2023-10-18T08:38:24.542 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-18T08:38:24.543 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi --verbose --owner scheduled_pdonnell@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} --archive /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719 -- /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719/orig.config.yaml 2023-10-18T08:38:24.593 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-18T08:38:24.673 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719 branch: wip-batrick-testing-20231016.203825 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} email: pdonnell@redhat.com first_in_suite: false job_id: '7430719' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20231016.203825 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd - disabled cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' client.0: debug client: 20 debug objectcacher: 20 debug objecter: 20 client.1: debug client: 20 debug objectcacher: 20 debug objecter: 20 global: ms die on skipped message: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd backfillfull ratio: 0.6 mon osd full ratio: 0.7 mon osd nearfull ratio: 0.6 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 memstore device bytes: 200000000 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 1.0 osd mon report interval: 5 osd objectstore: memstore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL - PG_RECOVERY_FULL - PG_DEGRADED sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20231016.203825 sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7703 sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb sleep_before_teardown: 0 subset: 153/256 suite: fs suite_branch: wip-batrick-testing-20231016.203825 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPru1b2IMuAaQvQ9WGQJurxzltIS6eqOeara4xXFpywLI6lhSRJD685TuJEGHY9LjnxtK5bfzffsA2ujB/Fq6ZE= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDKHTBvPWMvPC4sZtXUB9Xz9kdZblgeqYUCDaBSavudWyKwn8Od7f360DtIrNn+UGrWbBln0nEK2o6UC5gDSAS4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_full teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 8cdab074dcca9a68965bc5a50e9c30b691949723 timestamp: 2023-10-16_23:59:58 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-18T08:38:24.674 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa; will attempt to use it 2023-10-18T08:38:24.675 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks 2023-10-18T08:38:24.675 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-18T08:38:24.677 INFO:teuthology.task.internal:Checking packages... 2023-10-18T08:38:24.701 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f7420d7f8e55ceea786dcbf536c338ce920c21eb' 2023-10-18T08:38:24.701 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-18T08:38:24.702 INFO:teuthology.packaging:ref: None 2023-10-18T08:38:24.702 INFO:teuthology.packaging:tag: None 2023-10-18T08:38:24.702 INFO:teuthology.packaging:branch: wip-batrick-testing-20231016.203825 2023-10-18T08:38:24.703 INFO:teuthology.packaging:sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T08:38:24.703 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20231016.203825 2023-10-18T08:38:24.926 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6786.gf7420d7f 2023-10-18T08:38:24.926 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-18T08:38:24.933 INFO:teuthology.task.internal:no buildpackages task found 2023-10-18T08:38:24.934 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-18T08:38:24.945 INFO:teuthology.task.internal:Saving configuration 2023-10-18T08:38:24.964 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-18T08:38:24.970 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T08:38:24.995 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 08:33:22.094210', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPru1b2IMuAaQvQ9WGQJurxzltIS6eqOeara4xXFpywLI6lhSRJD685TuJEGHY9LjnxtK5bfzffsA2ujB/Fq6ZE='} 2023-10-18T08:38:25.019 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 08:33:22.095952', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDKHTBvPWMvPC4sZtXUB9Xz9kdZblgeqYUCDaBSavudWyKwn8Od7f360DtIrNn+UGrWbBln0nEK2o6UC5gDSAS4='} 2023-10-18T08:38:25.020 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-18T08:38:25.029 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-18T08:38:25.029 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-18T08:38:25.029 INFO:teuthology.run_tasks:Running task console_log... 2023-10-18T08:38:25.100 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff9eb14db80>, signals=[15]) 2023-10-18T08:38:25.101 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-18T08:38:25.107 INFO:teuthology.task.internal:Opening connections... 2023-10-18T08:38:25.107 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T08:38:25.109 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T08:38:25.210 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T08:38:25.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T08:38:25.292 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-18T08:38:25.299 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2023-10-18T08:38:25.317 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2023-10-18T08:38:25.317 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2023-10-18T08:38:25.375 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream" 2023-10-18T08:38:25.375 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9" 2023-10-18T08:38:25.375 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2023-10-18T08:38:25.376 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2023-10-18T08:38:25.376 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9" 2023-10-18T08:38:25.376 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9" 2023-10-18T08:38:25.376 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-18T08:38:25.376 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2023-10-18T08:38:25.376 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon" 2023-10-18T08:38:25.377 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-18T08:38:25.377 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/" 2023-10-18T08:38:25.377 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-18T08:38:25.377 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-18T08:38:25.378 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-18T08:38:25.378 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2023-10-18T08:38:25.405 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2023-10-18T08:38:25.422 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2023-10-18T08:38:25.423 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2023-10-18T08:38:25.478 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2023-10-18T08:38:25.479 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="9" 2023-10-18T08:38:25.479 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2023-10-18T08:38:25.479 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2023-10-18T08:38:25.479 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="9" 2023-10-18T08:38:25.480 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el9" 2023-10-18T08:38:25.480 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-18T08:38:25.480 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2023-10-18T08:38:25.480 INFO:teuthology.orchestra.run.smithi144.stdout:LOGO="fedora-logo-icon" 2023-10-18T08:38:25.480 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-18T08:38:25.481 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2023-10-18T08:38:25.481 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-18T08:38:25.481 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-18T08:38:25.481 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-18T08:38:25.481 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2023-10-18T08:38:25.504 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-18T08:38:25.513 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-18T08:38:25.519 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-18T08:38:25.520 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2023-10-18T08:38:25.523 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2023-10-18T08:38:25.539 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-18T08:38:25.545 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-18T08:38:25.546 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2023-10-18T08:38:25.580 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2023-10-18T08:38:25.694 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-18T08:38:25.743 INFO:teuthology.run_tasks:Running task kernel... 2023-10-18T08:38:25.757 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-18T08:38:25.758 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-10-18T08:38:25.758 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-10-18T08:38:25.759 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-18T08:38:25.760 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-18T08:38:25.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T08:38:25.777 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-10-18T08:38:25.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T08:38:25.781 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-10-18T08:38:26.008 DEBUG:teuthology.task.kernel:sha1 for client.0 is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T08:38:26.008 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-18T08:38:26.009 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2023-10-18T08:38:26.012 DEBUG:teuthology.task.kernel:sha1 for client.1 is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T08:38:26.012 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-18T08:38:26.012 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2023-10-18T08:38:26.026 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-361.el9.x86_64 2023-10-18T08:38:26.027 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T08:38:26.027 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-18T08:38:26.028 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum upgrade -y linux-firmware 2023-10-18T08:38:26.029 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-361.el9.x86_64 2023-10-18T08:38:26.030 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T08:38:26.030 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-18T08:38:26.030 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum upgrade -y linux-firmware 2023-10-18T08:38:28.074 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - AppStream 14 MB/s | 6.3 MB 00:00 2023-10-18T08:38:28.130 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-9 - AppStream 13 MB/s | 6.3 MB 00:00 2023-10-18T08:38:29.374 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - BaseOS 7.3 MB/s | 2.0 MB 00:00 2023-10-18T08:38:29.497 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-9 - BaseOS 7.6 MB/s | 2.0 MB 00:00 2023-10-18T08:38:30.371 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 7.9 MB 00:00 2023-10-18T08:38:30.648 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 8.4 MB/s | 7.9 MB 00:00 2023-10-18T08:38:32.433 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 23 MB/s | 18 MB 00:00 2023-10-18T08:38:32.747 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 14 MB/s | 18 MB 00:01 2023-10-18T08:38:36.167 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 12 MB/s | 5.7 MB 00:00 2023-10-18T08:38:38.401 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 14 kB 00:01 2023-10-18T08:38:39.164 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 5.7 MB 00:02 2023-10-18T08:38:39.863 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 14 MB/s | 19 MB 00:01 2023-10-18T08:38:40.732 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 14 kB 00:00 2023-10-18T08:38:41.858 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 19 MB/s | 19 MB 00:01 2023-10-18T08:38:44.352 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2023-10-18T08:38:46.464 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2023-10-18T08:38:46.472 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2023-10-18T08:38:46.473 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2023-10-18T08:38:46.473 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2023-10-18T08:38:46.473 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2023-10-18T08:38:46.473 INFO:teuthology.orchestra.run.smithi144.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-10-18T08:38:46.473 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:38:46.473 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2023-10-18T08:38:46.474 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2023-10-18T08:38:46.474 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 1 Package 2023-10-18T08:38:46.474 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:38:46.474 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 326 M 2023-10-18T08:38:46.474 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2023-10-18T08:38:46.566 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2023-10-18T08:38:48.721 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-10-18T08:38:48.736 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-10-18T08:38:48.736 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2023-10-18T08:38:48.737 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-10-18T08:38:48.737 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2023-10-18T08:38:48.737 INFO:teuthology.orchestra.run.smithi040.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-10-18T08:38:48.737 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:38:48.737 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-10-18T08:38:48.738 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2023-10-18T08:38:48.738 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 1 Package 2023-10-18T08:38:48.738 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:38:48.738 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 326 M 2023-10-18T08:38:48.738 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-10-18T08:38:53.269 INFO:teuthology.orchestra.run.smithi144.stdout:linux-firmware-20230926-140.el9.noarch.rpm 48 MB/s | 326 MB 00:06 2023-10-18T08:38:53.270 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2023-10-18T08:38:53.271 INFO:teuthology.orchestra.run.smithi144.stdout:Total 48 MB/s | 326 MB 00:06 2023-10-18T08:38:53.271 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2023-10-18T08:38:53.369 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2023-10-18T08:38:53.370 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2023-10-18T08:38:55.510 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2023-10-18T08:38:55.511 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2023-10-18T08:38:56.075 INFO:teuthology.orchestra.run.smithi040.stdout:linux-firmware-20230926-140.el9.noarch.rpm 44 MB/s | 326 MB 00:07 2023-10-18T08:38:56.076 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-10-18T08:38:56.077 INFO:teuthology.orchestra.run.smithi040.stdout:Total 44 MB/s | 326 MB 00:07 2023-10-18T08:38:56.077 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-10-18T08:38:56.154 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-10-18T08:38:56.154 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-10-18T08:38:58.308 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-10-18T08:38:58.308 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-10-18T08:39:01.022 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2023-10-18T08:39:01.612 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-18T08:39:01.784 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-18T08:39:02.728 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-10-18T08:39:02.728 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-18T08:39:03.148 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-18T08:39:03.149 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:39:03.149 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2023-10-18T08:39:03.149 INFO:teuthology.orchestra.run.smithi144.stdout: linux-firmware-20230926-140.el9.noarch 2023-10-18T08:39:03.149 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:39:03.150 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2023-10-18T08:39:03.428 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.1... 2023-10-18T08:39:03.428 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T08:39:03.655 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/ 2023-10-18T08:39:03.656 DEBUG:teuthology.orchestra.run.smithi144:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/ --input-file=- 2023-10-18T08:39:03.659 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi144.front.sepia.ceph.com to complete... 2023-10-18T08:39:04.184 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-10-18T08:39:04.582 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-18T08:39:04.683 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-18T08:39:05.747 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-10-18T08:39:05.747 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-18T08:39:06.141 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-18T08:39:06.142 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:39:06.142 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2023-10-18T08:39:06.142 INFO:teuthology.orchestra.run.smithi040.stdout: linux-firmware-20230926-140.el9.noarch 2023-10-18T08:39:06.142 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:39:06.142 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-10-18T08:39:06.420 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-18T08:39:06.420 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T08:39:06.564 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/ 2023-10-18T08:39:06.565 DEBUG:teuthology.orchestra.run.smithi040:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/ --input-file=- 2023-10-18T08:39:06.568 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi040.front.sepia.ceph.com to complete... 2023-10-18T08:39:29.178 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18 08:39:29 URL:https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm [320683515/320683515] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-18T08:39:29.178 INFO:teuthology.orchestra.run.smithi144.stderr:FINISHED --2023-10-18 08:39:29-- 2023-10-18T08:39:29.179 INFO:teuthology.orchestra.run.smithi144.stderr:Total wall clock time: 25s 2023-10-18T08:39:29.179 INFO:teuthology.orchestra.run.smithi144.stderr:Downloaded: 1 files, 306M in 25s (12.1 MB/s) 2023-10-18T08:39:29.180 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.1... 2023-10-18T08:39:29.180 DEBUG:teuthology.orchestra.run.smithi144:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-18T08:39:31.024 INFO:teuthology.orchestra.run.smithi144.stdout:Verifying... ######################################## 2023-10-18T08:39:31.073 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing... ######################################## 2023-10-18T08:39:31.152 INFO:teuthology.orchestra.run.smithi144.stdout:Updating / installing... 2023-10-18T08:39:31.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18 08:39:31 URL:https://2.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/9/flavors/default/x86_64/kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm [320683515/320683515] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-18T08:39:31.379 INFO:teuthology.orchestra.run.smithi040.stderr:FINISHED --2023-10-18 08:39:31-- 2023-10-18T08:39:31.379 INFO:teuthology.orchestra.run.smithi040.stderr:Total wall clock time: 25s 2023-10-18T08:39:31.379 INFO:teuthology.orchestra.run.smithi040.stderr:Downloaded: 1 files, 306M in 24s (12.5 MB/s) 2023-10-18T08:39:31.380 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-18T08:39:31.381 DEBUG:teuthology.orchestra.run.smithi040:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-18T08:39:33.313 INFO:teuthology.orchestra.run.smithi040.stdout:Verifying... ######################################## 2023-10-18T08:39:33.383 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing... ######################################## 2023-10-18T08:39:33.468 INFO:teuthology.orchestra.run.smithi040.stdout:Updating / installing... 2023-10-18T08:39:38.218 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-6.6.0_rc5_g47672119149b-1 ######################################## 2023-10-18T08:39:40.885 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-6.6.0_rc5_g47672119149b-1 ######################################## 2023-10-18T08:39:53.324 INFO:teuthology.orchestra.run.smithi144.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-10-18T08:39:56.330 INFO:teuthology.orchestra.run.smithi040.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-10-18T08:40:14.463 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi144.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-18T08:40:14.463 DEBUG:teuthology.orchestra.run.smithi144:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-18T08:40:14.530 INFO:teuthology.orchestra.run.smithi144.stdout:/boot/System.map-6.6.0-rc5-g47672119149b 2023-10-18T08:40:14.530 INFO:teuthology.orchestra.run.smithi144.stdout:/boot/config-6.6.0-rc5-g47672119149b 2023-10-18T08:40:14.530 INFO:teuthology.orchestra.run.smithi144.stdout:/boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-18T08:40:14.530 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b 2023-10-18T08:40:14.531 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel 2023-10-18T08:40:14.531 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch 2023-10-18T08:40:14.531 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86 2023-10-18T08:40:14.531 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto 2023-10-18T08:40:14.531 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-18T08:40:14.532 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-18T08:40:14.532 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-18T08:40:14.532 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-18T08:40:14.532 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-18T08:40:14.532 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-18T08:40:14.532 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-18T08:40:14.533 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-18T08:40:14.533 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-18T08:40:14.533 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-18T08:40:14.533 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-18T08:40:14.533 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-18T08:40:14.534 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-18T08:40:14.534 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-18T08:40:14.534 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-18T08:40:14.534 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-18T08:40:14.534 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-18T08:40:14.535 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-18T08:40:14.535 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-18T08:40:14.535 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-18T08:40:14.535 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel 2023-10-18T08:40:14.535 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu 2023-10-18T08:40:14.535 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce 2023-10-18T08:40:14.536 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-18T08:40:14.536 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm 2023-10-18T08:40:14.536 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-18T08:40:14.536 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-18T08:40:14.536 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-18T08:40:14.537 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto 2023-10-18T08:40:14.537 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-18T08:40:14.537 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-18T08:40:14.537 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-18T08:40:14.537 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx 2023-10-18T08:40:14.537 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-18T08:40:14.538 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-18T08:40:14.538 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-18T08:40:14.538 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-18T08:40:14.538 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-18T08:40:14.538 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-18T08:40:14.539 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-18T08:40:14.539 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-18T08:40:14.539 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-18T08:40:14.539 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-18T08:40:14.539 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-18T08:40:14.539 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-18T08:40:14.540 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-18T08:40:14.540 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-18T08:40:14.540 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-18T08:40:14.540 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-18T08:40:14.540 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-18T08:40:14.540 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-18T08:40:14.541 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-18T08:40:14.541 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-18T08:40:14.541 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-18T08:40:14.541 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-18T08:40:14.541 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-18T08:40:14.541 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-18T08:40:14.542 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-18T08:40:14.542 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-18T08:40:14.542 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-18T08:40:14.542 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-18T08:40:14.542 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-18T08:40:14.543 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-18T08:40:14.543 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-18T08:40:14.543 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-18T08:40:14.543 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-18T08:40:14.543 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-18T08:40:14.543 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-18T08:40:14.544 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-18T08:40:14.544 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-18T08:40:14.544 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-18T08:40:14.544 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-18T08:40:14.544 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-18T08:40:14.545 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-18T08:40:14.545 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-18T08:40:14.545 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-18T08:40:14.545 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-18T08:40:14.545 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-18T08:40:14.546 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers 2023-10-18T08:40:14.546 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi 2023-10-18T08:40:14.546 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-18T08:40:14.546 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-18T08:40:14.546 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/custom_method.ko 2023-10-18T08:40:14.546 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-18T08:40:14.547 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-18T08:40:14.547 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-18T08:40:14.547 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-18T08:40:14.547 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-18T08:40:14.547 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata 2023-10-18T08:40:14.548 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-18T08:40:14.548 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-18T08:40:14.548 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ata_generic.ko 2023-10-18T08:40:14.548 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-18T08:40:14.548 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-18T08:40:14.549 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-18T08:40:14.549 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-18T08:40:14.549 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-18T08:40:14.549 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-18T08:40:14.549 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-18T08:40:14.549 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-18T08:40:14.550 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-18T08:40:14.550 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-18T08:40:14.550 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-18T08:40:14.550 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-18T08:40:14.550 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-18T08:40:14.551 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-18T08:40:14.551 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-18T08:40:14.551 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-18T08:40:14.551 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-18T08:40:14.551 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-18T08:40:14.551 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-18T08:40:14.552 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-18T08:40:14.552 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-18T08:40:14.552 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-18T08:40:14.552 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-18T08:40:14.552 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-18T08:40:14.553 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-18T08:40:14.553 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-18T08:40:14.553 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-18T08:40:14.553 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-18T08:40:14.553 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-18T08:40:14.553 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-18T08:40:14.554 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-18T08:40:14.554 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-18T08:40:14.554 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-18T08:40:14.555 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-18T08:40:14.555 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-18T08:40:14.555 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sis.ko 2023-10-18T08:40:14.555 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-18T08:40:14.557 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-18T08:40:14.557 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-18T08:40:14.557 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-18T08:40:14.557 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-18T08:40:14.557 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-18T08:40:14.558 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-18T08:40:14.558 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-18T08:40:14.558 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-18T08:40:14.558 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-18T08:40:14.558 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-18T08:40:14.559 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-18T08:40:14.559 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-18T08:40:14.559 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-18T08:40:14.559 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-18T08:40:14.559 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-18T08:40:14.560 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-18T08:40:14.560 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay 2023-10-18T08:40:14.560 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-18T08:40:14.560 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base 2023-10-18T08:40:14.560 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap 2023-10-18T08:40:14.560 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-18T08:40:14.561 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-18T08:40:14.561 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma 2023-10-18T08:40:14.561 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-18T08:40:14.561 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block 2023-10-18T08:40:14.561 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe 2023-10-18T08:40:14.562 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-18T08:40:14.562 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/brd.ko 2023-10-18T08:40:14.562 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd 2023-10-18T08:40:14.562 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-18T08:40:14.562 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-18T08:40:14.562 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/loop.ko 2023-10-18T08:40:14.563 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx 2023-10-18T08:40:14.563 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-18T08:40:14.563 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-18T08:40:14.563 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk 2023-10-18T08:40:14.563 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-18T08:40:14.564 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-18T08:40:14.564 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-18T08:40:14.564 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/virtio_blk.ko 2023-10-18T08:40:14.564 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback 2023-10-18T08:40:14.564 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-18T08:40:14.565 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkfront.ko 2023-10-18T08:40:14.565 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram 2023-10-18T08:40:14.565 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-18T08:40:14.565 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char 2023-10-18T08:40:14.565 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-18T08:40:14.565 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random 2023-10-18T08:40:14.566 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-18T08:40:14.566 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-18T08:40:14.566 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-18T08:40:14.566 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-18T08:40:14.566 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-18T08:40:14.567 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi 2023-10-18T08:40:14.567 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-18T08:40:14.567 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-18T08:40:14.567 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-18T08:40:14.567 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-18T08:40:14.568 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-18T08:40:14.568 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-18T08:40:14.568 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-18T08:40:14.568 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave 2023-10-18T08:40:14.568 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-18T08:40:14.568 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-18T08:40:14.569 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-18T08:40:14.569 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm 2023-10-18T08:40:14.569 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm.ko 2023-10-18T08:40:14.569 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-18T08:40:14.569 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-18T08:40:14.570 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-18T08:40:14.570 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-18T08:40:14.570 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-18T08:40:14.570 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/uv_mmtimer.ko 2023-10-18T08:40:14.570 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/virtio_console.ko 2023-10-18T08:40:14.571 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq 2023-10-18T08:40:14.571 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-18T08:40:14.571 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-18T08:40:14.571 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-18T08:40:14.571 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-18T08:40:14.571 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-18T08:40:14.572 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-18T08:40:14.572 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto 2023-10-18T08:40:14.572 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp 2023-10-18T08:40:14.572 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-18T08:40:14.572 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-18T08:40:14.573 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel 2023-10-18T08:40:14.573 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat 2023-10-18T08:40:14.573 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-18T08:40:14.573 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-18T08:40:14.573 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-18T08:40:14.573 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-18T08:40:14.574 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-18T08:40:14.574 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-18T08:40:14.574 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio 2023-10-18T08:40:14.574 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-18T08:40:14.574 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca 2023-10-18T08:40:14.575 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-18T08:40:14.575 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq 2023-10-18T08:40:14.575 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-18T08:40:14.575 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma 2023-10-18T08:40:14.575 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw 2023-10-18T08:40:14.575 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-18T08:40:14.576 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat 2023-10-18T08:40:14.576 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-18T08:40:14.576 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac 2023-10-18T08:40:14.576 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-18T08:40:14.576 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-18T08:40:14.577 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-18T08:40:14.577 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-18T08:40:14.577 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-18T08:40:14.577 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-18T08:40:14.577 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-18T08:40:14.577 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-18T08:40:14.578 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-18T08:40:14.578 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-18T08:40:14.578 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-18T08:40:14.578 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-18T08:40:14.578 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-18T08:40:14.579 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware 2023-10-18T08:40:14.579 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/edd.ko 2023-10-18T08:40:14.579 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-18T08:40:14.579 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio 2023-10-18T08:40:14.579 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-18T08:40:14.580 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-18T08:40:14.580 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid 2023-10-18T08:40:14.580 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-18T08:40:14.580 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-18T08:40:14.580 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-18T08:40:14.580 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-18T08:40:14.581 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-18T08:40:14.581 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-18T08:40:14.581 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-18T08:40:14.581 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-18T08:40:14.581 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-18T08:40:14.582 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-18T08:40:14.582 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-18T08:40:14.582 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-18T08:40:14.582 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-18T08:40:14.582 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-18T08:40:14.583 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-18T08:40:14.583 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-18T08:40:14.583 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-18T08:40:14.583 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-18T08:40:14.583 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-18T08:40:14.583 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-18T08:40:14.584 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-18T08:40:14.584 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-18T08:40:14.584 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-18T08:40:14.584 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-18T08:40:14.584 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-18T08:40:14.585 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-18T08:40:14.585 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-18T08:40:14.585 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-18T08:40:14.585 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-18T08:40:14.585 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-18T08:40:14.586 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-18T08:40:14.586 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-18T08:40:14.586 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-18T08:40:14.586 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-18T08:40:14.586 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-18T08:40:14.586 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-18T08:40:14.587 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-18T08:40:14.587 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-18T08:40:14.587 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-18T08:40:14.587 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-18T08:40:14.587 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-18T08:40:14.588 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-18T08:40:14.588 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-18T08:40:14.588 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-18T08:40:14.588 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-18T08:40:14.588 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-18T08:40:14.589 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-18T08:40:14.589 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-18T08:40:14.589 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-18T08:40:14.589 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-18T08:40:14.589 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-18T08:40:14.590 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-18T08:40:14.590 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-18T08:40:14.590 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-18T08:40:14.590 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-18T08:40:14.590 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-18T08:40:14.590 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-18T08:40:14.591 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-18T08:40:14.591 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-18T08:40:14.591 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-18T08:40:14.591 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-18T08:40:14.591 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-18T08:40:14.592 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-18T08:40:14.592 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-18T08:40:14.592 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv 2023-10-18T08:40:14.592 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-18T08:40:14.592 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-18T08:40:14.593 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-18T08:40:14.593 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon 2023-10-18T08:40:14.593 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-18T08:40:14.593 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-18T08:40:14.593 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-18T08:40:14.593 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-18T08:40:14.594 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-18T08:40:14.594 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-18T08:40:14.594 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-18T08:40:14.594 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-18T08:40:14.594 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-18T08:40:14.595 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-18T08:40:14.595 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-18T08:40:14.595 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-18T08:40:14.595 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-18T08:40:14.595 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-18T08:40:14.596 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-18T08:40:14.596 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-18T08:40:14.596 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-18T08:40:14.596 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-18T08:40:14.596 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-18T08:40:14.596 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-18T08:40:14.597 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-18T08:40:14.597 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-18T08:40:14.597 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-18T08:40:14.597 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-18T08:40:14.597 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-18T08:40:14.598 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-18T08:40:14.598 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-18T08:40:14.598 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-18T08:40:14.598 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-18T08:40:14.598 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-18T08:40:14.599 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-18T08:40:14.599 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-18T08:40:14.599 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-18T08:40:14.599 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-18T08:40:14.599 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-18T08:40:14.599 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-18T08:40:14.600 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-18T08:40:14.600 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-18T08:40:14.600 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-18T08:40:14.600 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-18T08:40:14.600 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-18T08:40:14.601 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-18T08:40:14.601 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-18T08:40:14.601 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-18T08:40:14.601 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-18T08:40:14.601 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-18T08:40:14.601 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-18T08:40:14.602 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-18T08:40:14.602 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-18T08:40:14.602 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-18T08:40:14.602 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-18T08:40:14.602 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-18T08:40:14.603 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-18T08:40:14.603 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-18T08:40:14.603 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-18T08:40:14.603 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-18T08:40:14.603 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-18T08:40:14.603 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-18T08:40:14.604 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-18T08:40:14.604 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-18T08:40:14.604 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-18T08:40:14.604 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-18T08:40:14.604 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-18T08:40:14.605 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-18T08:40:14.605 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-18T08:40:14.605 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-18T08:40:14.605 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-18T08:40:14.605 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-18T08:40:14.606 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-18T08:40:14.606 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-18T08:40:14.606 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-18T08:40:14.606 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-18T08:40:14.606 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-18T08:40:14.606 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-18T08:40:14.607 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-18T08:40:14.607 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-18T08:40:14.607 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-18T08:40:14.607 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-18T08:40:14.607 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-18T08:40:14.608 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-18T08:40:14.608 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-18T08:40:14.608 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-18T08:40:14.608 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-18T08:40:14.608 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-18T08:40:14.608 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-18T08:40:14.609 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-18T08:40:14.609 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-18T08:40:14.609 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-18T08:40:14.609 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus 2023-10-18T08:40:14.609 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-18T08:40:14.610 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-18T08:40:14.610 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-18T08:40:14.610 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-18T08:40:14.610 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-18T08:40:14.610 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-18T08:40:14.610 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-18T08:40:14.611 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-18T08:40:14.611 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-18T08:40:14.611 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-18T08:40:14.611 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-18T08:40:14.611 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-18T08:40:14.612 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-18T08:40:14.612 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-18T08:40:14.612 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-18T08:40:14.612 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-18T08:40:14.612 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-18T08:40:14.612 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-18T08:40:14.613 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-18T08:40:14.613 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-18T08:40:14.613 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-18T08:40:14.613 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-18T08:40:14.613 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-18T08:40:14.614 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-18T08:40:14.614 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-18T08:40:14.614 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-18T08:40:14.614 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-18T08:40:14.614 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-18T08:40:14.615 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-18T08:40:14.615 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-18T08:40:14.615 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-18T08:40:14.615 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-18T08:40:14.615 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-18T08:40:14.615 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-18T08:40:14.616 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-18T08:40:14.616 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-18T08:40:14.616 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-18T08:40:14.616 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-18T08:40:14.616 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-18T08:40:14.617 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-18T08:40:14.617 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-18T08:40:14.617 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c 2023-10-18T08:40:14.617 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos 2023-10-18T08:40:14.617 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-18T08:40:14.618 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-18T08:40:14.618 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses 2023-10-18T08:40:14.618 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-18T08:40:14.618 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-18T08:40:14.618 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-18T08:40:14.618 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-18T08:40:14.619 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-18T08:40:14.619 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-18T08:40:14.619 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-18T08:40:14.619 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-18T08:40:14.619 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-18T08:40:14.620 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-18T08:40:14.620 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-18T08:40:14.620 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-18T08:40:14.620 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-18T08:40:14.620 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-18T08:40:14.621 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-18T08:40:14.621 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-18T08:40:14.621 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-18T08:40:14.621 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-18T08:40:14.621 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-18T08:40:14.621 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-18T08:40:14.622 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-18T08:40:14.622 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-18T08:40:14.622 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-18T08:40:14.622 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-dev.ko 2023-10-18T08:40:14.622 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-18T08:40:14.623 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-18T08:40:14.623 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-18T08:40:14.623 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-18T08:40:14.623 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband 2023-10-18T08:40:14.623 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core 2023-10-18T08:40:14.623 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-18T08:40:14.624 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-18T08:40:14.624 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-18T08:40:14.624 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-18T08:40:14.624 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-18T08:40:14.624 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-18T08:40:14.625 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-18T08:40:14.625 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw 2023-10-18T08:40:14.625 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4 2023-10-18T08:40:14.625 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-18T08:40:14.625 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4 2023-10-18T08:40:14.625 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-18T08:40:14.626 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5 2023-10-18T08:40:14.626 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-18T08:40:14.626 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca 2023-10-18T08:40:14.626 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-18T08:40:14.626 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp 2023-10-18T08:40:14.627 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib 2023-10-18T08:40:14.627 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-18T08:40:14.627 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser 2023-10-18T08:40:14.627 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-18T08:40:14.627 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert 2023-10-18T08:40:14.628 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-18T08:40:14.628 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp 2023-10-18T08:40:14.628 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-18T08:40:14.628 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt 2023-10-18T08:40:14.628 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-18T08:40:14.629 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input 2023-10-18T08:40:14.629 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport 2023-10-18T08:40:14.629 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-18T08:40:14.629 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-18T08:40:14.629 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-18T08:40:14.629 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-18T08:40:14.630 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard 2023-10-18T08:40:14.630 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-18T08:40:14.630 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse 2023-10-18T08:40:14.630 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-18T08:40:14.630 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-18T08:40:14.631 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-18T08:40:14.631 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-18T08:40:14.631 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-18T08:40:14.631 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-18T08:40:14.631 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-18T08:40:14.631 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-18T08:40:14.632 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4 2023-10-18T08:40:14.632 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-18T08:40:14.632 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio 2023-10-18T08:40:14.632 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-18T08:40:14.632 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-18T08:40:14.633 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-18T08:40:14.633 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-18T08:40:14.633 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-18T08:40:14.633 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu 2023-10-18T08:40:14.633 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd 2023-10-18T08:40:14.634 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-18T08:40:14.634 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds 2023-10-18T08:40:14.634 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-18T08:40:14.634 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-18T08:40:14.634 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-18T08:40:14.634 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-18T08:40:14.635 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger 2023-10-18T08:40:14.635 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-18T08:40:14.635 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-18T08:40:14.635 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-18T08:40:14.635 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-18T08:40:14.636 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-18T08:40:14.636 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-18T08:40:14.636 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-18T08:40:14.636 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-18T08:40:14.636 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md 2023-10-18T08:40:14.636 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache 2023-10-18T08:40:14.637 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-18T08:40:14.637 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-18T08:40:14.637 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-18T08:40:14.637 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-18T08:40:14.637 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-18T08:40:14.638 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-18T08:40:14.638 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-18T08:40:14.638 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-18T08:40:14.638 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-18T08:40:14.638 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-18T08:40:14.638 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-18T08:40:14.639 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-18T08:40:14.639 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-18T08:40:14.639 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-18T08:40:14.639 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-18T08:40:14.639 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-18T08:40:14.640 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-18T08:40:14.640 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-18T08:40:14.640 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-18T08:40:14.640 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data 2023-10-18T08:40:14.640 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-18T08:40:14.641 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-18T08:40:14.641 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-18T08:40:14.641 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-18T08:40:14.641 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-18T08:40:14.641 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message 2023-10-18T08:40:14.642 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion 2023-10-18T08:40:14.642 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-18T08:40:14.642 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-18T08:40:14.642 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-18T08:40:14.642 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-18T08:40:14.642 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-18T08:40:14.643 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-18T08:40:14.643 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-18T08:40:14.643 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd 2023-10-18T08:40:14.643 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-18T08:40:14.643 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-18T08:40:14.644 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mfd-core.ko 2023-10-18T08:40:14.644 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-18T08:40:14.644 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-18T08:40:14.644 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-18T08:40:14.644 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-18T08:40:14.645 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc 2023-10-18T08:40:14.645 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl 2023-10-18T08:40:14.645 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-18T08:40:14.645 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-18T08:40:14.645 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-18T08:40:14.645 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-18T08:40:14.646 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710 2023-10-18T08:40:14.646 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-18T08:40:14.646 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom 2023-10-18T08:40:14.646 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-18T08:40:14.646 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-18T08:40:14.647 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-18T08:40:14.647 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-18T08:40:14.647 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-18T08:40:14.647 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-18T08:40:14.647 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-18T08:40:14.648 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-18T08:40:14.648 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d 2023-10-18T08:40:14.648 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-18T08:40:14.648 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-18T08:40:14.648 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei 2023-10-18T08:40:14.648 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-18T08:40:14.649 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-18T08:40:14.649 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-18T08:40:14.649 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru 2023-10-18T08:40:14.649 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-18T08:40:14.649 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp 2023-10-18T08:40:14.650 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-18T08:40:14.650 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-18T08:40:14.650 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-18T08:40:14.650 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-18T08:40:14.650 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-18T08:40:14.651 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-18T08:40:14.651 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-18T08:40:14.651 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci 2023-10-18T08:40:14.651 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-18T08:40:14.651 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net 2023-10-18T08:40:14.651 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding 2023-10-18T08:40:14.652 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-18T08:40:14.652 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa 2023-10-18T08:40:14.652 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53 2023-10-18T08:40:14.652 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-18T08:40:14.652 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-18T08:40:14.653 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-18T08:40:14.653 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-18T08:40:14.653 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-18T08:40:14.653 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-18T08:40:14.653 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-18T08:40:14.654 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet 2023-10-18T08:40:14.654 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com 2023-10-18T08:40:14.654 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-18T08:40:14.654 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-18T08:40:14.654 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-18T08:40:14.654 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-18T08:40:14.655 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390 2023-10-18T08:40:14.655 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-18T08:40:14.655 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-18T08:40:14.655 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-18T08:40:14.655 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-18T08:40:14.656 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec 2023-10-18T08:40:14.656 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-18T08:40:14.656 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere 2023-10-18T08:40:14.656 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-18T08:40:14.656 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon 2023-10-18T08:40:14.656 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-18T08:40:14.657 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera 2023-10-18T08:40:14.657 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-18T08:40:14.657 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd 2023-10-18T08:40:14.657 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-18T08:40:14.657 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-18T08:40:14.658 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-18T08:40:14.658 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros 2023-10-18T08:40:14.658 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx 2023-10-18T08:40:14.658 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-18T08:40:14.658 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-18T08:40:14.659 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-18T08:40:14.659 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-18T08:40:14.659 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-18T08:40:14.659 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-18T08:40:14.659 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-18T08:40:14.659 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-18T08:40:14.660 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom 2023-10-18T08:40:14.660 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-18T08:40:14.660 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-18T08:40:14.660 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-18T08:40:14.660 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-18T08:40:14.661 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-18T08:40:14.661 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-18T08:40:14.661 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-18T08:40:14.661 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-18T08:40:14.661 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade 2023-10-18T08:40:14.662 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna 2023-10-18T08:40:14.662 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-18T08:40:14.662 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio 2023-10-18T08:40:14.662 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-18T08:40:14.662 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-18T08:40:14.662 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-18T08:40:14.663 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-18T08:40:14.663 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-18T08:40:14.663 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-18T08:40:14.663 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-18T08:40:14.663 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-18T08:40:14.664 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-18T08:40:14.664 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-18T08:40:14.664 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco 2023-10-18T08:40:14.664 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic 2023-10-18T08:40:14.664 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-18T08:40:14.664 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec 2023-10-18T08:40:14.665 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip 2023-10-18T08:40:14.665 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-18T08:40:14.665 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-18T08:40:14.665 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-18T08:40:14.665 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-18T08:40:14.666 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-18T08:40:14.666 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-18T08:40:14.666 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink 2023-10-18T08:40:14.666 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-18T08:40:14.666 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-18T08:40:14.667 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-18T08:40:14.667 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex 2023-10-18T08:40:14.667 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet 2023-10-18T08:40:14.667 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-18T08:40:14.667 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-18T08:40:14.667 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-18T08:40:14.668 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel 2023-10-18T08:40:14.668 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-18T08:40:14.668 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000 2023-10-18T08:40:14.668 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-18T08:40:14.668 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-18T08:40:14.669 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-18T08:40:14.669 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-18T08:40:14.669 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-18T08:40:14.669 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e 2023-10-18T08:40:14.669 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-18T08:40:14.670 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf 2023-10-18T08:40:14.670 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-18T08:40:14.670 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb 2023-10-18T08:40:14.670 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-18T08:40:14.670 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-18T08:40:14.671 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-18T08:40:14.671 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-18T08:40:14.671 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-18T08:40:14.671 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-18T08:40:14.671 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-18T08:40:14.671 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-18T08:40:14.672 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell 2023-10-18T08:40:14.672 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-18T08:40:14.672 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-18T08:40:14.672 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-18T08:40:14.672 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox 2023-10-18T08:40:14.672 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-18T08:40:14.673 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-18T08:40:14.673 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-18T08:40:14.673 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-18T08:40:14.673 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-18T08:40:14.673 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-18T08:40:14.673 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel 2023-10-18T08:40:14.674 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-18T08:40:14.674 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom 2023-10-18T08:40:14.674 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-18T08:40:14.674 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-18T08:40:14.674 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi 2023-10-18T08:40:14.675 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-18T08:40:14.675 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-18T08:40:14.675 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion 2023-10-18T08:40:14.675 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-18T08:40:14.675 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia 2023-10-18T08:40:14.676 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-18T08:40:14.676 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines 2023-10-18T08:40:14.676 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-18T08:40:14.676 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-18T08:40:14.676 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic 2023-10-18T08:40:14.676 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-18T08:40:14.677 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-18T08:40:14.677 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-18T08:40:14.677 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-18T08:40:14.677 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-18T08:40:14.677 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc 2023-10-18T08:40:14.678 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-18T08:40:14.678 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek 2023-10-18T08:40:14.678 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-18T08:40:14.678 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-18T08:40:14.678 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-18T08:40:14.678 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-18T08:40:14.679 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker 2023-10-18T08:40:14.679 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-18T08:40:14.679 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc 2023-10-18T08:40:14.679 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-18T08:40:14.679 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan 2023-10-18T08:40:14.680 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-18T08:40:14.680 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis 2023-10-18T08:40:14.680 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-18T08:40:14.680 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-18T08:40:14.680 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc 2023-10-18T08:40:14.680 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-18T08:40:14.681 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-18T08:40:14.681 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-18T08:40:14.681 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-18T08:40:14.681 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro 2023-10-18T08:40:14.681 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-18T08:40:14.682 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-18T08:40:14.682 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-18T08:40:14.682 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun 2023-10-18T08:40:14.682 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-18T08:40:14.682 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-18T08:40:14.683 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-18T08:40:14.683 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-18T08:40:14.683 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti 2023-10-18T08:40:14.683 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-18T08:40:14.683 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti 2023-10-18T08:40:14.684 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-18T08:40:14.684 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via 2023-10-18T08:40:14.684 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-18T08:40:14.684 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-18T08:40:14.684 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet 2023-10-18T08:40:14.684 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-18T08:40:14.685 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-18T08:40:14.685 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom 2023-10-18T08:40:14.685 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-18T08:40:14.685 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-18T08:40:14.685 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv 2023-10-18T08:40:14.686 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-18T08:40:14.686 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154 2023-10-18T08:40:14.686 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-18T08:40:14.686 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-18T08:40:14.686 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan 2023-10-18T08:40:14.687 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-18T08:40:14.687 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-18T08:40:14.687 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-18T08:40:14.687 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio 2023-10-18T08:40:14.687 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-18T08:40:14.687 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-18T08:40:14.688 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-18T08:40:14.688 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-18T08:40:14.688 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-18T08:40:14.688 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/net_failover.ko 2023-10-18T08:40:14.688 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-18T08:40:14.689 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-18T08:40:14.689 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs 2023-10-18T08:40:14.689 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-18T08:40:14.689 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-18T08:40:14.689 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy 2023-10-18T08:40:14.689 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-18T08:40:14.690 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-18T08:40:14.690 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-18T08:40:14.690 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-18T08:40:14.690 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-18T08:40:14.690 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-18T08:40:14.691 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-18T08:40:14.691 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-18T08:40:14.691 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/dp83640.ko 2023-10-18T08:40:14.691 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-18T08:40:14.691 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-18T08:40:14.692 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-18T08:40:14.692 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-18T08:40:14.692 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-18T08:40:14.692 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-18T08:40:14.692 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-18T08:40:14.692 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-18T08:40:14.693 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-18T08:40:14.693 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-18T08:40:14.693 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-18T08:40:14.693 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-18T08:40:14.693 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp 2023-10-18T08:40:14.693 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-18T08:40:14.694 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-18T08:40:14.694 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-18T08:40:14.694 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-18T08:40:14.694 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-18T08:40:14.694 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-18T08:40:14.695 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-18T08:40:14.695 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-18T08:40:14.695 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-18T08:40:14.695 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip 2023-10-18T08:40:14.695 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slhc.ko 2023-10-18T08:40:14.695 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-18T08:40:14.696 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-18T08:40:14.696 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-18T08:40:14.696 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team 2023-10-18T08:40:14.696 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-18T08:40:14.696 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-18T08:40:14.696 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-18T08:40:14.697 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-18T08:40:14.697 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-18T08:40:14.697 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-18T08:40:14.697 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tun.ko 2023-10-18T08:40:14.697 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-18T08:40:14.698 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/virtio_net.ko 2023-10-18T08:40:14.698 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3 2023-10-18T08:40:14.698 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-18T08:40:14.698 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan 2023-10-18T08:40:14.698 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-18T08:40:14.698 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback 2023-10-18T08:40:14.699 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-18T08:40:14.699 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netfront.ko 2023-10-18T08:40:14.699 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb 2023-10-18T08:40:14.699 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-18T08:40:14.699 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme 2023-10-18T08:40:14.700 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host 2023-10-18T08:40:14.700 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-18T08:40:14.700 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-18T08:40:14.700 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport 2023-10-18T08:40:14.700 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-18T08:40:14.701 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-18T08:40:14.701 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-18T08:40:14.701 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-18T08:40:14.701 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci 2023-10-18T08:40:14.701 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug 2023-10-18T08:40:14.701 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-18T08:40:14.702 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie 2023-10-18T08:40:14.702 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-18T08:40:14.702 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-18T08:40:14.702 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia 2023-10-18T08:40:14.702 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-18T08:40:14.703 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-18T08:40:14.703 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-18T08:40:14.703 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl 2023-10-18T08:40:14.703 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel 2023-10-18T08:40:14.703 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-18T08:40:14.703 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform 2023-10-18T08:40:14.704 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86 2023-10-18T08:40:14.704 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-18T08:40:14.704 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-18T08:40:14.704 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-18T08:40:14.704 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-18T08:40:14.705 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-18T08:40:14.705 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-18T08:40:14.705 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-18T08:40:14.705 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-18T08:40:14.705 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-18T08:40:14.706 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-18T08:40:14.706 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-18T08:40:14.706 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-18T08:40:14.706 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel 2023-10-18T08:40:14.706 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-18T08:40:14.707 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-18T08:40:14.707 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-18T08:40:14.707 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-18T08:40:14.707 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-18T08:40:14.707 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-18T08:40:14.707 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-18T08:40:14.708 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-18T08:40:14.708 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-18T08:40:14.708 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-18T08:40:14.708 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-18T08:40:14.708 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-18T08:40:14.709 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap 2023-10-18T08:40:14.709 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-18T08:40:14.709 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-18T08:40:14.709 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps 2023-10-18T08:40:14.709 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients 2023-10-18T08:40:14.709 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-18T08:40:14.710 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-18T08:40:14.710 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-18T08:40:14.710 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-18T08:40:14.710 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp 2023-10-18T08:40:14.710 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-18T08:40:14.711 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-18T08:40:14.711 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc 2023-10-18T08:40:14.711 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-18T08:40:14.711 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-18T08:40:14.711 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-18T08:40:14.711 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-18T08:40:14.712 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-18T08:40:14.712 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-18T08:40:14.712 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-18T08:40:14.712 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-18T08:40:14.712 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-18T08:40:14.713 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-18T08:40:14.713 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-18T08:40:14.713 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-18T08:40:14.713 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-18T08:40:14.713 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-18T08:40:14.713 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-18T08:40:14.714 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-18T08:40:14.714 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-18T08:40:14.714 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-18T08:40:14.714 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-18T08:40:14.714 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-18T08:40:14.715 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-18T08:40:14.715 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-18T08:40:14.715 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-18T08:40:14.715 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-18T08:40:14.715 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-18T08:40:14.716 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-18T08:40:14.716 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-18T08:40:14.716 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-18T08:40:14.716 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-18T08:40:14.716 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-18T08:40:14.716 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-18T08:40:14.717 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi 2023-10-18T08:40:14.717 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-18T08:40:14.717 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-18T08:40:14.717 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-18T08:40:14.717 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-18T08:40:14.718 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-18T08:40:14.718 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid 2023-10-18T08:40:14.718 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-18T08:40:14.718 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-18T08:40:14.718 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx 2023-10-18T08:40:14.719 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-18T08:40:14.719 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-18T08:40:14.719 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-18T08:40:14.719 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr 2023-10-18T08:40:14.719 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-18T08:40:14.719 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-18T08:40:14.720 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi 2023-10-18T08:40:14.720 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-18T08:40:14.720 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa 2023-10-18T08:40:14.720 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-18T08:40:14.720 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc 2023-10-18T08:40:14.721 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-18T08:40:14.721 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i 2023-10-18T08:40:14.721 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-18T08:40:14.721 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-18T08:40:14.721 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor 2023-10-18T08:40:14.721 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-18T08:40:14.722 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi 2023-10-18T08:40:14.722 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-18T08:40:14.722 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-18T08:40:14.722 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-18T08:40:14.722 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-18T08:40:14.723 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-18T08:40:14.723 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-18T08:40:14.723 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler 2023-10-18T08:40:14.723 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-18T08:40:14.723 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-18T08:40:14.723 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-18T08:40:14.724 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-18T08:40:14.724 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-18T08:40:14.724 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r 2023-10-18T08:40:14.724 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-18T08:40:14.724 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-18T08:40:14.725 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe 2023-10-18T08:40:14.725 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-18T08:40:14.725 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-18T08:40:14.725 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic 2023-10-18T08:40:14.725 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-18T08:40:14.725 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-18T08:40:14.726 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-18T08:40:14.726 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-18T08:40:14.726 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-18T08:40:14.726 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-18T08:40:14.726 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-18T08:40:14.727 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci 2023-10-18T08:40:14.727 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-18T08:40:14.727 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-18T08:40:14.727 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-18T08:40:14.727 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc 2023-10-18T08:40:14.727 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-18T08:40:14.728 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-18T08:40:14.728 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-18T08:40:14.728 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas 2023-10-18T08:40:14.728 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-18T08:40:14.728 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc 2023-10-18T08:40:14.728 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-18T08:40:14.729 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid 2023-10-18T08:40:14.729 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-18T08:40:14.729 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-18T08:40:14.729 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-18T08:40:14.729 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-18T08:40:14.730 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas 2023-10-18T08:40:14.730 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-18T08:40:14.730 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas 2023-10-18T08:40:14.730 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-18T08:40:14.730 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-18T08:40:14.730 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001 2023-10-18T08:40:14.731 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-18T08:40:14.731 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-18T08:40:14.731 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-18T08:40:14.731 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx 2023-10-18T08:40:14.731 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-18T08:40:14.731 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-18T08:40:14.732 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx 2023-10-18T08:40:14.732 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-18T08:40:14.732 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-18T08:40:14.732 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-18T08:40:14.732 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-18T08:40:14.733 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-18T08:40:14.733 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-18T08:40:14.733 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-18T08:40:14.733 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-18T08:40:14.733 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-18T08:40:14.734 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-18T08:40:14.734 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-18T08:40:14.734 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2 2023-10-18T08:40:14.734 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-18T08:40:14.734 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-18T08:40:14.734 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-18T08:40:14.735 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-18T08:40:14.735 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb 2023-10-18T08:40:14.735 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-18T08:40:14.735 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target 2023-10-18T08:40:14.735 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi 2023-10-18T08:40:14.736 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-18T08:40:14.736 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback 2023-10-18T08:40:14.736 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-18T08:40:14.736 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-18T08:40:14.736 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-18T08:40:14.736 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-18T08:40:14.737 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-18T08:40:14.737 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-18T08:40:14.737 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc 2023-10-18T08:40:14.737 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-18T08:40:14.737 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal 2023-10-18T08:40:14.737 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel 2023-10-18T08:40:14.738 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-18T08:40:14.738 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-18T08:40:14.738 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-18T08:40:14.738 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-18T08:40:14.738 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-18T08:40:14.739 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-18T08:40:14.739 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-18T08:40:14.739 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-18T08:40:14.739 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-18T08:40:14.739 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-18T08:40:14.740 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-18T08:40:14.740 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-18T08:40:14.740 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-18T08:40:14.740 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-18T08:40:14.740 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-18T08:40:14.741 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-18T08:40:14.741 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty 2023-10-18T08:40:14.741 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless 2023-10-18T08:40:14.741 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-18T08:40:14.741 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-18T08:40:14.741 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-18T08:40:14.742 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-18T08:40:14.742 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial 2023-10-18T08:40:14.742 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250 2023-10-18T08:40:14.742 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-18T08:40:14.742 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-18T08:40:14.743 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm 2023-10-18T08:40:14.743 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-18T08:40:14.743 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs 2023-10-18T08:40:14.743 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core 2023-10-18T08:40:14.743 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-18T08:40:14.744 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host 2023-10-18T08:40:14.744 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-18T08:40:14.744 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio 2023-10-18T08:40:14.744 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-18T08:40:14.744 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-18T08:40:14.744 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-18T08:40:14.745 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-18T08:40:14.745 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-18T08:40:14.745 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb 2023-10-18T08:40:14.745 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class 2023-10-18T08:40:14.745 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-18T08:40:14.746 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-18T08:40:14.746 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-18T08:40:14.746 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-18T08:40:14.746 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host 2023-10-18T08:40:14.746 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-18T08:40:14.747 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image 2023-10-18T08:40:14.747 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-18T08:40:14.747 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-18T08:40:14.747 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc 2023-10-18T08:40:14.747 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-18T08:40:14.747 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-18T08:40:14.748 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-18T08:40:14.748 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-18T08:40:14.748 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-18T08:40:14.748 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-18T08:40:14.748 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-18T08:40:14.748 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-18T08:40:14.749 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-18T08:40:14.749 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-18T08:40:14.749 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga 2023-10-18T08:40:14.749 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-18T08:40:14.749 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-18T08:40:14.750 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-18T08:40:14.750 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-18T08:40:14.750 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-18T08:40:14.750 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-18T08:40:14.750 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-18T08:40:14.750 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy 2023-10-18T08:40:14.751 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-18T08:40:14.751 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial 2023-10-18T08:40:14.751 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-18T08:40:14.751 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-18T08:40:14.751 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-18T08:40:14.752 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-18T08:40:14.752 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-18T08:40:14.752 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-18T08:40:14.752 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-18T08:40:14.752 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-18T08:40:14.752 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-18T08:40:14.753 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-18T08:40:14.753 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-18T08:40:14.753 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-18T08:40:14.753 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-18T08:40:14.753 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-18T08:40:14.754 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-18T08:40:14.754 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-18T08:40:14.754 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-18T08:40:14.754 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-18T08:40:14.754 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-18T08:40:14.754 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-18T08:40:14.755 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-18T08:40:14.755 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-18T08:40:14.755 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-18T08:40:14.755 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-18T08:40:14.755 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-18T08:40:14.756 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-18T08:40:14.756 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-18T08:40:14.756 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-18T08:40:14.756 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-18T08:40:14.756 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-18T08:40:14.756 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-18T08:40:14.757 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-18T08:40:14.757 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-18T08:40:14.757 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-18T08:40:14.757 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-18T08:40:14.757 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-18T08:40:14.757 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-18T08:40:14.758 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-18T08:40:14.758 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-18T08:40:14.758 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-18T08:40:14.758 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-18T08:40:14.758 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-18T08:40:14.759 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-18T08:40:14.759 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-18T08:40:14.759 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-18T08:40:14.759 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage 2023-10-18T08:40:14.759 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-18T08:40:14.759 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-18T08:40:14.760 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-18T08:40:14.760 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-18T08:40:14.760 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-18T08:40:14.760 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-18T08:40:14.760 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-18T08:40:14.761 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-18T08:40:14.761 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-18T08:40:14.761 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-18T08:40:14.761 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-18T08:40:14.761 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-18T08:40:14.762 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-18T08:40:14.762 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-18T08:40:14.762 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-18T08:40:14.762 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip 2023-10-18T08:40:14.762 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-18T08:40:14.763 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-18T08:40:14.763 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-18T08:40:14.763 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio 2023-10-18T08:40:14.763 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci 2023-10-18T08:40:14.763 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-18T08:40:14.763 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-18T08:40:14.764 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-18T08:40:14.764 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-18T08:40:14.764 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost 2023-10-18T08:40:14.764 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-18T08:40:14.764 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-18T08:40:14.764 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-18T08:40:14.765 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-18T08:40:14.765 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video 2023-10-18T08:40:14.765 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight 2023-10-18T08:40:14.765 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-18T08:40:14.765 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-18T08:40:14.766 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-18T08:40:14.766 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio 2023-10-18T08:40:14.766 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio.ko 2023-10-18T08:40:14.766 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-18T08:40:14.766 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-18T08:40:14.766 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci.ko 2023-10-18T08:40:14.767 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-18T08:40:14.767 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-18T08:40:14.767 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_ring.ko 2023-10-18T08:40:14.767 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1 2023-10-18T08:40:14.767 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters 2023-10-18T08:40:14.768 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-18T08:40:14.768 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-18T08:40:14.768 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves 2023-10-18T08:40:14.768 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-18T08:40:14.768 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-18T08:40:14.768 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-18T08:40:14.769 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-18T08:40:14.769 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-18T08:40:14.769 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-18T08:40:14.769 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-18T08:40:14.769 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-18T08:40:14.769 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-18T08:40:14.770 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-18T08:40:14.770 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-18T08:40:14.770 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-18T08:40:14.770 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog 2023-10-18T08:40:14.770 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-18T08:40:14.771 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-18T08:40:14.771 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-18T08:40:14.771 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-18T08:40:14.771 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-18T08:40:14.771 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-18T08:40:14.771 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-18T08:40:14.772 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-18T08:40:14.772 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-18T08:40:14.772 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-18T08:40:14.772 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-18T08:40:14.772 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-18T08:40:14.772 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-18T08:40:14.773 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-18T08:40:14.773 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-18T08:40:14.773 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-18T08:40:14.773 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-18T08:40:14.773 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-18T08:40:14.773 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-18T08:40:14.774 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-18T08:40:14.774 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-18T08:40:14.774 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-18T08:40:14.774 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-18T08:40:14.774 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-18T08:40:14.775 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-18T08:40:14.775 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-18T08:40:14.775 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen 2023-10-18T08:40:14.775 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-18T08:40:14.775 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-18T08:40:14.775 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-18T08:40:14.776 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-18T08:40:14.776 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback 2023-10-18T08:40:14.776 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-18T08:40:14.776 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-18T08:40:14.776 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs 2023-10-18T08:40:14.777 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-18T08:40:14.777 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs 2023-10-18T08:40:14.777 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p 2023-10-18T08:40:14.777 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-18T08:40:14.777 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs 2023-10-18T08:40:14.777 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-18T08:40:14.778 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs 2023-10-18T08:40:14.778 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-18T08:40:14.778 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-18T08:40:14.778 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs 2023-10-18T08:40:14.778 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-18T08:40:14.778 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles 2023-10-18T08:40:14.779 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-18T08:40:14.779 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph 2023-10-18T08:40:14.779 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-18T08:40:14.779 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda 2023-10-18T08:40:14.779 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-18T08:40:14.780 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs 2023-10-18T08:40:14.780 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-18T08:40:14.780 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm 2023-10-18T08:40:14.780 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-18T08:40:14.780 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs 2023-10-18T08:40:14.780 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-18T08:40:14.781 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs 2023-10-18T08:40:14.781 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-18T08:40:14.781 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat 2023-10-18T08:40:14.781 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/fat.ko 2023-10-18T08:40:14.781 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-18T08:40:14.782 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/vfat.ko 2023-10-18T08:40:14.782 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache 2023-10-18T08:40:14.782 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-18T08:40:14.782 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse 2023-10-18T08:40:14.782 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-18T08:40:14.783 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/fuse.ko 2023-10-18T08:40:14.783 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2 2023-10-18T08:40:14.783 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-18T08:40:14.783 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs 2023-10-18T08:40:14.783 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-18T08:40:14.783 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus 2023-10-18T08:40:14.784 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-18T08:40:14.784 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs 2023-10-18T08:40:14.784 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-18T08:40:14.784 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs 2023-10-18T08:40:14.784 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-18T08:40:14.785 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd 2023-10-18T08:40:14.785 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-18T08:40:14.785 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix 2023-10-18T08:40:14.785 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-18T08:40:14.785 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs 2023-10-18T08:40:14.785 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-18T08:40:14.786 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs 2023-10-18T08:40:14.786 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout 2023-10-18T08:40:14.786 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-18T08:40:14.786 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout 2023-10-18T08:40:14.786 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-18T08:40:14.787 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout 2023-10-18T08:40:14.787 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-18T08:40:14.787 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-18T08:40:14.787 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-18T08:40:14.787 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-18T08:40:14.787 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common 2023-10-18T08:40:14.788 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-18T08:40:14.788 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-18T08:40:14.788 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd 2023-10-18T08:40:14.788 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-18T08:40:14.788 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2 2023-10-18T08:40:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-18T08:40:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls 2023-10-18T08:40:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-18T08:40:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-18T08:40:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-18T08:40:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-18T08:40:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-18T08:40:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-18T08:40:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-18T08:40:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-18T08:40:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-18T08:40:14.791 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-18T08:40:14.791 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-18T08:40:14.791 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-18T08:40:14.791 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-18T08:40:14.791 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-18T08:40:14.792 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-18T08:40:14.792 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-18T08:40:14.792 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-18T08:40:14.792 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-18T08:40:14.792 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-18T08:40:14.792 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-18T08:40:14.793 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-18T08:40:14.793 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-18T08:40:14.793 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-18T08:40:14.793 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-18T08:40:14.793 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-18T08:40:14.793 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-18T08:40:14.794 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-18T08:40:14.794 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-18T08:40:14.794 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-18T08:40:14.794 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-18T08:40:14.794 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-18T08:40:14.795 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-18T08:40:14.795 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-18T08:40:14.795 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-18T08:40:14.795 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-18T08:40:14.795 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-18T08:40:14.795 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-18T08:40:14.796 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-18T08:40:14.796 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-18T08:40:14.796 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-18T08:40:14.796 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-18T08:40:14.796 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-18T08:40:14.796 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-18T08:40:14.797 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-18T08:40:14.797 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-18T08:40:14.797 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-18T08:40:14.797 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-18T08:40:14.797 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-18T08:40:14.798 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-18T08:40:14.798 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-18T08:40:14.798 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2 2023-10-18T08:40:14.798 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster 2023-10-18T08:40:14.798 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-18T08:40:14.798 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm 2023-10-18T08:40:14.799 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-18T08:40:14.799 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs 2023-10-18T08:40:14.799 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-18T08:40:14.799 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-18T08:40:14.799 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-18T08:40:14.800 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-18T08:40:14.800 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-18T08:40:14.800 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs 2023-10-18T08:40:14.800 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-18T08:40:14.800 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore 2023-10-18T08:40:14.800 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-18T08:40:14.801 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs 2023-10-18T08:40:14.801 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-18T08:40:14.801 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs 2023-10-18T08:40:14.801 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-18T08:40:14.801 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb 2023-10-18T08:40:14.801 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client 2023-10-18T08:40:14.802 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-18T08:40:14.802 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common 2023-10-18T08:40:14.802 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-18T08:40:14.802 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-18T08:40:14.802 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs 2023-10-18T08:40:14.803 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-18T08:40:14.803 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv 2023-10-18T08:40:14.803 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-18T08:40:14.803 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf 2023-10-18T08:40:14.803 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-18T08:40:14.803 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs 2023-10-18T08:40:14.804 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-18T08:40:14.804 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs 2023-10-18T08:40:14.804 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-18T08:40:14.804 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel 2023-10-18T08:40:14.804 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu 2023-10-18T08:40:14.804 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu/rcutorture.ko 2023-10-18T08:40:14.805 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-18T08:40:14.805 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace 2023-10-18T08:40:14.805 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-18T08:40:14.805 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib 2023-10-18T08:40:14.805 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/asn1_encoder.ko 2023-10-18T08:40:14.806 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-18T08:40:14.806 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-18T08:40:14.806 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto 2023-10-18T08:40:14.806 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-18T08:40:14.806 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-18T08:40:14.807 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-18T08:40:14.807 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-18T08:40:14.807 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4 2023-10-18T08:40:14.807 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-18T08:40:14.807 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-18T08:40:14.807 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math 2023-10-18T08:40:14.808 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-18T08:40:14.808 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6 2023-10-18T08:40:14.808 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-18T08:40:14.808 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon 2023-10-18T08:40:14.808 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-18T08:40:14.808 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-18T08:40:14.809 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-18T08:40:14.809 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-18T08:40:14.809 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd 2023-10-18T08:40:14.809 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd/zstd_compress.ko 2023-10-18T08:40:14.809 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm 2023-10-18T08:40:14.810 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-18T08:40:14.810 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net 2023-10-18T08:40:14.810 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan 2023-10-18T08:40:14.810 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-18T08:40:14.810 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-18T08:40:14.811 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-18T08:40:14.811 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-18T08:40:14.811 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-18T08:40:14.811 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-18T08:40:14.811 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-18T08:40:14.811 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-18T08:40:14.812 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802 2023-10-18T08:40:14.813 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-18T08:40:14.813 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-18T08:40:14.813 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-18T08:40:14.813 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-18T08:40:14.814 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-18T08:40:14.814 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q 2023-10-18T08:40:14.814 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-18T08:40:14.814 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p 2023-10-18T08:40:14.814 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-18T08:40:14.815 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-18T08:40:14.815 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-18T08:40:14.815 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-18T08:40:14.815 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv 2023-10-18T08:40:14.815 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-18T08:40:14.815 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge 2023-10-18T08:40:14.816 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-18T08:40:14.816 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-18T08:40:14.816 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter 2023-10-18T08:40:14.816 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-18T08:40:14.816 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-18T08:40:14.817 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-18T08:40:14.817 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-18T08:40:14.817 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-18T08:40:14.817 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-18T08:40:14.817 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-18T08:40:14.817 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-18T08:40:14.818 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-18T08:40:14.818 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-18T08:40:14.818 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-18T08:40:14.818 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-18T08:40:14.818 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-18T08:40:14.818 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-18T08:40:14.819 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-18T08:40:14.819 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-18T08:40:14.819 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-18T08:40:14.819 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-18T08:40:14.819 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-18T08:40:14.820 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-18T08:40:14.820 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-18T08:40:14.820 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-18T08:40:14.820 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-18T08:40:14.820 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-18T08:40:14.820 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph 2023-10-18T08:40:14.821 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-18T08:40:14.821 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core 2023-10-18T08:40:14.821 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/failover.ko 2023-10-18T08:40:14.821 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-18T08:40:14.821 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp 2023-10-18T08:40:14.821 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-18T08:40:14.822 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-18T08:40:14.822 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-18T08:40:14.822 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-18T08:40:14.822 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver 2023-10-18T08:40:14.822 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-18T08:40:14.823 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa 2023-10-18T08:40:14.823 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-18T08:40:14.823 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-18T08:40:14.823 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-18T08:40:14.823 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-18T08:40:14.823 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-18T08:40:14.824 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154 2023-10-18T08:40:14.824 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan 2023-10-18T08:40:14.824 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-18T08:40:14.824 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-18T08:40:14.824 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-18T08:40:14.825 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4 2023-10-18T08:40:14.825 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-18T08:40:14.825 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-18T08:40:14.825 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-18T08:40:14.825 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-18T08:40:14.826 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-18T08:40:14.826 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-18T08:40:14.826 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-18T08:40:14.826 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-18T08:40:14.826 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-18T08:40:14.826 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-18T08:40:14.827 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter 2023-10-18T08:40:14.827 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-18T08:40:14.827 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-18T08:40:14.827 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-18T08:40:14.827 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-18T08:40:14.827 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-18T08:40:14.828 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-18T08:40:14.828 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-18T08:40:14.828 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-18T08:40:14.828 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-18T08:40:14.828 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-18T08:40:14.829 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-18T08:40:14.829 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-18T08:40:14.829 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-18T08:40:14.829 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-18T08:40:14.829 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-18T08:40:14.829 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-18T08:40:14.830 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-18T08:40:14.830 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-18T08:40:14.830 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-18T08:40:14.830 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-18T08:40:14.830 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-18T08:40:14.831 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-18T08:40:14.831 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-18T08:40:14.831 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-18T08:40:14.831 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-18T08:40:14.831 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-18T08:40:14.832 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-18T08:40:14.832 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-18T08:40:14.832 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-18T08:40:14.832 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-18T08:40:14.832 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-18T08:40:14.832 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-18T08:40:14.833 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-18T08:40:14.833 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-18T08:40:14.833 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-18T08:40:14.833 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-18T08:40:14.833 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-18T08:40:14.833 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-18T08:40:14.834 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6 2023-10-18T08:40:14.834 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-18T08:40:14.834 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-18T08:40:14.834 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-18T08:40:14.834 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-18T08:40:14.835 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-18T08:40:14.835 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-18T08:40:14.835 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-18T08:40:14.835 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter 2023-10-18T08:40:14.835 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-18T08:40:14.835 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-18T08:40:14.836 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-18T08:40:14.836 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-18T08:40:14.836 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-18T08:40:14.836 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-18T08:40:14.836 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-18T08:40:14.836 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-18T08:40:14.837 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-18T08:40:14.837 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-18T08:40:14.837 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-18T08:40:14.837 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-18T08:40:14.837 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-18T08:40:14.838 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-18T08:40:14.838 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-18T08:40:14.838 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-18T08:40:14.838 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-18T08:40:14.838 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-18T08:40:14.838 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-18T08:40:14.839 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-18T08:40:14.839 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-18T08:40:14.839 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-18T08:40:14.839 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-18T08:40:14.839 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-18T08:40:14.839 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-18T08:40:14.840 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-18T08:40:14.840 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-18T08:40:14.840 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key 2023-10-18T08:40:14.840 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-18T08:40:14.840 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp 2023-10-18T08:40:14.841 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-18T08:40:14.841 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-18T08:40:14.841 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-18T08:40:14.841 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-18T08:40:14.841 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-18T08:40:14.841 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-18T08:40:14.842 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-18T08:40:14.842 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc 2023-10-18T08:40:14.842 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-18T08:40:14.842 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154 2023-10-18T08:40:14.842 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-18T08:40:14.843 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls 2023-10-18T08:40:14.843 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-18T08:40:14.843 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter 2023-10-18T08:40:14.843 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset 2023-10-18T08:40:14.843 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-18T08:40:14.843 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-18T08:40:14.844 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-18T08:40:14.844 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-18T08:40:14.844 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-18T08:40:14.844 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-18T08:40:14.844 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-18T08:40:14.845 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-18T08:40:14.845 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-18T08:40:14.845 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-18T08:40:14.845 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-18T08:40:14.845 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-18T08:40:14.845 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-18T08:40:14.846 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-18T08:40:14.846 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-18T08:40:14.846 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-18T08:40:14.846 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs 2023-10-18T08:40:14.846 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-18T08:40:14.847 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-18T08:40:14.847 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-18T08:40:14.847 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-18T08:40:14.847 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-18T08:40:14.847 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-18T08:40:14.847 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-18T08:40:14.848 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-18T08:40:14.848 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-18T08:40:14.848 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-18T08:40:14.848 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-18T08:40:14.848 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-18T08:40:14.849 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-18T08:40:14.849 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-18T08:40:14.849 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-18T08:40:14.849 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-18T08:40:14.849 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-18T08:40:14.849 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-18T08:40:14.850 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-18T08:40:14.850 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-18T08:40:14.850 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-18T08:40:14.850 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-18T08:40:14.850 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-18T08:40:14.851 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-18T08:40:14.851 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-18T08:40:14.851 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-18T08:40:14.851 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-18T08:40:14.851 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-18T08:40:14.852 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-18T08:40:14.852 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-18T08:40:14.852 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-18T08:40:14.852 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-18T08:40:14.852 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-18T08:40:14.852 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-18T08:40:14.853 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-18T08:40:14.853 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-18T08:40:14.853 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-18T08:40:14.853 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-18T08:40:14.853 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-18T08:40:14.854 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-18T08:40:14.854 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-18T08:40:14.854 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-18T08:40:14.854 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-18T08:40:14.854 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-18T08:40:14.854 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-18T08:40:14.855 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-18T08:40:14.855 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-18T08:40:14.855 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-18T08:40:14.855 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-18T08:40:14.855 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-18T08:40:14.855 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-18T08:40:14.856 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-18T08:40:14.856 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-18T08:40:14.856 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-18T08:40:14.856 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-18T08:40:14.856 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-18T08:40:14.857 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-18T08:40:14.857 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-18T08:40:14.857 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_quota.ko 2023-10-18T08:40:14.857 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-18T08:40:14.857 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-18T08:40:14.857 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-18T08:40:14.858 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-18T08:40:14.858 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tunnel.ko 2023-10-18T08:40:14.858 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-18T08:40:14.858 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-18T08:40:14.858 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-18T08:40:14.859 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-18T08:40:14.859 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-18T08:40:14.859 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-18T08:40:14.859 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-18T08:40:14.859 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-18T08:40:14.859 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-18T08:40:14.860 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-18T08:40:14.860 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-18T08:40:14.860 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-18T08:40:14.860 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-18T08:40:14.860 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-18T08:40:14.861 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-18T08:40:14.861 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-18T08:40:14.861 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-18T08:40:14.861 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-18T08:40:14.861 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-18T08:40:14.861 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-18T08:40:14.862 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-18T08:40:14.862 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-18T08:40:14.862 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-18T08:40:14.862 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-18T08:40:14.862 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-18T08:40:14.863 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-18T08:40:14.863 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-18T08:40:14.863 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-18T08:40:14.863 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-18T08:40:14.863 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-18T08:40:14.864 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-18T08:40:14.864 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-18T08:40:14.864 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-18T08:40:14.864 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-18T08:40:14.864 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-18T08:40:14.864 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-18T08:40:14.865 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-18T08:40:14.865 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-18T08:40:14.865 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-18T08:40:14.865 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-18T08:40:14.865 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-18T08:40:14.865 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-18T08:40:14.866 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-18T08:40:14.866 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-18T08:40:14.866 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-18T08:40:14.866 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-18T08:40:14.866 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-18T08:40:14.867 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-18T08:40:14.867 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-18T08:40:14.867 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-18T08:40:14.867 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-18T08:40:14.867 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-18T08:40:14.867 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-18T08:40:14.868 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-18T08:40:14.868 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-18T08:40:14.868 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-18T08:40:14.868 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-18T08:40:14.868 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-18T08:40:14.868 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-18T08:40:14.869 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-18T08:40:14.869 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-18T08:40:14.869 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-18T08:40:14.869 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-18T08:40:14.869 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-18T08:40:14.870 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-18T08:40:14.870 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-18T08:40:14.870 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-18T08:40:14.870 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-18T08:40:14.870 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-18T08:40:14.871 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-18T08:40:14.871 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-18T08:40:14.871 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink 2023-10-18T08:40:14.871 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-18T08:40:14.871 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh 2023-10-18T08:40:14.871 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-18T08:40:14.872 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch 2023-10-18T08:40:14.872 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-18T08:40:14.872 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-18T08:40:14.872 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-18T08:40:14.872 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-18T08:40:14.872 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet 2023-10-18T08:40:14.873 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-18T08:40:14.873 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds 2023-10-18T08:40:14.873 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-18T08:40:14.873 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-18T08:40:14.873 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-18T08:40:14.873 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched 2023-10-18T08:40:14.874 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-18T08:40:14.874 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-18T08:40:14.874 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-18T08:40:14.874 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-18T08:40:14.874 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-18T08:40:14.875 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-18T08:40:14.875 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-18T08:40:14.875 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-18T08:40:14.875 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-18T08:40:14.875 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-18T08:40:14.876 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-18T08:40:14.876 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-18T08:40:14.876 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-18T08:40:14.876 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-18T08:40:14.876 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-18T08:40:14.876 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-18T08:40:14.877 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-18T08:40:14.877 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-18T08:40:14.877 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-18T08:40:14.877 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-18T08:40:14.877 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-18T08:40:14.877 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-18T08:40:14.878 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-18T08:40:14.878 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-18T08:40:14.878 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-18T08:40:14.878 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-18T08:40:14.878 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-18T08:40:14.879 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-18T08:40:14.879 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-18T08:40:14.879 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-18T08:40:14.879 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-18T08:40:14.879 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-18T08:40:14.879 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-18T08:40:14.880 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-18T08:40:14.880 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-18T08:40:14.880 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-18T08:40:14.880 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-18T08:40:14.880 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-18T08:40:14.881 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-18T08:40:14.881 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-18T08:40:14.881 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-18T08:40:14.881 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-18T08:40:14.881 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-18T08:40:14.881 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-18T08:40:14.882 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp 2023-10-18T08:40:14.882 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-18T08:40:14.882 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-18T08:40:14.882 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc 2023-10-18T08:40:14.882 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss 2023-10-18T08:40:14.883 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-18T08:40:14.883 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-18T08:40:14.883 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-18T08:40:14.883 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma 2023-10-18T08:40:14.883 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-18T08:40:14.883 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc 2023-10-18T08:40:14.884 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-18T08:40:14.884 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-18T08:40:14.884 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix 2023-10-18T08:40:14.884 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-18T08:40:14.884 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock 2023-10-18T08:40:14.884 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-18T08:40:14.885 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-18T08:40:14.885 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-18T08:40:14.885 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-18T08:40:14.885 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-18T08:40:14.885 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm 2023-10-18T08:40:14.885 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-18T08:40:14.886 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security 2023-10-18T08:40:14.886 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys 2023-10-18T08:40:14.886 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys 2023-10-18T08:40:14.886 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-18T08:40:14.886 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys 2023-10-18T08:40:14.887 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-18T08:40:14.887 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt 2023-10-18T08:40:14.887 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib 2023-10-18T08:40:14.887 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-18T08:40:14.887 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-18T08:40:14.887 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-18T08:40:14.888 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-18T08:40:14.888 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.alias.bin 2023-10-18T08:40:14.888 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-18T08:40:14.888 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-18T08:40:14.888 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-18T08:40:14.888 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-18T08:40:14.889 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-18T08:40:14.889 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-18T08:40:14.889 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-18T08:40:14.889 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-18T08:40:14.889 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-18T08:40:14.890 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-18T08:40:14.890 DEBUG:teuthology.orchestra.run.smithi144:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-18T08:40:14.913 INFO:teuthology.orchestra.run.smithi144.stdout:postinstall scriptlet (using /bin/sh): 2023-10-18T08:40:14.914 INFO:teuthology.orchestra.run.smithi144.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g47672119149b -a -r /boot/System.map-6.6.0-rc5-g47672119149b ]; then 2023-10-18T08:40:14.914 INFO:teuthology.orchestra.run.smithi144.stdout:cp /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm 2023-10-18T08:40:14.914 INFO:teuthology.orchestra.run.smithi144.stdout:cp /boot/System.map-6.6.0-rc5-g47672119149b /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-18T08:40:14.914 INFO:teuthology.orchestra.run.smithi144.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/System.map-6.6.0-rc5-g47672119149b 2023-10-18T08:40:14.914 INFO:teuthology.orchestra.run.smithi144.stdout:/sbin/installkernel 6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-18T08:40:14.915 INFO:teuthology.orchestra.run.smithi144.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-18T08:40:14.915 INFO:teuthology.orchestra.run.smithi144.stdout:fi 2023-10-18T08:40:14.915 INFO:teuthology.orchestra.run.smithi144.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-18T08:40:14.915 INFO:teuthology.orchestra.run.smithi144.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-18T08:40:14.915 INFO:teuthology.orchestra.run.smithi144.stdout:new-kernel-pkg --remove 6.6.0-rc5-g47672119149b --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g47672119149b.img 2023-10-18T08:40:14.915 INFO:teuthology.orchestra.run.smithi144.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-18T08:40:14.916 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-install remove 6.6.0-rc5-g47672119149b 2023-10-18T08:40:14.916 INFO:teuthology.orchestra.run.smithi144.stdout:fi 2023-10-18T08:40:14.916 INFO:teuthology.orchestra.run.smithi144.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-18T08:40:14.916 INFO:teuthology.orchestra.run.smithi144.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-18T08:40:14.916 INFO:teuthology.orchestra.run.smithi144.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g47672119149b 2023-10-18T08:40:14.917 INFO:teuthology.orchestra.run.smithi144.stdout:fi 2023-10-18T08:40:14.917 DEBUG:teuthology.orchestra.run.smithi144:> sudo rpm -qi grub2-tools 2023-10-18T08:40:14.993 INFO:teuthology.orchestra.run.smithi144.stdout:Name : grub2-tools 2023-10-18T08:40:14.993 INFO:teuthology.orchestra.run.smithi144.stdout:Epoch : 1 2023-10-18T08:40:14.993 INFO:teuthology.orchestra.run.smithi144.stdout:Version : 2.06 2023-10-18T08:40:14.993 INFO:teuthology.orchestra.run.smithi144.stdout:Release : 68.el9 2023-10-18T08:40:14.994 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: x86_64 2023-10-18T08:40:14.994 INFO:teuthology.orchestra.run.smithi144.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-18T08:40:14.994 INFO:teuthology.orchestra.run.smithi144.stdout:Group : Unspecified 2023-10-18T08:40:14.994 INFO:teuthology.orchestra.run.smithi144.stdout:Size : 8264254 2023-10-18T08:40:14.994 INFO:teuthology.orchestra.run.smithi144.stdout:License : GPLv3+ 2023-10-18T08:40:14.995 INFO:teuthology.orchestra.run.smithi144.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-18T08:40:14.995 INFO:teuthology.orchestra.run.smithi144.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-18T08:40:14.995 INFO:teuthology.orchestra.run.smithi144.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-18T08:40:14.995 INFO:teuthology.orchestra.run.smithi144.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-18T08:40:14.995 INFO:teuthology.orchestra.run.smithi144.stdout:Packager : builder@centos.org 2023-10-18T08:40:14.996 INFO:teuthology.orchestra.run.smithi144.stdout:Vendor : CentOS 2023-10-18T08:40:14.996 INFO:teuthology.orchestra.run.smithi144.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-18T08:40:14.996 INFO:teuthology.orchestra.run.smithi144.stdout:Summary : Support tools for GRUB. 2023-10-18T08:40:14.996 INFO:teuthology.orchestra.run.smithi144.stdout:Description : 2023-10-18T08:40:14.996 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:40:14.996 INFO:teuthology.orchestra.run.smithi144.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-18T08:40:14.997 INFO:teuthology.orchestra.run.smithi144.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-18T08:40:14.997 INFO:teuthology.orchestra.run.smithi144.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-18T08:40:14.997 INFO:teuthology.orchestra.run.smithi144.stdout:hardware devices. 2023-10-18T08:40:14.997 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:40:14.997 INFO:teuthology.orchestra.run.smithi144.stdout:This subpackage provides tools for support of all platforms. 2023-10-18T08:40:14.998 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-18T08:40:14.999 INFO:teuthology.task.kernel:Updating grub on smithi144 to boot 6.6.0-rc5-g47672119149b 2023-10-18T08:40:14.999 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-18T08:40:15.608 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2023-10-18T08:40:17.035 INFO:teuthology.orchestra.run.smithi144.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-18T08:40:17.062 INFO:teuthology.orchestra.run.smithi144.stderr:done 2023-10-18T08:40:17.064 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2023-10-18T08:40:17.079 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.SdAdyhGWwk 2023-10-18T08:40:17.080 DEBUG:teuthology.orchestra.run.smithi144:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.SdAdyhGWwk 2023-10-18T08:40:17.148 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.SdAdyhGWwk 2023-10-18T08:40:17.283 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.SdAdyhGWwk is 6KB 2023-10-18T08:40:17.332 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.SdAdyhGWwk 2023-10-18T08:40:17.346 DEBUG:teuthology.orchestra.run.smithi144:> sudo /bin/ls /boot/loader/entries || true 2023-10-18T08:40:17.411 INFO:teuthology.orchestra.run.smithi144.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-18T08:40:17.411 INFO:teuthology.orchestra.run.smithi144.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-18T08:40:17.411 INFO:teuthology.orchestra.run.smithi144.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-18T08:40:17.412 INFO:teuthology.orchestra.run.smithi144.stdout:15949e3c88704300882482eddfd7baae-6.6.0-rc5-g47672119149b.conf 2023-10-18T08:40:17.413 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-rc5-g47672119149b 2023-10-18T08:40:17.573 DEBUG:teuthology.orchestra.run.smithi144:> sudo shutdown -r now 2023-10-18T08:40:18.370 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi040.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-18T08:40:18.371 DEBUG:teuthology.orchestra.run.smithi040:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-18T08:40:18.459 INFO:teuthology.orchestra.run.smithi040.stdout:/boot/System.map-6.6.0-rc5-g47672119149b 2023-10-18T08:40:18.460 INFO:teuthology.orchestra.run.smithi040.stdout:/boot/config-6.6.0-rc5-g47672119149b 2023-10-18T08:40:18.460 INFO:teuthology.orchestra.run.smithi040.stdout:/boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-18T08:40:18.460 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b 2023-10-18T08:40:18.461 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel 2023-10-18T08:40:18.461 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch 2023-10-18T08:40:18.461 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86 2023-10-18T08:40:18.461 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto 2023-10-18T08:40:18.462 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-18T08:40:18.462 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-18T08:40:18.463 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-18T08:40:18.463 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-18T08:40:18.463 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-18T08:40:18.463 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-18T08:40:18.463 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-18T08:40:18.463 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-18T08:40:18.464 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-18T08:40:18.464 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-18T08:40:18.464 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-18T08:40:18.464 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-18T08:40:18.464 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-18T08:40:18.465 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-18T08:40:18.465 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-18T08:40:18.465 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-18T08:40:18.465 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-18T08:40:18.465 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-18T08:40:18.465 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-18T08:40:18.466 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-18T08:40:18.466 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel 2023-10-18T08:40:18.466 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu 2023-10-18T08:40:18.466 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce 2023-10-18T08:40:18.466 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-18T08:40:18.467 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm 2023-10-18T08:40:18.467 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-18T08:40:18.467 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-18T08:40:18.467 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-18T08:40:18.468 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto 2023-10-18T08:40:18.468 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-18T08:40:18.468 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-18T08:40:18.468 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-18T08:40:18.468 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx 2023-10-18T08:40:18.468 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-18T08:40:18.469 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-18T08:40:18.469 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-18T08:40:18.469 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-18T08:40:18.469 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-18T08:40:18.469 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-18T08:40:18.470 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-18T08:40:18.470 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-18T08:40:18.470 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-18T08:40:18.470 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-18T08:40:18.470 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-18T08:40:18.471 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-18T08:40:18.471 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-18T08:40:18.471 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-18T08:40:18.471 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-18T08:40:18.471 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-18T08:40:18.472 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-18T08:40:18.472 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-18T08:40:18.472 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-18T08:40:18.472 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-18T08:40:18.472 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-18T08:40:18.472 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-18T08:40:18.473 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-18T08:40:18.473 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-18T08:40:18.473 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-18T08:40:18.473 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-18T08:40:18.473 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-18T08:40:18.473 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-18T08:40:18.474 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-18T08:40:18.474 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-18T08:40:18.474 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-18T08:40:18.474 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-18T08:40:18.474 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-18T08:40:18.475 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-18T08:40:18.475 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-18T08:40:18.475 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-18T08:40:18.475 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-18T08:40:18.475 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-18T08:40:18.476 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-18T08:40:18.476 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-18T08:40:18.476 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-18T08:40:18.476 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-18T08:40:18.476 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-18T08:40:18.476 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-18T08:40:18.477 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-18T08:40:18.477 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers 2023-10-18T08:40:18.477 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi 2023-10-18T08:40:18.477 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-18T08:40:18.477 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-18T08:40:18.478 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/custom_method.ko 2023-10-18T08:40:18.478 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-18T08:40:18.478 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-18T08:40:18.478 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-18T08:40:18.478 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-18T08:40:18.479 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-18T08:40:18.479 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata 2023-10-18T08:40:18.479 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-18T08:40:18.479 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-18T08:40:18.479 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ata_generic.ko 2023-10-18T08:40:18.480 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-18T08:40:18.480 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-18T08:40:18.480 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-18T08:40:18.480 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-18T08:40:18.480 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-18T08:40:18.480 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-18T08:40:18.481 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-18T08:40:18.481 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-18T08:40:18.481 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-18T08:40:18.481 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-18T08:40:18.481 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-18T08:40:18.482 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-18T08:40:18.482 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-18T08:40:18.482 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-18T08:40:18.482 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-18T08:40:18.482 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-18T08:40:18.483 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-18T08:40:18.483 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-18T08:40:18.483 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-18T08:40:18.483 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-18T08:40:18.483 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-18T08:40:18.483 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-18T08:40:18.484 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-18T08:40:18.484 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-18T08:40:18.484 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-18T08:40:18.484 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-18T08:40:18.485 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-18T08:40:18.485 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-18T08:40:18.485 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-18T08:40:18.485 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-18T08:40:18.485 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-18T08:40:18.485 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-18T08:40:18.486 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-18T08:40:18.486 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-18T08:40:18.486 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-18T08:40:18.486 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sis.ko 2023-10-18T08:40:18.486 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-18T08:40:18.487 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-18T08:40:18.487 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-18T08:40:18.487 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-18T08:40:18.487 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-18T08:40:18.487 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-18T08:40:18.488 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-18T08:40:18.488 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-18T08:40:18.488 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-18T08:40:18.488 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-18T08:40:18.488 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-18T08:40:18.488 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-18T08:40:18.489 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-18T08:40:18.489 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-18T08:40:18.489 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-18T08:40:18.489 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-18T08:40:18.489 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-18T08:40:18.490 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay 2023-10-18T08:40:18.490 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-18T08:40:18.490 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base 2023-10-18T08:40:18.490 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap 2023-10-18T08:40:18.490 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-18T08:40:18.491 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-18T08:40:18.491 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma 2023-10-18T08:40:18.491 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-18T08:40:18.491 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block 2023-10-18T08:40:18.491 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe 2023-10-18T08:40:18.491 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-18T08:40:18.492 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/brd.ko 2023-10-18T08:40:18.492 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd 2023-10-18T08:40:18.492 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-18T08:40:18.492 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-18T08:40:18.492 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/loop.ko 2023-10-18T08:40:18.493 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx 2023-10-18T08:40:18.493 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-18T08:40:18.493 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-18T08:40:18.493 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk 2023-10-18T08:40:18.493 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-18T08:40:18.493 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-18T08:40:18.494 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-18T08:40:18.494 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/virtio_blk.ko 2023-10-18T08:40:18.494 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback 2023-10-18T08:40:18.494 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-18T08:40:18.494 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkfront.ko 2023-10-18T08:40:18.495 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram 2023-10-18T08:40:18.495 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-18T08:40:18.495 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char 2023-10-18T08:40:18.495 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-18T08:40:18.495 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random 2023-10-18T08:40:18.495 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-18T08:40:18.496 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-18T08:40:18.496 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-18T08:40:18.496 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-18T08:40:18.496 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-18T08:40:18.496 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi 2023-10-18T08:40:18.497 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-18T08:40:18.497 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-18T08:40:18.497 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-18T08:40:18.497 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-18T08:40:18.497 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-18T08:40:18.498 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-18T08:40:18.498 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-18T08:40:18.498 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave 2023-10-18T08:40:18.498 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-18T08:40:18.498 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-18T08:40:18.499 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-18T08:40:18.499 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm 2023-10-18T08:40:18.499 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm.ko 2023-10-18T08:40:18.499 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-18T08:40:18.499 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-18T08:40:18.500 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-18T08:40:18.500 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-18T08:40:18.500 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-18T08:40:18.500 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/uv_mmtimer.ko 2023-10-18T08:40:18.500 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/virtio_console.ko 2023-10-18T08:40:18.501 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq 2023-10-18T08:40:18.501 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-18T08:40:18.501 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-18T08:40:18.501 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-18T08:40:18.501 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-18T08:40:18.502 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-18T08:40:18.502 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-18T08:40:18.502 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto 2023-10-18T08:40:18.502 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp 2023-10-18T08:40:18.502 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-18T08:40:18.503 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-18T08:40:18.503 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel 2023-10-18T08:40:18.503 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat 2023-10-18T08:40:18.503 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-18T08:40:18.503 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-18T08:40:18.503 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-18T08:40:18.504 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-18T08:40:18.504 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-18T08:40:18.504 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-18T08:40:18.504 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio 2023-10-18T08:40:18.504 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-18T08:40:18.505 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca 2023-10-18T08:40:18.505 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-18T08:40:18.505 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq 2023-10-18T08:40:18.505 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-18T08:40:18.505 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma 2023-10-18T08:40:18.506 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw 2023-10-18T08:40:18.506 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-18T08:40:18.506 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat 2023-10-18T08:40:18.506 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-18T08:40:18.506 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac 2023-10-18T08:40:18.506 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-18T08:40:18.507 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-18T08:40:18.507 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-18T08:40:18.507 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-18T08:40:18.507 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-18T08:40:18.507 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-18T08:40:18.508 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-18T08:40:18.508 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-18T08:40:18.508 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-18T08:40:18.508 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-18T08:40:18.508 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-18T08:40:18.508 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-18T08:40:18.509 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-18T08:40:18.509 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware 2023-10-18T08:40:18.509 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/edd.ko 2023-10-18T08:40:18.509 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-18T08:40:18.509 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio 2023-10-18T08:40:18.510 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-18T08:40:18.510 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-18T08:40:18.510 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid 2023-10-18T08:40:18.510 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-18T08:40:18.510 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-18T08:40:18.511 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-18T08:40:18.511 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-18T08:40:18.511 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-18T08:40:18.511 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-18T08:40:18.511 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-18T08:40:18.511 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-18T08:40:18.512 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-18T08:40:18.512 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-18T08:40:18.512 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-18T08:40:18.512 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-18T08:40:18.512 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-18T08:40:18.512 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-18T08:40:18.513 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-18T08:40:18.513 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-18T08:40:18.513 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-18T08:40:18.513 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-18T08:40:18.513 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-18T08:40:18.514 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-18T08:40:18.514 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-18T08:40:18.514 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-18T08:40:18.514 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-18T08:40:18.514 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-18T08:40:18.515 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-18T08:40:18.515 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-18T08:40:18.515 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-18T08:40:18.515 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-18T08:40:18.515 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-18T08:40:18.515 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-18T08:40:18.516 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-18T08:40:18.516 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-18T08:40:18.516 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-18T08:40:18.516 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-18T08:40:18.516 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-18T08:40:18.516 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-18T08:40:18.517 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-18T08:40:18.517 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-18T08:40:18.517 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-18T08:40:18.517 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-18T08:40:18.517 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-18T08:40:18.518 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-18T08:40:18.518 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-18T08:40:18.518 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-18T08:40:18.518 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-18T08:40:18.518 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-18T08:40:18.519 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-18T08:40:18.519 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-18T08:40:18.519 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-18T08:40:18.520 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-18T08:40:18.520 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-18T08:40:18.520 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-18T08:40:18.521 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-18T08:40:18.521 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-18T08:40:18.521 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-18T08:40:18.522 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-18T08:40:18.522 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-18T08:40:18.522 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-18T08:40:18.523 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-18T08:40:18.523 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-18T08:40:18.523 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-18T08:40:18.524 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-18T08:40:18.524 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-18T08:40:18.524 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-18T08:40:18.524 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv 2023-10-18T08:40:18.525 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-18T08:40:18.525 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-18T08:40:18.525 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-18T08:40:18.525 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon 2023-10-18T08:40:18.525 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-18T08:40:18.526 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-18T08:40:18.526 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-18T08:40:18.526 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-18T08:40:18.526 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-18T08:40:18.526 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-18T08:40:18.527 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-18T08:40:18.527 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-18T08:40:18.527 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-18T08:40:18.527 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-18T08:40:18.528 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-18T08:40:18.528 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-18T08:40:18.528 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-18T08:40:18.528 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-18T08:40:18.528 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-18T08:40:18.528 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-18T08:40:18.529 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-18T08:40:18.529 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-18T08:40:18.529 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-18T08:40:18.529 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-18T08:40:18.529 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-18T08:40:18.530 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-18T08:40:18.530 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-18T08:40:18.530 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-18T08:40:18.530 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-18T08:40:18.530 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-18T08:40:18.531 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-18T08:40:18.531 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-18T08:40:18.531 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-18T08:40:18.531 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-18T08:40:18.531 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-18T08:40:18.532 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-18T08:40:18.532 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-18T08:40:18.532 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-18T08:40:18.532 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-18T08:40:18.532 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-18T08:40:18.532 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-18T08:40:18.533 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-18T08:40:18.533 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-18T08:40:18.533 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-18T08:40:18.533 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-18T08:40:18.533 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-18T08:40:18.534 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-18T08:40:18.534 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-18T08:40:18.534 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-18T08:40:18.534 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-18T08:40:18.534 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-18T08:40:18.535 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-18T08:40:18.535 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-18T08:40:18.535 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-18T08:40:18.535 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-18T08:40:18.535 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-18T08:40:18.535 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-18T08:40:18.536 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-18T08:40:18.536 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-18T08:40:18.536 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-18T08:40:18.536 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-18T08:40:18.536 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-18T08:40:18.537 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-18T08:40:18.537 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-18T08:40:18.537 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-18T08:40:18.537 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-18T08:40:18.537 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-18T08:40:18.537 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-18T08:40:18.538 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-18T08:40:18.538 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-18T08:40:18.538 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-18T08:40:18.538 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-18T08:40:18.538 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-18T08:40:18.539 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-18T08:40:18.539 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-18T08:40:18.539 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-18T08:40:18.539 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-18T08:40:18.539 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-18T08:40:18.539 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-18T08:40:18.540 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-18T08:40:18.540 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-18T08:40:18.540 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-18T08:40:18.540 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-18T08:40:18.540 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-18T08:40:18.541 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-18T08:40:18.541 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-18T08:40:18.541 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-18T08:40:18.541 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-18T08:40:18.541 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-18T08:40:18.541 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-18T08:40:18.542 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-18T08:40:18.542 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-18T08:40:18.542 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus 2023-10-18T08:40:18.542 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-18T08:40:18.542 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-18T08:40:18.543 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-18T08:40:18.543 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-18T08:40:18.543 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-18T08:40:18.543 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-18T08:40:18.543 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-18T08:40:18.544 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-18T08:40:18.544 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-18T08:40:18.544 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-18T08:40:18.544 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-18T08:40:18.544 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-18T08:40:18.545 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-18T08:40:18.545 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-18T08:40:18.545 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-18T08:40:18.545 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-18T08:40:18.545 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-18T08:40:18.546 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-18T08:40:18.546 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-18T08:40:18.546 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-18T08:40:18.546 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-18T08:40:18.546 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-18T08:40:18.547 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-18T08:40:18.547 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-18T08:40:18.547 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-18T08:40:18.547 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-18T08:40:18.547 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-18T08:40:18.548 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-18T08:40:18.548 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-18T08:40:18.548 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-18T08:40:18.548 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-18T08:40:18.548 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-18T08:40:18.548 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-18T08:40:18.549 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-18T08:40:18.549 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-18T08:40:18.549 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-18T08:40:18.549 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-18T08:40:18.549 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-18T08:40:18.550 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-18T08:40:18.550 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-18T08:40:18.550 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-18T08:40:18.550 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c 2023-10-18T08:40:18.550 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos 2023-10-18T08:40:18.551 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-18T08:40:18.551 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-18T08:40:18.551 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses 2023-10-18T08:40:18.551 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-18T08:40:18.551 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-18T08:40:18.551 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-18T08:40:18.552 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-18T08:40:18.552 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-18T08:40:18.552 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-18T08:40:18.552 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-18T08:40:18.552 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-18T08:40:18.553 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-18T08:40:18.553 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-18T08:40:18.553 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-18T08:40:18.553 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-18T08:40:18.553 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-18T08:40:18.553 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-18T08:40:18.554 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-18T08:40:18.554 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-18T08:40:18.554 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-18T08:40:18.554 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-18T08:40:18.554 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-18T08:40:18.555 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-18T08:40:18.555 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-18T08:40:18.555 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-18T08:40:18.555 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-18T08:40:18.555 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-dev.ko 2023-10-18T08:40:18.556 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-18T08:40:18.556 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-18T08:40:18.556 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-18T08:40:18.556 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-18T08:40:18.556 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband 2023-10-18T08:40:18.557 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core 2023-10-18T08:40:18.557 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-18T08:40:18.557 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-18T08:40:18.557 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-18T08:40:18.557 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-18T08:40:18.558 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-18T08:40:18.558 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-18T08:40:18.558 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-18T08:40:18.558 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw 2023-10-18T08:40:18.558 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4 2023-10-18T08:40:18.558 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-18T08:40:18.559 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4 2023-10-18T08:40:18.559 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-18T08:40:18.559 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5 2023-10-18T08:40:18.559 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-18T08:40:18.559 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca 2023-10-18T08:40:18.560 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-18T08:40:18.560 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp 2023-10-18T08:40:18.560 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib 2023-10-18T08:40:18.560 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-18T08:40:18.560 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser 2023-10-18T08:40:18.561 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-18T08:40:18.561 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert 2023-10-18T08:40:18.561 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-18T08:40:18.561 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp 2023-10-18T08:40:18.561 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-18T08:40:18.561 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt 2023-10-18T08:40:18.562 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-18T08:40:18.562 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input 2023-10-18T08:40:18.562 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport 2023-10-18T08:40:18.562 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-18T08:40:18.562 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-18T08:40:18.563 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-18T08:40:18.563 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-18T08:40:18.563 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard 2023-10-18T08:40:18.563 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-18T08:40:18.563 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse 2023-10-18T08:40:18.564 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-18T08:40:18.564 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-18T08:40:18.564 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-18T08:40:18.564 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-18T08:40:18.564 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-18T08:40:18.564 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-18T08:40:18.565 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-18T08:40:18.565 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-18T08:40:18.565 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4 2023-10-18T08:40:18.565 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-18T08:40:18.565 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio 2023-10-18T08:40:18.566 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-18T08:40:18.566 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-18T08:40:18.566 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-18T08:40:18.566 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-18T08:40:18.566 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-18T08:40:18.566 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu 2023-10-18T08:40:18.567 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd 2023-10-18T08:40:18.567 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-18T08:40:18.567 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds 2023-10-18T08:40:18.567 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-18T08:40:18.567 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-18T08:40:18.568 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-18T08:40:18.568 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-18T08:40:18.568 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger 2023-10-18T08:40:18.568 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-18T08:40:18.568 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-18T08:40:18.568 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-18T08:40:18.569 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-18T08:40:18.569 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-18T08:40:18.569 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-18T08:40:18.569 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-18T08:40:18.569 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-18T08:40:18.570 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md 2023-10-18T08:40:18.570 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache 2023-10-18T08:40:18.570 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-18T08:40:18.570 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-18T08:40:18.570 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-18T08:40:18.571 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-18T08:40:18.571 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-18T08:40:18.571 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-18T08:40:18.571 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-18T08:40:18.571 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-18T08:40:18.571 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-18T08:40:18.572 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-18T08:40:18.572 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-18T08:40:18.572 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-18T08:40:18.572 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-18T08:40:18.572 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-18T08:40:18.573 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-18T08:40:18.573 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-18T08:40:18.573 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-18T08:40:18.573 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-18T08:40:18.573 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-18T08:40:18.573 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data 2023-10-18T08:40:18.574 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-18T08:40:18.574 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-18T08:40:18.574 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-18T08:40:18.574 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-18T08:40:18.574 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-18T08:40:18.575 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message 2023-10-18T08:40:18.575 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion 2023-10-18T08:40:18.575 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-18T08:40:18.575 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-18T08:40:18.575 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-18T08:40:18.576 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-18T08:40:18.576 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-18T08:40:18.576 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-18T08:40:18.576 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-18T08:40:18.576 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd 2023-10-18T08:40:18.577 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-18T08:40:18.577 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-18T08:40:18.577 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mfd-core.ko 2023-10-18T08:40:18.577 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-18T08:40:18.577 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-18T08:40:18.577 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-18T08:40:18.578 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-18T08:40:18.578 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc 2023-10-18T08:40:18.578 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl 2023-10-18T08:40:18.578 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-18T08:40:18.578 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-18T08:40:18.579 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-18T08:40:18.579 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-18T08:40:18.579 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710 2023-10-18T08:40:18.579 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-18T08:40:18.579 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom 2023-10-18T08:40:18.580 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-18T08:40:18.580 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-18T08:40:18.580 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-18T08:40:18.580 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-18T08:40:18.580 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-18T08:40:18.580 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-18T08:40:18.581 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-18T08:40:18.581 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-18T08:40:18.581 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d 2023-10-18T08:40:18.581 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-18T08:40:18.581 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-18T08:40:18.581 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei 2023-10-18T08:40:18.582 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-18T08:40:18.582 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-18T08:40:18.582 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-18T08:40:18.582 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru 2023-10-18T08:40:18.582 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-18T08:40:18.583 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp 2023-10-18T08:40:18.583 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-18T08:40:18.583 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-18T08:40:18.583 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-18T08:40:18.583 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-18T08:40:18.584 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-18T08:40:18.584 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-18T08:40:18.584 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-18T08:40:18.584 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci 2023-10-18T08:40:18.584 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-18T08:40:18.585 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net 2023-10-18T08:40:18.585 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding 2023-10-18T08:40:18.585 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-18T08:40:18.585 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa 2023-10-18T08:40:18.585 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53 2023-10-18T08:40:18.586 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-18T08:40:18.586 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-18T08:40:18.586 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-18T08:40:18.586 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-18T08:40:18.586 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-18T08:40:18.587 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-18T08:40:18.587 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-18T08:40:18.587 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet 2023-10-18T08:40:18.587 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com 2023-10-18T08:40:18.587 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-18T08:40:18.587 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-18T08:40:18.588 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-18T08:40:18.588 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-18T08:40:18.588 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390 2023-10-18T08:40:18.588 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-18T08:40:18.588 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-18T08:40:18.589 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-18T08:40:18.589 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-18T08:40:18.589 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec 2023-10-18T08:40:18.589 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-18T08:40:18.589 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere 2023-10-18T08:40:18.589 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-18T08:40:18.590 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon 2023-10-18T08:40:18.590 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-18T08:40:18.590 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera 2023-10-18T08:40:18.590 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-18T08:40:18.590 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd 2023-10-18T08:40:18.591 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-18T08:40:18.591 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-18T08:40:18.591 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-18T08:40:18.591 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros 2023-10-18T08:40:18.591 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx 2023-10-18T08:40:18.591 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-18T08:40:18.592 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-18T08:40:18.592 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-18T08:40:18.592 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-18T08:40:18.592 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-18T08:40:18.592 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-18T08:40:18.592 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-18T08:40:18.593 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-18T08:40:18.593 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom 2023-10-18T08:40:18.593 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-18T08:40:18.593 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-18T08:40:18.593 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-18T08:40:18.594 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-18T08:40:18.594 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-18T08:40:18.594 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-18T08:40:18.594 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-18T08:40:18.594 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-18T08:40:18.595 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade 2023-10-18T08:40:18.595 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna 2023-10-18T08:40:18.595 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-18T08:40:18.595 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio 2023-10-18T08:40:18.595 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-18T08:40:18.596 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-18T08:40:18.596 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-18T08:40:18.596 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-18T08:40:18.596 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-18T08:40:18.596 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-18T08:40:18.596 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-18T08:40:18.597 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-18T08:40:18.597 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-18T08:40:18.597 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-18T08:40:18.597 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco 2023-10-18T08:40:18.597 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic 2023-10-18T08:40:18.598 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-18T08:40:18.598 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec 2023-10-18T08:40:18.598 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip 2023-10-18T08:40:18.598 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-18T08:40:18.598 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-18T08:40:18.598 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-18T08:40:18.599 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-18T08:40:18.599 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-18T08:40:18.599 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-18T08:40:18.599 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink 2023-10-18T08:40:18.600 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-18T08:40:18.600 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-18T08:40:18.600 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-18T08:40:18.600 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex 2023-10-18T08:40:18.600 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet 2023-10-18T08:40:18.601 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-18T08:40:18.601 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-18T08:40:18.601 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-18T08:40:18.601 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel 2023-10-18T08:40:18.601 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-18T08:40:18.602 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000 2023-10-18T08:40:18.602 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-18T08:40:18.602 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-18T08:40:18.602 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-18T08:40:18.602 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-18T08:40:18.602 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-18T08:40:18.603 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e 2023-10-18T08:40:18.603 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-18T08:40:18.603 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf 2023-10-18T08:40:18.603 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-18T08:40:18.603 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb 2023-10-18T08:40:18.604 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-18T08:40:18.604 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-18T08:40:18.604 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-18T08:40:18.604 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-18T08:40:18.604 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-18T08:40:18.604 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-18T08:40:18.605 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-18T08:40:18.605 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-18T08:40:18.605 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell 2023-10-18T08:40:18.605 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-18T08:40:18.605 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-18T08:40:18.606 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-18T08:40:18.606 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox 2023-10-18T08:40:18.606 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-18T08:40:18.606 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-18T08:40:18.606 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-18T08:40:18.607 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-18T08:40:18.607 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-18T08:40:18.607 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-18T08:40:18.607 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel 2023-10-18T08:40:18.607 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-18T08:40:18.608 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom 2023-10-18T08:40:18.608 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-18T08:40:18.608 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-18T08:40:18.608 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi 2023-10-18T08:40:18.608 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-18T08:40:18.608 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-18T08:40:18.609 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion 2023-10-18T08:40:18.609 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-18T08:40:18.609 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia 2023-10-18T08:40:18.609 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-18T08:40:18.609 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines 2023-10-18T08:40:18.610 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-18T08:40:18.610 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-18T08:40:18.610 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic 2023-10-18T08:40:18.610 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-18T08:40:18.610 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-18T08:40:18.610 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-18T08:40:18.611 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-18T08:40:18.611 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-18T08:40:18.611 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc 2023-10-18T08:40:18.611 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-18T08:40:18.611 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek 2023-10-18T08:40:18.612 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-18T08:40:18.612 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-18T08:40:18.612 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-18T08:40:18.612 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-18T08:40:18.612 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker 2023-10-18T08:40:18.613 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-18T08:40:18.613 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc 2023-10-18T08:40:18.613 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-18T08:40:18.613 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan 2023-10-18T08:40:18.613 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-18T08:40:18.613 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis 2023-10-18T08:40:18.614 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-18T08:40:18.614 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-18T08:40:18.614 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc 2023-10-18T08:40:18.614 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-18T08:40:18.614 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-18T08:40:18.615 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-18T08:40:18.615 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-18T08:40:18.615 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro 2023-10-18T08:40:18.615 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-18T08:40:18.615 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-18T08:40:18.616 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-18T08:40:18.616 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun 2023-10-18T08:40:18.616 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-18T08:40:18.616 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-18T08:40:18.616 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-18T08:40:18.616 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-18T08:40:18.617 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti 2023-10-18T08:40:18.617 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-18T08:40:18.617 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti 2023-10-18T08:40:18.617 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-18T08:40:18.617 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via 2023-10-18T08:40:18.618 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-18T08:40:18.618 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-18T08:40:18.618 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet 2023-10-18T08:40:18.618 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-18T08:40:18.618 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-18T08:40:18.618 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom 2023-10-18T08:40:18.619 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-18T08:40:18.619 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-18T08:40:18.619 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv 2023-10-18T08:40:18.619 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-18T08:40:18.619 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154 2023-10-18T08:40:18.620 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-18T08:40:18.620 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-18T08:40:18.620 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan 2023-10-18T08:40:18.620 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-18T08:40:18.620 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-18T08:40:18.621 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-18T08:40:18.621 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio 2023-10-18T08:40:18.621 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-18T08:40:18.621 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-18T08:40:18.621 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-18T08:40:18.621 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-18T08:40:18.622 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-18T08:40:18.622 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/net_failover.ko 2023-10-18T08:40:18.622 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-18T08:40:18.622 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-18T08:40:18.622 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs 2023-10-18T08:40:18.623 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-18T08:40:18.623 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-18T08:40:18.623 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy 2023-10-18T08:40:18.623 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-18T08:40:18.623 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-18T08:40:18.623 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-18T08:40:18.624 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-18T08:40:18.624 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-18T08:40:18.624 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-18T08:40:18.624 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-18T08:40:18.624 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-18T08:40:18.625 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/dp83640.ko 2023-10-18T08:40:18.625 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-18T08:40:18.625 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-18T08:40:18.625 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-18T08:40:18.625 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-18T08:40:18.626 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-18T08:40:18.626 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-18T08:40:18.626 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-18T08:40:18.626 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-18T08:40:18.626 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-18T08:40:18.626 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-18T08:40:18.627 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-18T08:40:18.627 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-18T08:40:18.627 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp 2023-10-18T08:40:18.627 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-18T08:40:18.628 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-18T08:40:18.628 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-18T08:40:18.628 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-18T08:40:18.628 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-18T08:40:18.628 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-18T08:40:18.628 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-18T08:40:18.629 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-18T08:40:18.629 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-18T08:40:18.629 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip 2023-10-18T08:40:18.629 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slhc.ko 2023-10-18T08:40:18.629 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-18T08:40:18.630 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-18T08:40:18.630 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-18T08:40:18.630 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team 2023-10-18T08:40:18.630 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-18T08:40:18.630 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-18T08:40:18.631 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-18T08:40:18.631 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-18T08:40:18.631 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-18T08:40:18.631 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-18T08:40:18.631 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tun.ko 2023-10-18T08:40:18.631 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-18T08:40:18.632 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/virtio_net.ko 2023-10-18T08:40:18.632 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3 2023-10-18T08:40:18.632 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-18T08:40:18.632 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan 2023-10-18T08:40:18.632 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-18T08:40:18.633 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback 2023-10-18T08:40:18.633 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-18T08:40:18.633 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netfront.ko 2023-10-18T08:40:18.633 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb 2023-10-18T08:40:18.633 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-18T08:40:18.634 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme 2023-10-18T08:40:18.634 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host 2023-10-18T08:40:18.634 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-18T08:40:18.634 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-18T08:40:18.634 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport 2023-10-18T08:40:18.634 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-18T08:40:18.635 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-18T08:40:18.635 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-18T08:40:18.635 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-18T08:40:18.635 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci 2023-10-18T08:40:18.635 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug 2023-10-18T08:40:18.636 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-18T08:40:18.636 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie 2023-10-18T08:40:18.636 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-18T08:40:18.636 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-18T08:40:18.636 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia 2023-10-18T08:40:18.636 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-18T08:40:18.637 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-18T08:40:18.637 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-18T08:40:18.637 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl 2023-10-18T08:40:18.637 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel 2023-10-18T08:40:18.637 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-18T08:40:18.637 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform 2023-10-18T08:40:18.638 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86 2023-10-18T08:40:18.638 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-18T08:40:18.638 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-18T08:40:18.638 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-18T08:40:18.638 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-18T08:40:18.639 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-18T08:40:18.639 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-18T08:40:18.639 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-18T08:40:18.639 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-18T08:40:18.639 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-18T08:40:18.640 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-18T08:40:18.640 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-18T08:40:18.640 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-18T08:40:18.640 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel 2023-10-18T08:40:18.640 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-18T08:40:18.640 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-18T08:40:18.641 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-18T08:40:18.641 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-18T08:40:18.641 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-18T08:40:18.641 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-18T08:40:18.641 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-18T08:40:18.641 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-18T08:40:18.642 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-18T08:40:18.642 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-18T08:40:18.642 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-18T08:40:18.642 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-18T08:40:18.642 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap 2023-10-18T08:40:18.643 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-18T08:40:18.643 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-18T08:40:18.643 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps 2023-10-18T08:40:18.643 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients 2023-10-18T08:40:18.643 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-18T08:40:18.644 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-18T08:40:18.644 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-18T08:40:18.644 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-18T08:40:18.644 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp 2023-10-18T08:40:18.644 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-18T08:40:18.644 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-18T08:40:18.645 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc 2023-10-18T08:40:18.645 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-18T08:40:18.645 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-18T08:40:18.645 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-18T08:40:18.645 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-18T08:40:18.646 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-18T08:40:18.646 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-18T08:40:18.646 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-18T08:40:18.646 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-18T08:40:18.646 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-18T08:40:18.646 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-18T08:40:18.647 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-18T08:40:18.647 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-18T08:40:18.647 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-18T08:40:18.647 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-18T08:40:18.647 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-18T08:40:18.648 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-18T08:40:18.648 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-18T08:40:18.648 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-18T08:40:18.648 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-18T08:40:18.648 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-18T08:40:18.648 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-18T08:40:18.649 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-18T08:40:18.649 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-18T08:40:18.649 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-18T08:40:18.649 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-18T08:40:18.649 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-18T08:40:18.649 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-18T08:40:18.650 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-18T08:40:18.650 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-18T08:40:18.650 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-18T08:40:18.650 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-18T08:40:18.650 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi 2023-10-18T08:40:18.651 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-18T08:40:18.651 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-18T08:40:18.651 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-18T08:40:18.651 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-18T08:40:18.651 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-18T08:40:18.651 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid 2023-10-18T08:40:18.652 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-18T08:40:18.652 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-18T08:40:18.652 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx 2023-10-18T08:40:18.652 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-18T08:40:18.652 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-18T08:40:18.652 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-18T08:40:18.653 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr 2023-10-18T08:40:18.653 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-18T08:40:18.653 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-18T08:40:18.653 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi 2023-10-18T08:40:18.653 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-18T08:40:18.653 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa 2023-10-18T08:40:18.654 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-18T08:40:18.654 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc 2023-10-18T08:40:18.654 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-18T08:40:18.654 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i 2023-10-18T08:40:18.654 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-18T08:40:18.655 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-18T08:40:18.655 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor 2023-10-18T08:40:18.655 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-18T08:40:18.655 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi 2023-10-18T08:40:18.655 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-18T08:40:18.656 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-18T08:40:18.656 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-18T08:40:18.656 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-18T08:40:18.656 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-18T08:40:18.656 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-18T08:40:18.656 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler 2023-10-18T08:40:18.657 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-18T08:40:18.657 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-18T08:40:18.657 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-18T08:40:18.657 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-18T08:40:18.657 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-18T08:40:18.658 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r 2023-10-18T08:40:18.658 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-18T08:40:18.658 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-18T08:40:18.658 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe 2023-10-18T08:40:18.658 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-18T08:40:18.658 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-18T08:40:18.659 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic 2023-10-18T08:40:18.659 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-18T08:40:18.659 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-18T08:40:18.659 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-18T08:40:18.659 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-18T08:40:18.660 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-18T08:40:18.660 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-18T08:40:18.660 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-18T08:40:18.660 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci 2023-10-18T08:40:18.660 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-18T08:40:18.660 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-18T08:40:18.661 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-18T08:40:18.661 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc 2023-10-18T08:40:18.661 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-18T08:40:18.661 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-18T08:40:18.661 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-18T08:40:18.662 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas 2023-10-18T08:40:18.662 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-18T08:40:18.662 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc 2023-10-18T08:40:18.662 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-18T08:40:18.662 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid 2023-10-18T08:40:18.663 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-18T08:40:18.663 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-18T08:40:18.663 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-18T08:40:18.663 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-18T08:40:18.663 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas 2023-10-18T08:40:18.663 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-18T08:40:18.664 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas 2023-10-18T08:40:18.664 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-18T08:40:18.664 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-18T08:40:18.664 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001 2023-10-18T08:40:18.664 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-18T08:40:18.665 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-18T08:40:18.665 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-18T08:40:18.665 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx 2023-10-18T08:40:18.665 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-18T08:40:18.665 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-18T08:40:18.665 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx 2023-10-18T08:40:18.666 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-18T08:40:18.666 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-18T08:40:18.666 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-18T08:40:18.666 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-18T08:40:18.666 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-18T08:40:18.667 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-18T08:40:18.667 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-18T08:40:18.667 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-18T08:40:18.667 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-18T08:40:18.667 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-18T08:40:18.667 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-18T08:40:18.668 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2 2023-10-18T08:40:18.668 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-18T08:40:18.668 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-18T08:40:18.668 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-18T08:40:18.668 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-18T08:40:18.669 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb 2023-10-18T08:40:18.669 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-18T08:40:18.669 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target 2023-10-18T08:40:18.669 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi 2023-10-18T08:40:18.669 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-18T08:40:18.670 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback 2023-10-18T08:40:18.670 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-18T08:40:18.670 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-18T08:40:18.670 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-18T08:40:18.670 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-18T08:40:18.671 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-18T08:40:18.671 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-18T08:40:18.671 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc 2023-10-18T08:40:18.671 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-18T08:40:18.671 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal 2023-10-18T08:40:18.671 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel 2023-10-18T08:40:18.672 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-18T08:40:18.672 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-18T08:40:18.672 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-18T08:40:18.672 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-18T08:40:18.672 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-18T08:40:18.673 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-18T08:40:18.673 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-18T08:40:18.673 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-18T08:40:18.673 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-18T08:40:18.673 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-18T08:40:18.673 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-18T08:40:18.674 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-18T08:40:18.674 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-18T08:40:18.674 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-18T08:40:18.674 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-18T08:40:18.674 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-18T08:40:18.675 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty 2023-10-18T08:40:18.675 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless 2023-10-18T08:40:18.675 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-18T08:40:18.675 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-18T08:40:18.675 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-18T08:40:18.675 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-18T08:40:18.676 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial 2023-10-18T08:40:18.676 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250 2023-10-18T08:40:18.676 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-18T08:40:18.676 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-18T08:40:18.676 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm 2023-10-18T08:40:18.677 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-18T08:40:18.677 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs 2023-10-18T08:40:18.677 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core 2023-10-18T08:40:18.677 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-18T08:40:18.677 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host 2023-10-18T08:40:18.677 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-18T08:40:18.678 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio 2023-10-18T08:40:18.678 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-18T08:40:18.678 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-18T08:40:18.678 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-18T08:40:18.678 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-18T08:40:18.679 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-18T08:40:18.679 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb 2023-10-18T08:40:18.679 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class 2023-10-18T08:40:18.679 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-18T08:40:18.679 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-18T08:40:18.679 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-18T08:40:18.680 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-18T08:40:18.680 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host 2023-10-18T08:40:18.680 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-18T08:40:18.680 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image 2023-10-18T08:40:18.680 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-18T08:40:18.681 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-18T08:40:18.681 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc 2023-10-18T08:40:18.681 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-18T08:40:18.681 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-18T08:40:18.681 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-18T08:40:18.681 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-18T08:40:18.682 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-18T08:40:18.682 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-18T08:40:18.682 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-18T08:40:18.682 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-18T08:40:18.682 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-18T08:40:18.683 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-18T08:40:18.683 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga 2023-10-18T08:40:18.683 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-18T08:40:18.683 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-18T08:40:18.683 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-18T08:40:18.683 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-18T08:40:18.684 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-18T08:40:18.684 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-18T08:40:18.684 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-18T08:40:18.684 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy 2023-10-18T08:40:18.684 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-18T08:40:18.685 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial 2023-10-18T08:40:18.685 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-18T08:40:18.685 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-18T08:40:18.685 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-18T08:40:18.685 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-18T08:40:18.686 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-18T08:40:18.686 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-18T08:40:18.686 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-18T08:40:18.686 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-18T08:40:18.686 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-18T08:40:18.687 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-18T08:40:18.687 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-18T08:40:18.687 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-18T08:40:18.687 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-18T08:40:18.687 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-18T08:40:18.687 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-18T08:40:18.688 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-18T08:40:18.688 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-18T08:40:18.688 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-18T08:40:18.688 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-18T08:40:18.688 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-18T08:40:18.689 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-18T08:40:18.689 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-18T08:40:18.689 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-18T08:40:18.689 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-18T08:40:18.689 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-18T08:40:18.690 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-18T08:40:18.690 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-18T08:40:18.690 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-18T08:40:18.690 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-18T08:40:18.690 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-18T08:40:18.690 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-18T08:40:18.691 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-18T08:40:18.691 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-18T08:40:18.691 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-18T08:40:18.691 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-18T08:40:18.691 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-18T08:40:18.691 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-18T08:40:18.692 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-18T08:40:18.692 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-18T08:40:18.692 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-18T08:40:18.692 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-18T08:40:18.692 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-18T08:40:18.693 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-18T08:40:18.693 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-18T08:40:18.693 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-18T08:40:18.693 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage 2023-10-18T08:40:18.693 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-18T08:40:18.693 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-18T08:40:18.694 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-18T08:40:18.694 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-18T08:40:18.694 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-18T08:40:18.694 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-18T08:40:18.694 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-18T08:40:18.694 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-18T08:40:18.695 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-18T08:40:18.695 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-18T08:40:18.695 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-18T08:40:18.695 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-18T08:40:18.695 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-18T08:40:18.695 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-18T08:40:18.696 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-18T08:40:18.696 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip 2023-10-18T08:40:18.696 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-18T08:40:18.696 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-18T08:40:18.696 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-18T08:40:18.697 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio 2023-10-18T08:40:18.697 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci 2023-10-18T08:40:18.697 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-18T08:40:18.697 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-18T08:40:18.697 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-18T08:40:18.698 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-18T08:40:18.698 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost 2023-10-18T08:40:18.698 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-18T08:40:18.698 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-18T08:40:18.698 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-18T08:40:18.698 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-18T08:40:18.699 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video 2023-10-18T08:40:18.699 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight 2023-10-18T08:40:18.699 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-18T08:40:18.699 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-18T08:40:18.699 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-18T08:40:18.700 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio 2023-10-18T08:40:18.700 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio.ko 2023-10-18T08:40:18.700 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-18T08:40:18.700 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-18T08:40:18.700 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci.ko 2023-10-18T08:40:18.700 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-18T08:40:18.701 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-18T08:40:18.701 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_ring.ko 2023-10-18T08:40:18.701 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1 2023-10-18T08:40:18.701 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters 2023-10-18T08:40:18.701 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-18T08:40:18.702 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-18T08:40:18.702 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves 2023-10-18T08:40:18.702 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-18T08:40:18.702 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-18T08:40:18.702 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-18T08:40:18.702 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-18T08:40:18.703 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-18T08:40:18.703 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-18T08:40:18.703 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-18T08:40:18.703 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-18T08:40:18.703 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-18T08:40:18.704 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-18T08:40:18.704 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-18T08:40:18.704 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-18T08:40:18.704 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog 2023-10-18T08:40:18.704 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-18T08:40:18.704 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-18T08:40:18.705 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-18T08:40:18.705 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-18T08:40:18.705 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-18T08:40:18.705 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-18T08:40:18.705 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-18T08:40:18.706 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-18T08:40:18.706 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-18T08:40:18.706 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-18T08:40:18.706 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-18T08:40:18.706 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-18T08:40:18.706 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-18T08:40:18.707 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-18T08:40:18.707 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-18T08:40:18.707 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-18T08:40:18.707 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-18T08:40:18.707 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-18T08:40:18.708 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-18T08:40:18.708 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-18T08:40:18.708 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-18T08:40:18.708 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-18T08:40:18.708 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-18T08:40:18.708 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-18T08:40:18.709 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-18T08:40:18.709 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-18T08:40:18.709 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen 2023-10-18T08:40:18.709 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-18T08:40:18.709 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-18T08:40:18.710 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-18T08:40:18.710 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-18T08:40:18.710 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback 2023-10-18T08:40:18.710 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-18T08:40:18.710 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-18T08:40:18.710 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs 2023-10-18T08:40:18.711 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-18T08:40:18.711 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs 2023-10-18T08:40:18.711 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p 2023-10-18T08:40:18.711 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-18T08:40:18.711 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs 2023-10-18T08:40:18.712 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-18T08:40:18.712 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs 2023-10-18T08:40:18.712 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-18T08:40:18.712 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-18T08:40:18.712 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs 2023-10-18T08:40:18.712 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-18T08:40:18.713 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles 2023-10-18T08:40:18.713 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-18T08:40:18.713 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph 2023-10-18T08:40:18.713 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-18T08:40:18.713 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda 2023-10-18T08:40:18.714 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-18T08:40:18.714 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs 2023-10-18T08:40:18.714 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-18T08:40:18.714 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm 2023-10-18T08:40:18.714 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-18T08:40:18.714 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs 2023-10-18T08:40:18.715 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-18T08:40:18.715 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs 2023-10-18T08:40:18.715 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-18T08:40:18.715 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat 2023-10-18T08:40:18.715 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/fat.ko 2023-10-18T08:40:18.716 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-18T08:40:18.716 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/vfat.ko 2023-10-18T08:40:18.716 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache 2023-10-18T08:40:18.716 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-18T08:40:18.716 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse 2023-10-18T08:40:18.716 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-18T08:40:18.717 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/fuse.ko 2023-10-18T08:40:18.717 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2 2023-10-18T08:40:18.717 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-18T08:40:18.717 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs 2023-10-18T08:40:18.717 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-18T08:40:18.718 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus 2023-10-18T08:40:18.718 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-18T08:40:18.718 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs 2023-10-18T08:40:18.718 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-18T08:40:18.718 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs 2023-10-18T08:40:18.718 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-18T08:40:18.719 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd 2023-10-18T08:40:18.719 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-18T08:40:18.719 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix 2023-10-18T08:40:18.719 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-18T08:40:18.719 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs 2023-10-18T08:40:18.720 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-18T08:40:18.720 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs 2023-10-18T08:40:18.720 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout 2023-10-18T08:40:18.720 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-18T08:40:18.720 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout 2023-10-18T08:40:18.720 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-18T08:40:18.721 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout 2023-10-18T08:40:18.721 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-18T08:40:18.721 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-18T08:40:18.721 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-18T08:40:18.721 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-18T08:40:18.721 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common 2023-10-18T08:40:18.722 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-18T08:40:18.722 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-18T08:40:18.722 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd 2023-10-18T08:40:18.722 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-18T08:40:18.722 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2 2023-10-18T08:40:18.723 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-18T08:40:18.723 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls 2023-10-18T08:40:18.723 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-18T08:40:18.723 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-18T08:40:18.723 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-18T08:40:18.724 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-18T08:40:18.724 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-18T08:40:18.724 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-18T08:40:18.724 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-18T08:40:18.724 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-18T08:40:18.724 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-18T08:40:18.725 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-18T08:40:18.725 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-18T08:40:18.725 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-18T08:40:18.725 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-18T08:40:18.725 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-18T08:40:18.725 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-18T08:40:18.726 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-18T08:40:18.726 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-18T08:40:18.726 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-18T08:40:18.726 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-18T08:40:18.726 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-18T08:40:18.727 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-18T08:40:18.727 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-18T08:40:18.727 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-18T08:40:18.727 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-18T08:40:18.727 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-18T08:40:18.728 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-18T08:40:18.728 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-18T08:40:18.728 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-18T08:40:18.728 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-18T08:40:18.728 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-18T08:40:18.729 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-18T08:40:18.729 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-18T08:40:18.729 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-18T08:40:18.729 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-18T08:40:18.729 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-18T08:40:18.730 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-18T08:40:18.730 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-18T08:40:18.730 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-18T08:40:18.730 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-18T08:40:18.730 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-18T08:40:18.731 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-18T08:40:18.731 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-18T08:40:18.731 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-18T08:40:18.731 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-18T08:40:18.731 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-18T08:40:18.731 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-18T08:40:18.732 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-18T08:40:18.732 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-18T08:40:18.732 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-18T08:40:18.732 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-18T08:40:18.732 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2 2023-10-18T08:40:18.732 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster 2023-10-18T08:40:18.733 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-18T08:40:18.733 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm 2023-10-18T08:40:18.733 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-18T08:40:18.733 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs 2023-10-18T08:40:18.733 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-18T08:40:18.734 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-18T08:40:18.734 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-18T08:40:18.734 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-18T08:40:18.734 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-18T08:40:18.734 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs 2023-10-18T08:40:18.735 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-18T08:40:18.735 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore 2023-10-18T08:40:18.735 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-18T08:40:18.735 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs 2023-10-18T08:40:18.735 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-18T08:40:18.735 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs 2023-10-18T08:40:18.736 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-18T08:40:18.736 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb 2023-10-18T08:40:18.736 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client 2023-10-18T08:40:18.736 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-18T08:40:18.736 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common 2023-10-18T08:40:18.737 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-18T08:40:18.737 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-18T08:40:18.737 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs 2023-10-18T08:40:18.737 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-18T08:40:18.737 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv 2023-10-18T08:40:18.737 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-18T08:40:18.738 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf 2023-10-18T08:40:18.738 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-18T08:40:18.738 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs 2023-10-18T08:40:18.738 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-18T08:40:18.738 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs 2023-10-18T08:40:18.739 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-18T08:40:18.739 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel 2023-10-18T08:40:18.739 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu 2023-10-18T08:40:18.739 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu/rcutorture.ko 2023-10-18T08:40:18.739 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-18T08:40:18.739 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace 2023-10-18T08:40:18.740 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-18T08:40:18.740 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib 2023-10-18T08:40:18.740 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/asn1_encoder.ko 2023-10-18T08:40:18.740 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-18T08:40:18.740 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-18T08:40:18.741 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto 2023-10-18T08:40:18.741 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-18T08:40:18.741 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-18T08:40:18.741 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-18T08:40:18.741 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-18T08:40:18.741 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4 2023-10-18T08:40:18.742 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-18T08:40:18.742 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-18T08:40:18.742 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math 2023-10-18T08:40:18.742 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-18T08:40:18.742 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6 2023-10-18T08:40:18.743 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-18T08:40:18.743 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon 2023-10-18T08:40:18.743 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-18T08:40:18.743 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-18T08:40:18.743 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-18T08:40:18.744 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-18T08:40:18.744 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd 2023-10-18T08:40:18.744 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd/zstd_compress.ko 2023-10-18T08:40:18.744 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm 2023-10-18T08:40:18.744 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-18T08:40:18.744 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net 2023-10-18T08:40:18.745 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan 2023-10-18T08:40:18.745 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-18T08:40:18.745 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-18T08:40:18.745 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-18T08:40:18.745 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-18T08:40:18.745 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-18T08:40:18.746 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-18T08:40:18.746 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-18T08:40:18.746 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-18T08:40:18.746 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802 2023-10-18T08:40:18.746 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-18T08:40:18.747 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-18T08:40:18.747 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-18T08:40:18.747 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-18T08:40:18.747 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-18T08:40:18.747 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q 2023-10-18T08:40:18.748 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-18T08:40:18.748 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p 2023-10-18T08:40:18.748 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-18T08:40:18.748 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-18T08:40:18.748 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-18T08:40:18.748 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-18T08:40:18.749 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv 2023-10-18T08:40:18.749 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-18T08:40:18.749 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge 2023-10-18T08:40:18.749 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-18T08:40:18.749 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-18T08:40:18.750 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter 2023-10-18T08:40:18.750 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-18T08:40:18.750 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-18T08:40:18.750 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-18T08:40:18.750 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-18T08:40:18.750 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-18T08:40:18.751 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-18T08:40:18.751 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-18T08:40:18.751 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-18T08:40:18.751 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-18T08:40:18.751 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-18T08:40:18.752 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-18T08:40:18.752 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-18T08:40:18.752 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-18T08:40:18.752 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-18T08:40:18.752 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-18T08:40:18.752 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-18T08:40:18.753 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-18T08:40:18.753 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-18T08:40:18.753 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-18T08:40:18.753 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-18T08:40:18.753 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-18T08:40:18.753 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-18T08:40:18.754 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-18T08:40:18.754 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-18T08:40:18.754 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph 2023-10-18T08:40:18.754 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-18T08:40:18.754 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core 2023-10-18T08:40:18.755 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/failover.ko 2023-10-18T08:40:18.755 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-18T08:40:18.755 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp 2023-10-18T08:40:18.755 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-18T08:40:18.755 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-18T08:40:18.756 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-18T08:40:18.756 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-18T08:40:18.756 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver 2023-10-18T08:40:18.756 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-18T08:40:18.756 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa 2023-10-18T08:40:18.756 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-18T08:40:18.757 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-18T08:40:18.757 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-18T08:40:18.757 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-18T08:40:18.757 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-18T08:40:18.757 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154 2023-10-18T08:40:18.757 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan 2023-10-18T08:40:18.758 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-18T08:40:18.758 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-18T08:40:18.758 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-18T08:40:18.758 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4 2023-10-18T08:40:18.758 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-18T08:40:18.759 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-18T08:40:18.759 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-18T08:40:18.759 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-18T08:40:18.759 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-18T08:40:18.759 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-18T08:40:18.759 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-18T08:40:18.760 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-18T08:40:18.760 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-18T08:40:18.760 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-18T08:40:18.760 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter 2023-10-18T08:40:18.760 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-18T08:40:18.761 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-18T08:40:18.761 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-18T08:40:18.761 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-18T08:40:18.761 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-18T08:40:18.761 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-18T08:40:18.761 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-18T08:40:18.762 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-18T08:40:18.762 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-18T08:40:18.762 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-18T08:40:18.762 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-18T08:40:18.762 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-18T08:40:18.762 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-18T08:40:18.763 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-18T08:40:18.763 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-18T08:40:18.763 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-18T08:40:18.763 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-18T08:40:18.763 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-18T08:40:18.764 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-18T08:40:18.764 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-18T08:40:18.764 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-18T08:40:18.764 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-18T08:40:18.764 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-18T08:40:18.764 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-18T08:40:18.765 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-18T08:40:18.765 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-18T08:40:18.765 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-18T08:40:18.765 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-18T08:40:18.765 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-18T08:40:18.765 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-18T08:40:18.766 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-18T08:40:18.766 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-18T08:40:18.766 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-18T08:40:18.766 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-18T08:40:18.766 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-18T08:40:18.767 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-18T08:40:18.767 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-18T08:40:18.767 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-18T08:40:18.767 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6 2023-10-18T08:40:18.767 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-18T08:40:18.768 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-18T08:40:18.768 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-18T08:40:18.768 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-18T08:40:18.768 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-18T08:40:18.768 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-18T08:40:18.768 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-18T08:40:18.769 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter 2023-10-18T08:40:18.769 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-18T08:40:18.769 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-18T08:40:18.769 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-18T08:40:18.769 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-18T08:40:18.770 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-18T08:40:18.770 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-18T08:40:18.770 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-18T08:40:18.770 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-18T08:40:18.770 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-18T08:40:18.771 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-18T08:40:18.771 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-18T08:40:18.771 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-18T08:40:18.771 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-18T08:40:18.771 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-18T08:40:18.771 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-18T08:40:18.772 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-18T08:40:18.772 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-18T08:40:18.772 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-18T08:40:18.772 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-18T08:40:18.772 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-18T08:40:18.773 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-18T08:40:18.773 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-18T08:40:18.773 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-18T08:40:18.773 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-18T08:40:18.773 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-18T08:40:18.774 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-18T08:40:18.774 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-18T08:40:18.774 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key 2023-10-18T08:40:18.774 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-18T08:40:18.774 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp 2023-10-18T08:40:18.775 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-18T08:40:18.775 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-18T08:40:18.775 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-18T08:40:18.775 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-18T08:40:18.775 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-18T08:40:18.775 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-18T08:40:18.776 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-18T08:40:18.776 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc 2023-10-18T08:40:18.776 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-18T08:40:18.776 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154 2023-10-18T08:40:18.776 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-18T08:40:18.777 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls 2023-10-18T08:40:18.777 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-18T08:40:18.777 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter 2023-10-18T08:40:18.777 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset 2023-10-18T08:40:18.777 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-18T08:40:18.778 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-18T08:40:18.778 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-18T08:40:18.778 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-18T08:40:18.778 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-18T08:40:18.778 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-18T08:40:18.778 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-18T08:40:18.779 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-18T08:40:18.779 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-18T08:40:18.779 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-18T08:40:18.779 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-18T08:40:18.779 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-18T08:40:18.780 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-18T08:40:18.780 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-18T08:40:18.780 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-18T08:40:18.780 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-18T08:40:18.780 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs 2023-10-18T08:40:18.781 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-18T08:40:18.781 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-18T08:40:18.781 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-18T08:40:18.781 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-18T08:40:18.781 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-18T08:40:18.781 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-18T08:40:18.782 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-18T08:40:18.782 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-18T08:40:18.782 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-18T08:40:18.782 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-18T08:40:18.782 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-18T08:40:18.783 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-18T08:40:18.783 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-18T08:40:18.783 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-18T08:40:18.783 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-18T08:40:18.783 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-18T08:40:18.783 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-18T08:40:18.784 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-18T08:40:18.784 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-18T08:40:18.784 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-18T08:40:18.784 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-18T08:40:18.784 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-18T08:40:18.785 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-18T08:40:18.785 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-18T08:40:18.785 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-18T08:40:18.785 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-18T08:40:18.785 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-18T08:40:18.786 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-18T08:40:18.786 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-18T08:40:18.786 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-18T08:40:18.786 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-18T08:40:18.786 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-18T08:40:18.787 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-18T08:40:18.787 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-18T08:40:18.787 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-18T08:40:18.787 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-18T08:40:18.787 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-18T08:40:18.788 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-18T08:40:18.788 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-18T08:40:18.788 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-18T08:40:18.788 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-18T08:40:18.788 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-18T08:40:18.788 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-18T08:40:18.789 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-18T08:40:18.789 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-18T08:40:18.789 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-18T08:40:18.789 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-18T08:40:18.789 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-18T08:40:18.790 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-18T08:40:18.790 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-18T08:40:18.790 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-18T08:40:18.790 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-18T08:40:18.790 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-18T08:40:18.790 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-18T08:40:18.791 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-18T08:40:18.791 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-18T08:40:18.791 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-18T08:40:18.791 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-18T08:40:18.791 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_quota.ko 2023-10-18T08:40:18.792 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-18T08:40:18.792 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-18T08:40:18.792 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-18T08:40:18.792 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-18T08:40:18.792 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tunnel.ko 2023-10-18T08:40:18.793 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-18T08:40:18.793 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-18T08:40:18.793 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-18T08:40:18.793 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-18T08:40:18.793 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-18T08:40:18.793 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-18T08:40:18.794 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-18T08:40:18.794 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-18T08:40:18.794 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-18T08:40:18.794 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-18T08:40:18.794 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-18T08:40:18.795 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-18T08:40:18.795 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-18T08:40:18.795 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-18T08:40:18.795 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-18T08:40:18.795 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-18T08:40:18.795 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-18T08:40:18.796 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-18T08:40:18.796 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-18T08:40:18.796 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-18T08:40:18.796 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-18T08:40:18.796 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-18T08:40:18.797 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-18T08:40:18.797 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-18T08:40:18.797 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-18T08:40:18.797 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-18T08:40:18.797 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-18T08:40:18.797 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-18T08:40:18.798 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-18T08:40:18.798 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-18T08:40:18.798 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-18T08:40:18.798 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-18T08:40:18.798 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-18T08:40:18.799 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-18T08:40:18.799 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-18T08:40:18.799 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-18T08:40:18.799 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-18T08:40:18.799 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-18T08:40:18.800 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-18T08:40:18.800 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-18T08:40:18.800 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-18T08:40:18.800 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-18T08:40:18.800 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-18T08:40:18.801 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-18T08:40:18.801 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-18T08:40:18.801 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-18T08:40:18.801 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-18T08:40:18.801 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-18T08:40:18.801 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-18T08:40:18.802 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-18T08:40:18.802 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-18T08:40:18.802 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-18T08:40:18.802 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-18T08:40:18.802 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-18T08:40:18.803 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-18T08:40:18.803 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-18T08:40:18.803 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-18T08:40:18.803 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-18T08:40:18.803 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-18T08:40:18.803 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-18T08:40:18.804 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-18T08:40:18.804 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-18T08:40:18.804 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-18T08:40:18.804 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-18T08:40:18.804 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-18T08:40:18.805 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-18T08:40:18.805 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-18T08:40:18.805 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-18T08:40:18.805 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-18T08:40:18.805 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-18T08:40:18.805 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-18T08:40:18.806 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink 2023-10-18T08:40:18.806 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-18T08:40:18.806 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh 2023-10-18T08:40:18.806 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-18T08:40:18.806 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch 2023-10-18T08:40:18.807 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-18T08:40:18.807 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-18T08:40:18.807 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-18T08:40:18.807 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-18T08:40:18.807 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet 2023-10-18T08:40:18.807 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-18T08:40:18.808 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds 2023-10-18T08:40:18.808 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-18T08:40:18.808 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-18T08:40:18.808 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-18T08:40:18.808 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched 2023-10-18T08:40:18.809 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-18T08:40:18.809 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-18T08:40:18.809 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-18T08:40:18.809 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-18T08:40:18.809 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-18T08:40:18.809 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-18T08:40:18.810 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-18T08:40:18.810 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-18T08:40:18.810 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-18T08:40:18.810 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-18T08:40:18.810 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-18T08:40:18.811 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-18T08:40:18.811 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-18T08:40:18.811 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-18T08:40:18.811 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-18T08:40:18.811 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-18T08:40:18.812 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-18T08:40:18.812 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-18T08:40:18.812 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-18T08:40:18.812 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-18T08:40:18.812 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-18T08:40:18.812 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-18T08:40:18.813 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-18T08:40:18.813 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-18T08:40:18.813 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-18T08:40:18.813 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-18T08:40:18.813 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-18T08:40:18.813 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-18T08:40:18.814 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-18T08:40:18.814 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-18T08:40:18.814 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-18T08:40:18.814 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-18T08:40:18.814 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-18T08:40:18.815 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-18T08:40:18.815 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-18T08:40:18.815 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-18T08:40:18.815 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-18T08:40:18.815 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-18T08:40:18.816 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-18T08:40:18.816 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-18T08:40:18.816 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-18T08:40:18.816 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-18T08:40:18.817 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-18T08:40:18.817 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-18T08:40:18.817 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp 2023-10-18T08:40:18.817 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-18T08:40:18.818 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-18T08:40:18.818 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc 2023-10-18T08:40:18.818 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss 2023-10-18T08:40:18.818 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-18T08:40:18.818 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-18T08:40:18.819 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-18T08:40:18.819 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma 2023-10-18T08:40:18.819 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-18T08:40:18.819 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc 2023-10-18T08:40:18.819 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-18T08:40:18.819 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-18T08:40:18.820 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix 2023-10-18T08:40:18.820 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-18T08:40:18.820 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock 2023-10-18T08:40:18.820 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-18T08:40:18.820 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-18T08:40:18.821 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-18T08:40:18.821 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-18T08:40:18.821 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-18T08:40:18.821 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm 2023-10-18T08:40:18.821 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-18T08:40:18.822 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security 2023-10-18T08:40:18.822 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys 2023-10-18T08:40:18.822 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys 2023-10-18T08:40:18.822 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-18T08:40:18.822 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys 2023-10-18T08:40:18.823 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-18T08:40:18.823 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt 2023-10-18T08:40:18.823 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib 2023-10-18T08:40:18.823 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-18T08:40:18.823 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-18T08:40:18.823 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-18T08:40:18.824 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-18T08:40:18.824 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.alias.bin 2023-10-18T08:40:18.824 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-18T08:40:18.824 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-18T08:40:18.824 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-18T08:40:18.824 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-18T08:40:18.825 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-18T08:40:18.825 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-18T08:40:18.825 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-18T08:40:18.825 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-18T08:40:18.825 INFO:teuthology.orchestra.run.smithi040.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-18T08:40:18.826 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-18T08:40:18.826 DEBUG:teuthology.orchestra.run.smithi040:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-18T08:40:18.850 INFO:teuthology.orchestra.run.smithi040.stdout:postinstall scriptlet (using /bin/sh): 2023-10-18T08:40:18.851 INFO:teuthology.orchestra.run.smithi040.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g47672119149b -a -r /boot/System.map-6.6.0-rc5-g47672119149b ]; then 2023-10-18T08:40:18.851 INFO:teuthology.orchestra.run.smithi040.stdout:cp /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm 2023-10-18T08:40:18.851 INFO:teuthology.orchestra.run.smithi040.stdout:cp /boot/System.map-6.6.0-rc5-g47672119149b /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-18T08:40:18.851 INFO:teuthology.orchestra.run.smithi040.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/System.map-6.6.0-rc5-g47672119149b 2023-10-18T08:40:18.852 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/installkernel 6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-18T08:40:18.852 INFO:teuthology.orchestra.run.smithi040.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-18T08:40:18.852 INFO:teuthology.orchestra.run.smithi040.stdout:fi 2023-10-18T08:40:18.852 INFO:teuthology.orchestra.run.smithi040.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-18T08:40:18.852 INFO:teuthology.orchestra.run.smithi040.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-18T08:40:18.853 INFO:teuthology.orchestra.run.smithi040.stdout:new-kernel-pkg --remove 6.6.0-rc5-g47672119149b --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g47672119149b.img 2023-10-18T08:40:18.853 INFO:teuthology.orchestra.run.smithi040.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-18T08:40:18.853 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-install remove 6.6.0-rc5-g47672119149b 2023-10-18T08:40:18.853 INFO:teuthology.orchestra.run.smithi040.stdout:fi 2023-10-18T08:40:18.853 INFO:teuthology.orchestra.run.smithi040.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-18T08:40:18.853 INFO:teuthology.orchestra.run.smithi040.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-18T08:40:18.854 INFO:teuthology.orchestra.run.smithi040.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g47672119149b 2023-10-18T08:40:18.854 INFO:teuthology.orchestra.run.smithi040.stdout:fi 2023-10-18T08:40:18.855 DEBUG:teuthology.orchestra.run.smithi040:> sudo rpm -qi grub2-tools 2023-10-18T08:40:18.936 INFO:teuthology.orchestra.run.smithi040.stdout:Name : grub2-tools 2023-10-18T08:40:18.937 INFO:teuthology.orchestra.run.smithi040.stdout:Epoch : 1 2023-10-18T08:40:18.937 INFO:teuthology.orchestra.run.smithi040.stdout:Version : 2.06 2023-10-18T08:40:18.937 INFO:teuthology.orchestra.run.smithi040.stdout:Release : 68.el9 2023-10-18T08:40:18.937 INFO:teuthology.orchestra.run.smithi040.stdout:Architecture: x86_64 2023-10-18T08:40:18.937 INFO:teuthology.orchestra.run.smithi040.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-18T08:40:18.938 INFO:teuthology.orchestra.run.smithi040.stdout:Group : Unspecified 2023-10-18T08:40:18.938 INFO:teuthology.orchestra.run.smithi040.stdout:Size : 8264254 2023-10-18T08:40:18.938 INFO:teuthology.orchestra.run.smithi040.stdout:License : GPLv3+ 2023-10-18T08:40:18.938 INFO:teuthology.orchestra.run.smithi040.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-18T08:40:18.938 INFO:teuthology.orchestra.run.smithi040.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-18T08:40:18.939 INFO:teuthology.orchestra.run.smithi040.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-18T08:40:18.939 INFO:teuthology.orchestra.run.smithi040.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-18T08:40:18.939 INFO:teuthology.orchestra.run.smithi040.stdout:Packager : builder@centos.org 2023-10-18T08:40:18.939 INFO:teuthology.orchestra.run.smithi040.stdout:Vendor : CentOS 2023-10-18T08:40:18.939 INFO:teuthology.orchestra.run.smithi040.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-18T08:40:18.940 INFO:teuthology.orchestra.run.smithi040.stdout:Summary : Support tools for GRUB. 2023-10-18T08:40:18.940 INFO:teuthology.orchestra.run.smithi040.stdout:Description : 2023-10-18T08:40:18.940 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:40:18.940 INFO:teuthology.orchestra.run.smithi040.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-18T08:40:18.940 INFO:teuthology.orchestra.run.smithi040.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-18T08:40:18.940 INFO:teuthology.orchestra.run.smithi040.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-18T08:40:18.941 INFO:teuthology.orchestra.run.smithi040.stdout:hardware devices. 2023-10-18T08:40:18.941 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:40:18.941 INFO:teuthology.orchestra.run.smithi040.stdout:This subpackage provides tools for support of all platforms. 2023-10-18T08:40:18.942 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-18T08:40:18.942 INFO:teuthology.task.kernel:Updating grub on smithi040 to boot 6.6.0-rc5-g47672119149b 2023-10-18T08:40:18.942 DEBUG:teuthology.orchestra.run.smithi040:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-18T08:40:19.565 INFO:teuthology.orchestra.run.smithi040.stderr:Generating grub configuration file ... 2023-10-18T08:40:20.994 INFO:teuthology.orchestra.run.smithi040.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-18T08:40:21.022 INFO:teuthology.orchestra.run.smithi040.stderr:done 2023-10-18T08:40:21.025 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2023-10-18T08:40:21.040 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.2NOJtL9kdA 2023-10-18T08:40:21.040 DEBUG:teuthology.orchestra.run.smithi040:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2NOJtL9kdA 2023-10-18T08:40:21.112 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.2NOJtL9kdA 2023-10-18T08:40:21.256 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.2NOJtL9kdA is 6KB 2023-10-18T08:40:21.276 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.2NOJtL9kdA 2023-10-18T08:40:21.290 DEBUG:teuthology.orchestra.run.smithi040:> sudo /bin/ls /boot/loader/entries || true 2023-10-18T08:40:21.359 INFO:teuthology.orchestra.run.smithi040.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-18T08:40:21.360 INFO:teuthology.orchestra.run.smithi040.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-18T08:40:21.360 INFO:teuthology.orchestra.run.smithi040.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-18T08:40:21.360 INFO:teuthology.orchestra.run.smithi040.stdout:15949e3c88704300882482eddfd7baae-6.6.0-rc5-g47672119149b.conf 2023-10-18T08:40:21.361 DEBUG:teuthology.orchestra.run.smithi040:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-rc5-g47672119149b 2023-10-18T08:40:21.514 DEBUG:teuthology.orchestra.run.smithi040:> sudo shutdown -r now 2023-10-18T08:40:47.602 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-18T08:40:47.603 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-10-18T08:40:47.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T08:40:51.522 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-18T08:40:51.523 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-10-18T08:40:51.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T08:41:47.663 DEBUG:teuthology.orchestra.remote:timed out 2023-10-18T08:41:51.529 DEBUG:teuthology.orchestra.remote:timed out 2023-10-18T08:41:56.670 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-10-18T08:41:56.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T08:41:58.607 DEBUG:teuthology.orchestra.run.smithi144:> true 2023-10-18T08:41:59.232 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-10-18T08:41:59.232 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-rc5-g47672119149b"... 2023-10-18T08:41:59.233 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2023-10-18T08:41:59.273 INFO:teuthology.orchestra.run.smithi144.stdout:6.6.0-rc5-g47672119149b 2023-10-18T08:41:59.273 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-18T08:41:59.273 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-18T08:41:59.273 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-18T08:42:00.275 DEBUG:teuthology.parallel:result is None 2023-10-18T08:42:00.531 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-10-18T08:42:00.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T08:42:08.056 DEBUG:teuthology.orchestra.run.smithi040:> true 2023-10-18T08:42:08.648 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-10-18T08:42:08.649 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g47672119149b"... 2023-10-18T08:42:08.649 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2023-10-18T08:42:08.691 INFO:teuthology.orchestra.run.smithi040.stdout:6.6.0-rc5-g47672119149b 2023-10-18T08:42:08.691 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-18T08:42:08.691 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-18T08:42:08.692 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-18T08:42:09.693 DEBUG:teuthology.parallel:result is None 2023-10-18T08:42:09.694 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-18T08:42:09.701 INFO:teuthology.task.internal:Creating test directory... 2023-10-18T08:42:09.701 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-18T08:42:09.708 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-18T08:42:09.754 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-18T08:42:09.761 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-18T08:42:09.786 INFO:teuthology.task.internal:Creating archive directory... 2023-10-18T08:42:09.786 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-18T08:42:09.791 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-18T08:42:09.844 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-18T08:42:09.853 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-18T08:42:09.853 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-18T08:42:09.859 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-18T08:42:09.980 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T08:42:09.997 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T08:42:10.048 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T08:42:10.049 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T08:42:10.053 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-18T08:42:10.060 INFO:teuthology.task.internal:Configuring sudo... 2023-10-18T08:42:10.061 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-18T08:42:10.065 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-18T08:42:10.140 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-18T08:42:10.150 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-18T08:42:10.151 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-18T08:42:10.156 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-18T08:42:10.193 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T08:42:10.313 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T08:42:10.407 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T08:42:10.490 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T08:42:10.579 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:42:10.579 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-18T08:42:10.669 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T08:42:10.809 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T08:42:10.919 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T08:42:10.987 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T08:42:11.074 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:42:11.074 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-18T08:42:11.143 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2023-10-18T08:42:11.154 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2023-10-18T08:42:11.258 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T08:42:11.284 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T08:42:11.553 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-18T08:42:11.560 INFO:teuthology.task.internal:Starting timer... 2023-10-18T08:42:11.561 INFO:teuthology.run_tasks:Running task pcp... 2023-10-18T08:42:11.569 INFO:teuthology.run_tasks:Running task selinux... 2023-10-18T08:42:11.578 DEBUG:teuthology.orchestra.run.smithi040:> sudo service auditd rotate 2023-10-18T08:42:11.695 INFO:teuthology.orchestra.run.smithi040.stdout:Rotating logs: 2023-10-18T08:42:11.699 DEBUG:teuthology.orchestra.run.smithi144:> sudo service auditd rotate 2023-10-18T08:42:11.809 INFO:teuthology.orchestra.run.smithi144.stdout:Rotating logs: 2023-10-18T08:42:11.813 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-18T08:42:11.813 DEBUG:teuthology.orchestra.run.smithi040:> /usr/sbin/getenforce 2023-10-18T08:42:11.868 INFO:teuthology.orchestra.run.smithi040.stdout:Permissive 2023-10-18T08:42:11.869 DEBUG:teuthology.orchestra.run.smithi144:> /usr/sbin/getenforce 2023-10-18T08:42:11.926 INFO:teuthology.orchestra.run.smithi144.stdout:Permissive 2023-10-18T08:42:11.926 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi040.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi144.front.sepia.ceph.com': 'permissive'} 2023-10-18T08:42:11.927 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T08:42:12.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T08:42:12.019 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T08:42:12.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T08:42:12.103 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-18T08:42:12.104 DEBUG:teuthology.orchestra.run.smithi040:> sudo /usr/sbin/setenforce permissive 2023-10-18T08:42:12.202 DEBUG:teuthology.orchestra.run.smithi144:> sudo /usr/sbin/setenforce permissive 2023-10-18T08:42:12.288 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-18T08:42:12.297 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-18T08:42:12.337 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-18T08:42:12.499 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-18T08:42:12.584 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-18T08:42:12.586 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com,smithi144.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-18T08:52:30.323 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com')] 2023-10-18T08:52:30.325 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-10-18T08:52:30.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T08:52:30.476 DEBUG:teuthology.orchestra.run.smithi040:> true 2023-10-18T08:52:30.627 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-10-18T08:52:30.627 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-10-18T08:52:30.628 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T08:52:30.762 DEBUG:teuthology.orchestra.run.smithi144:> true 2023-10-18T08:52:30.915 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-10-18T08:52:30.916 INFO:teuthology.run_tasks:Running task clock... 2023-10-18T08:52:30.929 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-18T08:52:30.929 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-18T08:52:30.930 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T08:52:30.942 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-18T08:52:30.943 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T08:52:31.047 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-18T08:52:31.049 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-18T08:52:31.101 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-18T08:52:31.102 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-18T08:52:31.191 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: ntpd: command not found 2023-10-18T08:52:31.200 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: ntpd: command not found 2023-10-18T08:52:31.232 INFO:teuthology.orchestra.run.smithi144.stdout:506 Cannot talk to daemon 2023-10-18T08:52:31.242 INFO:teuthology.orchestra.run.smithi040.stdout:506 Cannot talk to daemon 2023-10-18T08:52:31.282 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-18T08:52:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-18T08:52:31.340 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-18T08:52:31.350 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-18T08:52:31.461 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2023-10-18T08:52:31.471 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T08:52:31.471 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2023-10-18T08:52:31.472 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T08:52:31.472 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T08:52:31.472 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T08:52:31.472 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T08:52:31.479 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2023-10-18T08:52:31.488 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T08:52:31.488 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2023-10-18T08:52:31.489 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T08:52:31.489 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T08:52:31.489 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T08:52:31.489 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T08:52:31.490 INFO:teuthology.run_tasks:Running task install... 2023-10-18T08:52:31.500 DEBUG:teuthology.task.install:project ceph 2023-10-18T08:52:31.500 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'}} 2023-10-18T08:52:31.500 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'} 2023-10-18T08:52:31.501 INFO:teuthology.task.install:Using flavor: default 2023-10-18T08:52:31.506 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-18T08:52:31.507 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-18T08:52:31.507 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'tag': None, 'wait_for_package': False} 2023-10-18T08:52:31.508 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T08:52:31.510 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'tag': None, 'wait_for_package': False} 2023-10-18T08:52:31.511 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T08:52:31.697 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/9/flavors/default/ 2023-10-18T08:52:31.698 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6786.gf7420d7f 2023-10-18T08:52:31.704 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/9/flavors/default/ 2023-10-18T08:52:31.704 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6786.gf7420d7f 2023-10-18T08:52:31.853 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-18T08:52:31.853 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:52:31.854 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-18T08:52:31.967 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-18T08:52:31.967 DEBUG:teuthology.orchestra.run.smithi040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f7420d7f8e55ceea786dcbf536c338ce920c21eb/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-18T08:52:32.115 DEBUG:teuthology.orchestra.run.smithi040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-18T08:52:32.171 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-18T08:52:32.171 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:52:32.171 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-18T08:52:32.259 DEBUG:teuthology.orchestra.run.smithi040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-18T08:52:32.273 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-18T08:52:32.273 DEBUG:teuthology.orchestra.run.smithi144:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f7420d7f8e55ceea786dcbf536c338ce920c21eb/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-18T08:52:32.361 INFO:teuthology.orchestra.run.smithi040.stdout:check_obsoletes = 1 2023-10-18T08:52:32.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum clean all 2023-10-18T08:52:32.414 DEBUG:teuthology.orchestra.run.smithi144:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-18T08:52:32.544 DEBUG:teuthology.orchestra.run.smithi144:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-18T08:52:32.641 INFO:teuthology.orchestra.run.smithi144.stdout:check_obsoletes = 1 2023-10-18T08:52:32.645 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum clean all 2023-10-18T08:52:32.732 INFO:teuthology.orchestra.run.smithi040.stdout:72 files removed 2023-10-18T08:52:32.772 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-18T08:52:32.984 INFO:teuthology.orchestra.run.smithi144.stdout:72 files removed 2023-10-18T08:52:33.022 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-18T08:52:34.233 INFO:teuthology.orchestra.run.smithi040.stdout:ceph packages for x86_64 84 kB/s | 78 kB 00:00 2023-10-18T08:52:34.255 INFO:teuthology.orchestra.run.smithi144.stdout:ceph packages for x86_64 108 kB/s | 78 kB 00:00 2023-10-18T08:52:34.983 INFO:teuthology.orchestra.run.smithi040.stdout:ceph noarch packages 14 kB/s | 9.4 kB 00:00 2023-10-18T08:52:35.030 INFO:teuthology.orchestra.run.smithi144.stdout:ceph noarch packages 14 kB/s | 9.4 kB 00:00 2023-10-18T08:52:35.776 INFO:teuthology.orchestra.run.smithi040.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2023-10-18T08:52:35.814 INFO:teuthology.orchestra.run.smithi144.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2023-10-18T08:52:36.496 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-9 - AppStream 9.8 MB/s | 6.3 MB 00:00 2023-10-18T08:52:36.618 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - AppStream 8.6 MB/s | 6.3 MB 00:00 2023-10-18T08:52:38.080 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS-9 - BaseOS 5.4 MB/s | 2.0 MB 00:00 2023-10-18T08:52:38.335 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS-9 - BaseOS 4.1 MB/s | 2.0 MB 00:00 2023-10-18T08:52:39.309 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 9.5 MB/s | 7.9 MB 00:00 2023-10-18T08:52:39.431 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 7.9 MB 00:00 2023-10-18T08:52:42.736 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 8.3 MB/s | 18 MB 00:02 2023-10-18T08:52:48.071 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 5.7 MB 00:01 2023-10-18T08:52:49.622 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 14 kB 00:00 2023-10-18T08:52:50.871 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 18 MB/s | 19 MB 00:01 2023-10-18T08:52:50.917 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 1.7 MB/s | 18 MB 00:10 2023-10-18T08:52:55.651 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 5.7 MB 00:00 2023-10-18T08:52:56.198 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 8.8 kB/s | 1.7 kB 00:00 2023-10-18T08:52:57.752 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 14 kB 00:00 2023-10-18T08:52:58.700 INFO:teuthology.orchestra.run.smithi144.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-18T08:52:58.702 INFO:teuthology.orchestra.run.smithi144.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-18T08:52:58.721 INFO:teuthology.orchestra.run.smithi144.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-10-18T08:52:58.722 INFO:teuthology.orchestra.run.smithi144.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-10-18T08:52:58.723 INFO:teuthology.orchestra.run.smithi144.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-10-18T08:52:58.724 INFO:teuthology.orchestra.run.smithi144.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-10-18T08:52:58.727 INFO:teuthology.orchestra.run.smithi144.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-10-18T08:52:58.730 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-10-18T08:52:58.731 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-10-18T08:52:58.809 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2023-10-18T08:52:58.834 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2023-10-18T08:52:58.834 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2023-10-18T08:52:58.834 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2023-10-18T08:52:58.835 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2023-10-18T08:52:58.835 INFO:teuthology.orchestra.run.smithi144.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-10-18T08:52:58.835 INFO:teuthology.orchestra.run.smithi144.stdout: ceph x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 6.5 k 2023-10-18T08:52:58.835 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 5.2 M 2023-10-18T08:52:58.835 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 857 k 2023-10-18T08:52:58.836 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 142 k 2023-10-18T08:52:58.836 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 1.5 M 2023-10-18T08:52:58.836 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 132 k 2023-10-18T08:52:58.836 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 1.8 M 2023-10-18T08:52:58.836 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 7.4 M 2023-10-18T08:52:58.837 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 47 k 2023-10-18T08:52:58.837 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 8.6 M 2023-10-18T08:52:58.837 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 41 M 2023-10-18T08:52:58.837 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 262 k 2023-10-18T08:52:58.837 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 270 k 2023-10-18T08:52:58.837 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 218 k 2023-10-18T08:52:58.838 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 18 k 2023-10-18T08:52:58.838 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-10-18T08:52:58.838 INFO:teuthology.orchestra.run.smithi144.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-10-18T08:52:58.838 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-10-18T08:52:58.838 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 31 k 2023-10-18T08:52:58.839 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 716 k 2023-10-18T08:52:58.839 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 127 k 2023-10-18T08:52:58.839 INFO:teuthology.orchestra.run.smithi144.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-10-18T08:52:58.839 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-10-18T08:52:58.839 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-10-18T08:52:58.840 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-10-18T08:52:58.840 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-10-18T08:52:58.840 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-10-18T08:52:58.840 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 162 k 2023-10-18T08:52:58.840 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 321 k 2023-10-18T08:52:58.841 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 297 k 2023-10-18T08:52:58.841 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 99 k 2023-10-18T08:52:58.841 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 86 k 2023-10-18T08:52:58.841 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 3.0 M 2023-10-18T08:52:58.841 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 169 k 2023-10-18T08:52:58.841 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-10-18T08:52:58.842 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2023-10-18T08:52:58.842 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager x86_64 1:1.45.4-1.el9 CentOS-BaseOS 2.2 M 2023-10-18T08:52:58.842 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm x86_64 1:1.45.4-1.el9 CentOS-BaseOS 1.8 M 2023-10-18T08:52:58.842 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team x86_64 1:1.45.4-1.el9 CentOS-BaseOS 41 k 2023-10-18T08:52:58.842 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui x86_64 1:1.45.4-1.el9 CentOS-BaseOS 244 k 2023-10-18T08:52:58.843 INFO:teuthology.orchestra.run.smithi144.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-10-18T08:52:58.843 INFO:teuthology.orchestra.run.smithi144.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-10-18T08:52:58.843 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-10-18T08:52:58.843 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 3.3 M 2023-10-18T08:52:58.843 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 3.0 M 2023-10-18T08:52:58.844 INFO:teuthology.orchestra.run.smithi144.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-10-18T08:52:58.844 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2023-10-18T08:52:58.844 INFO:teuthology.orchestra.run.smithi144.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-10-18T08:52:58.844 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-18T08:52:58.844 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 19 M 2023-10-18T08:52:58.845 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 23 k 2023-10-18T08:52:58.845 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 2.1 M 2023-10-18T08:52:58.845 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 242 k 2023-10-18T08:52:58.845 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 4.4 M 2023-10-18T08:52:58.845 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 18 M 2023-10-18T08:52:58.845 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 15 k 2023-10-18T08:52:58.846 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 24 k 2023-10-18T08:52:58.846 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-18T08:52:58.846 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-18T08:52:58.846 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-18T08:52:58.846 INFO:teuthology.orchestra.run.smithi144.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-18T08:52:58.847 INFO:teuthology.orchestra.run.smithi144.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-18T08:52:58.847 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-18T08:52:58.847 INFO:teuthology.orchestra.run.smithi144.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-18T08:52:58.847 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-18T08:52:58.847 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-18T08:52:58.848 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 164 k 2023-10-18T08:52:58.848 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-18T08:52:58.848 INFO:teuthology.orchestra.run.smithi144.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-18T08:52:58.848 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-18T08:52:58.848 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-18T08:52:58.848 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-18T08:52:58.849 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 472 k 2023-10-18T08:52:58.849 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-18T08:52:58.849 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 4.8 M 2023-10-18T08:52:58.849 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-18T08:52:58.849 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-10-18T08:52:58.850 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-18T08:52:58.850 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-18T08:52:58.850 INFO:teuthology.orchestra.run.smithi144.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-18T08:52:58.850 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-18T08:52:58.850 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-18T08:52:58.851 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-10-18T08:52:58.851 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-18T08:52:58.851 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-18T08:52:58.851 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-18T08:52:58.851 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-18T08:52:58.851 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 45 k 2023-10-18T08:52:58.852 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 127 k 2023-10-18T08:52:58.852 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-18T08:52:58.852 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-18T08:52:58.852 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-18T08:52:58.852 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-18T08:52:58.853 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-18T08:52:58.853 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-18T08:52:58.853 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth noarch 1:2.23.2-1.el9 epel 196 k 2023-10-18T08:52:58.853 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-18T08:52:58.853 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-18T08:52:58.853 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-18T08:52:58.854 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-18T08:52:58.854 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-18T08:52:58.854 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-18T08:52:58.854 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-18T08:52:58.854 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-18T08:52:58.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-18T08:52:58.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-18T08:52:58.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-18T08:52:58.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-18T08:52:58.855 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-18T08:52:58.856 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-18T08:52:58.856 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-18T08:52:58.856 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-18T08:52:58.856 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-18T08:52:58.857 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-18T08:52:58.857 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-18T08:52:58.857 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-18T08:52:58.857 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-18T08:52:58.857 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-18T08:52:58.858 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-18T08:52:58.858 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-18T08:52:58.858 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-18T08:52:58.858 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-18T08:52:58.858 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-18T08:52:58.859 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-18T08:52:58.859 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-18T08:52:58.859 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-18T08:52:58.859 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-18T08:52:58.860 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-18T08:52:58.860 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-18T08:52:58.860 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-18T08:52:58.860 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-18T08:52:58.860 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-18T08:52:58.860 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-10-18T08:52:58.861 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-18T08:52:58.861 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-18T08:52:58.861 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-18T08:52:58.861 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-18T08:52:58.861 INFO:teuthology.orchestra.run.smithi144.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-18T08:52:58.862 INFO:teuthology.orchestra.run.smithi144.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-18T08:52:58.862 INFO:teuthology.orchestra.run.smithi144.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-18T08:52:58.862 INFO:teuthology.orchestra.run.smithi144.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-10-18T08:52:58.862 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-18T08:52:58.862 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-10-18T08:52:58.863 INFO:teuthology.orchestra.run.smithi144.stdout:Installing weak dependencies: 2023-10-18T08:52:58.863 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-18T08:52:58.863 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:52:58.863 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2023-10-18T08:52:58.863 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================================ 2023-10-18T08:52:58.864 INFO:teuthology.orchestra.run.smithi144.stdout:Install 133 Packages 2023-10-18T08:52:58.864 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 10 Packages 2023-10-18T08:52:58.864 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:52:58.865 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 200 M 2023-10-18T08:52:58.865 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2023-10-18T08:52:59.600 INFO:teuthology.orchestra.run.smithi144.stdout:(1/143): ceph-18.0.0-6786.gf7420d7f.el9.x86_64. 23 kB/s | 6.5 kB 00:00 2023-10-18T08:52:59.800 INFO:teuthology.orchestra.run.smithi144.stdout:(2/143): ceph-fuse-18.0.0-6786.gf7420d7f.el9.x8 4.2 MB/s | 857 kB 00:00 2023-10-18T08:52:59.850 INFO:teuthology.orchestra.run.smithi144.stdout:(3/143): ceph-immutable-object-cache-18.0.0-678 2.8 MB/s | 142 kB 00:00 2023-10-18T08:53:00.126 INFO:teuthology.orchestra.run.smithi144.stdout:(4/143): ceph-mds-18.0.0-6786.gf7420d7f.el9.x86 7.6 MB/s | 2.1 MB 00:00 2023-10-18T08:53:00.334 INFO:teuthology.orchestra.run.smithi144.stdout:(5/143): ceph-mgr-18.0.0-6786.gf7420d7f.el9.x86 7.0 MB/s | 1.5 MB 00:00 2023-10-18T08:53:00.501 INFO:teuthology.orchestra.run.smithi144.stdout:(6/143): ceph-base-18.0.0-6786.gf7420d7f.el9.x8 4.4 MB/s | 5.2 MB 00:01 2023-10-18T08:53:00.910 INFO:teuthology.orchestra.run.smithi144.stdout:(7/143): ceph-mon-18.0.0-6786.gf7420d7f.el9.x86 7.6 MB/s | 4.4 MB 00:00 2023-10-18T08:53:01.327 INFO:teuthology.orchestra.run.smithi144.stdout:(8/143): ceph-common-18.0.0-6786.gf7420d7f.el9. 9.5 MB/s | 19 MB 00:02 2023-10-18T08:53:01.402 INFO:teuthology.orchestra.run.smithi144.stdout:(9/143): ceph-selinux-18.0.0-6786.gf7420d7f.el9 326 kB/s | 24 kB 00:00 2023-10-18T08:53:01.877 INFO:teuthology.orchestra.run.smithi144.stdout:(10/143): ceph-radosgw-18.0.0-6786.gf7420d7f.el 8.9 MB/s | 8.6 MB 00:00 2023-10-18T08:53:01.952 INFO:teuthology.orchestra.run.smithi144.stdout:(11/143): cephfs-mirror-18.0.0-6786.gf7420d7f.e 2.9 MB/s | 218 kB 00:00 2023-10-18T08:53:02.036 INFO:teuthology.orchestra.run.smithi144.stdout:(12/143): libcephfs-devel-18.0.0-6786.gf7420d7f 377 kB/s | 31 kB 00:00 2023-10-18T08:53:02.170 INFO:teuthology.orchestra.run.smithi144.stdout:(13/143): libcephfs2-18.0.0-6786.gf7420d7f.el9. 5.3 MB/s | 716 kB 00:00 2023-10-18T08:53:02.236 INFO:teuthology.orchestra.run.smithi144.stdout:(14/143): libcephsqlite-18.0.0-6786.gf7420d7f.e 2.4 MB/s | 164 kB 00:00 2023-10-18T08:53:02.303 INFO:teuthology.orchestra.run.smithi144.stdout:(15/143): librados-devel-18.0.0-6786.gf7420d7f. 1.9 MB/s | 127 kB 00:00 2023-10-18T08:53:02.395 INFO:teuthology.orchestra.run.smithi144.stdout:(16/143): libradosstriper1-18.0.0-6786.gf7420d7 5.1 MB/s | 472 kB 00:00 2023-10-18T08:53:02.754 INFO:teuthology.orchestra.run.smithi144.stdout:(17/143): ceph-osd-18.0.0-6786.gf7420d7f.el9.x8 7.9 MB/s | 18 MB 00:02 2023-10-18T08:53:02.863 INFO:teuthology.orchestra.run.smithi144.stdout:(18/143): python3-ceph-argparse-18.0.0-6786.gf7 420 kB/s | 45 kB 00:00 2023-10-18T08:53:02.971 INFO:teuthology.orchestra.run.smithi144.stdout:(19/143): librgw2-18.0.0-6786.gf7420d7f.el9.x86 8.3 MB/s | 4.8 MB 00:00 2023-10-18T08:53:03.005 INFO:teuthology.orchestra.run.smithi144.stdout:(20/143): python3-ceph-common-18.0.0-6786.gf742 897 kB/s | 127 kB 00:00 2023-10-18T08:53:03.038 INFO:teuthology.orchestra.run.smithi144.stdout:(21/143): python3-cephfs-18.0.0-6786.gf7420d7f. 2.4 MB/s | 162 kB 00:00 2023-10-18T08:53:03.105 INFO:teuthology.orchestra.run.smithi144.stdout:(22/143): python3-rbd-18.0.0-6786.gf7420d7f.el9 4.4 MB/s | 297 kB 00:00 2023-10-18T08:53:03.139 INFO:teuthology.orchestra.run.smithi144.stdout:(23/143): python3-rados-18.0.0-6786.gf7420d7f.e 2.4 MB/s | 321 kB 00:00 2023-10-18T08:53:03.173 INFO:teuthology.orchestra.run.smithi144.stdout:(24/143): python3-rgw-18.0.0-6786.gf7420d7f.el9 1.5 MB/s | 99 kB 00:00 2023-10-18T08:53:03.224 INFO:teuthology.orchestra.run.smithi144.stdout:(25/143): rbd-fuse-18.0.0-6786.gf7420d7f.el9.x8 1.0 MB/s | 86 kB 00:00 2023-10-18T08:53:03.308 INFO:teuthology.orchestra.run.smithi144.stdout:(26/143): rbd-nbd-18.0.0-6786.gf7420d7f.el9.x86 2.0 MB/s | 169 kB 00:00 2023-10-18T08:53:03.408 INFO:teuthology.orchestra.run.smithi144.stdout:(27/143): ceph-grafana-dashboards-18.0.0-6786.g 233 kB/s | 23 kB 00:00 2023-10-18T08:53:03.519 INFO:teuthology.orchestra.run.smithi144.stdout:(28/143): rbd-mirror-18.0.0-6786.gf7420d7f.el9. 8.7 MB/s | 3.0 MB 00:00 2023-10-18T08:53:03.569 INFO:teuthology.orchestra.run.smithi144.stdout:(29/143): ceph-mgr-cephadm-18.0.0-6786.gf7420d7 827 kB/s | 132 kB 00:00 2023-10-18T08:53:03.736 INFO:teuthology.orchestra.run.smithi144.stdout:(30/143): ceph-mgr-dashboard-18.0.0-6786.gf7420 8.2 MB/s | 1.8 MB 00:00 2023-10-18T08:53:03.845 INFO:teuthology.orchestra.run.smithi144.stdout:(31/143): ceph-mgr-modules-core-18.0.0-6786.gf7 2.2 MB/s | 242 kB 00:00 2023-10-18T08:53:03.937 INFO:teuthology.orchestra.run.smithi144.stdout:(32/143): ceph-mgr-rook-18.0.0-6786.gf7420d7f.e 524 kB/s | 47 kB 00:00 2023-10-18T08:53:04.029 INFO:teuthology.orchestra.run.smithi144.stdout:(33/143): ceph-prometheus-alerts-18.0.0-6786.gf 166 kB/s | 15 kB 00:00 2023-10-18T08:53:04.112 INFO:teuthology.orchestra.run.smithi144.stdout:(34/143): ceph-volume-18.0.0-6786.gf7420d7f.el9 3.1 MB/s | 262 kB 00:00 2023-10-18T08:53:04.159 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 3.1 MB/s | 19 MB 00:06 2023-10-18T08:53:04.196 INFO:teuthology.orchestra.run.smithi144.stdout:(35/143): cephadm-18.0.0-6786.gf7420d7f.el9.noa 3.2 MB/s | 270 kB 00:00 2023-10-18T08:53:04.646 INFO:teuthology.orchestra.run.smithi144.stdout:(36/143): ceph-test-18.0.0-6786.gf7420d7f.el9.x 13 MB/s | 41 MB 00:03 2023-10-18T08:53:04.688 INFO:teuthology.orchestra.run.smithi144.stdout:(37/143): cephfs-top-18.0.0-6786.gf7420d7f.el9. 37 kB/s | 18 kB 00:00 2023-10-18T08:53:04.930 INFO:teuthology.orchestra.run.smithi144.stdout:(38/143): ceph-mgr-diskprediction-local-18.0.0- 5.4 MB/s | 7.4 MB 00:01 2023-10-18T08:53:04.980 INFO:teuthology.orchestra.run.smithi144.stdout:(39/143): boost-program-options-1.75.0-8.el9.x8 364 kB/s | 106 kB 00:00 2023-10-18T08:53:05.023 INFO:teuthology.orchestra.run.smithi144.stdout:(40/143): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 920 kB 00:00 2023-10-18T08:53:05.059 INFO:teuthology.orchestra.run.smithi144.stdout:(41/143): flex-2.6.4-9.el9.x86_64.rpm 2.7 MB/s | 307 kB 00:00 2023-10-18T08:53:05.089 INFO:teuthology.orchestra.run.smithi144.stdout:(42/143): flexiblas-3.0.4-8.el9.x86_64.rpm 303 kB/s | 32 kB 00:00 2023-10-18T08:53:05.131 INFO:teuthology.orchestra.run.smithi144.stdout:(43/143): flexiblas-openblas-openmp-3.0.4-8.el9 209 kB/s | 17 kB 00:00 2023-10-18T08:53:05.257 INFO:teuthology.orchestra.run.smithi144.stdout:(44/143): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2023-10-18T08:53:05.299 INFO:teuthology.orchestra.run.smithi144.stdout:(45/143): libaio-devel-0.3.111-13.el9.x86_64.rp 48 kB/s | 10 kB 00:00 2023-10-18T08:53:05.332 INFO:teuthology.orchestra.run.smithi144.stdout:(46/143): libpmemobj-1.12.1-1.el9.x86_64.rpm 850 kB/s | 159 kB 00:00 2023-10-18T08:53:05.382 INFO:teuthology.orchestra.run.smithi144.stdout:(47/143): libtool-2.4.6-45.el9.x86_64.rpm 7.0 MB/s | 578 kB 00:00 2023-10-18T08:53:05.424 INFO:teuthology.orchestra.run.smithi144.stdout:(48/143): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 661 kB 00:00 2023-10-18T08:53:05.483 INFO:teuthology.orchestra.run.smithi144.stdout:(49/143): libzstd-devel-1.5.1-2.el9.x86_64.rpm 315 kB/s | 46 kB 00:00 2023-10-18T08:53:05.517 INFO:teuthology.orchestra.run.smithi144.stdout:(50/143): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 282 kB 00:00 2023-10-18T08:53:05.567 INFO:teuthology.orchestra.run.smithi144.stdout:(51/143): openblas-0.3.21-2.el9.x86_64.rpm 427 kB/s | 34 kB 00:00 2023-10-18T08:53:05.609 INFO:teuthology.orchestra.run.smithi144.stdout:(52/143): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.7 MB/s | 217 kB 00:00 2023-10-18T08:53:05.742 INFO:teuthology.orchestra.run.smithi144.stdout:(53/143): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2023-10-18T08:53:05.818 INFO:teuthology.orchestra.run.smithi144.stdout:(54/143): postgresql-13.11-1.el9.x86_64.rpm 6.5 MB/s | 1.5 MB 00:00 2023-10-18T08:53:05.880 INFO:teuthology.orchestra.run.smithi144.stdout:(55/143): postgresql-contrib-13.11-1.el9.x86_64 3.1 MB/s | 811 kB 00:00 2023-10-18T08:53:05.918 INFO:teuthology.orchestra.run.smithi144.stdout:(56/143): postgresql-private-libs-13.11-1.el9.x 824 kB/s | 134 kB 00:00 2023-10-18T08:53:06.127 INFO:teuthology.orchestra.run.smithi144.stdout:(57/143): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 5.8 MB 00:00 2023-10-18T08:53:06.236 INFO:teuthology.orchestra.run.smithi144.stdout:(58/143): postgresql-server-13.11-1.el9.x86_64. 16 MB/s | 5.7 MB 00:00 2023-10-18T08:53:06.261 INFO:teuthology.orchestra.run.smithi144.stdout:(59/143): python3-devel-3.9.17-2.el9.x86_64.rpm 607 kB/s | 205 kB 00:00 2023-10-18T08:53:06.294 INFO:teuthology.orchestra.run.smithi144.stdout:(60/143): python3-jinja2-2.11.3-4.el9.noarch.rp 1.4 MB/s | 228 kB 00:00 2023-10-18T08:53:06.336 INFO:teuthology.orchestra.run.smithi144.stdout:(61/143): python3-jsonpatch-1.21-16.el9.noarch. 336 kB/s | 24 kB 00:00 2023-10-18T08:53:06.378 INFO:teuthology.orchestra.run.smithi144.stdout:(62/143): python3-jsonpointer-2.0-4.el9.noarch. 158 kB/s | 18 kB 00:00 2023-10-18T08:53:06.403 INFO:teuthology.orchestra.run.smithi144.stdout:(63/143): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 144 kB 00:00 2023-10-18T08:53:06.462 INFO:teuthology.orchestra.run.smithi144.stdout:(64/143): python3-markupsafe-1.1.1-12.el9.x86_6 388 kB/s | 32 kB 00:00 2023-10-18T08:53:06.588 INFO:teuthology.orchestra.run.smithi144.stdout:(65/143): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 4.8 MB 00:00 2023-10-18T08:53:06.613 INFO:teuthology.orchestra.run.smithi144.stdout:(66/143): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 217 kB 00:00 2023-10-18T08:53:06.638 INFO:teuthology.orchestra.run.smithi144.stdout:(67/143): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2023-10-18T08:53:06.705 INFO:teuthology.orchestra.run.smithi144.stdout:(68/143): python3-prettytable-0.7.2-27.el9.noar 627 kB/s | 41 kB 00:00 2023-10-18T08:53:06.738 INFO:teuthology.orchestra.run.smithi144.stdout:(69/143): python3-pyasn1-modules-0.4.8-6.el9.no 2.1 MB/s | 211 kB 00:00 2023-10-18T08:53:06.764 INFO:teuthology.orchestra.run.smithi144.stdout:(70/143): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 132 kB 00:00 2023-10-18T08:53:06.806 INFO:teuthology.orchestra.run.smithi144.stdout:(71/143): python3-requests-oauthlib-1.3.0-12.el 435 kB/s | 43 kB 00:00 2023-10-18T08:53:07.139 INFO:teuthology.orchestra.run.smithi144.stdout:(72/143): python3-toml-0.10.2-6.el9.noarch.rpm 103 kB/s | 39 kB 00:00 2023-10-18T08:53:07.173 INFO:teuthology.orchestra.run.smithi144.stdout:(73/143): socat-1.7.4.1-5.el9.x86_64.rpm 845 kB/s | 300 kB 00:00 2023-10-18T08:53:07.415 INFO:teuthology.orchestra.run.smithi144.stdout:(74/143): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2023-10-18T08:53:07.457 INFO:teuthology.orchestra.run.smithi144.stdout:(75/143): uuid-1.6.2-55.el9.x86_64.rpm 179 kB/s | 56 kB 00:00 2023-10-18T08:53:07.499 INFO:teuthology.orchestra.run.smithi144.stdout:(76/143): xmlstarlet-1.6.1-20.el9.x86_64.rpm 200 kB/s | 63 kB 00:00 2023-10-18T08:53:07.524 INFO:teuthology.orchestra.run.smithi144.stdout:(77/143): fuse-2.9.9-15.el9.x86_64.rpm 734 kB/s | 79 kB 00:00 2023-10-18T08:53:07.558 INFO:teuthology.orchestra.run.smithi144.stdout:(78/143): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.4 MB/s | 195 kB 00:00 2023-10-18T08:53:07.608 INFO:teuthology.orchestra.run.smithi144.stdout:(79/143): libgfortran-11.4.1-2.3.el9.x86_64.rpm 7.3 MB/s | 803 kB 00:00 2023-10-18T08:53:07.633 INFO:teuthology.orchestra.run.smithi144.stdout:(80/143): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.2 MB/s | 241 kB 00:00 2023-10-18T08:53:07.658 INFO:teuthology.orchestra.run.smithi144.stdout:(81/143): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2023-10-18T08:53:07.683 INFO:teuthology.orchestra.run.smithi144.stdout:(82/143): python3-requests-2.25.1-7.el9.noarch. 2.3 MB/s | 113 kB 00:00 2023-10-18T08:53:07.717 INFO:teuthology.orchestra.run.smithi144.stdout:(83/143): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2023-10-18T08:53:07.742 INFO:teuthology.orchestra.run.smithi144.stdout:(84/143): python3-urllib3-1.26.5-3.el9.noarch.r 2.2 MB/s | 188 kB 00:00 2023-10-18T08:53:07.859 INFO:teuthology.orchestra.run.smithi144.stdout:(85/143): elfutils-libelf-devel-0.189-3.el9.x86 193 kB/s | 22 kB 00:00 2023-10-18T08:53:07.960 INFO:teuthology.orchestra.run.smithi144.stdout:(86/143): jq-1.6-15.el9.x86_64.rpm 1.8 MB/s | 187 kB 00:00 2023-10-18T08:53:08.002 INFO:teuthology.orchestra.run.smithi144.stdout:(87/143): python3-cryptography-36.0.1-4.el9.x86 3.9 MB/s | 1.2 MB 00:00 2023-10-18T08:53:08.044 INFO:teuthology.orchestra.run.smithi144.stdout:(88/143): librabbitmq-0.11.0-7.el9.x86_64.rpm 544 kB/s | 45 kB 00:00 2023-10-18T08:53:08.086 INFO:teuthology.orchestra.run.smithi144.stdout:(89/143): libuuid-devel-2.37.4-15.el9.x86_64.rp 308 kB/s | 26 kB 00:00 2023-10-18T08:53:08.128 INFO:teuthology.orchestra.run.smithi144.stdout:(90/143): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2023-10-18T08:53:08.170 INFO:teuthology.orchestra.run.smithi144.stdout:(91/143): python3-pytz-2021.1-5.el9.noarch.rpm 616 kB/s | 51 kB 00:00 2023-10-18T08:53:08.195 INFO:teuthology.orchestra.run.smithi144.stdout:(92/143): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.3 MB/s | 86 kB 00:00 2023-10-18T08:53:08.245 INFO:teuthology.orchestra.run.smithi144.stdout:(93/143): zlib-devel-1.2.11-40.el9.x86_64.rpm 606 kB/s | 45 kB 00:00 2023-10-18T08:53:08.412 INFO:teuthology.orchestra.run.smithi144.stdout:(94/143): openssl-devel-3.0.7-24.el9.x86_64.rpm 11 MB/s | 4.1 MB 00:00 2023-10-18T08:53:08.479 INFO:teuthology.orchestra.run.smithi144.stdout:(95/143): fmt-8.1.1-5.el9.x86_64.rpm 391 kB/s | 111 kB 00:00 2023-10-18T08:53:08.546 INFO:teuthology.orchestra.run.smithi144.stdout:(96/143): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.0 MB/s | 309 kB 00:00 2023-10-18T08:53:08.596 INFO:teuthology.orchestra.run.smithi144.stdout:(97/143): libarrow-doc-9.0.0-9.el9.noarch.rpm 210 kB/s | 24 kB 00:00 2023-10-18T08:53:08.622 INFO:teuthology.orchestra.run.smithi144.stdout:(98/143): liboath-2.6.7-2.el9.x86_64.rpm 652 kB/s | 49 kB 00:00 2023-10-18T08:53:08.664 INFO:teuthology.orchestra.run.smithi144.stdout:(99/143): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2023-10-18T08:53:08.789 INFO:teuthology.orchestra.run.smithi144.stdout:(100/143): python3-asyncssh-2.13.2-3.el9.noarch 4.3 MB/s | 548 kB 00:00 2023-10-18T08:53:08.839 INFO:teuthology.orchestra.run.smithi144.stdout:(101/143): parquet-libs-9.0.0-9.el9.x86_64.rpm 3.8 MB/s | 837 kB 00:00 2023-10-18T08:53:08.864 INFO:teuthology.orchestra.run.smithi144.stdout:(102/143): python3-bcrypt-3.2.2-1.el9.x86_64.rp 584 kB/s | 43 kB 00:00 2023-10-18T08:53:08.898 INFO:teuthology.orchestra.run.smithi144.stdout:(103/143): python3-cachetools-4.2.4-1.el9.noarc 557 kB/s | 32 kB 00:00 2023-10-18T08:53:08.923 INFO:teuthology.orchestra.run.smithi144.stdout:(104/143): python3-certifi-2021.10.8-2.el9.noar 259 kB/s | 15 kB 00:00 2023-10-18T08:53:08.982 INFO:teuthology.orchestra.run.smithi144.stdout:(105/143): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2023-10-18T08:53:09.015 INFO:teuthology.orchestra.run.smithi144.stdout:(106/143): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2023-10-18T08:53:09.049 INFO:teuthology.orchestra.run.smithi144.stdout:(107/143): python3-google-auth-2.23.2-1.el9.noa 2.9 MB/s | 196 kB 00:00 2023-10-18T08:53:09.141 INFO:teuthology.orchestra.run.smithi144.stdout:(108/143): libarrow-9.0.0-9.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2023-10-18T08:53:09.166 INFO:teuthology.orchestra.run.smithi144.stdout:(109/143): python3-jaraco-8.2.1-3.el9.noarch.rp 71 kB/s | 11 kB 00:00 2023-10-18T08:53:09.191 INFO:teuthology.orchestra.run.smithi144.stdout:(110/143): python3-jaraco-classes-3.2.1-5.el9.n 125 kB/s | 18 kB 00:00 2023-10-18T08:53:09.225 INFO:teuthology.orchestra.run.smithi144.stdout:(111/143): python3-jaraco-collections-3.0.0-8.e 402 kB/s | 23 kB 00:00 2023-10-18T08:53:09.250 INFO:teuthology.orchestra.run.smithi144.stdout:(112/143): python3-jaraco-functools-3.5.0-2.el9 236 kB/s | 19 kB 00:00 2023-10-18T08:53:09.275 INFO:teuthology.orchestra.run.smithi144.stdout:(113/143): python3-jaraco-text-3.2.0-6.el9.noar 236 kB/s | 20 kB 00:00 2023-10-18T08:53:09.301 INFO:teuthology.orchestra.run.smithi144.stdout:(114/143): python3-jwt+crypto-2.4.0-1.el9.noarc 184 kB/s | 9.0 kB 00:00 2023-10-18T08:53:09.318 INFO:teuthology.orchestra.run.smithi144.stdout:(115/143): python3-jwt-2.4.0-1.el9.noarch.rpm 621 kB/s | 41 kB 00:00 2023-10-18T08:53:09.351 INFO:teuthology.orchestra.run.smithi144.stdout:(116/143): python3-logutils-0.3.5-21.el9.noarch 935 kB/s | 46 kB 00:00 2023-10-18T08:53:09.385 INFO:teuthology.orchestra.run.smithi144.stdout:(117/143): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2023-10-18T08:53:09.402 INFO:teuthology.orchestra.run.smithi144.stdout:(118/143): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2023-10-18T08:53:09.452 INFO:teuthology.orchestra.run.smithi144.stdout:(119/143): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2023-10-18T08:53:09.478 INFO:teuthology.orchestra.run.smithi144.stdout:(120/143): python3-portend-3.1.0-2.el9.noarch.r 220 kB/s | 16 kB 00:00 2023-10-18T08:53:09.511 INFO:teuthology.orchestra.run.smithi144.stdout:(121/143): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2023-10-18T08:53:09.528 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2023-10-18T08:53:09.536 INFO:teuthology.orchestra.run.smithi144.stdout:(122/143): python3-repoze-lru-0.7-16.el9.noarch 531 kB/s | 31 kB 00:00 2023-10-18T08:53:09.562 INFO:teuthology.orchestra.run.smithi144.stdout:(123/143): python3-pyOpenSSL-21.0.0-1.el9.noarc 830 kB/s | 90 kB 00:00 2023-10-18T08:53:09.595 INFO:teuthology.orchestra.run.smithi144.stdout:(124/143): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2023-10-18T08:53:09.621 INFO:teuthology.orchestra.run.smithi144.stdout:(125/143): python3-tempora-5.0.0-2.el9.noarch.r 614 kB/s | 36 kB 00:00 2023-10-18T08:53:09.646 INFO:teuthology.orchestra.run.smithi144.stdout:(126/143): python3-rsa-4.9-2.el9.noarch.rpm 544 kB/s | 59 kB 00:00 2023-10-18T08:53:09.671 INFO:teuthology.orchestra.run.smithi144.stdout:(127/143): python3-typing-extensions-4.4.0-2.el 683 kB/s | 51 kB 00:00 2023-10-18T08:53:09.705 INFO:teuthology.orchestra.run.smithi144.stdout:(128/143): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2023-10-18T08:53:09.730 INFO:teuthology.orchestra.run.smithi144.stdout:(129/143): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2023-10-18T08:53:09.772 INFO:teuthology.orchestra.run.smithi144.stdout:(130/143): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2023-10-18T08:53:09.797 INFO:teuthology.orchestra.run.smithi144.stdout:(131/143): python3-zc-lockfile-2.0-10.el9.noarc 219 kB/s | 20 kB 00:00 2023-10-18T08:53:09.831 INFO:teuthology.orchestra.run.smithi144.stdout:(132/143): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2023-10-18T08:53:09.898 INFO:teuthology.orchestra.run.smithi144.stdout:(133/143): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2023-10-18T08:53:09.940 INFO:teuthology.orchestra.run.smithi144.stdout:(134/143): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2023-10-18T08:53:10.040 INFO:teuthology.orchestra.run.smithi144.stdout:(135/143): NetworkManager-1.45.4-1.el9.x86_64.r 23 MB/s | 2.2 MB 00:00 2023-10-18T08:53:10.124 INFO:teuthology.orchestra.run.smithi144.stdout:(136/143): NetworkManager-libnm-1.45.4-1.el9.x8 22 MB/s | 1.8 MB 00:00 2023-10-18T08:53:10.157 INFO:teuthology.orchestra.run.smithi144.stdout:(137/143): NetworkManager-team-1.45.4-1.el9.x86 1.2 MB/s | 41 kB 00:00 2023-10-18T08:53:10.200 INFO:teuthology.orchestra.run.smithi144.stdout:(138/143): NetworkManager-tui-1.45.4-1.el9.x86_ 5.9 MB/s | 244 kB 00:00 2023-10-18T08:53:10.242 INFO:teuthology.orchestra.run.smithi144.stdout:(139/143): acl-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 69 kB 00:00 2023-10-18T08:53:10.317 INFO:teuthology.orchestra.run.smithi144.stdout:(140/143): librados2-18.0.0-6786.gf7420d7f.el9. 6.4 MB/s | 3.3 MB 00:00 2023-10-18T08:53:10.342 INFO:teuthology.orchestra.run.smithi144.stdout:(141/143): libacl-2.3.1-4.el9.x86_64.rpm 220 kB/s | 22 kB 00:00 2023-10-18T08:53:10.376 INFO:teuthology.orchestra.run.smithi144.stdout:(142/143): xfsdump-3.1.12-4.el9.x86_64.rpm 5.5 MB/s | 321 kB 00:00 2023-10-18T08:53:10.468 INFO:teuthology.orchestra.run.smithi144.stdout:(143/143): librbd1-18.0.0-6786.gf7420d7f.el9.x8 4.8 MB/s | 3.0 MB 00:00 2023-10-18T08:53:10.475 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2023-10-18T08:53:10.476 INFO:teuthology.orchestra.run.smithi144.stdout:Total 17 MB/s | 200 MB 00:11 2023-10-18T08:53:10.911 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2023-10-18T08:53:11.113 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2023-10-18T08:53:11.114 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2023-10-18T08:53:12.163 INFO:teuthology.orchestra.run.smithi040.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-18T08:53:12.165 INFO:teuthology.orchestra.run.smithi040.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-18T08:53:12.187 INFO:teuthology.orchestra.run.smithi040.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-10-18T08:53:12.189 INFO:teuthology.orchestra.run.smithi040.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-10-18T08:53:12.190 INFO:teuthology.orchestra.run.smithi040.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-10-18T08:53:12.192 INFO:teuthology.orchestra.run.smithi040.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-10-18T08:53:12.195 INFO:teuthology.orchestra.run.smithi040.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-10-18T08:53:12.199 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-10-18T08:53:12.200 INFO:teuthology.orchestra.run.smithi040.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-10-18T08:53:12.283 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2023-10-18T08:53:12.303 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================================ 2023-10-18T08:53:12.303 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2023-10-18T08:53:12.304 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================================ 2023-10-18T08:53:12.304 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2023-10-18T08:53:12.304 INFO:teuthology.orchestra.run.smithi040.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-10-18T08:53:12.304 INFO:teuthology.orchestra.run.smithi040.stdout: ceph x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 6.5 k 2023-10-18T08:53:12.304 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 5.2 M 2023-10-18T08:53:12.305 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 857 k 2023-10-18T08:53:12.305 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 142 k 2023-10-18T08:53:12.305 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 1.5 M 2023-10-18T08:53:12.305 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 132 k 2023-10-18T08:53:12.305 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 1.8 M 2023-10-18T08:53:12.305 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 7.4 M 2023-10-18T08:53:12.306 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 47 k 2023-10-18T08:53:12.306 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 8.6 M 2023-10-18T08:53:12.306 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 41 M 2023-10-18T08:53:12.306 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 262 k 2023-10-18T08:53:12.306 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 270 k 2023-10-18T08:53:12.307 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 218 k 2023-10-18T08:53:12.307 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 18 k 2023-10-18T08:53:12.307 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-10-18T08:53:12.307 INFO:teuthology.orchestra.run.smithi040.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-10-18T08:53:12.307 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-10-18T08:53:12.308 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 31 k 2023-10-18T08:53:12.308 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 716 k 2023-10-18T08:53:12.308 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 127 k 2023-10-18T08:53:12.308 INFO:teuthology.orchestra.run.smithi040.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-10-18T08:53:12.308 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-10-18T08:53:12.308 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-10-18T08:53:12.309 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-10-18T08:53:12.309 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-10-18T08:53:12.309 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-10-18T08:53:12.309 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 162 k 2023-10-18T08:53:12.309 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 321 k 2023-10-18T08:53:12.309 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 297 k 2023-10-18T08:53:12.310 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 99 k 2023-10-18T08:53:12.310 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 86 k 2023-10-18T08:53:12.310 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 3.0 M 2023-10-18T08:53:12.310 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 169 k 2023-10-18T08:53:12.310 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-10-18T08:53:12.311 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2023-10-18T08:53:12.311 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager x86_64 1:1.45.4-1.el9 CentOS-BaseOS 2.2 M 2023-10-18T08:53:12.311 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-libnm x86_64 1:1.45.4-1.el9 CentOS-BaseOS 1.8 M 2023-10-18T08:53:12.311 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-team x86_64 1:1.45.4-1.el9 CentOS-BaseOS 41 k 2023-10-18T08:53:12.311 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-tui x86_64 1:1.45.4-1.el9 CentOS-BaseOS 244 k 2023-10-18T08:53:12.311 INFO:teuthology.orchestra.run.smithi040.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-10-18T08:53:12.312 INFO:teuthology.orchestra.run.smithi040.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-10-18T08:53:12.312 INFO:teuthology.orchestra.run.smithi040.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-10-18T08:53:12.312 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 3.3 M 2023-10-18T08:53:12.312 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 3.0 M 2023-10-18T08:53:12.312 INFO:teuthology.orchestra.run.smithi040.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-10-18T08:53:12.312 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2023-10-18T08:53:12.313 INFO:teuthology.orchestra.run.smithi040.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-10-18T08:53:12.313 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-18T08:53:12.313 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 19 M 2023-10-18T08:53:12.313 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 23 k 2023-10-18T08:53:12.313 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 2.1 M 2023-10-18T08:53:12.313 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 242 k 2023-10-18T08:53:12.314 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 4.4 M 2023-10-18T08:53:12.314 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 18 M 2023-10-18T08:53:12.314 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6786.gf7420d7f.el9 ceph-noarch 15 k 2023-10-18T08:53:12.314 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 24 k 2023-10-18T08:53:12.314 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-18T08:53:12.315 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-18T08:53:12.315 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-18T08:53:12.315 INFO:teuthology.orchestra.run.smithi040.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-18T08:53:12.315 INFO:teuthology.orchestra.run.smithi040.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-18T08:53:12.315 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-18T08:53:12.316 INFO:teuthology.orchestra.run.smithi040.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-18T08:53:12.316 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-18T08:53:12.316 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-18T08:53:12.316 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 164 k 2023-10-18T08:53:12.316 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-18T08:53:12.316 INFO:teuthology.orchestra.run.smithi040.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-18T08:53:12.317 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-18T08:53:12.317 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-18T08:53:12.317 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-18T08:53:12.317 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 472 k 2023-10-18T08:53:12.317 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-18T08:53:12.317 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2 x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 4.8 M 2023-10-18T08:53:12.318 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-18T08:53:12.318 INFO:teuthology.orchestra.run.smithi040.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-10-18T08:53:12.318 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-18T08:53:12.318 INFO:teuthology.orchestra.run.smithi040.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-18T08:53:12.318 INFO:teuthology.orchestra.run.smithi040.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-18T08:53:12.319 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-18T08:53:12.319 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-18T08:53:12.319 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-10-18T08:53:12.319 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-18T08:53:12.319 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-18T08:53:12.320 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-18T08:53:12.320 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-18T08:53:12.320 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 45 k 2023-10-18T08:53:12.320 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el9 ceph 127 k 2023-10-18T08:53:12.320 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-18T08:53:12.320 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-18T08:53:12.321 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-18T08:53:12.321 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-18T08:53:12.321 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-18T08:53:12.321 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-18T08:53:12.321 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth noarch 1:2.23.2-1.el9 epel 196 k 2023-10-18T08:53:12.322 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-18T08:53:12.322 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-18T08:53:12.322 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-18T08:53:12.322 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-18T08:53:12.322 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-18T08:53:12.323 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-18T08:53:12.323 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-18T08:53:12.323 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-18T08:53:12.323 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-18T08:53:12.323 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-18T08:53:12.324 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-18T08:53:12.324 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-18T08:53:12.324 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-18T08:53:12.324 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-18T08:53:12.324 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-18T08:53:12.324 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-18T08:53:12.325 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-18T08:53:12.325 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-18T08:53:12.325 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-18T08:53:12.325 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-18T08:53:12.325 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-18T08:53:12.325 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-18T08:53:12.326 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-18T08:53:12.326 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-18T08:53:12.326 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-18T08:53:12.326 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-18T08:53:12.326 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-18T08:53:12.327 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-18T08:53:12.327 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-18T08:53:12.327 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-18T08:53:12.327 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-18T08:53:12.327 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-18T08:53:12.328 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-18T08:53:12.328 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-18T08:53:12.328 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-18T08:53:12.328 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-18T08:53:12.328 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-10-18T08:53:12.329 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-18T08:53:12.329 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-18T08:53:12.329 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-18T08:53:12.329 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-18T08:53:12.329 INFO:teuthology.orchestra.run.smithi040.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-18T08:53:12.330 INFO:teuthology.orchestra.run.smithi040.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-18T08:53:12.330 INFO:teuthology.orchestra.run.smithi040.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-18T08:53:12.330 INFO:teuthology.orchestra.run.smithi040.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-10-18T08:53:12.330 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-18T08:53:12.331 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-10-18T08:53:12.331 INFO:teuthology.orchestra.run.smithi040.stdout:Installing weak dependencies: 2023-10-18T08:53:12.331 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-18T08:53:12.331 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:53:12.331 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2023-10-18T08:53:12.332 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================================ 2023-10-18T08:53:12.332 INFO:teuthology.orchestra.run.smithi040.stdout:Install 133 Packages 2023-10-18T08:53:12.332 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 10 Packages 2023-10-18T08:53:12.332 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:53:12.333 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 200 M 2023-10-18T08:53:12.333 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2023-10-18T08:53:13.170 INFO:teuthology.orchestra.run.smithi040.stdout:(1/143): ceph-18.0.0-6786.gf7420d7f.el9.x86_64. 26 kB/s | 6.5 kB 00:00 2023-10-18T08:53:13.311 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2023-10-18T08:53:13.312 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2023-10-18T08:53:13.362 INFO:teuthology.orchestra.run.smithi040.stdout:(2/143): ceph-fuse-18.0.0-6786.gf7420d7f.el9.x8 4.4 MB/s | 857 kB 00:00 2023-10-18T08:53:13.429 INFO:teuthology.orchestra.run.smithi040.stdout:(3/143): ceph-immutable-object-cache-18.0.0-678 2.1 MB/s | 142 kB 00:00 2023-10-18T08:53:13.713 INFO:teuthology.orchestra.run.smithi040.stdout:(4/143): ceph-base-18.0.0-6786.gf7420d7f.el9.x8 6.5 MB/s | 5.2 MB 00:00 2023-10-18T08:53:13.797 INFO:teuthology.orchestra.run.smithi040.stdout:(5/143): ceph-mds-18.0.0-6786.gf7420d7f.el9.x86 5.7 MB/s | 2.1 MB 00:00 2023-10-18T08:53:13.888 INFO:teuthology.orchestra.run.smithi040.stdout:(6/143): ceph-mgr-18.0.0-6786.gf7420d7f.el9.x86 8.4 MB/s | 1.5 MB 00:00 2023-10-18T08:53:14.239 INFO:teuthology.orchestra.run.smithi040.stdout:(7/143): ceph-mon-18.0.0-6786.gf7420d7f.el9.x86 10 MB/s | 4.4 MB 00:00 2023-10-18T08:53:15.205 INFO:teuthology.orchestra.run.smithi040.stdout:(8/143): ceph-radosgw-18.0.0-6786.gf7420d7f.el9 8.9 MB/s | 8.6 MB 00:00 2023-10-18T08:53:15.464 INFO:teuthology.orchestra.run.smithi040.stdout:(9/143): ceph-osd-18.0.0-6786.gf7420d7f.el9.x86 11 MB/s | 18 MB 00:01 2023-10-18T08:53:15.506 INFO:teuthology.orchestra.run.smithi040.stdout:(10/143): ceph-selinux-18.0.0-6786.gf7420d7f.el 80 kB/s | 24 kB 00:00 2023-10-18T08:53:15.606 INFO:teuthology.orchestra.run.smithi040.stdout:(11/143): cephfs-mirror-18.0.0-6786.gf7420d7f.e 2.2 MB/s | 218 kB 00:00 2023-10-18T08:53:15.698 INFO:teuthology.orchestra.run.smithi040.stdout:(12/143): libcephfs-devel-18.0.0-6786.gf7420d7f 345 kB/s | 31 kB 00:00 2023-10-18T08:53:15.849 INFO:teuthology.orchestra.run.smithi040.stdout:(13/143): libcephfs2-18.0.0-6786.gf7420d7f.el9. 4.7 MB/s | 716 kB 00:00 2023-10-18T08:53:15.924 INFO:teuthology.orchestra.run.smithi040.stdout:(14/143): libcephsqlite-18.0.0-6786.gf7420d7f.e 2.2 MB/s | 164 kB 00:00 2023-10-18T08:53:15.999 INFO:teuthology.orchestra.run.smithi040.stdout:(15/143): librados-devel-18.0.0-6786.gf7420d7f. 1.7 MB/s | 127 kB 00:00 2023-10-18T08:53:16.108 INFO:teuthology.orchestra.run.smithi040.stdout:(16/143): libradosstriper1-18.0.0-6786.gf7420d7 4.3 MB/s | 472 kB 00:00 2023-10-18T08:53:16.741 INFO:teuthology.orchestra.run.smithi040.stdout:(17/143): librgw2-18.0.0-6786.gf7420d7f.el9.x86 7.6 MB/s | 4.8 MB 00:00 2023-10-18T08:53:16.850 INFO:teuthology.orchestra.run.smithi040.stdout:(18/143): python3-ceph-argparse-18.0.0-6786.gf7 423 kB/s | 45 kB 00:00 2023-10-18T08:53:16.933 INFO:teuthology.orchestra.run.smithi040.stdout:(19/143): python3-ceph-common-18.0.0-6786.gf742 1.5 MB/s | 127 kB 00:00 2023-10-18T08:53:17.159 INFO:teuthology.orchestra.run.smithi040.stdout:(20/143): ceph-common-18.0.0-6786.gf7420d7f.el9 4.5 MB/s | 19 MB 00:04 2023-10-18T08:53:17.192 INFO:teuthology.orchestra.run.smithi040.stdout:(21/143): python3-cephfs-18.0.0-6786.gf7420d7f. 627 kB/s | 162 kB 00:00 2023-10-18T08:53:17.251 INFO:teuthology.orchestra.run.smithi040.stdout:(22/143): python3-rbd-18.0.0-6786.gf7420d7f.el9 5.1 MB/s | 297 kB 00:00 2023-10-18T08:53:17.284 INFO:teuthology.orchestra.run.smithi040.stdout:(23/143): python3-rados-18.0.0-6786.gf7420d7f.e 2.5 MB/s | 321 kB 00:00 2023-10-18T08:53:17.318 INFO:teuthology.orchestra.run.smithi040.stdout:(24/143): python3-rgw-18.0.0-6786.gf7420d7f.el9 1.5 MB/s | 99 kB 00:00 2023-10-18T08:53:17.371 INFO:teuthology.orchestra.run.smithi040.stdout:(25/143): rbd-fuse-18.0.0-6786.gf7420d7f.el9.x8 1.0 MB/s | 86 kB 00:00 2023-10-18T08:53:17.460 INFO:teuthology.orchestra.run.smithi040.stdout:(26/143): rbd-nbd-18.0.0-6786.gf7420d7f.el9.x86 1.9 MB/s | 169 kB 00:00 2023-10-18T08:53:17.586 INFO:teuthology.orchestra.run.smithi040.stdout:(27/143): ceph-grafana-dashboards-18.0.0-6786.g 187 kB/s | 23 kB 00:00 2023-10-18T08:53:17.669 INFO:teuthology.orchestra.run.smithi040.stdout:(28/143): rbd-mirror-18.0.0-6786.gf7420d7f.el9. 8.6 MB/s | 3.0 MB 00:00 2023-10-18T08:53:17.703 INFO:teuthology.orchestra.run.smithi040.stdout:(29/143): ceph-mgr-cephadm-18.0.0-6786.gf7420d7 1.1 MB/s | 132 kB 00:00 2023-10-18T08:53:17.854 INFO:teuthology.orchestra.run.smithi040.stdout:(30/143): ceph-mgr-dashboard-18.0.0-6786.gf7420 9.6 MB/s | 1.8 MB 00:00 2023-10-18T08:53:18.297 INFO:teuthology.orchestra.run.smithi040.stdout:(31/143): ceph-test-18.0.0-6786.gf7420d7f.el9.x 14 MB/s | 41 MB 00:02 2023-10-18T08:53:18.329 INFO:teuthology.orchestra.run.smithi040.stdout:(32/143): ceph-mgr-modules-core-18.0.0-6786.gf7 512 kB/s | 242 kB 00:00 2023-10-18T08:53:18.371 INFO:teuthology.orchestra.run.smithi040.stdout:(33/143): ceph-mgr-rook-18.0.0-6786.gf7420d7f.e 643 kB/s | 47 kB 00:00 2023-10-18T08:53:18.413 INFO:teuthology.orchestra.run.smithi040.stdout:(34/143): ceph-prometheus-alerts-18.0.0-6786.gf 182 kB/s | 15 kB 00:00 2023-10-18T08:53:18.446 INFO:teuthology.orchestra.run.smithi040.stdout:(35/143): ceph-volume-18.0.0-6786.gf7420d7f.el9 3.4 MB/s | 262 kB 00:00 2023-10-18T08:53:18.489 INFO:teuthology.orchestra.run.smithi040.stdout:(36/143): cephadm-18.0.0-6786.gf7420d7f.el9.noa 3.6 MB/s | 270 kB 00:00 2023-10-18T08:53:18.530 INFO:teuthology.orchestra.run.smithi040.stdout:(37/143): cephfs-top-18.0.0-6786.gf7420d7f.el9. 221 kB/s | 18 kB 00:00 2023-10-18T08:53:18.731 INFO:teuthology.orchestra.run.smithi040.stdout:(38/143): boost-program-options-1.75.0-8.el9.x8 532 kB/s | 106 kB 00:00 2023-10-18T08:53:18.772 INFO:teuthology.orchestra.run.smithi040.stdout:(39/143): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 920 kB 00:00 2023-10-18T08:53:18.798 INFO:teuthology.orchestra.run.smithi040.stdout:(40/143): flex-2.6.4-9.el9.x86_64.rpm 4.7 MB/s | 307 kB 00:00 2023-10-18T08:53:18.823 INFO:teuthology.orchestra.run.smithi040.stdout:(41/143): flexiblas-3.0.4-8.el9.x86_64.rpm 648 kB/s | 32 kB 00:00 2023-10-18T08:53:18.924 INFO:teuthology.orchestra.run.smithi040.stdout:(42/143): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2023-10-18T08:53:18.940 INFO:teuthology.orchestra.run.smithi040.stdout:(43/143): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 17 kB 00:00 2023-10-18T08:53:18.974 INFO:teuthology.orchestra.run.smithi040.stdout:(44/143): libaio-devel-0.3.111-13.el9.x86_64.rp 313 kB/s | 10 kB 00:00 2023-10-18T08:53:18.999 INFO:teuthology.orchestra.run.smithi040.stdout:(45/143): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2023-10-18T08:53:19.133 INFO:teuthology.orchestra.run.smithi040.stdout:(46/143): ceph-mgr-diskprediction-local-18.0.0- 5.2 MB/s | 7.4 MB 00:01 2023-10-18T08:53:19.136 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2023-10-18T08:53:19.183 INFO:teuthology.orchestra.run.smithi040.stdout:(47/143): libtool-2.4.6-45.el9.x86_64.rpm 3.1 MB/s | 578 kB 00:00 2023-10-18T08:53:19.206 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/153 2023-10-18T08:53:19.217 INFO:teuthology.orchestra.run.smithi040.stdout:(48/143): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 661 kB 00:00 2023-10-18T08:53:19.242 INFO:teuthology.orchestra.run.smithi040.stdout:(49/143): libzstd-devel-1.5.1-2.el9.x86_64.rpm 443 kB/s | 46 kB 00:00 2023-10-18T08:53:19.284 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/153 2023-10-18T08:53:19.300 INFO:teuthology.orchestra.run.smithi040.stdout:(50/143): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 282 kB 00:00 2023-10-18T08:53:19.326 INFO:teuthology.orchestra.run.smithi040.stdout:(51/143): openblas-0.3.21-2.el9.x86_64.rpm 427 kB/s | 34 kB 00:00 2023-10-18T08:53:19.370 INFO:teuthology.orchestra.run.smithi040.stdout:(52/143): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.7 MB/s | 217 kB 00:00 2023-10-18T08:53:19.467 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/153 2023-10-18T08:53:19.495 INFO:teuthology.orchestra.run.smithi040.stdout:(53/143): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2023-10-18T08:53:19.553 INFO:teuthology.orchestra.run.smithi040.stdout:(54/143): postgresql-13.11-1.el9.x86_64.rpm 7.3 MB/s | 1.5 MB 00:00 2023-10-18T08:53:19.571 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/153 2023-10-18T08:53:19.595 INFO:teuthology.orchestra.run.smithi040.stdout:(55/143): postgresql-contrib-13.11-1.el9.x86_64 3.8 MB/s | 811 kB 00:00 2023-10-18T08:53:19.620 INFO:teuthology.orchestra.run.smithi040.stdout:(56/143): postgresql-private-libs-13.11-1.el9.x 1.1 MB/s | 134 kB 00:00 2023-10-18T08:53:19.779 INFO:teuthology.orchestra.run.smithi040.stdout:(57/143): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 5.8 MB 00:00 2023-10-18T08:53:19.924 INFO:teuthology.orchestra.run.smithi040.stdout:(58/143): postgresql-server-13.11-1.el9.x86_64. 17 MB/s | 5.7 MB 00:00 2023-10-18T08:53:19.955 INFO:teuthology.orchestra.run.smithi040.stdout:(59/143): python3-devel-3.9.17-2.el9.x86_64.rpm 616 kB/s | 205 kB 00:00 2023-10-18T08:53:19.988 INFO:teuthology.orchestra.run.smithi040.stdout:(60/143): python3-jinja2-2.11.3-4.el9.noarch.rp 1.1 MB/s | 228 kB 00:00 2023-10-18T08:53:19.991 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/153 2023-10-18T08:53:20.014 INFO:teuthology.orchestra.run.smithi040.stdout:(61/143): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 18 kB 00:00 2023-10-18T08:53:20.039 INFO:teuthology.orchestra.run.smithi040.stdout:(62/143): python3-jsonpatch-1.21-16.el9.noarch. 303 kB/s | 24 kB 00:00 2023-10-18T08:53:20.046 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librados2-2:18.0.0-6786.gf7420d7f.el9.x86_64 6/153 2023-10-18T08:53:20.064 INFO:teuthology.orchestra.run.smithi040.stdout:(63/143): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2023-10-18T08:53:20.098 INFO:teuthology.orchestra.run.smithi040.stdout:(64/143): python3-markupsafe-1.1.1-12.el9.x86_6 387 kB/s | 32 kB 00:00 2023-10-18T08:53:20.133 INFO:teuthology.orchestra.run.smithi040.stdout:(65/143): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 217 kB 00:00 2023-10-18T08:53:20.194 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:18.0.0-6786.gf7420d7f.el9.x86_64 6/153 2023-10-18T08:53:20.222 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs2-2:18.0.0-6786.gf7420d7f.el9.x86_64 7/153 2023-10-18T08:53:20.257 INFO:teuthology.orchestra.run.smithi040.stdout:(66/143): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2023-10-18T08:53:20.282 INFO:teuthology.orchestra.run.smithi040.stdout:(67/143): python3-oauthlib-3.1.1-5.el9.noarch.r 956 kB/s | 174 kB 00:00 2023-10-18T08:53:20.307 INFO:teuthology.orchestra.run.smithi040.stdout:(68/143): python3-prettytable-0.7.2-27.el9.noar 256 kB/s | 41 kB 00:00 2023-10-18T08:53:20.349 INFO:teuthology.orchestra.run.smithi040.stdout:(69/143): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.2 MB/s | 132 kB 00:00 2023-10-18T08:53:20.368 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs2-2:18.0.0-6786.gf7420d7f.el9.x86_64 7/153 2023-10-18T08:53:20.374 INFO:teuthology.orchestra.run.smithi040.stdout:(70/143): python3-requests-oauthlib-1.3.0-12.el 659 kB/s | 43 kB 00:00 2023-10-18T08:53:20.407 INFO:teuthology.orchestra.run.smithi040.stdout:(71/143): python3-pyasn1-modules-0.4.8-6.el9.no 2.1 MB/s | 211 kB 00:00 2023-10-18T08:53:20.433 INFO:teuthology.orchestra.run.smithi040.stdout:(72/143): python3-toml-0.10.2-6.el9.noarch.rpm 694 kB/s | 39 kB 00:00 2023-10-18T08:53:20.440 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rados-2:18.0.0-6786.gf7420d7f.el9.x86_64 8/153 2023-10-18T08:53:20.484 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/153 2023-10-18T08:53:20.641 INFO:teuthology.orchestra.run.smithi040.stdout:(73/143): uuid-1.6.2-55.el9.x86_64.rpm 293 kB/s | 56 kB 00:00 2023-10-18T08:53:20.667 INFO:teuthology.orchestra.run.smithi040.stdout:(74/143): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 300 kB 00:00 2023-10-18T08:53:20.733 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/153 2023-10-18T08:53:20.794 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/153 2023-10-18T08:53:20.819 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephsqlite-2:18.0.0-6786.gf7420d7f.el9.x86_64 12/153 2023-10-18T08:53:20.859 INFO:teuthology.orchestra.run.smithi040.stdout:(75/143): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2023-10-18T08:53:20.875 INFO:teuthology.orchestra.run.smithi040.stdout:(76/143): xmlstarlet-1.6.1-20.el9.x86_64.rpm 269 kB/s | 63 kB 00:00 2023-10-18T08:53:20.901 INFO:teuthology.orchestra.run.smithi040.stdout:(77/143): fuse-2.9.9-15.el9.x86_64.rpm 344 kB/s | 79 kB 00:00 2023-10-18T08:53:20.934 INFO:teuthology.orchestra.run.smithi040.stdout:(78/143): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.4 MB/s | 195 kB 00:00 2023-10-18T08:53:20.964 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6786.gf7420d7f.el9.x86_64 12/153 2023-10-18T08:53:20.976 INFO:teuthology.orchestra.run.smithi040.stdout:(79/143): libgfortran-11.4.1-2.3.el9.x86_64.rpm 8.0 MB/s | 803 kB 00:00 2023-10-18T08:53:21.002 INFO:teuthology.orchestra.run.smithi040.stdout:(80/143): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.4 MB/s | 241 kB 00:00 2023-10-18T08:53:21.003 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libradosstriper1-2:18.0.0-6786.gf7420d7f.el9.x86 13/153 2023-10-18T08:53:21.018 INFO:teuthology.orchestra.run.smithi040.stdout:(81/143): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2023-10-18T08:53:21.053 INFO:teuthology.orchestra.run.smithi040.stdout:(82/143): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2023-10-18T08:53:21.077 INFO:teuthology.orchestra.run.smithi040.stdout:(83/143): python3-requests-2.25.1-7.el9.noarch. 1.5 MB/s | 113 kB 00:00 2023-10-18T08:53:21.086 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6786.gf7420d7f.el9.x86 13/153 2023-10-18T08:53:21.111 INFO:teuthology.orchestra.run.smithi040.stdout:(84/143): python3-urllib3-1.26.5-3.el9.noarch.r 2.0 MB/s | 188 kB 00:00 2023-10-18T08:53:21.211 INFO:teuthology.orchestra.run.smithi040.stdout:(85/143): elfutils-libelf-devel-0.189-3.el9.x86 225 kB/s | 22 kB 00:00 2023-10-18T08:53:21.345 INFO:teuthology.orchestra.run.smithi040.stdout:(86/143): jq-1.6-15.el9.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2023-10-18T08:53:21.395 INFO:teuthology.orchestra.run.smithi040.stdout:(87/143): librabbitmq-0.11.0-7.el9.x86_64.rpm 915 kB/s | 45 kB 00:00 2023-10-18T08:53:21.428 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/153 2023-10-18T08:53:21.445 INFO:teuthology.orchestra.run.smithi040.stdout:(88/143): libuuid-devel-2.37.4-15.el9.x86_64.rp 524 kB/s | 26 kB 00:00 2023-10-18T08:53:21.495 INFO:teuthology.orchestra.run.smithi040.stdout:(89/143): python3-cryptography-36.0.1-4.el9.x86 3.0 MB/s | 1.2 MB 00:00 2023-10-18T08:53:21.529 INFO:teuthology.orchestra.run.smithi040.stdout:(90/143): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 689 kB 00:00 2023-10-18T08:53:21.605 INFO:teuthology.orchestra.run.smithi040.stdout:(91/143): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.1 MB/s | 86 kB 00:00 2023-10-18T08:53:21.630 INFO:teuthology.orchestra.run.smithi040.stdout:(92/143): python3-pytz-2021.1-5.el9.noarch.rpm 383 kB/s | 51 kB 00:00 2023-10-18T08:53:21.633 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-18T08:53:21.655 INFO:teuthology.orchestra.run.smithi040.stdout:(93/143): zlib-devel-1.2.11-40.el9.x86_64.rpm 915 kB/s | 45 kB 00:00 2023-10-18T08:53:21.721 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-18T08:53:21.922 INFO:teuthology.orchestra.run.smithi040.stdout:(94/143): fmt-8.1.1-5.el9.x86_64.rpm 380 kB/s | 111 kB 00:00 2023-10-18T08:53:21.942 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-18T08:53:21.989 INFO:teuthology.orchestra.run.smithi040.stdout:(95/143): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.2 MB/s | 309 kB 00:00 2023-10-18T08:53:21.995 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-18T08:53:22.098 INFO:teuthology.orchestra.run.smithi040.stdout:(96/143): libarrow-doc-9.0.0-9.el9.noarch.rpm 226 kB/s | 24 kB 00:00 2023-10-18T08:53:22.273 INFO:teuthology.orchestra.run.smithi040.stdout:(97/143): libarrow-9.0.0-9.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2023-10-18T08:53:22.290 INFO:teuthology.orchestra.run.smithi040.stdout:(98/143): liboath-2.6.7-2.el9.x86_64.rpm 263 kB/s | 49 kB 00:00 2023-10-18T08:53:22.323 INFO:teuthology.orchestra.run.smithi040.stdout:(99/143): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2023-10-18T08:53:22.365 INFO:teuthology.orchestra.run.smithi040.stdout:(100/143): parquet-libs-9.0.0-9.el9.x86_64.rpm 11 MB/s | 837 kB 00:00 2023-10-18T08:53:22.411 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-18T08:53:22.459 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/153 2023-10-18T08:53:22.507 INFO:teuthology.orchestra.run.smithi040.stdout:(101/143): python3-asyncssh-2.13.2-3.el9.noarch 3.1 MB/s | 548 kB 00:00 2023-10-18T08:53:22.509 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/153 2023-10-18T08:53:22.524 INFO:teuthology.orchestra.run.smithi040.stdout:(102/143): python3-bcrypt-3.2.2-1.el9.x86_64.rp 274 kB/s | 43 kB 00:00 2023-10-18T08:53:22.549 INFO:teuthology.orchestra.run.smithi040.stdout:(103/143): python3-cachetools-4.2.4-1.el9.noarc 792 kB/s | 32 kB 00:00 2023-10-18T08:53:22.566 INFO:teuthology.orchestra.run.smithi040.stdout:(104/143): python3-certifi-2021.10.8-2.el9.noar 363 kB/s | 15 kB 00:00 2023-10-18T08:53:22.612 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/153 2023-10-18T08:53:22.650 INFO:teuthology.orchestra.run.smithi040.stdout:(105/143): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2023-10-18T08:53:22.674 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 20/153 2023-10-18T08:53:22.684 INFO:teuthology.orchestra.run.smithi040.stdout:(106/143): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2023-10-18T08:53:22.704 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/153 2023-10-18T08:53:22.717 INFO:teuthology.orchestra.run.smithi040.stdout:(107/143): python3-google-auth-2.23.2-1.el9.noa 3.1 MB/s | 196 kB 00:00 2023-10-18T08:53:22.734 INFO:teuthology.orchestra.run.smithi040.stdout:(108/143): python3-jaraco-8.2.1-3.el9.noarch.rp 225 kB/s | 11 kB 00:00 2023-10-18T08:53:22.755 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/153 2023-10-18T08:53:22.784 INFO:teuthology.orchestra.run.smithi040.stdout:(109/143): python3-jaraco-classes-3.2.1-5.el9.n 268 kB/s | 18 kB 00:00 2023-10-18T08:53:22.801 INFO:teuthology.orchestra.run.smithi040.stdout:(110/143): python3-jaraco-collections-3.0.0-8.e 349 kB/s | 23 kB 00:00 2023-10-18T08:53:22.844 INFO:teuthology.orchestra.run.smithi040.stdout:(111/143): python3-jaraco-functools-3.5.0-2.el9 338 kB/s | 19 kB 00:00 2023-10-18T08:53:22.857 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/153 2023-10-18T08:53:22.869 INFO:teuthology.orchestra.run.smithi040.stdout:(112/143): python3-jaraco-text-3.2.0-6.el9.noar 297 kB/s | 20 kB 00:00 2023-10-18T08:53:22.907 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/153 2023-10-18T08:53:22.911 INFO:teuthology.orchestra.run.smithi040.stdout:(113/143): python3-jwt+crypto-2.4.0-1.el9.noarc 137 kB/s | 9.0 kB 00:00 2023-10-18T08:53:22.936 INFO:teuthology.orchestra.run.smithi040.stdout:(114/143): python3-jwt-2.4.0-1.el9.noarch.rpm 618 kB/s | 41 kB 00:00 2023-10-18T08:53:22.940 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/153 2023-10-18T08:53:23.035 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/153 2023-10-18T08:53:23.062 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el9. 27/153 2023-10-18T08:53:23.069 INFO:teuthology.orchestra.run.smithi040.stdout:(115/143): openssl-devel-3.0.7-24.el9.x86_64.rp 2.5 MB/s | 4.1 MB 00:01 2023-10-18T08:53:23.108 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 28/153 2023-10-18T08:53:23.170 INFO:teuthology.orchestra.run.smithi040.stdout:(116/143): python3-kubernetes-26.1.0-2.el9.noar 4.0 MB/s | 1.0 MB 00:00 2023-10-18T08:53:23.195 INFO:teuthology.orchestra.run.smithi040.stdout:(117/143): python3-logutils-0.3.5-21.el9.noarch 179 kB/s | 46 kB 00:00 2023-10-18T08:53:23.212 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cephfs-2:18.0.0-6786.gf7420d7f.el9.x86_6 29/153 2023-10-18T08:53:23.220 INFO:teuthology.orchestra.run.smithi040.stdout:(118/143): python3-more-itertools-8.12.0-2.el9. 561 kB/s | 79 kB 00:00 2023-10-18T08:53:23.279 INFO:teuthology.orchestra.run.smithi040.stdout:(119/143): python3-natsort-7.1.1-5.el9.noarch.r 707 kB/s | 58 kB 00:00 2023-10-18T08:53:23.313 INFO:teuthology.orchestra.run.smithi040.stdout:(120/143): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2023-10-18T08:53:23.346 INFO:teuthology.orchestra.run.smithi040.stdout:(121/143): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2023-10-18T08:53:23.372 INFO:teuthology.orchestra.run.smithi040.stdout:(122/143): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2023-10-18T08:53:23.395 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/153 2023-10-18T08:53:23.397 INFO:teuthology.orchestra.run.smithi040.stdout:(123/143): python3-repoze-lru-0.7-16.el9.noarch 637 kB/s | 31 kB 00:00 2023-10-18T08:53:23.430 INFO:teuthology.orchestra.run.smithi040.stdout:(124/143): python3-routes-2.5.1-5.el9.noarch.rp 2.3 MB/s | 188 kB 00:00 2023-10-18T08:53:23.455 INFO:teuthology.orchestra.run.smithi040.stdout:(125/143): python3-rsa-4.9-2.el9.noarch.rpm 719 kB/s | 59 kB 00:00 2023-10-18T08:53:23.463 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/153 2023-10-18T08:53:23.481 INFO:teuthology.orchestra.run.smithi040.stdout:(126/143): python3-tempora-5.0.0-2.el9.noarch.r 444 kB/s | 36 kB 00:00 2023-10-18T08:53:23.506 INFO:teuthology.orchestra.run.smithi040.stdout:(127/143): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2023-10-18T08:53:23.540 INFO:teuthology.orchestra.run.smithi040.stdout:(128/143): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2023-10-18T08:53:23.573 INFO:teuthology.orchestra.run.smithi040.stdout:(129/143): python3-websocket-client-1.2.3-2.el9 985 kB/s | 90 kB 00:00 2023-10-18T08:53:23.607 INFO:teuthology.orchestra.run.smithi040.stdout:(130/143): python3-werkzeug-2.0.3-3.el9.1.noarc 4.3 MB/s | 427 kB 00:00 2023-10-18T08:53:23.632 INFO:teuthology.orchestra.run.smithi040.stdout:(131/143): python3-zc-lockfile-2.0-10.el9.noarc 230 kB/s | 20 kB 00:00 2023-10-18T08:53:23.674 INFO:teuthology.orchestra.run.smithi040.stdout:(132/143): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2023-10-18T08:53:23.763 INFO:teuthology.orchestra.run.smithi040.stdout:(133/143): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2023-10-18T08:53:23.808 INFO:teuthology.orchestra.run.smithi040.stdout:(134/143): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.5 MB/s | 68 kB 00:00 2023-10-18T08:53:23.909 INFO:teuthology.orchestra.run.smithi040.stdout:(135/143): NetworkManager-1.45.4-1.el9.x86_64.r 23 MB/s | 2.2 MB 00:00 2023-10-18T08:53:23.992 INFO:teuthology.orchestra.run.smithi040.stdout:(136/143): NetworkManager-libnm-1.45.4-1.el9.x8 22 MB/s | 1.8 MB 00:00 2023-10-18T08:53:24.026 INFO:teuthology.orchestra.run.smithi040.stdout:(137/143): NetworkManager-team-1.45.4-1.el9.x86 1.3 MB/s | 41 kB 00:00 2023-10-18T08:53:24.067 INFO:teuthology.orchestra.run.smithi040.stdout:(138/143): NetworkManager-tui-1.45.4-1.el9.x86_ 5.9 MB/s | 244 kB 00:00 2023-10-18T08:53:24.109 INFO:teuthology.orchestra.run.smithi040.stdout:(139/143): acl-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 69 kB 00:00 2023-10-18T08:53:24.143 INFO:teuthology.orchestra.run.smithi040.stdout:(140/143): libacl-2.3.1-4.el9.x86_64.rpm 679 kB/s | 22 kB 00:00 2023-10-18T08:53:24.144 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/153 2023-10-18T08:53:24.185 INFO:teuthology.orchestra.run.smithi040.stdout:(141/143): xfsdump-3.1.12-4.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2023-10-18T08:53:24.244 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/153 2023-10-18T08:53:24.268 INFO:teuthology.orchestra.run.smithi040.stdout:(142/143): librbd1-18.0.0-6786.gf7420d7f.el9.x8 5.2 MB/s | 3.0 MB 00:00 2023-10-18T08:53:24.290 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/153 2023-10-18T08:53:24.338 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librados-devel-2:18.0.0-6786.gf7420d7f.el9.x86_6 35/153 2023-10-18T08:53:24.352 INFO:teuthology.orchestra.run.smithi040.stdout:(143/143): librados2-18.0.0-6786.gf7420d7f.el9. 4.6 MB/s | 3.3 MB 00:00 2023-10-18T08:53:24.364 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2023-10-18T08:53:24.364 INFO:teuthology.orchestra.run.smithi040.stdout:Total 17 MB/s | 200 MB 00:12 2023-10-18T08:53:24.402 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/153 2023-10-18T08:53:24.560 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/153 2023-10-18T08:53:24.596 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/153 2023-10-18T08:53:24.662 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/153 2023-10-18T08:53:24.739 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/153 2023-10-18T08:53:24.821 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2023-10-18T08:53:24.914 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/153 2023-10-18T08:53:24.956 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/153 2023-10-18T08:53:25.029 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/153 2023-10-18T08:53:25.040 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2023-10-18T08:53:25.040 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2023-10-18T08:53:25.091 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/153 2023-10-18T08:53:25.157 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/153 2023-10-18T08:53:25.350 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/153 2023-10-18T08:53:25.419 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/153 2023-10-18T08:53:25.478 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/153 2023-10-18T08:53:25.609 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/153 2023-10-18T08:53:25.653 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-google-auth-1:2.23.2-1.el9.noarch 50/153 2023-10-18T08:53:25.707 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/153 2023-10-18T08:53:25.744 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/153 2023-10-18T08:53:25.783 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/153 2023-10-18T08:53:25.825 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fmt-8.1.1-5.el9.x86_64 54/153 2023-10-18T08:53:25.878 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 55/153 2023-10-18T08:53:26.033 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 56/153 2023-10-18T08:53:26.086 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : automake-1.16.2-8.el9.noarch 57/153 2023-10-18T08:53:26.168 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/153 2023-10-18T08:53:26.262 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/153 2023-10-18T08:53:26.527 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/153 2023-10-18T08:53:26.582 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/153 2023-10-18T08:53:26.755 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/153 2023-10-18T08:53:26.810 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/153 2023-10-18T08:53:26.833 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/153 2023-10-18T08:53:26.873 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/153 2023-10-18T08:53:26.924 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/153 2023-10-18T08:53:26.968 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/153 2023-10-18T08:53:27.022 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/153 2023-10-18T08:53:27.102 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/153 2023-10-18T08:53:27.152 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 70/153 2023-10-18T08:53:27.193 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/153 2023-10-18T08:53:27.300 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2023-10-18T08:53:27.300 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2023-10-18T08:53:27.319 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/153 2023-10-18T08:53:27.357 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/153 2023-10-18T08:53:27.392 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/153 2023-10-18T08:53:27.482 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/153 2023-10-18T08:53:27.530 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/153 2023-10-18T08:53:27.666 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/153 2023-10-18T08:53:27.709 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/153 2023-10-18T08:53:27.852 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/153 2023-10-18T08:53:27.925 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/153 2023-10-18T08:53:28.906 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/153 2023-10-18T08:53:28.965 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/153 2023-10-18T08:53:28.990 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/153 2023-10-18T08:53:29.117 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/153 2023-10-18T08:53:29.157 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 85/153 2023-10-18T08:53:29.387 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-0.3.21-2.el9.x86_64 86/153 2023-10-18T08:53:29.421 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 87/153 2023-10-18T08:53:29.514 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 88/153 2023-10-18T08:53:30.332 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 89/153 2023-10-18T08:53:30.525 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 90/153 2023-10-18T08:53:32.492 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 91/153 2023-10-18T08:53:32.584 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 92/153 2023-10-18T08:53:32.621 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 93/153 2023-10-18T08:53:32.652 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : jq-1.6-15.el9.x86_64 94/153 2023-10-18T08:53:32.724 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 95/153 2023-10-18T08:53:33.099 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/153 2023-10-18T08:53:33.140 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librbd1-2:18.0.0-6786.gf7420d7f.el9.x86_64 97/153 2023-10-18T08:53:33.261 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:18.0.0-6786.gf7420d7f.el9.x86_64 97/153 2023-10-18T08:53:33.296 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 98/153 2023-10-18T08:53:33.352 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/153 2023-10-18T08:53:33.454 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2023-10-18T08:53:33.534 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/153 2023-10-18T08:53:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/153 2023-10-18T08:53:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/153 2023-10-18T08:53:33.710 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/153 2023-10-18T08:53:33.845 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 100/153 2023-10-18T08:53:33.876 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librgw2-2:18.0.0-6786.gf7420d7f.el9.x86_64 101/153 2023-10-18T08:53:34.018 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librgw2-2:18.0.0-6786.gf7420d7f.el9.x86_64 101/153 2023-10-18T08:53:34.045 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 102/153 2023-10-18T08:53:34.142 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/153 2023-10-18T08:53:34.183 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librados2-2:18.0.0-6786.gf7420d7f.el9.x86_64 6/153 2023-10-18T08:53:34.356 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:18.0.0-6786.gf7420d7f.el9.x86_64 6/153 2023-10-18T08:53:34.392 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs2-2:18.0.0-6786.gf7420d7f.el9.x86_64 7/153 2023-10-18T08:53:34.523 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs2-2:18.0.0-6786.gf7420d7f.el9.x86_64 7/153 2023-10-18T08:53:34.583 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rados-2:18.0.0-6786.gf7420d7f.el9.x86_64 8/153 2023-10-18T08:53:34.625 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/153 2023-10-18T08:53:34.914 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/153 2023-10-18T08:53:34.980 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/153 2023-10-18T08:53:35.005 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephsqlite-2:18.0.0-6786.gf7420d7f.el9.x86_64 12/153 2023-10-18T08:53:35.186 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6786.gf7420d7f.el9.x86_64 12/153 2023-10-18T08:53:35.232 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libradosstriper1-2:18.0.0-6786.gf7420d7f.el9.x86 13/153 2023-10-18T08:53:35.333 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6786.gf7420d7f.el9.x86 13/153 2023-10-18T08:53:35.771 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/153 2023-10-18T08:53:35.941 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-18T08:53:36.034 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 15/153 2023-10-18T08:53:36.264 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-18T08:53:36.315 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-18T08:53:36.607 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 103/153 2023-10-18T08:53:36.658 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 103/153 2023-10-18T08:53:36.760 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 16/153 2023-10-18T08:53:36.803 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/153 2023-10-18T08:53:36.862 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/153 2023-10-18T08:53:36.989 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/153 2023-10-18T08:53:37.066 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 20/153 2023-10-18T08:53:37.108 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/153 2023-10-18T08:53:37.170 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/153 2023-10-18T08:53:37.285 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 103/153 2023-10-18T08:53:37.307 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/153 2023-10-18T08:53:37.328 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-base-2:18.0.0-6786.gf7420d7f.el9.x86_64 104/153 2023-10-18T08:53:37.359 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/153 2023-10-18T08:53:37.393 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/153 2023-10-18T08:53:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-base-2:18.0.0-6786.gf7420d7f.el9.x86_64 104/153 2023-10-18T08:53:37.510 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-18T08:53:37.510 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-18T08:53:37.510 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:53:37.513 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/153 2023-10-18T08:53:37.547 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el9. 27/153 2023-10-18T08:53:37.563 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-selinux-2:18.0.0-6786.gf7420d7f.el9.x86_64 105/153 2023-10-18T08:53:37.598 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 28/153 2023-10-18T08:53:37.711 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cephfs-2:18.0.0-6786.gf7420d7f.el9.x86_6 29/153 2023-10-18T08:53:37.894 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/153 2023-10-18T08:53:37.967 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/153 2023-10-18T08:53:38.714 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/153 2023-10-18T08:53:38.821 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/153 2023-10-18T08:53:38.868 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/153 2023-10-18T08:53:38.907 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados-devel-2:18.0.0-6786.gf7420d7f.el9.x86_6 35/153 2023-10-18T08:53:38.945 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/153 2023-10-18T08:53:39.108 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/153 2023-10-18T08:53:39.190 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/153 2023-10-18T08:53:39.265 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/153 2023-10-18T08:53:39.376 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/153 2023-10-18T08:53:39.416 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/153 2023-10-18T08:53:39.457 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/153 2023-10-18T08:53:39.527 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/153 2023-10-18T08:53:39.588 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/153 2023-10-18T08:53:39.650 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/153 2023-10-18T08:53:39.851 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/153 2023-10-18T08:53:39.897 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/153 2023-10-18T08:53:39.962 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/153 2023-10-18T08:53:40.096 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/153 2023-10-18T08:53:40.146 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-google-auth-1:2.23.2-1.el9.noarch 50/153 2023-10-18T08:53:40.206 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/153 2023-10-18T08:53:40.237 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/153 2023-10-18T08:53:40.277 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/153 2023-10-18T08:53:40.320 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fmt-8.1.1-5.el9.x86_64 54/153 2023-10-18T08:53:40.374 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 55/153 2023-10-18T08:53:40.563 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 56/153 2023-10-18T08:53:40.625 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : automake-1.16.2-8.el9.noarch 57/153 2023-10-18T08:53:40.707 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ply-3.11-14.el9.noarch 58/153 2023-10-18T08:53:40.802 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 59/153 2023-10-18T08:53:41.089 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 60/153 2023-10-18T08:53:41.157 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 61/153 2023-10-18T08:53:41.344 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 62/153 2023-10-18T08:53:41.402 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 63/153 2023-10-18T08:53:41.442 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 64/153 2023-10-18T08:53:41.482 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 65/153 2023-10-18T08:53:41.525 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 66/153 2023-10-18T08:53:41.568 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fuse-2.9.9-15.el9.x86_64 67/153 2023-10-18T08:53:41.616 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/153 2023-10-18T08:53:41.695 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/153 2023-10-18T08:53:41.752 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 70/153 2023-10-18T08:53:41.793 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/153 2023-10-18T08:53:41.924 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/153 2023-10-18T08:53:41.967 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/153 2023-10-18T08:53:42.001 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 74/153 2023-10-18T08:53:42.063 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/153 2023-10-18T08:53:42.106 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 76/153 2023-10-18T08:53:42.244 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 77/153 2023-10-18T08:53:42.286 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/153 2023-10-18T08:53:42.458 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/153 2023-10-18T08:53:42.527 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/153 2023-10-18T08:53:43.561 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/153 2023-10-18T08:53:43.642 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/153 2023-10-18T08:53:43.676 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/153 2023-10-18T08:53:43.814 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/153 2023-10-18T08:53:43.859 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 85/153 2023-10-18T08:53:44.111 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-0.3.21-2.el9.x86_64 86/153 2023-10-18T08:53:44.257 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 87/153 2023-10-18T08:53:44.390 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 88/153 2023-10-18T08:53:45.305 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 89/153 2023-10-18T08:53:45.442 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 90/153 2023-10-18T08:53:47.161 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 91/153 2023-10-18T08:53:47.260 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 92/153 2023-10-18T08:53:47.298 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 93/153 2023-10-18T08:53:47.328 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : jq-1.6-15.el9.x86_64 94/153 2023-10-18T08:53:47.368 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 95/153 2023-10-18T08:53:47.752 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/153 2023-10-18T08:53:47.783 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librbd1-2:18.0.0-6786.gf7420d7f.el9.x86_64 97/153 2023-10-18T08:53:47.905 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:18.0.0-6786.gf7420d7f.el9.x86_64 97/153 2023-10-18T08:53:47.939 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 98/153 2023-10-18T08:53:47.996 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/153 2023-10-18T08:53:48.510 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 100/153 2023-10-18T08:53:48.536 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librgw2-2:18.0.0-6786.gf7420d7f.el9.x86_64 101/153 2023-10-18T08:53:48.635 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librgw2-2:18.0.0-6786.gf7420d7f.el9.x86_64 101/153 2023-10-18T08:53:48.671 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 102/153 2023-10-18T08:53:50.005 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6786.gf7420d7f.el9.x86_64 105/153 2023-10-18T08:53:50.005 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /sys 2023-10-18T08:53:50.006 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /proc 2023-10-18T08:53:50.006 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /mnt 2023-10-18T08:53:50.006 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /var/tmp 2023-10-18T08:53:50.006 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /home 2023-10-18T08:53:50.006 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /root 2023-10-18T08:53:50.007 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /tmp 2023-10-18T08:53:50.007 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:53:50.234 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-volume-2:18.0.0-6786.gf7420d7f.el9.noarch 106/153 2023-10-18T08:53:50.289 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-volume-2:18.0.0-6786.gf7420d7f.el9.noarch 106/153 2023-10-18T08:53:50.290 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:53:50.290 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-18T08:53:50.290 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:53:51.276 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 103/153 2023-10-18T08:53:51.318 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 103/153 2023-10-18T08:53:51.989 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 103/153 2023-10-18T08:53:52.020 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-osd-2:18.0.0-6786.gf7420d7f.el9.x86_64 107/153 2023-10-18T08:53:52.037 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-base-2:18.0.0-6786.gf7420d7f.el9.x86_64 104/153 2023-10-18T08:53:52.105 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-osd-2:18.0.0-6786.gf7420d7f.el9.x86_64 107/153 2023-10-18T08:53:52.106 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:53:52.106 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-18T08:53:52.106 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-18T08:53:52.106 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-18T08:53:52.107 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:53:52.210 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-base-2:18.0.0-6786.gf7420d7f.el9.x86_64 104/153 2023-10-18T08:53:52.211 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-18T08:53:52.211 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-18T08:53:52.211 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:53:52.246 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-selinux-2:18.0.0-6786.gf7420d7f.el9.x86_64 105/153 2023-10-18T08:53:52.319 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mds-2:18.0.0-6786.gf7420d7f.el9.x86_64 108/153 2023-10-18T08:53:52.391 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mds-2:18.0.0-6786.gf7420d7f.el9.x86_64 108/153 2023-10-18T08:53:52.392 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:53:52.392 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-18T08:53:52.392 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-18T08:53:52.392 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-18T08:53:52.393 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:53:52.879 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mon-2:18.0.0-6786.gf7420d7f.el9.x86_64 109/153 2023-10-18T08:53:52.947 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mon-2:18.0.0-6786.gf7420d7f.el9.x86_64 109/153 2023-10-18T08:53:52.948 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:53:52.948 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-18T08:53:52.948 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-18T08:53:52.948 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-18T08:53:52.948 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:53:53.771 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 110/153 2023-10-18T08:53:53.807 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 110/153 2023-10-18T08:53:53.852 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 111/153 2023-10-18T08:53:53.965 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 112/153 2023-10-18T08:53:53.993 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el9.noa 113/153 2023-10-18T08:53:54.313 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el9.noa 113/153 2023-10-18T08:53:54.360 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el9.n 114/153 2023-10-18T08:53:55.489 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el9.n 114/153 2023-10-18T08:53:55.519 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 115/153 2023-10-18T08:53:55.661 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 115/153 2023-10-18T08:53:55.835 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 116/153 2023-10-18T08:53:55.863 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-2:18.0.0-6786.gf7420d7f.el9.x86_64 117/153 2023-10-18T08:53:55.954 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6786.gf7420d7f.el9.x86_64 117/153 2023-10-18T08:53:55.954 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:53:55.955 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-18T08:53:55.955 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-18T08:53:55.955 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-18T08:53:55.955 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:53:56.013 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el9.noarch 118/153 2023-10-18T08:53:56.053 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el9.noarch 118/153 2023-10-18T08:53:56.973 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-2:18.0.0-6786.gf7420d7f.el9.x86_64 119/153 2023-10-18T08:53:57.006 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 120/153 2023-10-18T08:53:57.100 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 120/153 2023-10-18T08:53:57.100 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:53:57.100 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-18T08:53:57.100 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-18T08:53:57.101 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-18T08:53:57.101 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:53:57.148 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 121/153 2023-10-18T08:53:57.250 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6786.gf7420 121/153 2023-10-18T08:53:57.250 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:53:57.250 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-18T08:53:57.251 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:53:57.358 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 122/153 2023-10-18T08:53:57.469 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 122/153 2023-10-18T08:53:57.469 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:53:57.470 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-10-18T08:53:57.470 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-18T08:53:57.470 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-18T08:53:57.470 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:53:57.867 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 123/153 2023-10-18T08:53:57.968 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 123/153 2023-10-18T08:53:57.969 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:53:57.969 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-18T08:53:57.969 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-18T08:53:57.969 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-18T08:53:57.970 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:02.126 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-test-2:18.0.0-6786.gf7420d7f.el9.x86_64 124/153 2023-10-18T08:54:02.182 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 125/153 2023-10-18T08:54:02.223 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-nbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 126/153 2023-10-18T08:54:02.491 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 127/153 2023-10-18T08:54:02.613 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 128/153 2023-10-18T08:54:02.677 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 129/153 2023-10-18T08:54:02.725 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/153 2023-10-18T08:54:02.762 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/153 2023-10-18T08:54:02.792 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el9.x86_ 132/153 2023-10-18T08:54:02.828 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-team-1:1.45.4-1.el9.x86_64 133/153 2023-10-18T08:54:02.938 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : NetworkManager-tui-1:1.45.4-1.el9.x86_64 134/153 2023-10-18T08:54:03.005 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 135/153 2023-10-18T08:54:03.046 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 136/153 2023-10-18T08:54:03.990 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-18T08:54:04.036 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-18T08:54:04.099 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-18T08:54:04.159 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-top-2:18.0.0-6786.gf7420d7f.el9.noarch 138/153 2023-10-18T08:54:05.554 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 139/153 2023-10-18T08:54:05.642 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 140/153 2023-10-18T08:54:05.688 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 141/153 2023-10-18T08:54:05.715 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6786.gf7420d7f.el9.x86_64 105/153 2023-10-18T08:54:05.716 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /sys 2023-10-18T08:54:05.716 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /proc 2023-10-18T08:54:05.716 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /mnt 2023-10-18T08:54:05.716 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /var/tmp 2023-10-18T08:54:05.716 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /home 2023-10-18T08:54:05.717 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /root 2023-10-18T08:54:05.717 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /tmp 2023-10-18T08:54:05.717 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:05.803 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flex-2.6.4-9.el9.x86_64 142/153 2023-10-18T08:54:05.844 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : bison-3.7.4-5.el9.x86_64 143/153 2023-10-18T08:54:05.872 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 144/153 2023-10-18T08:54:05.873 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-18T08:54:05.936 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-volume-2:18.0.0-6786.gf7420d7f.el9.noarch 106/153 2023-10-18T08:54:05.986 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-volume-2:18.0.0-6786.gf7420d7f.el9.noarch 106/153 2023-10-18T08:54:05.987 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:54:05.987 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-18T08:54:05.987 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:05.992 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-18T08:54:06.024 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 146/153 2023-10-18T08:54:06.056 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 147/153 2023-10-18T08:54:06.089 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 148/153 2023-10-18T08:54:06.128 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-18T08:54:06.129 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-18T08:54:06.463 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-18T08:54:06.466 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-18T08:54:06.603 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-18T08:54:06.634 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-18T08:54:06.634 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-18T08:54:06.787 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-18T08:54:06.913 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-18T08:54:07.151 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-18T08:54:07.151 INFO:teuthology.orchestra.run.smithi144.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-18T08:54:07.152 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-osd-2:18.0.0-6786.gf7420d7f.el9.x86_64 107/153 2023-10-18T08:54:07.843 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-osd-2:18.0.0-6786.gf7420d7f.el9.x86_64 107/153 2023-10-18T08:54:07.843 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:54:07.844 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-18T08:54:07.844 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-18T08:54:07.844 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-18T08:54:07.844 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:08.088 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mds-2:18.0.0-6786.gf7420d7f.el9.x86_64 108/153 2023-10-18T08:54:08.166 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mds-2:18.0.0-6786.gf7420d7f.el9.x86_64 108/153 2023-10-18T08:54:08.166 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:54:08.166 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-18T08:54:08.167 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-18T08:54:08.167 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-18T08:54:08.167 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:08.714 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mon-2:18.0.0-6786.gf7420d7f.el9.x86_64 109/153 2023-10-18T08:54:08.789 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mon-2:18.0.0-6786.gf7420d7f.el9.x86_64 109/153 2023-10-18T08:54:08.790 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:54:08.790 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-18T08:54:08.790 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-18T08:54:08.790 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-18T08:54:08.790 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:09.416 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 110/153 2023-10-18T08:54:09.452 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 110/153 2023-10-18T08:54:09.501 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 111/153 2023-10-18T08:54:09.618 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 112/153 2023-10-18T08:54:09.661 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el9.noa 113/153 2023-10-18T08:54:10.002 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el9.noa 113/153 2023-10-18T08:54:10.037 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el9.n 114/153 2023-10-18T08:54:11.221 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el9.n 114/153 2023-10-18T08:54:11.254 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 115/153 2023-10-18T08:54:11.404 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 115/153 2023-10-18T08:54:11.579 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 116/153 2023-10-18T08:54:11.606 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-2:18.0.0-6786.gf7420d7f.el9.x86_64 117/153 2023-10-18T08:54:11.683 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6786.gf7420d7f.el9.x86_64 117/153 2023-10-18T08:54:11.683 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:54:11.683 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-18T08:54:11.683 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-18T08:54:11.684 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-18T08:54:11.684 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:11.728 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-2:18.0.0-6786.gf7420d7f.el9.x86_64 1/153 2023-10-18T08:54:11.729 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-base-2:18.0.0-6786.gf7420d7f.el9.x86_64 2/153 2023-10-18T08:54:11.729 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 3/153 2023-10-18T08:54:11.729 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 4/153 2023-10-18T08:54:11.729 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 5/153 2023-10-18T08:54:11.730 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mds-2:18.0.0-6786.gf7420d7f.el9.x86_64 6/153 2023-10-18T08:54:11.730 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-2:18.0.0-6786.gf7420d7f.el9.x86_64 7/153 2023-10-18T08:54:11.730 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mon-2:18.0.0-6786.gf7420d7f.el9.x86_64 8/153 2023-10-18T08:54:11.730 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-osd-2:18.0.0-6786.gf7420d7f.el9.x86_64 9/153 2023-10-18T08:54:11.730 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 10/153 2023-10-18T08:54:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-selinux-2:18.0.0-6786.gf7420d7f.el9.x86_64 11/153 2023-10-18T08:54:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-test-2:18.0.0-6786.gf7420d7f.el9.x86_64 12/153 2023-10-18T08:54:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 13/153 2023-10-18T08:54:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el9.x86_ 14/153 2023-10-18T08:54:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs2-2:18.0.0-6786.gf7420d7f.el9.x86_64 15/153 2023-10-18T08:54:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephsqlite-2:18.0.0-6786.gf7420d7f.el9.x86_64 16/153 2023-10-18T08:54:11.732 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados-devel-2:18.0.0-6786.gf7420d7f.el9.x86_6 17/153 2023-10-18T08:54:11.732 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libradosstriper1-2:18.0.0-6786.gf7420d7f.el9.x86 18/153 2023-10-18T08:54:11.732 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librgw2-2:18.0.0-6786.gf7420d7f.el9.x86_64 19/153 2023-10-18T08:54:11.732 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 20/153 2023-10-18T08:54:11.732 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el9. 21/153 2023-10-18T08:54:11.733 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cephfs-2:18.0.0-6786.gf7420d7f.el9.x86_6 22/153 2023-10-18T08:54:11.733 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rados-2:18.0.0-6786.gf7420d7f.el9.x86_64 23/153 2023-10-18T08:54:11.733 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 24/153 2023-10-18T08:54:11.733 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 25/153 2023-10-18T08:54:11.733 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 26/153 2023-10-18T08:54:11.734 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 27/153 2023-10-18T08:54:11.734 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-nbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 28/153 2023-10-18T08:54:11.734 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 29/153 2023-10-18T08:54:11.734 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el9.noa 30/153 2023-10-18T08:54:11.734 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el9.n 31/153 2023-10-18T08:54:11.734 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 32/153 2023-10-18T08:54:11.735 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 33/153 2023-10-18T08:54:11.735 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el9.noarch 34/153 2023-10-18T08:54:11.735 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 35/153 2023-10-18T08:54:11.735 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-volume-2:18.0.0-6786.gf7420d7f.el9.noarch 36/153 2023-10-18T08:54:11.735 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 37/153 2023-10-18T08:54:11.736 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-top-2:18.0.0-6786.gf7420d7f.el9.noarch 38/153 2023-10-18T08:54:11.736 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/153 2023-10-18T08:54:11.736 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/153 2023-10-18T08:54:11.736 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/153 2023-10-18T08:54:11.736 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/153 2023-10-18T08:54:11.737 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/153 2023-10-18T08:54:11.737 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/153 2023-10-18T08:54:11.737 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/153 2023-10-18T08:54:11.737 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/153 2023-10-18T08:54:11.737 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/153 2023-10-18T08:54:11.737 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/153 2023-10-18T08:54:11.738 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/153 2023-10-18T08:54:11.738 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/153 2023-10-18T08:54:11.738 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/153 2023-10-18T08:54:11.738 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/153 2023-10-18T08:54:11.738 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/153 2023-10-18T08:54:11.739 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/153 2023-10-18T08:54:11.739 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/153 2023-10-18T08:54:11.739 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/153 2023-10-18T08:54:11.739 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/153 2023-10-18T08:54:11.739 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/153 2023-10-18T08:54:11.739 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/153 2023-10-18T08:54:11.740 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/153 2023-10-18T08:54:11.740 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/153 2023-10-18T08:54:11.740 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/153 2023-10-18T08:54:11.740 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/153 2023-10-18T08:54:11.740 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/153 2023-10-18T08:54:11.741 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/153 2023-10-18T08:54:11.741 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/153 2023-10-18T08:54:11.741 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/153 2023-10-18T08:54:11.741 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/153 2023-10-18T08:54:11.741 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/153 2023-10-18T08:54:11.741 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/153 2023-10-18T08:54:11.742 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/153 2023-10-18T08:54:11.742 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/153 2023-10-18T08:54:11.742 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/153 2023-10-18T08:54:11.742 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/153 2023-10-18T08:54:11.742 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/153 2023-10-18T08:54:11.743 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/153 2023-10-18T08:54:11.743 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/153 2023-10-18T08:54:11.743 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 78/153 2023-10-18T08:54:11.743 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 79/153 2023-10-18T08:54:11.743 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 80/153 2023-10-18T08:54:11.744 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ply-3.11-14.el9.noarch 81/153 2023-10-18T08:54:11.744 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 82/153 2023-10-18T08:54:11.744 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 83/153 2023-10-18T08:54:11.744 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 84/153 2023-10-18T08:54:11.744 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/153 2023-10-18T08:54:11.745 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/153 2023-10-18T08:54:11.745 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/153 2023-10-18T08:54:11.745 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : jq-1.6-15.el9.x86_64 88/153 2023-10-18T08:54:11.745 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/153 2023-10-18T08:54:11.745 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/153 2023-10-18T08:54:11.745 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/153 2023-10-18T08:54:11.746 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/153 2023-10-18T08:54:11.746 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/153 2023-10-18T08:54:11.746 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/153 2023-10-18T08:54:11.746 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/153 2023-10-18T08:54:11.746 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/153 2023-10-18T08:54:11.747 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/153 2023-10-18T08:54:11.747 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/153 2023-10-18T08:54:11.747 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/153 2023-10-18T08:54:11.747 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/153 2023-10-18T08:54:11.747 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/153 2023-10-18T08:54:11.748 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/153 2023-10-18T08:54:11.748 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/153 2023-10-18T08:54:11.748 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/153 2023-10-18T08:54:11.748 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/153 2023-10-18T08:54:11.748 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/153 2023-10-18T08:54:11.749 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/153 2023-10-18T08:54:11.749 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-google-auth-1:2.23.2-1.el9.noarch 108/153 2023-10-18T08:54:11.749 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/153 2023-10-18T08:54:11.749 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/153 2023-10-18T08:54:11.749 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/153 2023-10-18T08:54:11.749 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/153 2023-10-18T08:54:11.750 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/153 2023-10-18T08:54:11.750 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/153 2023-10-18T08:54:11.750 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/153 2023-10-18T08:54:11.750 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/153 2023-10-18T08:54:11.750 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/153 2023-10-18T08:54:11.751 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/153 2023-10-18T08:54:11.751 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/153 2023-10-18T08:54:11.751 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/153 2023-10-18T08:54:11.751 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/153 2023-10-18T08:54:11.751 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/153 2023-10-18T08:54:11.751 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/153 2023-10-18T08:54:11.752 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/153 2023-10-18T08:54:11.752 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/153 2023-10-18T08:54:11.752 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/153 2023-10-18T08:54:11.752 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/153 2023-10-18T08:54:11.753 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/153 2023-10-18T08:54:11.753 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/153 2023-10-18T08:54:11.753 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/153 2023-10-18T08:54:11.753 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/153 2023-10-18T08:54:11.753 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/153 2023-10-18T08:54:11.753 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/153 2023-10-18T08:54:11.754 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:18.0.0-6786.gf7420d7f.el9.x86_64 134/153 2023-10-18T08:54:11.754 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/153 2023-10-18T08:54:11.754 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:18.0.0-6786.gf7420d7f.el9.x86_64 136/153 2023-10-18T08:54:11.754 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/153 2023-10-18T08:54:11.754 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/153 2023-10-18T08:54:11.755 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/153 2023-10-18T08:54:11.755 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.45.4-1.el9.x86_64 140/153 2023-10-18T08:54:11.755 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 141/153 2023-10-18T08:54:11.755 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 142/153 2023-10-18T08:54:11.755 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 143/153 2023-10-18T08:54:11.755 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.45.4-1.el9.x86_64 144/153 2023-10-18T08:54:11.756 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 145/153 2023-10-18T08:54:11.756 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.45.4-1.el9.x86_64 146/153 2023-10-18T08:54:11.756 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 147/153 2023-10-18T08:54:11.756 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : acl-2.3.1-4.el9.x86_64 148/153 2023-10-18T08:54:11.756 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : acl-2.3.1-3.el9.x86_64 149/153 2023-10-18T08:54:11.757 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/153 2023-10-18T08:54:11.757 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-18T08:54:11.757 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/153 2023-10-18T08:54:11.759 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el9.noarch 118/153 2023-10-18T08:54:11.800 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el9.noarch 118/153 2023-10-18T08:54:12.774 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-2:18.0.0-6786.gf7420d7f.el9.x86_64 119/153 2023-10-18T08:54:12.816 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 120/153 2023-10-18T08:54:12.903 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 120/153 2023-10-18T08:54:12.903 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:54:12.903 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-18T08:54:12.904 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-18T08:54:12.904 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-18T08:54:12.904 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:12.958 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 121/153 2023-10-18T08:54:13.037 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6786.gf7420 121/153 2023-10-18T08:54:13.037 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:54:13.038 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-18T08:54:13.038 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:13.109 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 122/153 2023-10-18T08:54:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 122/153 2023-10-18T08:54:13.193 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:54:13.194 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-10-18T08:54:13.194 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-18T08:54:13.194 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-18T08:54:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:13.585 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 123/153 2023-10-18T08:54:13.664 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 123/153 2023-10-18T08:54:13.664 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-18T08:54:13.665 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-18T08:54:13.665 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-18T08:54:13.665 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-18T08:54:13.665 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-18T08:54:13.823 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:13.824 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2023-10-18T08:54:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-1:1.45.4-1.el9.x86_64 2023-10-18T08:54:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 2023-10-18T08:54:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-team-1:1.45.4-1.el9.x86_64 2023-10-18T08:54:13.824 INFO:teuthology.orchestra.run.smithi144.stdout: NetworkManager-tui-1:1.45.4-1.el9.x86_64 2023-10-18T08:54:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: acl-2.3.1-4.el9.x86_64 2023-10-18T08:54:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-2.3.1-4.el9.x86_64 2023-10-18T08:54:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-10-18T08:54:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: librados2-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.825 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-10-18T08:54:13.826 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2023-10-18T08:54:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: automake-1.16.2-8.el9.noarch 2023-10-18T08:54:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: bison-3.7.4-5.el9.x86_64 2023-10-18T08:54:13.826 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-18T08:54:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:13.827 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:13.828 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:13.829 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:13.830 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-10-18T08:54:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: flex-2.6.4-9.el9.x86_64 2023-10-18T08:54:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-18T08:54:13.831 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-18T08:54:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-18T08:54:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-18T08:54:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: fuse-2.9.9-15.el9.x86_64 2023-10-18T08:54:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-18T08:54:13.832 INFO:teuthology.orchestra.run.smithi144.stdout: jq-1.6-15.el9.x86_64 2023-10-18T08:54:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-10-18T08:54:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-18T08:54:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-18T08:54:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.833 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-18T08:54:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-18T08:54:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-18T08:54:13.834 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-18T08:54:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-18T08:54:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-18T08:54:13.835 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: libtool-2.4.6-45.el9.x86_64 2023-10-18T08:54:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-18T08:54:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-10-18T08:54:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-10-18T08:54:13.836 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-18T08:54:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-18T08:54:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-18T08:54:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-18T08:54:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-10-18T08:54:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-18T08:54:13.837 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-13.11-1.el9.x86_64 2023-10-18T08:54:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-10-18T08:54:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-10-18T08:54:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server-13.11-1.el9.x86_64 2023-10-18T08:54:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-18T08:54:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-18T08:54:13.838 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-18T08:54:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-18T08:54:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.839 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-18T08:54:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-18T08:54:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-18T08:54:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-18T08:54:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-18T08:54:13.840 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-18T08:54:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth-1:2.23.2-1.el9.noarch 2023-10-18T08:54:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-18T08:54:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-18T08:54:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-18T08:54:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-18T08:54:13.841 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-18T08:54:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-18T08:54:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-18T08:54:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-18T08:54:13.842 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-18T08:54:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-18T08:54:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-18T08:54:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-18T08:54:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-18T08:54:13.843 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-18T08:54:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-18T08:54:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-18T08:54:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-18T08:54:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-18T08:54:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-18T08:54:13.844 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-18T08:54:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply-3.11-14.el9.noarch 2023-10-18T08:54:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-18T08:54:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-18T08:54:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-18T08:54:13.845 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-18T08:54:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-18T08:54:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-18T08:54:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-18T08:54:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.846 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-18T08:54:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-18T08:54:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-18T08:54:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.847 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-18T08:54:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-18T08:54:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-18T08:54:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-18T08:54:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-18T08:54:13.848 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-18T08:54:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-10-18T08:54:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-18T08:54:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-18T08:54:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-18T08:54:13.849 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-18T08:54:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: re2-1:20211101-3.el9.x86_64 2023-10-18T08:54:13.850 INFO:teuthology.orchestra.run.smithi144.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-18T08:54:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-18T08:54:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: uuid-1.6.2-55.el9.x86_64 2023-10-18T08:54:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-10-18T08:54:13.851 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-18T08:54:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-10-18T08:54:13.852 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:13.852 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2023-10-18T08:54:14.338 DEBUG:teuthology.parallel:result is None 2023-10-18T08:54:18.041 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-test-2:18.0.0-6786.gf7420d7f.el9.x86_64 124/153 2023-10-18T08:54:18.096 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 125/153 2023-10-18T08:54:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-nbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 126/153 2023-10-18T08:54:18.435 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 127/153 2023-10-18T08:54:18.559 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 128/153 2023-10-18T08:54:18.626 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 129/153 2023-10-18T08:54:18.674 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/153 2023-10-18T08:54:18.711 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/153 2023-10-18T08:54:18.741 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el9.x86_ 132/153 2023-10-18T08:54:18.779 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : NetworkManager-team-1:1.45.4-1.el9.x86_64 133/153 2023-10-18T08:54:18.894 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : NetworkManager-tui-1:1.45.4-1.el9.x86_64 134/153 2023-10-18T08:54:18.970 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 135/153 2023-10-18T08:54:19.012 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 136/153 2023-10-18T08:54:19.931 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-18T08:54:19.976 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-18T08:54:20.038 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 137/153 2023-10-18T08:54:20.125 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephfs-top-2:18.0.0-6786.gf7420d7f.el9.noarch 138/153 2023-10-18T08:54:21.617 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 139/153 2023-10-18T08:54:21.708 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 140/153 2023-10-18T08:54:21.756 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 141/153 2023-10-18T08:54:21.903 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flex-2.6.4-9.el9.x86_64 142/153 2023-10-18T08:54:21.944 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : bison-3.7.4-5.el9.x86_64 143/153 2023-10-18T08:54:21.981 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 144/153 2023-10-18T08:54:21.981 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-18T08:54:22.109 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 145/153 2023-10-18T08:54:22.148 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 146/153 2023-10-18T08:54:22.180 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 147/153 2023-10-18T08:54:22.213 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 148/153 2023-10-18T08:54:22.253 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-18T08:54:22.253 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-18T08:54:22.596 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 149/153 2023-10-18T08:54:22.597 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-18T08:54:22.777 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 150/153 2023-10-18T08:54:22.808 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-18T08:54:22.809 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-18T08:54:22.947 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/153 2023-10-18T08:54:23.073 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-18T08:54:23.327 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-18T08:54:23.328 INFO:teuthology.orchestra.run.smithi040.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-18T08:54:23.328 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:28.033 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-2:18.0.0-6786.gf7420d7f.el9.x86_64 1/153 2023-10-18T08:54:28.034 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-base-2:18.0.0-6786.gf7420d7f.el9.x86_64 2/153 2023-10-18T08:54:28.034 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 3/153 2023-10-18T08:54:28.034 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 4/153 2023-10-18T08:54:28.035 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 5/153 2023-10-18T08:54:28.035 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mds-2:18.0.0-6786.gf7420d7f.el9.x86_64 6/153 2023-10-18T08:54:28.035 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-2:18.0.0-6786.gf7420d7f.el9.x86_64 7/153 2023-10-18T08:54:28.035 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mon-2:18.0.0-6786.gf7420d7f.el9.x86_64 8/153 2023-10-18T08:54:28.035 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-osd-2:18.0.0-6786.gf7420d7f.el9.x86_64 9/153 2023-10-18T08:54:28.036 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 10/153 2023-10-18T08:54:28.036 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-selinux-2:18.0.0-6786.gf7420d7f.el9.x86_64 11/153 2023-10-18T08:54:28.036 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-test-2:18.0.0-6786.gf7420d7f.el9.x86_64 12/153 2023-10-18T08:54:28.036 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 13/153 2023-10-18T08:54:28.036 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el9.x86_ 14/153 2023-10-18T08:54:28.037 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs2-2:18.0.0-6786.gf7420d7f.el9.x86_64 15/153 2023-10-18T08:54:28.037 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephsqlite-2:18.0.0-6786.gf7420d7f.el9.x86_64 16/153 2023-10-18T08:54:28.037 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados-devel-2:18.0.0-6786.gf7420d7f.el9.x86_6 17/153 2023-10-18T08:54:28.037 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libradosstriper1-2:18.0.0-6786.gf7420d7f.el9.x86 18/153 2023-10-18T08:54:28.037 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librgw2-2:18.0.0-6786.gf7420d7f.el9.x86_64 19/153 2023-10-18T08:54:28.037 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 20/153 2023-10-18T08:54:28.038 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el9. 21/153 2023-10-18T08:54:28.038 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cephfs-2:18.0.0-6786.gf7420d7f.el9.x86_6 22/153 2023-10-18T08:54:28.038 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rados-2:18.0.0-6786.gf7420d7f.el9.x86_64 23/153 2023-10-18T08:54:28.038 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 24/153 2023-10-18T08:54:28.038 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 25/153 2023-10-18T08:54:28.039 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 26/153 2023-10-18T08:54:28.039 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 27/153 2023-10-18T08:54:28.049 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-nbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 28/153 2023-10-18T08:54:28.049 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 29/153 2023-10-18T08:54:28.050 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el9.noa 30/153 2023-10-18T08:54:28.050 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el9.n 31/153 2023-10-18T08:54:28.050 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 32/153 2023-10-18T08:54:28.050 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 33/153 2023-10-18T08:54:28.050 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el9.noarch 34/153 2023-10-18T08:54:28.051 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 35/153 2023-10-18T08:54:28.051 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-volume-2:18.0.0-6786.gf7420d7f.el9.noarch 36/153 2023-10-18T08:54:28.051 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 37/153 2023-10-18T08:54:28.051 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephfs-top-2:18.0.0-6786.gf7420d7f.el9.noarch 38/153 2023-10-18T08:54:28.051 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/153 2023-10-18T08:54:28.052 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/153 2023-10-18T08:54:28.052 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/153 2023-10-18T08:54:28.052 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/153 2023-10-18T08:54:28.052 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/153 2023-10-18T08:54:28.052 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/153 2023-10-18T08:54:28.052 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/153 2023-10-18T08:54:28.053 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/153 2023-10-18T08:54:28.053 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/153 2023-10-18T08:54:28.053 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/153 2023-10-18T08:54:28.053 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/153 2023-10-18T08:54:28.053 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/153 2023-10-18T08:54:28.054 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/153 2023-10-18T08:54:28.054 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/153 2023-10-18T08:54:28.054 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/153 2023-10-18T08:54:28.054 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/153 2023-10-18T08:54:28.054 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/153 2023-10-18T08:54:28.055 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/153 2023-10-18T08:54:28.055 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/153 2023-10-18T08:54:28.055 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/153 2023-10-18T08:54:28.055 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/153 2023-10-18T08:54:28.055 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/153 2023-10-18T08:54:28.056 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/153 2023-10-18T08:54:28.056 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/153 2023-10-18T08:54:28.056 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/153 2023-10-18T08:54:28.056 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/153 2023-10-18T08:54:28.056 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/153 2023-10-18T08:54:28.056 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/153 2023-10-18T08:54:28.057 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/153 2023-10-18T08:54:28.057 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/153 2023-10-18T08:54:28.057 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/153 2023-10-18T08:54:28.057 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/153 2023-10-18T08:54:28.057 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/153 2023-10-18T08:54:28.058 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/153 2023-10-18T08:54:28.058 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/153 2023-10-18T08:54:28.058 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/153 2023-10-18T08:54:28.058 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/153 2023-10-18T08:54:28.058 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/153 2023-10-18T08:54:28.059 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/153 2023-10-18T08:54:28.059 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 78/153 2023-10-18T08:54:28.059 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 79/153 2023-10-18T08:54:28.059 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 80/153 2023-10-18T08:54:28.059 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ply-3.11-14.el9.noarch 81/153 2023-10-18T08:54:28.059 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 82/153 2023-10-18T08:54:28.060 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 83/153 2023-10-18T08:54:28.060 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 84/153 2023-10-18T08:54:28.060 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/153 2023-10-18T08:54:28.060 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/153 2023-10-18T08:54:28.060 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/153 2023-10-18T08:54:28.061 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : jq-1.6-15.el9.x86_64 88/153 2023-10-18T08:54:28.061 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/153 2023-10-18T08:54:28.061 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/153 2023-10-18T08:54:28.061 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/153 2023-10-18T08:54:28.061 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/153 2023-10-18T08:54:28.062 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/153 2023-10-18T08:54:28.062 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/153 2023-10-18T08:54:28.062 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/153 2023-10-18T08:54:28.062 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/153 2023-10-18T08:54:28.062 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/153 2023-10-18T08:54:28.063 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/153 2023-10-18T08:54:28.063 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/153 2023-10-18T08:54:28.063 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/153 2023-10-18T08:54:28.063 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/153 2023-10-18T08:54:28.063 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/153 2023-10-18T08:54:28.063 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/153 2023-10-18T08:54:28.064 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/153 2023-10-18T08:54:28.064 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/153 2023-10-18T08:54:28.064 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/153 2023-10-18T08:54:28.064 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/153 2023-10-18T08:54:28.064 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-google-auth-1:2.23.2-1.el9.noarch 108/153 2023-10-18T08:54:28.065 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/153 2023-10-18T08:54:28.065 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/153 2023-10-18T08:54:28.065 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/153 2023-10-18T08:54:28.065 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/153 2023-10-18T08:54:28.065 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/153 2023-10-18T08:54:28.066 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/153 2023-10-18T08:54:28.066 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/153 2023-10-18T08:54:28.066 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/153 2023-10-18T08:54:28.066 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/153 2023-10-18T08:54:28.066 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/153 2023-10-18T08:54:28.067 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/153 2023-10-18T08:54:28.067 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/153 2023-10-18T08:54:28.067 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/153 2023-10-18T08:54:28.067 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/153 2023-10-18T08:54:28.067 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/153 2023-10-18T08:54:28.067 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/153 2023-10-18T08:54:28.068 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/153 2023-10-18T08:54:28.068 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/153 2023-10-18T08:54:28.068 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/153 2023-10-18T08:54:28.068 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/153 2023-10-18T08:54:28.068 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/153 2023-10-18T08:54:28.069 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/153 2023-10-18T08:54:28.069 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/153 2023-10-18T08:54:28.069 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/153 2023-10-18T08:54:28.069 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/153 2023-10-18T08:54:28.069 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:18.0.0-6786.gf7420d7f.el9.x86_64 134/153 2023-10-18T08:54:28.070 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/153 2023-10-18T08:54:28.070 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:18.0.0-6786.gf7420d7f.el9.x86_64 136/153 2023-10-18T08:54:28.070 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/153 2023-10-18T08:54:28.070 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 138/153 2023-10-18T08:54:28.070 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 139/153 2023-10-18T08:54:28.071 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-1:1.45.4-1.el9.x86_64 140/153 2023-10-18T08:54:28.071 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 141/153 2023-10-18T08:54:28.071 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 142/153 2023-10-18T08:54:28.071 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 143/153 2023-10-18T08:54:28.071 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-team-1:1.45.4-1.el9.x86_64 144/153 2023-10-18T08:54:28.071 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 145/153 2023-10-18T08:54:28.072 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-tui-1:1.45.4-1.el9.x86_64 146/153 2023-10-18T08:54:28.072 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 147/153 2023-10-18T08:54:28.072 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : acl-2.3.1-4.el9.x86_64 148/153 2023-10-18T08:54:28.072 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : acl-2.3.1-3.el9.x86_64 149/153 2023-10-18T08:54:28.072 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 150/153 2023-10-18T08:54:28.073 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 151/153 2023-10-18T08:54:28.073 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 152/153 2023-10-18T08:54:30.429 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 153/153 2023-10-18T08:54:30.429 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:30.429 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2023-10-18T08:54:30.430 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-1:1.45.4-1.el9.x86_64 2023-10-18T08:54:30.430 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 2023-10-18T08:54:30.430 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-team-1:1.45.4-1.el9.x86_64 2023-10-18T08:54:30.430 INFO:teuthology.orchestra.run.smithi040.stdout: NetworkManager-tui-1:1.45.4-1.el9.x86_64 2023-10-18T08:54:30.430 INFO:teuthology.orchestra.run.smithi040.stdout: acl-2.3.1-4.el9.x86_64 2023-10-18T08:54:30.431 INFO:teuthology.orchestra.run.smithi040.stdout: libacl-2.3.1-4.el9.x86_64 2023-10-18T08:54:30.431 INFO:teuthology.orchestra.run.smithi040.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-10-18T08:54:30.431 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.431 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.431 INFO:teuthology.orchestra.run.smithi040.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-10-18T08:54:30.432 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2023-10-18T08:54:30.432 INFO:teuthology.orchestra.run.smithi040.stdout: automake-1.16.2-8.el9.noarch 2023-10-18T08:54:30.432 INFO:teuthology.orchestra.run.smithi040.stdout: bison-3.7.4-5.el9.x86_64 2023-10-18T08:54:30.432 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-18T08:54:30.432 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.432 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.433 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.433 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.433 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:30.433 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.433 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.434 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.434 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:30.434 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:30.434 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:30.434 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:30.435 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:30.435 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.435 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.435 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:30.435 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.435 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.436 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.436 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:30.436 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:30.436 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.436 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-top-2:18.0.0-6786.gf7420d7f.el9.noarch 2023-10-18T08:54:30.436 INFO:teuthology.orchestra.run.smithi040.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-10-18T08:54:30.437 INFO:teuthology.orchestra.run.smithi040.stdout: flex-2.6.4-9.el9.x86_64 2023-10-18T08:54:30.437 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-18T08:54:30.437 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-18T08:54:30.437 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-18T08:54:30.437 INFO:teuthology.orchestra.run.smithi040.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-18T08:54:30.438 INFO:teuthology.orchestra.run.smithi040.stdout: fuse-2.9.9-15.el9.x86_64 2023-10-18T08:54:30.438 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-18T08:54:30.438 INFO:teuthology.orchestra.run.smithi040.stdout: jq-1.6-15.el9.x86_64 2023-10-18T08:54:30.438 INFO:teuthology.orchestra.run.smithi040.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-10-18T08:54:30.438 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-18T08:54:30.438 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-18T08:54:30.439 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.439 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.439 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.439 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-18T08:54:30.439 INFO:teuthology.orchestra.run.smithi040.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-18T08:54:30.439 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-18T08:54:30.440 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-18T08:54:30.440 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-18T08:54:30.440 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.440 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.440 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-18T08:54:30.441 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.441 INFO:teuthology.orchestra.run.smithi040.stdout: libtool-2.4.6-45.el9.x86_64 2023-10-18T08:54:30.441 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-18T08:54:30.441 INFO:teuthology.orchestra.run.smithi040.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-10-18T08:54:30.441 INFO:teuthology.orchestra.run.smithi040.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-10-18T08:54:30.441 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-18T08:54:30.442 INFO:teuthology.orchestra.run.smithi040.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-18T08:54:30.442 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-18T08:54:30.442 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-18T08:54:30.442 INFO:teuthology.orchestra.run.smithi040.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-10-18T08:54:30.442 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-18T08:54:30.443 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-13.11-1.el9.x86_64 2023-10-18T08:54:30.443 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-10-18T08:54:30.443 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-10-18T08:54:30.443 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-server-13.11-1.el9.x86_64 2023-10-18T08:54:30.443 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-18T08:54:30.443 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-18T08:54:30.444 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-18T08:54:30.444 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-18T08:54:30.444 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.444 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.444 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-18T08:54:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-18T08:54:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-18T08:54:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-18T08:54:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-18T08:54:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-18T08:54:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth-1:2.23.2-1.el9.noarch 2023-10-18T08:54:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-18T08:54:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-18T08:54:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-18T08:54:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-18T08:54:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-18T08:54:30.447 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-18T08:54:30.447 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-18T08:54:30.447 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-18T08:54:30.447 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-18T08:54:30.447 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-18T08:54:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-18T08:54:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-18T08:54:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-18T08:54:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-18T08:54:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-18T08:54:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-18T08:54:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-18T08:54:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-18T08:54:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-18T08:54:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-18T08:54:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply-3.11-14.el9.noarch 2023-10-18T08:54:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-18T08:54:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-18T08:54:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-18T08:54:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-18T08:54:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-18T08:54:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-18T08:54:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-18T08:54:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-18T08:54:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-18T08:54:30.452 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-18T08:54:30.452 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.452 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-18T08:54:30.452 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-18T08:54:30.452 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-18T08:54:30.452 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-18T08:54:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-18T08:54:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-18T08:54:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-10-18T08:54:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-18T08:54:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-18T08:54:30.454 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-18T08:54:30.454 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-18T08:54:30.454 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.454 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.454 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-2:18.0.0-6786.gf7420d7f.el9.x86_64 2023-10-18T08:54:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: re2-1:20211101-3.el9.x86_64 2023-10-18T08:54:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-18T08:54:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-18T08:54:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: uuid-1.6.2-55.el9.x86_64 2023-10-18T08:54:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-10-18T08:54:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-18T08:54:30.456 INFO:teuthology.orchestra.run.smithi040.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-10-18T08:54:30.456 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:30.456 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2023-10-18T08:54:30.908 DEBUG:teuthology.parallel:result is None 2023-10-18T08:54:30.909 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T08:54:31.052 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-18T08:54:31.124 INFO:teuthology.orchestra.run.smithi040.stdout:18.0.0-6786.gf7420d7f.el9 2023-10-18T08:54:31.125 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6786.gf7420d7f.el9 2023-10-18T08:54:31.126 INFO:teuthology.task.install:The correct ceph version 18.0.0-6786.gf7420d7f is installed. 2023-10-18T08:54:31.126 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T08:54:31.268 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-18T08:54:31.344 INFO:teuthology.orchestra.run.smithi144.stdout:18.0.0-6786.gf7420d7f.el9 2023-10-18T08:54:31.345 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6786.gf7420d7f.el9 2023-10-18T08:54:31.345 INFO:teuthology.task.install:The correct ceph version 18.0.0-6786.gf7420d7f is installed. 2023-10-18T08:54:31.345 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-18T08:54:31.346 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:31.346 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-18T08:54:31.432 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:31.432 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-18T08:54:31.542 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-18T08:54:31.542 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:31.543 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2023-10-18T08:54:31.628 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-18T08:54:31.746 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:31.747 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2023-10-18T08:54:31.840 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-18T08:54:31.963 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-18T08:54:31.964 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:31.964 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-18T08:54:32.046 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-18T08:54:32.164 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:32.164 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-18T08:54:32.257 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-18T08:54:32.381 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-18T08:54:32.382 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:32.382 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2023-10-18T08:54:32.503 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-18T08:54:32.581 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:32.582 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2023-10-18T08:54:32.671 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-18T08:54:32.756 INFO:teuthology.run_tasks:Running task ceph... 2023-10-18T08:54:32.821 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-18T08:54:32.822 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /var/log/ceph 2023-10-18T08:54:32.828 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /var/log/ceph 2023-10-18T08:54:32.909 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-18T08:54:32.910 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-18T08:54:32.915 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-18T08:54:32.994 INFO:tasks.ceph:Creating extra log directories... 2023-10-18T08:54:32.995 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-18T08:54:32.999 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-18T08:54:33.086 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-18T08:54:33.087 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-18T08:54:33.087 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-18T08:54:33.087 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:33.088 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-18T08:54:33.131 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-18T08:54:33.359 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-18T08:54:33.435 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:33.435 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-18T08:54:33.490 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-18T08:54:33.760 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-18T08:54:33.870 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-18T08:54:33.870 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-18T08:54:33.871 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719', 'branch': 'wip-batrick-testing-20231016.203825', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7430719', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-batrick-testing-20231016.203825'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-batrick-testing-20231016.203825', 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7703, 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'sleep_before_teardown': 0, 'subset': '153/256', 'suite': 'fs', 'suite_branch': 'wip-batrick-testing-20231016.203825', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi040.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPru1b2IMuAaQvQ9WGQJurxzltIS6eqOeara4xXFpywLI6lhSRJD685TuJEGHY9LjnxtK5bfzffsA2ujB/Fq6ZE=', 'smithi144.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDKHTBvPWMvPC4sZtXUB9Xz9kdZblgeqYUCDaBSavudWyKwn8Od7f360DtIrNn+UGrWbBln0nEK2o6UC5gDSAS4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_full']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '8cdab074dcca9a68965bc5a50e9c30b691949723', 'timestamp': '2023-10-16_23:59:58', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-18T08:54:33.872 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-18T08:54:33.878 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-18T08:54:33.935 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/run/ceph 2023-10-18T08:54:33.967 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/run/ceph 2023-10-18T08:54:34.071 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:34.072 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2023-10-18T08:54:34.116 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-18T08:54:34.117 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2023-10-18T08:54:34.202 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-18T08:54:34.202 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T08:54:34.203 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 631 Links: 1 2023-10-18T08:54:34.203 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T08:54:34.203 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T08:54:34.203 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-10-18 08:54:23.824707091 +0000 2023-10-18T08:54:34.203 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-10-18 08:51:46.911785319 +0000 2023-10-18T08:54:34.204 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-10-18 08:51:46.911785319 +0000 2023-10-18T08:54:34.204 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2023-10-18 08:51:46.911785319 +0000 2023-10-18T08:54:34.204 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-18T08:54:34.315 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-10-18T08:54:34.315 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-10-18T08:54:34.316 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000348102 s, 1.5 MB/s 2023-10-18T08:54:34.318 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-18T08:54:34.362 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2023-10-18T08:54:34.444 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-18T08:54:34.445 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T08:54:34.445 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 652 Links: 1 2023-10-18T08:54:34.445 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T08:54:34.446 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T08:54:34.446 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-10-18 08:54:23.825707079 +0000 2023-10-18T08:54:34.446 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-10-18 08:51:48.000771738 +0000 2023-10-18T08:54:34.446 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-10-18 08:51:48.000771738 +0000 2023-10-18T08:54:34.447 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2023-10-18 08:51:48.000771738 +0000 2023-10-18T08:54:34.448 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-18T08:54:34.573 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-10-18T08:54:34.573 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-10-18T08:54:34.574 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000378497 s, 1.4 MB/s 2023-10-18T08:54:34.577 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-18T08:54:34.639 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2023-10-18T08:54:34.696 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-18T08:54:34.696 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T08:54:34.696 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 672 Links: 1 2023-10-18T08:54:34.696 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T08:54:34.697 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T08:54:34.697 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-10-18 08:54:23.826707067 +0000 2023-10-18T08:54:34.697 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-10-18 08:51:49.033758855 +0000 2023-10-18T08:54:34.697 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-10-18 08:51:49.033758855 +0000 2023-10-18T08:54:34.697 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2023-10-18 08:51:49.033758855 +0000 2023-10-18T08:54:34.698 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-18T08:54:34.796 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-10-18T08:54:34.796 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-10-18T08:54:34.797 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000364033 s, 1.4 MB/s 2023-10-18T08:54:34.800 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-18T08:54:34.857 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2023-10-18T08:54:34.910 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-18T08:54:34.910 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T08:54:34.910 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 688 Links: 1 2023-10-18T08:54:34.910 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T08:54:34.911 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T08:54:34.911 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-10-18 08:54:23.826707067 +0000 2023-10-18T08:54:34.911 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-10-18 08:51:50.120745300 +0000 2023-10-18T08:54:34.911 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-10-18 08:51:50.120745300 +0000 2023-10-18T08:54:34.912 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2023-10-18 08:51:50.120745300 +0000 2023-10-18T08:54:34.912 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-18T08:54:35.003 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-10-18T08:54:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-10-18T08:54:35.004 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000332672 s, 1.5 MB/s 2023-10-18T08:54:35.007 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-18T08:54:35.069 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T08:54:35.069 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:35.069 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2023-10-18T08:54:35.130 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-18T08:54:35.131 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2023-10-18T08:54:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-18T08:54:35.185 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T08:54:35.185 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 632 Links: 1 2023-10-18T08:54:35.185 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T08:54:35.185 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T08:54:35.185 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2023-10-18 08:54:07.593573461 +0000 2023-10-18T08:54:35.186 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2023-10-18 08:51:20.464787684 +0000 2023-10-18T08:54:35.186 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2023-10-18 08:51:20.464787684 +0000 2023-10-18T08:54:35.186 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2023-10-18 08:51:20.464787684 +0000 2023-10-18T08:54:35.186 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-18T08:54:35.280 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2023-10-18T08:54:35.280 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2023-10-18T08:54:35.280 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000349765 s, 1.5 MB/s 2023-10-18T08:54:35.284 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-18T08:54:35.345 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2023-10-18T08:54:35.397 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-18T08:54:35.397 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T08:54:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 650 Links: 1 2023-10-18T08:54:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T08:54:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T08:54:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2023-10-18 08:54:07.594573444 +0000 2023-10-18T08:54:35.398 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2023-10-18 08:51:21.463770213 +0000 2023-10-18T08:54:35.399 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2023-10-18 08:51:21.463770213 +0000 2023-10-18T08:54:35.399 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2023-10-18 08:51:21.463770213 +0000 2023-10-18T08:54:35.399 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-18T08:54:35.494 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2023-10-18T08:54:35.494 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2023-10-18T08:54:35.494 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000324785 s, 1.6 MB/s 2023-10-18T08:54:35.498 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-18T08:54:35.556 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2023-10-18T08:54:35.605 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-18T08:54:35.605 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T08:54:35.605 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 670 Links: 1 2023-10-18T08:54:35.605 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T08:54:35.606 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T08:54:35.606 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2023-10-18 08:54:07.595573426 +0000 2023-10-18T08:54:35.606 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2023-10-18 08:51:22.538751413 +0000 2023-10-18T08:54:35.606 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2023-10-18 08:51:22.538751413 +0000 2023-10-18T08:54:35.606 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2023-10-18 08:51:22.538751413 +0000 2023-10-18T08:54:35.607 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-18T08:54:35.691 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2023-10-18T08:54:35.691 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2023-10-18T08:54:35.691 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000316204 s, 1.6 MB/s 2023-10-18T08:54:35.694 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-18T08:54:35.751 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2023-10-18T08:54:35.803 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-18T08:54:35.804 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T08:54:35.804 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 690 Links: 1 2023-10-18T08:54:35.804 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T08:54:35.804 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T08:54:35.804 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2023-10-18 08:54:07.595573426 +0000 2023-10-18T08:54:35.805 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2023-10-18 08:51:23.572733330 +0000 2023-10-18T08:54:35.805 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2023-10-18 08:51:23.572733330 +0000 2023-10-18T08:54:35.805 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2023-10-18 08:51:23.572733330 +0000 2023-10-18T08:54:35.805 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-18T08:54:35.890 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2023-10-18T08:54:35.891 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2023-10-18T08:54:35.891 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000348239 s, 1.5 MB/s 2023-10-18T08:54:35.893 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-18T08:54:35.948 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T08:54:35.948 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-18T08:54:35.948 INFO:tasks.ceph:Generating config... 2023-10-18T08:54:35.949 INFO:tasks.ceph:[client] pid file = 2023-10-18T08:54:35.950 INFO:tasks.ceph:[client.0] debug client = 20 2023-10-18T08:54:35.950 INFO:tasks.ceph:[client.0] debug objectcacher = 20 2023-10-18T08:54:35.951 INFO:tasks.ceph:[client.0] debug objecter = 20 2023-10-18T08:54:35.951 INFO:tasks.ceph:[client.1] debug client = 20 2023-10-18T08:54:35.952 INFO:tasks.ceph:[client.1] debug objectcacher = 20 2023-10-18T08:54:35.952 INFO:tasks.ceph:[client.1] debug objecter = 20 2023-10-18T08:54:35.953 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-18T08:54:35.953 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-18T08:54:35.953 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-18T08:54:35.954 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-18T08:54:35.955 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-18T08:54:35.955 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-18T08:54:35.955 INFO:tasks.ceph:[mon] mon osd backfillfull ratio = 0.6 2023-10-18T08:54:35.956 INFO:tasks.ceph:[mon] mon osd full ratio = 0.7 2023-10-18T08:54:35.956 INFO:tasks.ceph:[mon] mon osd nearfull ratio = 0.6 2023-10-18T08:54:35.957 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-18T08:54:35.957 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-18T08:54:35.957 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-10-18T08:54:35.958 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-10-18T08:54:35.958 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-18T08:54:35.958 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-18T08:54:35.959 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-18T08:54:35.959 INFO:tasks.ceph:[osd] memstore device bytes = 200000000 2023-10-18T08:54:35.959 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-18T08:54:35.959 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-18T08:54:35.960 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-18T08:54:35.960 INFO:tasks.ceph:[osd] osd failsafe full ratio = 1.0 2023-10-18T08:54:35.960 INFO:tasks.ceph:[osd] osd mon report interval = 5 2023-10-18T08:54:35.960 INFO:tasks.ceph:[osd] osd objectstore = memstore 2023-10-18T08:54:35.961 INFO:tasks.ceph:Setting up mon.a... 2023-10-18T08:54:35.961 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-18T08:54:36.083 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.keyring 2023-10-18T08:54:36.091 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-18T08:54:36.202 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-18T08:54:36.278 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.40'), ('mon.b', '172.21.15.144'), ('mon.c', '[v2:172.21.15.144:3301,v1:172.21.15.144:6790]')] 2023-10-18T08:54:36.279 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.40,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}} 2023-10-18T08:54:36.280 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:36.280 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-18T08:54:36.328 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.40 --add b 172.21.15.144 --addv c '[v2:172.21.15.144:3301,v1:172.21.15.144:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-18T08:54:36.466 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-18T08:54:36.466 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: generated fsid 029855e3-538a-4478-b323-f54ab0f3a56d 2023-10-18T08:54:36.466 INFO:teuthology.orchestra.run.smithi040.stdout:setting min_mon_release = pacific 2023-10-18T08:54:36.467 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 0 2023-10-18T08:54:36.467 INFO:teuthology.orchestra.run.smithi040.stdout:fsid 029855e3-538a-4478-b323-f54ab0f3a56d 2023-10-18T08:54:36.467 INFO:teuthology.orchestra.run.smithi040.stdout:last_changed 2023-10-18T08:54:36.464506+0000 2023-10-18T08:54:36.467 INFO:teuthology.orchestra.run.smithi040.stdout:created 2023-10-18T08:54:36.464506+0000 2023-10-18T08:54:36.467 INFO:teuthology.orchestra.run.smithi040.stdout:min_mon_release 16 (pacific) 2023-10-18T08:54:36.468 INFO:teuthology.orchestra.run.smithi040.stdout:election_strategy: 1 2023-10-18T08:54:36.468 INFO:teuthology.orchestra.run.smithi040.stdout:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-10-18T08:54:36.468 INFO:teuthology.orchestra.run.smithi040.stdout:1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2023-10-18T08:54:36.468 INFO:teuthology.orchestra.run.smithi040.stdout:2: [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] mon.c 2023-10-18T08:54:36.468 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-18T08:54:36.470 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-18T08:54:36.510 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 029855e3-538a-4478-b323-f54ab0f3a56d... 2023-10-18T08:54:36.511 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-18T08:54:36.558 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-18T08:54:36.719 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2023-10-18T08:54:36.719 INFO:teuthology.orchestra.run.smithi040.stdout: chdir = "" 2023-10-18T08:54:36.719 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-18T08:54:36.720 INFO:teuthology.orchestra.run.smithi040.stdout: auth supported = cephx 2023-10-18T08:54:36.720 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.720 INFO:teuthology.orchestra.run.smithi040.stdout: filestore xattr use omap = true 2023-10-18T08:54:36.720 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.720 INFO:teuthology.orchestra.run.smithi040.stdout: mon clock drift allowed = 1.000 2023-10-18T08:54:36.720 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.721 INFO:teuthology.orchestra.run.smithi040.stdout: osd crush chooseleaf type = 0 2023-10-18T08:54:36.721 INFO:teuthology.orchestra.run.smithi040.stdout: auth debug = true 2023-10-18T08:54:36.721 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.721 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on old message = true 2023-10-18T08:54:36.721 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on bug = true 2023-10-18T08:54:36.722 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.722 INFO:teuthology.orchestra.run.smithi040.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-18T08:54:36.722 INFO:teuthology.orchestra.run.smithi040.stdout: mon pg warn max object skew = 0 2023-10-18T08:54:36.722 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.722 INFO:teuthology.orchestra.run.smithi040.stdout: # disable pg_autoscaler by default for new pools 2023-10-18T08:54:36.723 INFO:teuthology.orchestra.run.smithi040.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-18T08:54:36.723 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.723 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default size = 2 2023-10-18T08:54:36.723 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.723 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow primary affinity = true 2023-10-18T08:54:36.723 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow pg remap = true 2023-10-18T08:54:36.724 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on legacy crush tunables = false 2023-10-18T08:54:36.724 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on crush straw calc version zero = false 2023-10-18T08:54:36.724 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on no sortbitwise = false 2023-10-18T08:54:36.724 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on osd down out interval zero = false 2023-10-18T08:54:36.724 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on too few osds = false 2023-10-18T08:54:36.725 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-18T08:54:36.725 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-18T08:54:36.725 INFO:teuthology.orchestra.run.smithi040.stdout: mon_allow_pool_size_one = true 2023-10-18T08:54:36.725 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.725 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-18T08:54:36.726 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.726 INFO:teuthology.orchestra.run.smithi040.stdout: osd default data pool replay window = 5 2023-10-18T08:54:36.726 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.726 INFO:teuthology.orchestra.run.smithi040.stdout: mon allow pool delete = true 2023-10-18T08:54:36.726 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.727 INFO:teuthology.orchestra.run.smithi040.stdout: mon cluster log file level = debug 2023-10-18T08:54:36.727 INFO:teuthology.orchestra.run.smithi040.stdout: debug asserts on shutdown = true 2023-10-18T08:54:36.727 INFO:teuthology.orchestra.run.smithi040.stdout: mon health detail to clog = false 2023-10-18T08:54:36.727 INFO:teuthology.orchestra.run.smithi040.stdout: mon host = "172.21.15.40,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2023-10-18T08:54:36.727 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on skipped message = False 2023-10-18T08:54:36.728 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = 029855e3-538a-4478-b323-f54ab0f3a56d 2023-10-18T08:54:36.728 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.728 INFO:teuthology.orchestra.run.smithi040.stdout:[osd] 2023-10-18T08:54:36.728 INFO:teuthology.orchestra.run.smithi040.stdout: osd journal size = 100 2023-10-18T08:54:36.728 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.729 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub load threshold = 5.0 2023-10-18T08:54:36.729 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub max interval = 600 2023-10-18T08:54:36.729 INFO:teuthology.orchestra.run.smithi040.stdout: osd mclock profile = high_recovery_ops 2023-10-18T08:54:36.729 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.729 INFO:teuthology.orchestra.run.smithi040.stdout: osd recover clone overlap = true 2023-10-18T08:54:36.729 INFO:teuthology.orchestra.run.smithi040.stdout: osd recovery max chunk = 1048576 2023-10-18T08:54:36.730 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.730 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug shutdown = true 2023-10-18T08:54:36.730 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug op order = true 2023-10-18T08:54:36.730 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify stray on activate = true 2023-10-18T08:54:36.730 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.731 INFO:teuthology.orchestra.run.smithi040.stdout: osd open classes on start = true 2023-10-18T08:54:36.731 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug pg log writeout = true 2023-10-18T08:54:36.731 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.731 INFO:teuthology.orchestra.run.smithi040.stdout: osd deep scrub update digest min age = 30 2023-10-18T08:54:36.732 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.732 INFO:teuthology.orchestra.run.smithi040.stdout: osd map max advance = 10 2023-10-18T08:54:36.732 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.732 INFO:teuthology.orchestra.run.smithi040.stdout: journal zero on create = true 2023-10-18T08:54:36.732 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.733 INFO:teuthology.orchestra.run.smithi040.stdout: filestore ondisk finisher threads = 3 2023-10-18T08:54:36.733 INFO:teuthology.orchestra.run.smithi040.stdout: filestore apply finisher threads = 3 2023-10-18T08:54:36.733 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.733 INFO:teuthology.orchestra.run.smithi040.stdout: bdev debug aio = true 2023-10-18T08:54:36.733 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug misdirected ops = true 2023-10-18T08:54:36.734 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore block size = 96636764160 2023-10-18T08:54:36.734 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore fsck on mount = True 2023-10-18T08:54:36.734 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluefs = 20 2023-10-18T08:54:36.734 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluestore = 20 2023-10-18T08:54:36.735 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-10-18T08:54:36.735 INFO:teuthology.orchestra.run.smithi040.stdout: debug osd = 20 2023-10-18T08:54:36.735 INFO:teuthology.orchestra.run.smithi040.stdout: debug rocksdb = 10 2023-10-18T08:54:36.735 INFO:teuthology.orchestra.run.smithi040.stdout: memstore device bytes = 200000000 2023-10-18T08:54:36.735 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd backfillfull_ratio = 0.85 2023-10-18T08:54:36.736 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd full ratio = 0.9 2023-10-18T08:54:36.736 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd nearfull ratio = 0.8 2023-10-18T08:54:36.736 INFO:teuthology.orchestra.run.smithi040.stdout: osd failsafe full ratio = 1.0 2023-10-18T08:54:36.736 INFO:teuthology.orchestra.run.smithi040.stdout: osd mon report interval = 5 2023-10-18T08:54:36.736 INFO:teuthology.orchestra.run.smithi040.stdout: osd objectstore = memstore 2023-10-18T08:54:36.737 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.737 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr] 2023-10-18T08:54:36.737 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-10-18T08:54:36.737 INFO:teuthology.orchestra.run.smithi040.stdout: debug mgr = 20 2023-10-18T08:54:36.737 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2023-10-18T08:54:36.738 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2023-10-18T08:54:36.738 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2023-10-18T08:54:36.738 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2023-10-18T08:54:36.738 INFO:teuthology.orchestra.run.smithi040.stdout: mgr/telemetry/nag = false 2023-10-18T08:54:36.738 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.739 INFO:teuthology.orchestra.run.smithi040.stdout:[mon] 2023-10-18T08:54:36.739 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-10-18T08:54:36.739 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2023-10-18T08:54:36.739 INFO:teuthology.orchestra.run.smithi040.stdout: debug paxos = 20 2023-10-18T08:54:36.739 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2023-10-18T08:54:36.740 INFO:teuthology.orchestra.run.smithi040.stdout: mon data avail warn = 5 2023-10-18T08:54:36.740 INFO:teuthology.orchestra.run.smithi040.stdout: mon mgr mkfs grace = 240 2023-10-18T08:54:36.740 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2023-10-18T08:54:36.740 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd reporter subtree level = osd 2023-10-18T08:54:36.740 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd prime pg temp = true 2023-10-18T08:54:36.741 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2023-10-18T08:54:36.741 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.741 INFO:teuthology.orchestra.run.smithi040.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-18T08:54:36.741 INFO:teuthology.orchestra.run.smithi040.stdout: auth mon ticket ttl = 660 # 11m 2023-10-18T08:54:36.741 INFO:teuthology.orchestra.run.smithi040.stdout: auth service ticket ttl = 240 # 4m 2023-10-18T08:54:36.742 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.742 INFO:teuthology.orchestra.run.smithi040.stdout: # don't complain about insecure global_id in the test suite 2023-10-18T08:54:36.742 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-18T08:54:36.742 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-18T08:54:36.742 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.743 INFO:teuthology.orchestra.run.smithi040.stdout: # 1m isn't quite enough 2023-10-18T08:54:36.743 INFO:teuthology.orchestra.run.smithi040.stdout: mon_down_mkfs_grace = 2m 2023-10-18T08:54:36.743 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.743 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_filestore_osds = false 2023-10-18T08:54:36.743 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd backfillfull ratio = 0.6 2023-10-18T08:54:36.743 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd full ratio = 0.7 2023-10-18T08:54:36.744 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd nearfull ratio = 0.6 2023-10-18T08:54:36.744 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:54:36.744 INFO:teuthology.orchestra.run.smithi040.stdout:[client] 2023-10-18T08:54:36.744 INFO:teuthology.orchestra.run.smithi040.stdout: rgw cache enabled = true 2023-10-18T08:54:36.744 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable ops log = true 2023-10-18T08:54:36.745 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable usage log = true 2023-10-18T08:54:36.745 INFO:teuthology.orchestra.run.smithi040.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-18T08:54:36.745 INFO:teuthology.orchestra.run.smithi040.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-18T08:54:36.745 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = "" 2023-10-18T08:54:36.745 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.a] 2023-10-18T08:54:36.746 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.b] 2023-10-18T08:54:36.746 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.c] 2023-10-18T08:54:36.746 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2023-10-18T08:54:36.746 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c] 2023-10-18T08:54:36.746 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2023-10-18T08:54:36.747 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d] 2023-10-18T08:54:36.747 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2023-10-18T08:54:36.747 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2023-10-18T08:54:36.747 INFO:teuthology.orchestra.run.smithi040.stdout: debug objectcacher = 20 2023-10-18T08:54:36.747 INFO:teuthology.orchestra.run.smithi040.stdout: debug objecter = 20 2023-10-18T08:54:36.748 INFO:teuthology.orchestra.run.smithi040.stdout:[client.1] 2023-10-18T08:54:36.748 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2023-10-18T08:54:36.748 INFO:teuthology.orchestra.run.smithi040.stdout: debug objectcacher = 20 2023-10-18T08:54:36.748 INFO:teuthology.orchestra.run.smithi040.stdout: debug objecter = 20 2023-10-18T08:54:36.749 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2023-10-18T08:54:36.749 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2023-10-18T08:54:36.749 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-18T08:54:36.749 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2023-10-18T08:54:36.750 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.750 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2023-10-18T08:54:36.750 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.750 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2023-10-18T08:54:36.750 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.751 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2023-10-18T08:54:36.751 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2023-10-18T08:54:36.751 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.751 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2023-10-18T08:54:36.752 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2023-10-18T08:54:36.752 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.752 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-18T08:54:36.752 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2023-10-18T08:54:36.752 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.752 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2023-10-18T08:54:36.753 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-18T08:54:36.753 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.753 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2023-10-18T08:54:36.753 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.753 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2023-10-18T08:54:36.754 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2023-10-18T08:54:36.754 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2023-10-18T08:54:36.754 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2023-10-18T08:54:36.754 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2023-10-18T08:54:36.755 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2023-10-18T08:54:36.755 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2023-10-18T08:54:36.755 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-18T08:54:36.755 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-18T08:54:36.755 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2023-10-18T08:54:36.756 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.756 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-18T08:54:36.756 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.756 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2023-10-18T08:54:36.757 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.757 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2023-10-18T08:54:36.757 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.757 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2023-10-18T08:54:36.757 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2023-10-18T08:54:36.758 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2023-10-18T08:54:36.758 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.40,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2023-10-18T08:54:36.758 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on skipped message = False 2023-10-18T08:54:36.758 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = 029855e3-538a-4478-b323-f54ab0f3a56d 2023-10-18T08:54:36.758 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.759 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2023-10-18T08:54:36.759 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2023-10-18T08:54:36.759 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.759 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2023-10-18T08:54:36.759 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2023-10-18T08:54:36.760 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2023-10-18T08:54:36.760 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.760 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2023-10-18T08:54:36.760 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2023-10-18T08:54:36.760 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.761 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2023-10-18T08:54:36.761 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2023-10-18T08:54:36.761 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2023-10-18T08:54:36.761 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.761 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2023-10-18T08:54:36.762 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2023-10-18T08:54:36.762 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.762 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2023-10-18T08:54:36.762 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.763 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2023-10-18T08:54:36.763 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.763 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2023-10-18T08:54:36.763 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.763 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2023-10-18T08:54:36.764 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2023-10-18T08:54:36.764 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.764 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2023-10-18T08:54:36.764 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2023-10-18T08:54:36.764 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2023-10-18T08:54:36.765 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2023-10-18T08:54:36.765 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 20 2023-10-18T08:54:36.765 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 20 2023-10-18T08:54:36.765 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-10-18T08:54:36.765 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2023-10-18T08:54:36.766 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 10 2023-10-18T08:54:36.766 INFO:teuthology.orchestra.run.smithi144.stdout: memstore device bytes = 200000000 2023-10-18T08:54:36.766 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2023-10-18T08:54:36.766 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2023-10-18T08:54:36.766 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2023-10-18T08:54:36.767 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 1.0 2023-10-18T08:54:36.767 INFO:teuthology.orchestra.run.smithi144.stdout: osd mon report interval = 5 2023-10-18T08:54:36.767 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = memstore 2023-10-18T08:54:36.767 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.767 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2023-10-18T08:54:36.767 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-10-18T08:54:36.768 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2023-10-18T08:54:36.768 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2023-10-18T08:54:36.768 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2023-10-18T08:54:36.768 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2023-10-18T08:54:36.768 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2023-10-18T08:54:36.769 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2023-10-18T08:54:36.769 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.769 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2023-10-18T08:54:36.769 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2023-10-18T08:54:36.770 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2023-10-18T08:54:36.770 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2023-10-18T08:54:36.770 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2023-10-18T08:54:36.770 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2023-10-18T08:54:36.771 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2023-10-18T08:54:36.771 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2023-10-18T08:54:36.771 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2023-10-18T08:54:36.771 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2023-10-18T08:54:36.771 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2023-10-18T08:54:36.771 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.772 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-18T08:54:36.772 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2023-10-18T08:54:36.772 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2023-10-18T08:54:36.772 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.772 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2023-10-18T08:54:36.773 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-18T08:54:36.773 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-18T08:54:36.773 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.773 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2023-10-18T08:54:36.773 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2023-10-18T08:54:36.774 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.774 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2023-10-18T08:54:36.774 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull ratio = 0.6 2023-10-18T08:54:36.774 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.7 2023-10-18T08:54:36.774 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.6 2023-10-18T08:54:36.775 INFO:teuthology.orchestra.run.smithi144.stdout: 2023-10-18T08:54:36.775 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2023-10-18T08:54:36.775 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2023-10-18T08:54:36.775 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2023-10-18T08:54:36.775 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2023-10-18T08:54:36.775 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-18T08:54:36.776 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-18T08:54:36.776 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = "" 2023-10-18T08:54:36.776 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2023-10-18T08:54:36.776 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2023-10-18T08:54:36.776 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2023-10-18T08:54:36.777 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2023-10-18T08:54:36.777 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2023-10-18T08:54:36.777 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2023-10-18T08:54:36.777 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2023-10-18T08:54:36.777 INFO:teuthology.orchestra.run.smithi144.stdout:[client.0] 2023-10-18T08:54:36.778 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2023-10-18T08:54:36.778 INFO:teuthology.orchestra.run.smithi144.stdout: debug objectcacher = 20 2023-10-18T08:54:36.778 INFO:teuthology.orchestra.run.smithi144.stdout: debug objecter = 20 2023-10-18T08:54:36.778 INFO:teuthology.orchestra.run.smithi144.stdout:[client.1] 2023-10-18T08:54:36.778 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2023-10-18T08:54:36.779 INFO:teuthology.orchestra.run.smithi144.stdout: debug objectcacher = 20 2023-10-18T08:54:36.779 INFO:teuthology.orchestra.run.smithi144.stdout: debug objecter = 20 2023-10-18T08:54:36.780 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-18T08:54:36.781 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-18T08:54:36.917 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-18T08:54:36.918 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:36.919 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-18T08:54:36.966 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:36.966 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-18T08:54:37.059 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T08:54:37.060 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:37.061 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-18T08:54:37.061 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-18T08:54:37.204 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:37.205 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-18T08:54:37.307 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T08:54:37.308 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:37.309 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-18T08:54:37.309 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-18T08:54:37.466 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:37.467 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-18T08:54:37.566 INFO:tasks.ceph:Setting up mon nodes... 2023-10-18T08:54:37.567 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-18T08:54:37.567 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-18T08:54:37.779 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-18T08:54:37.790 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-18T08:54:37.990 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-18T08:54:38.001 INFO:tasks.ceph:Setting up mds nodes... 2023-10-18T08:54:38.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-18T08:54:38.202 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-18T08:54:38.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-18T08:54:38.314 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-18T08:54:38.513 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-18T08:54:38.522 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-18T08:54:38.625 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-18T08:54:38.813 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-18T08:54:38.822 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-18T08:54:38.917 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-18T08:54:39.102 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-18T08:54:39.111 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-18T08:54:39.209 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-18T08:54:39.210 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-18T08:54:39.348 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-18T08:54:39.401 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-18T08:54:39.536 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-18T08:54:39.587 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-18T08:54:39.721 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-18T08:54:39.773 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-18T08:54:39.901 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-18T08:54:39.950 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-18T08:54:39.951 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-18T08:54:39.951 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-18T08:54:40.058 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T08:54:40.059 INFO:tasks.ceph:role: osd.0 2023-10-18T08:54:40.059 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T08:54:40.059 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-18T08:54:40.168 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T08:54:40.168 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T08:54:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T08:54:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-18T08:54:40.169 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T08:54:40.169 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2023-10-18T08:54:40.169 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T08:54:40.169 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-18T08:54:40.170 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T08:54:40.170 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T08:54:40.632 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2023-10-18T08:54:40.639 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2023-10-18T08:54:40.639 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-18T08:54:40.823 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-18T08:54:40.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-18T08:54:41.035 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T08:54:41.035 INFO:tasks.ceph:role: osd.1 2023-10-18T08:54:41.036 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T08:54:41.036 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-18T08:54:41.143 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T08:54:41.143 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T08:54:41.144 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T08:54:41.144 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-18T08:54:41.144 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T08:54:41.144 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2023-10-18T08:54:41.144 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T08:54:41.145 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-18T08:54:41.145 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T08:54:41.145 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T08:54:41.612 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2023-10-18T08:54:41.619 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2023-10-18T08:54:41.620 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-18T08:54:41.775 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-18T08:54:41.883 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-18T08:54:41.985 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T08:54:41.985 INFO:tasks.ceph:role: osd.2 2023-10-18T08:54:41.985 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T08:54:41.986 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-18T08:54:42.095 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T08:54:42.095 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T08:54:42.095 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T08:54:42.096 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-18T08:54:42.096 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T08:54:42.096 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2023-10-18T08:54:42.096 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T08:54:42.096 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-18T08:54:42.096 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T08:54:42.097 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T08:54:42.620 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2023-10-18T08:54:42.627 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2023-10-18T08:54:42.627 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-18T08:54:42.833 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-18T08:54:42.947 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-18T08:54:43.049 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-18T08:54:43.050 INFO:tasks.ceph:role: osd.3 2023-10-18T08:54:43.050 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T08:54:43.050 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-18T08:54:43.153 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T08:54:43.153 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T08:54:43.153 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T08:54:43.153 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-18T08:54:43.154 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T08:54:43.154 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2023-10-18T08:54:43.154 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T08:54:43.154 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-18T08:54:43.154 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T08:54:43.155 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T08:54:43.626 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2023-10-18T08:54:43.633 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2023-10-18T08:54:43.634 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-18T08:54:43.781 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-18T08:54:43.887 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T08:54:44.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:44.043+0000 7f22b0dab500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-18T08:54:44.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:44.046+0000 7f22b0dab500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-18T08:54:44.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:44.056+0000 7f22b0dab500 -1 memstore(/var/lib/ceph/osd/ceph-0) /var/lib/ceph/osd/ceph-0 2023-10-18T08:54:44.092 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-18T08:54:44.184 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T08:54:44.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:44.333+0000 7fc28cc20500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-18T08:54:44.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:44.338+0000 7fc28cc20500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-18T08:54:44.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:44.347+0000 7fc28cc20500 -1 memstore(/var/lib/ceph/osd/ceph-1) /var/lib/ceph/osd/ceph-1 2023-10-18T08:54:44.381 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-18T08:54:44.484 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T08:54:44.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:44.636+0000 7f9591d1e500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-18T08:54:44.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:44.637+0000 7f9591d1e500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-18T08:54:44.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:44.638+0000 7f9591d1e500 -1 memstore(/var/lib/ceph/osd/ceph-2) /var/lib/ceph/osd/ceph-2 2023-10-18T08:54:44.663 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-18T08:54:44.750 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T08:54:44.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:44.900+0000 7fad1d539500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-18T08:54:44.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:44.900+0000 7fad1d539500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-18T08:54:44.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:44.903+0000 7fad1d539500 -1 memstore(/var/lib/ceph/osd/ceph-3) /var/lib/ceph/osd/ceph-3 2023-10-18T08:54:44.922 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-18T08:54:45.016 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-18T08:54:45.120 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T08:54:45.121 INFO:tasks.ceph:role: osd.4 2023-10-18T08:54:45.121 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T08:54:45.121 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-18T08:54:45.221 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T08:54:45.221 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T08:54:45.221 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T08:54:45.222 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-18T08:54:45.222 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T08:54:45.222 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2023-10-18T08:54:45.222 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T08:54:45.222 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-18T08:54:45.223 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T08:54:45.223 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T08:54:45.751 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2023-10-18T08:54:45.757 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2023-10-18T08:54:45.758 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-18T08:54:45.944 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-18T08:54:46.053 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-18T08:54:46.143 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T08:54:46.143 INFO:tasks.ceph:role: osd.5 2023-10-18T08:54:46.144 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T08:54:46.144 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-18T08:54:46.243 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T08:54:46.243 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T08:54:46.243 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T08:54:46.244 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-18T08:54:46.244 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T08:54:46.245 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2023-10-18T08:54:46.245 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T08:54:46.245 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-18T08:54:46.246 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T08:54:46.246 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T08:54:46.729 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2023-10-18T08:54:46.736 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2023-10-18T08:54:46.736 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-18T08:54:46.888 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-18T08:54:46.987 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-18T08:54:47.079 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T08:54:47.079 INFO:tasks.ceph:role: osd.6 2023-10-18T08:54:47.079 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T08:54:47.080 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-18T08:54:47.180 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T08:54:47.181 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T08:54:47.181 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T08:54:47.181 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-18T08:54:47.182 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T08:54:47.182 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2023-10-18T08:54:47.182 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T08:54:47.182 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-18T08:54:47.183 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T08:54:47.183 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T08:54:47.701 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2023-10-18T08:54:47.707 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2023-10-18T08:54:47.707 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-18T08:54:47.900 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-18T08:54:48.009 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-18T08:54:48.095 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-18T08:54:48.096 INFO:tasks.ceph:role: osd.7 2023-10-18T08:54:48.096 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T08:54:48.096 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-18T08:54:48.198 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-18T08:54:48.198 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-18T08:54:48.198 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-18T08:54:48.198 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-18T08:54:48.199 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-18T08:54:48.199 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2023-10-18T08:54:48.199 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-18T08:54:48.199 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-18T08:54:48.199 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-18T08:54:48.199 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-18T08:54:48.661 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2023-10-18T08:54:48.666 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2023-10-18T08:54:48.667 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-18T08:54:48.796 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-18T08:54:48.885 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T08:54:49.035 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:49.032+0000 7fe4b36cf500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-18T08:54:49.037 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:49.034+0000 7fe4b36cf500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-18T08:54:49.042 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:49.039+0000 7fe4b36cf500 -1 memstore(/var/lib/ceph/osd/ceph-4) /var/lib/ceph/osd/ceph-4 2023-10-18T08:54:49.064 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-18T08:54:49.158 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T08:54:49.293 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:49.289+0000 7f9288e9a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-18T08:54:49.296 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:49.293+0000 7f9288e9a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-18T08:54:49.306 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:49.303+0000 7f9288e9a500 -1 memstore(/var/lib/ceph/osd/ceph-5) /var/lib/ceph/osd/ceph-5 2023-10-18T08:54:49.338 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-18T08:54:49.429 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T08:54:49.565 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:49.562+0000 7f1a3d48c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-18T08:54:49.570 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:49.568+0000 7f1a3d48c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-18T08:54:49.579 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:49.576+0000 7f1a3d48c500 -1 memstore(/var/lib/ceph/osd/ceph-6) /var/lib/ceph/osd/ceph-6 2023-10-18T08:54:49.613 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-18T08:54:49.710 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-18T08:54:49.846 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:49.842+0000 7fc7c2d27500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-18T08:54:49.846 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:49.843+0000 7fc7c2d27500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-18T08:54:49.848 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:49.845+0000 7fc7c2d27500 -1 memstore(/var/lib/ceph/osd/ceph-7) /var/lib/ceph/osd/ceph-7 2023-10-18T08:54:49.866 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-18T08:54:49.958 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-18T08:54:49.959 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:49.959 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-18T08:54:50.072 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:50.073 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-18T08:54:50.177 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:50.178 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-18T08:54:50.281 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:50.281 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-18T08:54:50.383 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:50.384 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-18T08:54:50.479 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:50.479 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-18T08:54:50.577 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:50.578 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-18T08:54:50.683 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:50.683 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-18T08:54:50.789 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:50.789 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-18T08:54:50.887 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:50.887 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-18T08:54:50.986 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:50.987 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-18T08:54:51.082 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:51.082 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-18T08:54:51.176 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:51.176 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-18T08:54:51.274 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:51.274 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-18T08:54:51.371 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:51.372 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-18T08:54:51.438 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:51.438 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-18T08:54:51.497 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:51.498 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-18T08:54:51.557 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:51.558 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-18T08:54:51.610 INFO:tasks.ceph:Adding keys to all mons... 2023-10-18T08:54:51.610 DEBUG:teuthology.orchestra.run.smithi040:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-18T08:54:51.619 DEBUG:teuthology.orchestra.run.smithi144:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-18T08:54:51.710 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.y] 2023-10-18T08:54:51.711 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBNnS9loxRXLhAAQK1n1/4PB35umHxOKQzz7Q== 2023-10-18T08:54:51.711 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2023-10-18T08:54:51.711 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBOnS9lYsb5CxAAGButZ6+Usc62kJ0ksxZrYw== 2023-10-18T08:54:51.711 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2023-10-18T08:54:51.711 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBOnS9lPVh+HhAAzucSx+ksPDBkqJGa8+pyJw== 2023-10-18T08:54:51.712 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.0] 2023-10-18T08:54:51.712 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBUnS9lVx+hAhAA6ZpIyZKhbqy8tV84piGILA== 2023-10-18T08:54:51.712 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.1] 2023-10-18T08:54:51.712 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBUnS9lk9LwExAA7zFzQOn2hcvnck2cOZZ7AA== 2023-10-18T08:54:51.712 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.2] 2023-10-18T08:54:51.713 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBUnS9lK930JRAA1GyKRkIpXPHHGqY8owYWRQ== 2023-10-18T08:54:51.713 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.3] 2023-10-18T08:54:51.713 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBUnS9liBWyNRAAMHBBdG5nbrGQupqvUtuKMA== 2023-10-18T08:54:51.713 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.x] 2023-10-18T08:54:51.713 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBNnS9lCf/xOhAAJHVoKIswWUcVGKQpiaJ12A== 2023-10-18T08:54:51.714 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2023-10-18T08:54:51.714 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBOnS9lAqlhMBAApAhxCYB9d9bz1GibADDRSQ== 2023-10-18T08:54:51.714 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2023-10-18T08:54:51.714 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBPnS9lP7QCBhAAog7qDCgGVv0T7WKsRkATkA== 2023-10-18T08:54:51.714 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.4] 2023-10-18T08:54:51.715 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBZnS9loDoOAhAAjHZub4wnB7dCr86rxAtI5g== 2023-10-18T08:54:51.715 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.5] 2023-10-18T08:54:51.715 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBZnS9lhqNfERAAw410g3DwSXrK23YkxVBFVw== 2023-10-18T08:54:51.715 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.6] 2023-10-18T08:54:51.715 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBZnS9lRRScIRAAg0pmfxCQBCPYodj1Lko+Lw== 2023-10-18T08:54:51.716 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.7] 2023-10-18T08:54:51.716 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBZnS9ld1dXMhAAyovlrKOhVuynEiuPTgFcjQ== 2023-10-18T08:54:51.716 INFO:teuthology.orchestra.run.smithi144.stdout:[client.0] 2023-10-18T08:54:51.716 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w== 2023-10-18T08:54:51.716 INFO:teuthology.orchestra.run.smithi144.stdout:[client.2] 2023-10-18T08:54:51.716 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBPnS9lu57fHxAAdzsVU2KKsWRkaARRS8TGjw== 2023-10-18T08:54:51.717 INFO:teuthology.orchestra.run.smithi144.stdout:[client.1] 2023-10-18T08:54:51.717 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBPnS9lAB3oKhAAD43WFdC23WGF6WUi4JKsvg== 2023-10-18T08:54:51.717 INFO:teuthology.orchestra.run.smithi144.stdout:[client.3] 2023-10-18T08:54:51.717 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQBPnS9lygWsNRAA1OziXxk7hBtTw2Aq8OlSKQ== 2023-10-18T08:54:51.718 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.y] 2023-10-18T08:54:51.718 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBNnS9loxRXLhAAQK1n1/4PB35umHxOKQzz7Q== 2023-10-18T08:54:51.718 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2023-10-18T08:54:51.718 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBOnS9lYsb5CxAAGButZ6+Usc62kJ0ksxZrYw== 2023-10-18T08:54:51.719 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c] 2023-10-18T08:54:51.719 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBOnS9lPVh+HhAAzucSx+ksPDBkqJGa8+pyJw== 2023-10-18T08:54:51.719 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.0] 2023-10-18T08:54:51.719 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBUnS9lVx+hAhAA6ZpIyZKhbqy8tV84piGILA== 2023-10-18T08:54:51.719 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.1] 2023-10-18T08:54:51.719 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBUnS9lk9LwExAA7zFzQOn2hcvnck2cOZZ7AA== 2023-10-18T08:54:51.720 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.2] 2023-10-18T08:54:51.720 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBUnS9lK930JRAA1GyKRkIpXPHHGqY8owYWRQ== 2023-10-18T08:54:51.720 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.3] 2023-10-18T08:54:51.720 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBUnS9liBWyNRAAMHBBdG5nbrGQupqvUtuKMA== 2023-10-18T08:54:51.720 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.x] 2023-10-18T08:54:51.720 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBNnS9lCf/xOhAAJHVoKIswWUcVGKQpiaJ12A== 2023-10-18T08:54:51.721 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2023-10-18T08:54:51.721 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBOnS9lAqlhMBAApAhxCYB9d9bz1GibADDRSQ== 2023-10-18T08:54:51.721 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d] 2023-10-18T08:54:51.721 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBPnS9lP7QCBhAAog7qDCgGVv0T7WKsRkATkA== 2023-10-18T08:54:51.721 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.4] 2023-10-18T08:54:51.722 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBZnS9loDoOAhAAjHZub4wnB7dCr86rxAtI5g== 2023-10-18T08:54:51.722 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.5] 2023-10-18T08:54:51.722 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBZnS9lhqNfERAAw410g3DwSXrK23YkxVBFVw== 2023-10-18T08:54:51.722 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.6] 2023-10-18T08:54:51.722 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBZnS9lRRScIRAAg0pmfxCQBCPYodj1Lko+Lw== 2023-10-18T08:54:51.723 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.7] 2023-10-18T08:54:51.723 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBZnS9ld1dXMhAAyovlrKOhVuynEiuPTgFcjQ== 2023-10-18T08:54:51.723 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2023-10-18T08:54:51.723 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w== 2023-10-18T08:54:51.723 INFO:teuthology.orchestra.run.smithi040.stdout:[client.2] 2023-10-18T08:54:51.723 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBPnS9lu57fHxAAdzsVU2KKsWRkaARRS8TGjw== 2023-10-18T08:54:51.724 INFO:teuthology.orchestra.run.smithi040.stdout:[client.1] 2023-10-18T08:54:51.724 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBPnS9lAB3oKhAAD43WFdC23WGF6WUi4JKsvg== 2023-10-18T08:54:51.724 INFO:teuthology.orchestra.run.smithi040.stdout:[client.3] 2023-10-18T08:54:51.724 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBPnS9lygWsNRAA1OziXxk7hBtTw2Aq8OlSKQ== 2023-10-18T08:54:51.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T08:54:51.734 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T08:54:51.882 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T08:54:51.890 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T08:54:52.040 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T08:54:52.046 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T08:54:52.193 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T08:54:52.202 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T08:54:52.345 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T08:54:52.351 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T08:54:52.497 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T08:54:52.502 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T08:54:52.650 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T08:54:52.658 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T08:54:52.804 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T08:54:52.810 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-18T08:54:52.955 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T08:54:52.961 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T08:54:53.112 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T08:54:53.120 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-18T08:54:53.273 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T08:54:53.279 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T08:54:53.427 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T08:54:53.432 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T08:54:53.581 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T08:54:53.589 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T08:54:53.729 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T08:54:53.734 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-18T08:54:53.881 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T08:54:53.888 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T08:54:54.039 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T08:54:54.044 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T08:54:54.191 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T08:54:54.199 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T08:54:54.350 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T08:54:54.358 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-18T08:54:54.509 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-18T08:54:54.509 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-18T08:54:54.613 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-18T08:54:54.784 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-18T08:54:54.871 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-18T08:54:54.969 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-18T08:54:55.123 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-18T08:54:55.219 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-18T08:54:55.310 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-18T08:54:55.470 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-18T08:54:55.553 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-18T08:54:55.562 DEBUG:teuthology.orchestra.run.smithi144:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-18T08:54:55.615 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-18T08:54:55.616 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-18T08:54:55.617 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-18T08:54:55.623 INFO:tasks.ceph.mon.a:Started 2023-10-18T08:54:55.624 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-18T08:54:55.624 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-18T08:54:55.629 INFO:tasks.ceph.mon.b:Started 2023-10-18T08:54:55.629 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-18T08:54:55.630 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-18T08:54:55.634 INFO:tasks.ceph.mon.c:Started 2023-10-18T08:54:55.635 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-18T08:54:55.635 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-18T08:54:55.635 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-18T08:54:55.641 INFO:tasks.ceph.mgr.y:Started 2023-10-18T08:54:55.642 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-18T08:54:55.643 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-18T08:54:55.648 INFO:tasks.ceph.mgr.x:Started 2023-10-18T08:54:55.648 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2023-10-18T08:54:55.649 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2023-10-18T08:54:55.654 INFO:tasks.ceph:setting config [client] debug_client = 20 2023-10-18T08:54:55.655 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set client debug_client 20 2023-10-18T08:54:55.701 INFO:tasks.ceph:setting config [client] debug_ms = 1 2023-10-18T08:54:55.702 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set client debug_ms 1 2023-10-18T08:54:55.747 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2023-10-18T08:54:55.747 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2023-10-18T08:54:55.795 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2023-10-18T08:54:55.795 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2023-10-18T08:54:55.844 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2023-10-18T08:54:55.844 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set mds debug_mds 20 2023-10-18T08:54:55.893 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2023-10-18T08:54:55.894 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2023-10-18T08:54:55.945 INFO:tasks.ceph:setting config [mds] debug_ms = 1 2023-10-18T08:54:55.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set mds debug_ms 1 2023-10-18T08:54:55.997 INFO:tasks.ceph:setting config [mds] mds_bal_fragment_size_max = 10000 2023-10-18T08:54:55.997 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set mds mds_bal_fragment_size_max 10000 2023-10-18T08:54:56.047 INFO:tasks.ceph:setting config [mds] mds_bal_merge_size = 5 2023-10-18T08:54:56.047 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set mds mds_bal_merge_size 5 2023-10-18T08:54:56.099 INFO:tasks.ceph:setting config [mds] mds_bal_split_bits = 3 2023-10-18T08:54:56.099 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set mds mds_bal_split_bits 3 2023-10-18T08:54:56.145 INFO:tasks.ceph:setting config [mds] mds_bal_split_size = 100 2023-10-18T08:54:56.145 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set mds mds_bal_split_size 100 2023-10-18T08:54:56.170 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2023-10-18T08:54:56.170 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2023-10-18T08:54:56.223 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2023-10-18T08:54:56.224 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2023-10-18T08:54:56.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.214+0000 7f192e475640 1 -- 172.21.15.40:0/3263770023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19280cd5b0 msgr2=0x7f1928149e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.214+0000 7f192e475640 1 --2- 172.21.15.40:0/3263770023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19280cd5b0 0x7f1928149e90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f192000c6a0 tx=0x7f192000cb60 comp rx=0 tx=0).stop 2023-10-18T08:54:56.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.215+0000 7f192e475640 1 -- 172.21.15.40:0/3263770023 shutdown_connections 2023-10-18T08:54:56.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.215+0000 7f192e475640 1 --2- 172.21.15.40:0/3263770023 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1928157800 0x7f19281518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.215+0000 7f192e475640 1 --2- 172.21.15.40:0/3263770023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19280cd5b0 0x7f1928149e90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.215+0000 7f192e475640 1 --2- 172.21.15.40:0/3263770023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19280cca30 0x7f192814fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.215+0000 7f192e475640 1 -- 172.21.15.40:0/3263770023 >> 172.21.15.40:0/3263770023 conn(0x7f1928069720 msgr2=0x7f19280ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.215+0000 7f192e475640 1 -- 172.21.15.40:0/3263770023 shutdown_connections 2023-10-18T08:54:56.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.215+0000 7f192e475640 1 -- 172.21.15.40:0/3263770023 wait complete. 2023-10-18T08:54:56.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.223+0000 7fb0a1b21640 1 -- 172.21.15.40:0/2254058190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0940e31a0 msgr2=0x7fb09404f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.223+0000 7fb0a1b21640 1 --2- 172.21.15.40:0/2254058190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0940e31a0 0x7fb09404f260 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb090009830 tx=0x7fb09002bea0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.225+0000 7fb0a1b21640 1 -- 172.21.15.40:0/2254058190 shutdown_connections 2023-10-18T08:54:56.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.225+0000 7fb0a1b21640 1 --2- 172.21.15.40:0/2254058190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0940e31a0 0x7fb09404f260 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.225+0000 7fb0a1b21640 1 -- 172.21.15.40:0/2254058190 >> 172.21.15.40:0/2254058190 conn(0x7fb094009f80 msgr2=0x7fb09400a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.238 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2023-10-18T08:54:56.239 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2023-10-18T08:54:56.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.227+0000 7fb0a1b21640 1 -- 172.21.15.40:0/2254058190 shutdown_connections 2023-10-18T08:54:56.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.227+0000 7fb0a1b21640 1 -- 172.21.15.40:0/2254058190 wait complete. 2023-10-18T08:54:56.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.231+0000 7fb0a1b21640 1 Processor -- start 2023-10-18T08:54:56.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.232+0000 7fb0a1b21640 1 -- start start 2023-10-18T08:54:56.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.233+0000 7fb0a1b21640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb094038300 0x7fb0940e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.234+0000 7fb0a1b21640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb094053fe0 0x7fb0941790b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.234+0000 7fb0a1b21640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0940e31a0 0x7fb0940dd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.234+0000 7fb0a1b21640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0940e6b40 con 0x7fb094038300 2023-10-18T08:54:56.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.234+0000 7fb0a1b21640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0940ceb80 con 0x7fb094053fe0 2023-10-18T08:54:56.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.234+0000 7fb0a1b21640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0940dd420 con 0x7fb0940e31a0 2023-10-18T08:54:56.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.235+0000 7fb0a0b1f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb094038300 0x7fb0940e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.235+0000 7fb0a0b1f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb094038300 0x7fb0940e64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43152/0 (socket says 172.21.15.40:43152) 2023-10-18T08:54:56.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.235+0000 7fb09bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb094053fe0 0x7fb0941790b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.235+0000 7fb0a1320640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0940e31a0 0x7fb0940dd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.235+0000 7fb0a1320640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0940e31a0 0x7fb0940dd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:59000/0 (socket says 172.21.15.40:59000) 2023-10-18T08:54:56.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.235+0000 7fb0a1320640 1 -- 172.21.15.40:0/1314402100 learned_addr learned my addr 172.21.15.40:0/1314402100 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:56.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.236+0000 7fb0a1320640 1 -- 172.21.15.40:0/1314402100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb094053fe0 msgr2=0x7fb0941790b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.237+0000 7fb0a1320640 1 --2- 172.21.15.40:0/1314402100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb094053fe0 0x7fb0941790b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.237+0000 7fb0a1320640 1 -- 172.21.15.40:0/1314402100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb094038300 msgr2=0x7fb0940e64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.241+0000 7fb0a1320640 1 --2- 172.21.15.40:0/1314402100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb094038300 0x7fb0940e64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.241+0000 7fb0a1320640 1 -- 172.21.15.40:0/1314402100 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb090009490 con 0x7fb0940e31a0 2023-10-18T08:54:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.241+0000 7fb09bfff640 1 --2- 172.21.15.40:0/1314402100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb094053fe0 0x7fb0941790b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:54:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.241+0000 7fb0a0b1f640 1 --2- 172.21.15.40:0/1314402100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb094038300 0x7fb0940e64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:54:56.261 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2023-10-18T08:54:56.261 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2023-10-18T08:54:56.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.245+0000 7fb0a1320640 1 --2- 172.21.15.40:0/1314402100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0940e31a0 0x7fb0940dd920 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0633a0 tx=0x7fb09c073940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:56.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.246+0000 7fb0997fa640 1 -- 172.21.15.40:0/1314402100 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c0669a0 con 0x7fb0940e31a0 2023-10-18T08:54:56.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.249+0000 7fb0997fa640 1 -- 172.21.15.40:0/1314402100 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(3 keys) v1 ==== 79+0+0 (secure 0 0 0) 0x7fb09c0669a0 con 0x7fb0940e31a0 2023-10-18T08:54:56.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.249+0000 7fb0997fa640 1 -- 172.21.15.40:0/1314402100 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c06a8a0 con 0x7fb0940e31a0 2023-10-18T08:54:56.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.254+0000 7fb0a1b21640 1 -- 172.21.15.40:0/1314402100 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0940e3c90 con 0x7fb0940e31a0 2023-10-18T08:54:56.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.254+0000 7fb0a1b21640 1 -- 172.21.15.40:0/1314402100 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0940dde60 con 0x7fb0940e31a0 2023-10-18T08:54:56.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.256+0000 7fb0997fa640 1 -- 172.21.15.40:0/1314402100 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb09c08e020 con 0x7fb0940e31a0 2023-10-18T08:54:56.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.256+0000 7fb0997fa640 1 -- 172.21.15.40:0/1314402100 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb09c073180 con 0x7fb0940e31a0 2023-10-18T08:54:56.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.260+0000 7fb086ffd640 1 -- 172.21.15.40:0/1314402100 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0940089a0 con 0x7fb0940e31a0 2023-10-18T08:54:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.278+0000 7fb0997fa640 1 -- 172.21.15.40:0/1314402100 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fb09c06aa20 con 0x7fb0940e31a0 2023-10-18T08:54:56.297 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2023-10-18T08:54:56.297 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2023-10-18T08:54:56.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.311+0000 7f577891d640 1 -- 172.21.15.40:0/3186848224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f577406bad0 msgr2=0x7f5774157c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.329 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2023-10-18T08:54:56.329 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2023-10-18T08:54:56.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.312+0000 7f577891d640 1 --2- 172.21.15.40:0/3186848224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f577406bad0 0x7f5774157c00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f576c005fd0 tx=0x7f576c02c050 comp rx=0 tx=0).stop 2023-10-18T08:54:56.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.315+0000 7f577891d640 1 -- 172.21.15.40:0/3186848224 shutdown_connections 2023-10-18T08:54:56.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.315+0000 7f577891d640 1 --2- 172.21.15.40:0/3186848224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f577406bad0 0x7f5774157c00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.315+0000 7f577891d640 1 --2- 172.21.15.40:0/3186848224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5774157820 0x7f57740b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.315+0000 7f577891d640 1 -- 172.21.15.40:0/3186848224 >> 172.21.15.40:0/3186848224 conn(0x7f57740ccf80 msgr2=0x7f57740cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.318+0000 7f577891d640 1 -- 172.21.15.40:0/3186848224 shutdown_connections 2023-10-18T08:54:56.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.321+0000 7f577891d640 1 -- 172.21.15.40:0/3186848224 wait complete. 2023-10-18T08:54:56.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.332+0000 7f3d491a5640 1 -- 172.21.15.40:0/457159249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f3d44072d50 con 0x7f3d440cff00 2023-10-18T08:54:56.347 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2023-10-18T08:54:56.347 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2023-10-18T08:54:56.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.332+0000 7f577891d640 1 Processor -- start 2023-10-18T08:54:56.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.337+0000 7f577891d640 1 -- start start 2023-10-18T08:54:56.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.346+0000 7f577891d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f577406bad0 0x7f5774159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.346+0000 7f577891d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57740cc230 0x7f57740acb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.347+0000 7f3d3b7fe640 1 -- 172.21.15.40:0/457159249 <== mon.0 v2:172.21.15.40:3300/0 10 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v5)=0 v5) v1 ==== 123+0+0 (secure 0 0 0) 0x7f3d340560c0 con 0x7f3d440cff00 2023-10-18T08:54:56.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.347+0000 7f3d3b7fe640 1 -- 172.21.15.40:0/457159249 <== mon.0 v2:172.21.15.40:3300/0 11 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7f3d34051b50 con 0x7f3d440cff00 2023-10-18T08:54:56.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.347+0000 7f577891d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5774157820 0x7f57740ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.347+0000 7f577259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f577406bad0 0x7f5774159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.347+0000 7f577259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f577406bad0 0x7f5774159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.40:55826/0 (socket says 172.21.15.40:55826) 2023-10-18T08:54:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.347+0000 7f577891d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57740ac000 con 0x7f5774157820 2023-10-18T08:54:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.347+0000 7f5771d9b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57740cc230 0x7f57740acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.347+0000 7f5771d9b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57740cc230 0x7f57740acb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:59018/0 (socket says 172.21.15.40:59018) 2023-10-18T08:54:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.347+0000 7f577891d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57741f6860 con 0x7f577406bad0 2023-10-18T08:54:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.347+0000 7f577891d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57741f5aa0 con 0x7f57740cc230 2023-10-18T08:54:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.348+0000 7f5771d9b640 1 -- 172.21.15.40:0/3373651242 learned_addr learned my addr 172.21.15.40:0/3373651242 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.349+0000 7f4b15ffb640 1 -- 172.21.15.40:0/745818700 <== mon.0 v2:172.21.15.40:3300/0 8 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7f4b1804a650 con 0x7f4b20152b10 2023-10-18T08:54:56.371 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2023-10-18T08:54:56.371 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2023-10-18T08:54:56.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.359+0000 7f3d491a5640 1 -- 172.21.15.40:0/457159249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d440cff00 msgr2=0x7f3d440b69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.359+0000 7f3d491a5640 1 --2- 172.21.15.40:0/457159249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d440cff00 0x7f3d440b69f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3d34006010 tx=0x7f3d340046c0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.359+0000 7f3d491a5640 1 -- 172.21.15.40:0/457159249 shutdown_connections 2023-10-18T08:54:56.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.359+0000 7f3d491a5640 1 --2- 172.21.15.40:0/457159249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4415b6c0 0x7f3d440c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.361+0000 7f3d491a5640 1 --2- 172.21.15.40:0/457159249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d440d1c70 0x7f3d440c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.361+0000 7f3d491a5640 1 --2- 172.21.15.40:0/457159249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d440cff00 0x7f3d440b69f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.361+0000 7f3d491a5640 1 -- 172.21.15.40:0/457159249 >> 172.21.15.40:0/457159249 conn(0x7f3d4409a960 msgr2=0x7f3d440afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.361+0000 7f3d491a5640 1 -- 172.21.15.40:0/457159249 shutdown_connections 2023-10-18T08:54:56.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.361+0000 7f3d491a5640 1 -- 172.21.15.40:0/457159249 wait complete. 2023-10-18T08:54:56.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.351+0000 7fb0997fa640 1 -- 172.21.15.40:0/1314402100 <== mon.2 v2:172.21.15.144:3301/0 7 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7fb09c06ac00 con 0x7fb0940e31a0 2023-10-18T08:54:56.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.353+0000 7f5771d9b640 1 -- 172.21.15.40:0/3373651242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f577406bad0 msgr2=0x7f5774159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.353+0000 7f5771d9b640 1 --2- 172.21.15.40:0/3373651242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f577406bad0 0x7f5774159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.353+0000 7f5771d9b640 1 -- 172.21.15.40:0/3373651242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5774157820 msgr2=0x7f57740ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.353+0000 7f5771d9b640 1 --2- 172.21.15.40:0/3373651242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5774157820 0x7f57740ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.353+0000 7f5771d9b640 1 -- 172.21.15.40:0/3373651242 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c009840 con 0x7f57740cc230 2023-10-18T08:54:56.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.353+0000 7f5771d9b640 1 --2- 172.21.15.40:0/3373651242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57740cc230 0x7f57740acb90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f576800c4a0 tx=0x7f576800c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:56.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.354+0000 7f5762ffd640 1 -- 172.21.15.40:0/3373651242 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57680035f0 con 0x7f57740cc230 2023-10-18T08:54:56.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.354+0000 7f577891d640 1 -- 172.21.15.40:0/3373651242 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57741f5d80 con 0x7f57740cc230 2023-10-18T08:54:56.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.354+0000 7f577891d640 1 -- 172.21.15.40:0/3373651242 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57741f6340 con 0x7f57740cc230 2023-10-18T08:54:56.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.355+0000 7f5762ffd640 1 -- 172.21.15.40:0/3373651242 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7f5768003820 con 0x7f57740cc230 2023-10-18T08:54:56.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.355+0000 7f5762ffd640 1 -- 172.21.15.40:0/3373651242 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57680035f0 con 0x7f57740cc230 2023-10-18T08:54:56.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.363+0000 7f5762ffd640 1 -- 172.21.15.40:0/3373651242 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f576801c440 con 0x7f57740cc230 2023-10-18T08:54:56.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.363+0000 7f5760ff9640 1 -- 172.21.15.40:0/3373651242 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c005250 con 0x7f57740cc230 2023-10-18T08:54:56.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.363+0000 7f5762ffd640 1 -- 172.21.15.40:0/3373651242 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5768012740 con 0x7f57740cc230 2023-10-18T08:54:56.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.371+0000 7f5762ffd640 1 -- 172.21.15.40:0/3373651242 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f5768017070 con 0x7f57740cc230 2023-10-18T08:54:56.409 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2023-10-18T08:54:56.410 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2023-10-18T08:54:56.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.410+0000 7fad85f47640 1 -- 172.21.15.40:0/1650556474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad8015b420 msgr2=0x7fad800cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.410+0000 7fad85f47640 1 --2- 172.21.15.40:0/1650556474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad8015b420 0x7fad800cf010 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fad70009830 tx=0x7fad7002bb60 comp rx=0 tx=0).stop 2023-10-18T08:54:56.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.414+0000 7fad85f47640 1 -- 172.21.15.40:0/1650556474 shutdown_connections 2023-10-18T08:54:56.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.414+0000 7fad85f47640 1 --2- 172.21.15.40:0/1650556474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad8015b420 0x7fad800cf010 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.414+0000 7fad85f47640 1 -- 172.21.15.40:0/1650556474 >> 172.21.15.40:0/1650556474 conn(0x7fad8009a590 msgr2=0x7fad800d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.414+0000 7fad85f47640 1 -- 172.21.15.40:0/1650556474 shutdown_connections 2023-10-18T08:54:56.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.414+0000 7fad85f47640 1 -- 172.21.15.40:0/1650556474 wait complete. 2023-10-18T08:54:56.446 DEBUG:tasks.ceph:set 21 configs 2023-10-18T08:54:56.447 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi040') 2023-10-18T08:54:56.447 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi040') 2023-10-18T08:54:56.448 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi040') 2023-10-18T08:54:56.448 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi040') 2023-10-18T08:54:56.449 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi040') 2023-10-18T08:54:56.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.418+0000 7f4b273b7640 1 -- 172.21.15.40:0/745818700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7f4afc0052d0 con 0x7f4b20152b10 2023-10-18T08:54:56.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.424+0000 7f4b15ffb640 1 -- 172.21.15.40:0/745818700 <== mon.0 v2:172.21.15.40:3300/0 9 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) v1 ==== 123+0+0 (secure 0 0 0) 0x7f4b180590c0 con 0x7f4b20152b10 2023-10-18T08:54:56.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.424+0000 7f4b15ffb640 1 -- 172.21.15.40:0/745818700 <== mon.0 v2:172.21.15.40:3300/0 10 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4b1804a650 con 0x7f4b20152b10 2023-10-18T08:54:56.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.430+0000 7f4b273b7640 1 -- 172.21.15.40:0/745818700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b20152b10 msgr2=0x7f4b200cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.430+0000 7f4b273b7640 1 --2- 172.21.15.40:0/745818700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b20152b10 0x7f4b200cbb80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4b18006010 tx=0x7f4b18004a40 comp rx=0 tx=0).stop 2023-10-18T08:54:56.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.434+0000 7f4b273b7640 1 -- 172.21.15.40:0/745818700 shutdown_connections 2023-10-18T08:54:56.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.434+0000 7f4b273b7640 1 --2- 172.21.15.40:0/745818700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b20152b10 0x7f4b200cbb80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.434+0000 7f4b273b7640 1 --2- 172.21.15.40:0/745818700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b200b1b30 0x7f4b200cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.434+0000 7f4b273b7640 1 --2- 172.21.15.40:0/745818700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b200b1750 0x7f4b2014f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.434+0000 7f4b273b7640 1 -- 172.21.15.40:0/745818700 >> 172.21.15.40:0/745818700 conn(0x7f4b200ccf80 msgr2=0x7f4b2015f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.435+0000 7f4b273b7640 1 -- 172.21.15.40:0/745818700 shutdown_connections 2023-10-18T08:54:56.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.435+0000 7f4b273b7640 1 -- 172.21.15.40:0/745818700 wait complete. 2023-10-18T08:54:56.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.439+0000 7fb0997fa640 1 -- 172.21.15.40:0/1314402100 <== mon.2 v2:172.21.15.144:3301/0 8 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb09c06ad90 con 0x7fb0940e31a0 2023-10-18T08:54:56.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.428+0000 7f5762ffd640 1 -- 172.21.15.40:0/3373651242 <== mon.2 v2:172.21.15.144:3301/0 7 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5768012ad0 con 0x7f57740cc230 2023-10-18T08:54:56.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.438+0000 7fad85f47640 1 Processor -- start 2023-10-18T08:54:56.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.441+0000 7fad85f47640 1 -- start start 2023-10-18T08:54:56.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.447+0000 7fad85f47640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad800cdb70 0x7fad800c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.448+0000 7fad84f45640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad800cdb70 0x7fad800c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.452+0000 7fad84f45640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad800cdb70 0x7fad800c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43166/0 (socket says 172.21.15.40:43166) 2023-10-18T08:54:56.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.452+0000 7fad85f47640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad800cfee0 0x7fad80159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.452+0000 7fad84f45640 1 -- 172.21.15.40:0/2203767996 learned_addr learned my addr 172.21.15.40:0/2203767996 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:56.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.452+0000 7fad85f47640 1 --2- 172.21.15.40:0/2203767996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad8015b420 0x7fad800b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.452+0000 7fad85f47640 1 -- 172.21.15.40:0/2203767996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad800c5390 con 0x7fad800cdb70 2023-10-18T08:54:56.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.452+0000 7fad85f47640 1 -- 172.21.15.40:0/2203767996 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad800ca2f0 con 0x7fad800cfee0 2023-10-18T08:54:56.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.452+0000 7fad85f47640 1 -- 172.21.15.40:0/2203767996 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad800ca5d0 con 0x7fad8015b420 2023-10-18T08:54:56.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.453+0000 7fad7ffff640 1 --2- 172.21.15.40:0/2203767996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad800cfee0 0x7fad80159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.454+0000 7fad84f45640 1 -- 172.21.15.40:0/2203767996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad8015b420 msgr2=0x7fad800b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.454+0000 7fad85746640 1 --2- 172.21.15.40:0/2203767996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad8015b420 0x7fad800b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.456+0000 7fad84f45640 1 --2- 172.21.15.40:0/2203767996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad8015b420 0x7fad800b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.456+0000 7fad84f45640 1 -- 172.21.15.40:0/2203767996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad800cfee0 msgr2=0x7fad80159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.456+0000 7fad84f45640 1 --2- 172.21.15.40:0/2203767996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad800cfee0 0x7fad80159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.456+0000 7fad84f45640 1 -- 172.21.15.40:0/2203767996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad70009490 con 0x7fad800cdb70 2023-10-18T08:54:56.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.456+0000 7fad7ffff640 1 --2- 172.21.15.40:0/2203767996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad800cfee0 0x7fad80159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:54:56.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.456+0000 7fad85746640 1 --2- 172.21.15.40:0/2203767996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad8015b420 0x7fad800b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:54:56.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.458+0000 7fad84f45640 1 --2- 172.21.15.40:0/2203767996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad800cdb70 0x7fad800c75c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fad70009950 tx=0x7fad7002dec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:56.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.459+0000 7fad7d7fa640 1 -- 172.21.15.40:0/2203767996 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7003c070 con 0x7fad800cdb70 2023-10-18T08:54:56.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.459+0000 7fad85f47640 1 -- 172.21.15.40:0/2203767996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad800c4d40 con 0x7fad800cdb70 2023-10-18T08:54:56.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.459+0000 7fad85f47640 1 -- 172.21.15.40:0/2203767996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad800c7b00 con 0x7fad800cdb70 2023-10-18T08:54:56.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.459+0000 7fad7d7fa640 1 -- 172.21.15.40:0/2203767996 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fad700035f0 con 0x7fad800cdb70 2023-10-18T08:54:56.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.459+0000 7fad7d7fa640 1 -- 172.21.15.40:0/2203767996 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70005cd0 con 0x7fad800cdb70 2023-10-18T08:54:56.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.461+0000 7fad85f47640 1 -- 172.21.15.40:0/2203767996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad800cec10 con 0x7fad800cdb70 2023-10-18T08:54:56.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.462+0000 7fad7d7fa640 1 -- 172.21.15.40:0/2203767996 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fad70003a00 con 0x7fad800cdb70 2023-10-18T08:54:56.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.462+0000 7fad7d7fa640 1 -- 172.21.15.40:0/2203767996 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fad7002ed90 con 0x7fad800cdb70 2023-10-18T08:54:56.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.469+0000 7fad7d7fa640 1 -- 172.21.15.40:0/2203767996 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fad7002e470 con 0x7fad800cdb70 2023-10-18T08:54:56.479 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi040') 2023-10-18T08:54:56.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.508+0000 7f824872c640 1 -- 172.21.15.40:0/2802066466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82380df3c0 msgr2=0x7f8238054050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.508+0000 7f824872c640 1 --2- 172.21.15.40:0/2802066466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82380df3c0 0x7f8238054050 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f823c009830 tx=0x7f823c02bd50 comp rx=0 tx=0).stop 2023-10-18T08:54:56.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.508+0000 7f824872c640 1 -- 172.21.15.40:0/2802066466 shutdown_connections 2023-10-18T08:54:56.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.508+0000 7f824872c640 1 --2- 172.21.15.40:0/2802066466 >> v2:172.21.15.40:3300/0 conn(0x7f823804fca0 0x7f8238050130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.508+0000 7f824872c640 1 --2- 172.21.15.40:0/2802066466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82380df3c0 0x7f8238054050 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.508+0000 7f824872c640 1 -- 172.21.15.40:0/2802066466 >> 172.21.15.40:0/2802066466 conn(0x7f8238054c60 msgr2=0x7f8238055070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.511+0000 7f824872c640 1 -- 172.21.15.40:0/2802066466 shutdown_connections 2023-10-18T08:54:56.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.511+0000 7f824872c640 1 -- 172.21.15.40:0/2802066466 wait complete. 2023-10-18T08:54:56.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.522+0000 7f824872c640 1 Processor -- start 2023-10-18T08:54:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.522+0000 7f824872c640 1 -- start start 2023-10-18T08:54:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.524+0000 7f824872c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823804fca0 0x7f82380cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.524+0000 7f824872c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82380531f0 0x7f82381678c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.524+0000 7f824872c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82380df3c0 0x7f82381692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.524+0000 7f824872c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8238168210 con 0x7f823804fca0 2023-10-18T08:54:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.524+0000 7f824872c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82381684f0 con 0x7f82380df3c0 2023-10-18T08:54:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.524+0000 7f824872c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82381687d0 con 0x7f82380531f0 2023-10-18T08:54:56.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.530+0000 7f82474ca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823804fca0 0x7f82380cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.530+0000 7f82474ca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823804fca0 0x7f82380cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43178/0 (socket says 172.21.15.40:43178) 2023-10-18T08:54:56.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.530+0000 7f82474ca640 1 -- 172.21.15.40:0/468546022 learned_addr learned my addr 172.21.15.40:0/468546022 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:56.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.536+0000 7f82474ca640 1 -- 172.21.15.40:0/468546022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82380531f0 msgr2=0x7f82381678c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T08:54:56.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.536+0000 7f82474ca640 1 --2- 172.21.15.40:0/468546022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82380531f0 0x7f82381678c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.536+0000 7f82474ca640 1 -- 172.21.15.40:0/468546022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82380df3c0 msgr2=0x7f82381692b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T08:54:56.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.536+0000 7f82474ca640 1 --2- 172.21.15.40:0/468546022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82380df3c0 0x7f82381692b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.536+0000 7f82474ca640 1 -- 172.21.15.40:0/468546022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c009490 con 0x7f823804fca0 2023-10-18T08:54:56.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.540+0000 7f82474ca640 1 --2- 172.21.15.40:0/468546022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823804fca0 0x7f82380cbfe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f823c000c00 tx=0x7f823c0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:56.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.548+0000 7f8236ffd640 1 -- 172.21.15.40:0/468546022 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c042d30 con 0x7f823804fca0 2023-10-18T08:54:56.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.548+0000 7f8236ffd640 1 -- 172.21.15.40:0/468546022 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f823c003670 con 0x7f823804fca0 2023-10-18T08:54:56.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.548+0000 7f8236ffd640 1 -- 172.21.15.40:0/468546022 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c003a80 con 0x7f823804fca0 2023-10-18T08:54:56.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.549+0000 7f824872c640 1 -- 172.21.15.40:0/468546022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8238168ab0 con 0x7f823804fca0 2023-10-18T08:54:56.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.552+0000 7f824872c640 1 -- 172.21.15.40:0/468546022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82380cc520 con 0x7f823804fca0 2023-10-18T08:54:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.561+0000 7f8236ffd640 1 -- 172.21.15.40:0/468546022 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f823c03b070 con 0x7f823804fca0 2023-10-18T08:54:56.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.561+0000 7f8236ffd640 1 -- 172.21.15.40:0/468546022 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f823c034b10 con 0x7f823804fca0 2023-10-18T08:54:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.562+0000 7f8234ff9640 1 -- 172.21.15.40:0/468546022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821c005250 con 0x7f823804fca0 2023-10-18T08:54:56.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.569+0000 7f8236ffd640 1 -- 172.21.15.40:0/468546022 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f823c069b40 con 0x7f823804fca0 2023-10-18T08:54:56.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.576+0000 7fb086ffd640 1 -- 172.21.15.40:0/1314402100 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) v1 -- 0x7fb0740052b0 con 0x7fb0940e31a0 2023-10-18T08:54:56.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.586+0000 7f728c67c640 1 -- 172.21.15.40:0/2607725573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72840cc0a0 msgr2=0x7f72840cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.586+0000 7f728c67c640 1 --2- 172.21.15.40:0/2607725573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72840cc0a0 0x7f72840cc500 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7274009830 tx=0x7f727402bd30 comp rx=0 tx=0).stop 2023-10-18T08:54:56.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.587+0000 7f728c67c640 1 -- 172.21.15.40:0/2607725573 shutdown_connections 2023-10-18T08:54:56.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.587+0000 7f728c67c640 1 --2- 172.21.15.40:0/2607725573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72840cc0a0 0x7f72840cc500 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.587+0000 7f728c67c640 1 -- 172.21.15.40:0/2607725573 >> 172.21.15.40:0/2607725573 conn(0x7f72840ccf80 msgr2=0x7f72840cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.587+0000 7f728c67c640 1 -- 172.21.15.40:0/2607725573 shutdown_connections 2023-10-18T08:54:56.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.587+0000 7f728c67c640 1 -- 172.21.15.40:0/2607725573 wait complete. 2023-10-18T08:54:56.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.592+0000 7f728c67c640 1 Processor -- start 2023-10-18T08:54:56.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.595+0000 7f728c67c640 1 -- start start 2023-10-18T08:54:56.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.601+0000 7f728c67c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f728406b3c0 0x7f72841f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.605 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:56.602+0000 7f700aa30040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T08:54:56.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.605+0000 7fb0997fa640 1 -- 172.21.15.40:0/1314402100 <== mon.2 v2:172.21.15.144:3301/0 9 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) v1 ==== 108+0+0 (secure 0 0 0) 0x7fb09c0a00b0 con 0x7fb0940e31a0 2023-10-18T08:54:56.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.602+0000 7f728a418640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f728406b3c0 0x7f72841f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.602+0000 7f728a418640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f728406b3c0 0x7f72841f07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:59060/0 (socket says 172.21.15.40:59060) 2023-10-18T08:54:56.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.605+0000 7f728a418640 1 -- 172.21.15.40:0/3255749880 learned_addr learned my addr 172.21.15.40:0/3255749880 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:56.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.605+0000 7f728c67c640 1 --2- 172.21.15.40:0/3255749880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72840cc0a0 0x7f72841f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.605+0000 7f728c67c640 1 --2- 172.21.15.40:0/3255749880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f728413fa40 0x7f72841ef110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.607+0000 7f728c67c640 1 -- 172.21.15.40:0/3255749880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72841f0d10 con 0x7f72840cc0a0 2023-10-18T08:54:56.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.607+0000 7f728c67c640 1 -- 172.21.15.40:0/3255749880 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72841523e0 con 0x7f728413fa40 2023-10-18T08:54:56.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.607+0000 7f728ac19640 1 --2- 172.21.15.40:0/3255749880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f728413fa40 0x7f72841ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.608+0000 7f728c67c640 1 -- 172.21.15.40:0/3255749880 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72841526c0 con 0x7f728406b3c0 2023-10-18T08:54:56.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.609+0000 7f7289c17640 1 --2- 172.21.15.40:0/3255749880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72840cc0a0 0x7f72841f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.616+0000 7f7289c17640 1 -- 172.21.15.40:0/3255749880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f728406b3c0 msgr2=0x7f72841f07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.617+0000 7f7289c17640 1 --2- 172.21.15.40:0/3255749880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f728406b3c0 0x7f72841f07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.617+0000 7f7289c17640 1 -- 172.21.15.40:0/3255749880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f728413fa40 msgr2=0x7f72841ef110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.617+0000 7f7289c17640 1 --2- 172.21.15.40:0/3255749880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f728413fa40 0x7f72841ef110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.617+0000 7f7289c17640 1 -- 172.21.15.40:0/3255749880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7274009490 con 0x7f72840cc0a0 2023-10-18T08:54:56.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.617+0000 7f728ac19640 1 --2- 172.21.15.40:0/3255749880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f728413fa40 0x7f72841ef110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T08:54:56.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.618+0000 7f728a418640 1 --2- 172.21.15.40:0/3255749880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f728406b3c0 0x7f72841f07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T08:54:56.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.622+0000 7fb086ffd640 1 -- 172.21.15.40:0/1314402100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0940e31a0 msgr2=0x7fb0940dd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.622+0000 7fb086ffd640 1 --2- 172.21.15.40:0/1314402100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0940e31a0 0x7fb0940dd920 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0633a0 tx=0x7fb09c073940 comp rx=0 tx=0).stop 2023-10-18T08:54:56.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.622+0000 7f7289c17640 1 --2- 172.21.15.40:0/3255749880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72840cc0a0 0x7f72841f35b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7274033000 tx=0x7f72740038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:56.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.626+0000 7f727affd640 1 -- 172.21.15.40:0/3255749880 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274003ea0 con 0x7f72840cc0a0 2023-10-18T08:54:56.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.626+0000 7f728c67c640 1 -- 172.21.15.40:0/3255749880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728413ff60 con 0x7f72840cc0a0 2023-10-18T08:54:56.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.626+0000 7f728c67c640 1 -- 172.21.15.40:0/3255749880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7284152c80 con 0x7f72840cc0a0 2023-10-18T08:54:56.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.627+0000 7f727affd640 1 -- 172.21.15.40:0/3255749880 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7274034440 con 0x7f72840cc0a0 2023-10-18T08:54:56.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.631+0000 7f727affd640 1 -- 172.21.15.40:0/3255749880 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274003ea0 con 0x7f72840cc0a0 2023-10-18T08:54:56.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.636+0000 7fb086ffd640 1 -- 172.21.15.40:0/1314402100 shutdown_connections 2023-10-18T08:54:56.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.636+0000 7fb086ffd640 1 --2- 172.21.15.40:0/1314402100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0940e31a0 0x7fb0940dd920 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.636+0000 7fb086ffd640 1 --2- 172.21.15.40:0/1314402100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb094053fe0 0x7fb0941790b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.636+0000 7fb086ffd640 1 --2- 172.21.15.40:0/1314402100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb094038300 0x7fb0940e64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.636+0000 7fb086ffd640 1 -- 172.21.15.40:0/1314402100 >> 172.21.15.40:0/1314402100 conn(0x7fb094009f80 msgr2=0x7fb0940351d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.638+0000 7fb086ffd640 1 -- 172.21.15.40:0/1314402100 shutdown_connections 2023-10-18T08:54:56.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.638+0000 7fb086ffd640 1 -- 172.21.15.40:0/1314402100 wait complete. 2023-10-18T08:54:56.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.638+0000 7f727affd640 1 -- 172.21.15.40:0/3255749880 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f727403b040 con 0x7f72840cc0a0 2023-10-18T08:54:56.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.638+0000 7f727affd640 1 -- 172.21.15.40:0/3255749880 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7274046900 con 0x7f72840cc0a0 2023-10-18T08:54:56.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.653+0000 7f728c67c640 1 -- 172.21.15.40:0/3255749880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f726c005250 con 0x7f72840cc0a0 2023-10-18T08:54:56.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.665+0000 7f727affd640 1 -- 172.21.15.40:0/3255749880 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f727404b060 con 0x7f72840cc0a0 2023-10-18T08:54:56.668 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:56.665+0000 7f700aa30040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T08:54:56.688 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi040') 2023-10-18T08:54:56.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.689+0000 7fad85f47640 1 -- 172.21.15.40:0/2203767996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) v1 -- 0x7fad800d1fb0 con 0x7fad800cdb70 2023-10-18T08:54:56.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.690+0000 7f5760ff9640 1 -- 172.21.15.40:0/3373651242 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) v1 -- 0x7f574c0050c0 con 0x7f57740cc230 2023-10-18T08:54:56.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.703+0000 7fad7d7fa640 1 -- 172.21.15.40:0/2203767996 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v8)=0 v8) v1 ==== 106+0+0 (secure 0 0 0) 0x7fad7004f0b0 con 0x7fad800cdb70 2023-10-18T08:54:56.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.705+0000 7fad85f47640 1 -- 172.21.15.40:0/2203767996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad800cdb70 msgr2=0x7fad800c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.705+0000 7fad85f47640 1 --2- 172.21.15.40:0/2203767996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad800cdb70 0x7fad800c75c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fad70009950 tx=0x7fad7002dec0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.708+0000 7fad85f47640 1 -- 172.21.15.40:0/2203767996 shutdown_connections 2023-10-18T08:54:56.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.708+0000 7fad85f47640 1 --2- 172.21.15.40:0/2203767996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad8015b420 0x7fad800b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.708+0000 7fad85f47640 1 --2- 172.21.15.40:0/2203767996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad800cfee0 0x7fad80159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.708+0000 7fad85f47640 1 --2- 172.21.15.40:0/2203767996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad800cdb70 0x7fad800c75c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.708+0000 7fad85f47640 1 -- 172.21.15.40:0/2203767996 >> 172.21.15.40:0/2203767996 conn(0x7fad8009a590 msgr2=0x7fad800cc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.709+0000 7fad85f47640 1 -- 172.21.15.40:0/2203767996 shutdown_connections 2023-10-18T08:54:56.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.713+0000 7fad85f47640 1 -- 172.21.15.40:0/2203767996 wait complete. 2023-10-18T08:54:56.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.714+0000 7f29b3c39640 1 -- 172.21.15.40:0/1805099757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29ac071050 msgr2=0x7f29ac0cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.714+0000 7f29b3c39640 1 --2- 172.21.15.40:0/1805099757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29ac071050 0x7f29ac0cae80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f29ac06c3d0 tx=0x7f299c03cc50 comp rx=0 tx=0).stop 2023-10-18T08:54:56.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.716+0000 7f29b3c39640 1 -- 172.21.15.40:0/1805099757 shutdown_connections 2023-10-18T08:54:56.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.716+0000 7f29b3c39640 1 --2- 172.21.15.40:0/1805099757 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29ac070160 0x7f29ac0705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.716+0000 7f29b3c39640 1 --2- 172.21.15.40:0/1805099757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29ac071050 0x7f29ac0cae80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.716+0000 7f29b3c39640 1 --2- 172.21.15.40:0/1805099757 >> v2:172.21.15.40:3300/0 conn(0x7f29ac0b0070 0x7f29ac06bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.716+0000 7f29b3c39640 1 -- 172.21.15.40:0/1805099757 >> 172.21.15.40:0/1805099757 conn(0x7f29ac0ccf80 msgr2=0x7f29ac0cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.716+0000 7f29b3c39640 1 -- 172.21.15.40:0/1805099757 shutdown_connections 2023-10-18T08:54:56.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.716+0000 7f29b3c39640 1 -- 172.21.15.40:0/1805099757 wait complete. 2023-10-18T08:54:56.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.722+0000 7f5762ffd640 1 -- 172.21.15.40:0/3373651242 <== mon.2 v2:172.21.15.144:3301/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v9)=0 v9) v1 ==== 117+0+0 (secure 0 0 0) 0x7f5768017070 con 0x7f57740cc230 2023-10-18T08:54:56.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.725+0000 7f5760ff9640 1 -- 172.21.15.40:0/3373651242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57740cc230 msgr2=0x7f57740acb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.725+0000 7f5760ff9640 1 --2- 172.21.15.40:0/3373651242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57740cc230 0x7f57740acb90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f576800c4a0 tx=0x7f576800c960 comp rx=0 tx=0).stop 2023-10-18T08:54:56.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.722+0000 7f29b3c39640 1 Processor -- start 2023-10-18T08:54:56.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.728+0000 7f29b3c39640 1 -- start start 2023-10-18T08:54:56.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.732+0000 7f5760ff9640 1 -- 172.21.15.40:0/3373651242 shutdown_connections 2023-10-18T08:54:56.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.732+0000 7f5760ff9640 1 --2- 172.21.15.40:0/3373651242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5774157820 0x7f57740ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.732+0000 7f5760ff9640 1 --2- 172.21.15.40:0/3373651242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57740cc230 0x7f57740acb90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.732+0000 7f5760ff9640 1 --2- 172.21.15.40:0/3373651242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f577406bad0 0x7f5774159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.732+0000 7f5760ff9640 1 -- 172.21.15.40:0/3373651242 >> 172.21.15.40:0/3373651242 conn(0x7f57740ccf80 msgr2=0x7f57740ad9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.736+0000 7f5760ff9640 1 -- 172.21.15.40:0/3373651242 shutdown_connections 2023-10-18T08:54:56.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.737+0000 7f5760ff9640 1 -- 172.21.15.40:0/3373651242 wait complete. 2023-10-18T08:54:56.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.741+0000 7f29b3c39640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29ac070160 0x7f29ac1f0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.745+0000 7f29b3c39640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29ac071050 0x7f29ac15f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.745+0000 7f29b3c39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29ac0b0070 0x7f29ac161cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.745+0000 7f29b19d5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29ac070160 0x7f29ac1f0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.745+0000 7f29b19d5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29ac070160 0x7f29ac1f0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:59086/0 (socket says 172.21.15.40:59086) 2023-10-18T08:54:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.746+0000 7f29b21d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29ac0b0070 0x7f29ac161cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.746+0000 7f29b21d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29ac0b0070 0x7f29ac161cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43208/0 (socket says 172.21.15.40:43208) 2023-10-18T08:54:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.746+0000 7f29b19d5640 1 -- 172.21.15.40:0/1988801713 learned_addr learned my addr 172.21.15.40:0/1988801713 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.746+0000 7f29b3c39640 1 -- 172.21.15.40:0/1988801713 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac1f0680 con 0x7f29ac0b0070 2023-10-18T08:54:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.746+0000 7f29b3c39640 1 -- 172.21.15.40:0/1988801713 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac160870 con 0x7f29ac071050 2023-10-18T08:54:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.746+0000 7f29b3c39640 1 -- 172.21.15.40:0/1988801713 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac1eefa0 con 0x7f29ac070160 2023-10-18T08:54:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.746+0000 7f29b11d4640 1 --2- 172.21.15.40:0/1988801713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29ac071050 0x7f29ac15f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.751+0000 7f29b19d5640 1 -- 172.21.15.40:0/1988801713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29ac071050 msgr2=0x7f29ac15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.751+0000 7f29b19d5640 1 --2- 172.21.15.40:0/1988801713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29ac071050 0x7f29ac15f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.751+0000 7f29b19d5640 1 -- 172.21.15.40:0/1988801713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29ac0b0070 msgr2=0x7f29ac161cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.751+0000 7f29b19d5640 1 --2- 172.21.15.40:0/1988801713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29ac0b0070 0x7f29ac161cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.752+0000 7f29b19d5640 1 -- 172.21.15.40:0/1988801713 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c009490 con 0x7f29ac070160 2023-10-18T08:54:56.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.753+0000 7f4eff7b7640 1 -- 172.21.15.40:0/3524579317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ef80cbfc0 msgr2=0x7f4ef80cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.753+0000 7f4eff7b7640 1 --2- 172.21.15.40:0/3524579317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ef80cbfc0 0x7f4ef80cc3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80d2480 tx=0x7f4ef002c200 comp rx=0 tx=0).stop 2023-10-18T08:54:56.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.753+0000 7f29b19d5640 1 --2- 172.21.15.40:0/1988801713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29ac070160 0x7f29ac1f0140 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f29a800b940 tx=0x7f29a800be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:56.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.753+0000 7f29a27fc640 1 -- 172.21.15.40:0/1988801713 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a8008ba0 con 0x7f29ac070160 2023-10-18T08:54:56.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.753+0000 7f29a27fc640 1 -- 172.21.15.40:0/1988801713 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29a8008df0 con 0x7f29ac070160 2023-10-18T08:54:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.753+0000 7f29a27fc640 1 -- 172.21.15.40:0/1988801713 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a8005280 con 0x7f29ac070160 2023-10-18T08:54:56.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.754+0000 7f4eff7b7640 1 -- 172.21.15.40:0/3524579317 shutdown_connections 2023-10-18T08:54:56.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.754+0000 7f4eff7b7640 1 --2- 172.21.15.40:0/3524579317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ef80cbfc0 0x7f4ef80cc3f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.754+0000 7f4eff7b7640 1 --2- 172.21.15.40:0/3524579317 >> v2:172.21.15.40:3300/0 conn(0x7f4ef80cfc10 0x7f4ef80d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.754+0000 7f4eff7b7640 1 -- 172.21.15.40:0/3524579317 >> 172.21.15.40:0/3524579317 conn(0x7f4ef808ae20 msgr2=0x7f4ef809a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.754+0000 7f4eff7b7640 1 -- 172.21.15.40:0/3524579317 shutdown_connections 2023-10-18T08:54:56.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.758+0000 7f4eff7b7640 1 -- 172.21.15.40:0/3524579317 wait complete. 2023-10-18T08:54:56.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.754+0000 7f29b3c39640 1 -- 172.21.15.40:0/1988801713 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ac1ef890 con 0x7f29ac070160 2023-10-18T08:54:56.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.754+0000 7f29b3c39640 1 -- 172.21.15.40:0/1988801713 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ac1ee9e0 con 0x7f29ac070160 2023-10-18T08:54:56.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.762+0000 7f29a27fc640 1 -- 172.21.15.40:0/1988801713 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f29a80053e0 con 0x7f29ac070160 2023-10-18T08:54:56.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.762+0000 7f29a27fc640 1 -- 172.21.15.40:0/1988801713 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f29a801e440 con 0x7f29ac070160 2023-10-18T08:54:56.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.766+0000 7f2997fff640 1 -- 172.21.15.40:0/1988801713 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ac0b09c0 con 0x7f29ac070160 2023-10-18T08:54:56.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.770+0000 7f4eff7b7640 1 Processor -- start 2023-10-18T08:54:56.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.770+0000 7f4eff7b7640 1 -- start start 2023-10-18T08:54:56.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.774+0000 7f29a27fc640 1 -- 172.21.15.40:0/1988801713 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f29a8019020 con 0x7f29ac070160 2023-10-18T08:54:56.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.775+0000 7f4eff7b7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ef80cbfc0 0x7f4ef8156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.775+0000 7f4eff7b7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ef80cfc10 0x7f4ef80ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.776+0000 7f4eff7b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ef80d1360 0x7f4ef80caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.776+0000 7f4efcd52640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ef80cfc10 0x7f4ef80ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.776+0000 7f4efcd52640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ef80cfc10 0x7f4ef80ca990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:59100/0 (socket says 172.21.15.40:59100) 2023-10-18T08:54:56.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.776+0000 7f4efdd54640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ef80d1360 0x7f4ef80caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.777+0000 7f4efdd54640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ef80d1360 0x7f4ef80caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43222/0 (socket says 172.21.15.40:43222) 2023-10-18T08:54:56.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.777+0000 7f4efcd52640 1 -- 172.21.15.40:0/2340016990 learned_addr learned my addr 172.21.15.40:0/2340016990 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:56.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.781+0000 7f4eff7b7640 1 -- 172.21.15.40:0/2340016990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef80b5670 con 0x7f4ef80d1360 2023-10-18T08:54:56.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.781+0000 7f4eff7b7640 1 -- 172.21.15.40:0/2340016990 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef8156e50 con 0x7f4ef80cbfc0 2023-10-18T08:54:56.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.781+0000 7f4eff7b7640 1 -- 172.21.15.40:0/2340016990 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef80c90f0 con 0x7f4ef80cfc10 2023-10-18T08:54:56.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.782+0000 7f4efdd54640 1 -- 172.21.15.40:0/2340016990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ef80cfc10 msgr2=0x7f4ef80ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.782+0000 7f4efdd54640 1 --2- 172.21.15.40:0/2340016990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ef80cfc10 0x7f4ef80ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.782+0000 7f4efdd54640 1 -- 172.21.15.40:0/2340016990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ef80cbfc0 msgr2=0x7f4ef8156910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T08:54:56.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.782+0000 7f4efdd54640 1 --2- 172.21.15.40:0/2340016990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ef80cbfc0 0x7f4ef8156910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.782+0000 7f4efdd54640 1 -- 172.21.15.40:0/2340016990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef0009900 con 0x7f4ef80d1360 2023-10-18T08:54:56.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.783+0000 7f4efdd54640 1 --2- 172.21.15.40:0/2340016990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ef80d1360 0x7f4ef80caed0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0033000 tx=0x7f4ef0035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:56.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.792+0000 7f4eedffb640 1 -- 172.21.15.40:0/2340016990 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0034440 con 0x7f4ef80d1360 2023-10-18T08:54:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.793+0000 7f4eedffb640 1 -- 172.21.15.40:0/2340016990 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4ef0004540 con 0x7f4ef80d1360 2023-10-18T08:54:56.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.793+0000 7f4eedffb640 1 -- 172.21.15.40:0/2340016990 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef003f440 con 0x7f4ef80d1360 2023-10-18T08:54:56.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.797+0000 7f4eff7b7640 1 -- 172.21.15.40:0/2340016990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef80c93d0 con 0x7f4ef80d1360 2023-10-18T08:54:56.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.798+0000 7f4eedffb640 1 -- 172.21.15.40:0/2340016990 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4ef003b040 con 0x7f4ef80d1360 2023-10-18T08:54:56.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.798+0000 7f4eff7b7640 1 -- 172.21.15.40:0/2340016990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef80c9990 con 0x7f4ef80d1360 2023-10-18T08:54:56.801 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '1'], check_status=True, hostname='smithi040') 2023-10-18T08:54:56.802 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi040') 2023-10-18T08:54:56.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.799+0000 7f4eedffb640 1 -- 172.21.15.40:0/2340016990 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4ef003fa80 con 0x7f4ef80d1360 2023-10-18T08:54:56.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.808+0000 7f4eff7b7640 1 -- 172.21.15.40:0/2340016990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef8066fd0 con 0x7f4ef80d1360 2023-10-18T08:54:56.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.819+0000 7f4eedffb640 1 -- 172.21.15.40:0/2340016990 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f4ef0043020 con 0x7f4ef80d1360 2023-10-18T08:54:56.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.858+0000 7f8234ff9640 1 -- 172.21.15.40:0/468546022 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) v1 -- 0x7f821c0057d0 con 0x7f823804fca0 2023-10-18T08:54:56.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.856+0000 7fcb903a9640 1 -- 172.21.15.40:0/2680581015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb880ccb30 msgr2=0x7fcb8806caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.857+0000 7fcb903a9640 1 --2- 172.21.15.40:0/2680581015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb880ccb30 0x7fcb8806caa0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcb8806aeb0 tx=0x7fcb7803ce00 comp rx=0 tx=0).stop 2023-10-18T08:54:56.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.863+0000 7fcb903a9640 1 -- 172.21.15.40:0/2680581015 shutdown_connections 2023-10-18T08:54:56.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.863+0000 7fcb903a9640 1 --2- 172.21.15.40:0/2680581015 >> v2:172.21.15.144:3300/0 conn(0x7fcb880ac7e0 0x7fcb880ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.863+0000 7fcb903a9640 1 --2- 172.21.15.40:0/2680581015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb880ccb30 0x7fcb8806caa0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.863+0000 7fcb903a9640 1 --2- 172.21.15.40:0/2680581015 >> v2:172.21.15.40:3300/0 conn(0x7fcb880cc750 0x7fcb8815f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.864+0000 7fcb903a9640 1 -- 172.21.15.40:0/2680581015 >> 172.21.15.40:0/2680581015 conn(0x7fcb880aed70 msgr2=0x7fcb880af180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.864+0000 7fcb903a9640 1 -- 172.21.15.40:0/2680581015 shutdown_connections 2023-10-18T08:54:56.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.864+0000 7fcb903a9640 1 -- 172.21.15.40:0/2680581015 wait complete. 2023-10-18T08:54:56.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.867+0000 7fcb903a9640 1 Processor -- start 2023-10-18T08:54:56.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.867+0000 7fcb903a9640 1 -- start start 2023-10-18T08:54:56.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.868+0000 7f8236ffd640 1 -- 172.21.15.40:0/468546022 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v10)=0 v10) v1 ==== 127+0+0 (secure 0 0 0) 0x7f823c058030 con 0x7f823804fca0 2023-10-18T08:54:56.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.873+0000 7f8234ff9640 1 -- 172.21.15.40:0/468546022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823804fca0 msgr2=0x7f82380cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.873+0000 7f8234ff9640 1 --2- 172.21.15.40:0/468546022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823804fca0 0x7f82380cbfe0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f823c000c00 tx=0x7f823c0045c0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.873+0000 7f8234ff9640 1 -- 172.21.15.40:0/468546022 shutdown_connections 2023-10-18T08:54:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.873+0000 7f8234ff9640 1 --2- 172.21.15.40:0/468546022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82380df3c0 0x7f82381692b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.873+0000 7f8234ff9640 1 --2- 172.21.15.40:0/468546022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82380531f0 0x7f82381678c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.873+0000 7f8234ff9640 1 --2- 172.21.15.40:0/468546022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f823804fca0 0x7f82380cbfe0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.873+0000 7f8234ff9640 1 -- 172.21.15.40:0/468546022 >> 172.21.15.40:0/468546022 conn(0x7f8238054c60 msgr2=0x7f8238039210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.873+0000 7f8234ff9640 1 -- 172.21.15.40:0/468546022 shutdown_connections 2023-10-18T08:54:56.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.876+0000 7f8234ff9640 1 -- 172.21.15.40:0/468546022 wait complete. 2023-10-18T08:54:56.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.872+0000 7fdef3fff640 1 -- 172.21.15.40:0/3913759978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdef40aec00 msgr2=0x7fdef40aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.872+0000 7fdef3fff640 1 --2- 172.21.15.40:0/3913759978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdef40aec00 0x7fdef40aefe0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009830 tx=0x7fdee402bc60 comp rx=0 tx=0).stop 2023-10-18T08:54:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.876+0000 7fdef3fff640 1 -- 172.21.15.40:0/3913759978 shutdown_connections 2023-10-18T08:54:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.876+0000 7fdef3fff640 1 --2- 172.21.15.40:0/3913759978 >> v2:172.21.15.40:3300/0 conn(0x7fdef416bc60 0x7fdef416c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.876+0000 7fdef3fff640 1 --2- 172.21.15.40:0/3913759978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdef40aec00 0x7fdef40aefe0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.876+0000 7fdef3fff640 1 -- 172.21.15.40:0/3913759978 >> 172.21.15.40:0/3913759978 conn(0x7fdef4069b10 msgr2=0x7fdef4069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.876+0000 7fdef3fff640 1 -- 172.21.15.40:0/3913759978 shutdown_connections 2023-10-18T08:54:56.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.876+0000 7fdef3fff640 1 -- 172.21.15.40:0/3913759978 wait complete. 2023-10-18T08:54:56.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.876+0000 7fc8798c3640 1 -- 172.21.15.40:0/4011621142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc874142ad0 msgr2=0x7fc8740aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.876+0000 7fc8798c3640 1 --2- 172.21.15.40:0/4011621142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc874142ad0 0x7fc8740aec00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc864009900 tx=0x7fc86402be00 comp rx=0 tx=0).stop 2023-10-18T08:54:56.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.877+0000 7fc8798c3640 1 -- 172.21.15.40:0/4011621142 shutdown_connections 2023-10-18T08:54:56.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.877+0000 7fc8798c3640 1 --2- 172.21.15.40:0/4011621142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc874142ad0 0x7fc8740aec00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.877+0000 7fc8798c3640 1 -- 172.21.15.40:0/4011621142 >> 172.21.15.40:0/4011621142 conn(0x7fc874069b10 msgr2=0x7fc874069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.877+0000 7fc8798c3640 1 -- 172.21.15.40:0/4011621142 shutdown_connections 2023-10-18T08:54:56.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.875+0000 7fcb903a9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb880ac7e0 0x7fcb881547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.879+0000 7fcb903a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb880cc750 0x7fcb88159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.879+0000 7fcb903a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb880ccb30 0x7fcb88155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.879+0000 7fdef3fff640 1 Processor -- start 2023-10-18T08:54:56.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.879+0000 7fdef3fff640 1 -- start start 2023-10-18T08:54:56.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.880+0000 7fcb8e145640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb880ac7e0 0x7fcb881547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.880+0000 7fcb8e145640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb880ac7e0 0x7fcb881547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:59144/0 (socket says 172.21.15.40:59144) 2023-10-18T08:54:56.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.877+0000 7fc8798c3640 1 -- 172.21.15.40:0/4011621142 wait complete. 2023-10-18T08:54:56.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.880+0000 7fcb903a9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb881543d0 con 0x7fcb880cc750 2023-10-18T08:54:56.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.880+0000 7fcb903a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb88154d30 con 0x7fcb880ccb30 2023-10-18T08:54:56.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.880+0000 7fcb903a9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb88155aa0 con 0x7fcb880ac7e0 2023-10-18T08:54:56.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.880+0000 7fcb8e145640 1 -- 172.21.15.40:0/4290143094 learned_addr learned my addr 172.21.15.40:0/4290143094 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:56.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.880+0000 7fcb8e946640 1 --2- 172.21.15.40:0/4290143094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb880ccb30 0x7fcb88155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.881+0000 7fcb8d944640 1 --2- 172.21.15.40:0/4290143094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb880cc750 0x7fcb88159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.881+0000 7fcb8e145640 1 -- 172.21.15.40:0/4290143094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb880ccb30 msgr2=0x7fcb88155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.881+0000 7fcb8e145640 1 --2- 172.21.15.40:0/4290143094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb880ccb30 0x7fcb88155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.881+0000 7fcb8e145640 1 -- 172.21.15.40:0/4290143094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb880cc750 msgr2=0x7fcb88159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.881+0000 7fdef3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdef40aec00 0x7fdef415cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.882+0000 7fdef3fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdef40aff70 0x7fdef40ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.881+0000 7fbe69f47640 1 -- 172.21.15.40:0/3906008239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe6415af70 msgr2=0x7fbe6406c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.882+0000 7fbe69f47640 1 --2- 172.21.15.40:0/3906008239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe6415af70 0x7fbe6406c100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbe640cdf20 tx=0x7fbe5402c070 comp rx=0 tx=0).stop 2023-10-18T08:54:56.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.882+0000 7fdef3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdef416bc60 0x7fdef40ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.882+0000 7fc8798c3640 1 Processor -- start 2023-10-18T08:54:56.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.882+0000 7fdef3fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef4084270 con 0x7fdef416bc60 2023-10-18T08:54:56.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.883+0000 7fc8798c3640 1 -- start start 2023-10-18T08:54:56.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.882+0000 7fdef3fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef416a770 con 0x7fdef40aec00 2023-10-18T08:54:56.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.883+0000 7fdef3fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef415bab0 con 0x7fdef40aff70 2023-10-18T08:54:56.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.883+0000 7fbe69f47640 1 -- 172.21.15.40:0/3906008239 shutdown_connections 2023-10-18T08:54:56.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.883+0000 7fbe69f47640 1 --2- 172.21.15.40:0/3906008239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe6415af70 0x7fbe6406c100 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.883+0000 7fbe69f47640 1 --2- 172.21.15.40:0/3906008239 >> v2:172.21.15.40:3300/0 conn(0x7fbe640c9620 0x7fbe6406bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.883+0000 7fbe69f47640 1 -- 172.21.15.40:0/3906008239 >> 172.21.15.40:0/3906008239 conn(0x7fbe6406ca10 msgr2=0x7fbe6406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.884+0000 7fbe69f47640 1 -- 172.21.15.40:0/3906008239 shutdown_connections 2023-10-18T08:54:56.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.884+0000 7fbe69f47640 1 -- 172.21.15.40:0/3906008239 wait complete. 2023-10-18T08:54:56.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.882+0000 7fcb8e145640 1 --2- 172.21.15.40:0/4290143094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb880cc750 0x7fcb88159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.882+0000 7fcb8e145640 1 -- 172.21.15.40:0/4290143094 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7801a570 con 0x7fcb880ac7e0 2023-10-18T08:54:56.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.886+0000 7fcb8d944640 1 --2- 172.21.15.40:0/4290143094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb880cc750 0x7fcb88159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:54:56.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.886+0000 7fc8798c3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8740ac7e0 0x7fc87406ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.886+0000 7fc8798c3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8740b1010 0x7fc874137960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.886+0000 7fc8798c3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc874142ad0 0x7fc874137ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.887+0000 7fc8798c3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740acbe0 con 0x7fc874142ad0 2023-10-18T08:54:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.887+0000 7fc8790c2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc874142ad0 0x7fc874137ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.887+0000 7fc8788c1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8740ac7e0 0x7fc87406ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.887+0000 7fc873fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8740b1010 0x7fc874137960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.887+0000 7fc873fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8740b1010 0x7fc874137960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.40:55946/0 (socket says 172.21.15.40:55946) 2023-10-18T08:54:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.887+0000 7fc873fff640 1 -- 172.21.15.40:0/3131145573 learned_addr learned my addr 172.21.15.40:0/3131145573 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.887+0000 7fc8790c2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc874142ad0 0x7fc874137ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43266/0 (socket says 172.21.15.40:43266) 2023-10-18T08:54:56.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.887+0000 7fdef37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdef416bc60 0x7fdef40ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.887+0000 7fdef37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdef416bc60 0x7fdef40ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43262/0 (socket says 172.21.15.40:43262) 2023-10-18T08:54:56.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.887+0000 7fdef37fe640 1 -- 172.21.15.40:0/3467462250 learned_addr learned my addr 172.21.15.40:0/3467462250 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:56.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.887+0000 7fc8798c3640 1 -- 172.21.15.40:0/3131145573 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc874139b50 con 0x7fc8740b1010 2023-10-18T08:54:56.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.887+0000 7fcb8e145640 1 --2- 172.21.15.40:0/4290143094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb880ac7e0 0x7fcb881547f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400b450 tx=0x7fcb8400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:56.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.887+0000 7fc8798c3640 1 -- 172.21.15.40:0/3131145573 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87413a3b0 con 0x7fc8740ac7e0 2023-10-18T08:54:56.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.888+0000 7fbe69f47640 1 Processor -- start 2023-10-18T08:54:56.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.888+0000 7fcb7effd640 1 -- 172.21.15.40:0/4290143094 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84011890 con 0x7fcb880ac7e0 2023-10-18T08:54:56.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.888+0000 7fc873fff640 1 -- 172.21.15.40:0/3131145573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8740ac7e0 msgr2=0x7fc87406ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.888+0000 7fcb903a9640 1 -- 172.21.15.40:0/4290143094 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb88155d80 con 0x7fcb880ac7e0 2023-10-18T08:54:56.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.888+0000 7fc873fff640 1 --2- 172.21.15.40:0/3131145573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8740ac7e0 0x7fc87406ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.888+0000 7fc873fff640 1 -- 172.21.15.40:0/3131145573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc874142ad0 msgr2=0x7fc874137ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.888+0000 7fc873fff640 1 --2- 172.21.15.40:0/3131145573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc874142ad0 0x7fc874137ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.888+0000 7fc873fff640 1 -- 172.21.15.40:0/3131145573 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc864009560 con 0x7fc8740b1010 2023-10-18T08:54:56.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.889+0000 7fc873fff640 1 --2- 172.21.15.40:0/3131145573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8740b1010 0x7fc874137960 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc86800c490 tx=0x7fc86800c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:56.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.889+0000 7fc8717fa640 1 -- 172.21.15.40:0/3131145573 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8680052d0 con 0x7fc8740b1010 2023-10-18T08:54:56.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.890+0000 7fc8717fa640 1 -- 172.21.15.40:0/3131145573 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc868005430 con 0x7fc8740b1010 2023-10-18T08:54:56.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.890+0000 7fc8798c3640 1 -- 172.21.15.40:0/3131145573 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc874137530 con 0x7fc8740b1010 2023-10-18T08:54:56.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.888+0000 7fbe69f47640 1 -- start start 2023-10-18T08:54:56.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.890+0000 7fbe69f47640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe64070b60 0x7fbe64153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.890+0000 7fbe69f47640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe640c9620 0x7fbe64154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.890+0000 7fbe69f47640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe6415af70 0x7fbe641f9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.890+0000 7fbe69f47640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe64154a40 con 0x7fbe640c9620 2023-10-18T08:54:56.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.890+0000 7fbe69f47640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6415a4f0 con 0x7fbe6415af70 2023-10-18T08:54:56.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.890+0000 7fbe69f47640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe64158a20 con 0x7fbe64070b60 2023-10-18T08:54:56.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.891+0000 7fbe69746640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe6415af70 0x7fbe641f9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.891+0000 7fc8717fa640 1 -- 172.21.15.40:0/3131145573 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868012950 con 0x7fc8740b1010 2023-10-18T08:54:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.891+0000 7fdef37fe640 1 -- 172.21.15.40:0/3467462250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdef40aff70 msgr2=0x7fdef40ad4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T08:54:56.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.891+0000 7fdef37fe640 1 --2- 172.21.15.40:0/3467462250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdef40aff70 0x7fdef40ad4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.891+0000 7fdef37fe640 1 -- 172.21.15.40:0/3467462250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdef40aec00 msgr2=0x7fdef415cb50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T08:54:56.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.891+0000 7fdef37fe640 1 --2- 172.21.15.40:0/3467462250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdef40aec00 0x7fdef415cb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.891+0000 7fdef37fe640 1 -- 172.21.15.40:0/3467462250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee4009490 con 0x7fdef416bc60 2023-10-18T08:54:56.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.891+0000 7fbe69746640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe6415af70 0x7fbe641f9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.40:55956/0 (socket says 172.21.15.40:55956) 2023-10-18T08:54:56.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.893+0000 7fc8798c3640 1 -- 172.21.15.40:0/3131145573 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87413a0d0 con 0x7fc8740b1010 2023-10-18T08:54:56.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.888+0000 7fcb903a9640 1 -- 172.21.15.40:0/4290143094 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb88153ff0 con 0x7fcb880ac7e0 2023-10-18T08:54:56.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.888+0000 7fcb7effd640 1 -- 172.21.15.40:0/4290143094 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb84011ae0 con 0x7fcb880ac7e0 2023-10-18T08:54:56.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.888+0000 7fcb7effd640 1 -- 172.21.15.40:0/4290143094 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84011890 con 0x7fcb880ac7e0 2023-10-18T08:54:56.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.894+0000 7fcb7effd640 1 -- 172.21.15.40:0/4290143094 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcb8401a710 con 0x7fcb880ac7e0 2023-10-18T08:54:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.894+0000 7fcb7effd640 1 -- 172.21.15.40:0/4290143094 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcb84010960 con 0x7fcb880ac7e0 2023-10-18T08:54:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.893+0000 7fbe63fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe640c9620 0x7fbe64154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.893+0000 7fbe68f45640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe64070b60 0x7fbe64153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.894+0000 7fbe68f45640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe64070b60 0x7fbe64153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:59164/0 (socket says 172.21.15.40:59164) 2023-10-18T08:54:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.894+0000 7fbe68f45640 1 -- 172.21.15.40:0/1367543567 learned_addr learned my addr 172.21.15.40:0/1367543567 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.894+0000 7fc8717fa640 1 -- 172.21.15.40:0/3131145573 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc8680035f0 con 0x7fc8740b1010 2023-10-18T08:54:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.894+0000 7fc8717fa640 1 -- 172.21.15.40:0/3131145573 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc86801ba80 con 0x7fc8740b1010 2023-10-18T08:54:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.896+0000 7fbe63fff640 1 -- 172.21.15.40:0/1367543567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe64070b60 msgr2=0x7fbe64153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.896+0000 7fbe63fff640 1 --2- 172.21.15.40:0/1367543567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe64070b60 0x7fbe64153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.896+0000 7fbe63fff640 1 -- 172.21.15.40:0/1367543567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe6415af70 msgr2=0x7fbe641f9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.898+0000 7fc85affd640 1 -- 172.21.15.40:0/3131145573 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc874202050 con 0x7fc8740b1010 2023-10-18T08:54:56.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.896+0000 7fbe63fff640 1 --2- 172.21.15.40:0/1367543567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe6415af70 0x7fbe641f9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.898+0000 7fcb903a9640 1 -- 172.21.15.40:0/4290143094 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb70005250 con 0x7fcb880ac7e0 2023-10-18T08:54:56.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.899+0000 7fdef37fe640 1 --2- 172.21.15.40:0/3467462250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdef416bc60 0x7fdef40ada30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00eca0 tx=0x7fdeec00b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.899+0000 7fdedb7fe640 1 -- 172.21.15.40:0/3467462250 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec013070 con 0x7fdef416bc60 2023-10-18T08:54:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.899+0000 7fdedb7fe640 1 -- 172.21.15.40:0/3467462250 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdeec004b20 con 0x7fdef416bc60 2023-10-18T08:54:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.899+0000 7fdedb7fe640 1 -- 172.21.15.40:0/3467462250 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec005020 con 0x7fdef416bc60 2023-10-18T08:54:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.900+0000 7fdef3fff640 1 -- 172.21.15.40:0/3467462250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef415bd90 con 0x7fdef416bc60 2023-10-18T08:54:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.900+0000 7fdef3fff640 1 -- 172.21.15.40:0/3467462250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef415c350 con 0x7fdef416bc60 2023-10-18T08:54:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.901+0000 7fdedb7fe640 1 -- 172.21.15.40:0/3467462250 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdeec005480 con 0x7fdef416bc60 2023-10-18T08:54:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.901+0000 7fdedb7fe640 1 -- 172.21.15.40:0/3467462250 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdeec017850 con 0x7fdef416bc60 2023-10-18T08:54:56.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.896+0000 7fbe63fff640 1 -- 172.21.15.40:0/1367543567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe54009840 con 0x7fbe640c9620 2023-10-18T08:54:56.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.902+0000 7fdef3fff640 1 -- 172.21.15.40:0/3467462250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef40cbed0 con 0x7fdef416bc60 2023-10-18T08:54:56.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.902+0000 7fbe63fff640 1 --2- 172.21.15.40:0/1367543567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe640c9620 0x7fbe64154500 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009000 tx=0x7fbe540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:56.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.904+0000 7fcb7effd640 1 -- 172.21.15.40:0/4290143094 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fcb84015020 con 0x7fcb880ac7e0 2023-10-18T08:54:56.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.905+0000 7fc8717fa640 1 -- 172.21.15.40:0/3131145573 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fc8680038a0 con 0x7fc8740b1010 2023-10-18T08:54:56.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.907+0000 7fe0f7ae5640 1 -- 172.21.15.40:0/2949514855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc0047c0 msgr2=0x7fe0dc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.908+0000 7fe0f487f640 1 -- 172.21.15.40:0/2949514855 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e002d700 con 0x7fe0dc0047c0 2023-10-18T08:54:56.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.908+0000 7fe0f7ae5640 1 --2- 172.21.15.40:0/2949514855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc0047c0 0x7fe0dc004c40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009880 tx=0x7fe0dc0057f0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.911+0000 7fdedb7fe640 1 -- 172.21.15.40:0/3467462250 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fdeec025060 con 0x7fdef416bc60 2023-10-18T08:54:56.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.912+0000 7fe0f7ae5640 1 -- 172.21.15.40:0/2949514855 shutdown_connections 2023-10-18T08:54:56.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.912+0000 7fe0f7ae5640 1 --2- 172.21.15.40:0/2949514855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc0047c0 0x7fe0dc004c40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.912+0000 7fe0f7ae5640 1 -- 172.21.15.40:0/2949514855 >> 172.21.15.40:0/2949514855 conn(0x7fe0e803aa00 msgr2=0x7fe0e803ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.913+0000 7fe0f7ae5640 1 -- 172.21.15.40:0/2949514855 shutdown_connections 2023-10-18T08:54:56.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.913+0000 7fbe617fa640 1 -- 172.21.15.40:0/1367543567 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5403e460 con 0x7fbe640c9620 2023-10-18T08:54:56.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.913+0000 7fbe69f47640 1 -- 172.21.15.40:0/1367543567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe64158d00 con 0x7fbe640c9620 2023-10-18T08:54:56.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.913+0000 7fbe617fa640 1 -- 172.21.15.40:0/1367543567 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbe5403e6b0 con 0x7fbe640c9620 2023-10-18T08:54:56.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.913+0000 7fbe617fa640 1 -- 172.21.15.40:0/1367543567 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5403e460 con 0x7fbe640c9620 2023-10-18T08:54:56.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.914+0000 7fbe69f47640 1 -- 172.21.15.40:0/1367543567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe641592c0 con 0x7fbe640c9620 2023-10-18T08:54:56.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.917+0000 7fbe617fa640 1 -- 172.21.15.40:0/1367543567 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbe5403eb80 con 0x7fbe640c9620 2023-10-18T08:54:56.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.916+0000 7fe0f7ae5640 1 -- 172.21.15.40:0/2949514855 wait complete. 2023-10-18T08:54:56.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.918+0000 7fbe69f47640 1 -- 172.21.15.40:0/1367543567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe641594b0 con 0x7fbe640c9620 2023-10-18T08:54:56.928 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi040') 2023-10-18T08:54:56.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.938+0000 7f728c67c640 1 -- 172.21.15.40:0/3255749880 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) v1 -- 0x7f726c0050c0 con 0x7f72840cc0a0 2023-10-18T08:54:56.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.942+0000 7fbe617fa640 1 -- 172.21.15.40:0/1367543567 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbe54034430 con 0x7fbe640c9620 2023-10-18T08:54:56.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.944+0000 7fe0f7ae5640 1 Processor -- start 2023-10-18T08:54:56.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.945+0000 7fe0f7ae5640 1 -- start start 2023-10-18T08:54:56.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.946+0000 7fe0f7ae5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc0047c0 0x7fe0e804ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.946+0000 7fe0f7ae5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0e80d8f30 0x7fe0e80e37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.950+0000 7f85a28df640 1 -- 172.21.15.40:0/2729592122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859c06d390 msgr2=0x7f859c0cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.950+0000 7f85a28df640 1 --2- 172.21.15.40:0/2729592122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859c06d390 0x7f859c0cc5f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f859c0719a0 tx=0x7f858c02bbb0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.952+0000 7fbe617fa640 1 -- 172.21.15.40:0/1367543567 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fbe54065b40 con 0x7fbe640c9620 2023-10-18T08:54:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.951+0000 7f3fcd56e640 1 -- 172.21.15.40:0/3918924284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc806b630 msgr2=0x7f3fc80cc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.951+0000 7f3fcd56e640 1 --2- 172.21.15.40:0/3918924284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc806b630 0x7f3fc80cc650 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0005fd0 tx=0x7f3fc002c080 comp rx=0 tx=0).stop 2023-10-18T08:54:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.955+0000 7f3fcd56e640 1 -- 172.21.15.40:0/3918924284 shutdown_connections 2023-10-18T08:54:56.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.955+0000 7f3fcd56e640 1 --2- 172.21.15.40:0/3918924284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fc806ba10 0x7f3fc815fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.955+0000 7f3fcd56e640 1 --2- 172.21.15.40:0/3918924284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc806b630 0x7f3fc80cc650 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.955+0000 7f3fcd56e640 1 -- 172.21.15.40:0/3918924284 >> 172.21.15.40:0/3918924284 conn(0x7f3fc80ccf80 msgr2=0x7f3fc80cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.955+0000 7f3fcd56e640 1 -- 172.21.15.40:0/3918924284 shutdown_connections 2023-10-18T08:54:56.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.955+0000 7f3fcd56e640 1 -- 172.21.15.40:0/3918924284 wait complete. 2023-10-18T08:54:56.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.957+0000 7f727affd640 1 -- 172.21.15.40:0/3255749880 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v11)=0 v11) v1 ==== 116+0+0 (secure 0 0 0) 0x7f7274041070 con 0x7f72840cc0a0 2023-10-18T08:54:56.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.956+0000 7f85a28df640 1 -- 172.21.15.40:0/2729592122 shutdown_connections 2023-10-18T08:54:56.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.956+0000 7f85a28df640 1 --2- 172.21.15.40:0/2729592122 >> v2:172.21.15.144:3300/0 conn(0x7f859c0aec00 0x7f859c0af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.956+0000 7f85a28df640 1 --2- 172.21.15.40:0/2729592122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859c06d390 0x7f859c0cc5f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.956+0000 7f85a28df640 1 -- 172.21.15.40:0/2729592122 >> 172.21.15.40:0/2729592122 conn(0x7f859c069b10 msgr2=0x7f859c069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.956+0000 7f85a28df640 1 -- 172.21.15.40:0/2729592122 shutdown_connections 2023-10-18T08:54:56.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.956+0000 7f85a28df640 1 -- 172.21.15.40:0/2729592122 wait complete. 2023-10-18T08:54:56.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.947+0000 7fe0f7ae5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0e8039d40 0x7fe0e803a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.947+0000 7fe0f7ae5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e803a700 con 0x7fe0dc0047c0 2023-10-18T08:54:56.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.947+0000 7fe0f7ae5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e80d93b0 con 0x7fe0e8039d40 2023-10-18T08:54:56.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.947+0000 7fe0f7ae5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e80db7b0 con 0x7fe0e80d8f30 2023-10-18T08:54:56.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.947+0000 7fe0f5080640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0e80d8f30 0x7fe0e80e37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.947+0000 7fe0f5080640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0e80d8f30 0x7fe0e80e37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:59182/0 (socket says 172.21.15.40:59182) 2023-10-18T08:54:56.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.947+0000 7fe0f5080640 1 -- 172.21.15.40:0/2189688512 learned_addr learned my addr 172.21.15.40:0/2189688512 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:56.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.949+0000 7fe0f5881640 1 --2- 172.21.15.40:0/2189688512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc0047c0 0x7fe0e804ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.949+0000 7fe0f5881640 1 -- 172.21.15.40:0/2189688512 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0e80d8f30 msgr2=0x7fe0e80e37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.950+0000 7fe0f5881640 1 --2- 172.21.15.40:0/2189688512 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0e80d8f30 0x7fe0e80e37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.950+0000 7fe0f5881640 1 -- 172.21.15.40:0/2189688512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0e8039d40 msgr2=0x7fe0e803a1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T08:54:56.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.950+0000 7fe0f5881640 1 --2- 172.21.15.40:0/2189688512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0e8039d40 0x7fe0e803a1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.950+0000 7fe0f5881640 1 -- 172.21.15.40:0/2189688512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e80dba90 con 0x7fe0dc0047c0 2023-10-18T08:54:56.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.951+0000 7fe0f5881640 1 --2- 172.21.15.40:0/2189688512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc0047c0 0x7fe0e804ebd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec000f60 tx=0x7fe0ec00ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:56.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.955+0000 7fe0e67fc640 1 -- 172.21.15.40:0/2189688512 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec005270 con 0x7fe0dc0047c0 2023-10-18T08:54:56.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.955+0000 7fe0e67fc640 1 -- 172.21.15.40:0/2189688512 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe0ec015960 con 0x7fe0dc0047c0 2023-10-18T08:54:56.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.955+0000 7fe0e67fc640 1 -- 172.21.15.40:0/2189688512 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec015d30 con 0x7fe0dc0047c0 2023-10-18T08:54:56.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.955+0000 7fe0f7ae5640 1 -- 172.21.15.40:0/2189688512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e80cba70 con 0x7fe0dc0047c0 2023-10-18T08:54:56.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.955+0000 7fe0f7ae5640 1 -- 172.21.15.40:0/2189688512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e8050fa0 con 0x7fe0dc0047c0 2023-10-18T08:54:56.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.954+0000 7f80b909b640 1 -- 172.21.15.40:0/779235888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b415b670 msgr2=0x7f80b40ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.954+0000 7f80b909b640 1 --2- 172.21.15.40:0/779235888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b415b670 0x7f80b40ce520 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f80b40ceec0 tx=0x7f809c02bd30 comp rx=0 tx=0).stop 2023-10-18T08:54:56.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.955+0000 7f80b909b640 1 -- 172.21.15.40:0/779235888 shutdown_connections 2023-10-18T08:54:56.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.955+0000 7f80b909b640 1 --2- 172.21.15.40:0/779235888 >> v2:172.21.15.40:3300/0 conn(0x7f80b40d1a70 0x7f80b40cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.955+0000 7f80b909b640 1 --2- 172.21.15.40:0/779235888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b415b670 0x7f80b40ce520 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.955+0000 7f80b909b640 1 -- 172.21.15.40:0/779235888 >> 172.21.15.40:0/779235888 conn(0x7f80b408ae20 msgr2=0x7f80b409a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.958+0000 7f80b909b640 1 -- 172.21.15.40:0/779235888 shutdown_connections 2023-10-18T08:54:56.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.961+0000 7fe0e67fc640 1 -- 172.21.15.40:0/2189688512 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe0ec01b070 con 0x7fe0dc0047c0 2023-10-18T08:54:56.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.958+0000 7f80b909b640 1 -- 172.21.15.40:0/779235888 wait complete. 2023-10-18T08:54:56.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.961+0000 7fe0e67fc640 1 -- 172.21.15.40:0/2189688512 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe0ec00aaf0 con 0x7fe0dc0047c0 2023-10-18T08:54:56.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.961+0000 7f80b909b640 1 Processor -- start 2023-10-18T08:54:56.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.961+0000 7f85a28df640 1 Processor -- start 2023-10-18T08:54:56.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.961+0000 7f7278ff9640 1 -- 172.21.15.40:0/3255749880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72840cc0a0 msgr2=0x7f72841f35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.961+0000 7f7278ff9640 1 --2- 172.21.15.40:0/3255749880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72840cc0a0 0x7f72841f35b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7274033000 tx=0x7f72740038a0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.961+0000 7f80b909b640 1 -- start start 2023-10-18T08:54:56.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.962+0000 7f7278ff9640 1 -- 172.21.15.40:0/3255749880 shutdown_connections 2023-10-18T08:54:56.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.962+0000 7f7278ff9640 1 --2- 172.21.15.40:0/3255749880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f728413fa40 0x7f72841ef110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.962+0000 7f7278ff9640 1 --2- 172.21.15.40:0/3255749880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72840cc0a0 0x7f72841f35b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.962+0000 7f7278ff9640 1 --2- 172.21.15.40:0/3255749880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f728406b3c0 0x7f72841f07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.962+0000 7f7278ff9640 1 -- 172.21.15.40:0/3255749880 >> 172.21.15.40:0/3255749880 conn(0x7f72840ccf80 msgr2=0x7f728415c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:56.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.962+0000 7f85a28df640 1 -- start start 2023-10-18T08:54:56.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.962+0000 7f7278ff9640 1 -- 172.21.15.40:0/3255749880 shutdown_connections 2023-10-18T08:54:56.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.962+0000 7f7278ff9640 1 -- 172.21.15.40:0/3255749880 wait complete. 2023-10-18T08:54:56.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.964+0000 7fe0f7ae5640 1 -- 172.21.15.40:0/2189688512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e80cb8b0 con 0x7fe0dc0047c0 2023-10-18T08:54:56.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.968+0000 7f80b909b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80b40cea60 0x7f80b40c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.971+0000 7f85a28df640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f859c06d390 0x7f859c145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.971+0000 7f85a28df640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859c0aec00 0x7f859c0c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.971+0000 7f85a28df640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f859c0b1740 0x7f859c1f4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.971+0000 7f85a28df640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c0c6550 con 0x7f859c0aec00 2023-10-18T08:54:56.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.971+0000 7f85a28df640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c145a40 con 0x7f859c0b1740 2023-10-18T08:54:56.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.972+0000 7f85a28df640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859c145d20 con 0x7f859c06d390 2023-10-18T08:54:56.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.972+0000 7f80b909b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80b40d1a70 0x7f80b41fee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.969+0000 7f3fcd56e640 1 Processor -- start 2023-10-18T08:54:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.972+0000 7f80aa59c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80b40d1a70 0x7f80b41fee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.972+0000 7f85a18dd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f859c06d390 0x7f859c145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.973+0000 7f80aa59c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80b40d1a70 0x7f80b41fee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:59196/0 (socket says 172.21.15.40:59196) 2023-10-18T08:54:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.972+0000 7f85a20de640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f859c0b1740 0x7f859c1f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.973+0000 7f85a20de640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f859c0b1740 0x7f859c1f4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.40:56014/0 (socket says 172.21.15.40:56014) 2023-10-18T08:54:56.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.973+0000 7f85a20de640 1 -- 172.21.15.40:0/486104265 learned_addr learned my addr 172.21.15.40:0/486104265 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:56.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.973+0000 7f85a20de640 1 -- 172.21.15.40:0/486104265 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f859c06d390 msgr2=0x7f859c145460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.973+0000 7f85a20de640 1 --2- 172.21.15.40:0/486104265 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f859c06d390 0x7f859c145460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.973+0000 7f85a20de640 1 -- 172.21.15.40:0/486104265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859c0aec00 msgr2=0x7f859c0c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.973+0000 7f85a20de640 1 --2- 172.21.15.40:0/486104265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859c0aec00 0x7f859c0c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.974+0000 7f85a20de640 1 -- 172.21.15.40:0/486104265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c0081d0 con 0x7f859c0b1740 2023-10-18T08:54:56.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.974+0000 7f3fcd56e640 1 -- start start 2023-10-18T08:54:56.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.974+0000 7f85a20de640 1 --2- 172.21.15.40:0/486104265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f859c0b1740 0x7f859c1f4020 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f859400bcc0 tx=0x7f8594008600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:56.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.975+0000 7f85927fc640 1 -- 172.21.15.40:0/486104265 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8594019070 con 0x7f859c0b1740 2023-10-18T08:54:56.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.976+0000 7f80aa59c640 1 -- 172.21.15.40:0/2741572192 learned_addr learned my addr 172.21.15.40:0/2741572192 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:56.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.976+0000 7f80b909b640 1 --2- 172.21.15.40:0/2741572192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b415b670 0x7f80b4201c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.976+0000 7fe0e67fc640 1 -- 172.21.15.40:0/2189688512 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fe0ec00acb0 con 0x7fe0dc0047c0 2023-10-18T08:54:56.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.975+0000 7f85927fc640 1 -- 172.21.15.40:0/486104265 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8594004b20 con 0x7f859c0b1740 2023-10-18T08:54:56.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.975+0000 7f85927fc640 1 -- 172.21.15.40:0/486104265 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8594019070 con 0x7f859c0b1740 2023-10-18T08:54:56.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.979+0000 7f85a28df640 1 -- 172.21.15.40:0/486104265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859c146000 con 0x7f859c0b1740 2023-10-18T08:54:56.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.979+0000 7f85a28df640 1 -- 172.21.15.40:0/486104265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859c1465c0 con 0x7f859c0b1740 2023-10-18T08:54:56.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.979+0000 7f85927fc640 1 -- 172.21.15.40:0/486104265 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8594015720 con 0x7f859c0b1740 2023-10-18T08:54:56.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.980+0000 7f85927fc640 1 -- 172.21.15.40:0/486104265 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f859401e8a0 con 0x7f859c0b1740 2023-10-18T08:54:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.980+0000 7f80b2d9d640 1 --2- 172.21.15.40:0/2741572192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80b40cea60 0x7f80b40c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.980+0000 7f80b909b640 1 -- 172.21.15.40:0/2741572192 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b41f5630 con 0x7f80b40cea60 2023-10-18T08:54:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.981+0000 7f80b359e640 1 --2- 172.21.15.40:0/2741572192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b415b670 0x7f80b4201c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.981+0000 7f80b909b640 1 -- 172.21.15.40:0/2741572192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40c8280 con 0x7f80b415b670 2023-10-18T08:54:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.981+0000 7f80b909b640 1 -- 172.21.15.40:0/2741572192 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40c71e0 con 0x7f80b40d1a70 2023-10-18T08:54:56.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.981+0000 7f3fcd56e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fc806b630 0x7f3fc815e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.982+0000 7f3fcd56e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fc806ba10 0x7f3fc806f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.982+0000 7f3fcd56e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc806bfd0 0x7f3fc814cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.982+0000 7f3fc67fc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fc806ba10 0x7f3fc806f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.982+0000 7f3fc67fc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fc806ba10 0x7f3fc806f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:59222/0 (socket says 172.21.15.40:59222) 2023-10-18T08:54:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.982+0000 7f857ffff640 1 -- 172.21.15.40:0/486104265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8578005250 con 0x7f859c0b1740 2023-10-18T08:54:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.983+0000 7f80b359e640 1 -- 172.21.15.40:0/2741572192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80b40d1a70 msgr2=0x7f80b41fee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.983+0000 7f80b359e640 1 --2- 172.21.15.40:0/2741572192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80b40d1a70 0x7f80b41fee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.983+0000 7f80b359e640 1 -- 172.21.15.40:0/2741572192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80b40cea60 msgr2=0x7f80b40c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.983+0000 7f80b359e640 1 --2- 172.21.15.40:0/2741572192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80b40cea60 0x7f80b40c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.983+0000 7f80b359e640 1 -- 172.21.15.40:0/2741572192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c009560 con 0x7f80b415b670 2023-10-18T08:54:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.984+0000 7f80b359e640 1 --2- 172.21.15.40:0/2741572192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b415b670 0x7f80b4201c40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00bdf0 tx=0x7f80ac00bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:56.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.988+0000 7f3fcd56e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc8070040 con 0x7f3fc806b630 2023-10-18T08:54:56.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.988+0000 7f3fcd56e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc8070320 con 0x7f3fc806bfd0 2023-10-18T08:54:56.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.988+0000 7f3fcd56e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc8070600 con 0x7f3fc806ba10 2023-10-18T08:54:56.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.988+0000 7f3fc67fc640 1 -- 172.21.15.40:0/434856922 learned_addr learned my addr 172.21.15.40:0/434856922 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:56.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.989+0000 7f85927fc640 1 -- 172.21.15.40:0/486104265 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f85940159d0 con 0x7f859c0b1740 2023-10-18T08:54:56.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.989+0000 7f3fc6ffd640 1 --2- 172.21.15.40:0/434856922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fc806b630 0x7f3fc815e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:56.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.990+0000 7f3fc6ffd640 1 -- 172.21.15.40:0/434856922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fc806ba10 msgr2=0x7f3fc806f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.990+0000 7f3fc6ffd640 1 --2- 172.21.15.40:0/434856922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fc806ba10 0x7f3fc806f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.990+0000 7f3fc6ffd640 1 -- 172.21.15.40:0/434856922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc806bfd0 msgr2=0x7f3fc814cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.990+0000 7f3fc6ffd640 1 --2- 172.21.15.40:0/434856922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc806bfd0 0x7f3fc814cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.990+0000 7f3fc6ffd640 1 -- 172.21.15.40:0/434856922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc0009840 con 0x7f3fc806b630 2023-10-18T08:54:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.991+0000 7f3fc6ffd640 1 --2- 172.21.15.40:0/434856922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fc806b630 0x7f3fc815e870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009000 tx=0x7f3fc00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.991+0000 7f80abfff640 1 -- 172.21.15.40:0/2741572192 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00ce70 con 0x7f80b415b670 2023-10-18T08:54:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.991+0000 7f80abfff640 1 -- 172.21.15.40:0/2741572192 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f80ac014440 con 0x7f80b415b670 2023-10-18T08:54:56.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.991+0000 7f80abfff640 1 -- 172.21.15.40:0/2741572192 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac014890 con 0x7f80b415b670 2023-10-18T08:54:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.991+0000 7f3faf7fe640 1 -- 172.21.15.40:0/434856922 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc003d430 con 0x7f3fc806b630 2023-10-18T08:54:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.991+0000 7f3faf7fe640 1 -- 172.21.15.40:0/434856922 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3fc003d680 con 0x7f3fc806b630 2023-10-18T08:54:56.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.991+0000 7f3faf7fe640 1 -- 172.21.15.40:0/434856922 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc003da90 con 0x7f3fc806b630 2023-10-18T08:54:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.991+0000 7f80b909b640 1 -- 172.21.15.40:0/2741572192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b40c74c0 con 0x7f80b415b670 2023-10-18T08:54:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.991+0000 7f80b909b640 1 -- 172.21.15.40:0/2741572192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b40c7a80 con 0x7f80b415b670 2023-10-18T08:54:56.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.992+0000 7f80abfff640 1 -- 172.21.15.40:0/2741572192 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f80ac014d10 con 0x7f80b415b670 2023-10-18T08:54:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.993+0000 7f80abfff640 1 -- 172.21.15.40:0/2741572192 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f80ac01d6f0 con 0x7f80b415b670 2023-10-18T08:54:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.992+0000 7f3fcd56e640 1 -- 172.21.15.40:0/434856922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc80708e0 con 0x7f3fc806b630 2023-10-18T08:54:56.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.992+0000 7f3fcd56e640 1 -- 172.21.15.40:0/434856922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc8149fa0 con 0x7f3fc806b630 2023-10-18T08:54:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.995+0000 7f3faf7fe640 1 -- 172.21.15.40:0/434856922 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3fc0046450 con 0x7f3fc806b630 2023-10-18T08:54:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.995+0000 7f3faf7fe640 1 -- 172.21.15.40:0/434856922 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3fc0034430 con 0x7f3fc806b630 2023-10-18T08:54:57.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.995+0000 7f3fcd56e640 1 -- 172.21.15.40:0/434856922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc8070e00 con 0x7f3fc806b630 2023-10-18T08:54:57.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:56.999+0000 7f3faf7fe640 1 -- 172.21.15.40:0/434856922 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f3fc8070e00 con 0x7f3fc806b630 2023-10-18T08:54:57.009 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi040') 2023-10-18T08:54:57.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.008+0000 7f80b909b640 1 -- 172.21.15.40:0/2741572192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8098005250 con 0x7f80b415b670 2023-10-18T08:54:57.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.018+0000 7f80abfff640 1 -- 172.21.15.40:0/2741572192 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f80ac023060 con 0x7f80b415b670 2023-10-18T08:54:57.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.021+0000 7f4eff7b7640 1 -- 172.21.15.40:0/2340016990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) v1 -- 0x7f4ef815e720 con 0x7f4ef80d1360 2023-10-18T08:54:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.032+0000 7f4eedffb640 1 -- 172.21.15.40:0/2340016990 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v12)=0 v12) v1 ==== 116+0+0 (secure 0 0 0) 0x7f4ef0039070 con 0x7f4ef80d1360 2023-10-18T08:54:57.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.034+0000 7f4eff7b7640 1 -- 172.21.15.40:0/2340016990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ef80d1360 msgr2=0x7f4ef80caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.034+0000 7f4eff7b7640 1 --2- 172.21.15.40:0/2340016990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ef80d1360 0x7f4ef80caed0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0033000 tx=0x7f4ef0035db0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.034+0000 7f4eff7b7640 1 -- 172.21.15.40:0/2340016990 shutdown_connections 2023-10-18T08:54:57.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.034+0000 7f4eff7b7640 1 --2- 172.21.15.40:0/2340016990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ef80d1360 0x7f4ef80caed0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.034+0000 7f4eff7b7640 1 --2- 172.21.15.40:0/2340016990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ef80cfc10 0x7f4ef80ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.034+0000 7f4eff7b7640 1 --2- 172.21.15.40:0/2340016990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ef80cbfc0 0x7f4ef8156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.034+0000 7f4eff7b7640 1 -- 172.21.15.40:0/2340016990 >> 172.21.15.40:0/2340016990 conn(0x7f4ef808ae20 msgr2=0x7f4ef806b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:57.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.034+0000 7f4eff7b7640 1 -- 172.21.15.40:0/2340016990 shutdown_connections 2023-10-18T08:54:57.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.034+0000 7f4eff7b7640 1 -- 172.21.15.40:0/2340016990 wait complete. 2023-10-18T08:54:57.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.036+0000 7f2997fff640 1 -- 172.21.15.40:0/1988801713 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) v1 -- 0x7f29ac066fb0 con 0x7f29ac070160 2023-10-18T08:54:57.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.053+0000 7f29a27fc640 1 -- 172.21.15.40:0/1988801713 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v13)=0 v13) v1 ==== 118+0+0 (secure 0 0 0) 0x7f29a80047a0 con 0x7f29ac070160 2023-10-18T08:54:57.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.054+0000 7f2997fff640 1 -- 172.21.15.40:0/1988801713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29ac070160 msgr2=0x7f29ac1f0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.054+0000 7f2997fff640 1 --2- 172.21.15.40:0/1988801713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29ac070160 0x7f29ac1f0140 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f29a800b940 tx=0x7f29a800be00 comp rx=0 tx=0).stop 2023-10-18T08:54:57.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.054+0000 7f2997fff640 1 -- 172.21.15.40:0/1988801713 shutdown_connections 2023-10-18T08:54:57.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.054+0000 7f2997fff640 1 --2- 172.21.15.40:0/1988801713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29ac0b0070 0x7f29ac161cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.054+0000 7f2997fff640 1 --2- 172.21.15.40:0/1988801713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29ac071050 0x7f29ac15f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.054+0000 7f2997fff640 1 --2- 172.21.15.40:0/1988801713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29ac070160 0x7f29ac1f0140 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.054+0000 7f2997fff640 1 -- 172.21.15.40:0/1988801713 >> 172.21.15.40:0/1988801713 conn(0x7f29ac0ccf80 msgr2=0x7f29ac0c1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:57.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.051+0000 7f3d6d0fc640 1 -- 172.21.15.40:0/2361812275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d68071610 msgr2=0x7f3d680cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.051+0000 7f3d6d0fc640 1 --2- 172.21.15.40:0/2361812275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d68071610 0x7f3d680cadc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3d680cddf0 tx=0x7f3d5002bdb0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.055+0000 7f2997fff640 1 -- 172.21.15.40:0/1988801713 shutdown_connections 2023-10-18T08:54:57.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.057+0000 7fdef3fff640 1 -- 172.21.15.40:0/3467462250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) v1 -- 0x7fdef4066e00 con 0x7fdef416bc60 2023-10-18T08:54:57.062 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi040') 2023-10-18T08:54:57.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.060+0000 7f3d6d0fc640 1 -- 172.21.15.40:0/2361812275 shutdown_connections 2023-10-18T08:54:57.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.058+0000 7f2997fff640 1 -- 172.21.15.40:0/1988801713 wait complete. 2023-10-18T08:54:57.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.063+0000 7fdedb7fe640 1 -- 172.21.15.40:0/3467462250 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v14)=0 v14) v1 ==== 121+0+0 (secure 0 0 0) 0x7fdeec0058c0 con 0x7fdef416bc60 2023-10-18T08:54:57.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.065+0000 7fdef3fff640 1 -- 172.21.15.40:0/3467462250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdef416bc60 msgr2=0x7fdef40ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.065+0000 7f3d6d0fc640 1 --2- 172.21.15.40:0/2361812275 >> v2:172.21.15.144:3300/0 conn(0x7f3d680abba0 0x7f3d680ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.065+0000 7f3d6d0fc640 1 --2- 172.21.15.40:0/2361812275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d68071610 0x7f3d680cadc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.065+0000 7f3d6d0fc640 1 -- 172.21.15.40:0/2361812275 >> 172.21.15.40:0/2361812275 conn(0x7f3d68069b10 msgr2=0x7f3d68069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:57.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.066+0000 7f3d6d0fc640 1 -- 172.21.15.40:0/2361812275 shutdown_connections 2023-10-18T08:54:57.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.066+0000 7f3d6d0fc640 1 -- 172.21.15.40:0/2361812275 wait complete. 2023-10-18T08:54:57.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.069+0000 7f3d6d0fc640 1 Processor -- start 2023-10-18T08:54:57.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.070+0000 7f3d6d0fc640 1 -- start start 2023-10-18T08:54:57.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.072+0000 7f3d6d0fc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d68071610 0x7f3d680a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:57.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.069+0000 7fdef3fff640 1 --2- 172.21.15.40:0/3467462250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdef416bc60 0x7fdef40ada30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00eca0 tx=0x7fdeec00b690 comp rx=0 tx=0).stop 2023-10-18T08:54:57.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.073+0000 7f3d67fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d68071610 0x7f3d680a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:57.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.073+0000 7f3d67fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d68071610 0x7f3d680a8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:59228/0 (socket says 172.21.15.40:59228) 2023-10-18T08:54:57.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.074+0000 7f3d6d0fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d680abba0 0x7f3d680ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:57.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.074+0000 7f3d5f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d680abba0 0x7f3d680ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:57.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.075+0000 7f3d5f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d680abba0 0x7f3d680ae040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43362/0 (socket says 172.21.15.40:43362) 2023-10-18T08:54:57.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.075+0000 7fdef3fff640 1 -- 172.21.15.40:0/3467462250 shutdown_connections 2023-10-18T08:54:57.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.075+0000 7fdef3fff640 1 --2- 172.21.15.40:0/3467462250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdef416bc60 0x7fdef40ada30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.075+0000 7fdef3fff640 1 --2- 172.21.15.40:0/3467462250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdef40aff70 0x7fdef40ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.075+0000 7fdef3fff640 1 --2- 172.21.15.40:0/3467462250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdef40aec00 0x7fdef415cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.075+0000 7f3d67fff640 1 -- 172.21.15.40:0/3086116344 learned_addr learned my addr 172.21.15.40:0/3086116344 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:57.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.075+0000 7fdef3fff640 1 -- 172.21.15.40:0/3467462250 >> 172.21.15.40:0/3467462250 conn(0x7fdef4069b10 msgr2=0x7fdef40882b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:57.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.075+0000 7f3d6d0fc640 1 --2- 172.21.15.40:0/3086116344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d680aec00 0x7f3d680ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:57.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.075+0000 7fdef3fff640 1 -- 172.21.15.40:0/3467462250 shutdown_connections 2023-10-18T08:54:57.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.075+0000 7fdef3fff640 1 -- 172.21.15.40:0/3467462250 wait complete. 2023-10-18T08:54:57.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.076+0000 7f3d6d0fc640 1 -- 172.21.15.40:0/3086116344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d680a9260 con 0x7f3d680abba0 2023-10-18T08:54:57.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.076+0000 7f3d6d0fc640 1 -- 172.21.15.40:0/3086116344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d680a7360 con 0x7f3d680aec00 2023-10-18T08:54:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.076+0000 7f3d6c8fb640 1 --2- 172.21.15.40:0/3086116344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d680aec00 0x7f3d680ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.076+0000 7f3d6d0fc640 1 -- 172.21.15.40:0/3086116344 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d680a7640 con 0x7f3d68071610 2023-10-18T08:54:57.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.078+0000 7f3d5f7fe640 1 -- 172.21.15.40:0/3086116344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d68071610 msgr2=0x7f3d680a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.078+0000 7f3d5f7fe640 1 --2- 172.21.15.40:0/3086116344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d68071610 0x7f3d680a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.078+0000 7f3d5f7fe640 1 -- 172.21.15.40:0/3086116344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d680aec00 msgr2=0x7f3d680ae580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.078+0000 7f3d5f7fe640 1 --2- 172.21.15.40:0/3086116344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d680aec00 0x7f3d680ae580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.078+0000 7f3d5f7fe640 1 -- 172.21.15.40:0/3086116344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d50009490 con 0x7f3d680abba0 2023-10-18T08:54:57.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.079+0000 7f3d6c8fb640 1 --2- 172.21.15.40:0/3086116344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d680aec00 0x7f3d680ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:54:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.080+0000 7f3d5f7fe640 1 --2- 172.21.15.40:0/3086116344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d680abba0 0x7f3d680ae040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3d50033000 tx=0x7f3d50003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.081+0000 7f3d657fa640 1 -- 172.21.15.40:0/3086116344 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d50005020 con 0x7f3d680abba0 2023-10-18T08:54:57.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.082+0000 7f3d657fa640 1 -- 172.21.15.40:0/3086116344 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3d50005270 con 0x7f3d680abba0 2023-10-18T08:54:57.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.082+0000 7f3d6d0fc640 1 -- 172.21.15.40:0/3086116344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d680a7920 con 0x7f3d680abba0 2023-10-18T08:54:57.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.084+0000 7f3d657fa640 1 -- 172.21.15.40:0/3086116344 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d50005680 con 0x7f3d680abba0 2023-10-18T08:54:57.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.084+0000 7f3d6d0fc640 1 -- 172.21.15.40:0/3086116344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d680a7ee0 con 0x7f3d680abba0 2023-10-18T08:54:57.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.086+0000 7f3d657fa640 1 -- 172.21.15.40:0/3086116344 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3d50004aa0 con 0x7f3d680abba0 2023-10-18T08:54:57.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.086+0000 7f3d657fa640 1 -- 172.21.15.40:0/3086116344 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3d50034a10 con 0x7f3d680abba0 2023-10-18T08:54:57.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.089+0000 7f3d5e7fc640 1 -- 172.21.15.40:0/3086116344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d680a80d0 con 0x7f3d680abba0 2023-10-18T08:54:57.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.104+0000 7f3d657fa640 1 -- 172.21.15.40:0/3086116344 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f3d50042070 con 0x7f3d680abba0 2023-10-18T08:54:57.108 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi040') 2023-10-18T08:54:57.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.113+0000 7fc85affd640 1 -- 172.21.15.40:0/3131145573 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) v1 -- 0x7fc874066e00 con 0x7fc8740b1010 2023-10-18T08:54:57.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.118+0000 7fcb903a9640 1 -- 172.21.15.40:0/4290143094 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) v1 -- 0x7fcb700050c0 con 0x7fcb880ac7e0 2023-10-18T08:54:57.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.127+0000 7fc8717fa640 1 -- 172.21.15.40:0/3131145573 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v15)=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc868003a40 con 0x7fc8740b1010 2023-10-18T08:54:57.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.129+0000 7fc85affd640 1 -- 172.21.15.40:0/3131145573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8740b1010 msgr2=0x7fc874137960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.129+0000 7fc85affd640 1 --2- 172.21.15.40:0/3131145573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8740b1010 0x7fc874137960 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc86800c490 tx=0x7fc86800c950 comp rx=0 tx=0).stop 2023-10-18T08:54:57.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.130+0000 7fc85affd640 1 -- 172.21.15.40:0/3131145573 shutdown_connections 2023-10-18T08:54:57.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.130+0000 7fc85affd640 1 --2- 172.21.15.40:0/3131145573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc874142ad0 0x7fc874137ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.130+0000 7fc85affd640 1 --2- 172.21.15.40:0/3131145573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8740b1010 0x7fc874137960 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.130+0000 7fc85affd640 1 --2- 172.21.15.40:0/3131145573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8740ac7e0 0x7fc87406ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.130+0000 7fc85affd640 1 -- 172.21.15.40:0/3131145573 >> 172.21.15.40:0/3131145573 conn(0x7fc874069b10 msgr2=0x7fc874070c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:57.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.130+0000 7fc85affd640 1 -- 172.21.15.40:0/3131145573 shutdown_connections 2023-10-18T08:54:57.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.130+0000 7fc85affd640 1 -- 172.21.15.40:0/3131145573 wait complete. 2023-10-18T08:54:57.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.140+0000 7fcb7effd640 1 -- 172.21.15.40:0/4290143094 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v16)=0 v16) v1 ==== 115+0+0 (secure 0 0 0) 0x7fcb840103e0 con 0x7fcb880ac7e0 2023-10-18T08:54:57.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.140+0000 7fbe69f47640 1 -- 172.21.15.40:0/1367543567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) v1 -- 0x7fbe64066e00 con 0x7fbe640c9620 2023-10-18T08:54:57.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.143+0000 7fcb903a9640 1 -- 172.21.15.40:0/4290143094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb880ac7e0 msgr2=0x7fcb881547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.143+0000 7fcb903a9640 1 --2- 172.21.15.40:0/4290143094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb880ac7e0 0x7fcb881547f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400b450 tx=0x7fcb8400b910 comp rx=0 tx=0).stop 2023-10-18T08:54:57.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.143+0000 7fcb903a9640 1 -- 172.21.15.40:0/4290143094 shutdown_connections 2023-10-18T08:54:57.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.143+0000 7fcb903a9640 1 --2- 172.21.15.40:0/4290143094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb880ccb30 0x7fcb88155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.143+0000 7fcb903a9640 1 --2- 172.21.15.40:0/4290143094 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb880cc750 0x7fcb88159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.143+0000 7fcb903a9640 1 --2- 172.21.15.40:0/4290143094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb880ac7e0 0x7fcb881547f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.143+0000 7fcb903a9640 1 -- 172.21.15.40:0/4290143094 >> 172.21.15.40:0/4290143094 conn(0x7fcb880aed70 msgr2=0x7fcb880b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:57.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.144+0000 7fcb903a9640 1 -- 172.21.15.40:0/4290143094 shutdown_connections 2023-10-18T08:54:57.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.144+0000 7fcb903a9640 1 -- 172.21.15.40:0/4290143094 wait complete. 2023-10-18T08:54:57.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.152+0000 7fbe617fa640 1 -- 172.21.15.40:0/1367543567 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v17)=0 v17) v1 ==== 119+0+0 (secure 0 0 0) 0x7fbe54054020 con 0x7fbe640c9620 2023-10-18T08:54:57.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.156+0000 7f3fcd56e640 1 -- 172.21.15.40:0/434856922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f3fc8070740 con 0x7f3fc806b630 2023-10-18T08:54:57.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.156+0000 7fbe69f47640 1 -- 172.21.15.40:0/1367543567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe640c9620 msgr2=0x7fbe64154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.156+0000 7fbe69f47640 1 --2- 172.21.15.40:0/1367543567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe640c9620 0x7fbe64154500 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009000 tx=0x7fbe540042f0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.157+0000 7fbe69f47640 1 -- 172.21.15.40:0/1367543567 shutdown_connections 2023-10-18T08:54:57.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.157+0000 7fbe69f47640 1 --2- 172.21.15.40:0/1367543567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe6415af70 0x7fbe641f9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.157+0000 7fbe69f47640 1 --2- 172.21.15.40:0/1367543567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe640c9620 0x7fbe64154500 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.157+0000 7fbe69f47640 1 --2- 172.21.15.40:0/1367543567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe64070b60 0x7fbe64153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.157+0000 7fbe69f47640 1 -- 172.21.15.40:0/1367543567 >> 172.21.15.40:0/1367543567 conn(0x7fbe6406ca10 msgr2=0x7fbe6415b480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:57.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.158+0000 7fbe69f47640 1 -- 172.21.15.40:0/1367543567 shutdown_connections 2023-10-18T08:54:57.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.158+0000 7fbe69f47640 1 -- 172.21.15.40:0/1367543567 wait complete. 2023-10-18T08:54:57.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.161+0000 7f3faf7fe640 1 -- 172.21.15.40:0/434856922 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v18)=0 v18) v1 ==== 121+0+0 (secure 0 0 0) 0x7f3fc8070740 con 0x7f3fc806b630 2023-10-18T08:54:57.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.163+0000 7f3fcd56e640 1 -- 172.21.15.40:0/434856922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fc806b630 msgr2=0x7f3fc815e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.163+0000 7f3fcd56e640 1 --2- 172.21.15.40:0/434856922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fc806b630 0x7f3fc815e870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009000 tx=0x7f3fc00045e0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.164+0000 7f3fcd56e640 1 -- 172.21.15.40:0/434856922 shutdown_connections 2023-10-18T08:54:57.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.164+0000 7f3fcd56e640 1 --2- 172.21.15.40:0/434856922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc806bfd0 0x7f3fc814cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.164+0000 7f3fcd56e640 1 --2- 172.21.15.40:0/434856922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fc806ba10 0x7f3fc806f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.164+0000 7f3fcd56e640 1 --2- 172.21.15.40:0/434856922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fc806b630 0x7f3fc815e870 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.164+0000 7f3fcd56e640 1 -- 172.21.15.40:0/434856922 >> 172.21.15.40:0/434856922 conn(0x7f3fc80ccf80 msgr2=0x7f3fc80cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:57.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.164+0000 7f3fcd56e640 1 -- 172.21.15.40:0/434856922 shutdown_connections 2023-10-18T08:54:57.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.164+0000 7f3fcd56e640 1 -- 172.21.15.40:0/434856922 wait complete. 2023-10-18T08:54:57.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.165+0000 7f80b909b640 1 -- 172.21.15.40:0/2741572192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) v1 -- 0x7f80980050c0 con 0x7f80b415b670 2023-10-18T08:54:57.169 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi040') 2023-10-18T08:54:57.170 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi040') 2023-10-18T08:54:57.170 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi040') 2023-10-18T08:54:57.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.174+0000 7f80abfff640 1 -- 172.21.15.40:0/2741572192 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v19)=0 v19) v1 ==== 121+0+0 (secure 0 0 0) 0x7f80ac019070 con 0x7f80b415b670 2023-10-18T08:54:57.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.176+0000 7f80b909b640 1 -- 172.21.15.40:0/2741572192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b415b670 msgr2=0x7f80b4201c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.176+0000 7f80b909b640 1 --2- 172.21.15.40:0/2741572192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b415b670 0x7f80b4201c40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00bdf0 tx=0x7f80ac00bee0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.176+0000 7f80b909b640 1 -- 172.21.15.40:0/2741572192 shutdown_connections 2023-10-18T08:54:57.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.176+0000 7f80b909b640 1 --2- 172.21.15.40:0/2741572192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80b415b670 0x7f80b4201c40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.176+0000 7f80b909b640 1 --2- 172.21.15.40:0/2741572192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80b40d1a70 0x7f80b41fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.176+0000 7f80b909b640 1 --2- 172.21.15.40:0/2741572192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80b40cea60 0x7f80b40c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.176+0000 7f80b909b640 1 -- 172.21.15.40:0/2741572192 >> 172.21.15.40:0/2741572192 conn(0x7f80b408ae20 msgr2=0x7f80b40b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:57.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.176+0000 7f80b909b640 1 -- 172.21.15.40:0/2741572192 shutdown_connections 2023-10-18T08:54:57.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.176+0000 7f80b909b640 1 -- 172.21.15.40:0/2741572192 wait complete. 2023-10-18T08:54:57.190 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi040') 2023-10-18T08:54:57.191 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi040') 2023-10-18T08:54:57.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.189+0000 7f857ffff640 1 -- 172.21.15.40:0/486104265 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f85780050c0 con 0x7f859c0b1740 2023-10-18T08:54:57.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.198+0000 7f85927fc640 1 -- 172.21.15.40:0/486104265 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v20)=0 v20) v1 ==== 120+0+0 (secure 0 0 0) 0x7f85940159d0 con 0x7f859c0b1740 2023-10-18T08:54:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.199+0000 7f857ffff640 1 -- 172.21.15.40:0/486104265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f859c0b1740 msgr2=0x7f859c1f4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.199+0000 7f857ffff640 1 --2- 172.21.15.40:0/486104265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f859c0b1740 0x7f859c1f4020 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f859400bcc0 tx=0x7f8594008600 comp rx=0 tx=0).stop 2023-10-18T08:54:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.199+0000 7f857ffff640 1 -- 172.21.15.40:0/486104265 shutdown_connections 2023-10-18T08:54:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.199+0000 7f857ffff640 1 --2- 172.21.15.40:0/486104265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f859c0b1740 0x7f859c1f4020 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.199+0000 7f857ffff640 1 --2- 172.21.15.40:0/486104265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859c0aec00 0x7f859c0c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.199+0000 7f857ffff640 1 --2- 172.21.15.40:0/486104265 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f859c06d390 0x7f859c145460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.199+0000 7f857ffff640 1 -- 172.21.15.40:0/486104265 >> 172.21.15.40:0/486104265 conn(0x7f859c069b10 msgr2=0x7f859c158bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:57.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.199+0000 7f857ffff640 1 -- 172.21.15.40:0/486104265 shutdown_connections 2023-10-18T08:54:57.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.199+0000 7f857ffff640 1 -- 172.21.15.40:0/486104265 wait complete. 2023-10-18T08:54:57.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.210+0000 7fe0f7ae5640 1 -- 172.21.15.40:0/2189688512 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7fe0e80dbc20 con 0x7fe0dc0047c0 2023-10-18T08:54:57.216 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi040') 2023-10-18T08:54:57.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.214+0000 7fe0e67fc640 1 -- 172.21.15.40:0/2189688512 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v21)=0 v21) v1 ==== 120+0+0 (secure 0 0 0) 0x7fe0ec00acb0 con 0x7fe0dc0047c0 2023-10-18T08:54:57.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.215+0000 7fe0f7ae5640 1 -- 172.21.15.40:0/2189688512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc0047c0 msgr2=0x7fe0e804ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.215+0000 7fe0f7ae5640 1 --2- 172.21.15.40:0/2189688512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc0047c0 0x7fe0e804ebd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec000f60 tx=0x7fe0ec00ce10 comp rx=0 tx=0).stop 2023-10-18T08:54:57.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.216+0000 7fe0f7ae5640 1 -- 172.21.15.40:0/2189688512 shutdown_connections 2023-10-18T08:54:57.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.216+0000 7fe0f7ae5640 1 --2- 172.21.15.40:0/2189688512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0e8039d40 0x7fe0e803a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.216+0000 7fe0f7ae5640 1 --2- 172.21.15.40:0/2189688512 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0e80d8f30 0x7fe0e80e37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.216+0000 7fe0f7ae5640 1 --2- 172.21.15.40:0/2189688512 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0dc0047c0 0x7fe0e804ebd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.216+0000 7fe0f7ae5640 1 -- 172.21.15.40:0/2189688512 >> 172.21.15.40:0/2189688512 conn(0x7fe0e803aa00 msgr2=0x7fe0e804f260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:57.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.216+0000 7fe0f7ae5640 1 -- 172.21.15.40:0/2189688512 shutdown_connections 2023-10-18T08:54:57.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.216+0000 7fe0f7ae5640 1 -- 172.21.15.40:0/2189688512 wait complete. 2023-10-18T08:54:57.239 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi040') 2023-10-18T08:54:57.240 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi040') 2023-10-18T08:54:57.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.244+0000 7f3d5e7fc640 1 -- 172.21.15.40:0/3086116344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f3d680a7780 con 0x7f3d680abba0 2023-10-18T08:54:57.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.247+0000 7f3d657fa640 1 -- 172.21.15.40:0/3086116344 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v22)=0 v22) v1 ==== 121+0+0 (secure 0 0 0) 0x7f3d680a7780 con 0x7f3d680abba0 2023-10-18T08:54:57.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.248+0000 7f3d5e7fc640 1 -- 172.21.15.40:0/3086116344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d680abba0 msgr2=0x7f3d680ae040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.248+0000 7f3d5e7fc640 1 --2- 172.21.15.40:0/3086116344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d680abba0 0x7f3d680ae040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3d50033000 tx=0x7f3d50003b90 comp rx=0 tx=0).stop 2023-10-18T08:54:57.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.248+0000 7f3d5e7fc640 1 -- 172.21.15.40:0/3086116344 shutdown_connections 2023-10-18T08:54:57.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.248+0000 7f3d5e7fc640 1 --2- 172.21.15.40:0/3086116344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d680aec00 0x7f3d680ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.248+0000 7f3d5e7fc640 1 --2- 172.21.15.40:0/3086116344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d680abba0 0x7f3d680ae040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.248+0000 7f3d5e7fc640 1 --2- 172.21.15.40:0/3086116344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d68071610 0x7f3d680a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.249+0000 7f3d5e7fc640 1 -- 172.21.15.40:0/3086116344 >> 172.21.15.40:0/3086116344 conn(0x7f3d68069b10 msgr2=0x7f3d680ad5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.249+0000 7f3d5e7fc640 1 -- 172.21.15.40:0/3086116344 shutdown_connections 2023-10-18T08:54:57.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.249+0000 7f3d5e7fc640 1 -- 172.21.15.40:0/3086116344 wait complete. 2023-10-18T08:54:57.263 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:57.261+0000 7fa50ce11040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T08:54:57.269 INFO:tasks.ceph:Setting crush tunables to default 2023-10-18T08:54:57.270 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd crush tunables default 2023-10-18T08:54:57.302 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:57.299+0000 7f700aa30040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T08:54:57.326 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:57.324+0000 7fa50ce11040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T08:54:57.364 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:57.361+0000 7f700aa30040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T08:54:57.415 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:57.411+0000 7f700aa30040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T08:54:57.477 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:57.474+0000 7f700aa30040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T08:54:57.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.500+0000 7f64c2543640 1 -- 172.21.15.40:0/825666009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64bc15b740 msgr2=0x7f64bc0cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.500+0000 7f64c2543640 1 --2- 172.21.15.40:0/825666009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64bc15b740 0x7f64bc0cf850 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009830 tx=0x7f64b802bd60 comp rx=0 tx=0).stop 2023-10-18T08:54:57.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.500+0000 7f64c2543640 1 -- 172.21.15.40:0/825666009 shutdown_connections 2023-10-18T08:54:57.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.500+0000 7f64c2543640 1 --2- 172.21.15.40:0/825666009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64bc15b740 0x7f64bc0cf850 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.500+0000 7f64c2543640 1 -- 172.21.15.40:0/825666009 >> 172.21.15.40:0/825666009 conn(0x7f64bc09a560 msgr2=0x7f64bc0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:57.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.500+0000 7f64c2543640 1 -- 172.21.15.40:0/825666009 shutdown_connections 2023-10-18T08:54:57.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.500+0000 7f64c2543640 1 -- 172.21.15.40:0/825666009 wait complete. 2023-10-18T08:54:57.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.502+0000 7f64c2543640 1 Processor -- start 2023-10-18T08:54:57.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.502+0000 7f64c2543640 1 -- start start 2023-10-18T08:54:57.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c2543640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64bc0af9e0 0x7f64bc1f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:57.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c2543640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64bc0cc860 0x7f64bc15a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:57.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c2543640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64bc15b740 0x7f64bc0c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:57.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c2543640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc1f1b90 con 0x7f64bc0cc860 2023-10-18T08:54:57.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c2543640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc15adc0 con 0x7f64bc0af9e0 2023-10-18T08:54:57.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c2543640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc0c5c30 con 0x7f64bc15b740 2023-10-18T08:54:57.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c0d40640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64bc0cc860 0x7f64bc15a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:57.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c0d40640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64bc0cc860 0x7f64bc15a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43386/0 (socket says 172.21.15.40:43386) 2023-10-18T08:54:57.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c0d40640 1 -- 172.21.15.40:0/2096427361 learned_addr learned my addr 172.21.15.40:0/2096427361 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:54:57.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c1541640 1 --2- 172.21.15.40:0/2096427361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64bc0af9e0 0x7f64bc1f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:57.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c1d42640 1 --2- 172.21.15.40:0/2096427361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64bc15b740 0x7f64bc0c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:57.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c0d40640 1 -- 172.21.15.40:0/2096427361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64bc15b740 msgr2=0x7f64bc0c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c0d40640 1 --2- 172.21.15.40:0/2096427361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64bc15b740 0x7f64bc0c5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c0d40640 1 -- 172.21.15.40:0/2096427361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64bc0af9e0 msgr2=0x7f64bc1f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c0d40640 1 --2- 172.21.15.40:0/2096427361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64bc0af9e0 0x7f64bc1f1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c0d40640 1 -- 172.21.15.40:0/2096427361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b8009490 con 0x7f64bc0cc860 2023-10-18T08:54:57.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.503+0000 7f64c1541640 1 --2- 172.21.15.40:0/2096427361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64bc0af9e0 0x7f64bc1f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:54:57.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.504+0000 7f64c0d40640 1 --2- 172.21.15.40:0/2096427361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64bc0cc860 0x7f64bc15a880 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00c430 tx=0x7f64ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:57.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.504+0000 7f64b1ffb640 1 -- 172.21.15.40:0/2096427361 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac011790 con 0x7f64bc0cc860 2023-10-18T08:54:57.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.504+0000 7f64b1ffb640 1 -- 172.21.15.40:0/2096427361 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f64ac0119e0 con 0x7f64bc0cc860 2023-10-18T08:54:57.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.504+0000 7f64b1ffb640 1 -- 172.21.15.40:0/2096427361 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac011e30 con 0x7f64bc0cc860 2023-10-18T08:54:57.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.504+0000 7f64c2543640 1 -- 172.21.15.40:0/2096427361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64bc0c94e0 con 0x7f64bc0cc860 2023-10-18T08:54:57.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.504+0000 7f64c2543640 1 -- 172.21.15.40:0/2096427361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64bc0c9730 con 0x7f64bc0cc860 2023-10-18T08:54:57.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.504+0000 7f64b1ffb640 1 -- 172.21.15.40:0/2096427361 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f64ac01c760 con 0x7f64bc0cc860 2023-10-18T08:54:57.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.505+0000 7f64b1ffb640 1 -- 172.21.15.40:0/2096427361 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f64ac0129a0 con 0x7f64bc0cc860 2023-10-18T08:54:57.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.505+0000 7f64c2543640 1 -- 172.21.15.40:0/2096427361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64bc066cf0 con 0x7f64bc0cc860 2023-10-18T08:54:57.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.508+0000 7f64b1ffb640 1 -- 172.21.15.40:0/2096427361 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f64ac012420 con 0x7f64bc0cc860 2023-10-18T08:54:57.527 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:57.524+0000 7f700aa30040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T08:54:57.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.576+0000 7f64c2543640 1 -- 172.21.15.40:0/2096427361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f64bc0cfb10 con 0x7f64bc0cc860 2023-10-18T08:54:57.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.579+0000 7f64b1ffb640 1 -- 172.21.15.40:0/2096427361 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f64ac01cbf0 con 0x7f64bc0cc860 2023-10-18T08:54:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2023-10-18T08:54:57.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.581+0000 7f64c2543640 1 -- 172.21.15.40:0/2096427361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64bc0cc860 msgr2=0x7f64bc15a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:57.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.581+0000 7f64c2543640 1 --2- 172.21.15.40:0/2096427361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64bc0cc860 0x7f64bc15a880 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00c430 tx=0x7f64ac00c8f0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.581+0000 7f64c2543640 1 -- 172.21.15.40:0/2096427361 shutdown_connections 2023-10-18T08:54:57.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.581+0000 7f64c2543640 1 --2- 172.21.15.40:0/2096427361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64bc15b740 0x7f64bc0c5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.581+0000 7f64c2543640 1 --2- 172.21.15.40:0/2096427361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64bc0cc860 0x7f64bc15a880 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.581+0000 7f64c2543640 1 --2- 172.21.15.40:0/2096427361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64bc0af9e0 0x7f64bc1f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:57.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.581+0000 7f64c2543640 1 -- 172.21.15.40:0/2096427361 >> 172.21.15.40:0/2096427361 conn(0x7f64bc09a560 msgr2=0x7f64bc15c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:57.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.581+0000 7f64c2543640 1 -- 172.21.15.40:0/2096427361 shutdown_connections 2023-10-18T08:54:57.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:54:57.581+0000 7f64c2543640 1 -- 172.21.15.40:0/2096427361 wait complete. 2023-10-18T08:54:57.600 INFO:tasks.ceph:check_enable_crimson: False 2023-10-18T08:54:57.600 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-18T08:54:57.601 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:57.601 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-18T08:54:57.641 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:57.638+0000 7f700aa30040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T08:54:57.720 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:57.716+0000 7f700aa30040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T08:54:57.734 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:57.735 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-18T08:54:57.773 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:57.770+0000 7f700aa30040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T08:54:57.828 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:57.828 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-18T08:54:57.832 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:57.829+0000 7f700aa30040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T08:54:57.887 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:57.884+0000 7f700aa30040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T08:54:57.929 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:54:57.930 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-18T08:54:57.993 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:57.991+0000 7fa50ce11040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T08:54:58.023 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:58.023 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-18T08:54:58.060 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:58.057+0000 7fa50ce11040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T08:54:58.113 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:58.111+0000 7fa50ce11040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T08:54:58.117 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:58.117 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-18T08:54:58.190 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:58.188+0000 7fa50ce11040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T08:54:58.199 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:58.200 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-18T08:54:58.243 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:58.241+0000 7fa50ce11040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T08:54:58.334 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:54:58.334 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-18T08:54:58.356 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:58.353+0000 7f700aa30040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T08:54:58.364 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:58.362+0000 7fa50ce11040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T08:54:58.428 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 0 2023-10-18T08:54:58.449 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:58.446+0000 7fa50ce11040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T08:54:58.505 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:58.503+0000 7fa50ce11040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T08:54:58.524 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:58.521+0000 7f700aa30040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T08:54:58.569 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:58.567+0000 7fa50ce11040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T08:54:58.627 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:58.625+0000 7fa50ce11040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T08:54:58.666 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:58.663+0000 7f700aa30040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T08:54:58.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.668+0000 7f0b85f0c640 1 -- 172.21.15.144:0/2066825708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b800b22c0 msgr2=0x7f0b800cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:58.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.668+0000 7f0b85f0c640 1 --2- 172.21.15.144:0/2066825708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b800b22c0 0x7f0b800cd100 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c005fd0 tx=0x7f0b6c02d160 comp rx=0 tx=0).stop 2023-10-18T08:54:58.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.668+0000 7f0b85f0c640 1 -- 172.21.15.144:0/2066825708 shutdown_connections 2023-10-18T08:54:58.671 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.668+0000 7f0b85f0c640 1 --2- 172.21.15.144:0/2066825708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b800b22c0 0x7f0b800cd100 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:58.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.668+0000 7f0b85f0c640 1 -- 172.21.15.144:0/2066825708 >> 172.21.15.144:0/2066825708 conn(0x7f0b800660c0 msgr2=0x7f0b80069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:58.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.668+0000 7f0b85f0c640 1 -- 172.21.15.144:0/2066825708 shutdown_connections 2023-10-18T08:54:58.672 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.668+0000 7f0b85f0c640 1 -- 172.21.15.144:0/2066825708 wait complete. 2023-10-18T08:54:58.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.669+0000 7f0b85f0c640 1 Processor -- start 2023-10-18T08:54:58.673 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.670+0000 7f0b85f0c640 1 -- start start 2023-10-18T08:54:58.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.670+0000 7f0b85f0c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b800aed80 0x7f0b80141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:58.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.670+0000 7f0b85f0c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b800b1ee0 0x7f0b801e5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:58.675 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.670+0000 7f0b85f0c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b800b22c0 0x7f0b801e5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:58.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.670+0000 7f0b85f0c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8013fbd0 con 0x7f0b800b22c0 2023-10-18T08:54:58.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.670+0000 7f0b85f0c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8013fe60 con 0x7f0b800b1ee0 2023-10-18T08:54:58.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.670+0000 7f0b85f0c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b80140420 con 0x7f0b800aed80 2023-10-18T08:54:58.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.671+0000 7f0b7ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b800b1ee0 0x7f0b801e5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:58.676 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.671+0000 7f0b7ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b800b1ee0 0x7f0b801e5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50650/0 (socket says 172.21.15.144:50650) 2023-10-18T08:54:58.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.671+0000 7f0b7ffff640 1 -- 172.21.15.144:0/3083449699 learned_addr learned my addr 172.21.15.144:0/3083449699 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-18T08:54:58.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.671+0000 7f0b84f0a640 1 --2- 172.21.15.144:0/3083449699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b800aed80 0x7f0b80141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:58.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.671+0000 7f0b8570b640 1 --2- 172.21.15.144:0/3083449699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b800b22c0 0x7f0b801e5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:58.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.671+0000 7f0b7ffff640 1 -- 172.21.15.144:0/3083449699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b800aed80 msgr2=0x7f0b80141ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:58.677 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.671+0000 7f0b7ffff640 1 --2- 172.21.15.144:0/3083449699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b800aed80 0x7f0b80141ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:58.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.671+0000 7f0b7ffff640 1 -- 172.21.15.144:0/3083449699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b800b22c0 msgr2=0x7f0b801e5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:58.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.671+0000 7f0b7ffff640 1 --2- 172.21.15.144:0/3083449699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b800b22c0 0x7f0b801e5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:58.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.671+0000 7f0b7ffff640 1 -- 172.21.15.144:0/3083449699 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c00a670 con 0x7f0b800b1ee0 2023-10-18T08:54:58.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.671+0000 7f0b7ffff640 1 --2- 172.21.15.144:0/3083449699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b800b1ee0 0x7f0b801e5530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c037040 tx=0x7f0b6c004820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:58.678 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.672+0000 7f0b7d7fa640 1 -- 172.21.15.144:0/3083449699 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c004e20 con 0x7f0b800b1ee0 2023-10-18T08:54:58.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.672+0000 7f0b7d7fa640 1 -- 172.21.15.144:0/3083449699 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0b6c0035f0 con 0x7f0b800b1ee0 2023-10-18T08:54:58.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.672+0000 7f0b7d7fa640 1 -- 172.21.15.144:0/3083449699 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c0039a0 con 0x7f0b800b1ee0 2023-10-18T08:54:58.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.672+0000 7f0b85f0c640 1 -- 172.21.15.144:0/3083449699 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b80142970 con 0x7f0b800b1ee0 2023-10-18T08:54:58.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.672+0000 7f0b85f0c640 1 -- 172.21.15.144:0/3083449699 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b801400b0 con 0x7f0b800b1ee0 2023-10-18T08:54:58.679 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.672+0000 7f0b7d7fa640 1 -- 172.21.15.144:0/3083449699 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0b6c003b00 con 0x7f0b800b1ee0 2023-10-18T08:54:58.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.672+0000 7f0b7d7fa640 1 -- 172.21.15.144:0/3083449699 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0b6c03e9d0 con 0x7f0b800b1ee0 2023-10-18T08:54:58.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.673+0000 7f0b85f0c640 1 -- 172.21.15.144:0/3083449699 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b80142e90 con 0x7f0b800b1ee0 2023-10-18T08:54:58.680 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.677+0000 7f0b7d7fa640 1 -- 172.21.15.144:0/3083449699 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f0b6c042020 con 0x7f0b800b1ee0 2023-10-18T08:54:58.724 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:58.721+0000 7f700aa30040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T08:54:58.760 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.757+0000 7f0b85f0c640 1 -- 172.21.15.144:0/3083449699 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0} v 0) v1 -- 0x7f0b800b1ae0 con 0x7f0b800b1ee0 2023-10-18T08:54:58.767 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.764+0000 7f0b7d7fa640 1 -- 172.21.15.144:0/3083449699 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0b6c03e440 con 0x7f0b800b1ee0 2023-10-18T08:54:58.767 INFO:teuthology.orchestra.run.smithi144.stdout:0 2023-10-18T08:54:58.768 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.765+0000 7f0b85f0c640 1 -- 172.21.15.144:0/3083449699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b800b1ee0 msgr2=0x7f0b801e5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:58.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.765+0000 7f0b85f0c640 1 --2- 172.21.15.144:0/3083449699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b800b1ee0 0x7f0b801e5530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c037040 tx=0x7f0b6c004820 comp rx=0 tx=0).stop 2023-10-18T08:54:58.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.765+0000 7f0b85f0c640 1 -- 172.21.15.144:0/3083449699 shutdown_connections 2023-10-18T08:54:58.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.765+0000 7f0b85f0c640 1 --2- 172.21.15.144:0/3083449699 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b800b22c0 0x7f0b801e5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:58.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.765+0000 7f0b85f0c640 1 --2- 172.21.15.144:0/3083449699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b800b1ee0 0x7f0b801e5530 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:58.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.765+0000 7f0b85f0c640 1 --2- 172.21.15.144:0/3083449699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b800aed80 0x7f0b80141ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:58.769 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.765+0000 7f0b85f0c640 1 -- 172.21.15.144:0/3083449699 >> 172.21.15.144:0/3083449699 conn(0x7f0b800660c0 msgr2=0x7f0b800b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:58.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.765+0000 7f0b85f0c640 1 -- 172.21.15.144:0/3083449699 shutdown_connections 2023-10-18T08:54:58.770 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.765+0000 7f0b85f0c640 1 -- 172.21.15.144:0/3083449699 wait complete. 2023-10-18T08:54:58.784 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 1 2023-10-18T08:54:58.840 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:58.837+0000 7f700aa30040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T08:54:58.957 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:58.954+0000 7f700aa30040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T08:54:58.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.971+0000 7fa42bfff640 1 -- 172.21.15.144:0/651242948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa42c0cb9d0 msgr2=0x7fa42c0c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:58.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.971+0000 7fa42bfff640 1 --2- 172.21.15.144:0/651242948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa42c0cb9d0 0x7fa42c0c9a00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa42c15cb90 tx=0x7fa42002bcf0 comp rx=0 tx=0).stop 2023-10-18T08:54:58.975 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.971+0000 7fa42bfff640 1 -- 172.21.15.144:0/651242948 shutdown_connections 2023-10-18T08:54:58.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.971+0000 7fa42bfff640 1 --2- 172.21.15.144:0/651242948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa42c0cb9d0 0x7fa42c0c9a00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:58.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.971+0000 7fa42bfff640 1 --2- 172.21.15.144:0/651242948 >> v2:172.21.15.40:3300/0 conn(0x7fa42c150270 0x7fa42c15a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:58.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.971+0000 7fa42bfff640 1 -- 172.21.15.144:0/651242948 >> 172.21.15.144:0/651242948 conn(0x7fa42c0660c0 msgr2=0x7fa42c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:58.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.971+0000 7fa42bfff640 1 -- 172.21.15.144:0/651242948 shutdown_connections 2023-10-18T08:54:58.976 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.971+0000 7fa42bfff640 1 -- 172.21.15.144:0/651242948 wait complete. 2023-10-18T08:54:58.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.973+0000 7fa42bfff640 1 Processor -- start 2023-10-18T08:54:58.978 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.973+0000 7fa42bfff640 1 -- start start 2023-10-18T08:54:58.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.974+0000 7fa42bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa42c0af700 0x7fa42c14cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:58.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.974+0000 7fa42bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa42c0cb9d0 0x7fa42c158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:58.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.974+0000 7fa42bfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa42c150270 0x7fa42c14ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:58.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.974+0000 7fa42bfff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c1585c0 con 0x7fa42c0af700 2023-10-18T08:54:58.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.974+0000 7fa42bfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c0a6470 con 0x7fa42c0cb9d0 2023-10-18T08:54:58.979 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.974+0000 7fa42bfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c0a6750 con 0x7fa42c150270 2023-10-18T08:54:58.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.974+0000 7fa42a7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa42c0cb9d0 0x7fa42c158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:58.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.974+0000 7fa42b7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa42c150270 0x7fa42c14ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:58.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.974+0000 7fa42a7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa42c0cb9d0 0x7fa42c158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58518/0 (socket says 172.21.15.144:58518) 2023-10-18T08:54:58.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.974+0000 7fa42a7fc640 1 -- 172.21.15.144:0/1611332948 learned_addr learned my addr 172.21.15.144:0/1611332948 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-18T08:54:58.980 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.975+0000 7fa42affd640 1 --2- 172.21.15.144:0/1611332948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa42c0af700 0x7fa42c14cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:58.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.975+0000 7fa42a7fc640 1 -- 172.21.15.144:0/1611332948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa42c150270 msgr2=0x7fa42c14ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:58.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.975+0000 7fa42a7fc640 1 --2- 172.21.15.144:0/1611332948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa42c150270 0x7fa42c14ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:58.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.975+0000 7fa42a7fc640 1 -- 172.21.15.144:0/1611332948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa42c0af700 msgr2=0x7fa42c14cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:58.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.975+0000 7fa42a7fc640 1 --2- 172.21.15.144:0/1611332948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa42c0af700 0x7fa42c14cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:58.981 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.975+0000 7fa42a7fc640 1 -- 172.21.15.144:0/1611332948 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa420009490 con 0x7fa42c0cb9d0 2023-10-18T08:54:58.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.975+0000 7fa42b7fe640 1 --2- 172.21.15.144:0/1611332948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa42c150270 0x7fa42c14ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T08:54:58.982 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.975+0000 7fa42a7fc640 1 --2- 172.21.15.144:0/1611332948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa42c0cb9d0 0x7fa42c158080 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa42002c300 tx=0x7fa4200038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:58.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.976+0000 7fa413fff640 1 -- 172.21.15.144:0/1611332948 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420003ea0 con 0x7fa42c0cb9d0 2023-10-18T08:54:58.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.976+0000 7fa413fff640 1 -- 172.21.15.144:0/1611332948 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa420034440 con 0x7fa42c0cb9d0 2023-10-18T08:54:58.983 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.976+0000 7fa413fff640 1 -- 172.21.15.144:0/1611332948 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420034810 con 0x7fa42c0cb9d0 2023-10-18T08:54:58.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.976+0000 7fa42bfff640 1 -- 172.21.15.144:0/1611332948 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42c0aeb60 con 0x7fa42c0cb9d0 2023-10-18T08:54:58.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.976+0000 7fa42bfff640 1 -- 172.21.15.144:0/1611332948 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42c0af120 con 0x7fa42c0cb9d0 2023-10-18T08:54:58.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.976+0000 7fa413fff640 1 -- 172.21.15.144:0/1611332948 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa420034c70 con 0x7fa42c0cb9d0 2023-10-18T08:54:58.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.976+0000 7fa413fff640 1 -- 172.21.15.144:0/1611332948 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fa42003e7b0 con 0x7fa42c0cb9d0 2023-10-18T08:54:58.984 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.977+0000 7fa42bfff640 1 -- 172.21.15.144:0/1611332948 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa42c06a6d0 con 0x7fa42c0cb9d0 2023-10-18T08:54:58.985 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:58.979+0000 7fa413fff640 1 -- 172.21.15.144:0/1611332948 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fa420066b40 con 0x7fa42c0cb9d0 2023-10-18T08:54:59.059 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.056+0000 7fa42bfff640 1 -- 172.21.15.144:0/1611332948 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1} v 0) v1 -- 0x7fa42c0b2c20 con 0x7fa42c0cb9d0 2023-10-18T08:54:59.070 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.067+0000 7fa413fff640 1 -- 172.21.15.144:0/1611332948 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa420055020 con 0x7fa42c0cb9d0 2023-10-18T08:54:59.071 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-10-18T08:54:59.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.068+0000 7fa42bfff640 1 -- 172.21.15.144:0/1611332948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa42c0cb9d0 msgr2=0x7fa42c158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:59.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.068+0000 7fa42bfff640 1 --2- 172.21.15.144:0/1611332948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa42c0cb9d0 0x7fa42c158080 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa42002c300 tx=0x7fa4200038a0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.069+0000 7fa42bfff640 1 -- 172.21.15.144:0/1611332948 shutdown_connections 2023-10-18T08:54:59.072 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.069+0000 7fa42bfff640 1 --2- 172.21.15.144:0/1611332948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa42c150270 0x7fa42c14ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.069+0000 7fa42bfff640 1 --2- 172.21.15.144:0/1611332948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa42c0cb9d0 0x7fa42c158080 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.069+0000 7fa42bfff640 1 --2- 172.21.15.144:0/1611332948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa42c0af700 0x7fa42c14cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.069+0000 7fa42bfff640 1 -- 172.21.15.144:0/1611332948 >> 172.21.15.144:0/1611332948 conn(0x7fa42c0660c0 msgr2=0x7fa42c072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:59.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.069+0000 7fa42bfff640 1 -- 172.21.15.144:0/1611332948 shutdown_connections 2023-10-18T08:54:59.073 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.069+0000 7fa42bfff640 1 -- 172.21.15.144:0/1611332948 wait complete. 2023-10-18T08:54:59.087 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 2 2023-10-18T08:54:59.108 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:59.106+0000 7fa50ce11040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T08:54:59.127 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:59.124+0000 7f700aa30040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T08:54:59.180 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:59.176+0000 7f700aa30040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T08:54:59.277 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:59.275+0000 7fa50ce11040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T08:54:59.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.281+0000 7f4ec5fc4640 1 -- 172.21.15.144:0/1192556134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ec015eea0 msgr2=0x7f4ec00b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:59.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.281+0000 7f4ec5fc4640 1 --2- 172.21.15.144:0/1192556134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ec015eea0 0x7f4ec00b0be0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0069c40 tx=0x7f4eac02be40 comp rx=0 tx=0).stop 2023-10-18T08:54:59.285 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.282+0000 7f4ec5fc4640 1 -- 172.21.15.144:0/1192556134 shutdown_connections 2023-10-18T08:54:59.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.282+0000 7f4ec5fc4640 1 --2- 172.21.15.144:0/1192556134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ec015eea0 0x7f4ec00b0be0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.282+0000 7f4ec5fc4640 1 --2- 172.21.15.144:0/1192556134 >> v2:172.21.15.40:3300/0 conn(0x7f4ec015eac0 0x7f4ec00cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.282+0000 7f4ec5fc4640 1 -- 172.21.15.144:0/1192556134 >> 172.21.15.144:0/1192556134 conn(0x7f4ec009a610 msgr2=0x7f4ec00d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:59.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.282+0000 7f4ec5fc4640 1 -- 172.21.15.144:0/1192556134 shutdown_connections 2023-10-18T08:54:59.286 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.282+0000 7f4ec5fc4640 1 -- 172.21.15.144:0/1192556134 wait complete. 2023-10-18T08:54:59.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.283+0000 7f4ec5fc4640 1 Processor -- start 2023-10-18T08:54:59.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.283+0000 7f4ec5fc4640 1 -- start start 2023-10-18T08:54:59.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.284+0000 7f4ec5fc4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ec00d19b0 0x7f4ec014d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:59.289 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.284+0000 7f4ec5fc4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ec015eac0 0x7f4ec00c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:59.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.284+0000 7f4ec5fc4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec015eea0 0x7f4ec00b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:59.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.284+0000 7f4ec5fc4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec014dcb0 con 0x7f4ec015eea0 2023-10-18T08:54:59.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.284+0000 7f4ec5fc4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec00c8bc0 con 0x7f4ec00d19b0 2023-10-18T08:54:59.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.284+0000 7f4ec5fc4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec00c8ea0 con 0x7f4ec015eac0 2023-10-18T08:54:59.290 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.284+0000 7f4ec4fc2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ec00d19b0 0x7f4ec014d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:59.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.285+0000 7f4ec4fc2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ec00d19b0 0x7f4ec014d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58540/0 (socket says 172.21.15.144:58540) 2023-10-18T08:54:59.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.285+0000 7f4ec4fc2640 1 -- 172.21.15.144:0/1092697269 learned_addr learned my addr 172.21.15.144:0/1092697269 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-18T08:54:59.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.285+0000 7f4ebffff640 1 --2- 172.21.15.144:0/1092697269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ec015eac0 0x7f4ec00c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:59.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.285+0000 7f4ec57c3640 1 --2- 172.21.15.144:0/1092697269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec015eea0 0x7f4ec00b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:59.291 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.285+0000 7f4ec4fc2640 1 -- 172.21.15.144:0/1092697269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ec015eac0 msgr2=0x7f4ec00c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:59.292 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.285+0000 7f4ec4fc2640 1 --2- 172.21.15.144:0/1092697269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ec015eac0 0x7f4ec00c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.292 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.285+0000 7f4ec4fc2640 1 -- 172.21.15.144:0/1092697269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec015eea0 msgr2=0x7f4ec00b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:59.292 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.285+0000 7f4ec4fc2640 1 --2- 172.21.15.144:0/1092697269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec015eea0 0x7f4ec00b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.292 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.285+0000 7f4ec4fc2640 1 -- 172.21.15.144:0/1092697269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac009560 con 0x7f4ec00d19b0 2023-10-18T08:54:59.292 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.285+0000 7f4ec4fc2640 1 --2- 172.21.15.144:0/1092697269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ec00d19b0 0x7f4ec014d770 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400d950 tx=0x7f4eb400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:59.293 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.285+0000 7f4ebd7fa640 1 -- 172.21.15.144:0/1092697269 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4009c30 con 0x7f4ec00d19b0 2023-10-18T08:54:59.293 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.286+0000 7f4ebd7fa640 1 -- 172.21.15.144:0/1092697269 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4eb4009e80 con 0x7f4ec00d19b0 2023-10-18T08:54:59.293 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.286+0000 7f4ebd7fa640 1 -- 172.21.15.144:0/1092697269 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb40106c0 con 0x7f4ec00d19b0 2023-10-18T08:54:59.293 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.286+0000 7f4ec5fc4640 1 -- 172.21.15.144:0/1092697269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec00c9180 con 0x7f4ec00d19b0 2023-10-18T08:54:59.293 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.286+0000 7f4ec5fc4640 1 -- 172.21.15.144:0/1092697269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec00c9740 con 0x7f4ec00d19b0 2023-10-18T08:54:59.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.286+0000 7f4ebd7fa640 1 -- 172.21.15.144:0/1092697269 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4eb4010b20 con 0x7f4ec00d19b0 2023-10-18T08:54:59.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.286+0000 7f4ebd7fa640 1 -- 172.21.15.144:0/1092697269 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f4eb4010dd0 con 0x7f4ec00d19b0 2023-10-18T08:54:59.294 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.287+0000 7f4ec5fc4640 1 -- 172.21.15.144:0/1092697269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec0071da0 con 0x7f4ec00d19b0 2023-10-18T08:54:59.295 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.289+0000 7f4ebd7fa640 1 -- 172.21.15.144:0/1092697269 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f4eb4020060 con 0x7f4ec00d19b0 2023-10-18T08:54:59.345 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:54:59.341+0000 7f700aa30040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T08:54:59.369 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.366+0000 7f4ec5fc4640 1 -- 172.21.15.144:0/1092697269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2} v 0) v1 -- 0x7f4ec00c98d0 con 0x7f4ec00d19b0 2023-10-18T08:54:59.378 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.374+0000 7f4ebd7fa640 1 -- 172.21.15.144:0/1092697269 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4eb4017070 con 0x7f4ec00d19b0 2023-10-18T08:54:59.378 INFO:teuthology.orchestra.run.smithi144.stdout:2 2023-10-18T08:54:59.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.376+0000 7f4ec5fc4640 1 -- 172.21.15.144:0/1092697269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ec00d19b0 msgr2=0x7f4ec014d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:59.379 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.376+0000 7f4ec5fc4640 1 --2- 172.21.15.144:0/1092697269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ec00d19b0 0x7f4ec014d770 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400d950 tx=0x7f4eb400de10 comp rx=0 tx=0).stop 2023-10-18T08:54:59.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.376+0000 7f4ec5fc4640 1 -- 172.21.15.144:0/1092697269 shutdown_connections 2023-10-18T08:54:59.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.376+0000 7f4ec5fc4640 1 --2- 172.21.15.144:0/1092697269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec015eea0 0x7f4ec00b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.376+0000 7f4ec5fc4640 1 --2- 172.21.15.144:0/1092697269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ec015eac0 0x7f4ec00c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.376+0000 7f4ec5fc4640 1 --2- 172.21.15.144:0/1092697269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ec00d19b0 0x7f4ec014d770 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.376+0000 7f4ec5fc4640 1 -- 172.21.15.144:0/1092697269 >> 172.21.15.144:0/1092697269 conn(0x7f4ec009a610 msgr2=0x7f4ec00d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:59.380 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.376+0000 7f4ec5fc4640 1 -- 172.21.15.144:0/1092697269 shutdown_connections 2023-10-18T08:54:59.381 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.376+0000 7f4ec5fc4640 1 -- 172.21.15.144:0/1092697269 wait complete. 2023-10-18T08:54:59.396 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 3 2023-10-18T08:54:59.410 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:59.407+0000 7fa50ce11040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T08:54:59.465 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:59.463+0000 7fa50ce11040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T08:54:59.580 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:59.578+0000 7fa50ce11040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T08:54:59.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.642+0000 7f02e35e2640 1 -- 172.21.15.144:0/3424447141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02dc0cea60 msgr2=0x7f02dc0d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:59.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.642+0000 7f02e35e2640 1 --2- 172.21.15.144:0/3424447141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02dc0cea60 0x7f02dc0d1360 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f02cc005fd0 tx=0x7f02cc02bd80 comp rx=0 tx=0).stop 2023-10-18T08:54:59.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.642+0000 7f02e35e2640 1 -- 172.21.15.144:0/3424447141 shutdown_connections 2023-10-18T08:54:59.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.642+0000 7f02e35e2640 1 --2- 172.21.15.144:0/3424447141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02dc0cea60 0x7f02dc0d1360 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.646 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.642+0000 7f02e35e2640 1 --2- 172.21.15.144:0/3424447141 >> v2:172.21.15.144:3300/0 conn(0x7f02dc15b670 0x7f02dc0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.642+0000 7f02e35e2640 1 -- 172.21.15.144:0/3424447141 >> 172.21.15.144:0/3424447141 conn(0x7f02dc08ae20 msgr2=0x7f02dc09a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:59.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.642+0000 7f02e35e2640 1 -- 172.21.15.144:0/3424447141 shutdown_connections 2023-10-18T08:54:59.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.642+0000 7f02e35e2640 1 -- 172.21.15.144:0/3424447141 wait complete. 2023-10-18T08:54:59.647 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.644+0000 7f02e35e2640 1 Processor -- start 2023-10-18T08:54:59.648 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.644+0000 7f02e35e2640 1 -- start start 2023-10-18T08:54:59.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.645+0000 7f02e35e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02dc0cea60 0x7f02dc15d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:59.649 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.645+0000 7f02e35e2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02dc0d18a0 0x7f02dc0b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:59.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.646+0000 7f02e35e2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02dc15b670 0x7f02dc0c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:59.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.646+0000 7f02e35e2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc06bb70 con 0x7f02dc0cea60 2023-10-18T08:54:59.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.646+0000 7f02e35e2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc0b6760 con 0x7f02dc15b670 2023-10-18T08:54:59.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.646+0000 7f02e35e2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc15da00 con 0x7f02dc0d18a0 2023-10-18T08:54:59.650 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.646+0000 7f02e0b7d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02dc0d18a0 0x7f02dc0b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:59.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.646+0000 7f02e1b7f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02dc15b670 0x7f02dc0c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:59.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.646+0000 7f02e1b7f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02dc15b670 0x7f02dc0c79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58554/0 (socket says 172.21.15.144:58554) 2023-10-18T08:54:59.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.646+0000 7f02e1b7f640 1 -- 172.21.15.144:0/4254348821 learned_addr learned my addr 172.21.15.144:0/4254348821 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-18T08:54:59.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.646+0000 7f02e0b7d640 1 --2- 172.21.15.144:0/4254348821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02dc0d18a0 0x7f02dc0b6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:50552/0 (socket says 172.21.15.144:50552) 2023-10-18T08:54:59.651 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.646+0000 7f02e137e640 1 --2- 172.21.15.144:0/4254348821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02dc0cea60 0x7f02dc15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:59.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.646+0000 7f02e0b7d640 1 -- 172.21.15.144:0/4254348821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02dc15b670 msgr2=0x7f02dc0c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:59.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.646+0000 7f02e0b7d640 1 --2- 172.21.15.144:0/4254348821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02dc15b670 0x7f02dc0c79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.646+0000 7f02e0b7d640 1 -- 172.21.15.144:0/4254348821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02dc0cea60 msgr2=0x7f02dc15d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:59.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.646+0000 7f02e0b7d640 1 --2- 172.21.15.144:0/4254348821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02dc0cea60 0x7f02dc15d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.652 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.646+0000 7f02e0b7d640 1 -- 172.21.15.144:0/4254348821 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d8009490 con 0x7f02dc0d18a0 2023-10-18T08:54:59.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.647+0000 7f02e0b7d640 1 --2- 172.21.15.144:0/4254348821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02dc0d18a0 0x7f02dc0b6220 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f02cc033000 tx=0x7f02cc02d960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:59.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.647+0000 7f02d1ffb640 1 -- 172.21.15.144:0/4254348821 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc02de20 con 0x7f02dc0d18a0 2023-10-18T08:54:59.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.647+0000 7f02d1ffb640 1 -- 172.21.15.144:0/4254348821 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f02cc0036e0 con 0x7f02dc0d18a0 2023-10-18T08:54:59.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.647+0000 7f02e35e2640 1 -- 172.21.15.144:0/4254348821 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02cc009560 con 0x7f02dc0d18a0 2023-10-18T08:54:59.654 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.647+0000 7f02d1ffb640 1 -- 172.21.15.144:0/4254348821 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc003a90 con 0x7f02dc0d18a0 2023-10-18T08:54:59.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.647+0000 7f02e35e2640 1 -- 172.21.15.144:0/4254348821 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02dc15dfc0 con 0x7f02dc0d18a0 2023-10-18T08:54:59.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.647+0000 7f02d1ffb640 1 -- 172.21.15.144:0/4254348821 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f02cc03e450 con 0x7f02dc0d18a0 2023-10-18T08:54:59.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.647+0000 7f02d1ffb640 1 -- 172.21.15.144:0/4254348821 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f02cc0348d0 con 0x7f02dc0d18a0 2023-10-18T08:54:59.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.648+0000 7f02e35e2640 1 -- 172.21.15.144:0/4254348821 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02dc066fd0 con 0x7f02dc0d18a0 2023-10-18T08:54:59.655 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.651+0000 7f02d1ffb640 1 -- 172.21.15.144:0/4254348821 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f02cc066b40 con 0x7f02dc0d18a0 2023-10-18T08:54:59.697 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:59.695+0000 7fa50ce11040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T08:54:59.731 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.728+0000 7f02e35e2640 1 -- 172.21.15.144:0/4254348821 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3} v 0) v1 -- 0x7f02dc0b00b0 con 0x7f02dc0d18a0 2023-10-18T08:54:59.739 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.736+0000 7f02d1ffb640 1 -- 172.21.15.144:0/4254348821 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f02cc055020 con 0x7f02dc0d18a0 2023-10-18T08:54:59.739 INFO:teuthology.orchestra.run.smithi144.stdout:3 2023-10-18T08:54:59.740 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.737+0000 7f02e35e2640 1 -- 172.21.15.144:0/4254348821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02dc0d18a0 msgr2=0x7f02dc0b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:59.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.737+0000 7f02e35e2640 1 --2- 172.21.15.144:0/4254348821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02dc0d18a0 0x7f02dc0b6220 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f02cc033000 tx=0x7f02cc02d960 comp rx=0 tx=0).stop 2023-10-18T08:54:59.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.737+0000 7f02e35e2640 1 -- 172.21.15.144:0/4254348821 shutdown_connections 2023-10-18T08:54:59.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.737+0000 7f02e35e2640 1 --2- 172.21.15.144:0/4254348821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02dc15b670 0x7f02dc0c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.737+0000 7f02e35e2640 1 --2- 172.21.15.144:0/4254348821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02dc0d18a0 0x7f02dc0b6220 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.741 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.737+0000 7f02e35e2640 1 --2- 172.21.15.144:0/4254348821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02dc0cea60 0x7f02dc15d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.737+0000 7f02e35e2640 1 -- 172.21.15.144:0/4254348821 >> 172.21.15.144:0/4254348821 conn(0x7f02dc08ae20 msgr2=0x7f02dc0ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:59.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.737+0000 7f02e35e2640 1 -- 172.21.15.144:0/4254348821 shutdown_connections 2023-10-18T08:54:59.742 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.737+0000 7f02e35e2640 1 -- 172.21.15.144:0/4254348821 wait complete. 2023-10-18T08:54:59.756 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 4 2023-10-18T08:54:59.866 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:59.864+0000 7fa50ce11040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T08:54:59.918 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:54:59.916+0000 7fa50ce11040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T08:54:59.956 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.953+0000 7ffbbafd7640 1 -- 172.21.15.144:0/194454311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffba0003970 msgr2=0x7ffba0003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:59.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.953+0000 7ffbbafd7640 1 --2- 172.21.15.144:0/194454311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffba0003970 0x7ffba0003d70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000c290 tx=0x7ffba00057f0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.953+0000 7ffbbafd7640 1 -- 172.21.15.144:0/194454311 shutdown_connections 2023-10-18T08:54:59.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.953+0000 7ffbbafd7640 1 --2- 172.21.15.144:0/194454311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffba0003970 0x7ffba0003d70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.957 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.953+0000 7ffbbafd7640 1 -- 172.21.15.144:0/194454311 >> 172.21.15.144:0/194454311 conn(0x7ffbb409a580 msgr2=0x7ffbb40d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:54:59.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.953+0000 7ffbbafd7640 1 -- 172.21.15.144:0/194454311 shutdown_connections 2023-10-18T08:54:59.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.953+0000 7ffbbafd7640 1 -- 172.21.15.144:0/194454311 wait complete. 2023-10-18T08:54:59.958 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.955+0000 7ffbbafd7640 1 Processor -- start 2023-10-18T08:54:59.959 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.955+0000 7ffbbafd7640 1 -- start start 2023-10-18T08:54:59.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.956+0000 7ffbbafd7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffba0003970 0x7ffbb415aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:59.960 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.956+0000 7ffbbafd7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb409b9b0 0x7ffbb409be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:59.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.956+0000 7ffbbafd7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbb406ae10 0x7ffbb406b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:54:59.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.956+0000 7ffbbafd7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb409ef30 con 0x7ffbb406ae10 2023-10-18T08:54:59.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.956+0000 7ffbbafd7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb406b7d0 con 0x7ffbb409b9b0 2023-10-18T08:54:59.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.956+0000 7ffbbafd7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb415b010 con 0x7ffba0003970 2023-10-18T08:54:59.961 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.956+0000 7ffbb9fd5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffba0003970 0x7ffbb415aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:59.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.956+0000 7ffbb97d4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb409b9b0 0x7ffbb409be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:59.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.956+0000 7ffbb9fd5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffba0003970 0x7ffbb415aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:50562/0 (socket says 172.21.15.144:50562) 2023-10-18T08:54:59.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.956+0000 7ffbb9fd5640 1 -- 172.21.15.144:0/3301415915 learned_addr learned my addr 172.21.15.144:0/3301415915 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-18T08:54:59.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.957+0000 7ffbba7d6640 1 --2- 172.21.15.144:0/3301415915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbb406ae10 0x7ffbb406b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:54:59.962 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.957+0000 7ffbb9fd5640 1 -- 172.21.15.144:0/3301415915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb409b9b0 msgr2=0x7ffbb409be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:59.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.957+0000 7ffbb9fd5640 1 --2- 172.21.15.144:0/3301415915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb409b9b0 0x7ffbb409be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.957+0000 7ffbb9fd5640 1 -- 172.21.15.144:0/3301415915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbb406ae10 msgr2=0x7ffbb406b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:54:59.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.957+0000 7ffbb9fd5640 1 --2- 172.21.15.144:0/3301415915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbb406ae10 0x7ffbb406b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:54:59.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.957+0000 7ffbb9fd5640 1 -- 172.21.15.144:0/3301415915 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb409e2b0 con 0x7ffba0003970 2023-10-18T08:54:59.963 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.957+0000 7ffbb9fd5640 1 --2- 172.21.15.144:0/3301415915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffba0003970 0x7ffbb415aad0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000c3b0 tx=0x7ffbb0001540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:54:59.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.959+0000 7ffbaa7fc640 1 -- 172.21.15.144:0/3301415915 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0006310 con 0x7ffba0003970 2023-10-18T08:54:59.964 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.959+0000 7ffbaa7fc640 1 -- 172.21.15.144:0/3301415915 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffbb0006470 con 0x7ffba0003970 2023-10-18T08:54:59.965 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.959+0000 7ffbaa7fc640 1 -- 172.21.15.144:0/3301415915 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0003850 con 0x7ffba0003970 2023-10-18T08:54:59.965 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.959+0000 7ffbbafd7640 1 -- 172.21.15.144:0/3301415915 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb409e590 con 0x7ffba0003970 2023-10-18T08:54:59.965 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.959+0000 7ffbbafd7640 1 -- 172.21.15.144:0/3301415915 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb409eb50 con 0x7ffba0003970 2023-10-18T08:54:59.965 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.960+0000 7ffbaa7fc640 1 -- 172.21.15.144:0/3301415915 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffbb0031030 con 0x7ffba0003970 2023-10-18T08:54:59.965 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.960+0000 7ffbaa7fc640 1 -- 172.21.15.144:0/3301415915 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7ffbb00039b0 con 0x7ffba0003970 2023-10-18T08:54:59.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.961+0000 7ffbbafd7640 1 -- 172.21.15.144:0/3301415915 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb4073570 con 0x7ffba0003970 2023-10-18T08:54:59.966 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:54:59.963+0000 7ffbaa7fc640 1 -- 172.21.15.144:0/3301415915 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7ffbb0043070 con 0x7ffba0003970 2023-10-18T08:55:00.044 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.040+0000 7ffbbafd7640 1 -- 172.21.15.144:0/3301415915 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4} v 0) v1 -- 0x7ffbb40af000 con 0x7ffba0003970 2023-10-18T08:55:00.051 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.048+0000 7ffbaa7fc640 1 -- 172.21.15.144:0/3301415915 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffbb0039040 con 0x7ffba0003970 2023-10-18T08:55:00.051 INFO:teuthology.orchestra.run.smithi144.stdout:4 2023-10-18T08:55:00.052 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.049+0000 7ffbbafd7640 1 -- 172.21.15.144:0/3301415915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffba0003970 msgr2=0x7ffbb415aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:00.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.049+0000 7ffbbafd7640 1 --2- 172.21.15.144:0/3301415915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffba0003970 0x7ffbb415aad0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000c3b0 tx=0x7ffbb0001540 comp rx=0 tx=0).stop 2023-10-18T08:55:00.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.049+0000 7ffbbafd7640 1 -- 172.21.15.144:0/3301415915 shutdown_connections 2023-10-18T08:55:00.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.049+0000 7ffbbafd7640 1 --2- 172.21.15.144:0/3301415915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbb406ae10 0x7ffbb406b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.053 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.049+0000 7ffbbafd7640 1 --2- 172.21.15.144:0/3301415915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb409b9b0 0x7ffbb409be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.049+0000 7ffbbafd7640 1 --2- 172.21.15.144:0/3301415915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffba0003970 0x7ffbb415aad0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.049+0000 7ffbbafd7640 1 -- 172.21.15.144:0/3301415915 >> 172.21.15.144:0/3301415915 conn(0x7ffbb409a580 msgr2=0x7ffbb406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:00.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.049+0000 7ffbbafd7640 1 -- 172.21.15.144:0/3301415915 shutdown_connections 2023-10-18T08:55:00.054 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.049+0000 7ffbbafd7640 1 -- 172.21.15.144:0/3301415915 wait complete. 2023-10-18T08:55:00.068 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 5 2023-10-18T08:55:00.088 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:55:00.085+0000 7fa50ce11040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T08:55:00.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.254+0000 7f2f75026640 1 -- 172.21.15.144:0/60371251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7015b670 msgr2=0x7f2f700ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:00.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.254+0000 7f2f75026640 1 --2- 172.21.15.144:0/60371251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7015b670 0x7f2f700ce520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2f60005fd0 tx=0x7f2f6002bcf0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.258 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.254+0000 7f2f75026640 1 -- 172.21.15.144:0/60371251 shutdown_connections 2023-10-18T08:55:00.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.254+0000 7f2f75026640 1 --2- 172.21.15.144:0/60371251 >> v2:172.21.15.40:3300/0 conn(0x7f2f700cea60 0x7f2f700d1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.254+0000 7f2f75026640 1 --2- 172.21.15.144:0/60371251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7015b670 0x7f2f700ce520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.254+0000 7f2f75026640 1 -- 172.21.15.144:0/60371251 >> 172.21.15.144:0/60371251 conn(0x7f2f7008ae20 msgr2=0x7f2f7009a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:00.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.254+0000 7f2f75026640 1 -- 172.21.15.144:0/60371251 shutdown_connections 2023-10-18T08:55:00.259 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.254+0000 7f2f75026640 1 -- 172.21.15.144:0/60371251 wait complete. 2023-10-18T08:55:00.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.256+0000 7f2f75026640 1 Processor -- start 2023-10-18T08:55:00.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.256+0000 7f2f75026640 1 -- start start 2023-10-18T08:55:00.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.257+0000 7f2f75026640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f700cea60 0x7f2f701497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:00.262 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.257+0000 7f2f75026640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f700d18a0 0x7f2f700ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:00.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.257+0000 7f2f75026640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f7015b670 0x7f2f700cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:00.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.257+0000 7f2f75026640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f700c8540 con 0x7f2f7015b670 2023-10-18T08:55:00.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.257+0000 7f2f75026640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f70149d30 con 0x7f2f700d18a0 2023-10-18T08:55:00.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.257+0000 7f2f75026640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7015c8d0 con 0x7f2f700cea60 2023-10-18T08:55:00.263 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.257+0000 7f2f6ed9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f700cea60 0x7f2f701497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:00.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.258+0000 7f2f6e59c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f700d18a0 0x7f2f700ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:00.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.258+0000 7f2f6ed9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f700cea60 0x7f2f701497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:50574/0 (socket says 172.21.15.144:50574) 2023-10-18T08:55:00.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.258+0000 7f2f6ed9d640 1 -- 172.21.15.144:0/3625838399 learned_addr learned my addr 172.21.15.144:0/3625838399 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-18T08:55:00.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.258+0000 7f2f6f59e640 1 --2- 172.21.15.144:0/3625838399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f7015b670 0x7f2f700cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:00.264 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.258+0000 7f2f6ed9d640 1 -- 172.21.15.144:0/3625838399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f700d18a0 msgr2=0x7f2f700ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:00.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.258+0000 7f2f6ed9d640 1 --2- 172.21.15.144:0/3625838399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f700d18a0 0x7f2f700ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.258+0000 7f2f6ed9d640 1 -- 172.21.15.144:0/3625838399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f7015b670 msgr2=0x7f2f700cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:00.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.258+0000 7f2f6ed9d640 1 --2- 172.21.15.144:0/3625838399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f7015b670 0x7f2f700cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.258+0000 7f2f6ed9d640 1 -- 172.21.15.144:0/3625838399 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f60009560 con 0x7f2f700cea60 2023-10-18T08:55:00.265 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.258+0000 7f2f6e59c640 1 --2- 172.21.15.144:0/3625838399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f700d18a0 0x7f2f700ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T08:55:00.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.258+0000 7f2f6ed9d640 1 --2- 172.21.15.144:0/3625838399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f700cea60 0x7f2f701497f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009850 tx=0x7f2f6002d9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:00.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.259+0000 7f2f577fe640 1 -- 172.21.15.144:0/3625838399 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6002ddc0 con 0x7f2f700cea60 2023-10-18T08:55:00.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.259+0000 7f2f577fe640 1 -- 172.21.15.144:0/3625838399 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2f600036e0 con 0x7f2f700cea60 2023-10-18T08:55:00.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.259+0000 7f2f75026640 1 -- 172.21.15.144:0/3625838399 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f700c87b0 con 0x7f2f700cea60 2023-10-18T08:55:00.266 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.259+0000 7f2f577fe640 1 -- 172.21.15.144:0/3625838399 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f60003a90 con 0x7f2f700cea60 2023-10-18T08:55:00.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.259+0000 7f2f75026640 1 -- 172.21.15.144:0/3625838399 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7015cbb0 con 0x7f2f700cea60 2023-10-18T08:55:00.267 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.259+0000 7f2f577fe640 1 -- 172.21.15.144:0/3625838399 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2f6003e450 con 0x7f2f700cea60 2023-10-18T08:55:00.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.259+0000 7f2f577fe640 1 -- 172.21.15.144:0/3625838399 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f2f60034a10 con 0x7f2f700cea60 2023-10-18T08:55:00.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.260+0000 7f2f75026640 1 -- 172.21.15.144:0/3625838399 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f700c8cd0 con 0x7f2f700cea60 2023-10-18T08:55:00.268 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.262+0000 7f2f577fe640 1 -- 172.21.15.144:0/3625838399 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f2f700c8cd0 con 0x7f2f700cea60 2023-10-18T08:55:00.343 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.340+0000 7f2f75026640 1 -- 172.21.15.144:0/3625838399 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5} v 0) v1 -- 0x7f2f7015ed40 con 0x7f2f700cea60 2023-10-18T08:55:00.351 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.348+0000 7f2f577fe640 1 -- 172.21.15.144:0/3625838399 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2f60043020 con 0x7f2f700cea60 2023-10-18T08:55:00.351 INFO:teuthology.orchestra.run.smithi144.stdout:5 2023-10-18T08:55:00.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.349+0000 7f2f75026640 1 -- 172.21.15.144:0/3625838399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f700cea60 msgr2=0x7f2f701497f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:00.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.349+0000 7f2f75026640 1 --2- 172.21.15.144:0/3625838399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f700cea60 0x7f2f701497f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009850 tx=0x7f2f6002d9c0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.350+0000 7f2f75026640 1 -- 172.21.15.144:0/3625838399 shutdown_connections 2023-10-18T08:55:00.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.350+0000 7f2f75026640 1 --2- 172.21.15.144:0/3625838399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f7015b670 0x7f2f700cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.353 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.350+0000 7f2f75026640 1 --2- 172.21.15.144:0/3625838399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f700d18a0 0x7f2f700ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.350+0000 7f2f75026640 1 --2- 172.21.15.144:0/3625838399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f700cea60 0x7f2f701497f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.350+0000 7f2f75026640 1 -- 172.21.15.144:0/3625838399 >> 172.21.15.144:0/3625838399 conn(0x7f2f7008ae20 msgr2=0x7f2f7006b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:00.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.350+0000 7f2f75026640 1 -- 172.21.15.144:0/3625838399 shutdown_connections 2023-10-18T08:55:00.354 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.350+0000 7f2f75026640 1 -- 172.21.15.144:0/3625838399 wait complete. 2023-10-18T08:55:00.368 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 6 2023-10-18T08:55:00.375 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:55:00.372+0000 7f700aa30040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T08:55:00.552 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.549+0000 7fa622c2e640 1 -- 172.21.15.144:0/2936199326 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa61c15cd60 msgr2=0x7fa61c15d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:00.553 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.549+0000 7fa622c2e640 1 --2- 172.21.15.144:0/2936199326 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa61c15cd60 0x7fa61c15d1d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0711d0 tx=0x7fa61402c260 comp rx=0 tx=0).stop 2023-10-18T08:55:00.553 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.549+0000 7fa622c2e640 1 -- 172.21.15.144:0/2936199326 shutdown_connections 2023-10-18T08:55:00.553 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.549+0000 7fa622c2e640 1 --2- 172.21.15.144:0/2936199326 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa61c15cd60 0x7fa61c15d1d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.549+0000 7fa622c2e640 1 --2- 172.21.15.144:0/2936199326 >> v2:172.21.15.144:3300/0 conn(0x7fa61c070150 0x7fa61c0705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.549+0000 7fa622c2e640 1 --2- 172.21.15.144:0/2936199326 >> v2:172.21.15.40:3300/0 conn(0x7fa61c0acc90 0x7fa61c0b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.549+0000 7fa622c2e640 1 -- 172.21.15.144:0/2936199326 >> 172.21.15.144:0/2936199326 conn(0x7fa61c069b40 msgr2=0x7fa61c069f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:00.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.549+0000 7fa622c2e640 1 -- 172.21.15.144:0/2936199326 shutdown_connections 2023-10-18T08:55:00.554 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.549+0000 7fa622c2e640 1 -- 172.21.15.144:0/2936199326 wait complete. 2023-10-18T08:55:00.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.551+0000 7fa622c2e640 1 Processor -- start 2023-10-18T08:55:00.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.551+0000 7fa622c2e640 1 -- start start 2023-10-18T08:55:00.557 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.551+0000 7fa622c2e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c070150 0x7fa61c1f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:00.558 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.552+0000 7fa622c2e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa61c0acc90 0x7fa61c156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:00.558 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.552+0000 7fa622c2e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa61c15cd60 0x7fa61c0aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:00.558 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.552+0000 7fa622c2e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c156c40 con 0x7fa61c070150 2023-10-18T08:55:00.558 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.552+0000 7fa622c2e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0aadc0 con 0x7fa61c15cd60 2023-10-18T08:55:00.558 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.552+0000 7fa622c2e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0ab070 con 0x7fa61c0acc90 2023-10-18T08:55:00.559 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.552+0000 7fa62142b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa61c0acc90 0x7fa61c156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:00.559 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.552+0000 7fa62142b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa61c0acc90 0x7fa61c156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:50598/0 (socket says 172.21.15.144:50598) 2023-10-18T08:55:00.559 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.552+0000 7fa62142b640 1 -- 172.21.15.144:0/2875392076 learned_addr learned my addr 172.21.15.144:0/2875392076 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-18T08:55:00.560 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.552+0000 7fa62242d640 1 --2- 172.21.15.144:0/2875392076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa61c15cd60 0x7fa61c0aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:00.560 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.552+0000 7fa621c2c640 1 --2- 172.21.15.144:0/2875392076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c070150 0x7fa61c1f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:00.560 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.552+0000 7fa62142b640 1 -- 172.21.15.144:0/2875392076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa61c15cd60 msgr2=0x7fa61c0aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:00.560 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.552+0000 7fa62142b640 1 --2- 172.21.15.144:0/2875392076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa61c15cd60 0x7fa61c0aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.560 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.552+0000 7fa62142b640 1 -- 172.21.15.144:0/2875392076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c070150 msgr2=0x7fa61c1f25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:00.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.552+0000 7fa62142b640 1 --2- 172.21.15.144:0/2875392076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c070150 0x7fa61c1f25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.552+0000 7fa62142b640 1 -- 172.21.15.144:0/2875392076 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60c009490 con 0x7fa61c0acc90 2023-10-18T08:55:00.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.553+0000 7fa62142b640 1 --2- 172.21.15.144:0/2875392076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa61c0acc90 0x7fa61c156700 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00ea60 tx=0x7fa60c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:00.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.553+0000 7fa6127fc640 1 -- 172.21.15.144:0/2875392076 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c00cde0 con 0x7fa61c0acc90 2023-10-18T08:55:00.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.553+0000 7fa6127fc640 1 -- 172.21.15.144:0/2875392076 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa60c004020 con 0x7fa61c0acc90 2023-10-18T08:55:00.561 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.553+0000 7fa622c2e640 1 -- 172.21.15.144:0/2875392076 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa614009900 con 0x7fa61c0acc90 2023-10-18T08:55:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.553+0000 7fa6127fc640 1 -- 172.21.15.144:0/2875392076 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c004410 con 0x7fa61c0acc90 2023-10-18T08:55:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.553+0000 7fa622c2e640 1 -- 172.21.15.144:0/2875392076 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61c0ab630 con 0x7fa61c0acc90 2023-10-18T08:55:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.553+0000 7fa6127fc640 1 -- 172.21.15.144:0/2875392076 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa60c005020 con 0x7fa61c0acc90 2023-10-18T08:55:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.553+0000 7fa6127fc640 1 -- 172.21.15.144:0/2875392076 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fa60c014070 con 0x7fa61c0acc90 2023-10-18T08:55:00.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.554+0000 7fa622c2e640 1 -- 172.21.15.144:0/2875392076 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61c0ccb70 con 0x7fa61c0acc90 2023-10-18T08:55:00.563 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.557+0000 7fa6127fc640 1 -- 172.21.15.144:0/2875392076 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fa60c0047c0 con 0x7fa61c0acc90 2023-10-18T08:55:00.637 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.633+0000 7fa622c2e640 1 -- 172.21.15.144:0/2875392076 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6} v 0) v1 -- 0x7fa61c073570 con 0x7fa61c0acc90 2023-10-18T08:55:00.642 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.639+0000 7fa6127fc640 1 -- 172.21.15.144:0/2875392076 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa60c004c20 con 0x7fa61c0acc90 2023-10-18T08:55:00.642 INFO:teuthology.orchestra.run.smithi144.stdout:6 2023-10-18T08:55:00.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.640+0000 7fa622c2e640 1 -- 172.21.15.144:0/2875392076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa61c0acc90 msgr2=0x7fa61c156700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:00.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.640+0000 7fa622c2e640 1 --2- 172.21.15.144:0/2875392076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa61c0acc90 0x7fa61c156700 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00ea60 tx=0x7fa60c00ef20 comp rx=0 tx=0).stop 2023-10-18T08:55:00.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.640+0000 7fa622c2e640 1 -- 172.21.15.144:0/2875392076 shutdown_connections 2023-10-18T08:55:00.644 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.640+0000 7fa622c2e640 1 --2- 172.21.15.144:0/2875392076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa61c15cd60 0x7fa61c0aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.640+0000 7fa622c2e640 1 --2- 172.21.15.144:0/2875392076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa61c0acc90 0x7fa61c156700 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.640+0000 7fa622c2e640 1 --2- 172.21.15.144:0/2875392076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa61c070150 0x7fa61c1f25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.640+0000 7fa622c2e640 1 -- 172.21.15.144:0/2875392076 >> 172.21.15.144:0/2875392076 conn(0x7fa61c069b40 msgr2=0x7fa61c0cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:00.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.641+0000 7fa622c2e640 1 -- 172.21.15.144:0/2875392076 shutdown_connections 2023-10-18T08:55:00.645 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.641+0000 7fa622c2e640 1 -- 172.21.15.144:0/2875392076 wait complete. 2023-10-18T08:55:00.660 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 7 2023-10-18T08:55:00.693 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:55:00.690+0000 7f700aa30040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T08:55:00.756 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:55:00.753+0000 7f700aa30040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T08:55:00.874 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:55:00.871+0000 7f700aa30040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T08:55:00.901 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.897+0000 7f7d08505640 1 -- 172.21.15.144:0/3224347929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d0015b6c0 msgr2=0x7f7d000b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:00.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.897+0000 7f7d08505640 1 --2- 172.21.15.144:0/3224347929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d0015b6c0 0x7f7d000b1e40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc005fd0 tx=0x7f7cfc02bd90 comp rx=0 tx=0).stop 2023-10-18T08:55:00.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.898+0000 7f7d08505640 1 -- 172.21.15.144:0/3224347929 shutdown_connections 2023-10-18T08:55:00.902 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.898+0000 7f7d08505640 1 --2- 172.21.15.144:0/3224347929 >> v2:172.21.15.40:3300/0 conn(0x7f7d000cff00 0x7f7d000d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.898+0000 7f7d08505640 1 --2- 172.21.15.144:0/3224347929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d0015b6c0 0x7f7d000b1e40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.898+0000 7f7d08505640 1 -- 172.21.15.144:0/3224347929 >> 172.21.15.144:0/3224347929 conn(0x7f7d0009a960 msgr2=0x7f7d00088fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:00.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.898+0000 7f7d08505640 1 -- 172.21.15.144:0/3224347929 shutdown_connections 2023-10-18T08:55:00.903 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.898+0000 7f7d08505640 1 -- 172.21.15.144:0/3224347929 wait complete. 2023-10-18T08:55:00.904 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.900+0000 7f7d08505640 1 Processor -- start 2023-10-18T08:55:00.905 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.901+0000 7f7d08505640 1 -- start start 2023-10-18T08:55:00.906 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.902+0000 7f7d08505640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d000cff00 0x7f7d0015e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:00.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.902+0000 7f7d08505640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d000d1aa0 0x7f7d000b6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:00.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.902+0000 7f7d08505640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d0015b6c0 0x7f7d0015a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:00.907 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.902+0000 7f7d08505640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d000b6340 con 0x7f7d0015b6c0 2023-10-18T08:55:00.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.902+0000 7f7d08505640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0015e9c0 con 0x7f7d000cff00 2023-10-18T08:55:00.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.902+0000 7f7d08505640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0015ec10 con 0x7f7d000d1aa0 2023-10-18T08:55:00.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.902+0000 7f7d062a1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d000cff00 0x7f7d0015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:00.908 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.902+0000 7f7d05aa0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d000d1aa0 0x7f7d000b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:00.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.902+0000 7f7d062a1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d000cff00 0x7f7d0015e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58626/0 (socket says 172.21.15.144:58626) 2023-10-18T08:55:00.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.902+0000 7f7d062a1640 1 -- 172.21.15.144:0/439976246 learned_addr learned my addr 172.21.15.144:0/439976246 (peer_addr_for_me v2:172.21.15.144:0/0) 2023-10-18T08:55:00.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.903+0000 7f7d06aa2640 1 --2- 172.21.15.144:0/439976246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d0015b6c0 0x7f7d0015a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:00.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.903+0000 7f7d05aa0640 1 -- 172.21.15.144:0/439976246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d000cff00 msgr2=0x7f7d0015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:00.909 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.903+0000 7f7d05aa0640 1 --2- 172.21.15.144:0/439976246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d000cff00 0x7f7d0015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.903+0000 7f7d05aa0640 1 -- 172.21.15.144:0/439976246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d0015b6c0 msgr2=0x7f7d0015a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:00.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.903+0000 7f7d05aa0640 1 --2- 172.21.15.144:0/439976246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d0015b6c0 0x7f7d0015a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:00.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.903+0000 7f7d05aa0640 1 -- 172.21.15.144:0/439976246 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc009560 con 0x7f7d000d1aa0 2023-10-18T08:55:00.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.903+0000 7f7d062a1640 1 --2- 172.21.15.144:0/439976246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d000cff00 0x7f7d0015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T08:55:00.910 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.903+0000 7f7d05aa0640 1 --2- 172.21.15.144:0/439976246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d000d1aa0 0x7f7d000b6760 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000d750 tx=0x7f7cf000dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:00.911 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.903+0000 7f7cf6ffd640 1 -- 172.21.15.144:0/439976246 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0004220 con 0x7f7d000d1aa0 2023-10-18T08:55:00.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.903+0000 7f7cf6ffd640 1 -- 172.21.15.144:0/439976246 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7cf0004470 con 0x7f7d000d1aa0 2023-10-18T08:55:00.912 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.903+0000 7f7cf6ffd640 1 -- 172.21.15.144:0/439976246 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0009c10 con 0x7f7d000d1aa0 2023-10-18T08:55:00.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.903+0000 7f7d08505640 1 -- 172.21.15.144:0/439976246 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d000b59a0 con 0x7f7d000d1aa0 2023-10-18T08:55:00.913 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.904+0000 7f7d08505640 1 -- 172.21.15.144:0/439976246 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d000b5f60 con 0x7f7d000d1aa0 2023-10-18T08:55:00.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.905+0000 7f7cf6ffd640 1 -- 172.21.15.144:0/439976246 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 2) v1 ==== 44566+0+0 (secure 0 0 0) 0x7f7cf0005020 con 0x7f7d000d1aa0 2023-10-18T08:55:00.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.905+0000 7f7d08505640 1 -- 172.21.15.144:0/439976246 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d000b6480 con 0x7f7d000d1aa0 2023-10-18T08:55:00.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.905+0000 7f7cf6ffd640 1 -- 172.21.15.144:0/439976246 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f7cf0049a00 con 0x7f7d000d1aa0 2023-10-18T08:55:00.914 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.908+0000 7f7cf6ffd640 1 -- 172.21.15.144:0/439976246 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f7d000b6480 con 0x7f7d000d1aa0 2023-10-18T08:55:00.991 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.988+0000 7f7d08505640 1 -- 172.21.15.144:0/439976246 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7} v 0) v1 -- 0x7f7d000b6480 con 0x7f7d000d1aa0 2023-10-18T08:55:01.001 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.998+0000 7f7cf6ffd640 1 -- 172.21.15.144:0/439976246 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7d000b6480 con 0x7f7d000d1aa0 2023-10-18T08:55:01.001 INFO:teuthology.orchestra.run.smithi144.stdout:7 2023-10-18T08:55:01.002 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.999+0000 7f7d08505640 1 -- 172.21.15.144:0/439976246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d000d1aa0 msgr2=0x7f7d000b6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:01.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.999+0000 7f7d08505640 1 --2- 172.21.15.144:0/439976246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d000d1aa0 0x7f7d000b6760 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000d750 tx=0x7f7cf000dc10 comp rx=0 tx=0).stop 2023-10-18T08:55:01.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.999+0000 7f7d08505640 1 -- 172.21.15.144:0/439976246 shutdown_connections 2023-10-18T08:55:01.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.999+0000 7f7d08505640 1 --2- 172.21.15.144:0/439976246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d0015b6c0 0x7f7d0015a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:01.003 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.999+0000 7f7d08505640 1 --2- 172.21.15.144:0/439976246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d000d1aa0 0x7f7d000b6760 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:01.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.999+0000 7f7d08505640 1 --2- 172.21.15.144:0/439976246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d000cff00 0x7f7d0015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:01.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.999+0000 7f7d08505640 1 -- 172.21.15.144:0/439976246 >> 172.21.15.144:0/439976246 conn(0x7f7d0009a960 msgr2=0x7f7d000ca6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:01.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.999+0000 7f7d08505640 1 -- 172.21.15.144:0/439976246 shutdown_connections 2023-10-18T08:55:01.004 INFO:teuthology.orchestra.run.smithi144.stderr:2023-10-18T08:55:00.999+0000 7f7d08505640 1 -- 172.21.15.144:0/439976246 wait complete. 2023-10-18T08:55:01.017 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-18T08:55:01.018 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-18T08:55:01.027 INFO:tasks.ceph.osd.0:Started 2023-10-18T08:55:01.027 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-18T08:55:01.028 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-18T08:55:01.036 INFO:tasks.ceph.osd.1:Started 2023-10-18T08:55:01.037 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-18T08:55:01.037 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-18T08:55:01.044 INFO:tasks.ceph.osd.2:Started 2023-10-18T08:55:01.044 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-18T08:55:01.045 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-18T08:55:01.051 INFO:tasks.ceph.osd.3:Started 2023-10-18T08:55:01.052 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-18T08:55:01.052 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-18T08:55:01.060 INFO:tasks.ceph.osd.4:Started 2023-10-18T08:55:01.061 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-18T08:55:01.061 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-18T08:55:01.069 INFO:tasks.ceph.osd.5:Started 2023-10-18T08:55:01.069 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-18T08:55:01.069 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-18T08:55:01.074 INFO:tasks.ceph.osd.6:Started 2023-10-18T08:55:01.075 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-18T08:55:01.075 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-18T08:55:01.080 INFO:tasks.ceph.osd.7:Started 2023-10-18T08:55:01.080 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:01.213 INFO:tasks.ceph.osd.5.smithi144.stderr:2023-10-18T08:55:01.210+0000 7fd7790d4500 -1 Falling back to public interface 2023-10-18T08:55:01.216 INFO:tasks.ceph.osd.4.smithi144.stderr:2023-10-18T08:55:01.213+0000 7fe248813500 -1 Falling back to public interface 2023-10-18T08:55:01.225 INFO:tasks.ceph.osd.6.smithi144.stderr:2023-10-18T08:55:01.222+0000 7ff56e01a500 -1 Falling back to public interface 2023-10-18T08:55:01.231 INFO:tasks.ceph.osd.7.smithi144.stderr:2023-10-18T08:55:01.228+0000 7fb16f813500 -1 Falling back to public interface 2023-10-18T08:55:01.233 INFO:tasks.ceph.osd.5.smithi144.stderr:2023-10-18T08:55:01.230+0000 7fd7790d4500 -1 osd.5 0 log_to_monitors true 2023-10-18T08:55:01.238 INFO:tasks.ceph.osd.4.smithi144.stderr:2023-10-18T08:55:01.235+0000 7fe248813500 -1 osd.4 0 log_to_monitors true 2023-10-18T08:55:01.242 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-10-18T08:55:01.240+0000 7fa22d60c500 -1 Falling back to public interface 2023-10-18T08:55:01.247 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-10-18T08:55:01.245+0000 7f16ee310500 -1 Falling back to public interface 2023-10-18T08:55:01.251 INFO:tasks.ceph.osd.6.smithi144.stderr:2023-10-18T08:55:01.248+0000 7ff56e01a500 -1 osd.6 0 log_to_monitors true 2023-10-18T08:55:01.258 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-10-18T08:55:01.256+0000 7fa63aa14500 -1 Falling back to public interface 2023-10-18T08:55:01.259 INFO:tasks.ceph.osd.7.smithi144.stderr:2023-10-18T08:55:01.255+0000 7fb16f813500 -1 osd.7 0 log_to_monitors true 2023-10-18T08:55:01.267 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-10-18T08:55:01.265+0000 7f0bf5c69500 -1 Falling back to public interface 2023-10-18T08:55:01.270 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-10-18T08:55:01.268+0000 7fa22d60c500 -1 osd.0 0 log_to_monitors true 2023-10-18T08:55:01.279 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-10-18T08:55:01.277+0000 7f16ee310500 -1 osd.1 0 log_to_monitors true 2023-10-18T08:55:01.286 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-10-18T08:55:01.284+0000 7fa63aa14500 -1 osd.3 0 log_to_monitors true 2023-10-18T08:55:01.311 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-10-18T08:55:01.309+0000 7f0bf5c69500 -1 osd.2 0 log_to_monitors true 2023-10-18T08:55:01.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.327+0000 7f05903d0640 1 -- 172.21.15.40:0/3377132831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f058813c470 msgr2=0x7f058813f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:01.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.327+0000 7f05903d0640 1 --2- 172.21.15.40:0/3377132831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f058813c470 0x7f058813f5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f058806c830 tx=0x7f058402bd70 comp rx=0 tx=0).stop 2023-10-18T08:55:01.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.327+0000 7f05903d0640 1 -- 172.21.15.40:0/3377132831 shutdown_connections 2023-10-18T08:55:01.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.327+0000 7f05903d0640 1 --2- 172.21.15.40:0/3377132831 >> v2:172.21.15.40:3300/0 conn(0x7f05880ca210 0x7f058815d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:01.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.327+0000 7f05903d0640 1 --2- 172.21.15.40:0/3377132831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f058813c470 0x7f058813f5a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:01.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.327+0000 7f05903d0640 1 -- 172.21.15.40:0/3377132831 >> 172.21.15.40:0/3377132831 conn(0x7f058806cee0 msgr2=0x7f0588069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:01.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.328+0000 7f05903d0640 1 -- 172.21.15.40:0/3377132831 shutdown_connections 2023-10-18T08:55:01.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.328+0000 7f05903d0640 1 -- 172.21.15.40:0/3377132831 wait complete. 2023-10-18T08:55:01.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.329+0000 7f05903d0640 1 Processor -- start 2023-10-18T08:55:01.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.330+0000 7f05903d0640 1 -- start start 2023-10-18T08:55:01.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.331+0000 7f05903d0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05880ca210 0x7f05880710a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:01.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.331+0000 7f05903d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f058813c470 0x7f058813add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:01.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.331+0000 7f05903d0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f058815cf30 0x7f058813f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:01.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.331+0000 7f05903d0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0588070f00 con 0x7f058815cf30 2023-10-18T08:55:01.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.331+0000 7f05903d0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05880715e0 con 0x7f058813c470 2023-10-18T08:55:01.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.331+0000 7f05903d0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058813b310 con 0x7f05880ca210 2023-10-18T08:55:01.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.331+0000 7f058e16c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05880ca210 0x7f05880710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:01.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.331+0000 7f058e16c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05880ca210 0x7f05880710a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:59300/0 (socket says 172.21.15.40:59300) 2023-10-18T08:55:01.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.331+0000 7f058e16c640 1 -- 172.21.15.40:0/3331866319 learned_addr learned my addr 172.21.15.40:0/3331866319 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:01.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.331+0000 7f058e96d640 1 --2- 172.21.15.40:0/3331866319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f058815cf30 0x7f058813f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:01.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.331+0000 7f058d96b640 1 --2- 172.21.15.40:0/3331866319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f058813c470 0x7f058813add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:01.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.331+0000 7f058e16c640 1 -- 172.21.15.40:0/3331866319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f058813c470 msgr2=0x7f058813add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:01.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.331+0000 7f058e16c640 1 --2- 172.21.15.40:0/3331866319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f058813c470 0x7f058813add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:01.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.331+0000 7f058e16c640 1 -- 172.21.15.40:0/3331866319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f058815cf30 msgr2=0x7f058813f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:01.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.331+0000 7f058e16c640 1 --2- 172.21.15.40:0/3331866319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f058815cf30 0x7f058813f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:01.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.331+0000 7f058e16c640 1 -- 172.21.15.40:0/3331866319 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0584009490 con 0x7f05880ca210 2023-10-18T08:55:01.337 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:55:01.332+0000 7fa50ce11040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T08:55:01.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.332+0000 7f058e16c640 1 --2- 172.21.15.40:0/3331866319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05880ca210 0x7f05880710a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0584000c00 tx=0x7f05840038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:01.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.332+0000 7f057effd640 1 -- 172.21.15.40:0/3331866319 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584003ea0 con 0x7f05880ca210 2023-10-18T08:55:01.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.332+0000 7f057effd640 1 -- 172.21.15.40:0/3331866319 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0584034440 con 0x7f05880ca210 2023-10-18T08:55:01.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.332+0000 7f057effd640 1 -- 172.21.15.40:0/3331866319 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584034850 con 0x7f05880ca210 2023-10-18T08:55:01.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.332+0000 7f05903d0640 1 -- 172.21.15.40:0/3331866319 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05880ace20 con 0x7f05880ca210 2023-10-18T08:55:01.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.332+0000 7f05903d0640 1 -- 172.21.15.40:0/3331866319 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05880ad3e0 con 0x7f05880ca210 2023-10-18T08:55:01.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.333+0000 7f057effd640 1 -- 172.21.15.40:0/3331866319 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 2) v1 ==== 44566+0+0 (secure 0 0 0) 0x7f058403b040 con 0x7f05880ca210 2023-10-18T08:55:01.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.334+0000 7f05903d0640 1 -- 172.21.15.40:0/3331866319 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0588066fd0 con 0x7f05880ca210 2023-10-18T08:55:01.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.334+0000 7f057effd640 1 -- 172.21.15.40:0/3331866319 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f058406bac0 con 0x7f05880ca210 2023-10-18T08:55:01.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.336+0000 7f057effd640 1 -- 172.21.15.40:0/3331866319 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f0584033990 con 0x7f05880ca210 2023-10-18T08:55:01.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.404+0000 7f05903d0640 1 -- 172.21.15.40:0/3331866319 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f058815a250 con 0x7f05880ca210 2023-10-18T08:55:01.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.405+0000 7f057effd640 1 -- 172.21.15.40:0/3331866319 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0584041030 con 0x7f05880ca210 2023-10-18T08:55:01.408 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:01.408 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":10,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:00.991196+0000","last_up_change":"0.000000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:01.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.407+0000 7f05903d0640 1 -- 172.21.15.40:0/3331866319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05880ca210 msgr2=0x7f05880710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:01.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.407+0000 7f05903d0640 1 --2- 172.21.15.40:0/3331866319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05880ca210 0x7f05880710a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0584000c00 tx=0x7f05840038a0 comp rx=0 tx=0).stop 2023-10-18T08:55:01.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.407+0000 7f05903d0640 1 -- 172.21.15.40:0/3331866319 shutdown_connections 2023-10-18T08:55:01.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.407+0000 7f05903d0640 1 --2- 172.21.15.40:0/3331866319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f058815cf30 0x7f058813f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:01.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.407+0000 7f05903d0640 1 --2- 172.21.15.40:0/3331866319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f058813c470 0x7f058813add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:01.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.407+0000 7f05903d0640 1 --2- 172.21.15.40:0/3331866319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05880ca210 0x7f05880710a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:01.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.407+0000 7f05903d0640 1 -- 172.21.15.40:0/3331866319 >> 172.21.15.40:0/3331866319 conn(0x7f058806cee0 msgr2=0x7f058813c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:01.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.407+0000 7f05903d0640 1 -- 172.21.15.40:0/3331866319 shutdown_connections 2023-10-18T08:55:01.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:01.407+0000 7f05903d0640 1 -- 172.21.15.40:0/3331866319 wait complete. 2023-10-18T08:55:01.426 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-18T08:55:01.426 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-18T08:55:01.681 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:55:01.678+0000 7fa50ce11040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T08:55:01.744 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:55:01.742+0000 7fa50ce11040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T08:55:01.863 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-18T08:55:01.861+0000 7fa50ce11040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T08:55:01.922 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:55:01.919+0000 7f6fab662640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-18T08:55:01.925 INFO:tasks.ceph.mgr.x.smithi144.stderr:2023-10-18T08:55:01.922+0000 7f6fab662640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-18T08:55:02.232 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:02.433 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.431+0000 7fe3171ae640 1 -- 172.21.15.40:0/2532772745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe310158920 msgr2=0x7fe31006cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:02.434 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.431+0000 7fe3171ae640 1 --2- 172.21.15.40:0/2532772745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe310158920 0x7fe31006cfd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe300005fd0 tx=0x7fe30002d160 comp rx=0 tx=0).stop 2023-10-18T08:55:02.434 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.431+0000 7fe3171ae640 1 -- 172.21.15.40:0/2532772745 shutdown_connections 2023-10-18T08:55:02.434 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.431+0000 7fe3171ae640 1 --2- 172.21.15.40:0/2532772745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310070c60 0x7fe3100710b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:02.434 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.431+0000 7fe3171ae640 1 --2- 172.21.15.40:0/2532772745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe310158920 0x7fe31006cfd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:02.435 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.431+0000 7fe3171ae640 1 -- 172.21.15.40:0/2532772745 >> 172.21.15.40:0/2532772745 conn(0x7fe310069930 msgr2=0x7fe310069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:02.435 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.431+0000 7fe3171ae640 1 -- 172.21.15.40:0/2532772745 shutdown_connections 2023-10-18T08:55:02.435 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.431+0000 7fe3171ae640 1 -- 172.21.15.40:0/2532772745 wait complete. 2023-10-18T08:55:02.436 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.433+0000 7fe3171ae640 1 Processor -- start 2023-10-18T08:55:02.436 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.433+0000 7fe3171ae640 1 -- start start 2023-10-18T08:55:02.438 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.434+0000 7fe3171ae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310070c60 0x7fe31014b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:02.438 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.434+0000 7fe3171ae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3101576f0 0x7fe3101522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:02.438 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.434+0000 7fe3171ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe310158920 0x7fe310099dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:02.438 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.434+0000 7fe3171ae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe310097dc0 con 0x7fe310158920 2023-10-18T08:55:02.438 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.434+0000 7fe3171ae640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31014bc50 con 0x7fe3101576f0 2023-10-18T08:55:02.439 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.434+0000 7fe3171ae640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe310098600 con 0x7fe310070c60 2023-10-18T08:55:02.439 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.434+0000 7fe3169ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe310158920 0x7fe310099dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:02.439 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.434+0000 7fe3169ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe310158920 0x7fe310099dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43488/0 (socket says 172.21.15.40:43488) 2023-10-18T08:55:02.439 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.434+0000 7fe3169ad640 1 -- 172.21.15.40:0/3791828563 learned_addr learned my addr 172.21.15.40:0/3791828563 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:02.439 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.435+0000 7fe3159ab640 1 --2- 172.21.15.40:0/3791828563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3101576f0 0x7fe3101522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:02.440 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.435+0000 7fe3161ac640 1 --2- 172.21.15.40:0/3791828563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310070c60 0x7fe31014b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:02.440 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.435+0000 7fe3169ad640 1 -- 172.21.15.40:0/3791828563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310070c60 msgr2=0x7fe31014b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:02.440 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.435+0000 7fe3169ad640 1 --2- 172.21.15.40:0/3791828563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310070c60 0x7fe31014b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:02.440 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.435+0000 7fe3169ad640 1 -- 172.21.15.40:0/3791828563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3101576f0 msgr2=0x7fe3101522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:02.440 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.435+0000 7fe3169ad640 1 --2- 172.21.15.40:0/3791828563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3101576f0 0x7fe3101522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:02.441 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.435+0000 7fe3169ad640 1 -- 172.21.15.40:0/3791828563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30000a670 con 0x7fe310158920 2023-10-18T08:55:02.441 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.435+0000 7fe3161ac640 1 --2- 172.21.15.40:0/3791828563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310070c60 0x7fe31014b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:02.441 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.435+0000 7fe3159ab640 1 --2- 172.21.15.40:0/3791828563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3101576f0 0x7fe3101522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:02.441 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.435+0000 7fe3169ad640 1 --2- 172.21.15.40:0/3791828563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe310158920 0x7fe310099dd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe30800eb10 tx=0x7fe30800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:02.441 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.435+0000 7fe306ffd640 1 -- 172.21.15.40:0/3791828563 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe308004b20 con 0x7fe310158920 2023-10-18T08:55:02.442 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.435+0000 7fe306ffd640 1 -- 172.21.15.40:0/3791828563 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe308004ca0 con 0x7fe310158920 2023-10-18T08:55:02.442 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.435+0000 7fe306ffd640 1 -- 172.21.15.40:0/3791828563 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe308005280 con 0x7fe310158920 2023-10-18T08:55:02.442 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.435+0000 7fe3171ae640 1 -- 172.21.15.40:0/3791828563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe310152c90 con 0x7fe310158920 2023-10-18T08:55:02.442 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.436+0000 7fe3171ae640 1 -- 172.21.15.40:0/3791828563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe310153250 con 0x7fe310158920 2023-10-18T08:55:02.443 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.437+0000 7fe3171ae640 1 -- 172.21.15.40:0/3791828563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe310153440 con 0x7fe310158920 2023-10-18T08:55:02.443 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.438+0000 7fe306ffd640 1 -- 172.21.15.40:0/3791828563 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 3) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fe3080053e0 con 0x7fe310158920 2023-10-18T08:55:02.444 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.438+0000 7fe306ffd640 1 --2- 172.21.15.40:0/3791828563 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe2f406ac90 0x7fe2f406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:02.444 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.439+0000 7fe306ffd640 1 -- 172.21.15.40:0/3791828563 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1883+0+0 (secure 0 0 0) 0x7fe308014070 con 0x7fe310158920 2023-10-18T08:55:02.444 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.439+0000 7fe3161ac640 1 --2- 172.21.15.40:0/3791828563 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe2f406ac90 0x7fe2f406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:02.444 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.440+0000 7fe3161ac640 1 --2- 172.21.15.40:0/3791828563 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe2f406ac90 0x7fe2f406d150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe30c000f60 tx=0x7fe30c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:02.445 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.443+0000 7fe306ffd640 1 -- 172.21.15.40:0/3791828563 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3080553f0 con 0x7fe310158920 2023-10-18T08:55:02.641 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.639+0000 7fe3171ae640 1 -- 172.21.15.40:0/3791828563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe310066cf0 con 0x7fe310158920 2023-10-18T08:55:02.642 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.640+0000 7fe306ffd640 1 -- 172.21.15.40:0/3791828563 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fe308058a10 con 0x7fe310158920 2023-10-18T08:55:02.642 INFO:teuthology.misc.health.smithi040.stdout: 2023-10-18T08:55:02.643 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":11,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:01.994687+0000","last_up_change":"0.000000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:02.645 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.643+0000 7fe3171ae640 1 -- 172.21.15.40:0/3791828563 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe2f406ac90 msgr2=0x7fe2f406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:02.646 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.643+0000 7fe3171ae640 1 --2- 172.21.15.40:0/3791828563 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe2f406ac90 0x7fe2f406d150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe30c000f60 tx=0x7fe30c009040 comp rx=0 tx=0).stop 2023-10-18T08:55:02.646 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.643+0000 7fe3171ae640 1 -- 172.21.15.40:0/3791828563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe310158920 msgr2=0x7fe310099dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:02.646 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.643+0000 7fe3171ae640 1 --2- 172.21.15.40:0/3791828563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe310158920 0x7fe310099dd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe30800eb10 tx=0x7fe30800c2d0 comp rx=0 tx=0).stop 2023-10-18T08:55:02.646 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.643+0000 7fe3171ae640 1 -- 172.21.15.40:0/3791828563 shutdown_connections 2023-10-18T08:55:02.646 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.643+0000 7fe3171ae640 1 --2- 172.21.15.40:0/3791828563 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe2f406ac90 0x7fe2f406d150 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:02.647 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.643+0000 7fe3171ae640 1 --2- 172.21.15.40:0/3791828563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe310158920 0x7fe310099dd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:02.647 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.643+0000 7fe3171ae640 1 --2- 172.21.15.40:0/3791828563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3101576f0 0x7fe3101522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:02.647 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.643+0000 7fe3171ae640 1 --2- 172.21.15.40:0/3791828563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe310070c60 0x7fe31014b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:02.647 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.643+0000 7fe3171ae640 1 -- 172.21.15.40:0/3791828563 >> 172.21.15.40:0/3791828563 conn(0x7fe310069930 msgr2=0x7fe310069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:02.647 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.643+0000 7fe3171ae640 1 -- 172.21.15.40:0/3791828563 shutdown_connections 2023-10-18T08:55:02.648 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:02.643+0000 7fe3171ae640 1 -- 172.21.15.40:0/3791828563 wait complete. 2023-10-18T08:55:02.659 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-18T08:55:03.060 INFO:tasks.ceph.osd.4.smithi144.stderr:2023-10-18T08:55:03.057+0000 7fe2449f2640 -1 osd.4 0 waiting for initial osdmap 2023-10-18T08:55:03.060 INFO:tasks.ceph.osd.5.smithi144.stderr:2023-10-18T08:55:03.058+0000 7fd775060640 -1 osd.5 0 waiting for initial osdmap 2023-10-18T08:55:03.061 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-10-18T08:55:03.059+0000 7fa636340640 -1 osd.3 0 waiting for initial osdmap 2023-10-18T08:55:03.062 INFO:tasks.ceph.osd.4.smithi144.stderr:2023-10-18T08:55:03.059+0000 7fe23d810640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T08:55:03.063 INFO:tasks.ceph.osd.5.smithi144.stderr:2023-10-18T08:55:03.060+0000 7fd76e694640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T08:55:03.064 INFO:tasks.ceph.osd.6.smithi144.stderr:2023-10-18T08:55:03.061+0000 7ff569a03640 -1 osd.6 0 waiting for initial osdmap 2023-10-18T08:55:03.065 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-10-18T08:55:03.062+0000 7fa22908b640 -1 osd.0 0 waiting for initial osdmap 2023-10-18T08:55:03.065 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-10-18T08:55:03.063+0000 7fa62f974640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T08:55:03.066 INFO:tasks.ceph.osd.6.smithi144.stderr:2023-10-18T08:55:03.063+0000 7ff563037640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T08:55:03.066 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-10-18T08:55:03.064+0000 7f0bf1bf5640 -1 osd.2 0 waiting for initial osdmap 2023-10-18T08:55:03.067 INFO:tasks.ceph.osd.7.smithi144.stderr:2023-10-18T08:55:03.064+0000 7fb16b0be640 -1 osd.7 0 waiting for initial osdmap 2023-10-18T08:55:03.067 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-10-18T08:55:03.065+0000 7fa2226bf640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T08:55:03.069 INFO:tasks.ceph.osd.7.smithi144.stderr:2023-10-18T08:55:03.066+0000 7fb1646f2640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T08:55:03.069 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-10-18T08:55:03.067+0000 7f0beb229640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T08:55:03.070 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-10-18T08:55:03.068+0000 7f16eaab2640 -1 osd.1 0 waiting for initial osdmap 2023-10-18T08:55:03.073 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-10-18T08:55:03.072+0000 7f16e38d0640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T08:55:03.874 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:55:03.883 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:55:09.466 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:09.676 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.674+0000 7f8c81cea640 1 -- 172.21.15.40:0/245843567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7c0ce1c0 msgr2=0x7f8c7c0b09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:09.676 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.674+0000 7f8c81cea640 1 --2- 172.21.15.40:0/245843567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7c0ce1c0 0x7f8c7c0b09d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009860 tx=0x7f8c7402bde0 comp rx=0 tx=0).stop 2023-10-18T08:55:09.677 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.674+0000 7f8c81cea640 1 -- 172.21.15.40:0/245843567 shutdown_connections 2023-10-18T08:55:09.677 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.674+0000 7f8c81cea640 1 --2- 172.21.15.40:0/245843567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7c0ce1c0 0x7f8c7c0b09d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:09.677 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.674+0000 7f8c81cea640 1 -- 172.21.15.40:0/245843567 >> 172.21.15.40:0/245843567 conn(0x7f8c7c09a520 msgr2=0x7f8c7c0d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:09.677 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.674+0000 7f8c81cea640 1 -- 172.21.15.40:0/245843567 shutdown_connections 2023-10-18T08:55:09.677 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.674+0000 7f8c81cea640 1 -- 172.21.15.40:0/245843567 wait complete. 2023-10-18T08:55:09.678 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.676+0000 7f8c81cea640 1 Processor -- start 2023-10-18T08:55:09.678 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.676+0000 7f8c81cea640 1 -- start start 2023-10-18T08:55:09.680 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.677+0000 7f8c81cea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c7c0ce1c0 0x7f8c7c0c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:09.680 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.677+0000 7f8c81cea640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c7c0cfe80 0x7f8c7c1f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:09.680 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.677+0000 7f8c81cea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7c15e750 0x7f8c7c0aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:09.680 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.677+0000 7f8c81cea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c0c9af0 con 0x7f8c7c15e750 2023-10-18T08:55:09.680 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.677+0000 7f8c81cea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c156de0 con 0x7f8c7c0ce1c0 2023-10-18T08:55:09.681 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.677+0000 7f8c81cea640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c1570c0 con 0x7f8c7c0cfe80 2023-10-18T08:55:09.681 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.677+0000 7f8c814e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7c15e750 0x7f8c7c0aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:09.681 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.677+0000 7f8c814e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7c15e750 0x7f8c7c0aef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57400/0 (socket says 172.21.15.40:57400) 2023-10-18T08:55:09.681 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.677+0000 7f8c814e9640 1 -- 172.21.15.40:0/2354840419 learned_addr learned my addr 172.21.15.40:0/2354840419 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:09.681 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.677+0000 7f8c80ce8640 1 --2- 172.21.15.40:0/2354840419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c7c0ce1c0 0x7f8c7c0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:09.682 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.678+0000 7f8c7bfff640 1 --2- 172.21.15.40:0/2354840419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c7c0cfe80 0x7f8c7c1f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:09.682 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.678+0000 7f8c814e9640 1 -- 172.21.15.40:0/2354840419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c7c0cfe80 msgr2=0x7f8c7c1f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:09.682 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.678+0000 7f8c814e9640 1 --2- 172.21.15.40:0/2354840419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c7c0cfe80 0x7f8c7c1f1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:09.682 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.678+0000 7f8c814e9640 1 -- 172.21.15.40:0/2354840419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c7c0ce1c0 msgr2=0x7f8c7c0c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:09.682 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.678+0000 7f8c814e9640 1 --2- 172.21.15.40:0/2354840419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c7c0ce1c0 0x7f8c7c0c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:09.683 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.678+0000 7f8c814e9640 1 -- 172.21.15.40:0/2354840419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c74009490 con 0x7f8c7c15e750 2023-10-18T08:55:09.684 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.678+0000 7f8c814e9640 1 --2- 172.21.15.40:0/2354840419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7c15e750 0x7f8c7c0aef60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009980 tx=0x7f8c74003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:09.684 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.678+0000 7f8c797fa640 1 -- 172.21.15.40:0/2354840419 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74003e20 con 0x7f8c7c15e750 2023-10-18T08:55:09.684 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.678+0000 7f8c797fa640 1 -- 172.21.15.40:0/2354840419 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8c74034440 con 0x7f8c7c15e750 2023-10-18T08:55:09.684 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.678+0000 7f8c797fa640 1 -- 172.21.15.40:0/2354840419 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74034830 con 0x7f8c7c15e750 2023-10-18T08:55:09.684 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.678+0000 7f8c81cea640 1 -- 172.21.15.40:0/2354840419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7c1573a0 con 0x7f8c7c15e750 2023-10-18T08:55:09.685 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.679+0000 7f8c81cea640 1 -- 172.21.15.40:0/2354840419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7c157960 con 0x7f8c7c15e750 2023-10-18T08:55:09.685 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.680+0000 7f8c81cea640 1 -- 172.21.15.40:0/2354840419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c7c066cf0 con 0x7f8c7c15e750 2023-10-18T08:55:09.685 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.681+0000 7f8c797fa640 1 -- 172.21.15.40:0/2354840419 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8c7403b040 con 0x7f8c7c15e750 2023-10-18T08:55:09.685 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.682+0000 7f8c797fa640 1 --2- 172.21.15.40:0/2354840419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c5c06ae40 0x7f8c5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:09.685 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.682+0000 7f8c797fa640 1 -- 172.21.15.40:0/2354840419 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8c740ab770 con 0x7f8c7c15e750 2023-10-18T08:55:09.686 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.682+0000 7f8c80ce8640 1 --2- 172.21.15.40:0/2354840419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c5c06ae40 0x7f8c5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:09.686 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.684+0000 7f8c80ce8640 1 --2- 172.21.15.40:0/2354840419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c5c06ae40 0x7f8c5c06d300 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0016f0 tx=0x7f8c6c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:09.690 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.688+0000 7f8c797fa640 1 -- 172.21.15.40:0/2354840419 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c74076b70 con 0x7f8c7c15e750 2023-10-18T08:55:09.899 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.897+0000 7f8c81cea640 1 -- 172.21.15.40:0/2354840419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c7c0cd100 con 0x7f8c7c15e750 2023-10-18T08:55:09.901 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.899+0000 7f8c797fa640 1 -- 172.21.15.40:0/2354840419 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12862 (secure 0 0 0) 0x7f8c74041090 con 0x7f8c7c15e750 2023-10-18T08:55:09.902 INFO:teuthology.misc.health.smithi040.stdout: 2023-10-18T08:55:09.902 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":16,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:07.072557+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:09.905 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.902+0000 7f8c81cea640 1 -- 172.21.15.40:0/2354840419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c5c06ae40 msgr2=0x7f8c5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:09.905 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.902+0000 7f8c81cea640 1 --2- 172.21.15.40:0/2354840419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c5c06ae40 0x7f8c5c06d300 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0016f0 tx=0x7f8c6c009040 comp rx=0 tx=0).stop 2023-10-18T08:55:09.905 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.902+0000 7f8c81cea640 1 -- 172.21.15.40:0/2354840419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7c15e750 msgr2=0x7f8c7c0aef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:09.905 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.902+0000 7f8c81cea640 1 --2- 172.21.15.40:0/2354840419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7c15e750 0x7f8c7c0aef60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009980 tx=0x7f8c74003820 comp rx=0 tx=0).stop 2023-10-18T08:55:09.905 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.902+0000 7f8c81cea640 1 -- 172.21.15.40:0/2354840419 shutdown_connections 2023-10-18T08:55:09.906 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.902+0000 7f8c81cea640 1 --2- 172.21.15.40:0/2354840419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c5c06ae40 0x7f8c5c06d300 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:09.906 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.902+0000 7f8c81cea640 1 --2- 172.21.15.40:0/2354840419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c7c15e750 0x7f8c7c0aef60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:09.906 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.902+0000 7f8c81cea640 1 --2- 172.21.15.40:0/2354840419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c7c0cfe80 0x7f8c7c1f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:09.906 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.902+0000 7f8c81cea640 1 --2- 172.21.15.40:0/2354840419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c7c0ce1c0 0x7f8c7c0c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:09.906 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.902+0000 7f8c81cea640 1 -- 172.21.15.40:0/2354840419 >> 172.21.15.40:0/2354840419 conn(0x7f8c7c09a520 msgr2=0x7f8c7c0cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:09.907 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.902+0000 7f8c81cea640 1 -- 172.21.15.40:0/2354840419 shutdown_connections 2023-10-18T08:55:09.907 INFO:teuthology.misc.health.smithi040.stderr:2023-10-18T08:55:09.903+0000 7f8c81cea640 1 -- 172.21.15.40:0/2354840419 wait complete. 2023-10-18T08:55:09.920 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-18T08:55:09.920 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-18T08:55:09.921 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T08:55:09.921 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T08:55:09.929 INFO:tasks.ceph.mds.a:Started 2023-10-18T08:55:09.929 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T08:55:09.930 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T08:55:09.938 INFO:tasks.ceph.mds.c:Started 2023-10-18T08:55:09.938 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T08:55:09.939 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T08:55:09.948 INFO:tasks.ceph.mds.b:Started 2023-10-18T08:55:09.948 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T08:55:09.948 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T08:55:09.957 INFO:tasks.ceph.mds.d:Started 2023-10-18T08:55:09.958 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-18T08:55:09.958 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:10.090 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-10-18T08:55:10.107 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T08:55:10.109 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-18T08:55:10.123 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-18T08:55:10.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.155+0000 7f5e9f98e640 1 -- 172.21.15.40:0/104494862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e9815ba90 msgr2=0x7f5e980cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:10.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.155+0000 7f5e9f98e640 1 --2- 172.21.15.40:0/104494862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e9815ba90 0x7f5e980cc3b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5e94005fd0 tx=0x7f5e9402be20 comp rx=0 tx=0).stop 2023-10-18T08:55:10.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.155+0000 7f5e9f98e640 1 -- 172.21.15.40:0/104494862 shutdown_connections 2023-10-18T08:55:10.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.155+0000 7f5e9f98e640 1 --2- 172.21.15.40:0/104494862 >> v2:172.21.15.144:3300/0 conn(0x7f5e980b7000 0x7f5e980cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:10.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.155+0000 7f5e9f98e640 1 --2- 172.21.15.40:0/104494862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e9815ba90 0x7f5e980cc3b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:10.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.155+0000 7f5e9f98e640 1 -- 172.21.15.40:0/104494862 >> 172.21.15.40:0/104494862 conn(0x7f5e9806a780 msgr2=0x7f5e9806ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:10.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.155+0000 7f5e9f98e640 1 -- 172.21.15.40:0/104494862 shutdown_connections 2023-10-18T08:55:10.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.155+0000 7f5e9f98e640 1 -- 172.21.15.40:0/104494862 wait complete. 2023-10-18T08:55:10.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.157+0000 7f5e9f98e640 1 Processor -- start 2023-10-18T08:55:10.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.157+0000 7f5e9f98e640 1 -- start start 2023-10-18T08:55:10.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.158+0000 7f5e9f98e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e980b7000 0x7f5e98155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:10.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.158+0000 7f5e9f98e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e980ce430 0x7f5e981f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:10.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.158+0000 7f5e9f98e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e9815e500 0x7f5e981f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:10.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.158+0000 7f5e9f98e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e980ca430 con 0x7f5e980ce430 2023-10-18T08:55:10.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.158+0000 7f5e9f98e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e98159510 con 0x7f5e9815e500 2023-10-18T08:55:10.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.158+0000 7f5e9f98e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e980c9b70 con 0x7f5e980b7000 2023-10-18T08:55:10.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.159+0000 7f5e9cf29640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e980ce430 0x7f5e981f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:10.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.159+0000 7f5e9cf29640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e980ce430 0x7f5e981f31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57434/0 (socket says 172.21.15.40:57434) 2023-10-18T08:55:10.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.159+0000 7f5e9cf29640 1 -- 172.21.15.40:0/2391502675 learned_addr learned my addr 172.21.15.40:0/2391502675 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:10.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.159+0000 7f5e9d72a640 1 --2- 172.21.15.40:0/2391502675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e980b7000 0x7f5e98155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:10.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.159+0000 7f5e9df2b640 1 --2- 172.21.15.40:0/2391502675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e9815e500 0x7f5e981f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:10.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.159+0000 7f5e9cf29640 1 -- 172.21.15.40:0/2391502675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e980b7000 msgr2=0x7f5e98155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:10.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.159+0000 7f5e9cf29640 1 --2- 172.21.15.40:0/2391502675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e980b7000 0x7f5e98155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:10.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.159+0000 7f5e9cf29640 1 -- 172.21.15.40:0/2391502675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e9815e500 msgr2=0x7f5e981f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:10.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.159+0000 7f5e9cf29640 1 --2- 172.21.15.40:0/2391502675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e9815e500 0x7f5e981f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:10.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.159+0000 7f5e9cf29640 1 -- 172.21.15.40:0/2391502675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e88009490 con 0x7f5e980ce430 2023-10-18T08:55:10.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.159+0000 7f5e9d72a640 1 --2- 172.21.15.40:0/2391502675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e980b7000 0x7f5e98155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:55:10.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.159+0000 7f5e9cf29640 1 --2- 172.21.15.40:0/2391502675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e980ce430 0x7f5e981f31d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800ea60 tx=0x7f5e8800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:10.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.160+0000 7f5e8dffb640 1 -- 172.21.15.40:0/2391502675 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8800cde0 con 0x7f5e980ce430 2023-10-18T08:55:10.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.160+0000 7f5e8dffb640 1 -- 172.21.15.40:0/2391502675 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5e88004020 con 0x7f5e980ce430 2023-10-18T08:55:10.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.160+0000 7f5e8dffb640 1 -- 172.21.15.40:0/2391502675 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e88004410 con 0x7f5e980ce430 2023-10-18T08:55:10.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.160+0000 7f5e9f98e640 1 -- 172.21.15.40:0/2391502675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e94009560 con 0x7f5e980ce430 2023-10-18T08:55:10.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.160+0000 7f5e9f98e640 1 -- 172.21.15.40:0/2391502675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e981f2fa0 con 0x7f5e980ce430 2023-10-18T08:55:10.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.161+0000 7f5e9f98e640 1 -- 172.21.15.40:0/2391502675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e980ca5a0 con 0x7f5e980ce430 2023-10-18T08:55:10.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.162+0000 7f5e8dffb640 1 -- 172.21.15.40:0/2391502675 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5e880035f0 con 0x7f5e980ce430 2023-10-18T08:55:10.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.163+0000 7f5e8dffb640 1 --2- 172.21.15.40:0/2391502675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5e7c06ae70 0x7f5e7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:10.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.164+0000 7f5e8dffb640 1 -- 172.21.15.40:0/2391502675 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5e88014070 con 0x7f5e980ce430 2023-10-18T08:55:10.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.164+0000 7f5e9d72a640 1 --2- 172.21.15.40:0/2391502675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5e7c06ae70 0x7f5e7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:10.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.165+0000 7f5e9d72a640 1 --2- 172.21.15.40:0/2391502675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5e7c06ae70 0x7f5e7c06d330 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5e94035000 tx=0x7f5e94004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:10.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.170+0000 7f5e8dffb640 1 -- 172.21.15.40:0/2391502675 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e880552f0 con 0x7f5e980ce430 2023-10-18T08:55:10.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.367+0000 7f5e9f98e640 1 -- 172.21.15.40:0/2391502675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e98066fd0 con 0x7f5e980ce430 2023-10-18T08:55:10.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.369+0000 7f5e8dffb640 1 -- 172.21.15.40:0/2391502675 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12862 (secure 0 0 0) 0x7f5e88058910 con 0x7f5e980ce430 2023-10-18T08:55:10.372 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:10.372 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:07.072557+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.371+0000 7f5e9f98e640 1 -- 172.21.15.40:0/2391502675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5e7c06ae70 msgr2=0x7f5e7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.371+0000 7f5e9f98e640 1 --2- 172.21.15.40:0/2391502675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5e7c06ae70 0x7f5e7c06d330 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5e94035000 tx=0x7f5e94004730 comp rx=0 tx=0).stop 2023-10-18T08:55:10.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.371+0000 7f5e9f98e640 1 -- 172.21.15.40:0/2391502675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e980ce430 msgr2=0x7f5e981f31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:10.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.371+0000 7f5e9f98e640 1 --2- 172.21.15.40:0/2391502675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e980ce430 0x7f5e981f31d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800ea60 tx=0x7f5e8800ef20 comp rx=0 tx=0).stop 2023-10-18T08:55:10.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.372+0000 7f5e9f98e640 1 -- 172.21.15.40:0/2391502675 shutdown_connections 2023-10-18T08:55:10.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.372+0000 7f5e9f98e640 1 --2- 172.21.15.40:0/2391502675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5e7c06ae70 0x7f5e7c06d330 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:10.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.372+0000 7f5e9f98e640 1 --2- 172.21.15.40:0/2391502675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e9815e500 0x7f5e981f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:10.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.372+0000 7f5e9f98e640 1 --2- 172.21.15.40:0/2391502675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e980ce430 0x7f5e981f31d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:10.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.372+0000 7f5e9f98e640 1 --2- 172.21.15.40:0/2391502675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e980b7000 0x7f5e98155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:10.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.372+0000 7f5e9f98e640 1 -- 172.21.15.40:0/2391502675 >> 172.21.15.40:0/2391502675 conn(0x7f5e9806a780 msgr2=0x7f5e980b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:10.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.372+0000 7f5e9f98e640 1 -- 172.21.15.40:0/2391502675 shutdown_connections 2023-10-18T08:55:10.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.372+0000 7f5e9f98e640 1 -- 172.21.15.40:0/2391502675 wait complete. 2023-10-18T08:55:10.392 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:55:10.393 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:55:10.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.626+0000 7f4b5b580640 1 -- 172.21.15.40:0/3973554988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b400039a0 msgr2=0x7f4b40003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:10.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.626+0000 7f4b5b580640 1 --2- 172.21.15.40:0/3973554988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b400039a0 0x7f4b40003dc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009b20 tx=0x7f4b400057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:10.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.626+0000 7f4b5ad7f640 1 -- 172.21.15.40:0/3973554988 reap_dead start 2023-10-18T08:55:10.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.626+0000 7f4b5b580640 1 -- 172.21.15.40:0/3973554988 shutdown_connections 2023-10-18T08:55:10.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.626+0000 7f4b5b580640 1 -- 172.21.15.40:0/3973554988 >> 172.21.15.40:0/3973554988 conn(0x7f4b540cefc0 msgr2=0x7f4b540cf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:10.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.626+0000 7f4b5b580640 1 -- 172.21.15.40:0/3973554988 shutdown_connections 2023-10-18T08:55:10.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.626+0000 7f4b5b580640 1 -- 172.21.15.40:0/3973554988 wait complete. 2023-10-18T08:55:10.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.628+0000 7f4b5b580640 1 Processor -- start 2023-10-18T08:55:10.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.628+0000 7f4b5b580640 1 -- start start 2023-10-18T08:55:10.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.629+0000 7f4b5b580640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b400039a0 0x7f4b5414f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:10.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.629+0000 7f4b5b580640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b540b3ea0 0x7f4b540b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:10.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.629+0000 7f4b5b580640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b540b1b70 0x7f4b540b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:10.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.629+0000 7f4b5b580640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540b4860 con 0x7f4b400039a0 2023-10-18T08:55:10.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.629+0000 7f4b5b580640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540b2530 con 0x7f4b540b1b70 2023-10-18T08:55:10.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.629+0000 7f4b5b580640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b541569f0 con 0x7f4b540b3ea0 2023-10-18T08:55:10.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.629+0000 7f4b5a57e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b400039a0 0x7f4b5414f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:10.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.630+0000 7f4b5a57e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b400039a0 0x7f4b5414f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57448/0 (socket says 172.21.15.40:57448) 2023-10-18T08:55:10.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.630+0000 7f4b5a57e640 1 -- 172.21.15.40:0/1933073570 learned_addr learned my addr 172.21.15.40:0/1933073570 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:10.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.630+0000 7f4b5ad7f640 1 --2- 172.21.15.40:0/1933073570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b540b1b70 0x7f4b540b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:10.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.630+0000 7f4b59d7d640 1 --2- 172.21.15.40:0/1933073570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b540b3ea0 0x7f4b540b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:10.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.630+0000 7f4b5a57e640 1 -- 172.21.15.40:0/1933073570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b540b3ea0 msgr2=0x7f4b540b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:10.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.630+0000 7f4b5a57e640 1 --2- 172.21.15.40:0/1933073570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b540b3ea0 0x7f4b540b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:10.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.630+0000 7f4b5a57e640 1 -- 172.21.15.40:0/1933073570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b540b1b70 msgr2=0x7f4b540b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:10.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.630+0000 7f4b5a57e640 1 --2- 172.21.15.40:0/1933073570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b540b1b70 0x7f4b540b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:10.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.630+0000 7f4b5a57e640 1 -- 172.21.15.40:0/1933073570 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5414fba0 con 0x7f4b400039a0 2023-10-18T08:55:10.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.630+0000 7f4b59d7d640 1 --2- 172.21.15.40:0/1933073570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b540b3ea0 0x7f4b540b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:10.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.630+0000 7f4b5ad7f640 1 --2- 172.21.15.40:0/1933073570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b540b1b70 0x7f4b540b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:10.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.630+0000 7f4b5a57e640 1 --2- 172.21.15.40:0/1933073570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b400039a0 0x7f4b5414f660 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009c40 tx=0x7f4b50033c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:10.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.631+0000 7f4b4affd640 1 -- 172.21.15.40:0/1933073570 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50004220 con 0x7f4b400039a0 2023-10-18T08:55:10.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.631+0000 7f4b4affd640 1 -- 172.21.15.40:0/1933073570 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4b50004380 con 0x7f4b400039a0 2023-10-18T08:55:10.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.631+0000 7f4b4affd640 1 -- 172.21.15.40:0/1933073570 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5003d940 con 0x7f4b400039a0 2023-10-18T08:55:10.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.631+0000 7f4b5b580640 1 -- 172.21.15.40:0/1933073570 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b540b2750 con 0x7f4b400039a0 2023-10-18T08:55:10.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.631+0000 7f4b5b580640 1 -- 172.21.15.40:0/1933073570 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b540b2d10 con 0x7f4b400039a0 2023-10-18T08:55:10.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.632+0000 7f4b5b580640 1 -- 172.21.15.40:0/1933073570 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b54073570 con 0x7f4b400039a0 2023-10-18T08:55:10.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.633+0000 7f4b4affd640 1 -- 172.21.15.40:0/1933073570 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4b5003daa0 con 0x7f4b400039a0 2023-10-18T08:55:10.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.634+0000 7f4b4affd640 1 --2- 172.21.15.40:0/1933073570 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b3806ae70 0x7f4b3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:10.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.634+0000 7f4b4affd640 1 -- 172.21.15.40:0/1933073570 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4b50033e40 con 0x7f4b400039a0 2023-10-18T08:55:10.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.634+0000 7f4b59d7d640 1 --2- 172.21.15.40:0/1933073570 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b3806ae70 0x7f4b3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:10.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.635+0000 7f4b59d7d640 1 --2- 172.21.15.40:0/1933073570 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b3806ae70 0x7f4b3806d330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4b540ac220 tx=0x7f4b44006df0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:10.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.638+0000 7f4b4affd640 1 -- 172.21.15.40:0/1933073570 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4b50077250 con 0x7f4b400039a0 2023-10-18T08:55:10.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.829+0000 7f4b5b580640 1 -- 172.21.15.40:0/1933073570 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b540ac250 con 0x7f4b400039a0 2023-10-18T08:55:10.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.830+0000 7f4b4affd640 1 -- 172.21.15.40:0/1933073570 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b5004a020 con 0x7f4b400039a0 2023-10-18T08:55:10.832 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:55:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.833+0000 7f4b5b580640 1 -- 172.21.15.40:0/1933073570 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b3806ae70 msgr2=0x7f4b3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.833+0000 7f4b5b580640 1 --2- 172.21.15.40:0/1933073570 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b3806ae70 0x7f4b3806d330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4b540ac220 tx=0x7f4b44006df0 comp rx=0 tx=0).stop 2023-10-18T08:55:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.833+0000 7f4b5b580640 1 -- 172.21.15.40:0/1933073570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b400039a0 msgr2=0x7f4b5414f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.833+0000 7f4b5b580640 1 --2- 172.21.15.40:0/1933073570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b400039a0 0x7f4b5414f660 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009c40 tx=0x7f4b50033c50 comp rx=0 tx=0).stop 2023-10-18T08:55:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.833+0000 7f4b5b580640 1 -- 172.21.15.40:0/1933073570 shutdown_connections 2023-10-18T08:55:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.833+0000 7f4b5b580640 1 --2- 172.21.15.40:0/1933073570 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b3806ae70 0x7f4b3806d330 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.833+0000 7f4b5b580640 1 --2- 172.21.15.40:0/1933073570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b540b1b70 0x7f4b540b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.833+0000 7f4b5b580640 1 --2- 172.21.15.40:0/1933073570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b540b3ea0 0x7f4b540b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.833+0000 7f4b5b580640 1 --2- 172.21.15.40:0/1933073570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b400039a0 0x7f4b5414f660 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.833+0000 7f4b5b580640 1 -- 172.21.15.40:0/1933073570 >> 172.21.15.40:0/1933073570 conn(0x7f4b540cefc0 msgr2=0x7f4b540b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.833+0000 7f4b5b580640 1 -- 172.21.15.40:0/1933073570 shutdown_connections 2023-10-18T08:55:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:10.833+0000 7f4b5b580640 1 -- 172.21.15.40:0/1933073570 wait complete. 2023-10-18T08:55:10.855 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:55:11.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.085+0000 7f3a16597640 1 -- 172.21.15.40:0/2022346511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a1013c470 msgr2=0x7f3a1013f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:11.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.085+0000 7f3a16597640 1 --2- 172.21.15.40:0/2022346511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a1013c470 0x7f3a1013f5a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3a00005fd0 tx=0x7f3a0002c2e0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.086+0000 7f3a16597640 1 -- 172.21.15.40:0/2022346511 shutdown_connections 2023-10-18T08:55:11.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.086+0000 7f3a16597640 1 --2- 172.21.15.40:0/2022346511 >> v2:172.21.15.144:3300/0 conn(0x7f3a1015cf30 0x7f3a1015d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.086+0000 7f3a16597640 1 --2- 172.21.15.40:0/2022346511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a1013c470 0x7f3a1013f5a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.086+0000 7f3a16597640 1 -- 172.21.15.40:0/2022346511 >> 172.21.15.40:0/2022346511 conn(0x7f3a1006cee0 msgr2=0x7f3a10069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:11.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.086+0000 7f3a16597640 1 -- 172.21.15.40:0/2022346511 shutdown_connections 2023-10-18T08:55:11.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.086+0000 7f3a16597640 1 -- 172.21.15.40:0/2022346511 wait complete. 2023-10-18T08:55:11.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.087+0000 7f3a16597640 1 Processor -- start 2023-10-18T08:55:11.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.088+0000 7f3a16597640 1 -- start start 2023-10-18T08:55:11.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.088+0000 7f3a16597640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a100c9b00 0x7f3a10156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:11.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.089+0000 7f3a16597640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1013c470 0x7f3a100707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:11.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.089+0000 7f3a16597640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a1015cf30 0x7f3a10070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:11.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.089+0000 7f3a16597640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a101f2740 con 0x7f3a100c9b00 2023-10-18T08:55:11.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.089+0000 7f3a16597640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a101f2160 con 0x7f3a1013c470 2023-10-18T08:55:11.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.089+0000 7f3a16597640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a101f2440 con 0x7f3a1015cf30 2023-10-18T08:55:11.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.089+0000 7f3a0ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a100c9b00 0x7f3a10156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:11.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.089+0000 7f3a0ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a100c9b00 0x7f3a10156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57450/0 (socket says 172.21.15.40:57450) 2023-10-18T08:55:11.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.089+0000 7f3a0ffff640 1 -- 172.21.15.40:0/1026718565 learned_addr learned my addr 172.21.15.40:0/1026718565 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:11.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.089+0000 7f3a0f7fe640 1 --2- 172.21.15.40:0/1026718565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1013c470 0x7f3a100707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:11.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.089+0000 7f3a14b34640 1 --2- 172.21.15.40:0/1026718565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a1015cf30 0x7f3a10070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:11.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.089+0000 7f3a0ffff640 1 -- 172.21.15.40:0/1026718565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a1015cf30 msgr2=0x7f3a10070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:11.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.089+0000 7f3a0ffff640 1 --2- 172.21.15.40:0/1026718565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a1015cf30 0x7f3a10070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.089+0000 7f3a0ffff640 1 -- 172.21.15.40:0/1026718565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1013c470 msgr2=0x7f3a100707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.089+0000 7f3a0ffff640 1 --2- 172.21.15.40:0/1026718565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1013c470 0x7f3a100707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.089+0000 7f3a0ffff640 1 -- 172.21.15.40:0/1026718565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a00009900 con 0x7f3a100c9b00 2023-10-18T08:55:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.090+0000 7f3a0f7fe640 1 --2- 172.21.15.40:0/1026718565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1013c470 0x7f3a100707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:55:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.090+0000 7f3a0ffff640 1 --2- 172.21.15.40:0/1026718565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a100c9b00 0x7f3a10156d80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3a00006010 tx=0x7f3a00035b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.090+0000 7f3a0cff9640 1 -- 172.21.15.40:0/1026718565 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00004240 con 0x7f3a100c9b00 2023-10-18T08:55:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.090+0000 7f3a0cff9640 1 -- 172.21.15.40:0/1026718565 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3a00004490 con 0x7f3a100c9b00 2023-10-18T08:55:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.090+0000 7f3a0cff9640 1 -- 172.21.15.40:0/1026718565 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a00004840 con 0x7f3a100c9b00 2023-10-18T08:55:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.090+0000 7f3a16597640 1 -- 172.21.15.40:0/1026718565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a100ca4a0 con 0x7f3a100c9b00 2023-10-18T08:55:11.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.090+0000 7f3a16597640 1 -- 172.21.15.40:0/1026718565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1013cb30 con 0x7f3a100c9b00 2023-10-18T08:55:11.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.092+0000 7f3a16597640 1 -- 172.21.15.40:0/1026718565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1013f5a0 con 0x7f3a100c9b00 2023-10-18T08:55:11.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.093+0000 7f3a0cff9640 1 -- 172.21.15.40:0/1026718565 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3a0003b040 con 0x7f3a100c9b00 2023-10-18T08:55:11.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.093+0000 7f3a0cff9640 1 --2- 172.21.15.40:0/1026718565 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39f006ae40 0x7f39f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:11.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.094+0000 7f3a0cff9640 1 -- 172.21.15.40:0/1026718565 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3a000abfa0 con 0x7f3a100c9b00 2023-10-18T08:55:11.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.094+0000 7f3a0f7fe640 1 --2- 172.21.15.40:0/1026718565 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39f006ae40 0x7f39f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:11.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.095+0000 7f3a0f7fe640 1 --2- 172.21.15.40:0/1026718565 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39f006ae40 0x7f39f006d300 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3a1015c2e0 tx=0x7f39f8005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:11.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.098+0000 7f3a0cff9640 1 -- 172.21.15.40:0/1026718565 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3a00077420 con 0x7f3a100c9b00 2023-10-18T08:55:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.331+0000 7f3a16597640 1 -- 172.21.15.40:0/1026718565 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a10066fd0 con 0x7f3a100c9b00 2023-10-18T08:55:11.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.332+0000 7f3a0cff9640 1 -- 172.21.15.40:0/1026718565 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7f3a00043e90 con 0x7f3a100c9b00 2023-10-18T08:55:11.335 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:11.335 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1962150323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1962150323},{"type":"v1","addr":"172.21.15.144:6837","nonce":1962150323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4329,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2589720838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2589720838},{"type":"v1","addr":"172.21.15.40:6833","nonce":2589720838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3031491371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3031491371},{"type":"v1","addr":"172.21.15.40:6835","nonce":3031491371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4384,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/682820172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":682820172},{"type":"v1","addr":"172.21.15.144:6835","nonce":682820172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-18T08:55:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 3 2023-10-18T08:55:11.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.335+0000 7f3a16597640 1 -- 172.21.15.40:0/1026718565 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39f006ae40 msgr2=0x7f39f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.335+0000 7f3a16597640 1 --2- 172.21.15.40:0/1026718565 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39f006ae40 0x7f39f006d300 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3a1015c2e0 tx=0x7f39f8005f90 comp rx=0 tx=0).stop 2023-10-18T08:55:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.335+0000 7f3a16597640 1 -- 172.21.15.40:0/1026718565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a100c9b00 msgr2=0x7f3a10156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.335+0000 7f3a16597640 1 --2- 172.21.15.40:0/1026718565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a100c9b00 0x7f3a10156d80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3a00006010 tx=0x7f3a00035b50 comp rx=0 tx=0).stop 2023-10-18T08:55:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.335+0000 7f3a16597640 1 -- 172.21.15.40:0/1026718565 shutdown_connections 2023-10-18T08:55:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.335+0000 7f3a16597640 1 --2- 172.21.15.40:0/1026718565 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39f006ae40 0x7f39f006d300 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.335+0000 7f3a16597640 1 --2- 172.21.15.40:0/1026718565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a1015cf30 0x7f3a10070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.335+0000 7f3a16597640 1 --2- 172.21.15.40:0/1026718565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a1013c470 0x7f3a100707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.335+0000 7f3a16597640 1 --2- 172.21.15.40:0/1026718565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a100c9b00 0x7f3a10156d80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.335+0000 7f3a16597640 1 -- 172.21.15.40:0/1026718565 >> 172.21.15.40:0/1026718565 conn(0x7f3a1006cee0 msgr2=0x7f3a100abd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:11.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.335+0000 7f3a16597640 1 -- 172.21.15.40:0/1026718565 shutdown_connections 2023-10-18T08:55:11.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.336+0000 7f3a16597640 1 -- 172.21.15.40:0/1026718565 wait complete. 2023-10-18T08:55:11.358 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:11.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.583+0000 7f2fe59c4640 1 -- 172.21.15.40:0/2274598620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe015b380 msgr2=0x7f2fe00b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:11.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.583+0000 7f2fe59c4640 1 --2- 172.21.15.40:0/2274598620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe015b380 0x7f2fe00b1120 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0005fd0 tx=0x7f2fd002c050 comp rx=0 tx=0).stop 2023-10-18T08:55:11.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.583+0000 7f2fe59c4640 1 -- 172.21.15.40:0/2274598620 shutdown_connections 2023-10-18T08:55:11.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.583+0000 7f2fe59c4640 1 --2- 172.21.15.40:0/2274598620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe015b380 0x7f2fe00b1120 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.583+0000 7f2fe59c4640 1 -- 172.21.15.40:0/2274598620 >> 172.21.15.40:0/2274598620 conn(0x7f2fe008acb0 msgr2=0x7f2fe009a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.583+0000 7f2fe59c4640 1 -- 172.21.15.40:0/2274598620 shutdown_connections 2023-10-18T08:55:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.583+0000 7f2fe59c4640 1 -- 172.21.15.40:0/2274598620 wait complete. 2023-10-18T08:55:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.585+0000 7f2fe59c4640 1 Processor -- start 2023-10-18T08:55:11.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.585+0000 7f2fe59c4640 1 -- start start 2023-10-18T08:55:11.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.586+0000 7f2fe59c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fe00d1580 0x7f2fe014dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:11.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.586+0000 7f2fe59c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe015b380 0x7f2fe014e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.586+0000 7f2fe59c4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fe015b760 0x7f2fe015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.586+0000 7f2fe59c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe014f970 con 0x7f2fe015b380 2023-10-18T08:55:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.586+0000 7f2fe59c4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe014f0d0 con 0x7f2fe00d1580 2023-10-18T08:55:11.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.586+0000 7f2fe59c4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe014f3b0 con 0x7f2fe015b760 2023-10-18T08:55:11.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.586+0000 7f2fdffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe015b380 0x7f2fe014e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.586+0000 7f2fdffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe015b380 0x7f2fe014e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57468/0 (socket says 172.21.15.40:57468) 2023-10-18T08:55:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.587+0000 7f2fdffff640 1 -- 172.21.15.40:0/1804403160 learned_addr learned my addr 172.21.15.40:0/1804403160 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.587+0000 7f2fe49c2640 1 --2- 172.21.15.40:0/1804403160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fe00d1580 0x7f2fe014dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:11.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.587+0000 7f2fe51c3640 1 --2- 172.21.15.40:0/1804403160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fe015b760 0x7f2fe015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:11.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.587+0000 7f2fdffff640 1 -- 172.21.15.40:0/1804403160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fe015b760 msgr2=0x7f2fe015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:11.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.587+0000 7f2fdffff640 1 --2- 172.21.15.40:0/1804403160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fe015b760 0x7f2fe015dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.587+0000 7f2fdffff640 1 -- 172.21.15.40:0/1804403160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fe00d1580 msgr2=0x7f2fe014dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.587+0000 7f2fdffff640 1 --2- 172.21.15.40:0/1804403160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fe00d1580 0x7f2fe014dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.587+0000 7f2fdffff640 1 -- 172.21.15.40:0/1804403160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd0009840 con 0x7f2fe015b380 2023-10-18T08:55:11.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.587+0000 7f2fe49c2640 1 --2- 172.21.15.40:0/1804403160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fe00d1580 0x7f2fe014dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:55:11.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.587+0000 7f2fdffff640 1 --2- 172.21.15.40:0/1804403160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe015b380 0x7f2fe014e610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40011e0 tx=0x7f2fd400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:11.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.587+0000 7f2fdd7fa640 1 -- 172.21.15.40:0/1804403160 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400a7e0 con 0x7f2fe015b380 2023-10-18T08:55:11.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.588+0000 7f2fdd7fa640 1 -- 172.21.15.40:0/1804403160 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2fd400aa50 con 0x7f2fe015b380 2023-10-18T08:55:11.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.588+0000 7f2fdd7fa640 1 -- 172.21.15.40:0/1804403160 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400ae60 con 0x7f2fe015b380 2023-10-18T08:55:11.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.588+0000 7f2fe59c4640 1 -- 172.21.15.40:0/1804403160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe014f690 con 0x7f2fe015b380 2023-10-18T08:55:11.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.588+0000 7f2fe59c4640 1 -- 172.21.15.40:0/1804403160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe014fda0 con 0x7f2fe015b380 2023-10-18T08:55:11.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.589+0000 7f2fe59c4640 1 -- 172.21.15.40:0/1804403160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe015bee0 con 0x7f2fe015b380 2023-10-18T08:55:11.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.590+0000 7f2fdd7fa640 1 -- 172.21.15.40:0/1804403160 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2fd4012420 con 0x7f2fe015b380 2023-10-18T08:55:11.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.592+0000 7f2fdd7fa640 1 --2- 172.21.15.40:0/1804403160 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2fcc06ae70 0x7f2fcc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:11.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.592+0000 7f2fdd7fa640 1 -- 172.21.15.40:0/1804403160 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2fd4089940 con 0x7f2fe015b380 2023-10-18T08:55:11.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.592+0000 7f2fe49c2640 1 --2- 172.21.15.40:0/1804403160 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2fcc06ae70 0x7f2fcc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:11.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.593+0000 7f2fe49c2640 1 --2- 172.21.15.40:0/1804403160 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2fcc06ae70 0x7f2fcc06d330 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0035000 tx=0x7f2fd0004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:11.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.596+0000 7f2fdd7fa640 1 -- 172.21.15.40:0/1804403160 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2fe015bee0 con 0x7f2fe015b380 2023-10-18T08:55:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.788+0000 7f2fe59c4640 1 -- 172.21.15.40:0/1804403160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fe015bee0 con 0x7f2fe015b380 2023-10-18T08:55:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.791+0000 7f2fdd7fa640 1 -- 172.21.15.40:0/1804403160 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12862 (secure 0 0 0) 0x7f2fe015bee0 con 0x7f2fe015b380 2023-10-18T08:55:11.793 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:11.793 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:07.072557+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:11.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.793+0000 7f2fe59c4640 1 -- 172.21.15.40:0/1804403160 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2fcc06ae70 msgr2=0x7f2fcc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:11.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.793+0000 7f2fe59c4640 1 --2- 172.21.15.40:0/1804403160 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2fcc06ae70 0x7f2fcc06d330 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0035000 tx=0x7f2fd0004ab0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.793+0000 7f2fe59c4640 1 -- 172.21.15.40:0/1804403160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe015b380 msgr2=0x7f2fe014e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:11.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.793+0000 7f2fe59c4640 1 --2- 172.21.15.40:0/1804403160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe015b380 0x7f2fe014e610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40011e0 tx=0x7f2fd400cbe0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.793+0000 7f2fe59c4640 1 -- 172.21.15.40:0/1804403160 shutdown_connections 2023-10-18T08:55:11.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.793+0000 7f2fe59c4640 1 --2- 172.21.15.40:0/1804403160 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2fcc06ae70 0x7f2fcc06d330 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.793+0000 7f2fe59c4640 1 --2- 172.21.15.40:0/1804403160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fe015b760 0x7f2fe015dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.793+0000 7f2fe59c4640 1 --2- 172.21.15.40:0/1804403160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fe015b380 0x7f2fe014e610 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.793+0000 7f2fe59c4640 1 --2- 172.21.15.40:0/1804403160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fe00d1580 0x7f2fe014dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:11.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.793+0000 7f2fe59c4640 1 -- 172.21.15.40:0/1804403160 >> 172.21.15.40:0/1804403160 conn(0x7f2fe008acb0 msgr2=0x7f2fe006bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.793+0000 7f2fe59c4640 1 -- 172.21.15.40:0/1804403160 shutdown_connections 2023-10-18T08:55:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:11.794+0000 7f2fe59c4640 1 -- 172.21.15.40:0/1804403160 wait complete. 2023-10-18T08:55:11.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:55:11.817 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:55:12.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.044+0000 7fae7087b640 1 -- 172.21.15.40:0/385803921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c15b8b0 msgr2=0x7fae6c0d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:12.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.045+0000 7fae7087b640 1 --2- 172.21.15.40:0/385803921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c15b8b0 0x7fae6c0d1250 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0d22b0 tx=0x7fae5402be20 comp rx=0 tx=0).stop 2023-10-18T08:55:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.045+0000 7fae7087b640 1 -- 172.21.15.40:0/385803921 shutdown_connections 2023-10-18T08:55:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.045+0000 7fae7087b640 1 --2- 172.21.15.40:0/385803921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae6c0d1790 0x7fae6c0cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.045+0000 7fae7087b640 1 --2- 172.21.15.40:0/385803921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c15b8b0 0x7fae6c0d1250 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.045+0000 7fae7087b640 1 --2- 172.21.15.40:0/385803921 >> v2:172.21.15.144:3300/0 conn(0x7fae6c15b4d0 0x7fae6c0addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.045+0000 7fae7087b640 1 -- 172.21.15.40:0/385803921 >> 172.21.15.40:0/385803921 conn(0x7fae6c08ad30 msgr2=0x7fae6c09a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.045+0000 7fae7087b640 1 -- 172.21.15.40:0/385803921 shutdown_connections 2023-10-18T08:55:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.045+0000 7fae7087b640 1 -- 172.21.15.40:0/385803921 wait complete. 2023-10-18T08:55:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.047+0000 7fae7087b640 1 Processor -- start 2023-10-18T08:55:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.047+0000 7fae7087b640 1 -- start start 2023-10-18T08:55:12.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.048+0000 7fae7087b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d1790 0x7fae6c15c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:12.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.048+0000 7fae7087b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae6c15b4d0 0x7fae6c15dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:12.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.048+0000 7fae7087b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae6c15b8b0 0x7fae6c14f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:12.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.048+0000 7fae7087b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c0c8930 con 0x7fae6c0d1790 2023-10-18T08:55:12.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.048+0000 7fae7087b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c15ccb0 con 0x7fae6c15b4d0 2023-10-18T08:55:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.048+0000 7fae7087b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c15cf90 con 0x7fae6c15b8b0 2023-10-18T08:55:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.048+0000 7fae6ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d1790 0x7fae6c15c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.048+0000 7fae6ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d1790 0x7fae6c15c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57494/0 (socket says 172.21.15.40:57494) 2023-10-18T08:55:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.048+0000 7fae6ad9d640 1 -- 172.21.15.40:0/1020610776 learned_addr learned my addr 172.21.15.40:0/1020610776 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:12.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.048+0000 7fae63fff640 1 --2- 172.21.15.40:0/1020610776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae6c15b4d0 0x7fae6c15dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:12.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.048+0000 7fae6b59e640 1 --2- 172.21.15.40:0/1020610776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae6c15b8b0 0x7fae6c14f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:12.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.048+0000 7fae6ad9d640 1 -- 172.21.15.40:0/1020610776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae6c15b8b0 msgr2=0x7fae6c14f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:12.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.048+0000 7fae6ad9d640 1 --2- 172.21.15.40:0/1020610776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae6c15b8b0 0x7fae6c14f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.048+0000 7fae6ad9d640 1 -- 172.21.15.40:0/1020610776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae6c15b4d0 msgr2=0x7fae6c15dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:12.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.048+0000 7fae6ad9d640 1 --2- 172.21.15.40:0/1020610776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae6c15b4d0 0x7fae6c15dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.048+0000 7fae6ad9d640 1 -- 172.21.15.40:0/1020610776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae54009560 con 0x7fae6c0d1790 2023-10-18T08:55:12.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.049+0000 7fae6ad9d640 1 --2- 172.21.15.40:0/1020610776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d1790 0x7fae6c15c6d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00d950 tx=0x7fae5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:12.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.049+0000 7fae637fe640 1 -- 172.21.15.40:0/1020610776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c009c30 con 0x7fae6c0d1790 2023-10-18T08:55:12.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.049+0000 7fae637fe640 1 -- 172.21.15.40:0/1020610776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fae5c009e80 con 0x7fae6c0d1790 2023-10-18T08:55:12.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.049+0000 7fae7087b640 1 -- 172.21.15.40:0/1020610776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6c15d270 con 0x7fae6c0d1790 2023-10-18T08:55:12.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.049+0000 7fae637fe640 1 -- 172.21.15.40:0/1020610776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c0106c0 con 0x7fae6c0d1790 2023-10-18T08:55:12.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.049+0000 7fae7087b640 1 -- 172.21.15.40:0/1020610776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6c15e170 con 0x7fae6c0d1790 2023-10-18T08:55:12.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.050+0000 7fae7087b640 1 -- 172.21.15.40:0/1020610776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6c071da0 con 0x7fae6c0d1790 2023-10-18T08:55:12.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.051+0000 7fae637fe640 1 -- 172.21.15.40:0/1020610776 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fae5c010820 con 0x7fae6c0d1790 2023-10-18T08:55:12.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.053+0000 7fae637fe640 1 --2- 172.21.15.40:0/1020610776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fae4c06ae40 0x7fae4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:12.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.053+0000 7fae637fe640 1 -- 172.21.15.40:0/1020610776 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fae5c0897a0 con 0x7fae6c0d1790 2023-10-18T08:55:12.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.053+0000 7fae63fff640 1 --2- 172.21.15.40:0/1020610776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fae4c06ae40 0x7fae4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:12.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.055+0000 7fae63fff640 1 --2- 172.21.15.40:0/1020610776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fae4c06ae40 0x7fae4c06d300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fae54035000 tx=0x7fae54004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:12.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.057+0000 7fae637fe640 1 -- 172.21.15.40:0/1020610776 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fae5c054bb0 con 0x7fae6c0d1790 2023-10-18T08:55:12.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.254+0000 7fae7087b640 1 -- 172.21.15.40:0/1020610776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fae6c0c8a70 con 0x7fae6c0d1790 2023-10-18T08:55:12.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.254+0000 7fae637fe640 1 -- 172.21.15.40:0/1020610776 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fae5c0581d0 con 0x7fae6c0d1790 2023-10-18T08:55:12.257 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:55:12.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.257+0000 7fae7087b640 1 -- 172.21.15.40:0/1020610776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fae4c06ae40 msgr2=0x7fae4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:12.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.257+0000 7fae7087b640 1 --2- 172.21.15.40:0/1020610776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fae4c06ae40 0x7fae4c06d300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fae54035000 tx=0x7fae54004730 comp rx=0 tx=0).stop 2023-10-18T08:55:12.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.257+0000 7fae7087b640 1 -- 172.21.15.40:0/1020610776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d1790 msgr2=0x7fae6c15c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:12.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.257+0000 7fae7087b640 1 --2- 172.21.15.40:0/1020610776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d1790 0x7fae6c15c6d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00d950 tx=0x7fae5c00de10 comp rx=0 tx=0).stop 2023-10-18T08:55:12.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.257+0000 7fae7087b640 1 -- 172.21.15.40:0/1020610776 shutdown_connections 2023-10-18T08:55:12.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.257+0000 7fae7087b640 1 --2- 172.21.15.40:0/1020610776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fae4c06ae40 0x7fae4c06d300 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.257+0000 7fae7087b640 1 --2- 172.21.15.40:0/1020610776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae6c15b8b0 0x7fae6c14f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.257+0000 7fae7087b640 1 --2- 172.21.15.40:0/1020610776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae6c15b4d0 0x7fae6c15dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.257+0000 7fae7087b640 1 --2- 172.21.15.40:0/1020610776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0d1790 0x7fae6c15c6d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.257+0000 7fae7087b640 1 -- 172.21.15.40:0/1020610776 >> 172.21.15.40:0/1020610776 conn(0x7fae6c08ad30 msgr2=0x7fae6c0cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:12.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.258+0000 7fae7087b640 1 -- 172.21.15.40:0/1020610776 shutdown_connections 2023-10-18T08:55:12.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.258+0000 7fae7087b640 1 -- 172.21.15.40:0/1020610776 wait complete. 2023-10-18T08:55:12.279 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T08:55:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.508+0000 7f2a2de02640 1 -- 172.21.15.40:0/2807575922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a10003970 msgr2=0x7f2a10003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.508+0000 7f2a2de02640 1 --2- 172.21.15.40:0/2807575922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a10003970 0x7f2a10003d90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2a2400c260 tx=0x7f2a100057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.508+0000 7f2a2d601640 1 -- 172.21.15.40:0/2807575922 reap_dead start 2023-10-18T08:55:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.508+0000 7f2a2de02640 1 -- 172.21.15.40:0/2807575922 shutdown_connections 2023-10-18T08:55:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.508+0000 7f2a2de02640 1 -- 172.21.15.40:0/2807575922 >> 172.21.15.40:0/2807575922 conn(0x7f2a2808ad00 msgr2=0x7f2a2809a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.508+0000 7f2a2de02640 1 -- 172.21.15.40:0/2807575922 shutdown_connections 2023-10-18T08:55:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.508+0000 7f2a2de02640 1 -- 172.21.15.40:0/2807575922 wait complete. 2023-10-18T08:55:12.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.510+0000 7f2a2de02640 1 Processor -- start 2023-10-18T08:55:12.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.510+0000 7f2a2de02640 1 -- start start 2023-10-18T08:55:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.511+0000 7f2a2de02640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a10003970 0x7f2a28148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.511+0000 7f2a2de02640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a280b4900 0x7f2a2815d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.511+0000 7f2a2de02640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a280b4d80 0x7f2a280b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.512+0000 7f2a2de02640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2815c080 con 0x7f2a280b4d80 2023-10-18T08:55:12.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.512+0000 7f2a2de02640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2815c360 con 0x7f2a10003970 2023-10-18T08:55:12.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.512+0000 7f2a2de02640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2815c640 con 0x7f2a280b4900 2023-10-18T08:55:12.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.512+0000 7f2a2d601640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a280b4d80 0x7f2a280b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:12.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.512+0000 7f2a2d601640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a280b4d80 0x7f2a280b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57522/0 (socket says 172.21.15.40:57522) 2023-10-18T08:55:12.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.512+0000 7f2a2d601640 1 -- 172.21.15.40:0/2856416478 learned_addr learned my addr 172.21.15.40:0/2856416478 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:12.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.512+0000 7f2a2ce00640 1 --2- 172.21.15.40:0/2856416478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a10003970 0x7f2a28148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:12.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.512+0000 7f2a1ffff640 1 --2- 172.21.15.40:0/2856416478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a280b4900 0x7f2a2815d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:12.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.512+0000 7f2a2d601640 1 -- 172.21.15.40:0/2856416478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a280b4900 msgr2=0x7f2a2815d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:12.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.512+0000 7f2a2d601640 1 --2- 172.21.15.40:0/2856416478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a280b4900 0x7f2a2815d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.512+0000 7f2a2d601640 1 -- 172.21.15.40:0/2856416478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a10003970 msgr2=0x7f2a28148740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:12.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.512+0000 7f2a2d601640 1 --2- 172.21.15.40:0/2856416478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a10003970 0x7f2a28148740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.512+0000 7f2a2d601640 1 -- 172.21.15.40:0/2856416478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2815c920 con 0x7f2a280b4d80 2023-10-18T08:55:12.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.513+0000 7f2a2d601640 1 --2- 172.21.15.40:0/2856416478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a280b4d80 0x7f2a280b1430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a2815f150 tx=0x7f2a2000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:12.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.513+0000 7f2a1d7fa640 1 -- 172.21.15.40:0/2856416478 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a20018070 con 0x7f2a280b4d80 2023-10-18T08:55:12.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.513+0000 7f2a1d7fa640 1 -- 172.21.15.40:0/2856416478 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2a20004c10 con 0x7f2a280b4d80 2023-10-18T08:55:12.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.513+0000 7f2a1d7fa640 1 -- 172.21.15.40:0/2856416478 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2000cb30 con 0x7f2a280b4d80 2023-10-18T08:55:12.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.513+0000 7f2a2de02640 1 -- 172.21.15.40:0/2856416478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2815cc00 con 0x7f2a280b4d80 2023-10-18T08:55:12.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.513+0000 7f2a2de02640 1 -- 172.21.15.40:0/2856416478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2815bc30 con 0x7f2a280b4d80 2023-10-18T08:55:12.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.514+0000 7f2a2de02640 1 -- 172.21.15.40:0/2856416478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a28071da0 con 0x7f2a280b4d80 2023-10-18T08:55:12.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.515+0000 7f2a1d7fa640 1 -- 172.21.15.40:0/2856416478 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2a2000cc90 con 0x7f2a280b4d80 2023-10-18T08:55:12.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.516+0000 7f2a1d7fa640 1 --2- 172.21.15.40:0/2856416478 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a1406ae40 0x7f2a1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:12.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.516+0000 7f2a1d7fa640 1 -- 172.21.15.40:0/2856416478 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2a20088f90 con 0x7f2a280b4d80 2023-10-18T08:55:12.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.516+0000 7f2a2ce00640 1 --2- 172.21.15.40:0/2856416478 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a1406ae40 0x7f2a1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:12.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.517+0000 7f2a2ce00640 1 --2- 172.21.15.40:0/2856416478 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a1406ae40 0x7f2a1406d300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2a24034000 tx=0x7f2a24001a00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:12.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.520+0000 7f2a1d7fa640 1 -- 172.21.15.40:0/2856416478 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a20054380 con 0x7f2a280b4d80 2023-10-18T08:55:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.714+0000 7f2a2de02640 1 -- 172.21.15.40:0/2856416478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2a28066e00 con 0x7f2a280b4d80 2023-10-18T08:55:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.714+0000 7f2a1d7fa640 1 -- 172.21.15.40:0/2856416478 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2a200579a0 con 0x7f2a280b4d80 2023-10-18T08:55:12.717 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:12.717 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-10-18T08:55:12.717 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:55:12.717 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-10-18T08:55:12.718 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-10-18T08:55:12.718 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:55:12.718 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-10-18T08:55:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.717+0000 7f2a2de02640 1 -- 172.21.15.40:0/2856416478 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a1406ae40 msgr2=0x7f2a1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:12.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.717+0000 7f2a2de02640 1 --2- 172.21.15.40:0/2856416478 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a1406ae40 0x7f2a1406d300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2a24034000 tx=0x7f2a24001a00 comp rx=0 tx=0).stop 2023-10-18T08:55:12.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.717+0000 7f2a2de02640 1 -- 172.21.15.40:0/2856416478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a280b4d80 msgr2=0x7f2a280b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:12.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.717+0000 7f2a2de02640 1 --2- 172.21.15.40:0/2856416478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a280b4d80 0x7f2a280b1430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2a2815f150 tx=0x7f2a2000eec0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.717+0000 7f2a2de02640 1 -- 172.21.15.40:0/2856416478 shutdown_connections 2023-10-18T08:55:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.717+0000 7f2a2de02640 1 --2- 172.21.15.40:0/2856416478 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a1406ae40 0x7f2a1406d300 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.717+0000 7f2a2de02640 1 --2- 172.21.15.40:0/2856416478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a280b4d80 0x7f2a280b1430 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.717+0000 7f2a2de02640 1 --2- 172.21.15.40:0/2856416478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a280b4900 0x7f2a2815d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.717+0000 7f2a2de02640 1 --2- 172.21.15.40:0/2856416478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a10003970 0x7f2a28148740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.717+0000 7f2a2de02640 1 -- 172.21.15.40:0/2856416478 >> 172.21.15.40:0/2856416478 conn(0x7f2a2808ad00 msgr2=0x7f2a2806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:12.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.717+0000 7f2a2de02640 1 -- 172.21.15.40:0/2856416478 shutdown_connections 2023-10-18T08:55:12.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.717+0000 7f2a2de02640 1 -- 172.21.15.40:0/2856416478 wait complete. 2023-10-18T08:55:12.739 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T08:55:12.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T08:55:12.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.962+0000 7f87dddae640 1 -- 172.21.15.40:0/1237838891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87c4003f70 msgr2=0x7f87c40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:12.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.962+0000 7f87dddae640 1 --2- 172.21.15.40:0/1237838891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87c4003f70 0x7f87c40047c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f87c0006ed0 tx=0x7f87c40057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.962+0000 7f87dd5ad640 1 -- 172.21.15.40:0/1237838891 reap_dead start 2023-10-18T08:55:12.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.962+0000 7f87dddae640 1 -- 172.21.15.40:0/1237838891 shutdown_connections 2023-10-18T08:55:12.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.962+0000 7f87dddae640 1 -- 172.21.15.40:0/1237838891 >> 172.21.15.40:0/1237838891 conn(0x7f87d808acf0 msgr2=0x7f87d809a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:12.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.962+0000 7f87dddae640 1 -- 172.21.15.40:0/1237838891 shutdown_connections 2023-10-18T08:55:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.962+0000 7f87dddae640 1 -- 172.21.15.40:0/1237838891 wait complete. 2023-10-18T08:55:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.964+0000 7f87dddae640 1 Processor -- start 2023-10-18T08:55:12.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.964+0000 7f87dddae640 1 -- start start 2023-10-18T08:55:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.965+0000 7f87dddae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87c4003f70 0x7f87d81f1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.965+0000 7f87dddae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87d80c5930 0x7f87d815e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:12.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.965+0000 7f87dddae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87d80c5db0 0x7f87d80b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.965+0000 7f87dddae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d81f6160 con 0x7f87c4003f70 2023-10-18T08:55:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.965+0000 7f87dddae640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d81f55e0 con 0x7f87d80c5db0 2023-10-18T08:55:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.965+0000 7f87dddae640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d81f58c0 con 0x7f87d80c5930 2023-10-18T08:55:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.965+0000 7f87dcdac640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87c4003f70 0x7f87d81f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:12.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.965+0000 7f87dcdac640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87c4003f70 0x7f87d81f1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57546/0 (socket says 172.21.15.40:57546) 2023-10-18T08:55:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.965+0000 7f87dcdac640 1 -- 172.21.15.40:0/2704003435 learned_addr learned my addr 172.21.15.40:0/2704003435 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.965+0000 7f87dcdac640 1 -- 172.21.15.40:0/2704003435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87d80c5930 msgr2=0x7f87d815e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.965+0000 7f87dcdac640 1 --2- 172.21.15.40:0/2704003435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87d80c5930 0x7f87d815e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.965+0000 7f87dcdac640 1 -- 172.21.15.40:0/2704003435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87d80c5db0 msgr2=0x7f87d80b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.965+0000 7f87dcdac640 1 --2- 172.21.15.40:0/2704003435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87d80c5db0 0x7f87d80b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:12.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.965+0000 7f87dcdac640 1 -- 172.21.15.40:0/2704003435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d81f5ba0 con 0x7f87c4003f70 2023-10-18T08:55:12.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.966+0000 7f87dcdac640 1 --2- 172.21.15.40:0/2704003435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87c4003f70 0x7f87d81f1570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c840 tx=0x7f87c800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:12.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.966+0000 7f87d57fa640 1 -- 172.21.15.40:0/2704003435 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8003b70 con 0x7f87c4003f70 2023-10-18T08:55:12.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.966+0000 7f87d57fa640 1 -- 172.21.15.40:0/2704003435 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f87c8003cf0 con 0x7f87c4003f70 2023-10-18T08:55:12.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.966+0000 7f87d57fa640 1 -- 172.21.15.40:0/2704003435 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8003ea0 con 0x7f87c4003f70 2023-10-18T08:55:12.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.966+0000 7f87dddae640 1 -- 172.21.15.40:0/2704003435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d81f5e80 con 0x7f87c4003f70 2023-10-18T08:55:12.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.966+0000 7f87dddae640 1 -- 172.21.15.40:0/2704003435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d81f5240 con 0x7f87c4003f70 2023-10-18T08:55:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.967+0000 7f87dddae640 1 -- 172.21.15.40:0/2704003435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d8071da0 con 0x7f87c4003f70 2023-10-18T08:55:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.968+0000 7f87d57fa640 1 -- 172.21.15.40:0/2704003435 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f87c800aa30 con 0x7f87c4003f70 2023-10-18T08:55:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.969+0000 7f87d57fa640 1 --2- 172.21.15.40:0/2704003435 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87b806ae40 0x7f87b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.969+0000 7f87d57fa640 1 -- 172.21.15.40:0/2704003435 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f87c8089390 con 0x7f87c4003f70 2023-10-18T08:55:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.970+0000 7f87d7fff640 1 --2- 172.21.15.40:0/2704003435 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87b806ae40 0x7f87b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.971+0000 7f87d7fff640 1 --2- 172.21.15.40:0/2704003435 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87b806ae40 0x7f87b806d300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f87c0034000 tx=0x7f87c0039040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:12.973+0000 7f87d57fa640 1 -- 172.21.15.40:0/2704003435 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87c80546f0 con 0x7f87c4003f70 2023-10-18T08:55:13.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.157+0000 7f87dddae640 1 -- 172.21.15.40:0/2704003435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f87d8066ea0 con 0x7f87c4003f70 2023-10-18T08:55:13.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.265+0000 7f87d57fa640 1 -- 172.21.15.40:0/2704003435 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f87c8057d10 con 0x7f87c4003f70 2023-10-18T08:55:13.268 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-10-18T08:55:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.268+0000 7f87dddae640 1 -- 172.21.15.40:0/2704003435 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87b806ae40 msgr2=0x7f87b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.268+0000 7f87dddae640 1 --2- 172.21.15.40:0/2704003435 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87b806ae40 0x7f87b806d300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f87c0034000 tx=0x7f87c0039040 comp rx=0 tx=0).stop 2023-10-18T08:55:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.268+0000 7f87dddae640 1 -- 172.21.15.40:0/2704003435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87c4003f70 msgr2=0x7f87d81f1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:13.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.268+0000 7f87dddae640 1 --2- 172.21.15.40:0/2704003435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87c4003f70 0x7f87d81f1570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c840 tx=0x7f87c800cd00 comp rx=0 tx=0).stop 2023-10-18T08:55:13.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.269+0000 7f87dddae640 1 -- 172.21.15.40:0/2704003435 shutdown_connections 2023-10-18T08:55:13.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.269+0000 7f87dddae640 1 --2- 172.21.15.40:0/2704003435 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87b806ae40 0x7f87b806d300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:13.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.269+0000 7f87dddae640 1 --2- 172.21.15.40:0/2704003435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87d80c5db0 0x7f87d80b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:13.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.269+0000 7f87dddae640 1 --2- 172.21.15.40:0/2704003435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87d80c5930 0x7f87d815e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:13.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.269+0000 7f87dddae640 1 --2- 172.21.15.40:0/2704003435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87c4003f70 0x7f87d81f1570 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:13.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.269+0000 7f87dddae640 1 -- 172.21.15.40:0/2704003435 >> 172.21.15.40:0/2704003435 conn(0x7f87d808acf0 msgr2=0x7f87d80b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:13.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.269+0000 7f87dddae640 1 -- 172.21.15.40:0/2704003435 shutdown_connections 2023-10-18T08:55:13.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.270+0000 7f87dddae640 1 -- 172.21.15.40:0/2704003435 wait complete. 2023-10-18T08:55:13.293 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T08:55:13.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.514+0000 7f803f59e640 1 -- 172.21.15.40:0/1106286765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80400ac430 msgr2=0x7f8040155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:13.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.514+0000 7f803f59e640 1 --2- 172.21.15.40:0/1106286765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80400ac430 0x7f8040155330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8038005fd0 tx=0x7f803802c2e0 comp rx=0 tx=0).stop 2023-10-18T08:55:13.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.514+0000 7f803f59e640 1 -- 172.21.15.40:0/1106286765 shutdown_connections 2023-10-18T08:55:13.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.514+0000 7f803f59e640 1 --2- 172.21.15.40:0/1106286765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80400ac430 0x7f8040155330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:13.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.514+0000 7f803f59e640 1 --2- 172.21.15.40:0/1106286765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80400cc630 0x7f804006baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:13.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.514+0000 7f803f59e640 1 -- 172.21.15.40:0/1106286765 >> 172.21.15.40:0/1106286765 conn(0x7f804006ca10 msgr2=0x7f804006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:13.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.515+0000 7f803f59e640 1 -- 172.21.15.40:0/1106286765 shutdown_connections 2023-10-18T08:55:13.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.515+0000 7f803f59e640 1 -- 172.21.15.40:0/1106286765 wait complete. 2023-10-18T08:55:13.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.516+0000 7f803f59e640 1 Processor -- start 2023-10-18T08:55:13.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.516+0000 7f803f59e640 1 -- start start 2023-10-18T08:55:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.517+0000 7f803f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f804006c030 0x7f80401f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.517+0000 7f803f59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80400ac430 0x7f8040146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.517+0000 7f803f59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80400cc630 0x7f8040153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:13.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.517+0000 7f803f59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8040146f40 con 0x7f804006c030 2023-10-18T08:55:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.517+0000 7f803f59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80400afd90 con 0x7f80400cc630 2023-10-18T08:55:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.517+0000 7f803f59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80401541a0 con 0x7f80400ac430 2023-10-18T08:55:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.518+0000 7f803e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f804006c030 0x7f80401f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.518+0000 7f803e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f804006c030 0x7f80401f3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57566/0 (socket says 172.21.15.40:57566) 2023-10-18T08:55:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.518+0000 7f803e59c640 1 -- 172.21.15.40:0/3240898965 learned_addr learned my addr 172.21.15.40:0/3240898965 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.518+0000 7f803e59c640 1 -- 172.21.15.40:0/3240898965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80400ac430 msgr2=0x7f8040146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.518+0000 7f803e59c640 1 --2- 172.21.15.40:0/3240898965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80400ac430 0x7f8040146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.518+0000 7f803e59c640 1 -- 172.21.15.40:0/3240898965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80400cc630 msgr2=0x7f8040153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.518+0000 7f803e59c640 1 --2- 172.21.15.40:0/3240898965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80400cc630 0x7f8040153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:13.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.518+0000 7f803e59c640 1 -- 172.21.15.40:0/3240898965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8038009900 con 0x7f804006c030 2023-10-18T08:55:13.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.518+0000 7f803e59c640 1 --2- 172.21.15.40:0/3240898965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f804006c030 0x7f80401f3130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f803000d740 tx=0x7f803000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:13.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.519+0000 7f802effd640 1 -- 172.21.15.40:0/3240898965 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030009c30 con 0x7f804006c030 2023-10-18T08:55:13.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.519+0000 7f802effd640 1 -- 172.21.15.40:0/3240898965 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8030009e80 con 0x7f804006c030 2023-10-18T08:55:13.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.519+0000 7f802effd640 1 -- 172.21.15.40:0/3240898965 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80300106c0 con 0x7f804006c030 2023-10-18T08:55:13.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.519+0000 7f803f59e640 1 -- 172.21.15.40:0/3240898965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8040154480 con 0x7f804006c030 2023-10-18T08:55:13.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.519+0000 7f803f59e640 1 -- 172.21.15.40:0/3240898965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80400b0310 con 0x7f804006c030 2023-10-18T08:55:13.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.520+0000 7f803f59e640 1 -- 172.21.15.40:0/3240898965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8040066e00 con 0x7f804006c030 2023-10-18T08:55:13.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.521+0000 7f802effd640 1 -- 172.21.15.40:0/3240898965 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8030010820 con 0x7f804006c030 2023-10-18T08:55:13.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.523+0000 7f802effd640 1 --2- 172.21.15.40:0/3240898965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f802006ae70 0x7f802006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:13.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.523+0000 7f802effd640 1 -- 172.21.15.40:0/3240898965 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f80300899d0 con 0x7f804006c030 2023-10-18T08:55:13.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.523+0000 7f803dd9b640 1 --2- 172.21.15.40:0/3240898965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f802006ae70 0x7f802006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:13.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.524+0000 7f803dd9b640 1 --2- 172.21.15.40:0/3240898965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f802006ae70 0x7f802006d330 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8034000f60 tx=0x7f80340079d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:13.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.528+0000 7f802effd640 1 -- 172.21.15.40:0/3240898965 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8030054bc0 con 0x7f804006c030 2023-10-18T08:55:13.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:13.720+0000 7f803f59e640 1 -- 172.21.15.40:0/3240898965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f804006f620 con 0x7f804006c030 2023-10-18T08:55:14.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.271+0000 7f802effd640 1 -- 172.21.15.40:0/3240898965 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f80300210d0 con 0x7f804006c030 2023-10-18T08:55:14.274 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-10-18T08:55:14.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.275+0000 7f803f59e640 1 -- 172.21.15.40:0/3240898965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f802006ae70 msgr2=0x7f802006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:14.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.275+0000 7f803f59e640 1 --2- 172.21.15.40:0/3240898965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f802006ae70 0x7f802006d330 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8034000f60 tx=0x7f80340079d0 comp rx=0 tx=0).stop 2023-10-18T08:55:14.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.275+0000 7f803f59e640 1 -- 172.21.15.40:0/3240898965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f804006c030 msgr2=0x7f80401f3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:14.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.275+0000 7f803f59e640 1 --2- 172.21.15.40:0/3240898965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f804006c030 0x7f80401f3130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f803000d740 tx=0x7f803000dc00 comp rx=0 tx=0).stop 2023-10-18T08:55:14.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.275+0000 7f803f59e640 1 -- 172.21.15.40:0/3240898965 shutdown_connections 2023-10-18T08:55:14.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.275+0000 7f803f59e640 1 --2- 172.21.15.40:0/3240898965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f802006ae70 0x7f802006d330 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:14.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.275+0000 7f803f59e640 1 --2- 172.21.15.40:0/3240898965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80400cc630 0x7f8040153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:14.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.275+0000 7f803f59e640 1 --2- 172.21.15.40:0/3240898965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80400ac430 0x7f8040146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:14.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.275+0000 7f803f59e640 1 --2- 172.21.15.40:0/3240898965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f804006c030 0x7f80401f3130 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:14.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.275+0000 7f803f59e640 1 -- 172.21.15.40:0/3240898965 >> 172.21.15.40:0/3240898965 conn(0x7f804006ca10 msgr2=0x7f804015c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:14.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.275+0000 7f803f59e640 1 -- 172.21.15.40:0/3240898965 shutdown_connections 2023-10-18T08:55:14.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.275+0000 7f803f59e640 1 -- 172.21.15.40:0/3240898965 wait complete. 2023-10-18T08:55:14.306 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T08:55:14.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.506+0000 7f1d067c0640 1 -- 172.21.15.40:0/3202025844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d00159f00 msgr2=0x7f1d001589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.506+0000 7f1d067c0640 1 --2- 172.21.15.40:0/3202025844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d00159f00 0x7f1d001589c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009900 tx=0x7f1cf402be20 comp rx=0 tx=0).stop 2023-10-18T08:55:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.506+0000 7f1d067c0640 1 -- 172.21.15.40:0/3202025844 shutdown_connections 2023-10-18T08:55:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.506+0000 7f1d067c0640 1 --2- 172.21.15.40:0/3202025844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d00159f00 0x7f1d001589c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.506+0000 7f1d067c0640 1 -- 172.21.15.40:0/3202025844 >> 172.21.15.40:0/3202025844 conn(0x7f1d00069720 msgr2=0x7f1d00069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.506+0000 7f1d067c0640 1 -- 172.21.15.40:0/3202025844 shutdown_connections 2023-10-18T08:55:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.506+0000 7f1d067c0640 1 -- 172.21.15.40:0/3202025844 wait complete. 2023-10-18T08:55:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.508+0000 7f1d067c0640 1 Processor -- start 2023-10-18T08:55:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.509+0000 7f1d067c0640 1 -- start start 2023-10-18T08:55:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.510+0000 7f1d067c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d000b0190 0x7f1d001e30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.510+0000 7f1d067c0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d000b0a80 0x7f1d000cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.510+0000 7f1d067c0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d00159f00 0x7f1d000cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.510+0000 7f1d067c0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000cbec0 con 0x7f1d000b0190 2023-10-18T08:55:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.510+0000 7f1d067c0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000cc1a0 con 0x7f1d00159f00 2023-10-18T08:55:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.510+0000 7f1d067c0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d001e0ca0 con 0x7f1d000b0a80 2023-10-18T08:55:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.510+0000 7f1cfffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d000b0190 0x7f1d001e30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.510+0000 7f1cfffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d000b0190 0x7f1d001e30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57580/0 (socket says 172.21.15.40:57580) 2023-10-18T08:55:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.510+0000 7f1cfffff640 1 -- 172.21.15.40:0/425982100 learned_addr learned my addr 172.21.15.40:0/425982100 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.511+0000 7f1cfffff640 1 -- 172.21.15.40:0/425982100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d000b0a80 msgr2=0x7f1d000cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.511+0000 7f1cfffff640 1 --2- 172.21.15.40:0/425982100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d000b0a80 0x7f1d000cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.511+0000 7f1cfffff640 1 -- 172.21.15.40:0/425982100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d00159f00 msgr2=0x7f1d000cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.511+0000 7f1cfffff640 1 --2- 172.21.15.40:0/425982100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d00159f00 0x7f1d000cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.511+0000 7f1cfffff640 1 -- 172.21.15.40:0/425982100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf4009560 con 0x7f1d000b0190 2023-10-18T08:55:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.511+0000 7f1cfffff640 1 --2- 172.21.15.40:0/425982100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d000b0190 0x7f1d001e30f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000c450 tx=0x7f1cf000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.511+0000 7f1cfcff9640 1 -- 172.21.15.40:0/425982100 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00052d0 con 0x7f1d000b0190 2023-10-18T08:55:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.511+0000 7f1cfcff9640 1 -- 172.21.15.40:0/425982100 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1cf0005430 con 0x7f1d000b0190 2023-10-18T08:55:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.511+0000 7f1cfcff9640 1 -- 172.21.15.40:0/425982100 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0012890 con 0x7f1d000b0190 2023-10-18T08:55:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.511+0000 7f1d067c0640 1 -- 172.21.15.40:0/425982100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d001e0f80 con 0x7f1d000b0190 2023-10-18T08:55:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.512+0000 7f1d067c0640 1 -- 172.21.15.40:0/425982100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d001424d0 con 0x7f1d000b0190 2023-10-18T08:55:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.513+0000 7f1d067c0640 1 -- 172.21.15.40:0/425982100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d00066fd0 con 0x7f1d000b0190 2023-10-18T08:55:14.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.514+0000 7f1cfcff9640 1 -- 172.21.15.40:0/425982100 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1cf00035f0 con 0x7f1d000b0190 2023-10-18T08:55:14.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.515+0000 7f1cfcff9640 1 --2- 172.21.15.40:0/425982100 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cec06ae40 0x7f1cec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:14.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.515+0000 7f1cfcff9640 1 -- 172.21.15.40:0/425982100 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f1cf0089300 con 0x7f1d000b0190 2023-10-18T08:55:14.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.515+0000 7f1cff7fe640 1 --2- 172.21.15.40:0/425982100 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cec06ae40 0x7f1cec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:14.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.516+0000 7f1cff7fe640 1 --2- 172.21.15.40:0/425982100 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cec06ae40 0x7f1cec06d300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4035000 tx=0x7f1cf40046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.520+0000 7f1cfcff9640 1 -- 172.21.15.40:0/425982100 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1cf0054370 con 0x7f1d000b0190 2023-10-18T08:55:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:14.751+0000 7f1d067c0640 1 -- 172.21.15.40:0/425982100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1d001599e0 con 0x7f1d000b0190 2023-10-18T08:55:15.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.293+0000 7f1cfcff9640 1 -- 172.21.15.40:0/425982100 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1cf0057990 con 0x7f1d000b0190 2023-10-18T08:55:15.298 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 2 and data pool 3 2023-10-18T08:55:15.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.304+0000 7f1d067c0640 1 -- 172.21.15.40:0/425982100 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cec06ae40 msgr2=0x7f1cec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:15.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.304+0000 7f1d067c0640 1 --2- 172.21.15.40:0/425982100 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cec06ae40 0x7f1cec06d300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4035000 tx=0x7f1cf40046c0 comp rx=0 tx=0).stop 2023-10-18T08:55:15.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.304+0000 7f1d067c0640 1 -- 172.21.15.40:0/425982100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d000b0190 msgr2=0x7f1d001e30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:15.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.304+0000 7f1d067c0640 1 --2- 172.21.15.40:0/425982100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d000b0190 0x7f1d001e30f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000c450 tx=0x7f1cf000c910 comp rx=0 tx=0).stop 2023-10-18T08:55:15.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.306+0000 7f1d067c0640 1 -- 172.21.15.40:0/425982100 shutdown_connections 2023-10-18T08:55:15.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.306+0000 7f1d067c0640 1 --2- 172.21.15.40:0/425982100 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cec06ae40 0x7f1cec06d300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:15.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.306+0000 7f1d067c0640 1 --2- 172.21.15.40:0/425982100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d00159f00 0x7f1d000cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:15.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.306+0000 7f1d067c0640 1 --2- 172.21.15.40:0/425982100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d000b0a80 0x7f1d000cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:15.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.306+0000 7f1d067c0640 1 --2- 172.21.15.40:0/425982100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d000b0190 0x7f1d001e30f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:15.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.306+0000 7f1d067c0640 1 -- 172.21.15.40:0/425982100 >> 172.21.15.40:0/425982100 conn(0x7f1d00069720 msgr2=0x7f1d0015aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:15.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.308+0000 7f1d067c0640 1 -- 172.21.15.40:0/425982100 shutdown_connections 2023-10-18T08:55:15.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.308+0000 7f1d067c0640 1 -- 172.21.15.40:0/425982100 wait complete. 2023-10-18T08:55:15.333 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:15.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.529+0000 7f8dca290640 1 -- 172.21.15.40:0/777377034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db0003fd0 msgr2=0x7f8db00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:15.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.529+0000 7f8dca290640 1 --2- 172.21.15.40:0/777377034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db0003fd0 0x7f8db00047c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8db4006fd0 tx=0x7f8db00057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:15.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.529+0000 7f8dc9a8f640 1 -- 172.21.15.40:0/777377034 reap_dead start 2023-10-18T08:55:15.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.529+0000 7f8dca290640 1 -- 172.21.15.40:0/777377034 shutdown_connections 2023-10-18T08:55:15.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.529+0000 7f8dca290640 1 -- 172.21.15.40:0/777377034 >> 172.21.15.40:0/777377034 conn(0x7f8dc408aca0 msgr2=0x7f8dc409a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:15.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.529+0000 7f8dca290640 1 -- 172.21.15.40:0/777377034 shutdown_connections 2023-10-18T08:55:15.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.529+0000 7f8dca290640 1 -- 172.21.15.40:0/777377034 wait complete. 2023-10-18T08:55:15.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.531+0000 7f8dca290640 1 Processor -- start 2023-10-18T08:55:15.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.531+0000 7f8dca290640 1 -- start start 2023-10-18T08:55:15.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.532+0000 7f8dca290640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db0003fd0 0x7f8dc40c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:15.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.532+0000 7f8dca290640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8dc40c5850 0x7f8dc415bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:15.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.532+0000 7f8dca290640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dc40c5cb0 0x7f8dc40ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:15.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.532+0000 7f8dca290640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc40c9820 con 0x7f8db0003fd0 2023-10-18T08:55:15.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.532+0000 7f8dca290640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc40c9b00 con 0x7f8dc40c5cb0 2023-10-18T08:55:15.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.532+0000 7f8dca290640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc40c9de0 con 0x7f8dc40c5850 2023-10-18T08:55:15.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.532+0000 7f8dc928e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db0003fd0 0x7f8dc40c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:15.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.532+0000 7f8dc928e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db0003fd0 0x7f8dc40c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36902/0 (socket says 172.21.15.40:36902) 2023-10-18T08:55:15.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.532+0000 7f8dc928e640 1 -- 172.21.15.40:0/1570623712 learned_addr learned my addr 172.21.15.40:0/1570623712 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:15.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.532+0000 7f8dc8a8d640 1 --2- 172.21.15.40:0/1570623712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8dc40c5850 0x7f8dc415bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:15.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.532+0000 7f8dc9a8f640 1 --2- 172.21.15.40:0/1570623712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dc40c5cb0 0x7f8dc40ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:15.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.532+0000 7f8dc928e640 1 -- 172.21.15.40:0/1570623712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8dc40c5850 msgr2=0x7f8dc415bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:15.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.532+0000 7f8dc928e640 1 --2- 172.21.15.40:0/1570623712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8dc40c5850 0x7f8dc415bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:15.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.532+0000 7f8dc928e640 1 -- 172.21.15.40:0/1570623712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dc40c5cb0 msgr2=0x7f8dc40ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:15.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.532+0000 7f8dc928e640 1 --2- 172.21.15.40:0/1570623712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dc40c5cb0 0x7f8dc40ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:15.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.532+0000 7f8dc928e640 1 -- 172.21.15.40:0/1570623712 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc40ca0c0 con 0x7f8db0003fd0 2023-10-18T08:55:15.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.533+0000 7f8dc928e640 1 --2- 172.21.15.40:0/1570623712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db0003fd0 0x7f8dc40c75d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0004e20 tx=0x7f8dc0001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:15.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.533+0000 7f8db9ffb640 1 -- 172.21.15.40:0/1570623712 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc00107d0 con 0x7f8db0003fd0 2023-10-18T08:55:15.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.533+0000 7f8db9ffb640 1 -- 172.21.15.40:0/1570623712 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8dc0010a40 con 0x7f8db0003fd0 2023-10-18T08:55:15.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.533+0000 7f8db9ffb640 1 -- 172.21.15.40:0/1570623712 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0010e50 con 0x7f8db0003fd0 2023-10-18T08:55:15.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.533+0000 7f8dca290640 1 -- 172.21.15.40:0/1570623712 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc40ca3a0 con 0x7f8db0003fd0 2023-10-18T08:55:15.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.533+0000 7f8dca290640 1 -- 172.21.15.40:0/1570623712 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc40c7b10 con 0x7f8db0003fd0 2023-10-18T08:55:15.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.534+0000 7f8dca290640 1 -- 172.21.15.40:0/1570623712 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc4071da0 con 0x7f8db0003fd0 2023-10-18T08:55:15.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.535+0000 7f8db9ffb640 1 -- 172.21.15.40:0/1570623712 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8dc001b460 con 0x7f8db0003fd0 2023-10-18T08:55:15.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.536+0000 7f8db9ffb640 1 --2- 172.21.15.40:0/1570623712 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8da806ae70 0x7f8da806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:15.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.536+0000 7f8db9ffb640 1 -- 172.21.15.40:0/1570623712 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8dc008a200 con 0x7f8db0003fd0 2023-10-18T08:55:15.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.537+0000 7f8dc8a8d640 1 --2- 172.21.15.40:0/1570623712 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8da806ae70 0x7f8da806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:15.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.538+0000 7f8dc8a8d640 1 --2- 172.21.15.40:0/1570623712 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8da806ae70 0x7f8da806d330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8db4034000 tx=0x7f8db402e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:15.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.541+0000 7f8db9ffb640 1 -- 172.21.15.40:0/1570623712 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8dc0055180 con 0x7f8db0003fd0 2023-10-18T08:55:15.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.738+0000 7f8dca290640 1 -- 172.21.15.40:0/1570623712 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8dc40667d0 con 0x7f8db0003fd0 2023-10-18T08:55:15.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.741+0000 7f8db9ffb640 1 -- 172.21.15.40:0/1570623712 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16530 (secure 0 0 0) 0x7f8dc00587a0 con 0x7f8db0003fd0 2023-10-18T08:55:15.744 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:15.744 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":19,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:14.752924+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:15.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.743+0000 7f8dca290640 1 -- 172.21.15.40:0/1570623712 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8da806ae70 msgr2=0x7f8da806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:15.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.743+0000 7f8dca290640 1 --2- 172.21.15.40:0/1570623712 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8da806ae70 0x7f8da806d330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8db4034000 tx=0x7f8db402e000 comp rx=0 tx=0).stop 2023-10-18T08:55:15.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.743+0000 7f8dca290640 1 -- 172.21.15.40:0/1570623712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db0003fd0 msgr2=0x7f8dc40c75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:15.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.743+0000 7f8dca290640 1 --2- 172.21.15.40:0/1570623712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db0003fd0 0x7f8dc40c75d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0004e20 tx=0x7f8dc0001630 comp rx=0 tx=0).stop 2023-10-18T08:55:15.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.743+0000 7f8dca290640 1 -- 172.21.15.40:0/1570623712 shutdown_connections 2023-10-18T08:55:15.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.743+0000 7f8dca290640 1 --2- 172.21.15.40:0/1570623712 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8da806ae70 0x7f8da806d330 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:15.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.743+0000 7f8dca290640 1 --2- 172.21.15.40:0/1570623712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dc40c5cb0 0x7f8dc40ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:15.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.744+0000 7f8dca290640 1 --2- 172.21.15.40:0/1570623712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8dc40c5850 0x7f8dc415bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:15.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.744+0000 7f8dca290640 1 --2- 172.21.15.40:0/1570623712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8db0003fd0 0x7f8dc40c75d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:15.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.744+0000 7f8dca290640 1 -- 172.21.15.40:0/1570623712 >> 172.21.15.40:0/1570623712 conn(0x7f8dc408aca0 msgr2=0x7f8dc40adf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:15.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.744+0000 7f8dca290640 1 -- 172.21.15.40:0/1570623712 shutdown_connections 2023-10-18T08:55:15.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.744+0000 7f8dca290640 1 -- 172.21.15.40:0/1570623712 wait complete. 2023-10-18T08:55:15.764 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:15.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.992+0000 7ff450ff5640 1 -- 172.21.15.40:0/2565426099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4380047c0 msgr2=0x7ff438004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:15.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.992+0000 7ff450ff5640 1 --2- 172.21.15.40:0/2565426099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4380047c0 0x7ff438004c40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff434006f40 tx=0x7ff4380057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:15.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.992+0000 7ff44ad9d640 1 -- 172.21.15.40:0/2565426099 reap_dead start 2023-10-18T08:55:15.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.992+0000 7ff450ff5640 1 -- 172.21.15.40:0/2565426099 shutdown_connections 2023-10-18T08:55:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.992+0000 7ff450ff5640 1 -- 172.21.15.40:0/2565426099 >> 172.21.15.40:0/2565426099 conn(0x7ff44c06cee0 msgr2=0x7ff44c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.992+0000 7ff450ff5640 1 -- 172.21.15.40:0/2565426099 shutdown_connections 2023-10-18T08:55:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.992+0000 7ff450ff5640 1 -- 172.21.15.40:0/2565426099 wait complete. 2023-10-18T08:55:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.995+0000 7ff450ff5640 1 Processor -- start 2023-10-18T08:55:15.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.995+0000 7ff450ff5640 1 -- start start 2023-10-18T08:55:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.996+0000 7ff450ff5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4380047c0 0x7ff44c152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.996+0000 7ff450ff5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff44c06f620 0x7ff44c1f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.996+0000 7ff450ff5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff44c1f5280 0x7ff44c156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.996+0000 7ff450ff5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c1f5800 con 0x7ff44c06f620 2023-10-18T08:55:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.996+0000 7ff450ff5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c1f5ae0 con 0x7ff4380047c0 2023-10-18T08:55:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.996+0000 7ff450ff5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c156130 con 0x7ff44c1f5280 2023-10-18T08:55:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.997+0000 7ff443fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff44c06f620 0x7ff44c1f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.997+0000 7ff443fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff44c06f620 0x7ff44c1f4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36910/0 (socket says 172.21.15.40:36910) 2023-10-18T08:55:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.997+0000 7ff443fff640 1 -- 172.21.15.40:0/1269660127 learned_addr learned my addr 172.21.15.40:0/1269660127 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.997+0000 7ff44ad9d640 1 --2- 172.21.15.40:0/1269660127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff44c1f5280 0x7ff44c156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.997+0000 7ff44a59c640 1 --2- 172.21.15.40:0/1269660127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4380047c0 0x7ff44c152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.997+0000 7ff443fff640 1 -- 172.21.15.40:0/1269660127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff44c1f5280 msgr2=0x7ff44c156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.997+0000 7ff443fff640 1 --2- 172.21.15.40:0/1269660127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff44c1f5280 0x7ff44c156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:16.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.997+0000 7ff443fff640 1 -- 172.21.15.40:0/1269660127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4380047c0 msgr2=0x7ff44c152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:16.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.997+0000 7ff443fff640 1 --2- 172.21.15.40:0/1269660127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4380047c0 0x7ff44c152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:16.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.997+0000 7ff443fff640 1 -- 172.21.15.40:0/1269660127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c156410 con 0x7ff44c06f620 2023-10-18T08:55:16.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.998+0000 7ff443fff640 1 --2- 172.21.15.40:0/1269660127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff44c06f620 0x7ff44c1f4d40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff43400c350 tx=0x7ff434006510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:16.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.998+0000 7ff442ffd640 1 -- 172.21.15.40:0/1269660127 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff434003a00 con 0x7ff44c06f620 2023-10-18T08:55:16.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.998+0000 7ff442ffd640 1 -- 172.21.15.40:0/1269660127 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff434003c50 con 0x7ff44c06f620 2023-10-18T08:55:16.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.998+0000 7ff450ff5640 1 -- 172.21.15.40:0/1269660127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44c142340 con 0x7ff44c06f620 2023-10-18T08:55:16.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.998+0000 7ff442ffd640 1 -- 172.21.15.40:0/1269660127 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff434035470 con 0x7ff44c06f620 2023-10-18T08:55:16.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.998+0000 7ff450ff5640 1 -- 172.21.15.40:0/1269660127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44c06faa0 con 0x7ff44c06f620 2023-10-18T08:55:16.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:15.999+0000 7ff450ff5640 1 -- 172.21.15.40:0/1269660127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44c1565a0 con 0x7ff44c06f620 2023-10-18T08:55:16.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.000+0000 7ff442ffd640 1 -- 172.21.15.40:0/1269660127 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff434032070 con 0x7ff44c06f620 2023-10-18T08:55:16.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.001+0000 7ff442ffd640 1 --2- 172.21.15.40:0/1269660127 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff42c06ae40 0x7ff42c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:16.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.001+0000 7ff442ffd640 1 -- 172.21.15.40:0/1269660127 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4340aacb0 con 0x7ff44c06f620 2023-10-18T08:55:16.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.002+0000 7ff44a59c640 1 --2- 172.21.15.40:0/1269660127 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff42c06ae40 0x7ff42c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:16.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.003+0000 7ff44a59c640 1 --2- 172.21.15.40:0/1269660127 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff42c06ae40 0x7ff42c06d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff43c006990 tx=0x7ff43c006b20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:16.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.007+0000 7ff442ffd640 1 -- 172.21.15.40:0/1269660127 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff434075ed0 con 0x7ff44c06f620 2023-10-18T08:55:16.118 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-18T08:55:16.116+0000 7f00c28bc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T08:55:16.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.202+0000 7ff450ff5640 1 -- 172.21.15.40:0/1269660127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff44c1f5c50 con 0x7ff44c06f620 2023-10-18T08:55:16.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.205+0000 7ff442ffd640 1 -- 172.21.15.40:0/1269660127 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16530 (secure 0 0 0) 0x7ff434043070 con 0x7ff44c06f620 2023-10-18T08:55:16.209 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:16.209 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":19,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:14.752924+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:16.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.208+0000 7ff450ff5640 1 -- 172.21.15.40:0/1269660127 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff42c06ae40 msgr2=0x7ff42c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:16.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.208+0000 7ff450ff5640 1 --2- 172.21.15.40:0/1269660127 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff42c06ae40 0x7ff42c06d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff43c006990 tx=0x7ff43c006b20 comp rx=0 tx=0).stop 2023-10-18T08:55:16.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.208+0000 7ff450ff5640 1 -- 172.21.15.40:0/1269660127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff44c06f620 msgr2=0x7ff44c1f4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:16.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.208+0000 7ff450ff5640 1 --2- 172.21.15.40:0/1269660127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff44c06f620 0x7ff44c1f4d40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff43400c350 tx=0x7ff434006510 comp rx=0 tx=0).stop 2023-10-18T08:55:16.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.208+0000 7ff450ff5640 1 -- 172.21.15.40:0/1269660127 shutdown_connections 2023-10-18T08:55:16.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.208+0000 7ff450ff5640 1 --2- 172.21.15.40:0/1269660127 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff42c06ae40 0x7ff42c06d300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:16.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.208+0000 7ff450ff5640 1 --2- 172.21.15.40:0/1269660127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff44c1f5280 0x7ff44c156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:16.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.208+0000 7ff450ff5640 1 --2- 172.21.15.40:0/1269660127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff44c06f620 0x7ff44c1f4d40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:16.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.208+0000 7ff450ff5640 1 --2- 172.21.15.40:0/1269660127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4380047c0 0x7ff44c152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:16.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.208+0000 7ff450ff5640 1 -- 172.21.15.40:0/1269660127 >> 172.21.15.40:0/1269660127 conn(0x7ff44c06cee0 msgr2=0x7ff44c13b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:16.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.209+0000 7ff450ff5640 1 -- 172.21.15.40:0/1269660127 shutdown_connections 2023-10-18T08:55:16.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.209+0000 7ff450ff5640 1 -- 172.21.15.40:0/1269660127 wait complete. 2023-10-18T08:55:16.229 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T08:55:16.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.463+0000 7fd00359e640 1 -- 172.21.15.40:0/4118038731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0040d1600 msgr2=0x7fd0040cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:16.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.463+0000 7fd00359e640 1 --2- 172.21.15.40:0/4118038731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0040d1600 0x7fd0040cf2a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcffc005fd0 tx=0x7fcffc02c0e0 comp rx=0 tx=0).stop 2023-10-18T08:55:16.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.464+0000 7fd00359e640 1 -- 172.21.15.40:0/4118038731 shutdown_connections 2023-10-18T08:55:16.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.464+0000 7fd00359e640 1 --2- 172.21.15.40:0/4118038731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0040d1600 0x7fd0040cf2a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:16.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.464+0000 7fd00359e640 1 --2- 172.21.15.40:0/4118038731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd00415b400 0x7fd0040b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:16.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.464+0000 7fd00359e640 1 -- 172.21.15.40:0/4118038731 >> 172.21.15.40:0/4118038731 conn(0x7fd00408ad30 msgr2=0x7fd00409a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:16.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.464+0000 7fd00359e640 1 -- 172.21.15.40:0/4118038731 shutdown_connections 2023-10-18T08:55:16.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.464+0000 7fd00359e640 1 -- 172.21.15.40:0/4118038731 wait complete. 2023-10-18T08:55:16.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.466+0000 7fd00359e640 1 Processor -- start 2023-10-18T08:55:16.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.467+0000 7fd00359e640 1 -- start start 2023-10-18T08:55:16.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.468+0000 7fd00359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0040d1600 0x7fd0041e4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:16.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.468+0000 7fd00359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00415b400 0x7fd0041f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:16.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.468+0000 7fd00359e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd00415b7e0 0x7fd0041f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:16.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.468+0000 7fd00359e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040c4fa0 con 0x7fd00415b400 2023-10-18T08:55:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.468+0000 7fd00359e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0041e5130 con 0x7fd0040d1600 2023-10-18T08:55:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.468+0000 7fd00359e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0041e5410 con 0x7fd00415b7e0 2023-10-18T08:55:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.468+0000 7fcffbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00415b400 0x7fd0041f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.468+0000 7fcffbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00415b400 0x7fd0041f5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36924/0 (socket says 172.21.15.40:36924) 2023-10-18T08:55:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.468+0000 7fcffbfff640 1 -- 172.21.15.40:0/2274605263 learned_addr learned my addr 172.21.15.40:0/2274605263 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.469+0000 7fd00259c640 1 --2- 172.21.15.40:0/2274605263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0040d1600 0x7fd0041e4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.469+0000 7fd002d9d640 1 --2- 172.21.15.40:0/2274605263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd00415b7e0 0x7fd0041f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.469+0000 7fcffbfff640 1 -- 172.21.15.40:0/2274605263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd00415b7e0 msgr2=0x7fd0041f58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:16.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.469+0000 7fcffbfff640 1 --2- 172.21.15.40:0/2274605263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd00415b7e0 0x7fd0041f58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:16.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.469+0000 7fcffbfff640 1 -- 172.21.15.40:0/2274605263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0040d1600 msgr2=0x7fd0041e4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:16.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.469+0000 7fcffbfff640 1 --2- 172.21.15.40:0/2274605263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0040d1600 0x7fd0041e4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.469+0000 7fcffbfff640 1 -- 172.21.15.40:0/2274605263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcffc009840 con 0x7fd00415b400 2023-10-18T08:55:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.469+0000 7fd00259c640 1 --2- 172.21.15.40:0/2274605263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0040d1600 0x7fd0041e4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:55:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.469+0000 7fcffbfff640 1 --2- 172.21.15.40:0/2274605263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00415b400 0x7fd0041f5370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00c3c0 tx=0x7fcfec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.470+0000 7fcffaffd640 1 -- 172.21.15.40:0/2274605263 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0052d0 con 0x7fd00415b400 2023-10-18T08:55:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.470+0000 7fcffaffd640 1 -- 172.21.15.40:0/2274605263 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcfec0035f0 con 0x7fd00415b400 2023-10-18T08:55:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.470+0000 7fcffaffd640 1 -- 172.21.15.40:0/2274605263 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec003a00 con 0x7fd00415b400 2023-10-18T08:55:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.470+0000 7fd00359e640 1 -- 172.21.15.40:0/2274605263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0040c5260 con 0x7fd00415b400 2023-10-18T08:55:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.470+0000 7fd00359e640 1 -- 172.21.15.40:0/2274605263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0040cb380 con 0x7fd00415b400 2023-10-18T08:55:16.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.471+0000 7fd00359e640 1 -- 172.21.15.40:0/2274605263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd004071da0 con 0x7fd00415b400 2023-10-18T08:55:16.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.472+0000 7fcffaffd640 1 -- 172.21.15.40:0/2274605263 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcfec003b60 con 0x7fd00415b400 2023-10-18T08:55:16.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.473+0000 7fcffaffd640 1 --2- 172.21.15.40:0/2274605263 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfe406ae40 0x7fcfe406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:16.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.474+0000 7fcffaffd640 1 -- 172.21.15.40:0/2274605263 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfec088db0 con 0x7fd00415b400 2023-10-18T08:55:16.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.474+0000 7fd00259c640 1 --2- 172.21.15.40:0/2274605263 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfe406ae40 0x7fcfe406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:16.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.475+0000 7fd00259c640 1 --2- 172.21.15.40:0/2274605263 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfe406ae40 0x7fcfe406d300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcff4005fd0 tx=0x7fcff4009260 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:16.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.478+0000 7fcffaffd640 1 -- 172.21.15.40:0/2274605263 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcfec08d070 con 0x7fd00415b400 2023-10-18T08:55:16.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:16.712+0000 7fd00359e640 1 -- 172.21.15.40:0/2274605263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd004066e00 con 0x7fd00415b400 2023-10-18T08:55:17.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.307+0000 7fcffaffd640 1 -- 172.21.15.40:0/2274605263 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcfec0170b0 con 0x7fd00415b400 2023-10-18T08:55:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.311+0000 7fd00359e640 1 -- 172.21.15.40:0/2274605263 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfe406ae40 msgr2=0x7fcfe406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.311+0000 7fd00359e640 1 --2- 172.21.15.40:0/2274605263 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfe406ae40 0x7fcfe406d300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcff4005fd0 tx=0x7fcff4009260 comp rx=0 tx=0).stop 2023-10-18T08:55:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.311+0000 7fd00359e640 1 -- 172.21.15.40:0/2274605263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00415b400 msgr2=0x7fd0041f5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:17.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.311+0000 7fd00359e640 1 --2- 172.21.15.40:0/2274605263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00415b400 0x7fd0041f5370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00c3c0 tx=0x7fcfec00c880 comp rx=0 tx=0).stop 2023-10-18T08:55:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.311+0000 7fd00359e640 1 -- 172.21.15.40:0/2274605263 shutdown_connections 2023-10-18T08:55:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.311+0000 7fd00359e640 1 --2- 172.21.15.40:0/2274605263 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfe406ae40 0x7fcfe406d300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.311+0000 7fd00359e640 1 --2- 172.21.15.40:0/2274605263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd00415b7e0 0x7fd0041f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.311+0000 7fd00359e640 1 --2- 172.21.15.40:0/2274605263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00415b400 0x7fd0041f5370 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:17.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.311+0000 7fd00359e640 1 --2- 172.21.15.40:0/2274605263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0040d1600 0x7fd0041e4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:17.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.311+0000 7fd00359e640 1 -- 172.21.15.40:0/2274605263 >> 172.21.15.40:0/2274605263 conn(0x7fd00408ad30 msgr2=0x7fd00406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:17.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.312+0000 7fd00359e640 1 -- 172.21.15.40:0/2274605263 shutdown_connections 2023-10-18T08:55:17.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.312+0000 7fd00359e640 1 -- 172.21.15.40:0/2274605263 wait complete. 2023-10-18T08:55:17.336 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']} 2023-10-18T08:55:17.337 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T08:55:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.574+0000 7f4ca11f5640 1 -- 172.21.15.40:0/2882859291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c9c070d40 msgr2=0x7f4c9c0711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.574+0000 7f4ca11f5640 1 --2- 172.21.15.40:0/2882859291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c9c070d40 0x7f4c9c0711c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0cab20 tx=0x7f4c9002bdb0 comp rx=0 tx=0).stop 2023-10-18T08:55:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.575+0000 7f4ca11f5640 1 -- 172.21.15.40:0/2882859291 shutdown_connections 2023-10-18T08:55:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.575+0000 7f4ca11f5640 1 --2- 172.21.15.40:0/2882859291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c9c070d40 0x7f4c9c0711c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.575+0000 7f4ca11f5640 1 --2- 172.21.15.40:0/2882859291 >> v2:172.21.15.144:3300/0 conn(0x7f4c9c13ec50 0x7f4c9c13f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.575+0000 7f4ca11f5640 1 -- 172.21.15.40:0/2882859291 >> 172.21.15.40:0/2882859291 conn(0x7f4c9c069720 msgr2=0x7f4c9c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.575+0000 7f4ca11f5640 1 -- 172.21.15.40:0/2882859291 shutdown_connections 2023-10-18T08:55:17.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.575+0000 7f4ca11f5640 1 -- 172.21.15.40:0/2882859291 wait complete. 2023-10-18T08:55:17.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.576+0000 7f4ca11f5640 1 Processor -- start 2023-10-18T08:55:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.577+0000 7f4ca11f5640 1 -- start start 2023-10-18T08:55:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.578+0000 7f4ca11f5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9c070d40 0x7f4c9c1f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.578+0000 7f4ca11f5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c9c0cae80 0x7f4c9c0b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.578+0000 7f4ca11f5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c9c13ec50 0x7f4c9c1f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.578+0000 7f4ca11f5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0ac620 con 0x7f4c9c0cae80 2023-10-18T08:55:17.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.578+0000 7f4ca11f5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c1f6120 con 0x7f4c9c13ec50 2023-10-18T08:55:17.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.578+0000 7f4ca11f5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c1e6310 con 0x7f4c9c070d40 2023-10-18T08:55:17.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.578+0000 7f4c9a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c9c0cae80 0x7f4c9c0b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:17.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.578+0000 7f4c9a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c9c0cae80 0x7f4c9c0b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36942/0 (socket says 172.21.15.40:36942) 2023-10-18T08:55:17.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.578+0000 7f4c9a59c640 1 -- 172.21.15.40:0/2854682021 learned_addr learned my addr 172.21.15.40:0/2854682021 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:17.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.578+0000 7f4c9ad9d640 1 --2- 172.21.15.40:0/2854682021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9c070d40 0x7f4c9c1f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:17.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.578+0000 7f4c9b59e640 1 --2- 172.21.15.40:0/2854682021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c9c13ec50 0x7f4c9c1f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:17.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.578+0000 7f4c9a59c640 1 -- 172.21.15.40:0/2854682021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9c070d40 msgr2=0x7f4c9c1f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:17.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.578+0000 7f4c9a59c640 1 --2- 172.21.15.40:0/2854682021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9c070d40 0x7f4c9c1f5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:17.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.578+0000 7f4c9a59c640 1 -- 172.21.15.40:0/2854682021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c9c13ec50 msgr2=0x7f4c9c1f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:17.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.578+0000 7f4c9a59c640 1 --2- 172.21.15.40:0/2854682021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c9c13ec50 0x7f4c9c1f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:17.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.578+0000 7f4c9a59c640 1 -- 172.21.15.40:0/2854682021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c90009560 con 0x7f4c9c0cae80 2023-10-18T08:55:17.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.579+0000 7f4c9a59c640 1 --2- 172.21.15.40:0/2854682021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c9c0cae80 0x7f4c9c0b1330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009850 tx=0x7f4c9002dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.579+0000 7f4c837fe640 1 -- 172.21.15.40:0/2854682021 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90003800 con 0x7f4c9c0cae80 2023-10-18T08:55:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.579+0000 7f4c837fe640 1 -- 172.21.15.40:0/2854682021 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4c90003a50 con 0x7f4c9c0cae80 2023-10-18T08:55:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.579+0000 7f4c837fe640 1 -- 172.21.15.40:0/2854682021 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90003e20 con 0x7f4c9c0cae80 2023-10-18T08:55:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.579+0000 7f4ca11f5640 1 -- 172.21.15.40:0/2854682021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9c1e5700 con 0x7f4c9c0cae80 2023-10-18T08:55:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.579+0000 7f4ca11f5640 1 -- 172.21.15.40:0/2854682021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9c1e5cc0 con 0x7f4c9c0cae80 2023-10-18T08:55:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.581+0000 7f4ca11f5640 1 -- 172.21.15.40:0/2854682021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9c066fd0 con 0x7f4c9c0cae80 2023-10-18T08:55:17.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.581+0000 7f4c837fe640 1 -- 172.21.15.40:0/2854682021 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4c9003e450 con 0x7f4c9c0cae80 2023-10-18T08:55:17.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.583+0000 7f4c837fe640 1 --2- 172.21.15.40:0/2854682021 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4c8806ae70 0x7f4c8806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:17.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.583+0000 7f4c837fe640 1 -- 172.21.15.40:0/2854682021 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c900ac0b0 con 0x7f4c9c0cae80 2023-10-18T08:55:17.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.583+0000 7f4c9ad9d640 1 --2- 172.21.15.40:0/2854682021 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4c8806ae70 0x7f4c8806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:17.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.584+0000 7f4c9ad9d640 1 --2- 172.21.15.40:0/2854682021 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4c8806ae70 0x7f4c8806d330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c005fd0 tx=0x7f4c8c005e40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:17.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.587+0000 7f4c837fe640 1 -- 172.21.15.40:0/2854682021 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c90077270 con 0x7f4c9c0cae80 2023-10-18T08:55:17.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:17.817+0000 7f4ca11f5640 1 -- 172.21.15.40:0/2854682021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4c9c1e6450 con 0x7f4c9c0cae80 2023-10-18T08:55:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.311+0000 7f4c837fe640 1 -- 172.21.15.40:0/2854682021 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4c9007a890 con 0x7f4c9c0cae80 2023-10-18T08:55:18.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.315+0000 7f4ca11f5640 1 -- 172.21.15.40:0/2854682021 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4c8806ae70 msgr2=0x7f4c8806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:18.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.315+0000 7f4ca11f5640 1 --2- 172.21.15.40:0/2854682021 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4c8806ae70 0x7f4c8806d330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c005fd0 tx=0x7f4c8c005e40 comp rx=0 tx=0).stop 2023-10-18T08:55:18.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.315+0000 7f4ca11f5640 1 -- 172.21.15.40:0/2854682021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c9c0cae80 msgr2=0x7f4c9c0b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:18.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.315+0000 7f4ca11f5640 1 --2- 172.21.15.40:0/2854682021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c9c0cae80 0x7f4c9c0b1330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009850 tx=0x7f4c9002dd00 comp rx=0 tx=0).stop 2023-10-18T08:55:18.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.316+0000 7f4ca11f5640 1 -- 172.21.15.40:0/2854682021 shutdown_connections 2023-10-18T08:55:18.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.316+0000 7f4ca11f5640 1 --2- 172.21.15.40:0/2854682021 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4c8806ae70 0x7f4c8806d330 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:18.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.316+0000 7f4ca11f5640 1 --2- 172.21.15.40:0/2854682021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c9c13ec50 0x7f4c9c1f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:18.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.316+0000 7f4ca11f5640 1 --2- 172.21.15.40:0/2854682021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c9c0cae80 0x7f4c9c0b1330 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:18.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.316+0000 7f4ca11f5640 1 --2- 172.21.15.40:0/2854682021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9c070d40 0x7f4c9c1f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:18.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.316+0000 7f4ca11f5640 1 -- 172.21.15.40:0/2854682021 >> 172.21.15.40:0/2854682021 conn(0x7f4c9c069720 msgr2=0x7f4c9c13cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:18.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.316+0000 7f4ca11f5640 1 -- 172.21.15.40:0/2854682021 shutdown_connections 2023-10-18T08:55:18.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.316+0000 7f4ca11f5640 1 -- 172.21.15.40:0/2854682021 wait complete. 2023-10-18T08:55:18.339 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T08:55:18.340 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:55:18.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.555+0000 7f737670c640 1 -- 172.21.15.40:0/2358104742 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73700b03c0 msgr2=0x7f73700cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:18.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.555+0000 7f737670c640 1 --2- 172.21.15.40:0/2358104742 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73700b03c0 0x7f73700cf130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f736c005fd0 tx=0x7f736c02be20 comp rx=0 tx=0).stop 2023-10-18T08:55:18.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.556+0000 7f737670c640 1 -- 172.21.15.40:0/2358104742 shutdown_connections 2023-10-18T08:55:18.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.556+0000 7f737670c640 1 --2- 172.21.15.40:0/2358104742 >> v2:172.21.15.144:3300/0 conn(0x7f737015c640 0x7f73700d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:18.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.556+0000 7f737670c640 1 --2- 172.21.15.40:0/2358104742 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f73700b03c0 0x7f73700cf130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:18.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.556+0000 7f737670c640 1 -- 172.21.15.40:0/2358104742 >> 172.21.15.40:0/2358104742 conn(0x7f7370088fa0 msgr2=0x7f737009a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:18.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.556+0000 7f737670c640 1 -- 172.21.15.40:0/2358104742 shutdown_connections 2023-10-18T08:55:18.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.556+0000 7f737670c640 1 -- 172.21.15.40:0/2358104742 wait complete. 2023-10-18T08:55:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.557+0000 7f737670c640 1 Processor -- start 2023-10-18T08:55:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.558+0000 7f737670c640 1 -- start start 2023-10-18T08:55:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f737670c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73700b03c0 0x7f73700c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f737670c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73700d14e0 0x7f7370155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:18.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f737670c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f737015c640 0x7f737015ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:18.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f737670c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73700c63f0 con 0x7f73700b03c0 2023-10-18T08:55:18.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f737670c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73700c7d50 con 0x7f73700d14e0 2023-10-18T08:55:18.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f737670c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7370159030 con 0x7f737015c640 2023-10-18T08:55:18.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f737570a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73700b03c0 0x7f73700c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:18.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f737570a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73700b03c0 0x7f73700c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36950/0 (socket says 172.21.15.40:36950) 2023-10-18T08:55:18.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f737570a640 1 -- 172.21.15.40:0/127363932 learned_addr learned my addr 172.21.15.40:0/127363932 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:18.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f7374f09640 1 --2- 172.21.15.40:0/127363932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73700d14e0 0x7f7370155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:18.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f7375f0b640 1 --2- 172.21.15.40:0/127363932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f737015c640 0x7f737015ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:18.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f737570a640 1 -- 172.21.15.40:0/127363932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f737015c640 msgr2=0x7f737015ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:18.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f737570a640 1 --2- 172.21.15.40:0/127363932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f737015c640 0x7f737015ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:18.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f737570a640 1 -- 172.21.15.40:0/127363932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73700d14e0 msgr2=0x7f7370155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:18.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f737570a640 1 --2- 172.21.15.40:0/127363932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73700d14e0 0x7f7370155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:18.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f737570a640 1 -- 172.21.15.40:0/127363932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c009560 con 0x7f73700b03c0 2023-10-18T08:55:18.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f7374f09640 1 --2- 172.21.15.40:0/127363932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73700d14e0 0x7f7370155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:55:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.559+0000 7f7375f0b640 1 --2- 172.21.15.40:0/127363932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f737015c640 0x7f737015ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.560+0000 7f737570a640 1 --2- 172.21.15.40:0/127363932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73700b03c0 0x7f73700c7810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f736c006010 tx=0x7f736c02da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.560+0000 7f7365ffb640 1 -- 172.21.15.40:0/127363932 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c02de80 con 0x7f73700b03c0 2023-10-18T08:55:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.560+0000 7f7365ffb640 1 -- 172.21.15.40:0/127363932 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f736c0036e0 con 0x7f73700b03c0 2023-10-18T08:55:18.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.560+0000 7f7365ffb640 1 -- 172.21.15.40:0/127363932 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c003ad0 con 0x7f73700b03c0 2023-10-18T08:55:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.560+0000 7f737670c640 1 -- 172.21.15.40:0/127363932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73700c7fa0 con 0x7f73700b03c0 2023-10-18T08:55:18.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.560+0000 7f737670c640 1 -- 172.21.15.40:0/127363932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73701f5620 con 0x7f73700b03c0 2023-10-18T08:55:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.562+0000 7f737670c640 1 -- 172.21.15.40:0/127363932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7370073570 con 0x7f73700b03c0 2023-10-18T08:55:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.562+0000 7f7365ffb640 1 -- 172.21.15.40:0/127363932 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f736c003c30 con 0x7f73700b03c0 2023-10-18T08:55:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.563+0000 7f7365ffb640 1 --2- 172.21.15.40:0/127363932 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f735406ae40 0x7f735406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:18.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.563+0000 7f7365ffb640 1 -- 172.21.15.40:0/127363932 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f736c0ac310 con 0x7f73700b03c0 2023-10-18T08:55:18.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.564+0000 7f7374f09640 1 --2- 172.21.15.40:0/127363932 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f735406ae40 0x7f735406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:18.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.565+0000 7f7374f09640 1 --2- 172.21.15.40:0/127363932 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f735406ae40 0x7f735406d300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f73700b1590 tx=0x7f7360005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:18.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.569+0000 7f7365ffb640 1 -- 172.21.15.40:0/127363932 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f736c077440 con 0x7f73700b03c0 2023-10-18T08:55:18.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.802+0000 7f737670c640 1 -- 172.21.15.40:0/127363932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f737006bc00 con 0x7f73700b03c0 2023-10-18T08:55:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.803+0000 7f7365ffb640 1 -- 172.21.15.40:0/127363932 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f736c07aa60 con 0x7f73700b03c0 2023-10-18T08:55:18.806 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:18.806 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1962150323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1962150323},{"type":"v1","addr":"172.21.15.144:6837","nonce":1962150323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4329,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2589720838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2589720838},{"type":"v1","addr":"172.21.15.40:6833","nonce":2589720838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3031491371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3031491371},{"type":"v1","addr":"172.21.15.40:6835","nonce":3031491371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:55:14.753828+0000","modified":"2023-10-18T08:55:17.818729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/682820172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":682820172},{"type":"v1","addr":"172.21.15.144:6835","nonce":682820172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T08:55:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2023-10-18T08:55:18.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.806+0000 7f737670c640 1 -- 172.21.15.40:0/127363932 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f735406ae40 msgr2=0x7f735406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:18.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.806+0000 7f737670c640 1 --2- 172.21.15.40:0/127363932 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f735406ae40 0x7f735406d300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f73700b1590 tx=0x7f7360005f90 comp rx=0 tx=0).stop 2023-10-18T08:55:18.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.806+0000 7f737670c640 1 -- 172.21.15.40:0/127363932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73700b03c0 msgr2=0x7f73700c7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:18.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.806+0000 7f737670c640 1 --2- 172.21.15.40:0/127363932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73700b03c0 0x7f73700c7810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f736c006010 tx=0x7f736c02da80 comp rx=0 tx=0).stop 2023-10-18T08:55:18.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.806+0000 7f737670c640 1 -- 172.21.15.40:0/127363932 shutdown_connections 2023-10-18T08:55:18.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.806+0000 7f737670c640 1 --2- 172.21.15.40:0/127363932 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f735406ae40 0x7f735406d300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:18.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.806+0000 7f737670c640 1 --2- 172.21.15.40:0/127363932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f737015c640 0x7f737015ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:18.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.806+0000 7f737670c640 1 --2- 172.21.15.40:0/127363932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73700d14e0 0x7f7370155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:18.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.806+0000 7f737670c640 1 --2- 172.21.15.40:0/127363932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73700b03c0 0x7f73700c7810 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:18.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.806+0000 7f737670c640 1 -- 172.21.15.40:0/127363932 >> 172.21.15.40:0/127363932 conn(0x7f7370088fa0 msgr2=0x7f737006a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:18.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.806+0000 7f737670c640 1 -- 172.21.15.40:0/127363932 shutdown_connections 2023-10-18T08:55:18.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:18.806+0000 7f737670c640 1 -- 172.21.15.40:0/127363932 wait complete. 2023-10-18T08:55:18.826 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:19.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.049+0000 7f35c0885640 1 -- 172.21.15.40:0/3129077803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a8003970 msgr2=0x7f35a8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:19.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.049+0000 7f35c0885640 1 --2- 172.21.15.40:0/3129077803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a8003970 0x7f35a8003d90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f35ac005fd0 tx=0x7f35a80057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.049+0000 7f35bbfff640 1 -- 172.21.15.40:0/3129077803 reap_dead start 2023-10-18T08:55:19.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.049+0000 7f35c0885640 1 -- 172.21.15.40:0/3129077803 shutdown_connections 2023-10-18T08:55:19.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.049+0000 7f35c0885640 1 -- 172.21.15.40:0/3129077803 >> 172.21.15.40:0/3129077803 conn(0x7f35bc069720 msgr2=0x7f35bc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:19.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.049+0000 7f35c0885640 1 -- 172.21.15.40:0/3129077803 shutdown_connections 2023-10-18T08:55:19.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.049+0000 7f35c0885640 1 -- 172.21.15.40:0/3129077803 wait complete. 2023-10-18T08:55:19.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.051+0000 7f35c0885640 1 Processor -- start 2023-10-18T08:55:19.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.051+0000 7f35c0885640 1 -- start start 2023-10-18T08:55:19.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.052+0000 7f35c0885640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35a8003970 0x7f35bc14c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:19.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.052+0000 7f35c0885640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35bc1597e0 0x7f35bc1545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:19.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.053+0000 7f35c0885640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35bc159c60 0x7f35bc154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:19.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.053+0000 7f35c0885640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc1e29a0 con 0x7f35bc159c60 2023-10-18T08:55:19.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.053+0000 7f35c0885640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc1e3fa0 con 0x7f35bc1597e0 2023-10-18T08:55:19.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.053+0000 7f35c0885640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc1e2bf0 con 0x7f35a8003970 2023-10-18T08:55:19.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.053+0000 7f35bbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35bc159c60 0x7f35bc154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:19.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.053+0000 7f35bbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35bc159c60 0x7f35bc154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36962/0 (socket says 172.21.15.40:36962) 2023-10-18T08:55:19.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.053+0000 7f35bbfff640 1 -- 172.21.15.40:0/2667874221 learned_addr learned my addr 172.21.15.40:0/2667874221 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:19.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.053+0000 7f35baffd640 1 --2- 172.21.15.40:0/2667874221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35bc1597e0 0x7f35bc1545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:19.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.053+0000 7f35bb7fe640 1 --2- 172.21.15.40:0/2667874221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35a8003970 0x7f35bc14c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:19.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.053+0000 7f35bbfff640 1 -- 172.21.15.40:0/2667874221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35a8003970 msgr2=0x7f35bc14c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:19.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.053+0000 7f35bbfff640 1 --2- 172.21.15.40:0/2667874221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35a8003970 0x7f35bc14c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.053+0000 7f35bbfff640 1 -- 172.21.15.40:0/2667874221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35bc1597e0 msgr2=0x7f35bc1545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:19.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.053+0000 7f35bbfff640 1 --2- 172.21.15.40:0/2667874221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35bc1597e0 0x7f35bc1545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.053+0000 7f35bbfff640 1 -- 172.21.15.40:0/2667874221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc1e2ed0 con 0x7f35bc159c60 2023-10-18T08:55:19.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.054+0000 7f35bbfff640 1 --2- 172.21.15.40:0/2667874221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35bc159c60 0x7f35bc154b30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f35b4000c00 tx=0x7f35b400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:19.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.054+0000 7f35b3fff640 1 -- 172.21.15.40:0/2667874221 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4005c90 con 0x7f35bc159c60 2023-10-18T08:55:19.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.054+0000 7f35b3fff640 1 -- 172.21.15.40:0/2667874221 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f35b4005e10 con 0x7f35bc159c60 2023-10-18T08:55:19.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.054+0000 7f35b3fff640 1 -- 172.21.15.40:0/2667874221 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4007330 con 0x7f35bc159c60 2023-10-18T08:55:19.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.054+0000 7f35c0885640 1 -- 172.21.15.40:0/2667874221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35bc1e31b0 con 0x7f35bc159c60 2023-10-18T08:55:19.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.054+0000 7f35c0885640 1 -- 172.21.15.40:0/2667874221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35bc14d020 con 0x7f35bc159c60 2023-10-18T08:55:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.055+0000 7f35c0885640 1 -- 172.21.15.40:0/2667874221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35bc1e4430 con 0x7f35bc159c60 2023-10-18T08:55:19.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.056+0000 7f35b3fff640 1 -- 172.21.15.40:0/2667874221 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f35b4007490 con 0x7f35bc159c60 2023-10-18T08:55:19.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.057+0000 7f35b3fff640 1 --2- 172.21.15.40:0/2667874221 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f359c06ae70 0x7f359c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:19.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.057+0000 7f35b3fff640 1 -- 172.21.15.40:0/2667874221 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35b40887e0 con 0x7f35bc159c60 2023-10-18T08:55:19.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.058+0000 7f35bb7fe640 1 --2- 172.21.15.40:0/2667874221 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f359c06ae70 0x7f359c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:19.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.059+0000 7f35bb7fe640 1 --2- 172.21.15.40:0/2667874221 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f359c06ae70 0x7f359c06d330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f35ac036000 tx=0x7f35ac035040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:19.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.061+0000 7f35b3fff640 1 -- 172.21.15.40:0/2667874221 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35bc1e4430 con 0x7f35bc159c60 2023-10-18T08:55:19.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.254+0000 7f35c0885640 1 -- 172.21.15.40:0/2667874221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35bc1e4430 con 0x7f35bc159c60 2023-10-18T08:55:19.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.257+0000 7f35b3fff640 1 -- 172.21.15.40:0/2667874221 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f35bc1e4430 con 0x7f35bc159c60 2023-10-18T08:55:19.259 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:19.260 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:16.275677+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:19.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.259+0000 7f35c0885640 1 -- 172.21.15.40:0/2667874221 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f359c06ae70 msgr2=0x7f359c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:19.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.259+0000 7f35c0885640 1 --2- 172.21.15.40:0/2667874221 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f359c06ae70 0x7f359c06d330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f35ac036000 tx=0x7f35ac035040 comp rx=0 tx=0).stop 2023-10-18T08:55:19.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.259+0000 7f35c0885640 1 -- 172.21.15.40:0/2667874221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35bc159c60 msgr2=0x7f35bc154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:19.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.259+0000 7f35c0885640 1 --2- 172.21.15.40:0/2667874221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35bc159c60 0x7f35bc154b30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f35b4000c00 tx=0x7f35b400c8e0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.260+0000 7f35c0885640 1 -- 172.21.15.40:0/2667874221 shutdown_connections 2023-10-18T08:55:19.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.260+0000 7f35c0885640 1 --2- 172.21.15.40:0/2667874221 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f359c06ae70 0x7f359c06d330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.260+0000 7f35c0885640 1 --2- 172.21.15.40:0/2667874221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35bc159c60 0x7f35bc154b30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.260+0000 7f35c0885640 1 --2- 172.21.15.40:0/2667874221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35bc1597e0 0x7f35bc1545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.260+0000 7f35c0885640 1 --2- 172.21.15.40:0/2667874221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35a8003970 0x7f35bc14c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.260+0000 7f35c0885640 1 -- 172.21.15.40:0/2667874221 >> 172.21.15.40:0/2667874221 conn(0x7f35bc069720 msgr2=0x7f35bc0aa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.260+0000 7f35c0885640 1 -- 172.21.15.40:0/2667874221 shutdown_connections 2023-10-18T08:55:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.260+0000 7f35c0885640 1 -- 172.21.15.40:0/2667874221 wait complete. 2023-10-18T08:55:19.282 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T08:55:19.282 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:55:19.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.511+0000 7fe555928640 1 -- 172.21.15.40:0/3500491271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5501420f0 msgr2=0x7fe5500b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:19.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.511+0000 7fe555928640 1 --2- 172.21.15.40:0/3500491271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5501420f0 0x7fe5500b0a80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe544009830 tx=0x7fe54402bdf0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.511+0000 7fe555928640 1 -- 172.21.15.40:0/3500491271 shutdown_connections 2023-10-18T08:55:19.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.511+0000 7fe555928640 1 --2- 172.21.15.40:0/3500491271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5501420f0 0x7fe5500b0a80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.511+0000 7fe555928640 1 -- 172.21.15.40:0/3500491271 >> 172.21.15.40:0/3500491271 conn(0x7fe550069720 msgr2=0x7fe550069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:19.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.511+0000 7fe555928640 1 -- 172.21.15.40:0/3500491271 shutdown_connections 2023-10-18T08:55:19.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.511+0000 7fe555928640 1 -- 172.21.15.40:0/3500491271 wait complete. 2023-10-18T08:55:19.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.513+0000 7fe555928640 1 Processor -- start 2023-10-18T08:55:19.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.513+0000 7fe555928640 1 -- start start 2023-10-18T08:55:19.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.514+0000 7fe555928640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5500ca160 0x7fe550159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:19.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.514+0000 7fe555928640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5501420f0 0x7fe550158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:19.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.514+0000 7fe555928640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5501425a0 0x7fe550140240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:19.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.514+0000 7fe555928640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5500a65a0 con 0x7fe5501420f0 2023-10-18T08:55:19.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.514+0000 7fe555928640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5501590d0 con 0x7fe5500ca160 2023-10-18T08:55:19.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.514+0000 7fe555928640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe550159fa0 con 0x7fe5501425a0 2023-10-18T08:55:19.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.514+0000 7fe54ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5501420f0 0x7fe550158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:19.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.515+0000 7fe54ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5501420f0 0x7fe550158b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36974/0 (socket says 172.21.15.40:36974) 2023-10-18T08:55:19.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.515+0000 7fe54ffff640 1 -- 172.21.15.40:0/3002268914 learned_addr learned my addr 172.21.15.40:0/3002268914 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:19.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.515+0000 7fe555127640 1 --2- 172.21.15.40:0/3002268914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5501425a0 0x7fe550140240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:19.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.515+0000 7fe54ffff640 1 -- 172.21.15.40:0/3002268914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5501425a0 msgr2=0x7fe550140240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:19.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.515+0000 7fe54ffff640 1 --2- 172.21.15.40:0/3002268914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5501425a0 0x7fe550140240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.515+0000 7fe54ffff640 1 -- 172.21.15.40:0/3002268914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5500ca160 msgr2=0x7fe550159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:19.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.515+0000 7fe54ffff640 1 --2- 172.21.15.40:0/3002268914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5500ca160 0x7fe550159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.515+0000 7fe54ffff640 1 -- 172.21.15.40:0/3002268914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe544009490 con 0x7fe5501420f0 2023-10-18T08:55:19.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.515+0000 7fe555127640 1 --2- 172.21.15.40:0/3002268914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5501425a0 0x7fe550140240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:19.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.515+0000 7fe54ffff640 1 --2- 172.21.15.40:0/3002268914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5501420f0 0x7fe550158b90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00c3c0 tx=0x7fe53c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:19.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.516+0000 7fe54d7fa640 1 -- 172.21.15.40:0/3002268914 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53c011790 con 0x7fe5501420f0 2023-10-18T08:55:19.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.516+0000 7fe54d7fa640 1 -- 172.21.15.40:0/3002268914 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe53c0119e0 con 0x7fe5501420f0 2023-10-18T08:55:19.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.516+0000 7fe555928640 1 -- 172.21.15.40:0/3002268914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55015a280 con 0x7fe5501420f0 2023-10-18T08:55:19.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.516+0000 7fe54d7fa640 1 -- 172.21.15.40:0/3002268914 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53c011e10 con 0x7fe5501420f0 2023-10-18T08:55:19.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.516+0000 7fe555928640 1 -- 172.21.15.40:0/3002268914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55015a840 con 0x7fe5501420f0 2023-10-18T08:55:19.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.517+0000 7fe555928640 1 -- 172.21.15.40:0/3002268914 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe550066e00 con 0x7fe5501420f0 2023-10-18T08:55:19.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.518+0000 7fe54d7fa640 1 -- 172.21.15.40:0/3002268914 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe53c01c460 con 0x7fe5501420f0 2023-10-18T08:55:19.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.519+0000 7fe54d7fa640 1 --2- 172.21.15.40:0/3002268914 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe53406ae40 0x7fe53406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:19.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.519+0000 7fe54d7fa640 1 -- 172.21.15.40:0/3002268914 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe53c0896e0 con 0x7fe5501420f0 2023-10-18T08:55:19.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.520+0000 7fe554926640 1 --2- 172.21.15.40:0/3002268914 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe53406ae40 0x7fe53406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:19.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.521+0000 7fe554926640 1 --2- 172.21.15.40:0/3002268914 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe53406ae40 0x7fe53406d300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe544036000 tx=0x7fe5440046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:19.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.524+0000 7fe54d7fa640 1 -- 172.21.15.40:0/3002268914 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe53c0546f0 con 0x7fe5501420f0 2023-10-18T08:55:19.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.717+0000 7fe555928640 1 -- 172.21.15.40:0/3002268914 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5500a66e0 con 0x7fe53406ae40 2023-10-18T08:55:19.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.741+0000 7fe54d7fa640 1 -- 172.21.15.40:0/3002268914 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251454 (secure 0 0 0) 0x7fe5500a66e0 con 0x7fe53406ae40 2023-10-18T08:55:19.745 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:19.748 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:55:19.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.746+0000 7fe555928640 1 -- 172.21.15.40:0/3002268914 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe53406ae40 msgr2=0x7fe53406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:19.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.746+0000 7fe555928640 1 --2- 172.21.15.40:0/3002268914 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe53406ae40 0x7fe53406d300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe544036000 tx=0x7fe5440046c0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.746+0000 7fe555928640 1 -- 172.21.15.40:0/3002268914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5501420f0 msgr2=0x7fe550158b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:19.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.746+0000 7fe555928640 1 --2- 172.21.15.40:0/3002268914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5501420f0 0x7fe550158b90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00c3c0 tx=0x7fe53c00c880 comp rx=0 tx=0).stop 2023-10-18T08:55:19.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.747+0000 7fe555928640 1 -- 172.21.15.40:0/3002268914 shutdown_connections 2023-10-18T08:55:19.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.747+0000 7fe555928640 1 --2- 172.21.15.40:0/3002268914 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe53406ae40 0x7fe53406d300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.747+0000 7fe555928640 1 --2- 172.21.15.40:0/3002268914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5501425a0 0x7fe550140240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.747+0000 7fe555928640 1 --2- 172.21.15.40:0/3002268914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5501420f0 0x7fe550158b90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.747+0000 7fe555928640 1 --2- 172.21.15.40:0/3002268914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5500ca160 0x7fe550159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:19.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.747+0000 7fe555928640 1 -- 172.21.15.40:0/3002268914 >> 172.21.15.40:0/3002268914 conn(0x7fe550069720 msgr2=0x7fe550072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:19.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.747+0000 7fe555928640 1 -- 172.21.15.40:0/3002268914 shutdown_connections 2023-10-18T08:55:19.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:19.747+0000 7fe555928640 1 -- 172.21.15.40:0/3002268914 wait complete. 2023-10-18T08:55:19.767 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-18T08:55:18.894334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":110,"acting":110,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1534,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560965,"statfs":{"total":1600000000,"available":1598428668,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.625914"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.283854+0000","last_change":"2023-10-18T08:55:14.283854+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.283854+0000","last_undegraded":"2023-10-18T08:55:14.283854+0000","last_fullsized":"2023-10-18T08:55:14.283854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.285218+0000","last_change":"2023-10-18T08:55:14.285218+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.285218+0000","last_undegraded":"2023-10-18T08:55:14.285218+0000","last_fullsized":"2023-10-18T08:55:14.285218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.287117+0000","last_change":"2023-10-18T08:55:14.287117+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.287117+0000","last_undegraded":"2023-10-18T08:55:14.287117+0000","last_fullsized":"2023-10-18T08:55:14.287117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.284309+0000","last_change":"2023-10-18T08:55:14.284309+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.284309+0000","last_undegraded":"2023-10-18T08:55:14.284309+0000","last_fullsized":"2023-10-18T08:55:14.284309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.288904+0000","last_change":"2023-10-18T08:55:14.288904+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.288904+0000","last_undegraded":"2023-10-18T08:55:14.288904+0000","last_fullsized":"2023-10-18T08:55:14.288904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.283719+0000","last_change":"2023-10-18T08:55:14.283719+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.283719+0000","last_undegraded":"2023-10-18T08:55:14.283719+0000","last_fullsized":"2023-10-18T08:55:14.283719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.285735+0000","last_change":"2023-10-18T08:55:14.285735+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.285735+0000","last_undegraded":"2023-10-18T08:55:14.285735+0000","last_fullsized":"2023-10-18T08:55:14.285735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.282795+0000","last_change":"2023-10-18T08:55:14.282795+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.282795+0000","last_undegraded":"2023-10-18T08:55:14.282795+0000","last_fullsized":"2023-10-18T08:55:14.282795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.282221+0000","last_change":"2023-10-18T08:55:14.282221+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.282221+0000","last_undegraded":"2023-10-18T08:55:14.282221+0000","last_fullsized":"2023-10-18T08:55:14.282221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.281281+0000","last_change":"2023-10-18T08:55:14.281281+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.281281+0000","last_undegraded":"2023-10-18T08:55:14.281281+0000","last_fullsized":"2023-10-18T08:55:14.281281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.290866+0000","last_change":"2023-10-18T08:55:14.290336+0000","last_active":"2023-10-18T08:55:14.290866+0000","last_peered":"2023-10-18T08:55:14.290866+0000","last_clean":"2023-10-18T08:55:14.290866+0000","last_became_active":"2023-10-18T08:55:14.290051+0000","last_became_peered":"2023-10-18T08:55:14.290051+0000","last_unstale":"2023-10-18T08:55:14.290866+0000","last_undegraded":"2023-10-18T08:55:14.290866+0000","last_fullsized":"2023-10-18T08:55:14.290866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:41.860553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.274511+0000","last_change":"2023-10-18T08:55:13.274511+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.274511+0000","last_undegraded":"2023-10-18T08:55:13.274511+0000","last_fullsized":"2023-10-18T08:55:13.274511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.293436+0000","last_change":"2023-10-18T08:55:14.293089+0000","last_active":"2023-10-18T08:55:14.293436+0000","last_peered":"2023-10-18T08:55:14.293436+0000","last_clean":"2023-10-18T08:55:14.293436+0000","last_became_active":"2023-10-18T08:55:14.292273+0000","last_became_peered":"2023-10-18T08:55:14.292273+0000","last_unstale":"2023-10-18T08:55:14.293436+0000","last_undegraded":"2023-10-18T08:55:14.293436+0000","last_fullsized":"2023-10-18T08:55:14.293436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:08:01.871452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.279973+0000","last_change":"2023-10-18T08:55:14.279973+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.279973+0000","last_undegraded":"2023-10-18T08:55:14.279973+0000","last_fullsized":"2023-10-18T08:55:14.279973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.279586+0000","last_change":"2023-10-18T08:55:13.279586+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.279586+0000","last_undegraded":"2023-10-18T08:55:13.279586+0000","last_fullsized":"2023-10-18T08:55:13.279586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.272896+0000","last_change":"2023-10-18T08:55:13.272896+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.272896+0000","last_undegraded":"2023-10-18T08:55:13.272896+0000","last_fullsized":"2023-10-18T08:55:13.272896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.277583+0000","last_change":"2023-10-18T08:55:13.277583+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.277583+0000","last_undegraded":"2023-10-18T08:55:13.277583+0000","last_fullsized":"2023-10-18T08:55:13.277583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.276345+0000","last_change":"2023-10-18T08:55:13.276345+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.276345+0000","last_undegraded":"2023-10-18T08:55:13.276345+0000","last_fullsized":"2023-10-18T08:55:13.276345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.274657+0000","last_change":"2023-10-18T08:55:13.274657+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.274657+0000","last_undegraded":"2023-10-18T08:55:13.274657+0000","last_fullsized":"2023-10-18T08:55:13.274657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.280707+0000","last_change":"2023-10-18T08:55:14.280707+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.280707+0000","last_undegraded":"2023-10-18T08:55:14.280707+0000","last_fullsized":"2023-10-18T08:55:14.280707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.271900+0000","last_change":"2023-10-18T08:55:13.271900+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.271900+0000","last_undegraded":"2023-10-18T08:55:13.271900+0000","last_fullsized":"2023-10-18T08:55:13.271900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.279889+0000","last_change":"2023-10-18T08:55:13.279889+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.279889+0000","last_undegraded":"2023-10-18T08:55:13.279889+0000","last_fullsized":"2023-10-18T08:55:13.279889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.282597+0000","last_change":"2023-10-18T08:55:14.282597+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.282597+0000","last_undegraded":"2023-10-18T08:55:14.282597+0000","last_fullsized":"2023-10-18T08:55:14.282597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275196+0000","last_change":"2023-10-18T08:55:13.275196+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275196+0000","last_undegraded":"2023-10-18T08:55:13.275196+0000","last_fullsized":"2023-10-18T08:55:13.275196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.279023+0000","last_change":"2023-10-18T08:55:13.279023+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.279023+0000","last_undegraded":"2023-10-18T08:55:13.279023+0000","last_fullsized":"2023-10-18T08:55:13.279023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.286371+0000","last_change":"2023-10-18T08:55:14.286371+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.286371+0000","last_undegraded":"2023-10-18T08:55:14.286371+0000","last_fullsized":"2023-10-18T08:55:14.286371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.271644+0000","last_change":"2023-10-18T08:55:13.271644+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.271644+0000","last_undegraded":"2023-10-18T08:55:13.271644+0000","last_fullsized":"2023-10-18T08:55:13.271644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.281780+0000","last_change":"2023-10-18T08:55:14.281780+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.281780+0000","last_undegraded":"2023-10-18T08:55:14.281780+0000","last_fullsized":"2023-10-18T08:55:14.281780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.278604+0000","last_change":"2023-10-18T08:55:13.278604+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.278604+0000","last_undegraded":"2023-10-18T08:55:13.278604+0000","last_fullsized":"2023-10-18T08:55:13.278604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.284994+0000","last_change":"2023-10-18T08:55:14.284994+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.284994+0000","last_undegraded":"2023-10-18T08:55:14.284994+0000","last_fullsized":"2023-10-18T08:55:14.284994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275071+0000","last_change":"2023-10-18T08:55:13.275071+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275071+0000","last_undegraded":"2023-10-18T08:55:13.275071+0000","last_fullsized":"2023-10-18T08:55:13.275071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.278652+0000","last_change":"2023-10-18T08:55:13.278652+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.278652+0000","last_undegraded":"2023-10-18T08:55:13.278652+0000","last_fullsized":"2023-10-18T08:55:13.278652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.282939+0000","last_change":"2023-10-18T08:55:14.282939+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.282939+0000","last_undegraded":"2023-10-18T08:55:14.282939+0000","last_fullsized":"2023-10-18T08:55:14.282939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-10-18T08:55:13.278928+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:55:13.278928+0000","last_peered":"2023-10-18T08:55:13.278928+0000","last_clean":"2023-10-18T08:55:13.278928+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:55:13.278928+0000","last_undegraded":"2023-10-18T08:55:13.278928+0000","last_fullsized":"2023-10-18T08:55:13.278928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:34.881744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.276534+0000","last_change":"2023-10-18T08:55:13.276534+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.276534+0000","last_undegraded":"2023-10-18T08:55:13.276534+0000","last_fullsized":"2023-10-18T08:55:13.276534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.280345+0000","last_change":"2023-10-18T08:55:14.280345+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.280345+0000","last_undegraded":"2023-10-18T08:55:14.280345+0000","last_fullsized":"2023-10-18T08:55:14.280345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.299595+0000","last_change":"2023-10-18T08:55:14.299430+0000","last_active":"2023-10-18T08:55:14.299595+0000","last_peered":"2023-10-18T08:55:14.299595+0000","last_clean":"2023-10-18T08:55:14.299595+0000","last_became_active":"2023-10-18T08:55:14.299004+0000","last_became_peered":"2023-10-18T08:55:14.299004+0000","last_unstale":"2023-10-18T08:55:14.299595+0000","last_undegraded":"2023-10-18T08:55:14.299595+0000","last_fullsized":"2023-10-18T08:55:14.299595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:06:39.141484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.278422+0000","last_change":"2023-10-18T08:55:13.278422+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.278422+0000","last_undegraded":"2023-10-18T08:55:13.278422+0000","last_fullsized":"2023-10-18T08:55:13.278422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.276507+0000","last_change":"2023-10-18T08:55:13.276507+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.276507+0000","last_undegraded":"2023-10-18T08:55:13.276507+0000","last_fullsized":"2023-10-18T08:55:13.276507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275063+0000","last_change":"2023-10-18T08:55:13.275063+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275063+0000","last_undegraded":"2023-10-18T08:55:13.275063+0000","last_fullsized":"2023-10-18T08:55:13.275063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.299558+0000","last_change":"2023-10-18T08:55:14.299281+0000","last_active":"2023-10-18T08:55:14.299558+0000","last_peered":"2023-10-18T08:55:14.299558+0000","last_clean":"2023-10-18T08:55:14.299558+0000","last_became_active":"2023-10-18T08:55:14.298698+0000","last_became_peered":"2023-10-18T08:55:14.298698+0000","last_unstale":"2023-10-18T08:55:14.299558+0000","last_undegraded":"2023-10-18T08:55:14.299558+0000","last_fullsized":"2023-10-18T08:55:14.299558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:16:56.668322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275194+0000","last_change":"2023-10-18T08:55:13.275194+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275194+0000","last_undegraded":"2023-10-18T08:55:13.275194+0000","last_fullsized":"2023-10-18T08:55:13.275194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.281833+0000","last_change":"2023-10-18T08:55:14.281833+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.281833+0000","last_undegraded":"2023-10-18T08:55:14.281833+0000","last_fullsized":"2023-10-18T08:55:14.281833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.273062+0000","last_change":"2023-10-18T08:55:13.273062+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.273062+0000","last_undegraded":"2023-10-18T08:55:13.273062+0000","last_fullsized":"2023-10-18T08:55:13.273062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.299579+0000","last_change":"2023-10-18T08:55:14.299395+0000","last_active":"2023-10-18T08:55:14.299579+0000","last_peered":"2023-10-18T08:55:14.299579+0000","last_clean":"2023-10-18T08:55:14.299579+0000","last_became_active":"2023-10-18T08:55:14.299130+0000","last_became_peered":"2023-10-18T08:55:14.299130+0000","last_unstale":"2023-10-18T08:55:14.299579+0000","last_undegraded":"2023-10-18T08:55:14.299579+0000","last_fullsized":"2023-10-18T08:55:14.299579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:02:57.867822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275906+0000","last_change":"2023-10-18T08:55:13.275906+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275906+0000","last_undegraded":"2023-10-18T08:55:13.275906+0000","last_fullsized":"2023-10-18T08:55:13.275906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.280662+0000","last_change":"2023-10-18T08:55:14.280662+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.280662+0000","last_undegraded":"2023-10-18T08:55:14.280662+0000","last_fullsized":"2023-10-18T08:55:14.280662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275062+0000","last_change":"2023-10-18T08:55:13.275062+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275062+0000","last_undegraded":"2023-10-18T08:55:13.275062+0000","last_fullsized":"2023-10-18T08:55:13.275062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.280357+0000","last_change":"2023-10-18T08:55:14.280357+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.280357+0000","last_undegraded":"2023-10-18T08:55:14.280357+0000","last_fullsized":"2023-10-18T08:55:14.280357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.292912+0000","last_change":"2023-10-18T08:55:14.292655+0000","last_active":"2023-10-18T08:55:14.292912+0000","last_peered":"2023-10-18T08:55:14.292912+0000","last_clean":"2023-10-18T08:55:14.292912+0000","last_became_active":"2023-10-18T08:55:14.292291+0000","last_became_peered":"2023-10-18T08:55:14.292291+0000","last_unstale":"2023-10-18T08:55:14.292912+0000","last_undegraded":"2023-10-18T08:55:14.292912+0000","last_fullsized":"2023-10-18T08:55:14.292912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:08:01.871452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.301172+0000","last_change":"2023-10-18T08:55:14.301102+0000","last_active":"2023-10-18T08:55:14.301172+0000","last_peered":"2023-10-18T08:55:14.301172+0000","last_clean":"2023-10-18T08:55:14.301172+0000","last_became_active":"2023-10-18T08:55:14.300855+0000","last_became_peered":"2023-10-18T08:55:14.300855+0000","last_unstale":"2023-10-18T08:55:14.301172+0000","last_undegraded":"2023-10-18T08:55:14.301172+0000","last_fullsized":"2023-10-18T08:55:14.301172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:41.860553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.294048+0000","last_change":"2023-10-18T08:55:14.293731+0000","last_active":"2023-10-18T08:55:14.294048+0000","last_peered":"2023-10-18T08:55:14.294048+0000","last_clean":"2023-10-18T08:55:14.294048+0000","last_became_active":"2023-10-18T08:55:14.293420+0000","last_became_peered":"2023-10-18T08:55:14.293420+0000","last_unstale":"2023-10-18T08:55:14.294048+0000","last_undegraded":"2023-10-18T08:55:14.294048+0000","last_fullsized":"2023-10-18T08:55:14.294048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:06:48.633699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.298685+0000","last_change":"2023-10-18T08:55:14.297189+0000","last_active":"2023-10-18T08:55:14.298685+0000","last_peered":"2023-10-18T08:55:14.298685+0000","last_clean":"2023-10-18T08:55:14.298685+0000","last_became_active":"2023-10-18T08:55:14.296794+0000","last_became_peered":"2023-10-18T08:55:14.296794+0000","last_unstale":"2023-10-18T08:55:14.298685+0000","last_undegraded":"2023-10-18T08:55:14.298685+0000","last_fullsized":"2023-10-18T08:55:14.298685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:10:34.964947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.279865+0000","last_change":"2023-10-18T08:55:14.279865+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.279865+0000","last_undegraded":"2023-10-18T08:55:14.279865+0000","last_fullsized":"2023-10-18T08:55:14.279865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.271895+0000","last_change":"2023-10-18T08:55:13.271895+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.271895+0000","last_undegraded":"2023-10-18T08:55:13.271895+0000","last_fullsized":"2023-10-18T08:55:13.271895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6639999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.506}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5129999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4630000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.708}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6830000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949631,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949631,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":620,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194692,"statfs":{"total":200000000,"available":199365080,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]}]}],"pool_statfs":[]}} 2023-10-18T08:55:19.773 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:55:20.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.049+0000 7f5a54cd7640 1 -- 172.21.15.40:0/1377758604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a501586a0 msgr2=0x7f5a500b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:20.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.049+0000 7f5a54cd7640 1 --2- 172.21.15.40:0/1377758604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a501586a0 0x7f5a500b0bc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5a5006a6d0 tx=0x7f5a3802bc50 comp rx=0 tx=0).stop 2023-10-18T08:55:20.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.049+0000 7f5a54cd7640 1 -- 172.21.15.40:0/1377758604 shutdown_connections 2023-10-18T08:55:20.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.049+0000 7f5a54cd7640 1 --2- 172.21.15.40:0/1377758604 >> v2:172.21.15.144:3300/0 conn(0x7f5a500b1100 0x7f5a501625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:20.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.049+0000 7f5a54cd7640 1 --2- 172.21.15.40:0/1377758604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a501586a0 0x7f5a500b0bc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:20.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.049+0000 7f5a54cd7640 1 -- 172.21.15.40:0/1377758604 >> 172.21.15.40:0/1377758604 conn(0x7f5a5006cee0 msgr2=0x7f5a50069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:20.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.049+0000 7f5a54cd7640 1 -- 172.21.15.40:0/1377758604 shutdown_connections 2023-10-18T08:55:20.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.049+0000 7f5a54cd7640 1 -- 172.21.15.40:0/1377758604 wait complete. 2023-10-18T08:55:20.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.051+0000 7f5a54cd7640 1 Processor -- start 2023-10-18T08:55:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.051+0000 7f5a54cd7640 1 -- start start 2023-10-18T08:55:20.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.052+0000 7f5a54cd7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a5006d710 0x7f5a500aa670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:20.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.052+0000 7f5a54cd7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a500b1100 0x7f5a500a8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:20.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.052+0000 7f5a54cd7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a501586a0 0x7f5a500abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:20.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.052+0000 7f5a54cd7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5014ed80 con 0x7f5a500b1100 2023-10-18T08:55:20.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.052+0000 7f5a54cd7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a500ac250 con 0x7f5a501586a0 2023-10-18T08:55:20.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.052+0000 7f5a54cd7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a501426b0 con 0x7f5a5006d710 2023-10-18T08:55:20.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.053+0000 7f5a47fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a500b1100 0x7f5a500a8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:20.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.053+0000 7f5a47fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a500b1100 0x7f5a500a8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36992/0 (socket says 172.21.15.40:36992) 2023-10-18T08:55:20.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.053+0000 7f5a47fff640 1 -- 172.21.15.40:0/224138427 learned_addr learned my addr 172.21.15.40:0/224138427 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:20.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.053+0000 7f5a4e59c640 1 --2- 172.21.15.40:0/224138427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a5006d710 0x7f5a500aa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:20.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.053+0000 7f5a4ed9d640 1 --2- 172.21.15.40:0/224138427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a501586a0 0x7f5a500abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.053+0000 7f5a47fff640 1 -- 172.21.15.40:0/224138427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a5006d710 msgr2=0x7f5a500aa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.053+0000 7f5a47fff640 1 --2- 172.21.15.40:0/224138427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a5006d710 0x7f5a500aa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.053+0000 7f5a47fff640 1 -- 172.21.15.40:0/224138427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a501586a0 msgr2=0x7f5a500abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:20.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.053+0000 7f5a47fff640 1 --2- 172.21.15.40:0/224138427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a501586a0 0x7f5a500abd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:20.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.053+0000 7f5a47fff640 1 -- 172.21.15.40:0/224138427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a38009490 con 0x7f5a500b1100 2023-10-18T08:55:20.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.053+0000 7f5a4e59c640 1 --2- 172.21.15.40:0/224138427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a5006d710 0x7f5a500aa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:55:20.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.053+0000 7f5a47fff640 1 --2- 172.21.15.40:0/224138427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a500b1100 0x7f5a500a8e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009920 tx=0x7f5a3802cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:20.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.054+0000 7f5a46ffd640 1 -- 172.21.15.40:0/224138427 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38003a00 con 0x7f5a500b1100 2023-10-18T08:55:20.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.054+0000 7f5a46ffd640 1 -- 172.21.15.40:0/224138427 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5a38003c50 con 0x7f5a500b1100 2023-10-18T08:55:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.054+0000 7f5a46ffd640 1 -- 172.21.15.40:0/224138427 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38005c90 con 0x7f5a500b1100 2023-10-18T08:55:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.054+0000 7f5a54cd7640 1 -- 172.21.15.40:0/224138427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a500aaca0 con 0x7f5a500b1100 2023-10-18T08:55:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.054+0000 7f5a54cd7640 1 -- 172.21.15.40:0/224138427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a500ab510 con 0x7f5a500b1100 2023-10-18T08:55:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.055+0000 7f5a54cd7640 1 -- 172.21.15.40:0/224138427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a50066fd0 con 0x7f5a500b1100 2023-10-18T08:55:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.056+0000 7f5a46ffd640 1 -- 172.21.15.40:0/224138427 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5a3803b040 con 0x7f5a500b1100 2023-10-18T08:55:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.057+0000 7f5a46ffd640 1 --2- 172.21.15.40:0/224138427 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a3006ae40 0x7f5a3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.057+0000 7f5a46ffd640 1 -- 172.21.15.40:0/224138427 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a380ab5d0 con 0x7f5a500b1100 2023-10-18T08:55:20.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.058+0000 7f5a4e59c640 1 --2- 172.21.15.40:0/224138427 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a3006ae40 0x7f5a3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:20.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.059+0000 7f5a4e59c640 1 --2- 172.21.15.40:0/224138427 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a3006ae40 0x7f5a3006d300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5a40001eb0 tx=0x7f5a40009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:20.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.062+0000 7f5a46ffd640 1 -- 172.21.15.40:0/224138427 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a38076670 con 0x7f5a500b1100 2023-10-18T08:55:20.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.253+0000 7f5a54cd7640 1 -- 172.21.15.40:0/224138427 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5a500cdaa0 con 0x7f5a3006ae40 2023-10-18T08:55:20.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.274+0000 7f5a46ffd640 1 -- 172.21.15.40:0/224138427 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251454 (secure 0 0 0) 0x7f5a4000e870 con 0x7f5a3006ae40 2023-10-18T08:55:20.281 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:20.283 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:55:20.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.280+0000 7f5a54cd7640 1 -- 172.21.15.40:0/224138427 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a3006ae40 msgr2=0x7f5a3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:20.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.280+0000 7f5a54cd7640 1 --2- 172.21.15.40:0/224138427 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a3006ae40 0x7f5a3006d300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5a40001eb0 tx=0x7f5a40009040 comp rx=0 tx=0).stop 2023-10-18T08:55:20.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.280+0000 7f5a54cd7640 1 -- 172.21.15.40:0/224138427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a500b1100 msgr2=0x7f5a500a8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:20.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.280+0000 7f5a54cd7640 1 --2- 172.21.15.40:0/224138427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a500b1100 0x7f5a500a8e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009920 tx=0x7f5a3802cdf0 comp rx=0 tx=0).stop 2023-10-18T08:55:20.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.281+0000 7f5a54cd7640 1 -- 172.21.15.40:0/224138427 shutdown_connections 2023-10-18T08:55:20.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.281+0000 7f5a54cd7640 1 --2- 172.21.15.40:0/224138427 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a3006ae40 0x7f5a3006d300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:20.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.281+0000 7f5a54cd7640 1 --2- 172.21.15.40:0/224138427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a501586a0 0x7f5a500abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:20.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.281+0000 7f5a54cd7640 1 --2- 172.21.15.40:0/224138427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a500b1100 0x7f5a500a8e80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:20.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.281+0000 7f5a54cd7640 1 --2- 172.21.15.40:0/224138427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a5006d710 0x7f5a500aa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:20.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.281+0000 7f5a54cd7640 1 -- 172.21.15.40:0/224138427 >> 172.21.15.40:0/224138427 conn(0x7f5a5006cee0 msgr2=0x7f5a50150870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:20.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.281+0000 7f5a54cd7640 1 -- 172.21.15.40:0/224138427 shutdown_connections 2023-10-18T08:55:20.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.281+0000 7f5a54cd7640 1 -- 172.21.15.40:0/224138427 wait complete. 2023-10-18T08:55:20.303 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-18T08:55:18.894334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":110,"acting":110,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1534,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560965,"statfs":{"total":1600000000,"available":1598428668,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.625914"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.283854+0000","last_change":"2023-10-18T08:55:14.283854+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.283854+0000","last_undegraded":"2023-10-18T08:55:14.283854+0000","last_fullsized":"2023-10-18T08:55:14.283854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.285218+0000","last_change":"2023-10-18T08:55:14.285218+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.285218+0000","last_undegraded":"2023-10-18T08:55:14.285218+0000","last_fullsized":"2023-10-18T08:55:14.285218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.287117+0000","last_change":"2023-10-18T08:55:14.287117+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.287117+0000","last_undegraded":"2023-10-18T08:55:14.287117+0000","last_fullsized":"2023-10-18T08:55:14.287117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.284309+0000","last_change":"2023-10-18T08:55:14.284309+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.284309+0000","last_undegraded":"2023-10-18T08:55:14.284309+0000","last_fullsized":"2023-10-18T08:55:14.284309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.288904+0000","last_change":"2023-10-18T08:55:14.288904+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.288904+0000","last_undegraded":"2023-10-18T08:55:14.288904+0000","last_fullsized":"2023-10-18T08:55:14.288904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.283719+0000","last_change":"2023-10-18T08:55:14.283719+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.283719+0000","last_undegraded":"2023-10-18T08:55:14.283719+0000","last_fullsized":"2023-10-18T08:55:14.283719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.285735+0000","last_change":"2023-10-18T08:55:14.285735+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.285735+0000","last_undegraded":"2023-10-18T08:55:14.285735+0000","last_fullsized":"2023-10-18T08:55:14.285735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.282795+0000","last_change":"2023-10-18T08:55:14.282795+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.282795+0000","last_undegraded":"2023-10-18T08:55:14.282795+0000","last_fullsized":"2023-10-18T08:55:14.282795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.282221+0000","last_change":"2023-10-18T08:55:14.282221+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.282221+0000","last_undegraded":"2023-10-18T08:55:14.282221+0000","last_fullsized":"2023-10-18T08:55:14.282221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.281281+0000","last_change":"2023-10-18T08:55:14.281281+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.281281+0000","last_undegraded":"2023-10-18T08:55:14.281281+0000","last_fullsized":"2023-10-18T08:55:14.281281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.290866+0000","last_change":"2023-10-18T08:55:14.290336+0000","last_active":"2023-10-18T08:55:14.290866+0000","last_peered":"2023-10-18T08:55:14.290866+0000","last_clean":"2023-10-18T08:55:14.290866+0000","last_became_active":"2023-10-18T08:55:14.290051+0000","last_became_peered":"2023-10-18T08:55:14.290051+0000","last_unstale":"2023-10-18T08:55:14.290866+0000","last_undegraded":"2023-10-18T08:55:14.290866+0000","last_fullsized":"2023-10-18T08:55:14.290866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:41.860553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.274511+0000","last_change":"2023-10-18T08:55:13.274511+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.274511+0000","last_undegraded":"2023-10-18T08:55:13.274511+0000","last_fullsized":"2023-10-18T08:55:13.274511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.293436+0000","last_change":"2023-10-18T08:55:14.293089+0000","last_active":"2023-10-18T08:55:14.293436+0000","last_peered":"2023-10-18T08:55:14.293436+0000","last_clean":"2023-10-18T08:55:14.293436+0000","last_became_active":"2023-10-18T08:55:14.292273+0000","last_became_peered":"2023-10-18T08:55:14.292273+0000","last_unstale":"2023-10-18T08:55:14.293436+0000","last_undegraded":"2023-10-18T08:55:14.293436+0000","last_fullsized":"2023-10-18T08:55:14.293436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:08:01.871452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.279973+0000","last_change":"2023-10-18T08:55:14.279973+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.279973+0000","last_undegraded":"2023-10-18T08:55:14.279973+0000","last_fullsized":"2023-10-18T08:55:14.279973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.279586+0000","last_change":"2023-10-18T08:55:13.279586+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.279586+0000","last_undegraded":"2023-10-18T08:55:13.279586+0000","last_fullsized":"2023-10-18T08:55:13.279586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.272896+0000","last_change":"2023-10-18T08:55:13.272896+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.272896+0000","last_undegraded":"2023-10-18T08:55:13.272896+0000","last_fullsized":"2023-10-18T08:55:13.272896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.277583+0000","last_change":"2023-10-18T08:55:13.277583+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.277583+0000","last_undegraded":"2023-10-18T08:55:13.277583+0000","last_fullsized":"2023-10-18T08:55:13.277583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.276345+0000","last_change":"2023-10-18T08:55:13.276345+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.276345+0000","last_undegraded":"2023-10-18T08:55:13.276345+0000","last_fullsized":"2023-10-18T08:55:13.276345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.274657+0000","last_change":"2023-10-18T08:55:13.274657+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.274657+0000","last_undegraded":"2023-10-18T08:55:13.274657+0000","last_fullsized":"2023-10-18T08:55:13.274657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.280707+0000","last_change":"2023-10-18T08:55:14.280707+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.280707+0000","last_undegraded":"2023-10-18T08:55:14.280707+0000","last_fullsized":"2023-10-18T08:55:14.280707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.271900+0000","last_change":"2023-10-18T08:55:13.271900+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.271900+0000","last_undegraded":"2023-10-18T08:55:13.271900+0000","last_fullsized":"2023-10-18T08:55:13.271900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.279889+0000","last_change":"2023-10-18T08:55:13.279889+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.279889+0000","last_undegraded":"2023-10-18T08:55:13.279889+0000","last_fullsized":"2023-10-18T08:55:13.279889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.282597+0000","last_change":"2023-10-18T08:55:14.282597+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.282597+0000","last_undegraded":"2023-10-18T08:55:14.282597+0000","last_fullsized":"2023-10-18T08:55:14.282597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275196+0000","last_change":"2023-10-18T08:55:13.275196+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275196+0000","last_undegraded":"2023-10-18T08:55:13.275196+0000","last_fullsized":"2023-10-18T08:55:13.275196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.279023+0000","last_change":"2023-10-18T08:55:13.279023+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.279023+0000","last_undegraded":"2023-10-18T08:55:13.279023+0000","last_fullsized":"2023-10-18T08:55:13.279023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.286371+0000","last_change":"2023-10-18T08:55:14.286371+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.286371+0000","last_undegraded":"2023-10-18T08:55:14.286371+0000","last_fullsized":"2023-10-18T08:55:14.286371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.271644+0000","last_change":"2023-10-18T08:55:13.271644+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.271644+0000","last_undegraded":"2023-10-18T08:55:13.271644+0000","last_fullsized":"2023-10-18T08:55:13.271644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.281780+0000","last_change":"2023-10-18T08:55:14.281780+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.281780+0000","last_undegraded":"2023-10-18T08:55:14.281780+0000","last_fullsized":"2023-10-18T08:55:14.281780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.278604+0000","last_change":"2023-10-18T08:55:13.278604+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.278604+0000","last_undegraded":"2023-10-18T08:55:13.278604+0000","last_fullsized":"2023-10-18T08:55:13.278604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.284994+0000","last_change":"2023-10-18T08:55:14.284994+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.284994+0000","last_undegraded":"2023-10-18T08:55:14.284994+0000","last_fullsized":"2023-10-18T08:55:14.284994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275071+0000","last_change":"2023-10-18T08:55:13.275071+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275071+0000","last_undegraded":"2023-10-18T08:55:13.275071+0000","last_fullsized":"2023-10-18T08:55:13.275071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.278652+0000","last_change":"2023-10-18T08:55:13.278652+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.278652+0000","last_undegraded":"2023-10-18T08:55:13.278652+0000","last_fullsized":"2023-10-18T08:55:13.278652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.282939+0000","last_change":"2023-10-18T08:55:14.282939+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.282939+0000","last_undegraded":"2023-10-18T08:55:14.282939+0000","last_fullsized":"2023-10-18T08:55:14.282939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-10-18T08:55:13.278928+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:55:13.278928+0000","last_peered":"2023-10-18T08:55:13.278928+0000","last_clean":"2023-10-18T08:55:13.278928+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:55:13.278928+0000","last_undegraded":"2023-10-18T08:55:13.278928+0000","last_fullsized":"2023-10-18T08:55:13.278928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:34.881744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.276534+0000","last_change":"2023-10-18T08:55:13.276534+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.276534+0000","last_undegraded":"2023-10-18T08:55:13.276534+0000","last_fullsized":"2023-10-18T08:55:13.276534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.280345+0000","last_change":"2023-10-18T08:55:14.280345+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.280345+0000","last_undegraded":"2023-10-18T08:55:14.280345+0000","last_fullsized":"2023-10-18T08:55:14.280345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.299595+0000","last_change":"2023-10-18T08:55:14.299430+0000","last_active":"2023-10-18T08:55:14.299595+0000","last_peered":"2023-10-18T08:55:14.299595+0000","last_clean":"2023-10-18T08:55:14.299595+0000","last_became_active":"2023-10-18T08:55:14.299004+0000","last_became_peered":"2023-10-18T08:55:14.299004+0000","last_unstale":"2023-10-18T08:55:14.299595+0000","last_undegraded":"2023-10-18T08:55:14.299595+0000","last_fullsized":"2023-10-18T08:55:14.299595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:06:39.141484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.278422+0000","last_change":"2023-10-18T08:55:13.278422+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.278422+0000","last_undegraded":"2023-10-18T08:55:13.278422+0000","last_fullsized":"2023-10-18T08:55:13.278422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.276507+0000","last_change":"2023-10-18T08:55:13.276507+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.276507+0000","last_undegraded":"2023-10-18T08:55:13.276507+0000","last_fullsized":"2023-10-18T08:55:13.276507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275063+0000","last_change":"2023-10-18T08:55:13.275063+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275063+0000","last_undegraded":"2023-10-18T08:55:13.275063+0000","last_fullsized":"2023-10-18T08:55:13.275063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.299558+0000","last_change":"2023-10-18T08:55:14.299281+0000","last_active":"2023-10-18T08:55:14.299558+0000","last_peered":"2023-10-18T08:55:14.299558+0000","last_clean":"2023-10-18T08:55:14.299558+0000","last_became_active":"2023-10-18T08:55:14.298698+0000","last_became_peered":"2023-10-18T08:55:14.298698+0000","last_unstale":"2023-10-18T08:55:14.299558+0000","last_undegraded":"2023-10-18T08:55:14.299558+0000","last_fullsized":"2023-10-18T08:55:14.299558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:16:56.668322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275194+0000","last_change":"2023-10-18T08:55:13.275194+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275194+0000","last_undegraded":"2023-10-18T08:55:13.275194+0000","last_fullsized":"2023-10-18T08:55:13.275194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.281833+0000","last_change":"2023-10-18T08:55:14.281833+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.281833+0000","last_undegraded":"2023-10-18T08:55:14.281833+0000","last_fullsized":"2023-10-18T08:55:14.281833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.273062+0000","last_change":"2023-10-18T08:55:13.273062+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.273062+0000","last_undegraded":"2023-10-18T08:55:13.273062+0000","last_fullsized":"2023-10-18T08:55:13.273062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.299579+0000","last_change":"2023-10-18T08:55:14.299395+0000","last_active":"2023-10-18T08:55:14.299579+0000","last_peered":"2023-10-18T08:55:14.299579+0000","last_clean":"2023-10-18T08:55:14.299579+0000","last_became_active":"2023-10-18T08:55:14.299130+0000","last_became_peered":"2023-10-18T08:55:14.299130+0000","last_unstale":"2023-10-18T08:55:14.299579+0000","last_undegraded":"2023-10-18T08:55:14.299579+0000","last_fullsized":"2023-10-18T08:55:14.299579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:02:57.867822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275906+0000","last_change":"2023-10-18T08:55:13.275906+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275906+0000","last_undegraded":"2023-10-18T08:55:13.275906+0000","last_fullsized":"2023-10-18T08:55:13.275906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.280662+0000","last_change":"2023-10-18T08:55:14.280662+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.280662+0000","last_undegraded":"2023-10-18T08:55:14.280662+0000","last_fullsized":"2023-10-18T08:55:14.280662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275062+0000","last_change":"2023-10-18T08:55:13.275062+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275062+0000","last_undegraded":"2023-10-18T08:55:13.275062+0000","last_fullsized":"2023-10-18T08:55:13.275062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.280357+0000","last_change":"2023-10-18T08:55:14.280357+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.280357+0000","last_undegraded":"2023-10-18T08:55:14.280357+0000","last_fullsized":"2023-10-18T08:55:14.280357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.292912+0000","last_change":"2023-10-18T08:55:14.292655+0000","last_active":"2023-10-18T08:55:14.292912+0000","last_peered":"2023-10-18T08:55:14.292912+0000","last_clean":"2023-10-18T08:55:14.292912+0000","last_became_active":"2023-10-18T08:55:14.292291+0000","last_became_peered":"2023-10-18T08:55:14.292291+0000","last_unstale":"2023-10-18T08:55:14.292912+0000","last_undegraded":"2023-10-18T08:55:14.292912+0000","last_fullsized":"2023-10-18T08:55:14.292912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:08:01.871452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.301172+0000","last_change":"2023-10-18T08:55:14.301102+0000","last_active":"2023-10-18T08:55:14.301172+0000","last_peered":"2023-10-18T08:55:14.301172+0000","last_clean":"2023-10-18T08:55:14.301172+0000","last_became_active":"2023-10-18T08:55:14.300855+0000","last_became_peered":"2023-10-18T08:55:14.300855+0000","last_unstale":"2023-10-18T08:55:14.301172+0000","last_undegraded":"2023-10-18T08:55:14.301172+0000","last_fullsized":"2023-10-18T08:55:14.301172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:41.860553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.294048+0000","last_change":"2023-10-18T08:55:14.293731+0000","last_active":"2023-10-18T08:55:14.294048+0000","last_peered":"2023-10-18T08:55:14.294048+0000","last_clean":"2023-10-18T08:55:14.294048+0000","last_became_active":"2023-10-18T08:55:14.293420+0000","last_became_peered":"2023-10-18T08:55:14.293420+0000","last_unstale":"2023-10-18T08:55:14.294048+0000","last_undegraded":"2023-10-18T08:55:14.294048+0000","last_fullsized":"2023-10-18T08:55:14.294048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:06:48.633699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.298685+0000","last_change":"2023-10-18T08:55:14.297189+0000","last_active":"2023-10-18T08:55:14.298685+0000","last_peered":"2023-10-18T08:55:14.298685+0000","last_clean":"2023-10-18T08:55:14.298685+0000","last_became_active":"2023-10-18T08:55:14.296794+0000","last_became_peered":"2023-10-18T08:55:14.296794+0000","last_unstale":"2023-10-18T08:55:14.298685+0000","last_undegraded":"2023-10-18T08:55:14.298685+0000","last_fullsized":"2023-10-18T08:55:14.298685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:10:34.964947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.279865+0000","last_change":"2023-10-18T08:55:14.279865+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.279865+0000","last_undegraded":"2023-10-18T08:55:14.279865+0000","last_fullsized":"2023-10-18T08:55:14.279865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.271895+0000","last_change":"2023-10-18T08:55:13.271895+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.271895+0000","last_undegraded":"2023-10-18T08:55:13.271895+0000","last_fullsized":"2023-10-18T08:55:13.271895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6639999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.506}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5129999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4630000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.708}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6830000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949631,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949631,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":620,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194692,"statfs":{"total":200000000,"available":199365080,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]}]}],"pool_statfs":[]}} 2023-10-18T08:55:20.309 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:55:20.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.581+0000 7f954587f640 1 -- 172.21.15.40:0/813883327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f952c004ab0 msgr2=0x7f952c004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:20.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.581+0000 7f954587f640 1 --2- 172.21.15.40:0/813883327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f952c004ab0 0x7f952c004ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9530001260 tx=0x7f952c006900 comp rx=0 tx=0).stop 2023-10-18T08:55:20.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.581+0000 7f954507e640 1 -- 172.21.15.40:0/813883327 reap_dead start 2023-10-18T08:55:20.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.581+0000 7f954587f640 1 -- 172.21.15.40:0/813883327 shutdown_connections 2023-10-18T08:55:20.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.581+0000 7f954587f640 1 -- 172.21.15.40:0/813883327 >> 172.21.15.40:0/813883327 conn(0x7f9540069720 msgr2=0x7f9540069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:20.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.581+0000 7f954587f640 1 -- 172.21.15.40:0/813883327 shutdown_connections 2023-10-18T08:55:20.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.581+0000 7f954587f640 1 -- 172.21.15.40:0/813883327 wait complete. 2023-10-18T08:55:20.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.584+0000 7f954587f640 1 Processor -- start 2023-10-18T08:55:20.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.584+0000 7f954587f640 1 -- start start 2023-10-18T08:55:20.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.585+0000 7f954587f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f952c004ab0 0x7f954015a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:20.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.585+0000 7f954587f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95401e2930 0x7f95400acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:20.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.585+0000 7f954587f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954013c770 0x7f95400cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:20.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.585+0000 7f954587f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954015a3c0 con 0x7f952c004ab0 2023-10-18T08:55:20.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.585+0000 7f954587f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95401597b0 con 0x7f95401e2930 2023-10-18T08:55:20.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.585+0000 7f954587f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9540159a90 con 0x7f954013c770 2023-10-18T08:55:20.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.585+0000 7f954487d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f952c004ab0 0x7f954015a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:20.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.586+0000 7f954487d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f952c004ab0 0x7f954015a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37000/0 (socket says 172.21.15.40:37000) 2023-10-18T08:55:20.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.586+0000 7f954487d640 1 -- 172.21.15.40:0/33908655 learned_addr learned my addr 172.21.15.40:0/33908655 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:20.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.586+0000 7f953ffff640 1 --2- 172.21.15.40:0/33908655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95401e2930 0x7f95400acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:20.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.586+0000 7f954507e640 1 --2- 172.21.15.40:0/33908655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954013c770 0x7f95400cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:20.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.586+0000 7f954487d640 1 -- 172.21.15.40:0/33908655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954013c770 msgr2=0x7f95400cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:20.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.586+0000 7f954487d640 1 --2- 172.21.15.40:0/33908655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954013c770 0x7f95400cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:20.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.586+0000 7f954487d640 1 -- 172.21.15.40:0/33908655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95401e2930 msgr2=0x7f95400acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:20.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.586+0000 7f954487d640 1 --2- 172.21.15.40:0/33908655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95401e2930 0x7f95400acae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:20.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.586+0000 7f954487d640 1 -- 172.21.15.40:0/33908655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9540159d70 con 0x7f952c004ab0 2023-10-18T08:55:20.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.586+0000 7f954487d640 1 --2- 172.21.15.40:0/33908655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f952c004ab0 0x7f954015a850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f953002e3d0 tx=0x7f9530006290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:20.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.587+0000 7f953d7fa640 1 -- 172.21.15.40:0/33908655 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953002ec70 con 0x7f952c004ab0 2023-10-18T08:55:20.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.587+0000 7f953d7fa640 1 -- 172.21.15.40:0/33908655 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f95300035f0 con 0x7f952c004ab0 2023-10-18T08:55:20.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.587+0000 7f953d7fa640 1 -- 172.21.15.40:0/33908655 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530003a00 con 0x7f952c004ab0 2023-10-18T08:55:20.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.587+0000 7f954587f640 1 -- 172.21.15.40:0/33908655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954015a3c0 con 0x7f952c004ab0 2023-10-18T08:55:20.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.587+0000 7f954587f640 1 -- 172.21.15.40:0/33908655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95400ad020 con 0x7f952c004ab0 2023-10-18T08:55:20.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.588+0000 7f954587f640 1 -- 172.21.15.40:0/33908655 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9540066e00 con 0x7f952c004ab0 2023-10-18T08:55:20.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.590+0000 7f953d7fa640 1 -- 172.21.15.40:0/33908655 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9530003b60 con 0x7f952c004ab0 2023-10-18T08:55:20.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.591+0000 7f953d7fa640 1 --2- 172.21.15.40:0/33908655 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f952006ae40 0x7f952006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:20.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.591+0000 7f953d7fa640 1 -- 172.21.15.40:0/33908655 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95300acb80 con 0x7f952c004ab0 2023-10-18T08:55:20.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.591+0000 7f953ffff640 1 --2- 172.21.15.40:0/33908655 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f952006ae40 0x7f952006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:20.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.592+0000 7f953ffff640 1 --2- 172.21.15.40:0/33908655 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f952006ae40 0x7f952006d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f95280011e0 tx=0x7f9528009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:20.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.595+0000 7f953d7fa640 1 -- 172.21.15.40:0/33908655 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9530077b90 con 0x7f952c004ab0 2023-10-18T08:55:20.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.785+0000 7f954587f640 1 -- 172.21.15.40:0/33908655 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f95400b2270 con 0x7f952006ae40 2023-10-18T08:55:20.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.804+0000 7f953d7fa640 1 -- 172.21.15.40:0/33908655 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251454 (secure 0 0 0) 0x7f9528001a20 con 0x7f952006ae40 2023-10-18T08:55:20.811 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:20.812 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:55:20.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.809+0000 7f954587f640 1 -- 172.21.15.40:0/33908655 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f952006ae40 msgr2=0x7f952006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:20.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.809+0000 7f954587f640 1 --2- 172.21.15.40:0/33908655 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f952006ae40 0x7f952006d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f95280011e0 tx=0x7f9528009040 comp rx=0 tx=0).stop 2023-10-18T08:55:20.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.809+0000 7f954587f640 1 -- 172.21.15.40:0/33908655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f952c004ab0 msgr2=0x7f954015a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:20.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.809+0000 7f954587f640 1 --2- 172.21.15.40:0/33908655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f952c004ab0 0x7f954015a850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f953002e3d0 tx=0x7f9530006290 comp rx=0 tx=0).stop 2023-10-18T08:55:20.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.809+0000 7f954587f640 1 -- 172.21.15.40:0/33908655 shutdown_connections 2023-10-18T08:55:20.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.809+0000 7f954587f640 1 --2- 172.21.15.40:0/33908655 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f952006ae40 0x7f952006d300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:20.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.809+0000 7f954587f640 1 --2- 172.21.15.40:0/33908655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954013c770 0x7f95400cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:20.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.809+0000 7f954587f640 1 --2- 172.21.15.40:0/33908655 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95401e2930 0x7f95400acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:20.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.809+0000 7f954587f640 1 --2- 172.21.15.40:0/33908655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f952c004ab0 0x7f954015a850 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:20.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.809+0000 7f954587f640 1 -- 172.21.15.40:0/33908655 >> 172.21.15.40:0/33908655 conn(0x7f9540069720 msgr2=0x7f9540069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:20.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.809+0000 7f954587f640 1 -- 172.21.15.40:0/33908655 shutdown_connections 2023-10-18T08:55:20.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:20.810+0000 7f954587f640 1 -- 172.21.15.40:0/33908655 wait complete. 2023-10-18T08:55:20.834 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-18T08:55:18.894334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":110,"acting":110,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1534,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560965,"statfs":{"total":1600000000,"available":1598428668,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.625914"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.283854+0000","last_change":"2023-10-18T08:55:14.283854+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.283854+0000","last_undegraded":"2023-10-18T08:55:14.283854+0000","last_fullsized":"2023-10-18T08:55:14.283854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.285218+0000","last_change":"2023-10-18T08:55:14.285218+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.285218+0000","last_undegraded":"2023-10-18T08:55:14.285218+0000","last_fullsized":"2023-10-18T08:55:14.285218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.287117+0000","last_change":"2023-10-18T08:55:14.287117+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.287117+0000","last_undegraded":"2023-10-18T08:55:14.287117+0000","last_fullsized":"2023-10-18T08:55:14.287117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.284309+0000","last_change":"2023-10-18T08:55:14.284309+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.284309+0000","last_undegraded":"2023-10-18T08:55:14.284309+0000","last_fullsized":"2023-10-18T08:55:14.284309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.288904+0000","last_change":"2023-10-18T08:55:14.288904+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.288904+0000","last_undegraded":"2023-10-18T08:55:14.288904+0000","last_fullsized":"2023-10-18T08:55:14.288904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.283719+0000","last_change":"2023-10-18T08:55:14.283719+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.283719+0000","last_undegraded":"2023-10-18T08:55:14.283719+0000","last_fullsized":"2023-10-18T08:55:14.283719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.285735+0000","last_change":"2023-10-18T08:55:14.285735+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.285735+0000","last_undegraded":"2023-10-18T08:55:14.285735+0000","last_fullsized":"2023-10-18T08:55:14.285735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.282795+0000","last_change":"2023-10-18T08:55:14.282795+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.282795+0000","last_undegraded":"2023-10-18T08:55:14.282795+0000","last_fullsized":"2023-10-18T08:55:14.282795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.282221+0000","last_change":"2023-10-18T08:55:14.282221+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.282221+0000","last_undegraded":"2023-10-18T08:55:14.282221+0000","last_fullsized":"2023-10-18T08:55:14.282221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.281281+0000","last_change":"2023-10-18T08:55:14.281281+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.281281+0000","last_undegraded":"2023-10-18T08:55:14.281281+0000","last_fullsized":"2023-10-18T08:55:14.281281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.290866+0000","last_change":"2023-10-18T08:55:14.290336+0000","last_active":"2023-10-18T08:55:14.290866+0000","last_peered":"2023-10-18T08:55:14.290866+0000","last_clean":"2023-10-18T08:55:14.290866+0000","last_became_active":"2023-10-18T08:55:14.290051+0000","last_became_peered":"2023-10-18T08:55:14.290051+0000","last_unstale":"2023-10-18T08:55:14.290866+0000","last_undegraded":"2023-10-18T08:55:14.290866+0000","last_fullsized":"2023-10-18T08:55:14.290866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:41.860553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.274511+0000","last_change":"2023-10-18T08:55:13.274511+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.274511+0000","last_undegraded":"2023-10-18T08:55:13.274511+0000","last_fullsized":"2023-10-18T08:55:13.274511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.293436+0000","last_change":"2023-10-18T08:55:14.293089+0000","last_active":"2023-10-18T08:55:14.293436+0000","last_peered":"2023-10-18T08:55:14.293436+0000","last_clean":"2023-10-18T08:55:14.293436+0000","last_became_active":"2023-10-18T08:55:14.292273+0000","last_became_peered":"2023-10-18T08:55:14.292273+0000","last_unstale":"2023-10-18T08:55:14.293436+0000","last_undegraded":"2023-10-18T08:55:14.293436+0000","last_fullsized":"2023-10-18T08:55:14.293436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:08:01.871452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.279973+0000","last_change":"2023-10-18T08:55:14.279973+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.279973+0000","last_undegraded":"2023-10-18T08:55:14.279973+0000","last_fullsized":"2023-10-18T08:55:14.279973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.279586+0000","last_change":"2023-10-18T08:55:13.279586+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.279586+0000","last_undegraded":"2023-10-18T08:55:13.279586+0000","last_fullsized":"2023-10-18T08:55:13.279586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.272896+0000","last_change":"2023-10-18T08:55:13.272896+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.272896+0000","last_undegraded":"2023-10-18T08:55:13.272896+0000","last_fullsized":"2023-10-18T08:55:13.272896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.277583+0000","last_change":"2023-10-18T08:55:13.277583+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.277583+0000","last_undegraded":"2023-10-18T08:55:13.277583+0000","last_fullsized":"2023-10-18T08:55:13.277583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.276345+0000","last_change":"2023-10-18T08:55:13.276345+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.276345+0000","last_undegraded":"2023-10-18T08:55:13.276345+0000","last_fullsized":"2023-10-18T08:55:13.276345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.274657+0000","last_change":"2023-10-18T08:55:13.274657+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.274657+0000","last_undegraded":"2023-10-18T08:55:13.274657+0000","last_fullsized":"2023-10-18T08:55:13.274657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.280707+0000","last_change":"2023-10-18T08:55:14.280707+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.280707+0000","last_undegraded":"2023-10-18T08:55:14.280707+0000","last_fullsized":"2023-10-18T08:55:14.280707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.271900+0000","last_change":"2023-10-18T08:55:13.271900+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.271900+0000","last_undegraded":"2023-10-18T08:55:13.271900+0000","last_fullsized":"2023-10-18T08:55:13.271900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.279889+0000","last_change":"2023-10-18T08:55:13.279889+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.279889+0000","last_undegraded":"2023-10-18T08:55:13.279889+0000","last_fullsized":"2023-10-18T08:55:13.279889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.282597+0000","last_change":"2023-10-18T08:55:14.282597+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.282597+0000","last_undegraded":"2023-10-18T08:55:14.282597+0000","last_fullsized":"2023-10-18T08:55:14.282597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275196+0000","last_change":"2023-10-18T08:55:13.275196+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275196+0000","last_undegraded":"2023-10-18T08:55:13.275196+0000","last_fullsized":"2023-10-18T08:55:13.275196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.279023+0000","last_change":"2023-10-18T08:55:13.279023+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.279023+0000","last_undegraded":"2023-10-18T08:55:13.279023+0000","last_fullsized":"2023-10-18T08:55:13.279023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.286371+0000","last_change":"2023-10-18T08:55:14.286371+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.286371+0000","last_undegraded":"2023-10-18T08:55:14.286371+0000","last_fullsized":"2023-10-18T08:55:14.286371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.271644+0000","last_change":"2023-10-18T08:55:13.271644+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.271644+0000","last_undegraded":"2023-10-18T08:55:13.271644+0000","last_fullsized":"2023-10-18T08:55:13.271644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.281780+0000","last_change":"2023-10-18T08:55:14.281780+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.281780+0000","last_undegraded":"2023-10-18T08:55:14.281780+0000","last_fullsized":"2023-10-18T08:55:14.281780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.278604+0000","last_change":"2023-10-18T08:55:13.278604+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.278604+0000","last_undegraded":"2023-10-18T08:55:13.278604+0000","last_fullsized":"2023-10-18T08:55:13.278604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.284994+0000","last_change":"2023-10-18T08:55:14.284994+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.284994+0000","last_undegraded":"2023-10-18T08:55:14.284994+0000","last_fullsized":"2023-10-18T08:55:14.284994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275071+0000","last_change":"2023-10-18T08:55:13.275071+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275071+0000","last_undegraded":"2023-10-18T08:55:13.275071+0000","last_fullsized":"2023-10-18T08:55:13.275071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.278652+0000","last_change":"2023-10-18T08:55:13.278652+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.278652+0000","last_undegraded":"2023-10-18T08:55:13.278652+0000","last_fullsized":"2023-10-18T08:55:13.278652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.282939+0000","last_change":"2023-10-18T08:55:14.282939+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.282939+0000","last_undegraded":"2023-10-18T08:55:14.282939+0000","last_fullsized":"2023-10-18T08:55:14.282939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-10-18T08:55:13.278928+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:55:13.278928+0000","last_peered":"2023-10-18T08:55:13.278928+0000","last_clean":"2023-10-18T08:55:13.278928+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:55:13.278928+0000","last_undegraded":"2023-10-18T08:55:13.278928+0000","last_fullsized":"2023-10-18T08:55:13.278928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:34.881744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.276534+0000","last_change":"2023-10-18T08:55:13.276534+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.276534+0000","last_undegraded":"2023-10-18T08:55:13.276534+0000","last_fullsized":"2023-10-18T08:55:13.276534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.280345+0000","last_change":"2023-10-18T08:55:14.280345+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.280345+0000","last_undegraded":"2023-10-18T08:55:14.280345+0000","last_fullsized":"2023-10-18T08:55:14.280345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.299595+0000","last_change":"2023-10-18T08:55:14.299430+0000","last_active":"2023-10-18T08:55:14.299595+0000","last_peered":"2023-10-18T08:55:14.299595+0000","last_clean":"2023-10-18T08:55:14.299595+0000","last_became_active":"2023-10-18T08:55:14.299004+0000","last_became_peered":"2023-10-18T08:55:14.299004+0000","last_unstale":"2023-10-18T08:55:14.299595+0000","last_undegraded":"2023-10-18T08:55:14.299595+0000","last_fullsized":"2023-10-18T08:55:14.299595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:06:39.141484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.278422+0000","last_change":"2023-10-18T08:55:13.278422+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.278422+0000","last_undegraded":"2023-10-18T08:55:13.278422+0000","last_fullsized":"2023-10-18T08:55:13.278422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.276507+0000","last_change":"2023-10-18T08:55:13.276507+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.276507+0000","last_undegraded":"2023-10-18T08:55:13.276507+0000","last_fullsized":"2023-10-18T08:55:13.276507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275063+0000","last_change":"2023-10-18T08:55:13.275063+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275063+0000","last_undegraded":"2023-10-18T08:55:13.275063+0000","last_fullsized":"2023-10-18T08:55:13.275063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.299558+0000","last_change":"2023-10-18T08:55:14.299281+0000","last_active":"2023-10-18T08:55:14.299558+0000","last_peered":"2023-10-18T08:55:14.299558+0000","last_clean":"2023-10-18T08:55:14.299558+0000","last_became_active":"2023-10-18T08:55:14.298698+0000","last_became_peered":"2023-10-18T08:55:14.298698+0000","last_unstale":"2023-10-18T08:55:14.299558+0000","last_undegraded":"2023-10-18T08:55:14.299558+0000","last_fullsized":"2023-10-18T08:55:14.299558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:16:56.668322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275194+0000","last_change":"2023-10-18T08:55:13.275194+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275194+0000","last_undegraded":"2023-10-18T08:55:13.275194+0000","last_fullsized":"2023-10-18T08:55:13.275194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.281833+0000","last_change":"2023-10-18T08:55:14.281833+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.281833+0000","last_undegraded":"2023-10-18T08:55:14.281833+0000","last_fullsized":"2023-10-18T08:55:14.281833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.273062+0000","last_change":"2023-10-18T08:55:13.273062+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.273062+0000","last_undegraded":"2023-10-18T08:55:13.273062+0000","last_fullsized":"2023-10-18T08:55:13.273062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.299579+0000","last_change":"2023-10-18T08:55:14.299395+0000","last_active":"2023-10-18T08:55:14.299579+0000","last_peered":"2023-10-18T08:55:14.299579+0000","last_clean":"2023-10-18T08:55:14.299579+0000","last_became_active":"2023-10-18T08:55:14.299130+0000","last_became_peered":"2023-10-18T08:55:14.299130+0000","last_unstale":"2023-10-18T08:55:14.299579+0000","last_undegraded":"2023-10-18T08:55:14.299579+0000","last_fullsized":"2023-10-18T08:55:14.299579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:02:57.867822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275906+0000","last_change":"2023-10-18T08:55:13.275906+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275906+0000","last_undegraded":"2023-10-18T08:55:13.275906+0000","last_fullsized":"2023-10-18T08:55:13.275906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.280662+0000","last_change":"2023-10-18T08:55:14.280662+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.280662+0000","last_undegraded":"2023-10-18T08:55:14.280662+0000","last_fullsized":"2023-10-18T08:55:14.280662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.275062+0000","last_change":"2023-10-18T08:55:13.275062+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.275062+0000","last_undegraded":"2023-10-18T08:55:13.275062+0000","last_fullsized":"2023-10-18T08:55:13.275062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.280357+0000","last_change":"2023-10-18T08:55:14.280357+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.280357+0000","last_undegraded":"2023-10-18T08:55:14.280357+0000","last_fullsized":"2023-10-18T08:55:14.280357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.292912+0000","last_change":"2023-10-18T08:55:14.292655+0000","last_active":"2023-10-18T08:55:14.292912+0000","last_peered":"2023-10-18T08:55:14.292912+0000","last_clean":"2023-10-18T08:55:14.292912+0000","last_became_active":"2023-10-18T08:55:14.292291+0000","last_became_peered":"2023-10-18T08:55:14.292291+0000","last_unstale":"2023-10-18T08:55:14.292912+0000","last_undegraded":"2023-10-18T08:55:14.292912+0000","last_fullsized":"2023-10-18T08:55:14.292912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:08:01.871452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.301172+0000","last_change":"2023-10-18T08:55:14.301102+0000","last_active":"2023-10-18T08:55:14.301172+0000","last_peered":"2023-10-18T08:55:14.301172+0000","last_clean":"2023-10-18T08:55:14.301172+0000","last_became_active":"2023-10-18T08:55:14.300855+0000","last_became_peered":"2023-10-18T08:55:14.300855+0000","last_unstale":"2023-10-18T08:55:14.301172+0000","last_undegraded":"2023-10-18T08:55:14.301172+0000","last_fullsized":"2023-10-18T08:55:14.301172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:41.860553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.294048+0000","last_change":"2023-10-18T08:55:14.293731+0000","last_active":"2023-10-18T08:55:14.294048+0000","last_peered":"2023-10-18T08:55:14.294048+0000","last_clean":"2023-10-18T08:55:14.294048+0000","last_became_active":"2023-10-18T08:55:14.293420+0000","last_became_peered":"2023-10-18T08:55:14.293420+0000","last_unstale":"2023-10-18T08:55:14.294048+0000","last_undegraded":"2023-10-18T08:55:14.294048+0000","last_fullsized":"2023-10-18T08:55:14.294048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:06:48.633699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:55:14.265731+0000","last_change":"2023-10-18T08:55:14.265731+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.265731+0000","last_undegraded":"2023-10-18T08:55:14.265731+0000","last_fullsized":"2023-10-18T08:55:14.265731+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-10-18T08:55:14.298685+0000","last_change":"2023-10-18T08:55:14.297189+0000","last_active":"2023-10-18T08:55:14.298685+0000","last_peered":"2023-10-18T08:55:14.298685+0000","last_clean":"2023-10-18T08:55:14.298685+0000","last_became_active":"2023-10-18T08:55:14.296794+0000","last_became_peered":"2023-10-18T08:55:14.296794+0000","last_unstale":"2023-10-18T08:55:14.298685+0000","last_undegraded":"2023-10-18T08:55:14.298685+0000","last_fullsized":"2023-10-18T08:55:14.298685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:10:34.964947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-10-18T08:55:14.279865+0000","last_change":"2023-10-18T08:55:14.279865+0000","last_active":"2023-10-18T08:55:14.265731+0000","last_peered":"2023-10-18T08:55:14.265731+0000","last_clean":"2023-10-18T08:55:14.265731+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:14.279865+0000","last_undegraded":"2023-10-18T08:55:14.279865+0000","last_fullsized":"2023-10-18T08:55:14.279865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-10-18T08:55:13.271895+0000","last_change":"2023-10-18T08:55:13.271895+0000","last_active":"2023-10-18T08:55:13.261024+0000","last_peered":"2023-10-18T08:55:13.261024+0000","last_clean":"2023-10-18T08:55:13.261024+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:55:13.271895+0000","last_undegraded":"2023-10-18T08:55:13.271895+0000","last_fullsized":"2023-10-18T08:55:13.271895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":44,"acting":44,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6639999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.506}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5129999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4630000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.708}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4650000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6830000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949631,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.246}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":49,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195263,"statfs":{"total":200000000,"available":199949631,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.212}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":620,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194692,"statfs":{"total":200000000,"available":199365080,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]}]}],"pool_statfs":[]}} 2023-10-18T08:55:20.839 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2023-10-18T08:55:20.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.283854+0000', 'last_change': '2023-10-18T08:55:14.283854+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.283854+0000', 'last_undegraded': '2023-10-18T08:55:14.283854+0000', 'last_fullsized': '2023-10-18T08:55:14.283854+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:55:20.840 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2023-10-18T08:55:20.840 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2023-10-18T08:55:20.841 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.841 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2023-10-18T08:55:20.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.842 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2023-10-18T08:55:20.842 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.285218+0000', 'last_change': '2023-10-18T08:55:14.285218+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.285218+0000', 'last_undegraded': '2023-10-18T08:55:14.285218+0000', 'last_fullsized': '2023-10-18T08:55:14.285218+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:55:20.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2023-10-18T08:55:20.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2023-10-18T08:55:20.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.287117+0000', 'last_change': '2023-10-18T08:55:14.287117+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.287117+0000', 'last_undegraded': '2023-10-18T08:55:14.287117+0000', 'last_fullsized': '2023-10-18T08:55:14.287117+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:55:20.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2023-10-18T08:55:20.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.284309+0000', 'last_change': '2023-10-18T08:55:14.284309+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.284309+0000', 'last_undegraded': '2023-10-18T08:55:14.284309+0000', 'last_fullsized': '2023-10-18T08:55:14.284309+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:55:20.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2023-10-18T08:55:20.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.288904+0000', 'last_change': '2023-10-18T08:55:14.288904+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.288904+0000', 'last_undegraded': '2023-10-18T08:55:14.288904+0000', 'last_fullsized': '2023-10-18T08:55:14.288904+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:55:20.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2023-10-18T08:55:20.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2023-10-18T08:55:20.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2023-10-18T08:55:20.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2023-10-18T08:55:20.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.283719+0000', 'last_change': '2023-10-18T08:55:14.283719+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.283719+0000', 'last_undegraded': '2023-10-18T08:55:14.283719+0000', 'last_fullsized': '2023-10-18T08:55:14.283719+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:55:20.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2023-10-18T08:55:20.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2023-10-18T08:55:20.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2023-10-18T08:55:20.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2023-10-18T08:55:20.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2023-10-18T08:55:20.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2023-10-18T08:55:20.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.285735+0000', 'last_change': '2023-10-18T08:55:14.285735+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.285735+0000', 'last_undegraded': '2023-10-18T08:55:14.285735+0000', 'last_fullsized': '2023-10-18T08:55:14.285735+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:55:20.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2023-10-18T08:55:20.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2023-10-18T08:55:20.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.854 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2023-10-18T08:55:20.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2023-10-18T08:55:20.855 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.282795+0000', 'last_change': '2023-10-18T08:55:14.282795+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.282795+0000', 'last_undegraded': '2023-10-18T08:55:14.282795+0000', 'last_fullsized': '2023-10-18T08:55:14.282795+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:55:20.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2023-10-18T08:55:20.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2023-10-18T08:55:20.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.282221+0000', 'last_change': '2023-10-18T08:55:14.282221+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.282221+0000', 'last_undegraded': '2023-10-18T08:55:14.282221+0000', 'last_fullsized': '2023-10-18T08:55:14.282221+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:55:20.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2023-10-18T08:55:20.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2023-10-18T08:55:20.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2023-10-18T08:55:20.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.281281+0000', 'last_change': '2023-10-18T08:55:14.281281+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.281281+0000', 'last_undegraded': '2023-10-18T08:55:14.281281+0000', 'last_fullsized': '2023-10-18T08:55:14.281281+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:55:20.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2023-10-18T08:55:20.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2023-10-18T08:55:20.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2023-10-18T08:55:20.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2023-10-18T08:55:20.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2023-10-18T08:55:20.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2023-10-18T08:55:20.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2023-10-18T08:55:20.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.274511+0000', 'last_change': '2023-10-18T08:55:13.274511+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.274511+0000', 'last_undegraded': '2023-10-18T08:55:13.274511+0000', 'last_fullsized': '2023-10-18T08:55:13.274511+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T08:55:20.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2023-10-18T08:55:20.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.279973+0000', 'last_change': '2023-10-18T08:55:14.279973+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.279973+0000', 'last_undegraded': '2023-10-18T08:55:14.279973+0000', 'last_fullsized': '2023-10-18T08:55:14.279973+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:55:20.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2023-10-18T08:55:20.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.279586+0000', 'last_change': '2023-10-18T08:55:13.279586+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.279586+0000', 'last_undegraded': '2023-10-18T08:55:13.279586+0000', 'last_fullsized': '2023-10-18T08:55:13.279586+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:55:20.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2023-10-18T08:55:20.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2023-10-18T08:55:20.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.272896+0000', 'last_change': '2023-10-18T08:55:13.272896+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.272896+0000', 'last_undegraded': '2023-10-18T08:55:13.272896+0000', 'last_fullsized': '2023-10-18T08:55:13.272896+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T08:55:20.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2023-10-18T08:55:20.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2023-10-18T08:55:20.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.277583+0000', 'last_change': '2023-10-18T08:55:13.277583+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.277583+0000', 'last_undegraded': '2023-10-18T08:55:13.277583+0000', 'last_fullsized': '2023-10-18T08:55:13.277583+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:55:20.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2023-10-18T08:55:20.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2023-10-18T08:55:20.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.276345+0000', 'last_change': '2023-10-18T08:55:13.276345+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.276345+0000', 'last_undegraded': '2023-10-18T08:55:13.276345+0000', 'last_fullsized': '2023-10-18T08:55:13.276345+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T08:55:20.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2023-10-18T08:55:20.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2023-10-18T08:55:20.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.274657+0000', 'last_change': '2023-10-18T08:55:13.274657+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.274657+0000', 'last_undegraded': '2023-10-18T08:55:13.274657+0000', 'last_fullsized': '2023-10-18T08:55:13.274657+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T08:55:20.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2023-10-18T08:55:20.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.280707+0000', 'last_change': '2023-10-18T08:55:14.280707+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.280707+0000', 'last_undegraded': '2023-10-18T08:55:14.280707+0000', 'last_fullsized': '2023-10-18T08:55:14.280707+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:55:20.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2023-10-18T08:55:20.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.271900+0000', 'last_change': '2023-10-18T08:55:13.271900+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.271900+0000', 'last_undegraded': '2023-10-18T08:55:13.271900+0000', 'last_fullsized': '2023-10-18T08:55:13.271900+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:55:20.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2023-10-18T08:55:20.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2023-10-18T08:55:20.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.279889+0000', 'last_change': '2023-10-18T08:55:13.279889+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.279889+0000', 'last_undegraded': '2023-10-18T08:55:13.279889+0000', 'last_fullsized': '2023-10-18T08:55:13.279889+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:55:20.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2023-10-18T08:55:20.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.282597+0000', 'last_change': '2023-10-18T08:55:14.282597+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.282597+0000', 'last_undegraded': '2023-10-18T08:55:14.282597+0000', 'last_fullsized': '2023-10-18T08:55:14.282597+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:55:20.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2023-10-18T08:55:20.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.275196+0000', 'last_change': '2023-10-18T08:55:13.275196+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.275196+0000', 'last_undegraded': '2023-10-18T08:55:13.275196+0000', 'last_fullsized': '2023-10-18T08:55:13.275196+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T08:55:20.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2023-10-18T08:55:20.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2023-10-18T08:55:20.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.279023+0000', 'last_change': '2023-10-18T08:55:13.279023+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.279023+0000', 'last_undegraded': '2023-10-18T08:55:13.279023+0000', 'last_fullsized': '2023-10-18T08:55:13.279023+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T08:55:20.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2023-10-18T08:55:20.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.286371+0000', 'last_change': '2023-10-18T08:55:14.286371+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.286371+0000', 'last_undegraded': '2023-10-18T08:55:14.286371+0000', 'last_fullsized': '2023-10-18T08:55:14.286371+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:55:20.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2023-10-18T08:55:20.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.271644+0000', 'last_change': '2023-10-18T08:55:13.271644+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.271644+0000', 'last_undegraded': '2023-10-18T08:55:13.271644+0000', 'last_fullsized': '2023-10-18T08:55:13.271644+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T08:55:20.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2023-10-18T08:55:20.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.281780+0000', 'last_change': '2023-10-18T08:55:14.281780+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.281780+0000', 'last_undegraded': '2023-10-18T08:55:14.281780+0000', 'last_fullsized': '2023-10-18T08:55:14.281780+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:55:20.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2023-10-18T08:55:20.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.278604+0000', 'last_change': '2023-10-18T08:55:13.278604+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.278604+0000', 'last_undegraded': '2023-10-18T08:55:13.278604+0000', 'last_fullsized': '2023-10-18T08:55:13.278604+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T08:55:20.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2023-10-18T08:55:20.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.284994+0000', 'last_change': '2023-10-18T08:55:14.284994+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.284994+0000', 'last_undegraded': '2023-10-18T08:55:14.284994+0000', 'last_fullsized': '2023-10-18T08:55:14.284994+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:55:20.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2023-10-18T08:55:20.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.275071+0000', 'last_change': '2023-10-18T08:55:13.275071+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.275071+0000', 'last_undegraded': '2023-10-18T08:55:13.275071+0000', 'last_fullsized': '2023-10-18T08:55:13.275071+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T08:55:20.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2023-10-18T08:55:20.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2023-10-18T08:55:20.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.278652+0000', 'last_change': '2023-10-18T08:55:13.278652+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.278652+0000', 'last_undegraded': '2023-10-18T08:55:13.278652+0000', 'last_fullsized': '2023-10-18T08:55:13.278652+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T08:55:20.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2023-10-18T08:55:20.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.282939+0000', 'last_change': '2023-10-18T08:55:14.282939+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.282939+0000', 'last_undegraded': '2023-10-18T08:55:14.282939+0000', 'last_fullsized': '2023-10-18T08:55:14.282939+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:55:20.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2023-10-18T08:55:20.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.276534+0000', 'last_change': '2023-10-18T08:55:13.276534+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.276534+0000', 'last_undegraded': '2023-10-18T08:55:13.276534+0000', 'last_fullsized': '2023-10-18T08:55:13.276534+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:55:20.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2023-10-18T08:55:20.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.280345+0000', 'last_change': '2023-10-18T08:55:14.280345+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.280345+0000', 'last_undegraded': '2023-10-18T08:55:14.280345+0000', 'last_fullsized': '2023-10-18T08:55:14.280345+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:55:20.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2023-10-18T08:55:20.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2023-10-18T08:55:20.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.278422+0000', 'last_change': '2023-10-18T08:55:13.278422+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.278422+0000', 'last_undegraded': '2023-10-18T08:55:13.278422+0000', 'last_fullsized': '2023-10-18T08:55:13.278422+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T08:55:20.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2023-10-18T08:55:20.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2023-10-18T08:55:20.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.276507+0000', 'last_change': '2023-10-18T08:55:13.276507+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.276507+0000', 'last_undegraded': '2023-10-18T08:55:13.276507+0000', 'last_fullsized': '2023-10-18T08:55:13.276507+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:55:20.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2023-10-18T08:55:20.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2023-10-18T08:55:20.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.275063+0000', 'last_change': '2023-10-18T08:55:13.275063+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.275063+0000', 'last_undegraded': '2023-10-18T08:55:13.275063+0000', 'last_fullsized': '2023-10-18T08:55:13.275063+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T08:55:20.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2023-10-18T08:55:20.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2023-10-18T08:55:20.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2023-10-18T08:55:20.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.275194+0000', 'last_change': '2023-10-18T08:55:13.275194+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.275194+0000', 'last_undegraded': '2023-10-18T08:55:13.275194+0000', 'last_fullsized': '2023-10-18T08:55:13.275194+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T08:55:20.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2023-10-18T08:55:20.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.281833+0000', 'last_change': '2023-10-18T08:55:14.281833+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.281833+0000', 'last_undegraded': '2023-10-18T08:55:14.281833+0000', 'last_fullsized': '2023-10-18T08:55:14.281833+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:55:20.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2023-10-18T08:55:20.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.273062+0000', 'last_change': '2023-10-18T08:55:13.273062+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.273062+0000', 'last_undegraded': '2023-10-18T08:55:13.273062+0000', 'last_fullsized': '2023-10-18T08:55:13.273062+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T08:55:20.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2023-10-18T08:55:20.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2023-10-18T08:55:20.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2023-10-18T08:55:20.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.275906+0000', 'last_change': '2023-10-18T08:55:13.275906+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.275906+0000', 'last_undegraded': '2023-10-18T08:55:13.275906+0000', 'last_fullsized': '2023-10-18T08:55:13.275906+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T08:55:20.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2023-10-18T08:55:20.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.280662+0000', 'last_change': '2023-10-18T08:55:14.280662+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.280662+0000', 'last_undegraded': '2023-10-18T08:55:14.280662+0000', 'last_fullsized': '2023-10-18T08:55:14.280662+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:55:20.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2023-10-18T08:55:20.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.275062+0000', 'last_change': '2023-10-18T08:55:13.275062+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.275062+0000', 'last_undegraded': '2023-10-18T08:55:13.275062+0000', 'last_fullsized': '2023-10-18T08:55:13.275062+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T08:55:20.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2023-10-18T08:55:20.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.280357+0000', 'last_change': '2023-10-18T08:55:14.280357+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.280357+0000', 'last_undegraded': '2023-10-18T08:55:14.280357+0000', 'last_fullsized': '2023-10-18T08:55:14.280357+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:55:20.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2023-10-18T08:55:20.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2023-10-18T08:55:20.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2023-10-18T08:55:20.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2023-10-18T08:55:20.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:55:14.265731+0000', 'last_change': '2023-10-18T08:55:14.265731+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.265731+0000', 'last_undegraded': '2023-10-18T08:55:14.265731+0000', 'last_fullsized': '2023-10-18T08:55:14.265731+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:55:20.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2023-10-18T08:55:20.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:14.279865+0000', 'last_change': '2023-10-18T08:55:14.279865+0000', 'last_active': '2023-10-18T08:55:14.265731+0000', 'last_peered': '2023-10-18T08:55:14.265731+0000', 'last_clean': '2023-10-18T08:55:14.265731+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:14.279865+0000', 'last_undegraded': '2023-10-18T08:55:14.279865+0000', 'last_fullsized': '2023-10-18T08:55:14.279865+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:14.265731+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:55:20.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2023-10-18T08:55:20.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:55:13.271895+0000', 'last_change': '2023-10-18T08:55:13.271895+0000', 'last_active': '2023-10-18T08:55:13.261024+0000', 'last_peered': '2023-10-18T08:55:13.261024+0000', 'last_clean': '2023-10-18T08:55:13.261024+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:55:13.271895+0000', 'last_undegraded': '2023-10-18T08:55:13.271895+0000', 'last_fullsized': '2023-10-18T08:55:13.271895+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'last_clean_scrub_stamp': '2023-10-18T08:55:13.261024+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:55:20.901 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T08:55:21.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.136+0000 7f65f7fff640 1 -- 172.21.15.40:0/454037455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65e4003970 msgr2=0x7f65e4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:21.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.136+0000 7f65f7fff640 1 --2- 172.21.15.40:0/454037455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65e4003970 0x7f65e4003d90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f65e8006fd0 tx=0x7f65e40057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.137+0000 7f65f7fff640 1 -- 172.21.15.40:0/454037455 shutdown_connections 2023-10-18T08:55:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.137+0000 7f65f7fff640 1 --2- 172.21.15.40:0/454037455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65e4003970 0x7f65e4003d90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.137+0000 7f65f7fff640 1 -- 172.21.15.40:0/454037455 >> 172.21.15.40:0/454037455 conn(0x7f65f80aed70 msgr2=0x7f65f80af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.137+0000 7f65f7fff640 1 -- 172.21.15.40:0/454037455 shutdown_connections 2023-10-18T08:55:21.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.137+0000 7f65f7fff640 1 -- 172.21.15.40:0/454037455 wait complete. 2023-10-18T08:55:21.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.139+0000 7f65f7fff640 1 Processor -- start 2023-10-18T08:55:21.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.139+0000 7f65f7fff640 1 -- start start 2023-10-18T08:55:21.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.140+0000 7f65f7fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65e4003970 0x7f65f81f2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:21.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.140+0000 7f65f7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65f81f21a0 0x7f65f80c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:21.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.140+0000 7f65f7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65f81e46d0 0x7f65f815ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:21.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.140+0000 7f65f7fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f81f2f80 con 0x7f65f81f21a0 2023-10-18T08:55:21.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.140+0000 7f65f7fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f81f3260 con 0x7f65f81e46d0 2023-10-18T08:55:21.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.140+0000 7f65f7fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f81f3540 con 0x7f65e4003970 2023-10-18T08:55:21.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.141+0000 7f65effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65f81f21a0 0x7f65f80c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:21.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.141+0000 7f65effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65f81f21a0 0x7f65f80c9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37026/0 (socket says 172.21.15.40:37026) 2023-10-18T08:55:21.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.141+0000 7f65effff640 1 -- 172.21.15.40:0/300235553 learned_addr learned my addr 172.21.15.40:0/300235553 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.141+0000 7f65f6ffd640 1 --2- 172.21.15.40:0/300235553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65e4003970 0x7f65f81f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.141+0000 7f65f77fe640 1 --2- 172.21.15.40:0/300235553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65f81e46d0 0x7f65f815ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.141+0000 7f65effff640 1 -- 172.21.15.40:0/300235553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65e4003970 msgr2=0x7f65f81f2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.141+0000 7f65effff640 1 --2- 172.21.15.40:0/300235553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65e4003970 0x7f65f81f2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.141+0000 7f65effff640 1 -- 172.21.15.40:0/300235553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65f81e46d0 msgr2=0x7f65f815ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:21.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.141+0000 7f65effff640 1 --2- 172.21.15.40:0/300235553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65f81e46d0 0x7f65f815ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:21.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.141+0000 7f65effff640 1 -- 172.21.15.40:0/300235553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f81f3820 con 0x7f65f81f21a0 2023-10-18T08:55:21.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.141+0000 7f65f6ffd640 1 --2- 172.21.15.40:0/300235553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65e4003970 0x7f65f81f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:55:21.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.141+0000 7f65effff640 1 --2- 172.21.15.40:0/300235553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65f81f21a0 0x7f65f80c9a40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f65e000d4b0 tx=0x7f65e000d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:21.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.142+0000 7f65ef7fe640 1 -- 172.21.15.40:0/300235553 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e0012630 con 0x7f65f81f21a0 2023-10-18T08:55:21.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.142+0000 7f65ef7fe640 1 -- 172.21.15.40:0/300235553 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f65e0012790 con 0x7f65f81f21a0 2023-10-18T08:55:21.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.142+0000 7f65f7fff640 1 -- 172.21.15.40:0/300235553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f81f3b00 con 0x7f65f81f21a0 2023-10-18T08:55:21.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.142+0000 7f65ef7fe640 1 -- 172.21.15.40:0/300235553 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e0012b60 con 0x7f65f81f21a0 2023-10-18T08:55:21.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.142+0000 7f65f7fff640 1 -- 172.21.15.40:0/300235553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f80c9f80 con 0x7f65f81f21a0 2023-10-18T08:55:21.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.143+0000 7f65f7fff640 1 -- 172.21.15.40:0/300235553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f8066e00 con 0x7f65f81f21a0 2023-10-18T08:55:21.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.144+0000 7f65ef7fe640 1 -- 172.21.15.40:0/300235553 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f65e001a3b0 con 0x7f65f81f21a0 2023-10-18T08:55:21.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.145+0000 7f65ef7fe640 1 --2- 172.21.15.40:0/300235553 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f65d806ae40 0x7f65d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:21.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.145+0000 7f65ef7fe640 1 -- 172.21.15.40:0/300235553 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65e0089510 con 0x7f65f81f21a0 2023-10-18T08:55:21.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.146+0000 7f65f6ffd640 1 --2- 172.21.15.40:0/300235553 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f65d806ae40 0x7f65d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:21.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.147+0000 7f65f6ffd640 1 --2- 172.21.15.40:0/300235553 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f65d806ae40 0x7f65d806d300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f65e8034000 tx=0x7f65e802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:21.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.150+0000 7f65ef7fe640 1 -- 172.21.15.40:0/300235553 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f65e0054520 con 0x7f65f81f21a0 2023-10-18T08:55:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.415+0000 7f65f7fff640 1 -- 172.21.15.40:0/300235553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f65f80ab190 con 0x7f65f81f21a0 2023-10-18T08:55:21.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.417+0000 7f65ef7fe640 1 -- 172.21.15.40:0/300235553 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1100 (secure 0 0 0) 0x7f65e0001120 con 0x7f65f81f21a0 2023-10-18T08:55:21.420 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:21.420 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":25,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1697619304,"num_in_osds":8,"osd_in_since":1697619300,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":45},{"state_name":"unknown","count":42},{"state_name":"active+clean","count":10}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":1571332,"bytes_avail":1598428668,"bytes_total":1600000000,"unknown_pgs_ratio":0.4329896867275238,"inactive_pgs_ratio":0.46391752362251282},"fsmap":{"epoch":7,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4384}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-10-18T08:54:55.882672+0000","services":{}},"progress_events":{}} 2023-10-18T08:55:21.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.419+0000 7f65f7fff640 1 -- 172.21.15.40:0/300235553 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f65d806ae40 msgr2=0x7f65d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:21.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.419+0000 7f65f7fff640 1 --2- 172.21.15.40:0/300235553 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f65d806ae40 0x7f65d806d300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f65e8034000 tx=0x7f65e802e000 comp rx=0 tx=0).stop 2023-10-18T08:55:21.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.419+0000 7f65f7fff640 1 -- 172.21.15.40:0/300235553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65f81f21a0 msgr2=0x7f65f80c9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:21.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.419+0000 7f65f7fff640 1 --2- 172.21.15.40:0/300235553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65f81f21a0 0x7f65f80c9a40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f65e000d4b0 tx=0x7f65e000d970 comp rx=0 tx=0).stop 2023-10-18T08:55:21.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.419+0000 7f65f7fff640 1 -- 172.21.15.40:0/300235553 shutdown_connections 2023-10-18T08:55:21.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.419+0000 7f65f7fff640 1 --2- 172.21.15.40:0/300235553 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f65d806ae40 0x7f65d806d300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:21.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.419+0000 7f65f7fff640 1 --2- 172.21.15.40:0/300235553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65f81e46d0 0x7f65f815ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:21.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.419+0000 7f65f7fff640 1 --2- 172.21.15.40:0/300235553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65f81f21a0 0x7f65f80c9a40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:21.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.419+0000 7f65f7fff640 1 --2- 172.21.15.40:0/300235553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65e4003970 0x7f65f81f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:21.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.419+0000 7f65f7fff640 1 -- 172.21.15.40:0/300235553 >> 172.21.15.40:0/300235553 conn(0x7f65f80aed70 msgr2=0x7f65f81415b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:21.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.419+0000 7f65f7fff640 1 -- 172.21.15.40:0/300235553 shutdown_connections 2023-10-18T08:55:21.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.420+0000 7f65f7fff640 1 -- 172.21.15.40:0/300235553 wait complete. 2023-10-18T08:55:21.440 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T08:55:21.441 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:55:21.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.663+0000 7f54f024b640 1 -- 172.21.15.40:0/2302606624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d4004d00 msgr2=0x7f54d4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.663+0000 7f54f024b640 1 --2- 172.21.15.40:0/2302606624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d4004d00 0x7f54d4005180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c610 tx=0x7f54d40057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.664+0000 7f54ee7e8640 1 -- 172.21.15.40:0/2302606624 reap_dead start 2023-10-18T08:55:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.664+0000 7f54f024b640 1 -- 172.21.15.40:0/2302606624 shutdown_connections 2023-10-18T08:55:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.664+0000 7f54f024b640 1 -- 172.21.15.40:0/2302606624 >> 172.21.15.40:0/2302606624 conn(0x7f54e806cee0 msgr2=0x7f54e8069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.664+0000 7f54f024b640 1 -- 172.21.15.40:0/2302606624 shutdown_connections 2023-10-18T08:55:21.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.664+0000 7f54f024b640 1 -- 172.21.15.40:0/2302606624 wait complete. 2023-10-18T08:55:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.666+0000 7f54f024b640 1 Processor -- start 2023-10-18T08:55:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.666+0000 7f54f024b640 1 -- start start 2023-10-18T08:55:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.667+0000 7f54f024b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d4004d00 0x7f54e813c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:21.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.667+0000 7f54f024b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54e80c9fc0 0x7f54e81f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:21.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.667+0000 7f54f024b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54e80ca440 0x7f54e8149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:21.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.667+0000 7f54f024b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e814bbf0 con 0x7f54d4004d00 2023-10-18T08:55:21.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.667+0000 7f54f024b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e81f5930 con 0x7f54e80ca440 2023-10-18T08:55:21.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.667+0000 7f54f024b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e81f5c10 con 0x7f54e80c9fc0 2023-10-18T08:55:21.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.667+0000 7f54edfe7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d4004d00 0x7f54e813c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:21.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.667+0000 7f54edfe7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d4004d00 0x7f54e813c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37044/0 (socket says 172.21.15.40:37044) 2023-10-18T08:55:21.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.667+0000 7f54edfe7640 1 -- 172.21.15.40:0/3231376822 learned_addr learned my addr 172.21.15.40:0/3231376822 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:21.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.667+0000 7f54ed7e6640 1 --2- 172.21.15.40:0/3231376822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54e80c9fc0 0x7f54e81f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:21.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.668+0000 7f54ee7e8640 1 --2- 172.21.15.40:0/3231376822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54e80ca440 0x7f54e8149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:21.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.668+0000 7f54edfe7640 1 -- 172.21.15.40:0/3231376822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54e80c9fc0 msgr2=0x7f54e81f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:21.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.668+0000 7f54edfe7640 1 --2- 172.21.15.40:0/3231376822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54e80c9fc0 0x7f54e81f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:21.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.668+0000 7f54edfe7640 1 -- 172.21.15.40:0/3231376822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54e80ca440 msgr2=0x7f54e8149700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:21.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.668+0000 7f54edfe7640 1 --2- 172.21.15.40:0/3231376822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54e80ca440 0x7f54e8149700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:21.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.668+0000 7f54edfe7640 1 -- 172.21.15.40:0/3231376822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e814b880 con 0x7f54d4004d00 2023-10-18T08:55:21.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.668+0000 7f54ee7e8640 1 --2- 172.21.15.40:0/3231376822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54e80ca440 0x7f54e8149700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:21.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.668+0000 7f54edfe7640 1 --2- 172.21.15.40:0/3231376822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d4004d00 0x7f54e813c8a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f54e4011a50 tx=0x7f54e4011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:21.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.668+0000 7f54de7fc640 1 -- 172.21.15.40:0/3231376822 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e400fcd0 con 0x7f54d4004d00 2023-10-18T08:55:21.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.668+0000 7f54de7fc640 1 -- 172.21.15.40:0/3231376822 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f54e4012450 con 0x7f54d4004d00 2023-10-18T08:55:21.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.668+0000 7f54de7fc640 1 -- 172.21.15.40:0/3231376822 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e4012840 con 0x7f54d4004d00 2023-10-18T08:55:21.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.668+0000 7f54f024b640 1 -- 172.21.15.40:0/3231376822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e814b570 con 0x7f54d4004d00 2023-10-18T08:55:21.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.669+0000 7f54f024b640 1 -- 172.21.15.40:0/3231376822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e81f53b0 con 0x7f54d4004d00 2023-10-18T08:55:21.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.670+0000 7f54f024b640 1 -- 172.21.15.40:0/3231376822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e8066fd0 con 0x7f54d4004d00 2023-10-18T08:55:21.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.671+0000 7f54de7fc640 1 -- 172.21.15.40:0/3231376822 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f54e40129a0 con 0x7f54d4004d00 2023-10-18T08:55:21.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.672+0000 7f54de7fc640 1 --2- 172.21.15.40:0/3231376822 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f54cc06ae40 0x7f54cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:21.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.672+0000 7f54de7fc640 1 -- 172.21.15.40:0/3231376822 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54e408a490 con 0x7f54d4004d00 2023-10-18T08:55:21.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.672+0000 7f54ed7e6640 1 --2- 172.21.15.40:0/3231376822 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f54cc06ae40 0x7f54cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:21.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.673+0000 7f54ed7e6640 1 --2- 172.21.15.40:0/3231376822 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f54cc06ae40 0x7f54cc06d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f54d8005d00 tx=0x7f54d8005c70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:21.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.676+0000 7f54de7fc640 1 -- 172.21.15.40:0/3231376822 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f54e40555f0 con 0x7f54d4004d00 2023-10-18T08:55:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.868+0000 7f54f024b640 1 -- 172.21.15.40:0/3231376822 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54e8151fb0 con 0x7f54cc06ae40 2023-10-18T08:55:21.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.889+0000 7f54de7fc640 1 -- 172.21.15.40:0/3231376822 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260059 (secure 0 0 0) 0x7f54d800c8e0 con 0x7f54cc06ae40 2023-10-18T08:55:21.896 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:21.897 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:55:21.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.895+0000 7f54f024b640 1 -- 172.21.15.40:0/3231376822 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f54cc06ae40 msgr2=0x7f54cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:21.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.895+0000 7f54f024b640 1 --2- 172.21.15.40:0/3231376822 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f54cc06ae40 0x7f54cc06d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f54d8005d00 tx=0x7f54d8005c70 comp rx=0 tx=0).stop 2023-10-18T08:55:21.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.895+0000 7f54f024b640 1 -- 172.21.15.40:0/3231376822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d4004d00 msgr2=0x7f54e813c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:21.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.895+0000 7f54f024b640 1 --2- 172.21.15.40:0/3231376822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d4004d00 0x7f54e813c8a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f54e4011a50 tx=0x7f54e4011f10 comp rx=0 tx=0).stop 2023-10-18T08:55:21.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.895+0000 7f54f024b640 1 -- 172.21.15.40:0/3231376822 shutdown_connections 2023-10-18T08:55:21.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.895+0000 7f54f024b640 1 --2- 172.21.15.40:0/3231376822 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f54cc06ae40 0x7f54cc06d300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:21.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.895+0000 7f54f024b640 1 --2- 172.21.15.40:0/3231376822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54e80ca440 0x7f54e8149700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:21.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.895+0000 7f54f024b640 1 --2- 172.21.15.40:0/3231376822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54e80c9fc0 0x7f54e81f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:21.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.895+0000 7f54f024b640 1 --2- 172.21.15.40:0/3231376822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54d4004d00 0x7f54e813c8a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:21.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.895+0000 7f54f024b640 1 -- 172.21.15.40:0/3231376822 >> 172.21.15.40:0/3231376822 conn(0x7f54e806cee0 msgr2=0x7f54e8150750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:21.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.895+0000 7f54f024b640 1 -- 172.21.15.40:0/3231376822 shutdown_connections 2023-10-18T08:55:21.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:21.895+0000 7f54f024b640 1 -- 172.21.15.40:0/3231376822 wait complete. 2023-10-18T08:55:21.918 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-18T08:55:20.894970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560792,"statfs":{"total":1600000000,"available":1598251597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.620215"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282778+0000","last_change":"2023-10-18T08:55:15.307275+0000","last_active":"2023-10-18T08:55:16.282778+0000","last_peered":"2023-10-18T08:55:16.282778+0000","last_clean":"2023-10-18T08:55:16.282778+0000","last_became_active":"2023-10-18T08:55:15.304759+0000","last_became_peered":"2023-10-18T08:55:15.304759+0000","last_unstale":"2023-10-18T08:55:16.282778+0000","last_undegraded":"2023-10-18T08:55:16.282778+0000","last_fullsized":"2023-10-18T08:55:16.282778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:41.755639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281449+0000","last_change":"2023-10-18T08:55:15.306465+0000","last_active":"2023-10-18T08:55:16.281449+0000","last_peered":"2023-10-18T08:55:16.281449+0000","last_clean":"2023-10-18T08:55:16.281449+0000","last_became_active":"2023-10-18T08:55:15.305909+0000","last_became_peered":"2023-10-18T08:55:15.305909+0000","last_unstale":"2023-10-18T08:55:16.281449+0000","last_undegraded":"2023-10-18T08:55:16.281449+0000","last_fullsized":"2023-10-18T08:55:16.281449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:02.173932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688851+0000","last_change":"2023-10-18T08:55:15.302350+0000","last_active":"2023-10-18T08:55:16.688851+0000","last_peered":"2023-10-18T08:55:16.688851+0000","last_clean":"2023-10-18T08:55:16.688851+0000","last_became_active":"2023-10-18T08:55:15.302090+0000","last_became_peered":"2023-10-18T08:55:15.302090+0000","last_unstale":"2023-10-18T08:55:16.688851+0000","last_undegraded":"2023-10-18T08:55:16.688851+0000","last_fullsized":"2023-10-18T08:55:16.688851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:19.879235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688836+0000","last_change":"2023-10-18T08:55:15.307679+0000","last_active":"2023-10-18T08:55:16.688836+0000","last_peered":"2023-10-18T08:55:16.688836+0000","last_clean":"2023-10-18T08:55:16.688836+0000","last_became_active":"2023-10-18T08:55:15.306999+0000","last_became_peered":"2023-10-18T08:55:15.306999+0000","last_unstale":"2023-10-18T08:55:16.688836+0000","last_undegraded":"2023-10-18T08:55:16.688836+0000","last_fullsized":"2023-10-18T08:55:16.688836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282830+0000","last_change":"2023-10-18T08:55:15.293552+0000","last_active":"2023-10-18T08:55:16.282830+0000","last_peered":"2023-10-18T08:55:16.282830+0000","last_clean":"2023-10-18T08:55:16.282830+0000","last_became_active":"2023-10-18T08:55:15.292895+0000","last_became_peered":"2023-10-18T08:55:15.292895+0000","last_unstale":"2023-10-18T08:55:16.282830+0000","last_undegraded":"2023-10-18T08:55:16.282830+0000","last_fullsized":"2023-10-18T08:55:16.282830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:07:58.310623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691708+0000","last_change":"2023-10-18T08:55:15.319351+0000","last_active":"2023-10-18T08:55:16.691708+0000","last_peered":"2023-10-18T08:55:16.691708+0000","last_clean":"2023-10-18T08:55:16.691708+0000","last_became_active":"2023-10-18T08:55:15.317663+0000","last_became_peered":"2023-10-18T08:55:15.317663+0000","last_unstale":"2023-10-18T08:55:16.691708+0000","last_undegraded":"2023-10-18T08:55:16.691708+0000","last_fullsized":"2023-10-18T08:55:16.691708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:10.917539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282757+0000","last_change":"2023-10-18T08:55:15.298227+0000","last_active":"2023-10-18T08:55:16.282757+0000","last_peered":"2023-10-18T08:55:16.282757+0000","last_clean":"2023-10-18T08:55:16.282757+0000","last_became_active":"2023-10-18T08:55:15.297630+0000","last_became_peered":"2023-10-18T08:55:15.297630+0000","last_unstale":"2023-10-18T08:55:16.282757+0000","last_undegraded":"2023-10-18T08:55:16.282757+0000","last_fullsized":"2023-10-18T08:55:16.282757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:04.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.279841+0000","last_change":"2023-10-18T08:55:15.307773+0000","last_active":"2023-10-18T08:55:16.279841+0000","last_peered":"2023-10-18T08:55:16.279841+0000","last_clean":"2023-10-18T08:55:16.279841+0000","last_became_active":"2023-10-18T08:55:15.307252+0000","last_became_peered":"2023-10-18T08:55:15.307252+0000","last_unstale":"2023-10-18T08:55:16.279841+0000","last_undegraded":"2023-10-18T08:55:16.279841+0000","last_fullsized":"2023-10-18T08:55:16.279841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:19.879235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282164+0000","last_change":"2023-10-18T08:55:15.308028+0000","last_active":"2023-10-18T08:55:16.282164+0000","last_peered":"2023-10-18T08:55:16.282164+0000","last_clean":"2023-10-18T08:55:16.282164+0000","last_became_active":"2023-10-18T08:55:15.307786+0000","last_became_peered":"2023-10-18T08:55:15.307786+0000","last_unstale":"2023-10-18T08:55:16.282164+0000","last_undegraded":"2023-10-18T08:55:16.282164+0000","last_fullsized":"2023-10-18T08:55:16.282164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:31.888781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691934+0000","last_change":"2023-10-18T08:55:15.307031+0000","last_active":"2023-10-18T08:55:16.691934+0000","last_peered":"2023-10-18T08:55:16.691934+0000","last_clean":"2023-10-18T08:55:16.691934+0000","last_became_active":"2023-10-18T08:55:15.306786+0000","last_became_peered":"2023-10-18T08:55:15.306786+0000","last_unstale":"2023-10-18T08:55:16.691934+0000","last_undegraded":"2023-10-18T08:55:16.691934+0000","last_fullsized":"2023-10-18T08:55:16.691934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:36.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689019+0000","last_change":"2023-10-18T08:55:15.321350+0000","last_active":"2023-10-18T08:55:16.689019+0000","last_peered":"2023-10-18T08:55:16.689019+0000","last_clean":"2023-10-18T08:55:16.689019+0000","last_became_active":"2023-10-18T08:55:15.321114+0000","last_became_peered":"2023-10-18T08:55:15.321114+0000","last_unstale":"2023-10-18T08:55:16.689019+0000","last_undegraded":"2023-10-18T08:55:16.689019+0000","last_fullsized":"2023-10-18T08:55:16.689019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:02.173932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689997+0000","last_change":"2023-10-18T08:55:15.307879+0000","last_active":"2023-10-18T08:55:16.689997+0000","last_peered":"2023-10-18T08:55:16.689997+0000","last_clean":"2023-10-18T08:55:16.689997+0000","last_became_active":"2023-10-18T08:55:15.307599+0000","last_became_peered":"2023-10-18T08:55:15.307599+0000","last_unstale":"2023-10-18T08:55:16.689997+0000","last_undegraded":"2023-10-18T08:55:16.689997+0000","last_fullsized":"2023-10-18T08:55:16.689997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:02.474966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280131+0000","last_change":"2023-10-18T08:55:15.306959+0000","last_active":"2023-10-18T08:55:16.280131+0000","last_peered":"2023-10-18T08:55:16.280131+0000","last_clean":"2023-10-18T08:55:16.280131+0000","last_became_active":"2023-10-18T08:55:15.306003+0000","last_became_peered":"2023-10-18T08:55:15.306003+0000","last_unstale":"2023-10-18T08:55:16.280131+0000","last_undegraded":"2023-10-18T08:55:16.280131+0000","last_fullsized":"2023-10-18T08:55:16.280131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:06.219128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688286+0000","last_change":"2023-10-18T08:55:15.302306+0000","last_active":"2023-10-18T08:55:16.688286+0000","last_peered":"2023-10-18T08:55:16.688286+0000","last_clean":"2023-10-18T08:55:16.688286+0000","last_became_active":"2023-10-18T08:55:15.302027+0000","last_became_peered":"2023-10-18T08:55:15.302027+0000","last_unstale":"2023-10-18T08:55:16.688286+0000","last_undegraded":"2023-10-18T08:55:16.688286+0000","last_fullsized":"2023-10-18T08:55:16.688286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:45.159195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282045+0000","last_change":"2023-10-18T08:55:15.316075+0000","last_active":"2023-10-18T08:55:16.282045+0000","last_peered":"2023-10-18T08:55:16.282045+0000","last_clean":"2023-10-18T08:55:16.282045+0000","last_became_active":"2023-10-18T08:55:15.315569+0000","last_became_peered":"2023-10-18T08:55:15.315569+0000","last_unstale":"2023-10-18T08:55:16.282045+0000","last_undegraded":"2023-10-18T08:55:16.282045+0000","last_fullsized":"2023-10-18T08:55:16.282045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691359+0000","last_change":"2023-10-18T08:55:15.320240+0000","last_active":"2023-10-18T08:55:16.691359+0000","last_peered":"2023-10-18T08:55:16.691359+0000","last_clean":"2023-10-18T08:55:16.691359+0000","last_became_active":"2023-10-18T08:55:15.319558+0000","last_became_peered":"2023-10-18T08:55:15.319558+0000","last_unstale":"2023-10-18T08:55:16.691359+0000","last_undegraded":"2023-10-18T08:55:16.691359+0000","last_fullsized":"2023-10-18T08:55:16.691359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:10.355168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690434+0000","last_change":"2023-10-18T08:55:15.315655+0000","last_active":"2023-10-18T08:55:16.690434+0000","last_peered":"2023-10-18T08:55:16.690434+0000","last_clean":"2023-10-18T08:55:16.690434+0000","last_became_active":"2023-10-18T08:55:15.311919+0000","last_became_peered":"2023-10-18T08:55:15.311919+0000","last_unstale":"2023-10-18T08:55:16.690434+0000","last_undegraded":"2023-10-18T08:55:16.690434+0000","last_fullsized":"2023-10-18T08:55:16.690434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:23:50.828764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.687786+0000","last_change":"2023-10-18T08:55:15.306321+0000","last_active":"2023-10-18T08:55:16.687786+0000","last_peered":"2023-10-18T08:55:16.687786+0000","last_clean":"2023-10-18T08:55:16.687786+0000","last_became_active":"2023-10-18T08:55:15.305696+0000","last_became_peered":"2023-10-18T08:55:15.305696+0000","last_unstale":"2023-10-18T08:55:16.687786+0000","last_undegraded":"2023-10-18T08:55:16.687786+0000","last_fullsized":"2023-10-18T08:55:16.687786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:05.190948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282965+0000","last_change":"2023-10-18T08:55:15.294264+0000","last_active":"2023-10-18T08:55:16.282965+0000","last_peered":"2023-10-18T08:55:16.282965+0000","last_clean":"2023-10-18T08:55:16.282965+0000","last_became_active":"2023-10-18T08:55:15.293654+0000","last_became_peered":"2023-10-18T08:55:15.293654+0000","last_unstale":"2023-10-18T08:55:16.282965+0000","last_undegraded":"2023-10-18T08:55:16.282965+0000","last_fullsized":"2023-10-18T08:55:16.282965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:26.310762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.692252+0000","last_change":"2023-10-18T08:55:15.315803+0000","last_active":"2023-10-18T08:55:16.692252+0000","last_peered":"2023-10-18T08:55:16.692252+0000","last_clean":"2023-10-18T08:55:16.692252+0000","last_became_active":"2023-10-18T08:55:15.315544+0000","last_became_peered":"2023-10-18T08:55:15.315544+0000","last_unstale":"2023-10-18T08:55:16.692252+0000","last_undegraded":"2023-10-18T08:55:16.692252+0000","last_fullsized":"2023-10-18T08:55:16.692252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689490+0000","last_change":"2023-10-18T08:55:15.308154+0000","last_active":"2023-10-18T08:55:16.689490+0000","last_peered":"2023-10-18T08:55:16.689490+0000","last_clean":"2023-10-18T08:55:16.689490+0000","last_became_active":"2023-10-18T08:55:15.307592+0000","last_became_peered":"2023-10-18T08:55:15.307592+0000","last_unstale":"2023-10-18T08:55:16.689490+0000","last_undegraded":"2023-10-18T08:55:16.689490+0000","last_fullsized":"2023-10-18T08:55:16.689490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:06.219128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691182+0000","last_change":"2023-10-18T08:55:15.312342+0000","last_active":"2023-10-18T08:55:16.691182+0000","last_peered":"2023-10-18T08:55:16.691182+0000","last_clean":"2023-10-18T08:55:16.691182+0000","last_became_active":"2023-10-18T08:55:15.311286+0000","last_became_peered":"2023-10-18T08:55:15.311286+0000","last_unstale":"2023-10-18T08:55:16.691182+0000","last_undegraded":"2023-10-18T08:55:16.691182+0000","last_fullsized":"2023-10-18T08:55:16.691182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:07:58.310623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280368+0000","last_change":"2023-10-18T08:55:15.307636+0000","last_active":"2023-10-18T08:55:16.280368+0000","last_peered":"2023-10-18T08:55:16.280368+0000","last_clean":"2023-10-18T08:55:16.280368+0000","last_became_active":"2023-10-18T08:55:15.306703+0000","last_became_peered":"2023-10-18T08:55:15.306703+0000","last_unstale":"2023-10-18T08:55:16.280368+0000","last_undegraded":"2023-10-18T08:55:16.280368+0000","last_fullsized":"2023-10-18T08:55:16.280368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:36.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689173+0000","last_change":"2023-10-18T08:55:15.307528+0000","last_active":"2023-10-18T08:55:16.689173+0000","last_peered":"2023-10-18T08:55:16.689173+0000","last_clean":"2023-10-18T08:55:16.689173+0000","last_became_active":"2023-10-18T08:55:15.306757+0000","last_became_peered":"2023-10-18T08:55:15.306757+0000","last_unstale":"2023-10-18T08:55:16.689173+0000","last_undegraded":"2023-10-18T08:55:16.689173+0000","last_fullsized":"2023-10-18T08:55:16.689173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:31.888781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280839+0000","last_change":"2023-10-18T08:55:15.316136+0000","last_active":"2023-10-18T08:55:16.280839+0000","last_peered":"2023-10-18T08:55:16.280839+0000","last_clean":"2023-10-18T08:55:16.280839+0000","last_became_active":"2023-10-18T08:55:15.315888+0000","last_became_peered":"2023-10-18T08:55:15.315888+0000","last_unstale":"2023-10-18T08:55:16.280839+0000","last_undegraded":"2023-10-18T08:55:16.280839+0000","last_fullsized":"2023-10-18T08:55:16.280839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:23:50.828764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689385+0000","last_change":"2023-10-18T08:55:15.309215+0000","last_active":"2023-10-18T08:55:16.689385+0000","last_peered":"2023-10-18T08:55:16.689385+0000","last_clean":"2023-10-18T08:55:16.689385+0000","last_became_active":"2023-10-18T08:55:15.308858+0000","last_became_peered":"2023-10-18T08:55:15.308858+0000","last_unstale":"2023-10-18T08:55:16.689385+0000","last_undegraded":"2023-10-18T08:55:16.689385+0000","last_fullsized":"2023-10-18T08:55:16.689385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:06.219128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281659+0000","last_change":"2023-10-18T08:55:15.308660+0000","last_active":"2023-10-18T08:55:16.281659+0000","last_peered":"2023-10-18T08:55:16.281659+0000","last_clean":"2023-10-18T08:55:16.281659+0000","last_became_active":"2023-10-18T08:55:15.306230+0000","last_became_peered":"2023-10-18T08:55:15.306230+0000","last_unstale":"2023-10-18T08:55:16.281659+0000","last_undegraded":"2023-10-18T08:55:16.281659+0000","last_fullsized":"2023-10-18T08:55:16.281659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:06.219128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280534+0000","last_change":"2023-10-18T08:55:15.307486+0000","last_active":"2023-10-18T08:55:16.280534+0000","last_peered":"2023-10-18T08:55:16.280534+0000","last_clean":"2023-10-18T08:55:16.280534+0000","last_became_active":"2023-10-18T08:55:15.306357+0000","last_became_peered":"2023-10-18T08:55:15.306357+0000","last_unstale":"2023-10-18T08:55:16.280534+0000","last_undegraded":"2023-10-18T08:55:16.280534+0000","last_fullsized":"2023-10-18T08:55:16.280534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:31.888781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688669+0000","last_change":"2023-10-18T08:55:15.300645+0000","last_active":"2023-10-18T08:55:16.688669+0000","last_peered":"2023-10-18T08:55:16.688669+0000","last_clean":"2023-10-18T08:55:16.688669+0000","last_became_active":"2023-10-18T08:55:15.299795+0000","last_became_peered":"2023-10-18T08:55:15.299795+0000","last_unstale":"2023-10-18T08:55:16.688669+0000","last_undegraded":"2023-10-18T08:55:16.688669+0000","last_fullsized":"2023-10-18T08:55:16.688669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:21.842439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690857+0000","last_change":"2023-10-18T08:55:15.305929+0000","last_active":"2023-10-18T08:55:16.690857+0000","last_peered":"2023-10-18T08:55:16.690857+0000","last_clean":"2023-10-18T08:55:16.690857+0000","last_became_active":"2023-10-18T08:55:15.301322+0000","last_became_peered":"2023-10-18T08:55:15.301322+0000","last_unstale":"2023-10-18T08:55:16.690857+0000","last_undegraded":"2023-10-18T08:55:16.690857+0000","last_fullsized":"2023-10-18T08:55:16.690857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:41.755639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.687612+0000","last_change":"2023-10-18T08:55:15.320868+0000","last_active":"2023-10-18T08:55:16.687612+0000","last_peered":"2023-10-18T08:55:16.687612+0000","last_clean":"2023-10-18T08:55:16.687612+0000","last_became_active":"2023-10-18T08:55:15.320606+0000","last_became_peered":"2023-10-18T08:55:15.320606+0000","last_unstale":"2023-10-18T08:55:16.687612+0000","last_undegraded":"2023-10-18T08:55:16.687612+0000","last_fullsized":"2023-10-18T08:55:16.687612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:04.364364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690001+0000","last_change":"2023-10-18T08:55:15.314269+0000","last_active":"2023-10-18T08:55:16.690001+0000","last_peered":"2023-10-18T08:55:16.690001+0000","last_clean":"2023-10-18T08:55:16.690001+0000","last_became_active":"2023-10-18T08:55:15.314011+0000","last_became_peered":"2023-10-18T08:55:15.314011+0000","last_unstale":"2023-10-18T08:55:16.690001+0000","last_undegraded":"2023-10-18T08:55:16.690001+0000","last_fullsized":"2023-10-18T08:55:16.690001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:28.148282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689559+0000","last_change":"2023-10-18T08:55:15.306134+0000","last_active":"2023-10-18T08:55:16.689559+0000","last_peered":"2023-10-18T08:55:16.689559+0000","last_clean":"2023-10-18T08:55:16.689559+0000","last_became_active":"2023-10-18T08:55:15.305389+0000","last_became_peered":"2023-10-18T08:55:15.305389+0000","last_unstale":"2023-10-18T08:55:16.689559+0000","last_undegraded":"2023-10-18T08:55:16.689559+0000","last_fullsized":"2023-10-18T08:55:16.689559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:32.273116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.327693+0000","last_change":"2023-10-18T08:55:14.290336+0000","last_active":"2023-10-18T08:55:15.327693+0000","last_peered":"2023-10-18T08:55:15.327693+0000","last_clean":"2023-10-18T08:55:15.327693+0000","last_became_active":"2023-10-18T08:55:14.290051+0000","last_became_peered":"2023-10-18T08:55:14.290051+0000","last_unstale":"2023-10-18T08:55:15.327693+0000","last_undegraded":"2023-10-18T08:55:15.327693+0000","last_fullsized":"2023-10-18T08:55:15.327693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:38.968705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689230+0000","last_change":"2023-10-18T08:55:15.302202+0000","last_active":"2023-10-18T08:55:16.689230+0000","last_peered":"2023-10-18T08:55:16.689230+0000","last_clean":"2023-10-18T08:55:16.689230+0000","last_became_active":"2023-10-18T08:55:15.301534+0000","last_became_peered":"2023-10-18T08:55:15.301534+0000","last_unstale":"2023-10-18T08:55:16.689230+0000","last_undegraded":"2023-10-18T08:55:16.689230+0000","last_fullsized":"2023-10-18T08:55:16.689230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:27.650596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.326541+0000","last_change":"2023-10-18T08:55:14.296292+0000","last_active":"2023-10-18T08:55:15.326541+0000","last_peered":"2023-10-18T08:55:15.326541+0000","last_clean":"2023-10-18T08:55:15.326541+0000","last_became_active":"2023-10-18T08:55:14.295937+0000","last_became_peered":"2023-10-18T08:55:14.295937+0000","last_unstale":"2023-10-18T08:55:15.326541+0000","last_undegraded":"2023-10-18T08:55:15.326541+0000","last_fullsized":"2023-10-18T08:55:15.326541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:38.968705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.280268+0000","last_change":"2023-10-18T08:55:14.293089+0000","last_active":"2023-10-18T08:55:15.280268+0000","last_peered":"2023-10-18T08:55:15.280268+0000","last_clean":"2023-10-18T08:55:15.280268+0000","last_became_active":"2023-10-18T08:55:14.292273+0000","last_became_peered":"2023-10-18T08:55:14.292273+0000","last_unstale":"2023-10-18T08:55:15.280268+0000","last_undegraded":"2023-10-18T08:55:15.280268+0000","last_fullsized":"2023-10-18T08:55:15.280268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:03.359656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281004+0000","last_change":"2023-10-18T08:55:15.297969+0000","last_active":"2023-10-18T08:55:16.281004+0000","last_peered":"2023-10-18T08:55:16.281004+0000","last_clean":"2023-10-18T08:55:16.281004+0000","last_became_active":"2023-10-18T08:55:15.297730+0000","last_became_peered":"2023-10-18T08:55:15.297730+0000","last_unstale":"2023-10-18T08:55:16.281004+0000","last_undegraded":"2023-10-18T08:55:16.281004+0000","last_fullsized":"2023-10-18T08:55:16.281004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:04.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.301050+0000","last_change":"2023-10-18T08:55:14.299406+0000","last_active":"2023-10-18T08:55:15.301050+0000","last_peered":"2023-10-18T08:55:15.301050+0000","last_clean":"2023-10-18T08:55:15.301050+0000","last_became_active":"2023-10-18T08:55:14.299181+0000","last_became_peered":"2023-10-18T08:55:14.299181+0000","last_unstale":"2023-10-18T08:55:15.301050+0000","last_undegraded":"2023-10-18T08:55:15.301050+0000","last_fullsized":"2023-10-18T08:55:15.301050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:18.874528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281450+0000","last_change":"2023-10-18T08:55:15.309032+0000","last_active":"2023-10-18T08:55:16.281450+0000","last_peered":"2023-10-18T08:55:16.281450+0000","last_clean":"2023-10-18T08:55:16.281450+0000","last_became_active":"2023-10-18T08:55:15.308756+0000","last_became_peered":"2023-10-18T08:55:15.308756+0000","last_unstale":"2023-10-18T08:55:16.281450+0000","last_undegraded":"2023-10-18T08:55:16.281450+0000","last_fullsized":"2023-10-18T08:55:16.281450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:27.650596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.297946+0000","last_change":"2023-10-18T08:55:14.304174+0000","last_active":"2023-10-18T08:55:15.297946+0000","last_peered":"2023-10-18T08:55:15.297946+0000","last_clean":"2023-10-18T08:55:15.297946+0000","last_became_active":"2023-10-18T08:55:14.303968+0000","last_became_peered":"2023-10-18T08:55:14.303968+0000","last_unstale":"2023-10-18T08:55:15.297946+0000","last_undegraded":"2023-10-18T08:55:15.297946+0000","last_fullsized":"2023-10-18T08:55:15.297946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:04.186241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690815+0000","last_change":"2023-10-18T08:55:15.312484+0000","last_active":"2023-10-18T08:55:16.690815+0000","last_peered":"2023-10-18T08:55:16.690815+0000","last_clean":"2023-10-18T08:55:16.690815+0000","last_became_active":"2023-10-18T08:55:15.311626+0000","last_became_peered":"2023-10-18T08:55:15.311626+0000","last_unstale":"2023-10-18T08:55:16.690815+0000","last_undegraded":"2023-10-18T08:55:16.690815+0000","last_fullsized":"2023-10-18T08:55:16.690815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:04.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.300021+0000","last_change":"2023-10-18T08:55:14.302651+0000","last_active":"2023-10-18T08:55:15.300021+0000","last_peered":"2023-10-18T08:55:15.300021+0000","last_clean":"2023-10-18T08:55:15.300021+0000","last_became_active":"2023-10-18T08:55:14.302029+0000","last_became_peered":"2023-10-18T08:55:14.302029+0000","last_unstale":"2023-10-18T08:55:15.300021+0000","last_undegraded":"2023-10-18T08:55:15.300021+0000","last_fullsized":"2023-10-18T08:55:15.300021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:44.154488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689826+0000","last_change":"2023-10-18T08:55:15.322350+0000","last_active":"2023-10-18T08:55:16.689826+0000","last_peered":"2023-10-18T08:55:16.689826+0000","last_clean":"2023-10-18T08:55:16.689826+0000","last_became_active":"2023-10-18T08:55:15.322002+0000","last_became_peered":"2023-10-18T08:55:15.322002+0000","last_unstale":"2023-10-18T08:55:16.689826+0000","last_undegraded":"2023-10-18T08:55:16.689826+0000","last_fullsized":"2023-10-18T08:55:16.689826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:36.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.326702+0000","last_change":"2023-10-18T08:55:14.300549+0000","last_active":"2023-10-18T08:55:15.326702+0000","last_peered":"2023-10-18T08:55:15.326702+0000","last_clean":"2023-10-18T08:55:15.326702+0000","last_became_active":"2023-10-18T08:55:14.299990+0000","last_became_peered":"2023-10-18T08:55:14.299990+0000","last_unstale":"2023-10-18T08:55:15.326702+0000","last_undegraded":"2023-10-18T08:55:15.326702+0000","last_fullsized":"2023-10-18T08:55:15.326702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:37.505319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690659+0000","last_change":"2023-10-18T08:55:15.314324+0000","last_active":"2023-10-18T08:55:16.690659+0000","last_peered":"2023-10-18T08:55:16.690659+0000","last_clean":"2023-10-18T08:55:16.690659+0000","last_became_active":"2023-10-18T08:55:15.313890+0000","last_became_peered":"2023-10-18T08:55:15.313890+0000","last_unstale":"2023-10-18T08:55:16.690659+0000","last_undegraded":"2023-10-18T08:55:16.690659+0000","last_fullsized":"2023-10-18T08:55:16.690659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:05.190948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.329951+0000","last_change":"2023-10-18T08:55:14.292962+0000","last_active":"2023-10-18T08:55:15.329951+0000","last_peered":"2023-10-18T08:55:15.329951+0000","last_clean":"2023-10-18T08:55:15.329951+0000","last_became_active":"2023-10-18T08:55:14.292424+0000","last_became_peered":"2023-10-18T08:55:14.292424+0000","last_unstale":"2023-10-18T08:55:15.329951+0000","last_undegraded":"2023-10-18T08:55:15.329951+0000","last_fullsized":"2023-10-18T08:55:15.329951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:20.837731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282227+0000","last_change":"2023-10-18T08:55:15.296745+0000","last_active":"2023-10-18T08:55:16.282227+0000","last_peered":"2023-10-18T08:55:16.282227+0000","last_clean":"2023-10-18T08:55:16.282227+0000","last_became_active":"2023-10-18T08:55:15.296095+0000","last_became_peered":"2023-10-18T08:55:15.296095+0000","last_unstale":"2023-10-18T08:55:16.282227+0000","last_undegraded":"2023-10-18T08:55:16.282227+0000","last_fullsized":"2023-10-18T08:55:16.282227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.688019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.328191+0000","last_change":"2023-10-18T08:55:14.299318+0000","last_active":"2023-10-18T08:55:15.328191+0000","last_peered":"2023-10-18T08:55:15.328191+0000","last_clean":"2023-10-18T08:55:15.328191+0000","last_became_active":"2023-10-18T08:55:14.299069+0000","last_became_peered":"2023-10-18T08:55:14.299069+0000","last_unstale":"2023-10-18T08:55:15.328191+0000","last_undegraded":"2023-10-18T08:55:15.328191+0000","last_fullsized":"2023-10-18T08:55:15.328191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:03.359656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690146+0000","last_change":"2023-10-18T08:55:15.307886+0000","last_active":"2023-10-18T08:55:16.690146+0000","last_peered":"2023-10-18T08:55:16.690146+0000","last_clean":"2023-10-18T08:55:16.690146+0000","last_became_active":"2023-10-18T08:55:15.306465+0000","last_became_peered":"2023-10-18T08:55:15.306465+0000","last_unstale":"2023-10-18T08:55:16.690146+0000","last_undegraded":"2023-10-18T08:55:16.690146+0000","last_fullsized":"2023-10-18T08:55:16.690146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:31.888781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.316152+0000","last_change":"2023-10-18T08:55:14.305151+0000","last_active":"2023-10-18T08:55:15.316152+0000","last_peered":"2023-10-18T08:55:15.316152+0000","last_clean":"2023-10-18T08:55:15.316152+0000","last_became_active":"2023-10-18T08:55:14.303161+0000","last_became_peered":"2023-10-18T08:55:14.303161+0000","last_unstale":"2023-10-18T08:55:15.316152+0000","last_undegraded":"2023-10-18T08:55:15.316152+0000","last_fullsized":"2023-10-18T08:55:15.316152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:26.645889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.283192+0000","last_change":"2023-10-18T08:55:15.317138+0000","last_active":"2023-10-18T08:55:16.283192+0000","last_peered":"2023-10-18T08:55:16.283192+0000","last_clean":"2023-10-18T08:55:16.283192+0000","last_became_active":"2023-10-18T08:55:15.316743+0000","last_became_peered":"2023-10-18T08:55:15.316743+0000","last_unstale":"2023-10-18T08:55:16.283192+0000","last_undegraded":"2023-10-18T08:55:16.283192+0000","last_fullsized":"2023-10-18T08:55:16.283192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:46.921031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.303320+0000","last_change":"2023-10-18T08:55:14.294348+0000","last_active":"2023-10-18T08:55:15.303320+0000","last_peered":"2023-10-18T08:55:15.303320+0000","last_clean":"2023-10-18T08:55:15.303320+0000","last_became_active":"2023-10-18T08:55:14.293456+0000","last_became_peered":"2023-10-18T08:55:14.293456+0000","last_unstale":"2023-10-18T08:55:15.303320+0000","last_undegraded":"2023-10-18T08:55:15.303320+0000","last_fullsized":"2023-10-18T08:55:15.303320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:37.505319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689020+0000","last_change":"2023-10-18T08:55:15.304772+0000","last_active":"2023-10-18T08:55:16.689020+0000","last_peered":"2023-10-18T08:55:16.689020+0000","last_clean":"2023-10-18T08:55:16.689020+0000","last_became_active":"2023-10-18T08:55:15.304528+0000","last_became_peered":"2023-10-18T08:55:15.304528+0000","last_unstale":"2023-10-18T08:55:16.689020+0000","last_undegraded":"2023-10-18T08:55:16.689020+0000","last_fullsized":"2023-10-18T08:55:16.689020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:27.650596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.330488+0000","last_change":"2023-10-18T08:55:14.297308+0000","last_active":"2023-10-18T08:55:15.330488+0000","last_peered":"2023-10-18T08:55:15.330488+0000","last_clean":"2023-10-18T08:55:15.330488+0000","last_became_active":"2023-10-18T08:55:14.297069+0000","last_became_peered":"2023-10-18T08:55:14.297069+0000","last_unstale":"2023-10-18T08:55:15.330488+0000","last_undegraded":"2023-10-18T08:55:15.330488+0000","last_fullsized":"2023-10-18T08:55:15.330488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:10.814889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.283167+0000","last_change":"2023-10-18T08:55:15.303171+0000","last_active":"2023-10-18T08:55:16.283167+0000","last_peered":"2023-10-18T08:55:16.283167+0000","last_clean":"2023-10-18T08:55:16.283167+0000","last_became_active":"2023-10-18T08:55:15.301815+0000","last_became_peered":"2023-10-18T08:55:15.301815+0000","last_unstale":"2023-10-18T08:55:16.283167+0000","last_undegraded":"2023-10-18T08:55:16.283167+0000","last_fullsized":"2023-10-18T08:55:16.283167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:10.355168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.325923+0000","last_change":"2023-10-18T08:55:14.298243+0000","last_active":"2023-10-18T08:55:15.325923+0000","last_peered":"2023-10-18T08:55:15.325923+0000","last_clean":"2023-10-18T08:55:15.325923+0000","last_became_active":"2023-10-18T08:55:14.297343+0000","last_became_peered":"2023-10-18T08:55:14.297343+0000","last_unstale":"2023-10-18T08:55:15.325923+0000","last_undegraded":"2023-10-18T08:55:15.325923+0000","last_fullsized":"2023-10-18T08:55:15.325923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:10.814889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.283511+0000","last_change":"2023-10-18T08:55:15.316986+0000","last_active":"2023-10-18T08:55:16.283511+0000","last_peered":"2023-10-18T08:55:16.283511+0000","last_clean":"2023-10-18T08:55:16.283511+0000","last_became_active":"2023-10-18T08:55:15.316398+0000","last_became_peered":"2023-10-18T08:55:15.316398+0000","last_unstale":"2023-10-18T08:55:16.283511+0000","last_undegraded":"2023-10-18T08:55:16.283511+0000","last_fullsized":"2023-10-18T08:55:16.283511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:56:45.727823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.329694+0000","last_change":"2023-10-18T08:55:14.290070+0000","last_active":"2023-10-18T08:55:15.329694+0000","last_peered":"2023-10-18T08:55:15.329694+0000","last_clean":"2023-10-18T08:55:15.329694+0000","last_became_active":"2023-10-18T08:55:14.289756+0000","last_became_peered":"2023-10-18T08:55:14.289756+0000","last_unstale":"2023-10-18T08:55:15.329694+0000","last_undegraded":"2023-10-18T08:55:15.329694+0000","last_fullsized":"2023-10-18T08:55:15.329694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:01.470259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282548+0000","last_change":"2023-10-18T08:55:15.311283+0000","last_active":"2023-10-18T08:55:16.282548+0000","last_peered":"2023-10-18T08:55:16.282548+0000","last_clean":"2023-10-18T08:55:16.282548+0000","last_became_active":"2023-10-18T08:55:15.311053+0000","last_became_peered":"2023-10-18T08:55:15.311053+0000","last_unstale":"2023-10-18T08:55:16.282548+0000","last_undegraded":"2023-10-18T08:55:16.282548+0000","last_fullsized":"2023-10-18T08:55:16.282548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:23:50.828764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.325915+0000","last_change":"2023-10-18T08:55:14.294635+0000","last_active":"2023-10-18T08:55:15.325915+0000","last_peered":"2023-10-18T08:55:15.325915+0000","last_clean":"2023-10-18T08:55:15.325915+0000","last_became_active":"2023-10-18T08:55:14.294341+0000","last_became_peered":"2023-10-18T08:55:14.294341+0000","last_unstale":"2023-10-18T08:55:15.325915+0000","last_undegraded":"2023-10-18T08:55:15.325915+0000","last_fullsized":"2023-10-18T08:55:15.325915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:01.470259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691402+0000","last_change":"2023-10-18T08:55:15.312145+0000","last_active":"2023-10-18T08:55:16.691402+0000","last_peered":"2023-10-18T08:55:16.691402+0000","last_clean":"2023-10-18T08:55:16.691402+0000","last_became_active":"2023-10-18T08:55:15.310296+0000","last_became_peered":"2023-10-18T08:55:15.310296+0000","last_unstale":"2023-10-18T08:55:16.691402+0000","last_undegraded":"2023-10-18T08:55:16.691402+0000","last_fullsized":"2023-10-18T08:55:16.691402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:46.921031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.282183+0000","last_change":"2023-10-18T08:55:14.303903+0000","last_active":"2023-10-18T08:55:15.282183+0000","last_peered":"2023-10-18T08:55:15.282183+0000","last_clean":"2023-10-18T08:55:15.282183+0000","last_became_active":"2023-10-18T08:55:14.303633+0000","last_became_peered":"2023-10-18T08:55:14.303633+0000","last_unstale":"2023-10-18T08:55:15.282183+0000","last_undegraded":"2023-10-18T08:55:15.282183+0000","last_fullsized":"2023-10-18T08:55:15.282183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:16:56.668322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.283316+0000","last_change":"2023-10-18T08:55:15.318598+0000","last_active":"2023-10-18T08:55:16.283316+0000","last_peered":"2023-10-18T08:55:16.283316+0000","last_clean":"2023-10-18T08:55:16.283316+0000","last_became_active":"2023-10-18T08:55:15.318353+0000","last_became_peered":"2023-10-18T08:55:15.318353+0000","last_unstale":"2023-10-18T08:55:16.283316+0000","last_undegraded":"2023-10-18T08:55:16.283316+0000","last_fullsized":"2023-10-18T08:55:16.283316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:10.917539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.287158+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:55:15.287158+0000","last_peered":"2023-10-18T08:55:15.287158+0000","last_clean":"2023-10-18T08:55:15.287158+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:55:15.287158+0000","last_undegraded":"2023-10-18T08:55:15.287158+0000","last_fullsized":"2023-10-18T08:55:15.287158+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:34.881744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.316324+0000","last_change":"2023-10-18T08:55:14.305589+0000","last_active":"2023-10-18T08:55:15.316324+0000","last_peered":"2023-10-18T08:55:15.316324+0000","last_clean":"2023-10-18T08:55:15.316324+0000","last_became_active":"2023-10-18T08:55:14.303723+0000","last_became_peered":"2023-10-18T08:55:14.303723+0000","last_unstale":"2023-10-18T08:55:15.316324+0000","last_undegraded":"2023-10-18T08:55:15.316324+0000","last_fullsized":"2023-10-18T08:55:15.316324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:05.214421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280180+0000","last_change":"2023-10-18T08:55:15.307598+0000","last_active":"2023-10-18T08:55:16.280180+0000","last_peered":"2023-10-18T08:55:16.280180+0000","last_clean":"2023-10-18T08:55:16.280180+0000","last_became_active":"2023-10-18T08:55:15.307263+0000","last_became_peered":"2023-10-18T08:55:15.307263+0000","last_unstale":"2023-10-18T08:55:16.280180+0000","last_undegraded":"2023-10-18T08:55:16.280180+0000","last_fullsized":"2023-10-18T08:55:16.280180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:28.148282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.286898+0000","last_change":"2023-10-18T08:55:14.299430+0000","last_active":"2023-10-18T08:55:15.286898+0000","last_peered":"2023-10-18T08:55:15.286898+0000","last_clean":"2023-10-18T08:55:15.286898+0000","last_became_active":"2023-10-18T08:55:14.299004+0000","last_became_peered":"2023-10-18T08:55:14.299004+0000","last_unstale":"2023-10-18T08:55:15.286898+0000","last_undegraded":"2023-10-18T08:55:15.286898+0000","last_fullsized":"2023-10-18T08:55:15.286898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:26.645889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689865+0000","last_change":"2023-10-18T08:55:15.308337+0000","last_active":"2023-10-18T08:55:16.689865+0000","last_peered":"2023-10-18T08:55:16.689865+0000","last_clean":"2023-10-18T08:55:16.689865+0000","last_became_active":"2023-10-18T08:55:15.305146+0000","last_became_peered":"2023-10-18T08:55:15.305146+0000","last_unstale":"2023-10-18T08:55:16.689865+0000","last_undegraded":"2023-10-18T08:55:16.689865+0000","last_fullsized":"2023-10-18T08:55:16.689865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.283153+0000","last_change":"2023-10-18T08:55:14.287811+0000","last_active":"2023-10-18T08:55:15.283153+0000","last_peered":"2023-10-18T08:55:15.283153+0000","last_clean":"2023-10-18T08:55:15.283153+0000","last_became_active":"2023-10-18T08:55:14.287466+0000","last_became_peered":"2023-10-18T08:55:14.287466+0000","last_unstale":"2023-10-18T08:55:15.283153+0000","last_undegraded":"2023-10-18T08:55:15.283153+0000","last_fullsized":"2023-10-18T08:55:15.283153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:31.268409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281786+0000","last_change":"2023-10-18T08:55:15.301141+0000","last_active":"2023-10-18T08:55:16.281786+0000","last_peered":"2023-10-18T08:55:16.281786+0000","last_clean":"2023-10-18T08:55:16.281786+0000","last_became_active":"2023-10-18T08:55:15.300535+0000","last_became_peered":"2023-10-18T08:55:15.300535+0000","last_unstale":"2023-10-18T08:55:16.281786+0000","last_undegraded":"2023-10-18T08:55:16.281786+0000","last_fullsized":"2023-10-18T08:55:16.281786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:28.148282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.300341+0000","last_change":"2023-10-18T08:55:14.302819+0000","last_active":"2023-10-18T08:55:15.300341+0000","last_peered":"2023-10-18T08:55:15.300341+0000","last_clean":"2023-10-18T08:55:15.300341+0000","last_became_active":"2023-10-18T08:55:14.302381+0000","last_became_peered":"2023-10-18T08:55:14.302381+0000","last_unstale":"2023-10-18T08:55:15.300341+0000","last_undegraded":"2023-10-18T08:55:15.300341+0000","last_fullsized":"2023-10-18T08:55:15.300341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:20.837731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690460+0000","last_change":"2023-10-18T08:55:15.309720+0000","last_active":"2023-10-18T08:55:16.690460+0000","last_peered":"2023-10-18T08:55:16.690460+0000","last_clean":"2023-10-18T08:55:16.690460+0000","last_became_active":"2023-10-18T08:55:15.309469+0000","last_became_peered":"2023-10-18T08:55:15.309469+0000","last_unstale":"2023-10-18T08:55:16.690460+0000","last_undegraded":"2023-10-18T08:55:16.690460+0000","last_fullsized":"2023-10-18T08:55:16.690460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:04.364364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.295166+0000","last_change":"2023-10-18T08:55:14.296298+0000","last_active":"2023-10-18T08:55:15.295166+0000","last_peered":"2023-10-18T08:55:15.295166+0000","last_clean":"2023-10-18T08:55:15.295166+0000","last_became_active":"2023-10-18T08:55:14.295788+0000","last_became_peered":"2023-10-18T08:55:14.295788+0000","last_unstale":"2023-10-18T08:55:15.295166+0000","last_undegraded":"2023-10-18T08:55:15.295166+0000","last_fullsized":"2023-10-18T08:55:15.295166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:38.968705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689511+0000","last_change":"2023-10-18T08:55:15.307391+0000","last_active":"2023-10-18T08:55:16.689511+0000","last_peered":"2023-10-18T08:55:16.689511+0000","last_clean":"2023-10-18T08:55:16.689511+0000","last_became_active":"2023-10-18T08:55:15.306422+0000","last_became_peered":"2023-10-18T08:55:15.306422+0000","last_unstale":"2023-10-18T08:55:16.689511+0000","last_undegraded":"2023-10-18T08:55:16.689511+0000","last_fullsized":"2023-10-18T08:55:16.689511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.688019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.286110+0000","last_change":"2023-10-18T08:55:14.299281+0000","last_active":"2023-10-18T08:55:15.286110+0000","last_peered":"2023-10-18T08:55:15.286110+0000","last_clean":"2023-10-18T08:55:15.286110+0000","last_became_active":"2023-10-18T08:55:14.298698+0000","last_became_peered":"2023-10-18T08:55:14.298698+0000","last_unstale":"2023-10-18T08:55:15.286110+0000","last_undegraded":"2023-10-18T08:55:15.286110+0000","last_fullsized":"2023-10-18T08:55:15.286110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:44.154488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281874+0000","last_change":"2023-10-18T08:55:15.299892+0000","last_active":"2023-10-18T08:55:16.281874+0000","last_peered":"2023-10-18T08:55:16.281874+0000","last_clean":"2023-10-18T08:55:16.281874+0000","last_became_active":"2023-10-18T08:55:15.299281+0000","last_became_peered":"2023-10-18T08:55:15.299281+0000","last_unstale":"2023-10-18T08:55:16.281874+0000","last_undegraded":"2023-10-18T08:55:16.281874+0000","last_fullsized":"2023-10-18T08:55:16.281874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:36.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.296680+0000","last_change":"2023-10-18T08:55:14.300359+0000","last_active":"2023-10-18T08:55:15.296680+0000","last_peered":"2023-10-18T08:55:15.296680+0000","last_clean":"2023-10-18T08:55:15.296680+0000","last_became_active":"2023-10-18T08:55:14.299423+0000","last_became_peered":"2023-10-18T08:55:14.299423+0000","last_unstale":"2023-10-18T08:55:15.296680+0000","last_undegraded":"2023-10-18T08:55:15.296680+0000","last_fullsized":"2023-10-18T08:55:15.296680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:44.154488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280011+0000","last_change":"2023-10-18T08:55:15.288327+0000","last_active":"2023-10-18T08:55:16.280011+0000","last_peered":"2023-10-18T08:55:16.280011+0000","last_clean":"2023-10-18T08:55:16.280011+0000","last_became_active":"2023-10-18T08:55:15.287648+0000","last_became_peered":"2023-10-18T08:55:15.287648+0000","last_unstale":"2023-10-18T08:55:16.280011+0000","last_undegraded":"2023-10-18T08:55:16.280011+0000","last_fullsized":"2023-10-18T08:55:16.280011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:27.650596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.327662+0000","last_change":"2023-10-18T08:55:14.294992+0000","last_active":"2023-10-18T08:55:15.327662+0000","last_peered":"2023-10-18T08:55:15.327662+0000","last_clean":"2023-10-18T08:55:15.327662+0000","last_became_active":"2023-10-18T08:55:14.294682+0000","last_became_peered":"2023-10-18T08:55:14.294682+0000","last_unstale":"2023-10-18T08:55:15.327662+0000","last_undegraded":"2023-10-18T08:55:15.327662+0000","last_fullsized":"2023-10-18T08:55:15.327662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:44.154488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689214+0000","last_change":"2023-10-18T08:55:15.308496+0000","last_active":"2023-10-18T08:55:16.689214+0000","last_peered":"2023-10-18T08:55:16.689214+0000","last_clean":"2023-10-18T08:55:16.689214+0000","last_became_active":"2023-10-18T08:55:15.306095+0000","last_became_peered":"2023-10-18T08:55:15.306095+0000","last_unstale":"2023-10-18T08:55:16.689214+0000","last_undegraded":"2023-10-18T08:55:16.689214+0000","last_fullsized":"2023-10-18T08:55:16.689214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:02.173932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.288529+0000","last_change":"2023-10-18T08:55:14.299395+0000","last_active":"2023-10-18T08:55:15.288529+0000","last_peered":"2023-10-18T08:55:15.288529+0000","last_clean":"2023-10-18T08:55:15.288529+0000","last_became_active":"2023-10-18T08:55:14.299130+0000","last_became_peered":"2023-10-18T08:55:14.299130+0000","last_unstale":"2023-10-18T08:55:15.288529+0000","last_undegraded":"2023-10-18T08:55:15.288529+0000","last_fullsized":"2023-10-18T08:55:15.288529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:35.197625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.693084+0000","last_change":"2023-10-18T08:55:15.314704+0000","last_active":"2023-10-18T08:55:16.693084+0000","last_peered":"2023-10-18T08:55:16.693084+0000","last_clean":"2023-10-18T08:55:16.693084+0000","last_became_active":"2023-10-18T08:55:15.314153+0000","last_became_peered":"2023-10-18T08:55:15.314153+0000","last_unstale":"2023-10-18T08:55:16.693084+0000","last_undegraded":"2023-10-18T08:55:16.693084+0000","last_fullsized":"2023-10-18T08:55:16.693084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:39.973412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.295022+0000","last_change":"2023-10-18T08:55:14.297463+0000","last_active":"2023-10-18T08:55:15.295022+0000","last_peered":"2023-10-18T08:55:15.295022+0000","last_clean":"2023-10-18T08:55:15.295022+0000","last_became_active":"2023-10-18T08:55:14.297199+0000","last_became_peered":"2023-10-18T08:55:14.297199+0000","last_unstale":"2023-10-18T08:55:15.295022+0000","last_undegraded":"2023-10-18T08:55:15.295022+0000","last_fullsized":"2023-10-18T08:55:15.295022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:04.186241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.279960+0000","last_change":"2023-10-18T08:55:15.303892+0000","last_active":"2023-10-18T08:55:16.279960+0000","last_peered":"2023-10-18T08:55:16.279960+0000","last_clean":"2023-10-18T08:55:16.279960+0000","last_became_active":"2023-10-18T08:55:15.303545+0000","last_became_peered":"2023-10-18T08:55:15.303545+0000","last_unstale":"2023-10-18T08:55:16.279960+0000","last_undegraded":"2023-10-18T08:55:16.279960+0000","last_fullsized":"2023-10-18T08:55:16.279960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:02.173932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.297222+0000","last_change":"2023-10-18T08:55:14.294024+0000","last_active":"2023-10-18T08:55:15.297222+0000","last_peered":"2023-10-18T08:55:15.297222+0000","last_clean":"2023-10-18T08:55:15.297222+0000","last_became_active":"2023-10-18T08:55:14.293763+0000","last_became_peered":"2023-10-18T08:55:14.293763+0000","last_unstale":"2023-10-18T08:55:15.297222+0000","last_undegraded":"2023-10-18T08:55:15.297222+0000","last_fullsized":"2023-10-18T08:55:15.297222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:20.837731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280694+0000","last_change":"2023-10-18T08:55:15.302069+0000","last_active":"2023-10-18T08:55:16.280694+0000","last_peered":"2023-10-18T08:55:16.280694+0000","last_clean":"2023-10-18T08:55:16.280694+0000","last_became_active":"2023-10-18T08:55:15.301806+0000","last_became_peered":"2023-10-18T08:55:15.301806+0000","last_unstale":"2023-10-18T08:55:16.280694+0000","last_undegraded":"2023-10-18T08:55:16.280694+0000","last_fullsized":"2023-10-18T08:55:16.280694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.688019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.327881+0000","last_change":"2023-10-18T08:55:14.292655+0000","last_active":"2023-10-18T08:55:15.327881+0000","last_peered":"2023-10-18T08:55:15.327881+0000","last_clean":"2023-10-18T08:55:15.327881+0000","last_became_active":"2023-10-18T08:55:14.292291+0000","last_became_peered":"2023-10-18T08:55:14.292291+0000","last_unstale":"2023-10-18T08:55:15.327881+0000","last_undegraded":"2023-10-18T08:55:15.327881+0000","last_fullsized":"2023-10-18T08:55:15.327881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:57.132126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688748+0000","last_change":"2023-10-18T08:55:15.315320+0000","last_active":"2023-10-18T08:55:16.688748+0000","last_peered":"2023-10-18T08:55:16.688748+0000","last_clean":"2023-10-18T08:55:16.688748+0000","last_became_active":"2023-10-18T08:55:15.315007+0000","last_became_peered":"2023-10-18T08:55:15.315007+0000","last_unstale":"2023-10-18T08:55:16.688748+0000","last_undegraded":"2023-10-18T08:55:16.688748+0000","last_fullsized":"2023-10-18T08:55:16.688748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:19.879235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691247+0000","last_change":"2023-10-18T08:55:15.317460+0000","last_active":"2023-10-18T08:55:16.691247+0000","last_peered":"2023-10-18T08:55:16.691247+0000","last_clean":"2023-10-18T08:55:16.691247+0000","last_became_active":"2023-10-18T08:55:15.317194+0000","last_became_peered":"2023-10-18T08:55:15.317194+0000","last_unstale":"2023-10-18T08:55:16.691247+0000","last_undegraded":"2023-10-18T08:55:16.691247+0000","last_fullsized":"2023-10-18T08:55:16.691247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:26.310762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.328212+0000","last_change":"2023-10-18T08:55:14.301102+0000","last_active":"2023-10-18T08:55:15.328212+0000","last_peered":"2023-10-18T08:55:15.328212+0000","last_clean":"2023-10-18T08:55:15.328212+0000","last_became_active":"2023-10-18T08:55:14.300855+0000","last_became_peered":"2023-10-18T08:55:14.300855+0000","last_unstale":"2023-10-18T08:55:15.328212+0000","last_undegraded":"2023-10-18T08:55:15.328212+0000","last_fullsized":"2023-10-18T08:55:15.328212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:27.143575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.692035+0000","last_change":"2023-10-18T08:55:15.320111+0000","last_active":"2023-10-18T08:55:16.692035+0000","last_peered":"2023-10-18T08:55:16.692035+0000","last_clean":"2023-10-18T08:55:16.692035+0000","last_became_active":"2023-10-18T08:55:15.319103+0000","last_became_peered":"2023-10-18T08:55:15.319103+0000","last_unstale":"2023-10-18T08:55:16.692035+0000","last_undegraded":"2023-10-18T08:55:16.692035+0000","last_fullsized":"2023-10-18T08:55:16.692035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:56:45.727823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.327995+0000","last_change":"2023-10-18T08:55:14.293731+0000","last_active":"2023-10-18T08:55:15.327995+0000","last_peered":"2023-10-18T08:55:15.327995+0000","last_clean":"2023-10-18T08:55:15.327995+0000","last_became_active":"2023-10-18T08:55:14.293420+0000","last_became_peered":"2023-10-18T08:55:14.293420+0000","last_unstale":"2023-10-18T08:55:15.327995+0000","last_undegraded":"2023-10-18T08:55:15.327995+0000","last_fullsized":"2023-10-18T08:55:15.327995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:01.169224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689541+0000","last_change":"2023-10-18T08:55:15.307386+0000","last_active":"2023-10-18T08:55:16.689541+0000","last_peered":"2023-10-18T08:55:16.689541+0000","last_clean":"2023-10-18T08:55:16.689541+0000","last_became_active":"2023-10-18T08:55:15.304519+0000","last_became_peered":"2023-10-18T08:55:15.304519+0000","last_unstale":"2023-10-18T08:55:16.689541+0000","last_undegraded":"2023-10-18T08:55:16.689541+0000","last_fullsized":"2023-10-18T08:55:16.689541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:28.148282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.329282+0000","last_change":"2023-10-18T08:55:14.297189+0000","last_active":"2023-10-18T08:55:15.329282+0000","last_peered":"2023-10-18T08:55:15.329282+0000","last_clean":"2023-10-18T08:55:15.329282+0000","last_became_active":"2023-10-18T08:55:14.296794+0000","last_became_peered":"2023-10-18T08:55:14.296794+0000","last_unstale":"2023-10-18T08:55:15.329282+0000","last_undegraded":"2023-10-18T08:55:15.329282+0000","last_fullsized":"2023-10-18T08:55:15.329282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:10.814889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280421+0000","last_change":"2023-10-18T08:55:15.309830+0000","last_active":"2023-10-18T08:55:16.280421+0000","last_peered":"2023-10-18T08:55:16.280421+0000","last_clean":"2023-10-18T08:55:16.280421+0000","last_became_active":"2023-10-18T08:55:15.309632+0000","last_became_peered":"2023-10-18T08:55:15.309632+0000","last_unstale":"2023-10-18T08:55:16.280421+0000","last_undegraded":"2023-10-18T08:55:16.280421+0000","last_fullsized":"2023-10-18T08:55:16.280421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.328808+0000","last_change":"2023-10-18T08:55:14.297412+0000","last_active":"2023-10-18T08:55:15.328808+0000","last_peered":"2023-10-18T08:55:15.328808+0000","last_clean":"2023-10-18T08:55:15.328808+0000","last_became_active":"2023-10-18T08:55:14.296983+0000","last_became_peered":"2023-10-18T08:55:14.296983+0000","last_unstale":"2023-10-18T08:55:15.328808+0000","last_undegraded":"2023-10-18T08:55:15.328808+0000","last_fullsized":"2023-10-18T08:55:15.328808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:38.968705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338312,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3080000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4590000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2889999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.546}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5069999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930155,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5349999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.613}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5549999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.609}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4179999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4590000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930121,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4470000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.417}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194667,"statfs":{"total":200000000,"available":199339275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.647}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4810000000000001}]}]}],"pool_statfs":[]}} 2023-10-18T08:55:24.924 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:55:25.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.162+0000 7fddc551b640 1 -- 172.21.15.40:0/4030798184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc01425a0 msgr2=0x7fddc00cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:25.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.162+0000 7fddc551b640 1 --2- 172.21.15.40:0/4030798184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc01425a0 0x7fddc00cafc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fddb8005fd0 tx=0x7fddb802c060 comp rx=0 tx=0).stop 2023-10-18T08:55:25.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.162+0000 7fddc551b640 1 -- 172.21.15.40:0/4030798184 shutdown_connections 2023-10-18T08:55:25.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.162+0000 7fddc551b640 1 --2- 172.21.15.40:0/4030798184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc01425a0 0x7fddc00cafc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.162+0000 7fddc551b640 1 --2- 172.21.15.40:0/4030798184 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddc00ca160 0x7fddc00b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.162+0000 7fddc551b640 1 -- 172.21.15.40:0/4030798184 >> 172.21.15.40:0/4030798184 conn(0x7fddc0069720 msgr2=0x7fddc0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:25.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.162+0000 7fddc551b640 1 -- 172.21.15.40:0/4030798184 shutdown_connections 2023-10-18T08:55:25.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.162+0000 7fddc551b640 1 -- 172.21.15.40:0/4030798184 wait complete. 2023-10-18T08:55:25.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.164+0000 7fddc551b640 1 Processor -- start 2023-10-18T08:55:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.164+0000 7fddc551b640 1 -- start start 2023-10-18T08:55:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.165+0000 7fddc551b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc00ca160 0x7fddc015a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.165+0000 7fddc551b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddc01420f0 0x7fddc015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.165+0000 7fddc551b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc01425a0 0x7fddc00ae310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:25.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.165+0000 7fddc551b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc015b420 con 0x7fddc00ca160 2023-10-18T08:55:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.165+0000 7fddc551b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc015b670 con 0x7fddc01425a0 2023-10-18T08:55:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.165+0000 7fddc551b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc015b950 con 0x7fddc01420f0 2023-10-18T08:55:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.165+0000 7fddbffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc00ca160 0x7fddc015a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.165+0000 7fddbffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc00ca160 0x7fddc015a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47770/0 (socket says 172.21.15.40:47770) 2023-10-18T08:55:25.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.165+0000 7fddbffff640 1 -- 172.21.15.40:0/3129336708 learned_addr learned my addr 172.21.15.40:0/3129336708 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:25.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.165+0000 7fddb7fff640 1 --2- 172.21.15.40:0/3129336708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddc01420f0 0x7fddc015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:25.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.166+0000 7fddc4d1a640 1 --2- 172.21.15.40:0/3129336708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc01425a0 0x7fddc00ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:25.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.166+0000 7fddbffff640 1 -- 172.21.15.40:0/3129336708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddc01420f0 msgr2=0x7fddc015aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:25.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.166+0000 7fddbffff640 1 --2- 172.21.15.40:0/3129336708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddc01420f0 0x7fddc015aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.166+0000 7fddbffff640 1 -- 172.21.15.40:0/3129336708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc01425a0 msgr2=0x7fddc00ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:25.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.166+0000 7fddbffff640 1 --2- 172.21.15.40:0/3129336708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc01425a0 0x7fddc00ae310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.166+0000 7fddbffff640 1 -- 172.21.15.40:0/3129336708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb8009840 con 0x7fddc00ca160 2023-10-18T08:55:25.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.166+0000 7fddc4d1a640 1 --2- 172.21.15.40:0/3129336708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc01425a0 0x7fddc00ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:25.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.166+0000 7fddbffff640 1 --2- 172.21.15.40:0/3129336708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc00ca160 0x7fddc015a9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fddb000c490 tx=0x7fddb000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:25.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.166+0000 7fddbd7fa640 1 -- 172.21.15.40:0/3129336708 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0012830 con 0x7fddc00ca160 2023-10-18T08:55:25.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.166+0000 7fddbd7fa640 1 -- 172.21.15.40:0/3129336708 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fddb0012a80 con 0x7fddc00ca160 2023-10-18T08:55:25.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.166+0000 7fddbd7fa640 1 -- 172.21.15.40:0/3129336708 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0011380 con 0x7fddc00ca160 2023-10-18T08:55:25.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.166+0000 7fddc551b640 1 -- 172.21.15.40:0/3129336708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc015bc30 con 0x7fddc00ca160 2023-10-18T08:55:25.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.166+0000 7fddc551b640 1 -- 172.21.15.40:0/3129336708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc015c1f0 con 0x7fddc00ca160 2023-10-18T08:55:25.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.168+0000 7fddc551b640 1 -- 172.21.15.40:0/3129336708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc0066e00 con 0x7fddc00ca160 2023-10-18T08:55:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.169+0000 7fddbd7fa640 1 -- 172.21.15.40:0/3129336708 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fddb00114e0 con 0x7fddc00ca160 2023-10-18T08:55:25.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.170+0000 7fddbd7fa640 1 --2- 172.21.15.40:0/3129336708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdda006ae40 0x7fdda006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.170+0000 7fddbd7fa640 1 -- 172.21.15.40:0/3129336708 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddb0089890 con 0x7fddc00ca160 2023-10-18T08:55:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.171+0000 7fddb7fff640 1 --2- 172.21.15.40:0/3129336708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdda006ae40 0x7fdda006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:25.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.172+0000 7fddb7fff640 1 --2- 172.21.15.40:0/3129336708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdda006ae40 0x7fdda006d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdda8005fd0 tx=0x7fdda8005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:25.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.175+0000 7fddbd7fa640 1 -- 172.21.15.40:0/3129336708 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fddb0054930 con 0x7fddc00ca160 2023-10-18T08:55:25.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.366+0000 7fddc551b640 1 -- 172.21.15.40:0/3129336708 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fddc015bac0 con 0x7fdda006ae40 2023-10-18T08:55:25.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.387+0000 7fddbd7fa640 1 -- 172.21.15.40:0/3129336708 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259992 (secure 0 0 0) 0x7fddc015bac0 con 0x7fdda006ae40 2023-10-18T08:55:25.394 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:25.398 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:55:25.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.392+0000 7fddc551b640 1 -- 172.21.15.40:0/3129336708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdda006ae40 msgr2=0x7fdda006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:25.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.392+0000 7fddc551b640 1 --2- 172.21.15.40:0/3129336708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdda006ae40 0x7fdda006d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdda8005fd0 tx=0x7fdda8005850 comp rx=0 tx=0).stop 2023-10-18T08:55:25.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.392+0000 7fddc551b640 1 -- 172.21.15.40:0/3129336708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc00ca160 msgr2=0x7fddc015a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:25.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.392+0000 7fddc551b640 1 --2- 172.21.15.40:0/3129336708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc00ca160 0x7fddc015a9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fddb000c490 tx=0x7fddb000c950 comp rx=0 tx=0).stop 2023-10-18T08:55:25.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.392+0000 7fddc551b640 1 -- 172.21.15.40:0/3129336708 shutdown_connections 2023-10-18T08:55:25.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.392+0000 7fddc551b640 1 --2- 172.21.15.40:0/3129336708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdda006ae40 0x7fdda006d300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.392+0000 7fddc551b640 1 --2- 172.21.15.40:0/3129336708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc01425a0 0x7fddc00ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.392+0000 7fddc551b640 1 --2- 172.21.15.40:0/3129336708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddc01420f0 0x7fddc015aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.392+0000 7fddc551b640 1 --2- 172.21.15.40:0/3129336708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddc00ca160 0x7fddc015a9a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.392+0000 7fddc551b640 1 -- 172.21.15.40:0/3129336708 >> 172.21.15.40:0/3129336708 conn(0x7fddc0069720 msgr2=0x7fddc013e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:25.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.392+0000 7fddc551b640 1 -- 172.21.15.40:0/3129336708 shutdown_connections 2023-10-18T08:55:25.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.393+0000 7fddc551b640 1 -- 172.21.15.40:0/3129336708 wait complete. 2023-10-18T08:55:25.416 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-18T08:55:24.896680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560792,"statfs":{"total":1600000000,"available":1598251597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.616314"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282778+0000","last_change":"2023-10-18T08:55:15.307275+0000","last_active":"2023-10-18T08:55:16.282778+0000","last_peered":"2023-10-18T08:55:16.282778+0000","last_clean":"2023-10-18T08:55:16.282778+0000","last_became_active":"2023-10-18T08:55:15.304759+0000","last_became_peered":"2023-10-18T08:55:15.304759+0000","last_unstale":"2023-10-18T08:55:16.282778+0000","last_undegraded":"2023-10-18T08:55:16.282778+0000","last_fullsized":"2023-10-18T08:55:16.282778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:41.755639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281449+0000","last_change":"2023-10-18T08:55:15.306465+0000","last_active":"2023-10-18T08:55:16.281449+0000","last_peered":"2023-10-18T08:55:16.281449+0000","last_clean":"2023-10-18T08:55:16.281449+0000","last_became_active":"2023-10-18T08:55:15.305909+0000","last_became_peered":"2023-10-18T08:55:15.305909+0000","last_unstale":"2023-10-18T08:55:16.281449+0000","last_undegraded":"2023-10-18T08:55:16.281449+0000","last_fullsized":"2023-10-18T08:55:16.281449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:02.173932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688851+0000","last_change":"2023-10-18T08:55:15.302350+0000","last_active":"2023-10-18T08:55:16.688851+0000","last_peered":"2023-10-18T08:55:16.688851+0000","last_clean":"2023-10-18T08:55:16.688851+0000","last_became_active":"2023-10-18T08:55:15.302090+0000","last_became_peered":"2023-10-18T08:55:15.302090+0000","last_unstale":"2023-10-18T08:55:16.688851+0000","last_undegraded":"2023-10-18T08:55:16.688851+0000","last_fullsized":"2023-10-18T08:55:16.688851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:19.879235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688836+0000","last_change":"2023-10-18T08:55:15.307679+0000","last_active":"2023-10-18T08:55:16.688836+0000","last_peered":"2023-10-18T08:55:16.688836+0000","last_clean":"2023-10-18T08:55:16.688836+0000","last_became_active":"2023-10-18T08:55:15.306999+0000","last_became_peered":"2023-10-18T08:55:15.306999+0000","last_unstale":"2023-10-18T08:55:16.688836+0000","last_undegraded":"2023-10-18T08:55:16.688836+0000","last_fullsized":"2023-10-18T08:55:16.688836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282830+0000","last_change":"2023-10-18T08:55:15.293552+0000","last_active":"2023-10-18T08:55:16.282830+0000","last_peered":"2023-10-18T08:55:16.282830+0000","last_clean":"2023-10-18T08:55:16.282830+0000","last_became_active":"2023-10-18T08:55:15.292895+0000","last_became_peered":"2023-10-18T08:55:15.292895+0000","last_unstale":"2023-10-18T08:55:16.282830+0000","last_undegraded":"2023-10-18T08:55:16.282830+0000","last_fullsized":"2023-10-18T08:55:16.282830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:07:58.310623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691708+0000","last_change":"2023-10-18T08:55:15.319351+0000","last_active":"2023-10-18T08:55:16.691708+0000","last_peered":"2023-10-18T08:55:16.691708+0000","last_clean":"2023-10-18T08:55:16.691708+0000","last_became_active":"2023-10-18T08:55:15.317663+0000","last_became_peered":"2023-10-18T08:55:15.317663+0000","last_unstale":"2023-10-18T08:55:16.691708+0000","last_undegraded":"2023-10-18T08:55:16.691708+0000","last_fullsized":"2023-10-18T08:55:16.691708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:10.917539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282757+0000","last_change":"2023-10-18T08:55:15.298227+0000","last_active":"2023-10-18T08:55:16.282757+0000","last_peered":"2023-10-18T08:55:16.282757+0000","last_clean":"2023-10-18T08:55:16.282757+0000","last_became_active":"2023-10-18T08:55:15.297630+0000","last_became_peered":"2023-10-18T08:55:15.297630+0000","last_unstale":"2023-10-18T08:55:16.282757+0000","last_undegraded":"2023-10-18T08:55:16.282757+0000","last_fullsized":"2023-10-18T08:55:16.282757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:04.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.279841+0000","last_change":"2023-10-18T08:55:15.307773+0000","last_active":"2023-10-18T08:55:16.279841+0000","last_peered":"2023-10-18T08:55:16.279841+0000","last_clean":"2023-10-18T08:55:16.279841+0000","last_became_active":"2023-10-18T08:55:15.307252+0000","last_became_peered":"2023-10-18T08:55:15.307252+0000","last_unstale":"2023-10-18T08:55:16.279841+0000","last_undegraded":"2023-10-18T08:55:16.279841+0000","last_fullsized":"2023-10-18T08:55:16.279841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:19.879235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282164+0000","last_change":"2023-10-18T08:55:15.308028+0000","last_active":"2023-10-18T08:55:16.282164+0000","last_peered":"2023-10-18T08:55:16.282164+0000","last_clean":"2023-10-18T08:55:16.282164+0000","last_became_active":"2023-10-18T08:55:15.307786+0000","last_became_peered":"2023-10-18T08:55:15.307786+0000","last_unstale":"2023-10-18T08:55:16.282164+0000","last_undegraded":"2023-10-18T08:55:16.282164+0000","last_fullsized":"2023-10-18T08:55:16.282164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:31.888781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691934+0000","last_change":"2023-10-18T08:55:15.307031+0000","last_active":"2023-10-18T08:55:16.691934+0000","last_peered":"2023-10-18T08:55:16.691934+0000","last_clean":"2023-10-18T08:55:16.691934+0000","last_became_active":"2023-10-18T08:55:15.306786+0000","last_became_peered":"2023-10-18T08:55:15.306786+0000","last_unstale":"2023-10-18T08:55:16.691934+0000","last_undegraded":"2023-10-18T08:55:16.691934+0000","last_fullsized":"2023-10-18T08:55:16.691934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:36.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689019+0000","last_change":"2023-10-18T08:55:15.321350+0000","last_active":"2023-10-18T08:55:16.689019+0000","last_peered":"2023-10-18T08:55:16.689019+0000","last_clean":"2023-10-18T08:55:16.689019+0000","last_became_active":"2023-10-18T08:55:15.321114+0000","last_became_peered":"2023-10-18T08:55:15.321114+0000","last_unstale":"2023-10-18T08:55:16.689019+0000","last_undegraded":"2023-10-18T08:55:16.689019+0000","last_fullsized":"2023-10-18T08:55:16.689019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:02.173932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689997+0000","last_change":"2023-10-18T08:55:15.307879+0000","last_active":"2023-10-18T08:55:16.689997+0000","last_peered":"2023-10-18T08:55:16.689997+0000","last_clean":"2023-10-18T08:55:16.689997+0000","last_became_active":"2023-10-18T08:55:15.307599+0000","last_became_peered":"2023-10-18T08:55:15.307599+0000","last_unstale":"2023-10-18T08:55:16.689997+0000","last_undegraded":"2023-10-18T08:55:16.689997+0000","last_fullsized":"2023-10-18T08:55:16.689997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:02.474966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280131+0000","last_change":"2023-10-18T08:55:15.306959+0000","last_active":"2023-10-18T08:55:16.280131+0000","last_peered":"2023-10-18T08:55:16.280131+0000","last_clean":"2023-10-18T08:55:16.280131+0000","last_became_active":"2023-10-18T08:55:15.306003+0000","last_became_peered":"2023-10-18T08:55:15.306003+0000","last_unstale":"2023-10-18T08:55:16.280131+0000","last_undegraded":"2023-10-18T08:55:16.280131+0000","last_fullsized":"2023-10-18T08:55:16.280131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:06.219128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688286+0000","last_change":"2023-10-18T08:55:15.302306+0000","last_active":"2023-10-18T08:55:16.688286+0000","last_peered":"2023-10-18T08:55:16.688286+0000","last_clean":"2023-10-18T08:55:16.688286+0000","last_became_active":"2023-10-18T08:55:15.302027+0000","last_became_peered":"2023-10-18T08:55:15.302027+0000","last_unstale":"2023-10-18T08:55:16.688286+0000","last_undegraded":"2023-10-18T08:55:16.688286+0000","last_fullsized":"2023-10-18T08:55:16.688286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:45.159195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282045+0000","last_change":"2023-10-18T08:55:15.316075+0000","last_active":"2023-10-18T08:55:16.282045+0000","last_peered":"2023-10-18T08:55:16.282045+0000","last_clean":"2023-10-18T08:55:16.282045+0000","last_became_active":"2023-10-18T08:55:15.315569+0000","last_became_peered":"2023-10-18T08:55:15.315569+0000","last_unstale":"2023-10-18T08:55:16.282045+0000","last_undegraded":"2023-10-18T08:55:16.282045+0000","last_fullsized":"2023-10-18T08:55:16.282045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691359+0000","last_change":"2023-10-18T08:55:15.320240+0000","last_active":"2023-10-18T08:55:16.691359+0000","last_peered":"2023-10-18T08:55:16.691359+0000","last_clean":"2023-10-18T08:55:16.691359+0000","last_became_active":"2023-10-18T08:55:15.319558+0000","last_became_peered":"2023-10-18T08:55:15.319558+0000","last_unstale":"2023-10-18T08:55:16.691359+0000","last_undegraded":"2023-10-18T08:55:16.691359+0000","last_fullsized":"2023-10-18T08:55:16.691359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:10.355168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690434+0000","last_change":"2023-10-18T08:55:15.315655+0000","last_active":"2023-10-18T08:55:16.690434+0000","last_peered":"2023-10-18T08:55:16.690434+0000","last_clean":"2023-10-18T08:55:16.690434+0000","last_became_active":"2023-10-18T08:55:15.311919+0000","last_became_peered":"2023-10-18T08:55:15.311919+0000","last_unstale":"2023-10-18T08:55:16.690434+0000","last_undegraded":"2023-10-18T08:55:16.690434+0000","last_fullsized":"2023-10-18T08:55:16.690434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:23:50.828764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.687786+0000","last_change":"2023-10-18T08:55:15.306321+0000","last_active":"2023-10-18T08:55:16.687786+0000","last_peered":"2023-10-18T08:55:16.687786+0000","last_clean":"2023-10-18T08:55:16.687786+0000","last_became_active":"2023-10-18T08:55:15.305696+0000","last_became_peered":"2023-10-18T08:55:15.305696+0000","last_unstale":"2023-10-18T08:55:16.687786+0000","last_undegraded":"2023-10-18T08:55:16.687786+0000","last_fullsized":"2023-10-18T08:55:16.687786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:05.190948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282965+0000","last_change":"2023-10-18T08:55:15.294264+0000","last_active":"2023-10-18T08:55:16.282965+0000","last_peered":"2023-10-18T08:55:16.282965+0000","last_clean":"2023-10-18T08:55:16.282965+0000","last_became_active":"2023-10-18T08:55:15.293654+0000","last_became_peered":"2023-10-18T08:55:15.293654+0000","last_unstale":"2023-10-18T08:55:16.282965+0000","last_undegraded":"2023-10-18T08:55:16.282965+0000","last_fullsized":"2023-10-18T08:55:16.282965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:26.310762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.692252+0000","last_change":"2023-10-18T08:55:15.315803+0000","last_active":"2023-10-18T08:55:16.692252+0000","last_peered":"2023-10-18T08:55:16.692252+0000","last_clean":"2023-10-18T08:55:16.692252+0000","last_became_active":"2023-10-18T08:55:15.315544+0000","last_became_peered":"2023-10-18T08:55:15.315544+0000","last_unstale":"2023-10-18T08:55:16.692252+0000","last_undegraded":"2023-10-18T08:55:16.692252+0000","last_fullsized":"2023-10-18T08:55:16.692252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689490+0000","last_change":"2023-10-18T08:55:15.308154+0000","last_active":"2023-10-18T08:55:16.689490+0000","last_peered":"2023-10-18T08:55:16.689490+0000","last_clean":"2023-10-18T08:55:16.689490+0000","last_became_active":"2023-10-18T08:55:15.307592+0000","last_became_peered":"2023-10-18T08:55:15.307592+0000","last_unstale":"2023-10-18T08:55:16.689490+0000","last_undegraded":"2023-10-18T08:55:16.689490+0000","last_fullsized":"2023-10-18T08:55:16.689490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:06.219128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691182+0000","last_change":"2023-10-18T08:55:15.312342+0000","last_active":"2023-10-18T08:55:16.691182+0000","last_peered":"2023-10-18T08:55:16.691182+0000","last_clean":"2023-10-18T08:55:16.691182+0000","last_became_active":"2023-10-18T08:55:15.311286+0000","last_became_peered":"2023-10-18T08:55:15.311286+0000","last_unstale":"2023-10-18T08:55:16.691182+0000","last_undegraded":"2023-10-18T08:55:16.691182+0000","last_fullsized":"2023-10-18T08:55:16.691182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:07:58.310623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280368+0000","last_change":"2023-10-18T08:55:15.307636+0000","last_active":"2023-10-18T08:55:16.280368+0000","last_peered":"2023-10-18T08:55:16.280368+0000","last_clean":"2023-10-18T08:55:16.280368+0000","last_became_active":"2023-10-18T08:55:15.306703+0000","last_became_peered":"2023-10-18T08:55:15.306703+0000","last_unstale":"2023-10-18T08:55:16.280368+0000","last_undegraded":"2023-10-18T08:55:16.280368+0000","last_fullsized":"2023-10-18T08:55:16.280368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:36.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689173+0000","last_change":"2023-10-18T08:55:15.307528+0000","last_active":"2023-10-18T08:55:16.689173+0000","last_peered":"2023-10-18T08:55:16.689173+0000","last_clean":"2023-10-18T08:55:16.689173+0000","last_became_active":"2023-10-18T08:55:15.306757+0000","last_became_peered":"2023-10-18T08:55:15.306757+0000","last_unstale":"2023-10-18T08:55:16.689173+0000","last_undegraded":"2023-10-18T08:55:16.689173+0000","last_fullsized":"2023-10-18T08:55:16.689173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:31.888781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280839+0000","last_change":"2023-10-18T08:55:15.316136+0000","last_active":"2023-10-18T08:55:16.280839+0000","last_peered":"2023-10-18T08:55:16.280839+0000","last_clean":"2023-10-18T08:55:16.280839+0000","last_became_active":"2023-10-18T08:55:15.315888+0000","last_became_peered":"2023-10-18T08:55:15.315888+0000","last_unstale":"2023-10-18T08:55:16.280839+0000","last_undegraded":"2023-10-18T08:55:16.280839+0000","last_fullsized":"2023-10-18T08:55:16.280839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:23:50.828764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689385+0000","last_change":"2023-10-18T08:55:15.309215+0000","last_active":"2023-10-18T08:55:16.689385+0000","last_peered":"2023-10-18T08:55:16.689385+0000","last_clean":"2023-10-18T08:55:16.689385+0000","last_became_active":"2023-10-18T08:55:15.308858+0000","last_became_peered":"2023-10-18T08:55:15.308858+0000","last_unstale":"2023-10-18T08:55:16.689385+0000","last_undegraded":"2023-10-18T08:55:16.689385+0000","last_fullsized":"2023-10-18T08:55:16.689385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:06.219128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281659+0000","last_change":"2023-10-18T08:55:15.308660+0000","last_active":"2023-10-18T08:55:16.281659+0000","last_peered":"2023-10-18T08:55:16.281659+0000","last_clean":"2023-10-18T08:55:16.281659+0000","last_became_active":"2023-10-18T08:55:15.306230+0000","last_became_peered":"2023-10-18T08:55:15.306230+0000","last_unstale":"2023-10-18T08:55:16.281659+0000","last_undegraded":"2023-10-18T08:55:16.281659+0000","last_fullsized":"2023-10-18T08:55:16.281659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:06.219128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280534+0000","last_change":"2023-10-18T08:55:15.307486+0000","last_active":"2023-10-18T08:55:16.280534+0000","last_peered":"2023-10-18T08:55:16.280534+0000","last_clean":"2023-10-18T08:55:16.280534+0000","last_became_active":"2023-10-18T08:55:15.306357+0000","last_became_peered":"2023-10-18T08:55:15.306357+0000","last_unstale":"2023-10-18T08:55:16.280534+0000","last_undegraded":"2023-10-18T08:55:16.280534+0000","last_fullsized":"2023-10-18T08:55:16.280534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:31.888781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688669+0000","last_change":"2023-10-18T08:55:15.300645+0000","last_active":"2023-10-18T08:55:16.688669+0000","last_peered":"2023-10-18T08:55:16.688669+0000","last_clean":"2023-10-18T08:55:16.688669+0000","last_became_active":"2023-10-18T08:55:15.299795+0000","last_became_peered":"2023-10-18T08:55:15.299795+0000","last_unstale":"2023-10-18T08:55:16.688669+0000","last_undegraded":"2023-10-18T08:55:16.688669+0000","last_fullsized":"2023-10-18T08:55:16.688669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:21.842439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690857+0000","last_change":"2023-10-18T08:55:15.305929+0000","last_active":"2023-10-18T08:55:16.690857+0000","last_peered":"2023-10-18T08:55:16.690857+0000","last_clean":"2023-10-18T08:55:16.690857+0000","last_became_active":"2023-10-18T08:55:15.301322+0000","last_became_peered":"2023-10-18T08:55:15.301322+0000","last_unstale":"2023-10-18T08:55:16.690857+0000","last_undegraded":"2023-10-18T08:55:16.690857+0000","last_fullsized":"2023-10-18T08:55:16.690857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:41.755639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.687612+0000","last_change":"2023-10-18T08:55:15.320868+0000","last_active":"2023-10-18T08:55:16.687612+0000","last_peered":"2023-10-18T08:55:16.687612+0000","last_clean":"2023-10-18T08:55:16.687612+0000","last_became_active":"2023-10-18T08:55:15.320606+0000","last_became_peered":"2023-10-18T08:55:15.320606+0000","last_unstale":"2023-10-18T08:55:16.687612+0000","last_undegraded":"2023-10-18T08:55:16.687612+0000","last_fullsized":"2023-10-18T08:55:16.687612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:04.364364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690001+0000","last_change":"2023-10-18T08:55:15.314269+0000","last_active":"2023-10-18T08:55:16.690001+0000","last_peered":"2023-10-18T08:55:16.690001+0000","last_clean":"2023-10-18T08:55:16.690001+0000","last_became_active":"2023-10-18T08:55:15.314011+0000","last_became_peered":"2023-10-18T08:55:15.314011+0000","last_unstale":"2023-10-18T08:55:16.690001+0000","last_undegraded":"2023-10-18T08:55:16.690001+0000","last_fullsized":"2023-10-18T08:55:16.690001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:28.148282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689559+0000","last_change":"2023-10-18T08:55:15.306134+0000","last_active":"2023-10-18T08:55:16.689559+0000","last_peered":"2023-10-18T08:55:16.689559+0000","last_clean":"2023-10-18T08:55:16.689559+0000","last_became_active":"2023-10-18T08:55:15.305389+0000","last_became_peered":"2023-10-18T08:55:15.305389+0000","last_unstale":"2023-10-18T08:55:16.689559+0000","last_undegraded":"2023-10-18T08:55:16.689559+0000","last_fullsized":"2023-10-18T08:55:16.689559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:32.273116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.327693+0000","last_change":"2023-10-18T08:55:14.290336+0000","last_active":"2023-10-18T08:55:15.327693+0000","last_peered":"2023-10-18T08:55:15.327693+0000","last_clean":"2023-10-18T08:55:15.327693+0000","last_became_active":"2023-10-18T08:55:14.290051+0000","last_became_peered":"2023-10-18T08:55:14.290051+0000","last_unstale":"2023-10-18T08:55:15.327693+0000","last_undegraded":"2023-10-18T08:55:15.327693+0000","last_fullsized":"2023-10-18T08:55:15.327693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:38.968705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689230+0000","last_change":"2023-10-18T08:55:15.302202+0000","last_active":"2023-10-18T08:55:16.689230+0000","last_peered":"2023-10-18T08:55:16.689230+0000","last_clean":"2023-10-18T08:55:16.689230+0000","last_became_active":"2023-10-18T08:55:15.301534+0000","last_became_peered":"2023-10-18T08:55:15.301534+0000","last_unstale":"2023-10-18T08:55:16.689230+0000","last_undegraded":"2023-10-18T08:55:16.689230+0000","last_fullsized":"2023-10-18T08:55:16.689230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:27.650596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.326541+0000","last_change":"2023-10-18T08:55:14.296292+0000","last_active":"2023-10-18T08:55:15.326541+0000","last_peered":"2023-10-18T08:55:15.326541+0000","last_clean":"2023-10-18T08:55:15.326541+0000","last_became_active":"2023-10-18T08:55:14.295937+0000","last_became_peered":"2023-10-18T08:55:14.295937+0000","last_unstale":"2023-10-18T08:55:15.326541+0000","last_undegraded":"2023-10-18T08:55:15.326541+0000","last_fullsized":"2023-10-18T08:55:15.326541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:38.968705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.280268+0000","last_change":"2023-10-18T08:55:14.293089+0000","last_active":"2023-10-18T08:55:15.280268+0000","last_peered":"2023-10-18T08:55:15.280268+0000","last_clean":"2023-10-18T08:55:15.280268+0000","last_became_active":"2023-10-18T08:55:14.292273+0000","last_became_peered":"2023-10-18T08:55:14.292273+0000","last_unstale":"2023-10-18T08:55:15.280268+0000","last_undegraded":"2023-10-18T08:55:15.280268+0000","last_fullsized":"2023-10-18T08:55:15.280268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:03.359656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281004+0000","last_change":"2023-10-18T08:55:15.297969+0000","last_active":"2023-10-18T08:55:16.281004+0000","last_peered":"2023-10-18T08:55:16.281004+0000","last_clean":"2023-10-18T08:55:16.281004+0000","last_became_active":"2023-10-18T08:55:15.297730+0000","last_became_peered":"2023-10-18T08:55:15.297730+0000","last_unstale":"2023-10-18T08:55:16.281004+0000","last_undegraded":"2023-10-18T08:55:16.281004+0000","last_fullsized":"2023-10-18T08:55:16.281004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:04.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.301050+0000","last_change":"2023-10-18T08:55:14.299406+0000","last_active":"2023-10-18T08:55:15.301050+0000","last_peered":"2023-10-18T08:55:15.301050+0000","last_clean":"2023-10-18T08:55:15.301050+0000","last_became_active":"2023-10-18T08:55:14.299181+0000","last_became_peered":"2023-10-18T08:55:14.299181+0000","last_unstale":"2023-10-18T08:55:15.301050+0000","last_undegraded":"2023-10-18T08:55:15.301050+0000","last_fullsized":"2023-10-18T08:55:15.301050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:18.874528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281450+0000","last_change":"2023-10-18T08:55:15.309032+0000","last_active":"2023-10-18T08:55:16.281450+0000","last_peered":"2023-10-18T08:55:16.281450+0000","last_clean":"2023-10-18T08:55:16.281450+0000","last_became_active":"2023-10-18T08:55:15.308756+0000","last_became_peered":"2023-10-18T08:55:15.308756+0000","last_unstale":"2023-10-18T08:55:16.281450+0000","last_undegraded":"2023-10-18T08:55:16.281450+0000","last_fullsized":"2023-10-18T08:55:16.281450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:27.650596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.297946+0000","last_change":"2023-10-18T08:55:14.304174+0000","last_active":"2023-10-18T08:55:15.297946+0000","last_peered":"2023-10-18T08:55:15.297946+0000","last_clean":"2023-10-18T08:55:15.297946+0000","last_became_active":"2023-10-18T08:55:14.303968+0000","last_became_peered":"2023-10-18T08:55:14.303968+0000","last_unstale":"2023-10-18T08:55:15.297946+0000","last_undegraded":"2023-10-18T08:55:15.297946+0000","last_fullsized":"2023-10-18T08:55:15.297946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:04.186241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690815+0000","last_change":"2023-10-18T08:55:15.312484+0000","last_active":"2023-10-18T08:55:16.690815+0000","last_peered":"2023-10-18T08:55:16.690815+0000","last_clean":"2023-10-18T08:55:16.690815+0000","last_became_active":"2023-10-18T08:55:15.311626+0000","last_became_peered":"2023-10-18T08:55:15.311626+0000","last_unstale":"2023-10-18T08:55:16.690815+0000","last_undegraded":"2023-10-18T08:55:16.690815+0000","last_fullsized":"2023-10-18T08:55:16.690815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:04.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.300021+0000","last_change":"2023-10-18T08:55:14.302651+0000","last_active":"2023-10-18T08:55:15.300021+0000","last_peered":"2023-10-18T08:55:15.300021+0000","last_clean":"2023-10-18T08:55:15.300021+0000","last_became_active":"2023-10-18T08:55:14.302029+0000","last_became_peered":"2023-10-18T08:55:14.302029+0000","last_unstale":"2023-10-18T08:55:15.300021+0000","last_undegraded":"2023-10-18T08:55:15.300021+0000","last_fullsized":"2023-10-18T08:55:15.300021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:44.154488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689826+0000","last_change":"2023-10-18T08:55:15.322350+0000","last_active":"2023-10-18T08:55:16.689826+0000","last_peered":"2023-10-18T08:55:16.689826+0000","last_clean":"2023-10-18T08:55:16.689826+0000","last_became_active":"2023-10-18T08:55:15.322002+0000","last_became_peered":"2023-10-18T08:55:15.322002+0000","last_unstale":"2023-10-18T08:55:16.689826+0000","last_undegraded":"2023-10-18T08:55:16.689826+0000","last_fullsized":"2023-10-18T08:55:16.689826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:36.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.326702+0000","last_change":"2023-10-18T08:55:14.300549+0000","last_active":"2023-10-18T08:55:15.326702+0000","last_peered":"2023-10-18T08:55:15.326702+0000","last_clean":"2023-10-18T08:55:15.326702+0000","last_became_active":"2023-10-18T08:55:14.299990+0000","last_became_peered":"2023-10-18T08:55:14.299990+0000","last_unstale":"2023-10-18T08:55:15.326702+0000","last_undegraded":"2023-10-18T08:55:15.326702+0000","last_fullsized":"2023-10-18T08:55:15.326702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:37.505319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690659+0000","last_change":"2023-10-18T08:55:15.314324+0000","last_active":"2023-10-18T08:55:16.690659+0000","last_peered":"2023-10-18T08:55:16.690659+0000","last_clean":"2023-10-18T08:55:16.690659+0000","last_became_active":"2023-10-18T08:55:15.313890+0000","last_became_peered":"2023-10-18T08:55:15.313890+0000","last_unstale":"2023-10-18T08:55:16.690659+0000","last_undegraded":"2023-10-18T08:55:16.690659+0000","last_fullsized":"2023-10-18T08:55:16.690659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:05.190948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.329951+0000","last_change":"2023-10-18T08:55:14.292962+0000","last_active":"2023-10-18T08:55:15.329951+0000","last_peered":"2023-10-18T08:55:15.329951+0000","last_clean":"2023-10-18T08:55:15.329951+0000","last_became_active":"2023-10-18T08:55:14.292424+0000","last_became_peered":"2023-10-18T08:55:14.292424+0000","last_unstale":"2023-10-18T08:55:15.329951+0000","last_undegraded":"2023-10-18T08:55:15.329951+0000","last_fullsized":"2023-10-18T08:55:15.329951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:20.837731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282227+0000","last_change":"2023-10-18T08:55:15.296745+0000","last_active":"2023-10-18T08:55:16.282227+0000","last_peered":"2023-10-18T08:55:16.282227+0000","last_clean":"2023-10-18T08:55:16.282227+0000","last_became_active":"2023-10-18T08:55:15.296095+0000","last_became_peered":"2023-10-18T08:55:15.296095+0000","last_unstale":"2023-10-18T08:55:16.282227+0000","last_undegraded":"2023-10-18T08:55:16.282227+0000","last_fullsized":"2023-10-18T08:55:16.282227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.688019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.328191+0000","last_change":"2023-10-18T08:55:14.299318+0000","last_active":"2023-10-18T08:55:15.328191+0000","last_peered":"2023-10-18T08:55:15.328191+0000","last_clean":"2023-10-18T08:55:15.328191+0000","last_became_active":"2023-10-18T08:55:14.299069+0000","last_became_peered":"2023-10-18T08:55:14.299069+0000","last_unstale":"2023-10-18T08:55:15.328191+0000","last_undegraded":"2023-10-18T08:55:15.328191+0000","last_fullsized":"2023-10-18T08:55:15.328191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:03.359656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690146+0000","last_change":"2023-10-18T08:55:15.307886+0000","last_active":"2023-10-18T08:55:16.690146+0000","last_peered":"2023-10-18T08:55:16.690146+0000","last_clean":"2023-10-18T08:55:16.690146+0000","last_became_active":"2023-10-18T08:55:15.306465+0000","last_became_peered":"2023-10-18T08:55:15.306465+0000","last_unstale":"2023-10-18T08:55:16.690146+0000","last_undegraded":"2023-10-18T08:55:16.690146+0000","last_fullsized":"2023-10-18T08:55:16.690146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:31.888781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.316152+0000","last_change":"2023-10-18T08:55:14.305151+0000","last_active":"2023-10-18T08:55:15.316152+0000","last_peered":"2023-10-18T08:55:15.316152+0000","last_clean":"2023-10-18T08:55:15.316152+0000","last_became_active":"2023-10-18T08:55:14.303161+0000","last_became_peered":"2023-10-18T08:55:14.303161+0000","last_unstale":"2023-10-18T08:55:15.316152+0000","last_undegraded":"2023-10-18T08:55:15.316152+0000","last_fullsized":"2023-10-18T08:55:15.316152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:26.645889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.283192+0000","last_change":"2023-10-18T08:55:15.317138+0000","last_active":"2023-10-18T08:55:16.283192+0000","last_peered":"2023-10-18T08:55:16.283192+0000","last_clean":"2023-10-18T08:55:16.283192+0000","last_became_active":"2023-10-18T08:55:15.316743+0000","last_became_peered":"2023-10-18T08:55:15.316743+0000","last_unstale":"2023-10-18T08:55:16.283192+0000","last_undegraded":"2023-10-18T08:55:16.283192+0000","last_fullsized":"2023-10-18T08:55:16.283192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:46.921031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.303320+0000","last_change":"2023-10-18T08:55:14.294348+0000","last_active":"2023-10-18T08:55:15.303320+0000","last_peered":"2023-10-18T08:55:15.303320+0000","last_clean":"2023-10-18T08:55:15.303320+0000","last_became_active":"2023-10-18T08:55:14.293456+0000","last_became_peered":"2023-10-18T08:55:14.293456+0000","last_unstale":"2023-10-18T08:55:15.303320+0000","last_undegraded":"2023-10-18T08:55:15.303320+0000","last_fullsized":"2023-10-18T08:55:15.303320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:37.505319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689020+0000","last_change":"2023-10-18T08:55:15.304772+0000","last_active":"2023-10-18T08:55:16.689020+0000","last_peered":"2023-10-18T08:55:16.689020+0000","last_clean":"2023-10-18T08:55:16.689020+0000","last_became_active":"2023-10-18T08:55:15.304528+0000","last_became_peered":"2023-10-18T08:55:15.304528+0000","last_unstale":"2023-10-18T08:55:16.689020+0000","last_undegraded":"2023-10-18T08:55:16.689020+0000","last_fullsized":"2023-10-18T08:55:16.689020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:27.650596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.330488+0000","last_change":"2023-10-18T08:55:14.297308+0000","last_active":"2023-10-18T08:55:15.330488+0000","last_peered":"2023-10-18T08:55:15.330488+0000","last_clean":"2023-10-18T08:55:15.330488+0000","last_became_active":"2023-10-18T08:55:14.297069+0000","last_became_peered":"2023-10-18T08:55:14.297069+0000","last_unstale":"2023-10-18T08:55:15.330488+0000","last_undegraded":"2023-10-18T08:55:15.330488+0000","last_fullsized":"2023-10-18T08:55:15.330488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:10.814889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.283167+0000","last_change":"2023-10-18T08:55:15.303171+0000","last_active":"2023-10-18T08:55:16.283167+0000","last_peered":"2023-10-18T08:55:16.283167+0000","last_clean":"2023-10-18T08:55:16.283167+0000","last_became_active":"2023-10-18T08:55:15.301815+0000","last_became_peered":"2023-10-18T08:55:15.301815+0000","last_unstale":"2023-10-18T08:55:16.283167+0000","last_undegraded":"2023-10-18T08:55:16.283167+0000","last_fullsized":"2023-10-18T08:55:16.283167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:10.355168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.325923+0000","last_change":"2023-10-18T08:55:14.298243+0000","last_active":"2023-10-18T08:55:15.325923+0000","last_peered":"2023-10-18T08:55:15.325923+0000","last_clean":"2023-10-18T08:55:15.325923+0000","last_became_active":"2023-10-18T08:55:14.297343+0000","last_became_peered":"2023-10-18T08:55:14.297343+0000","last_unstale":"2023-10-18T08:55:15.325923+0000","last_undegraded":"2023-10-18T08:55:15.325923+0000","last_fullsized":"2023-10-18T08:55:15.325923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:10.814889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.283511+0000","last_change":"2023-10-18T08:55:15.316986+0000","last_active":"2023-10-18T08:55:16.283511+0000","last_peered":"2023-10-18T08:55:16.283511+0000","last_clean":"2023-10-18T08:55:16.283511+0000","last_became_active":"2023-10-18T08:55:15.316398+0000","last_became_peered":"2023-10-18T08:55:15.316398+0000","last_unstale":"2023-10-18T08:55:16.283511+0000","last_undegraded":"2023-10-18T08:55:16.283511+0000","last_fullsized":"2023-10-18T08:55:16.283511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:56:45.727823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.329694+0000","last_change":"2023-10-18T08:55:14.290070+0000","last_active":"2023-10-18T08:55:15.329694+0000","last_peered":"2023-10-18T08:55:15.329694+0000","last_clean":"2023-10-18T08:55:15.329694+0000","last_became_active":"2023-10-18T08:55:14.289756+0000","last_became_peered":"2023-10-18T08:55:14.289756+0000","last_unstale":"2023-10-18T08:55:15.329694+0000","last_undegraded":"2023-10-18T08:55:15.329694+0000","last_fullsized":"2023-10-18T08:55:15.329694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:01.470259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282548+0000","last_change":"2023-10-18T08:55:15.311283+0000","last_active":"2023-10-18T08:55:16.282548+0000","last_peered":"2023-10-18T08:55:16.282548+0000","last_clean":"2023-10-18T08:55:16.282548+0000","last_became_active":"2023-10-18T08:55:15.311053+0000","last_became_peered":"2023-10-18T08:55:15.311053+0000","last_unstale":"2023-10-18T08:55:16.282548+0000","last_undegraded":"2023-10-18T08:55:16.282548+0000","last_fullsized":"2023-10-18T08:55:16.282548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:23:50.828764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.325915+0000","last_change":"2023-10-18T08:55:14.294635+0000","last_active":"2023-10-18T08:55:15.325915+0000","last_peered":"2023-10-18T08:55:15.325915+0000","last_clean":"2023-10-18T08:55:15.325915+0000","last_became_active":"2023-10-18T08:55:14.294341+0000","last_became_peered":"2023-10-18T08:55:14.294341+0000","last_unstale":"2023-10-18T08:55:15.325915+0000","last_undegraded":"2023-10-18T08:55:15.325915+0000","last_fullsized":"2023-10-18T08:55:15.325915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:01.470259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691402+0000","last_change":"2023-10-18T08:55:15.312145+0000","last_active":"2023-10-18T08:55:16.691402+0000","last_peered":"2023-10-18T08:55:16.691402+0000","last_clean":"2023-10-18T08:55:16.691402+0000","last_became_active":"2023-10-18T08:55:15.310296+0000","last_became_peered":"2023-10-18T08:55:15.310296+0000","last_unstale":"2023-10-18T08:55:16.691402+0000","last_undegraded":"2023-10-18T08:55:16.691402+0000","last_fullsized":"2023-10-18T08:55:16.691402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:46.921031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.282183+0000","last_change":"2023-10-18T08:55:14.303903+0000","last_active":"2023-10-18T08:55:15.282183+0000","last_peered":"2023-10-18T08:55:15.282183+0000","last_clean":"2023-10-18T08:55:15.282183+0000","last_became_active":"2023-10-18T08:55:14.303633+0000","last_became_peered":"2023-10-18T08:55:14.303633+0000","last_unstale":"2023-10-18T08:55:15.282183+0000","last_undegraded":"2023-10-18T08:55:15.282183+0000","last_fullsized":"2023-10-18T08:55:15.282183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:16:56.668322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.283316+0000","last_change":"2023-10-18T08:55:15.318598+0000","last_active":"2023-10-18T08:55:16.283316+0000","last_peered":"2023-10-18T08:55:16.283316+0000","last_clean":"2023-10-18T08:55:16.283316+0000","last_became_active":"2023-10-18T08:55:15.318353+0000","last_became_peered":"2023-10-18T08:55:15.318353+0000","last_unstale":"2023-10-18T08:55:16.283316+0000","last_undegraded":"2023-10-18T08:55:16.283316+0000","last_fullsized":"2023-10-18T08:55:16.283316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:10.917539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.287158+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:55:15.287158+0000","last_peered":"2023-10-18T08:55:15.287158+0000","last_clean":"2023-10-18T08:55:15.287158+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:55:15.287158+0000","last_undegraded":"2023-10-18T08:55:15.287158+0000","last_fullsized":"2023-10-18T08:55:15.287158+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:34.881744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.316324+0000","last_change":"2023-10-18T08:55:14.305589+0000","last_active":"2023-10-18T08:55:15.316324+0000","last_peered":"2023-10-18T08:55:15.316324+0000","last_clean":"2023-10-18T08:55:15.316324+0000","last_became_active":"2023-10-18T08:55:14.303723+0000","last_became_peered":"2023-10-18T08:55:14.303723+0000","last_unstale":"2023-10-18T08:55:15.316324+0000","last_undegraded":"2023-10-18T08:55:15.316324+0000","last_fullsized":"2023-10-18T08:55:15.316324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:05.214421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280180+0000","last_change":"2023-10-18T08:55:15.307598+0000","last_active":"2023-10-18T08:55:16.280180+0000","last_peered":"2023-10-18T08:55:16.280180+0000","last_clean":"2023-10-18T08:55:16.280180+0000","last_became_active":"2023-10-18T08:55:15.307263+0000","last_became_peered":"2023-10-18T08:55:15.307263+0000","last_unstale":"2023-10-18T08:55:16.280180+0000","last_undegraded":"2023-10-18T08:55:16.280180+0000","last_fullsized":"2023-10-18T08:55:16.280180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:28.148282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.286898+0000","last_change":"2023-10-18T08:55:14.299430+0000","last_active":"2023-10-18T08:55:15.286898+0000","last_peered":"2023-10-18T08:55:15.286898+0000","last_clean":"2023-10-18T08:55:15.286898+0000","last_became_active":"2023-10-18T08:55:14.299004+0000","last_became_peered":"2023-10-18T08:55:14.299004+0000","last_unstale":"2023-10-18T08:55:15.286898+0000","last_undegraded":"2023-10-18T08:55:15.286898+0000","last_fullsized":"2023-10-18T08:55:15.286898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:26.645889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689865+0000","last_change":"2023-10-18T08:55:15.308337+0000","last_active":"2023-10-18T08:55:16.689865+0000","last_peered":"2023-10-18T08:55:16.689865+0000","last_clean":"2023-10-18T08:55:16.689865+0000","last_became_active":"2023-10-18T08:55:15.305146+0000","last_became_peered":"2023-10-18T08:55:15.305146+0000","last_unstale":"2023-10-18T08:55:16.689865+0000","last_undegraded":"2023-10-18T08:55:16.689865+0000","last_fullsized":"2023-10-18T08:55:16.689865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.283153+0000","last_change":"2023-10-18T08:55:14.287811+0000","last_active":"2023-10-18T08:55:15.283153+0000","last_peered":"2023-10-18T08:55:15.283153+0000","last_clean":"2023-10-18T08:55:15.283153+0000","last_became_active":"2023-10-18T08:55:14.287466+0000","last_became_peered":"2023-10-18T08:55:14.287466+0000","last_unstale":"2023-10-18T08:55:15.283153+0000","last_undegraded":"2023-10-18T08:55:15.283153+0000","last_fullsized":"2023-10-18T08:55:15.283153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:31.268409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281786+0000","last_change":"2023-10-18T08:55:15.301141+0000","last_active":"2023-10-18T08:55:16.281786+0000","last_peered":"2023-10-18T08:55:16.281786+0000","last_clean":"2023-10-18T08:55:16.281786+0000","last_became_active":"2023-10-18T08:55:15.300535+0000","last_became_peered":"2023-10-18T08:55:15.300535+0000","last_unstale":"2023-10-18T08:55:16.281786+0000","last_undegraded":"2023-10-18T08:55:16.281786+0000","last_fullsized":"2023-10-18T08:55:16.281786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:28.148282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.300341+0000","last_change":"2023-10-18T08:55:14.302819+0000","last_active":"2023-10-18T08:55:15.300341+0000","last_peered":"2023-10-18T08:55:15.300341+0000","last_clean":"2023-10-18T08:55:15.300341+0000","last_became_active":"2023-10-18T08:55:14.302381+0000","last_became_peered":"2023-10-18T08:55:14.302381+0000","last_unstale":"2023-10-18T08:55:15.300341+0000","last_undegraded":"2023-10-18T08:55:15.300341+0000","last_fullsized":"2023-10-18T08:55:15.300341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:20.837731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690460+0000","last_change":"2023-10-18T08:55:15.309720+0000","last_active":"2023-10-18T08:55:16.690460+0000","last_peered":"2023-10-18T08:55:16.690460+0000","last_clean":"2023-10-18T08:55:16.690460+0000","last_became_active":"2023-10-18T08:55:15.309469+0000","last_became_peered":"2023-10-18T08:55:15.309469+0000","last_unstale":"2023-10-18T08:55:16.690460+0000","last_undegraded":"2023-10-18T08:55:16.690460+0000","last_fullsized":"2023-10-18T08:55:16.690460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:04.364364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.295166+0000","last_change":"2023-10-18T08:55:14.296298+0000","last_active":"2023-10-18T08:55:15.295166+0000","last_peered":"2023-10-18T08:55:15.295166+0000","last_clean":"2023-10-18T08:55:15.295166+0000","last_became_active":"2023-10-18T08:55:14.295788+0000","last_became_peered":"2023-10-18T08:55:14.295788+0000","last_unstale":"2023-10-18T08:55:15.295166+0000","last_undegraded":"2023-10-18T08:55:15.295166+0000","last_fullsized":"2023-10-18T08:55:15.295166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:38.968705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689511+0000","last_change":"2023-10-18T08:55:15.307391+0000","last_active":"2023-10-18T08:55:16.689511+0000","last_peered":"2023-10-18T08:55:16.689511+0000","last_clean":"2023-10-18T08:55:16.689511+0000","last_became_active":"2023-10-18T08:55:15.306422+0000","last_became_peered":"2023-10-18T08:55:15.306422+0000","last_unstale":"2023-10-18T08:55:16.689511+0000","last_undegraded":"2023-10-18T08:55:16.689511+0000","last_fullsized":"2023-10-18T08:55:16.689511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.688019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.286110+0000","last_change":"2023-10-18T08:55:14.299281+0000","last_active":"2023-10-18T08:55:15.286110+0000","last_peered":"2023-10-18T08:55:15.286110+0000","last_clean":"2023-10-18T08:55:15.286110+0000","last_became_active":"2023-10-18T08:55:14.298698+0000","last_became_peered":"2023-10-18T08:55:14.298698+0000","last_unstale":"2023-10-18T08:55:15.286110+0000","last_undegraded":"2023-10-18T08:55:15.286110+0000","last_fullsized":"2023-10-18T08:55:15.286110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:44.154488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281874+0000","last_change":"2023-10-18T08:55:15.299892+0000","last_active":"2023-10-18T08:55:16.281874+0000","last_peered":"2023-10-18T08:55:16.281874+0000","last_clean":"2023-10-18T08:55:16.281874+0000","last_became_active":"2023-10-18T08:55:15.299281+0000","last_became_peered":"2023-10-18T08:55:15.299281+0000","last_unstale":"2023-10-18T08:55:16.281874+0000","last_undegraded":"2023-10-18T08:55:16.281874+0000","last_fullsized":"2023-10-18T08:55:16.281874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:36.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281731+0000","last_change":"2023-10-18T08:55:14.300359+0000","last_active":"2023-10-18T08:55:16.281731+0000","last_peered":"2023-10-18T08:55:16.281731+0000","last_clean":"2023-10-18T08:55:16.281731+0000","last_became_active":"2023-10-18T08:55:14.299423+0000","last_became_peered":"2023-10-18T08:55:14.299423+0000","last_unstale":"2023-10-18T08:55:16.281731+0000","last_undegraded":"2023-10-18T08:55:16.281731+0000","last_fullsized":"2023-10-18T08:55:16.281731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:44.154488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280011+0000","last_change":"2023-10-18T08:55:15.288327+0000","last_active":"2023-10-18T08:55:16.280011+0000","last_peered":"2023-10-18T08:55:16.280011+0000","last_clean":"2023-10-18T08:55:16.280011+0000","last_became_active":"2023-10-18T08:55:15.287648+0000","last_became_peered":"2023-10-18T08:55:15.287648+0000","last_unstale":"2023-10-18T08:55:16.280011+0000","last_undegraded":"2023-10-18T08:55:16.280011+0000","last_fullsized":"2023-10-18T08:55:16.280011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:27.650596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.327662+0000","last_change":"2023-10-18T08:55:14.294992+0000","last_active":"2023-10-18T08:55:15.327662+0000","last_peered":"2023-10-18T08:55:15.327662+0000","last_clean":"2023-10-18T08:55:15.327662+0000","last_became_active":"2023-10-18T08:55:14.294682+0000","last_became_peered":"2023-10-18T08:55:14.294682+0000","last_unstale":"2023-10-18T08:55:15.327662+0000","last_undegraded":"2023-10-18T08:55:15.327662+0000","last_fullsized":"2023-10-18T08:55:15.327662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:44.154488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689214+0000","last_change":"2023-10-18T08:55:15.308496+0000","last_active":"2023-10-18T08:55:16.689214+0000","last_peered":"2023-10-18T08:55:16.689214+0000","last_clean":"2023-10-18T08:55:16.689214+0000","last_became_active":"2023-10-18T08:55:15.306095+0000","last_became_peered":"2023-10-18T08:55:15.306095+0000","last_unstale":"2023-10-18T08:55:16.689214+0000","last_undegraded":"2023-10-18T08:55:16.689214+0000","last_fullsized":"2023-10-18T08:55:16.689214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:02.173932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.288529+0000","last_change":"2023-10-18T08:55:14.299395+0000","last_active":"2023-10-18T08:55:15.288529+0000","last_peered":"2023-10-18T08:55:15.288529+0000","last_clean":"2023-10-18T08:55:15.288529+0000","last_became_active":"2023-10-18T08:55:14.299130+0000","last_became_peered":"2023-10-18T08:55:14.299130+0000","last_unstale":"2023-10-18T08:55:15.288529+0000","last_undegraded":"2023-10-18T08:55:15.288529+0000","last_fullsized":"2023-10-18T08:55:15.288529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:35.197625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.693084+0000","last_change":"2023-10-18T08:55:15.314704+0000","last_active":"2023-10-18T08:55:16.693084+0000","last_peered":"2023-10-18T08:55:16.693084+0000","last_clean":"2023-10-18T08:55:16.693084+0000","last_became_active":"2023-10-18T08:55:15.314153+0000","last_became_peered":"2023-10-18T08:55:15.314153+0000","last_unstale":"2023-10-18T08:55:16.693084+0000","last_undegraded":"2023-10-18T08:55:16.693084+0000","last_fullsized":"2023-10-18T08:55:16.693084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:39.973412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.295022+0000","last_change":"2023-10-18T08:55:14.297463+0000","last_active":"2023-10-18T08:55:15.295022+0000","last_peered":"2023-10-18T08:55:15.295022+0000","last_clean":"2023-10-18T08:55:15.295022+0000","last_became_active":"2023-10-18T08:55:14.297199+0000","last_became_peered":"2023-10-18T08:55:14.297199+0000","last_unstale":"2023-10-18T08:55:15.295022+0000","last_undegraded":"2023-10-18T08:55:15.295022+0000","last_fullsized":"2023-10-18T08:55:15.295022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:04.186241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.279960+0000","last_change":"2023-10-18T08:55:15.303892+0000","last_active":"2023-10-18T08:55:16.279960+0000","last_peered":"2023-10-18T08:55:16.279960+0000","last_clean":"2023-10-18T08:55:16.279960+0000","last_became_active":"2023-10-18T08:55:15.303545+0000","last_became_peered":"2023-10-18T08:55:15.303545+0000","last_unstale":"2023-10-18T08:55:16.279960+0000","last_undegraded":"2023-10-18T08:55:16.279960+0000","last_fullsized":"2023-10-18T08:55:16.279960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:02.173932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.297222+0000","last_change":"2023-10-18T08:55:14.294024+0000","last_active":"2023-10-18T08:55:15.297222+0000","last_peered":"2023-10-18T08:55:15.297222+0000","last_clean":"2023-10-18T08:55:15.297222+0000","last_became_active":"2023-10-18T08:55:14.293763+0000","last_became_peered":"2023-10-18T08:55:14.293763+0000","last_unstale":"2023-10-18T08:55:15.297222+0000","last_undegraded":"2023-10-18T08:55:15.297222+0000","last_fullsized":"2023-10-18T08:55:15.297222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:20.837731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280694+0000","last_change":"2023-10-18T08:55:15.302069+0000","last_active":"2023-10-18T08:55:16.280694+0000","last_peered":"2023-10-18T08:55:16.280694+0000","last_clean":"2023-10-18T08:55:16.280694+0000","last_became_active":"2023-10-18T08:55:15.301806+0000","last_became_peered":"2023-10-18T08:55:15.301806+0000","last_unstale":"2023-10-18T08:55:16.280694+0000","last_undegraded":"2023-10-18T08:55:16.280694+0000","last_fullsized":"2023-10-18T08:55:16.280694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.688019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.327881+0000","last_change":"2023-10-18T08:55:14.292655+0000","last_active":"2023-10-18T08:55:15.327881+0000","last_peered":"2023-10-18T08:55:15.327881+0000","last_clean":"2023-10-18T08:55:15.327881+0000","last_became_active":"2023-10-18T08:55:14.292291+0000","last_became_peered":"2023-10-18T08:55:14.292291+0000","last_unstale":"2023-10-18T08:55:15.327881+0000","last_undegraded":"2023-10-18T08:55:15.327881+0000","last_fullsized":"2023-10-18T08:55:15.327881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:57.132126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688748+0000","last_change":"2023-10-18T08:55:15.315320+0000","last_active":"2023-10-18T08:55:16.688748+0000","last_peered":"2023-10-18T08:55:16.688748+0000","last_clean":"2023-10-18T08:55:16.688748+0000","last_became_active":"2023-10-18T08:55:15.315007+0000","last_became_peered":"2023-10-18T08:55:15.315007+0000","last_unstale":"2023-10-18T08:55:16.688748+0000","last_undegraded":"2023-10-18T08:55:16.688748+0000","last_fullsized":"2023-10-18T08:55:16.688748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:19.879235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691247+0000","last_change":"2023-10-18T08:55:15.317460+0000","last_active":"2023-10-18T08:55:16.691247+0000","last_peered":"2023-10-18T08:55:16.691247+0000","last_clean":"2023-10-18T08:55:16.691247+0000","last_became_active":"2023-10-18T08:55:15.317194+0000","last_became_peered":"2023-10-18T08:55:15.317194+0000","last_unstale":"2023-10-18T08:55:16.691247+0000","last_undegraded":"2023-10-18T08:55:16.691247+0000","last_fullsized":"2023-10-18T08:55:16.691247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:26.310762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.328212+0000","last_change":"2023-10-18T08:55:14.301102+0000","last_active":"2023-10-18T08:55:15.328212+0000","last_peered":"2023-10-18T08:55:15.328212+0000","last_clean":"2023-10-18T08:55:15.328212+0000","last_became_active":"2023-10-18T08:55:14.300855+0000","last_became_peered":"2023-10-18T08:55:14.300855+0000","last_unstale":"2023-10-18T08:55:15.328212+0000","last_undegraded":"2023-10-18T08:55:15.328212+0000","last_fullsized":"2023-10-18T08:55:15.328212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:27.143575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.692035+0000","last_change":"2023-10-18T08:55:15.320111+0000","last_active":"2023-10-18T08:55:16.692035+0000","last_peered":"2023-10-18T08:55:16.692035+0000","last_clean":"2023-10-18T08:55:16.692035+0000","last_became_active":"2023-10-18T08:55:15.319103+0000","last_became_peered":"2023-10-18T08:55:15.319103+0000","last_unstale":"2023-10-18T08:55:16.692035+0000","last_undegraded":"2023-10-18T08:55:16.692035+0000","last_fullsized":"2023-10-18T08:55:16.692035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:56:45.727823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.327995+0000","last_change":"2023-10-18T08:55:14.293731+0000","last_active":"2023-10-18T08:55:15.327995+0000","last_peered":"2023-10-18T08:55:15.327995+0000","last_clean":"2023-10-18T08:55:15.327995+0000","last_became_active":"2023-10-18T08:55:14.293420+0000","last_became_peered":"2023-10-18T08:55:14.293420+0000","last_unstale":"2023-10-18T08:55:15.327995+0000","last_undegraded":"2023-10-18T08:55:15.327995+0000","last_fullsized":"2023-10-18T08:55:15.327995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:01.169224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689541+0000","last_change":"2023-10-18T08:55:15.307386+0000","last_active":"2023-10-18T08:55:16.689541+0000","last_peered":"2023-10-18T08:55:16.689541+0000","last_clean":"2023-10-18T08:55:16.689541+0000","last_became_active":"2023-10-18T08:55:15.304519+0000","last_became_peered":"2023-10-18T08:55:15.304519+0000","last_unstale":"2023-10-18T08:55:16.689541+0000","last_undegraded":"2023-10-18T08:55:16.689541+0000","last_fullsized":"2023-10-18T08:55:16.689541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:28.148282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.329282+0000","last_change":"2023-10-18T08:55:14.297189+0000","last_active":"2023-10-18T08:55:15.329282+0000","last_peered":"2023-10-18T08:55:15.329282+0000","last_clean":"2023-10-18T08:55:15.329282+0000","last_became_active":"2023-10-18T08:55:14.296794+0000","last_became_peered":"2023-10-18T08:55:14.296794+0000","last_unstale":"2023-10-18T08:55:15.329282+0000","last_undegraded":"2023-10-18T08:55:15.329282+0000","last_fullsized":"2023-10-18T08:55:15.329282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:10.814889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280421+0000","last_change":"2023-10-18T08:55:15.309830+0000","last_active":"2023-10-18T08:55:16.280421+0000","last_peered":"2023-10-18T08:55:16.280421+0000","last_clean":"2023-10-18T08:55:16.280421+0000","last_became_active":"2023-10-18T08:55:15.309632+0000","last_became_peered":"2023-10-18T08:55:15.309632+0000","last_unstale":"2023-10-18T08:55:16.280421+0000","last_undegraded":"2023-10-18T08:55:16.280421+0000","last_fullsized":"2023-10-18T08:55:16.280421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.328808+0000","last_change":"2023-10-18T08:55:14.297412+0000","last_active":"2023-10-18T08:55:15.328808+0000","last_peered":"2023-10-18T08:55:15.328808+0000","last_clean":"2023-10-18T08:55:15.328808+0000","last_became_active":"2023-10-18T08:55:14.296983+0000","last_became_peered":"2023-10-18T08:55:14.296983+0000","last_unstale":"2023-10-18T08:55:15.328808+0000","last_undegraded":"2023-10-18T08:55:15.328808+0000","last_fullsized":"2023-10-18T08:55:15.328808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:38.968705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338312,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3080000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.397}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930155,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5549999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.609}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4179999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930121,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.371}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4930000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194667,"statfs":{"total":200000000,"available":199339275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.589}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.48}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7050000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]}]}],"pool_statfs":[]}} 2023-10-18T08:55:25.421 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T08:55:25.422 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2023-10-18T08:55:25.423 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-18T08:55:25.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.693+0000 7fc15ca75640 1 -- 172.21.15.40:0/1366092953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc15813c470 msgr2=0x7fc15813f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:25.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.693+0000 7fc15ca75640 1 --2- 172.21.15.40:0/1366092953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc15813c470 0x7fc15813f5a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc140005fd0 tx=0x7fc14002be10 comp rx=0 tx=0).stop 2023-10-18T08:55:25.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.693+0000 7fc15ca75640 1 -- 172.21.15.40:0/1366092953 shutdown_connections 2023-10-18T08:55:25.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.693+0000 7fc15ca75640 1 --2- 172.21.15.40:0/1366092953 >> v2:172.21.15.144:3300/0 conn(0x7fc1580ca210 0x7fc15815d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.693+0000 7fc15ca75640 1 --2- 172.21.15.40:0/1366092953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc15813c470 0x7fc15813f5a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.693+0000 7fc15ca75640 1 -- 172.21.15.40:0/1366092953 >> 172.21.15.40:0/1366092953 conn(0x7fc15806cee0 msgr2=0x7fc158069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:25.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.693+0000 7fc15ca75640 1 -- 172.21.15.40:0/1366092953 shutdown_connections 2023-10-18T08:55:25.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.693+0000 7fc15ca75640 1 -- 172.21.15.40:0/1366092953 wait complete. 2023-10-18T08:55:25.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.696+0000 7fc15ca75640 1 Processor -- start 2023-10-18T08:55:25.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.696+0000 7fc15ca75640 1 -- start start 2023-10-18T08:55:25.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.697+0000 7fc15ca75640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1580ca210 0x7fc15813a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:25.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.697+0000 7fc15ca75640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15813c470 0x7fc15813ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:25.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.697+0000 7fc15ca75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc15815cf30 0x7fc15813d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:25.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.697+0000 7fc15ca75640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1580aa880 con 0x7fc15815cf30 2023-10-18T08:55:25.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.697+0000 7fc15ca75640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1580aab60 con 0x7fc15813c470 2023-10-18T08:55:25.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.697+0000 7fc15ca75640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc158070eb0 con 0x7fc1580ca210 2023-10-18T08:55:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.697+0000 7fc156d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc15815cf30 0x7fc15813d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.697+0000 7fc156d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc15815cf30 0x7fc15813d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47772/0 (socket says 172.21.15.40:47772) 2023-10-18T08:55:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.697+0000 7fc156d9d640 1 -- 172.21.15.40:0/2457417215 learned_addr learned my addr 172.21.15.40:0/2457417215 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.698+0000 7fc15659c640 1 --2- 172.21.15.40:0/2457417215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1580ca210 0x7fc15813a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.698+0000 7fc155d9b640 1 --2- 172.21.15.40:0/2457417215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15813c470 0x7fc15813ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.698+0000 7fc156d9d640 1 -- 172.21.15.40:0/2457417215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1580ca210 msgr2=0x7fc15813a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.698+0000 7fc156d9d640 1 --2- 172.21.15.40:0/2457417215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1580ca210 0x7fc15813a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.698+0000 7fc156d9d640 1 -- 172.21.15.40:0/2457417215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15813c470 msgr2=0x7fc15813ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.698+0000 7fc156d9d640 1 --2- 172.21.15.40:0/2457417215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15813c470 0x7fc15813ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.698+0000 7fc156d9d640 1 -- 172.21.15.40:0/2457417215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc140009560 con 0x7fc15815cf30 2023-10-18T08:55:25.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.698+0000 7fc155d9b640 1 --2- 172.21.15.40:0/2457417215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15813c470 0x7fc15813ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:25.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.698+0000 7fc156d9d640 1 --2- 172.21.15.40:0/2457417215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc15815cf30 0x7fc15813d780 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc15000eb40 tx=0x7fc15000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:25.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.698+0000 7fc14effd640 1 -- 172.21.15.40:0/2457417215 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150004b20 con 0x7fc15815cf30 2023-10-18T08:55:25.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.699+0000 7fc14effd640 1 -- 172.21.15.40:0/2457417215 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc150004ca0 con 0x7fc15815cf30 2023-10-18T08:55:25.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.699+0000 7fc14effd640 1 -- 172.21.15.40:0/2457417215 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150005280 con 0x7fc15815cf30 2023-10-18T08:55:25.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.699+0000 7fc15ca75640 1 -- 172.21.15.40:0/2457417215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc158071190 con 0x7fc15815cf30 2023-10-18T08:55:25.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.699+0000 7fc15ca75640 1 -- 172.21.15.40:0/2457417215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15813a990 con 0x7fc15815cf30 2023-10-18T08:55:25.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.700+0000 7fc15ca75640 1 -- 172.21.15.40:0/2457417215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc158066fd0 con 0x7fc15815cf30 2023-10-18T08:55:25.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.701+0000 7fc14effd640 1 -- 172.21.15.40:0/2457417215 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc1500053e0 con 0x7fc15815cf30 2023-10-18T08:55:25.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.702+0000 7fc14effd640 1 --2- 172.21.15.40:0/2457417215 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc13806ae40 0x7fc13806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.702+0000 7fc14effd640 1 -- 172.21.15.40:0/2457417215 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc150014070 con 0x7fc15815cf30 2023-10-18T08:55:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.703+0000 7fc15659c640 1 --2- 172.21.15.40:0/2457417215 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc13806ae40 0x7fc13806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.704+0000 7fc15659c640 1 --2- 172.21.15.40:0/2457417215 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc13806ae40 0x7fc13806d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc140035000 tx=0x7fc140004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.706+0000 7fc14effd640 1 -- 172.21.15.40:0/2457417215 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc1500554d0 con 0x7fc15815cf30 2023-10-18T08:55:25.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.903+0000 7fc15ca75640 1 -- 172.21.15.40:0/2457417215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7fc1580aaca0 con 0x7fc15815cf30 2023-10-18T08:55:25.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.903+0000 7fc14effd640 1 -- 172.21.15.40:0/2457417215 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v22) v1 ==== 86+0+146 (secure 0 0 0) 0x7fc150058af0 con 0x7fc15815cf30 2023-10-18T08:55:25.907 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:25.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.906+0000 7fc15ca75640 1 -- 172.21.15.40:0/2457417215 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc13806ae40 msgr2=0x7fc13806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:25.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.906+0000 7fc15ca75640 1 --2- 172.21.15.40:0/2457417215 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc13806ae40 0x7fc13806d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc140035000 tx=0x7fc140004730 comp rx=0 tx=0).stop 2023-10-18T08:55:25.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.906+0000 7fc15ca75640 1 -- 172.21.15.40:0/2457417215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc15815cf30 msgr2=0x7fc15813d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:25.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.906+0000 7fc15ca75640 1 --2- 172.21.15.40:0/2457417215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc15815cf30 0x7fc15813d780 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc15000eb40 tx=0x7fc15000c2d0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.906+0000 7fc15ca75640 1 -- 172.21.15.40:0/2457417215 shutdown_connections 2023-10-18T08:55:25.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.906+0000 7fc15ca75640 1 --2- 172.21.15.40:0/2457417215 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc13806ae40 0x7fc13806d300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.906+0000 7fc15ca75640 1 --2- 172.21.15.40:0/2457417215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc15815cf30 0x7fc15813d780 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.906+0000 7fc15ca75640 1 --2- 172.21.15.40:0/2457417215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15813c470 0x7fc15813ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.906+0000 7fc15ca75640 1 --2- 172.21.15.40:0/2457417215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1580ca210 0x7fc15813a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:25.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.906+0000 7fc15ca75640 1 -- 172.21.15.40:0/2457417215 >> 172.21.15.40:0/2457417215 conn(0x7fc15806cee0 msgr2=0x7fc15815b150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:25.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.906+0000 7fc15ca75640 1 -- 172.21.15.40:0/2457417215 shutdown_connections 2023-10-18T08:55:25.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:25.906+0000 7fc15ca75640 1 -- 172.21.15.40:0/2457417215 wait complete. 2023-10-18T08:55:25.927 INFO:teuthology.orchestra.run.smithi040.stdout:[{"version":22,"timestamp":"2023-10-18T08:54:57.245272+0000","name":"","changes":[{"name":"osd/osd_op_complaint_time","new_value":"180.000000"}]}] 2023-10-18T08:55:25.928 INFO:tasks.ceph_manager:config epoch is 22 2023-10-18T08:55:25.928 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-18T08:55:25.929 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-18T08:55:25.929 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-18T08:55:26.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.159+0000 7f963e3ae640 1 -- 172.21.15.40:0/4154503962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96380b2270 msgr2=0x7f96380c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:26.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.159+0000 7f963e3ae640 1 --2- 172.21.15.40:0/4154503962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96380b2270 0x7f96380c9f00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9630005fd0 tx=0x7f963002c0f0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.159+0000 7f963e3ae640 1 -- 172.21.15.40:0/4154503962 shutdown_connections 2023-10-18T08:55:26.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.159+0000 7f963e3ae640 1 --2- 172.21.15.40:0/4154503962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96380b2270 0x7f96380c9f00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.159+0000 7f963e3ae640 1 --2- 172.21.15.40:0/4154503962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96380b1e10 0x7f96380c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.159+0000 7f963e3ae640 1 -- 172.21.15.40:0/4154503962 >> 172.21.15.40:0/4154503962 conn(0x7f9638069720 msgr2=0x7f9638069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:26.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.159+0000 7f963e3ae640 1 -- 172.21.15.40:0/4154503962 shutdown_connections 2023-10-18T08:55:26.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.159+0000 7f963e3ae640 1 -- 172.21.15.40:0/4154503962 wait complete. 2023-10-18T08:55:26.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.161+0000 7f963e3ae640 1 Processor -- start 2023-10-18T08:55:26.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.161+0000 7f963e3ae640 1 -- start start 2023-10-18T08:55:26.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.162+0000 7f963e3ae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96380b15b0 0x7f9638143970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:26.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.162+0000 7f963e3ae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96380b1e10 0x7f96380ab290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:26.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.162+0000 7f963e3ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96380b2270 0x7f9638145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:26.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.162+0000 7f963e3ae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963813f430 con 0x7f96380b2270 2023-10-18T08:55:26.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.162+0000 7f963e3ae640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9638143eb0 con 0x7f96380b1e10 2023-10-18T08:55:26.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.162+0000 7f963e3ae640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96380ac1c0 con 0x7f96380b15b0 2023-10-18T08:55:26.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.162+0000 7f963dbad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96380b2270 0x7f9638145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:26.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.162+0000 7f963dbad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96380b2270 0x7f9638145400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47796/0 (socket says 172.21.15.40:47796) 2023-10-18T08:55:26.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.162+0000 7f963dbad640 1 -- 172.21.15.40:0/1503857487 learned_addr learned my addr 172.21.15.40:0/1503857487 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:26.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.162+0000 7f963d3ac640 1 --2- 172.21.15.40:0/1503857487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96380b15b0 0x7f9638143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:26.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.162+0000 7f963cbab640 1 --2- 172.21.15.40:0/1503857487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96380b1e10 0x7f96380ab290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:26.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.163+0000 7f963dbad640 1 -- 172.21.15.40:0/1503857487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96380b15b0 msgr2=0x7f9638143970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:26.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.163+0000 7f963dbad640 1 --2- 172.21.15.40:0/1503857487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96380b15b0 0x7f9638143970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.163+0000 7f963dbad640 1 -- 172.21.15.40:0/1503857487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96380b1e10 msgr2=0x7f96380ab290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:26.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.163+0000 7f963dbad640 1 --2- 172.21.15.40:0/1503857487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96380b1e10 0x7f96380ab290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.163+0000 7f963dbad640 1 -- 172.21.15.40:0/1503857487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9630009840 con 0x7f96380b2270 2023-10-18T08:55:26.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.163+0000 7f963dbad640 1 --2- 172.21.15.40:0/1503857487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96380b2270 0x7f9638145400 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9630009cd0 tx=0x7f9630005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:26.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.163+0000 7f962dffb640 1 -- 172.21.15.40:0/1503857487 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630004680 con 0x7f96380b2270 2023-10-18T08:55:26.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.163+0000 7f962dffb640 1 -- 172.21.15.40:0/1503857487 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f96300345a0 con 0x7f96380b2270 2023-10-18T08:55:26.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.163+0000 7f963e3ae640 1 -- 172.21.15.40:0/1503857487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96380abc00 con 0x7f96380b2270 2023-10-18T08:55:26.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.163+0000 7f962dffb640 1 -- 172.21.15.40:0/1503857487 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96300349b0 con 0x7f96380b2270 2023-10-18T08:55:26.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.164+0000 7f963e3ae640 1 -- 172.21.15.40:0/1503857487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963814a5f0 con 0x7f96380b2270 2023-10-18T08:55:26.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.165+0000 7f963e3ae640 1 -- 172.21.15.40:0/1503857487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9638066e00 con 0x7f96380b2270 2023-10-18T08:55:26.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.166+0000 7f962dffb640 1 -- 172.21.15.40:0/1503857487 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9630034b10 con 0x7f96380b2270 2023-10-18T08:55:26.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.166+0000 7f962dffb640 1 --2- 172.21.15.40:0/1503857487 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f961c06ae40 0x7f961c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.167+0000 7f962dffb640 1 -- 172.21.15.40:0/1503857487 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96300ac7c0 con 0x7f96380b2270 2023-10-18T08:55:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.167+0000 7f963d3ac640 1 --2- 172.21.15.40:0/1503857487 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f961c06ae40 0x7f961c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.168+0000 7f963d3ac640 1 --2- 172.21.15.40:0/1503857487 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f961c06ae40 0x7f961c06d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f96340010e0 tx=0x7f9634009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.171+0000 7f962dffb640 1 -- 172.21.15.40:0/1503857487 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9630077900 con 0x7f96380b2270 2023-10-18T08:55:26.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.417+0000 7f963e3ae640 1 -- 172.21.15.40:0/1503857487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f96380cc110 con 0x7f96380b2270 2023-10-18T08:55:26.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.422+0000 7f962dffb640 1 -- 172.21.15.40:0/1503857487 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171036 (secure 0 0 0) 0x7f963007af20 con 0x7f96380b2270 2023-10-18T08:55:26.427 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:26.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.426+0000 7f963e3ae640 1 -- 172.21.15.40:0/1503857487 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f961c06ae40 msgr2=0x7f961c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:26.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.426+0000 7f963e3ae640 1 --2- 172.21.15.40:0/1503857487 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f961c06ae40 0x7f961c06d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f96340010e0 tx=0x7f9634009040 comp rx=0 tx=0).stop 2023-10-18T08:55:26.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.426+0000 7f963e3ae640 1 -- 172.21.15.40:0/1503857487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96380b2270 msgr2=0x7f9638145400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:26.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.426+0000 7f963e3ae640 1 --2- 172.21.15.40:0/1503857487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96380b2270 0x7f9638145400 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9630009cd0 tx=0x7f9630005c80 comp rx=0 tx=0).stop 2023-10-18T08:55:26.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.426+0000 7f963e3ae640 1 -- 172.21.15.40:0/1503857487 shutdown_connections 2023-10-18T08:55:26.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.426+0000 7f963e3ae640 1 --2- 172.21.15.40:0/1503857487 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f961c06ae40 0x7f961c06d300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.426+0000 7f963e3ae640 1 --2- 172.21.15.40:0/1503857487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96380b2270 0x7f9638145400 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.426+0000 7f963e3ae640 1 --2- 172.21.15.40:0/1503857487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96380b1e10 0x7f96380ab290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.426+0000 7f963e3ae640 1 --2- 172.21.15.40:0/1503857487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96380b15b0 0x7f9638143970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.426+0000 7f963e3ae640 1 -- 172.21.15.40:0/1503857487 >> 172.21.15.40:0/1503857487 conn(0x7f9638069720 msgr2=0x7f9638159c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:26.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.426+0000 7f963e3ae640 1 -- 172.21.15.40:0/1503857487 shutdown_connections 2023-10-18T08:55:26.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.426+0000 7f963e3ae640 1 -- 172.21.15.40:0/1503857487 wait complete. 2023-10-18T08:55:26.448 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":5,"active_gid":4114,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":3843888630},{"type":"v1","addr":"172.21.15.144:6801","nonce":3843888630}]},"active_addr":"172.21.15.144:6801/3843888630","active_change":"2023-10-18T08:55:00.878429+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":965787339}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":1638189781}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":3078039673}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":3294213717}]}]} 2023-10-18T08:55:26.453 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-18T08:55:26.453 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-18T08:55:26.454 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:26.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.686+0000 7f56ed1ef640 1 -- 172.21.15.40:0/1971254134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56e806d6f0 msgr2=0x7f56e80b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:26.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.686+0000 7f56ed1ef640 1 --2- 172.21.15.40:0/1971254134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56e806d6f0 0x7f56e80b15b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f56e80caf30 tx=0x7f56d802bd70 comp rx=0 tx=0).stop 2023-10-18T08:55:26.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.686+0000 7f56ed1ef640 1 -- 172.21.15.40:0/1971254134 shutdown_connections 2023-10-18T08:55:26.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.686+0000 7f56ed1ef640 1 --2- 172.21.15.40:0/1971254134 >> v2:172.21.15.144:3300/0 conn(0x7f56e80ca0d0 0x7f56e8157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.686+0000 7f56ed1ef640 1 --2- 172.21.15.40:0/1971254134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56e806d6f0 0x7f56e80b15b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.686+0000 7f56ed1ef640 1 -- 172.21.15.40:0/1971254134 >> 172.21.15.40:0/1971254134 conn(0x7f56e8069720 msgr2=0x7f56e8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:26.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.686+0000 7f56ed1ef640 1 -- 172.21.15.40:0/1971254134 shutdown_connections 2023-10-18T08:55:26.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.686+0000 7f56ed1ef640 1 -- 172.21.15.40:0/1971254134 wait complete. 2023-10-18T08:55:26.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.688+0000 7f56ed1ef640 1 Processor -- start 2023-10-18T08:55:26.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.688+0000 7f56ed1ef640 1 -- start start 2023-10-18T08:55:26.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.689+0000 7f56ed1ef640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56e806d6f0 0x7f56e8159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:26.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.689+0000 7f56ed1ef640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56e80b0630 0x7f56e81447f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:26.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.689+0000 7f56ed1ef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56e80ca0d0 0x7f56e80cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:26.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.689+0000 7f56ed1ef640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e815a500 con 0x7f56e80ca0d0 2023-10-18T08:55:26.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.689+0000 7f56ed1ef640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e81596a0 con 0x7f56e806d6f0 2023-10-18T08:55:26.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.689+0000 7f56ed1ef640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e81598f0 con 0x7f56e80b0630 2023-10-18T08:55:26.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.690+0000 7f56ec9ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56e80ca0d0 0x7f56e80cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:26.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.690+0000 7f56ec9ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56e80ca0d0 0x7f56e80cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47810/0 (socket says 172.21.15.40:47810) 2023-10-18T08:55:26.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.690+0000 7f56ec9ee640 1 -- 172.21.15.40:0/1632353763 learned_addr learned my addr 172.21.15.40:0/1632353763 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:26.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.690+0000 7f56e7fff640 1 --2- 172.21.15.40:0/1632353763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56e806d6f0 0x7f56e8159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:26.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.690+0000 7f56e77fe640 1 --2- 172.21.15.40:0/1632353763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56e80b0630 0x7f56e81447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:26.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.690+0000 7f56ec9ee640 1 -- 172.21.15.40:0/1632353763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56e80b0630 msgr2=0x7f56e81447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:26.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.690+0000 7f56ec9ee640 1 --2- 172.21.15.40:0/1632353763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56e80b0630 0x7f56e81447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.690+0000 7f56ec9ee640 1 -- 172.21.15.40:0/1632353763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56e806d6f0 msgr2=0x7f56e8159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:26.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.690+0000 7f56ec9ee640 1 --2- 172.21.15.40:0/1632353763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56e806d6f0 0x7f56e8159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.690+0000 7f56ec9ee640 1 -- 172.21.15.40:0/1632353763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d8009490 con 0x7f56e80ca0d0 2023-10-18T08:55:26.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.691+0000 7f56ec9ee640 1 --2- 172.21.15.40:0/1632353763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56e80ca0d0 0x7f56e80cc3b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f56e000ddc0 tx=0x7f56e000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:26.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.691+0000 7f56e4ff9640 1 -- 172.21.15.40:0/1632353763 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0004d20 con 0x7f56e80ca0d0 2023-10-18T08:55:26.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.691+0000 7f56e4ff9640 1 -- 172.21.15.40:0/1632353763 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f56e0004e80 con 0x7f56e80ca0d0 2023-10-18T08:55:26.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.691+0000 7f56ed1ef640 1 -- 172.21.15.40:0/1632353763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e8159bd0 con 0x7f56e80ca0d0 2023-10-18T08:55:26.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.691+0000 7f56e4ff9640 1 -- 172.21.15.40:0/1632353763 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e000e8a0 con 0x7f56e80ca0d0 2023-10-18T08:55:26.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.691+0000 7f56ed1ef640 1 -- 172.21.15.40:0/1632353763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e815a190 con 0x7f56e80ca0d0 2023-10-18T08:55:26.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.692+0000 7f56ed1ef640 1 -- 172.21.15.40:0/1632353763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e815a6b0 con 0x7f56e80ca0d0 2023-10-18T08:55:26.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.693+0000 7f56e4ff9640 1 -- 172.21.15.40:0/1632353763 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f56e000ea00 con 0x7f56e80ca0d0 2023-10-18T08:55:26.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.694+0000 7f56e4ff9640 1 --2- 172.21.15.40:0/1632353763 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f56d406ae70 0x7f56d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:26.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.694+0000 7f56e4ff9640 1 -- 172.21.15.40:0/1632353763 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56e008a3f0 con 0x7f56e80ca0d0 2023-10-18T08:55:26.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.695+0000 7f56e7fff640 1 --2- 172.21.15.40:0/1632353763 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f56d406ae70 0x7f56d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:26.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.696+0000 7f56e7fff640 1 --2- 172.21.15.40:0/1632353763 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f56d406ae70 0x7f56d406d330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f56d8036000 tx=0x7f56d8004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:26.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.698+0000 7f56e4ff9640 1 -- 172.21.15.40:0/1632353763 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f56e815a6b0 con 0x7f56e80ca0d0 2023-10-18T08:55:26.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.899+0000 7f56ed1ef640 1 -- 172.21.15.40:0/1632353763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56e815bed0 con 0x7f56e80ca0d0 2023-10-18T08:55:26.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.901+0000 7f56e4ff9640 1 -- 172.21.15.40:0/1632353763 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f56e0055370 con 0x7f56e80ca0d0 2023-10-18T08:55:26.905 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:26.905 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:16.275677+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:26.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.904+0000 7f56ed1ef640 1 -- 172.21.15.40:0/1632353763 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f56d406ae70 msgr2=0x7f56d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:26.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.904+0000 7f56ed1ef640 1 --2- 172.21.15.40:0/1632353763 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f56d406ae70 0x7f56d406d330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f56d8036000 tx=0x7f56d8004730 comp rx=0 tx=0).stop 2023-10-18T08:55:26.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.904+0000 7f56ed1ef640 1 -- 172.21.15.40:0/1632353763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56e80ca0d0 msgr2=0x7f56e80cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:26.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.904+0000 7f56ed1ef640 1 --2- 172.21.15.40:0/1632353763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56e80ca0d0 0x7f56e80cc3b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f56e000ddc0 tx=0x7f56e000b730 comp rx=0 tx=0).stop 2023-10-18T08:55:26.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.904+0000 7f56ed1ef640 1 -- 172.21.15.40:0/1632353763 shutdown_connections 2023-10-18T08:55:26.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.904+0000 7f56ed1ef640 1 --2- 172.21.15.40:0/1632353763 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f56d406ae70 0x7f56d406d330 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.904+0000 7f56ed1ef640 1 --2- 172.21.15.40:0/1632353763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56e80ca0d0 0x7f56e80cc3b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.904+0000 7f56ed1ef640 1 --2- 172.21.15.40:0/1632353763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56e80b0630 0x7f56e81447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.904+0000 7f56ed1ef640 1 --2- 172.21.15.40:0/1632353763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56e806d6f0 0x7f56e8159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:26.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.904+0000 7f56ed1ef640 1 -- 172.21.15.40:0/1632353763 >> 172.21.15.40:0/1632353763 conn(0x7f56e8069720 msgr2=0x7f56e80abef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:26.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.904+0000 7f56ed1ef640 1 -- 172.21.15.40:0/1632353763 shutdown_connections 2023-10-18T08:55:26.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:26.904+0000 7f56ed1ef640 1 -- 172.21.15.40:0/1632353763 wait complete. 2023-10-18T08:55:26.926 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-18T08:55:26.927 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:27.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.155+0000 7fe4a3fff640 1 -- 172.21.15.40:0/157747164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4a40b15b0 msgr2=0x7fe4a40c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.155+0000 7fe4a3fff640 1 --2- 172.21.15.40:0/157747164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4a40b15b0 0x7fe4a40c99c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe48c005fd0 tx=0x7fe48c02c2e0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.155+0000 7fe4a3fff640 1 -- 172.21.15.40:0/157747164 shutdown_connections 2023-10-18T08:55:27.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.155+0000 7fe4a3fff640 1 --2- 172.21.15.40:0/157747164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4a40c9f00 0x7fe4a4157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.155+0000 7fe4a3fff640 1 --2- 172.21.15.40:0/157747164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4a40b15b0 0x7fe4a40c99c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.155+0000 7fe4a3fff640 1 -- 172.21.15.40:0/157747164 >> 172.21.15.40:0/157747164 conn(0x7fe4a4069720 msgr2=0x7fe4a4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.155+0000 7fe4a3fff640 1 -- 172.21.15.40:0/157747164 shutdown_connections 2023-10-18T08:55:27.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.155+0000 7fe4a3fff640 1 -- 172.21.15.40:0/157747164 wait complete. 2023-10-18T08:55:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.157+0000 7fe4a3fff640 1 Processor -- start 2023-10-18T08:55:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.157+0000 7fe4a3fff640 1 -- start start 2023-10-18T08:55:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe4a3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4a406d6f0 0x7fe4a4144920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe4a3fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4a40b15b0 0x7fe4a40af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe4a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4a40c9f00 0x7fe4a41e1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe4a3fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a4144e60 con 0x7fe4a40c9f00 2023-10-18T08:55:27.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe4a3fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a41e5090 con 0x7fe4a406d6f0 2023-10-18T08:55:27.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe4a3fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a41e5370 con 0x7fe4a40b15b0 2023-10-18T08:55:27.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe4a37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4a40c9f00 0x7fe4a41e1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe4a37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4a40c9f00 0x7fe4a41e1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47836/0 (socket says 172.21.15.40:47836) 2023-10-18T08:55:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe4a37fe640 1 -- 172.21.15.40:0/838239098 learned_addr learned my addr 172.21.15.40:0/838239098 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe49a7fc640 1 --2- 172.21.15.40:0/838239098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4a40b15b0 0x7fe4a40af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe4a2ffd640 1 --2- 172.21.15.40:0/838239098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4a406d6f0 0x7fe4a4144920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe4a37fe640 1 -- 172.21.15.40:0/838239098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4a40b15b0 msgr2=0x7fe4a40af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe4a37fe640 1 --2- 172.21.15.40:0/838239098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4a40b15b0 0x7fe4a40af5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe4a37fe640 1 -- 172.21.15.40:0/838239098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4a406d6f0 msgr2=0x7fe4a4144920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe4a37fe640 1 --2- 172.21.15.40:0/838239098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4a406d6f0 0x7fe4a4144920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe4a37fe640 1 -- 172.21.15.40:0/838239098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48c009900 con 0x7fe4a40c9f00 2023-10-18T08:55:27.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe4a2ffd640 1 --2- 172.21.15.40:0/838239098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4a406d6f0 0x7fe4a4144920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:27.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.158+0000 7fe49a7fc640 1 --2- 172.21.15.40:0/838239098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4a40b15b0 0x7fe4a40af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:27.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.159+0000 7fe4a37fe640 1 --2- 172.21.15.40:0/838239098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4a40c9f00 0x7fe4a41e1f70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00da80 tx=0x7fe49c00df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.159+0000 7fe49bfff640 1 -- 172.21.15.40:0/838239098 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c00ae20 con 0x7fe4a40c9f00 2023-10-18T08:55:27.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.159+0000 7fe49bfff640 1 -- 172.21.15.40:0/838239098 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe49c005110 con 0x7fe4a40c9f00 2023-10-18T08:55:27.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.159+0000 7fe4a3fff640 1 -- 172.21.15.40:0/838239098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a415bc10 con 0x7fe4a40c9f00 2023-10-18T08:55:27.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.159+0000 7fe49bfff640 1 -- 172.21.15.40:0/838239098 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c005520 con 0x7fe4a40c9f00 2023-10-18T08:55:27.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.159+0000 7fe4a3fff640 1 -- 172.21.15.40:0/838239098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a41e5930 con 0x7fe4a40c9f00 2023-10-18T08:55:27.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.160+0000 7fe4a3fff640 1 -- 172.21.15.40:0/838239098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a4066e00 con 0x7fe4a40c9f00 2023-10-18T08:55:27.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.161+0000 7fe49bfff640 1 -- 172.21.15.40:0/838239098 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe49c0047a0 con 0x7fe4a40c9f00 2023-10-18T08:55:27.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.162+0000 7fe49bfff640 1 --2- 172.21.15.40:0/838239098 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe48406ae40 0x7fe48406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.162+0000 7fe49bfff640 1 -- 172.21.15.40:0/838239098 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe49c089db0 con 0x7fe4a40c9f00 2023-10-18T08:55:27.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.163+0000 7fe4a2ffd640 1 --2- 172.21.15.40:0/838239098 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe48406ae40 0x7fe48406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.164+0000 7fe4a2ffd640 1 --2- 172.21.15.40:0/838239098 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe48406ae40 0x7fe48406d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe494000f60 tx=0x7fe494009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.166+0000 7fe49bfff640 1 -- 172.21.15.40:0/838239098 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe49c08e070 con 0x7fe4a40c9f00 2023-10-18T08:55:27.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.363+0000 7fe4a3fff640 1 -- 172.21.15.40:0/838239098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4a40caf90 con 0x7fe4a40c9f00 2023-10-18T08:55:27.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.365+0000 7fe49bfff640 1 -- 172.21.15.40:0/838239098 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7fe49c054e50 con 0x7fe4a40c9f00 2023-10-18T08:55:27.369 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:27.369 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:16.275677+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:27.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.368+0000 7fe4a3fff640 1 -- 172.21.15.40:0/838239098 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe48406ae40 msgr2=0x7fe48406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.368+0000 7fe4a3fff640 1 --2- 172.21.15.40:0/838239098 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe48406ae40 0x7fe48406d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe494000f60 tx=0x7fe494009040 comp rx=0 tx=0).stop 2023-10-18T08:55:27.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.368+0000 7fe4a3fff640 1 -- 172.21.15.40:0/838239098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4a40c9f00 msgr2=0x7fe4a41e1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.368+0000 7fe4a3fff640 1 --2- 172.21.15.40:0/838239098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4a40c9f00 0x7fe4a41e1f70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00da80 tx=0x7fe49c00df40 comp rx=0 tx=0).stop 2023-10-18T08:55:27.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.368+0000 7fe4a3fff640 1 -- 172.21.15.40:0/838239098 shutdown_connections 2023-10-18T08:55:27.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.368+0000 7fe4a3fff640 1 --2- 172.21.15.40:0/838239098 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe48406ae40 0x7fe48406d300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.368+0000 7fe4a3fff640 1 --2- 172.21.15.40:0/838239098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4a40c9f00 0x7fe4a41e1f70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.368+0000 7fe4a3fff640 1 --2- 172.21.15.40:0/838239098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4a40b15b0 0x7fe4a40af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.368+0000 7fe4a3fff640 1 --2- 172.21.15.40:0/838239098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4a406d6f0 0x7fe4a4144920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.368+0000 7fe4a3fff640 1 -- 172.21.15.40:0/838239098 >> 172.21.15.40:0/838239098 conn(0x7fe4a4069720 msgr2=0x7fe4a41549d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.369+0000 7fe4a3fff640 1 -- 172.21.15.40:0/838239098 shutdown_connections 2023-10-18T08:55:27.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.369+0000 7fe4a3fff640 1 -- 172.21.15.40:0/838239098 wait complete. 2023-10-18T08:55:27.392 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-18T08:55:27.392 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-18T08:55:27.393 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-18T08:55:27.394 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-18T08:55:27.394 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-18T08:55:27.395 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-18T08:55:27.395 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-18T08:55:27.396 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-18T08:55:27.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.718+0000 7f881acc3640 1 -- 172.21.15.40:0/4067806780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8804005180 msgr2=0x7f880400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.718+0000 7f881acc3640 1 --2- 172.21.15.40:0/4067806780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8804005180 0x7f880400db90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f880c00a600 tx=0x7f88040057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.718+0000 7f881a4c2640 1 -- 172.21.15.40:0/4067806780 reap_dead start 2023-10-18T08:55:27.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.719+0000 7f881acc3640 1 -- 172.21.15.40:0/4067806780 shutdown_connections 2023-10-18T08:55:27.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.719+0000 7f881acc3640 1 -- 172.21.15.40:0/4067806780 >> 172.21.15.40:0/4067806780 conn(0x7f8814069720 msgr2=0x7f8814069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.719+0000 7f881acc3640 1 -- 172.21.15.40:0/4067806780 shutdown_connections 2023-10-18T08:55:27.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.720+0000 7f881acc3640 1 -- 172.21.15.40:0/4067806780 wait complete. 2023-10-18T08:55:27.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.723+0000 7f881acc3640 1 Processor -- start 2023-10-18T08:55:27.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.724+0000 7f881acc3640 1 -- start start 2023-10-18T08:55:27.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.725+0000 7f9738c35640 1 -- 172.21.15.40:0/3421098176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97240047c0 msgr2=0x7f9724004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.725+0000 7f9738c35640 1 --2- 172.21.15.40:0/3421098176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97240047c0 0x7f9724004c40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f97200017d0 tx=0x7f97240057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.726+0000 7f9733fff640 1 -- 172.21.15.40:0/3421098176 reap_dead start 2023-10-18T08:55:27.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.726+0000 7f9738c35640 1 -- 172.21.15.40:0/3421098176 shutdown_connections 2023-10-18T08:55:27.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.726+0000 7f9738c35640 1 -- 172.21.15.40:0/3421098176 >> 172.21.15.40:0/3421098176 conn(0x7f972c009f80 msgr2=0x7f972c00a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.726+0000 7f8c62541640 1 -- 172.21.15.40:0/812641140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c0cf7e0 msgr2=0x7f8c5c0cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.726+0000 7f8c5affd640 1 -- 172.21.15.40:0/812641140 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0049e0 con 0x7f8c5c0cf7e0 2023-10-18T08:55:27.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.726+0000 7f8c62541640 1 --2- 172.21.15.40:0/812641140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c0cf7e0 0x7f8c5c0cfbc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c005fd0 tx=0x7f8c4c02bd90 comp rx=0 tx=0).stop 2023-10-18T08:55:27.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.726+0000 7f881acc3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f881414b5d0 0x7f881414e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.726+0000 7f8c62541640 1 -- 172.21.15.40:0/812641140 shutdown_connections 2023-10-18T08:55:27.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.726+0000 7f8c62541640 1 --2- 172.21.15.40:0/812641140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c5c0cedf0 0x7f8c5c0ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.726+0000 7f9738c35640 1 -- 172.21.15.40:0/3421098176 shutdown_connections 2023-10-18T08:55:27.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.726+0000 7f8c62541640 1 --2- 172.21.15.40:0/812641140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c0cf7e0 0x7f8c5c0cfbc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.726+0000 7f8c62541640 1 -- 172.21.15.40:0/812641140 >> 172.21.15.40:0/812641140 conn(0x7f8c5c09a580 msgr2=0x7f8c5c0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.727+0000 7f8819cc1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f881414b5d0 0x7f881414e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.727+0000 7f8819cc1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f881414b5d0 0x7f881414e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47882/0 (socket says 172.21.15.40:47882) 2023-10-18T08:55:27.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.727+0000 7f8819cc1640 1 -- 172.21.15.40:0/2780907168 learned_addr learned my addr 172.21.15.40:0/2780907168 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:27.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.726+0000 7f1c69690640 1 -- 172.21.15.40:0/3359317884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c4c003970 msgr2=0x7f1c4c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.726+0000 7f1c69690640 1 --2- 172.21.15.40:0/3359317884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c4c003970 0x7f1c4c003d90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00c470 tx=0x7f1c4c0057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.726+0000 7f9738c35640 1 -- 172.21.15.40:0/3421098176 wait complete. 2023-10-18T08:55:27.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.727+0000 7f1c637fe640 1 -- 172.21.15.40:0/3359317884 reap_dead start 2023-10-18T08:55:27.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.727+0000 7f1c69690640 1 -- 172.21.15.40:0/3359317884 shutdown_connections 2023-10-18T08:55:27.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.727+0000 7f1c69690640 1 -- 172.21.15.40:0/3359317884 >> 172.21.15.40:0/3359317884 conn(0x7f1c6406a780 msgr2=0x7f1c6406ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.727+0000 7f881acc3640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f881414b9b0 0x7f88141592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.728+0000 7f1c69690640 1 -- 172.21.15.40:0/3359317884 shutdown_connections 2023-10-18T08:55:27.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.728+0000 7f881acc3640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88140ad980 0x7f88141e1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.728+0000 7f881acc3640 1 -- 172.21.15.40:0/2780907168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881414f4e0 con 0x7f881414b5d0 2023-10-18T08:55:27.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.728+0000 7f881acc3640 1 -- 172.21.15.40:0/2780907168 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881414e960 con 0x7f881414b9b0 2023-10-18T08:55:27.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.728+0000 7f88194c0640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f881414b9b0 0x7f88141592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.728+0000 7f881a4c2640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88140ad980 0x7f88141e1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.728+0000 7f881acc3640 1 -- 172.21.15.40:0/2780907168 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f881414ec40 con 0x7f88140ad980 2023-10-18T08:55:27.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.728+0000 7f1c69690640 1 -- 172.21.15.40:0/3359317884 wait complete. 2023-10-18T08:55:27.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.729+0000 7f8c62541640 1 -- 172.21.15.40:0/812641140 shutdown_connections 2023-10-18T08:55:27.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.729+0000 7f8819cc1640 1 -- 172.21.15.40:0/2780907168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88140ad980 msgr2=0x7f88141e1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.729+0000 7f8819cc1640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88140ad980 0x7f88141e1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.729+0000 7f8819cc1640 1 -- 172.21.15.40:0/2780907168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f881414b9b0 msgr2=0x7f88141592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.729+0000 7f9738c35640 1 Processor -- start 2023-10-18T08:55:27.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.730+0000 7f9738c35640 1 -- start start 2023-10-18T08:55:27.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.729+0000 7f8c62541640 1 -- 172.21.15.40:0/812641140 wait complete. 2023-10-18T08:55:27.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.731+0000 7f8819cc1640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f881414b9b0 0x7f88141592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.731+0000 7f9738c35640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97240047c0 0x7f972c0d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.731+0000 7f9738c35640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f972c04ab60 0x7f972c04c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.731+0000 7f9738c35640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c04c610 0x7f972c04ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.731+0000 7f9738c35640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c034670 con 0x7f972c04c610 2023-10-18T08:55:27.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.731+0000 7f9738c35640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c04cfd0 con 0x7f972c04ab60 2023-10-18T08:55:27.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.731+0000 7f9738c35640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972c034bc0 con 0x7f97240047c0 2023-10-18T08:55:27.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.731+0000 7f8819cc1640 1 -- 172.21.15.40:0/2780907168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881414ef20 con 0x7f881414b5d0 2023-10-18T08:55:27.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.731+0000 7f8c62541640 1 Processor -- start 2023-10-18T08:55:27.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.732+0000 7f8c62541640 1 -- start start 2023-10-18T08:55:27.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.732+0000 7f35fad6e640 1 -- 172.21.15.40:0/102723718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec050170 msgr2=0x7f35ec038050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.732+0000 7f35fad6e640 1 --2- 172.21.15.40:0/102723718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec050170 0x7f35ec038050 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009830 tx=0x7f35e402bcd0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.731+0000 7f88194c0640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f881414b9b0 0x7f88141592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:55:27.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.732+0000 7f1c69690640 1 Processor -- start 2023-10-18T08:55:27.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.732+0000 7f9733fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c04c610 0x7f972c04ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.732+0000 7f97337fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97240047c0 0x7f972c0d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.732+0000 7f97337fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97240047c0 0x7f972c0d4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:45220/0 (socket says 172.21.15.40:45220) 2023-10-18T08:55:27.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.732+0000 7f97337fe640 1 -- 172.21.15.40:0/3253221272 learned_addr learned my addr 172.21.15.40:0/3253221272 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:27.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.732+0000 7f9732ffd640 1 --2- 172.21.15.40:0/3253221272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f972c04ab60 0x7f972c04c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.732+0000 7f35fad6e640 1 -- 172.21.15.40:0/102723718 shutdown_connections 2023-10-18T08:55:27.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.732+0000 7f35fad6e640 1 --2- 172.21.15.40:0/102723718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35ec053c40 0x7f35ec0069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.732+0000 7f35fad6e640 1 --2- 172.21.15.40:0/102723718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec050170 0x7f35ec038050 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.732+0000 7f35fad6e640 1 -- 172.21.15.40:0/102723718 >> 172.21.15.40:0/102723718 conn(0x7f35ec009f80 msgr2=0x7f35ec00a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.732+0000 7f8819cc1640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f881414b5d0 0x7f881414e380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f880800e900 tx=0x7f880800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.732+0000 7f35fad6e640 1 -- 172.21.15.40:0/102723718 shutdown_connections 2023-10-18T08:55:27.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.732+0000 7f1c69690640 1 -- start start 2023-10-18T08:55:27.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.733+0000 7f9733fff640 1 -- 172.21.15.40:0/3253221272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97240047c0 msgr2=0x7f972c0d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.733+0000 7f9733fff640 1 --2- 172.21.15.40:0/3253221272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97240047c0 0x7f972c0d4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.733+0000 7f9733fff640 1 -- 172.21.15.40:0/3253221272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f972c04ab60 msgr2=0x7f972c04c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.733+0000 7f9733fff640 1 --2- 172.21.15.40:0/3253221272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f972c04ab60 0x7f972c04c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.733+0000 7f9733fff640 1 -- 172.21.15.40:0/3253221272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c04afc0 con 0x7f972c04c610 2023-10-18T08:55:27.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.733+0000 7f88127fc640 1 -- 172.21.15.40:0/2780907168 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880800cc50 con 0x7f881414b5d0 2023-10-18T08:55:27.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.733+0000 7f88127fc640 1 -- 172.21.15.40:0/2780907168 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8808011460 con 0x7f881414b5d0 2023-10-18T08:55:27.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.733+0000 7f88127fc640 1 -- 172.21.15.40:0/2780907168 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880800cea0 con 0x7f881414b5d0 2023-10-18T08:55:27.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.733+0000 7f881acc3640 1 -- 172.21.15.40:0/2780907168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881414f200 con 0x7f881414b5d0 2023-10-18T08:55:27.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.733+0000 7f881acc3640 1 -- 172.21.15.40:0/2780907168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88141e2450 con 0x7f881414b5d0 2023-10-18T08:55:27.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.733+0000 7f9733fff640 1 --2- 172.21.15.40:0/3253221272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c04c610 0x7f972c04ca90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f97340633f0 tx=0x7f9734073940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.734+0000 7f8c62541640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c0cca70 0x7f8c5c14d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.734+0000 7f8c62541640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c5c0cedf0 0x7f8c5c1514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.734+0000 7f8c62541640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c5c0cf7e0 0x7f8c5c151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.734+0000 7f8c62541640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c153460 con 0x7f8c5c0cf7e0 2023-10-18T08:55:27.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.734+0000 7f8c62541640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c15bf80 con 0x7f8c5c0cca70 2023-10-18T08:55:27.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.734+0000 7f8c62541640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c15c260 con 0x7f8c5c0cedf0 2023-10-18T08:55:27.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.733+0000 7f35fad6e640 1 -- 172.21.15.40:0/102723718 wait complete. 2023-10-18T08:55:27.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.734+0000 7f9cb3fff640 1 -- 172.21.15.40:0/1114782324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cac0df1f0 msgr2=0x7f9cac038050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f1c69690640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c4c003970 0x7f1c641500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f1c69690640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c640b33a0 0x7f1c640b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f1c69690640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c64155b40 0x7f1c64155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f1c69690640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c640caac0 con 0x7f1c4c003970 2023-10-18T08:55:27.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f1c69690640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c64156500 con 0x7f1c64155b40 2023-10-18T08:55:27.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f1c69690640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c64150610 con 0x7f1c640b33a0 2023-10-18T08:55:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f971ffff640 1 -- 172.21.15.40:0/3253221272 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973408e040 con 0x7f972c04c610 2023-10-18T08:55:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.734+0000 7f9cb3fff640 1 --2- 172.21.15.40:0/1114782324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cac0df1f0 0x7f9cac038050 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009830 tx=0x7f9ca802bcd0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f9cb3fff640 1 -- 172.21.15.40:0/1114782324 shutdown_connections 2023-10-18T08:55:27.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f9cb3fff640 1 --2- 172.21.15.40:0/1114782324 >> v2:172.21.15.144:3300/0 conn(0x7f9cac050170 0x7f9cac0069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f9cb3fff640 1 --2- 172.21.15.40:0/1114782324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cac0df1f0 0x7f9cac038050 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f9cb3fff640 1 -- 172.21.15.40:0/1114782324 >> 172.21.15.40:0/1114782324 conn(0x7f9cac009f80 msgr2=0x7f9cac00a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f8c5bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c0cca70 0x7f8c5c14d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f8c5b7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c5c0cedf0 0x7f8c5c1514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f8c5bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c0cca70 0x7f8c5c14d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.40:34574/0 (socket says 172.21.15.40:34574) 2023-10-18T08:55:27.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f8c5bfff640 1 -- 172.21.15.40:0/1948687784 learned_addr learned my addr 172.21.15.40:0/1948687784 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:27.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f8c60ade640 1 --2- 172.21.15.40:0/1948687784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c5c0cf7e0 0x7f8c5c151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f9cb3fff640 1 -- 172.21.15.40:0/1114782324 shutdown_connections 2023-10-18T08:55:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.736+0000 7f1c62ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c4c003970 0x7f1c641500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.736+0000 7f1c62ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c4c003970 0x7f1c641500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47908/0 (socket says 172.21.15.40:47908) 2023-10-18T08:55:27.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.736+0000 7f1c62ffd640 1 -- 172.21.15.40:0/969310818 learned_addr learned my addr 172.21.15.40:0/969310818 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.735+0000 7f35fad6e640 1 Processor -- start 2023-10-18T08:55:27.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.736+0000 7f1c637fe640 1 --2- 172.21.15.40:0/969310818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c64155b40 0x7f1c64155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.736+0000 7f9738c35640 1 -- 172.21.15.40:0/3253221272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972c0d7b50 con 0x7f972c04c610 2023-10-18T08:55:27.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.736+0000 7f971ffff640 1 -- 172.21.15.40:0/3253221272 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9734066850 con 0x7f972c04c610 2023-10-18T08:55:27.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.736+0000 7f8c60ade640 1 -- 172.21.15.40:0/1948687784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c5c0cedf0 msgr2=0x7f8c5c1514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.736+0000 7f8c60ade640 1 --2- 172.21.15.40:0/1948687784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c5c0cedf0 0x7f8c5c1514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.736+0000 7f8c60ade640 1 -- 172.21.15.40:0/1948687784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c0cca70 msgr2=0x7f8c5c14d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.736+0000 7f8c60ade640 1 --2- 172.21.15.40:0/1948687784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c5c0cca70 0x7f8c5c14d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.736+0000 7f8c60ade640 1 -- 172.21.15.40:0/1948687784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c009560 con 0x7f8c5c0cf7e0 2023-10-18T08:55:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.738+0000 7f88127fc640 1 -- 172.21.15.40:0/2780907168 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8808011870 con 0x7f881414b5d0 2023-10-18T08:55:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.738+0000 7f881acc3640 1 -- 172.21.15.40:0/2780907168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f88140b1a50 con 0x7f881414b5d0 2023-10-18T08:55:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.737+0000 7f1c62ffd640 1 -- 172.21.15.40:0/969310818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c640b33a0 msgr2=0x7f1c640b3820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T08:55:27.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.737+0000 7f1c62ffd640 1 --2- 172.21.15.40:0/969310818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c640b33a0 0x7f1c640b3820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.737+0000 7f1c62ffd640 1 -- 172.21.15.40:0/969310818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c64155b40 msgr2=0x7f1c64155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.737+0000 7f1c62ffd640 1 --2- 172.21.15.40:0/969310818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c64155b40 0x7f1c64155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.737+0000 7f1c62ffd640 1 -- 172.21.15.40:0/969310818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c64151050 con 0x7f1c4c003970 2023-10-18T08:55:27.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.736+0000 7f971ffff640 1 -- 172.21.15.40:0/3253221272 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97340699a0 con 0x7f972c04c610 2023-10-18T08:55:27.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.736+0000 7f8c5b7fe640 1 --2- 172.21.15.40:0/1948687784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c5c0cedf0 0x7f8c5c1514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:55:27.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.736+0000 7f9cb3fff640 1 -- 172.21.15.40:0/1114782324 wait complete. 2023-10-18T08:55:27.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.736+0000 7f35fad6e640 1 -- start start 2023-10-18T08:55:27.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.737+0000 7f35fad6e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec050170 0x7f35ec04e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.737+0000 7f35fad6e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35ec053c40 0x7f35ec0d99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.737+0000 7f35fad6e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35ec0df1f0 0x7f35ec0d9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.737+0000 7f35fad6e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec0d8310 con 0x7f35ec053c40 2023-10-18T08:55:27.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.737+0000 7f35fad6e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec0db020 con 0x7f35ec050170 2023-10-18T08:55:27.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.737+0000 7f35fad6e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ec0dab90 con 0x7f35ec0df1f0 2023-10-18T08:55:27.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.738+0000 7f1c62ffd640 1 --2- 172.21.15.40:0/969310818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c4c003970 0x7f1c641500d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00c590 tx=0x7f1c5c001910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.737+0000 7f9738c35640 1 -- 172.21.15.40:0/3253221272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972c034850 con 0x7f972c04c610 2023-10-18T08:55:27.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.738+0000 7f9738c35640 1 -- 172.21.15.40:0/3253221272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f972c033ec0 con 0x7f972c04c610 2023-10-18T08:55:27.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.737+0000 7f8c60ade640 1 --2- 172.21.15.40:0/1948687784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c5c0cf7e0 0x7f8c5c151a10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400c780 tx=0x7f8c5400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f9cb3fff640 1 Processor -- start 2023-10-18T08:55:27.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f35fa56d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35ec0df1f0 0x7f35ec0d9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f35f9d6c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec050170 0x7f35ec04e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f35fa56d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35ec0df1f0 0x7f35ec0d9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:45250/0 (socket says 172.21.15.40:45250) 2023-10-18T08:55:27.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f35fa56d640 1 -- 172.21.15.40:0/2750618205 learned_addr learned my addr 172.21.15.40:0/2750618205 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:27.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f35f956b640 1 --2- 172.21.15.40:0/2750618205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35ec053c40 0x7f35ec0d99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f8c58ff9640 1 -- 172.21.15.40:0/1948687784 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54003da0 con 0x7f8c5c0cf7e0 2023-10-18T08:55:27.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f8c58ff9640 1 -- 172.21.15.40:0/1948687784 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8c5400ce30 con 0x7f8c5c0cf7e0 2023-10-18T08:55:27.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f8c58ff9640 1 -- 172.21.15.40:0/1948687784 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c540052b0 con 0x7f8c5c0cf7e0 2023-10-18T08:55:27.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f88127fc640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87f806af10 0x7f87f806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f9cb3fff640 1 -- start start 2023-10-18T08:55:27.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f1c4b7fe640 1 -- 172.21.15.40:0/969310818 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c003b70 con 0x7f1c4c003970 2023-10-18T08:55:27.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f1c4b7fe640 1 -- 172.21.15.40:0/969310818 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1c5c003cd0 con 0x7f1c4c003970 2023-10-18T08:55:27.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f1c4b7fe640 1 -- 172.21.15.40:0/969310818 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c006320 con 0x7f1c4c003970 2023-10-18T08:55:27.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f88127fc640 1 -- 172.21.15.40:0/2780907168 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f880808a0d0 con 0x7f881414b5d0 2023-10-18T08:55:27.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f35fa56d640 1 -- 172.21.15.40:0/2750618205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec050170 msgr2=0x7f35ec04e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f35fa56d640 1 --2- 172.21.15.40:0/2750618205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35ec050170 0x7f35ec04e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f35fa56d640 1 -- 172.21.15.40:0/2750618205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35ec053c40 msgr2=0x7f35ec0d99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f35fa56d640 1 --2- 172.21.15.40:0/2750618205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35ec053c40 0x7f35ec0d99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.739+0000 7f35fa56d640 1 -- 172.21.15.40:0/2750618205 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4009490 con 0x7f35ec0df1f0 2023-10-18T08:55:27.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.740+0000 7f8c62541640 1 -- 172.21.15.40:0/1948687784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5c15c540 con 0x7f8c5c0cf7e0 2023-10-18T08:55:27.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.740+0000 7f88127fc640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.40:6800/1070603306,v1:172.21.15.40:6801/1070603306] conn(0x7f87f80730b0 0x7f87f80754d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.740+0000 7f88127fc640 1 -- 172.21.15.40:0/2780907168 --> [v2:172.21.15.40:6800/1070603306,v1:172.21.15.40:6801/1070603306] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87f8075b80 con 0x7f87f80730b0 2023-10-18T08:55:27.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.740+0000 7f88194c0640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87f806af10 0x7f87f806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.740+0000 7f1c69690640 1 -- 172.21.15.40:0/969310818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c640b3ef0 con 0x7f1c4c003970 2023-10-18T08:55:27.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.740+0000 7f881a4c2640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.40:6800/1070603306,v1:172.21.15.40:6801/1070603306] conn(0x7f87f80730b0 0x7f87f80754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.740+0000 7f1c69690640 1 -- 172.21.15.40:0/969310818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c640ca400 con 0x7f1c4c003970 2023-10-18T08:55:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.741+0000 7f9cb3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac050170 0x7f9cac04a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.741+0000 7f8c62541640 1 -- 172.21.15.40:0/1948687784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5c15cb00 con 0x7f8c5c0cf7e0 2023-10-18T08:55:27.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.741+0000 7f9cb3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cac053c40 0x7f9cac0d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.741+0000 7f9cb3fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cac0df1f0 0x7f9cac0d9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.742+0000 7f9cb3fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0e3910 con 0x7f9cac053c40 2023-10-18T08:55:27.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.742+0000 7f9cb3fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0e2cd0 con 0x7f9cac050170 2023-10-18T08:55:27.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.742+0000 7f9cb3fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0e2f70 con 0x7f9cac0df1f0 2023-10-18T08:55:27.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.742+0000 7f35fa56d640 1 --2- 172.21.15.40:0/2750618205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35ec0df1f0 0x7f35ec0d9f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f35f4063080 tx=0x7f35f4073940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.742+0000 7f881a4c2640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.40:6800/1070603306,v1:172.21.15.40:6801/1070603306] conn(0x7f87f80730b0 0x7f87f80754d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.742+0000 7f88127fc640 1 -- 172.21.15.40:0/2780907168 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8808001ee0 con 0x7f881414b5d0 2023-10-18T08:55:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.742+0000 7f9cb37fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cac0df1f0 0x7f9cac0d9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.742+0000 7f9cb37fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cac0df1f0 0x7f9cac0d9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:45254/0 (socket says 172.21.15.40:45254) 2023-10-18T08:55:27.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.742+0000 7f9cb37fe640 1 -- 172.21.15.40:0/1651990644 learned_addr learned my addr 172.21.15.40:0/1651990644 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:27.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.743+0000 7f9cb27fc640 1 --2- 172.21.15.40:0/1651990644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cac053c40 0x7f9cac0d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.743+0000 7f9cb2ffd640 1 --2- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac050170 0x7f9cac04a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.743+0000 7f35ea7fc640 1 -- 172.21.15.40:0/2750618205 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4069920 con 0x7f35ec0df1f0 2023-10-18T08:55:27.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.743+0000 7f35ea7fc640 1 -- 172.21.15.40:0/2750618205 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f35f4069bb0 con 0x7f35ec0df1f0 2023-10-18T08:55:27.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.743+0000 7f88127fc640 1 -- 172.21.15.40:0/2780907168 <== osd.0 v2:172.21.15.40:6800/1070603306 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f87f8075b80 con 0x7f87f80730b0 2023-10-18T08:55:27.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.743+0000 7f35ea7fc640 1 -- 172.21.15.40:0/2750618205 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4066990 con 0x7f35ec0df1f0 2023-10-18T08:55:27.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.743+0000 7f88194c0640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87f806af10 0x7f87f806d3d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8800006fd0 tx=0x7f8800006050 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.744+0000 7f35fad6e640 1 -- 172.21.15.40:0/2750618205 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ec0d8030 con 0x7f35ec0df1f0 2023-10-18T08:55:27.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.743+0000 7f9cb37fe640 1 -- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac050170 msgr2=0x7f9cac04a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.744+0000 7f971ffff640 1 -- 172.21.15.40:0/3253221272 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9734069b00 con 0x7f972c04c610 2023-10-18T08:55:27.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.744+0000 7f35fad6e640 1 -- 172.21.15.40:0/2750618205 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec0da820 con 0x7f35ec0df1f0 2023-10-18T08:55:27.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.744+0000 7f1c69690640 1 -- 172.21.15.40:0/969310818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c64066fd0 con 0x7f1c4c003970 2023-10-18T08:55:27.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.744+0000 7f8c62541640 1 -- 172.21.15.40:0/1948687784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8c5c0cd0d0 con 0x7f8c5c0cf7e0 2023-10-18T08:55:27.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.743+0000 7f9cb37fe640 1 --2- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac050170 0x7f9cac04a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.744+0000 7f9cb37fe640 1 -- 172.21.15.40:0/1651990644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cac053c40 msgr2=0x7f9cac0d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.744+0000 7f9cb37fe640 1 --2- 172.21.15.40:0/1651990644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cac053c40 0x7f9cac0d5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.744+0000 7f9cb37fe640 1 -- 172.21.15.40:0/1651990644 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca8009490 con 0x7f9cac0df1f0 2023-10-18T08:55:27.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.744+0000 7f9cb2ffd640 1 --2- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cac050170 0x7f9cac04a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:27.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.744+0000 7f9cb27fc640 1 --2- 172.21.15.40:0/1651990644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cac053c40 0x7f9cac0d5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:55:27.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.744+0000 7f9cb37fe640 1 --2- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cac0df1f0 0x7f9cac0d9260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40633a0 tx=0x7f9cb4073940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.745+0000 7f9c9b7fe640 1 -- 172.21.15.40:0/1651990644 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4066990 con 0x7f9cac0df1f0 2023-10-18T08:55:27.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.745+0000 7f9cb3fff640 1 -- 172.21.15.40:0/1651990644 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cac0e3250 con 0x7f9cac0df1f0 2023-10-18T08:55:27.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.745+0000 7f9cb3fff640 1 -- 172.21.15.40:0/1651990644 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cac0d10e0 con 0x7f9cac0df1f0 2023-10-18T08:55:27.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.745+0000 7f9c9b7fe640 1 -- 172.21.15.40:0/1651990644 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9cb4069720 con 0x7f9cac0df1f0 2023-10-18T08:55:27.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.745+0000 7f9c9b7fe640 1 -- 172.21.15.40:0/1651990644 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4069b10 con 0x7f9cac0df1f0 2023-10-18T08:55:27.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.746+0000 7f35fad6e640 1 -- 172.21.15.40:0/2750618205 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f35ec033ec0 con 0x7f35ec0df1f0 2023-10-18T08:55:27.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.745+0000 7f8c58ff9640 1 -- 172.21.15.40:0/1948687784 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8c54005410 con 0x7f8c5c0cf7e0 2023-10-18T08:55:27.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.747+0000 7f9cb3fff640 1 -- 172.21.15.40:0/1651990644 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9c94000fc0 con 0x7f9cac0df1f0 2023-10-18T08:55:27.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.747+0000 7f8c58ff9640 1 --2- 172.21.15.40:0/1948687784 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c4806ae40 0x7f8c4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.747+0000 7f971ffff640 1 --2- 172.21.15.40:0/3253221272 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f971806ae40 0x7f971806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.747+0000 7f971ffff640 1 -- 172.21.15.40:0/3253221272 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97341016f0 con 0x7f972c04c610 2023-10-18T08:55:27.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.748+0000 7f971ffff640 1 --2- 172.21.15.40:0/3253221272 >> [v2:172.21.15.40:6823/1187944209,v1:172.21.15.40:6825/1187944209] conn(0x7f9718072fe0 0x7f9718075400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.748+0000 7f971ffff640 1 -- 172.21.15.40:0/3253221272 --> [v2:172.21.15.40:6823/1187944209,v1:172.21.15.40:6825/1187944209] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9718075ab0 con 0x7f9718072fe0 2023-10-18T08:55:27.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.748+0000 7f971ffff640 1 -- 172.21.15.40:0/3253221272 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9734101b40 con 0x7f972c04c610 2023-10-18T08:55:27.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.748+0000 7f8c58ff9640 1 -- 172.21.15.40:0/1948687784 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c540890d0 con 0x7f8c5c0cf7e0 2023-10-18T08:55:27.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.748+0000 7f97337fe640 1 --2- 172.21.15.40:0/3253221272 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f971806ae40 0x7f971806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.748+0000 7f9732ffd640 1 --2- 172.21.15.40:0/3253221272 >> [v2:172.21.15.40:6823/1187944209,v1:172.21.15.40:6825/1187944209] conn(0x7f9718072fe0 0x7f9718075400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.748+0000 7f1c4b7fe640 1 -- 172.21.15.40:0/969310818 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1c5c031030 con 0x7f1c4c003970 2023-10-18T08:55:27.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.749+0000 7f97337fe640 1 --2- 172.21.15.40:0/3253221272 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f971806ae40 0x7f971806d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f97280017a0 tx=0x7f9728001470 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.748+0000 7f8c58ff9640 1 --2- 172.21.15.40:0/1948687784 >> [v2:172.21.15.40:6816/3903517270,v1:172.21.15.40:6817/3903517270] conn(0x7f8c48072fe0 0x7f8c48075400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.750+0000 7f8c58ff9640 1 -- 172.21.15.40:0/1948687784 --> [v2:172.21.15.40:6816/3903517270,v1:172.21.15.40:6817/3903517270] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c48075ab0 con 0x7f8c48072fe0 2023-10-18T08:55:27.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.750+0000 7f8c58ff9640 1 -- 172.21.15.40:0/1948687784 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c5408b790 con 0x7f8c5c0cf7e0 2023-10-18T08:55:27.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.750+0000 7f8c5bfff640 1 --2- 172.21.15.40:0/1948687784 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c4806ae40 0x7f8c4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.750+0000 7f8c5b7fe640 1 --2- 172.21.15.40:0/1948687784 >> [v2:172.21.15.40:6816/3903517270,v1:172.21.15.40:6817/3903517270] conn(0x7f8c48072fe0 0x7f8c48075400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.750+0000 7f35ea7fc640 1 -- 172.21.15.40:0/2750618205 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f35f4072b00 con 0x7f35ec0df1f0 2023-10-18T08:55:27.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.750+0000 7f9732ffd640 1 --2- 172.21.15.40:0/3253221272 >> [v2:172.21.15.40:6823/1187944209,v1:172.21.15.40:6825/1187944209] conn(0x7f9718072fe0 0x7f9718075400 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.751+0000 7f9c9b7fe640 1 -- 172.21.15.40:0/1651990644 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9cb4072b00 con 0x7f9cac0df1f0 2023-10-18T08:55:27.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.751+0000 7f1c4b7fe640 1 --2- 172.21.15.40:0/969310818 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1c5006af10 0x7f1c5006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.751+0000 7f1c4b7fe640 1 -- 172.21.15.40:0/969310818 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c5c0abd40 con 0x7f1c4c003970 2023-10-18T08:55:27.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.751+0000 7f1c4b7fe640 1 --2- 172.21.15.40:0/969310818 >> [v2:172.21.15.40:6808/2762063371,v1:172.21.15.40:6809/2762063371] conn(0x7f1c50073130 0x7f1c50075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.751+0000 7f1c4b7fe640 1 -- 172.21.15.40:0/969310818 --> [v2:172.21.15.40:6808/2762063371,v1:172.21.15.40:6809/2762063371] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c50075c00 con 0x7f1c50073130 2023-10-18T08:55:27.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.751+0000 7f1c4b7fe640 1 -- 172.21.15.40:0/969310818 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c5c0ac0f0 con 0x7f1c4c003970 2023-10-18T08:55:27.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.751+0000 7f1c627fc640 1 --2- 172.21.15.40:0/969310818 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1c5006af10 0x7f1c5006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.751+0000 7f01930f3640 1 -- 172.21.15.40:0/3311626767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c0ca890 msgr2=0x7f018c0b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.751+0000 7f01930f3640 1 --2- 172.21.15.40:0/3311626767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c0ca890 0x7f018c0b1d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f017c00a940 tx=0x7f017c02d000 comp rx=0 tx=0).stop 2023-10-18T08:55:27.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.751+0000 7f35ea7fc640 1 --2- 172.21.15.40:0/2750618205 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35d806f2d0 0x7f35d8071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f1d17b6c640 1 -- 172.21.15.40:0/1446842666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d10151960 msgr2=0x7f1d100b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f1d17b6c640 1 --2- 172.21.15.40:0/1446842666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d10151960 0x7f1d100b0bc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1d100b1d40 tx=0x7f1d0802c2c0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f01930f3640 1 -- 172.21.15.40:0/3311626767 shutdown_connections 2023-10-18T08:55:27.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f01930f3640 1 --2- 172.21.15.40:0/3311626767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c0ca890 0x7f018c0b1d10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f01930f3640 1 --2- 172.21.15.40:0/3311626767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f018c157870 0x7f018c0ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f01930f3640 1 -- 172.21.15.40:0/3311626767 >> 172.21.15.40:0/3311626767 conn(0x7f018c069720 msgr2=0x7f018c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f35ea7fc640 1 -- 172.21.15.40:0/2750618205 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35f4101e70 con 0x7f35ec0df1f0 2023-10-18T08:55:27.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f1d17b6c640 1 -- 172.21.15.40:0/1446842666 shutdown_connections 2023-10-18T08:55:27.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f1d17b6c640 1 --2- 172.21.15.40:0/1446842666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d10151e10 0x7f1d100b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f1d17b6c640 1 --2- 172.21.15.40:0/1446842666 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d10151960 0x7f1d100b0bc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f1d17b6c640 1 -- 172.21.15.40:0/1446842666 >> 172.21.15.40:0/1446842666 conn(0x7f1d1006cee0 msgr2=0x7f1d10069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f01930f3640 1 -- 172.21.15.40:0/3311626767 shutdown_connections 2023-10-18T08:55:27.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f01930f3640 1 -- 172.21.15.40:0/3311626767 wait complete. 2023-10-18T08:55:27.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f1d17b6c640 1 -- 172.21.15.40:0/1446842666 shutdown_connections 2023-10-18T08:55:27.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f8c5bfff640 1 --2- 172.21.15.40:0/1948687784 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c4806ae40 0x7f8c4806d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c006010 tx=0x7f8c4c0046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f35f9d6c640 1 --2- 172.21.15.40:0/2750618205 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35d806f2d0 0x7f35d8071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.752+0000 7f1d17b6c640 1 -- 172.21.15.40:0/1446842666 wait complete. 2023-10-18T08:55:27.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.753+0000 7f35ea7fc640 1 --2- 172.21.15.40:0/2750618205 >> [v2:172.21.15.144:6802/1833904789,v1:172.21.15.144:6803/1833904789] conn(0x7f35d8077470 0x7f35d8079890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.753+0000 7f9c9b7fe640 1 --2- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9ca006ae40 0x7f9ca006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.753+0000 7f9c9b7fe640 1 -- 172.21.15.40:0/1651990644 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9cb4101e10 con 0x7f9cac0df1f0 2023-10-18T08:55:27.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.753+0000 7f35ea7fc640 1 -- 172.21.15.40:0/2750618205 --> [v2:172.21.15.144:6802/1833904789,v1:172.21.15.144:6803/1833904789] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35d8079f40 con 0x7f35d8077470 2023-10-18T08:55:27.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.753+0000 7f1c637fe640 1 --2- 172.21.15.40:0/969310818 >> [v2:172.21.15.40:6808/2762063371,v1:172.21.15.40:6809/2762063371] conn(0x7f1c50073130 0x7f1c50075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.753+0000 7f8c5b7fe640 1 --2- 172.21.15.40:0/1948687784 >> [v2:172.21.15.40:6816/3903517270,v1:172.21.15.40:6817/3903517270] conn(0x7f8c48072fe0 0x7f8c48075400 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.753+0000 7f971ffff640 1 -- 172.21.15.40:0/3253221272 <== osd.2 v2:172.21.15.40:6823/1187944209 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f9718075ab0 con 0x7f9718072fe0 2023-10-18T08:55:27.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.753+0000 7f9c9b7fe640 1 --2- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:6810/1718994813,v1:172.21.15.144:6811/1718994813] conn(0x7f9ca0073060 0x7f9ca0075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.753+0000 7f9c9b7fe640 1 -- 172.21.15.40:0/1651990644 --> [v2:172.21.15.144:6810/1718994813,v1:172.21.15.144:6811/1718994813] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ca0075b30 con 0x7f9ca0073060 2023-10-18T08:55:27.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.753+0000 7f9c9b7fe640 1 -- 172.21.15.40:0/1651990644 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cb41021c0 con 0x7f9cac0df1f0 2023-10-18T08:55:27.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.753+0000 7f35ea7fc640 1 -- 172.21.15.40:0/2750618205 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35f40ccee0 con 0x7f35ec0df1f0 2023-10-18T08:55:27.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.753+0000 7f1c627fc640 1 --2- 172.21.15.40:0/969310818 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1c5006af10 0x7f1c5006d3d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c54005c80 tx=0x7f1c54005a20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.754+0000 7f1c637fe640 1 --2- 172.21.15.40:0/969310818 >> [v2:172.21.15.40:6808/2762063371,v1:172.21.15.40:6809/2762063371] conn(0x7f1c50073130 0x7f1c50075550 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.754+0000 7f9cb27fc640 1 --2- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:6810/1718994813,v1:172.21.15.144:6811/1718994813] conn(0x7f9ca0073060 0x7f9ca0075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.754+0000 7f1d17b6c640 1 Processor -- start 2023-10-18T08:55:27.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.755+0000 7f8c58ff9640 1 -- 172.21.15.40:0/1948687784 <== osd.3 v2:172.21.15.40:6816/3903517270 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f8c48075ab0 con 0x7f8c48072fe0 2023-10-18T08:55:27.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.755+0000 7f9cb2ffd640 1 --2- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9ca006ae40 0x7f9ca006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.755+0000 7f1d17b6c640 1 -- start start 2023-10-18T08:55:27.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.754+0000 7f35f956b640 1 --2- 172.21.15.40:0/2750618205 >> [v2:172.21.15.144:6802/1833904789,v1:172.21.15.144:6803/1833904789] conn(0x7f35d8077470 0x7f35d8079890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.755+0000 7f35f9d6c640 1 --2- 172.21.15.40:0/2750618205 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35d806f2d0 0x7f35d8071790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f35f0007fd0 tx=0x7f35f0007cc0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.755+0000 7f1c4b7fe640 1 -- 172.21.15.40:0/969310818 <== osd.1 v2:172.21.15.40:6808/2762063371 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f1c50075c00 con 0x7f1c50073130 2023-10-18T08:55:27.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.755+0000 7f01930f3640 1 Processor -- start 2023-10-18T08:55:27.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.756+0000 7f01930f3640 1 -- start start 2023-10-18T08:55:27.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.756+0000 7f1d17b6c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d1006d710 0x7f1d10145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.756+0000 7f9cb27fc640 1 --2- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:6810/1718994813,v1:172.21.15.144:6811/1718994813] conn(0x7f9ca0073060 0x7f9ca0075480 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.756+0000 7f1d17b6c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d10151960 0x7f1d101f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.756+0000 7f1d17b6c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d10151e10 0x7f1d101fa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.756+0000 7f1d17b6c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1014ead0 con 0x7f1d10151960 2023-10-18T08:55:27.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.756+0000 7f1d17b6c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1014dec0 con 0x7f1d1006d710 2023-10-18T08:55:27.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.756+0000 7f1d17b6c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1014e1a0 con 0x7f1d10151e10 2023-10-18T08:55:27.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.756+0000 7f9cb2ffd640 1 --2- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9ca006ae40 0x7f9ca006d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8036000 tx=0x7f9ca8004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.756+0000 7f35f956b640 1 --2- 172.21.15.40:0/2750618205 >> [v2:172.21.15.144:6802/1833904789,v1:172.21.15.144:6803/1833904789] conn(0x7f35d8077470 0x7f35d8079890 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.756+0000 7f1d15107640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d10151960 0x7f1d101f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.757+0000 7f1d15107640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d10151960 0x7f1d101f9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47928/0 (socket says 172.21.15.40:47928) 2023-10-18T08:55:27.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.757+0000 7f1d15107640 1 -- 172.21.15.40:0/3310635705 learned_addr learned my addr 172.21.15.40:0/3310635705 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:27.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.757+0000 7f1d15908640 1 --2- 172.21.15.40:0/3310635705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d1006d710 0x7f1d10145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.757+0000 7f1d16109640 1 --2- 172.21.15.40:0/3310635705 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d10151e10 0x7f1d101fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.757+0000 7f01930f3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c0b2250 0x7f018c1459a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.757+0000 7f01930f3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f018c0ca890 0x7f018c154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.757+0000 7f01930f3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f018c157870 0x7f018c15b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.757+0000 7f01930f3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c1624a0 con 0x7f018c157870 2023-10-18T08:55:27.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.757+0000 7f01930f3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c145ee0 con 0x7f018c0b2250 2023-10-18T08:55:27.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.757+0000 7f01930f3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018c1557b0 con 0x7f018c0ca890 2023-10-18T08:55:27.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.757+0000 7f1d15107640 1 -- 172.21.15.40:0/3310635705 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d10151e10 msgr2=0x7f1d101fa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.757+0000 7f1d15107640 1 --2- 172.21.15.40:0/3310635705 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d10151e10 0x7f1d101fa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.757+0000 7f1d15107640 1 -- 172.21.15.40:0/3310635705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d1006d710 msgr2=0x7f1d10145c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.757+0000 7f1d15107640 1 --2- 172.21.15.40:0/3310635705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d1006d710 0x7f1d10145c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.757+0000 7f1d15107640 1 -- 172.21.15.40:0/3310635705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d08009900 con 0x7f1d10151960 2023-10-18T08:55:27.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.758+0000 7f01928f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f018c157870 0x7f018c15b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.758+0000 7f01918f0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f018c0ca890 0x7f018c154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.758+0000 7f01918f0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f018c0ca890 0x7f018c154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:45292/0 (socket says 172.21.15.40:45292) 2023-10-18T08:55:27.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.758+0000 7f01918f0640 1 -- 172.21.15.40:0/3677948029 learned_addr learned my addr 172.21.15.40:0/3677948029 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:27.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.758+0000 7f01928f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f018c157870 0x7f018c15b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47940/0 (socket says 172.21.15.40:47940) 2023-10-18T08:55:27.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.758+0000 7f01920f1640 1 --2- 172.21.15.40:0/3677948029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c0b2250 0x7f018c1459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.758+0000 7f1d15107640 1 --2- 172.21.15.40:0/3310635705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d10151960 0x7f1d101f9c40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1d08000f80 tx=0x7f1d08003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.759+0000 7f1d027fc640 1 -- 172.21.15.40:0/3310635705 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d08004240 con 0x7f1d10151960 2023-10-18T08:55:27.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.759+0000 7f01928f2640 1 -- 172.21.15.40:0/3677948029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f018c0ca890 msgr2=0x7f018c154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.759+0000 7f01928f2640 1 --2- 172.21.15.40:0/3677948029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f018c0ca890 0x7f018c154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.759+0000 7f01928f2640 1 -- 172.21.15.40:0/3677948029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c0b2250 msgr2=0x7f018c1459a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.759+0000 7f01928f2640 1 --2- 172.21.15.40:0/3677948029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c0b2250 0x7f018c1459a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.759+0000 7f01928f2640 1 -- 172.21.15.40:0/3677948029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017c00a5a0 con 0x7f018c157870 2023-10-18T08:55:27.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.759+0000 7f01920f1640 1 --2- 172.21.15.40:0/3677948029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c0b2250 0x7f018c1459a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T08:55:27.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.759+0000 7f9c9b7fe640 1 -- 172.21.15.40:0/1651990644 <== osd.4 v2:172.21.15.144:6810/1718994813 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f9ca0075b30 con 0x7f9ca0073060 2023-10-18T08:55:27.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.759+0000 7f1d027fc640 1 -- 172.21.15.40:0/3310635705 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1d08004490 con 0x7f1d10151960 2023-10-18T08:55:27.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.759+0000 7f1d17b6c640 1 -- 172.21.15.40:0/3310635705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1014e480 con 0x7f1d10151960 2023-10-18T08:55:27.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.759+0000 7f1d027fc640 1 -- 172.21.15.40:0/3310635705 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d08004240 con 0x7f1d10151960 2023-10-18T08:55:27.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.759+0000 7f1d17b6c640 1 -- 172.21.15.40:0/3310635705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d101461d0 con 0x7f1d10151960 2023-10-18T08:55:27.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.759+0000 7f35ea7fc640 1 -- 172.21.15.40:0/2750618205 <== osd.5 v2:172.21.15.144:6802/1833904789 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f35d8079f40 con 0x7f35d8077470 2023-10-18T08:55:27.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.760+0000 7f01928f2640 1 --2- 172.21.15.40:0/3677948029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f018c157870 0x7f018c15b5b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f018400caa0 tx=0x7f018400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.760+0000 7f0182ffd640 1 -- 172.21.15.40:0/3677948029 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018400ac00 con 0x7f018c157870 2023-10-18T08:55:27.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.760+0000 7f0182ffd640 1 -- 172.21.15.40:0/3677948029 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f018400ae50 con 0x7f018c157870 2023-10-18T08:55:27.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.760+0000 7f0182ffd640 1 -- 172.21.15.40:0/3677948029 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0184012680 con 0x7f018c157870 2023-10-18T08:55:27.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.762+0000 7f01930f3640 1 -- 172.21.15.40:0/3677948029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f018c155a90 con 0x7f018c157870 2023-10-18T08:55:27.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.762+0000 7f01930f3640 1 -- 172.21.15.40:0/3677948029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f018c156050 con 0x7f018c157870 2023-10-18T08:55:27.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.763+0000 7f01930f3640 1 -- 172.21.15.40:0/3677948029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f018c0c9ac0 con 0x7f018c157870 2023-10-18T08:55:27.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.763+0000 7f1d17b6c640 1 -- 172.21.15.40:0/3310635705 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1d10066fd0 con 0x7f1d10151960 2023-10-18T08:55:27.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.765+0000 7f1d027fc640 1 -- 172.21.15.40:0/3310635705 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1d0803b040 con 0x7f1d10151960 2023-10-18T08:55:27.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.767+0000 7f1d027fc640 1 --2- 172.21.15.40:0/3310635705 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cf406aea0 0x7f1cf406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.767+0000 7f1d027fc640 1 -- 172.21.15.40:0/3310635705 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d080ac0d0 con 0x7f1d10151960 2023-10-18T08:55:27.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.768+0000 7f1d027fc640 1 --2- 172.21.15.40:0/3310635705 >> [v2:172.21.15.144:6824/3907866946,v1:172.21.15.144:6826/3907866946] conn(0x7f1cf4073040 0x7f1cf4075460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.768+0000 7f1d027fc640 1 -- 172.21.15.40:0/3310635705 --> [v2:172.21.15.144:6824/3907866946,v1:172.21.15.144:6826/3907866946] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1cf4075b10 con 0x7f1cf4073040 2023-10-18T08:55:27.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.767+0000 7f0182ffd640 1 -- 172.21.15.40:0/3677948029 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0184024430 con 0x7f018c157870 2023-10-18T08:55:27.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.768+0000 7f1d027fc640 1 -- 172.21.15.40:0/3310635705 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d080771a0 con 0x7f1d10151960 2023-10-18T08:55:27.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.769+0000 7f1d15908640 1 --2- 172.21.15.40:0/3310635705 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cf406aea0 0x7f1cf406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.769+0000 7f1d16109640 1 --2- 172.21.15.40:0/3310635705 >> [v2:172.21.15.144:6824/3907866946,v1:172.21.15.144:6826/3907866946] conn(0x7f1cf4073040 0x7f1cf4075460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.769+0000 7f0182ffd640 1 --2- 172.21.15.40:0/3677948029 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f017006aec0 0x7f017006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.770+0000 7f0182ffd640 1 -- 172.21.15.40:0/3677948029 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f018408a080 con 0x7f018c157870 2023-10-18T08:55:27.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.770+0000 7f1d16109640 1 --2- 172.21.15.40:0/3310635705 >> [v2:172.21.15.144:6824/3907866946,v1:172.21.15.144:6826/3907866946] conn(0x7f1cf4073040 0x7f1cf4075460 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.770+0000 7f1d15908640 1 --2- 172.21.15.40:0/3310635705 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cf406aea0 0x7f1cf406d360 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1d10069e00 tx=0x7f1d04009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.770+0000 7f01920f1640 1 --2- 172.21.15.40:0/3677948029 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f017006aec0 0x7f017006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.771+0000 7f0182ffd640 1 --2- 172.21.15.40:0/3677948029 >> [v2:172.21.15.144:6818/1988482345,v1:172.21.15.144:6819/1988482345] conn(0x7f01700730e0 0x7f0170075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:27.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.771+0000 7f0182ffd640 1 -- 172.21.15.40:0/3677948029 --> [v2:172.21.15.144:6818/1988482345,v1:172.21.15.144:6819/1988482345] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0170075bb0 con 0x7f01700730e0 2023-10-18T08:55:27.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.771+0000 7f0182ffd640 1 -- 172.21.15.40:0/3677948029 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f018408a520 con 0x7f018c157870 2023-10-18T08:55:27.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.771+0000 7f01918f0640 1 --2- 172.21.15.40:0/3677948029 >> [v2:172.21.15.144:6818/1988482345,v1:172.21.15.144:6819/1988482345] conn(0x7f01700730e0 0x7f0170075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:27.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.772+0000 7f01920f1640 1 --2- 172.21.15.40:0/3677948029 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f017006aec0 0x7f017006d380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0188009f20 tx=0x7f0188005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.772+0000 7f1d027fc640 1 -- 172.21.15.40:0/3310635705 <== osd.7 v2:172.21.15.144:6824/3907866946 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f1cf4075b10 con 0x7f1cf4073040 2023-10-18T08:55:27.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.772+0000 7f01918f0640 1 --2- 172.21.15.40:0/3677948029 >> [v2:172.21.15.144:6818/1988482345,v1:172.21.15.144:6819/1988482345] conn(0x7f01700730e0 0x7f0170075500 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:27.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.774+0000 7f0182ffd640 1 -- 172.21.15.40:0/3677948029 <== osd.6 v2:172.21.15.144:6818/1988482345 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f0170075bb0 con 0x7f01700730e0 2023-10-18T08:55:27.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.778+0000 7f9738c35640 1 -- 172.21.15.40:0/3253221272 --> [v2:172.21.15.40:6823/1187944209,v1:172.21.15.40:6825/1187944209] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f972c0d8070 con 0x7f9718072fe0 2023-10-18T08:55:27.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.778+0000 7f971ffff640 1 -- 172.21.15.40:0/3253221272 <== osd.2 v2:172.21.15.40:6823/1187944209 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f972c0d8070 con 0x7f9718072fe0 2023-10-18T08:55:27.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.779+0000 7f9738c35640 1 -- 172.21.15.40:0/3253221272 >> [v2:172.21.15.40:6823/1187944209,v1:172.21.15.40:6825/1187944209] conn(0x7f9718072fe0 msgr2=0x7f9718075400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.779+0000 7f9738c35640 1 --2- 172.21.15.40:0/3253221272 >> [v2:172.21.15.40:6823/1187944209,v1:172.21.15.40:6825/1187944209] conn(0x7f9718072fe0 0x7f9718075400 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.779+0000 7f9738c35640 1 -- 172.21.15.40:0/3253221272 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f971806ae40 msgr2=0x7f971806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.779+0000 7f9738c35640 1 --2- 172.21.15.40:0/3253221272 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f971806ae40 0x7f971806d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f97280017a0 tx=0x7f9728001470 comp rx=0 tx=0).stop 2023-10-18T08:55:27.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.779+0000 7f9738c35640 1 -- 172.21.15.40:0/3253221272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c04c610 msgr2=0x7f972c04ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.779+0000 7f9738c35640 1 --2- 172.21.15.40:0/3253221272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f972c04c610 0x7f972c04ca90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f97340633f0 tx=0x7f9734073940 comp rx=0 tx=0).stop 2023-10-18T08:55:27.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.779+0000 7f881acc3640 1 -- 172.21.15.40:0/2780907168 --> [v2:172.21.15.40:6800/1070603306,v1:172.21.15.40:6801/1070603306] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f881406d900 con 0x7f87f80730b0 2023-10-18T08:55:27.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.779+0000 7f9733fff640 1 -- 172.21.15.40:0/3253221272 reap_dead start 2023-10-18T08:55:27.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.780+0000 7f9738c35640 1 -- 172.21.15.40:0/3253221272 shutdown_connections 2023-10-18T08:55:27.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.780+0000 7f9738c35640 1 -- 172.21.15.40:0/3253221272 >> 172.21.15.40:0/3253221272 conn(0x7f972c009f80 msgr2=0x7f972c0e1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.780+0000 7f9738c35640 1 -- 172.21.15.40:0/3253221272 shutdown_connections 2023-10-18T08:55:27.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.780+0000 7f9738c35640 1 -- 172.21.15.40:0/3253221272 wait complete. 2023-10-18T08:55:27.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.785+0000 7f35fad6e640 1 -- 172.21.15.40:0/2750618205 --> [v2:172.21.15.144:6802/1833904789,v1:172.21.15.144:6803/1833904789] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f35ec005360 con 0x7f35d8077470 2023-10-18T08:55:27.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.785+0000 7f88127fc640 1 -- 172.21.15.40:0/2780907168 <== osd.0 v2:172.21.15.40:6800/1070603306 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f880c02de20 con 0x7f87f80730b0 2023-10-18T08:55:27.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.786+0000 7f881acc3640 1 -- 172.21.15.40:0/2780907168 >> [v2:172.21.15.40:6800/1070603306,v1:172.21.15.40:6801/1070603306] conn(0x7f87f80730b0 msgr2=0x7f87f80754d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.786+0000 7f881acc3640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.40:6800/1070603306,v1:172.21.15.40:6801/1070603306] conn(0x7f87f80730b0 0x7f87f80754d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.786+0000 7f881acc3640 1 -- 172.21.15.40:0/2780907168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87f806af10 msgr2=0x7f87f806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.786+0000 7f881acc3640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87f806af10 0x7f87f806d3d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8800006fd0 tx=0x7f8800006050 comp rx=0 tx=0).stop 2023-10-18T08:55:27.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.786+0000 7f881acc3640 1 -- 172.21.15.40:0/2780907168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f881414b5d0 msgr2=0x7f881414e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.786+0000 7f881acc3640 1 --2- 172.21.15.40:0/2780907168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f881414b5d0 0x7f881414e380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f880800e900 tx=0x7f880800edc0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.787+0000 7f881a4c2640 1 -- 172.21.15.40:0/2780907168 reap_dead start 2023-10-18T08:55:27.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.787+0000 7f881acc3640 1 -- 172.21.15.40:0/2780907168 shutdown_connections 2023-10-18T08:55:27.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.787+0000 7f881acc3640 1 -- 172.21.15.40:0/2780907168 >> 172.21.15.40:0/2780907168 conn(0x7f8814069720 msgr2=0x7f88141555f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.787+0000 7f881acc3640 1 -- 172.21.15.40:0/2780907168 shutdown_connections 2023-10-18T08:55:27.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.787+0000 7f881acc3640 1 -- 172.21.15.40:0/2780907168 wait complete. 2023-10-18T08:55:27.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.787+0000 7f1c69690640 1 -- 172.21.15.40:0/969310818 --> [v2:172.21.15.40:6808/2762063371,v1:172.21.15.40:6809/2762063371] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1c640cb980 con 0x7f1c50073130 2023-10-18T08:55:27.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.788+0000 7f35ea7fc640 1 -- 172.21.15.40:0/2750618205 <== osd.5 v2:172.21.15.144:6802/1833904789 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f35ec005360 con 0x7f35d8077470 2023-10-18T08:55:27.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.788+0000 7f1c4b7fe640 1 -- 172.21.15.40:0/969310818 <== osd.1 v2:172.21.15.40:6808/2762063371 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1c640cb980 con 0x7f1c50073130 2023-10-18T08:55:27.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.788+0000 7f8c62541640 1 -- 172.21.15.40:0/1948687784 --> [v2:172.21.15.40:6816/3903517270,v1:172.21.15.40:6817/3903517270] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8c5c15c3d0 con 0x7f8c48072fe0 2023-10-18T08:55:27.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.789+0000 7f35fad6e640 1 -- 172.21.15.40:0/2750618205 >> [v2:172.21.15.144:6802/1833904789,v1:172.21.15.144:6803/1833904789] conn(0x7f35d8077470 msgr2=0x7f35d8079890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.789+0000 7f35fad6e640 1 --2- 172.21.15.40:0/2750618205 >> [v2:172.21.15.144:6802/1833904789,v1:172.21.15.144:6803/1833904789] conn(0x7f35d8077470 0x7f35d8079890 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.789+0000 7f35fad6e640 1 -- 172.21.15.40:0/2750618205 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35d806f2d0 msgr2=0x7f35d8071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.789+0000 7f35fad6e640 1 --2- 172.21.15.40:0/2750618205 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35d806f2d0 0x7f35d8071790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f35f0007fd0 tx=0x7f35f0007cc0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.789+0000 7f35fad6e640 1 -- 172.21.15.40:0/2750618205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35ec0df1f0 msgr2=0x7f35ec0d9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.789+0000 7f35fad6e640 1 --2- 172.21.15.40:0/2750618205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35ec0df1f0 0x7f35ec0d9f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f35f4063080 tx=0x7f35f4073940 comp rx=0 tx=0).stop 2023-10-18T08:55:27.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.789+0000 7f35fa56d640 1 -- 172.21.15.40:0/2750618205 reap_dead start 2023-10-18T08:55:27.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.789+0000 7f35fad6e640 1 -- 172.21.15.40:0/2750618205 shutdown_connections 2023-10-18T08:55:27.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.789+0000 7f35fad6e640 1 -- 172.21.15.40:0/2750618205 >> 172.21.15.40:0/2750618205 conn(0x7f35ec009f80 msgr2=0x7f35ec036df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.790+0000 7f1c69690640 1 -- 172.21.15.40:0/969310818 >> [v2:172.21.15.40:6808/2762063371,v1:172.21.15.40:6809/2762063371] conn(0x7f1c50073130 msgr2=0x7f1c50075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.790+0000 7f1c69690640 1 --2- 172.21.15.40:0/969310818 >> [v2:172.21.15.40:6808/2762063371,v1:172.21.15.40:6809/2762063371] conn(0x7f1c50073130 0x7f1c50075550 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.790+0000 7f1c69690640 1 -- 172.21.15.40:0/969310818 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1c5006af10 msgr2=0x7f1c5006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.790+0000 7f1c69690640 1 --2- 172.21.15.40:0/969310818 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1c5006af10 0x7f1c5006d3d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c54005c80 tx=0x7f1c54005a20 comp rx=0 tx=0).stop 2023-10-18T08:55:27.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.790+0000 7f1c69690640 1 -- 172.21.15.40:0/969310818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c4c003970 msgr2=0x7f1c641500d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.790+0000 7f1c69690640 1 --2- 172.21.15.40:0/969310818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c4c003970 0x7f1c641500d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00c590 tx=0x7f1c5c001910 comp rx=0 tx=0).stop 2023-10-18T08:55:27.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.790+0000 7f1c637fe640 1 -- 172.21.15.40:0/969310818 reap_dead start 2023-10-18T08:55:27.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.790+0000 7f1c69690640 1 -- 172.21.15.40:0/969310818 shutdown_connections 2023-10-18T08:55:27.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.790+0000 7f8c58ff9640 1 -- 172.21.15.40:0/1948687784 <== osd.3 v2:172.21.15.40:6816/3903517270 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8c5c15c3d0 con 0x7f8c48072fe0 2023-10-18T08:55:27.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.790+0000 7f1c69690640 1 -- 172.21.15.40:0/969310818 >> 172.21.15.40:0/969310818 conn(0x7f1c6406a780 msgr2=0x7f1c6415afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.791+0000 7f1c69690640 1 -- 172.21.15.40:0/969310818 shutdown_connections 2023-10-18T08:55:27.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.791+0000 7f9cb3fff640 1 -- 172.21.15.40:0/1651990644 --> [v2:172.21.15.144:6810/1718994813,v1:172.21.15.144:6811/1718994813] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9c94002cb0 con 0x7f9ca0073060 2023-10-18T08:55:27.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.791+0000 7f8c62541640 1 -- 172.21.15.40:0/1948687784 >> [v2:172.21.15.40:6816/3903517270,v1:172.21.15.40:6817/3903517270] conn(0x7f8c48072fe0 msgr2=0x7f8c48075400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.791+0000 7f8c62541640 1 --2- 172.21.15.40:0/1948687784 >> [v2:172.21.15.40:6816/3903517270,v1:172.21.15.40:6817/3903517270] conn(0x7f8c48072fe0 0x7f8c48075400 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.792+0000 7f35fad6e640 1 -- 172.21.15.40:0/2750618205 shutdown_connections 2023-10-18T08:55:27.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.792+0000 7f35fad6e640 1 -- 172.21.15.40:0/2750618205 wait complete. 2023-10-18T08:55:27.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.792+0000 7f1c69690640 1 -- 172.21.15.40:0/969310818 wait complete. 2023-10-18T08:55:27.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.792+0000 7f9c9b7fe640 1 -- 172.21.15.40:0/1651990644 <== osd.4 v2:172.21.15.144:6810/1718994813 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9c94002cb0 con 0x7f9ca0073060 2023-10-18T08:55:27.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.793+0000 7f8c62541640 1 -- 172.21.15.40:0/1948687784 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c4806ae40 msgr2=0x7f8c4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.793+0000 7f8c62541640 1 --2- 172.21.15.40:0/1948687784 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c4806ae40 0x7f8c4806d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c006010 tx=0x7f8c4c0046c0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.793+0000 7f8c62541640 1 -- 172.21.15.40:0/1948687784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c5c0cf7e0 msgr2=0x7f8c5c151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.793+0000 7f8c62541640 1 --2- 172.21.15.40:0/1948687784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c5c0cf7e0 0x7f8c5c151a10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400c780 tx=0x7f8c5400cc40 comp rx=0 tx=0).stop 2023-10-18T08:55:27.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.793+0000 7f8c60ade640 1 -- 172.21.15.40:0/1948687784 reap_dead start 2023-10-18T08:55:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.793+0000 7f8c62541640 1 -- 172.21.15.40:0/1948687784 shutdown_connections 2023-10-18T08:55:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.793+0000 7f8c62541640 1 -- 172.21.15.40:0/1948687784 >> 172.21.15.40:0/1948687784 conn(0x7f8c5c09a580 msgr2=0x7f8c5c0c6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.793+0000 7f8c62541640 1 -- 172.21.15.40:0/1948687784 shutdown_connections 2023-10-18T08:55:27.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.793+0000 7f8c62541640 1 -- 172.21.15.40:0/1948687784 wait complete. 2023-10-18T08:55:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.794+0000 7f9cb3fff640 1 -- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:6810/1718994813,v1:172.21.15.144:6811/1718994813] conn(0x7f9ca0073060 msgr2=0x7f9ca0075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.794+0000 7f9cb3fff640 1 --2- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:6810/1718994813,v1:172.21.15.144:6811/1718994813] conn(0x7f9ca0073060 0x7f9ca0075480 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.794+0000 7f9cb3fff640 1 -- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9ca006ae40 msgr2=0x7f9ca006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.794+0000 7f9cb3fff640 1 --2- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9ca006ae40 0x7f9ca006d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8036000 tx=0x7f9ca8004730 comp rx=0 tx=0).stop 2023-10-18T08:55:27.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.794+0000 7f9cb3fff640 1 -- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cac0df1f0 msgr2=0x7f9cac0d9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.794+0000 7f9cb3fff640 1 --2- 172.21.15.40:0/1651990644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cac0df1f0 0x7f9cac0d9260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40633a0 tx=0x7f9cb4073940 comp rx=0 tx=0).stop 2023-10-18T08:55:27.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.794+0000 7f9cb37fe640 1 -- 172.21.15.40:0/1651990644 reap_dead start 2023-10-18T08:55:27.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.794+0000 7f9cb3fff640 1 -- 172.21.15.40:0/1651990644 shutdown_connections 2023-10-18T08:55:27.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.794+0000 7f9cb3fff640 1 -- 172.21.15.40:0/1651990644 >> 172.21.15.40:0/1651990644 conn(0x7f9cac009f80 msgr2=0x7f9cac004900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.795+0000 7f9cb3fff640 1 -- 172.21.15.40:0/1651990644 shutdown_connections 2023-10-18T08:55:27.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.795+0000 7f9cb3fff640 1 -- 172.21.15.40:0/1651990644 wait complete. 2023-10-18T08:55:27.864 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2023-10-18T08:55:27.864 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-18T08:55:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.802+0000 7f01930f3640 1 -- 172.21.15.40:0/3677948029 --> [v2:172.21.15.144:6818/1988482345,v1:172.21.15.144:6819/1988482345] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f018c066e00 con 0x7f01700730e0 2023-10-18T08:55:27.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.804+0000 7f0182ffd640 1 -- 172.21.15.40:0/3677948029 <== osd.6 v2:172.21.15.144:6818/1988482345 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f018c066e00 con 0x7f01700730e0 2023-10-18T08:55:27.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.804+0000 7f1d17b6c640 1 -- 172.21.15.40:0/3310635705 --> [v2:172.21.15.144:6824/3907866946,v1:172.21.15.144:6826/3907866946] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1d1014ec10 con 0x7f1cf4073040 2023-10-18T08:55:27.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.804+0000 7f01930f3640 1 -- 172.21.15.40:0/3677948029 >> [v2:172.21.15.144:6818/1988482345,v1:172.21.15.144:6819/1988482345] conn(0x7f01700730e0 msgr2=0x7f0170075500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.804+0000 7f01930f3640 1 --2- 172.21.15.40:0/3677948029 >> [v2:172.21.15.144:6818/1988482345,v1:172.21.15.144:6819/1988482345] conn(0x7f01700730e0 0x7f0170075500 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.804+0000 7f01930f3640 1 -- 172.21.15.40:0/3677948029 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f017006aec0 msgr2=0x7f017006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.804+0000 7f01930f3640 1 --2- 172.21.15.40:0/3677948029 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f017006aec0 0x7f017006d380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0188009f20 tx=0x7f0188005f90 comp rx=0 tx=0).stop 2023-10-18T08:55:27.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.804+0000 7f01930f3640 1 -- 172.21.15.40:0/3677948029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f018c157870 msgr2=0x7f018c15b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.804+0000 7f01930f3640 1 --2- 172.21.15.40:0/3677948029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f018c157870 0x7f018c15b5b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f018400caa0 tx=0x7f018400cf60 comp rx=0 tx=0).stop 2023-10-18T08:55:27.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.805+0000 7f01928f2640 1 -- 172.21.15.40:0/3677948029 reap_dead start 2023-10-18T08:55:27.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.805+0000 7f01930f3640 1 -- 172.21.15.40:0/3677948029 shutdown_connections 2023-10-18T08:55:27.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.805+0000 7f01930f3640 1 -- 172.21.15.40:0/3677948029 >> 172.21.15.40:0/3677948029 conn(0x7f018c069720 msgr2=0x7f018c161a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.805+0000 7f01930f3640 1 -- 172.21.15.40:0/3677948029 shutdown_connections 2023-10-18T08:55:27.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.805+0000 7f01930f3640 1 -- 172.21.15.40:0/3677948029 wait complete. 2023-10-18T08:55:27.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.805+0000 7f1d027fc640 1 -- 172.21.15.40:0/3310635705 <== osd.7 v2:172.21.15.144:6824/3907866946 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1d1014ec10 con 0x7f1cf4073040 2023-10-18T08:55:27.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.806+0000 7f1d17b6c640 1 -- 172.21.15.40:0/3310635705 >> [v2:172.21.15.144:6824/3907866946,v1:172.21.15.144:6826/3907866946] conn(0x7f1cf4073040 msgr2=0x7f1cf4075460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.806+0000 7f1d17b6c640 1 --2- 172.21.15.40:0/3310635705 >> [v2:172.21.15.144:6824/3907866946,v1:172.21.15.144:6826/3907866946] conn(0x7f1cf4073040 0x7f1cf4075460 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:27.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.806+0000 7f1d17b6c640 1 -- 172.21.15.40:0/3310635705 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cf406aea0 msgr2=0x7f1cf406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.807+0000 7f1d17b6c640 1 --2- 172.21.15.40:0/3310635705 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cf406aea0 0x7f1cf406d360 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1d10069e00 tx=0x7f1d04009040 comp rx=0 tx=0).stop 2023-10-18T08:55:27.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.807+0000 7f1d17b6c640 1 -- 172.21.15.40:0/3310635705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d10151960 msgr2=0x7f1d101f9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:27.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.807+0000 7f1d17b6c640 1 --2- 172.21.15.40:0/3310635705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d10151960 0x7f1d101f9c40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1d08000f80 tx=0x7f1d08003f90 comp rx=0 tx=0).stop 2023-10-18T08:55:27.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.807+0000 7f1d16109640 1 -- 172.21.15.40:0/3310635705 reap_dead start 2023-10-18T08:55:27.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.807+0000 7f1d17b6c640 1 -- 172.21.15.40:0/3310635705 shutdown_connections 2023-10-18T08:55:27.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.807+0000 7f1d17b6c640 1 -- 172.21.15.40:0/3310635705 >> 172.21.15.40:0/3310635705 conn(0x7f1d1006cee0 msgr2=0x7f1d10150a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:27.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.808+0000 7f1d17b6c640 1 -- 172.21.15.40:0/3310635705 shutdown_connections 2023-10-18T08:55:27.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:27.808+0000 7f1d17b6c640 1 -- 172.21.15.40:0/3310635705 wait complete. 2023-10-18T08:55:27.873 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-18T08:55:27.874 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-18T08:55:27.875 INFO:teuthology.orchestra.run.smithi040.stdout:55834574856 2023-10-18T08:55:27.875 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-18T08:55:27.876 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-18T08:55:27.877 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-18T08:55:27.877 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-18T08:55:27.878 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-18T08:55:27.879 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2023-10-18T08:55:27.879 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-18T08:55:27.880 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-18T08:55:27.880 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-18T08:55:27.881 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-18T08:55:27.882 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-18T08:55:28.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.225+0000 7f298fc21640 1 -- 172.21.15.40:0/3003162472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2974004d00 msgr2=0x7f2974005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.225+0000 7f298fc21640 1 --2- 172.21.15.40:0/3003162472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2974004d00 0x7f2974005180 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2988068160 tx=0x7f29740057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.225+0000 7f298e1be640 1 -- 172.21.15.40:0/3003162472 reap_dead start 2023-10-18T08:55:28.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.225+0000 7f298fc21640 1 -- 172.21.15.40:0/3003162472 shutdown_connections 2023-10-18T08:55:28.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.225+0000 7f298fc21640 1 -- 172.21.15.40:0/3003162472 >> 172.21.15.40:0/3003162472 conn(0x7f29800559b0 msgr2=0x7f2980055dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.227+0000 7f298fc21640 1 -- 172.21.15.40:0/3003162472 shutdown_connections 2023-10-18T08:55:28.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.227+0000 7f298fc21640 1 -- 172.21.15.40:0/3003162472 wait complete. 2023-10-18T08:55:28.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.229+0000 7f298fc21640 1 Processor -- start 2023-10-18T08:55:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.230+0000 7f298fc21640 1 -- start start 2023-10-18T08:55:28.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.232+0000 7f298fc21640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2974004d00 0x7f2980052830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.232+0000 7f298fc21640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f298017e8f0 0x7f2980050fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.232+0000 7f298fc21640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2980039d90 0x7f298004eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.232+0000 7f298fc21640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29800523a0 con 0x7f298017e8f0 2023-10-18T08:55:28.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.232+0000 7f298fc21640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298017ed70 con 0x7f2974004d00 2023-10-18T08:55:28.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.232+0000 7f298fc21640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29800514f0 con 0x7f2980039d90 2023-10-18T08:55:28.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.233+0000 7f298d9bd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2974004d00 0x7f2980052830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.233+0000 7f298e1be640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2980039d90 0x7f298004eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.233+0000 7f298d9bd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2974004d00 0x7f2980052830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.40:34674/0 (socket says 172.21.15.40:34674) 2023-10-18T08:55:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.233+0000 7f298d9bd640 1 -- 172.21.15.40:0/2083821929 learned_addr learned my addr 172.21.15.40:0/2083821929 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.233+0000 7f298d1bc640 1 --2- 172.21.15.40:0/2083821929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f298017e8f0 0x7f2980050fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.233+0000 7f298e1be640 1 -- 172.21.15.40:0/2083821929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2974004d00 msgr2=0x7f2980052830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.234+0000 7f298e1be640 1 --2- 172.21.15.40:0/2083821929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2974004d00 0x7f2980052830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.234+0000 7f298e1be640 1 -- 172.21.15.40:0/2083821929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f298017e8f0 msgr2=0x7f2980050fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.234+0000 7f298e1be640 1 --2- 172.21.15.40:0/2083821929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f298017e8f0 0x7f2980050fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.234+0000 7f298e1be640 1 -- 172.21.15.40:0/2083821929 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2980051790 con 0x7f2980039d90 2023-10-18T08:55:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.234+0000 7f298d1bc640 1 --2- 172.21.15.40:0/2083821929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f298017e8f0 0x7f2980050fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:28.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.236+0000 7f298e1be640 1 --2- 172.21.15.40:0/2083821929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2980039d90 0x7f298004eb90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f29880633f0 tx=0x7f29880a5750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:28.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.236+0000 7f297e7fc640 1 -- 172.21.15.40:0/2083821929 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29880a5c60 con 0x7f2980039d90 2023-10-18T08:55:28.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.236+0000 7f297e7fc640 1 -- 172.21.15.40:0/2083821929 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2988061040 con 0x7f2980039d90 2023-10-18T08:55:28.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.236+0000 7f297e7fc640 1 -- 172.21.15.40:0/2083821929 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29880a72d0 con 0x7f2980039d90 2023-10-18T08:55:28.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.236+0000 7f298fc21640 1 -- 172.21.15.40:0/2083821929 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2980051a70 con 0x7f2980039d90 2023-10-18T08:55:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.237+0000 7fa841de2640 1 -- 172.21.15.40:0/3271405070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8340e12e0 msgr2=0x7fa83403a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.237+0000 7fa841de2640 1 --2- 172.21.15.40:0/3271405070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8340e12e0 0x7fa83403a140 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa828005fd0 tx=0x7fa82802bef0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.238+0000 7fa841de2640 1 -- 172.21.15.40:0/3271405070 shutdown_connections 2023-10-18T08:55:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.236+0000 7f298fc21640 1 -- 172.21.15.40:0/2083821929 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2980052030 con 0x7f2980039d90 2023-10-18T08:55:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.238+0000 7fa841de2640 1 --2- 172.21.15.40:0/3271405070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8340e12e0 0x7fa83403a140 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.238+0000 7f298fc21640 1 -- 172.21.15.40:0/2083821929 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29800521c0 con 0x7f2980039d90 2023-10-18T08:55:28.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.238+0000 7fa841de2640 1 -- 172.21.15.40:0/3271405070 >> 172.21.15.40:0/3271405070 conn(0x7fa834056ca0 msgr2=0x7fa8340570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:28.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.240+0000 7fa841de2640 1 -- 172.21.15.40:0/3271405070 shutdown_connections 2023-10-18T08:55:28.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.240+0000 7fa841de2640 1 -- 172.21.15.40:0/3271405070 wait complete. 2023-10-18T08:55:28.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.240+0000 7f297e7fc640 1 -- 172.21.15.40:0/2083821929 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f29880b4040 con 0x7f2980039d90 2023-10-18T08:55:28.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.242+0000 7f297e7fc640 1 --2- 172.21.15.40:0/2083821929 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f296c06ae70 0x7f296c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.242+0000 7f297e7fc640 1 -- 172.21.15.40:0/2083821929 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2988122bd0 con 0x7f2980039d90 2023-10-18T08:55:28.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.242+0000 7fa841de2640 1 Processor -- start 2023-10-18T08:55:28.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.243+0000 7fa841de2640 1 -- start start 2023-10-18T08:55:28.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.243+0000 7f298d9bd640 1 --2- 172.21.15.40:0/2083821929 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f296c06ae70 0x7f296c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.245+0000 7fa841de2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83400a430 0x7fa834048cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.245+0000 7fa841de2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa83400ab10 0x7fa8340d4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.245+0000 7fa841de2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8340e12e0 0x7fa834009500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.246+0000 7fa841de2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa834049eb0 con 0x7fa83400ab10 2023-10-18T08:55:28.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.246+0000 7fa840de0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83400a430 0x7fa834048cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.246+0000 7fa840de0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83400a430 0x7fa834048cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.40:34684/0 (socket says 172.21.15.40:34684) 2023-10-18T08:55:28.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.246+0000 7fa840de0640 1 -- 172.21.15.40:0/3465470387 learned_addr learned my addr 172.21.15.40:0/3465470387 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:28.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.246+0000 7fa83bfff640 1 --2- 172.21.15.40:0/3465470387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa83400ab10 0x7fa8340d4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.246+0000 7fa841de2640 1 -- 172.21.15.40:0/3465470387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340498f0 con 0x7fa83400a430 2023-10-18T08:55:28.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.246+0000 7fa841de2640 1 -- 172.21.15.40:0/3465470387 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa834049bd0 con 0x7fa8340e12e0 2023-10-18T08:55:28.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.246+0000 7fa8415e1640 1 --2- 172.21.15.40:0/3465470387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8340e12e0 0x7fa834009500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.247+0000 7f298d9bd640 1 --2- 172.21.15.40:0/2083821929 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f296c06ae70 0x7f296c06d330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2984006fd0 tx=0x7f2984006b70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:28.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.247+0000 7fa840de0640 1 -- 172.21.15.40:0/3465470387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8340e12e0 msgr2=0x7fa834009500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.248+0000 7fa840de0640 1 --2- 172.21.15.40:0/3465470387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8340e12e0 0x7fa834009500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.248+0000 7fa840de0640 1 -- 172.21.15.40:0/3465470387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa83400ab10 msgr2=0x7fa8340d4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.248+0000 7fa840de0640 1 --2- 172.21.15.40:0/3465470387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa83400ab10 0x7fa8340d4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.248+0000 7fa840de0640 1 -- 172.21.15.40:0/3465470387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa828009560 con 0x7fa83400a430 2023-10-18T08:55:28.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.248+0000 7f2275e00640 1 -- 172.21.15.40:0/2250062472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f227006cac0 msgr2=0x7f227006cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.248+0000 7f2275e00640 1 --2- 172.21.15.40:0/2250062472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f227006cac0 0x7f227006cf20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2268009830 tx=0x7f226802bd50 comp rx=0 tx=0).stop 2023-10-18T08:55:28.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.248+0000 7fa83bfff640 1 --2- 172.21.15.40:0/3465470387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa83400ab10 0x7fa8340d4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T08:55:28.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.249+0000 7f2275e00640 1 -- 172.21.15.40:0/2250062472 shutdown_connections 2023-10-18T08:55:28.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.249+0000 7f2275e00640 1 --2- 172.21.15.40:0/2250062472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f227006cac0 0x7f227006cf20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.249+0000 7f2275e00640 1 --2- 172.21.15.40:0/2250062472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f227006be20 0x7f227015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.249+0000 7f2275e00640 1 -- 172.21.15.40:0/2250062472 >> 172.21.15.40:0/2250062472 conn(0x7f22700aede0 msgr2=0x7f22700af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:28.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.249+0000 7f2275e00640 1 -- 172.21.15.40:0/2250062472 shutdown_connections 2023-10-18T08:55:28.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.249+0000 7fa840de0640 1 --2- 172.21.15.40:0/3465470387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83400a430 0x7fa834048cb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa828009850 tx=0x7fa828003870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:28.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.250+0000 7fa8397fa640 1 -- 172.21.15.40:0/3465470387 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828003e70 con 0x7fa83400a430 2023-10-18T08:55:28.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.250+0000 7fa841de2640 1 -- 172.21.15.40:0/3465470387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa834008d00 con 0x7fa83400a430 2023-10-18T08:55:28.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.249+0000 7f2275e00640 1 -- 172.21.15.40:0/2250062472 wait complete. 2023-10-18T08:55:28.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.250+0000 7fa8397fa640 1 -- 172.21.15.40:0/3465470387 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa82803d530 con 0x7fa83400a430 2023-10-18T08:55:28.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.250+0000 7fa8397fa640 1 -- 172.21.15.40:0/3465470387 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82803d900 con 0x7fa83400a430 2023-10-18T08:55:28.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.250+0000 7fa841de2640 1 -- 172.21.15.40:0/3465470387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8340491f0 con 0x7fa83400a430 2023-10-18T08:55:28.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.251+0000 7f2275e00640 1 Processor -- start 2023-10-18T08:55:28.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.251+0000 7f2275e00640 1 -- start start 2023-10-18T08:55:28.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.251+0000 7fa841de2640 1 -- 172.21.15.40:0/3465470387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa834008b60 con 0x7fa83400a430 2023-10-18T08:55:28.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.253+0000 7f2275e00640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227006be20 0x7f227013ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.253+0000 7f2275e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f227006cac0 0x7f2270158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.253+0000 7f2275e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f227013c200 0x7f227013ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.253+0000 7f2275e00640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2270158f70 con 0x7f227006be20 2023-10-18T08:55:28.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.253+0000 7f2275e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22701585a0 con 0x7f227006cac0 2023-10-18T08:55:28.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.253+0000 7f2275e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2270157f70 con 0x7f227013c200 2023-10-18T08:55:28.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.253+0000 7f226f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227006be20 0x7f227013ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.253+0000 7f226ffff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f227013c200 0x7f227013ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.253+0000 7f226f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227006be20 0x7f227013ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47994/0 (socket says 172.21.15.40:47994) 2023-10-18T08:55:28.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.253+0000 7f226f7fe640 1 -- 172.21.15.40:0/1772266028 learned_addr learned my addr 172.21.15.40:0/1772266028 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:28.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.253+0000 7f2266ffd640 1 --2- 172.21.15.40:0/1772266028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f227006cac0 0x7f2270158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.254+0000 7fa8397fa640 1 -- 172.21.15.40:0/3465470387 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa82803da60 con 0x7fa83400a430 2023-10-18T08:55:28.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.254+0000 7f226f7fe640 1 -- 172.21.15.40:0/1772266028 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f227013c200 msgr2=0x7f227013ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.254+0000 7f226f7fe640 1 --2- 172.21.15.40:0/1772266028 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f227013c200 0x7f227013ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.254+0000 7f226f7fe640 1 -- 172.21.15.40:0/1772266028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f227006cac0 msgr2=0x7f2270158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.254+0000 7f226f7fe640 1 --2- 172.21.15.40:0/1772266028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f227006cac0 0x7f2270158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.254+0000 7f226f7fe640 1 -- 172.21.15.40:0/1772266028 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2268009490 con 0x7f227006be20 2023-10-18T08:55:28.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.254+0000 7f226f7fe640 1 --2- 172.21.15.40:0/1772266028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227006be20 0x7f227013ba40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f225800c430 tx=0x7f225800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:28.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.255+0000 7f226cff9640 1 -- 172.21.15.40:0/1772266028 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2258011790 con 0x7f227006be20 2023-10-18T08:55:28.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.255+0000 7f226cff9640 1 -- 172.21.15.40:0/1772266028 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2258011a00 con 0x7f227006be20 2023-10-18T08:55:28.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.255+0000 7f2275e00640 1 -- 172.21.15.40:0/1772266028 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227014ff70 con 0x7f227006be20 2023-10-18T08:55:28.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.255+0000 7f2275e00640 1 -- 172.21.15.40:0/1772266028 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2270150530 con 0x7f227006be20 2023-10-18T08:55:28.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.255+0000 7f226cff9640 1 -- 172.21.15.40:0/1772266028 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2258011e10 con 0x7f227006be20 2023-10-18T08:55:28.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.255+0000 7fa8397fa640 1 --2- 172.21.15.40:0/3465470387 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa82006ae40 0x7fa82006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.255+0000 7fa8397fa640 1 -- 172.21.15.40:0/3465470387 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8280ac2d0 con 0x7fa83400a430 2023-10-18T08:55:28.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.256+0000 7fa83bfff640 1 --2- 172.21.15.40:0/3465470387 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa82006ae40 0x7fa82006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.257+0000 7f2275e00640 1 -- 172.21.15.40:0/1772266028 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2254005250 con 0x7f227006be20 2023-10-18T08:55:28.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.257+0000 7fa83bfff640 1 --2- 172.21.15.40:0/3465470387 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa82006ae40 0x7fa82006d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa83403b4d0 tx=0x7fa830009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:28.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.257+0000 7f226cff9640 1 -- 172.21.15.40:0/1772266028 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f225801c460 con 0x7f227006be20 2023-10-18T08:55:28.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.258+0000 7ff8b651d640 1 -- 172.21.15.40:0/3354776281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8b01586a0 msgr2=0x7ff8b00cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.258+0000 7ff8b651d640 1 --2- 172.21.15.40:0/3354776281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8b01586a0 0x7ff8b00cc990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400a940 tx=0x7ff8a402d140 comp rx=0 tx=0).stop 2023-10-18T08:55:28.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.258+0000 7ff8b651d640 1 -- 172.21.15.40:0/3354776281 shutdown_connections 2023-10-18T08:55:28.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.258+0000 7ff8b651d640 1 --2- 172.21.15.40:0/3354776281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8b01586a0 0x7ff8b00cc990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.258+0000 7ff8b651d640 1 -- 172.21.15.40:0/3354776281 >> 172.21.15.40:0/3354776281 conn(0x7ff8b006cee0 msgr2=0x7ff8b0069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:28.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.258+0000 7ff8b651d640 1 -- 172.21.15.40:0/3354776281 shutdown_connections 2023-10-18T08:55:28.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.258+0000 7ff8b651d640 1 -- 172.21.15.40:0/3354776281 wait complete. 2023-10-18T08:55:28.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.258+0000 7f226cff9640 1 --2- 172.21.15.40:0/1772266028 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f225006ae40 0x7f225006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.258+0000 7f226cff9640 1 -- 172.21.15.40:0/1772266028 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2258089520 con 0x7f227006be20 2023-10-18T08:55:28.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.259+0000 7f2266ffd640 1 --2- 172.21.15.40:0/1772266028 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f225006ae40 0x7f225006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.260+0000 7ff8b651d640 1 Processor -- start 2023-10-18T08:55:28.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.260+0000 7ff8b651d640 1 -- start start 2023-10-18T08:55:28.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.260+0000 7f2266ffd640 1 --2- 172.21.15.40:0/1772266028 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f225006ae40 0x7f225006d300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f22701e4d90 tx=0x7f22600075b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:28.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.261+0000 7ff8b651d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b006d710 0x7ff8b01593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.261+0000 7ff8b651d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b00cced0 0x7ff8b0159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.261+0000 7ff8b651d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8b01586a0 0x7ff8b01f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.261+0000 7ff8b651d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b014dd00 con 0x7ff8b01586a0 2023-10-18T08:55:28.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.261+0000 7ff8b651d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b014d0f0 con 0x7ff8b00cced0 2023-10-18T08:55:28.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.261+0000 7ff8b651d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b014d3d0 con 0x7ff8b006d710 2023-10-18T08:55:28.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.261+0000 7f297e7fc640 1 -- 172.21.15.40:0/2083821929 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29880edbe0 con 0x7f2980039d90 2023-10-18T08:55:28.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.262+0000 7f6da91d0640 1 -- 172.21.15.40:0/583089345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da40cc1a0 msgr2=0x7f6da40cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.262+0000 7f6da91d0640 1 --2- 172.21.15.40:0/583089345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da40cc1a0 0x7f6da40cc630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c005fd0 tx=0x7f6d9c02c170 comp rx=0 tx=0).stop 2023-10-18T08:55:28.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.262+0000 7ff8af7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b00cced0 0x7ff8b0159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.262+0000 7ff8af7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b00cced0 0x7ff8b0159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.40:34714/0 (socket says 172.21.15.40:34714) 2023-10-18T08:55:28.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.262+0000 7ff8af7fe640 1 -- 172.21.15.40:0/2450723602 learned_addr learned my addr 172.21.15.40:0/2450723602 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:28.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.262+0000 7f6da91d0640 1 -- 172.21.15.40:0/583089345 shutdown_connections 2023-10-18T08:55:28.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.262+0000 7f6da91d0640 1 --2- 172.21.15.40:0/583089345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da40cc1a0 0x7f6da40cc630 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.262+0000 7f6da91d0640 1 --2- 172.21.15.40:0/583089345 >> v2:172.21.15.40:3300/0 conn(0x7f6da40d1480 0x7f6da40d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.262+0000 7f6da91d0640 1 -- 172.21.15.40:0/583089345 >> 172.21.15.40:0/583089345 conn(0x7f6da409a9a0 msgr2=0x7f6da4088fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:28.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.262+0000 7ff8b4aba640 1 --2- 172.21.15.40:0/2450723602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8b01586a0 0x7ff8b01f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.262+0000 7f6da91d0640 1 -- 172.21.15.40:0/583089345 shutdown_connections 2023-10-18T08:55:28.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.262+0000 7ff8affff640 1 --2- 172.21.15.40:0/2450723602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b006d710 0x7ff8b01593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.262+0000 7fb3e8bf0640 1 -- 172.21.15.40:0/3163998702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e40b0bc0 msgr2=0x7fb3e40b1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.262+0000 7fb3e8bf0640 1 --2- 172.21.15.40:0/3163998702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e40b0bc0 0x7fb3e40b1040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc007380 tx=0x7fb3cc02ca50 comp rx=0 tx=0).stop 2023-10-18T08:55:28.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.262+0000 7f6da91d0640 1 -- 172.21.15.40:0/583089345 wait complete. 2023-10-18T08:55:28.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.263+0000 7fb3e8bf0640 1 -- 172.21.15.40:0/3163998702 shutdown_connections 2023-10-18T08:55:28.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.263+0000 7fb3e8bf0640 1 --2- 172.21.15.40:0/3163998702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e40b0bc0 0x7fb3e40b1040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.263+0000 7fb3e8bf0640 1 -- 172.21.15.40:0/3163998702 >> 172.21.15.40:0/3163998702 conn(0x7fb3e406cee0 msgr2=0x7fb3e4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:28.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.263+0000 7fb3e8bf0640 1 -- 172.21.15.40:0/3163998702 shutdown_connections 2023-10-18T08:55:28.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.263+0000 7ff8af7fe640 1 -- 172.21.15.40:0/2450723602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b006d710 msgr2=0x7ff8b01593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.263+0000 7ff8af7fe640 1 --2- 172.21.15.40:0/2450723602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b006d710 0x7ff8b01593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.263+0000 7ff8af7fe640 1 -- 172.21.15.40:0/2450723602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8b01586a0 msgr2=0x7ff8b01f0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.263+0000 7fb3e8bf0640 1 -- 172.21.15.40:0/3163998702 wait complete. 2023-10-18T08:55:28.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.264+0000 7ff8af7fe640 1 --2- 172.21.15.40:0/2450723602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8b01586a0 0x7ff8b01f0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.264+0000 7ff8af7fe640 1 -- 172.21.15.40:0/2450723602 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a400a5a0 con 0x7ff8b00cced0 2023-10-18T08:55:28.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.264+0000 7ff8b4aba640 1 --2- 172.21.15.40:0/2450723602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8b01586a0 0x7ff8b01f0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T08:55:28.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.264+0000 7f6da91d0640 1 Processor -- start 2023-10-18T08:55:28.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.264+0000 7f6da91d0640 1 -- start start 2023-10-18T08:55:28.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.264+0000 7ff8af7fe640 1 --2- 172.21.15.40:0/2450723602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b00cced0 0x7ff8b0159930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4000c00 tx=0x7ff8a4036db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:28.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.265+0000 7fb3e8bf0640 1 Processor -- start 2023-10-18T08:55:28.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.265+0000 7f226cff9640 1 -- 172.21.15.40:0/1772266028 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22580545e0 con 0x7f227006be20 2023-10-18T08:55:28.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.265+0000 7fb3e8bf0640 1 -- start start 2023-10-18T08:55:28.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.266+0000 7f6da91d0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da40cc1a0 0x7f6da40c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.266+0000 7f6da91d0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da40d1480 0x7f6da414c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.266+0000 7f6da91d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6da415b760 0x7f6da406a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.266+0000 7f6da91d0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da40cb070 con 0x7f6da40d1480 2023-10-18T08:55:28.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.266+0000 7f6da91d0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da40ca460 con 0x7f6da415b760 2023-10-18T08:55:28.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.266+0000 7f6da91d0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da40ca740 con 0x7f6da40cc1a0 2023-10-18T08:55:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.266+0000 7f6da2d9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da40cc1a0 0x7f6da40c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.266+0000 7f6da2d9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da40cc1a0 0x7f6da40c9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:45376/0 (socket says 172.21.15.40:45376) 2023-10-18T08:55:28.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.266+0000 7f6da2d9d640 1 -- 172.21.15.40:0/1758233718 learned_addr learned my addr 172.21.15.40:0/1758233718 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7fb3e8bf0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e40cc990 0x7fb3e413dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7fb3e8bf0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e4151bd0 0x7fb3e41e2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7fb3e8bf0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e4159580 0x7fb3e4159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7fb3e8bf0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e4159f50 con 0x7fb3e4151bd0 2023-10-18T08:55:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7fb3e8bf0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e413e500 con 0x7fb3e40cc990 2023-10-18T08:55:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7fb3e8bf0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e41e2d90 con 0x7fb3e4159580 2023-10-18T08:55:28.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7f6da259c640 1 --2- 172.21.15.40:0/1758233718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da40d1480 0x7f6da414c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7f4579df5640 1 -- 172.21.15.40:0/1036462951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f457406e460 msgr2=0x7f45740b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7f4579df5640 1 --2- 172.21.15.40:0/1036462951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f457406e460 0x7f45740b2a50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4568009830 tx=0x7f456802bdf0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7f6da359e640 1 --2- 172.21.15.40:0/1758233718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6da415b760 0x7f6da406a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7fb3e1d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e4151bd0 0x7fb3e41e2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7fb3e259c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e40cc990 0x7fb3e413dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7fb3e1d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e4151bd0 0x7fb3e41e2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48032/0 (socket says 172.21.15.40:48032) 2023-10-18T08:55:28.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7fb3e1d9b640 1 -- 172.21.15.40:0/2299745425 learned_addr learned my addr 172.21.15.40:0/2299745425 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7f4579df5640 1 -- 172.21.15.40:0/1036462951 shutdown_connections 2023-10-18T08:55:28.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7f4579df5640 1 --2- 172.21.15.40:0/1036462951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f457406e460 0x7f45740b2a50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7f4579df5640 1 -- 172.21.15.40:0/1036462951 >> 172.21.15.40:0/1036462951 conn(0x7f45740655f0 msgr2=0x7f45740659d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7f4579df5640 1 -- 172.21.15.40:0/1036462951 shutdown_connections 2023-10-18T08:55:28.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.268+0000 7f6da2d9d640 1 -- 172.21.15.40:0/1758233718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6da415b760 msgr2=0x7f6da406a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.268+0000 7f6da2d9d640 1 --2- 172.21.15.40:0/1758233718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6da415b760 0x7f6da406a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.268+0000 7f6da2d9d640 1 -- 172.21.15.40:0/1758233718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da40d1480 msgr2=0x7f6da414c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.268+0000 7f6da2d9d640 1 --2- 172.21.15.40:0/1758233718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da40d1480 0x7f6da414c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.268+0000 7f6da2d9d640 1 -- 172.21.15.40:0/1758233718 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c009840 con 0x7f6da40cc1a0 2023-10-18T08:55:28.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.267+0000 7fb3e2d9d640 1 --2- 172.21.15.40:0/2299745425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e4159580 0x7fb3e4159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.268+0000 7fb3e1d9b640 1 -- 172.21.15.40:0/2299745425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e4159580 msgr2=0x7fb3e4159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.268+0000 7fb3e1d9b640 1 --2- 172.21.15.40:0/2299745425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e4159580 0x7fb3e4159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.268+0000 7fb3e1d9b640 1 -- 172.21.15.40:0/2299745425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e40cc990 msgr2=0x7fb3e413dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.268+0000 7fb3e1d9b640 1 --2- 172.21.15.40:0/2299745425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e40cc990 0x7fb3e413dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.268+0000 7fb3e1d9b640 1 -- 172.21.15.40:0/2299745425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3cc00ae30 con 0x7fb3e4151bd0 2023-10-18T08:55:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.268+0000 7f4579df5640 1 -- 172.21.15.40:0/1036462951 wait complete. 2023-10-18T08:55:28.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.268+0000 7f6da2d9d640 1 --2- 172.21.15.40:0/1758233718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da40cc1a0 0x7f6da40c9ba0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400c410 tx=0x7f6d9400c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.268+0000 7fb3e1d9b640 1 --2- 172.21.15.40:0/2299745425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e4151bd0 0x7fb3e41e2850 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0074a0 tx=0x7fb3cc00a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:28.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.269+0000 7fb3daffd640 1 -- 172.21.15.40:0/2299745425 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc00a0d0 con 0x7fb3e4151bd0 2023-10-18T08:55:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.269+0000 7fb3daffd640 1 -- 172.21.15.40:0/2299745425 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb3cc007d40 con 0x7fb3e4151bd0 2023-10-18T08:55:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.269+0000 7fb3daffd640 1 -- 172.21.15.40:0/2299745425 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc003850 con 0x7fb3e4151bd0 2023-10-18T08:55:28.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.269+0000 7fb3e8bf0640 1 -- 172.21.15.40:0/2299745425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e413e930 con 0x7fb3e4151bd0 2023-10-18T08:55:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.269+0000 7f6d8b7fe640 1 -- 172.21.15.40:0/1758233718 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d940035f0 con 0x7f6da40cc1a0 2023-10-18T08:55:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.269+0000 7f6d8b7fe640 1 -- 172.21.15.40:0/1758233718 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6d94003750 con 0x7f6da40cc1a0 2023-10-18T08:55:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.269+0000 7f6d8b7fe640 1 -- 172.21.15.40:0/1758233718 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94003ba0 con 0x7f6da40cc1a0 2023-10-18T08:55:28.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.269+0000 7f6da91d0640 1 -- 172.21.15.40:0/1758233718 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da40caa20 con 0x7f6da40cc1a0 2023-10-18T08:55:28.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.269+0000 7fb3e8bf0640 1 -- 172.21.15.40:0/2299745425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e413eef0 con 0x7fb3e4151bd0 2023-10-18T08:55:28.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.269+0000 7f6da91d0640 1 -- 172.21.15.40:0/1758233718 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da40ca0e0 con 0x7f6da40cc1a0 2023-10-18T08:55:28.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.270+0000 7fb3e8bf0640 1 -- 172.21.15.40:0/2299745425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e4151780 con 0x7fb3e4151bd0 2023-10-18T08:55:28.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.271+0000 7f4579df5640 1 Processor -- start 2023-10-18T08:55:28.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.271+0000 7f4579df5640 1 -- start start 2023-10-18T08:55:28.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.271+0000 7f6da91d0640 1 -- 172.21.15.40:0/1758233718 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d84005250 con 0x7f6da40cc1a0 2023-10-18T08:55:28.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.272+0000 7f4579df5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f457406e460 0x7f45740b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.272+0000 7f4579df5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45740ae700 0x7f457413dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.272+0000 7f4579df5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4574142200 0x7f45741de5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.272+0000 7f4579df5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45740abb50 con 0x7f4574142200 2023-10-18T08:55:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.272+0000 7f4579df5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45740abe30 con 0x7f45740ae700 2023-10-18T08:55:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.272+0000 7f4579df5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45740ac110 con 0x7f457406e460 2023-10-18T08:55:28.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.272+0000 7fa8397fa640 1 -- 172.21.15.40:0/3465470387 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa828077410 con 0x7fa83400a430 2023-10-18T08:55:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.272+0000 7fb3daffd640 1 -- 172.21.15.40:0/2299745425 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb3cc0039b0 con 0x7fb3e4151bd0 2023-10-18T08:55:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.272+0000 7ff8acff9640 1 -- 172.21.15.40:0/2450723602 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4003ac0 con 0x7ff8b00cced0 2023-10-18T08:55:28.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.272+0000 7ff8acff9640 1 -- 172.21.15.40:0/2450723602 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff8a4003d10 con 0x7ff8b00cced0 2023-10-18T08:55:28.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.272+0000 7ff8acff9640 1 -- 172.21.15.40:0/2450723602 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a403e690 con 0x7ff8b00cced0 2023-10-18T08:55:28.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.273+0000 7ff8b651d640 1 -- 172.21.15.40:0/2450723602 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b015c1d0 con 0x7ff8b00cced0 2023-10-18T08:55:28.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.273+0000 7ff8b651d640 1 -- 172.21.15.40:0/2450723602 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b014d990 con 0x7ff8b00cced0 2023-10-18T08:55:28.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.274+0000 7f45737fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f457406e460 0x7f45740b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.274+0000 7fb3daffd640 1 --2- 172.21.15.40:0/2299745425 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3c406af10 0x7fb3c406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.274+0000 7fb3daffd640 1 -- 172.21.15.40:0/2299745425 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3cc0ab6b0 con 0x7fb3e4151bd0 2023-10-18T08:55:28.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.274+0000 7f45737fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f457406e460 0x7f45740b1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:45388/0 (socket says 172.21.15.40:45388) 2023-10-18T08:55:28.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.274+0000 7f45737fe640 1 -- 172.21.15.40:0/1184481720 learned_addr learned my addr 172.21.15.40:0/1184481720 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:28.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.274+0000 7f6d8b7fe640 1 -- 172.21.15.40:0/1758233718 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6d94003d00 con 0x7f6da40cc1a0 2023-10-18T08:55:28.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.274+0000 7fcd3e6fc640 1 -- 172.21.15.40:0/21034416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380ce430 msgr2=0x7fcd380b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.274+0000 7fcd3e6fc640 1 --2- 172.21.15.40:0/21034416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380ce430 0x7fcd380b4e00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcd380cc150 tx=0x7fcd3002bd70 comp rx=0 tx=0).stop 2023-10-18T08:55:28.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.275+0000 7ff8b651d640 1 -- 172.21.15.40:0/2450723602 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b0066fd0 con 0x7ff8b00cced0 2023-10-18T08:55:28.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.275+0000 7fb3e259c640 1 --2- 172.21.15.40:0/2299745425 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3c406af10 0x7fb3c406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.275+0000 7fcd3e6fc640 1 -- 172.21.15.40:0/21034416 shutdown_connections 2023-10-18T08:55:28.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.275+0000 7f45737fe640 1 -- 172.21.15.40:0/1184481720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45740ae700 msgr2=0x7f457413dc10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T08:55:28.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.275+0000 7f45737fe640 1 --2- 172.21.15.40:0/1184481720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45740ae700 0x7f457413dc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.275+0000 7fcd3e6fc640 1 --2- 172.21.15.40:0/21034416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380ce430 0x7fcd380b4e00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.275+0000 7fcd3e6fc640 1 --2- 172.21.15.40:0/21034416 >> v2:172.21.15.40:3300/0 conn(0x7fcd380b7000 0x7fcd380cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.275+0000 7fcd3e6fc640 1 -- 172.21.15.40:0/21034416 >> 172.21.15.40:0/21034416 conn(0x7fcd3806a780 msgr2=0x7fcd3806ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:28.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.275+0000 7f45737fe640 1 -- 172.21.15.40:0/1184481720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4574142200 msgr2=0x7f45741de5e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T08:55:28.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.275+0000 7f45737fe640 1 --2- 172.21.15.40:0/1184481720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4574142200 0x7f45741de5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.275+0000 7f45737fe640 1 -- 172.21.15.40:0/1184481720 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4568009490 con 0x7f457406e460 2023-10-18T08:55:28.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.276+0000 7f6d8b7fe640 1 --2- 172.21.15.40:0/1758233718 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6d9006ae40 0x7f6d9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.276+0000 7f6d8b7fe640 1 -- 172.21.15.40:0/1758233718 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d94088b80 con 0x7f6da40cc1a0 2023-10-18T08:55:28.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.276+0000 7f45737fe640 1 --2- 172.21.15.40:0/1184481720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f457406e460 0x7f45740b1e70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f456400c400 tx=0x7f456400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:28.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.276+0000 7fb3e259c640 1 --2- 172.21.15.40:0/2299745425 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3c406af10 0x7fb3c406d3d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4000f60 tx=0x7fb3d4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:28.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.277+0000 7f6da259c640 1 --2- 172.21.15.40:0/1758233718 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6d9006ae40 0x7f6d9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.277+0000 7fcd3e6fc640 1 -- 172.21.15.40:0/21034416 shutdown_connections 2023-10-18T08:55:28.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.277+0000 7ff8acff9640 1 -- 172.21.15.40:0/2450723602 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff8a403c040 con 0x7ff8b00cced0 2023-10-18T08:55:28.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.278+0000 7f6da259c640 1 --2- 172.21.15.40:0/1758233718 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6d9006ae40 0x7f6d9006d300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6da406b6e0 tx=0x7f6d9800a2d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:28.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.278+0000 7ff8acff9640 1 --2- 172.21.15.40:0/2450723602 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff89c06ae90 0x7ff89c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.278+0000 7ff8acff9640 1 -- 172.21.15.40:0/2450723602 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8a40ac590 con 0x7ff8b00cced0 2023-10-18T08:55:28.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.277+0000 7fcd3e6fc640 1 -- 172.21.15.40:0/21034416 wait complete. 2023-10-18T08:55:28.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.278+0000 7fcd3e6fc640 1 Processor -- start 2023-10-18T08:55:28.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.279+0000 7ff8affff640 1 --2- 172.21.15.40:0/2450723602 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff89c06ae90 0x7ff89c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.279+0000 7fcd3e6fc640 1 -- start start 2023-10-18T08:55:28.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.280+0000 7fcd3e6fc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd380b7000 0x7fcd380caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.280+0000 7fcd3e6fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd380ce430 0x7fcd381f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.280+0000 7fcd3e6fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380b20c0 0x7fcd380b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.280+0000 7fcd3e6fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380c9480 con 0x7fcd380ce430 2023-10-18T08:55:28.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.280+0000 7fcd3e6fc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380b2a80 con 0x7fcd380b20c0 2023-10-18T08:55:28.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.280+0000 7fcd3e6fc640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380c85b0 con 0x7fcd380b7000 2023-10-18T08:55:28.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.280+0000 7fcd2f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd380ce430 0x7fcd381f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.280+0000 7fcd2f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd380ce430 0x7fcd381f5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48044/0 (socket says 172.21.15.40:48044) 2023-10-18T08:55:28.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.280+0000 7fcd2f7fe640 1 -- 172.21.15.40:0/311884279 learned_addr learned my addr 172.21.15.40:0/311884279 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:28.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.281+0000 7fcd37fff640 1 --2- 172.21.15.40:0/311884279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd380b7000 0x7fcd380caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.281+0000 7ff8affff640 1 --2- 172.21.15.40:0/2450723602 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff89c06ae90 0x7ff89c06d350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00046b0 tx=0x7ff8a0007eb0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:28.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.281+0000 7fcd3cc99640 1 --2- 172.21.15.40:0/311884279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380b20c0 0x7fcd380b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.281+0000 7fcd2f7fe640 1 -- 172.21.15.40:0/311884279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd380b7000 msgr2=0x7fcd380caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.281+0000 7fcd2f7fe640 1 --2- 172.21.15.40:0/311884279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd380b7000 0x7fcd380caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.281+0000 7fcd2f7fe640 1 -- 172.21.15.40:0/311884279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380b20c0 msgr2=0x7fcd380b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.281+0000 7fcd2f7fe640 1 --2- 172.21.15.40:0/311884279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380b20c0 0x7fcd380b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.281+0000 7fcd2f7fe640 1 -- 172.21.15.40:0/311884279 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd30009490 con 0x7fcd380ce430 2023-10-18T08:55:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.281+0000 7fcd2f7fe640 1 --2- 172.21.15.40:0/311884279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd380ce430 0x7fcd381f5690 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000e9f0 tx=0x7fcd2000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.282+0000 7fcd357fa640 1 -- 172.21.15.40:0/311884279 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20009700 con 0x7fcd380ce430 2023-10-18T08:55:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.282+0000 7fcd357fa640 1 -- 172.21.15.40:0/311884279 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcd2000ce70 con 0x7fcd380ce430 2023-10-18T08:55:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.282+0000 7fcd3e6fc640 1 -- 172.21.15.40:0/311884279 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd380c8800 con 0x7fcd380ce430 2023-10-18T08:55:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.282+0000 7fcd357fa640 1 -- 172.21.15.40:0/311884279 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20010670 con 0x7fcd380ce430 2023-10-18T08:55:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.282+0000 7fb3daffd640 1 -- 172.21.15.40:0/2299745425 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3cc0766b0 con 0x7fb3e4151bd0 2023-10-18T08:55:28.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.283+0000 7fcd3e6fc640 1 -- 172.21.15.40:0/311884279 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd380c8dc0 con 0x7fcd380ce430 2023-10-18T08:55:28.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.284+0000 7fcd3e6fc640 1 -- 172.21.15.40:0/311884279 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd380c95c0 con 0x7fcd380ce430 2023-10-18T08:55:28.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.285+0000 7fcd357fa640 1 -- 172.21.15.40:0/311884279 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcd200107d0 con 0x7fcd380ce430 2023-10-18T08:55:28.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.286+0000 7fcd357fa640 1 --2- 172.21.15.40:0/311884279 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcd1806ae40 0x7fcd1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.286+0000 7fcd357fa640 1 -- 172.21.15.40:0/311884279 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd20014070 con 0x7fcd380ce430 2023-10-18T08:55:28.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.286+0000 7fcd37fff640 1 --2- 172.21.15.40:0/311884279 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcd1806ae40 0x7fcd1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.289+0000 7f455bfff640 1 -- 172.21.15.40:0/1184481720 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45640052d0 con 0x7f457406e460 2023-10-18T08:55:28.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.289+0000 7f455bfff640 1 -- 172.21.15.40:0/1184481720 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f45640035f0 con 0x7f457406e460 2023-10-18T08:55:28.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.289+0000 7f455bfff640 1 -- 172.21.15.40:0/1184481720 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4564003a00 con 0x7f457406e460 2023-10-18T08:55:28.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.289+0000 7f4579df5640 1 -- 172.21.15.40:0/1184481720 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45740ac3f0 con 0x7f457406e460 2023-10-18T08:55:28.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.290+0000 7fcd37fff640 1 --2- 172.21.15.40:0/311884279 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcd1806ae40 0x7fcd1806d300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcd28001560 tx=0x7fcd28001a30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:28.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.290+0000 7f4579df5640 1 -- 172.21.15.40:0/1184481720 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45740ad800 con 0x7f457406e460 2023-10-18T08:55:28.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.290+0000 7f6d8b7fe640 1 -- 172.21.15.40:0/1758233718 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d94053c40 con 0x7f6da40cc1a0 2023-10-18T08:55:28.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.291+0000 7f4579df5640 1 -- 172.21.15.40:0/1184481720 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45740ac910 con 0x7f457406e460 2023-10-18T08:55:28.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.293+0000 7ff8acff9640 1 -- 172.21.15.40:0/2450723602 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff8a4077860 con 0x7ff8b00cced0 2023-10-18T08:55:28.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.293+0000 7f455bfff640 1 -- 172.21.15.40:0/1184481720 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4564003b60 con 0x7f457406e460 2023-10-18T08:55:28.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.295+0000 7f455bfff640 1 --2- 172.21.15.40:0/1184481720 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f456006af10 0x7f456006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:28.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.295+0000 7f455bfff640 1 -- 172.21.15.40:0/1184481720 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4564088eb0 con 0x7f457406e460 2023-10-18T08:55:28.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.296+0000 7fcd357fa640 1 -- 172.21.15.40:0/311884279 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd20051ca0 con 0x7fcd380ce430 2023-10-18T08:55:28.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.296+0000 7f4572ffd640 1 --2- 172.21.15.40:0/1184481720 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f456006af10 0x7f456006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:28.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.297+0000 7f4572ffd640 1 --2- 172.21.15.40:0/1184481720 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f456006af10 0x7f456006d3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4568036000 tx=0x7f4568004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:28.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.308+0000 7f455bfff640 1 -- 172.21.15.40:0/1184481720 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f45740ac910 con 0x7f457406e460 2023-10-18T08:55:28.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.661+0000 7f298fc21640 1 -- 172.21.15.40:0/2083821929 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2980052510 con 0x7f2980039d90 2023-10-18T08:55:28.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.662+0000 7f297e7fc640 1 -- 172.21.15.40:0/2083821929 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f29880f1200 con 0x7f2980039d90 2023-10-18T08:55:28.665 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-18T08:55:28.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.665+0000 7f298fc21640 1 -- 172.21.15.40:0/2083821929 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f296c06ae70 msgr2=0x7f296c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.665+0000 7f298fc21640 1 --2- 172.21.15.40:0/2083821929 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f296c06ae70 0x7f296c06d330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2984006fd0 tx=0x7f2984006b70 comp rx=0 tx=0).stop 2023-10-18T08:55:28.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.666+0000 7f298fc21640 1 -- 172.21.15.40:0/2083821929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2980039d90 msgr2=0x7f298004eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.666+0000 7f298fc21640 1 --2- 172.21.15.40:0/2083821929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2980039d90 0x7f298004eb90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f29880633f0 tx=0x7f29880a5750 comp rx=0 tx=0).stop 2023-10-18T08:55:28.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.666+0000 7f298fc21640 1 -- 172.21.15.40:0/2083821929 shutdown_connections 2023-10-18T08:55:28.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.666+0000 7f298fc21640 1 --2- 172.21.15.40:0/2083821929 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f296c06ae70 0x7f296c06d330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.666+0000 7f298fc21640 1 --2- 172.21.15.40:0/2083821929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2980039d90 0x7f298004eb90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.666+0000 7f298fc21640 1 --2- 172.21.15.40:0/2083821929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f298017e8f0 0x7f2980050fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.666+0000 7f298fc21640 1 --2- 172.21.15.40:0/2083821929 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2974004d00 0x7f2980052830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.666+0000 7f298fc21640 1 -- 172.21.15.40:0/2083821929 >> 172.21.15.40:0/2083821929 conn(0x7f29800559b0 msgr2=0x7f298003bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:28.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.666+0000 7f298fc21640 1 -- 172.21.15.40:0/2083821929 shutdown_connections 2023-10-18T08:55:28.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.667+0000 7f298fc21640 1 -- 172.21.15.40:0/2083821929 wait complete. 2023-10-18T08:55:28.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-10-18T08:55:28.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.697+0000 7f2275e00640 1 -- 172.21.15.40:0/1772266028 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f22540050c0 con 0x7f227006be20 2023-10-18T08:55:28.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.698+0000 7f226cff9640 1 -- 172.21.15.40:0/1772266028 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2258057c00 con 0x7f227006be20 2023-10-18T08:55:28.702 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2023-10-18T08:55:28.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.702+0000 7f2275e00640 1 -- 172.21.15.40:0/1772266028 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f225006ae40 msgr2=0x7f225006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.702+0000 7f2275e00640 1 --2- 172.21.15.40:0/1772266028 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f225006ae40 0x7f225006d300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f22701e4d90 tx=0x7f22600075b0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.702+0000 7f2275e00640 1 -- 172.21.15.40:0/1772266028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227006be20 msgr2=0x7f227013ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.702+0000 7f2275e00640 1 --2- 172.21.15.40:0/1772266028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227006be20 0x7f227013ba40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f225800c430 tx=0x7f225800c8f0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.702+0000 7f2275e00640 1 -- 172.21.15.40:0/1772266028 shutdown_connections 2023-10-18T08:55:28.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.702+0000 7f2275e00640 1 --2- 172.21.15.40:0/1772266028 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f225006ae40 0x7f225006d300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.702+0000 7f2275e00640 1 --2- 172.21.15.40:0/1772266028 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f227013c200 0x7f227013ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.702+0000 7f2275e00640 1 --2- 172.21.15.40:0/1772266028 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f227006cac0 0x7f2270158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.702+0000 7f2275e00640 1 --2- 172.21.15.40:0/1772266028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f227006be20 0x7f227013ba40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.702+0000 7f2275e00640 1 -- 172.21.15.40:0/1772266028 >> 172.21.15.40:0/1772266028 conn(0x7f22700aede0 msgr2=0x7f227006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.702+0000 7f2275e00640 1 -- 172.21.15.40:0/1772266028 shutdown_connections 2023-10-18T08:55:28.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.703+0000 7f2275e00640 1 -- 172.21.15.40:0/1772266028 wait complete. 2023-10-18T08:55:28.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.706+0000 7fa841de2640 1 -- 172.21.15.40:0/3465470387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa834008b60 con 0x7fa83400a430 2023-10-18T08:55:28.710 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2023-10-18T08:55:28.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.707+0000 7fa8397fa640 1 -- 172.21.15.40:0/3465470387 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa834008b60 con 0x7fa83400a430 2023-10-18T08:55:28.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.710+0000 7fa841de2640 1 -- 172.21.15.40:0/3465470387 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa82006ae40 msgr2=0x7fa82006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.710+0000 7fa841de2640 1 --2- 172.21.15.40:0/3465470387 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa82006ae40 0x7fa82006d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa83403b4d0 tx=0x7fa830009040 comp rx=0 tx=0).stop 2023-10-18T08:55:28.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.710+0000 7fa841de2640 1 -- 172.21.15.40:0/3465470387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83400a430 msgr2=0x7fa834048cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.710+0000 7fa841de2640 1 --2- 172.21.15.40:0/3465470387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83400a430 0x7fa834048cb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa828009850 tx=0x7fa828003870 comp rx=0 tx=0).stop 2023-10-18T08:55:28.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.711+0000 7fa841de2640 1 -- 172.21.15.40:0/3465470387 shutdown_connections 2023-10-18T08:55:28.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.711+0000 7fa841de2640 1 --2- 172.21.15.40:0/3465470387 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa82006ae40 0x7fa82006d300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.711+0000 7fa841de2640 1 --2- 172.21.15.40:0/3465470387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8340e12e0 0x7fa834009500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.711+0000 7fa841de2640 1 --2- 172.21.15.40:0/3465470387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa83400ab10 0x7fa8340d4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.711+0000 7fa841de2640 1 --2- 172.21.15.40:0/3465470387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83400a430 0x7fa834048cb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.711+0000 7fa841de2640 1 -- 172.21.15.40:0/3465470387 >> 172.21.15.40:0/3465470387 conn(0x7fa834056ca0 msgr2=0x7fa8340e0510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:28.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.712+0000 7fa841de2640 1 -- 172.21.15.40:0/3465470387 shutdown_connections 2023-10-18T08:55:28.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.712+0000 7fa841de2640 1 -- 172.21.15.40:0/3465470387 wait complete. 2023-10-18T08:55:28.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.713+0000 7f4579df5640 1 -- 172.21.15.40:0/1184481720 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f45740ac910 con 0x7f457406e460 2023-10-18T08:55:28.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.713+0000 7f455bfff640 1 -- 172.21.15.40:0/1184481720 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f45740ac910 con 0x7f457406e460 2023-10-18T08:55:28.717 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2023-10-18T08:55:28.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.716+0000 7f4579df5640 1 -- 172.21.15.40:0/1184481720 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f456006af10 msgr2=0x7f456006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.716+0000 7f4579df5640 1 --2- 172.21.15.40:0/1184481720 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f456006af10 0x7f456006d3d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4568036000 tx=0x7f4568004730 comp rx=0 tx=0).stop 2023-10-18T08:55:28.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.716+0000 7f4579df5640 1 -- 172.21.15.40:0/1184481720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f457406e460 msgr2=0x7f45740b1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.716+0000 7f4579df5640 1 --2- 172.21.15.40:0/1184481720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f457406e460 0x7f45740b1e70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f456400c400 tx=0x7f456400c8c0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.716+0000 7f4579df5640 1 -- 172.21.15.40:0/1184481720 shutdown_connections 2023-10-18T08:55:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.716+0000 7f4579df5640 1 --2- 172.21.15.40:0/1184481720 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f456006af10 0x7f456006d3d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.716+0000 7f4579df5640 1 --2- 172.21.15.40:0/1184481720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4574142200 0x7f45741de5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.716+0000 7f4579df5640 1 --2- 172.21.15.40:0/1184481720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45740ae700 0x7f457413dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.716+0000 7f4579df5640 1 --2- 172.21.15.40:0/1184481720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f457406e460 0x7f45740b1e70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.716+0000 7f4579df5640 1 -- 172.21.15.40:0/1184481720 >> 172.21.15.40:0/1184481720 conn(0x7f45740655f0 msgr2=0x7f4574158f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.717+0000 7f4579df5640 1 -- 172.21.15.40:0/1184481720 shutdown_connections 2023-10-18T08:55:28.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.717+0000 7f4579df5640 1 -- 172.21.15.40:0/1184481720 wait complete. 2023-10-18T08:55:28.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.723+0000 7fb3e8bf0640 1 -- 172.21.15.40:0/2299745425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb3e4066fd0 con 0x7fb3e4151bd0 2023-10-18T08:55:28.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.724+0000 7fb3daffd640 1 -- 172.21.15.40:0/2299745425 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb3cc079cd0 con 0x7fb3e4151bd0 2023-10-18T08:55:28.727 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-18T08:55:28.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.726+0000 7ff8b651d640 1 -- 172.21.15.40:0/2450723602 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff8b014ce10 con 0x7ff8b00cced0 2023-10-18T08:55:28.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.726+0000 7fb3e8bf0640 1 -- 172.21.15.40:0/2299745425 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3c406af10 msgr2=0x7fb3c406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.726+0000 7fb3e8bf0640 1 --2- 172.21.15.40:0/2299745425 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3c406af10 0x7fb3c406d3d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4000f60 tx=0x7fb3d4009040 comp rx=0 tx=0).stop 2023-10-18T08:55:28.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.726+0000 7fb3e8bf0640 1 -- 172.21.15.40:0/2299745425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e4151bd0 msgr2=0x7fb3e41e2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.726+0000 7fb3e8bf0640 1 --2- 172.21.15.40:0/2299745425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e4151bd0 0x7fb3e41e2850 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0074a0 tx=0x7fb3cc00a6a0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2023-10-18T08:55:28.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.727+0000 7ff8acff9640 1 -- 172.21.15.40:0/2450723602 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff8a4042020 con 0x7ff8b00cced0 2023-10-18T08:55:28.732 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2023-10-18T08:55:28.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7fb3e8bf0640 1 -- 172.21.15.40:0/2299745425 shutdown_connections 2023-10-18T08:55:28.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7fb3e8bf0640 1 --2- 172.21.15.40:0/2299745425 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3c406af10 0x7fb3c406d3d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7fb3e8bf0640 1 --2- 172.21.15.40:0/2299745425 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3e4159580 0x7fb3e4159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7fb3e8bf0640 1 --2- 172.21.15.40:0/2299745425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3e4151bd0 0x7fb3e41e2850 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7fb3e8bf0640 1 --2- 172.21.15.40:0/2299745425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3e40cc990 0x7fb3e413dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7fb3e8bf0640 1 -- 172.21.15.40:0/2299745425 >> 172.21.15.40:0/2299745425 conn(0x7fb3e406cee0 msgr2=0x7fb3e4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:28.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7ff8b651d640 1 -- 172.21.15.40:0/2450723602 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff89c06ae90 msgr2=0x7ff89c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7ff8b651d640 1 --2- 172.21.15.40:0/2450723602 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff89c06ae90 0x7ff89c06d350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00046b0 tx=0x7ff8a0007eb0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7ff8b651d640 1 -- 172.21.15.40:0/2450723602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b00cced0 msgr2=0x7ff8b0159930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7ff8b651d640 1 --2- 172.21.15.40:0/2450723602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b00cced0 0x7ff8b0159930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4000c00 tx=0x7ff8a4036db0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7ff8b651d640 1 -- 172.21.15.40:0/2450723602 shutdown_connections 2023-10-18T08:55:28.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7ff8b651d640 1 --2- 172.21.15.40:0/2450723602 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff89c06ae90 0x7ff89c06d350 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7ff8b651d640 1 --2- 172.21.15.40:0/2450723602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8b01586a0 0x7ff8b01f0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7ff8b651d640 1 --2- 172.21.15.40:0/2450723602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8b00cced0 0x7ff8b0159930 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7ff8b651d640 1 --2- 172.21.15.40:0/2450723602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8b006d710 0x7ff8b01593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7ff8b651d640 1 -- 172.21.15.40:0/2450723602 >> 172.21.15.40:0/2450723602 conn(0x7ff8b006cee0 msgr2=0x7ff8b00cd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:28.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7fcd3e6fc640 1 -- 172.21.15.40:0/311884279 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcd380d0980 con 0x7fcd380ce430 2023-10-18T08:55:28.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7ff8b651d640 1 -- 172.21.15.40:0/2450723602 shutdown_connections 2023-10-18T08:55:28.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7fb3e8bf0640 1 -- 172.21.15.40:0/2299745425 shutdown_connections 2023-10-18T08:55:28.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.729+0000 7fb3e8bf0640 1 -- 172.21.15.40:0/2299745425 wait complete. 2023-10-18T08:55:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.730+0000 7ff8b651d640 1 -- 172.21.15.40:0/2450723602 wait complete. 2023-10-18T08:55:28.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.731+0000 7fcd357fa640 1 -- 172.21.15.40:0/311884279 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcd380c95c0 con 0x7fcd380ce430 2023-10-18T08:55:28.740 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2023-10-18T08:55:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.734+0000 7fcd3e6fc640 1 -- 172.21.15.40:0/311884279 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcd1806ae40 msgr2=0x7fcd1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.734+0000 7fcd3e6fc640 1 --2- 172.21.15.40:0/311884279 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcd1806ae40 0x7fcd1806d300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcd28001560 tx=0x7fcd28001a30 comp rx=0 tx=0).stop 2023-10-18T08:55:28.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.734+0000 7fcd3e6fc640 1 -- 172.21.15.40:0/311884279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd380ce430 msgr2=0x7fcd381f5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.734+0000 7fcd3e6fc640 1 --2- 172.21.15.40:0/311884279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd380ce430 0x7fcd381f5690 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000e9f0 tx=0x7fcd2000eeb0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-10-18T08:55:28.743 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2023-10-18T08:55:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.737+0000 7fcd3e6fc640 1 -- 172.21.15.40:0/311884279 shutdown_connections 2023-10-18T08:55:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.737+0000 7fcd3e6fc640 1 --2- 172.21.15.40:0/311884279 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcd1806ae40 0x7fcd1806d300 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.737+0000 7fcd3e6fc640 1 --2- 172.21.15.40:0/311884279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd380b20c0 0x7fcd380b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.737+0000 7fcd3e6fc640 1 --2- 172.21.15.40:0/311884279 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd380ce430 0x7fcd381f5690 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.737+0000 7fcd3e6fc640 1 --2- 172.21.15.40:0/311884279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd380b7000 0x7fcd380caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.737+0000 7fcd3e6fc640 1 -- 172.21.15.40:0/311884279 >> 172.21.15.40:0/311884279 conn(0x7fcd3806a780 msgr2=0x7fcd3815a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.737+0000 7fcd3e6fc640 1 -- 172.21.15.40:0/311884279 shutdown_connections 2023-10-18T08:55:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.737+0000 7fcd3e6fc640 1 -- 172.21.15.40:0/311884279 wait complete. 2023-10-18T08:55:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.739+0000 7f6da91d0640 1 -- 172.21.15.40:0/1758233718 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6d840050c0 con 0x7f6da40cc1a0 2023-10-18T08:55:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.739+0000 7f6d8b7fe640 1 -- 172.21.15.40:0/1758233718 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6d94017020 con 0x7f6da40cc1a0 2023-10-18T08:55:28.746 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2023-10-18T08:55:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.742+0000 7f6da91d0640 1 -- 172.21.15.40:0/1758233718 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6d9006ae40 msgr2=0x7f6d9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.742+0000 7f6da91d0640 1 --2- 172.21.15.40:0/1758233718 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6d9006ae40 0x7f6d9006d300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6da406b6e0 tx=0x7f6d9800a2d0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.742+0000 7f6da91d0640 1 -- 172.21.15.40:0/1758233718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da40cc1a0 msgr2=0x7f6da40c9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.742+0000 7f6da91d0640 1 --2- 172.21.15.40:0/1758233718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da40cc1a0 0x7f6da40c9ba0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400c410 tx=0x7f6d9400c810 comp rx=0 tx=0).stop 2023-10-18T08:55:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.742+0000 7f6da91d0640 1 -- 172.21.15.40:0/1758233718 shutdown_connections 2023-10-18T08:55:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.742+0000 7f6da91d0640 1 --2- 172.21.15.40:0/1758233718 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6d9006ae40 0x7f6d9006d300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.742+0000 7f6da91d0640 1 --2- 172.21.15.40:0/1758233718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6da415b760 0x7f6da406a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.742+0000 7f6da91d0640 1 --2- 172.21.15.40:0/1758233718 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da40d1480 0x7f6da414c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.742+0000 7f6da91d0640 1 --2- 172.21.15.40:0/1758233718 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6da40cc1a0 0x7f6da40c9ba0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.742+0000 7f6da91d0640 1 -- 172.21.15.40:0/1758233718 >> 172.21.15.40:0/1758233718 conn(0x7f6da409a9a0 msgr2=0x7f6da406c090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.742+0000 7f6da91d0640 1 -- 172.21.15.40:0/1758233718 shutdown_connections 2023-10-18T08:55:28.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:28.742+0000 7f6da91d0640 1 -- 172.21.15.40:0/1758233718 wait complete. 2023-10-18T08:55:28.760 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-10-18T08:55:28.762 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2023-10-18T08:55:28.762 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2023-10-18T08:55:28.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2023-10-18T08:55:29.698 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-18T08:55:29.733 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-18T08:55:29.743 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-18T08:55:29.743 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-18T08:55:29.762 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-18T08:55:29.763 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-18T08:55:29.765 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-18T08:55:29.765 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-18T08:55:29.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:29.993+0000 7f1313fff640 1 -- 172.21.15.40:0/1776054862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c0df380 msgr2=0x7f130c04f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:29.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:29.993+0000 7f1313fff640 1 --2- 172.21.15.40:0/1776054862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c0df380 0x7f130c04f7b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f130c007a30 tx=0x7f130402bc40 comp rx=0 tx=0).stop 2023-10-18T08:55:29.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:29.994+0000 7f1313fff640 1 -- 172.21.15.40:0/1776054862 shutdown_connections 2023-10-18T08:55:29.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:29.994+0000 7f1313fff640 1 --2- 172.21.15.40:0/1776054862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f130c050230 0x7f130c0e2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:29.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:29.994+0000 7f1313fff640 1 --2- 172.21.15.40:0/1776054862 >> v2:172.21.15.40:3300/0 conn(0x7f130c053b30 0x7f130c04fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:29.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:29.994+0000 7f1313fff640 1 --2- 172.21.15.40:0/1776054862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c0df380 0x7f130c04f7b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:29.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:29.994+0000 7f1313fff640 1 -- 172.21.15.40:0/1776054862 >> 172.21.15.40:0/1776054862 conn(0x7f130c009f80 msgr2=0x7f130c00a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:29.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:29.994+0000 7f1313fff640 1 -- 172.21.15.40:0/1776054862 shutdown_connections 2023-10-18T08:55:30.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:29.994+0000 7f1313fff640 1 -- 172.21.15.40:0/1776054862 wait complete. 2023-10-18T08:55:30.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:29.997+0000 7f1313fff640 1 Processor -- start 2023-10-18T08:55:30.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:29.998+0000 7f1313fff640 1 -- start start 2023-10-18T08:55:30.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.000+0000 7f1313fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c050230 0x7f130c0d8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.000+0000 7f1313fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c053b30 0x7f130c039a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.001+0000 7f1313fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f130c0df380 0x7f130c039fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.001+0000 7f1312ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c050230 0x7f130c0d8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.001+0000 7f13127fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c053b30 0x7f130c039a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.001+0000 7f1312ffd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c050230 0x7f130c0d8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.40:34784/0 (socket says 172.21.15.40:34784) 2023-10-18T08:55:30.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.001+0000 7f13127fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c053b30 0x7f130c039a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48054/0 (socket says 172.21.15.40:48054) 2023-10-18T08:55:30.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.002+0000 7f1313fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c049ad0 con 0x7f130c053b30 2023-10-18T08:55:30.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.002+0000 7f1313fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c048ec0 con 0x7f130c050230 2023-10-18T08:55:30.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.002+0000 7f1313fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0491a0 con 0x7f130c0df380 2023-10-18T08:55:30.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.002+0000 7f1312ffd640 1 -- 172.21.15.40:0/1511685348 learned_addr learned my addr 172.21.15.40:0/1511685348 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:30.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.002+0000 7f13137fe640 1 --2- 172.21.15.40:0/1511685348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f130c0df380 0x7f130c039fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.003+0000 7f1312ffd640 1 -- 172.21.15.40:0/1511685348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f130c0df380 msgr2=0x7f130c039fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.003+0000 7f1312ffd640 1 --2- 172.21.15.40:0/1511685348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f130c0df380 0x7f130c039fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.003+0000 7f1312ffd640 1 -- 172.21.15.40:0/1511685348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c053b30 msgr2=0x7f130c039a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.003+0000 7f1312ffd640 1 --2- 172.21.15.40:0/1511685348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c053b30 0x7f130c039a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.004+0000 7f1312ffd640 1 -- 172.21.15.40:0/1511685348 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1304009490 con 0x7f130c050230 2023-10-18T08:55:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.004+0000 7f13127fc640 1 --2- 172.21.15.40:0/1511685348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c053b30 0x7f130c039a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T08:55:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.005+0000 7f1312ffd640 1 --2- 172.21.15.40:0/1511685348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c050230 0x7f130c0d8970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1304037040 tx=0x7f1304004450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.005+0000 7f12fb7fe640 1 -- 172.21.15.40:0/1511685348 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304005cb0 con 0x7f130c050230 2023-10-18T08:55:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.006+0000 7f1313fff640 1 -- 172.21.15.40:0/1511685348 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130c054740 con 0x7f130c050230 2023-10-18T08:55:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.006+0000 7f1313fff640 1 -- 172.21.15.40:0/1511685348 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c049760 con 0x7f130c050230 2023-10-18T08:55:30.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.006+0000 7f12fb7fe640 1 -- 172.21.15.40:0/1511685348 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1304003af0 con 0x7f130c050230 2023-10-18T08:55:30.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.006+0000 7f12fb7fe640 1 -- 172.21.15.40:0/1511685348 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1304003ee0 con 0x7f130c050230 2023-10-18T08:55:30.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.007+0000 7f1313fff640 1 -- 172.21.15.40:0/1511685348 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f4005250 con 0x7f130c050230 2023-10-18T08:55:30.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.010+0000 7f12fb7fe640 1 -- 172.21.15.40:0/1511685348 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f130404e440 con 0x7f130c050230 2023-10-18T08:55:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.012+0000 7f12fb7fe640 1 --2- 172.21.15.40:0/1511685348 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f130006ae40 0x7f130006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.012+0000 7f12fb7fe640 1 -- 172.21.15.40:0/1511685348 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13040bcf90 con 0x7f130c050230 2023-10-18T08:55:30.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.012+0000 7f13127fc640 1 --2- 172.21.15.40:0/1511685348 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f130006ae40 0x7f130006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.018+0000 7f13127fc640 1 --2- 172.21.15.40:0/1511685348 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f130006ae40 0x7f130006d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1314068ee0 tx=0x7f131408b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.022+0000 7f12fb7fe640 1 -- 172.21.15.40:0/1511685348 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1304088140 con 0x7f130c050230 2023-10-18T08:55:30.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.032+0000 7ff6b1ebb640 1 -- 172.21.15.40:0/3771695852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6ac0b0bc0 msgr2=0x7ff6ac0b1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.032+0000 7ff6b1ebb640 1 --2- 172.21.15.40:0/3771695852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6ac0b0bc0 0x7ff6ac0b1070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0acc20 tx=0x7ff69c02cf90 comp rx=0 tx=0).stop 2023-10-18T08:55:30.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.032+0000 7ff6b1ebb640 1 -- 172.21.15.40:0/3771695852 shutdown_connections 2023-10-18T08:55:30.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.032+0000 7ff6b1ebb640 1 --2- 172.21.15.40:0/3771695852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6ac0b0bc0 0x7ff6ac0b1070 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0acc20 tx=0x7ff69c02cf90 comp rx=0 tx=0).stop 2023-10-18T08:55:30.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.032+0000 7ff6b1ebb640 1 --2- 172.21.15.40:0/3771695852 >> v2:172.21.15.144:3300/0 conn(0x7ff6ac151bd0 0x7ff6ac1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.033+0000 7ff6b1ebb640 1 -- 172.21.15.40:0/3771695852 >> 172.21.15.40:0/3771695852 conn(0x7ff6ac06cee0 msgr2=0x7ff6ac069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.035+0000 7ff6b1ebb640 1 -- 172.21.15.40:0/3771695852 shutdown_connections 2023-10-18T08:55:30.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.035+0000 7ff6b1ebb640 1 -- 172.21.15.40:0/3771695852 wait complete. 2023-10-18T08:55:30.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.038+0000 7ff6b1ebb640 1 Processor -- start 2023-10-18T08:55:30.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.038+0000 7ff6b1ebb640 1 -- start start 2023-10-18T08:55:30.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.041+0000 7ff6b1ebb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6ac06b710 0x7ff6ac141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.041+0000 7ff6b1ebb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6ac151bd0 0x7ff6ac13dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.042+0000 7ff6b1ebb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6ac1e2340 0x7ff6ac1e1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.042+0000 7ff6b1ebb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac1e2d20 con 0x7ff6ac1e2340 2023-10-18T08:55:30.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.042+0000 7ff6b1ebb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac1e3000 con 0x7ff6ac06b710 2023-10-18T08:55:30.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.042+0000 7ff6b1ebb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac1e32e0 con 0x7ff6ac151bd0 2023-10-18T08:55:30.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.042+0000 7fb2f0087640 1 -- 172.21.15.40:0/739238187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e806df10 msgr2=0x7fb2e8158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.042+0000 7fb2f0087640 1 --2- 172.21.15.40:0/739238187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e806df10 0x7fb2e8158ac0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0005fd0 tx=0x7fb2e002c0f0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.042+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6ac06b710 0x7ff6ac141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.042+0000 7ff6ab7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6ac06b710 0x7ff6ac141940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.40:34816/0 (socket says 172.21.15.40:34816) 2023-10-18T08:55:30.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.042+0000 7ff6ab7fe640 1 -- 172.21.15.40:0/869444172 learned_addr learned my addr 172.21.15.40:0/869444172 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:30.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.042+0000 7ff6aaffd640 1 --2- 172.21.15.40:0/869444172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6ac151bd0 0x7ff6ac13dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.043+0000 7fb2f0087640 1 -- 172.21.15.40:0/739238187 shutdown_connections 2023-10-18T08:55:30.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.043+0000 7fb2f0087640 1 --2- 172.21.15.40:0/739238187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e806df10 0x7fb2e8158ac0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.043+0000 7fb2f0087640 1 -- 172.21.15.40:0/739238187 >> 172.21.15.40:0/739238187 conn(0x7fb2e80655f0 msgr2=0x7fb2e80659d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:30.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.043+0000 7fb2f0087640 1 -- 172.21.15.40:0/739238187 shutdown_connections 2023-10-18T08:55:30.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.043+0000 7ff6ab7fe640 1 -- 172.21.15.40:0/869444172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6ac151bd0 msgr2=0x7ff6ac13dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.043+0000 7ff6ab7fe640 1 --2- 172.21.15.40:0/869444172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6ac151bd0 0x7ff6ac13dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.043+0000 7ff6ab7fe640 1 -- 172.21.15.40:0/869444172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6ac1e2340 msgr2=0x7ff6ac1e1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.043+0000 7ff6ab7fe640 1 --2- 172.21.15.40:0/869444172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6ac1e2340 0x7ff6ac1e1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.043+0000 7ff6ab7fe640 1 -- 172.21.15.40:0/869444172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c009560 con 0x7ff6ac06b710 2023-10-18T08:55:30.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.043+0000 7f88e5fa0640 1 -- 172.21.15.40:0/1999078084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d80073b0 msgr2=0x7f88d8037bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.043+0000 7f88e5fa0640 1 --2- 172.21.15.40:0/1999078084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d80073b0 0x7f88d8037bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f88e0067cf0 tx=0x7f88e00a4570 comp rx=0 tx=0).stop 2023-10-18T08:55:30.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.044+0000 7f88e5fa0640 1 -- 172.21.15.40:0/1999078084 shutdown_connections 2023-10-18T08:55:30.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.043+0000 7fb2f0087640 1 -- 172.21.15.40:0/739238187 wait complete. 2023-10-18T08:55:30.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.044+0000 7f88e5fa0640 1 --2- 172.21.15.40:0/1999078084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d80073b0 0x7f88d8037bf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.044+0000 7f88e5fa0640 1 --2- 172.21.15.40:0/1999078084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88d804f260 0x7f88d8006e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.044+0000 7f88e5fa0640 1 -- 172.21.15.40:0/1999078084 >> 172.21.15.40:0/1999078084 conn(0x7f88d8009f80 msgr2=0x7f88d800a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:30.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.044+0000 7f88e5fa0640 1 -- 172.21.15.40:0/1999078084 shutdown_connections 2023-10-18T08:55:30.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.044+0000 7f88e5fa0640 1 -- 172.21.15.40:0/1999078084 wait complete. 2023-10-18T08:55:30.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.044+0000 7ff6ab7fe640 1 --2- 172.21.15.40:0/869444172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6ac06b710 0x7ff6ac141940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff69400b410 tx=0x7ff69400b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.045+0000 7ff6a3fff640 1 -- 172.21.15.40:0/869444172 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694011830 con 0x7ff6ac06b710 2023-10-18T08:55:30.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.045+0000 7ff6a3fff640 1 -- 172.21.15.40:0/869444172 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff694011aa0 con 0x7ff6ac06b710 2023-10-18T08:55:30.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.045+0000 7ff6a3fff640 1 -- 172.21.15.40:0/869444172 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694010380 con 0x7ff6ac06b710 2023-10-18T08:55:30.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.046+0000 7ff6b1ebb640 1 -- 172.21.15.40:0/869444172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac1e35c0 con 0x7ff6ac06b710 2023-10-18T08:55:30.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.047+0000 7f88e5fa0640 1 Processor -- start 2023-10-18T08:55:30.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.047+0000 7fb2f0087640 1 Processor -- start 2023-10-18T08:55:30.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.047+0000 7ff6b1ebb640 1 -- 172.21.15.40:0/869444172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac13e1d0 con 0x7ff6ac06b710 2023-10-18T08:55:30.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.047+0000 7f88e5fa0640 1 -- start start 2023-10-18T08:55:30.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.047+0000 7fb2f0087640 1 -- start start 2023-10-18T08:55:30.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.049+0000 7f88e5fa0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88d80073b0 0x7f88d816e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.049+0000 7f88e5fa0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d804f260 0x7f88d8036690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.049+0000 7f88e5fa0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d80e31a0 0x7f88d8036bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.049+0000 7f88e5fa0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d80d52a0 con 0x7f88d804f260 2023-10-18T08:55:30.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.049+0000 7f88e5fa0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d80d56a0 con 0x7f88d80e31a0 2023-10-18T08:55:30.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.049+0000 7f88e5fa0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d80d46b0 con 0x7f88d80073b0 2023-10-18T08:55:30.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.049+0000 7fb2f0087640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e806df10 0x7fb2e8157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.049+0000 7ff6b1ebb640 1 -- 172.21.15.40:0/869444172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ac1e3ae0 con 0x7ff6ac06b710 2023-10-18T08:55:30.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.049+0000 7fb2f0087640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e806e2f0 0x7fb2e80c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.050+0000 7fb2f0087640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2e80cfd70 0x7fb2e80c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.050+0000 7f88e579f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d80e31a0 0x7f88d8036bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.050+0000 7f88e579f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d80e31a0 0x7f88d8036bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.40:34822/0 (socket says 172.21.15.40:34822) 2023-10-18T08:55:30.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.050+0000 7f88e579f640 1 -- 172.21.15.40:0/376615360 learned_addr learned my addr 172.21.15.40:0/376615360 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:30.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.050+0000 7f88e4f9e640 1 --2- 172.21.15.40:0/376615360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88d80073b0 0x7f88d816e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.051+0000 7f88e579f640 1 -- 172.21.15.40:0/376615360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88d80073b0 msgr2=0x7f88d816e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.051+0000 7f88e579f640 1 --2- 172.21.15.40:0/376615360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88d80073b0 0x7f88d816e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.051+0000 7f88e579f640 1 -- 172.21.15.40:0/376615360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d804f260 msgr2=0x7f88d8036690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.051+0000 7f88e579f640 1 --2- 172.21.15.40:0/376615360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d804f260 0x7f88d8036690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.051+0000 7f88e579f640 1 -- 172.21.15.40:0/376615360 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e004df90 con 0x7f88d80e31a0 2023-10-18T08:55:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.050+0000 7fb2ed622640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e806e2f0 0x7fb2e80c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.050+0000 7fb2ed622640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e806e2f0 0x7fb2e80c4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:45446/0 (socket says 172.21.15.40:45446) 2023-10-18T08:55:30.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.050+0000 7fb2ee624640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2e80cfd70 0x7fb2e80c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.051+0000 7fb2ede23640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e806df10 0x7fb2e8157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.051+0000 7fb2f0087640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e80c6c40 con 0x7fb2e80cfd70 2023-10-18T08:55:30.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.051+0000 7fb2f0087640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e81583f0 con 0x7fb2e806df10 2023-10-18T08:55:30.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.051+0000 7fb2f0087640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e80c5ab0 con 0x7fb2e806e2f0 2023-10-18T08:55:30.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.051+0000 7fb2ee624640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2e80cfd70 0x7fb2e80c5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48110/0 (socket says 172.21.15.40:48110) 2023-10-18T08:55:30.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.051+0000 7fb2ee624640 1 -- 172.21.15.40:0/2052557534 learned_addr learned my addr 172.21.15.40:0/2052557534 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:30.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.051+0000 7ff6a3fff640 1 -- 172.21.15.40:0/869444172 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff6940104e0 con 0x7ff6ac06b710 2023-10-18T08:55:30.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.052+0000 7fb2ed622640 1 -- 172.21.15.40:0/2052557534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e806df10 msgr2=0x7fb2e8157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.052+0000 7fb2ed622640 1 --2- 172.21.15.40:0/2052557534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e806df10 0x7fb2e8157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.052+0000 7fb2ed622640 1 -- 172.21.15.40:0/2052557534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2e80cfd70 msgr2=0x7fb2e80c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.052+0000 7fb2ed622640 1 --2- 172.21.15.40:0/2052557534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2e80cfd70 0x7fb2e80c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.052+0000 7fb2ed622640 1 -- 172.21.15.40:0/2052557534 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e0009840 con 0x7fb2e806e2f0 2023-10-18T08:55:30.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.052+0000 7f88e579f640 1 --2- 172.21.15.40:0/376615360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d80e31a0 0x7f88d8036bd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f88e00633a0 tx=0x7f88e00ab700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.052+0000 7fb2ee624640 1 --2- 172.21.15.40:0/2052557534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2e80cfd70 0x7fb2e80c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T08:55:30.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.052+0000 7fb2ed622640 1 --2- 172.21.15.40:0/2052557534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e806e2f0 0x7fb2e80c4f40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400c390 tx=0x7fb2e4001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.053+0000 7fb2da7fc640 1 -- 172.21.15.40:0/2052557534 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40052d0 con 0x7fb2e806e2f0 2023-10-18T08:55:30.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.053+0000 7fb2da7fc640 1 -- 172.21.15.40:0/2052557534 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb2e40035f0 con 0x7fb2e806e2f0 2023-10-18T08:55:30.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.053+0000 7fb2da7fc640 1 -- 172.21.15.40:0/2052557534 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40039c0 con 0x7fb2e806e2f0 2023-10-18T08:55:30.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.053+0000 7ff6a3fff640 1 --2- 172.21.15.40:0/869444172 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff68c06ae90 0x7ff68c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.053+0000 7ff6a3fff640 1 -- 172.21.15.40:0/869444172 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6940897f0 con 0x7ff6ac06b710 2023-10-18T08:55:30.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.054+0000 7ff6aaffd640 1 --2- 172.21.15.40:0/869444172 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff68c06ae90 0x7ff68c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.053+0000 7fb2f0087640 1 -- 172.21.15.40:0/2052557534 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e80c6e90 con 0x7fb2e806e2f0 2023-10-18T08:55:30.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.053+0000 7fb2f0087640 1 -- 172.21.15.40:0/2052557534 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e80c7450 con 0x7fb2e806e2f0 2023-10-18T08:55:30.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.056+0000 7ff6aaffd640 1 --2- 172.21.15.40:0/869444172 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff68c06ae90 0x7ff68c06d350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff69c036000 tx=0x7ff69c02f040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.056+0000 7fb2f0087640 1 -- 172.21.15.40:0/2052557534 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d0005250 con 0x7fb2e806e2f0 2023-10-18T08:55:30.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.058+0000 7fb2da7fc640 1 -- 172.21.15.40:0/2052557534 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb2e4003b20 con 0x7fb2e806e2f0 2023-10-18T08:55:30.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.060+0000 7fb2da7fc640 1 --2- 172.21.15.40:0/2052557534 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb2cc06ae40 0x7fb2cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.060+0000 7fb2da7fc640 1 -- 172.21.15.40:0/2052557534 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2e4088ba0 con 0x7fb2e806e2f0 2023-10-18T08:55:30.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.061+0000 7fb2ede23640 1 --2- 172.21.15.40:0/2052557534 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb2cc06ae40 0x7fb2cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.062+0000 7fb2ede23640 1 --2- 172.21.15.40:0/2052557534 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb2cc06ae40 0x7fb2cc06d300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0035000 tx=0x7fb2e0004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.067+0000 7f88dd7fa640 1 -- 172.21.15.40:0/376615360 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e00b7430 con 0x7f88d80e31a0 2023-10-18T08:55:30.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.067+0000 7f88dd7fa640 1 -- 172.21.15.40:0/376615360 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f88e00b7680 con 0x7f88d80e31a0 2023-10-18T08:55:30.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.067+0000 7f88dd7fa640 1 -- 172.21.15.40:0/376615360 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e00b7a30 con 0x7f88d80e31a0 2023-10-18T08:55:30.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.067+0000 7f88e5fa0640 1 -- 172.21.15.40:0/376615360 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d80d4900 con 0x7f88d80e31a0 2023-10-18T08:55:30.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.068+0000 7f88e5fa0640 1 -- 172.21.15.40:0/376615360 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d80d4ec0 con 0x7f88d80e31a0 2023-10-18T08:55:30.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.068+0000 7ff6a3fff640 1 -- 172.21.15.40:0/869444172 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff6ac1e3ae0 con 0x7ff6ac06b710 2023-10-18T08:55:30.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.070+0000 7f88e5fa0640 1 -- 172.21.15.40:0/376615360 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d80089a0 con 0x7f88d80e31a0 2023-10-18T08:55:30.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.073+0000 7fb2da7fc640 1 -- 172.21.15.40:0/2052557534 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb2e4053d70 con 0x7fb2e806e2f0 2023-10-18T08:55:30.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.072+0000 7f88dd7fa640 1 -- 172.21.15.40:0/376615360 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f88e00a9070 con 0x7f88d80e31a0 2023-10-18T08:55:30.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.074+0000 7f88dd7fa640 1 --2- 172.21.15.40:0/376615360 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f88c406ae90 0x7f88c406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.074+0000 7f88dd7fa640 1 -- 172.21.15.40:0/376615360 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88e00f3f30 con 0x7f88d80e31a0 2023-10-18T08:55:30.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.074+0000 7f88e4f9e640 1 --2- 172.21.15.40:0/376615360 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f88c406ae90 0x7f88c406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.077+0000 7f88e4f9e640 1 --2- 172.21.15.40:0/376615360 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f88c406ae90 0x7f88c406d350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f88d4006fd0 tx=0x7f88d4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.085+0000 7f88dd7fa640 1 -- 172.21.15.40:0/376615360 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f88e00f0300 con 0x7f88d80e31a0 2023-10-18T08:55:30.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.116+0000 7f4a3557d640 1 -- 172.21.15.40:0/1292404481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a30141e60 msgr2=0x7f4a3015c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.116+0000 7f4a3557d640 1 --2- 172.21.15.40:0/1292404481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a30141e60 0x7f4a3015c7f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4a24005fd0 tx=0x7f4a2402bd80 comp rx=0 tx=0).stop 2023-10-18T08:55:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.116+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2650468787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f815ed60 msgr2=0x7fb3f80affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.116+0000 7fb3fd4df640 1 --2- 172.21.15.40:0/2650468787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f815ed60 0x7fb3f80affb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb3f808a6b0 tx=0x7fb3e802bc60 comp rx=0 tx=0).stop 2023-10-18T08:55:30.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.117+0000 7f4a3557d640 1 -- 172.21.15.40:0/1292404481 shutdown_connections 2023-10-18T08:55:30.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.117+0000 7f4a3557d640 1 --2- 172.21.15.40:0/1292404481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a3006be20 0x7f4a30158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.117+0000 7f4a3557d640 1 --2- 172.21.15.40:0/1292404481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a30141e60 0x7f4a3015c7f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.117+0000 7f4a3557d640 1 -- 172.21.15.40:0/1292404481 >> 172.21.15.40:0/1292404481 conn(0x7f4a300aede0 msgr2=0x7f4a300af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.117+0000 7f8a1b59e640 1 -- 172.21.15.40:0/1990108403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a140e3130 msgr2=0x7f8a140e7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.117+0000 7f8a1b59e640 1 --2- 172.21.15.40:0/1990108403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a140e3130 0x7f8a140e7750 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009830 tx=0x7f8a0c02bd50 comp rx=0 tx=0).stop 2023-10-18T08:55:30.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.117+0000 7f4a3557d640 1 -- 172.21.15.40:0/1292404481 shutdown_connections 2023-10-18T08:55:30.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.117+0000 7f4a3557d640 1 -- 172.21.15.40:0/1292404481 wait complete. 2023-10-18T08:55:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.117+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2650468787 shutdown_connections 2023-10-18T08:55:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.117+0000 7fb3fd4df640 1 --2- 172.21.15.40:0/2650468787 >> v2:172.21.15.144:3300/0 conn(0x7fb3f80b0dd0 0x7fb3f80b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.117+0000 7fb3fd4df640 1 --2- 172.21.15.40:0/2650468787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f80cdbc0 0x7fb3f80ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.117+0000 7fb3fd4df640 1 --2- 172.21.15.40:0/2650468787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f815ed60 0x7fb3f80affb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.117+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2650468787 >> 172.21.15.40:0/2650468787 conn(0x7fb3f809a580 msgr2=0x7fb3f80d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.117+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2650468787 shutdown_connections 2023-10-18T08:55:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.119+0000 7f8a1b59e640 1 -- 172.21.15.40:0/1990108403 shutdown_connections 2023-10-18T08:55:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.119+0000 7f8a1b59e640 1 --2- 172.21.15.40:0/1990108403 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a14053c80 0x7f8a14054110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.119+0000 7f8a1b59e640 1 --2- 172.21.15.40:0/1990108403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a140e3130 0x7f8a140e7750 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.119+0000 7f8a1b59e640 1 -- 172.21.15.40:0/1990108403 >> 172.21.15.40:0/1990108403 conn(0x7f8a14058eb0 msgr2=0x7f8a14059290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.119+0000 7f8a1b59e640 1 -- 172.21.15.40:0/1990108403 shutdown_connections 2023-10-18T08:55:30.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.119+0000 7f8a1b59e640 1 -- 172.21.15.40:0/1990108403 wait complete. 2023-10-18T08:55:30.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.119+0000 7f4a3557d640 1 Processor -- start 2023-10-18T08:55:30.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.119+0000 7f4a3557d640 1 -- start start 2023-10-18T08:55:30.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.121+0000 7f4a3557d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a3006be20 0x7f4a30152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.121+0000 7f4a3557d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a300b00e0 0x7f4a30153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.121+0000 7f4a3557d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a30141e60 0x7f4a3015aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.121+0000 7f4a3557d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a30136330 con 0x7f4a3006be20 2023-10-18T08:55:30.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.121+0000 7f4a3557d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3015bc20 con 0x7f4a30141e60 2023-10-18T08:55:30.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.121+0000 7f4a3557d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a30139450 con 0x7f4a300b00e0 2023-10-18T08:55:30.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.121+0000 7f4a2effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a3006be20 0x7f4a30152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.121+0000 7f4a2effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a3006be20 0x7f4a30152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48148/0 (socket says 172.21.15.40:48148) 2023-10-18T08:55:30.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.121+0000 7f4a2effd640 1 -- 172.21.15.40:0/3257097860 learned_addr learned my addr 172.21.15.40:0/3257097860 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:30.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.122+0000 7f4a2e7fc640 1 --2- 172.21.15.40:0/3257097860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a300b00e0 0x7f4a30153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.122+0000 7f4a2effd640 1 -- 172.21.15.40:0/3257097860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a300b00e0 msgr2=0x7f4a30153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.122+0000 7f4a2effd640 1 --2- 172.21.15.40:0/3257097860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a300b00e0 0x7f4a30153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.122+0000 7f4a2effd640 1 -- 172.21.15.40:0/3257097860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a30141e60 msgr2=0x7f4a3015aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.122+0000 7f8a1b59e640 1 Processor -- start 2023-10-18T08:55:30.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.122+0000 7f4a2f7fe640 1 --2- 172.21.15.40:0/3257097860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a30141e60 0x7f4a3015aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.124+0000 7f4a2effd640 1 --2- 172.21.15.40:0/3257097860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a30141e60 0x7f4a3015aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.124+0000 7f4a2effd640 1 -- 172.21.15.40:0/3257097860 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a24009560 con 0x7f4a3006be20 2023-10-18T08:55:30.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.124+0000 7f4a2e7fc640 1 --2- 172.21.15.40:0/3257097860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a300b00e0 0x7f4a30153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:30.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.124+0000 7f4a2f7fe640 1 --2- 172.21.15.40:0/3257097860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a30141e60 0x7f4a3015aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:30.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.122+0000 7f8a1b59e640 1 -- start start 2023-10-18T08:55:30.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.124+0000 7f4a2effd640 1 --2- 172.21.15.40:0/3257097860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a3006be20 0x7f4a30152f20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4a24009850 tx=0x7f4a2402da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.124+0000 7f8a1b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a1403c070 0x7f8a140e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.124+0000 7f8a1b59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a14053c80 0x7f8a14048690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.124+0000 7f8a1b59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a140e3130 0x7f8a14048bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.124+0000 7f8a1b59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a14049110 con 0x7f8a1403c070 2023-10-18T08:55:30.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.124+0000 7f8a1b59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a140cf060 con 0x7f8a14053c80 2023-10-18T08:55:30.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.124+0000 7f8a1b59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a14052940 con 0x7f8a140e3130 2023-10-18T08:55:30.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.120+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2650468787 wait complete. 2023-10-18T08:55:30.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.125+0000 7fb3fd4df640 1 Processor -- start 2023-10-18T08:55:30.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.125+0000 7f8a1ad9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a140e3130 0x7f8a14048bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.125+0000 7f4a177fe640 1 -- 172.21.15.40:0/3257097860 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2402de50 con 0x7f4a3006be20 2023-10-18T08:55:30.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.125+0000 7f4a177fe640 1 -- 172.21.15.40:0/3257097860 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4a240036e0 con 0x7f4a3006be20 2023-10-18T08:55:30.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.125+0000 7f4a177fe640 1 -- 172.21.15.40:0/3257097860 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a24003a90 con 0x7f4a3006be20 2023-10-18T08:55:30.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.125+0000 7fb3fd4df640 1 -- start start 2023-10-18T08:55:30.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.125+0000 7f8a1ad9d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a140e3130 0x7f8a14048bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:45498/0 (socket says 172.21.15.40:45498) 2023-10-18T08:55:30.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.125+0000 7f8a1ad9d640 1 -- 172.21.15.40:0/1761904080 learned_addr learned my addr 172.21.15.40:0/1761904080 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:30.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.125+0000 7f4a3557d640 1 -- 172.21.15.40:0/3257097860 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3013dc90 con 0x7f4a3006be20 2023-10-18T08:55:30.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.126+0000 7f8a1a59c640 1 --2- 172.21.15.40:0/1761904080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a1403c070 0x7f8a140e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.125+0000 7f4a3557d640 1 -- 172.21.15.40:0/3257097860 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a301364c0 con 0x7f4a3006be20 2023-10-18T08:55:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.126+0000 7f8a1ad9d640 1 -- 172.21.15.40:0/1761904080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a14053c80 msgr2=0x7f8a14048690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T08:55:30.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.126+0000 7f8a1ad9d640 1 --2- 172.21.15.40:0/1761904080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a14053c80 0x7f8a14048690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.126+0000 7f8a1ad9d640 1 -- 172.21.15.40:0/1761904080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a1403c070 msgr2=0x7f8a140e4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.126+0000 7f8a1ad9d640 1 --2- 172.21.15.40:0/1761904080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a1403c070 0x7f8a140e4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.126+0000 7f8a1ad9d640 1 -- 172.21.15.40:0/1761904080 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c009490 con 0x7f8a140e3130 2023-10-18T08:55:30.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.127+0000 7fb3fd4df640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f80b0dd0 0x7fb3f809d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.128+0000 7fb3fd4df640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3f80cdbc0 0x7fb3f809de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.128+0000 7fb3fd4df640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f815ed60 0x7fb3f814e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.128+0000 7fb3f7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f80b0dd0 0x7fb3f809d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.128+0000 7f8a1ad9d640 1 --2- 172.21.15.40:0/1761904080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a140e3130 0x7f8a14048bd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c063080 tx=0x7f8a1c073940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.128+0000 7fb3f7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f80b0dd0 0x7fb3f809d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48160/0 (socket says 172.21.15.40:48160) 2023-10-18T08:55:30.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.129+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2652962573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f809ebf0 con 0x7fb3f80b0dd0 2023-10-18T08:55:30.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.128+0000 7fb3f7fff640 1 -- 172.21.15.40:0/2652962573 learned_addr learned my addr 172.21.15.40:0/2652962573 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:30.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.129+0000 7f8a0affd640 1 -- 172.21.15.40:0/1761904080 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c066990 con 0x7f8a140e3130 2023-10-18T08:55:30.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.129+0000 7f8a0affd640 1 -- 172.21.15.40:0/1761904080 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8a1c069720 con 0x7f8a140e3130 2023-10-18T08:55:30.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.129+0000 7f8a1b59e640 1 -- 172.21.15.40:0/1761904080 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a14052380 con 0x7f8a140e3130 2023-10-18T08:55:30.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.129+0000 7f4a177fe640 1 -- 172.21.15.40:0/3257097860 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4a24003bf0 con 0x7f4a3006be20 2023-10-18T08:55:30.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.129+0000 7fb3f77fe640 1 --2- 172.21.15.40:0/2652962573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3f80cdbc0 0x7fb3f809de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.129+0000 7f8a0affd640 1 -- 172.21.15.40:0/1761904080 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c069b30 con 0x7f8a140e3130 2023-10-18T08:55:30.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.129+0000 7f8a1b59e640 1 -- 172.21.15.40:0/1761904080 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a140cecf0 con 0x7f8a140e3130 2023-10-18T08:55:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.129+0000 7fb3fccde640 1 --2- 172.21.15.40:0/2652962573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f815ed60 0x7fb3f814e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.129+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2652962573 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f809eed0 con 0x7fb3f80cdbc0 2023-10-18T08:55:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.130+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2652962573 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f809d110 con 0x7fb3f815ed60 2023-10-18T08:55:30.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.131+0000 7f4a3557d640 1 -- 172.21.15.40:0/3257097860 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a30136650 con 0x7f4a3006be20 2023-10-18T08:55:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.131+0000 7fb3fccde640 1 -- 172.21.15.40:0/2652962573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3f80cdbc0 msgr2=0x7fb3f809de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.131+0000 7fb3fccde640 1 --2- 172.21.15.40:0/2652962573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3f80cdbc0 0x7fb3f809de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.131+0000 7fb3fccde640 1 -- 172.21.15.40:0/2652962573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f80b0dd0 msgr2=0x7fb3f809d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.131+0000 7fb3fccde640 1 --2- 172.21.15.40:0/2652962573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f80b0dd0 0x7fb3f809d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.131+0000 7fb3fccde640 1 -- 172.21.15.40:0/2652962573 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e8009490 con 0x7fb3f815ed60 2023-10-18T08:55:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.132+0000 7f4a177fe640 1 --2- 172.21.15.40:0/3257097860 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a1c06af10 0x7f4a1c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.132+0000 7fb3fccde640 1 --2- 172.21.15.40:0/2652962573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f815ed60 0x7fb3f814e530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000ef00 tx=0x7fb3f000c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.132+0000 7fb3f7fff640 1 --2- 172.21.15.40:0/2652962573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f80b0dd0 0x7fb3f809d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:55:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.132+0000 7fda0bfff640 1 -- 172.21.15.40:0/1530385810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda0c15ed60 msgr2=0x7fda0c0affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.132+0000 7fda0bfff640 1 --2- 172.21.15.40:0/1530385810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda0c15ed60 0x7fda0c0affb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fda04005fd0 tx=0x7fda0402c060 comp rx=0 tx=0).stop 2023-10-18T08:55:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.132+0000 7fda0bfff640 1 -- 172.21.15.40:0/1530385810 shutdown_connections 2023-10-18T08:55:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.132+0000 7fda0bfff640 1 --2- 172.21.15.40:0/1530385810 >> v2:172.21.15.144:3300/0 conn(0x7fda0c0cdbc0 0x7fda0c0ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.132+0000 7fda0bfff640 1 --2- 172.21.15.40:0/1530385810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda0c15ed60 0x7fda0c0affb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.132+0000 7fda0bfff640 1 -- 172.21.15.40:0/1530385810 >> 172.21.15.40:0/1530385810 conn(0x7fda0c09a580 msgr2=0x7fda0c0d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.132+0000 7f4a177fe640 1 -- 172.21.15.40:0/3257097860 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a240ac080 con 0x7f4a3006be20 2023-10-18T08:55:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.132+0000 7f8a1b59e640 1 -- 172.21.15.40:0/1761904080 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a14052150 con 0x7f8a140e3130 2023-10-18T08:55:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.133+0000 7fb3f4ff9640 1 -- 172.21.15.40:0/2652962573 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0004b20 con 0x7fb3f815ed60 2023-10-18T08:55:30.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.133+0000 7f4a2e7fc640 1 --2- 172.21.15.40:0/3257097860 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a1c06af10 0x7f4a1c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.133+0000 7fda0bfff640 1 -- 172.21.15.40:0/1530385810 shutdown_connections 2023-10-18T08:55:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.133+0000 7fb3f4ff9640 1 -- 172.21.15.40:0/2652962573 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb3f0004d70 con 0x7fb3f815ed60 2023-10-18T08:55:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.133+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2652962573 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f809d3f0 con 0x7fb3f815ed60 2023-10-18T08:55:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.133+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2652962573 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f809b850 con 0x7fb3f815ed60 2023-10-18T08:55:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.135+0000 7fb3f4ff9640 1 -- 172.21.15.40:0/2652962573 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0003bd0 con 0x7fb3f815ed60 2023-10-18T08:55:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.135+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2652962573 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f8073570 con 0x7fb3f815ed60 2023-10-18T08:55:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.133+0000 7fda0bfff640 1 -- 172.21.15.40:0/1530385810 wait complete. 2023-10-18T08:55:30.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.136+0000 7f4a2e7fc640 1 --2- 172.21.15.40:0/3257097860 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a1c06af10 0x7f4a1c06d3d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4a300ccc30 tx=0x7f4a20009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.136+0000 7fda0bfff640 1 Processor -- start 2023-10-18T08:55:30.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.136+0000 7f8a0affd640 1 -- 172.21.15.40:0/1761904080 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8a1c069c90 con 0x7f8a140e3130 2023-10-18T08:55:30.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.137+0000 7fda0bfff640 1 -- start start 2023-10-18T08:55:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.137+0000 7f8a0affd640 1 --2- 172.21.15.40:0/1761904080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f89fc06ae40 0x7f89fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.137+0000 7f8a0affd640 1 -- 172.21.15.40:0/1761904080 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a1c101fb0 con 0x7f8a140e3130 2023-10-18T08:55:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.138+0000 7f8a1a59c640 1 --2- 172.21.15.40:0/1761904080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f89fc06ae40 0x7f89fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.138+0000 7fda0bfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda0c0b0dd0 0x7fda0c0c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.138+0000 7fda0bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c0cdbc0 0x7fda0c0b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.138+0000 7fda0bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c15ed60 0x7fda0c1581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.138+0000 7fda0bfff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c0c8140 con 0x7fda0c15ed60 2023-10-18T08:55:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.138+0000 7fda0bfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c1fe920 con 0x7fda0c0cdbc0 2023-10-18T08:55:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.138+0000 7fda0bfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c0c8380 con 0x7fda0c0b0dd0 2023-10-18T08:55:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.139+0000 7fda0affd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda0c0b0dd0 0x7fda0c0c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.139+0000 7fda0a7fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c0cdbc0 0x7fda0c0b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.139+0000 7fda0affd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda0c0b0dd0 0x7fda0c0c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:45502/0 (socket says 172.21.15.40:45502) 2023-10-18T08:55:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.139+0000 7fda0affd640 1 -- 172.21.15.40:0/1104074969 learned_addr learned my addr 172.21.15.40:0/1104074969 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.139+0000 7fda0b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c15ed60 0x7fda0c1581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.140+0000 7f8a1a59c640 1 --2- 172.21.15.40:0/1761904080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f89fc06ae40 0x7f89fc06d300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c036000 tx=0x7f8a0c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.140+0000 7fda0b7fe640 1 -- 172.21.15.40:0/1104074969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda0c0b0dd0 msgr2=0x7fda0c0c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.140+0000 7f4a177fe640 1 -- 172.21.15.40:0/3257097860 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a2407ad50 con 0x7f4a3006be20 2023-10-18T08:55:30.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.140+0000 7fda0b7fe640 1 --2- 172.21.15.40:0/1104074969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda0c0b0dd0 0x7fda0c0c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.140+0000 7fda0b7fe640 1 -- 172.21.15.40:0/1104074969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c0cdbc0 msgr2=0x7fda0c0b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.140+0000 7fda0b7fe640 1 --2- 172.21.15.40:0/1104074969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c0cdbc0 0x7fda0c0b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.140+0000 7fda0b7fe640 1 -- 172.21.15.40:0/1104074969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda04009840 con 0x7fda0c15ed60 2023-10-18T08:55:30.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.140+0000 7fda0b7fe640 1 --2- 172.21.15.40:0/1104074969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c15ed60 0x7fda0c1581f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00c3c0 tx=0x7fd9fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.141+0000 7fda037fe640 1 -- 172.21.15.40:0/1104074969 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc004230 con 0x7fda0c15ed60 2023-10-18T08:55:30.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.141+0000 7fda037fe640 1 -- 172.21.15.40:0/1104074969 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd9fc004480 con 0x7fda0c15ed60 2023-10-18T08:55:30.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.141+0000 7fda037fe640 1 -- 172.21.15.40:0/1104074969 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc003850 con 0x7fda0c15ed60 2023-10-18T08:55:30.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.141+0000 7fda0bfff640 1 -- 172.21.15.40:0/1104074969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0c0c8660 con 0x7fda0c15ed60 2023-10-18T08:55:30.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.142+0000 7fda0bfff640 1 -- 172.21.15.40:0/1104074969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0c0c8c20 con 0x7fda0c15ed60 2023-10-18T08:55:30.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.143+0000 7fda0bfff640 1 -- 172.21.15.40:0/1104074969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda0c067030 con 0x7fda0c15ed60 2023-10-18T08:55:30.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.145+0000 7fda037fe640 1 -- 172.21.15.40:0/1104074969 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd9fc0039b0 con 0x7fda0c15ed60 2023-10-18T08:55:30.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.146+0000 7fda037fe640 1 --2- 172.21.15.40:0/1104074969 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd9ec06aec0 0x7fd9ec06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.147+0000 7fda037fe640 1 -- 172.21.15.40:0/1104074969 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9fc088bc0 con 0x7fda0c15ed60 2023-10-18T08:55:30.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.147+0000 7fda0affd640 1 --2- 172.21.15.40:0/1104074969 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd9ec06aec0 0x7fd9ec06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.149+0000 7fda0affd640 1 --2- 172.21.15.40:0/1104074969 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd9ec06aec0 0x7fd9ec06d380 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fda04000f80 tx=0x7fda04004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.150+0000 7f8a0affd640 1 -- 172.21.15.40:0/1761904080 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a1c0cd0d0 con 0x7f8a140e3130 2023-10-18T08:55:30.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.156+0000 7fb3f4ff9640 1 -- 172.21.15.40:0/2652962573 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb3f000a040 con 0x7fb3f815ed60 2023-10-18T08:55:30.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.157+0000 7fda037fe640 1 -- 172.21.15.40:0/1104074969 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9fc053d90 con 0x7fda0c15ed60 2023-10-18T08:55:30.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.159+0000 7fb3f4ff9640 1 --2- 172.21.15.40:0/2652962573 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3e406ae40 0x7fb3e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.159+0000 7fb3f4ff9640 1 -- 172.21.15.40:0/2652962573 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3f008a460 con 0x7fb3f815ed60 2023-10-18T08:55:30.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.159+0000 7fb3f7fff640 1 --2- 172.21.15.40:0/2652962573 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3e406ae40 0x7fb3e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.161+0000 7fb3f7fff640 1 --2- 172.21.15.40:0/2652962573 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3e406ae40 0x7fb3e406d300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80661a0 tx=0x7fb3e8004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.162+0000 7fb3f4ff9640 1 -- 172.21.15.40:0/2652962573 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3f00555f0 con 0x7fb3f815ed60 2023-10-18T08:55:30.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.449+0000 7f1313fff640 1 -- 172.21.15.40:0/1511685348 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f12f40050c0 con 0x7f130c050230 2023-10-18T08:55:30.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.450+0000 7f12fb7fe640 1 -- 172.21.15.40:0/1511685348 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f130408b760 con 0x7f130c050230 2023-10-18T08:55:30.454 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2023-10-18T08:55:30.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.454+0000 7f1313fff640 1 -- 172.21.15.40:0/1511685348 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f130006ae40 msgr2=0x7f130006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.454+0000 7f1313fff640 1 --2- 172.21.15.40:0/1511685348 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f130006ae40 0x7f130006d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1314068ee0 tx=0x7f131408b040 comp rx=0 tx=0).stop 2023-10-18T08:55:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.455+0000 7f1313fff640 1 -- 172.21.15.40:0/1511685348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c050230 msgr2=0x7f130c0d8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.455+0000 7f1313fff640 1 --2- 172.21.15.40:0/1511685348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c050230 0x7f130c0d8970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1304037040 tx=0x7f1304004450 comp rx=0 tx=0).stop 2023-10-18T08:55:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.455+0000 7f1313fff640 1 -- 172.21.15.40:0/1511685348 shutdown_connections 2023-10-18T08:55:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.455+0000 7f1313fff640 1 --2- 172.21.15.40:0/1511685348 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f130006ae40 0x7f130006d300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.455+0000 7f1313fff640 1 --2- 172.21.15.40:0/1511685348 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f130c0df380 0x7f130c039fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.455+0000 7f1313fff640 1 --2- 172.21.15.40:0/1511685348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f130c053b30 0x7f130c039a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.455+0000 7f1313fff640 1 --2- 172.21.15.40:0/1511685348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f130c050230 0x7f130c0d8970 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.455+0000 7f1313fff640 1 -- 172.21.15.40:0/1511685348 >> 172.21.15.40:0/1511685348 conn(0x7f130c009f80 msgr2=0x7f130c0e1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.456+0000 7f1313fff640 1 -- 172.21.15.40:0/1511685348 shutdown_connections 2023-10-18T08:55:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.456+0000 7f1313fff640 1 -- 172.21.15.40:0/1511685348 wait complete. 2023-10-18T08:55:30.486 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-10-18T08:55:30.486 DEBUG:teuthology.parallel:result is None 2023-10-18T08:55:30.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.487+0000 7ff6b1ebb640 1 -- 172.21.15.40:0/869444172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff6ac1e3ae0 con 0x7ff6ac06b710 2023-10-18T08:55:30.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.488+0000 7ff6a3fff640 1 -- 172.21.15.40:0/869444172 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff6ac1e3ae0 con 0x7ff6ac06b710 2023-10-18T08:55:30.491 INFO:teuthology.orchestra.run.smithi040.stdout:55834574856 2023-10-18T08:55:30.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.491+0000 7ff6b1ebb640 1 -- 172.21.15.40:0/869444172 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff68c06ae90 msgr2=0x7ff68c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.491+0000 7ff6b1ebb640 1 --2- 172.21.15.40:0/869444172 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff68c06ae90 0x7ff68c06d350 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff69c036000 tx=0x7ff69c02f040 comp rx=0 tx=0).stop 2023-10-18T08:55:30.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.491+0000 7ff6b1ebb640 1 -- 172.21.15.40:0/869444172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6ac06b710 msgr2=0x7ff6ac141940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.491+0000 7ff6b1ebb640 1 --2- 172.21.15.40:0/869444172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6ac06b710 0x7ff6ac141940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff69400b410 tx=0x7ff69400b8d0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.491+0000 7ff6b1ebb640 1 -- 172.21.15.40:0/869444172 shutdown_connections 2023-10-18T08:55:30.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.491+0000 7ff6b1ebb640 1 --2- 172.21.15.40:0/869444172 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff68c06ae90 0x7ff68c06d350 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.491+0000 7ff6b1ebb640 1 --2- 172.21.15.40:0/869444172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6ac1e2340 0x7ff6ac1e1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.491+0000 7ff6b1ebb640 1 --2- 172.21.15.40:0/869444172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6ac151bd0 0x7ff6ac13dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.491+0000 7ff6b1ebb640 1 --2- 172.21.15.40:0/869444172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6ac06b710 0x7ff6ac141940 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.491+0000 7ff6b1ebb640 1 -- 172.21.15.40:0/869444172 >> 172.21.15.40:0/869444172 conn(0x7ff6ac06cee0 msgr2=0x7ff6ac150b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:30.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.492+0000 7ff6b1ebb640 1 -- 172.21.15.40:0/869444172 shutdown_connections 2023-10-18T08:55:30.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.492+0000 7ff6b1ebb640 1 -- 172.21.15.40:0/869444172 wait complete. 2023-10-18T08:55:30.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.498+0000 7fb2f0087640 1 -- 172.21.15.40:0/2052557534 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb2d00050c0 con 0x7fb2e806e2f0 2023-10-18T08:55:30.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.499+0000 7fb2da7fc640 1 -- 172.21.15.40:0/2052557534 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb2e4017090 con 0x7fb2e806e2f0 2023-10-18T08:55:30.502 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-18T08:55:30.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.502+0000 7f88e5fa0640 1 -- 172.21.15.40:0/376615360 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f88d8007800 con 0x7f88d80e31a0 2023-10-18T08:55:30.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.502+0000 7fb2f0087640 1 -- 172.21.15.40:0/2052557534 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb2cc06ae40 msgr2=0x7fb2cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.502+0000 7fb2f0087640 1 --2- 172.21.15.40:0/2052557534 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb2cc06ae40 0x7fb2cc06d300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0035000 tx=0x7fb2e0004ab0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.502+0000 7fb2f0087640 1 -- 172.21.15.40:0/2052557534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e806e2f0 msgr2=0x7fb2e80c4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.502+0000 7fb2f0087640 1 --2- 172.21.15.40:0/2052557534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e806e2f0 0x7fb2e80c4f40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400c390 tx=0x7fb2e4001e70 comp rx=0 tx=0).stop 2023-10-18T08:55:30.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.502+0000 7fb2f0087640 1 -- 172.21.15.40:0/2052557534 shutdown_connections 2023-10-18T08:55:30.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.502+0000 7fb2f0087640 1 --2- 172.21.15.40:0/2052557534 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb2cc06ae40 0x7fb2cc06d300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.502+0000 7fb2f0087640 1 --2- 172.21.15.40:0/2052557534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2e80cfd70 0x7fb2e80c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.502+0000 7fb2f0087640 1 --2- 172.21.15.40:0/2052557534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e806e2f0 0x7fb2e80c4f40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.502+0000 7fb2f0087640 1 --2- 172.21.15.40:0/2052557534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e806df10 0x7fb2e8157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.502+0000 7fb2f0087640 1 -- 172.21.15.40:0/2052557534 >> 172.21.15.40:0/2052557534 conn(0x7fb2e80655f0 msgr2=0x7fb2e8159ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:30.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.503+0000 7fb2f0087640 1 -- 172.21.15.40:0/2052557534 shutdown_connections 2023-10-18T08:55:30.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.503+0000 7fb2f0087640 1 -- 172.21.15.40:0/2052557534 wait complete. 2023-10-18T08:55:30.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.503+0000 7f88dd7fa640 1 -- 172.21.15.40:0/376615360 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f88e00bb030 con 0x7f88d80e31a0 2023-10-18T08:55:30.509 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-18T08:55:30.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.506+0000 7f88e5fa0640 1 -- 172.21.15.40:0/376615360 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f88c406ae90 msgr2=0x7f88c406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.506+0000 7f88e5fa0640 1 --2- 172.21.15.40:0/376615360 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f88c406ae90 0x7f88c406d350 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f88d4006fd0 tx=0x7f88d4008040 comp rx=0 tx=0).stop 2023-10-18T08:55:30.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.506+0000 7f88e5fa0640 1 -- 172.21.15.40:0/376615360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d80e31a0 msgr2=0x7f88d8036bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.506+0000 7f88e5fa0640 1 --2- 172.21.15.40:0/376615360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d80e31a0 0x7f88d8036bd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f88e00633a0 tx=0x7f88e00ab700 comp rx=0 tx=0).stop 2023-10-18T08:55:30.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.506+0000 7f88e5fa0640 1 -- 172.21.15.40:0/376615360 shutdown_connections 2023-10-18T08:55:30.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.506+0000 7f88e5fa0640 1 --2- 172.21.15.40:0/376615360 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f88c406ae90 0x7f88c406d350 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.506+0000 7f88e5fa0640 1 --2- 172.21.15.40:0/376615360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d80e31a0 0x7f88d8036bd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.506+0000 7f88e5fa0640 1 --2- 172.21.15.40:0/376615360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88d804f260 0x7f88d8036690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.506+0000 7f88e5fa0640 1 --2- 172.21.15.40:0/376615360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f88d80073b0 0x7f88d816e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.506+0000 7f88e5fa0640 1 -- 172.21.15.40:0/376615360 >> 172.21.15.40:0/376615360 conn(0x7f88d8009f80 msgr2=0x7f88d8034e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:30.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.506+0000 7f88e5fa0640 1 -- 172.21.15.40:0/376615360 shutdown_connections 2023-10-18T08:55:30.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.507+0000 7f88e5fa0640 1 -- 172.21.15.40:0/376615360 wait complete. 2023-10-18T08:55:30.523 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2023-10-18T08:55:30.523 DEBUG:teuthology.parallel:result is None 2023-10-18T08:55:30.527 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-10-18T08:55:30.527 DEBUG:teuthology.parallel:result is None 2023-10-18T08:55:30.531 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-10-18T08:55:30.532 DEBUG:teuthology.parallel:result is None 2023-10-18T08:55:30.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.536+0000 7f4a3557d640 1 -- 172.21.15.40:0/3257097860 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4a30066fd0 con 0x7f4a3006be20 2023-10-18T08:55:30.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.536+0000 7f4a177fe640 1 -- 172.21.15.40:0/3257097860 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4a24043090 con 0x7f4a3006be20 2023-10-18T08:55:30.540 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2023-10-18T08:55:30.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.538+0000 7f4a3557d640 1 -- 172.21.15.40:0/3257097860 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a1c06af10 msgr2=0x7f4a1c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.538+0000 7f4a3557d640 1 --2- 172.21.15.40:0/3257097860 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a1c06af10 0x7f4a1c06d3d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4a300ccc30 tx=0x7f4a20009040 comp rx=0 tx=0).stop 2023-10-18T08:55:30.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.538+0000 7f4a3557d640 1 -- 172.21.15.40:0/3257097860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a3006be20 msgr2=0x7f4a30152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.538+0000 7f4a3557d640 1 --2- 172.21.15.40:0/3257097860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a3006be20 0x7f4a30152f20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4a24009850 tx=0x7f4a2402da50 comp rx=0 tx=0).stop 2023-10-18T08:55:30.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.539+0000 7f4a3557d640 1 -- 172.21.15.40:0/3257097860 shutdown_connections 2023-10-18T08:55:30.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.539+0000 7f4a3557d640 1 --2- 172.21.15.40:0/3257097860 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a1c06af10 0x7f4a1c06d3d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.539+0000 7f4a3557d640 1 --2- 172.21.15.40:0/3257097860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a30141e60 0x7f4a3015aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.539+0000 7f4a3557d640 1 --2- 172.21.15.40:0/3257097860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a300b00e0 0x7f4a30153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.539+0000 7f4a3557d640 1 --2- 172.21.15.40:0/3257097860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a3006be20 0x7f4a30152f20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.539+0000 7f4a3557d640 1 -- 172.21.15.40:0/3257097860 >> 172.21.15.40:0/3257097860 conn(0x7f4a300aede0 msgr2=0x7f4a300cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:30.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.539+0000 7f8a1b59e640 1 -- 172.21.15.40:0/1761904080 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8a140ceb30 con 0x7f8a140e3130 2023-10-18T08:55:30.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.539+0000 7f4a3557d640 1 -- 172.21.15.40:0/3257097860 shutdown_connections 2023-10-18T08:55:30.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.539+0000 7f4a3557d640 1 -- 172.21.15.40:0/3257097860 wait complete. 2023-10-18T08:55:30.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.539+0000 7f8a0affd640 1 -- 172.21.15.40:0/1761904080 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8a1c097020 con 0x7f8a140e3130 2023-10-18T08:55:30.545 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-18T08:55:30.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.542+0000 7f8a1b59e640 1 -- 172.21.15.40:0/1761904080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f89fc06ae40 msgr2=0x7f89fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.542+0000 7f8a1b59e640 1 --2- 172.21.15.40:0/1761904080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f89fc06ae40 0x7f89fc06d300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c036000 tx=0x7f8a0c004730 comp rx=0 tx=0).stop 2023-10-18T08:55:30.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.542+0000 7f8a1b59e640 1 -- 172.21.15.40:0/1761904080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a140e3130 msgr2=0x7f8a14048bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.542+0000 7f8a1b59e640 1 --2- 172.21.15.40:0/1761904080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a140e3130 0x7f8a14048bd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c063080 tx=0x7f8a1c073940 comp rx=0 tx=0).stop 2023-10-18T08:55:30.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.542+0000 7f8a1b59e640 1 -- 172.21.15.40:0/1761904080 shutdown_connections 2023-10-18T08:55:30.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.542+0000 7f8a1b59e640 1 --2- 172.21.15.40:0/1761904080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f89fc06ae40 0x7f89fc06d300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.542+0000 7f8a1b59e640 1 --2- 172.21.15.40:0/1761904080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a140e3130 0x7f8a14048bd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.542+0000 7f8a1b59e640 1 --2- 172.21.15.40:0/1761904080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a14053c80 0x7f8a14048690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.542+0000 7f8a1b59e640 1 --2- 172.21.15.40:0/1761904080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a1403c070 0x7f8a140e4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.542+0000 7f8a1b59e640 1 -- 172.21.15.40:0/1761904080 >> 172.21.15.40:0/1761904080 conn(0x7f8a14058eb0 msgr2=0x7f8a140e6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:30.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.542+0000 7f8a1b59e640 1 -- 172.21.15.40:0/1761904080 shutdown_connections 2023-10-18T08:55:30.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.542+0000 7f8a1b59e640 1 -- 172.21.15.40:0/1761904080 wait complete. 2023-10-18T08:55:30.558 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-10-18T08:55:30.559 DEBUG:teuthology.parallel:result is None 2023-10-18T08:55:30.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-10-18T08:55:30.561 DEBUG:teuthology.parallel:result is None 2023-10-18T08:55:30.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.586+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2652962573 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb3f809d250 con 0x7fb3f815ed60 2023-10-18T08:55:30.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.586+0000 7fda0bfff640 1 -- 172.21.15.40:0/1104074969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fda0c073570 con 0x7fda0c15ed60 2023-10-18T08:55:30.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.587+0000 7fb3f4ff9640 1 -- 172.21.15.40:0/2652962573 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb3f809d250 con 0x7fb3f815ed60 2023-10-18T08:55:30.591 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-18T08:55:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.587+0000 7fda037fe640 1 -- 172.21.15.40:0/1104074969 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd9fc017070 con 0x7fda0c15ed60 2023-10-18T08:55:30.591 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-18T08:55:30.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.589+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2652962573 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3e406ae40 msgr2=0x7fb3e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.589+0000 7fb3fd4df640 1 --2- 172.21.15.40:0/2652962573 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3e406ae40 0x7fb3e406d300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80661a0 tx=0x7fb3e8004730 comp rx=0 tx=0).stop 2023-10-18T08:55:30.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.589+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2652962573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f815ed60 msgr2=0x7fb3f814e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.589+0000 7fb3fd4df640 1 --2- 172.21.15.40:0/2652962573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f815ed60 0x7fb3f814e530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000ef00 tx=0x7fb3f000c740 comp rx=0 tx=0).stop 2023-10-18T08:55:30.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.589+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2652962573 shutdown_connections 2023-10-18T08:55:30.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.589+0000 7fb3fd4df640 1 --2- 172.21.15.40:0/2652962573 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3e406ae40 0x7fb3e406d300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.589+0000 7fb3fd4df640 1 --2- 172.21.15.40:0/2652962573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3f815ed60 0x7fb3f814e530 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.589+0000 7fb3fd4df640 1 --2- 172.21.15.40:0/2652962573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3f80cdbc0 0x7fb3f809de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.589+0000 7fb3fd4df640 1 --2- 172.21.15.40:0/2652962573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3f80b0dd0 0x7fb3f809d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.589+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2652962573 >> 172.21.15.40:0/2652962573 conn(0x7fb3f809a580 msgr2=0x7fb3f815c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.590+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2652962573 shutdown_connections 2023-10-18T08:55:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.590+0000 7fb3fd4df640 1 -- 172.21.15.40:0/2652962573 wait complete. 2023-10-18T08:55:30.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.590+0000 7fda0bfff640 1 -- 172.21.15.40:0/1104074969 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd9ec06aec0 msgr2=0x7fd9ec06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.590+0000 7fda0bfff640 1 --2- 172.21.15.40:0/1104074969 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd9ec06aec0 0x7fd9ec06d380 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fda04000f80 tx=0x7fda04004ab0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.590+0000 7fda0bfff640 1 -- 172.21.15.40:0/1104074969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c15ed60 msgr2=0x7fda0c1581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.590+0000 7fda0bfff640 1 --2- 172.21.15.40:0/1104074969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c15ed60 0x7fda0c1581f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00c3c0 tx=0x7fd9fc00c880 comp rx=0 tx=0).stop 2023-10-18T08:55:30.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.590+0000 7fda0bfff640 1 -- 172.21.15.40:0/1104074969 shutdown_connections 2023-10-18T08:55:30.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.590+0000 7fda0bfff640 1 --2- 172.21.15.40:0/1104074969 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd9ec06aec0 0x7fd9ec06d380 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.590+0000 7fda0bfff640 1 --2- 172.21.15.40:0/1104074969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda0c15ed60 0x7fda0c1581f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.590+0000 7fda0bfff640 1 --2- 172.21.15.40:0/1104074969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda0c0cdbc0 0x7fda0c0b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.590+0000 7fda0bfff640 1 --2- 172.21.15.40:0/1104074969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda0c0b0dd0 0x7fda0c0c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.590+0000 7fda0bfff640 1 -- 172.21.15.40:0/1104074969 >> 172.21.15.40:0/1104074969 conn(0x7fda0c09a580 msgr2=0x7fda0c15c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:30.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.590+0000 7fda0bfff640 1 -- 172.21.15.40:0/1104074969 shutdown_connections 2023-10-18T08:55:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.590+0000 7fda0bfff640 1 -- 172.21.15.40:0/1104074969 wait complete. 2023-10-18T08:55:30.609 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-10-18T08:55:30.610 DEBUG:teuthology.parallel:result is None 2023-10-18T08:55:30.610 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2023-10-18T08:55:30.611 DEBUG:teuthology.parallel:result is None 2023-10-18T08:55:30.611 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-18T08:55:30.611 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:55:30.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.870+0000 7f0aa9287640 1 -- 172.21.15.40:0/119554813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa40b00e0 msgr2=0x7f0aa40b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.870+0000 7f0aa9287640 1 --2- 172.21.15.40:0/119554813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa40b00e0 0x7f0aa40b16c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009830 tx=0x7f0a9802bd50 comp rx=0 tx=0).stop 2023-10-18T08:55:30.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.870+0000 7f0aa9287640 1 -- 172.21.15.40:0/119554813 shutdown_connections 2023-10-18T08:55:30.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.870+0000 7f0aa9287640 1 --2- 172.21.15.40:0/119554813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0aa40b0550 0x7f0aa40b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.870+0000 7f0aa9287640 1 --2- 172.21.15.40:0/119554813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa40b00e0 0x7f0aa40b16c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.870+0000 7f0aa9287640 1 -- 172.21.15.40:0/119554813 >> 172.21.15.40:0/119554813 conn(0x7f0aa406cee0 msgr2=0x7f0aa4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:30.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.871+0000 7f0aa9287640 1 -- 172.21.15.40:0/119554813 shutdown_connections 2023-10-18T08:55:30.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.871+0000 7f0aa9287640 1 -- 172.21.15.40:0/119554813 wait complete. 2023-10-18T08:55:30.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.872+0000 7f0aa9287640 1 Processor -- start 2023-10-18T08:55:30.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.872+0000 7f0aa9287640 1 -- start start 2023-10-18T08:55:30.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.873+0000 7f0aa9287640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa40b00e0 0x7f0aa4149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.873+0000 7f0aa9287640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa40b0550 0x7f0aa4157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.873+0000 7f0aa9287640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0aa40cd610 0x7f0aa4157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.873+0000 7f0aa9287640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa41592b0 con 0x7f0aa40b00e0 2023-10-18T08:55:30.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.873+0000 7f0aa9287640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa414d610 con 0x7f0aa40b0550 2023-10-18T08:55:30.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.873+0000 7f0aa9287640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa4159500 con 0x7f0aa40cd610 2023-10-18T08:55:30.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.874+0000 7f0aa2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa40b00e0 0x7f0aa4149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.874+0000 7f0aa2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa40b00e0 0x7f0aa4149c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48174/0 (socket says 172.21.15.40:48174) 2023-10-18T08:55:30.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.874+0000 7f0aa2ffd640 1 -- 172.21.15.40:0/518030242 learned_addr learned my addr 172.21.15.40:0/518030242 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:30.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.874+0000 7f0aa37fe640 1 --2- 172.21.15.40:0/518030242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0aa40cd610 0x7f0aa4157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.874+0000 7f0aa27fc640 1 --2- 172.21.15.40:0/518030242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa40b0550 0x7f0aa4157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.874+0000 7f0aa2ffd640 1 -- 172.21.15.40:0/518030242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0aa40cd610 msgr2=0x7f0aa4157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.874+0000 7f0aa2ffd640 1 --2- 172.21.15.40:0/518030242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0aa40cd610 0x7f0aa4157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.874+0000 7f0aa2ffd640 1 -- 172.21.15.40:0/518030242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa40b0550 msgr2=0x7f0aa4157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:30.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.874+0000 7f0aa2ffd640 1 --2- 172.21.15.40:0/518030242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa40b0550 0x7f0aa4157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:30.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.874+0000 7f0aa2ffd640 1 -- 172.21.15.40:0/518030242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a98009490 con 0x7f0aa40b00e0 2023-10-18T08:55:30.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.874+0000 7f0aa37fe640 1 --2- 172.21.15.40:0/518030242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0aa40cd610 0x7f0aa4157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:30.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.874+0000 7f0aa27fc640 1 --2- 172.21.15.40:0/518030242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa40b0550 0x7f0aa4157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:30.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.874+0000 7f0aa2ffd640 1 --2- 172.21.15.40:0/518030242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa40b00e0 0x7f0aa4149c10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400c410 tx=0x7f0a9400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.875+0000 7f0a8b7fe640 1 -- 172.21.15.40:0/518030242 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a940035f0 con 0x7f0aa40b00e0 2023-10-18T08:55:30.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.875+0000 7f0a8b7fe640 1 -- 172.21.15.40:0/518030242 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0a94003770 con 0x7f0aa40b00e0 2023-10-18T08:55:30.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.875+0000 7f0a8b7fe640 1 -- 172.21.15.40:0/518030242 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94003b80 con 0x7f0aa40b00e0 2023-10-18T08:55:30.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.875+0000 7f0aa9287640 1 -- 172.21.15.40:0/518030242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa41597e0 con 0x7f0aa40b00e0 2023-10-18T08:55:30.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.875+0000 7f0aa9287640 1 -- 172.21.15.40:0/518030242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa414a150 con 0x7f0aa40b00e0 2023-10-18T08:55:30.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.876+0000 7f0aa9287640 1 -- 172.21.15.40:0/518030242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa4159d00 con 0x7f0aa40b00e0 2023-10-18T08:55:30.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.877+0000 7f0a8b7fe640 1 -- 172.21.15.40:0/518030242 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0a94003ce0 con 0x7f0aa40b00e0 2023-10-18T08:55:30.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.878+0000 7f0a8b7fe640 1 --2- 172.21.15.40:0/518030242 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a9006ae40 0x7f0a9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:30.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.878+0000 7f0a8b7fe640 1 -- 172.21.15.40:0/518030242 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a94088d30 con 0x7f0aa40b00e0 2023-10-18T08:55:30.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.878+0000 7f0aa27fc640 1 --2- 172.21.15.40:0/518030242 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a9006ae40 0x7f0a9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:30.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.880+0000 7f0aa27fc640 1 --2- 172.21.15.40:0/518030242 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a9006ae40 0x7f0a9006d300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0a98036000 tx=0x7f0a98004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:30.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:30.882+0000 7f0a8b7fe640 1 -- 172.21.15.40:0/518030242 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0aa4159d00 con 0x7f0aa40b00e0 2023-10-18T08:55:31.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.062+0000 7f0aa9287640 1 -- 172.21.15.40:0/518030242 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0aa4159d00 con 0x7f0a9006ae40 2023-10-18T08:55:31.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.083+0000 7f0a8b7fe640 1 -- 172.21.15.40:0/518030242 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259981 (secure 0 0 0) 0x7f0aa4159d00 con 0x7f0a9006ae40 2023-10-18T08:55:31.089 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:31.091 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:55:31.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.088+0000 7f0aa9287640 1 -- 172.21.15.40:0/518030242 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a9006ae40 msgr2=0x7f0a9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.088+0000 7f0aa9287640 1 --2- 172.21.15.40:0/518030242 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a9006ae40 0x7f0a9006d300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0a98036000 tx=0x7f0a98004730 comp rx=0 tx=0).stop 2023-10-18T08:55:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.089+0000 7f0aa9287640 1 -- 172.21.15.40:0/518030242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa40b00e0 msgr2=0x7f0aa4149c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.089+0000 7f0aa9287640 1 --2- 172.21.15.40:0/518030242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa40b00e0 0x7f0aa4149c10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400c410 tx=0x7f0a9400c8d0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.089+0000 7f0aa9287640 1 -- 172.21.15.40:0/518030242 shutdown_connections 2023-10-18T08:55:31.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.089+0000 7f0aa9287640 1 --2- 172.21.15.40:0/518030242 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a9006ae40 0x7f0a9006d300 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.089+0000 7f0aa9287640 1 --2- 172.21.15.40:0/518030242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0aa40cd610 0x7f0aa4157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.089+0000 7f0aa9287640 1 --2- 172.21.15.40:0/518030242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa40b0550 0x7f0aa4157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.089+0000 7f0aa9287640 1 --2- 172.21.15.40:0/518030242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa40b00e0 0x7f0aa4149c10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.089+0000 7f0aa9287640 1 -- 172.21.15.40:0/518030242 >> 172.21.15.40:0/518030242 conn(0x7f0aa406cee0 msgr2=0x7f0aa40af140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.089+0000 7f0aa9287640 1 -- 172.21.15.40:0/518030242 shutdown_connections 2023-10-18T08:55:31.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.089+0000 7f0aa9287640 1 -- 172.21.15.40:0/518030242 wait complete. 2023-10-18T08:55:31.110 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-18T08:55:30.899225+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560792,"statfs":{"total":1600000000,"available":1598251597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004891"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282778+0000","last_change":"2023-10-18T08:55:15.307275+0000","last_active":"2023-10-18T08:55:16.282778+0000","last_peered":"2023-10-18T08:55:16.282778+0000","last_clean":"2023-10-18T08:55:16.282778+0000","last_became_active":"2023-10-18T08:55:15.304759+0000","last_became_peered":"2023-10-18T08:55:15.304759+0000","last_unstale":"2023-10-18T08:55:16.282778+0000","last_undegraded":"2023-10-18T08:55:16.282778+0000","last_fullsized":"2023-10-18T08:55:16.282778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:41.755639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281449+0000","last_change":"2023-10-18T08:55:15.306465+0000","last_active":"2023-10-18T08:55:16.281449+0000","last_peered":"2023-10-18T08:55:16.281449+0000","last_clean":"2023-10-18T08:55:16.281449+0000","last_became_active":"2023-10-18T08:55:15.305909+0000","last_became_peered":"2023-10-18T08:55:15.305909+0000","last_unstale":"2023-10-18T08:55:16.281449+0000","last_undegraded":"2023-10-18T08:55:16.281449+0000","last_fullsized":"2023-10-18T08:55:16.281449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:02.173932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688851+0000","last_change":"2023-10-18T08:55:15.302350+0000","last_active":"2023-10-18T08:55:16.688851+0000","last_peered":"2023-10-18T08:55:16.688851+0000","last_clean":"2023-10-18T08:55:16.688851+0000","last_became_active":"2023-10-18T08:55:15.302090+0000","last_became_peered":"2023-10-18T08:55:15.302090+0000","last_unstale":"2023-10-18T08:55:16.688851+0000","last_undegraded":"2023-10-18T08:55:16.688851+0000","last_fullsized":"2023-10-18T08:55:16.688851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:19.879235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688836+0000","last_change":"2023-10-18T08:55:15.307679+0000","last_active":"2023-10-18T08:55:16.688836+0000","last_peered":"2023-10-18T08:55:16.688836+0000","last_clean":"2023-10-18T08:55:16.688836+0000","last_became_active":"2023-10-18T08:55:15.306999+0000","last_became_peered":"2023-10-18T08:55:15.306999+0000","last_unstale":"2023-10-18T08:55:16.688836+0000","last_undegraded":"2023-10-18T08:55:16.688836+0000","last_fullsized":"2023-10-18T08:55:16.688836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282830+0000","last_change":"2023-10-18T08:55:15.293552+0000","last_active":"2023-10-18T08:55:16.282830+0000","last_peered":"2023-10-18T08:55:16.282830+0000","last_clean":"2023-10-18T08:55:16.282830+0000","last_became_active":"2023-10-18T08:55:15.292895+0000","last_became_peered":"2023-10-18T08:55:15.292895+0000","last_unstale":"2023-10-18T08:55:16.282830+0000","last_undegraded":"2023-10-18T08:55:16.282830+0000","last_fullsized":"2023-10-18T08:55:16.282830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:07:58.310623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691708+0000","last_change":"2023-10-18T08:55:15.319351+0000","last_active":"2023-10-18T08:55:16.691708+0000","last_peered":"2023-10-18T08:55:16.691708+0000","last_clean":"2023-10-18T08:55:16.691708+0000","last_became_active":"2023-10-18T08:55:15.317663+0000","last_became_peered":"2023-10-18T08:55:15.317663+0000","last_unstale":"2023-10-18T08:55:16.691708+0000","last_undegraded":"2023-10-18T08:55:16.691708+0000","last_fullsized":"2023-10-18T08:55:16.691708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:10.917539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282757+0000","last_change":"2023-10-18T08:55:15.298227+0000","last_active":"2023-10-18T08:55:16.282757+0000","last_peered":"2023-10-18T08:55:16.282757+0000","last_clean":"2023-10-18T08:55:16.282757+0000","last_became_active":"2023-10-18T08:55:15.297630+0000","last_became_peered":"2023-10-18T08:55:15.297630+0000","last_unstale":"2023-10-18T08:55:16.282757+0000","last_undegraded":"2023-10-18T08:55:16.282757+0000","last_fullsized":"2023-10-18T08:55:16.282757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:04.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.279841+0000","last_change":"2023-10-18T08:55:15.307773+0000","last_active":"2023-10-18T08:55:16.279841+0000","last_peered":"2023-10-18T08:55:16.279841+0000","last_clean":"2023-10-18T08:55:16.279841+0000","last_became_active":"2023-10-18T08:55:15.307252+0000","last_became_peered":"2023-10-18T08:55:15.307252+0000","last_unstale":"2023-10-18T08:55:16.279841+0000","last_undegraded":"2023-10-18T08:55:16.279841+0000","last_fullsized":"2023-10-18T08:55:16.279841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:19.879235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282164+0000","last_change":"2023-10-18T08:55:15.308028+0000","last_active":"2023-10-18T08:55:16.282164+0000","last_peered":"2023-10-18T08:55:16.282164+0000","last_clean":"2023-10-18T08:55:16.282164+0000","last_became_active":"2023-10-18T08:55:15.307786+0000","last_became_peered":"2023-10-18T08:55:15.307786+0000","last_unstale":"2023-10-18T08:55:16.282164+0000","last_undegraded":"2023-10-18T08:55:16.282164+0000","last_fullsized":"2023-10-18T08:55:16.282164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:31.888781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691934+0000","last_change":"2023-10-18T08:55:15.307031+0000","last_active":"2023-10-18T08:55:16.691934+0000","last_peered":"2023-10-18T08:55:16.691934+0000","last_clean":"2023-10-18T08:55:16.691934+0000","last_became_active":"2023-10-18T08:55:15.306786+0000","last_became_peered":"2023-10-18T08:55:15.306786+0000","last_unstale":"2023-10-18T08:55:16.691934+0000","last_undegraded":"2023-10-18T08:55:16.691934+0000","last_fullsized":"2023-10-18T08:55:16.691934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:36.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689019+0000","last_change":"2023-10-18T08:55:15.321350+0000","last_active":"2023-10-18T08:55:16.689019+0000","last_peered":"2023-10-18T08:55:16.689019+0000","last_clean":"2023-10-18T08:55:16.689019+0000","last_became_active":"2023-10-18T08:55:15.321114+0000","last_became_peered":"2023-10-18T08:55:15.321114+0000","last_unstale":"2023-10-18T08:55:16.689019+0000","last_undegraded":"2023-10-18T08:55:16.689019+0000","last_fullsized":"2023-10-18T08:55:16.689019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:02.173932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689997+0000","last_change":"2023-10-18T08:55:15.307879+0000","last_active":"2023-10-18T08:55:16.689997+0000","last_peered":"2023-10-18T08:55:16.689997+0000","last_clean":"2023-10-18T08:55:16.689997+0000","last_became_active":"2023-10-18T08:55:15.307599+0000","last_became_peered":"2023-10-18T08:55:15.307599+0000","last_unstale":"2023-10-18T08:55:16.689997+0000","last_undegraded":"2023-10-18T08:55:16.689997+0000","last_fullsized":"2023-10-18T08:55:16.689997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:02.474966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280131+0000","last_change":"2023-10-18T08:55:15.306959+0000","last_active":"2023-10-18T08:55:16.280131+0000","last_peered":"2023-10-18T08:55:16.280131+0000","last_clean":"2023-10-18T08:55:16.280131+0000","last_became_active":"2023-10-18T08:55:15.306003+0000","last_became_peered":"2023-10-18T08:55:15.306003+0000","last_unstale":"2023-10-18T08:55:16.280131+0000","last_undegraded":"2023-10-18T08:55:16.280131+0000","last_fullsized":"2023-10-18T08:55:16.280131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:06.219128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688286+0000","last_change":"2023-10-18T08:55:15.302306+0000","last_active":"2023-10-18T08:55:16.688286+0000","last_peered":"2023-10-18T08:55:16.688286+0000","last_clean":"2023-10-18T08:55:16.688286+0000","last_became_active":"2023-10-18T08:55:15.302027+0000","last_became_peered":"2023-10-18T08:55:15.302027+0000","last_unstale":"2023-10-18T08:55:16.688286+0000","last_undegraded":"2023-10-18T08:55:16.688286+0000","last_fullsized":"2023-10-18T08:55:16.688286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:45.159195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282045+0000","last_change":"2023-10-18T08:55:15.316075+0000","last_active":"2023-10-18T08:55:16.282045+0000","last_peered":"2023-10-18T08:55:16.282045+0000","last_clean":"2023-10-18T08:55:16.282045+0000","last_became_active":"2023-10-18T08:55:15.315569+0000","last_became_peered":"2023-10-18T08:55:15.315569+0000","last_unstale":"2023-10-18T08:55:16.282045+0000","last_undegraded":"2023-10-18T08:55:16.282045+0000","last_fullsized":"2023-10-18T08:55:16.282045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691359+0000","last_change":"2023-10-18T08:55:15.320240+0000","last_active":"2023-10-18T08:55:16.691359+0000","last_peered":"2023-10-18T08:55:16.691359+0000","last_clean":"2023-10-18T08:55:16.691359+0000","last_became_active":"2023-10-18T08:55:15.319558+0000","last_became_peered":"2023-10-18T08:55:15.319558+0000","last_unstale":"2023-10-18T08:55:16.691359+0000","last_undegraded":"2023-10-18T08:55:16.691359+0000","last_fullsized":"2023-10-18T08:55:16.691359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:10.355168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690434+0000","last_change":"2023-10-18T08:55:15.315655+0000","last_active":"2023-10-18T08:55:16.690434+0000","last_peered":"2023-10-18T08:55:16.690434+0000","last_clean":"2023-10-18T08:55:16.690434+0000","last_became_active":"2023-10-18T08:55:15.311919+0000","last_became_peered":"2023-10-18T08:55:15.311919+0000","last_unstale":"2023-10-18T08:55:16.690434+0000","last_undegraded":"2023-10-18T08:55:16.690434+0000","last_fullsized":"2023-10-18T08:55:16.690434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:23:50.828764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.687786+0000","last_change":"2023-10-18T08:55:15.306321+0000","last_active":"2023-10-18T08:55:16.687786+0000","last_peered":"2023-10-18T08:55:16.687786+0000","last_clean":"2023-10-18T08:55:16.687786+0000","last_became_active":"2023-10-18T08:55:15.305696+0000","last_became_peered":"2023-10-18T08:55:15.305696+0000","last_unstale":"2023-10-18T08:55:16.687786+0000","last_undegraded":"2023-10-18T08:55:16.687786+0000","last_fullsized":"2023-10-18T08:55:16.687786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:05.190948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282965+0000","last_change":"2023-10-18T08:55:15.294264+0000","last_active":"2023-10-18T08:55:16.282965+0000","last_peered":"2023-10-18T08:55:16.282965+0000","last_clean":"2023-10-18T08:55:16.282965+0000","last_became_active":"2023-10-18T08:55:15.293654+0000","last_became_peered":"2023-10-18T08:55:15.293654+0000","last_unstale":"2023-10-18T08:55:16.282965+0000","last_undegraded":"2023-10-18T08:55:16.282965+0000","last_fullsized":"2023-10-18T08:55:16.282965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:26.310762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.692252+0000","last_change":"2023-10-18T08:55:15.315803+0000","last_active":"2023-10-18T08:55:16.692252+0000","last_peered":"2023-10-18T08:55:16.692252+0000","last_clean":"2023-10-18T08:55:16.692252+0000","last_became_active":"2023-10-18T08:55:15.315544+0000","last_became_peered":"2023-10-18T08:55:15.315544+0000","last_unstale":"2023-10-18T08:55:16.692252+0000","last_undegraded":"2023-10-18T08:55:16.692252+0000","last_fullsized":"2023-10-18T08:55:16.692252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689490+0000","last_change":"2023-10-18T08:55:15.308154+0000","last_active":"2023-10-18T08:55:16.689490+0000","last_peered":"2023-10-18T08:55:16.689490+0000","last_clean":"2023-10-18T08:55:16.689490+0000","last_became_active":"2023-10-18T08:55:15.307592+0000","last_became_peered":"2023-10-18T08:55:15.307592+0000","last_unstale":"2023-10-18T08:55:16.689490+0000","last_undegraded":"2023-10-18T08:55:16.689490+0000","last_fullsized":"2023-10-18T08:55:16.689490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:06.219128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691182+0000","last_change":"2023-10-18T08:55:15.312342+0000","last_active":"2023-10-18T08:55:16.691182+0000","last_peered":"2023-10-18T08:55:16.691182+0000","last_clean":"2023-10-18T08:55:16.691182+0000","last_became_active":"2023-10-18T08:55:15.311286+0000","last_became_peered":"2023-10-18T08:55:15.311286+0000","last_unstale":"2023-10-18T08:55:16.691182+0000","last_undegraded":"2023-10-18T08:55:16.691182+0000","last_fullsized":"2023-10-18T08:55:16.691182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:07:58.310623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280368+0000","last_change":"2023-10-18T08:55:15.307636+0000","last_active":"2023-10-18T08:55:16.280368+0000","last_peered":"2023-10-18T08:55:16.280368+0000","last_clean":"2023-10-18T08:55:16.280368+0000","last_became_active":"2023-10-18T08:55:15.306703+0000","last_became_peered":"2023-10-18T08:55:15.306703+0000","last_unstale":"2023-10-18T08:55:16.280368+0000","last_undegraded":"2023-10-18T08:55:16.280368+0000","last_fullsized":"2023-10-18T08:55:16.280368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:36.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689173+0000","last_change":"2023-10-18T08:55:15.307528+0000","last_active":"2023-10-18T08:55:16.689173+0000","last_peered":"2023-10-18T08:55:16.689173+0000","last_clean":"2023-10-18T08:55:16.689173+0000","last_became_active":"2023-10-18T08:55:15.306757+0000","last_became_peered":"2023-10-18T08:55:15.306757+0000","last_unstale":"2023-10-18T08:55:16.689173+0000","last_undegraded":"2023-10-18T08:55:16.689173+0000","last_fullsized":"2023-10-18T08:55:16.689173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:31.888781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280839+0000","last_change":"2023-10-18T08:55:15.316136+0000","last_active":"2023-10-18T08:55:16.280839+0000","last_peered":"2023-10-18T08:55:16.280839+0000","last_clean":"2023-10-18T08:55:16.280839+0000","last_became_active":"2023-10-18T08:55:15.315888+0000","last_became_peered":"2023-10-18T08:55:15.315888+0000","last_unstale":"2023-10-18T08:55:16.280839+0000","last_undegraded":"2023-10-18T08:55:16.280839+0000","last_fullsized":"2023-10-18T08:55:16.280839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:23:50.828764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689385+0000","last_change":"2023-10-18T08:55:15.309215+0000","last_active":"2023-10-18T08:55:16.689385+0000","last_peered":"2023-10-18T08:55:16.689385+0000","last_clean":"2023-10-18T08:55:16.689385+0000","last_became_active":"2023-10-18T08:55:15.308858+0000","last_became_peered":"2023-10-18T08:55:15.308858+0000","last_unstale":"2023-10-18T08:55:16.689385+0000","last_undegraded":"2023-10-18T08:55:16.689385+0000","last_fullsized":"2023-10-18T08:55:16.689385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:06.219128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281659+0000","last_change":"2023-10-18T08:55:15.308660+0000","last_active":"2023-10-18T08:55:16.281659+0000","last_peered":"2023-10-18T08:55:16.281659+0000","last_clean":"2023-10-18T08:55:16.281659+0000","last_became_active":"2023-10-18T08:55:15.306230+0000","last_became_peered":"2023-10-18T08:55:15.306230+0000","last_unstale":"2023-10-18T08:55:16.281659+0000","last_undegraded":"2023-10-18T08:55:16.281659+0000","last_fullsized":"2023-10-18T08:55:16.281659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:06.219128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280534+0000","last_change":"2023-10-18T08:55:15.307486+0000","last_active":"2023-10-18T08:55:16.280534+0000","last_peered":"2023-10-18T08:55:16.280534+0000","last_clean":"2023-10-18T08:55:16.280534+0000","last_became_active":"2023-10-18T08:55:15.306357+0000","last_became_peered":"2023-10-18T08:55:15.306357+0000","last_unstale":"2023-10-18T08:55:16.280534+0000","last_undegraded":"2023-10-18T08:55:16.280534+0000","last_fullsized":"2023-10-18T08:55:16.280534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:31.888781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688669+0000","last_change":"2023-10-18T08:55:15.300645+0000","last_active":"2023-10-18T08:55:16.688669+0000","last_peered":"2023-10-18T08:55:16.688669+0000","last_clean":"2023-10-18T08:55:16.688669+0000","last_became_active":"2023-10-18T08:55:15.299795+0000","last_became_peered":"2023-10-18T08:55:15.299795+0000","last_unstale":"2023-10-18T08:55:16.688669+0000","last_undegraded":"2023-10-18T08:55:16.688669+0000","last_fullsized":"2023-10-18T08:55:16.688669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:21.842439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690857+0000","last_change":"2023-10-18T08:55:15.305929+0000","last_active":"2023-10-18T08:55:16.690857+0000","last_peered":"2023-10-18T08:55:16.690857+0000","last_clean":"2023-10-18T08:55:16.690857+0000","last_became_active":"2023-10-18T08:55:15.301322+0000","last_became_peered":"2023-10-18T08:55:15.301322+0000","last_unstale":"2023-10-18T08:55:16.690857+0000","last_undegraded":"2023-10-18T08:55:16.690857+0000","last_fullsized":"2023-10-18T08:55:16.690857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:41.755639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.687612+0000","last_change":"2023-10-18T08:55:15.320868+0000","last_active":"2023-10-18T08:55:16.687612+0000","last_peered":"2023-10-18T08:55:16.687612+0000","last_clean":"2023-10-18T08:55:16.687612+0000","last_became_active":"2023-10-18T08:55:15.320606+0000","last_became_peered":"2023-10-18T08:55:15.320606+0000","last_unstale":"2023-10-18T08:55:16.687612+0000","last_undegraded":"2023-10-18T08:55:16.687612+0000","last_fullsized":"2023-10-18T08:55:16.687612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:04.364364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690001+0000","last_change":"2023-10-18T08:55:15.314269+0000","last_active":"2023-10-18T08:55:16.690001+0000","last_peered":"2023-10-18T08:55:16.690001+0000","last_clean":"2023-10-18T08:55:16.690001+0000","last_became_active":"2023-10-18T08:55:15.314011+0000","last_became_peered":"2023-10-18T08:55:15.314011+0000","last_unstale":"2023-10-18T08:55:16.690001+0000","last_undegraded":"2023-10-18T08:55:16.690001+0000","last_fullsized":"2023-10-18T08:55:16.690001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:28.148282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689559+0000","last_change":"2023-10-18T08:55:15.306134+0000","last_active":"2023-10-18T08:55:16.689559+0000","last_peered":"2023-10-18T08:55:16.689559+0000","last_clean":"2023-10-18T08:55:16.689559+0000","last_became_active":"2023-10-18T08:55:15.305389+0000","last_became_peered":"2023-10-18T08:55:15.305389+0000","last_unstale":"2023-10-18T08:55:16.689559+0000","last_undegraded":"2023-10-18T08:55:16.689559+0000","last_fullsized":"2023-10-18T08:55:16.689559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:32.273116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.327693+0000","last_change":"2023-10-18T08:55:14.290336+0000","last_active":"2023-10-18T08:55:15.327693+0000","last_peered":"2023-10-18T08:55:15.327693+0000","last_clean":"2023-10-18T08:55:15.327693+0000","last_became_active":"2023-10-18T08:55:14.290051+0000","last_became_peered":"2023-10-18T08:55:14.290051+0000","last_unstale":"2023-10-18T08:55:15.327693+0000","last_undegraded":"2023-10-18T08:55:15.327693+0000","last_fullsized":"2023-10-18T08:55:15.327693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:38.968705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689230+0000","last_change":"2023-10-18T08:55:15.302202+0000","last_active":"2023-10-18T08:55:16.689230+0000","last_peered":"2023-10-18T08:55:16.689230+0000","last_clean":"2023-10-18T08:55:16.689230+0000","last_became_active":"2023-10-18T08:55:15.301534+0000","last_became_peered":"2023-10-18T08:55:15.301534+0000","last_unstale":"2023-10-18T08:55:16.689230+0000","last_undegraded":"2023-10-18T08:55:16.689230+0000","last_fullsized":"2023-10-18T08:55:16.689230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:27.650596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.326541+0000","last_change":"2023-10-18T08:55:14.296292+0000","last_active":"2023-10-18T08:55:15.326541+0000","last_peered":"2023-10-18T08:55:15.326541+0000","last_clean":"2023-10-18T08:55:15.326541+0000","last_became_active":"2023-10-18T08:55:14.295937+0000","last_became_peered":"2023-10-18T08:55:14.295937+0000","last_unstale":"2023-10-18T08:55:15.326541+0000","last_undegraded":"2023-10-18T08:55:15.326541+0000","last_fullsized":"2023-10-18T08:55:15.326541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:38.968705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.280268+0000","last_change":"2023-10-18T08:55:14.293089+0000","last_active":"2023-10-18T08:55:15.280268+0000","last_peered":"2023-10-18T08:55:15.280268+0000","last_clean":"2023-10-18T08:55:15.280268+0000","last_became_active":"2023-10-18T08:55:14.292273+0000","last_became_peered":"2023-10-18T08:55:14.292273+0000","last_unstale":"2023-10-18T08:55:15.280268+0000","last_undegraded":"2023-10-18T08:55:15.280268+0000","last_fullsized":"2023-10-18T08:55:15.280268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:03.359656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281004+0000","last_change":"2023-10-18T08:55:15.297969+0000","last_active":"2023-10-18T08:55:16.281004+0000","last_peered":"2023-10-18T08:55:16.281004+0000","last_clean":"2023-10-18T08:55:16.281004+0000","last_became_active":"2023-10-18T08:55:15.297730+0000","last_became_peered":"2023-10-18T08:55:15.297730+0000","last_unstale":"2023-10-18T08:55:16.281004+0000","last_undegraded":"2023-10-18T08:55:16.281004+0000","last_fullsized":"2023-10-18T08:55:16.281004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:04.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.301050+0000","last_change":"2023-10-18T08:55:14.299406+0000","last_active":"2023-10-18T08:55:15.301050+0000","last_peered":"2023-10-18T08:55:15.301050+0000","last_clean":"2023-10-18T08:55:15.301050+0000","last_became_active":"2023-10-18T08:55:14.299181+0000","last_became_peered":"2023-10-18T08:55:14.299181+0000","last_unstale":"2023-10-18T08:55:15.301050+0000","last_undegraded":"2023-10-18T08:55:15.301050+0000","last_fullsized":"2023-10-18T08:55:15.301050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:18.874528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281450+0000","last_change":"2023-10-18T08:55:15.309032+0000","last_active":"2023-10-18T08:55:16.281450+0000","last_peered":"2023-10-18T08:55:16.281450+0000","last_clean":"2023-10-18T08:55:16.281450+0000","last_became_active":"2023-10-18T08:55:15.308756+0000","last_became_peered":"2023-10-18T08:55:15.308756+0000","last_unstale":"2023-10-18T08:55:16.281450+0000","last_undegraded":"2023-10-18T08:55:16.281450+0000","last_fullsized":"2023-10-18T08:55:16.281450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:27.650596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.297946+0000","last_change":"2023-10-18T08:55:14.304174+0000","last_active":"2023-10-18T08:55:15.297946+0000","last_peered":"2023-10-18T08:55:15.297946+0000","last_clean":"2023-10-18T08:55:15.297946+0000","last_became_active":"2023-10-18T08:55:14.303968+0000","last_became_peered":"2023-10-18T08:55:14.303968+0000","last_unstale":"2023-10-18T08:55:15.297946+0000","last_undegraded":"2023-10-18T08:55:15.297946+0000","last_fullsized":"2023-10-18T08:55:15.297946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:04.186241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690815+0000","last_change":"2023-10-18T08:55:15.312484+0000","last_active":"2023-10-18T08:55:16.690815+0000","last_peered":"2023-10-18T08:55:16.690815+0000","last_clean":"2023-10-18T08:55:16.690815+0000","last_became_active":"2023-10-18T08:55:15.311626+0000","last_became_peered":"2023-10-18T08:55:15.311626+0000","last_unstale":"2023-10-18T08:55:16.690815+0000","last_undegraded":"2023-10-18T08:55:16.690815+0000","last_fullsized":"2023-10-18T08:55:16.690815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:04.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.300021+0000","last_change":"2023-10-18T08:55:14.302651+0000","last_active":"2023-10-18T08:55:15.300021+0000","last_peered":"2023-10-18T08:55:15.300021+0000","last_clean":"2023-10-18T08:55:15.300021+0000","last_became_active":"2023-10-18T08:55:14.302029+0000","last_became_peered":"2023-10-18T08:55:14.302029+0000","last_unstale":"2023-10-18T08:55:15.300021+0000","last_undegraded":"2023-10-18T08:55:15.300021+0000","last_fullsized":"2023-10-18T08:55:15.300021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:44.154488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689826+0000","last_change":"2023-10-18T08:55:15.322350+0000","last_active":"2023-10-18T08:55:16.689826+0000","last_peered":"2023-10-18T08:55:16.689826+0000","last_clean":"2023-10-18T08:55:16.689826+0000","last_became_active":"2023-10-18T08:55:15.322002+0000","last_became_peered":"2023-10-18T08:55:15.322002+0000","last_unstale":"2023-10-18T08:55:16.689826+0000","last_undegraded":"2023-10-18T08:55:16.689826+0000","last_fullsized":"2023-10-18T08:55:16.689826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:36.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.326702+0000","last_change":"2023-10-18T08:55:14.300549+0000","last_active":"2023-10-18T08:55:15.326702+0000","last_peered":"2023-10-18T08:55:15.326702+0000","last_clean":"2023-10-18T08:55:15.326702+0000","last_became_active":"2023-10-18T08:55:14.299990+0000","last_became_peered":"2023-10-18T08:55:14.299990+0000","last_unstale":"2023-10-18T08:55:15.326702+0000","last_undegraded":"2023-10-18T08:55:15.326702+0000","last_fullsized":"2023-10-18T08:55:15.326702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:37.505319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690659+0000","last_change":"2023-10-18T08:55:15.314324+0000","last_active":"2023-10-18T08:55:16.690659+0000","last_peered":"2023-10-18T08:55:16.690659+0000","last_clean":"2023-10-18T08:55:16.690659+0000","last_became_active":"2023-10-18T08:55:15.313890+0000","last_became_peered":"2023-10-18T08:55:15.313890+0000","last_unstale":"2023-10-18T08:55:16.690659+0000","last_undegraded":"2023-10-18T08:55:16.690659+0000","last_fullsized":"2023-10-18T08:55:16.690659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:05.190948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.329951+0000","last_change":"2023-10-18T08:55:14.292962+0000","last_active":"2023-10-18T08:55:15.329951+0000","last_peered":"2023-10-18T08:55:15.329951+0000","last_clean":"2023-10-18T08:55:15.329951+0000","last_became_active":"2023-10-18T08:55:14.292424+0000","last_became_peered":"2023-10-18T08:55:14.292424+0000","last_unstale":"2023-10-18T08:55:15.329951+0000","last_undegraded":"2023-10-18T08:55:15.329951+0000","last_fullsized":"2023-10-18T08:55:15.329951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:20.837731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282227+0000","last_change":"2023-10-18T08:55:15.296745+0000","last_active":"2023-10-18T08:55:16.282227+0000","last_peered":"2023-10-18T08:55:16.282227+0000","last_clean":"2023-10-18T08:55:16.282227+0000","last_became_active":"2023-10-18T08:55:15.296095+0000","last_became_peered":"2023-10-18T08:55:15.296095+0000","last_unstale":"2023-10-18T08:55:16.282227+0000","last_undegraded":"2023-10-18T08:55:16.282227+0000","last_fullsized":"2023-10-18T08:55:16.282227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.688019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.328191+0000","last_change":"2023-10-18T08:55:14.299318+0000","last_active":"2023-10-18T08:55:15.328191+0000","last_peered":"2023-10-18T08:55:15.328191+0000","last_clean":"2023-10-18T08:55:15.328191+0000","last_became_active":"2023-10-18T08:55:14.299069+0000","last_became_peered":"2023-10-18T08:55:14.299069+0000","last_unstale":"2023-10-18T08:55:15.328191+0000","last_undegraded":"2023-10-18T08:55:15.328191+0000","last_fullsized":"2023-10-18T08:55:15.328191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:03.359656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690146+0000","last_change":"2023-10-18T08:55:15.307886+0000","last_active":"2023-10-18T08:55:16.690146+0000","last_peered":"2023-10-18T08:55:16.690146+0000","last_clean":"2023-10-18T08:55:16.690146+0000","last_became_active":"2023-10-18T08:55:15.306465+0000","last_became_peered":"2023-10-18T08:55:15.306465+0000","last_unstale":"2023-10-18T08:55:16.690146+0000","last_undegraded":"2023-10-18T08:55:16.690146+0000","last_fullsized":"2023-10-18T08:55:16.690146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:31.888781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.316152+0000","last_change":"2023-10-18T08:55:14.305151+0000","last_active":"2023-10-18T08:55:15.316152+0000","last_peered":"2023-10-18T08:55:15.316152+0000","last_clean":"2023-10-18T08:55:15.316152+0000","last_became_active":"2023-10-18T08:55:14.303161+0000","last_became_peered":"2023-10-18T08:55:14.303161+0000","last_unstale":"2023-10-18T08:55:15.316152+0000","last_undegraded":"2023-10-18T08:55:15.316152+0000","last_fullsized":"2023-10-18T08:55:15.316152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:26.645889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.283192+0000","last_change":"2023-10-18T08:55:15.317138+0000","last_active":"2023-10-18T08:55:16.283192+0000","last_peered":"2023-10-18T08:55:16.283192+0000","last_clean":"2023-10-18T08:55:16.283192+0000","last_became_active":"2023-10-18T08:55:15.316743+0000","last_became_peered":"2023-10-18T08:55:15.316743+0000","last_unstale":"2023-10-18T08:55:16.283192+0000","last_undegraded":"2023-10-18T08:55:16.283192+0000","last_fullsized":"2023-10-18T08:55:16.283192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:46.921031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.303320+0000","last_change":"2023-10-18T08:55:14.294348+0000","last_active":"2023-10-18T08:55:15.303320+0000","last_peered":"2023-10-18T08:55:15.303320+0000","last_clean":"2023-10-18T08:55:15.303320+0000","last_became_active":"2023-10-18T08:55:14.293456+0000","last_became_peered":"2023-10-18T08:55:14.293456+0000","last_unstale":"2023-10-18T08:55:15.303320+0000","last_undegraded":"2023-10-18T08:55:15.303320+0000","last_fullsized":"2023-10-18T08:55:15.303320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:37.505319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689020+0000","last_change":"2023-10-18T08:55:15.304772+0000","last_active":"2023-10-18T08:55:16.689020+0000","last_peered":"2023-10-18T08:55:16.689020+0000","last_clean":"2023-10-18T08:55:16.689020+0000","last_became_active":"2023-10-18T08:55:15.304528+0000","last_became_peered":"2023-10-18T08:55:15.304528+0000","last_unstale":"2023-10-18T08:55:16.689020+0000","last_undegraded":"2023-10-18T08:55:16.689020+0000","last_fullsized":"2023-10-18T08:55:16.689020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:27.650596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.330488+0000","last_change":"2023-10-18T08:55:14.297308+0000","last_active":"2023-10-18T08:55:15.330488+0000","last_peered":"2023-10-18T08:55:15.330488+0000","last_clean":"2023-10-18T08:55:15.330488+0000","last_became_active":"2023-10-18T08:55:14.297069+0000","last_became_peered":"2023-10-18T08:55:14.297069+0000","last_unstale":"2023-10-18T08:55:15.330488+0000","last_undegraded":"2023-10-18T08:55:15.330488+0000","last_fullsized":"2023-10-18T08:55:15.330488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:10.814889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.283167+0000","last_change":"2023-10-18T08:55:15.303171+0000","last_active":"2023-10-18T08:55:16.283167+0000","last_peered":"2023-10-18T08:55:16.283167+0000","last_clean":"2023-10-18T08:55:16.283167+0000","last_became_active":"2023-10-18T08:55:15.301815+0000","last_became_peered":"2023-10-18T08:55:15.301815+0000","last_unstale":"2023-10-18T08:55:16.283167+0000","last_undegraded":"2023-10-18T08:55:16.283167+0000","last_fullsized":"2023-10-18T08:55:16.283167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:10.355168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.325923+0000","last_change":"2023-10-18T08:55:14.298243+0000","last_active":"2023-10-18T08:55:15.325923+0000","last_peered":"2023-10-18T08:55:15.325923+0000","last_clean":"2023-10-18T08:55:15.325923+0000","last_became_active":"2023-10-18T08:55:14.297343+0000","last_became_peered":"2023-10-18T08:55:14.297343+0000","last_unstale":"2023-10-18T08:55:15.325923+0000","last_undegraded":"2023-10-18T08:55:15.325923+0000","last_fullsized":"2023-10-18T08:55:15.325923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:10.814889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.283511+0000","last_change":"2023-10-18T08:55:15.316986+0000","last_active":"2023-10-18T08:55:16.283511+0000","last_peered":"2023-10-18T08:55:16.283511+0000","last_clean":"2023-10-18T08:55:16.283511+0000","last_became_active":"2023-10-18T08:55:15.316398+0000","last_became_peered":"2023-10-18T08:55:15.316398+0000","last_unstale":"2023-10-18T08:55:16.283511+0000","last_undegraded":"2023-10-18T08:55:16.283511+0000","last_fullsized":"2023-10-18T08:55:16.283511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:56:45.727823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.329694+0000","last_change":"2023-10-18T08:55:14.290070+0000","last_active":"2023-10-18T08:55:15.329694+0000","last_peered":"2023-10-18T08:55:15.329694+0000","last_clean":"2023-10-18T08:55:15.329694+0000","last_became_active":"2023-10-18T08:55:14.289756+0000","last_became_peered":"2023-10-18T08:55:14.289756+0000","last_unstale":"2023-10-18T08:55:15.329694+0000","last_undegraded":"2023-10-18T08:55:15.329694+0000","last_fullsized":"2023-10-18T08:55:15.329694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:01.470259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282548+0000","last_change":"2023-10-18T08:55:15.311283+0000","last_active":"2023-10-18T08:55:16.282548+0000","last_peered":"2023-10-18T08:55:16.282548+0000","last_clean":"2023-10-18T08:55:16.282548+0000","last_became_active":"2023-10-18T08:55:15.311053+0000","last_became_peered":"2023-10-18T08:55:15.311053+0000","last_unstale":"2023-10-18T08:55:16.282548+0000","last_undegraded":"2023-10-18T08:55:16.282548+0000","last_fullsized":"2023-10-18T08:55:16.282548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:23:50.828764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.325915+0000","last_change":"2023-10-18T08:55:14.294635+0000","last_active":"2023-10-18T08:55:15.325915+0000","last_peered":"2023-10-18T08:55:15.325915+0000","last_clean":"2023-10-18T08:55:15.325915+0000","last_became_active":"2023-10-18T08:55:14.294341+0000","last_became_peered":"2023-10-18T08:55:14.294341+0000","last_unstale":"2023-10-18T08:55:15.325915+0000","last_undegraded":"2023-10-18T08:55:15.325915+0000","last_fullsized":"2023-10-18T08:55:15.325915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:01.470259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691402+0000","last_change":"2023-10-18T08:55:15.312145+0000","last_active":"2023-10-18T08:55:16.691402+0000","last_peered":"2023-10-18T08:55:16.691402+0000","last_clean":"2023-10-18T08:55:16.691402+0000","last_became_active":"2023-10-18T08:55:15.310296+0000","last_became_peered":"2023-10-18T08:55:15.310296+0000","last_unstale":"2023-10-18T08:55:16.691402+0000","last_undegraded":"2023-10-18T08:55:16.691402+0000","last_fullsized":"2023-10-18T08:55:16.691402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:46.921031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.282183+0000","last_change":"2023-10-18T08:55:14.303903+0000","last_active":"2023-10-18T08:55:15.282183+0000","last_peered":"2023-10-18T08:55:15.282183+0000","last_clean":"2023-10-18T08:55:15.282183+0000","last_became_active":"2023-10-18T08:55:14.303633+0000","last_became_peered":"2023-10-18T08:55:14.303633+0000","last_unstale":"2023-10-18T08:55:15.282183+0000","last_undegraded":"2023-10-18T08:55:15.282183+0000","last_fullsized":"2023-10-18T08:55:15.282183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:16:56.668322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.283316+0000","last_change":"2023-10-18T08:55:15.318598+0000","last_active":"2023-10-18T08:55:16.283316+0000","last_peered":"2023-10-18T08:55:16.283316+0000","last_clean":"2023-10-18T08:55:16.283316+0000","last_became_active":"2023-10-18T08:55:15.318353+0000","last_became_peered":"2023-10-18T08:55:15.318353+0000","last_unstale":"2023-10-18T08:55:16.283316+0000","last_undegraded":"2023-10-18T08:55:16.283316+0000","last_fullsized":"2023-10-18T08:55:16.283316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:10.917539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.287158+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:55:15.287158+0000","last_peered":"2023-10-18T08:55:15.287158+0000","last_clean":"2023-10-18T08:55:15.287158+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:55:15.287158+0000","last_undegraded":"2023-10-18T08:55:15.287158+0000","last_fullsized":"2023-10-18T08:55:15.287158+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:34.881744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.316324+0000","last_change":"2023-10-18T08:55:14.305589+0000","last_active":"2023-10-18T08:55:15.316324+0000","last_peered":"2023-10-18T08:55:15.316324+0000","last_clean":"2023-10-18T08:55:15.316324+0000","last_became_active":"2023-10-18T08:55:14.303723+0000","last_became_peered":"2023-10-18T08:55:14.303723+0000","last_unstale":"2023-10-18T08:55:15.316324+0000","last_undegraded":"2023-10-18T08:55:15.316324+0000","last_fullsized":"2023-10-18T08:55:15.316324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:05.214421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280180+0000","last_change":"2023-10-18T08:55:15.307598+0000","last_active":"2023-10-18T08:55:16.280180+0000","last_peered":"2023-10-18T08:55:16.280180+0000","last_clean":"2023-10-18T08:55:16.280180+0000","last_became_active":"2023-10-18T08:55:15.307263+0000","last_became_peered":"2023-10-18T08:55:15.307263+0000","last_unstale":"2023-10-18T08:55:16.280180+0000","last_undegraded":"2023-10-18T08:55:16.280180+0000","last_fullsized":"2023-10-18T08:55:16.280180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:28.148282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.286898+0000","last_change":"2023-10-18T08:55:14.299430+0000","last_active":"2023-10-18T08:55:15.286898+0000","last_peered":"2023-10-18T08:55:15.286898+0000","last_clean":"2023-10-18T08:55:15.286898+0000","last_became_active":"2023-10-18T08:55:14.299004+0000","last_became_peered":"2023-10-18T08:55:14.299004+0000","last_unstale":"2023-10-18T08:55:15.286898+0000","last_undegraded":"2023-10-18T08:55:15.286898+0000","last_fullsized":"2023-10-18T08:55:15.286898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:26.645889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689865+0000","last_change":"2023-10-18T08:55:15.308337+0000","last_active":"2023-10-18T08:55:16.689865+0000","last_peered":"2023-10-18T08:55:16.689865+0000","last_clean":"2023-10-18T08:55:16.689865+0000","last_became_active":"2023-10-18T08:55:15.305146+0000","last_became_peered":"2023-10-18T08:55:15.305146+0000","last_unstale":"2023-10-18T08:55:16.689865+0000","last_undegraded":"2023-10-18T08:55:16.689865+0000","last_fullsized":"2023-10-18T08:55:16.689865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.283153+0000","last_change":"2023-10-18T08:55:14.287811+0000","last_active":"2023-10-18T08:55:15.283153+0000","last_peered":"2023-10-18T08:55:15.283153+0000","last_clean":"2023-10-18T08:55:15.283153+0000","last_became_active":"2023-10-18T08:55:14.287466+0000","last_became_peered":"2023-10-18T08:55:14.287466+0000","last_unstale":"2023-10-18T08:55:15.283153+0000","last_undegraded":"2023-10-18T08:55:15.283153+0000","last_fullsized":"2023-10-18T08:55:15.283153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:31.268409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281786+0000","last_change":"2023-10-18T08:55:15.301141+0000","last_active":"2023-10-18T08:55:16.281786+0000","last_peered":"2023-10-18T08:55:16.281786+0000","last_clean":"2023-10-18T08:55:16.281786+0000","last_became_active":"2023-10-18T08:55:15.300535+0000","last_became_peered":"2023-10-18T08:55:15.300535+0000","last_unstale":"2023-10-18T08:55:16.281786+0000","last_undegraded":"2023-10-18T08:55:16.281786+0000","last_fullsized":"2023-10-18T08:55:16.281786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:28.148282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.300341+0000","last_change":"2023-10-18T08:55:14.302819+0000","last_active":"2023-10-18T08:55:15.300341+0000","last_peered":"2023-10-18T08:55:15.300341+0000","last_clean":"2023-10-18T08:55:15.300341+0000","last_became_active":"2023-10-18T08:55:14.302381+0000","last_became_peered":"2023-10-18T08:55:14.302381+0000","last_unstale":"2023-10-18T08:55:15.300341+0000","last_undegraded":"2023-10-18T08:55:15.300341+0000","last_fullsized":"2023-10-18T08:55:15.300341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:20.837731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690460+0000","last_change":"2023-10-18T08:55:15.309720+0000","last_active":"2023-10-18T08:55:16.690460+0000","last_peered":"2023-10-18T08:55:16.690460+0000","last_clean":"2023-10-18T08:55:16.690460+0000","last_became_active":"2023-10-18T08:55:15.309469+0000","last_became_peered":"2023-10-18T08:55:15.309469+0000","last_unstale":"2023-10-18T08:55:16.690460+0000","last_undegraded":"2023-10-18T08:55:16.690460+0000","last_fullsized":"2023-10-18T08:55:16.690460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:04.364364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.295166+0000","last_change":"2023-10-18T08:55:14.296298+0000","last_active":"2023-10-18T08:55:15.295166+0000","last_peered":"2023-10-18T08:55:15.295166+0000","last_clean":"2023-10-18T08:55:15.295166+0000","last_became_active":"2023-10-18T08:55:14.295788+0000","last_became_peered":"2023-10-18T08:55:14.295788+0000","last_unstale":"2023-10-18T08:55:15.295166+0000","last_undegraded":"2023-10-18T08:55:15.295166+0000","last_fullsized":"2023-10-18T08:55:15.295166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:38.968705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689511+0000","last_change":"2023-10-18T08:55:15.307391+0000","last_active":"2023-10-18T08:55:16.689511+0000","last_peered":"2023-10-18T08:55:16.689511+0000","last_clean":"2023-10-18T08:55:16.689511+0000","last_became_active":"2023-10-18T08:55:15.306422+0000","last_became_peered":"2023-10-18T08:55:15.306422+0000","last_unstale":"2023-10-18T08:55:16.689511+0000","last_undegraded":"2023-10-18T08:55:16.689511+0000","last_fullsized":"2023-10-18T08:55:16.689511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.688019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.286110+0000","last_change":"2023-10-18T08:55:14.299281+0000","last_active":"2023-10-18T08:55:15.286110+0000","last_peered":"2023-10-18T08:55:15.286110+0000","last_clean":"2023-10-18T08:55:15.286110+0000","last_became_active":"2023-10-18T08:55:14.298698+0000","last_became_peered":"2023-10-18T08:55:14.298698+0000","last_unstale":"2023-10-18T08:55:15.286110+0000","last_undegraded":"2023-10-18T08:55:15.286110+0000","last_fullsized":"2023-10-18T08:55:15.286110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:44.154488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281874+0000","last_change":"2023-10-18T08:55:15.299892+0000","last_active":"2023-10-18T08:55:16.281874+0000","last_peered":"2023-10-18T08:55:16.281874+0000","last_clean":"2023-10-18T08:55:16.281874+0000","last_became_active":"2023-10-18T08:55:15.299281+0000","last_became_peered":"2023-10-18T08:55:15.299281+0000","last_unstale":"2023-10-18T08:55:16.281874+0000","last_undegraded":"2023-10-18T08:55:16.281874+0000","last_fullsized":"2023-10-18T08:55:16.281874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:36.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281731+0000","last_change":"2023-10-18T08:55:14.300359+0000","last_active":"2023-10-18T08:55:16.281731+0000","last_peered":"2023-10-18T08:55:16.281731+0000","last_clean":"2023-10-18T08:55:16.281731+0000","last_became_active":"2023-10-18T08:55:14.299423+0000","last_became_peered":"2023-10-18T08:55:14.299423+0000","last_unstale":"2023-10-18T08:55:16.281731+0000","last_undegraded":"2023-10-18T08:55:16.281731+0000","last_fullsized":"2023-10-18T08:55:16.281731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:44.154488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280011+0000","last_change":"2023-10-18T08:55:15.288327+0000","last_active":"2023-10-18T08:55:16.280011+0000","last_peered":"2023-10-18T08:55:16.280011+0000","last_clean":"2023-10-18T08:55:16.280011+0000","last_became_active":"2023-10-18T08:55:15.287648+0000","last_became_peered":"2023-10-18T08:55:15.287648+0000","last_unstale":"2023-10-18T08:55:16.280011+0000","last_undegraded":"2023-10-18T08:55:16.280011+0000","last_fullsized":"2023-10-18T08:55:16.280011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:27.650596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.327662+0000","last_change":"2023-10-18T08:55:14.294992+0000","last_active":"2023-10-18T08:55:15.327662+0000","last_peered":"2023-10-18T08:55:15.327662+0000","last_clean":"2023-10-18T08:55:15.327662+0000","last_became_active":"2023-10-18T08:55:14.294682+0000","last_became_peered":"2023-10-18T08:55:14.294682+0000","last_unstale":"2023-10-18T08:55:15.327662+0000","last_undegraded":"2023-10-18T08:55:15.327662+0000","last_fullsized":"2023-10-18T08:55:15.327662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:44.154488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689214+0000","last_change":"2023-10-18T08:55:15.308496+0000","last_active":"2023-10-18T08:55:16.689214+0000","last_peered":"2023-10-18T08:55:16.689214+0000","last_clean":"2023-10-18T08:55:16.689214+0000","last_became_active":"2023-10-18T08:55:15.306095+0000","last_became_peered":"2023-10-18T08:55:15.306095+0000","last_unstale":"2023-10-18T08:55:16.689214+0000","last_undegraded":"2023-10-18T08:55:16.689214+0000","last_fullsized":"2023-10-18T08:55:16.689214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:02.173932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.288529+0000","last_change":"2023-10-18T08:55:14.299395+0000","last_active":"2023-10-18T08:55:15.288529+0000","last_peered":"2023-10-18T08:55:15.288529+0000","last_clean":"2023-10-18T08:55:15.288529+0000","last_became_active":"2023-10-18T08:55:14.299130+0000","last_became_peered":"2023-10-18T08:55:14.299130+0000","last_unstale":"2023-10-18T08:55:15.288529+0000","last_undegraded":"2023-10-18T08:55:15.288529+0000","last_fullsized":"2023-10-18T08:55:15.288529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:35.197625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.693084+0000","last_change":"2023-10-18T08:55:15.314704+0000","last_active":"2023-10-18T08:55:16.693084+0000","last_peered":"2023-10-18T08:55:16.693084+0000","last_clean":"2023-10-18T08:55:16.693084+0000","last_became_active":"2023-10-18T08:55:15.314153+0000","last_became_peered":"2023-10-18T08:55:15.314153+0000","last_unstale":"2023-10-18T08:55:16.693084+0000","last_undegraded":"2023-10-18T08:55:16.693084+0000","last_fullsized":"2023-10-18T08:55:16.693084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:39.973412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.295022+0000","last_change":"2023-10-18T08:55:14.297463+0000","last_active":"2023-10-18T08:55:15.295022+0000","last_peered":"2023-10-18T08:55:15.295022+0000","last_clean":"2023-10-18T08:55:15.295022+0000","last_became_active":"2023-10-18T08:55:14.297199+0000","last_became_peered":"2023-10-18T08:55:14.297199+0000","last_unstale":"2023-10-18T08:55:15.295022+0000","last_undegraded":"2023-10-18T08:55:15.295022+0000","last_fullsized":"2023-10-18T08:55:15.295022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:04.186241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.279960+0000","last_change":"2023-10-18T08:55:15.303892+0000","last_active":"2023-10-18T08:55:16.279960+0000","last_peered":"2023-10-18T08:55:16.279960+0000","last_clean":"2023-10-18T08:55:16.279960+0000","last_became_active":"2023-10-18T08:55:15.303545+0000","last_became_peered":"2023-10-18T08:55:15.303545+0000","last_unstale":"2023-10-18T08:55:16.279960+0000","last_undegraded":"2023-10-18T08:55:16.279960+0000","last_fullsized":"2023-10-18T08:55:16.279960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:02.173932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.297222+0000","last_change":"2023-10-18T08:55:14.294024+0000","last_active":"2023-10-18T08:55:15.297222+0000","last_peered":"2023-10-18T08:55:15.297222+0000","last_clean":"2023-10-18T08:55:15.297222+0000","last_became_active":"2023-10-18T08:55:14.293763+0000","last_became_peered":"2023-10-18T08:55:14.293763+0000","last_unstale":"2023-10-18T08:55:15.297222+0000","last_undegraded":"2023-10-18T08:55:15.297222+0000","last_fullsized":"2023-10-18T08:55:15.297222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:20.837731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280694+0000","last_change":"2023-10-18T08:55:15.302069+0000","last_active":"2023-10-18T08:55:16.280694+0000","last_peered":"2023-10-18T08:55:16.280694+0000","last_clean":"2023-10-18T08:55:16.280694+0000","last_became_active":"2023-10-18T08:55:15.301806+0000","last_became_peered":"2023-10-18T08:55:15.301806+0000","last_unstale":"2023-10-18T08:55:16.280694+0000","last_undegraded":"2023-10-18T08:55:16.280694+0000","last_fullsized":"2023-10-18T08:55:16.280694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.688019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.327881+0000","last_change":"2023-10-18T08:55:14.292655+0000","last_active":"2023-10-18T08:55:15.327881+0000","last_peered":"2023-10-18T08:55:15.327881+0000","last_clean":"2023-10-18T08:55:15.327881+0000","last_became_active":"2023-10-18T08:55:14.292291+0000","last_became_peered":"2023-10-18T08:55:14.292291+0000","last_unstale":"2023-10-18T08:55:15.327881+0000","last_undegraded":"2023-10-18T08:55:15.327881+0000","last_fullsized":"2023-10-18T08:55:15.327881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:57.132126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688748+0000","last_change":"2023-10-18T08:55:15.315320+0000","last_active":"2023-10-18T08:55:16.688748+0000","last_peered":"2023-10-18T08:55:16.688748+0000","last_clean":"2023-10-18T08:55:16.688748+0000","last_became_active":"2023-10-18T08:55:15.315007+0000","last_became_peered":"2023-10-18T08:55:15.315007+0000","last_unstale":"2023-10-18T08:55:16.688748+0000","last_undegraded":"2023-10-18T08:55:16.688748+0000","last_fullsized":"2023-10-18T08:55:16.688748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:19.879235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691247+0000","last_change":"2023-10-18T08:55:15.317460+0000","last_active":"2023-10-18T08:55:16.691247+0000","last_peered":"2023-10-18T08:55:16.691247+0000","last_clean":"2023-10-18T08:55:16.691247+0000","last_became_active":"2023-10-18T08:55:15.317194+0000","last_became_peered":"2023-10-18T08:55:15.317194+0000","last_unstale":"2023-10-18T08:55:16.691247+0000","last_undegraded":"2023-10-18T08:55:16.691247+0000","last_fullsized":"2023-10-18T08:55:16.691247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:26.310762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.328212+0000","last_change":"2023-10-18T08:55:14.301102+0000","last_active":"2023-10-18T08:55:15.328212+0000","last_peered":"2023-10-18T08:55:15.328212+0000","last_clean":"2023-10-18T08:55:15.328212+0000","last_became_active":"2023-10-18T08:55:14.300855+0000","last_became_peered":"2023-10-18T08:55:14.300855+0000","last_unstale":"2023-10-18T08:55:15.328212+0000","last_undegraded":"2023-10-18T08:55:15.328212+0000","last_fullsized":"2023-10-18T08:55:15.328212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:27.143575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.692035+0000","last_change":"2023-10-18T08:55:15.320111+0000","last_active":"2023-10-18T08:55:16.692035+0000","last_peered":"2023-10-18T08:55:16.692035+0000","last_clean":"2023-10-18T08:55:16.692035+0000","last_became_active":"2023-10-18T08:55:15.319103+0000","last_became_peered":"2023-10-18T08:55:15.319103+0000","last_unstale":"2023-10-18T08:55:16.692035+0000","last_undegraded":"2023-10-18T08:55:16.692035+0000","last_fullsized":"2023-10-18T08:55:16.692035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:56:45.727823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.327995+0000","last_change":"2023-10-18T08:55:14.293731+0000","last_active":"2023-10-18T08:55:15.327995+0000","last_peered":"2023-10-18T08:55:15.327995+0000","last_clean":"2023-10-18T08:55:15.327995+0000","last_became_active":"2023-10-18T08:55:14.293420+0000","last_became_peered":"2023-10-18T08:55:14.293420+0000","last_unstale":"2023-10-18T08:55:15.327995+0000","last_undegraded":"2023-10-18T08:55:15.327995+0000","last_fullsized":"2023-10-18T08:55:15.327995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:01.169224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689541+0000","last_change":"2023-10-18T08:55:15.307386+0000","last_active":"2023-10-18T08:55:16.689541+0000","last_peered":"2023-10-18T08:55:16.689541+0000","last_clean":"2023-10-18T08:55:16.689541+0000","last_became_active":"2023-10-18T08:55:15.304519+0000","last_became_peered":"2023-10-18T08:55:15.304519+0000","last_unstale":"2023-10-18T08:55:16.689541+0000","last_undegraded":"2023-10-18T08:55:16.689541+0000","last_fullsized":"2023-10-18T08:55:16.689541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:28.148282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.329282+0000","last_change":"2023-10-18T08:55:14.297189+0000","last_active":"2023-10-18T08:55:15.329282+0000","last_peered":"2023-10-18T08:55:15.329282+0000","last_clean":"2023-10-18T08:55:15.329282+0000","last_became_active":"2023-10-18T08:55:14.296794+0000","last_became_peered":"2023-10-18T08:55:14.296794+0000","last_unstale":"2023-10-18T08:55:15.329282+0000","last_undegraded":"2023-10-18T08:55:15.329282+0000","last_fullsized":"2023-10-18T08:55:15.329282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:10.814889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280421+0000","last_change":"2023-10-18T08:55:15.309830+0000","last_active":"2023-10-18T08:55:16.280421+0000","last_peered":"2023-10-18T08:55:16.280421+0000","last_clean":"2023-10-18T08:55:16.280421+0000","last_became_active":"2023-10-18T08:55:15.309632+0000","last_became_peered":"2023-10-18T08:55:15.309632+0000","last_unstale":"2023-10-18T08:55:16.280421+0000","last_undegraded":"2023-10-18T08:55:16.280421+0000","last_fullsized":"2023-10-18T08:55:16.280421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.328808+0000","last_change":"2023-10-18T08:55:14.297412+0000","last_active":"2023-10-18T08:55:15.328808+0000","last_peered":"2023-10-18T08:55:15.328808+0000","last_clean":"2023-10-18T08:55:15.328808+0000","last_became_active":"2023-10-18T08:55:14.296983+0000","last_became_peered":"2023-10-18T08:55:14.296983+0000","last_unstale":"2023-10-18T08:55:15.328808+0000","last_undegraded":"2023-10-18T08:55:15.328808+0000","last_fullsized":"2023-10-18T08:55:15.328808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:38.968705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338312,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4550000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.341}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5249999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.698}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.841}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7549999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930155,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5149999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.796}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5309999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4670000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.585}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930121,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.657}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194667,"statfs":{"total":200000000,"available":199339275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4420000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7639999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.492}]}]}],"pool_statfs":[]}} 2023-10-18T08:55:31.115 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:55:31.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.378+0000 7f3e9c724640 1 -- 172.21.15.40:0/3576595631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e941586a0 msgr2=0x7f3e940b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:31.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.378+0000 7f3e9c724640 1 --2- 172.21.15.40:0/3576595631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e941586a0 0x7f3e940b0bc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3e940acc20 tx=0x7f3e8402bda0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.378+0000 7f3e9c724640 1 -- 172.21.15.40:0/3576595631 shutdown_connections 2023-10-18T08:55:31.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.378+0000 7f3e9c724640 1 --2- 172.21.15.40:0/3576595631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e941586a0 0x7f3e940b0bc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.378+0000 7f3e9c724640 1 --2- 172.21.15.40:0/3576595631 >> v2:172.21.15.144:3300/0 conn(0x7f3e9406d710 0x7f3e94151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.378+0000 7f3e9c724640 1 -- 172.21.15.40:0/3576595631 >> 172.21.15.40:0/3576595631 conn(0x7f3e9406cee0 msgr2=0x7f3e94069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:31.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.378+0000 7f3e9c724640 1 -- 172.21.15.40:0/3576595631 shutdown_connections 2023-10-18T08:55:31.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.378+0000 7f3e9c724640 1 -- 172.21.15.40:0/3576595631 wait complete. 2023-10-18T08:55:31.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.380+0000 7f3e9c724640 1 Processor -- start 2023-10-18T08:55:31.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.380+0000 7f3e9c724640 1 -- start start 2023-10-18T08:55:31.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.381+0000 7f3e9c724640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e9406d710 0x7f3e9413fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:31.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.381+0000 7f3e9c724640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e940b1100 0x7f3e9414e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:31.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.381+0000 7f3e9c724640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e941586a0 0x7f3e940cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:31.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.381+0000 7f3e9c724640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e941400d0 con 0x7f3e940b1100 2023-10-18T08:55:31.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.381+0000 7f3e9c724640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e940caef0 con 0x7f3e941586a0 2023-10-18T08:55:31.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.381+0000 7f3e9c724640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9414da50 con 0x7f3e9406d710 2023-10-18T08:55:31.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.381+0000 7f3e99cbf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e940b1100 0x7f3e9414e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:31.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.381+0000 7f3e99cbf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e940b1100 0x7f3e9414e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48192/0 (socket says 172.21.15.40:48192) 2023-10-18T08:55:31.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.381+0000 7f3e99cbf640 1 -- 172.21.15.40:0/168063349 learned_addr learned my addr 172.21.15.40:0/168063349 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.382+0000 7f3e99cbf640 1 -- 172.21.15.40:0/168063349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e9406d710 msgr2=0x7f3e9413fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.382+0000 7f3e9a4c0640 1 --2- 172.21.15.40:0/168063349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e9406d710 0x7f3e9413fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.382+0000 7f3e9acc1640 1 --2- 172.21.15.40:0/168063349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e941586a0 0x7f3e940cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.382+0000 7f3e99cbf640 1 --2- 172.21.15.40:0/168063349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e9406d710 0x7f3e9413fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.382+0000 7f3e99cbf640 1 -- 172.21.15.40:0/168063349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e941586a0 msgr2=0x7f3e940cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:31.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.382+0000 7f3e99cbf640 1 --2- 172.21.15.40:0/168063349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e941586a0 0x7f3e940cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.382+0000 7f3e99cbf640 1 -- 172.21.15.40:0/168063349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e84009560 con 0x7f3e940b1100 2023-10-18T08:55:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.382+0000 7f3e9a4c0640 1 --2- 172.21.15.40:0/168063349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e9406d710 0x7f3e9413fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.382+0000 7f3e9acc1640 1 --2- 172.21.15.40:0/168063349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e941586a0 0x7f3e940cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:31.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.382+0000 7f3e99cbf640 1 --2- 172.21.15.40:0/168063349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e940b1100 0x7f3e9414e530 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3e84036040 tx=0x7f3e8402dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:31.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.382+0000 7f3e8affd640 1 -- 172.21.15.40:0/168063349 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84003800 con 0x7f3e940b1100 2023-10-18T08:55:31.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.382+0000 7f3e8affd640 1 -- 172.21.15.40:0/168063349 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3e84003a50 con 0x7f3e940b1100 2023-10-18T08:55:31.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.382+0000 7f3e8affd640 1 -- 172.21.15.40:0/168063349 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84003de0 con 0x7f3e940b1100 2023-10-18T08:55:31.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.382+0000 7f3e9c724640 1 -- 172.21.15.40:0/168063349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9414dd30 con 0x7f3e940b1100 2023-10-18T08:55:31.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.383+0000 7f3e9c724640 1 -- 172.21.15.40:0/168063349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9414d700 con 0x7f3e940b1100 2023-10-18T08:55:31.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.384+0000 7f3e9c724640 1 -- 172.21.15.40:0/168063349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9414e250 con 0x7f3e940b1100 2023-10-18T08:55:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.385+0000 7f3e8affd640 1 -- 172.21.15.40:0/168063349 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3e8403e450 con 0x7f3e940b1100 2023-10-18T08:55:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.385+0000 7f3e8affd640 1 --2- 172.21.15.40:0/168063349 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e7806ae40 0x7f3e7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.386+0000 7f3e8affd640 1 -- 172.21.15.40:0/168063349 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e840abf90 con 0x7f3e940b1100 2023-10-18T08:55:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.386+0000 7f3e9a4c0640 1 --2- 172.21.15.40:0/168063349 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e7806ae40 0x7f3e7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.387+0000 7f3e9a4c0640 1 --2- 172.21.15.40:0/168063349 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e7806ae40 0x7f3e7806d300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3e90006fd0 tx=0x7f3e90008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:31.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.389+0000 7f3e8affd640 1 -- 172.21.15.40:0/168063349 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e840deca0 con 0x7f3e940b1100 2023-10-18T08:55:31.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.573+0000 7f3e9c724640 1 -- 172.21.15.40:0/168063349 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e940cdaa0 con 0x7f3e7806ae40 2023-10-18T08:55:31.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.593+0000 7f3e8affd640 1 -- 172.21.15.40:0/168063349 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259981 (secure 0 0 0) 0x7f3e9000ae20 con 0x7f3e7806ae40 2023-10-18T08:55:31.600 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:31.601 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:55:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.598+0000 7f3e9c724640 1 -- 172.21.15.40:0/168063349 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e7806ae40 msgr2=0x7f3e7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.598+0000 7f3e9c724640 1 --2- 172.21.15.40:0/168063349 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e7806ae40 0x7f3e7806d300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3e90006fd0 tx=0x7f3e90008040 comp rx=0 tx=0).stop 2023-10-18T08:55:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.598+0000 7f3e9c724640 1 -- 172.21.15.40:0/168063349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e940b1100 msgr2=0x7f3e9414e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.598+0000 7f3e9c724640 1 --2- 172.21.15.40:0/168063349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e940b1100 0x7f3e9414e530 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3e84036040 tx=0x7f3e8402dd70 comp rx=0 tx=0).stop 2023-10-18T08:55:31.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.599+0000 7f3e9c724640 1 -- 172.21.15.40:0/168063349 shutdown_connections 2023-10-18T08:55:31.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.599+0000 7f3e9c724640 1 --2- 172.21.15.40:0/168063349 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e7806ae40 0x7f3e7806d300 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.599+0000 7f3e9c724640 1 --2- 172.21.15.40:0/168063349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e941586a0 0x7f3e940cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.599+0000 7f3e9c724640 1 --2- 172.21.15.40:0/168063349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e940b1100 0x7f3e9414e530 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.599+0000 7f3e9c724640 1 --2- 172.21.15.40:0/168063349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e9406d710 0x7f3e9413fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.599+0000 7f3e9c724640 1 -- 172.21.15.40:0/168063349 >> 172.21.15.40:0/168063349 conn(0x7f3e9406cee0 msgr2=0x7f3e94071200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:31.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.599+0000 7f3e9c724640 1 -- 172.21.15.40:0/168063349 shutdown_connections 2023-10-18T08:55:31.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.599+0000 7f3e9c724640 1 -- 172.21.15.40:0/168063349 wait complete. 2023-10-18T08:55:31.621 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-18T08:55:30.899225+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560792,"statfs":{"total":1600000000,"available":1598251597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004891"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282778+0000","last_change":"2023-10-18T08:55:15.307275+0000","last_active":"2023-10-18T08:55:16.282778+0000","last_peered":"2023-10-18T08:55:16.282778+0000","last_clean":"2023-10-18T08:55:16.282778+0000","last_became_active":"2023-10-18T08:55:15.304759+0000","last_became_peered":"2023-10-18T08:55:15.304759+0000","last_unstale":"2023-10-18T08:55:16.282778+0000","last_undegraded":"2023-10-18T08:55:16.282778+0000","last_fullsized":"2023-10-18T08:55:16.282778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:41.755639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281449+0000","last_change":"2023-10-18T08:55:15.306465+0000","last_active":"2023-10-18T08:55:16.281449+0000","last_peered":"2023-10-18T08:55:16.281449+0000","last_clean":"2023-10-18T08:55:16.281449+0000","last_became_active":"2023-10-18T08:55:15.305909+0000","last_became_peered":"2023-10-18T08:55:15.305909+0000","last_unstale":"2023-10-18T08:55:16.281449+0000","last_undegraded":"2023-10-18T08:55:16.281449+0000","last_fullsized":"2023-10-18T08:55:16.281449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:02.173932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688851+0000","last_change":"2023-10-18T08:55:15.302350+0000","last_active":"2023-10-18T08:55:16.688851+0000","last_peered":"2023-10-18T08:55:16.688851+0000","last_clean":"2023-10-18T08:55:16.688851+0000","last_became_active":"2023-10-18T08:55:15.302090+0000","last_became_peered":"2023-10-18T08:55:15.302090+0000","last_unstale":"2023-10-18T08:55:16.688851+0000","last_undegraded":"2023-10-18T08:55:16.688851+0000","last_fullsized":"2023-10-18T08:55:16.688851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:19.879235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688836+0000","last_change":"2023-10-18T08:55:15.307679+0000","last_active":"2023-10-18T08:55:16.688836+0000","last_peered":"2023-10-18T08:55:16.688836+0000","last_clean":"2023-10-18T08:55:16.688836+0000","last_became_active":"2023-10-18T08:55:15.306999+0000","last_became_peered":"2023-10-18T08:55:15.306999+0000","last_unstale":"2023-10-18T08:55:16.688836+0000","last_undegraded":"2023-10-18T08:55:16.688836+0000","last_fullsized":"2023-10-18T08:55:16.688836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282830+0000","last_change":"2023-10-18T08:55:15.293552+0000","last_active":"2023-10-18T08:55:16.282830+0000","last_peered":"2023-10-18T08:55:16.282830+0000","last_clean":"2023-10-18T08:55:16.282830+0000","last_became_active":"2023-10-18T08:55:15.292895+0000","last_became_peered":"2023-10-18T08:55:15.292895+0000","last_unstale":"2023-10-18T08:55:16.282830+0000","last_undegraded":"2023-10-18T08:55:16.282830+0000","last_fullsized":"2023-10-18T08:55:16.282830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:07:58.310623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691708+0000","last_change":"2023-10-18T08:55:15.319351+0000","last_active":"2023-10-18T08:55:16.691708+0000","last_peered":"2023-10-18T08:55:16.691708+0000","last_clean":"2023-10-18T08:55:16.691708+0000","last_became_active":"2023-10-18T08:55:15.317663+0000","last_became_peered":"2023-10-18T08:55:15.317663+0000","last_unstale":"2023-10-18T08:55:16.691708+0000","last_undegraded":"2023-10-18T08:55:16.691708+0000","last_fullsized":"2023-10-18T08:55:16.691708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:10.917539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282757+0000","last_change":"2023-10-18T08:55:15.298227+0000","last_active":"2023-10-18T08:55:16.282757+0000","last_peered":"2023-10-18T08:55:16.282757+0000","last_clean":"2023-10-18T08:55:16.282757+0000","last_became_active":"2023-10-18T08:55:15.297630+0000","last_became_peered":"2023-10-18T08:55:15.297630+0000","last_unstale":"2023-10-18T08:55:16.282757+0000","last_undegraded":"2023-10-18T08:55:16.282757+0000","last_fullsized":"2023-10-18T08:55:16.282757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:04.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.279841+0000","last_change":"2023-10-18T08:55:15.307773+0000","last_active":"2023-10-18T08:55:16.279841+0000","last_peered":"2023-10-18T08:55:16.279841+0000","last_clean":"2023-10-18T08:55:16.279841+0000","last_became_active":"2023-10-18T08:55:15.307252+0000","last_became_peered":"2023-10-18T08:55:15.307252+0000","last_unstale":"2023-10-18T08:55:16.279841+0000","last_undegraded":"2023-10-18T08:55:16.279841+0000","last_fullsized":"2023-10-18T08:55:16.279841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:19.879235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282164+0000","last_change":"2023-10-18T08:55:15.308028+0000","last_active":"2023-10-18T08:55:16.282164+0000","last_peered":"2023-10-18T08:55:16.282164+0000","last_clean":"2023-10-18T08:55:16.282164+0000","last_became_active":"2023-10-18T08:55:15.307786+0000","last_became_peered":"2023-10-18T08:55:15.307786+0000","last_unstale":"2023-10-18T08:55:16.282164+0000","last_undegraded":"2023-10-18T08:55:16.282164+0000","last_fullsized":"2023-10-18T08:55:16.282164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:31.888781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691934+0000","last_change":"2023-10-18T08:55:15.307031+0000","last_active":"2023-10-18T08:55:16.691934+0000","last_peered":"2023-10-18T08:55:16.691934+0000","last_clean":"2023-10-18T08:55:16.691934+0000","last_became_active":"2023-10-18T08:55:15.306786+0000","last_became_peered":"2023-10-18T08:55:15.306786+0000","last_unstale":"2023-10-18T08:55:16.691934+0000","last_undegraded":"2023-10-18T08:55:16.691934+0000","last_fullsized":"2023-10-18T08:55:16.691934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:36.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689019+0000","last_change":"2023-10-18T08:55:15.321350+0000","last_active":"2023-10-18T08:55:16.689019+0000","last_peered":"2023-10-18T08:55:16.689019+0000","last_clean":"2023-10-18T08:55:16.689019+0000","last_became_active":"2023-10-18T08:55:15.321114+0000","last_became_peered":"2023-10-18T08:55:15.321114+0000","last_unstale":"2023-10-18T08:55:16.689019+0000","last_undegraded":"2023-10-18T08:55:16.689019+0000","last_fullsized":"2023-10-18T08:55:16.689019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:02.173932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689997+0000","last_change":"2023-10-18T08:55:15.307879+0000","last_active":"2023-10-18T08:55:16.689997+0000","last_peered":"2023-10-18T08:55:16.689997+0000","last_clean":"2023-10-18T08:55:16.689997+0000","last_became_active":"2023-10-18T08:55:15.307599+0000","last_became_peered":"2023-10-18T08:55:15.307599+0000","last_unstale":"2023-10-18T08:55:16.689997+0000","last_undegraded":"2023-10-18T08:55:16.689997+0000","last_fullsized":"2023-10-18T08:55:16.689997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:02.474966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280131+0000","last_change":"2023-10-18T08:55:15.306959+0000","last_active":"2023-10-18T08:55:16.280131+0000","last_peered":"2023-10-18T08:55:16.280131+0000","last_clean":"2023-10-18T08:55:16.280131+0000","last_became_active":"2023-10-18T08:55:15.306003+0000","last_became_peered":"2023-10-18T08:55:15.306003+0000","last_unstale":"2023-10-18T08:55:16.280131+0000","last_undegraded":"2023-10-18T08:55:16.280131+0000","last_fullsized":"2023-10-18T08:55:16.280131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:06.219128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688286+0000","last_change":"2023-10-18T08:55:15.302306+0000","last_active":"2023-10-18T08:55:16.688286+0000","last_peered":"2023-10-18T08:55:16.688286+0000","last_clean":"2023-10-18T08:55:16.688286+0000","last_became_active":"2023-10-18T08:55:15.302027+0000","last_became_peered":"2023-10-18T08:55:15.302027+0000","last_unstale":"2023-10-18T08:55:16.688286+0000","last_undegraded":"2023-10-18T08:55:16.688286+0000","last_fullsized":"2023-10-18T08:55:16.688286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:45.159195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282045+0000","last_change":"2023-10-18T08:55:15.316075+0000","last_active":"2023-10-18T08:55:16.282045+0000","last_peered":"2023-10-18T08:55:16.282045+0000","last_clean":"2023-10-18T08:55:16.282045+0000","last_became_active":"2023-10-18T08:55:15.315569+0000","last_became_peered":"2023-10-18T08:55:15.315569+0000","last_unstale":"2023-10-18T08:55:16.282045+0000","last_undegraded":"2023-10-18T08:55:16.282045+0000","last_fullsized":"2023-10-18T08:55:16.282045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691359+0000","last_change":"2023-10-18T08:55:15.320240+0000","last_active":"2023-10-18T08:55:16.691359+0000","last_peered":"2023-10-18T08:55:16.691359+0000","last_clean":"2023-10-18T08:55:16.691359+0000","last_became_active":"2023-10-18T08:55:15.319558+0000","last_became_peered":"2023-10-18T08:55:15.319558+0000","last_unstale":"2023-10-18T08:55:16.691359+0000","last_undegraded":"2023-10-18T08:55:16.691359+0000","last_fullsized":"2023-10-18T08:55:16.691359+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:10.355168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690434+0000","last_change":"2023-10-18T08:55:15.315655+0000","last_active":"2023-10-18T08:55:16.690434+0000","last_peered":"2023-10-18T08:55:16.690434+0000","last_clean":"2023-10-18T08:55:16.690434+0000","last_became_active":"2023-10-18T08:55:15.311919+0000","last_became_peered":"2023-10-18T08:55:15.311919+0000","last_unstale":"2023-10-18T08:55:16.690434+0000","last_undegraded":"2023-10-18T08:55:16.690434+0000","last_fullsized":"2023-10-18T08:55:16.690434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:23:50.828764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.687786+0000","last_change":"2023-10-18T08:55:15.306321+0000","last_active":"2023-10-18T08:55:16.687786+0000","last_peered":"2023-10-18T08:55:16.687786+0000","last_clean":"2023-10-18T08:55:16.687786+0000","last_became_active":"2023-10-18T08:55:15.305696+0000","last_became_peered":"2023-10-18T08:55:15.305696+0000","last_unstale":"2023-10-18T08:55:16.687786+0000","last_undegraded":"2023-10-18T08:55:16.687786+0000","last_fullsized":"2023-10-18T08:55:16.687786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:05.190948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282965+0000","last_change":"2023-10-18T08:55:15.294264+0000","last_active":"2023-10-18T08:55:16.282965+0000","last_peered":"2023-10-18T08:55:16.282965+0000","last_clean":"2023-10-18T08:55:16.282965+0000","last_became_active":"2023-10-18T08:55:15.293654+0000","last_became_peered":"2023-10-18T08:55:15.293654+0000","last_unstale":"2023-10-18T08:55:16.282965+0000","last_undegraded":"2023-10-18T08:55:16.282965+0000","last_fullsized":"2023-10-18T08:55:16.282965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:26.310762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.692252+0000","last_change":"2023-10-18T08:55:15.315803+0000","last_active":"2023-10-18T08:55:16.692252+0000","last_peered":"2023-10-18T08:55:16.692252+0000","last_clean":"2023-10-18T08:55:16.692252+0000","last_became_active":"2023-10-18T08:55:15.315544+0000","last_became_peered":"2023-10-18T08:55:15.315544+0000","last_unstale":"2023-10-18T08:55:16.692252+0000","last_undegraded":"2023-10-18T08:55:16.692252+0000","last_fullsized":"2023-10-18T08:55:16.692252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689490+0000","last_change":"2023-10-18T08:55:15.308154+0000","last_active":"2023-10-18T08:55:16.689490+0000","last_peered":"2023-10-18T08:55:16.689490+0000","last_clean":"2023-10-18T08:55:16.689490+0000","last_became_active":"2023-10-18T08:55:15.307592+0000","last_became_peered":"2023-10-18T08:55:15.307592+0000","last_unstale":"2023-10-18T08:55:16.689490+0000","last_undegraded":"2023-10-18T08:55:16.689490+0000","last_fullsized":"2023-10-18T08:55:16.689490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:06.219128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691182+0000","last_change":"2023-10-18T08:55:15.312342+0000","last_active":"2023-10-18T08:55:16.691182+0000","last_peered":"2023-10-18T08:55:16.691182+0000","last_clean":"2023-10-18T08:55:16.691182+0000","last_became_active":"2023-10-18T08:55:15.311286+0000","last_became_peered":"2023-10-18T08:55:15.311286+0000","last_unstale":"2023-10-18T08:55:16.691182+0000","last_undegraded":"2023-10-18T08:55:16.691182+0000","last_fullsized":"2023-10-18T08:55:16.691182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:07:58.310623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280368+0000","last_change":"2023-10-18T08:55:15.307636+0000","last_active":"2023-10-18T08:55:16.280368+0000","last_peered":"2023-10-18T08:55:16.280368+0000","last_clean":"2023-10-18T08:55:16.280368+0000","last_became_active":"2023-10-18T08:55:15.306703+0000","last_became_peered":"2023-10-18T08:55:15.306703+0000","last_unstale":"2023-10-18T08:55:16.280368+0000","last_undegraded":"2023-10-18T08:55:16.280368+0000","last_fullsized":"2023-10-18T08:55:16.280368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:36.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689173+0000","last_change":"2023-10-18T08:55:15.307528+0000","last_active":"2023-10-18T08:55:16.689173+0000","last_peered":"2023-10-18T08:55:16.689173+0000","last_clean":"2023-10-18T08:55:16.689173+0000","last_became_active":"2023-10-18T08:55:15.306757+0000","last_became_peered":"2023-10-18T08:55:15.306757+0000","last_unstale":"2023-10-18T08:55:16.689173+0000","last_undegraded":"2023-10-18T08:55:16.689173+0000","last_fullsized":"2023-10-18T08:55:16.689173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:31.888781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280839+0000","last_change":"2023-10-18T08:55:15.316136+0000","last_active":"2023-10-18T08:55:16.280839+0000","last_peered":"2023-10-18T08:55:16.280839+0000","last_clean":"2023-10-18T08:55:16.280839+0000","last_became_active":"2023-10-18T08:55:15.315888+0000","last_became_peered":"2023-10-18T08:55:15.315888+0000","last_unstale":"2023-10-18T08:55:16.280839+0000","last_undegraded":"2023-10-18T08:55:16.280839+0000","last_fullsized":"2023-10-18T08:55:16.280839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:23:50.828764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689385+0000","last_change":"2023-10-18T08:55:15.309215+0000","last_active":"2023-10-18T08:55:16.689385+0000","last_peered":"2023-10-18T08:55:16.689385+0000","last_clean":"2023-10-18T08:55:16.689385+0000","last_became_active":"2023-10-18T08:55:15.308858+0000","last_became_peered":"2023-10-18T08:55:15.308858+0000","last_unstale":"2023-10-18T08:55:16.689385+0000","last_undegraded":"2023-10-18T08:55:16.689385+0000","last_fullsized":"2023-10-18T08:55:16.689385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:06.219128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281659+0000","last_change":"2023-10-18T08:55:15.308660+0000","last_active":"2023-10-18T08:55:16.281659+0000","last_peered":"2023-10-18T08:55:16.281659+0000","last_clean":"2023-10-18T08:55:16.281659+0000","last_became_active":"2023-10-18T08:55:15.306230+0000","last_became_peered":"2023-10-18T08:55:15.306230+0000","last_unstale":"2023-10-18T08:55:16.281659+0000","last_undegraded":"2023-10-18T08:55:16.281659+0000","last_fullsized":"2023-10-18T08:55:16.281659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:06.219128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280534+0000","last_change":"2023-10-18T08:55:15.307486+0000","last_active":"2023-10-18T08:55:16.280534+0000","last_peered":"2023-10-18T08:55:16.280534+0000","last_clean":"2023-10-18T08:55:16.280534+0000","last_became_active":"2023-10-18T08:55:15.306357+0000","last_became_peered":"2023-10-18T08:55:15.306357+0000","last_unstale":"2023-10-18T08:55:16.280534+0000","last_undegraded":"2023-10-18T08:55:16.280534+0000","last_fullsized":"2023-10-18T08:55:16.280534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:31.888781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688669+0000","last_change":"2023-10-18T08:55:15.300645+0000","last_active":"2023-10-18T08:55:16.688669+0000","last_peered":"2023-10-18T08:55:16.688669+0000","last_clean":"2023-10-18T08:55:16.688669+0000","last_became_active":"2023-10-18T08:55:15.299795+0000","last_became_peered":"2023-10-18T08:55:15.299795+0000","last_unstale":"2023-10-18T08:55:16.688669+0000","last_undegraded":"2023-10-18T08:55:16.688669+0000","last_fullsized":"2023-10-18T08:55:16.688669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:21.842439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690857+0000","last_change":"2023-10-18T08:55:15.305929+0000","last_active":"2023-10-18T08:55:16.690857+0000","last_peered":"2023-10-18T08:55:16.690857+0000","last_clean":"2023-10-18T08:55:16.690857+0000","last_became_active":"2023-10-18T08:55:15.301322+0000","last_became_peered":"2023-10-18T08:55:15.301322+0000","last_unstale":"2023-10-18T08:55:16.690857+0000","last_undegraded":"2023-10-18T08:55:16.690857+0000","last_fullsized":"2023-10-18T08:55:16.690857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:41.755639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.687612+0000","last_change":"2023-10-18T08:55:15.320868+0000","last_active":"2023-10-18T08:55:16.687612+0000","last_peered":"2023-10-18T08:55:16.687612+0000","last_clean":"2023-10-18T08:55:16.687612+0000","last_became_active":"2023-10-18T08:55:15.320606+0000","last_became_peered":"2023-10-18T08:55:15.320606+0000","last_unstale":"2023-10-18T08:55:16.687612+0000","last_undegraded":"2023-10-18T08:55:16.687612+0000","last_fullsized":"2023-10-18T08:55:16.687612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:04.364364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690001+0000","last_change":"2023-10-18T08:55:15.314269+0000","last_active":"2023-10-18T08:55:16.690001+0000","last_peered":"2023-10-18T08:55:16.690001+0000","last_clean":"2023-10-18T08:55:16.690001+0000","last_became_active":"2023-10-18T08:55:15.314011+0000","last_became_peered":"2023-10-18T08:55:15.314011+0000","last_unstale":"2023-10-18T08:55:16.690001+0000","last_undegraded":"2023-10-18T08:55:16.690001+0000","last_fullsized":"2023-10-18T08:55:16.690001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:28.148282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689559+0000","last_change":"2023-10-18T08:55:15.306134+0000","last_active":"2023-10-18T08:55:16.689559+0000","last_peered":"2023-10-18T08:55:16.689559+0000","last_clean":"2023-10-18T08:55:16.689559+0000","last_became_active":"2023-10-18T08:55:15.305389+0000","last_became_peered":"2023-10-18T08:55:15.305389+0000","last_unstale":"2023-10-18T08:55:16.689559+0000","last_undegraded":"2023-10-18T08:55:16.689559+0000","last_fullsized":"2023-10-18T08:55:16.689559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:32.273116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.327693+0000","last_change":"2023-10-18T08:55:14.290336+0000","last_active":"2023-10-18T08:55:15.327693+0000","last_peered":"2023-10-18T08:55:15.327693+0000","last_clean":"2023-10-18T08:55:15.327693+0000","last_became_active":"2023-10-18T08:55:14.290051+0000","last_became_peered":"2023-10-18T08:55:14.290051+0000","last_unstale":"2023-10-18T08:55:15.327693+0000","last_undegraded":"2023-10-18T08:55:15.327693+0000","last_fullsized":"2023-10-18T08:55:15.327693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:38.968705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689230+0000","last_change":"2023-10-18T08:55:15.302202+0000","last_active":"2023-10-18T08:55:16.689230+0000","last_peered":"2023-10-18T08:55:16.689230+0000","last_clean":"2023-10-18T08:55:16.689230+0000","last_became_active":"2023-10-18T08:55:15.301534+0000","last_became_peered":"2023-10-18T08:55:15.301534+0000","last_unstale":"2023-10-18T08:55:16.689230+0000","last_undegraded":"2023-10-18T08:55:16.689230+0000","last_fullsized":"2023-10-18T08:55:16.689230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:27.650596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.326541+0000","last_change":"2023-10-18T08:55:14.296292+0000","last_active":"2023-10-18T08:55:15.326541+0000","last_peered":"2023-10-18T08:55:15.326541+0000","last_clean":"2023-10-18T08:55:15.326541+0000","last_became_active":"2023-10-18T08:55:14.295937+0000","last_became_peered":"2023-10-18T08:55:14.295937+0000","last_unstale":"2023-10-18T08:55:15.326541+0000","last_undegraded":"2023-10-18T08:55:15.326541+0000","last_fullsized":"2023-10-18T08:55:15.326541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:38.968705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.280268+0000","last_change":"2023-10-18T08:55:14.293089+0000","last_active":"2023-10-18T08:55:15.280268+0000","last_peered":"2023-10-18T08:55:15.280268+0000","last_clean":"2023-10-18T08:55:15.280268+0000","last_became_active":"2023-10-18T08:55:14.292273+0000","last_became_peered":"2023-10-18T08:55:14.292273+0000","last_unstale":"2023-10-18T08:55:15.280268+0000","last_undegraded":"2023-10-18T08:55:15.280268+0000","last_fullsized":"2023-10-18T08:55:15.280268+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:03.359656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281004+0000","last_change":"2023-10-18T08:55:15.297969+0000","last_active":"2023-10-18T08:55:16.281004+0000","last_peered":"2023-10-18T08:55:16.281004+0000","last_clean":"2023-10-18T08:55:16.281004+0000","last_became_active":"2023-10-18T08:55:15.297730+0000","last_became_peered":"2023-10-18T08:55:15.297730+0000","last_unstale":"2023-10-18T08:55:16.281004+0000","last_undegraded":"2023-10-18T08:55:16.281004+0000","last_fullsized":"2023-10-18T08:55:16.281004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:04.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.301050+0000","last_change":"2023-10-18T08:55:14.299406+0000","last_active":"2023-10-18T08:55:15.301050+0000","last_peered":"2023-10-18T08:55:15.301050+0000","last_clean":"2023-10-18T08:55:15.301050+0000","last_became_active":"2023-10-18T08:55:14.299181+0000","last_became_peered":"2023-10-18T08:55:14.299181+0000","last_unstale":"2023-10-18T08:55:15.301050+0000","last_undegraded":"2023-10-18T08:55:15.301050+0000","last_fullsized":"2023-10-18T08:55:15.301050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:18.874528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281450+0000","last_change":"2023-10-18T08:55:15.309032+0000","last_active":"2023-10-18T08:55:16.281450+0000","last_peered":"2023-10-18T08:55:16.281450+0000","last_clean":"2023-10-18T08:55:16.281450+0000","last_became_active":"2023-10-18T08:55:15.308756+0000","last_became_peered":"2023-10-18T08:55:15.308756+0000","last_unstale":"2023-10-18T08:55:16.281450+0000","last_undegraded":"2023-10-18T08:55:16.281450+0000","last_fullsized":"2023-10-18T08:55:16.281450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:27.650596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.297946+0000","last_change":"2023-10-18T08:55:14.304174+0000","last_active":"2023-10-18T08:55:15.297946+0000","last_peered":"2023-10-18T08:55:15.297946+0000","last_clean":"2023-10-18T08:55:15.297946+0000","last_became_active":"2023-10-18T08:55:14.303968+0000","last_became_peered":"2023-10-18T08:55:14.303968+0000","last_unstale":"2023-10-18T08:55:15.297946+0000","last_undegraded":"2023-10-18T08:55:15.297946+0000","last_fullsized":"2023-10-18T08:55:15.297946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:04.186241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690815+0000","last_change":"2023-10-18T08:55:15.312484+0000","last_active":"2023-10-18T08:55:16.690815+0000","last_peered":"2023-10-18T08:55:16.690815+0000","last_clean":"2023-10-18T08:55:16.690815+0000","last_became_active":"2023-10-18T08:55:15.311626+0000","last_became_peered":"2023-10-18T08:55:15.311626+0000","last_unstale":"2023-10-18T08:55:16.690815+0000","last_undegraded":"2023-10-18T08:55:16.690815+0000","last_fullsized":"2023-10-18T08:55:16.690815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:19:04.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.300021+0000","last_change":"2023-10-18T08:55:14.302651+0000","last_active":"2023-10-18T08:55:15.300021+0000","last_peered":"2023-10-18T08:55:15.300021+0000","last_clean":"2023-10-18T08:55:15.300021+0000","last_became_active":"2023-10-18T08:55:14.302029+0000","last_became_peered":"2023-10-18T08:55:14.302029+0000","last_unstale":"2023-10-18T08:55:15.300021+0000","last_undegraded":"2023-10-18T08:55:15.300021+0000","last_fullsized":"2023-10-18T08:55:15.300021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:44.154488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689826+0000","last_change":"2023-10-18T08:55:15.322350+0000","last_active":"2023-10-18T08:55:16.689826+0000","last_peered":"2023-10-18T08:55:16.689826+0000","last_clean":"2023-10-18T08:55:16.689826+0000","last_became_active":"2023-10-18T08:55:15.322002+0000","last_became_peered":"2023-10-18T08:55:15.322002+0000","last_unstale":"2023-10-18T08:55:16.689826+0000","last_undegraded":"2023-10-18T08:55:16.689826+0000","last_fullsized":"2023-10-18T08:55:16.689826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:36.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.326702+0000","last_change":"2023-10-18T08:55:14.300549+0000","last_active":"2023-10-18T08:55:15.326702+0000","last_peered":"2023-10-18T08:55:15.326702+0000","last_clean":"2023-10-18T08:55:15.326702+0000","last_became_active":"2023-10-18T08:55:14.299990+0000","last_became_peered":"2023-10-18T08:55:14.299990+0000","last_unstale":"2023-10-18T08:55:15.326702+0000","last_undegraded":"2023-10-18T08:55:15.326702+0000","last_fullsized":"2023-10-18T08:55:15.326702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:37.505319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690659+0000","last_change":"2023-10-18T08:55:15.314324+0000","last_active":"2023-10-18T08:55:16.690659+0000","last_peered":"2023-10-18T08:55:16.690659+0000","last_clean":"2023-10-18T08:55:16.690659+0000","last_became_active":"2023-10-18T08:55:15.313890+0000","last_became_peered":"2023-10-18T08:55:15.313890+0000","last_unstale":"2023-10-18T08:55:16.690659+0000","last_undegraded":"2023-10-18T08:55:16.690659+0000","last_fullsized":"2023-10-18T08:55:16.690659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:05.190948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.329951+0000","last_change":"2023-10-18T08:55:14.292962+0000","last_active":"2023-10-18T08:55:15.329951+0000","last_peered":"2023-10-18T08:55:15.329951+0000","last_clean":"2023-10-18T08:55:15.329951+0000","last_became_active":"2023-10-18T08:55:14.292424+0000","last_became_peered":"2023-10-18T08:55:14.292424+0000","last_unstale":"2023-10-18T08:55:15.329951+0000","last_undegraded":"2023-10-18T08:55:15.329951+0000","last_fullsized":"2023-10-18T08:55:15.329951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:20.837731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282227+0000","last_change":"2023-10-18T08:55:15.296745+0000","last_active":"2023-10-18T08:55:16.282227+0000","last_peered":"2023-10-18T08:55:16.282227+0000","last_clean":"2023-10-18T08:55:16.282227+0000","last_became_active":"2023-10-18T08:55:15.296095+0000","last_became_peered":"2023-10-18T08:55:15.296095+0000","last_unstale":"2023-10-18T08:55:16.282227+0000","last_undegraded":"2023-10-18T08:55:16.282227+0000","last_fullsized":"2023-10-18T08:55:16.282227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.688019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.328191+0000","last_change":"2023-10-18T08:55:14.299318+0000","last_active":"2023-10-18T08:55:15.328191+0000","last_peered":"2023-10-18T08:55:15.328191+0000","last_clean":"2023-10-18T08:55:15.328191+0000","last_became_active":"2023-10-18T08:55:14.299069+0000","last_became_peered":"2023-10-18T08:55:14.299069+0000","last_unstale":"2023-10-18T08:55:15.328191+0000","last_undegraded":"2023-10-18T08:55:15.328191+0000","last_fullsized":"2023-10-18T08:55:15.328191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:03.359656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690146+0000","last_change":"2023-10-18T08:55:15.307886+0000","last_active":"2023-10-18T08:55:16.690146+0000","last_peered":"2023-10-18T08:55:16.690146+0000","last_clean":"2023-10-18T08:55:16.690146+0000","last_became_active":"2023-10-18T08:55:15.306465+0000","last_became_peered":"2023-10-18T08:55:15.306465+0000","last_unstale":"2023-10-18T08:55:16.690146+0000","last_undegraded":"2023-10-18T08:55:16.690146+0000","last_fullsized":"2023-10-18T08:55:16.690146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:31.888781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.316152+0000","last_change":"2023-10-18T08:55:14.305151+0000","last_active":"2023-10-18T08:55:15.316152+0000","last_peered":"2023-10-18T08:55:15.316152+0000","last_clean":"2023-10-18T08:55:15.316152+0000","last_became_active":"2023-10-18T08:55:14.303161+0000","last_became_peered":"2023-10-18T08:55:14.303161+0000","last_unstale":"2023-10-18T08:55:15.316152+0000","last_undegraded":"2023-10-18T08:55:15.316152+0000","last_fullsized":"2023-10-18T08:55:15.316152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:26.645889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.283192+0000","last_change":"2023-10-18T08:55:15.317138+0000","last_active":"2023-10-18T08:55:16.283192+0000","last_peered":"2023-10-18T08:55:16.283192+0000","last_clean":"2023-10-18T08:55:16.283192+0000","last_became_active":"2023-10-18T08:55:15.316743+0000","last_became_peered":"2023-10-18T08:55:15.316743+0000","last_unstale":"2023-10-18T08:55:16.283192+0000","last_undegraded":"2023-10-18T08:55:16.283192+0000","last_fullsized":"2023-10-18T08:55:16.283192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:46.921031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.303320+0000","last_change":"2023-10-18T08:55:14.294348+0000","last_active":"2023-10-18T08:55:15.303320+0000","last_peered":"2023-10-18T08:55:15.303320+0000","last_clean":"2023-10-18T08:55:15.303320+0000","last_became_active":"2023-10-18T08:55:14.293456+0000","last_became_peered":"2023-10-18T08:55:14.293456+0000","last_unstale":"2023-10-18T08:55:15.303320+0000","last_undegraded":"2023-10-18T08:55:15.303320+0000","last_fullsized":"2023-10-18T08:55:15.303320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:37.505319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689020+0000","last_change":"2023-10-18T08:55:15.304772+0000","last_active":"2023-10-18T08:55:16.689020+0000","last_peered":"2023-10-18T08:55:16.689020+0000","last_clean":"2023-10-18T08:55:16.689020+0000","last_became_active":"2023-10-18T08:55:15.304528+0000","last_became_peered":"2023-10-18T08:55:15.304528+0000","last_unstale":"2023-10-18T08:55:16.689020+0000","last_undegraded":"2023-10-18T08:55:16.689020+0000","last_fullsized":"2023-10-18T08:55:16.689020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:27.650596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.330488+0000","last_change":"2023-10-18T08:55:14.297308+0000","last_active":"2023-10-18T08:55:15.330488+0000","last_peered":"2023-10-18T08:55:15.330488+0000","last_clean":"2023-10-18T08:55:15.330488+0000","last_became_active":"2023-10-18T08:55:14.297069+0000","last_became_peered":"2023-10-18T08:55:14.297069+0000","last_unstale":"2023-10-18T08:55:15.330488+0000","last_undegraded":"2023-10-18T08:55:15.330488+0000","last_fullsized":"2023-10-18T08:55:15.330488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:10.814889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.283167+0000","last_change":"2023-10-18T08:55:15.303171+0000","last_active":"2023-10-18T08:55:16.283167+0000","last_peered":"2023-10-18T08:55:16.283167+0000","last_clean":"2023-10-18T08:55:16.283167+0000","last_became_active":"2023-10-18T08:55:15.301815+0000","last_became_peered":"2023-10-18T08:55:15.301815+0000","last_unstale":"2023-10-18T08:55:16.283167+0000","last_undegraded":"2023-10-18T08:55:16.283167+0000","last_fullsized":"2023-10-18T08:55:16.283167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:10.355168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.325923+0000","last_change":"2023-10-18T08:55:14.298243+0000","last_active":"2023-10-18T08:55:15.325923+0000","last_peered":"2023-10-18T08:55:15.325923+0000","last_clean":"2023-10-18T08:55:15.325923+0000","last_became_active":"2023-10-18T08:55:14.297343+0000","last_became_peered":"2023-10-18T08:55:14.297343+0000","last_unstale":"2023-10-18T08:55:15.325923+0000","last_undegraded":"2023-10-18T08:55:15.325923+0000","last_fullsized":"2023-10-18T08:55:15.325923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:10.814889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.283511+0000","last_change":"2023-10-18T08:55:15.316986+0000","last_active":"2023-10-18T08:55:16.283511+0000","last_peered":"2023-10-18T08:55:16.283511+0000","last_clean":"2023-10-18T08:55:16.283511+0000","last_became_active":"2023-10-18T08:55:15.316398+0000","last_became_peered":"2023-10-18T08:55:15.316398+0000","last_unstale":"2023-10-18T08:55:16.283511+0000","last_undegraded":"2023-10-18T08:55:16.283511+0000","last_fullsized":"2023-10-18T08:55:16.283511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:56:45.727823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.329694+0000","last_change":"2023-10-18T08:55:14.290070+0000","last_active":"2023-10-18T08:55:15.329694+0000","last_peered":"2023-10-18T08:55:15.329694+0000","last_clean":"2023-10-18T08:55:15.329694+0000","last_became_active":"2023-10-18T08:55:14.289756+0000","last_became_peered":"2023-10-18T08:55:14.289756+0000","last_unstale":"2023-10-18T08:55:15.329694+0000","last_undegraded":"2023-10-18T08:55:15.329694+0000","last_fullsized":"2023-10-18T08:55:15.329694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:01.470259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.282548+0000","last_change":"2023-10-18T08:55:15.311283+0000","last_active":"2023-10-18T08:55:16.282548+0000","last_peered":"2023-10-18T08:55:16.282548+0000","last_clean":"2023-10-18T08:55:16.282548+0000","last_became_active":"2023-10-18T08:55:15.311053+0000","last_became_peered":"2023-10-18T08:55:15.311053+0000","last_unstale":"2023-10-18T08:55:16.282548+0000","last_undegraded":"2023-10-18T08:55:16.282548+0000","last_fullsized":"2023-10-18T08:55:16.282548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:23:50.828764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.325915+0000","last_change":"2023-10-18T08:55:14.294635+0000","last_active":"2023-10-18T08:55:15.325915+0000","last_peered":"2023-10-18T08:55:15.325915+0000","last_clean":"2023-10-18T08:55:15.325915+0000","last_became_active":"2023-10-18T08:55:14.294341+0000","last_became_peered":"2023-10-18T08:55:14.294341+0000","last_unstale":"2023-10-18T08:55:15.325915+0000","last_undegraded":"2023-10-18T08:55:15.325915+0000","last_fullsized":"2023-10-18T08:55:15.325915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:01.470259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691402+0000","last_change":"2023-10-18T08:55:15.312145+0000","last_active":"2023-10-18T08:55:16.691402+0000","last_peered":"2023-10-18T08:55:16.691402+0000","last_clean":"2023-10-18T08:55:16.691402+0000","last_became_active":"2023-10-18T08:55:15.310296+0000","last_became_peered":"2023-10-18T08:55:15.310296+0000","last_unstale":"2023-10-18T08:55:16.691402+0000","last_undegraded":"2023-10-18T08:55:16.691402+0000","last_fullsized":"2023-10-18T08:55:16.691402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:46.921031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.282183+0000","last_change":"2023-10-18T08:55:14.303903+0000","last_active":"2023-10-18T08:55:15.282183+0000","last_peered":"2023-10-18T08:55:15.282183+0000","last_clean":"2023-10-18T08:55:15.282183+0000","last_became_active":"2023-10-18T08:55:14.303633+0000","last_became_peered":"2023-10-18T08:55:14.303633+0000","last_unstale":"2023-10-18T08:55:15.282183+0000","last_undegraded":"2023-10-18T08:55:15.282183+0000","last_fullsized":"2023-10-18T08:55:15.282183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:16:56.668322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.283316+0000","last_change":"2023-10-18T08:55:15.318598+0000","last_active":"2023-10-18T08:55:16.283316+0000","last_peered":"2023-10-18T08:55:16.283316+0000","last_clean":"2023-10-18T08:55:16.283316+0000","last_became_active":"2023-10-18T08:55:15.318353+0000","last_became_peered":"2023-10-18T08:55:15.318353+0000","last_unstale":"2023-10-18T08:55:16.283316+0000","last_undegraded":"2023-10-18T08:55:16.283316+0000","last_fullsized":"2023-10-18T08:55:16.283316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:10.917539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.287158+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:55:15.287158+0000","last_peered":"2023-10-18T08:55:15.287158+0000","last_clean":"2023-10-18T08:55:15.287158+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:55:15.287158+0000","last_undegraded":"2023-10-18T08:55:15.287158+0000","last_fullsized":"2023-10-18T08:55:15.287158+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:34.881744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.316324+0000","last_change":"2023-10-18T08:55:14.305589+0000","last_active":"2023-10-18T08:55:15.316324+0000","last_peered":"2023-10-18T08:55:15.316324+0000","last_clean":"2023-10-18T08:55:15.316324+0000","last_became_active":"2023-10-18T08:55:14.303723+0000","last_became_peered":"2023-10-18T08:55:14.303723+0000","last_unstale":"2023-10-18T08:55:15.316324+0000","last_undegraded":"2023-10-18T08:55:15.316324+0000","last_fullsized":"2023-10-18T08:55:15.316324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:05.214421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280180+0000","last_change":"2023-10-18T08:55:15.307598+0000","last_active":"2023-10-18T08:55:16.280180+0000","last_peered":"2023-10-18T08:55:16.280180+0000","last_clean":"2023-10-18T08:55:16.280180+0000","last_became_active":"2023-10-18T08:55:15.307263+0000","last_became_peered":"2023-10-18T08:55:15.307263+0000","last_unstale":"2023-10-18T08:55:16.280180+0000","last_undegraded":"2023-10-18T08:55:16.280180+0000","last_fullsized":"2023-10-18T08:55:16.280180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:28.148282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.286898+0000","last_change":"2023-10-18T08:55:14.299430+0000","last_active":"2023-10-18T08:55:15.286898+0000","last_peered":"2023-10-18T08:55:15.286898+0000","last_clean":"2023-10-18T08:55:15.286898+0000","last_became_active":"2023-10-18T08:55:14.299004+0000","last_became_peered":"2023-10-18T08:55:14.299004+0000","last_unstale":"2023-10-18T08:55:15.286898+0000","last_undegraded":"2023-10-18T08:55:15.286898+0000","last_fullsized":"2023-10-18T08:55:15.286898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:26.645889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689865+0000","last_change":"2023-10-18T08:55:15.308337+0000","last_active":"2023-10-18T08:55:16.689865+0000","last_peered":"2023-10-18T08:55:16.689865+0000","last_clean":"2023-10-18T08:55:16.689865+0000","last_became_active":"2023-10-18T08:55:15.305146+0000","last_became_peered":"2023-10-18T08:55:15.305146+0000","last_unstale":"2023-10-18T08:55:16.689865+0000","last_undegraded":"2023-10-18T08:55:16.689865+0000","last_fullsized":"2023-10-18T08:55:16.689865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.283153+0000","last_change":"2023-10-18T08:55:14.287811+0000","last_active":"2023-10-18T08:55:15.283153+0000","last_peered":"2023-10-18T08:55:15.283153+0000","last_clean":"2023-10-18T08:55:15.283153+0000","last_became_active":"2023-10-18T08:55:14.287466+0000","last_became_peered":"2023-10-18T08:55:14.287466+0000","last_unstale":"2023-10-18T08:55:15.283153+0000","last_undegraded":"2023-10-18T08:55:15.283153+0000","last_fullsized":"2023-10-18T08:55:15.283153+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:31.268409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281786+0000","last_change":"2023-10-18T08:55:15.301141+0000","last_active":"2023-10-18T08:55:16.281786+0000","last_peered":"2023-10-18T08:55:16.281786+0000","last_clean":"2023-10-18T08:55:16.281786+0000","last_became_active":"2023-10-18T08:55:15.300535+0000","last_became_peered":"2023-10-18T08:55:15.300535+0000","last_unstale":"2023-10-18T08:55:16.281786+0000","last_undegraded":"2023-10-18T08:55:16.281786+0000","last_fullsized":"2023-10-18T08:55:16.281786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:28.148282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.300341+0000","last_change":"2023-10-18T08:55:14.302819+0000","last_active":"2023-10-18T08:55:15.300341+0000","last_peered":"2023-10-18T08:55:15.300341+0000","last_clean":"2023-10-18T08:55:15.300341+0000","last_became_active":"2023-10-18T08:55:14.302381+0000","last_became_peered":"2023-10-18T08:55:14.302381+0000","last_unstale":"2023-10-18T08:55:15.300341+0000","last_undegraded":"2023-10-18T08:55:15.300341+0000","last_fullsized":"2023-10-18T08:55:15.300341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:20.837731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.690460+0000","last_change":"2023-10-18T08:55:15.309720+0000","last_active":"2023-10-18T08:55:16.690460+0000","last_peered":"2023-10-18T08:55:16.690460+0000","last_clean":"2023-10-18T08:55:16.690460+0000","last_became_active":"2023-10-18T08:55:15.309469+0000","last_became_peered":"2023-10-18T08:55:15.309469+0000","last_unstale":"2023-10-18T08:55:16.690460+0000","last_undegraded":"2023-10-18T08:55:16.690460+0000","last_fullsized":"2023-10-18T08:55:16.690460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:04.364364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.295166+0000","last_change":"2023-10-18T08:55:14.296298+0000","last_active":"2023-10-18T08:55:15.295166+0000","last_peered":"2023-10-18T08:55:15.295166+0000","last_clean":"2023-10-18T08:55:15.295166+0000","last_became_active":"2023-10-18T08:55:14.295788+0000","last_became_peered":"2023-10-18T08:55:14.295788+0000","last_unstale":"2023-10-18T08:55:15.295166+0000","last_undegraded":"2023-10-18T08:55:15.295166+0000","last_fullsized":"2023-10-18T08:55:15.295166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:38.968705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689511+0000","last_change":"2023-10-18T08:55:15.307391+0000","last_active":"2023-10-18T08:55:16.689511+0000","last_peered":"2023-10-18T08:55:16.689511+0000","last_clean":"2023-10-18T08:55:16.689511+0000","last_became_active":"2023-10-18T08:55:15.306422+0000","last_became_peered":"2023-10-18T08:55:15.306422+0000","last_unstale":"2023-10-18T08:55:16.689511+0000","last_undegraded":"2023-10-18T08:55:16.689511+0000","last_fullsized":"2023-10-18T08:55:16.689511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.688019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.286110+0000","last_change":"2023-10-18T08:55:14.299281+0000","last_active":"2023-10-18T08:55:15.286110+0000","last_peered":"2023-10-18T08:55:15.286110+0000","last_clean":"2023-10-18T08:55:15.286110+0000","last_became_active":"2023-10-18T08:55:14.298698+0000","last_became_peered":"2023-10-18T08:55:14.298698+0000","last_unstale":"2023-10-18T08:55:15.286110+0000","last_undegraded":"2023-10-18T08:55:15.286110+0000","last_fullsized":"2023-10-18T08:55:15.286110+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:44.154488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281874+0000","last_change":"2023-10-18T08:55:15.299892+0000","last_active":"2023-10-18T08:55:16.281874+0000","last_peered":"2023-10-18T08:55:16.281874+0000","last_clean":"2023-10-18T08:55:16.281874+0000","last_became_active":"2023-10-18T08:55:15.299281+0000","last_became_peered":"2023-10-18T08:55:15.299281+0000","last_unstale":"2023-10-18T08:55:16.281874+0000","last_undegraded":"2023-10-18T08:55:16.281874+0000","last_fullsized":"2023-10-18T08:55:16.281874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:36.202332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.281731+0000","last_change":"2023-10-18T08:55:14.300359+0000","last_active":"2023-10-18T08:55:16.281731+0000","last_peered":"2023-10-18T08:55:16.281731+0000","last_clean":"2023-10-18T08:55:16.281731+0000","last_became_active":"2023-10-18T08:55:14.299423+0000","last_became_peered":"2023-10-18T08:55:14.299423+0000","last_unstale":"2023-10-18T08:55:16.281731+0000","last_undegraded":"2023-10-18T08:55:16.281731+0000","last_fullsized":"2023-10-18T08:55:16.281731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:44.154488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280011+0000","last_change":"2023-10-18T08:55:15.288327+0000","last_active":"2023-10-18T08:55:16.280011+0000","last_peered":"2023-10-18T08:55:16.280011+0000","last_clean":"2023-10-18T08:55:16.280011+0000","last_became_active":"2023-10-18T08:55:15.287648+0000","last_became_peered":"2023-10-18T08:55:15.287648+0000","last_unstale":"2023-10-18T08:55:16.280011+0000","last_undegraded":"2023-10-18T08:55:16.280011+0000","last_fullsized":"2023-10-18T08:55:16.280011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:27.650596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.327662+0000","last_change":"2023-10-18T08:55:14.294992+0000","last_active":"2023-10-18T08:55:15.327662+0000","last_peered":"2023-10-18T08:55:15.327662+0000","last_clean":"2023-10-18T08:55:15.327662+0000","last_became_active":"2023-10-18T08:55:14.294682+0000","last_became_peered":"2023-10-18T08:55:14.294682+0000","last_unstale":"2023-10-18T08:55:15.327662+0000","last_undegraded":"2023-10-18T08:55:15.327662+0000","last_fullsized":"2023-10-18T08:55:15.327662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:04:44.154488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689214+0000","last_change":"2023-10-18T08:55:15.308496+0000","last_active":"2023-10-18T08:55:16.689214+0000","last_peered":"2023-10-18T08:55:16.689214+0000","last_clean":"2023-10-18T08:55:16.689214+0000","last_became_active":"2023-10-18T08:55:15.306095+0000","last_became_peered":"2023-10-18T08:55:15.306095+0000","last_unstale":"2023-10-18T08:55:16.689214+0000","last_undegraded":"2023-10-18T08:55:16.689214+0000","last_fullsized":"2023-10-18T08:55:16.689214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:02.173932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.288529+0000","last_change":"2023-10-18T08:55:14.299395+0000","last_active":"2023-10-18T08:55:15.288529+0000","last_peered":"2023-10-18T08:55:15.288529+0000","last_clean":"2023-10-18T08:55:15.288529+0000","last_became_active":"2023-10-18T08:55:14.299130+0000","last_became_peered":"2023-10-18T08:55:14.299130+0000","last_unstale":"2023-10-18T08:55:15.288529+0000","last_undegraded":"2023-10-18T08:55:15.288529+0000","last_fullsized":"2023-10-18T08:55:15.288529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:35.197625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.693084+0000","last_change":"2023-10-18T08:55:15.314704+0000","last_active":"2023-10-18T08:55:16.693084+0000","last_peered":"2023-10-18T08:55:16.693084+0000","last_clean":"2023-10-18T08:55:16.693084+0000","last_became_active":"2023-10-18T08:55:15.314153+0000","last_became_peered":"2023-10-18T08:55:15.314153+0000","last_unstale":"2023-10-18T08:55:16.693084+0000","last_undegraded":"2023-10-18T08:55:16.693084+0000","last_fullsized":"2023-10-18T08:55:16.693084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:39.973412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.295022+0000","last_change":"2023-10-18T08:55:14.297463+0000","last_active":"2023-10-18T08:55:15.295022+0000","last_peered":"2023-10-18T08:55:15.295022+0000","last_clean":"2023-10-18T08:55:15.295022+0000","last_became_active":"2023-10-18T08:55:14.297199+0000","last_became_peered":"2023-10-18T08:55:14.297199+0000","last_unstale":"2023-10-18T08:55:15.295022+0000","last_undegraded":"2023-10-18T08:55:15.295022+0000","last_fullsized":"2023-10-18T08:55:15.295022+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:04.186241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.279960+0000","last_change":"2023-10-18T08:55:15.303892+0000","last_active":"2023-10-18T08:55:16.279960+0000","last_peered":"2023-10-18T08:55:16.279960+0000","last_clean":"2023-10-18T08:55:16.279960+0000","last_became_active":"2023-10-18T08:55:15.303545+0000","last_became_peered":"2023-10-18T08:55:15.303545+0000","last_unstale":"2023-10-18T08:55:16.279960+0000","last_undegraded":"2023-10-18T08:55:16.279960+0000","last_fullsized":"2023-10-18T08:55:16.279960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:02.173932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.297222+0000","last_change":"2023-10-18T08:55:14.294024+0000","last_active":"2023-10-18T08:55:15.297222+0000","last_peered":"2023-10-18T08:55:15.297222+0000","last_clean":"2023-10-18T08:55:15.297222+0000","last_became_active":"2023-10-18T08:55:14.293763+0000","last_became_peered":"2023-10-18T08:55:14.293763+0000","last_unstale":"2023-10-18T08:55:15.297222+0000","last_undegraded":"2023-10-18T08:55:15.297222+0000","last_fullsized":"2023-10-18T08:55:15.297222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:30:20.837731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280694+0000","last_change":"2023-10-18T08:55:15.302069+0000","last_active":"2023-10-18T08:55:16.280694+0000","last_peered":"2023-10-18T08:55:16.280694+0000","last_clean":"2023-10-18T08:55:16.280694+0000","last_became_active":"2023-10-18T08:55:15.301806+0000","last_became_peered":"2023-10-18T08:55:15.301806+0000","last_unstale":"2023-10-18T08:55:16.280694+0000","last_undegraded":"2023-10-18T08:55:16.280694+0000","last_fullsized":"2023-10-18T08:55:16.280694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:03.688019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.327881+0000","last_change":"2023-10-18T08:55:14.292655+0000","last_active":"2023-10-18T08:55:15.327881+0000","last_peered":"2023-10-18T08:55:15.327881+0000","last_clean":"2023-10-18T08:55:15.327881+0000","last_became_active":"2023-10-18T08:55:14.292291+0000","last_became_peered":"2023-10-18T08:55:14.292291+0000","last_unstale":"2023-10-18T08:55:15.327881+0000","last_undegraded":"2023-10-18T08:55:15.327881+0000","last_fullsized":"2023-10-18T08:55:15.327881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:58:57.132126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.688748+0000","last_change":"2023-10-18T08:55:15.315320+0000","last_active":"2023-10-18T08:55:16.688748+0000","last_peered":"2023-10-18T08:55:16.688748+0000","last_clean":"2023-10-18T08:55:16.688748+0000","last_became_active":"2023-10-18T08:55:15.315007+0000","last_became_peered":"2023-10-18T08:55:15.315007+0000","last_unstale":"2023-10-18T08:55:16.688748+0000","last_undegraded":"2023-10-18T08:55:16.688748+0000","last_fullsized":"2023-10-18T08:55:16.688748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:19.879235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.691247+0000","last_change":"2023-10-18T08:55:15.317460+0000","last_active":"2023-10-18T08:55:16.691247+0000","last_peered":"2023-10-18T08:55:16.691247+0000","last_clean":"2023-10-18T08:55:16.691247+0000","last_became_active":"2023-10-18T08:55:15.317194+0000","last_became_peered":"2023-10-18T08:55:15.317194+0000","last_unstale":"2023-10-18T08:55:16.691247+0000","last_undegraded":"2023-10-18T08:55:16.691247+0000","last_fullsized":"2023-10-18T08:55:16.691247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:26.310762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.328212+0000","last_change":"2023-10-18T08:55:14.301102+0000","last_active":"2023-10-18T08:55:15.328212+0000","last_peered":"2023-10-18T08:55:15.328212+0000","last_clean":"2023-10-18T08:55:15.328212+0000","last_became_active":"2023-10-18T08:55:14.300855+0000","last_became_peered":"2023-10-18T08:55:14.300855+0000","last_unstale":"2023-10-18T08:55:15.328212+0000","last_undegraded":"2023-10-18T08:55:15.328212+0000","last_fullsized":"2023-10-18T08:55:15.328212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:27.143575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.692035+0000","last_change":"2023-10-18T08:55:15.320111+0000","last_active":"2023-10-18T08:55:16.692035+0000","last_peered":"2023-10-18T08:55:16.692035+0000","last_clean":"2023-10-18T08:55:16.692035+0000","last_became_active":"2023-10-18T08:55:15.319103+0000","last_became_peered":"2023-10-18T08:55:15.319103+0000","last_unstale":"2023-10-18T08:55:16.692035+0000","last_undegraded":"2023-10-18T08:55:16.692035+0000","last_fullsized":"2023-10-18T08:55:16.692035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:56:45.727823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.327995+0000","last_change":"2023-10-18T08:55:14.293731+0000","last_active":"2023-10-18T08:55:15.327995+0000","last_peered":"2023-10-18T08:55:15.327995+0000","last_clean":"2023-10-18T08:55:15.327995+0000","last_became_active":"2023-10-18T08:55:14.293420+0000","last_became_peered":"2023-10-18T08:55:14.293420+0000","last_unstale":"2023-10-18T08:55:15.327995+0000","last_undegraded":"2023-10-18T08:55:15.327995+0000","last_fullsized":"2023-10-18T08:55:15.327995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:01.169224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.689541+0000","last_change":"2023-10-18T08:55:15.307386+0000","last_active":"2023-10-18T08:55:16.689541+0000","last_peered":"2023-10-18T08:55:16.689541+0000","last_clean":"2023-10-18T08:55:16.689541+0000","last_became_active":"2023-10-18T08:55:15.304519+0000","last_became_peered":"2023-10-18T08:55:15.304519+0000","last_unstale":"2023-10-18T08:55:16.689541+0000","last_undegraded":"2023-10-18T08:55:16.689541+0000","last_fullsized":"2023-10-18T08:55:16.689541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:28.148282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.329282+0000","last_change":"2023-10-18T08:55:14.297189+0000","last_active":"2023-10-18T08:55:15.329282+0000","last_peered":"2023-10-18T08:55:15.329282+0000","last_clean":"2023-10-18T08:55:15.329282+0000","last_became_active":"2023-10-18T08:55:14.296794+0000","last_became_peered":"2023-10-18T08:55:14.296794+0000","last_unstale":"2023-10-18T08:55:15.329282+0000","last_undegraded":"2023-10-18T08:55:15.329282+0000","last_fullsized":"2023-10-18T08:55:15.329282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:10.814889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-18T08:55:16.280421+0000","last_change":"2023-10-18T08:55:15.309830+0000","last_active":"2023-10-18T08:55:16.280421+0000","last_peered":"2023-10-18T08:55:16.280421+0000","last_clean":"2023-10-18T08:55:16.280421+0000","last_became_active":"2023-10-18T08:55:15.309632+0000","last_became_peered":"2023-10-18T08:55:15.309632+0000","last_unstale":"2023-10-18T08:55:16.280421+0000","last_undegraded":"2023-10-18T08:55:16.280421+0000","last_fullsized":"2023-10-18T08:55:16.280421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:14.265731+0000","last_clean_scrub_stamp":"2023-10-18T08:55:14.265731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:41:20.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-18T08:55:15.328808+0000","last_change":"2023-10-18T08:55:14.297412+0000","last_active":"2023-10-18T08:55:15.328808+0000","last_peered":"2023-10-18T08:55:15.328808+0000","last_clean":"2023-10-18T08:55:15.328808+0000","last_became_active":"2023-10-18T08:55:14.296983+0000","last_became_peered":"2023-10-18T08:55:14.296983+0000","last_unstale":"2023-10-18T08:55:15.328808+0000","last_undegraded":"2023-10-18T08:55:15.328808+0000","last_fullsized":"2023-10-18T08:55:15.328808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:13.261024+0000","last_clean_scrub_stamp":"2023-10-18T08:55:13.261024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:38.968705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338312,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4550000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.341}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5249999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.698}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.841}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7549999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930155,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5149999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.796}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5309999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4670000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.585}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":68,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195244,"statfs":{"total":200000000,"available":199930121,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.657}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194667,"statfs":{"total":200000000,"available":199339275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4420000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7639999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.492}]}]}],"pool_statfs":[]}} 2023-10-18T08:55:31.627 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-18T08:55:31.627 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-18T08:55:31.627 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-18T08:55:31.627 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-18T08:55:31.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.892+0000 7f6d8d193640 1 -- 172.21.15.40:0/1606099764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d880b0dd0 msgr2=0x7f6d8815a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:31.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.892+0000 7f6d8d193640 1 --2- 172.21.15.40:0/1606099764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d880b0dd0 0x7f6d8815a350 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009830 tx=0x7f6d7802bcd0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.892+0000 7f6d8d193640 1 -- 172.21.15.40:0/1606099764 shutdown_connections 2023-10-18T08:55:31.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.892+0000 7f6d8d193640 1 --2- 172.21.15.40:0/1606099764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6d880b0070 0x7f6d8806b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.892+0000 7f6d8d193640 1 --2- 172.21.15.40:0/1606099764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d880b0dd0 0x7f6d8815a350 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.892+0000 7f6d8d193640 1 -- 172.21.15.40:0/1606099764 >> 172.21.15.40:0/1606099764 conn(0x7f6d880aed70 msgr2=0x7f6d880af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:31.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.892+0000 7f6d8d193640 1 -- 172.21.15.40:0/1606099764 shutdown_connections 2023-10-18T08:55:31.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.892+0000 7f6d8d193640 1 -- 172.21.15.40:0/1606099764 wait complete. 2023-10-18T08:55:31.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.894+0000 7f6d8d193640 1 Processor -- start 2023-10-18T08:55:31.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.895+0000 7f6d8d193640 1 -- start start 2023-10-18T08:55:31.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.896+0000 7f6d8d193640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d880b0070 0x7f6d88159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:31.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.896+0000 7f6d8d193640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d880b0dd0 0x7f6d880c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:31.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.896+0000 7f6d8d193640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6d8815a990 0x7f6d880a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:31.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.896+0000 7f6d8d193640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d88132b70 con 0x7f6d880b0070 2023-10-18T08:55:31.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.896+0000 7f6d8d193640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d88131f60 con 0x7f6d880b0dd0 2023-10-18T08:55:31.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.896+0000 7f6d8d193640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d88132240 con 0x7f6d8815a990 2023-10-18T08:55:31.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.896+0000 7f6d87fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d880b0070 0x7f6d88159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:31.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.896+0000 7f6d87fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d880b0070 0x7f6d88159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48198/0 (socket says 172.21.15.40:48198) 2023-10-18T08:55:31.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.896+0000 7f6d87fff640 1 -- 172.21.15.40:0/688283534 learned_addr learned my addr 172.21.15.40:0/688283534 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:31.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.897+0000 7f6d87fff640 1 -- 172.21.15.40:0/688283534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6d8815a990 msgr2=0x7f6d880a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:31.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.897+0000 7f6d87fff640 1 --2- 172.21.15.40:0/688283534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6d8815a990 0x7f6d880a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.897+0000 7f6d87fff640 1 -- 172.21.15.40:0/688283534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d880b0dd0 msgr2=0x7f6d880c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:31.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.897+0000 7f6d7ffff640 1 --2- 172.21.15.40:0/688283534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d880b0dd0 0x7f6d880c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:31.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.897+0000 7f6d87fff640 1 --2- 172.21.15.40:0/688283534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d880b0dd0 0x7f6d880c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:31.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.897+0000 7f6d87fff640 1 -- 172.21.15.40:0/688283534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d78009490 con 0x7f6d880b0070 2023-10-18T08:55:31.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.897+0000 7f6d7ffff640 1 --2- 172.21.15.40:0/688283534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d880b0dd0 0x7f6d880c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:31.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.897+0000 7f6d87fff640 1 --2- 172.21.15.40:0/688283534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d880b0070 0x7f6d88159b00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009780 tx=0x7f6d780038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:31.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.897+0000 7f6d857fa640 1 -- 172.21.15.40:0/688283534 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d78003ea0 con 0x7f6d880b0070 2023-10-18T08:55:31.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.897+0000 7f6d857fa640 1 -- 172.21.15.40:0/688283534 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6d78034440 con 0x7f6d880b0070 2023-10-18T08:55:31.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.897+0000 7f6d8d193640 1 -- 172.21.15.40:0/688283534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d88132520 con 0x7f6d880b0070 2023-10-18T08:55:31.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.897+0000 7f6d857fa640 1 -- 172.21.15.40:0/688283534 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d78034850 con 0x7f6d880b0070 2023-10-18T08:55:31.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.898+0000 7f6d8d193640 1 -- 172.21.15.40:0/688283534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d88131aa0 con 0x7f6d880b0070 2023-10-18T08:55:31.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.899+0000 7f6d8d193640 1 -- 172.21.15.40:0/688283534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d88066e00 con 0x7f6d880b0070 2023-10-18T08:55:31.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.900+0000 7f6d857fa640 1 -- 172.21.15.40:0/688283534 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6d7803b040 con 0x7f6d880b0070 2023-10-18T08:55:31.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.901+0000 7f6d857fa640 1 --2- 172.21.15.40:0/688283534 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6d6806ae40 0x7f6d6806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:31.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.901+0000 7f6d857fa640 1 -- 172.21.15.40:0/688283534 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d780abc20 con 0x7f6d880b0070 2023-10-18T08:55:31.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.901+0000 7f6d7ffff640 1 --2- 172.21.15.40:0/688283534 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6d6806ae40 0x7f6d6806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:31.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.903+0000 7f6d7ffff640 1 --2- 172.21.15.40:0/688283534 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6d6806ae40 0x7f6d6806d300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6d8815a680 tx=0x7f6d70009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:31.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:31.907+0000 7f6d857fa640 1 -- 172.21.15.40:0/688283534 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d78076bb0 con 0x7f6d880b0070 2023-10-18T08:55:32.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.139+0000 7f6d8d193640 1 -- 172.21.15.40:0/688283534 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6d8815a3d0 con 0x7f6d880b0070 2023-10-18T08:55:32.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.140+0000 7f6d857fa640 1 -- 172.21.15.40:0/688283534 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f6d78041040 con 0x7f6d880b0070 2023-10-18T08:55:32.143 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:32.144 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-18T08:55:32.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.142+0000 7f6d8d193640 1 -- 172.21.15.40:0/688283534 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6d6806ae40 msgr2=0x7f6d6806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:32.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.142+0000 7f6d8d193640 1 --2- 172.21.15.40:0/688283534 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6d6806ae40 0x7f6d6806d300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6d8815a680 tx=0x7f6d70009040 comp rx=0 tx=0).stop 2023-10-18T08:55:32.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.143+0000 7f6d8d193640 1 -- 172.21.15.40:0/688283534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d880b0070 msgr2=0x7f6d88159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:32.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.143+0000 7f6d8d193640 1 --2- 172.21.15.40:0/688283534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d880b0070 0x7f6d88159b00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009780 tx=0x7f6d780038a0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.143+0000 7f6d8d193640 1 -- 172.21.15.40:0/688283534 shutdown_connections 2023-10-18T08:55:32.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.143+0000 7f6d8d193640 1 --2- 172.21.15.40:0/688283534 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6d6806ae40 0x7f6d6806d300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.143+0000 7f6d8d193640 1 --2- 172.21.15.40:0/688283534 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6d8815a990 0x7f6d880a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.143+0000 7f6d8d193640 1 --2- 172.21.15.40:0/688283534 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d880b0dd0 0x7f6d880c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.143+0000 7f6d8d193640 1 --2- 172.21.15.40:0/688283534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d880b0070 0x7f6d88159b00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.143+0000 7f6d8d193640 1 -- 172.21.15.40:0/688283534 >> 172.21.15.40:0/688283534 conn(0x7f6d880aed70 msgr2=0x7f6d880af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:32.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.143+0000 7f6d8d193640 1 -- 172.21.15.40:0/688283534 shutdown_connections 2023-10-18T08:55:32.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.143+0000 7f6d8d193640 1 -- 172.21.15.40:0/688283534 wait complete. 2023-10-18T08:55:32.166 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-18T08:55:32.167 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:32.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.389+0000 7ffa287a9640 1 -- 172.21.15.40:0/2166076893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa2006d710 msgr2=0x7ffa20151bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:32.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.389+0000 7ffa287a9640 1 --2- 172.21.15.40:0/2166076893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa2006d710 0x7ffa20151bd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffa2006c840 tx=0x7ffa1c02c110 comp rx=0 tx=0).stop 2023-10-18T08:55:32.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.390+0000 7ffa287a9640 1 -- 172.21.15.40:0/2166076893 shutdown_connections 2023-10-18T08:55:32.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.390+0000 7ffa287a9640 1 --2- 172.21.15.40:0/2166076893 >> v2:172.21.15.144:3300/0 conn(0x7ffa201586a0 0x7ffa200b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.390+0000 7ffa287a9640 1 --2- 172.21.15.40:0/2166076893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa2006d710 0x7ffa20151bd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.390+0000 7ffa287a9640 1 -- 172.21.15.40:0/2166076893 >> 172.21.15.40:0/2166076893 conn(0x7ffa2006cee0 msgr2=0x7ffa20069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:32.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.390+0000 7ffa287a9640 1 -- 172.21.15.40:0/2166076893 shutdown_connections 2023-10-18T08:55:32.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.390+0000 7ffa287a9640 1 -- 172.21.15.40:0/2166076893 wait complete. 2023-10-18T08:55:32.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.391+0000 7ffa287a9640 1 Processor -- start 2023-10-18T08:55:32.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.392+0000 7ffa287a9640 1 -- start start 2023-10-18T08:55:32.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.392+0000 7ffa287a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa2006d710 0x7ffa20145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:32.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.392+0000 7ffa287a9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffa200b1100 0x7ffa20148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:32.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.393+0000 7ffa287a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa201586a0 0x7ffa20148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:32.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.393+0000 7ffa287a9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa20146130 con 0x7ffa201586a0 2023-10-18T08:55:32.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.393+0000 7ffa287a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa200abc50 con 0x7ffa2006d710 2023-10-18T08:55:32.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.393+0000 7ffa287a9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa200abf30 con 0x7ffa200b1100 2023-10-18T08:55:32.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.393+0000 7ffa26d46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa201586a0 0x7ffa20148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:32.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.393+0000 7ffa26d46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa201586a0 0x7ffa20148f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48218/0 (socket says 172.21.15.40:48218) 2023-10-18T08:55:32.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.393+0000 7ffa26d46640 1 -- 172.21.15.40:0/216572833 learned_addr learned my addr 172.21.15.40:0/216572833 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:32.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.393+0000 7ffa26545640 1 --2- 172.21.15.40:0/216572833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa2006d710 0x7ffa20145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:32.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.393+0000 7ffa26d46640 1 -- 172.21.15.40:0/216572833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffa200b1100 msgr2=0x7ffa20148a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:32.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.393+0000 7ffa26d46640 1 --2- 172.21.15.40:0/216572833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffa200b1100 0x7ffa20148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.393+0000 7ffa26d46640 1 -- 172.21.15.40:0/216572833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa2006d710 msgr2=0x7ffa20145bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:32.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.393+0000 7ffa26d46640 1 --2- 172.21.15.40:0/216572833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa2006d710 0x7ffa20145bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.393+0000 7ffa26d46640 1 -- 172.21.15.40:0/216572833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1c009840 con 0x7ffa201586a0 2023-10-18T08:55:32.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.393+0000 7ffa26545640 1 --2- 172.21.15.40:0/216572833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa2006d710 0x7ffa20145bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:32.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.394+0000 7ffa26d46640 1 --2- 172.21.15.40:0/216572833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa201586a0 0x7ffa20148f40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800c430 tx=0x7ffa1800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:32.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.394+0000 7ffa16ffd640 1 -- 172.21.15.40:0/216572833 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18011790 con 0x7ffa201586a0 2023-10-18T08:55:32.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.394+0000 7ffa16ffd640 1 -- 172.21.15.40:0/216572833 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffa180119e0 con 0x7ffa201586a0 2023-10-18T08:55:32.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.394+0000 7ffa287a9640 1 -- 172.21.15.40:0/216572833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa200ac210 con 0x7ffa201586a0 2023-10-18T08:55:32.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.394+0000 7ffa16ffd640 1 -- 172.21.15.40:0/216572833 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18011e10 con 0x7ffa201586a0 2023-10-18T08:55:32.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.394+0000 7ffa287a9640 1 -- 172.21.15.40:0/216572833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa200aaed0 con 0x7ffa201586a0 2023-10-18T08:55:32.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.395+0000 7ffa287a9640 1 -- 172.21.15.40:0/216572833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa200c9fc0 con 0x7ffa201586a0 2023-10-18T08:55:32.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.397+0000 7ffa16ffd640 1 -- 172.21.15.40:0/216572833 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffa1801c460 con 0x7ffa201586a0 2023-10-18T08:55:32.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.397+0000 7ffa16ffd640 1 --2- 172.21.15.40:0/216572833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffa0406ae40 0x7ffa0406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:32.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.398+0000 7ffa16ffd640 1 -- 172.21.15.40:0/216572833 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa18089820 con 0x7ffa201586a0 2023-10-18T08:55:32.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.398+0000 7ffa26545640 1 --2- 172.21.15.40:0/216572833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffa0406ae40 0x7ffa0406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:32.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.399+0000 7ffa26545640 1 --2- 172.21.15.40:0/216572833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffa0406ae40 0x7ffa0406d300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c035000 tx=0x7ffa1c004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:32.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.402+0000 7ffa16ffd640 1 -- 172.21.15.40:0/216572833 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa200c9fc0 con 0x7ffa201586a0 2023-10-18T08:55:32.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.595+0000 7ffa287a9640 1 -- 172.21.15.40:0/216572833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa200ca3a0 con 0x7ffa201586a0 2023-10-18T08:55:32.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.597+0000 7ffa16ffd640 1 -- 172.21.15.40:0/216572833 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7ffa200ca3a0 con 0x7ffa201586a0 2023-10-18T08:55:32.601 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:32.601 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:16.275677+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:32.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.600+0000 7ffa287a9640 1 -- 172.21.15.40:0/216572833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffa0406ae40 msgr2=0x7ffa0406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:32.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.600+0000 7ffa287a9640 1 --2- 172.21.15.40:0/216572833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffa0406ae40 0x7ffa0406d300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c035000 tx=0x7ffa1c004ab0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.600+0000 7ffa287a9640 1 -- 172.21.15.40:0/216572833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa201586a0 msgr2=0x7ffa20148f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:32.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.600+0000 7ffa287a9640 1 --2- 172.21.15.40:0/216572833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa201586a0 0x7ffa20148f40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800c430 tx=0x7ffa1800c8f0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.600+0000 7ffa287a9640 1 -- 172.21.15.40:0/216572833 shutdown_connections 2023-10-18T08:55:32.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.600+0000 7ffa287a9640 1 --2- 172.21.15.40:0/216572833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffa0406ae40 0x7ffa0406d300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.600+0000 7ffa287a9640 1 --2- 172.21.15.40:0/216572833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa201586a0 0x7ffa20148f40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.600+0000 7ffa287a9640 1 --2- 172.21.15.40:0/216572833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffa200b1100 0x7ffa20148a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.600+0000 7ffa287a9640 1 --2- 172.21.15.40:0/216572833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa2006d710 0x7ffa20145bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.600+0000 7ffa287a9640 1 -- 172.21.15.40:0/216572833 >> 172.21.15.40:0/216572833 conn(0x7ffa2006cee0 msgr2=0x7ffa200afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:32.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.601+0000 7ffa287a9640 1 -- 172.21.15.40:0/216572833 shutdown_connections 2023-10-18T08:55:32.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.601+0000 7ffa287a9640 1 -- 172.21.15.40:0/216572833 wait complete. 2023-10-18T08:55:32.624 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:55:13.210788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:55:13.772849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:55:32.625 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:55:32.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.860+0000 7f01b936c640 1 -- 172.21.15.40:0/824040122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f019c0047c0 msgr2=0x7f019c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:32.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.860+0000 7f01b936c640 1 --2- 172.21.15.40:0/824040122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f019c0047c0 0x7f019c004c40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f01b40b0600 tx=0x7f019c0057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.860+0000 7f01b8b6b640 1 -- 172.21.15.40:0/824040122 reap_dead start 2023-10-18T08:55:32.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.860+0000 7f01b936c640 1 -- 172.21.15.40:0/824040122 shutdown_connections 2023-10-18T08:55:32.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.860+0000 7f01b936c640 1 -- 172.21.15.40:0/824040122 >> 172.21.15.40:0/824040122 conn(0x7f01b409a560 msgr2=0x7f01b40d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:32.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.860+0000 7f01b936c640 1 -- 172.21.15.40:0/824040122 shutdown_connections 2023-10-18T08:55:32.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.860+0000 7f01b936c640 1 -- 172.21.15.40:0/824040122 wait complete. 2023-10-18T08:55:32.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.862+0000 7f01b936c640 1 Processor -- start 2023-10-18T08:55:32.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.862+0000 7f01b936c640 1 -- start start 2023-10-18T08:55:32.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.863+0000 7f01b936c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f019c0047c0 0x7f01b41f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:32.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.863+0000 7f01b936c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01b409c140 0x7f01b409c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:32.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.863+0000 7f01b936c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01b40c82c0 0x7f01b40c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:32.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.863+0000 7f01b936c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40c9be0 con 0x7f01b40c82c0 2023-10-18T08:55:32.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.863+0000 7f01b936c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b409cb00 con 0x7f019c0047c0 2023-10-18T08:55:32.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.863+0000 7f01b936c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40c8f60 con 0x7f01b409c140 2023-10-18T08:55:32.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.863+0000 7f01b8b6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01b40c82c0 0x7f01b40c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:32.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.863+0000 7f01b8b6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01b40c82c0 0x7f01b40c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48240/0 (socket says 172.21.15.40:48240) 2023-10-18T08:55:32.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.863+0000 7f01b8b6b640 1 -- 172.21.15.40:0/2902995371 learned_addr learned my addr 172.21.15.40:0/2902995371 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:32.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.863+0000 7f01b3fff640 1 --2- 172.21.15.40:0/2902995371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f019c0047c0 0x7f01b41f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:32.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.863+0000 7f01b37fe640 1 --2- 172.21.15.40:0/2902995371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01b409c140 0x7f01b409c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:32.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.863+0000 7f01b8b6b640 1 -- 172.21.15.40:0/2902995371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01b409c140 msgr2=0x7f01b409c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:32.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.863+0000 7f01b8b6b640 1 --2- 172.21.15.40:0/2902995371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01b409c140 0x7f01b409c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.863+0000 7f01b8b6b640 1 -- 172.21.15.40:0/2902995371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f019c0047c0 msgr2=0x7f01b41f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:32.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.863+0000 7f01b8b6b640 1 --2- 172.21.15.40:0/2902995371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f019c0047c0 0x7f01b41f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:32.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.863+0000 7f01b8b6b640 1 -- 172.21.15.40:0/2902995371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b415f0e0 con 0x7f01b40c82c0 2023-10-18T08:55:32.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.864+0000 7f01b8b6b640 1 --2- 172.21.15.40:0/2902995371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01b40c82c0 0x7f01b40c8740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f01b40b72a0 tx=0x7f01ac011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:32.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.864+0000 7f01b0ff9640 1 -- 172.21.15.40:0/2902995371 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac004230 con 0x7f01b40c82c0 2023-10-18T08:55:32.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.864+0000 7f01b0ff9640 1 -- 172.21.15.40:0/2902995371 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f01ac004480 con 0x7f01b40c82c0 2023-10-18T08:55:32.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.864+0000 7f01b936c640 1 -- 172.21.15.40:0/2902995371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b40c9240 con 0x7f01b40c82c0 2023-10-18T08:55:32.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.864+0000 7f01b0ff9640 1 -- 172.21.15.40:0/2902995371 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac00fa90 con 0x7f01b40c82c0 2023-10-18T08:55:32.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.864+0000 7f01b936c640 1 -- 172.21.15.40:0/2902995371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b40c9800 con 0x7f01b40c82c0 2023-10-18T08:55:32.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.866+0000 7f01b936c640 1 -- 172.21.15.40:0/2902995371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b40c9d20 con 0x7f01b40c82c0 2023-10-18T08:55:32.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.867+0000 7f01b0ff9640 1 -- 172.21.15.40:0/2902995371 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f01ac018070 con 0x7f01b40c82c0 2023-10-18T08:55:32.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.867+0000 7f01b0ff9640 1 --2- 172.21.15.40:0/2902995371 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f01a006ae40 0x7f01a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:32.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.868+0000 7f01b0ff9640 1 -- 172.21.15.40:0/2902995371 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01ac08a170 con 0x7f01b40c82c0 2023-10-18T08:55:32.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.868+0000 7f01b3fff640 1 --2- 172.21.15.40:0/2902995371 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f01a006ae40 0x7f01a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:32.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.869+0000 7f01b3fff640 1 --2- 172.21.15.40:0/2902995371 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f01a006ae40 0x7f01a006d300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f01a4005fd0 tx=0x7f01a4009500 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:32.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:32.872+0000 7f01b0ff9640 1 -- 172.21.15.40:0/2902995371 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f01b40c9d20 con 0x7f01b40c82c0 2023-10-18T08:55:33.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.067+0000 7f01b936c640 1 -- 172.21.15.40:0/2902995371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01b40c9d20 con 0x7f01b40c82c0 2023-10-18T08:55:33.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.068+0000 7f01b0ff9640 1 -- 172.21.15.40:0/2902995371 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01b40c9d20 con 0x7f01b40c82c0 2023-10-18T08:55:33.072 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:55:33.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.071+0000 7f01b936c640 1 -- 172.21.15.40:0/2902995371 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f01a006ae40 msgr2=0x7f01a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:33.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.071+0000 7f01b936c640 1 --2- 172.21.15.40:0/2902995371 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f01a006ae40 0x7f01a006d300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f01a4005fd0 tx=0x7f01a4009500 comp rx=0 tx=0).stop 2023-10-18T08:55:33.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.071+0000 7f01b936c640 1 -- 172.21.15.40:0/2902995371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01b40c82c0 msgr2=0x7f01b40c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:33.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.071+0000 7f01b936c640 1 --2- 172.21.15.40:0/2902995371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01b40c82c0 0x7f01b40c8740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f01b40b72a0 tx=0x7f01ac011b10 comp rx=0 tx=0).stop 2023-10-18T08:55:33.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.071+0000 7f01b936c640 1 -- 172.21.15.40:0/2902995371 shutdown_connections 2023-10-18T08:55:33.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.071+0000 7f01b936c640 1 --2- 172.21.15.40:0/2902995371 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f01a006ae40 0x7f01a006d300 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.071+0000 7f01b936c640 1 --2- 172.21.15.40:0/2902995371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01b40c82c0 0x7f01b40c8740 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.071+0000 7f01b936c640 1 --2- 172.21.15.40:0/2902995371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01b409c140 0x7f01b409c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.071+0000 7f01b936c640 1 --2- 172.21.15.40:0/2902995371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f019c0047c0 0x7f01b41f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.071+0000 7f01b936c640 1 -- 172.21.15.40:0/2902995371 >> 172.21.15.40:0/2902995371 conn(0x7f01b409a560 msgr2=0x7f01b415a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:33.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.071+0000 7f01b936c640 1 -- 172.21.15.40:0/2902995371 shutdown_connections 2023-10-18T08:55:33.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.071+0000 7f01b936c640 1 -- 172.21.15.40:0/2902995371 wait complete. 2023-10-18T08:55:33.098 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:55:33.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.327+0000 7f18beafc640 1 -- 172.21.15.40:0/3696048223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b81593c0 msgr2=0x7f18b80cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:33.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.327+0000 7f18beafc640 1 --2- 172.21.15.40:0/3696048223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b81593c0 0x7f18b80cadc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f18b80aedb0 tx=0x7f18b402bd70 comp rx=0 tx=0).stop 2023-10-18T08:55:33.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.327+0000 7f18beafc640 1 -- 172.21.15.40:0/3696048223 shutdown_connections 2023-10-18T08:55:33.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.327+0000 7f18beafc640 1 --2- 172.21.15.40:0/3696048223 >> v2:172.21.15.144:3300/0 conn(0x7f18b806baf0 0x7f18b806bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.327+0000 7f18beafc640 1 --2- 172.21.15.40:0/3696048223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b81593c0 0x7f18b80cadc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.327+0000 7f18beafc640 1 -- 172.21.15.40:0/3696048223 >> 172.21.15.40:0/3696048223 conn(0x7f18b806ca10 msgr2=0x7f18b806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:33.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.327+0000 7f18beafc640 1 -- 172.21.15.40:0/3696048223 shutdown_connections 2023-10-18T08:55:33.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.328+0000 7f18beafc640 1 -- 172.21.15.40:0/3696048223 wait complete. 2023-10-18T08:55:33.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.329+0000 7f18beafc640 1 Processor -- start 2023-10-18T08:55:33.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.329+0000 7f18beafc640 1 -- start start 2023-10-18T08:55:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.330+0000 7f18beafc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b806baf0 0x7f18b80ab480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.330+0000 7f18beafc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b81593c0 0x7f18b814c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.330+0000 7f18beafc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18b815a490 0x7f18b80adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:33.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.330+0000 7f18beafc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b81578e0 con 0x7f18b81593c0 2023-10-18T08:55:33.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.330+0000 7f18beafc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b80ab9c0 con 0x7f18b806baf0 2023-10-18T08:55:33.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.330+0000 7f18beafc640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b8157b30 con 0x7f18b815a490 2023-10-18T08:55:33.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.331+0000 7f18bd2f9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b81593c0 0x7f18b814c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:33.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.331+0000 7f18bd2f9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b81593c0 0x7f18b814c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48258/0 (socket says 172.21.15.40:48258) 2023-10-18T08:55:33.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.331+0000 7f18bd2f9640 1 -- 172.21.15.40:0/2101953174 learned_addr learned my addr 172.21.15.40:0/2101953174 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:33.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.331+0000 7f18bdafa640 1 --2- 172.21.15.40:0/2101953174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b806baf0 0x7f18b80ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:33.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.331+0000 7f18be2fb640 1 --2- 172.21.15.40:0/2101953174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18b815a490 0x7f18b80adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:33.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.331+0000 7f18bd2f9640 1 -- 172.21.15.40:0/2101953174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18b815a490 msgr2=0x7f18b80adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:33.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.331+0000 7f18bd2f9640 1 --2- 172.21.15.40:0/2101953174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18b815a490 0x7f18b80adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.331+0000 7f18bd2f9640 1 -- 172.21.15.40:0/2101953174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b806baf0 msgr2=0x7f18b80ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:33.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.331+0000 7f18bd2f9640 1 --2- 172.21.15.40:0/2101953174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b806baf0 0x7f18b80ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.331+0000 7f18bd2f9640 1 -- 172.21.15.40:0/2101953174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b4009490 con 0x7f18b81593c0 2023-10-18T08:55:33.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.332+0000 7f18bd2f9640 1 --2- 172.21.15.40:0/2101953174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b81593c0 0x7f18b814c180 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c410 tx=0x7f18a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:33.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.332+0000 7f18ae7fc640 1 -- 172.21.15.40:0/2101953174 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a80035f0 con 0x7f18b81593c0 2023-10-18T08:55:33.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.332+0000 7f18ae7fc640 1 -- 172.21.15.40:0/2101953174 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f18a8003770 con 0x7f18b81593c0 2023-10-18T08:55:33.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.332+0000 7f18ae7fc640 1 -- 172.21.15.40:0/2101953174 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8003b80 con 0x7f18b81593c0 2023-10-18T08:55:33.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.332+0000 7f18beafc640 1 -- 172.21.15.40:0/2101953174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b8157e10 con 0x7f18b81593c0 2023-10-18T08:55:33.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.332+0000 7f18beafc640 1 -- 172.21.15.40:0/2101953174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b81583d0 con 0x7f18b81593c0 2023-10-18T08:55:33.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.333+0000 7f18beafc640 1 -- 172.21.15.40:0/2101953174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b8066e00 con 0x7f18b81593c0 2023-10-18T08:55:33.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.334+0000 7f18ae7fc640 1 -- 172.21.15.40:0/2101953174 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f18a8003ce0 con 0x7f18b81593c0 2023-10-18T08:55:33.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.335+0000 7f18ae7fc640 1 --2- 172.21.15.40:0/2101953174 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f189c06ae40 0x7f189c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:33.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.335+0000 7f18ae7fc640 1 -- 172.21.15.40:0/2101953174 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18a8088db0 con 0x7f18b81593c0 2023-10-18T08:55:33.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.336+0000 7f18bdafa640 1 --2- 172.21.15.40:0/2101953174 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f189c06ae40 0x7f189c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:33.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.337+0000 7f18bdafa640 1 --2- 172.21.15.40:0/2101953174 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f189c06ae40 0x7f189c06d300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f18b4036000 tx=0x7f18b4004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:33.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.339+0000 7f18ae7fc640 1 -- 172.21.15.40:0/2101953174 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18a808d070 con 0x7f18b81593c0 2023-10-18T08:55:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.530+0000 7f18beafc640 1 -- 172.21.15.40:0/2101953174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f18b80cdc90 con 0x7f18b81593c0 2023-10-18T08:55:33.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.531+0000 7f18ae7fc640 1 -- 172.21.15.40:0/2101953174 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f18a8017020 con 0x7f18b81593c0 2023-10-18T08:55:33.535 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:55:33.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.533+0000 7f18beafc640 1 -- 172.21.15.40:0/2101953174 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f189c06ae40 msgr2=0x7f189c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:33.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.533+0000 7f18beafc640 1 --2- 172.21.15.40:0/2101953174 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f189c06ae40 0x7f189c06d300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f18b4036000 tx=0x7f18b4004730 comp rx=0 tx=0).stop 2023-10-18T08:55:33.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.533+0000 7f18beafc640 1 -- 172.21.15.40:0/2101953174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b81593c0 msgr2=0x7f18b814c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:33.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.533+0000 7f18beafc640 1 --2- 172.21.15.40:0/2101953174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b81593c0 0x7f18b814c180 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c410 tx=0x7f18a800c8d0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.534+0000 7f18beafc640 1 -- 172.21.15.40:0/2101953174 shutdown_connections 2023-10-18T08:55:33.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.534+0000 7f18beafc640 1 --2- 172.21.15.40:0/2101953174 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f189c06ae40 0x7f189c06d300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.534+0000 7f18beafc640 1 --2- 172.21.15.40:0/2101953174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18b815a490 0x7f18b80adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.534+0000 7f18beafc640 1 --2- 172.21.15.40:0/2101953174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b81593c0 0x7f18b814c180 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.534+0000 7f18beafc640 1 --2- 172.21.15.40:0/2101953174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18b806baf0 0x7f18b80ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.534+0000 7f18beafc640 1 -- 172.21.15.40:0/2101953174 >> 172.21.15.40:0/2101953174 conn(0x7f18b806ca10 msgr2=0x7f18b80acbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:33.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.534+0000 7f18beafc640 1 -- 172.21.15.40:0/2101953174 shutdown_connections 2023-10-18T08:55:33.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.534+0000 7f18beafc640 1 -- 172.21.15.40:0/2101953174 wait complete. 2023-10-18T08:55:33.556 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:55:33.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.788+0000 7f9aa2cc9640 1 -- 172.21.15.40:0/1830253780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c06d660 msgr2=0x7f9a9c0b1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:33.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.788+0000 7f9aa2cc9640 1 --2- 172.21.15.40:0/1830253780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c06d660 0x7f9a9c0b1cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c0acb90 tx=0x7f9a9002bd30 comp rx=0 tx=0).stop 2023-10-18T08:55:33.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.788+0000 7f9aa2cc9640 1 -- 172.21.15.40:0/1830253780 shutdown_connections 2023-10-18T08:55:33.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.788+0000 7f9aa2cc9640 1 --2- 172.21.15.40:0/1830253780 >> v2:172.21.15.144:3300/0 conn(0x7f9a9c157800 0x7f9a9c0cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.788+0000 7f9aa2cc9640 1 --2- 172.21.15.40:0/1830253780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c06d660 0x7f9a9c0b1cc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.788+0000 7f9aa2cc9640 1 -- 172.21.15.40:0/1830253780 >> 172.21.15.40:0/1830253780 conn(0x7f9a9c069720 msgr2=0x7f9a9c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:33.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.789+0000 7f9aa2cc9640 1 -- 172.21.15.40:0/1830253780 shutdown_connections 2023-10-18T08:55:33.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.789+0000 7f9aa2cc9640 1 -- 172.21.15.40:0/1830253780 wait complete. 2023-10-18T08:55:33.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.791+0000 7f9aa2cc9640 1 Processor -- start 2023-10-18T08:55:33.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.791+0000 7f9aa2cc9640 1 -- start start 2023-10-18T08:55:33.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.793+0000 7f9aa2cc9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c06d660 0x7f9a9c145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:33.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.793+0000 7f9aa2cc9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a9c0ccf70 0x7f9a9c0adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:33.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.793+0000 7f9aa2cc9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a9c157800 0x7f9a9c0ae050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:33.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.793+0000 7f9aa2cc9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c145fe0 con 0x7f9a9c06d660 2023-10-18T08:55:33.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.793+0000 7f9aa2cc9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c1e1e10 con 0x7f9a9c0ccf70 2023-10-18T08:55:33.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.793+0000 7f9aa2cc9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c1e20f0 con 0x7f9a9c157800 2023-10-18T08:55:33.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.793+0000 7f9aa0a65640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c06d660 0x7f9a9c145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:33.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.793+0000 7f9aa0a65640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c06d660 0x7f9a9c145aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48276/0 (socket says 172.21.15.40:48276) 2023-10-18T08:55:33.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.793+0000 7f9aa0a65640 1 -- 172.21.15.40:0/3488301188 learned_addr learned my addr 172.21.15.40:0/3488301188 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:33.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.793+0000 7f9aa1266640 1 --2- 172.21.15.40:0/3488301188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a9c157800 0x7f9a9c0ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:33.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.793+0000 7f9a9bfff640 1 --2- 172.21.15.40:0/3488301188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a9c0ccf70 0x7f9a9c0adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:33.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.794+0000 7f9aa0a65640 1 -- 172.21.15.40:0/3488301188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a9c157800 msgr2=0x7f9a9c0ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:33.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.794+0000 7f9aa0a65640 1 --2- 172.21.15.40:0/3488301188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a9c157800 0x7f9a9c0ae050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.794+0000 7f9aa0a65640 1 -- 172.21.15.40:0/3488301188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a9c0ccf70 msgr2=0x7f9a9c0adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:33.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.794+0000 7f9aa0a65640 1 --2- 172.21.15.40:0/3488301188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a9c0ccf70 0x7f9a9c0adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:33.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.794+0000 7f9aa0a65640 1 -- 172.21.15.40:0/3488301188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a90009560 con 0x7f9a9c06d660 2023-10-18T08:55:33.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.794+0000 7f9aa0a65640 1 --2- 172.21.15.40:0/3488301188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c06d660 0x7f9a9c145aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9a90000c00 tx=0x7f9a9002dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:33.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.794+0000 7f9a997fa640 1 -- 172.21.15.40:0/3488301188 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90005670 con 0x7f9a9c06d660 2023-10-18T08:55:33.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.794+0000 7f9a997fa640 1 -- 172.21.15.40:0/3488301188 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9a900038f0 con 0x7f9a9c06d660 2023-10-18T08:55:33.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.794+0000 7f9a997fa640 1 -- 172.21.15.40:0/3488301188 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90003ce0 con 0x7f9a9c06d660 2023-10-18T08:55:33.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.794+0000 7f9aa2cc9640 1 -- 172.21.15.40:0/3488301188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9c1e23d0 con 0x7f9a9c06d660 2023-10-18T08:55:33.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.795+0000 7f9aa2cc9640 1 -- 172.21.15.40:0/3488301188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9c1e2990 con 0x7f9a9c06d660 2023-10-18T08:55:33.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.796+0000 7f9aa2cc9640 1 -- 172.21.15.40:0/3488301188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9c066fd0 con 0x7f9a9c06d660 2023-10-18T08:55:33.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.797+0000 7f9a997fa640 1 -- 172.21.15.40:0/3488301188 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9a9003e450 con 0x7f9a9c06d660 2023-10-18T08:55:33.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.798+0000 7f9a997fa640 1 --2- 172.21.15.40:0/3488301188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a8006ae40 0x7f9a8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:33.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.798+0000 7f9a997fa640 1 -- 172.21.15.40:0/3488301188 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a900abf50 con 0x7f9a9c06d660 2023-10-18T08:55:33.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.799+0000 7f9a9bfff640 1 --2- 172.21.15.40:0/3488301188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a8006ae40 0x7f9a8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.800+0000 7f9a9bfff640 1 --2- 172.21.15.40:0/3488301188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a8006ae40 0x7f9a8006d300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9a88005d70 tx=0x7f9a88005d00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:33.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:33.804+0000 7f9a997fa640 1 -- 172.21.15.40:0/3488301188 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9a90076f60 con 0x7f9a9c06d660 2023-10-18T08:55:34.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.006+0000 7f9aa2cc9640 1 -- 172.21.15.40:0/3488301188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9a9c0cc450 con 0x7f9a9c06d660 2023-10-18T08:55:34.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.007+0000 7f9a997fa640 1 -- 172.21.15.40:0/3488301188 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9a9c0cc450 con 0x7f9a9c06d660 2023-10-18T08:55:34.010 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:55:34.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.010+0000 7f9aa2cc9640 1 -- 172.21.15.40:0/3488301188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a8006ae40 msgr2=0x7f9a8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:34.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.010+0000 7f9aa2cc9640 1 --2- 172.21.15.40:0/3488301188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a8006ae40 0x7f9a8006d300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9a88005d70 tx=0x7f9a88005d00 comp rx=0 tx=0).stop 2023-10-18T08:55:34.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.010+0000 7f9aa2cc9640 1 -- 172.21.15.40:0/3488301188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c06d660 msgr2=0x7f9a9c145aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:34.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.010+0000 7f9aa2cc9640 1 --2- 172.21.15.40:0/3488301188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c06d660 0x7f9a9c145aa0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9a90000c00 tx=0x7f9a9002dd00 comp rx=0 tx=0).stop 2023-10-18T08:55:34.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.010+0000 7f9aa2cc9640 1 -- 172.21.15.40:0/3488301188 shutdown_connections 2023-10-18T08:55:34.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.010+0000 7f9aa2cc9640 1 --2- 172.21.15.40:0/3488301188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a8006ae40 0x7f9a8006d300 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.010+0000 7f9aa2cc9640 1 --2- 172.21.15.40:0/3488301188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a9c157800 0x7f9a9c0ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.010+0000 7f9aa2cc9640 1 --2- 172.21.15.40:0/3488301188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a9c0ccf70 0x7f9a9c0adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.010+0000 7f9aa2cc9640 1 --2- 172.21.15.40:0/3488301188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a9c06d660 0x7f9a9c145aa0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.010+0000 7f9aa2cc9640 1 -- 172.21.15.40:0/3488301188 >> 172.21.15.40:0/3488301188 conn(0x7f9a9c069720 msgr2=0x7f9a9c156950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:34.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.010+0000 7f9aa2cc9640 1 -- 172.21.15.40:0/3488301188 shutdown_connections 2023-10-18T08:55:34.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.011+0000 7f9aa2cc9640 1 -- 172.21.15.40:0/3488301188 wait complete. 2023-10-18T08:55:34.032 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:55:34.053 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:55:34.084 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:55:34.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.261+0000 7fbf2d5d0640 1 -- 172.21.15.40:0/3657513658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf2815b640 msgr2=0x7fbf280d1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:34.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.261+0000 7fbf2d5d0640 1 --2- 172.21.15.40:0/3657513658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf2815b640 0x7fbf280d1a90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbf18005fd0 tx=0x7fbf1802bc60 comp rx=0 tx=0).stop 2023-10-18T08:55:34.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.262+0000 7fbf2d5d0640 1 -- 172.21.15.40:0/3657513658 shutdown_connections 2023-10-18T08:55:34.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.262+0000 7fbf2d5d0640 1 --2- 172.21.15.40:0/3657513658 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf280b01e0 0x7fbf280cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.262+0000 7fbf2d5d0640 1 --2- 172.21.15.40:0/3657513658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf2815b640 0x7fbf280d1a90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.262+0000 7fbf2d5d0640 1 -- 172.21.15.40:0/3657513658 >> 172.21.15.40:0/3657513658 conn(0x7fbf2808ad00 msgr2=0x7fbf2809a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:34.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.262+0000 7fbf2d5d0640 1 -- 172.21.15.40:0/3657513658 shutdown_connections 2023-10-18T08:55:34.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.262+0000 7fbf2d5d0640 1 -- 172.21.15.40:0/3657513658 wait complete. 2023-10-18T08:55:34.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.263+0000 7fbf2d5d0640 1 Processor -- start 2023-10-18T08:55:34.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.264+0000 7fbf2d5d0640 1 -- start start 2023-10-18T08:55:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.264+0000 7fbf2d5d0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf280b01e0 0x7fbf2815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.264+0000 7fbf2d5d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf280cfe50 0x7fbf280ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:34.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.265+0000 7fbf2d5d0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf2815b640 0x7fbf28152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:34.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.265+0000 7fbf2d5d0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf280ca800 con 0x7fbf2815b640 2023-10-18T08:55:34.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.265+0000 7fbf2d5d0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf280b53d0 con 0x7fbf280cfe50 2023-10-18T08:55:34.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.265+0000 7fbf2d5d0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf280ca580 con 0x7fbf280b01e0 2023-10-18T08:55:34.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.265+0000 7fbf2cdcf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf2815b640 0x7fbf28152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.265+0000 7fbf2cdcf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf2815b640 0x7fbf28152970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48288/0 (socket says 172.21.15.40:48288) 2023-10-18T08:55:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.265+0000 7fbf2cdcf640 1 -- 172.21.15.40:0/2574226047 learned_addr learned my addr 172.21.15.40:0/2574226047 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.265+0000 7fbf27fff640 1 --2- 172.21.15.40:0/2574226047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf280b01e0 0x7fbf2815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.265+0000 7fbf2cdcf640 1 -- 172.21.15.40:0/2574226047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf280b01e0 msgr2=0x7fbf2815ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.265+0000 7fbf277fe640 1 --2- 172.21.15.40:0/2574226047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf280cfe50 0x7fbf280ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.265+0000 7fbf2cdcf640 1 --2- 172.21.15.40:0/2574226047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf280b01e0 0x7fbf2815ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.265+0000 7fbf2cdcf640 1 -- 172.21.15.40:0/2574226047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf280cfe50 msgr2=0x7fbf280ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:34.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.265+0000 7fbf2cdcf640 1 --2- 172.21.15.40:0/2574226047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf280cfe50 0x7fbf280ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.265+0000 7fbf2cdcf640 1 -- 172.21.15.40:0/2574226047 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf18009560 con 0x7fbf2815b640 2023-10-18T08:55:34.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.266+0000 7fbf2cdcf640 1 --2- 172.21.15.40:0/2574226047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf2815b640 0x7fbf28152970 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000c7b0 tx=0x7fbf2000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:34.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.266+0000 7fbf24ff9640 1 -- 172.21.15.40:0/2574226047 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20004b60 con 0x7fbf2815b640 2023-10-18T08:55:34.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.266+0000 7fbf24ff9640 1 -- 172.21.15.40:0/2574226047 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbf20004db0 con 0x7fbf2815b640 2023-10-18T08:55:34.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.266+0000 7fbf24ff9640 1 -- 172.21.15.40:0/2574226047 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2000aa50 con 0x7fbf2815b640 2023-10-18T08:55:34.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.266+0000 7fbf2d5d0640 1 -- 172.21.15.40:0/2574226047 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf280b4aa0 con 0x7fbf2815b640 2023-10-18T08:55:34.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.266+0000 7fbf2d5d0640 1 -- 172.21.15.40:0/2574226047 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf280b5060 con 0x7fbf2815b640 2023-10-18T08:55:34.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.268+0000 7fbf2d5d0640 1 -- 172.21.15.40:0/2574226047 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf28071da0 con 0x7fbf2815b640 2023-10-18T08:55:34.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.268+0000 7fbf24ff9640 1 -- 172.21.15.40:0/2574226047 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbf2000abb0 con 0x7fbf2815b640 2023-10-18T08:55:34.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.270+0000 7fbf24ff9640 1 --2- 172.21.15.40:0/2574226047 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbf1406ae40 0x7fbf1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.270+0000 7fbf24ff9640 1 -- 172.21.15.40:0/2574226047 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf20089b20 con 0x7fbf2815b640 2023-10-18T08:55:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.271+0000 7fbf27fff640 1 --2- 172.21.15.40:0/2574226047 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbf1406ae40 0x7fbf1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.272+0000 7fbf27fff640 1 --2- 172.21.15.40:0/2574226047 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbf1406ae40 0x7fbf1406d300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbf18035000 tx=0x7fbf18004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.275+0000 7fbf24ff9640 1 -- 172.21.15.40:0/2574226047 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbf20054aa0 con 0x7fbf2815b640 2023-10-18T08:55:34.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.505+0000 7fbf2d5d0640 1 -- 172.21.15.40:0/2574226047 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf28066e00 con 0x7fbf2815b640 2023-10-18T08:55:34.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.506+0000 7fbf24ff9640 1 -- 172.21.15.40:0/2574226047 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4910 (secure 0 0 0) 0x7fbf2000ae60 con 0x7fbf2815b640 2023-10-18T08:55:34.510 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:34.510 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1962150323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1962150323},{"type":"v1","addr":"172.21.15.144:6837","nonce":1962150323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4329,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2589720838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2589720838},{"type":"v1","addr":"172.21.15.40:6833","nonce":2589720838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3031491371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3031491371},{"type":"v1","addr":"172.21.15.40:6835","nonce":3031491371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:55:14.753828+0000","modified":"2023-10-18T08:55:17.818729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/682820172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":682820172},{"type":"v1","addr":"172.21.15.144:6835","nonce":682820172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T08:55:34.510 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2023-10-18T08:55:34.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.509+0000 7fbf2d5d0640 1 -- 172.21.15.40:0/2574226047 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbf1406ae40 msgr2=0x7fbf1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:34.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.509+0000 7fbf2d5d0640 1 --2- 172.21.15.40:0/2574226047 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbf1406ae40 0x7fbf1406d300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbf18035000 tx=0x7fbf18004730 comp rx=0 tx=0).stop 2023-10-18T08:55:34.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.509+0000 7fbf2d5d0640 1 -- 172.21.15.40:0/2574226047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf2815b640 msgr2=0x7fbf28152970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:34.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.509+0000 7fbf2d5d0640 1 --2- 172.21.15.40:0/2574226047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf2815b640 0x7fbf28152970 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000c7b0 tx=0x7fbf2000cc70 comp rx=0 tx=0).stop 2023-10-18T08:55:34.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.509+0000 7fbf2d5d0640 1 -- 172.21.15.40:0/2574226047 shutdown_connections 2023-10-18T08:55:34.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.509+0000 7fbf2d5d0640 1 --2- 172.21.15.40:0/2574226047 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbf1406ae40 0x7fbf1406d300 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.509+0000 7fbf2d5d0640 1 --2- 172.21.15.40:0/2574226047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf2815b640 0x7fbf28152970 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.509+0000 7fbf2d5d0640 1 --2- 172.21.15.40:0/2574226047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf280cfe50 0x7fbf280ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.509+0000 7fbf2d5d0640 1 --2- 172.21.15.40:0/2574226047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbf280b01e0 0x7fbf2815ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.509+0000 7fbf2d5d0640 1 -- 172.21.15.40:0/2574226047 >> 172.21.15.40:0/2574226047 conn(0x7fbf2808ad00 msgr2=0x7fbf280cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:34.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.509+0000 7fbf2d5d0640 1 -- 172.21.15.40:0/2574226047 shutdown_connections 2023-10-18T08:55:34.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.509+0000 7fbf2d5d0640 1 -- 172.21.15.40:0/2574226047 wait complete. 2023-10-18T08:55:34.535 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.754+0000 7ff02d24b640 1 -- 172.21.15.40:0/2641026673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0280b1b10 msgr2=0x7ff028163170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.754+0000 7ff02d24b640 1 --2- 172.21.15.40:0/2641026673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0280b1b10 0x7ff028163170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff020009830 tx=0x7ff02002bd70 comp rx=0 tx=0).stop 2023-10-18T08:55:34.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.755+0000 7ff02d24b640 1 -- 172.21.15.40:0/2641026673 shutdown_connections 2023-10-18T08:55:34.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.755+0000 7ff02d24b640 1 --2- 172.21.15.40:0/2641026673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0280b1b10 0x7ff028163170 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.755+0000 7ff02d24b640 1 -- 172.21.15.40:0/2641026673 >> 172.21.15.40:0/2641026673 conn(0x7ff02806cee0 msgr2=0x7ff028069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:34.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.755+0000 7ff02d24b640 1 -- 172.21.15.40:0/2641026673 shutdown_connections 2023-10-18T08:55:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.755+0000 7ff02d24b640 1 -- 172.21.15.40:0/2641026673 wait complete. 2023-10-18T08:55:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.757+0000 7ff02d24b640 1 Processor -- start 2023-10-18T08:55:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.757+0000 7ff02d24b640 1 -- start start 2023-10-18T08:55:34.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.758+0000 7ff02d24b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff02806f620 0x7ff02814f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:34.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.758+0000 7ff02d24b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0280b1b10 0x7ff028148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:34.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.758+0000 7ff02d24b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff028158160 0x7ff0281523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:34.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.758+0000 7ff02d24b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff028149380 con 0x7ff02806f620 2023-10-18T08:55:34.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.758+0000 7ff02d24b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02814ec20 con 0x7ff028158160 2023-10-18T08:55:34.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.758+0000 7ff02d24b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff028152cc0 con 0x7ff0280b1b10 2023-10-18T08:55:34.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.759+0000 7ff026d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff02806f620 0x7ff02814f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:34.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.759+0000 7ff026d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff02806f620 0x7ff02814f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48314/0 (socket says 172.21.15.40:48314) 2023-10-18T08:55:34.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.759+0000 7ff026d9d640 1 -- 172.21.15.40:0/1431579217 learned_addr learned my addr 172.21.15.40:0/1431579217 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:34.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.759+0000 7ff02659c640 1 --2- 172.21.15.40:0/1431579217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0280b1b10 0x7ff028148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:34.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.759+0000 7ff02759e640 1 --2- 172.21.15.40:0/1431579217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff028158160 0x7ff0281523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:34.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.759+0000 7ff026d9d640 1 -- 172.21.15.40:0/1431579217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0280b1b10 msgr2=0x7ff028148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:34.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.759+0000 7ff026d9d640 1 --2- 172.21.15.40:0/1431579217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0280b1b10 0x7ff028148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.759+0000 7ff026d9d640 1 -- 172.21.15.40:0/1431579217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff028158160 msgr2=0x7ff0281523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:34.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.759+0000 7ff026d9d640 1 --2- 172.21.15.40:0/1431579217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff028158160 0x7ff0281523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.759+0000 7ff026d9d640 1 -- 172.21.15.40:0/1431579217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff020009490 con 0x7ff02806f620 2023-10-18T08:55:34.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.760+0000 7ff026d9d640 1 --2- 172.21.15.40:0/1431579217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff02806f620 0x7ff02814f9a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff01800cb50 tx=0x7ff01800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:34.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.760+0000 7ff01f7fe640 1 -- 172.21.15.40:0/1431579217 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01800ad40 con 0x7ff02806f620 2023-10-18T08:55:34.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.760+0000 7ff01f7fe640 1 -- 172.21.15.40:0/1431579217 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff018012420 con 0x7ff02806f620 2023-10-18T08:55:34.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.760+0000 7ff01f7fe640 1 -- 172.21.15.40:0/1431579217 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff018012870 con 0x7ff02806f620 2023-10-18T08:55:34.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.760+0000 7ff02d24b640 1 -- 172.21.15.40:0/1431579217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02814eec0 con 0x7ff02806f620 2023-10-18T08:55:34.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.760+0000 7ff02d24b640 1 -- 172.21.15.40:0/1431579217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02814f480 con 0x7ff02806f620 2023-10-18T08:55:34.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.762+0000 7ff02d24b640 1 -- 172.21.15.40:0/1431579217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff028163630 con 0x7ff02806f620 2023-10-18T08:55:34.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.762+0000 7ff01f7fe640 1 -- 172.21.15.40:0/1431579217 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff01801a3b0 con 0x7ff02806f620 2023-10-18T08:55:34.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.763+0000 7ff01f7fe640 1 --2- 172.21.15.40:0/1431579217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff00806ae40 0x7ff00806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:34.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.763+0000 7ff01f7fe640 1 -- 172.21.15.40:0/1431579217 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff018089f50 con 0x7ff02806f620 2023-10-18T08:55:34.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.764+0000 7ff02659c640 1 --2- 172.21.15.40:0/1431579217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff00806ae40 0x7ff00806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:34.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.765+0000 7ff02659c640 1 --2- 172.21.15.40:0/1431579217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff00806ae40 0x7ff00806d300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff010001640 tx=0x7ff010009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:34.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.768+0000 7ff01f7fe640 1 -- 172.21.15.40:0/1431579217 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff018054f60 con 0x7ff02806f620 2023-10-18T08:55:34.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.962+0000 7ff02d24b640 1 -- 172.21.15.40:0/1431579217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff028066fd0 con 0x7ff02806f620 2023-10-18T08:55:34.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.964+0000 7ff01f7fe640 1 -- 172.21.15.40:0/1431579217 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7ff018016090 con 0x7ff02806f620 2023-10-18T08:55:34.968 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:34.968 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:16.275677+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:34.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.968+0000 7ff02d24b640 1 -- 172.21.15.40:0/1431579217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff00806ae40 msgr2=0x7ff00806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:34.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.968+0000 7ff02d24b640 1 --2- 172.21.15.40:0/1431579217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff00806ae40 0x7ff00806d300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff010001640 tx=0x7ff010009040 comp rx=0 tx=0).stop 2023-10-18T08:55:34.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.968+0000 7ff02d24b640 1 -- 172.21.15.40:0/1431579217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff02806f620 msgr2=0x7ff02814f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:34.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.968+0000 7ff02d24b640 1 --2- 172.21.15.40:0/1431579217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff02806f620 0x7ff02814f9a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff01800cb50 tx=0x7ff01800a430 comp rx=0 tx=0).stop 2023-10-18T08:55:34.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.968+0000 7ff02d24b640 1 -- 172.21.15.40:0/1431579217 shutdown_connections 2023-10-18T08:55:34.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.968+0000 7ff02d24b640 1 --2- 172.21.15.40:0/1431579217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff00806ae40 0x7ff00806d300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.968+0000 7ff02d24b640 1 --2- 172.21.15.40:0/1431579217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff028158160 0x7ff0281523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.968+0000 7ff02d24b640 1 --2- 172.21.15.40:0/1431579217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0280b1b10 0x7ff028148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.968+0000 7ff02d24b640 1 --2- 172.21.15.40:0/1431579217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff02806f620 0x7ff02814f9a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:34.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.968+0000 7ff02d24b640 1 -- 172.21.15.40:0/1431579217 >> 172.21.15.40:0/1431579217 conn(0x7ff02806cee0 msgr2=0x7ff0280af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:34.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.968+0000 7ff02d24b640 1 -- 172.21.15.40:0/1431579217 shutdown_connections 2023-10-18T08:55:34.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:34.968+0000 7ff02d24b640 1 -- 172.21.15.40:0/1431579217 wait complete. 2023-10-18T08:55:34.988 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:55:13.210788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:55:13.772849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:55:34.989 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:55:35.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.214+0000 7f40faaca640 1 -- 172.21.15.40:0/3402249344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f40aec00 msgr2=0x7f40f40af070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:35.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.214+0000 7f40faaca640 1 --2- 172.21.15.40:0/3402249344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f40aec00 0x7f40f40af070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f40f40afca0 tx=0x7f40ec02c1d0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.215+0000 7f40faaca640 1 -- 172.21.15.40:0/3402249344 shutdown_connections 2023-10-18T08:55:35.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.215+0000 7f40faaca640 1 --2- 172.21.15.40:0/3402249344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f40aec00 0x7f40f40af070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.215+0000 7f40faaca640 1 --2- 172.21.15.40:0/3402249344 >> v2:172.21.15.144:3300/0 conn(0x7f40f40b2c20 0x7f40f40ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.215+0000 7f40faaca640 1 -- 172.21.15.40:0/3402249344 >> 172.21.15.40:0/3402249344 conn(0x7f40f4069720 msgr2=0x7f40f4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:35.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.215+0000 7f40faaca640 1 -- 172.21.15.40:0/3402249344 shutdown_connections 2023-10-18T08:55:35.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.215+0000 7f40faaca640 1 -- 172.21.15.40:0/3402249344 wait complete. 2023-10-18T08:55:35.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.216+0000 7f40faaca640 1 Processor -- start 2023-10-18T08:55:35.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.217+0000 7f40faaca640 1 -- start start 2023-10-18T08:55:35.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.217+0000 7f40faaca640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40f40aec00 0x7f40f41532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:35.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.218+0000 7f40faaca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40f40b2c20 0x7f40f4146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:35.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.218+0000 7f40faaca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f415a200 0x7f40f4151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:35.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.218+0000 7f40faaca640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f41f4ff0 con 0x7f40f415a200 2023-10-18T08:55:35.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.218+0000 7f40faaca640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f4147110 con 0x7f40f40b2c20 2023-10-18T08:55:35.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.218+0000 7f40faaca640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f41f5240 con 0x7f40f40aec00 2023-10-18T08:55:35.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.218+0000 7f40fa2c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f415a200 0x7f40f4151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:35.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.218+0000 7f40fa2c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f415a200 0x7f40f4151880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52030/0 (socket says 172.21.15.40:52030) 2023-10-18T08:55:35.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.218+0000 7f40fa2c9640 1 -- 172.21.15.40:0/3733484398 learned_addr learned my addr 172.21.15.40:0/3733484398 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:35.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.218+0000 7f40f9ac8640 1 --2- 172.21.15.40:0/3733484398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40f40aec00 0x7f40f41532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:35.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.218+0000 7f40f92c7640 1 --2- 172.21.15.40:0/3733484398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40f40b2c20 0x7f40f4146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:35.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.218+0000 7f40fa2c9640 1 -- 172.21.15.40:0/3733484398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40f40aec00 msgr2=0x7f40f41532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:35.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.218+0000 7f40fa2c9640 1 --2- 172.21.15.40:0/3733484398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40f40aec00 0x7f40f41532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.219+0000 7f40fa2c9640 1 -- 172.21.15.40:0/3733484398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40f40b2c20 msgr2=0x7f40f4146600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:35.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.219+0000 7f40fa2c9640 1 --2- 172.21.15.40:0/3733484398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40f40b2c20 0x7f40f4146600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.219+0000 7f40fa2c9640 1 -- 172.21.15.40:0/3733484398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec009900 con 0x7f40f415a200 2023-10-18T08:55:35.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.219+0000 7f40f9ac8640 1 --2- 172.21.15.40:0/3733484398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40f40aec00 0x7f40f41532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:55:35.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.219+0000 7f40fa2c9640 1 --2- 172.21.15.40:0/3733484398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f415a200 0x7f40f4151880 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f40ec037040 tx=0x7f40ec035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:35.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.219+0000 7f40ea7fc640 1 -- 172.21.15.40:0/3733484398 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec034440 con 0x7f40f415a200 2023-10-18T08:55:35.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.219+0000 7f40ea7fc640 1 -- 172.21.15.40:0/3733484398 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f40ec004540 con 0x7f40f415a200 2023-10-18T08:55:35.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.219+0000 7f40ea7fc640 1 -- 172.21.15.40:0/3733484398 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec003e60 con 0x7f40f415a200 2023-10-18T08:55:35.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.219+0000 7f40faaca640 1 -- 172.21.15.40:0/3733484398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f41f5520 con 0x7f40f415a200 2023-10-18T08:55:35.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.219+0000 7f40faaca640 1 -- 172.21.15.40:0/3733484398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f41f5ae0 con 0x7f40f415a200 2023-10-18T08:55:35.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.221+0000 7f40faaca640 1 -- 172.21.15.40:0/3733484398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f41f5d00 con 0x7f40f415a200 2023-10-18T08:55:35.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.222+0000 7f40ea7fc640 1 -- 172.21.15.40:0/3733484398 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f40ec03b040 con 0x7f40f415a200 2023-10-18T08:55:35.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.223+0000 7f40ea7fc640 1 --2- 172.21.15.40:0/3733484398 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f40d806ae40 0x7f40d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:35.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.223+0000 7f40ea7fc640 1 -- 172.21.15.40:0/3733484398 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40ec0ac0a0 con 0x7f40f415a200 2023-10-18T08:55:35.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.223+0000 7f40f9ac8640 1 --2- 172.21.15.40:0/3733484398 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f40d806ae40 0x7f40d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:35.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.224+0000 7f40f9ac8640 1 --2- 172.21.15.40:0/3733484398 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f40d806ae40 0x7f40d806d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40f4153a20 tx=0x7f40f0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:35.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.227+0000 7f40ea7fc640 1 -- 172.21.15.40:0/3733484398 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40ec077140 con 0x7f40f415a200 2023-10-18T08:55:35.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.419+0000 7f40faaca640 1 -- 172.21.15.40:0/3733484398 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40f4066e00 con 0x7f40f415a200 2023-10-18T08:55:35.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.420+0000 7f40ea7fc640 1 -- 172.21.15.40:0/3733484398 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40ec07a760 con 0x7f40f415a200 2023-10-18T08:55:35.424 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:55:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.423+0000 7f40faaca640 1 -- 172.21.15.40:0/3733484398 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f40d806ae40 msgr2=0x7f40d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:35.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.423+0000 7f40faaca640 1 --2- 172.21.15.40:0/3733484398 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f40d806ae40 0x7f40d806d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40f4153a20 tx=0x7f40f0008040 comp rx=0 tx=0).stop 2023-10-18T08:55:35.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.423+0000 7f40faaca640 1 -- 172.21.15.40:0/3733484398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f415a200 msgr2=0x7f40f4151880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:35.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.423+0000 7f40faaca640 1 --2- 172.21.15.40:0/3733484398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f415a200 0x7f40f4151880 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f40ec037040 tx=0x7f40ec035db0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.423+0000 7f40faaca640 1 -- 172.21.15.40:0/3733484398 shutdown_connections 2023-10-18T08:55:35.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.423+0000 7f40faaca640 1 --2- 172.21.15.40:0/3733484398 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f40d806ae40 0x7f40d806d300 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.423+0000 7f40faaca640 1 --2- 172.21.15.40:0/3733484398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f415a200 0x7f40f4151880 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.423+0000 7f40faaca640 1 --2- 172.21.15.40:0/3733484398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40f40b2c20 0x7f40f4146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.423+0000 7f40faaca640 1 --2- 172.21.15.40:0/3733484398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40f40aec00 0x7f40f41532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.423+0000 7f40faaca640 1 -- 172.21.15.40:0/3733484398 >> 172.21.15.40:0/3733484398 conn(0x7f40f4069720 msgr2=0x7f40f40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:35.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.423+0000 7f40faaca640 1 -- 172.21.15.40:0/3733484398 shutdown_connections 2023-10-18T08:55:35.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.423+0000 7f40faaca640 1 -- 172.21.15.40:0/3733484398 wait complete. 2023-10-18T08:55:35.445 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:55:35.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.675+0000 7f1fa1847640 1 -- 172.21.15.40:0/1189678963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f9c15af70 msgr2=0x7f1f9c0710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:35.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.675+0000 7f1fa1847640 1 --2- 172.21.15.40:0/1189678963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f9c15af70 0x7f1f9c0710a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1f94005fd0 tx=0x7f1f9402c240 comp rx=0 tx=0).stop 2023-10-18T08:55:35.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.675+0000 7f1fa1847640 1 -- 172.21.15.40:0/1189678963 shutdown_connections 2023-10-18T08:55:35.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.675+0000 7f1fa1847640 1 --2- 172.21.15.40:0/1189678963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f9c15af70 0x7f1f9c0710a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.675+0000 7f1fa1847640 1 --2- 172.21.15.40:0/1189678963 >> v2:172.21.15.144:3300/0 conn(0x7f1f9c06c2d0 0x7f1f9c070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.675+0000 7f1fa1847640 1 -- 172.21.15.40:0/1189678963 >> 172.21.15.40:0/1189678963 conn(0x7f1f9c06ca10 msgr2=0x7f1f9c06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.676+0000 7f1fa1847640 1 -- 172.21.15.40:0/1189678963 shutdown_connections 2023-10-18T08:55:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.676+0000 7f1fa1847640 1 -- 172.21.15.40:0/1189678963 wait complete. 2023-10-18T08:55:35.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.677+0000 7f1fa1847640 1 Processor -- start 2023-10-18T08:55:35.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.677+0000 7f1fa1847640 1 -- start start 2023-10-18T08:55:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.678+0000 7f1fa1847640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f9c06c2d0 0x7f1f9c14ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.678+0000 7f1fa1847640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f9c0c9620 0x7f1f9c0abe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.678+0000 7f1fa1847640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f9c15af70 0x7f1f9c0ac360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.678+0000 7f1fa1847640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c156e10 con 0x7f1f9c06c2d0 2023-10-18T08:55:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.678+0000 7f1fa1847640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c156540 con 0x7f1f9c15af70 2023-10-18T08:55:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.678+0000 7f1fa1847640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c0c2020 con 0x7f1f9c0c9620 2023-10-18T08:55:35.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.679+0000 7f1fa0845640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f9c06c2d0 0x7f1f9c14ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:35.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.679+0000 7f1fa0845640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f9c06c2d0 0x7f1f9c14ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52046/0 (socket says 172.21.15.40:52046) 2023-10-18T08:55:35.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.679+0000 7f1fa0845640 1 -- 172.21.15.40:0/694493694 learned_addr learned my addr 172.21.15.40:0/694493694 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:35.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.679+0000 7f1fa1046640 1 --2- 172.21.15.40:0/694493694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f9c15af70 0x7f1f9c0ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.679+0000 7f1f93fff640 1 --2- 172.21.15.40:0/694493694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f9c0c9620 0x7f1f9c0abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.680+0000 7f1fa0845640 1 -- 172.21.15.40:0/694493694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f9c0c9620 msgr2=0x7f1f9c0abe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.680+0000 7f1fa0845640 1 --2- 172.21.15.40:0/694493694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f9c0c9620 0x7f1f9c0abe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.680+0000 7f1fa0845640 1 -- 172.21.15.40:0/694493694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f9c15af70 msgr2=0x7f1f9c0ac360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.680+0000 7f1fa0845640 1 --2- 172.21.15.40:0/694493694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f9c15af70 0x7f1f9c0ac360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.680+0000 7f1fa0845640 1 -- 172.21.15.40:0/694493694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c009490 con 0x7f1f9c06c2d0 2023-10-18T08:55:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.680+0000 7f1f93fff640 1 --2- 172.21.15.40:0/694493694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f9c0c9620 0x7f1f9c0abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:55:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.680+0000 7f1fa0845640 1 --2- 172.21.15.40:0/694493694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f9c06c2d0 0x7f1f9c14ee20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800c490 tx=0x7f1f9800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.680+0000 7f1f917fa640 1 -- 172.21.15.40:0/694493694 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f980052d0 con 0x7f1f9c06c2d0 2023-10-18T08:55:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.680+0000 7f1f917fa640 1 -- 172.21.15.40:0/694493694 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1f98005430 con 0x7f1f9c06c2d0 2023-10-18T08:55:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.680+0000 7f1f917fa640 1 -- 172.21.15.40:0/694493694 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f98012890 con 0x7f1f9c06c2d0 2023-10-18T08:55:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.680+0000 7f1fa1847640 1 -- 172.21.15.40:0/694493694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f94009900 con 0x7f1f9c06c2d0 2023-10-18T08:55:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.681+0000 7f1fa1847640 1 -- 172.21.15.40:0/694493694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9c156750 con 0x7f1f9c06c2d0 2023-10-18T08:55:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.682+0000 7f1fa1847640 1 -- 172.21.15.40:0/694493694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9c0cdfb0 con 0x7f1f9c06c2d0 2023-10-18T08:55:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.683+0000 7f1f917fa640 1 -- 172.21.15.40:0/694493694 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1f980035f0 con 0x7f1f9c06c2d0 2023-10-18T08:55:35.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.684+0000 7f1f917fa640 1 --2- 172.21.15.40:0/694493694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f8806ae40 0x7f1f8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:35.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.684+0000 7f1f917fa640 1 -- 172.21.15.40:0/694493694 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f98089360 con 0x7f1f9c06c2d0 2023-10-18T08:55:35.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.684+0000 7f1f93fff640 1 --2- 172.21.15.40:0/694493694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f8806ae40 0x7f1f8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:35.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.685+0000 7f1f93fff640 1 --2- 172.21.15.40:0/694493694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f8806ae40 0x7f1f8806d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c1f53f0 tx=0x7f1f8c005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:35.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.688+0000 7f1f917fa640 1 -- 172.21.15.40:0/694493694 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f98054370 con 0x7f1f9c06c2d0 2023-10-18T08:55:35.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.881+0000 7f1fa1847640 1 -- 172.21.15.40:0/694493694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1f9c066e00 con 0x7f1f9c06c2d0 2023-10-18T08:55:35.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.883+0000 7f1f917fa640 1 -- 172.21.15.40:0/694493694 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1f98057990 con 0x7f1f9c06c2d0 2023-10-18T08:55:35.886 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:55:35.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.885+0000 7f1fa1847640 1 -- 172.21.15.40:0/694493694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f8806ae40 msgr2=0x7f1f8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:35.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.885+0000 7f1fa1847640 1 --2- 172.21.15.40:0/694493694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f8806ae40 0x7f1f8806d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c1f53f0 tx=0x7f1f8c005f90 comp rx=0 tx=0).stop 2023-10-18T08:55:35.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.885+0000 7f1fa1847640 1 -- 172.21.15.40:0/694493694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f9c06c2d0 msgr2=0x7f1f9c14ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:35.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.885+0000 7f1fa1847640 1 --2- 172.21.15.40:0/694493694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f9c06c2d0 0x7f1f9c14ee20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800c490 tx=0x7f1f9800c950 comp rx=0 tx=0).stop 2023-10-18T08:55:35.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.885+0000 7f1fa1847640 1 -- 172.21.15.40:0/694493694 shutdown_connections 2023-10-18T08:55:35.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.885+0000 7f1fa1847640 1 --2- 172.21.15.40:0/694493694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f8806ae40 0x7f1f8806d300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.885+0000 7f1fa1847640 1 --2- 172.21.15.40:0/694493694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f9c15af70 0x7f1f9c0ac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.886+0000 7f1fa1847640 1 --2- 172.21.15.40:0/694493694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f9c0c9620 0x7f1f9c0abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.886+0000 7f1fa1847640 1 --2- 172.21.15.40:0/694493694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f9c06c2d0 0x7f1f9c14ee20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:35.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.886+0000 7f1fa1847640 1 -- 172.21.15.40:0/694493694 >> 172.21.15.40:0/694493694 conn(0x7f1f9c06ca10 msgr2=0x7f1f9c0ccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:35.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.886+0000 7f1fa1847640 1 -- 172.21.15.40:0/694493694 shutdown_connections 2023-10-18T08:55:35.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:35.886+0000 7f1fa1847640 1 -- 172.21.15.40:0/694493694 wait complete. 2023-10-18T08:55:35.908 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:55:36.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.134+0000 7f31f9125640 1 -- 172.21.15.40:0/198276803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31dc0047c0 msgr2=0x7f31dc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:36.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.134+0000 7f31f9125640 1 --2- 172.21.15.40:0/198276803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31dc0047c0 0x7f31dc004c40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f31f4158ae0 tx=0x7f31dc0057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.134+0000 7f31f359e640 1 -- 172.21.15.40:0/198276803 reap_dead start 2023-10-18T08:55:36.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.134+0000 7f31f9125640 1 -- 172.21.15.40:0/198276803 shutdown_connections 2023-10-18T08:55:36.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.134+0000 7f31f9125640 1 -- 172.21.15.40:0/198276803 >> 172.21.15.40:0/198276803 conn(0x7f31f4069720 msgr2=0x7f31f4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:36.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.134+0000 7f31f9125640 1 -- 172.21.15.40:0/198276803 shutdown_connections 2023-10-18T08:55:36.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.135+0000 7f31f9125640 1 -- 172.21.15.40:0/198276803 wait complete. 2023-10-18T08:55:36.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.136+0000 7f31f9125640 1 Processor -- start 2023-10-18T08:55:36.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.137+0000 7f31f9125640 1 -- start start 2023-10-18T08:55:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.138+0000 7f31f9125640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31dc0047c0 0x7f31f4154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.138+0000 7f31f9125640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31f4150bc0 0x7f31f41485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.138+0000 7f31f9125640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31f4143000 0x7f31f4143460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:36.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.138+0000 7f31f9125640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f4156220 con 0x7f31f4150bc0 2023-10-18T08:55:36.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.138+0000 7f31f9125640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f41f0250 con 0x7f31f4143000 2023-10-18T08:55:36.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.138+0000 7f31f9125640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f41439a0 con 0x7f31dc0047c0 2023-10-18T08:55:36.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.138+0000 7f31f259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31f4150bc0 0x7f31f41485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:36.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.138+0000 7f31f259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31f4150bc0 0x7f31f41485b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52064/0 (socket says 172.21.15.40:52064) 2023-10-18T08:55:36.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.138+0000 7f31f259c640 1 -- 172.21.15.40:0/1680023213 learned_addr learned my addr 172.21.15.40:0/1680023213 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:36.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.138+0000 7f31f2d9d640 1 --2- 172.21.15.40:0/1680023213 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31dc0047c0 0x7f31f4154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:36.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.138+0000 7f31f359e640 1 --2- 172.21.15.40:0/1680023213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31f4143000 0x7f31f4143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:36.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.138+0000 7f31f259c640 1 -- 172.21.15.40:0/1680023213 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31dc0047c0 msgr2=0x7f31f4154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:36.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.138+0000 7f31f259c640 1 --2- 172.21.15.40:0/1680023213 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31dc0047c0 0x7f31f4154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.139+0000 7f31f259c640 1 -- 172.21.15.40:0/1680023213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31f4143000 msgr2=0x7f31f4143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:36.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.139+0000 7f31f259c640 1 --2- 172.21.15.40:0/1680023213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31f4143000 0x7f31f4143460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.139+0000 7f31f259c640 1 -- 172.21.15.40:0/1680023213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f4155440 con 0x7f31f4150bc0 2023-10-18T08:55:36.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.139+0000 7f31f359e640 1 --2- 172.21.15.40:0/1680023213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31f4143000 0x7f31f4143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:36.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.139+0000 7f31f259c640 1 --2- 172.21.15.40:0/1680023213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31f4150bc0 0x7f31f41485b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f31e800bbe0 tx=0x7f31e80366b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:36.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.139+0000 7f31db7fe640 1 -- 172.21.15.40:0/1680023213 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8036cb0 con 0x7f31f4150bc0 2023-10-18T08:55:36.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.139+0000 7f31db7fe640 1 -- 172.21.15.40:0/1680023213 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f31e8004030 con 0x7f31f4150bc0 2023-10-18T08:55:36.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.139+0000 7f31f9125640 1 -- 172.21.15.40:0/1680023213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f4155720 con 0x7f31f4150bc0 2023-10-18T08:55:36.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.139+0000 7f31db7fe640 1 -- 172.21.15.40:0/1680023213 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8004400 con 0x7f31f4150bc0 2023-10-18T08:55:36.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.139+0000 7f31f9125640 1 -- 172.21.15.40:0/1680023213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f4156470 con 0x7f31f4150bc0 2023-10-18T08:55:36.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.141+0000 7f31f9125640 1 -- 172.21.15.40:0/1680023213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f4157b10 con 0x7f31f4150bc0 2023-10-18T08:55:36.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.142+0000 7f31db7fe640 1 -- 172.21.15.40:0/1680023213 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f31e80035f0 con 0x7f31f4150bc0 2023-10-18T08:55:36.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.143+0000 7f31db7fe640 1 --2- 172.21.15.40:0/1680023213 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31e006ae70 0x7f31e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:36.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.143+0000 7f31db7fe640 1 -- 172.21.15.40:0/1680023213 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31e80ab6c0 con 0x7f31f4150bc0 2023-10-18T08:55:36.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.144+0000 7f31f2d9d640 1 --2- 172.21.15.40:0/1680023213 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31e006ae70 0x7f31e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:36.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.145+0000 7f31f2d9d640 1 --2- 172.21.15.40:0/1680023213 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31e006ae70 0x7f31e006d330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f31e40017a0 tx=0x7f31e4001be0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:36.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.148+0000 7f31db7fe640 1 -- 172.21.15.40:0/1680023213 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f31e8076770 con 0x7f31f4150bc0 2023-10-18T08:55:36.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.344+0000 7f31f9125640 1 -- 172.21.15.40:0/1680023213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f31f4066fd0 con 0x7f31f4150bc0 2023-10-18T08:55:36.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.345+0000 7f31db7fe640 1 -- 172.21.15.40:0/1680023213 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f31e8032020 con 0x7f31f4150bc0 2023-10-18T08:55:36.349 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:55:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.348+0000 7f31f9125640 1 -- 172.21.15.40:0/1680023213 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31e006ae70 msgr2=0x7f31e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.348+0000 7f31f9125640 1 --2- 172.21.15.40:0/1680023213 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31e006ae70 0x7f31e006d330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f31e40017a0 tx=0x7f31e4001be0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.348+0000 7f31f9125640 1 -- 172.21.15.40:0/1680023213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31f4150bc0 msgr2=0x7f31f41485b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.348+0000 7f31f9125640 1 --2- 172.21.15.40:0/1680023213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31f4150bc0 0x7f31f41485b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f31e800bbe0 tx=0x7f31e80366b0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.348+0000 7f31f9125640 1 -- 172.21.15.40:0/1680023213 shutdown_connections 2023-10-18T08:55:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.348+0000 7f31f9125640 1 --2- 172.21.15.40:0/1680023213 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31e006ae70 0x7f31e006d330 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.348+0000 7f31f9125640 1 --2- 172.21.15.40:0/1680023213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31f4143000 0x7f31f4143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.348+0000 7f31f9125640 1 --2- 172.21.15.40:0/1680023213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31f4150bc0 0x7f31f41485b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.348+0000 7f31f9125640 1 --2- 172.21.15.40:0/1680023213 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31dc0047c0 0x7f31f4154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.348+0000 7f31f9125640 1 -- 172.21.15.40:0/1680023213 >> 172.21.15.40:0/1680023213 conn(0x7f31f4069720 msgr2=0x7f31f4149db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:36.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.349+0000 7f31f9125640 1 -- 172.21.15.40:0/1680023213 shutdown_connections 2023-10-18T08:55:36.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.349+0000 7f31f9125640 1 -- 172.21.15.40:0/1680023213 wait complete. 2023-10-18T08:55:36.374 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:55:36.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.593+0000 7f7f7bfff640 1 -- 172.21.15.40:0/1588642879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f7c0ca0d0 msgr2=0x7f7f7c157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.593+0000 7f7f7bfff640 1 --2- 172.21.15.40:0/1588642879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f7c0ca0d0 0x7f7f7c157d40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7f74005fd0 tx=0x7f7f7402c2e0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.593+0000 7f7f7bfff640 1 -- 172.21.15.40:0/1588642879 shutdown_connections 2023-10-18T08:55:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.593+0000 7f7f7bfff640 1 --2- 172.21.15.40:0/1588642879 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f7c0ca0d0 0x7f7f7c157d40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.593+0000 7f7f7bfff640 1 --2- 172.21.15.40:0/1588642879 >> v2:172.21.15.144:3300/0 conn(0x7f7f7c06d6f0 0x7f7f7c0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.593+0000 7f7f7bfff640 1 -- 172.21.15.40:0/1588642879 >> 172.21.15.40:0/1588642879 conn(0x7f7f7c069720 msgr2=0x7f7f7c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.593+0000 7f7f7bfff640 1 -- 172.21.15.40:0/1588642879 shutdown_connections 2023-10-18T08:55:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.593+0000 7f7f7bfff640 1 -- 172.21.15.40:0/1588642879 wait complete. 2023-10-18T08:55:36.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.595+0000 7f7f7bfff640 1 Processor -- start 2023-10-18T08:55:36.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.595+0000 7f7f7bfff640 1 -- start start 2023-10-18T08:55:36.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.596+0000 7f7f7bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f7c06d6f0 0x7f7f7c13a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:36.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.596+0000 7f7f7bfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f7c0b0630 0x7f7f7c157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:36.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.596+0000 7f7f7bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f7c0ca0d0 0x7f7f7c0cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:36.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.596+0000 7f7f7bfff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c159380 con 0x7f7f7c06d6f0 2023-10-18T08:55:36.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.596+0000 7f7f7bfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c158700 con 0x7f7f7c0ca0d0 2023-10-18T08:55:36.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.596+0000 7f7f7bfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c1589e0 con 0x7f7f7c0b0630 2023-10-18T08:55:36.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.597+0000 7f7f7affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f7c06d6f0 0x7f7f7c13a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:36.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.597+0000 7f7f7affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f7c06d6f0 0x7f7f7c13a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52066/0 (socket says 172.21.15.40:52066) 2023-10-18T08:55:36.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.597+0000 7f7f7affd640 1 -- 172.21.15.40:0/4044972217 learned_addr learned my addr 172.21.15.40:0/4044972217 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:36.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.597+0000 7f7f7a7fc640 1 --2- 172.21.15.40:0/4044972217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f7c0b0630 0x7f7f7c157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:36.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.597+0000 7f7f7b7fe640 1 --2- 172.21.15.40:0/4044972217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f7c0ca0d0 0x7f7f7c0cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:36.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.597+0000 7f7f7affd640 1 -- 172.21.15.40:0/4044972217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f7c0b0630 msgr2=0x7f7f7c157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:36.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.597+0000 7f7f7affd640 1 --2- 172.21.15.40:0/4044972217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f7c0b0630 0x7f7f7c157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.597+0000 7f7f7affd640 1 -- 172.21.15.40:0/4044972217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f7c0ca0d0 msgr2=0x7f7f7c0cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:36.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.597+0000 7f7f7affd640 1 --2- 172.21.15.40:0/4044972217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f7c0ca0d0 0x7f7f7c0cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.597+0000 7f7f7affd640 1 -- 172.21.15.40:0/4044972217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f74009900 con 0x7f7f7c06d6f0 2023-10-18T08:55:36.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.598+0000 7f7f7affd640 1 --2- 172.21.15.40:0/4044972217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f7c06d6f0 0x7f7f7c13a730 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00ea20 tx=0x7f7f6c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:36.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.598+0000 7f7f637fe640 1 -- 172.21.15.40:0/4044972217 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00cd40 con 0x7f7f7c06d6f0 2023-10-18T08:55:36.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.598+0000 7f7f637fe640 1 -- 172.21.15.40:0/4044972217 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7f6c016410 con 0x7f7f7c06d6f0 2023-10-18T08:55:36.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.598+0000 7f7f7bfff640 1 -- 172.21.15.40:0/4044972217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7c158cc0 con 0x7f7f7c06d6f0 2023-10-18T08:55:36.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.598+0000 7f7f637fe640 1 -- 172.21.15.40:0/4044972217 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c016820 con 0x7f7f7c06d6f0 2023-10-18T08:55:36.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.598+0000 7f7f7bfff640 1 -- 172.21.15.40:0/4044972217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7c1597a0 con 0x7f7f7c06d6f0 2023-10-18T08:55:36.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.599+0000 7f7f7bfff640 1 -- 172.21.15.40:0/4044972217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7c066e00 con 0x7f7f7c06d6f0 2023-10-18T08:55:36.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.600+0000 7f7f637fe640 1 -- 172.21.15.40:0/4044972217 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7f6c016980 con 0x7f7f7c06d6f0 2023-10-18T08:55:36.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.601+0000 7f7f637fe640 1 --2- 172.21.15.40:0/4044972217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7f6806ae40 0x7f7f6806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:36.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.601+0000 7f7f637fe640 1 -- 172.21.15.40:0/4044972217 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f6c089ba0 con 0x7f7f7c06d6f0 2023-10-18T08:55:36.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.602+0000 7f7f7a7fc640 1 --2- 172.21.15.40:0/4044972217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7f6806ae40 0x7f7f6806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:36.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.603+0000 7f7f7a7fc640 1 --2- 172.21.15.40:0/4044972217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7f6806ae40 0x7f7f6806d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7f70007fd0 tx=0x7f7f70009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:36.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.606+0000 7f7f637fe640 1 -- 172.21.15.40:0/4044972217 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f6c054cf0 con 0x7f7f7c06d6f0 2023-10-18T08:55:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.839+0000 7f7f7bfff640 1 -- 172.21.15.40:0/4044972217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f7c15bed0 con 0x7f7f7c06d6f0 2023-10-18T08:55:36.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.840+0000 7f7f637fe640 1 -- 172.21.15.40:0/4044972217 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f7f6c058310 con 0x7f7f7c06d6f0 2023-10-18T08:55:36.843 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:36.844 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1962150323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1962150323},{"type":"v1","addr":"172.21.15.144:6837","nonce":1962150323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4329,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2589720838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2589720838},{"type":"v1","addr":"172.21.15.40:6833","nonce":2589720838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3031491371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3031491371},{"type":"v1","addr":"172.21.15.40:6835","nonce":3031491371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:55:14.753828+0000","modified":"2023-10-18T08:55:17.818729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/682820172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":682820172},{"type":"v1","addr":"172.21.15.144:6835","nonce":682820172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T08:55:36.844 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2023-10-18T08:55:36.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.842+0000 7f7f7bfff640 1 -- 172.21.15.40:0/4044972217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7f6806ae40 msgr2=0x7f7f6806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:36.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.842+0000 7f7f7bfff640 1 --2- 172.21.15.40:0/4044972217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7f6806ae40 0x7f7f6806d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7f70007fd0 tx=0x7f7f70009040 comp rx=0 tx=0).stop 2023-10-18T08:55:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.842+0000 7f7f7bfff640 1 -- 172.21.15.40:0/4044972217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f7c06d6f0 msgr2=0x7f7f7c13a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.842+0000 7f7f7bfff640 1 --2- 172.21.15.40:0/4044972217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f7c06d6f0 0x7f7f7c13a730 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00ea20 tx=0x7f7f6c00eee0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.843+0000 7f7f7bfff640 1 -- 172.21.15.40:0/4044972217 shutdown_connections 2023-10-18T08:55:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.843+0000 7f7f7bfff640 1 --2- 172.21.15.40:0/4044972217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7f6806ae40 0x7f7f6806d300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.843+0000 7f7f7bfff640 1 --2- 172.21.15.40:0/4044972217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f7c0ca0d0 0x7f7f7c0cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.843+0000 7f7f7bfff640 1 --2- 172.21.15.40:0/4044972217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f7c0b0630 0x7f7f7c157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.843+0000 7f7f7bfff640 1 --2- 172.21.15.40:0/4044972217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f7c06d6f0 0x7f7f7c13a730 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.843+0000 7f7f7bfff640 1 -- 172.21.15.40:0/4044972217 >> 172.21.15.40:0/4044972217 conn(0x7f7f7c069720 msgr2=0x7f7f7c1556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.843+0000 7f7f7bfff640 1 -- 172.21.15.40:0/4044972217 shutdown_connections 2023-10-18T08:55:36.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:36.843+0000 7f7f7bfff640 1 -- 172.21.15.40:0/4044972217 wait complete. 2023-10-18T08:55:36.865 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:37.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.084+0000 7ff3c7ac2640 1 -- 172.21.15.40:0/3080882840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac005180 msgr2=0x7ff3ac00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:37.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.084+0000 7ff3c7ac2640 1 --2- 172.21.15.40:0/3080882840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac005180 0x7ff3ac00b980 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009ed0 tx=0x7ff3ac0057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.085+0000 7ff3c7ac2640 1 -- 172.21.15.40:0/3080882840 shutdown_connections 2023-10-18T08:55:37.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.085+0000 7ff3c7ac2640 1 --2- 172.21.15.40:0/3080882840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac005180 0x7ff3ac00b980 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.085+0000 7ff3c7ac2640 1 -- 172.21.15.40:0/3080882840 >> 172.21.15.40:0/3080882840 conn(0x7ff3c006cee0 msgr2=0x7ff3c0069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:37.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.085+0000 7ff3c7ac2640 1 -- 172.21.15.40:0/3080882840 shutdown_connections 2023-10-18T08:55:37.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.085+0000 7ff3c7ac2640 1 -- 172.21.15.40:0/3080882840 wait complete. 2023-10-18T08:55:37.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.086+0000 7ff3c7ac2640 1 Processor -- start 2023-10-18T08:55:37.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.087+0000 7ff3c7ac2640 1 -- start start 2023-10-18T08:55:37.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.087+0000 7ff3c7ac2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3ac005180 0x7ff3c01522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:37.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.088+0000 7ff3c7ac2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c0150110 0x7ff3c0150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:37.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.088+0000 7ff3c7ac2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3c0132c00 0x7ff3c0133080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:37.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.088+0000 7ff3c7ac2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c0150ad0 con 0x7ff3c0150110 2023-10-18T08:55:37.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.088+0000 7ff3c7ac2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c01308c0 con 0x7ff3c0132c00 2023-10-18T08:55:37.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.088+0000 7ff3c7ac2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c0130ba0 con 0x7ff3ac005180 2023-10-18T08:55:37.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.088+0000 7ff3c505d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c0150110 0x7ff3c0150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:37.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.088+0000 7ff3c505d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c0150110 0x7ff3c0150590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52092/0 (socket says 172.21.15.40:52092) 2023-10-18T08:55:37.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.088+0000 7ff3c505d640 1 -- 172.21.15.40:0/874661043 learned_addr learned my addr 172.21.15.40:0/874661043 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:37.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.088+0000 7ff3c605f640 1 --2- 172.21.15.40:0/874661043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3c0132c00 0x7ff3c0133080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:37.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.088+0000 7ff3c585e640 1 --2- 172.21.15.40:0/874661043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3ac005180 0x7ff3c01522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:37.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.088+0000 7ff3c505d640 1 -- 172.21.15.40:0/874661043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3ac005180 msgr2=0x7ff3c01522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:37.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.088+0000 7ff3c505d640 1 --2- 172.21.15.40:0/874661043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3ac005180 0x7ff3c01522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.088+0000 7ff3c505d640 1 -- 172.21.15.40:0/874661043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3c0132c00 msgr2=0x7ff3c0133080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:37.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.088+0000 7ff3c505d640 1 --2- 172.21.15.40:0/874661043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3c0132c00 0x7ff3c0133080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.088+0000 7ff3c505d640 1 -- 172.21.15.40:0/874661043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c0130e80 con 0x7ff3c0150110 2023-10-18T08:55:37.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.088+0000 7ff3c605f640 1 --2- 172.21.15.40:0/874661043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3c0132c00 0x7ff3c0133080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:55:37.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.089+0000 7ff3c505d640 1 --2- 172.21.15.40:0/874661043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c0150110 0x7ff3c0150590 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0001050 tx=0x7ff3b000ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:37.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.089+0000 7ff3b67fc640 1 -- 172.21.15.40:0/874661043 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0010890 con 0x7ff3c0150110 2023-10-18T08:55:37.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.089+0000 7ff3b67fc640 1 -- 172.21.15.40:0/874661043 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff3b0010b20 con 0x7ff3c0150110 2023-10-18T08:55:37.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.089+0000 7ff3c7ac2640 1 -- 172.21.15.40:0/874661043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c0131160 con 0x7ff3c0150110 2023-10-18T08:55:37.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.089+0000 7ff3b67fc640 1 -- 172.21.15.40:0/874661043 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0011460 con 0x7ff3c0150110 2023-10-18T08:55:37.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.089+0000 7ff3c7ac2640 1 -- 172.21.15.40:0/874661043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c01529c0 con 0x7ff3c0150110 2023-10-18T08:55:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.090+0000 7ff3c7ac2640 1 -- 172.21.15.40:0/874661043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c0066fd0 con 0x7ff3c0150110 2023-10-18T08:55:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.092+0000 7ff3b67fc640 1 -- 172.21.15.40:0/874661043 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff3b00115c0 con 0x7ff3c0150110 2023-10-18T08:55:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.093+0000 7ff3b67fc640 1 --2- 172.21.15.40:0/874661043 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff3a406ae70 0x7ff3a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:37.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.093+0000 7ff3b67fc640 1 -- 172.21.15.40:0/874661043 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3b0089dc0 con 0x7ff3c0150110 2023-10-18T08:55:37.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.093+0000 7ff3c585e640 1 --2- 172.21.15.40:0/874661043 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff3a406ae70 0x7ff3a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:37.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.094+0000 7ff3c585e640 1 --2- 172.21.15.40:0/874661043 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff3a406ae70 0x7ff3a406d330 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc000c00 tx=0x7ff3bc006050 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:37.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.097+0000 7ff3b67fc640 1 -- 172.21.15.40:0/874661043 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff3b0054de0 con 0x7ff3c0150110 2023-10-18T08:55:37.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.295+0000 7ff3c7ac2640 1 -- 172.21.15.40:0/874661043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3c006fb50 con 0x7ff3c0150110 2023-10-18T08:55:37.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.297+0000 7ff3b67fc640 1 -- 172.21.15.40:0/874661043 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7ff3b001f070 con 0x7ff3c0150110 2023-10-18T08:55:37.300 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:37.301 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:16.275677+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:37.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.300+0000 7ff3c7ac2640 1 -- 172.21.15.40:0/874661043 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff3a406ae70 msgr2=0x7ff3a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:37.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.300+0000 7ff3c7ac2640 1 --2- 172.21.15.40:0/874661043 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff3a406ae70 0x7ff3a406d330 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc000c00 tx=0x7ff3bc006050 comp rx=0 tx=0).stop 2023-10-18T08:55:37.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.300+0000 7ff3c7ac2640 1 -- 172.21.15.40:0/874661043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c0150110 msgr2=0x7ff3c0150590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:37.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.300+0000 7ff3c7ac2640 1 --2- 172.21.15.40:0/874661043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c0150110 0x7ff3c0150590 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0001050 tx=0x7ff3b000ea20 comp rx=0 tx=0).stop 2023-10-18T08:55:37.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.300+0000 7ff3c7ac2640 1 -- 172.21.15.40:0/874661043 shutdown_connections 2023-10-18T08:55:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.300+0000 7ff3c7ac2640 1 --2- 172.21.15.40:0/874661043 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff3a406ae70 0x7ff3a406d330 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.300+0000 7ff3c7ac2640 1 --2- 172.21.15.40:0/874661043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3c0132c00 0x7ff3c0133080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.300+0000 7ff3c7ac2640 1 --2- 172.21.15.40:0/874661043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c0150110 0x7ff3c0150590 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.300+0000 7ff3c7ac2640 1 --2- 172.21.15.40:0/874661043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3ac005180 0x7ff3c01522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.300+0000 7ff3c7ac2640 1 -- 172.21.15.40:0/874661043 >> 172.21.15.40:0/874661043 conn(0x7ff3c006cee0 msgr2=0x7ff3c00ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:37.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.300+0000 7ff3c7ac2640 1 -- 172.21.15.40:0/874661043 shutdown_connections 2023-10-18T08:55:37.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.300+0000 7ff3c7ac2640 1 -- 172.21.15.40:0/874661043 wait complete. 2023-10-18T08:55:37.322 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:55:37.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.553+0000 7f1875b06640 1 -- 172.21.15.40:0/359397727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1870071440 msgr2=0x7f18700aab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:37.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.553+0000 7f1875b06640 1 --2- 172.21.15.40:0/359397727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1870071440 0x7f18700aab90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f185c009860 tx=0x7f185c02bde0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.553+0000 7f1875b06640 1 -- 172.21.15.40:0/359397727 shutdown_connections 2023-10-18T08:55:37.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.553+0000 7f1875b06640 1 --2- 172.21.15.40:0/359397727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1870071440 0x7f18700aab90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.553+0000 7f1875b06640 1 --2- 172.21.15.40:0/359397727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1870152560 0x7f1870070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.553+0000 7f1875b06640 1 -- 172.21.15.40:0/359397727 >> 172.21.15.40:0/359397727 conn(0x7f1870069720 msgr2=0x7f1870069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:37.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.553+0000 7f1875b06640 1 -- 172.21.15.40:0/359397727 shutdown_connections 2023-10-18T08:55:37.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.553+0000 7f1875b06640 1 -- 172.21.15.40:0/359397727 wait complete. 2023-10-18T08:55:37.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.555+0000 7f1875b06640 1 Processor -- start 2023-10-18T08:55:37.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.555+0000 7f1875b06640 1 -- start start 2023-10-18T08:55:37.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.556+0000 7f1875b06640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1870071440 0x7f1870147820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:37.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.556+0000 7f1875b06640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18700ab0d0 0x7f1870148170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:37.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.556+0000 7f1875b06640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1870152560 0x7f1870149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:37.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.556+0000 7f1875b06640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187014ee70 con 0x7f18700ab0d0 2023-10-18T08:55:37.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.556+0000 7f1875b06640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18701489c0 con 0x7f1870152560 2023-10-18T08:55:37.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.556+0000 7f1875b06640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1870148ca0 con 0x7f1870071440 2023-10-18T08:55:37.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.557+0000 7f186ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18700ab0d0 0x7f1870148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:37.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.557+0000 7f186ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18700ab0d0 0x7f1870148170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52106/0 (socket says 172.21.15.40:52106) 2023-10-18T08:55:37.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.557+0000 7f186ffff640 1 -- 172.21.15.40:0/1610493275 learned_addr learned my addr 172.21.15.40:0/1610493275 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:37.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.557+0000 7f1875305640 1 --2- 172.21.15.40:0/1610493275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1870152560 0x7f1870149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:37.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.557+0000 7f1874b04640 1 --2- 172.21.15.40:0/1610493275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1870071440 0x7f1870147820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:37.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.557+0000 7f186ffff640 1 -- 172.21.15.40:0/1610493275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1870071440 msgr2=0x7f1870147820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:37.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.557+0000 7f186ffff640 1 --2- 172.21.15.40:0/1610493275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1870071440 0x7f1870147820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.557+0000 7f186ffff640 1 -- 172.21.15.40:0/1610493275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1870152560 msgr2=0x7f1870149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:37.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.557+0000 7f186ffff640 1 --2- 172.21.15.40:0/1610493275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1870152560 0x7f1870149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.557+0000 7f186ffff640 1 -- 172.21.15.40:0/1610493275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c009490 con 0x7f18700ab0d0 2023-10-18T08:55:37.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.557+0000 7f186ffff640 1 --2- 172.21.15.40:0/1610493275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18700ab0d0 0x7f1870148170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f185c009980 tx=0x7f185c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:37.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.558+0000 7f186d7fa640 1 -- 172.21.15.40:0/1610493275 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c003ea0 con 0x7f18700ab0d0 2023-10-18T08:55:37.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.558+0000 7f186d7fa640 1 -- 172.21.15.40:0/1610493275 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f185c034440 con 0x7f18700ab0d0 2023-10-18T08:55:37.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.558+0000 7f186d7fa640 1 -- 172.21.15.40:0/1610493275 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c034850 con 0x7f18700ab0d0 2023-10-18T08:55:37.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.558+0000 7f1875b06640 1 -- 172.21.15.40:0/1610493275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187014eb00 con 0x7f18700ab0d0 2023-10-18T08:55:37.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.558+0000 7f1875b06640 1 -- 172.21.15.40:0/1610493275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18701486b0 con 0x7f18700ab0d0 2023-10-18T08:55:37.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.559+0000 7f1875b06640 1 -- 172.21.15.40:0/1610493275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1870066e00 con 0x7f18700ab0d0 2023-10-18T08:55:37.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.560+0000 7f186d7fa640 1 -- 172.21.15.40:0/1610493275 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f185c03b040 con 0x7f18700ab0d0 2023-10-18T08:55:37.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.561+0000 7f186d7fa640 1 --2- 172.21.15.40:0/1610493275 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f185406ae40 0x7f185406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:37.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.561+0000 7f186d7fa640 1 -- 172.21.15.40:0/1610493275 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f185c0abb90 con 0x7f18700ab0d0 2023-10-18T08:55:37.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.562+0000 7f1874b04640 1 --2- 172.21.15.40:0/1610493275 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f185406ae40 0x7f185406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:37.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.563+0000 7f1874b04640 1 --2- 172.21.15.40:0/1610493275 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f185406ae40 0x7f185406d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1864009ac0 tx=0x7f1864005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:37.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.565+0000 7f186d7fa640 1 -- 172.21.15.40:0/1610493275 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f185c076ba0 con 0x7f18700ab0d0 2023-10-18T08:55:37.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.797+0000 7f1875b06640 1 -- 172.21.15.40:0/1610493275 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18700aac20 con 0x7f18700ab0d0 2023-10-18T08:55:37.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.798+0000 7f186d7fa640 1 -- 172.21.15.40:0/1610493275 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f185c041090 con 0x7f18700ab0d0 2023-10-18T08:55:37.801 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:37.802 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1962150323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1962150323},{"type":"v1","addr":"172.21.15.144:6837","nonce":1962150323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4329,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2589720838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2589720838},{"type":"v1","addr":"172.21.15.40:6833","nonce":2589720838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3031491371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3031491371},{"type":"v1","addr":"172.21.15.40:6835","nonce":3031491371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:55:14.753828+0000","modified":"2023-10-18T08:55:17.818729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/682820172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":682820172},{"type":"v1","addr":"172.21.15.144:6835","nonce":682820172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T08:55:37.802 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2023-10-18T08:55:37.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.801+0000 7f1875b06640 1 -- 172.21.15.40:0/1610493275 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f185406ae40 msgr2=0x7f185406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:37.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.801+0000 7f1875b06640 1 --2- 172.21.15.40:0/1610493275 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f185406ae40 0x7f185406d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1864009ac0 tx=0x7f1864005f90 comp rx=0 tx=0).stop 2023-10-18T08:55:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.801+0000 7f1875b06640 1 -- 172.21.15.40:0/1610493275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18700ab0d0 msgr2=0x7f1870148170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.801+0000 7f1875b06640 1 --2- 172.21.15.40:0/1610493275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18700ab0d0 0x7f1870148170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f185c009980 tx=0x7f185c0038a0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.801+0000 7f1875b06640 1 -- 172.21.15.40:0/1610493275 shutdown_connections 2023-10-18T08:55:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.801+0000 7f1875b06640 1 --2- 172.21.15.40:0/1610493275 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f185406ae40 0x7f185406d300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.801+0000 7f1875b06640 1 --2- 172.21.15.40:0/1610493275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1870152560 0x7f1870149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.801+0000 7f1875b06640 1 --2- 172.21.15.40:0/1610493275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18700ab0d0 0x7f1870148170 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.801+0000 7f1875b06640 1 --2- 172.21.15.40:0/1610493275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1870071440 0x7f1870147820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.801+0000 7f1875b06640 1 -- 172.21.15.40:0/1610493275 >> 172.21.15.40:0/1610493275 conn(0x7f1870069720 msgr2=0x7f187015d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.801+0000 7f1875b06640 1 -- 172.21.15.40:0/1610493275 shutdown_connections 2023-10-18T08:55:37.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:37.801+0000 7f1875b06640 1 -- 172.21.15.40:0/1610493275 wait complete. 2023-10-18T08:55:38.224 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T08:55:14.753828+0000', 'modified': '2023-10-18T08:55:17.818729+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4384}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4384': {'gid': 4384, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.144:6835/682820172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6834', 'nonce': 682820172}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 682820172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T08:55:38.225 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T08:55:38.225 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-18T08:55:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.465+0000 7ff76a16e640 1 -- 172.21.15.40:0/3090515590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7640b01e0 msgr2=0x7ff7640d1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.465+0000 7ff76a16e640 1 --2- 172.21.15.40:0/3090515590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7640b01e0 0x7ff7640d1a90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009830 tx=0x7ff75c02bcd0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.465+0000 7ff76a16e640 1 -- 172.21.15.40:0/3090515590 shutdown_connections 2023-10-18T08:55:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.465+0000 7ff76a16e640 1 --2- 172.21.15.40:0/3090515590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7640b01e0 0x7ff7640d1a90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.466+0000 7ff76a16e640 1 -- 172.21.15.40:0/3090515590 >> 172.21.15.40:0/3090515590 conn(0x7ff76408ad00 msgr2=0x7ff76409a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.466+0000 7ff76a16e640 1 -- 172.21.15.40:0/3090515590 shutdown_connections 2023-10-18T08:55:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.466+0000 7ff76a16e640 1 -- 172.21.15.40:0/3090515590 wait complete. 2023-10-18T08:55:38.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.468+0000 7ff76a16e640 1 Processor -- start 2023-10-18T08:55:38.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.468+0000 7ff76a16e640 1 -- start start 2023-10-18T08:55:38.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.469+0000 7ff76a16e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7640b01e0 0x7ff7641e0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.469+0000 7ff76a16e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7640d0020 0x7ff7641589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.470+0000 7ff76a16e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff76415b640 0x7ff7640b5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.470+0000 7ff76a16e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76406bb10 con 0x7ff7640b01e0 2023-10-18T08:55:38.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.470+0000 7ff76a16e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7641e3af0 con 0x7ff76415b640 2023-10-18T08:55:38.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.470+0000 7ff76a16e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7641e3dd0 con 0x7ff7640d0020 2023-10-18T08:55:38.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.470+0000 7ff76916c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7640b01e0 0x7ff7641e0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.470+0000 7ff76916c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7640b01e0 0x7ff7641e0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52118/0 (socket says 172.21.15.40:52118) 2023-10-18T08:55:38.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.470+0000 7ff76916c640 1 -- 172.21.15.40:0/1580264206 learned_addr learned my addr 172.21.15.40:0/1580264206 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:38.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.470+0000 7ff76996d640 1 --2- 172.21.15.40:0/1580264206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff76415b640 0x7ff7640b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.470+0000 7ff76896b640 1 --2- 172.21.15.40:0/1580264206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7640d0020 0x7ff7641589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.470+0000 7ff76916c640 1 -- 172.21.15.40:0/1580264206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7640d0020 msgr2=0x7ff7641589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:38.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.470+0000 7ff76916c640 1 --2- 172.21.15.40:0/1580264206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7640d0020 0x7ff7641589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.470+0000 7ff76916c640 1 -- 172.21.15.40:0/1580264206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff76415b640 msgr2=0x7ff7640b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:38.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.470+0000 7ff76916c640 1 --2- 172.21.15.40:0/1580264206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff76415b640 0x7ff7640b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.470+0000 7ff76916c640 1 -- 172.21.15.40:0/1580264206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c009490 con 0x7ff7640b01e0 2023-10-18T08:55:38.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.471+0000 7ff76916c640 1 --2- 172.21.15.40:0/1580264206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7640b01e0 0x7ff7641e0dc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff75400cbc0 tx=0x7ff75400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:38.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.471+0000 7ff761ffb640 1 -- 172.21.15.40:0/1580264206 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75400ad40 con 0x7ff7640b01e0 2023-10-18T08:55:38.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.471+0000 7ff761ffb640 1 -- 172.21.15.40:0/1580264206 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff754012420 con 0x7ff7640b01e0 2023-10-18T08:55:38.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.471+0000 7ff761ffb640 1 -- 172.21.15.40:0/1580264206 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754012830 con 0x7ff7640b01e0 2023-10-18T08:55:38.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.471+0000 7ff76a16e640 1 -- 172.21.15.40:0/1580264206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7641e40b0 con 0x7ff7640b01e0 2023-10-18T08:55:38.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.471+0000 7ff76a16e640 1 -- 172.21.15.40:0/1580264206 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7641e4670 con 0x7ff7640b01e0 2023-10-18T08:55:38.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.474+0000 7ff761ffb640 1 -- 172.21.15.40:0/1580264206 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff75401a3b0 con 0x7ff7640b01e0 2023-10-18T08:55:38.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.476+0000 7ff761ffb640 1 --2- 172.21.15.40:0/1580264206 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff75006ae40 0x7ff75006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.476+0000 7ff761ffb640 1 -- 172.21.15.40:0/1580264206 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff754089e10 con 0x7ff7640b01e0 2023-10-18T08:55:38.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.476+0000 7ff76df64740 1 Processor -- start 2023-10-18T08:55:38.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.476+0000 7ff76df64740 1 -- start start 2023-10-18T08:55:38.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.476+0000 7ff76896b640 1 --2- 172.21.15.40:0/1580264206 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff75006ae40 0x7ff75006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.477+0000 7ff76df64740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a3240ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.477+0000 7ff76df64740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a326dd50 0x55b7a3270700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.477+0000 7ff76df64740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a32410e0 0x55b7a3270c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.477+0000 7ff76df64740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a31b3740 con 0x55b7a326d970 2023-10-18T08:55:38.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.477+0000 7ff76df64740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a31b0e30 con 0x55b7a32410e0 2023-10-18T08:55:38.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.477+0000 7ff76df64740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7a31ac720 con 0x55b7a326dd50 2023-10-18T08:55:38.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.478+0000 7ff76896b640 1 --2- 172.21.15.40:0/1580264206 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff75006ae40 0x7ff75006d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff758007fd0 tx=0x7ff758009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:38.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.478+0000 7ff76916c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a3240ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.478+0000 7ff76916c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a3240ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52126/0 (socket says 172.21.15.40:52126) 2023-10-18T08:55:38.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.478+0000 7ff76916c640 1 -- 172.21.15.40:0/3306517070 learned_addr learned my addr 172.21.15.40:0/3306517070 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:38.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.478+0000 7ff76996d640 1 --2- 172.21.15.40:0/3306517070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a32410e0 0x55b7a3270c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.478+0000 7ff76896b640 1 --2- 172.21.15.40:0/3306517070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a326dd50 0x55b7a3270700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.478+0000 7ff76916c640 1 -- 172.21.15.40:0/3306517070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a326dd50 msgr2=0x55b7a3270700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:55:38.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.478+0000 7ff76916c640 1 --2- 172.21.15.40:0/3306517070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a326dd50 0x55b7a3270700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.478+0000 7ff76916c640 1 -- 172.21.15.40:0/3306517070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a32410e0 msgr2=0x55b7a3270c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:55:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.478+0000 7ff76916c640 1 --2- 172.21.15.40:0/3306517070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a32410e0 0x55b7a3270c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.478+0000 7ff76916c640 1 -- 172.21.15.40:0/3306517070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7a3170910 con 0x55b7a326d970 2023-10-18T08:55:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.478+0000 7ff76916c640 1 --2- 172.21.15.40:0/3306517070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a3240ba0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff754055070 tx=0x7ff754019310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccd25d50da9f1e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.478+0000 7ff74affd640 1 -- 172.21.15.40:0/3306517070 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754059ea0 con 0x55b7a326d970 2023-10-18T08:55:38.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.479+0000 7ff74affd640 1 -- 172.21.15.40:0/3306517070 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff75405ccc0 con 0x55b7a326d970 2023-10-18T08:55:38.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.479+0000 7ff74affd640 1 -- 172.21.15.40:0/3306517070 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754060e70 con 0x55b7a326d970 2023-10-18T08:55:38.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.479+0000 7ff76df64740 1 -- 172.21.15.40:0/3306517070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 msgr2=0x55b7a3240ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:38.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.479+0000 7ff76df64740 1 --2- 172.21.15.40:0/3306517070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a3240ba0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff754055070 tx=0x7ff754019310 comp rx=0 tx=0).stop 2023-10-18T08:55:38.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.479+0000 7ff76df64740 1 -- 172.21.15.40:0/3306517070 shutdown_connections 2023-10-18T08:55:38.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.479+0000 7ff76df64740 1 --2- 172.21.15.40:0/3306517070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a32410e0 0x55b7a3270c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.479+0000 7ff76df64740 1 --2- 172.21.15.40:0/3306517070 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a326dd50 0x55b7a3270700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.479+0000 7ff76df64740 1 --2- 172.21.15.40:0/3306517070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a3240ba0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.479+0000 7ff76df64740 1 -- 172.21.15.40:0/3306517070 >> 172.21.15.40:0/3306517070 conn(0x55b7a3242e10 msgr2=0x55b7a328b700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:38.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.479+0000 7ff76df64740 1 -- 172.21.15.40:0/3306517070 shutdown_connections 2023-10-18T08:55:38.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.479+0000 7ff76df64740 1 -- 172.21.15.40:0/3306517070 wait complete. 2023-10-18T08:55:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.480+0000 7ff76df64740 1 -- start start 2023-10-18T08:55:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.480+0000 7ff76df64740 1 Processor -- start 2023-10-18T08:55:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.481+0000 7ff76df64740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a32667c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.481+0000 7ff76df64740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a326dd50 0x55b7a3268720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.481+0000 7ff76df64740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a3268c60 0x55b7a3254440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.481+0000 7ff76df64740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a314a970 con 0x55b7a326d970 2023-10-18T08:55:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.481+0000 7ff76df64740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a314a610 con 0x55b7a3268c60 2023-10-18T08:55:38.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.481+0000 7ff76df64740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7a2f10f20 con 0x55b7a326dd50 2023-10-18T08:55:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.481+0000 7ff76df64740 10 client.0 _ll_register_callbacks cb 0x55b7a316e0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T08:55:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.482+0000 7ff76916c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a32667c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.482+0000 7ff76916c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a32667c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52128/0 (socket says 172.21.15.40:52128) 2023-10-18T08:55:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.482+0000 7ff76916c640 1 -- 172.21.15.40:0/3145363502 learned_addr learned my addr 172.21.15.40:0/3145363502 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:38.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.482+0000 7ff76896b640 1 --2- 172.21.15.40:0/3145363502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a326dd50 0x55b7a3268720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.482+0000 7ff76996d640 1 --2- 172.21.15.40:0/3145363502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a3268c60 0x55b7a3254440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.482+0000 7ff76916c640 1 -- 172.21.15.40:0/3145363502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a326dd50 msgr2=0x55b7a3268720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:55:38.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.482+0000 7ff76916c640 1 --2- 172.21.15.40:0/3145363502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a326dd50 0x55b7a3268720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.482+0000 7ff76916c640 1 -- 172.21.15.40:0/3145363502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a3268c60 msgr2=0x55b7a3254440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:55:38.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.482+0000 7ff76916c640 1 --2- 172.21.15.40:0/3145363502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a3268c60 0x55b7a3254440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.482+0000 7ff76916c640 1 -- 172.21.15.40:0/3145363502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7641e4670 con 0x55b7a326d970 2023-10-18T08:55:38.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.482+0000 7ff76996d640 1 --2- 172.21.15.40:0/3145363502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a3268c60 0x55b7a3254440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:38.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.482+0000 7ff76916c640 1 --2- 172.21.15.40:0/3145363502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a32667c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff754055070 tx=0x7ff754060f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d55a605aede9b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:38.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.482+0000 7ff73ffff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T08:55:38.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.483+0000 7ff73ffff640 1 -- 172.21.15.40:0/3145363502 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754059ea0 con 0x55b7a326d970 2023-10-18T08:55:38.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.483+0000 7ff73ffff640 1 -- 172.21.15.40:0/3145363502 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff754060be0 con 0x55b7a326d970 2023-10-18T08:55:38.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.483+0000 7ff76df64740 1 -- 172.21.15.40:0/3145363502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7a31a5700 con 0x55b7a326d970 2023-10-18T08:55:38.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.483+0000 7ff73ffff640 1 -- 172.21.15.40:0/3145363502 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff754070170 con 0x55b7a326d970 2023-10-18T08:55:38.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.483+0000 7ff76df64740 10 client.4731 fetch_fsmap learned FSMap version 7 2023-10-18T08:55:38.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.483+0000 7ff76df64740 1 -- 172.21.15.40:0/3145363502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55b7a31aada0 con 0x55b7a326d970 2023-10-18T08:55:38.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.483+0000 7ff73ffff640 1 -- 172.21.15.40:0/3145363502 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55b7a31a5700 con 0x55b7a326d970 2023-10-18T08:55:38.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.483+0000 7ff73ffff640 1 -- 172.21.15.40:0/3145363502 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 7) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff754022020 con 0x55b7a326d970 2023-10-18T08:55:38.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.483+0000 7ff76df64740 10 client.4731 fetch_fsmap finished waiting for FSMap version 7 2023-10-18T08:55:38.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.483+0000 7ff76df64740 10 client.4731 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-18T08:55:38.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.483+0000 7ff76df64740 20 client.4731 populate_metadata read hostname 'smithi040' 2023-10-18T08:55:38.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.483+0000 7ff76df64740 1 --2- 172.21.15.40:0/3145363502 >> [v2:172.21.15.144:6834/682820172,v1:172.21.15.144:6835/682820172] conn(0x55b7a326f860 0x55b7a326fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.483+0000 7ff76df64740 4 client.4731 mds_command: new command op to 4384 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T08:55:38.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.483+0000 7ff76df64740 1 -- 172.21.15.40:0/3145363502 --> [v2:172.21.15.144:6834/682820172,v1:172.21.15.144:6835/682820172] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7a31a5420 con 0x55b7a326f860 2023-10-18T08:55:38.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.484+0000 7ff76896b640 1 --2- 172.21.15.40:0/3145363502 >> [v2:172.21.15.144:6834/682820172,v1:172.21.15.144:6835/682820172] conn(0x55b7a326f860 0x55b7a326fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.485+0000 7ff76896b640 1 --2- 172.21.15.40:0/3145363502 >> [v2:172.21.15.144:6834/682820172,v1:172.21.15.144:6835/682820172] conn(0x55b7a326f860 0x55b7a326fc40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c2cacb8014e80ed server_cookie=4833e2b2ec027167 in_seq=0 out_seq=0 2023-10-18T08:55:38.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.485+0000 7ff73ffff640 10 client.4731 ms_handle_connect on v2:172.21.15.144:6834/682820172 2023-10-18T08:55:38.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.487+0000 7ff73ffff640 1 -- 172.21.15.40:0/3145363502 <== mds.0 v2:172.21.15.144:6834/682820172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b7a31a5420 con 0x55b7a326f860 2023-10-18T08:55:38.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.488+0000 7ff73ffff640 10 client.4731 handle_command_reply: tid=0 2023-10-18T08:55:38.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.488+0000 7ff76df64740 1 client.4731 shutdown 2023-10-18T08:55:38.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.488+0000 7ff76df64740 1 -- 172.21.15.40:0/3145363502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 msgr2=0x55b7a32667c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:38.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.488+0000 7ff76df64740 1 --2- 172.21.15.40:0/3145363502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a32667c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff754055070 tx=0x7ff754060f60 comp rx=0 tx=0).stop 2023-10-18T08:55:38.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.488+0000 7ff76df64740 1 -- 172.21.15.40:0/3145363502 shutdown_connections 2023-10-18T08:55:38.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.488+0000 7ff76df64740 1 --2- 172.21.15.40:0/3145363502 >> [v2:172.21.15.144:6834/682820172,v1:172.21.15.144:6835/682820172] conn(0x55b7a326f860 0x55b7a326fc40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.488+0000 7ff76df64740 1 --2- 172.21.15.40:0/3145363502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a3268c60 0x55b7a3254440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.488+0000 7ff76df64740 1 --2- 172.21.15.40:0/3145363502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a326dd50 0x55b7a3268720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.488+0000 7ff73ffff640 0 client.4731 ms_handle_reset on v2:172.21.15.144:6834/682820172 2023-10-18T08:55:38.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.488+0000 7ff76df64740 1 --2- 172.21.15.40:0/3145363502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a32667c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.488+0000 7ff76df64740 1 -- 172.21.15.40:0/3145363502 >> 172.21.15.40:0/3145363502 conn(0x55b7a32410e0 msgr2=0x55b7a328ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:38.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.488+0000 7ff76df64740 1 -- 172.21.15.40:0/3145363502 shutdown_connections 2023-10-18T08:55:38.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.488+0000 7ff76df64740 1 -- 172.21.15.40:0/3145363502 wait complete. 2023-10-18T08:55:38.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.488+0000 7ff76df64740 20 client.4731 trim_cache size 0 max 16384 2023-10-18T08:55:38.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.505+0000 7ff76df64740 1 Processor -- start 2023-10-18T08:55:38.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.505+0000 7ff76df64740 1 -- start start 2023-10-18T08:55:38.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.506+0000 7ff76df64740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a3268c60 0x55b7a3244470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.506+0000 7ff76df64740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a32449b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.506+0000 7ff76df64740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a326dd50 0x55b7a329ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.506+0000 7ff76df64740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a326f340 con 0x55b7a326d970 2023-10-18T08:55:38.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.506+0000 7ff76df64740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a326f120 con 0x55b7a326dd50 2023-10-18T08:55:38.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.506+0000 7ff76df64740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7a32404c0 con 0x55b7a3268c60 2023-10-18T08:55:38.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.506+0000 7ff76896b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a32449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.506+0000 7ff76896b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a32449b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52140/0 (socket says 172.21.15.40:52140) 2023-10-18T08:55:38.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.506+0000 7ff76896b640 1 -- 172.21.15.40:0/2228120575 learned_addr learned my addr 172.21.15.40:0/2228120575 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:38.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.507+0000 7ff76916c640 1 --2- 172.21.15.40:0/2228120575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a3268c60 0x55b7a3244470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.507+0000 7ff76896b640 1 -- 172.21.15.40:0/2228120575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a3268c60 msgr2=0x55b7a3244470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:55:38.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.507+0000 7ff76996d640 1 --2- 172.21.15.40:0/2228120575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a326dd50 0x55b7a329ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.507+0000 7ff76896b640 1 --2- 172.21.15.40:0/2228120575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a3268c60 0x55b7a3244470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.507+0000 7ff76896b640 1 -- 172.21.15.40:0/2228120575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a326dd50 msgr2=0x55b7a329ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:55:38.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.507+0000 7ff76896b640 1 --2- 172.21.15.40:0/2228120575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a326dd50 0x55b7a329ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.507+0000 7ff76896b640 1 -- 172.21.15.40:0/2228120575 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7a3149b80 con 0x55b7a326d970 2023-10-18T08:55:38.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.507+0000 7ff76916c640 1 --2- 172.21.15.40:0/2228120575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a3268c60 0x55b7a3244470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:38.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.507+0000 7ff76996d640 1 --2- 172.21.15.40:0/2228120575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a326dd50 0x55b7a329ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:38.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.507+0000 7ff76896b640 1 --2- 172.21.15.40:0/2228120575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a32449b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff7640cfa80 tx=0x7ff758012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe95c045b8b13d83 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:38.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.508+0000 7ff74b7fe640 1 -- 172.21.15.40:0/2228120575 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7580253d0 con 0x55b7a326d970 2023-10-18T08:55:38.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.508+0000 7ff74b7fe640 1 -- 172.21.15.40:0/2228120575 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff758003ac0 con 0x55b7a326d970 2023-10-18T08:55:38.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.508+0000 7ff74b7fe640 1 -- 172.21.15.40:0/2228120575 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758003c90 con 0x55b7a326d970 2023-10-18T08:55:38.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.508+0000 7ff76df64740 1 -- 172.21.15.40:0/2228120575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 msgr2=0x55b7a32449b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:38.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.508+0000 7ff76df64740 1 --2- 172.21.15.40:0/2228120575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a32449b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff7640cfa80 tx=0x7ff758012000 comp rx=0 tx=0).stop 2023-10-18T08:55:38.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.508+0000 7ff76df64740 1 -- 172.21.15.40:0/2228120575 shutdown_connections 2023-10-18T08:55:38.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.508+0000 7ff76df64740 1 --2- 172.21.15.40:0/2228120575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a326dd50 0x55b7a329ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.508+0000 7ff76df64740 1 --2- 172.21.15.40:0/2228120575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326d970 0x55b7a32449b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.508+0000 7ff76df64740 1 --2- 172.21.15.40:0/2228120575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a3268c60 0x55b7a3244470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.508+0000 7ff76df64740 1 -- 172.21.15.40:0/2228120575 >> 172.21.15.40:0/2228120575 conn(0x55b7a32410e0 msgr2=0x55b7a3247930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:38.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.508+0000 7ff76df64740 1 -- 172.21.15.40:0/2228120575 shutdown_connections 2023-10-18T08:55:38.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.508+0000 7ff76df64740 1 -- 172.21.15.40:0/2228120575 wait complete. 2023-10-18T08:55:38.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.509+0000 7ff76df64740 1 -- start start 2023-10-18T08:55:38.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.509+0000 7ff76df64740 1 Processor -- start 2023-10-18T08:55:38.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.510+0000 7ff76df64740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a3268c60 0x55b7a3270150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.510+0000 7ff76df64740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a326d970 0x55b7a32fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.510+0000 7ff76df64740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326dd50 0x55b7a3266e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.510+0000 7ff76df64740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a32af930 con 0x55b7a326dd50 2023-10-18T08:55:38.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.510+0000 7ff76df64740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7a32fc7c0 con 0x55b7a326d970 2023-10-18T08:55:38.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.510+0000 7ff76df64740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7a328db90 con 0x55b7a3268c60 2023-10-18T08:55:38.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.510+0000 7ff76df64740 10 client.0 _ll_register_callbacks cb 0x7ff754054f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T08:55:38.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.511+0000 7ff76996d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326dd50 0x55b7a3266e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.511+0000 7ff76996d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326dd50 0x55b7a3266e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52144/0 (socket says 172.21.15.40:52144) 2023-10-18T08:55:38.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.511+0000 7ff76996d640 1 -- 172.21.15.40:0/3640933404 learned_addr learned my addr 172.21.15.40:0/3640933404 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:38.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.511+0000 7ff76896b640 1 --2- 172.21.15.40:0/3640933404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a326d970 0x55b7a32fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.511+0000 7ff76916c640 1 --2- 172.21.15.40:0/3640933404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a3268c60 0x55b7a3270150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.511+0000 7ff76996d640 1 -- 172.21.15.40:0/3640933404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a3268c60 msgr2=0x55b7a3270150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:55:38.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.511+0000 7ff76996d640 1 --2- 172.21.15.40:0/3640933404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a3268c60 0x55b7a3270150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.511+0000 7ff76996d640 1 -- 172.21.15.40:0/3640933404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a326d970 msgr2=0x55b7a32fbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:55:38.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.511+0000 7ff76996d640 1 --2- 172.21.15.40:0/3640933404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a326d970 0x55b7a32fbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.511+0000 7ff76996d640 1 -- 172.21.15.40:0/3640933404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75801d040 con 0x55b7a326dd50 2023-10-18T08:55:38.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.511+0000 7ff76916c640 1 --2- 172.21.15.40:0/3640933404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a3268c60 0x55b7a3270150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:38.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.511+0000 7ff76996d640 1 --2- 172.21.15.40:0/3640933404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326dd50 0x55b7a3266e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009920 tx=0x7ff75c004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc2281ccaa7e58a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:38.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.511+0000 7ff73ffff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T08:55:38.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.512+0000 7ff73ffff640 1 -- 172.21.15.40:0/3640933404 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c003a00 con 0x55b7a326dd50 2023-10-18T08:55:38.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.512+0000 7ff73ffff640 1 -- 172.21.15.40:0/3640933404 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff75c003b60 con 0x55b7a326dd50 2023-10-18T08:55:38.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.512+0000 7ff76df64740 1 -- 172.21.15.40:0/3640933404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff75405ccc0 con 0x55b7a326dd50 2023-10-18T08:55:38.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.512+0000 7ff73ffff640 1 -- 172.21.15.40:0/3640933404 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff75c003d20 con 0x55b7a326dd50 2023-10-18T08:55:38.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.512+0000 7ff76df64740 10 client.4737 fetch_fsmap learned FSMap version 7 2023-10-18T08:55:38.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.512+0000 7ff76df64740 1 -- 172.21.15.40:0/3640933404 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55b7a316f170 con 0x55b7a326dd50 2023-10-18T08:55:38.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.512+0000 7ff73ffff640 1 -- 172.21.15.40:0/3640933404 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff75405ccc0 con 0x55b7a326dd50 2023-10-18T08:55:38.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.512+0000 7ff73ffff640 1 -- 172.21.15.40:0/3640933404 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 7) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff75c045020 con 0x55b7a326dd50 2023-10-18T08:55:38.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.512+0000 7ff76df64740 10 client.4737 fetch_fsmap finished waiting for FSMap version 7 2023-10-18T08:55:38.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.512+0000 7ff76df64740 10 client.4737 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-18T08:55:38.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.512+0000 7ff76df64740 20 client.4737 populate_metadata read hostname 'smithi040' 2023-10-18T08:55:38.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.512+0000 7ff76df64740 1 --2- 172.21.15.40:0/3640933404 >> [v2:172.21.15.144:6834/682820172,v1:172.21.15.144:6835/682820172] conn(0x55b7a3267390 0x55b7a3267770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:38.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.512+0000 7ff76df64740 4 client.4737 mds_command: new command op to 4384 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T08:55:38.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.512+0000 7ff76df64740 1 -- 172.21.15.40:0/3640933404 --> [v2:172.21.15.144:6834/682820172,v1:172.21.15.144:6835/682820172] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b7a2eae6c0 con 0x55b7a3267390 2023-10-18T08:55:38.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.513+0000 7ff76916c640 1 --2- 172.21.15.40:0/3640933404 >> [v2:172.21.15.144:6834/682820172,v1:172.21.15.144:6835/682820172] conn(0x55b7a3267390 0x55b7a3267770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:38.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.514+0000 7ff76916c640 1 --2- 172.21.15.40:0/3640933404 >> [v2:172.21.15.144:6834/682820172,v1:172.21.15.144:6835/682820172] conn(0x55b7a3267390 0x55b7a3267770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b68c21f379e43353 server_cookie=763e4d200f7b6cc8 in_seq=0 out_seq=0 2023-10-18T08:55:38.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.514+0000 7ff73ffff640 10 client.4737 ms_handle_connect on v2:172.21.15.144:6834/682820172 2023-10-18T08:55:38.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.514+0000 7ff73ffff640 1 -- 172.21.15.40:0/3640933404 <== mds.0 v2:172.21.15.144:6834/682820172 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55b7a2eae6c0 con 0x55b7a3267390 2023-10-18T08:55:38.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.514+0000 7ff73ffff640 10 client.4737 handle_command_reply: tid=0 2023-10-18T08:55:38.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.514+0000 7ff76df64740 1 client.4737 shutdown 2023-10-18T08:55:38.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.515+0000 7ff76df64740 1 -- 172.21.15.40:0/3640933404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326dd50 msgr2=0x55b7a3266e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:38.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.515+0000 7ff76df64740 1 --2- 172.21.15.40:0/3640933404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326dd50 0x55b7a3266e50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009920 tx=0x7ff75c004460 comp rx=0 tx=0).stop 2023-10-18T08:55:38.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.515+0000 7ff76df64740 1 -- 172.21.15.40:0/3640933404 shutdown_connections 2023-10-18T08:55:38.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.515+0000 7ff76df64740 1 --2- 172.21.15.40:0/3640933404 >> [v2:172.21.15.144:6834/682820172,v1:172.21.15.144:6835/682820172] conn(0x55b7a3267390 0x55b7a3267770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.515+0000 7ff76df64740 1 --2- 172.21.15.40:0/3640933404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b7a326dd50 0x55b7a3266e50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.515+0000 7ff76df64740 1 --2- 172.21.15.40:0/3640933404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b7a326d970 0x55b7a32fbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.515+0000 7ff76df64740 1 --2- 172.21.15.40:0/3640933404 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b7a3268c60 0x55b7a3270150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.515+0000 7ff73ffff640 0 client.4737 ms_handle_reset on v2:172.21.15.144:6834/682820172 2023-10-18T08:55:38.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.515+0000 7ff76df64740 1 -- 172.21.15.40:0/3640933404 >> 172.21.15.40:0/3640933404 conn(0x55b7a32410e0 msgr2=0x55b7a3247930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:38.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.515+0000 7ff76df64740 1 -- 172.21.15.40:0/3640933404 shutdown_connections 2023-10-18T08:55:38.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.515+0000 7ff76df64740 1 -- 172.21.15.40:0/3640933404 wait complete. 2023-10-18T08:55:38.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.515+0000 7ff76df64740 20 client.4737 trim_cache size 0 max 16384 2023-10-18T08:55:38.532 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T08:55:38.532 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T08:55:38.532 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-10-18T08:55:38.533 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4384, 2023-10-18T08:55:38.533 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-10-18T08:55:38.533 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-10-18T08:55:38.533 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-10-18T08:55:38.533 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 23.211339464999998, 2023-10-18T08:55:38.534 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 7, 2023-10-18T08:55:38.534 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 20, 2023-10-18T08:55:38.534 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 20, 2023-10-18T08:55:38.534 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 28.402736282999999 2023-10-18T08:55:38.535 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T08:55:38.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.516+0000 7ff76a16e640 1 -- 172.21.15.40:0/1580264206 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff75006ae40 msgr2=0x7ff75006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:38.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.516+0000 7ff76a16e640 1 --2- 172.21.15.40:0/1580264206 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff75006ae40 0x7ff75006d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff758007fd0 tx=0x7ff758009040 comp rx=0 tx=0).stop 2023-10-18T08:55:38.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.516+0000 7ff76a16e640 1 -- 172.21.15.40:0/1580264206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7640b01e0 msgr2=0x7ff7641e0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:38.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.516+0000 7ff76a16e640 1 --2- 172.21.15.40:0/1580264206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7640b01e0 0x7ff7641e0dc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff75400cbc0 tx=0x7ff75400a430 comp rx=0 tx=0).stop 2023-10-18T08:55:38.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.516+0000 7ff76a16e640 1 -- 172.21.15.40:0/1580264206 shutdown_connections 2023-10-18T08:55:38.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.516+0000 7ff76a16e640 1 --2- 172.21.15.40:0/1580264206 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff75006ae40 0x7ff75006d300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.516+0000 7ff76a16e640 1 --2- 172.21.15.40:0/1580264206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff76415b640 0x7ff7640b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.516+0000 7ff76a16e640 1 --2- 172.21.15.40:0/1580264206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7640d0020 0x7ff7641589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.516+0000 7ff76a16e640 1 --2- 172.21.15.40:0/1580264206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7640b01e0 0x7ff7641e0dc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:38.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.516+0000 7ff76a16e640 1 -- 172.21.15.40:0/1580264206 >> 172.21.15.40:0/1580264206 conn(0x7ff76408ad00 msgr2=0x7ff76415a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:38.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.516+0000 7ff76a16e640 1 -- 172.21.15.40:0/1580264206 shutdown_connections 2023-10-18T08:55:38.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:38.516+0000 7ff76a16e640 1 -- 172.21.15.40:0/1580264206 wait complete. 2023-10-18T08:55:38.539 INFO:teuthology.run_tasks:Running task kclient... 2023-10-18T08:55:38.563 INFO:tasks.kclient:Mounting kernel clients... 2023-10-18T08:55:38.563 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-18T08:55:38.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:38.565 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-18T08:55:38.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:38.626 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2023-10-18T08:55:38.734 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2023-10-18T08:55:38.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T08:55:38.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:38.738 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-18T08:55:38.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:38.792 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2023-10-18T08:55:38.893 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2023-10-18T08:55:38.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T08:55:38.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:38.898 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-18T08:55:38.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:38.959 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2023-10-18T08:55:39.063 INFO:teuthology.orchestra.run.smithi144.stderr:Cannot find device "ceph-brx" 2023-10-18T08:55:39.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T08:55:39.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:39.069 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-18T08:55:39.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:39.122 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2023-10-18T08:55:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:Cannot find device "ceph-brx" 2023-10-18T08:55:39.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T08:55:39.225 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-10-18T08:55:39.226 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T08:55:39.226 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T08:55:39.226 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T08:55:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.462+0000 7f7aba7c4640 1 -- 172.21.15.40:0/3796127890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ab406b4d0 msgr2=0x7f7ab406b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:39.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.462+0000 7f7aba7c4640 1 --2- 172.21.15.40:0/3796127890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ab406b4d0 0x7f7ab406b950 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40665e0 tx=0x7f7aa402bda0 comp rx=0 tx=0).stop 2023-10-18T08:55:39.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.462+0000 7f7aba7c4640 1 -- 172.21.15.40:0/3796127890 shutdown_connections 2023-10-18T08:55:39.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.462+0000 7f7aba7c4640 1 --2- 172.21.15.40:0/3796127890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ab406b4d0 0x7f7ab406b950 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:39.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.462+0000 7f7aba7c4640 1 --2- 172.21.15.40:0/3796127890 >> v2:172.21.15.144:3300/0 conn(0x7f7ab406d6f0 0x7f7ab4071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.462+0000 7f7aba7c4640 1 -- 172.21.15.40:0/3796127890 >> 172.21.15.40:0/3796127890 conn(0x7f7ab40aed70 msgr2=0x7f7ab40af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.462+0000 7f7aba7c4640 1 -- 172.21.15.40:0/3796127890 shutdown_connections 2023-10-18T08:55:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.463+0000 7f7aba7c4640 1 -- 172.21.15.40:0/3796127890 wait complete. 2023-10-18T08:55:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.464+0000 7f7aba7c4640 1 Processor -- start 2023-10-18T08:55:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.464+0000 7f7aba7c4640 1 -- start start 2023-10-18T08:55:39.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.465+0000 7f7aba7c4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ab406b4d0 0x7f7ab40cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:39.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.465+0000 7f7aba7c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ab406d6f0 0x7f7ab413d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:39.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.465+0000 7f7aba7c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ab4157cd0 0x7f7ab4137b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:39.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.465+0000 7f7aba7c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab40cb920 con 0x7f7ab4157cd0 2023-10-18T08:55:39.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.465+0000 7f7aba7c4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab40cbc00 con 0x7f7ab406d6f0 2023-10-18T08:55:39.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.465+0000 7f7aba7c4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab40cbee0 con 0x7f7ab406b4d0 2023-10-18T08:55:39.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.466+0000 7f7ab9fc3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ab4157cd0 0x7f7ab4137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:39.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.466+0000 7f7ab9fc3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ab4157cd0 0x7f7ab4137b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52156/0 (socket says 172.21.15.40:52156) 2023-10-18T08:55:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.466+0000 7f7ab9fc3640 1 -- 172.21.15.40:0/847354883 learned_addr learned my addr 172.21.15.40:0/847354883 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.466+0000 7f7ab8fc1640 1 --2- 172.21.15.40:0/847354883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ab406d6f0 0x7f7ab413d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.466+0000 7f7ab97c2640 1 --2- 172.21.15.40:0/847354883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ab406b4d0 0x7f7ab40cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.466+0000 7f7ab9fc3640 1 -- 172.21.15.40:0/847354883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ab406b4d0 msgr2=0x7f7ab40cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.466+0000 7f7ab9fc3640 1 --2- 172.21.15.40:0/847354883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ab406b4d0 0x7f7ab40cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:39.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.466+0000 7f7ab9fc3640 1 -- 172.21.15.40:0/847354883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ab406d6f0 msgr2=0x7f7ab413d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:39.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.466+0000 7f7ab9fc3640 1 --2- 172.21.15.40:0/847354883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ab406d6f0 0x7f7ab413d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:39.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.466+0000 7f7ab9fc3640 1 -- 172.21.15.40:0/847354883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa4009560 con 0x7f7ab4157cd0 2023-10-18T08:55:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.466+0000 7f7ab9fc3640 1 --2- 172.21.15.40:0/847354883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ab4157cd0 0x7f7ab4137b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00c780 tx=0x7f7aac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.467+0000 7f7aa9ffb640 1 -- 172.21.15.40:0/847354883 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac004b60 con 0x7f7ab4157cd0 2023-10-18T08:55:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.467+0000 7f7aa9ffb640 1 -- 172.21.15.40:0/847354883 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7aac00ce90 con 0x7f7ab4157cd0 2023-10-18T08:55:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.467+0000 7f7aa9ffb640 1 -- 172.21.15.40:0/847354883 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac00aab0 con 0x7f7ab4157cd0 2023-10-18T08:55:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.467+0000 7f7aba7c4640 1 -- 172.21.15.40:0/847354883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab40cc1c0 con 0x7f7ab4157cd0 2023-10-18T08:55:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.467+0000 7f7aba7c4640 1 -- 172.21.15.40:0/847354883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab4138ae0 con 0x7f7ab4157cd0 2023-10-18T08:55:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.468+0000 7f7aba7c4640 1 -- 172.21.15.40:0/847354883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab40cc6e0 con 0x7f7ab4157cd0 2023-10-18T08:55:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.469+0000 7f7aa9ffb640 1 -- 172.21.15.40:0/847354883 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7aac00ac10 con 0x7f7ab4157cd0 2023-10-18T08:55:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.470+0000 7f7aa9ffb640 1 --2- 172.21.15.40:0/847354883 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7a9806ae40 0x7f7a9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.470+0000 7f7aa9ffb640 1 -- 172.21.15.40:0/847354883 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7aac089b10 con 0x7f7ab4157cd0 2023-10-18T08:55:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.471+0000 7f7ab97c2640 1 --2- 172.21.15.40:0/847354883 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7a9806ae40 0x7f7a9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:39.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.472+0000 7f7ab97c2640 1 --2- 172.21.15.40:0/847354883 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7a9806ae40 0x7f7a9806d300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009890 tx=0x7f7ab0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:39.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.474+0000 7f7aa9ffb640 1 -- 172.21.15.40:0/847354883 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ab40cc6e0 con 0x7f7ab4157cd0 2023-10-18T08:55:39.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.710+0000 7f7aba7c4640 1 -- 172.21.15.40:0/847354883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7ab40cc6e0 con 0x7f7ab4157cd0 2023-10-18T08:55:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.711+0000 7f7aa9ffb640 1 -- 172.21.15.40:0/847354883 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7ab40cc6e0 con 0x7f7ab4157cd0 2023-10-18T08:55:39.714 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T08:55:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.713+0000 7f7aba7c4640 1 -- 172.21.15.40:0/847354883 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7a9806ae40 msgr2=0x7f7a9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:39.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.713+0000 7f7aba7c4640 1 --2- 172.21.15.40:0/847354883 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7a9806ae40 0x7f7a9806d300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009890 tx=0x7f7ab0008040 comp rx=0 tx=0).stop 2023-10-18T08:55:39.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.713+0000 7f7aba7c4640 1 -- 172.21.15.40:0/847354883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ab4157cd0 msgr2=0x7f7ab4137b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:39.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.713+0000 7f7aba7c4640 1 --2- 172.21.15.40:0/847354883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ab4157cd0 0x7f7ab4137b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00c780 tx=0x7f7aac00cc40 comp rx=0 tx=0).stop 2023-10-18T08:55:39.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.713+0000 7f7aba7c4640 1 -- 172.21.15.40:0/847354883 shutdown_connections 2023-10-18T08:55:39.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.713+0000 7f7aba7c4640 1 --2- 172.21.15.40:0/847354883 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7a9806ae40 0x7f7a9806d300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:39.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.713+0000 7f7aba7c4640 1 --2- 172.21.15.40:0/847354883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ab4157cd0 0x7f7ab4137b00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:39.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.714+0000 7f7aba7c4640 1 --2- 172.21.15.40:0/847354883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ab406d6f0 0x7f7ab413d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:39.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.714+0000 7f7aba7c4640 1 --2- 172.21.15.40:0/847354883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ab406b4d0 0x7f7ab40cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:39.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.714+0000 7f7aba7c4640 1 -- 172.21.15.40:0/847354883 >> 172.21.15.40:0/847354883 conn(0x7f7ab40aed70 msgr2=0x7f7ab40aba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:39.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.714+0000 7f7aba7c4640 1 -- 172.21.15.40:0/847354883 shutdown_connections 2023-10-18T08:55:39.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:39.714+0000 7f7aba7c4640 1 -- 172.21.15.40:0/847354883 wait complete. 2023-10-18T08:55:39.736 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T08:55:39.736 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T08:55:39.737 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-10-18T08:55:39.737 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T08:55:39.737 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T08:55:39.737 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T08:55:39.738 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T08:55:39.738 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T08:55:39.738 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T08:55:39.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:39.739 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-10-18T08:55:39.795 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T08:55:39.795 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T08:55:39.795 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T08:55:39.795 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:39.796 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-18T08:55:39.796 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:39.796 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:55:39.796 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:39.796 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-10-18T08:55:39.797 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:55:39.797 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:39.797 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-10-18T08:55:39.797 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T08:55:39.798 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:39.798 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-10-18T08:55:39.798 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-18T08:55:39.798 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42385sec preferred_lft 42385sec 2023-10-18T08:55:39.798 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2023-10-18T08:55:39.799 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:39.799 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:55:39.799 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:39.799 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-10-18T08:55:39.800 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-18T08:55:39.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:39.801 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T08:55:39.801 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T08:55:39.801 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add name ceph-brx type bridge 2023-10-18T08:55:39.802 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr flush dev ceph-brx 2023-10-18T08:55:39.802 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set ceph-brx up 2023-10-18T08:55:39.802 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-18T08:55:39.803 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T08:55:40.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:40.301 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T08:55:40.382 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-10-18T08:55:40.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:40.386 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-10-18T08:55:40.435 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-10-18T08:55:40.435 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T08:55:40.436 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-18T08:55:40.436 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-18T08:55:40.436 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T08:55:40.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:40.437 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T08:55:40.437 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T08:55:40.438 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-18T08:55:40.438 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-18T08:55:40.438 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-18T08:55:40.439 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T08:55:41.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:41.203 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-18T08:55:41.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:41.264 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2023-10-18T08:55:41.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:41.323 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T08:55:41.323 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T08:55:41.323 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T08:55:41.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-18T08:55:41.324 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T08:55:41.483 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-18T08:55:41.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:41.486 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T08:55:41.486 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T08:55:41.486 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-18T08:55:41.487 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-18T08:55:41.487 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-18T08:55:41.487 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-18T08:55:41.487 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-18T08:55:41.488 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T08:55:41.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:41.784 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T08:55:41.784 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T08:55:41.785 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-10-18T08:55:41.786 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.0 master ceph-brx 2023-10-18T08:55:41.786 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T08:55:41.965 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:55:41.965 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T08:55:42.013 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T08:55:42.013 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:55:42.014 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T08:55:42.114 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-18T08:55:42.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:42.114 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:55:42.269 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:55:42.270 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:55:42.749 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:55:42.749 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:55:42.749 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T08:55:42.749 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T08:55:42.750 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T08:55:42.754 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-18T08:55:42.754 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2023-10-18T08:55:42.755 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T08:55:42.755 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T08:55:42.755 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T08:55:42.759 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T08:55:42.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:42.759 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T08:55:42.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:42.867 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:55:42.950 INFO:teuthology.orchestra.run.smithi040.stdout:client4735 2023-10-18T08:55:42.950 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:55:42.951 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:55:43.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:43.012 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:55:43.095 INFO:teuthology.orchestra.run.smithi040.stdout:client4735 2023-10-18T08:55:43.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:43.096 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:55:43.181 INFO:teuthology.orchestra.run.smithi040.stdout:client4735 2023-10-18T08:55:43.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:43.182 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/029855e3-538a-4478-b323-f54ab0f3a56d.client4735/status') 2023-10-18T08:55:43.304 INFO:teuthology.orchestra.run.smithi040.stdout:instance: client.4735 (3)172.21.15.40:0/2325493630 2023-10-18T08:55:43.304 INFO:teuthology.orchestra.run.smithi040.stdout:blocklisted: false 2023-10-18T08:55:43.305 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2023-10-18T08:55:43.305 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2023-10-18T08:55:43.305 INFO:teuthology.orchestra.run.smithi040.stderr:70 bytes copied, 7.9025e-05 s, 886 kB/s 2023-10-18T08:55:43.309 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-10-18T08:55:43.309 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T08:55:43.309 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-18T08:55:43.310 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T08:55:43.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.537+0000 7fbbd9f05640 1 -- 172.21.15.40:0/3759450363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbd40703e0 msgr2=0x7fbbd4157c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:43.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.537+0000 7fbbd9f05640 1 --2- 172.21.15.40:0/3759450363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbd40703e0 0x7fbbd4157c00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbbd406ab40 tx=0x7fbbcc02bd70 comp rx=0 tx=0).stop 2023-10-18T08:55:43.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.537+0000 7fbbd9f05640 1 -- 172.21.15.40:0/3759450363 shutdown_connections 2023-10-18T08:55:43.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.537+0000 7fbbd9f05640 1 --2- 172.21.15.40:0/3759450363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbd40703e0 0x7fbbd4157c00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:43.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.537+0000 7fbbd9f05640 1 --2- 172.21.15.40:0/3759450363 >> v2:172.21.15.144:3300/0 conn(0x7fbbd4151cb0 0x7fbbd406fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:43.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.537+0000 7fbbd9f05640 1 -- 172.21.15.40:0/3759450363 >> 172.21.15.40:0/3759450363 conn(0x7fbbd4069720 msgr2=0x7fbbd4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:43.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.538+0000 7fbbd9f05640 1 -- 172.21.15.40:0/3759450363 shutdown_connections 2023-10-18T08:55:43.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.538+0000 7fbbd9f05640 1 -- 172.21.15.40:0/3759450363 wait complete. 2023-10-18T08:55:43.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.539+0000 7fbbd9f05640 1 Processor -- start 2023-10-18T08:55:43.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.540+0000 7fbbd9f05640 1 -- start start 2023-10-18T08:55:43.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.540+0000 7fbbd9f05640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbd406d710 0x7fbbd414a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:43.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbd9f05640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbd40703e0 0x7fbbd414a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:43.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbd9f05640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbd4151cb0 0x7fbbd414ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbd9f05640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd41534f0 con 0x7fbbd4151cb0 2023-10-18T08:55:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbd9f05640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd414a5a0 con 0x7fbbd406d710 2023-10-18T08:55:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbd9f05640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd4154f90 con 0x7fbbd40703e0 2023-10-18T08:55:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbd3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbd4151cb0 0x7fbbd414ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:43.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbd3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbd4151cb0 0x7fbbd414ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52172/0 (socket says 172.21.15.40:52172) 2023-10-18T08:55:43.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbd3fff640 1 -- 172.21.15.40:0/1693136956 learned_addr learned my addr 172.21.15.40:0/1693136956 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:43.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbcaffd640 1 --2- 172.21.15.40:0/1693136956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbd40703e0 0x7fbbd414a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:43.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbd37fe640 1 --2- 172.21.15.40:0/1693136956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbd406d710 0x7fbbd414a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:43.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbd3fff640 1 -- 172.21.15.40:0/1693136956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbd40703e0 msgr2=0x7fbbd414a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:43.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbd3fff640 1 --2- 172.21.15.40:0/1693136956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbd40703e0 0x7fbbd414a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:43.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbd3fff640 1 -- 172.21.15.40:0/1693136956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbd406d710 msgr2=0x7fbbd414a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:43.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbd3fff640 1 --2- 172.21.15.40:0/1693136956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbd406d710 0x7fbbd414a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:43.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbd3fff640 1 -- 172.21.15.40:0/1693136956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc009490 con 0x7fbbd4151cb0 2023-10-18T08:55:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbcaffd640 1 --2- 172.21.15.40:0/1693136956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbd40703e0 0x7fbbd414a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.541+0000 7fbbd37fe640 1 --2- 172.21.15.40:0/1693136956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbd406d710 0x7fbbd414a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.542+0000 7fbbd3fff640 1 --2- 172.21.15.40:0/1693136956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbd4151cb0 0x7fbbd414ad10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009920 tx=0x7fbbcc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.542+0000 7fbbd0ff9640 1 -- 172.21.15.40:0/1693136956 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc003e20 con 0x7fbbd4151cb0 2023-10-18T08:55:43.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.542+0000 7fbbd0ff9640 1 -- 172.21.15.40:0/1693136956 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbbcc034440 con 0x7fbbd4151cb0 2023-10-18T08:55:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.542+0000 7fbbd9f05640 1 -- 172.21.15.40:0/1693136956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd4152e30 con 0x7fbbd4151cb0 2023-10-18T08:55:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.542+0000 7fbbd0ff9640 1 -- 172.21.15.40:0/1693136956 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc034830 con 0x7fbbd4151cb0 2023-10-18T08:55:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.542+0000 7fbbd9f05640 1 -- 172.21.15.40:0/1693136956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd4152b40 con 0x7fbbd4151cb0 2023-10-18T08:55:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.543+0000 7fbbd9f05640 1 -- 172.21.15.40:0/1693136956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd4066fd0 con 0x7fbbd4151cb0 2023-10-18T08:55:43.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.544+0000 7fbbd0ff9640 1 -- 172.21.15.40:0/1693136956 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbbcc03b040 con 0x7fbbd4151cb0 2023-10-18T08:55:43.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.545+0000 7fbbd0ff9640 1 --2- 172.21.15.40:0/1693136956 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbbb406ae40 0x7fbbb406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:43.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.545+0000 7fbbd0ff9640 1 -- 172.21.15.40:0/1693136956 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbcc0abbb0 con 0x7fbbd4151cb0 2023-10-18T08:55:43.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.546+0000 7fbbd37fe640 1 --2- 172.21.15.40:0/1693136956 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbbb406ae40 0x7fbbb406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:43.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.547+0000 7fbbd37fe640 1 --2- 172.21.15.40:0/1693136956 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbbb406ae40 0x7fbbb406d300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40016f0 tx=0x7fbbc4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:43.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.550+0000 7fbbd0ff9640 1 -- 172.21.15.40:0/1693136956 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbbcc076bc0 con 0x7fbbd4151cb0 2023-10-18T08:55:43.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.781+0000 7fbbd9f05640 1 -- 172.21.15.40:0/1693136956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbbd40b0590 con 0x7fbbd4151cb0 2023-10-18T08:55:43.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.782+0000 7fbbd0ff9640 1 -- 172.21.15.40:0/1693136956 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbbcc039070 con 0x7fbbd4151cb0 2023-10-18T08:55:43.798 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T08:55:43.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.786+0000 7fbbd9f05640 1 -- 172.21.15.40:0/1693136956 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbbb406ae40 msgr2=0x7fbbb406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:43.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.786+0000 7fbbd9f05640 1 --2- 172.21.15.40:0/1693136956 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbbb406ae40 0x7fbbb406d300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40016f0 tx=0x7fbbc4009040 comp rx=0 tx=0).stop 2023-10-18T08:55:43.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.786+0000 7fbbd9f05640 1 -- 172.21.15.40:0/1693136956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbd4151cb0 msgr2=0x7fbbd414ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:43.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.786+0000 7fbbd9f05640 1 --2- 172.21.15.40:0/1693136956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbd4151cb0 0x7fbbd414ad10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009920 tx=0x7fbbcc003820 comp rx=0 tx=0).stop 2023-10-18T08:55:43.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.786+0000 7fbbd9f05640 1 -- 172.21.15.40:0/1693136956 shutdown_connections 2023-10-18T08:55:43.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.786+0000 7fbbd9f05640 1 --2- 172.21.15.40:0/1693136956 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbbb406ae40 0x7fbbb406d300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:43.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.786+0000 7fbbd9f05640 1 --2- 172.21.15.40:0/1693136956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbd4151cb0 0x7fbbd414ad10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:43.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.786+0000 7fbbd9f05640 1 --2- 172.21.15.40:0/1693136956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbd40703e0 0x7fbbd414a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:43.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.786+0000 7fbbd9f05640 1 --2- 172.21.15.40:0/1693136956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbd406d710 0x7fbbd414a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:43.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.786+0000 7fbbd9f05640 1 -- 172.21.15.40:0/1693136956 >> 172.21.15.40:0/1693136956 conn(0x7fbbd4069720 msgr2=0x7fbbd415c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:43.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.786+0000 7fbbd9f05640 1 -- 172.21.15.40:0/1693136956 shutdown_connections 2023-10-18T08:55:43.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:43.786+0000 7fbbd9f05640 1 -- 172.21.15.40:0/1693136956 wait complete. 2023-10-18T08:55:43.808 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T08:55:43.808 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T08:55:43.809 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-10-18T08:55:43.809 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-18T08:55:43.809 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-18T08:55:43.809 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T08:55:43.810 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T08:55:43.810 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T08:55:43.810 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-18T08:55:43.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:43.811 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-10-18T08:55:43.868 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T08:55:43.868 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T08:55:43.868 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T08:55:43.869 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:43.869 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-18T08:55:43.870 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:43.870 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:55:43.870 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:43.871 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-10-18T08:55:43.871 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:55:43.871 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:43.872 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-10-18T08:55:43.872 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T08:55:43.873 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:43.873 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-10-18T08:55:43.873 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-18T08:55:43.874 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42380sec preferred_lft 42380sec 2023-10-18T08:55:43.874 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2023-10-18T08:55:43.875 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:43.875 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:55:43.875 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:43.876 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-10-18T08:55:43.876 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T08:55:43.876 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:43.877 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T08:55:43.877 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:43.878 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2026:95ff:fea9:a463/64 scope link 2023-10-18T08:55:43.878 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:43.879 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T08:55:43.879 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T08:55:43.880 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::80bd:e5ff:fe0f:6f68/64 scope link 2023-10-18T08:55:43.880 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:43.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:43.882 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T08:55:43.980 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-10-18T08:55:43.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:43.985 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-10-18T08:55:44.040 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-10-18T08:55:44.040 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T08:55:44.041 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-18T08:55:44.041 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-18T08:55:44.041 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T08:55:44.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:44.043 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T08:55:44.044 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T08:55:44.044 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-18T08:55:44.044 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-18T08:55:44.045 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-18T08:55:44.045 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T08:55:44.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:44.250 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-18T08:55:44.304 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T08:55:44.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:44.306 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2023-10-18T08:55:44.361 INFO:teuthology.orchestra.run.smithi040.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-18T08:55:44.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:44.363 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T08:55:44.363 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T08:55:44.363 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T08:55:44.364 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-18T08:55:44.364 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T08:55:44.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:44.518 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-18T08:55:44.614 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T08:55:44.614 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T08:55:44.614 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T08:55:44.614 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:44.615 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-18T08:55:44.615 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:44.615 INFO:teuthology.orchestra.run.smithi040.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T08:55:44.615 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether e2:14:da:34:42:25 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-18T08:55:44.615 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-18T08:55:44.616 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:44.616 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::e014:daff:fe34:4225/64 scope link 2023-10-18T08:55:44.616 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:44.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:44.618 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-18T08:55:44.718 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T08:55:44.719 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T08:55:44.719 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T08:55:44.719 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:44.720 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-18T08:55:44.720 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:44.720 INFO:teuthology.orchestra.run.smithi040.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T08:55:44.720 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether e2:14:da:34:42:25 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-18T08:55:44.720 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-18T08:55:44.721 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:44.721 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::e014:daff:fe34:4225/64 scope link 2023-10-18T08:55:44.721 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:44.722 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-18T08:55:44.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:44.723 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T08:55:44.724 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T08:55:44.724 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-18T08:55:44.724 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-18T08:55:44.725 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-18T08:55:44.725 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-18T08:55:44.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-18T08:55:44.726 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T08:55:45.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:45.025 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T08:55:45.025 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T08:55:45.025 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.1 up 2023-10-18T08:55:45.026 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.1 master ceph-brx 2023-10-18T08:55:45.026 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T08:55:45.209 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:55:45.209 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-18T08:55:45.264 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-18T08:55:45.264 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:55:45.265 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-18T08:55:45.318 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-10-18T08:55:45.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:45.319 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:55:45.470 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:55:45.470 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:55:45.506 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:55:45.506 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:55:45.506 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T08:55:45.507 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 2@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T08:55:45.507 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T08:55:45.508 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2023-10-18T08:55:45.508 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2023-10-18T08:55:45.508 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T08:55:45.508 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T08:55:45.509 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T08:55:45.512 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T08:55:45.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:45.513 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-18T08:55:45.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:45.605 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:55:45.666 INFO:teuthology.orchestra.run.smithi040.stdout:client4741 2023-10-18T08:55:45.667 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:55:45.667 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:55:45.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:45.720 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:55:45.779 INFO:teuthology.orchestra.run.smithi040.stdout:client4741 2023-10-18T08:55:45.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:45.780 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:55:45.839 INFO:teuthology.orchestra.run.smithi040.stdout:client4741 2023-10-18T08:55:45.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:45.840 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/029855e3-538a-4478-b323-f54ab0f3a56d.client4741/status') 2023-10-18T08:55:45.936 INFO:teuthology.orchestra.run.smithi040.stdout:instance: client.4741 (3)172.21.15.40:0/2759378953 2023-10-18T08:55:45.937 INFO:teuthology.orchestra.run.smithi040.stdout:blocklisted: false 2023-10-18T08:55:45.937 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records in 2023-10-18T08:55:45.937 INFO:teuthology.orchestra.run.smithi040.stderr:0+1 records out 2023-10-18T08:55:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:70 bytes copied, 6.9618e-05 s, 1.0 MB/s 2023-10-18T08:55:45.941 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-10-18T08:55:45.942 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T08:55:45.942 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T08:55:45.943 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T08:55:46.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.170+0000 7fcf1ae58640 1 -- 172.21.15.40:0/813700032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf14151cb0 msgr2=0x7fcf14158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:46.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.170+0000 7fcf1ae58640 1 --2- 172.21.15.40:0/813700032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf14151cb0 0x7fcf14158ca0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800b3f0 tx=0x7fcf0802c610 comp rx=0 tx=0).stop 2023-10-18T08:55:46.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.171+0000 7fcf1ae58640 1 -- 172.21.15.40:0/813700032 shutdown_connections 2023-10-18T08:55:46.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.171+0000 7fcf1ae58640 1 --2- 172.21.15.40:0/813700032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf14158310 0x7fcf14070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:46.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.171+0000 7fcf1ae58640 1 --2- 172.21.15.40:0/813700032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf14151cb0 0x7fcf14158ca0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:46.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.171+0000 7fcf1ae58640 1 -- 172.21.15.40:0/813700032 >> 172.21.15.40:0/813700032 conn(0x7fcf14069720 msgr2=0x7fcf14069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:46.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.171+0000 7fcf1ae58640 1 -- 172.21.15.40:0/813700032 shutdown_connections 2023-10-18T08:55:46.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.171+0000 7fcf1ae58640 1 -- 172.21.15.40:0/813700032 wait complete. 2023-10-18T08:55:46.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.172+0000 7fcf1ae58640 1 Processor -- start 2023-10-18T08:55:46.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.173+0000 7fcf1ae58640 1 -- start start 2023-10-18T08:55:46.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.173+0000 7fcf1ae58640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf1406fea0 0x7fcf141593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:46.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.173+0000 7fcf1ae58640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf14151cb0 0x7fcf141598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:46.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.173+0000 7fcf1ae58640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf14158310 0x7fcf141392e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:46.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.174+0000 7fcf1ae58640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1415ac10 con 0x7fcf1406fea0 2023-10-18T08:55:46.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.174+0000 7fcf1ae58640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf14137030 con 0x7fcf14151cb0 2023-10-18T08:55:46.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.174+0000 7fcf1ae58640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf140ab1e0 con 0x7fcf14158310 2023-10-18T08:55:46.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.174+0000 7fcf18bf4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf1406fea0 0x7fcf141593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:46.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.174+0000 7fcf13fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf14151cb0 0x7fcf141598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:46.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.174+0000 7fcf13fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf14151cb0 0x7fcf141598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.40:37766/0 (socket says 172.21.15.40:37766) 2023-10-18T08:55:46.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.174+0000 7fcf13fff640 1 -- 172.21.15.40:0/678254621 learned_addr learned my addr 172.21.15.40:0/678254621 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:46.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.174+0000 7fcf18bf4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf1406fea0 0x7fcf141593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39154/0 (socket says 172.21.15.40:39154) 2023-10-18T08:55:46.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.174+0000 7fcf193f5640 1 --2- 172.21.15.40:0/678254621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf14158310 0x7fcf141392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:46.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.175+0000 7fcf18bf4640 1 -- 172.21.15.40:0/678254621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf14158310 msgr2=0x7fcf141392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:46.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.175+0000 7fcf18bf4640 1 --2- 172.21.15.40:0/678254621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf14158310 0x7fcf141392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:46.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.175+0000 7fcf18bf4640 1 -- 172.21.15.40:0/678254621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf14151cb0 msgr2=0x7fcf141598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:46.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.175+0000 7fcf18bf4640 1 --2- 172.21.15.40:0/678254621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf14151cb0 0x7fcf141598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:46.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.175+0000 7fcf18bf4640 1 -- 172.21.15.40:0/678254621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf08009df0 con 0x7fcf1406fea0 2023-10-18T08:55:46.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.175+0000 7fcf18bf4640 1 --2- 172.21.15.40:0/678254621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf1406fea0 0x7fcf141593b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcf08033000 tx=0x7fcf08004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:46.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.175+0000 7fcf117fa640 1 -- 172.21.15.40:0/678254621 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08004460 con 0x7fcf1406fea0 2023-10-18T08:55:46.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.175+0000 7fcf117fa640 1 -- 172.21.15.40:0/678254621 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcf0802ce90 con 0x7fcf1406fea0 2023-10-18T08:55:46.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.175+0000 7fcf117fa640 1 -- 172.21.15.40:0/678254621 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08004ea0 con 0x7fcf1406fea0 2023-10-18T08:55:46.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.175+0000 7fcf1ae58640 1 -- 172.21.15.40:0/678254621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf14159f90 con 0x7fcf1406fea0 2023-10-18T08:55:46.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.176+0000 7fcf1ae58640 1 -- 172.21.15.40:0/678254621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1415a550 con 0x7fcf1406fea0 2023-10-18T08:55:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.177+0000 7fcf1ae58640 1 -- 172.21.15.40:0/678254621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf14066fd0 con 0x7fcf1406fea0 2023-10-18T08:55:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.178+0000 7fcf117fa640 1 -- 172.21.15.40:0/678254621 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcf08005000 con 0x7fcf1406fea0 2023-10-18T08:55:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.179+0000 7fcf117fa640 1 --2- 172.21.15.40:0/678254621 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcef806ae40 0x7fcef806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.179+0000 7fcf117fa640 1 -- 172.21.15.40:0/678254621 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf080ac240 con 0x7fcf1406fea0 2023-10-18T08:55:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.179+0000 7fcf13fff640 1 --2- 172.21.15.40:0/678254621 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcef806ae40 0x7fcef806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.181+0000 7fcf13fff640 1 --2- 172.21.15.40:0/678254621 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcef806ae40 0x7fcef806d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcf141548f0 tx=0x7fcf00007ef0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.184+0000 7fcf117fa640 1 -- 172.21.15.40:0/678254621 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcf08077390 con 0x7fcf1406fea0 2023-10-18T08:55:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.418+0000 7fcf1ae58640 1 -- 172.21.15.40:0/678254621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcf140b0590 con 0x7fcf1406fea0 2023-10-18T08:55:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.419+0000 7fcf117fa640 1 -- 172.21.15.40:0/678254621 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcf08042060 con 0x7fcf1406fea0 2023-10-18T08:55:46.422 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T08:55:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.421+0000 7fcf1ae58640 1 -- 172.21.15.40:0/678254621 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcef806ae40 msgr2=0x7fcef806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.421+0000 7fcf1ae58640 1 --2- 172.21.15.40:0/678254621 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcef806ae40 0x7fcef806d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcf141548f0 tx=0x7fcf00007ef0 comp rx=0 tx=0).stop 2023-10-18T08:55:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.421+0000 7fcf1ae58640 1 -- 172.21.15.40:0/678254621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf1406fea0 msgr2=0x7fcf141593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.421+0000 7fcf1ae58640 1 --2- 172.21.15.40:0/678254621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf1406fea0 0x7fcf141593b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcf08033000 tx=0x7fcf08004a50 comp rx=0 tx=0).stop 2023-10-18T08:55:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.421+0000 7fcf1ae58640 1 -- 172.21.15.40:0/678254621 shutdown_connections 2023-10-18T08:55:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.421+0000 7fcf1ae58640 1 --2- 172.21.15.40:0/678254621 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcef806ae40 0x7fcef806d300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.422+0000 7fcf1ae58640 1 --2- 172.21.15.40:0/678254621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf14158310 0x7fcf141392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.422+0000 7fcf1ae58640 1 --2- 172.21.15.40:0/678254621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf14151cb0 0x7fcf141598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.422+0000 7fcf1ae58640 1 --2- 172.21.15.40:0/678254621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf1406fea0 0x7fcf141593b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:46.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.422+0000 7fcf1ae58640 1 -- 172.21.15.40:0/678254621 >> 172.21.15.40:0/678254621 conn(0x7fcf14069720 msgr2=0x7fcf141560c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:46.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.422+0000 7fcf1ae58640 1 -- 172.21.15.40:0/678254621 shutdown_connections 2023-10-18T08:55:46.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:46.422+0000 7fcf1ae58640 1 -- 172.21.15.40:0/678254621 wait complete. 2023-10-18T08:55:46.444 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T08:55:46.444 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T08:55:46.445 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-10-18T08:55:46.445 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-18T08:55:46.445 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T08:55:46.445 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T08:55:46.446 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T08:55:46.446 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T08:55:46.447 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T08:55:46.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:46.447 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-10-18T08:55:46.507 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T08:55:46.507 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T08:55:46.508 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T08:55:46.508 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:46.508 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-18T08:55:46.508 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:46.508 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:55:46.509 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:46.509 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2023-10-18T08:55:46.509 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:55:46.509 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:46.509 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2023-10-18T08:55:46.510 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:55:46.510 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:46.510 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T08:55:46.510 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:46.510 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T08:55:46.511 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42371sec preferred_lft 42371sec 2023-10-18T08:55:46.511 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2023-10-18T08:55:46.511 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:46.511 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-18T08:55:46.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:46.512 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T08:55:46.512 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T08:55:46.513 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add name ceph-brx type bridge 2023-10-18T08:55:46.513 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip addr flush dev ceph-brx 2023-10-18T08:55:46.513 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set ceph-brx up 2023-10-18T08:55:46.513 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-18T08:55:46.514 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T08:55:46.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:46.976 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T08:55:47.065 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-10-18T08:55:47.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:47.069 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-10-18T08:55:47.120 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-10-18T08:55:47.120 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T08:55:47.121 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T08:55:47.121 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T08:55:47.121 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T08:55:47.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:47.122 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T08:55:47.122 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T08:55:47.122 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T08:55:47.123 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T08:55:47.123 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T08:55:47.123 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T08:55:47.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:47.824 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-18T08:55:47.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:47.881 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list-id 2023-10-18T08:55:47.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:47.938 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T08:55:47.939 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T08:55:47.939 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T08:55:47.939 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-18T08:55:47.940 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T08:55:48.083 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-18T08:55:48.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:48.084 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T08:55:48.085 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T08:55:48.085 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-18T08:55:48.085 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-18T08:55:48.086 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-18T08:55:48.086 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-18T08:55:48.086 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-18T08:55:48.086 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T08:55:48.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:48.371 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T08:55:48.371 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T08:55:48.371 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2023-10-18T08:55:48.372 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set dev brx.0 master ceph-brx 2023-10-18T08:55:48.372 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T08:55:48.535 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:55:48.535 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-18T08:55:48.584 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-18T08:55:48.585 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:55:48.585 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-18T08:55:48.683 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-18T08:55:48.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:48.684 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:55:48.833 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:55:48.833 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:55:49.316 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:55:49.316 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:55:49.316 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T08:55:49.317 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with new device syntax: 1@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T08:55:49.317 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T08:55:49.319 INFO:teuthology.orchestra.run.smithi144.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-18T08:55:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: You just mounted a file system that supports labels which does not 2023-10-18T08:55:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T08:55:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T08:55:49.320 INFO:teuthology.orchestra.run.smithi144.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T08:55:49.323 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T08:55:49.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:49.324 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-18T08:55:49.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:49.412 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:55:49.465 INFO:teuthology.orchestra.run.smithi144.stdout:client4580 2023-10-18T08:55:49.465 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:55:49.466 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:55:49.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:49.519 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:55:49.574 INFO:teuthology.orchestra.run.smithi144.stdout:client4580 2023-10-18T08:55:49.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:49.575 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:55:49.626 INFO:teuthology.orchestra.run.smithi144.stdout:client4580 2023-10-18T08:55:49.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:49.627 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/029855e3-538a-4478-b323-f54ab0f3a56d.client4580/status') 2023-10-18T08:55:49.717 INFO:teuthology.orchestra.run.smithi144.stdout:instance: client.4580 (3)192.168.0.1:0/2563975350 2023-10-18T08:55:49.717 INFO:teuthology.orchestra.run.smithi144.stdout:blocklisted: false 2023-10-18T08:55:49.718 INFO:teuthology.orchestra.run.smithi144.stderr:0+1 records in 2023-10-18T08:55:49.718 INFO:teuthology.orchestra.run.smithi144.stderr:0+1 records out 2023-10-18T08:55:49.718 INFO:teuthology.orchestra.run.smithi144.stderr:69 bytes copied, 7.4237e-05 s, 929 kB/s 2023-10-18T08:55:49.721 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-10-18T08:55:49.722 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T08:55:49.722 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-18T08:55:49.722 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T08:55:49.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.965+0000 7f137cdef640 1 -- 172.21.15.40:0/551001631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1360003ef0 msgr2=0x7f13600047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:49.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.965+0000 7f137cdef640 1 --2- 172.21.15.40:0/551001631 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1360003ef0 0x7f13600047c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f136c009b50 tx=0x7f13600057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:49.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.965+0000 7f1377fff640 1 -- 172.21.15.40:0/551001631 reap_dead start 2023-10-18T08:55:49.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.965+0000 7f137cdef640 1 -- 172.21.15.40:0/551001631 shutdown_connections 2023-10-18T08:55:49.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.965+0000 7f137cdef640 1 -- 172.21.15.40:0/551001631 >> 172.21.15.40:0/551001631 conn(0x7f137806ca10 msgr2=0x7f137806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.965+0000 7f137cdef640 1 -- 172.21.15.40:0/551001631 shutdown_connections 2023-10-18T08:55:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.965+0000 7f137cdef640 1 -- 172.21.15.40:0/551001631 wait complete. 2023-10-18T08:55:49.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.967+0000 7f137cdef640 1 Processor -- start 2023-10-18T08:55:49.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.967+0000 7f137cdef640 1 -- start start 2023-10-18T08:55:49.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.969+0000 7f137cdef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1360003ef0 0x7f1378151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:49.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.969+0000 7f137cdef640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f137814dfd0 0x7f137813daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:49.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.969+0000 7f137cdef640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1378153860 0x7f1378153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:49.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.969+0000 7f137cdef640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1378154220 con 0x7f1360003ef0 2023-10-18T08:55:49.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.969+0000 7f137cdef640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137814f3a0 con 0x7f137814dfd0 2023-10-18T08:55:49.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.969+0000 7f137cdef640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137813dfe0 con 0x7f1378153860 2023-10-18T08:55:49.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.969+0000 7f13777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1360003ef0 0x7f1378151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:49.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.969+0000 7f13777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1360003ef0 0x7f1378151450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39176/0 (socket says 172.21.15.40:39176) 2023-10-18T08:55:49.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.969+0000 7f13777fe640 1 -- 172.21.15.40:0/2407964641 learned_addr learned my addr 172.21.15.40:0/2407964641 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:49.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.969+0000 7f1376ffd640 1 --2- 172.21.15.40:0/2407964641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f137814dfd0 0x7f137813daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:49.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.969+0000 7f1377fff640 1 --2- 172.21.15.40:0/2407964641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1378153860 0x7f1378153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:49.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.969+0000 7f13777fe640 1 -- 172.21.15.40:0/2407964641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1378153860 msgr2=0x7f1378153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:49.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.969+0000 7f13777fe640 1 --2- 172.21.15.40:0/2407964641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1378153860 0x7f1378153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:49.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.969+0000 7f13777fe640 1 -- 172.21.15.40:0/2407964641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f137814dfd0 msgr2=0x7f137813daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:49.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.969+0000 7f13777fe640 1 --2- 172.21.15.40:0/2407964641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f137814dfd0 0x7f137813daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:49.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.969+0000 7f13777fe640 1 -- 172.21.15.40:0/2407964641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137814e790 con 0x7f1360003ef0 2023-10-18T08:55:49.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.970+0000 7f1376ffd640 1 --2- 172.21.15.40:0/2407964641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f137814dfd0 0x7f137813daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:49.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.970+0000 7f1377fff640 1 --2- 172.21.15.40:0/2407964641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1378153860 0x7f1378153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:49.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.970+0000 7f13777fe640 1 --2- 172.21.15.40:0/2407964641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1360003ef0 0x7f1378151450 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f136800fc30 tx=0x7f136800d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:49.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.970+0000 7f135ffff640 1 -- 172.21.15.40:0/2407964641 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136800dee0 con 0x7f1360003ef0 2023-10-18T08:55:49.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.970+0000 7f135ffff640 1 -- 172.21.15.40:0/2407964641 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1368011640 con 0x7f1360003ef0 2023-10-18T08:55:49.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.970+0000 7f135ffff640 1 -- 172.21.15.40:0/2407964641 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1368011a50 con 0x7f1360003ef0 2023-10-18T08:55:49.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.970+0000 7f137cdef640 1 -- 172.21.15.40:0/2407964641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137814ea70 con 0x7f1360003ef0 2023-10-18T08:55:49.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.970+0000 7f137cdef640 1 -- 172.21.15.40:0/2407964641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137814f030 con 0x7f1360003ef0 2023-10-18T08:55:49.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.972+0000 7f137cdef640 1 -- 172.21.15.40:0/2407964641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1378066e00 con 0x7f1360003ef0 2023-10-18T08:55:49.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.973+0000 7f135ffff640 1 -- 172.21.15.40:0/2407964641 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1368011bb0 con 0x7f1360003ef0 2023-10-18T08:55:49.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.973+0000 7f135ffff640 1 --2- 172.21.15.40:0/2407964641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f136406ae40 0x7f136406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:49.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.974+0000 7f135ffff640 1 -- 172.21.15.40:0/2407964641 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f136808a5e0 con 0x7f1360003ef0 2023-10-18T08:55:49.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.974+0000 7f1376ffd640 1 --2- 172.21.15.40:0/2407964641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f136406ae40 0x7f136406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:49.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.975+0000 7f1376ffd640 1 --2- 172.21.15.40:0/2407964641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f136406ae40 0x7f136406d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f136c036000 tx=0x7f136c035040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:49.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:49.978+0000 7f135ffff640 1 -- 172.21.15.40:0/2407964641 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1368055690 con 0x7f1360003ef0 2023-10-18T08:55:50.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:50.213+0000 7f137cdef640 1 -- 172.21.15.40:0/2407964641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f137806a730 con 0x7f1360003ef0 2023-10-18T08:55:50.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:50.214+0000 7f135ffff640 1 -- 172.21.15.40:0/2407964641 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1368011e60 con 0x7f1360003ef0 2023-10-18T08:55:50.217 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T08:55:50.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:50.217+0000 7f137cdef640 1 -- 172.21.15.40:0/2407964641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f136406ae40 msgr2=0x7f136406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:50.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:50.217+0000 7f137cdef640 1 --2- 172.21.15.40:0/2407964641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f136406ae40 0x7f136406d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f136c036000 tx=0x7f136c035040 comp rx=0 tx=0).stop 2023-10-18T08:55:50.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:50.217+0000 7f137cdef640 1 -- 172.21.15.40:0/2407964641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1360003ef0 msgr2=0x7f1378151450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:50.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:50.217+0000 7f137cdef640 1 --2- 172.21.15.40:0/2407964641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1360003ef0 0x7f1378151450 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f136800fc30 tx=0x7f136800d6b0 comp rx=0 tx=0).stop 2023-10-18T08:55:50.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:50.217+0000 7f137cdef640 1 -- 172.21.15.40:0/2407964641 shutdown_connections 2023-10-18T08:55:50.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:50.217+0000 7f137cdef640 1 --2- 172.21.15.40:0/2407964641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f136406ae40 0x7f136406d300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:50.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:50.217+0000 7f137cdef640 1 --2- 172.21.15.40:0/2407964641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1378153860 0x7f1378153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:50.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:50.217+0000 7f137cdef640 1 --2- 172.21.15.40:0/2407964641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f137814dfd0 0x7f137813daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:50.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:50.217+0000 7f137cdef640 1 --2- 172.21.15.40:0/2407964641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1360003ef0 0x7f1378151450 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:50.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:50.217+0000 7f137cdef640 1 -- 172.21.15.40:0/2407964641 >> 172.21.15.40:0/2407964641 conn(0x7f137806ca10 msgr2=0x7f13780cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:50.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:50.217+0000 7f137cdef640 1 -- 172.21.15.40:0/2407964641 shutdown_connections 2023-10-18T08:55:50.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:50.217+0000 7f137cdef640 1 -- 172.21.15.40:0/2407964641 wait complete. 2023-10-18T08:55:50.238 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T08:55:50.239 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T08:55:50.239 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-10-18T08:55:50.239 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-18T08:55:50.239 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-18T08:55:50.240 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T08:55:50.240 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T08:55:50.240 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T08:55:50.240 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-18T08:55:50.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:50.241 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-10-18T08:55:50.301 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T08:55:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T08:55:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T08:55:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:50.302 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-18T08:55:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:50.303 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:55:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:50.303 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2023-10-18T08:55:50.303 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:55:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2023-10-18T08:55:50.304 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:55:50.304 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:50.304 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T08:55:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T08:55:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42368sec preferred_lft 42368sec 2023-10-18T08:55:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2023-10-18T08:55:50.305 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:50.306 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T08:55:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 4a:6e:a9:f0:92:62 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:55:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T08:55:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:50.306 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1493:a0ff:fed2:2c1/64 scope link 2023-10-18T08:55:50.307 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:50.307 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T08:55:50.307 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 4a:6e:a9:f0:92:62 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T08:55:50.307 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::486e:a9ff:fef0:9262/64 scope link 2023-10-18T08:55:50.307 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:50.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:50.309 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T08:55:50.405 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-10-18T08:55:50.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:50.409 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-10-18T08:55:50.464 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-10-18T08:55:50.465 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T08:55:50.465 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T08:55:50.465 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T08:55:50.466 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T08:55:50.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:50.467 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T08:55:50.467 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T08:55:50.467 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T08:55:50.468 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T08:55:50.468 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T08:55:50.468 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T08:55:50.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:50.648 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-18T08:55:50.695 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T08:55:50.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:50.696 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list-id 2023-10-18T08:55:50.794 INFO:teuthology.orchestra.run.smithi144.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-18T08:55:50.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:50.796 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T08:55:50.796 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T08:55:50.796 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T08:55:50.797 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-18T08:55:50.797 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T08:55:50.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:50.942 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-18T08:55:51.029 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T08:55:51.029 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T08:55:51.030 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T08:55:51.030 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:51.030 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-18T08:55:51.030 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:51.030 INFO:teuthology.orchestra.run.smithi144.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T08:55:51.031 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether be:47:09:a0:b0:8f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-18T08:55:51.031 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-18T08:55:51.031 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:51.031 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::bc47:9ff:fea0:b08f/64 scope link 2023-10-18T08:55:51.031 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:51.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:51.033 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-18T08:55:51.123 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T08:55:51.124 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T08:55:51.124 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T08:55:51.124 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:51.124 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-18T08:55:51.124 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:51.125 INFO:teuthology.orchestra.run.smithi144.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T08:55:51.125 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether be:47:09:a0:b0:8f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-18T08:55:51.125 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-18T08:55:51.125 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:51.125 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::bc47:9ff:fea0:b08f/64 scope link 2023-10-18T08:55:51.126 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:55:51.128 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-18T08:55:51.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:51.129 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T08:55:51.129 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T08:55:51.130 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-18T08:55:51.130 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-18T08:55:51.131 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-18T08:55:51.131 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-18T08:55:51.131 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-18T08:55:51.131 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T08:55:51.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:51.395 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T08:55:51.396 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T08:55:51.396 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.1 up 2023-10-18T08:55:51.396 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set dev brx.1 master ceph-brx 2023-10-18T08:55:51.397 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T08:55:51.568 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:55:51.568 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-18T08:55:51.619 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-18T08:55:51.620 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:55:51.620 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-18T08:55:51.669 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-10-18T08:55:51.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:51.670 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:55:51.809 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:55:51.809 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:55:51.842 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:55:51.842 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:55:51.843 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T08:55:51.843 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with new device syntax: 3@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T08:55:51.843 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T08:55:51.844 INFO:teuthology.orchestra.run.smithi144.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2023-10-18T08:55:51.844 INFO:teuthology.orchestra.run.smithi144.stdout: You just mounted a file system that supports labels which does not 2023-10-18T08:55:51.844 INFO:teuthology.orchestra.run.smithi144.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T08:55:51.844 INFO:teuthology.orchestra.run.smithi144.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T08:55:51.845 INFO:teuthology.orchestra.run.smithi144.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T08:55:51.848 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T08:55:51.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:51.849 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-18T08:55:51.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:51.938 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:55:51.997 INFO:teuthology.orchestra.run.smithi144.stdout:client4767 2023-10-18T08:55:51.998 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:55:51.998 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:55:52.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:52.047 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:55:52.150 INFO:teuthology.orchestra.run.smithi144.stdout:client4767 2023-10-18T08:55:52.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:52.151 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:55:52.212 INFO:teuthology.orchestra.run.smithi144.stdout:client4767 2023-10-18T08:55:52.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:55:52.213 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/029855e3-538a-4478-b323-f54ab0f3a56d.client4767/status') 2023-10-18T08:55:52.300 INFO:teuthology.orchestra.run.smithi144.stdout:instance: client.4767 (3)172.21.15.144:0/603862188 2023-10-18T08:55:52.300 INFO:teuthology.orchestra.run.smithi144.stdout:blocklisted: false 2023-10-18T08:55:52.301 INFO:teuthology.orchestra.run.smithi144.stderr:0+1 records in 2023-10-18T08:55:52.301 INFO:teuthology.orchestra.run.smithi144.stderr:0+1 records out 2023-10-18T08:55:52.301 INFO:teuthology.orchestra.run.smithi144.stderr:70 bytes copied, 7.5956e-05 s, 922 kB/s 2023-10-18T08:55:52.304 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-18T08:55:52.319 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:52.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.549+0000 7f95cedbf640 1 -- 172.21.15.40:0/2710487709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c80b15b0 msgr2=0x7f95c80c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:52.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.549+0000 7f95cedbf640 1 --2- 172.21.15.40:0/2710487709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c80b15b0 0x7f95c80c99c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f95b8005fd0 tx=0x7f95b802be20 comp rx=0 tx=0).stop 2023-10-18T08:55:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.550+0000 7f95cedbf640 1 -- 172.21.15.40:0/2710487709 shutdown_connections 2023-10-18T08:55:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.550+0000 7f95cedbf640 1 --2- 172.21.15.40:0/2710487709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c80c9f00 0x7f95c8157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.550+0000 7f95cedbf640 1 --2- 172.21.15.40:0/2710487709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c80b15b0 0x7f95c80c99c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.550+0000 7f95cedbf640 1 -- 172.21.15.40:0/2710487709 >> 172.21.15.40:0/2710487709 conn(0x7f95c8069720 msgr2=0x7f95c8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.550+0000 7f95cedbf640 1 -- 172.21.15.40:0/2710487709 shutdown_connections 2023-10-18T08:55:52.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.550+0000 7f95cedbf640 1 -- 172.21.15.40:0/2710487709 wait complete. 2023-10-18T08:55:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.551+0000 7f95cedbf640 1 Processor -- start 2023-10-18T08:55:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.552+0000 7f95cedbf640 1 -- start start 2023-10-18T08:55:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.552+0000 7f95cedbf640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c806d6f0 0x7f95c80acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.552+0000 7f95cedbf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c80b15b0 0x7f95c80cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.552+0000 7f95cedbf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c80c9f00 0x7f95c8147590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:52.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.552+0000 7f95cedbf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c81586c0 con 0x7f95c80c9f00 2023-10-18T08:55:52.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.552+0000 7f95cedbf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c80cc8e0 con 0x7f95c80b15b0 2023-10-18T08:55:52.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.552+0000 7f95cedbf640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c80ad4e0 con 0x7f95c806d6f0 2023-10-18T08:55:52.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.553+0000 7f95ce5be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c80c9f00 0x7f95c8147590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:52.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.553+0000 7f95ce5be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c80c9f00 0x7f95c8147590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39202/0 (socket says 172.21.15.40:39202) 2023-10-18T08:55:52.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.553+0000 7f95ce5be640 1 -- 172.21.15.40:0/2380634323 learned_addr learned my addr 172.21.15.40:0/2380634323 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.553+0000 7f95cd5bc640 1 --2- 172.21.15.40:0/2380634323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c80b15b0 0x7f95c80cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.553+0000 7f95cddbd640 1 --2- 172.21.15.40:0/2380634323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c806d6f0 0x7f95c80acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.553+0000 7f95ce5be640 1 -- 172.21.15.40:0/2380634323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c806d6f0 msgr2=0x7f95c80acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.553+0000 7f95ce5be640 1 --2- 172.21.15.40:0/2380634323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c806d6f0 0x7f95c80acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.553+0000 7f95ce5be640 1 -- 172.21.15.40:0/2380634323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c80b15b0 msgr2=0x7f95c80cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.553+0000 7f95ce5be640 1 --2- 172.21.15.40:0/2380634323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c80b15b0 0x7f95c80cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.553+0000 7f95ce5be640 1 -- 172.21.15.40:0/2380634323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b8009560 con 0x7f95c80c9f00 2023-10-18T08:55:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.553+0000 7f95cd5bc640 1 --2- 172.21.15.40:0/2380634323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c80b15b0 0x7f95c80cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.553+0000 7f95cddbd640 1 --2- 172.21.15.40:0/2380634323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c806d6f0 0x7f95c80acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.554+0000 7f95ce5be640 1 --2- 172.21.15.40:0/2380634323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c80c9f00 0x7f95c8147590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f95c000e960 tx=0x7f95c000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.554+0000 7f95be7fc640 1 -- 172.21.15.40:0/2380634323 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0004d20 con 0x7f95c80c9f00 2023-10-18T08:55:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.554+0000 7f95be7fc640 1 -- 172.21.15.40:0/2380634323 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f95c000cb20 con 0x7f95c80c9f00 2023-10-18T08:55:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.554+0000 7f95be7fc640 1 -- 172.21.15.40:0/2380634323 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0005020 con 0x7f95c80c9f00 2023-10-18T08:55:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.554+0000 7f95cedbf640 1 -- 172.21.15.40:0/2380634323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c80ccb80 con 0x7f95c80c9f00 2023-10-18T08:55:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.554+0000 7f95cedbf640 1 -- 172.21.15.40:0/2380634323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c814e6c0 con 0x7f95c80c9f00 2023-10-18T08:55:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.555+0000 7f95cedbf640 1 -- 172.21.15.40:0/2380634323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c80ccd70 con 0x7f95c80c9f00 2023-10-18T08:55:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.556+0000 7f95be7fc640 1 -- 172.21.15.40:0/2380634323 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f95c0005180 con 0x7f95c80c9f00 2023-10-18T08:55:52.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.558+0000 7f95be7fc640 1 --2- 172.21.15.40:0/2380634323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f95ac06ae40 0x7f95ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:52.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.558+0000 7f95be7fc640 1 -- 172.21.15.40:0/2380634323 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95c008a020 con 0x7f95c80c9f00 2023-10-18T08:55:52.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.558+0000 7f95cddbd640 1 --2- 172.21.15.40:0/2380634323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f95ac06ae40 0x7f95ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:52.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.559+0000 7f95cddbd640 1 --2- 172.21.15.40:0/2380634323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f95ac06ae40 0x7f95ac06d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000f60 tx=0x7f95c4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:52.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.563+0000 7f95be7fc640 1 -- 172.21.15.40:0/2380634323 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95c0055030 con 0x7f95c80c9f00 2023-10-18T08:55:52.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.762+0000 7f95cedbf640 1 -- 172.21.15.40:0/2380634323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95c8158b50 con 0x7f95c80c9f00 2023-10-18T08:55:52.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.765+0000 7f95be7fc640 1 -- 172.21.15.40:0/2380634323 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f95c8158b50 con 0x7f95c80c9f00 2023-10-18T08:55:52.769 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:52.769 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:16.275677+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:52.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.768+0000 7f95cedbf640 1 -- 172.21.15.40:0/2380634323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f95ac06ae40 msgr2=0x7f95ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:52.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.768+0000 7f95cedbf640 1 --2- 172.21.15.40:0/2380634323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f95ac06ae40 0x7f95ac06d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000f60 tx=0x7f95c4009040 comp rx=0 tx=0).stop 2023-10-18T08:55:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.768+0000 7f95cedbf640 1 -- 172.21.15.40:0/2380634323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c80c9f00 msgr2=0x7f95c8147590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.768+0000 7f95cedbf640 1 --2- 172.21.15.40:0/2380634323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c80c9f00 0x7f95c8147590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f95c000e960 tx=0x7f95c000ee20 comp rx=0 tx=0).stop 2023-10-18T08:55:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.768+0000 7f95cedbf640 1 -- 172.21.15.40:0/2380634323 shutdown_connections 2023-10-18T08:55:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.768+0000 7f95cedbf640 1 --2- 172.21.15.40:0/2380634323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f95ac06ae40 0x7f95ac06d300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.768+0000 7f95cedbf640 1 --2- 172.21.15.40:0/2380634323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95c80c9f00 0x7f95c8147590 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.768+0000 7f95cedbf640 1 --2- 172.21.15.40:0/2380634323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c80b15b0 0x7f95c80cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.768+0000 7f95cedbf640 1 --2- 172.21.15.40:0/2380634323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c806d6f0 0x7f95c80acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.768+0000 7f95cedbf640 1 -- 172.21.15.40:0/2380634323 >> 172.21.15.40:0/2380634323 conn(0x7f95c8069720 msgr2=0x7f95c8153420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:52.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.768+0000 7f95cedbf640 1 -- 172.21.15.40:0/2380634323 shutdown_connections 2023-10-18T08:55:52.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:52.768+0000 7f95cedbf640 1 -- 172.21.15.40:0/2380634323 wait complete. 2023-10-18T08:55:52.790 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:55:13.210788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:55:13.772849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:55:52.790 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:55:53.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.018+0000 7f024cdd0640 1 -- 172.21.15.40:0/3144769809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024814e730 msgr2=0x7f02480b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:53.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.018+0000 7f024cdd0640 1 --2- 172.21.15.40:0/3144769809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024814e730 0x7f02480b1950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f02380092e0 tx=0x7f023802bca0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.018+0000 7f024cdd0640 1 -- 172.21.15.40:0/3144769809 shutdown_connections 2023-10-18T08:55:53.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.018+0000 7f024cdd0640 1 --2- 172.21.15.40:0/3144769809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02480715c0 0x7f02480aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.018+0000 7f024cdd0640 1 --2- 172.21.15.40:0/3144769809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024814e730 0x7f02480b1950 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.018+0000 7f024cdd0640 1 -- 172.21.15.40:0/3144769809 >> 172.21.15.40:0/3144769809 conn(0x7f0248069720 msgr2=0x7f0248069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:53.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.018+0000 7f024cdd0640 1 -- 172.21.15.40:0/3144769809 shutdown_connections 2023-10-18T08:55:53.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.018+0000 7f024cdd0640 1 -- 172.21.15.40:0/3144769809 wait complete. 2023-10-18T08:55:53.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.020+0000 7f024cdd0640 1 Processor -- start 2023-10-18T08:55:53.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.020+0000 7f024cdd0640 1 -- start start 2023-10-18T08:55:53.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.021+0000 7f024cdd0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02480715c0 0x7f024813b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:53.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.021+0000 7f024cdd0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02480af240 0x7f02481572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:53.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.021+0000 7f024cdd0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024814e730 0x7f0248143860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:53.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.021+0000 7f024cdd0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024813b8f0 con 0x7f024814e730 2023-10-18T08:55:53.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.021+0000 7f024cdd0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0248138cc0 con 0x7f02480af240 2023-10-18T08:55:53.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.021+0000 7f024cdd0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0248138fa0 con 0x7f02480715c0 2023-10-18T08:55:53.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.021+0000 7f0246d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024814e730 0x7f0248143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:53.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.021+0000 7f0246d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024814e730 0x7f0248143860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39232/0 (socket says 172.21.15.40:39232) 2023-10-18T08:55:53.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.021+0000 7f0246d9d640 1 -- 172.21.15.40:0/1803506895 learned_addr learned my addr 172.21.15.40:0/1803506895 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:53.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.022+0000 7f0245d9b640 1 --2- 172.21.15.40:0/1803506895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02480af240 0x7f02481572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:53.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.022+0000 7f024659c640 1 --2- 172.21.15.40:0/1803506895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02480715c0 0x7f024813b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:53.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.022+0000 7f0246d9d640 1 -- 172.21.15.40:0/1803506895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02480715c0 msgr2=0x7f024813b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:53.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.022+0000 7f0246d9d640 1 --2- 172.21.15.40:0/1803506895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02480715c0 0x7f024813b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.022+0000 7f0246d9d640 1 -- 172.21.15.40:0/1803506895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02480af240 msgr2=0x7f02481572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:53.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.022+0000 7f0246d9d640 1 --2- 172.21.15.40:0/1803506895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02480af240 0x7f02481572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.022+0000 7f0246d9d640 1 -- 172.21.15.40:0/1803506895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0238009d20 con 0x7f024814e730 2023-10-18T08:55:53.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.022+0000 7f0245d9b640 1 --2- 172.21.15.40:0/1803506895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02480af240 0x7f02481572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:55:53.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.022+0000 7f0246d9d640 1 --2- 172.21.15.40:0/1803506895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024814e730 0x7f0248143860 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0240001560 tx=0x7f024000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:53.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.023+0000 7f0236ffd640 1 -- 172.21.15.40:0/1803506895 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240003da0 con 0x7f024814e730 2023-10-18T08:55:53.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.023+0000 7f0236ffd640 1 -- 172.21.15.40:0/1803506895 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0240005070 con 0x7f024814e730 2023-10-18T08:55:53.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.023+0000 7f0236ffd640 1 -- 172.21.15.40:0/1803506895 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240005460 con 0x7f024814e730 2023-10-18T08:55:53.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.023+0000 7f024cdd0640 1 -- 172.21.15.40:0/1803506895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0248139280 con 0x7f024814e730 2023-10-18T08:55:53.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.023+0000 7f024cdd0640 1 -- 172.21.15.40:0/1803506895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0248139840 con 0x7f024814e730 2023-10-18T08:55:53.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.024+0000 7f024cdd0640 1 -- 172.21.15.40:0/1803506895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0248139a60 con 0x7f024814e730 2023-10-18T08:55:53.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.025+0000 7f0236ffd640 1 -- 172.21.15.40:0/1803506895 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f02400055c0 con 0x7f024814e730 2023-10-18T08:55:53.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.027+0000 7f0236ffd640 1 --2- 172.21.15.40:0/1803506895 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f022806ae40 0x7f022806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:53.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.027+0000 7f0236ffd640 1 -- 172.21.15.40:0/1803506895 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02400891e0 con 0x7f024814e730 2023-10-18T08:55:53.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.028+0000 7f024659c640 1 --2- 172.21.15.40:0/1803506895 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f022806ae40 0x7f022806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:53.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.029+0000 7f024659c640 1 --2- 172.21.15.40:0/1803506895 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f022806ae40 0x7f022806d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0238036000 tx=0x7f0238004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:53.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.031+0000 7f0236ffd640 1 -- 172.21.15.40:0/1803506895 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f02400541f0 con 0x7f024814e730 2023-10-18T08:55:53.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.222+0000 7f024cdd0640 1 -- 172.21.15.40:0/1803506895 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f02480b23c0 con 0x7f024814e730 2023-10-18T08:55:53.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.224+0000 7f0236ffd640 1 -- 172.21.15.40:0/1803506895 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0240017020 con 0x7f024814e730 2023-10-18T08:55:53.227 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:55:53.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.226+0000 7f024cdd0640 1 -- 172.21.15.40:0/1803506895 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f022806ae40 msgr2=0x7f022806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:53.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.226+0000 7f024cdd0640 1 --2- 172.21.15.40:0/1803506895 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f022806ae40 0x7f022806d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0238036000 tx=0x7f0238004730 comp rx=0 tx=0).stop 2023-10-18T08:55:53.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.226+0000 7f024cdd0640 1 -- 172.21.15.40:0/1803506895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024814e730 msgr2=0x7f0248143860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:53.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.226+0000 7f024cdd0640 1 --2- 172.21.15.40:0/1803506895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024814e730 0x7f0248143860 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0240001560 tx=0x7f024000cc70 comp rx=0 tx=0).stop 2023-10-18T08:55:53.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.226+0000 7f024cdd0640 1 -- 172.21.15.40:0/1803506895 shutdown_connections 2023-10-18T08:55:53.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.226+0000 7f024cdd0640 1 --2- 172.21.15.40:0/1803506895 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f022806ae40 0x7f022806d300 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.226+0000 7f024cdd0640 1 --2- 172.21.15.40:0/1803506895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f024814e730 0x7f0248143860 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.226+0000 7f024cdd0640 1 --2- 172.21.15.40:0/1803506895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02480af240 0x7f02481572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.226+0000 7f024cdd0640 1 --2- 172.21.15.40:0/1803506895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02480715c0 0x7f024813b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.226+0000 7f024cdd0640 1 -- 172.21.15.40:0/1803506895 >> 172.21.15.40:0/1803506895 conn(0x7f0248069720 msgr2=0x7f024814dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.227+0000 7f024cdd0640 1 -- 172.21.15.40:0/1803506895 shutdown_connections 2023-10-18T08:55:53.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.227+0000 7f024cdd0640 1 -- 172.21.15.40:0/1803506895 wait complete. 2023-10-18T08:55:53.250 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:55:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.479+0000 7f206d01a640 1 -- 172.21.15.40:0/202192088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f206806d5d0 msgr2=0x7f20680c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.479+0000 7f206d01a640 1 --2- 172.21.15.40:0/202192088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f206806d5d0 0x7f20680c9620 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2058009830 tx=0x7f205802bcd0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.479+0000 7f206d01a640 1 -- 172.21.15.40:0/202192088 shutdown_connections 2023-10-18T08:55:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.479+0000 7f206d01a640 1 --2- 172.21.15.40:0/202192088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f206806d5d0 0x7f20680c9620 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.479+0000 7f206d01a640 1 -- 172.21.15.40:0/202192088 >> 172.21.15.40:0/202192088 conn(0x7f206806ca10 msgr2=0x7f206806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.479+0000 7f206d01a640 1 -- 172.21.15.40:0/202192088 shutdown_connections 2023-10-18T08:55:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.479+0000 7f206d01a640 1 -- 172.21.15.40:0/202192088 wait complete. 2023-10-18T08:55:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.481+0000 7f206d01a640 1 Processor -- start 2023-10-18T08:55:53.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.481+0000 7f206d01a640 1 -- start start 2023-10-18T08:55:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.482+0000 7f206d01a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f206806d5d0 0x7f2068153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.482+0000 7f206d01a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2068071270 0x7f20680c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.482+0000 7f206d01a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f206815af70 0x7f2068152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.482+0000 7f206d01a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20680c7f70 con 0x7f2068071270 2023-10-18T08:55:53.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.482+0000 7f206d01a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20680c6d00 con 0x7f206815af70 2023-10-18T08:55:53.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.482+0000 7f206d01a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2068153280 con 0x7f206806d5d0 2023-10-18T08:55:53.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.482+0000 7f205ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2068071270 0x7f20680c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:53.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.483+0000 7f205ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2068071270 0x7f20680c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39256/0 (socket says 172.21.15.40:39256) 2023-10-18T08:55:53.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.483+0000 7f205ffff640 1 -- 172.21.15.40:0/568856776 learned_addr learned my addr 172.21.15.40:0/568856776 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:53.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.483+0000 7f2067fff640 1 --2- 172.21.15.40:0/568856776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f206806d5d0 0x7f2068153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:53.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.483+0000 7f205ffff640 1 -- 172.21.15.40:0/568856776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f206806d5d0 msgr2=0x7f2068153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:53.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.483+0000 7f205ffff640 1 --2- 172.21.15.40:0/568856776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f206806d5d0 0x7f2068153a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.483+0000 7f205ffff640 1 -- 172.21.15.40:0/568856776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f206815af70 msgr2=0x7f2068152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:53.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.483+0000 7f206c819640 1 --2- 172.21.15.40:0/568856776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f206815af70 0x7f2068152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:53.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.483+0000 7f205ffff640 1 --2- 172.21.15.40:0/568856776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f206815af70 0x7f2068152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.483+0000 7f205ffff640 1 -- 172.21.15.40:0/568856776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2058009490 con 0x7f2068071270 2023-10-18T08:55:53.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.483+0000 7f205ffff640 1 --2- 172.21.15.40:0/568856776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2068071270 0x7f20680c7a30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f205000c410 tx=0x7f205000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:53.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.484+0000 7f20657fa640 1 -- 172.21.15.40:0/568856776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20500035f0 con 0x7f2068071270 2023-10-18T08:55:53.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.484+0000 7f20657fa640 1 -- 172.21.15.40:0/568856776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2050003770 con 0x7f2068071270 2023-10-18T08:55:53.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.484+0000 7f20657fa640 1 -- 172.21.15.40:0/568856776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2050003b80 con 0x7f2068071270 2023-10-18T08:55:53.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.484+0000 7f206d01a640 1 -- 172.21.15.40:0/568856776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2068154100 con 0x7f2068071270 2023-10-18T08:55:53.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.484+0000 7f206d01a640 1 -- 172.21.15.40:0/568856776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20680c7230 con 0x7f2068071270 2023-10-18T08:55:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.485+0000 7f206d01a640 1 -- 172.21.15.40:0/568856776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2068066e00 con 0x7f2068071270 2023-10-18T08:55:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.487+0000 7f20657fa640 1 -- 172.21.15.40:0/568856776 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2050003ce0 con 0x7f2068071270 2023-10-18T08:55:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.488+0000 7f20657fa640 1 --2- 172.21.15.40:0/568856776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f204806ae40 0x7f204806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:53.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.488+0000 7f20657fa640 1 -- 172.21.15.40:0/568856776 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2050088c50 con 0x7f2068071270 2023-10-18T08:55:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.488+0000 7f2067fff640 1 --2- 172.21.15.40:0/568856776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f204806ae40 0x7f204806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:53.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.489+0000 7f2067fff640 1 --2- 172.21.15.40:0/568856776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f204806ae40 0x7f204806d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2058036000 tx=0x7f2058004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:53.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.491+0000 7f20657fa640 1 -- 172.21.15.40:0/568856776 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2050053cf0 con 0x7f2068071270 2023-10-18T08:55:53.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.683+0000 7f206d01a640 1 -- 172.21.15.40:0/568856776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20680c9b30 con 0x7f2068071270 2023-10-18T08:55:53.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.684+0000 7f20657fa640 1 -- 172.21.15.40:0/568856776 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2050017020 con 0x7f2068071270 2023-10-18T08:55:53.687 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:55:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.686+0000 7f206d01a640 1 -- 172.21.15.40:0/568856776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f204806ae40 msgr2=0x7f204806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:53.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.686+0000 7f206d01a640 1 --2- 172.21.15.40:0/568856776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f204806ae40 0x7f204806d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2058036000 tx=0x7f2058004730 comp rx=0 tx=0).stop 2023-10-18T08:55:53.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.686+0000 7f206d01a640 1 -- 172.21.15.40:0/568856776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2068071270 msgr2=0x7f20680c7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:53.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.686+0000 7f206d01a640 1 --2- 172.21.15.40:0/568856776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2068071270 0x7f20680c7a30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f205000c410 tx=0x7f205000c8d0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.688+0000 7f206d01a640 1 -- 172.21.15.40:0/568856776 shutdown_connections 2023-10-18T08:55:53.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.688+0000 7f206d01a640 1 --2- 172.21.15.40:0/568856776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f204806ae40 0x7f204806d300 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.688+0000 7f206d01a640 1 --2- 172.21.15.40:0/568856776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f206815af70 0x7f2068152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.688+0000 7f206d01a640 1 --2- 172.21.15.40:0/568856776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2068071270 0x7f20680c7a30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.688+0000 7f206d01a640 1 --2- 172.21.15.40:0/568856776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f206806d5d0 0x7f2068153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.688+0000 7f206d01a640 1 -- 172.21.15.40:0/568856776 >> 172.21.15.40:0/568856776 conn(0x7f206806ca10 msgr2=0x7f20680afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:53.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.688+0000 7f206d01a640 1 -- 172.21.15.40:0/568856776 shutdown_connections 2023-10-18T08:55:53.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.688+0000 7f206d01a640 1 -- 172.21.15.40:0/568856776 wait complete. 2023-10-18T08:55:53.710 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:55:53.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.936+0000 7fe06d5b8640 1 -- 172.21.15.40:0/3406827101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe050003970 msgr2=0x7fe050003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.936+0000 7fe06d5b8640 1 --2- 172.21.15.40:0/3406827101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe050003970 0x7fe050003d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe05800bb20 tx=0x7fe0500057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.936+0000 7fe06cdb7640 1 -- 172.21.15.40:0/3406827101 reap_dead start 2023-10-18T08:55:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.936+0000 7fe06d5b8640 1 -- 172.21.15.40:0/3406827101 shutdown_connections 2023-10-18T08:55:53.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.936+0000 7fe06d5b8640 1 -- 172.21.15.40:0/3406827101 >> 172.21.15.40:0/3406827101 conn(0x7fe06806ca10 msgr2=0x7fe06806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:53.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.936+0000 7fe06d5b8640 1 -- 172.21.15.40:0/3406827101 shutdown_connections 2023-10-18T08:55:53.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.936+0000 7fe06d5b8640 1 -- 172.21.15.40:0/3406827101 wait complete. 2023-10-18T08:55:53.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.938+0000 7fe06d5b8640 1 Processor -- start 2023-10-18T08:55:53.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.938+0000 7fe06d5b8640 1 -- start start 2023-10-18T08:55:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.939+0000 7fe06d5b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe050003970 0x7fe0680ac180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:53.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.939+0000 7fe06d5b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe068158930 0x7fe0681f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.939+0000 7fe06d5b8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe06814c5a0 0x7fe068145680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.939+0000 7fe06d5b8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06814f380 con 0x7fe068158930 2023-10-18T08:55:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.939+0000 7fe06d5b8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe068158db0 con 0x7fe06814c5a0 2023-10-18T08:55:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.939+0000 7fe06d5b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0681f5520 con 0x7fe050003970 2023-10-18T08:55:53.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.939+0000 7fe0677fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe068158930 0x7fe0681f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.939+0000 7fe0677fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe068158930 0x7fe0681f4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39280/0 (socket says 172.21.15.40:39280) 2023-10-18T08:55:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.939+0000 7fe0677fe640 1 -- 172.21.15.40:0/2038328457 learned_addr learned my addr 172.21.15.40:0/2038328457 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.940+0000 7fe06cdb7640 1 --2- 172.21.15.40:0/2038328457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe06814c5a0 0x7fe068145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:53.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.940+0000 7fe0677fe640 1 -- 172.21.15.40:0/2038328457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe050003970 msgr2=0x7fe0680ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.940+0000 7fe067fff640 1 --2- 172.21.15.40:0/2038328457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe050003970 0x7fe0680ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.940+0000 7fe0677fe640 1 --2- 172.21.15.40:0/2038328457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe050003970 0x7fe0680ac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.940+0000 7fe0677fe640 1 -- 172.21.15.40:0/2038328457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe06814c5a0 msgr2=0x7fe068145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.940+0000 7fe0677fe640 1 --2- 172.21.15.40:0/2038328457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe06814c5a0 0x7fe068145680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:53.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.940+0000 7fe0677fe640 1 -- 172.21.15.40:0/2038328457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0681f5800 con 0x7fe068158930 2023-10-18T08:55:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.940+0000 7fe06cdb7640 1 --2- 172.21.15.40:0/2038328457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe06814c5a0 0x7fe068145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.940+0000 7fe067fff640 1 --2- 172.21.15.40:0/2038328457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe050003970 0x7fe0680ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.940+0000 7fe0677fe640 1 --2- 172.21.15.40:0/2038328457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe068158930 0x7fe0681f4cd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00c700 tx=0x7fe05c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.940+0000 7fe064ff9640 1 -- 172.21.15.40:0/2038328457 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c005480 con 0x7fe068158930 2023-10-18T08:55:53.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.941+0000 7fe064ff9640 1 -- 172.21.15.40:0/2038328457 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe05c00a640 con 0x7fe068158930 2023-10-18T08:55:53.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.941+0000 7fe064ff9640 1 -- 172.21.15.40:0/2038328457 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00aa50 con 0x7fe068158930 2023-10-18T08:55:53.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.941+0000 7fe06d5b8640 1 -- 172.21.15.40:0/2038328457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0681f5ae0 con 0x7fe068158930 2023-10-18T08:55:53.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.941+0000 7fe06d5b8640 1 -- 172.21.15.40:0/2038328457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06814ef50 con 0x7fe068158930 2023-10-18T08:55:53.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.942+0000 7fe06d5b8640 1 -- 172.21.15.40:0/2038328457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0680cab30 con 0x7fe068158930 2023-10-18T08:55:53.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.943+0000 7fe064ff9640 1 -- 172.21.15.40:0/2038328457 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe05c00abb0 con 0x7fe068158930 2023-10-18T08:55:53.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.945+0000 7fe064ff9640 1 --2- 172.21.15.40:0/2038328457 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe05406ae40 0x7fe05406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:53.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.945+0000 7fe064ff9640 1 -- 172.21.15.40:0/2038328457 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe05c088f30 con 0x7fe068158930 2023-10-18T08:55:53.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.945+0000 7fe067fff640 1 --2- 172.21.15.40:0/2038328457 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe05406ae40 0x7fe05406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:53.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.946+0000 7fe067fff640 1 --2- 172.21.15.40:0/2038328457 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe05406ae40 0x7fe05406d300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe058037000 tx=0x7fe05802e040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:53.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:53.949+0000 7fe064ff9640 1 -- 172.21.15.40:0/2038328457 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe05c053fd0 con 0x7fe068158930 2023-10-18T08:55:54.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.142+0000 7fe06d5b8640 1 -- 172.21.15.40:0/2038328457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0680c9b60 con 0x7fe068158930 2023-10-18T08:55:54.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.144+0000 7fe064ff9640 1 -- 172.21.15.40:0/2038328457 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe0680c9b60 con 0x7fe068158930 2023-10-18T08:55:54.147 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:55:54.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.146+0000 7fe06d5b8640 1 -- 172.21.15.40:0/2038328457 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe05406ae40 msgr2=0x7fe05406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:54.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.146+0000 7fe06d5b8640 1 --2- 172.21.15.40:0/2038328457 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe05406ae40 0x7fe05406d300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe058037000 tx=0x7fe05802e040 comp rx=0 tx=0).stop 2023-10-18T08:55:54.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.146+0000 7fe06d5b8640 1 -- 172.21.15.40:0/2038328457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe068158930 msgr2=0x7fe0681f4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:54.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.146+0000 7fe06d5b8640 1 --2- 172.21.15.40:0/2038328457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe068158930 0x7fe0681f4cd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00c700 tx=0x7fe05c00cbc0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.146+0000 7fe06d5b8640 1 -- 172.21.15.40:0/2038328457 shutdown_connections 2023-10-18T08:55:54.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.146+0000 7fe06d5b8640 1 --2- 172.21.15.40:0/2038328457 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe05406ae40 0x7fe05406d300 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.146+0000 7fe06d5b8640 1 --2- 172.21.15.40:0/2038328457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe06814c5a0 0x7fe068145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.146+0000 7fe06d5b8640 1 --2- 172.21.15.40:0/2038328457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe068158930 0x7fe0681f4cd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.146+0000 7fe06d5b8640 1 --2- 172.21.15.40:0/2038328457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe050003970 0x7fe0680ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.146+0000 7fe06d5b8640 1 -- 172.21.15.40:0/2038328457 >> 172.21.15.40:0/2038328457 conn(0x7fe06806ca10 msgr2=0x7fe0680b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:54.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.146+0000 7fe06d5b8640 1 -- 172.21.15.40:0/2038328457 shutdown_connections 2023-10-18T08:55:54.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.146+0000 7fe06d5b8640 1 -- 172.21.15.40:0/2038328457 wait complete. 2023-10-18T08:55:54.170 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:54.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.404+0000 7f52fee9e640 1 -- 172.21.15.40:0/1975381357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52f815b4a0 msgr2=0x7f52f80cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:54.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.404+0000 7f52fee9e640 1 --2- 172.21.15.40:0/1975381357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52f815b4a0 0x7f52f80cfe60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f52f80d0d50 tx=0x7f52f402c1e0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.405+0000 7f52fee9e640 1 -- 172.21.15.40:0/1975381357 shutdown_connections 2023-10-18T08:55:54.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.405+0000 7f52fee9e640 1 --2- 172.21.15.40:0/1975381357 >> v2:172.21.15.144:3300/0 conn(0x7f52f815b880 0x7f52f80cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.405+0000 7f52fee9e640 1 --2- 172.21.15.40:0/1975381357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52f815b4a0 0x7f52f80cfe60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.405+0000 7f52fee9e640 1 -- 172.21.15.40:0/1975381357 >> 172.21.15.40:0/1975381357 conn(0x7f52f808aca0 msgr2=0x7f52f809a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:54.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.405+0000 7f52fee9e640 1 -- 172.21.15.40:0/1975381357 shutdown_connections 2023-10-18T08:55:54.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.405+0000 7f52fee9e640 1 -- 172.21.15.40:0/1975381357 wait complete. 2023-10-18T08:55:54.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.407+0000 7f52fee9e640 1 Processor -- start 2023-10-18T08:55:54.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.407+0000 7f52fee9e640 1 -- start start 2023-10-18T08:55:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.408+0000 7f52fee9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52f80cdca0 0x7f52f80c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.408+0000 7f52fee9e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52f815b4a0 0x7f52f80b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.408+0000 7f52fee9e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52f80c4dc0 0x7f52f80c5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.408+0000 7f52fee9e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80b3e70 con 0x7f52f80cdca0 2023-10-18T08:55:54.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.408+0000 7f52fee9e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80b6e00 con 0x7f52f80c4dc0 2023-10-18T08:55:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.408+0000 7f52fee9e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80c52c0 con 0x7f52f815b4a0 2023-10-18T08:55:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.408+0000 7f52fde9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52f80cdca0 0x7f52f80c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.408+0000 7f52fde9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52f80cdca0 0x7f52f80c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39300/0 (socket says 172.21.15.40:39300) 2023-10-18T08:55:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.408+0000 7f52fde9c640 1 -- 172.21.15.40:0/2583071314 learned_addr learned my addr 172.21.15.40:0/2583071314 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:54.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.408+0000 7f52fd69b640 1 --2- 172.21.15.40:0/2583071314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52f815b4a0 0x7f52f80b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:54.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.408+0000 7f52fde9c640 1 -- 172.21.15.40:0/2583071314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52f815b4a0 msgr2=0x7f52f80b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:54.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.409+0000 7f52fde9c640 1 --2- 172.21.15.40:0/2583071314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52f815b4a0 0x7f52f80b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.409+0000 7f52fde9c640 1 -- 172.21.15.40:0/2583071314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52f80c4dc0 msgr2=0x7f52f80c5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:54.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.409+0000 7f52fe69d640 1 --2- 172.21.15.40:0/2583071314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52f80c4dc0 0x7f52f80c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:54.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.409+0000 7f52fde9c640 1 --2- 172.21.15.40:0/2583071314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52f80c4dc0 0x7f52f80c5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.409+0000 7f52fde9c640 1 -- 172.21.15.40:0/2583071314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f4009900 con 0x7f52f80cdca0 2023-10-18T08:55:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.409+0000 7f52fd69b640 1 --2- 172.21.15.40:0/2583071314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52f815b4a0 0x7f52f80b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.409+0000 7f52fe69d640 1 --2- 172.21.15.40:0/2583071314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52f80c4dc0 0x7f52f80c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.409+0000 7f52fde9c640 1 --2- 172.21.15.40:0/2583071314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52f80cdca0 0x7f52f80c8360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f52f4000f80 tx=0x7f52f4035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.409+0000 7f52ee7fc640 1 -- 172.21.15.40:0/2583071314 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4034440 con 0x7f52f80cdca0 2023-10-18T08:55:54.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.409+0000 7f52ee7fc640 1 -- 172.21.15.40:0/2583071314 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f52f4004450 con 0x7f52f80cdca0 2023-10-18T08:55:54.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.409+0000 7f52ee7fc640 1 -- 172.21.15.40:0/2583071314 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4004820 con 0x7f52f80cdca0 2023-10-18T08:55:54.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.409+0000 7f52fee9e640 1 -- 172.21.15.40:0/2583071314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f80b4110 con 0x7f52f80cdca0 2023-10-18T08:55:54.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.410+0000 7f52fee9e640 1 -- 172.21.15.40:0/2583071314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f80b46d0 con 0x7f52f80cdca0 2023-10-18T08:55:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.411+0000 7f52fee9e640 1 -- 172.21.15.40:0/2583071314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f8071da0 con 0x7f52f80cdca0 2023-10-18T08:55:54.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.412+0000 7f52ee7fc640 1 -- 172.21.15.40:0/2583071314 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f52f403b040 con 0x7f52f80cdca0 2023-10-18T08:55:54.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.413+0000 7f52ee7fc640 1 --2- 172.21.15.40:0/2583071314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f52dc06ae70 0x7f52dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:54.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.413+0000 7f52ee7fc640 1 -- 172.21.15.40:0/2583071314 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52f40ac5d0 con 0x7f52f80cdca0 2023-10-18T08:55:54.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.414+0000 7f52fd69b640 1 --2- 172.21.15.40:0/2583071314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f52dc06ae70 0x7f52dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:54.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.415+0000 7f52fd69b640 1 --2- 172.21.15.40:0/2583071314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f52dc06ae70 0x7f52dc06d330 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f52f80afc60 tx=0x7f52e8005cf0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:54.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.418+0000 7f52ee7fc640 1 -- 172.21.15.40:0/2583071314 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f52f40775e0 con 0x7f52f80cdca0 2023-10-18T08:55:54.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.611+0000 7f52fee9e640 1 -- 172.21.15.40:0/2583071314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52f80667d0 con 0x7f52f80cdca0 2023-10-18T08:55:54.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.614+0000 7f52ee7fc640 1 -- 172.21.15.40:0/2583071314 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f52f403d3f0 con 0x7f52f80cdca0 2023-10-18T08:55:54.617 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:54.617 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:16.275677+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.617+0000 7f52fee9e640 1 -- 172.21.15.40:0/2583071314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f52dc06ae70 msgr2=0x7f52dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:54.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.617+0000 7f52fee9e640 1 --2- 172.21.15.40:0/2583071314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f52dc06ae70 0x7f52dc06d330 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f52f80afc60 tx=0x7f52e8005cf0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.617+0000 7f52fee9e640 1 -- 172.21.15.40:0/2583071314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52f80cdca0 msgr2=0x7f52f80c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:54.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.617+0000 7f52fee9e640 1 --2- 172.21.15.40:0/2583071314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52f80cdca0 0x7f52f80c8360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f52f4000f80 tx=0x7f52f4035db0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.617+0000 7f52fee9e640 1 -- 172.21.15.40:0/2583071314 shutdown_connections 2023-10-18T08:55:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.617+0000 7f52fee9e640 1 --2- 172.21.15.40:0/2583071314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f52dc06ae70 0x7f52dc06d330 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.617+0000 7f52fee9e640 1 --2- 172.21.15.40:0/2583071314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52f80c4dc0 0x7f52f80c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.617+0000 7f52fee9e640 1 --2- 172.21.15.40:0/2583071314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52f815b4a0 0x7f52f80b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.617+0000 7f52fee9e640 1 --2- 172.21.15.40:0/2583071314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52f80cdca0 0x7f52f80c8360 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.617+0000 7f52fee9e640 1 -- 172.21.15.40:0/2583071314 >> 172.21.15.40:0/2583071314 conn(0x7f52f808aca0 msgr2=0x7f52f80cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.617+0000 7f52fee9e640 1 -- 172.21.15.40:0/2583071314 shutdown_connections 2023-10-18T08:55:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.617+0000 7f52fee9e640 1 -- 172.21.15.40:0/2583071314 wait complete. 2023-10-18T08:55:54.640 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:55:13.210788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:55:13.772849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:55:54.640 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:55:54.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.877+0000 7f2c74602640 1 -- 172.21.15.40:0/1956336478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c6c13c580 msgr2=0x7f2c6c15a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:54.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.877+0000 7f2c74602640 1 --2- 172.21.15.40:0/1956336478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c6c13c580 0x7f2c6c15a270 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c0ce000 tx=0x7f2c6802c250 comp rx=0 tx=0).stop 2023-10-18T08:55:54.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.877+0000 7f2c74602640 1 -- 172.21.15.40:0/1956336478 shutdown_connections 2023-10-18T08:55:54.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.877+0000 7f2c74602640 1 --2- 172.21.15.40:0/1956336478 >> v2:172.21.15.144:3300/0 conn(0x7f2c6c0708e0 0x7f2c6c070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.877+0000 7f2c74602640 1 --2- 172.21.15.40:0/1956336478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c6c0aed00 0x7f2c6c0af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.877+0000 7f2c74602640 1 --2- 172.21.15.40:0/1956336478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c6c13c580 0x7f2c6c15a270 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.877+0000 7f2c74602640 1 -- 172.21.15.40:0/1956336478 >> 172.21.15.40:0/1956336478 conn(0x7f2c6c069720 msgr2=0x7f2c6c069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:54.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.877+0000 7f2c74602640 1 -- 172.21.15.40:0/1956336478 shutdown_connections 2023-10-18T08:55:54.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.877+0000 7f2c74602640 1 -- 172.21.15.40:0/1956336478 wait complete. 2023-10-18T08:55:54.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.880+0000 7f2c74602640 1 Processor -- start 2023-10-18T08:55:54.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.880+0000 7f2c74602640 1 -- start start 2023-10-18T08:55:54.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.881+0000 7f2c74602640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c6c0708e0 0x7f2c6c139070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.881+0000 7f2c74602640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c6c0aed00 0x7f2c6c15ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.881+0000 7f2c74602640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c6c13c580 0x7f2c6c15b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.881+0000 7f2c74602640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c138be0 con 0x7f2c6c0708e0 2023-10-18T08:55:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.881+0000 7f2c74602640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c1395b0 con 0x7f2c6c13c580 2023-10-18T08:55:54.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.881+0000 7f2c74602640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c137fd0 con 0x7f2c6c0aed00 2023-10-18T08:55:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.881+0000 7f2c7239e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c6c0708e0 0x7f2c6c139070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.881+0000 7f2c7239e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c6c0708e0 0x7f2c6c139070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39330/0 (socket says 172.21.15.40:39330) 2023-10-18T08:55:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.881+0000 7f2c7239e640 1 -- 172.21.15.40:0/997714821 learned_addr learned my addr 172.21.15.40:0/997714821 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.882+0000 7f2c72b9f640 1 --2- 172.21.15.40:0/997714821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c6c13c580 0x7f2c6c15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:54.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.882+0000 7f2c71b9d640 1 --2- 172.21.15.40:0/997714821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c6c0aed00 0x7f2c6c15ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:54.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.882+0000 7f2c7239e640 1 -- 172.21.15.40:0/997714821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c6c0aed00 msgr2=0x7f2c6c15ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:54.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.882+0000 7f2c7239e640 1 --2- 172.21.15.40:0/997714821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c6c0aed00 0x7f2c6c15ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.882+0000 7f2c7239e640 1 -- 172.21.15.40:0/997714821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c6c13c580 msgr2=0x7f2c6c15b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:54.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.882+0000 7f2c7239e640 1 --2- 172.21.15.40:0/997714821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c6c13c580 0x7f2c6c15b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:54.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.882+0000 7f2c7239e640 1 -- 172.21.15.40:0/997714821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c68009900 con 0x7f2c6c0708e0 2023-10-18T08:55:54.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.882+0000 7f2c7239e640 1 --2- 172.21.15.40:0/997714821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c6c0708e0 0x7f2c6c139070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2c68000f80 tx=0x7f2c68035d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:54.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.883+0000 7f2c62ffd640 1 -- 172.21.15.40:0/997714821 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68034440 con 0x7f2c6c0708e0 2023-10-18T08:55:54.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.883+0000 7f2c62ffd640 1 -- 172.21.15.40:0/997714821 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2c68003e60 con 0x7f2c6c0708e0 2023-10-18T08:55:54.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.883+0000 7f2c62ffd640 1 -- 172.21.15.40:0/997714821 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c680044b0 con 0x7f2c6c0708e0 2023-10-18T08:55:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.883+0000 7f2c74602640 1 -- 172.21.15.40:0/997714821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6c1382b0 con 0x7f2c6c0708e0 2023-10-18T08:55:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.883+0000 7f2c74602640 1 -- 172.21.15.40:0/997714821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6c138870 con 0x7f2c6c0708e0 2023-10-18T08:55:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.884+0000 7f2c74602640 1 -- 172.21.15.40:0/997714821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6c138a00 con 0x7f2c6c0708e0 2023-10-18T08:55:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.885+0000 7f2c62ffd640 1 -- 172.21.15.40:0/997714821 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2c6803b040 con 0x7f2c6c0708e0 2023-10-18T08:55:54.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.886+0000 7f2c62ffd640 1 --2- 172.21.15.40:0/997714821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2c5006ae40 0x7f2c5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:54.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.886+0000 7f2c62ffd640 1 -- 172.21.15.40:0/997714821 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c680ac100 con 0x7f2c6c0708e0 2023-10-18T08:55:54.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.887+0000 7f2c71b9d640 1 --2- 172.21.15.40:0/997714821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2c5006ae40 0x7f2c5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:54.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.888+0000 7f2c71b9d640 1 --2- 172.21.15.40:0/997714821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2c5006ae40 0x7f2c5006d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c153190 tx=0x7f2c5c005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:54.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:54.891+0000 7f2c62ffd640 1 -- 172.21.15.40:0/997714821 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c680771a0 con 0x7f2c6c0708e0 2023-10-18T08:55:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.085+0000 7f2c74602640 1 -- 172.21.15.40:0/997714821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c6c066fd0 con 0x7f2c6c0708e0 2023-10-18T08:55:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.086+0000 7f2c62ffd640 1 -- 172.21.15.40:0/997714821 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c6807a7c0 con 0x7f2c6c0708e0 2023-10-18T08:55:55.089 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:55:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.089+0000 7f2c74602640 1 -- 172.21.15.40:0/997714821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2c5006ae40 msgr2=0x7f2c5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.089+0000 7f2c74602640 1 --2- 172.21.15.40:0/997714821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2c5006ae40 0x7f2c5006d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c153190 tx=0x7f2c5c005f90 comp rx=0 tx=0).stop 2023-10-18T08:55:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.089+0000 7f2c74602640 1 -- 172.21.15.40:0/997714821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c6c0708e0 msgr2=0x7f2c6c139070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.089+0000 7f2c74602640 1 --2- 172.21.15.40:0/997714821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c6c0708e0 0x7f2c6c139070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2c68000f80 tx=0x7f2c68035d80 comp rx=0 tx=0).stop 2023-10-18T08:55:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.089+0000 7f2c74602640 1 -- 172.21.15.40:0/997714821 shutdown_connections 2023-10-18T08:55:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.089+0000 7f2c74602640 1 --2- 172.21.15.40:0/997714821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2c5006ae40 0x7f2c5006d300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.089+0000 7f2c74602640 1 --2- 172.21.15.40:0/997714821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c6c13c580 0x7f2c6c15b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.089+0000 7f2c74602640 1 --2- 172.21.15.40:0/997714821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c6c0aed00 0x7f2c6c15ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.089+0000 7f2c74602640 1 --2- 172.21.15.40:0/997714821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c6c0708e0 0x7f2c6c139070 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.089+0000 7f2c74602640 1 -- 172.21.15.40:0/997714821 >> 172.21.15.40:0/997714821 conn(0x7f2c6c069720 msgr2=0x7f2c6c13bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.089+0000 7f2c74602640 1 -- 172.21.15.40:0/997714821 shutdown_connections 2023-10-18T08:55:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.089+0000 7f2c74602640 1 -- 172.21.15.40:0/997714821 wait complete. 2023-10-18T08:55:55.110 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:55:55.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.340+0000 7f0ef92d7640 1 -- 172.21.15.40:0/1761121572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef415af70 msgr2=0x7f0ef406c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:55.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.340+0000 7f0ef92d7640 1 --2- 172.21.15.40:0/1761121572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef415af70 0x7f0ef406c100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40aef60 tx=0x7f0ee802bd30 comp rx=0 tx=0).stop 2023-10-18T08:55:55.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.340+0000 7f0ef92d7640 1 -- 172.21.15.40:0/1761121572 shutdown_connections 2023-10-18T08:55:55.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.340+0000 7f0ef92d7640 1 --2- 172.21.15.40:0/1761121572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef415af70 0x7f0ef406c100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:55.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.340+0000 7f0ef92d7640 1 --2- 172.21.15.40:0/1761121572 >> v2:172.21.15.144:3300/0 conn(0x7f0ef40c9620 0x7f0ef406bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:55.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.340+0000 7f0ef92d7640 1 -- 172.21.15.40:0/1761121572 >> 172.21.15.40:0/1761121572 conn(0x7f0ef406ca10 msgr2=0x7f0ef406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:55.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.340+0000 7f0ef92d7640 1 -- 172.21.15.40:0/1761121572 shutdown_connections 2023-10-18T08:55:55.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.340+0000 7f0ef92d7640 1 -- 172.21.15.40:0/1761121572 wait complete. 2023-10-18T08:55:55.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.342+0000 7f0ef92d7640 1 Processor -- start 2023-10-18T08:55:55.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.342+0000 7f0ef92d7640 1 -- start start 2023-10-18T08:55:55.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.343+0000 7f0ef92d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ef4070b60 0x7f0ef41501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:55.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.343+0000 7f0ef92d7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ef40c9620 0x7f0ef40ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:55.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.343+0000 7f0ef92d7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef415af70 0x7f0ef40cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:55.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.343+0000 7f0ef92d7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4150b80 con 0x7f0ef415af70 2023-10-18T08:55:55.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.343+0000 7f0ef92d7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef414eef0 con 0x7f0ef4070b60 2023-10-18T08:55:55.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.343+0000 7f0ef92d7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef414f1d0 con 0x7f0ef40c9620 2023-10-18T08:55:55.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.343+0000 7f0ef8ad6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef415af70 0x7f0ef40cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:55.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.343+0000 7f0ef8ad6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef415af70 0x7f0ef40cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52378/0 (socket says 172.21.15.40:52378) 2023-10-18T08:55:55.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.343+0000 7f0ef8ad6640 1 -- 172.21.15.40:0/1221633541 learned_addr learned my addr 172.21.15.40:0/1221633541 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:55.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.343+0000 7f0ef37fe640 1 --2- 172.21.15.40:0/1221633541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ef40c9620 0x7f0ef40ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:55.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.344+0000 7f0ef8ad6640 1 -- 172.21.15.40:0/1221633541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ef40c9620 msgr2=0x7f0ef40ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:55.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.344+0000 7f0ef3fff640 1 --2- 172.21.15.40:0/1221633541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ef4070b60 0x7f0ef41501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:55.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.344+0000 7f0ef8ad6640 1 --2- 172.21.15.40:0/1221633541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ef40c9620 0x7f0ef40ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:55.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.344+0000 7f0ef8ad6640 1 -- 172.21.15.40:0/1221633541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ef4070b60 msgr2=0x7f0ef41501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:55.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.344+0000 7f0ef8ad6640 1 --2- 172.21.15.40:0/1221633541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ef4070b60 0x7f0ef41501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:55.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.344+0000 7f0ef8ad6640 1 -- 172.21.15.40:0/1221633541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee8009560 con 0x7f0ef415af70 2023-10-18T08:55:55.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.344+0000 7f0ef8ad6640 1 --2- 172.21.15.40:0/1221633541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef415af70 0x7f0ef40cd2d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00c780 tx=0x7f0eec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:55.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.344+0000 7f0ef0ff9640 1 -- 172.21.15.40:0/1221633541 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec004b60 con 0x7f0ef415af70 2023-10-18T08:55:55.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.344+0000 7f0ef0ff9640 1 -- 172.21.15.40:0/1221633541 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0eec00ce90 con 0x7f0ef415af70 2023-10-18T08:55:55.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.344+0000 7f0ef92d7640 1 -- 172.21.15.40:0/1221633541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef414f4b0 con 0x7f0ef415af70 2023-10-18T08:55:55.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.344+0000 7f0ef0ff9640 1 -- 172.21.15.40:0/1221633541 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec00aab0 con 0x7f0ef415af70 2023-10-18T08:55:55.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.344+0000 7f0ef92d7640 1 -- 172.21.15.40:0/1221633541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef414fcb0 con 0x7f0ef415af70 2023-10-18T08:55:55.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.346+0000 7f0ef92d7640 1 -- 172.21.15.40:0/1221633541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef414f9d0 con 0x7f0ef415af70 2023-10-18T08:55:55.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.347+0000 7f0ef0ff9640 1 -- 172.21.15.40:0/1221633541 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0eec00ac10 con 0x7f0ef415af70 2023-10-18T08:55:55.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.347+0000 7f0ef0ff9640 1 --2- 172.21.15.40:0/1221633541 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ee006ae70 0x7f0ee006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:55.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.347+0000 7f0ef0ff9640 1 -- 172.21.15.40:0/1221633541 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0eec089940 con 0x7f0ef415af70 2023-10-18T08:55:55.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.348+0000 7f0ef3fff640 1 --2- 172.21.15.40:0/1221633541 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ee006ae70 0x7f0ee006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:55.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.349+0000 7f0ef3fff640 1 --2- 172.21.15.40:0/1221633541 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ee006ae70 0x7f0ee006d330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4005fd0 tx=0x7f0ee4005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:55.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.352+0000 7f0ef0ff9640 1 -- 172.21.15.40:0/1221633541 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ef414f9d0 con 0x7f0ef415af70 2023-10-18T08:55:55.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.544+0000 7f0ef92d7640 1 -- 172.21.15.40:0/1221633541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ef414f9d0 con 0x7f0ef415af70 2023-10-18T08:55:55.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.544+0000 7f0ef0ff9640 1 -- 172.21.15.40:0/1221633541 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0ef414f9d0 con 0x7f0ef415af70 2023-10-18T08:55:55.547 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:55:55.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.547+0000 7f0ef92d7640 1 -- 172.21.15.40:0/1221633541 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ee006ae70 msgr2=0x7f0ee006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:55.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.547+0000 7f0ef92d7640 1 --2- 172.21.15.40:0/1221633541 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ee006ae70 0x7f0ee006d330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4005fd0 tx=0x7f0ee4005850 comp rx=0 tx=0).stop 2023-10-18T08:55:55.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.547+0000 7f0ef92d7640 1 -- 172.21.15.40:0/1221633541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef415af70 msgr2=0x7f0ef40cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:55.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.547+0000 7f0ef92d7640 1 --2- 172.21.15.40:0/1221633541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef415af70 0x7f0ef40cd2d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00c780 tx=0x7f0eec00cc40 comp rx=0 tx=0).stop 2023-10-18T08:55:55.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.547+0000 7f0ef92d7640 1 -- 172.21.15.40:0/1221633541 shutdown_connections 2023-10-18T08:55:55.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.547+0000 7f0ef92d7640 1 --2- 172.21.15.40:0/1221633541 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ee006ae70 0x7f0ee006d330 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:55.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.547+0000 7f0ef92d7640 1 --2- 172.21.15.40:0/1221633541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ef415af70 0x7f0ef40cd2d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:55.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.547+0000 7f0ef92d7640 1 --2- 172.21.15.40:0/1221633541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ef40c9620 0x7f0ef40ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:55.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.547+0000 7f0ef92d7640 1 --2- 172.21.15.40:0/1221633541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ef4070b60 0x7f0ef41501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:55.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.547+0000 7f0ef92d7640 1 -- 172.21.15.40:0/1221633541 >> 172.21.15.40:0/1221633541 conn(0x7f0ef406ca10 msgr2=0x7f0ef414e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:55.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.548+0000 7f0ef92d7640 1 -- 172.21.15.40:0/1221633541 shutdown_connections 2023-10-18T08:55:55.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.548+0000 7f0ef92d7640 1 -- 172.21.15.40:0/1221633541 wait complete. 2023-10-18T08:55:55.569 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:55:55.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.791+0000 7fb1a10c3640 1 -- 172.21.15.40:0/2031970732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb19c0ae670 msgr2=0x7fb19c0c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:55.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.791+0000 7fb1a10c3640 1 --2- 172.21.15.40:0/2031970732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb19c0ae670 0x7fb19c0c9650 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0ccbd0 tx=0x7fb18c02bd90 comp rx=0 tx=0).stop 2023-10-18T08:55:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.791+0000 7fb1a10c3640 1 -- 172.21.15.40:0/2031970732 shutdown_connections 2023-10-18T08:55:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.791+0000 7fb1a10c3640 1 --2- 172.21.15.40:0/2031970732 >> v2:172.21.15.144:3300/0 conn(0x7fb19c0af7a0 0x7fb19c0cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.791+0000 7fb1a10c3640 1 --2- 172.21.15.40:0/2031970732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb19c0ae670 0x7fb19c0c9650 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.791+0000 7fb1a10c3640 1 -- 172.21.15.40:0/2031970732 >> 172.21.15.40:0/2031970732 conn(0x7fb19c0cf000 msgr2=0x7fb19c0cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:55.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.791+0000 7fb1a10c3640 1 -- 172.21.15.40:0/2031970732 shutdown_connections 2023-10-18T08:55:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.791+0000 7fb1a10c3640 1 -- 172.21.15.40:0/2031970732 wait complete. 2023-10-18T08:55:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.793+0000 7fb1a10c3640 1 Processor -- start 2023-10-18T08:55:55.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.793+0000 7fb1a10c3640 1 -- start start 2023-10-18T08:55:55.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.794+0000 7fb1a10c3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb19c0ae670 0x7fb19c0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.794+0000 7fb1a10c3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb19c0af7a0 0x7fb19c1f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.794+0000 7fb1a10c3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb19c0cb880 0x7fb19c1f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.794+0000 7fb1a10c3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c1f2de0 con 0x7fb19c0ae670 2023-10-18T08:55:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.794+0000 7fb1a10c3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c1f6340 con 0x7fb19c0af7a0 2023-10-18T08:55:55.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.794+0000 7fb1a10c3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c1f1de0 con 0x7fb19c0cb880 2023-10-18T08:55:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.794+0000 7fb19bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb19c0ae670 0x7fb19c0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.794+0000 7fb19bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb19c0ae670 0x7fb19c0b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52390/0 (socket says 172.21.15.40:52390) 2023-10-18T08:55:55.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.794+0000 7fb19bfff640 1 -- 172.21.15.40:0/2432082941 learned_addr learned my addr 172.21.15.40:0/2432082941 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:55.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.795+0000 7fb19bfff640 1 -- 172.21.15.40:0/2432082941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb19c0cb880 msgr2=0x7fb19c1f5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:55.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.795+0000 7fb19bfff640 1 --2- 172.21.15.40:0/2432082941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb19c0cb880 0x7fb19c1f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:55.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.795+0000 7fb19bfff640 1 -- 172.21.15.40:0/2432082941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb19c0af7a0 msgr2=0x7fb19c1f28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:55.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.795+0000 7fb193fff640 1 --2- 172.21.15.40:0/2432082941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb19c0af7a0 0x7fb19c1f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:55.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.795+0000 7fb19bfff640 1 --2- 172.21.15.40:0/2432082941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb19c0af7a0 0x7fb19c1f28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.795+0000 7fb19bfff640 1 -- 172.21.15.40:0/2432082941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c009560 con 0x7fb19c0ae670 2023-10-18T08:55:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.795+0000 7fb19bfff640 1 --2- 172.21.15.40:0/2432082941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb19c0ae670 0x7fb19c0b2d50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009850 tx=0x7fb18c02db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.795+0000 7fb1997fa640 1 -- 172.21.15.40:0/2432082941 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c0035f0 con 0x7fb19c0ae670 2023-10-18T08:55:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.795+0000 7fb1997fa640 1 -- 172.21.15.40:0/2432082941 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb18c003750 con 0x7fb19c0ae670 2023-10-18T08:55:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.795+0000 7fb1a10c3640 1 -- 172.21.15.40:0/2432082941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19c0b33f0 con 0x7fb19c0ae670 2023-10-18T08:55:55.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.795+0000 7fb1997fa640 1 -- 172.21.15.40:0/2432082941 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18c003b00 con 0x7fb19c0ae670 2023-10-18T08:55:55.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.796+0000 7fb1a10c3640 1 -- 172.21.15.40:0/2432082941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19c1f20a0 con 0x7fb19c0ae670 2023-10-18T08:55:55.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.797+0000 7fb1a10c3640 1 -- 172.21.15.40:0/2432082941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19c066e00 con 0x7fb19c0ae670 2023-10-18T08:55:55.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.798+0000 7fb1997fa640 1 -- 172.21.15.40:0/2432082941 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb18c003c60 con 0x7fb19c0ae670 2023-10-18T08:55:55.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.799+0000 7fb1997fa640 1 --2- 172.21.15.40:0/2432082941 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb17c06ae40 0x7fb17c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:55.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.799+0000 7fb1997fa640 1 -- 172.21.15.40:0/2432082941 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb18c0ac240 con 0x7fb19c0ae670 2023-10-18T08:55:55.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.799+0000 7fb193fff640 1 --2- 172.21.15.40:0/2432082941 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb17c06ae40 0x7fb17c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:55.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.800+0000 7fb193fff640 1 --2- 172.21.15.40:0/2432082941 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb17c06ae40 0x7fb17c06d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0c3f40 tx=0x7fb184008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:55.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:55.803+0000 7fb1997fa640 1 -- 172.21.15.40:0/2432082941 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb18c077380 con 0x7fb19c0ae670 2023-10-18T08:55:56.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.000+0000 7fb1a10c3640 1 -- 172.21.15.40:0/2432082941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb19c0b38b0 con 0x7fb19c0ae670 2023-10-18T08:55:56.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.001+0000 7fb1997fa640 1 -- 172.21.15.40:0/2432082941 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb18c07a9a0 con 0x7fb19c0ae670 2023-10-18T08:55:56.003 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:55:56.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.004+0000 7fb1a10c3640 1 -- 172.21.15.40:0/2432082941 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb17c06ae40 msgr2=0x7fb17c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:56.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.004+0000 7fb1a10c3640 1 --2- 172.21.15.40:0/2432082941 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb17c06ae40 0x7fb17c06d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0c3f40 tx=0x7fb184008040 comp rx=0 tx=0).stop 2023-10-18T08:55:56.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.004+0000 7fb1a10c3640 1 -- 172.21.15.40:0/2432082941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb19c0ae670 msgr2=0x7fb19c0b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:56.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.004+0000 7fb1a10c3640 1 --2- 172.21.15.40:0/2432082941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb19c0ae670 0x7fb19c0b2d50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009850 tx=0x7fb18c02db10 comp rx=0 tx=0).stop 2023-10-18T08:55:56.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.004+0000 7fb1a10c3640 1 -- 172.21.15.40:0/2432082941 shutdown_connections 2023-10-18T08:55:56.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.004+0000 7fb1a10c3640 1 --2- 172.21.15.40:0/2432082941 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb17c06ae40 0x7fb17c06d300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.004+0000 7fb1a10c3640 1 --2- 172.21.15.40:0/2432082941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb19c0cb880 0x7fb19c1f5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.004+0000 7fb1a10c3640 1 --2- 172.21.15.40:0/2432082941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb19c0af7a0 0x7fb19c1f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.004+0000 7fb1a10c3640 1 --2- 172.21.15.40:0/2432082941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb19c0ae670 0x7fb19c0b2d50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.004+0000 7fb1a10c3640 1 -- 172.21.15.40:0/2432082941 >> 172.21.15.40:0/2432082941 conn(0x7fb19c0cf000 msgr2=0x7fb19c15e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:56.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.004+0000 7fb1a10c3640 1 -- 172.21.15.40:0/2432082941 shutdown_connections 2023-10-18T08:55:56.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.004+0000 7fb1a10c3640 1 -- 172.21.15.40:0/2432082941 wait complete. 2023-10-18T08:55:56.026 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:56.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.245+0000 7f30debae640 1 -- 172.21.15.40:0/1218550698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30d80b0630 msgr2=0x7f30d80c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:56.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.245+0000 7f30debae640 1 --2- 172.21.15.40:0/1218550698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30d80b0630 0x7f30d80c99c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f30d806b050 tx=0x7f30c802bd90 comp rx=0 tx=0).stop 2023-10-18T08:55:56.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.245+0000 7f30debae640 1 -- 172.21.15.40:0/1218550698 shutdown_connections 2023-10-18T08:55:56.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.245+0000 7f30debae640 1 --2- 172.21.15.40:0/1218550698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30d80b0630 0x7f30d80c99c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.245+0000 7f30debae640 1 --2- 172.21.15.40:0/1218550698 >> v2:172.21.15.144:3300/0 conn(0x7f30d806d6f0 0x7f30d80b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.245+0000 7f30debae640 1 -- 172.21.15.40:0/1218550698 >> 172.21.15.40:0/1218550698 conn(0x7f30d8069720 msgr2=0x7f30d8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:56.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.245+0000 7f30debae640 1 -- 172.21.15.40:0/1218550698 shutdown_connections 2023-10-18T08:55:56.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.245+0000 7f30debae640 1 -- 172.21.15.40:0/1218550698 wait complete. 2023-10-18T08:55:56.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.247+0000 7f30debae640 1 Processor -- start 2023-10-18T08:55:56.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.247+0000 7f30debae640 1 -- start start 2023-10-18T08:55:56.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.248+0000 7f30debae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30d806d6f0 0x7f30d81f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:56.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.248+0000 7f30debae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30d80b0630 0x7f30d8151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:56.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.248+0000 7f30debae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30d80c9f00 0x7f30d81e3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:56.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.248+0000 7f30debae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d8153390 con 0x7f30d80c9f00 2023-10-18T08:55:56.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.248+0000 7f30debae640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d81f74d0 con 0x7f30d806d6f0 2023-10-18T08:55:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.248+0000 7f30debae640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d8152040 con 0x7f30d80b0630 2023-10-18T08:55:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.249+0000 7f30de3ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30d80c9f00 0x7f30d81e3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.249+0000 7f30de3ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30d80c9f00 0x7f30d81e3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52408/0 (socket says 172.21.15.40:52408) 2023-10-18T08:55:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.249+0000 7f30de3ad640 1 -- 172.21.15.40:0/2613958791 learned_addr learned my addr 172.21.15.40:0/2613958791 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.249+0000 7f30ddbac640 1 --2- 172.21.15.40:0/2613958791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30d806d6f0 0x7f30d81f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:56.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.249+0000 7f30dd3ab640 1 --2- 172.21.15.40:0/2613958791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30d80b0630 0x7f30d8151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.249+0000 7f30de3ad640 1 -- 172.21.15.40:0/2613958791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30d80b0630 msgr2=0x7f30d8151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.249+0000 7f30de3ad640 1 --2- 172.21.15.40:0/2613958791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30d80b0630 0x7f30d8151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.249+0000 7f30de3ad640 1 -- 172.21.15.40:0/2613958791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30d806d6f0 msgr2=0x7f30d81f6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.249+0000 7f30de3ad640 1 --2- 172.21.15.40:0/2613958791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30d806d6f0 0x7f30d81f6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.249+0000 7f30de3ad640 1 -- 172.21.15.40:0/2613958791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c8009560 con 0x7f30d80c9f00 2023-10-18T08:55:56.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.250+0000 7f30de3ad640 1 --2- 172.21.15.40:0/2613958791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30d80c9f00 0x7f30d81e3790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f30d000b4c0 tx=0x7f30d000b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:56.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.250+0000 7f30ce7fc640 1 -- 172.21.15.40:0/2613958791 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0011830 con 0x7f30d80c9f00 2023-10-18T08:55:56.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.250+0000 7f30ce7fc640 1 -- 172.21.15.40:0/2613958791 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f30d0011aa0 con 0x7f30d80c9f00 2023-10-18T08:55:56.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.250+0000 7f30ce7fc640 1 -- 172.21.15.40:0/2613958791 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0010380 con 0x7f30d80c9f00 2023-10-18T08:55:56.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.250+0000 7f30debae640 1 -- 172.21.15.40:0/2613958791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d8152730 con 0x7f30d80c9f00 2023-10-18T08:55:56.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.250+0000 7f30debae640 1 -- 172.21.15.40:0/2613958791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d8152cd0 con 0x7f30d80c9f00 2023-10-18T08:55:56.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.251+0000 7f30debae640 1 -- 172.21.15.40:0/2613958791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d81534d0 con 0x7f30d80c9f00 2023-10-18T08:55:56.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.252+0000 7f30ce7fc640 1 -- 172.21.15.40:0/2613958791 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f30d00104e0 con 0x7f30d80c9f00 2023-10-18T08:55:56.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.253+0000 7f30ce7fc640 1 --2- 172.21.15.40:0/2613958791 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f30bc06ae40 0x7f30bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:56.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.253+0000 7f30ce7fc640 1 -- 172.21.15.40:0/2613958791 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30d0089970 con 0x7f30d80c9f00 2023-10-18T08:55:56.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.254+0000 7f30ddbac640 1 --2- 172.21.15.40:0/2613958791 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f30bc06ae40 0x7f30bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:56.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.255+0000 7f30ddbac640 1 --2- 172.21.15.40:0/2613958791 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f30bc06ae40 0x7f30bc06d300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009f20 tx=0x7f30d40094f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:56.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.258+0000 7f30ce7fc640 1 -- 172.21.15.40:0/2613958791 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f30d0054a10 con 0x7f30d80c9f00 2023-10-18T08:55:56.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.456+0000 7f30debae640 1 -- 172.21.15.40:0/2613958791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30d8066e00 con 0x7f30d80c9f00 2023-10-18T08:55:56.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.459+0000 7f30ce7fc640 1 -- 172.21.15.40:0/2613958791 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f30d0015090 con 0x7f30d80c9f00 2023-10-18T08:55:56.462 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:56.462 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:16.275677+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:56.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.461+0000 7f30debae640 1 -- 172.21.15.40:0/2613958791 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f30bc06ae40 msgr2=0x7f30bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:56.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.461+0000 7f30debae640 1 --2- 172.21.15.40:0/2613958791 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f30bc06ae40 0x7f30bc06d300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009f20 tx=0x7f30d40094f0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.461+0000 7f30debae640 1 -- 172.21.15.40:0/2613958791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30d80c9f00 msgr2=0x7f30d81e3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:56.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.461+0000 7f30debae640 1 --2- 172.21.15.40:0/2613958791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30d80c9f00 0x7f30d81e3790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f30d000b4c0 tx=0x7f30d000b980 comp rx=0 tx=0).stop 2023-10-18T08:55:56.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.461+0000 7f30debae640 1 -- 172.21.15.40:0/2613958791 shutdown_connections 2023-10-18T08:55:56.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.461+0000 7f30debae640 1 --2- 172.21.15.40:0/2613958791 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f30bc06ae40 0x7f30bc06d300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.461+0000 7f30debae640 1 --2- 172.21.15.40:0/2613958791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30d80c9f00 0x7f30d81e3790 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.461+0000 7f30debae640 1 --2- 172.21.15.40:0/2613958791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30d80b0630 0x7f30d8151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.461+0000 7f30debae640 1 --2- 172.21.15.40:0/2613958791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30d806d6f0 0x7f30d81f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.461+0000 7f30debae640 1 -- 172.21.15.40:0/2613958791 >> 172.21.15.40:0/2613958791 conn(0x7f30d8069720 msgr2=0x7f30d80cc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:56.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.462+0000 7f30debae640 1 -- 172.21.15.40:0/2613958791 shutdown_connections 2023-10-18T08:55:56.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.462+0000 7f30debae640 1 -- 172.21.15.40:0/2613958791 wait complete. 2023-10-18T08:55:56.484 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:55:13.210788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:55:13.772849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:55:56.484 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:55:56.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.719+0000 7efcc555e640 1 -- 172.21.15.40:0/2472474246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcc00afa40 msgr2=0x7efcc0158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:56.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.719+0000 7efcc555e640 1 --2- 172.21.15.40:0/2472474246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcc00afa40 0x7efcc0158b10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efca8009830 tx=0x7efca802bd50 comp rx=0 tx=0).stop 2023-10-18T08:55:56.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.719+0000 7efcc555e640 1 -- 172.21.15.40:0/2472474246 shutdown_connections 2023-10-18T08:55:56.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.719+0000 7efcc555e640 1 --2- 172.21.15.40:0/2472474246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcc00afa40 0x7efcc0158b10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.719+0000 7efcc555e640 1 --2- 172.21.15.40:0/2472474246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcc00cd610 0x7efcc00cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.719+0000 7efcc555e640 1 -- 172.21.15.40:0/2472474246 >> 172.21.15.40:0/2472474246 conn(0x7efcc0069720 msgr2=0x7efcc0069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:56.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.719+0000 7efcc555e640 1 -- 172.21.15.40:0/2472474246 shutdown_connections 2023-10-18T08:55:56.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.719+0000 7efcc555e640 1 -- 172.21.15.40:0/2472474246 wait complete. 2023-10-18T08:55:56.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.721+0000 7efcc555e640 1 Processor -- start 2023-10-18T08:55:56.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.721+0000 7efcc555e640 1 -- start start 2023-10-18T08:55:56.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.722+0000 7efcc555e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcc00afa40 0x7efcc014ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:56.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.722+0000 7efcc555e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcc00caf40 0x7efcc014b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:56.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.722+0000 7efcc555e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcc00cd610 0x7efcc00ae330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:56.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.722+0000 7efcc555e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc0149de0 con 0x7efcc00afa40 2023-10-18T08:55:56.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.722+0000 7efcc555e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc014b370 con 0x7efcc00caf40 2023-10-18T08:55:56.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.722+0000 7efcc555e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc014d3d0 con 0x7efcc00cd610 2023-10-18T08:55:56.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.722+0000 7efcbeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcc00afa40 0x7efcc014ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:56.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.722+0000 7efcbeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcc00afa40 0x7efcc014ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52420/0 (socket says 172.21.15.40:52420) 2023-10-18T08:55:56.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.722+0000 7efcbeffd640 1 -- 172.21.15.40:0/4152529025 learned_addr learned my addr 172.21.15.40:0/4152529025 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:56.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.723+0000 7efcbeffd640 1 -- 172.21.15.40:0/4152529025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcc00cd610 msgr2=0x7efcc00ae330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:56.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.723+0000 7efcbeffd640 1 --2- 172.21.15.40:0/4152529025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcc00cd610 0x7efcc00ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.723+0000 7efcbeffd640 1 -- 172.21.15.40:0/4152529025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcc00caf40 msgr2=0x7efcc014b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:56.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.723+0000 7efcbeffd640 1 --2- 172.21.15.40:0/4152529025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcc00caf40 0x7efcc014b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.723+0000 7efcbeffd640 1 -- 172.21.15.40:0/4152529025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca8009490 con 0x7efcc00afa40 2023-10-18T08:55:56.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.723+0000 7efcbeffd640 1 --2- 172.21.15.40:0/4152529025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcc00afa40 0x7efcc014ce90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efcb000ea20 tx=0x7efcb000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:56.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.723+0000 7efcb77fe640 1 -- 172.21.15.40:0/4152529025 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb000cde0 con 0x7efcc00afa40 2023-10-18T08:55:56.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.723+0000 7efcb77fe640 1 -- 172.21.15.40:0/4152529025 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efcb0011560 con 0x7efcc00afa40 2023-10-18T08:55:56.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.723+0000 7efcc555e640 1 -- 172.21.15.40:0/4152529025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc0149540 con 0x7efcc00afa40 2023-10-18T08:55:56.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.723+0000 7efcb77fe640 1 -- 172.21.15.40:0/4152529025 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb0011950 con 0x7efcc00afa40 2023-10-18T08:55:56.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.723+0000 7efcc555e640 1 -- 172.21.15.40:0/4152529025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc0149b00 con 0x7efcc00afa40 2023-10-18T08:55:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.724+0000 7efcc555e640 1 -- 172.21.15.40:0/4152529025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc014b140 con 0x7efcc00afa40 2023-10-18T08:55:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.726+0000 7efcb77fe640 1 -- 172.21.15.40:0/4152529025 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efcb0011ab0 con 0x7efcc00afa40 2023-10-18T08:55:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.726+0000 7efcb77fe640 1 --2- 172.21.15.40:0/4152529025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efca006ae40 0x7efca006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.727+0000 7efcb77fe640 1 -- 172.21.15.40:0/4152529025 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efcb008a340 con 0x7efcc00afa40 2023-10-18T08:55:56.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.727+0000 7efcbe7fc640 1 --2- 172.21.15.40:0/4152529025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efca006ae40 0x7efca006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:56.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.728+0000 7efcbe7fc640 1 --2- 172.21.15.40:0/4152529025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efca006ae40 0x7efca006d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efca8036000 tx=0x7efca80046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:56.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.731+0000 7efcb77fe640 1 -- 172.21.15.40:0/4152529025 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efcc014b140 con 0x7efcc00afa40 2023-10-18T08:55:56.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.924+0000 7efcc555e640 1 -- 172.21.15.40:0/4152529025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efcc0066fd0 con 0x7efcc00afa40 2023-10-18T08:55:56.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.925+0000 7efcb77fe640 1 -- 172.21.15.40:0/4152529025 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7efcb001f020 con 0x7efcc00afa40 2023-10-18T08:55:56.927 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:55:56.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.928+0000 7efcc555e640 1 -- 172.21.15.40:0/4152529025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efca006ae40 msgr2=0x7efca006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:56.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.928+0000 7efcc555e640 1 --2- 172.21.15.40:0/4152529025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efca006ae40 0x7efca006d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efca8036000 tx=0x7efca80046c0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.928+0000 7efcc555e640 1 -- 172.21.15.40:0/4152529025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcc00afa40 msgr2=0x7efcc014ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:56.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.928+0000 7efcc555e640 1 --2- 172.21.15.40:0/4152529025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcc00afa40 0x7efcc014ce90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efcb000ea20 tx=0x7efcb000eee0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.928+0000 7efcc555e640 1 -- 172.21.15.40:0/4152529025 shutdown_connections 2023-10-18T08:55:56.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.928+0000 7efcc555e640 1 --2- 172.21.15.40:0/4152529025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efca006ae40 0x7efca006d300 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.928+0000 7efcc555e640 1 --2- 172.21.15.40:0/4152529025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcc00cd610 0x7efcc00ae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.928+0000 7efcc555e640 1 --2- 172.21.15.40:0/4152529025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efcc00caf40 0x7efcc014b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.928+0000 7efcc555e640 1 --2- 172.21.15.40:0/4152529025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efcc00afa40 0x7efcc014ce90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:56.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.928+0000 7efcc555e640 1 -- 172.21.15.40:0/4152529025 >> 172.21.15.40:0/4152529025 conn(0x7efcc0069720 msgr2=0x7efcc006b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:56.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.928+0000 7efcc555e640 1 -- 172.21.15.40:0/4152529025 shutdown_connections 2023-10-18T08:55:56.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:56.928+0000 7efcc555e640 1 -- 172.21.15.40:0/4152529025 wait complete. 2023-10-18T08:55:56.949 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:55:57.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.186+0000 7f6ec6e90640 1 -- 172.21.15.40:0/1664764755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ec006ff80 msgr2=0x7f6ec00cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:57.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.186+0000 7f6ec6e90640 1 --2- 172.21.15.40:0/1664764755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ec006ff80 0x7f6ec00cb290 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6ec00af400 tx=0x7f6eb802c2f0 comp rx=0 tx=0).stop 2023-10-18T08:55:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.187+0000 7f6ec6e90640 1 -- 172.21.15.40:0/1664764755 shutdown_connections 2023-10-18T08:55:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.187+0000 7f6ec6e90640 1 --2- 172.21.15.40:0/1664764755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ec006ff80 0x7f6ec00cb290 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.187+0000 7f6ec6e90640 1 --2- 172.21.15.40:0/1664764755 >> v2:172.21.15.144:3300/0 conn(0x7f6ec0155d00 0x7f6ec006bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.187+0000 7f6ec6e90640 1 -- 172.21.15.40:0/1664764755 >> 172.21.15.40:0/1664764755 conn(0x7f6ec006ca10 msgr2=0x7f6ec006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:57.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.187+0000 7f6ec6e90640 1 -- 172.21.15.40:0/1664764755 shutdown_connections 2023-10-18T08:55:57.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.187+0000 7f6ec6e90640 1 -- 172.21.15.40:0/1664764755 wait complete. 2023-10-18T08:55:57.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.188+0000 7f6ec6e90640 1 Processor -- start 2023-10-18T08:55:57.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.189+0000 7f6ec6e90640 1 -- start start 2023-10-18T08:55:57.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.190+0000 7f6ec6e90640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ec006c090 0x7f6ec0146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:57.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.190+0000 7f6ec6e90640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ec006ff80 0x7f6ec0154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:57.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.190+0000 7f6ec6e90640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ec0155d00 0x7f6ec01577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:57.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.190+0000 7f6ec6e90640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec01522f0 con 0x7f6ec006ff80 2023-10-18T08:55:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.190+0000 7f6ec6e90640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec01525d0 con 0x7f6ec006c090 2023-10-18T08:55:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.190+0000 7f6ec6e90640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec01528b0 con 0x7f6ec0155d00 2023-10-18T08:55:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.190+0000 7f6ec568d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ec006ff80 0x7f6ec0154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.190+0000 7f6ec568d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ec006ff80 0x7f6ec0154d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52434/0 (socket says 172.21.15.40:52434) 2023-10-18T08:55:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.190+0000 7f6ec568d640 1 -- 172.21.15.40:0/1603377264 learned_addr learned my addr 172.21.15.40:0/1603377264 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.190+0000 7f6ec5e8e640 1 --2- 172.21.15.40:0/1603377264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ec006c090 0x7f6ec0146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.190+0000 7f6ec668f640 1 --2- 172.21.15.40:0/1603377264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ec0155d00 0x7f6ec01577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.191+0000 7f6ec568d640 1 -- 172.21.15.40:0/1603377264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ec0155d00 msgr2=0x7f6ec01577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.191+0000 7f6ec568d640 1 --2- 172.21.15.40:0/1603377264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ec0155d00 0x7f6ec01577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.191+0000 7f6ec568d640 1 -- 172.21.15.40:0/1603377264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ec006c090 msgr2=0x7f6ec0146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:57.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.191+0000 7f6ec568d640 1 --2- 172.21.15.40:0/1603377264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ec006c090 0x7f6ec0146cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:57.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.191+0000 7f6ec568d640 1 -- 172.21.15.40:0/1603377264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb8009900 con 0x7f6ec006ff80 2023-10-18T08:55:57.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.191+0000 7f6ec568d640 1 --2- 172.21.15.40:0/1603377264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ec006ff80 0x7f6ec0154d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000c4b0 tx=0x7f6eb000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.191+0000 7f6eb67fc640 1 -- 172.21.15.40:0/1603377264 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0005480 con 0x7f6ec006ff80 2023-10-18T08:55:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.191+0000 7f6eb67fc640 1 -- 172.21.15.40:0/1603377264 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6eb00036e0 con 0x7f6ec006ff80 2023-10-18T08:55:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.191+0000 7f6eb67fc640 1 -- 172.21.15.40:0/1603377264 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0003b30 con 0x7f6ec006ff80 2023-10-18T08:55:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.191+0000 7f6ec6e90640 1 -- 172.21.15.40:0/1603377264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec0159d80 con 0x7f6ec006ff80 2023-10-18T08:55:57.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.192+0000 7f6ec6e90640 1 -- 172.21.15.40:0/1603377264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec01552c0 con 0x7f6ec006ff80 2023-10-18T08:55:57.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.193+0000 7f6ec6e90640 1 -- 172.21.15.40:0/1603377264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec0155540 con 0x7f6ec006ff80 2023-10-18T08:55:57.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.194+0000 7f6eb67fc640 1 -- 172.21.15.40:0/1603377264 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6eb0003c90 con 0x7f6ec006ff80 2023-10-18T08:55:57.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.195+0000 7f6eb67fc640 1 --2- 172.21.15.40:0/1603377264 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ea406ae40 0x7f6ea406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:57.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.195+0000 7f6eb67fc640 1 -- 172.21.15.40:0/1603377264 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6eb0088ea0 con 0x7f6ec006ff80 2023-10-18T08:55:57.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.195+0000 7f6ec5e8e640 1 --2- 172.21.15.40:0/1603377264 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ea406ae40 0x7f6ea406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:57.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.196+0000 7f6ec5e8e640 1 --2- 172.21.15.40:0/1603377264 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ea406ae40 0x7f6ea406d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc005fd0 tx=0x7f6ebc005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.199+0000 7f6eb67fc640 1 -- 172.21.15.40:0/1603377264 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6eb0053eb0 con 0x7f6ec006ff80 2023-10-18T08:55:57.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.394+0000 7f6ec6e90640 1 -- 172.21.15.40:0/1603377264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ec00cdfb0 con 0x7f6ec006ff80 2023-10-18T08:55:57.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.395+0000 7f6eb67fc640 1 -- 172.21.15.40:0/1603377264 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6eb0017020 con 0x7f6ec006ff80 2023-10-18T08:55:57.398 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:55:57.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.398+0000 7f6ec6e90640 1 -- 172.21.15.40:0/1603377264 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ea406ae40 msgr2=0x7f6ea406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:57.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.398+0000 7f6ec6e90640 1 --2- 172.21.15.40:0/1603377264 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ea406ae40 0x7f6ea406d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc005fd0 tx=0x7f6ebc005850 comp rx=0 tx=0).stop 2023-10-18T08:55:57.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.398+0000 7f6ec6e90640 1 -- 172.21.15.40:0/1603377264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ec006ff80 msgr2=0x7f6ec0154d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:57.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.398+0000 7f6ec6e90640 1 --2- 172.21.15.40:0/1603377264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ec006ff80 0x7f6ec0154d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000c4b0 tx=0x7f6eb000c970 comp rx=0 tx=0).stop 2023-10-18T08:55:57.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.398+0000 7f6ec6e90640 1 -- 172.21.15.40:0/1603377264 shutdown_connections 2023-10-18T08:55:57.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.398+0000 7f6ec6e90640 1 --2- 172.21.15.40:0/1603377264 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ea406ae40 0x7f6ea406d300 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:57.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.398+0000 7f6ec6e90640 1 --2- 172.21.15.40:0/1603377264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ec0155d00 0x7f6ec01577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:57.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.398+0000 7f6ec6e90640 1 --2- 172.21.15.40:0/1603377264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ec006ff80 0x7f6ec0154d80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:57.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.398+0000 7f6ec6e90640 1 --2- 172.21.15.40:0/1603377264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ec006c090 0x7f6ec0146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:57.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.398+0000 7f6ec6e90640 1 -- 172.21.15.40:0/1603377264 >> 172.21.15.40:0/1603377264 conn(0x7f6ec006ca10 msgr2=0x7f6ec006f510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:57.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.398+0000 7f6ec6e90640 1 -- 172.21.15.40:0/1603377264 shutdown_connections 2023-10-18T08:55:57.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.398+0000 7f6ec6e90640 1 -- 172.21.15.40:0/1603377264 wait complete. 2023-10-18T08:55:57.420 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:55:57.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.652+0000 7f8788c13640 1 -- 172.21.15.40:0/3694928445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f878406d5d0 msgr2=0x7f87840c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:57.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.652+0000 7f8788c13640 1 --2- 172.21.15.40:0/3694928445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f878406d5d0 0x7f87840c9620 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8774005fd0 tx=0x7f877402bd90 comp rx=0 tx=0).stop 2023-10-18T08:55:57.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.653+0000 7f8788c13640 1 -- 172.21.15.40:0/3694928445 shutdown_connections 2023-10-18T08:55:57.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.653+0000 7f8788c13640 1 --2- 172.21.15.40:0/3694928445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f878406d5d0 0x7f87840c9620 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:57.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.653+0000 7f8788c13640 1 -- 172.21.15.40:0/3694928445 >> 172.21.15.40:0/3694928445 conn(0x7f878406ca10 msgr2=0x7f878406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:57.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.653+0000 7f8788c13640 1 -- 172.21.15.40:0/3694928445 shutdown_connections 2023-10-18T08:55:57.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.653+0000 7f8788c13640 1 -- 172.21.15.40:0/3694928445 wait complete. 2023-10-18T08:55:57.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.655+0000 7f8788c13640 1 Processor -- start 2023-10-18T08:55:57.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.655+0000 7f8788c13640 1 -- start start 2023-10-18T08:55:57.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.656+0000 7f8788c13640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f878406c2d0 0x7f87841e1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:57.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.656+0000 7f8788c13640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f878406d5d0 0x7f8784150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:57.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.656+0000 7f8788c13640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f878415af70 0x7f87841e27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:57.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.656+0000 7f8788c13640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87840c7710 con 0x7f878415af70 2023-10-18T08:55:57.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.656+0000 7f8788c13640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87841e24a0 con 0x7f878406c2d0 2023-10-18T08:55:57.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.656+0000 7f8788c13640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87840c6de0 con 0x7f878406d5d0 2023-10-18T08:55:57.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.656+0000 7f8783fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f878415af70 0x7f87841e27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:57.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.656+0000 7f8783fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f878415af70 0x7f87841e27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52440/0 (socket says 172.21.15.40:52440) 2023-10-18T08:55:57.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.656+0000 7f8783fff640 1 -- 172.21.15.40:0/176379188 learned_addr learned my addr 172.21.15.40:0/176379188 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:57.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.656+0000 7f87837fe640 1 --2- 172.21.15.40:0/176379188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f878406c2d0 0x7f87841e1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:57.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.657+0000 7f8783fff640 1 -- 172.21.15.40:0/176379188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f878406d5d0 msgr2=0x7f8784150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:57.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.657+0000 7f8782ffd640 1 --2- 172.21.15.40:0/176379188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f878406d5d0 0x7f8784150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:57.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.657+0000 7f8783fff640 1 --2- 172.21.15.40:0/176379188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f878406d5d0 0x7f8784150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:57.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.657+0000 7f8783fff640 1 -- 172.21.15.40:0/176379188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f878406c2d0 msgr2=0x7f87841e1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:57.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.657+0000 7f8783fff640 1 --2- 172.21.15.40:0/176379188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f878406c2d0 0x7f87841e1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:57.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.657+0000 7f8783fff640 1 -- 172.21.15.40:0/176379188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8774009560 con 0x7f878415af70 2023-10-18T08:55:57.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.657+0000 7f87837fe640 1 --2- 172.21.15.40:0/176379188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f878406c2d0 0x7f87841e1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:57.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.657+0000 7f8782ffd640 1 --2- 172.21.15.40:0/176379188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f878406d5d0 0x7f8784150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:55:57.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.657+0000 7f8783fff640 1 --2- 172.21.15.40:0/176379188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f878415af70 0x7f87841e27d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f877c00c780 tx=0x7f877c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:57.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.657+0000 7f876bfff640 1 -- 172.21.15.40:0/176379188 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c004b60 con 0x7f878415af70 2023-10-18T08:55:57.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.657+0000 7f876bfff640 1 -- 172.21.15.40:0/176379188 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f877c00ce90 con 0x7f878415af70 2023-10-18T08:55:57.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.657+0000 7f8788c13640 1 -- 172.21.15.40:0/176379188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87840c70c0 con 0x7f878415af70 2023-10-18T08:55:57.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.657+0000 7f876bfff640 1 -- 172.21.15.40:0/176379188 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c00aab0 con 0x7f878415af70 2023-10-18T08:55:57.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.657+0000 7f8788c13640 1 -- 172.21.15.40:0/176379188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8784150650 con 0x7f878415af70 2023-10-18T08:55:57.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.659+0000 7f8788c13640 1 -- 172.21.15.40:0/176379188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87840c7880 con 0x7f878415af70 2023-10-18T08:55:57.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.660+0000 7f876bfff640 1 -- 172.21.15.40:0/176379188 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f877c00ac10 con 0x7f878415af70 2023-10-18T08:55:57.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.660+0000 7f876bfff640 1 --2- 172.21.15.40:0/176379188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f877006ae40 0x7f877006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:57.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.661+0000 7f876bfff640 1 -- 172.21.15.40:0/176379188 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f877c089aa0 con 0x7f878415af70 2023-10-18T08:55:57.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.661+0000 7f87837fe640 1 --2- 172.21.15.40:0/176379188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f877006ae40 0x7f877006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:57.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.662+0000 7f87837fe640 1 --2- 172.21.15.40:0/176379188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f877006ae40 0x7f877006d300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8774035000 tx=0x7f8774004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:57.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.665+0000 7f876bfff640 1 -- 172.21.15.40:0/176379188 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f877c054b40 con 0x7f878415af70 2023-10-18T08:55:57.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.860+0000 7f8788c13640 1 -- 172.21.15.40:0/176379188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8784066e00 con 0x7f878415af70 2023-10-18T08:55:57.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.860+0000 7f876bfff640 1 -- 172.21.15.40:0/176379188 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f877c058160 con 0x7f878415af70 2023-10-18T08:55:57.863 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:55:57.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.863+0000 7f8788c13640 1 -- 172.21.15.40:0/176379188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f877006ae40 msgr2=0x7f877006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:57.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.863+0000 7f8788c13640 1 --2- 172.21.15.40:0/176379188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f877006ae40 0x7f877006d300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8774035000 tx=0x7f8774004730 comp rx=0 tx=0).stop 2023-10-18T08:55:57.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.863+0000 7f8788c13640 1 -- 172.21.15.40:0/176379188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f878415af70 msgr2=0x7f87841e27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:57.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.863+0000 7f8788c13640 1 --2- 172.21.15.40:0/176379188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f878415af70 0x7f87841e27d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f877c00c780 tx=0x7f877c00cc40 comp rx=0 tx=0).stop 2023-10-18T08:55:57.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.863+0000 7f8788c13640 1 -- 172.21.15.40:0/176379188 shutdown_connections 2023-10-18T08:55:57.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.863+0000 7f8788c13640 1 --2- 172.21.15.40:0/176379188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f877006ae40 0x7f877006d300 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:57.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.863+0000 7f8788c13640 1 --2- 172.21.15.40:0/176379188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f878415af70 0x7f87841e27d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:57.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.863+0000 7f8788c13640 1 --2- 172.21.15.40:0/176379188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f878406d5d0 0x7f8784150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:57.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.863+0000 7f8788c13640 1 --2- 172.21.15.40:0/176379188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f878406c2d0 0x7f87841e1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:57.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.863+0000 7f8788c13640 1 -- 172.21.15.40:0/176379188 >> 172.21.15.40:0/176379188 conn(0x7f878406ca10 msgr2=0x7f878406cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.863+0000 7f8788c13640 1 -- 172.21.15.40:0/176379188 shutdown_connections 2023-10-18T08:55:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:57.863+0000 7f8788c13640 1 -- 172.21.15.40:0/176379188 wait complete. 2023-10-18T08:55:57.885 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:58.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.119+0000 7f9474de2640 1 -- 172.21.15.40:0/1014364378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94700b15b0 msgr2=0x7f94700b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:58.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.119+0000 7f9474de2640 1 --2- 172.21.15.40:0/1014364378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94700b15b0 0x7f94700b0630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9460005fd0 tx=0x7f946002c2d0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.119+0000 7f9474de2640 1 -- 172.21.15.40:0/1014364378 shutdown_connections 2023-10-18T08:55:58.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.119+0000 7f9474de2640 1 --2- 172.21.15.40:0/1014364378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94700ca0d0 0x7f9470157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.119+0000 7f9474de2640 1 --2- 172.21.15.40:0/1014364378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94700b15b0 0x7f94700b0630 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.119+0000 7f9474de2640 1 -- 172.21.15.40:0/1014364378 >> 172.21.15.40:0/1014364378 conn(0x7f9470069720 msgr2=0x7f9470069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:58.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.119+0000 7f9474de2640 1 -- 172.21.15.40:0/1014364378 shutdown_connections 2023-10-18T08:55:58.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.119+0000 7f9474de2640 1 -- 172.21.15.40:0/1014364378 wait complete. 2023-10-18T08:55:58.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.121+0000 7f9474de2640 1 Processor -- start 2023-10-18T08:55:58.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.121+0000 7f9474de2640 1 -- start start 2023-10-18T08:55:58.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.122+0000 7f9474de2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94700b15b0 0x7f9470158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:58.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.123+0000 7f9474de2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94700b1e10 0x7f9470145f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:58.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.123+0000 7f9474de2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94700ca0d0 0x7f947014c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:58.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.123+0000 7f9474de2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9470159e80 con 0x7f94700b1e10 2023-10-18T08:55:58.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.123+0000 7f9474de2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9470144ab0 con 0x7f94700ca0d0 2023-10-18T08:55:58.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.123+0000 7f9474de2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9470144d90 con 0x7f94700b15b0 2023-10-18T08:55:58.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.123+0000 7f9466ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94700b1e10 0x7f9470145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:58.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.123+0000 7f9466ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94700b1e10 0x7f9470145f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52442/0 (socket says 172.21.15.40:52442) 2023-10-18T08:55:58.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.123+0000 7f9466ffd640 1 -- 172.21.15.40:0/323505163 learned_addr learned my addr 172.21.15.40:0/323505163 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:58.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.123+0000 7f946f7fe640 1 --2- 172.21.15.40:0/323505163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94700b15b0 0x7f9470158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:58.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.123+0000 7f946ffff640 1 --2- 172.21.15.40:0/323505163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94700ca0d0 0x7f947014c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:58.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.123+0000 7f9466ffd640 1 -- 172.21.15.40:0/323505163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94700b15b0 msgr2=0x7f9470158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:58.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.123+0000 7f9466ffd640 1 --2- 172.21.15.40:0/323505163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94700b15b0 0x7f9470158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.123+0000 7f9466ffd640 1 -- 172.21.15.40:0/323505163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94700ca0d0 msgr2=0x7f947014c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:58.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.123+0000 7f9466ffd640 1 --2- 172.21.15.40:0/323505163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94700ca0d0 0x7f947014c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.123+0000 7f9466ffd640 1 -- 172.21.15.40:0/323505163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9460009900 con 0x7f94700b1e10 2023-10-18T08:55:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.124+0000 7f9466ffd640 1 --2- 172.21.15.40:0/323505163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94700b1e10 0x7f9470145f10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f945800c500 tx=0x7f945800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.124+0000 7f946cff9640 1 -- 172.21.15.40:0/323505163 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94580052d0 con 0x7f94700b1e10 2023-10-18T08:55:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.124+0000 7f946cff9640 1 -- 172.21.15.40:0/323505163 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f94580035f0 con 0x7f94700b1e10 2023-10-18T08:55:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.124+0000 7f946cff9640 1 -- 172.21.15.40:0/323505163 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458003a40 con 0x7f94700b1e10 2023-10-18T08:55:58.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.124+0000 7f9474de2640 1 -- 172.21.15.40:0/323505163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947015a0f0 con 0x7f94700b1e10 2023-10-18T08:55:58.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.124+0000 7f9474de2640 1 -- 172.21.15.40:0/323505163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9470144740 con 0x7f94700b1e10 2023-10-18T08:55:58.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.125+0000 7f9474de2640 1 -- 172.21.15.40:0/323505163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947015a610 con 0x7f94700b1e10 2023-10-18T08:55:58.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.126+0000 7f946cff9640 1 -- 172.21.15.40:0/323505163 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9458003ba0 con 0x7f94700b1e10 2023-10-18T08:55:58.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.127+0000 7f946cff9640 1 --2- 172.21.15.40:0/323505163 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f945006ae40 0x7f945006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:58.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.127+0000 7f946cff9640 1 -- 172.21.15.40:0/323505163 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9458088f00 con 0x7f94700b1e10 2023-10-18T08:55:58.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.128+0000 7f946f7fe640 1 --2- 172.21.15.40:0/323505163 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f945006ae40 0x7f945006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:58.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.129+0000 7f946f7fe640 1 --2- 172.21.15.40:0/323505163 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f945006ae40 0x7f945006d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9460036000 tx=0x7f9460004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:58.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.132+0000 7f946cff9640 1 -- 172.21.15.40:0/323505163 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f947015a610 con 0x7f94700b1e10 2023-10-18T08:55:58.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.325+0000 7f9474de2640 1 -- 172.21.15.40:0/323505163 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f947015a610 con 0x7f94700b1e10 2023-10-18T08:55:58.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.328+0000 7f946cff9640 1 -- 172.21.15.40:0/323505163 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f947015a610 con 0x7f94700b1e10 2023-10-18T08:55:58.331 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:55:58.331 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:16.275677+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:55:58.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.331+0000 7f9474de2640 1 -- 172.21.15.40:0/323505163 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f945006ae40 msgr2=0x7f945006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:58.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.331+0000 7f9474de2640 1 --2- 172.21.15.40:0/323505163 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f945006ae40 0x7f945006d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9460036000 tx=0x7f9460004ab0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.331+0000 7f9474de2640 1 -- 172.21.15.40:0/323505163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94700b1e10 msgr2=0x7f9470145f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:58.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.331+0000 7f9474de2640 1 --2- 172.21.15.40:0/323505163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94700b1e10 0x7f9470145f10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f945800c500 tx=0x7f945800c9c0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.331+0000 7f9474de2640 1 -- 172.21.15.40:0/323505163 shutdown_connections 2023-10-18T08:55:58.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.331+0000 7f9474de2640 1 --2- 172.21.15.40:0/323505163 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f945006ae40 0x7f945006d300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.331+0000 7f9474de2640 1 --2- 172.21.15.40:0/323505163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94700ca0d0 0x7f947014c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.331+0000 7f9474de2640 1 --2- 172.21.15.40:0/323505163 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94700b1e10 0x7f9470145f10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.331+0000 7f9474de2640 1 --2- 172.21.15.40:0/323505163 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94700b15b0 0x7f9470158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.331+0000 7f9474de2640 1 -- 172.21.15.40:0/323505163 >> 172.21.15.40:0/323505163 conn(0x7f9470069720 msgr2=0x7f9470155f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:58.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.331+0000 7f9474de2640 1 -- 172.21.15.40:0/323505163 shutdown_connections 2023-10-18T08:55:58.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.331+0000 7f9474de2640 1 -- 172.21.15.40:0/323505163 wait complete. 2023-10-18T08:55:58.354 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:55:13.210788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:55:13.772849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:55:58.355 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:55:58.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.580+0000 7f22fffff640 1 -- 172.21.15.40:0/579124409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ec005180 msgr2=0x7f22ec00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:58.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.580+0000 7f22fffff640 1 --2- 172.21.15.40:0/579124409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ec005180 0x7f22ec00b980 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f22f8006f00 tx=0x7f22ec0057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.580+0000 7f22fffff640 1 -- 172.21.15.40:0/579124409 shutdown_connections 2023-10-18T08:55:58.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.580+0000 7f22fffff640 1 --2- 172.21.15.40:0/579124409 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ec005180 0x7f22ec00b980 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.580+0000 7f22fffff640 1 -- 172.21.15.40:0/579124409 >> 172.21.15.40:0/579124409 conn(0x7f230008ad30 msgr2=0x7f230009a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:58.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.580+0000 7f22fffff640 1 -- 172.21.15.40:0/579124409 shutdown_connections 2023-10-18T08:55:58.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.580+0000 7f22fffff640 1 -- 172.21.15.40:0/579124409 wait complete. 2023-10-18T08:55:58.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.582+0000 7f22fffff640 1 Processor -- start 2023-10-18T08:55:58.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.582+0000 7f22fffff640 1 -- start start 2023-10-18T08:55:58.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.583+0000 7f22fffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ec005180 0x7f2300154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:58.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.583+0000 7f22fffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23000caf20 0x7f2300156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:58.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.583+0000 7f22fffff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2300156ea0 0x7f23000c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:58.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.583+0000 7f22fffff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23001549d0 con 0x7f22ec005180 2023-10-18T08:55:58.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.583+0000 7f22fffff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2300157350 con 0x7f23000caf20 2023-10-18T08:55:58.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.583+0000 7f22fffff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23001556e0 con 0x7f2300156ea0 2023-10-18T08:55:58.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.583+0000 7f22feffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ec005180 0x7f2300154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:58.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.583+0000 7f22feffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ec005180 0x7f2300154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52458/0 (socket says 172.21.15.40:52458) 2023-10-18T08:55:58.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.583+0000 7f22feffd640 1 -- 172.21.15.40:0/2308499171 learned_addr learned my addr 172.21.15.40:0/2308499171 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:58.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.584+0000 7f22fe7fc640 1 --2- 172.21.15.40:0/2308499171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23000caf20 0x7f2300156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:58.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.584+0000 7f22ff7fe640 1 --2- 172.21.15.40:0/2308499171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2300156ea0 0x7f23000c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:58.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.584+0000 7f22feffd640 1 -- 172.21.15.40:0/2308499171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2300156ea0 msgr2=0x7f23000c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:58.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.584+0000 7f22feffd640 1 --2- 172.21.15.40:0/2308499171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2300156ea0 0x7f23000c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.584+0000 7f22feffd640 1 -- 172.21.15.40:0/2308499171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23000caf20 msgr2=0x7f2300156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:58.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.584+0000 7f22feffd640 1 --2- 172.21.15.40:0/2308499171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23000caf20 0x7f2300156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.584+0000 7f22feffd640 1 -- 172.21.15.40:0/2308499171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23001559c0 con 0x7f22ec005180 2023-10-18T08:55:58.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.584+0000 7f22feffd640 1 --2- 172.21.15.40:0/2308499171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ec005180 0x7f2300154110 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f22f0010b80 tx=0x7f22f000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:58.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.584+0000 7f22f77fe640 1 -- 172.21.15.40:0/2308499171 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0003b70 con 0x7f22ec005180 2023-10-18T08:55:58.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.584+0000 7f22f77fe640 1 -- 172.21.15.40:0/2308499171 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f22f0015070 con 0x7f22ec005180 2023-10-18T08:55:58.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.584+0000 7f22f77fe640 1 -- 172.21.15.40:0/2308499171 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0010e80 con 0x7f22ec005180 2023-10-18T08:55:58.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.584+0000 7f22fffff640 1 -- 172.21.15.40:0/2308499171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2300155ca0 con 0x7f22ec005180 2023-10-18T08:55:58.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.584+0000 7f22fffff640 1 -- 172.21.15.40:0/2308499171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2300156260 con 0x7f22ec005180 2023-10-18T08:55:58.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.586+0000 7f22fffff640 1 -- 172.21.15.40:0/2308499171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2300071da0 con 0x7f22ec005180 2023-10-18T08:55:58.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.587+0000 7f22f77fe640 1 -- 172.21.15.40:0/2308499171 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f22f0004020 con 0x7f22ec005180 2023-10-18T08:55:58.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.588+0000 7f22f77fe640 1 --2- 172.21.15.40:0/2308499171 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f22e006ae70 0x7f22e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:58.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.588+0000 7f22f77fe640 1 -- 172.21.15.40:0/2308499171 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22f0089fe0 con 0x7f22ec005180 2023-10-18T08:55:58.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.588+0000 7f22fe7fc640 1 --2- 172.21.15.40:0/2308499171 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f22e006ae70 0x7f22e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:58.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.589+0000 7f22fe7fc640 1 --2- 172.21.15.40:0/2308499171 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f22e006ae70 0x7f22e006d330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f230009f0f0 tx=0x7f22e8006070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:58.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.592+0000 7f22f77fe640 1 -- 172.21.15.40:0/2308499171 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22f0055100 con 0x7f22ec005180 2023-10-18T08:55:58.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.785+0000 7f22fffff640 1 -- 172.21.15.40:0/2308499171 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2300069c90 con 0x7f22ec005180 2023-10-18T08:55:58.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.786+0000 7f22f77fe640 1 -- 172.21.15.40:0/2308499171 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22f0058720 con 0x7f22ec005180 2023-10-18T08:55:58.788 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:55:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.789+0000 7f22fffff640 1 -- 172.21.15.40:0/2308499171 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f22e006ae70 msgr2=0x7f22e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.789+0000 7f22fffff640 1 --2- 172.21.15.40:0/2308499171 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f22e006ae70 0x7f22e006d330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f230009f0f0 tx=0x7f22e8006070 comp rx=0 tx=0).stop 2023-10-18T08:55:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.789+0000 7f22fffff640 1 -- 172.21.15.40:0/2308499171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ec005180 msgr2=0x7f2300154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.789+0000 7f22fffff640 1 --2- 172.21.15.40:0/2308499171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ec005180 0x7f2300154110 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f22f0010b80 tx=0x7f22f000e720 comp rx=0 tx=0).stop 2023-10-18T08:55:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.789+0000 7f22fffff640 1 -- 172.21.15.40:0/2308499171 shutdown_connections 2023-10-18T08:55:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.789+0000 7f22fffff640 1 --2- 172.21.15.40:0/2308499171 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f22e006ae70 0x7f22e006d330 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.789+0000 7f22fffff640 1 --2- 172.21.15.40:0/2308499171 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2300156ea0 0x7f23000c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.789+0000 7f22fffff640 1 --2- 172.21.15.40:0/2308499171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23000caf20 0x7f2300156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.789+0000 7f22fffff640 1 --2- 172.21.15.40:0/2308499171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22ec005180 0x7f2300154110 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.789+0000 7f22fffff640 1 -- 172.21.15.40:0/2308499171 >> 172.21.15.40:0/2308499171 conn(0x7f230008ad30 msgr2=0x7f2300072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.789+0000 7f22fffff640 1 -- 172.21.15.40:0/2308499171 shutdown_connections 2023-10-18T08:55:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:58.789+0000 7f22fffff640 1 -- 172.21.15.40:0/2308499171 wait complete. 2023-10-18T08:55:58.809 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:55:59.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.039+0000 7f8ca280d640 1 -- 172.21.15.40:0/779791079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c880039a0 msgr2=0x7f8c88003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:59.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.039+0000 7f8ca280d640 1 --2- 172.21.15.40:0/779791079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c880039a0 0x7f8c88003dc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8c98001680 tx=0x7f8c880057f0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.039+0000 7f8ca200c640 1 -- 172.21.15.40:0/779791079 reap_dead start 2023-10-18T08:55:59.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.040+0000 7f8ca280d640 1 -- 172.21.15.40:0/779791079 shutdown_connections 2023-10-18T08:55:59.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.040+0000 7f8ca280d640 1 -- 172.21.15.40:0/779791079 >> 172.21.15.40:0/779791079 conn(0x7f8c9c0af450 msgr2=0x7f8c9c0af860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:59.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.040+0000 7f8ca280d640 1 -- 172.21.15.40:0/779791079 shutdown_connections 2023-10-18T08:55:59.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.040+0000 7f8ca280d640 1 -- 172.21.15.40:0/779791079 wait complete. 2023-10-18T08:55:59.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.041+0000 7f8ca280d640 1 Processor -- start 2023-10-18T08:55:59.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.042+0000 7f8ca280d640 1 -- start start 2023-10-18T08:55:59.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.042+0000 7f8ca280d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c880039a0 0x7f8c9c1536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:59.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.042+0000 7f8ca280d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c9c1524f0 0x7f8c9c152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:59.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.043+0000 7f8ca280d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c14dd50 0x7f8c9c14e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:59.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.043+0000 7f8ca280d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c14f6e0 con 0x7f8c9c14dd50 2023-10-18T08:55:59.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.043+0000 7f8ca280d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c152e70 con 0x7f8c9c1524f0 2023-10-18T08:55:59.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.043+0000 7f8ca280d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c14ead0 con 0x7f8c880039a0 2023-10-18T08:55:59.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.043+0000 7f8ca200c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c14dd50 0x7f8c9c14e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:59.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.043+0000 7f8ca200c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c14dd50 0x7f8c9c14e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52464/0 (socket says 172.21.15.40:52464) 2023-10-18T08:55:59.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.043+0000 7f8ca200c640 1 -- 172.21.15.40:0/1548080550 learned_addr learned my addr 172.21.15.40:0/1548080550 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:59.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.043+0000 7f8ca100a640 1 --2- 172.21.15.40:0/1548080550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c9c1524f0 0x7f8c9c152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:59.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.043+0000 7f8ca200c640 1 -- 172.21.15.40:0/1548080550 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c880039a0 msgr2=0x7f8c9c1536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:59.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.043+0000 7f8ca200c640 1 --2- 172.21.15.40:0/1548080550 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c880039a0 0x7f8c9c1536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.043+0000 7f8ca200c640 1 -- 172.21.15.40:0/1548080550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c9c1524f0 msgr2=0x7f8c9c152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:59.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.043+0000 7f8ca200c640 1 --2- 172.21.15.40:0/1548080550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c9c1524f0 0x7f8c9c152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.043+0000 7f8ca200c640 1 -- 172.21.15.40:0/1548080550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c14edb0 con 0x7f8c9c14dd50 2023-10-18T08:55:59.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.044+0000 7f8ca200c640 1 --2- 172.21.15.40:0/1548080550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c14dd50 0x7f8c9c14e1b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0657f0 tx=0x7f8c9400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:59.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.044+0000 7f8c927fc640 1 -- 172.21.15.40:0/1548080550 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94018070 con 0x7f8c9c14dd50 2023-10-18T08:55:59.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.044+0000 7f8c927fc640 1 -- 172.21.15.40:0/1548080550 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8c9400cd90 con 0x7f8c9c14dd50 2023-10-18T08:55:59.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.044+0000 7f8c927fc640 1 -- 172.21.15.40:0/1548080550 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94003bd0 con 0x7f8c9c14dd50 2023-10-18T08:55:59.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.044+0000 7f8ca280d640 1 -- 172.21.15.40:0/1548080550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9c14f090 con 0x7f8c9c14dd50 2023-10-18T08:55:59.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.044+0000 7f8ca280d640 1 -- 172.21.15.40:0/1548080550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9c14fb70 con 0x7f8c9c14dd50 2023-10-18T08:55:59.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.045+0000 7f8ca280d640 1 -- 172.21.15.40:0/1548080550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9c0667d0 con 0x7f8c9c14dd50 2023-10-18T08:55:59.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.046+0000 7f8c927fc640 1 -- 172.21.15.40:0/1548080550 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8c94010430 con 0x7f8c9c14dd50 2023-10-18T08:55:59.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.047+0000 7f8c927fc640 1 --2- 172.21.15.40:0/1548080550 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c8006ae40 0x7f8c8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:59.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.047+0000 7f8c927fc640 1 -- 172.21.15.40:0/1548080550 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c94089560 con 0x7f8c9c14dd50 2023-10-18T08:55:59.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.048+0000 7f8ca180b640 1 --2- 172.21.15.40:0/1548080550 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c8006ae40 0x7f8c8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:59.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.049+0000 7f8ca180b640 1 --2- 172.21.15.40:0/1548080550 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c8006ae40 0x7f8c8006d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8c98034000 tx=0x7f8c98001120 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:59.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.051+0000 7f8c927fc640 1 -- 172.21.15.40:0/1548080550 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c94054600 con 0x7f8c9c14dd50 2023-10-18T08:55:59.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.246+0000 7f8ca280d640 1 -- 172.21.15.40:0/1548080550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c9c0708c0 con 0x7f8c9c14dd50 2023-10-18T08:55:59.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.247+0000 7f8c927fc640 1 -- 172.21.15.40:0/1548080550 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8c94057c20 con 0x7f8c9c14dd50 2023-10-18T08:55:59.249 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:55:59.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.249+0000 7f8ca280d640 1 -- 172.21.15.40:0/1548080550 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c8006ae40 msgr2=0x7f8c8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:59.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.249+0000 7f8ca280d640 1 --2- 172.21.15.40:0/1548080550 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c8006ae40 0x7f8c8006d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8c98034000 tx=0x7f8c98001120 comp rx=0 tx=0).stop 2023-10-18T08:55:59.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.249+0000 7f8ca280d640 1 -- 172.21.15.40:0/1548080550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c14dd50 msgr2=0x7f8c9c14e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:59.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.249+0000 7f8ca280d640 1 --2- 172.21.15.40:0/1548080550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c14dd50 0x7f8c9c14e1b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0657f0 tx=0x7f8c9400c6a0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.250+0000 7f8ca280d640 1 -- 172.21.15.40:0/1548080550 shutdown_connections 2023-10-18T08:55:59.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.250+0000 7f8ca280d640 1 --2- 172.21.15.40:0/1548080550 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c8006ae40 0x7f8c8006d300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.250+0000 7f8ca280d640 1 --2- 172.21.15.40:0/1548080550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c9c14dd50 0x7f8c9c14e1b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.250+0000 7f8ca280d640 1 --2- 172.21.15.40:0/1548080550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c9c1524f0 0x7f8c9c152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.250+0000 7f8ca280d640 1 --2- 172.21.15.40:0/1548080550 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c880039a0 0x7f8c9c1536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.250+0000 7f8ca280d640 1 -- 172.21.15.40:0/1548080550 >> 172.21.15.40:0/1548080550 conn(0x7f8c9c0af450 msgr2=0x7f8c9c1561e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:59.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.250+0000 7f8ca280d640 1 -- 172.21.15.40:0/1548080550 shutdown_connections 2023-10-18T08:55:59.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.250+0000 7f8ca280d640 1 -- 172.21.15.40:0/1548080550 wait complete. 2023-10-18T08:55:59.270 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:55:59.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.495+0000 7f19623e2640 1 -- 172.21.15.40:0/1115517361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f195c071350 msgr2=0x7f195c0cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:59.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.495+0000 7f19623e2640 1 --2- 172.21.15.40:0/1115517361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f195c071350 0x7f195c0cb410 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f195c06c780 tx=0x7f194c02bcf0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.495+0000 7f19623e2640 1 -- 172.21.15.40:0/1115517361 shutdown_connections 2023-10-18T08:55:59.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.495+0000 7f19623e2640 1 --2- 172.21.15.40:0/1115517361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f195c071350 0x7f195c0cb410 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.495+0000 7f19623e2640 1 --2- 172.21.15.40:0/1115517361 >> v2:172.21.15.144:3300/0 conn(0x7f195c155ad0 0x7f195c0cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.495+0000 7f19623e2640 1 -- 172.21.15.40:0/1115517361 >> 172.21.15.40:0/1115517361 conn(0x7f195c06ca10 msgr2=0x7f195c06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:59.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.495+0000 7f19623e2640 1 -- 172.21.15.40:0/1115517361 shutdown_connections 2023-10-18T08:55:59.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.495+0000 7f19623e2640 1 -- 172.21.15.40:0/1115517361 wait complete. 2023-10-18T08:55:59.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.497+0000 7f19623e2640 1 Processor -- start 2023-10-18T08:55:59.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.497+0000 7f19623e2640 1 -- start start 2023-10-18T08:55:59.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.498+0000 7f19623e2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f195c071350 0x7f195c15af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:59.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.498+0000 7f19623e2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f195c0cb950 0x7f195c1584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:59.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.498+0000 7f19623e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f195c155ad0 0x7f195c158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:59.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.498+0000 7f19623e2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c15aad0 con 0x7f195c155ad0 2023-10-18T08:55:59.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.498+0000 7f19623e2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c154060 con 0x7f195c0cb950 2023-10-18T08:55:59.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.498+0000 7f19623e2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195c15a4c0 con 0x7f195c071350 2023-10-18T08:55:59.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.498+0000 7f1961be1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f195c155ad0 0x7f195c158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:59.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.498+0000 7f1961be1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f195c155ad0 0x7f195c158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52482/0 (socket says 172.21.15.40:52482) 2023-10-18T08:55:59.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.498+0000 7f1961be1640 1 -- 172.21.15.40:0/3138219970 learned_addr learned my addr 172.21.15.40:0/3138219970 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:59.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.498+0000 7f1960bdf640 1 --2- 172.21.15.40:0/3138219970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f195c0cb950 0x7f195c1584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:59.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.498+0000 7f19613e0640 1 --2- 172.21.15.40:0/3138219970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f195c071350 0x7f195c15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:59.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.499+0000 7f1961be1640 1 -- 172.21.15.40:0/3138219970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f195c071350 msgr2=0x7f195c15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:59.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.499+0000 7f1961be1640 1 --2- 172.21.15.40:0/3138219970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f195c071350 0x7f195c15af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.499+0000 7f1961be1640 1 -- 172.21.15.40:0/3138219970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f195c0cb950 msgr2=0x7f195c1584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:59.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.499+0000 7f1961be1640 1 --2- 172.21.15.40:0/3138219970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f195c0cb950 0x7f195c1584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.499+0000 7f1961be1640 1 -- 172.21.15.40:0/3138219970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c009490 con 0x7f195c155ad0 2023-10-18T08:55:59.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.499+0000 7f1961be1640 1 --2- 172.21.15.40:0/3138219970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f195c155ad0 0x7f195c158a10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f195400cae0 tx=0x7f195400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:59.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.499+0000 7f1951ffb640 1 -- 172.21.15.40:0/3138219970 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195400ac40 con 0x7f195c155ad0 2023-10-18T08:55:59.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.500+0000 7f1951ffb640 1 -- 172.21.15.40:0/3138219970 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f195400ae90 con 0x7f195c155ad0 2023-10-18T08:55:59.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.500+0000 7f1951ffb640 1 -- 172.21.15.40:0/3138219970 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954003bd0 con 0x7f195c155ad0 2023-10-18T08:55:59.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.500+0000 7f19623e2640 1 -- 172.21.15.40:0/3138219970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195c154300 con 0x7f195c155ad0 2023-10-18T08:55:59.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.500+0000 7f19623e2640 1 -- 172.21.15.40:0/3138219970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195c15b4a0 con 0x7f195c155ad0 2023-10-18T08:55:59.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.501+0000 7f19623e2640 1 -- 172.21.15.40:0/3138219970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195c154540 con 0x7f195c155ad0 2023-10-18T08:55:59.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.502+0000 7f1951ffb640 1 -- 172.21.15.40:0/3138219970 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1954003d30 con 0x7f195c155ad0 2023-10-18T08:55:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.503+0000 7f1951ffb640 1 --2- 172.21.15.40:0/3138219970 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f194006ae40 0x7f194006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.503+0000 7f1951ffb640 1 -- 172.21.15.40:0/3138219970 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f195408a100 con 0x7f195c155ad0 2023-10-18T08:55:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.504+0000 7f19613e0640 1 --2- 172.21.15.40:0/3138219970 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f194006ae40 0x7f194006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.505+0000 7f19613e0640 1 --2- 172.21.15.40:0/3138219970 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f194006ae40 0x7f194006d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1958005fd0 tx=0x7f1958005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.507+0000 7f1951ffb640 1 -- 172.21.15.40:0/3138219970 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f195c154540 con 0x7f195c155ad0 2023-10-18T08:55:59.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.704+0000 7f19623e2640 1 -- 172.21.15.40:0/3138219970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f195c154540 con 0x7f195c155ad0 2023-10-18T08:55:59.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.704+0000 7f1951ffb640 1 -- 172.21.15.40:0/3138219970 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f195c154540 con 0x7f195c155ad0 2023-10-18T08:55:59.707 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:55:59.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.707+0000 7f19623e2640 1 -- 172.21.15.40:0/3138219970 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f194006ae40 msgr2=0x7f194006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:59.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.707+0000 7f19623e2640 1 --2- 172.21.15.40:0/3138219970 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f194006ae40 0x7f194006d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1958005fd0 tx=0x7f1958005850 comp rx=0 tx=0).stop 2023-10-18T08:55:59.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.707+0000 7f19623e2640 1 -- 172.21.15.40:0/3138219970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f195c155ad0 msgr2=0x7f195c158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:59.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.707+0000 7f19623e2640 1 --2- 172.21.15.40:0/3138219970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f195c155ad0 0x7f195c158a10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f195400cae0 tx=0x7f195400a060 comp rx=0 tx=0).stop 2023-10-18T08:55:59.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.707+0000 7f19623e2640 1 -- 172.21.15.40:0/3138219970 shutdown_connections 2023-10-18T08:55:59.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.707+0000 7f19623e2640 1 --2- 172.21.15.40:0/3138219970 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f194006ae40 0x7f194006d300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.707+0000 7f19623e2640 1 --2- 172.21.15.40:0/3138219970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f195c155ad0 0x7f195c158a10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.707+0000 7f19623e2640 1 --2- 172.21.15.40:0/3138219970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f195c0cb950 0x7f195c1584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.707+0000 7f19623e2640 1 --2- 172.21.15.40:0/3138219970 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f195c071350 0x7f195c15af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.707+0000 7f19623e2640 1 -- 172.21.15.40:0/3138219970 >> 172.21.15.40:0/3138219970 conn(0x7f195c06ca10 msgr2=0x7f195c0ae470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:59.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.708+0000 7f19623e2640 1 -- 172.21.15.40:0/3138219970 shutdown_connections 2023-10-18T08:55:59.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.708+0000 7f19623e2640 1 -- 172.21.15.40:0/3138219970 wait complete. 2023-10-18T08:55:59.748 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestClusterFull) That when an OSD epoch barrier is set on an MDS, subsequently 2023-10-18T08:55:59.749 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestClusterFull) 2023-10-18T08:55:59.749 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:55:59.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.979+0000 7fdb1af9a640 1 -- 172.21.15.40:0/2526361814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb00004ad0 msgr2=0x7fdb00004ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:59.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.979+0000 7fdb1af9a640 1 --2- 172.21.15.40:0/2526361814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb00004ad0 0x7fdb00004ef0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009b50 tx=0x7fdb00006900 comp rx=0 tx=0).stop 2023-10-18T08:55:59.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.979+0000 7fdb1af9a640 1 -- 172.21.15.40:0/2526361814 shutdown_connections 2023-10-18T08:55:59.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.979+0000 7fdb1af9a640 1 --2- 172.21.15.40:0/2526361814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb00004ad0 0x7fdb00004ef0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.979+0000 7fdb1af9a640 1 -- 172.21.15.40:0/2526361814 >> 172.21.15.40:0/2526361814 conn(0x7fdb14069720 msgr2=0x7fdb14069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:55:59.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.979+0000 7fdb1af9a640 1 -- 172.21.15.40:0/2526361814 shutdown_connections 2023-10-18T08:55:59.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.979+0000 7fdb1af9a640 1 -- 172.21.15.40:0/2526361814 wait complete. 2023-10-18T08:55:59.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.981+0000 7fdb1af9a640 1 Processor -- start 2023-10-18T08:55:59.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.981+0000 7fdb1af9a640 1 -- start start 2023-10-18T08:55:59.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.982+0000 7fdb1af9a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb00004ad0 0x7fdb141f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:59.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.982+0000 7fdb1af9a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb14151400 0x7fdb1413d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:59.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.982+0000 7fdb1af9a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1414ed70 0x7fdb14159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:59.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.982+0000 7fdb1af9a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1413ddf0 con 0x7fdb00004ad0 2023-10-18T08:55:59.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.982+0000 7fdb1af9a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1413f280 con 0x7fdb1414ed70 2023-10-18T08:55:59.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.982+0000 7fdb1af9a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1413f560 con 0x7fdb14151400 2023-10-18T08:55:59.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.982+0000 7fdb19f98640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb00004ad0 0x7fdb141f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:59.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.983+0000 7fdb19f98640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb00004ad0 0x7fdb141f2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52510/0 (socket says 172.21.15.40:52510) 2023-10-18T08:55:59.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.983+0000 7fdb19f98640 1 -- 172.21.15.40:0/294291296 learned_addr learned my addr 172.21.15.40:0/294291296 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:55:59.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.983+0000 7fdb19f98640 1 -- 172.21.15.40:0/294291296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb14151400 msgr2=0x7fdb1413d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:59.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.983+0000 7fdb19f98640 1 --2- 172.21.15.40:0/294291296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb14151400 0x7fdb1413d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.983+0000 7fdb19f98640 1 -- 172.21.15.40:0/294291296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1414ed70 msgr2=0x7fdb14159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:55:59.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.983+0000 7fdb19f98640 1 --2- 172.21.15.40:0/294291296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1414ed70 0x7fdb14159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:55:59.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.983+0000 7fdb19f98640 1 -- 172.21.15.40:0/294291296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1413f840 con 0x7fdb00004ad0 2023-10-18T08:55:59.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.983+0000 7fdb19f98640 1 --2- 172.21.15.40:0/294291296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb00004ad0 0x7fdb141f2070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009b50 tx=0x7fdb10033bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:59.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.984+0000 7fdb0a7fc640 1 -- 172.21.15.40:0/294291296 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10004230 con 0x7fdb00004ad0 2023-10-18T08:55:59.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.984+0000 7fdb0a7fc640 1 -- 172.21.15.40:0/294291296 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdb10004390 con 0x7fdb00004ad0 2023-10-18T08:55:59.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.984+0000 7fdb0a7fc640 1 -- 172.21.15.40:0/294291296 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10003850 con 0x7fdb00004ad0 2023-10-18T08:55:59.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.984+0000 7fdb1af9a640 1 -- 172.21.15.40:0/294291296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1413fb20 con 0x7fdb00004ad0 2023-10-18T08:55:59.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.984+0000 7fdb1af9a640 1 -- 172.21.15.40:0/294291296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1414f1f0 con 0x7fdb00004ad0 2023-10-18T08:55:59.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.985+0000 7fdb1af9a640 1 -- 172.21.15.40:0/294291296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1414f380 con 0x7fdb00004ad0 2023-10-18T08:55:59.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.986+0000 7fdb0a7fc640 1 -- 172.21.15.40:0/294291296 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdb100039b0 con 0x7fdb00004ad0 2023-10-18T08:55:59.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.987+0000 7fdb0a7fc640 1 --2- 172.21.15.40:0/294291296 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdaf806ae70 0x7fdaf806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:55:59.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.987+0000 7fdb0a7fc640 1 -- 172.21.15.40:0/294291296 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb100ac1e0 con 0x7fdb00004ad0 2023-10-18T08:55:59.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.987+0000 7fdb19797640 1 --2- 172.21.15.40:0/294291296 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdaf806ae70 0x7fdaf806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:55:59.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.988+0000 7fdb19797640 1 --2- 172.21.15.40:0/294291296 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdaf806ae70 0x7fdaf806d330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdb141f5540 tx=0x7fdb040079d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:55:59.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:55:59.991+0000 7fdb0a7fc640 1 -- 172.21.15.40:0/294291296 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb100771f0 con 0x7fdb00004ad0 2023-10-18T08:56:00.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.189+0000 7fdb1af9a640 1 -- 172.21.15.40:0/294291296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb1413ddf0 con 0x7fdb00004ad0 2023-10-18T08:56:00.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.192+0000 7fdb0a7fc640 1 -- 172.21.15.40:0/294291296 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7fdb1007a810 con 0x7fdb00004ad0 2023-10-18T08:56:00.195 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:00.196 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:55:16.275677+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:00.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.195+0000 7fdb1af9a640 1 -- 172.21.15.40:0/294291296 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdaf806ae70 msgr2=0x7fdaf806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:00.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.195+0000 7fdb1af9a640 1 --2- 172.21.15.40:0/294291296 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdaf806ae70 0x7fdaf806d330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdb141f5540 tx=0x7fdb040079d0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.195+0000 7fdb1af9a640 1 -- 172.21.15.40:0/294291296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb00004ad0 msgr2=0x7fdb141f2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:00.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.195+0000 7fdb1af9a640 1 --2- 172.21.15.40:0/294291296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb00004ad0 0x7fdb141f2070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009b50 tx=0x7fdb10033bd0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.195+0000 7fdb1af9a640 1 -- 172.21.15.40:0/294291296 shutdown_connections 2023-10-18T08:56:00.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.195+0000 7fdb1af9a640 1 --2- 172.21.15.40:0/294291296 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdaf806ae70 0x7fdaf806d330 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.195+0000 7fdb1af9a640 1 --2- 172.21.15.40:0/294291296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1414ed70 0x7fdb14159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.195+0000 7fdb1af9a640 1 --2- 172.21.15.40:0/294291296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb14151400 0x7fdb1413d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.195+0000 7fdb1af9a640 1 --2- 172.21.15.40:0/294291296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb00004ad0 0x7fdb141f2070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.195+0000 7fdb1af9a640 1 -- 172.21.15.40:0/294291296 >> 172.21.15.40:0/294291296 conn(0x7fdb14069720 msgr2=0x7fdb14154530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:00.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.195+0000 7fdb1af9a640 1 -- 172.21.15.40:0/294291296 shutdown_connections 2023-10-18T08:56:00.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.195+0000 7fdb1af9a640 1 -- 172.21.15.40:0/294291296 wait complete. 2023-10-18T08:56:00.218 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:55:13.210788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:55:13.772849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:56:00.220 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:56:00.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.460+0000 7fec7ad25640 1 -- 172.21.15.40:0/152900072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec7414e730 msgr2=0x7fec740b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:00.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.461+0000 7fec7ad25640 1 --2- 172.21.15.40:0/152900072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec7414e730 0x7fec740b1950 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fec68005fd0 tx=0x7fec6802bd90 comp rx=0 tx=0).stop 2023-10-18T08:56:00.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.461+0000 7fec7ad25640 1 -- 172.21.15.40:0/152900072 shutdown_connections 2023-10-18T08:56:00.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.461+0000 7fec7ad25640 1 --2- 172.21.15.40:0/152900072 >> v2:172.21.15.144:3300/0 conn(0x7fec740715c0 0x7fec740aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.461+0000 7fec7ad25640 1 --2- 172.21.15.40:0/152900072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec7414e730 0x7fec740b1950 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.461+0000 7fec7ad25640 1 -- 172.21.15.40:0/152900072 >> 172.21.15.40:0/152900072 conn(0x7fec74069720 msgr2=0x7fec74069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:00.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.461+0000 7fec7ad25640 1 -- 172.21.15.40:0/152900072 shutdown_connections 2023-10-18T08:56:00.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.461+0000 7fec7ad25640 1 -- 172.21.15.40:0/152900072 wait complete. 2023-10-18T08:56:00.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.463+0000 7fec7ad25640 1 Processor -- start 2023-10-18T08:56:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.463+0000 7fec7ad25640 1 -- start start 2023-10-18T08:56:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec7ad25640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec740715c0 0x7fec741e4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec7ad25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec740af240 0x7fec741e2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:00.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec7ad25640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec7414e730 0x7fec741e3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec7ad25640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec741e2730 con 0x7fec740af240 2023-10-18T08:56:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec7ad25640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7414d280 con 0x7fec7414e730 2023-10-18T08:56:00.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec7ad25640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec741e35d0 con 0x7fec740715c0 2023-10-18T08:56:00.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec73fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec740af240 0x7fec741e2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec73fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec740af240 0x7fec741e2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52512/0 (socket says 172.21.15.40:52512) 2023-10-18T08:56:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec73fff640 1 -- 172.21.15.40:0/3031384616 learned_addr learned my addr 172.21.15.40:0/3031384616 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec78ac1640 1 --2- 172.21.15.40:0/3031384616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec740715c0 0x7fec741e4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:00.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec792c2640 1 --2- 172.21.15.40:0/3031384616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec7414e730 0x7fec741e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:00.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec73fff640 1 -- 172.21.15.40:0/3031384616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec740715c0 msgr2=0x7fec741e4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:00.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec73fff640 1 --2- 172.21.15.40:0/3031384616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec740715c0 0x7fec741e4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec73fff640 1 -- 172.21.15.40:0/3031384616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec7414e730 msgr2=0x7fec741e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:00.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec73fff640 1 --2- 172.21.15.40:0/3031384616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec7414e730 0x7fec741e3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec73fff640 1 -- 172.21.15.40:0/3031384616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec68009df0 con 0x7fec740af240 2023-10-18T08:56:00.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.464+0000 7fec792c2640 1 --2- 172.21.15.40:0/3031384616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec7414e730 0x7fec741e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:00.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.465+0000 7fec73fff640 1 --2- 172.21.15.40:0/3031384616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec740af240 0x7fec741e2b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec6000ea50 tx=0x7fec6000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:00.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.465+0000 7fec717fa640 1 -- 172.21.15.40:0/3031384616 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6000cd20 con 0x7fec740af240 2023-10-18T08:56:00.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.465+0000 7fec717fa640 1 -- 172.21.15.40:0/3031384616 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fec60016410 con 0x7fec740af240 2023-10-18T08:56:00.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.465+0000 7fec7ad25640 1 -- 172.21.15.40:0/3031384616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7414a6f0 con 0x7fec740af240 2023-10-18T08:56:00.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.465+0000 7fec717fa640 1 -- 172.21.15.40:0/3031384616 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec60016820 con 0x7fec740af240 2023-10-18T08:56:00.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.465+0000 7fec7ad25640 1 -- 172.21.15.40:0/3031384616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec741e2070 con 0x7fec740af240 2023-10-18T08:56:00.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.466+0000 7fec7ad25640 1 -- 172.21.15.40:0/3031384616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec74066fd0 con 0x7fec740af240 2023-10-18T08:56:00.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.467+0000 7fec717fa640 1 -- 172.21.15.40:0/3031384616 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fec60016980 con 0x7fec740af240 2023-10-18T08:56:00.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.468+0000 7fec717fa640 1 --2- 172.21.15.40:0/3031384616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fec5806ae40 0x7fec5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:00.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.468+0000 7fec717fa640 1 -- 172.21.15.40:0/3031384616 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec60089cb0 con 0x7fec740af240 2023-10-18T08:56:00.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.469+0000 7fec78ac1640 1 --2- 172.21.15.40:0/3031384616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fec5806ae40 0x7fec5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:00.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.470+0000 7fec78ac1640 1 --2- 172.21.15.40:0/3031384616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fec5806ae40 0x7fec5806d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fec68035000 tx=0x7fec68004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:00.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.472+0000 7fec717fa640 1 -- 172.21.15.40:0/3031384616 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec60058810 con 0x7fec740af240 2023-10-18T08:56:00.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.662+0000 7fec7ad25640 1 -- 172.21.15.40:0/3031384616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec740c9d60 con 0x7fec740af240 2023-10-18T08:56:00.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.664+0000 7fec717fa640 1 -- 172.21.15.40:0/3031384616 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec600583a0 con 0x7fec740af240 2023-10-18T08:56:00.666 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:56:00.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.666+0000 7fec7ad25640 1 -- 172.21.15.40:0/3031384616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fec5806ae40 msgr2=0x7fec5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:00.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.666+0000 7fec7ad25640 1 --2- 172.21.15.40:0/3031384616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fec5806ae40 0x7fec5806d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fec68035000 tx=0x7fec68004730 comp rx=0 tx=0).stop 2023-10-18T08:56:00.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.666+0000 7fec7ad25640 1 -- 172.21.15.40:0/3031384616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec740af240 msgr2=0x7fec741e2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:00.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.666+0000 7fec7ad25640 1 --2- 172.21.15.40:0/3031384616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec740af240 0x7fec741e2b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fec6000ea50 tx=0x7fec6000ef10 comp rx=0 tx=0).stop 2023-10-18T08:56:00.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.666+0000 7fec7ad25640 1 -- 172.21.15.40:0/3031384616 shutdown_connections 2023-10-18T08:56:00.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.666+0000 7fec7ad25640 1 --2- 172.21.15.40:0/3031384616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fec5806ae40 0x7fec5806d300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.666+0000 7fec7ad25640 1 --2- 172.21.15.40:0/3031384616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec7414e730 0x7fec741e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.666+0000 7fec7ad25640 1 --2- 172.21.15.40:0/3031384616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec740af240 0x7fec741e2b50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.666+0000 7fec7ad25640 1 --2- 172.21.15.40:0/3031384616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec740715c0 0x7fec741e4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.666+0000 7fec7ad25640 1 -- 172.21.15.40:0/3031384616 >> 172.21.15.40:0/3031384616 conn(0x7fec74069720 msgr2=0x7fec74157120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:00.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.666+0000 7fec7ad25640 1 -- 172.21.15.40:0/3031384616 shutdown_connections 2023-10-18T08:56:00.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.666+0000 7fec7ad25640 1 -- 172.21.15.40:0/3031384616 wait complete. 2023-10-18T08:56:00.690 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:56:00.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.925+0000 7f9545993640 1 -- 172.21.15.40:0/1557166060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f954015eea0 msgr2=0x7f95400cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:00.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.925+0000 7f9545993640 1 --2- 172.21.15.40:0/1557166060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f954015eea0 0x7f95400cff70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f95400b72b0 tx=0x7f952802bd70 comp rx=0 tx=0).stop 2023-10-18T08:56:00.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.926+0000 7f9545993640 1 -- 172.21.15.40:0/1557166060 shutdown_connections 2023-10-18T08:56:00.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.926+0000 7f9545993640 1 --2- 172.21.15.40:0/1557166060 >> v2:172.21.15.144:3300/0 conn(0x7f95400d19b0 0x7f95400ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.926+0000 7f9545993640 1 --2- 172.21.15.40:0/1557166060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f954015eea0 0x7f95400cff70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.926+0000 7f9545993640 1 -- 172.21.15.40:0/1557166060 >> 172.21.15.40:0/1557166060 conn(0x7f954009a610 msgr2=0x7f95400d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:00.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.926+0000 7f9545993640 1 -- 172.21.15.40:0/1557166060 shutdown_connections 2023-10-18T08:56:00.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.926+0000 7f9545993640 1 -- 172.21.15.40:0/1557166060 wait complete. 2023-10-18T08:56:00.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.927+0000 7f9545993640 1 Processor -- start 2023-10-18T08:56:00.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.928+0000 7f9545993640 1 -- start start 2023-10-18T08:56:00.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.928+0000 7f9545993640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95400d19b0 0x7f95400c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:00.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.928+0000 7f9545993640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954015eac0 0x7f95400b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:00.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.928+0000 7f9545993640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f954015eea0 0x7f954015a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:00.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.928+0000 7f9545993640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954015a650 con 0x7f95400d19b0 2023-10-18T08:56:00.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.928+0000 7f9545993640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95400c70a0 con 0x7f954015eea0 2023-10-18T08:56:00.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.928+0000 7f9545993640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954015ab20 con 0x7f954015eac0 2023-10-18T08:56:00.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.929+0000 7f9544991640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95400d19b0 0x7f95400c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:00.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.929+0000 7f9544991640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95400d19b0 0x7f95400c5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52538/0 (socket says 172.21.15.40:52538) 2023-10-18T08:56:00.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.929+0000 7f9544991640 1 -- 172.21.15.40:0/4174361417 learned_addr learned my addr 172.21.15.40:0/4174361417 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:00.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.929+0000 7f9544991640 1 -- 172.21.15.40:0/4174361417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954015eac0 msgr2=0x7f95400b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:00.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.929+0000 7f9544991640 1 --2- 172.21.15.40:0/4174361417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954015eac0 0x7f95400b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.929+0000 7f9544991640 1 -- 172.21.15.40:0/4174361417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f954015eea0 msgr2=0x7f954015a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:00.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.929+0000 7f9544991640 1 --2- 172.21.15.40:0/4174361417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f954015eea0 0x7f954015a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:00.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.929+0000 7f9544991640 1 -- 172.21.15.40:0/4174361417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9528009490 con 0x7f95400d19b0 2023-10-18T08:56:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.930+0000 7f9544991640 1 --2- 172.21.15.40:0/4174361417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95400d19b0 0x7f95400c5e00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f953000cc80 tx=0x7f953000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.930+0000 7f953d7fa640 1 -- 172.21.15.40:0/4174361417 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953000ce90 con 0x7f95400d19b0 2023-10-18T08:56:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.930+0000 7f953d7fa640 1 -- 172.21.15.40:0/4174361417 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9530012420 con 0x7f95400d19b0 2023-10-18T08:56:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.930+0000 7f953d7fa640 1 -- 172.21.15.40:0/4174361417 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530012810 con 0x7f95400d19b0 2023-10-18T08:56:00.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.930+0000 7f9545993640 1 -- 172.21.15.40:0/4174361417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95400c6490 con 0x7f95400d19b0 2023-10-18T08:56:00.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.930+0000 7f9545993640 1 -- 172.21.15.40:0/4174361417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95400c6a50 con 0x7f95400d19b0 2023-10-18T08:56:00.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.931+0000 7f9545993640 1 -- 172.21.15.40:0/4174361417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9540071da0 con 0x7f95400d19b0 2023-10-18T08:56:00.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.932+0000 7f953d7fa640 1 -- 172.21.15.40:0/4174361417 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f953001b400 con 0x7f95400d19b0 2023-10-18T08:56:00.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.933+0000 7f953d7fa640 1 --2- 172.21.15.40:0/4174361417 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f952006ae70 0x7f952006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:00.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.933+0000 7f953d7fa640 1 -- 172.21.15.40:0/4174361417 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f953008a210 con 0x7f95400d19b0 2023-10-18T08:56:00.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.934+0000 7f953ffff640 1 --2- 172.21.15.40:0/4174361417 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f952006ae70 0x7f952006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:00.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.935+0000 7f953ffff640 1 --2- 172.21.15.40:0/4174361417 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f952006ae70 0x7f952006d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9528036000 tx=0x7f95280046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:00.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:00.938+0000 7f953d7fa640 1 -- 172.21.15.40:0/4174361417 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95300552b0 con 0x7f95400d19b0 2023-10-18T08:56:01.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.134+0000 7f9545993640 1 -- 172.21.15.40:0/4174361417 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9540066e00 con 0x7f95400d19b0 2023-10-18T08:56:01.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.134+0000 7f953d7fa640 1 -- 172.21.15.40:0/4174361417 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f95300588d0 con 0x7f95400d19b0 2023-10-18T08:56:01.137 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:56:01.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.137+0000 7f9545993640 1 -- 172.21.15.40:0/4174361417 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f952006ae70 msgr2=0x7f952006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:01.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.137+0000 7f9545993640 1 --2- 172.21.15.40:0/4174361417 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f952006ae70 0x7f952006d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9528036000 tx=0x7f95280046c0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.137+0000 7f9545993640 1 -- 172.21.15.40:0/4174361417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95400d19b0 msgr2=0x7f95400c5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:01.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.137+0000 7f9545993640 1 --2- 172.21.15.40:0/4174361417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95400d19b0 0x7f95400c5e00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f953000cc80 tx=0x7f953000a430 comp rx=0 tx=0).stop 2023-10-18T08:56:01.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.137+0000 7f9545993640 1 -- 172.21.15.40:0/4174361417 shutdown_connections 2023-10-18T08:56:01.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.137+0000 7f9545993640 1 --2- 172.21.15.40:0/4174361417 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f952006ae70 0x7f952006d330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.137+0000 7f9545993640 1 --2- 172.21.15.40:0/4174361417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f954015eea0 0x7f954015a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.137+0000 7f9545993640 1 --2- 172.21.15.40:0/4174361417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f954015eac0 0x7f95400b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.137+0000 7f9545993640 1 --2- 172.21.15.40:0/4174361417 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95400d19b0 0x7f95400c5e00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.137+0000 7f9545993640 1 -- 172.21.15.40:0/4174361417 >> 172.21.15.40:0/4174361417 conn(0x7f954009a610 msgr2=0x7f95400d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:01.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.138+0000 7f9545993640 1 -- 172.21.15.40:0/4174361417 shutdown_connections 2023-10-18T08:56:01.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.138+0000 7f9545993640 1 -- 172.21.15.40:0/4174361417 wait complete. 2023-10-18T08:56:01.159 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:56:01.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.398+0000 7feaa45f5640 1 -- 172.21.15.40:0/895586953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9c06f570 msgr2=0x7fea9c06f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:01.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.398+0000 7feaa45f5640 1 --2- 172.21.15.40:0/895586953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9c06f570 0x7fea9c06f9e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fea94005fd0 tx=0x7fea9402c0f0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.399+0000 7feaa45f5640 1 -- 172.21.15.40:0/895586953 shutdown_connections 2023-10-18T08:56:01.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.399+0000 7feaa45f5640 1 --2- 172.21.15.40:0/895586953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9c06f570 0x7fea9c06f9e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.399+0000 7feaa45f5640 1 --2- 172.21.15.40:0/895586953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea9c157800 0x7fea9c0b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.399+0000 7feaa45f5640 1 -- 172.21.15.40:0/895586953 >> 172.21.15.40:0/895586953 conn(0x7fea9c069720 msgr2=0x7fea9c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:01.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.399+0000 7feaa45f5640 1 -- 172.21.15.40:0/895586953 shutdown_connections 2023-10-18T08:56:01.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.399+0000 7feaa45f5640 1 -- 172.21.15.40:0/895586953 wait complete. 2023-10-18T08:56:01.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.401+0000 7feaa45f5640 1 Processor -- start 2023-10-18T08:56:01.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.401+0000 7feaa45f5640 1 -- start start 2023-10-18T08:56:01.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.402+0000 7feaa45f5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea9c06d660 0x7fea9c0aad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:01.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.402+0000 7feaa45f5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea9c06f570 0x7fea9c142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:01.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.402+0000 7feaa45f5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9c157800 0x7fea9c142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:01.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.402+0000 7feaa45f5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c1f6510 con 0x7fea9c157800 2023-10-18T08:56:01.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.402+0000 7feaa45f5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c145e50 con 0x7fea9c06f570 2023-10-18T08:56:01.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.402+0000 7feaa45f5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c142b90 con 0x7fea9c06d660 2023-10-18T08:56:01.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.402+0000 7feaa2b92640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9c157800 0x7fea9c142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:01.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.402+0000 7feaa2b92640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9c157800 0x7fea9c142650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52552/0 (socket says 172.21.15.40:52552) 2023-10-18T08:56:01.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.402+0000 7feaa2b92640 1 -- 172.21.15.40:0/3429012923 learned_addr learned my addr 172.21.15.40:0/3429012923 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:01.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.403+0000 7feaa2391640 1 --2- 172.21.15.40:0/3429012923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea9c06d660 0x7fea9c0aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:01.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.403+0000 7feaa1b90640 1 --2- 172.21.15.40:0/3429012923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea9c06f570 0x7fea9c142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:01.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.403+0000 7feaa2b92640 1 -- 172.21.15.40:0/3429012923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea9c06d660 msgr2=0x7fea9c0aad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:01.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.403+0000 7feaa2b92640 1 --2- 172.21.15.40:0/3429012923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea9c06d660 0x7fea9c0aad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.403+0000 7feaa2b92640 1 -- 172.21.15.40:0/3429012923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea9c06f570 msgr2=0x7fea9c142110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:01.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.403+0000 7feaa2b92640 1 --2- 172.21.15.40:0/3429012923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea9c06f570 0x7fea9c142110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.403+0000 7feaa2b92640 1 -- 172.21.15.40:0/3429012923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea94009840 con 0x7fea9c157800 2023-10-18T08:56:01.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.403+0000 7feaa2b92640 1 --2- 172.21.15.40:0/3429012923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9c157800 0x7fea9c142650 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fea94036040 tx=0x7fea94004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:01.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.403+0000 7fea92ffd640 1 -- 172.21.15.40:0/3429012923 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea94004730 con 0x7fea9c157800 2023-10-18T08:56:01.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.403+0000 7fea92ffd640 1 -- 172.21.15.40:0/3429012923 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fea940345a0 con 0x7fea9c157800 2023-10-18T08:56:01.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.403+0000 7fea92ffd640 1 -- 172.21.15.40:0/3429012923 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea940349b0 con 0x7fea9c157800 2023-10-18T08:56:01.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.403+0000 7feaa45f5640 1 -- 172.21.15.40:0/3429012923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9c1f5750 con 0x7fea9c157800 2023-10-18T08:56:01.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.404+0000 7feaa45f5640 1 -- 172.21.15.40:0/3429012923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9c1f5d10 con 0x7fea9c157800 2023-10-18T08:56:01.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.405+0000 7feaa45f5640 1 -- 172.21.15.40:0/3429012923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9c066fd0 con 0x7fea9c157800 2023-10-18T08:56:01.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.406+0000 7fea92ffd640 1 -- 172.21.15.40:0/3429012923 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fea94034b10 con 0x7fea9c157800 2023-10-18T08:56:01.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.407+0000 7fea92ffd640 1 --2- 172.21.15.40:0/3429012923 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea8006ae70 0x7fea8006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:01.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.407+0000 7fea92ffd640 1 -- 172.21.15.40:0/3429012923 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea940ac910 con 0x7fea9c157800 2023-10-18T08:56:01.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.408+0000 7feaa2391640 1 --2- 172.21.15.40:0/3429012923 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea8006ae70 0x7fea8006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:01.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.409+0000 7feaa2391640 1 --2- 172.21.15.40:0/3429012923 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea8006ae70 0x7fea8006d330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea980046b0 tx=0x7fea98009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:01.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.412+0000 7fea92ffd640 1 -- 172.21.15.40:0/3429012923 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea940779b0 con 0x7fea9c157800 2023-10-18T08:56:01.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.609+0000 7feaa45f5640 1 -- 172.21.15.40:0/3429012923 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fea9c0cc810 con 0x7fea9c157800 2023-10-18T08:56:01.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.610+0000 7fea92ffd640 1 -- 172.21.15.40:0/3429012923 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fea940430a0 con 0x7fea9c157800 2023-10-18T08:56:01.613 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:56:01.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.613+0000 7feaa45f5640 1 -- 172.21.15.40:0/3429012923 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea8006ae70 msgr2=0x7fea8006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:01.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.613+0000 7feaa45f5640 1 --2- 172.21.15.40:0/3429012923 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea8006ae70 0x7fea8006d330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea980046b0 tx=0x7fea98009040 comp rx=0 tx=0).stop 2023-10-18T08:56:01.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.613+0000 7feaa45f5640 1 -- 172.21.15.40:0/3429012923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9c157800 msgr2=0x7fea9c142650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:01.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.613+0000 7feaa45f5640 1 --2- 172.21.15.40:0/3429012923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9c157800 0x7fea9c142650 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fea94036040 tx=0x7fea94004270 comp rx=0 tx=0).stop 2023-10-18T08:56:01.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.613+0000 7feaa45f5640 1 -- 172.21.15.40:0/3429012923 shutdown_connections 2023-10-18T08:56:01.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.613+0000 7feaa45f5640 1 --2- 172.21.15.40:0/3429012923 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea8006ae70 0x7fea8006d330 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.613+0000 7feaa45f5640 1 --2- 172.21.15.40:0/3429012923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea9c157800 0x7fea9c142650 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.613+0000 7feaa45f5640 1 --2- 172.21.15.40:0/3429012923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea9c06f570 0x7fea9c142110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.613+0000 7feaa45f5640 1 --2- 172.21.15.40:0/3429012923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea9c06d660 0x7fea9c0aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.613+0000 7feaa45f5640 1 -- 172.21.15.40:0/3429012923 >> 172.21.15.40:0/3429012923 conn(0x7fea9c069720 msgr2=0x7fea9c156ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:01.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.613+0000 7feaa45f5640 1 -- 172.21.15.40:0/3429012923 shutdown_connections 2023-10-18T08:56:01.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.613+0000 7feaa45f5640 1 -- 172.21.15.40:0/3429012923 wait complete. 2023-10-18T08:56:01.634 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2023-10-18T08:56:01.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.865+0000 7f8a1359e640 1 -- 172.21.15.40:0/1019023815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a141420f0 msgr2=0x7f8a140b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:01.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.865+0000 7f8a1359e640 1 --2- 172.21.15.40:0/1019023815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a141420f0 0x7f8a140b0a80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009900 tx=0x7f8a0402be00 comp rx=0 tx=0).stop 2023-10-18T08:56:01.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.865+0000 7f8a1359e640 1 -- 172.21.15.40:0/1019023815 shutdown_connections 2023-10-18T08:56:01.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.865+0000 7f8a1359e640 1 --2- 172.21.15.40:0/1019023815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a141425a0 0x7f8a140c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.865+0000 7f8a1359e640 1 --2- 172.21.15.40:0/1019023815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a141420f0 0x7f8a140b0a80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.865+0000 7f8a1359e640 1 -- 172.21.15.40:0/1019023815 >> 172.21.15.40:0/1019023815 conn(0x7f8a14069720 msgr2=0x7f8a14069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:01.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.865+0000 7f8a1359e640 1 -- 172.21.15.40:0/1019023815 shutdown_connections 2023-10-18T08:56:01.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.865+0000 7f8a1359e640 1 -- 172.21.15.40:0/1019023815 wait complete. 2023-10-18T08:56:01.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.867+0000 7f8a1359e640 1 Processor -- start 2023-10-18T08:56:01.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.867+0000 7f8a1359e640 1 -- start start 2023-10-18T08:56:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.868+0000 7f8a1359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a140c9f90 0x7f8a14140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.868+0000 7f8a1359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a141420f0 0x7f8a1413f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.868+0000 7f8a1359e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a141425a0 0x7f8a14158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.869+0000 7f8a1359e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a14141520 con 0x7f8a140c9f90 2023-10-18T08:56:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.869+0000 7f8a1359e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a14158ca0 con 0x7f8a141420f0 2023-10-18T08:56:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.869+0000 7f8a1359e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a14140910 con 0x7f8a141425a0 2023-10-18T08:56:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.869+0000 7f8a1259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a140c9f90 0x7f8a14140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.869+0000 7f8a1259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a140c9f90 0x7f8a14140230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52570/0 (socket says 172.21.15.40:52570) 2023-10-18T08:56:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.869+0000 7f8a1259c640 1 -- 172.21.15.40:0/1028671350 learned_addr learned my addr 172.21.15.40:0/1028671350 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.869+0000 7f8a0bfff640 1 --2- 172.21.15.40:0/1028671350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a141420f0 0x7f8a1413f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:01.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.869+0000 7f8a12d9d640 1 --2- 172.21.15.40:0/1028671350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a141425a0 0x7f8a14158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:01.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.870+0000 7f8a1259c640 1 -- 172.21.15.40:0/1028671350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a141425a0 msgr2=0x7f8a14158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:01.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.870+0000 7f8a1259c640 1 --2- 172.21.15.40:0/1028671350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a141425a0 0x7f8a14158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.870+0000 7f8a1259c640 1 -- 172.21.15.40:0/1028671350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a141420f0 msgr2=0x7f8a1413f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:01.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.870+0000 7f8a1259c640 1 --2- 172.21.15.40:0/1028671350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a141420f0 0x7f8a1413f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.870+0000 7f8a1259c640 1 -- 172.21.15.40:0/1028671350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a04009560 con 0x7f8a140c9f90 2023-10-18T08:56:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.870+0000 7f8a1259c640 1 --2- 172.21.15.40:0/1028671350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a140c9f90 0x7f8a14140230 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009230 tx=0x7f8a0402dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.871+0000 7f8a0affd640 1 -- 172.21.15.40:0/1028671350 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04003800 con 0x7f8a140c9f90 2023-10-18T08:56:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.871+0000 7f8a0affd640 1 -- 172.21.15.40:0/1028671350 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8a04003a50 con 0x7f8a140c9f90 2023-10-18T08:56:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.871+0000 7f8a0affd640 1 -- 172.21.15.40:0/1028671350 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04003e00 con 0x7f8a140c9f90 2023-10-18T08:56:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.871+0000 7f8a1359e640 1 -- 172.21.15.40:0/1028671350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a14140bf0 con 0x7f8a140c9f90 2023-10-18T08:56:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.871+0000 7f8a1359e640 1 -- 172.21.15.40:0/1028671350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a141411b0 con 0x7f8a140c9f90 2023-10-18T08:56:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.873+0000 7f8a1359e640 1 -- 172.21.15.40:0/1028671350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a14066e00 con 0x7f8a140c9f90 2023-10-18T08:56:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.874+0000 7f8a0affd640 1 -- 172.21.15.40:0/1028671350 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8a0403e450 con 0x7f8a140c9f90 2023-10-18T08:56:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.875+0000 7f8a0affd640 1 --2- 172.21.15.40:0/1028671350 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f89f406ae40 0x7f89f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.875+0000 7f8a0affd640 1 -- 172.21.15.40:0/1028671350 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8a040ac200 con 0x7f8a140c9f90 2023-10-18T08:56:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.879+0000 7f8a0bfff640 1 --2- 172.21.15.40:0/1028671350 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f89f406ae40 0x7f89f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:01.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.880+0000 7f8a0affd640 1 -- 172.21.15.40:0/1028671350 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a040772c0 con 0x7f8a140c9f90 2023-10-18T08:56:01.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:01.881+0000 7f8a0bfff640 1 --2- 172.21.15.40:0/1028671350 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f89f406ae40 0x7f89f406d300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0058c0 tx=0x7f89fc005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:02.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:02.147+0000 7f8a1359e640 1 -- 172.21.15.40:0/1028671350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7f8a1406d920 con 0x7f8a140c9f90 2023-10-18T08:56:03.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.047+0000 7f8a0affd640 1 -- 172.21.15.40:0/1028671350 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v63) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8a0407a8e0 con 0x7f8a140c9f90 2023-10-18T08:56:03.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.051+0000 7f8a1359e640 1 -- 172.21.15.40:0/1028671350 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f89f406ae40 msgr2=0x7f89f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:03.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.051+0000 7f8a1359e640 1 --2- 172.21.15.40:0/1028671350 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f89f406ae40 0x7f89f406d300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0058c0 tx=0x7f89fc005850 comp rx=0 tx=0).stop 2023-10-18T08:56:03.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.051+0000 7f8a1359e640 1 -- 172.21.15.40:0/1028671350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a140c9f90 msgr2=0x7f8a14140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:03.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.051+0000 7f8a1359e640 1 --2- 172.21.15.40:0/1028671350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a140c9f90 0x7f8a14140230 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009230 tx=0x7f8a0402dad0 comp rx=0 tx=0).stop 2023-10-18T08:56:03.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.051+0000 7f8a1359e640 1 -- 172.21.15.40:0/1028671350 shutdown_connections 2023-10-18T08:56:03.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.051+0000 7f8a1359e640 1 --2- 172.21.15.40:0/1028671350 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f89f406ae40 0x7f89f406d300 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:03.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.051+0000 7f8a1359e640 1 --2- 172.21.15.40:0/1028671350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a141425a0 0x7f8a14158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:03.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.051+0000 7f8a1359e640 1 --2- 172.21.15.40:0/1028671350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a141420f0 0x7f8a1413f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:03.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.051+0000 7f8a1359e640 1 --2- 172.21.15.40:0/1028671350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a140c9f90 0x7f8a14140230 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:03.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.051+0000 7f8a1359e640 1 -- 172.21.15.40:0/1028671350 >> 172.21.15.40:0/1028671350 conn(0x7f8a14069720 msgr2=0x7f8a1406f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:03.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.051+0000 7f8a1359e640 1 -- 172.21.15.40:0/1028671350 shutdown_connections 2023-10-18T08:56:03.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.051+0000 7f8a1359e640 1 -- 172.21.15.40:0/1028671350 wait complete. 2023-10-18T08:56:03.076 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-18T08:56:03.303 INFO:teuthology.orchestra.run.smithi040.stdout:{"osd_objectstore":"memstore"} 2023-10-18T08:56:03.304 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-18T08:56:03.304 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T08:56:03.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.534+0000 7fcf52f47640 1 -- 172.21.15.40:0/4016231291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf380047c0 msgr2=0x7fcf38004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:03.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.534+0000 7fcf52f47640 1 --2- 172.21.15.40:0/4016231291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf380047c0 0x7fcf38004c40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c15adb0 tx=0x7fcf380057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:03.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.534+0000 7fcf52746640 1 -- 172.21.15.40:0/4016231291 reap_dead start 2023-10-18T08:56:03.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.534+0000 7fcf52f47640 1 -- 172.21.15.40:0/4016231291 shutdown_connections 2023-10-18T08:56:03.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.534+0000 7fcf52f47640 1 -- 172.21.15.40:0/4016231291 >> 172.21.15.40:0/4016231291 conn(0x7fcf4c0aed70 msgr2=0x7fcf4c0af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:03.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.534+0000 7fcf52f47640 1 -- 172.21.15.40:0/4016231291 shutdown_connections 2023-10-18T08:56:03.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.534+0000 7fcf52f47640 1 -- 172.21.15.40:0/4016231291 wait complete. 2023-10-18T08:56:03.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.536+0000 7fcf52f47640 1 Processor -- start 2023-10-18T08:56:03.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.536+0000 7fcf52f47640 1 -- start start 2023-10-18T08:56:03.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.537+0000 7fcf52f47640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf380047c0 0x7fcf4c130750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:03.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.537+0000 7fcf52f47640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf4c12d3f0 0x7fcf4c0a9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:03.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.537+0000 7fcf52f47640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf4c140830 0x7fcf4c140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:03.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.537+0000 7fcf52f47640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c1411f0 con 0x7fcf4c12d3f0 2023-10-18T08:56:03.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.537+0000 7fcf52f47640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c130c90 con 0x7fcf4c140830 2023-10-18T08:56:03.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.537+0000 7fcf52f47640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c12e050 con 0x7fcf380047c0 2023-10-18T08:56:03.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.537+0000 7fcf51744640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf4c12d3f0 0x7fcf4c0a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:03.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.537+0000 7fcf51744640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf4c12d3f0 0x7fcf4c0a9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52584/0 (socket says 172.21.15.40:52584) 2023-10-18T08:56:03.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.537+0000 7fcf51744640 1 -- 172.21.15.40:0/1252650707 learned_addr learned my addr 172.21.15.40:0/1252650707 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:03.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.538+0000 7fcf51744640 1 -- 172.21.15.40:0/1252650707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf380047c0 msgr2=0x7fcf4c130750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:03.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.538+0000 7fcf51744640 1 --2- 172.21.15.40:0/1252650707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf380047c0 0x7fcf4c130750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:03.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.538+0000 7fcf51744640 1 -- 172.21.15.40:0/1252650707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf4c140830 msgr2=0x7fcf4c140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:03.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.538+0000 7fcf51744640 1 --2- 172.21.15.40:0/1252650707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf4c140830 0x7fcf4c140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:03.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.538+0000 7fcf51744640 1 -- 172.21.15.40:0/1252650707 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c12e330 con 0x7fcf4c12d3f0 2023-10-18T08:56:03.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.538+0000 7fcf51744640 1 --2- 172.21.15.40:0/1252650707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf4c12d3f0 0x7fcf4c0a9380 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c02c650 tx=0x7fcf3c036690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:03.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.538+0000 7fcf427fc640 1 -- 172.21.15.40:0/1252650707 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c036c90 con 0x7fcf4c12d3f0 2023-10-18T08:56:03.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.538+0000 7fcf427fc640 1 -- 172.21.15.40:0/1252650707 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcf3c004030 con 0x7fcf4c12d3f0 2023-10-18T08:56:03.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.538+0000 7fcf427fc640 1 -- 172.21.15.40:0/1252650707 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c004440 con 0x7fcf4c12d3f0 2023-10-18T08:56:03.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.538+0000 7fcf52f47640 1 -- 172.21.15.40:0/1252650707 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf4c12e610 con 0x7fcf4c12d3f0 2023-10-18T08:56:03.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.539+0000 7fcf52f47640 1 -- 172.21.15.40:0/1252650707 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf4c12ebd0 con 0x7fcf4c12d3f0 2023-10-18T08:56:03.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.540+0000 7fcf52f47640 1 -- 172.21.15.40:0/1252650707 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf4c066e00 con 0x7fcf4c12d3f0 2023-10-18T08:56:03.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.541+0000 7fcf427fc640 1 -- 172.21.15.40:0/1252650707 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcf3c0035f0 con 0x7fcf4c12d3f0 2023-10-18T08:56:03.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.542+0000 7fcf427fc640 1 --2- 172.21.15.40:0/1252650707 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf3006ae70 0x7fcf3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:03.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.542+0000 7fcf427fc640 1 -- 172.21.15.40:0/1252650707 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fcf3c0ab7c0 con 0x7fcf4c12d3f0 2023-10-18T08:56:03.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.543+0000 7fcf51f45640 1 --2- 172.21.15.40:0/1252650707 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf3006ae70 0x7fcf3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:03.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.544+0000 7fcf51f45640 1 --2- 172.21.15.40:0/1252650707 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf3006ae70 0x7fcf3006d330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcf48005d00 tx=0x7fcf48005ed0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:03.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.547+0000 7fcf427fc640 1 -- 172.21.15.40:0/1252650707 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcf3c076750 con 0x7fcf4c12d3f0 2023-10-18T08:56:03.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.742+0000 7fcf52f47640 1 -- 172.21.15.40:0/1252650707 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7fcf4c15a3f0 con 0x7fcf4c12d3f0 2023-10-18T08:56:03.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.743+0000 7fcf427fc640 1 -- 172.21.15.40:0/1252650707 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v22) v1 ==== 59+0+1452 (secure 0 0 0) 0x7fcf3c0325c0 con 0x7fcf4c12d3f0 2023-10-18T08:56:03.745 INFO:teuthology.orchestra.run.smithi040.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T08:56:03.746 INFO:teuthology.orchestra.run.smithi040.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T08:56:03.746 INFO:teuthology.orchestra.run.smithi040.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T08:56:03.746 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds 20/20 2023-10-18T08:56:03.746 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T08:56:03.747 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_ms 1/1 2023-10-18T08:56:03.747 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T08:56:03.747 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T08:56:03.747 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T08:56:03.747 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_size 100 2023-10-18T08:56:03.748 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_frag true 2023-10-18T08:56:03.748 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_scatterstat true 2023-10-18T08:56:03.748 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T08:56:03.748 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_verify_scatter true 2023-10-18T08:56:03.748 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T08:56:03.748 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T08:56:03.749 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T08:56:03.749 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced client_mount_timeout 600 2023-10-18T08:56:03.749 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_client 20/20 2023-10-18T08:56:03.749 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_ms 1/1 2023-10-18T08:56:03.749 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T08:56:03.750 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T08:56:03.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.746+0000 7fcf52f47640 1 -- 172.21.15.40:0/1252650707 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf3006ae70 msgr2=0x7fcf3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:03.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.746+0000 7fcf52f47640 1 --2- 172.21.15.40:0/1252650707 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf3006ae70 0x7fcf3006d330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcf48005d00 tx=0x7fcf48005ed0 comp rx=0 tx=0).stop 2023-10-18T08:56:03.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.746+0000 7fcf52f47640 1 -- 172.21.15.40:0/1252650707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf4c12d3f0 msgr2=0x7fcf4c0a9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:03.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.746+0000 7fcf52f47640 1 --2- 172.21.15.40:0/1252650707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf4c12d3f0 0x7fcf4c0a9380 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c02c650 tx=0x7fcf3c036690 comp rx=0 tx=0).stop 2023-10-18T08:56:03.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.746+0000 7fcf52f47640 1 -- 172.21.15.40:0/1252650707 shutdown_connections 2023-10-18T08:56:03.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.746+0000 7fcf52f47640 1 --2- 172.21.15.40:0/1252650707 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf3006ae70 0x7fcf3006d330 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:03.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.746+0000 7fcf52f47640 1 --2- 172.21.15.40:0/1252650707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf4c140830 0x7fcf4c140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:03.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.746+0000 7fcf52f47640 1 --2- 172.21.15.40:0/1252650707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf4c12d3f0 0x7fcf4c0a9380 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:03.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.746+0000 7fcf52f47640 1 --2- 172.21.15.40:0/1252650707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf380047c0 0x7fcf4c130750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:03.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.746+0000 7fcf52f47640 1 -- 172.21.15.40:0/1252650707 >> 172.21.15.40:0/1252650707 conn(0x7fcf4c0aed70 msgr2=0x7fcf4c141930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:03.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.746+0000 7fcf52f47640 1 -- 172.21.15.40:0/1252650707 shutdown_connections 2023-10-18T08:56:03.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:03.746+0000 7fcf52f47640 1 -- 172.21.15.40:0/1252650707 wait complete. 2023-10-18T08:56:03.768 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T08:56:04.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.000+0000 7f89556c3640 1 -- 172.21.15.40:0/312185495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89500c9620 msgr2=0x7f895006bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:04.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.000+0000 7f89556c3640 1 --2- 172.21.15.40:0/312185495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89500c9620 0x7f895006bbc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8940005fd0 tx=0x7f894002c620 comp rx=0 tx=0).stop 2023-10-18T08:56:04.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.000+0000 7f89556c3640 1 -- 172.21.15.40:0/312185495 shutdown_connections 2023-10-18T08:56:04.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.000+0000 7f89556c3640 1 --2- 172.21.15.40:0/312185495 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f895015af70 0x7f895006c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:04.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.000+0000 7f89556c3640 1 --2- 172.21.15.40:0/312185495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89500c9620 0x7f895006bbc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:04.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.000+0000 7f89556c3640 1 -- 172.21.15.40:0/312185495 >> 172.21.15.40:0/312185495 conn(0x7f895006ca10 msgr2=0x7f895006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:04.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.000+0000 7f89556c3640 1 -- 172.21.15.40:0/312185495 shutdown_connections 2023-10-18T08:56:04.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.000+0000 7f89556c3640 1 -- 172.21.15.40:0/312185495 wait complete. 2023-10-18T08:56:04.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.002+0000 7f89556c3640 1 Processor -- start 2023-10-18T08:56:04.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.002+0000 7f89556c3640 1 -- start start 2023-10-18T08:56:04.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.003+0000 7f89556c3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8950070b60 0x7f89501561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:04.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.003+0000 7f89556c3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89500c9620 0x7f8950158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:04.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.003+0000 7f89556c3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f895015af70 0x7f8950158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:04.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.003+0000 7f89556c3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8950158a90 con 0x7f8950070b60 2023-10-18T08:56:04.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.003+0000 7f89556c3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8950158d20 con 0x7f89500c9620 2023-10-18T08:56:04.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.003+0000 7f89556c3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8950159000 con 0x7f895015af70 2023-10-18T08:56:04.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.003+0000 7f894ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8950070b60 0x7f89501561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:04.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.003+0000 7f894ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8950070b60 0x7f89501561e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52610/0 (socket says 172.21.15.40:52610) 2023-10-18T08:56:04.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.003+0000 7f894ffff640 1 -- 172.21.15.40:0/573968634 learned_addr learned my addr 172.21.15.40:0/573968634 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:04.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.004+0000 7f894ffff640 1 -- 172.21.15.40:0/573968634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f895015af70 msgr2=0x7f8950158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:04.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.004+0000 7f8954ec2640 1 --2- 172.21.15.40:0/573968634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f895015af70 0x7f8950158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:04.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.004+0000 7f894ffff640 1 --2- 172.21.15.40:0/573968634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f895015af70 0x7f8950158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:04.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.004+0000 7f894ffff640 1 -- 172.21.15.40:0/573968634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89500c9620 msgr2=0x7f8950158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:04.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.004+0000 7f894f7fe640 1 --2- 172.21.15.40:0/573968634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89500c9620 0x7f8950158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:04.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.004+0000 7f894ffff640 1 --2- 172.21.15.40:0/573968634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89500c9620 0x7f8950158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:04.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.004+0000 7f894ffff640 1 -- 172.21.15.40:0/573968634 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8940009df0 con 0x7f8950070b60 2023-10-18T08:56:04.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.004+0000 7f894ffff640 1 --2- 172.21.15.40:0/573968634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8950070b60 0x7f89501561e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8940006010 tx=0x7f8940009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:04.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.004+0000 7f894cff9640 1 -- 172.21.15.40:0/573968634 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8940004c70 con 0x7f8950070b60 2023-10-18T08:56:04.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.004+0000 7f894cff9640 1 -- 172.21.15.40:0/573968634 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8940004ee0 con 0x7f8950070b60 2023-10-18T08:56:04.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.004+0000 7f894cff9640 1 -- 172.21.15.40:0/573968634 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894003e6d0 con 0x7f8950070b60 2023-10-18T08:56:04.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.004+0000 7f89556c3640 1 -- 172.21.15.40:0/573968634 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89501592e0 con 0x7f8950070b60 2023-10-18T08:56:04.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.005+0000 7f89556c3640 1 -- 172.21.15.40:0/573968634 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89501598a0 con 0x7f8950070b60 2023-10-18T08:56:04.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.006+0000 7f89556c3640 1 -- 172.21.15.40:0/573968634 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8950066e00 con 0x7f8950070b60 2023-10-18T08:56:04.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.007+0000 7f894cff9640 1 -- 172.21.15.40:0/573968634 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f894003e830 con 0x7f8950070b60 2023-10-18T08:56:04.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.008+0000 7f894cff9640 1 --2- 172.21.15.40:0/573968634 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f893c06ae40 0x7f893c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:04.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.008+0000 7f894cff9640 1 -- 172.21.15.40:0/573968634 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f89400ac5a0 con 0x7f8950070b60 2023-10-18T08:56:04.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.008+0000 7f894f7fe640 1 --2- 172.21.15.40:0/573968634 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f893c06ae40 0x7f893c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:04.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.009+0000 7f894f7fe640 1 --2- 172.21.15.40:0/573968634 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f893c06ae40 0x7f893c06d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f89501f5060 tx=0x7f8944008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:04.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.012+0000 7f894cff9640 1 -- 172.21.15.40:0/573968634 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8940077660 con 0x7f8950070b60 2023-10-18T08:56:04.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:56:04.222 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:56:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.213+0000 7f89556c3640 1 -- 172.21.15.40:0/573968634 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f89500c9b10 con 0x7f8950070b60 2023-10-18T08:56:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.218+0000 7f894cff9640 1 -- 172.21.15.40:0/573968634 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v23)=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7f894007ac80 con 0x7f8950070b60 2023-10-18T08:56:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.221+0000 7f89556c3640 1 -- 172.21.15.40:0/573968634 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f893c06ae40 msgr2=0x7f893c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.221+0000 7f89556c3640 1 --2- 172.21.15.40:0/573968634 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f893c06ae40 0x7f893c06d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f89501f5060 tx=0x7f8944008040 comp rx=0 tx=0).stop 2023-10-18T08:56:04.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.221+0000 7f89556c3640 1 -- 172.21.15.40:0/573968634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8950070b60 msgr2=0x7f89501561e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.221+0000 7f89556c3640 1 --2- 172.21.15.40:0/573968634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8950070b60 0x7f89501561e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8940006010 tx=0x7f8940009890 comp rx=0 tx=0).stop 2023-10-18T08:56:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.221+0000 7f89556c3640 1 -- 172.21.15.40:0/573968634 shutdown_connections 2023-10-18T08:56:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.221+0000 7f89556c3640 1 --2- 172.21.15.40:0/573968634 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f893c06ae40 0x7f893c06d300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.221+0000 7f89556c3640 1 --2- 172.21.15.40:0/573968634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f895015af70 0x7f8950158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.221+0000 7f89556c3640 1 --2- 172.21.15.40:0/573968634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89500c9620 0x7f8950158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:04.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.221+0000 7f89556c3640 1 --2- 172.21.15.40:0/573968634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8950070b60 0x7f89501561e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:04.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.221+0000 7f89556c3640 1 -- 172.21.15.40:0/573968634 >> 172.21.15.40:0/573968634 conn(0x7f895006ca10 msgr2=0x7f89500ccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:04.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.221+0000 7f89556c3640 1 -- 172.21.15.40:0/573968634 shutdown_connections 2023-10-18T08:56:04.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.221+0000 7f89556c3640 1 -- 172.21.15.40:0/573968634 wait complete. 2023-10-18T08:56:04.243 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T08:56:04.244 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T08:56:04.411 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T08:56:04.538 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T08:56:04.539 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:56:04.540 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:04.607 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:56:04.608 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:04.678 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:56:04.678 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:04.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:04.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.974+0000 7ff9e4d3c640 1 -- 172.21.15.40:0/2885865787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9e00b15b0 msgr2=0x7ff9e00b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:04.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.974+0000 7ff9e4d3c640 1 --2- 172.21.15.40:0/2885865787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9e00b15b0 0x7ff9e00b0630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0005fd0 tx=0x7ff9d002be10 comp rx=0 tx=0).stop 2023-10-18T08:56:04.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.974+0000 7ff9e4d3c640 1 -- 172.21.15.40:0/2885865787 shutdown_connections 2023-10-18T08:56:04.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.974+0000 7ff9e4d3c640 1 --2- 172.21.15.40:0/2885865787 >> v2:172.21.15.144:3300/0 conn(0x7ff9e00b1e10 0x7ff9e00c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:04.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.974+0000 7ff9e4d3c640 1 --2- 172.21.15.40:0/2885865787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9e00b15b0 0x7ff9e00b0630 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:04.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.974+0000 7ff9e4d3c640 1 -- 172.21.15.40:0/2885865787 >> 172.21.15.40:0/2885865787 conn(0x7ff9e0069720 msgr2=0x7ff9e0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:04.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.974+0000 7ff9e4d3c640 1 -- 172.21.15.40:0/2885865787 shutdown_connections 2023-10-18T08:56:04.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.974+0000 7ff9e4d3c640 1 -- 172.21.15.40:0/2885865787 wait complete. 2023-10-18T08:56:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.976+0000 7ff9e4d3c640 1 Processor -- start 2023-10-18T08:56:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.976+0000 7ff9e4d3c640 1 -- start start 2023-10-18T08:56:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.977+0000 7ff9e4d3c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00b15b0 0x7ff9e01524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.977+0000 7ff9e4d3c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9e00b1e10 0x7ff9e014c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.977+0000 7ff9e4d3c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9e00b2270 0x7ff9e01e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.977+0000 7ff9e4d3c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e01520d0 con 0x7ff9e00b15b0 2023-10-18T08:56:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.977+0000 7ff9e4d3c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e01e1890 con 0x7ff9e00b2270 2023-10-18T08:56:04.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.977+0000 7ff9e4d3c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e0151a10 con 0x7ff9e00b1e10 2023-10-18T08:56:04.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.977+0000 7ff9df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00b15b0 0x7ff9e01524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:04.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.977+0000 7ff9df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00b15b0 0x7ff9e01524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33996/0 (socket says 172.21.15.40:33996) 2023-10-18T08:56:04.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.977+0000 7ff9df7fe640 1 -- 172.21.15.40:0/1520575145 learned_addr learned my addr 172.21.15.40:0/1520575145 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:04.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.978+0000 7ff9deffd640 1 --2- 172.21.15.40:0/1520575145 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9e00b1e10 0x7ff9e014c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:04.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.978+0000 7ff9df7fe640 1 -- 172.21.15.40:0/1520575145 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9e00b1e10 msgr2=0x7ff9e014c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:04.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.978+0000 7ff9df7fe640 1 --2- 172.21.15.40:0/1520575145 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9e00b1e10 0x7ff9e014c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:04.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.978+0000 7ff9dffff640 1 --2- 172.21.15.40:0/1520575145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9e00b2270 0x7ff9e01e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:04.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.978+0000 7ff9df7fe640 1 -- 172.21.15.40:0/1520575145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9e00b2270 msgr2=0x7ff9e01e0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:04.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.978+0000 7ff9df7fe640 1 --2- 172.21.15.40:0/1520575145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9e00b2270 0x7ff9e01e0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:04.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.978+0000 7ff9df7fe640 1 -- 172.21.15.40:0/1520575145 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d0009560 con 0x7ff9e00b15b0 2023-10-18T08:56:04.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.978+0000 7ff9dffff640 1 --2- 172.21.15.40:0/1520575145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9e00b2270 0x7ff9e01e0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:04.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.978+0000 7ff9df7fe640 1 --2- 172.21.15.40:0/1520575145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00b15b0 0x7ff9e01524f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009850 tx=0x7ff9d002da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:04.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.978+0000 7ff9c7fff640 1 -- 172.21.15.40:0/1520575145 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d002de50 con 0x7ff9e00b15b0 2023-10-18T08:56:04.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.979+0000 7ff9c7fff640 1 -- 172.21.15.40:0/1520575145 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff9d00036e0 con 0x7ff9e00b15b0 2023-10-18T08:56:04.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.979+0000 7ff9e4d3c640 1 -- 172.21.15.40:0/1520575145 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e0151cf0 con 0x7ff9e00b15b0 2023-10-18T08:56:04.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.979+0000 7ff9c7fff640 1 -- 172.21.15.40:0/1520575145 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0003a90 con 0x7ff9e00b15b0 2023-10-18T08:56:04.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.979+0000 7ff9e4d3c640 1 -- 172.21.15.40:0/1520575145 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e014ce10 con 0x7ff9e00b15b0 2023-10-18T08:56:04.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.980+0000 7ff9e4d3c640 1 -- 172.21.15.40:0/1520575145 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e0066e00 con 0x7ff9e00b15b0 2023-10-18T08:56:04.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.981+0000 7ff9c7fff640 1 -- 172.21.15.40:0/1520575145 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff9d0003bf0 con 0x7ff9e00b15b0 2023-10-18T08:56:04.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.982+0000 7ff9c7fff640 1 --2- 172.21.15.40:0/1520575145 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9cc06ae40 0x7ff9cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:04.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.982+0000 7ff9c7fff640 1 -- 172.21.15.40:0/1520575145 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff9d00ac1f0 con 0x7ff9e00b15b0 2023-10-18T08:56:04.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.982+0000 7ff9deffd640 1 --2- 172.21.15.40:0/1520575145 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9cc06ae40 0x7ff9cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:04.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.983+0000 7ff9deffd640 1 --2- 172.21.15.40:0/1520575145 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9cc06ae40 0x7ff9cc06d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0154bb0 tx=0x7ff9d4005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:04.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:04.986+0000 7ff9c7fff640 1 -- 172.21.15.40:0/1520575145 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff9d0077330 con 0x7ff9e00b15b0 2023-10-18T08:56:05.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.185+0000 7ff9e4d3c640 1 -- 172.21.15.40:0/1520575145 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9e00caf90 con 0x7ff9e00b15b0 2023-10-18T08:56:05.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.186+0000 7ff9c7fff640 1 -- 172.21.15.40:0/1520575145 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16766 (secure 0 0 0) 0x7ff9d007a950 con 0x7ff9e00b15b0 2023-10-18T08:56:05.189 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:05.189 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:02.868692+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:05.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.190+0000 7ff9e4d3c640 1 -- 172.21.15.40:0/1520575145 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9cc06ae40 msgr2=0x7ff9cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:05.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.190+0000 7ff9e4d3c640 1 --2- 172.21.15.40:0/1520575145 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9cc06ae40 0x7ff9cc06d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0154bb0 tx=0x7ff9d4005f90 comp rx=0 tx=0).stop 2023-10-18T08:56:05.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.190+0000 7ff9e4d3c640 1 -- 172.21.15.40:0/1520575145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00b15b0 msgr2=0x7ff9e01524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.190+0000 7ff9e4d3c640 1 --2- 172.21.15.40:0/1520575145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00b15b0 0x7ff9e01524f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009850 tx=0x7ff9d002da50 comp rx=0 tx=0).stop 2023-10-18T08:56:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.190+0000 7ff9e4d3c640 1 -- 172.21.15.40:0/1520575145 shutdown_connections 2023-10-18T08:56:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.190+0000 7ff9e4d3c640 1 --2- 172.21.15.40:0/1520575145 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9cc06ae40 0x7ff9cc06d300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.190+0000 7ff9e4d3c640 1 --2- 172.21.15.40:0/1520575145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9e00b2270 0x7ff9e01e0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.190+0000 7ff9e4d3c640 1 --2- 172.21.15.40:0/1520575145 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9e00b1e10 0x7ff9e014c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.190+0000 7ff9e4d3c640 1 --2- 172.21.15.40:0/1520575145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e00b15b0 0x7ff9e01524f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.190+0000 7ff9e4d3c640 1 -- 172.21.15.40:0/1520575145 >> 172.21.15.40:0/1520575145 conn(0x7ff9e0069720 msgr2=0x7ff9e014da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:05.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.190+0000 7ff9e4d3c640 1 -- 172.21.15.40:0/1520575145 shutdown_connections 2023-10-18T08:56:05.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.190+0000 7ff9e4d3c640 1 -- 172.21.15.40:0/1520575145 wait complete. 2023-10-18T08:56:05.212 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:55:13.210788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:55:13.772849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:56:05.213 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:56:05.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.459+0000 7f75d1a35640 1 -- 172.21.15.40:0/1625489927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75bc005180 msgr2=0x7f75bc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:05.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.459+0000 7f75d1a35640 1 --2- 172.21.15.40:0/1625489927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75bc005180 0x7f75bc00b980 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f75c4001d80 tx=0x7f75bc0057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.459+0000 7f75d1a35640 1 -- 172.21.15.40:0/1625489927 shutdown_connections 2023-10-18T08:56:05.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.459+0000 7f75d1a35640 1 --2- 172.21.15.40:0/1625489927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75bc005180 0x7f75bc00b980 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.459+0000 7f75d1a35640 1 -- 172.21.15.40:0/1625489927 >> 172.21.15.40:0/1625489927 conn(0x7f75cc069720 msgr2=0x7f75cc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:05.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.459+0000 7f75d1a35640 1 -- 172.21.15.40:0/1625489927 shutdown_connections 2023-10-18T08:56:05.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.459+0000 7f75d1a35640 1 -- 172.21.15.40:0/1625489927 wait complete. 2023-10-18T08:56:05.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.462+0000 7f75d1a35640 1 Processor -- start 2023-10-18T08:56:05.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.462+0000 7f75d1a35640 1 -- start start 2023-10-18T08:56:05.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.463+0000 7f75d1a35640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75bc005180 0x7f75cc139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:05.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.463+0000 7f75d1a35640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75cc06ff20 0x7f75cc13c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:05.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.463+0000 7f75d1a35640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75cc0703a0 0x7f75cc13cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:05.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.463+0000 7f75d1a35640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc13ac90 con 0x7f75cc06ff20 2023-10-18T08:56:05.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.463+0000 7f75d1a35640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc13d4c0 con 0x7f75bc005180 2023-10-18T08:56:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.463+0000 7f75d1a35640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc14bfa0 con 0x7f75cc0703a0 2023-10-18T08:56:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.464+0000 7f75cbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75cc06ff20 0x7f75cc13c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.464+0000 7f75cbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75cc06ff20 0x7f75cc13c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34022/0 (socket says 172.21.15.40:34022) 2023-10-18T08:56:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.464+0000 7f75cbfff640 1 -- 172.21.15.40:0/3230686001 learned_addr learned my addr 172.21.15.40:0/3230686001 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.464+0000 7f75cbfff640 1 -- 172.21.15.40:0/3230686001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75cc0703a0 msgr2=0x7f75cc13cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.464+0000 7f75d1234640 1 --2- 172.21.15.40:0/3230686001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75cc0703a0 0x7f75cc13cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:05.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.464+0000 7f75cbfff640 1 --2- 172.21.15.40:0/3230686001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75cc0703a0 0x7f75cc13cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.464+0000 7f75cbfff640 1 -- 172.21.15.40:0/3230686001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75bc005180 msgr2=0x7f75cc139e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:05.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.464+0000 7f75d0a33640 1 --2- 172.21.15.40:0/3230686001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75bc005180 0x7f75cc139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:05.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.464+0000 7f75cbfff640 1 --2- 172.21.15.40:0/3230686001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75bc005180 0x7f75cc139e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.464+0000 7f75cbfff640 1 -- 172.21.15.40:0/3230686001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc14c280 con 0x7f75cc06ff20 2023-10-18T08:56:05.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.464+0000 7f75d0a33640 1 --2- 172.21.15.40:0/3230686001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75bc005180 0x7f75cc139e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:05.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.465+0000 7f75cbfff640 1 --2- 172.21.15.40:0/3230686001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75cc06ff20 0x7f75cc13c5f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f75b8010a10 tx=0x7f75b8010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:05.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.465+0000 7f75c97fa640 1 -- 172.21.15.40:0/3230686001 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b80096a0 con 0x7f75cc06ff20 2023-10-18T08:56:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.465+0000 7f75c97fa640 1 -- 172.21.15.40:0/3230686001 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f75b800ece0 con 0x7f75cc06ff20 2023-10-18T08:56:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.465+0000 7f75c97fa640 1 -- 172.21.15.40:0/3230686001 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b8005280 con 0x7f75cc06ff20 2023-10-18T08:56:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.465+0000 7f75d1a35640 1 -- 172.21.15.40:0/3230686001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75cc13a5d0 con 0x7f75cc06ff20 2023-10-18T08:56:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.465+0000 7f75d1a35640 1 -- 172.21.15.40:0/3230686001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc14bc30 con 0x7f75cc06ff20 2023-10-18T08:56:05.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.466+0000 7f75d1a35640 1 -- 172.21.15.40:0/3230686001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75cc13add0 con 0x7f75cc06ff20 2023-10-18T08:56:05.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.467+0000 7f75c97fa640 1 -- 172.21.15.40:0/3230686001 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f75b80053e0 con 0x7f75cc06ff20 2023-10-18T08:56:05.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.468+0000 7f75c97fa640 1 --2- 172.21.15.40:0/3230686001 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f75b006ae70 0x7f75b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:05.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.468+0000 7f75c97fa640 1 -- 172.21.15.40:0/3230686001 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f75b8016070 con 0x7f75cc06ff20 2023-10-18T08:56:05.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.469+0000 7f75d0a33640 1 --2- 172.21.15.40:0/3230686001 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f75b006ae70 0x7f75b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:05.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.470+0000 7f75d0a33640 1 --2- 172.21.15.40:0/3230686001 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f75b006ae70 0x7f75b006d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f75c0001200 tx=0x7f75c0000f50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:05.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.472+0000 7f75c97fa640 1 -- 172.21.15.40:0/3230686001 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f75cc13add0 con 0x7f75cc06ff20 2023-10-18T08:56:05.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.666+0000 7f75d1a35640 1 -- 172.21.15.40:0/3230686001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75cc13add0 con 0x7f75cc06ff20 2023-10-18T08:56:05.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.666+0000 7f75c97fa640 1 -- 172.21.15.40:0/3230686001 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f75cc13add0 con 0x7f75cc06ff20 2023-10-18T08:56:05.669 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:56:05.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.669+0000 7f75d1a35640 1 -- 172.21.15.40:0/3230686001 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f75b006ae70 msgr2=0x7f75b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:05.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.669+0000 7f75d1a35640 1 --2- 172.21.15.40:0/3230686001 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f75b006ae70 0x7f75b006d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f75c0001200 tx=0x7f75c0000f50 comp rx=0 tx=0).stop 2023-10-18T08:56:05.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.669+0000 7f75d1a35640 1 -- 172.21.15.40:0/3230686001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75cc06ff20 msgr2=0x7f75cc13c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:05.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.669+0000 7f75d1a35640 1 --2- 172.21.15.40:0/3230686001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75cc06ff20 0x7f75cc13c5f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f75b8010a10 tx=0x7f75b8010ed0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.669+0000 7f75d1a35640 1 -- 172.21.15.40:0/3230686001 shutdown_connections 2023-10-18T08:56:05.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.669+0000 7f75d1a35640 1 --2- 172.21.15.40:0/3230686001 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f75b006ae70 0x7f75b006d330 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.669+0000 7f75d1a35640 1 --2- 172.21.15.40:0/3230686001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75cc0703a0 0x7f75cc13cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.669+0000 7f75d1a35640 1 --2- 172.21.15.40:0/3230686001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75cc06ff20 0x7f75cc13c5f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.669+0000 7f75d1a35640 1 --2- 172.21.15.40:0/3230686001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75bc005180 0x7f75cc139e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.669+0000 7f75d1a35640 1 -- 172.21.15.40:0/3230686001 >> 172.21.15.40:0/3230686001 conn(0x7f75cc069720 msgr2=0x7f75cc0cb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:05.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.669+0000 7f75d1a35640 1 -- 172.21.15.40:0/3230686001 shutdown_connections 2023-10-18T08:56:05.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.669+0000 7f75d1a35640 1 -- 172.21.15.40:0/3230686001 wait complete. 2023-10-18T08:56:05.691 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:56:05.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.929+0000 7f39b066f640 1 -- 172.21.15.40:0/1073606247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39a80cff00 msgr2=0x7f39a80d1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:05.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.929+0000 7f39b066f640 1 --2- 172.21.15.40:0/1073606247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39a80cff00 0x7f39a80d1560 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3998005fd0 tx=0x7f399802be20 comp rx=0 tx=0).stop 2023-10-18T08:56:05.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.929+0000 7f39b066f640 1 -- 172.21.15.40:0/1073606247 shutdown_connections 2023-10-18T08:56:05.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.929+0000 7f39b066f640 1 --2- 172.21.15.40:0/1073606247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39a80cff00 0x7f39a80d1560 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.929+0000 7f39b066f640 1 --2- 172.21.15.40:0/1073606247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39a815b6c0 0x7f39a80b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.929+0000 7f39b066f640 1 -- 172.21.15.40:0/1073606247 >> 172.21.15.40:0/1073606247 conn(0x7f39a809a960 msgr2=0x7f39a8088fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:05.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.929+0000 7f39b066f640 1 -- 172.21.15.40:0/1073606247 shutdown_connections 2023-10-18T08:56:05.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.929+0000 7f39b066f640 1 -- 172.21.15.40:0/1073606247 wait complete. 2023-10-18T08:56:05.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.931+0000 7f39b066f640 1 Processor -- start 2023-10-18T08:56:05.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.931+0000 7f39b066f640 1 -- start start 2023-10-18T08:56:05.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.932+0000 7f39b066f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a80cff00 0x7f39a815a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:05.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.932+0000 7f39b066f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39a80d1aa0 0x7f39a80c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:05.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.932+0000 7f39b066f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39a815b6c0 0x7f39a815e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:05.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.932+0000 7f39b066f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a815a580 con 0x7f39a80cff00 2023-10-18T08:56:05.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.932+0000 7f39b066f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a815eeb0 con 0x7f39a80d1aa0 2023-10-18T08:56:05.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.932+0000 7f39b066f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a80c64b0 con 0x7f39a815b6c0 2023-10-18T08:56:05.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.932+0000 7f39ae40b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a80cff00 0x7f39a815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:05.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.932+0000 7f39ae40b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a80cff00 0x7f39a815a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34026/0 (socket says 172.21.15.40:34026) 2023-10-18T08:56:05.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.932+0000 7f39ae40b640 1 -- 172.21.15.40:0/2613707758 learned_addr learned my addr 172.21.15.40:0/2613707758 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:05.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.933+0000 7f39adc0a640 1 --2- 172.21.15.40:0/2613707758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39a80d1aa0 0x7f39a80c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:05.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.933+0000 7f39ae40b640 1 -- 172.21.15.40:0/2613707758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39a815b6c0 msgr2=0x7f39a815e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:05.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.933+0000 7f39ae40b640 1 --2- 172.21.15.40:0/2613707758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39a815b6c0 0x7f39a815e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.933+0000 7f39ae40b640 1 -- 172.21.15.40:0/2613707758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39a80d1aa0 msgr2=0x7f39a80c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:05.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.933+0000 7f39ae40b640 1 --2- 172.21.15.40:0/2613707758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39a80d1aa0 0x7f39a80c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:05.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.933+0000 7f39ae40b640 1 -- 172.21.15.40:0/2613707758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a4009840 con 0x7f39a80cff00 2023-10-18T08:56:05.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.933+0000 7f39ae40b640 1 --2- 172.21.15.40:0/2613707758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a80cff00 0x7f39a815a040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f39a400ecb0 tx=0x7f39a400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:05.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.933+0000 7f399effd640 1 -- 172.21.15.40:0/2613707758 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4009aa0 con 0x7f39a80cff00 2023-10-18T08:56:05.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.933+0000 7f399effd640 1 -- 172.21.15.40:0/2613707758 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f39a4003b70 con 0x7f39a80cff00 2023-10-18T08:56:05.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.933+0000 7f399effd640 1 -- 172.21.15.40:0/2613707758 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4004020 con 0x7f39a80cff00 2023-10-18T08:56:05.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.933+0000 7f39b066f640 1 -- 172.21.15.40:0/2613707758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3998009560 con 0x7f39a80cff00 2023-10-18T08:56:05.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.934+0000 7f39b066f640 1 -- 172.21.15.40:0/2613707758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a80c6840 con 0x7f39a80cff00 2023-10-18T08:56:05.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.935+0000 7f39b066f640 1 -- 172.21.15.40:0/2613707758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a80739c0 con 0x7f39a80cff00 2023-10-18T08:56:05.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.936+0000 7f399effd640 1 -- 172.21.15.40:0/2613707758 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f39a4004180 con 0x7f39a80cff00 2023-10-18T08:56:05.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.937+0000 7f399effd640 1 --2- 172.21.15.40:0/2613707758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f398c06ae40 0x7f398c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:05.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.937+0000 7f399effd640 1 -- 172.21.15.40:0/2613707758 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f39a4014070 con 0x7f39a80cff00 2023-10-18T08:56:05.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.937+0000 7f39adc0a640 1 --2- 172.21.15.40:0/2613707758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f398c06ae40 0x7f398c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:05.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.938+0000 7f39adc0a640 1 --2- 172.21.15.40:0/2613707758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f398c06ae40 0x7f398c06d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3998035000 tx=0x7f3998004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:05.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:05.941+0000 7f399effd640 1 -- 172.21.15.40:0/2613707758 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39a40555d0 con 0x7f39a80cff00 2023-10-18T08:56:06.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.137+0000 7f39b066f640 1 -- 172.21.15.40:0/2613707758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f39a80af0c0 con 0x7f39a80cff00 2023-10-18T08:56:06.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.138+0000 7f399effd640 1 -- 172.21.15.40:0/2613707758 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f39a4058bf0 con 0x7f39a80cff00 2023-10-18T08:56:06.140 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:56:06.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.140+0000 7f39b066f640 1 -- 172.21.15.40:0/2613707758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f398c06ae40 msgr2=0x7f398c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:06.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.140+0000 7f39b066f640 1 --2- 172.21.15.40:0/2613707758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f398c06ae40 0x7f398c06d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3998035000 tx=0x7f3998004730 comp rx=0 tx=0).stop 2023-10-18T08:56:06.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.140+0000 7f39b066f640 1 -- 172.21.15.40:0/2613707758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a80cff00 msgr2=0x7f39a815a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:06.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.140+0000 7f39b066f640 1 --2- 172.21.15.40:0/2613707758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a80cff00 0x7f39a815a040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f39a400ecb0 tx=0x7f39a400c6a0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.141+0000 7f39b066f640 1 -- 172.21.15.40:0/2613707758 shutdown_connections 2023-10-18T08:56:06.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.141+0000 7f39b066f640 1 --2- 172.21.15.40:0/2613707758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f398c06ae40 0x7f398c06d300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.141+0000 7f39b066f640 1 --2- 172.21.15.40:0/2613707758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39a815b6c0 0x7f39a815e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.141+0000 7f39b066f640 1 --2- 172.21.15.40:0/2613707758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39a80d1aa0 0x7f39a80c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.141+0000 7f39b066f640 1 --2- 172.21.15.40:0/2613707758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39a80cff00 0x7f39a815a040 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.141+0000 7f39b066f640 1 -- 172.21.15.40:0/2613707758 >> 172.21.15.40:0/2613707758 conn(0x7f39a809a960 msgr2=0x7f39a80ae500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:06.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.141+0000 7f39b066f640 1 -- 172.21.15.40:0/2613707758 shutdown_connections 2023-10-18T08:56:06.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.141+0000 7f39b066f640 1 -- 172.21.15.40:0/2613707758 wait complete. 2023-10-18T08:56:06.162 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:56:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.393+0000 7f8d62b2c640 1 -- 172.21.15.40:0/353864744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d5c15eea0 msgr2=0x7f8d5c0cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:06.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.393+0000 7f8d62b2c640 1 --2- 172.21.15.40:0/353864744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d5c15eea0 0x7f8d5c0cff70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8d50009830 tx=0x7f8d5002bd70 comp rx=0 tx=0).stop 2023-10-18T08:56:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.394+0000 7f8d62b2c640 1 -- 172.21.15.40:0/353864744 shutdown_connections 2023-10-18T08:56:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.394+0000 7f8d62b2c640 1 --2- 172.21.15.40:0/353864744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d5c15eea0 0x7f8d5c0cff70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.394+0000 7f8d62b2c640 1 -- 172.21.15.40:0/353864744 >> 172.21.15.40:0/353864744 conn(0x7f8d5c09a610 msgr2=0x7f8d5c0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.394+0000 7f8d62b2c640 1 -- 172.21.15.40:0/353864744 shutdown_connections 2023-10-18T08:56:06.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.394+0000 7f8d62b2c640 1 -- 172.21.15.40:0/353864744 wait complete. 2023-10-18T08:56:06.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.396+0000 7f8d62b2c640 1 Processor -- start 2023-10-18T08:56:06.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.396+0000 7f8d62b2c640 1 -- start start 2023-10-18T08:56:06.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.397+0000 7f8d62b2c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d5c0d19b0 0x7f8d5c0c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:06.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.397+0000 7f8d62b2c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d5c15eac0 0x7f8d5c154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:06.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.397+0000 7f8d62b2c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d5c15eea0 0x7f8d5c0c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:06.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.397+0000 7f8d62b2c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c158b40 con 0x7f8d5c0d19b0 2023-10-18T08:56:06.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.397+0000 7f8d62b2c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c0c6880 con 0x7f8d5c15eac0 2023-10-18T08:56:06.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.397+0000 7f8d62b2c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c0c6b60 con 0x7f8d5c15eea0 2023-10-18T08:56:06.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.397+0000 7f8d61b2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d5c0d19b0 0x7f8d5c0c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:06.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.397+0000 7f8d61b2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d5c0d19b0 0x7f8d5c0c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34048/0 (socket says 172.21.15.40:34048) 2023-10-18T08:56:06.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.397+0000 7f8d61b2a640 1 -- 172.21.15.40:0/842613126 learned_addr learned my addr 172.21.15.40:0/842613126 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:06.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.398+0000 7f8d61b2a640 1 -- 172.21.15.40:0/842613126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d5c15eea0 msgr2=0x7f8d5c0c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:06.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.398+0000 7f8d61b2a640 1 --2- 172.21.15.40:0/842613126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d5c15eea0 0x7f8d5c0c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.398+0000 7f8d61b2a640 1 -- 172.21.15.40:0/842613126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d5c15eac0 msgr2=0x7f8d5c154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:06.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.398+0000 7f8d61b2a640 1 --2- 172.21.15.40:0/842613126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d5c15eac0 0x7f8d5c154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.398+0000 7f8d61b2a640 1 -- 172.21.15.40:0/842613126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d50009490 con 0x7f8d5c0d19b0 2023-10-18T08:56:06.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.398+0000 7f8d61b2a640 1 --2- 172.21.15.40:0/842613126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d5c0d19b0 0x7f8d5c0c8370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800c500 tx=0x7f8d5800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:06.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.398+0000 7f8d4e7fc640 1 -- 172.21.15.40:0/842613126 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58012830 con 0x7f8d5c0d19b0 2023-10-18T08:56:06.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.398+0000 7f8d4e7fc640 1 -- 172.21.15.40:0/842613126 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d58012a80 con 0x7f8d5c0d19b0 2023-10-18T08:56:06.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.398+0000 7f8d4e7fc640 1 -- 172.21.15.40:0/842613126 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58012e90 con 0x7f8d5c0d19b0 2023-10-18T08:56:06.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.398+0000 7f8d62b2c640 1 -- 172.21.15.40:0/842613126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5c0c6e40 con 0x7f8d5c0d19b0 2023-10-18T08:56:06.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.398+0000 7f8d62b2c640 1 -- 172.21.15.40:0/842613126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5c0c7c30 con 0x7f8d5c0d19b0 2023-10-18T08:56:06.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.400+0000 7f8d62b2c640 1 -- 172.21.15.40:0/842613126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5c071da0 con 0x7f8d5c0d19b0 2023-10-18T08:56:06.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.401+0000 7f8d4e7fc640 1 -- 172.21.15.40:0/842613126 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8d5801a3b0 con 0x7f8d5c0d19b0 2023-10-18T08:56:06.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.402+0000 7f8d4e7fc640 1 --2- 172.21.15.40:0/842613126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d4006ae40 0x7f8d4006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:06.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.402+0000 7f8d4e7fc640 1 -- 172.21.15.40:0/842613126 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8d58089640 con 0x7f8d5c0d19b0 2023-10-18T08:56:06.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.402+0000 7f8d61329640 1 --2- 172.21.15.40:0/842613126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d4006ae40 0x7f8d4006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:06.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.403+0000 7f8d61329640 1 --2- 172.21.15.40:0/842613126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d4006ae40 0x7f8d4006d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8d50036000 tx=0x7f8d500046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:06.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.406+0000 7f8d4e7fc640 1 -- 172.21.15.40:0/842613126 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8d58058070 con 0x7f8d5c0d19b0 2023-10-18T08:56:06.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.602+0000 7f8d62b2c640 1 -- 172.21.15.40:0/842613126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8d5c066e00 con 0x7f8d5c0d19b0 2023-10-18T08:56:06.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.603+0000 7f8d4e7fc640 1 -- 172.21.15.40:0/842613126 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8d58057c00 con 0x7f8d5c0d19b0 2023-10-18T08:56:06.605 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:56:06.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.606+0000 7f8d62b2c640 1 -- 172.21.15.40:0/842613126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d4006ae40 msgr2=0x7f8d4006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:06.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.606+0000 7f8d62b2c640 1 --2- 172.21.15.40:0/842613126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d4006ae40 0x7f8d4006d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8d50036000 tx=0x7f8d500046c0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.606+0000 7f8d62b2c640 1 -- 172.21.15.40:0/842613126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d5c0d19b0 msgr2=0x7f8d5c0c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.606+0000 7f8d62b2c640 1 --2- 172.21.15.40:0/842613126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d5c0d19b0 0x7f8d5c0c8370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800c500 tx=0x7f8d5800c9c0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.606+0000 7f8d62b2c640 1 -- 172.21.15.40:0/842613126 shutdown_connections 2023-10-18T08:56:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.606+0000 7f8d62b2c640 1 --2- 172.21.15.40:0/842613126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d4006ae40 0x7f8d4006d300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.606+0000 7f8d62b2c640 1 --2- 172.21.15.40:0/842613126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d5c15eea0 0x7f8d5c0c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.606+0000 7f8d62b2c640 1 --2- 172.21.15.40:0/842613126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d5c15eac0 0x7f8d5c154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.606+0000 7f8d62b2c640 1 --2- 172.21.15.40:0/842613126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d5c0d19b0 0x7f8d5c0c8370 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.606+0000 7f8d62b2c640 1 -- 172.21.15.40:0/842613126 >> 172.21.15.40:0/842613126 conn(0x7f8d5c09a610 msgr2=0x7f8d5c0b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:06.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.606+0000 7f8d62b2c640 1 -- 172.21.15.40:0/842613126 shutdown_connections 2023-10-18T08:56:06.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.606+0000 7f8d62b2c640 1 -- 172.21.15.40:0/842613126 wait complete. 2023-10-18T08:56:06.628 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T08:56:06.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.837+0000 7fa336ee5640 1 -- 172.21.15.40:0/2864349143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa33015ba90 msgr2=0x7fa3300cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:06.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.837+0000 7fa336ee5640 1 --2- 172.21.15.40:0/2864349143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa33015ba90 0x7fa3300cc3b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009830 tx=0x7fa32c02bd60 comp rx=0 tx=0).stop 2023-10-18T08:56:06.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.838+0000 7fa336ee5640 1 -- 172.21.15.40:0/2864349143 shutdown_connections 2023-10-18T08:56:06.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.838+0000 7fa336ee5640 1 --2- 172.21.15.40:0/2864349143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3300ce600 0x7fa3300b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.838+0000 7fa336ee5640 1 --2- 172.21.15.40:0/2864349143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa33015ba90 0x7fa3300cc3b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.838+0000 7fa336ee5640 1 -- 172.21.15.40:0/2864349143 >> 172.21.15.40:0/2864349143 conn(0x7fa33006a780 msgr2=0x7fa33006ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:06.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.838+0000 7fa336ee5640 1 -- 172.21.15.40:0/2864349143 shutdown_connections 2023-10-18T08:56:06.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.838+0000 7fa336ee5640 1 -- 172.21.15.40:0/2864349143 wait complete. 2023-10-18T08:56:06.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.841+0000 7fa336ee5640 1 Processor -- start 2023-10-18T08:56:06.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.841+0000 7fa336ee5640 1 -- start start 2023-10-18T08:56:06.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.842+0000 7fa336ee5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3300b7000 0x7fa33014f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:06.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.842+0000 7fa336ee5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3300ce600 0x7fa3301e43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:06.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.842+0000 7fa336ee5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3300c5cf0 0x7fa3300b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:06.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.843+0000 7fa336ee5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3301e5540 con 0x7fa3300ce600 2023-10-18T08:56:06.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.843+0000 7fa336ee5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3301e4ca0 con 0x7fa3300c5cf0 2023-10-18T08:56:06.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.843+0000 7fa336ee5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3301e4f80 con 0x7fa3300b7000 2023-10-18T08:56:06.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.843+0000 7fa327fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3300ce600 0x7fa3301e43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:06.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.843+0000 7fa327fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3300ce600 0x7fa3301e43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34074/0 (socket says 172.21.15.40:34074) 2023-10-18T08:56:06.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.843+0000 7fa327fff640 1 -- 172.21.15.40:0/1736812487 learned_addr learned my addr 172.21.15.40:0/1736812487 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:06.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.843+0000 7fa335482640 1 --2- 172.21.15.40:0/1736812487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3300c5cf0 0x7fa3300b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:06.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.843+0000 7fa334c81640 1 --2- 172.21.15.40:0/1736812487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3300b7000 0x7fa33014f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:06.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.843+0000 7fa327fff640 1 -- 172.21.15.40:0/1736812487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3300b7000 msgr2=0x7fa33014f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:06.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.843+0000 7fa327fff640 1 --2- 172.21.15.40:0/1736812487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3300b7000 0x7fa33014f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.843+0000 7fa327fff640 1 -- 172.21.15.40:0/1736812487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3300c5cf0 msgr2=0x7fa3300b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:06.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.843+0000 7fa327fff640 1 --2- 172.21.15.40:0/1736812487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3300c5cf0 0x7fa3300b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:06.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.843+0000 7fa327fff640 1 -- 172.21.15.40:0/1736812487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c009490 con 0x7fa3300ce600 2023-10-18T08:56:06.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.843+0000 7fa334c81640 1 --2- 172.21.15.40:0/1736812487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3300b7000 0x7fa33014f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:06.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.844+0000 7fa327fff640 1 --2- 172.21.15.40:0/1736812487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3300ce600 0x7fa3301e43a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa32000c500 tx=0x7fa32000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:06.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.844+0000 7fa3257fa640 1 -- 172.21.15.40:0/1736812487 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320012830 con 0x7fa3300ce600 2023-10-18T08:56:06.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.844+0000 7fa3257fa640 1 -- 172.21.15.40:0/1736812487 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa320012a80 con 0x7fa3300ce600 2023-10-18T08:56:06.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.844+0000 7fa336ee5640 1 -- 172.21.15.40:0/1736812487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3301e5260 con 0x7fa3300ce600 2023-10-18T08:56:06.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.844+0000 7fa3257fa640 1 -- 172.21.15.40:0/1736812487 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320012e50 con 0x7fa3300ce600 2023-10-18T08:56:06.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.844+0000 7fa336ee5640 1 -- 172.21.15.40:0/1736812487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3301e48e0 con 0x7fa3300ce600 2023-10-18T08:56:06.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.846+0000 7fa336ee5640 1 -- 172.21.15.40:0/1736812487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3300c63f0 con 0x7fa3300ce600 2023-10-18T08:56:06.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.847+0000 7fa3257fa640 1 -- 172.21.15.40:0/1736812487 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa32001b400 con 0x7fa3300ce600 2023-10-18T08:56:06.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.848+0000 7fa3257fa640 1 --2- 172.21.15.40:0/1736812487 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa31c06ae70 0x7fa31c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:06.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.849+0000 7fa3257fa640 1 -- 172.21.15.40:0/1736812487 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa320089840 con 0x7fa3300ce600 2023-10-18T08:56:06.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.849+0000 7fa334c81640 1 --2- 172.21.15.40:0/1736812487 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa31c06ae70 0x7fa31c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:06.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.850+0000 7fa334c81640 1 --2- 172.21.15.40:0/1736812487 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa31c06ae70 0x7fa31c06d330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa32c036000 tx=0x7fa32c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:06.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:06.853+0000 7fa3257fa640 1 -- 172.21.15.40:0/1736812487 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa3300c63f0 con 0x7fa3300ce600 2023-10-18T08:56:07.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.061+0000 7fa336ee5640 1 -- 172.21.15.40:0/1736812487 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa3300c63f0 con 0x7fa3300ce600 2023-10-18T08:56:07.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.062+0000 7fa3257fa640 1 -- 172.21.15.40:0/1736812487 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa3300c63f0 con 0x7fa3300ce600 2023-10-18T08:56:07.064 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T08:56:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.064+0000 7fa336ee5640 1 -- 172.21.15.40:0/1736812487 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa31c06ae70 msgr2=0x7fa31c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.064+0000 7fa336ee5640 1 --2- 172.21.15.40:0/1736812487 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa31c06ae70 0x7fa31c06d330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa32c036000 tx=0x7fa32c004730 comp rx=0 tx=0).stop 2023-10-18T08:56:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.064+0000 7fa336ee5640 1 -- 172.21.15.40:0/1736812487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3300ce600 msgr2=0x7fa3301e43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:07.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.064+0000 7fa336ee5640 1 --2- 172.21.15.40:0/1736812487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3300ce600 0x7fa3301e43a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa32000c500 tx=0x7fa32000c9c0 comp rx=0 tx=0).stop 2023-10-18T08:56:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.064+0000 7fa336ee5640 1 -- 172.21.15.40:0/1736812487 shutdown_connections 2023-10-18T08:56:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.065+0000 7fa336ee5640 1 --2- 172.21.15.40:0/1736812487 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa31c06ae70 0x7fa31c06d330 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.065+0000 7fa336ee5640 1 --2- 172.21.15.40:0/1736812487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3300c5cf0 0x7fa3300b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.065+0000 7fa336ee5640 1 --2- 172.21.15.40:0/1736812487 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3300ce600 0x7fa3301e43a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.065+0000 7fa336ee5640 1 --2- 172.21.15.40:0/1736812487 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3300b7000 0x7fa33014f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.065+0000 7fa336ee5640 1 -- 172.21.15.40:0/1736812487 >> 172.21.15.40:0/1736812487 conn(0x7fa33006a780 msgr2=0x7fa33015a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.065+0000 7fa336ee5640 1 -- 172.21.15.40:0/1736812487 shutdown_connections 2023-10-18T08:56:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.065+0000 7fa336ee5640 1 -- 172.21.15.40:0/1736812487 wait complete. 2023-10-18T08:56:07.089 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-18T08:56:07.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:07.090 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-18T08:56:07.180 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T08:56:07.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:07.181 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T08:56:07.231 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:56:07.232 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:07.292 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:56:07.293 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:07.346 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:56:07.347 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:07.398 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:07.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.614+0000 7f348f59e640 1 -- 172.21.15.40:0/1656838353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0039c0 msgr2=0x7f347c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:07.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.614+0000 7f348f59e640 1 --2- 172.21.15.40:0/1656838353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0039c0 0x7f347c003de0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3480001680 tx=0x7f347c0057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:07.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.615+0000 7f348f59e640 1 -- 172.21.15.40:0/1656838353 shutdown_connections 2023-10-18T08:56:07.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.615+0000 7f348f59e640 1 --2- 172.21.15.40:0/1656838353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0039c0 0x7f347c003de0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:07.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.615+0000 7f348f59e640 1 -- 172.21.15.40:0/1656838353 >> 172.21.15.40:0/1656838353 conn(0x7f3490069720 msgr2=0x7f3490069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:07.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.615+0000 7f348f59e640 1 -- 172.21.15.40:0/1656838353 shutdown_connections 2023-10-18T08:56:07.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.615+0000 7f348f59e640 1 -- 172.21.15.40:0/1656838353 wait complete. 2023-10-18T08:56:07.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.616+0000 7f348f59e640 1 Processor -- start 2023-10-18T08:56:07.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.617+0000 7f348f59e640 1 -- start start 2023-10-18T08:56:07.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.617+0000 7f348f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0039c0 0x7f34900c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:07.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f348f59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34901f1860 0x7f3490159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:07.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f348f59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34901f4ea0 0x7f34901f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:07.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f348f59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34901f5890 con 0x7f347c0039c0 2023-10-18T08:56:07.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f348f59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349013f260 con 0x7f34901f4ea0 2023-10-18T08:56:07.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f348f59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34901f1ce0 con 0x7f34901f1860 2023-10-18T08:56:07.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f348e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0039c0 0x7f34900c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:07.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f348e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0039c0 0x7f34900c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34090/0 (socket says 172.21.15.40:34090) 2023-10-18T08:56:07.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f348e59c640 1 -- 172.21.15.40:0/2645688381 learned_addr learned my addr 172.21.15.40:0/2645688381 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:07.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f3487fff640 1 --2- 172.21.15.40:0/2645688381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34901f1860 0x7f3490159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:07.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f348ed9d640 1 --2- 172.21.15.40:0/2645688381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34901f4ea0 0x7f34901f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:07.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f348e59c640 1 -- 172.21.15.40:0/2645688381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34901f1860 msgr2=0x7f3490159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:07.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f348e59c640 1 --2- 172.21.15.40:0/2645688381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34901f1860 0x7f3490159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:07.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f348e59c640 1 -- 172.21.15.40:0/2645688381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34901f4ea0 msgr2=0x7f34901f5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:07.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f348e59c640 1 --2- 172.21.15.40:0/2645688381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34901f4ea0 0x7f34901f5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:07.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f348e59c640 1 -- 172.21.15.40:0/2645688381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349015a020 con 0x7f347c0039c0 2023-10-18T08:56:07.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f3487fff640 1 --2- 172.21.15.40:0/2645688381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34901f1860 0x7f3490159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:07.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.618+0000 7f348ed9d640 1 --2- 172.21.15.40:0/2645688381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34901f4ea0 0x7f34901f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:07.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.619+0000 7f348e59c640 1 --2- 172.21.15.40:0/2645688381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0039c0 0x7f34900c9000 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f348002c0d0 tx=0x7f34800035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:07.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.619+0000 7f3486ffd640 1 -- 172.21.15.40:0/2645688381 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480003bf0 con 0x7f347c0039c0 2023-10-18T08:56:07.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.619+0000 7f3486ffd640 1 -- 172.21.15.40:0/2645688381 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3480003e40 con 0x7f347c0039c0 2023-10-18T08:56:07.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.619+0000 7f3486ffd640 1 -- 172.21.15.40:0/2645688381 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348002c790 con 0x7f347c0039c0 2023-10-18T08:56:07.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.619+0000 7f348f59e640 1 -- 172.21.15.40:0/2645688381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34901f1f70 con 0x7f347c0039c0 2023-10-18T08:56:07.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.619+0000 7f348f59e640 1 -- 172.21.15.40:0/2645688381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34901f2530 con 0x7f347c0039c0 2023-10-18T08:56:07.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.621+0000 7f348f59e640 1 -- 172.21.15.40:0/2645688381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3490066e00 con 0x7f347c0039c0 2023-10-18T08:56:07.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.621+0000 7f3486ffd640 1 -- 172.21.15.40:0/2645688381 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f348002c8f0 con 0x7f347c0039c0 2023-10-18T08:56:07.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.622+0000 7f3486ffd640 1 --2- 172.21.15.40:0/2645688381 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f347006ae70 0x7f347006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:07.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.622+0000 7f3486ffd640 1 -- 172.21.15.40:0/2645688381 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f34800aa630 con 0x7f347c0039c0 2023-10-18T08:56:07.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.623+0000 7f3487fff640 1 --2- 172.21.15.40:0/2645688381 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f347006ae70 0x7f347006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:07.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.624+0000 7f3487fff640 1 --2- 172.21.15.40:0/2645688381 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f347006ae70 0x7f347006d330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f349015c490 tx=0x7f3478005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:07.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.630+0000 7f3486ffd640 1 -- 172.21.15.40:0/2645688381 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3480075660 con 0x7f347c0039c0 2023-10-18T08:56:07.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.823+0000 7f348f59e640 1 -- 172.21.15.40:0/2645688381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f349006d920 con 0x7f347c0039c0 2023-10-18T08:56:07.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.825+0000 7f3486ffd640 1 -- 172.21.15.40:0/2645688381 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16766 (secure 0 0 0) 0x7f3480078c80 con 0x7f347c0039c0 2023-10-18T08:56:07.828 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:07.828 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:02.868692+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:07.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.827+0000 7f348f59e640 1 -- 172.21.15.40:0/2645688381 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f347006ae70 msgr2=0x7f347006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:07.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.827+0000 7f348f59e640 1 --2- 172.21.15.40:0/2645688381 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f347006ae70 0x7f347006d330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f349015c490 tx=0x7f3478005f90 comp rx=0 tx=0).stop 2023-10-18T08:56:07.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.827+0000 7f348f59e640 1 -- 172.21.15.40:0/2645688381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0039c0 msgr2=0x7f34900c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:07.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.827+0000 7f348f59e640 1 --2- 172.21.15.40:0/2645688381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0039c0 0x7f34900c9000 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f348002c0d0 tx=0x7f34800035f0 comp rx=0 tx=0).stop 2023-10-18T08:56:07.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.828+0000 7f348f59e640 1 -- 172.21.15.40:0/2645688381 shutdown_connections 2023-10-18T08:56:07.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.828+0000 7f348f59e640 1 --2- 172.21.15.40:0/2645688381 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f347006ae70 0x7f347006d330 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:07.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.828+0000 7f348f59e640 1 --2- 172.21.15.40:0/2645688381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34901f4ea0 0x7f34901f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:07.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.828+0000 7f348f59e640 1 --2- 172.21.15.40:0/2645688381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34901f1860 0x7f3490159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:07.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.828+0000 7f348f59e640 1 --2- 172.21.15.40:0/2645688381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f347c0039c0 0x7f34900c9000 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:07.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.828+0000 7f348f59e640 1 -- 172.21.15.40:0/2645688381 >> 172.21.15.40:0/2645688381 conn(0x7f3490069720 msgr2=0x7f34900cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:07.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.828+0000 7f348f59e640 1 -- 172.21.15.40:0/2645688381 shutdown_connections 2023-10-18T08:56:07.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:07.828+0000 7f348f59e640 1 -- 172.21.15.40:0/2645688381 wait complete. 2023-10-18T08:56:07.848 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:55:13.210788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:55:13.772849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:56:07.850 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:56:08.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.053+0000 7f0a1a032640 1 -- 172.21.15.40:0/3926156352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a1415f6d0 msgr2=0x7f0a1415fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:08.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.053+0000 7f0a1a032640 1 --2- 172.21.15.40:0/3926156352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a1415f6d0 0x7f0a1415fb60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a140d0250 tx=0x7f0a0c02bd70 comp rx=0 tx=0).stop 2023-10-18T08:56:08.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.053+0000 7f0a1a032640 1 -- 172.21.15.40:0/3926156352 shutdown_connections 2023-10-18T08:56:08.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.053+0000 7f0a1a032640 1 --2- 172.21.15.40:0/3926156352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a1415f6d0 0x7f0a1415fb60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.053+0000 7f0a1a032640 1 --2- 172.21.15.40:0/3926156352 >> v2:172.21.15.144:3300/0 conn(0x7f0a140d0d00 0x7f0a1415f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.053+0000 7f0a1a032640 1 -- 172.21.15.40:0/3926156352 >> 172.21.15.40:0/3926156352 conn(0x7f0a140cdda0 msgr2=0x7f0a140ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:08.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.053+0000 7f0a1a032640 1 -- 172.21.15.40:0/3926156352 shutdown_connections 2023-10-18T08:56:08.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.054+0000 7f0a1a032640 1 -- 172.21.15.40:0/3926156352 wait complete. 2023-10-18T08:56:08.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.055+0000 7f0a1a032640 1 Processor -- start 2023-10-18T08:56:08.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.055+0000 7f0a1a032640 1 -- start start 2023-10-18T08:56:08.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.056+0000 7f0a1a032640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a140cc0f0 0x7f0a140b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:08.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.056+0000 7f0a1a032640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a140d0d00 0x7f0a140c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:08.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.056+0000 7f0a1a032640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a1415f6d0 0x7f0a1415c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:08.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.056+0000 7f0a1a032640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a140ca3d0 con 0x7f0a140d0d00 2023-10-18T08:56:08.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.056+0000 7f0a1a032640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a140c9da0 con 0x7f0a140cc0f0 2023-10-18T08:56:08.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.056+0000 7f0a1a032640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1415c050 con 0x7f0a1415f6d0 2023-10-18T08:56:08.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.056+0000 7f0a0affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a140d0d00 0x7f0a140c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:08.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.056+0000 7f0a0affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a140d0d00 0x7f0a140c4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34102/0 (socket says 172.21.15.40:34102) 2023-10-18T08:56:08.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.056+0000 7f0a0affd640 1 -- 172.21.15.40:0/159386927 learned_addr learned my addr 172.21.15.40:0/159386927 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:08.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.057+0000 7f0a13fff640 1 --2- 172.21.15.40:0/159386927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a1415f6d0 0x7f0a1415c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:08.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.057+0000 7f0a137fe640 1 --2- 172.21.15.40:0/159386927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a140cc0f0 0x7f0a140b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:08.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.057+0000 7f0a0affd640 1 -- 172.21.15.40:0/159386927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a1415f6d0 msgr2=0x7f0a1415c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:08.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.057+0000 7f0a0affd640 1 --2- 172.21.15.40:0/159386927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a1415f6d0 0x7f0a1415c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.057+0000 7f0a0affd640 1 -- 172.21.15.40:0/159386927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a140cc0f0 msgr2=0x7f0a140b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:08.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.057+0000 7f0a0affd640 1 --2- 172.21.15.40:0/159386927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a140cc0f0 0x7f0a140b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.057+0000 7f0a0affd640 1 -- 172.21.15.40:0/159386927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c009560 con 0x7f0a140d0d00 2023-10-18T08:56:08.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.057+0000 7f0a137fe640 1 --2- 172.21.15.40:0/159386927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a140cc0f0 0x7f0a140b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:08.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.057+0000 7f0a0affd640 1 --2- 172.21.15.40:0/159386927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a140d0d00 0x7f0a140c4e50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00e9f0 tx=0x7f09fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:08.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.057+0000 7f0a10ff9640 1 -- 172.21.15.40:0/159386927 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc009700 con 0x7f0a140d0d00 2023-10-18T08:56:08.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.058+0000 7f0a10ff9640 1 -- 172.21.15.40:0/159386927 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f09fc00ce70 con 0x7f0a140d0d00 2023-10-18T08:56:08.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.058+0000 7f0a10ff9640 1 -- 172.21.15.40:0/159386927 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc010670 con 0x7f0a140d0d00 2023-10-18T08:56:08.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.058+0000 7f0a1a032640 1 -- 172.21.15.40:0/159386927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a140ca060 con 0x7f0a140d0d00 2023-10-18T08:56:08.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.058+0000 7f0a1a032640 1 -- 172.21.15.40:0/159386927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a140b28b0 con 0x7f0a140d0d00 2023-10-18T08:56:08.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.059+0000 7f0a1a032640 1 -- 172.21.15.40:0/159386927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a14066fd0 con 0x7f0a140d0d00 2023-10-18T08:56:08.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.060+0000 7f0a10ff9640 1 -- 172.21.15.40:0/159386927 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f09fc0107d0 con 0x7f0a140d0d00 2023-10-18T08:56:08.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.061+0000 7f0a10ff9640 1 --2- 172.21.15.40:0/159386927 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09f406ae70 0x7f09f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:08.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.061+0000 7f0a10ff9640 1 -- 172.21.15.40:0/159386927 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f09fc014070 con 0x7f0a140d0d00 2023-10-18T08:56:08.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.061+0000 7f0a137fe640 1 --2- 172.21.15.40:0/159386927 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09f406ae70 0x7f09f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:08.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.062+0000 7f0a137fe640 1 --2- 172.21.15.40:0/159386927 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09f406ae70 0x7f09f406d330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0a04001d70 tx=0x7f0a04009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:08.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.065+0000 7f0a10ff9640 1 -- 172.21.15.40:0/159386927 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f09fc051ca0 con 0x7f0a140d0d00 2023-10-18T08:56:08.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.261+0000 7f0a1a032640 1 -- 172.21.15.40:0/159386927 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a140d0ae0 con 0x7f0a140d0d00 2023-10-18T08:56:08.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.262+0000 7f0a10ff9640 1 -- 172.21.15.40:0/159386927 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09fc058d70 con 0x7f0a140d0d00 2023-10-18T08:56:08.265 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:56:08.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.265+0000 7f0a1a032640 1 -- 172.21.15.40:0/159386927 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09f406ae70 msgr2=0x7f09f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:08.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.265+0000 7f0a1a032640 1 --2- 172.21.15.40:0/159386927 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09f406ae70 0x7f09f406d330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0a04001d70 tx=0x7f0a04009040 comp rx=0 tx=0).stop 2023-10-18T08:56:08.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.265+0000 7f0a1a032640 1 -- 172.21.15.40:0/159386927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a140d0d00 msgr2=0x7f0a140c4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:08.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.265+0000 7f0a1a032640 1 --2- 172.21.15.40:0/159386927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a140d0d00 0x7f0a140c4e50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00e9f0 tx=0x7f09fc00eeb0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.265+0000 7f0a1a032640 1 -- 172.21.15.40:0/159386927 shutdown_connections 2023-10-18T08:56:08.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.265+0000 7f0a1a032640 1 --2- 172.21.15.40:0/159386927 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09f406ae70 0x7f09f406d330 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.265+0000 7f0a1a032640 1 --2- 172.21.15.40:0/159386927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a1415f6d0 0x7f0a1415c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.266+0000 7f0a1a032640 1 --2- 172.21.15.40:0/159386927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a140d0d00 0x7f0a140c4e50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.266+0000 7f0a1a032640 1 --2- 172.21.15.40:0/159386927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a140cc0f0 0x7f0a140b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.266+0000 7f0a1a032640 1 -- 172.21.15.40:0/159386927 >> 172.21.15.40:0/159386927 conn(0x7f0a140cdda0 msgr2=0x7f0a141578b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:08.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.266+0000 7f0a1a032640 1 -- 172.21.15.40:0/159386927 shutdown_connections 2023-10-18T08:56:08.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.266+0000 7f0a1a032640 1 -- 172.21.15.40:0/159386927 wait complete. 2023-10-18T08:56:08.285 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:56:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.514+0000 7f1822733640 1 -- 172.21.15.40:0/636306063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f181c071270 msgr2=0x7f181c0c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.514+0000 7f1822733640 1 --2- 172.21.15.40:0/636306063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f181c071270 0x7f181c0c9620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f180c009830 tx=0x7f180c02bcc0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.515+0000 7f1822733640 1 -- 172.21.15.40:0/636306063 shutdown_connections 2023-10-18T08:56:08.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.515+0000 7f1822733640 1 --2- 172.21.15.40:0/636306063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f181c15af70 0x7f181c157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.515+0000 7f1822733640 1 --2- 172.21.15.40:0/636306063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f181c071270 0x7f181c0c9620 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.515+0000 7f1822733640 1 -- 172.21.15.40:0/636306063 >> 172.21.15.40:0/636306063 conn(0x7f181c06ca10 msgr2=0x7f181c06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:08.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.515+0000 7f1822733640 1 -- 172.21.15.40:0/636306063 shutdown_connections 2023-10-18T08:56:08.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.515+0000 7f1822733640 1 -- 172.21.15.40:0/636306063 wait complete. 2023-10-18T08:56:08.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.516+0000 7f1822733640 1 Processor -- start 2023-10-18T08:56:08.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.517+0000 7f1822733640 1 -- start start 2023-10-18T08:56:08.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.517+0000 7f1822733640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f181c06d5d0 0x7f181c155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:08.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.518+0000 7f1822733640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f181c071270 0x7f181c154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:08.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.518+0000 7f1822733640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f181c15af70 0x7f181c1521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:08.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.518+0000 7f1822733640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c148400 con 0x7f181c15af70 2023-10-18T08:56:08.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.518+0000 7f1822733640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c156700 con 0x7f181c06d5d0 2023-10-18T08:56:08.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.518+0000 7f1822733640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181c152700 con 0x7f181c071270 2023-10-18T08:56:08.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.518+0000 7f1821f32640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f181c15af70 0x7f181c1521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:08.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.518+0000 7f1821f32640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f181c15af70 0x7f181c1521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34108/0 (socket says 172.21.15.40:34108) 2023-10-18T08:56:08.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.518+0000 7f1821f32640 1 -- 172.21.15.40:0/2655486945 learned_addr learned my addr 172.21.15.40:0/2655486945 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:08.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.518+0000 7f1821731640 1 --2- 172.21.15.40:0/2655486945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f181c06d5d0 0x7f181c155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:08.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.518+0000 7f1820f30640 1 --2- 172.21.15.40:0/2655486945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f181c071270 0x7f181c154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:08.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.518+0000 7f1821f32640 1 -- 172.21.15.40:0/2655486945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f181c071270 msgr2=0x7f181c154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:08.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.518+0000 7f1821f32640 1 --2- 172.21.15.40:0/2655486945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f181c071270 0x7f181c154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.518+0000 7f1821f32640 1 -- 172.21.15.40:0/2655486945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f181c06d5d0 msgr2=0x7f181c155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:08.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.518+0000 7f1821f32640 1 --2- 172.21.15.40:0/2655486945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f181c06d5d0 0x7f181c155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.518+0000 7f1821f32640 1 -- 172.21.15.40:0/2655486945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c009490 con 0x7f181c15af70 2023-10-18T08:56:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.518+0000 7f1820f30640 1 --2- 172.21.15.40:0/2655486945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f181c071270 0x7f181c154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.519+0000 7f1821f32640 1 --2- 172.21.15.40:0/2655486945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f181c15af70 0x7f181c1521c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f181400dda0 tx=0x7f181400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.519+0000 7f1811ffb640 1 -- 172.21.15.40:0/2655486945 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814003b70 con 0x7f181c15af70 2023-10-18T08:56:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.519+0000 7f1811ffb640 1 -- 172.21.15.40:0/2655486945 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1814003dc0 con 0x7f181c15af70 2023-10-18T08:56:08.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.519+0000 7f1811ffb640 1 -- 172.21.15.40:0/2655486945 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814004280 con 0x7f181c15af70 2023-10-18T08:56:08.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.519+0000 7f1822733640 1 -- 172.21.15.40:0/2655486945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f181c155af0 con 0x7f181c15af70 2023-10-18T08:56:08.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.519+0000 7f1822733640 1 -- 172.21.15.40:0/2655486945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181c1560b0 con 0x7f181c15af70 2023-10-18T08:56:08.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.521+0000 7f1822733640 1 -- 172.21.15.40:0/2655486945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f181c156890 con 0x7f181c15af70 2023-10-18T08:56:08.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.521+0000 7f1811ffb640 1 -- 172.21.15.40:0/2655486945 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f18140043e0 con 0x7f181c15af70 2023-10-18T08:56:08.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.522+0000 7f1811ffb640 1 --2- 172.21.15.40:0/2655486945 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f180006ae40 0x7f180006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:08.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.522+0000 7f1811ffb640 1 -- 172.21.15.40:0/2655486945 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f181408a1e0 con 0x7f181c15af70 2023-10-18T08:56:08.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.523+0000 7f1821731640 1 --2- 172.21.15.40:0/2655486945 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f180006ae40 0x7f180006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:08.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.524+0000 7f1821731640 1 --2- 172.21.15.40:0/2655486945 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f180006ae40 0x7f180006d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1818001370 tx=0x7f1818009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:08.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.527+0000 7f1811ffb640 1 -- 172.21.15.40:0/2655486945 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1814055170 con 0x7f181c15af70 2023-10-18T08:56:08.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.718+0000 7f1822733640 1 -- 172.21.15.40:0/2655486945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f181c066e00 con 0x7f181c15af70 2023-10-18T08:56:08.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.719+0000 7f1811ffb640 1 -- 172.21.15.40:0/2655486945 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1814058790 con 0x7f181c15af70 2023-10-18T08:56:08.722 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:56:08.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.722+0000 7f1822733640 1 -- 172.21.15.40:0/2655486945 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f180006ae40 msgr2=0x7f180006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:08.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.722+0000 7f1822733640 1 --2- 172.21.15.40:0/2655486945 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f180006ae40 0x7f180006d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1818001370 tx=0x7f1818009040 comp rx=0 tx=0).stop 2023-10-18T08:56:08.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.722+0000 7f1822733640 1 -- 172.21.15.40:0/2655486945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f181c15af70 msgr2=0x7f181c1521c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:08.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.722+0000 7f1822733640 1 --2- 172.21.15.40:0/2655486945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f181c15af70 0x7f181c1521c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f181400dda0 tx=0x7f181400b730 comp rx=0 tx=0).stop 2023-10-18T08:56:08.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.722+0000 7f1822733640 1 -- 172.21.15.40:0/2655486945 shutdown_connections 2023-10-18T08:56:08.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.722+0000 7f1822733640 1 --2- 172.21.15.40:0/2655486945 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f180006ae40 0x7f180006d300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.722+0000 7f1822733640 1 --2- 172.21.15.40:0/2655486945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f181c15af70 0x7f181c1521c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.722+0000 7f1822733640 1 --2- 172.21.15.40:0/2655486945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f181c071270 0x7f181c154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.722+0000 7f1822733640 1 --2- 172.21.15.40:0/2655486945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f181c06d5d0 0x7f181c155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.722+0000 7f1822733640 1 -- 172.21.15.40:0/2655486945 >> 172.21.15.40:0/2655486945 conn(0x7f181c06ca10 msgr2=0x7f181c0b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:08.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.722+0000 7f1822733640 1 -- 172.21.15.40:0/2655486945 shutdown_connections 2023-10-18T08:56:08.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.722+0000 7f1822733640 1 -- 172.21.15.40:0/2655486945 wait complete. 2023-10-18T08:56:08.746 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:56:08.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.982+0000 7f246bede640 1 -- 172.21.15.40:0/936398412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24640cd140 msgr2=0x7f2464162610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:08.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.982+0000 7f246bede640 1 --2- 172.21.15.40:0/936398412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24640cd140 0x7f2464162610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f245c005fd0 tx=0x7f245c02c050 comp rx=0 tx=0).stop 2023-10-18T08:56:08.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.982+0000 7f246bede640 1 -- 172.21.15.40:0/936398412 shutdown_connections 2023-10-18T08:56:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.982+0000 7f246bede640 1 --2- 172.21.15.40:0/936398412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24640cd140 0x7f2464162610 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.982+0000 7f246bede640 1 --2- 172.21.15.40:0/936398412 >> v2:172.21.15.144:3300/0 conn(0x7f246406d660 0x7f24640b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.982+0000 7f246bede640 1 -- 172.21.15.40:0/936398412 >> 172.21.15.40:0/936398412 conn(0x7f2464069720 msgr2=0x7f2464069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.983+0000 7f246bede640 1 -- 172.21.15.40:0/936398412 shutdown_connections 2023-10-18T08:56:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.983+0000 7f246bede640 1 -- 172.21.15.40:0/936398412 wait complete. 2023-10-18T08:56:08.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.984+0000 7f246bede640 1 Processor -- start 2023-10-18T08:56:08.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.984+0000 7f246bede640 1 -- start start 2023-10-18T08:56:08.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.985+0000 7f246bede640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246406d660 0x7f246414f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:08.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.985+0000 7f246bede640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24640cd140 0x7f24641f7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:08.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.985+0000 7f246bede640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2464157800 0x7f2464151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:08.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.985+0000 7f246bede640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24641f7d70 con 0x7f2464157800 2023-10-18T08:56:08.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.985+0000 7f246bede640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24641f8050 con 0x7f246406d660 2023-10-18T08:56:08.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.985+0000 7f246bede640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24641f8330 con 0x7f24640cd140 2023-10-18T08:56:08.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.986+0000 7f246a47b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2464157800 0x7f2464151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:08.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.986+0000 7f246a47b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2464157800 0x7f2464151ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34118/0 (socket says 172.21.15.40:34118) 2023-10-18T08:56:08.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.986+0000 7f246a47b640 1 -- 172.21.15.40:0/3297163728 learned_addr learned my addr 172.21.15.40:0/3297163728 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:08.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.986+0000 7f2469c7a640 1 --2- 172.21.15.40:0/3297163728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246406d660 0x7f246414f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:08.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.986+0000 7f2469479640 1 --2- 172.21.15.40:0/3297163728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24640cd140 0x7f24641f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:08.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.987+0000 7f246a47b640 1 -- 172.21.15.40:0/3297163728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24640cd140 msgr2=0x7f24641f7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:08.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.987+0000 7f246a47b640 1 --2- 172.21.15.40:0/3297163728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24640cd140 0x7f24641f7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.987+0000 7f246a47b640 1 -- 172.21.15.40:0/3297163728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246406d660 msgr2=0x7f246414f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:08.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.987+0000 7f246a47b640 1 --2- 172.21.15.40:0/3297163728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246406d660 0x7f246414f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:08.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.987+0000 7f246a47b640 1 -- 172.21.15.40:0/3297163728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2460009490 con 0x7f2464157800 2023-10-18T08:56:08.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.987+0000 7f246a47b640 1 --2- 172.21.15.40:0/3297163728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2464157800 0x7f2464151ee0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f245c009b30 tx=0x7f245c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:08.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.987+0000 7f245a7fc640 1 -- 172.21.15.40:0/3297163728 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c004870 con 0x7f2464157800 2023-10-18T08:56:08.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.987+0000 7f245a7fc640 1 -- 172.21.15.40:0/3297163728 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f245c03e550 con 0x7f2464157800 2023-10-18T08:56:08.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.987+0000 7f246bede640 1 -- 172.21.15.40:0/3297163728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245c009840 con 0x7f2464157800 2023-10-18T08:56:08.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.987+0000 7f245a7fc640 1 -- 172.21.15.40:0/3297163728 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c03e960 con 0x7f2464157800 2023-10-18T08:56:08.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.987+0000 7f246bede640 1 -- 172.21.15.40:0/3297163728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24641f88f0 con 0x7f2464157800 2023-10-18T08:56:08.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.988+0000 7f246bede640 1 -- 172.21.15.40:0/3297163728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24641f8b10 con 0x7f2464157800 2023-10-18T08:56:08.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.990+0000 7f245a7fc640 1 -- 172.21.15.40:0/3297163728 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f245c03eac0 con 0x7f2464157800 2023-10-18T08:56:08.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.990+0000 7f245a7fc640 1 --2- 172.21.15.40:0/3297163728 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f244806ae40 0x7f244806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:08.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.991+0000 7f245a7fc640 1 -- 172.21.15.40:0/3297163728 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f245c0ac5f0 con 0x7f2464157800 2023-10-18T08:56:08.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.991+0000 7f2469c7a640 1 --2- 172.21.15.40:0/3297163728 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f244806ae40 0x7f244806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:08.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.992+0000 7f2469c7a640 1 --2- 172.21.15.40:0/3297163728 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f244806ae40 0x7f244806d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2460009d60 tx=0x7f2460005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:08.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:08.995+0000 7f245a7fc640 1 -- 172.21.15.40:0/3297163728 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f245c077580 con 0x7f2464157800 2023-10-18T08:56:09.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.184+0000 7f246bede640 1 -- 172.21.15.40:0/3297163728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f24640b0a40 con 0x7f2464157800 2023-10-18T08:56:09.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.185+0000 7f245a7fc640 1 -- 172.21.15.40:0/3297163728 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f245c042020 con 0x7f2464157800 2023-10-18T08:56:09.187 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:56:09.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.187+0000 7f246bede640 1 -- 172.21.15.40:0/3297163728 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f244806ae40 msgr2=0x7f244806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:09.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.187+0000 7f246bede640 1 --2- 172.21.15.40:0/3297163728 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f244806ae40 0x7f244806d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2460009d60 tx=0x7f2460005f90 comp rx=0 tx=0).stop 2023-10-18T08:56:09.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.187+0000 7f246bede640 1 -- 172.21.15.40:0/3297163728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2464157800 msgr2=0x7f2464151ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:09.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.187+0000 7f246bede640 1 --2- 172.21.15.40:0/3297163728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2464157800 0x7f2464151ee0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f245c009b30 tx=0x7f245c004270 comp rx=0 tx=0).stop 2023-10-18T08:56:09.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.188+0000 7f246bede640 1 -- 172.21.15.40:0/3297163728 shutdown_connections 2023-10-18T08:56:09.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.188+0000 7f246bede640 1 --2- 172.21.15.40:0/3297163728 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f244806ae40 0x7f244806d300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:09.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.188+0000 7f246bede640 1 --2- 172.21.15.40:0/3297163728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2464157800 0x7f2464151ee0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:09.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.188+0000 7f246bede640 1 --2- 172.21.15.40:0/3297163728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24640cd140 0x7f24641f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:09.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.188+0000 7f246bede640 1 --2- 172.21.15.40:0/3297163728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f246406d660 0x7f246414f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:09.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.188+0000 7f246bede640 1 -- 172.21.15.40:0/3297163728 >> 172.21.15.40:0/3297163728 conn(0x7f2464069720 msgr2=0x7f246406fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:09.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.188+0000 7f246bede640 1 -- 172.21.15.40:0/3297163728 shutdown_connections 2023-10-18T08:56:09.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.188+0000 7f246bede640 1 -- 172.21.15.40:0/3297163728 wait complete. 2023-10-18T08:56:09.208 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T08:56:09.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.426+0000 7fb7bffff640 1 -- 172.21.15.40:0/3611507030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7c00c9f00 msgr2=0x7fb7c0157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:09.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.426+0000 7fb7bffff640 1 --2- 172.21.15.40:0/3611507030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7c00c9f00 0x7fb7c0157d40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00caf30 tx=0x7fb7b802bd70 comp rx=0 tx=0).stop 2023-10-18T08:56:09.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.426+0000 7fb7bffff640 1 -- 172.21.15.40:0/3611507030 shutdown_connections 2023-10-18T08:56:09.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.426+0000 7fb7bffff640 1 --2- 172.21.15.40:0/3611507030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7c00c9f00 0x7fb7c0157d40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:09.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.426+0000 7fb7bffff640 1 --2- 172.21.15.40:0/3611507030 >> v2:172.21.15.144:3300/0 conn(0x7fb7c00b15b0 0x7fb7c00c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:09.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.426+0000 7fb7bffff640 1 -- 172.21.15.40:0/3611507030 >> 172.21.15.40:0/3611507030 conn(0x7fb7c0069720 msgr2=0x7fb7c0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:09.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.426+0000 7fb7bffff640 1 -- 172.21.15.40:0/3611507030 shutdown_connections 2023-10-18T08:56:09.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.426+0000 7fb7bffff640 1 -- 172.21.15.40:0/3611507030 wait complete. 2023-10-18T08:56:09.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.428+0000 7fb7bffff640 1 Processor -- start 2023-10-18T08:56:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.428+0000 7fb7bffff640 1 -- start start 2023-10-18T08:56:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.429+0000 7fb7bffff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7c006d6f0 0x7fb7c014c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.429+0000 7fb7bffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7c00b15b0 0x7fb7c00cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:09.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.429+0000 7fb7bffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7c00c9f00 0x7fb7c00cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:09.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.430+0000 7fb7bffff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c013de70 con 0x7fb7c00b15b0 2023-10-18T08:56:09.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.430+0000 7fb7bffff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c014d600 con 0x7fb7c00c9f00 2023-10-18T08:56:09.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.430+0000 7fb7bffff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c014c980 con 0x7fb7c006d6f0 2023-10-18T08:56:09.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.430+0000 7fb7be7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7c00b15b0 0x7fb7c00cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:09.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.430+0000 7fb7be7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7c00b15b0 0x7fb7c00cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34146/0 (socket says 172.21.15.40:34146) 2023-10-18T08:56:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.430+0000 7fb7be7fc640 1 -- 172.21.15.40:0/4226108143 learned_addr learned my addr 172.21.15.40:0/4226108143 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.430+0000 7fb7beffd640 1 --2- 172.21.15.40:0/4226108143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7c006d6f0 0x7fb7c014c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.430+0000 7fb7bf7fe640 1 --2- 172.21.15.40:0/4226108143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7c00c9f00 0x7fb7c00cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.430+0000 7fb7be7fc640 1 -- 172.21.15.40:0/4226108143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7c006d6f0 msgr2=0x7fb7c014c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:09.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.430+0000 7fb7be7fc640 1 --2- 172.21.15.40:0/4226108143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7c006d6f0 0x7fb7c014c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:09.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.430+0000 7fb7be7fc640 1 -- 172.21.15.40:0/4226108143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7c00c9f00 msgr2=0x7fb7c00cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:09.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.430+0000 7fb7be7fc640 1 --2- 172.21.15.40:0/4226108143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7c00c9f00 0x7fb7c00cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:09.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.430+0000 7fb7be7fc640 1 -- 172.21.15.40:0/4226108143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b8009560 con 0x7fb7c00b15b0 2023-10-18T08:56:09.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.430+0000 7fb7bf7fe640 1 --2- 172.21.15.40:0/4226108143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7c00c9f00 0x7fb7c00cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:09.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.431+0000 7fb7be7fc640 1 --2- 172.21.15.40:0/4226108143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7c00b15b0 0x7fb7c00cb930 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800c920 tx=0x7fb7a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:09.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.431+0000 7fb7b77fe640 1 -- 172.21.15.40:0/4226108143 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8007c70 con 0x7fb7c00b15b0 2023-10-18T08:56:09.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.431+0000 7fb7b77fe640 1 -- 172.21.15.40:0/4226108143 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb7a800f450 con 0x7fb7c00b15b0 2023-10-18T08:56:09.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.431+0000 7fb7b77fe640 1 -- 172.21.15.40:0/4226108143 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a800f860 con 0x7fb7c00b15b0 2023-10-18T08:56:09.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.431+0000 7fb7bffff640 1 -- 172.21.15.40:0/4226108143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c014cc60 con 0x7fb7c00b15b0 2023-10-18T08:56:09.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.431+0000 7fb7bffff640 1 -- 172.21.15.40:0/4226108143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c014d220 con 0x7fb7c00b15b0 2023-10-18T08:56:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.432+0000 7fb7bffff640 1 -- 172.21.15.40:0/4226108143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c0066e00 con 0x7fb7c00b15b0 2023-10-18T08:56:09.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.433+0000 7fb7b77fe640 1 -- 172.21.15.40:0/4226108143 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb7a800f9c0 con 0x7fb7c00b15b0 2023-10-18T08:56:09.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.434+0000 7fb7b77fe640 1 --2- 172.21.15.40:0/4226108143 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7a006ae40 0x7fb7a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:09.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.434+0000 7fb7b77fe640 1 -- 172.21.15.40:0/4226108143 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb7a8089f40 con 0x7fb7c00b15b0 2023-10-18T08:56:09.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.435+0000 7fb7beffd640 1 --2- 172.21.15.40:0/4226108143 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7a006ae40 0x7fb7a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:09.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.436+0000 7fb7beffd640 1 --2- 172.21.15.40:0/4226108143 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7a006ae40 0x7fb7a006d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0005520 tx=0x7fb7b0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:09.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.439+0000 7fb7b77fe640 1 -- 172.21.15.40:0/4226108143 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb7a8054f60 con 0x7fb7c00b15b0 2023-10-18T08:56:09.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.652+0000 7fb7bffff640 1 -- 172.21.15.40:0/4226108143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb7c014ea70 con 0x7fb7c00b15b0 2023-10-18T08:56:09.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.653+0000 7fb7b77fe640 1 -- 172.21.15.40:0/4226108143 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb7a801d020 con 0x7fb7c00b15b0 2023-10-18T08:56:09.656 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T08:56:09.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.656+0000 7fb7bffff640 1 -- 172.21.15.40:0/4226108143 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7a006ae40 msgr2=0x7fb7a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:09.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.656+0000 7fb7bffff640 1 --2- 172.21.15.40:0/4226108143 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7a006ae40 0x7fb7a006d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0005520 tx=0x7fb7b0009040 comp rx=0 tx=0).stop 2023-10-18T08:56:09.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.656+0000 7fb7bffff640 1 -- 172.21.15.40:0/4226108143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7c00b15b0 msgr2=0x7fb7c00cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:09.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.656+0000 7fb7bffff640 1 --2- 172.21.15.40:0/4226108143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7c00b15b0 0x7fb7c00cb930 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800c920 tx=0x7fb7a800cde0 comp rx=0 tx=0).stop 2023-10-18T08:56:09.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.656+0000 7fb7bffff640 1 -- 172.21.15.40:0/4226108143 shutdown_connections 2023-10-18T08:56:09.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.656+0000 7fb7bffff640 1 --2- 172.21.15.40:0/4226108143 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7a006ae40 0x7fb7a006d300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:09.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.656+0000 7fb7bffff640 1 --2- 172.21.15.40:0/4226108143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7c00c9f00 0x7fb7c00cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:09.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.656+0000 7fb7bffff640 1 --2- 172.21.15.40:0/4226108143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7c00b15b0 0x7fb7c00cb930 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:09.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.656+0000 7fb7bffff640 1 --2- 172.21.15.40:0/4226108143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7c006d6f0 0x7fb7c014c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:09.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.656+0000 7fb7bffff640 1 -- 172.21.15.40:0/4226108143 >> 172.21.15.40:0/4226108143 conn(0x7fb7c0069720 msgr2=0x7fb7c0151cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:09.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.656+0000 7fb7bffff640 1 -- 172.21.15.40:0/4226108143 shutdown_connections 2023-10-18T08:56:09.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:09.657+0000 7fb7bffff640 1 -- 172.21.15.40:0/4226108143 wait complete. 2023-10-18T08:56:09.678 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-18T08:56:09.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:09.679 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-18T08:56:09.797 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T08:56:09.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:09.798 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T08:56:09.862 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:56:09.863 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:09.927 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:56:09.928 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:09.985 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:56:09.986 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:10.043 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:10.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.273+0000 7fe2e4ad7640 1 -- 172.21.15.40:0/2336659476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2e006d6f0 msgr2=0x7fe2e00b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:10.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.273+0000 7fe2e4ad7640 1 --2- 172.21.15.40:0/2336659476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2e006d6f0 0x7fe2e00b15b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0005fd0 tx=0x7fe2d002be10 comp rx=0 tx=0).stop 2023-10-18T08:56:10.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.273+0000 7fe2e4ad7640 1 -- 172.21.15.40:0/2336659476 shutdown_connections 2023-10-18T08:56:10.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.273+0000 7fe2e4ad7640 1 --2- 172.21.15.40:0/2336659476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00b0630 0x7fe2e00c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:10.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.273+0000 7fe2e4ad7640 1 --2- 172.21.15.40:0/2336659476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2e006d6f0 0x7fe2e00b15b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:10.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.273+0000 7fe2e4ad7640 1 -- 172.21.15.40:0/2336659476 >> 172.21.15.40:0/2336659476 conn(0x7fe2e0069720 msgr2=0x7fe2e0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:10.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.273+0000 7fe2e4ad7640 1 -- 172.21.15.40:0/2336659476 shutdown_connections 2023-10-18T08:56:10.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.273+0000 7fe2e4ad7640 1 -- 172.21.15.40:0/2336659476 wait complete. 2023-10-18T08:56:10.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.275+0000 7fe2e4ad7640 1 Processor -- start 2023-10-18T08:56:10.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.275+0000 7fe2e4ad7640 1 -- start start 2023-10-18T08:56:10.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.276+0000 7fe2e4ad7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e006d6f0 0x7fe2e0139f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:10.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.276+0000 7fe2e4ad7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00b0630 0x7fe2e0155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:10.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.276+0000 7fe2e4ad7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2e00c9f00 0x7fe2e0155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:10.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.276+0000 7fe2e4ad7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e015a900 con 0x7fe2e00c9f00 2023-10-18T08:56:10.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.276+0000 7fe2e4ad7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0159590 con 0x7fe2e006d6f0 2023-10-18T08:56:10.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.276+0000 7fe2e4ad7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0159a20 con 0x7fe2e00b0630 2023-10-18T08:56:10.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.277+0000 7fe2dffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2e00c9f00 0x7fe2e0155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:10.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.277+0000 7fe2dffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2e00c9f00 0x7fe2e0155b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34168/0 (socket says 172.21.15.40:34168) 2023-10-18T08:56:10.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.277+0000 7fe2dffff640 1 -- 172.21.15.40:0/3676163158 learned_addr learned my addr 172.21.15.40:0/3676163158 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:10.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.277+0000 7fe2df7fe640 1 --2- 172.21.15.40:0/3676163158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e006d6f0 0x7fe2e0139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:10.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.277+0000 7fe2d7fff640 1 --2- 172.21.15.40:0/3676163158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00b0630 0x7fe2e0155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:10.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.277+0000 7fe2dffff640 1 -- 172.21.15.40:0/3676163158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00b0630 msgr2=0x7fe2e0155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:10.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.277+0000 7fe2dffff640 1 --2- 172.21.15.40:0/3676163158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00b0630 0x7fe2e0155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:10.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.277+0000 7fe2dffff640 1 -- 172.21.15.40:0/3676163158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e006d6f0 msgr2=0x7fe2e0139f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:10.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.277+0000 7fe2dffff640 1 --2- 172.21.15.40:0/3676163158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e006d6f0 0x7fe2e0139f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:10.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.277+0000 7fe2dffff640 1 -- 172.21.15.40:0/3676163158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d0009560 con 0x7fe2e00c9f00 2023-10-18T08:56:10.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.277+0000 7fe2d7fff640 1 --2- 172.21.15.40:0/3676163158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00b0630 0x7fe2e0155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:10.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.277+0000 7fe2dffff640 1 --2- 172.21.15.40:0/3676163158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2e00c9f00 0x7fe2e0155b50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800c730 tx=0x7fe2d800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:10.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.278+0000 7fe2dcff9640 1 -- 172.21.15.40:0/3676163158 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8006d50 con 0x7fe2e00c9f00 2023-10-18T08:56:10.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.278+0000 7fe2dcff9640 1 -- 172.21.15.40:0/3676163158 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe2d8003970 con 0x7fe2e00c9f00 2023-10-18T08:56:10.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.278+0000 7fe2dcff9640 1 -- 172.21.15.40:0/3676163158 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8003d80 con 0x7fe2e00c9f00 2023-10-18T08:56:10.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.278+0000 7fe2e4ad7640 1 -- 172.21.15.40:0/3676163158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e013d8f0 con 0x7fe2e00c9f00 2023-10-18T08:56:10.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.278+0000 7fe2e4ad7640 1 -- 172.21.15.40:0/3676163158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e0158f40 con 0x7fe2e00c9f00 2023-10-18T08:56:10.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.279+0000 7fe2e4ad7640 1 -- 172.21.15.40:0/3676163158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e0159720 con 0x7fe2e00c9f00 2023-10-18T08:56:10.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.280+0000 7fe2dcff9640 1 -- 172.21.15.40:0/3676163158 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe2d80050d0 con 0x7fe2e00c9f00 2023-10-18T08:56:10.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.281+0000 7fe2dcff9640 1 --2- 172.21.15.40:0/3676163158 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe2c006ae40 0x7fe2c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:10.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.281+0000 7fe2dcff9640 1 -- 172.21.15.40:0/3676163158 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe2d8089340 con 0x7fe2e00c9f00 2023-10-18T08:56:10.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.282+0000 7fe2df7fe640 1 --2- 172.21.15.40:0/3676163158 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe2c006ae40 0x7fe2c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:10.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.283+0000 7fe2df7fe640 1 --2- 172.21.15.40:0/3676163158 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe2c006ae40 0x7fe2c006d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0035000 tx=0x7fe2d0004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:10.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.286+0000 7fe2dcff9640 1 -- 172.21.15.40:0/3676163158 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe2d8054400 con 0x7fe2e00c9f00 2023-10-18T08:56:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.481+0000 7fe2e4ad7640 1 -- 172.21.15.40:0/3676163158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2e0066e00 con 0x7fe2e00c9f00 2023-10-18T08:56:10.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.484+0000 7fe2dcff9640 1 -- 172.21.15.40:0/3676163158 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16766 (secure 0 0 0) 0x7fe2d8057a20 con 0x7fe2e00c9f00 2023-10-18T08:56:10.487 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:10.487 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:02.868692+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:10.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.487+0000 7fe2e4ad7640 1 -- 172.21.15.40:0/3676163158 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe2c006ae40 msgr2=0x7fe2c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:10.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.487+0000 7fe2e4ad7640 1 --2- 172.21.15.40:0/3676163158 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe2c006ae40 0x7fe2c006d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0035000 tx=0x7fe2d0004730 comp rx=0 tx=0).stop 2023-10-18T08:56:10.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.487+0000 7fe2e4ad7640 1 -- 172.21.15.40:0/3676163158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2e00c9f00 msgr2=0x7fe2e0155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:10.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.487+0000 7fe2e4ad7640 1 --2- 172.21.15.40:0/3676163158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2e00c9f00 0x7fe2e0155b50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800c730 tx=0x7fe2d800cb30 comp rx=0 tx=0).stop 2023-10-18T08:56:10.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.487+0000 7fe2e4ad7640 1 -- 172.21.15.40:0/3676163158 shutdown_connections 2023-10-18T08:56:10.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.487+0000 7fe2e4ad7640 1 --2- 172.21.15.40:0/3676163158 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe2c006ae40 0x7fe2c006d300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:10.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.487+0000 7fe2e4ad7640 1 --2- 172.21.15.40:0/3676163158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2e00c9f00 0x7fe2e0155b50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:10.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.487+0000 7fe2e4ad7640 1 --2- 172.21.15.40:0/3676163158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2e00b0630 0x7fe2e0155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:10.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.487+0000 7fe2e4ad7640 1 --2- 172.21.15.40:0/3676163158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2e006d6f0 0x7fe2e0139f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:10.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.487+0000 7fe2e4ad7640 1 -- 172.21.15.40:0/3676163158 >> 172.21.15.40:0/3676163158 conn(0x7fe2e0069720 msgr2=0x7fe2e0156b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:10.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.487+0000 7fe2e4ad7640 1 -- 172.21.15.40:0/3676163158 shutdown_connections 2023-10-18T08:56:10.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.487+0000 7fe2e4ad7640 1 -- 172.21.15.40:0/3676163158 wait complete. 2023-10-18T08:56:10.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:55:13.210788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:55:13.772849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:56:10.508 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:56:10.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.734+0000 7f94baf6c640 1 -- 172.21.15.40:0/897885994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a40039a0 msgr2=0x7f94a4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:10.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.734+0000 7f94baf6c640 1 --2- 172.21.15.40:0/897885994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a40039a0 0x7f94a4003dc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f94a8006fd0 tx=0x7f94a40057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:10.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.734+0000 7f94b9509640 1 -- 172.21.15.40:0/897885994 reap_dead start 2023-10-18T08:56:10.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.734+0000 7f94baf6c640 1 -- 172.21.15.40:0/897885994 shutdown_connections 2023-10-18T08:56:10.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.734+0000 7f94baf6c640 1 -- 172.21.15.40:0/897885994 >> 172.21.15.40:0/897885994 conn(0x7f94b4088fa0 msgr2=0x7f94b40cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:10.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.734+0000 7f94baf6c640 1 -- 172.21.15.40:0/897885994 shutdown_connections 2023-10-18T08:56:10.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.734+0000 7f94baf6c640 1 -- 172.21.15.40:0/897885994 wait complete. 2023-10-18T08:56:10.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.736+0000 7f94baf6c640 1 Processor -- start 2023-10-18T08:56:10.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.736+0000 7f94baf6c640 1 -- start start 2023-10-18T08:56:10.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.737+0000 7f94baf6c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a40039a0 0x7f94b40c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:10.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.737+0000 7f94baf6c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b41e5d00 0x7f94b41f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:10.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.737+0000 7f94baf6c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94b41e48b0 0x7f94b41e2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:10.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.737+0000 7f94baf6c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b4159a60 con 0x7f94b41e5d00 2023-10-18T08:56:10.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.737+0000 7f94baf6c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b415a5a0 con 0x7f94a40039a0 2023-10-18T08:56:10.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.737+0000 7f94baf6c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b4159c70 con 0x7f94b41e48b0 2023-10-18T08:56:10.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.738+0000 7f94b3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b41e5d00 0x7f94b41f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:10.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.738+0000 7f94b3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b41e5d00 0x7f94b41f3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34186/0 (socket says 172.21.15.40:34186) 2023-10-18T08:56:10.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.738+0000 7f94b3fff640 1 -- 172.21.15.40:0/75532053 learned_addr learned my addr 172.21.15.40:0/75532053 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:10.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.738+0000 7f94b3fff640 1 -- 172.21.15.40:0/75532053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94b41e48b0 msgr2=0x7f94b41e2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:10.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.738+0000 7f94b3fff640 1 --2- 172.21.15.40:0/75532053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94b41e48b0 0x7f94b41e2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:10.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.738+0000 7f94b3fff640 1 -- 172.21.15.40:0/75532053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a40039a0 msgr2=0x7f94b40c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:10.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.738+0000 7f94b8d08640 1 --2- 172.21.15.40:0/75532053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a40039a0 0x7f94b40c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:10.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.738+0000 7f94b3fff640 1 --2- 172.21.15.40:0/75532053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a40039a0 0x7f94b40c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:10.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.738+0000 7f94b3fff640 1 -- 172.21.15.40:0/75532053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b40ce320 con 0x7f94b41e5d00 2023-10-18T08:56:10.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.738+0000 7f94b8d08640 1 --2- 172.21.15.40:0/75532053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a40039a0 0x7f94b40c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:10.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.739+0000 7f94b3fff640 1 --2- 172.21.15.40:0/75532053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b41e5d00 0x7f94b41f3080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f94a0011750 tx=0x7f94a0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:10.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.739+0000 7f94b17fa640 1 -- 172.21.15.40:0/75532053 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a0004460 con 0x7f94b41e5d00 2023-10-18T08:56:10.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.739+0000 7f94b17fa640 1 -- 172.21.15.40:0/75532053 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f94a000f940 con 0x7f94b41e5d00 2023-10-18T08:56:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.739+0000 7f94b17fa640 1 -- 172.21.15.40:0/75532053 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a000fcd0 con 0x7f94b41e5d00 2023-10-18T08:56:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.739+0000 7f94baf6c640 1 -- 172.21.15.40:0/75532053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b4159f50 con 0x7f94b41e5d00 2023-10-18T08:56:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.739+0000 7f94baf6c640 1 -- 172.21.15.40:0/75532053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b41f35c0 con 0x7f94b41e5d00 2023-10-18T08:56:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.740+0000 7f94baf6c640 1 -- 172.21.15.40:0/75532053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b4066fd0 con 0x7f94b41e5d00 2023-10-18T08:56:10.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.741+0000 7f94b17fa640 1 -- 172.21.15.40:0/75532053 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f94a00035f0 con 0x7f94b41e5d00 2023-10-18T08:56:10.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.742+0000 7f94b17fa640 1 --2- 172.21.15.40:0/75532053 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f949806ae40 0x7f949806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:10.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.742+0000 7f94b17fa640 1 -- 172.21.15.40:0/75532053 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f94a008a490 con 0x7f94b41e5d00 2023-10-18T08:56:10.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.743+0000 7f94b8d08640 1 --2- 172.21.15.40:0/75532053 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f949806ae40 0x7f949806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:10.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.744+0000 7f94b8d08640 1 --2- 172.21.15.40:0/75532053 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f949806ae40 0x7f949806d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f94a8034000 tx=0x7f94a802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:10.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.746+0000 7f94b17fa640 1 -- 172.21.15.40:0/75532053 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f94a0055420 con 0x7f94b41e5d00 2023-10-18T08:56:10.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.942+0000 7f94baf6c640 1 -- 172.21.15.40:0/75532053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94b415a6e0 con 0x7f94b41e5d00 2023-10-18T08:56:10.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.943+0000 7f94b17fa640 1 -- 172.21.15.40:0/75532053 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94a0058a40 con 0x7f94b41e5d00 2023-10-18T08:56:10.945 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:56:10.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.946+0000 7f94baf6c640 1 -- 172.21.15.40:0/75532053 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f949806ae40 msgr2=0x7f949806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:10.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.946+0000 7f94baf6c640 1 --2- 172.21.15.40:0/75532053 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f949806ae40 0x7f949806d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f94a8034000 tx=0x7f94a802e000 comp rx=0 tx=0).stop 2023-10-18T08:56:10.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.946+0000 7f94baf6c640 1 -- 172.21.15.40:0/75532053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b41e5d00 msgr2=0x7f94b41f3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:10.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.946+0000 7f94baf6c640 1 --2- 172.21.15.40:0/75532053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b41e5d00 0x7f94b41f3080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f94a0011750 tx=0x7f94a0011c10 comp rx=0 tx=0).stop 2023-10-18T08:56:10.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.946+0000 7f94baf6c640 1 -- 172.21.15.40:0/75532053 shutdown_connections 2023-10-18T08:56:10.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.946+0000 7f94baf6c640 1 --2- 172.21.15.40:0/75532053 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f949806ae40 0x7f949806d300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:10.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.946+0000 7f94baf6c640 1 --2- 172.21.15.40:0/75532053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94b41e48b0 0x7f94b41e2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:10.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.946+0000 7f94baf6c640 1 --2- 172.21.15.40:0/75532053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94b41e5d00 0x7f94b41f3080 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:10.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.946+0000 7f94baf6c640 1 --2- 172.21.15.40:0/75532053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94a40039a0 0x7f94b40c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:10.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.946+0000 7f94baf6c640 1 -- 172.21.15.40:0/75532053 >> 172.21.15.40:0/75532053 conn(0x7f94b4088fa0 msgr2=0x7f94b40cb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:10.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.946+0000 7f94baf6c640 1 -- 172.21.15.40:0/75532053 shutdown_connections 2023-10-18T08:56:10.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:10.946+0000 7f94baf6c640 1 -- 172.21.15.40:0/75532053 wait complete. 2023-10-18T08:56:10.966 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:56:11.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.198+0000 7f8d7622e640 1 -- 172.21.15.40:0/4121776749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d7015eea0 msgr2=0x7f8d700b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.198+0000 7f8d7622e640 1 --2- 172.21.15.40:0/4121776749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d7015eea0 0x7f8d700b0be0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8d60005fd0 tx=0x7f8d6002be20 comp rx=0 tx=0).stop 2023-10-18T08:56:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.199+0000 7f8d7622e640 1 -- 172.21.15.40:0/4121776749 shutdown_connections 2023-10-18T08:56:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.199+0000 7f8d7622e640 1 --2- 172.21.15.40:0/4121776749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d7015eea0 0x7f8d700b0be0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.199+0000 7f8d7622e640 1 --2- 172.21.15.40:0/4121776749 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d7015eac0 0x7f8d700cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.199+0000 7f8d7622e640 1 -- 172.21.15.40:0/4121776749 >> 172.21.15.40:0/4121776749 conn(0x7f8d7009a610 msgr2=0x7f8d700d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.199+0000 7f8d7622e640 1 -- 172.21.15.40:0/4121776749 shutdown_connections 2023-10-18T08:56:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.199+0000 7f8d7622e640 1 -- 172.21.15.40:0/4121776749 wait complete. 2023-10-18T08:56:11.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.201+0000 7f8d7622e640 1 Processor -- start 2023-10-18T08:56:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.201+0000 7f8d7622e640 1 -- start start 2023-10-18T08:56:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.202+0000 7f8d7622e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d700d19b0 0x7f8d700c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.202+0000 7f8d7622e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d7015eac0 0x7f8d700c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.202+0000 7f8d7622e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d7015eea0 0x7f8d700c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.202+0000 7f8d7622e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700b4b40 con 0x7f8d700d19b0 2023-10-18T08:56:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.202+0000 7f8d7622e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700b3f30 con 0x7f8d7015eea0 2023-10-18T08:56:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.202+0000 7f8d7622e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700b4210 con 0x7f8d7015eac0 2023-10-18T08:56:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.202+0000 7f8d7522c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d700d19b0 0x7f8d700c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:11.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.202+0000 7f8d7522c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d700d19b0 0x7f8d700c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34198/0 (socket says 172.21.15.40:34198) 2023-10-18T08:56:11.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.202+0000 7f8d7522c640 1 -- 172.21.15.40:0/2005754919 learned_addr learned my addr 172.21.15.40:0/2005754919 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:11.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.202+0000 7f8d74a2b640 1 --2- 172.21.15.40:0/2005754919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d7015eac0 0x7f8d700c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:11.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.202+0000 7f8d7522c640 1 -- 172.21.15.40:0/2005754919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d7015eac0 msgr2=0x7f8d700c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:11.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.202+0000 7f8d7522c640 1 --2- 172.21.15.40:0/2005754919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d7015eac0 0x7f8d700c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:11.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.202+0000 7f8d7522c640 1 -- 172.21.15.40:0/2005754919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d7015eea0 msgr2=0x7f8d700c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.202+0000 7f8d7522c640 1 --2- 172.21.15.40:0/2005754919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d7015eea0 0x7f8d700c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.202+0000 7f8d7522c640 1 -- 172.21.15.40:0/2005754919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d60009560 con 0x7f8d700d19b0 2023-10-18T08:56:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.203+0000 7f8d7522c640 1 --2- 172.21.15.40:0/2005754919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d700d19b0 0x7f8d700c68b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00d950 tx=0x7f8d6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.203+0000 7f8d65ffb640 1 -- 172.21.15.40:0/2005754919 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c009c30 con 0x7f8d700d19b0 2023-10-18T08:56:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.203+0000 7f8d65ffb640 1 -- 172.21.15.40:0/2005754919 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d6c009e80 con 0x7f8d700d19b0 2023-10-18T08:56:11.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.203+0000 7f8d65ffb640 1 -- 172.21.15.40:0/2005754919 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0106c0 con 0x7f8d700d19b0 2023-10-18T08:56:11.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.203+0000 7f8d7622e640 1 -- 172.21.15.40:0/2005754919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d700b44f0 con 0x7f8d700d19b0 2023-10-18T08:56:11.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.203+0000 7f8d7622e640 1 -- 172.21.15.40:0/2005754919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d700c7870 con 0x7f8d700d19b0 2023-10-18T08:56:11.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.204+0000 7f8d7622e640 1 -- 172.21.15.40:0/2005754919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d70071da0 con 0x7f8d700d19b0 2023-10-18T08:56:11.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.205+0000 7f8d65ffb640 1 -- 172.21.15.40:0/2005754919 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8d6c010820 con 0x7f8d700d19b0 2023-10-18T08:56:11.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.206+0000 7f8d65ffb640 1 --2- 172.21.15.40:0/2005754919 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d5406ae40 0x7f8d5406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.206+0000 7f8d65ffb640 1 -- 172.21.15.40:0/2005754919 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8d6c089c20 con 0x7f8d700d19b0 2023-10-18T08:56:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.207+0000 7f8d74a2b640 1 --2- 172.21.15.40:0/2005754919 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d5406ae40 0x7f8d5406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.209+0000 7f8d74a2b640 1 --2- 172.21.15.40:0/2005754919 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d5406ae40 0x7f8d5406d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8d60035000 tx=0x7f8d60004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:11.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.211+0000 7f8d65ffb640 1 -- 172.21.15.40:0/2005754919 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8d6c054bb0 con 0x7f8d700d19b0 2023-10-18T08:56:11.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.406+0000 7f8d7622e640 1 -- 172.21.15.40:0/2005754919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d700c7a60 con 0x7f8d700d19b0 2023-10-18T08:56:11.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.407+0000 7f8d65ffb640 1 -- 172.21.15.40:0/2005754919 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d6c017020 con 0x7f8d700d19b0 2023-10-18T08:56:11.410 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:56:11.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.409+0000 7f8d7622e640 1 -- 172.21.15.40:0/2005754919 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d5406ae40 msgr2=0x7f8d5406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:11.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.409+0000 7f8d7622e640 1 --2- 172.21.15.40:0/2005754919 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d5406ae40 0x7f8d5406d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8d60035000 tx=0x7f8d60004730 comp rx=0 tx=0).stop 2023-10-18T08:56:11.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.409+0000 7f8d7622e640 1 -- 172.21.15.40:0/2005754919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d700d19b0 msgr2=0x7f8d700c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:11.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.409+0000 7f8d7622e640 1 --2- 172.21.15.40:0/2005754919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d700d19b0 0x7f8d700c68b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00d950 tx=0x7f8d6c00de10 comp rx=0 tx=0).stop 2023-10-18T08:56:11.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.409+0000 7f8d7622e640 1 -- 172.21.15.40:0/2005754919 shutdown_connections 2023-10-18T08:56:11.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.409+0000 7f8d7622e640 1 --2- 172.21.15.40:0/2005754919 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d5406ae40 0x7f8d5406d300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:11.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.409+0000 7f8d7622e640 1 --2- 172.21.15.40:0/2005754919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d7015eea0 0x7f8d700c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:11.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.409+0000 7f8d7622e640 1 --2- 172.21.15.40:0/2005754919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d7015eac0 0x7f8d700c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:11.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.409+0000 7f8d7622e640 1 --2- 172.21.15.40:0/2005754919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d700d19b0 0x7f8d700c68b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:11.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.409+0000 7f8d7622e640 1 -- 172.21.15.40:0/2005754919 >> 172.21.15.40:0/2005754919 conn(0x7f8d7009a610 msgr2=0x7f8d700af5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:11.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.409+0000 7f8d7622e640 1 -- 172.21.15.40:0/2005754919 shutdown_connections 2023-10-18T08:56:11.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.410+0000 7f8d7622e640 1 -- 172.21.15.40:0/2005754919 wait complete. 2023-10-18T08:56:11.429 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:56:11.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.655+0000 7f7d7f475640 1 -- 172.21.15.40:0/3806803999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d64005180 msgr2=0x7f7d6400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:11.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.655+0000 7f7d7f475640 1 --2- 172.21.15.40:0/3806803999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d64005180 0x7f7d6400b980 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009e20 tx=0x7f7d640057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:11.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.655+0000 7f7d7f475640 1 -- 172.21.15.40:0/3806803999 shutdown_connections 2023-10-18T08:56:11.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.655+0000 7f7d7f475640 1 --2- 172.21.15.40:0/3806803999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d64005180 0x7f7d6400b980 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:11.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.655+0000 7f7d7f475640 1 -- 172.21.15.40:0/3806803999 >> 172.21.15.40:0/3806803999 conn(0x7f7d7806ca10 msgr2=0x7f7d7806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:11.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.655+0000 7f7d7f475640 1 -- 172.21.15.40:0/3806803999 shutdown_connections 2023-10-18T08:56:11.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.656+0000 7f7d7f475640 1 -- 172.21.15.40:0/3806803999 wait complete. 2023-10-18T08:56:11.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.657+0000 7f7d7f475640 1 Processor -- start 2023-10-18T08:56:11.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.657+0000 7f7d7f475640 1 -- start start 2023-10-18T08:56:11.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.658+0000 7f7d7f475640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d64005180 0x7f7d78150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:11.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.658+0000 7f7d7f475640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d781524e0 0x7f7d7815a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:11.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.658+0000 7f7d7f475640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d7815a9a0 0x7f7d7815df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:11.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.658+0000 7f7d7f475640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7815ece0 con 0x7f7d781524e0 2023-10-18T08:56:11.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.658+0000 7f7d7f475640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7815ae00 con 0x7f7d7815a9a0 2023-10-18T08:56:11.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.658+0000 7f7d7f475640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d78152050 con 0x7f7d64005180 2023-10-18T08:56:11.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.659+0000 7f7d7dc72640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d781524e0 0x7f7d7815a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.659+0000 7f7d7dc72640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d781524e0 0x7f7d7815a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34206/0 (socket says 172.21.15.40:34206) 2023-10-18T08:56:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.659+0000 7f7d7dc72640 1 -- 172.21.15.40:0/1854687384 learned_addr learned my addr 172.21.15.40:0/1854687384 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:11.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.659+0000 7f7d7e473640 1 --2- 172.21.15.40:0/1854687384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d64005180 0x7f7d78150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:11.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.659+0000 7f7d7ec74640 1 --2- 172.21.15.40:0/1854687384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d7815a9a0 0x7f7d7815df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:11.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.659+0000 7f7d7dc72640 1 -- 172.21.15.40:0/1854687384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d64005180 msgr2=0x7f7d78150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:11.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.659+0000 7f7d7dc72640 1 --2- 172.21.15.40:0/1854687384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d64005180 0x7f7d78150a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:11.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.659+0000 7f7d7dc72640 1 -- 172.21.15.40:0/1854687384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d7815a9a0 msgr2=0x7f7d7815df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:11.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.659+0000 7f7d7dc72640 1 --2- 172.21.15.40:0/1854687384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d7815a9a0 0x7f7d7815df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:11.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.659+0000 7f7d7dc72640 1 -- 172.21.15.40:0/1854687384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d78151720 con 0x7f7d781524e0 2023-10-18T08:56:11.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.659+0000 7f7d7dc72640 1 --2- 172.21.15.40:0/1854687384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d781524e0 0x7f7d7815a460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800c790 tx=0x7f7d6800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:11.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.660+0000 7f7d6effd640 1 -- 172.21.15.40:0/1854687384 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d68005270 con 0x7f7d781524e0 2023-10-18T08:56:11.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.660+0000 7f7d6effd640 1 -- 172.21.15.40:0/1854687384 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7d6800a640 con 0x7f7d781524e0 2023-10-18T08:56:11.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.660+0000 7f7d6effd640 1 -- 172.21.15.40:0/1854687384 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6800aa50 con 0x7f7d781524e0 2023-10-18T08:56:11.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.660+0000 7f7d7f475640 1 -- 172.21.15.40:0/1854687384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d78151a00 con 0x7f7d781524e0 2023-10-18T08:56:11.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.660+0000 7f7d7f475640 1 -- 172.21.15.40:0/1854687384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d78150f50 con 0x7f7d781524e0 2023-10-18T08:56:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.661+0000 7f7d7f475640 1 -- 172.21.15.40:0/1854687384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d78066e00 con 0x7f7d781524e0 2023-10-18T08:56:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.662+0000 7f7d6effd640 1 -- 172.21.15.40:0/1854687384 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7d6800abb0 con 0x7f7d781524e0 2023-10-18T08:56:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.663+0000 7f7d6effd640 1 --2- 172.21.15.40:0/1854687384 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7d5c06ae40 0x7f7d5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.663+0000 7f7d6effd640 1 -- 172.21.15.40:0/1854687384 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f7d68088fe0 con 0x7f7d781524e0 2023-10-18T08:56:11.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.663+0000 7f7d7e473640 1 --2- 172.21.15.40:0/1854687384 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7d5c06ae40 0x7f7d5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:11.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.664+0000 7f7d7e473640 1 --2- 172.21.15.40:0/1854687384 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7d5c06ae40 0x7f7d5c06d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7d74000c00 tx=0x7f7d74006a00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:11.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.667+0000 7f7d6effd640 1 -- 172.21.15.40:0/1854687384 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d68053fd0 con 0x7f7d781524e0 2023-10-18T08:56:11.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.862+0000 7f7d7f475640 1 -- 172.21.15.40:0/1854687384 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7d78152190 con 0x7f7d781524e0 2023-10-18T08:56:11.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.863+0000 7f7d6effd640 1 -- 172.21.15.40:0/1854687384 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7d680170a0 con 0x7f7d781524e0 2023-10-18T08:56:11.865 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:56:11.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.866+0000 7f7d7f475640 1 -- 172.21.15.40:0/1854687384 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7d5c06ae40 msgr2=0x7f7d5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:11.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.866+0000 7f7d7f475640 1 --2- 172.21.15.40:0/1854687384 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7d5c06ae40 0x7f7d5c06d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7d74000c00 tx=0x7f7d74006a00 comp rx=0 tx=0).stop 2023-10-18T08:56:11.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.866+0000 7f7d7f475640 1 -- 172.21.15.40:0/1854687384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d781524e0 msgr2=0x7f7d7815a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.866+0000 7f7d7f475640 1 --2- 172.21.15.40:0/1854687384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d781524e0 0x7f7d7815a460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800c790 tx=0x7f7d6800cc50 comp rx=0 tx=0).stop 2023-10-18T08:56:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.866+0000 7f7d7f475640 1 -- 172.21.15.40:0/1854687384 shutdown_connections 2023-10-18T08:56:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.866+0000 7f7d7f475640 1 --2- 172.21.15.40:0/1854687384 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7d5c06ae40 0x7f7d5c06d300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.866+0000 7f7d7f475640 1 --2- 172.21.15.40:0/1854687384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d7815a9a0 0x7f7d7815df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.866+0000 7f7d7f475640 1 --2- 172.21.15.40:0/1854687384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d781524e0 0x7f7d7815a460 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.866+0000 7f7d7f475640 1 --2- 172.21.15.40:0/1854687384 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d64005180 0x7f7d78150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.866+0000 7f7d7f475640 1 -- 172.21.15.40:0/1854687384 >> 172.21.15.40:0/1854687384 conn(0x7f7d7806ca10 msgr2=0x7f7d780ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.866+0000 7f7d7f475640 1 -- 172.21.15.40:0/1854687384 shutdown_connections 2023-10-18T08:56:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:11.866+0000 7f7d7f475640 1 -- 172.21.15.40:0/1854687384 wait complete. 2023-10-18T08:56:11.889 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T08:56:12.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.119+0000 7fa45895d640 1 -- 172.21.15.40:0/1765036327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43c0047c0 msgr2=0x7fa43c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.119+0000 7fa45895d640 1 --2- 172.21.15.40:0/1765036327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa43c0047c0 0x7fa43c004c40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa448009b50 tx=0x7fa43c0057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.119+0000 7fa45359e640 1 -- 172.21.15.40:0/1765036327 reap_dead start 2023-10-18T08:56:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.119+0000 7fa45895d640 1 -- 172.21.15.40:0/1765036327 shutdown_connections 2023-10-18T08:56:12.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.119+0000 7fa45895d640 1 -- 172.21.15.40:0/1765036327 >> 172.21.15.40:0/1765036327 conn(0x7fa454069720 msgr2=0x7fa454069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:12.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.119+0000 7fa45895d640 1 -- 172.21.15.40:0/1765036327 shutdown_connections 2023-10-18T08:56:12.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.119+0000 7fa45895d640 1 -- 172.21.15.40:0/1765036327 wait complete. 2023-10-18T08:56:12.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.121+0000 7fa45895d640 1 Processor -- start 2023-10-18T08:56:12.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.121+0000 7fa45895d640 1 -- start start 2023-10-18T08:56:12.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.122+0000 7fa45895d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa43c0047c0 0x7fa454151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:12.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.122+0000 7fa45895d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa45415a280 0x7fa45415a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:12.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.122+0000 7fa45895d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa45414d960 0x7fa45414de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:12.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.122+0000 7fa45895d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa454156150 con 0x7fa45414d960 2023-10-18T08:56:12.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.122+0000 7fa45895d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4541563f0 con 0x7fa45415a280 2023-10-18T08:56:12.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.122+0000 7fa45895d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4541566d0 con 0x7fa43c0047c0 2023-10-18T08:56:12.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.122+0000 7fa45359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa45414d960 0x7fa45414de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:12.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.122+0000 7fa45359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa45414d960 0x7fa45414de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34222/0 (socket says 172.21.15.40:34222) 2023-10-18T08:56:12.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.122+0000 7fa45359e640 1 -- 172.21.15.40:0/2083700198 learned_addr learned my addr 172.21.15.40:0/2083700198 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:12.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.123+0000 7fa45359e640 1 -- 172.21.15.40:0/2083700198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa43c0047c0 msgr2=0x7fa454151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:12.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.123+0000 7fa45359e640 1 --2- 172.21.15.40:0/2083700198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa43c0047c0 0x7fa454151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:12.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.123+0000 7fa45359e640 1 -- 172.21.15.40:0/2083700198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa45415a280 msgr2=0x7fa45415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:12.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.123+0000 7fa45359e640 1 --2- 172.21.15.40:0/2083700198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa45415a280 0x7fa45415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:12.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.123+0000 7fa45359e640 1 -- 172.21.15.40:0/2083700198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4541569b0 con 0x7fa45414d960 2023-10-18T08:56:12.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.123+0000 7fa45359e640 1 --2- 172.21.15.40:0/2083700198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa45414d960 0x7fa45414de10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa45406b050 tx=0x7fa44c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:12.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.123+0000 7fa43b7fe640 1 -- 172.21.15.40:0/2083700198 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c00ad10 con 0x7fa45414d960 2023-10-18T08:56:12.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.123+0000 7fa43b7fe640 1 -- 172.21.15.40:0/2083700198 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa44c007070 con 0x7fa45414d960 2023-10-18T08:56:12.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.123+0000 7fa45895d640 1 -- 172.21.15.40:0/2083700198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45414e350 con 0x7fa45414d960 2023-10-18T08:56:12.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.123+0000 7fa43b7fe640 1 -- 172.21.15.40:0/2083700198 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c007480 con 0x7fa45414d960 2023-10-18T08:56:12.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.123+0000 7fa45895d640 1 -- 172.21.15.40:0/2083700198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45415ad20 con 0x7fa45414d960 2023-10-18T08:56:12.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.125+0000 7fa45895d640 1 -- 172.21.15.40:0/2083700198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4540aab10 con 0x7fa45414d960 2023-10-18T08:56:12.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.125+0000 7fa43b7fe640 1 -- 172.21.15.40:0/2083700198 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa44c0075e0 con 0x7fa45414d960 2023-10-18T08:56:12.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.126+0000 7fa43b7fe640 1 --2- 172.21.15.40:0/2083700198 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa44006ae70 0x7fa44006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:12.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.126+0000 7fa43b7fe640 1 -- 172.21.15.40:0/2083700198 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa44c089790 con 0x7fa45414d960 2023-10-18T08:56:12.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.127+0000 7fa452d9d640 1 --2- 172.21.15.40:0/2083700198 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa44006ae70 0x7fa44006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:12.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.128+0000 7fa452d9d640 1 --2- 172.21.15.40:0/2083700198 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa44006ae70 0x7fa44006d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa4440058c0 tx=0x7fa444005780 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:12.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.131+0000 7fa43b7fe640 1 -- 172.21.15.40:0/2083700198 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa44c054720 con 0x7fa45414d960 2023-10-18T08:56:12.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.339+0000 7fa45895d640 1 -- 172.21.15.40:0/2083700198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa454071890 con 0x7fa45414d960 2023-10-18T08:56:12.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.340+0000 7fa43b7fe640 1 -- 172.21.15.40:0/2083700198 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa454071890 con 0x7fa45414d960 2023-10-18T08:56:12.342 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T08:56:12.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.342+0000 7fa45895d640 1 -- 172.21.15.40:0/2083700198 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa44006ae70 msgr2=0x7fa44006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:12.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.342+0000 7fa45895d640 1 --2- 172.21.15.40:0/2083700198 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa44006ae70 0x7fa44006d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa4440058c0 tx=0x7fa444005780 comp rx=0 tx=0).stop 2023-10-18T08:56:12.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.342+0000 7fa45895d640 1 -- 172.21.15.40:0/2083700198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa45414d960 msgr2=0x7fa45414de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:12.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.342+0000 7fa45895d640 1 --2- 172.21.15.40:0/2083700198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa45414d960 0x7fa45414de10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa45406b050 tx=0x7fa44c00a430 comp rx=0 tx=0).stop 2023-10-18T08:56:12.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.342+0000 7fa45895d640 1 -- 172.21.15.40:0/2083700198 shutdown_connections 2023-10-18T08:56:12.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.342+0000 7fa45895d640 1 --2- 172.21.15.40:0/2083700198 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa44006ae70 0x7fa44006d330 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:12.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.342+0000 7fa45895d640 1 --2- 172.21.15.40:0/2083700198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa45414d960 0x7fa45414de10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:12.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.342+0000 7fa45895d640 1 --2- 172.21.15.40:0/2083700198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa45415a280 0x7fa45415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:12.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.342+0000 7fa45895d640 1 --2- 172.21.15.40:0/2083700198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa43c0047c0 0x7fa454151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:12.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.343+0000 7fa45895d640 1 -- 172.21.15.40:0/2083700198 >> 172.21.15.40:0/2083700198 conn(0x7fa454069720 msgr2=0x7fa45415d870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:12.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.343+0000 7fa45895d640 1 -- 172.21.15.40:0/2083700198 shutdown_connections 2023-10-18T08:56:12.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.343+0000 7fa45895d640 1 -- 172.21.15.40:0/2083700198 wait complete. 2023-10-18T08:56:12.363 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-18T08:56:12.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:12.364 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-18T08:56:12.469 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T08:56:12.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:12.470 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-18T08:56:12.525 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:56:12.526 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:12.588 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:56:12.588 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:12.642 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:56:12.642 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:12.697 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:12.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.939+0000 7fe00e6ca640 1 -- 172.21.15.40:0/3103521925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0080caec0 msgr2=0x7fe0080cb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:12.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.939+0000 7fe00e6ca640 1 --2- 172.21.15.40:0/3103521925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0080caec0 0x7fe0080cb310 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe00806a5a0 tx=0x7fe00002c260 comp rx=0 tx=0).stop 2023-10-18T08:56:12.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.939+0000 7fe00e6ca640 1 -- 172.21.15.40:0/3103521925 shutdown_connections 2023-10-18T08:56:12.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.939+0000 7fe00e6ca640 1 --2- 172.21.15.40:0/3103521925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0080caec0 0x7fe0080cb310 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:12.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.939+0000 7fe00e6ca640 1 --2- 172.21.15.40:0/3103521925 >> v2:172.21.15.144:3300/0 conn(0x7fe008157870 0x7fe0080ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:12.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.940+0000 7fe00e6ca640 1 -- 172.21.15.40:0/3103521925 >> 172.21.15.40:0/3103521925 conn(0x7fe008069720 msgr2=0x7fe008069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:12.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.940+0000 7fe00e6ca640 1 -- 172.21.15.40:0/3103521925 shutdown_connections 2023-10-18T08:56:12.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.940+0000 7fe00e6ca640 1 -- 172.21.15.40:0/3103521925 wait complete. 2023-10-18T08:56:12.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.941+0000 7fe00e6ca640 1 Processor -- start 2023-10-18T08:56:12.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.942+0000 7fe00e6ca640 1 -- start start 2023-10-18T08:56:12.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.942+0000 7fe00e6ca640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0080ca890 0x7fe0080abf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:12.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.943+0000 7fe00e6ca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0080caec0 0x7fe00814d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:12.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.943+0000 7fe00e6ca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe008157870 0x7fe0081490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:12.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.943+0000 7fe00e6ca640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080aad20 con 0x7fe008157870 2023-10-18T08:56:12.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.943+0000 7fe00e6ca640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080ab000 con 0x7fe0080caec0 2023-10-18T08:56:12.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.943+0000 7fe00e6ca640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080ab2e0 con 0x7fe0080ca890 2023-10-18T08:56:12.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.943+0000 7fe00dec9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe008157870 0x7fe0081490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:12.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.943+0000 7fe00dec9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe008157870 0x7fe0081490c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34242/0 (socket says 172.21.15.40:34242) 2023-10-18T08:56:12.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.943+0000 7fe00dec9640 1 -- 172.21.15.40:0/3891581353 learned_addr learned my addr 172.21.15.40:0/3891581353 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:12.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.943+0000 7fe00d6c8640 1 --2- 172.21.15.40:0/3891581353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0080ca890 0x7fe0080abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:12.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.943+0000 7fe00cec7640 1 --2- 172.21.15.40:0/3891581353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0080caec0 0x7fe00814d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:12.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.944+0000 7fe00dec9640 1 -- 172.21.15.40:0/3891581353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0080ca890 msgr2=0x7fe0080abf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:12.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.944+0000 7fe00dec9640 1 --2- 172.21.15.40:0/3891581353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0080ca890 0x7fe0080abf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:12.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.944+0000 7fe00dec9640 1 -- 172.21.15.40:0/3891581353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0080caec0 msgr2=0x7fe00814d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:12.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.944+0000 7fe00dec9640 1 --2- 172.21.15.40:0/3891581353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0080caec0 0x7fe00814d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:12.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.944+0000 7fe00dec9640 1 -- 172.21.15.40:0/3891581353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe000009900 con 0x7fe008157870 2023-10-18T08:56:12.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.944+0000 7fe00dec9640 1 --2- 172.21.15.40:0/3891581353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe008157870 0x7fe0081490c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe000037040 tx=0x7fe000003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:12.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.944+0000 7fdffdffb640 1 -- 172.21.15.40:0/3891581353 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000004250 con 0x7fe008157870 2023-10-18T08:56:12.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.944+0000 7fdffdffb640 1 -- 172.21.15.40:0/3891581353 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe0000044a0 con 0x7fe008157870 2023-10-18T08:56:12.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.944+0000 7fe00e6ca640 1 -- 172.21.15.40:0/3891581353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0080ab5c0 con 0x7fe008157870 2023-10-18T08:56:12.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.944+0000 7fdffdffb640 1 -- 172.21.15.40:0/3891581353 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000004890 con 0x7fe008157870 2023-10-18T08:56:12.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.944+0000 7fe00e6ca640 1 -- 172.21.15.40:0/3891581353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0080ac4b0 con 0x7fe008157870 2023-10-18T08:56:12.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.946+0000 7fe00e6ca640 1 -- 172.21.15.40:0/3891581353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0081312c0 con 0x7fe008157870 2023-10-18T08:56:12.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.947+0000 7fdffdffb640 1 -- 172.21.15.40:0/3891581353 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe00003b040 con 0x7fe008157870 2023-10-18T08:56:12.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.947+0000 7fdffdffb640 1 --2- 172.21.15.40:0/3891581353 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdfec06ae40 0x7fdfec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:12.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.948+0000 7fdffdffb640 1 -- 172.21.15.40:0/3891581353 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe0000ac490 con 0x7fe008157870 2023-10-18T08:56:12.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.948+0000 7fe00d6c8640 1 --2- 172.21.15.40:0/3891581353 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdfec06ae40 0x7fdfec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.949+0000 7fe00d6c8640 1 --2- 172.21.15.40:0/3891581353 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdfec06ae40 0x7fdfec06d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe0040098e0 tx=0x7fe004009060 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:12.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:12.952+0000 7fdffdffb640 1 -- 172.21.15.40:0/3891581353 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe0081312c0 con 0x7fe008157870 2023-10-18T08:56:13.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.146+0000 7fe00e6ca640 1 -- 172.21.15.40:0/3891581353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0081312c0 con 0x7fe008157870 2023-10-18T08:56:13.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.148+0000 7fdffdffb640 1 -- 172.21.15.40:0/3891581353 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16766 (secure 0 0 0) 0x7fe0081312c0 con 0x7fe008157870 2023-10-18T08:56:13.150 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:13.151 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:02.868692+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:13.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.151+0000 7fe00e6ca640 1 -- 172.21.15.40:0/3891581353 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdfec06ae40 msgr2=0x7fdfec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:13.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.151+0000 7fe00e6ca640 1 --2- 172.21.15.40:0/3891581353 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdfec06ae40 0x7fdfec06d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe0040098e0 tx=0x7fe004009060 comp rx=0 tx=0).stop 2023-10-18T08:56:13.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.151+0000 7fe00e6ca640 1 -- 172.21.15.40:0/3891581353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe008157870 msgr2=0x7fe0081490c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:13.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.151+0000 7fe00e6ca640 1 --2- 172.21.15.40:0/3891581353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe008157870 0x7fe0081490c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe000037040 tx=0x7fe000003c10 comp rx=0 tx=0).stop 2023-10-18T08:56:13.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.151+0000 7fe00e6ca640 1 -- 172.21.15.40:0/3891581353 shutdown_connections 2023-10-18T08:56:13.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.151+0000 7fe00e6ca640 1 --2- 172.21.15.40:0/3891581353 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdfec06ae40 0x7fdfec06d300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.151+0000 7fe00e6ca640 1 --2- 172.21.15.40:0/3891581353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe008157870 0x7fe0081490c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.151+0000 7fe00e6ca640 1 --2- 172.21.15.40:0/3891581353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0080caec0 0x7fe00814d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.151+0000 7fe00e6ca640 1 --2- 172.21.15.40:0/3891581353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0080ca890 0x7fe0080abf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.151+0000 7fe00e6ca640 1 -- 172.21.15.40:0/3891581353 >> 172.21.15.40:0/3891581353 conn(0x7fe008069720 msgr2=0x7fe0080af3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.151+0000 7fe00e6ca640 1 -- 172.21.15.40:0/3891581353 shutdown_connections 2023-10-18T08:56:13.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.151+0000 7fe00e6ca640 1 -- 172.21.15.40:0/3891581353 wait complete. 2023-10-18T08:56:13.173 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:55:13.210788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:55:13.772849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:56:13.174 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:56:13.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.407+0000 7f55ab59e640 1 -- 172.21.15.40:0/3998139075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac070790 msgr2=0x7f55ac0cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:13.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.407+0000 7f55ab59e640 1 --2- 172.21.15.40:0/3998139075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac070790 0x7f55ac0cc980 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f55a4005fd0 tx=0x7f55a402c2d0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.407+0000 7f55ab59e640 1 -- 172.21.15.40:0/3998139075 shutdown_connections 2023-10-18T08:56:13.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.407+0000 7f55ab59e640 1 --2- 172.21.15.40:0/3998139075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac070790 0x7f55ac0cc980 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.407+0000 7f55ab59e640 1 --2- 172.21.15.40:0/3998139075 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55ac0b0a50 0x7f55ac070250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.407+0000 7f55ab59e640 1 -- 172.21.15.40:0/3998139075 >> 172.21.15.40:0/3998139075 conn(0x7f55ac069720 msgr2=0x7f55ac069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:13.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.407+0000 7f55ab59e640 1 -- 172.21.15.40:0/3998139075 shutdown_connections 2023-10-18T08:56:13.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.407+0000 7f55ab59e640 1 -- 172.21.15.40:0/3998139075 wait complete. 2023-10-18T08:56:13.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.409+0000 7f55ab59e640 1 Processor -- start 2023-10-18T08:56:13.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.409+0000 7f55ab59e640 1 -- start start 2023-10-18T08:56:13.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.410+0000 7f55ab59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55ac070790 0x7f55ac13bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:13.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.410+0000 7f55ab59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55ac0b0a50 0x7f55ac13f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:13.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.410+0000 7f55ab59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac1411b0 0x7f55ac13fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:13.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.410+0000 7f55ab59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac0c35d0 con 0x7f55ac1411b0 2023-10-18T08:56:13.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.410+0000 7f55ab59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac1597f0 con 0x7f55ac0b0a50 2023-10-18T08:56:13.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.410+0000 7f55ab59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac0c2d30 con 0x7f55ac070790 2023-10-18T08:56:13.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.410+0000 7f55aad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac1411b0 0x7f55ac13fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:13.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.410+0000 7f55aad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac1411b0 0x7f55ac13fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34256/0 (socket says 172.21.15.40:34256) 2023-10-18T08:56:13.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.410+0000 7f55aad9d640 1 -- 172.21.15.40:0/2972522810 learned_addr learned my addr 172.21.15.40:0/2972522810 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:13.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.411+0000 7f55aa59c640 1 --2- 172.21.15.40:0/2972522810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55ac070790 0x7f55ac13bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:13.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.411+0000 7f55a3fff640 1 --2- 172.21.15.40:0/2972522810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55ac0b0a50 0x7f55ac13f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:13.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.411+0000 7f55aad9d640 1 -- 172.21.15.40:0/2972522810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55ac070790 msgr2=0x7f55ac13bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:13.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.411+0000 7f55aad9d640 1 --2- 172.21.15.40:0/2972522810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55ac070790 0x7f55ac13bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.411+0000 7f55aad9d640 1 -- 172.21.15.40:0/2972522810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55ac0b0a50 msgr2=0x7f55ac13f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:13.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.411+0000 7f55aad9d640 1 --2- 172.21.15.40:0/2972522810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55ac0b0a50 0x7f55ac13f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.411+0000 7f55aad9d640 1 -- 172.21.15.40:0/2972522810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a4009900 con 0x7f55ac1411b0 2023-10-18T08:56:13.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.411+0000 7f55aad9d640 1 --2- 172.21.15.40:0/2972522810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac1411b0 0x7f55ac13fd30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f55a4037040 tx=0x7f55a4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:13.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.412+0000 7f55a2ffd640 1 -- 172.21.15.40:0/2972522810 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a4004240 con 0x7f55ac1411b0 2023-10-18T08:56:13.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.412+0000 7f55a2ffd640 1 -- 172.21.15.40:0/2972522810 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f55a4004490 con 0x7f55ac1411b0 2023-10-18T08:56:13.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.412+0000 7f55a2ffd640 1 -- 172.21.15.40:0/2972522810 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a4004820 con 0x7f55ac1411b0 2023-10-18T08:56:13.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.412+0000 7f55ab59e640 1 -- 172.21.15.40:0/2972522810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ac0c3010 con 0x7f55ac1411b0 2023-10-18T08:56:13.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.412+0000 7f55ab59e640 1 -- 172.21.15.40:0/2972522810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ac159a00 con 0x7f55ac1411b0 2023-10-18T08:56:13.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.413+0000 7f55ab59e640 1 -- 172.21.15.40:0/2972522810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55ac066e00 con 0x7f55ac1411b0 2023-10-18T08:56:13.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.414+0000 7f55a2ffd640 1 -- 172.21.15.40:0/2972522810 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f55a403b040 con 0x7f55ac1411b0 2023-10-18T08:56:13.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.415+0000 7f55a2ffd640 1 --2- 172.21.15.40:0/2972522810 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f558c06ae40 0x7f558c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:13.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.415+0000 7f55a2ffd640 1 -- 172.21.15.40:0/2972522810 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f55a40ac380 con 0x7f55ac1411b0 2023-10-18T08:56:13.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.416+0000 7f55aa59c640 1 --2- 172.21.15.40:0/2972522810 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f558c06ae40 0x7f558c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:13.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.417+0000 7f55aa59c640 1 --2- 172.21.15.40:0/2972522810 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f558c06ae40 0x7f558c06d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f559c000f60 tx=0x7f559c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:13.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.419+0000 7f55a2ffd640 1 -- 172.21.15.40:0/2972522810 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55a4077310 con 0x7f55ac1411b0 2023-10-18T08:56:13.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.619+0000 7f55ab59e640 1 -- 172.21.15.40:0/2972522810 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f55ac0cb3f0 con 0x7f55ac1411b0 2023-10-18T08:56:13.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.619+0000 7f55a2ffd640 1 -- 172.21.15.40:0/2972522810 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55a407fe60 con 0x7f55ac1411b0 2023-10-18T08:56:13.622 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:56:13.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.622+0000 7f55ab59e640 1 -- 172.21.15.40:0/2972522810 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f558c06ae40 msgr2=0x7f558c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:13.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.622+0000 7f55ab59e640 1 --2- 172.21.15.40:0/2972522810 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f558c06ae40 0x7f558c06d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f559c000f60 tx=0x7f559c009040 comp rx=0 tx=0).stop 2023-10-18T08:56:13.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.622+0000 7f55ab59e640 1 -- 172.21.15.40:0/2972522810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac1411b0 msgr2=0x7f55ac13fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:13.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.622+0000 7f55ab59e640 1 --2- 172.21.15.40:0/2972522810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac1411b0 0x7f55ac13fd30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f55a4037040 tx=0x7f55a4003ba0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.622+0000 7f55ab59e640 1 -- 172.21.15.40:0/2972522810 shutdown_connections 2023-10-18T08:56:13.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.622+0000 7f55ab59e640 1 --2- 172.21.15.40:0/2972522810 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f558c06ae40 0x7f558c06d300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.622+0000 7f55ab59e640 1 --2- 172.21.15.40:0/2972522810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac1411b0 0x7f55ac13fd30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.622+0000 7f55ab59e640 1 --2- 172.21.15.40:0/2972522810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55ac0b0a50 0x7f55ac13f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.622+0000 7f55ab59e640 1 --2- 172.21.15.40:0/2972522810 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55ac070790 0x7f55ac13bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.622+0000 7f55ab59e640 1 -- 172.21.15.40:0/2972522810 >> 172.21.15.40:0/2972522810 conn(0x7f55ac069720 msgr2=0x7f55ac0acb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:13.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.623+0000 7f55ab59e640 1 -- 172.21.15.40:0/2972522810 shutdown_connections 2023-10-18T08:56:13.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.623+0000 7f55ab59e640 1 -- 172.21.15.40:0/2972522810 wait complete. 2023-10-18T08:56:13.645 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:56:13.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.866+0000 7f8c9319a640 1 -- 172.21.15.40:0/3648197018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c8c06c240 msgr2=0x7f8c8c0cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:13.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.866+0000 7f8c9319a640 1 --2- 172.21.15.40:0/3648197018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c8c06c240 0x7f8c8c0cb960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c005fd0 tx=0x7f8c7c02be20 comp rx=0 tx=0).stop 2023-10-18T08:56:13.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.866+0000 7f8c9319a640 1 -- 172.21.15.40:0/3648197018 shutdown_connections 2023-10-18T08:56:13.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.866+0000 7f8c9319a640 1 --2- 172.21.15.40:0/3648197018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c8c0cbea0 0x7f8c8c0b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.866+0000 7f8c9319a640 1 --2- 172.21.15.40:0/3648197018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c8c06c240 0x7f8c8c0cb960 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.866+0000 7f8c9319a640 1 -- 172.21.15.40:0/3648197018 >> 172.21.15.40:0/3648197018 conn(0x7f8c8c06ca10 msgr2=0x7f8c8c06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:13.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.866+0000 7f8c9319a640 1 -- 172.21.15.40:0/3648197018 shutdown_connections 2023-10-18T08:56:13.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.866+0000 7f8c9319a640 1 -- 172.21.15.40:0/3648197018 wait complete. 2023-10-18T08:56:13.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.868+0000 7f8c9319a640 1 Processor -- start 2023-10-18T08:56:13.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.869+0000 7f8c9319a640 1 -- start start 2023-10-18T08:56:13.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.870+0000 7f8c9319a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c8c06c240 0x7f8c8c1f1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:13.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.870+0000 7f8c9319a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c8c0cbea0 0x7f8c8c15c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:13.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.870+0000 7f8c9319a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c8c159330 0x7f8c8c153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:13.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.870+0000 7f8c9319a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c0c1ee0 con 0x7f8c8c0cbea0 2023-10-18T08:56:13.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.870+0000 7f8c9319a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c15c830 con 0x7f8c8c159330 2023-10-18T08:56:13.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.870+0000 7f8c9319a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c0c2130 con 0x7f8c8c06c240 2023-10-18T08:56:13.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.870+0000 7f8c91997640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c8c0cbea0 0x7f8c8c15c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:13.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.870+0000 7f8c91997640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c8c0cbea0 0x7f8c8c15c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34266/0 (socket says 172.21.15.40:34266) 2023-10-18T08:56:13.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.870+0000 7f8c91997640 1 -- 172.21.15.40:0/1419049073 learned_addr learned my addr 172.21.15.40:0/1419049073 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:13.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.871+0000 7f8c92198640 1 --2- 172.21.15.40:0/1419049073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c8c06c240 0x7f8c8c1f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:13.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.871+0000 7f8c92999640 1 --2- 172.21.15.40:0/1419049073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c8c159330 0x7f8c8c153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:13.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.871+0000 7f8c91997640 1 -- 172.21.15.40:0/1419049073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c8c06c240 msgr2=0x7f8c8c1f1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:13.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.871+0000 7f8c91997640 1 --2- 172.21.15.40:0/1419049073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c8c06c240 0x7f8c8c1f1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.871+0000 7f8c91997640 1 -- 172.21.15.40:0/1419049073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c8c159330 msgr2=0x7f8c8c153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:13.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.871+0000 7f8c91997640 1 --2- 172.21.15.40:0/1419049073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c8c159330 0x7f8c8c153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:13.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.871+0000 7f8c91997640 1 -- 172.21.15.40:0/1419049073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c009560 con 0x7f8c8c0cbea0 2023-10-18T08:56:13.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.871+0000 7f8c92198640 1 --2- 172.21.15.40:0/1419049073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c8c06c240 0x7f8c8c1f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:13.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.871+0000 7f8c91997640 1 --2- 172.21.15.40:0/1419049073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c8c0cbea0 0x7f8c8c15c2f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c036040 tx=0x7f8c7c02daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:13.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.871+0000 7f8c82ffd640 1 -- 172.21.15.40:0/1419049073 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c003800 con 0x7f8c8c0cbea0 2023-10-18T08:56:13.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.871+0000 7f8c82ffd640 1 -- 172.21.15.40:0/1419049073 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8c7c003a50 con 0x7f8c8c0cbea0 2023-10-18T08:56:13.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.871+0000 7f8c82ffd640 1 -- 172.21.15.40:0/1419049073 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c003e00 con 0x7f8c8c0cbea0 2023-10-18T08:56:13.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.871+0000 7f8c9319a640 1 -- 172.21.15.40:0/1419049073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8c0c2410 con 0x7f8c8c0cbea0 2023-10-18T08:56:13.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.872+0000 7f8c9319a640 1 -- 172.21.15.40:0/1419049073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8c0c29d0 con 0x7f8c8c0cbea0 2023-10-18T08:56:13.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.873+0000 7f8c9319a640 1 -- 172.21.15.40:0/1419049073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8c066e00 con 0x7f8c8c0cbea0 2023-10-18T08:56:13.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.874+0000 7f8c82ffd640 1 -- 172.21.15.40:0/1419049073 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8c7c03e450 con 0x7f8c8c0cbea0 2023-10-18T08:56:13.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.875+0000 7f8c82ffd640 1 --2- 172.21.15.40:0/1419049073 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c7006ae40 0x7f8c7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:13.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.875+0000 7f8c82ffd640 1 -- 172.21.15.40:0/1419049073 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8c7c0ac2d0 con 0x7f8c8c0cbea0 2023-10-18T08:56:13.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.875+0000 7f8c92198640 1 --2- 172.21.15.40:0/1419049073 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c7006ae40 0x7f8c7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:13.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.877+0000 7f8c92198640 1 --2- 172.21.15.40:0/1419049073 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c7006ae40 0x7f8c7006d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8c88000f60 tx=0x7f8c88009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:13.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:13.879+0000 7f8c82ffd640 1 -- 172.21.15.40:0/1419049073 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c7c077380 con 0x7f8c8c0cbea0 2023-10-18T08:56:14.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.073+0000 7f8c9319a640 1 -- 172.21.15.40:0/1419049073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c8c0b1ef0 con 0x7f8c8c0cbea0 2023-10-18T08:56:14.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.074+0000 7f8c82ffd640 1 -- 172.21.15.40:0/1419049073 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8c7c07a9a0 con 0x7f8c8c0cbea0 2023-10-18T08:56:14.076 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:56:14.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.077+0000 7f8c9319a640 1 -- 172.21.15.40:0/1419049073 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c7006ae40 msgr2=0x7f8c7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:14.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.077+0000 7f8c9319a640 1 --2- 172.21.15.40:0/1419049073 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c7006ae40 0x7f8c7006d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8c88000f60 tx=0x7f8c88009040 comp rx=0 tx=0).stop 2023-10-18T08:56:14.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.077+0000 7f8c9319a640 1 -- 172.21.15.40:0/1419049073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c8c0cbea0 msgr2=0x7f8c8c15c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:14.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.077+0000 7f8c9319a640 1 --2- 172.21.15.40:0/1419049073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c8c0cbea0 0x7f8c8c15c2f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c036040 tx=0x7f8c7c02daf0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.077+0000 7f8c9319a640 1 -- 172.21.15.40:0/1419049073 shutdown_connections 2023-10-18T08:56:14.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.077+0000 7f8c9319a640 1 --2- 172.21.15.40:0/1419049073 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8c7006ae40 0x7f8c7006d300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.077+0000 7f8c9319a640 1 --2- 172.21.15.40:0/1419049073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c8c159330 0x7f8c8c153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.077+0000 7f8c9319a640 1 --2- 172.21.15.40:0/1419049073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c8c0cbea0 0x7f8c8c15c2f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.077+0000 7f8c9319a640 1 --2- 172.21.15.40:0/1419049073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c8c06c240 0x7f8c8c1f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.077+0000 7f8c9319a640 1 -- 172.21.15.40:0/1419049073 >> 172.21.15.40:0/1419049073 conn(0x7f8c8c06ca10 msgr2=0x7f8c8c157050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:14.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.077+0000 7f8c9319a640 1 -- 172.21.15.40:0/1419049073 shutdown_connections 2023-10-18T08:56:14.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.077+0000 7f8c9319a640 1 -- 172.21.15.40:0/1419049073 wait complete. 2023-10-18T08:56:14.097 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:56:14.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.324+0000 7fc0c11ae640 1 -- 172.21.15.40:0/832641822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc071270 msgr2=0x7fc0bc06c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:14.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.324+0000 7fc0c11ae640 1 --2- 172.21.15.40:0/832641822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc071270 0x7fc0bc06c100 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4005fd0 tx=0x7fc0b402c050 comp rx=0 tx=0).stop 2023-10-18T08:56:14.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.324+0000 7fc0c11ae640 1 -- 172.21.15.40:0/832641822 shutdown_connections 2023-10-18T08:56:14.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.324+0000 7fc0c11ae640 1 --2- 172.21.15.40:0/832641822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc071270 0x7fc0bc06c100 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.324+0000 7fc0c11ae640 1 --2- 172.21.15.40:0/832641822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0bc0c9620 0x7fc0bc06bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.324+0000 7fc0c11ae640 1 -- 172.21.15.40:0/832641822 >> 172.21.15.40:0/832641822 conn(0x7fc0bc06ca10 msgr2=0x7fc0bc06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:14.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.324+0000 7fc0c11ae640 1 -- 172.21.15.40:0/832641822 shutdown_connections 2023-10-18T08:56:14.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.324+0000 7fc0c11ae640 1 -- 172.21.15.40:0/832641822 wait complete. 2023-10-18T08:56:14.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.326+0000 7fc0c11ae640 1 Processor -- start 2023-10-18T08:56:14.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.326+0000 7fc0c11ae640 1 -- start start 2023-10-18T08:56:14.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.327+0000 7fc0c11ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc071270 0x7fc0bc145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:14.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.327+0000 7fc0c11ae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0bc0c9620 0x7fc0bc0c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:14.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.327+0000 7fc0c11ae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0bc15af70 0x7fc0bc0c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:14.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.327+0000 7fc0c11ae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc0c76f0 con 0x7fc0bc071270 2023-10-18T08:56:14.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.327+0000 7fc0c11ae640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc0c79d0 con 0x7fc0bc0c9620 2023-10-18T08:56:14.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.327+0000 7fc0c11ae640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc0c7cb0 con 0x7fc0bc15af70 2023-10-18T08:56:14.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.328+0000 7fc0bbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc071270 0x7fc0bc145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:14.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.328+0000 7fc0bbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc071270 0x7fc0bc145a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34288/0 (socket says 172.21.15.40:34288) 2023-10-18T08:56:14.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.328+0000 7fc0bbfff640 1 -- 172.21.15.40:0/874762211 learned_addr learned my addr 172.21.15.40:0/874762211 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:14.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.328+0000 7fc0bbfff640 1 -- 172.21.15.40:0/874762211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0bc15af70 msgr2=0x7fc0bc0c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:14.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.328+0000 7fc0bbfff640 1 --2- 172.21.15.40:0/874762211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0bc15af70 0x7fc0bc0c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.328+0000 7fc0bbfff640 1 -- 172.21.15.40:0/874762211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0bc0c9620 msgr2=0x7fc0bc0c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:14.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.328+0000 7fc0bbfff640 1 --2- 172.21.15.40:0/874762211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0bc0c9620 0x7fc0bc0c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.328+0000 7fc0bbfff640 1 -- 172.21.15.40:0/874762211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b4009840 con 0x7fc0bc071270 2023-10-18T08:56:14.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.329+0000 7fc0bbfff640 1 --2- 172.21.15.40:0/874762211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc071270 0x7fc0bc145a90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00d950 tx=0x7fc0ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:14.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.329+0000 7fc0b8ff9640 1 -- 172.21.15.40:0/874762211 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac009c30 con 0x7fc0bc071270 2023-10-18T08:56:14.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.329+0000 7fc0b8ff9640 1 -- 172.21.15.40:0/874762211 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc0ac009e80 con 0x7fc0bc071270 2023-10-18T08:56:14.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.329+0000 7fc0b8ff9640 1 -- 172.21.15.40:0/874762211 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0106c0 con 0x7fc0bc071270 2023-10-18T08:56:14.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.329+0000 7fc0c11ae640 1 -- 172.21.15.40:0/874762211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0bc0c7f90 con 0x7fc0bc071270 2023-10-18T08:56:14.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.329+0000 7fc0c11ae640 1 -- 172.21.15.40:0/874762211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0bc145fd0 con 0x7fc0bc071270 2023-10-18T08:56:14.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.330+0000 7fc0c11ae640 1 -- 172.21.15.40:0/874762211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0bc0b2370 con 0x7fc0bc071270 2023-10-18T08:56:14.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.331+0000 7fc0b8ff9640 1 -- 172.21.15.40:0/874762211 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc0ac010820 con 0x7fc0bc071270 2023-10-18T08:56:14.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.332+0000 7fc0b8ff9640 1 --2- 172.21.15.40:0/874762211 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc0a806ae40 0x7fc0a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:14.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.332+0000 7fc0b8ff9640 1 -- 172.21.15.40:0/874762211 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc0ac089c20 con 0x7fc0bc071270 2023-10-18T08:56:14.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.333+0000 7fc0bb7fe640 1 --2- 172.21.15.40:0/874762211 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc0a806ae40 0x7fc0a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:14.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.334+0000 7fc0bb7fe640 1 --2- 172.21.15.40:0/874762211 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc0a806ae40 0x7fc0a806d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0007fd0 tx=0x7fc0b0007920 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:14.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.336+0000 7fc0b8ff9640 1 -- 172.21.15.40:0/874762211 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc0ac054bb0 con 0x7fc0bc071270 2023-10-18T08:56:14.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.528+0000 7fc0c11ae640 1 -- 172.21.15.40:0/874762211 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc0bc0c9b10 con 0x7fc0bc071270 2023-10-18T08:56:14.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.529+0000 7fc0b8ff9640 1 -- 172.21.15.40:0/874762211 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc0ac0170a0 con 0x7fc0bc071270 2023-10-18T08:56:14.532 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:56:14.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.532+0000 7fc0c11ae640 1 -- 172.21.15.40:0/874762211 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc0a806ae40 msgr2=0x7fc0a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:14.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.532+0000 7fc0c11ae640 1 --2- 172.21.15.40:0/874762211 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc0a806ae40 0x7fc0a806d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0007fd0 tx=0x7fc0b0007920 comp rx=0 tx=0).stop 2023-10-18T08:56:14.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.532+0000 7fc0c11ae640 1 -- 172.21.15.40:0/874762211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc071270 msgr2=0x7fc0bc145a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:14.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.532+0000 7fc0c11ae640 1 --2- 172.21.15.40:0/874762211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc071270 0x7fc0bc145a90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00d950 tx=0x7fc0ac00de10 comp rx=0 tx=0).stop 2023-10-18T08:56:14.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.532+0000 7fc0c11ae640 1 -- 172.21.15.40:0/874762211 shutdown_connections 2023-10-18T08:56:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.532+0000 7fc0c11ae640 1 --2- 172.21.15.40:0/874762211 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc0a806ae40 0x7fc0a806d300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.532+0000 7fc0c11ae640 1 --2- 172.21.15.40:0/874762211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0bc15af70 0x7fc0bc0c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.532+0000 7fc0c11ae640 1 --2- 172.21.15.40:0/874762211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0bc0c9620 0x7fc0bc0c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.532+0000 7fc0c11ae640 1 --2- 172.21.15.40:0/874762211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0bc071270 0x7fc0bc145a90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.532+0000 7fc0c11ae640 1 -- 172.21.15.40:0/874762211 >> 172.21.15.40:0/874762211 conn(0x7fc0bc06ca10 msgr2=0x7fc0bc15a260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.533+0000 7fc0c11ae640 1 -- 172.21.15.40:0/874762211 shutdown_connections 2023-10-18T08:56:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.533+0000 7fc0c11ae640 1 -- 172.21.15.40:0/874762211 wait complete. 2023-10-18T08:56:14.555 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T08:56:14.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.789+0000 7f16e1260640 1 -- 172.21.15.40:0/2523556868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16c80039c0 msgr2=0x7f16c8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:14.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.789+0000 7f16e1260640 1 --2- 172.21.15.40:0/2523556868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16c80039c0 0x7f16c8003de0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f16cc006f00 tx=0x7f16c80057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.789+0000 7f16e1260640 1 -- 172.21.15.40:0/2523556868 shutdown_connections 2023-10-18T08:56:14.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.789+0000 7f16e1260640 1 --2- 172.21.15.40:0/2523556868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16c80039c0 0x7f16c8003de0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.789+0000 7f16e1260640 1 -- 172.21.15.40:0/2523556868 >> 172.21.15.40:0/2523556868 conn(0x7f16dc08ad00 msgr2=0x7f16dc09a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:14.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.790+0000 7f16e1260640 1 -- 172.21.15.40:0/2523556868 shutdown_connections 2023-10-18T08:56:14.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.790+0000 7f16e1260640 1 -- 172.21.15.40:0/2523556868 wait complete. 2023-10-18T08:56:14.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.791+0000 7f16e1260640 1 Processor -- start 2023-10-18T08:56:14.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.792+0000 7f16e1260640 1 -- start start 2023-10-18T08:56:14.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.792+0000 7f16e1260640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16c80039c0 0x7f16dc14f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:14.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.792+0000 7f16e1260640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16dc15d010 0x7f16dc1f18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:14.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.792+0000 7f16e1260640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16dc15e7f0 0x7f16dc0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:14.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.792+0000 7f16e1260640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc0b50e0 con 0x7f16dc15d010 2023-10-18T08:56:14.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.792+0000 7f16e1260640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc14f670 con 0x7f16c80039c0 2023-10-18T08:56:14.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.793+0000 7f16e1260640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc0b5380 con 0x7f16dc15e7f0 2023-10-18T08:56:14.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.793+0000 7f16d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16dc15d010 0x7f16dc1f18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:14.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.793+0000 7f16d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16dc15d010 0x7f16dc1f18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34306/0 (socket says 172.21.15.40:34306) 2023-10-18T08:56:14.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.793+0000 7f16d3fff640 1 -- 172.21.15.40:0/3793692473 learned_addr learned my addr 172.21.15.40:0/3793692473 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:14.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.793+0000 7f16dbfff640 1 --2- 172.21.15.40:0/3793692473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16c80039c0 0x7f16dc14f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:14.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.793+0000 7f16e0a5f640 1 --2- 172.21.15.40:0/3793692473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16dc15e7f0 0x7f16dc0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:14.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.793+0000 7f16d3fff640 1 -- 172.21.15.40:0/3793692473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16dc15e7f0 msgr2=0x7f16dc0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:14.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.793+0000 7f16d3fff640 1 --2- 172.21.15.40:0/3793692473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16dc15e7f0 0x7f16dc0ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.793+0000 7f16d3fff640 1 -- 172.21.15.40:0/3793692473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16c80039c0 msgr2=0x7f16dc14f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:14.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.793+0000 7f16d3fff640 1 --2- 172.21.15.40:0/3793692473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16c80039c0 0x7f16dc14f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:14.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.793+0000 7f16d3fff640 1 -- 172.21.15.40:0/3793692473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc0b5660 con 0x7f16dc15d010 2023-10-18T08:56:14.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.793+0000 7f16dbfff640 1 --2- 172.21.15.40:0/3793692473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16c80039c0 0x7f16dc14f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:56:14.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.794+0000 7f16d3fff640 1 --2- 172.21.15.40:0/3793692473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16dc15d010 0x7f16dc1f18c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f16c400d5b0 tx=0x7f16c400da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:14.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.794+0000 7f16d97fa640 1 -- 172.21.15.40:0/3793692473 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c40062e0 con 0x7f16dc15d010 2023-10-18T08:56:14.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.794+0000 7f16d97fa640 1 -- 172.21.15.40:0/3793692473 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f16c4003610 con 0x7f16dc15d010 2023-10-18T08:56:14.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.794+0000 7f16d97fa640 1 -- 172.21.15.40:0/3793692473 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c40039c0 con 0x7f16dc15d010 2023-10-18T08:56:14.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.794+0000 7f16e1260640 1 -- 172.21.15.40:0/3793692473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16dc0b5940 con 0x7f16dc15d010 2023-10-18T08:56:14.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.794+0000 7f16e1260640 1 -- 172.21.15.40:0/3793692473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16dc1f1e00 con 0x7f16dc15d010 2023-10-18T08:56:14.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.795+0000 7f16e1260640 1 -- 172.21.15.40:0/3793692473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16dc071da0 con 0x7f16dc15d010 2023-10-18T08:56:14.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.796+0000 7f16d97fa640 1 -- 172.21.15.40:0/3793692473 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f16c4003b20 con 0x7f16dc15d010 2023-10-18T08:56:14.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.797+0000 7f16d97fa640 1 --2- 172.21.15.40:0/3793692473 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f16bc06ae90 0x7f16bc06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:14.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.797+0000 7f16d97fa640 1 -- 172.21.15.40:0/3793692473 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f16c4089e10 con 0x7f16dc15d010 2023-10-18T08:56:14.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.798+0000 7f16dbfff640 1 --2- 172.21.15.40:0/3793692473 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f16bc06ae90 0x7f16bc06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:14.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.799+0000 7f16dbfff640 1 --2- 172.21.15.40:0/3793692473 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f16bc06ae90 0x7f16bc06d350 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f16cc034000 tx=0x7f16cc039040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:14.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:14.801+0000 7f16d97fa640 1 -- 172.21.15.40:0/3793692473 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f16c408d030 con 0x7f16dc15d010 2023-10-18T08:56:15.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.008+0000 7f16e1260640 1 -- 172.21.15.40:0/3793692473 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f16dc066e00 con 0x7f16dc15d010 2023-10-18T08:56:15.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.009+0000 7f16d97fa640 1 -- 172.21.15.40:0/3793692473 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f16c4014070 con 0x7f16dc15d010 2023-10-18T08:56:15.011 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T08:56:15.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.012+0000 7f16e1260640 1 -- 172.21.15.40:0/3793692473 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f16bc06ae90 msgr2=0x7f16bc06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:15.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.012+0000 7f16e1260640 1 --2- 172.21.15.40:0/3793692473 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f16bc06ae90 0x7f16bc06d350 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f16cc034000 tx=0x7f16cc039040 comp rx=0 tx=0).stop 2023-10-18T08:56:15.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.012+0000 7f16e1260640 1 -- 172.21.15.40:0/3793692473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16dc15d010 msgr2=0x7f16dc1f18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:15.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.012+0000 7f16e1260640 1 --2- 172.21.15.40:0/3793692473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16dc15d010 0x7f16dc1f18c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f16c400d5b0 tx=0x7f16c400da70 comp rx=0 tx=0).stop 2023-10-18T08:56:15.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.012+0000 7f16e1260640 1 -- 172.21.15.40:0/3793692473 shutdown_connections 2023-10-18T08:56:15.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.012+0000 7f16e1260640 1 --2- 172.21.15.40:0/3793692473 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f16bc06ae90 0x7f16bc06d350 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:15.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.012+0000 7f16e1260640 1 --2- 172.21.15.40:0/3793692473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16dc15e7f0 0x7f16dc0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:15.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.012+0000 7f16e1260640 1 --2- 172.21.15.40:0/3793692473 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16dc15d010 0x7f16dc1f18c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:15.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.012+0000 7f16e1260640 1 --2- 172.21.15.40:0/3793692473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16c80039c0 0x7f16dc14f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:15.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.012+0000 7f16e1260640 1 -- 172.21.15.40:0/3793692473 >> 172.21.15.40:0/3793692473 conn(0x7f16dc08ad00 msgr2=0x7f16dc06b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:15.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.012+0000 7f16e1260640 1 -- 172.21.15.40:0/3793692473 shutdown_connections 2023-10-18T08:56:15.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.012+0000 7f16e1260640 1 -- 172.21.15.40:0/3793692473 wait complete. 2023-10-18T08:56:15.034 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-18T08:56:15.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:15.035 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-18T08:56:15.142 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T08:56:15.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:15.144 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-18T08:56:15.192 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T08:56:15.193 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:56:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.433+0000 7f9d70851640 1 -- 172.21.15.40:0/893314182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d54003970 msgr2=0x7f9d54003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:15.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.433+0000 7f9d70851640 1 --2- 172.21.15.40:0/893314182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d54003970 0x7f9d54003d90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c006fd0 tx=0x7f9d540057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.434+0000 7f9d70851640 1 -- 172.21.15.40:0/893314182 shutdown_connections 2023-10-18T08:56:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.434+0000 7f9d70851640 1 --2- 172.21.15.40:0/893314182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d54003970 0x7f9d54003d90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.434+0000 7f9d70851640 1 -- 172.21.15.40:0/893314182 >> 172.21.15.40:0/893314182 conn(0x7f9d6c06cee0 msgr2=0x7f9d6c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:15.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.434+0000 7f9d70851640 1 -- 172.21.15.40:0/893314182 shutdown_connections 2023-10-18T08:56:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.434+0000 7f9d70851640 1 -- 172.21.15.40:0/893314182 wait complete. 2023-10-18T08:56:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.436+0000 7f9d70851640 1 Processor -- start 2023-10-18T08:56:15.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.436+0000 7f9d70851640 1 -- start start 2023-10-18T08:56:15.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.437+0000 7f9d70851640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d54003970 0x7f9d6c149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:15.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.437+0000 7f9d70851640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d6c150420 0x7f9d6c0abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:15.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.437+0000 7f9d70851640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d6c14b480 0x7f9d6c1f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:15.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.437+0000 7f9d70851640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c14a290 con 0x7f9d6c14b480 2023-10-18T08:56:15.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.437+0000 7f9d70851640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0ac3d0 con 0x7f9d54003970 2023-10-18T08:56:15.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.437+0000 7f9d70851640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c150990 con 0x7f9d6c150420 2023-10-18T08:56:15.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.437+0000 7f9d6ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d6c14b480 0x7f9d6c1f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:15.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.437+0000 7f9d6ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d6c14b480 0x7f9d6c1f1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35314/0 (socket says 172.21.15.40:35314) 2023-10-18T08:56:15.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.437+0000 7f9d6ad9d640 1 -- 172.21.15.40:0/3866097447 learned_addr learned my addr 172.21.15.40:0/3866097447 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:15.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.438+0000 7f9d69d9b640 1 --2- 172.21.15.40:0/3866097447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d6c150420 0x7f9d6c0abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:15.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.438+0000 7f9d6ad9d640 1 -- 172.21.15.40:0/3866097447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d6c150420 msgr2=0x7f9d6c0abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:15.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.438+0000 7f9d6ad9d640 1 --2- 172.21.15.40:0/3866097447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d6c150420 0x7f9d6c0abe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:15.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.438+0000 7f9d6ad9d640 1 -- 172.21.15.40:0/3866097447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d54003970 msgr2=0x7f9d6c149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:15.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.438+0000 7f9d6a59c640 1 --2- 172.21.15.40:0/3866097447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d54003970 0x7f9d6c149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:15.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.438+0000 7f9d6ad9d640 1 --2- 172.21.15.40:0/3866097447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d54003970 0x7f9d6c149d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:15.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.438+0000 7f9d6ad9d640 1 -- 172.21.15.40:0/3866097447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c0ad150 con 0x7f9d6c14b480 2023-10-18T08:56:15.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.438+0000 7f9d6ad9d640 1 --2- 172.21.15.40:0/3866097447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d6c14b480 0x7f9d6c1f1f80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c06d610 tx=0x7f9d64001c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:15.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.438+0000 7f9d5affd640 1 -- 172.21.15.40:0/3866097447 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64011840 con 0x7f9d6c14b480 2023-10-18T08:56:15.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.439+0000 7f9d5affd640 1 -- 172.21.15.40:0/3866097447 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9d640119a0 con 0x7f9d6c14b480 2023-10-18T08:56:15.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.439+0000 7f9d70851640 1 -- 172.21.15.40:0/3866097447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6c0ad430 con 0x7f9d6c14b480 2023-10-18T08:56:15.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.439+0000 7f9d5affd640 1 -- 172.21.15.40:0/3866097447 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d64011db0 con 0x7f9d6c14b480 2023-10-18T08:56:15.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.439+0000 7f9d70851640 1 -- 172.21.15.40:0/3866097447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c0ad9f0 con 0x7f9d6c14b480 2023-10-18T08:56:15.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.440+0000 7f9d70851640 1 -- 172.21.15.40:0/3866097447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6c0adc10 con 0x7f9d6c14b480 2023-10-18T08:56:15.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.441+0000 7f9d5affd640 1 -- 172.21.15.40:0/3866097447 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9d640103c0 con 0x7f9d6c14b480 2023-10-18T08:56:15.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.442+0000 7f9d5affd640 1 --2- 172.21.15.40:0/3866097447 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9d4c06ae70 0x7f9d4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:15.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.442+0000 7f9d5affd640 1 -- 172.21.15.40:0/3866097447 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9d6408a6a0 con 0x7f9d6c14b480 2023-10-18T08:56:15.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.443+0000 7f9d6a59c640 1 --2- 172.21.15.40:0/3866097447 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9d4c06ae70 0x7f9d4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:15.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.444+0000 7f9d6a59c640 1 --2- 172.21.15.40:0/3866097447 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9d4c06ae70 0x7f9d4c06d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c034000 tx=0x7f9d5c036040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:15.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.447+0000 7f9d5affd640 1 -- 172.21.15.40:0/3866097447 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d640556c0 con 0x7f9d6c14b480 2023-10-18T08:56:15.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.678+0000 7f9d70851640 1 -- 172.21.15.40:0/3866097447 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d6c066fd0 con 0x7f9d6c14b480 2023-10-18T08:56:15.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.679+0000 7f9d5affd640 1 -- 172.21.15.40:0/3866097447 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f9d64015090 con 0x7f9d6c14b480 2023-10-18T08:56:15.682 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:15.682 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1962150323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1962150323},{"type":"v1","addr":"172.21.15.144:6837","nonce":1962150323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4329,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2589720838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2589720838},{"type":"v1","addr":"172.21.15.40:6833","nonce":2589720838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3031491371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3031491371},{"type":"v1","addr":"172.21.15.40:6835","nonce":3031491371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:55:14.753828+0000","modified":"2023-10-18T08:55:17.818729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/682820172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":682820172},{"type":"v1","addr":"172.21.15.144:6835","nonce":682820172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T08:56:15.683 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2023-10-18T08:56:15.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.682+0000 7f9d70851640 1 -- 172.21.15.40:0/3866097447 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9d4c06ae70 msgr2=0x7f9d4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:15.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.682+0000 7f9d70851640 1 --2- 172.21.15.40:0/3866097447 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9d4c06ae70 0x7f9d4c06d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c034000 tx=0x7f9d5c036040 comp rx=0 tx=0).stop 2023-10-18T08:56:15.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.682+0000 7f9d70851640 1 -- 172.21.15.40:0/3866097447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d6c14b480 msgr2=0x7f9d6c1f1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:15.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.682+0000 7f9d70851640 1 --2- 172.21.15.40:0/3866097447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d6c14b480 0x7f9d6c1f1f80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c06d610 tx=0x7f9d64001c30 comp rx=0 tx=0).stop 2023-10-18T08:56:15.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.682+0000 7f9d70851640 1 -- 172.21.15.40:0/3866097447 shutdown_connections 2023-10-18T08:56:15.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.682+0000 7f9d70851640 1 --2- 172.21.15.40:0/3866097447 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9d4c06ae70 0x7f9d4c06d330 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:15.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.682+0000 7f9d70851640 1 --2- 172.21.15.40:0/3866097447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d6c14b480 0x7f9d6c1f1f80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:15.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.682+0000 7f9d70851640 1 --2- 172.21.15.40:0/3866097447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d6c150420 0x7f9d6c0abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:15.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.682+0000 7f9d70851640 1 --2- 172.21.15.40:0/3866097447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d54003970 0x7f9d6c149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:15.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.682+0000 7f9d70851640 1 -- 172.21.15.40:0/3866097447 >> 172.21.15.40:0/3866097447 conn(0x7f9d6c06cee0 msgr2=0x7f9d6c070930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:15.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.682+0000 7f9d70851640 1 -- 172.21.15.40:0/3866097447 shutdown_connections 2023-10-18T08:56:15.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.682+0000 7f9d70851640 1 -- 172.21.15.40:0/3866097447 wait complete. 2023-10-18T08:56:15.702 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:15.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.932+0000 7f6397e15640 1 -- 172.21.15.40:0/1153432632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f639013c470 msgr2=0x7f639013f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:15.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.932+0000 7f6397e15640 1 --2- 172.21.15.40:0/1153432632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f639013c470 0x7f639013f5a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f638c009830 tx=0x7f638c02bd60 comp rx=0 tx=0).stop 2023-10-18T08:56:15.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.933+0000 7f6397e15640 1 -- 172.21.15.40:0/1153432632 shutdown_connections 2023-10-18T08:56:15.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.933+0000 7f6397e15640 1 --2- 172.21.15.40:0/1153432632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f639013c470 0x7f639013f5a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:15.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.933+0000 7f6397e15640 1 -- 172.21.15.40:0/1153432632 >> 172.21.15.40:0/1153432632 conn(0x7f639006cee0 msgr2=0x7f6390069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:15.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.933+0000 7f6397e15640 1 -- 172.21.15.40:0/1153432632 shutdown_connections 2023-10-18T08:56:15.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.933+0000 7f6397e15640 1 -- 172.21.15.40:0/1153432632 wait complete. 2023-10-18T08:56:15.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.934+0000 7f6397e15640 1 Processor -- start 2023-10-18T08:56:15.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.935+0000 7f6397e15640 1 -- start start 2023-10-18T08:56:15.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.935+0000 7f6397e15640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63900ca210 0x7f639013b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:15.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.935+0000 7f6397e15640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f639015cf30 0x7f6390158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:15.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.935+0000 7f6397e15640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6390158ca0 0x7f63900aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:15.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.935+0000 7f6397e15640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63901516d0 con 0x7f63900ca210 2023-10-18T08:56:15.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.936+0000 7f6397e15640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6390150b00 con 0x7f639015cf30 2023-10-18T08:56:15.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.936+0000 7f6397e15640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639013b880 con 0x7f6390158ca0 2023-10-18T08:56:15.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.936+0000 7f6395bb1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63900ca210 0x7f639013b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:15.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.936+0000 7f6395bb1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63900ca210 0x7f639013b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35328/0 (socket says 172.21.15.40:35328) 2023-10-18T08:56:15.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.936+0000 7f6395bb1640 1 -- 172.21.15.40:0/45987393 learned_addr learned my addr 172.21.15.40:0/45987393 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:15.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.936+0000 7f63953b0640 1 --2- 172.21.15.40:0/45987393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f639015cf30 0x7f6390158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:15.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.936+0000 7f63963b2640 1 --2- 172.21.15.40:0/45987393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6390158ca0 0x7f63900aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:15.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.936+0000 7f6395bb1640 1 -- 172.21.15.40:0/45987393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6390158ca0 msgr2=0x7f63900aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:15.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.936+0000 7f6395bb1640 1 --2- 172.21.15.40:0/45987393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6390158ca0 0x7f63900aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:15.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.936+0000 7f6395bb1640 1 -- 172.21.15.40:0/45987393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f639015cf30 msgr2=0x7f6390158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:15.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.936+0000 7f6395bb1640 1 --2- 172.21.15.40:0/45987393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f639015cf30 0x7f6390158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:15.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.936+0000 7f6395bb1640 1 -- 172.21.15.40:0/45987393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c009490 con 0x7f63900ca210 2023-10-18T08:56:15.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.936+0000 7f63953b0640 1 --2- 172.21.15.40:0/45987393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f639015cf30 0x7f6390158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:15.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.937+0000 7f6395bb1640 1 --2- 172.21.15.40:0/45987393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63900ca210 0x7f639013b340 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f638c000c00 tx=0x7f638c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:15.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.937+0000 7f63827fc640 1 -- 172.21.15.40:0/45987393 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c041070 con 0x7f63900ca210 2023-10-18T08:56:15.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.937+0000 7f63827fc640 1 -- 172.21.15.40:0/45987393 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f638c003de0 con 0x7f63900ca210 2023-10-18T08:56:15.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.937+0000 7f63827fc640 1 -- 172.21.15.40:0/45987393 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c0346a0 con 0x7f63900ca210 2023-10-18T08:56:15.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.937+0000 7f6397e15640 1 -- 172.21.15.40:0/45987393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6390150da0 con 0x7f63900ca210 2023-10-18T08:56:15.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.937+0000 7f6397e15640 1 -- 172.21.15.40:0/45987393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6390151360 con 0x7f63900ca210 2023-10-18T08:56:15.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.939+0000 7f6397e15640 1 -- 172.21.15.40:0/45987393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63901514f0 con 0x7f63900ca210 2023-10-18T08:56:15.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.939+0000 7f63827fc640 1 -- 172.21.15.40:0/45987393 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f638c03b040 con 0x7f63900ca210 2023-10-18T08:56:15.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.940+0000 7f63827fc640 1 --2- 172.21.15.40:0/45987393 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f637406ae40 0x7f637406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:15.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.940+0000 7f63827fc640 1 -- 172.21.15.40:0/45987393 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f638c0ab350 con 0x7f63900ca210 2023-10-18T08:56:15.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.941+0000 7f63953b0640 1 --2- 172.21.15.40:0/45987393 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f637406ae40 0x7f637406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:15.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.942+0000 7f63953b0640 1 --2- 172.21.15.40:0/45987393 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f637406ae40 0x7f637406d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f63901f5380 tx=0x7f6384009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:15.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:15.945+0000 7f63827fc640 1 -- 172.21.15.40:0/45987393 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f638c076410 con 0x7f63900ca210 2023-10-18T08:56:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.138+0000 7f6397e15640 1 -- 172.21.15.40:0/45987393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6390066fd0 con 0x7f63900ca210 2023-10-18T08:56:16.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.140+0000 7f63827fc640 1 -- 172.21.15.40:0/45987393 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16766 (secure 0 0 0) 0x7f638c079a30 con 0x7f63900ca210 2023-10-18T08:56:16.143 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:16.143 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:02.868692+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:16.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.143+0000 7f6397e15640 1 -- 172.21.15.40:0/45987393 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f637406ae40 msgr2=0x7f637406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:16.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.143+0000 7f6397e15640 1 --2- 172.21.15.40:0/45987393 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f637406ae40 0x7f637406d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f63901f5380 tx=0x7f6384009040 comp rx=0 tx=0).stop 2023-10-18T08:56:16.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.143+0000 7f6397e15640 1 -- 172.21.15.40:0/45987393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63900ca210 msgr2=0x7f639013b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:16.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.143+0000 7f6397e15640 1 --2- 172.21.15.40:0/45987393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63900ca210 0x7f639013b340 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f638c000c00 tx=0x7f638c0039e0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.143+0000 7f6397e15640 1 -- 172.21.15.40:0/45987393 shutdown_connections 2023-10-18T08:56:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.143+0000 7f6397e15640 1 --2- 172.21.15.40:0/45987393 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f637406ae40 0x7f637406d300 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.143+0000 7f6397e15640 1 --2- 172.21.15.40:0/45987393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6390158ca0 0x7f63900aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.143+0000 7f6397e15640 1 --2- 172.21.15.40:0/45987393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f639015cf30 0x7f6390158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.143+0000 7f6397e15640 1 --2- 172.21.15.40:0/45987393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63900ca210 0x7f639013b340 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.143+0000 7f6397e15640 1 -- 172.21.15.40:0/45987393 >> 172.21.15.40:0/45987393 conn(0x7f639006cee0 msgr2=0x7f639013da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:16.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.144+0000 7f6397e15640 1 -- 172.21.15.40:0/45987393 shutdown_connections 2023-10-18T08:56:16.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.144+0000 7f6397e15640 1 -- 172.21.15.40:0/45987393 wait complete. 2023-10-18T08:56:16.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:55:13.210788+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:55:13.772849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:56:16.168 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:56:16.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.406+0000 7fa0a5647640 1 -- 172.21.15.40:0/753166811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0a00b1900 msgr2=0x7fa0a006fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:16.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.406+0000 7fa0a5647640 1 --2- 172.21.15.40:0/753166811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0a00b1900 0x7fa0a006fa50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa098009830 tx=0x7fa09802bd50 comp rx=0 tx=0).stop 2023-10-18T08:56:16.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.406+0000 7fa0a5647640 1 -- 172.21.15.40:0/753166811 shutdown_connections 2023-10-18T08:56:16.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.406+0000 7fa0a5647640 1 --2- 172.21.15.40:0/753166811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0a00b1900 0x7fa0a006fa50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.406+0000 7fa0a5647640 1 --2- 172.21.15.40:0/753166811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0a00c98a0 0x7fa0a006f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.406+0000 7fa0a5647640 1 -- 172.21.15.40:0/753166811 >> 172.21.15.40:0/753166811 conn(0x7fa0a0069720 msgr2=0x7fa0a0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:16.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.406+0000 7fa0a5647640 1 -- 172.21.15.40:0/753166811 shutdown_connections 2023-10-18T08:56:16.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.406+0000 7fa0a5647640 1 -- 172.21.15.40:0/753166811 wait complete. 2023-10-18T08:56:16.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.408+0000 7fa0a5647640 1 Processor -- start 2023-10-18T08:56:16.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.408+0000 7fa0a5647640 1 -- start start 2023-10-18T08:56:16.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.409+0000 7fa0a5647640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0a00b1900 0x7fa0a01579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:16.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.409+0000 7fa0a5647640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0a00c98a0 0x7fa0a0156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:16.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.409+0000 7fa0a5647640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0a01405a0 0x7fa0a0159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:16.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.409+0000 7fa0a5647640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00af4c0 con 0x7fa0a01405a0 2023-10-18T08:56:16.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.409+0000 7fa0a5647640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a00acd40 con 0x7fa0a00b1900 2023-10-18T08:56:16.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.409+0000 7fa0a5647640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a0157f00 con 0x7fa0a00c98a0 2023-10-18T08:56:16.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.409+0000 7fa0a4e46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0a01405a0 0x7fa0a0159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:16.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.409+0000 7fa0a4e46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0a01405a0 0x7fa0a0159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35348/0 (socket says 172.21.15.40:35348) 2023-10-18T08:56:16.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.409+0000 7fa0a4e46640 1 -- 172.21.15.40:0/770149405 learned_addr learned my addr 172.21.15.40:0/770149405 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:16.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.409+0000 7fa09f7fe640 1 --2- 172.21.15.40:0/770149405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0a00c98a0 0x7fa0a0156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:16.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.409+0000 7fa09ffff640 1 --2- 172.21.15.40:0/770149405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0a00b1900 0x7fa0a01579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:16.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.410+0000 7fa0a4e46640 1 -- 172.21.15.40:0/770149405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0a00c98a0 msgr2=0x7fa0a0156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:16.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.410+0000 7fa0a4e46640 1 --2- 172.21.15.40:0/770149405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0a00c98a0 0x7fa0a0156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.410+0000 7fa0a4e46640 1 -- 172.21.15.40:0/770149405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0a00b1900 msgr2=0x7fa0a01579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:16.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.410+0000 7fa0a4e46640 1 --2- 172.21.15.40:0/770149405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0a00b1900 0x7fa0a01579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.410+0000 7fa0a4e46640 1 -- 172.21.15.40:0/770149405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa098009490 con 0x7fa0a01405a0 2023-10-18T08:56:16.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.410+0000 7fa09ffff640 1 --2- 172.21.15.40:0/770149405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0a00b1900 0x7fa0a01579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:16.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.410+0000 7fa0a4e46640 1 --2- 172.21.15.40:0/770149405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0a01405a0 0x7fa0a0159390 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa098009780 tx=0x7fa098003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:16.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.410+0000 7fa09cff9640 1 -- 172.21.15.40:0/770149405 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098005cb0 con 0x7fa0a01405a0 2023-10-18T08:56:16.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.410+0000 7fa09cff9640 1 -- 172.21.15.40:0/770149405 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa098034440 con 0x7fa0a01405a0 2023-10-18T08:56:16.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.410+0000 7fa0a5647640 1 -- 172.21.15.40:0/770149405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a013d350 con 0x7fa0a01405a0 2023-10-18T08:56:16.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.410+0000 7fa09cff9640 1 -- 172.21.15.40:0/770149405 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa098034850 con 0x7fa0a01405a0 2023-10-18T08:56:16.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.410+0000 7fa0a5647640 1 -- 172.21.15.40:0/770149405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a013d910 con 0x7fa0a01405a0 2023-10-18T08:56:16.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.411+0000 7fa0a5647640 1 -- 172.21.15.40:0/770149405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a0066e00 con 0x7fa0a01405a0 2023-10-18T08:56:16.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.412+0000 7fa09cff9640 1 -- 172.21.15.40:0/770149405 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa09803b040 con 0x7fa0a01405a0 2023-10-18T08:56:16.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.414+0000 7fa09cff9640 1 --2- 172.21.15.40:0/770149405 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa08c06ae40 0x7fa08c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:16.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.414+0000 7fa09cff9640 1 -- 172.21.15.40:0/770149405 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa0980abb20 con 0x7fa0a01405a0 2023-10-18T08:56:16.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.415+0000 7fa09ffff640 1 --2- 172.21.15.40:0/770149405 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa08c06ae40 0x7fa08c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:16.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.416+0000 7fa09ffff640 1 --2- 172.21.15.40:0/770149405 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa08c06ae40 0x7fa08c06d300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa090005fd0 tx=0x7fa0900093e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:16.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.418+0000 7fa09cff9640 1 -- 172.21.15.40:0/770149405 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa098076b40 con 0x7fa0a01405a0 2023-10-18T08:56:16.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.609+0000 7fa0a5647640 1 -- 172.21.15.40:0/770149405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa0a00b2ee0 con 0x7fa0a01405a0 2023-10-18T08:56:16.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.611+0000 7fa09cff9640 1 -- 172.21.15.40:0/770149405 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa098039020 con 0x7fa0a01405a0 2023-10-18T08:56:16.613 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:56:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.613+0000 7fa0a5647640 1 -- 172.21.15.40:0/770149405 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa08c06ae40 msgr2=0x7fa08c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.613+0000 7fa0a5647640 1 --2- 172.21.15.40:0/770149405 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa08c06ae40 0x7fa08c06d300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa090005fd0 tx=0x7fa0900093e0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.613+0000 7fa0a5647640 1 -- 172.21.15.40:0/770149405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0a01405a0 msgr2=0x7fa0a0159390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.613+0000 7fa0a5647640 1 --2- 172.21.15.40:0/770149405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0a01405a0 0x7fa0a0159390 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa098009780 tx=0x7fa098003820 comp rx=0 tx=0).stop 2023-10-18T08:56:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.614+0000 7fa0a5647640 1 -- 172.21.15.40:0/770149405 shutdown_connections 2023-10-18T08:56:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.614+0000 7fa0a5647640 1 --2- 172.21.15.40:0/770149405 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa08c06ae40 0x7fa08c06d300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.614+0000 7fa0a5647640 1 --2- 172.21.15.40:0/770149405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0a01405a0 0x7fa0a0159390 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.614+0000 7fa0a5647640 1 --2- 172.21.15.40:0/770149405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0a00c98a0 0x7fa0a0156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.614+0000 7fa0a5647640 1 --2- 172.21.15.40:0/770149405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0a00b1900 0x7fa0a01579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.614+0000 7fa0a5647640 1 -- 172.21.15.40:0/770149405 >> 172.21.15.40:0/770149405 conn(0x7fa0a0069720 msgr2=0x7fa0a00aba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.614+0000 7fa0a5647640 1 -- 172.21.15.40:0/770149405 shutdown_connections 2023-10-18T08:56:16.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.614+0000 7fa0a5647640 1 -- 172.21.15.40:0/770149405 wait complete. 2023-10-18T08:56:16.635 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:56:16.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.870+0000 7f5a5684e640 1 -- 172.21.15.40:0/682375583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a3c0039a0 msgr2=0x7f5a3c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.870+0000 7f5a5684e640 1 --2- 172.21.15.40:0/682375583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a3c0039a0 0x7f5a3c003dc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00c640 tx=0x7f5a3c0057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.870+0000 7f5a5604d640 1 -- 172.21.15.40:0/682375583 reap_dead start 2023-10-18T08:56:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.870+0000 7f5a5684e640 1 -- 172.21.15.40:0/682375583 shutdown_connections 2023-10-18T08:56:16.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.870+0000 7f5a5684e640 1 -- 172.21.15.40:0/682375583 >> 172.21.15.40:0/682375583 conn(0x7f5a5009a580 msgr2=0x7f5a500d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:16.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.870+0000 7f5a5684e640 1 -- 172.21.15.40:0/682375583 shutdown_connections 2023-10-18T08:56:16.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.870+0000 7f5a5684e640 1 -- 172.21.15.40:0/682375583 wait complete. 2023-10-18T08:56:16.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.872+0000 7f5a5684e640 1 Processor -- start 2023-10-18T08:56:16.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.872+0000 7f5a5684e640 1 -- start start 2023-10-18T08:56:16.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.873+0000 7f5a5684e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a3c0039a0 0x7f5a5015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:16.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.873+0000 7f5a5684e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a5015f1e0 0x7f5a500b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:16.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.873+0000 7f5a5684e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a50161400 0x7f5a50161880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:16.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.873+0000 7f5a5684e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5015a830 con 0x7f5a5015f1e0 2023-10-18T08:56:16.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.873+0000 7f5a5684e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5015f710 con 0x7f5a3c0039a0 2023-10-18T08:56:16.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.873+0000 7f5a5684e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5015f9f0 con 0x7f5a50161400 2023-10-18T08:56:16.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.873+0000 7f5a5504b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a5015f1e0 0x7f5a500b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:16.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.873+0000 7f5a5504b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a5015f1e0 0x7f5a500b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35358/0 (socket says 172.21.15.40:35358) 2023-10-18T08:56:16.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.873+0000 7f5a5504b640 1 -- 172.21.15.40:0/3649994523 learned_addr learned my addr 172.21.15.40:0/3649994523 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:16.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.874+0000 7f5a5584c640 1 --2- 172.21.15.40:0/3649994523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a3c0039a0 0x7f5a5015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:16.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.874+0000 7f5a5604d640 1 --2- 172.21.15.40:0/3649994523 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a50161400 0x7f5a50161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:16.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.874+0000 7f5a5504b640 1 -- 172.21.15.40:0/3649994523 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a50161400 msgr2=0x7f5a50161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:16.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.874+0000 7f5a5504b640 1 --2- 172.21.15.40:0/3649994523 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a50161400 0x7f5a50161880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.874+0000 7f5a5504b640 1 -- 172.21.15.40:0/3649994523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a3c0039a0 msgr2=0x7f5a5015a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:16.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.874+0000 7f5a5504b640 1 --2- 172.21.15.40:0/3649994523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a3c0039a0 0x7f5a5015a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:16.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.874+0000 7f5a5504b640 1 -- 172.21.15.40:0/3649994523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5015fcd0 con 0x7f5a5015f1e0 2023-10-18T08:56:16.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.874+0000 7f5a5504b640 1 --2- 172.21.15.40:0/3649994523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a5015f1e0 0x7f5a500b62b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000fa00 tx=0x7f5a4000fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:16.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.875+0000 7f5a467fc640 1 -- 172.21.15.40:0/3649994523 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4000bcb0 con 0x7f5a5015f1e0 2023-10-18T08:56:16.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.875+0000 7f5a467fc640 1 -- 172.21.15.40:0/3649994523 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5a40011440 con 0x7f5a5015f1e0 2023-10-18T08:56:16.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.875+0000 7f5a467fc640 1 -- 172.21.15.40:0/3649994523 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40011830 con 0x7f5a5015f1e0 2023-10-18T08:56:16.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.875+0000 7f5a5684e640 1 -- 172.21.15.40:0/3649994523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a500c5720 con 0x7f5a5015f1e0 2023-10-18T08:56:16.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.875+0000 7f5a5684e640 1 -- 172.21.15.40:0/3649994523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a500b67f0 con 0x7f5a5015f1e0 2023-10-18T08:56:16.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.876+0000 7f5a5684e640 1 -- 172.21.15.40:0/3649994523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5015fec0 con 0x7f5a5015f1e0 2023-10-18T08:56:16.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.877+0000 7f5a467fc640 1 -- 172.21.15.40:0/3649994523 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5a40011990 con 0x7f5a5015f1e0 2023-10-18T08:56:16.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.878+0000 7f5a467fc640 1 --2- 172.21.15.40:0/3649994523 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a3406ae40 0x7f5a3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:16.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.878+0000 7f5a467fc640 1 -- 172.21.15.40:0/3649994523 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5a4008a390 con 0x7f5a5015f1e0 2023-10-18T08:56:16.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.879+0000 7f5a5584c640 1 --2- 172.21.15.40:0/3649994523 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a3406ae40 0x7f5a3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:16.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.880+0000 7f5a5584c640 1 --2- 172.21.15.40:0/3649994523 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a3406ae40 0x7f5a3406d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c034000 tx=0x7f5a4c02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:16.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:16.882+0000 7f5a467fc640 1 -- 172.21.15.40:0/3649994523 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a40055320 con 0x7f5a5015f1e0 2023-10-18T08:56:17.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.072+0000 7f5a5684e640 1 -- 172.21.15.40:0/3649994523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a50073570 con 0x7f5a5015f1e0 2023-10-18T08:56:17.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.073+0000 7f5a467fc640 1 -- 172.21.15.40:0/3649994523 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a40058940 con 0x7f5a5015f1e0 2023-10-18T08:56:17.075 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:56:17.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.075+0000 7f5a5684e640 1 -- 172.21.15.40:0/3649994523 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a3406ae40 msgr2=0x7f5a3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:17.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.075+0000 7f5a5684e640 1 --2- 172.21.15.40:0/3649994523 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a3406ae40 0x7f5a3406d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c034000 tx=0x7f5a4c02e000 comp rx=0 tx=0).stop 2023-10-18T08:56:17.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.075+0000 7f5a5684e640 1 -- 172.21.15.40:0/3649994523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a5015f1e0 msgr2=0x7f5a500b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:17.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.075+0000 7f5a5684e640 1 --2- 172.21.15.40:0/3649994523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a5015f1e0 0x7f5a500b62b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000fa00 tx=0x7f5a4000fec0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.076+0000 7f5a5684e640 1 -- 172.21.15.40:0/3649994523 shutdown_connections 2023-10-18T08:56:17.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.076+0000 7f5a5684e640 1 --2- 172.21.15.40:0/3649994523 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a3406ae40 0x7f5a3406d300 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.076+0000 7f5a5684e640 1 --2- 172.21.15.40:0/3649994523 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a50161400 0x7f5a50161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.076+0000 7f5a5684e640 1 --2- 172.21.15.40:0/3649994523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a5015f1e0 0x7f5a500b62b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.076+0000 7f5a5684e640 1 --2- 172.21.15.40:0/3649994523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a3c0039a0 0x7f5a5015a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.076+0000 7f5a5684e640 1 -- 172.21.15.40:0/3649994523 >> 172.21.15.40:0/3649994523 conn(0x7f5a5009a580 msgr2=0x7f5a5006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:17.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.076+0000 7f5a5684e640 1 -- 172.21.15.40:0/3649994523 shutdown_connections 2023-10-18T08:56:17.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.076+0000 7f5a5684e640 1 -- 172.21.15.40:0/3649994523 wait complete. 2023-10-18T08:56:17.095 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:56:17.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.318+0000 7fece5d28640 1 -- 172.21.15.40:0/885058381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece00d19b0 msgr2=0x7fece00ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:17.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.318+0000 7fece5d28640 1 --2- 172.21.15.40:0/885058381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece00d19b0 0x7fece00ccbf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fece0069c40 tx=0x7fecd802bdb0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.318+0000 7fece5d28640 1 -- 172.21.15.40:0/885058381 shutdown_connections 2023-10-18T08:56:17.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.318+0000 7fece5d28640 1 --2- 172.21.15.40:0/885058381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece00d19b0 0x7fece00ccbf0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.318+0000 7fece5d28640 1 --2- 172.21.15.40:0/885058381 >> v2:172.21.15.144:3300/0 conn(0x7fece015eea0 0x7fece00cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.318+0000 7fece5d28640 1 -- 172.21.15.40:0/885058381 >> 172.21.15.40:0/885058381 conn(0x7fece009a610 msgr2=0x7fece00d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:17.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.318+0000 7fece5d28640 1 -- 172.21.15.40:0/885058381 shutdown_connections 2023-10-18T08:56:17.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.318+0000 7fece5d28640 1 -- 172.21.15.40:0/885058381 wait complete. 2023-10-18T08:56:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.320+0000 7fece5d28640 1 Processor -- start 2023-10-18T08:56:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.320+0000 7fece5d28640 1 -- start start 2023-10-18T08:56:17.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.321+0000 7fece5d28640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece00d19b0 0x7fece00c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.321+0000 7fece5d28640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fece015eac0 0x7fece00c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.321+0000 7fece5d28640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece015eea0 0x7fece006b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.321+0000 7fece5d28640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece00ca310 con 0x7fece015eea0 2023-10-18T08:56:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.321+0000 7fece5d28640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece00c6a70 con 0x7fece015eac0 2023-10-18T08:56:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.321+0000 7fece5d28640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece00c6d50 con 0x7fece00d19b0 2023-10-18T08:56:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.321+0000 7fece5527640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece015eea0 0x7fece006b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.322+0000 7fece5527640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece015eea0 0x7fece006b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35378/0 (socket says 172.21.15.40:35378) 2023-10-18T08:56:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.322+0000 7fece5527640 1 -- 172.21.15.40:0/3346735084 learned_addr learned my addr 172.21.15.40:0/3346735084 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.322+0000 7fece5527640 1 -- 172.21.15.40:0/3346735084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece00d19b0 msgr2=0x7fece00c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.322+0000 7fece5527640 1 --2- 172.21.15.40:0/3346735084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece00d19b0 0x7fece00c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.322+0000 7fece5527640 1 -- 172.21.15.40:0/3346735084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fece015eac0 msgr2=0x7fece00c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.322+0000 7fece5527640 1 --2- 172.21.15.40:0/3346735084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fece015eac0 0x7fece00c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.322+0000 7fece5527640 1 -- 172.21.15.40:0/3346735084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd8009560 con 0x7fece015eea0 2023-10-18T08:56:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.322+0000 7fece5527640 1 --2- 172.21.15.40:0/3346735084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece015eea0 0x7fece006b0f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fecd8036040 tx=0x7fecd802dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.322+0000 7fecd57fa640 1 -- 172.21.15.40:0/3346735084 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd8003800 con 0x7fece015eea0 2023-10-18T08:56:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.323+0000 7fecd57fa640 1 -- 172.21.15.40:0/3346735084 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fecd8003a50 con 0x7fece015eea0 2023-10-18T08:56:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.323+0000 7fecd57fa640 1 -- 172.21.15.40:0/3346735084 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd8003e60 con 0x7fece015eea0 2023-10-18T08:56:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.323+0000 7fece5d28640 1 -- 172.21.15.40:0/3346735084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece00c7030 con 0x7fece015eea0 2023-10-18T08:56:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.323+0000 7fece5d28640 1 -- 172.21.15.40:0/3346735084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece00c75f0 con 0x7fece015eea0 2023-10-18T08:56:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.324+0000 7fece5d28640 1 -- 172.21.15.40:0/3346735084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece0071da0 con 0x7fece015eea0 2023-10-18T08:56:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.325+0000 7fecd57fa640 1 -- 172.21.15.40:0/3346735084 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fecd803e450 con 0x7fece015eea0 2023-10-18T08:56:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.326+0000 7fecd57fa640 1 --2- 172.21.15.40:0/3346735084 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feccc06ae40 0x7feccc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.326+0000 7fecd57fa640 1 -- 172.21.15.40:0/3346735084 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fecd80ac090 con 0x7fece015eea0 2023-10-18T08:56:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.326+0000 7fece4d26640 1 --2- 172.21.15.40:0/3346735084 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feccc06ae40 0x7feccc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.327+0000 7fece4d26640 1 --2- 172.21.15.40:0/3346735084 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feccc06ae40 0x7feccc06d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fecdc001c60 tx=0x7fecdc007d10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.330+0000 7fecd57fa640 1 -- 172.21.15.40:0/3346735084 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fecd80771d0 con 0x7fece015eea0 2023-10-18T08:56:17.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.547+0000 7fece5d28640 1 -- 172.21.15.40:0/3346735084 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fece0066e00 con 0x7fece015eea0 2023-10-18T08:56:17.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.548+0000 7fecd57fa640 1 -- 172.21.15.40:0/3346735084 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fecd8042020 con 0x7fece015eea0 2023-10-18T08:56:17.551 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:56:17.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.551+0000 7fece5d28640 1 -- 172.21.15.40:0/3346735084 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feccc06ae40 msgr2=0x7feccc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:17.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.551+0000 7fece5d28640 1 --2- 172.21.15.40:0/3346735084 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feccc06ae40 0x7feccc06d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fecdc001c60 tx=0x7fecdc007d10 comp rx=0 tx=0).stop 2023-10-18T08:56:17.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.551+0000 7fece5d28640 1 -- 172.21.15.40:0/3346735084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece015eea0 msgr2=0x7fece006b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:17.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.551+0000 7fece5d28640 1 --2- 172.21.15.40:0/3346735084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece015eea0 0x7fece006b0f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fecd8036040 tx=0x7fecd802dd00 comp rx=0 tx=0).stop 2023-10-18T08:56:17.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.551+0000 7fece5d28640 1 -- 172.21.15.40:0/3346735084 shutdown_connections 2023-10-18T08:56:17.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.551+0000 7fece5d28640 1 --2- 172.21.15.40:0/3346735084 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feccc06ae40 0x7feccc06d300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.551+0000 7fece5d28640 1 --2- 172.21.15.40:0/3346735084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fece015eea0 0x7fece006b0f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.551+0000 7fece5d28640 1 --2- 172.21.15.40:0/3346735084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fece015eac0 0x7fece00c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.551+0000 7fece5d28640 1 --2- 172.21.15.40:0/3346735084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece00d19b0 0x7fece00c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.551+0000 7fece5d28640 1 -- 172.21.15.40:0/3346735084 >> 172.21.15.40:0/3346735084 conn(0x7fece009a610 msgr2=0x7fece0072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:17.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.551+0000 7fece5d28640 1 -- 172.21.15.40:0/3346735084 shutdown_connections 2023-10-18T08:56:17.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.551+0000 7fece5d28640 1 -- 172.21.15.40:0/3346735084 wait complete. 2023-10-18T08:56:17.575 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:56:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.760+0000 7f3619213640 1 -- 172.21.15.40:0/2402469043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36140cdca0 msgr2=0x7f36140ce130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.760+0000 7f3619213640 1 --2- 172.21.15.40:0/2402469043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36140cdca0 0x7f36140ce130 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f36140cba10 tx=0x7f360c02c110 comp rx=0 tx=0).stop 2023-10-18T08:56:17.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.760+0000 7f3619213640 1 -- 172.21.15.40:0/2402469043 shutdown_connections 2023-10-18T08:56:17.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.760+0000 7f3619213640 1 --2- 172.21.15.40:0/2402469043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36140cdca0 0x7f36140ce130 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.760+0000 7f3619213640 1 --2- 172.21.15.40:0/2402469043 >> v2:172.21.15.144:3300/0 conn(0x7f361415b880 0x7f36140cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.760+0000 7f3619213640 1 -- 172.21.15.40:0/2402469043 >> 172.21.15.40:0/2402469043 conn(0x7f361408aca0 msgr2=0x7f361409a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:17.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.760+0000 7f3619213640 1 -- 172.21.15.40:0/2402469043 shutdown_connections 2023-10-18T08:56:17.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.760+0000 7f3619213640 1 -- 172.21.15.40:0/2402469043 wait complete. 2023-10-18T08:56:17.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.761+0000 7f3619213640 1 Processor -- start 2023-10-18T08:56:17.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.762+0000 7f3619213640 1 -- start start 2023-10-18T08:56:17.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.762+0000 7f3619213640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36140cdca0 0x7f36140b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:17.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.762+0000 7f3619213640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f361415b4a0 0x7f361414f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:17.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.762+0000 7f3619213640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f361415bc70 0x7f36141f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:17.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.762+0000 7f3619213640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36140b5f50 con 0x7f361415bc70 2023-10-18T08:56:17.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.763+0000 7f3619213640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36140b6220 con 0x7f36140cdca0 2023-10-18T08:56:17.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.763+0000 7f3619213640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36140c4dc0 con 0x7f361415b4a0 2023-10-18T08:56:17.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.763+0000 7f3613fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36140cdca0 0x7f36140b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:17.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.763+0000 7f3618a12640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f361415bc70 0x7f36141f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:17.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.763+0000 7f3613fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36140cdca0 0x7f36140b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.40:33568/0 (socket says 172.21.15.40:33568) 2023-10-18T08:56:17.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.763+0000 7f3613fff640 1 -- 172.21.15.40:0/70420663 learned_addr learned my addr 172.21.15.40:0/70420663 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:17.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.763+0000 7f360bfff640 1 --2- 172.21.15.40:0/70420663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f361415b4a0 0x7f361414f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:17.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.764+0000 7f3618a12640 1 -- 172.21.15.40:0/70420663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f361415b4a0 msgr2=0x7f361414f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:17.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.764+0000 7f3618a12640 1 --2- 172.21.15.40:0/70420663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f361415b4a0 0x7f361414f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.764+0000 7f3618a12640 1 -- 172.21.15.40:0/70420663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36140cdca0 msgr2=0x7f36140b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:17.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.764+0000 7f3618a12640 1 --2- 172.21.15.40:0/70420663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36140cdca0 0x7f36140b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:17.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.764+0000 7f3618a12640 1 -- 172.21.15.40:0/70420663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c009840 con 0x7f361415bc70 2023-10-18T08:56:17.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.764+0000 7f3618a12640 1 --2- 172.21.15.40:0/70420663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f361415bc70 0x7f36141f5660 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f360c009000 tx=0x7f360c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:17.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.765+0000 7f36117fa640 1 -- 172.21.15.40:0/70420663 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c0047a0 con 0x7f361415bc70 2023-10-18T08:56:17.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.765+0000 7f36117fa640 1 -- 172.21.15.40:0/70420663 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f360c0345a0 con 0x7f361415bc70 2023-10-18T08:56:17.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.765+0000 7f36117fa640 1 -- 172.21.15.40:0/70420663 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c0349b0 con 0x7f361415bc70 2023-10-18T08:56:17.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.765+0000 7f3619213640 1 -- 172.21.15.40:0/70420663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36140c50a0 con 0x7f361415bc70 2023-10-18T08:56:17.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.765+0000 7f3619213640 1 -- 172.21.15.40:0/70420663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36140c5660 con 0x7f361415bc70 2023-10-18T08:56:17.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.766+0000 7f3619213640 1 -- 172.21.15.40:0/70420663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3614071da0 con 0x7f361415bc70 2023-10-18T08:56:17.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.767+0000 7f36117fa640 1 -- 172.21.15.40:0/70420663 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f360c034b10 con 0x7f361415bc70 2023-10-18T08:56:17.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.768+0000 7f36117fa640 1 --2- 172.21.15.40:0/70420663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35f406ae40 0x7f35f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:17.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.768+0000 7f36117fa640 1 -- 172.21.15.40:0/70420663 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f360c0ac930 con 0x7f361415bc70 2023-10-18T08:56:17.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.769+0000 7f3613fff640 1 --2- 172.21.15.40:0/70420663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35f406ae40 0x7f35f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:17.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.770+0000 7f3613fff640 1 --2- 172.21.15.40:0/70420663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35f406ae40 0x7f35f406d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f36040046b0 tx=0x7f3604007f70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:17.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:17.772+0000 7f36117fa640 1 -- 172.21.15.40:0/70420663 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f360c0778c0 con 0x7f361415bc70 2023-10-18T08:56:18.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.003+0000 7f3619213640 1 -- 172.21.15.40:0/70420663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36140667d0 con 0x7f361415bc70 2023-10-18T08:56:18.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.004+0000 7f36117fa640 1 -- 172.21.15.40:0/70420663 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f360c043020 con 0x7f361415bc70 2023-10-18T08:56:18.006 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:18.007 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1962150323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1962150323},{"type":"v1","addr":"172.21.15.144:6837","nonce":1962150323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4329,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2589720838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2589720838},{"type":"v1","addr":"172.21.15.40:6833","nonce":2589720838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3031491371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3031491371},{"type":"v1","addr":"172.21.15.40:6835","nonce":3031491371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:55:14.753828+0000","modified":"2023-10-18T08:55:17.818729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/682820172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":682820172},{"type":"v1","addr":"172.21.15.144:6835","nonce":682820172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T08:56:18.007 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2023-10-18T08:56:18.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.006+0000 7f3619213640 1 -- 172.21.15.40:0/70420663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35f406ae40 msgr2=0x7f35f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:18.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.006+0000 7f3619213640 1 --2- 172.21.15.40:0/70420663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35f406ae40 0x7f35f406d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f36040046b0 tx=0x7f3604007f70 comp rx=0 tx=0).stop 2023-10-18T08:56:18.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.006+0000 7f3619213640 1 -- 172.21.15.40:0/70420663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f361415bc70 msgr2=0x7f36141f5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:18.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.006+0000 7f3619213640 1 --2- 172.21.15.40:0/70420663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f361415bc70 0x7f36141f5660 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f360c009000 tx=0x7f360c0042e0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.006+0000 7f3619213640 1 -- 172.21.15.40:0/70420663 shutdown_connections 2023-10-18T08:56:18.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.006+0000 7f3619213640 1 --2- 172.21.15.40:0/70420663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35f406ae40 0x7f35f406d300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.006+0000 7f3619213640 1 --2- 172.21.15.40:0/70420663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f361415bc70 0x7f36141f5660 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.006+0000 7f3619213640 1 --2- 172.21.15.40:0/70420663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f361415b4a0 0x7f361414f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.006+0000 7f3619213640 1 --2- 172.21.15.40:0/70420663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36140cdca0 0x7f36140b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.007+0000 7f3619213640 1 -- 172.21.15.40:0/70420663 >> 172.21.15.40:0/70420663 conn(0x7f361408aca0 msgr2=0x7f3614151750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:18.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.007+0000 7f3619213640 1 -- 172.21.15.40:0/70420663 shutdown_connections 2023-10-18T08:56:18.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.007+0000 7f3619213640 1 -- 172.21.15.40:0/70420663 wait complete. 2023-10-18T08:56:18.027 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:18.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.249+0000 7f3ddab35640 1 -- 172.21.15.40:0/3404410643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc00047c0 msgr2=0x7f3dc0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.249+0000 7f3ddab35640 1 --2- 172.21.15.40:0/3404410643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc00047c0 0x7f3dc0004c40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4006ef0 tx=0x7f3dc00057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.249+0000 7f3ddab35640 1 -- 172.21.15.40:0/3404410643 shutdown_connections 2023-10-18T08:56:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.249+0000 7f3ddab35640 1 --2- 172.21.15.40:0/3404410643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dc00047c0 0x7f3dc0004c40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.249+0000 7f3ddab35640 1 -- 172.21.15.40:0/3404410643 >> 172.21.15.40:0/3404410643 conn(0x7f3dd4069720 msgr2=0x7f3dd4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.249+0000 7f3ddab35640 1 -- 172.21.15.40:0/3404410643 shutdown_connections 2023-10-18T08:56:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.249+0000 7f3ddab35640 1 -- 172.21.15.40:0/3404410643 wait complete. 2023-10-18T08:56:18.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.250+0000 7f3ddab35640 1 Processor -- start 2023-10-18T08:56:18.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.251+0000 7f3ddab35640 1 -- start start 2023-10-18T08:56:18.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.251+0000 7f3ddab35640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dc00047c0 0x7f3dd41f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:18.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.251+0000 7f3ddab35640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dd4159f50 0x7f3dd41e6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:18.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.251+0000 7f3ddab35640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dd40cc280 0x7f3dd40cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:18.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.252+0000 7f3ddab35640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd40ccc50 con 0x7f3dd40cc280 2023-10-18T08:56:18.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.252+0000 7f3ddab35640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd415acd0 con 0x7f3dc00047c0 2023-10-18T08:56:18.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.252+0000 7f3ddab35640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd41f1df0 con 0x7f3dd4159f50 2023-10-18T08:56:18.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.252+0000 7f3dda334640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dd40cc280 0x7f3dd40cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:18.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.252+0000 7f3dda334640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dd40cc280 0x7f3dd40cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35422/0 (socket says 172.21.15.40:35422) 2023-10-18T08:56:18.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.252+0000 7f3dda334640 1 -- 172.21.15.40:0/2597898444 learned_addr learned my addr 172.21.15.40:0/2597898444 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:18.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.252+0000 7f3dd9b33640 1 --2- 172.21.15.40:0/2597898444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dc00047c0 0x7f3dd41f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:18.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.252+0000 7f3dd9332640 1 --2- 172.21.15.40:0/2597898444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dd4159f50 0x7f3dd41e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:18.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.252+0000 7f3dda334640 1 -- 172.21.15.40:0/2597898444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dd4159f50 msgr2=0x7f3dd41e6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:18.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.252+0000 7f3dda334640 1 --2- 172.21.15.40:0/2597898444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dd4159f50 0x7f3dd41e6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.252+0000 7f3dda334640 1 -- 172.21.15.40:0/2597898444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dc00047c0 msgr2=0x7f3dd41f18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:18.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.252+0000 7f3dda334640 1 --2- 172.21.15.40:0/2597898444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dc00047c0 0x7f3dd41f18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.252+0000 7f3dda334640 1 -- 172.21.15.40:0/2597898444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd415a430 con 0x7f3dd40cc280 2023-10-18T08:56:18.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.252+0000 7f3dd9b33640 1 --2- 172.21.15.40:0/2597898444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dc00047c0 0x7f3dd41f18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:18.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.252+0000 7f3dda334640 1 --2- 172.21.15.40:0/2597898444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dd40cc280 0x7f3dd40cc710 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40cad50 tx=0x7f3dcc00b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:18.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.253+0000 7f3dca7fc640 1 -- 172.21.15.40:0/2597898444 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc004b20 con 0x7f3dd40cc280 2023-10-18T08:56:18.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.253+0000 7f3dca7fc640 1 -- 172.21.15.40:0/2597898444 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3dcc004d90 con 0x7f3dd40cc280 2023-10-18T08:56:18.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.253+0000 7f3dca7fc640 1 -- 172.21.15.40:0/2597898444 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc005280 con 0x7f3dd40cc280 2023-10-18T08:56:18.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.253+0000 7f3ddab35640 1 -- 172.21.15.40:0/2597898444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd415a710 con 0x7f3dd40cc280 2023-10-18T08:56:18.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.253+0000 7f3ddab35640 1 -- 172.21.15.40:0/2597898444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd41e4640 con 0x7f3dd40cc280 2023-10-18T08:56:18.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.254+0000 7f3ddab35640 1 -- 172.21.15.40:0/2597898444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd4066e00 con 0x7f3dd40cc280 2023-10-18T08:56:18.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.255+0000 7f3dca7fc640 1 -- 172.21.15.40:0/2597898444 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3dcc0053e0 con 0x7f3dd40cc280 2023-10-18T08:56:18.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.256+0000 7f3dca7fc640 1 --2- 172.21.15.40:0/2597898444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3db806ae70 0x7f3db806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:18.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.256+0000 7f3dca7fc640 1 -- 172.21.15.40:0/2597898444 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3dcc089ea0 con 0x7f3dd40cc280 2023-10-18T08:56:18.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.256+0000 7f3dd9b33640 1 --2- 172.21.15.40:0/2597898444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3db806ae70 0x7f3db806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:18.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.258+0000 7f3dd9b33640 1 --2- 172.21.15.40:0/2597898444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3db806ae70 0x7f3db806d330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4063540 tx=0x7f3dd0006a70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:18.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.260+0000 7f3dca7fc640 1 -- 172.21.15.40:0/2597898444 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3dcc054e30 con 0x7f3dd40cc280 2023-10-18T08:56:18.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.456+0000 7f3ddab35640 1 -- 172.21.15.40:0/2597898444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3dd4070930 con 0x7f3dd40cc280 2023-10-18T08:56:18.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.458+0000 7f3dca7fc640 1 -- 172.21.15.40:0/2597898444 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16766 (secure 0 0 0) 0x7f3dcc058450 con 0x7f3dd40cc280 2023-10-18T08:56:18.461 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:18.462 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:02.868692+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:18.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.461+0000 7f3ddab35640 1 -- 172.21.15.40:0/2597898444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3db806ae70 msgr2=0x7f3db806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.461+0000 7f3ddab35640 1 --2- 172.21.15.40:0/2597898444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3db806ae70 0x7f3db806d330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4063540 tx=0x7f3dd0006a70 comp rx=0 tx=0).stop 2023-10-18T08:56:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.461+0000 7f3ddab35640 1 -- 172.21.15.40:0/2597898444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dd40cc280 msgr2=0x7f3dd40cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.461+0000 7f3ddab35640 1 --2- 172.21.15.40:0/2597898444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dd40cc280 0x7f3dd40cc710 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40cad50 tx=0x7f3dcc00b360 comp rx=0 tx=0).stop 2023-10-18T08:56:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.461+0000 7f3ddab35640 1 -- 172.21.15.40:0/2597898444 shutdown_connections 2023-10-18T08:56:18.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.461+0000 7f3ddab35640 1 --2- 172.21.15.40:0/2597898444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3db806ae70 0x7f3db806d330 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.461+0000 7f3ddab35640 1 --2- 172.21.15.40:0/2597898444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3dd40cc280 0x7f3dd40cc710 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.461+0000 7f3ddab35640 1 --2- 172.21.15.40:0/2597898444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dd4159f50 0x7f3dd41e6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.461+0000 7f3ddab35640 1 --2- 172.21.15.40:0/2597898444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dc00047c0 0x7f3dd41f18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.461+0000 7f3ddab35640 1 -- 172.21.15.40:0/2597898444 >> 172.21.15.40:0/2597898444 conn(0x7f3dd4069720 msgr2=0x7f3dd40cb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:18.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.461+0000 7f3ddab35640 1 -- 172.21.15.40:0/2597898444 shutdown_connections 2023-10-18T08:56:18.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.461+0000 7f3ddab35640 1 -- 172.21.15.40:0/2597898444 wait complete. 2023-10-18T08:56:18.483 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T08:56:18.484 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:56:18.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.699+0000 7f334b0e1640 1 -- 172.21.15.40:0/2169616832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440cdbb0 msgr2=0x7f334415a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:18.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.699+0000 7f334b0e1640 1 --2- 172.21.15.40:0/2169616832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440cdbb0 0x7f334415a470 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f334406c300 tx=0x7f333802bda0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.699+0000 7f334b0e1640 1 -- 172.21.15.40:0/2169616832 shutdown_connections 2023-10-18T08:56:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.699+0000 7f334b0e1640 1 --2- 172.21.15.40:0/2169616832 >> v2:172.21.15.144:3300/0 conn(0x7f33440cbc20 0x7f33440c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.699+0000 7f334b0e1640 1 --2- 172.21.15.40:0/2169616832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440cdbb0 0x7f334415a470 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.699+0000 7f334b0e1640 1 -- 172.21.15.40:0/2169616832 >> 172.21.15.40:0/2169616832 conn(0x7f334406cee0 msgr2=0x7f3344069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.699+0000 7f334b0e1640 1 -- 172.21.15.40:0/2169616832 shutdown_connections 2023-10-18T08:56:18.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.699+0000 7f334b0e1640 1 -- 172.21.15.40:0/2169616832 wait complete. 2023-10-18T08:56:18.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.701+0000 7f334b0e1640 1 Processor -- start 2023-10-18T08:56:18.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.701+0000 7f334b0e1640 1 -- start start 2023-10-18T08:56:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.702+0000 7f334b0e1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f334406b710 0x7f3344152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.702+0000 7f334b0e1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33440cbc20 0x7f33441564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.702+0000 7f334b0e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440cdbb0 0x7f334414f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.702+0000 7f334b0e1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33440adc10 con 0x7f33440cdbb0 2023-10-18T08:56:18.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.702+0000 7f334b0e1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33440ad8c0 con 0x7f33440cbc20 2023-10-18T08:56:18.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.702+0000 7f334b0e1640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3344153110 con 0x7f334406b710 2023-10-18T08:56:18.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.702+0000 7f334967e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440cdbb0 0x7f334414f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:18.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.702+0000 7f334967e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440cdbb0 0x7f334414f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35448/0 (socket says 172.21.15.40:35448) 2023-10-18T08:56:18.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.702+0000 7f334967e640 1 -- 172.21.15.40:0/2300961592 learned_addr learned my addr 172.21.15.40:0/2300961592 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.702+0000 7f3343fff640 1 --2- 172.21.15.40:0/2300961592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33440cbc20 0x7f33441564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.703+0000 7f3348e7d640 1 --2- 172.21.15.40:0/2300961592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f334406b710 0x7f3344152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.703+0000 7f334967e640 1 -- 172.21.15.40:0/2300961592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f334406b710 msgr2=0x7f3344152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.703+0000 7f334967e640 1 --2- 172.21.15.40:0/2300961592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f334406b710 0x7f3344152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.703+0000 7f334967e640 1 -- 172.21.15.40:0/2300961592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33440cbc20 msgr2=0x7f33441564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.703+0000 7f334967e640 1 --2- 172.21.15.40:0/2300961592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33440cbc20 0x7f33441564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.703+0000 7f334967e640 1 -- 172.21.15.40:0/2300961592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3338009560 con 0x7f33440cdbb0 2023-10-18T08:56:18.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.703+0000 7f3348e7d640 1 --2- 172.21.15.40:0/2300961592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f334406b710 0x7f3344152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:18.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.703+0000 7f3343fff640 1 --2- 172.21.15.40:0/2300961592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33440cbc20 0x7f33441564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:56:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.703+0000 7f334967e640 1 --2- 172.21.15.40:0/2300961592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440cdbb0 0x7f334414f8f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f333c00c780 tx=0x7f333c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.703+0000 7f33417fa640 1 -- 172.21.15.40:0/2300961592 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c004b60 con 0x7f33440cdbb0 2023-10-18T08:56:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.703+0000 7f33417fa640 1 -- 172.21.15.40:0/2300961592 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f333c00ce90 con 0x7f33440cdbb0 2023-10-18T08:56:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.703+0000 7f33417fa640 1 -- 172.21.15.40:0/2300961592 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c00aab0 con 0x7f33440cdbb0 2023-10-18T08:56:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.704+0000 7f334b0e1640 1 -- 172.21.15.40:0/2300961592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33441533f0 con 0x7f33440cdbb0 2023-10-18T08:56:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.704+0000 7f334b0e1640 1 -- 172.21.15.40:0/2300961592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3344156a20 con 0x7f33440cdbb0 2023-10-18T08:56:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.705+0000 7f334b0e1640 1 -- 172.21.15.40:0/2300961592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33440add50 con 0x7f33440cdbb0 2023-10-18T08:56:18.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.706+0000 7f33417fa640 1 -- 172.21.15.40:0/2300961592 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f333c00ac10 con 0x7f33440cdbb0 2023-10-18T08:56:18.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.707+0000 7f33417fa640 1 --2- 172.21.15.40:0/2300961592 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f332806ae70 0x7f332806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:18.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.707+0000 7f33417fa640 1 -- 172.21.15.40:0/2300961592 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f333c089a90 con 0x7f33440cdbb0 2023-10-18T08:56:18.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.707+0000 7f3348e7d640 1 --2- 172.21.15.40:0/2300961592 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f332806ae70 0x7f332806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:18.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.708+0000 7f3348e7d640 1 --2- 172.21.15.40:0/2300961592 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f332806ae70 0x7f332806d330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3338035000 tx=0x7f3338004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:18.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.711+0000 7f33417fa640 1 -- 172.21.15.40:0/2300961592 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33440add50 con 0x7f33440cdbb0 2023-10-18T08:56:18.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.942+0000 7f334b0e1640 1 -- 172.21.15.40:0/2300961592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33440add50 con 0x7f33440cdbb0 2023-10-18T08:56:18.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.943+0000 7f33417fa640 1 -- 172.21.15.40:0/2300961592 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f33440add50 con 0x7f33440cdbb0 2023-10-18T08:56:18.946 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:18.946 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1962150323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1962150323},{"type":"v1","addr":"172.21.15.144:6837","nonce":1962150323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4329,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2589720838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2589720838},{"type":"v1","addr":"172.21.15.40:6833","nonce":2589720838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3031491371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3031491371},{"type":"v1","addr":"172.21.15.40:6835","nonce":3031491371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:55:14.753828+0000","modified":"2023-10-18T08:55:17.818729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/682820172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":682820172},{"type":"v1","addr":"172.21.15.144:6835","nonce":682820172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T08:56:18.946 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2023-10-18T08:56:18.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.946+0000 7f334b0e1640 1 -- 172.21.15.40:0/2300961592 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f332806ae70 msgr2=0x7f332806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:18.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.946+0000 7f334b0e1640 1 --2- 172.21.15.40:0/2300961592 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f332806ae70 0x7f332806d330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3338035000 tx=0x7f3338004730 comp rx=0 tx=0).stop 2023-10-18T08:56:18.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.946+0000 7f334b0e1640 1 -- 172.21.15.40:0/2300961592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440cdbb0 msgr2=0x7f334414f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.946+0000 7f334b0e1640 1 --2- 172.21.15.40:0/2300961592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440cdbb0 0x7f334414f8f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f333c00c780 tx=0x7f333c00cc40 comp rx=0 tx=0).stop 2023-10-18T08:56:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.946+0000 7f334b0e1640 1 -- 172.21.15.40:0/2300961592 shutdown_connections 2023-10-18T08:56:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.946+0000 7f334b0e1640 1 --2- 172.21.15.40:0/2300961592 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f332806ae70 0x7f332806d330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.946+0000 7f334b0e1640 1 --2- 172.21.15.40:0/2300961592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33440cdbb0 0x7f334414f8f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.946+0000 7f334b0e1640 1 --2- 172.21.15.40:0/2300961592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33440cbc20 0x7f33441564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.946+0000 7f334b0e1640 1 --2- 172.21.15.40:0/2300961592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f334406b710 0x7f3344152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.946+0000 7f334b0e1640 1 -- 172.21.15.40:0/2300961592 >> 172.21.15.40:0/2300961592 conn(0x7f334406cee0 msgr2=0x7f334415a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.946+0000 7f334b0e1640 1 -- 172.21.15.40:0/2300961592 shutdown_connections 2023-10-18T08:56:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:18.946+0000 7f334b0e1640 1 -- 172.21.15.40:0/2300961592 wait complete. 2023-10-18T08:56:18.968 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:56:19.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.201+0000 7f93d4a60640 1 -- 172.21.15.40:0/4041085661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93bc0047c0 msgr2=0x7f93bc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:19.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.201+0000 7f93d4a60640 1 --2- 172.21.15.40:0/4041085661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93bc0047c0 0x7f93bc004c40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009b50 tx=0x7f93bc0057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.201+0000 7f93cffff640 1 -- 172.21.15.40:0/4041085661 reap_dead start 2023-10-18T08:56:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.201+0000 7f93d4a60640 1 -- 172.21.15.40:0/4041085661 shutdown_connections 2023-10-18T08:56:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.201+0000 7f93d4a60640 1 -- 172.21.15.40:0/4041085661 >> 172.21.15.40:0/4041085661 conn(0x7f93d0069720 msgr2=0x7f93d0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.201+0000 7f93d4a60640 1 -- 172.21.15.40:0/4041085661 shutdown_connections 2023-10-18T08:56:19.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.201+0000 7f93d4a60640 1 -- 172.21.15.40:0/4041085661 wait complete. 2023-10-18T08:56:19.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.203+0000 7f93d4a60640 1 Processor -- start 2023-10-18T08:56:19.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.203+0000 7f93d4a60640 1 -- start start 2023-10-18T08:56:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.204+0000 7f93d4a60640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93bc0047c0 0x7f93d00cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.204+0000 7f93d4a60640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d01401d0 0x7f93d0159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.204+0000 7f93d4a60640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d0141350 0x7f93d01417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:19.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.204+0000 7f93d4a60640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d0141d20 con 0x7f93bc0047c0 2023-10-18T08:56:19.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.204+0000 7f93d4a60640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d0159a80 con 0x7f93d01401d0 2023-10-18T08:56:19.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.204+0000 7f93d4a60640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d0159d60 con 0x7f93d0141350 2023-10-18T08:56:19.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.204+0000 7f93cf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93bc0047c0 0x7f93d00cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:19.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.204+0000 7f93cf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93bc0047c0 0x7f93d00cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35472/0 (socket says 172.21.15.40:35472) 2023-10-18T08:56:19.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.204+0000 7f93cf7fe640 1 -- 172.21.15.40:0/1735056560 learned_addr learned my addr 172.21.15.40:0/1735056560 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:19.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.204+0000 7f93c7fff640 1 --2- 172.21.15.40:0/1735056560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d01401d0 0x7f93d0159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:19.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.204+0000 7f93cf7fe640 1 -- 172.21.15.40:0/1735056560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d0141350 msgr2=0x7f93d01417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:19.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.204+0000 7f93cf7fe640 1 --2- 172.21.15.40:0/1735056560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d0141350 0x7f93d01417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:19.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.205+0000 7f93cf7fe640 1 -- 172.21.15.40:0/1735056560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d01401d0 msgr2=0x7f93d0159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:19.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.205+0000 7f93cf7fe640 1 --2- 172.21.15.40:0/1735056560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d01401d0 0x7f93d0159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:19.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.205+0000 7f93cf7fe640 1 -- 172.21.15.40:0/1735056560 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d015a040 con 0x7f93bc0047c0 2023-10-18T08:56:19.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.205+0000 7f93c7fff640 1 --2- 172.21.15.40:0/1735056560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d01401d0 0x7f93d0159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:19.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.205+0000 7f93cf7fe640 1 --2- 172.21.15.40:0/1735056560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93bc0047c0 0x7f93d00cbdf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f93c0011a50 tx=0x7f93c0011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:19.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.205+0000 7f93ccff9640 1 -- 172.21.15.40:0/1735056560 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c000fd90 con 0x7f93bc0047c0 2023-10-18T08:56:19.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.205+0000 7f93ccff9640 1 -- 172.21.15.40:0/1735056560 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f93c0012450 con 0x7f93bc0047c0 2023-10-18T08:56:19.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.205+0000 7f93ccff9640 1 -- 172.21.15.40:0/1735056560 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c0012800 con 0x7f93bc0047c0 2023-10-18T08:56:19.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.205+0000 7f93d4a60640 1 -- 172.21.15.40:0/1735056560 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d015a320 con 0x7f93bc0047c0 2023-10-18T08:56:19.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.205+0000 7f93d4a60640 1 -- 172.21.15.40:0/1735056560 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d0159790 con 0x7f93bc0047c0 2023-10-18T08:56:19.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.207+0000 7f93d4a60640 1 -- 172.21.15.40:0/1735056560 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d0066e00 con 0x7f93bc0047c0 2023-10-18T08:56:19.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.207+0000 7f93ccff9640 1 -- 172.21.15.40:0/1735056560 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f93c0012960 con 0x7f93bc0047c0 2023-10-18T08:56:19.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.208+0000 7f93ccff9640 1 --2- 172.21.15.40:0/1735056560 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f93b006ae40 0x7f93b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:19.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.208+0000 7f93ccff9640 1 -- 172.21.15.40:0/1735056560 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f93c008a650 con 0x7f93bc0047c0 2023-10-18T08:56:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.209+0000 7f93c7fff640 1 --2- 172.21.15.40:0/1735056560 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f93b006ae40 0x7f93b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.210+0000 7f93c7fff640 1 --2- 172.21.15.40:0/1735056560 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f93b006ae40 0x7f93b006d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93b8036000 tx=0x7f93b8035040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.213+0000 7f93ccff9640 1 -- 172.21.15.40:0/1735056560 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93c0055690 con 0x7f93bc0047c0 2023-10-18T08:56:19.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.453+0000 7f93d4a60640 1 -- 172.21.15.40:0/1735056560 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93d00caf90 con 0x7f93bc0047c0 2023-10-18T08:56:19.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.454+0000 7f93ccff9640 1 -- 172.21.15.40:0/1735056560 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f93c0058cb0 con 0x7f93bc0047c0 2023-10-18T08:56:19.457 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:19.457 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4310,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1962150323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1962150323},{"type":"v1","addr":"172.21.15.144:6837","nonce":1962150323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4329,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2589720838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2589720838},{"type":"v1","addr":"172.21.15.40:6833","nonce":2589720838}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3031491371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3031491371},{"type":"v1","addr":"172.21.15.40:6835","nonce":3031491371}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:55:14.753828+0000","modified":"2023-10-18T08:55:17.818729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/682820172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":682820172},{"type":"v1","addr":"172.21.15.144:6835","nonce":682820172}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T08:56:19.457 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2023-10-18T08:56:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.457+0000 7f93d4a60640 1 -- 172.21.15.40:0/1735056560 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f93b006ae40 msgr2=0x7f93b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.457+0000 7f93d4a60640 1 --2- 172.21.15.40:0/1735056560 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f93b006ae40 0x7f93b006d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93b8036000 tx=0x7f93b8035040 comp rx=0 tx=0).stop 2023-10-18T08:56:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.457+0000 7f93d4a60640 1 -- 172.21.15.40:0/1735056560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93bc0047c0 msgr2=0x7f93d00cbdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.457+0000 7f93d4a60640 1 --2- 172.21.15.40:0/1735056560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93bc0047c0 0x7f93d00cbdf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f93c0011a50 tx=0x7f93c0011f10 comp rx=0 tx=0).stop 2023-10-18T08:56:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.457+0000 7f93d4a60640 1 -- 172.21.15.40:0/1735056560 shutdown_connections 2023-10-18T08:56:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.457+0000 7f93d4a60640 1 --2- 172.21.15.40:0/1735056560 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f93b006ae40 0x7f93b006d300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.457+0000 7f93d4a60640 1 --2- 172.21.15.40:0/1735056560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93d0141350 0x7f93d01417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.457+0000 7f93d4a60640 1 --2- 172.21.15.40:0/1735056560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93d01401d0 0x7f93d0159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.457+0000 7f93d4a60640 1 --2- 172.21.15.40:0/1735056560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93bc0047c0 0x7f93d00cbdf0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.457+0000 7f93d4a60640 1 -- 172.21.15.40:0/1735056560 >> 172.21.15.40:0/1735056560 conn(0x7f93d0069720 msgr2=0x7f93d006f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:19.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.457+0000 7f93d4a60640 1 -- 172.21.15.40:0/1735056560 shutdown_connections 2023-10-18T08:56:19.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.457+0000 7f93d4a60640 1 -- 172.21.15.40:0/1735056560 wait complete. 2023-10-18T08:56:19.479 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:19.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.691+0000 7fb9d6626640 1 -- 172.21.15.40:0/3431768084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b8003970 msgr2=0x7fb9b8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:19.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.691+0000 7fb9d6626640 1 --2- 172.21.15.40:0/3431768084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b8003970 0x7fb9b8003d90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0006f20 tx=0x7fb9b80057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:19.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.691+0000 7fb9d4bc3640 1 -- 172.21.15.40:0/3431768084 reap_dead start 2023-10-18T08:56:19.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.691+0000 7fb9d6626640 1 -- 172.21.15.40:0/3431768084 shutdown_connections 2023-10-18T08:56:19.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.691+0000 7fb9d6626640 1 -- 172.21.15.40:0/3431768084 >> 172.21.15.40:0/3431768084 conn(0x7fb9d006cee0 msgr2=0x7fb9d0069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:19.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.691+0000 7fb9d6626640 1 -- 172.21.15.40:0/3431768084 shutdown_connections 2023-10-18T08:56:19.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.691+0000 7fb9d6626640 1 -- 172.21.15.40:0/3431768084 wait complete. 2023-10-18T08:56:19.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.693+0000 7fb9d6626640 1 Processor -- start 2023-10-18T08:56:19.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.694+0000 7fb9d6626640 1 -- start start 2023-10-18T08:56:19.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.695+0000 7fb9d6626640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9b8003970 0x7fb9d0141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:19.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.695+0000 7fb9d6626640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9d0149240 0x7fb9d0159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:19.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.695+0000 7fb9d6626640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d01596b0 0x7fb9d0142f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:19.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.695+0000 7fb9d6626640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d0142b20 con 0x7fb9d01596b0 2023-10-18T08:56:19.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.695+0000 7fb9d6626640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d0159b30 con 0x7fb9d0149240 2023-10-18T08:56:19.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.695+0000 7fb9d6626640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d01420a0 con 0x7fb9b8003970 2023-10-18T08:56:19.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.695+0000 7fb9d4bc3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d01596b0 0x7fb9d0142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:19.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.695+0000 7fb9d4bc3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d01596b0 0x7fb9d0142f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35498/0 (socket says 172.21.15.40:35498) 2023-10-18T08:56:19.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.695+0000 7fb9d4bc3640 1 -- 172.21.15.40:0/2739802424 learned_addr learned my addr 172.21.15.40:0/2739802424 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:19.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.696+0000 7fb9d4bc3640 1 -- 172.21.15.40:0/2739802424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9b8003970 msgr2=0x7fb9d0141b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:19.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.696+0000 7fb9d4bc3640 1 --2- 172.21.15.40:0/2739802424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9b8003970 0x7fb9d0141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:19.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.696+0000 7fb9d4bc3640 1 -- 172.21.15.40:0/2739802424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9d0149240 msgr2=0x7fb9d0159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:19.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.696+0000 7fb9d4bc3640 1 --2- 172.21.15.40:0/2739802424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9d0149240 0x7fb9d0159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:19.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.696+0000 7fb9d4bc3640 1 -- 172.21.15.40:0/2739802424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d014a490 con 0x7fb9d01596b0 2023-10-18T08:56:19.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.696+0000 7fb9d4bc3640 1 --2- 172.21.15.40:0/2739802424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d01596b0 0x7fb9d0142f40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb9d006a700 tx=0x7fb9c8010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:19.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.697+0000 7fb9ccff9640 1 -- 172.21.15.40:0/2739802424 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c800eed0 con 0x7fb9d01596b0 2023-10-18T08:56:19.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.697+0000 7fb9ccff9640 1 -- 172.21.15.40:0/2739802424 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb9c8003a60 con 0x7fb9d01596b0 2023-10-18T08:56:19.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.697+0000 7fb9ccff9640 1 -- 172.21.15.40:0/2739802424 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8003e70 con 0x7fb9d01596b0 2023-10-18T08:56:19.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.697+0000 7fb9d6626640 1 -- 172.21.15.40:0/2739802424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d014a770 con 0x7fb9d01596b0 2023-10-18T08:56:19.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.697+0000 7fb9d6626640 1 -- 172.21.15.40:0/2739802424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d0142740 con 0x7fb9d01596b0 2023-10-18T08:56:19.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.698+0000 7fb9d6626640 1 -- 172.21.15.40:0/2739802424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d0142900 con 0x7fb9d01596b0 2023-10-18T08:56:19.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.699+0000 7fb9ccff9640 1 -- 172.21.15.40:0/2739802424 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb9c8019030 con 0x7fb9d01596b0 2023-10-18T08:56:19.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.700+0000 7fb9ccff9640 1 --2- 172.21.15.40:0/2739802424 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb9bc06ae40 0x7fb9bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:19.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.700+0000 7fb9ccff9640 1 -- 172.21.15.40:0/2739802424 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb9c8017070 con 0x7fb9d01596b0 2023-10-18T08:56:19.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.700+0000 7fb9cffff640 1 --2- 172.21.15.40:0/2739802424 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb9bc06ae40 0x7fb9bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:19.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.702+0000 7fb9cffff640 1 --2- 172.21.15.40:0/2739802424 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb9bc06ae40 0x7fb9bc06d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0034000 tx=0x7fb9d0157c20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:19.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.704+0000 7fb9ccff9640 1 -- 172.21.15.40:0/2739802424 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb9c80554e0 con 0x7fb9d01596b0 2023-10-18T08:56:19.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.888+0000 7fb9d6626640 1 -- 172.21.15.40:0/2739802424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9d0066fd0 con 0x7fb9d01596b0 2023-10-18T08:56:19.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.890+0000 7fb9ccff9640 1 -- 172.21.15.40:0/2739802424 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16766 (secure 0 0 0) 0x7fb9c8058b00 con 0x7fb9d01596b0 2023-10-18T08:56:19.893 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:19.893 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:02.868692+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:55:13.210788+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T08:55:13.772849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:19.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.892+0000 7fb9d6626640 1 -- 172.21.15.40:0/2739802424 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb9bc06ae40 msgr2=0x7fb9bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:19.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.892+0000 7fb9d6626640 1 --2- 172.21.15.40:0/2739802424 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb9bc06ae40 0x7fb9bc06d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0034000 tx=0x7fb9d0157c20 comp rx=0 tx=0).stop 2023-10-18T08:56:19.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.892+0000 7fb9d6626640 1 -- 172.21.15.40:0/2739802424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d01596b0 msgr2=0x7fb9d0142f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:19.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.892+0000 7fb9d6626640 1 --2- 172.21.15.40:0/2739802424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d01596b0 0x7fb9d0142f40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb9d006a700 tx=0x7fb9c8010f40 comp rx=0 tx=0).stop 2023-10-18T08:56:19.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.892+0000 7fb9d6626640 1 -- 172.21.15.40:0/2739802424 shutdown_connections 2023-10-18T08:56:19.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.892+0000 7fb9d6626640 1 --2- 172.21.15.40:0/2739802424 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb9bc06ae40 0x7fb9bc06d300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:19.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.892+0000 7fb9d6626640 1 --2- 172.21.15.40:0/2739802424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9d01596b0 0x7fb9d0142f40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:19.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.892+0000 7fb9d6626640 1 --2- 172.21.15.40:0/2739802424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9d0149240 0x7fb9d0159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:19.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.892+0000 7fb9d6626640 1 --2- 172.21.15.40:0/2739802424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9b8003970 0x7fb9d0141b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.893+0000 7fb9d6626640 1 -- 172.21.15.40:0/2739802424 >> 172.21.15.40:0/2739802424 conn(0x7fb9d006cee0 msgr2=0x7fb9d014da90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.893+0000 7fb9d6626640 1 -- 172.21.15.40:0/2739802424 shutdown_connections 2023-10-18T08:56:19.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:19.893+0000 7fb9d6626640 1 -- 172.21.15.40:0/2739802424 wait complete. 2023-10-18T08:56:19.915 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T08:56:20.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.142+0000 7f53bb59e640 1 -- 172.21.15.40:0/1445177259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53bc0b2320 msgr2=0x7f53bc06f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:20.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.142+0000 7f53bb59e640 1 --2- 172.21.15.40:0/1445177259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53bc0b2320 0x7f53bc06f600 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f53bc071ab0 tx=0x7f53b402c250 comp rx=0 tx=0).stop 2023-10-18T08:56:20.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.142+0000 7f53bb59e640 1 -- 172.21.15.40:0/1445177259 shutdown_connections 2023-10-18T08:56:20.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.142+0000 7f53bb59e640 1 --2- 172.21.15.40:0/1445177259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53bc0b2320 0x7f53bc06f600 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:20.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.142+0000 7f53bb59e640 1 --2- 172.21.15.40:0/1445177259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53bc0cc190 0x7f53bc0b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:20.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.142+0000 7f53bb59e640 1 --2- 172.21.15.40:0/1445177259 >> v2:172.21.15.144:3300/0 conn(0x7f53bc141c80 0x7f53bc15bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:20.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.142+0000 7f53bb59e640 1 -- 172.21.15.40:0/1445177259 >> 172.21.15.40:0/1445177259 conn(0x7f53bc069720 msgr2=0x7f53bc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:20.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.142+0000 7f53bb59e640 1 -- 172.21.15.40:0/1445177259 shutdown_connections 2023-10-18T08:56:20.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.142+0000 7f53bb59e640 1 -- 172.21.15.40:0/1445177259 wait complete. 2023-10-18T08:56:20.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.144+0000 7f53bb59e640 1 Processor -- start 2023-10-18T08:56:20.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.144+0000 7f53bb59e640 1 -- start start 2023-10-18T08:56:20.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.145+0000 7f53bb59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53bc0cc190 0x7f53bc0af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:20.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.145+0000 7f53bb59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53bc141c80 0x7f53bc15a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:20.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.145+0000 7f53bb59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53bc1f33b0 0x7f53bc136f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:20.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.145+0000 7f53bb59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc159960 con 0x7f53bc0cc190 2023-10-18T08:56:20.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.145+0000 7f53bb59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc1397a0 con 0x7f53bc141c80 2023-10-18T08:56:20.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.145+0000 7f53bb59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc0af9e0 con 0x7f53bc1f33b0 2023-10-18T08:56:20.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.145+0000 7f53ba59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53bc0cc190 0x7f53bc0af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:20.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.145+0000 7f53ba59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53bc0cc190 0x7f53bc0af3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35512/0 (socket says 172.21.15.40:35512) 2023-10-18T08:56:20.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.145+0000 7f53ba59c640 1 -- 172.21.15.40:0/1797777693 learned_addr learned my addr 172.21.15.40:0/1797777693 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:20.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.146+0000 7f53ba59c640 1 -- 172.21.15.40:0/1797777693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53bc1f33b0 msgr2=0x7f53bc136f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:20.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.146+0000 7f53ba59c640 1 --2- 172.21.15.40:0/1797777693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53bc1f33b0 0x7f53bc136f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:20.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.146+0000 7f53ba59c640 1 -- 172.21.15.40:0/1797777693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53bc141c80 msgr2=0x7f53bc15a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:20.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.146+0000 7f53ba59c640 1 --2- 172.21.15.40:0/1797777693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53bc141c80 0x7f53bc15a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:20.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.146+0000 7f53ba59c640 1 -- 172.21.15.40:0/1797777693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b4009900 con 0x7f53bc0cc190 2023-10-18T08:56:20.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.146+0000 7f53ba59c640 1 --2- 172.21.15.40:0/1797777693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53bc0cc190 0x7f53bc0af3d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00b410 tx=0x7f53ac00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:20.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.146+0000 7f53b2ffd640 1 -- 172.21.15.40:0/1797777693 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac011830 con 0x7f53bc0cc190 2023-10-18T08:56:20.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.146+0000 7f53b2ffd640 1 -- 172.21.15.40:0/1797777693 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f53ac011aa0 con 0x7f53bc0cc190 2023-10-18T08:56:20.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.146+0000 7f53b2ffd640 1 -- 172.21.15.40:0/1797777693 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac010380 con 0x7f53bc0cc190 2023-10-18T08:56:20.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.146+0000 7f53bb59e640 1 -- 172.21.15.40:0/1797777693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53bc159bb0 con 0x7f53bc0cc190 2023-10-18T08:56:20.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.146+0000 7f53bb59e640 1 -- 172.21.15.40:0/1797777693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53bc15a170 con 0x7f53bc0cc190 2023-10-18T08:56:20.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.148+0000 7f53bb59e640 1 -- 172.21.15.40:0/1797777693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53bc15a390 con 0x7f53bc0cc190 2023-10-18T08:56:20.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.148+0000 7f53b2ffd640 1 -- 172.21.15.40:0/1797777693 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f53ac0104e0 con 0x7f53bc0cc190 2023-10-18T08:56:20.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.149+0000 7f53b2ffd640 1 --2- 172.21.15.40:0/1797777693 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f539c06ae40 0x7f539c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:20.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.150+0000 7f53b2ffd640 1 -- 172.21.15.40:0/1797777693 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f53ac089a40 con 0x7f53bc0cc190 2023-10-18T08:56:20.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.150+0000 7f53b3fff640 1 --2- 172.21.15.40:0/1797777693 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f539c06ae40 0x7f539c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:20.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.151+0000 7f53b3fff640 1 --2- 172.21.15.40:0/1797777693 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f539c06ae40 0x7f539c06d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0af910 tx=0x7f53a4006c90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:20.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.153+0000 7f53b2ffd640 1 -- 172.21.15.40:0/1797777693 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f53ac054a60 con 0x7f53bc0cc190 2023-10-18T08:56:20.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.386+0000 7f53bb59e640 1 -- 172.21.15.40:0/1797777693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f53bc15c380 con 0x7f53bc0cc190 2023-10-18T08:56:20.390 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T08:56:20.388+0000 7f6a0f710640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T08:56:20.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.392+0000 7f53b2ffd640 1 -- 172.21.15.40:0/1797777693 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f53ac015090 con 0x7f53bc0cc190 2023-10-18T08:56:20.394 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T08:56:20.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.395+0000 7f53bb59e640 1 -- 172.21.15.40:0/1797777693 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f539c06ae40 msgr2=0x7f539c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:20.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.395+0000 7f53bb59e640 1 --2- 172.21.15.40:0/1797777693 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f539c06ae40 0x7f539c06d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0af910 tx=0x7f53a4006c90 comp rx=0 tx=0).stop 2023-10-18T08:56:20.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.395+0000 7f53bb59e640 1 -- 172.21.15.40:0/1797777693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53bc0cc190 msgr2=0x7f53bc0af3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:20.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.395+0000 7f53bb59e640 1 --2- 172.21.15.40:0/1797777693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53bc0cc190 0x7f53bc0af3d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00b410 tx=0x7f53ac00b8d0 comp rx=0 tx=0).stop 2023-10-18T08:56:20.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.395+0000 7f53bb59e640 1 -- 172.21.15.40:0/1797777693 shutdown_connections 2023-10-18T08:56:20.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.395+0000 7f53bb59e640 1 --2- 172.21.15.40:0/1797777693 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f539c06ae40 0x7f539c06d300 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:20.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.395+0000 7f53bb59e640 1 --2- 172.21.15.40:0/1797777693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53bc1f33b0 0x7f53bc136f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:20.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.395+0000 7f53bb59e640 1 --2- 172.21.15.40:0/1797777693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53bc141c80 0x7f53bc15a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:20.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.395+0000 7f53bb59e640 1 --2- 172.21.15.40:0/1797777693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53bc0cc190 0x7f53bc0af3d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:20.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.395+0000 7f53bb59e640 1 -- 172.21.15.40:0/1797777693 >> 172.21.15.40:0/1797777693 conn(0x7f53bc069720 msgr2=0x7f53bc1624b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:20.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.395+0000 7f53bb59e640 1 -- 172.21.15.40:0/1797777693 shutdown_connections 2023-10-18T08:56:20.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.395+0000 7f53bb59e640 1 -- 172.21.15.40:0/1797777693 wait complete. 2023-10-18T08:56:20.415 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T08:56:20.434 INFO:tasks.ceph.mds.b.smithi144.stderr: -7801> 2023-10-18T08:55:16.116+0000 7f00c28bc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T08:56:20.523 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-18T08:56:20.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.639+0000 7fcab6385640 1 -- 172.21.15.40:0/201190447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca9c003f20 msgr2=0x7fca9c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:20.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.639+0000 7fcab6385640 1 --2- 172.21.15.40:0/201190447 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca9c003f20 0x7fca9c0047c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcaac005fd0 tx=0x7fca9c0057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:20.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.639+0000 7fcab5b84640 1 -- 172.21.15.40:0/201190447 reap_dead start 2023-10-18T08:56:20.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.639+0000 7fcab6385640 1 -- 172.21.15.40:0/201190447 shutdown_connections 2023-10-18T08:56:20.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.639+0000 7fcab6385640 1 -- 172.21.15.40:0/201190447 >> 172.21.15.40:0/201190447 conn(0x7fcab009a610 msgr2=0x7fcab00d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:20.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.639+0000 7fcab6385640 1 -- 172.21.15.40:0/201190447 shutdown_connections 2023-10-18T08:56:20.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.639+0000 7fcab6385640 1 -- 172.21.15.40:0/201190447 wait complete. 2023-10-18T08:56:20.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.641+0000 7fcab6385640 1 Processor -- start 2023-10-18T08:56:20.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.641+0000 7fcab6385640 1 -- start start 2023-10-18T08:56:20.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.642+0000 7fcab6385640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca9c003f20 0x7fcab00ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:20.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.642+0000 7fcab6385640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcab00c6bb0 0x7fcab00c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:20.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.642+0000 7fcab6385640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab00c9900 0x7fcab00c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:20.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.642+0000 7fcab6385640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab00c7d60 con 0x7fcab00c9900 2023-10-18T08:56:20.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.642+0000 7fcab6385640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab00c7150 con 0x7fca9c003f20 2023-10-18T08:56:20.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.642+0000 7fcab6385640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab00c7430 con 0x7fcab00c6bb0 2023-10-18T08:56:20.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.642+0000 7fcab5b84640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab00c9900 0x7fcab00c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:20.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.642+0000 7fcab5b84640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab00c9900 0x7fcab00c9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35522/0 (socket says 172.21.15.40:35522) 2023-10-18T08:56:20.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.642+0000 7fcab5b84640 1 -- 172.21.15.40:0/1237755833 learned_addr learned my addr 172.21.15.40:0/1237755833 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:20.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.642+0000 7fcab5383640 1 --2- 172.21.15.40:0/1237755833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca9c003f20 0x7fcab00ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:20.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.643+0000 7fcab5b84640 1 -- 172.21.15.40:0/1237755833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcab00c6bb0 msgr2=0x7fcab00c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:20.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.643+0000 7fcab5b84640 1 --2- 172.21.15.40:0/1237755833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcab00c6bb0 0x7fcab00c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:20.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.643+0000 7fcab5b84640 1 -- 172.21.15.40:0/1237755833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca9c003f20 msgr2=0x7fcab00ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:20.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.643+0000 7fcab5b84640 1 --2- 172.21.15.40:0/1237755833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca9c003f20 0x7fcab00ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:20.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.643+0000 7fcab5b84640 1 -- 172.21.15.40:0/1237755833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab00c7710 con 0x7fcab00c9900 2023-10-18T08:56:20.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.643+0000 7fcab5383640 1 --2- 172.21.15.40:0/1237755833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca9c003f20 0x7fcab00ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:20.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.643+0000 7fcab5b84640 1 --2- 172.21.15.40:0/1237755833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab00c9900 0x7fcab00c9d80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcab0089660 tx=0x7fcaa800da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.643+0000 7fcaa1ffb640 1 -- 172.21.15.40:0/1237755833 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8006440 con 0x7fcab00c9900 2023-10-18T08:56:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.643+0000 7fcaa1ffb640 1 -- 172.21.15.40:0/1237755833 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcaa80035f0 con 0x7fcab00c9900 2023-10-18T08:56:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.643+0000 7fcaa1ffb640 1 -- 172.21.15.40:0/1237755833 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8003a20 con 0x7fcab00c9900 2023-10-18T08:56:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.643+0000 7fcab6385640 1 -- 172.21.15.40:0/1237755833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab00c79f0 con 0x7fcab00c9900 2023-10-18T08:56:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.643+0000 7fcab6385640 1 -- 172.21.15.40:0/1237755833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab00caf00 con 0x7fcab00c9900 2023-10-18T08:56:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.644+0000 7fcab6385640 1 -- 172.21.15.40:0/1237755833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab00b3ec0 con 0x7fcab00c9900 2023-10-18T08:56:20.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.645+0000 7fcaa1ffb640 1 -- 172.21.15.40:0/1237755833 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcaa8003b80 con 0x7fcab00c9900 2023-10-18T08:56:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.646+0000 7fcaa1ffb640 1 --2- 172.21.15.40:0/1237755833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fca9406ae70 0x7fca9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.647+0000 7fcaa1ffb640 1 -- 172.21.15.40:0/1237755833 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fcaa8088c80 con 0x7fcab00c9900 2023-10-18T08:56:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.647+0000 7fcab5383640 1 --2- 172.21.15.40:0/1237755833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fca9406ae70 0x7fca9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.648+0000 7fcab5383640 1 --2- 172.21.15.40:0/1237755833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fca9406ae70 0x7fca9406d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40017a0 tx=0x7fcaa4001e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:20.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.651+0000 7fcaa1ffb640 1 -- 172.21.15.40:0/1237755833 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcaa8053bc0 con 0x7fcab00c9900 2023-10-18T08:56:20.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:20.883+0000 7fcab6385640 1 -- 172.21.15.40:0/1237755833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcab0071da0 con 0x7fcab00c9900 2023-10-18T08:56:21.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.395+0000 7fcaa1ffb640 1 -- 172.21.15.40:0/1237755833 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcaa80160a0 con 0x7fcab00c9900 2023-10-18T08:56:21.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.399+0000 7fcab6385640 1 -- 172.21.15.40:0/1237755833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fca9406ae70 msgr2=0x7fca9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:21.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.399+0000 7fcab6385640 1 --2- 172.21.15.40:0/1237755833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fca9406ae70 0x7fca9406d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40017a0 tx=0x7fcaa4001e50 comp rx=0 tx=0).stop 2023-10-18T08:56:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.399+0000 7fcab6385640 1 -- 172.21.15.40:0/1237755833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab00c9900 msgr2=0x7fcab00c9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.399+0000 7fcab6385640 1 --2- 172.21.15.40:0/1237755833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab00c9900 0x7fcab00c9d80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcab0089660 tx=0x7fcaa800da10 comp rx=0 tx=0).stop 2023-10-18T08:56:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.399+0000 7fcab6385640 1 -- 172.21.15.40:0/1237755833 shutdown_connections 2023-10-18T08:56:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.399+0000 7fcab6385640 1 --2- 172.21.15.40:0/1237755833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fca9406ae70 0x7fca9406d330 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.399+0000 7fcab6385640 1 --2- 172.21.15.40:0/1237755833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab00c9900 0x7fcab00c9d80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.399+0000 7fcab6385640 1 --2- 172.21.15.40:0/1237755833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcab00c6bb0 0x7fcab00c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.399+0000 7fcab6385640 1 --2- 172.21.15.40:0/1237755833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca9c003f20 0x7fcab00ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.399+0000 7fcab6385640 1 -- 172.21.15.40:0/1237755833 >> 172.21.15.40:0/1237755833 conn(0x7fcab009a610 msgr2=0x7fcab015a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.399+0000 7fcab6385640 1 -- 172.21.15.40:0/1237755833 shutdown_connections 2023-10-18T08:56:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.399+0000 7fcab6385640 1 -- 172.21.15.40:0/1237755833 wait complete. 2023-10-18T08:56:21.422 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T08:56:21.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.656+0000 7f955ab2c640 1 -- 172.21.15.40:0/2628591272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9554157ce0 msgr2=0x7f955406fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:21.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.656+0000 7f955ab2c640 1 --2- 172.21.15.40:0/2628591272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9554157ce0 0x7f955406fd30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f95540acb40 tx=0x7f954002bd10 comp rx=0 tx=0).stop 2023-10-18T08:56:21.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.657+0000 7f955ab2c640 1 -- 172.21.15.40:0/2628591272 shutdown_connections 2023-10-18T08:56:21.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.657+0000 7f955ab2c640 1 --2- 172.21.15.40:0/2628591272 >> v2:172.21.15.144:3300/0 conn(0x7f9554070d20 0x7f955406b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:21.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.657+0000 7f955ab2c640 1 --2- 172.21.15.40:0/2628591272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9554157ce0 0x7f955406fd30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:21.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.657+0000 7f955ab2c640 1 -- 172.21.15.40:0/2628591272 >> 172.21.15.40:0/2628591272 conn(0x7f95540aec00 msgr2=0x7f95540aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:21.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.657+0000 7f955ab2c640 1 -- 172.21.15.40:0/2628591272 shutdown_connections 2023-10-18T08:56:21.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.657+0000 7f955ab2c640 1 -- 172.21.15.40:0/2628591272 wait complete. 2023-10-18T08:56:21.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.658+0000 7f955ab2c640 1 Processor -- start 2023-10-18T08:56:21.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.659+0000 7f955ab2c640 1 -- start start 2023-10-18T08:56:21.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.660+0000 7f955ab2c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9554070d20 0x7f9554149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:21.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.660+0000 7f955ab2c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9554157900 0x7f95540c40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:21.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.660+0000 7f955ab2c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9554157ce0 0x7f955414ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:21.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.660+0000 7f955ab2c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95540c4620 con 0x7f9554157900 2023-10-18T08:56:21.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.660+0000 7f955ab2c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95540abf00 con 0x7f9554070d20 2023-10-18T08:56:21.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.660+0000 7f955ab2c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95540c6ed0 con 0x7f9554157ce0 2023-10-18T08:56:21.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.660+0000 7f9553fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9554157900 0x7f95540c40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:21.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.660+0000 7f9553fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9554157900 0x7f95540c40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35544/0 (socket says 172.21.15.40:35544) 2023-10-18T08:56:21.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.660+0000 7f9553fff640 1 -- 172.21.15.40:0/2096354077 learned_addr learned my addr 172.21.15.40:0/2096354077 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:21.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.660+0000 7f95588c8640 1 --2- 172.21.15.40:0/2096354077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9554070d20 0x7f9554149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:21.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.660+0000 7f9553fff640 1 -- 172.21.15.40:0/2096354077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9554157ce0 msgr2=0x7f955414ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:21.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.660+0000 7f9553fff640 1 --2- 172.21.15.40:0/2096354077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9554157ce0 0x7f955414ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:21.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.660+0000 7f9553fff640 1 -- 172.21.15.40:0/2096354077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9554070d20 msgr2=0x7f9554149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:21.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.660+0000 7f9553fff640 1 --2- 172.21.15.40:0/2096354077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9554070d20 0x7f9554149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:21.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.660+0000 7f9553fff640 1 -- 172.21.15.40:0/2096354077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9540009560 con 0x7f9554157900 2023-10-18T08:56:21.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.660+0000 7f95588c8640 1 --2- 172.21.15.40:0/2096354077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9554070d20 0x7f9554149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:21.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.661+0000 7f9553fff640 1 --2- 172.21.15.40:0/2096354077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9554157900 0x7f95540c40e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9540036040 tx=0x7f954002dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:21.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.661+0000 7f95517fa640 1 -- 172.21.15.40:0/2096354077 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540003800 con 0x7f9554157900 2023-10-18T08:56:21.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.661+0000 7f95517fa640 1 -- 172.21.15.40:0/2096354077 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9540003a50 con 0x7f9554157900 2023-10-18T08:56:21.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.661+0000 7f95517fa640 1 -- 172.21.15.40:0/2096354077 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540003de0 con 0x7f9554157900 2023-10-18T08:56:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.661+0000 7f955ab2c640 1 -- 172.21.15.40:0/2096354077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95540ab420 con 0x7f9554157900 2023-10-18T08:56:21.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.661+0000 7f955ab2c640 1 -- 172.21.15.40:0/2096354077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95540ab9e0 con 0x7f9554157900 2023-10-18T08:56:21.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.662+0000 7f955ab2c640 1 -- 172.21.15.40:0/2096354077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9554066fd0 con 0x7f9554157900 2023-10-18T08:56:21.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.664+0000 7f95517fa640 1 -- 172.21.15.40:0/2096354077 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f954003e450 con 0x7f9554157900 2023-10-18T08:56:21.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.665+0000 7f95517fa640 1 --2- 172.21.15.40:0/2096354077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f953806ae40 0x7f953806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:21.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.665+0000 7f95517fa640 1 -- 172.21.15.40:0/2096354077 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f95400ac190 con 0x7f9554157900 2023-10-18T08:56:21.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.665+0000 7f95588c8640 1 --2- 172.21.15.40:0/2096354077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f953806ae40 0x7f953806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:21.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.666+0000 7f95588c8640 1 --2- 172.21.15.40:0/2096354077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f953806ae40 0x7f953806d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9548003eb0 tx=0x7f9548009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:21.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.669+0000 7f95517fa640 1 -- 172.21.15.40:0/2096354077 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95400771f0 con 0x7f9554157900 2023-10-18T08:56:21.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:21.864+0000 7f955ab2c640 1 -- 172.21.15.40:0/2096354077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f95540ac160 con 0x7f9554157900 2023-10-18T08:56:22.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.398+0000 7f95517fa640 1 -- 172.21.15.40:0/2096354077 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f95400056b0 con 0x7f9554157900 2023-10-18T08:56:22.401 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-10-18T08:56:22.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.405+0000 7f955ab2c640 1 -- 172.21.15.40:0/2096354077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f953806ae40 msgr2=0x7f953806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:22.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.405+0000 7f955ab2c640 1 --2- 172.21.15.40:0/2096354077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f953806ae40 0x7f953806d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9548003eb0 tx=0x7f9548009040 comp rx=0 tx=0).stop 2023-10-18T08:56:22.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.405+0000 7f955ab2c640 1 -- 172.21.15.40:0/2096354077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9554157900 msgr2=0x7f95540c40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:22.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.405+0000 7f955ab2c640 1 --2- 172.21.15.40:0/2096354077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9554157900 0x7f95540c40e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9540036040 tx=0x7f954002dce0 comp rx=0 tx=0).stop 2023-10-18T08:56:22.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.405+0000 7f955ab2c640 1 -- 172.21.15.40:0/2096354077 shutdown_connections 2023-10-18T08:56:22.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.405+0000 7f955ab2c640 1 --2- 172.21.15.40:0/2096354077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f953806ae40 0x7f953806d300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:22.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.406+0000 7f955ab2c640 1 --2- 172.21.15.40:0/2096354077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9554157ce0 0x7f955414ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:22.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.406+0000 7f955ab2c640 1 --2- 172.21.15.40:0/2096354077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9554157900 0x7f95540c40e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:22.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.406+0000 7f955ab2c640 1 --2- 172.21.15.40:0/2096354077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9554070d20 0x7f9554149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:22.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.406+0000 7f955ab2c640 1 -- 172.21.15.40:0/2096354077 >> 172.21.15.40:0/2096354077 conn(0x7f95540aec00 msgr2=0x7f95540ae3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:22.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.407+0000 7f955ab2c640 1 -- 172.21.15.40:0/2096354077 shutdown_connections 2023-10-18T08:56:22.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.407+0000 7f955ab2c640 1 -- 172.21.15.40:0/2096354077 wait complete. 2023-10-18T08:56:22.433 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T08:56:22.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.669+0000 7f0e70a01640 1 -- 172.21.15.40:0/2431404876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e6c0ad2a0 msgr2=0x7f0e6c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:22.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.669+0000 7f0e70a01640 1 --2- 172.21.15.40:0/2431404876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e6c0ad2a0 0x7f0e6c06c290 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009830 tx=0x7f0e5402bdf0 comp rx=0 tx=0).stop 2023-10-18T08:56:22.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.670+0000 7f0e70a01640 1 -- 172.21.15.40:0/2431404876 shutdown_connections 2023-10-18T08:56:22.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.670+0000 7f0e70a01640 1 --2- 172.21.15.40:0/2431404876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e6c0ad2a0 0x7f0e6c06c290 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:22.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.670+0000 7f0e70a01640 1 -- 172.21.15.40:0/2431404876 >> 172.21.15.40:0/2431404876 conn(0x7f0e6c0ca990 msgr2=0x7f0e6c0cada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:22.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.670+0000 7f0e70a01640 1 -- 172.21.15.40:0/2431404876 shutdown_connections 2023-10-18T08:56:22.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.670+0000 7f0e70a01640 1 -- 172.21.15.40:0/2431404876 wait complete. 2023-10-18T08:56:22.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.672+0000 7f0e70a01640 1 Processor -- start 2023-10-18T08:56:22.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.672+0000 7f0e70a01640 1 -- start start 2023-10-18T08:56:22.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.673+0000 7f0e70a01640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e6c06c7d0 0x7f0e6c152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:22.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.673+0000 7f0e70a01640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e6c0ad2a0 0x7f0e6c154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:22.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.673+0000 7f0e70a01640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e6c15b7a0 0x7f0e6c0ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:22.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.673+0000 7f0e70a01640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c154360 con 0x7f0e6c06c7d0 2023-10-18T08:56:22.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.673+0000 7f0e70a01640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c152d50 con 0x7f0e6c0ad2a0 2023-10-18T08:56:22.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.673+0000 7f0e70a01640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c1534b0 con 0x7f0e6c15b7a0 2023-10-18T08:56:22.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.673+0000 7f0e6ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e6c06c7d0 0x7f0e6c152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:22.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.673+0000 7f0e6ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e6c06c7d0 0x7f0e6c152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35566/0 (socket says 172.21.15.40:35566) 2023-10-18T08:56:22.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.673+0000 7f0e6ad9d640 1 -- 172.21.15.40:0/3016991770 learned_addr learned my addr 172.21.15.40:0/3016991770 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:22.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.673+0000 7f0e6ad9d640 1 -- 172.21.15.40:0/3016991770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e6c15b7a0 msgr2=0x7f0e6c0ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:22.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.673+0000 7f0e6ad9d640 1 --2- 172.21.15.40:0/3016991770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e6c15b7a0 0x7f0e6c0ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:22.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.673+0000 7f0e6ad9d640 1 -- 172.21.15.40:0/3016991770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e6c0ad2a0 msgr2=0x7f0e6c154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:22.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.673+0000 7f0e6ad9d640 1 --2- 172.21.15.40:0/3016991770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e6c0ad2a0 0x7f0e6c154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:22.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.673+0000 7f0e6ad9d640 1 -- 172.21.15.40:0/3016991770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e54009490 con 0x7f0e6c06c7d0 2023-10-18T08:56:22.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.674+0000 7f0e6ad9d640 1 --2- 172.21.15.40:0/3016991770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e6c06c7d0 0x7f0e6c152810 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00c400 tx=0x7f0e5c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:22.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.674+0000 7f0e637fe640 1 -- 172.21.15.40:0/3016991770 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c0052d0 con 0x7f0e6c06c7d0 2023-10-18T08:56:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.674+0000 7f0e637fe640 1 -- 172.21.15.40:0/3016991770 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0e5c0035f0 con 0x7f0e6c06c7d0 2023-10-18T08:56:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.674+0000 7f0e637fe640 1 -- 172.21.15.40:0/3016991770 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c003a00 con 0x7f0e6c06c7d0 2023-10-18T08:56:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.674+0000 7f0e70a01640 1 -- 172.21.15.40:0/3016991770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6c153750 con 0x7f0e6c06c7d0 2023-10-18T08:56:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.674+0000 7f0e70a01640 1 -- 172.21.15.40:0/3016991770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c153d10 con 0x7f0e6c06c7d0 2023-10-18T08:56:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.675+0000 7f0e70a01640 1 -- 172.21.15.40:0/3016991770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6c06fd10 con 0x7f0e6c06c7d0 2023-10-18T08:56:22.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.676+0000 7f0e637fe640 1 -- 172.21.15.40:0/3016991770 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0e5c003b60 con 0x7f0e6c06c7d0 2023-10-18T08:56:22.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.677+0000 7f0e637fe640 1 --2- 172.21.15.40:0/3016991770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0e4c06ae40 0x7f0e4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:22.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.677+0000 7f0e637fe640 1 -- 172.21.15.40:0/3016991770 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f0e5c088dc0 con 0x7f0e6c06c7d0 2023-10-18T08:56:22.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.678+0000 7f0e63fff640 1 --2- 172.21.15.40:0/3016991770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0e4c06ae40 0x7f0e4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:22.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.680+0000 7f0e63fff640 1 --2- 172.21.15.40:0/3016991770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0e4c06ae40 0x7f0e4c06d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e54036000 tx=0x7f0e540046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:22.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.682+0000 7f0e637fe640 1 -- 172.21.15.40:0/3016991770 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0e5c053eb0 con 0x7f0e6c06c7d0 2023-10-18T08:56:22.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:22.880+0000 7f0e70a01640 1 -- 172.21.15.40:0/3016991770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0e6c073570 con 0x7f0e6c06c7d0 2023-10-18T08:56:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.409+0000 7f0e637fe640 1 -- 172.21.15.40:0/3016991770 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0e5c0170d0 con 0x7f0e6c06c7d0 2023-10-18T08:56:23.412 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-10-18T08:56:23.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.414+0000 7f0e70a01640 1 -- 172.21.15.40:0/3016991770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0e4c06ae40 msgr2=0x7f0e4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:23.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.414+0000 7f0e70a01640 1 --2- 172.21.15.40:0/3016991770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0e4c06ae40 0x7f0e4c06d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0e54036000 tx=0x7f0e540046c0 comp rx=0 tx=0).stop 2023-10-18T08:56:23.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.414+0000 7f0e70a01640 1 -- 172.21.15.40:0/3016991770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e6c06c7d0 msgr2=0x7f0e6c152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:23.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.414+0000 7f0e70a01640 1 --2- 172.21.15.40:0/3016991770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e6c06c7d0 0x7f0e6c152810 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00c400 tx=0x7f0e5c00c8c0 comp rx=0 tx=0).stop 2023-10-18T08:56:23.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.414+0000 7f0e70a01640 1 -- 172.21.15.40:0/3016991770 shutdown_connections 2023-10-18T08:56:23.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.415+0000 7f0e70a01640 1 --2- 172.21.15.40:0/3016991770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0e4c06ae40 0x7f0e4c06d300 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:23.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.415+0000 7f0e70a01640 1 --2- 172.21.15.40:0/3016991770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e6c15b7a0 0x7f0e6c0ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.415+0000 7f0e70a01640 1 --2- 172.21.15.40:0/3016991770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e6c0ad2a0 0x7f0e6c154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.415+0000 7f0e70a01640 1 --2- 172.21.15.40:0/3016991770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e6c06c7d0 0x7f0e6c152810 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.415+0000 7f0e70a01640 1 -- 172.21.15.40:0/3016991770 >> 172.21.15.40:0/3016991770 conn(0x7f0e6c0ca990 msgr2=0x7f0e6c06cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.415+0000 7f0e70a01640 1 -- 172.21.15.40:0/3016991770 shutdown_connections 2023-10-18T08:56:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.415+0000 7f0e70a01640 1 -- 172.21.15.40:0/3016991770 wait complete. 2023-10-18T08:56:23.442 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T08:56:23.442 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T08:56:23.443 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T08:56:23.443 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T08:56:23.444 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T08:56:23.444 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T08:56:23.444 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T08:56:23.444 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T08:56:23.445 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T08:56:23.445 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T08:56:23.445 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T08:56:23.445 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T08:56:23.446 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T08:56:23.446 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T08:56:23.447 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T08:56:23.447 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T08:56:23.464 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T08:56:23.464 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T08:56:23.465 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T08:56:23.465 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T08:56:23.467 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T08:56:23.467 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T08:56:23.468 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T08:56:23.469 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T08:56:23.513 INFO:tasks.ceph.mds.c:Started 2023-10-18T08:56:23.513 DEBUG:teuthology.parallel:result is None 2023-10-18T08:56:23.516 INFO:tasks.ceph.mds.d:Started 2023-10-18T08:56:23.516 DEBUG:teuthology.parallel:result is None 2023-10-18T08:56:23.520 INFO:tasks.ceph.mds.a:Started 2023-10-18T08:56:23.520 DEBUG:teuthology.parallel:result is None 2023-10-18T08:56:23.524 INFO:tasks.ceph.mds.b:Started 2023-10-18T08:56:23.524 DEBUG:teuthology.parallel:result is None 2023-10-18T08:56:23.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T08:56:23.675 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T08:56:23.678 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-18T08:56:23.679 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-18T08:56:23.681 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-10-18T08:56:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.720+0000 7ff389af2640 1 -- 172.21.15.40:0/1323605934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff38406d710 msgr2=0x7ff384151bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.720+0000 7ff389af2640 1 --2- 172.21.15.40:0/1323605934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff38406d710 0x7ff384151bd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff374005fd0 tx=0x7ff37402d0d0 comp rx=0 tx=0).stop 2023-10-18T08:56:23.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.721+0000 7ff389af2640 1 -- 172.21.15.40:0/1323605934 shutdown_connections 2023-10-18T08:56:23.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.721+0000 7ff389af2640 1 --2- 172.21.15.40:0/1323605934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3840cd9a0 0x7ff38406be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:23.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.721+0000 7ff389af2640 1 --2- 172.21.15.40:0/1323605934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff38406d710 0x7ff384151bd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:23.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.721+0000 7ff389af2640 1 -- 172.21.15.40:0/1323605934 >> 172.21.15.40:0/1323605934 conn(0x7ff38406cee0 msgr2=0x7ff384069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:23.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.721+0000 7ff389af2640 1 -- 172.21.15.40:0/1323605934 shutdown_connections 2023-10-18T08:56:23.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.721+0000 7ff389af2640 1 -- 172.21.15.40:0/1323605934 wait complete. 2023-10-18T08:56:23.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.722+0000 7ff389af2640 1 Processor -- start 2023-10-18T08:56:23.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.723+0000 7ff389af2640 1 -- start start 2023-10-18T08:56:23.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.724+0000 7ff389af2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff38406d710 0x7ff384149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:23.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.724+0000 7ff389af2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3840cd9a0 0x7ff3841495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:23.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.724+0000 7ff389af2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3841586a0 0x7ff38414fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:23.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.724+0000 7ff389af2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3840cabb0 con 0x7ff3840cd9a0 2023-10-18T08:56:23.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.724+0000 7ff389af2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38414d310 con 0x7ff38406d710 2023-10-18T08:56:23.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.724+0000 7ff389af2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3840c9fc0 con 0x7ff3841586a0 2023-10-18T08:56:23.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.724+0000 7ff3837fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff38406d710 0x7ff384149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:23.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.724+0000 7ff37bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3840cd9a0 0x7ff3841495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:23.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.724+0000 7ff3837fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff38406d710 0x7ff384149e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.40:33782/0 (socket says 172.21.15.40:33782) 2023-10-18T08:56:23.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.724+0000 7ff3837fe640 1 -- 172.21.15.40:0/995836626 learned_addr learned my addr 172.21.15.40:0/995836626 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:23.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.724+0000 7ff383fff640 1 --2- 172.21.15.40:0/995836626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3841586a0 0x7ff38414fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:23.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.724+0000 7ff3837fe640 1 -- 172.21.15.40:0/995836626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3841586a0 msgr2=0x7ff38414fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:23.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.724+0000 7ff3837fe640 1 --2- 172.21.15.40:0/995836626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3841586a0 0x7ff38414fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:23.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.724+0000 7ff3837fe640 1 -- 172.21.15.40:0/995836626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3840cd9a0 msgr2=0x7ff3841495e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:23.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.724+0000 7ff3837fe640 1 --2- 172.21.15.40:0/995836626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3840cd9a0 0x7ff3841495e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:23.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.724+0000 7ff3837fe640 1 -- 172.21.15.40:0/995836626 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37400a670 con 0x7ff38406d710 2023-10-18T08:56:23.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.725+0000 7ff3837fe640 1 --2- 172.21.15.40:0/995836626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff38406d710 0x7ff384149e00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff374037040 tx=0x7ff374004a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:23.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.725+0000 7ff380ff9640 1 -- 172.21.15.40:0/995836626 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3740037f0 con 0x7ff38406d710 2023-10-18T08:56:23.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.725+0000 7ff380ff9640 1 -- 172.21.15.40:0/995836626 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff374003a40 con 0x7ff38406d710 2023-10-18T08:56:23.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.725+0000 7ff380ff9640 1 -- 172.21.15.40:0/995836626 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff374003df0 con 0x7ff38406d710 2023-10-18T08:56:23.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.725+0000 7ff389af2640 1 -- 172.21.15.40:0/995836626 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38414a340 con 0x7ff38406d710 2023-10-18T08:56:23.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.725+0000 7ff389af2640 1 -- 172.21.15.40:0/995836626 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3840ca4f0 con 0x7ff38406d710 2023-10-18T08:56:23.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.726+0000 7ff389af2640 1 -- 172.21.15.40:0/995836626 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff384066fd0 con 0x7ff38406d710 2023-10-18T08:56:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.727+0000 7ff380ff9640 1 -- 172.21.15.40:0/995836626 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff374004e50 con 0x7ff38406d710 2023-10-18T08:56:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.728+0000 7ff380ff9640 1 --2- 172.21.15.40:0/995836626 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff36406ae40 0x7ff36406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.728+0000 7ff380ff9640 1 -- 172.21.15.40:0/995836626 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff3740ac520 con 0x7ff38406d710 2023-10-18T08:56:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.729+0000 7ff37bfff640 1 --2- 172.21.15.40:0/995836626 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff36406ae40 0x7ff36406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:23.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.730+0000 7ff37bfff640 1 --2- 172.21.15.40:0/995836626 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff36406ae40 0x7ff36406d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff3840718a0 tx=0x7ff36c007e30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:23.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.734+0000 7ff380ff9640 1 -- 172.21.15.40:0/995836626 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff374077970 con 0x7ff38406d710 2023-10-18T08:56:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:23.941+0000 7ff389af2640 1 -- 172.21.15.40:0/995836626 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff38406fb50 con 0x7ff38406d710 2023-10-18T08:56:24.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.438+0000 7ff380ff9640 1 -- 172.21.15.40:0/995836626 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff374042080 con 0x7ff38406d710 2023-10-18T08:56:24.441 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2023-10-18T08:56:24.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.443+0000 7ff389af2640 1 -- 172.21.15.40:0/995836626 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff36406ae40 msgr2=0x7ff36406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:24.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.443+0000 7ff389af2640 1 --2- 172.21.15.40:0/995836626 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff36406ae40 0x7ff36406d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff3840718a0 tx=0x7ff36c007e30 comp rx=0 tx=0).stop 2023-10-18T08:56:24.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.443+0000 7ff389af2640 1 -- 172.21.15.40:0/995836626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff38406d710 msgr2=0x7ff384149e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:24.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.443+0000 7ff389af2640 1 --2- 172.21.15.40:0/995836626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff38406d710 0x7ff384149e00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff374037040 tx=0x7ff374004a50 comp rx=0 tx=0).stop 2023-10-18T08:56:24.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.443+0000 7ff389af2640 1 -- 172.21.15.40:0/995836626 shutdown_connections 2023-10-18T08:56:24.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.443+0000 7ff389af2640 1 --2- 172.21.15.40:0/995836626 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff36406ae40 0x7ff36406d300 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:24.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.443+0000 7ff389af2640 1 --2- 172.21.15.40:0/995836626 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3841586a0 0x7ff38414fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:24.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.443+0000 7ff389af2640 1 --2- 172.21.15.40:0/995836626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3840cd9a0 0x7ff3841495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:24.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.443+0000 7ff389af2640 1 --2- 172.21.15.40:0/995836626 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff38406d710 0x7ff384149e00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:24.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.443+0000 7ff389af2640 1 -- 172.21.15.40:0/995836626 >> 172.21.15.40:0/995836626 conn(0x7ff38406cee0 msgr2=0x7ff3840cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:24.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.444+0000 7ff389af2640 1 -- 172.21.15.40:0/995836626 shutdown_connections 2023-10-18T08:56:24.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.444+0000 7ff389af2640 1 -- 172.21.15.40:0/995836626 wait complete. 2023-10-18T08:56:24.468 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T08:56:24.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.699+0000 7f2db6402640 1 -- 172.21.15.40:0/489936365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9c0039c0 msgr2=0x7f2d9c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:24.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.699+0000 7f2db6402640 1 --2- 172.21.15.40:0/489936365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9c0039c0 0x7f2d9c003de0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2dac001680 tx=0x7f2d9c0057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:24.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.699+0000 7f2db6402640 1 -- 172.21.15.40:0/489936365 shutdown_connections 2023-10-18T08:56:24.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.699+0000 7f2db6402640 1 --2- 172.21.15.40:0/489936365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2d9c0039c0 0x7f2d9c003de0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:24.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.699+0000 7f2db6402640 1 -- 172.21.15.40:0/489936365 >> 172.21.15.40:0/489936365 conn(0x7f2db008ad00 msgr2=0x7f2db009a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.699+0000 7f2db6402640 1 -- 172.21.15.40:0/489936365 shutdown_connections 2023-10-18T08:56:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.699+0000 7f2db6402640 1 -- 172.21.15.40:0/489936365 wait complete. 2023-10-18T08:56:24.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.701+0000 7f2db6402640 1 Processor -- start 2023-10-18T08:56:24.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.701+0000 7f2db6402640 1 -- start start 2023-10-18T08:56:24.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.702+0000 7f2db6402640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d9c0039c0 0x7f2db00b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:24.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.702+0000 7f2db6402640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db00c5850 0x7f2db00c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:24.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.702+0000 7f2db6402640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2db0152b00 0x7f2db015a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:24.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.702+0000 7f2db6402640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00c6cf0 con 0x7f2db00c5850 2023-10-18T08:56:24.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.702+0000 7f2db6402640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00c4e40 con 0x7f2db0152b00 2023-10-18T08:56:24.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.702+0000 7f2db6402640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00c5cd0 con 0x7f2d9c0039c0 2023-10-18T08:56:24.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.702+0000 7f2db4bff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db00c5850 0x7f2db00c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:24.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.702+0000 7f2db4bff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db00c5850 0x7f2db00c5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35654/0 (socket says 172.21.15.40:35654) 2023-10-18T08:56:24.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.702+0000 7f2db4bff640 1 -- 172.21.15.40:0/2952436069 learned_addr learned my addr 172.21.15.40:0/2952436069 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:24.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.703+0000 7f2db5c01640 1 --2- 172.21.15.40:0/2952436069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2db0152b00 0x7f2db015a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:24.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.703+0000 7f2db4bff640 1 -- 172.21.15.40:0/2952436069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d9c0039c0 msgr2=0x7f2db00b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:24.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.703+0000 7f2db4bff640 1 --2- 172.21.15.40:0/2952436069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d9c0039c0 0x7f2db00b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:24.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.703+0000 7f2db4bff640 1 -- 172.21.15.40:0/2952436069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2db0152b00 msgr2=0x7f2db015a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:24.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.703+0000 7f2db4bff640 1 --2- 172.21.15.40:0/2952436069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2db0152b00 0x7f2db015a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.703+0000 7f2db4bff640 1 -- 172.21.15.40:0/2952436069 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db00c6730 con 0x7f2db00c5850 2023-10-18T08:56:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.703+0000 7f2db5c01640 1 --2- 172.21.15.40:0/2952436069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2db0152b00 0x7f2db015a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.703+0000 7f2db4bff640 1 --2- 172.21.15.40:0/2952436069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db00c5850 0x7f2db00c5ee0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2da000e4f0 tx=0x7f2da000e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.704+0000 7f2da5ffb640 1 -- 172.21.15.40:0/2952436069 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da0010890 con 0x7f2db00c5850 2023-10-18T08:56:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.704+0000 7f2da5ffb640 1 -- 172.21.15.40:0/2952436069 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2da0010ae0 con 0x7f2db00c5850 2023-10-18T08:56:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.704+0000 7f2db6402640 1 -- 172.21.15.40:0/2952436069 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db00c6a10 con 0x7f2db00c5850 2023-10-18T08:56:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.704+0000 7f2da5ffb640 1 -- 172.21.15.40:0/2952436069 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da0011460 con 0x7f2db00c5850 2023-10-18T08:56:24.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.704+0000 7f2db6402640 1 -- 172.21.15.40:0/2952436069 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db00c5330 con 0x7f2db00c5850 2023-10-18T08:56:24.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.705+0000 7f2db6402640 1 -- 172.21.15.40:0/2952436069 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db0071da0 con 0x7f2db00c5850 2023-10-18T08:56:24.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.706+0000 7f2da5ffb640 1 -- 172.21.15.40:0/2952436069 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2da00115c0 con 0x7f2db00c5850 2023-10-18T08:56:24.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.707+0000 7f2da5ffb640 1 --2- 172.21.15.40:0/2952436069 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2d9406ae40 0x7f2d9406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:24.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.708+0000 7f2da5ffb640 1 -- 172.21.15.40:0/2952436069 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2da0089a90 con 0x7f2db00c5850 2023-10-18T08:56:24.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.708+0000 7f2db5400640 1 --2- 172.21.15.40:0/2952436069 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2d9406ae40 0x7f2d9406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:24.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.709+0000 7f2db5400640 1 --2- 172.21.15.40:0/2952436069 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2d9406ae40 0x7f2d9406d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2dac034000 tx=0x7f2dac001120 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:24.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.712+0000 7f2da5ffb640 1 -- 172.21.15.40:0/2952436069 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2da0054df0 con 0x7f2db00c5850 2023-10-18T08:56:24.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.978+0000 7f2db6402640 1 -- 172.21.15.40:0/2952436069 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2db0066e00 con 0x7f2db00c5850 2023-10-18T08:56:24.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.979+0000 7f2da5ffb640 1 -- 172.21.15.40:0/2952436069 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f2da001f020 con 0x7f2db00c5850 2023-10-18T08:56:24.984 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:24.984 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T08:56:24.985 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2023-10-18T08:56:24.985 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:24.985 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2023-10-18T08:56:24.985 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lYsb5CxAAGButZ6+Usc62kJ0ksxZrYw==", 2023-10-18T08:56:24.985 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:24.986 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:56:24.986 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T08:56:24.986 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T08:56:24.986 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:56:24.987 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:24.987 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:24.987 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:24.987 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2023-10-18T08:56:24.987 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lAqlhMBAApAhxCYB9d9bz1GibADDRSQ==", 2023-10-18T08:56:24.988 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:24.988 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:56:24.988 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T08:56:24.988 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T08:56:24.988 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:56:24.988 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:24.989 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:24.989 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:24.989 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2023-10-18T08:56:24.989 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lPVh+HhAAzucSx+ksPDBkqJGa8+pyJw==", 2023-10-18T08:56:24.990 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:24.990 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:56:24.990 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T08:56:24.990 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T08:56:24.990 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:56:24.991 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:24.991 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:24.991 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:24.991 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2023-10-18T08:56:24.991 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lP7QCBhAAog7qDCgGVv0T7WKsRkATkA==", 2023-10-18T08:56:24.992 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:24.992 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:56:24.992 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T08:56:24.992 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T08:56:24.992 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:56:24.993 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:24.993 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:24.993 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:24.993 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2023-10-18T08:56:24.993 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lVx+hAhAA6ZpIyZKhbqy8tV84piGILA==", 2023-10-18T08:56:24.994 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:24.994 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:56:24.994 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:56:24.995 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:56:24.995 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:24.995 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:24.995 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:24.996 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2023-10-18T08:56:24.996 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lk9LwExAA7zFzQOn2hcvnck2cOZZ7AA==", 2023-10-18T08:56:24.996 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:24.996 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:56:24.997 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:56:24.997 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:56:24.997 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:24.998 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:24.998 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:24.998 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2023-10-18T08:56:24.998 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lK930JRAA1GyKRkIpXPHHGqY8owYWRQ==", 2023-10-18T08:56:24.999 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:24.999 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:56:24.999 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:56:24.999 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:56:25.000 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.000 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.000 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.000 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2023-10-18T08:56:25.000 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9liBWyNRAAMHBBdG5nbrGQupqvUtuKMA==", 2023-10-18T08:56:25.001 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.001 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:56:25.001 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:56:25.001 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:56:25.002 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.002 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.002 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.002 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2023-10-18T08:56:25.002 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9loDoOAhAAjHZub4wnB7dCr86rxAtI5g==", 2023-10-18T08:56:25.003 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.003 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:56:25.003 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:56:25.003 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:56:25.003 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.004 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.004 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.004 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2023-10-18T08:56:25.004 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lhqNfERAAw410g3DwSXrK23YkxVBFVw==", 2023-10-18T08:56:25.005 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.005 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:56:25.005 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:56:25.005 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:56:25.006 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.006 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.006 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.006 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2023-10-18T08:56:25.007 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lRRScIRAAg0pmfxCQBCPYodj1Lko+Lw==", 2023-10-18T08:56:25.007 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.007 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:56:25.007 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:56:25.007 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:56:25.008 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.008 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.008 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.008 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2023-10-18T08:56:25.008 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9ld1dXMhAAyovlrKOhVuynEiuPTgFcjQ==", 2023-10-18T08:56:25.009 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.009 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:56:25.009 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:56:25.009 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:56:25.009 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.010 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.010 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.010 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2023-10-18T08:56:25.010 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w==", 2023-10-18T08:56:25.011 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.011 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:56:25.011 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T08:56:25.011 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T08:56:25.012 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T08:56:25.012 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.012 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.012 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.012 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2023-10-18T08:56:25.013 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lAB3oKhAAD43WFdC23WGF6WUi4JKsvg==", 2023-10-18T08:56:25.013 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.013 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:56:25.013 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T08:56:25.013 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T08:56:25.014 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T08:56:25.014 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.014 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.015 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2023-10-18T08:56:25.015 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lu57fHxAAdzsVU2KKsWRkaARRS8TGjw==", 2023-10-18T08:56:25.015 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.015 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:56:25.015 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T08:56:25.015 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T08:56:25.016 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T08:56:25.016 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.016 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.016 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.017 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2023-10-18T08:56:25.017 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lygWsNRAA1OziXxk7hBtTw2Aq8OlSKQ==", 2023-10-18T08:56:25.017 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.017 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:56:25.017 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T08:56:25.017 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T08:56:25.018 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T08:56:25.018 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.018 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.019 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2023-10-18T08:56:25.019 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBMnS9lnrUjNhAAHP2eezRHF4/Jgox7iPwy7g==", 2023-10-18T08:56:25.019 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.020 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T08:56:25.020 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T08:56:25.020 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T08:56:25.021 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:56:25.021 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.021 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.021 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.022 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-mds", 2023-10-18T08:56:25.022 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBfnS9luxuINBAAIRtHWb/6OjTCUPerGDiDKg==", 2023-10-18T08:56:25.022 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.023 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-mds" 2023-10-18T08:56:25.023 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.023 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.024 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.024 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-mgr", 2023-10-18T08:56:25.024 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBfnS9lAC2INBAAA4oqUYSnLsNh+cask35PSA==", 2023-10-18T08:56:25.024 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.025 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-18T08:56:25.025 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.025 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.026 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.026 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-osd", 2023-10-18T08:56:25.026 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBfnS9lzj2INBAAUzljxuj5zKg95ku+jx3oeA==", 2023-10-18T08:56:25.027 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.027 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-osd" 2023-10-18T08:56:25.027 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.028 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.028 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rbd", 2023-10-18T08:56:25.029 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBfnS9lsU2INBAAsWjTsE6roZhPvSfECFWXIg==", 2023-10-18T08:56:25.029 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.029 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-18T08:56:25.029 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.030 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.030 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-18T08:56:25.031 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBfnS9l9l6INBAAWjdX3geajU8SB2qqbIpYwA==", 2023-10-18T08:56:25.031 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.031 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-18T08:56:25.031 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.032 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.032 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.032 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.bootstrap-rgw", 2023-10-18T08:56:25.032 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBfnS9lBXOINBAA8AGFC7BWfeVWQR4kasyKjA==", 2023-10-18T08:56:25.033 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.033 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-18T08:56:25.033 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.033 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.033 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.034 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2023-10-18T08:56:25.034 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9lCf/xOhAAJHVoKIswWUcVGKQpiaJ12A==", 2023-10-18T08:56:25.034 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.035 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T08:56:25.035 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T08:56:25.035 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:56:25.035 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.036 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:25.036 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:25.036 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2023-10-18T08:56:25.036 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9loxRXLhAAQK1n1/4PB35umHxOKQzz7Q==", 2023-10-18T08:56:25.036 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:56:25.037 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T08:56:25.037 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T08:56:25.037 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:56:25.037 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.038 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:25.038 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:56:25.038 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T08:56:25.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.982+0000 7f2db6402640 1 -- 172.21.15.40:0/2952436069 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2d9406ae40 msgr2=0x7f2d9406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.982+0000 7f2db6402640 1 --2- 172.21.15.40:0/2952436069 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2d9406ae40 0x7f2d9406d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2dac034000 tx=0x7f2dac001120 comp rx=0 tx=0).stop 2023-10-18T08:56:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.982+0000 7f2db6402640 1 -- 172.21.15.40:0/2952436069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db00c5850 msgr2=0x7f2db00c5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.982+0000 7f2db6402640 1 --2- 172.21.15.40:0/2952436069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db00c5850 0x7f2db00c5ee0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2da000e4f0 tx=0x7f2da000e9b0 comp rx=0 tx=0).stop 2023-10-18T08:56:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.982+0000 7f2db6402640 1 -- 172.21.15.40:0/2952436069 shutdown_connections 2023-10-18T08:56:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.982+0000 7f2db6402640 1 --2- 172.21.15.40:0/2952436069 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2d9406ae40 0x7f2d9406d300 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.982+0000 7f2db6402640 1 --2- 172.21.15.40:0/2952436069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2db0152b00 0x7f2db015a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.982+0000 7f2db6402640 1 --2- 172.21.15.40:0/2952436069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2db00c5850 0x7f2db00c5ee0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.982+0000 7f2db6402640 1 --2- 172.21.15.40:0/2952436069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d9c0039c0 0x7f2db00b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.982+0000 7f2db6402640 1 -- 172.21.15.40:0/2952436069 >> 172.21.15.40:0/2952436069 conn(0x7f2db008ad00 msgr2=0x7f2db00ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.983+0000 7f2db6402640 1 -- 172.21.15.40:0/2952436069 shutdown_connections 2023-10-18T08:56:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:24.983+0000 7f2db6402640 1 -- 172.21.15.40:0/2952436069 wait complete. 2023-10-18T08:56:25.042 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-18T08:56:25.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.286+0000 7fc174a13640 1 -- 172.21.15.40:0/2968544483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc15c004d00 msgr2=0x7fc15c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:25.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.286+0000 7fc174a13640 1 --2- 172.21.15.40:0/2968544483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc15c004d00 0x7fc15c005160 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc168006f00 tx=0x7fc15c0057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:25.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.286+0000 7fc16ffff640 1 -- 172.21.15.40:0/2968544483 reap_dead start 2023-10-18T08:56:25.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.286+0000 7fc174a13640 1 -- 172.21.15.40:0/2968544483 shutdown_connections 2023-10-18T08:56:25.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.286+0000 7fc174a13640 1 -- 172.21.15.40:0/2968544483 >> 172.21.15.40:0/2968544483 conn(0x7fc17008ad00 msgr2=0x7fc17009a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:25.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.286+0000 7fc174a13640 1 -- 172.21.15.40:0/2968544483 shutdown_connections 2023-10-18T08:56:25.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.286+0000 7fc174a13640 1 -- 172.21.15.40:0/2968544483 wait complete. 2023-10-18T08:56:25.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.288+0000 7fc174a13640 1 Processor -- start 2023-10-18T08:56:25.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.288+0000 7fc174a13640 1 -- start start 2023-10-18T08:56:25.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.289+0000 7fc174a13640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15c004d00 0x7fc1700cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:25.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.289+0000 7fc174a13640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1700cce20 0x7fc17015de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:25.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.289+0000 7fc174a13640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc17015e340 0x7fc17015e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:25.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.289+0000 7fc174a13640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1701f1e40 con 0x7fc1700cce20 2023-10-18T08:56:25.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.289+0000 7fc174a13640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1700cd2a0 con 0x7fc15c004d00 2023-10-18T08:56:25.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.289+0000 7fc174a13640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1701f14c0 con 0x7fc17015e340 2023-10-18T08:56:25.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.289+0000 7fc16effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1700cce20 0x7fc17015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:25.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.289+0000 7fc16effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1700cce20 0x7fc17015de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59698/0 (socket says 172.21.15.40:59698) 2023-10-18T08:56:25.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.289+0000 7fc16effd640 1 -- 172.21.15.40:0/463523168 learned_addr learned my addr 172.21.15.40:0/463523168 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:25.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.289+0000 7fc16f7fe640 1 --2- 172.21.15.40:0/463523168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15c004d00 0x7fc1700cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:25.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.289+0000 7fc16ffff640 1 --2- 172.21.15.40:0/463523168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc17015e340 0x7fc17015e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:25.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.290+0000 7fc16effd640 1 -- 172.21.15.40:0/463523168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc17015e340 msgr2=0x7fc17015e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:25.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.290+0000 7fc16effd640 1 --2- 172.21.15.40:0/463523168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc17015e340 0x7fc17015e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:25.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.290+0000 7fc16effd640 1 -- 172.21.15.40:0/463523168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15c004d00 msgr2=0x7fc1700cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:25.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.290+0000 7fc16effd640 1 --2- 172.21.15.40:0/463523168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15c004d00 0x7fc1700cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:25.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.290+0000 7fc16effd640 1 -- 172.21.15.40:0/463523168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17014ced0 con 0x7fc1700cce20 2023-10-18T08:56:25.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.290+0000 7fc16ffff640 1 --2- 172.21.15.40:0/463523168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc17015e340 0x7fc17015e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:56:25.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.290+0000 7fc16effd640 1 --2- 172.21.15.40:0/463523168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1700cce20 0x7fc17015de00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc158011a50 tx=0x7fc158011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.290+0000 7fc167fff640 1 -- 172.21.15.40:0/463523168 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15800fcd0 con 0x7fc1700cce20 2023-10-18T08:56:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.290+0000 7fc167fff640 1 -- 172.21.15.40:0/463523168 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc158012450 con 0x7fc1700cce20 2023-10-18T08:56:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.290+0000 7fc167fff640 1 -- 172.21.15.40:0/463523168 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158012840 con 0x7fc1700cce20 2023-10-18T08:56:25.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.290+0000 7fc174a13640 1 -- 172.21.15.40:0/463523168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1701f1780 con 0x7fc1700cce20 2023-10-18T08:56:25.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.291+0000 7fc174a13640 1 -- 172.21.15.40:0/463523168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1701f2260 con 0x7fc1700cce20 2023-10-18T08:56:25.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.292+0000 7fc174a13640 1 -- 172.21.15.40:0/463523168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc170071da0 con 0x7fc1700cce20 2023-10-18T08:56:25.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.293+0000 7fc167fff640 1 -- 172.21.15.40:0/463523168 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc1580129a0 con 0x7fc1700cce20 2023-10-18T08:56:25.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.295+0000 7fc167fff640 1 --2- 172.21.15.40:0/463523168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc15006ae40 0x7fc15006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:25.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.295+0000 7fc167fff640 1 -- 172.21.15.40:0/463523168 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc15808a0d0 con 0x7fc1700cce20 2023-10-18T08:56:25.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.295+0000 7fc16f7fe640 1 --2- 172.21.15.40:0/463523168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc15006ae40 0x7fc15006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:25.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.296+0000 7fc16f7fe640 1 --2- 172.21.15.40:0/463523168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc15006ae40 0x7fc15006d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc160006fd0 tx=0x7fc160006e60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:25.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.300+0000 7fc167fff640 1 -- 172.21.15.40:0/463523168 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc158055610 con 0x7fc1700cce20 2023-10-18T08:56:25.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.569+0000 7fc174a13640 1 -- 172.21.15.40:0/463523168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fc170066e00 con 0x7fc1700cce20 2023-10-18T08:56:25.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.574+0000 7fc167fff640 1 -- 172.21.15.40:0/463523168 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc15801f0a0 con 0x7fc1700cce20 2023-10-18T08:56:25.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.576+0000 7fc174a13640 1 -- 172.21.15.40:0/463523168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc15006ae40 msgr2=0x7fc15006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:25.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.576+0000 7fc174a13640 1 --2- 172.21.15.40:0/463523168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc15006ae40 0x7fc15006d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc160006fd0 tx=0x7fc160006e60 comp rx=0 tx=0).stop 2023-10-18T08:56:25.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.576+0000 7fc174a13640 1 -- 172.21.15.40:0/463523168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1700cce20 msgr2=0x7fc17015de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:25.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.576+0000 7fc174a13640 1 --2- 172.21.15.40:0/463523168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1700cce20 0x7fc17015de00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc158011a50 tx=0x7fc158011f10 comp rx=0 tx=0).stop 2023-10-18T08:56:25.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.577+0000 7fc174a13640 1 -- 172.21.15.40:0/463523168 shutdown_connections 2023-10-18T08:56:25.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.577+0000 7fc174a13640 1 --2- 172.21.15.40:0/463523168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc15006ae40 0x7fc15006d300 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:25.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.577+0000 7fc174a13640 1 --2- 172.21.15.40:0/463523168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc17015e340 0x7fc17015e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:25.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.577+0000 7fc174a13640 1 --2- 172.21.15.40:0/463523168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1700cce20 0x7fc17015de00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:25.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.577+0000 7fc174a13640 1 --2- 172.21.15.40:0/463523168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc15c004d00 0x7fc1700cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:25.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.577+0000 7fc174a13640 1 -- 172.21.15.40:0/463523168 >> 172.21.15.40:0/463523168 conn(0x7fc17008ad00 msgr2=0x7fc17015a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:25.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.577+0000 7fc174a13640 1 -- 172.21.15.40:0/463523168 shutdown_connections 2023-10-18T08:56:25.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.577+0000 7fc174a13640 1 -- 172.21.15.40:0/463523168 wait complete. 2023-10-18T08:56:25.599 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-18T08:56:25.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.825+0000 7f39e1547640 1 -- 172.21.15.40:0/2008278536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c8003970 msgr2=0x7f39c8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.825+0000 7f39e1547640 1 --2- 172.21.15.40:0/2008278536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c8003970 0x7f39c8003d90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009a10 tx=0x7f39c80057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.825+0000 7f39e0d46640 1 -- 172.21.15.40:0/2008278536 reap_dead start 2023-10-18T08:56:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.825+0000 7f39e1547640 1 -- 172.21.15.40:0/2008278536 shutdown_connections 2023-10-18T08:56:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.825+0000 7f39e1547640 1 -- 172.21.15.40:0/2008278536 >> 172.21.15.40:0/2008278536 conn(0x7f39dc069720 msgr2=0x7f39dc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.825+0000 7f39e1547640 1 -- 172.21.15.40:0/2008278536 shutdown_connections 2023-10-18T08:56:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.825+0000 7f39e1547640 1 -- 172.21.15.40:0/2008278536 wait complete. 2023-10-18T08:56:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.827+0000 7f39e1547640 1 Processor -- start 2023-10-18T08:56:25.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.827+0000 7f39e1547640 1 -- start start 2023-10-18T08:56:25.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.828+0000 7f39e1547640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39c8003970 0x7f39dc156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:25.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.828+0000 7f39e1547640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39dc139b40 0x7f39dc15af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:25.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.828+0000 7f39e1547640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39dc0aecd0 0x7f39dc0af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:25.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.828+0000 7f39e1547640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc0af6c0 con 0x7f39dc139b40 2023-10-18T08:56:25.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.828+0000 7f39e1547640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc13a020 con 0x7f39dc0aecd0 2023-10-18T08:56:25.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.828+0000 7f39e1547640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc13a300 con 0x7f39c8003970 2023-10-18T08:56:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.829+0000 7f39d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39dc139b40 0x7f39dc15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.829+0000 7f39d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39dc139b40 0x7f39dc15af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59722/0 (socket says 172.21.15.40:59722) 2023-10-18T08:56:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.829+0000 7f39d3fff640 1 -- 172.21.15.40:0/1261361188 learned_addr learned my addr 172.21.15.40:0/1261361188 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.829+0000 7f39dbfff640 1 --2- 172.21.15.40:0/1261361188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39c8003970 0x7f39dc156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:25.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.829+0000 7f39e0d46640 1 --2- 172.21.15.40:0/1261361188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39dc0aecd0 0x7f39dc0af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:25.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.829+0000 7f39d3fff640 1 -- 172.21.15.40:0/1261361188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39c8003970 msgr2=0x7f39dc156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:25.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.829+0000 7f39d3fff640 1 --2- 172.21.15.40:0/1261361188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39c8003970 0x7f39dc156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:25.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.829+0000 7f39d3fff640 1 -- 172.21.15.40:0/1261361188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39dc0aecd0 msgr2=0x7f39dc0af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:25.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.829+0000 7f39d3fff640 1 --2- 172.21.15.40:0/1261361188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39dc0aecd0 0x7f39dc0af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:25.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.829+0000 7f39d3fff640 1 -- 172.21.15.40:0/1261361188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc13a5e0 con 0x7f39dc139b40 2023-10-18T08:56:25.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.830+0000 7f39d3fff640 1 --2- 172.21.15.40:0/1261361188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39dc139b40 0x7f39dc15af60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f39c4001050 tx=0x7f39c400ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:25.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.830+0000 7f39d97fa640 1 -- 172.21.15.40:0/1261361188 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4010890 con 0x7f39dc139b40 2023-10-18T08:56:25.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.830+0000 7f39d97fa640 1 -- 172.21.15.40:0/1261361188 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f39c4010b20 con 0x7f39dc139b40 2023-10-18T08:56:25.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.830+0000 7f39d97fa640 1 -- 172.21.15.40:0/1261361188 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4011460 con 0x7f39dc139b40 2023-10-18T08:56:25.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.830+0000 7f39e1547640 1 -- 172.21.15.40:0/1261361188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39dc13a8c0 con 0x7f39dc139b40 2023-10-18T08:56:25.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.830+0000 7f39e1547640 1 -- 172.21.15.40:0/1261361188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39dc157310 con 0x7f39dc139b40 2023-10-18T08:56:25.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.831+0000 7f39e1547640 1 -- 172.21.15.40:0/1261361188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39dc134050 con 0x7f39dc139b40 2023-10-18T08:56:25.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.832+0000 7f39d97fa640 1 -- 172.21.15.40:0/1261361188 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f39c40115c0 con 0x7f39dc139b40 2023-10-18T08:56:25.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.833+0000 7f39d97fa640 1 --2- 172.21.15.40:0/1261361188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39bc06ae70 0x7f39bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:25.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.833+0000 7f39d97fa640 1 -- 172.21.15.40:0/1261361188 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f39c4089990 con 0x7f39dc139b40 2023-10-18T08:56:25.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.834+0000 7f39dbfff640 1 --2- 172.21.15.40:0/1261361188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39bc06ae70 0x7f39bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:25.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.835+0000 7f39dbfff640 1 --2- 172.21.15.40:0/1261361188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39bc06ae70 0x7f39bc06d330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f39cc036000 tx=0x7f39cc0046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:25.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:25.837+0000 7f39d97fa640 1 -- 172.21.15.40:0/1261361188 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39dc134050 con 0x7f39dc139b40 2023-10-18T08:56:26.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.105+0000 7f39e1547640 1 -- 172.21.15.40:0/1261361188 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f39dc134030 con 0x7f39dc139b40 2023-10-18T08:56:26.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.109+0000 7f39d97fa640 1 -- 172.21.15.40:0/1261361188 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f39c401f0a0 con 0x7f39dc139b40 2023-10-18T08:56:26.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.111+0000 7f39e1547640 1 -- 172.21.15.40:0/1261361188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39bc06ae70 msgr2=0x7f39bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:26.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.112+0000 7f39e1547640 1 --2- 172.21.15.40:0/1261361188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39bc06ae70 0x7f39bc06d330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f39cc036000 tx=0x7f39cc0046c0 comp rx=0 tx=0).stop 2023-10-18T08:56:26.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.112+0000 7f39e1547640 1 -- 172.21.15.40:0/1261361188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39dc139b40 msgr2=0x7f39dc15af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:26.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.112+0000 7f39e1547640 1 --2- 172.21.15.40:0/1261361188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39dc139b40 0x7f39dc15af60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f39c4001050 tx=0x7f39c400ea00 comp rx=0 tx=0).stop 2023-10-18T08:56:26.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.112+0000 7f39e1547640 1 -- 172.21.15.40:0/1261361188 shutdown_connections 2023-10-18T08:56:26.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.112+0000 7f39e1547640 1 --2- 172.21.15.40:0/1261361188 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39bc06ae70 0x7f39bc06d330 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:26.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.112+0000 7f39e1547640 1 --2- 172.21.15.40:0/1261361188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39dc0aecd0 0x7f39dc0af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:26.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.112+0000 7f39e1547640 1 --2- 172.21.15.40:0/1261361188 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39dc139b40 0x7f39dc15af60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:26.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.112+0000 7f39e1547640 1 --2- 172.21.15.40:0/1261361188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39c8003970 0x7f39dc156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:26.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.112+0000 7f39e1547640 1 -- 172.21.15.40:0/1261361188 >> 172.21.15.40:0/1261361188 conn(0x7f39dc069720 msgr2=0x7f39dc072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:26.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.112+0000 7f39e1547640 1 -- 172.21.15.40:0/1261361188 shutdown_connections 2023-10-18T08:56:26.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.112+0000 7f39e1547640 1 -- 172.21.15.40:0/1261361188 wait complete. 2023-10-18T08:56:26.133 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-18T08:56:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.366+0000 7fd6db59e640 1 -- 172.21.15.40:0/1899276720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6c8003f40 msgr2=0x7fd6c80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:26.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.366+0000 7fd6db59e640 1 --2- 172.21.15.40:0/1899276720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6c8003f40 0x7fd6c80047c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009b80 tx=0x7fd6c80057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:26.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.366+0000 7fd6dad9d640 1 -- 172.21.15.40:0/1899276720 reap_dead start 2023-10-18T08:56:26.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.367+0000 7fd6db59e640 1 -- 172.21.15.40:0/1899276720 shutdown_connections 2023-10-18T08:56:26.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.367+0000 7fd6db59e640 1 -- 172.21.15.40:0/1899276720 >> 172.21.15.40:0/1899276720 conn(0x7fd6dc08ad70 msgr2=0x7fd6dc09a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:26.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.367+0000 7fd6db59e640 1 -- 172.21.15.40:0/1899276720 shutdown_connections 2023-10-18T08:56:26.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.367+0000 7fd6db59e640 1 -- 172.21.15.40:0/1899276720 wait complete. 2023-10-18T08:56:26.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.368+0000 7fd6db59e640 1 Processor -- start 2023-10-18T08:56:26.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.369+0000 7fd6db59e640 1 -- start start 2023-10-18T08:56:26.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.370+0000 7fd6db59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c8003f40 0x7fd6dc15b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:26.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.370+0000 7fd6db59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6dc0b4f70 0x7fd6dc0c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:26.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.370+0000 7fd6db59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc15d820 0x7fd6dc15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:26.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.370+0000 7fd6db59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc15e1e0 con 0x7fd6dc0b4f70 2023-10-18T08:56:26.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.370+0000 7fd6db59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc0b6aa0 con 0x7fd6c8003f40 2023-10-18T08:56:26.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.370+0000 7fd6db59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc1f2a20 con 0x7fd6dc15d820 2023-10-18T08:56:26.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.370+0000 7fd6d9d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6dc0b4f70 0x7fd6dc0c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:26.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.370+0000 7fd6d9d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6dc0b4f70 0x7fd6dc0c7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59742/0 (socket says 172.21.15.40:59742) 2023-10-18T08:56:26.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.370+0000 7fd6d9d9b640 1 -- 172.21.15.40:0/900856460 learned_addr learned my addr 172.21.15.40:0/900856460 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:26.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.370+0000 7fd6dad9d640 1 --2- 172.21.15.40:0/900856460 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc15d820 0x7fd6dc15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:26.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.370+0000 7fd6da59c640 1 --2- 172.21.15.40:0/900856460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c8003f40 0x7fd6dc15b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:26.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.370+0000 7fd6d9d9b640 1 -- 172.21.15.40:0/900856460 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc15d820 msgr2=0x7fd6dc15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:26.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.370+0000 7fd6d9d9b640 1 --2- 172.21.15.40:0/900856460 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc15d820 0x7fd6dc15dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:26.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.370+0000 7fd6d9d9b640 1 -- 172.21.15.40:0/900856460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c8003f40 msgr2=0x7fd6dc15b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:26.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.370+0000 7fd6d9d9b640 1 --2- 172.21.15.40:0/900856460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c8003f40 0x7fd6dc15b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:26.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.370+0000 7fd6d9d9b640 1 -- 172.21.15.40:0/900856460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc1f1ea0 con 0x7fd6dc0b4f70 2023-10-18T08:56:26.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.371+0000 7fd6d9d9b640 1 --2- 172.21.15.40:0/900856460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6dc0b4f70 0x7fd6dc0c7490 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009b20 tx=0x7fd6c4033c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:26.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.371+0000 7fd6d2ffd640 1 -- 172.21.15.40:0/900856460 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4004460 con 0x7fd6dc0b4f70 2023-10-18T08:56:26.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.371+0000 7fd6d2ffd640 1 -- 172.21.15.40:0/900856460 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd6c4033e60 con 0x7fd6dc0b4f70 2023-10-18T08:56:26.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.371+0000 7fd6d2ffd640 1 -- 172.21.15.40:0/900856460 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4003830 con 0x7fd6dc0b4f70 2023-10-18T08:56:26.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.371+0000 7fd6db59e640 1 -- 172.21.15.40:0/900856460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6dc1f2180 con 0x7fd6dc0b4f70 2023-10-18T08:56:26.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.371+0000 7fd6db59e640 1 -- 172.21.15.40:0/900856460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6dc1f2740 con 0x7fd6dc0b4f70 2023-10-18T08:56:26.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.372+0000 7fd6db59e640 1 -- 172.21.15.40:0/900856460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6dc071da0 con 0x7fd6dc0b4f70 2023-10-18T08:56:26.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.373+0000 7fd6d2ffd640 1 -- 172.21.15.40:0/900856460 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd6c4003990 con 0x7fd6dc0b4f70 2023-10-18T08:56:26.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.374+0000 7fd6d2ffd640 1 --2- 172.21.15.40:0/900856460 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd6bc06ae70 0x7fd6bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:26.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.374+0000 7fd6d2ffd640 1 -- 172.21.15.40:0/900856460 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd6c40ac250 con 0x7fd6dc0b4f70 2023-10-18T08:56:26.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.375+0000 7fd6da59c640 1 --2- 172.21.15.40:0/900856460 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd6bc06ae70 0x7fd6bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:26.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.376+0000 7fd6da59c640 1 --2- 172.21.15.40:0/900856460 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd6bc06ae70 0x7fd6bc06d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc005fd0 tx=0x7fd6cc009500 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:26.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.379+0000 7fd6d2ffd640 1 -- 172.21.15.40:0/900856460 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6c407b040 con 0x7fd6dc0b4f70 2023-10-18T08:56:26.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.656+0000 7fd6db59e640 1 -- 172.21.15.40:0/900856460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fd6dc066e00 con 0x7fd6dc0b4f70 2023-10-18T08:56:26.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.661+0000 7fd6d2ffd640 1 -- 172.21.15.40:0/900856460 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd6c404a0a0 con 0x7fd6dc0b4f70 2023-10-18T08:56:26.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.664+0000 7fd6db59e640 1 -- 172.21.15.40:0/900856460 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd6bc06ae70 msgr2=0x7fd6bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:26.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.664+0000 7fd6db59e640 1 --2- 172.21.15.40:0/900856460 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd6bc06ae70 0x7fd6bc06d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc005fd0 tx=0x7fd6cc009500 comp rx=0 tx=0).stop 2023-10-18T08:56:26.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.664+0000 7fd6db59e640 1 -- 172.21.15.40:0/900856460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6dc0b4f70 msgr2=0x7fd6dc0c7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:26.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.664+0000 7fd6db59e640 1 --2- 172.21.15.40:0/900856460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6dc0b4f70 0x7fd6dc0c7490 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009b20 tx=0x7fd6c4033c30 comp rx=0 tx=0).stop 2023-10-18T08:56:26.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.665+0000 7fd6db59e640 1 -- 172.21.15.40:0/900856460 shutdown_connections 2023-10-18T08:56:26.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.665+0000 7fd6db59e640 1 --2- 172.21.15.40:0/900856460 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd6bc06ae70 0x7fd6bc06d330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:26.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.665+0000 7fd6db59e640 1 --2- 172.21.15.40:0/900856460 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6dc15d820 0x7fd6dc15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:26.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.665+0000 7fd6db59e640 1 --2- 172.21.15.40:0/900856460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6dc0b4f70 0x7fd6dc0c7490 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:26.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.665+0000 7fd6db59e640 1 --2- 172.21.15.40:0/900856460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c8003f40 0x7fd6dc15b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:26.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.665+0000 7fd6db59e640 1 -- 172.21.15.40:0/900856460 >> 172.21.15.40:0/900856460 conn(0x7fd6dc08ad70 msgr2=0x7fd6dc151990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:26.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.665+0000 7fd6db59e640 1 -- 172.21.15.40:0/900856460 shutdown_connections 2023-10-18T08:56:26.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.665+0000 7fd6db59e640 1 -- 172.21.15.40:0/900856460 wait complete. 2023-10-18T08:56:26.685 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-18T08:56:26.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.907+0000 7f8a6481a640 1 -- 172.21.15.40:0/642620561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a4c004d00 msgr2=0x7f8a4c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:26.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.907+0000 7f8a6481a640 1 --2- 172.21.15.40:0/642620561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a4c004d00 0x7f8a4c005180 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8a58001680 tx=0x7f8a4c0057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:26.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.907+0000 7f8a5ed9d640 1 -- 172.21.15.40:0/642620561 reap_dead start 2023-10-18T08:56:26.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.907+0000 7f8a6481a640 1 -- 172.21.15.40:0/642620561 shutdown_connections 2023-10-18T08:56:26.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.907+0000 7f8a6481a640 1 -- 172.21.15.40:0/642620561 >> 172.21.15.40:0/642620561 conn(0x7f8a6006cee0 msgr2=0x7f8a60069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:26.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.907+0000 7f8a6481a640 1 -- 172.21.15.40:0/642620561 shutdown_connections 2023-10-18T08:56:26.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.907+0000 7f8a6481a640 1 -- 172.21.15.40:0/642620561 wait complete. 2023-10-18T08:56:26.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.909+0000 7f8a6481a640 1 Processor -- start 2023-10-18T08:56:26.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.909+0000 7f8a6481a640 1 -- start start 2023-10-18T08:56:26.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.910+0000 7f8a6481a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a4c004d00 0x7f8a6013aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:26.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.910+0000 7f8a6481a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a600cbf70 0x7f8a601f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:26.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.910+0000 7f8a6481a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a601f2de0 0x7f8a6014ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:26.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.910+0000 7f8a6481a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6013b1d0 con 0x7f8a601f2de0 2023-10-18T08:56:26.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.910+0000 7f8a6481a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6013af50 con 0x7f8a600cbf70 2023-10-18T08:56:26.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.910+0000 7f8a6481a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a600cb1b0 con 0x7f8a4c004d00 2023-10-18T08:56:26.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.911+0000 7f8a5ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a601f2de0 0x7f8a6014ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:26.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.911+0000 7f8a5ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a601f2de0 0x7f8a6014ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59750/0 (socket says 172.21.15.40:59750) 2023-10-18T08:56:26.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.911+0000 7f8a5ed9d640 1 -- 172.21.15.40:0/2765857329 learned_addr learned my addr 172.21.15.40:0/2765857329 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:26.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.911+0000 7f8a5dd9b640 1 --2- 172.21.15.40:0/2765857329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a600cbf70 0x7f8a601f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:26.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.911+0000 7f8a5ed9d640 1 -- 172.21.15.40:0/2765857329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a4c004d00 msgr2=0x7f8a6013aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:26.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.911+0000 7f8a5e59c640 1 --2- 172.21.15.40:0/2765857329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a4c004d00 0x7f8a6013aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:26.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.911+0000 7f8a5ed9d640 1 --2- 172.21.15.40:0/2765857329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a4c004d00 0x7f8a6013aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:26.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.911+0000 7f8a5ed9d640 1 -- 172.21.15.40:0/2765857329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a600cbf70 msgr2=0x7f8a601f28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:26.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.911+0000 7f8a5ed9d640 1 --2- 172.21.15.40:0/2765857329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a600cbf70 0x7f8a601f28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:26.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.911+0000 7f8a5ed9d640 1 -- 172.21.15.40:0/2765857329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6015c1d0 con 0x7f8a601f2de0 2023-10-18T08:56:26.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.911+0000 7f8a5dd9b640 1 --2- 172.21.15.40:0/2765857329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a600cbf70 0x7f8a601f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:26.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.912+0000 7f8a5ed9d640 1 --2- 172.21.15.40:0/2765857329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a601f2de0 0x7f8a6014ff40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8a5802c0f0 tx=0x7f8a580062e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:26.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.912+0000 7f8a56ffd640 1 -- 172.21.15.40:0/2765857329 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58003a00 con 0x7f8a601f2de0 2023-10-18T08:56:26.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.912+0000 7f8a56ffd640 1 -- 172.21.15.40:0/2765857329 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8a58003c70 con 0x7f8a601f2de0 2023-10-18T08:56:26.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.912+0000 7f8a6481a640 1 -- 172.21.15.40:0/2765857329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6013b1d0 con 0x7f8a601f2de0 2023-10-18T08:56:26.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.912+0000 7f8a56ffd640 1 -- 172.21.15.40:0/2765857329 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5802c7b0 con 0x7f8a601f2de0 2023-10-18T08:56:26.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.912+0000 7f8a6481a640 1 -- 172.21.15.40:0/2765857329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a600cb770 con 0x7f8a601f2de0 2023-10-18T08:56:26.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.913+0000 7f8a6481a640 1 -- 172.21.15.40:0/2765857329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a60066fd0 con 0x7f8a601f2de0 2023-10-18T08:56:26.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.914+0000 7f8a56ffd640 1 -- 172.21.15.40:0/2765857329 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8a5803d070 con 0x7f8a601f2de0 2023-10-18T08:56:26.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.915+0000 7f8a56ffd640 1 --2- 172.21.15.40:0/2765857329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8a4006ae40 0x7f8a4006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:26.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.915+0000 7f8a56ffd640 1 -- 172.21.15.40:0/2765857329 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8a580aa400 con 0x7f8a601f2de0 2023-10-18T08:56:26.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.916+0000 7f8a5e59c640 1 --2- 172.21.15.40:0/2765857329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8a4006ae40 0x7f8a4006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:26.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.917+0000 7f8a5e59c640 1 --2- 172.21.15.40:0/2765857329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8a4006ae40 0x7f8a4006d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8a50006d50 tx=0x7f8a50001510 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:26.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:26.919+0000 7f8a56ffd640 1 -- 172.21.15.40:0/2765857329 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a580757f0 con 0x7f8a601f2de0 2023-10-18T08:56:27.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.183+0000 7f8a6481a640 1 -- 172.21.15.40:0/2765857329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f8a600b0b70 con 0x7f8a601f2de0 2023-10-18T08:56:27.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.188+0000 7f8a56ffd640 1 -- 172.21.15.40:0/2765857329 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8a58078e10 con 0x7f8a601f2de0 2023-10-18T08:56:27.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.190+0000 7f8a6481a640 1 -- 172.21.15.40:0/2765857329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8a4006ae40 msgr2=0x7f8a4006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:27.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.190+0000 7f8a6481a640 1 --2- 172.21.15.40:0/2765857329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8a4006ae40 0x7f8a4006d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8a50006d50 tx=0x7f8a50001510 comp rx=0 tx=0).stop 2023-10-18T08:56:27.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.190+0000 7f8a6481a640 1 -- 172.21.15.40:0/2765857329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a601f2de0 msgr2=0x7f8a6014ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:27.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.190+0000 7f8a6481a640 1 --2- 172.21.15.40:0/2765857329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a601f2de0 0x7f8a6014ff40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8a5802c0f0 tx=0x7f8a580062e0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.190+0000 7f8a6481a640 1 -- 172.21.15.40:0/2765857329 shutdown_connections 2023-10-18T08:56:27.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.190+0000 7f8a6481a640 1 --2- 172.21.15.40:0/2765857329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8a4006ae40 0x7f8a4006d300 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.190+0000 7f8a6481a640 1 --2- 172.21.15.40:0/2765857329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a601f2de0 0x7f8a6014ff40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.190+0000 7f8a6481a640 1 --2- 172.21.15.40:0/2765857329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a600cbf70 0x7f8a601f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.190+0000 7f8a6481a640 1 --2- 172.21.15.40:0/2765857329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a4c004d00 0x7f8a6013aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.190+0000 7f8a6481a640 1 -- 172.21.15.40:0/2765857329 >> 172.21.15.40:0/2765857329 conn(0x7f8a6006cee0 msgr2=0x7f8a6014ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:27.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.190+0000 7f8a6481a640 1 -- 172.21.15.40:0/2765857329 shutdown_connections 2023-10-18T08:56:27.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.190+0000 7f8a6481a640 1 -- 172.21.15.40:0/2765857329 wait complete. 2023-10-18T08:56:27.210 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-18T08:56:27.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.439+0000 7fed8493e640 1 -- 172.21.15.40:0/1536169892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed800b0bc0 msgr2=0x7fed800b1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:27.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.439+0000 7fed8493e640 1 --2- 172.21.15.40:0/1536169892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed800b0bc0 0x7fed800b1040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fed6800a940 tx=0x7fed6802d090 comp rx=0 tx=0).stop 2023-10-18T08:56:27.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.439+0000 7fed8493e640 1 -- 172.21.15.40:0/1536169892 shutdown_connections 2023-10-18T08:56:27.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.439+0000 7fed8493e640 1 --2- 172.21.15.40:0/1536169892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed800b0bc0 0x7fed800b1040 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.439+0000 7fed8493e640 1 --2- 172.21.15.40:0/1536169892 >> v2:172.21.15.144:3300/0 conn(0x7fed80151bd0 0x7fed801586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.439+0000 7fed8493e640 1 -- 172.21.15.40:0/1536169892 >> 172.21.15.40:0/1536169892 conn(0x7fed8006cee0 msgr2=0x7fed80069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:27.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.439+0000 7fed8493e640 1 -- 172.21.15.40:0/1536169892 shutdown_connections 2023-10-18T08:56:27.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.439+0000 7fed8493e640 1 -- 172.21.15.40:0/1536169892 wait complete. 2023-10-18T08:56:27.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.440+0000 7fed8493e640 1 Processor -- start 2023-10-18T08:56:27.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.441+0000 7fed8493e640 1 -- start start 2023-10-18T08:56:27.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.442+0000 7fed8493e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed8006b710 0x7fed80140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:27.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.442+0000 7fed8493e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed800b0bc0 0x7fed80141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:27.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.442+0000 7fed8493e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed80151bd0 0x7fed8014f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:27.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.442+0000 7fed8493e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed800cb600 con 0x7fed800b0bc0 2023-10-18T08:56:27.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.442+0000 7fed8493e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8014c7c0 con 0x7fed80151bd0 2023-10-18T08:56:27.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.442+0000 7fed8493e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8014ca10 con 0x7fed8006b710 2023-10-18T08:56:27.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.442+0000 7fed75d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed800b0bc0 0x7fed80141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:27.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.442+0000 7fed75d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed800b0bc0 0x7fed80141460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59752/0 (socket says 172.21.15.40:59752) 2023-10-18T08:56:27.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.442+0000 7fed75d9b640 1 -- 172.21.15.40:0/1459770812 learned_addr learned my addr 172.21.15.40:0/1459770812 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:27.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.442+0000 7fed7e59c640 1 --2- 172.21.15.40:0/1459770812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed8006b710 0x7fed80140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:27.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.442+0000 7fed75d9b640 1 -- 172.21.15.40:0/1459770812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed8006b710 msgr2=0x7fed80140aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:27.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.442+0000 7fed75d9b640 1 --2- 172.21.15.40:0/1459770812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed8006b710 0x7fed80140aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.442+0000 7fed75d9b640 1 -- 172.21.15.40:0/1459770812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed80151bd0 msgr2=0x7fed8014f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:27.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.442+0000 7fed75d9b640 1 --2- 172.21.15.40:0/1459770812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed80151bd0 0x7fed8014f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.442+0000 7fed75d9b640 1 -- 172.21.15.40:0/1459770812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6800a5a0 con 0x7fed800b0bc0 2023-10-18T08:56:27.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.443+0000 7fed75d9b640 1 --2- 172.21.15.40:0/1459770812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed800b0bc0 0x7fed80141460 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fed68000c00 tx=0x7fed68036c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:27.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.443+0000 7fed777fe640 1 -- 172.21.15.40:0/1459770812 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68003a00 con 0x7fed800b0bc0 2023-10-18T08:56:27.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.443+0000 7fed777fe640 1 -- 172.21.15.40:0/1459770812 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fed68003c50 con 0x7fed800b0bc0 2023-10-18T08:56:27.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.443+0000 7fed777fe640 1 -- 172.21.15.40:0/1459770812 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68036e40 con 0x7fed800b0bc0 2023-10-18T08:56:27.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.443+0000 7fed8493e640 1 -- 172.21.15.40:0/1459770812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed8014ccf0 con 0x7fed800b0bc0 2023-10-18T08:56:27.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.443+0000 7fed8493e640 1 -- 172.21.15.40:0/1459770812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed80140fe0 con 0x7fed800b0bc0 2023-10-18T08:56:27.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.445+0000 7fed8493e640 1 -- 172.21.15.40:0/1459770812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed800cb740 con 0x7fed800b0bc0 2023-10-18T08:56:27.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.446+0000 7fed777fe640 1 -- 172.21.15.40:0/1459770812 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fed6803c040 con 0x7fed800b0bc0 2023-10-18T08:56:27.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.447+0000 7fed777fe640 1 --2- 172.21.15.40:0/1459770812 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fed6006ae40 0x7fed6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:27.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.447+0000 7fed777fe640 1 -- 172.21.15.40:0/1459770812 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fed680abf50 con 0x7fed800b0bc0 2023-10-18T08:56:27.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.447+0000 7fed7e59c640 1 --2- 172.21.15.40:0/1459770812 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fed6006ae40 0x7fed6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:27.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.448+0000 7fed7e59c640 1 --2- 172.21.15.40:0/1459770812 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fed6006ae40 0x7fed6006d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fed70005fd0 tx=0x7fed70009260 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:27.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.451+0000 7fed777fe640 1 -- 172.21.15.40:0/1459770812 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed800cb740 con 0x7fed800b0bc0 2023-10-18T08:56:27.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.704+0000 7fed8493e640 1 -- 172.21.15.40:0/1459770812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fed800cb740 con 0x7fed800b0bc0 2023-10-18T08:56:27.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.709+0000 7fed777fe640 1 -- 172.21.15.40:0/1459770812 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fed800cb740 con 0x7fed800b0bc0 2023-10-18T08:56:27.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.711+0000 7fed8493e640 1 -- 172.21.15.40:0/1459770812 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fed6006ae40 msgr2=0x7fed6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:27.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.711+0000 7fed8493e640 1 --2- 172.21.15.40:0/1459770812 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fed6006ae40 0x7fed6006d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fed70005fd0 tx=0x7fed70009260 comp rx=0 tx=0).stop 2023-10-18T08:56:27.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.711+0000 7fed8493e640 1 -- 172.21.15.40:0/1459770812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed800b0bc0 msgr2=0x7fed80141460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:27.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.711+0000 7fed8493e640 1 --2- 172.21.15.40:0/1459770812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed800b0bc0 0x7fed80141460 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fed68000c00 tx=0x7fed68036c50 comp rx=0 tx=0).stop 2023-10-18T08:56:27.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.711+0000 7fed8493e640 1 -- 172.21.15.40:0/1459770812 shutdown_connections 2023-10-18T08:56:27.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.711+0000 7fed8493e640 1 --2- 172.21.15.40:0/1459770812 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fed6006ae40 0x7fed6006d300 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.711+0000 7fed8493e640 1 --2- 172.21.15.40:0/1459770812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed80151bd0 0x7fed8014f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.711+0000 7fed8493e640 1 --2- 172.21.15.40:0/1459770812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed800b0bc0 0x7fed80141460 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.711+0000 7fed8493e640 1 --2- 172.21.15.40:0/1459770812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed8006b710 0x7fed80140aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.711+0000 7fed8493e640 1 -- 172.21.15.40:0/1459770812 >> 172.21.15.40:0/1459770812 conn(0x7fed8006cee0 msgr2=0x7fed8014bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:27.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.712+0000 7fed8493e640 1 -- 172.21.15.40:0/1459770812 shutdown_connections 2023-10-18T08:56:27.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.712+0000 7fed8493e640 1 -- 172.21.15.40:0/1459770812 wait complete. 2023-10-18T08:56:27.730 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-18T08:56:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.951+0000 7f18a65d7640 1 -- 172.21.15.40:0/2543694049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a00b2270 msgr2=0x7f18a00c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.951+0000 7f18a65d7640 1 --2- 172.21.15.40:0/2543694049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a00b2270 0x7f18a00c9f00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f189800afd0 tx=0x7f189802ce20 comp rx=0 tx=0).stop 2023-10-18T08:56:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.951+0000 7f18a65d7640 1 -- 172.21.15.40:0/2543694049 shutdown_connections 2023-10-18T08:56:27.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.951+0000 7f18a65d7640 1 --2- 172.21.15.40:0/2543694049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a00b2270 0x7f18a00c9f00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.951+0000 7f18a65d7640 1 --2- 172.21.15.40:0/2543694049 >> v2:172.21.15.144:3300/0 conn(0x7f18a00b1e10 0x7f18a00c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.951+0000 7f18a65d7640 1 -- 172.21.15.40:0/2543694049 >> 172.21.15.40:0/2543694049 conn(0x7f18a0069720 msgr2=0x7f18a0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:27.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.951+0000 7f18a65d7640 1 -- 172.21.15.40:0/2543694049 shutdown_connections 2023-10-18T08:56:27.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.951+0000 7f18a65d7640 1 -- 172.21.15.40:0/2543694049 wait complete. 2023-10-18T08:56:27.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.953+0000 7f18a65d7640 1 Processor -- start 2023-10-18T08:56:27.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.953+0000 7f18a65d7640 1 -- start start 2023-10-18T08:56:27.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.954+0000 7f18a65d7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a00b15b0 0x7f18a0156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:27.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.954+0000 7f18a65d7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a00b1e10 0x7f18a01e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:27.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.954+0000 7f18a65d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a00b2270 0x7f18a01e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:27.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.954+0000 7f18a65d7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a01e3b50 con 0x7f18a00b1e10 2023-10-18T08:56:27.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.954+0000 7f18a65d7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a01e2c80 con 0x7f18a00b2270 2023-10-18T08:56:27.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.954+0000 7f18a65d7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a01e2ed0 con 0x7f18a00b15b0 2023-10-18T08:56:27.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.954+0000 7f18a4dd4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a00b1e10 0x7f18a01e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:27.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.955+0000 7f18a4dd4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a00b1e10 0x7f18a01e4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59768/0 (socket says 172.21.15.40:59768) 2023-10-18T08:56:27.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.955+0000 7f18a4dd4640 1 -- 172.21.15.40:0/1989862021 learned_addr learned my addr 172.21.15.40:0/1989862021 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:27.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.955+0000 7f18a5dd6640 1 --2- 172.21.15.40:0/1989862021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a00b2270 0x7f18a01e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:27.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.955+0000 7f18a55d5640 1 --2- 172.21.15.40:0/1989862021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a00b15b0 0x7f18a0156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:27.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.955+0000 7f18a4dd4640 1 -- 172.21.15.40:0/1989862021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a00b15b0 msgr2=0x7f18a0156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:27.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.955+0000 7f18a4dd4640 1 --2- 172.21.15.40:0/1989862021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a00b15b0 0x7f18a0156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.955+0000 7f18a4dd4640 1 -- 172.21.15.40:0/1989862021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a00b2270 msgr2=0x7f18a01e4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:27.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.955+0000 7f18a4dd4640 1 --2- 172.21.15.40:0/1989862021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a00b2270 0x7f18a01e4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:27.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.955+0000 7f18a4dd4640 1 -- 172.21.15.40:0/1989862021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1898007490 con 0x7f18a00b1e10 2023-10-18T08:56:27.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.955+0000 7f18a4dd4640 1 --2- 172.21.15.40:0/1989862021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a00b1e10 0x7f18a01e4a30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f189000e9f0 tx=0x7f189000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:27.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.956+0000 7f1895ffb640 1 -- 172.21.15.40:0/1989862021 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890009700 con 0x7f18a00b1e10 2023-10-18T08:56:27.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.956+0000 7f1895ffb640 1 -- 172.21.15.40:0/1989862021 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f189000ce70 con 0x7f18a00b1e10 2023-10-18T08:56:27.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.956+0000 7f1895ffb640 1 -- 172.21.15.40:0/1989862021 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890010670 con 0x7f18a00b1e10 2023-10-18T08:56:27.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.956+0000 7f18a65d7640 1 -- 172.21.15.40:0/1989862021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a01e31b0 con 0x7f18a00b1e10 2023-10-18T08:56:27.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.956+0000 7f18a65d7640 1 -- 172.21.15.40:0/1989862021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a01e3770 con 0x7f18a00b1e10 2023-10-18T08:56:27.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.957+0000 7f18a65d7640 1 -- 172.21.15.40:0/1989862021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a01e3dc0 con 0x7f18a00b1e10 2023-10-18T08:56:27.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.959+0000 7f1895ffb640 1 -- 172.21.15.40:0/1989862021 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f18900107d0 con 0x7f18a00b1e10 2023-10-18T08:56:27.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.960+0000 7f1895ffb640 1 --2- 172.21.15.40:0/1989862021 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f188406ae40 0x7f188406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:27.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.960+0000 7f1895ffb640 1 -- 172.21.15.40:0/1989862021 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1890014070 con 0x7f18a00b1e10 2023-10-18T08:56:27.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.961+0000 7f18a55d5640 1 --2- 172.21.15.40:0/1989862021 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f188406ae40 0x7f188406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:27.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.962+0000 7f18a55d5640 1 --2- 172.21.15.40:0/1989862021 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f188406ae40 0x7f188406d300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f189c000f60 tx=0x7f189c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:27.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:27.964+0000 7f1895ffb640 1 -- 172.21.15.40:0/1989862021 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1890051ca0 con 0x7f18a00b1e10 2023-10-18T08:56:28.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.230+0000 7f18a65d7640 1 -- 172.21.15.40:0/1989862021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f18a0066e00 con 0x7f18a00b1e10 2023-10-18T08:56:28.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.234+0000 7f1895ffb640 1 -- 172.21.15.40:0/1989862021 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1890058e40 con 0x7f18a00b1e10 2023-10-18T08:56:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.237+0000 7f18a65d7640 1 -- 172.21.15.40:0/1989862021 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f188406ae40 msgr2=0x7f188406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.237+0000 7f18a65d7640 1 --2- 172.21.15.40:0/1989862021 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f188406ae40 0x7f188406d300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f189c000f60 tx=0x7f189c009040 comp rx=0 tx=0).stop 2023-10-18T08:56:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.237+0000 7f18a65d7640 1 -- 172.21.15.40:0/1989862021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a00b1e10 msgr2=0x7f18a01e4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.237+0000 7f18a65d7640 1 --2- 172.21.15.40:0/1989862021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a00b1e10 0x7f18a01e4a30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f189000e9f0 tx=0x7f189000eeb0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.237+0000 7f18a65d7640 1 -- 172.21.15.40:0/1989862021 shutdown_connections 2023-10-18T08:56:28.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.237+0000 7f18a65d7640 1 --2- 172.21.15.40:0/1989862021 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f188406ae40 0x7f188406d300 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.237+0000 7f18a65d7640 1 --2- 172.21.15.40:0/1989862021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a00b2270 0x7f18a01e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.237+0000 7f18a65d7640 1 --2- 172.21.15.40:0/1989862021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a00b1e10 0x7f18a01e4a30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.237+0000 7f18a65d7640 1 --2- 172.21.15.40:0/1989862021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18a00b15b0 0x7f18a0156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.237+0000 7f18a65d7640 1 -- 172.21.15.40:0/1989862021 >> 172.21.15.40:0/1989862021 conn(0x7f18a0069720 msgr2=0x7f18a0158990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:28.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.238+0000 7f18a65d7640 1 -- 172.21.15.40:0/1989862021 shutdown_connections 2023-10-18T08:56:28.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.238+0000 7f18a65d7640 1 -- 172.21.15.40:0/1989862021 wait complete. 2023-10-18T08:56:28.259 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:28.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.487+0000 7fe72fc44640 1 -- 172.21.15.40:0/3542871727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714004a80 msgr2=0x7fe714004e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:28.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.487+0000 7fe72fc44640 1 --2- 172.21.15.40:0/3542871727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714004a80 0x7fe714004e80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe72400c6e0 tx=0x7fe714006900 comp rx=0 tx=0).stop 2023-10-18T08:56:28.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.487+0000 7fe72fc44640 1 -- 172.21.15.40:0/3542871727 shutdown_connections 2023-10-18T08:56:28.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.487+0000 7fe72fc44640 1 --2- 172.21.15.40:0/3542871727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714004a80 0x7fe714004e80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.487+0000 7fe72fc44640 1 -- 172.21.15.40:0/3542871727 >> 172.21.15.40:0/3542871727 conn(0x7fe72806cee0 msgr2=0x7fe728069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:28.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.487+0000 7fe72fc44640 1 -- 172.21.15.40:0/3542871727 shutdown_connections 2023-10-18T08:56:28.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.487+0000 7fe72fc44640 1 -- 172.21.15.40:0/3542871727 wait complete. 2023-10-18T08:56:28.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.489+0000 7fe72fc44640 1 Processor -- start 2023-10-18T08:56:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.489+0000 7fe72fc44640 1 -- start start 2023-10-18T08:56:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.490+0000 7fe72fc44640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714004a80 0x7fe728146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:28.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.490+0000 7fe72fc44640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7280cba70 0x7fe7280cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:28.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.490+0000 7fe72fc44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7281496e0 0x7fe728149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:28.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.490+0000 7fe72fc44640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7280cc430 con 0x7fe714004a80 2023-10-18T08:56:28.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.490+0000 7fe72fc44640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72814a960 con 0x7fe7281496e0 2023-10-18T08:56:28.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.490+0000 7fe72fc44640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72814a3a0 con 0x7fe7280cba70 2023-10-18T08:56:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.490+0000 7fe72d9e0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714004a80 0x7fe728146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.491+0000 7fe72d9e0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714004a80 0x7fe728146c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59784/0 (socket says 172.21.15.40:59784) 2023-10-18T08:56:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.491+0000 7fe72d9e0640 1 -- 172.21.15.40:0/3716577416 learned_addr learned my addr 172.21.15.40:0/3716577416 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.491+0000 7fe72e1e1640 1 --2- 172.21.15.40:0/3716577416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7281496e0 0x7fe728149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:28.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.491+0000 7fe72d1df640 1 --2- 172.21.15.40:0/3716577416 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7280cba70 0x7fe7280cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:28.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.491+0000 7fe72d9e0640 1 -- 172.21.15.40:0/3716577416 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7280cba70 msgr2=0x7fe7280cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:28.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.491+0000 7fe72d9e0640 1 --2- 172.21.15.40:0/3716577416 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7280cba70 0x7fe7280cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.491+0000 7fe72d9e0640 1 -- 172.21.15.40:0/3716577416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7281496e0 msgr2=0x7fe728149b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:28.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.491+0000 7fe72d9e0640 1 --2- 172.21.15.40:0/3716577416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7281496e0 0x7fe728149b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.491+0000 7fe72d9e0640 1 -- 172.21.15.40:0/3716577416 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72814a680 con 0x7fe714004a80 2023-10-18T08:56:28.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.491+0000 7fe72d1df640 1 --2- 172.21.15.40:0/3716577416 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7280cba70 0x7fe7280cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:28.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.491+0000 7fe72e1e1640 1 --2- 172.21.15.40:0/3716577416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7281496e0 0x7fe728149b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:28.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.491+0000 7fe72d9e0640 1 --2- 172.21.15.40:0/3716577416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714004a80 0x7fe728146c00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe72400c800 tx=0x7fe724004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:28.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.492+0000 7fe71e7fc640 1 -- 172.21.15.40:0/3716577416 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72403d070 con 0x7fe714004a80 2023-10-18T08:56:28.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.492+0000 7fe71e7fc640 1 -- 172.21.15.40:0/3716577416 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe724006260 con 0x7fe714004a80 2023-10-18T08:56:28.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.492+0000 7fe71e7fc640 1 -- 172.21.15.40:0/3716577416 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe724037690 con 0x7fe714004a80 2023-10-18T08:56:28.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.492+0000 7fe72fc44640 1 -- 172.21.15.40:0/3716577416 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72814ae20 con 0x7fe714004a80 2023-10-18T08:56:28.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.492+0000 7fe72fc44640 1 -- 172.21.15.40:0/3716577416 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72814a080 con 0x7fe714004a80 2023-10-18T08:56:28.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.493+0000 7fe72fc44640 1 -- 172.21.15.40:0/3716577416 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe728066fd0 con 0x7fe714004a80 2023-10-18T08:56:28.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.494+0000 7fe71e7fc640 1 -- 172.21.15.40:0/3716577416 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe7240377f0 con 0x7fe714004a80 2023-10-18T08:56:28.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.495+0000 7fe71e7fc640 1 --2- 172.21.15.40:0/3716577416 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe70c06ae40 0x7fe70c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:28.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.495+0000 7fe71e7fc640 1 -- 172.21.15.40:0/3716577416 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe7240aac70 con 0x7fe714004a80 2023-10-18T08:56:28.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.496+0000 7fe72d1df640 1 --2- 172.21.15.40:0/3716577416 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe70c06ae40 0x7fe70c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:28.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.497+0000 7fe72d1df640 1 --2- 172.21.15.40:0/3716577416 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe70c06ae40 0x7fe70c06d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe7281ee470 tx=0x7fe718001560 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:28.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.499+0000 7fe71e7fc640 1 -- 172.21.15.40:0/3716577416 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe724076180 con 0x7fe714004a80 2023-10-18T08:56:28.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.698+0000 7fe72fc44640 1 -- 172.21.15.40:0/3716577416 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7280cc430 con 0x7fe714004a80 2023-10-18T08:56:28.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.700+0000 7fe71e7fc640 1 -- 172.21.15.40:0/3716577416 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12869 (secure 0 0 0) 0x7fe7280cc430 con 0x7fe714004a80 2023-10-18T08:56:28.703 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:28.703 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":29,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:24.444632+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:28.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.703+0000 7fe72fc44640 1 -- 172.21.15.40:0/3716577416 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe70c06ae40 msgr2=0x7fe70c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:28.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.703+0000 7fe72fc44640 1 --2- 172.21.15.40:0/3716577416 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe70c06ae40 0x7fe70c06d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe7281ee470 tx=0x7fe718001560 comp rx=0 tx=0).stop 2023-10-18T08:56:28.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.703+0000 7fe72fc44640 1 -- 172.21.15.40:0/3716577416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714004a80 msgr2=0x7fe728146c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:28.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.703+0000 7fe72fc44640 1 --2- 172.21.15.40:0/3716577416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714004a80 0x7fe728146c00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe72400c800 tx=0x7fe724004df0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.703+0000 7fe72fc44640 1 -- 172.21.15.40:0/3716577416 shutdown_connections 2023-10-18T08:56:28.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.703+0000 7fe72fc44640 1 --2- 172.21.15.40:0/3716577416 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe70c06ae40 0x7fe70c06d300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.703+0000 7fe72fc44640 1 --2- 172.21.15.40:0/3716577416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7281496e0 0x7fe728149b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.703+0000 7fe72fc44640 1 --2- 172.21.15.40:0/3716577416 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7280cba70 0x7fe7280cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.703+0000 7fe72fc44640 1 --2- 172.21.15.40:0/3716577416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe714004a80 0x7fe728146c00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.703+0000 7fe72fc44640 1 -- 172.21.15.40:0/3716577416 >> 172.21.15.40:0/3716577416 conn(0x7fe72806cee0 msgr2=0x7fe72806fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:28.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.703+0000 7fe72fc44640 1 -- 172.21.15.40:0/3716577416 shutdown_connections 2023-10-18T08:56:28.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.703+0000 7fe72fc44640 1 -- 172.21.15.40:0/3716577416 wait complete. 2023-10-18T08:56:28.726 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:56:28.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:56:28.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.959+0000 7f4d6f6e5640 1 -- 172.21.15.40:0/3019475591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d6806d6f0 msgr2=0x7f4d6814e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:28.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.959+0000 7f4d6f6e5640 1 --2- 172.21.15.40:0/3019475591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d6806d6f0 0x7f4d6814e730 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4d68071d20 tx=0x7f4d6402be20 comp rx=0 tx=0).stop 2023-10-18T08:56:28.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.959+0000 7f4d6f6e5640 1 -- 172.21.15.40:0/3019475591 shutdown_connections 2023-10-18T08:56:28.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.959+0000 7f4d6f6e5640 1 --2- 172.21.15.40:0/3019475591 >> v2:172.21.15.144:3300/0 conn(0x7f4d680715c0 0x7f4d680b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.959+0000 7f4d6f6e5640 1 --2- 172.21.15.40:0/3019475591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d6806d6f0 0x7f4d6814e730 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.959+0000 7f4d6f6e5640 1 -- 172.21.15.40:0/3019475591 >> 172.21.15.40:0/3019475591 conn(0x7f4d68069720 msgr2=0x7f4d68069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:28.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.959+0000 7f4d6f6e5640 1 -- 172.21.15.40:0/3019475591 shutdown_connections 2023-10-18T08:56:28.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.959+0000 7f4d6f6e5640 1 -- 172.21.15.40:0/3019475591 wait complete. 2023-10-18T08:56:28.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.961+0000 7f4d6f6e5640 1 Processor -- start 2023-10-18T08:56:28.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.961+0000 7f4d6f6e5640 1 -- start start 2023-10-18T08:56:28.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.962+0000 7f4d6f6e5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d6806d6f0 0x7f4d68142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:28.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.962+0000 7f4d6f6e5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d680715c0 0x7f4d681f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:28.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.962+0000 7f4d6f6e5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680aed00 0x7f4d6806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:28.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.962+0000 7f4d6f6e5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d68143420 con 0x7f4d680aed00 2023-10-18T08:56:28.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.962+0000 7f4d6f6e5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d681f2920 con 0x7f4d680715c0 2023-10-18T08:56:28.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.962+0000 7f4d6f6e5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d68147da0 con 0x7f4d6806d6f0 2023-10-18T08:56:28.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.962+0000 7f4d6dc82640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680aed00 0x7f4d6806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:28.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.962+0000 7f4d6dc82640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680aed00 0x7f4d6806b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59798/0 (socket says 172.21.15.40:59798) 2023-10-18T08:56:28.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.962+0000 7f4d6dc82640 1 -- 172.21.15.40:0/518703526 learned_addr learned my addr 172.21.15.40:0/518703526 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:28.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.963+0000 7f4d6d481640 1 --2- 172.21.15.40:0/518703526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d6806d6f0 0x7f4d68142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:28.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.963+0000 7f4d6cc80640 1 --2- 172.21.15.40:0/518703526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d680715c0 0x7f4d681f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:28.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.963+0000 7f4d6dc82640 1 -- 172.21.15.40:0/518703526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d6806d6f0 msgr2=0x7f4d68142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:28.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.963+0000 7f4d6dc82640 1 --2- 172.21.15.40:0/518703526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d6806d6f0 0x7f4d68142ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.963+0000 7f4d6dc82640 1 -- 172.21.15.40:0/518703526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d680715c0 msgr2=0x7f4d681f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:28.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.963+0000 7f4d6dc82640 1 --2- 172.21.15.40:0/518703526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d680715c0 0x7f4d681f23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:28.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.963+0000 7f4d6dc82640 1 -- 172.21.15.40:0/518703526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d64009560 con 0x7f4d680aed00 2023-10-18T08:56:28.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.963+0000 7f4d6dc82640 1 --2- 172.21.15.40:0/518703526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680aed00 0x7f4d6806b7b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000c710 tx=0x7f4d6000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:28.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.963+0000 7f4d59ffb640 1 -- 172.21.15.40:0/518703526 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60003da0 con 0x7f4d680aed00 2023-10-18T08:56:28.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.963+0000 7f4d59ffb640 1 -- 172.21.15.40:0/518703526 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4d6000ce20 con 0x7f4d680aed00 2023-10-18T08:56:28.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.963+0000 7f4d59ffb640 1 -- 172.21.15.40:0/518703526 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d600052d0 con 0x7f4d680aed00 2023-10-18T08:56:28.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.963+0000 7f4d6f6e5640 1 -- 172.21.15.40:0/518703526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d68148060 con 0x7f4d680aed00 2023-10-18T08:56:28.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.964+0000 7f4d6f6e5640 1 -- 172.21.15.40:0/518703526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d68148620 con 0x7f4d680aed00 2023-10-18T08:56:28.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.965+0000 7f4d6f6e5640 1 -- 172.21.15.40:0/518703526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d68066fd0 con 0x7f4d680aed00 2023-10-18T08:56:28.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.966+0000 7f4d59ffb640 1 -- 172.21.15.40:0/518703526 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4d60005430 con 0x7f4d680aed00 2023-10-18T08:56:28.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.967+0000 7f4d59ffb640 1 --2- 172.21.15.40:0/518703526 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4d4c06ae40 0x7f4d4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:28.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.967+0000 7f4d59ffb640 1 -- 172.21.15.40:0/518703526 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4d60088e10 con 0x7f4d680aed00 2023-10-18T08:56:28.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.968+0000 7f4d6d481640 1 --2- 172.21.15.40:0/518703526 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4d4c06ae40 0x7f4d4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:28.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.969+0000 7f4d6d481640 1 --2- 172.21.15.40:0/518703526 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4d4c06ae40 0x7f4d4c06d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4d64035000 tx=0x7f4d64004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:28.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:28.972+0000 7f4d59ffb640 1 -- 172.21.15.40:0/518703526 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d6008c030 con 0x7f4d680aed00 2023-10-18T08:56:29.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.164+0000 7f4d6f6e5640 1 -- 172.21.15.40:0/518703526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4d680c9d60 con 0x7f4d680aed00 2023-10-18T08:56:29.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.165+0000 7f4d59ffb640 1 -- 172.21.15.40:0/518703526 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4d60017020 con 0x7f4d680aed00 2023-10-18T08:56:29.167 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:56:29.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.167+0000 7f4d6f6e5640 1 -- 172.21.15.40:0/518703526 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4d4c06ae40 msgr2=0x7f4d4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:29.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.167+0000 7f4d6f6e5640 1 --2- 172.21.15.40:0/518703526 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4d4c06ae40 0x7f4d4c06d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4d64035000 tx=0x7f4d64004730 comp rx=0 tx=0).stop 2023-10-18T08:56:29.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.167+0000 7f4d6f6e5640 1 -- 172.21.15.40:0/518703526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680aed00 msgr2=0x7f4d6806b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:29.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.167+0000 7f4d6f6e5640 1 --2- 172.21.15.40:0/518703526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680aed00 0x7f4d6806b7b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000c710 tx=0x7f4d6000cbd0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.167+0000 7f4d6f6e5640 1 -- 172.21.15.40:0/518703526 shutdown_connections 2023-10-18T08:56:29.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.168+0000 7f4d6f6e5640 1 --2- 172.21.15.40:0/518703526 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4d4c06ae40 0x7f4d4c06d300 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.168+0000 7f4d6f6e5640 1 --2- 172.21.15.40:0/518703526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d680aed00 0x7f4d6806b7b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.168+0000 7f4d6f6e5640 1 --2- 172.21.15.40:0/518703526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d680715c0 0x7f4d681f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.168+0000 7f4d6f6e5640 1 --2- 172.21.15.40:0/518703526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d6806d6f0 0x7f4d68142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.168+0000 7f4d6f6e5640 1 -- 172.21.15.40:0/518703526 >> 172.21.15.40:0/518703526 conn(0x7f4d68069720 msgr2=0x7f4d6814dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:29.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.168+0000 7f4d6f6e5640 1 -- 172.21.15.40:0/518703526 shutdown_connections 2023-10-18T08:56:29.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.168+0000 7f4d6f6e5640 1 -- 172.21.15.40:0/518703526 wait complete. 2023-10-18T08:56:29.190 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T08:56:29.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.414+0000 7fcffed6e640 1 -- 172.21.15.40:0/2459902533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff815b6b0 msgr2=0x7fcff806be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:29.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.414+0000 7fcffed6e640 1 --2- 172.21.15.40:0/2459902533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff815b6b0 0x7fcff806be40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009830 tx=0x7fcfe802bd50 comp rx=0 tx=0).stop 2023-10-18T08:56:29.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.414+0000 7fcffed6e640 1 -- 172.21.15.40:0/2459902533 shutdown_connections 2023-10-18T08:56:29.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.414+0000 7fcffed6e640 1 --2- 172.21.15.40:0/2459902533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff815b6b0 0x7fcff806be40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.414+0000 7fcffed6e640 1 --2- 172.21.15.40:0/2459902533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcff815b2d0 0x7fcff80b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.414+0000 7fcffed6e640 1 -- 172.21.15.40:0/2459902533 >> 172.21.15.40:0/2459902533 conn(0x7fcff8069b40 msgr2=0x7fcff8069f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:29.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.414+0000 7fcffed6e640 1 -- 172.21.15.40:0/2459902533 shutdown_connections 2023-10-18T08:56:29.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.414+0000 7fcffed6e640 1 -- 172.21.15.40:0/2459902533 wait complete. 2023-10-18T08:56:29.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.416+0000 7fcffed6e640 1 Processor -- start 2023-10-18T08:56:29.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.416+0000 7fcffed6e640 1 -- start start 2023-10-18T08:56:29.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.417+0000 7fcffed6e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff80b07c0 0x7fcff80ad430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:29.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.417+0000 7fcffed6e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff815b2d0 0x7fcff8159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:29.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.417+0000 7fcffed6e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcff815b6b0 0x7fcff815a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:29.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.417+0000 7fcffed6e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff80aee80 con 0x7fcff815b2d0 2023-10-18T08:56:29.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.417+0000 7fcffed6e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff80adcc0 con 0x7fcff80b07c0 2023-10-18T08:56:29.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.417+0000 7fcffed6e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff80adfa0 con 0x7fcff815b6b0 2023-10-18T08:56:29.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.417+0000 7fcffd56b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff815b2d0 0x7fcff8159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:29.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.418+0000 7fcffd56b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff815b2d0 0x7fcff8159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59812/0 (socket says 172.21.15.40:59812) 2023-10-18T08:56:29.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.418+0000 7fcffd56b640 1 -- 172.21.15.40:0/1035541149 learned_addr learned my addr 172.21.15.40:0/1035541149 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.418+0000 7fcffd56b640 1 -- 172.21.15.40:0/1035541149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcff815b6b0 msgr2=0x7fcff815a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.418+0000 7fcffd56b640 1 --2- 172.21.15.40:0/1035541149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcff815b6b0 0x7fcff815a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.418+0000 7fcffdd6c640 1 --2- 172.21.15.40:0/1035541149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff80b07c0 0x7fcff80ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:29.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.418+0000 7fcffd56b640 1 -- 172.21.15.40:0/1035541149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff80b07c0 msgr2=0x7fcff80ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:29.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.418+0000 7fcffd56b640 1 --2- 172.21.15.40:0/1035541149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff80b07c0 0x7fcff80ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.418+0000 7fcffd56b640 1 -- 172.21.15.40:0/1035541149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe8009490 con 0x7fcff815b2d0 2023-10-18T08:56:29.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.418+0000 7fcffdd6c640 1 --2- 172.21.15.40:0/1035541149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff80b07c0 0x7fcff80ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:29.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.418+0000 7fcffd56b640 1 --2- 172.21.15.40:0/1035541149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff815b2d0 0x7fcff8159d70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009780 tx=0x7fcfe80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:29.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.419+0000 7fcfee7fc640 1 -- 172.21.15.40:0/1035541149 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8003ea0 con 0x7fcff815b2d0 2023-10-18T08:56:29.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.419+0000 7fcfee7fc640 1 -- 172.21.15.40:0/1035541149 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcfe8034550 con 0x7fcff815b2d0 2023-10-18T08:56:29.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.419+0000 7fcfee7fc640 1 -- 172.21.15.40:0/1035541149 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8034960 con 0x7fcff815b2d0 2023-10-18T08:56:29.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.419+0000 7fcffed6e640 1 -- 172.21.15.40:0/1035541149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff80ae280 con 0x7fcff815b2d0 2023-10-18T08:56:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.419+0000 7fcffed6e640 1 -- 172.21.15.40:0/1035541149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff80ad970 con 0x7fcff815b2d0 2023-10-18T08:56:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.420+0000 7fcffed6e640 1 -- 172.21.15.40:0/1035541149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff80ae7a0 con 0x7fcff815b2d0 2023-10-18T08:56:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.421+0000 7fcfee7fc640 1 -- 172.21.15.40:0/1035541149 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcfe803b040 con 0x7fcff815b2d0 2023-10-18T08:56:29.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.422+0000 7fcfee7fc640 1 --2- 172.21.15.40:0/1035541149 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfdc06ae40 0x7fcfdc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:29.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.422+0000 7fcfee7fc640 1 -- 172.21.15.40:0/1035541149 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcfe80ab910 con 0x7fcff815b2d0 2023-10-18T08:56:29.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.423+0000 7fcffdd6c640 1 --2- 172.21.15.40:0/1035541149 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfdc06ae40 0x7fcfdc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:29.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.424+0000 7fcffdd6c640 1 --2- 172.21.15.40:0/1035541149 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfdc06ae40 0x7fcfdc06d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009b40 tx=0x7fcff4005e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:29.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.426+0000 7fcfee7fc640 1 -- 172.21.15.40:0/1035541149 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcfe8076c70 con 0x7fcff815b2d0 2023-10-18T08:56:29.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.620+0000 7fcffed6e640 1 -- 172.21.15.40:0/1035541149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcff8066e00 con 0x7fcff815b2d0 2023-10-18T08:56:29.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.621+0000 7fcfee7fc640 1 -- 172.21.15.40:0/1035541149 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcfe8039020 con 0x7fcff815b2d0 2023-10-18T08:56:29.623 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:29.623 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-10-18T08:56:29.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:29.624 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-10-18T08:56:29.624 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-10-18T08:56:29.624 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:29.624 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-10-18T08:56:29.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.623+0000 7fcffed6e640 1 -- 172.21.15.40:0/1035541149 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfdc06ae40 msgr2=0x7fcfdc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:29.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.623+0000 7fcffed6e640 1 --2- 172.21.15.40:0/1035541149 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfdc06ae40 0x7fcfdc06d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009b40 tx=0x7fcff4005e50 comp rx=0 tx=0).stop 2023-10-18T08:56:29.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.624+0000 7fcffed6e640 1 -- 172.21.15.40:0/1035541149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff815b2d0 msgr2=0x7fcff8159d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:29.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.624+0000 7fcffed6e640 1 --2- 172.21.15.40:0/1035541149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff815b2d0 0x7fcff8159d70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8009780 tx=0x7fcfe80038a0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.624+0000 7fcffed6e640 1 -- 172.21.15.40:0/1035541149 shutdown_connections 2023-10-18T08:56:29.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.624+0000 7fcffed6e640 1 --2- 172.21.15.40:0/1035541149 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfdc06ae40 0x7fcfdc06d300 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.624+0000 7fcffed6e640 1 --2- 172.21.15.40:0/1035541149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcff815b6b0 0x7fcff815a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.624+0000 7fcffed6e640 1 --2- 172.21.15.40:0/1035541149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff815b2d0 0x7fcff8159d70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.624+0000 7fcffed6e640 1 --2- 172.21.15.40:0/1035541149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff80b07c0 0x7fcff80ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.624+0000 7fcffed6e640 1 -- 172.21.15.40:0/1035541149 >> 172.21.15.40:0/1035541149 conn(0x7fcff8069b40 msgr2=0x7fcff81583a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:29.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.624+0000 7fcffed6e640 1 -- 172.21.15.40:0/1035541149 shutdown_connections 2023-10-18T08:56:29.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.624+0000 7fcffed6e640 1 -- 172.21.15.40:0/1035541149 wait complete. 2023-10-18T08:56:29.645 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T08:56:29.646 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T08:56:29.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.867+0000 7f2f7b1ed640 1 -- 172.21.15.40:0/998817100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f7406d570 msgr2=0x7f2f74157320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:29.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.867+0000 7f2f7b1ed640 1 --2- 172.21.15.40:0/998817100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f7406d570 0x7f2f74157320 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2f7406d950 tx=0x7f2f7002c1c0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.867+0000 7f2f7b1ed640 1 -- 172.21.15.40:0/998817100 shutdown_connections 2023-10-18T08:56:29.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.867+0000 7f2f7b1ed640 1 --2- 172.21.15.40:0/998817100 >> v2:172.21.15.144:3300/0 conn(0x7f2f7406f480 0x7f2f7406f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.867+0000 7f2f7b1ed640 1 --2- 172.21.15.40:0/998817100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f7406d570 0x7f2f74157320 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.867+0000 7f2f7b1ed640 1 -- 172.21.15.40:0/998817100 >> 172.21.15.40:0/998817100 conn(0x7f2f7406b7d0 msgr2=0x7f2f7406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:29.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.867+0000 7f2f7b1ed640 1 -- 172.21.15.40:0/998817100 shutdown_connections 2023-10-18T08:56:29.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.867+0000 7f2f7b1ed640 1 -- 172.21.15.40:0/998817100 wait complete. 2023-10-18T08:56:29.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.869+0000 7f2f7b1ed640 1 Processor -- start 2023-10-18T08:56:29.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.869+0000 7f2f7b1ed640 1 -- start start 2023-10-18T08:56:29.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.871+0000 7f2f7b1ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f7406d570 0x7f2f741529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:29.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.871+0000 7f2f7b1ed640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7406f480 0x7f2f7415aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:29.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.871+0000 7f2f7b1ed640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f740b2a20 0x7f2f7415b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:29.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.871+0000 7f2f7b1ed640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f74159610 con 0x7f2f7406d570 2023-10-18T08:56:29.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.871+0000 7f2f7b1ed640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f741598f0 con 0x7f2f740b2a20 2023-10-18T08:56:29.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.871+0000 7f2f7b1ed640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f74159bd0 con 0x7f2f7406f480 2023-10-18T08:56:29.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.871+0000 7f2f7a1eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f7406d570 0x7f2f741529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:29.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.871+0000 7f2f7a1eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f7406d570 0x7f2f741529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59832/0 (socket says 172.21.15.40:59832) 2023-10-18T08:56:29.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.871+0000 7f2f7a1eb640 1 -- 172.21.15.40:0/259974972 learned_addr learned my addr 172.21.15.40:0/259974972 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.872+0000 7f2f7a1eb640 1 -- 172.21.15.40:0/259974972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7406f480 msgr2=0x7f2f7415aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.871+0000 7f2f7a9ec640 1 --2- 172.21.15.40:0/259974972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f740b2a20 0x7f2f7415b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.872+0000 7f2f799ea640 1 --2- 172.21.15.40:0/259974972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7406f480 0x7f2f7415aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:29.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.872+0000 7f2f7a1eb640 1 --2- 172.21.15.40:0/259974972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7406f480 0x7f2f7415aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.872+0000 7f2f7a1eb640 1 -- 172.21.15.40:0/259974972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f740b2a20 msgr2=0x7f2f7415b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:29.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.872+0000 7f2f7a1eb640 1 --2- 172.21.15.40:0/259974972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f740b2a20 0x7f2f7415b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:29.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.872+0000 7f2f7a1eb640 1 -- 172.21.15.40:0/259974972 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f70009900 con 0x7f2f7406d570 2023-10-18T08:56:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.872+0000 7f2f799ea640 1 --2- 172.21.15.40:0/259974972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7406f480 0x7f2f7415aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.872+0000 7f2f7a9ec640 1 --2- 172.21.15.40:0/259974972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f740b2a20 0x7f2f7415b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.872+0000 7f2f7a1eb640 1 --2- 172.21.15.40:0/259974972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f7406d570 0x7f2f741529e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2f70006010 tx=0x7f2f70035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.872+0000 7f2f6affd640 1 -- 172.21.15.40:0/259974972 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f70034440 con 0x7f2f7406d570 2023-10-18T08:56:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.872+0000 7f2f6affd640 1 -- 172.21.15.40:0/259974972 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2f70004540 con 0x7f2f7406d570 2023-10-18T08:56:29.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.872+0000 7f2f7b1ed640 1 -- 172.21.15.40:0/259974972 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f74159eb0 con 0x7f2f7406d570 2023-10-18T08:56:29.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.872+0000 7f2f6affd640 1 -- 172.21.15.40:0/259974972 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7003f440 con 0x7f2f7406d570 2023-10-18T08:56:29.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.873+0000 7f2f7b1ed640 1 -- 172.21.15.40:0/259974972 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f74152f20 con 0x7f2f7406d570 2023-10-18T08:56:29.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.874+0000 7f2f7b1ed640 1 -- 172.21.15.40:0/259974972 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f74066ea0 con 0x7f2f7406d570 2023-10-18T08:56:29.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.875+0000 7f2f6affd640 1 -- 172.21.15.40:0/259974972 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2f7003b040 con 0x7f2f7406d570 2023-10-18T08:56:29.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.875+0000 7f2f6affd640 1 --2- 172.21.15.40:0/259974972 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2f5806ae70 0x7f2f5806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:29.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.876+0000 7f2f6affd640 1 -- 172.21.15.40:0/259974972 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2f700ac020 con 0x7f2f7406d570 2023-10-18T08:56:29.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.876+0000 7f2f799ea640 1 --2- 172.21.15.40:0/259974972 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2f5806ae70 0x7f2f5806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:29.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.877+0000 7f2f799ea640 1 --2- 172.21.15.40:0/259974972 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2f5806ae70 0x7f2f5806d330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400a830 tx=0x7f2f64008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:29.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:29.880+0000 7f2f6affd640 1 -- 172.21.15.40:0/259974972 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2f70077420 con 0x7f2f7406d570 2023-10-18T08:56:30.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.073+0000 7f2f7b1ed640 1 -- 172.21.15.40:0/259974972 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2f7406fe10 con 0x7f2f7406d570 2023-10-18T08:56:30.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.693+0000 7f2f6affd640 1 -- 172.21.15.40:0/259974972 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2f7406fe10 con 0x7f2f7406d570 2023-10-18T08:56:30.696 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-10-18T08:56:30.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.697+0000 7f2f7b1ed640 1 -- 172.21.15.40:0/259974972 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2f5806ae70 msgr2=0x7f2f5806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:30.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.697+0000 7f2f7b1ed640 1 --2- 172.21.15.40:0/259974972 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2f5806ae70 0x7f2f5806d330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400a830 tx=0x7f2f64008040 comp rx=0 tx=0).stop 2023-10-18T08:56:30.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.697+0000 7f2f7b1ed640 1 -- 172.21.15.40:0/259974972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f7406d570 msgr2=0x7f2f741529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:30.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.697+0000 7f2f7b1ed640 1 --2- 172.21.15.40:0/259974972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f7406d570 0x7f2f741529e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2f70006010 tx=0x7f2f70035e30 comp rx=0 tx=0).stop 2023-10-18T08:56:30.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.697+0000 7f2f7b1ed640 1 -- 172.21.15.40:0/259974972 shutdown_connections 2023-10-18T08:56:30.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.697+0000 7f2f7b1ed640 1 --2- 172.21.15.40:0/259974972 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2f5806ae70 0x7f2f5806d330 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:30.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.697+0000 7f2f7b1ed640 1 --2- 172.21.15.40:0/259974972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f740b2a20 0x7f2f7415b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:30.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.697+0000 7f2f7b1ed640 1 --2- 172.21.15.40:0/259974972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f7406f480 0x7f2f7415aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:30.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.697+0000 7f2f7b1ed640 1 --2- 172.21.15.40:0/259974972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f7406d570 0x7f2f741529e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:30.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.697+0000 7f2f7b1ed640 1 -- 172.21.15.40:0/259974972 >> 172.21.15.40:0/259974972 conn(0x7f2f7406b7d0 msgr2=0x7f2f7406cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:30.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.698+0000 7f2f7b1ed640 1 -- 172.21.15.40:0/259974972 shutdown_connections 2023-10-18T08:56:30.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.698+0000 7f2f7b1ed640 1 -- 172.21.15.40:0/259974972 wait complete. 2023-10-18T08:56:30.722 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T08:56:30.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.961+0000 7f8013d39640 1 -- 172.21.15.40:0/614649873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800c13f190 msgr2=0x7f800c070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:30.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.961+0000 7f8013d39640 1 --2- 172.21.15.40:0/614649873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800c13f190 0x7f800c070d00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f800c06a5c0 tx=0x7f7ffc02be40 comp rx=0 tx=0).stop 2023-10-18T08:56:30.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.961+0000 7f8013d39640 1 -- 172.21.15.40:0/614649873 shutdown_connections 2023-10-18T08:56:30.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.961+0000 7f8013d39640 1 --2- 172.21.15.40:0/614649873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800c13f190 0x7f800c070d00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:30.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.961+0000 7f8013d39640 1 --2- 172.21.15.40:0/614649873 >> v2:172.21.15.144:3300/0 conn(0x7f800c13c550 0x7f800c13ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:30.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.961+0000 7f8013d39640 1 -- 172.21.15.40:0/614649873 >> 172.21.15.40:0/614649873 conn(0x7f800c069720 msgr2=0x7f800c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:30.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.962+0000 7f8013d39640 1 -- 172.21.15.40:0/614649873 shutdown_connections 2023-10-18T08:56:30.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.962+0000 7f8013d39640 1 -- 172.21.15.40:0/614649873 wait complete. 2023-10-18T08:56:30.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.963+0000 7f8013d39640 1 Processor -- start 2023-10-18T08:56:30.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.963+0000 7f8013d39640 1 -- start start 2023-10-18T08:56:30.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.964+0000 7f8013d39640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f800c071240 0x7f800c152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:30.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.964+0000 7f8013d39640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f800c13c550 0x7f800c0aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.964+0000 7f8013d39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800c13f190 0x7f800c0af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.964+0000 7f8013d39640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c0b1290 con 0x7f800c13f190 2023-10-18T08:56:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.964+0000 7f8013d39640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c0b1570 con 0x7f800c13c550 2023-10-18T08:56:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.964+0000 7f8013d39640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c0b0390 con 0x7f800c071240 2023-10-18T08:56:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.965+0000 7f80122d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800c13f190 0x7f800c0af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:30.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.965+0000 7f80122d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800c13f190 0x7f800c0af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59838/0 (socket says 172.21.15.40:59838) 2023-10-18T08:56:30.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.965+0000 7f80122d6640 1 -- 172.21.15.40:0/1078964652 learned_addr learned my addr 172.21.15.40:0/1078964652 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:30.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.965+0000 7f8011ad5640 1 --2- 172.21.15.40:0/1078964652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f800c071240 0x7f800c152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:30.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.965+0000 7f80112d4640 1 --2- 172.21.15.40:0/1078964652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f800c13c550 0x7f800c0aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:30.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.965+0000 7f80122d6640 1 -- 172.21.15.40:0/1078964652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f800c071240 msgr2=0x7f800c152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.965+0000 7f80122d6640 1 --2- 172.21.15.40:0/1078964652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f800c071240 0x7f800c152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.965+0000 7f80122d6640 1 -- 172.21.15.40:0/1078964652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f800c13c550 msgr2=0x7f800c0aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.965+0000 7f80122d6640 1 --2- 172.21.15.40:0/1078964652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f800c13c550 0x7f800c0aec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.965+0000 7f80122d6640 1 -- 172.21.15.40:0/1078964652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc009560 con 0x7f800c13f190 2023-10-18T08:56:30.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.966+0000 7f80122d6640 1 --2- 172.21.15.40:0/1078964652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800c13f190 0x7f800c0af160 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8004001eb0 tx=0x7f800400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.966+0000 7f80027fc640 1 -- 172.21.15.40:0/1078964652 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040052d0 con 0x7f800c13f190 2023-10-18T08:56:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.966+0000 7f80027fc640 1 -- 172.21.15.40:0/1078964652 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f80040035f0 con 0x7f800c13f190 2023-10-18T08:56:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.966+0000 7f80027fc640 1 -- 172.21.15.40:0/1078964652 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040039e0 con 0x7f800c13f190 2023-10-18T08:56:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.966+0000 7f8013d39640 1 -- 172.21.15.40:0/1078964652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c0b08f0 con 0x7f800c13f190 2023-10-18T08:56:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.966+0000 7f8013d39640 1 -- 172.21.15.40:0/1078964652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c0b0eb0 con 0x7f800c13f190 2023-10-18T08:56:30.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.967+0000 7f8013d39640 1 -- 172.21.15.40:0/1078964652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800c0b13d0 con 0x7f800c13f190 2023-10-18T08:56:30.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.968+0000 7f80027fc640 1 -- 172.21.15.40:0/1078964652 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8004003b40 con 0x7f800c13f190 2023-10-18T08:56:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.969+0000 7f80027fc640 1 --2- 172.21.15.40:0/1078964652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ff006ae40 0x7f7ff006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.969+0000 7f80027fc640 1 -- 172.21.15.40:0/1078964652 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f8004088c90 con 0x7f800c13f190 2023-10-18T08:56:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.970+0000 7f8011ad5640 1 --2- 172.21.15.40:0/1078964652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ff006ae40 0x7f7ff006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:30.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.971+0000 7f8011ad5640 1 --2- 172.21.15.40:0/1078964652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ff006ae40 0x7f7ff006d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8008005fd0 tx=0x7f8008009870 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:30.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:30.974+0000 7f80027fc640 1 -- 172.21.15.40:0/1078964652 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f800c0b13d0 con 0x7f800c13f190 2023-10-18T08:56:31.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.168+0000 7f8013d39640 1 -- 172.21.15.40:0/1078964652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f800c0b13d0 con 0x7f800c13f190 2023-10-18T08:56:31.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.700+0000 7f80027fc640 1 -- 172.21.15.40:0/1078964652 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7f800c0b13d0 con 0x7f800c13f190 2023-10-18T08:56:31.703 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-10-18T08:56:31.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.704+0000 7f8013d39640 1 -- 172.21.15.40:0/1078964652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ff006ae40 msgr2=0x7f7ff006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:31.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.704+0000 7f8013d39640 1 --2- 172.21.15.40:0/1078964652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ff006ae40 0x7f7ff006d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8008005fd0 tx=0x7f8008009870 comp rx=0 tx=0).stop 2023-10-18T08:56:31.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.704+0000 7f8013d39640 1 -- 172.21.15.40:0/1078964652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800c13f190 msgr2=0x7f800c0af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:31.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.704+0000 7f8013d39640 1 --2- 172.21.15.40:0/1078964652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800c13f190 0x7f800c0af160 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8004001eb0 tx=0x7f800400c890 comp rx=0 tx=0).stop 2023-10-18T08:56:31.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.705+0000 7f8013d39640 1 -- 172.21.15.40:0/1078964652 shutdown_connections 2023-10-18T08:56:31.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.705+0000 7f8013d39640 1 --2- 172.21.15.40:0/1078964652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ff006ae40 0x7f7ff006d300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:31.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.705+0000 7f8013d39640 1 --2- 172.21.15.40:0/1078964652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800c13f190 0x7f800c0af160 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:31.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.705+0000 7f8013d39640 1 --2- 172.21.15.40:0/1078964652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f800c13c550 0x7f800c0aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:31.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.705+0000 7f8013d39640 1 --2- 172.21.15.40:0/1078964652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f800c071240 0x7f800c152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:31.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.705+0000 7f8013d39640 1 -- 172.21.15.40:0/1078964652 >> 172.21.15.40:0/1078964652 conn(0x7f800c069720 msgr2=0x7f800c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:31.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.708+0000 7f8013d39640 1 -- 172.21.15.40:0/1078964652 shutdown_connections 2023-10-18T08:56:31.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.711+0000 7f8013d39640 1 -- 172.21.15.40:0/1078964652 wait complete. 2023-10-18T08:56:31.745 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T08:56:31.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.953+0000 7f25790da640 1 -- 172.21.15.40:0/2173223833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f257415af70 msgr2=0x7f257406c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:31.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.953+0000 7f25790da640 1 --2- 172.21.15.40:0/2173223833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f257415af70 0x7f257406c100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f255c005fd0 tx=0x7f255c02bd90 comp rx=0 tx=0).stop 2023-10-18T08:56:31.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.953+0000 7f25790da640 1 -- 172.21.15.40:0/2173223833 shutdown_connections 2023-10-18T08:56:31.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.953+0000 7f25790da640 1 --2- 172.21.15.40:0/2173223833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f257415af70 0x7f257406c100 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:31.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.953+0000 7f25790da640 1 --2- 172.21.15.40:0/2173223833 >> v2:172.21.15.144:3300/0 conn(0x7f25740c9620 0x7f257406bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:31.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.953+0000 7f25790da640 1 -- 172.21.15.40:0/2173223833 >> 172.21.15.40:0/2173223833 conn(0x7f257406ca10 msgr2=0x7f257406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:31.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.953+0000 7f25790da640 1 -- 172.21.15.40:0/2173223833 shutdown_connections 2023-10-18T08:56:31.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.953+0000 7f25790da640 1 -- 172.21.15.40:0/2173223833 wait complete. 2023-10-18T08:56:31.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.955+0000 7f25790da640 1 Processor -- start 2023-10-18T08:56:31.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.955+0000 7f25790da640 1 -- start start 2023-10-18T08:56:31.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.956+0000 7f25790da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2574070b60 0x7f257414f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:31.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.956+0000 7f25790da640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25740c9620 0x7f2574152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:31.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.956+0000 7f25790da640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f257415af70 0x7f2574152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:31.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.956+0000 7f25790da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25740ab6a0 con 0x7f2574070b60 2023-10-18T08:56:31.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.956+0000 7f25790da640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25740aaa20 con 0x7f25740c9620 2023-10-18T08:56:31.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.956+0000 7f25790da640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25740aad00 con 0x7f257415af70 2023-10-18T08:56:31.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.956+0000 7f2573fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2574070b60 0x7f257414f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:31.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.956+0000 7f2573fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2574070b60 0x7f257414f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59852/0 (socket says 172.21.15.40:59852) 2023-10-18T08:56:31.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.956+0000 7f2573fff640 1 -- 172.21.15.40:0/2677166572 learned_addr learned my addr 172.21.15.40:0/2677166572 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:31.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.957+0000 7f256bfff640 1 --2- 172.21.15.40:0/2677166572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25740c9620 0x7f2574152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:31.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.957+0000 7f25788d9640 1 --2- 172.21.15.40:0/2677166572 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f257415af70 0x7f2574152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:31.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.957+0000 7f2573fff640 1 -- 172.21.15.40:0/2677166572 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f257415af70 msgr2=0x7f2574152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:31.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.957+0000 7f2573fff640 1 --2- 172.21.15.40:0/2677166572 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f257415af70 0x7f2574152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:31.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.957+0000 7f2573fff640 1 -- 172.21.15.40:0/2677166572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25740c9620 msgr2=0x7f2574152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:31.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.957+0000 7f2573fff640 1 --2- 172.21.15.40:0/2677166572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25740c9620 0x7f2574152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:31.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.957+0000 7f2573fff640 1 -- 172.21.15.40:0/2677166572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c009560 con 0x7f2574070b60 2023-10-18T08:56:31.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.957+0000 7f256bfff640 1 --2- 172.21.15.40:0/2677166572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25740c9620 0x7f2574152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:31.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.957+0000 7f25788d9640 1 --2- 172.21.15.40:0/2677166572 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f257415af70 0x7f2574152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:31.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.957+0000 7f2573fff640 1 --2- 172.21.15.40:0/2677166572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2574070b60 0x7f257414f3f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f256400e9f0 tx=0x7f256400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:31.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.958+0000 7f25717fa640 1 -- 172.21.15.40:0/2677166572 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2564009700 con 0x7f2574070b60 2023-10-18T08:56:31.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.958+0000 7f25717fa640 1 -- 172.21.15.40:0/2677166572 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f256400ce70 con 0x7f2574070b60 2023-10-18T08:56:31.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.958+0000 7f25717fa640 1 -- 172.21.15.40:0/2677166572 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2564010670 con 0x7f2574070b60 2023-10-18T08:56:31.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.958+0000 7f25790da640 1 -- 172.21.15.40:0/2677166572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25740aafe0 con 0x7f2574070b60 2023-10-18T08:56:31.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.958+0000 7f25790da640 1 -- 172.21.15.40:0/2677166572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25741514a0 con 0x7f2574070b60 2023-10-18T08:56:31.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.959+0000 7f25790da640 1 -- 172.21.15.40:0/2677166572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25740ab910 con 0x7f2574070b60 2023-10-18T08:56:31.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.960+0000 7f25717fa640 1 -- 172.21.15.40:0/2677166572 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f25640107d0 con 0x7f2574070b60 2023-10-18T08:56:31.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.961+0000 7f25717fa640 1 --2- 172.21.15.40:0/2677166572 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f255406ae70 0x7f255406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:31.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.961+0000 7f25717fa640 1 -- 172.21.15.40:0/2677166572 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2564014070 con 0x7f2574070b60 2023-10-18T08:56:31.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.962+0000 7f256bfff640 1 --2- 172.21.15.40:0/2677166572 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f255406ae70 0x7f255406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:31.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.963+0000 7f256bfff640 1 --2- 172.21.15.40:0/2677166572 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f255406ae70 0x7f255406d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f255c035000 tx=0x7f255c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:31.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:31.966+0000 7f25717fa640 1 -- 172.21.15.40:0/2677166572 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2564051ca0 con 0x7f2574070b60 2023-10-18T08:56:32.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:32.200+0000 7f25790da640 1 -- 172.21.15.40:0/2677166572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f25740c9b10 con 0x7f2574070b60 2023-10-18T08:56:32.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:32.878+0000 7f25717fa640 1 -- 172.21.15.40:0/2677166572 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2564058ae0 con 0x7f2574070b60 2023-10-18T08:56:32.881 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 4 and data pool 5 2023-10-18T08:56:32.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:32.882+0000 7f25790da640 1 -- 172.21.15.40:0/2677166572 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f255406ae70 msgr2=0x7f255406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:32.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:32.882+0000 7f25790da640 1 --2- 172.21.15.40:0/2677166572 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f255406ae70 0x7f255406d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f255c035000 tx=0x7f255c004730 comp rx=0 tx=0).stop 2023-10-18T08:56:32.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:32.882+0000 7f25790da640 1 -- 172.21.15.40:0/2677166572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2574070b60 msgr2=0x7f257414f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:32.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:32.883+0000 7f25790da640 1 --2- 172.21.15.40:0/2677166572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2574070b60 0x7f257414f3f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f256400e9f0 tx=0x7f256400eeb0 comp rx=0 tx=0).stop 2023-10-18T08:56:32.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:32.883+0000 7f25790da640 1 -- 172.21.15.40:0/2677166572 shutdown_connections 2023-10-18T08:56:32.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:32.883+0000 7f25790da640 1 --2- 172.21.15.40:0/2677166572 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f255406ae70 0x7f255406d330 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:32.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:32.883+0000 7f25790da640 1 --2- 172.21.15.40:0/2677166572 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f257415af70 0x7f2574152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:32.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:32.883+0000 7f25790da640 1 --2- 172.21.15.40:0/2677166572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25740c9620 0x7f2574152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:32.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:32.883+0000 7f25790da640 1 --2- 172.21.15.40:0/2677166572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2574070b60 0x7f257414f3f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:32.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:32.883+0000 7f25790da640 1 -- 172.21.15.40:0/2677166572 >> 172.21.15.40:0/2677166572 conn(0x7f257406ca10 msgr2=0x7f25740afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:32.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:32.883+0000 7f25790da640 1 -- 172.21.15.40:0/2677166572 shutdown_connections 2023-10-18T08:56:32.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:32.883+0000 7f25790da640 1 -- 172.21.15.40:0/2677166572 wait complete. 2023-10-18T08:56:32.908 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:33.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.109+0000 7fea4c94c640 1 -- 172.21.15.40:0/2995141398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea480b1b90 msgr2=0x7fea480cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:33.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.109+0000 7fea4c94c640 1 --2- 172.21.15.40:0/2995141398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea480b1b90 0x7fea480cd2f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fea30009860 tx=0x7fea3002bd50 comp rx=0 tx=0).stop 2023-10-18T08:56:33.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.109+0000 7fea4c94c640 1 -- 172.21.15.40:0/2995141398 shutdown_connections 2023-10-18T08:56:33.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.109+0000 7fea4c94c640 1 --2- 172.21.15.40:0/2995141398 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea480b1b90 0x7fea480cd2f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.110+0000 7fea4c94c640 1 --2- 172.21.15.40:0/2995141398 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea4815eaa0 0x7fea480ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.110+0000 7fea4c94c640 1 -- 172.21.15.40:0/2995141398 >> 172.21.15.40:0/2995141398 conn(0x7fea4806cee0 msgr2=0x7fea48069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:33.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.110+0000 7fea4c94c640 1 -- 172.21.15.40:0/2995141398 shutdown_connections 2023-10-18T08:56:33.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.110+0000 7fea4c94c640 1 -- 172.21.15.40:0/2995141398 wait complete. 2023-10-18T08:56:33.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.111+0000 7fea4c94c640 1 Processor -- start 2023-10-18T08:56:33.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.112+0000 7fea4c94c640 1 -- start start 2023-10-18T08:56:33.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.112+0000 7fea4c94c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea4806ffe0 0x7fea4814b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:33.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.113+0000 7fea4c94c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea480b1b90 0x7fea480cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:33.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.113+0000 7fea4c94c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea4815eaa0 0x7fea4815c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:33.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.113+0000 7fea4c94c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea480caee0 con 0x7fea4815eaa0 2023-10-18T08:56:33.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.113+0000 7fea4c94c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4815c470 con 0x7fea4806ffe0 2023-10-18T08:56:33.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.113+0000 7fea4c94c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea480ca5b0 con 0x7fea480b1b90 2023-10-18T08:56:33.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.113+0000 7fea46d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea4815eaa0 0x7fea4815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:33.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.113+0000 7fea46d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea4815eaa0 0x7fea4815c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59870/0 (socket says 172.21.15.40:59870) 2023-10-18T08:56:33.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.113+0000 7fea46d9d640 1 -- 172.21.15.40:0/2122161352 learned_addr learned my addr 172.21.15.40:0/2122161352 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:33.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.113+0000 7fea45d9b640 1 --2- 172.21.15.40:0/2122161352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea480b1b90 0x7fea480cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:33.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.113+0000 7fea4659c640 1 --2- 172.21.15.40:0/2122161352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea4806ffe0 0x7fea4814b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:33.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.113+0000 7fea46d9d640 1 -- 172.21.15.40:0/2122161352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea480b1b90 msgr2=0x7fea480cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:33.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.113+0000 7fea46d9d640 1 --2- 172.21.15.40:0/2122161352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea480b1b90 0x7fea480cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.113+0000 7fea46d9d640 1 -- 172.21.15.40:0/2122161352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea4806ffe0 msgr2=0x7fea4814b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:33.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.113+0000 7fea46d9d640 1 --2- 172.21.15.40:0/2122161352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea4806ffe0 0x7fea4814b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.114+0000 7fea46d9d640 1 -- 172.21.15.40:0/2122161352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea30009490 con 0x7fea4815eaa0 2023-10-18T08:56:33.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.114+0000 7fea45d9b640 1 --2- 172.21.15.40:0/2122161352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea480b1b90 0x7fea480cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:56:33.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.114+0000 7fea46d9d640 1 --2- 172.21.15.40:0/2122161352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea4815eaa0 0x7fea4815c880 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c430 tx=0x7fea4000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:33.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.114+0000 7fea3effd640 1 -- 172.21.15.40:0/2122161352 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40011790 con 0x7fea4815eaa0 2023-10-18T08:56:33.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.114+0000 7fea3effd640 1 -- 172.21.15.40:0/2122161352 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fea400119e0 con 0x7fea4815eaa0 2023-10-18T08:56:33.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.114+0000 7fea3effd640 1 -- 172.21.15.40:0/2122161352 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40011e10 con 0x7fea4815eaa0 2023-10-18T08:56:33.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.114+0000 7fea4c94c640 1 -- 172.21.15.40:0/2122161352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea480ca890 con 0x7fea4815eaa0 2023-10-18T08:56:33.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.114+0000 7fea4c94c640 1 -- 172.21.15.40:0/2122161352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea481543f0 con 0x7fea4815eaa0 2023-10-18T08:56:33.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.115+0000 7fea4c94c640 1 -- 172.21.15.40:0/2122161352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea480706f0 con 0x7fea4815eaa0 2023-10-18T08:56:33.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.117+0000 7fea3effd640 1 -- 172.21.15.40:0/2122161352 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fea4001c460 con 0x7fea4815eaa0 2023-10-18T08:56:33.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.117+0000 7fea3effd640 1 --2- 172.21.15.40:0/2122161352 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea2806ae40 0x7fea2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:33.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.118+0000 7fea3effd640 1 -- 172.21.15.40:0/2122161352 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea40089810 con 0x7fea4815eaa0 2023-10-18T08:56:33.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.118+0000 7fea4659c640 1 --2- 172.21.15.40:0/2122161352 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea2806ae40 0x7fea2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:33.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.119+0000 7fea4659c640 1 --2- 172.21.15.40:0/2122161352 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea2806ae40 0x7fea2806d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea3800a2e0 tx=0x7fea3800a270 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:33.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.122+0000 7fea3effd640 1 -- 172.21.15.40:0/2122161352 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea40054820 con 0x7fea4815eaa0 2023-10-18T08:56:33.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.318+0000 7fea4c94c640 1 -- 172.21.15.40:0/2122161352 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea48155440 con 0x7fea4815eaa0 2023-10-18T08:56:33.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.320+0000 7fea3effd640 1 -- 172.21.15.40:0/2122161352 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16594 (secure 0 0 0) 0x7fea40057e40 con 0x7fea4815eaa0 2023-10-18T08:56:33.323 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:33.324 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":32,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:32.202312+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:56:30.127184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-18T08:56:31.225270+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:33.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.323+0000 7fea4c94c640 1 -- 172.21.15.40:0/2122161352 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea2806ae40 msgr2=0x7fea2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:33.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.323+0000 7fea4c94c640 1 --2- 172.21.15.40:0/2122161352 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea2806ae40 0x7fea2806d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea3800a2e0 tx=0x7fea3800a270 comp rx=0 tx=0).stop 2023-10-18T08:56:33.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.323+0000 7fea4c94c640 1 -- 172.21.15.40:0/2122161352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea4815eaa0 msgr2=0x7fea4815c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:33.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.323+0000 7fea4c94c640 1 --2- 172.21.15.40:0/2122161352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea4815eaa0 0x7fea4815c880 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c430 tx=0x7fea4000c8f0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.323+0000 7fea4c94c640 1 -- 172.21.15.40:0/2122161352 shutdown_connections 2023-10-18T08:56:33.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.323+0000 7fea4c94c640 1 --2- 172.21.15.40:0/2122161352 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea2806ae40 0x7fea2806d300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.323+0000 7fea4c94c640 1 --2- 172.21.15.40:0/2122161352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea4815eaa0 0x7fea4815c880 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.323+0000 7fea4c94c640 1 --2- 172.21.15.40:0/2122161352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea480b1b90 0x7fea480cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.323+0000 7fea4c94c640 1 --2- 172.21.15.40:0/2122161352 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea4806ffe0 0x7fea4814b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.323+0000 7fea4c94c640 1 -- 172.21.15.40:0/2122161352 >> 172.21.15.40:0/2122161352 conn(0x7fea4806cee0 msgr2=0x7fea480adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.323+0000 7fea4c94c640 1 -- 172.21.15.40:0/2122161352 shutdown_connections 2023-10-18T08:56:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.323+0000 7fea4c94c640 1 -- 172.21.15.40:0/2122161352 wait complete. 2023-10-18T08:56:33.344 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:33.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.562+0000 7fab64a56640 1 -- 172.21.15.40:0/3208562078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab6015a6c0 msgr2=0x7fab600c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:33.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.562+0000 7fab64a56640 1 --2- 172.21.15.40:0/3208562078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab6015a6c0 0x7fab600c9b00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fab48009830 tx=0x7fab4802bd70 comp rx=0 tx=0).stop 2023-10-18T08:56:33.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.563+0000 7fab64a56640 1 -- 172.21.15.40:0/3208562078 shutdown_connections 2023-10-18T08:56:33.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.563+0000 7fab64a56640 1 --2- 172.21.15.40:0/3208562078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab6015a6c0 0x7fab600c9b00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.563+0000 7fab64a56640 1 -- 172.21.15.40:0/3208562078 >> 172.21.15.40:0/3208562078 conn(0x7fab6006cee0 msgr2=0x7fab60069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:33.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.563+0000 7fab64a56640 1 -- 172.21.15.40:0/3208562078 shutdown_connections 2023-10-18T08:56:33.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.563+0000 7fab64a56640 1 -- 172.21.15.40:0/3208562078 wait complete. 2023-10-18T08:56:33.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.565+0000 7fab64a56640 1 Processor -- start 2023-10-18T08:56:33.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.565+0000 7fab64a56640 1 -- start start 2023-10-18T08:56:33.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.566+0000 7fab64a56640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab600ca040 0x7fab60141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:33.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.566+0000 7fab64a56640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab6015a2e0 0x7fab600a7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:33.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.566+0000 7fab64a56640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab6015a6c0 0x7fab60158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:33.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.566+0000 7fab64a56640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab60142110 con 0x7fab600ca040 2023-10-18T08:56:33.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.566+0000 7fab64a56640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab600a6470 con 0x7fab6015a2e0 2023-10-18T08:56:33.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.566+0000 7fab64a56640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab600a7c60 con 0x7fab6015a6c0 2023-10-18T08:56:33.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.566+0000 7fab5e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab600ca040 0x7fab60141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:33.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.566+0000 7fab5e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab600ca040 0x7fab60141bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59892/0 (socket says 172.21.15.40:59892) 2023-10-18T08:56:33.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.566+0000 7fab5e59c640 1 -- 172.21.15.40:0/2997619821 learned_addr learned my addr 172.21.15.40:0/2997619821 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:33.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.567+0000 7fab5dd9b640 1 --2- 172.21.15.40:0/2997619821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab6015a2e0 0x7fab600a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:33.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.567+0000 7fab5ed9d640 1 --2- 172.21.15.40:0/2997619821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab6015a6c0 0x7fab60158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:33.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.567+0000 7fab5e59c640 1 -- 172.21.15.40:0/2997619821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab6015a6c0 msgr2=0x7fab60158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:33.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.567+0000 7fab5e59c640 1 --2- 172.21.15.40:0/2997619821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab6015a6c0 0x7fab60158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.567+0000 7fab5e59c640 1 -- 172.21.15.40:0/2997619821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab6015a2e0 msgr2=0x7fab600a7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:33.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.567+0000 7fab5e59c640 1 --2- 172.21.15.40:0/2997619821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab6015a2e0 0x7fab600a7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.567+0000 7fab5e59c640 1 -- 172.21.15.40:0/2997619821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab48009490 con 0x7fab600ca040 2023-10-18T08:56:33.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.567+0000 7fab5e59c640 1 --2- 172.21.15.40:0/2997619821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab600ca040 0x7fab60141bd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fab5000c7f0 tx=0x7fab5000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:33.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.567+0000 7fab56ffd640 1 -- 172.21.15.40:0/2997619821 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000aa50 con 0x7fab600ca040 2023-10-18T08:56:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.567+0000 7fab56ffd640 1 -- 172.21.15.40:0/2997619821 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fab5000aca0 con 0x7fab600ca040 2023-10-18T08:56:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.567+0000 7fab56ffd640 1 -- 172.21.15.40:0/2997619821 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50012680 con 0x7fab600ca040 2023-10-18T08:56:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.567+0000 7fab64a56640 1 -- 172.21.15.40:0/2997619821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab600a7290 con 0x7fab600ca040 2023-10-18T08:56:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.568+0000 7fab64a56640 1 -- 172.21.15.40:0/2997619821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab600a6960 con 0x7fab600ca040 2023-10-18T08:56:33.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.569+0000 7fab64a56640 1 -- 172.21.15.40:0/2997619821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6006a5a0 con 0x7fab600ca040 2023-10-18T08:56:33.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.570+0000 7fab56ffd640 1 -- 172.21.15.40:0/2997619821 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fab5001b400 con 0x7fab600ca040 2023-10-18T08:56:33.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.571+0000 7fab56ffd640 1 --2- 172.21.15.40:0/2997619821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fab4006ae40 0x7fab4006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:33.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.571+0000 7fab56ffd640 1 -- 172.21.15.40:0/2997619821 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab50089b00 con 0x7fab600ca040 2023-10-18T08:56:33.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.572+0000 7fab5dd9b640 1 --2- 172.21.15.40:0/2997619821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fab4006ae40 0x7fab4006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:33.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.573+0000 7fab5dd9b640 1 --2- 172.21.15.40:0/2997619821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fab4006ae40 0x7fab4006d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fab48036000 tx=0x7fab48004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:33.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.576+0000 7fab56ffd640 1 -- 172.21.15.40:0/2997619821 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab50054bb0 con 0x7fab600ca040 2023-10-18T08:56:33.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.776+0000 7fab64a56640 1 -- 172.21.15.40:0/2997619821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab60066fd0 con 0x7fab600ca040 2023-10-18T08:56:33.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.778+0000 7fab56ffd640 1 -- 172.21.15.40:0/2997619821 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16581 (secure 0 0 0) 0x7fab50016090 con 0x7fab600ca040 2023-10-18T08:56:33.781 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:33.781 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":33,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:33.724325+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:56:30.127184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-18T08:56:31.225270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:33.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.782+0000 7fab64a56640 1 -- 172.21.15.40:0/2997619821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fab4006ae40 msgr2=0x7fab4006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:33.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.782+0000 7fab64a56640 1 --2- 172.21.15.40:0/2997619821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fab4006ae40 0x7fab4006d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fab48036000 tx=0x7fab48004730 comp rx=0 tx=0).stop 2023-10-18T08:56:33.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.782+0000 7fab64a56640 1 -- 172.21.15.40:0/2997619821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab600ca040 msgr2=0x7fab60141bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:33.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.782+0000 7fab64a56640 1 --2- 172.21.15.40:0/2997619821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab600ca040 0x7fab60141bd0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fab5000c7f0 tx=0x7fab5000ccb0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.782+0000 7fab64a56640 1 -- 172.21.15.40:0/2997619821 shutdown_connections 2023-10-18T08:56:33.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.782+0000 7fab64a56640 1 --2- 172.21.15.40:0/2997619821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fab4006ae40 0x7fab4006d300 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.782+0000 7fab64a56640 1 --2- 172.21.15.40:0/2997619821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab6015a6c0 0x7fab60158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.782+0000 7fab64a56640 1 --2- 172.21.15.40:0/2997619821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab6015a2e0 0x7fab600a7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.782+0000 7fab64a56640 1 --2- 172.21.15.40:0/2997619821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab600ca040 0x7fab60141bd0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:33.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.782+0000 7fab64a56640 1 -- 172.21.15.40:0/2997619821 >> 172.21.15.40:0/2997619821 conn(0x7fab6006cee0 msgr2=0x7fab600cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:33.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.782+0000 7fab64a56640 1 -- 172.21.15.40:0/2997619821 shutdown_connections 2023-10-18T08:56:33.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:33.782+0000 7fab64a56640 1 -- 172.21.15.40:0/2997619821 wait complete. 2023-10-18T08:56:33.803 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T08:56:34.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.036+0000 7f0f83fff640 1 -- 172.21.15.40:0/2234923757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f70005020 msgr2=0x7f0f7000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:34.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.036+0000 7f0f83fff640 1 --2- 172.21.15.40:0/2234923757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f70005020 0x7f0f7000b980 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c006d60 tx=0x7f0f700057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:34.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.036+0000 7f0f837fe640 1 -- 172.21.15.40:0/2234923757 reap_dead start 2023-10-18T08:56:34.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.036+0000 7f0f83fff640 1 -- 172.21.15.40:0/2234923757 shutdown_connections 2023-10-18T08:56:34.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.036+0000 7f0f83fff640 1 -- 172.21.15.40:0/2234923757 >> 172.21.15.40:0/2234923757 conn(0x7f0f8406ca10 msgr2=0x7f0f8406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:34.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.036+0000 7f0f83fff640 1 -- 172.21.15.40:0/2234923757 shutdown_connections 2023-10-18T08:56:34.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.036+0000 7f0f83fff640 1 -- 172.21.15.40:0/2234923757 wait complete. 2023-10-18T08:56:34.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.038+0000 7f0f83fff640 1 Processor -- start 2023-10-18T08:56:34.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.038+0000 7f0f83fff640 1 -- start start 2023-10-18T08:56:34.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.039+0000 7f0f83fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f70005020 0x7f0f8414d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:34.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.039+0000 7f0f83fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f840ab8d0 0x7f0f840abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:34.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.039+0000 7f0f83fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f84156280 0x7f0f84156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:34.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.039+0000 7f0f83fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f840ac290 con 0x7f0f84156280 2023-10-18T08:56:34.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.039+0000 7f0f83fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f84155df0 con 0x7f0f840ab8d0 2023-10-18T08:56:34.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.039+0000 7f0f83fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f84159780 con 0x7f0f70005020 2023-10-18T08:56:34.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.039+0000 7f0f837fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f84156280 0x7f0f84156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:34.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.039+0000 7f0f837fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f84156280 0x7f0f84156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59920/0 (socket says 172.21.15.40:59920) 2023-10-18T08:56:34.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.039+0000 7f0f837fe640 1 -- 172.21.15.40:0/2577156251 learned_addr learned my addr 172.21.15.40:0/2577156251 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.040+0000 7f0f827fc640 1 --2- 172.21.15.40:0/2577156251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f840ab8d0 0x7f0f840abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.040+0000 7f0f837fe640 1 -- 172.21.15.40:0/2577156251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f70005020 msgr2=0x7f0f8414d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.040+0000 7f0f837fe640 1 --2- 172.21.15.40:0/2577156251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f70005020 0x7f0f8414d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:34.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.040+0000 7f0f837fe640 1 -- 172.21.15.40:0/2577156251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f840ab8d0 msgr2=0x7f0f840abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.040+0000 7f0f837fe640 1 --2- 172.21.15.40:0/2577156251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f840ab8d0 0x7f0f840abd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.040+0000 7f0f837fe640 1 -- 172.21.15.40:0/2577156251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f841554c0 con 0x7f0f84156280 2023-10-18T08:56:34.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.040+0000 7f0f837fe640 1 --2- 172.21.15.40:0/2577156251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f84156280 0x7f0f84156700 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00c320 tx=0x7f0f7c001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:34.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.040+0000 7f0f7b7fe640 1 -- 172.21.15.40:0/2577156251 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0017c0 con 0x7f0f84156280 2023-10-18T08:56:34.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.041+0000 7f0f7b7fe640 1 -- 172.21.15.40:0/2577156251 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0f7c0060d0 con 0x7f0f84156280 2023-10-18T08:56:34.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.041+0000 7f0f7b7fe640 1 -- 172.21.15.40:0/2577156251 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0064a0 con 0x7f0f84156280 2023-10-18T08:56:34.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.041+0000 7f0f83fff640 1 -- 172.21.15.40:0/2577156251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f841557a0 con 0x7f0f84156280 2023-10-18T08:56:34.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.041+0000 7f0f83fff640 1 -- 172.21.15.40:0/2577156251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8414dd40 con 0x7f0f84156280 2023-10-18T08:56:34.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.042+0000 7f0f83fff640 1 -- 172.21.15.40:0/2577156251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f84066e00 con 0x7f0f84156280 2023-10-18T08:56:34.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.043+0000 7f0f7b7fe640 1 -- 172.21.15.40:0/2577156251 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0f7c033030 con 0x7f0f84156280 2023-10-18T08:56:34.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.044+0000 7f0f7b7fe640 1 --2- 172.21.15.40:0/2577156251 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f6406ae70 0x7f0f6406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:34.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.044+0000 7f0f7b7fe640 1 -- 172.21.15.40:0/2577156251 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f7c0aa2e0 con 0x7f0f84156280 2023-10-18T08:56:34.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.045+0000 7f0f82ffd640 1 --2- 172.21.15.40:0/2577156251 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f6406ae70 0x7f0f6406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:34.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.046+0000 7f0f82ffd640 1 --2- 172.21.15.40:0/2577156251 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f6406ae70 0x7f0f6406d330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0f74005c40 tx=0x7f0f74005bd0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:34.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.049+0000 7f0f7b7fe640 1 -- 172.21.15.40:0/2577156251 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0f7c0752f0 con 0x7f0f84156280 2023-10-18T08:56:34.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.282+0000 7f0f83fff640 1 -- 172.21.15.40:0/2577156251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0f840c9b60 con 0x7f0f84156280 2023-10-18T08:56:34.378 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:56:34.387 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:56:34.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.904+0000 7f0f7b7fe640 1 -- 172.21.15.40:0/2577156251 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0f840c9b60 con 0x7f0f84156280 2023-10-18T08:56:34.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.907+0000 7f0f83fff640 1 -- 172.21.15.40:0/2577156251 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f6406ae70 msgr2=0x7f0f6406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:34.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.907+0000 7f0f83fff640 1 --2- 172.21.15.40:0/2577156251 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f6406ae70 0x7f0f6406d330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0f74005c40 tx=0x7f0f74005bd0 comp rx=0 tx=0).stop 2023-10-18T08:56:34.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.907+0000 7f0f83fff640 1 -- 172.21.15.40:0/2577156251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f84156280 msgr2=0x7f0f84156700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:34.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.907+0000 7f0f83fff640 1 --2- 172.21.15.40:0/2577156251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f84156280 0x7f0f84156700 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00c320 tx=0x7f0f7c001130 comp rx=0 tx=0).stop 2023-10-18T08:56:34.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.908+0000 7f0f83fff640 1 -- 172.21.15.40:0/2577156251 shutdown_connections 2023-10-18T08:56:34.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.908+0000 7f0f83fff640 1 --2- 172.21.15.40:0/2577156251 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f6406ae70 0x7f0f6406d330 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:34.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.908+0000 7f0f83fff640 1 --2- 172.21.15.40:0/2577156251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f84156280 0x7f0f84156700 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:34.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.908+0000 7f0f83fff640 1 --2- 172.21.15.40:0/2577156251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f840ab8d0 0x7f0f840abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:34.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.908+0000 7f0f83fff640 1 --2- 172.21.15.40:0/2577156251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f70005020 0x7f0f8414d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:34.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.908+0000 7f0f83fff640 1 -- 172.21.15.40:0/2577156251 >> 172.21.15.40:0/2577156251 conn(0x7f0f8406ca10 msgr2=0x7f0f840b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:34.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.908+0000 7f0f83fff640 1 -- 172.21.15.40:0/2577156251 shutdown_connections 2023-10-18T08:56:34.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:34.908+0000 7f0f83fff640 1 -- 172.21.15.40:0/2577156251 wait complete. 2023-10-18T08:56:34.932 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T08:56:34.932 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T08:56:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.160+0000 7f773a056640 1 -- 172.21.15.40:0/3273358724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f773406c410 msgr2=0x7f7734070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:35.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.160+0000 7f773a056640 1 --2- 172.21.15.40:0/3273358724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f773406c410 0x7f7734070d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f771c009830 tx=0x7f771c02bd50 comp rx=0 tx=0).stop 2023-10-18T08:56:35.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.161+0000 7f773a056640 1 -- 172.21.15.40:0/3273358724 shutdown_connections 2023-10-18T08:56:35.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.161+0000 7f773a056640 1 --2- 172.21.15.40:0/3273358724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f773406c410 0x7f7734070d40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:35.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.161+0000 7f773a056640 1 --2- 172.21.15.40:0/3273358724 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f773413ec50 0x7f773413f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:35.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.161+0000 7f773a056640 1 -- 172.21.15.40:0/3273358724 >> 172.21.15.40:0/3273358724 conn(0x7f7734069720 msgr2=0x7f7734069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:35.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.161+0000 7f773a056640 1 -- 172.21.15.40:0/3273358724 shutdown_connections 2023-10-18T08:56:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.161+0000 7f773a056640 1 -- 172.21.15.40:0/3273358724 wait complete. 2023-10-18T08:56:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.163+0000 7f773a056640 1 Processor -- start 2023-10-18T08:56:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.163+0000 7f773a056640 1 -- start start 2023-10-18T08:56:35.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f773a056640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7734071280 0x7f7734139b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:35.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f773a056640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f773413ec50 0x7f77340cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:35.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f773a056640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77340cc700 0x7f77341f13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:35.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f773a056640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77341f2480 con 0x7f773413ec50 2023-10-18T08:56:35.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f773a056640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77340ccb80 con 0x7f77340cc700 2023-10-18T08:56:35.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f773a056640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77341f21a0 con 0x7f7734071280 2023-10-18T08:56:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f7732ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f773413ec50 0x7f77340cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f7732ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f773413ec50 0x7f77340cc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50916/0 (socket says 172.21.15.40:50916) 2023-10-18T08:56:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f7732ffd640 1 -- 172.21.15.40:0/40169951 learned_addr learned my addr 172.21.15.40:0/40169951 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f77337fe640 1 --2- 172.21.15.40:0/40169951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7734071280 0x7f7734139b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f7732ffd640 1 -- 172.21.15.40:0/40169951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7734071280 msgr2=0x7f7734139b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f7732ffd640 1 --2- 172.21.15.40:0/40169951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7734071280 0x7f7734139b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f7732ffd640 1 -- 172.21.15.40:0/40169951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77340cc700 msgr2=0x7f77341f13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f7733fff640 1 --2- 172.21.15.40:0/40169951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77340cc700 0x7f77341f13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f7732ffd640 1 --2- 172.21.15.40:0/40169951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77340cc700 0x7f77341f13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f7732ffd640 1 -- 172.21.15.40:0/40169951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c009490 con 0x7f773413ec50 2023-10-18T08:56:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.165+0000 7f7733fff640 1 --2- 172.21.15.40:0/40169951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77340cc700 0x7f77341f13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.166+0000 7f7732ffd640 1 --2- 172.21.15.40:0/40169951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f773413ec50 0x7f77340cc1c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f771c000c00 tx=0x7f771c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.166+0000 7f772bfff640 1 -- 172.21.15.40:0/40169951 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c003e20 con 0x7f773413ec50 2023-10-18T08:56:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.166+0000 7f772bfff640 1 -- 172.21.15.40:0/40169951 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f771c034440 con 0x7f773413ec50 2023-10-18T08:56:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.166+0000 7f773a056640 1 -- 172.21.15.40:0/40169951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77340afeb0 con 0x7f773413ec50 2023-10-18T08:56:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.166+0000 7f772bfff640 1 -- 172.21.15.40:0/40169951 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c034850 con 0x7f773413ec50 2023-10-18T08:56:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.166+0000 7f773a056640 1 -- 172.21.15.40:0/40169951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77340b0470 con 0x7f773413ec50 2023-10-18T08:56:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.168+0000 7f773a056640 1 -- 172.21.15.40:0/40169951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77340afcb0 con 0x7f773413ec50 2023-10-18T08:56:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.169+0000 7f772bfff640 1 -- 172.21.15.40:0/40169951 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f771c0349b0 con 0x7f773413ec50 2023-10-18T08:56:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.169+0000 7f772bfff640 1 --2- 172.21.15.40:0/40169951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f771406ae40 0x7f771406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.170+0000 7f772bfff640 1 -- 172.21.15.40:0/40169951 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f771c0abe70 con 0x7f773413ec50 2023-10-18T08:56:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.170+0000 7f77337fe640 1 --2- 172.21.15.40:0/40169951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f771406ae40 0x7f771406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.171+0000 7f77337fe640 1 --2- 172.21.15.40:0/40169951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f771406ae40 0x7f771406d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7724005fd0 tx=0x7f7724009500 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.174+0000 7f772bfff640 1 -- 172.21.15.40:0/40169951 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f771c076ea0 con 0x7f773413ec50 2023-10-18T08:56:35.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.402+0000 7f773a056640 1 -- 172.21.15.40:0/40169951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7734066fd0 con 0x7f773413ec50 2023-10-18T08:56:35.447 INFO:tasks.ceph.mds.a.smithi040.stderr:2023-10-18T08:56:35.445+0000 7f1aafcdc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T08:56:35.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.914+0000 7f772bfff640 1 -- 172.21.15.40:0/40169951 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f771c0390c0 con 0x7f773413ec50 2023-10-18T08:56:35.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.919+0000 7f773a056640 1 -- 172.21.15.40:0/40169951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f771406ae40 msgr2=0x7f771406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:35.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.919+0000 7f773a056640 1 --2- 172.21.15.40:0/40169951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f771406ae40 0x7f771406d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7724005fd0 tx=0x7f7724009500 comp rx=0 tx=0).stop 2023-10-18T08:56:35.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.919+0000 7f773a056640 1 -- 172.21.15.40:0/40169951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f773413ec50 msgr2=0x7f77340cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:35.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.919+0000 7f773a056640 1 --2- 172.21.15.40:0/40169951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f773413ec50 0x7f77340cc1c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f771c000c00 tx=0x7f771c003820 comp rx=0 tx=0).stop 2023-10-18T08:56:35.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.919+0000 7f773a056640 1 -- 172.21.15.40:0/40169951 shutdown_connections 2023-10-18T08:56:35.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.919+0000 7f773a056640 1 --2- 172.21.15.40:0/40169951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f771406ae40 0x7f771406d300 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:35.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.919+0000 7f773a056640 1 --2- 172.21.15.40:0/40169951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77340cc700 0x7f77341f13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:35.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.919+0000 7f773a056640 1 --2- 172.21.15.40:0/40169951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f773413ec50 0x7f77340cc1c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:35.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.919+0000 7f773a056640 1 --2- 172.21.15.40:0/40169951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7734071280 0x7f7734139b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:35.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.919+0000 7f773a056640 1 -- 172.21.15.40:0/40169951 >> 172.21.15.40:0/40169951 conn(0x7f7734069720 msgr2=0x7f77340cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:35.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.919+0000 7f773a056640 1 -- 172.21.15.40:0/40169951 shutdown_connections 2023-10-18T08:56:35.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:35.920+0000 7f773a056640 1 -- 172.21.15.40:0/40169951 wait complete. 2023-10-18T08:56:35.944 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T08:56:35.944 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:56:36.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.167+0000 7f05deebb640 1 -- 172.21.15.40:0/1748258818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05d815eaa0 msgr2=0x7f05d80ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:36.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.167+0000 7f05deebb640 1 --2- 172.21.15.40:0/1748258818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05d815eaa0 0x7f05d80ccdb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f05d806d720 tx=0x7f05c802c260 comp rx=0 tx=0).stop 2023-10-18T08:56:36.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.167+0000 7f05deebb640 1 -- 172.21.15.40:0/1748258818 shutdown_connections 2023-10-18T08:56:36.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.167+0000 7f05deebb640 1 --2- 172.21.15.40:0/1748258818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05d806ffe0 0x7f05d806b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.167+0000 7f05deebb640 1 --2- 172.21.15.40:0/1748258818 >> v2:172.21.15.144:3300/0 conn(0x7f05d80b1b90 0x7f05d80cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.167+0000 7f05deebb640 1 --2- 172.21.15.40:0/1748258818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05d815eaa0 0x7f05d80ccdb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.167+0000 7f05deebb640 1 -- 172.21.15.40:0/1748258818 >> 172.21.15.40:0/1748258818 conn(0x7f05d806cee0 msgr2=0x7f05d8069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:36.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.167+0000 7f05deebb640 1 -- 172.21.15.40:0/1748258818 shutdown_connections 2023-10-18T08:56:36.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.167+0000 7f05deebb640 1 -- 172.21.15.40:0/1748258818 wait complete. 2023-10-18T08:56:36.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.169+0000 7f05deebb640 1 Processor -- start 2023-10-18T08:56:36.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.169+0000 7f05deebb640 1 -- start start 2023-10-18T08:56:36.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.170+0000 7f05deebb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05d806ffe0 0x7f05d813ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:36.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.170+0000 7f05deebb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05d80b1b90 0x7f05d815c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:36.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.170+0000 7f05deebb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05d815eaa0 0x7f05d80cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:36.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.170+0000 7f05deebb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d814f790 con 0x7f05d806ffe0 2023-10-18T08:56:36.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.170+0000 7f05deebb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d815cd30 con 0x7f05d80b1b90 2023-10-18T08:56:36.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.170+0000 7f05deebb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d814ee30 con 0x7f05d815eaa0 2023-10-18T08:56:36.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.170+0000 7f05dcc57640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05d806ffe0 0x7f05d813ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:36.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.170+0000 7f05dcc57640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05d806ffe0 0x7f05d813ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50932/0 (socket says 172.21.15.40:50932) 2023-10-18T08:56:36.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.170+0000 7f05dcc57640 1 -- 172.21.15.40:0/4126531625 learned_addr learned my addr 172.21.15.40:0/4126531625 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:36.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.170+0000 7f05d7fff640 1 --2- 172.21.15.40:0/4126531625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05d80b1b90 0x7f05d815c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:36.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.171+0000 7f05dd458640 1 --2- 172.21.15.40:0/4126531625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05d815eaa0 0x7f05d80cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:36.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.171+0000 7f05dcc57640 1 -- 172.21.15.40:0/4126531625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05d815eaa0 msgr2=0x7f05d80cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:36.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.171+0000 7f05dcc57640 1 --2- 172.21.15.40:0/4126531625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05d815eaa0 0x7f05d80cc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.171+0000 7f05dcc57640 1 -- 172.21.15.40:0/4126531625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05d80b1b90 msgr2=0x7f05d815c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:36.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.171+0000 7f05dcc57640 1 --2- 172.21.15.40:0/4126531625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05d80b1b90 0x7f05d815c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.171+0000 7f05dcc57640 1 -- 172.21.15.40:0/4126531625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c8009900 con 0x7f05d806ffe0 2023-10-18T08:56:36.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.171+0000 7f05dcc57640 1 --2- 172.21.15.40:0/4126531625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05d806ffe0 0x7f05d813ca60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f05c8000f80 tx=0x7f05c8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:36.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.172+0000 7f05d57fa640 1 -- 172.21.15.40:0/4126531625 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8004450 con 0x7f05d806ffe0 2023-10-18T08:56:36.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.172+0000 7f05d57fa640 1 -- 172.21.15.40:0/4126531625 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f05c80046a0 con 0x7f05d806ffe0 2023-10-18T08:56:36.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.172+0000 7f05deebb640 1 -- 172.21.15.40:0/4126531625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d815ca50 con 0x7f05d806ffe0 2023-10-18T08:56:36.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.172+0000 7f05d57fa640 1 -- 172.21.15.40:0/4126531625 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c803f6a0 con 0x7f05d806ffe0 2023-10-18T08:56:36.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.172+0000 7f05deebb640 1 -- 172.21.15.40:0/4126531625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d814f3b0 con 0x7f05d806ffe0 2023-10-18T08:56:36.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.173+0000 7f05deebb640 1 -- 172.21.15.40:0/4126531625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d813d000 con 0x7f05d806ffe0 2023-10-18T08:56:36.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.176+0000 7f05d57fa640 1 -- 172.21.15.40:0/4126531625 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f05c803b040 con 0x7f05d806ffe0 2023-10-18T08:56:36.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.177+0000 7f05d57fa640 1 --2- 172.21.15.40:0/4126531625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f05b806ae40 0x7f05b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:36.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.177+0000 7f05d57fa640 1 -- 172.21.15.40:0/4126531625 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05c80ac500 con 0x7f05d806ffe0 2023-10-18T08:56:36.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.178+0000 7f05d7fff640 1 --2- 172.21.15.40:0/4126531625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f05b806ae40 0x7f05b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:36.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.179+0000 7f05d7fff640 1 --2- 172.21.15.40:0/4126531625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f05b806ae40 0x7f05b806d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f05c0005e10 tx=0x7f05c0005da0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:36.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.182+0000 7f05d57fa640 1 -- 172.21.15.40:0/4126531625 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f05c80775a0 con 0x7f05d806ffe0 2023-10-18T08:56:36.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.414+0000 7f05deebb640 1 -- 172.21.15.40:0/4126531625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05d8066fd0 con 0x7f05d806ffe0 2023-10-18T08:56:36.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.415+0000 7f05d57fa640 1 -- 172.21.15.40:0/4126531625 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f05c807abc0 con 0x7f05d806ffe0 2023-10-18T08:56:36.418 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:36.418 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/837790640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":837790640},{"type":"v1","addr":"172.21.15.40:6833","nonce":837790640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3377302775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3377302775},{"type":"v1","addr":"172.21.15.144:6836","nonce":3377302775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4939,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3162305748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3162305748},{"type":"v1","addr":"172.21.15.144:6837","nonce":3162305748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:56:32.203221+0000","modified":"2023-10-18T08:56:35.403466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5190":{"gid":5190,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3167454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3167454356},{"type":"v1","addr":"172.21.15.40:6835","nonce":3167454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T08:56:36.418 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2023-10-18T08:56:36.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.418+0000 7f05deebb640 1 -- 172.21.15.40:0/4126531625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f05b806ae40 msgr2=0x7f05b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:36.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.418+0000 7f05deebb640 1 --2- 172.21.15.40:0/4126531625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f05b806ae40 0x7f05b806d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f05c0005e10 tx=0x7f05c0005da0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.418+0000 7f05deebb640 1 -- 172.21.15.40:0/4126531625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05d806ffe0 msgr2=0x7f05d813ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:36.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.418+0000 7f05deebb640 1 --2- 172.21.15.40:0/4126531625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05d806ffe0 0x7f05d813ca60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f05c8000f80 tx=0x7f05c8003c20 comp rx=0 tx=0).stop 2023-10-18T08:56:36.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.418+0000 7f05deebb640 1 -- 172.21.15.40:0/4126531625 shutdown_connections 2023-10-18T08:56:36.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.418+0000 7f05deebb640 1 --2- 172.21.15.40:0/4126531625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f05b806ae40 0x7f05b806d300 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.418+0000 7f05deebb640 1 --2- 172.21.15.40:0/4126531625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05d815eaa0 0x7f05d80cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.418+0000 7f05deebb640 1 --2- 172.21.15.40:0/4126531625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05d80b1b90 0x7f05d815c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.418+0000 7f05deebb640 1 --2- 172.21.15.40:0/4126531625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05d806ffe0 0x7f05d813ca60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.418+0000 7f05deebb640 1 -- 172.21.15.40:0/4126531625 >> 172.21.15.40:0/4126531625 conn(0x7f05d806cee0 msgr2=0x7f05d8154b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:36.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.418+0000 7f05deebb640 1 -- 172.21.15.40:0/4126531625 shutdown_connections 2023-10-18T08:56:36.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.418+0000 7f05deebb640 1 -- 172.21.15.40:0/4126531625 wait complete. 2023-10-18T08:56:36.438 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:36.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.650+0000 7fd16cd2c640 1 -- 172.21.15.40:0/195791760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1680cc1a0 msgr2=0x7fd1680cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:36.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.650+0000 7fd16cd2c640 1 --2- 172.21.15.40:0/195791760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1680cc1a0 0x7fd1680cc630 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd16809b0d0 tx=0x7fd16002c2f0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.650+0000 7fd16cd2c640 1 -- 172.21.15.40:0/195791760 shutdown_connections 2023-10-18T08:56:36.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.650+0000 7fd16cd2c640 1 --2- 172.21.15.40:0/195791760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1680cc1a0 0x7fd1680cc630 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.650+0000 7fd16cd2c640 1 --2- 172.21.15.40:0/195791760 >> v2:172.21.15.144:3300/0 conn(0x7fd1680d1480 0x7fd1680d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.650+0000 7fd16cd2c640 1 -- 172.21.15.40:0/195791760 >> 172.21.15.40:0/195791760 conn(0x7fd16809a9a0 msgr2=0x7fd168088fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:36.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.650+0000 7fd16cd2c640 1 -- 172.21.15.40:0/195791760 shutdown_connections 2023-10-18T08:56:36.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.650+0000 7fd16cd2c640 1 -- 172.21.15.40:0/195791760 wait complete. 2023-10-18T08:56:36.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.652+0000 7fd16cd2c640 1 Processor -- start 2023-10-18T08:56:36.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.652+0000 7fd16cd2c640 1 -- start start 2023-10-18T08:56:36.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.653+0000 7fd16cd2c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1680cc1a0 0x7fd168159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:36.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.653+0000 7fd16cd2c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1680d1480 0x7fd1681e6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:36.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.653+0000 7fd16cd2c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd16815b760 0x7fd16806b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:36.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.653+0000 7fd16cd2c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16815de80 con 0x7fd1680cc1a0 2023-10-18T08:56:36.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.653+0000 7fd16cd2c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1681e5450 con 0x7fd1680d1480 2023-10-18T08:56:36.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.653+0000 7fd16cd2c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1681e5730 con 0x7fd16815b760 2023-10-18T08:56:36.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.653+0000 7fd16659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1680cc1a0 0x7fd168159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:36.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.653+0000 7fd16659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1680cc1a0 0x7fd168159590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50952/0 (socket says 172.21.15.40:50952) 2023-10-18T08:56:36.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.653+0000 7fd16659c640 1 -- 172.21.15.40:0/686031077 learned_addr learned my addr 172.21.15.40:0/686031077 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:36.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.654+0000 7fd15dd9b640 1 --2- 172.21.15.40:0/686031077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1680d1480 0x7fd1681e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:36.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.654+0000 7fd16659c640 1 -- 172.21.15.40:0/686031077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd16815b760 msgr2=0x7fd16806b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:36.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.654+0000 7fd16659c640 1 --2- 172.21.15.40:0/686031077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd16815b760 0x7fd16806b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.654+0000 7fd16659c640 1 -- 172.21.15.40:0/686031077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1680d1480 msgr2=0x7fd1681e6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.654+0000 7fd16659c640 1 --2- 172.21.15.40:0/686031077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1680d1480 0x7fd1681e6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.654+0000 7fd16659c640 1 -- 172.21.15.40:0/686031077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd160009900 con 0x7fd1680cc1a0 2023-10-18T08:56:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.654+0000 7fd16659c640 1 --2- 172.21.15.40:0/686031077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1680cc1a0 0x7fd168159590 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd15800b3e0 tx=0x7fd15800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.654+0000 7fd15f7fe640 1 -- 172.21.15.40:0/686031077 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd158011830 con 0x7fd1680cc1a0 2023-10-18T08:56:36.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.654+0000 7fd15f7fe640 1 -- 172.21.15.40:0/686031077 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd158011aa0 con 0x7fd1680cc1a0 2023-10-18T08:56:36.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.654+0000 7fd15f7fe640 1 -- 172.21.15.40:0/686031077 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd158010380 con 0x7fd1680cc1a0 2023-10-18T08:56:36.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.654+0000 7fd16cd2c640 1 -- 172.21.15.40:0/686031077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1681e5a10 con 0x7fd1680cc1a0 2023-10-18T08:56:36.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.654+0000 7fd16cd2c640 1 -- 172.21.15.40:0/686031077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd168159ad0 con 0x7fd1680cc1a0 2023-10-18T08:56:36.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.656+0000 7fd16cd2c640 1 -- 172.21.15.40:0/686031077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1680739c0 con 0x7fd1680cc1a0 2023-10-18T08:56:36.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.656+0000 7fd15f7fe640 1 -- 172.21.15.40:0/686031077 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd1580104e0 con 0x7fd1680cc1a0 2023-10-18T08:56:36.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.657+0000 7fd15f7fe640 1 --2- 172.21.15.40:0/686031077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd14806ae40 0x7fd14806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:36.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.657+0000 7fd15f7fe640 1 -- 172.21.15.40:0/686031077 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd158089950 con 0x7fd1680cc1a0 2023-10-18T08:56:36.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.658+0000 7fd15dd9b640 1 --2- 172.21.15.40:0/686031077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd14806ae40 0x7fd14806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:36.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.659+0000 7fd15dd9b640 1 --2- 172.21.15.40:0/686031077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd14806ae40 0x7fd14806d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd1681f4cd0 tx=0x7fd150005c60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:36.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.662+0000 7fd15f7fe640 1 -- 172.21.15.40:0/686031077 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd158054a90 con 0x7fd1680cc1a0 2023-10-18T08:56:36.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.866+0000 7fd16cd2c640 1 -- 172.21.15.40:0/686031077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1681e5f30 con 0x7fd1680cc1a0 2023-10-18T08:56:36.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.868+0000 7fd15f7fe640 1 -- 172.21.15.40:0/686031077 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16581 (secure 0 0 0) 0x7fd158015090 con 0x7fd1680cc1a0 2023-10-18T08:56:36.870 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:36.871 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":33,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:33.724325+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:56:30.127184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-18T08:56:31.225270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:36.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.870+0000 7fd16cd2c640 1 -- 172.21.15.40:0/686031077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd14806ae40 msgr2=0x7fd14806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.870+0000 7fd16cd2c640 1 --2- 172.21.15.40:0/686031077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd14806ae40 0x7fd14806d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd1681f4cd0 tx=0x7fd150005c60 comp rx=0 tx=0).stop 2023-10-18T08:56:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.870+0000 7fd16cd2c640 1 -- 172.21.15.40:0/686031077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1680cc1a0 msgr2=0x7fd168159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.870+0000 7fd16cd2c640 1 --2- 172.21.15.40:0/686031077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1680cc1a0 0x7fd168159590 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd15800b3e0 tx=0x7fd15800b8a0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.870+0000 7fd16cd2c640 1 -- 172.21.15.40:0/686031077 shutdown_connections 2023-10-18T08:56:36.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.870+0000 7fd16cd2c640 1 --2- 172.21.15.40:0/686031077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd14806ae40 0x7fd14806d300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.870+0000 7fd16cd2c640 1 --2- 172.21.15.40:0/686031077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd16815b760 0x7fd16806b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.870+0000 7fd16cd2c640 1 --2- 172.21.15.40:0/686031077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1680d1480 0x7fd1681e6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.870+0000 7fd16cd2c640 1 --2- 172.21.15.40:0/686031077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1680cc1a0 0x7fd168159590 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:36.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.870+0000 7fd16cd2c640 1 -- 172.21.15.40:0/686031077 >> 172.21.15.40:0/686031077 conn(0x7fd16809a9a0 msgr2=0x7fd16815a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:36.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.870+0000 7fd16cd2c640 1 -- 172.21.15.40:0/686031077 shutdown_connections 2023-10-18T08:56:36.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:36.870+0000 7fd16cd2c640 1 -- 172.21.15.40:0/686031077 wait complete. 2023-10-18T08:56:36.890 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T08:56:36.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:56:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.095+0000 7f6af360e640 1 -- 172.21.15.40:0/628873773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ad8003970 msgr2=0x7f6ad8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.095+0000 7f6af360e640 1 --2- 172.21.15.40:0/628873773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ad8003970 0x7f6ad8003d90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8006fd0 tx=0x7f6ad80057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.095+0000 7f6af1bab640 1 -- 172.21.15.40:0/628873773 reap_dead start 2023-10-18T08:56:37.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.095+0000 7f6af360e640 1 -- 172.21.15.40:0/628873773 shutdown_connections 2023-10-18T08:56:37.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.095+0000 7f6af360e640 1 -- 172.21.15.40:0/628873773 >> 172.21.15.40:0/628873773 conn(0x7f6aec06cee0 msgr2=0x7f6aec069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:37.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.095+0000 7f6af360e640 1 -- 172.21.15.40:0/628873773 shutdown_connections 2023-10-18T08:56:37.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.095+0000 7f6af360e640 1 -- 172.21.15.40:0/628873773 wait complete. 2023-10-18T08:56:37.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.098+0000 7f6af360e640 1 Processor -- start 2023-10-18T08:56:37.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.098+0000 7f6af360e640 1 -- start start 2023-10-18T08:56:37.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.099+0000 7f6af360e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad8003970 0x7f6aec13eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:37.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.099+0000 7f6af360e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aec150b30 0x7f6aec150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:37.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.099+0000 7f6af360e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec152ea0 0x7f6aec153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:37.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.099+0000 7f6af360e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec1514f0 con 0x7f6aec152ea0 2023-10-18T08:56:37.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.099+0000 7f6af360e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec153860 con 0x7f6ad8003970 2023-10-18T08:56:37.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.099+0000 7f6af360e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec152300 con 0x7f6aec150b30 2023-10-18T08:56:37.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.099+0000 7f6af1bab640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec152ea0 0x7f6aec153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:37.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.099+0000 7f6af1bab640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec152ea0 0x7f6aec153320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50958/0 (socket says 172.21.15.40:50958) 2023-10-18T08:56:37.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.099+0000 7f6af1bab640 1 -- 172.21.15.40:0/2768528796 learned_addr learned my addr 172.21.15.40:0/2768528796 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:37.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.100+0000 7f6af1bab640 1 -- 172.21.15.40:0/2768528796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aec150b30 msgr2=0x7f6aec150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:37.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.100+0000 7f6af1bab640 1 --2- 172.21.15.40:0/2768528796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aec150b30 0x7f6aec150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:37.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.100+0000 7f6af13aa640 1 --2- 172.21.15.40:0/2768528796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad8003970 0x7f6aec13eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:37.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.100+0000 7f6af1bab640 1 -- 172.21.15.40:0/2768528796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad8003970 msgr2=0x7f6aec13eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:37.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.100+0000 7f6af1bab640 1 --2- 172.21.15.40:0/2768528796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad8003970 0x7f6aec13eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:37.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.100+0000 7f6af1bab640 1 -- 172.21.15.40:0/2768528796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec149190 con 0x7f6aec152ea0 2023-10-18T08:56:37.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.100+0000 7f6af1bab640 1 --2- 172.21.15.40:0/2768528796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec152ea0 0x7f6aec153320 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005fd0 tx=0x7f6ae400ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:37.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.100+0000 7f6ae1ffb640 1 -- 172.21.15.40:0/2768528796 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4018070 con 0x7f6aec152ea0 2023-10-18T08:56:37.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.100+0000 7f6ae1ffb640 1 -- 172.21.15.40:0/2768528796 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6ae4004b20 con 0x7f6aec152ea0 2023-10-18T08:56:37.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.101+0000 7f6ae1ffb640 1 -- 172.21.15.40:0/2768528796 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae400ca00 con 0x7f6aec152ea0 2023-10-18T08:56:37.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.101+0000 7f6af360e640 1 -- 172.21.15.40:0/2768528796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec1516f0 con 0x7f6aec152ea0 2023-10-18T08:56:37.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.101+0000 7f6af360e640 1 -- 172.21.15.40:0/2768528796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec151cb0 con 0x7f6aec152ea0 2023-10-18T08:56:37.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.102+0000 7f6af360e640 1 -- 172.21.15.40:0/2768528796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aec06a5a0 con 0x7f6aec152ea0 2023-10-18T08:56:37.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.103+0000 7f6ae1ffb640 1 -- 172.21.15.40:0/2768528796 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6ae400cb60 con 0x7f6aec152ea0 2023-10-18T08:56:37.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.103+0000 7f6ae1ffb640 1 --2- 172.21.15.40:0/2768528796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ad006ae40 0x7f6ad006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:37.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.104+0000 7f6ae1ffb640 1 -- 172.21.15.40:0/2768528796 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ae4089540 con 0x7f6aec152ea0 2023-10-18T08:56:37.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.104+0000 7f6af13aa640 1 --2- 172.21.15.40:0/2768528796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ad006ae40 0x7f6ad006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:37.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.105+0000 7f6af13aa640 1 --2- 172.21.15.40:0/2768528796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ad006ae40 0x7f6ad006d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8034000 tx=0x7f6ae8001a70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:37.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.108+0000 7f6ae1ffb640 1 -- 172.21.15.40:0/2768528796 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6ae4054550 con 0x7f6aec152ea0 2023-10-18T08:56:37.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.292+0000 7f6af360e640 1 -- 172.21.15.40:0/2768528796 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6aec0c9e00 con 0x7f6ad006ae40 2023-10-18T08:56:37.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.320+0000 7f6ae1ffb640 1 -- 172.21.15.40:0/2768528796 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282473 (secure 0 0 0) 0x7f6ae8006050 con 0x7f6ad006ae40 2023-10-18T08:56:37.326 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:37.328 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:56:37.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.326+0000 7f6af360e640 1 -- 172.21.15.40:0/2768528796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ad006ae40 msgr2=0x7f6ad006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:37.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.326+0000 7f6af360e640 1 --2- 172.21.15.40:0/2768528796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ad006ae40 0x7f6ad006d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8034000 tx=0x7f6ae8001a70 comp rx=0 tx=0).stop 2023-10-18T08:56:37.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.326+0000 7f6af360e640 1 -- 172.21.15.40:0/2768528796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec152ea0 msgr2=0x7f6aec153320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:37.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.326+0000 7f6af360e640 1 --2- 172.21.15.40:0/2768528796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec152ea0 0x7f6aec153320 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005fd0 tx=0x7f6ae400ed20 comp rx=0 tx=0).stop 2023-10-18T08:56:37.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.326+0000 7f6af360e640 1 -- 172.21.15.40:0/2768528796 shutdown_connections 2023-10-18T08:56:37.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.326+0000 7f6af360e640 1 --2- 172.21.15.40:0/2768528796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ad006ae40 0x7f6ad006d300 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:37.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.326+0000 7f6af360e640 1 --2- 172.21.15.40:0/2768528796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec152ea0 0x7f6aec153320 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:37.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.326+0000 7f6af360e640 1 --2- 172.21.15.40:0/2768528796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aec150b30 0x7f6aec150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:37.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.326+0000 7f6af360e640 1 --2- 172.21.15.40:0/2768528796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad8003970 0x7f6aec13eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:37.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.326+0000 7f6af360e640 1 -- 172.21.15.40:0/2768528796 >> 172.21.15.40:0/2768528796 conn(0x7f6aec06cee0 msgr2=0x7f6aec15b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:37.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.327+0000 7f6af360e640 1 -- 172.21.15.40:0/2768528796 shutdown_connections 2023-10-18T08:56:37.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.327+0000 7f6af360e640 1 -- 172.21.15.40:0/2768528796 wait complete. 2023-10-18T08:56:37.347 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2023-10-18T08:56:36.922967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2264,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560235,"statfs":{"total":1600000000,"available":1597681157,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002154"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.895861+0000","last_change":"2023-10-18T08:56:32.753645+0000","last_active":"2023-10-18T08:56:33.895861+0000","last_peered":"2023-10-18T08:56:33.895861+0000","last_clean":"2023-10-18T08:56:33.895861+0000","last_became_active":"2023-10-18T08:56:32.753349+0000","last_became_peered":"2023-10-18T08:56:32.753349+0000","last_unstale":"2023-10-18T08:56:33.895861+0000","last_undegraded":"2023-10-18T08:56:33.895861+0000","last_fullsized":"2023-10-18T08:56:33.895861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:53:51.600283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730113+0000","last_change":"2023-10-18T08:56:32.764736+0000","last_active":"2023-10-18T08:56:33.730113+0000","last_peered":"2023-10-18T08:56:33.730113+0000","last_clean":"2023-10-18T08:56:33.730113+0000","last_became_active":"2023-10-18T08:56:32.764429+0000","last_became_peered":"2023-10-18T08:56:32.764429+0000","last_unstale":"2023-10-18T08:56:33.730113+0000","last_undegraded":"2023-10-18T08:56:33.730113+0000","last_fullsized":"2023-10-18T08:56:33.730113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:17:05.996948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898868+0000","last_change":"2023-10-18T08:56:32.753507+0000","last_active":"2023-10-18T08:56:33.898868+0000","last_peered":"2023-10-18T08:56:33.898868+0000","last_clean":"2023-10-18T08:56:33.898868+0000","last_became_active":"2023-10-18T08:56:32.752727+0000","last_became_peered":"2023-10-18T08:56:32.752727+0000","last_unstale":"2023-10-18T08:56:33.898868+0000","last_undegraded":"2023-10-18T08:56:33.898868+0000","last_fullsized":"2023-10-18T08:56:33.898868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:54.633588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.735041+0000","last_change":"2023-10-18T08:56:32.756278+0000","last_active":"2023-10-18T08:56:33.735041+0000","last_peered":"2023-10-18T08:56:33.735041+0000","last_clean":"2023-10-18T08:56:33.735041+0000","last_became_active":"2023-10-18T08:56:32.755805+0000","last_became_peered":"2023-10-18T08:56:32.755805+0000","last_unstale":"2023-10-18T08:56:33.735041+0000","last_undegraded":"2023-10-18T08:56:33.735041+0000","last_fullsized":"2023-10-18T08:56:33.735041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:03:31.460178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894667+0000","last_change":"2023-10-18T08:56:32.760311+0000","last_active":"2023-10-18T08:56:33.894667+0000","last_peered":"2023-10-18T08:56:33.894667+0000","last_clean":"2023-10-18T08:56:33.894667+0000","last_became_active":"2023-10-18T08:56:32.760054+0000","last_became_peered":"2023-10-18T08:56:32.760054+0000","last_unstale":"2023-10-18T08:56:33.894667+0000","last_undegraded":"2023-10-18T08:56:33.894667+0000","last_fullsized":"2023-10-18T08:56:33.894667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:31.691404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730700+0000","last_change":"2023-10-18T08:56:32.755115+0000","last_active":"2023-10-18T08:56:33.730700+0000","last_peered":"2023-10-18T08:56:33.730700+0000","last_clean":"2023-10-18T08:56:33.730700+0000","last_became_active":"2023-10-18T08:56:32.754534+0000","last_became_peered":"2023-10-18T08:56:32.754534+0000","last_unstale":"2023-10-18T08:56:33.730700+0000","last_undegraded":"2023-10-18T08:56:33.730700+0000","last_fullsized":"2023-10-18T08:56:33.730700+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:21:01.706092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898490+0000","last_change":"2023-10-18T08:56:32.749058+0000","last_active":"2023-10-18T08:56:33.898490+0000","last_peered":"2023-10-18T08:56:33.898490+0000","last_clean":"2023-10-18T08:56:33.898490+0000","last_became_active":"2023-10-18T08:56:32.747939+0000","last_became_peered":"2023-10-18T08:56:32.747939+0000","last_unstale":"2023-10-18T08:56:33.898490+0000","last_undegraded":"2023-10-18T08:56:33.898490+0000","last_fullsized":"2023-10-18T08:56:33.898490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:17:05.996948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.896726+0000","last_change":"2023-10-18T08:56:32.749127+0000","last_active":"2023-10-18T08:56:33.896726+0000","last_peered":"2023-10-18T08:56:33.896726+0000","last_clean":"2023-10-18T08:56:33.896726+0000","last_became_active":"2023-10-18T08:56:32.748595+0000","last_became_peered":"2023-10-18T08:56:32.748595+0000","last_unstale":"2023-10-18T08:56:33.896726+0000","last_undegraded":"2023-10-18T08:56:33.896726+0000","last_fullsized":"2023-10-18T08:56:33.896726+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:41.017146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897819+0000","last_change":"2023-10-18T08:56:32.756936+0000","last_active":"2023-10-18T08:56:33.897819+0000","last_peered":"2023-10-18T08:56:33.897819+0000","last_clean":"2023-10-18T08:56:33.897819+0000","last_became_active":"2023-10-18T08:56:32.756495+0000","last_became_peered":"2023-10-18T08:56:32.756495+0000","last_unstale":"2023-10-18T08:56:33.897819+0000","last_undegraded":"2023-10-18T08:56:33.897819+0000","last_fullsized":"2023-10-18T08:56:33.897819+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:24.954589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.735528+0000","last_change":"2023-10-18T08:56:32.754846+0000","last_active":"2023-10-18T08:56:33.735528+0000","last_peered":"2023-10-18T08:56:33.735528+0000","last_clean":"2023-10-18T08:56:33.735528+0000","last_became_active":"2023-10-18T08:56:32.754271+0000","last_became_peered":"2023-10-18T08:56:32.754271+0000","last_unstale":"2023-10-18T08:56:33.735528+0000","last_undegraded":"2023-10-18T08:56:33.735528+0000","last_fullsized":"2023-10-18T08:56:33.735528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:43.146105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897411+0000","last_change":"2023-10-18T08:56:32.756696+0000","last_active":"2023-10-18T08:56:33.897411+0000","last_peered":"2023-10-18T08:56:33.897411+0000","last_clean":"2023-10-18T08:56:33.897411+0000","last_became_active":"2023-10-18T08:56:32.756425+0000","last_became_peered":"2023-10-18T08:56:32.756425+0000","last_unstale":"2023-10-18T08:56:33.897411+0000","last_undegraded":"2023-10-18T08:56:33.897411+0000","last_fullsized":"2023-10-18T08:56:33.897411+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:43.146105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899004+0000","last_change":"2023-10-18T08:56:32.758579+0000","last_active":"2023-10-18T08:56:33.899004+0000","last_peered":"2023-10-18T08:56:33.899004+0000","last_clean":"2023-10-18T08:56:33.899004+0000","last_became_active":"2023-10-18T08:56:32.758178+0000","last_became_peered":"2023-10-18T08:56:32.758178+0000","last_unstale":"2023-10-18T08:56:33.899004+0000","last_undegraded":"2023-10-18T08:56:33.899004+0000","last_fullsized":"2023-10-18T08:56:33.899004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:34.532079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894064+0000","last_change":"2023-10-18T08:56:32.757310+0000","last_active":"2023-10-18T08:56:33.894064+0000","last_peered":"2023-10-18T08:56:33.894064+0000","last_clean":"2023-10-18T08:56:33.894064+0000","last_became_active":"2023-10-18T08:56:32.756825+0000","last_became_peered":"2023-10-18T08:56:32.756825+0000","last_unstale":"2023-10-18T08:56:33.894064+0000","last_undegraded":"2023-10-18T08:56:33.894064+0000","last_fullsized":"2023-10-18T08:56:33.894064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:40.241394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730946+0000","last_change":"2023-10-18T08:56:32.751825+0000","last_active":"2023-10-18T08:56:33.730946+0000","last_peered":"2023-10-18T08:56:33.730946+0000","last_clean":"2023-10-18T08:56:33.730946+0000","last_became_active":"2023-10-18T08:56:32.751338+0000","last_became_peered":"2023-10-18T08:56:32.751338+0000","last_unstale":"2023-10-18T08:56:33.730946+0000","last_undegraded":"2023-10-18T08:56:33.730946+0000","last_fullsized":"2023-10-18T08:56:33.730946+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:41.097642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.900017+0000","last_change":"2023-10-18T08:56:32.754683+0000","last_active":"2023-10-18T08:56:33.900017+0000","last_peered":"2023-10-18T08:56:33.900017+0000","last_clean":"2023-10-18T08:56:33.900017+0000","last_became_active":"2023-10-18T08:56:32.754425+0000","last_became_peered":"2023-10-18T08:56:32.754425+0000","last_unstale":"2023-10-18T08:56:33.900017+0000","last_undegraded":"2023-10-18T08:56:33.900017+0000","last_fullsized":"2023-10-18T08:56:33.900017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:38.281069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897738+0000","last_change":"2023-10-18T08:56:32.737915+0000","last_active":"2023-10-18T08:56:33.897738+0000","last_peered":"2023-10-18T08:56:33.897738+0000","last_clean":"2023-10-18T08:56:33.897738+0000","last_became_active":"2023-10-18T08:56:32.737477+0000","last_became_peered":"2023-10-18T08:56:32.737477+0000","last_unstale":"2023-10-18T08:56:33.897738+0000","last_undegraded":"2023-10-18T08:56:33.897738+0000","last_fullsized":"2023-10-18T08:56:33.897738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:34.532079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897839+0000","last_change":"2023-10-18T08:56:32.749232+0000","last_active":"2023-10-18T08:56:33.897839+0000","last_peered":"2023-10-18T08:56:33.897839+0000","last_clean":"2023-10-18T08:56:33.897839+0000","last_became_active":"2023-10-18T08:56:32.748383+0000","last_became_peered":"2023-10-18T08:56:32.748383+0000","last_unstale":"2023-10-18T08:56:33.897839+0000","last_undegraded":"2023-10-18T08:56:33.897839+0000","last_fullsized":"2023-10-18T08:56:33.897839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:30:27.859747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730344+0000","last_change":"2023-10-18T08:56:32.756729+0000","last_active":"2023-10-18T08:56:33.730344+0000","last_peered":"2023-10-18T08:56:33.730344+0000","last_clean":"2023-10-18T08:56:33.730344+0000","last_became_active":"2023-10-18T08:56:32.756311+0000","last_became_peered":"2023-10-18T08:56:32.756311+0000","last_unstale":"2023-10-18T08:56:33.730344+0000","last_undegraded":"2023-10-18T08:56:33.730344+0000","last_fullsized":"2023-10-18T08:56:33.730344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:38.281069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730050+0000","last_change":"2023-10-18T08:56:32.758356+0000","last_active":"2023-10-18T08:56:33.730050+0000","last_peered":"2023-10-18T08:56:33.730050+0000","last_clean":"2023-10-18T08:56:33.730050+0000","last_became_active":"2023-10-18T08:56:32.757860+0000","last_became_peered":"2023-10-18T08:56:32.757860+0000","last_unstale":"2023-10-18T08:56:33.730050+0000","last_undegraded":"2023-10-18T08:56:33.730050+0000","last_fullsized":"2023-10-18T08:56:33.730050+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:58.348622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894888+0000","last_change":"2023-10-18T08:56:32.753658+0000","last_active":"2023-10-18T08:56:33.894888+0000","last_peered":"2023-10-18T08:56:33.894888+0000","last_clean":"2023-10-18T08:56:33.894888+0000","last_became_active":"2023-10-18T08:56:32.752555+0000","last_became_peered":"2023-10-18T08:56:32.752555+0000","last_unstale":"2023-10-18T08:56:33.894888+0000","last_undegraded":"2023-10-18T08:56:33.894888+0000","last_fullsized":"2023-10-18T08:56:33.894888+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:53:51.600283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898523+0000","last_change":"2023-10-18T08:56:32.764321+0000","last_active":"2023-10-18T08:56:33.898523+0000","last_peered":"2023-10-18T08:56:33.898523+0000","last_clean":"2023-10-18T08:56:33.898523+0000","last_became_active":"2023-10-18T08:56:32.764095+0000","last_became_peered":"2023-10-18T08:56:32.764095+0000","last_unstale":"2023-10-18T08:56:33.898523+0000","last_undegraded":"2023-10-18T08:56:33.898523+0000","last_fullsized":"2023-10-18T08:56:33.898523+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:46.641699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898759+0000","last_change":"2023-10-18T08:56:32.739350+0000","last_active":"2023-10-18T08:56:33.898759+0000","last_peered":"2023-10-18T08:56:33.898759+0000","last_clean":"2023-10-18T08:56:33.898759+0000","last_became_active":"2023-10-18T08:56:32.738980+0000","last_became_peered":"2023-10-18T08:56:32.738980+0000","last_unstale":"2023-10-18T08:56:33.898759+0000","last_undegraded":"2023-10-18T08:56:33.898759+0000","last_fullsized":"2023-10-18T08:56:33.898759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:53.992426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.734396+0000","last_change":"2023-10-18T08:56:32.765823+0000","last_active":"2023-10-18T08:56:33.734396+0000","last_peered":"2023-10-18T08:56:33.734396+0000","last_clean":"2023-10-18T08:56:33.734396+0000","last_became_active":"2023-10-18T08:56:32.765470+0000","last_became_peered":"2023-10-18T08:56:32.765470+0000","last_unstale":"2023-10-18T08:56:33.734396+0000","last_undegraded":"2023-10-18T08:56:33.734396+0000","last_fullsized":"2023-10-18T08:56:33.734396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:15.179607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899718+0000","last_change":"2023-10-18T08:56:32.755055+0000","last_active":"2023-10-18T08:56:33.899718+0000","last_peered":"2023-10-18T08:56:33.899718+0000","last_clean":"2023-10-18T08:56:33.899718+0000","last_became_active":"2023-10-18T08:56:32.753565+0000","last_became_peered":"2023-10-18T08:56:32.753565+0000","last_unstale":"2023-10-18T08:56:33.899718+0000","last_undegraded":"2023-10-18T08:56:33.899718+0000","last_fullsized":"2023-10-18T08:56:33.899718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:17:05.996948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.734871+0000","last_change":"2023-10-18T08:56:32.759339+0000","last_active":"2023-10-18T08:56:33.734871+0000","last_peered":"2023-10-18T08:56:33.734871+0000","last_clean":"2023-10-18T08:56:33.734871+0000","last_became_active":"2023-10-18T08:56:32.758800+0000","last_became_peered":"2023-10-18T08:56:32.758800+0000","last_unstale":"2023-10-18T08:56:33.734871+0000","last_undegraded":"2023-10-18T08:56:33.734871+0000","last_fullsized":"2023-10-18T08:56:33.734871+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:46.641699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.731027+0000","last_change":"2023-10-18T08:56:32.738926+0000","last_active":"2023-10-18T08:56:33.731027+0000","last_peered":"2023-10-18T08:56:33.731027+0000","last_clean":"2023-10-18T08:56:33.731027+0000","last_became_active":"2023-10-18T08:56:32.738061+0000","last_became_peered":"2023-10-18T08:56:32.738061+0000","last_unstale":"2023-10-18T08:56:33.731027+0000","last_undegraded":"2023-10-18T08:56:33.731027+0000","last_fullsized":"2023-10-18T08:56:33.731027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:03.028759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.736889+0000","last_change":"2023-10-18T08:56:32.762596+0000","last_active":"2023-10-18T08:56:33.736889+0000","last_peered":"2023-10-18T08:56:33.736889+0000","last_clean":"2023-10-18T08:56:33.736889+0000","last_became_active":"2023-10-18T08:56:32.759815+0000","last_became_peered":"2023-10-18T08:56:32.759815+0000","last_unstale":"2023-10-18T08:56:33.736889+0000","last_undegraded":"2023-10-18T08:56:33.736889+0000","last_fullsized":"2023-10-18T08:56:33.736889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:30:27.859747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897995+0000","last_change":"2023-10-18T08:56:32.766204+0000","last_active":"2023-10-18T08:56:33.897995+0000","last_peered":"2023-10-18T08:56:33.897995+0000","last_clean":"2023-10-18T08:56:33.897995+0000","last_became_active":"2023-10-18T08:56:32.765917+0000","last_became_peered":"2023-10-18T08:56:32.765917+0000","last_unstale":"2023-10-18T08:56:33.897995+0000","last_undegraded":"2023-10-18T08:56:33.897995+0000","last_fullsized":"2023-10-18T08:56:33.897995+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:53.992426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898126+0000","last_change":"2023-10-18T08:56:32.740545+0000","last_active":"2023-10-18T08:56:33.898126+0000","last_peered":"2023-10-18T08:56:33.898126+0000","last_clean":"2023-10-18T08:56:33.898126+0000","last_became_active":"2023-10-18T08:56:32.739922+0000","last_became_peered":"2023-10-18T08:56:32.739922+0000","last_unstale":"2023-10-18T08:56:33.898126+0000","last_undegraded":"2023-10-18T08:56:33.898126+0000","last_fullsized":"2023-10-18T08:56:33.898126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:03:31.460178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899205+0000","last_change":"2023-10-18T08:56:32.755907+0000","last_active":"2023-10-18T08:56:33.899205+0000","last_peered":"2023-10-18T08:56:33.899205+0000","last_clean":"2023-10-18T08:56:33.899205+0000","last_became_active":"2023-10-18T08:56:32.753213+0000","last_became_peered":"2023-10-18T08:56:32.753213+0000","last_unstale":"2023-10-18T08:56:33.899205+0000","last_undegraded":"2023-10-18T08:56:33.899205+0000","last_fullsized":"2023-10-18T08:56:33.899205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:38.281069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.896171+0000","last_change":"2023-10-18T08:56:32.756402+0000","last_active":"2023-10-18T08:56:33.896171+0000","last_peered":"2023-10-18T08:56:33.896171+0000","last_clean":"2023-10-18T08:56:33.896171+0000","last_became_active":"2023-10-18T08:56:32.756114+0000","last_became_peered":"2023-10-18T08:56:32.756114+0000","last_unstale":"2023-10-18T08:56:33.896171+0000","last_undegraded":"2023-10-18T08:56:33.896171+0000","last_fullsized":"2023-10-18T08:56:33.896171+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:35:08.254437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898145+0000","last_change":"2023-10-18T08:56:32.747103+0000","last_active":"2023-10-18T08:56:33.898145+0000","last_peered":"2023-10-18T08:56:33.898145+0000","last_clean":"2023-10-18T08:56:33.898145+0000","last_became_active":"2023-10-18T08:56:32.744772+0000","last_became_peered":"2023-10-18T08:56:32.744772+0000","last_unstale":"2023-10-18T08:56:33.898145+0000","last_undegraded":"2023-10-18T08:56:33.898145+0000","last_fullsized":"2023-10-18T08:56:33.898145+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:58.348622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.732272+0000","last_change":"2023-10-18T08:56:32.766097+0000","last_active":"2023-10-18T08:56:33.732272+0000","last_peered":"2023-10-18T08:56:33.732272+0000","last_clean":"2023-10-18T08:56:33.732272+0000","last_became_active":"2023-10-18T08:56:32.765353+0000","last_became_peered":"2023-10-18T08:56:32.765353+0000","last_unstale":"2023-10-18T08:56:33.732272+0000","last_undegraded":"2023-10-18T08:56:33.732272+0000","last_fullsized":"2023-10-18T08:56:33.732272+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:34.808391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897247+0000","last_change":"2023-10-18T08:56:31.734202+0000","last_active":"2023-10-18T08:56:33.897247+0000","last_peered":"2023-10-18T08:56:33.897247+0000","last_clean":"2023-10-18T08:56:33.897247+0000","last_became_active":"2023-10-18T08:56:31.733975+0000","last_became_peered":"2023-10-18T08:56:31.733975+0000","last_unstale":"2023-10-18T08:56:33.897247+0000","last_undegraded":"2023-10-18T08:56:33.897247+0000","last_fullsized":"2023-10-18T08:56:33.897247+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:31.408555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897005+0000","last_change":"2023-10-18T08:56:32.756370+0000","last_active":"2023-10-18T08:56:33.897005+0000","last_peered":"2023-10-18T08:56:33.897005+0000","last_clean":"2023-10-18T08:56:33.897005+0000","last_became_active":"2023-10-18T08:56:32.756114+0000","last_became_peered":"2023-10-18T08:56:32.756114+0000","last_unstale":"2023-10-18T08:56:33.897005+0000","last_undegraded":"2023-10-18T08:56:33.897005+0000","last_fullsized":"2023-10-18T08:56:33.897005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:19.090190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.731918+0000","last_change":"2023-10-18T08:56:31.725969+0000","last_active":"2023-10-18T08:56:33.731918+0000","last_peered":"2023-10-18T08:56:33.731918+0000","last_clean":"2023-10-18T08:56:33.731918+0000","last_became_active":"2023-10-18T08:56:31.725551+0000","last_became_peered":"2023-10-18T08:56:31.725551+0000","last_unstale":"2023-10-18T08:56:33.731918+0000","last_undegraded":"2023-10-18T08:56:33.731918+0000","last_fullsized":"2023-10-18T08:56:33.731918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:30.677707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898896+0000","last_change":"2023-10-18T08:56:32.747635+0000","last_active":"2023-10-18T08:56:33.898896+0000","last_peered":"2023-10-18T08:56:33.898896+0000","last_clean":"2023-10-18T08:56:33.898896+0000","last_became_active":"2023-10-18T08:56:32.745274+0000","last_became_peered":"2023-10-18T08:56:32.745274+0000","last_unstale":"2023-10-18T08:56:33.898896+0000","last_undegraded":"2023-10-18T08:56:33.898896+0000","last_fullsized":"2023-10-18T08:56:33.898896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:43.146105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730918+0000","last_change":"2023-10-18T08:56:31.720258+0000","last_active":"2023-10-18T08:56:33.730918+0000","last_peered":"2023-10-18T08:56:33.730918+0000","last_clean":"2023-10-18T08:56:33.730918+0000","last_became_active":"2023-10-18T08:56:31.720050+0000","last_became_peered":"2023-10-18T08:56:31.720050+0000","last_unstale":"2023-10-18T08:56:33.730918+0000","last_undegraded":"2023-10-18T08:56:33.730918+0000","last_fullsized":"2023-10-18T08:56:33.730918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:21.749663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.735200+0000","last_change":"2023-10-18T08:56:32.758132+0000","last_active":"2023-10-18T08:56:33.735200+0000","last_peered":"2023-10-18T08:56:33.735200+0000","last_clean":"2023-10-18T08:56:33.735200+0000","last_became_active":"2023-10-18T08:56:32.756996+0000","last_became_peered":"2023-10-18T08:56:32.756996+0000","last_unstale":"2023-10-18T08:56:33.735200+0000","last_undegraded":"2023-10-18T08:56:33.735200+0000","last_fullsized":"2023-10-18T08:56:33.735200+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:41.017146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894902+0000","last_change":"2023-10-18T08:56:31.719392+0000","last_active":"2023-10-18T08:56:33.894902+0000","last_peered":"2023-10-18T08:56:33.894902+0000","last_clean":"2023-10-18T08:56:33.894902+0000","last_became_active":"2023-10-18T08:56:31.719094+0000","last_became_peered":"2023-10-18T08:56:31.719094+0000","last_unstale":"2023-10-18T08:56:33.894902+0000","last_undegraded":"2023-10-18T08:56:33.894902+0000","last_fullsized":"2023-10-18T08:56:33.894902+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:51:25.842852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897200+0000","last_change":"2023-10-18T08:56:32.747036+0000","last_active":"2023-10-18T08:56:33.897200+0000","last_peered":"2023-10-18T08:56:33.897200+0000","last_clean":"2023-10-18T08:56:33.897200+0000","last_became_active":"2023-10-18T08:56:32.745877+0000","last_became_peered":"2023-10-18T08:56:32.745877+0000","last_unstale":"2023-10-18T08:56:33.897200+0000","last_undegraded":"2023-10-18T08:56:33.897200+0000","last_fullsized":"2023-10-18T08:56:33.897200+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:53.992426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898408+0000","last_change":"2023-10-18T08:56:31.732775+0000","last_active":"2023-10-18T08:56:33.898408+0000","last_peered":"2023-10-18T08:56:33.898408+0000","last_clean":"2023-10-18T08:56:33.898408+0000","last_became_active":"2023-10-18T08:56:31.732333+0000","last_became_peered":"2023-10-18T08:56:31.732333+0000","last_unstale":"2023-10-18T08:56:33.898408+0000","last_undegraded":"2023-10-18T08:56:33.898408+0000","last_fullsized":"2023-10-18T08:56:33.898408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:00:26.389152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899317+0000","last_change":"2023-10-18T08:56:32.765589+0000","last_active":"2023-10-18T08:56:33.899317+0000","last_peered":"2023-10-18T08:56:33.899317+0000","last_clean":"2023-10-18T08:56:33.899317+0000","last_became_active":"2023-10-18T08:56:32.765264+0000","last_became_peered":"2023-10-18T08:56:32.765264+0000","last_unstale":"2023-10-18T08:56:33.899317+0000","last_undegraded":"2023-10-18T08:56:33.899317+0000","last_fullsized":"2023-10-18T08:56:33.899317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:30:27.859747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898064+0000","last_change":"2023-10-18T08:56:31.730226+0000","last_active":"2023-10-18T08:56:33.898064+0000","last_peered":"2023-10-18T08:56:33.898064+0000","last_clean":"2023-10-18T08:56:33.898064+0000","last_became_active":"2023-10-18T08:56:31.729915+0000","last_became_peered":"2023-10-18T08:56:31.729915+0000","last_unstale":"2023-10-18T08:56:33.898064+0000","last_undegraded":"2023-10-18T08:56:33.898064+0000","last_fullsized":"2023-10-18T08:56:33.898064+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:30:32.002656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898705+0000","last_change":"2023-10-18T08:56:32.755938+0000","last_active":"2023-10-18T08:56:33.898705+0000","last_peered":"2023-10-18T08:56:33.898705+0000","last_clean":"2023-10-18T08:56:33.898705+0000","last_became_active":"2023-10-18T08:56:32.754686+0000","last_became_peered":"2023-10-18T08:56:32.754686+0000","last_unstale":"2023-10-18T08:56:33.898705+0000","last_undegraded":"2023-10-18T08:56:33.898705+0000","last_fullsized":"2023-10-18T08:56:33.898705+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:41.017146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.731163+0000","last_change":"2023-10-18T08:56:31.709122+0000","last_active":"2023-10-18T08:56:33.731163+0000","last_peered":"2023-10-18T08:56:33.731163+0000","last_clean":"2023-10-18T08:56:33.731163+0000","last_became_active":"2023-10-18T08:56:31.708306+0000","last_became_peered":"2023-10-18T08:56:31.708306+0000","last_unstale":"2023-10-18T08:56:33.731163+0000","last_undegraded":"2023-10-18T08:56:33.731163+0000","last_fullsized":"2023-10-18T08:56:33.731163+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:40.003448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.735387+0000","last_change":"2023-10-18T08:56:32.755223+0000","last_active":"2023-10-18T08:56:33.735387+0000","last_peered":"2023-10-18T08:56:33.735387+0000","last_clean":"2023-10-18T08:56:33.735387+0000","last_became_active":"2023-10-18T08:56:32.752298+0000","last_became_peered":"2023-10-18T08:56:32.752298+0000","last_unstale":"2023-10-18T08:56:33.735387+0000","last_undegraded":"2023-10-18T08:56:33.735387+0000","last_fullsized":"2023-10-18T08:56:33.735387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:53.992426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899377+0000","last_change":"2023-10-18T08:56:31.732204+0000","last_active":"2023-10-18T08:56:33.899377+0000","last_peered":"2023-10-18T08:56:33.899377+0000","last_clean":"2023-10-18T08:56:33.899377+0000","last_became_active":"2023-10-18T08:56:31.731739+0000","last_became_peered":"2023-10-18T08:56:31.731739+0000","last_unstale":"2023-10-18T08:56:33.899377+0000","last_undegraded":"2023-10-18T08:56:33.899377+0000","last_fullsized":"2023-10-18T08:56:33.899377+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:35:07.240740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.900125+0000","last_change":"2023-10-18T08:56:32.758417+0000","last_active":"2023-10-18T08:56:33.900125+0000","last_peered":"2023-10-18T08:56:33.900125+0000","last_clean":"2023-10-18T08:56:33.900125+0000","last_became_active":"2023-10-18T08:56:32.757491+0000","last_became_peered":"2023-10-18T08:56:32.757491+0000","last_unstale":"2023-10-18T08:56:33.900125+0000","last_undegraded":"2023-10-18T08:56:33.900125+0000","last_fullsized":"2023-10-18T08:56:33.900125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:44:45.934778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.731555+0000","last_change":"2023-10-18T08:56:31.723089+0000","last_active":"2023-10-18T08:56:33.731555+0000","last_peered":"2023-10-18T08:56:33.731555+0000","last_clean":"2023-10-18T08:56:33.731555+0000","last_became_active":"2023-10-18T08:56:31.722557+0000","last_became_peered":"2023-10-18T08:56:31.722557+0000","last_unstale":"2023-10-18T08:56:33.731555+0000","last_undegraded":"2023-10-18T08:56:33.731555+0000","last_fullsized":"2023-10-18T08:56:33.731555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:53.619890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.731333+0000","last_change":"2023-10-18T08:56:32.759844+0000","last_active":"2023-10-18T08:56:33.731333+0000","last_peered":"2023-10-18T08:56:33.731333+0000","last_clean":"2023-10-18T08:56:33.731333+0000","last_became_active":"2023-10-18T08:56:32.759473+0000","last_became_peered":"2023-10-18T08:56:32.759473+0000","last_unstale":"2023-10-18T08:56:33.731333+0000","last_undegraded":"2023-10-18T08:56:33.731333+0000","last_fullsized":"2023-10-18T08:56:33.731333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:14:48.818841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.731719+0000","last_change":"2023-10-18T08:56:31.715307+0000","last_active":"2023-10-18T08:56:33.731719+0000","last_peered":"2023-10-18T08:56:33.731719+0000","last_clean":"2023-10-18T08:56:33.731719+0000","last_became_active":"2023-10-18T08:56:31.715076+0000","last_became_peered":"2023-10-18T08:56:31.715076+0000","last_unstale":"2023-10-18T08:56:33.731719+0000","last_undegraded":"2023-10-18T08:56:33.731719+0000","last_fullsized":"2023-10-18T08:56:33.731719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:20:23.956053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.732182+0000","last_change":"2023-10-18T08:56:32.740242+0000","last_active":"2023-10-18T08:56:33.732182+0000","last_peered":"2023-10-18T08:56:33.732182+0000","last_clean":"2023-10-18T08:56:33.732182+0000","last_became_active":"2023-10-18T08:56:32.739609+0000","last_became_peered":"2023-10-18T08:56:32.739609+0000","last_unstale":"2023-10-18T08:56:33.732182+0000","last_undegraded":"2023-10-18T08:56:33.732182+0000","last_fullsized":"2023-10-18T08:56:33.732182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:49.645329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899112+0000","last_change":"2023-10-18T08:56:31.732827+0000","last_active":"2023-10-18T08:56:33.899112+0000","last_peered":"2023-10-18T08:56:33.899112+0000","last_clean":"2023-10-18T08:56:33.899112+0000","last_became_active":"2023-10-18T08:56:31.731216+0000","last_became_peered":"2023-10-18T08:56:31.731216+0000","last_unstale":"2023-10-18T08:56:33.899112+0000","last_undegraded":"2023-10-18T08:56:33.899112+0000","last_fullsized":"2023-10-18T08:56:33.899112+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:51.285612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898518+0000","last_change":"2023-10-18T08:56:32.753748+0000","last_active":"2023-10-18T08:56:33.898518+0000","last_peered":"2023-10-18T08:56:33.898518+0000","last_clean":"2023-10-18T08:56:33.898518+0000","last_became_active":"2023-10-18T08:56:32.753183+0000","last_became_peered":"2023-10-18T08:56:32.753183+0000","last_unstale":"2023-10-18T08:56:33.898518+0000","last_undegraded":"2023-10-18T08:56:33.898518+0000","last_fullsized":"2023-10-18T08:56:33.898518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:43.146105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.731836+0000","last_change":"2023-10-18T08:56:31.723370+0000","last_active":"2023-10-18T08:56:33.731836+0000","last_peered":"2023-10-18T08:56:33.731836+0000","last_clean":"2023-10-18T08:56:33.731836+0000","last_became_active":"2023-10-18T08:56:31.720962+0000","last_became_peered":"2023-10-18T08:56:31.720962+0000","last_unstale":"2023-10-18T08:56:33.731836+0000","last_undegraded":"2023-10-18T08:56:33.731836+0000","last_fullsized":"2023-10-18T08:56:33.731836+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:42.132408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.736795+0000","last_change":"2023-10-18T08:56:32.755368+0000","last_active":"2023-10-18T08:56:33.736795+0000","last_peered":"2023-10-18T08:56:33.736795+0000","last_clean":"2023-10-18T08:56:33.736795+0000","last_became_active":"2023-10-18T08:56:32.754948+0000","last_became_peered":"2023-10-18T08:56:32.754948+0000","last_unstale":"2023-10-18T08:56:33.736795+0000","last_undegraded":"2023-10-18T08:56:33.736795+0000","last_fullsized":"2023-10-18T08:56:33.736795+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:34.532079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899798+0000","last_change":"2023-10-18T08:56:31.730844+0000","last_active":"2023-10-18T08:56:33.899798+0000","last_peered":"2023-10-18T08:56:33.899798+0000","last_clean":"2023-10-18T08:56:33.899798+0000","last_became_active":"2023-10-18T08:56:31.728788+0000","last_became_peered":"2023-10-18T08:56:31.728788+0000","last_unstale":"2023-10-18T08:56:33.899798+0000","last_undegraded":"2023-10-18T08:56:33.899798+0000","last_fullsized":"2023-10-18T08:56:33.899798+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:18.076493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730743+0000","last_change":"2023-10-18T08:56:32.764857+0000","last_active":"2023-10-18T08:56:33.730743+0000","last_peered":"2023-10-18T08:56:33.730743+0000","last_clean":"2023-10-18T08:56:33.730743+0000","last_became_active":"2023-10-18T08:56:32.764600+0000","last_became_peered":"2023-10-18T08:56:32.764600+0000","last_unstale":"2023-10-18T08:56:33.730743+0000","last_undegraded":"2023-10-18T08:56:33.730743+0000","last_fullsized":"2023-10-18T08:56:33.730743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:19.595855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.737271+0000","last_change":"2023-10-18T08:56:31.722738+0000","last_active":"2023-10-18T08:56:33.737271+0000","last_peered":"2023-10-18T08:56:33.737271+0000","last_clean":"2023-10-18T08:56:33.737271+0000","last_became_active":"2023-10-18T08:56:31.722482+0000","last_became_peered":"2023-10-18T08:56:31.722482+0000","last_unstale":"2023-10-18T08:56:33.737271+0000","last_undegraded":"2023-10-18T08:56:33.737271+0000","last_fullsized":"2023-10-18T08:56:33.737271+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:14.165909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897500+0000","last_change":"2023-10-18T08:56:32.747307+0000","last_active":"2023-10-18T08:56:33.897500+0000","last_peered":"2023-10-18T08:56:33.897500+0000","last_clean":"2023-10-18T08:56:33.897500+0000","last_became_active":"2023-10-18T08:56:32.746573+0000","last_became_peered":"2023-10-18T08:56:32.746573+0000","last_unstale":"2023-10-18T08:56:33.897500+0000","last_undegraded":"2023-10-18T08:56:33.897500+0000","last_fullsized":"2023-10-18T08:56:33.897500+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:54.633588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730480+0000","last_change":"2023-10-18T08:56:31.722528+0000","last_active":"2023-10-18T08:56:33.730480+0000","last_peered":"2023-10-18T08:56:33.730480+0000","last_clean":"2023-10-18T08:56:33.730480+0000","last_became_active":"2023-10-18T08:56:31.722303+0000","last_became_peered":"2023-10-18T08:56:31.722303+0000","last_unstale":"2023-10-18T08:56:33.730480+0000","last_undegraded":"2023-10-18T08:56:33.730480+0000","last_fullsized":"2023-10-18T08:56:33.730480+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:45.628001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.896674+0000","last_change":"2023-10-18T08:56:32.754761+0000","last_active":"2023-10-18T08:56:33.896674+0000","last_peered":"2023-10-18T08:56:33.896674+0000","last_clean":"2023-10-18T08:56:33.896674+0000","last_became_active":"2023-10-18T08:56:32.754473+0000","last_became_peered":"2023-10-18T08:56:32.754473+0000","last_unstale":"2023-10-18T08:56:33.896674+0000","last_undegraded":"2023-10-18T08:56:33.896674+0000","last_fullsized":"2023-10-18T08:56:33.896674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:22.763360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730616+0000","last_change":"2023-10-18T08:56:31.719426+0000","last_active":"2023-10-18T08:56:33.730616+0000","last_peered":"2023-10-18T08:56:33.730616+0000","last_clean":"2023-10-18T08:56:33.730616+0000","last_became_active":"2023-10-18T08:56:31.719267+0000","last_became_peered":"2023-10-18T08:56:31.719267+0000","last_unstale":"2023-10-18T08:56:33.730616+0000","last_undegraded":"2023-10-18T08:56:33.730616+0000","last_fullsized":"2023-10-18T08:56:33.730616+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:51.285612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899185+0000","last_change":"2023-10-18T08:56:32.755498+0000","last_active":"2023-10-18T08:56:33.899185+0000","last_peered":"2023-10-18T08:56:33.899185+0000","last_clean":"2023-10-18T08:56:33.899185+0000","last_became_active":"2023-10-18T08:56:32.754310+0000","last_became_peered":"2023-10-18T08:56:32.754310+0000","last_unstale":"2023-10-18T08:56:33.899185+0000","last_undegraded":"2023-10-18T08:56:33.899185+0000","last_fullsized":"2023-10-18T08:56:33.899185+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:54.633588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.736035+0000","last_change":"2023-10-18T08:56:31.718149+0000","last_active":"2023-10-18T08:56:33.736035+0000","last_peered":"2023-10-18T08:56:33.736035+0000","last_clean":"2023-10-18T08:56:33.736035+0000","last_became_active":"2023-10-18T08:56:31.717895+0000","last_became_peered":"2023-10-18T08:56:31.717895+0000","last_unstale":"2023-10-18T08:56:33.736035+0000","last_undegraded":"2023-10-18T08:56:33.736035+0000","last_fullsized":"2023-10-18T08:56:33.736035+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:51.285612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.900407+0000","last_change":"2023-10-18T08:56:32.751361+0000","last_active":"2023-10-18T08:56:33.900407+0000","last_peered":"2023-10-18T08:56:33.900407+0000","last_clean":"2023-10-18T08:56:33.900407+0000","last_became_active":"2023-10-18T08:56:32.751119+0000","last_became_peered":"2023-10-18T08:56:32.751119+0000","last_unstale":"2023-10-18T08:56:33.900407+0000","last_undegraded":"2023-10-18T08:56:33.900407+0000","last_fullsized":"2023-10-18T08:56:33.900407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:21:01.706092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.900031+0000","last_change":"2023-10-18T08:56:31.730291+0000","last_active":"2023-10-18T08:56:33.900031+0000","last_peered":"2023-10-18T08:56:33.900031+0000","last_clean":"2023-10-18T08:56:33.900031+0000","last_became_active":"2023-10-18T08:56:31.728222+0000","last_became_peered":"2023-10-18T08:56:31.728222+0000","last_unstale":"2023-10-18T08:56:33.900031+0000","last_undegraded":"2023-10-18T08:56:33.900031+0000","last_fullsized":"2023-10-18T08:56:33.900031+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:51.285612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.736299+0000","last_change":"2023-10-18T08:56:32.762892+0000","last_active":"2023-10-18T08:56:33.736299+0000","last_peered":"2023-10-18T08:56:33.736299+0000","last_clean":"2023-10-18T08:56:33.736299+0000","last_became_active":"2023-10-18T08:56:32.760333+0000","last_became_peered":"2023-10-18T08:56:32.760333+0000","last_unstale":"2023-10-18T08:56:33.736299+0000","last_undegraded":"2023-10-18T08:56:33.736299+0000","last_fullsized":"2023-10-18T08:56:33.736299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:54.633588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899139+0000","last_change":"2023-10-18T08:56:31.728578+0000","last_active":"2023-10-18T08:56:33.899139+0000","last_peered":"2023-10-18T08:56:33.899139+0000","last_clean":"2023-10-18T08:56:33.899139+0000","last_became_active":"2023-10-18T08:56:31.728153+0000","last_became_peered":"2023-10-18T08:56:31.728153+0000","last_unstale":"2023-10-18T08:56:33.899139+0000","last_undegraded":"2023-10-18T08:56:33.899139+0000","last_fullsized":"2023-10-18T08:56:33.899139+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:30.677707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894557+0000","last_change":"2023-10-18T08:56:32.749958+0000","last_active":"2023-10-18T08:56:33.894557+0000","last_peered":"2023-10-18T08:56:33.894557+0000","last_clean":"2023-10-18T08:56:33.894557+0000","last_became_active":"2023-10-18T08:56:32.749411+0000","last_became_peered":"2023-10-18T08:56:32.749411+0000","last_unstale":"2023-10-18T08:56:33.894557+0000","last_undegraded":"2023-10-18T08:56:33.894557+0000","last_fullsized":"2023-10-18T08:56:33.894557+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:02.177975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898962+0000","last_change":"2023-10-18T08:56:31.732577+0000","last_active":"2023-10-18T08:56:33.898962+0000","last_peered":"2023-10-18T08:56:33.898962+0000","last_clean":"2023-10-18T08:56:33.898962+0000","last_became_active":"2023-10-18T08:56:31.731994+0000","last_became_peered":"2023-10-18T08:56:31.731994+0000","last_unstale":"2023-10-18T08:56:33.898962+0000","last_undegraded":"2023-10-18T08:56:33.898962+0000","last_fullsized":"2023-10-18T08:56:33.898962+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:01.164277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894045+0000","last_change":"2023-10-18T08:56:32.756361+0000","last_active":"2023-10-18T08:56:33.894045+0000","last_peered":"2023-10-18T08:56:33.894045+0000","last_clean":"2023-10-18T08:56:33.894045+0000","last_became_active":"2023-10-18T08:56:32.755618+0000","last_became_peered":"2023-10-18T08:56:32.755618+0000","last_unstale":"2023-10-18T08:56:33.894045+0000","last_undegraded":"2023-10-18T08:56:33.894045+0000","last_fullsized":"2023-10-18T08:56:33.894045+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:00.916693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.734659+0000","last_change":"2023-10-18T08:56:31.721732+0000","last_active":"2023-10-18T08:56:33.734659+0000","last_peered":"2023-10-18T08:56:33.734659+0000","last_clean":"2023-10-18T08:56:33.734659+0000","last_became_active":"2023-10-18T08:56:31.721101+0000","last_became_peered":"2023-10-18T08:56:31.721101+0000","last_unstale":"2023-10-18T08:56:33.734659+0000","last_undegraded":"2023-10-18T08:56:33.734659+0000","last_fullsized":"2023-10-18T08:56:33.734659+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:35:07.240740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894255+0000","last_change":"2023-10-18T08:56:32.766008+0000","last_active":"2023-10-18T08:56:33.894255+0000","last_peered":"2023-10-18T08:56:33.894255+0000","last_clean":"2023-10-18T08:56:33.894255+0000","last_became_active":"2023-10-18T08:56:32.765753+0000","last_became_peered":"2023-10-18T08:56:32.765753+0000","last_unstale":"2023-10-18T08:56:33.894255+0000","last_undegraded":"2023-10-18T08:56:33.894255+0000","last_fullsized":"2023-10-18T08:56:33.894255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:20:24.969751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.895128+0000","last_change":"2023-10-18T08:56:31.722881+0000","last_active":"2023-10-18T08:56:33.895128+0000","last_peered":"2023-10-18T08:56:33.895128+0000","last_clean":"2023-10-18T08:56:33.895128+0000","last_became_active":"2023-10-18T08:56:31.722436+0000","last_became_peered":"2023-10-18T08:56:31.722436+0000","last_unstale":"2023-10-18T08:56:33.895128+0000","last_undegraded":"2023-10-18T08:56:33.895128+0000","last_fullsized":"2023-10-18T08:56:33.895128+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:30:32.002656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894781+0000","last_change":"2023-10-18T08:56:32.764825+0000","last_active":"2023-10-18T08:56:33.894781+0000","last_peered":"2023-10-18T08:56:33.894781+0000","last_clean":"2023-10-18T08:56:33.894781+0000","last_became_active":"2023-10-18T08:56:32.764497+0000","last_became_peered":"2023-10-18T08:56:32.764497+0000","last_unstale":"2023-10-18T08:56:33.894781+0000","last_undegraded":"2023-10-18T08:56:33.894781+0000","last_fullsized":"2023-10-18T08:56:33.894781+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:32.422253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899557+0000","last_change":"2023-10-18T08:56:31.731502+0000","last_active":"2023-10-18T08:56:33.899557+0000","last_peered":"2023-10-18T08:56:33.899557+0000","last_clean":"2023-10-18T08:56:33.899557+0000","last_became_active":"2023-10-18T08:56:31.730568+0000","last_became_peered":"2023-10-18T08:56:31.730568+0000","last_unstale":"2023-10-18T08:56:33.899557+0000","last_undegraded":"2023-10-18T08:56:33.899557+0000","last_fullsized":"2023-10-18T08:56:33.899557+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:21.749663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899521+0000","last_change":"2023-10-18T08:56:32.753952+0000","last_active":"2023-10-18T08:56:33.899521+0000","last_peered":"2023-10-18T08:56:33.899521+0000","last_clean":"2023-10-18T08:56:33.899521+0000","last_became_active":"2023-10-18T08:56:32.751870+0000","last_became_peered":"2023-10-18T08:56:32.751870+0000","last_unstale":"2023-10-18T08:56:33.899521+0000","last_undegraded":"2023-10-18T08:56:33.899521+0000","last_fullsized":"2023-10-18T08:56:33.899521+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:44:45.934778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.896824+0000","last_change":"2023-10-18T08:56:31.729579+0000","last_active":"2023-10-18T08:56:33.896824+0000","last_peered":"2023-10-18T08:56:33.896824+0000","last_clean":"2023-10-18T08:56:33.896824+0000","last_became_active":"2023-10-18T08:56:31.726536+0000","last_became_peered":"2023-10-18T08:56:31.726536+0000","last_unstale":"2023-10-18T08:56:33.896824+0000","last_undegraded":"2023-10-18T08:56:33.896824+0000","last_fullsized":"2023-10-18T08:56:33.896824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:40.003448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894422+0000","last_change":"2023-10-18T08:56:32.750467+0000","last_active":"2023-10-18T08:56:33.894422+0000","last_peered":"2023-10-18T08:56:33.894422+0000","last_clean":"2023-10-18T08:56:33.894422+0000","last_became_active":"2023-10-18T08:56:32.749813+0000","last_became_peered":"2023-10-18T08:56:32.749813+0000","last_unstale":"2023-10-18T08:56:33.894422+0000","last_undegraded":"2023-10-18T08:56:33.894422+0000","last_fullsized":"2023-10-18T08:56:33.894422+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:21:14.799223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.736540+0000","last_change":"2023-10-18T08:56:31.730514+0000","last_active":"2023-10-18T08:56:33.736540+0000","last_peered":"2023-10-18T08:56:33.736540+0000","last_clean":"2023-10-18T08:56:33.736540+0000","last_became_active":"2023-10-18T08:56:31.730043+0000","last_became_peered":"2023-10-18T08:56:31.730043+0000","last_unstale":"2023-10-18T08:56:33.736540+0000","last_undegraded":"2023-10-18T08:56:33.736540+0000","last_fullsized":"2023-10-18T08:56:33.736540+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:18.076493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899665+0000","last_change":"2023-10-18T08:56:32.753042+0000","last_active":"2023-10-18T08:56:33.899665+0000","last_peered":"2023-10-18T08:56:33.899665+0000","last_clean":"2023-10-18T08:56:33.899665+0000","last_became_active":"2023-10-18T08:56:32.752790+0000","last_became_peered":"2023-10-18T08:56:32.752790+0000","last_unstale":"2023-10-18T08:56:33.899665+0000","last_undegraded":"2023-10-18T08:56:33.899665+0000","last_fullsized":"2023-10-18T08:56:33.899665+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:58.348622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.734880+0000","last_change":"2023-10-18T08:56:31.721895+0000","last_active":"2023-10-18T08:56:33.734880+0000","last_peered":"2023-10-18T08:56:33.734880+0000","last_clean":"2023-10-18T08:56:33.734880+0000","last_became_active":"2023-10-18T08:56:31.721423+0000","last_became_peered":"2023-10-18T08:56:31.721423+0000","last_unstale":"2023-10-18T08:56:33.734880+0000","last_undegraded":"2023-10-18T08:56:33.734880+0000","last_fullsized":"2023-10-18T08:56:33.734880+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:21.749663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.731649+0000","last_change":"2023-10-18T08:56:32.766263+0000","last_active":"2023-10-18T08:56:33.731649+0000","last_peered":"2023-10-18T08:56:33.731649+0000","last_clean":"2023-10-18T08:56:33.731649+0000","last_became_active":"2023-10-18T08:56:32.765809+0000","last_became_peered":"2023-10-18T08:56:32.765809+0000","last_unstale":"2023-10-18T08:56:33.731649+0000","last_undegraded":"2023-10-18T08:56:33.731649+0000","last_fullsized":"2023-10-18T08:56:33.731649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:19.090190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.736088+0000","last_change":"2023-10-18T08:56:31.718328+0000","last_active":"2023-10-18T08:56:33.736088+0000","last_peered":"2023-10-18T08:56:33.736088+0000","last_clean":"2023-10-18T08:56:33.736088+0000","last_became_active":"2023-10-18T08:56:31.717951+0000","last_became_peered":"2023-10-18T08:56:31.717951+0000","last_unstale":"2023-10-18T08:56:33.736088+0000","last_undegraded":"2023-10-18T08:56:33.736088+0000","last_fullsized":"2023-10-18T08:56:33.736088+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:23.940891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897013+0000","last_change":"2023-10-18T08:56:32.765411+0000","last_active":"2023-10-18T08:56:33.897013+0000","last_peered":"2023-10-18T08:56:33.897013+0000","last_clean":"2023-10-18T08:56:33.897013+0000","last_became_active":"2023-10-18T08:56:32.765146+0000","last_became_peered":"2023-10-18T08:56:32.765146+0000","last_unstale":"2023-10-18T08:56:33.897013+0000","last_undegraded":"2023-10-18T08:56:33.897013+0000","last_fullsized":"2023-10-18T08:56:33.897013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:52.299309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.732542+0000","last_change":"2023-10-18T08:56:31.725638+0000","last_active":"2023-10-18T08:56:33.732542+0000","last_peered":"2023-10-18T08:56:33.732542+0000","last_clean":"2023-10-18T08:56:33.732542+0000","last_became_active":"2023-10-18T08:56:31.724903+0000","last_became_peered":"2023-10-18T08:56:31.724903+0000","last_unstale":"2023-10-18T08:56:33.732542+0000","last_undegraded":"2023-10-18T08:56:33.732542+0000","last_fullsized":"2023-10-18T08:56:33.732542+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:53:50.586585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.732138+0000","last_change":"2023-10-18T08:56:32.765163+0000","last_active":"2023-10-18T08:56:33.732138+0000","last_peered":"2023-10-18T08:56:33.732138+0000","last_clean":"2023-10-18T08:56:33.732138+0000","last_became_active":"2023-10-18T08:56:32.764872+0000","last_became_peered":"2023-10-18T08:56:32.764872+0000","last_unstale":"2023-10-18T08:56:33.732138+0000","last_undegraded":"2023-10-18T08:56:33.732138+0000","last_fullsized":"2023-10-18T08:56:33.732138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:52.299309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.737085+0000","last_change":"2023-10-18T08:56:31.730319+0000","last_active":"2023-10-18T08:56:33.737085+0000","last_peered":"2023-10-18T08:56:33.737085+0000","last_clean":"2023-10-18T08:56:33.737085+0000","last_became_active":"2023-10-18T08:56:31.729683+0000","last_became_peered":"2023-10-18T08:56:31.729683+0000","last_unstale":"2023-10-18T08:56:33.737085+0000","last_undegraded":"2023-10-18T08:56:33.737085+0000","last_fullsized":"2023-10-18T08:56:33.737085+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:33.794693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730394+0000","last_change":"2023-10-18T08:56:32.758182+0000","last_active":"2023-10-18T08:56:33.730394+0000","last_peered":"2023-10-18T08:56:33.730394+0000","last_clean":"2023-10-18T08:56:33.730394+0000","last_became_active":"2023-10-18T08:56:32.756265+0000","last_became_peered":"2023-10-18T08:56:32.756265+0000","last_unstale":"2023-10-18T08:56:33.730394+0000","last_undegraded":"2023-10-18T08:56:33.730394+0000","last_fullsized":"2023-10-18T08:56:33.730394+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:44:45.934778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899390+0000","last_change":"2023-10-18T08:56:31.733508+0000","last_active":"2023-10-18T08:56:33.899390+0000","last_peered":"2023-10-18T08:56:33.899390+0000","last_clean":"2023-10-18T08:56:33.899390+0000","last_became_active":"2023-10-18T08:56:31.733283+0000","last_became_peered":"2023-10-18T08:56:31.733283+0000","last_unstale":"2023-10-18T08:56:33.899390+0000","last_undegraded":"2023-10-18T08:56:33.899390+0000","last_fullsized":"2023-10-18T08:56:33.899390+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:39.227696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897809+0000","last_change":"2023-10-18T08:56:32.737481+0000","last_active":"2023-10-18T08:56:33.897809+0000","last_peered":"2023-10-18T08:56:33.897809+0000","last_clean":"2023-10-18T08:56:33.897809+0000","last_became_active":"2023-10-18T08:56:32.734847+0000","last_became_peered":"2023-10-18T08:56:32.734847+0000","last_unstale":"2023-10-18T08:56:33.897809+0000","last_undegraded":"2023-10-18T08:56:33.897809+0000","last_fullsized":"2023-10-18T08:56:33.897809+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:15.179607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.732396+0000","last_change":"2023-10-18T08:56:31.725825+0000","last_active":"2023-10-18T08:56:33.732396+0000","last_peered":"2023-10-18T08:56:33.732396+0000","last_clean":"2023-10-18T08:56:33.732396+0000","last_became_active":"2023-10-18T08:56:31.725148+0000","last_became_peered":"2023-10-18T08:56:31.725148+0000","last_unstale":"2023-10-18T08:56:33.732396+0000","last_undegraded":"2023-10-18T08:56:33.732396+0000","last_fullsized":"2023-10-18T08:56:33.732396+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:30:26.846049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899854+0000","last_change":"2023-10-18T08:56:32.758487+0000","last_active":"2023-10-18T08:56:33.899854+0000","last_peered":"2023-10-18T08:56:33.899854+0000","last_clean":"2023-10-18T08:56:33.899854+0000","last_became_active":"2023-10-18T08:56:32.756653+0000","last_became_peered":"2023-10-18T08:56:32.756653+0000","last_unstale":"2023-10-18T08:56:33.899854+0000","last_undegraded":"2023-10-18T08:56:33.899854+0000","last_fullsized":"2023-10-18T08:56:33.899854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:21:01.706092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.895372+0000","last_change":"2023-10-18T08:56:31.733662+0000","last_active":"2023-10-18T08:56:33.895372+0000","last_peered":"2023-10-18T08:56:33.895372+0000","last_clean":"2023-10-18T08:56:33.895372+0000","last_became_active":"2023-10-18T08:56:31.732888+0000","last_became_peered":"2023-10-18T08:56:31.732888+0000","last_unstale":"2023-10-18T08:56:33.895372+0000","last_undegraded":"2023-10-18T08:56:33.895372+0000","last_fullsized":"2023-10-18T08:56:33.895372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:59.902995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-18T08:56:32.730066+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:56:32.730066+0000","last_peered":"2023-10-18T08:56:32.730066+0000","last_clean":"2023-10-18T08:56:32.730066+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:56:32.730066+0000","last_undegraded":"2023-10-18T08:56:32.730066+0000","last_fullsized":"2023-10-18T08:56:32.730066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:34.881744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":719,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194592,"statfs":{"total":200000000,"available":199262774,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:56:05 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.2849999999999999},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":1.667}]},{"osd":1,"last update":"Wed Oct 18 08:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.024},{"interface":"front","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.325}]},{"osd":2,"last update":"Wed Oct 18 08:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.456},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":1.4179999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.5309999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.613},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.5049999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":1.552}]},{"osd":6,"last update":"Wed Oct 18 08:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.383}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":137,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195174,"statfs":{"total":200000000,"available":199858916,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":1.3360000000000001}]},{"osd":1,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":1.04},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.9609999999999999,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":0.94299999999999995}]},{"osd":2,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.01},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.127}]},{"osd":3,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.383}]},{"osd":4,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.593},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.0860000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.2150000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.794,"5min":2.794,"15min":2.794},"last":1.288}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":137,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195174,"statfs":{"total":200000000,"available":199858860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":1.474},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.4239999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":1.3280000000000001}]},{"osd":2,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":3.8180000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.266}]},{"osd":3,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":4.7119999999999997,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":1.153},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.0109999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":5.4580000000000002,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":1.0649999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":5.141,"5min":5.141,"15min":5.141},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.603,"5min":1.603,"15min":1.603},"min":{"1min":1.171,"5min":1.171,"15min":1.171},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.1919999999999999}]},{"osd":7,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":1.014,"5min":1.014,"15min":1.014},"max":{"1min":5.5490000000000004,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":4.8419999999999996,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":1.2390000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":136,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199860447,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.1439999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.92500000000000004}]},{"osd":2,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.3380000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":2.8010000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.4610000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"min":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.4219999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.573},{"interface":"front","average":{"1min":1.583,"5min":1.583,"15min":1.583},"min":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"max":{"1min":3.6859999999999999,"5min":3.6859999999999999,"15min":3.6859999999999999},"last":1.6200000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.524,"15min":1.524},"min":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":1.23,"5min":1.23,"15min":1.23},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.665}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":144,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195167,"statfs":{"total":200000000,"available":199851598,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":1.075},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.73599999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.272}]},{"osd":2,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.1599999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.054}]},{"osd":5,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":1.006},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.335}]},{"osd":6,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.466},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":1.296}]},{"osd":7,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.397}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":137,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195175,"statfs":{"total":200000000,"available":199859462,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":4.2210000000000001,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.77700000000000002}]},{"osd":1,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":4.29,"5min":4.29,"15min":4.29},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.97699999999999998}]},{"osd":3,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":7.5570000000000004,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":1.1180000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":7.6189999999999998,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":1.228},{"interface":"front","average":{"1min":1.468,"5min":1.468,"15min":1.468},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.663}]},{"osd":5,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":4.4320000000000004,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":1.012},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":7.657,"5min":7.657,"15min":7.657},"last":1.4119999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":3.5299999999999998,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":4.3920000000000003,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":1.516}]},{"osd":7,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.595,"15min":1.595},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":3.3620000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":4.3390000000000004,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.4590000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":137,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195174,"statfs":{"total":200000000,"available":199858917,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.67200000000000004}]},{"osd":2,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.65200000000000002}]},{"osd":3,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.7010000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":0.74399999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.127},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.94099999999999995}]},{"osd":5,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.147},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.0900000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.272,"15min":1.272},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.212}]},{"osd":7,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.1759999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":712,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194599,"statfs":{"total":200000000,"available":199270183,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":3.4420000000000002,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.254}]},{"osd":2,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.92600000000000005}]},{"osd":3,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":3.5720000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.2889999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":4.492,"5min":4.492,"15min":4.492},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.1259999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":4.2779999999999996,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.758}]},{"osd":6,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":3.5299999999999998,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":4.4349999999999996,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.173}]},{"osd":7,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":3.492,"5min":3.492,"15min":3.492},"last":1.603},{"interface":"front","average":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":4.5419999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.048}]}]}],"pool_statfs":[]}} 2023-10-18T08:56:37.354 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:56:37.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.606+0000 7f3d71bf4640 1 -- 172.21.15.40:0/1431562250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d5c0039a0 msgr2=0x7f3d5c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:37.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.606+0000 7f3d71bf4640 1 --2- 172.21.15.40:0/1431562250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d5c0039a0 0x7f3d5c003dc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c06db10 tx=0x7f3d5c0057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:37.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.606+0000 7f3d713f3640 1 -- 172.21.15.40:0/1431562250 reap_dead start 2023-10-18T08:56:37.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.606+0000 7f3d71bf4640 1 -- 172.21.15.40:0/1431562250 shutdown_connections 2023-10-18T08:56:37.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.606+0000 7f3d71bf4640 1 -- 172.21.15.40:0/1431562250 >> 172.21.15.40:0/1431562250 conn(0x7f3d6c069720 msgr2=0x7f3d6c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:37.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.606+0000 7f3d71bf4640 1 -- 172.21.15.40:0/1431562250 shutdown_connections 2023-10-18T08:56:37.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.607+0000 7f3d71bf4640 1 -- 172.21.15.40:0/1431562250 wait complete. 2023-10-18T08:56:37.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.608+0000 7f3d71bf4640 1 Processor -- start 2023-10-18T08:56:37.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.608+0000 7f3d71bf4640 1 -- start start 2023-10-18T08:56:37.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.609+0000 7f3d71bf4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c0039a0 0x7f3d6c1e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:37.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.609+0000 7f3d71bf4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d6c14a9c0 0x7f3d6c15a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:37.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.609+0000 7f3d71bf4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d6c145cc0 0x7f3d6c154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:37.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.609+0000 7f3d71bf4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c14a5a0 con 0x7f3d6c145cc0 2023-10-18T08:56:37.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.609+0000 7f3d71bf4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c151d70 con 0x7f3d6c14a9c0 2023-10-18T08:56:37.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.609+0000 7f3d71bf4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c1499b0 con 0x7f3d5c0039a0 2023-10-18T08:56:37.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.610+0000 7f3d713f3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d6c145cc0 0x7f3d6c154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:37.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.610+0000 7f3d713f3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d6c145cc0 0x7f3d6c154ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50972/0 (socket says 172.21.15.40:50972) 2023-10-18T08:56:37.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.610+0000 7f3d713f3640 1 -- 172.21.15.40:0/1234850762 learned_addr learned my addr 172.21.15.40:0/1234850762 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:37.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.610+0000 7f3d6bfff640 1 --2- 172.21.15.40:0/1234850762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d6c14a9c0 0x7f3d6c15a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:37.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.610+0000 7f3d70bf2640 1 --2- 172.21.15.40:0/1234850762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c0039a0 0x7f3d6c1e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:37.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.610+0000 7f3d713f3640 1 -- 172.21.15.40:0/1234850762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c0039a0 msgr2=0x7f3d6c1e64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:37.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.610+0000 7f3d713f3640 1 --2- 172.21.15.40:0/1234850762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c0039a0 0x7f3d6c1e64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:37.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.610+0000 7f3d713f3640 1 -- 172.21.15.40:0/1234850762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d6c14a9c0 msgr2=0x7f3d6c15a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:37.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.610+0000 7f3d713f3640 1 --2- 172.21.15.40:0/1234850762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d6c14a9c0 0x7f3d6c15a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:37.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.610+0000 7f3d713f3640 1 -- 172.21.15.40:0/1234850762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c15adc0 con 0x7f3d6c145cc0 2023-10-18T08:56:37.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.610+0000 7f3d713f3640 1 --2- 172.21.15.40:0/1234850762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d6c145cc0 0x7f3d6c154ac0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3d64005fd0 tx=0x7f3d6400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:37.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.611+0000 7f3d697fa640 1 -- 172.21.15.40:0/1234850762 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d64004d20 con 0x7f3d6c145cc0 2023-10-18T08:56:37.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.611+0000 7f3d697fa640 1 -- 172.21.15.40:0/1234850762 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3d6400ca60 con 0x7f3d6c145cc0 2023-10-18T08:56:37.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.611+0000 7f3d697fa640 1 -- 172.21.15.40:0/1234850762 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400ce70 con 0x7f3d6c145cc0 2023-10-18T08:56:37.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.611+0000 7f3d71bf4640 1 -- 172.21.15.40:0/1234850762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6c149c00 con 0x7f3d6c145cc0 2023-10-18T08:56:37.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.611+0000 7f3d71bf4640 1 -- 172.21.15.40:0/1234850762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6c14a1c0 con 0x7f3d6c145cc0 2023-10-18T08:56:37.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.612+0000 7f3d71bf4640 1 -- 172.21.15.40:0/1234850762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6c14a6e0 con 0x7f3d6c145cc0 2023-10-18T08:56:37.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.613+0000 7f3d697fa640 1 -- 172.21.15.40:0/1234850762 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3d64011470 con 0x7f3d6c145cc0 2023-10-18T08:56:37.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.614+0000 7f3d697fa640 1 --2- 172.21.15.40:0/1234850762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d5006ae40 0x7f3d5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:37.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.614+0000 7f3d697fa640 1 -- 172.21.15.40:0/1234850762 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d64089a20 con 0x7f3d6c145cc0 2023-10-18T08:56:37.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.615+0000 7f3d70bf2640 1 --2- 172.21.15.40:0/1234850762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d5006ae40 0x7f3d5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:37.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.616+0000 7f3d70bf2640 1 --2- 172.21.15.40:0/1234850762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d5006ae40 0x7f3d5006d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d60034000 tx=0x7f3d6002e070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:37.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.618+0000 7f3d697fa640 1 -- 172.21.15.40:0/1234850762 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3d6c14a6e0 con 0x7f3d6c145cc0 2023-10-18T08:56:37.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.802+0000 7f3d71bf4640 1 -- 172.21.15.40:0/1234850762 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3d6c14a6e0 con 0x7f3d5006ae40 2023-10-18T08:56:37.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.824+0000 7f3d697fa640 1 -- 172.21.15.40:0/1234850762 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282473 (secure 0 0 0) 0x7f3d6c14a6e0 con 0x7f3d5006ae40 2023-10-18T08:56:37.830 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:37.833 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:56:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.828+0000 7f3d71bf4640 1 -- 172.21.15.40:0/1234850762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d5006ae40 msgr2=0x7f3d5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:37.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.828+0000 7f3d71bf4640 1 --2- 172.21.15.40:0/1234850762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d5006ae40 0x7f3d5006d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d60034000 tx=0x7f3d6002e070 comp rx=0 tx=0).stop 2023-10-18T08:56:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.828+0000 7f3d71bf4640 1 -- 172.21.15.40:0/1234850762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d6c145cc0 msgr2=0x7f3d6c154ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.828+0000 7f3d71bf4640 1 --2- 172.21.15.40:0/1234850762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d6c145cc0 0x7f3d6c154ac0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3d64005fd0 tx=0x7f3d6400eda0 comp rx=0 tx=0).stop 2023-10-18T08:56:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.828+0000 7f3d71bf4640 1 -- 172.21.15.40:0/1234850762 shutdown_connections 2023-10-18T08:56:37.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.828+0000 7f3d71bf4640 1 --2- 172.21.15.40:0/1234850762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d5006ae40 0x7f3d5006d300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:37.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.828+0000 7f3d71bf4640 1 --2- 172.21.15.40:0/1234850762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d6c145cc0 0x7f3d6c154ac0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:37.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.828+0000 7f3d71bf4640 1 --2- 172.21.15.40:0/1234850762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d6c14a9c0 0x7f3d6c15a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:37.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.829+0000 7f3d71bf4640 1 --2- 172.21.15.40:0/1234850762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c0039a0 0x7f3d6c1e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:37.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.829+0000 7f3d71bf4640 1 -- 172.21.15.40:0/1234850762 >> 172.21.15.40:0/1234850762 conn(0x7f3d6c069720 msgr2=0x7f3d6c15ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:37.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.829+0000 7f3d71bf4640 1 -- 172.21.15.40:0/1234850762 shutdown_connections 2023-10-18T08:56:37.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:37.829+0000 7f3d71bf4640 1 -- 172.21.15.40:0/1234850762 wait complete. 2023-10-18T08:56:37.848 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2023-10-18T08:56:36.922967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2264,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560235,"statfs":{"total":1600000000,"available":1597681157,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002154"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.895861+0000","last_change":"2023-10-18T08:56:32.753645+0000","last_active":"2023-10-18T08:56:33.895861+0000","last_peered":"2023-10-18T08:56:33.895861+0000","last_clean":"2023-10-18T08:56:33.895861+0000","last_became_active":"2023-10-18T08:56:32.753349+0000","last_became_peered":"2023-10-18T08:56:32.753349+0000","last_unstale":"2023-10-18T08:56:33.895861+0000","last_undegraded":"2023-10-18T08:56:33.895861+0000","last_fullsized":"2023-10-18T08:56:33.895861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:53:51.600283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730113+0000","last_change":"2023-10-18T08:56:32.764736+0000","last_active":"2023-10-18T08:56:33.730113+0000","last_peered":"2023-10-18T08:56:33.730113+0000","last_clean":"2023-10-18T08:56:33.730113+0000","last_became_active":"2023-10-18T08:56:32.764429+0000","last_became_peered":"2023-10-18T08:56:32.764429+0000","last_unstale":"2023-10-18T08:56:33.730113+0000","last_undegraded":"2023-10-18T08:56:33.730113+0000","last_fullsized":"2023-10-18T08:56:33.730113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:17:05.996948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898868+0000","last_change":"2023-10-18T08:56:32.753507+0000","last_active":"2023-10-18T08:56:33.898868+0000","last_peered":"2023-10-18T08:56:33.898868+0000","last_clean":"2023-10-18T08:56:33.898868+0000","last_became_active":"2023-10-18T08:56:32.752727+0000","last_became_peered":"2023-10-18T08:56:32.752727+0000","last_unstale":"2023-10-18T08:56:33.898868+0000","last_undegraded":"2023-10-18T08:56:33.898868+0000","last_fullsized":"2023-10-18T08:56:33.898868+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:54.633588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.735041+0000","last_change":"2023-10-18T08:56:32.756278+0000","last_active":"2023-10-18T08:56:33.735041+0000","last_peered":"2023-10-18T08:56:33.735041+0000","last_clean":"2023-10-18T08:56:33.735041+0000","last_became_active":"2023-10-18T08:56:32.755805+0000","last_became_peered":"2023-10-18T08:56:32.755805+0000","last_unstale":"2023-10-18T08:56:33.735041+0000","last_undegraded":"2023-10-18T08:56:33.735041+0000","last_fullsized":"2023-10-18T08:56:33.735041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:03:31.460178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894667+0000","last_change":"2023-10-18T08:56:32.760311+0000","last_active":"2023-10-18T08:56:33.894667+0000","last_peered":"2023-10-18T08:56:33.894667+0000","last_clean":"2023-10-18T08:56:33.894667+0000","last_became_active":"2023-10-18T08:56:32.760054+0000","last_became_peered":"2023-10-18T08:56:32.760054+0000","last_unstale":"2023-10-18T08:56:33.894667+0000","last_undegraded":"2023-10-18T08:56:33.894667+0000","last_fullsized":"2023-10-18T08:56:33.894667+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:31.691404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730700+0000","last_change":"2023-10-18T08:56:32.755115+0000","last_active":"2023-10-18T08:56:33.730700+0000","last_peered":"2023-10-18T08:56:33.730700+0000","last_clean":"2023-10-18T08:56:33.730700+0000","last_became_active":"2023-10-18T08:56:32.754534+0000","last_became_peered":"2023-10-18T08:56:32.754534+0000","last_unstale":"2023-10-18T08:56:33.730700+0000","last_undegraded":"2023-10-18T08:56:33.730700+0000","last_fullsized":"2023-10-18T08:56:33.730700+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:21:01.706092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898490+0000","last_change":"2023-10-18T08:56:32.749058+0000","last_active":"2023-10-18T08:56:33.898490+0000","last_peered":"2023-10-18T08:56:33.898490+0000","last_clean":"2023-10-18T08:56:33.898490+0000","last_became_active":"2023-10-18T08:56:32.747939+0000","last_became_peered":"2023-10-18T08:56:32.747939+0000","last_unstale":"2023-10-18T08:56:33.898490+0000","last_undegraded":"2023-10-18T08:56:33.898490+0000","last_fullsized":"2023-10-18T08:56:33.898490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:17:05.996948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.896726+0000","last_change":"2023-10-18T08:56:32.749127+0000","last_active":"2023-10-18T08:56:33.896726+0000","last_peered":"2023-10-18T08:56:33.896726+0000","last_clean":"2023-10-18T08:56:33.896726+0000","last_became_active":"2023-10-18T08:56:32.748595+0000","last_became_peered":"2023-10-18T08:56:32.748595+0000","last_unstale":"2023-10-18T08:56:33.896726+0000","last_undegraded":"2023-10-18T08:56:33.896726+0000","last_fullsized":"2023-10-18T08:56:33.896726+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:41.017146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897819+0000","last_change":"2023-10-18T08:56:32.756936+0000","last_active":"2023-10-18T08:56:33.897819+0000","last_peered":"2023-10-18T08:56:33.897819+0000","last_clean":"2023-10-18T08:56:33.897819+0000","last_became_active":"2023-10-18T08:56:32.756495+0000","last_became_peered":"2023-10-18T08:56:32.756495+0000","last_unstale":"2023-10-18T08:56:33.897819+0000","last_undegraded":"2023-10-18T08:56:33.897819+0000","last_fullsized":"2023-10-18T08:56:33.897819+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:24.954589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.735528+0000","last_change":"2023-10-18T08:56:32.754846+0000","last_active":"2023-10-18T08:56:33.735528+0000","last_peered":"2023-10-18T08:56:33.735528+0000","last_clean":"2023-10-18T08:56:33.735528+0000","last_became_active":"2023-10-18T08:56:32.754271+0000","last_became_peered":"2023-10-18T08:56:32.754271+0000","last_unstale":"2023-10-18T08:56:33.735528+0000","last_undegraded":"2023-10-18T08:56:33.735528+0000","last_fullsized":"2023-10-18T08:56:33.735528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:43.146105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897411+0000","last_change":"2023-10-18T08:56:32.756696+0000","last_active":"2023-10-18T08:56:33.897411+0000","last_peered":"2023-10-18T08:56:33.897411+0000","last_clean":"2023-10-18T08:56:33.897411+0000","last_became_active":"2023-10-18T08:56:32.756425+0000","last_became_peered":"2023-10-18T08:56:32.756425+0000","last_unstale":"2023-10-18T08:56:33.897411+0000","last_undegraded":"2023-10-18T08:56:33.897411+0000","last_fullsized":"2023-10-18T08:56:33.897411+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:43.146105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899004+0000","last_change":"2023-10-18T08:56:32.758579+0000","last_active":"2023-10-18T08:56:33.899004+0000","last_peered":"2023-10-18T08:56:33.899004+0000","last_clean":"2023-10-18T08:56:33.899004+0000","last_became_active":"2023-10-18T08:56:32.758178+0000","last_became_peered":"2023-10-18T08:56:32.758178+0000","last_unstale":"2023-10-18T08:56:33.899004+0000","last_undegraded":"2023-10-18T08:56:33.899004+0000","last_fullsized":"2023-10-18T08:56:33.899004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:34.532079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894064+0000","last_change":"2023-10-18T08:56:32.757310+0000","last_active":"2023-10-18T08:56:33.894064+0000","last_peered":"2023-10-18T08:56:33.894064+0000","last_clean":"2023-10-18T08:56:33.894064+0000","last_became_active":"2023-10-18T08:56:32.756825+0000","last_became_peered":"2023-10-18T08:56:32.756825+0000","last_unstale":"2023-10-18T08:56:33.894064+0000","last_undegraded":"2023-10-18T08:56:33.894064+0000","last_fullsized":"2023-10-18T08:56:33.894064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:40.241394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730946+0000","last_change":"2023-10-18T08:56:32.751825+0000","last_active":"2023-10-18T08:56:33.730946+0000","last_peered":"2023-10-18T08:56:33.730946+0000","last_clean":"2023-10-18T08:56:33.730946+0000","last_became_active":"2023-10-18T08:56:32.751338+0000","last_became_peered":"2023-10-18T08:56:32.751338+0000","last_unstale":"2023-10-18T08:56:33.730946+0000","last_undegraded":"2023-10-18T08:56:33.730946+0000","last_fullsized":"2023-10-18T08:56:33.730946+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:41.097642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.900017+0000","last_change":"2023-10-18T08:56:32.754683+0000","last_active":"2023-10-18T08:56:33.900017+0000","last_peered":"2023-10-18T08:56:33.900017+0000","last_clean":"2023-10-18T08:56:33.900017+0000","last_became_active":"2023-10-18T08:56:32.754425+0000","last_became_peered":"2023-10-18T08:56:32.754425+0000","last_unstale":"2023-10-18T08:56:33.900017+0000","last_undegraded":"2023-10-18T08:56:33.900017+0000","last_fullsized":"2023-10-18T08:56:33.900017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:38.281069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897738+0000","last_change":"2023-10-18T08:56:32.737915+0000","last_active":"2023-10-18T08:56:33.897738+0000","last_peered":"2023-10-18T08:56:33.897738+0000","last_clean":"2023-10-18T08:56:33.897738+0000","last_became_active":"2023-10-18T08:56:32.737477+0000","last_became_peered":"2023-10-18T08:56:32.737477+0000","last_unstale":"2023-10-18T08:56:33.897738+0000","last_undegraded":"2023-10-18T08:56:33.897738+0000","last_fullsized":"2023-10-18T08:56:33.897738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:34.532079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897839+0000","last_change":"2023-10-18T08:56:32.749232+0000","last_active":"2023-10-18T08:56:33.897839+0000","last_peered":"2023-10-18T08:56:33.897839+0000","last_clean":"2023-10-18T08:56:33.897839+0000","last_became_active":"2023-10-18T08:56:32.748383+0000","last_became_peered":"2023-10-18T08:56:32.748383+0000","last_unstale":"2023-10-18T08:56:33.897839+0000","last_undegraded":"2023-10-18T08:56:33.897839+0000","last_fullsized":"2023-10-18T08:56:33.897839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:30:27.859747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730344+0000","last_change":"2023-10-18T08:56:32.756729+0000","last_active":"2023-10-18T08:56:33.730344+0000","last_peered":"2023-10-18T08:56:33.730344+0000","last_clean":"2023-10-18T08:56:33.730344+0000","last_became_active":"2023-10-18T08:56:32.756311+0000","last_became_peered":"2023-10-18T08:56:32.756311+0000","last_unstale":"2023-10-18T08:56:33.730344+0000","last_undegraded":"2023-10-18T08:56:33.730344+0000","last_fullsized":"2023-10-18T08:56:33.730344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:38.281069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730050+0000","last_change":"2023-10-18T08:56:32.758356+0000","last_active":"2023-10-18T08:56:33.730050+0000","last_peered":"2023-10-18T08:56:33.730050+0000","last_clean":"2023-10-18T08:56:33.730050+0000","last_became_active":"2023-10-18T08:56:32.757860+0000","last_became_peered":"2023-10-18T08:56:32.757860+0000","last_unstale":"2023-10-18T08:56:33.730050+0000","last_undegraded":"2023-10-18T08:56:33.730050+0000","last_fullsized":"2023-10-18T08:56:33.730050+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:58.348622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894888+0000","last_change":"2023-10-18T08:56:32.753658+0000","last_active":"2023-10-18T08:56:33.894888+0000","last_peered":"2023-10-18T08:56:33.894888+0000","last_clean":"2023-10-18T08:56:33.894888+0000","last_became_active":"2023-10-18T08:56:32.752555+0000","last_became_peered":"2023-10-18T08:56:32.752555+0000","last_unstale":"2023-10-18T08:56:33.894888+0000","last_undegraded":"2023-10-18T08:56:33.894888+0000","last_fullsized":"2023-10-18T08:56:33.894888+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:53:51.600283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898523+0000","last_change":"2023-10-18T08:56:32.764321+0000","last_active":"2023-10-18T08:56:33.898523+0000","last_peered":"2023-10-18T08:56:33.898523+0000","last_clean":"2023-10-18T08:56:33.898523+0000","last_became_active":"2023-10-18T08:56:32.764095+0000","last_became_peered":"2023-10-18T08:56:32.764095+0000","last_unstale":"2023-10-18T08:56:33.898523+0000","last_undegraded":"2023-10-18T08:56:33.898523+0000","last_fullsized":"2023-10-18T08:56:33.898523+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:46.641699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898759+0000","last_change":"2023-10-18T08:56:32.739350+0000","last_active":"2023-10-18T08:56:33.898759+0000","last_peered":"2023-10-18T08:56:33.898759+0000","last_clean":"2023-10-18T08:56:33.898759+0000","last_became_active":"2023-10-18T08:56:32.738980+0000","last_became_peered":"2023-10-18T08:56:32.738980+0000","last_unstale":"2023-10-18T08:56:33.898759+0000","last_undegraded":"2023-10-18T08:56:33.898759+0000","last_fullsized":"2023-10-18T08:56:33.898759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:53.992426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.734396+0000","last_change":"2023-10-18T08:56:32.765823+0000","last_active":"2023-10-18T08:56:33.734396+0000","last_peered":"2023-10-18T08:56:33.734396+0000","last_clean":"2023-10-18T08:56:33.734396+0000","last_became_active":"2023-10-18T08:56:32.765470+0000","last_became_peered":"2023-10-18T08:56:32.765470+0000","last_unstale":"2023-10-18T08:56:33.734396+0000","last_undegraded":"2023-10-18T08:56:33.734396+0000","last_fullsized":"2023-10-18T08:56:33.734396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:15.179607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899718+0000","last_change":"2023-10-18T08:56:32.755055+0000","last_active":"2023-10-18T08:56:33.899718+0000","last_peered":"2023-10-18T08:56:33.899718+0000","last_clean":"2023-10-18T08:56:33.899718+0000","last_became_active":"2023-10-18T08:56:32.753565+0000","last_became_peered":"2023-10-18T08:56:32.753565+0000","last_unstale":"2023-10-18T08:56:33.899718+0000","last_undegraded":"2023-10-18T08:56:33.899718+0000","last_fullsized":"2023-10-18T08:56:33.899718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:17:05.996948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.734871+0000","last_change":"2023-10-18T08:56:32.759339+0000","last_active":"2023-10-18T08:56:33.734871+0000","last_peered":"2023-10-18T08:56:33.734871+0000","last_clean":"2023-10-18T08:56:33.734871+0000","last_became_active":"2023-10-18T08:56:32.758800+0000","last_became_peered":"2023-10-18T08:56:32.758800+0000","last_unstale":"2023-10-18T08:56:33.734871+0000","last_undegraded":"2023-10-18T08:56:33.734871+0000","last_fullsized":"2023-10-18T08:56:33.734871+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:46.641699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.731027+0000","last_change":"2023-10-18T08:56:32.738926+0000","last_active":"2023-10-18T08:56:33.731027+0000","last_peered":"2023-10-18T08:56:33.731027+0000","last_clean":"2023-10-18T08:56:33.731027+0000","last_became_active":"2023-10-18T08:56:32.738061+0000","last_became_peered":"2023-10-18T08:56:32.738061+0000","last_unstale":"2023-10-18T08:56:33.731027+0000","last_undegraded":"2023-10-18T08:56:33.731027+0000","last_fullsized":"2023-10-18T08:56:33.731027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:03.028759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.736889+0000","last_change":"2023-10-18T08:56:32.762596+0000","last_active":"2023-10-18T08:56:33.736889+0000","last_peered":"2023-10-18T08:56:33.736889+0000","last_clean":"2023-10-18T08:56:33.736889+0000","last_became_active":"2023-10-18T08:56:32.759815+0000","last_became_peered":"2023-10-18T08:56:32.759815+0000","last_unstale":"2023-10-18T08:56:33.736889+0000","last_undegraded":"2023-10-18T08:56:33.736889+0000","last_fullsized":"2023-10-18T08:56:33.736889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:30:27.859747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897995+0000","last_change":"2023-10-18T08:56:32.766204+0000","last_active":"2023-10-18T08:56:33.897995+0000","last_peered":"2023-10-18T08:56:33.897995+0000","last_clean":"2023-10-18T08:56:33.897995+0000","last_became_active":"2023-10-18T08:56:32.765917+0000","last_became_peered":"2023-10-18T08:56:32.765917+0000","last_unstale":"2023-10-18T08:56:33.897995+0000","last_undegraded":"2023-10-18T08:56:33.897995+0000","last_fullsized":"2023-10-18T08:56:33.897995+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:53.992426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898126+0000","last_change":"2023-10-18T08:56:32.740545+0000","last_active":"2023-10-18T08:56:33.898126+0000","last_peered":"2023-10-18T08:56:33.898126+0000","last_clean":"2023-10-18T08:56:33.898126+0000","last_became_active":"2023-10-18T08:56:32.739922+0000","last_became_peered":"2023-10-18T08:56:32.739922+0000","last_unstale":"2023-10-18T08:56:33.898126+0000","last_undegraded":"2023-10-18T08:56:33.898126+0000","last_fullsized":"2023-10-18T08:56:33.898126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:03:31.460178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899205+0000","last_change":"2023-10-18T08:56:32.755907+0000","last_active":"2023-10-18T08:56:33.899205+0000","last_peered":"2023-10-18T08:56:33.899205+0000","last_clean":"2023-10-18T08:56:33.899205+0000","last_became_active":"2023-10-18T08:56:32.753213+0000","last_became_peered":"2023-10-18T08:56:32.753213+0000","last_unstale":"2023-10-18T08:56:33.899205+0000","last_undegraded":"2023-10-18T08:56:33.899205+0000","last_fullsized":"2023-10-18T08:56:33.899205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:38.281069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.896171+0000","last_change":"2023-10-18T08:56:32.756402+0000","last_active":"2023-10-18T08:56:33.896171+0000","last_peered":"2023-10-18T08:56:33.896171+0000","last_clean":"2023-10-18T08:56:33.896171+0000","last_became_active":"2023-10-18T08:56:32.756114+0000","last_became_peered":"2023-10-18T08:56:32.756114+0000","last_unstale":"2023-10-18T08:56:33.896171+0000","last_undegraded":"2023-10-18T08:56:33.896171+0000","last_fullsized":"2023-10-18T08:56:33.896171+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:35:08.254437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898145+0000","last_change":"2023-10-18T08:56:32.747103+0000","last_active":"2023-10-18T08:56:33.898145+0000","last_peered":"2023-10-18T08:56:33.898145+0000","last_clean":"2023-10-18T08:56:33.898145+0000","last_became_active":"2023-10-18T08:56:32.744772+0000","last_became_peered":"2023-10-18T08:56:32.744772+0000","last_unstale":"2023-10-18T08:56:33.898145+0000","last_undegraded":"2023-10-18T08:56:33.898145+0000","last_fullsized":"2023-10-18T08:56:33.898145+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:58.348622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.732272+0000","last_change":"2023-10-18T08:56:32.766097+0000","last_active":"2023-10-18T08:56:33.732272+0000","last_peered":"2023-10-18T08:56:33.732272+0000","last_clean":"2023-10-18T08:56:33.732272+0000","last_became_active":"2023-10-18T08:56:32.765353+0000","last_became_peered":"2023-10-18T08:56:32.765353+0000","last_unstale":"2023-10-18T08:56:33.732272+0000","last_undegraded":"2023-10-18T08:56:33.732272+0000","last_fullsized":"2023-10-18T08:56:33.732272+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:34.808391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897247+0000","last_change":"2023-10-18T08:56:31.734202+0000","last_active":"2023-10-18T08:56:33.897247+0000","last_peered":"2023-10-18T08:56:33.897247+0000","last_clean":"2023-10-18T08:56:33.897247+0000","last_became_active":"2023-10-18T08:56:31.733975+0000","last_became_peered":"2023-10-18T08:56:31.733975+0000","last_unstale":"2023-10-18T08:56:33.897247+0000","last_undegraded":"2023-10-18T08:56:33.897247+0000","last_fullsized":"2023-10-18T08:56:33.897247+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:31.408555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897005+0000","last_change":"2023-10-18T08:56:32.756370+0000","last_active":"2023-10-18T08:56:33.897005+0000","last_peered":"2023-10-18T08:56:33.897005+0000","last_clean":"2023-10-18T08:56:33.897005+0000","last_became_active":"2023-10-18T08:56:32.756114+0000","last_became_peered":"2023-10-18T08:56:32.756114+0000","last_unstale":"2023-10-18T08:56:33.897005+0000","last_undegraded":"2023-10-18T08:56:33.897005+0000","last_fullsized":"2023-10-18T08:56:33.897005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:19.090190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.731918+0000","last_change":"2023-10-18T08:56:31.725969+0000","last_active":"2023-10-18T08:56:33.731918+0000","last_peered":"2023-10-18T08:56:33.731918+0000","last_clean":"2023-10-18T08:56:33.731918+0000","last_became_active":"2023-10-18T08:56:31.725551+0000","last_became_peered":"2023-10-18T08:56:31.725551+0000","last_unstale":"2023-10-18T08:56:33.731918+0000","last_undegraded":"2023-10-18T08:56:33.731918+0000","last_fullsized":"2023-10-18T08:56:33.731918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:30.677707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898896+0000","last_change":"2023-10-18T08:56:32.747635+0000","last_active":"2023-10-18T08:56:33.898896+0000","last_peered":"2023-10-18T08:56:33.898896+0000","last_clean":"2023-10-18T08:56:33.898896+0000","last_became_active":"2023-10-18T08:56:32.745274+0000","last_became_peered":"2023-10-18T08:56:32.745274+0000","last_unstale":"2023-10-18T08:56:33.898896+0000","last_undegraded":"2023-10-18T08:56:33.898896+0000","last_fullsized":"2023-10-18T08:56:33.898896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:43.146105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730918+0000","last_change":"2023-10-18T08:56:31.720258+0000","last_active":"2023-10-18T08:56:33.730918+0000","last_peered":"2023-10-18T08:56:33.730918+0000","last_clean":"2023-10-18T08:56:33.730918+0000","last_became_active":"2023-10-18T08:56:31.720050+0000","last_became_peered":"2023-10-18T08:56:31.720050+0000","last_unstale":"2023-10-18T08:56:33.730918+0000","last_undegraded":"2023-10-18T08:56:33.730918+0000","last_fullsized":"2023-10-18T08:56:33.730918+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:21.749663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.735200+0000","last_change":"2023-10-18T08:56:32.758132+0000","last_active":"2023-10-18T08:56:33.735200+0000","last_peered":"2023-10-18T08:56:33.735200+0000","last_clean":"2023-10-18T08:56:33.735200+0000","last_became_active":"2023-10-18T08:56:32.756996+0000","last_became_peered":"2023-10-18T08:56:32.756996+0000","last_unstale":"2023-10-18T08:56:33.735200+0000","last_undegraded":"2023-10-18T08:56:33.735200+0000","last_fullsized":"2023-10-18T08:56:33.735200+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:41.017146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894902+0000","last_change":"2023-10-18T08:56:31.719392+0000","last_active":"2023-10-18T08:56:33.894902+0000","last_peered":"2023-10-18T08:56:33.894902+0000","last_clean":"2023-10-18T08:56:33.894902+0000","last_became_active":"2023-10-18T08:56:31.719094+0000","last_became_peered":"2023-10-18T08:56:31.719094+0000","last_unstale":"2023-10-18T08:56:33.894902+0000","last_undegraded":"2023-10-18T08:56:33.894902+0000","last_fullsized":"2023-10-18T08:56:33.894902+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:51:25.842852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897200+0000","last_change":"2023-10-18T08:56:32.747036+0000","last_active":"2023-10-18T08:56:33.897200+0000","last_peered":"2023-10-18T08:56:33.897200+0000","last_clean":"2023-10-18T08:56:33.897200+0000","last_became_active":"2023-10-18T08:56:32.745877+0000","last_became_peered":"2023-10-18T08:56:32.745877+0000","last_unstale":"2023-10-18T08:56:33.897200+0000","last_undegraded":"2023-10-18T08:56:33.897200+0000","last_fullsized":"2023-10-18T08:56:33.897200+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:53.992426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898408+0000","last_change":"2023-10-18T08:56:31.732775+0000","last_active":"2023-10-18T08:56:33.898408+0000","last_peered":"2023-10-18T08:56:33.898408+0000","last_clean":"2023-10-18T08:56:33.898408+0000","last_became_active":"2023-10-18T08:56:31.732333+0000","last_became_peered":"2023-10-18T08:56:31.732333+0000","last_unstale":"2023-10-18T08:56:33.898408+0000","last_undegraded":"2023-10-18T08:56:33.898408+0000","last_fullsized":"2023-10-18T08:56:33.898408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:00:26.389152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899317+0000","last_change":"2023-10-18T08:56:32.765589+0000","last_active":"2023-10-18T08:56:33.899317+0000","last_peered":"2023-10-18T08:56:33.899317+0000","last_clean":"2023-10-18T08:56:33.899317+0000","last_became_active":"2023-10-18T08:56:32.765264+0000","last_became_peered":"2023-10-18T08:56:32.765264+0000","last_unstale":"2023-10-18T08:56:33.899317+0000","last_undegraded":"2023-10-18T08:56:33.899317+0000","last_fullsized":"2023-10-18T08:56:33.899317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:30:27.859747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898064+0000","last_change":"2023-10-18T08:56:31.730226+0000","last_active":"2023-10-18T08:56:33.898064+0000","last_peered":"2023-10-18T08:56:33.898064+0000","last_clean":"2023-10-18T08:56:33.898064+0000","last_became_active":"2023-10-18T08:56:31.729915+0000","last_became_peered":"2023-10-18T08:56:31.729915+0000","last_unstale":"2023-10-18T08:56:33.898064+0000","last_undegraded":"2023-10-18T08:56:33.898064+0000","last_fullsized":"2023-10-18T08:56:33.898064+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:30:32.002656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898705+0000","last_change":"2023-10-18T08:56:32.755938+0000","last_active":"2023-10-18T08:56:33.898705+0000","last_peered":"2023-10-18T08:56:33.898705+0000","last_clean":"2023-10-18T08:56:33.898705+0000","last_became_active":"2023-10-18T08:56:32.754686+0000","last_became_peered":"2023-10-18T08:56:32.754686+0000","last_unstale":"2023-10-18T08:56:33.898705+0000","last_undegraded":"2023-10-18T08:56:33.898705+0000","last_fullsized":"2023-10-18T08:56:33.898705+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:41.017146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.731163+0000","last_change":"2023-10-18T08:56:31.709122+0000","last_active":"2023-10-18T08:56:33.731163+0000","last_peered":"2023-10-18T08:56:33.731163+0000","last_clean":"2023-10-18T08:56:33.731163+0000","last_became_active":"2023-10-18T08:56:31.708306+0000","last_became_peered":"2023-10-18T08:56:31.708306+0000","last_unstale":"2023-10-18T08:56:33.731163+0000","last_undegraded":"2023-10-18T08:56:33.731163+0000","last_fullsized":"2023-10-18T08:56:33.731163+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:40.003448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.735387+0000","last_change":"2023-10-18T08:56:32.755223+0000","last_active":"2023-10-18T08:56:33.735387+0000","last_peered":"2023-10-18T08:56:33.735387+0000","last_clean":"2023-10-18T08:56:33.735387+0000","last_became_active":"2023-10-18T08:56:32.752298+0000","last_became_peered":"2023-10-18T08:56:32.752298+0000","last_unstale":"2023-10-18T08:56:33.735387+0000","last_undegraded":"2023-10-18T08:56:33.735387+0000","last_fullsized":"2023-10-18T08:56:33.735387+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:53.992426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899377+0000","last_change":"2023-10-18T08:56:31.732204+0000","last_active":"2023-10-18T08:56:33.899377+0000","last_peered":"2023-10-18T08:56:33.899377+0000","last_clean":"2023-10-18T08:56:33.899377+0000","last_became_active":"2023-10-18T08:56:31.731739+0000","last_became_peered":"2023-10-18T08:56:31.731739+0000","last_unstale":"2023-10-18T08:56:33.899377+0000","last_undegraded":"2023-10-18T08:56:33.899377+0000","last_fullsized":"2023-10-18T08:56:33.899377+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:35:07.240740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.900125+0000","last_change":"2023-10-18T08:56:32.758417+0000","last_active":"2023-10-18T08:56:33.900125+0000","last_peered":"2023-10-18T08:56:33.900125+0000","last_clean":"2023-10-18T08:56:33.900125+0000","last_became_active":"2023-10-18T08:56:32.757491+0000","last_became_peered":"2023-10-18T08:56:32.757491+0000","last_unstale":"2023-10-18T08:56:33.900125+0000","last_undegraded":"2023-10-18T08:56:33.900125+0000","last_fullsized":"2023-10-18T08:56:33.900125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:44:45.934778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.731555+0000","last_change":"2023-10-18T08:56:31.723089+0000","last_active":"2023-10-18T08:56:33.731555+0000","last_peered":"2023-10-18T08:56:33.731555+0000","last_clean":"2023-10-18T08:56:33.731555+0000","last_became_active":"2023-10-18T08:56:31.722557+0000","last_became_peered":"2023-10-18T08:56:31.722557+0000","last_unstale":"2023-10-18T08:56:33.731555+0000","last_undegraded":"2023-10-18T08:56:33.731555+0000","last_fullsized":"2023-10-18T08:56:33.731555+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:53.619890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.731333+0000","last_change":"2023-10-18T08:56:32.759844+0000","last_active":"2023-10-18T08:56:33.731333+0000","last_peered":"2023-10-18T08:56:33.731333+0000","last_clean":"2023-10-18T08:56:33.731333+0000","last_became_active":"2023-10-18T08:56:32.759473+0000","last_became_peered":"2023-10-18T08:56:32.759473+0000","last_unstale":"2023-10-18T08:56:33.731333+0000","last_undegraded":"2023-10-18T08:56:33.731333+0000","last_fullsized":"2023-10-18T08:56:33.731333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:14:48.818841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.731719+0000","last_change":"2023-10-18T08:56:31.715307+0000","last_active":"2023-10-18T08:56:33.731719+0000","last_peered":"2023-10-18T08:56:33.731719+0000","last_clean":"2023-10-18T08:56:33.731719+0000","last_became_active":"2023-10-18T08:56:31.715076+0000","last_became_peered":"2023-10-18T08:56:31.715076+0000","last_unstale":"2023-10-18T08:56:33.731719+0000","last_undegraded":"2023-10-18T08:56:33.731719+0000","last_fullsized":"2023-10-18T08:56:33.731719+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:20:23.956053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.732182+0000","last_change":"2023-10-18T08:56:32.740242+0000","last_active":"2023-10-18T08:56:33.732182+0000","last_peered":"2023-10-18T08:56:33.732182+0000","last_clean":"2023-10-18T08:56:33.732182+0000","last_became_active":"2023-10-18T08:56:32.739609+0000","last_became_peered":"2023-10-18T08:56:32.739609+0000","last_unstale":"2023-10-18T08:56:33.732182+0000","last_undegraded":"2023-10-18T08:56:33.732182+0000","last_fullsized":"2023-10-18T08:56:33.732182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:49.645329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899112+0000","last_change":"2023-10-18T08:56:31.732827+0000","last_active":"2023-10-18T08:56:33.899112+0000","last_peered":"2023-10-18T08:56:33.899112+0000","last_clean":"2023-10-18T08:56:33.899112+0000","last_became_active":"2023-10-18T08:56:31.731216+0000","last_became_peered":"2023-10-18T08:56:31.731216+0000","last_unstale":"2023-10-18T08:56:33.899112+0000","last_undegraded":"2023-10-18T08:56:33.899112+0000","last_fullsized":"2023-10-18T08:56:33.899112+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:51.285612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898518+0000","last_change":"2023-10-18T08:56:32.753748+0000","last_active":"2023-10-18T08:56:33.898518+0000","last_peered":"2023-10-18T08:56:33.898518+0000","last_clean":"2023-10-18T08:56:33.898518+0000","last_became_active":"2023-10-18T08:56:32.753183+0000","last_became_peered":"2023-10-18T08:56:32.753183+0000","last_unstale":"2023-10-18T08:56:33.898518+0000","last_undegraded":"2023-10-18T08:56:33.898518+0000","last_fullsized":"2023-10-18T08:56:33.898518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:43.146105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.731836+0000","last_change":"2023-10-18T08:56:31.723370+0000","last_active":"2023-10-18T08:56:33.731836+0000","last_peered":"2023-10-18T08:56:33.731836+0000","last_clean":"2023-10-18T08:56:33.731836+0000","last_became_active":"2023-10-18T08:56:31.720962+0000","last_became_peered":"2023-10-18T08:56:31.720962+0000","last_unstale":"2023-10-18T08:56:33.731836+0000","last_undegraded":"2023-10-18T08:56:33.731836+0000","last_fullsized":"2023-10-18T08:56:33.731836+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:42.132408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.736795+0000","last_change":"2023-10-18T08:56:32.755368+0000","last_active":"2023-10-18T08:56:33.736795+0000","last_peered":"2023-10-18T08:56:33.736795+0000","last_clean":"2023-10-18T08:56:33.736795+0000","last_became_active":"2023-10-18T08:56:32.754948+0000","last_became_peered":"2023-10-18T08:56:32.754948+0000","last_unstale":"2023-10-18T08:56:33.736795+0000","last_undegraded":"2023-10-18T08:56:33.736795+0000","last_fullsized":"2023-10-18T08:56:33.736795+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:34.532079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899798+0000","last_change":"2023-10-18T08:56:31.730844+0000","last_active":"2023-10-18T08:56:33.899798+0000","last_peered":"2023-10-18T08:56:33.899798+0000","last_clean":"2023-10-18T08:56:33.899798+0000","last_became_active":"2023-10-18T08:56:31.728788+0000","last_became_peered":"2023-10-18T08:56:31.728788+0000","last_unstale":"2023-10-18T08:56:33.899798+0000","last_undegraded":"2023-10-18T08:56:33.899798+0000","last_fullsized":"2023-10-18T08:56:33.899798+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:18.076493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730743+0000","last_change":"2023-10-18T08:56:32.764857+0000","last_active":"2023-10-18T08:56:33.730743+0000","last_peered":"2023-10-18T08:56:33.730743+0000","last_clean":"2023-10-18T08:56:33.730743+0000","last_became_active":"2023-10-18T08:56:32.764600+0000","last_became_peered":"2023-10-18T08:56:32.764600+0000","last_unstale":"2023-10-18T08:56:33.730743+0000","last_undegraded":"2023-10-18T08:56:33.730743+0000","last_fullsized":"2023-10-18T08:56:33.730743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:19.595855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.737271+0000","last_change":"2023-10-18T08:56:31.722738+0000","last_active":"2023-10-18T08:56:33.737271+0000","last_peered":"2023-10-18T08:56:33.737271+0000","last_clean":"2023-10-18T08:56:33.737271+0000","last_became_active":"2023-10-18T08:56:31.722482+0000","last_became_peered":"2023-10-18T08:56:31.722482+0000","last_unstale":"2023-10-18T08:56:33.737271+0000","last_undegraded":"2023-10-18T08:56:33.737271+0000","last_fullsized":"2023-10-18T08:56:33.737271+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:14.165909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897500+0000","last_change":"2023-10-18T08:56:32.747307+0000","last_active":"2023-10-18T08:56:33.897500+0000","last_peered":"2023-10-18T08:56:33.897500+0000","last_clean":"2023-10-18T08:56:33.897500+0000","last_became_active":"2023-10-18T08:56:32.746573+0000","last_became_peered":"2023-10-18T08:56:32.746573+0000","last_unstale":"2023-10-18T08:56:33.897500+0000","last_undegraded":"2023-10-18T08:56:33.897500+0000","last_fullsized":"2023-10-18T08:56:33.897500+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:54.633588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730480+0000","last_change":"2023-10-18T08:56:31.722528+0000","last_active":"2023-10-18T08:56:33.730480+0000","last_peered":"2023-10-18T08:56:33.730480+0000","last_clean":"2023-10-18T08:56:33.730480+0000","last_became_active":"2023-10-18T08:56:31.722303+0000","last_became_peered":"2023-10-18T08:56:31.722303+0000","last_unstale":"2023-10-18T08:56:33.730480+0000","last_undegraded":"2023-10-18T08:56:33.730480+0000","last_fullsized":"2023-10-18T08:56:33.730480+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:45.628001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.896674+0000","last_change":"2023-10-18T08:56:32.754761+0000","last_active":"2023-10-18T08:56:33.896674+0000","last_peered":"2023-10-18T08:56:33.896674+0000","last_clean":"2023-10-18T08:56:33.896674+0000","last_became_active":"2023-10-18T08:56:32.754473+0000","last_became_peered":"2023-10-18T08:56:32.754473+0000","last_unstale":"2023-10-18T08:56:33.896674+0000","last_undegraded":"2023-10-18T08:56:33.896674+0000","last_fullsized":"2023-10-18T08:56:33.896674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:22.763360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730616+0000","last_change":"2023-10-18T08:56:31.719426+0000","last_active":"2023-10-18T08:56:33.730616+0000","last_peered":"2023-10-18T08:56:33.730616+0000","last_clean":"2023-10-18T08:56:33.730616+0000","last_became_active":"2023-10-18T08:56:31.719267+0000","last_became_peered":"2023-10-18T08:56:31.719267+0000","last_unstale":"2023-10-18T08:56:33.730616+0000","last_undegraded":"2023-10-18T08:56:33.730616+0000","last_fullsized":"2023-10-18T08:56:33.730616+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:51.285612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899185+0000","last_change":"2023-10-18T08:56:32.755498+0000","last_active":"2023-10-18T08:56:33.899185+0000","last_peered":"2023-10-18T08:56:33.899185+0000","last_clean":"2023-10-18T08:56:33.899185+0000","last_became_active":"2023-10-18T08:56:32.754310+0000","last_became_peered":"2023-10-18T08:56:32.754310+0000","last_unstale":"2023-10-18T08:56:33.899185+0000","last_undegraded":"2023-10-18T08:56:33.899185+0000","last_fullsized":"2023-10-18T08:56:33.899185+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:54.633588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.736035+0000","last_change":"2023-10-18T08:56:31.718149+0000","last_active":"2023-10-18T08:56:33.736035+0000","last_peered":"2023-10-18T08:56:33.736035+0000","last_clean":"2023-10-18T08:56:33.736035+0000","last_became_active":"2023-10-18T08:56:31.717895+0000","last_became_peered":"2023-10-18T08:56:31.717895+0000","last_unstale":"2023-10-18T08:56:33.736035+0000","last_undegraded":"2023-10-18T08:56:33.736035+0000","last_fullsized":"2023-10-18T08:56:33.736035+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:51.285612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.900407+0000","last_change":"2023-10-18T08:56:32.751361+0000","last_active":"2023-10-18T08:56:33.900407+0000","last_peered":"2023-10-18T08:56:33.900407+0000","last_clean":"2023-10-18T08:56:33.900407+0000","last_became_active":"2023-10-18T08:56:32.751119+0000","last_became_peered":"2023-10-18T08:56:32.751119+0000","last_unstale":"2023-10-18T08:56:33.900407+0000","last_undegraded":"2023-10-18T08:56:33.900407+0000","last_fullsized":"2023-10-18T08:56:33.900407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:21:01.706092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.900031+0000","last_change":"2023-10-18T08:56:31.730291+0000","last_active":"2023-10-18T08:56:33.900031+0000","last_peered":"2023-10-18T08:56:33.900031+0000","last_clean":"2023-10-18T08:56:33.900031+0000","last_became_active":"2023-10-18T08:56:31.728222+0000","last_became_peered":"2023-10-18T08:56:31.728222+0000","last_unstale":"2023-10-18T08:56:33.900031+0000","last_undegraded":"2023-10-18T08:56:33.900031+0000","last_fullsized":"2023-10-18T08:56:33.900031+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:51.285612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.736299+0000","last_change":"2023-10-18T08:56:32.762892+0000","last_active":"2023-10-18T08:56:33.736299+0000","last_peered":"2023-10-18T08:56:33.736299+0000","last_clean":"2023-10-18T08:56:33.736299+0000","last_became_active":"2023-10-18T08:56:32.760333+0000","last_became_peered":"2023-10-18T08:56:32.760333+0000","last_unstale":"2023-10-18T08:56:33.736299+0000","last_undegraded":"2023-10-18T08:56:33.736299+0000","last_fullsized":"2023-10-18T08:56:33.736299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:08:54.633588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899139+0000","last_change":"2023-10-18T08:56:31.728578+0000","last_active":"2023-10-18T08:56:33.899139+0000","last_peered":"2023-10-18T08:56:33.899139+0000","last_clean":"2023-10-18T08:56:33.899139+0000","last_became_active":"2023-10-18T08:56:31.728153+0000","last_became_peered":"2023-10-18T08:56:31.728153+0000","last_unstale":"2023-10-18T08:56:33.899139+0000","last_undegraded":"2023-10-18T08:56:33.899139+0000","last_fullsized":"2023-10-18T08:56:33.899139+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:06:30.677707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894557+0000","last_change":"2023-10-18T08:56:32.749958+0000","last_active":"2023-10-18T08:56:33.894557+0000","last_peered":"2023-10-18T08:56:33.894557+0000","last_clean":"2023-10-18T08:56:33.894557+0000","last_became_active":"2023-10-18T08:56:32.749411+0000","last_became_peered":"2023-10-18T08:56:32.749411+0000","last_unstale":"2023-10-18T08:56:33.894557+0000","last_undegraded":"2023-10-18T08:56:33.894557+0000","last_fullsized":"2023-10-18T08:56:33.894557+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:02.177975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.898962+0000","last_change":"2023-10-18T08:56:31.732577+0000","last_active":"2023-10-18T08:56:33.898962+0000","last_peered":"2023-10-18T08:56:33.898962+0000","last_clean":"2023-10-18T08:56:33.898962+0000","last_became_active":"2023-10-18T08:56:31.731994+0000","last_became_peered":"2023-10-18T08:56:31.731994+0000","last_unstale":"2023-10-18T08:56:33.898962+0000","last_undegraded":"2023-10-18T08:56:33.898962+0000","last_fullsized":"2023-10-18T08:56:33.898962+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:20:01.164277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894045+0000","last_change":"2023-10-18T08:56:32.756361+0000","last_active":"2023-10-18T08:56:33.894045+0000","last_peered":"2023-10-18T08:56:33.894045+0000","last_clean":"2023-10-18T08:56:33.894045+0000","last_became_active":"2023-10-18T08:56:32.755618+0000","last_became_peered":"2023-10-18T08:56:32.755618+0000","last_unstale":"2023-10-18T08:56:33.894045+0000","last_undegraded":"2023-10-18T08:56:33.894045+0000","last_fullsized":"2023-10-18T08:56:33.894045+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:00.916693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.734659+0000","last_change":"2023-10-18T08:56:31.721732+0000","last_active":"2023-10-18T08:56:33.734659+0000","last_peered":"2023-10-18T08:56:33.734659+0000","last_clean":"2023-10-18T08:56:33.734659+0000","last_became_active":"2023-10-18T08:56:31.721101+0000","last_became_peered":"2023-10-18T08:56:31.721101+0000","last_unstale":"2023-10-18T08:56:33.734659+0000","last_undegraded":"2023-10-18T08:56:33.734659+0000","last_fullsized":"2023-10-18T08:56:33.734659+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:35:07.240740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894255+0000","last_change":"2023-10-18T08:56:32.766008+0000","last_active":"2023-10-18T08:56:33.894255+0000","last_peered":"2023-10-18T08:56:33.894255+0000","last_clean":"2023-10-18T08:56:33.894255+0000","last_became_active":"2023-10-18T08:56:32.765753+0000","last_became_peered":"2023-10-18T08:56:32.765753+0000","last_unstale":"2023-10-18T08:56:33.894255+0000","last_undegraded":"2023-10-18T08:56:33.894255+0000","last_fullsized":"2023-10-18T08:56:33.894255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:20:24.969751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.895128+0000","last_change":"2023-10-18T08:56:31.722881+0000","last_active":"2023-10-18T08:56:33.895128+0000","last_peered":"2023-10-18T08:56:33.895128+0000","last_clean":"2023-10-18T08:56:33.895128+0000","last_became_active":"2023-10-18T08:56:31.722436+0000","last_became_peered":"2023-10-18T08:56:31.722436+0000","last_unstale":"2023-10-18T08:56:33.895128+0000","last_undegraded":"2023-10-18T08:56:33.895128+0000","last_fullsized":"2023-10-18T08:56:33.895128+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:30:32.002656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894781+0000","last_change":"2023-10-18T08:56:32.764825+0000","last_active":"2023-10-18T08:56:33.894781+0000","last_peered":"2023-10-18T08:56:33.894781+0000","last_clean":"2023-10-18T08:56:33.894781+0000","last_became_active":"2023-10-18T08:56:32.764497+0000","last_became_peered":"2023-10-18T08:56:32.764497+0000","last_unstale":"2023-10-18T08:56:33.894781+0000","last_undegraded":"2023-10-18T08:56:33.894781+0000","last_fullsized":"2023-10-18T08:56:33.894781+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:32.422253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899557+0000","last_change":"2023-10-18T08:56:31.731502+0000","last_active":"2023-10-18T08:56:33.899557+0000","last_peered":"2023-10-18T08:56:33.899557+0000","last_clean":"2023-10-18T08:56:33.899557+0000","last_became_active":"2023-10-18T08:56:31.730568+0000","last_became_peered":"2023-10-18T08:56:31.730568+0000","last_unstale":"2023-10-18T08:56:33.899557+0000","last_undegraded":"2023-10-18T08:56:33.899557+0000","last_fullsized":"2023-10-18T08:56:33.899557+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:21.749663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899521+0000","last_change":"2023-10-18T08:56:32.753952+0000","last_active":"2023-10-18T08:56:33.899521+0000","last_peered":"2023-10-18T08:56:33.899521+0000","last_clean":"2023-10-18T08:56:33.899521+0000","last_became_active":"2023-10-18T08:56:32.751870+0000","last_became_peered":"2023-10-18T08:56:32.751870+0000","last_unstale":"2023-10-18T08:56:33.899521+0000","last_undegraded":"2023-10-18T08:56:33.899521+0000","last_fullsized":"2023-10-18T08:56:33.899521+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:44:45.934778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.896824+0000","last_change":"2023-10-18T08:56:31.729579+0000","last_active":"2023-10-18T08:56:33.896824+0000","last_peered":"2023-10-18T08:56:33.896824+0000","last_clean":"2023-10-18T08:56:33.896824+0000","last_became_active":"2023-10-18T08:56:31.726536+0000","last_became_peered":"2023-10-18T08:56:31.726536+0000","last_unstale":"2023-10-18T08:56:33.896824+0000","last_undegraded":"2023-10-18T08:56:33.896824+0000","last_fullsized":"2023-10-18T08:56:33.896824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:40.003448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.894422+0000","last_change":"2023-10-18T08:56:32.750467+0000","last_active":"2023-10-18T08:56:33.894422+0000","last_peered":"2023-10-18T08:56:33.894422+0000","last_clean":"2023-10-18T08:56:33.894422+0000","last_became_active":"2023-10-18T08:56:32.749813+0000","last_became_peered":"2023-10-18T08:56:32.749813+0000","last_unstale":"2023-10-18T08:56:33.894422+0000","last_undegraded":"2023-10-18T08:56:33.894422+0000","last_fullsized":"2023-10-18T08:56:33.894422+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:21:14.799223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.736540+0000","last_change":"2023-10-18T08:56:31.730514+0000","last_active":"2023-10-18T08:56:33.736540+0000","last_peered":"2023-10-18T08:56:33.736540+0000","last_clean":"2023-10-18T08:56:33.736540+0000","last_became_active":"2023-10-18T08:56:31.730043+0000","last_became_peered":"2023-10-18T08:56:31.730043+0000","last_unstale":"2023-10-18T08:56:33.736540+0000","last_undegraded":"2023-10-18T08:56:33.736540+0000","last_fullsized":"2023-10-18T08:56:33.736540+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:18.076493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899665+0000","last_change":"2023-10-18T08:56:32.753042+0000","last_active":"2023-10-18T08:56:33.899665+0000","last_peered":"2023-10-18T08:56:33.899665+0000","last_clean":"2023-10-18T08:56:33.899665+0000","last_became_active":"2023-10-18T08:56:32.752790+0000","last_became_peered":"2023-10-18T08:56:32.752790+0000","last_unstale":"2023-10-18T08:56:33.899665+0000","last_undegraded":"2023-10-18T08:56:33.899665+0000","last_fullsized":"2023-10-18T08:56:33.899665+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:58.348622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.734880+0000","last_change":"2023-10-18T08:56:31.721895+0000","last_active":"2023-10-18T08:56:33.734880+0000","last_peered":"2023-10-18T08:56:33.734880+0000","last_clean":"2023-10-18T08:56:33.734880+0000","last_became_active":"2023-10-18T08:56:31.721423+0000","last_became_peered":"2023-10-18T08:56:31.721423+0000","last_unstale":"2023-10-18T08:56:33.734880+0000","last_undegraded":"2023-10-18T08:56:33.734880+0000","last_fullsized":"2023-10-18T08:56:33.734880+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:37:21.749663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.731649+0000","last_change":"2023-10-18T08:56:32.766263+0000","last_active":"2023-10-18T08:56:33.731649+0000","last_peered":"2023-10-18T08:56:33.731649+0000","last_clean":"2023-10-18T08:56:33.731649+0000","last_became_active":"2023-10-18T08:56:32.765809+0000","last_became_peered":"2023-10-18T08:56:32.765809+0000","last_unstale":"2023-10-18T08:56:33.731649+0000","last_undegraded":"2023-10-18T08:56:33.731649+0000","last_fullsized":"2023-10-18T08:56:33.731649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:19.090190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.736088+0000","last_change":"2023-10-18T08:56:31.718328+0000","last_active":"2023-10-18T08:56:33.736088+0000","last_peered":"2023-10-18T08:56:33.736088+0000","last_clean":"2023-10-18T08:56:33.736088+0000","last_became_active":"2023-10-18T08:56:31.717951+0000","last_became_peered":"2023-10-18T08:56:31.717951+0000","last_unstale":"2023-10-18T08:56:33.736088+0000","last_undegraded":"2023-10-18T08:56:33.736088+0000","last_fullsized":"2023-10-18T08:56:33.736088+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:23.940891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897013+0000","last_change":"2023-10-18T08:56:32.765411+0000","last_active":"2023-10-18T08:56:33.897013+0000","last_peered":"2023-10-18T08:56:33.897013+0000","last_clean":"2023-10-18T08:56:33.897013+0000","last_became_active":"2023-10-18T08:56:32.765146+0000","last_became_peered":"2023-10-18T08:56:32.765146+0000","last_unstale":"2023-10-18T08:56:33.897013+0000","last_undegraded":"2023-10-18T08:56:33.897013+0000","last_fullsized":"2023-10-18T08:56:33.897013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:52.299309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.732542+0000","last_change":"2023-10-18T08:56:31.725638+0000","last_active":"2023-10-18T08:56:33.732542+0000","last_peered":"2023-10-18T08:56:33.732542+0000","last_clean":"2023-10-18T08:56:33.732542+0000","last_became_active":"2023-10-18T08:56:31.724903+0000","last_became_peered":"2023-10-18T08:56:31.724903+0000","last_unstale":"2023-10-18T08:56:33.732542+0000","last_undegraded":"2023-10-18T08:56:33.732542+0000","last_fullsized":"2023-10-18T08:56:33.732542+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:53:50.586585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.732138+0000","last_change":"2023-10-18T08:56:32.765163+0000","last_active":"2023-10-18T08:56:33.732138+0000","last_peered":"2023-10-18T08:56:33.732138+0000","last_clean":"2023-10-18T08:56:33.732138+0000","last_became_active":"2023-10-18T08:56:32.764872+0000","last_became_peered":"2023-10-18T08:56:32.764872+0000","last_unstale":"2023-10-18T08:56:33.732138+0000","last_undegraded":"2023-10-18T08:56:33.732138+0000","last_fullsized":"2023-10-18T08:56:33.732138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:52.299309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.737085+0000","last_change":"2023-10-18T08:56:31.730319+0000","last_active":"2023-10-18T08:56:33.737085+0000","last_peered":"2023-10-18T08:56:33.737085+0000","last_clean":"2023-10-18T08:56:33.737085+0000","last_became_active":"2023-10-18T08:56:31.729683+0000","last_became_peered":"2023-10-18T08:56:31.729683+0000","last_unstale":"2023-10-18T08:56:33.737085+0000","last_undegraded":"2023-10-18T08:56:33.737085+0000","last_fullsized":"2023-10-18T08:56:33.737085+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:33.794693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.730394+0000","last_change":"2023-10-18T08:56:32.758182+0000","last_active":"2023-10-18T08:56:33.730394+0000","last_peered":"2023-10-18T08:56:33.730394+0000","last_clean":"2023-10-18T08:56:33.730394+0000","last_became_active":"2023-10-18T08:56:32.756265+0000","last_became_peered":"2023-10-18T08:56:32.756265+0000","last_unstale":"2023-10-18T08:56:33.730394+0000","last_undegraded":"2023-10-18T08:56:33.730394+0000","last_fullsized":"2023-10-18T08:56:33.730394+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:44:45.934778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899390+0000","last_change":"2023-10-18T08:56:31.733508+0000","last_active":"2023-10-18T08:56:33.899390+0000","last_peered":"2023-10-18T08:56:33.899390+0000","last_clean":"2023-10-18T08:56:33.899390+0000","last_became_active":"2023-10-18T08:56:31.733283+0000","last_became_peered":"2023-10-18T08:56:31.733283+0000","last_unstale":"2023-10-18T08:56:33.899390+0000","last_undegraded":"2023-10-18T08:56:33.899390+0000","last_fullsized":"2023-10-18T08:56:33.899390+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:39.227696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.897809+0000","last_change":"2023-10-18T08:56:32.737481+0000","last_active":"2023-10-18T08:56:33.897809+0000","last_peered":"2023-10-18T08:56:33.897809+0000","last_clean":"2023-10-18T08:56:33.897809+0000","last_became_active":"2023-10-18T08:56:32.734847+0000","last_became_peered":"2023-10-18T08:56:32.734847+0000","last_unstale":"2023-10-18T08:56:33.897809+0000","last_undegraded":"2023-10-18T08:56:33.897809+0000","last_fullsized":"2023-10-18T08:56:33.897809+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:30:15.179607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.732396+0000","last_change":"2023-10-18T08:56:31.725825+0000","last_active":"2023-10-18T08:56:33.732396+0000","last_peered":"2023-10-18T08:56:33.732396+0000","last_clean":"2023-10-18T08:56:33.732396+0000","last_became_active":"2023-10-18T08:56:31.725148+0000","last_became_peered":"2023-10-18T08:56:31.725148+0000","last_unstale":"2023-10-18T08:56:33.732396+0000","last_undegraded":"2023-10-18T08:56:33.732396+0000","last_fullsized":"2023-10-18T08:56:33.732396+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:30:26.846049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.899854+0000","last_change":"2023-10-18T08:56:32.758487+0000","last_active":"2023-10-18T08:56:33.899854+0000","last_peered":"2023-10-18T08:56:33.899854+0000","last_clean":"2023-10-18T08:56:33.899854+0000","last_became_active":"2023-10-18T08:56:32.756653+0000","last_became_peered":"2023-10-18T08:56:32.756653+0000","last_unstale":"2023-10-18T08:56:33.899854+0000","last_undegraded":"2023-10-18T08:56:33.899854+0000","last_fullsized":"2023-10-18T08:56:33.899854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:31.693686+0000","last_clean_scrub_stamp":"2023-10-18T08:56:31.693686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:21:01.706092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-18T08:56:33.895372+0000","last_change":"2023-10-18T08:56:31.733662+0000","last_active":"2023-10-18T08:56:33.895372+0000","last_peered":"2023-10-18T08:56:33.895372+0000","last_clean":"2023-10-18T08:56:33.895372+0000","last_became_active":"2023-10-18T08:56:31.732888+0000","last_became_peered":"2023-10-18T08:56:31.732888+0000","last_unstale":"2023-10-18T08:56:33.895372+0000","last_undegraded":"2023-10-18T08:56:33.895372+0000","last_fullsized":"2023-10-18T08:56:33.895372+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:56:30.679988+0000","last_clean_scrub_stamp":"2023-10-18T08:56:30.679988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:36:59.902995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-18T08:56:32.730066+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:56:32.730066+0000","last_peered":"2023-10-18T08:56:32.730066+0000","last_clean":"2023-10-18T08:56:32.730066+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:56:32.730066+0000","last_undegraded":"2023-10-18T08:56:32.730066+0000","last_fullsized":"2023-10-18T08:56:32.730066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:34.881744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":719,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194592,"statfs":{"total":200000000,"available":199262774,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:56:05 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.2849999999999999},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":1.667}]},{"osd":1,"last update":"Wed Oct 18 08:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.024},{"interface":"front","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.325}]},{"osd":2,"last update":"Wed Oct 18 08:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.456},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":1.4179999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.5309999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.613},{"interface":"front","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.5169999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.5049999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":1.552}]},{"osd":6,"last update":"Wed Oct 18 08:56:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.383}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":137,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195174,"statfs":{"total":200000000,"available":199858916,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":1.3360000000000001}]},{"osd":1,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":1.04},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.9609999999999999,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":0.94299999999999995}]},{"osd":2,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.01},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.127}]},{"osd":3,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.383}]},{"osd":4,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.593},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.0860000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.2150000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.794,"5min":2.794,"15min":2.794},"last":1.288}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":137,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195174,"statfs":{"total":200000000,"available":199858860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":1.474},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.4239999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":3.7050000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":1.3280000000000001}]},{"osd":2,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":3.8180000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.266}]},{"osd":3,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":4.7119999999999997,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":1.153},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"min":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.0109999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":5.4580000000000002,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":1.0649999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":5.141,"5min":5.141,"15min":5.141},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.603,"5min":1.603,"15min":1.603},"min":{"1min":1.171,"5min":1.171,"15min":1.171},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.1919999999999999}]},{"osd":7,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"min":{"1min":1.014,"5min":1.014,"15min":1.014},"max":{"1min":5.5490000000000004,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":4.8419999999999996,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":1.2390000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":136,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199860447,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.2599999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.1439999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.92500000000000004}]},{"osd":2,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.3380000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"max":{"1min":2.8010000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.4610000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"min":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":1.4219999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.998,"5min":0.998,"15min":0.998},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.573},{"interface":"front","average":{"1min":1.583,"5min":1.583,"15min":1.583},"min":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"max":{"1min":3.6859999999999999,"5min":3.6859999999999999,"15min":3.6859999999999999},"last":1.6200000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:56:04 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.524,"15min":1.524},"min":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":1.23,"5min":1.23,"15min":1.23},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.665}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":144,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195167,"statfs":{"total":200000000,"available":199851598,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":1.075},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.73599999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.272}]},{"osd":2,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.1599999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.054}]},{"osd":5,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":1.006},{"interface":"front","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":1.335}]},{"osd":6,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.466},{"interface":"front","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":1.296}]},{"osd":7,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.298,"15min":1.298},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.397}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":137,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195175,"statfs":{"total":200000000,"available":199859462,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":4.2210000000000001,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.77700000000000002}]},{"osd":1,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":4.29,"5min":4.29,"15min":4.29},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.97699999999999998}]},{"osd":3,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":7.5570000000000004,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":1.1180000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":7.6189999999999998,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":1.228},{"interface":"front","average":{"1min":1.468,"5min":1.468,"15min":1.468},"min":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.663}]},{"osd":5,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":4.4320000000000004,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":1.012},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":7.657,"5min":7.657,"15min":7.657},"last":1.4119999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":3.5299999999999998,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":4.3920000000000003,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":1.516}]},{"osd":7,"last update":"Wed Oct 18 08:56:06 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.595,"15min":1.595},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":3.3620000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":4.3390000000000004,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.4590000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":137,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195174,"statfs":{"total":200000000,"available":199858917,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.67200000000000004}]},{"osd":2,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.65200000000000002}]},{"osd":3,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.7010000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":0.74399999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.127},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.94099999999999995}]},{"osd":5,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.5750000000000002,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.147},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":2.3769999999999998,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.0900000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.272,"15min":1.272},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.212}]},{"osd":7,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.1759999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":712,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194599,"statfs":{"total":200000000,"available":199270183,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":3.4420000000000002,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.254}]},{"osd":2,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.92600000000000005}]},{"osd":3,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":3.5720000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.2889999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":4.492,"5min":4.492,"15min":4.492},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.1259999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":4.2779999999999996,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.758}]},{"osd":6,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"min":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"max":{"1min":3.5299999999999998,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":4.4349999999999996,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.173}]},{"osd":7,"last update":"Wed Oct 18 08:56:07 2023","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":3.492,"5min":3.492,"15min":3.492},"last":1.603},{"interface":"front","average":{"1min":1.7589999999999999,"5min":1.7589999999999999,"15min":1.7589999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":4.5419999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.048}]}]}],"pool_statfs":[]}} 2023-10-18T08:56:37.854 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T08:56:37.855 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T08:56:38.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.134+0000 7f5521f35640 1 -- 172.21.15.40:0/422892187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f550c0039a0 msgr2=0x7f550c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:38.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.134+0000 7f5521f35640 1 --2- 172.21.15.40:0/422892187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f550c0039a0 0x7f550c003dc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5510009b20 tx=0x7f550c0057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:38.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.134+0000 7f5521734640 1 -- 172.21.15.40:0/422892187 reap_dead start 2023-10-18T08:56:38.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.134+0000 7f5521f35640 1 -- 172.21.15.40:0/422892187 shutdown_connections 2023-10-18T08:56:38.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.134+0000 7f5521f35640 1 -- 172.21.15.40:0/422892187 >> 172.21.15.40:0/422892187 conn(0x7f551c06ca10 msgr2=0x7f551c06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:38.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.134+0000 7f5521f35640 1 -- 172.21.15.40:0/422892187 shutdown_connections 2023-10-18T08:56:38.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.134+0000 7f5521f35640 1 -- 172.21.15.40:0/422892187 wait complete. 2023-10-18T08:56:38.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.136+0000 7f5521f35640 1 Processor -- start 2023-10-18T08:56:38.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.136+0000 7f5521f35640 1 -- start start 2023-10-18T08:56:38.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.137+0000 7f5521f35640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f550c0039a0 0x7f551c140f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:38.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.137+0000 7f5521f35640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c144130 0x7f551c1416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:38.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.137+0000 7f5521f35640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f551c0aa880 0x7f551c0aad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:38.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.137+0000 7f5521f35640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c0ab240 con 0x7f551c144130 2023-10-18T08:56:38.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.137+0000 7f5521f35640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c0c7260 con 0x7f550c0039a0 2023-10-18T08:56:38.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.137+0000 7f5521f35640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551c0c7540 con 0x7f551c0aa880 2023-10-18T08:56:38.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.137+0000 7f551bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c144130 0x7f551c1416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:38.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.137+0000 7f551bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c144130 0x7f551c1416d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50984/0 (socket says 172.21.15.40:50984) 2023-10-18T08:56:38.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.137+0000 7f551bfff640 1 -- 172.21.15.40:0/883580737 learned_addr learned my addr 172.21.15.40:0/883580737 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:38.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.138+0000 7f551bfff640 1 -- 172.21.15.40:0/883580737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f551c0aa880 msgr2=0x7f551c0aad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:38.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.138+0000 7f551bfff640 1 --2- 172.21.15.40:0/883580737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f551c0aa880 0x7f551c0aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:38.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.138+0000 7f551bfff640 1 -- 172.21.15.40:0/883580737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f550c0039a0 msgr2=0x7f551c140f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:38.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.138+0000 7f551bfff640 1 --2- 172.21.15.40:0/883580737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f550c0039a0 0x7f551c140f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:38.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.138+0000 7f551bfff640 1 -- 172.21.15.40:0/883580737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c0c7820 con 0x7f551c144130 2023-10-18T08:56:38.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.138+0000 7f551bfff640 1 --2- 172.21.15.40:0/883580737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c144130 0x7f551c1416d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f550800d580 tx=0x7f550800da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:38.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.138+0000 7f55197fa640 1 -- 172.21.15.40:0/883580737 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508012630 con 0x7f551c144130 2023-10-18T08:56:38.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.138+0000 7f55197fa640 1 -- 172.21.15.40:0/883580737 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5508012790 con 0x7f551c144130 2023-10-18T08:56:38.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.138+0000 7f55197fa640 1 -- 172.21.15.40:0/883580737 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508012bc0 con 0x7f551c144130 2023-10-18T08:56:38.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.138+0000 7f5521f35640 1 -- 172.21.15.40:0/883580737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551c0c7b00 con 0x7f551c144130 2023-10-18T08:56:38.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.138+0000 7f5521f35640 1 -- 172.21.15.40:0/883580737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551c144590 con 0x7f551c144130 2023-10-18T08:56:38.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.140+0000 7f5521f35640 1 -- 172.21.15.40:0/883580737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f551c0ccb20 con 0x7f551c144130 2023-10-18T08:56:38.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.141+0000 7f55197fa640 1 -- 172.21.15.40:0/883580737 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f550801b400 con 0x7f551c144130 2023-10-18T08:56:38.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.141+0000 7f55197fa640 1 --2- 172.21.15.40:0/883580737 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f550006ae70 0x7f550006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:38.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.142+0000 7f55197fa640 1 -- 172.21.15.40:0/883580737 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5508089960 con 0x7f551c144130 2023-10-18T08:56:38.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.142+0000 7f5520f33640 1 --2- 172.21.15.40:0/883580737 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f550006ae70 0x7f550006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:38.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.143+0000 7f5520f33640 1 --2- 172.21.15.40:0/883580737 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f550006ae70 0x7f550006d330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5510036000 tx=0x7f5510035040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:38.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.146+0000 7f55197fa640 1 -- 172.21.15.40:0/883580737 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5508054980 con 0x7f551c144130 2023-10-18T08:56:38.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.414+0000 7f5521f35640 1 -- 172.21.15.40:0/883580737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f551c066e00 con 0x7f551c144130 2023-10-18T08:56:38.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.417+0000 7f55197fa640 1 -- 172.21.15.40:0/883580737 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f5508057fa0 con 0x7f551c144130 2023-10-18T08:56:38.420 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2023-10-18T08:56:38.420 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w== 2023-10-18T08:56:38.421 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2023-10-18T08:56:38.421 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2023-10-18T08:56:38.421 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T08:56:38.421 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2023-10-18T08:56:38.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.421+0000 7f5521f35640 1 -- 172.21.15.40:0/883580737 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f550006ae70 msgr2=0x7f550006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:38.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.421+0000 7f5521f35640 1 --2- 172.21.15.40:0/883580737 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f550006ae70 0x7f550006d330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5510036000 tx=0x7f5510035040 comp rx=0 tx=0).stop 2023-10-18T08:56:38.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.421+0000 7f5521f35640 1 -- 172.21.15.40:0/883580737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c144130 msgr2=0x7f551c1416d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:38.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.421+0000 7f5521f35640 1 --2- 172.21.15.40:0/883580737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c144130 0x7f551c1416d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f550800d580 tx=0x7f550800da40 comp rx=0 tx=0).stop 2023-10-18T08:56:38.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.421+0000 7f5521f35640 1 -- 172.21.15.40:0/883580737 shutdown_connections 2023-10-18T08:56:38.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.421+0000 7f5521f35640 1 --2- 172.21.15.40:0/883580737 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f550006ae70 0x7f550006d330 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:38.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.421+0000 7f5521f35640 1 --2- 172.21.15.40:0/883580737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f551c0aa880 0x7f551c0aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:38.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.421+0000 7f5521f35640 1 --2- 172.21.15.40:0/883580737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c144130 0x7f551c1416d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:38.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.421+0000 7f5521f35640 1 --2- 172.21.15.40:0/883580737 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f550c0039a0 0x7f551c140f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:38.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.421+0000 7f5521f35640 1 -- 172.21.15.40:0/883580737 >> 172.21.15.40:0/883580737 conn(0x7f551c06ca10 msgr2=0x7f551c06cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:38.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.421+0000 7f5521f35640 1 -- 172.21.15.40:0/883580737 shutdown_connections 2023-10-18T08:56:38.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.421+0000 7f5521f35640 1 -- 172.21.15.40:0/883580737 wait complete. 2023-10-18T08:56:38.441 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:56:38.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.671+0000 7fb7d9244640 1 -- 172.21.15.40:0/3565982947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7c0003970 msgr2=0x7fb7c0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:38.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.671+0000 7fb7d9244640 1 --2- 172.21.15.40:0/3565982947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7c0003970 0x7fb7c0003d90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009b50 tx=0x7fb7c00057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:38.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.671+0000 7fb7d359e640 1 -- 172.21.15.40:0/3565982947 reap_dead start 2023-10-18T08:56:38.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.671+0000 7fb7d9244640 1 -- 172.21.15.40:0/3565982947 shutdown_connections 2023-10-18T08:56:38.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.671+0000 7fb7d9244640 1 -- 172.21.15.40:0/3565982947 >> 172.21.15.40:0/3565982947 conn(0x7fb7d406cee0 msgr2=0x7fb7d4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:38.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.671+0000 7fb7d9244640 1 -- 172.21.15.40:0/3565982947 shutdown_connections 2023-10-18T08:56:38.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.672+0000 7fb7d9244640 1 -- 172.21.15.40:0/3565982947 wait complete. 2023-10-18T08:56:38.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.673+0000 7fb7d9244640 1 Processor -- start 2023-10-18T08:56:38.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.673+0000 7fb7d9244640 1 -- start start 2023-10-18T08:56:38.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.674+0000 7fb7d9244640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7c0003970 0x7fb7d4142340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:38.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.674+0000 7fb7d9244640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d4156200 0x7fb7d4070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:38.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.674+0000 7fb7d9244640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7d4156660 0x7fb7d4070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:38.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.674+0000 7fb7d9244640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d413bf30 con 0x7fb7d4156200 2023-10-18T08:56:38.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.674+0000 7fb7d9244640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d41574d0 con 0x7fb7d4156660 2023-10-18T08:56:38.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.674+0000 7fb7d9244640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d413ff30 con 0x7fb7c0003970 2023-10-18T08:56:38.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.675+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d4156200 0x7fb7d4070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:38.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.675+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d4156200 0x7fb7d4070ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51002/0 (socket says 172.21.15.40:51002) 2023-10-18T08:56:38.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.675+0000 7fb7cbfff640 1 -- 172.21.15.40:0/3312848368 learned_addr learned my addr 172.21.15.40:0/3312848368 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:38.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.675+0000 7fb7cbfff640 1 -- 172.21.15.40:0/3312848368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7c0003970 msgr2=0x7fb7d4142340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:38.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.675+0000 7fb7cbfff640 1 --2- 172.21.15.40:0/3312848368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7c0003970 0x7fb7d4142340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:38.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.675+0000 7fb7cbfff640 1 -- 172.21.15.40:0/3312848368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7d4156660 msgr2=0x7fb7d4070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:38.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.675+0000 7fb7cbfff640 1 --2- 172.21.15.40:0/3312848368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7d4156660 0x7fb7d4070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:38.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.675+0000 7fb7cbfff640 1 -- 172.21.15.40:0/3312848368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d4140210 con 0x7fb7d4156200 2023-10-18T08:56:38.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.675+0000 7fb7cbfff640 1 --2- 172.21.15.40:0/3312848368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d4156200 0x7fb7d4070ab0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc010a10 tx=0x7fb7bc010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:38.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.676+0000 7fb7cb7fe640 1 -- 172.21.15.40:0/3312848368 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc009720 con 0x7fb7d4156200 2023-10-18T08:56:38.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.676+0000 7fb7cb7fe640 1 -- 172.21.15.40:0/3312848368 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb7bc00eda0 con 0x7fb7d4156200 2023-10-18T08:56:38.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.676+0000 7fb7cb7fe640 1 -- 172.21.15.40:0/3312848368 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc005280 con 0x7fb7d4156200 2023-10-18T08:56:38.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.676+0000 7fb7d9244640 1 -- 172.21.15.40:0/3312848368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d4157790 con 0x7fb7d4156200 2023-10-18T08:56:38.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.676+0000 7fb7d9244640 1 -- 172.21.15.40:0/3312848368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d4142880 con 0x7fb7d4156200 2023-10-18T08:56:38.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.677+0000 7fb7d9244640 1 -- 172.21.15.40:0/3312848368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d4140400 con 0x7fb7d4156200 2023-10-18T08:56:38.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.678+0000 7fb7cb7fe640 1 -- 172.21.15.40:0/3312848368 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb7bc0053e0 con 0x7fb7d4156200 2023-10-18T08:56:38.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.679+0000 7fb7cb7fe640 1 --2- 172.21.15.40:0/3312848368 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7b406ae40 0x7fb7b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:38.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.680+0000 7fb7cb7fe640 1 -- 172.21.15.40:0/3312848368 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7bc016070 con 0x7fb7d4156200 2023-10-18T08:56:38.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.680+0000 7fb7d2d9d640 1 --2- 172.21.15.40:0/3312848368 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7b406ae40 0x7fb7b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:38.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.681+0000 7fb7d2d9d640 1 --2- 172.21.15.40:0/3312848368 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7b406ae40 0x7fb7b406d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4036000 tx=0x7fb7c4035040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:38.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.684+0000 7fb7cb7fe640 1 -- 172.21.15.40:0/3312848368 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb7bc0554d0 con 0x7fb7d4156200 2023-10-18T08:56:38.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.911+0000 7fb7d9244640 1 -- 172.21.15.40:0/3312848368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7d4066fd0 con 0x7fb7d4156200 2023-10-18T08:56:38.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.912+0000 7fb7cb7fe640 1 -- 172.21.15.40:0/3312848368 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fb7bc058af0 con 0x7fb7d4156200 2023-10-18T08:56:38.915 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:38.915 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/837790640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":837790640},{"type":"v1","addr":"172.21.15.40:6833","nonce":837790640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3377302775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3377302775},{"type":"v1","addr":"172.21.15.144:6836","nonce":3377302775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4939,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3162305748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3162305748},{"type":"v1","addr":"172.21.15.144:6837","nonce":3162305748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:56:32.203221+0000","modified":"2023-10-18T08:56:35.403466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5190":{"gid":5190,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3167454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3167454356},{"type":"v1","addr":"172.21.15.40:6835","nonce":3167454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T08:56:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2023-10-18T08:56:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.915+0000 7fb7d9244640 1 -- 172.21.15.40:0/3312848368 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7b406ae40 msgr2=0x7fb7b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.915+0000 7fb7d9244640 1 --2- 172.21.15.40:0/3312848368 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7b406ae40 0x7fb7b406d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4036000 tx=0x7fb7c4035040 comp rx=0 tx=0).stop 2023-10-18T08:56:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.915+0000 7fb7d9244640 1 -- 172.21.15.40:0/3312848368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d4156200 msgr2=0x7fb7d4070ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.915+0000 7fb7d9244640 1 --2- 172.21.15.40:0/3312848368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d4156200 0x7fb7d4070ab0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc010a10 tx=0x7fb7bc010e10 comp rx=0 tx=0).stop 2023-10-18T08:56:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.915+0000 7fb7d9244640 1 -- 172.21.15.40:0/3312848368 shutdown_connections 2023-10-18T08:56:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.915+0000 7fb7d9244640 1 --2- 172.21.15.40:0/3312848368 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7b406ae40 0x7fb7b406d300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.915+0000 7fb7d9244640 1 --2- 172.21.15.40:0/3312848368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7d4156660 0x7fb7d4070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.915+0000 7fb7d9244640 1 --2- 172.21.15.40:0/3312848368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7d4156200 0x7fb7d4070ab0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.915+0000 7fb7d9244640 1 --2- 172.21.15.40:0/3312848368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7c0003970 0x7fb7d4142340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:38.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.915+0000 7fb7d9244640 1 -- 172.21.15.40:0/3312848368 >> 172.21.15.40:0/3312848368 conn(0x7fb7d406cee0 msgr2=0x7fb7d40abb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:38.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.915+0000 7fb7d9244640 1 -- 172.21.15.40:0/3312848368 shutdown_connections 2023-10-18T08:56:38.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:38.916+0000 7fb7d9244640 1 -- 172.21.15.40:0/3312848368 wait complete. 2023-10-18T08:56:39.339 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T08:56:32.203221+0000', 'modified': '2023-10-18T08:56:35.403466+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5190}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5190': {'gid': 5190, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.40:6835/3167454356', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 3167454356}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 3167454356}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T08:56:39.340 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T08:56:39.340 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-18T08:56:39.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.579+0000 7fbc7aff4640 1 -- 172.21.15.40:0/2804906444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc7415b880 msgr2=0x7fbc740cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:39.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.579+0000 7fbc7aff4640 1 --2- 172.21.15.40:0/2804906444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc7415b880 0x7fbc740cfe60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbc64005fd0 tx=0x7fbc6402be20 comp rx=0 tx=0).stop 2023-10-18T08:56:39.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.579+0000 7fbc7aff4640 1 -- 172.21.15.40:0/2804906444 shutdown_connections 2023-10-18T08:56:39.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.579+0000 7fbc7aff4640 1 --2- 172.21.15.40:0/2804906444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc740cdca0 0x7fbc740ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.579+0000 7fbc7aff4640 1 --2- 172.21.15.40:0/2804906444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc7415b880 0x7fbc740cfe60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.579+0000 7fbc7aff4640 1 -- 172.21.15.40:0/2804906444 >> 172.21.15.40:0/2804906444 conn(0x7fbc7408aca0 msgr2=0x7fbc7409a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:39.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.579+0000 7fbc7aff4640 1 -- 172.21.15.40:0/2804906444 shutdown_connections 2023-10-18T08:56:39.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.579+0000 7fbc7aff4640 1 -- 172.21.15.40:0/2804906444 wait complete. 2023-10-18T08:56:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.581+0000 7fbc7aff4640 1 Processor -- start 2023-10-18T08:56:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.581+0000 7fbc7aff4640 1 -- start start 2023-10-18T08:56:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.582+0000 7fbc7aff4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc740cdca0 0x7fbc7415e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.582+0000 7fbc7aff4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc7415b4a0 0x7fbc740b6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.582+0000 7fbc7aff4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc740b3eb0 0x7fbc740d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.582+0000 7fbc7aff4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7415ce70 con 0x7fbc740cdca0 2023-10-18T08:56:39.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.582+0000 7fbc7aff4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc740c4dc0 con 0x7fbc740b3eb0 2023-10-18T08:56:39.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.582+0000 7fbc7aff4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc740c50a0 con 0x7fbc7415b4a0 2023-10-18T08:56:39.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.582+0000 7fbc79ff2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc740cdca0 0x7fbc7415e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.582+0000 7fbc79ff2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc740cdca0 0x7fbc7415e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51026/0 (socket says 172.21.15.40:51026) 2023-10-18T08:56:39.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.582+0000 7fbc79ff2640 1 -- 172.21.15.40:0/2841386376 learned_addr learned my addr 172.21.15.40:0/2841386376 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:39.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.583+0000 7fbc797f1640 1 --2- 172.21.15.40:0/2841386376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc7415b4a0 0x7fbc740b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.583+0000 7fbc7a7f3640 1 --2- 172.21.15.40:0/2841386376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc740b3eb0 0x7fbc740d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.583+0000 7fbc79ff2640 1 -- 172.21.15.40:0/2841386376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc7415b4a0 msgr2=0x7fbc740b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:39.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.583+0000 7fbc79ff2640 1 --2- 172.21.15.40:0/2841386376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc7415b4a0 0x7fbc740b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.583+0000 7fbc79ff2640 1 -- 172.21.15.40:0/2841386376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc740b3eb0 msgr2=0x7fbc740d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:39.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.583+0000 7fbc79ff2640 1 --2- 172.21.15.40:0/2841386376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc740b3eb0 0x7fbc740d0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.583+0000 7fbc79ff2640 1 -- 172.21.15.40:0/2841386376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc64009560 con 0x7fbc740cdca0 2023-10-18T08:56:39.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.583+0000 7fbc797f1640 1 --2- 172.21.15.40:0/2841386376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc7415b4a0 0x7fbc740b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:39.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.583+0000 7fbc7a7f3640 1 --2- 172.21.15.40:0/2841386376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc740b3eb0 0x7fbc740d0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:39.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.583+0000 7fbc79ff2640 1 --2- 172.21.15.40:0/2841386376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc740cdca0 0x7fbc7415e6c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000c870 tx=0x7fbc7000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:39.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.583+0000 7fbc6a7fc640 1 -- 172.21.15.40:0/2841386376 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70003b70 con 0x7fbc740cdca0 2023-10-18T08:56:39.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.583+0000 7fbc6a7fc640 1 -- 172.21.15.40:0/2841386376 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbc70003cf0 con 0x7fbc740cdca0 2023-10-18T08:56:39.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.583+0000 7fbc6a7fc640 1 -- 172.21.15.40:0/2841386376 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70003ea0 con 0x7fbc740cdca0 2023-10-18T08:56:39.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.583+0000 7fbc7aff4640 1 -- 172.21.15.40:0/2841386376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7415c7b0 con 0x7fbc740cdca0 2023-10-18T08:56:39.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.584+0000 7fbc7aff4640 1 -- 172.21.15.40:0/2841386376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc740c5380 con 0x7fbc740cdca0 2023-10-18T08:56:39.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.587+0000 7fbc6a7fc640 1 -- 172.21.15.40:0/2841386376 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbc7000aab0 con 0x7fbc740cdca0 2023-10-18T08:56:39.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.588+0000 7fbc6a7fc640 1 --2- 172.21.15.40:0/2841386376 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc5806ae40 0x7fbc5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.588+0000 7fbc6a7fc640 1 -- 172.21.15.40:0/2841386376 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc700895d0 con 0x7fbc740cdca0 2023-10-18T08:56:39.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.588+0000 7fbc7edea740 1 Processor -- start 2023-10-18T08:56:39.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.588+0000 7fbc797f1640 1 --2- 172.21.15.40:0/2841386376 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc5806ae40 0x7fbc5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.588+0000 7fbc7edea740 1 -- start start 2023-10-18T08:56:39.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.589+0000 7fbc7edea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4ed970 0x55696f4c0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.589+0000 7fbc7edea740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f4f0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.589+0000 7fbc7edea740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4c10e0 0x55696f4f0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.589+0000 7fbc7edea740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55696f433740 con 0x55696f4edd50 2023-10-18T08:56:39.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.589+0000 7fbc7edea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55696f430e30 con 0x55696f4ed970 2023-10-18T08:56:39.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.589+0000 7fbc7edea740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55696f42c720 con 0x55696f4c10e0 2023-10-18T08:56:39.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.590+0000 7fbc797f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f4f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.590+0000 7fbc797f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f4f0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51038/0 (socket says 172.21.15.40:51038) 2023-10-18T08:56:39.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.590+0000 7fbc797f1640 1 -- 172.21.15.40:0/1567034864 learned_addr learned my addr 172.21.15.40:0/1567034864 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:39.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.590+0000 7fbc797f1640 1 --2- 172.21.15.40:0/2841386376 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc5806ae40 0x7fbc5806d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc64035000 tx=0x7fbc64004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:39.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.590+0000 7fbc79ff2640 1 --2- 172.21.15.40:0/1567034864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4ed970 0x55696f4c0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.590+0000 7fbc7a7f3640 1 --2- 172.21.15.40:0/1567034864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4c10e0 0x55696f4f0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.590+0000 7fbc797f1640 1 -- 172.21.15.40:0/1567034864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4c10e0 msgr2=0x55696f4f0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:56:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.590+0000 7fbc797f1640 1 --2- 172.21.15.40:0/1567034864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4c10e0 0x55696f4f0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.590+0000 7fbc797f1640 1 -- 172.21.15.40:0/1567034864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4ed970 msgr2=0x55696f4c0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:56:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.590+0000 7fbc797f1640 1 --2- 172.21.15.40:0/1567034864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4ed970 0x55696f4c0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.590+0000 7fbc797f1640 1 -- 172.21.15.40:0/1567034864 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55696f3f0910 con 0x55696f4edd50 2023-10-18T08:56:39.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.590+0000 7fbc7a7f3640 1 --2- 172.21.15.40:0/1567034864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4c10e0 0x55696f4f0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:39.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.590+0000 7fbc79ff2640 1 --2- 172.21.15.40:0/1567034864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4ed970 0x55696f4c0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:39.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.591+0000 7fbc797f1640 1 --2- 172.21.15.40:0/1567034864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f4f0700 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbc64000c00 tx=0x7fbc6403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f319436c32d18cb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:39.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.591+0000 7fbc577fe640 1 -- 172.21.15.40:0/1567034864 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc640035f0 con 0x55696f4edd50 2023-10-18T08:56:39.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.591+0000 7fbc577fe640 1 -- 172.21.15.40:0/1567034864 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbc64044070 con 0x55696f4edd50 2023-10-18T08:56:39.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.591+0000 7fbc577fe640 1 -- 172.21.15.40:0/1567034864 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc640039d0 con 0x55696f4edd50 2023-10-18T08:56:39.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.591+0000 7fbc7edea740 1 -- 172.21.15.40:0/1567034864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 msgr2=0x55696f4f0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:39.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.591+0000 7fbc7edea740 1 --2- 172.21.15.40:0/1567034864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f4f0700 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbc64000c00 tx=0x7fbc6403d000 comp rx=0 tx=0).stop 2023-10-18T08:56:39.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.592+0000 7fbc7edea740 1 -- 172.21.15.40:0/1567034864 shutdown_connections 2023-10-18T08:56:39.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.592+0000 7fbc7edea740 1 --2- 172.21.15.40:0/1567034864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4c10e0 0x55696f4f0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.592+0000 7fbc7edea740 1 --2- 172.21.15.40:0/1567034864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f4f0700 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.592+0000 7fbc7edea740 1 --2- 172.21.15.40:0/1567034864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4ed970 0x55696f4c0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.592+0000 7fbc7edea740 1 -- 172.21.15.40:0/1567034864 >> 172.21.15.40:0/1567034864 conn(0x55696f4c2e10 msgr2=0x55696f50b700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:39.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.592+0000 7fbc7edea740 1 -- 172.21.15.40:0/1567034864 shutdown_connections 2023-10-18T08:56:39.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.592+0000 7fbc7edea740 1 -- 172.21.15.40:0/1567034864 wait complete. 2023-10-18T08:56:39.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.592+0000 7fbc7edea740 1 -- start start 2023-10-18T08:56:39.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.593+0000 7fbc7edea740 1 Processor -- start 2023-10-18T08:56:39.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc7edea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4ed970 0x55696f50bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc7edea740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f4f2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc7edea740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4f2c80 0x55696f50ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc7edea740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55696f3ca970 con 0x55696f4edd50 2023-10-18T08:56:39.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc7edea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55696f3ca610 con 0x55696f4ed970 2023-10-18T08:56:39.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc7edea740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55696f190f20 con 0x55696f4f2c80 2023-10-18T08:56:39.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc7edea740 10 client.0 _ll_register_callbacks cb 0x55696f3ee0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T08:56:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc797f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f4f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc797f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f4f2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51042/0 (socket says 172.21.15.40:51042) 2023-10-18T08:56:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc797f1640 1 -- 172.21.15.40:0/78928851 learned_addr learned my addr 172.21.15.40:0/78928851 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc7a7f3640 1 --2- 172.21.15.40:0/78928851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4f2c80 0x55696f50ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc79ff2640 1 --2- 172.21.15.40:0/78928851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4ed970 0x55696f50bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc797f1640 1 -- 172.21.15.40:0/78928851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4f2c80 msgr2=0x55696f50ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:56:39.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc797f1640 1 --2- 172.21.15.40:0/78928851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4f2c80 0x55696f50ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc797f1640 1 -- 172.21.15.40:0/78928851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4ed970 msgr2=0x55696f50bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:56:39.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc797f1640 1 --2- 172.21.15.40:0/78928851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4ed970 0x55696f50bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.594+0000 7fbc797f1640 1 -- 172.21.15.40:0/78928851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc740cc9b0 con 0x55696f4edd50 2023-10-18T08:56:39.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.595+0000 7fbc7a7f3640 1 --2- 172.21.15.40:0/78928851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4f2c80 0x55696f50ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:39.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.595+0000 7fbc797f1640 1 --2- 172.21.15.40:0/78928851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f4f2740 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbc64000c00 tx=0x7fbc64003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630ee31c4086a7f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:39.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.595+0000 7fbc54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T08:56:39.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.595+0000 7fbc54ff9640 1 -- 172.21.15.40:0/78928851 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64040470 con 0x55696f4edd50 2023-10-18T08:56:39.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.595+0000 7fbc54ff9640 1 -- 172.21.15.40:0/78928851 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc6405fae0 con 0x55696f4edd50 2023-10-18T08:56:39.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.595+0000 7fbc7edea740 1 -- 172.21.15.40:0/78928851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55696f425700 con 0x55696f4edd50 2023-10-18T08:56:39.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.595+0000 7fbc54ff9640 1 -- 172.21.15.40:0/78928851 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc6405fca0 con 0x55696f4edd50 2023-10-18T08:56:39.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.595+0000 7fbc7edea740 10 client.5355 fetch_fsmap learned FSMap version 16 2023-10-18T08:56:39.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.595+0000 7fbc7edea740 1 -- 172.21.15.40:0/78928851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55696f42ada0 con 0x55696f4edd50 2023-10-18T08:56:39.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.596+0000 7fbc54ff9640 1 -- 172.21.15.40:0/78928851 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55696f425700 con 0x55696f4edd50 2023-10-18T08:56:39.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.596+0000 7fbc54ff9640 1 -- 172.21.15.40:0/78928851 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbc6403c040 con 0x55696f4edd50 2023-10-18T08:56:39.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.596+0000 7fbc7edea740 10 client.5355 fetch_fsmap finished waiting for FSMap version 16 2023-10-18T08:56:39.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.596+0000 7fbc7edea740 10 client.5355 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T08:56:39.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.596+0000 7fbc7edea740 20 client.5355 populate_metadata read hostname 'smithi040' 2023-10-18T08:56:39.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.596+0000 7fbc7edea740 1 --2- 172.21.15.40:0/78928851 >> [v2:172.21.15.40:6834/3167454356,v1:172.21.15.40:6835/3167454356] conn(0x55696f4de400 0x55696f4dc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.596+0000 7fbc7edea740 4 client.5355 mds_command: new command op to 5190 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T08:56:39.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.596+0000 7fbc7edea740 1 -- 172.21.15.40:0/78928851 --> [v2:172.21.15.40:6834/3167454356,v1:172.21.15.40:6835/3167454356] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55696f425420 con 0x55696f4de400 2023-10-18T08:56:39.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.596+0000 7fbc79ff2640 1 --2- 172.21.15.40:0/78928851 >> [v2:172.21.15.40:6834/3167454356,v1:172.21.15.40:6835/3167454356] conn(0x55696f4de400 0x55696f4dc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.597+0000 7fbc79ff2640 1 --2- 172.21.15.40:0/78928851 >> [v2:172.21.15.40:6834/3167454356,v1:172.21.15.40:6835/3167454356] conn(0x55696f4de400 0x55696f4dc270 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50ddaa164e1e078d server_cookie=ce0dd9eda688eb00 in_seq=0 out_seq=0 2023-10-18T08:56:39.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.597+0000 7fbc54ff9640 10 client.5355 ms_handle_connect on v2:172.21.15.40:6834/3167454356 2023-10-18T08:56:39.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.597+0000 7fbc54ff9640 1 -- 172.21.15.40:0/78928851 <== mds.0 v2:172.21.15.40:6834/3167454356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55696f425420 con 0x55696f4de400 2023-10-18T08:56:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.597+0000 7fbc54ff9640 10 client.5355 handle_command_reply: tid=0 2023-10-18T08:56:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.598+0000 7fbc7edea740 1 client.5355 shutdown 2023-10-18T08:56:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.598+0000 7fbc7edea740 1 -- 172.21.15.40:0/78928851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 msgr2=0x55696f4f2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.598+0000 7fbc7edea740 1 --2- 172.21.15.40:0/78928851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f4f2740 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbc64000c00 tx=0x7fbc64003d20 comp rx=0 tx=0).stop 2023-10-18T08:56:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.598+0000 7fbc7edea740 1 -- 172.21.15.40:0/78928851 shutdown_connections 2023-10-18T08:56:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.598+0000 7fbc7edea740 1 --2- 172.21.15.40:0/78928851 >> [v2:172.21.15.40:6834/3167454356,v1:172.21.15.40:6835/3167454356] conn(0x55696f4de400 0x55696f4dc270 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.598+0000 7fbc7edea740 1 --2- 172.21.15.40:0/78928851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4f2c80 0x55696f50ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.598+0000 7fbc7edea740 1 --2- 172.21.15.40:0/78928851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f4f2740 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.598+0000 7fbc7edea740 1 --2- 172.21.15.40:0/78928851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4ed970 0x55696f50bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.598+0000 7fbc54ff9640 0 client.5355 ms_handle_reset on v2:172.21.15.40:6834/3167454356 2023-10-18T08:56:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.598+0000 7fbc7edea740 1 -- 172.21.15.40:0/78928851 >> 172.21.15.40:0/78928851 conn(0x55696f4c10e0 msgr2=0x55696f4c4720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:39.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.598+0000 7fbc7edea740 1 -- 172.21.15.40:0/78928851 shutdown_connections 2023-10-18T08:56:39.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.598+0000 7fbc7edea740 1 -- 172.21.15.40:0/78928851 wait complete. 2023-10-18T08:56:39.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.598+0000 7fbc7edea740 20 client.5355 trim_cache size 0 max 16384 2023-10-18T08:56:39.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.615+0000 7fbc7edea740 1 Processor -- start 2023-10-18T08:56:39.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.615+0000 7fbc7edea740 1 -- start start 2023-10-18T08:56:39.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.616+0000 7fbc7edea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4ed970 0x55696f4dd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.616+0000 7fbc7edea740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4edd50 0x55696f4f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.616+0000 7fbc7edea740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4f2c80 0x55696f4f4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.616+0000 7fbc7edea740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55696f51ab40 con 0x55696f4f2c80 2023-10-18T08:56:39.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.616+0000 7fbc7edea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55696f4ee2a0 con 0x55696f4ed970 2023-10-18T08:56:39.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.616+0000 7fbc7edea740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55696f50e3d0 con 0x55696f4edd50 2023-10-18T08:56:39.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.616+0000 7fbc7a7f3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4f2c80 0x55696f4f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.616+0000 7fbc7a7f3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4f2c80 0x55696f4f4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51052/0 (socket says 172.21.15.40:51052) 2023-10-18T08:56:39.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.616+0000 7fbc7a7f3640 1 -- 172.21.15.40:0/2388672175 learned_addr learned my addr 172.21.15.40:0/2388672175 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:39.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.616+0000 7fbc797f1640 1 --2- 172.21.15.40:0/2388672175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4edd50 0x55696f4f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.617+0000 7fbc79ff2640 1 --2- 172.21.15.40:0/2388672175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4ed970 0x55696f4dd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.617+0000 7fbc7a7f3640 1 -- 172.21.15.40:0/2388672175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4edd50 msgr2=0x55696f4f06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:56:39.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.617+0000 7fbc7a7f3640 1 --2- 172.21.15.40:0/2388672175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4edd50 0x55696f4f06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.617+0000 7fbc7a7f3640 1 -- 172.21.15.40:0/2388672175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4ed970 msgr2=0x55696f4dd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:56:39.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.617+0000 7fbc7a7f3640 1 --2- 172.21.15.40:0/2388672175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4ed970 0x55696f4dd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.617+0000 7fbc7a7f3640 1 -- 172.21.15.40:0/2388672175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55696f3c9b80 con 0x55696f4f2c80 2023-10-18T08:56:39.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.617+0000 7fbc7a7f3640 1 --2- 172.21.15.40:0/2388672175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4f2c80 0x55696f4f4180 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbc741f1bd0 tx=0x7fbc6c00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7440796218c1fbd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:39.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.617+0000 7fbc57fff640 1 -- 172.21.15.40:0/2388672175 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c00fe10 con 0x55696f4f2c80 2023-10-18T08:56:39.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.617+0000 7fbc57fff640 1 -- 172.21.15.40:0/2388672175 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbc6c017430 con 0x55696f4f2c80 2023-10-18T08:56:39.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.617+0000 7fbc57fff640 1 -- 172.21.15.40:0/2388672175 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c017840 con 0x55696f4f2c80 2023-10-18T08:56:39.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.618+0000 7fbc7edea740 1 -- 172.21.15.40:0/2388672175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4f2c80 msgr2=0x55696f4f4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:39.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.618+0000 7fbc7edea740 1 --2- 172.21.15.40:0/2388672175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4f2c80 0x55696f4f4180 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbc741f1bd0 tx=0x7fbc6c00dc40 comp rx=0 tx=0).stop 2023-10-18T08:56:39.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.618+0000 7fbc7edea740 1 -- 172.21.15.40:0/2388672175 shutdown_connections 2023-10-18T08:56:39.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.618+0000 7fbc7edea740 1 --2- 172.21.15.40:0/2388672175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4f2c80 0x55696f4f4180 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.618+0000 7fbc7edea740 1 --2- 172.21.15.40:0/2388672175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4edd50 0x55696f4f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.618+0000 7fbc7edea740 1 --2- 172.21.15.40:0/2388672175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4ed970 0x55696f4dd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.618+0000 7fbc7edea740 1 -- 172.21.15.40:0/2388672175 >> 172.21.15.40:0/2388672175 conn(0x55696f4c10e0 msgr2=0x55696f4c6740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.618+0000 7fbc7edea740 1 -- 172.21.15.40:0/2388672175 shutdown_connections 2023-10-18T08:56:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.618+0000 7fbc7edea740 1 -- 172.21.15.40:0/2388672175 wait complete. 2023-10-18T08:56:39.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.619+0000 7fbc7edea740 1 -- start start 2023-10-18T08:56:39.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.619+0000 7fbc7edea740 1 Processor -- start 2023-10-18T08:56:39.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.620+0000 7fbc7edea740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4ed970 0x55696f4dc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.620+0000 7fbc7edea740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f519880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.620+0000 7fbc7edea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4f2c80 0x55696f4d99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.620+0000 7fbc7edea740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55696f4f14e0 con 0x55696f4edd50 2023-10-18T08:56:39.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.620+0000 7fbc7edea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55696f4f1940 con 0x55696f4f2c80 2023-10-18T08:56:39.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.620+0000 7fbc7edea740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55696f4da800 con 0x55696f4ed970 2023-10-18T08:56:39.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.620+0000 7fbc7edea740 10 client.0 _ll_register_callbacks cb 0x7fbc64009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T08:56:39.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.620+0000 7fbc797f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f519880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.620+0000 7fbc797f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f519880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51060/0 (socket says 172.21.15.40:51060) 2023-10-18T08:56:39.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.620+0000 7fbc797f1640 1 -- 172.21.15.40:0/1376553360 learned_addr learned my addr 172.21.15.40:0/1376553360 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:39.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.620+0000 7fbc797f1640 1 -- 172.21.15.40:0/1376553360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4ed970 msgr2=0x55696f4dc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:56:39.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.620+0000 7fbc79ff2640 1 --2- 172.21.15.40:0/1376553360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4ed970 0x55696f4dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.620+0000 7fbc7a7f3640 1 --2- 172.21.15.40:0/1376553360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4f2c80 0x55696f4d99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.620+0000 7fbc797f1640 1 --2- 172.21.15.40:0/1376553360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4ed970 0x55696f4dc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.620+0000 7fbc797f1640 1 -- 172.21.15.40:0/1376553360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4f2c80 msgr2=0x55696f4d99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:56:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.620+0000 7fbc797f1640 1 --2- 172.21.15.40:0/1376553360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4f2c80 0x55696f4d99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.621+0000 7fbc797f1640 1 -- 172.21.15.40:0/1376553360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6c00d7e0 con 0x55696f4edd50 2023-10-18T08:56:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.621+0000 7fbc79ff2640 1 --2- 172.21.15.40:0/1376553360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4ed970 0x55696f4dc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.621+0000 7fbc7a7f3640 1 --2- 172.21.15.40:0/1376553360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4f2c80 0x55696f4d99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.621+0000 7fbc797f1640 1 --2- 172.21.15.40:0/1376553360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f519880 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbc64036040 tx=0x7fbc6402ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b492bb81486ccf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:39.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.621+0000 7fbc54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T08:56:39.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.621+0000 7fbc54ff9640 1 -- 172.21.15.40:0/1376553360 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64040470 con 0x55696f4edd50 2023-10-18T08:56:39.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.621+0000 7fbc54ff9640 1 -- 172.21.15.40:0/1376553360 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc640405d0 con 0x55696f4edd50 2023-10-18T08:56:39.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.621+0000 7fbc7edea740 1 -- 172.21.15.40:0/1376553360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc64044070 con 0x55696f4edd50 2023-10-18T08:56:39.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.621+0000 7fbc54ff9640 1 -- 172.21.15.40:0/1376553360 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc64040790 con 0x55696f4edd50 2023-10-18T08:56:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.621+0000 7fbc7edea740 10 client.5361 fetch_fsmap learned FSMap version 16 2023-10-18T08:56:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.621+0000 7fbc7edea740 1 -- 172.21.15.40:0/1376553360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55696f3ef170 con 0x55696f4edd50 2023-10-18T08:56:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.622+0000 7fbc54ff9640 1 -- 172.21.15.40:0/1376553360 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbc64044070 con 0x55696f4edd50 2023-10-18T08:56:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.622+0000 7fbc54ff9640 1 -- 172.21.15.40:0/1376553360 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbc6403c040 con 0x55696f4edd50 2023-10-18T08:56:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.622+0000 7fbc7edea740 10 client.5361 fetch_fsmap finished waiting for FSMap version 16 2023-10-18T08:56:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.622+0000 7fbc7edea740 10 client.5361 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T08:56:39.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.622+0000 7fbc7edea740 20 client.5361 populate_metadata read hostname 'smithi040' 2023-10-18T08:56:39.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.622+0000 7fbc7edea740 1 --2- 172.21.15.40:0/1376553360 >> [v2:172.21.15.40:6834/3167454356,v1:172.21.15.40:6835/3167454356] conn(0x55696f519dc0 0x55696f4d9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.622+0000 7fbc7edea740 4 client.5361 mds_command: new command op to 5190 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T08:56:39.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.622+0000 7fbc7edea740 1 -- 172.21.15.40:0/1376553360 --> [v2:172.21.15.40:6834/3167454356,v1:172.21.15.40:6835/3167454356] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55696f12e6c0 con 0x55696f519dc0 2023-10-18T08:56:39.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.622+0000 7fbc79ff2640 1 --2- 172.21.15.40:0/1376553360 >> [v2:172.21.15.40:6834/3167454356,v1:172.21.15.40:6835/3167454356] conn(0x55696f519dc0 0x55696f4d9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.623+0000 7fbc54ff9640 10 client.5361 ms_handle_connect on v2:172.21.15.40:6834/3167454356 2023-10-18T08:56:39.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.623+0000 7fbc79ff2640 1 --2- 172.21.15.40:0/1376553360 >> [v2:172.21.15.40:6834/3167454356,v1:172.21.15.40:6835/3167454356] conn(0x55696f519dc0 0x55696f4d9f00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=becb6e6dc05306b2 server_cookie=50383e107805bb27 in_seq=0 out_seq=0 2023-10-18T08:56:39.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.623+0000 7fbc54ff9640 1 -- 172.21.15.40:0/1376553360 <== mds.0 v2:172.21.15.40:6834/3167454356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55696f12e6c0 con 0x55696f519dc0 2023-10-18T08:56:39.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.623+0000 7fbc54ff9640 10 client.5361 handle_command_reply: tid=0 2023-10-18T08:56:39.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.623+0000 7fbc7edea740 1 client.5361 shutdown 2023-10-18T08:56:39.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.623+0000 7fbc7edea740 1 -- 172.21.15.40:0/1376553360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 msgr2=0x55696f519880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.623+0000 7fbc7edea740 1 --2- 172.21.15.40:0/1376553360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f519880 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbc64036040 tx=0x7fbc6402ce40 comp rx=0 tx=0).stop 2023-10-18T08:56:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.623+0000 7fbc7edea740 1 -- 172.21.15.40:0/1376553360 shutdown_connections 2023-10-18T08:56:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.623+0000 7fbc7edea740 1 --2- 172.21.15.40:0/1376553360 >> [v2:172.21.15.40:6834/3167454356,v1:172.21.15.40:6835/3167454356] conn(0x55696f519dc0 0x55696f4d9f00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.623+0000 7fbc7edea740 1 --2- 172.21.15.40:0/1376553360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55696f4f2c80 0x55696f4d99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.623+0000 7fbc7edea740 1 --2- 172.21.15.40:0/1376553360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55696f4edd50 0x55696f519880 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.623+0000 7fbc7edea740 1 --2- 172.21.15.40:0/1376553360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55696f4ed970 0x55696f4dc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.623+0000 7fbc54ff9640 0 client.5361 ms_handle_reset on v2:172.21.15.40:6834/3167454356 2023-10-18T08:56:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.623+0000 7fbc7edea740 1 -- 172.21.15.40:0/1376553360 >> 172.21.15.40:0/1376553360 conn(0x55696f4c10e0 msgr2=0x55696f4bc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.623+0000 7fbc7edea740 1 -- 172.21.15.40:0/1376553360 shutdown_connections 2023-10-18T08:56:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.623+0000 7fbc7edea740 1 -- 172.21.15.40:0/1376553360 wait complete. 2023-10-18T08:56:39.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.624+0000 7fbc7edea740 20 client.5361 trim_cache size 0 max 16384 2023-10-18T08:56:39.639 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T08:56:39.639 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T08:56:39.639 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-10-18T08:56:39.640 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 5190, 2023-10-18T08:56:39.640 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-10-18T08:56:39.640 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-10-18T08:56:39.640 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-10-18T08:56:39.640 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.7427787090000004, 2023-10-18T08:56:39.641 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 16, 2023-10-18T08:56:39.641 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 33, 2023-10-18T08:56:39.641 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 33, 2023-10-18T08:56:39.641 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 15.938661290000001 2023-10-18T08:56:39.641 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T08:56:39.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.624+0000 7fbc7aff4640 1 -- 172.21.15.40:0/2841386376 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc5806ae40 msgr2=0x7fbc5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:39.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.624+0000 7fbc7aff4640 1 --2- 172.21.15.40:0/2841386376 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc5806ae40 0x7fbc5806d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc64035000 tx=0x7fbc64004730 comp rx=0 tx=0).stop 2023-10-18T08:56:39.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.624+0000 7fbc7aff4640 1 -- 172.21.15.40:0/2841386376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc740cdca0 msgr2=0x7fbc7415e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:39.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.624+0000 7fbc7aff4640 1 --2- 172.21.15.40:0/2841386376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc740cdca0 0x7fbc7415e6c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000c870 tx=0x7fbc7000cd30 comp rx=0 tx=0).stop 2023-10-18T08:56:39.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.624+0000 7fbc7aff4640 1 -- 172.21.15.40:0/2841386376 shutdown_connections 2023-10-18T08:56:39.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.624+0000 7fbc7aff4640 1 --2- 172.21.15.40:0/2841386376 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc5806ae40 0x7fbc5806d300 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.624+0000 7fbc7aff4640 1 --2- 172.21.15.40:0/2841386376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc740b3eb0 0x7fbc740d0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.624+0000 7fbc7aff4640 1 --2- 172.21.15.40:0/2841386376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc7415b4a0 0x7fbc740b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.624+0000 7fbc7aff4640 1 --2- 172.21.15.40:0/2841386376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc740cdca0 0x7fbc7415e6c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.624+0000 7fbc7aff4640 1 -- 172.21.15.40:0/2841386376 >> 172.21.15.40:0/2841386376 conn(0x7fbc7408aca0 msgr2=0x7fbc740cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:39.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.624+0000 7fbc7aff4640 1 -- 172.21.15.40:0/2841386376 shutdown_connections 2023-10-18T08:56:39.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.624+0000 7fbc7aff4640 1 -- 172.21.15.40:0/2841386376 wait complete. 2023-10-18T08:56:39.645 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T08:56:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.851+0000 7f4462597640 1 -- 172.21.15.40:0/3964180489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f445c155c60 msgr2=0x7f445c070de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.851+0000 7f4462597640 1 --2- 172.21.15.40:0/3964180489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f445c155c60 0x7f445c070de0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f445c0aa980 tx=0x7f444c02bc70 comp rx=0 tx=0).stop 2023-10-18T08:56:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.851+0000 7f4462597640 1 -- 172.21.15.40:0/3964180489 shutdown_connections 2023-10-18T08:56:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.851+0000 7f4462597640 1 --2- 172.21.15.40:0/3964180489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f445c0af2b0 0x7f445c15f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.851+0000 7f4462597640 1 --2- 172.21.15.40:0/3964180489 >> v2:172.21.15.144:3300/0 conn(0x7f445c071320 0x7f445c0aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.851+0000 7f4462597640 1 --2- 172.21.15.40:0/3964180489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f445c155c60 0x7f445c070de0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.851+0000 7f4462597640 1 -- 172.21.15.40:0/3964180489 >> 172.21.15.40:0/3964180489 conn(0x7f445c069720 msgr2=0x7f445c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.852+0000 7f4462597640 1 -- 172.21.15.40:0/3964180489 shutdown_connections 2023-10-18T08:56:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.852+0000 7f4462597640 1 -- 172.21.15.40:0/3964180489 wait complete. 2023-10-18T08:56:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.854+0000 7f4462597640 1 Processor -- start 2023-10-18T08:56:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.854+0000 7f4462597640 1 -- start start 2023-10-18T08:56:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.855+0000 7f4462597640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f445c071320 0x7f445c15b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.855+0000 7f4462597640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f445c0af2b0 0x7f445c14e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.855+0000 7f4462597640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f445c155c60 0x7f445c14e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.855+0000 7f4462597640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c147650 con 0x7f445c0af2b0 2023-10-18T08:56:39.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.855+0000 7f4462597640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c150eb0 con 0x7f445c155c60 2023-10-18T08:56:39.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.855+0000 7f4462597640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445c14ede0 con 0x7f445c071320 2023-10-18T08:56:39.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.856+0000 7f4453fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f445c0af2b0 0x7f445c14e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.856+0000 7f4453fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f445c0af2b0 0x7f445c14e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51064/0 (socket says 172.21.15.40:51064) 2023-10-18T08:56:39.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.856+0000 7f4453fff640 1 -- 172.21.15.40:0/2879500954 learned_addr learned my addr 172.21.15.40:0/2879500954 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.856+0000 7f4460b34640 1 --2- 172.21.15.40:0/2879500954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f445c155c60 0x7f445c14e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.856+0000 7f445bfff640 1 --2- 172.21.15.40:0/2879500954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f445c071320 0x7f445c15b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.856+0000 7f4453fff640 1 -- 172.21.15.40:0/2879500954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f445c071320 msgr2=0x7f445c15b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.856+0000 7f4453fff640 1 --2- 172.21.15.40:0/2879500954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f445c071320 0x7f445c15b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.856+0000 7f4453fff640 1 -- 172.21.15.40:0/2879500954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f445c155c60 msgr2=0x7f445c14e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.856+0000 7f4453fff640 1 --2- 172.21.15.40:0/2879500954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f445c155c60 0x7f445c14e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.856+0000 7f4453fff640 1 -- 172.21.15.40:0/2879500954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c009560 con 0x7f445c0af2b0 2023-10-18T08:56:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.857+0000 7f4453fff640 1 --2- 172.21.15.40:0/2879500954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f445c0af2b0 0x7f445c14e210 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f444400c950 tx=0x7f444400ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.857+0000 7f44597fa640 1 -- 172.21.15.40:0/2879500954 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444007c70 con 0x7f445c0af2b0 2023-10-18T08:56:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.857+0000 7f44597fa640 1 -- 172.21.15.40:0/2879500954 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f444400f450 con 0x7f445c0af2b0 2023-10-18T08:56:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.857+0000 7f4462597640 1 -- 172.21.15.40:0/2879500954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445c1510e0 con 0x7f445c0af2b0 2023-10-18T08:56:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.857+0000 7f44597fa640 1 -- 172.21.15.40:0/2879500954 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444400f860 con 0x7f445c0af2b0 2023-10-18T08:56:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.857+0000 7f4462597640 1 -- 172.21.15.40:0/2879500954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445c1516a0 con 0x7f445c0af2b0 2023-10-18T08:56:39.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.859+0000 7f4462597640 1 -- 172.21.15.40:0/2879500954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445c066fd0 con 0x7f445c0af2b0 2023-10-18T08:56:39.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.859+0000 7f44597fa640 1 -- 172.21.15.40:0/2879500954 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f444400f9c0 con 0x7f445c0af2b0 2023-10-18T08:56:39.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.861+0000 7f44597fa640 1 --2- 172.21.15.40:0/2879500954 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f443c06ae40 0x7f443c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:39.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.861+0000 7f44597fa640 1 -- 172.21.15.40:0/2879500954 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4444089ee0 con 0x7f445c0af2b0 2023-10-18T08:56:39.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.862+0000 7f445bfff640 1 --2- 172.21.15.40:0/2879500954 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f443c06ae40 0x7f443c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:39.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.863+0000 7f445bfff640 1 --2- 172.21.15.40:0/2879500954 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f443c06ae40 0x7f443c06d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f444c035000 tx=0x7f444c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:39.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:39.866+0000 7f44597fa640 1 -- 172.21.15.40:0/2879500954 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4444054f90 con 0x7f445c0af2b0 2023-10-18T08:56:40.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:40.101+0000 7f4462597640 1 -- 172.21.15.40:0/2879500954 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f445c0ab630 con 0x7f445c0af2b0 2023-10-18T08:56:40.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:40.102+0000 7f44597fa640 1 -- 172.21.15.40:0/2879500954 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f44440585b0 con 0x7f445c0af2b0 2023-10-18T08:56:40.104 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T08:56:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:40.104+0000 7f4462597640 1 -- 172.21.15.40:0/2879500954 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f443c06ae40 msgr2=0x7f443c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:40.104+0000 7f4462597640 1 --2- 172.21.15.40:0/2879500954 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f443c06ae40 0x7f443c06d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f444c035000 tx=0x7f444c004730 comp rx=0 tx=0).stop 2023-10-18T08:56:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:40.104+0000 7f4462597640 1 -- 172.21.15.40:0/2879500954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f445c0af2b0 msgr2=0x7f445c14e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:40.104+0000 7f4462597640 1 --2- 172.21.15.40:0/2879500954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f445c0af2b0 0x7f445c14e210 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f444400c950 tx=0x7f444400ce10 comp rx=0 tx=0).stop 2023-10-18T08:56:40.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:40.105+0000 7f4462597640 1 -- 172.21.15.40:0/2879500954 shutdown_connections 2023-10-18T08:56:40.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:40.105+0000 7f4462597640 1 --2- 172.21.15.40:0/2879500954 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f443c06ae40 0x7f443c06d300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:40.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:40.105+0000 7f4462597640 1 --2- 172.21.15.40:0/2879500954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f445c155c60 0x7f445c14e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:40.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:40.105+0000 7f4462597640 1 --2- 172.21.15.40:0/2879500954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f445c0af2b0 0x7f445c14e210 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:40.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:40.105+0000 7f4462597640 1 --2- 172.21.15.40:0/2879500954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f445c071320 0x7f445c15b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:40.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:40.105+0000 7f4462597640 1 -- 172.21.15.40:0/2879500954 >> 172.21.15.40:0/2879500954 conn(0x7f445c069720 msgr2=0x7f445c15cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:40.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:40.105+0000 7f4462597640 1 -- 172.21.15.40:0/2879500954 shutdown_connections 2023-10-18T08:56:40.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:40.105+0000 7f4462597640 1 -- 172.21.15.40:0/2879500954 wait complete. 2023-10-18T08:56:40.126 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T08:56:40.127 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T08:56:40.127 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-10-18T08:56:40.127 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T08:56:40.128 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T08:56:40.128 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T08:56:40.128 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T08:56:40.128 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T08:56:40.129 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T08:56:40.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:40.129 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-10-18T08:56:40.183 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T08:56:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T08:56:40.183 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T08:56:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:56:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-18T08:56:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:56:40.184 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:56:40.184 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:56:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-10-18T08:56:40.185 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:56:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:56:40.185 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-10-18T08:56:40.185 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T08:56:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-10-18T08:56:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-10-18T08:56:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-18T08:56:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42324sec preferred_lft 42324sec 2023-10-18T08:56:40.186 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2023-10-18T08:56:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:56:40.187 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:56:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-10-18T08:56:40.187 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-10-18T08:56:40.187 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T08:56:40.188 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:56:40.188 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T08:56:40.188 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:56:40.188 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2026:95ff:fea9:a463/64 scope link 2023-10-18T08:56:40.188 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:56:40.189 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T08:56:40.189 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T08:56:40.189 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::80bd:e5ff:fe0f:6f68/64 scope link 2023-10-18T08:56:40.189 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:56:40.190 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T08:56:40.190 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether ea:2b:21:f5:06:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T08:56:40.190 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::e82b:21ff:fef5:6af/64 scope link 2023-10-18T08:56:40.190 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:56:40.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:40.191 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T08:56:40.285 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-10-18T08:56:40.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:40.291 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-10-18T08:56:40.349 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-10-18T08:56:40.349 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T08:56:40.350 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-18T08:56:40.350 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-18T08:56:40.350 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T08:56:40.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:40.351 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T08:56:40.352 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T08:56:40.352 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-18T08:56:40.352 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-18T08:56:40.353 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-18T08:56:40.353 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T08:56:40.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:40.553 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-18T08:56:40.609 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T08:56:40.610 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T08:56:40.611 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T08:56:40.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:40.612 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-10-18T08:56:40.720 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:56:40.720 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T08:56:40.787 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T08:56:40.787 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:56:40.788 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T08:56:40.845 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-18T08:56:40.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:40.845 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:56:41.000 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:56:41.001 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:56:41.032 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:56:41.033 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:56:41.033 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T08:56:41.033 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T08:56:41.033 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T08:56:41.034 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-18T08:56:41.035 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2023-10-18T08:56:41.035 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T08:56:41.035 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T08:56:41.035 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T08:56:41.038 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T08:56:41.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:41.039 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T08:56:41.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:41.140 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:56:41.212 INFO:teuthology.orchestra.run.smithi040.stdout:client5026 2023-10-18T08:56:41.213 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:56:41.213 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:41.273 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T08:56:41.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.502+0000 7f3ad30fc640 1 -- 172.21.15.40:0/2125719971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15b6c0 msgr2=0x7f3acc0b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:41.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.502+0000 7f3ad30fc640 1 --2- 172.21.15.40:0/2125719971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15b6c0 0x7f3acc0b1e40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0b2380 tx=0x7f3ac802be40 comp rx=0 tx=0).stop 2023-10-18T08:56:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.503+0000 7f3ad30fc640 1 -- 172.21.15.40:0/2125719971 shutdown_connections 2023-10-18T08:56:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.503+0000 7f3ad30fc640 1 --2- 172.21.15.40:0/2125719971 >> v2:172.21.15.144:3300/0 conn(0x7f3acc0d1c70 0x7f3acc0cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.503+0000 7f3ad30fc640 1 --2- 172.21.15.40:0/2125719971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15b6c0 0x7f3acc0b1e40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.503+0000 7f3ad30fc640 1 -- 172.21.15.40:0/2125719971 >> 172.21.15.40:0/2125719971 conn(0x7f3acc09a960 msgr2=0x7f3acc088fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:41.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.503+0000 7f3ad30fc640 1 -- 172.21.15.40:0/2125719971 shutdown_connections 2023-10-18T08:56:41.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.503+0000 7f3ad30fc640 1 -- 172.21.15.40:0/2125719971 wait complete. 2023-10-18T08:56:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.504+0000 7f3ad30fc640 1 Processor -- start 2023-10-18T08:56:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.505+0000 7f3ad30fc640 1 -- start start 2023-10-18T08:56:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.505+0000 7f3ad30fc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3acc0cff00 0x7f3acc0b6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:41.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.505+0000 7f3ad30fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3acc0d1c70 0x7f3acc09e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.506+0000 7f3ad30fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15b6c0 0x7f3acc09ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.506+0000 7f3ad30fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc1f2950 con 0x7f3acc15b6c0 2023-10-18T08:56:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.506+0000 7f3ad30fc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0b4780 con 0x7f3acc0d1c70 2023-10-18T08:56:41.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.506+0000 7f3ad30fc640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0b4a60 con 0x7f3acc0cff00 2023-10-18T08:56:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.506+0000 7f3ad1699640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15b6c0 0x7f3acc09ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.506+0000 7f3ad1699640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15b6c0 0x7f3acc09ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51092/0 (socket says 172.21.15.40:51092) 2023-10-18T08:56:41.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.506+0000 7f3ad1699640 1 -- 172.21.15.40:0/2828355628 learned_addr learned my addr 172.21.15.40:0/2828355628 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:41.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.506+0000 7f3ad1699640 1 -- 172.21.15.40:0/2828355628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3acc0cff00 msgr2=0x7f3acc0b6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:41.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.506+0000 7f3ad1699640 1 --2- 172.21.15.40:0/2828355628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3acc0cff00 0x7f3acc0b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:41.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.506+0000 7f3ad1699640 1 -- 172.21.15.40:0/2828355628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3acc0d1c70 msgr2=0x7f3acc09e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:41.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.506+0000 7f3ad1699640 1 --2- 172.21.15.40:0/2828355628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3acc0d1c70 0x7f3acc09e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:41.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.506+0000 7f3ad1699640 1 -- 172.21.15.40:0/2828355628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac8009560 con 0x7f3acc15b6c0 2023-10-18T08:56:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.507+0000 7f3ad1699640 1 --2- 172.21.15.40:0/2828355628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15b6c0 0x7f3acc09ee30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400cab0 tx=0x7f3ac400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.507+0000 7f3ac17fa640 1 -- 172.21.15.40:0/2828355628 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4007e10 con 0x7f3acc15b6c0 2023-10-18T08:56:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.507+0000 7f3ac17fa640 1 -- 172.21.15.40:0/2828355628 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3ac400f450 con 0x7f3acc15b6c0 2023-10-18T08:56:41.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.507+0000 7f3ad30fc640 1 -- 172.21.15.40:0/2828355628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3acc0b4d40 con 0x7f3acc15b6c0 2023-10-18T08:56:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.507+0000 7f3ac17fa640 1 -- 172.21.15.40:0/2828355628 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac400f880 con 0x7f3acc15b6c0 2023-10-18T08:56:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.507+0000 7f3ad30fc640 1 -- 172.21.15.40:0/2828355628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3acc0b5540 con 0x7f3acc15b6c0 2023-10-18T08:56:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.508+0000 7f3ad30fc640 1 -- 172.21.15.40:0/2828355628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3acc0b5260 con 0x7f3acc15b6c0 2023-10-18T08:56:41.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.509+0000 7f3ac17fa640 1 -- 172.21.15.40:0/2828355628 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3ac400f9e0 con 0x7f3acc15b6c0 2023-10-18T08:56:41.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.510+0000 7f3ac17fa640 1 --2- 172.21.15.40:0/2828355628 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ab806ae40 0x7f3ab806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:41.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.510+0000 7f3ac17fa640 1 -- 172.21.15.40:0/2828355628 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ac408a150 con 0x7f3acc15b6c0 2023-10-18T08:56:41.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.511+0000 7f3ad0e98640 1 --2- 172.21.15.40:0/2828355628 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ab806ae40 0x7f3ab806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:41.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.512+0000 7f3ad0e98640 1 --2- 172.21.15.40:0/2828355628 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ab806ae40 0x7f3ab806d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8035000 tx=0x7f3ac80046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:41.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.515+0000 7f3ac17fa640 1 -- 172.21.15.40:0/2828355628 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3acc0b5260 con 0x7f3acc15b6c0 2023-10-18T08:56:41.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.745+0000 7f3ad30fc640 1 -- 172.21.15.40:0/2828355628 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3acc0b5260 con 0x7f3acc15b6c0 2023-10-18T08:56:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.746+0000 7f3ac17fa640 1 -- 172.21.15.40:0/2828355628 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3acc0b5260 con 0x7f3acc15b6c0 2023-10-18T08:56:41.749 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T08:56:41.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.748+0000 7f3ad30fc640 1 -- 172.21.15.40:0/2828355628 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ab806ae40 msgr2=0x7f3ab806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:41.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.748+0000 7f3ad30fc640 1 --2- 172.21.15.40:0/2828355628 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ab806ae40 0x7f3ab806d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8035000 tx=0x7f3ac80046c0 comp rx=0 tx=0).stop 2023-10-18T08:56:41.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.749+0000 7f3ad30fc640 1 -- 172.21.15.40:0/2828355628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15b6c0 msgr2=0x7f3acc09ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:41.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.749+0000 7f3ad30fc640 1 --2- 172.21.15.40:0/2828355628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15b6c0 0x7f3acc09ee30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400cab0 tx=0x7f3ac400cf70 comp rx=0 tx=0).stop 2023-10-18T08:56:41.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.749+0000 7f3ad30fc640 1 -- 172.21.15.40:0/2828355628 shutdown_connections 2023-10-18T08:56:41.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.749+0000 7f3ad30fc640 1 --2- 172.21.15.40:0/2828355628 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ab806ae40 0x7f3ab806d300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:41.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.749+0000 7f3ad30fc640 1 --2- 172.21.15.40:0/2828355628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3acc15b6c0 0x7f3acc09ee30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:41.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.749+0000 7f3ad30fc640 1 --2- 172.21.15.40:0/2828355628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3acc0d1c70 0x7f3acc09e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:41.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.749+0000 7f3ad30fc640 1 --2- 172.21.15.40:0/2828355628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3acc0cff00 0x7f3acc0b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:41.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.749+0000 7f3ad30fc640 1 -- 172.21.15.40:0/2828355628 >> 172.21.15.40:0/2828355628 conn(0x7f3acc09a960 msgr2=0x7f3acc15abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:41.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.749+0000 7f3ad30fc640 1 -- 172.21.15.40:0/2828355628 shutdown_connections 2023-10-18T08:56:41.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:41.749+0000 7f3ad30fc640 1 -- 172.21.15.40:0/2828355628 wait complete. 2023-10-18T08:56:41.772 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T08:56:41.773 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T08:56:41.774 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-10-18T08:56:41.774 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-18T08:56:41.775 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T08:56:41.775 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T08:56:41.776 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T08:56:41.776 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T08:56:41.776 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T08:56:41.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:41.777 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-10-18T08:56:41.839 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T08:56:41.839 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T08:56:41.839 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T08:56:41.840 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:56:41.840 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-18T08:56:41.840 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:56:41.840 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:56:41.841 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:56:41.841 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2023-10-18T08:56:41.841 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:56:41.841 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:56:41.841 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2023-10-18T08:56:41.842 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:56:41.842 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-10-18T08:56:41.842 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T08:56:41.842 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-10-18T08:56:41.843 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T08:56:41.843 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42316sec preferred_lft 42316sec 2023-10-18T08:56:41.843 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2023-10-18T08:56:41.843 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:56:41.844 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T08:56:41.844 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff 2023-10-18T08:56:41.844 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T08:56:41.844 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:56:41.845 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1493:a0ff:fed2:2c1/64 scope link 2023-10-18T08:56:41.845 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:56:41.845 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T08:56:41.845 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 4a:6e:a9:f0:92:62 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T08:56:41.845 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::486e:a9ff:fef0:9262/64 scope link 2023-10-18T08:56:41.846 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:56:41.846 INFO:teuthology.orchestra.run.smithi144.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T08:56:41.846 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T08:56:41.846 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::345a:c5ff:fe54:de6e/64 scope link 2023-10-18T08:56:41.846 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:56:41.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:41.849 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T08:56:41.945 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-10-18T08:56:41.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:41.950 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-10-18T08:56:41.996 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-10-18T08:56:41.997 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T08:56:41.997 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T08:56:41.997 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T08:56:41.997 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T08:56:41.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:41.998 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T08:56:41.999 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T08:56:41.999 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T08:56:41.999 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T08:56:42.000 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T08:56:42.000 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T08:56:42.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:42.200 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-18T08:56:42.244 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T08:56:42.244 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-18T08:56:42.244 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T08:56:42.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:42.245 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2023-10-18T08:56:42.358 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:56:42.359 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-18T08:56:42.398 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-18T08:56:42.398 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:56:42.399 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-18T08:56:42.482 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-18T08:56:42.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:42.483 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:56:42.659 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:56:42.660 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:56:42.693 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:56:42.693 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:56:42.693 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T08:56:42.694 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with new device syntax: 1@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T08:56:42.694 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T08:56:42.695 INFO:teuthology.orchestra.run.smithi144.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-18T08:56:42.695 INFO:teuthology.orchestra.run.smithi144.stdout: You just mounted a file system that supports labels which does not 2023-10-18T08:56:42.695 INFO:teuthology.orchestra.run.smithi144.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T08:56:42.696 INFO:teuthology.orchestra.run.smithi144.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T08:56:42.696 INFO:teuthology.orchestra.run.smithi144.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T08:56:42.699 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T08:56:42.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:42.700 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-18T08:56:42.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:42.779 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:56:42.832 INFO:teuthology.orchestra.run.smithi144.stdout:client5029 2023-10-18T08:56:42.832 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:56:42.833 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:42.880 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:56:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.126+0000 7f8b47fff640 1 -- 172.21.15.40:0/2503778664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b480ca160 msgr2=0x7f8b480cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.126+0000 7f8b47fff640 1 --2- 172.21.15.40:0/2503778664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b480ca160 0x7f8b480cafc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8b40005fd0 tx=0x7f8b4002c240 comp rx=0 tx=0).stop 2023-10-18T08:56:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.126+0000 7f8b47fff640 1 -- 172.21.15.40:0/2503778664 shutdown_connections 2023-10-18T08:56:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.126+0000 7f8b47fff640 1 --2- 172.21.15.40:0/2503778664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b480ca160 0x7f8b480cafc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.126+0000 7f8b47fff640 1 --2- 172.21.15.40:0/2503778664 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b481420f0 0x7f8b480b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.126+0000 7f8b47fff640 1 -- 172.21.15.40:0/2503778664 >> 172.21.15.40:0/2503778664 conn(0x7f8b48069720 msgr2=0x7f8b48069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.126+0000 7f8b47fff640 1 -- 172.21.15.40:0/2503778664 shutdown_connections 2023-10-18T08:56:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.126+0000 7f8b47fff640 1 -- 172.21.15.40:0/2503778664 wait complete. 2023-10-18T08:56:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.128+0000 7f8b47fff640 1 Processor -- start 2023-10-18T08:56:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.128+0000 7f8b47fff640 1 -- start start 2023-10-18T08:56:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.129+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b480ca160 0x7f8b481f8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.129+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b481420f0 0x7f8b480aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.129+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b481425a0 0x7f8b480af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.129+0000 7f8b47fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4815a5c0 con 0x7f8b480ca160 2023-10-18T08:56:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.129+0000 7f8b47fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b481599f0 con 0x7f8b481425a0 2023-10-18T08:56:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.129+0000 7f8b47fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b48159c90 con 0x7f8b481420f0 2023-10-18T08:56:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.129+0000 7f8b46ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b480ca160 0x7f8b481f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.129+0000 7f8b46ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b480ca160 0x7f8b481f8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51114/0 (socket says 172.21.15.40:51114) 2023-10-18T08:56:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.129+0000 7f8b46ffd640 1 -- 172.21.15.40:0/1825262048 learned_addr learned my addr 172.21.15.40:0/1825262048 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.130+0000 7f8b46ffd640 1 -- 172.21.15.40:0/1825262048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b481420f0 msgr2=0x7f8b480aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.130+0000 7f8b46ffd640 1 --2- 172.21.15.40:0/1825262048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b481420f0 0x7f8b480aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.130+0000 7f8b477fe640 1 --2- 172.21.15.40:0/1825262048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b481425a0 0x7f8b480af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.130+0000 7f8b46ffd640 1 -- 172.21.15.40:0/1825262048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b481425a0 msgr2=0x7f8b480af470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.130+0000 7f8b46ffd640 1 --2- 172.21.15.40:0/1825262048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b481425a0 0x7f8b480af470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.130+0000 7f8b46ffd640 1 -- 172.21.15.40:0/1825262048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40009900 con 0x7f8b480ca160 2023-10-18T08:56:43.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.130+0000 7f8b46ffd640 1 --2- 172.21.15.40:0/1825262048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b480ca160 0x7f8b481f8de0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800c970 tx=0x7f8b3800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:43.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.130+0000 7f8b3f7fe640 1 -- 172.21.15.40:0/1825262048 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38007bb0 con 0x7f8b480ca160 2023-10-18T08:56:43.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.131+0000 7f8b3f7fe640 1 -- 172.21.15.40:0/1825262048 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8b38007e20 con 0x7f8b480ca160 2023-10-18T08:56:43.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.131+0000 7f8b3f7fe640 1 -- 172.21.15.40:0/1825262048 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38005280 con 0x7f8b480ca160 2023-10-18T08:56:43.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.131+0000 7f8b47fff640 1 -- 172.21.15.40:0/1825262048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b48159f70 con 0x7f8b480ca160 2023-10-18T08:56:43.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.131+0000 7f8b47fff640 1 -- 172.21.15.40:0/1825262048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b481f9320 con 0x7f8b480ca160 2023-10-18T08:56:43.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.132+0000 7f8b47fff640 1 -- 172.21.15.40:0/1825262048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b48072100 con 0x7f8b480ca160 2023-10-18T08:56:43.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.133+0000 7f8b3f7fe640 1 -- 172.21.15.40:0/1825262048 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8b380053e0 con 0x7f8b480ca160 2023-10-18T08:56:43.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.134+0000 7f8b3f7fe640 1 --2- 172.21.15.40:0/1825262048 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b2806ae40 0x7f8b2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:43.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.134+0000 7f8b3f7fe640 1 -- 172.21.15.40:0/1825262048 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b38089c20 con 0x7f8b480ca160 2023-10-18T08:56:43.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.134+0000 7f8b3ffff640 1 --2- 172.21.15.40:0/1825262048 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b2806ae40 0x7f8b2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:43.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.135+0000 7f8b3ffff640 1 --2- 172.21.15.40:0/1825262048 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b2806ae40 0x7f8b2806d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b481fa720 tx=0x7f8b30006ba0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:43.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.138+0000 7f8b3f7fe640 1 -- 172.21.15.40:0/1825262048 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8b38054cc0 con 0x7f8b480ca160 2023-10-18T08:56:43.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.375+0000 7f8b47fff640 1 -- 172.21.15.40:0/1825262048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b48066e00 con 0x7f8b480ca160 2023-10-18T08:56:43.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.376+0000 7f8b3f7fe640 1 -- 172.21.15.40:0/1825262048 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f8b3801d070 con 0x7f8b480ca160 2023-10-18T08:56:43.380 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:43.380 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/837790640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":837790640},{"type":"v1","addr":"172.21.15.40:6833","nonce":837790640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3377302775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3377302775},{"type":"v1","addr":"172.21.15.144:6836","nonce":3377302775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4939,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3162305748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3162305748},{"type":"v1","addr":"172.21.15.144:6837","nonce":3162305748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:56:32.203221+0000","modified":"2023-10-18T08:56:35.403466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5190":{"gid":5190,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3167454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3167454356},{"type":"v1","addr":"172.21.15.40:6835","nonce":3167454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T08:56:43.380 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2023-10-18T08:56:43.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.379+0000 7f8b47fff640 1 -- 172.21.15.40:0/1825262048 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b2806ae40 msgr2=0x7f8b2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:43.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.379+0000 7f8b47fff640 1 --2- 172.21.15.40:0/1825262048 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b2806ae40 0x7f8b2806d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b481fa720 tx=0x7f8b30006ba0 comp rx=0 tx=0).stop 2023-10-18T08:56:43.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.379+0000 7f8b47fff640 1 -- 172.21.15.40:0/1825262048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b480ca160 msgr2=0x7f8b481f8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:43.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.379+0000 7f8b47fff640 1 --2- 172.21.15.40:0/1825262048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b480ca160 0x7f8b481f8de0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800c970 tx=0x7f8b3800ce30 comp rx=0 tx=0).stop 2023-10-18T08:56:43.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.379+0000 7f8b47fff640 1 -- 172.21.15.40:0/1825262048 shutdown_connections 2023-10-18T08:56:43.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.379+0000 7f8b47fff640 1 --2- 172.21.15.40:0/1825262048 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b2806ae40 0x7f8b2806d300 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:43.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.379+0000 7f8b47fff640 1 --2- 172.21.15.40:0/1825262048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b481425a0 0x7f8b480af470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:43.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.379+0000 7f8b47fff640 1 --2- 172.21.15.40:0/1825262048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b481420f0 0x7f8b480aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:43.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.379+0000 7f8b47fff640 1 --2- 172.21.15.40:0/1825262048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b480ca160 0x7f8b481f8de0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:43.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.379+0000 7f8b47fff640 1 -- 172.21.15.40:0/1825262048 >> 172.21.15.40:0/1825262048 conn(0x7f8b48069720 msgr2=0x7f8b4813ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:43.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.379+0000 7f8b47fff640 1 -- 172.21.15.40:0/1825262048 shutdown_connections 2023-10-18T08:56:43.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.379+0000 7f8b47fff640 1 -- 172.21.15.40:0/1825262048 wait complete. 2023-10-18T08:56:43.400 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-18T08:56:43.641 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":5190,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.74315807,"mdsmap_epoch":16,"osdmap_epoch":33,"osdmap_epoch_barrier":33,"uptime":19.939054482} 2023-10-18T08:56:43.642 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 5190, "mdsmap_epoch": 16, "osdmap_epoch": 33, "osdmap_epoch_barrier": 33, "rank_uptime": 10.74315807, "state": "up:active", "uptime": 19.939054482, "want_state": "up:active", "whoami": 0 } 2023-10-18T08:56:43.642 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2023-10-18T08:56:43.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.866+0000 7f723ebe2640 1 -- 172.21.15.40:0/1751393244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72380cb540 msgr2=0x7f7238159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:43.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.866+0000 7f723ebe2640 1 --2- 172.21.15.40:0/1751393244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72380cb540 0x7f7238159cc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7230005fd0 tx=0x7f723002c0f0 comp rx=0 tx=0).stop 2023-10-18T08:56:43.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.866+0000 7f723ebe2640 1 -- 172.21.15.40:0/1751393244 shutdown_connections 2023-10-18T08:56:43.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.866+0000 7f723ebe2640 1 --2- 172.21.15.40:0/1751393244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72380cb540 0x7f7238159cc0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:43.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.866+0000 7f723ebe2640 1 --2- 172.21.15.40:0/1751393244 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72380af410 0x7f7238159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:43.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.866+0000 7f723ebe2640 1 -- 172.21.15.40:0/1751393244 >> 172.21.15.40:0/1751393244 conn(0x7f7238069720 msgr2=0x7f7238069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:43.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.867+0000 7f723ebe2640 1 -- 172.21.15.40:0/1751393244 shutdown_connections 2023-10-18T08:56:43.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.867+0000 7f723ebe2640 1 -- 172.21.15.40:0/1751393244 wait complete. 2023-10-18T08:56:43.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.868+0000 7f723ebe2640 1 Processor -- start 2023-10-18T08:56:43.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.868+0000 7f723ebe2640 1 -- start start 2023-10-18T08:56:43.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.869+0000 7f723ebe2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72380af410 0x7f72380aa930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:43.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.869+0000 7f723ebe2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72380cb540 0x7f72380abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:43.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.869+0000 7f723ebe2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723813c450 0x7f7238070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:43.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.869+0000 7f723ebe2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380abb90 con 0x7f723813c450 2023-10-18T08:56:43.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.869+0000 7f723ebe2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380aaf10 con 0x7f72380cb540 2023-10-18T08:56:43.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.869+0000 7f723ebe2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72380ab1f0 con 0x7f72380af410 2023-10-18T08:56:43.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.870+0000 7f723e3e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723813c450 0x7f7238070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:43.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.870+0000 7f723e3e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723813c450 0x7f7238070460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51132/0 (socket says 172.21.15.40:51132) 2023-10-18T08:56:43.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.870+0000 7f723e3e1640 1 -- 172.21.15.40:0/335025664 learned_addr learned my addr 172.21.15.40:0/335025664 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:43.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.870+0000 7f723d3df640 1 --2- 172.21.15.40:0/335025664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72380cb540 0x7f72380abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:43.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.870+0000 7f723dbe0640 1 --2- 172.21.15.40:0/335025664 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72380af410 0x7f72380aa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:43.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.870+0000 7f723e3e1640 1 -- 172.21.15.40:0/335025664 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72380af410 msgr2=0x7f72380aa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:43.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.870+0000 7f723e3e1640 1 --2- 172.21.15.40:0/335025664 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72380af410 0x7f72380aa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:43.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.870+0000 7f723e3e1640 1 -- 172.21.15.40:0/335025664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72380cb540 msgr2=0x7f72380abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:43.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.870+0000 7f723e3e1640 1 --2- 172.21.15.40:0/335025664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72380cb540 0x7f72380abfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:43.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.870+0000 7f723e3e1640 1 -- 172.21.15.40:0/335025664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7230009840 con 0x7f723813c450 2023-10-18T08:56:43.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.870+0000 7f723dbe0640 1 --2- 172.21.15.40:0/335025664 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72380af410 0x7f72380aa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:56:43.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.871+0000 7f723e3e1640 1 --2- 172.21.15.40:0/335025664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723813c450 0x7f7238070460 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7230036040 tx=0x7f7230005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:43.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.871+0000 7f722e7fc640 1 -- 172.21.15.40:0/335025664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7230004680 con 0x7f723813c450 2023-10-18T08:56:43.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.871+0000 7f722e7fc640 1 -- 172.21.15.40:0/335025664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f72300345a0 con 0x7f723813c450 2023-10-18T08:56:43.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.871+0000 7f722e7fc640 1 -- 172.21.15.40:0/335025664 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72300349b0 con 0x7f723813c450 2023-10-18T08:56:43.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.871+0000 7f723ebe2640 1 -- 172.21.15.40:0/335025664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72380ab4d0 con 0x7f723813c450 2023-10-18T08:56:43.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.871+0000 7f723ebe2640 1 -- 172.21.15.40:0/335025664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7238157b30 con 0x7f723813c450 2023-10-18T08:56:43.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.872+0000 7f723ebe2640 1 -- 172.21.15.40:0/335025664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72380abcd0 con 0x7f723813c450 2023-10-18T08:56:43.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.873+0000 7f722e7fc640 1 -- 172.21.15.40:0/335025664 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7230034b10 con 0x7f723813c450 2023-10-18T08:56:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.874+0000 7f722e7fc640 1 --2- 172.21.15.40:0/335025664 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f721c06ae40 0x7f721c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.874+0000 7f722e7fc640 1 -- 172.21.15.40:0/335025664 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72300ac8b0 con 0x7f723813c450 2023-10-18T08:56:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.875+0000 7f723dbe0640 1 --2- 172.21.15.40:0/335025664 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f721c06ae40 0x7f721c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:43.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.876+0000 7f723dbe0640 1 --2- 172.21.15.40:0/335025664 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f721c06ae40 0x7f721c06d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7234001660 tx=0x7f7234009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:43.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:43.878+0000 7f722e7fc640 1 -- 172.21.15.40:0/335025664 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7230077950 con 0x7f723813c450 2023-10-18T08:56:44.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.142+0000 7f723ebe2640 1 -- 172.21.15.40:0/335025664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f723806c4a0 con 0x7f723813c450 2023-10-18T08:56:44.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.143+0000 7f722e7fc640 1 -- 172.21.15.40:0/335025664 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1620 (secure 0 0 0) 0x7f7230043090 con 0x7f723813c450 2023-10-18T08:56:44.146 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:44.147 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T08:56:44.147 INFO:teuthology.orchestra.run.smithi040.stdout: "stats": { 2023-10-18T08:56:44.147 INFO:teuthology.orchestra.run.smithi040.stdout: "total_bytes": 1600000000, 2023-10-18T08:56:44.147 INFO:teuthology.orchestra.run.smithi040.stdout: "total_avail_bytes": 1597643357, 2023-10-18T08:56:44.147 INFO:teuthology.orchestra.run.smithi040.stdout: "total_used_bytes": 2356643, 2023-10-18T08:56:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: "total_used_raw_bytes": 2356643, 2023-10-18T08:56:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: "total_used_raw_ratio": 0.0014729019021615386, 2023-10-18T08:56:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: "num_osds": 8, 2023-10-18T08:56:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: "num_per_pool_osds": 0, 2023-10-18T08:56:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: "num_per_pool_omap_osds": 0 2023-10-18T08:56:44.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:44.149 INFO:teuthology.orchestra.run.smithi040.stdout: "stats_by_class": { 2023-10-18T08:56:44.149 INFO:teuthology.orchestra.run.smithi040.stdout: "hdd": { 2023-10-18T08:56:44.149 INFO:teuthology.orchestra.run.smithi040.stdout: "total_bytes": 1600000000, 2023-10-18T08:56:44.149 INFO:teuthology.orchestra.run.smithi040.stdout: "total_avail_bytes": 1597643357, 2023-10-18T08:56:44.149 INFO:teuthology.orchestra.run.smithi040.stdout: "total_used_bytes": 2356643, 2023-10-18T08:56:44.150 INFO:teuthology.orchestra.run.smithi040.stdout: "total_used_raw_bytes": 2356643, 2023-10-18T08:56:44.150 INFO:teuthology.orchestra.run.smithi040.stdout: "total_used_raw_ratio": 0.0014729019021615386 2023-10-18T08:56:44.150 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:44.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:44.150 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T08:56:44.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:44.151 INFO:teuthology.orchestra.run.smithi040.stdout: "name": ".mgr", 2023-10-18T08:56:44.151 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 1, 2023-10-18T08:56:44.151 INFO:teuthology.orchestra.run.smithi040.stdout: "stats": { 2023-10-18T08:56:44.151 INFO:teuthology.orchestra.run.smithi040.stdout: "stored": 590368, 2023-10-18T08:56:44.151 INFO:teuthology.orchestra.run.smithi040.stdout: "objects": 2, 2023-10-18T08:56:44.151 INFO:teuthology.orchestra.run.smithi040.stdout: "kb_used": 577, 2023-10-18T08:56:44.152 INFO:teuthology.orchestra.run.smithi040.stdout: "bytes_used": 590368, 2023-10-18T08:56:44.152 INFO:teuthology.orchestra.run.smithi040.stdout: "percent_used": 0.00052962568588554859, 2023-10-18T08:56:44.152 INFO:teuthology.orchestra.run.smithi040.stdout: "max_avail": 557049408 2023-10-18T08:56:44.152 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:44.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:44.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:44.153 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cephfs_metadata", 2023-10-18T08:56:44.153 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4, 2023-10-18T08:56:44.153 INFO:teuthology.orchestra.run.smithi040.stdout: "stats": { 2023-10-18T08:56:44.153 INFO:teuthology.orchestra.run.smithi040.stdout: "stored": 2410, 2023-10-18T08:56:44.153 INFO:teuthology.orchestra.run.smithi040.stdout: "objects": 22, 2023-10-18T08:56:44.153 INFO:teuthology.orchestra.run.smithi040.stdout: "kb_used": 3, 2023-10-18T08:56:44.154 INFO:teuthology.orchestra.run.smithi040.stdout: "bytes_used": 2410, 2023-10-18T08:56:44.154 INFO:teuthology.orchestra.run.smithi040.stdout: "percent_used": 2.1631785784848034e-06, 2023-10-18T08:56:44.154 INFO:teuthology.orchestra.run.smithi040.stdout: "max_avail": 557049408 2023-10-18T08:56:44.154 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:44.154 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:56:44.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:56:44.155 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "cephfs_data", 2023-10-18T08:56:44.155 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 5, 2023-10-18T08:56:44.155 INFO:teuthology.orchestra.run.smithi040.stdout: "stats": { 2023-10-18T08:56:44.155 INFO:teuthology.orchestra.run.smithi040.stdout: "stored": 0, 2023-10-18T08:56:44.155 INFO:teuthology.orchestra.run.smithi040.stdout: "objects": 0, 2023-10-18T08:56:44.156 INFO:teuthology.orchestra.run.smithi040.stdout: "kb_used": 0, 2023-10-18T08:56:44.156 INFO:teuthology.orchestra.run.smithi040.stdout: "bytes_used": 0, 2023-10-18T08:56:44.156 INFO:teuthology.orchestra.run.smithi040.stdout: "percent_used": 0, 2023-10-18T08:56:44.156 INFO:teuthology.orchestra.run.smithi040.stdout: "max_avail": 557049408 2023-10-18T08:56:44.156 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:44.156 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:56:44.157 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:56:44.157 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T08:56:44.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.146+0000 7f723ebe2640 1 -- 172.21.15.40:0/335025664 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f721c06ae40 msgr2=0x7f721c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:44.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.146+0000 7f723ebe2640 1 --2- 172.21.15.40:0/335025664 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f721c06ae40 0x7f721c06d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7234001660 tx=0x7f7234009040 comp rx=0 tx=0).stop 2023-10-18T08:56:44.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.146+0000 7f723ebe2640 1 -- 172.21.15.40:0/335025664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723813c450 msgr2=0x7f7238070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:44.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.146+0000 7f723ebe2640 1 --2- 172.21.15.40:0/335025664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723813c450 0x7f7238070460 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7230036040 tx=0x7f7230005c80 comp rx=0 tx=0).stop 2023-10-18T08:56:44.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.146+0000 7f723ebe2640 1 -- 172.21.15.40:0/335025664 shutdown_connections 2023-10-18T08:56:44.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.146+0000 7f723ebe2640 1 --2- 172.21.15.40:0/335025664 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f721c06ae40 0x7f721c06d300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:44.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.146+0000 7f723ebe2640 1 --2- 172.21.15.40:0/335025664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723813c450 0x7f7238070460 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:44.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.146+0000 7f723ebe2640 1 --2- 172.21.15.40:0/335025664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72380cb540 0x7f72380abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:44.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.146+0000 7f723ebe2640 1 --2- 172.21.15.40:0/335025664 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72380af410 0x7f72380aa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:44.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.146+0000 7f723ebe2640 1 -- 172.21.15.40:0/335025664 >> 172.21.15.40:0/335025664 conn(0x7f7238069720 msgr2=0x7f72381559a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:44.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.146+0000 7f723ebe2640 1 -- 172.21.15.40:0/335025664 shutdown_connections 2023-10-18T08:56:44.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.147+0000 7f723ebe2640 1 -- 172.21.15.40:0/335025664 wait complete. 2023-10-18T08:56:44.168 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2023-10-18T08:56:44.168 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-10-18T08:56:44.168 DEBUG:teuthology.orchestra.run.smithi040:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-18T08:56:44.169 DEBUG:teuthology.orchestra.run.smithi040:> os.fchmod(fd, 0O666) 2023-10-18T08:56:44.169 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(fd) 2023-10-18T08:56:44.169 DEBUG:teuthology.orchestra.run.smithi040:> os.close(fd) 2023-10-18T08:56:44.170 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-10-18T08:56:44.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T08:56:44.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:44.344 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:56:44.450 INFO:teuthology.orchestra.run.smithi040.stdout:client5026 2023-10-18T08:56:44.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:44.451 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/029855e3-538a-4478-b323-f54ab0f3a56d.client5026/osdmap') 2023-10-18T08:56:44.542 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 33 barrier 33 flags 0x588000 2023-10-18T08:56:44.543 INFO:teuthology.orchestra.run.smithi040.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T08:56:44.543 INFO:teuthology.orchestra.run.smithi040.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T08:56:44.543 INFO:teuthology.orchestra.run.smithi040.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T08:56:44.543 INFO:teuthology.orchestra.run.smithi040.stdout:osd0 (2)172.21.15.40:6800 100% (exists, up) 100% -1 2023-10-18T08:56:44.543 INFO:teuthology.orchestra.run.smithi040.stdout:osd1 (2)172.21.15.40:6808 100% (exists, up) 100% -1 2023-10-18T08:56:44.543 INFO:teuthology.orchestra.run.smithi040.stdout:osd2 (2)172.21.15.40:6823 100% (exists, up) 100% -1 2023-10-18T08:56:44.544 INFO:teuthology.orchestra.run.smithi040.stdout:osd3 (2)172.21.15.40:6816 100% (exists, up) 100% -1 2023-10-18T08:56:44.544 INFO:teuthology.orchestra.run.smithi040.stdout:osd4 (2)172.21.15.144:6810 100% (exists, up) 100% -1 2023-10-18T08:56:44.544 INFO:teuthology.orchestra.run.smithi040.stdout:osd5 (2)172.21.15.144:6802 100% (exists, up) 100% -1 2023-10-18T08:56:44.544 INFO:teuthology.orchestra.run.smithi040.stdout:osd6 (2)172.21.15.144:6818 100% (exists, up) 100% -1 2023-10-18T08:56:44.544 INFO:teuthology.orchestra.run.smithi040.stdout:osd7 (2)172.21.15.144:6824 100% (exists, up) 100% -1 2023-10-18T08:56:44.545 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records in 2023-10-18T08:56:44.545 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records out 2023-10-18T08:56:44.545 INFO:teuthology.orchestra.run.smithi040.stderr:795 bytes copied, 0.000140358 s, 5.7 MB/s 2023-10-18T08:56:44.546 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2023-10-18T08:56:44.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.764+0000 7fe3b3b2c640 1 -- 172.21.15.40:0/2179809996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ac15eaa0 msgr2=0x7fe3ac0ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:44.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.764+0000 7fe3b3b2c640 1 --2- 172.21.15.40:0/2179809996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ac15eaa0 0x7fe3ac0ccdb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009860 tx=0x7fe3a802bd50 comp rx=0 tx=0).stop 2023-10-18T08:56:44.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.764+0000 7fe3b3b2c640 1 -- 172.21.15.40:0/2179809996 shutdown_connections 2023-10-18T08:56:44.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.764+0000 7fe3b3b2c640 1 --2- 172.21.15.40:0/2179809996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3ac0b1b90 0x7fe3ac0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:44.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.764+0000 7fe3b3b2c640 1 --2- 172.21.15.40:0/2179809996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ac15eaa0 0x7fe3ac0ccdb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:44.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.764+0000 7fe3b3b2c640 1 -- 172.21.15.40:0/2179809996 >> 172.21.15.40:0/2179809996 conn(0x7fe3ac06cee0 msgr2=0x7fe3ac069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:44.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.764+0000 7fe3b3b2c640 1 -- 172.21.15.40:0/2179809996 shutdown_connections 2023-10-18T08:56:44.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.764+0000 7fe3b3b2c640 1 -- 172.21.15.40:0/2179809996 wait complete. 2023-10-18T08:56:44.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.766+0000 7fe3b3b2c640 1 Processor -- start 2023-10-18T08:56:44.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.766+0000 7fe3b3b2c640 1 -- start start 2023-10-18T08:56:44.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.767+0000 7fe3b3b2c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3ac06ffe0 0x7fe3ac141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:44.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.767+0000 7fe3b3b2c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ac0b1b90 0x7fe3ac13ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:44.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.767+0000 7fe3b3b2c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3ac15eaa0 0x7fe3ac13ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:44.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.767+0000 7fe3b3b2c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac152750 con 0x7fe3ac15eaa0 2023-10-18T08:56:44.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.767+0000 7fe3b3b2c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac13f4b0 con 0x7fe3ac0b1b90 2023-10-18T08:56:44.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.767+0000 7fe3b3b2c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac152470 con 0x7fe3ac06ffe0 2023-10-18T08:56:44.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.768+0000 7fe3b20c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3ac15eaa0 0x7fe3ac13ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:44.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.768+0000 7fe3b20c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3ac15eaa0 0x7fe3ac13ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51142/0 (socket says 172.21.15.40:51142) 2023-10-18T08:56:44.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.768+0000 7fe3b20c9640 1 -- 172.21.15.40:0/3807616468 learned_addr learned my addr 172.21.15.40:0/3807616468 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:44.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.768+0000 7fe3b10c7640 1 --2- 172.21.15.40:0/3807616468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ac0b1b90 0x7fe3ac13ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:44.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.768+0000 7fe3b18c8640 1 --2- 172.21.15.40:0/3807616468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3ac06ffe0 0x7fe3ac141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:44.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.768+0000 7fe3b20c9640 1 -- 172.21.15.40:0/3807616468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3ac06ffe0 msgr2=0x7fe3ac141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:44.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.768+0000 7fe3b20c9640 1 --2- 172.21.15.40:0/3807616468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3ac06ffe0 0x7fe3ac141a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:44.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.768+0000 7fe3b20c9640 1 -- 172.21.15.40:0/3807616468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ac0b1b90 msgr2=0x7fe3ac13ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:44.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.768+0000 7fe3b20c9640 1 --2- 172.21.15.40:0/3807616468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ac0b1b90 0x7fe3ac13ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:44.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.768+0000 7fe3b20c9640 1 -- 172.21.15.40:0/3807616468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a8009490 con 0x7fe3ac15eaa0 2023-10-18T08:56:44.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.769+0000 7fe3b20c9640 1 --2- 172.21.15.40:0/3807616468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3ac15eaa0 0x7fe3ac13ef70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400c7d0 tx=0x7fe3a400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:44.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.769+0000 7fe3a27fc640 1 -- 172.21.15.40:0/3807616468 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4003b70 con 0x7fe3ac15eaa0 2023-10-18T08:56:44.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.769+0000 7fe3a27fc640 1 -- 172.21.15.40:0/3807616468 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe3a4003cf0 con 0x7fe3ac15eaa0 2023-10-18T08:56:44.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.769+0000 7fe3b3b2c640 1 -- 172.21.15.40:0/3807616468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3ac14f4d0 con 0x7fe3ac15eaa0 2023-10-18T08:56:44.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.769+0000 7fe3a27fc640 1 -- 172.21.15.40:0/3807616468 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4003ea0 con 0x7fe3ac15eaa0 2023-10-18T08:56:44.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.769+0000 7fe3b3b2c640 1 -- 172.21.15.40:0/3807616468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ac14fa90 con 0x7fe3ac15eaa0 2023-10-18T08:56:44.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.770+0000 7fe3b3b2c640 1 -- 172.21.15.40:0/3807616468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3ac0706f0 con 0x7fe3ac15eaa0 2023-10-18T08:56:44.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.771+0000 7fe3a27fc640 1 -- 172.21.15.40:0/3807616468 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe3a400aab0 con 0x7fe3ac15eaa0 2023-10-18T08:56:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.773+0000 7fe3a27fc640 1 --2- 172.21.15.40:0/3807616468 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe39006ae70 0x7fe39006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.773+0000 7fe3a27fc640 1 -- 172.21.15.40:0/3807616468 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3a40895c0 con 0x7fe3ac15eaa0 2023-10-18T08:56:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.773+0000 7fe3b18c8640 1 --2- 172.21.15.40:0/3807616468 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe39006ae70 0x7fe39006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:44.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.775+0000 7fe3b18c8640 1 --2- 172.21.15.40:0/3807616468 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe39006ae70 0x7fe39006d330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8036000 tx=0x7fe3a8004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:44.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.777+0000 7fe3a27fc640 1 -- 172.21.15.40:0/3807616468 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3a40545d0 con 0x7fe3ac15eaa0 2023-10-18T08:56:44.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:44.980+0000 7fe3b3b2c640 1 -- 172.21.15.40:0/3807616468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7fe3ac06c360 con 0x7fe3ac15eaa0 2023-10-18T08:56:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:45.966+0000 7fe3a27fc640 1 -- 172.21.15.40:0/3807616468 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v34) v1 ==== 93+0+0 (secure 0 0 0) 0x7fe3a4057bf0 con 0x7fe3ac15eaa0 2023-10-18T08:56:45.969 INFO:teuthology.orchestra.run.smithi040.stderr:pauserd,pausewr is set 2023-10-18T08:56:45.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:45.969+0000 7fe3b3b2c640 1 -- 172.21.15.40:0/3807616468 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe39006ae70 msgr2=0x7fe39006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:45.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:45.969+0000 7fe3b3b2c640 1 --2- 172.21.15.40:0/3807616468 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe39006ae70 0x7fe39006d330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8036000 tx=0x7fe3a8004730 comp rx=0 tx=0).stop 2023-10-18T08:56:45.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:45.969+0000 7fe3b3b2c640 1 -- 172.21.15.40:0/3807616468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3ac15eaa0 msgr2=0x7fe3ac13ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:45.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:45.969+0000 7fe3b3b2c640 1 --2- 172.21.15.40:0/3807616468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3ac15eaa0 0x7fe3ac13ef70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400c7d0 tx=0x7fe3a400cbd0 comp rx=0 tx=0).stop 2023-10-18T08:56:45.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:45.970+0000 7fe3b3b2c640 1 -- 172.21.15.40:0/3807616468 shutdown_connections 2023-10-18T08:56:45.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:45.970+0000 7fe3b3b2c640 1 --2- 172.21.15.40:0/3807616468 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe39006ae70 0x7fe39006d330 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:45.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:45.970+0000 7fe3b3b2c640 1 --2- 172.21.15.40:0/3807616468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3ac15eaa0 0x7fe3ac13ef70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:45.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:45.970+0000 7fe3b3b2c640 1 --2- 172.21.15.40:0/3807616468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ac0b1b90 0x7fe3ac13ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:45.970+0000 7fe3b3b2c640 1 --2- 172.21.15.40:0/3807616468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3ac06ffe0 0x7fe3ac141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:45.970+0000 7fe3b3b2c640 1 -- 172.21.15.40:0/3807616468 >> 172.21.15.40:0/3807616468 conn(0x7fe3ac06cee0 msgr2=0x7fe3ac0aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:45.970+0000 7fe3b3b2c640 1 -- 172.21.15.40:0/3807616468 shutdown_connections 2023-10-18T08:56:45.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:45.970+0000 7fe3b3b2c640 1 -- 172.21.15.40:0/3807616468 wait complete. 2023-10-18T08:56:45.995 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2023-10-18T08:56:46.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.213+0000 7f86ce602640 1 -- 172.21.15.40:0/409022655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80c9f90 msgr2=0x7f86c806b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:46.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.213+0000 7f86ce602640 1 --2- 172.21.15.40:0/409022655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80c9f90 0x7f86c806b4d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f86c806af60 tx=0x7f86c002c240 comp rx=0 tx=0).stop 2023-10-18T08:56:46.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.213+0000 7f86ce602640 1 -- 172.21.15.40:0/409022655 shutdown_connections 2023-10-18T08:56:46.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.213+0000 7f86ce602640 1 --2- 172.21.15.40:0/409022655 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80c9f90 0x7f86c806b4d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:46.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.213+0000 7f86ce602640 1 --2- 172.21.15.40:0/409022655 >> v2:172.21.15.144:3300/0 conn(0x7f86c81425a0 0x7f86c80c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:46.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.213+0000 7f86ce602640 1 --2- 172.21.15.40:0/409022655 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86c81420f0 0x7f86c80b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:46.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.213+0000 7f86ce602640 1 -- 172.21.15.40:0/409022655 >> 172.21.15.40:0/409022655 conn(0x7f86c8069720 msgr2=0x7f86c8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:46.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.213+0000 7f86ce602640 1 -- 172.21.15.40:0/409022655 shutdown_connections 2023-10-18T08:56:46.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.213+0000 7f86ce602640 1 -- 172.21.15.40:0/409022655 wait complete. 2023-10-18T08:56:46.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.215+0000 7f86ce602640 1 Processor -- start 2023-10-18T08:56:46.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.215+0000 7f86ce602640 1 -- start start 2023-10-18T08:56:46.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.216+0000 7f86ce602640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86c80c9f90 0x7f86c8158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:46.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.216+0000 7f86ce602640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86c81420f0 0x7f86c8140860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:46.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.216+0000 7f86ce602640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c81425a0 0x7f86c8140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:46.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.216+0000 7f86ce602640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c80abda0 con 0x7f86c81425a0 2023-10-18T08:56:46.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.216+0000 7f86ce602640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c8157f90 con 0x7f86c80c9f90 2023-10-18T08:56:46.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.216+0000 7f86ce602640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c80712b0 con 0x7f86c81420f0 2023-10-18T08:56:46.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.216+0000 7f86cde01640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c81425a0 0x7f86c8140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:46.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.216+0000 7f86cde01640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c81425a0 0x7f86c8140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41690/0 (socket says 172.21.15.40:41690) 2023-10-18T08:56:46.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.216+0000 7f86cde01640 1 -- 172.21.15.40:0/988464610 learned_addr learned my addr 172.21.15.40:0/988464610 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:46.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.217+0000 7f86cde01640 1 -- 172.21.15.40:0/988464610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86c81420f0 msgr2=0x7f86c8140860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:46.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.217+0000 7f86cde01640 1 --2- 172.21.15.40:0/988464610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86c81420f0 0x7f86c8140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:46.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.217+0000 7f86cde01640 1 -- 172.21.15.40:0/988464610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86c80c9f90 msgr2=0x7f86c8158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:46.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.217+0000 7f86cd600640 1 --2- 172.21.15.40:0/988464610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86c80c9f90 0x7f86c8158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:46.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.217+0000 7f86cde01640 1 --2- 172.21.15.40:0/988464610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86c80c9f90 0x7f86c8158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:46.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.217+0000 7f86cde01640 1 -- 172.21.15.40:0/988464610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c0009900 con 0x7f86c81425a0 2023-10-18T08:56:46.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.217+0000 7f86cd600640 1 --2- 172.21.15.40:0/988464610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86c80c9f90 0x7f86c8158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:46.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.217+0000 7f86cde01640 1 --2- 172.21.15.40:0/988464610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c81425a0 0x7f86c8140da0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f86c0037040 tx=0x7f86c0035a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:46.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.217+0000 7f86bdffb640 1 -- 172.21.15.40:0/988464610 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0034440 con 0x7f86c81425a0 2023-10-18T08:56:46.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.218+0000 7f86bdffb640 1 -- 172.21.15.40:0/988464610 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f86c0004330 con 0x7f86c81425a0 2023-10-18T08:56:46.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.218+0000 7f86bdffb640 1 -- 172.21.15.40:0/988464610 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c00046c0 con 0x7f86c81425a0 2023-10-18T08:56:46.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.218+0000 7f86ce602640 1 -- 172.21.15.40:0/988464610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c8158230 con 0x7f86c81425a0 2023-10-18T08:56:46.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.218+0000 7f86ce602640 1 -- 172.21.15.40:0/988464610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c8071590 con 0x7f86c81425a0 2023-10-18T08:56:46.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.219+0000 7f86ce602640 1 -- 172.21.15.40:0/988464610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c8158750 con 0x7f86c81425a0 2023-10-18T08:56:46.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.220+0000 7f86bdffb640 1 -- 172.21.15.40:0/988464610 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f86c003b040 con 0x7f86c81425a0 2023-10-18T08:56:46.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.221+0000 7f86bdffb640 1 --2- 172.21.15.40:0/988464610 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f86ac06ae40 0x7f86ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:46.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.221+0000 7f86bdffb640 1 -- 172.21.15.40:0/988464610 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86c00ac340 con 0x7f86c81425a0 2023-10-18T08:56:46.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.221+0000 7f86bdffb640 1 -- 172.21.15.40:0/988464610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=35+}) v3 -- 0x7f86ac072970 con 0x7f86c81425a0 2023-10-18T08:56:46.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.222+0000 7f86cd600640 1 --2- 172.21.15.40:0/988464610 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f86ac06ae40 0x7f86ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:46.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.223+0000 7f86cd600640 1 --2- 172.21.15.40:0/988464610 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f86ac06ae40 0x7f86ac06d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f86c400baa0 tx=0x7f86c4005d20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:46.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.226+0000 7f86bdffb640 1 -- 172.21.15.40:0/988464610 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f86c8158750 con 0x7f86c81425a0 2023-10-18T08:56:46.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.429+0000 7f86ce602640 1 -- 172.21.15.40:0/988464610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7f86c8158750 con 0x7f86c81425a0 2023-10-18T08:56:46.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.970+0000 7f86bdffb640 1 -- 172.21.15.40:0/988464610 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v35) v1 ==== 97+0+0 (secure 0 0 0) 0x7f86c8158750 con 0x7f86c81425a0 2023-10-18T08:56:46.974 INFO:teuthology.orchestra.run.smithi040.stderr:pauserd,pausewr is unset 2023-10-18T08:56:46.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.970+0000 7f86bdffb640 1 -- 172.21.15.40:0/988464610 <== mon.0 v2:172.21.15.40:3300/0 8 ==== osd_map(35..35 src has 1..35) v4 ==== 288+0+0 (secure 0 0 0) 0x7f86c0035e40 con 0x7f86c81425a0 2023-10-18T08:56:46.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.971+0000 7f86bdffb640 1 -- 172.21.15.40:0/988464610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=36}) v3 -- 0x7f86ac073370 con 0x7f86c81425a0 2023-10-18T08:56:46.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.974+0000 7f86ce602640 1 -- 172.21.15.40:0/988464610 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f86ac06ae40 msgr2=0x7f86ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:46.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.974+0000 7f86ce602640 1 --2- 172.21.15.40:0/988464610 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f86ac06ae40 0x7f86ac06d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f86c400baa0 tx=0x7f86c4005d20 comp rx=0 tx=0).stop 2023-10-18T08:56:46.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.974+0000 7f86ce602640 1 -- 172.21.15.40:0/988464610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c81425a0 msgr2=0x7f86c8140da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:46.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.974+0000 7f86ce602640 1 --2- 172.21.15.40:0/988464610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c81425a0 0x7f86c8140da0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f86c0037040 tx=0x7f86c0035a40 comp rx=0 tx=0).stop 2023-10-18T08:56:46.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.974+0000 7f86ce602640 1 -- 172.21.15.40:0/988464610 shutdown_connections 2023-10-18T08:56:46.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.974+0000 7f86ce602640 1 --2- 172.21.15.40:0/988464610 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f86ac06ae40 0x7f86ac06d300 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f86c400baa0 tx=0x7f86c4005d20 comp rx=0 tx=0).stop 2023-10-18T08:56:46.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.974+0000 7f86ce602640 1 --2- 172.21.15.40:0/988464610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c81425a0 0x7f86c8140da0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:46.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.974+0000 7f86ce602640 1 --2- 172.21.15.40:0/988464610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86c81420f0 0x7f86c8140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:46.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.974+0000 7f86ce602640 1 --2- 172.21.15.40:0/988464610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86c80c9f90 0x7f86c8158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:46.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.974+0000 7f86ce602640 1 -- 172.21.15.40:0/988464610 >> 172.21.15.40:0/988464610 conn(0x7f86c8069720 msgr2=0x7f86c813b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:46.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.975+0000 7f86ce602640 1 -- 172.21.15.40:0/988464610 shutdown_connections 2023-10-18T08:56:46.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:46.975+0000 7f86ce602640 1 -- 172.21.15.40:0/988464610 wait complete. 2023-10-18T08:56:46.999 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:47.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.212+0000 7fd471a32640 1 -- 172.21.15.40:0/2016284864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd46c15cf30 msgr2=0x7fd46c15d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:47.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.212+0000 7fd471a32640 1 --2- 172.21.15.40:0/2016284864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd46c15cf30 0x7fd46c15d390 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd454009830 tx=0x7fd45402bd60 comp rx=0 tx=0).stop 2023-10-18T08:56:47.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.213+0000 7fd471a32640 1 -- 172.21.15.40:0/2016284864 shutdown_connections 2023-10-18T08:56:47.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.213+0000 7fd471a32640 1 --2- 172.21.15.40:0/2016284864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd46c15cf30 0x7fd46c15d390 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:47.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.213+0000 7fd471a32640 1 --2- 172.21.15.40:0/2016284864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd46c13c470 0x7fd46c13f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:47.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.213+0000 7fd471a32640 1 -- 172.21.15.40:0/2016284864 >> 172.21.15.40:0/2016284864 conn(0x7fd46c06cee0 msgr2=0x7fd46c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:47.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.213+0000 7fd471a32640 1 -- 172.21.15.40:0/2016284864 shutdown_connections 2023-10-18T08:56:47.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.213+0000 7fd471a32640 1 -- 172.21.15.40:0/2016284864 wait complete. 2023-10-18T08:56:47.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.214+0000 7fd471a32640 1 Processor -- start 2023-10-18T08:56:47.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.215+0000 7fd471a32640 1 -- start start 2023-10-18T08:56:47.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.215+0000 7fd471a32640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd46c0c9b00 0x7fd46c13b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:47.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.215+0000 7fd471a32640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd46c13c470 0x7fd46c13ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:47.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.216+0000 7fd471a32640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd46c15cf30 0x7fd46c0701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:47.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.216+0000 7fd471a32640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c156e60 con 0x7fd46c13c470 2023-10-18T08:56:47.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.216+0000 7fd471a32640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c13b660 con 0x7fd46c15cf30 2023-10-18T08:56:47.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.216+0000 7fd471a32640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c1542a0 con 0x7fd46c0c9b00 2023-10-18T08:56:47.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.216+0000 7fd46a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd46c13c470 0x7fd46c13ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:47.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.216+0000 7fd46a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd46c13c470 0x7fd46c13ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41708/0 (socket says 172.21.15.40:41708) 2023-10-18T08:56:47.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.216+0000 7fd46a7fc640 1 -- 172.21.15.40:0/998576306 learned_addr learned my addr 172.21.15.40:0/998576306 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:47.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.216+0000 7fd46affd640 1 --2- 172.21.15.40:0/998576306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd46c0c9b00 0x7fd46c13b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:47.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.216+0000 7fd46a7fc640 1 -- 172.21.15.40:0/998576306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd46c0c9b00 msgr2=0x7fd46c13b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:47.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.216+0000 7fd46a7fc640 1 --2- 172.21.15.40:0/998576306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd46c0c9b00 0x7fd46c13b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:47.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.216+0000 7fd46a7fc640 1 -- 172.21.15.40:0/998576306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd46c15cf30 msgr2=0x7fd46c0701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:47.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.216+0000 7fd46a7fc640 1 --2- 172.21.15.40:0/998576306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd46c15cf30 0x7fd46c0701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:47.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.216+0000 7fd46a7fc640 1 -- 172.21.15.40:0/998576306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd454009490 con 0x7fd46c13c470 2023-10-18T08:56:47.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.216+0000 7fd46affd640 1 --2- 172.21.15.40:0/998576306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd46c0c9b00 0x7fd46c13b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:47.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.217+0000 7fd46a7fc640 1 --2- 172.21.15.40:0/998576306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd46c13c470 0x7fd46c13ff00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd454009780 tx=0x7fd4540038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:47.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.217+0000 7fd463fff640 1 -- 172.21.15.40:0/998576306 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd454003ea0 con 0x7fd46c13c470 2023-10-18T08:56:47.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.217+0000 7fd463fff640 1 -- 172.21.15.40:0/998576306 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd454034550 con 0x7fd46c13c470 2023-10-18T08:56:47.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.217+0000 7fd463fff640 1 -- 172.21.15.40:0/998576306 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd454034960 con 0x7fd46c13c470 2023-10-18T08:56:47.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.217+0000 7fd471a32640 1 -- 172.21.15.40:0/998576306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46c0ca4a0 con 0x7fd46c13c470 2023-10-18T08:56:47.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.217+0000 7fd471a32640 1 -- 172.21.15.40:0/998576306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46c154b40 con 0x7fd46c13c470 2023-10-18T08:56:47.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.218+0000 7fd471a32640 1 -- 172.21.15.40:0/998576306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46c066fd0 con 0x7fd46c13c470 2023-10-18T08:56:47.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.219+0000 7fd463fff640 1 -- 172.21.15.40:0/998576306 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd45403b040 con 0x7fd46c13c470 2023-10-18T08:56:47.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.220+0000 7fd463fff640 1 --2- 172.21.15.40:0/998576306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd44c06ae70 0x7fd44c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:47.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.220+0000 7fd463fff640 1 -- 172.21.15.40:0/998576306 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4540abd90 con 0x7fd46c13c470 2023-10-18T08:56:47.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.221+0000 7fd46affd640 1 --2- 172.21.15.40:0/998576306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd44c06ae70 0x7fd44c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:47.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.222+0000 7fd46affd640 1 --2- 172.21.15.40:0/998576306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd44c06ae70 0x7fd44c06d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00baa0 tx=0x7fd45c005d90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:47.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.224+0000 7fd463fff640 1 -- 172.21.15.40:0/998576306 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd454076d10 con 0x7fd46c13c470 2023-10-18T08:56:47.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.418+0000 7fd471a32640 1 -- 172.21.15.40:0/998576306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd46c0ac4d0 con 0x7fd46c13c470 2023-10-18T08:56:47.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.421+0000 7fd463fff640 1 -- 172.21.15.40:0/998576306 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16581 (secure 0 0 0) 0x7fd454039020 con 0x7fd46c13c470 2023-10-18T08:56:47.425 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:47.425 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":35,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:46.967356+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:56:30.127184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-18T08:56:31.225270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:47.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.424+0000 7fd471a32640 1 -- 172.21.15.40:0/998576306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd44c06ae70 msgr2=0x7fd44c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:47.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.424+0000 7fd471a32640 1 --2- 172.21.15.40:0/998576306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd44c06ae70 0x7fd44c06d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00baa0 tx=0x7fd45c005d90 comp rx=0 tx=0).stop 2023-10-18T08:56:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.424+0000 7fd471a32640 1 -- 172.21.15.40:0/998576306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd46c13c470 msgr2=0x7fd46c13ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.424+0000 7fd471a32640 1 --2- 172.21.15.40:0/998576306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd46c13c470 0x7fd46c13ff00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd454009780 tx=0x7fd4540038a0 comp rx=0 tx=0).stop 2023-10-18T08:56:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.424+0000 7fd471a32640 1 -- 172.21.15.40:0/998576306 shutdown_connections 2023-10-18T08:56:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.424+0000 7fd471a32640 1 --2- 172.21.15.40:0/998576306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd44c06ae70 0x7fd44c06d330 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.424+0000 7fd471a32640 1 --2- 172.21.15.40:0/998576306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd46c15cf30 0x7fd46c0701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:47.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.424+0000 7fd471a32640 1 --2- 172.21.15.40:0/998576306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd46c13c470 0x7fd46c13ff00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:47.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.424+0000 7fd471a32640 1 --2- 172.21.15.40:0/998576306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd46c0c9b00 0x7fd46c13b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:47.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.424+0000 7fd471a32640 1 -- 172.21.15.40:0/998576306 >> 172.21.15.40:0/998576306 conn(0x7fd46c06cee0 msgr2=0x7fd46c13c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:47.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.424+0000 7fd471a32640 1 -- 172.21.15.40:0/998576306 shutdown_connections 2023-10-18T08:56:47.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47.425+0000 7fd471a32640 1 -- 172.21.15.40:0/998576306 wait complete. 2023-10-18T08:56:47.447 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2023-10-18T08:56:47.448 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-10-18T08:56:47.448 DEBUG:teuthology.orchestra.run.smithi040:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-18T08:56:47.448 DEBUG:teuthology.orchestra.run.smithi040:> os.fchmod(fd, 0O666) 2023-10-18T08:56:47.449 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(fd) 2023-10-18T08:56:47.449 DEBUG:teuthology.orchestra.run.smithi040:> os.close(fd) 2023-10-18T08:56:47.449 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-10-18T08:56:47.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T08:56:47.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:47.618 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:56:47.726 INFO:teuthology.orchestra.run.smithi040.stdout:client5026 2023-10-18T08:56:47.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:47.727 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/029855e3-538a-4478-b323-f54ab0f3a56d.client5026/osdmap') 2023-10-18T08:56:47.835 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 33 barrier 33 flags 0x588000 2023-10-18T08:56:47.835 INFO:teuthology.orchestra.run.smithi040.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T08:56:47.835 INFO:teuthology.orchestra.run.smithi040.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T08:56:47.836 INFO:teuthology.orchestra.run.smithi040.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T08:56:47.836 INFO:teuthology.orchestra.run.smithi040.stdout:osd0 (2)172.21.15.40:6800 100% (exists, up) 100% -1 2023-10-18T08:56:47.836 INFO:teuthology.orchestra.run.smithi040.stdout:osd1 (2)172.21.15.40:6808 100% (exists, up) 100% -1 2023-10-18T08:56:47.836 INFO:teuthology.orchestra.run.smithi040.stdout:osd2 (2)172.21.15.40:6823 100% (exists, up) 100% -1 2023-10-18T08:56:47.836 INFO:teuthology.orchestra.run.smithi040.stdout:osd3 (2)172.21.15.40:6816 100% (exists, up) 100% -1 2023-10-18T08:56:47.836 INFO:teuthology.orchestra.run.smithi040.stdout:osd4 (2)172.21.15.144:6810 100% (exists, up) 100% -1 2023-10-18T08:56:47.837 INFO:teuthology.orchestra.run.smithi040.stdout:osd5 (2)172.21.15.144:6802 100% (exists, up) 100% -1 2023-10-18T08:56:47.837 INFO:teuthology.orchestra.run.smithi040.stdout:osd6 (2)172.21.15.144:6818 100% (exists, up) 100% -1 2023-10-18T08:56:47.837 INFO:teuthology.orchestra.run.smithi040.stdout:osd7 (2)172.21.15.144:6824 100% (exists, up) 100% -1 2023-10-18T08:56:47.837 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records in 2023-10-18T08:56:47.838 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records out 2023-10-18T08:56:47.838 INFO:teuthology.orchestra.run.smithi040.stderr:795 bytes copied, 0.000106897 s, 7.4 MB/s 2023-10-18T08:56:47.839 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:56:48.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.068+0000 7f9f8e516640 1 -- 172.21.15.40:0/1940515549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f88159920 msgr2=0x7f9f8815dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:48.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.068+0000 7f9f8e516640 1 --2- 172.21.15.40:0/1940515549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f88159920 0x7f9f8815dd30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009830 tx=0x7f9f7c02bcc0 comp rx=0 tx=0).stop 2023-10-18T08:56:48.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.068+0000 7f9f8e516640 1 -- 172.21.15.40:0/1940515549 shutdown_connections 2023-10-18T08:56:48.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.068+0000 7f9f8e516640 1 --2- 172.21.15.40:0/1940515549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f880b4fc0 0x7f9f880ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:48.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.068+0000 7f9f8e516640 1 --2- 172.21.15.40:0/1940515549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f88159920 0x7f9f8815dd30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:48.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.068+0000 7f9f8e516640 1 -- 172.21.15.40:0/1940515549 >> 172.21.15.40:0/1940515549 conn(0x7f9f880cf3e0 msgr2=0x7f9f880cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:48.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.068+0000 7f9f8e516640 1 -- 172.21.15.40:0/1940515549 shutdown_connections 2023-10-18T08:56:48.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.068+0000 7f9f8e516640 1 -- 172.21.15.40:0/1940515549 wait complete. 2023-10-18T08:56:48.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.070+0000 7f9f8e516640 1 Processor -- start 2023-10-18T08:56:48.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.070+0000 7f9f8e516640 1 -- start start 2023-10-18T08:56:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.071+0000 7f9f8e516640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f880b4fc0 0x7f9f881587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.071+0000 7f9f8e516640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f880c99a0 0x7f9f880c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.071+0000 7f9f8e516640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f88159920 0x7f9f880c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.071+0000 7f9f8e516640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8815b930 con 0x7f9f88159920 2023-10-18T08:56:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.071+0000 7f9f8e516640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8815c790 con 0x7f9f880b4fc0 2023-10-18T08:56:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.071+0000 7f9f8e516640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8815bb80 con 0x7f9f880c99a0 2023-10-18T08:56:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.071+0000 7f9f8cab3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f88159920 0x7f9f880c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.071+0000 7f9f8cab3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f88159920 0x7f9f880c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41718/0 (socket says 172.21.15.40:41718) 2023-10-18T08:56:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.071+0000 7f9f8cab3640 1 -- 172.21.15.40:0/3339935378 learned_addr learned my addr 172.21.15.40:0/3339935378 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.072+0000 7f9f877fe640 1 --2- 172.21.15.40:0/3339935378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f880c99a0 0x7f9f880c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.072+0000 7f9f87fff640 1 --2- 172.21.15.40:0/3339935378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f880b4fc0 0x7f9f881587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.072+0000 7f9f8cab3640 1 -- 172.21.15.40:0/3339935378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f880c99a0 msgr2=0x7f9f880c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:48.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.072+0000 7f9f8cab3640 1 --2- 172.21.15.40:0/3339935378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f880c99a0 0x7f9f880c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:48.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.072+0000 7f9f8cab3640 1 -- 172.21.15.40:0/3339935378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f880b4fc0 msgr2=0x7f9f881587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:48.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.072+0000 7f9f8cab3640 1 --2- 172.21.15.40:0/3339935378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f880b4fc0 0x7f9f881587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:48.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.072+0000 7f9f8cab3640 1 -- 172.21.15.40:0/3339935378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f80009490 con 0x7f9f88159920 2023-10-18T08:56:48.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.072+0000 7f9f87fff640 1 --2- 172.21.15.40:0/3339935378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f880b4fc0 0x7f9f881587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.072+0000 7f9f877fe640 1 --2- 172.21.15.40:0/3339935378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f880c99a0 0x7f9f880c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.072+0000 7f9f8cab3640 1 --2- 172.21.15.40:0/3339935378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f88159920 0x7f9f880c76b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000e9e0 tx=0x7f9f8000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.072+0000 7f9f84ff9640 1 -- 172.21.15.40:0/3339935378 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8000cdb0 con 0x7f9f88159920 2023-10-18T08:56:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.072+0000 7f9f84ff9640 1 -- 172.21.15.40:0/3339935378 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9f80004020 con 0x7f9f88159920 2023-10-18T08:56:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.072+0000 7f9f84ff9640 1 -- 172.21.15.40:0/3339935378 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f80004410 con 0x7f9f88159920 2023-10-18T08:56:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.072+0000 7f9f8e516640 1 -- 172.21.15.40:0/3339935378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f7c009490 con 0x7f9f88159920 2023-10-18T08:56:48.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.073+0000 7f9f8e516640 1 -- 172.21.15.40:0/3339935378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8815c140 con 0x7f9f88159920 2023-10-18T08:56:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.074+0000 7f9f8e516640 1 -- 172.21.15.40:0/3339935378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f88066d40 con 0x7f9f88159920 2023-10-18T08:56:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.075+0000 7f9f84ff9640 1 -- 172.21.15.40:0/3339935378 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9f800035f0 con 0x7f9f88159920 2023-10-18T08:56:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.076+0000 7f9f84ff9640 1 --2- 172.21.15.40:0/3339935378 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9f7406ae40 0x7f9f7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.076+0000 7f9f84ff9640 1 -- 172.21.15.40:0/3339935378 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f80014070 con 0x7f9f88159920 2023-10-18T08:56:48.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.076+0000 7f9f87fff640 1 --2- 172.21.15.40:0/3339935378 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9f7406ae40 0x7f9f7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:48.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.077+0000 7f9f87fff640 1 --2- 172.21.15.40:0/3339935378 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9f7406ae40 0x7f9f7406d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f780021a0 tx=0x7f9f78009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:48.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.080+0000 7f9f84ff9640 1 -- 172.21.15.40:0/3339935378 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9f80055280 con 0x7f9f88159920 2023-10-18T08:56:48.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.310+0000 7f9f8e516640 1 -- 172.21.15.40:0/3339935378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f880caa60 con 0x7f9f88159920 2023-10-18T08:56:48.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.311+0000 7f9f84ff9640 1 -- 172.21.15.40:0/3339935378 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f9f800588a0 con 0x7f9f88159920 2023-10-18T08:56:48.315 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:48.315 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/837790640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":837790640},{"type":"v1","addr":"172.21.15.40:6833","nonce":837790640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3377302775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3377302775},{"type":"v1","addr":"172.21.15.144:6836","nonce":3377302775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4939,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3162305748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3162305748},{"type":"v1","addr":"172.21.15.144:6837","nonce":3162305748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:56:32.203221+0000","modified":"2023-10-18T08:56:35.403466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5190":{"gid":5190,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3167454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3167454356},{"type":"v1","addr":"172.21.15.40:6835","nonce":3167454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T08:56:48.316 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2023-10-18T08:56:48.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.314+0000 7f9f8e516640 1 -- 172.21.15.40:0/3339935378 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9f7406ae40 msgr2=0x7f9f7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:48.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.314+0000 7f9f8e516640 1 --2- 172.21.15.40:0/3339935378 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9f7406ae40 0x7f9f7406d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f780021a0 tx=0x7f9f78009040 comp rx=0 tx=0).stop 2023-10-18T08:56:48.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.314+0000 7f9f8e516640 1 -- 172.21.15.40:0/3339935378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f88159920 msgr2=0x7f9f880c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:48.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.314+0000 7f9f8e516640 1 --2- 172.21.15.40:0/3339935378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f88159920 0x7f9f880c76b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9f8000e9e0 tx=0x7f9f8000eea0 comp rx=0 tx=0).stop 2023-10-18T08:56:48.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.315+0000 7f9f8e516640 1 -- 172.21.15.40:0/3339935378 shutdown_connections 2023-10-18T08:56:48.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.315+0000 7f9f8e516640 1 --2- 172.21.15.40:0/3339935378 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9f7406ae40 0x7f9f7406d300 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:48.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.315+0000 7f9f8e516640 1 --2- 172.21.15.40:0/3339935378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f88159920 0x7f9f880c76b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:48.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.315+0000 7f9f8e516640 1 --2- 172.21.15.40:0/3339935378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f880c99a0 0x7f9f880c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:48.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.315+0000 7f9f8e516640 1 --2- 172.21.15.40:0/3339935378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f880b4fc0 0x7f9f881587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:48.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.315+0000 7f9f8e516640 1 -- 172.21.15.40:0/3339935378 >> 172.21.15.40:0/3339935378 conn(0x7f9f880cf3e0 msgr2=0x7f9f880b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:48.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.315+0000 7f9f8e516640 1 -- 172.21.15.40:0/3339935378 shutdown_connections 2023-10-18T08:56:48.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48.315+0000 7f9f8e516640 1 -- 172.21.15.40:0/3339935378 wait complete. 2023-10-18T08:56:48.336 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json osdmap barrier 35 2023-10-18T08:56:48.580 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-18T08:56:48.581 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2023-10-18T08:56:48.581 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-10-18T08:56:48.582 DEBUG:teuthology.orchestra.run.smithi040:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-18T08:56:48.582 DEBUG:teuthology.orchestra.run.smithi040:> os.fchmod(fd, 0O666) 2023-10-18T08:56:48.583 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(fd) 2023-10-18T08:56:48.583 DEBUG:teuthology.orchestra.run.smithi040:> os.close(fd) 2023-10-18T08:56:48.583 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-10-18T08:56:48.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T08:56:48.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:48.751 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:56:48.858 INFO:teuthology.orchestra.run.smithi040.stdout:client5026 2023-10-18T08:56:48.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:48.859 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/029855e3-538a-4478-b323-f54ab0f3a56d.client5026/osdmap') 2023-10-18T08:56:48.959 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 35 barrier 35 flags 0x588000 2023-10-18T08:56:48.959 INFO:teuthology.orchestra.run.smithi040.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T08:56:48.959 INFO:teuthology.orchestra.run.smithi040.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T08:56:48.960 INFO:teuthology.orchestra.run.smithi040.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T08:56:48.960 INFO:teuthology.orchestra.run.smithi040.stdout:osd0 (2)172.21.15.40:6800 100% (exists, up) 100% -1 2023-10-18T08:56:48.960 INFO:teuthology.orchestra.run.smithi040.stdout:osd1 (2)172.21.15.40:6808 100% (exists, up) 100% -1 2023-10-18T08:56:48.960 INFO:teuthology.orchestra.run.smithi040.stdout:osd2 (2)172.21.15.40:6823 100% (exists, up) 100% -1 2023-10-18T08:56:48.960 INFO:teuthology.orchestra.run.smithi040.stdout:osd3 (2)172.21.15.40:6816 100% (exists, up) 100% -1 2023-10-18T08:56:48.961 INFO:teuthology.orchestra.run.smithi040.stdout:osd4 (2)172.21.15.144:6810 100% (exists, up) 100% -1 2023-10-18T08:56:48.961 INFO:teuthology.orchestra.run.smithi040.stdout:osd5 (2)172.21.15.144:6802 100% (exists, up) 100% -1 2023-10-18T08:56:48.961 INFO:teuthology.orchestra.run.smithi040.stdout:osd6 (2)172.21.15.144:6818 100% (exists, up) 100% -1 2023-10-18T08:56:48.961 INFO:teuthology.orchestra.run.smithi040.stdout:osd7 (2)172.21.15.144:6824 100% (exists, up) 100% -1 2023-10-18T08:56:48.961 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records in 2023-10-18T08:56:48.962 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records out 2023-10-18T08:56:48.962 INFO:teuthology.orchestra.run.smithi040.stderr:795 bytes copied, 0.000102873 s, 7.7 MB/s 2023-10-18T08:56:48.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:48.963 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:56:49.024 INFO:teuthology.orchestra.run.smithi040.stdout:client5026 2023-10-18T08:56:49.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:49.025 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/029855e3-538a-4478-b323-f54ab0f3a56d.client5026/osdmap') 2023-10-18T08:56:49.124 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 35 barrier 35 flags 0x588000 2023-10-18T08:56:49.125 INFO:teuthology.orchestra.run.smithi040.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T08:56:49.125 INFO:teuthology.orchestra.run.smithi040.stdout:pool 4 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T08:56:49.125 INFO:teuthology.orchestra.run.smithi040.stdout:pool 5 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T08:56:49.125 INFO:teuthology.orchestra.run.smithi040.stdout:osd0 (2)172.21.15.40:6800 100% (exists, up) 100% -1 2023-10-18T08:56:49.125 INFO:teuthology.orchestra.run.smithi040.stdout:osd1 (2)172.21.15.40:6808 100% (exists, up) 100% -1 2023-10-18T08:56:49.126 INFO:teuthology.orchestra.run.smithi040.stdout:osd2 (2)172.21.15.40:6823 100% (exists, up) 100% -1 2023-10-18T08:56:49.126 INFO:teuthology.orchestra.run.smithi040.stdout:osd3 (2)172.21.15.40:6816 100% (exists, up) 100% -1 2023-10-18T08:56:49.126 INFO:teuthology.orchestra.run.smithi040.stdout:osd4 (2)172.21.15.144:6810 100% (exists, up) 100% -1 2023-10-18T08:56:49.126 INFO:teuthology.orchestra.run.smithi040.stdout:osd5 (2)172.21.15.144:6802 100% (exists, up) 100% -1 2023-10-18T08:56:49.126 INFO:teuthology.orchestra.run.smithi040.stdout:osd6 (2)172.21.15.144:6818 100% (exists, up) 100% -1 2023-10-18T08:56:49.127 INFO:teuthology.orchestra.run.smithi040.stdout:osd7 (2)172.21.15.144:6824 100% (exists, up) 100% -1 2023-10-18T08:56:49.127 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records in 2023-10-18T08:56:49.127 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records out 2023-10-18T08:56:49.127 INFO:teuthology.orchestra.run.smithi040.stderr:795 bytes copied, 0.000129762 s, 6.1 MB/s 2023-10-18T08:56:49.129 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-18T08:56:49.129 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T08:56:49.130 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T08:56:49.259 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T08:56:49.386 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T08:56:49.386 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:56:49.387 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:49.449 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:56:49.449 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:49.507 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:49.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.736+0000 7feafd0bb640 1 -- 172.21.15.40:0/3784548741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf806b8c0 msgr2=0x7feaf80ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:49.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.736+0000 7feafd0bb640 1 --2- 172.21.15.40:0/3784548741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf806b8c0 0x7feaf80ca4b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009830 tx=0x7feaf002bdf0 comp rx=0 tx=0).stop 2023-10-18T08:56:49.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.737+0000 7feafd0bb640 1 -- 172.21.15.40:0/3784548741 shutdown_connections 2023-10-18T08:56:49.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.737+0000 7feafd0bb640 1 --2- 172.21.15.40:0/3784548741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf806b8c0 0x7feaf80ca4b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:49.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.737+0000 7feafd0bb640 1 -- 172.21.15.40:0/3784548741 >> 172.21.15.40:0/3784548741 conn(0x7feaf80b0af0 msgr2=0x7feaf80b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:49.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.737+0000 7feafd0bb640 1 -- 172.21.15.40:0/3784548741 shutdown_connections 2023-10-18T08:56:49.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.737+0000 7feafd0bb640 1 -- 172.21.15.40:0/3784548741 wait complete. 2023-10-18T08:56:49.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.739+0000 7feafd0bb640 1 Processor -- start 2023-10-18T08:56:49.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.739+0000 7feafd0bb640 1 -- start start 2023-10-18T08:56:49.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.741+0000 7feafd0bb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feaf806ac60 0x7feaf81fa140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:49.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.741+0000 7feafd0bb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaf806b8c0 0x7feaf80c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:49.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.741+0000 7feafd0bb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf80b4a00 0x7feaf80c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:49.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.741+0000 7feafd0bb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf80c6610 con 0x7feaf80b4a00 2023-10-18T08:56:49.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.741+0000 7feafd0bb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf80c6870 con 0x7feaf806b8c0 2023-10-18T08:56:49.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.741+0000 7feafd0bb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf80c6b50 con 0x7feaf806ac60 2023-10-18T08:56:49.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.741+0000 7feafc8ba640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf80b4a00 0x7feaf80c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:49.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.741+0000 7feafc8ba640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf80b4a00 0x7feaf80c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41724/0 (socket says 172.21.15.40:41724) 2023-10-18T08:56:49.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.741+0000 7feafc8ba640 1 -- 172.21.15.40:0/2732630904 learned_addr learned my addr 172.21.15.40:0/2732630904 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:49.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.741+0000 7feaf77fe640 1 --2- 172.21.15.40:0/2732630904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaf806b8c0 0x7feaf80c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:49.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.741+0000 7feaf7fff640 1 --2- 172.21.15.40:0/2732630904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feaf806ac60 0x7feaf81fa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:49.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.742+0000 7feafc8ba640 1 -- 172.21.15.40:0/2732630904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feaf806ac60 msgr2=0x7feaf81fa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:49.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.742+0000 7feafc8ba640 1 --2- 172.21.15.40:0/2732630904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feaf806ac60 0x7feaf81fa140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:49.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.742+0000 7feafc8ba640 1 -- 172.21.15.40:0/2732630904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaf806b8c0 msgr2=0x7feaf80c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:49.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.742+0000 7feafc8ba640 1 --2- 172.21.15.40:0/2732630904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaf806b8c0 0x7feaf80c60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:49.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.742+0000 7feafc8ba640 1 -- 172.21.15.40:0/2732630904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf0009490 con 0x7feaf80b4a00 2023-10-18T08:56:49.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.742+0000 7feafc8ba640 1 --2- 172.21.15.40:0/2732630904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf80b4a00 0x7feaf80c7550 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009780 tx=0x7feaf00038f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:49.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.742+0000 7feaf4ff9640 1 -- 172.21.15.40:0/2732630904 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf0034440 con 0x7feaf80b4a00 2023-10-18T08:56:49.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.742+0000 7feaf4ff9640 1 -- 172.21.15.40:0/2732630904 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feaf0034690 con 0x7feaf80b4a00 2023-10-18T08:56:49.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.742+0000 7feaf4ff9640 1 -- 172.21.15.40:0/2732630904 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf0034aa0 con 0x7feaf80b4a00 2023-10-18T08:56:49.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.742+0000 7feafd0bb640 1 -- 172.21.15.40:0/2732630904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf80c6e30 con 0x7feaf80b4a00 2023-10-18T08:56:49.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.743+0000 7feafd0bb640 1 -- 172.21.15.40:0/2732630904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf8153a30 con 0x7feaf80b4a00 2023-10-18T08:56:49.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.744+0000 7feafd0bb640 1 -- 172.21.15.40:0/2732630904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf8066e00 con 0x7feaf80b4a00 2023-10-18T08:56:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.745+0000 7feaf4ff9640 1 -- 172.21.15.40:0/2732630904 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feaf003b040 con 0x7feaf80b4a00 2023-10-18T08:56:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.746+0000 7feaf4ff9640 1 --2- 172.21.15.40:0/2732630904 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feae406ae40 0x7feae406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.746+0000 7feaf4ff9640 1 -- 172.21.15.40:0/2732630904 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feaf00abce0 con 0x7feaf80b4a00 2023-10-18T08:56:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.746+0000 7feaf7fff640 1 --2- 172.21.15.40:0/2732630904 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feae406ae40 0x7feae406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:49.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.747+0000 7feaf7fff640 1 --2- 172.21.15.40:0/2732630904 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feae406ae40 0x7feae406d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feae8009f20 tx=0x7feae80094d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:49.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.750+0000 7feaf4ff9640 1 -- 172.21.15.40:0/2732630904 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feaf0076cf0 con 0x7feaf80b4a00 2023-10-18T08:56:49.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.945+0000 7feafd0bb640 1 -- 172.21.15.40:0/2732630904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feaf815e050 con 0x7feaf80b4a00 2023-10-18T08:56:49.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.947+0000 7feaf4ff9640 1 -- 172.21.15.40:0/2732630904 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16581 (secure 0 0 0) 0x7feaf0039090 con 0x7feaf80b4a00 2023-10-18T08:56:49.951 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:49.951 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":35,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:46.967356+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:56:30.127184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-18T08:56:31.225270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:49.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.950+0000 7feafd0bb640 1 -- 172.21.15.40:0/2732630904 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feae406ae40 msgr2=0x7feae406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:49.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.950+0000 7feafd0bb640 1 --2- 172.21.15.40:0/2732630904 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feae406ae40 0x7feae406d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feae8009f20 tx=0x7feae80094d0 comp rx=0 tx=0).stop 2023-10-18T08:56:49.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.950+0000 7feafd0bb640 1 -- 172.21.15.40:0/2732630904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf80b4a00 msgr2=0x7feaf80c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:49.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.950+0000 7feafd0bb640 1 --2- 172.21.15.40:0/2732630904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf80b4a00 0x7feaf80c7550 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009780 tx=0x7feaf00038f0 comp rx=0 tx=0).stop 2023-10-18T08:56:49.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.951+0000 7feafd0bb640 1 -- 172.21.15.40:0/2732630904 shutdown_connections 2023-10-18T08:56:49.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.951+0000 7feafd0bb640 1 --2- 172.21.15.40:0/2732630904 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feae406ae40 0x7feae406d300 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:49.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.951+0000 7feafd0bb640 1 --2- 172.21.15.40:0/2732630904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf80b4a00 0x7feaf80c7550 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:49.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.951+0000 7feafd0bb640 1 --2- 172.21.15.40:0/2732630904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaf806b8c0 0x7feaf80c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:49.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.951+0000 7feafd0bb640 1 --2- 172.21.15.40:0/2732630904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feaf806ac60 0x7feaf81fa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:49.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.951+0000 7feafd0bb640 1 -- 172.21.15.40:0/2732630904 >> 172.21.15.40:0/2732630904 conn(0x7feaf80b0af0 msgr2=0x7feaf81555b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:49.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.951+0000 7feafd0bb640 1 -- 172.21.15.40:0/2732630904 shutdown_connections 2023-10-18T08:56:49.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:49.951+0000 7feafd0bb640 1 -- 172.21.15.40:0/2732630904 wait complete. 2023-10-18T08:56:49.973 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:56:30.127184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:56:31.225270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:56:49.973 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:56:50.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.203+0000 7f959359e640 1 -- 172.21.15.40:0/3611937168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95941411b0 msgr2=0x7f95941416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:50.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.203+0000 7f959359e640 1 --2- 172.21.15.40:0/3611937168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95941411b0 0x7f95941416a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9584005fd0 tx=0x7f958402be20 comp rx=0 tx=0).stop 2023-10-18T08:56:50.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.204+0000 7f959359e640 1 -- 172.21.15.40:0/3611937168 shutdown_connections 2023-10-18T08:56:50.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.204+0000 7f959359e640 1 --2- 172.21.15.40:0/3611937168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9594070960 0x7f9594141be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:50.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.204+0000 7f959359e640 1 --2- 172.21.15.40:0/3611937168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95941411b0 0x7f95941416a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:50.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.204+0000 7f959359e640 1 -- 172.21.15.40:0/3611937168 >> 172.21.15.40:0/3611937168 conn(0x7f9594069720 msgr2=0x7f9594069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:50.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.204+0000 7f959359e640 1 -- 172.21.15.40:0/3611937168 shutdown_connections 2023-10-18T08:56:50.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.204+0000 7f959359e640 1 -- 172.21.15.40:0/3611937168 wait complete. 2023-10-18T08:56:50.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.206+0000 7f959359e640 1 Processor -- start 2023-10-18T08:56:50.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.206+0000 7f959359e640 1 -- start start 2023-10-18T08:56:50.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.207+0000 7f959359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9594070960 0x7f95940adb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:50.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.207+0000 7f959359e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95940b0a50 0x7f95941f2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:50.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.207+0000 7f959359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95941411b0 0x7f959415a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:50.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.207+0000 7f959359e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95940aefc0 con 0x7f95941411b0 2023-10-18T08:56:50.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.207+0000 7f959359e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95941566c0 con 0x7f9594070960 2023-10-18T08:56:50.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.207+0000 7f959359e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95940ae070 con 0x7f95940b0a50 2023-10-18T08:56:50.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.207+0000 7f9592d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95941411b0 0x7f959415a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:50.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.207+0000 7f9592d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95941411b0 0x7f959415a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41738/0 (socket says 172.21.15.40:41738) 2023-10-18T08:56:50.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.207+0000 7f9592d9d640 1 -- 172.21.15.40:0/2284160706 learned_addr learned my addr 172.21.15.40:0/2284160706 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:50.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.207+0000 7f9592d9d640 1 -- 172.21.15.40:0/2284160706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95940b0a50 msgr2=0x7f95941f2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:50.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.207+0000 7f959259c640 1 --2- 172.21.15.40:0/2284160706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9594070960 0x7f95940adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:50.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.207+0000 7f958bfff640 1 --2- 172.21.15.40:0/2284160706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95940b0a50 0x7f95941f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:50.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.207+0000 7f9592d9d640 1 --2- 172.21.15.40:0/2284160706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95940b0a50 0x7f95941f2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:50.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.207+0000 7f9592d9d640 1 -- 172.21.15.40:0/2284160706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9594070960 msgr2=0x7f95940adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:50.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.207+0000 7f9592d9d640 1 --2- 172.21.15.40:0/2284160706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9594070960 0x7f95940adb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:50.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.207+0000 7f9592d9d640 1 -- 172.21.15.40:0/2284160706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9584009560 con 0x7f95941411b0 2023-10-18T08:56:50.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.208+0000 7f959259c640 1 --2- 172.21.15.40:0/2284160706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9594070960 0x7f95940adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:50.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.208+0000 7f9592d9d640 1 --2- 172.21.15.40:0/2284160706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95941411b0 0x7f959415a640 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f958c00e960 tx=0x7f958c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:50.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.208+0000 7f958affd640 1 -- 172.21.15.40:0/2284160706 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c004d20 con 0x7f95941411b0 2023-10-18T08:56:50.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.208+0000 7f958affd640 1 -- 172.21.15.40:0/2284160706 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f958c00cac0 con 0x7f95941411b0 2023-10-18T08:56:50.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.208+0000 7f958affd640 1 -- 172.21.15.40:0/2284160706 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c005020 con 0x7f95941411b0 2023-10-18T08:56:50.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.208+0000 7f959359e640 1 -- 172.21.15.40:0/2284160706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95940ae340 con 0x7f95941411b0 2023-10-18T08:56:50.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.208+0000 7f959359e640 1 -- 172.21.15.40:0/2284160706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95940ae900 con 0x7f95941411b0 2023-10-18T08:56:50.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.209+0000 7f959359e640 1 -- 172.21.15.40:0/2284160706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9594066e00 con 0x7f95941411b0 2023-10-18T08:56:50.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.210+0000 7f958affd640 1 -- 172.21.15.40:0/2284160706 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f958c005180 con 0x7f95941411b0 2023-10-18T08:56:50.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.211+0000 7f958affd640 1 --2- 172.21.15.40:0/2284160706 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f957406ae40 0x7f957406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:50.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.211+0000 7f958affd640 1 -- 172.21.15.40:0/2284160706 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f958c089f10 con 0x7f95941411b0 2023-10-18T08:56:50.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.212+0000 7f959259c640 1 --2- 172.21.15.40:0/2284160706 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f957406ae40 0x7f957406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:50.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.213+0000 7f959259c640 1 --2- 172.21.15.40:0/2284160706 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f957406ae40 0x7f957406d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9584035000 tx=0x7f9584004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:50.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.215+0000 7f958affd640 1 -- 172.21.15.40:0/2284160706 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f958c055050 con 0x7f95941411b0 2023-10-18T08:56:50.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.408+0000 7f959359e640 1 -- 172.21.15.40:0/2284160706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95941558f0 con 0x7f95941411b0 2023-10-18T08:56:50.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.409+0000 7f958affd640 1 -- 172.21.15.40:0/2284160706 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95941558f0 con 0x7f95941411b0 2023-10-18T08:56:50.413 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:56:50.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.411+0000 7f959359e640 1 -- 172.21.15.40:0/2284160706 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f957406ae40 msgr2=0x7f957406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:50.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.411+0000 7f959359e640 1 --2- 172.21.15.40:0/2284160706 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f957406ae40 0x7f957406d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9584035000 tx=0x7f9584004730 comp rx=0 tx=0).stop 2023-10-18T08:56:50.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.412+0000 7f959359e640 1 -- 172.21.15.40:0/2284160706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95941411b0 msgr2=0x7f959415a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:50.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.412+0000 7f959359e640 1 --2- 172.21.15.40:0/2284160706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95941411b0 0x7f959415a640 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f958c00e960 tx=0x7f958c00ee20 comp rx=0 tx=0).stop 2023-10-18T08:56:50.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.412+0000 7f959359e640 1 -- 172.21.15.40:0/2284160706 shutdown_connections 2023-10-18T08:56:50.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.412+0000 7f959359e640 1 --2- 172.21.15.40:0/2284160706 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f957406ae40 0x7f957406d300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:50.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.412+0000 7f959359e640 1 --2- 172.21.15.40:0/2284160706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95941411b0 0x7f959415a640 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:50.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.412+0000 7f959359e640 1 --2- 172.21.15.40:0/2284160706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95940b0a50 0x7f95941f2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:50.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.412+0000 7f959359e640 1 --2- 172.21.15.40:0/2284160706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9594070960 0x7f95940adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:50.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.412+0000 7f959359e640 1 -- 172.21.15.40:0/2284160706 >> 172.21.15.40:0/2284160706 conn(0x7f9594069720 msgr2=0x7f959413cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:50.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.412+0000 7f959359e640 1 -- 172.21.15.40:0/2284160706 shutdown_connections 2023-10-18T08:56:50.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.412+0000 7f959359e640 1 -- 172.21.15.40:0/2284160706 wait complete. 2023-10-18T08:56:50.432 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:56:50.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.667+0000 7f3288cd7640 1 -- 172.21.15.40:0/2018969728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f328406f570 msgr2=0x7f328406f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:50.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.667+0000 7f3288cd7640 1 --2- 172.21.15.40:0/2018969728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f328406f570 0x7f328406f9e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f327c005fd0 tx=0x7f327c02bd80 comp rx=0 tx=0).stop 2023-10-18T08:56:50.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.667+0000 7f3288cd7640 1 -- 172.21.15.40:0/2018969728 shutdown_connections 2023-10-18T08:56:50.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.667+0000 7f3288cd7640 1 --2- 172.21.15.40:0/2018969728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f328406f570 0x7f328406f9e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:50.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.667+0000 7f3288cd7640 1 --2- 172.21.15.40:0/2018969728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f328406d660 0x7f32840cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:50.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.667+0000 7f3288cd7640 1 -- 172.21.15.40:0/2018969728 >> 172.21.15.40:0/2018969728 conn(0x7f3284069720 msgr2=0x7f3284069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:50.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.667+0000 7f3288cd7640 1 -- 172.21.15.40:0/2018969728 shutdown_connections 2023-10-18T08:56:50.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.667+0000 7f3288cd7640 1 -- 172.21.15.40:0/2018969728 wait complete. 2023-10-18T08:56:50.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.669+0000 7f3288cd7640 1 Processor -- start 2023-10-18T08:56:50.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.669+0000 7f3288cd7640 1 -- start start 2023-10-18T08:56:50.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.670+0000 7f3288cd7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f328406d660 0x7f3284152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:50.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.670+0000 7f3288cd7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f328406f570 0x7f32840ab920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:50.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.670+0000 7f3288cd7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3284157800 0x7f32840abe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:50.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.670+0000 7f3288cd7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3284153420 con 0x7f3284157800 2023-10-18T08:56:50.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.670+0000 7f3288cd7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3284157ce0 con 0x7f328406f570 2023-10-18T08:56:50.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.670+0000 7f3288cd7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3284157fc0 con 0x7f328406d660 2023-10-18T08:56:50.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.670+0000 7f3282d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3284157800 0x7f32840abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:50.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.670+0000 7f3282d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3284157800 0x7f32840abe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41766/0 (socket says 172.21.15.40:41766) 2023-10-18T08:56:50.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.670+0000 7f3282d9d640 1 -- 172.21.15.40:0/582838607 learned_addr learned my addr 172.21.15.40:0/582838607 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:50.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.671+0000 7f3282d9d640 1 -- 172.21.15.40:0/582838607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f328406d660 msgr2=0x7f3284152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:50.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.671+0000 7f3282d9d640 1 --2- 172.21.15.40:0/582838607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f328406d660 0x7f3284152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:50.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.671+0000 7f3282d9d640 1 -- 172.21.15.40:0/582838607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f328406f570 msgr2=0x7f32840ab920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:50.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.671+0000 7f3282d9d640 1 --2- 172.21.15.40:0/582838607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f328406f570 0x7f32840ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:50.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.671+0000 7f3282d9d640 1 -- 172.21.15.40:0/582838607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c009560 con 0x7f3284157800 2023-10-18T08:56:50.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.671+0000 7f3282d9d640 1 --2- 172.21.15.40:0/582838607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3284157800 0x7f32840abe60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f327c036040 tx=0x7f327c02da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:50.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.671+0000 7f327affd640 1 -- 172.21.15.40:0/582838607 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c02de30 con 0x7f3284157800 2023-10-18T08:56:50.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.671+0000 7f327affd640 1 -- 172.21.15.40:0/582838607 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f327c0036e0 con 0x7f3284157800 2023-10-18T08:56:50.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.671+0000 7f327affd640 1 -- 172.21.15.40:0/582838607 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c003af0 con 0x7f3284157800 2023-10-18T08:56:50.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.671+0000 7f3288cd7640 1 -- 172.21.15.40:0/582838607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32841582a0 con 0x7f3284157800 2023-10-18T08:56:50.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.671+0000 7f3288cd7640 1 -- 172.21.15.40:0/582838607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3284158860 con 0x7f3284157800 2023-10-18T08:56:50.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.673+0000 7f3288cd7640 1 -- 172.21.15.40:0/582838607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3284066fd0 con 0x7f3284157800 2023-10-18T08:56:50.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.674+0000 7f327affd640 1 -- 172.21.15.40:0/582838607 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f327c003c50 con 0x7f3284157800 2023-10-18T08:56:50.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.675+0000 7f327affd640 1 --2- 172.21.15.40:0/582838607 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f326406ae40 0x7f326406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:50.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.675+0000 7f327affd640 1 -- 172.21.15.40:0/582838607 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f327c0abf00 con 0x7f3284157800 2023-10-18T08:56:50.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.676+0000 7f328259c640 1 --2- 172.21.15.40:0/582838607 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f326406ae40 0x7f326406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:50.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.677+0000 7f328259c640 1 --2- 172.21.15.40:0/582838607 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f326406ae40 0x7f326406d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f32840ac520 tx=0x7f327400a780 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.679+0000 7f327affd640 1 -- 172.21.15.40:0/582838607 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f327c077160 con 0x7f3284157800 2023-10-18T08:56:50.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.874+0000 7f3288cd7640 1 -- 172.21.15.40:0/582838607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f32840cc450 con 0x7f3284157800 2023-10-18T08:56:50.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.876+0000 7f327affd640 1 -- 172.21.15.40:0/582838607 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f32840cc450 con 0x7f3284157800 2023-10-18T08:56:50.880 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:56:50.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.879+0000 7f3288cd7640 1 -- 172.21.15.40:0/582838607 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f326406ae40 msgr2=0x7f326406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:50.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.879+0000 7f3288cd7640 1 --2- 172.21.15.40:0/582838607 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f326406ae40 0x7f326406d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f32840ac520 tx=0x7f327400a780 comp rx=0 tx=0).stop 2023-10-18T08:56:50.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.879+0000 7f3288cd7640 1 -- 172.21.15.40:0/582838607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3284157800 msgr2=0x7f32840abe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:50.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.879+0000 7f3288cd7640 1 --2- 172.21.15.40:0/582838607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3284157800 0x7f32840abe60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f327c036040 tx=0x7f327c02da30 comp rx=0 tx=0).stop 2023-10-18T08:56:50.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.879+0000 7f3288cd7640 1 -- 172.21.15.40:0/582838607 shutdown_connections 2023-10-18T08:56:50.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.879+0000 7f3288cd7640 1 --2- 172.21.15.40:0/582838607 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f326406ae40 0x7f326406d300 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:50.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.879+0000 7f3288cd7640 1 --2- 172.21.15.40:0/582838607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3284157800 0x7f32840abe60 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:50.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.879+0000 7f3288cd7640 1 --2- 172.21.15.40:0/582838607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f328406f570 0x7f32840ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:50.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.879+0000 7f3288cd7640 1 --2- 172.21.15.40:0/582838607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f328406d660 0x7f3284152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:50.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.879+0000 7f3288cd7640 1 -- 172.21.15.40:0/582838607 >> 172.21.15.40:0/582838607 conn(0x7f3284069720 msgr2=0x7f32840acc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:50.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.879+0000 7f3288cd7640 1 -- 172.21.15.40:0/582838607 shutdown_connections 2023-10-18T08:56:50.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:50.880+0000 7f3288cd7640 1 -- 172.21.15.40:0/582838607 wait complete. 2023-10-18T08:56:50.900 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:56:51.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.128+0000 7faa73fff640 1 -- 172.21.15.40:0/4168524740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7415eac0 msgr2=0x7faa740cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:51.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.128+0000 7faa73fff640 1 --2- 172.21.15.40:0/4168524740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7415eac0 0x7faa740cff70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7faa64005fd0 tx=0x7faa6402c240 comp rx=0 tx=0).stop 2023-10-18T08:56:51.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.128+0000 7faa73fff640 1 -- 172.21.15.40:0/4168524740 shutdown_connections 2023-10-18T08:56:51.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.128+0000 7faa73fff640 1 --2- 172.21.15.40:0/4168524740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa7415eea0 0x7faa740b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.128+0000 7faa73fff640 1 --2- 172.21.15.40:0/4168524740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa7415eac0 0x7faa740cff70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.128+0000 7faa73fff640 1 -- 172.21.15.40:0/4168524740 >> 172.21.15.40:0/4168524740 conn(0x7faa7409a610 msgr2=0x7faa740d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:51.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.128+0000 7faa73fff640 1 -- 172.21.15.40:0/4168524740 shutdown_connections 2023-10-18T08:56:51.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.128+0000 7faa73fff640 1 -- 172.21.15.40:0/4168524740 wait complete. 2023-10-18T08:56:51.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.130+0000 7faa73fff640 1 Processor -- start 2023-10-18T08:56:51.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.130+0000 7faa73fff640 1 -- start start 2023-10-18T08:56:51.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.131+0000 7faa73fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa740d19b0 0x7faa74158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:51.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.131+0000 7faa73fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7415eac0 0x7faa740b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:51.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.131+0000 7faa73fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa7415eea0 0x7faa740b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:51.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.131+0000 7faa73fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa741e57b0 con 0x7faa740d19b0 2023-10-18T08:56:51.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.131+0000 7faa73fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa740b5be0 con 0x7faa7415eac0 2023-10-18T08:56:51.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.131+0000 7faa73fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa741e4ba0 con 0x7faa7415eea0 2023-10-18T08:56:51.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.131+0000 7faa72ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa740d19b0 0x7faa74158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:51.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.131+0000 7faa72ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa740d19b0 0x7faa74158680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41788/0 (socket says 172.21.15.40:41788) 2023-10-18T08:56:51.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.131+0000 7faa72ffd640 1 -- 172.21.15.40:0/2578751062 learned_addr learned my addr 172.21.15.40:0/2578751062 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:51.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.131+0000 7faa72ffd640 1 -- 172.21.15.40:0/2578751062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa7415eea0 msgr2=0x7faa740b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:51.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.131+0000 7faa72ffd640 1 --2- 172.21.15.40:0/2578751062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa7415eea0 0x7faa740b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.131+0000 7faa72ffd640 1 -- 172.21.15.40:0/2578751062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7415eac0 msgr2=0x7faa740b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:51.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.131+0000 7faa72ffd640 1 --2- 172.21.15.40:0/2578751062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7415eac0 0x7faa740b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.131+0000 7faa72ffd640 1 -- 172.21.15.40:0/2578751062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa64009900 con 0x7faa740d19b0 2023-10-18T08:56:51.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.132+0000 7faa72ffd640 1 --2- 172.21.15.40:0/2578751062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa740d19b0 0x7faa74158680 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7faa64006010 tx=0x7faa64035ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:51.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.132+0000 7faa5b7fe640 1 -- 172.21.15.40:0/2578751062 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa64034440 con 0x7faa740d19b0 2023-10-18T08:56:51.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.132+0000 7faa5b7fe640 1 -- 172.21.15.40:0/2578751062 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faa64004330 con 0x7faa740d19b0 2023-10-18T08:56:51.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.132+0000 7faa73fff640 1 -- 172.21.15.40:0/2578751062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa741e4e80 con 0x7faa740d19b0 2023-10-18T08:56:51.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.132+0000 7faa5b7fe640 1 -- 172.21.15.40:0/2578751062 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa640046e0 con 0x7faa740d19b0 2023-10-18T08:56:51.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.132+0000 7faa73fff640 1 -- 172.21.15.40:0/2578751062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa741e5440 con 0x7faa740d19b0 2023-10-18T08:56:51.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.133+0000 7faa73fff640 1 -- 172.21.15.40:0/2578751062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa74071da0 con 0x7faa740d19b0 2023-10-18T08:56:51.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.134+0000 7faa5b7fe640 1 -- 172.21.15.40:0/2578751062 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faa6403b040 con 0x7faa740d19b0 2023-10-18T08:56:51.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.135+0000 7faa5b7fe640 1 --2- 172.21.15.40:0/2578751062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa6006ae70 0x7faa6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:51.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.135+0000 7faa5b7fe640 1 -- 172.21.15.40:0/2578751062 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa640ac220 con 0x7faa740d19b0 2023-10-18T08:56:51.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.136+0000 7faa727fc640 1 --2- 172.21.15.40:0/2578751062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa6006ae70 0x7faa6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:51.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.137+0000 7faa727fc640 1 --2- 172.21.15.40:0/2578751062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa6006ae70 0x7faa6006d330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faa740b3d20 tx=0x7faa68005b50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:51.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.140+0000 7faa5b7fe640 1 -- 172.21.15.40:0/2578751062 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa64077480 con 0x7faa740d19b0 2023-10-18T08:56:51.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.332+0000 7faa73fff640 1 -- 172.21.15.40:0/2578751062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faa74066e00 con 0x7faa740d19b0 2023-10-18T08:56:51.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.333+0000 7faa5b7fe640 1 -- 172.21.15.40:0/2578751062 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7faa6407aaa0 con 0x7faa740d19b0 2023-10-18T08:56:51.336 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:56:51.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.336+0000 7faa73fff640 1 -- 172.21.15.40:0/2578751062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa6006ae70 msgr2=0x7faa6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:51.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.336+0000 7faa73fff640 1 --2- 172.21.15.40:0/2578751062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa6006ae70 0x7faa6006d330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faa740b3d20 tx=0x7faa68005b50 comp rx=0 tx=0).stop 2023-10-18T08:56:51.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.336+0000 7faa73fff640 1 -- 172.21.15.40:0/2578751062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa740d19b0 msgr2=0x7faa74158680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:51.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.336+0000 7faa73fff640 1 --2- 172.21.15.40:0/2578751062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa740d19b0 0x7faa74158680 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7faa64006010 tx=0x7faa64035ac0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.336+0000 7faa73fff640 1 -- 172.21.15.40:0/2578751062 shutdown_connections 2023-10-18T08:56:51.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.336+0000 7faa73fff640 1 --2- 172.21.15.40:0/2578751062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa6006ae70 0x7faa6006d330 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.336+0000 7faa73fff640 1 --2- 172.21.15.40:0/2578751062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa7415eea0 0x7faa740b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.336+0000 7faa73fff640 1 --2- 172.21.15.40:0/2578751062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa7415eac0 0x7faa740b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.336+0000 7faa73fff640 1 --2- 172.21.15.40:0/2578751062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa740d19b0 0x7faa74158680 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.336+0000 7faa73fff640 1 -- 172.21.15.40:0/2578751062 >> 172.21.15.40:0/2578751062 conn(0x7faa7409a610 msgr2=0x7faa7415a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:51.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.336+0000 7faa73fff640 1 -- 172.21.15.40:0/2578751062 shutdown_connections 2023-10-18T08:56:51.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.336+0000 7faa73fff640 1 -- 172.21.15.40:0/2578751062 wait complete. 2023-10-18T08:56:51.358 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T08:56:51.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.584+0000 7f79eb59e640 1 -- 172.21.15.40:0/2761238127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ec0d1910 msgr2=0x7f79ec06a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:51.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.584+0000 7f79eb59e640 1 --2- 172.21.15.40:0/2761238127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ec0d1910 0x7f79ec06a5a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0d0fa0 tx=0x7f79e002bd70 comp rx=0 tx=0).stop 2023-10-18T08:56:51.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.584+0000 7f79eb59e640 1 -- 172.21.15.40:0/2761238127 shutdown_connections 2023-10-18T08:56:51.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.584+0000 7f79eb59e640 1 --2- 172.21.15.40:0/2761238127 >> v2:172.21.15.144:3300/0 conn(0x7f79ec06aae0 0x7f79ec0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.584+0000 7f79eb59e640 1 --2- 172.21.15.40:0/2761238127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ec0d1910 0x7f79ec06a5a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.584+0000 7f79eb59e640 1 -- 172.21.15.40:0/2761238127 >> 172.21.15.40:0/2761238127 conn(0x7f79ec08ac60 msgr2=0x7f79ec09a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:51.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.584+0000 7f79eb59e640 1 -- 172.21.15.40:0/2761238127 shutdown_connections 2023-10-18T08:56:51.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.584+0000 7f79eb59e640 1 -- 172.21.15.40:0/2761238127 wait complete. 2023-10-18T08:56:51.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.586+0000 7f79eb59e640 1 Processor -- start 2023-10-18T08:56:51.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.586+0000 7f79eb59e640 1 -- start start 2023-10-18T08:56:51.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.587+0000 7f79eb59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79ec06aae0 0x7f79ec1531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:51.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.587+0000 7f79eb59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ec0d1910 0x7f79ec0ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:51.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.587+0000 7f79eb59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79ec15b530 0x7f79ec15d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:51.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.587+0000 7f79eb59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec153720 con 0x7f79ec0d1910 2023-10-18T08:56:51.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.587+0000 7f79eb59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec1f5310 con 0x7f79ec06aae0 2023-10-18T08:56:51.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.587+0000 7f79eb59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec1f55f0 con 0x7f79ec15b530 2023-10-18T08:56:51.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.588+0000 7f79e9d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ec0d1910 0x7f79ec0ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:51.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.588+0000 7f79e9d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ec0d1910 0x7f79ec0ca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41816/0 (socket says 172.21.15.40:41816) 2023-10-18T08:56:51.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.588+0000 7f79e9d9b640 1 -- 172.21.15.40:0/344973401 learned_addr learned my addr 172.21.15.40:0/344973401 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:51.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.588+0000 7f79ead9d640 1 --2- 172.21.15.40:0/344973401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79ec15b530 0x7f79ec15d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:51.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.588+0000 7f79ea59c640 1 --2- 172.21.15.40:0/344973401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79ec06aae0 0x7f79ec1531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:51.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.588+0000 7f79e9d9b640 1 -- 172.21.15.40:0/344973401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79ec15b530 msgr2=0x7f79ec15d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:51.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.588+0000 7f79e9d9b640 1 --2- 172.21.15.40:0/344973401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79ec15b530 0x7f79ec15d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.588+0000 7f79e9d9b640 1 -- 172.21.15.40:0/344973401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79ec06aae0 msgr2=0x7f79ec1531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:51.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.588+0000 7f79e9d9b640 1 --2- 172.21.15.40:0/344973401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79ec06aae0 0x7f79ec1531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.588+0000 7f79e9d9b640 1 -- 172.21.15.40:0/344973401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e0009490 con 0x7f79ec0d1910 2023-10-18T08:56:51.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.588+0000 7f79e9d9b640 1 --2- 172.21.15.40:0/344973401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ec0d1910 0x7f79ec0ca670 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f79e0000c00 tx=0x7f79e00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:51.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.589+0000 7f79daffd640 1 -- 172.21.15.40:0/344973401 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0034440 con 0x7f79ec0d1910 2023-10-18T08:56:51.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.589+0000 7f79daffd640 1 -- 172.21.15.40:0/344973401 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f79e00346b0 con 0x7f79ec0d1910 2023-10-18T08:56:51.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.589+0000 7f79daffd640 1 -- 172.21.15.40:0/344973401 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0034ac0 con 0x7f79ec0d1910 2023-10-18T08:56:51.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.589+0000 7f79eb59e640 1 -- 172.21.15.40:0/344973401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ec1f58d0 con 0x7f79ec0d1910 2023-10-18T08:56:51.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.589+0000 7f79eb59e640 1 -- 172.21.15.40:0/344973401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ec1f5e90 con 0x7f79ec0d1910 2023-10-18T08:56:51.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.590+0000 7f79eb59e640 1 -- 172.21.15.40:0/344973401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ec066e00 con 0x7f79ec0d1910 2023-10-18T08:56:51.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.591+0000 7f79daffd640 1 -- 172.21.15.40:0/344973401 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f79e003b040 con 0x7f79ec0d1910 2023-10-18T08:56:51.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.592+0000 7f79daffd640 1 --2- 172.21.15.40:0/344973401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79cc06ae70 0x7f79cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:51.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.592+0000 7f79daffd640 1 -- 172.21.15.40:0/344973401 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79e00abd40 con 0x7f79ec0d1910 2023-10-18T08:56:51.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.593+0000 7f79ea59c640 1 --2- 172.21.15.40:0/344973401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79cc06ae70 0x7f79cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:51.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.594+0000 7f79ea59c640 1 --2- 172.21.15.40:0/344973401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79cc06ae70 0x7f79cc06d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f79dc007fd0 tx=0x7f79dc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:51.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.597+0000 7f79daffd640 1 -- 172.21.15.40:0/344973401 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79e0076d50 con 0x7f79ec0d1910 2023-10-18T08:56:51.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.803+0000 7f79eb59e640 1 -- 172.21.15.40:0/344973401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f79ec0cb140 con 0x7f79ec0d1910 2023-10-18T08:56:51.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.804+0000 7f79daffd640 1 -- 172.21.15.40:0/344973401 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7f79e0039080 con 0x7f79ec0d1910 2023-10-18T08:56:51.808 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T08:56:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.807+0000 7f79eb59e640 1 -- 172.21.15.40:0/344973401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79cc06ae70 msgr2=0x7f79cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:51.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.807+0000 7f79eb59e640 1 --2- 172.21.15.40:0/344973401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79cc06ae70 0x7f79cc06d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f79dc007fd0 tx=0x7f79dc009040 comp rx=0 tx=0).stop 2023-10-18T08:56:51.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.807+0000 7f79eb59e640 1 -- 172.21.15.40:0/344973401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ec0d1910 msgr2=0x7f79ec0ca670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:51.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.807+0000 7f79eb59e640 1 --2- 172.21.15.40:0/344973401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ec0d1910 0x7f79ec0ca670 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f79e0000c00 tx=0x7f79e00039e0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.807+0000 7f79eb59e640 1 -- 172.21.15.40:0/344973401 shutdown_connections 2023-10-18T08:56:51.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.807+0000 7f79eb59e640 1 --2- 172.21.15.40:0/344973401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79cc06ae70 0x7f79cc06d330 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.807+0000 7f79eb59e640 1 --2- 172.21.15.40:0/344973401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79ec15b530 0x7f79ec15d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.807+0000 7f79eb59e640 1 --2- 172.21.15.40:0/344973401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79ec0d1910 0x7f79ec0ca670 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.807+0000 7f79eb59e640 1 --2- 172.21.15.40:0/344973401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79ec06aae0 0x7f79ec1531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.807+0000 7f79eb59e640 1 -- 172.21.15.40:0/344973401 >> 172.21.15.40:0/344973401 conn(0x7f79ec08ac60 msgr2=0x7f79ec15bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.807+0000 7f79eb59e640 1 -- 172.21.15.40:0/344973401 shutdown_connections 2023-10-18T08:56:51.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:51.807+0000 7f79eb59e640 1 -- 172.21.15.40:0/344973401 wait complete. 2023-10-18T08:56:51.828 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-18T08:56:51.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:51.829 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-18T08:56:51.981 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T08:56:51.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:51.982 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T08:56:52.026 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:56:52.027 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:52.071 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:56:52.072 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:52.166 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:52.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.397+0000 7f4ddb483640 1 -- 172.21.15.40:0/3868583320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dd406d6f0 msgr2=0x7f4dd40b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:52.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.397+0000 7f4ddb483640 1 --2- 172.21.15.40:0/3868583320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dd406d6f0 0x7f4dd40b15b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0005fd0 tx=0x7f4dd002be20 comp rx=0 tx=0).stop 2023-10-18T08:56:52.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.397+0000 7f4ddb483640 1 -- 172.21.15.40:0/3868583320 shutdown_connections 2023-10-18T08:56:52.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.397+0000 7f4ddb483640 1 --2- 172.21.15.40:0/3868583320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dd406d6f0 0x7f4dd40b15b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:52.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.397+0000 7f4ddb483640 1 -- 172.21.15.40:0/3868583320 >> 172.21.15.40:0/3868583320 conn(0x7f4dd4069720 msgr2=0x7f4dd4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:52.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.397+0000 7f4ddb483640 1 -- 172.21.15.40:0/3868583320 shutdown_connections 2023-10-18T08:56:52.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.397+0000 7f4ddb483640 1 -- 172.21.15.40:0/3868583320 wait complete. 2023-10-18T08:56:52.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.399+0000 7f4ddb483640 1 Processor -- start 2023-10-18T08:56:52.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.399+0000 7f4ddb483640 1 -- start start 2023-10-18T08:56:52.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.400+0000 7f4ddb483640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4dd406d6f0 0x7f4dd4144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:52.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.400+0000 7f4ddb483640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd40b0630 0x7f4dd4157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:52.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.400+0000 7f4ddb483640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dd40c99c0 0x7f4dd4150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:52.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.400+0000 7f4ddb483640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd4144dd0 con 0x7f4dd40c99c0 2023-10-18T08:56:52.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.400+0000 7f4ddb483640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40af060 con 0x7f4dd40b0630 2023-10-18T08:56:52.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.400+0000 7f4ddb483640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd40af340 con 0x7f4dd406d6f0 2023-10-18T08:56:52.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.401+0000 7f4ddac82640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dd40c99c0 0x7f4dd4150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:52.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.401+0000 7f4ddac82640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dd40c99c0 0x7f4dd4150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41832/0 (socket says 172.21.15.40:41832) 2023-10-18T08:56:52.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.401+0000 7f4ddac82640 1 -- 172.21.15.40:0/4132853848 learned_addr learned my addr 172.21.15.40:0/4132853848 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:52.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.401+0000 7f4dda481640 1 --2- 172.21.15.40:0/4132853848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4dd406d6f0 0x7f4dd4144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:52.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.401+0000 7f4dd9c80640 1 --2- 172.21.15.40:0/4132853848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd40b0630 0x7f4dd4157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:52.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.401+0000 7f4ddac82640 1 -- 172.21.15.40:0/4132853848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4dd406d6f0 msgr2=0x7f4dd4144890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:52.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.401+0000 7f4ddac82640 1 --2- 172.21.15.40:0/4132853848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4dd406d6f0 0x7f4dd4144890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:52.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.401+0000 7f4ddac82640 1 -- 172.21.15.40:0/4132853848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd40b0630 msgr2=0x7f4dd4157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:52.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.401+0000 7f4ddac82640 1 --2- 172.21.15.40:0/4132853848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd40b0630 0x7f4dd4157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:52.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.401+0000 7f4ddac82640 1 -- 172.21.15.40:0/4132853848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd0009560 con 0x7f4dd40c99c0 2023-10-18T08:56:52.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.402+0000 7f4ddac82640 1 --2- 172.21.15.40:0/4132853848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dd40c99c0 0x7f4dd4150910 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00b990 tx=0x7f4dcc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:52.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.402+0000 7f4dcaffd640 1 -- 172.21.15.40:0/4132853848 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc00cba0 con 0x7f4dd40c99c0 2023-10-18T08:56:52.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.402+0000 7f4dcaffd640 1 -- 172.21.15.40:0/4132853848 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4dcc00cdf0 con 0x7f4dd40c99c0 2023-10-18T08:56:52.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.402+0000 7f4dcaffd640 1 -- 172.21.15.40:0/4132853848 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc0146a0 con 0x7f4dd40c99c0 2023-10-18T08:56:52.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.402+0000 7f4ddb483640 1 -- 172.21.15.40:0/4132853848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd40af620 con 0x7f4dd40c99c0 2023-10-18T08:56:52.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.402+0000 7f4ddb483640 1 -- 172.21.15.40:0/4132853848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd40aa880 con 0x7f4dd40c99c0 2023-10-18T08:56:52.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.403+0000 7f4ddb483640 1 -- 172.21.15.40:0/4132853848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd40aaaa0 con 0x7f4dd40c99c0 2023-10-18T08:56:52.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.404+0000 7f4dcaffd640 1 -- 172.21.15.40:0/4132853848 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4dcc01d3b0 con 0x7f4dd40c99c0 2023-10-18T08:56:52.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.405+0000 7f4dcaffd640 1 --2- 172.21.15.40:0/4132853848 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4db806ae40 0x7f4db806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:52.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.405+0000 7f4dcaffd640 1 -- 172.21.15.40:0/4132853848 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4dcc0897f0 con 0x7f4dd40c99c0 2023-10-18T08:56:52.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.406+0000 7f4dda481640 1 --2- 172.21.15.40:0/4132853848 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4db806ae40 0x7f4db806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:52.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.407+0000 7f4dda481640 1 --2- 172.21.15.40:0/4132853848 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4db806ae40 0x7f4db806d300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0035000 tx=0x7f4dd0004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.409+0000 7f4dcaffd640 1 -- 172.21.15.40:0/4132853848 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4dcc0548b0 con 0x7f4dd40c99c0 2023-10-18T08:56:52.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.604+0000 7f4ddb483640 1 -- 172.21.15.40:0/4132853848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4dd4066e00 con 0x7f4dd40c99c0 2023-10-18T08:56:52.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.606+0000 7f4dcaffd640 1 -- 172.21.15.40:0/4132853848 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16581 (secure 0 0 0) 0x7f4dcc019090 con 0x7f4dd40c99c0 2023-10-18T08:56:52.610 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:52.610 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":35,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:46.967356+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:56:30.127184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-18T08:56:31.225270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:52.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.609+0000 7f4ddb483640 1 -- 172.21.15.40:0/4132853848 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4db806ae40 msgr2=0x7f4db806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:52.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.609+0000 7f4ddb483640 1 --2- 172.21.15.40:0/4132853848 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4db806ae40 0x7f4db806d300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0035000 tx=0x7f4dd0004730 comp rx=0 tx=0).stop 2023-10-18T08:56:52.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.609+0000 7f4ddb483640 1 -- 172.21.15.40:0/4132853848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dd40c99c0 msgr2=0x7f4dd4150910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:52.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.609+0000 7f4ddb483640 1 --2- 172.21.15.40:0/4132853848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dd40c99c0 0x7f4dd4150910 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc00b990 tx=0x7f4dcc00be50 comp rx=0 tx=0).stop 2023-10-18T08:56:52.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.609+0000 7f4ddb483640 1 -- 172.21.15.40:0/4132853848 shutdown_connections 2023-10-18T08:56:52.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.609+0000 7f4ddb483640 1 --2- 172.21.15.40:0/4132853848 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4db806ae40 0x7f4db806d300 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:52.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.609+0000 7f4ddb483640 1 --2- 172.21.15.40:0/4132853848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dd40c99c0 0x7f4dd4150910 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:52.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.609+0000 7f4ddb483640 1 --2- 172.21.15.40:0/4132853848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dd40b0630 0x7f4dd4157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:52.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.609+0000 7f4ddb483640 1 --2- 172.21.15.40:0/4132853848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4dd406d6f0 0x7f4dd4144890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:52.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.609+0000 7f4ddb483640 1 -- 172.21.15.40:0/4132853848 >> 172.21.15.40:0/4132853848 conn(0x7f4dd4069720 msgr2=0x7f4dd4153390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:52.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.609+0000 7f4ddb483640 1 -- 172.21.15.40:0/4132853848 shutdown_connections 2023-10-18T08:56:52.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.609+0000 7f4ddb483640 1 -- 172.21.15.40:0/4132853848 wait complete. 2023-10-18T08:56:52.631 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:56:30.127184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:56:31.225270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:56:52.632 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:56:52.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.858+0000 7f32fb0da640 1 -- 172.21.15.40:0/2024069715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f4070790 msgr2=0x7f32f40cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:52.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.858+0000 7f32fb0da640 1 --2- 172.21.15.40:0/2024069715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f4070790 0x7f32f40cc980 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f32f415bb90 tx=0x7f32ec02c260 comp rx=0 tx=0).stop 2023-10-18T08:56:52.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.859+0000 7f32fb0da640 1 -- 172.21.15.40:0/2024069715 shutdown_connections 2023-10-18T08:56:52.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.859+0000 7f32fb0da640 1 --2- 172.21.15.40:0/2024069715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f4070790 0x7f32f40cc980 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:52.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.859+0000 7f32fb0da640 1 --2- 172.21.15.40:0/2024069715 >> v2:172.21.15.144:3300/0 conn(0x7f32f40b0a50 0x7f32f4070250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:52.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.859+0000 7f32fb0da640 1 -- 172.21.15.40:0/2024069715 >> 172.21.15.40:0/2024069715 conn(0x7f32f4069720 msgr2=0x7f32f4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:52.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.859+0000 7f32fb0da640 1 -- 172.21.15.40:0/2024069715 shutdown_connections 2023-10-18T08:56:52.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.859+0000 7f32fb0da640 1 -- 172.21.15.40:0/2024069715 wait complete. 2023-10-18T08:56:52.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.861+0000 7f32fb0da640 1 Processor -- start 2023-10-18T08:56:52.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.861+0000 7f32fb0da640 1 -- start start 2023-10-18T08:56:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.863+0000 7f32fb0da640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f4070790 0x7f32f413aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.863+0000 7f32fb0da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f40b0a50 0x7f32f4158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.863+0000 7f32fb0da640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f41411b0 0x7f32f4158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.863+0000 7f32fb0da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f413b000 con 0x7f32f40b0a50 2023-10-18T08:56:52.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.863+0000 7f32fb0da640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f4159ed0 con 0x7f32f41411b0 2023-10-18T08:56:52.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.863+0000 7f32fb0da640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f415a360 con 0x7f32f4070790 2023-10-18T08:56:52.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.863+0000 7f32f98d7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f40b0a50 0x7f32f4158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:52.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.863+0000 7f32f98d7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f40b0a50 0x7f32f4158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41850/0 (socket says 172.21.15.40:41850) 2023-10-18T08:56:52.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.863+0000 7f32f98d7640 1 -- 172.21.15.40:0/3514514695 learned_addr learned my addr 172.21.15.40:0/3514514695 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:52.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.863+0000 7f32fa0d8640 1 --2- 172.21.15.40:0/3514514695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f4070790 0x7f32f413aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:52.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.864+0000 7f32fa8d9640 1 --2- 172.21.15.40:0/3514514695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f41411b0 0x7f32f4158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:52.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.864+0000 7f32f98d7640 1 -- 172.21.15.40:0/3514514695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f4070790 msgr2=0x7f32f413aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:52.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.864+0000 7f32f98d7640 1 --2- 172.21.15.40:0/3514514695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f4070790 0x7f32f413aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:52.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.864+0000 7f32f98d7640 1 -- 172.21.15.40:0/3514514695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f41411b0 msgr2=0x7f32f4158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:52.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.864+0000 7f32f98d7640 1 --2- 172.21.15.40:0/3514514695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f41411b0 0x7f32f4158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:52.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.864+0000 7f32f98d7640 1 -- 172.21.15.40:0/3514514695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec009900 con 0x7f32f40b0a50 2023-10-18T08:56:52.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.864+0000 7f32f98d7640 1 --2- 172.21.15.40:0/3514514695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f40b0a50 0x7f32f4158750 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f32e800d770 tx=0x7f32e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:52.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.864+0000 7f32e6ffd640 1 -- 172.21.15.40:0/3514514695 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8009c30 con 0x7f32f40b0a50 2023-10-18T08:56:52.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.864+0000 7f32e6ffd640 1 -- 172.21.15.40:0/3514514695 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f32e8009e80 con 0x7f32f40b0a50 2023-10-18T08:56:52.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.864+0000 7f32e6ffd640 1 -- 172.21.15.40:0/3514514695 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e80106a0 con 0x7f32f40b0a50 2023-10-18T08:56:52.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.864+0000 7f32fb0da640 1 -- 172.21.15.40:0/3514514695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f40af6c0 con 0x7f32f40b0a50 2023-10-18T08:56:52.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.865+0000 7f32fb0da640 1 -- 172.21.15.40:0/3514514695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f41595a0 con 0x7f32f40b0a50 2023-10-18T08:56:52.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.866+0000 7f32fb0da640 1 -- 172.21.15.40:0/3514514695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f415a080 con 0x7f32f40b0a50 2023-10-18T08:56:52.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.867+0000 7f32e6ffd640 1 -- 172.21.15.40:0/3514514695 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f32e8010800 con 0x7f32f40b0a50 2023-10-18T08:56:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.868+0000 7f32e6ffd640 1 --2- 172.21.15.40:0/3514514695 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f32d806ae40 0x7f32d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.869+0000 7f32e6ffd640 1 -- 172.21.15.40:0/3514514695 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32e8089a80 con 0x7f32f40b0a50 2023-10-18T08:56:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.869+0000 7f32fa0d8640 1 --2- 172.21.15.40:0/3514514695 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f32d806ae40 0x7f32d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:52.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.870+0000 7f32fa0d8640 1 --2- 172.21.15.40:0/3514514695 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f32d806ae40 0x7f32d806d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f32f0006fd0 tx=0x7f32f0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:52.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:52.874+0000 7f32e6ffd640 1 -- 172.21.15.40:0/3514514695 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32f415a080 con 0x7f32f40b0a50 2023-10-18T08:56:53.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.067+0000 7f32fb0da640 1 -- 172.21.15.40:0/3514514695 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32f415a080 con 0x7f32f40b0a50 2023-10-18T08:56:53.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.068+0000 7f32e6ffd640 1 -- 172.21.15.40:0/3514514695 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32f415a080 con 0x7f32f40b0a50 2023-10-18T08:56:53.072 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:56:53.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.071+0000 7f32fb0da640 1 -- 172.21.15.40:0/3514514695 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f32d806ae40 msgr2=0x7f32d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:53.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.071+0000 7f32fb0da640 1 --2- 172.21.15.40:0/3514514695 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f32d806ae40 0x7f32d806d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f32f0006fd0 tx=0x7f32f0008040 comp rx=0 tx=0).stop 2023-10-18T08:56:53.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.071+0000 7f32fb0da640 1 -- 172.21.15.40:0/3514514695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f40b0a50 msgr2=0x7f32f4158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:53.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.071+0000 7f32fb0da640 1 --2- 172.21.15.40:0/3514514695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f40b0a50 0x7f32f4158750 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f32e800d770 tx=0x7f32e800dc30 comp rx=0 tx=0).stop 2023-10-18T08:56:53.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.071+0000 7f32fb0da640 1 -- 172.21.15.40:0/3514514695 shutdown_connections 2023-10-18T08:56:53.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.071+0000 7f32fb0da640 1 --2- 172.21.15.40:0/3514514695 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f32d806ae40 0x7f32d806d300 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.071+0000 7f32fb0da640 1 --2- 172.21.15.40:0/3514514695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f41411b0 0x7f32f4158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.071+0000 7f32fb0da640 1 --2- 172.21.15.40:0/3514514695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32f40b0a50 0x7f32f4158750 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.071+0000 7f32fb0da640 1 --2- 172.21.15.40:0/3514514695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f4070790 0x7f32f413aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.071+0000 7f32fb0da640 1 -- 172.21.15.40:0/3514514695 >> 172.21.15.40:0/3514514695 conn(0x7f32f4069720 msgr2=0x7f32f4141680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:53.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.071+0000 7f32fb0da640 1 -- 172.21.15.40:0/3514514695 shutdown_connections 2023-10-18T08:56:53.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.071+0000 7f32fb0da640 1 -- 172.21.15.40:0/3514514695 wait complete. 2023-10-18T08:56:53.094 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:56:53.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.315+0000 7fd0c3fff640 1 -- 172.21.15.40:0/4237189151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0c415b7e0 msgr2=0x7fd0c40cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:53.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.315+0000 7fd0c3fff640 1 --2- 172.21.15.40:0/4237189151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0c415b7e0 0x7fd0c40cf2a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd0c408b860 tx=0x7fd0b802bda0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.315+0000 7fd0c3fff640 1 -- 172.21.15.40:0/4237189151 shutdown_connections 2023-10-18T08:56:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.315+0000 7fd0c3fff640 1 --2- 172.21.15.40:0/4237189151 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0c415b7e0 0x7fd0c40cf2a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.315+0000 7fd0c3fff640 1 --2- 172.21.15.40:0/4237189151 >> v2:172.21.15.144:3300/0 conn(0x7fd0c415b400 0x7fd0c40b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.315+0000 7fd0c3fff640 1 -- 172.21.15.40:0/4237189151 >> 172.21.15.40:0/4237189151 conn(0x7fd0c408ad30 msgr2=0x7fd0c409a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:53.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.316+0000 7fd0c3fff640 1 -- 172.21.15.40:0/4237189151 shutdown_connections 2023-10-18T08:56:53.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.316+0000 7fd0c3fff640 1 -- 172.21.15.40:0/4237189151 wait complete. 2023-10-18T08:56:53.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.317+0000 7fd0c3fff640 1 Processor -- start 2023-10-18T08:56:53.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.317+0000 7fd0c3fff640 1 -- start start 2023-10-18T08:56:53.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.318+0000 7fd0c3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c40cdbe0 0x7fd0c40af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:53.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.318+0000 7fd0c3fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c415b400 0x7fd0c415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:53.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.318+0000 7fd0c3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0c415b7e0 0x7fd0c41f7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:53.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.318+0000 7fd0c3fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c41591f0 con 0x7fd0c415b7e0 2023-10-18T08:56:53.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.318+0000 7fd0c3fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c415c140 con 0x7fd0c40cdbe0 2023-10-18T08:56:53.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.318+0000 7fd0c3fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c415c420 con 0x7fd0c415b400 2023-10-18T08:56:53.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.319+0000 7fd0c37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0c415b7e0 0x7fd0c41f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:53.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.319+0000 7fd0c37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0c415b7e0 0x7fd0c41f7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41866/0 (socket says 172.21.15.40:41866) 2023-10-18T08:56:53.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.319+0000 7fd0c37fe640 1 -- 172.21.15.40:0/2285308702 learned_addr learned my addr 172.21.15.40:0/2285308702 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.319+0000 7fd0c27fc640 1 --2- 172.21.15.40:0/2285308702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c415b400 0x7fd0c415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.319+0000 7fd0c37fe640 1 -- 172.21.15.40:0/2285308702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c415b400 msgr2=0x7fd0c415e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.319+0000 7fd0c37fe640 1 --2- 172.21.15.40:0/2285308702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c415b400 0x7fd0c415e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.319+0000 7fd0c37fe640 1 -- 172.21.15.40:0/2285308702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c40cdbe0 msgr2=0x7fd0c40af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:53.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.319+0000 7fd0c37fe640 1 --2- 172.21.15.40:0/2285308702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c40cdbe0 0x7fd0c40af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.319+0000 7fd0c37fe640 1 -- 172.21.15.40:0/2285308702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b8009560 con 0x7fd0c415b7e0 2023-10-18T08:56:53.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.319+0000 7fd0c37fe640 1 --2- 172.21.15.40:0/2285308702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0c415b7e0 0x7fd0c41f7370 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00c780 tx=0x7fd0bc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:53.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.320+0000 7fd0ab7fe640 1 -- 172.21.15.40:0/2285308702 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc004b60 con 0x7fd0c415b7e0 2023-10-18T08:56:53.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.320+0000 7fd0ab7fe640 1 -- 172.21.15.40:0/2285308702 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd0bc00ce90 con 0x7fd0c415b7e0 2023-10-18T08:56:53.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.320+0000 7fd0ab7fe640 1 -- 172.21.15.40:0/2285308702 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc00aab0 con 0x7fd0c415b7e0 2023-10-18T08:56:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.320+0000 7fd0c3fff640 1 -- 172.21.15.40:0/2285308702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c415c700 con 0x7fd0c415b7e0 2023-10-18T08:56:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.320+0000 7fd0c3fff640 1 -- 172.21.15.40:0/2285308702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c415c9e0 con 0x7fd0c415b7e0 2023-10-18T08:56:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.321+0000 7fd0c3fff640 1 -- 172.21.15.40:0/2285308702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c4071da0 con 0x7fd0c415b7e0 2023-10-18T08:56:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.322+0000 7fd0ab7fe640 1 -- 172.21.15.40:0/2285308702 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd0bc00ac10 con 0x7fd0c415b7e0 2023-10-18T08:56:53.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.324+0000 7fd0ab7fe640 1 --2- 172.21.15.40:0/2285308702 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd0b006ae40 0x7fd0b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:53.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.324+0000 7fd0ab7fe640 1 -- 172.21.15.40:0/2285308702 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0bc089b10 con 0x7fd0c415b7e0 2023-10-18T08:56:53.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.324+0000 7fd0c2ffd640 1 --2- 172.21.15.40:0/2285308702 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd0b006ae40 0x7fd0b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:53.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.325+0000 7fd0c2ffd640 1 --2- 172.21.15.40:0/2285308702 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd0b006ae40 0x7fd0b006d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400a990 tx=0x7fd0b4005c70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:53.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.328+0000 7fd0ab7fe640 1 -- 172.21.15.40:0/2285308702 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd0bc054b20 con 0x7fd0c415b7e0 2023-10-18T08:56:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.526+0000 7fd0c3fff640 1 -- 172.21.15.40:0/2285308702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd0c4066e00 con 0x7fd0c415b7e0 2023-10-18T08:56:53.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.526+0000 7fd0ab7fe640 1 -- 172.21.15.40:0/2285308702 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd0bc058140 con 0x7fd0c415b7e0 2023-10-18T08:56:53.530 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:56:53.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.529+0000 7fd0c3fff640 1 -- 172.21.15.40:0/2285308702 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd0b006ae40 msgr2=0x7fd0b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:53.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.529+0000 7fd0c3fff640 1 --2- 172.21.15.40:0/2285308702 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd0b006ae40 0x7fd0b006d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400a990 tx=0x7fd0b4005c70 comp rx=0 tx=0).stop 2023-10-18T08:56:53.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.529+0000 7fd0c3fff640 1 -- 172.21.15.40:0/2285308702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0c415b7e0 msgr2=0x7fd0c41f7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:53.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.529+0000 7fd0c3fff640 1 --2- 172.21.15.40:0/2285308702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0c415b7e0 0x7fd0c41f7370 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00c780 tx=0x7fd0bc00cc40 comp rx=0 tx=0).stop 2023-10-18T08:56:53.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.529+0000 7fd0c3fff640 1 -- 172.21.15.40:0/2285308702 shutdown_connections 2023-10-18T08:56:53.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.529+0000 7fd0c3fff640 1 --2- 172.21.15.40:0/2285308702 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd0b006ae40 0x7fd0b006d300 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.529+0000 7fd0c3fff640 1 --2- 172.21.15.40:0/2285308702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0c415b7e0 0x7fd0c41f7370 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.529+0000 7fd0c3fff640 1 --2- 172.21.15.40:0/2285308702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0c415b400 0x7fd0c415e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.529+0000 7fd0c3fff640 1 --2- 172.21.15.40:0/2285308702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0c40cdbe0 0x7fd0c40af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.529+0000 7fd0c3fff640 1 -- 172.21.15.40:0/2285308702 >> 172.21.15.40:0/2285308702 conn(0x7fd0c408ad30 msgr2=0x7fd0c415aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:53.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.529+0000 7fd0c3fff640 1 -- 172.21.15.40:0/2285308702 shutdown_connections 2023-10-18T08:56:53.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.529+0000 7fd0c3fff640 1 -- 172.21.15.40:0/2285308702 wait complete. 2023-10-18T08:56:53.551 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:56:53.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.782+0000 7f5c0aeea640 1 -- 172.21.15.40:0/1938032766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c041589c0 msgr2=0x7f5c040aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:53.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.782+0000 7f5c0aeea640 1 --2- 172.21.15.40:0/1938032766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c041589c0 0x7f5c040aec00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009830 tx=0x7f5bf402bd50 comp rx=0 tx=0).stop 2023-10-18T08:56:53.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.782+0000 7f5c0aeea640 1 -- 172.21.15.40:0/1938032766 shutdown_connections 2023-10-18T08:56:53.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.782+0000 7f5c0aeea640 1 --2- 172.21.15.40:0/1938032766 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c041589c0 0x7f5c040aec00 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.782+0000 7f5c0aeea640 1 --2- 172.21.15.40:0/1938032766 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c041585e0 0x7f5c0406f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.782+0000 7f5c0aeea640 1 -- 172.21.15.40:0/1938032766 >> 172.21.15.40:0/1938032766 conn(0x7f5c0406ca10 msgr2=0x7f5c0406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:53.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.782+0000 7f5c0aeea640 1 -- 172.21.15.40:0/1938032766 shutdown_connections 2023-10-18T08:56:53.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.782+0000 7f5c0aeea640 1 -- 172.21.15.40:0/1938032766 wait complete. 2023-10-18T08:56:53.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.784+0000 7f5c0aeea640 1 Processor -- start 2023-10-18T08:56:53.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.784+0000 7f5c0aeea640 1 -- start start 2023-10-18T08:56:53.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.785+0000 7f5c0aeea640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c040af140 0x7f5c041f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:53.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.785+0000 7f5c0aeea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c041585e0 0x7f5c0414eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:53.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.785+0000 7f5c0aeea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c041589c0 0x7f5c0414c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:53.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.785+0000 7f5c0aeea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c041f5560 con 0x7f5c041589c0 2023-10-18T08:56:53.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.785+0000 7f5c0aeea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0414f470 con 0x7f5c041585e0 2023-10-18T08:56:53.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.785+0000 7f5c0aeea640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0414f750 con 0x7f5c040af140 2023-10-18T08:56:53.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.785+0000 7f5c0a6e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c041589c0 0x7f5c0414c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:53.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.785+0000 7f5c0a6e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c041589c0 0x7f5c0414c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41888/0 (socket says 172.21.15.40:41888) 2023-10-18T08:56:53.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.785+0000 7f5c0a6e9640 1 -- 172.21.15.40:0/3789399049 learned_addr learned my addr 172.21.15.40:0/3789399049 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:53.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.786+0000 7f5c09ee8640 1 --2- 172.21.15.40:0/3789399049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c040af140 0x7f5c041f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:53.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.786+0000 7f5c096e7640 1 --2- 172.21.15.40:0/3789399049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c041585e0 0x7f5c0414eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:53.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.786+0000 7f5c0a6e9640 1 -- 172.21.15.40:0/3789399049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c040af140 msgr2=0x7f5c041f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.786+0000 7f5c0a6e9640 1 --2- 172.21.15.40:0/3789399049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c040af140 0x7f5c041f5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.786+0000 7f5c0a6e9640 1 -- 172.21.15.40:0/3789399049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c041585e0 msgr2=0x7f5c0414eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.786+0000 7f5c0a6e9640 1 --2- 172.21.15.40:0/3789399049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c041585e0 0x7f5c0414eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.786+0000 7f5c0a6e9640 1 -- 172.21.15.40:0/3789399049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf4009490 con 0x7f5c041589c0 2023-10-18T08:56:53.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.786+0000 7f5c096e7640 1 --2- 172.21.15.40:0/3789399049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c041585e0 0x7f5c0414eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.786+0000 7f5c09ee8640 1 --2- 172.21.15.40:0/3789399049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c040af140 0x7f5c041f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.786+0000 7f5c0a6e9640 1 --2- 172.21.15.40:0/3789399049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c041589c0 0x7f5c0414c0c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc001560 tx=0x7f5bfc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.786+0000 7f5bfa7fc640 1 -- 172.21.15.40:0/3789399049 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc003da0 con 0x7f5c041589c0 2023-10-18T08:56:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.786+0000 7f5bfa7fc640 1 -- 172.21.15.40:0/3789399049 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5bfc005070 con 0x7f5c041589c0 2023-10-18T08:56:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.786+0000 7f5bfa7fc640 1 -- 172.21.15.40:0/3789399049 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc005460 con 0x7f5c041589c0 2023-10-18T08:56:53.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.786+0000 7f5c0aeea640 1 -- 172.21.15.40:0/3789399049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c041f57b0 con 0x7f5c041589c0 2023-10-18T08:56:53.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.787+0000 7f5c0aeea640 1 -- 172.21.15.40:0/3789399049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0414c600 con 0x7f5c041589c0 2023-10-18T08:56:53.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.788+0000 7f5c0aeea640 1 -- 172.21.15.40:0/3789399049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c041f5cd0 con 0x7f5c041589c0 2023-10-18T08:56:53.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.789+0000 7f5bfa7fc640 1 -- 172.21.15.40:0/3789399049 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5bfc0055c0 con 0x7f5c041589c0 2023-10-18T08:56:53.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.789+0000 7f5bfa7fc640 1 --2- 172.21.15.40:0/3789399049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5be806ae40 0x7f5be806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:53.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.790+0000 7f5bfa7fc640 1 -- 172.21.15.40:0/3789399049 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bfc089140 con 0x7f5c041589c0 2023-10-18T08:56:53.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.790+0000 7f5c09ee8640 1 --2- 172.21.15.40:0/3789399049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5be806ae40 0x7f5be806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:53.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.791+0000 7f5c09ee8640 1 --2- 172.21.15.40:0/3789399049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5be806ae40 0x7f5be806d300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009b40 tx=0x7f5c00005e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:53.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.794+0000 7f5bfa7fc640 1 -- 172.21.15.40:0/3789399049 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c041f5cd0 con 0x7f5c041589c0 2023-10-18T08:56:53.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.989+0000 7f5c0aeea640 1 -- 172.21.15.40:0/3789399049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c041f5cd0 con 0x7f5c041589c0 2023-10-18T08:56:53.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.989+0000 7f5bfa7fc640 1 -- 172.21.15.40:0/3789399049 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5bfc017020 con 0x7f5c041589c0 2023-10-18T08:56:53.993 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:56:53.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.992+0000 7f5c0aeea640 1 -- 172.21.15.40:0/3789399049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5be806ae40 msgr2=0x7f5be806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:53.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.992+0000 7f5c0aeea640 1 --2- 172.21.15.40:0/3789399049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5be806ae40 0x7f5be806d300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009b40 tx=0x7f5c00005e50 comp rx=0 tx=0).stop 2023-10-18T08:56:53.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.992+0000 7f5c0aeea640 1 -- 172.21.15.40:0/3789399049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c041589c0 msgr2=0x7f5c0414c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:53.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.992+0000 7f5c0aeea640 1 --2- 172.21.15.40:0/3789399049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c041589c0 0x7f5c0414c0c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc001560 tx=0x7f5bfc00cc70 comp rx=0 tx=0).stop 2023-10-18T08:56:53.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.992+0000 7f5c0aeea640 1 -- 172.21.15.40:0/3789399049 shutdown_connections 2023-10-18T08:56:53.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.992+0000 7f5c0aeea640 1 --2- 172.21.15.40:0/3789399049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5be806ae40 0x7f5be806d300 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.992+0000 7f5c0aeea640 1 --2- 172.21.15.40:0/3789399049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5c041589c0 0x7f5c0414c0c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.992+0000 7f5c0aeea640 1 --2- 172.21.15.40:0/3789399049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c041585e0 0x7f5c0414eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.992+0000 7f5c0aeea640 1 --2- 172.21.15.40:0/3789399049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c040af140 0x7f5c041f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:53.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.992+0000 7f5c0aeea640 1 -- 172.21.15.40:0/3789399049 >> 172.21.15.40:0/3789399049 conn(0x7f5c0406ca10 msgr2=0x7f5c04069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:53.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.992+0000 7f5c0aeea640 1 -- 172.21.15.40:0/3789399049 shutdown_connections 2023-10-18T08:56:53.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:53.992+0000 7f5c0aeea640 1 -- 172.21.15.40:0/3789399049 wait complete. 2023-10-18T08:56:54.016 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T08:56:54.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.238+0000 7f699359e640 1 -- 172.21.15.40:0/2654023216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69940d1740 msgr2=0x7f699406a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:54.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.238+0000 7f699359e640 1 --2- 172.21.15.40:0/2654023216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69940d1740 0x7f699406a5a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6994069e10 tx=0x7f697c02c260 comp rx=0 tx=0).stop 2023-10-18T08:56:54.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.238+0000 7f699359e640 1 -- 172.21.15.40:0/2654023216 shutdown_connections 2023-10-18T08:56:54.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.238+0000 7f699359e640 1 --2- 172.21.15.40:0/2654023216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f699406aae0 0x7f699415f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:54.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.238+0000 7f699359e640 1 --2- 172.21.15.40:0/2654023216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69940d1740 0x7f699406a5a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:54.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.238+0000 7f699359e640 1 --2- 172.21.15.40:0/2654023216 >> v2:172.21.15.144:3300/0 conn(0x7f699415b530 0x7f69940d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:54.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.238+0000 7f699359e640 1 -- 172.21.15.40:0/2654023216 >> 172.21.15.40:0/2654023216 conn(0x7f699408ac60 msgr2=0x7f699409a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:54.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.238+0000 7f699359e640 1 -- 172.21.15.40:0/2654023216 shutdown_connections 2023-10-18T08:56:54.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.238+0000 7f699359e640 1 -- 172.21.15.40:0/2654023216 wait complete. 2023-10-18T08:56:54.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.240+0000 7f699359e640 1 Processor -- start 2023-10-18T08:56:54.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.240+0000 7f699359e640 1 -- start start 2023-10-18T08:56:54.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.241+0000 7f699359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f699406aae0 0x7f69940b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:54.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.241+0000 7f699359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69940d1740 0x7f69940b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:54.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.241+0000 7f699359e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f699415b530 0x7f69940af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:54.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.241+0000 7f699359e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69940b71f0 con 0x7f699406aae0 2023-10-18T08:56:54.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.241+0000 7f699359e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69940b5c20 con 0x7f69940d1740 2023-10-18T08:56:54.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.241+0000 7f699359e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69940b4fd0 con 0x7f699415b530 2023-10-18T08:56:54.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.241+0000 7f699259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f699406aae0 0x7f69940b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:54.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.241+0000 7f699259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f699406aae0 0x7f69940b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41900/0 (socket says 172.21.15.40:41900) 2023-10-18T08:56:54.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.241+0000 7f699259c640 1 -- 172.21.15.40:0/1770755568 learned_addr learned my addr 172.21.15.40:0/1770755568 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:54.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.242+0000 7f699259c640 1 -- 172.21.15.40:0/1770755568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f699415b530 msgr2=0x7f69940af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:54.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.242+0000 7f699259c640 1 --2- 172.21.15.40:0/1770755568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f699415b530 0x7f69940af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:54.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.242+0000 7f699259c640 1 -- 172.21.15.40:0/1770755568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69940d1740 msgr2=0x7f69940b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:54.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.242+0000 7f699259c640 1 --2- 172.21.15.40:0/1770755568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69940d1740 0x7f69940b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:54.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.242+0000 7f699259c640 1 -- 172.21.15.40:0/1770755568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c009900 con 0x7f699406aae0 2023-10-18T08:56:54.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.242+0000 7f699259c640 1 --2- 172.21.15.40:0/1770755568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f699406aae0 0x7f69940b4a60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f698400b410 tx=0x7f698400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.242+0000 7f698b7fe640 1 -- 172.21.15.40:0/1770755568 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6984011830 con 0x7f699406aae0 2023-10-18T08:56:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.242+0000 7f698b7fe640 1 -- 172.21.15.40:0/1770755568 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6984011aa0 con 0x7f699406aae0 2023-10-18T08:56:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.242+0000 7f699359e640 1 -- 172.21.15.40:0/1770755568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69940b6f10 con 0x7f699406aae0 2023-10-18T08:56:54.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.243+0000 7f698b7fe640 1 -- 172.21.15.40:0/1770755568 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6984010380 con 0x7f699406aae0 2023-10-18T08:56:54.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.243+0000 7f699359e640 1 -- 172.21.15.40:0/1770755568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69940b19c0 con 0x7f699406aae0 2023-10-18T08:56:54.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.244+0000 7f699359e640 1 -- 172.21.15.40:0/1770755568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69940b1bb0 con 0x7f699406aae0 2023-10-18T08:56:54.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.245+0000 7f698b7fe640 1 -- 172.21.15.40:0/1770755568 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f69840104e0 con 0x7f699406aae0 2023-10-18T08:56:54.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.246+0000 7f698b7fe640 1 --2- 172.21.15.40:0/1770755568 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f697406ae40 0x7f697406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:54.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.246+0000 7f698b7fe640 1 -- 172.21.15.40:0/1770755568 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69840899c0 con 0x7f699406aae0 2023-10-18T08:56:54.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.246+0000 7f6989d9b640 1 --2- 172.21.15.40:0/1770755568 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f697406ae40 0x7f697406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:54.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.247+0000 7f6989d9b640 1 --2- 172.21.15.40:0/1770755568 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f697406ae40 0x7f697406d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f697c036000 tx=0x7f697c004a40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:54.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.250+0000 7f698b7fe640 1 -- 172.21.15.40:0/1770755568 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6984054a60 con 0x7f699406aae0 2023-10-18T08:56:54.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.456+0000 7f699359e640 1 -- 172.21.15.40:0/1770755568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6994066e00 con 0x7f699406aae0 2023-10-18T08:56:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.457+0000 7f698b7fe640 1 -- 172.21.15.40:0/1770755568 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6984015080 con 0x7f699406aae0 2023-10-18T08:56:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T08:56:54.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.460+0000 7f699359e640 1 -- 172.21.15.40:0/1770755568 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f697406ae40 msgr2=0x7f697406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:54.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.460+0000 7f699359e640 1 --2- 172.21.15.40:0/1770755568 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f697406ae40 0x7f697406d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f697c036000 tx=0x7f697c004a40 comp rx=0 tx=0).stop 2023-10-18T08:56:54.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.460+0000 7f699359e640 1 -- 172.21.15.40:0/1770755568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f699406aae0 msgr2=0x7f69940b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:54.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.460+0000 7f699359e640 1 --2- 172.21.15.40:0/1770755568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f699406aae0 0x7f69940b4a60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f698400b410 tx=0x7f698400b8d0 comp rx=0 tx=0).stop 2023-10-18T08:56:54.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.460+0000 7f699359e640 1 -- 172.21.15.40:0/1770755568 shutdown_connections 2023-10-18T08:56:54.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.460+0000 7f699359e640 1 --2- 172.21.15.40:0/1770755568 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f697406ae40 0x7f697406d300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:54.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.460+0000 7f699359e640 1 --2- 172.21.15.40:0/1770755568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f699415b530 0x7f69940af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:54.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.460+0000 7f699359e640 1 --2- 172.21.15.40:0/1770755568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69940d1740 0x7f69940b6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:54.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.460+0000 7f699359e640 1 --2- 172.21.15.40:0/1770755568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f699406aae0 0x7f69940b4a60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:54.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.460+0000 7f699359e640 1 -- 172.21.15.40:0/1770755568 >> 172.21.15.40:0/1770755568 conn(0x7f699408ac60 msgr2=0x7f699415dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:54.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.460+0000 7f699359e640 1 -- 172.21.15.40:0/1770755568 shutdown_connections 2023-10-18T08:56:54.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.460+0000 7f699359e640 1 -- 172.21.15.40:0/1770755568 wait complete. 2023-10-18T08:56:54.479 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-18T08:56:54.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:54.480 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-18T08:56:54.586 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T08:56:54.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:56:54.587 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T08:56:54.637 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:56:54.637 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:54.698 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T08:56:54.699 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:56:54.699 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:56:54.758 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T08:56:54.759 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:56:55.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.997+0000 7fbbabfff640 1 -- 172.21.15.40:0/1207344912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbac0b1e10 msgr2=0x7fbbac0c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:55.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.997+0000 7fbbabfff640 1 --2- 172.21.15.40:0/1207344912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbac0b1e10 0x7fbbac0c99c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbbac06af60 tx=0x7fbba002bd80 comp rx=0 tx=0).stop 2023-10-18T08:56:55.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.997+0000 7fbbabfff640 1 -- 172.21.15.40:0/1207344912 shutdown_connections 2023-10-18T08:56:55.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.997+0000 7fbbabfff640 1 --2- 172.21.15.40:0/1207344912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbac0b1e10 0x7fbbac0c99c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.997+0000 7fbbabfff640 1 --2- 172.21.15.40:0/1207344912 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbac0b15b0 0x7fbbac0b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.997+0000 7fbbabfff640 1 -- 172.21.15.40:0/1207344912 >> 172.21.15.40:0/1207344912 conn(0x7fbbac069720 msgr2=0x7fbbac069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:55.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.997+0000 7fbbabfff640 1 -- 172.21.15.40:0/1207344912 shutdown_connections 2023-10-18T08:56:55.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.998+0000 7fbbabfff640 1 -- 172.21.15.40:0/1207344912 wait complete. 2023-10-18T08:56:55.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:54.999+0000 7fbbabfff640 1 Processor -- start 2023-10-18T08:56:55.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.000+0000 7fbbabfff640 1 -- start start 2023-10-18T08:56:55.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.001+0000 7fbbabfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbac0b15b0 0x7fbbac159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:55.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.001+0000 7fbbabfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac0b1e10 0x7fbbac1585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:55.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.001+0000 7fbbabfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbac0b2270 0x7fbbac158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:55.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.001+0000 7fbbabfff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac15a090 con 0x7fbbac0b2270 2023-10-18T08:56:55.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.001+0000 7fbbabfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac14d6d0 con 0x7fbbac0b1e10 2023-10-18T08:56:55.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.001+0000 7fbbabfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac14d920 con 0x7fbbac0b15b0 2023-10-18T08:56:55.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.001+0000 7fbbab7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbac0b2270 0x7fbbac158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:55.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.001+0000 7fbbab7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbac0b2270 0x7fbbac158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33602/0 (socket says 172.21.15.40:33602) 2023-10-18T08:56:55.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.001+0000 7fbbab7fe640 1 -- 172.21.15.40:0/653573942 learned_addr learned my addr 172.21.15.40:0/653573942 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:55.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.001+0000 7fbbaa7fc640 1 --2- 172.21.15.40:0/653573942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac0b1e10 0x7fbbac1585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:55.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.002+0000 7fbbaaffd640 1 --2- 172.21.15.40:0/653573942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbac0b15b0 0x7fbbac159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:55.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.002+0000 7fbbab7fe640 1 -- 172.21.15.40:0/653573942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbac0b15b0 msgr2=0x7fbbac159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:55.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.002+0000 7fbbab7fe640 1 --2- 172.21.15.40:0/653573942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbac0b15b0 0x7fbbac159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.002+0000 7fbbab7fe640 1 -- 172.21.15.40:0/653573942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac0b1e10 msgr2=0x7fbbac1585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:55.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.002+0000 7fbbab7fe640 1 --2- 172.21.15.40:0/653573942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac0b1e10 0x7fbbac1585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.002+0000 7fbbab7fe640 1 -- 172.21.15.40:0/653573942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba0009560 con 0x7fbbac0b2270 2023-10-18T08:56:55.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.002+0000 7fbbaaffd640 1 --2- 172.21.15.40:0/653573942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbac0b15b0 0x7fbbac159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:55.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.002+0000 7fbbab7fe640 1 --2- 172.21.15.40:0/653573942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbac0b2270 0x7fbbac158b30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbba400b9e0 tx=0x7fbba400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:55.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.002+0000 7fbb93fff640 1 -- 172.21.15.40:0/653573942 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400cb20 con 0x7fbbac0b2270 2023-10-18T08:56:55.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.002+0000 7fbbabfff640 1 -- 172.21.15.40:0/653573942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbac14dc00 con 0x7fbbac0b2270 2023-10-18T08:56:55.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.002+0000 7fbb93fff640 1 -- 172.21.15.40:0/653573942 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbba400cd70 con 0x7fbbac0b2270 2023-10-18T08:56:55.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.002+0000 7fbb93fff640 1 -- 172.21.15.40:0/653573942 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba40146a0 con 0x7fbbac0b2270 2023-10-18T08:56:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.002+0000 7fbbabfff640 1 -- 172.21.15.40:0/653573942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbac14e1c0 con 0x7fbbac0b2270 2023-10-18T08:56:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.004+0000 7fbbabfff640 1 -- 172.21.15.40:0/653573942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbac14e3e0 con 0x7fbbac0b2270 2023-10-18T08:56:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.005+0000 7fbb93fff640 1 -- 172.21.15.40:0/653573942 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbba401d3b0 con 0x7fbbac0b2270 2023-10-18T08:56:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.006+0000 7fbb93fff640 1 --2- 172.21.15.40:0/653573942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb9806ae40 0x7fbb9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:55.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.006+0000 7fbb93fff640 1 -- 172.21.15.40:0/653573942 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbba40898e0 con 0x7fbbac0b2270 2023-10-18T08:56:55.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.006+0000 7fbbaaffd640 1 --2- 172.21.15.40:0/653573942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb9806ae40 0x7fbb9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:55.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.007+0000 7fbbaaffd640 1 --2- 172.21.15.40:0/653573942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb9806ae40 0x7fbb9806d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0058c0 tx=0x7fbb9c005d50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:55.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.013+0000 7fbb93fff640 1 -- 172.21.15.40:0/653573942 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbba4054870 con 0x7fbbac0b2270 2023-10-18T08:56:55.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.245+0000 7fbbabfff640 1 -- 172.21.15.40:0/653573942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbac072100 con 0x7fbbac0b2270 2023-10-18T08:56:55.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.246+0000 7fbb93fff640 1 -- 172.21.15.40:0/653573942 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fbba4057e90 con 0x7fbbac0b2270 2023-10-18T08:56:55.250 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:55.250 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/837790640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":837790640},{"type":"v1","addr":"172.21.15.40:6833","nonce":837790640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3377302775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3377302775},{"type":"v1","addr":"172.21.15.144:6836","nonce":3377302775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4939,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3162305748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3162305748},{"type":"v1","addr":"172.21.15.144:6837","nonce":3162305748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:56:32.203221+0000","modified":"2023-10-18T08:56:35.403466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5190":{"gid":5190,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3167454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3167454356},{"type":"v1","addr":"172.21.15.40:6835","nonce":3167454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T08:56:55.251 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2023-10-18T08:56:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.249+0000 7fbbabfff640 1 -- 172.21.15.40:0/653573942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb9806ae40 msgr2=0x7fbb9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.249+0000 7fbbabfff640 1 --2- 172.21.15.40:0/653573942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb9806ae40 0x7fbb9806d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0058c0 tx=0x7fbb9c005d50 comp rx=0 tx=0).stop 2023-10-18T08:56:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.249+0000 7fbbabfff640 1 -- 172.21.15.40:0/653573942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbac0b2270 msgr2=0x7fbbac158b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.249+0000 7fbbabfff640 1 --2- 172.21.15.40:0/653573942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbac0b2270 0x7fbbac158b30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbba400b9e0 tx=0x7fbba400bea0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.249+0000 7fbbabfff640 1 -- 172.21.15.40:0/653573942 shutdown_connections 2023-10-18T08:56:55.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.249+0000 7fbbabfff640 1 --2- 172.21.15.40:0/653573942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb9806ae40 0x7fbb9806d300 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.249+0000 7fbbabfff640 1 --2- 172.21.15.40:0/653573942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbac0b2270 0x7fbbac158b30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.249+0000 7fbbabfff640 1 --2- 172.21.15.40:0/653573942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbac0b1e10 0x7fbbac1585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.249+0000 7fbbabfff640 1 --2- 172.21.15.40:0/653573942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbac0b15b0 0x7fbbac159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.249+0000 7fbbabfff640 1 -- 172.21.15.40:0/653573942 >> 172.21.15.40:0/653573942 conn(0x7fbbac069720 msgr2=0x7fbbac0aa970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:55.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.250+0000 7fbbabfff640 1 -- 172.21.15.40:0/653573942 shutdown_connections 2023-10-18T08:56:55.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.250+0000 7fbbabfff640 1 -- 172.21.15.40:0/653573942 wait complete. 2023-10-18T08:56:55.272 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:55.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.497+0000 7fd04b1c4640 1 -- 172.21.15.40:0/649884640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0440ca160 msgr2=0x7fd0440b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:55.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.497+0000 7fd04b1c4640 1 --2- 172.21.15.40:0/649884640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0440ca160 0x7fd0440b0a80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd034005fd0 tx=0x7fd03402bd90 comp rx=0 tx=0).stop 2023-10-18T08:56:55.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.497+0000 7fd04b1c4640 1 -- 172.21.15.40:0/649884640 shutdown_connections 2023-10-18T08:56:55.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.497+0000 7fd04b1c4640 1 --2- 172.21.15.40:0/649884640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0441425a0 0x7fd0440cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.497+0000 7fd04b1c4640 1 --2- 172.21.15.40:0/649884640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0440ca160 0x7fd0440b0a80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.497+0000 7fd04b1c4640 1 -- 172.21.15.40:0/649884640 >> 172.21.15.40:0/649884640 conn(0x7fd044069720 msgr2=0x7fd044069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:55.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.497+0000 7fd04b1c4640 1 -- 172.21.15.40:0/649884640 shutdown_connections 2023-10-18T08:56:55.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.497+0000 7fd04b1c4640 1 -- 172.21.15.40:0/649884640 wait complete. 2023-10-18T08:56:55.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.499+0000 7fd04b1c4640 1 Processor -- start 2023-10-18T08:56:55.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.499+0000 7fd04b1c4640 1 -- start start 2023-10-18T08:56:55.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.501+0000 7fd04b1c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0440ca160 0x7fd0441f0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:55.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.501+0000 7fd04b1c4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0441420f0 0x7fd044140860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:55.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.501+0000 7fd04b1c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0441425a0 0x7fd044140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:55.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.501+0000 7fd04b1c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0441f0b70 con 0x7fd0440ca160 2023-10-18T08:56:55.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.501+0000 7fd04b1c4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd044159c00 con 0x7fd0441425a0 2023-10-18T08:56:55.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.501+0000 7fd04b1c4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd044159ee0 con 0x7fd0441420f0 2023-10-18T08:56:55.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.501+0000 7fd04a1c2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0440ca160 0x7fd0441f0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:55.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.501+0000 7fd04a1c2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0440ca160 0x7fd0441f0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33618/0 (socket says 172.21.15.40:33618) 2023-10-18T08:56:55.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.501+0000 7fd04a1c2640 1 -- 172.21.15.40:0/2091507323 learned_addr learned my addr 172.21.15.40:0/2091507323 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:55.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.501+0000 7fd0499c1640 1 --2- 172.21.15.40:0/2091507323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0441420f0 0x7fd044140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:55.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.501+0000 7fd04a1c2640 1 -- 172.21.15.40:0/2091507323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0441420f0 msgr2=0x7fd044140860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:55.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.501+0000 7fd04a1c2640 1 --2- 172.21.15.40:0/2091507323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0441420f0 0x7fd044140860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.501+0000 7fd04a1c2640 1 -- 172.21.15.40:0/2091507323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0441425a0 msgr2=0x7fd044140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:55.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.501+0000 7fd04a1c2640 1 --2- 172.21.15.40:0/2091507323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0441425a0 0x7fd044140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.501+0000 7fd04a1c2640 1 -- 172.21.15.40:0/2091507323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd034009560 con 0x7fd0440ca160 2023-10-18T08:56:55.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.502+0000 7fd04a1c2640 1 --2- 172.21.15.40:0/2091507323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0440ca160 0x7fd0441f0dd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd04000c430 tx=0x7fd04000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:55.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.502+0000 7fd03affd640 1 -- 172.21.15.40:0/2091507323 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040011790 con 0x7fd0440ca160 2023-10-18T08:56:55.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.502+0000 7fd03affd640 1 -- 172.21.15.40:0/2091507323 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd0400119e0 con 0x7fd0440ca160 2023-10-18T08:56:55.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.502+0000 7fd03affd640 1 -- 172.21.15.40:0/2091507323 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040011e30 con 0x7fd0440ca160 2023-10-18T08:56:55.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.502+0000 7fd04b1c4640 1 -- 172.21.15.40:0/2091507323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04415a1c0 con 0x7fd0440ca160 2023-10-18T08:56:55.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.502+0000 7fd04b1c4640 1 -- 172.21.15.40:0/2091507323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0441f1310 con 0x7fd0440ca160 2023-10-18T08:56:55.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.504+0000 7fd04b1c4640 1 -- 172.21.15.40:0/2091507323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0440c98c0 con 0x7fd0440ca160 2023-10-18T08:56:55.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.504+0000 7fd03affd640 1 -- 172.21.15.40:0/2091507323 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd04001c460 con 0x7fd0440ca160 2023-10-18T08:56:55.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.505+0000 7fd03affd640 1 --2- 172.21.15.40:0/2091507323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd02806ae40 0x7fd02806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:55.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.505+0000 7fd03affd640 1 -- 172.21.15.40:0/2091507323 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd040089890 con 0x7fd0440ca160 2023-10-18T08:56:55.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.506+0000 7fd0499c1640 1 --2- 172.21.15.40:0/2091507323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd02806ae40 0x7fd02806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:55.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.507+0000 7fd0499c1640 1 --2- 172.21.15.40:0/2091507323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd02806ae40 0x7fd02806d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd034035000 tx=0x7fd034004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:55.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.510+0000 7fd03affd640 1 -- 172.21.15.40:0/2091507323 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd040054930 con 0x7fd0440ca160 2023-10-18T08:56:55.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.705+0000 7fd04b1c4640 1 -- 172.21.15.40:0/2091507323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd04415a6e0 con 0x7fd0440ca160 2023-10-18T08:56:55.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.708+0000 7fd03affd640 1 -- 172.21.15.40:0/2091507323 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16581 (secure 0 0 0) 0x7fd04415a6e0 con 0x7fd0440ca160 2023-10-18T08:56:55.712 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:55.712 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":35,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:46.967356+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:56:30.127184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-18T08:56:31.225270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:55.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.711+0000 7fd04b1c4640 1 -- 172.21.15.40:0/2091507323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd02806ae40 msgr2=0x7fd02806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:55.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.711+0000 7fd04b1c4640 1 --2- 172.21.15.40:0/2091507323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd02806ae40 0x7fd02806d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd034035000 tx=0x7fd034004730 comp rx=0 tx=0).stop 2023-10-18T08:56:55.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.712+0000 7fd04b1c4640 1 -- 172.21.15.40:0/2091507323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0440ca160 msgr2=0x7fd0441f0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:55.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.712+0000 7fd04b1c4640 1 --2- 172.21.15.40:0/2091507323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0440ca160 0x7fd0441f0dd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd04000c430 tx=0x7fd04000c8f0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.712+0000 7fd04b1c4640 1 -- 172.21.15.40:0/2091507323 shutdown_connections 2023-10-18T08:56:55.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.712+0000 7fd04b1c4640 1 --2- 172.21.15.40:0/2091507323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd02806ae40 0x7fd02806d300 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.712+0000 7fd04b1c4640 1 --2- 172.21.15.40:0/2091507323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0441425a0 0x7fd044140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.712+0000 7fd04b1c4640 1 --2- 172.21.15.40:0/2091507323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0441420f0 0x7fd044140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.712+0000 7fd04b1c4640 1 --2- 172.21.15.40:0/2091507323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0440ca160 0x7fd0441f0dd0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.712+0000 7fd04b1c4640 1 -- 172.21.15.40:0/2091507323 >> 172.21.15.40:0/2091507323 conn(0x7fd044069720 msgr2=0x7fd0440c9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:55.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.712+0000 7fd04b1c4640 1 -- 172.21.15.40:0/2091507323 shutdown_connections 2023-10-18T08:56:55.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.712+0000 7fd04b1c4640 1 -- 172.21.15.40:0/2091507323 wait complete. 2023-10-18T08:56:55.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:56:30.127184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:56:31.225270+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:56:55.737 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:56:55.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.953+0000 7f0fa51f4640 1 -- 172.21.15.40:0/4119633817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa00701d0 msgr2=0x7f0fa00c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:55.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.953+0000 7f0fa51f4640 1 --2- 172.21.15.40:0/4119633817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa00701d0 0x7f0fa00c98f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00cda50 tx=0x7f0f9802d420 comp rx=0 tx=0).stop 2023-10-18T08:56:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.954+0000 7f0fa51f4640 1 -- 172.21.15.40:0/4119633817 shutdown_connections 2023-10-18T08:56:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.954+0000 7f0fa51f4640 1 --2- 172.21.15.40:0/4119633817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa00701d0 0x7f0fa00c98f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.954+0000 7f0fa51f4640 1 --2- 172.21.15.40:0/4119633817 >> v2:172.21.15.144:3300/0 conn(0x7f0fa0157d40 0x7f0fa006fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.954+0000 7f0fa51f4640 1 -- 172.21.15.40:0/4119633817 >> 172.21.15.40:0/4119633817 conn(0x7f0fa0069720 msgr2=0x7f0fa0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.954+0000 7f0fa51f4640 1 -- 172.21.15.40:0/4119633817 shutdown_connections 2023-10-18T08:56:55.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.954+0000 7f0fa51f4640 1 -- 172.21.15.40:0/4119633817 wait complete. 2023-10-18T08:56:55.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.955+0000 7f0fa51f4640 1 Processor -- start 2023-10-18T08:56:55.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.956+0000 7f0fa51f4640 1 -- start start 2023-10-18T08:56:55.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.956+0000 7f0fa51f4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa00701d0 0x7f0fa014fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:55.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0fa51f4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa00af410 0x7f0fa00ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:55.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0fa51f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa0157d40 0x7f0fa00b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:55.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0fa51f4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa00ade80 con 0x7f0fa00af410 2023-10-18T08:56:55.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0fa51f4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa0150200 con 0x7f0fa0157d40 2023-10-18T08:56:55.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0fa51f4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa00ad270 con 0x7f0fa00701d0 2023-10-18T08:56:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0f97fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa00af410 0x7f0fa00ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0f97fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa00af410 0x7f0fa00ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33648/0 (socket says 172.21.15.40:33648) 2023-10-18T08:56:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0f97fff640 1 -- 172.21.15.40:0/2809297329 learned_addr learned my addr 172.21.15.40:0/2809297329 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0fa49f3640 1 --2- 172.21.15.40:0/2809297329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa0157d40 0x7f0fa00b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:55.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0f9ffff640 1 --2- 172.21.15.40:0/2809297329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa00701d0 0x7f0fa014fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0f97fff640 1 -- 172.21.15.40:0/2809297329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa00701d0 msgr2=0x7f0fa014fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0f97fff640 1 --2- 172.21.15.40:0/2809297329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa00701d0 0x7f0fa014fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0f97fff640 1 -- 172.21.15.40:0/2809297329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa0157d40 msgr2=0x7f0fa00b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0f97fff640 1 --2- 172.21.15.40:0/2809297329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa0157d40 0x7f0fa00b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:55.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0f97fff640 1 -- 172.21.15.40:0/2809297329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9800a950 con 0x7f0fa00af410 2023-10-18T08:56:55.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0f9ffff640 1 --2- 172.21.15.40:0/2809297329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa00701d0 0x7f0fa014fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:55.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.957+0000 7f0fa49f3640 1 --2- 172.21.15.40:0/2809297329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa0157d40 0x7f0fa00b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:55.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.958+0000 7f0f97fff640 1 --2- 172.21.15.40:0/2809297329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa00af410 0x7f0fa00ac860 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800d950 tx=0x7f0f8800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:55.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.958+0000 7f0f9d7fa640 1 -- 172.21.15.40:0/2809297329 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88009c30 con 0x7f0fa00af410 2023-10-18T08:56:55.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.958+0000 7f0f9d7fa640 1 -- 172.21.15.40:0/2809297329 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0f88009e80 con 0x7f0fa00af410 2023-10-18T08:56:55.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.958+0000 7f0f9d7fa640 1 -- 172.21.15.40:0/2809297329 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f880106c0 con 0x7f0fa00af410 2023-10-18T08:56:55.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.958+0000 7f0fa51f4640 1 -- 172.21.15.40:0/2809297329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa00ad550 con 0x7f0fa00af410 2023-10-18T08:56:55.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.958+0000 7f0fa51f4640 1 -- 172.21.15.40:0/2809297329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa00adb10 con 0x7f0fa00af410 2023-10-18T08:56:55.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.959+0000 7f0fa51f4640 1 -- 172.21.15.40:0/2809297329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa01f1600 con 0x7f0fa00af410 2023-10-18T08:56:55.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.960+0000 7f0f9d7fa640 1 -- 172.21.15.40:0/2809297329 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0f88010820 con 0x7f0fa00af410 2023-10-18T08:56:55.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.961+0000 7f0f9d7fa640 1 --2- 172.21.15.40:0/2809297329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f8006ae40 0x7f0f8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:55.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.961+0000 7f0f9d7fa640 1 -- 172.21.15.40:0/2809297329 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f88089c30 con 0x7f0fa00af410 2023-10-18T08:56:55.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.962+0000 7f0f9ffff640 1 --2- 172.21.15.40:0/2809297329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f8006ae40 0x7f0f8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:55.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.963+0000 7f0f9ffff640 1 --2- 172.21.15.40:0/2809297329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f8006ae40 0x7f0f8006d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0f900011e0 tx=0x7f0f90009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:55.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:55.966+0000 7f0f9d7fa640 1 -- 172.21.15.40:0/2809297329 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0f88054bb0 con 0x7f0fa00af410 2023-10-18T08:56:56.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.160+0000 7f0fa51f4640 1 -- 172.21.15.40:0/2809297329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0fa0066e00 con 0x7f0fa00af410 2023-10-18T08:56:56.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.161+0000 7f0f9d7fa640 1 -- 172.21.15.40:0/2809297329 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0f88017020 con 0x7f0fa00af410 2023-10-18T08:56:56.164 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:56:56.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.163+0000 7f0fa51f4640 1 -- 172.21.15.40:0/2809297329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f8006ae40 msgr2=0x7f0f8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:56.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.163+0000 7f0fa51f4640 1 --2- 172.21.15.40:0/2809297329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f8006ae40 0x7f0f8006d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0f900011e0 tx=0x7f0f90009040 comp rx=0 tx=0).stop 2023-10-18T08:56:56.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.163+0000 7f0fa51f4640 1 -- 172.21.15.40:0/2809297329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa00af410 msgr2=0x7f0fa00ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:56.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.163+0000 7f0fa51f4640 1 --2- 172.21.15.40:0/2809297329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa00af410 0x7f0fa00ac860 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800d950 tx=0x7f0f8800de10 comp rx=0 tx=0).stop 2023-10-18T08:56:56.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.164+0000 7f0fa51f4640 1 -- 172.21.15.40:0/2809297329 shutdown_connections 2023-10-18T08:56:56.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.164+0000 7f0fa51f4640 1 --2- 172.21.15.40:0/2809297329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f8006ae40 0x7f0f8006d300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.164+0000 7f0fa51f4640 1 --2- 172.21.15.40:0/2809297329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa0157d40 0x7f0fa00b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.164+0000 7f0fa51f4640 1 --2- 172.21.15.40:0/2809297329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa00af410 0x7f0fa00ac860 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.164+0000 7f0fa51f4640 1 --2- 172.21.15.40:0/2809297329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa00701d0 0x7f0fa014fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.164+0000 7f0fa51f4640 1 -- 172.21.15.40:0/2809297329 >> 172.21.15.40:0/2809297329 conn(0x7f0fa0069720 msgr2=0x7f0fa00b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:56.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.164+0000 7f0fa51f4640 1 -- 172.21.15.40:0/2809297329 shutdown_connections 2023-10-18T08:56:56.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.164+0000 7f0fa51f4640 1 -- 172.21.15.40:0/2809297329 wait complete. 2023-10-18T08:56:56.187 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:56:56.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.419+0000 7f7a0ede9640 1 -- 172.21.15.40:0/4116831116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a08098120 msgr2=0x7f7a080caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:56.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.419+0000 7f7a0ede9640 1 --2- 172.21.15.40:0/4116831116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a08098120 0x7f7a080caa20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7a00005fd0 tx=0x7f7a0002c0e0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.419+0000 7f7a0ede9640 1 -- 172.21.15.40:0/4116831116 shutdown_connections 2023-10-18T08:56:56.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.419+0000 7f7a0ede9640 1 --2- 172.21.15.40:0/4116831116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a08098120 0x7f7a080caa20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.419+0000 7f7a0ede9640 1 -- 172.21.15.40:0/4116831116 >> 172.21.15.40:0/4116831116 conn(0x7f7a080b0ad0 msgr2=0x7f7a080b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:56.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.419+0000 7f7a0ede9640 1 -- 172.21.15.40:0/4116831116 shutdown_connections 2023-10-18T08:56:56.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.419+0000 7f7a0ede9640 1 -- 172.21.15.40:0/4116831116 wait complete. 2023-10-18T08:56:56.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.421+0000 7f7a0ede9640 1 Processor -- start 2023-10-18T08:56:56.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.421+0000 7f7a0ede9640 1 -- start start 2023-10-18T08:56:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.422+0000 7f7a0ede9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a0806a680 0x7f7a080c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.422+0000 7f7a0ede9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a08098120 0x7f7a08140620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.422+0000 7f7a0ede9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0815b5b0 0x7f7a081433c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:56.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.422+0000 7f7a0ede9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a08140b60 con 0x7f7a0806a680 2023-10-18T08:56:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.422+0000 7f7a0ede9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a080c8fa0 con 0x7f7a08098120 2023-10-18T08:56:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.422+0000 7f7a0ede9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a080c7f00 con 0x7f7a0815b5b0 2023-10-18T08:56:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.422+0000 7f7a0dde7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a0806a680 0x7f7a080c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.423+0000 7f7a0dde7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a0806a680 0x7f7a080c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33666/0 (socket says 172.21.15.40:33666) 2023-10-18T08:56:56.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.423+0000 7f7a0dde7640 1 -- 172.21.15.40:0/3255713345 learned_addr learned my addr 172.21.15.40:0/3255713345 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:56.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.423+0000 7f7a0d5e6640 1 --2- 172.21.15.40:0/3255713345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a08098120 0x7f7a08140620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:56.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.423+0000 7f7a0e5e8640 1 --2- 172.21.15.40:0/3255713345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0815b5b0 0x7f7a081433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:56.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.423+0000 7f7a0dde7640 1 -- 172.21.15.40:0/3255713345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0815b5b0 msgr2=0x7f7a081433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:56.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.423+0000 7f7a0dde7640 1 --2- 172.21.15.40:0/3255713345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0815b5b0 0x7f7a081433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.423+0000 7f7a0dde7640 1 -- 172.21.15.40:0/3255713345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a08098120 msgr2=0x7f7a08140620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:56.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.423+0000 7f7a0dde7640 1 --2- 172.21.15.40:0/3255713345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a08098120 0x7f7a08140620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.423+0000 7f7a0dde7640 1 -- 172.21.15.40:0/3255713345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a00009840 con 0x7f7a0806a680 2023-10-18T08:56:56.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.424+0000 7f7a0dde7640 1 --2- 172.21.15.40:0/3255713345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a0806a680 0x7f7a080c7820 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400c410 tx=0x7f7a0400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:56.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.424+0000 7f79fe7fc640 1 -- 172.21.15.40:0/3255713345 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a040035f0 con 0x7f7a0806a680 2023-10-18T08:56:56.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.424+0000 7f79fe7fc640 1 -- 172.21.15.40:0/3255713345 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7a04003770 con 0x7f7a0806a680 2023-10-18T08:56:56.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.424+0000 7f79fe7fc640 1 -- 172.21.15.40:0/3255713345 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04003b80 con 0x7f7a0806a680 2023-10-18T08:56:56.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.424+0000 7f7a0ede9640 1 -- 172.21.15.40:0/3255713345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a080c81e0 con 0x7f7a0806a680 2023-10-18T08:56:56.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.424+0000 7f7a0ede9640 1 -- 172.21.15.40:0/3255713345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a080c87a0 con 0x7f7a0806a680 2023-10-18T08:56:56.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.425+0000 7f7a0ede9640 1 -- 172.21.15.40:0/3255713345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a08073570 con 0x7f7a0806a680 2023-10-18T08:56:56.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.426+0000 7f79fe7fc640 1 -- 172.21.15.40:0/3255713345 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7a04003ce0 con 0x7f7a0806a680 2023-10-18T08:56:56.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.427+0000 7f79fe7fc640 1 --2- 172.21.15.40:0/3255713345 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79ec06ae70 0x7f79ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:56.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.427+0000 7f79fe7fc640 1 -- 172.21.15.40:0/3255713345 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a04088dd0 con 0x7f7a0806a680 2023-10-18T08:56:56.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.428+0000 7f7a0d5e6640 1 --2- 172.21.15.40:0/3255713345 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79ec06ae70 0x7f79ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:56.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.429+0000 7f7a0d5e6640 1 --2- 172.21.15.40:0/3255713345 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79ec06ae70 0x7f79ec06d330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7a08158de0 tx=0x7f79f8007e30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:56.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.431+0000 7f79fe7fc640 1 -- 172.21.15.40:0/3255713345 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7a04053e00 con 0x7f7a0806a680 2023-10-18T08:56:56.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.633+0000 7f7a0ede9640 1 -- 172.21.15.40:0/3255713345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a0815bad0 con 0x7f7a0806a680 2023-10-18T08:56:56.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.634+0000 7f79fe7fc640 1 -- 172.21.15.40:0/3255713345 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a04017070 con 0x7f7a0806a680 2023-10-18T08:56:56.637 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:56:56.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.636+0000 7f7a0ede9640 1 -- 172.21.15.40:0/3255713345 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79ec06ae70 msgr2=0x7f79ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:56.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.636+0000 7f7a0ede9640 1 --2- 172.21.15.40:0/3255713345 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79ec06ae70 0x7f79ec06d330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7a08158de0 tx=0x7f79f8007e30 comp rx=0 tx=0).stop 2023-10-18T08:56:56.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.636+0000 7f7a0ede9640 1 -- 172.21.15.40:0/3255713345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a0806a680 msgr2=0x7f7a080c7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:56.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.636+0000 7f7a0ede9640 1 --2- 172.21.15.40:0/3255713345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a0806a680 0x7f7a080c7820 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400c410 tx=0x7f7a0400c8d0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.636+0000 7f7a0ede9640 1 -- 172.21.15.40:0/3255713345 shutdown_connections 2023-10-18T08:56:56.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.636+0000 7f7a0ede9640 1 --2- 172.21.15.40:0/3255713345 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79ec06ae70 0x7f79ec06d330 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.636+0000 7f7a0ede9640 1 --2- 172.21.15.40:0/3255713345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a0815b5b0 0x7f7a081433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.636+0000 7f7a0ede9640 1 --2- 172.21.15.40:0/3255713345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a08098120 0x7f7a08140620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.636+0000 7f7a0ede9640 1 --2- 172.21.15.40:0/3255713345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a0806a680 0x7f7a080c7820 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.636+0000 7f7a0ede9640 1 -- 172.21.15.40:0/3255713345 >> 172.21.15.40:0/3255713345 conn(0x7f7a080b0ad0 msgr2=0x7f7a0815a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:56.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.637+0000 7f7a0ede9640 1 -- 172.21.15.40:0/3255713345 shutdown_connections 2023-10-18T08:56:56.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.637+0000 7f7a0ede9640 1 -- 172.21.15.40:0/3255713345 wait complete. 2023-10-18T08:56:56.660 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:56:56.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.899+0000 7fdf74f06640 1 -- 172.21.15.40:0/1857800273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf7006d5d0 msgr2=0x7fdf700c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:56.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.899+0000 7fdf74f06640 1 --2- 172.21.15.40:0/1857800273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf7006d5d0 0x7fdf700c9620 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdf60005fd0 tx=0x7fdf6002c1b0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.899+0000 7fdf74f06640 1 -- 172.21.15.40:0/1857800273 shutdown_connections 2023-10-18T08:56:56.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.899+0000 7fdf74f06640 1 --2- 172.21.15.40:0/1857800273 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf7015af70 0x7fdf70070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.899+0000 7fdf74f06640 1 --2- 172.21.15.40:0/1857800273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf7006d5d0 0x7fdf700c9620 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.899+0000 7fdf74f06640 1 -- 172.21.15.40:0/1857800273 >> 172.21.15.40:0/1857800273 conn(0x7fdf7006ca10 msgr2=0x7fdf7006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:56.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.900+0000 7fdf74f06640 1 -- 172.21.15.40:0/1857800273 shutdown_connections 2023-10-18T08:56:56.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.900+0000 7fdf74f06640 1 -- 172.21.15.40:0/1857800273 wait complete. 2023-10-18T08:56:56.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.902+0000 7fdf74f06640 1 Processor -- start 2023-10-18T08:56:56.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.902+0000 7fdf74f06640 1 -- start start 2023-10-18T08:56:56.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.904+0000 7fdf74f06640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf7006d5d0 0x7fdf701f08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:56.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.904+0000 7fdf74f06640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf700710a0 0x7fdf7014dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:56.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.904+0000 7fdf74f06640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf7015af70 0x7fdf700afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.904+0000 7fdf74f06640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf701f0740 con 0x7fdf7015af70 2023-10-18T08:56:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.904+0000 7fdf74f06640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf701f0e30 con 0x7fdf7006d5d0 2023-10-18T08:56:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.904+0000 7fdf74f06640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf701f0460 con 0x7fdf700710a0 2023-10-18T08:56:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.904+0000 7fdf6ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf7015af70 0x7fdf700afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.904+0000 7fdf6ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf7015af70 0x7fdf700afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33686/0 (socket says 172.21.15.40:33686) 2023-10-18T08:56:56.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.904+0000 7fdf6ffff640 1 -- 172.21.15.40:0/3312551828 learned_addr learned my addr 172.21.15.40:0/3312551828 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:56.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.904+0000 7fdf66ffd640 1 --2- 172.21.15.40:0/3312551828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf700710a0 0x7fdf7014dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:56.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.905+0000 7fdf6f7fe640 1 --2- 172.21.15.40:0/3312551828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf7006d5d0 0x7fdf701f08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:56.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.905+0000 7fdf6ffff640 1 -- 172.21.15.40:0/3312551828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf700710a0 msgr2=0x7fdf7014dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:56.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.905+0000 7fdf6ffff640 1 --2- 172.21.15.40:0/3312551828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf700710a0 0x7fdf7014dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.905+0000 7fdf6ffff640 1 -- 172.21.15.40:0/3312551828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf7006d5d0 msgr2=0x7fdf701f08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:56.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.905+0000 7fdf6ffff640 1 --2- 172.21.15.40:0/3312551828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf7006d5d0 0x7fdf701f08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:56.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.905+0000 7fdf6ffff640 1 -- 172.21.15.40:0/3312551828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf60009900 con 0x7fdf7015af70 2023-10-18T08:56:56.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.905+0000 7fdf6f7fe640 1 --2- 172.21.15.40:0/3312551828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf7006d5d0 0x7fdf701f08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:56.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.905+0000 7fdf6ffff640 1 --2- 172.21.15.40:0/3312551828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf7015af70 0x7fdf700afdf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800c440 tx=0x7fdf6800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.905+0000 7fdf6cff9640 1 -- 172.21.15.40:0/3312551828 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf68011790 con 0x7fdf7015af70 2023-10-18T08:56:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.905+0000 7fdf6cff9640 1 -- 172.21.15.40:0/3312551828 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdf680119e0 con 0x7fdf7015af70 2023-10-18T08:56:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.905+0000 7fdf6cff9640 1 -- 172.21.15.40:0/3312551828 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf68011df0 con 0x7fdf7015af70 2023-10-18T08:56:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.905+0000 7fdf74f06640 1 -- 172.21.15.40:0/3312551828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf70150ca0 con 0x7fdf7015af70 2023-10-18T08:56:56.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.906+0000 7fdf74f06640 1 -- 172.21.15.40:0/3312551828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf70151260 con 0x7fdf7015af70 2023-10-18T08:56:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.907+0000 7fdf74f06640 1 -- 172.21.15.40:0/3312551828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf70150b00 con 0x7fdf7015af70 2023-10-18T08:56:56.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.908+0000 7fdf6cff9640 1 -- 172.21.15.40:0/3312551828 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdf6801c460 con 0x7fdf7015af70 2023-10-18T08:56:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.909+0000 7fdf6cff9640 1 --2- 172.21.15.40:0/3312551828 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf5006ae40 0x7fdf5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.909+0000 7fdf6cff9640 1 -- 172.21.15.40:0/3312551828 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf680897e0 con 0x7fdf7015af70 2023-10-18T08:56:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.909+0000 7fdf6f7fe640 1 --2- 172.21.15.40:0/3312551828 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf5006ae40 0x7fdf5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:56.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.911+0000 7fdf6f7fe640 1 --2- 172.21.15.40:0/3312551828 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf5006ae40 0x7fdf5006d300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdf60036000 tx=0x7fdf60004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:56.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:56.915+0000 7fdf6cff9640 1 -- 172.21.15.40:0/3312551828 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf70150b00 con 0x7fdf7015af70 2023-10-18T08:56:57.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.108+0000 7fdf74f06640 1 -- 172.21.15.40:0/3312551828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf70150b00 con 0x7fdf7015af70 2023-10-18T08:56:57.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.109+0000 7fdf6cff9640 1 -- 172.21.15.40:0/3312551828 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf70150b00 con 0x7fdf7015af70 2023-10-18T08:56:57.113 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:56:57.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.112+0000 7fdf74f06640 1 -- 172.21.15.40:0/3312551828 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf5006ae40 msgr2=0x7fdf5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:57.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.112+0000 7fdf74f06640 1 --2- 172.21.15.40:0/3312551828 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf5006ae40 0x7fdf5006d300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdf60036000 tx=0x7fdf60004ab0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.112+0000 7fdf74f06640 1 -- 172.21.15.40:0/3312551828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf7015af70 msgr2=0x7fdf700afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:57.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.112+0000 7fdf74f06640 1 --2- 172.21.15.40:0/3312551828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf7015af70 0x7fdf700afdf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800c440 tx=0x7fdf6800c900 comp rx=0 tx=0).stop 2023-10-18T08:56:57.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.112+0000 7fdf74f06640 1 -- 172.21.15.40:0/3312551828 shutdown_connections 2023-10-18T08:56:57.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.112+0000 7fdf74f06640 1 --2- 172.21.15.40:0/3312551828 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf5006ae40 0x7fdf5006d300 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.112+0000 7fdf74f06640 1 --2- 172.21.15.40:0/3312551828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf7015af70 0x7fdf700afdf0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.112+0000 7fdf74f06640 1 --2- 172.21.15.40:0/3312551828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf700710a0 0x7fdf7014dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.112+0000 7fdf74f06640 1 --2- 172.21.15.40:0/3312551828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf7006d5d0 0x7fdf701f08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.112+0000 7fdf74f06640 1 -- 172.21.15.40:0/3312551828 >> 172.21.15.40:0/3312551828 conn(0x7fdf7006ca10 msgr2=0x7fdf7006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:57.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.112+0000 7fdf74f06640 1 -- 172.21.15.40:0/3312551828 shutdown_connections 2023-10-18T08:56:57.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.112+0000 7fdf74f06640 1 -- 172.21.15.40:0/3312551828 wait complete. 2023-10-18T08:56:57.134 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:56:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.362+0000 7f366cab4640 1 -- 172.21.15.40:0/3448865323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36500039c0 msgr2=0x7f3650003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:57.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.362+0000 7f366cab4640 1 --2- 172.21.15.40:0/3448865323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36500039c0 0x7f3650003de0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3658006ed0 tx=0x7f36500057f0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.363+0000 7f366cab4640 1 -- 172.21.15.40:0/3448865323 shutdown_connections 2023-10-18T08:56:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.363+0000 7f366cab4640 1 --2- 172.21.15.40:0/3448865323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36500039c0 0x7f3650003de0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.363+0000 7f366cab4640 1 -- 172.21.15.40:0/3448865323 >> 172.21.15.40:0/3448865323 conn(0x7f36680afdf0 msgr2=0x7f36680b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.363+0000 7f366cab4640 1 -- 172.21.15.40:0/3448865323 shutdown_connections 2023-10-18T08:56:57.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.363+0000 7f366cab4640 1 -- 172.21.15.40:0/3448865323 wait complete. 2023-10-18T08:56:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.364+0000 7f366cab4640 1 Processor -- start 2023-10-18T08:56:57.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.365+0000 7f366cab4640 1 -- start start 2023-10-18T08:56:57.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.366+0000 7f366cab4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36500039c0 0x7f36680ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:57.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.366+0000 7f366cab4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36681f6190 0x7f366815c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:57.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.366+0000 7f366cab4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36680acbc0 0x7f366815c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:57.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.366+0000 7f366cab4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36681f5d00 con 0x7f36680acbc0 2023-10-18T08:56:57.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.366+0000 7f366cab4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36680ad100 con 0x7f36681f6190 2023-10-18T08:56:57.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.366+0000 7f366cab4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36681f56b0 con 0x7f36500039c0 2023-10-18T08:56:57.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.366+0000 7f3667fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36680acbc0 0x7f366815c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:57.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.366+0000 7f3667fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36680acbc0 0x7f366815c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33696/0 (socket says 172.21.15.40:33696) 2023-10-18T08:56:57.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.366+0000 7f3667fff640 1 -- 172.21.15.40:0/699814189 learned_addr learned my addr 172.21.15.40:0/699814189 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:57.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.366+0000 7f3666ffd640 1 --2- 172.21.15.40:0/699814189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36681f6190 0x7f366815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:57.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.366+0000 7f36677fe640 1 --2- 172.21.15.40:0/699814189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36500039c0 0x7f36680ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:57.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.366+0000 7f3667fff640 1 -- 172.21.15.40:0/699814189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36500039c0 msgr2=0x7f36680ac680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:57.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.367+0000 7f3667fff640 1 --2- 172.21.15.40:0/699814189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36500039c0 0x7f36680ac680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.367+0000 7f3667fff640 1 -- 172.21.15.40:0/699814189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36681f6190 msgr2=0x7f366815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:57.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.367+0000 7f3667fff640 1 --2- 172.21.15.40:0/699814189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36681f6190 0x7f366815c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.367+0000 7f3667fff640 1 -- 172.21.15.40:0/699814189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36681f5990 con 0x7f36680acbc0 2023-10-18T08:56:57.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.367+0000 7f3667fff640 1 --2- 172.21.15.40:0/699814189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36680acbc0 0x7f366815c5c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f36680b11a0 tx=0x7f36600019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:57.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.367+0000 7f364ffff640 1 -- 172.21.15.40:0/699814189 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660011870 con 0x7f36680acbc0 2023-10-18T08:56:57.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.367+0000 7f364ffff640 1 -- 172.21.15.40:0/699814189 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3660011ac0 con 0x7f36680acbc0 2023-10-18T08:56:57.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.367+0000 7f364ffff640 1 -- 172.21.15.40:0/699814189 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660002090 con 0x7f36680acbc0 2023-10-18T08:56:57.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.367+0000 7f366cab4640 1 -- 172.21.15.40:0/699814189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366813db20 con 0x7f36680acbc0 2023-10-18T08:56:57.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.367+0000 7f366cab4640 1 -- 172.21.15.40:0/699814189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36680b13d0 con 0x7f36680acbc0 2023-10-18T08:56:57.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.369+0000 7f366cab4640 1 -- 172.21.15.40:0/699814189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36681f5510 con 0x7f36680acbc0 2023-10-18T08:56:57.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.370+0000 7f364ffff640 1 -- 172.21.15.40:0/699814189 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f366001a400 con 0x7f36680acbc0 2023-10-18T08:56:57.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.370+0000 7f364ffff640 1 --2- 172.21.15.40:0/699814189 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f365406ae40 0x7f365406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:57.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.371+0000 7f364ffff640 1 -- 172.21.15.40:0/699814189 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f366008a340 con 0x7f36680acbc0 2023-10-18T08:56:57.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.371+0000 7f36677fe640 1 --2- 172.21.15.40:0/699814189 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f365406ae40 0x7f365406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:57.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.372+0000 7f36677fe640 1 --2- 172.21.15.40:0/699814189 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f365406ae40 0x7f365406d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3658034000 tx=0x7f3658039040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:57.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.375+0000 7f364ffff640 1 -- 172.21.15.40:0/699814189 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36681f5510 con 0x7f36680acbc0 2023-10-18T08:56:57.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.603+0000 7f366cab4640 1 -- 172.21.15.40:0/699814189 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36681f5510 con 0x7f36680acbc0 2023-10-18T08:56:57.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.604+0000 7f364ffff640 1 -- 172.21.15.40:0/699814189 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f36681f5510 con 0x7f36680acbc0 2023-10-18T08:56:57.607 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:57.608 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/837790640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":837790640},{"type":"v1","addr":"172.21.15.40:6833","nonce":837790640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3377302775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3377302775},{"type":"v1","addr":"172.21.15.144:6836","nonce":3377302775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4939,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3162305748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3162305748},{"type":"v1","addr":"172.21.15.144:6837","nonce":3162305748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:56:32.203221+0000","modified":"2023-10-18T08:56:35.403466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5190":{"gid":5190,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3167454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3167454356},{"type":"v1","addr":"172.21.15.40:6835","nonce":3167454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T08:56:57.608 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2023-10-18T08:56:57.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.606+0000 7f366cab4640 1 -- 172.21.15.40:0/699814189 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f365406ae40 msgr2=0x7f365406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:57.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.606+0000 7f366cab4640 1 --2- 172.21.15.40:0/699814189 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f365406ae40 0x7f365406d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3658034000 tx=0x7f3658039040 comp rx=0 tx=0).stop 2023-10-18T08:56:57.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.606+0000 7f366cab4640 1 -- 172.21.15.40:0/699814189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36680acbc0 msgr2=0x7f366815c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:57.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.606+0000 7f366cab4640 1 --2- 172.21.15.40:0/699814189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36680acbc0 0x7f366815c5c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f36680b11a0 tx=0x7f36600019b0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.607+0000 7f366cab4640 1 -- 172.21.15.40:0/699814189 shutdown_connections 2023-10-18T08:56:57.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.607+0000 7f366cab4640 1 --2- 172.21.15.40:0/699814189 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f365406ae40 0x7f365406d300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.607+0000 7f366cab4640 1 --2- 172.21.15.40:0/699814189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36680acbc0 0x7f366815c5c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.607+0000 7f366cab4640 1 --2- 172.21.15.40:0/699814189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36681f6190 0x7f366815c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.607+0000 7f366cab4640 1 --2- 172.21.15.40:0/699814189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36500039c0 0x7f36680ac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.607+0000 7f366cab4640 1 -- 172.21.15.40:0/699814189 >> 172.21.15.40:0/699814189 conn(0x7f36680afdf0 msgr2=0x7f366815ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:57.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.607+0000 7f366cab4640 1 -- 172.21.15.40:0/699814189 shutdown_connections 2023-10-18T08:56:57.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.607+0000 7f366cab4640 1 -- 172.21.15.40:0/699814189 wait complete. 2023-10-18T08:56:57.628 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:57.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.855+0000 7f33d0f9e640 1 -- 172.21.15.40:0/3319266555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33cc0b2270 msgr2=0x7f33cc0c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:57.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.855+0000 7f33d0f9e640 1 --2- 172.21.15.40:0/3319266555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33cc0b2270 0x7f33cc0c9f00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f33cc06aeb0 tx=0x7f33c402cd90 comp rx=0 tx=0).stop 2023-10-18T08:56:57.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.855+0000 7f33d0f9e640 1 -- 172.21.15.40:0/3319266555 shutdown_connections 2023-10-18T08:56:57.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.855+0000 7f33d0f9e640 1 --2- 172.21.15.40:0/3319266555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33cc0b2270 0x7f33cc0c9f00 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.855+0000 7f33d0f9e640 1 --2- 172.21.15.40:0/3319266555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33cc0b1e10 0x7f33cc0c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.855+0000 7f33d0f9e640 1 --2- 172.21.15.40:0/3319266555 >> v2:172.21.15.144:3300/0 conn(0x7f33cc0b15b0 0x7f33cc0b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.855+0000 7f33d0f9e640 1 -- 172.21.15.40:0/3319266555 >> 172.21.15.40:0/3319266555 conn(0x7f33cc069720 msgr2=0x7f33cc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:57.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.855+0000 7f33d0f9e640 1 -- 172.21.15.40:0/3319266555 shutdown_connections 2023-10-18T08:56:57.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.855+0000 7f33d0f9e640 1 -- 172.21.15.40:0/3319266555 wait complete. 2023-10-18T08:56:57.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.857+0000 7f33d0f9e640 1 Processor -- start 2023-10-18T08:56:57.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.857+0000 7f33d0f9e640 1 -- start start 2023-10-18T08:56:57.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.858+0000 7f33d0f9e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33cc0b15b0 0x7f33cc15a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:57.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.858+0000 7f33d0f9e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33cc0b1e10 0x7f33cc1449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:57.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.858+0000 7f33d0f9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33cc0b2270 0x7f33cc1e2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:57.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.858+0000 7f33d0f9e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc144ef0 con 0x7f33cc0b2270 2023-10-18T08:56:57.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.858+0000 7f33d0f9e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc1e42f0 con 0x7f33cc0b1e10 2023-10-18T08:56:57.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.858+0000 7f33d0f9e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc1e45d0 con 0x7f33cc0b15b0 2023-10-18T08:56:57.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.858+0000 7f33cbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33cc0b2270 0x7f33cc1e2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:57.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.858+0000 7f33cbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33cc0b2270 0x7f33cc1e2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33712/0 (socket says 172.21.15.40:33712) 2023-10-18T08:56:57.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.858+0000 7f33cbfff640 1 -- 172.21.15.40:0/875943870 learned_addr learned my addr 172.21.15.40:0/875943870 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:57.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.859+0000 7f33cbfff640 1 -- 172.21.15.40:0/875943870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33cc0b15b0 msgr2=0x7f33cc15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:57.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.859+0000 7f33cbfff640 1 --2- 172.21.15.40:0/875943870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33cc0b15b0 0x7f33cc15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.859+0000 7f33cbfff640 1 -- 172.21.15.40:0/875943870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33cc0b1e10 msgr2=0x7f33cc1449b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:57.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.859+0000 7f33cbfff640 1 --2- 172.21.15.40:0/875943870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33cc0b1e10 0x7f33cc1449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:57.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.859+0000 7f33cbfff640 1 -- 172.21.15.40:0/875943870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c4007490 con 0x7f33cc0b2270 2023-10-18T08:56:57.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.859+0000 7f33cbfff640 1 --2- 172.21.15.40:0/875943870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33cc0b2270 0x7f33cc1e2f60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f33c4007780 tx=0x7f33c4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:57.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.859+0000 7f33c8ff9640 1 -- 172.21.15.40:0/875943870 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c4043070 con 0x7f33cc0b2270 2023-10-18T08:56:57.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.859+0000 7f33c8ff9640 1 -- 172.21.15.40:0/875943870 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f33c4004250 con 0x7f33cc0b2270 2023-10-18T08:56:57.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.859+0000 7f33c8ff9640 1 -- 172.21.15.40:0/875943870 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c40046c0 con 0x7f33cc0b2270 2023-10-18T08:56:57.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.859+0000 7f33d0f9e640 1 -- 172.21.15.40:0/875943870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33cc1e48b0 con 0x7f33cc0b2270 2023-10-18T08:56:57.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.860+0000 7f33d0f9e640 1 -- 172.21.15.40:0/875943870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33cc1e4e70 con 0x7f33cc0b2270 2023-10-18T08:56:57.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.861+0000 7f33d0f9e640 1 -- 172.21.15.40:0/875943870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33cc069c50 con 0x7f33cc0b2270 2023-10-18T08:56:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.862+0000 7f33c8ff9640 1 -- 172.21.15.40:0/875943870 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f33c400a0d0 con 0x7f33cc0b2270 2023-10-18T08:56:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.863+0000 7f33c8ff9640 1 --2- 172.21.15.40:0/875943870 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f33ac06ae40 0x7f33ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.863+0000 7f33c8ff9640 1 -- 172.21.15.40:0/875943870 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33c40aba80 con 0x7f33cc0b2270 2023-10-18T08:56:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.864+0000 7f33cb7fe640 1 --2- 172.21.15.40:0/875943870 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f33ac06ae40 0x7f33ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:57.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.865+0000 7f33cb7fe640 1 --2- 172.21.15.40:0/875943870 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f33ac06ae40 0x7f33ac06d300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00a8f0 tx=0x7f33bc005bf0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:57.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:57.867+0000 7f33c8ff9640 1 -- 172.21.15.40:0/875943870 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33c4076aa0 con 0x7f33cc0b2270 2023-10-18T08:56:58.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.065+0000 7f33d0f9e640 1 -- 172.21.15.40:0/875943870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33cc072100 con 0x7f33cc0b2270 2023-10-18T08:56:58.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.067+0000 7f33c8ff9640 1 -- 172.21.15.40:0/875943870 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16581 (secure 0 0 0) 0x7f33c407a0c0 con 0x7f33cc0b2270 2023-10-18T08:56:58.070 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:58.071 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":35,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:46.967356+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:56:30.127184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-18T08:56:31.225270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:58.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.069+0000 7f33d0f9e640 1 -- 172.21.15.40:0/875943870 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f33ac06ae40 msgr2=0x7f33ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:58.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.069+0000 7f33d0f9e640 1 --2- 172.21.15.40:0/875943870 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f33ac06ae40 0x7f33ac06d300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00a8f0 tx=0x7f33bc005bf0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.069+0000 7f33d0f9e640 1 -- 172.21.15.40:0/875943870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33cc0b2270 msgr2=0x7f33cc1e2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:58.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.069+0000 7f33d0f9e640 1 --2- 172.21.15.40:0/875943870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33cc0b2270 0x7f33cc1e2f60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f33c4007780 tx=0x7f33c4003f10 comp rx=0 tx=0).stop 2023-10-18T08:56:58.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.070+0000 7f33d0f9e640 1 -- 172.21.15.40:0/875943870 shutdown_connections 2023-10-18T08:56:58.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.070+0000 7f33d0f9e640 1 --2- 172.21.15.40:0/875943870 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f33ac06ae40 0x7f33ac06d300 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.070+0000 7f33d0f9e640 1 --2- 172.21.15.40:0/875943870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33cc0b2270 0x7f33cc1e2f60 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.070+0000 7f33d0f9e640 1 --2- 172.21.15.40:0/875943870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33cc0b1e10 0x7f33cc1449b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.070+0000 7f33d0f9e640 1 --2- 172.21.15.40:0/875943870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33cc0b15b0 0x7f33cc15a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.070+0000 7f33d0f9e640 1 -- 172.21.15.40:0/875943870 >> 172.21.15.40:0/875943870 conn(0x7f33cc069720 msgr2=0x7f33cc155610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:58.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.070+0000 7f33d0f9e640 1 -- 172.21.15.40:0/875943870 shutdown_connections 2023-10-18T08:56:58.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.070+0000 7f33d0f9e640 1 -- 172.21.15.40:0/875943870 wait complete. 2023-10-18T08:56:58.091 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T08:56:58.092 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:56:58.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.316+0000 7f145cde2640 1 -- 172.21.15.40:0/2968833388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14580c99c0 msgr2=0x7f14580c9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:58.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.316+0000 7f145cde2640 1 --2- 172.21.15.40:0/2968833388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14580c99c0 0x7f14580c9e30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f145806f590 tx=0x7f145002c210 comp rx=0 tx=0).stop 2023-10-18T08:56:58.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.316+0000 7f145cde2640 1 -- 172.21.15.40:0/2968833388 shutdown_connections 2023-10-18T08:56:58.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.316+0000 7f145cde2640 1 --2- 172.21.15.40:0/2968833388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14580c99c0 0x7f14580c9e30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.316+0000 7f145cde2640 1 --2- 172.21.15.40:0/2968833388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580b15b0 0x7f14580b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.316+0000 7f145cde2640 1 -- 172.21.15.40:0/2968833388 >> 172.21.15.40:0/2968833388 conn(0x7f1458069720 msgr2=0x7f1458069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:58.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.316+0000 7f145cde2640 1 -- 172.21.15.40:0/2968833388 shutdown_connections 2023-10-18T08:56:58.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.316+0000 7f145cde2640 1 -- 172.21.15.40:0/2968833388 wait complete. 2023-10-18T08:56:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.318+0000 7f145cde2640 1 Processor -- start 2023-10-18T08:56:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.318+0000 7f145cde2640 1 -- start start 2023-10-18T08:56:58.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.319+0000 7f145cde2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145806d6f0 0x7f14580af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:58.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.319+0000 7f145cde2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580b15b0 0x7f14581582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:58.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.319+0000 7f145cde2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14580c99c0 0x7f145814ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:58.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.319+0000 7f145cde2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145814b4f0 con 0x7f14580c99c0 2023-10-18T08:56:58.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.319+0000 7f145cde2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14580afc30 con 0x7f145806d6f0 2023-10-18T08:56:58.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.319+0000 7f145cde2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1458158c10 con 0x7f14580b15b0 2023-10-18T08:56:58.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.320+0000 7f1457fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14580c99c0 0x7f145814ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:58.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.320+0000 7f1457fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14580c99c0 0x7f145814ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33742/0 (socket says 172.21.15.40:33742) 2023-10-18T08:56:58.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.320+0000 7f1457fff640 1 -- 172.21.15.40:0/1418822708 learned_addr learned my addr 172.21.15.40:0/1418822708 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:58.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.320+0000 7f14577fe640 1 --2- 172.21.15.40:0/1418822708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145806d6f0 0x7f14580af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:58.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.320+0000 7f1456ffd640 1 --2- 172.21.15.40:0/1418822708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580b15b0 0x7f14581582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:58.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.320+0000 7f1457fff640 1 -- 172.21.15.40:0/1418822708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580b15b0 msgr2=0x7f14581582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:58.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.320+0000 7f1457fff640 1 --2- 172.21.15.40:0/1418822708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580b15b0 0x7f14581582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.320+0000 7f1457fff640 1 -- 172.21.15.40:0/1418822708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145806d6f0 msgr2=0x7f14580af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:58.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.320+0000 7f1457fff640 1 --2- 172.21.15.40:0/1418822708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145806d6f0 0x7f14580af6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.320+0000 7f1457fff640 1 -- 172.21.15.40:0/1418822708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1450009900 con 0x7f14580c99c0 2023-10-18T08:56:58.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.320+0000 7f14577fe640 1 --2- 172.21.15.40:0/1418822708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145806d6f0 0x7f14580af6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:58.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.320+0000 7f1457fff640 1 --2- 172.21.15.40:0/1418822708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14580c99c0 0x7f145814ffc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1450037040 tx=0x7f1450035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:58.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.321+0000 7f143ffff640 1 -- 172.21.15.40:0/1418822708 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1450034440 con 0x7f14580c99c0 2023-10-18T08:56:58.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.321+0000 7f143ffff640 1 -- 172.21.15.40:0/1418822708 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1450004450 con 0x7f14580c99c0 2023-10-18T08:56:58.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.321+0000 7f145cde2640 1 -- 172.21.15.40:0/1418822708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1458158ef0 con 0x7f14580c99c0 2023-10-18T08:56:58.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.321+0000 7f143ffff640 1 -- 172.21.15.40:0/1418822708 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1450004840 con 0x7f14580c99c0 2023-10-18T08:56:58.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.321+0000 7f145cde2640 1 -- 172.21.15.40:0/1418822708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14581594b0 con 0x7f14580c99c0 2023-10-18T08:56:58.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.322+0000 7f145cde2640 1 -- 172.21.15.40:0/1418822708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145814b660 con 0x7f14580c99c0 2023-10-18T08:56:58.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.323+0000 7f143ffff640 1 -- 172.21.15.40:0/1418822708 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f145003b040 con 0x7f14580c99c0 2023-10-18T08:56:58.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.324+0000 7f143ffff640 1 --2- 172.21.15.40:0/1418822708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f144406ae40 0x7f144406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:58.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.324+0000 7f143ffff640 1 -- 172.21.15.40:0/1418822708 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14500ac4e0 con 0x7f14580c99c0 2023-10-18T08:56:58.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.325+0000 7f14577fe640 1 --2- 172.21.15.40:0/1418822708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f144406ae40 0x7f144406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:58.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.326+0000 7f14577fe640 1 --2- 172.21.15.40:0/1418822708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f144406ae40 0x7f144406d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f145815acb0 tx=0x7f1448009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:58.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.329+0000 7f143ffff640 1 -- 172.21.15.40:0/1418822708 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1450077580 con 0x7f14580c99c0 2023-10-18T08:56:58.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.557+0000 7f145cde2640 1 -- 172.21.15.40:0/1418822708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1458066e00 con 0x7f14580c99c0 2023-10-18T08:56:58.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.559+0000 7f143ffff640 1 -- 172.21.15.40:0/1418822708 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f145007aba0 con 0x7f14580c99c0 2023-10-18T08:56:58.562 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:58.563 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/837790640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":837790640},{"type":"v1","addr":"172.21.15.40:6833","nonce":837790640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3377302775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3377302775},{"type":"v1","addr":"172.21.15.144:6836","nonce":3377302775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4939,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3162305748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3162305748},{"type":"v1","addr":"172.21.15.144:6837","nonce":3162305748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:56:32.203221+0000","modified":"2023-10-18T08:56:35.403466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5190":{"gid":5190,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3167454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3167454356},{"type":"v1","addr":"172.21.15.40:6835","nonce":3167454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T08:56:58.563 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2023-10-18T08:56:58.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.561+0000 7f145cde2640 1 -- 172.21.15.40:0/1418822708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f144406ae40 msgr2=0x7f144406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:58.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.561+0000 7f145cde2640 1 --2- 172.21.15.40:0/1418822708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f144406ae40 0x7f144406d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f145815acb0 tx=0x7f1448009040 comp rx=0 tx=0).stop 2023-10-18T08:56:58.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.561+0000 7f145cde2640 1 -- 172.21.15.40:0/1418822708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14580c99c0 msgr2=0x7f145814ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:58.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.561+0000 7f145cde2640 1 --2- 172.21.15.40:0/1418822708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14580c99c0 0x7f145814ffc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1450037040 tx=0x7f1450035db0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.561+0000 7f145cde2640 1 -- 172.21.15.40:0/1418822708 shutdown_connections 2023-10-18T08:56:58.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.561+0000 7f145cde2640 1 --2- 172.21.15.40:0/1418822708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f144406ae40 0x7f144406d300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.561+0000 7f145cde2640 1 --2- 172.21.15.40:0/1418822708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14580c99c0 0x7f145814ffc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.561+0000 7f145cde2640 1 --2- 172.21.15.40:0/1418822708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14580b15b0 0x7f14581582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.561+0000 7f145cde2640 1 --2- 172.21.15.40:0/1418822708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145806d6f0 0x7f14580af6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.561+0000 7f145cde2640 1 -- 172.21.15.40:0/1418822708 >> 172.21.15.40:0/1418822708 conn(0x7f1458069720 msgr2=0x7f145815d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:58.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.562+0000 7f145cde2640 1 -- 172.21.15.40:0/1418822708 shutdown_connections 2023-10-18T08:56:58.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.562+0000 7f145cde2640 1 -- 172.21.15.40:0/1418822708 wait complete. 2023-10-18T08:56:58.584 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:56:58.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.809+0000 7f0bf17ee640 1 -- 172.21.15.40:0/89082854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bec15a2e0 msgr2=0x7f0bec0cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:58.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.809+0000 7f0bf17ee640 1 --2- 172.21.15.40:0/89082854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bec15a2e0 0x7f0bec0cbc20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc005fd0 tx=0x7f0bdc02bd90 comp rx=0 tx=0).stop 2023-10-18T08:56:58.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.809+0000 7f0bf17ee640 1 -- 172.21.15.40:0/89082854 shutdown_connections 2023-10-18T08:56:58.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.809+0000 7f0bf17ee640 1 --2- 172.21.15.40:0/89082854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bec15a6c0 0x7f0bec06b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.809+0000 7f0bf17ee640 1 --2- 172.21.15.40:0/89082854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bec15a2e0 0x7f0bec0cbc20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.809+0000 7f0bf17ee640 1 -- 172.21.15.40:0/89082854 >> 172.21.15.40:0/89082854 conn(0x7f0bec06cee0 msgr2=0x7f0bec069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:58.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.809+0000 7f0bf17ee640 1 -- 172.21.15.40:0/89082854 shutdown_connections 2023-10-18T08:56:58.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.809+0000 7f0bf17ee640 1 -- 172.21.15.40:0/89082854 wait complete. 2023-10-18T08:56:58.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.811+0000 7f0bf17ee640 1 Processor -- start 2023-10-18T08:56:58.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.811+0000 7f0bf17ee640 1 -- start start 2023-10-18T08:56:58.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.812+0000 7f0bf17ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bec15a2e0 0x7f0bec0c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:58.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.812+0000 7f0bf17ee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bec15a6c0 0x7f0bec0aad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:58.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.812+0000 7f0bf17ee640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bec0ab250 0x7f0bec0a7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:58.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.812+0000 7f0bf17ee640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec1452d0 con 0x7f0bec15a2e0 2023-10-18T08:56:58.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.812+0000 7f0bf17ee640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0a8050 con 0x7f0bec15a6c0 2023-10-18T08:56:58.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.812+0000 7f0bf17ee640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0ab700 con 0x7f0bec0ab250 2023-10-18T08:56:58.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.812+0000 7f0beaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bec15a2e0 0x7f0bec0c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:58.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.812+0000 7f0beaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bec15a2e0 0x7f0bec0c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33774/0 (socket says 172.21.15.40:33774) 2023-10-18T08:56:58.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.812+0000 7f0beaffd640 1 -- 172.21.15.40:0/1810581652 learned_addr learned my addr 172.21.15.40:0/1810581652 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:58.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.813+0000 7f0beaffd640 1 -- 172.21.15.40:0/1810581652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bec0ab250 msgr2=0x7f0bec0a7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:58.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.813+0000 7f0beaffd640 1 --2- 172.21.15.40:0/1810581652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bec0ab250 0x7f0bec0a7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.813+0000 7f0beaffd640 1 -- 172.21.15.40:0/1810581652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bec15a6c0 msgr2=0x7f0bec0aad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:58.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.813+0000 7f0beaffd640 1 --2- 172.21.15.40:0/1810581652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bec15a6c0 0x7f0bec0aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:58.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.813+0000 7f0beaffd640 1 -- 172.21.15.40:0/1810581652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bdc009560 con 0x7f0bec15a2e0 2023-10-18T08:56:58.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.813+0000 7f0beaffd640 1 --2- 172.21.15.40:0/1810581652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bec15a2e0 0x7f0bec0c9060 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009850 tx=0x7f0bdc02db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:58.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.813+0000 7f0be3fff640 1 -- 172.21.15.40:0/1810581652 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc0035f0 con 0x7f0bec15a2e0 2023-10-18T08:56:58.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.813+0000 7f0be3fff640 1 -- 172.21.15.40:0/1810581652 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0bdc003750 con 0x7f0bec15a2e0 2023-10-18T08:56:58.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.813+0000 7f0be3fff640 1 -- 172.21.15.40:0/1810581652 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc003b20 con 0x7f0bec15a2e0 2023-10-18T08:56:58.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.813+0000 7f0bf17ee640 1 -- 172.21.15.40:0/1810581652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bec0a7d70 con 0x7f0bec15a2e0 2023-10-18T08:56:58.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.813+0000 7f0bf17ee640 1 -- 172.21.15.40:0/1810581652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bec144ef0 con 0x7f0bec15a2e0 2023-10-18T08:56:58.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.815+0000 7f0bf17ee640 1 -- 172.21.15.40:0/1810581652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bec06c020 con 0x7f0bec15a2e0 2023-10-18T08:56:58.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.816+0000 7f0be3fff640 1 -- 172.21.15.40:0/1810581652 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0bdc003c80 con 0x7f0bec15a2e0 2023-10-18T08:56:58.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.817+0000 7f0be3fff640 1 --2- 172.21.15.40:0/1810581652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0bcc06ae40 0x7f0bcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:58.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.817+0000 7f0be3fff640 1 -- 172.21.15.40:0/1810581652 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bdc0ac2c0 con 0x7f0bec15a2e0 2023-10-18T08:56:58.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.817+0000 7f0be27fc640 1 --2- 172.21.15.40:0/1810581652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0bcc06ae40 0x7f0bcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:58.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.818+0000 7f0be27fc640 1 --2- 172.21.15.40:0/1810581652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0bcc06ae40 0x7f0bcc06d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0b2430 tx=0x7f0bd4007990 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:58.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:58.821+0000 7f0be3fff640 1 -- 172.21.15.40:0/1810581652 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0bdc077490 con 0x7f0bec15a2e0 2023-10-18T08:56:59.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.056+0000 7f0bf17ee640 1 -- 172.21.15.40:0/1810581652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bec145080 con 0x7f0bec15a2e0 2023-10-18T08:56:59.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.058+0000 7f0be3fff640 1 -- 172.21.15.40:0/1810581652 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f0bdc07aab0 con 0x7f0bec15a2e0 2023-10-18T08:56:59.061 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:59.061 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/837790640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":837790640},{"type":"v1","addr":"172.21.15.40:6833","nonce":837790640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3377302775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3377302775},{"type":"v1","addr":"172.21.15.144:6836","nonce":3377302775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4939,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3162305748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3162305748},{"type":"v1","addr":"172.21.15.144:6837","nonce":3162305748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:56:32.203221+0000","modified":"2023-10-18T08:56:35.403466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5190":{"gid":5190,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.40:6835/3167454356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3167454356},{"type":"v1","addr":"172.21.15.40:6835","nonce":3167454356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-18T08:56:59.062 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 16 2023-10-18T08:56:59.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.060+0000 7f0bf17ee640 1 -- 172.21.15.40:0/1810581652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0bcc06ae40 msgr2=0x7f0bcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:59.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.061+0000 7f0bf17ee640 1 --2- 172.21.15.40:0/1810581652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0bcc06ae40 0x7f0bcc06d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0b2430 tx=0x7f0bd4007990 comp rx=0 tx=0).stop 2023-10-18T08:56:59.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.061+0000 7f0bf17ee640 1 -- 172.21.15.40:0/1810581652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bec15a2e0 msgr2=0x7f0bec0c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:59.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.061+0000 7f0bf17ee640 1 --2- 172.21.15.40:0/1810581652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bec15a2e0 0x7f0bec0c9060 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc009850 tx=0x7f0bdc02db10 comp rx=0 tx=0).stop 2023-10-18T08:56:59.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.061+0000 7f0bf17ee640 1 -- 172.21.15.40:0/1810581652 shutdown_connections 2023-10-18T08:56:59.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.061+0000 7f0bf17ee640 1 --2- 172.21.15.40:0/1810581652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0bcc06ae40 0x7f0bcc06d300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.061+0000 7f0bf17ee640 1 --2- 172.21.15.40:0/1810581652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bec0ab250 0x7f0bec0a7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.061+0000 7f0bf17ee640 1 --2- 172.21.15.40:0/1810581652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bec15a6c0 0x7f0bec0aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.061+0000 7f0bf17ee640 1 --2- 172.21.15.40:0/1810581652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bec15a2e0 0x7f0bec0c9060 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.061+0000 7f0bf17ee640 1 -- 172.21.15.40:0/1810581652 >> 172.21.15.40:0/1810581652 conn(0x7f0bec06cee0 msgr2=0x7f0bec15ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:59.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.061+0000 7f0bf17ee640 1 -- 172.21.15.40:0/1810581652 shutdown_connections 2023-10-18T08:56:59.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.061+0000 7f0bf17ee640 1 -- 172.21.15.40:0/1810581652 wait complete. 2023-10-18T08:56:59.083 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:56:59.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.304+0000 7f6af9047640 1 -- 172.21.15.40:0/508784921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40ca0d0 msgr2=0x7f6af4157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:59.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.304+0000 7f6af9047640 1 --2- 172.21.15.40:0/508784921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40ca0d0 0x7f6af4157d40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6af40caf30 tx=0x7f6aec02d150 comp rx=0 tx=0).stop 2023-10-18T08:56:59.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.304+0000 7f6af9047640 1 -- 172.21.15.40:0/508784921 shutdown_connections 2023-10-18T08:56:59.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.304+0000 7f6af9047640 1 --2- 172.21.15.40:0/508784921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40ca0d0 0x7f6af4157d40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.304+0000 7f6af9047640 1 --2- 172.21.15.40:0/508784921 >> v2:172.21.15.144:3300/0 conn(0x7f6af406d6f0 0x7f6af40b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.304+0000 7f6af9047640 1 -- 172.21.15.40:0/508784921 >> 172.21.15.40:0/508784921 conn(0x7f6af4069720 msgr2=0x7f6af4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:59.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.304+0000 7f6af9047640 1 -- 172.21.15.40:0/508784921 shutdown_connections 2023-10-18T08:56:59.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.304+0000 7f6af9047640 1 -- 172.21.15.40:0/508784921 wait complete. 2023-10-18T08:56:59.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.306+0000 7f6af9047640 1 Processor -- start 2023-10-18T08:56:59.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.306+0000 7f6af9047640 1 -- start start 2023-10-18T08:56:59.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.307+0000 7f6af9047640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af406d6f0 0x7f6af40adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:59.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.307+0000 7f6af9047640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40b0630 0x7f6af41e33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:59.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.307+0000 7f6af9047640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6af40ca0d0 0x7f6af41e41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:59.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.307+0000 7f6af9047640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40af3f0 con 0x7f6af40b0630 2023-10-18T08:56:59.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.307+0000 7f6af9047640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40af880 con 0x7f6af40ca0d0 2023-10-18T08:56:59.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.307+0000 7f6af9047640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40ae7e0 con 0x7f6af406d6f0 2023-10-18T08:56:59.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.307+0000 7f6aebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40b0630 0x7f6af41e33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:59.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.307+0000 7f6aebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40b0630 0x7f6af41e33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33782/0 (socket says 172.21.15.40:33782) 2023-10-18T08:56:59.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.307+0000 7f6aebfff640 1 -- 172.21.15.40:0/2968150796 learned_addr learned my addr 172.21.15.40:0/2968150796 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:59.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.307+0000 7f6af3fff640 1 --2- 172.21.15.40:0/2968150796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af406d6f0 0x7f6af40adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:59.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.308+0000 7f6af8846640 1 --2- 172.21.15.40:0/2968150796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6af40ca0d0 0x7f6af41e41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:59.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.308+0000 7f6aebfff640 1 -- 172.21.15.40:0/2968150796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af406d6f0 msgr2=0x7f6af40adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:59.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.308+0000 7f6aebfff640 1 --2- 172.21.15.40:0/2968150796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af406d6f0 0x7f6af40adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.308+0000 7f6aebfff640 1 -- 172.21.15.40:0/2968150796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6af40ca0d0 msgr2=0x7f6af41e41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:59.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.308+0000 7f6aebfff640 1 --2- 172.21.15.40:0/2968150796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6af40ca0d0 0x7f6af41e41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.308+0000 7f6aebfff640 1 -- 172.21.15.40:0/2968150796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec00aa10 con 0x7f6af40b0630 2023-10-18T08:56:59.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.308+0000 7f6aebfff640 1 --2- 172.21.15.40:0/2968150796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40b0630 0x7f6af41e33a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00c410 tx=0x7f6adc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:59.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.308+0000 7f6af17fa640 1 -- 172.21.15.40:0/2968150796 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0035f0 con 0x7f6af40b0630 2023-10-18T08:56:59.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.308+0000 7f6af17fa640 1 -- 172.21.15.40:0/2968150796 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6adc003750 con 0x7f6af40b0630 2023-10-18T08:56:59.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.308+0000 7f6af9047640 1 -- 172.21.15.40:0/2968150796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af40aeac0 con 0x7f6af40b0630 2023-10-18T08:56:59.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.308+0000 7f6af17fa640 1 -- 172.21.15.40:0/2968150796 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc003ba0 con 0x7f6af40b0630 2023-10-18T08:56:59.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.309+0000 7f6af9047640 1 -- 172.21.15.40:0/2968150796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af40af080 con 0x7f6af40b0630 2023-10-18T08:56:59.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.310+0000 7f6af9047640 1 -- 172.21.15.40:0/2968150796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af4066e00 con 0x7f6af40b0630 2023-10-18T08:56:59.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.311+0000 7f6af17fa640 1 -- 172.21.15.40:0/2968150796 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6adc003d00 con 0x7f6af40b0630 2023-10-18T08:56:59.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.312+0000 7f6af17fa640 1 --2- 172.21.15.40:0/2968150796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ad406ae40 0x7f6ad406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:59.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.312+0000 7f6af17fa640 1 -- 172.21.15.40:0/2968150796 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6adc088a60 con 0x7f6af40b0630 2023-10-18T08:56:59.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.312+0000 7f6af3fff640 1 --2- 172.21.15.40:0/2968150796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ad406ae40 0x7f6ad406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:59.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.313+0000 7f6af3fff640 1 --2- 172.21.15.40:0/2968150796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ad406ae40 0x7f6ad406d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005fd0 tx=0x7f6ae4005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:59.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.318+0000 7f6af17fa640 1 -- 172.21.15.40:0/2968150796 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6adc053a80 con 0x7f6af40b0630 2023-10-18T08:56:59.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.510+0000 7f6af9047640 1 -- 172.21.15.40:0/2968150796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6af40af530 con 0x7f6af40b0630 2023-10-18T08:56:59.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.512+0000 7f6af17fa640 1 -- 172.21.15.40:0/2968150796 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16581 (secure 0 0 0) 0x7f6adc0570a0 con 0x7f6af40b0630 2023-10-18T08:56:59.515 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:56:59.516 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":35,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:56:46.967356+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:56:30.127184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-18T08:56:31.225270+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:56:59.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.514+0000 7f6af9047640 1 -- 172.21.15.40:0/2968150796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ad406ae40 msgr2=0x7f6ad406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:59.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.514+0000 7f6af9047640 1 --2- 172.21.15.40:0/2968150796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ad406ae40 0x7f6ad406d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005fd0 tx=0x7f6ae4005850 comp rx=0 tx=0).stop 2023-10-18T08:56:59.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.514+0000 7f6af9047640 1 -- 172.21.15.40:0/2968150796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40b0630 msgr2=0x7f6af41e33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:59.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.514+0000 7f6af9047640 1 --2- 172.21.15.40:0/2968150796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40b0630 0x7f6af41e33a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00c410 tx=0x7f6adc00c8d0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.514+0000 7f6af9047640 1 -- 172.21.15.40:0/2968150796 shutdown_connections 2023-10-18T08:56:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.514+0000 7f6af9047640 1 --2- 172.21.15.40:0/2968150796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ad406ae40 0x7f6ad406d300 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.514+0000 7f6af9047640 1 --2- 172.21.15.40:0/2968150796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6af40ca0d0 0x7f6af41e41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.514+0000 7f6af9047640 1 --2- 172.21.15.40:0/2968150796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6af40b0630 0x7f6af41e33a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.514+0000 7f6af9047640 1 --2- 172.21.15.40:0/2968150796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af406d6f0 0x7f6af40adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.514+0000 7f6af9047640 1 -- 172.21.15.40:0/2968150796 >> 172.21.15.40:0/2968150796 conn(0x7f6af4069720 msgr2=0x7f6af4154560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.515+0000 7f6af9047640 1 -- 172.21.15.40:0/2968150796 shutdown_connections 2023-10-18T08:56:59.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.515+0000 7f6af9047640 1 -- 172.21.15.40:0/2968150796 wait complete. 2023-10-18T08:56:59.537 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T08:56:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.749+0000 7f759647f640 1 -- 172.21.15.40:0/1652402253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7590152560 msgr2=0x7f7590070f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:59.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.749+0000 7f759647f640 1 --2- 172.21.15.40:0/1652402253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7590152560 0x7f7590070f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f758c009900 tx=0x7f758c02beb0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.749+0000 7f759647f640 1 -- 172.21.15.40:0/1652402253 shutdown_connections 2023-10-18T08:56:59.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.749+0000 7f759647f640 1 --2- 172.21.15.40:0/1652402253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75900ab2a0 0x7f75900cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.749+0000 7f759647f640 1 --2- 172.21.15.40:0/1652402253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7590152560 0x7f7590070f00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.749+0000 7f759647f640 1 -- 172.21.15.40:0/1652402253 >> 172.21.15.40:0/1652402253 conn(0x7f7590069720 msgr2=0x7f7590069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:56:59.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.749+0000 7f759647f640 1 -- 172.21.15.40:0/1652402253 shutdown_connections 2023-10-18T08:56:59.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.749+0000 7f759647f640 1 -- 172.21.15.40:0/1652402253 wait complete. 2023-10-18T08:56:59.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.752+0000 7f759647f640 1 Processor -- start 2023-10-18T08:56:59.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.752+0000 7f759647f640 1 -- start start 2023-10-18T08:56:59.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.753+0000 7f759647f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7590071440 0x7f7590151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:59.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.753+0000 7f759647f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75900ab2a0 0x7f759014ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:59.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.753+0000 7f759647f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7590152560 0x7f7590154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:59.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.753+0000 7f759647f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7590151a70 con 0x7f75900ab2a0 2023-10-18T08:56:59.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.753+0000 7f759647f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759014b050 con 0x7f7590071440 2023-10-18T08:56:59.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.753+0000 7f759647f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759014ee00 con 0x7f7590152560 2023-10-18T08:56:59.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.753+0000 7f7594c7c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75900ab2a0 0x7f759014ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:59.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.753+0000 7f7594c7c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75900ab2a0 0x7f759014ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33792/0 (socket says 172.21.15.40:33792) 2023-10-18T08:56:59.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.753+0000 7f7594c7c640 1 -- 172.21.15.40:0/2663377680 learned_addr learned my addr 172.21.15.40:0/2663377680 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:56:59.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.754+0000 7f759547d640 1 --2- 172.21.15.40:0/2663377680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7590071440 0x7f7590151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:59.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.754+0000 7f7594c7c640 1 -- 172.21.15.40:0/2663377680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7590152560 msgr2=0x7f7590154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:59.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.754+0000 7f7595c7e640 1 --2- 172.21.15.40:0/2663377680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7590152560 0x7f7590154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:59.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.754+0000 7f7594c7c640 1 --2- 172.21.15.40:0/2663377680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7590152560 0x7f7590154830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.754+0000 7f7594c7c640 1 -- 172.21.15.40:0/2663377680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7590071440 msgr2=0x7f7590151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:56:59.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.754+0000 7f7594c7c640 1 --2- 172.21.15.40:0/2663377680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7590071440 0x7f7590151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:56:59.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.754+0000 7f7594c7c640 1 -- 172.21.15.40:0/2663377680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c009560 con 0x7f75900ab2a0 2023-10-18T08:56:59.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.754+0000 7f759547d640 1 --2- 172.21.15.40:0/2663377680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7590071440 0x7f7590151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:56:59.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.754+0000 7f7594c7c640 1 --2- 172.21.15.40:0/2663377680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75900ab2a0 0x7f759014ab10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f758000c450 tx=0x7f758000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:59.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.754+0000 7f7585ffb640 1 -- 172.21.15.40:0/2663377680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75800052d0 con 0x7f75900ab2a0 2023-10-18T08:56:59.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.755+0000 7f759647f640 1 -- 172.21.15.40:0/2663377680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759014f0e0 con 0x7f75900ab2a0 2023-10-18T08:56:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.755+0000 7f7585ffb640 1 -- 172.21.15.40:0/2663377680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7580005430 con 0x7f75900ab2a0 2023-10-18T08:56:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.755+0000 7f7585ffb640 1 -- 172.21.15.40:0/2663377680 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580012890 con 0x7f75900ab2a0 2023-10-18T08:56:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.755+0000 7f759647f640 1 -- 172.21.15.40:0/2663377680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759014f6a0 con 0x7f75900ab2a0 2023-10-18T08:56:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.756+0000 7f759647f640 1 -- 172.21.15.40:0/2663377680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7590066e00 con 0x7f75900ab2a0 2023-10-18T08:56:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.757+0000 7f7585ffb640 1 -- 172.21.15.40:0/2663377680 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f75800035f0 con 0x7f75900ab2a0 2023-10-18T08:56:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.758+0000 7f7585ffb640 1 --2- 172.21.15.40:0/2663377680 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f757406ae40 0x7f757406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:56:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.758+0000 7f7585ffb640 1 -- 172.21.15.40:0/2663377680 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75800891a0 con 0x7f75900ab2a0 2023-10-18T08:56:59.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.758+0000 7f759547d640 1 --2- 172.21.15.40:0/2663377680 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f757406ae40 0x7f757406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:56:59.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.759+0000 7f759547d640 1 --2- 172.21.15.40:0/2663377680 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f757406ae40 0x7f757406d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f758c035000 tx=0x7f758c02e040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:56:59.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:56:59.763+0000 7f7585ffb640 1 -- 172.21.15.40:0/2663377680 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f75800541c0 con 0x7f75900ab2a0 2023-10-18T08:57:00.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.002+0000 7f759647f640 1 -- 172.21.15.40:0/2663377680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7590071890 con 0x7f75900ab2a0 2023-10-18T08:57:00.005 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T08:57:00.002+0000 7f6a0f710640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T08:57:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.005+0000 7f7585ffb640 1 -- 172.21.15.40:0/2663377680 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7590071890 con 0x7f75900ab2a0 2023-10-18T08:57:00.009 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T08:57:00.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.010+0000 7f759647f640 1 -- 172.21.15.40:0/2663377680 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f757406ae40 msgr2=0x7f757406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:00.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.010+0000 7f759647f640 1 --2- 172.21.15.40:0/2663377680 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f757406ae40 0x7f757406d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f758c035000 tx=0x7f758c02e040 comp rx=0 tx=0).stop 2023-10-18T08:57:00.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.010+0000 7f759647f640 1 -- 172.21.15.40:0/2663377680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75900ab2a0 msgr2=0x7f759014ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:00.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.010+0000 7f759647f640 1 --2- 172.21.15.40:0/2663377680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75900ab2a0 0x7f759014ab10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f758000c450 tx=0x7f758000c910 comp rx=0 tx=0).stop 2023-10-18T08:57:00.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.010+0000 7f759647f640 1 -- 172.21.15.40:0/2663377680 shutdown_connections 2023-10-18T08:57:00.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.010+0000 7f759647f640 1 --2- 172.21.15.40:0/2663377680 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f757406ae40 0x7f757406d300 secure :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f758c035000 tx=0x7f758c02e040 comp rx=0 tx=0).stop 2023-10-18T08:57:00.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.010+0000 7f759647f640 1 --2- 172.21.15.40:0/2663377680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7590152560 0x7f7590154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:00.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.010+0000 7f759647f640 1 --2- 172.21.15.40:0/2663377680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75900ab2a0 0x7f759014ab10 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:00.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.010+0000 7f759647f640 1 --2- 172.21.15.40:0/2663377680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7590071440 0x7f7590151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:00.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.010+0000 7f759647f640 1 -- 172.21.15.40:0/2663377680 >> 172.21.15.40:0/2663377680 conn(0x7f7590069720 msgr2=0x7f7590155770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:00.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.012+0000 7f759647f640 1 -- 172.21.15.40:0/2663377680 shutdown_connections 2023-10-18T08:57:00.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.012+0000 7f759647f640 1 -- 172.21.15.40:0/2663377680 wait complete. 2023-10-18T08:57:00.022 INFO:tasks.ceph.mds.a.smithi040.stderr: -4967> 2023-10-18T08:56:35.445+0000 7f1aafcdc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T08:57:00.033 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T08:57:00.101 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-10-18T08:57:00.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.236+0000 7fccd35b6640 1 -- 172.21.15.40:0/627302403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcccc15b740 msgr2=0x7fcccc070860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:00.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.236+0000 7fccd35b6640 1 --2- 172.21.15.40:0/627302403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcccc15b740 0x7fcccc070860 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fccc8005fd0 tx=0x7fccc802be10 comp rx=0 tx=0).stop 2023-10-18T08:57:00.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.236+0000 7fccd35b6640 1 -- 172.21.15.40:0/627302403 shutdown_connections 2023-10-18T08:57:00.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.236+0000 7fccd35b6640 1 --2- 172.21.15.40:0/627302403 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcccc0c8480 0x7fcccc0c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:00.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.236+0000 7fccd35b6640 1 --2- 172.21.15.40:0/627302403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcccc15b740 0x7fcccc070860 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:00.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.236+0000 7fccd35b6640 1 -- 172.21.15.40:0/627302403 >> 172.21.15.40:0/627302403 conn(0x7fcccc0cd840 msgr2=0x7fcccc0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:00.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.236+0000 7fccd35b6640 1 -- 172.21.15.40:0/627302403 shutdown_connections 2023-10-18T08:57:00.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.236+0000 7fccd35b6640 1 -- 172.21.15.40:0/627302403 wait complete. 2023-10-18T08:57:00.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.238+0000 7fccd35b6640 1 Processor -- start 2023-10-18T08:57:00.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.238+0000 7fccd35b6640 1 -- start start 2023-10-18T08:57:00.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.239+0000 7fccd35b6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcccc0c8480 0x7fcccc1f1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:00.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.239+0000 7fccd35b6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcccc0cd3f0 0x7fcccc143570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:00.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.239+0000 7fccd35b6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcccc15b740 0x7fcccc143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:00.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.239+0000 7fccd35b6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc0c7510 con 0x7fcccc15b740 2023-10-18T08:57:00.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.239+0000 7fccd35b6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc1f1c80 con 0x7fcccc0cd3f0 2023-10-18T08:57:00.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.239+0000 7fccd35b6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc0c6900 con 0x7fcccc0c8480 2023-10-18T08:57:00.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.239+0000 7fccd1b53640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcccc15b740 0x7fcccc143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:00.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.239+0000 7fccd1b53640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcccc15b740 0x7fcccc143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33812/0 (socket says 172.21.15.40:33812) 2023-10-18T08:57:00.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.239+0000 7fccd1b53640 1 -- 172.21.15.40:0/1094495956 learned_addr learned my addr 172.21.15.40:0/1094495956 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:00.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.239+0000 7fccd1352640 1 --2- 172.21.15.40:0/1094495956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcccc0c8480 0x7fcccc1f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:00.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.239+0000 7fccd0b51640 1 --2- 172.21.15.40:0/1094495956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcccc0cd3f0 0x7fcccc143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:00.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.240+0000 7fccd1b53640 1 -- 172.21.15.40:0/1094495956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcccc0c8480 msgr2=0x7fcccc1f1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:00.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.240+0000 7fccd1b53640 1 --2- 172.21.15.40:0/1094495956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcccc0c8480 0x7fcccc1f1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:00.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.240+0000 7fccd1b53640 1 -- 172.21.15.40:0/1094495956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcccc0cd3f0 msgr2=0x7fcccc143570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:00.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.240+0000 7fccd1b53640 1 --2- 172.21.15.40:0/1094495956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcccc0cd3f0 0x7fcccc143570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:00.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.240+0000 7fccd1b53640 1 -- 172.21.15.40:0/1094495956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc8009560 con 0x7fcccc15b740 2023-10-18T08:57:00.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.240+0000 7fccd1352640 1 --2- 172.21.15.40:0/1094495956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcccc0c8480 0x7fcccc1f1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:57:00.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.240+0000 7fccd1b53640 1 --2- 172.21.15.40:0/1094495956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcccc15b740 0x7fcccc143ab0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fccc400eb40 tx=0x7fccc400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:00.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.240+0000 7fccc1ffb640 1 -- 172.21.15.40:0/1094495956 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4004b20 con 0x7fcccc15b740 2023-10-18T08:57:00.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.240+0000 7fccc1ffb640 1 -- 172.21.15.40:0/1094495956 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fccc4004ca0 con 0x7fcccc15b740 2023-10-18T08:57:00.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.240+0000 7fccc1ffb640 1 -- 172.21.15.40:0/1094495956 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4005280 con 0x7fcccc15b740 2023-10-18T08:57:00.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.240+0000 7fccd35b6640 1 -- 172.21.15.40:0/1094495956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcccc0cbe90 con 0x7fcccc15b740 2023-10-18T08:57:00.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.241+0000 7fccd35b6640 1 -- 172.21.15.40:0/1094495956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcccc0c6ec0 con 0x7fcccc15b740 2023-10-18T08:57:00.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.242+0000 7fccd35b6640 1 -- 172.21.15.40:0/1094495956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcccc0b0b90 con 0x7fcccc15b740 2023-10-18T08:57:00.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.243+0000 7fccc1ffb640 1 -- 172.21.15.40:0/1094495956 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fccc40053e0 con 0x7fcccc15b740 2023-10-18T08:57:00.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.244+0000 7fccc1ffb640 1 --2- 172.21.15.40:0/1094495956 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fccb006ae70 0x7fccb006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:00.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.244+0000 7fccc1ffb640 1 -- 172.21.15.40:0/1094495956 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fccc4014070 con 0x7fcccc15b740 2023-10-18T08:57:00.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.244+0000 7fccd1352640 1 --2- 172.21.15.40:0/1094495956 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fccb006ae70 0x7fccb006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:00.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.245+0000 7fccd1352640 1 --2- 172.21.15.40:0/1094495956 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fccb006ae70 0x7fccb006d330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fccc8035000 tx=0x7fccc8004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:00.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.248+0000 7fccc1ffb640 1 -- 172.21.15.40:0/1094495956 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcccc0b0b90 con 0x7fcccc15b740 2023-10-18T08:57:00.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:00.479+0000 7fccd35b6640 1 -- 172.21.15.40:0/1094495956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcccc0b0b90 con 0x7fcccc15b740 2023-10-18T08:57:01.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.039+0000 7fccc1ffb640 1 -- 172.21.15.40:0/1094495956 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcccc0b0b90 con 0x7fcccc15b740 2023-10-18T08:57:01.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.043+0000 7fccd35b6640 1 -- 172.21.15.40:0/1094495956 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fccb006ae70 msgr2=0x7fccb006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:01.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.043+0000 7fccd35b6640 1 --2- 172.21.15.40:0/1094495956 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fccb006ae70 0x7fccb006d330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fccc8035000 tx=0x7fccc8004730 comp rx=0 tx=0).stop 2023-10-18T08:57:01.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.043+0000 7fccd35b6640 1 -- 172.21.15.40:0/1094495956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcccc15b740 msgr2=0x7fcccc143ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:01.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.043+0000 7fccd35b6640 1 --2- 172.21.15.40:0/1094495956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcccc15b740 0x7fcccc143ab0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fccc400eb40 tx=0x7fccc400c2d0 comp rx=0 tx=0).stop 2023-10-18T08:57:01.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.043+0000 7fccd35b6640 1 -- 172.21.15.40:0/1094495956 shutdown_connections 2023-10-18T08:57:01.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.043+0000 7fccd35b6640 1 --2- 172.21.15.40:0/1094495956 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fccb006ae70 0x7fccb006d330 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:01.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.043+0000 7fccd35b6640 1 --2- 172.21.15.40:0/1094495956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcccc15b740 0x7fcccc143ab0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:01.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.043+0000 7fccd35b6640 1 --2- 172.21.15.40:0/1094495956 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcccc0cd3f0 0x7fcccc143570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:01.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.043+0000 7fccd35b6640 1 --2- 172.21.15.40:0/1094495956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcccc0c8480 0x7fcccc1f1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:01.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.043+0000 7fccd35b6640 1 -- 172.21.15.40:0/1094495956 >> 172.21.15.40:0/1094495956 conn(0x7fcccc0cd840 msgr2=0x7fcccc06cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:01.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.043+0000 7fccd35b6640 1 -- 172.21.15.40:0/1094495956 shutdown_connections 2023-10-18T08:57:01.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.044+0000 7fccd35b6640 1 -- 172.21.15.40:0/1094495956 wait complete. 2023-10-18T08:57:01.067 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T08:57:01.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.278+0000 7ff1e88e0640 1 -- 172.21.15.40:0/2307735427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e413c220 msgr2=0x7ff1e40cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:01.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.278+0000 7ff1e88e0640 1 --2- 172.21.15.40:0/2307735427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e413c220 0x7ff1e40cc770 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4071ba0 tx=0x7ff1d402bda0 comp rx=0 tx=0).stop 2023-10-18T08:57:01.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.279+0000 7ff1e88e0640 1 -- 172.21.15.40:0/2307735427 shutdown_connections 2023-10-18T08:57:01.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.279+0000 7ff1e88e0640 1 --2- 172.21.15.40:0/2307735427 >> v2:172.21.15.144:3300/0 conn(0x7ff1e40c9830 0x7ff1e40c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:01.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.279+0000 7ff1e88e0640 1 --2- 172.21.15.40:0/2307735427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e413c220 0x7ff1e40cc770 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:01.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.279+0000 7ff1e88e0640 1 -- 172.21.15.40:0/2307735427 >> 172.21.15.40:0/2307735427 conn(0x7ff1e40aeb90 msgr2=0x7ff1e40aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:01.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.279+0000 7ff1e88e0640 1 -- 172.21.15.40:0/2307735427 shutdown_connections 2023-10-18T08:57:01.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.279+0000 7ff1e88e0640 1 -- 172.21.15.40:0/2307735427 wait complete. 2023-10-18T08:57:01.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.280+0000 7ff1e88e0640 1 Processor -- start 2023-10-18T08:57:01.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.280+0000 7ff1e88e0640 1 -- start start 2023-10-18T08:57:01.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.281+0000 7ff1e88e0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1e40c9830 0x7ff1e41ef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:01.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.281+0000 7ff1e88e0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e413c220 0x7ff1e40aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:01.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.281+0000 7ff1e88e0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1e413eaa0 0x7ff1e40b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:01.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.281+0000 7ff1e88e0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e4151cf0 con 0x7ff1e413c220 2023-10-18T08:57:01.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.281+0000 7ff1e88e0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e40b1f50 con 0x7ff1e413eaa0 2023-10-18T08:57:01.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.281+0000 7ff1e88e0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e40b16b0 con 0x7ff1e40c9830 2023-10-18T08:57:01.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.282+0000 7ff1dbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e413c220 0x7ff1e40aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:01.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.282+0000 7ff1dbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e413c220 0x7ff1e40aa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33840/0 (socket says 172.21.15.40:33840) 2023-10-18T08:57:01.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.282+0000 7ff1dbfff640 1 -- 172.21.15.40:0/747360684 learned_addr learned my addr 172.21.15.40:0/747360684 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:01.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.282+0000 7ff1e2d9d640 1 --2- 172.21.15.40:0/747360684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1e40c9830 0x7ff1e41ef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:01.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.282+0000 7ff1dbfff640 1 -- 172.21.15.40:0/747360684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1e40c9830 msgr2=0x7ff1e41ef4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:01.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.282+0000 7ff1dbfff640 1 --2- 172.21.15.40:0/747360684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1e40c9830 0x7ff1e41ef4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:01.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.282+0000 7ff1e359e640 1 --2- 172.21.15.40:0/747360684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1e413eaa0 0x7ff1e40b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:01.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.282+0000 7ff1dbfff640 1 -- 172.21.15.40:0/747360684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1e413eaa0 msgr2=0x7ff1e40b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:01.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.282+0000 7ff1dbfff640 1 --2- 172.21.15.40:0/747360684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1e413eaa0 0x7ff1e40b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:01.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.282+0000 7ff1dbfff640 1 -- 172.21.15.40:0/747360684 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d4009560 con 0x7ff1e413c220 2023-10-18T08:57:01.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.282+0000 7ff1e2d9d640 1 --2- 172.21.15.40:0/747360684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1e40c9830 0x7ff1e41ef4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:01.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.282+0000 7ff1dbfff640 1 --2- 172.21.15.40:0/747360684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e413c220 0x7ff1e40aa880 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00c890 tx=0x7ff1cc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:01.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.283+0000 7ff1db7fe640 1 -- 172.21.15.40:0/747360684 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc004220 con 0x7ff1e413c220 2023-10-18T08:57:01.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.283+0000 7ff1db7fe640 1 -- 172.21.15.40:0/747360684 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff1cc004470 con 0x7ff1e413c220 2023-10-18T08:57:01.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.283+0000 7ff1db7fe640 1 -- 172.21.15.40:0/747360684 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc007c10 con 0x7ff1e413c220 2023-10-18T08:57:01.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.283+0000 7ff1e88e0640 1 -- 172.21.15.40:0/747360684 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e40b1990 con 0x7ff1e413c220 2023-10-18T08:57:01.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.283+0000 7ff1e88e0640 1 -- 172.21.15.40:0/747360684 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e4155130 con 0x7ff1e413c220 2023-10-18T08:57:01.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.284+0000 7ff1e88e0640 1 -- 172.21.15.40:0/747360684 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e4155350 con 0x7ff1e413c220 2023-10-18T08:57:01.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.285+0000 7ff1db7fe640 1 -- 172.21.15.40:0/747360684 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff1cc005020 con 0x7ff1e413c220 2023-10-18T08:57:01.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.286+0000 7ff1db7fe640 1 --2- 172.21.15.40:0/747360684 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1c406ae40 0x7ff1c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:01.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.286+0000 7ff1db7fe640 1 -- 172.21.15.40:0/747360684 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff1cc089ca0 con 0x7ff1e413c220 2023-10-18T08:57:01.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.287+0000 7ff1e2d9d640 1 --2- 172.21.15.40:0/747360684 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1c406ae40 0x7ff1c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:01.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.288+0000 7ff1e2d9d640 1 --2- 172.21.15.40:0/747360684 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1c406ae40 0x7ff1c406d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4035000 tx=0x7ff1d4004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:01.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.291+0000 7ff1db7fe640 1 -- 172.21.15.40:0/747360684 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff1cc054c50 con 0x7ff1e413c220 2023-10-18T08:57:01.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:01.479+0000 7ff1e88e0640 1 -- 172.21.15.40:0/747360684 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff1e4066e00 con 0x7ff1e413c220 2023-10-18T08:57:02.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.059+0000 7ff1db7fe640 1 -- 172.21.15.40:0/747360684 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff1cc01d070 con 0x7ff1e413c220 2023-10-18T08:57:02.063 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-10-18T08:57:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.064+0000 7ff1e88e0640 1 -- 172.21.15.40:0/747360684 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1c406ae40 msgr2=0x7ff1c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.064+0000 7ff1e88e0640 1 --2- 172.21.15.40:0/747360684 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1c406ae40 0x7ff1c406d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4035000 tx=0x7ff1d4004730 comp rx=0 tx=0).stop 2023-10-18T08:57:02.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.064+0000 7ff1e88e0640 1 -- 172.21.15.40:0/747360684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e413c220 msgr2=0x7ff1e40aa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:02.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.064+0000 7ff1e88e0640 1 --2- 172.21.15.40:0/747360684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e413c220 0x7ff1e40aa880 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00c890 tx=0x7ff1cc00cd50 comp rx=0 tx=0).stop 2023-10-18T08:57:02.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.064+0000 7ff1e88e0640 1 -- 172.21.15.40:0/747360684 shutdown_connections 2023-10-18T08:57:02.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.065+0000 7ff1e88e0640 1 --2- 172.21.15.40:0/747360684 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1c406ae40 0x7ff1c406d300 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:02.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.065+0000 7ff1e88e0640 1 --2- 172.21.15.40:0/747360684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1e413eaa0 0x7ff1e40b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:02.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.065+0000 7ff1e88e0640 1 --2- 172.21.15.40:0/747360684 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1e413c220 0x7ff1e40aa880 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:02.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.065+0000 7ff1e88e0640 1 --2- 172.21.15.40:0/747360684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1e40c9830 0x7ff1e41ef4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:02.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.065+0000 7ff1e88e0640 1 -- 172.21.15.40:0/747360684 >> 172.21.15.40:0/747360684 conn(0x7ff1e40aeb90 msgr2=0x7ff1e40c9080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:02.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.065+0000 7ff1e88e0640 1 -- 172.21.15.40:0/747360684 shutdown_connections 2023-10-18T08:57:02.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.065+0000 7ff1e88e0640 1 -- 172.21.15.40:0/747360684 wait complete. 2023-10-18T08:57:02.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T08:57:02.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.309+0000 7f9799267640 1 -- 172.21.15.40:0/3856500135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f979414e350 msgr2=0x7f979406b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.309+0000 7f9799267640 1 --2- 172.21.15.40:0/3856500135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f979414e350 0x7f979406b710 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9784005fd0 tx=0x7f978402c240 comp rx=0 tx=0).stop 2023-10-18T08:57:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.309+0000 7f9799267640 1 -- 172.21.15.40:0/3856500135 shutdown_connections 2023-10-18T08:57:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.309+0000 7f9799267640 1 --2- 172.21.15.40:0/3856500135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f979406fbc0 0x7f97940cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.309+0000 7f9799267640 1 --2- 172.21.15.40:0/3856500135 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f979414e350 0x7f979406b710 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.309+0000 7f9799267640 1 -- 172.21.15.40:0/3856500135 >> 172.21.15.40:0/3856500135 conn(0x7f97940aeb90 msgr2=0x7f97940aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.309+0000 7f9799267640 1 -- 172.21.15.40:0/3856500135 shutdown_connections 2023-10-18T08:57:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.309+0000 7f9799267640 1 -- 172.21.15.40:0/3856500135 wait complete. 2023-10-18T08:57:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.311+0000 7f9799267640 1 Processor -- start 2023-10-18T08:57:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.312+0000 7f9799267640 1 -- start start 2023-10-18T08:57:02.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.313+0000 7f9799267640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f979406fbc0 0x7f9794150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:02.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.313+0000 7f9799267640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9794070020 0x7f979414d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:02.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.313+0000 7f9799267640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f979414e350 0x7f9794147000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:02.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.313+0000 7f9799267640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9794156290 con 0x7f979406fbc0 2023-10-18T08:57:02.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.313+0000 7f9799267640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97941512d0 con 0x7f9794070020 2023-10-18T08:57:02.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.313+0000 7f9799267640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97941515b0 con 0x7f979414e350 2023-10-18T08:57:02.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.313+0000 7f9793fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f979406fbc0 0x7f9794150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.313+0000 7f9793fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f979406fbc0 0x7f9794150cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33852/0 (socket says 172.21.15.40:33852) 2023-10-18T08:57:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.313+0000 7f9793fff640 1 -- 172.21.15.40:0/15078694 learned_addr learned my addr 172.21.15.40:0/15078694 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.314+0000 7f97937fe640 1 --2- 172.21.15.40:0/15078694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9794070020 0x7f979414d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:02.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.314+0000 7f9793fff640 1 -- 172.21.15.40:0/15078694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f979414e350 msgr2=0x7f9794147000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:02.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.314+0000 7f9798a66640 1 --2- 172.21.15.40:0/15078694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f979414e350 0x7f9794147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:02.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.314+0000 7f9793fff640 1 --2- 172.21.15.40:0/15078694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f979414e350 0x7f9794147000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:02.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.314+0000 7f9793fff640 1 -- 172.21.15.40:0/15078694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9794070020 msgr2=0x7f979414d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:02.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.314+0000 7f9793fff640 1 --2- 172.21.15.40:0/15078694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9794070020 0x7f979414d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:02.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.314+0000 7f9793fff640 1 -- 172.21.15.40:0/15078694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9784009900 con 0x7f979406fbc0 2023-10-18T08:57:02.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.314+0000 7f9793fff640 1 --2- 172.21.15.40:0/15078694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f979406fbc0 0x7f9794150cd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9784006010 tx=0x7f9784035ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:02.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.314+0000 7f9790ff9640 1 -- 172.21.15.40:0/15078694 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9784034440 con 0x7f979406fbc0 2023-10-18T08:57:02.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.314+0000 7f9790ff9640 1 -- 172.21.15.40:0/15078694 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9784004330 con 0x7f979406fbc0 2023-10-18T08:57:02.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.314+0000 7f9790ff9640 1 -- 172.21.15.40:0/15078694 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97840046e0 con 0x7f979406fbc0 2023-10-18T08:57:02.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.315+0000 7f9799267640 1 -- 172.21.15.40:0/15078694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9794151890 con 0x7f979406fbc0 2023-10-18T08:57:02.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.315+0000 7f9799267640 1 -- 172.21.15.40:0/15078694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9794147b70 con 0x7f979406fbc0 2023-10-18T08:57:02.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.316+0000 7f9799267640 1 -- 172.21.15.40:0/15078694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9794147d00 con 0x7f979406fbc0 2023-10-18T08:57:02.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.317+0000 7f9790ff9640 1 -- 172.21.15.40:0/15078694 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f978403b040 con 0x7f979406fbc0 2023-10-18T08:57:02.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.318+0000 7f9790ff9640 1 --2- 172.21.15.40:0/15078694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f978006ae70 0x7f978006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:02.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.318+0000 7f9790ff9640 1 -- 172.21.15.40:0/15078694 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f97840ac160 con 0x7f979406fbc0 2023-10-18T08:57:02.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.318+0000 7f97937fe640 1 --2- 172.21.15.40:0/15078694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f978006ae70 0x7f978006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:02.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.320+0000 7f97937fe640 1 --2- 172.21.15.40:0/15078694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f978006ae70 0x7f978006d330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9788005e10 tx=0x7f9788005da0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:02.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.322+0000 7f9790ff9640 1 -- 172.21.15.40:0/15078694 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9784077480 con 0x7f979406fbc0 2023-10-18T08:57:02.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:02.515+0000 7f9799267640 1 -- 172.21.15.40:0/15078694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9794066e00 con 0x7f979406fbc0 2023-10-18T08:57:03.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.070+0000 7f9790ff9640 1 -- 172.21.15.40:0/15078694 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) v1 ==== 169+0+0 (secure 0 0 0) 0x7f978407aaa0 con 0x7f979406fbc0 2023-10-18T08:57:03.075 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-10-18T08:57:03.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.076+0000 7f9799267640 1 -- 172.21.15.40:0/15078694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f978006ae70 msgr2=0x7f978006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:03.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.076+0000 7f9799267640 1 --2- 172.21.15.40:0/15078694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f978006ae70 0x7f978006d330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9788005e10 tx=0x7f9788005da0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.076+0000 7f9799267640 1 -- 172.21.15.40:0/15078694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f979406fbc0 msgr2=0x7f9794150cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:03.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.076+0000 7f9799267640 1 --2- 172.21.15.40:0/15078694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f979406fbc0 0x7f9794150cd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9784006010 tx=0x7f9784035ac0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.076+0000 7f9799267640 1 -- 172.21.15.40:0/15078694 shutdown_connections 2023-10-18T08:57:03.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.076+0000 7f9799267640 1 --2- 172.21.15.40:0/15078694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f978006ae70 0x7f978006d330 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.076+0000 7f9799267640 1 --2- 172.21.15.40:0/15078694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f979414e350 0x7f9794147000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.076+0000 7f9799267640 1 --2- 172.21.15.40:0/15078694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9794070020 0x7f979414d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.076+0000 7f9799267640 1 --2- 172.21.15.40:0/15078694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f979406fbc0 0x7f9794150cd0 secure :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9784006010 tx=0x7f9784035ac0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.076+0000 7f9799267640 1 -- 172.21.15.40:0/15078694 >> 172.21.15.40:0/15078694 conn(0x7f97940aeb90 msgr2=0x7f97940ad240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:03.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.077+0000 7f9799267640 1 -- 172.21.15.40:0/15078694 shutdown_connections 2023-10-18T08:57:03.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.077+0000 7f9799267640 1 -- 172.21.15.40:0/15078694 wait complete. 2023-10-18T08:57:03.105 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 22 2023-10-18T08:57:03.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.331+0000 7fcff48a9640 1 -- 172.21.15.40:0/153510964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff015a470 msgr2=0x7fcff00c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:03.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.331+0000 7fcff48a9640 1 --2- 172.21.15.40:0/153510964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff015a470 0x7fcff00c9b00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009830 tx=0x7fcfd802bdf0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.331+0000 7fcff48a9640 1 -- 172.21.15.40:0/153510964 shutdown_connections 2023-10-18T08:57:03.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.331+0000 7fcff48a9640 1 --2- 172.21.15.40:0/153510964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff015a470 0x7fcff00c9b00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.331+0000 7fcff48a9640 1 -- 172.21.15.40:0/153510964 >> 172.21.15.40:0/153510964 conn(0x7fcff006cee0 msgr2=0x7fcff0069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:03.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.331+0000 7fcff48a9640 1 -- 172.21.15.40:0/153510964 shutdown_connections 2023-10-18T08:57:03.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.331+0000 7fcff48a9640 1 -- 172.21.15.40:0/153510964 wait complete. 2023-10-18T08:57:03.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.333+0000 7fcff48a9640 1 Processor -- start 2023-10-18T08:57:03.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.333+0000 7fcff48a9640 1 -- start start 2023-10-18T08:57:03.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.334+0000 7fcff48a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff006b710 0x7fcff01e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:03.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.334+0000 7fcff48a9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcff00cdbb0 0x7fcff01529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:03.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.334+0000 7fcff48a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff015a470 0x7fcff015aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:03.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.334+0000 7fcff48a9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff01e7fa0 con 0x7fcff006b710 2023-10-18T08:57:03.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.334+0000 7fcff48a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff0151800 con 0x7fcff015a470 2023-10-18T08:57:03.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.334+0000 7fcff48a9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff0151ae0 con 0x7fcff00cdbb0 2023-10-18T08:57:03.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.334+0000 7fcfee59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff006b710 0x7fcff01e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:03.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.334+0000 7fcfee59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff006b710 0x7fcff01e7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33870/0 (socket says 172.21.15.40:33870) 2023-10-18T08:57:03.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.334+0000 7fcfee59c640 1 -- 172.21.15.40:0/1316927702 learned_addr learned my addr 172.21.15.40:0/1316927702 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:03.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.334+0000 7fcfee59c640 1 -- 172.21.15.40:0/1316927702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcff00cdbb0 msgr2=0x7fcff01529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:03.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.334+0000 7fcfee59c640 1 --2- 172.21.15.40:0/1316927702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcff00cdbb0 0x7fcff01529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.334+0000 7fcfee59c640 1 -- 172.21.15.40:0/1316927702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff015a470 msgr2=0x7fcff015aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:03.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.334+0000 7fcfeed9d640 1 --2- 172.21.15.40:0/1316927702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff015a470 0x7fcff015aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:03.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.335+0000 7fcfee59c640 1 --2- 172.21.15.40:0/1316927702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff015a470 0x7fcff015aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.335+0000 7fcfee59c640 1 -- 172.21.15.40:0/1316927702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd8009490 con 0x7fcff006b710 2023-10-18T08:57:03.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.335+0000 7fcfeed9d640 1 --2- 172.21.15.40:0/1316927702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff015a470 0x7fcff015aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:03.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.335+0000 7fcfee59c640 1 --2- 172.21.15.40:0/1316927702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff006b710 0x7fcff01e7a60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000c450 tx=0x7fcfe000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:03.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.335+0000 7fcfe6ffd640 1 -- 172.21.15.40:0/1316927702 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe00052d0 con 0x7fcff006b710 2023-10-18T08:57:03.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.335+0000 7fcfe6ffd640 1 -- 172.21.15.40:0/1316927702 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcfe0005430 con 0x7fcff006b710 2023-10-18T08:57:03.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.335+0000 7fcfe6ffd640 1 -- 172.21.15.40:0/1316927702 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0012890 con 0x7fcff006b710 2023-10-18T08:57:03.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.335+0000 7fcff48a9640 1 -- 172.21.15.40:0/1316927702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff0151dc0 con 0x7fcff006b710 2023-10-18T08:57:03.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.335+0000 7fcff48a9640 1 -- 172.21.15.40:0/1316927702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff0151370 con 0x7fcff006b710 2023-10-18T08:57:03.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.337+0000 7fcff48a9640 1 -- 172.21.15.40:0/1316927702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff01515c0 con 0x7fcff006b710 2023-10-18T08:57:03.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.338+0000 7fcfe6ffd640 1 -- 172.21.15.40:0/1316927702 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcfe00035f0 con 0x7fcff006b710 2023-10-18T08:57:03.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.339+0000 7fcfe6ffd640 1 --2- 172.21.15.40:0/1316927702 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfd006ae40 0x7fcfd006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:03.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.339+0000 7fcfe6ffd640 1 -- 172.21.15.40:0/1316927702 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcfe0088fe0 con 0x7fcff006b710 2023-10-18T08:57:03.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.340+0000 7fcfe7fff640 1 --2- 172.21.15.40:0/1316927702 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfd006ae40 0x7fcfd006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:03.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.341+0000 7fcfe7fff640 1 --2- 172.21.15.40:0/1316927702 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfd006ae40 0x7fcfd006d300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8036000 tx=0x7fcfd80046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:03.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.343+0000 7fcfe6ffd640 1 -- 172.21.15.40:0/1316927702 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcff01515c0 con 0x7fcff006b710 2023-10-18T08:57:03.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.537+0000 7fcff48a9640 1 -- 172.21.15.40:0/1316927702 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config reset", "num": 22} v 0) v1 -- 0x7fcff01515c0 con 0x7fcff006b710 2023-10-18T08:57:03.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.543+0000 7fcfe6ffd640 1 -- 172.21.15.40:0/1316927702 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 22}]=0 v24) v1 ==== 71+0+0 (secure 0 0 0) 0x7fcff01515c0 con 0x7fcff006b710 2023-10-18T08:57:03.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.546+0000 7fcff48a9640 1 -- 172.21.15.40:0/1316927702 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfd006ae40 msgr2=0x7fcfd006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:03.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.546+0000 7fcff48a9640 1 --2- 172.21.15.40:0/1316927702 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfd006ae40 0x7fcfd006d300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8036000 tx=0x7fcfd80046c0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.546+0000 7fcff48a9640 1 -- 172.21.15.40:0/1316927702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff006b710 msgr2=0x7fcff01e7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:03.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.546+0000 7fcff48a9640 1 --2- 172.21.15.40:0/1316927702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff006b710 0x7fcff01e7a60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000c450 tx=0x7fcfe000c910 comp rx=0 tx=0).stop 2023-10-18T08:57:03.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.546+0000 7fcff48a9640 1 -- 172.21.15.40:0/1316927702 shutdown_connections 2023-10-18T08:57:03.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.546+0000 7fcff48a9640 1 --2- 172.21.15.40:0/1316927702 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfd006ae40 0x7fcfd006d300 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.546+0000 7fcff48a9640 1 --2- 172.21.15.40:0/1316927702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcff015a470 0x7fcff015aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.546+0000 7fcff48a9640 1 --2- 172.21.15.40:0/1316927702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcff00cdbb0 0x7fcff01529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.546+0000 7fcff48a9640 1 --2- 172.21.15.40:0/1316927702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcff006b710 0x7fcff01e7a60 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.546+0000 7fcff48a9640 1 -- 172.21.15.40:0/1316927702 >> 172.21.15.40:0/1316927702 conn(0x7fcff006cee0 msgr2=0x7fcff0070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:03.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.546+0000 7fcff48a9640 1 -- 172.21.15.40:0/1316927702 shutdown_connections 2023-10-18T08:57:03.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.546+0000 7fcff48a9640 1 -- 172.21.15.40:0/1316927702 wait complete. 2023-10-18T08:57:03.569 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2023-10-18T08:57:03.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.803+0000 7f78cbfff640 1 -- 172.21.15.40:0/1481634089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc0cc190 msgr2=0x7f78cc0b1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:03.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.803+0000 7f78cbfff640 1 --2- 172.21.15.40:0/1481634089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc0cc190 0x7f78cc0b1de0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f78cc06a2a0 tx=0x7f78b402bd70 comp rx=0 tx=0).stop 2023-10-18T08:57:03.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.803+0000 7f78cbfff640 1 -- 172.21.15.40:0/1481634089 shutdown_connections 2023-10-18T08:57:03.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.803+0000 7f78cbfff640 1 --2- 172.21.15.40:0/1481634089 >> v2:172.21.15.144:3300/0 conn(0x7f78cc0b2320 0x7f78cc06f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.803+0000 7f78cbfff640 1 --2- 172.21.15.40:0/1481634089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc0cc190 0x7f78cc0b1de0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.803+0000 7f78cbfff640 1 -- 172.21.15.40:0/1481634089 >> 172.21.15.40:0/1481634089 conn(0x7f78cc069720 msgr2=0x7f78cc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:03.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.803+0000 7f78cbfff640 1 -- 172.21.15.40:0/1481634089 shutdown_connections 2023-10-18T08:57:03.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.804+0000 7f78cbfff640 1 -- 172.21.15.40:0/1481634089 wait complete. 2023-10-18T08:57:03.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.805+0000 7f78cbfff640 1 Processor -- start 2023-10-18T08:57:03.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.805+0000 7f78cbfff640 1 -- start start 2023-10-18T08:57:03.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.806+0000 7f78cbfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78cc0cc190 0x7f78cc0c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:03.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.806+0000 7f78cbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78cc141c80 0x7f78cc13c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:03.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.806+0000 7f78cbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc1f1860 0x7f78cc15a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:03.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.806+0000 7f78cbfff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc0c2810 con 0x7f78cc1f1860 2023-10-18T08:57:03.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.806+0000 7f78cbfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc0c2af0 con 0x7f78cc141c80 2023-10-18T08:57:03.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.806+0000 7f78cbfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc0c2dd0 con 0x7f78cc0cc190 2023-10-18T08:57:03.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.807+0000 7f78cb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc1f1860 0x7f78cc15a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:03.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.807+0000 7f78cb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc1f1860 0x7f78cc15a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33882/0 (socket says 172.21.15.40:33882) 2023-10-18T08:57:03.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.807+0000 7f78cb7fe640 1 -- 172.21.15.40:0/1466256507 learned_addr learned my addr 172.21.15.40:0/1466256507 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:03.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.807+0000 7f78caffd640 1 --2- 172.21.15.40:0/1466256507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78cc0cc190 0x7f78cc0c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:03.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.807+0000 7f78c3fff640 1 --2- 172.21.15.40:0/1466256507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78cc141c80 0x7f78cc13c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:03.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.807+0000 7f78cb7fe640 1 -- 172.21.15.40:0/1466256507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78cc0cc190 msgr2=0x7f78cc0c1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:03.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.807+0000 7f78cb7fe640 1 --2- 172.21.15.40:0/1466256507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78cc0cc190 0x7f78cc0c1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.807+0000 7f78cb7fe640 1 -- 172.21.15.40:0/1466256507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78cc141c80 msgr2=0x7f78cc13c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:03.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.807+0000 7f78cb7fe640 1 --2- 172.21.15.40:0/1466256507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78cc141c80 0x7f78cc13c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:03.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.807+0000 7f78cb7fe640 1 -- 172.21.15.40:0/1466256507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b4009490 con 0x7f78cc1f1860 2023-10-18T08:57:03.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.807+0000 7f78c3fff640 1 --2- 172.21.15.40:0/1466256507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78cc141c80 0x7f78cc13c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:03.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.808+0000 7f78cb7fe640 1 --2- 172.21.15.40:0/1466256507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc1f1860 0x7f78cc15a7f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f78c400eaa0 tx=0x7f78c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:03.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.808+0000 7f78c37fe640 1 -- 172.21.15.40:0/1466256507 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c400cdb0 con 0x7f78cc1f1860 2023-10-18T08:57:03.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.808+0000 7f78c37fe640 1 -- 172.21.15.40:0/1466256507 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f78c4004b20 con 0x7f78cc1f1860 2023-10-18T08:57:03.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.808+0000 7f78c37fe640 1 -- 172.21.15.40:0/1466256507 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4005020 con 0x7f78cc1f1860 2023-10-18T08:57:03.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.808+0000 7f78cbfff640 1 -- 172.21.15.40:0/1466256507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78cc0c30b0 con 0x7f78cc1f1860 2023-10-18T08:57:03.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.808+0000 7f78cbfff640 1 -- 172.21.15.40:0/1466256507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78cc0c2520 con 0x7f78cc1f1860 2023-10-18T08:57:03.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.809+0000 7f78cbfff640 1 -- 172.21.15.40:0/1466256507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78cc066e00 con 0x7f78cc1f1860 2023-10-18T08:57:03.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.810+0000 7f78c37fe640 1 -- 172.21.15.40:0/1466256507 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f78c4005180 con 0x7f78cc1f1860 2023-10-18T08:57:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.811+0000 7f78c37fe640 1 --2- 172.21.15.40:0/1466256507 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78ac06ae70 0x7f78ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.811+0000 7f78c37fe640 1 -- 172.21.15.40:0/1466256507 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f78c4089d70 con 0x7f78cc1f1860 2023-10-18T08:57:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.812+0000 7f78caffd640 1 --2- 172.21.15.40:0/1466256507 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78ac06ae70 0x7f78ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:03.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.813+0000 7f78caffd640 1 --2- 172.21.15.40:0/1466256507 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78ac06ae70 0x7f78ac06d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0015b0 tx=0x7f78bc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:03.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:03.816+0000 7f78c37fe640 1 -- 172.21.15.40:0/1466256507 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78c4055080 con 0x7f78cc1f1860 2023-10-18T08:57:04.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:04.080+0000 7f78cbfff640 1 -- 172.21.15.40:0/1466256507 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7f78cc070530 con 0x7f78cc1f1860 2023-10-18T08:57:04.567 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:57:04.577 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:57:05.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.083+0000 7f78c37fe640 1 -- 172.21.15.40:0/1466256507 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v123) v1 ==== 130+0+0 (secure 0 0 0) 0x7f78c401f020 con 0x7f78cc1f1860 2023-10-18T08:57:05.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.086+0000 7f78cbfff640 1 -- 172.21.15.40:0/1466256507 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78ac06ae70 msgr2=0x7f78ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:05.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.086+0000 7f78cbfff640 1 --2- 172.21.15.40:0/1466256507 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78ac06ae70 0x7f78ac06d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0015b0 tx=0x7f78bc009040 comp rx=0 tx=0).stop 2023-10-18T08:57:05.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.086+0000 7f78cbfff640 1 -- 172.21.15.40:0/1466256507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc1f1860 msgr2=0x7f78cc15a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:05.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.086+0000 7f78cbfff640 1 --2- 172.21.15.40:0/1466256507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc1f1860 0x7f78cc15a7f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f78c400eaa0 tx=0x7f78c400ef60 comp rx=0 tx=0).stop 2023-10-18T08:57:05.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.086+0000 7f78cbfff640 1 -- 172.21.15.40:0/1466256507 shutdown_connections 2023-10-18T08:57:05.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.086+0000 7f78cbfff640 1 --2- 172.21.15.40:0/1466256507 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78ac06ae70 0x7f78ac06d330 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:05.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.086+0000 7f78cbfff640 1 --2- 172.21.15.40:0/1466256507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc1f1860 0x7f78cc15a7f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:05.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.086+0000 7f78cbfff640 1 --2- 172.21.15.40:0/1466256507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78cc141c80 0x7f78cc13c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:05.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.086+0000 7f78cbfff640 1 --2- 172.21.15.40:0/1466256507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78cc0cc190 0x7f78cc0c1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:05.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.086+0000 7f78cbfff640 1 -- 172.21.15.40:0/1466256507 >> 172.21.15.40:0/1466256507 conn(0x7f78cc069720 msgr2=0x7f78cc072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:05.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.087+0000 7f78cbfff640 1 -- 172.21.15.40:0/1466256507 shutdown_connections 2023-10-18T08:57:05.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.087+0000 7f78cbfff640 1 -- 172.21.15.40:0/1466256507 wait complete. 2023-10-18T08:57:05.112 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2023-10-18T08:57:05.112 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestClusterFull) 2023-10-18T08:57:05.113 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:57:05.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.341+0000 7f2029bf4640 1 -- 172.21.15.40:0/106527532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20240b1e10 msgr2=0x7f20240c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:05.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.341+0000 7f2029bf4640 1 --2- 172.21.15.40:0/106527532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20240b1e10 0x7f20240c99c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f201000a940 tx=0x7f201002d140 comp rx=0 tx=0).stop 2023-10-18T08:57:05.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.341+0000 7f2029bf4640 1 -- 172.21.15.40:0/106527532 shutdown_connections 2023-10-18T08:57:05.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.341+0000 7f2029bf4640 1 --2- 172.21.15.40:0/106527532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20240b1e10 0x7f20240c99c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.341+0000 7f2029bf4640 1 -- 172.21.15.40:0/106527532 >> 172.21.15.40:0/106527532 conn(0x7f2024069720 msgr2=0x7f2024069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.341+0000 7f2029bf4640 1 -- 172.21.15.40:0/106527532 shutdown_connections 2023-10-18T08:57:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.341+0000 7f2029bf4640 1 -- 172.21.15.40:0/106527532 wait complete. 2023-10-18T08:57:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.343+0000 7f2029bf4640 1 Processor -- start 2023-10-18T08:57:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.343+0000 7f2029bf4640 1 -- start start 2023-10-18T08:57:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.344+0000 7f2029bf4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20240b15b0 0x7f2024148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.344+0000 7f2029bf4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20240b1e10 0x7f20241520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.344+0000 7f2029bf4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20240b2270 0x7f2024150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:05.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.344+0000 7f2029bf4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20241506e0 con 0x7f20240b15b0 2023-10-18T08:57:05.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.344+0000 7f2029bf4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202414b010 con 0x7f20240b1e10 2023-10-18T08:57:05.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.344+0000 7f2029bf4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202414b2f0 con 0x7f20240b2270 2023-10-18T08:57:05.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.345+0000 7f2028bf2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20240b15b0 0x7f2024148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:05.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.345+0000 7f2028bf2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20240b15b0 0x7f2024148110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40796/0 (socket says 172.21.15.40:40796) 2023-10-18T08:57:05.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.345+0000 7f2028bf2640 1 -- 172.21.15.40:0/2441842074 learned_addr learned my addr 172.21.15.40:0/2441842074 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:05.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.345+0000 7f2023fff640 1 --2- 172.21.15.40:0/2441842074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20240b1e10 0x7f20241520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:05.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.345+0000 7f20293f3640 1 --2- 172.21.15.40:0/2441842074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20240b2270 0x7f2024150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:05.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.345+0000 7f2028bf2640 1 -- 172.21.15.40:0/2441842074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20240b2270 msgr2=0x7f2024150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:05.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.345+0000 7f2028bf2640 1 --2- 172.21.15.40:0/2441842074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20240b2270 0x7f2024150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:05.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.345+0000 7f2028bf2640 1 -- 172.21.15.40:0/2441842074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20240b1e10 msgr2=0x7f20241520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:05.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.345+0000 7f2028bf2640 1 --2- 172.21.15.40:0/2441842074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20240b1e10 0x7f20241520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:05.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.345+0000 7f2028bf2640 1 -- 172.21.15.40:0/2441842074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201000a5a0 con 0x7f20240b15b0 2023-10-18T08:57:05.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.345+0000 7f2023fff640 1 --2- 172.21.15.40:0/2441842074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20240b1e10 0x7f20241520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:57:05.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.346+0000 7f2028bf2640 1 --2- 172.21.15.40:0/2441842074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20240b15b0 0x7f2024148110 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f201800c490 tx=0x7f201800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:05.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.346+0000 7f20217fa640 1 -- 172.21.15.40:0/2441842074 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018012830 con 0x7f20240b15b0 2023-10-18T08:57:05.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.346+0000 7f20217fa640 1 -- 172.21.15.40:0/2441842074 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2018012a80 con 0x7f20240b15b0 2023-10-18T08:57:05.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.346+0000 7f20217fa640 1 -- 172.21.15.40:0/2441842074 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018011380 con 0x7f20240b15b0 2023-10-18T08:57:05.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.346+0000 7f2029bf4640 1 -- 172.21.15.40:0/2441842074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2024150980 con 0x7f20240b15b0 2023-10-18T08:57:05.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.346+0000 7f2029bf4640 1 -- 172.21.15.40:0/2441842074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2024148650 con 0x7f20240b15b0 2023-10-18T08:57:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.347+0000 7f2029bf4640 1 -- 172.21.15.40:0/2441842074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2024066e00 con 0x7f20240b15b0 2023-10-18T08:57:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.348+0000 7f20217fa640 1 -- 172.21.15.40:0/2441842074 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f20180114e0 con 0x7f20240b15b0 2023-10-18T08:57:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.349+0000 7f20217fa640 1 --2- 172.21.15.40:0/2441842074 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f200806ae40 0x7f200806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.349+0000 7f20217fa640 1 -- 172.21.15.40:0/2441842074 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2018089500 con 0x7f20240b15b0 2023-10-18T08:57:05.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.350+0000 7f2023fff640 1 --2- 172.21.15.40:0/2441842074 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f200806ae40 0x7f200806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:05.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.351+0000 7f2023fff640 1 --2- 172.21.15.40:0/2441842074 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f200806ae40 0x7f200806d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2010037000 tx=0x7f2010004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:05.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.353+0000 7f20217fa640 1 -- 172.21.15.40:0/2441842074 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2018058330 con 0x7f20240b15b0 2023-10-18T08:57:05.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.542+0000 7f2029bf4640 1 -- 172.21.15.40:0/2441842074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20240caf90 con 0x7f20240b15b0 2023-10-18T08:57:05.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.544+0000 7f20217fa640 1 -- 172.21.15.40:0/2441842074 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+12998 (secure 0 0 0) 0x7f2018057ec0 con 0x7f20240b15b0 2023-10-18T08:57:05.548 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:05.548 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":38,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:57:03.062148+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3167454356":"2023-10-19T08:57:00.004125+0000","172.21.15.40:6834/3167454356":"2023-10-19T08:57:00.004125+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:57:05.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.546+0000 7f2029bf4640 1 -- 172.21.15.40:0/2441842074 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f200806ae40 msgr2=0x7f200806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:05.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.546+0000 7f2029bf4640 1 --2- 172.21.15.40:0/2441842074 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f200806ae40 0x7f200806d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2010037000 tx=0x7f2010004730 comp rx=0 tx=0).stop 2023-10-18T08:57:05.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.546+0000 7f2029bf4640 1 -- 172.21.15.40:0/2441842074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20240b15b0 msgr2=0x7f2024148110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:05.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.546+0000 7f2029bf4640 1 --2- 172.21.15.40:0/2441842074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20240b15b0 0x7f2024148110 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f201800c490 tx=0x7f201800c950 comp rx=0 tx=0).stop 2023-10-18T08:57:05.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.546+0000 7f2029bf4640 1 -- 172.21.15.40:0/2441842074 shutdown_connections 2023-10-18T08:57:05.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.546+0000 7f2029bf4640 1 --2- 172.21.15.40:0/2441842074 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f200806ae40 0x7f200806d300 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:05.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.546+0000 7f2029bf4640 1 --2- 172.21.15.40:0/2441842074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20240b2270 0x7f2024150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:05.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.546+0000 7f2029bf4640 1 --2- 172.21.15.40:0/2441842074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20240b1e10 0x7f20241520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:05.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.546+0000 7f2029bf4640 1 --2- 172.21.15.40:0/2441842074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20240b15b0 0x7f2024148110 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:05.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.546+0000 7f2029bf4640 1 -- 172.21.15.40:0/2441842074 >> 172.21.15.40:0/2441842074 conn(0x7f2024069720 msgr2=0x7f202414fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:05.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.547+0000 7f2029bf4640 1 -- 172.21.15.40:0/2441842074 shutdown_connections 2023-10-18T08:57:05.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.547+0000 7f2029bf4640 1 -- 172.21.15.40:0/2441842074 wait complete. 2023-10-18T08:57:05.567 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:57:05.567 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:57:05.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.793+0000 7fb267fff640 1 -- 172.21.15.40:0/756685660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb268069b80 msgr2=0x7fb268159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.793+0000 7fb267fff640 1 --2- 172.21.15.40:0/756685660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb268069b80 0x7fb268159db0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb2680cded0 tx=0x7fb26002c260 comp rx=0 tx=0).stop 2023-10-18T08:57:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.793+0000 7fb267fff640 1 -- 172.21.15.40:0/756685660 shutdown_connections 2023-10-18T08:57:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.793+0000 7fb267fff640 1 --2- 172.21.15.40:0/756685660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb268069b80 0x7fb268159db0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.793+0000 7fb267fff640 1 --2- 172.21.15.40:0/756685660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb268069720 0x7fb268159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.793+0000 7fb267fff640 1 --2- 172.21.15.40:0/756685660 >> v2:172.21.15.144:3300/0 conn(0x7fb2680cc880 0x7fb2680ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:05.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.793+0000 7fb267fff640 1 -- 172.21.15.40:0/756685660 >> 172.21.15.40:0/756685660 conn(0x7fb2680cdf70 msgr2=0x7fb2680ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:05.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.793+0000 7fb267fff640 1 -- 172.21.15.40:0/756685660 shutdown_connections 2023-10-18T08:57:05.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.793+0000 7fb267fff640 1 -- 172.21.15.40:0/756685660 wait complete. 2023-10-18T08:57:05.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.795+0000 7fb267fff640 1 Processor -- start 2023-10-18T08:57:05.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.795+0000 7fb267fff640 1 -- start start 2023-10-18T08:57:05.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.796+0000 7fb267fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb268069720 0x7fb2681f1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:05.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.796+0000 7fb267fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb268069b80 0x7fb268152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:05.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.796+0000 7fb267fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2680cc880 0x7fb268152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:05.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.796+0000 7fb267fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb268155c90 con 0x7fb2680cc880 2023-10-18T08:57:05.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.796+0000 7fb267fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb268155f70 con 0x7fb268069b80 2023-10-18T08:57:05.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.796+0000 7fb267fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb268156250 con 0x7fb268069720 2023-10-18T08:57:05.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.796+0000 7fb2677fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2680cc880 0x7fb268152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:05.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.796+0000 7fb2677fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2680cc880 0x7fb268152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40816/0 (socket says 172.21.15.40:40816) 2023-10-18T08:57:05.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.796+0000 7fb2677fe640 1 -- 172.21.15.40:0/1412712252 learned_addr learned my addr 172.21.15.40:0/1412712252 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:05.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.796+0000 7fb266ffd640 1 --2- 172.21.15.40:0/1412712252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb268069720 0x7fb2681f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.796+0000 7fb25e7fc640 1 --2- 172.21.15.40:0/1412712252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb268069b80 0x7fb268152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.797+0000 7fb2677fe640 1 -- 172.21.15.40:0/1412712252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb268069720 msgr2=0x7fb2681f1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.797+0000 7fb2677fe640 1 --2- 172.21.15.40:0/1412712252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb268069720 0x7fb2681f1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.797+0000 7fb2677fe640 1 -- 172.21.15.40:0/1412712252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb268069b80 msgr2=0x7fb268152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.797+0000 7fb2677fe640 1 --2- 172.21.15.40:0/1412712252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb268069b80 0x7fb268152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.797+0000 7fb2677fe640 1 -- 172.21.15.40:0/1412712252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb260009900 con 0x7fb2680cc880 2023-10-18T08:57:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.797+0000 7fb2677fe640 1 --2- 172.21.15.40:0/1412712252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2680cc880 0x7fb268152cc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb260037040 tx=0x7fb260003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:05.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.797+0000 7fb25ffff640 1 -- 172.21.15.40:0/1412712252 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb260004250 con 0x7fb2680cc880 2023-10-18T08:57:05.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.797+0000 7fb25ffff640 1 -- 172.21.15.40:0/1412712252 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb2600044a0 con 0x7fb2680cc880 2023-10-18T08:57:05.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.797+0000 7fb25ffff640 1 -- 172.21.15.40:0/1412712252 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb260004890 con 0x7fb2680cc880 2023-10-18T08:57:05.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.797+0000 7fb267fff640 1 -- 172.21.15.40:0/1412712252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb268156530 con 0x7fb2680cc880 2023-10-18T08:57:05.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.798+0000 7fb267fff640 1 -- 172.21.15.40:0/1412712252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2681f1660 con 0x7fb2680cc880 2023-10-18T08:57:05.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.799+0000 7fb267fff640 1 -- 172.21.15.40:0/1412712252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2680cc6f0 con 0x7fb2680cc880 2023-10-18T08:57:05.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.799+0000 7fb25ffff640 1 -- 172.21.15.40:0/1412712252 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb26003b040 con 0x7fb2680cc880 2023-10-18T08:57:05.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.800+0000 7fb25ffff640 1 --2- 172.21.15.40:0/1412712252 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb24806ae40 0x7fb24806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:05.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.800+0000 7fb25ffff640 1 -- 172.21.15.40:0/1412712252 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb2600ac250 con 0x7fb2680cc880 2023-10-18T08:57:05.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.801+0000 7fb266ffd640 1 --2- 172.21.15.40:0/1412712252 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb24806ae40 0x7fb24806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:05.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.802+0000 7fb266ffd640 1 --2- 172.21.15.40:0/1412712252 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb24806ae40 0x7fb24806d300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb25800ca40 tx=0x7fb25800c420 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:05.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.806+0000 7fb25ffff640 1 -- 172.21.15.40:0/1412712252 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb260077560 con 0x7fb2680cc880 2023-10-18T08:57:06.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.998+0000 7fb267fff640 1 -- 172.21.15.40:0/1412712252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb268073570 con 0x7fb2680cc880 2023-10-18T08:57:06.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:05.998+0000 7fb25ffff640 1 -- 172.21.15.40:0/1412712252 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb26007ab80 con 0x7fb2680cc880 2023-10-18T08:57:06.003 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:57:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.001+0000 7fb267fff640 1 -- 172.21.15.40:0/1412712252 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb24806ae40 msgr2=0x7fb24806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.001+0000 7fb267fff640 1 --2- 172.21.15.40:0/1412712252 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb24806ae40 0x7fb24806d300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb25800ca40 tx=0x7fb25800c420 comp rx=0 tx=0).stop 2023-10-18T08:57:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.001+0000 7fb267fff640 1 -- 172.21.15.40:0/1412712252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2680cc880 msgr2=0x7fb268152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.001+0000 7fb267fff640 1 --2- 172.21.15.40:0/1412712252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2680cc880 0x7fb268152cc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb260037040 tx=0x7fb260003c10 comp rx=0 tx=0).stop 2023-10-18T08:57:06.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.001+0000 7fb267fff640 1 -- 172.21.15.40:0/1412712252 shutdown_connections 2023-10-18T08:57:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.001+0000 7fb267fff640 1 --2- 172.21.15.40:0/1412712252 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb24806ae40 0x7fb24806d300 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.001+0000 7fb267fff640 1 --2- 172.21.15.40:0/1412712252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2680cc880 0x7fb268152cc0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.001+0000 7fb267fff640 1 --2- 172.21.15.40:0/1412712252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb268069b80 0x7fb268152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:06.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.001+0000 7fb267fff640 1 --2- 172.21.15.40:0/1412712252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb268069720 0x7fb2681f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:06.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.001+0000 7fb267fff640 1 -- 172.21.15.40:0/1412712252 >> 172.21.15.40:0/1412712252 conn(0x7fb2680cdf70 msgr2=0x7fb268069fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:06.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.001+0000 7fb267fff640 1 -- 172.21.15.40:0/1412712252 shutdown_connections 2023-10-18T08:57:06.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.001+0000 7fb267fff640 1 -- 172.21.15.40:0/1412712252 wait complete. 2023-10-18T08:57:06.023 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2023-10-18T08:57:06.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.247+0000 7f5d78abb640 1 -- 172.21.15.40:0/1105789455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d740b15b0 msgr2=0x7f5d740c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:06.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.247+0000 7f5d78abb640 1 --2- 172.21.15.40:0/1105789455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d740b15b0 0x7f5d740c99c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5d7406b050 tx=0x7f5d6802bd70 comp rx=0 tx=0).stop 2023-10-18T08:57:06.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.248+0000 7f5d78abb640 1 -- 172.21.15.40:0/1105789455 shutdown_connections 2023-10-18T08:57:06.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.248+0000 7f5d78abb640 1 --2- 172.21.15.40:0/1105789455 >> v2:172.21.15.144:3300/0 conn(0x7f5d740c9f00 0x7f5d74157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:06.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.248+0000 7f5d78abb640 1 --2- 172.21.15.40:0/1105789455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d740b15b0 0x7f5d740c99c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:06.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.248+0000 7f5d78abb640 1 -- 172.21.15.40:0/1105789455 >> 172.21.15.40:0/1105789455 conn(0x7f5d74069720 msgr2=0x7f5d74069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:06.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.248+0000 7f5d78abb640 1 -- 172.21.15.40:0/1105789455 shutdown_connections 2023-10-18T08:57:06.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.248+0000 7f5d78abb640 1 -- 172.21.15.40:0/1105789455 wait complete. 2023-10-18T08:57:06.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.249+0000 7f5d78abb640 1 Processor -- start 2023-10-18T08:57:06.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.250+0000 7f5d78abb640 1 -- start start 2023-10-18T08:57:06.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.250+0000 7f5d78abb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d7406d6f0 0x7f5d741535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:06.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.251+0000 7f5d78abb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d740b15b0 0x7f5d7414b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:06.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.251+0000 7f5d78abb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d740c9f00 0x7f5d741f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:06.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.251+0000 7f5d78abb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d74153ca0 con 0x7f5d7406d6f0 2023-10-18T08:57:06.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.251+0000 7f5d78abb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d740aee90 con 0x7f5d740b15b0 2023-10-18T08:57:06.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.251+0000 7f5d78abb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7414bc10 con 0x7f5d740c9f00 2023-10-18T08:57:06.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.251+0000 7f5d737fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d7406d6f0 0x7f5d741535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:06.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.251+0000 7f5d737fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d7406d6f0 0x7f5d741535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40828/0 (socket says 172.21.15.40:40828) 2023-10-18T08:57:06.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.251+0000 7f5d737fe640 1 -- 172.21.15.40:0/1818211453 learned_addr learned my addr 172.21.15.40:0/1818211453 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:06.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.251+0000 7f5d72ffd640 1 --2- 172.21.15.40:0/1818211453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d740b15b0 0x7f5d7414b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:06.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.251+0000 7f5d737fe640 1 -- 172.21.15.40:0/1818211453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d740c9f00 msgr2=0x7f5d741f60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:06.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.251+0000 7f5d737fe640 1 --2- 172.21.15.40:0/1818211453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d740c9f00 0x7f5d741f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:06.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.251+0000 7f5d737fe640 1 -- 172.21.15.40:0/1818211453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d740b15b0 msgr2=0x7f5d7414b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:06.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.251+0000 7f5d737fe640 1 --2- 172.21.15.40:0/1818211453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d740b15b0 0x7f5d7414b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:06.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.251+0000 7f5d737fe640 1 -- 172.21.15.40:0/1818211453 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d68009560 con 0x7f5d7406d6f0 2023-10-18T08:57:06.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.252+0000 7f5d737fe640 1 --2- 172.21.15.40:0/1818211453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d7406d6f0 0x7f5d741535e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5d6400c430 tx=0x7f5d6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:06.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.252+0000 7f5d5bfff640 1 -- 172.21.15.40:0/1818211453 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d64011790 con 0x7f5d7406d6f0 2023-10-18T08:57:06.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.252+0000 7f5d5bfff640 1 -- 172.21.15.40:0/1818211453 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5d640119e0 con 0x7f5d7406d6f0 2023-10-18T08:57:06.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.252+0000 7f5d5bfff640 1 -- 172.21.15.40:0/1818211453 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d64011df0 con 0x7f5d7406d6f0 2023-10-18T08:57:06.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.252+0000 7f5d78abb640 1 -- 172.21.15.40:0/1818211453 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d740ae4f0 con 0x7f5d7406d6f0 2023-10-18T08:57:06.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.252+0000 7f5d78abb640 1 -- 172.21.15.40:0/1818211453 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d740aeab0 con 0x7f5d7406d6f0 2023-10-18T08:57:06.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.253+0000 7f5d78abb640 1 -- 172.21.15.40:0/1818211453 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d74066e00 con 0x7f5d7406d6f0 2023-10-18T08:57:06.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.254+0000 7f5d5bfff640 1 -- 172.21.15.40:0/1818211453 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5d6401c460 con 0x7f5d7406d6f0 2023-10-18T08:57:06.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.255+0000 7f5d5bfff640 1 --2- 172.21.15.40:0/1818211453 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d6006ae40 0x7f5d6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:06.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.255+0000 7f5d5bfff640 1 -- 172.21.15.40:0/1818211453 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5d64089500 con 0x7f5d7406d6f0 2023-10-18T08:57:06.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.256+0000 7f5d72ffd640 1 --2- 172.21.15.40:0/1818211453 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d6006ae40 0x7f5d6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:06.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.257+0000 7f5d72ffd640 1 --2- 172.21.15.40:0/1818211453 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d6006ae40 0x7f5d6006d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5d68035000 tx=0x7f5d68004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:06.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.260+0000 7f5d5bfff640 1 -- 172.21.15.40:0/1818211453 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d640548a0 con 0x7f5d7406d6f0 2023-10-18T08:57:06.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:06.524+0000 7f5d78abb640 1 -- 172.21.15.40:0/1818211453 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f5d740caf90 con 0x7f5d7406d6f0 2023-10-18T08:57:07.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.098+0000 7f5d5bfff640 1 -- 172.21.15.40:0/1818211453 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v125) v1 ==== 145+0+0 (secure 0 0 0) 0x7f5d64057ec0 con 0x7f5d7406d6f0 2023-10-18T08:57:07.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.102+0000 7f5d78abb640 1 -- 172.21.15.40:0/1818211453 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d6006ae40 msgr2=0x7f5d6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:07.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.102+0000 7f5d78abb640 1 --2- 172.21.15.40:0/1818211453 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d6006ae40 0x7f5d6006d300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5d68035000 tx=0x7f5d68004730 comp rx=0 tx=0).stop 2023-10-18T08:57:07.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.102+0000 7f5d78abb640 1 -- 172.21.15.40:0/1818211453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d7406d6f0 msgr2=0x7f5d741535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:07.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.102+0000 7f5d78abb640 1 --2- 172.21.15.40:0/1818211453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d7406d6f0 0x7f5d741535e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5d6400c430 tx=0x7f5d6400c8f0 comp rx=0 tx=0).stop 2023-10-18T08:57:07.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.102+0000 7f5d78abb640 1 -- 172.21.15.40:0/1818211453 shutdown_connections 2023-10-18T08:57:07.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.102+0000 7f5d78abb640 1 --2- 172.21.15.40:0/1818211453 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d6006ae40 0x7f5d6006d300 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:07.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.102+0000 7f5d78abb640 1 --2- 172.21.15.40:0/1818211453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d740c9f00 0x7f5d741f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:07.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.102+0000 7f5d78abb640 1 --2- 172.21.15.40:0/1818211453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d740b15b0 0x7f5d7414b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:07.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.102+0000 7f5d78abb640 1 --2- 172.21.15.40:0/1818211453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d7406d6f0 0x7f5d741535e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.102+0000 7f5d78abb640 1 -- 172.21.15.40:0/1818211453 >> 172.21.15.40:0/1818211453 conn(0x7f5d74069720 msgr2=0x7f5d740cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.103+0000 7f5d78abb640 1 -- 172.21.15.40:0/1818211453 shutdown_connections 2023-10-18T08:57:07.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.103+0000 7f5d78abb640 1 -- 172.21.15.40:0/1818211453 wait complete. 2023-10-18T08:57:07.129 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-18T08:57:07.377 INFO:teuthology.orchestra.run.smithi040.stdout:{"osd_objectstore":"memstore"} 2023-10-18T08:57:07.377 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-18T08:57:07.378 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T08:57:07.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.609+0000 7fc370855640 1 -- 172.21.15.40:0/2180888371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc36c071310 msgr2=0x7fc36c06b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:07.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.609+0000 7fc370855640 1 --2- 172.21.15.40:0/2180888371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc36c071310 0x7fc36c06b4d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc35c005fd0 tx=0x7fc35c02bd90 comp rx=0 tx=0).stop 2023-10-18T08:57:07.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.609+0000 7fc370855640 1 -- 172.21.15.40:0/2180888371 shutdown_connections 2023-10-18T08:57:07.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.609+0000 7fc370855640 1 --2- 172.21.15.40:0/2180888371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc36c06ba10 0x7fc36c157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:07.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.609+0000 7fc370855640 1 --2- 172.21.15.40:0/2180888371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc36c071310 0x7fc36c06b4d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:07.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.609+0000 7fc370855640 1 -- 172.21.15.40:0/2180888371 >> 172.21.15.40:0/2180888371 conn(0x7fc36c0aed70 msgr2=0x7fc36c0af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:07.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.609+0000 7fc370855640 1 -- 172.21.15.40:0/2180888371 shutdown_connections 2023-10-18T08:57:07.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.609+0000 7fc370855640 1 -- 172.21.15.40:0/2180888371 wait complete. 2023-10-18T08:57:07.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.611+0000 7fc370855640 1 Processor -- start 2023-10-18T08:57:07.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.611+0000 7fc370855640 1 -- start start 2023-10-18T08:57:07.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.612+0000 7fc370855640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc36c06ba10 0x7fc36c13bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:07.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.612+0000 7fc370855640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc36c071310 0x7fc36c1510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:07.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.612+0000 7fc370855640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc36c158210 0x7fc36c1515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:07.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.612+0000 7fc370855640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c1f54f0 con 0x7fc36c06ba10 2023-10-18T08:57:07.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.612+0000 7fc370855640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c1f57d0 con 0x7fc36c071310 2023-10-18T08:57:07.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.612+0000 7fc370855640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c1f5ab0 con 0x7fc36c158210 2023-10-18T08:57:07.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.613+0000 7fc36ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc36c06ba10 0x7fc36c13bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:07.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.613+0000 7fc36ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc36c06ba10 0x7fc36c13bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40852/0 (socket says 172.21.15.40:40852) 2023-10-18T08:57:07.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.613+0000 7fc36ad9d640 1 -- 172.21.15.40:0/3666236114 learned_addr learned my addr 172.21.15.40:0/3666236114 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:07.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.613+0000 7fc36a59c640 1 --2- 172.21.15.40:0/3666236114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc36c071310 0x7fc36c1510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:07.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.613+0000 7fc36b59e640 1 --2- 172.21.15.40:0/3666236114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc36c158210 0x7fc36c1515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:07.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.613+0000 7fc36ad9d640 1 -- 172.21.15.40:0/3666236114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc36c158210 msgr2=0x7fc36c1515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:07.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.613+0000 7fc36ad9d640 1 --2- 172.21.15.40:0/3666236114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc36c158210 0x7fc36c1515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:07.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.613+0000 7fc36ad9d640 1 -- 172.21.15.40:0/3666236114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc36c071310 msgr2=0x7fc36c1510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:07.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.613+0000 7fc36ad9d640 1 --2- 172.21.15.40:0/3666236114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc36c071310 0x7fc36c1510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:07.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.613+0000 7fc36ad9d640 1 -- 172.21.15.40:0/3666236114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c009560 con 0x7fc36c06ba10 2023-10-18T08:57:07.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.613+0000 7fc36b59e640 1 --2- 172.21.15.40:0/3666236114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc36c158210 0x7fc36c1515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:07.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.614+0000 7fc36ad9d640 1 --2- 172.21.15.40:0/3666236114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc36c06ba10 0x7fc36c13bc20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009850 tx=0x7fc35c02db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:07.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.614+0000 7fc3537fe640 1 -- 172.21.15.40:0/3666236114 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0035f0 con 0x7fc36c06ba10 2023-10-18T08:57:07.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.614+0000 7fc3537fe640 1 -- 172.21.15.40:0/3666236114 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc35c003750 con 0x7fc36c06ba10 2023-10-18T08:57:07.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.614+0000 7fc3537fe640 1 -- 172.21.15.40:0/3666236114 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c003b00 con 0x7fc36c06ba10 2023-10-18T08:57:07.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.614+0000 7fc370855640 1 -- 172.21.15.40:0/3666236114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36c1f5d90 con 0x7fc36c06ba10 2023-10-18T08:57:07.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.614+0000 7fc370855640 1 -- 172.21.15.40:0/3666236114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c1f6070 con 0x7fc36c06ba10 2023-10-18T08:57:07.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.615+0000 7fc370855640 1 -- 172.21.15.40:0/3666236114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36c06b700 con 0x7fc36c06ba10 2023-10-18T08:57:07.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.616+0000 7fc3537fe640 1 -- 172.21.15.40:0/3666236114 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc35c003c60 con 0x7fc36c06ba10 2023-10-18T08:57:07.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.617+0000 7fc3537fe640 1 --2- 172.21.15.40:0/3666236114 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc35806ae40 0x7fc35806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:07.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.617+0000 7fc3537fe640 1 -- 172.21.15.40:0/3666236114 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc35c0abf70 con 0x7fc36c06ba10 2023-10-18T08:57:07.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.618+0000 7fc36a59c640 1 --2- 172.21.15.40:0/3666236114 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc35806ae40 0x7fc35806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:07.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.619+0000 7fc36a59c640 1 --2- 172.21.15.40:0/3666236114 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc35806ae40 0x7fc35806d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc36c15a8d0 tx=0x7fc360005d80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:07.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.622+0000 7fc3537fe640 1 -- 172.21.15.40:0/3666236114 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc35c077430 con 0x7fc36c06ba10 2023-10-18T08:57:07.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.816+0000 7fc370855640 1 -- 172.21.15.40:0/3666236114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7fc36c0b2230 con 0x7fc36c06ba10 2023-10-18T08:57:07.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.817+0000 7fc3537fe640 1 -- 172.21.15.40:0/3666236114 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v24) v1 ==== 59+0+1452 (secure 0 0 0) 0x7fc35c043080 con 0x7fc36c06ba10 2023-10-18T08:57:07.821 INFO:teuthology.orchestra.run.smithi040.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T08:57:07.821 INFO:teuthology.orchestra.run.smithi040.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T08:57:07.821 INFO:teuthology.orchestra.run.smithi040.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T08:57:07.821 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds 20/20 2023-10-18T08:57:07.821 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T08:57:07.822 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_ms 1/1 2023-10-18T08:57:07.822 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T08:57:07.822 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T08:57:07.822 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T08:57:07.822 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_size 100 2023-10-18T08:57:07.823 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_frag true 2023-10-18T08:57:07.823 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_scatterstat true 2023-10-18T08:57:07.823 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T08:57:07.823 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_verify_scatter true 2023-10-18T08:57:07.823 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T08:57:07.824 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T08:57:07.824 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T08:57:07.824 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced client_mount_timeout 600 2023-10-18T08:57:07.824 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_client 20/20 2023-10-18T08:57:07.824 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_ms 1/1 2023-10-18T08:57:07.825 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T08:57:07.825 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T08:57:07.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.820+0000 7fc370855640 1 -- 172.21.15.40:0/3666236114 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc35806ae40 msgr2=0x7fc35806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:07.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.820+0000 7fc370855640 1 --2- 172.21.15.40:0/3666236114 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc35806ae40 0x7fc35806d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc36c15a8d0 tx=0x7fc360005d80 comp rx=0 tx=0).stop 2023-10-18T08:57:07.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.820+0000 7fc370855640 1 -- 172.21.15.40:0/3666236114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc36c06ba10 msgr2=0x7fc36c13bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:07.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.820+0000 7fc370855640 1 --2- 172.21.15.40:0/3666236114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc36c06ba10 0x7fc36c13bc20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009850 tx=0x7fc35c02db10 comp rx=0 tx=0).stop 2023-10-18T08:57:07.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.820+0000 7fc370855640 1 -- 172.21.15.40:0/3666236114 shutdown_connections 2023-10-18T08:57:07.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.820+0000 7fc370855640 1 --2- 172.21.15.40:0/3666236114 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc35806ae40 0x7fc35806d300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:07.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.820+0000 7fc370855640 1 --2- 172.21.15.40:0/3666236114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc36c158210 0x7fc36c1515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:07.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.820+0000 7fc370855640 1 --2- 172.21.15.40:0/3666236114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc36c071310 0x7fc36c1510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:07.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.820+0000 7fc370855640 1 --2- 172.21.15.40:0/3666236114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc36c06ba10 0x7fc36c13bc20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:07.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.820+0000 7fc370855640 1 -- 172.21.15.40:0/3666236114 >> 172.21.15.40:0/3666236114 conn(0x7fc36c0aed70 msgr2=0x7fc36c156ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:07.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.820+0000 7fc370855640 1 -- 172.21.15.40:0/3666236114 shutdown_connections 2023-10-18T08:57:07.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:07.820+0000 7fc370855640 1 -- 172.21.15.40:0/3666236114 wait complete. 2023-10-18T08:57:07.841 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T08:57:08.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.068+0000 7ff01d139640 1 -- 172.21.15.40:0/520350441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff01806d660 msgr2=0x7ff0180b1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:08.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.068+0000 7ff01d139640 1 --2- 172.21.15.40:0/520350441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff01806d660 0x7ff0180b1cc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff008005fd0 tx=0x7ff00802c2e0 comp rx=0 tx=0).stop 2023-10-18T08:57:08.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.068+0000 7ff01d139640 1 -- 172.21.15.40:0/520350441 shutdown_connections 2023-10-18T08:57:08.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.068+0000 7ff01d139640 1 --2- 172.21.15.40:0/520350441 >> v2:172.21.15.144:3300/0 conn(0x7ff0180cd140 0x7ff018162610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:08.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.068+0000 7ff01d139640 1 --2- 172.21.15.40:0/520350441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff01806d660 0x7ff0180b1cc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:08.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.068+0000 7ff01d139640 1 -- 172.21.15.40:0/520350441 >> 172.21.15.40:0/520350441 conn(0x7ff018069720 msgr2=0x7ff018069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:08.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.068+0000 7ff01d139640 1 -- 172.21.15.40:0/520350441 shutdown_connections 2023-10-18T08:57:08.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.068+0000 7ff01d139640 1 -- 172.21.15.40:0/520350441 wait complete. 2023-10-18T08:57:08.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.070+0000 7ff01d139640 1 Processor -- start 2023-10-18T08:57:08.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.071+0000 7ff01d139640 1 -- start start 2023-10-18T08:57:08.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.072+0000 7ff01d139640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff01806d660 0x7ff0181f0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:08.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.072+0000 7ff01d139640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0180cd140 0x7ff0181601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:08.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.072+0000 7ff01d139640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff018157800 0x7ff018153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:08.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.072+0000 7ff01d139640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff018161710 con 0x7ff018157800 2023-10-18T08:57:08.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.072+0000 7ff01d139640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff018160b00 con 0x7ff0180cd140 2023-10-18T08:57:08.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.072+0000 7ff01d139640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff018160de0 con 0x7ff01806d660 2023-10-18T08:57:08.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.072+0000 7ff01759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff018157800 0x7ff018153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:08.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.072+0000 7ff01759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff018157800 0x7ff018153f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40854/0 (socket says 172.21.15.40:40854) 2023-10-18T08:57:08.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.072+0000 7ff01759e640 1 -- 172.21.15.40:0/2297978027 learned_addr learned my addr 172.21.15.40:0/2297978027 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:08.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.072+0000 7ff016d9d640 1 --2- 172.21.15.40:0/2297978027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff01806d660 0x7ff0181f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:08.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.073+0000 7ff01659c640 1 --2- 172.21.15.40:0/2297978027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0180cd140 0x7ff0181601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:08.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.073+0000 7ff01759e640 1 -- 172.21.15.40:0/2297978027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff01806d660 msgr2=0x7ff0181f0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:08.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.073+0000 7ff01759e640 1 --2- 172.21.15.40:0/2297978027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff01806d660 0x7ff0181f0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:08.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.073+0000 7ff01759e640 1 -- 172.21.15.40:0/2297978027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0180cd140 msgr2=0x7ff0181601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:08.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.073+0000 7ff01759e640 1 --2- 172.21.15.40:0/2297978027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0180cd140 0x7ff0181601d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:08.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.073+0000 7ff01759e640 1 -- 172.21.15.40:0/2297978027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff008009900 con 0x7ff018157800 2023-10-18T08:57:08.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.073+0000 7ff016d9d640 1 --2- 172.21.15.40:0/2297978027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff01806d660 0x7ff0181f0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:57:08.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.073+0000 7ff01759e640 1 --2- 172.21.15.40:0/2297978027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff018157800 0x7ff018153f70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff01000e970 tx=0x7ff01000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:08.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.073+0000 7fefff7fe640 1 -- 172.21.15.40:0/2297978027 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff010009720 con 0x7ff018157800 2023-10-18T08:57:08.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.073+0000 7fefff7fe640 1 -- 172.21.15.40:0/2297978027 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff01000ccc0 con 0x7ff018157800 2023-10-18T08:57:08.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.073+0000 7fefff7fe640 1 -- 172.21.15.40:0/2297978027 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff010005280 con 0x7ff018157800 2023-10-18T08:57:08.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.073+0000 7ff01d139640 1 -- 172.21.15.40:0/2297978027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0181610c0 con 0x7ff018157800 2023-10-18T08:57:08.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.074+0000 7ff01d139640 1 -- 172.21.15.40:0/2297978027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff018160710 con 0x7ff018157800 2023-10-18T08:57:08.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.075+0000 7ff01d139640 1 -- 172.21.15.40:0/2297978027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0181618c0 con 0x7ff018157800 2023-10-18T08:57:08.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.076+0000 7fefff7fe640 1 -- 172.21.15.40:0/2297978027 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff0100053e0 con 0x7ff018157800 2023-10-18T08:57:08.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.077+0000 7fefff7fe640 1 --2- 172.21.15.40:0/2297978027 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff00406ae70 0x7ff00406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:08.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.077+0000 7fefff7fe640 1 -- 172.21.15.40:0/2297978027 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff010014070 con 0x7ff018157800 2023-10-18T08:57:08.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.077+0000 7ff016d9d640 1 --2- 172.21.15.40:0/2297978027 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff00406ae70 0x7ff00406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:08.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.079+0000 7ff016d9d640 1 --2- 172.21.15.40:0/2297978027 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff00406ae70 0x7ff00406d330 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff008036000 tx=0x7ff008004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.081+0000 7fefff7fe640 1 -- 172.21.15.40:0/2297978027 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff0181618c0 con 0x7ff018157800 2023-10-18T08:57:08.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.279+0000 7ff01d139640 1 -- 172.21.15.40:0/2297978027 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff0180b2280 con 0x7ff018157800 2023-10-18T08:57:08.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.286+0000 7fefff7fe640 1 -- 172.21.15.40:0/2297978027 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v25)=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff010058bd0 con 0x7ff018157800 2023-10-18T08:57:08.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.289+0000 7ff01d139640 1 -- 172.21.15.40:0/2297978027 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff00406ae70 msgr2=0x7ff00406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:08.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.289+0000 7ff01d139640 1 --2- 172.21.15.40:0/2297978027 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff00406ae70 0x7ff00406d330 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff008036000 tx=0x7ff008004ab0 comp rx=0 tx=0).stop 2023-10-18T08:57:08.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.289+0000 7ff01d139640 1 -- 172.21.15.40:0/2297978027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff018157800 msgr2=0x7ff018153f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:08.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.289+0000 7ff01d139640 1 --2- 172.21.15.40:0/2297978027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff018157800 0x7ff018153f70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff01000e970 tx=0x7ff01000ed70 comp rx=0 tx=0).stop 2023-10-18T08:57:08.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.289+0000 7ff01d139640 1 -- 172.21.15.40:0/2297978027 shutdown_connections 2023-10-18T08:57:08.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.289+0000 7ff01d139640 1 --2- 172.21.15.40:0/2297978027 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff00406ae70 0x7ff00406d330 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:08.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.289+0000 7ff01d139640 1 --2- 172.21.15.40:0/2297978027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff018157800 0x7ff018153f70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:08.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.289+0000 7ff01d139640 1 --2- 172.21.15.40:0/2297978027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0180cd140 0x7ff0181601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:08.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.289+0000 7ff01d139640 1 --2- 172.21.15.40:0/2297978027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff01806d660 0x7ff0181f0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:08.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.289+0000 7ff01d139640 1 -- 172.21.15.40:0/2297978027 >> 172.21.15.40:0/2297978027 conn(0x7ff018069720 msgr2=0x7ff0180afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.290+0000 7ff01d139640 1 -- 172.21.15.40:0/2297978027 shutdown_connections 2023-10-18T08:57:08.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:08.290+0000 7ff01d139640 1 -- 172.21.15.40:0/2297978027 wait complete. 2023-10-18T08:57:08.311 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T08:57:08.311 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T08:57:08.427 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T08:57:08.547 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T08:57:08.548 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:57:08.548 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:57:08.611 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T08:57:08.612 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:57:08.612 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:57:08.671 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T08:57:08.672 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:57:08.672 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:57:08.735 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T08:57:08.735 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:57:08.735 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:57:08.797 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T08:57:08.798 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T08:57:08.798 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:57:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.036+0000 7f2a83000640 1 -- 172.21.15.40:0/3408031039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a7c1586a0 msgr2=0x7f2a7c06b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.036+0000 7f2a83000640 1 --2- 172.21.15.40:0/3408031039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a7c1586a0 0x7f2a7c06b710 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0acc20 tx=0x7f2a6802bd90 comp rx=0 tx=0).stop 2023-10-18T08:57:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.036+0000 7f2a83000640 1 -- 172.21.15.40:0/3408031039 shutdown_connections 2023-10-18T08:57:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.036+0000 7f2a83000640 1 --2- 172.21.15.40:0/3408031039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a7c06bc50 0x7f2a7c06c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.036+0000 7f2a83000640 1 --2- 172.21.15.40:0/3408031039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a7c1586a0 0x7f2a7c06b710 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.036+0000 7f2a83000640 1 --2- 172.21.15.40:0/3408031039 >> v2:172.21.15.144:3300/0 conn(0x7f2a7c06d710 0x7f2a7c151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.036+0000 7f2a83000640 1 -- 172.21.15.40:0/3408031039 >> 172.21.15.40:0/3408031039 conn(0x7f2a7c06cee0 msgr2=0x7f2a7c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.036+0000 7f2a83000640 1 -- 172.21.15.40:0/3408031039 shutdown_connections 2023-10-18T08:57:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.036+0000 7f2a83000640 1 -- 172.21.15.40:0/3408031039 wait complete. 2023-10-18T08:57:09.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.038+0000 7f2a83000640 1 Processor -- start 2023-10-18T08:57:09.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.038+0000 7f2a83000640 1 -- start start 2023-10-18T08:57:09.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.039+0000 7f2a83000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a7c06bc50 0x7f2a7c140200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:09.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.039+0000 7f2a83000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a7c06d710 0x7f2a7c14ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:09.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.039+0000 7f2a83000640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a7c1586a0 0x7f2a7c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:09.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.039+0000 7f2a83000640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c141270 con 0x7f2a7c1586a0 2023-10-18T08:57:09.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.039+0000 7f2a83000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c14d8d0 con 0x7f2a7c06d710 2023-10-18T08:57:09.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.039+0000 7f2a83000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c14dbb0 con 0x7f2a7c06bc50 2023-10-18T08:57:09.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.040+0000 7f2a8159d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a7c1586a0 0x7f2a7c0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:09.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.040+0000 7f2a8159d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a7c1586a0 0x7f2a7c0cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40874/0 (socket says 172.21.15.40:40874) 2023-10-18T08:57:09.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.040+0000 7f2a8159d640 1 -- 172.21.15.40:0/396390480 learned_addr learned my addr 172.21.15.40:0/396390480 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:09.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.040+0000 7f2a7bfff640 1 --2- 172.21.15.40:0/396390480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a7c06d710 0x7f2a7c14ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:09.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.040+0000 7f2a80d9c640 1 --2- 172.21.15.40:0/396390480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a7c06bc50 0x7f2a7c140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:09.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.040+0000 7f2a8159d640 1 -- 172.21.15.40:0/396390480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a7c06bc50 msgr2=0x7f2a7c140200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:09.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.040+0000 7f2a8159d640 1 --2- 172.21.15.40:0/396390480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a7c06bc50 0x7f2a7c140200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:09.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.040+0000 7f2a8159d640 1 -- 172.21.15.40:0/396390480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a7c06d710 msgr2=0x7f2a7c14ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:09.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.040+0000 7f2a8159d640 1 --2- 172.21.15.40:0/396390480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a7c06d710 0x7f2a7c14ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:09.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.040+0000 7f2a8159d640 1 -- 172.21.15.40:0/396390480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a68009560 con 0x7f2a7c1586a0 2023-10-18T08:57:09.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.040+0000 7f2a80d9c640 1 --2- 172.21.15.40:0/396390480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a7c06bc50 0x7f2a7c140200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:09.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.040+0000 7f2a7bfff640 1 --2- 172.21.15.40:0/396390480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a7c06d710 0x7f2a7c14ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:09.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.040+0000 7f2a8159d640 1 --2- 172.21.15.40:0/396390480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a7c1586a0 0x7f2a7c0cc900 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400e980 tx=0x7f2a7400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:09.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.041+0000 7f2a797fa640 1 -- 172.21.15.40:0/396390480 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74004d20 con 0x7f2a7c1586a0 2023-10-18T08:57:09.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.041+0000 7f2a797fa640 1 -- 172.21.15.40:0/396390480 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2a7400cb20 con 0x7f2a7c1586a0 2023-10-18T08:57:09.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.041+0000 7f2a83000640 1 -- 172.21.15.40:0/396390480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7c14de90 con 0x7f2a7c1586a0 2023-10-18T08:57:09.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.041+0000 7f2a797fa640 1 -- 172.21.15.40:0/396390480 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74005020 con 0x7f2a7c1586a0 2023-10-18T08:57:09.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.041+0000 7f2a83000640 1 -- 172.21.15.40:0/396390480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7c14e450 con 0x7f2a7c1586a0 2023-10-18T08:57:09.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.042+0000 7f2a83000640 1 -- 172.21.15.40:0/396390480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7c066fd0 con 0x7f2a7c1586a0 2023-10-18T08:57:09.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.043+0000 7f2a797fa640 1 -- 172.21.15.40:0/396390480 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2a74005180 con 0x7f2a7c1586a0 2023-10-18T08:57:09.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.044+0000 7f2a797fa640 1 --2- 172.21.15.40:0/396390480 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a6006ae70 0x7f2a6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:09.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.044+0000 7f2a797fa640 1 -- 172.21.15.40:0/396390480 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2a74089cc0 con 0x7f2a7c1586a0 2023-10-18T08:57:09.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.045+0000 7f2a80d9c640 1 --2- 172.21.15.40:0/396390480 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a6006ae70 0x7f2a6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:09.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.046+0000 7f2a80d9c640 1 --2- 172.21.15.40:0/396390480 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a6006ae70 0x7f2a6006d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000cb50 tx=0x7f2a70005e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:09.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.048+0000 7f2a797fa640 1 -- 172.21.15.40:0/396390480 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a74090080 con 0x7f2a7c1586a0 2023-10-18T08:57:09.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.283+0000 7f2a83000640 1 -- 172.21.15.40:0/396390480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a7c14fa20 con 0x7f2a7c1586a0 2023-10-18T08:57:09.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.285+0000 7f2a797fa640 1 -- 172.21.15.40:0/396390480 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f2a7401f070 con 0x7f2a7c1586a0 2023-10-18T08:57:09.288 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:09.288 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/837790640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":837790640},{"type":"v1","addr":"172.21.15.40:6833","nonce":837790640}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3377302775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3377302775},{"type":"v1","addr":"172.21.15.144:6836","nonce":3377302775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4939,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3162305748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3162305748},{"type":"v1","addr":"172.21.15.144:6837","nonce":3162305748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1921545729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1921545729},{"type":"v1","addr":"172.21.15.40:6835","nonce":1921545729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[]} 2023-10-18T08:57:09.289 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 18 2023-10-18T08:57:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.288+0000 7f2a83000640 1 -- 172.21.15.40:0/396390480 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a6006ae70 msgr2=0x7f2a6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.288+0000 7f2a83000640 1 --2- 172.21.15.40:0/396390480 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a6006ae70 0x7f2a6006d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000cb50 tx=0x7f2a70005e50 comp rx=0 tx=0).stop 2023-10-18T08:57:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.288+0000 7f2a83000640 1 -- 172.21.15.40:0/396390480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a7c1586a0 msgr2=0x7f2a7c0cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:09.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.288+0000 7f2a83000640 1 --2- 172.21.15.40:0/396390480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a7c1586a0 0x7f2a7c0cc900 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400e980 tx=0x7f2a7400ee40 comp rx=0 tx=0).stop 2023-10-18T08:57:09.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.289+0000 7f2a83000640 1 -- 172.21.15.40:0/396390480 shutdown_connections 2023-10-18T08:57:09.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.289+0000 7f2a83000640 1 --2- 172.21.15.40:0/396390480 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a6006ae70 0x7f2a6006d330 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:09.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.289+0000 7f2a83000640 1 --2- 172.21.15.40:0/396390480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a7c1586a0 0x7f2a7c0cc900 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:09.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.289+0000 7f2a83000640 1 --2- 172.21.15.40:0/396390480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a7c06d710 0x7f2a7c14ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:09.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.289+0000 7f2a83000640 1 --2- 172.21.15.40:0/396390480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a7c06bc50 0x7f2a7c140200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:09.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.289+0000 7f2a83000640 1 -- 172.21.15.40:0/396390480 >> 172.21.15.40:0/396390480 conn(0x7f2a7c06cee0 msgr2=0x7f2a7c0cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:09.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.289+0000 7f2a83000640 1 -- 172.21.15.40:0/396390480 shutdown_connections 2023-10-18T08:57:09.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.289+0000 7f2a83000640 1 -- 172.21.15.40:0/396390480 wait complete. 2023-10-18T08:57:09.312 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T08:57:09.312 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T08:57:09.313 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T08:57:09.313 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T08:57:09.313 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T08:57:09.313 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T08:57:09.314 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T08:57:09.314 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T08:57:09.314 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T08:57:09.314 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T08:57:09.315 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T08:57:09.315 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T08:57:09.315 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T08:57:09.315 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T08:57:09.316 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T08:57:09.316 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T08:57:09.332 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T08:57:09.333 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T08:57:09.334 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T08:57:09.334 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T08:57:09.336 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T08:57:09.336 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T08:57:09.338 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T08:57:09.338 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T08:57:09.384 INFO:tasks.ceph.mds.a:Started 2023-10-18T08:57:09.385 DEBUG:teuthology.parallel:result is None 2023-10-18T08:57:09.387 INFO:tasks.ceph.mds.b:Started 2023-10-18T08:57:09.387 DEBUG:teuthology.parallel:result is None 2023-10-18T08:57:09.392 INFO:tasks.ceph.mds.c:Started 2023-10-18T08:57:09.392 DEBUG:teuthology.parallel:result is None 2023-10-18T08:57:09.395 INFO:tasks.ceph.mds.d:Started 2023-10-18T08:57:09.395 DEBUG:teuthology.parallel:result is None 2023-10-18T08:57:09.396 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T08:57:09.549 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T08:57:09.550 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-10-18T08:57:09.557 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-18T08:57:09.558 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-18T08:57:09.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.596+0000 7ff80d490640 1 -- 172.21.15.40:0/1621757156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8080cbc20 msgr2=0x7ff8080c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:09.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.596+0000 7ff80d490640 1 --2- 172.21.15.40:0/1621757156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8080cbc20 0x7ff8080c9b00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff80806a510 tx=0x7ff7fc02bd70 comp rx=0 tx=0).stop 2023-10-18T08:57:09.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.596+0000 7ff80d490640 1 -- 172.21.15.40:0/1621757156 shutdown_connections 2023-10-18T08:57:09.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.596+0000 7ff80d490640 1 --2- 172.21.15.40:0/1621757156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8080cbc20 0x7ff8080c9b00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:09.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.596+0000 7ff80d490640 1 --2- 172.21.15.40:0/1621757156 >> v2:172.21.15.144:3300/0 conn(0x7ff8080cdbb0 0x7ff80815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:09.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.596+0000 7ff80d490640 1 -- 172.21.15.40:0/1621757156 >> 172.21.15.40:0/1621757156 conn(0x7ff80806cee0 msgr2=0x7ff808069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:09.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.596+0000 7ff80d490640 1 -- 172.21.15.40:0/1621757156 shutdown_connections 2023-10-18T08:57:09.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.596+0000 7ff80d490640 1 -- 172.21.15.40:0/1621757156 wait complete. 2023-10-18T08:57:09.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.598+0000 7ff80d490640 1 Processor -- start 2023-10-18T08:57:09.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.598+0000 7ff80d490640 1 -- start start 2023-10-18T08:57:09.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.599+0000 7ff80d490640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff80806b710 0x7ff80814f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:09.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.599+0000 7ff80d490640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8080cbc20 0x7ff8081f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:09.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.599+0000 7ff80d490640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8080cdbb0 0x7ff8081f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:09.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.599+0000 7ff80d490640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80814f610 con 0x7ff8080cbc20 2023-10-18T08:57:09.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.599+0000 7ff80d490640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8081558d0 con 0x7ff80806b710 2023-10-18T08:57:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.599+0000 7ff80d490640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8081f35b0 con 0x7ff8080cdbb0 2023-10-18T08:57:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.599+0000 7ff8067fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8080cbc20 0x7ff8081f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.599+0000 7ff8067fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8080cbc20 0x7ff8081f4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40914/0 (socket says 172.21.15.40:40914) 2023-10-18T08:57:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.599+0000 7ff8067fc640 1 -- 172.21.15.40:0/246240787 learned_addr learned my addr 172.21.15.40:0/246240787 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.600+0000 7ff8077fe640 1 --2- 172.21.15.40:0/246240787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8080cdbb0 0x7ff8081f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:09.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.600+0000 7ff806ffd640 1 --2- 172.21.15.40:0/246240787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff80806b710 0x7ff80814f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:09.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.600+0000 7ff8067fc640 1 -- 172.21.15.40:0/246240787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8080cdbb0 msgr2=0x7ff8081f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:09.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.600+0000 7ff8067fc640 1 --2- 172.21.15.40:0/246240787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8080cdbb0 0x7ff8081f54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:09.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.600+0000 7ff8067fc640 1 -- 172.21.15.40:0/246240787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff80806b710 msgr2=0x7ff80814f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:09.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.600+0000 7ff8067fc640 1 --2- 172.21.15.40:0/246240787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff80806b710 0x7ff80814f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.600+0000 7ff8067fc640 1 -- 172.21.15.40:0/246240787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7fc009490 con 0x7ff8080cbc20 2023-10-18T08:57:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.600+0000 7ff8077fe640 1 --2- 172.21.15.40:0/246240787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8080cdbb0 0x7ff8081f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.600+0000 7ff8067fc640 1 --2- 172.21.15.40:0/246240787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8080cbc20 0x7ff8081f4f80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc000c00 tx=0x7ff7fc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.600+0000 7ff7ef7fe640 1 -- 172.21.15.40:0/246240787 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc003ea0 con 0x7ff8080cbc20 2023-10-18T08:57:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.601+0000 7ff7ef7fe640 1 -- 172.21.15.40:0/246240787 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff7fc034440 con 0x7ff8080cbc20 2023-10-18T08:57:09.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.601+0000 7ff7ef7fe640 1 -- 172.21.15.40:0/246240787 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc034850 con 0x7ff8080cbc20 2023-10-18T08:57:09.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.601+0000 7ff80d490640 1 -- 172.21.15.40:0/246240787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8081f2c80 con 0x7ff8080cbc20 2023-10-18T08:57:09.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.601+0000 7ff80d490640 1 -- 172.21.15.40:0/246240787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8081f3240 con 0x7ff8080cbc20 2023-10-18T08:57:09.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.602+0000 7ff80d490640 1 -- 172.21.15.40:0/246240787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff808150370 con 0x7ff8080cbc20 2023-10-18T08:57:09.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.603+0000 7ff7ef7fe640 1 -- 172.21.15.40:0/246240787 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff7fc03b040 con 0x7ff8080cbc20 2023-10-18T08:57:09.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.604+0000 7ff7ef7fe640 1 --2- 172.21.15.40:0/246240787 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7f406ae40 0x7ff7f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:09.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.604+0000 7ff7ef7fe640 1 -- 172.21.15.40:0/246240787 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff7fc0ab8a0 con 0x7ff8080cbc20 2023-10-18T08:57:09.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.604+0000 7ff806ffd640 1 --2- 172.21.15.40:0/246240787 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7f406ae40 0x7ff7f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:09.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.605+0000 7ff806ffd640 1 --2- 172.21.15.40:0/246240787 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7f406ae40 0x7ff7f406d300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff8080ac630 tx=0x7ff7f8005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:09.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.608+0000 7ff7ef7fe640 1 -- 172.21.15.40:0/246240787 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff7fc076bb0 con 0x7ff8080cbc20 2023-10-18T08:57:09.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:09.816+0000 7ff80d490640 1 -- 172.21.15.40:0/246240787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff8081f3400 con 0x7ff8080cbc20 2023-10-18T08:57:10.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.313+0000 7ff7ef7fe640 1 -- 172.21.15.40:0/246240787 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v40) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff7fc039080 con 0x7ff8080cbc20 2023-10-18T08:57:10.317 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2023-10-18T08:57:10.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.317+0000 7ff80d490640 1 -- 172.21.15.40:0/246240787 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7f406ae40 msgr2=0x7ff7f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:10.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.317+0000 7ff80d490640 1 --2- 172.21.15.40:0/246240787 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7f406ae40 0x7ff7f406d300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff8080ac630 tx=0x7ff7f8005f90 comp rx=0 tx=0).stop 2023-10-18T08:57:10.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.317+0000 7ff80d490640 1 -- 172.21.15.40:0/246240787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8080cbc20 msgr2=0x7ff8081f4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:10.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.317+0000 7ff80d490640 1 --2- 172.21.15.40:0/246240787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8080cbc20 0x7ff8081f4f80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc000c00 tx=0x7ff7fc0038a0 comp rx=0 tx=0).stop 2023-10-18T08:57:10.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.317+0000 7ff80d490640 1 -- 172.21.15.40:0/246240787 shutdown_connections 2023-10-18T08:57:10.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.317+0000 7ff80d490640 1 --2- 172.21.15.40:0/246240787 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7f406ae40 0x7ff7f406d300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:10.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.317+0000 7ff80d490640 1 --2- 172.21.15.40:0/246240787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8080cdbb0 0x7ff8081f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:10.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.318+0000 7ff80d490640 1 --2- 172.21.15.40:0/246240787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8080cbc20 0x7ff8081f4f80 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:10.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.318+0000 7ff80d490640 1 --2- 172.21.15.40:0/246240787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff80806b710 0x7ff80814f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:10.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.318+0000 7ff80d490640 1 -- 172.21.15.40:0/246240787 >> 172.21.15.40:0/246240787 conn(0x7ff80806cee0 msgr2=0x7ff8080b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:10.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.318+0000 7ff80d490640 1 -- 172.21.15.40:0/246240787 shutdown_connections 2023-10-18T08:57:10.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.318+0000 7ff80d490640 1 -- 172.21.15.40:0/246240787 wait complete. 2023-10-18T08:57:10.342 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T08:57:10.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.560+0000 7f7e1a683640 1 -- 172.21.15.40:0/2363645571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e1406d6f0 msgr2=0x7f7e140b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:10.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.560+0000 7f7e1a683640 1 --2- 172.21.15.40:0/2363645571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e1406d6f0 0x7f7e140b15b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7e10005fd0 tx=0x7f7e1002bd90 comp rx=0 tx=0).stop 2023-10-18T08:57:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.561+0000 7f7e1a683640 1 -- 172.21.15.40:0/2363645571 shutdown_connections 2023-10-18T08:57:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.561+0000 7f7e1a683640 1 --2- 172.21.15.40:0/2363645571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e140b0630 0x7f7e140c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.561+0000 7f7e1a683640 1 --2- 172.21.15.40:0/2363645571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e1406d6f0 0x7f7e140b15b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.561+0000 7f7e1a683640 1 -- 172.21.15.40:0/2363645571 >> 172.21.15.40:0/2363645571 conn(0x7f7e14069720 msgr2=0x7f7e14069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:10.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.561+0000 7f7e1a683640 1 -- 172.21.15.40:0/2363645571 shutdown_connections 2023-10-18T08:57:10.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.561+0000 7f7e1a683640 1 -- 172.21.15.40:0/2363645571 wait complete. 2023-10-18T08:57:10.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.563+0000 7f7e1a683640 1 Processor -- start 2023-10-18T08:57:10.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.563+0000 7f7e1a683640 1 -- start start 2023-10-18T08:57:10.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.565+0000 7f7e1a683640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e1406d6f0 0x7f7e140c2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:10.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.565+0000 7f7e1a683640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e140b0630 0x7f7e14144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:10.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.565+0000 7f7e1a683640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e140c9f00 0x7f7e140ad040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:10.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.565+0000 7f7e1a683640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e14148310 con 0x7f7e140c9f00 2023-10-18T08:57:10.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.565+0000 7f7e1a683640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e14145850 con 0x7f7e1406d6f0 2023-10-18T08:57:10.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.565+0000 7f7e1a683640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e14145b30 con 0x7f7e140b0630 2023-10-18T08:57:10.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.565+0000 7f7e19e82640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e140c9f00 0x7f7e140ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:10.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.565+0000 7f7e19e82640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e140c9f00 0x7f7e140ad040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40934/0 (socket says 172.21.15.40:40934) 2023-10-18T08:57:10.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.565+0000 7f7e19e82640 1 -- 172.21.15.40:0/1544271910 learned_addr learned my addr 172.21.15.40:0/1544271910 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:10.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.566+0000 7f7e18e80640 1 --2- 172.21.15.40:0/1544271910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e140b0630 0x7f7e14144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:10.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.566+0000 7f7e19681640 1 --2- 172.21.15.40:0/1544271910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e1406d6f0 0x7f7e140c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:10.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.566+0000 7f7e19e82640 1 -- 172.21.15.40:0/1544271910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e140b0630 msgr2=0x7f7e14144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:10.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.566+0000 7f7e19e82640 1 --2- 172.21.15.40:0/1544271910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e140b0630 0x7f7e14144a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:10.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.566+0000 7f7e19e82640 1 -- 172.21.15.40:0/1544271910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e1406d6f0 msgr2=0x7f7e140c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:10.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.566+0000 7f7e19e82640 1 --2- 172.21.15.40:0/1544271910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e1406d6f0 0x7f7e140c2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:10.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.566+0000 7f7e19e82640 1 -- 172.21.15.40:0/1544271910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e10009560 con 0x7f7e140c9f00 2023-10-18T08:57:10.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.566+0000 7f7e19e82640 1 --2- 172.21.15.40:0/1544271910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e140c9f00 0x7f7e140ad040 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c00b970 tx=0x7f7e0c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:10.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.566+0000 7f7e09ffb640 1 -- 172.21.15.40:0/1544271910 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c00cb40 con 0x7f7e140c9f00 2023-10-18T08:57:10.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.566+0000 7f7e09ffb640 1 -- 172.21.15.40:0/1544271910 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7e0c00cd90 con 0x7f7e140c9f00 2023-10-18T08:57:10.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.566+0000 7f7e09ffb640 1 -- 172.21.15.40:0/1544271910 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c0146a0 con 0x7f7e140c9f00 2023-10-18T08:57:10.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.566+0000 7f7e1a683640 1 -- 172.21.15.40:0/1544271910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e14145e10 con 0x7f7e140c9f00 2023-10-18T08:57:10.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.566+0000 7f7e1a683640 1 -- 172.21.15.40:0/1544271910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e141463d0 con 0x7f7e140c9f00 2023-10-18T08:57:10.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.568+0000 7f7e1a683640 1 -- 172.21.15.40:0/1544271910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e141465f0 con 0x7f7e140c9f00 2023-10-18T08:57:10.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.568+0000 7f7e09ffb640 1 -- 172.21.15.40:0/1544271910 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7e0c01e440 con 0x7f7e140c9f00 2023-10-18T08:57:10.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.569+0000 7f7e09ffb640 1 --2- 172.21.15.40:0/1544271910 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7df806ae40 0x7f7df806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:10.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.569+0000 7f7e09ffb640 1 -- 172.21.15.40:0/1544271910 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7e0c089330 con 0x7f7e140c9f00 2023-10-18T08:57:10.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.570+0000 7f7e19681640 1 --2- 172.21.15.40:0/1544271910 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7df806ae40 0x7f7df806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:10.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.571+0000 7f7e19681640 1 --2- 172.21.15.40:0/1544271910 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7df806ae40 0x7f7df806d300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7e10035000 tx=0x7f7e10004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:10.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.574+0000 7f7e09ffb640 1 -- 172.21.15.40:0/1544271910 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e0c0547c0 con 0x7f7e140c9f00 2023-10-18T08:57:10.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.840+0000 7f7e1a683640 1 -- 172.21.15.40:0/1544271910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7e14066e00 con 0x7f7e140c9f00 2023-10-18T08:57:10.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.841+0000 7f7e09ffb640 1 -- 172.21.15.40:0/1544271910 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7e0c057de0 con 0x7f7e140c9f00 2023-10-18T08:57:10.844 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:10.845 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T08:57:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2023-10-18T08:57:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2023-10-18T08:57:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lYsb5CxAAGButZ6+Usc62kJ0ksxZrYw==", 2023-10-18T08:57:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:57:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T08:57:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T08:57:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:57:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2023-10-18T08:57:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lAqlhMBAApAhxCYB9d9bz1GibADDRSQ==", 2023-10-18T08:57:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:57:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T08:57:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T08:57:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:57:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2023-10-18T08:57:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lPVh+HhAAzucSx+ksPDBkqJGa8+pyJw==", 2023-10-18T08:57:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.850 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:57:10.850 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T08:57:10.850 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T08:57:10.850 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:57:10.850 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.851 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2023-10-18T08:57:10.851 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lP7QCBhAAog7qDCgGVv0T7WKsRkATkA==", 2023-10-18T08:57:10.851 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.851 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:57:10.852 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T08:57:10.852 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T08:57:10.852 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:57:10.852 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.852 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.853 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.853 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2023-10-18T08:57:10.853 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lVx+hAhAA6ZpIyZKhbqy8tV84piGILA==", 2023-10-18T08:57:10.853 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.853 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:57:10.854 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:57:10.854 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:57:10.854 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.854 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.854 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.854 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2023-10-18T08:57:10.855 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lk9LwExAA7zFzQOn2hcvnck2cOZZ7AA==", 2023-10-18T08:57:10.855 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.855 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:57:10.855 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:57:10.855 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:57:10.856 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.856 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2023-10-18T08:57:10.856 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lK930JRAA1GyKRkIpXPHHGqY8owYWRQ==", 2023-10-18T08:57:10.857 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.857 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:57:10.857 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:57:10.857 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:57:10.857 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.858 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.858 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2023-10-18T08:57:10.858 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9liBWyNRAAMHBBdG5nbrGQupqvUtuKMA==", 2023-10-18T08:57:10.858 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.858 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:57:10.859 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:57:10.859 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:57:10.859 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.859 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.859 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.859 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2023-10-18T08:57:10.860 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9loDoOAhAAjHZub4wnB7dCr86rxAtI5g==", 2023-10-18T08:57:10.860 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.860 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:57:10.860 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:57:10.860 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:57:10.861 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.861 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.861 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2023-10-18T08:57:10.861 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lhqNfERAAw410g3DwSXrK23YkxVBFVw==", 2023-10-18T08:57:10.861 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.862 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:57:10.862 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:57:10.862 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:57:10.862 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.862 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.863 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2023-10-18T08:57:10.863 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lRRScIRAAg0pmfxCQBCPYodj1Lko+Lw==", 2023-10-18T08:57:10.863 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.863 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:57:10.864 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:57:10.864 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:57:10.864 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.865 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2023-10-18T08:57:10.865 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9ld1dXMhAAyovlrKOhVuynEiuPTgFcjQ==", 2023-10-18T08:57:10.865 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.865 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:57:10.865 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:57:10.865 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:57:10.866 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.866 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.866 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2023-10-18T08:57:10.867 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w==", 2023-10-18T08:57:10.867 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.867 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:57:10.867 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2023-10-18T08:57:10.867 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T08:57:10.867 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.868 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.868 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.868 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2023-10-18T08:57:10.868 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lAB3oKhAAD43WFdC23WGF6WUi4JKsvg==", 2023-10-18T08:57:10.868 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.869 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:57:10.869 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T08:57:10.869 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T08:57:10.869 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T08:57:10.869 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2023-10-18T08:57:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lu57fHxAAdzsVU2KKsWRkaARRS8TGjw==", 2023-10-18T08:57:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:57:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T08:57:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T08:57:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T08:57:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2023-10-18T08:57:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lygWsNRAA1OziXxk7hBtTw2Aq8OlSKQ==", 2023-10-18T08:57:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:57:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T08:57:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T08:57:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T08:57:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2023-10-18T08:57:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBMnS9lnrUjNhAAHP2eezRHF4/Jgox7iPwy7g==", 2023-10-18T08:57:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T08:57:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T08:57:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T08:57:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:57:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.876 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2023-10-18T08:57:10.876 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9lCf/xOhAAJHVoKIswWUcVGKQpiaJ12A==", 2023-10-18T08:57:10.877 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T08:57:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T08:57:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:57:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2023-10-18T08:57:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9loxRXLhAAQK1n1/4PB35umHxOKQzz7Q==", 2023-10-18T08:57:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:57:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T08:57:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T08:57:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:57:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:10.880 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T08:57:10.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.844+0000 7f7e1a683640 1 -- 172.21.15.40:0/1544271910 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7df806ae40 msgr2=0x7f7df806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:10.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.844+0000 7f7e1a683640 1 --2- 172.21.15.40:0/1544271910 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7df806ae40 0x7f7df806d300 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7e10035000 tx=0x7f7e10004730 comp rx=0 tx=0).stop 2023-10-18T08:57:10.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.844+0000 7f7e1a683640 1 -- 172.21.15.40:0/1544271910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e140c9f00 msgr2=0x7f7e140ad040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:10.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.844+0000 7f7e1a683640 1 --2- 172.21.15.40:0/1544271910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e140c9f00 0x7f7e140ad040 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c00b970 tx=0x7f7e0c00be30 comp rx=0 tx=0).stop 2023-10-18T08:57:10.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.844+0000 7f7e1a683640 1 -- 172.21.15.40:0/1544271910 shutdown_connections 2023-10-18T08:57:10.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.844+0000 7f7e1a683640 1 --2- 172.21.15.40:0/1544271910 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7df806ae40 0x7f7df806d300 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:10.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.844+0000 7f7e1a683640 1 --2- 172.21.15.40:0/1544271910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e140c9f00 0x7f7e140ad040 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:10.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.844+0000 7f7e1a683640 1 --2- 172.21.15.40:0/1544271910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e140b0630 0x7f7e14144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.844+0000 7f7e1a683640 1 --2- 172.21.15.40:0/1544271910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e1406d6f0 0x7f7e140c2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.844+0000 7f7e1a683640 1 -- 172.21.15.40:0/1544271910 >> 172.21.15.40:0/1544271910 conn(0x7f7e14069720 msgr2=0x7f7e140b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.844+0000 7f7e1a683640 1 -- 172.21.15.40:0/1544271910 shutdown_connections 2023-10-18T08:57:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:10.844+0000 7f7e1a683640 1 -- 172.21.15.40:0/1544271910 wait complete. 2023-10-18T08:57:10.885 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:57:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.115+0000 7fe628064640 1 -- 172.21.15.40:0/2150908115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6201586a0 msgr2=0x7fe6200b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.115+0000 7fe628064640 1 --2- 172.21.15.40:0/2150908115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6201586a0 0x7fe6200b0bc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe610005fd0 tx=0x7fe61002be10 comp rx=0 tx=0).stop 2023-10-18T08:57:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.115+0000 7fe628064640 1 -- 172.21.15.40:0/2150908115 shutdown_connections 2023-10-18T08:57:11.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.115+0000 7fe628064640 1 --2- 172.21.15.40:0/2150908115 >> v2:172.21.15.144:3300/0 conn(0x7fe6200b1100 0x7fe6201625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.115+0000 7fe628064640 1 --2- 172.21.15.40:0/2150908115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6201586a0 0x7fe6200b0bc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.115+0000 7fe628064640 1 -- 172.21.15.40:0/2150908115 >> 172.21.15.40:0/2150908115 conn(0x7fe62006cee0 msgr2=0x7fe620069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:11.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.115+0000 7fe628064640 1 -- 172.21.15.40:0/2150908115 shutdown_connections 2023-10-18T08:57:11.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.115+0000 7fe628064640 1 -- 172.21.15.40:0/2150908115 wait complete. 2023-10-18T08:57:11.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.117+0000 7fe628064640 1 Processor -- start 2023-10-18T08:57:11.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.117+0000 7fe628064640 1 -- start start 2023-10-18T08:57:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.118+0000 7fe628064640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe62006d710 0x7fe62014f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:11.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.118+0000 7fe628064640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6200b1100 0x7fe62014d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:11.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.118+0000 7fe628064640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6201586a0 0x7fe6201e56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:11.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.118+0000 7fe628064640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe620150b20 con 0x7fe6200b1100 2023-10-18T08:57:11.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.118+0000 7fe628064640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62014fea0 con 0x7fe62006d710 2023-10-18T08:57:11.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.118+0000 7fe628064640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe620150180 con 0x7fe6201586a0 2023-10-18T08:57:11.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.119+0000 7fe6255ff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6200b1100 0x7fe62014d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:11.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.119+0000 7fe6255ff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6200b1100 0x7fe62014d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40950/0 (socket says 172.21.15.40:40950) 2023-10-18T08:57:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.119+0000 7fe6255ff640 1 -- 172.21.15.40:0/3726476704 learned_addr learned my addr 172.21.15.40:0/3726476704 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.119+0000 7fe626601640 1 --2- 172.21.15.40:0/3726476704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6201586a0 0x7fe6201e56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.119+0000 7fe625e00640 1 --2- 172.21.15.40:0/3726476704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe62006d710 0x7fe62014f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.119+0000 7fe6255ff640 1 -- 172.21.15.40:0/3726476704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6201586a0 msgr2=0x7fe6201e56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:11.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.119+0000 7fe6255ff640 1 --2- 172.21.15.40:0/3726476704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6201586a0 0x7fe6201e56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.119+0000 7fe6255ff640 1 -- 172.21.15.40:0/3726476704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe62006d710 msgr2=0x7fe62014f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:11.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.119+0000 7fe6255ff640 1 --2- 172.21.15.40:0/3726476704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe62006d710 0x7fe62014f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.119+0000 7fe6255ff640 1 -- 172.21.15.40:0/3726476704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe610009560 con 0x7fe6200b1100 2023-10-18T08:57:11.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.120+0000 7fe6255ff640 1 --2- 172.21.15.40:0/3726476704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6200b1100 0x7fe62014d330 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe610036040 tx=0x7fe610005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:11.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.120+0000 7fe6167fc640 1 -- 172.21.15.40:0/3726476704 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61002de00 con 0x7fe6200b1100 2023-10-18T08:57:11.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.120+0000 7fe6167fc640 1 -- 172.21.15.40:0/3726476704 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe6100036e0 con 0x7fe6200b1100 2023-10-18T08:57:11.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.120+0000 7fe6167fc640 1 -- 172.21.15.40:0/3726476704 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610003a90 con 0x7fe6200b1100 2023-10-18T08:57:11.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.120+0000 7fe628064640 1 -- 172.21.15.40:0/3726476704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe620150460 con 0x7fe6200b1100 2023-10-18T08:57:11.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.120+0000 7fe628064640 1 -- 172.21.15.40:0/3726476704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62014bdf0 con 0x7fe6200b1100 2023-10-18T08:57:11.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.121+0000 7fe628064640 1 -- 172.21.15.40:0/3726476704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe620150c60 con 0x7fe6200b1100 2023-10-18T08:57:11.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.122+0000 7fe6167fc640 1 -- 172.21.15.40:0/3726476704 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe610003bf0 con 0x7fe6200b1100 2023-10-18T08:57:11.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.123+0000 7fe6167fc640 1 --2- 172.21.15.40:0/3726476704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe60406ae40 0x7fe60406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:11.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.123+0000 7fe6167fc640 1 -- 172.21.15.40:0/3726476704 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe6100abdc0 con 0x7fe6200b1100 2023-10-18T08:57:11.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.124+0000 7fe625e00640 1 --2- 172.21.15.40:0/3726476704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe60406ae40 0x7fe60406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:11.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.125+0000 7fe625e00640 1 --2- 172.21.15.40:0/3726476704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe60406ae40 0x7fe60406d300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe61c001370 tx=0x7fe61c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:11.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.128+0000 7fe6167fc640 1 -- 172.21.15.40:0/3726476704 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe620150c60 con 0x7fe6200b1100 2023-10-18T08:57:11.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.324+0000 7fe628064640 1 -- 172.21.15.40:0/3726476704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe620150c60 con 0x7fe6200b1100 2023-10-18T08:57:11.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.325+0000 7fe6167fc640 1 -- 172.21.15.40:0/3726476704 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+12869 (secure 0 0 0) 0x7fe620150c60 con 0x7fe6200b1100 2023-10-18T08:57:11.329 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:11.329 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":42,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:57:10.326049+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:57:11.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.328+0000 7fe628064640 1 -- 172.21.15.40:0/3726476704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe60406ae40 msgr2=0x7fe60406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:11.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.328+0000 7fe628064640 1 --2- 172.21.15.40:0/3726476704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe60406ae40 0x7fe60406d300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe61c001370 tx=0x7fe61c009040 comp rx=0 tx=0).stop 2023-10-18T08:57:11.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.328+0000 7fe628064640 1 -- 172.21.15.40:0/3726476704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6200b1100 msgr2=0x7fe62014d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:11.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.328+0000 7fe628064640 1 --2- 172.21.15.40:0/3726476704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6200b1100 0x7fe62014d330 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe610036040 tx=0x7fe610005e80 comp rx=0 tx=0).stop 2023-10-18T08:57:11.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.328+0000 7fe628064640 1 -- 172.21.15.40:0/3726476704 shutdown_connections 2023-10-18T08:57:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.329+0000 7fe628064640 1 --2- 172.21.15.40:0/3726476704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe60406ae40 0x7fe60406d300 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.329+0000 7fe628064640 1 --2- 172.21.15.40:0/3726476704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6201586a0 0x7fe6201e56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.329+0000 7fe628064640 1 --2- 172.21.15.40:0/3726476704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6200b1100 0x7fe62014d330 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.329+0000 7fe628064640 1 --2- 172.21.15.40:0/3726476704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe62006d710 0x7fe62014f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.329+0000 7fe628064640 1 -- 172.21.15.40:0/3726476704 >> 172.21.15.40:0/3726476704 conn(0x7fe62006cee0 msgr2=0x7fe620070150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:11.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.329+0000 7fe628064640 1 -- 172.21.15.40:0/3726476704 shutdown_connections 2023-10-18T08:57:11.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.329+0000 7fe628064640 1 -- 172.21.15.40:0/3726476704 wait complete. 2023-10-18T08:57:11.350 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:57:11.351 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:57:11.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.580+0000 7fa7734f2640 1 -- 172.21.15.40:0/2797113128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa76c14e730 msgr2=0x7fa76c0b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:11.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.580+0000 7fa7734f2640 1 --2- 172.21.15.40:0/2797113128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa76c14e730 0x7fa76c0b1950 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa76800a940 tx=0x7fa76802d080 comp rx=0 tx=0).stop 2023-10-18T08:57:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.580+0000 7fa7734f2640 1 -- 172.21.15.40:0/2797113128 shutdown_connections 2023-10-18T08:57:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.580+0000 7fa7734f2640 1 --2- 172.21.15.40:0/2797113128 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa76c0af410 0x7fa76c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.580+0000 7fa7734f2640 1 --2- 172.21.15.40:0/2797113128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa76c14e730 0x7fa76c0b1950 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.580+0000 7fa7734f2640 1 -- 172.21.15.40:0/2797113128 >> 172.21.15.40:0/2797113128 conn(0x7fa76c069720 msgr2=0x7fa76c069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.581+0000 7fa7734f2640 1 -- 172.21.15.40:0/2797113128 shutdown_connections 2023-10-18T08:57:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.581+0000 7fa7734f2640 1 -- 172.21.15.40:0/2797113128 wait complete. 2023-10-18T08:57:11.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.582+0000 7fa7734f2640 1 Processor -- start 2023-10-18T08:57:11.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.583+0000 7fa7734f2640 1 -- start start 2023-10-18T08:57:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.583+0000 7fa7734f2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa76c0715c0 0x7fa76c13a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.583+0000 7fa7734f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0af410 0x7fa76c1e0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.584+0000 7fa7734f2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa76c14e730 0x7fa76c13c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.584+0000 7fa7734f2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c1e2ab0 con 0x7fa76c0af410 2023-10-18T08:57:11.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.584+0000 7fa7734f2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c13d5f0 con 0x7fa76c0715c0 2023-10-18T08:57:11.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.584+0000 7fa7734f2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c13ab90 con 0x7fa76c14e730 2023-10-18T08:57:11.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.584+0000 7fa770a8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0af410 0x7fa76c1e0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:11.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.584+0000 7fa770a8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0af410 0x7fa76c1e0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40962/0 (socket says 172.21.15.40:40962) 2023-10-18T08:57:11.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.584+0000 7fa770a8d640 1 -- 172.21.15.40:0/3789804589 learned_addr learned my addr 172.21.15.40:0/3789804589 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:11.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.584+0000 7fa77128e640 1 --2- 172.21.15.40:0/3789804589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa76c0715c0 0x7fa76c13a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:11.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.584+0000 7fa771a8f640 1 --2- 172.21.15.40:0/3789804589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa76c14e730 0x7fa76c13c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:11.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.584+0000 7fa770a8d640 1 -- 172.21.15.40:0/3789804589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa76c14e730 msgr2=0x7fa76c13c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:11.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.584+0000 7fa770a8d640 1 --2- 172.21.15.40:0/3789804589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa76c14e730 0x7fa76c13c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.584+0000 7fa770a8d640 1 -- 172.21.15.40:0/3789804589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa76c0715c0 msgr2=0x7fa76c13a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:11.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.584+0000 7fa770a8d640 1 --2- 172.21.15.40:0/3789804589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa76c0715c0 0x7fa76c13a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.584+0000 7fa770a8d640 1 -- 172.21.15.40:0/3789804589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa764009900 con 0x7fa76c0af410 2023-10-18T08:57:11.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.585+0000 7fa770a8d640 1 --2- 172.21.15.40:0/3789804589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0af410 0x7fa76c1e0a50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00c430 tx=0x7fa75c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:11.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.585+0000 7fa761ffb640 1 -- 172.21.15.40:0/3789804589 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0052d0 con 0x7fa76c0af410 2023-10-18T08:57:11.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.585+0000 7fa761ffb640 1 -- 172.21.15.40:0/3789804589 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa75c0035f0 con 0x7fa76c0af410 2023-10-18T08:57:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.585+0000 7fa761ffb640 1 -- 172.21.15.40:0/3789804589 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c003a00 con 0x7fa76c0af410 2023-10-18T08:57:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.585+0000 7fa7734f2640 1 -- 172.21.15.40:0/3789804589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76800a5a0 con 0x7fa76c0af410 2023-10-18T08:57:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.585+0000 7fa7734f2640 1 -- 172.21.15.40:0/3789804589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76c13d310 con 0x7fa76c0af410 2023-10-18T08:57:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.586+0000 7fa7734f2640 1 -- 172.21.15.40:0/3789804589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76c1e2c60 con 0x7fa76c0af410 2023-10-18T08:57:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.587+0000 7fa761ffb640 1 -- 172.21.15.40:0/3789804589 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa75c003b60 con 0x7fa76c0af410 2023-10-18T08:57:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.588+0000 7fa761ffb640 1 --2- 172.21.15.40:0/3789804589 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa75006ae70 0x7fa75006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:11.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.588+0000 7fa761ffb640 1 -- 172.21.15.40:0/3789804589 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa75c088b90 con 0x7fa76c0af410 2023-10-18T08:57:11.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.589+0000 7fa77128e640 1 --2- 172.21.15.40:0/3789804589 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa75006ae70 0x7fa75006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:11.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.590+0000 7fa77128e640 1 --2- 172.21.15.40:0/3789804589 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa75006ae70 0x7fa75006d330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa768037000 tx=0x7fa768004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:11.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.593+0000 7fa761ffb640 1 -- 172.21.15.40:0/3789804589 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa76c1e2c60 con 0x7fa76c0af410 2023-10-18T08:57:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.783+0000 7fa7734f2640 1 -- 172.21.15.40:0/3789804589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa76c066fd0 con 0x7fa76c0af410 2023-10-18T08:57:11.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.784+0000 7fa761ffb640 1 -- 172.21.15.40:0/3789804589 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa75c0170a0 con 0x7fa76c0af410 2023-10-18T08:57:11.788 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:57:11.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.787+0000 7fa7734f2640 1 -- 172.21.15.40:0/3789804589 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa75006ae70 msgr2=0x7fa75006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.787+0000 7fa7734f2640 1 --2- 172.21.15.40:0/3789804589 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa75006ae70 0x7fa75006d330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa768037000 tx=0x7fa768004730 comp rx=0 tx=0).stop 2023-10-18T08:57:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.787+0000 7fa7734f2640 1 -- 172.21.15.40:0/3789804589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0af410 msgr2=0x7fa76c1e0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.787+0000 7fa7734f2640 1 --2- 172.21.15.40:0/3789804589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0af410 0x7fa76c1e0a50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00c430 tx=0x7fa75c00c8f0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.787+0000 7fa7734f2640 1 -- 172.21.15.40:0/3789804589 shutdown_connections 2023-10-18T08:57:11.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.787+0000 7fa7734f2640 1 --2- 172.21.15.40:0/3789804589 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa75006ae70 0x7fa75006d330 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.787+0000 7fa7734f2640 1 --2- 172.21.15.40:0/3789804589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa76c14e730 0x7fa76c13c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.787+0000 7fa7734f2640 1 --2- 172.21.15.40:0/3789804589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0af410 0x7fa76c1e0a50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.787+0000 7fa7734f2640 1 --2- 172.21.15.40:0/3789804589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa76c0715c0 0x7fa76c13a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.787+0000 7fa7734f2640 1 -- 172.21.15.40:0/3789804589 >> 172.21.15.40:0/3789804589 conn(0x7fa76c069720 msgr2=0x7fa76c14daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.787+0000 7fa7734f2640 1 -- 172.21.15.40:0/3789804589 shutdown_connections 2023-10-18T08:57:11.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:11.787+0000 7fa7734f2640 1 -- 172.21.15.40:0/3789804589 wait complete. 2023-10-18T08:57:11.808 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T08:57:12.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.033+0000 7fbb50202640 1 -- 172.21.15.40:0/2367856088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb480b1b90 msgr2=0x7fbb480cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.033+0000 7fbb50202640 1 --2- 172.21.15.40:0/2367856088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb480b1b90 0x7fbb480cd2f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009860 tx=0x7fbb3802bd40 comp rx=0 tx=0).stop 2023-10-18T08:57:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.033+0000 7fbb50202640 1 -- 172.21.15.40:0/2367856088 shutdown_connections 2023-10-18T08:57:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.033+0000 7fbb50202640 1 --2- 172.21.15.40:0/2367856088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb480b1b90 0x7fbb480cd2f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:12.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.033+0000 7fbb50202640 1 --2- 172.21.15.40:0/2367856088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb4815eaa0 0x7fbb480ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.033+0000 7fbb50202640 1 -- 172.21.15.40:0/2367856088 >> 172.21.15.40:0/2367856088 conn(0x7fbb4806cee0 msgr2=0x7fbb48069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.033+0000 7fbb50202640 1 -- 172.21.15.40:0/2367856088 shutdown_connections 2023-10-18T08:57:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.033+0000 7fbb50202640 1 -- 172.21.15.40:0/2367856088 wait complete. 2023-10-18T08:57:12.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.036+0000 7fbb50202640 1 Processor -- start 2023-10-18T08:57:12.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.036+0000 7fbb50202640 1 -- start start 2023-10-18T08:57:12.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.037+0000 7fbb50202640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb4806ffe0 0x7fbb48156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:12.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.037+0000 7fbb50202640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb480b1b90 0x7fbb481571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:12.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.037+0000 7fbb50202640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb4815eaa0 0x7fbb48155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:12.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.037+0000 7fbb50202640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb48156ba0 con 0x7fbb4806ffe0 2023-10-18T08:57:12.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.037+0000 7fbb50202640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb48157720 con 0x7fbb4815eaa0 2023-10-18T08:57:12.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.037+0000 7fbb50202640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb48157b30 con 0x7fbb480b1b90 2023-10-18T08:57:12.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.037+0000 7fbb4df9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb4806ffe0 0x7fbb48156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:12.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.037+0000 7fbb4df9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb4806ffe0 0x7fbb48156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40970/0 (socket says 172.21.15.40:40970) 2023-10-18T08:57:12.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.037+0000 7fbb4df9e640 1 -- 172.21.15.40:0/2089614132 learned_addr learned my addr 172.21.15.40:0/2089614132 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:12.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.038+0000 7fbb4df9e640 1 -- 172.21.15.40:0/2089614132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb480b1b90 msgr2=0x7fbb481571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:12.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.038+0000 7fbb4df9e640 1 --2- 172.21.15.40:0/2089614132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb480b1b90 0x7fbb481571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:12.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.038+0000 7fbb4df9e640 1 -- 172.21.15.40:0/2089614132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb4815eaa0 msgr2=0x7fbb48155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:12.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.038+0000 7fbb4e79f640 1 --2- 172.21.15.40:0/2089614132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb4815eaa0 0x7fbb48155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:12.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.038+0000 7fbb4df9e640 1 --2- 172.21.15.40:0/2089614132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb4815eaa0 0x7fbb48155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:12.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.038+0000 7fbb4df9e640 1 -- 172.21.15.40:0/2089614132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb38009490 con 0x7fbb4806ffe0 2023-10-18T08:57:12.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.038+0000 7fbb4e79f640 1 --2- 172.21.15.40:0/2089614132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb4815eaa0 0x7fbb48155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.038+0000 7fbb4df9e640 1 --2- 172.21.15.40:0/2089614132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb4806ffe0 0x7fbb48156660 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400e790 tx=0x7fbb4400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.038+0000 7fbb3e7fc640 1 -- 172.21.15.40:0/2089614132 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4400cc60 con 0x7fbb4806ffe0 2023-10-18T08:57:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.039+0000 7fbb3e7fc640 1 -- 172.21.15.40:0/2089614132 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbb4400eea0 con 0x7fbb4806ffe0 2023-10-18T08:57:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.039+0000 7fbb3e7fc640 1 -- 172.21.15.40:0/2089614132 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44010690 con 0x7fbb4806ffe0 2023-10-18T08:57:12.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.039+0000 7fbb50202640 1 -- 172.21.15.40:0/2089614132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb48157e10 con 0x7fbb4806ffe0 2023-10-18T08:57:12.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.039+0000 7fbb50202640 1 -- 172.21.15.40:0/2089614132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb481583d0 con 0x7fbb4806ffe0 2023-10-18T08:57:12.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.040+0000 7fbb50202640 1 -- 172.21.15.40:0/2089614132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb48066fd0 con 0x7fbb4806ffe0 2023-10-18T08:57:12.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.041+0000 7fbb3e7fc640 1 -- 172.21.15.40:0/2089614132 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbb440107f0 con 0x7fbb4806ffe0 2023-10-18T08:57:12.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.042+0000 7fbb3e7fc640 1 --2- 172.21.15.40:0/2089614132 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb2c06ae40 0x7fbb2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:12.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.042+0000 7fbb3e7fc640 1 -- 172.21.15.40:0/2089614132 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbb44014070 con 0x7fbb4806ffe0 2023-10-18T08:57:12.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.043+0000 7fbb4d79d640 1 --2- 172.21.15.40:0/2089614132 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb2c06ae40 0x7fbb2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:12.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.044+0000 7fbb4d79d640 1 --2- 172.21.15.40:0/2089614132 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb2c06ae40 0x7fbb2c06d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbb38036000 tx=0x7fbb380046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:12.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.046+0000 7fbb3e7fc640 1 -- 172.21.15.40:0/2089614132 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb44051ca0 con 0x7fbb4806ffe0 2023-10-18T08:57:12.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.244+0000 7fbb50202640 1 -- 172.21.15.40:0/2089614132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbb480aa950 con 0x7fbb4806ffe0 2023-10-18T08:57:12.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.245+0000 7fbb3e7fc640 1 -- 172.21.15.40:0/2089614132 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbb44055160 con 0x7fbb4806ffe0 2023-10-18T08:57:12.248 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:12.249 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-10-18T08:57:12.249 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:12.249 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-10-18T08:57:12.249 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-10-18T08:57:12.249 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:12.250 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-10-18T08:57:12.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.248+0000 7fbb50202640 1 -- 172.21.15.40:0/2089614132 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb2c06ae40 msgr2=0x7fbb2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:12.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.248+0000 7fbb50202640 1 --2- 172.21.15.40:0/2089614132 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb2c06ae40 0x7fbb2c06d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbb38036000 tx=0x7fbb380046c0 comp rx=0 tx=0).stop 2023-10-18T08:57:12.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.248+0000 7fbb50202640 1 -- 172.21.15.40:0/2089614132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb4806ffe0 msgr2=0x7fbb48156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:12.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.248+0000 7fbb50202640 1 --2- 172.21.15.40:0/2089614132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb4806ffe0 0x7fbb48156660 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400e790 tx=0x7fbb4400ec50 comp rx=0 tx=0).stop 2023-10-18T08:57:12.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.248+0000 7fbb50202640 1 -- 172.21.15.40:0/2089614132 shutdown_connections 2023-10-18T08:57:12.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.248+0000 7fbb50202640 1 --2- 172.21.15.40:0/2089614132 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb2c06ae40 0x7fbb2c06d300 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:12.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.248+0000 7fbb50202640 1 --2- 172.21.15.40:0/2089614132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb4815eaa0 0x7fbb48155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:12.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.248+0000 7fbb50202640 1 --2- 172.21.15.40:0/2089614132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb480b1b90 0x7fbb481571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:12.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.248+0000 7fbb50202640 1 --2- 172.21.15.40:0/2089614132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb4806ffe0 0x7fbb48156660 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:12.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.248+0000 7fbb50202640 1 -- 172.21.15.40:0/2089614132 >> 172.21.15.40:0/2089614132 conn(0x7fbb4806cee0 msgr2=0x7fbb480aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:12.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.248+0000 7fbb50202640 1 -- 172.21.15.40:0/2089614132 shutdown_connections 2023-10-18T08:57:12.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.248+0000 7fbb50202640 1 -- 172.21.15.40:0/2089614132 wait complete. 2023-10-18T08:57:12.270 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T08:57:12.270 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T08:57:12.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.501+0000 7f205291d640 1 -- 172.21.15.40:0/196416679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f202c004d00 msgr2=0x7f202c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.501+0000 7f205291d640 1 --2- 172.21.15.40:0/196416679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f202c004d00 0x7f202c005180 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2044009ed0 tx=0x7f202c0057f0 comp rx=0 tx=0).stop 2023-10-18T08:57:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.501+0000 7f2050eba640 1 -- 172.21.15.40:0/196416679 reap_dead start 2023-10-18T08:57:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.501+0000 7f205291d640 1 -- 172.21.15.40:0/196416679 shutdown_connections 2023-10-18T08:57:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.501+0000 7f205291d640 1 -- 172.21.15.40:0/196416679 >> 172.21.15.40:0/196416679 conn(0x7f204c0ce030 msgr2=0x7f204c0ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.501+0000 7f205291d640 1 -- 172.21.15.40:0/196416679 shutdown_connections 2023-10-18T08:57:12.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.501+0000 7f205291d640 1 -- 172.21.15.40:0/196416679 wait complete. 2023-10-18T08:57:12.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.503+0000 7f205291d640 1 Processor -- start 2023-10-18T08:57:12.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.503+0000 7f205291d640 1 -- start start 2023-10-18T08:57:12.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.504+0000 7f205291d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f202c004d00 0x7f204c1f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.504+0000 7f205291d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c0b2a40 0x7f204c0b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.504+0000 7f205291d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f204c150dc0 0x7f204c13e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.504+0000 7f205291d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c13b2d0 con 0x7f204c0b2a40 2023-10-18T08:57:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.504+0000 7f205291d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c151830 con 0x7f204c150dc0 2023-10-18T08:57:12.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.504+0000 7f205291d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c151550 con 0x7f202c004d00 2023-10-18T08:57:12.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.504+0000 7f20437fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c0b2a40 0x7f204c0b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:12.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.504+0000 7f20437fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c0b2a40 0x7f204c0b3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40988/0 (socket says 172.21.15.40:40988) 2023-10-18T08:57:12.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.504+0000 7f20437fe640 1 -- 172.21.15.40:0/3240687641 learned_addr learned my addr 172.21.15.40:0/3240687641 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:12.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.505+0000 7f204bfff640 1 --2- 172.21.15.40:0/3240687641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f202c004d00 0x7f204c1f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.505+0000 7f2050eba640 1 --2- 172.21.15.40:0/3240687641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f204c150dc0 0x7f204c13e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.505+0000 7f20437fe640 1 -- 172.21.15.40:0/3240687641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f202c004d00 msgr2=0x7f204c1f14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.505+0000 7f20437fe640 1 --2- 172.21.15.40:0/3240687641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f202c004d00 0x7f204c1f14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.505+0000 7f20437fe640 1 -- 172.21.15.40:0/3240687641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f204c150dc0 msgr2=0x7f204c13e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.505+0000 7f20437fe640 1 --2- 172.21.15.40:0/3240687641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f204c150dc0 0x7f204c13e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.505+0000 7f20437fe640 1 -- 172.21.15.40:0/3240687641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c0b1f60 con 0x7f204c0b2a40 2023-10-18T08:57:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.505+0000 7f2050eba640 1 --2- 172.21.15.40:0/3240687641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f204c150dc0 0x7f204c13e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.505+0000 7f20437fe640 1 --2- 172.21.15.40:0/3240687641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c0b2a40 0x7f204c0b3810 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f203400e970 tx=0x7f203400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.505+0000 7f20497fa640 1 -- 172.21.15.40:0/3240687641 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203400ccc0 con 0x7f204c0b2a40 2023-10-18T08:57:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.505+0000 7f20497fa640 1 -- 172.21.15.40:0/3240687641 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f203400ce20 con 0x7f204c0b2a40 2023-10-18T08:57:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.505+0000 7f20497fa640 1 -- 172.21.15.40:0/3240687641 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034005280 con 0x7f204c0b2a40 2023-10-18T08:57:12.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.505+0000 7f205291d640 1 -- 172.21.15.40:0/3240687641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204c0b2240 con 0x7f204c0b2a40 2023-10-18T08:57:12.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.506+0000 7f205291d640 1 -- 172.21.15.40:0/3240687641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204c151250 con 0x7f204c0b2a40 2023-10-18T08:57:12.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.507+0000 7f205291d640 1 -- 172.21.15.40:0/3240687641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204c0b1dc0 con 0x7f204c0b2a40 2023-10-18T08:57:12.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.508+0000 7f20497fa640 1 -- 172.21.15.40:0/3240687641 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f20340053e0 con 0x7f204c0b2a40 2023-10-18T08:57:12.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.508+0000 7f20497fa640 1 --2- 172.21.15.40:0/3240687641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f203006ae70 0x7f203006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:12.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.509+0000 7f20497fa640 1 -- 172.21.15.40:0/3240687641 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2034014070 con 0x7f204c0b2a40 2023-10-18T08:57:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.509+0000 7f204bfff640 1 --2- 172.21.15.40:0/3240687641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f203006ae70 0x7f203006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.510+0000 7f204bfff640 1 --2- 172.21.15.40:0/3240687641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f203006ae70 0x7f203006d330 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f203c000f60 tx=0x7f203c001c10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.513+0000 7f20497fa640 1 -- 172.21.15.40:0/3240687641 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f204c0b1dc0 con 0x7f204c0b2a40 2023-10-18T08:57:12.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:12.707+0000 7f205291d640 1 -- 172.21.15.40:0/3240687641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f204c0b1dc0 con 0x7f204c0b2a40 2023-10-18T08:57:13.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.333+0000 7f20497fa640 1 -- 172.21.15.40:0/3240687641 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) v1 ==== 138+0+0 (secure 0 0 0) 0x7f204c0b1dc0 con 0x7f204c0b2a40 2023-10-18T08:57:13.337 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-10-18T08:57:13.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.336+0000 7f205291d640 1 -- 172.21.15.40:0/3240687641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f203006ae70 msgr2=0x7f203006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:13.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.337+0000 7f205291d640 1 --2- 172.21.15.40:0/3240687641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f203006ae70 0x7f203006d330 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f203c000f60 tx=0x7f203c001c10 comp rx=0 tx=0).stop 2023-10-18T08:57:13.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.337+0000 7f205291d640 1 -- 172.21.15.40:0/3240687641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c0b2a40 msgr2=0x7f204c0b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:13.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.337+0000 7f205291d640 1 --2- 172.21.15.40:0/3240687641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c0b2a40 0x7f204c0b3810 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f203400e970 tx=0x7f203400ee30 comp rx=0 tx=0).stop 2023-10-18T08:57:13.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.337+0000 7f205291d640 1 -- 172.21.15.40:0/3240687641 shutdown_connections 2023-10-18T08:57:13.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.337+0000 7f205291d640 1 --2- 172.21.15.40:0/3240687641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f203006ae70 0x7f203006d330 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:13.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.337+0000 7f205291d640 1 --2- 172.21.15.40:0/3240687641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f204c150dc0 0x7f204c13e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:13.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.337+0000 7f205291d640 1 --2- 172.21.15.40:0/3240687641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f204c0b2a40 0x7f204c0b3810 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:13.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.337+0000 7f205291d640 1 --2- 172.21.15.40:0/3240687641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f202c004d00 0x7f204c1f14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:13.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.337+0000 7f205291d640 1 -- 172.21.15.40:0/3240687641 >> 172.21.15.40:0/3240687641 conn(0x7f204c0ce030 msgr2=0x7f204c070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:13.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.337+0000 7f205291d640 1 -- 172.21.15.40:0/3240687641 shutdown_connections 2023-10-18T08:57:13.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.337+0000 7f205291d640 1 -- 172.21.15.40:0/3240687641 wait complete. 2023-10-18T08:57:13.364 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T08:57:13.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.590+0000 7f9c43fff640 1 -- 172.21.15.40:0/1162397893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c30003970 msgr2=0x7f9c30003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.590+0000 7f9c43fff640 1 --2- 172.21.15.40:0/1162397893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c30003970 0x7f9c30003d90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9c34005fd0 tx=0x7f9c300057f0 comp rx=0 tx=0).stop 2023-10-18T08:57:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.590+0000 7f9c437fe640 1 -- 172.21.15.40:0/1162397893 reap_dead start 2023-10-18T08:57:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.590+0000 7f9c43fff640 1 -- 172.21.15.40:0/1162397893 shutdown_connections 2023-10-18T08:57:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.590+0000 7f9c43fff640 1 -- 172.21.15.40:0/1162397893 >> 172.21.15.40:0/1162397893 conn(0x7f9c4406ca10 msgr2=0x7f9c4406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.590+0000 7f9c43fff640 1 -- 172.21.15.40:0/1162397893 shutdown_connections 2023-10-18T08:57:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.590+0000 7f9c43fff640 1 -- 172.21.15.40:0/1162397893 wait complete. 2023-10-18T08:57:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.593+0000 7f9c43fff640 1 Processor -- start 2023-10-18T08:57:13.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.593+0000 7f9c43fff640 1 -- start start 2023-10-18T08:57:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c43fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c30003970 0x7f9c441530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c43fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c440ad990 0x7f9c440ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c43fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c441425a0 0x7f9c44142a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c43fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c44140e20 con 0x7f9c440ad990 2023-10-18T08:57:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c43fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c440ae350 con 0x7f9c441425a0 2023-10-18T08:57:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c43fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c44140510 con 0x7f9c30003970 2023-10-18T08:57:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c427fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c440ad990 0x7f9c440ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c427fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c440ad990 0x7f9c440ade10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41000/0 (socket says 172.21.15.40:41000) 2023-10-18T08:57:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c427fc640 1 -- 172.21.15.40:0/4038596936 learned_addr learned my addr 172.21.15.40:0/4038596936 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c42ffd640 1 --2- 172.21.15.40:0/4038596936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c30003970 0x7f9c441530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c427fc640 1 -- 172.21.15.40:0/4038596936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c30003970 msgr2=0x7f9c441530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c427fc640 1 --2- 172.21.15.40:0/4038596936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c30003970 0x7f9c441530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c427fc640 1 -- 172.21.15.40:0/4038596936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c441425a0 msgr2=0x7f9c44142a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c427fc640 1 --2- 172.21.15.40:0/4038596936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c441425a0 0x7f9c44142a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c427fc640 1 -- 172.21.15.40:0/4038596936 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c44151590 con 0x7f9c440ad990 2023-10-18T08:57:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c427fc640 1 --2- 172.21.15.40:0/4038596936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c440ad990 0x7f9c440ade10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00d780 tx=0x7f9c2c00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c3b7fe640 1 -- 172.21.15.40:0/4038596936 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c005220 con 0x7f9c440ad990 2023-10-18T08:57:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c3b7fe640 1 -- 172.21.15.40:0/4038596936 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9c2c005490 con 0x7f9c440ad990 2023-10-18T08:57:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.594+0000 7f9c3b7fe640 1 -- 172.21.15.40:0/4038596936 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c009a20 con 0x7f9c440ad990 2023-10-18T08:57:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.595+0000 7f9c43fff640 1 -- 172.21.15.40:0/4038596936 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c44143310 con 0x7f9c440ad990 2023-10-18T08:57:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.595+0000 7f9c43fff640 1 -- 172.21.15.40:0/4038596936 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c44140a40 con 0x7f9c440ad990 2023-10-18T08:57:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.596+0000 7f9c43fff640 1 -- 172.21.15.40:0/4038596936 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c44140f60 con 0x7f9c440ad990 2023-10-18T08:57:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.598+0000 7f9c3b7fe640 1 -- 172.21.15.40:0/4038596936 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9c2c009b80 con 0x7f9c440ad990 2023-10-18T08:57:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.600+0000 7f9c3b7fe640 1 --2- 172.21.15.40:0/4038596936 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9c2406ae70 0x7f9c2406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.600+0000 7f9c3b7fe640 1 -- 172.21.15.40:0/4038596936 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9c2c0896f0 con 0x7f9c440ad990 2023-10-18T08:57:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.600+0000 7f9c42ffd640 1 --2- 172.21.15.40:0/4038596936 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9c2406ae70 0x7f9c2406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.600+0000 7f9c42ffd640 1 --2- 172.21.15.40:0/4038596936 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9c2406ae70 0x7f9c2406d330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9c34038000 tx=0x7f9c34004a40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:13.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.603+0000 7f9c3b7fe640 1 -- 172.21.15.40:0/4038596936 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c2c054af0 con 0x7f9c440ad990 2023-10-18T08:57:13.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:13.796+0000 7f9c43fff640 1 -- 172.21.15.40:0/4038596936 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9c44140f60 con 0x7f9c440ad990 2023-10-18T08:57:14.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.340+0000 7f9c3b7fe640 1 -- 172.21.15.40:0/4038596936 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9c44140f60 con 0x7f9c440ad990 2023-10-18T08:57:14.343 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-10-18T08:57:14.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.345+0000 7f9c43fff640 1 -- 172.21.15.40:0/4038596936 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9c2406ae70 msgr2=0x7f9c2406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:14.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.345+0000 7f9c43fff640 1 --2- 172.21.15.40:0/4038596936 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9c2406ae70 0x7f9c2406d330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9c34038000 tx=0x7f9c34004a40 comp rx=0 tx=0).stop 2023-10-18T08:57:14.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.345+0000 7f9c43fff640 1 -- 172.21.15.40:0/4038596936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c440ad990 msgr2=0x7f9c440ade10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:14.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.345+0000 7f9c43fff640 1 --2- 172.21.15.40:0/4038596936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c440ad990 0x7f9c440ade10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00d780 tx=0x7f9c2c00dc40 comp rx=0 tx=0).stop 2023-10-18T08:57:14.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.345+0000 7f9c43fff640 1 -- 172.21.15.40:0/4038596936 shutdown_connections 2023-10-18T08:57:14.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.345+0000 7f9c43fff640 1 --2- 172.21.15.40:0/4038596936 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9c2406ae70 0x7f9c2406d330 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:14.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.345+0000 7f9c43fff640 1 --2- 172.21.15.40:0/4038596936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c441425a0 0x7f9c44142a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:14.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.345+0000 7f9c43fff640 1 --2- 172.21.15.40:0/4038596936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c440ad990 0x7f9c440ade10 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:14.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.345+0000 7f9c43fff640 1 --2- 172.21.15.40:0/4038596936 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c30003970 0x7f9c441530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:14.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.345+0000 7f9c43fff640 1 -- 172.21.15.40:0/4038596936 >> 172.21.15.40:0/4038596936 conn(0x7f9c4406ca10 msgr2=0x7f9c440acbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:14.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.346+0000 7f9c43fff640 1 -- 172.21.15.40:0/4038596936 shutdown_connections 2023-10-18T08:57:14.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.346+0000 7f9c43fff640 1 -- 172.21.15.40:0/4038596936 wait complete. 2023-10-18T08:57:14.378 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T08:57:14.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.578+0000 7f47c5b43640 1 -- 172.21.15.40:0/2830806695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a80047c0 msgr2=0x7f47a8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:14.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.578+0000 7f47c5b43640 1 --2- 172.21.15.40:0/2830806695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a80047c0 0x7f47a8004c40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f47b0001680 tx=0x7f47a80057f0 comp rx=0 tx=0).stop 2023-10-18T08:57:14.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.578+0000 7f47c5342640 1 -- 172.21.15.40:0/2830806695 reap_dead start 2023-10-18T08:57:14.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.578+0000 7f47c5b43640 1 -- 172.21.15.40:0/2830806695 shutdown_connections 2023-10-18T08:57:14.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.578+0000 7f47c5b43640 1 -- 172.21.15.40:0/2830806695 >> 172.21.15.40:0/2830806695 conn(0x7f47c006ca10 msgr2=0x7f47c006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:14.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.578+0000 7f47c5b43640 1 -- 172.21.15.40:0/2830806695 shutdown_connections 2023-10-18T08:57:14.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.578+0000 7f47c5b43640 1 -- 172.21.15.40:0/2830806695 wait complete. 2023-10-18T08:57:14.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.580+0000 7f47c5b43640 1 Processor -- start 2023-10-18T08:57:14.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.580+0000 7f47c5b43640 1 -- start start 2023-10-18T08:57:14.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.581+0000 7f47c5b43640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a80047c0 0x7f47c00cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:14.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.581+0000 7f47c5b43640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47c01f6040 0x7f47c00cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:14.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.581+0000 7f47c5b43640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47c014ee60 0x7f47c01f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:14.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.581+0000 7f47c5b43640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c01f5380 con 0x7f47a80047c0 2023-10-18T08:57:14.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.581+0000 7f47c5b43640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c01f5bb0 con 0x7f47c014ee60 2023-10-18T08:57:14.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.581+0000 7f47c5b43640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c01f2ac0 con 0x7f47c01f6040 2023-10-18T08:57:14.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.581+0000 7f47c4b41640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a80047c0 0x7f47c00cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:14.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.581+0000 7f47c4b41640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a80047c0 0x7f47c00cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41012/0 (socket says 172.21.15.40:41012) 2023-10-18T08:57:14.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.581+0000 7f47c4b41640 1 -- 172.21.15.40:0/1246649036 learned_addr learned my addr 172.21.15.40:0/1246649036 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:14.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.582+0000 7f47c4b41640 1 -- 172.21.15.40:0/1246649036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47c01f6040 msgr2=0x7f47c00cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:14.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.582+0000 7f47c4b41640 1 --2- 172.21.15.40:0/1246649036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47c01f6040 0x7f47c00cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:14.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.582+0000 7f47c4b41640 1 -- 172.21.15.40:0/1246649036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47c014ee60 msgr2=0x7f47c01f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:14.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.582+0000 7f47c4b41640 1 --2- 172.21.15.40:0/1246649036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47c014ee60 0x7f47c01f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:14.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.582+0000 7f47c4b41640 1 -- 172.21.15.40:0/1246649036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c01f5560 con 0x7f47a80047c0 2023-10-18T08:57:14.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.582+0000 7f47c4b41640 1 --2- 172.21.15.40:0/1246649036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a80047c0 0x7f47c00cd020 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f47b400e970 tx=0x7f47b400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:14.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.582+0000 7f47bd7fa640 1 -- 172.21.15.40:0/1246649036 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b400ccc0 con 0x7f47a80047c0 2023-10-18T08:57:14.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.582+0000 7f47bd7fa640 1 -- 172.21.15.40:0/1246649036 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f47b400ce40 con 0x7f47a80047c0 2023-10-18T08:57:14.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.582+0000 7f47c5b43640 1 -- 172.21.15.40:0/1246649036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c01f5840 con 0x7f47a80047c0 2023-10-18T08:57:14.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.582+0000 7f47bd7fa640 1 -- 172.21.15.40:0/1246649036 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4005280 con 0x7f47a80047c0 2023-10-18T08:57:14.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.583+0000 7f47c5b43640 1 -- 172.21.15.40:0/1246649036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c01f24a0 con 0x7f47a80047c0 2023-10-18T08:57:14.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.584+0000 7f47c5b43640 1 -- 172.21.15.40:0/1246649036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c0066e00 con 0x7f47a80047c0 2023-10-18T08:57:14.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.585+0000 7f47bd7fa640 1 -- 172.21.15.40:0/1246649036 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f47b40053e0 con 0x7f47a80047c0 2023-10-18T08:57:14.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.586+0000 7f47bd7fa640 1 --2- 172.21.15.40:0/1246649036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f47ac06ae70 0x7f47ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:14.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.586+0000 7f47bd7fa640 1 -- 172.21.15.40:0/1246649036 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f47b4014070 con 0x7f47a80047c0 2023-10-18T08:57:14.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.587+0000 7f47bffff640 1 --2- 172.21.15.40:0/1246649036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f47ac06ae70 0x7f47ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:14.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.588+0000 7f47bffff640 1 --2- 172.21.15.40:0/1246649036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f47ac06ae70 0x7f47ac06d330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f47b0034000 tx=0x7f47b0001120 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:14.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.591+0000 7f47bd7fa640 1 -- 172.21.15.40:0/1246649036 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f47b4055060 con 0x7f47a80047c0 2023-10-18T08:57:14.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:14.823+0000 7f47c5b43640 1 -- 172.21.15.40:0/1246649036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f47c00aba60 con 0x7f47a80047c0 2023-10-18T08:57:15.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.388+0000 7f47bd7fa640 1 -- 172.21.15.40:0/1246649036 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7f47b4058680 con 0x7f47a80047c0 2023-10-18T08:57:15.394 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 6 and data pool 7 2023-10-18T08:57:15.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.394+0000 7f47c5b43640 1 -- 172.21.15.40:0/1246649036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f47ac06ae70 msgr2=0x7f47ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:15.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.394+0000 7f47c5b43640 1 --2- 172.21.15.40:0/1246649036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f47ac06ae70 0x7f47ac06d330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f47b0034000 tx=0x7f47b0001120 comp rx=0 tx=0).stop 2023-10-18T08:57:15.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.394+0000 7f47c5b43640 1 -- 172.21.15.40:0/1246649036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a80047c0 msgr2=0x7f47c00cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:15.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.394+0000 7f47c5b43640 1 --2- 172.21.15.40:0/1246649036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a80047c0 0x7f47c00cd020 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f47b400e970 tx=0x7f47b400ee30 comp rx=0 tx=0).stop 2023-10-18T08:57:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.396+0000 7f47c5b43640 1 -- 172.21.15.40:0/1246649036 shutdown_connections 2023-10-18T08:57:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.396+0000 7f47c5b43640 1 --2- 172.21.15.40:0/1246649036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f47ac06ae70 0x7f47ac06d330 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.396+0000 7f47c5b43640 1 --2- 172.21.15.40:0/1246649036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47c014ee60 0x7f47c01f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.396+0000 7f47c5b43640 1 --2- 172.21.15.40:0/1246649036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47c01f6040 0x7f47c00cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:15.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.396+0000 7f47c5b43640 1 --2- 172.21.15.40:0/1246649036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47a80047c0 0x7f47c00cd020 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:15.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.396+0000 7f47c5b43640 1 -- 172.21.15.40:0/1246649036 >> 172.21.15.40:0/1246649036 conn(0x7f47c006ca10 msgr2=0x7f47c0070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:15.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.397+0000 7f47c5b43640 1 -- 172.21.15.40:0/1246649036 shutdown_connections 2023-10-18T08:57:15.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.398+0000 7f47c5b43640 1 -- 172.21.15.40:0/1246649036 wait complete. 2023-10-18T08:57:15.428 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:57:15.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.623+0000 7f43c17ed640 1 -- 172.21.15.40:0/3761807288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43bc152560 msgr2=0x7f43bc070f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:15.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.623+0000 7f43c17ed640 1 --2- 172.21.15.40:0/3761807288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43bc152560 0x7f43bc070f00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f43ac005fd0 tx=0x7f43ac02beb0 comp rx=0 tx=0).stop 2023-10-18T08:57:15.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.623+0000 7f43c17ed640 1 -- 172.21.15.40:0/3761807288 shutdown_connections 2023-10-18T08:57:15.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.623+0000 7f43c17ed640 1 --2- 172.21.15.40:0/3761807288 >> v2:172.21.15.144:3300/0 conn(0x7f43bc071440 0x7f43bc0aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:15.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.623+0000 7f43c17ed640 1 --2- 172.21.15.40:0/3761807288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43bc152560 0x7f43bc070f00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:15.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.623+0000 7f43c17ed640 1 -- 172.21.15.40:0/3761807288 >> 172.21.15.40:0/3761807288 conn(0x7f43bc069720 msgr2=0x7f43bc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:15.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.623+0000 7f43c17ed640 1 -- 172.21.15.40:0/3761807288 shutdown_connections 2023-10-18T08:57:15.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.623+0000 7f43c17ed640 1 -- 172.21.15.40:0/3761807288 wait complete. 2023-10-18T08:57:15.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.625+0000 7f43c17ed640 1 Processor -- start 2023-10-18T08:57:15.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.625+0000 7f43c17ed640 1 -- start start 2023-10-18T08:57:15.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.626+0000 7f43c17ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43bc071440 0x7f43bc155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:15.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.626+0000 7f43c17ed640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43bc0ab0d0 0x7f43bc14f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:15.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.626+0000 7f43c17ed640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43bc152560 0x7f43bc148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:15.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.626+0000 7f43c17ed640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc153290 con 0x7f43bc071440 2023-10-18T08:57:15.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.626+0000 7f43c17ed640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc14ea60 con 0x7f43bc152560 2023-10-18T08:57:15.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.626+0000 7f43c17ed640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc15a880 con 0x7f43bc0ab0d0 2023-10-18T08:57:15.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.626+0000 7f43bbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43bc071440 0x7f43bc155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:15.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.626+0000 7f43bbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43bc071440 0x7f43bc155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53988/0 (socket says 172.21.15.40:53988) 2023-10-18T08:57:15.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.626+0000 7f43bbfff640 1 -- 172.21.15.40:0/1327248371 learned_addr learned my addr 172.21.15.40:0/1327248371 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:15.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.627+0000 7f43b3fff640 1 --2- 172.21.15.40:0/1327248371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43bc0ab0d0 0x7f43bc14f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:15.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.627+0000 7f43c0fec640 1 --2- 172.21.15.40:0/1327248371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43bc152560 0x7f43bc148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:15.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.627+0000 7f43bbfff640 1 -- 172.21.15.40:0/1327248371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43bc0ab0d0 msgr2=0x7f43bc14f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:15.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.627+0000 7f43bbfff640 1 --2- 172.21.15.40:0/1327248371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43bc0ab0d0 0x7f43bc14f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:15.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.627+0000 7f43bbfff640 1 -- 172.21.15.40:0/1327248371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43bc152560 msgr2=0x7f43bc148070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:15.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.627+0000 7f43bbfff640 1 --2- 172.21.15.40:0/1327248371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43bc152560 0x7f43bc148070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:15.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.627+0000 7f43bbfff640 1 -- 172.21.15.40:0/1327248371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac009560 con 0x7f43bc071440 2023-10-18T08:57:15.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.627+0000 7f43c0fec640 1 --2- 172.21.15.40:0/1327248371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43bc152560 0x7f43bc148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:15.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.627+0000 7f43bbfff640 1 --2- 172.21.15.40:0/1327248371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43bc071440 0x7f43bc155230 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0099f0 tx=0x7f43ac0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:15.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.627+0000 7f43b97fa640 1 -- 172.21.15.40:0/1327248371 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac003ea0 con 0x7f43bc071440 2023-10-18T08:57:15.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.628+0000 7f43b97fa640 1 -- 172.21.15.40:0/1327248371 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f43ac03d440 con 0x7f43bc071440 2023-10-18T08:57:15.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.628+0000 7f43b97fa640 1 -- 172.21.15.40:0/1327248371 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac03d850 con 0x7f43bc071440 2023-10-18T08:57:15.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.628+0000 7f43c17ed640 1 -- 172.21.15.40:0/1327248371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc15aff0 con 0x7f43bc071440 2023-10-18T08:57:15.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.628+0000 7f43c17ed640 1 -- 172.21.15.40:0/1327248371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc14ef90 con 0x7f43bc071440 2023-10-18T08:57:15.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.629+0000 7f43c17ed640 1 -- 172.21.15.40:0/1327248371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43bc072100 con 0x7f43bc071440 2023-10-18T08:57:15.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.630+0000 7f43b97fa640 1 -- 172.21.15.40:0/1327248371 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f43ac03d9b0 con 0x7f43bc071440 2023-10-18T08:57:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.631+0000 7f43b97fa640 1 --2- 172.21.15.40:0/1327248371 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f439c06ae40 0x7f439c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.632+0000 7f43b97fa640 1 -- 172.21.15.40:0/1327248371 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43ac0abfd0 con 0x7f43bc071440 2023-10-18T08:57:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.632+0000 7f43b3fff640 1 --2- 172.21.15.40:0/1327248371 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f439c06ae40 0x7f439c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:15.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.633+0000 7f43b3fff640 1 --2- 172.21.15.40:0/1327248371 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f439c06ae40 0x7f439c06d300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0b2030 tx=0x7f43a4005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:15.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.636+0000 7f43b97fa640 1 -- 172.21.15.40:0/1327248371 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f43ac077070 con 0x7f43bc071440 2023-10-18T08:57:15.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.830+0000 7f43c17ed640 1 -- 172.21.15.40:0/1327248371 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43bc066e00 con 0x7f43bc071440 2023-10-18T08:57:15.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.833+0000 7f43b97fa640 1 -- 172.21.15.40:0/1327248371 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16538 (secure 0 0 0) 0x7f43ac07a690 con 0x7f43bc071440 2023-10-18T08:57:15.836 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:15.836 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":45,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:57:14.825201+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:57:12.765969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T08:57:13.851110+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:57:15.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.835+0000 7f43c17ed640 1 -- 172.21.15.40:0/1327248371 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f439c06ae40 msgr2=0x7f439c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:15.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.835+0000 7f43c17ed640 1 --2- 172.21.15.40:0/1327248371 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f439c06ae40 0x7f439c06d300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f43bc0b2030 tx=0x7f43a4005f90 comp rx=0 tx=0).stop 2023-10-18T08:57:15.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.835+0000 7f43c17ed640 1 -- 172.21.15.40:0/1327248371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43bc071440 msgr2=0x7f43bc155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:15.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.835+0000 7f43c17ed640 1 --2- 172.21.15.40:0/1327248371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43bc071440 0x7f43bc155230 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0099f0 tx=0x7f43ac0038a0 comp rx=0 tx=0).stop 2023-10-18T08:57:15.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.836+0000 7f43c17ed640 1 -- 172.21.15.40:0/1327248371 shutdown_connections 2023-10-18T08:57:15.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.836+0000 7f43c17ed640 1 --2- 172.21.15.40:0/1327248371 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f439c06ae40 0x7f439c06d300 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:15.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.836+0000 7f43c17ed640 1 --2- 172.21.15.40:0/1327248371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43bc152560 0x7f43bc148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:15.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.836+0000 7f43c17ed640 1 --2- 172.21.15.40:0/1327248371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43bc0ab0d0 0x7f43bc14f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:15.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.836+0000 7f43c17ed640 1 --2- 172.21.15.40:0/1327248371 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43bc071440 0x7f43bc155230 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:15.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.836+0000 7f43c17ed640 1 -- 172.21.15.40:0/1327248371 >> 172.21.15.40:0/1327248371 conn(0x7f43bc069720 msgr2=0x7f43bc15ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:15.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.836+0000 7f43c17ed640 1 -- 172.21.15.40:0/1327248371 shutdown_connections 2023-10-18T08:57:15.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:15.836+0000 7f43c17ed640 1 -- 172.21.15.40:0/1327248371 wait complete. 2023-10-18T08:57:15.858 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:57:16.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.087+0000 7f9902e47640 1 -- 172.21.15.40:0/1530669169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98fc152560 msgr2=0x7f98fc070f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:16.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.087+0000 7f9902e47640 1 --2- 172.21.15.40:0/1530669169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98fc152560 0x7f98fc070f00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009830 tx=0x7f98f802be80 comp rx=0 tx=0).stop 2023-10-18T08:57:16.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.088+0000 7f9902e47640 1 -- 172.21.15.40:0/1530669169 shutdown_connections 2023-10-18T08:57:16.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.088+0000 7f9902e47640 1 --2- 172.21.15.40:0/1530669169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98fc152560 0x7f98fc070f00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:16.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.088+0000 7f9902e47640 1 -- 172.21.15.40:0/1530669169 >> 172.21.15.40:0/1530669169 conn(0x7f98fc069720 msgr2=0x7f98fc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:16.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.088+0000 7f9902e47640 1 -- 172.21.15.40:0/1530669169 shutdown_connections 2023-10-18T08:57:16.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.088+0000 7f9902e47640 1 -- 172.21.15.40:0/1530669169 wait complete. 2023-10-18T08:57:16.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.089+0000 7f9902e47640 1 Processor -- start 2023-10-18T08:57:16.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.090+0000 7f9902e47640 1 -- start start 2023-10-18T08:57:16.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.090+0000 7f9902e47640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98fc071440 0x7f98fc15dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:16.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.090+0000 7f9902e47640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98fc0ab2a0 0x7f98fc15a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:16.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.091+0000 7f9902e47640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98fc152560 0x7f98fc15ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:16.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.091+0000 7f9902e47640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc150030 con 0x7f98fc152560 2023-10-18T08:57:16.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.091+0000 7f9902e47640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc14f790 con 0x7f98fc071440 2023-10-18T08:57:16.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.091+0000 7f9902e47640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc14fa70 con 0x7f98fc0ab2a0 2023-10-18T08:57:16.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.091+0000 7f9902646640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98fc152560 0x7f98fc15ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:16.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.091+0000 7f9902646640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98fc152560 0x7f98fc15ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54018/0 (socket says 172.21.15.40:54018) 2023-10-18T08:57:16.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.091+0000 7f9902646640 1 -- 172.21.15.40:0/3947552025 learned_addr learned my addr 172.21.15.40:0/3947552025 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:16.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.091+0000 7f9901e45640 1 --2- 172.21.15.40:0/3947552025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98fc071440 0x7f98fc15dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:16.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.091+0000 7f9901644640 1 --2- 172.21.15.40:0/3947552025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98fc0ab2a0 0x7f98fc15a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:16.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.091+0000 7f9902646640 1 -- 172.21.15.40:0/3947552025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98fc0ab2a0 msgr2=0x7f98fc15a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:16.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.091+0000 7f9902646640 1 --2- 172.21.15.40:0/3947552025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98fc0ab2a0 0x7f98fc15a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:16.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.091+0000 7f9902646640 1 -- 172.21.15.40:0/3947552025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98fc071440 msgr2=0x7f98fc15dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:16.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.091+0000 7f9902646640 1 --2- 172.21.15.40:0/3947552025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98fc071440 0x7f98fc15dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:16.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.091+0000 7f9902646640 1 -- 172.21.15.40:0/3947552025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f8009490 con 0x7f98fc152560 2023-10-18T08:57:16.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.092+0000 7f9902646640 1 --2- 172.21.15.40:0/3947552025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98fc152560 0x7f98fc15ae00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f98f400c7a0 tx=0x7f98f400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:16.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.092+0000 7f98f27fc640 1 -- 172.21.15.40:0/3947552025 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4003b70 con 0x7f98fc152560 2023-10-18T08:57:16.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.092+0000 7f98f27fc640 1 -- 172.21.15.40:0/3947552025 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f98f4003cf0 con 0x7f98fc152560 2023-10-18T08:57:16.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.092+0000 7f98f27fc640 1 -- 172.21.15.40:0/3947552025 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f400aab0 con 0x7f98fc152560 2023-10-18T08:57:16.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.092+0000 7f9902e47640 1 -- 172.21.15.40:0/3947552025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98fc14fd50 con 0x7f98fc152560 2023-10-18T08:57:16.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.092+0000 7f9902e47640 1 -- 172.21.15.40:0/3947552025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc15e520 con 0x7f98fc152560 2023-10-18T08:57:16.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.094+0000 7f9902e47640 1 -- 172.21.15.40:0/3947552025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98fc1f1d90 con 0x7f98fc152560 2023-10-18T08:57:16.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.094+0000 7f98f27fc640 1 -- 172.21.15.40:0/3947552025 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f98f400ac10 con 0x7f98fc152560 2023-10-18T08:57:16.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.095+0000 7f98f27fc640 1 --2- 172.21.15.40:0/3947552025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f98e006ae40 0x7f98e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:16.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.095+0000 7f98f27fc640 1 -- 172.21.15.40:0/3947552025 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98f4089610 con 0x7f98fc152560 2023-10-18T08:57:16.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.096+0000 7f9901e45640 1 --2- 172.21.15.40:0/3947552025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f98e006ae40 0x7f98e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:16.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.098+0000 7f9901e45640 1 --2- 172.21.15.40:0/3947552025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f98e006ae40 0x7f98e006d300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f98f8036000 tx=0x7f98f8004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:16.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.100+0000 7f98f27fc640 1 -- 172.21.15.40:0/3947552025 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f98f40546c0 con 0x7f98fc152560 2023-10-18T08:57:16.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.293+0000 7f9902e47640 1 -- 172.21.15.40:0/3947552025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98fc15e6e0 con 0x7f98fc152560 2023-10-18T08:57:16.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.295+0000 7f98f27fc640 1 -- 172.21.15.40:0/3947552025 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16538 (secure 0 0 0) 0x7f98f4057ce0 con 0x7f98fc152560 2023-10-18T08:57:16.298 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:16.298 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":45,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:57:14.825201+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:57:12.765969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T08:57:13.851110+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:57:16.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.298+0000 7f9902e47640 1 -- 172.21.15.40:0/3947552025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f98e006ae40 msgr2=0x7f98e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:16.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.298+0000 7f9902e47640 1 --2- 172.21.15.40:0/3947552025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f98e006ae40 0x7f98e006d300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f98f8036000 tx=0x7f98f8004730 comp rx=0 tx=0).stop 2023-10-18T08:57:16.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.298+0000 7f9902e47640 1 -- 172.21.15.40:0/3947552025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98fc152560 msgr2=0x7f98fc15ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:16.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.298+0000 7f9902e47640 1 --2- 172.21.15.40:0/3947552025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98fc152560 0x7f98fc15ae00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f98f400c7a0 tx=0x7f98f400cc60 comp rx=0 tx=0).stop 2023-10-18T08:57:16.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.298+0000 7f9902e47640 1 -- 172.21.15.40:0/3947552025 shutdown_connections 2023-10-18T08:57:16.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.298+0000 7f9902e47640 1 --2- 172.21.15.40:0/3947552025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f98e006ae40 0x7f98e006d300 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:16.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.298+0000 7f9902e47640 1 --2- 172.21.15.40:0/3947552025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98fc152560 0x7f98fc15ae00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:16.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.298+0000 7f9902e47640 1 --2- 172.21.15.40:0/3947552025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98fc0ab2a0 0x7f98fc15a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:16.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.298+0000 7f9902e47640 1 --2- 172.21.15.40:0/3947552025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98fc071440 0x7f98fc15dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:16.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.298+0000 7f9902e47640 1 -- 172.21.15.40:0/3947552025 >> 172.21.15.40:0/3947552025 conn(0x7f98fc069720 msgr2=0x7f98fc06f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:16.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.298+0000 7f9902e47640 1 -- 172.21.15.40:0/3947552025 shutdown_connections 2023-10-18T08:57:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.298+0000 7f9902e47640 1 -- 172.21.15.40:0/3947552025 wait complete. 2023-10-18T08:57:16.321 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T08:57:16.332 INFO:tasks.ceph.mds.b.smithi144.stderr:2023-10-18T08:57:16.330+0000 7f7031efc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T08:57:16.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.564+0000 7f36bf27c640 1 -- 172.21.15.40:0/1022467391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36b806d660 msgr2=0x7f36b80b1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:16.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.564+0000 7f36bf27c640 1 --2- 172.21.15.40:0/1022467391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36b806d660 0x7f36b80b1cc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f36b4005fd0 tx=0x7f36b402c260 comp rx=0 tx=0).stop 2023-10-18T08:57:16.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.564+0000 7f36bf27c640 1 -- 172.21.15.40:0/1022467391 shutdown_connections 2023-10-18T08:57:16.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.564+0000 7f36bf27c640 1 --2- 172.21.15.40:0/1022467391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36b806d660 0x7f36b80b1cc0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:16.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.564+0000 7f36bf27c640 1 -- 172.21.15.40:0/1022467391 >> 172.21.15.40:0/1022467391 conn(0x7f36b8069720 msgr2=0x7f36b8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:16.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.564+0000 7f36bf27c640 1 -- 172.21.15.40:0/1022467391 shutdown_connections 2023-10-18T08:57:16.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.564+0000 7f36bf27c640 1 -- 172.21.15.40:0/1022467391 wait complete. 2023-10-18T08:57:16.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.567+0000 7f36bf27c640 1 Processor -- start 2023-10-18T08:57:16.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.567+0000 7f36bf27c640 1 -- start start 2023-10-18T08:57:16.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.568+0000 7f36bf27c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36b806d660 0x7f36b8154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:16.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.568+0000 7f36bf27c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b80cd140 0x7f36b81539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:16.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.568+0000 7f36bf27c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36b8157800 0x7f36b8070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:16.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.568+0000 7f36bf27c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b80ac070 con 0x7f36b806d660 2023-10-18T08:57:16.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.568+0000 7f36bf27c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b80ac500 con 0x7f36b80cd140 2023-10-18T08:57:16.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.568+0000 7f36bf27c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b8156d30 con 0x7f36b8157800 2023-10-18T08:57:16.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.568+0000 7f36bd018640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36b806d660 0x7f36b8154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:16.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.568+0000 7f36bd018640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36b806d660 0x7f36b8154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54038/0 (socket says 172.21.15.40:54038) 2023-10-18T08:57:16.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.568+0000 7f36bd018640 1 -- 172.21.15.40:0/4121935693 learned_addr learned my addr 172.21.15.40:0/4121935693 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:16.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.569+0000 7f36bc817640 1 --2- 172.21.15.40:0/4121935693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b80cd140 0x7f36b81539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:16.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.569+0000 7f36bd018640 1 -- 172.21.15.40:0/4121935693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36b8157800 msgr2=0x7f36b8070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:16.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.569+0000 7f36bd018640 1 --2- 172.21.15.40:0/4121935693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36b8157800 0x7f36b8070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:16.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.569+0000 7f36bd018640 1 -- 172.21.15.40:0/4121935693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b80cd140 msgr2=0x7f36b81539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.569+0000 7f36bd018640 1 --2- 172.21.15.40:0/4121935693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b80cd140 0x7f36b81539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.569+0000 7f36bd018640 1 -- 172.21.15.40:0/4121935693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b4009900 con 0x7f36b806d660 2023-10-18T08:57:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.569+0000 7f36bc817640 1 --2- 172.21.15.40:0/4121935693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b80cd140 0x7f36b81539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.569+0000 7f36bd018640 1 --2- 172.21.15.40:0/4121935693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36b806d660 0x7f36b8154180 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f36b4006010 tx=0x7f36b4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:16.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.570+0000 7f36adffb640 1 -- 172.21.15.40:0/4121935693 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b4004450 con 0x7f36b806d660 2023-10-18T08:57:16.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.570+0000 7f36adffb640 1 -- 172.21.15.40:0/4121935693 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f36b40046a0 con 0x7f36b806d660 2023-10-18T08:57:16.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.570+0000 7f36adffb640 1 -- 172.21.15.40:0/4121935693 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b403f6a0 con 0x7f36b806d660 2023-10-18T08:57:16.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.570+0000 7f36bf27c640 1 -- 172.21.15.40:0/4121935693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b8157010 con 0x7f36b806d660 2023-10-18T08:57:16.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.570+0000 7f36bf27c640 1 -- 172.21.15.40:0/4121935693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b8153f00 con 0x7f36b806d660 2023-10-18T08:57:16.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.571+0000 7f36bf27c640 1 -- 172.21.15.40:0/4121935693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b80b0d20 con 0x7f36b806d660 2023-10-18T08:57:16.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.573+0000 7f36adffb640 1 -- 172.21.15.40:0/4121935693 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f36b403b040 con 0x7f36b806d660 2023-10-18T08:57:16.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.574+0000 7f36adffb640 1 --2- 172.21.15.40:0/4121935693 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f369c06ae40 0x7f369c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:16.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.574+0000 7f36adffb640 1 -- 172.21.15.40:0/4121935693 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36b40ac5e0 con 0x7f36b806d660 2023-10-18T08:57:16.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.575+0000 7f36bc817640 1 --2- 172.21.15.40:0/4121935693 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f369c06ae40 0x7f369c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:16.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.576+0000 7f36bc817640 1 --2- 172.21.15.40:0/4121935693 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f369c06ae40 0x7f369c06d300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f36b8162eb0 tx=0x7f36a8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.578+0000 7f36adffb640 1 -- 172.21.15.40:0/4121935693 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36b4077710 con 0x7f36b806d660 2023-10-18T08:57:16.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:16.809+0000 7f36bf27c640 1 -- 172.21.15.40:0/4121935693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f36b8066fd0 con 0x7f36b806d660 2023-10-18T08:57:17.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.391+0000 7f36adffb640 1 -- 172.21.15.40:0/4121935693 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f36b407ad30 con 0x7f36b806d660 2023-10-18T08:57:17.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.394+0000 7f36bf27c640 1 -- 172.21.15.40:0/4121935693 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f369c06ae40 msgr2=0x7f369c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:17.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.394+0000 7f36bf27c640 1 --2- 172.21.15.40:0/4121935693 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f369c06ae40 0x7f369c06d300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f36b8162eb0 tx=0x7f36a8009040 comp rx=0 tx=0).stop 2023-10-18T08:57:17.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.394+0000 7f36bf27c640 1 -- 172.21.15.40:0/4121935693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36b806d660 msgr2=0x7f36b8154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:17.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.394+0000 7f36bf27c640 1 --2- 172.21.15.40:0/4121935693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36b806d660 0x7f36b8154180 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f36b4006010 tx=0x7f36b4003c20 comp rx=0 tx=0).stop 2023-10-18T08:57:17.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.395+0000 7f36bf27c640 1 -- 172.21.15.40:0/4121935693 shutdown_connections 2023-10-18T08:57:17.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.395+0000 7f36bf27c640 1 --2- 172.21.15.40:0/4121935693 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f369c06ae40 0x7f369c06d300 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:17.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.395+0000 7f36bf27c640 1 --2- 172.21.15.40:0/4121935693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36b8157800 0x7f36b8070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:17.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.395+0000 7f36bf27c640 1 --2- 172.21.15.40:0/4121935693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b80cd140 0x7f36b81539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.395+0000 7f36bf27c640 1 --2- 172.21.15.40:0/4121935693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36b806d660 0x7f36b8154180 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.395+0000 7f36bf27c640 1 -- 172.21.15.40:0/4121935693 >> 172.21.15.40:0/4121935693 conn(0x7f36b8069720 msgr2=0x7f36b8157d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.395+0000 7f36bf27c640 1 -- 172.21.15.40:0/4121935693 shutdown_connections 2023-10-18T08:57:17.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.395+0000 7f36bf27c640 1 -- 172.21.15.40:0/4121935693 wait complete. 2023-10-18T08:57:17.421 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T08:57:17.422 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T08:57:17.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.655+0000 7fecf4852640 1 -- 172.21.15.40:0/1201509205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf006ea90 msgr2=0x7fecf01441b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:17.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.655+0000 7fecf4852640 1 --2- 172.21.15.40:0/1201509205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf006ea90 0x7fecf01441b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fecd8009830 tx=0x7fecd802bb90 comp rx=0 tx=0).stop 2023-10-18T08:57:17.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.655+0000 7fecf4852640 1 -- 172.21.15.40:0/1201509205 shutdown_connections 2023-10-18T08:57:17.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.655+0000 7fecf4852640 1 --2- 172.21.15.40:0/1201509205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf006ea90 0x7fecf01441b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:17.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.655+0000 7fecf4852640 1 --2- 172.21.15.40:0/1201509205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecf0069720 0x7fecf0069b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:17.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.655+0000 7fecf4852640 1 -- 172.21.15.40:0/1201509205 >> 172.21.15.40:0/1201509205 conn(0x7fecf00afc80 msgr2=0x7fecf00b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:17.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.655+0000 7fecf4852640 1 -- 172.21.15.40:0/1201509205 shutdown_connections 2023-10-18T08:57:17.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.655+0000 7fecf4852640 1 -- 172.21.15.40:0/1201509205 wait complete. 2023-10-18T08:57:17.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.658+0000 7fecf4852640 1 Processor -- start 2023-10-18T08:57:17.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.658+0000 7fecf4852640 1 -- start start 2023-10-18T08:57:17.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.659+0000 7fecf4852640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecf0069720 0x7fecf00abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:17.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.659+0000 7fecf4852640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf006ea90 0x7fecf01f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:17.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.659+0000 7fecf4852640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecf01446f0 0x7fecf00b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:17.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.659+0000 7fecf4852640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf01f55a0 con 0x7fecf0069720 2023-10-18T08:57:17.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.659+0000 7fecf4852640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf01f5200 con 0x7fecf006ea90 2023-10-18T08:57:17.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.659+0000 7fecf4852640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf00b1f90 con 0x7fecf01446f0 2023-10-18T08:57:17.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.659+0000 7fecef7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecf0069720 0x7fecf00abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:17.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.659+0000 7fecef7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecf0069720 0x7fecf00abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54042/0 (socket says 172.21.15.40:54042) 2023-10-18T08:57:17.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.659+0000 7fecef7fe640 1 -- 172.21.15.40:0/351919593 learned_addr learned my addr 172.21.15.40:0/351919593 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:17.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.660+0000 7fecef7fe640 1 -- 172.21.15.40:0/351919593 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecf01446f0 msgr2=0x7fecf00b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:17.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.660+0000 7fecef7fe640 1 --2- 172.21.15.40:0/351919593 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecf01446f0 0x7fecf00b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:17.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.660+0000 7fecef7fe640 1 -- 172.21.15.40:0/351919593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf006ea90 msgr2=0x7fecf01f59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:17.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.660+0000 7feceeffd640 1 --2- 172.21.15.40:0/351919593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf006ea90 0x7fecf01f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:17.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.660+0000 7fecef7fe640 1 --2- 172.21.15.40:0/351919593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf006ea90 0x7fecf01f59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:17.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.660+0000 7fecef7fe640 1 -- 172.21.15.40:0/351919593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece0009840 con 0x7fecf0069720 2023-10-18T08:57:17.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.660+0000 7feceeffd640 1 --2- 172.21.15.40:0/351919593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf006ea90 0x7fecf01f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:17.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.660+0000 7fecef7fe640 1 --2- 172.21.15.40:0/351919593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecf0069720 0x7fecf00abda0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fece0009d50 tx=0x7fece000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:17.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.660+0000 7fece7fff640 1 -- 172.21.15.40:0/351919593 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0003b70 con 0x7fecf0069720 2023-10-18T08:57:17.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.660+0000 7fece7fff640 1 -- 172.21.15.40:0/351919593 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fece0003cd0 con 0x7fecf0069720 2023-10-18T08:57:17.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.660+0000 7fece7fff640 1 -- 172.21.15.40:0/351919593 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0004280 con 0x7fecf0069720 2023-10-18T08:57:17.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.660+0000 7fecf4852640 1 -- 172.21.15.40:0/351919593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd8009490 con 0x7fecf0069720 2023-10-18T08:57:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.661+0000 7fecf4852640 1 -- 172.21.15.40:0/351919593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf00ac2e0 con 0x7fecf0069720 2023-10-18T08:57:17.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.662+0000 7fecf4852640 1 -- 172.21.15.40:0/351919593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf0066a90 con 0x7fecf0069720 2023-10-18T08:57:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.663+0000 7fece7fff640 1 -- 172.21.15.40:0/351919593 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fece00043e0 con 0x7fecf0069720 2023-10-18T08:57:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.664+0000 7fece7fff640 1 --2- 172.21.15.40:0/351919593 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fecd006ae40 0x7fecd006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.665+0000 7fece7fff640 1 -- 172.21.15.40:0/351919593 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fece0014070 con 0x7fecf0069720 2023-10-18T08:57:17.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.665+0000 7feceeffd640 1 --2- 172.21.15.40:0/351919593 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fecd006ae40 0x7fecd006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:17.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.666+0000 7feceeffd640 1 --2- 172.21.15.40:0/351919593 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fecd006ae40 0x7fecd006d300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fecd8036000 tx=0x7fecd8004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:17.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.669+0000 7fece7fff640 1 -- 172.21.15.40:0/351919593 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fece0055660 con 0x7fecf0069720 2023-10-18T08:57:17.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:17.901+0000 7fecf4852640 1 -- 172.21.15.40:0/351919593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fecf0073570 con 0x7fecf0069720 2023-10-18T08:57:18.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.396+0000 7fece7fff640 1 -- 172.21.15.40:0/351919593 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fece0003e60 con 0x7fecf0069720 2023-10-18T08:57:18.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.399+0000 7fecf4852640 1 -- 172.21.15.40:0/351919593 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fecd006ae40 msgr2=0x7fecd006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:18.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.399+0000 7fecf4852640 1 --2- 172.21.15.40:0/351919593 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fecd006ae40 0x7fecd006d300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fecd8036000 tx=0x7fecd8004730 comp rx=0 tx=0).stop 2023-10-18T08:57:18.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.399+0000 7fecf4852640 1 -- 172.21.15.40:0/351919593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecf0069720 msgr2=0x7fecf00abda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:18.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.399+0000 7fecf4852640 1 --2- 172.21.15.40:0/351919593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecf0069720 0x7fecf00abda0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fece0009d50 tx=0x7fece000c6a0 comp rx=0 tx=0).stop 2023-10-18T08:57:18.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.399+0000 7fecf4852640 1 -- 172.21.15.40:0/351919593 shutdown_connections 2023-10-18T08:57:18.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.399+0000 7fecf4852640 1 --2- 172.21.15.40:0/351919593 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fecd006ae40 0x7fecd006d300 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:18.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.400+0000 7fecf4852640 1 --2- 172.21.15.40:0/351919593 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecf01446f0 0x7fecf00b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:18.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.400+0000 7fecf4852640 1 --2- 172.21.15.40:0/351919593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf006ea90 0x7fecf01f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:18.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.400+0000 7fecf4852640 1 --2- 172.21.15.40:0/351919593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecf0069720 0x7fecf00abda0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:18.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.400+0000 7fecf4852640 1 -- 172.21.15.40:0/351919593 >> 172.21.15.40:0/351919593 conn(0x7fecf00afc80 msgr2=0x7fecf015b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:18.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.400+0000 7fecf4852640 1 -- 172.21.15.40:0/351919593 shutdown_connections 2023-10-18T08:57:18.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.400+0000 7fecf4852640 1 -- 172.21.15.40:0/351919593 wait complete. 2023-10-18T08:57:18.422 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T08:57:18.422 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:57:18.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.649+0000 7f9b150b4640 1 -- 172.21.15.40:0/3882844340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b100ab0d0 msgr2=0x7f9b100cb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:18.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.649+0000 7f9b150b4640 1 --2- 172.21.15.40:0/3882844340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b100ab0d0 0x7f9b100cb3d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9b08005fd0 tx=0x7f9b0802c170 comp rx=0 tx=0).stop 2023-10-18T08:57:18.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.649+0000 7f9b150b4640 1 -- 172.21.15.40:0/3882844340 shutdown_connections 2023-10-18T08:57:18.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.649+0000 7f9b150b4640 1 --2- 172.21.15.40:0/3882844340 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b100ab0d0 0x7f9b100cb3d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:18.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.649+0000 7f9b150b4640 1 --2- 172.21.15.40:0/3882844340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b10071610 0x7f9b100aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:18.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.649+0000 7f9b150b4640 1 -- 172.21.15.40:0/3882844340 >> 172.21.15.40:0/3882844340 conn(0x7f9b10069720 msgr2=0x7f9b10069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:18.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.649+0000 7f9b150b4640 1 -- 172.21.15.40:0/3882844340 shutdown_connections 2023-10-18T08:57:18.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.649+0000 7f9b150b4640 1 -- 172.21.15.40:0/3882844340 wait complete. 2023-10-18T08:57:18.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.651+0000 7f9b150b4640 1 Processor -- start 2023-10-18T08:57:18.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.651+0000 7f9b150b4640 1 -- start start 2023-10-18T08:57:18.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.652+0000 7f9b150b4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b10071610 0x7f9b1015ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:18.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.652+0000 7f9b150b4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b100ab0d0 0x7f9b101560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:18.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.652+0000 7f9b150b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b10152560 0x7f9b1013f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:18.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.652+0000 7f9b150b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b10151250 con 0x7f9b10152560 2023-10-18T08:57:18.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.652+0000 7f9b150b4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b10153670 con 0x7f9b10071610 2023-10-18T08:57:18.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.652+0000 7f9b150b4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b10150a10 con 0x7f9b100ab0d0 2023-10-18T08:57:18.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.652+0000 7f9b148b3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b10152560 0x7f9b1013f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:18.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.652+0000 7f9b148b3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b10152560 0x7f9b1013f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54068/0 (socket says 172.21.15.40:54068) 2023-10-18T08:57:18.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.652+0000 7f9b148b3640 1 -- 172.21.15.40:0/1007547762 learned_addr learned my addr 172.21.15.40:0/1007547762 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:18.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.652+0000 7f9b0ffff640 1 --2- 172.21.15.40:0/1007547762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b10071610 0x7f9b1015ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:18.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.652+0000 7f9b0f7fe640 1 --2- 172.21.15.40:0/1007547762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b100ab0d0 0x7f9b101560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:18.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.653+0000 7f9b148b3640 1 -- 172.21.15.40:0/1007547762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b100ab0d0 msgr2=0x7f9b101560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:18.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.653+0000 7f9b148b3640 1 --2- 172.21.15.40:0/1007547762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b100ab0d0 0x7f9b101560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:18.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.653+0000 7f9b148b3640 1 -- 172.21.15.40:0/1007547762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b10071610 msgr2=0x7f9b1015ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:18.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.653+0000 7f9b148b3640 1 --2- 172.21.15.40:0/1007547762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b10071610 0x7f9b1015ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:18.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.653+0000 7f9b148b3640 1 -- 172.21.15.40:0/1007547762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b08009840 con 0x7f9b10152560 2023-10-18T08:57:18.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.653+0000 7f9b148b3640 1 --2- 172.21.15.40:0/1007547762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b10152560 0x7f9b1013f310 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9b08036040 tx=0x7f9b08004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:18.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.653+0000 7f9b0cff9640 1 -- 172.21.15.40:0/1007547762 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08004670 con 0x7f9b10152560 2023-10-18T08:57:18.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.653+0000 7f9b0cff9640 1 -- 172.21.15.40:0/1007547762 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9b080345a0 con 0x7f9b10152560 2023-10-18T08:57:18.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.653+0000 7f9b0cff9640 1 -- 172.21.15.40:0/1007547762 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b080349b0 con 0x7f9b10152560 2023-10-18T08:57:18.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.653+0000 7f9b150b4640 1 -- 172.21.15.40:0/1007547762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b10150c90 con 0x7f9b10152560 2023-10-18T08:57:18.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.653+0000 7f9b150b4640 1 -- 172.21.15.40:0/1007547762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b10153290 con 0x7f9b10152560 2023-10-18T08:57:18.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.655+0000 7f9b150b4640 1 -- 172.21.15.40:0/1007547762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b101530f0 con 0x7f9b10152560 2023-10-18T08:57:18.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.656+0000 7f9b0cff9640 1 -- 172.21.15.40:0/1007547762 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9b08034b10 con 0x7f9b10152560 2023-10-18T08:57:18.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.657+0000 7f9b0cff9640 1 --2- 172.21.15.40:0/1007547762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9afc06ae40 0x7f9afc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:18.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.657+0000 7f9b0cff9640 1 -- 172.21.15.40:0/1007547762 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b080ac8b0 con 0x7f9b10152560 2023-10-18T08:57:18.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.658+0000 7f9b0ffff640 1 --2- 172.21.15.40:0/1007547762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9afc06ae40 0x7f9afc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:18.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.659+0000 7f9b0ffff640 1 --2- 172.21.15.40:0/1007547762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9afc06ae40 0x7f9afc06d300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9b000011e0 tx=0x7f9b00009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:18.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.661+0000 7f9b0cff9640 1 -- 172.21.15.40:0/1007547762 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9b101530f0 con 0x7f9b10152560 2023-10-18T08:57:18.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.896+0000 7f9b150b4640 1 -- 172.21.15.40:0/1007547762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b101537b0 con 0x7f9b10152560 2023-10-18T08:57:18.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.897+0000 7f9b0cff9640 1 -- 172.21.15.40:0/1007547762 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f9b101537b0 con 0x7f9b10152560 2023-10-18T08:57:18.900 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:18.900 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3668356540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3668356540},{"type":"v1","addr":"172.21.15.144:6836","nonce":3668356540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4222679844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4222679844},{"type":"v1","addr":"172.21.15.40:6834","nonce":4222679844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2829399368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2829399368},{"type":"v1","addr":"172.21.15.40:6835","nonce":2829399368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:57:14.826066+0000","modified":"2023-10-18T08:57:17.903233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1438456692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1438456692},{"type":"v1","addr":"172.21.15.144:6837","nonce":1438456692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-18T08:57:18.900 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 24 2023-10-18T08:57:18.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.900+0000 7f9b150b4640 1 -- 172.21.15.40:0/1007547762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9afc06ae40 msgr2=0x7f9afc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:18.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.900+0000 7f9b150b4640 1 --2- 172.21.15.40:0/1007547762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9afc06ae40 0x7f9afc06d300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9b000011e0 tx=0x7f9b00009040 comp rx=0 tx=0).stop 2023-10-18T08:57:18.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.900+0000 7f9b150b4640 1 -- 172.21.15.40:0/1007547762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b10152560 msgr2=0x7f9b1013f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:18.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.900+0000 7f9b150b4640 1 --2- 172.21.15.40:0/1007547762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b10152560 0x7f9b1013f310 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9b08036040 tx=0x7f9b08004270 comp rx=0 tx=0).stop 2023-10-18T08:57:18.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.900+0000 7f9b150b4640 1 -- 172.21.15.40:0/1007547762 shutdown_connections 2023-10-18T08:57:18.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.900+0000 7f9b150b4640 1 --2- 172.21.15.40:0/1007547762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9afc06ae40 0x7f9afc06d300 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:18.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.900+0000 7f9b150b4640 1 --2- 172.21.15.40:0/1007547762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b10152560 0x7f9b1013f310 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:18.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.900+0000 7f9b150b4640 1 --2- 172.21.15.40:0/1007547762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b100ab0d0 0x7f9b101560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:18.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.900+0000 7f9b150b4640 1 --2- 172.21.15.40:0/1007547762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b10071610 0x7f9b1015ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:18.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.900+0000 7f9b150b4640 1 -- 172.21.15.40:0/1007547762 >> 172.21.15.40:0/1007547762 conn(0x7f9b10069720 msgr2=0x7f9b1014a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:18.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.901+0000 7f9b150b4640 1 -- 172.21.15.40:0/1007547762 shutdown_connections 2023-10-18T08:57:18.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:18.901+0000 7f9b150b4640 1 -- 172.21.15.40:0/1007547762 wait complete. 2023-10-18T08:57:18.924 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:57:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.154+0000 7fe5d1947640 1 -- 172.21.15.40:0/353968278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5cc0ca160 msgr2=0x7fe5cc0cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:19.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.154+0000 7fe5d1947640 1 --2- 172.21.15.40:0/353968278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5cc0ca160 0x7fe5cc0cafc0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4005fd0 tx=0x7fe5c402bd90 comp rx=0 tx=0).stop 2023-10-18T08:57:19.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.154+0000 7fe5d1947640 1 -- 172.21.15.40:0/353968278 shutdown_connections 2023-10-18T08:57:19.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.154+0000 7fe5d1947640 1 --2- 172.21.15.40:0/353968278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5cc0ca160 0x7fe5cc0cafc0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.154+0000 7fe5d1947640 1 --2- 172.21.15.40:0/353968278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5cc1420f0 0x7fe5cc0b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.154+0000 7fe5d1947640 1 -- 172.21.15.40:0/353968278 >> 172.21.15.40:0/353968278 conn(0x7fe5cc069720 msgr2=0x7fe5cc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.154+0000 7fe5d1947640 1 -- 172.21.15.40:0/353968278 shutdown_connections 2023-10-18T08:57:19.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.154+0000 7fe5d1947640 1 -- 172.21.15.40:0/353968278 wait complete. 2023-10-18T08:57:19.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.156+0000 7fe5d1947640 1 Processor -- start 2023-10-18T08:57:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.156+0000 7fe5d1947640 1 -- start start 2023-10-18T08:57:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.157+0000 7fe5d1947640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5cc0ca160 0x7fe5cc15c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.157+0000 7fe5d1947640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc1420f0 0x7fe5cc13a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.157+0000 7fe5d1947640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5cc1425a0 0x7fe5cc14d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:19.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.157+0000 7fe5d1947640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc15b4a0 con 0x7fe5cc0ca160 2023-10-18T08:57:19.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.157+0000 7fe5d1947640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc15a920 con 0x7fe5cc1420f0 2023-10-18T08:57:19.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.157+0000 7fe5d1947640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc15ac00 con 0x7fe5cc1425a0 2023-10-18T08:57:19.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.157+0000 7fe5d0945640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5cc0ca160 0x7fe5cc15c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:19.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.157+0000 7fe5d0945640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5cc0ca160 0x7fe5cc15c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54100/0 (socket says 172.21.15.40:54100) 2023-10-18T08:57:19.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.157+0000 7fe5d0945640 1 -- 172.21.15.40:0/1980284660 learned_addr learned my addr 172.21.15.40:0/1980284660 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:19.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.158+0000 7fe5d1146640 1 --2- 172.21.15.40:0/1980284660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5cc1425a0 0x7fe5cc14d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:19.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.158+0000 7fe5c3fff640 1 --2- 172.21.15.40:0/1980284660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc1420f0 0x7fe5cc13a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:19.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.158+0000 7fe5d0945640 1 -- 172.21.15.40:0/1980284660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5cc1425a0 msgr2=0x7fe5cc14d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:19.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.158+0000 7fe5d0945640 1 --2- 172.21.15.40:0/1980284660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5cc1425a0 0x7fe5cc14d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.158+0000 7fe5d0945640 1 -- 172.21.15.40:0/1980284660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc1420f0 msgr2=0x7fe5cc13a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:19.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.158+0000 7fe5d0945640 1 --2- 172.21.15.40:0/1980284660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc1420f0 0x7fe5cc13a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.158+0000 7fe5d0945640 1 -- 172.21.15.40:0/1980284660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c4009560 con 0x7fe5cc0ca160 2023-10-18T08:57:19.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.158+0000 7fe5d0945640 1 --2- 172.21.15.40:0/1980284660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5cc0ca160 0x7fe5cc15c6e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800cce0 tx=0x7fe5c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:19.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.158+0000 7fe5c17fa640 1 -- 172.21.15.40:0/1980284660 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8004b20 con 0x7fe5cc0ca160 2023-10-18T08:57:19.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.158+0000 7fe5c17fa640 1 -- 172.21.15.40:0/1980284660 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe5c8004d70 con 0x7fe5cc0ca160 2023-10-18T08:57:19.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.158+0000 7fe5c17fa640 1 -- 172.21.15.40:0/1980284660 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8005280 con 0x7fe5cc0ca160 2023-10-18T08:57:19.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.158+0000 7fe5d1947640 1 -- 172.21.15.40:0/1980284660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5cc15aee0 con 0x7fe5cc0ca160 2023-10-18T08:57:19.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.159+0000 7fe5d1947640 1 -- 172.21.15.40:0/1980284660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5cc15cc20 con 0x7fe5cc0ca160 2023-10-18T08:57:19.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.160+0000 7fe5d1947640 1 -- 172.21.15.40:0/1980284660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5cc066e00 con 0x7fe5cc0ca160 2023-10-18T08:57:19.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.161+0000 7fe5c17fa640 1 -- 172.21.15.40:0/1980284660 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe5c80053e0 con 0x7fe5cc0ca160 2023-10-18T08:57:19.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.161+0000 7fe5c17fa640 1 --2- 172.21.15.40:0/1980284660 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe5b806ae70 0x7fe5b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:19.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.162+0000 7fe5c17fa640 1 -- 172.21.15.40:0/1980284660 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5c8089e80 con 0x7fe5cc0ca160 2023-10-18T08:57:19.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.162+0000 7fe5c3fff640 1 --2- 172.21.15.40:0/1980284660 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe5b806ae70 0x7fe5b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:19.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.163+0000 7fe5c3fff640 1 --2- 172.21.15.40:0/1980284660 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe5b806ae70 0x7fe5b806d330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0c9f10 tx=0x7fe5bc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:19.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.166+0000 7fe5c17fa640 1 -- 172.21.15.40:0/1980284660 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5c8054f30 con 0x7fe5cc0ca160 2023-10-18T08:57:19.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.366+0000 7fe5d1947640 1 -- 172.21.15.40:0/1980284660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5cc06d920 con 0x7fe5cc0ca160 2023-10-18T08:57:19.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.368+0000 7fe5c17fa640 1 -- 172.21.15.40:0/1980284660 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16525 (secure 0 0 0) 0x7fe5c801d070 con 0x7fe5cc0ca160 2023-10-18T08:57:19.371 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:19.371 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":46,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:57:16.366742+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:57:12.765969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T08:57:13.851110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:57:19.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.370+0000 7fe5d1947640 1 -- 172.21.15.40:0/1980284660 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe5b806ae70 msgr2=0x7fe5b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:19.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.370+0000 7fe5d1947640 1 --2- 172.21.15.40:0/1980284660 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe5b806ae70 0x7fe5b806d330 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0c9f10 tx=0x7fe5bc008040 comp rx=0 tx=0).stop 2023-10-18T08:57:19.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.370+0000 7fe5d1947640 1 -- 172.21.15.40:0/1980284660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5cc0ca160 msgr2=0x7fe5cc15c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:19.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.370+0000 7fe5d1947640 1 --2- 172.21.15.40:0/1980284660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5cc0ca160 0x7fe5cc15c6e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800cce0 tx=0x7fe5c800a620 comp rx=0 tx=0).stop 2023-10-18T08:57:19.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.371+0000 7fe5d1947640 1 -- 172.21.15.40:0/1980284660 shutdown_connections 2023-10-18T08:57:19.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.371+0000 7fe5d1947640 1 --2- 172.21.15.40:0/1980284660 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe5b806ae70 0x7fe5b806d330 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.371+0000 7fe5d1947640 1 --2- 172.21.15.40:0/1980284660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5cc1425a0 0x7fe5cc14d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.371+0000 7fe5d1947640 1 --2- 172.21.15.40:0/1980284660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5cc1420f0 0x7fe5cc13a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.371+0000 7fe5d1947640 1 --2- 172.21.15.40:0/1980284660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5cc0ca160 0x7fe5cc15c6e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.371+0000 7fe5d1947640 1 -- 172.21.15.40:0/1980284660 >> 172.21.15.40:0/1980284660 conn(0x7fe5cc069720 msgr2=0x7fe5cc0b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:19.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.371+0000 7fe5d1947640 1 -- 172.21.15.40:0/1980284660 shutdown_connections 2023-10-18T08:57:19.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.371+0000 7fe5d1947640 1 -- 172.21.15.40:0/1980284660 wait complete. 2023-10-18T08:57:19.393 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T08:57:19.394 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:57:19.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.614+0000 7f183ae75640 1 -- 172.21.15.40:0/2389855802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f183415b760 msgr2=0x7f18340cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:19.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.614+0000 7f183ae75640 1 --2- 172.21.15.40:0/2389855802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f183415b760 0x7f18340cfbc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1824009ca0 tx=0x7f182402c2e0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.614+0000 7f183ae75640 1 -- 172.21.15.40:0/2389855802 shutdown_connections 2023-10-18T08:57:19.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.614+0000 7f183ae75640 1 --2- 172.21.15.40:0/2389855802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18340d0100 0x7f18340d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.614+0000 7f183ae75640 1 --2- 172.21.15.40:0/2389855802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f183415b760 0x7f18340cfbc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.614+0000 7f183ae75640 1 -- 172.21.15.40:0/2389855802 >> 172.21.15.40:0/2389855802 conn(0x7f183409a9a0 msgr2=0x7f1834088fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:19.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.614+0000 7f183ae75640 1 -- 172.21.15.40:0/2389855802 shutdown_connections 2023-10-18T08:57:19.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.614+0000 7f183ae75640 1 -- 172.21.15.40:0/2389855802 wait complete. 2023-10-18T08:57:19.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.616+0000 7f183ae75640 1 Processor -- start 2023-10-18T08:57:19.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.616+0000 7f183ae75640 1 -- start start 2023-10-18T08:57:19.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.617+0000 7f183ae75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18340d0100 0x7f183414cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:19.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.617+0000 7f183ae75640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18340d19c0 0x7f18340c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:19.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.617+0000 7f183ae75640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f183415b760 0x7f18340af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:19.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.617+0000 7f183ae75640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18340c8f50 con 0x7f18340d0100 2023-10-18T08:57:19.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.617+0000 7f183ae75640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18340cae90 con 0x7f183415b760 2023-10-18T08:57:19.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.617+0000 7f183ae75640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18340c7840 con 0x7f18340d19c0 2023-10-18T08:57:19.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.618+0000 7f1838c11640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18340d0100 0x7f183414cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:19.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.618+0000 7f1838c11640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18340d0100 0x7f183414cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54108/0 (socket says 172.21.15.40:54108) 2023-10-18T08:57:19.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.618+0000 7f1838c11640 1 -- 172.21.15.40:0/2154062618 learned_addr learned my addr 172.21.15.40:0/2154062618 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:19.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.618+0000 7f1833fff640 1 --2- 172.21.15.40:0/2154062618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18340d19c0 0x7f18340c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:19.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.618+0000 7f1838c11640 1 -- 172.21.15.40:0/2154062618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18340d19c0 msgr2=0x7f18340c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:19.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.618+0000 7f1839412640 1 --2- 172.21.15.40:0/2154062618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f183415b760 0x7f18340af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:19.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.618+0000 7f1838c11640 1 --2- 172.21.15.40:0/2154062618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18340d19c0 0x7f18340c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.618+0000 7f1838c11640 1 -- 172.21.15.40:0/2154062618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f183415b760 msgr2=0x7f18340af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:19.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.618+0000 7f1838c11640 1 --2- 172.21.15.40:0/2154062618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f183415b760 0x7f18340af740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.618+0000 7f1838c11640 1 -- 172.21.15.40:0/2154062618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1824009900 con 0x7f18340d0100 2023-10-18T08:57:19.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.618+0000 7f1833fff640 1 --2- 172.21.15.40:0/2154062618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18340d19c0 0x7f18340c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:19.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.618+0000 7f1838c11640 1 --2- 172.21.15.40:0/2154062618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18340d0100 0x7f183414cb30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1824037040 tx=0x7f1824035b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:19.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.619+0000 7f18317fa640 1 -- 172.21.15.40:0/2154062618 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824004240 con 0x7f18340d0100 2023-10-18T08:57:19.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.619+0000 7f18317fa640 1 -- 172.21.15.40:0/2154062618 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1824004490 con 0x7f18340d0100 2023-10-18T08:57:19.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.619+0000 7f18317fa640 1 -- 172.21.15.40:0/2154062618 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824004840 con 0x7f18340d0100 2023-10-18T08:57:19.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.619+0000 7f183ae75640 1 -- 172.21.15.40:0/2154062618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18340c6990 con 0x7f18340d0100 2023-10-18T08:57:19.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.619+0000 7f183ae75640 1 -- 172.21.15.40:0/2154062618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18340c6f10 con 0x7f18340d0100 2023-10-18T08:57:19.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.620+0000 7f183ae75640 1 -- 172.21.15.40:0/2154062618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18340739c0 con 0x7f18340d0100 2023-10-18T08:57:19.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.621+0000 7f18317fa640 1 -- 172.21.15.40:0/2154062618 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f182403b040 con 0x7f18340d0100 2023-10-18T08:57:19.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.622+0000 7f18317fa640 1 --2- 172.21.15.40:0/2154062618 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f182006ae40 0x7f182006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:19.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.622+0000 7f18317fa640 1 -- 172.21.15.40:0/2154062618 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18240ac340 con 0x7f18340d0100 2023-10-18T08:57:19.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.623+0000 7f1833fff640 1 --2- 172.21.15.40:0/2154062618 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f182006ae40 0x7f182006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:19.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.624+0000 7f1833fff640 1 --2- 172.21.15.40:0/2154062618 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f182006ae40 0x7f182006d300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f18340b0670 tx=0x7f1828005e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:19.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.626+0000 7f18317fa640 1 -- 172.21.15.40:0/2154062618 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1824077400 con 0x7f18340d0100 2023-10-18T08:57:19.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.817+0000 7f183ae75640 1 -- 172.21.15.40:0/2154062618 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f18340d15b0 con 0x7f182006ae40 2023-10-18T08:57:19.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.841+0000 7f18317fa640 1 -- 172.21.15.40:0/2154062618 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+273765 (secure 0 0 0) 0x7f18340d15b0 con 0x7f182006ae40 2023-10-18T08:57:19.847 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:19.849 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:57:19.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.848+0000 7f183ae75640 1 -- 172.21.15.40:0/2154062618 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f182006ae40 msgr2=0x7f182006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:19.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.848+0000 7f183ae75640 1 --2- 172.21.15.40:0/2154062618 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f182006ae40 0x7f182006d300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f18340b0670 tx=0x7f1828005e50 comp rx=0 tx=0).stop 2023-10-18T08:57:19.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.848+0000 7f183ae75640 1 -- 172.21.15.40:0/2154062618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18340d0100 msgr2=0x7f183414cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:19.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.848+0000 7f183ae75640 1 --2- 172.21.15.40:0/2154062618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18340d0100 0x7f183414cb30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1824037040 tx=0x7f1824035b50 comp rx=0 tx=0).stop 2023-10-18T08:57:19.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.848+0000 7f183ae75640 1 -- 172.21.15.40:0/2154062618 shutdown_connections 2023-10-18T08:57:19.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.848+0000 7f183ae75640 1 --2- 172.21.15.40:0/2154062618 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f182006ae40 0x7f182006d300 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.848+0000 7f183ae75640 1 --2- 172.21.15.40:0/2154062618 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f183415b760 0x7f18340af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.848+0000 7f183ae75640 1 --2- 172.21.15.40:0/2154062618 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18340d19c0 0x7f18340c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.848+0000 7f183ae75640 1 --2- 172.21.15.40:0/2154062618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18340d0100 0x7f183414cb30 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:19.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.848+0000 7f183ae75640 1 -- 172.21.15.40:0/2154062618 >> 172.21.15.40:0/2154062618 conn(0x7f183409a9a0 msgr2=0x7f183415a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:19.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.848+0000 7f183ae75640 1 -- 172.21.15.40:0/2154062618 shutdown_connections 2023-10-18T08:57:19.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:19.848+0000 7f183ae75640 1 -- 172.21.15.40:0/2154062618 wait complete. 2023-10-18T08:57:19.870 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2023-10-18T08:57:18.937983+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":96,"acting":96,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2732,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559767,"statfs":{"total":1600000000,"available":1597201933,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.599581"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.354858+0000","last_change":"2023-10-18T08:57:14.354858+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.354858+0000","last_undegraded":"2023-10-18T08:57:14.354858+0000","last_fullsized":"2023-10-18T08:57:14.354858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.357660+0000","last_change":"2023-10-18T08:57:14.357660+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.357660+0000","last_undegraded":"2023-10-18T08:57:14.357660+0000","last_fullsized":"2023-10-18T08:57:14.357660+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.353317+0000","last_change":"2023-10-18T08:57:14.353317+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.353317+0000","last_undegraded":"2023-10-18T08:57:14.353317+0000","last_fullsized":"2023-10-18T08:57:14.353317+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.362613+0000","last_change":"2023-10-18T08:57:14.362613+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.362613+0000","last_undegraded":"2023-10-18T08:57:14.362613+0000","last_fullsized":"2023-10-18T08:57:14.362613+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.356438+0000","last_change":"2023-10-18T08:57:14.356438+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.356438+0000","last_undegraded":"2023-10-18T08:57:14.356438+0000","last_fullsized":"2023-10-18T08:57:14.356438+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.355530+0000","last_change":"2023-10-18T08:57:14.355530+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.355530+0000","last_undegraded":"2023-10-18T08:57:14.355530+0000","last_fullsized":"2023-10-18T08:57:14.355530+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.371711+0000","last_change":"2023-10-18T08:57:14.371648+0000","last_active":"2023-10-18T08:57:14.371711+0000","last_peered":"2023-10-18T08:57:14.371711+0000","last_clean":"2023-10-18T08:57:14.371711+0000","last_became_active":"2023-10-18T08:57:14.371360+0000","last_became_peered":"2023-10-18T08:57:14.371360+0000","last_unstale":"2023-10-18T08:57:14.371711+0000","last_undegraded":"2023-10-18T08:57:14.371711+0000","last_fullsized":"2023-10-18T08:57:14.371711+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:28.358388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.344547+0000","last_change":"2023-10-18T08:57:13.344547+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.344547+0000","last_undegraded":"2023-10-18T08:57:13.344547+0000","last_fullsized":"2023-10-18T08:57:13.344547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.339855+0000","last_change":"2023-10-18T08:57:13.339855+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.339855+0000","last_undegraded":"2023-10-18T08:57:13.339855+0000","last_fullsized":"2023-10-18T08:57:13.339855+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.373956+0000","last_change":"2023-10-18T08:57:14.373706+0000","last_active":"2023-10-18T08:57:14.373956+0000","last_peered":"2023-10-18T08:57:14.373956+0000","last_clean":"2023-10-18T08:57:14.373956+0000","last_became_active":"2023-10-18T08:57:14.373294+0000","last_became_peered":"2023-10-18T08:57:14.373294+0000","last_unstale":"2023-10-18T08:57:14.373956+0000","last_undegraded":"2023-10-18T08:57:14.373956+0000","last_fullsized":"2023-10-18T08:57:14.373956+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:26:09.061307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.365216+0000","last_change":"2023-10-18T08:57:14.365008+0000","last_active":"2023-10-18T08:57:14.365216+0000","last_peered":"2023-10-18T08:57:14.365216+0000","last_clean":"2023-10-18T08:57:14.365216+0000","last_became_active":"2023-10-18T08:57:14.361849+0000","last_became_peered":"2023-10-18T08:57:14.361849+0000","last_unstale":"2023-10-18T08:57:14.365216+0000","last_undegraded":"2023-10-18T08:57:14.365216+0000","last_fullsized":"2023-10-18T08:57:14.365216+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:59:52.547228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.344538+0000","last_change":"2023-10-18T08:57:13.344538+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.344538+0000","last_undegraded":"2023-10-18T08:57:13.344538+0000","last_fullsized":"2023-10-18T08:57:13.344538+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.341960+0000","last_change":"2023-10-18T08:57:13.341960+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.341960+0000","last_undegraded":"2023-10-18T08:57:13.341960+0000","last_fullsized":"2023-10-18T08:57:13.341960+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.344612+0000","last_change":"2023-10-18T08:57:13.344612+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.344612+0000","last_undegraded":"2023-10-18T08:57:13.344612+0000","last_fullsized":"2023-10-18T08:57:13.344612+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.346128+0000","last_change":"2023-10-18T08:57:13.346128+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.346128+0000","last_undegraded":"2023-10-18T08:57:13.346128+0000","last_fullsized":"2023-10-18T08:57:13.346128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.352399+0000","last_change":"2023-10-18T08:57:14.352399+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.352399+0000","last_undegraded":"2023-10-18T08:57:14.352399+0000","last_fullsized":"2023-10-18T08:57:14.352399+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.344191+0000","last_change":"2023-10-18T08:57:13.344191+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.344191+0000","last_undegraded":"2023-10-18T08:57:13.344191+0000","last_fullsized":"2023-10-18T08:57:13.344191+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.361992+0000","last_change":"2023-10-18T08:57:14.361992+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.361992+0000","last_undegraded":"2023-10-18T08:57:14.361992+0000","last_fullsized":"2023-10-18T08:57:14.361992+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.373105+0000","last_change":"2023-10-18T08:57:14.372079+0000","last_active":"2023-10-18T08:57:14.373105+0000","last_peered":"2023-10-18T08:57:14.373105+0000","last_clean":"2023-10-18T08:57:14.373105+0000","last_became_active":"2023-10-18T08:57:14.371832+0000","last_became_peered":"2023-10-18T08:57:14.371832+0000","last_unstale":"2023-10-18T08:57:14.373105+0000","last_undegraded":"2023-10-18T08:57:14.373105+0000","last_fullsized":"2023-10-18T08:57:14.373105+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:05.351228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.350346+0000","last_change":"2023-10-18T08:57:14.350346+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.350346+0000","last_undegraded":"2023-10-18T08:57:14.350346+0000","last_fullsized":"2023-10-18T08:57:14.350346+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.348782+0000","last_change":"2023-10-18T08:57:13.348782+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.348782+0000","last_undegraded":"2023-10-18T08:57:13.348782+0000","last_fullsized":"2023-10-18T08:57:13.348782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.364976+0000","last_change":"2023-10-18T08:57:14.363761+0000","last_active":"2023-10-18T08:57:14.364976+0000","last_peered":"2023-10-18T08:57:14.364976+0000","last_clean":"2023-10-18T08:57:14.364976+0000","last_became_active":"2023-10-18T08:57:14.363321+0000","last_became_peered":"2023-10-18T08:57:14.363321+0000","last_unstale":"2023-10-18T08:57:14.364976+0000","last_undegraded":"2023-10-18T08:57:14.364976+0000","last_fullsized":"2023-10-18T08:57:14.364976+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:08.465913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.367726+0000","last_change":"2023-10-18T08:57:14.367626+0000","last_active":"2023-10-18T08:57:14.367726+0000","last_peered":"2023-10-18T08:57:14.367726+0000","last_clean":"2023-10-18T08:57:14.367726+0000","last_became_active":"2023-10-18T08:57:14.364913+0000","last_became_peered":"2023-10-18T08:57:14.364913+0000","last_unstale":"2023-10-18T08:57:14.367726+0000","last_undegraded":"2023-10-18T08:57:14.367726+0000","last_fullsized":"2023-10-18T08:57:14.367726+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:35.097485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.354808+0000","last_change":"2023-10-18T08:57:14.354808+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.354808+0000","last_undegraded":"2023-10-18T08:57:14.354808+0000","last_fullsized":"2023-10-18T08:57:14.354808+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.368309+0000","last_change":"2023-10-18T08:57:14.368186+0000","last_active":"2023-10-18T08:57:14.368309+0000","last_peered":"2023-10-18T08:57:14.368309+0000","last_clean":"2023-10-18T08:57:14.368309+0000","last_became_active":"2023-10-18T08:57:14.367105+0000","last_became_peered":"2023-10-18T08:57:14.367105+0000","last_unstale":"2023-10-18T08:57:14.368309+0000","last_undegraded":"2023-10-18T08:57:14.368309+0000","last_fullsized":"2023-10-18T08:57:14.368309+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:08.232425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.352501+0000","last_change":"2023-10-18T08:57:14.352501+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.352501+0000","last_undegraded":"2023-10-18T08:57:14.352501+0000","last_fullsized":"2023-10-18T08:57:14.352501+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.343633+0000","last_change":"2023-10-18T08:57:13.343633+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.343633+0000","last_undegraded":"2023-10-18T08:57:13.343633+0000","last_fullsized":"2023-10-18T08:57:13.343633+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.347605+0000","last_change":"2023-10-18T08:57:13.347605+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.347605+0000","last_undegraded":"2023-10-18T08:57:13.347605+0000","last_fullsized":"2023-10-18T08:57:13.347605+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.345028+0000","last_change":"2023-10-18T08:57:13.345028+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.345028+0000","last_undegraded":"2023-10-18T08:57:13.345028+0000","last_fullsized":"2023-10-18T08:57:13.345028+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.341352+0000","last_change":"2023-10-18T08:57:13.341352+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.341352+0000","last_undegraded":"2023-10-18T08:57:13.341352+0000","last_fullsized":"2023-10-18T08:57:13.341352+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.372035+0000","last_change":"2023-10-18T08:57:14.371953+0000","last_active":"2023-10-18T08:57:14.372035+0000","last_peered":"2023-10-18T08:57:14.372035+0000","last_clean":"2023-10-18T08:57:14.372035+0000","last_became_active":"2023-10-18T08:57:14.371639+0000","last_became_peered":"2023-10-18T08:57:14.371639+0000","last_unstale":"2023-10-18T08:57:14.372035+0000","last_undegraded":"2023-10-18T08:57:14.372035+0000","last_fullsized":"2023-10-18T08:57:14.372035+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:20:37.681848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.354927+0000","last_change":"2023-10-18T08:57:14.354927+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.354927+0000","last_undegraded":"2023-10-18T08:57:14.354927+0000","last_fullsized":"2023-10-18T08:57:14.354927+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.357026+0000","last_change":"2023-10-18T08:57:14.356889+0000","last_active":"2023-10-18T08:57:14.357026+0000","last_peered":"2023-10-18T08:57:14.357026+0000","last_clean":"2023-10-18T08:57:14.357026+0000","last_became_active":"2023-10-18T08:57:14.355993+0000","last_became_peered":"2023-10-18T08:57:14.355993+0000","last_unstale":"2023-10-18T08:57:14.357026+0000","last_undegraded":"2023-10-18T08:57:14.357026+0000","last_fullsized":"2023-10-18T08:57:14.357026+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:07.495221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.341122+0000","last_change":"2023-10-18T08:57:13.341122+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.341122+0000","last_undegraded":"2023-10-18T08:57:13.341122+0000","last_fullsized":"2023-10-18T08:57:13.341122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.344125+0000","last_change":"2023-10-18T08:57:13.344125+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.344125+0000","last_undegraded":"2023-10-18T08:57:13.344125+0000","last_fullsized":"2023-10-18T08:57:13.344125+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.368653+0000","last_change":"2023-10-18T08:57:14.368515+0000","last_active":"2023-10-18T08:57:14.368653+0000","last_peered":"2023-10-18T08:57:14.368653+0000","last_clean":"2023-10-18T08:57:14.368653+0000","last_became_active":"2023-10-18T08:57:14.368173+0000","last_became_peered":"2023-10-18T08:57:14.368173+0000","last_unstale":"2023-10-18T08:57:14.368653+0000","last_undegraded":"2023-10-18T08:57:14.368653+0000","last_fullsized":"2023-10-18T08:57:14.368653+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:08.465913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.366758+0000","last_change":"2023-10-18T08:57:14.366229+0000","last_active":"2023-10-18T08:57:14.366758+0000","last_peered":"2023-10-18T08:57:14.366758+0000","last_clean":"2023-10-18T08:57:14.366758+0000","last_became_active":"2023-10-18T08:57:14.365496+0000","last_became_peered":"2023-10-18T08:57:14.365496+0000","last_unstale":"2023-10-18T08:57:14.366758+0000","last_undegraded":"2023-10-18T08:57:14.366758+0000","last_fullsized":"2023-10-18T08:57:14.366758+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:30.531145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.373512+0000","last_change":"2023-10-18T08:57:14.373352+0000","last_active":"2023-10-18T08:57:14.373512+0000","last_peered":"2023-10-18T08:57:14.373512+0000","last_clean":"2023-10-18T08:57:14.373512+0000","last_became_active":"2023-10-18T08:57:14.373089+0000","last_became_peered":"2023-10-18T08:57:14.373089+0000","last_unstale":"2023-10-18T08:57:14.373512+0000","last_undegraded":"2023-10-18T08:57:14.373512+0000","last_fullsized":"2023-10-18T08:57:14.373512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:08:14.491862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.348920+0000","last_change":"2023-10-18T08:57:14.348920+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.348920+0000","last_undegraded":"2023-10-18T08:57:14.348920+0000","last_fullsized":"2023-10-18T08:57:14.348920+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.339494+0000","last_change":"2023-10-18T08:57:13.339494+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.339494+0000","last_undegraded":"2023-10-18T08:57:13.339494+0000","last_fullsized":"2023-10-18T08:57:13.339494+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.357648+0000","last_change":"2023-10-18T08:57:14.357648+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.357648+0000","last_undegraded":"2023-10-18T08:57:14.357648+0000","last_fullsized":"2023-10-18T08:57:14.357648+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.373937+0000","last_change":"2023-10-18T08:57:14.373541+0000","last_active":"2023-10-18T08:57:14.373937+0000","last_peered":"2023-10-18T08:57:14.373937+0000","last_clean":"2023-10-18T08:57:14.373937+0000","last_became_active":"2023-10-18T08:57:14.372892+0000","last_became_peered":"2023-10-18T08:57:14.372892+0000","last_unstale":"2023-10-18T08:57:14.373937+0000","last_undegraded":"2023-10-18T08:57:14.373937+0000","last_fullsized":"2023-10-18T08:57:14.373937+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:59:52.547228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.354120+0000","last_change":"2023-10-18T08:57:14.354120+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.354120+0000","last_undegraded":"2023-10-18T08:57:14.354120+0000","last_fullsized":"2023-10-18T08:57:14.354120+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.345024+0000","last_change":"2023-10-18T08:57:13.345024+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.345024+0000","last_undegraded":"2023-10-18T08:57:13.345024+0000","last_fullsized":"2023-10-18T08:57:13.345024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.339514+0000","last_change":"2023-10-18T08:57:13.339514+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.339514+0000","last_undegraded":"2023-10-18T08:57:13.339514+0000","last_fullsized":"2023-10-18T08:57:13.339514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.343518+0000","last_change":"2023-10-18T08:57:13.343518+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.343518+0000","last_undegraded":"2023-10-18T08:57:13.343518+0000","last_fullsized":"2023-10-18T08:57:13.343518+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.341302+0000","last_change":"2023-10-18T08:57:13.341302+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.341302+0000","last_undegraded":"2023-10-18T08:57:13.341302+0000","last_fullsized":"2023-10-18T08:57:13.341302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-18T08:57:13.347025+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:57:13.347025+0000","last_peered":"2023-10-18T08:57:13.347025+0000","last_clean":"2023-10-18T08:57:13.347025+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:57:13.347025+0000","last_undegraded":"2023-10-18T08:57:13.347025+0000","last_fullsized":"2023-10-18T08:57:13.347025+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:34.881744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":30,"acting":30,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":780,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194532,"statfs":{"total":200000000,"available":199200929,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.998,"15min":0.98999999999999999},"min":{"1min":0.67800000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3979999999999999,"5min":1.538,"15min":1.538},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.014,"15min":1.002},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":1.4019999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1399999999999999,"15min":1.1519999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.393,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.123,"15min":1.1160000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7849999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.98499999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.218,"15min":1.224},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.589,"5min":1.694,"15min":1.694},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1839999999999999,"15min":1.1859999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.526}]},{"osd":3,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.254,"15min":1.2609999999999999},"min":{"1min":0.91500000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.72,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.379},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3089999999999999,"15min":1.3129999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.8029999999999999,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.1339999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.1859999999999999,"15min":1.1579999999999999},"min":{"1min":0.92800000000000005,"5min":0.746,"15min":0.746},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.292},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.407,"15min":1.411},"min":{"1min":1.0369999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.885,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.4390000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3520000000000001,"15min":1.353},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.615,"5min":1.893,"15min":1.893},"last":1.343},{"interface":"front","average":{"1min":1.389,"5min":1.417,"15min":1.421},"min":{"1min":1.0940000000000001,"5min":0.874,"15min":0.874},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.617}]},{"osd":6,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.464,"15min":1.458},"min":{"1min":1.0169999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.9339999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.516,"15min":1.518},"min":{"1min":1.23,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.978,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.4990000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":192,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195120,"statfs":{"total":200000000,"available":199803381,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0800000000000001,"15min":1.073},"min":{"1min":0.65400000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.5529999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.133,"5min":1.216,"15min":1.23},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.016}]},{"osd":1,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0780000000000001,"15min":1.081},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.3420000000000001,"5min":1.524,"15min":1.524},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1659999999999999,"15min":1.1819999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.474,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":0.85699999999999998}]},{"osd":2,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.1579999999999999,"15min":1.1719999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.3},{"interface":"front","average":{"1min":1.165,"5min":1.1870000000000001,"15min":1.1910000000000001},"min":{"1min":0.82099999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5309999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.139}]},{"osd":3,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2509999999999999,"15min":1.2569999999999999},"min":{"1min":0.84799999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.502,"5min":1.784,"15min":1.784},"last":1.401},{"interface":"front","average":{"1min":1.222,"5min":1.2170000000000001,"15min":1.216},"min":{"1min":1.0329999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.494,"5min":1.5,"15min":1.5},"last":1.2110000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.1950000000000001,"15min":1.169},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.246},{"interface":"front","average":{"1min":1.284,"5min":1.272,"15min":1.27},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6990000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.607}]},{"osd":5,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3560000000000001,"15min":1.3500000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.665,"5min":1.667,"15min":1.667},"last":1.548},{"interface":"front","average":{"1min":1.331,"5min":1.2330000000000001,"15min":1.2170000000000001},"min":{"1min":0.54900000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":1.103}]},{"osd":7,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4019999999999999,"15min":1.3959999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.373},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5249999999999999,"15min":1.53},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8500000000000001,"5min":2.794,"15min":2.794},"last":1.831}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":197,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195114,"statfs":{"total":200000000,"available":199797564,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2010000000000001,"15min":1.1850000000000001},"min":{"1min":0.77800000000000002,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.7,"5min":1.915,"15min":1.915},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.288,"15min":1.292},"min":{"1min":0.746,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7130000000000001,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.0289999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.2030000000000001,"15min":1.2210000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.613,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.76000000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.581,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.94499999999999995}]},{"osd":2,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2689999999999999,"15min":1.268},"min":{"1min":0.77700000000000002,"5min":0.755,"15min":0.755},"max":{"1min":1.7749999999999999,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.47},{"interface":"front","average":{"1min":1.27,"5min":1.29,"15min":1.2929999999999999},"min":{"1min":0.96099999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.7210000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.669}]},{"osd":3,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.232,"15min":1.24},"min":{"1min":0.84799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5640000000000001,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.321,"15min":1.327},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.784,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.627}]},{"osd":4,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.417,"15min":1.4199999999999999},"min":{"1min":0.84899999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8799999999999999,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.792},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.5069999999999999,"15min":1.5369999999999999},"min":{"1min":0.90100000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7929999999999999,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":1.8999999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.3839999999999999,"15min":1.4139999999999999},"min":{"1min":0.873,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6799999999999999,"5min":5.141,"15min":5.141},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.5720000000000001,"15min":1.5920000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.931,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":2.0590000000000002}]},{"osd":7,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.661,"15min":1.677},"min":{"1min":1.038,"5min":1.014,"15min":1.014},"max":{"1min":1.9990000000000001,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":1.956},{"interface":"front","average":{"1min":1.423,"5min":1.508,"15min":1.522},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.907,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":1.7230000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":196,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195115,"statfs":{"total":200000000,"available":199798583,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.105,"15min":1.109},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.1679999999999999,"15min":1.1819999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0659999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.262}]},{"osd":1,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.0980000000000001,"15min":1.095},"min":{"1min":0.76100000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.583,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.167,"5min":1.2310000000000001,"15min":1.2410000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.929,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.1699999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.1779999999999999,"15min":1.1759999999999999},"min":{"1min":0.75600000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.365},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.29,"15min":1.3100000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.601,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.3959999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.292,"15min":1.302},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4159999999999999,"15min":1.413},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.0899999999999999,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.5169999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3120000000000001,"15min":1.3},"min":{"1min":0.84099999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.8080000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.365,"15min":1.3720000000000001},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.4199999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.397,"15min":1.3819999999999999},"min":{"1min":1.169,"5min":0.998,"15min":0.998},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.51,"5min":1.5680000000000001,"15min":1.5780000000000001},"min":{"1min":1.04,"5min":1.04,"15min":1.04},"max":{"1min":1.8939999999999999,"5min":3.6859999999999999,"15min":3.6859999999999999},"last":1.329}]},{"osd":7,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"max":{"1min":2.2759999999999998,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.659},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.5629999999999999,"15min":1.554},"min":{"1min":1.3089999999999999,"5min":1.23,"15min":1.23},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.569}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":203,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195108,"statfs":{"total":200000000,"available":199791297,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":0.82399999999999995,"15min":0.76500000000000001},"min":{"1min":0.52500000000000002,"5min":0.439,"15min":0.439},"max":{"1min":5.5730000000000004,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.365,"5min":0.93999999999999995,"15min":0.86899999999999999},"min":{"1min":0.53700000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.5599999999999996,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":1.0549999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":0.98499999999999999,"15min":0.94399999999999995},"min":{"1min":0.59399999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":6.8099999999999996,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.238,"5min":1.1259999999999999,"15min":1.1080000000000001},"min":{"1min":0.69099999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.466}]},{"osd":2,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.0620000000000001,"15min":1.0169999999999999},"min":{"1min":0.61199999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":7.4580000000000002,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.0880000000000001,"15min":1.048},"min":{"1min":0.56699999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":3.5110000000000001,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":1.613}]},{"osd":4,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":2.3849999999999998,"5min":1.4570000000000001,"15min":1.3029999999999999},"min":{"1min":0.99099999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":18.664999999999999,"5min":18.664999999999999,"15min":18.664999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.1899999999999999,"15min":1.1439999999999999},"min":{"1min":0.83999999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":7.0700000000000003,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":1.244}]},{"osd":5,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.266,"15min":1.214},"min":{"1min":0.872,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":6.9139999999999997,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":1.175},{"interface":"front","average":{"1min":2.306,"5min":1.4690000000000001,"15min":1.329},"min":{"1min":1.0269999999999999,"5min":0.748,"15min":0.748},"max":{"1min":16.390999999999998,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.6799999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.8049999999999999,"5min":1.4750000000000001,"15min":1.4199999999999999},"min":{"1min":1.022,"5min":0.625,"15min":0.625},"max":{"1min":7.7270000000000003,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.7190000000000001,"5min":1.3819999999999999,"15min":1.3260000000000001},"min":{"1min":0.96499999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":7.242,"5min":7.242,"15min":7.242},"last":1.3180000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.643,"5min":1.367,"15min":1.321},"min":{"1min":0.94299999999999995,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.74},{"interface":"front","average":{"1min":2.1309999999999998,"5min":1.54,"15min":1.4419999999999999},"min":{"1min":0.84999999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":12.946,"5min":12.946,"15min":12.946},"last":1.544}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":192,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195120,"statfs":{"total":200000000,"available":199803381,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.92000000000000004,"15min":0.93999999999999995},"min":{"1min":0.433,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.395,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.024,"15min":1.0569999999999999},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3700000000000001,"5min":1.651,"15min":1.651},"last":0.63100000000000001}]},{"osd":1,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":1.04,"15min":1.0600000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3660000000000001,"5min":4.29,"15min":4.29},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.87,"5min":1.0580000000000001,"15min":1.089},"min":{"1min":0.57499999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3009999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.171}]},{"osd":3,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.105,"15min":1.133},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.9139999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.111,"5min":1.4419999999999999,"15min":1.4970000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.609,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":1.546}]},{"osd":4,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.47,"15min":1.502},"min":{"1min":0.90900000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.7,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":1.7450000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.4239999999999999,"15min":1.4530000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.663,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.6080000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.248,"15min":1.272},"min":{"1min":0.79400000000000004,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5389999999999999,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":1.056},{"interface":"front","average":{"1min":1.256,"5min":1.542,"15min":1.5900000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.7569999999999999,"5min":7.657,"15min":7.657},"last":1.863}]},{"osd":6,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.4950000000000001,"15min":1.5389999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7170000000000001,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.698},{"interface":"front","average":{"1min":1.244,"5min":1.4059999999999999,"15min":1.4330000000000001},"min":{"1min":0.94099999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.796,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":1.3759999999999999}]},{"osd":7,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.5449999999999999,"15min":1.5780000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7350000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.4490000000000001,"15min":1.474},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.6899999999999999,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.335}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":196,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195115,"statfs":{"total":200000000,"available":199798583,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.88},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.871,"15min":0.88400000000000001},"min":{"1min":0.54400000000000004,"5min":0.443,"15min":0.443},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.59299999999999997}]},{"osd":2,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.90000000000000002,"15min":0.91900000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.222,"5min":1.859,"15min":1.859},"last":0.754},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.82599999999999996,"15min":0.83299999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.73999999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.98499999999999999,"15min":1.0089999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.024,"15min":1.048},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.7110000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":0.64400000000000002}]},{"osd":4,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.224,"15min":1.236},"min":{"1min":0.71499999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.53,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.1830000000000001,"15min":1.1890000000000001},"min":{"1min":0.85999999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.99,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.0509999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.262,"15min":1.2789999999999999},"min":{"1min":0.82899999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.6299999999999999,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.335,"15min":1.341},"min":{"1min":0.84999999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.8480000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.708}]},{"osd":6,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2529999999999999,"15min":1.2649999999999999},"min":{"1min":0.86199999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.2050000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.095},{"interface":"front","average":{"1min":1.143,"5min":1.228,"15min":1.242},"min":{"1min":0.72999999999999998,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.853,"5min":1.853,"15min":1.853},"last":1.236}]},{"osd":7,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.3160000000000001,"15min":1.333},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.0419999999999998,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.333,"15min":1.3380000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.8169999999999999,"5min":1.994,"15min":1.994},"last":1.8169999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":773,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194539,"statfs":{"total":200000000,"available":199208215,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.92600000000000005,"15min":0.94599999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.25,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":1.079},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.077,"15min":1.083},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4990000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.6319999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.339,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.125,"15min":1.125},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.2450000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.2849999999999999,"15min":1.327},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.294,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.5109999999999999,"15min":1.5429999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.8640000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.3720000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.2909999999999999,"15min":1.3260000000000001},"min":{"1min":0.72899999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.3460000000000001,"5min":4.492,"15min":4.492},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2509999999999999,"15min":1.2470000000000001},"min":{"1min":0.78500000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":1.3029999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.417,"15min":1.4630000000000001},"min":{"1min":0.79000000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.4770000000000001,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3879999999999999,"15min":1.3959999999999999},"min":{"1min":0.81299999999999994,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.758,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.8240000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5980000000000001,"15min":1.6220000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.952,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.5069999999999999,"15min":1.532},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":2.0910000000000002,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.351}]},{"osd":7,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.647,"15min":1.671},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.149,"5min":3.492,"15min":3.492},"last":1.9339999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.6830000000000001,"15min":1.7330000000000001},"min":{"1min":0.96799999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8859999999999999,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.5569999999999999}]}]}],"pool_statfs":[]}} 2023-10-18T08:57:19.876 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:57:20.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.146+0000 7fa2ba6f3640 1 -- 172.21.15.40:0/2909557310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b40cc480 msgr2=0x7fa2b40b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:20.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.146+0000 7fa2ba6f3640 1 --2- 172.21.15.40:0/2909557310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b40cc480 0x7fa2b40b1340 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009830 tx=0x7fa2ac02bde0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.146+0000 7fa2ba6f3640 1 -- 172.21.15.40:0/2909557310 shutdown_connections 2023-10-18T08:57:20.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.146+0000 7fa2ba6f3640 1 --2- 172.21.15.40:0/2909557310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b40cc480 0x7fa2b40b1340 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.146+0000 7fa2ba6f3640 1 --2- 172.21.15.40:0/2909557310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b415b3d0 0x7fa2b40cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.146+0000 7fa2ba6f3640 1 -- 172.21.15.40:0/2909557310 >> 172.21.15.40:0/2909557310 conn(0x7fa2b40afdf0 msgr2=0x7fa2b40b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:20.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.146+0000 7fa2ba6f3640 1 -- 172.21.15.40:0/2909557310 shutdown_connections 2023-10-18T08:57:20.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.146+0000 7fa2ba6f3640 1 -- 172.21.15.40:0/2909557310 wait complete. 2023-10-18T08:57:20.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.148+0000 7fa2ba6f3640 1 Processor -- start 2023-10-18T08:57:20.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.148+0000 7fa2ba6f3640 1 -- start start 2023-10-18T08:57:20.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.149+0000 7fa2ba6f3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b40cc480 0x7fa2b40c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:20.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.149+0000 7fa2ba6f3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b415b3d0 0x7fa2b41e54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:20.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.149+0000 7fa2ba6f3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b4149bb0 0x7fa2b41e5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:20.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.149+0000 7fa2ba6f3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b40c8c80 con 0x7fa2b40cc480 2023-10-18T08:57:20.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.149+0000 7fa2ba6f3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b41e6260 con 0x7fa2b4149bb0 2023-10-18T08:57:20.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.149+0000 7fa2ba6f3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b40c85c0 con 0x7fa2b415b3d0 2023-10-18T08:57:20.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.149+0000 7fa2b96f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b40cc480 0x7fa2b40c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:20.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.150+0000 7fa2b96f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b40cc480 0x7fa2b40c64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54128/0 (socket says 172.21.15.40:54128) 2023-10-18T08:57:20.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.150+0000 7fa2b96f1640 1 -- 172.21.15.40:0/335551897 learned_addr learned my addr 172.21.15.40:0/335551897 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:20.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.150+0000 7fa2b9ef2640 1 --2- 172.21.15.40:0/335551897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b4149bb0 0x7fa2b41e5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:20.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.150+0000 7fa2b8ef0640 1 --2- 172.21.15.40:0/335551897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b415b3d0 0x7fa2b41e54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:20.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.150+0000 7fa2b96f1640 1 -- 172.21.15.40:0/335551897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b415b3d0 msgr2=0x7fa2b41e54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:20.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.150+0000 7fa2b96f1640 1 --2- 172.21.15.40:0/335551897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b415b3d0 0x7fa2b41e54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.150+0000 7fa2b96f1640 1 -- 172.21.15.40:0/335551897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b4149bb0 msgr2=0x7fa2b41e5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:20.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.150+0000 7fa2b96f1640 1 --2- 172.21.15.40:0/335551897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b4149bb0 0x7fa2b41e5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.150+0000 7fa2b96f1640 1 -- 172.21.15.40:0/335551897 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac009490 con 0x7fa2b40cc480 2023-10-18T08:57:20.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.150+0000 7fa2b9ef2640 1 --2- 172.21.15.40:0/335551897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b4149bb0 0x7fa2b41e5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:20.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.150+0000 7fa2b8ef0640 1 --2- 172.21.15.40:0/335551897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b415b3d0 0x7fa2b41e54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:20.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.150+0000 7fa2b96f1640 1 --2- 172.21.15.40:0/335551897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b40cc480 0x7fa2b40c64a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000ea20 tx=0x7fa2b000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:20.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.151+0000 7fa2a9ffb640 1 -- 172.21.15.40:0/335551897 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b000cde0 con 0x7fa2b40cc480 2023-10-18T08:57:20.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.151+0000 7fa2a9ffb640 1 -- 172.21.15.40:0/335551897 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa2b0011560 con 0x7fa2b40cc480 2023-10-18T08:57:20.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.151+0000 7fa2a9ffb640 1 -- 172.21.15.40:0/335551897 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0011950 con 0x7fa2b40cc480 2023-10-18T08:57:20.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.151+0000 7fa2ba6f3640 1 -- 172.21.15.40:0/335551897 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b40c88a0 con 0x7fa2b40cc480 2023-10-18T08:57:20.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.151+0000 7fa2ba6f3640 1 -- 172.21.15.40:0/335551897 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b40c8220 con 0x7fa2b40cc480 2023-10-18T08:57:20.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.152+0000 7fa2ba6f3640 1 -- 172.21.15.40:0/335551897 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b40b1f50 con 0x7fa2b40cc480 2023-10-18T08:57:20.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.153+0000 7fa2a9ffb640 1 -- 172.21.15.40:0/335551897 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa2b0011ab0 con 0x7fa2b40cc480 2023-10-18T08:57:20.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.154+0000 7fa2a9ffb640 1 --2- 172.21.15.40:0/335551897 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa29806ae70 0x7fa29806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:20.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.154+0000 7fa2a9ffb640 1 -- 172.21.15.40:0/335551897 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2b008a360 con 0x7fa2b40cc480 2023-10-18T08:57:20.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.155+0000 7fa2b8ef0640 1 --2- 172.21.15.40:0/335551897 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa29806ae70 0x7fa29806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:20.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.156+0000 7fa2b8ef0640 1 --2- 172.21.15.40:0/335551897 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa29806ae70 0x7fa29806d330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa2b41f9870 tx=0x7fa2a4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:20.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.158+0000 7fa2a9ffb640 1 -- 172.21.15.40:0/335551897 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2b0055370 con 0x7fa2b40cc480 2023-10-18T08:57:20.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.354+0000 7fa2ba6f3640 1 -- 172.21.15.40:0/335551897 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa2b415c850 con 0x7fa29806ae70 2023-10-18T08:57:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.376+0000 7fa2a9ffb640 1 -- 172.21.15.40:0/335551897 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+273765 (secure 0 0 0) 0x7fa2a40013d0 con 0x7fa29806ae70 2023-10-18T08:57:20.382 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:20.384 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:57:20.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.383+0000 7fa2ba6f3640 1 -- 172.21.15.40:0/335551897 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa29806ae70 msgr2=0x7fa29806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:20.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.383+0000 7fa2ba6f3640 1 --2- 172.21.15.40:0/335551897 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa29806ae70 0x7fa29806d330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa2b41f9870 tx=0x7fa2a4009040 comp rx=0 tx=0).stop 2023-10-18T08:57:20.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.383+0000 7fa2ba6f3640 1 -- 172.21.15.40:0/335551897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b40cc480 msgr2=0x7fa2b40c64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:20.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.383+0000 7fa2ba6f3640 1 --2- 172.21.15.40:0/335551897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b40cc480 0x7fa2b40c64a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000ea20 tx=0x7fa2b000eee0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.383+0000 7fa2ba6f3640 1 -- 172.21.15.40:0/335551897 shutdown_connections 2023-10-18T08:57:20.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.383+0000 7fa2ba6f3640 1 --2- 172.21.15.40:0/335551897 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa29806ae70 0x7fa29806d330 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.383+0000 7fa2ba6f3640 1 --2- 172.21.15.40:0/335551897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b4149bb0 0x7fa2b41e5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.383+0000 7fa2ba6f3640 1 --2- 172.21.15.40:0/335551897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b415b3d0 0x7fa2b41e54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.383+0000 7fa2ba6f3640 1 --2- 172.21.15.40:0/335551897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b40cc480 0x7fa2b40c64a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.383+0000 7fa2ba6f3640 1 -- 172.21.15.40:0/335551897 >> 172.21.15.40:0/335551897 conn(0x7fa2b40afdf0 msgr2=0x7fa2b4070600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:20.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.384+0000 7fa2ba6f3640 1 -- 172.21.15.40:0/335551897 shutdown_connections 2023-10-18T08:57:20.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.384+0000 7fa2ba6f3640 1 -- 172.21.15.40:0/335551897 wait complete. 2023-10-18T08:57:20.406 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2023-10-18T08:57:18.937983+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":96,"acting":96,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2732,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559767,"statfs":{"total":1600000000,"available":1597201933,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.599581"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.354858+0000","last_change":"2023-10-18T08:57:14.354858+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.354858+0000","last_undegraded":"2023-10-18T08:57:14.354858+0000","last_fullsized":"2023-10-18T08:57:14.354858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.357660+0000","last_change":"2023-10-18T08:57:14.357660+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.357660+0000","last_undegraded":"2023-10-18T08:57:14.357660+0000","last_fullsized":"2023-10-18T08:57:14.357660+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.353317+0000","last_change":"2023-10-18T08:57:14.353317+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.353317+0000","last_undegraded":"2023-10-18T08:57:14.353317+0000","last_fullsized":"2023-10-18T08:57:14.353317+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.362613+0000","last_change":"2023-10-18T08:57:14.362613+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.362613+0000","last_undegraded":"2023-10-18T08:57:14.362613+0000","last_fullsized":"2023-10-18T08:57:14.362613+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.356438+0000","last_change":"2023-10-18T08:57:14.356438+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.356438+0000","last_undegraded":"2023-10-18T08:57:14.356438+0000","last_fullsized":"2023-10-18T08:57:14.356438+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.355530+0000","last_change":"2023-10-18T08:57:14.355530+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.355530+0000","last_undegraded":"2023-10-18T08:57:14.355530+0000","last_fullsized":"2023-10-18T08:57:14.355530+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.371711+0000","last_change":"2023-10-18T08:57:14.371648+0000","last_active":"2023-10-18T08:57:14.371711+0000","last_peered":"2023-10-18T08:57:14.371711+0000","last_clean":"2023-10-18T08:57:14.371711+0000","last_became_active":"2023-10-18T08:57:14.371360+0000","last_became_peered":"2023-10-18T08:57:14.371360+0000","last_unstale":"2023-10-18T08:57:14.371711+0000","last_undegraded":"2023-10-18T08:57:14.371711+0000","last_fullsized":"2023-10-18T08:57:14.371711+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:28.358388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.344547+0000","last_change":"2023-10-18T08:57:13.344547+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.344547+0000","last_undegraded":"2023-10-18T08:57:13.344547+0000","last_fullsized":"2023-10-18T08:57:13.344547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.339855+0000","last_change":"2023-10-18T08:57:13.339855+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.339855+0000","last_undegraded":"2023-10-18T08:57:13.339855+0000","last_fullsized":"2023-10-18T08:57:13.339855+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.373956+0000","last_change":"2023-10-18T08:57:14.373706+0000","last_active":"2023-10-18T08:57:14.373956+0000","last_peered":"2023-10-18T08:57:14.373956+0000","last_clean":"2023-10-18T08:57:14.373956+0000","last_became_active":"2023-10-18T08:57:14.373294+0000","last_became_peered":"2023-10-18T08:57:14.373294+0000","last_unstale":"2023-10-18T08:57:14.373956+0000","last_undegraded":"2023-10-18T08:57:14.373956+0000","last_fullsized":"2023-10-18T08:57:14.373956+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:26:09.061307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.365216+0000","last_change":"2023-10-18T08:57:14.365008+0000","last_active":"2023-10-18T08:57:14.365216+0000","last_peered":"2023-10-18T08:57:14.365216+0000","last_clean":"2023-10-18T08:57:14.365216+0000","last_became_active":"2023-10-18T08:57:14.361849+0000","last_became_peered":"2023-10-18T08:57:14.361849+0000","last_unstale":"2023-10-18T08:57:14.365216+0000","last_undegraded":"2023-10-18T08:57:14.365216+0000","last_fullsized":"2023-10-18T08:57:14.365216+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:59:52.547228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.344538+0000","last_change":"2023-10-18T08:57:13.344538+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.344538+0000","last_undegraded":"2023-10-18T08:57:13.344538+0000","last_fullsized":"2023-10-18T08:57:13.344538+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.341960+0000","last_change":"2023-10-18T08:57:13.341960+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.341960+0000","last_undegraded":"2023-10-18T08:57:13.341960+0000","last_fullsized":"2023-10-18T08:57:13.341960+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.344612+0000","last_change":"2023-10-18T08:57:13.344612+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.344612+0000","last_undegraded":"2023-10-18T08:57:13.344612+0000","last_fullsized":"2023-10-18T08:57:13.344612+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.346128+0000","last_change":"2023-10-18T08:57:13.346128+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.346128+0000","last_undegraded":"2023-10-18T08:57:13.346128+0000","last_fullsized":"2023-10-18T08:57:13.346128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.352399+0000","last_change":"2023-10-18T08:57:14.352399+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.352399+0000","last_undegraded":"2023-10-18T08:57:14.352399+0000","last_fullsized":"2023-10-18T08:57:14.352399+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.344191+0000","last_change":"2023-10-18T08:57:13.344191+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.344191+0000","last_undegraded":"2023-10-18T08:57:13.344191+0000","last_fullsized":"2023-10-18T08:57:13.344191+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.361992+0000","last_change":"2023-10-18T08:57:14.361992+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.361992+0000","last_undegraded":"2023-10-18T08:57:14.361992+0000","last_fullsized":"2023-10-18T08:57:14.361992+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.373105+0000","last_change":"2023-10-18T08:57:14.372079+0000","last_active":"2023-10-18T08:57:14.373105+0000","last_peered":"2023-10-18T08:57:14.373105+0000","last_clean":"2023-10-18T08:57:14.373105+0000","last_became_active":"2023-10-18T08:57:14.371832+0000","last_became_peered":"2023-10-18T08:57:14.371832+0000","last_unstale":"2023-10-18T08:57:14.373105+0000","last_undegraded":"2023-10-18T08:57:14.373105+0000","last_fullsized":"2023-10-18T08:57:14.373105+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:05.351228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.350346+0000","last_change":"2023-10-18T08:57:14.350346+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.350346+0000","last_undegraded":"2023-10-18T08:57:14.350346+0000","last_fullsized":"2023-10-18T08:57:14.350346+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.348782+0000","last_change":"2023-10-18T08:57:13.348782+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.348782+0000","last_undegraded":"2023-10-18T08:57:13.348782+0000","last_fullsized":"2023-10-18T08:57:13.348782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.364976+0000","last_change":"2023-10-18T08:57:14.363761+0000","last_active":"2023-10-18T08:57:14.364976+0000","last_peered":"2023-10-18T08:57:14.364976+0000","last_clean":"2023-10-18T08:57:14.364976+0000","last_became_active":"2023-10-18T08:57:14.363321+0000","last_became_peered":"2023-10-18T08:57:14.363321+0000","last_unstale":"2023-10-18T08:57:14.364976+0000","last_undegraded":"2023-10-18T08:57:14.364976+0000","last_fullsized":"2023-10-18T08:57:14.364976+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:08.465913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.367726+0000","last_change":"2023-10-18T08:57:14.367626+0000","last_active":"2023-10-18T08:57:14.367726+0000","last_peered":"2023-10-18T08:57:14.367726+0000","last_clean":"2023-10-18T08:57:14.367726+0000","last_became_active":"2023-10-18T08:57:14.364913+0000","last_became_peered":"2023-10-18T08:57:14.364913+0000","last_unstale":"2023-10-18T08:57:14.367726+0000","last_undegraded":"2023-10-18T08:57:14.367726+0000","last_fullsized":"2023-10-18T08:57:14.367726+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:35.097485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.354808+0000","last_change":"2023-10-18T08:57:14.354808+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.354808+0000","last_undegraded":"2023-10-18T08:57:14.354808+0000","last_fullsized":"2023-10-18T08:57:14.354808+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.368309+0000","last_change":"2023-10-18T08:57:14.368186+0000","last_active":"2023-10-18T08:57:14.368309+0000","last_peered":"2023-10-18T08:57:14.368309+0000","last_clean":"2023-10-18T08:57:14.368309+0000","last_became_active":"2023-10-18T08:57:14.367105+0000","last_became_peered":"2023-10-18T08:57:14.367105+0000","last_unstale":"2023-10-18T08:57:14.368309+0000","last_undegraded":"2023-10-18T08:57:14.368309+0000","last_fullsized":"2023-10-18T08:57:14.368309+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:08.232425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.352501+0000","last_change":"2023-10-18T08:57:14.352501+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.352501+0000","last_undegraded":"2023-10-18T08:57:14.352501+0000","last_fullsized":"2023-10-18T08:57:14.352501+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.343633+0000","last_change":"2023-10-18T08:57:13.343633+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.343633+0000","last_undegraded":"2023-10-18T08:57:13.343633+0000","last_fullsized":"2023-10-18T08:57:13.343633+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.347605+0000","last_change":"2023-10-18T08:57:13.347605+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.347605+0000","last_undegraded":"2023-10-18T08:57:13.347605+0000","last_fullsized":"2023-10-18T08:57:13.347605+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.345028+0000","last_change":"2023-10-18T08:57:13.345028+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.345028+0000","last_undegraded":"2023-10-18T08:57:13.345028+0000","last_fullsized":"2023-10-18T08:57:13.345028+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.341352+0000","last_change":"2023-10-18T08:57:13.341352+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.341352+0000","last_undegraded":"2023-10-18T08:57:13.341352+0000","last_fullsized":"2023-10-18T08:57:13.341352+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.372035+0000","last_change":"2023-10-18T08:57:14.371953+0000","last_active":"2023-10-18T08:57:14.372035+0000","last_peered":"2023-10-18T08:57:14.372035+0000","last_clean":"2023-10-18T08:57:14.372035+0000","last_became_active":"2023-10-18T08:57:14.371639+0000","last_became_peered":"2023-10-18T08:57:14.371639+0000","last_unstale":"2023-10-18T08:57:14.372035+0000","last_undegraded":"2023-10-18T08:57:14.372035+0000","last_fullsized":"2023-10-18T08:57:14.372035+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:20:37.681848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.354927+0000","last_change":"2023-10-18T08:57:14.354927+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.354927+0000","last_undegraded":"2023-10-18T08:57:14.354927+0000","last_fullsized":"2023-10-18T08:57:14.354927+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.357026+0000","last_change":"2023-10-18T08:57:14.356889+0000","last_active":"2023-10-18T08:57:14.357026+0000","last_peered":"2023-10-18T08:57:14.357026+0000","last_clean":"2023-10-18T08:57:14.357026+0000","last_became_active":"2023-10-18T08:57:14.355993+0000","last_became_peered":"2023-10-18T08:57:14.355993+0000","last_unstale":"2023-10-18T08:57:14.357026+0000","last_undegraded":"2023-10-18T08:57:14.357026+0000","last_fullsized":"2023-10-18T08:57:14.357026+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:07.495221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.341122+0000","last_change":"2023-10-18T08:57:13.341122+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.341122+0000","last_undegraded":"2023-10-18T08:57:13.341122+0000","last_fullsized":"2023-10-18T08:57:13.341122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.344125+0000","last_change":"2023-10-18T08:57:13.344125+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.344125+0000","last_undegraded":"2023-10-18T08:57:13.344125+0000","last_fullsized":"2023-10-18T08:57:13.344125+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.368653+0000","last_change":"2023-10-18T08:57:14.368515+0000","last_active":"2023-10-18T08:57:14.368653+0000","last_peered":"2023-10-18T08:57:14.368653+0000","last_clean":"2023-10-18T08:57:14.368653+0000","last_became_active":"2023-10-18T08:57:14.368173+0000","last_became_peered":"2023-10-18T08:57:14.368173+0000","last_unstale":"2023-10-18T08:57:14.368653+0000","last_undegraded":"2023-10-18T08:57:14.368653+0000","last_fullsized":"2023-10-18T08:57:14.368653+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:08.465913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.366758+0000","last_change":"2023-10-18T08:57:14.366229+0000","last_active":"2023-10-18T08:57:14.366758+0000","last_peered":"2023-10-18T08:57:14.366758+0000","last_clean":"2023-10-18T08:57:14.366758+0000","last_became_active":"2023-10-18T08:57:14.365496+0000","last_became_peered":"2023-10-18T08:57:14.365496+0000","last_unstale":"2023-10-18T08:57:14.366758+0000","last_undegraded":"2023-10-18T08:57:14.366758+0000","last_fullsized":"2023-10-18T08:57:14.366758+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:30.531145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.373512+0000","last_change":"2023-10-18T08:57:14.373352+0000","last_active":"2023-10-18T08:57:14.373512+0000","last_peered":"2023-10-18T08:57:14.373512+0000","last_clean":"2023-10-18T08:57:14.373512+0000","last_became_active":"2023-10-18T08:57:14.373089+0000","last_became_peered":"2023-10-18T08:57:14.373089+0000","last_unstale":"2023-10-18T08:57:14.373512+0000","last_undegraded":"2023-10-18T08:57:14.373512+0000","last_fullsized":"2023-10-18T08:57:14.373512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:08:14.491862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.348920+0000","last_change":"2023-10-18T08:57:14.348920+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.348920+0000","last_undegraded":"2023-10-18T08:57:14.348920+0000","last_fullsized":"2023-10-18T08:57:14.348920+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.339494+0000","last_change":"2023-10-18T08:57:13.339494+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.339494+0000","last_undegraded":"2023-10-18T08:57:13.339494+0000","last_fullsized":"2023-10-18T08:57:13.339494+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.357648+0000","last_change":"2023-10-18T08:57:14.357648+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.357648+0000","last_undegraded":"2023-10-18T08:57:14.357648+0000","last_fullsized":"2023-10-18T08:57:14.357648+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.373937+0000","last_change":"2023-10-18T08:57:14.373541+0000","last_active":"2023-10-18T08:57:14.373937+0000","last_peered":"2023-10-18T08:57:14.373937+0000","last_clean":"2023-10-18T08:57:14.373937+0000","last_became_active":"2023-10-18T08:57:14.372892+0000","last_became_peered":"2023-10-18T08:57:14.372892+0000","last_unstale":"2023-10-18T08:57:14.373937+0000","last_undegraded":"2023-10-18T08:57:14.373937+0000","last_fullsized":"2023-10-18T08:57:14.373937+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:59:52.547228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.354120+0000","last_change":"2023-10-18T08:57:14.354120+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.354120+0000","last_undegraded":"2023-10-18T08:57:14.354120+0000","last_fullsized":"2023-10-18T08:57:14.354120+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.345024+0000","last_change":"2023-10-18T08:57:13.345024+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.345024+0000","last_undegraded":"2023-10-18T08:57:13.345024+0000","last_fullsized":"2023-10-18T08:57:13.345024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.339514+0000","last_change":"2023-10-18T08:57:13.339514+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.339514+0000","last_undegraded":"2023-10-18T08:57:13.339514+0000","last_fullsized":"2023-10-18T08:57:13.339514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.343518+0000","last_change":"2023-10-18T08:57:13.343518+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.343518+0000","last_undegraded":"2023-10-18T08:57:13.343518+0000","last_fullsized":"2023-10-18T08:57:13.343518+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.341302+0000","last_change":"2023-10-18T08:57:13.341302+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.341302+0000","last_undegraded":"2023-10-18T08:57:13.341302+0000","last_fullsized":"2023-10-18T08:57:13.341302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-18T08:57:13.347025+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:57:13.347025+0000","last_peered":"2023-10-18T08:57:13.347025+0000","last_clean":"2023-10-18T08:57:13.347025+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:57:13.347025+0000","last_undegraded":"2023-10-18T08:57:13.347025+0000","last_fullsized":"2023-10-18T08:57:13.347025+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:34.881744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":30,"acting":30,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":780,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194532,"statfs":{"total":200000000,"available":199200929,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.998,"15min":0.98999999999999999},"min":{"1min":0.67800000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3979999999999999,"5min":1.538,"15min":1.538},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.014,"15min":1.002},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":1.4019999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1399999999999999,"15min":1.1519999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.393,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.123,"15min":1.1160000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7849999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.98499999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.218,"15min":1.224},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.589,"5min":1.694,"15min":1.694},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1839999999999999,"15min":1.1859999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.526}]},{"osd":3,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.254,"15min":1.2609999999999999},"min":{"1min":0.91500000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.72,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.379},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3089999999999999,"15min":1.3129999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.8029999999999999,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.1339999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.1859999999999999,"15min":1.1579999999999999},"min":{"1min":0.92800000000000005,"5min":0.746,"15min":0.746},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.292},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.407,"15min":1.411},"min":{"1min":1.0369999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.885,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.4390000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3520000000000001,"15min":1.353},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.615,"5min":1.893,"15min":1.893},"last":1.343},{"interface":"front","average":{"1min":1.389,"5min":1.417,"15min":1.421},"min":{"1min":1.0940000000000001,"5min":0.874,"15min":0.874},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.617}]},{"osd":6,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.464,"15min":1.458},"min":{"1min":1.0169999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.9339999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.516,"15min":1.518},"min":{"1min":1.23,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.978,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.4990000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":192,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195120,"statfs":{"total":200000000,"available":199803381,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0800000000000001,"15min":1.073},"min":{"1min":0.65400000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.5529999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.133,"5min":1.216,"15min":1.23},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.016}]},{"osd":1,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0780000000000001,"15min":1.081},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.3420000000000001,"5min":1.524,"15min":1.524},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1659999999999999,"15min":1.1819999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.474,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":0.85699999999999998}]},{"osd":2,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.1579999999999999,"15min":1.1719999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.3},{"interface":"front","average":{"1min":1.165,"5min":1.1870000000000001,"15min":1.1910000000000001},"min":{"1min":0.82099999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5309999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.139}]},{"osd":3,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2509999999999999,"15min":1.2569999999999999},"min":{"1min":0.84799999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.502,"5min":1.784,"15min":1.784},"last":1.401},{"interface":"front","average":{"1min":1.222,"5min":1.2170000000000001,"15min":1.216},"min":{"1min":1.0329999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.494,"5min":1.5,"15min":1.5},"last":1.2110000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.1950000000000001,"15min":1.169},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.246},{"interface":"front","average":{"1min":1.284,"5min":1.272,"15min":1.27},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6990000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.607}]},{"osd":5,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3560000000000001,"15min":1.3500000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.665,"5min":1.667,"15min":1.667},"last":1.548},{"interface":"front","average":{"1min":1.331,"5min":1.2330000000000001,"15min":1.2170000000000001},"min":{"1min":0.54900000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":1.103}]},{"osd":7,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4019999999999999,"15min":1.3959999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.373},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5249999999999999,"15min":1.53},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8500000000000001,"5min":2.794,"15min":2.794},"last":1.831}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":197,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195114,"statfs":{"total":200000000,"available":199797564,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2010000000000001,"15min":1.1850000000000001},"min":{"1min":0.77800000000000002,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.7,"5min":1.915,"15min":1.915},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.288,"15min":1.292},"min":{"1min":0.746,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7130000000000001,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.0289999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.2030000000000001,"15min":1.2210000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.613,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.76000000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.581,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.94499999999999995}]},{"osd":2,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2689999999999999,"15min":1.268},"min":{"1min":0.77700000000000002,"5min":0.755,"15min":0.755},"max":{"1min":1.7749999999999999,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.47},{"interface":"front","average":{"1min":1.27,"5min":1.29,"15min":1.2929999999999999},"min":{"1min":0.96099999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.7210000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.669}]},{"osd":3,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.232,"15min":1.24},"min":{"1min":0.84799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5640000000000001,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.321,"15min":1.327},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.784,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.627}]},{"osd":4,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.417,"15min":1.4199999999999999},"min":{"1min":0.84899999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8799999999999999,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.792},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.5069999999999999,"15min":1.5369999999999999},"min":{"1min":0.90100000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7929999999999999,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":1.8999999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.3839999999999999,"15min":1.4139999999999999},"min":{"1min":0.873,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6799999999999999,"5min":5.141,"15min":5.141},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.5720000000000001,"15min":1.5920000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.931,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":2.0590000000000002}]},{"osd":7,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.661,"15min":1.677},"min":{"1min":1.038,"5min":1.014,"15min":1.014},"max":{"1min":1.9990000000000001,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":1.956},{"interface":"front","average":{"1min":1.423,"5min":1.508,"15min":1.522},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.907,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":1.7230000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":196,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195115,"statfs":{"total":200000000,"available":199798583,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.105,"15min":1.109},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.1679999999999999,"15min":1.1819999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0659999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.262}]},{"osd":1,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.0980000000000001,"15min":1.095},"min":{"1min":0.76100000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.583,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.167,"5min":1.2310000000000001,"15min":1.2410000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.929,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.1699999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.1779999999999999,"15min":1.1759999999999999},"min":{"1min":0.75600000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.365},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.29,"15min":1.3100000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.601,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.3959999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.292,"15min":1.302},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4159999999999999,"15min":1.413},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.0899999999999999,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.5169999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3120000000000001,"15min":1.3},"min":{"1min":0.84099999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.8080000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.365,"15min":1.3720000000000001},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.4199999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.397,"15min":1.3819999999999999},"min":{"1min":1.169,"5min":0.998,"15min":0.998},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.51,"5min":1.5680000000000001,"15min":1.5780000000000001},"min":{"1min":1.04,"5min":1.04,"15min":1.04},"max":{"1min":1.8939999999999999,"5min":3.6859999999999999,"15min":3.6859999999999999},"last":1.329}]},{"osd":7,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"max":{"1min":2.2759999999999998,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.659},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.5629999999999999,"15min":1.554},"min":{"1min":1.3089999999999999,"5min":1.23,"15min":1.23},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.569}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":203,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195108,"statfs":{"total":200000000,"available":199791297,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":0.82399999999999995,"15min":0.76500000000000001},"min":{"1min":0.52500000000000002,"5min":0.439,"15min":0.439},"max":{"1min":5.5730000000000004,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.365,"5min":0.93999999999999995,"15min":0.86899999999999999},"min":{"1min":0.53700000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.5599999999999996,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":1.0549999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":0.98499999999999999,"15min":0.94399999999999995},"min":{"1min":0.59399999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":6.8099999999999996,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.238,"5min":1.1259999999999999,"15min":1.1080000000000001},"min":{"1min":0.69099999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.466}]},{"osd":2,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.0620000000000001,"15min":1.0169999999999999},"min":{"1min":0.61199999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":7.4580000000000002,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.0880000000000001,"15min":1.048},"min":{"1min":0.56699999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":3.5110000000000001,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":1.613}]},{"osd":4,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":2.3849999999999998,"5min":1.4570000000000001,"15min":1.3029999999999999},"min":{"1min":0.99099999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":18.664999999999999,"5min":18.664999999999999,"15min":18.664999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.1899999999999999,"15min":1.1439999999999999},"min":{"1min":0.83999999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":7.0700000000000003,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":1.244}]},{"osd":5,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.266,"15min":1.214},"min":{"1min":0.872,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":6.9139999999999997,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":1.175},{"interface":"front","average":{"1min":2.306,"5min":1.4690000000000001,"15min":1.329},"min":{"1min":1.0269999999999999,"5min":0.748,"15min":0.748},"max":{"1min":16.390999999999998,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.6799999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.8049999999999999,"5min":1.4750000000000001,"15min":1.4199999999999999},"min":{"1min":1.022,"5min":0.625,"15min":0.625},"max":{"1min":7.7270000000000003,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.7190000000000001,"5min":1.3819999999999999,"15min":1.3260000000000001},"min":{"1min":0.96499999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":7.242,"5min":7.242,"15min":7.242},"last":1.3180000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.643,"5min":1.367,"15min":1.321},"min":{"1min":0.94299999999999995,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.74},{"interface":"front","average":{"1min":2.1309999999999998,"5min":1.54,"15min":1.4419999999999999},"min":{"1min":0.84999999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":12.946,"5min":12.946,"15min":12.946},"last":1.544}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":192,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195120,"statfs":{"total":200000000,"available":199803381,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.92000000000000004,"15min":0.93999999999999995},"min":{"1min":0.433,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.395,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.024,"15min":1.0569999999999999},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3700000000000001,"5min":1.651,"15min":1.651},"last":0.63100000000000001}]},{"osd":1,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":1.04,"15min":1.0600000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3660000000000001,"5min":4.29,"15min":4.29},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.87,"5min":1.0580000000000001,"15min":1.089},"min":{"1min":0.57499999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3009999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.171}]},{"osd":3,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.105,"15min":1.133},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.9139999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.111,"5min":1.4419999999999999,"15min":1.4970000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.609,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":1.546}]},{"osd":4,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.47,"15min":1.502},"min":{"1min":0.90900000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.7,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":1.7450000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.4239999999999999,"15min":1.4530000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.663,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.6080000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.248,"15min":1.272},"min":{"1min":0.79400000000000004,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5389999999999999,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":1.056},{"interface":"front","average":{"1min":1.256,"5min":1.542,"15min":1.5900000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.7569999999999999,"5min":7.657,"15min":7.657},"last":1.863}]},{"osd":6,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.4950000000000001,"15min":1.5389999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7170000000000001,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.698},{"interface":"front","average":{"1min":1.244,"5min":1.4059999999999999,"15min":1.4330000000000001},"min":{"1min":0.94099999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.796,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":1.3759999999999999}]},{"osd":7,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.5449999999999999,"15min":1.5780000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7350000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.4490000000000001,"15min":1.474},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.6899999999999999,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.335}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":196,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195115,"statfs":{"total":200000000,"available":199798583,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.88},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.871,"15min":0.88400000000000001},"min":{"1min":0.54400000000000004,"5min":0.443,"15min":0.443},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.59299999999999997}]},{"osd":2,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.90000000000000002,"15min":0.91900000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.222,"5min":1.859,"15min":1.859},"last":0.754},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.82599999999999996,"15min":0.83299999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.73999999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.98499999999999999,"15min":1.0089999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.024,"15min":1.048},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.7110000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":0.64400000000000002}]},{"osd":4,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.224,"15min":1.236},"min":{"1min":0.71499999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.53,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.1830000000000001,"15min":1.1890000000000001},"min":{"1min":0.85999999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.99,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.0509999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.262,"15min":1.2789999999999999},"min":{"1min":0.82899999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.6299999999999999,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.335,"15min":1.341},"min":{"1min":0.84999999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.8480000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.708}]},{"osd":6,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2529999999999999,"15min":1.2649999999999999},"min":{"1min":0.86199999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.2050000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.095},{"interface":"front","average":{"1min":1.143,"5min":1.228,"15min":1.242},"min":{"1min":0.72999999999999998,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.853,"5min":1.853,"15min":1.853},"last":1.236}]},{"osd":7,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.3160000000000001,"15min":1.333},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.0419999999999998,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.333,"15min":1.3380000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.8169999999999999,"5min":1.994,"15min":1.994},"last":1.8169999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":773,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194539,"statfs":{"total":200000000,"available":199208215,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.92600000000000005,"15min":0.94599999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.25,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":1.079},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.077,"15min":1.083},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4990000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.6319999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.339,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.125,"15min":1.125},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.2450000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.2849999999999999,"15min":1.327},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.294,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.5109999999999999,"15min":1.5429999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.8640000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.3720000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.2909999999999999,"15min":1.3260000000000001},"min":{"1min":0.72899999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.3460000000000001,"5min":4.492,"15min":4.492},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2509999999999999,"15min":1.2470000000000001},"min":{"1min":0.78500000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":1.3029999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.417,"15min":1.4630000000000001},"min":{"1min":0.79000000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.4770000000000001,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3879999999999999,"15min":1.3959999999999999},"min":{"1min":0.81299999999999994,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.758,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.8240000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5980000000000001,"15min":1.6220000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.952,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.5069999999999999,"15min":1.532},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":2.0910000000000002,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.351}]},{"osd":7,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.647,"15min":1.671},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.149,"5min":3.492,"15min":3.492},"last":1.9339999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.6830000000000001,"15min":1.7330000000000001},"min":{"1min":0.96799999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8859999999999999,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.5569999999999999}]}]}],"pool_statfs":[]}} 2023-10-18T08:57:20.412 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:57:20.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.680+0000 7fd1e189b640 1 -- 172.21.15.40:0/3918001445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1dc0ce430 msgr2=0x7fd1dc0b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:20.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.680+0000 7fd1e189b640 1 --2- 172.21.15.40:0/3918001445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1dc0ce430 0x7fd1dc0b4e00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4005fd0 tx=0x7fd1d402c0f0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.680+0000 7fd1e189b640 1 -- 172.21.15.40:0/3918001445 shutdown_connections 2023-10-18T08:57:20.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.680+0000 7fd1e189b640 1 --2- 172.21.15.40:0/3918001445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1dc0ce430 0x7fd1dc0b4e00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.680+0000 7fd1e189b640 1 --2- 172.21.15.40:0/3918001445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1dc0b7000 0x7fd1dc0cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.680+0000 7fd1e189b640 1 -- 172.21.15.40:0/3918001445 >> 172.21.15.40:0/3918001445 conn(0x7fd1dc06a780 msgr2=0x7fd1dc06ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:20.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.680+0000 7fd1e189b640 1 -- 172.21.15.40:0/3918001445 shutdown_connections 2023-10-18T08:57:20.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.680+0000 7fd1e189b640 1 -- 172.21.15.40:0/3918001445 wait complete. 2023-10-18T08:57:20.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.682+0000 7fd1e189b640 1 Processor -- start 2023-10-18T08:57:20.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.682+0000 7fd1e189b640 1 -- start start 2023-10-18T08:57:20.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.683+0000 7fd1e189b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1dc0b7000 0x7fd1dc1f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:20.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.683+0000 7fd1e189b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1dc0ce430 0x7fd1dc0cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:20.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.683+0000 7fd1e189b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1dc15aa50 0x7fd1dc1591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:20.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.683+0000 7fd1e189b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc150210 con 0x7fd1dc0b7000 2023-10-18T08:57:20.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.683+0000 7fd1e189b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc15a5c0 con 0x7fd1dc15aa50 2023-10-18T08:57:20.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.683+0000 7fd1e189b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0cb210 con 0x7fd1dc0ce430 2023-10-18T08:57:20.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.683+0000 7fd1daffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1dc0b7000 0x7fd1dc1f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:20.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.683+0000 7fd1daffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1dc0b7000 0x7fd1dc1f2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54136/0 (socket says 172.21.15.40:54136) 2023-10-18T08:57:20.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.683+0000 7fd1daffd640 1 -- 172.21.15.40:0/1355827226 learned_addr learned my addr 172.21.15.40:0/1355827226 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:20.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.683+0000 7fd1d3fff640 1 --2- 172.21.15.40:0/1355827226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1dc0ce430 0x7fd1dc0cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:20.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.684+0000 7fd1db7fe640 1 --2- 172.21.15.40:0/1355827226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1dc15aa50 0x7fd1dc1591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:20.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.684+0000 7fd1daffd640 1 -- 172.21.15.40:0/1355827226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1dc0ce430 msgr2=0x7fd1dc0cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:20.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.684+0000 7fd1daffd640 1 --2- 172.21.15.40:0/1355827226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1dc0ce430 0x7fd1dc0cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.684+0000 7fd1daffd640 1 -- 172.21.15.40:0/1355827226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1dc15aa50 msgr2=0x7fd1dc1591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:20.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.684+0000 7fd1daffd640 1 --2- 172.21.15.40:0/1355827226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1dc15aa50 0x7fd1dc1591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.684+0000 7fd1daffd640 1 -- 172.21.15.40:0/1355827226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d4009840 con 0x7fd1dc0b7000 2023-10-18T08:57:20.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.684+0000 7fd1daffd640 1 --2- 172.21.15.40:0/1355827226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1dc0b7000 0x7fd1dc1f2a10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00c390 tx=0x7fd1cc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:20.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.684+0000 7fd1d8ff9640 1 -- 172.21.15.40:0/1355827226 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc004230 con 0x7fd1dc0b7000 2023-10-18T08:57:20.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.684+0000 7fd1d8ff9640 1 -- 172.21.15.40:0/1355827226 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd1cc0044a0 con 0x7fd1dc0b7000 2023-10-18T08:57:20.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.684+0000 7fd1d8ff9640 1 -- 172.21.15.40:0/1355827226 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc003850 con 0x7fd1dc0b7000 2023-10-18T08:57:20.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.684+0000 7fd1e189b640 1 -- 172.21.15.40:0/1355827226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1dc159710 con 0x7fd1dc0b7000 2023-10-18T08:57:20.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.685+0000 7fd1e189b640 1 -- 172.21.15.40:0/1355827226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1dc159c90 con 0x7fd1dc0b7000 2023-10-18T08:57:20.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.686+0000 7fd1e189b640 1 -- 172.21.15.40:0/1355827226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1dc066fd0 con 0x7fd1dc0b7000 2023-10-18T08:57:20.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.687+0000 7fd1d8ff9640 1 -- 172.21.15.40:0/1355827226 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd1cc0039d0 con 0x7fd1dc0b7000 2023-10-18T08:57:20.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.687+0000 7fd1d8ff9640 1 --2- 172.21.15.40:0/1355827226 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd1bc06ae70 0x7fd1bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:20.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.688+0000 7fd1d8ff9640 1 -- 172.21.15.40:0/1355827226 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1cc088d70 con 0x7fd1dc0b7000 2023-10-18T08:57:20.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.688+0000 7fd1d3fff640 1 --2- 172.21.15.40:0/1355827226 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd1bc06ae70 0x7fd1bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:20.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.689+0000 7fd1d3fff640 1 --2- 172.21.15.40:0/1355827226 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd1bc06ae70 0x7fd1bc06d330 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4005fd0 tx=0x7fd1c4005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:20.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.692+0000 7fd1d8ff9640 1 -- 172.21.15.40:0/1355827226 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1cc053ee0 con 0x7fd1dc0b7000 2023-10-18T08:57:20.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.881+0000 7fd1e189b640 1 -- 172.21.15.40:0/1355827226 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1dc15cd50 con 0x7fd1bc06ae70 2023-10-18T08:57:20.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.907+0000 7fd1d8ff9640 1 -- 172.21.15.40:0/1355827226 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+273765 (secure 0 0 0) 0x7fd1dc15cd50 con 0x7fd1bc06ae70 2023-10-18T08:57:20.912 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:20.915 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:57:20.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.914+0000 7fd1e189b640 1 -- 172.21.15.40:0/1355827226 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd1bc06ae70 msgr2=0x7fd1bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:20.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.914+0000 7fd1e189b640 1 --2- 172.21.15.40:0/1355827226 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd1bc06ae70 0x7fd1bc06d330 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4005fd0 tx=0x7fd1c4005850 comp rx=0 tx=0).stop 2023-10-18T08:57:20.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.914+0000 7fd1e189b640 1 -- 172.21.15.40:0/1355827226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1dc0b7000 msgr2=0x7fd1dc1f2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:20.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.914+0000 7fd1e189b640 1 --2- 172.21.15.40:0/1355827226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1dc0b7000 0x7fd1dc1f2a10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00c390 tx=0x7fd1cc00c850 comp rx=0 tx=0).stop 2023-10-18T08:57:20.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.914+0000 7fd1e189b640 1 -- 172.21.15.40:0/1355827226 shutdown_connections 2023-10-18T08:57:20.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.914+0000 7fd1e189b640 1 --2- 172.21.15.40:0/1355827226 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd1bc06ae70 0x7fd1bc06d330 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.914+0000 7fd1e189b640 1 --2- 172.21.15.40:0/1355827226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1dc15aa50 0x7fd1dc1591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.914+0000 7fd1e189b640 1 --2- 172.21.15.40:0/1355827226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1dc0ce430 0x7fd1dc0cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.914+0000 7fd1e189b640 1 --2- 172.21.15.40:0/1355827226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1dc0b7000 0x7fd1dc1f2a10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:20.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.914+0000 7fd1e189b640 1 -- 172.21.15.40:0/1355827226 >> 172.21.15.40:0/1355827226 conn(0x7fd1dc06a780 msgr2=0x7fd1dc15e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:20.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.914+0000 7fd1e189b640 1 -- 172.21.15.40:0/1355827226 shutdown_connections 2023-10-18T08:57:20.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:20.914+0000 7fd1e189b640 1 -- 172.21.15.40:0/1355827226 wait complete. 2023-10-18T08:57:20.936 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2023-10-18T08:57:18.937983+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":96,"acting":96,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2732,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559767,"statfs":{"total":1600000000,"available":1597201933,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.599581"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.354858+0000","last_change":"2023-10-18T08:57:14.354858+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.354858+0000","last_undegraded":"2023-10-18T08:57:14.354858+0000","last_fullsized":"2023-10-18T08:57:14.354858+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.357660+0000","last_change":"2023-10-18T08:57:14.357660+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.357660+0000","last_undegraded":"2023-10-18T08:57:14.357660+0000","last_fullsized":"2023-10-18T08:57:14.357660+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.353317+0000","last_change":"2023-10-18T08:57:14.353317+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.353317+0000","last_undegraded":"2023-10-18T08:57:14.353317+0000","last_fullsized":"2023-10-18T08:57:14.353317+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.362613+0000","last_change":"2023-10-18T08:57:14.362613+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.362613+0000","last_undegraded":"2023-10-18T08:57:14.362613+0000","last_fullsized":"2023-10-18T08:57:14.362613+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.356438+0000","last_change":"2023-10-18T08:57:14.356438+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.356438+0000","last_undegraded":"2023-10-18T08:57:14.356438+0000","last_fullsized":"2023-10-18T08:57:14.356438+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.355530+0000","last_change":"2023-10-18T08:57:14.355530+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.355530+0000","last_undegraded":"2023-10-18T08:57:14.355530+0000","last_fullsized":"2023-10-18T08:57:14.355530+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.371711+0000","last_change":"2023-10-18T08:57:14.371648+0000","last_active":"2023-10-18T08:57:14.371711+0000","last_peered":"2023-10-18T08:57:14.371711+0000","last_clean":"2023-10-18T08:57:14.371711+0000","last_became_active":"2023-10-18T08:57:14.371360+0000","last_became_peered":"2023-10-18T08:57:14.371360+0000","last_unstale":"2023-10-18T08:57:14.371711+0000","last_undegraded":"2023-10-18T08:57:14.371711+0000","last_fullsized":"2023-10-18T08:57:14.371711+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:28.358388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.344547+0000","last_change":"2023-10-18T08:57:13.344547+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.344547+0000","last_undegraded":"2023-10-18T08:57:13.344547+0000","last_fullsized":"2023-10-18T08:57:13.344547+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.339855+0000","last_change":"2023-10-18T08:57:13.339855+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.339855+0000","last_undegraded":"2023-10-18T08:57:13.339855+0000","last_fullsized":"2023-10-18T08:57:13.339855+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.373956+0000","last_change":"2023-10-18T08:57:14.373706+0000","last_active":"2023-10-18T08:57:14.373956+0000","last_peered":"2023-10-18T08:57:14.373956+0000","last_clean":"2023-10-18T08:57:14.373956+0000","last_became_active":"2023-10-18T08:57:14.373294+0000","last_became_peered":"2023-10-18T08:57:14.373294+0000","last_unstale":"2023-10-18T08:57:14.373956+0000","last_undegraded":"2023-10-18T08:57:14.373956+0000","last_fullsized":"2023-10-18T08:57:14.373956+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:26:09.061307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.365216+0000","last_change":"2023-10-18T08:57:14.365008+0000","last_active":"2023-10-18T08:57:14.365216+0000","last_peered":"2023-10-18T08:57:14.365216+0000","last_clean":"2023-10-18T08:57:14.365216+0000","last_became_active":"2023-10-18T08:57:14.361849+0000","last_became_peered":"2023-10-18T08:57:14.361849+0000","last_unstale":"2023-10-18T08:57:14.365216+0000","last_undegraded":"2023-10-18T08:57:14.365216+0000","last_fullsized":"2023-10-18T08:57:14.365216+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:59:52.547228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.344538+0000","last_change":"2023-10-18T08:57:13.344538+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.344538+0000","last_undegraded":"2023-10-18T08:57:13.344538+0000","last_fullsized":"2023-10-18T08:57:13.344538+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.341960+0000","last_change":"2023-10-18T08:57:13.341960+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.341960+0000","last_undegraded":"2023-10-18T08:57:13.341960+0000","last_fullsized":"2023-10-18T08:57:13.341960+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.344612+0000","last_change":"2023-10-18T08:57:13.344612+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.344612+0000","last_undegraded":"2023-10-18T08:57:13.344612+0000","last_fullsized":"2023-10-18T08:57:13.344612+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.346128+0000","last_change":"2023-10-18T08:57:13.346128+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.346128+0000","last_undegraded":"2023-10-18T08:57:13.346128+0000","last_fullsized":"2023-10-18T08:57:13.346128+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.352399+0000","last_change":"2023-10-18T08:57:14.352399+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.352399+0000","last_undegraded":"2023-10-18T08:57:14.352399+0000","last_fullsized":"2023-10-18T08:57:14.352399+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.344191+0000","last_change":"2023-10-18T08:57:13.344191+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.344191+0000","last_undegraded":"2023-10-18T08:57:13.344191+0000","last_fullsized":"2023-10-18T08:57:13.344191+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.361992+0000","last_change":"2023-10-18T08:57:14.361992+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.361992+0000","last_undegraded":"2023-10-18T08:57:14.361992+0000","last_fullsized":"2023-10-18T08:57:14.361992+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.373105+0000","last_change":"2023-10-18T08:57:14.372079+0000","last_active":"2023-10-18T08:57:14.373105+0000","last_peered":"2023-10-18T08:57:14.373105+0000","last_clean":"2023-10-18T08:57:14.373105+0000","last_became_active":"2023-10-18T08:57:14.371832+0000","last_became_peered":"2023-10-18T08:57:14.371832+0000","last_unstale":"2023-10-18T08:57:14.373105+0000","last_undegraded":"2023-10-18T08:57:14.373105+0000","last_fullsized":"2023-10-18T08:57:14.373105+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:05.351228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.350346+0000","last_change":"2023-10-18T08:57:14.350346+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.350346+0000","last_undegraded":"2023-10-18T08:57:14.350346+0000","last_fullsized":"2023-10-18T08:57:14.350346+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.348782+0000","last_change":"2023-10-18T08:57:13.348782+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.348782+0000","last_undegraded":"2023-10-18T08:57:13.348782+0000","last_fullsized":"2023-10-18T08:57:13.348782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.364976+0000","last_change":"2023-10-18T08:57:14.363761+0000","last_active":"2023-10-18T08:57:14.364976+0000","last_peered":"2023-10-18T08:57:14.364976+0000","last_clean":"2023-10-18T08:57:14.364976+0000","last_became_active":"2023-10-18T08:57:14.363321+0000","last_became_peered":"2023-10-18T08:57:14.363321+0000","last_unstale":"2023-10-18T08:57:14.364976+0000","last_undegraded":"2023-10-18T08:57:14.364976+0000","last_fullsized":"2023-10-18T08:57:14.364976+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:08.465913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.367726+0000","last_change":"2023-10-18T08:57:14.367626+0000","last_active":"2023-10-18T08:57:14.367726+0000","last_peered":"2023-10-18T08:57:14.367726+0000","last_clean":"2023-10-18T08:57:14.367726+0000","last_became_active":"2023-10-18T08:57:14.364913+0000","last_became_peered":"2023-10-18T08:57:14.364913+0000","last_unstale":"2023-10-18T08:57:14.367726+0000","last_undegraded":"2023-10-18T08:57:14.367726+0000","last_fullsized":"2023-10-18T08:57:14.367726+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:35.097485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.354808+0000","last_change":"2023-10-18T08:57:14.354808+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.354808+0000","last_undegraded":"2023-10-18T08:57:14.354808+0000","last_fullsized":"2023-10-18T08:57:14.354808+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.368309+0000","last_change":"2023-10-18T08:57:14.368186+0000","last_active":"2023-10-18T08:57:14.368309+0000","last_peered":"2023-10-18T08:57:14.368309+0000","last_clean":"2023-10-18T08:57:14.368309+0000","last_became_active":"2023-10-18T08:57:14.367105+0000","last_became_peered":"2023-10-18T08:57:14.367105+0000","last_unstale":"2023-10-18T08:57:14.368309+0000","last_undegraded":"2023-10-18T08:57:14.368309+0000","last_fullsized":"2023-10-18T08:57:14.368309+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:08.232425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.352501+0000","last_change":"2023-10-18T08:57:14.352501+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.352501+0000","last_undegraded":"2023-10-18T08:57:14.352501+0000","last_fullsized":"2023-10-18T08:57:14.352501+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.343633+0000","last_change":"2023-10-18T08:57:13.343633+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.343633+0000","last_undegraded":"2023-10-18T08:57:13.343633+0000","last_fullsized":"2023-10-18T08:57:13.343633+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.347605+0000","last_change":"2023-10-18T08:57:13.347605+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.347605+0000","last_undegraded":"2023-10-18T08:57:13.347605+0000","last_fullsized":"2023-10-18T08:57:13.347605+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.345028+0000","last_change":"2023-10-18T08:57:13.345028+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.345028+0000","last_undegraded":"2023-10-18T08:57:13.345028+0000","last_fullsized":"2023-10-18T08:57:13.345028+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.341352+0000","last_change":"2023-10-18T08:57:13.341352+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.341352+0000","last_undegraded":"2023-10-18T08:57:13.341352+0000","last_fullsized":"2023-10-18T08:57:13.341352+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.372035+0000","last_change":"2023-10-18T08:57:14.371953+0000","last_active":"2023-10-18T08:57:14.372035+0000","last_peered":"2023-10-18T08:57:14.372035+0000","last_clean":"2023-10-18T08:57:14.372035+0000","last_became_active":"2023-10-18T08:57:14.371639+0000","last_became_peered":"2023-10-18T08:57:14.371639+0000","last_unstale":"2023-10-18T08:57:14.372035+0000","last_undegraded":"2023-10-18T08:57:14.372035+0000","last_fullsized":"2023-10-18T08:57:14.372035+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:20:37.681848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.354927+0000","last_change":"2023-10-18T08:57:14.354927+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.354927+0000","last_undegraded":"2023-10-18T08:57:14.354927+0000","last_fullsized":"2023-10-18T08:57:14.354927+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.357026+0000","last_change":"2023-10-18T08:57:14.356889+0000","last_active":"2023-10-18T08:57:14.357026+0000","last_peered":"2023-10-18T08:57:14.357026+0000","last_clean":"2023-10-18T08:57:14.357026+0000","last_became_active":"2023-10-18T08:57:14.355993+0000","last_became_peered":"2023-10-18T08:57:14.355993+0000","last_unstale":"2023-10-18T08:57:14.357026+0000","last_undegraded":"2023-10-18T08:57:14.357026+0000","last_fullsized":"2023-10-18T08:57:14.357026+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:07.495221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.341122+0000","last_change":"2023-10-18T08:57:13.341122+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.341122+0000","last_undegraded":"2023-10-18T08:57:13.341122+0000","last_fullsized":"2023-10-18T08:57:13.341122+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.344125+0000","last_change":"2023-10-18T08:57:13.344125+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.344125+0000","last_undegraded":"2023-10-18T08:57:13.344125+0000","last_fullsized":"2023-10-18T08:57:13.344125+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.368653+0000","last_change":"2023-10-18T08:57:14.368515+0000","last_active":"2023-10-18T08:57:14.368653+0000","last_peered":"2023-10-18T08:57:14.368653+0000","last_clean":"2023-10-18T08:57:14.368653+0000","last_became_active":"2023-10-18T08:57:14.368173+0000","last_became_peered":"2023-10-18T08:57:14.368173+0000","last_unstale":"2023-10-18T08:57:14.368653+0000","last_undegraded":"2023-10-18T08:57:14.368653+0000","last_fullsized":"2023-10-18T08:57:14.368653+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:08.465913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.366758+0000","last_change":"2023-10-18T08:57:14.366229+0000","last_active":"2023-10-18T08:57:14.366758+0000","last_peered":"2023-10-18T08:57:14.366758+0000","last_clean":"2023-10-18T08:57:14.366758+0000","last_became_active":"2023-10-18T08:57:14.365496+0000","last_became_peered":"2023-10-18T08:57:14.365496+0000","last_unstale":"2023-10-18T08:57:14.366758+0000","last_undegraded":"2023-10-18T08:57:14.366758+0000","last_fullsized":"2023-10-18T08:57:14.366758+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:30.531145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.373512+0000","last_change":"2023-10-18T08:57:14.373352+0000","last_active":"2023-10-18T08:57:14.373512+0000","last_peered":"2023-10-18T08:57:14.373512+0000","last_clean":"2023-10-18T08:57:14.373512+0000","last_became_active":"2023-10-18T08:57:14.373089+0000","last_became_peered":"2023-10-18T08:57:14.373089+0000","last_unstale":"2023-10-18T08:57:14.373512+0000","last_undegraded":"2023-10-18T08:57:14.373512+0000","last_fullsized":"2023-10-18T08:57:14.373512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:08:14.491862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.348920+0000","last_change":"2023-10-18T08:57:14.348920+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.348920+0000","last_undegraded":"2023-10-18T08:57:14.348920+0000","last_fullsized":"2023-10-18T08:57:14.348920+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.339494+0000","last_change":"2023-10-18T08:57:13.339494+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.339494+0000","last_undegraded":"2023-10-18T08:57:13.339494+0000","last_fullsized":"2023-10-18T08:57:13.339494+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.357648+0000","last_change":"2023-10-18T08:57:14.357648+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.357648+0000","last_undegraded":"2023-10-18T08:57:14.357648+0000","last_fullsized":"2023-10-18T08:57:14.357648+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-18T08:57:14.373937+0000","last_change":"2023-10-18T08:57:14.373541+0000","last_active":"2023-10-18T08:57:14.373937+0000","last_peered":"2023-10-18T08:57:14.373937+0000","last_clean":"2023-10-18T08:57:14.373937+0000","last_became_active":"2023-10-18T08:57:14.372892+0000","last_became_peered":"2023-10-18T08:57:14.372892+0000","last_unstale":"2023-10-18T08:57:14.373937+0000","last_undegraded":"2023-10-18T08:57:14.373937+0000","last_fullsized":"2023-10-18T08:57:14.373937+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:59:52.547228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2023-10-18T08:57:14.354120+0000","last_change":"2023-10-18T08:57:14.354120+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.354120+0000","last_undegraded":"2023-10-18T08:57:14.354120+0000","last_fullsized":"2023-10-18T08:57:14.354120+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.345024+0000","last_change":"2023-10-18T08:57:13.345024+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.345024+0000","last_undegraded":"2023-10-18T08:57:13.345024+0000","last_fullsized":"2023-10-18T08:57:13.345024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.339514+0000","last_change":"2023-10-18T08:57:13.339514+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.339514+0000","last_undegraded":"2023-10-18T08:57:13.339514+0000","last_fullsized":"2023-10-18T08:57:13.339514+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.343518+0000","last_change":"2023-10-18T08:57:13.343518+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.343518+0000","last_undegraded":"2023-10-18T08:57:13.343518+0000","last_fullsized":"2023-10-18T08:57:13.343518+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T08:57:14.334221+0000","last_change":"2023-10-18T08:57:14.334221+0000","last_active":"2023-10-18T08:57:14.334221+0000","last_peered":"2023-10-18T08:57:14.334221+0000","last_clean":"2023-10-18T08:57:14.334221+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:14.334221+0000","last_undegraded":"2023-10-18T08:57:14.334221+0000","last_fullsized":"2023-10-18T08:57:14.334221+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2023-10-18T08:57:13.341302+0000","last_change":"2023-10-18T08:57:13.341302+0000","last_active":"2023-10-18T08:57:13.322296+0000","last_peered":"2023-10-18T08:57:13.322296+0000","last_clean":"2023-10-18T08:57:13.322296+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:57:13.341302+0000","last_undegraded":"2023-10-18T08:57:13.341302+0000","last_fullsized":"2023-10-18T08:57:13.341302+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-18T08:57:13.347025+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:57:13.347025+0000","last_peered":"2023-10-18T08:57:13.347025+0000","last_clean":"2023-10-18T08:57:13.347025+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:57:13.347025+0000","last_undegraded":"2023-10-18T08:57:13.347025+0000","last_fullsized":"2023-10-18T08:57:13.347025+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:34.881744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":30,"acting":30,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":780,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194532,"statfs":{"total":200000000,"available":199200929,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.998,"15min":0.98999999999999999},"min":{"1min":0.67800000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3979999999999999,"5min":1.538,"15min":1.538},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.014,"15min":1.002},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":1.4019999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1399999999999999,"15min":1.1519999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.393,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.123,"15min":1.1160000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7849999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.98499999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.218,"15min":1.224},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.589,"5min":1.694,"15min":1.694},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1839999999999999,"15min":1.1859999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.526}]},{"osd":3,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.254,"15min":1.2609999999999999},"min":{"1min":0.91500000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.72,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.379},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3089999999999999,"15min":1.3129999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.8029999999999999,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.1339999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.1859999999999999,"15min":1.1579999999999999},"min":{"1min":0.92800000000000005,"5min":0.746,"15min":0.746},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.292},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.407,"15min":1.411},"min":{"1min":1.0369999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.885,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.4390000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3520000000000001,"15min":1.353},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.615,"5min":1.893,"15min":1.893},"last":1.343},{"interface":"front","average":{"1min":1.389,"5min":1.417,"15min":1.421},"min":{"1min":1.0940000000000001,"5min":0.874,"15min":0.874},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.617}]},{"osd":6,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.464,"15min":1.458},"min":{"1min":1.0169999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.9339999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.516,"15min":1.518},"min":{"1min":1.23,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.978,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.4990000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":192,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195120,"statfs":{"total":200000000,"available":199803381,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0800000000000001,"15min":1.073},"min":{"1min":0.65400000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.5529999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.133,"5min":1.216,"15min":1.23},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.016}]},{"osd":1,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0780000000000001,"15min":1.081},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.3420000000000001,"5min":1.524,"15min":1.524},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1659999999999999,"15min":1.1819999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.474,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":0.85699999999999998}]},{"osd":2,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.1579999999999999,"15min":1.1719999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.3},{"interface":"front","average":{"1min":1.165,"5min":1.1870000000000001,"15min":1.1910000000000001},"min":{"1min":0.82099999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5309999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.139}]},{"osd":3,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2509999999999999,"15min":1.2569999999999999},"min":{"1min":0.84799999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.502,"5min":1.784,"15min":1.784},"last":1.401},{"interface":"front","average":{"1min":1.222,"5min":1.2170000000000001,"15min":1.216},"min":{"1min":1.0329999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.494,"5min":1.5,"15min":1.5},"last":1.2110000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.1950000000000001,"15min":1.169},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.246},{"interface":"front","average":{"1min":1.284,"5min":1.272,"15min":1.27},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6990000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.607}]},{"osd":5,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3560000000000001,"15min":1.3500000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.665,"5min":1.667,"15min":1.667},"last":1.548},{"interface":"front","average":{"1min":1.331,"5min":1.2330000000000001,"15min":1.2170000000000001},"min":{"1min":0.54900000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":1.103}]},{"osd":7,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4019999999999999,"15min":1.3959999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.373},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5249999999999999,"15min":1.53},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8500000000000001,"5min":2.794,"15min":2.794},"last":1.831}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":197,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195114,"statfs":{"total":200000000,"available":199797564,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2010000000000001,"15min":1.1850000000000001},"min":{"1min":0.77800000000000002,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.7,"5min":1.915,"15min":1.915},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.288,"15min":1.292},"min":{"1min":0.746,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7130000000000001,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.0289999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.2030000000000001,"15min":1.2210000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.613,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.76000000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.581,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.94499999999999995}]},{"osd":2,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2689999999999999,"15min":1.268},"min":{"1min":0.77700000000000002,"5min":0.755,"15min":0.755},"max":{"1min":1.7749999999999999,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.47},{"interface":"front","average":{"1min":1.27,"5min":1.29,"15min":1.2929999999999999},"min":{"1min":0.96099999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.7210000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.669}]},{"osd":3,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.232,"15min":1.24},"min":{"1min":0.84799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5640000000000001,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.321,"15min":1.327},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.784,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.627}]},{"osd":4,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.417,"15min":1.4199999999999999},"min":{"1min":0.84899999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8799999999999999,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.792},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.5069999999999999,"15min":1.5369999999999999},"min":{"1min":0.90100000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7929999999999999,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":1.8999999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.3839999999999999,"15min":1.4139999999999999},"min":{"1min":0.873,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6799999999999999,"5min":5.141,"15min":5.141},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.5720000000000001,"15min":1.5920000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.931,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":2.0590000000000002}]},{"osd":7,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.661,"15min":1.677},"min":{"1min":1.038,"5min":1.014,"15min":1.014},"max":{"1min":1.9990000000000001,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":1.956},{"interface":"front","average":{"1min":1.423,"5min":1.508,"15min":1.522},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.907,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":1.7230000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":196,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195115,"statfs":{"total":200000000,"available":199798583,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.105,"15min":1.109},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.1679999999999999,"15min":1.1819999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0659999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.262}]},{"osd":1,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.0980000000000001,"15min":1.095},"min":{"1min":0.76100000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.583,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.167,"5min":1.2310000000000001,"15min":1.2410000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.929,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.1699999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.1779999999999999,"15min":1.1759999999999999},"min":{"1min":0.75600000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.365},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.29,"15min":1.3100000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.601,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.3959999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.292,"15min":1.302},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4159999999999999,"15min":1.413},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.0899999999999999,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.5169999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3120000000000001,"15min":1.3},"min":{"1min":0.84099999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.8080000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.365,"15min":1.3720000000000001},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.4199999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.397,"15min":1.3819999999999999},"min":{"1min":1.169,"5min":0.998,"15min":0.998},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.51,"5min":1.5680000000000001,"15min":1.5780000000000001},"min":{"1min":1.04,"5min":1.04,"15min":1.04},"max":{"1min":1.8939999999999999,"5min":3.6859999999999999,"15min":3.6859999999999999},"last":1.329}]},{"osd":7,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"max":{"1min":2.2759999999999998,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.659},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.5629999999999999,"15min":1.554},"min":{"1min":1.3089999999999999,"5min":1.23,"15min":1.23},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.569}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":203,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195108,"statfs":{"total":200000000,"available":199791297,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":0.82399999999999995,"15min":0.76500000000000001},"min":{"1min":0.52500000000000002,"5min":0.439,"15min":0.439},"max":{"1min":5.5730000000000004,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.365,"5min":0.93999999999999995,"15min":0.86899999999999999},"min":{"1min":0.53700000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.5599999999999996,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":1.0549999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":0.98499999999999999,"15min":0.94399999999999995},"min":{"1min":0.59399999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":6.8099999999999996,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.238,"5min":1.1259999999999999,"15min":1.1080000000000001},"min":{"1min":0.69099999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.466}]},{"osd":2,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.0620000000000001,"15min":1.0169999999999999},"min":{"1min":0.61199999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":7.4580000000000002,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.0880000000000001,"15min":1.048},"min":{"1min":0.56699999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":3.5110000000000001,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":1.613}]},{"osd":4,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":2.3849999999999998,"5min":1.4570000000000001,"15min":1.3029999999999999},"min":{"1min":0.99099999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":18.664999999999999,"5min":18.664999999999999,"15min":18.664999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.1899999999999999,"15min":1.1439999999999999},"min":{"1min":0.83999999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":7.0700000000000003,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":1.244}]},{"osd":5,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.266,"15min":1.214},"min":{"1min":0.872,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":6.9139999999999997,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":1.175},{"interface":"front","average":{"1min":2.306,"5min":1.4690000000000001,"15min":1.329},"min":{"1min":1.0269999999999999,"5min":0.748,"15min":0.748},"max":{"1min":16.390999999999998,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.6799999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.8049999999999999,"5min":1.4750000000000001,"15min":1.4199999999999999},"min":{"1min":1.022,"5min":0.625,"15min":0.625},"max":{"1min":7.7270000000000003,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.7190000000000001,"5min":1.3819999999999999,"15min":1.3260000000000001},"min":{"1min":0.96499999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":7.242,"5min":7.242,"15min":7.242},"last":1.3180000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.643,"5min":1.367,"15min":1.321},"min":{"1min":0.94299999999999995,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.74},{"interface":"front","average":{"1min":2.1309999999999998,"5min":1.54,"15min":1.4419999999999999},"min":{"1min":0.84999999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":12.946,"5min":12.946,"15min":12.946},"last":1.544}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":192,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195120,"statfs":{"total":200000000,"available":199803381,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.92000000000000004,"15min":0.93999999999999995},"min":{"1min":0.433,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.395,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.024,"15min":1.0569999999999999},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3700000000000001,"5min":1.651,"15min":1.651},"last":0.63100000000000001}]},{"osd":1,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":1.04,"15min":1.0600000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3660000000000001,"5min":4.29,"15min":4.29},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.87,"5min":1.0580000000000001,"15min":1.089},"min":{"1min":0.57499999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3009999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.171}]},{"osd":3,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.105,"15min":1.133},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.9139999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.111,"5min":1.4419999999999999,"15min":1.4970000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.609,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":1.546}]},{"osd":4,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.47,"15min":1.502},"min":{"1min":0.90900000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.7,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":1.7450000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.4239999999999999,"15min":1.4530000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.663,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.6080000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.248,"15min":1.272},"min":{"1min":0.79400000000000004,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5389999999999999,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":1.056},{"interface":"front","average":{"1min":1.256,"5min":1.542,"15min":1.5900000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.7569999999999999,"5min":7.657,"15min":7.657},"last":1.863}]},{"osd":6,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.4950000000000001,"15min":1.5389999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7170000000000001,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.698},{"interface":"front","average":{"1min":1.244,"5min":1.4059999999999999,"15min":1.4330000000000001},"min":{"1min":0.94099999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.796,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":1.3759999999999999}]},{"osd":7,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.5449999999999999,"15min":1.5780000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7350000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.4490000000000001,"15min":1.474},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.6899999999999999,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.335}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":196,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195115,"statfs":{"total":200000000,"available":199798583,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.88},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.871,"15min":0.88400000000000001},"min":{"1min":0.54400000000000004,"5min":0.443,"15min":0.443},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.59299999999999997}]},{"osd":2,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.90000000000000002,"15min":0.91900000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.222,"5min":1.859,"15min":1.859},"last":0.754},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.82599999999999996,"15min":0.83299999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.73999999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.98499999999999999,"15min":1.0089999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.024,"15min":1.048},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.7110000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":0.64400000000000002}]},{"osd":4,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.224,"15min":1.236},"min":{"1min":0.71499999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.53,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.1830000000000001,"15min":1.1890000000000001},"min":{"1min":0.85999999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.99,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.0509999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.262,"15min":1.2789999999999999},"min":{"1min":0.82899999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.6299999999999999,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.335,"15min":1.341},"min":{"1min":0.84999999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.8480000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.708}]},{"osd":6,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2529999999999999,"15min":1.2649999999999999},"min":{"1min":0.86199999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.2050000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.095},{"interface":"front","average":{"1min":1.143,"5min":1.228,"15min":1.242},"min":{"1min":0.72999999999999998,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.853,"5min":1.853,"15min":1.853},"last":1.236}]},{"osd":7,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.3160000000000001,"15min":1.333},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.0419999999999998,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.333,"15min":1.3380000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.8169999999999999,"5min":1.994,"15min":1.994},"last":1.8169999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":773,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194539,"statfs":{"total":200000000,"available":199208215,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.92600000000000005,"15min":0.94599999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.25,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":1.079},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.077,"15min":1.083},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4990000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.6319999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.339,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.125,"15min":1.125},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.2450000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.2849999999999999,"15min":1.327},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.294,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.5109999999999999,"15min":1.5429999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.8640000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.3720000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.2909999999999999,"15min":1.3260000000000001},"min":{"1min":0.72899999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.3460000000000001,"5min":4.492,"15min":4.492},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2509999999999999,"15min":1.2470000000000001},"min":{"1min":0.78500000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":1.3029999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.417,"15min":1.4630000000000001},"min":{"1min":0.79000000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.4770000000000001,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3879999999999999,"15min":1.3959999999999999},"min":{"1min":0.81299999999999994,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.758,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.8240000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5980000000000001,"15min":1.6220000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.952,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.5069999999999999,"15min":1.532},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":2.0910000000000002,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.351}]},{"osd":7,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.647,"15min":1.671},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.149,"5min":3.492,"15min":3.492},"last":1.9339999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.6830000000000001,"15min":1.7330000000000001},"min":{"1min":0.96799999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8859999999999999,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.5569999999999999}]}]}],"pool_statfs":[]}} 2023-10-18T08:57:20.942 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2023-10-18T08:57:20.942 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:14.354858+0000', 'last_change': '2023-10-18T08:57:14.354858+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.354858+0000', 'last_undegraded': '2023-10-18T08:57:14.354858+0000', 'last_fullsized': '2023-10-18T08:57:14.354858+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:57:20.943 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2023-10-18T08:57:20.943 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.944 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2023-10-18T08:57:20.944 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.944 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2023-10-18T08:57:20.945 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.945 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2023-10-18T08:57:20.946 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.946 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2023-10-18T08:57:20.946 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:14.357660+0000', 'last_change': '2023-10-18T08:57:14.357660+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.357660+0000', 'last_undegraded': '2023-10-18T08:57:14.357660+0000', 'last_fullsized': '2023-10-18T08:57:14.357660+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:57:20.947 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2023-10-18T08:57:20.947 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:14.353317+0000', 'last_change': '2023-10-18T08:57:14.353317+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.353317+0000', 'last_undegraded': '2023-10-18T08:57:14.353317+0000', 'last_fullsized': '2023-10-18T08:57:14.353317+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:57:20.947 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2023-10-18T08:57:20.948 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.948 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2023-10-18T08:57:20.949 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.949 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2023-10-18T08:57:20.949 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.950 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2023-10-18T08:57:20.950 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2023-10-18T08:57:20.951 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.951 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2023-10-18T08:57:20.952 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.952 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2023-10-18T08:57:20.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.953 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2023-10-18T08:57:20.953 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:14.362613+0000', 'last_change': '2023-10-18T08:57:14.362613+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.362613+0000', 'last_undegraded': '2023-10-18T08:57:14.362613+0000', 'last_fullsized': '2023-10-18T08:57:14.362613+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:57:20.954 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2023-10-18T08:57:20.954 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.958 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2023-10-18T08:57:20.958 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.959 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2023-10-18T08:57:20.959 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.960 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2023-10-18T08:57:20.960 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2023-10-18T08:57:20.961 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.961 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2023-10-18T08:57:20.962 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:14.356438+0000', 'last_change': '2023-10-18T08:57:14.356438+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.356438+0000', 'last_undegraded': '2023-10-18T08:57:14.356438+0000', 'last_fullsized': '2023-10-18T08:57:14.356438+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:57:20.962 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2023-10-18T08:57:20.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.963 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2023-10-18T08:57:20.963 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2023-10-18T08:57:20.964 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2023-10-18T08:57:20.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2023-10-18T08:57:20.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2023-10-18T08:57:20.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2023-10-18T08:57:20.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2023-10-18T08:57:20.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2023-10-18T08:57:20.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2023-10-18T08:57:20.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:14.355530+0000', 'last_change': '2023-10-18T08:57:14.355530+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.355530+0000', 'last_undegraded': '2023-10-18T08:57:14.355530+0000', 'last_fullsized': '2023-10-18T08:57:14.355530+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:57:20.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2023-10-18T08:57:20.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2023-10-18T08:57:20.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2023-10-18T08:57:20.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2023-10-18T08:57:20.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.344547+0000', 'last_change': '2023-10-18T08:57:13.344547+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.344547+0000', 'last_undegraded': '2023-10-18T08:57:13.344547+0000', 'last_fullsized': '2023-10-18T08:57:13.344547+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T08:57:20.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2023-10-18T08:57:20.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2023-10-18T08:57:20.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.339855+0000', 'last_change': '2023-10-18T08:57:13.339855+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.339855+0000', 'last_undegraded': '2023-10-18T08:57:13.339855+0000', 'last_fullsized': '2023-10-18T08:57:13.339855+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T08:57:20.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2023-10-18T08:57:20.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2023-10-18T08:57:20.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2023-10-18T08:57:20.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2023-10-18T08:57:20.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.344538+0000', 'last_change': '2023-10-18T08:57:13.344538+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.344538+0000', 'last_undegraded': '2023-10-18T08:57:13.344538+0000', 'last_fullsized': '2023-10-18T08:57:13.344538+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T08:57:20.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2023-10-18T08:57:20.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2023-10-18T08:57:20.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.341960+0000', 'last_change': '2023-10-18T08:57:13.341960+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.341960+0000', 'last_undegraded': '2023-10-18T08:57:13.341960+0000', 'last_fullsized': '2023-10-18T08:57:13.341960+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:57:20.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2023-10-18T08:57:20.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2023-10-18T08:57:20.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.344612+0000', 'last_change': '2023-10-18T08:57:13.344612+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.344612+0000', 'last_undegraded': '2023-10-18T08:57:13.344612+0000', 'last_fullsized': '2023-10-18T08:57:13.344612+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:57:20.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2023-10-18T08:57:20.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2023-10-18T08:57:20.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.346128+0000', 'last_change': '2023-10-18T08:57:13.346128+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.346128+0000', 'last_undegraded': '2023-10-18T08:57:13.346128+0000', 'last_fullsized': '2023-10-18T08:57:13.346128+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T08:57:20.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2023-10-18T08:57:20.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:14.352399+0000', 'last_change': '2023-10-18T08:57:14.352399+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.352399+0000', 'last_undegraded': '2023-10-18T08:57:14.352399+0000', 'last_fullsized': '2023-10-18T08:57:14.352399+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:57:20.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2023-10-18T08:57:20.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.344191+0000', 'last_change': '2023-10-18T08:57:13.344191+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.344191+0000', 'last_undegraded': '2023-10-18T08:57:13.344191+0000', 'last_fullsized': '2023-10-18T08:57:13.344191+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:57:20.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2023-10-18T08:57:20.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:14.361992+0000', 'last_change': '2023-10-18T08:57:14.361992+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.361992+0000', 'last_undegraded': '2023-10-18T08:57:14.361992+0000', 'last_fullsized': '2023-10-18T08:57:14.361992+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:57:20.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2023-10-18T08:57:20.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:14.350346+0000', 'last_change': '2023-10-18T08:57:14.350346+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.350346+0000', 'last_undegraded': '2023-10-18T08:57:14.350346+0000', 'last_fullsized': '2023-10-18T08:57:14.350346+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:57:20.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2023-10-18T08:57:20.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.348782+0000', 'last_change': '2023-10-18T08:57:13.348782+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.348782+0000', 'last_undegraded': '2023-10-18T08:57:13.348782+0000', 'last_fullsized': '2023-10-18T08:57:13.348782+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T08:57:20.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2023-10-18T08:57:20.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2023-10-18T08:57:20.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2023-10-18T08:57:20.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:14.354808+0000', 'last_change': '2023-10-18T08:57:14.354808+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.354808+0000', 'last_undegraded': '2023-10-18T08:57:14.354808+0000', 'last_fullsized': '2023-10-18T08:57:14.354808+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:57:20.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2023-10-18T08:57:20.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:14.352501+0000', 'last_change': '2023-10-18T08:57:14.352501+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.352501+0000', 'last_undegraded': '2023-10-18T08:57:14.352501+0000', 'last_fullsized': '2023-10-18T08:57:14.352501+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:57:20.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2023-10-18T08:57:20.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.343633+0000', 'last_change': '2023-10-18T08:57:13.343633+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.343633+0000', 'last_undegraded': '2023-10-18T08:57:13.343633+0000', 'last_fullsized': '2023-10-18T08:57:13.343633+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:57:20.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2023-10-18T08:57:20.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2023-10-18T08:57:20.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.347605+0000', 'last_change': '2023-10-18T08:57:13.347605+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.347605+0000', 'last_undegraded': '2023-10-18T08:57:13.347605+0000', 'last_fullsized': '2023-10-18T08:57:13.347605+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T08:57:20.992 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2023-10-18T08:57:20.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.993 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2023-10-18T08:57:20.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.345028+0000', 'last_change': '2023-10-18T08:57:13.345028+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.345028+0000', 'last_undegraded': '2023-10-18T08:57:13.345028+0000', 'last_fullsized': '2023-10-18T08:57:13.345028+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T08:57:20.994 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2023-10-18T08:57:20.994 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.994 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2023-10-18T08:57:20.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.341352+0000', 'last_change': '2023-10-18T08:57:13.341352+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.341352+0000', 'last_undegraded': '2023-10-18T08:57:13.341352+0000', 'last_fullsized': '2023-10-18T08:57:13.341352+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:57:20.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2023-10-18T08:57:20.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2023-10-18T08:57:20.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:14.354927+0000', 'last_change': '2023-10-18T08:57:14.354927+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.354927+0000', 'last_undegraded': '2023-10-18T08:57:14.354927+0000', 'last_fullsized': '2023-10-18T08:57:14.354927+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:57:20.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2023-10-18T08:57:20.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2023-10-18T08:57:20.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.341122+0000', 'last_change': '2023-10-18T08:57:13.341122+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.341122+0000', 'last_undegraded': '2023-10-18T08:57:13.341122+0000', 'last_fullsized': '2023-10-18T08:57:13.341122+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:57:20.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2023-10-18T08:57:20.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:20.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2023-10-18T08:57:20.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.344125+0000', 'last_change': '2023-10-18T08:57:13.344125+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.344125+0000', 'last_undegraded': '2023-10-18T08:57:13.344125+0000', 'last_fullsized': '2023-10-18T08:57:13.344125+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:57:21.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2023-10-18T08:57:21.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:21.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2023-10-18T08:57:21.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:21.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2023-10-18T08:57:21.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:21.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2023-10-18T08:57:21.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:14.348920+0000', 'last_change': '2023-10-18T08:57:14.348920+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.348920+0000', 'last_undegraded': '2023-10-18T08:57:14.348920+0000', 'last_fullsized': '2023-10-18T08:57:14.348920+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-18T08:57:21.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2023-10-18T08:57:21.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.339494+0000', 'last_change': '2023-10-18T08:57:13.339494+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.339494+0000', 'last_undegraded': '2023-10-18T08:57:13.339494+0000', 'last_fullsized': '2023-10-18T08:57:13.339494+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T08:57:21.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2023-10-18T08:57:21.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:14.357648+0000', 'last_change': '2023-10-18T08:57:14.357648+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.357648+0000', 'last_undegraded': '2023-10-18T08:57:14.357648+0000', 'last_fullsized': '2023-10-18T08:57:14.357648+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:57:21.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2023-10-18T08:57:21.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 44, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:14.354120+0000', 'last_change': '2023-10-18T08:57:14.354120+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.354120+0000', 'last_undegraded': '2023-10-18T08:57:14.354120+0000', 'last_fullsized': '2023-10-18T08:57:14.354120+0000', 'mapping_epoch': 44, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 44, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-18T08:57:21.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2023-10-18T08:57:21.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.345024+0000', 'last_change': '2023-10-18T08:57:13.345024+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.345024+0000', 'last_undegraded': '2023-10-18T08:57:13.345024+0000', 'last_fullsized': '2023-10-18T08:57:13.345024+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T08:57:21.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2023-10-18T08:57:21.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:21.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2023-10-18T08:57:21.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.339514+0000', 'last_change': '2023-10-18T08:57:13.339514+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.339514+0000', 'last_undegraded': '2023-10-18T08:57:13.339514+0000', 'last_fullsized': '2023-10-18T08:57:13.339514+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T08:57:21.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2023-10-18T08:57:21.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:21.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2023-10-18T08:57:21.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.343518+0000', 'last_change': '2023-10-18T08:57:13.343518+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.343518+0000', 'last_undegraded': '2023-10-18T08:57:13.343518+0000', 'last_fullsized': '2023-10-18T08:57:13.343518+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:57:21.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2023-10-18T08:57:21.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-18T08:57:14.334221+0000', 'last_change': '2023-10-18T08:57:14.334221+0000', 'last_active': '2023-10-18T08:57:14.334221+0000', 'last_peered': '2023-10-18T08:57:14.334221+0000', 'last_clean': '2023-10-18T08:57:14.334221+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:14.334221+0000', 'last_undegraded': '2023-10-18T08:57:14.334221+0000', 'last_fullsized': '2023-10-18T08:57:14.334221+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:14.334221+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-18T08:57:21.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2023-10-18T08:57:21.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:57:13.341302+0000', 'last_change': '2023-10-18T08:57:13.341302+0000', 'last_active': '2023-10-18T08:57:13.322296+0000', 'last_peered': '2023-10-18T08:57:13.322296+0000', 'last_clean': '2023-10-18T08:57:13.322296+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:57:13.341302+0000', 'last_undegraded': '2023-10-18T08:57:13.341302+0000', 'last_fullsized': '2023-10-18T08:57:13.341302+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'last_clean_scrub_stamp': '2023-10-18T08:57:13.322296+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:57:21.011 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T08:57:21.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.249+0000 7fd6bb59e640 1 -- 172.21.15.40:0/594916352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6bc15a990 msgr2=0x7fd6bc15a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:21.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.249+0000 7fd6bb59e640 1 --2- 172.21.15.40:0/594916352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6bc15a990 0x7fd6bc15a350 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009830 tx=0x7fd6a402bcd0 comp rx=0 tx=0).stop 2023-10-18T08:57:21.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.250+0000 7fd6bb59e640 1 -- 172.21.15.40:0/594916352 shutdown_connections 2023-10-18T08:57:21.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.250+0000 7fd6bb59e640 1 --2- 172.21.15.40:0/594916352 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6bc0b0070 0x7fd6bc06b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:21.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.250+0000 7fd6bb59e640 1 --2- 172.21.15.40:0/594916352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6bc15a990 0x7fd6bc15a350 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:21.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.250+0000 7fd6bb59e640 1 -- 172.21.15.40:0/594916352 >> 172.21.15.40:0/594916352 conn(0x7fd6bc0aed70 msgr2=0x7fd6bc0af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:21.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.250+0000 7fd6bb59e640 1 -- 172.21.15.40:0/594916352 shutdown_connections 2023-10-18T08:57:21.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.250+0000 7fd6bb59e640 1 -- 172.21.15.40:0/594916352 wait complete. 2023-10-18T08:57:21.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.252+0000 7fd6bb59e640 1 Processor -- start 2023-10-18T08:57:21.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.252+0000 7fd6bb59e640 1 -- start start 2023-10-18T08:57:21.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.253+0000 7fd6bb59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6bc0b0070 0x7fd6bc14cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:21.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.253+0000 7fd6bb59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6bc0b0dd0 0x7fd6bc158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:21.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.253+0000 7fd6bb59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6bc15a990 0x7fd6bc14a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:21.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.253+0000 7fd6bb59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc13b670 con 0x7fd6bc0b0070 2023-10-18T08:57:21.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.253+0000 7fd6bb59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc13a780 con 0x7fd6bc0b0dd0 2023-10-18T08:57:21.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.253+0000 7fd6bb59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc13aa60 con 0x7fd6bc15a990 2023-10-18T08:57:21.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.253+0000 7fd6ba59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6bc0b0070 0x7fd6bc14cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:21.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.253+0000 7fd6ba59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6bc0b0070 0x7fd6bc14cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54156/0 (socket says 172.21.15.40:54156) 2023-10-18T08:57:21.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.253+0000 7fd6ba59c640 1 -- 172.21.15.40:0/640856085 learned_addr learned my addr 172.21.15.40:0/640856085 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:21.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.253+0000 7fd6b9d9b640 1 --2- 172.21.15.40:0/640856085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6bc0b0dd0 0x7fd6bc158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:21.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.253+0000 7fd6bad9d640 1 --2- 172.21.15.40:0/640856085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6bc15a990 0x7fd6bc14a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:21.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.253+0000 7fd6ba59c640 1 -- 172.21.15.40:0/640856085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6bc15a990 msgr2=0x7fd6bc14a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:21.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.253+0000 7fd6ba59c640 1 --2- 172.21.15.40:0/640856085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6bc15a990 0x7fd6bc14a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:21.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.253+0000 7fd6ba59c640 1 -- 172.21.15.40:0/640856085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6bc0b0dd0 msgr2=0x7fd6bc158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:21.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.253+0000 7fd6ba59c640 1 --2- 172.21.15.40:0/640856085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6bc0b0dd0 0x7fd6bc158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:21.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.254+0000 7fd6ba59c640 1 -- 172.21.15.40:0/640856085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a4009490 con 0x7fd6bc0b0070 2023-10-18T08:57:21.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.254+0000 7fd6ba59c640 1 --2- 172.21.15.40:0/640856085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6bc0b0070 0x7fd6bc14cc60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00cc50 tx=0x7fd6ac00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:21.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.254+0000 7fd6b2ffd640 1 -- 172.21.15.40:0/640856085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac00ce60 con 0x7fd6bc0b0070 2023-10-18T08:57:21.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.254+0000 7fd6b2ffd640 1 -- 172.21.15.40:0/640856085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd6ac012420 con 0x7fd6bc0b0070 2023-10-18T08:57:21.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.254+0000 7fd6b2ffd640 1 -- 172.21.15.40:0/640856085 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac012810 con 0x7fd6bc0b0070 2023-10-18T08:57:21.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.254+0000 7fd6bb59e640 1 -- 172.21.15.40:0/640856085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6bc13ad40 con 0x7fd6bc0b0070 2023-10-18T08:57:21.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.254+0000 7fd6bb59e640 1 -- 172.21.15.40:0/640856085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6bc13b300 con 0x7fd6bc0b0070 2023-10-18T08:57:21.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.255+0000 7fd6bb59e640 1 -- 172.21.15.40:0/640856085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6bc066e00 con 0x7fd6bc0b0070 2023-10-18T08:57:21.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.256+0000 7fd6b2ffd640 1 -- 172.21.15.40:0/640856085 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd6ac01a3b0 con 0x7fd6bc0b0070 2023-10-18T08:57:21.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.257+0000 7fd6b2ffd640 1 --2- 172.21.15.40:0/640856085 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd69c06ae40 0x7fd69c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:21.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.257+0000 7fd6b2ffd640 1 -- 172.21.15.40:0/640856085 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6ac089f60 con 0x7fd6bc0b0070 2023-10-18T08:57:21.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.258+0000 7fd6b9d9b640 1 --2- 172.21.15.40:0/640856085 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd69c06ae40 0x7fd69c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:21.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.259+0000 7fd6b9d9b640 1 --2- 172.21.15.40:0/640856085 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd69c06ae40 0x7fd69c06d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4036000 tx=0x7fd6a4004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:21.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.262+0000 7fd6b2ffd640 1 -- 172.21.15.40:0/640856085 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6ac054f70 con 0x7fd6bc0b0070 2023-10-18T08:57:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.524+0000 7fd6bb59e640 1 -- 172.21.15.40:0/640856085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd6bc159890 con 0x7fd6bc0b0070 2023-10-18T08:57:21.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.525+0000 7fd6b2ffd640 1 -- 172.21.15.40:0/640856085 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1521 (secure 0 0 0) 0x7fd6ac016610 con 0x7fd6bc0b0070 2023-10-18T08:57:21.528 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:21.528 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":145,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":46,"num_osds":8,"num_up_osds":8,"osd_up_since":1697619304,"num_in_osds":8,"osd_in_since":1697619300,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":49},{"state_name":"creating+peering","count":34},{"state_name":"active+clean","count":14}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2798067,"bytes_avail":1597201933,"bytes_total":1600000000,"unknown_pgs_ratio":0.50515460968017578,"inactive_pgs_ratio":0.3505154550075531},"fsmap":{"epoch":24,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"b","status":"up:active","gid":5655}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2023-10-18T08:57:00.932481+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-18T08:57:21.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.528+0000 7fd6bb59e640 1 -- 172.21.15.40:0/640856085 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd69c06ae40 msgr2=0x7fd69c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:21.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.528+0000 7fd6bb59e640 1 --2- 172.21.15.40:0/640856085 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd69c06ae40 0x7fd69c06d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4036000 tx=0x7fd6a4004730 comp rx=0 tx=0).stop 2023-10-18T08:57:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.528+0000 7fd6bb59e640 1 -- 172.21.15.40:0/640856085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6bc0b0070 msgr2=0x7fd6bc14cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:21.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.528+0000 7fd6bb59e640 1 --2- 172.21.15.40:0/640856085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6bc0b0070 0x7fd6bc14cc60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00cc50 tx=0x7fd6ac00a430 comp rx=0 tx=0).stop 2023-10-18T08:57:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.529+0000 7fd6bb59e640 1 -- 172.21.15.40:0/640856085 shutdown_connections 2023-10-18T08:57:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.529+0000 7fd6bb59e640 1 --2- 172.21.15.40:0/640856085 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd69c06ae40 0x7fd69c06d300 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.529+0000 7fd6bb59e640 1 --2- 172.21.15.40:0/640856085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6bc15a990 0x7fd6bc14a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.529+0000 7fd6bb59e640 1 --2- 172.21.15.40:0/640856085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6bc0b0dd0 0x7fd6bc158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:21.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.529+0000 7fd6bb59e640 1 --2- 172.21.15.40:0/640856085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6bc0b0070 0x7fd6bc14cc60 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.529+0000 7fd6bb59e640 1 -- 172.21.15.40:0/640856085 >> 172.21.15.40:0/640856085 conn(0x7fd6bc0aed70 msgr2=0x7fd6bc0b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.529+0000 7fd6bb59e640 1 -- 172.21.15.40:0/640856085 shutdown_connections 2023-10-18T08:57:21.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.529+0000 7fd6bb59e640 1 -- 172.21.15.40:0/640856085 wait complete. 2023-10-18T08:57:21.551 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T08:57:21.552 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:57:21.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.788+0000 7f9075142640 1 -- 172.21.15.40:0/2158065548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f905c0047c0 msgr2=0x7f905c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:21.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.788+0000 7f9075142640 1 --2- 172.21.15.40:0/2158065548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f905c0047c0 0x7f905c004c40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f905800bb20 tx=0x7f905c0057f0 comp rx=0 tx=0).stop 2023-10-18T08:57:21.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.788+0000 7f9074941640 1 -- 172.21.15.40:0/2158065548 reap_dead start 2023-10-18T08:57:21.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.788+0000 7f9075142640 1 -- 172.21.15.40:0/2158065548 shutdown_connections 2023-10-18T08:57:21.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.788+0000 7f9075142640 1 -- 172.21.15.40:0/2158065548 >> 172.21.15.40:0/2158065548 conn(0x7f9070069720 msgr2=0x7f9070069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:21.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.788+0000 7f9075142640 1 -- 172.21.15.40:0/2158065548 shutdown_connections 2023-10-18T08:57:21.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.788+0000 7f9075142640 1 -- 172.21.15.40:0/2158065548 wait complete. 2023-10-18T08:57:21.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.790+0000 7f9075142640 1 Processor -- start 2023-10-18T08:57:21.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.790+0000 7f9075142640 1 -- start start 2023-10-18T08:57:21.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.791+0000 7f9075142640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f905c0047c0 0x7f90700c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.791+0000 7f9075142640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90700ac970 0x7f907015ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.791+0000 7f9075142640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907015b140 0x7f90700adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.791+0000 7f9075142640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90700ad910 con 0x7f907015b140 2023-10-18T08:57:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.791+0000 7f9075142640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90700ad350 con 0x7f905c0047c0 2023-10-18T08:57:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.791+0000 7f9075142640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90700ad630 con 0x7f90700ac970 2023-10-18T08:57:21.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.791+0000 7f9074941640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907015b140 0x7f90700adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:21.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.792+0000 7f9074941640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907015b140 0x7f90700adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54174/0 (socket says 172.21.15.40:54174) 2023-10-18T08:57:21.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.792+0000 7f9074941640 1 -- 172.21.15.40:0/436648499 learned_addr learned my addr 172.21.15.40:0/436648499 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:21.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.792+0000 7f9074941640 1 -- 172.21.15.40:0/436648499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90700ac970 msgr2=0x7f907015ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:21.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.792+0000 7f9074941640 1 --2- 172.21.15.40:0/436648499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90700ac970 0x7f907015ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:21.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.792+0000 7f9074941640 1 -- 172.21.15.40:0/436648499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f905c0047c0 msgr2=0x7f90700c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:21.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.792+0000 7f906ffff640 1 --2- 172.21.15.40:0/436648499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f905c0047c0 0x7f90700c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:21.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.792+0000 7f9074941640 1 --2- 172.21.15.40:0/436648499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f905c0047c0 0x7f90700c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:21.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.792+0000 7f9074941640 1 -- 172.21.15.40:0/436648499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907015a400 con 0x7f907015b140 2023-10-18T08:57:21.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.792+0000 7f9074941640 1 --2- 172.21.15.40:0/436648499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907015b140 0x7f90700adf50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f907006f590 tx=0x7f906800e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:21.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.792+0000 7f906d7fa640 1 -- 172.21.15.40:0/436648499 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906800c070 con 0x7f907015b140 2023-10-18T08:57:21.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.793+0000 7f906d7fa640 1 -- 172.21.15.40:0/436648499 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9068013610 con 0x7f907015b140 2023-10-18T08:57:21.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.793+0000 7f906d7fa640 1 -- 172.21.15.40:0/436648499 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068013a20 con 0x7f907015b140 2023-10-18T08:57:21.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.793+0000 7f9075142640 1 -- 172.21.15.40:0/436648499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907015a6e0 con 0x7f907015b140 2023-10-18T08:57:21.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.793+0000 7f9075142640 1 -- 172.21.15.40:0/436648499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90701f1b70 con 0x7f907015b140 2023-10-18T08:57:21.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.794+0000 7f9075142640 1 -- 172.21.15.40:0/436648499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907015a8d0 con 0x7f907015b140 2023-10-18T08:57:21.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.795+0000 7f906d7fa640 1 -- 172.21.15.40:0/436648499 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f906801c3b0 con 0x7f907015b140 2023-10-18T08:57:21.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.796+0000 7f906d7fa640 1 --2- 172.21.15.40:0/436648499 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f905006ae70 0x7f905006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:21.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.796+0000 7f906d7fa640 1 -- 172.21.15.40:0/436648499 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9068088b00 con 0x7f907015b140 2023-10-18T08:57:21.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.796+0000 7f906ffff640 1 --2- 172.21.15.40:0/436648499 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f905006ae70 0x7f905006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:21.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.798+0000 7f906ffff640 1 --2- 172.21.15.40:0/436648499 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f905006ae70 0x7f905006d330 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9060001050 tx=0x7f9060001d90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:21.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.800+0000 7f906d7fa640 1 -- 172.21.15.40:0/436648499 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f906808ae60 con 0x7f907015b140 2023-10-18T08:57:21.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:21.994+0000 7f9075142640 1 -- 172.21.15.40:0/436648499 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90700ad7a0 con 0x7f905006ae70 2023-10-18T08:57:22.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:22.017+0000 7f906d7fa640 1 -- 172.21.15.40:0/436648499 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282385 (secure 0 0 0) 0x7f90700ad7a0 con 0x7f905006ae70 2023-10-18T08:57:22.020 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:22.025 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:57:22.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:22.023+0000 7f9075142640 1 -- 172.21.15.40:0/436648499 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f905006ae70 msgr2=0x7f905006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:22.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:22.023+0000 7f9075142640 1 --2- 172.21.15.40:0/436648499 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f905006ae70 0x7f905006d330 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9060001050 tx=0x7f9060001d90 comp rx=0 tx=0).stop 2023-10-18T08:57:22.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:22.023+0000 7f9075142640 1 -- 172.21.15.40:0/436648499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907015b140 msgr2=0x7f90700adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:22.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:22.023+0000 7f9075142640 1 --2- 172.21.15.40:0/436648499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907015b140 0x7f90700adf50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f907006f590 tx=0x7f906800e9f0 comp rx=0 tx=0).stop 2023-10-18T08:57:22.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:22.023+0000 7f9075142640 1 -- 172.21.15.40:0/436648499 shutdown_connections 2023-10-18T08:57:22.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:22.023+0000 7f9075142640 1 --2- 172.21.15.40:0/436648499 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f905006ae70 0x7f905006d330 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:22.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:22.023+0000 7f9075142640 1 --2- 172.21.15.40:0/436648499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f907015b140 0x7f90700adf50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:22.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:22.023+0000 7f9075142640 1 --2- 172.21.15.40:0/436648499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90700ac970 0x7f907015ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:22.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:22.023+0000 7f9075142640 1 --2- 172.21.15.40:0/436648499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f905c0047c0 0x7f90700c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:22.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:22.023+0000 7f9075142640 1 -- 172.21.15.40:0/436648499 >> 172.21.15.40:0/436648499 conn(0x7f9070069720 msgr2=0x7f907006f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:22.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:22.023+0000 7f9075142640 1 -- 172.21.15.40:0/436648499 shutdown_connections 2023-10-18T08:57:22.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:22.023+0000 7f9075142640 1 -- 172.21.15.40:0/436648499 wait complete. 2023-10-18T08:57:22.047 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2023-10-18T08:57:20.938568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2887,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559612,"statfs":{"total":1600000000,"available":1597043277,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.593641"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.374986+0000","last_change":"2023-10-18T08:57:15.389269+0000","last_active":"2023-10-18T08:57:16.374986+0000","last_peered":"2023-10-18T08:57:16.374986+0000","last_clean":"2023-10-18T08:57:16.374986+0000","last_became_active":"2023-10-18T08:57:15.388586+0000","last_became_peered":"2023-10-18T08:57:15.388586+0000","last_unstale":"2023-10-18T08:57:16.374986+0000","last_undegraded":"2023-10-18T08:57:16.374986+0000","last_fullsized":"2023-10-18T08:57:16.374986+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:10.615157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.090744+0000","last_change":"2023-10-18T08:57:15.403603+0000","last_active":"2023-10-18T08:57:18.090744+0000","last_peered":"2023-10-18T08:57:18.090744+0000","last_clean":"2023-10-18T08:57:18.090744+0000","last_became_active":"2023-10-18T08:57:15.403352+0000","last_became_peered":"2023-10-18T08:57:15.403352+0000","last_unstale":"2023-10-18T08:57:18.090744+0000","last_undegraded":"2023-10-18T08:57:18.090744+0000","last_fullsized":"2023-10-18T08:57:18.090744+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:28.288612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.375196+0000","last_change":"2023-10-18T08:57:15.400440+0000","last_active":"2023-10-18T08:57:16.375196+0000","last_peered":"2023-10-18T08:57:16.375196+0000","last_clean":"2023-10-18T08:57:16.375196+0000","last_became_active":"2023-10-18T08:57:15.399898+0000","last_became_peered":"2023-10-18T08:57:15.399898+0000","last_unstale":"2023-10-18T08:57:16.375196+0000","last_undegraded":"2023-10-18T08:57:16.375196+0000","last_fullsized":"2023-10-18T08:57:16.375196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:16.125626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.301864+0000","last_change":"2023-10-18T08:57:15.404432+0000","last_active":"2023-10-18T08:57:17.301864+0000","last_peered":"2023-10-18T08:57:17.301864+0000","last_clean":"2023-10-18T08:57:17.301864+0000","last_became_active":"2023-10-18T08:57:15.404044+0000","last_became_peered":"2023-10-18T08:57:15.404044+0000","last_unstale":"2023-10-18T08:57:17.301864+0000","last_undegraded":"2023-10-18T08:57:17.301864+0000","last_fullsized":"2023-10-18T08:57:17.301864+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:31.543070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.091161+0000","last_change":"2023-10-18T08:57:15.408728+0000","last_active":"2023-10-18T08:57:18.091161+0000","last_peered":"2023-10-18T08:57:18.091161+0000","last_clean":"2023-10-18T08:57:18.091161+0000","last_became_active":"2023-10-18T08:57:15.408457+0000","last_became_peered":"2023-10-18T08:57:15.408457+0000","last_unstale":"2023-10-18T08:57:18.091161+0000","last_undegraded":"2023-10-18T08:57:18.091161+0000","last_fullsized":"2023-10-18T08:57:18.091161+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:22.712989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.373009+0000","last_change":"2023-10-18T08:57:15.379923+0000","last_active":"2023-10-18T08:57:16.373009+0000","last_peered":"2023-10-18T08:57:16.373009+0000","last_clean":"2023-10-18T08:57:16.373009+0000","last_became_active":"2023-10-18T08:57:15.378979+0000","last_became_peered":"2023-10-18T08:57:15.378979+0000","last_unstale":"2023-10-18T08:57:16.373009+0000","last_undegraded":"2023-10-18T08:57:16.373009+0000","last_fullsized":"2023-10-18T08:57:16.373009+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:29:20.959175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.374306+0000","last_change":"2023-10-18T08:57:15.401109+0000","last_active":"2023-10-18T08:57:16.374306+0000","last_peered":"2023-10-18T08:57:16.374306+0000","last_clean":"2023-10-18T08:57:16.374306+0000","last_became_active":"2023-10-18T08:57:15.400850+0000","last_became_peered":"2023-10-18T08:57:15.400850+0000","last_unstale":"2023-10-18T08:57:16.374306+0000","last_undegraded":"2023-10-18T08:57:16.374306+0000","last_fullsized":"2023-10-18T08:57:16.374306+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:42.529724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.301895+0000","last_change":"2023-10-18T08:57:15.404043+0000","last_active":"2023-10-18T08:57:17.301895+0000","last_peered":"2023-10-18T08:57:17.301895+0000","last_clean":"2023-10-18T08:57:17.301895+0000","last_became_active":"2023-10-18T08:57:15.403796+0000","last_became_peered":"2023-10-18T08:57:15.403796+0000","last_unstale":"2023-10-18T08:57:17.301895+0000","last_undegraded":"2023-10-18T08:57:17.301895+0000","last_fullsized":"2023-10-18T08:57:17.301895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:19.959565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302230+0000","last_change":"2023-10-18T08:57:15.404275+0000","last_active":"2023-10-18T08:57:17.302230+0000","last_peered":"2023-10-18T08:57:17.302230+0000","last_clean":"2023-10-18T08:57:17.302230+0000","last_became_active":"2023-10-18T08:57:15.403525+0000","last_became_peered":"2023-10-18T08:57:15.403525+0000","last_unstale":"2023-10-18T08:57:17.302230+0000","last_undegraded":"2023-10-18T08:57:17.302230+0000","last_fullsized":"2023-10-18T08:57:17.302230+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:34.381463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.301895+0000","last_change":"2023-10-18T08:57:15.410588+0000","last_active":"2023-10-18T08:57:17.301895+0000","last_peered":"2023-10-18T08:57:17.301895+0000","last_clean":"2023-10-18T08:57:17.301895+0000","last_became_active":"2023-10-18T08:57:15.410362+0000","last_became_peered":"2023-10-18T08:57:15.410362+0000","last_unstale":"2023-10-18T08:57:17.301895+0000","last_undegraded":"2023-10-18T08:57:17.301895+0000","last_fullsized":"2023-10-18T08:57:17.301895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:01.633676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302286+0000","last_change":"2023-10-18T08:57:15.400199+0000","last_active":"2023-10-18T08:57:17.302286+0000","last_peered":"2023-10-18T08:57:17.302286+0000","last_clean":"2023-10-18T08:57:17.302286+0000","last_became_active":"2023-10-18T08:57:15.399889+0000","last_became_peered":"2023-10-18T08:57:15.399889+0000","last_unstale":"2023-10-18T08:57:17.302286+0000","last_undegraded":"2023-10-18T08:57:17.302286+0000","last_fullsized":"2023-10-18T08:57:17.302286+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:05:08.684704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.090952+0000","last_change":"2023-10-18T08:57:15.398078+0000","last_active":"2023-10-18T08:57:18.090952+0000","last_peered":"2023-10-18T08:57:18.090952+0000","last_clean":"2023-10-18T08:57:18.090952+0000","last_became_active":"2023-10-18T08:57:15.397874+0000","last_became_peered":"2023-10-18T08:57:15.397874+0000","last_unstale":"2023-10-18T08:57:18.090952+0000","last_undegraded":"2023-10-18T08:57:18.090952+0000","last_fullsized":"2023-10-18T08:57:18.090952+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:05:08.684704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.091291+0000","last_change":"2023-10-18T08:57:15.405598+0000","last_active":"2023-10-18T08:57:18.091291+0000","last_peered":"2023-10-18T08:57:18.091291+0000","last_clean":"2023-10-18T08:57:18.091291+0000","last_became_active":"2023-10-18T08:57:15.405278+0000","last_became_peered":"2023-10-18T08:57:15.405278+0000","last_unstale":"2023-10-18T08:57:18.091291+0000","last_undegraded":"2023-10-18T08:57:18.091291+0000","last_fullsized":"2023-10-18T08:57:18.091291+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:19:38.945359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.303220+0000","last_change":"2023-10-18T08:57:15.403771+0000","last_active":"2023-10-18T08:57:17.303220+0000","last_peered":"2023-10-18T08:57:17.303220+0000","last_clean":"2023-10-18T08:57:17.303220+0000","last_became_active":"2023-10-18T08:57:15.402907+0000","last_became_peered":"2023-10-18T08:57:15.402907+0000","last_unstale":"2023-10-18T08:57:17.303220+0000","last_undegraded":"2023-10-18T08:57:17.303220+0000","last_fullsized":"2023-10-18T08:57:17.303220+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:05:27.740292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.372706+0000","last_change":"2023-10-18T08:57:15.390844+0000","last_active":"2023-10-18T08:57:16.372706+0000","last_peered":"2023-10-18T08:57:16.372706+0000","last_clean":"2023-10-18T08:57:16.372706+0000","last_became_active":"2023-10-18T08:57:15.390204+0000","last_became_peered":"2023-10-18T08:57:15.390204+0000","last_unstale":"2023-10-18T08:57:16.372706+0000","last_undegraded":"2023-10-18T08:57:16.372706+0000","last_fullsized":"2023-10-18T08:57:16.372706+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:06.663782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.184671+0000","last_change":"2023-10-18T08:57:15.406162+0000","last_active":"2023-10-18T08:57:17.184671+0000","last_peered":"2023-10-18T08:57:17.184671+0000","last_clean":"2023-10-18T08:57:17.184671+0000","last_became_active":"2023-10-18T08:57:15.405881+0000","last_became_peered":"2023-10-18T08:57:15.405881+0000","last_unstale":"2023-10-18T08:57:17.184671+0000","last_undegraded":"2023-10-18T08:57:17.184671+0000","last_fullsized":"2023-10-18T08:57:17.184671+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:06.363154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.376820+0000","last_change":"2023-10-18T08:57:15.399690+0000","last_active":"2023-10-18T08:57:16.376820+0000","last_peered":"2023-10-18T08:57:16.376820+0000","last_clean":"2023-10-18T08:57:16.376820+0000","last_became_active":"2023-10-18T08:57:15.397149+0000","last_became_peered":"2023-10-18T08:57:15.397149+0000","last_unstale":"2023-10-18T08:57:16.376820+0000","last_undegraded":"2023-10-18T08:57:16.376820+0000","last_fullsized":"2023-10-18T08:57:16.376820+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:22.712989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.376683+0000","last_change":"2023-10-18T08:57:15.400280+0000","last_active":"2023-10-18T08:57:16.376683+0000","last_peered":"2023-10-18T08:57:16.376683+0000","last_clean":"2023-10-18T08:57:16.376683+0000","last_became_active":"2023-10-18T08:57:15.399540+0000","last_became_peered":"2023-10-18T08:57:15.399540+0000","last_unstale":"2023-10-18T08:57:16.376683+0000","last_undegraded":"2023-10-18T08:57:16.376683+0000","last_fullsized":"2023-10-18T08:57:16.376683+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:05:08.684704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.185536+0000","last_change":"2023-10-18T08:57:15.406156+0000","last_active":"2023-10-18T08:57:17.185536+0000","last_peered":"2023-10-18T08:57:17.185536+0000","last_clean":"2023-10-18T08:57:17.185536+0000","last_became_active":"2023-10-18T08:57:15.405706+0000","last_became_peered":"2023-10-18T08:57:15.405706+0000","last_unstale":"2023-10-18T08:57:17.185536+0000","last_undegraded":"2023-10-18T08:57:17.185536+0000","last_fullsized":"2023-10-18T08:57:17.185536+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:08.507146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.090762+0000","last_change":"2023-10-18T08:57:15.408499+0000","last_active":"2023-10-18T08:57:18.090762+0000","last_peered":"2023-10-18T08:57:18.090762+0000","last_clean":"2023-10-18T08:57:18.090762+0000","last_became_active":"2023-10-18T08:57:15.408271+0000","last_became_peered":"2023-10-18T08:57:15.408271+0000","last_unstale":"2023-10-18T08:57:18.090762+0000","last_undegraded":"2023-10-18T08:57:18.090762+0000","last_fullsized":"2023-10-18T08:57:18.090762+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:40:23.738176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.373281+0000","last_change":"2023-10-18T08:57:15.400776+0000","last_active":"2023-10-18T08:57:16.373281+0000","last_peered":"2023-10-18T08:57:16.373281+0000","last_clean":"2023-10-18T08:57:16.373281+0000","last_became_active":"2023-10-18T08:57:15.400519+0000","last_became_peered":"2023-10-18T08:57:15.400519+0000","last_unstale":"2023-10-18T08:57:16.373281+0000","last_undegraded":"2023-10-18T08:57:16.373281+0000","last_fullsized":"2023-10-18T08:57:16.373281+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:02.402794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302344+0000","last_change":"2023-10-18T08:57:15.409255+0000","last_active":"2023-10-18T08:57:17.302344+0000","last_peered":"2023-10-18T08:57:17.302344+0000","last_clean":"2023-10-18T08:57:17.302344+0000","last_became_active":"2023-10-18T08:57:15.408910+0000","last_became_peered":"2023-10-18T08:57:15.408910+0000","last_unstale":"2023-10-18T08:57:17.302344+0000","last_undegraded":"2023-10-18T08:57:17.302344+0000","last_fullsized":"2023-10-18T08:57:17.302344+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:22.712989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.185285+0000","last_change":"2023-10-18T08:57:15.403446+0000","last_active":"2023-10-18T08:57:17.185285+0000","last_peered":"2023-10-18T08:57:17.185285+0000","last_clean":"2023-10-18T08:57:17.185285+0000","last_became_active":"2023-10-18T08:57:15.403176+0000","last_became_peered":"2023-10-18T08:57:15.403176+0000","last_unstale":"2023-10-18T08:57:17.185285+0000","last_undegraded":"2023-10-18T08:57:17.185285+0000","last_fullsized":"2023-10-18T08:57:17.185285+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:09.244351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.184459+0000","last_change":"2023-10-18T08:57:15.405352+0000","last_active":"2023-10-18T08:57:17.184459+0000","last_peered":"2023-10-18T08:57:17.184459+0000","last_clean":"2023-10-18T08:57:17.184459+0000","last_became_active":"2023-10-18T08:57:15.404942+0000","last_became_peered":"2023-10-18T08:57:15.404942+0000","last_unstale":"2023-10-18T08:57:17.184459+0000","last_undegraded":"2023-10-18T08:57:17.184459+0000","last_fullsized":"2023-10-18T08:57:17.184459+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:42:22.676749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.184750+0000","last_change":"2023-10-18T08:57:15.404895+0000","last_active":"2023-10-18T08:57:17.184750+0000","last_peered":"2023-10-18T08:57:17.184750+0000","last_clean":"2023-10-18T08:57:17.184750+0000","last_became_active":"2023-10-18T08:57:15.403675+0000","last_became_peered":"2023-10-18T08:57:15.403675+0000","last_unstale":"2023-10-18T08:57:17.184750+0000","last_undegraded":"2023-10-18T08:57:17.184750+0000","last_fullsized":"2023-10-18T08:57:17.184750+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:26:10.073233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.091374+0000","last_change":"2023-10-18T08:57:15.395124+0000","last_active":"2023-10-18T08:57:18.091374+0000","last_peered":"2023-10-18T08:57:18.091374+0000","last_clean":"2023-10-18T08:57:18.091374+0000","last_became_active":"2023-10-18T08:57:15.394746+0000","last_became_peered":"2023-10-18T08:57:15.394746+0000","last_unstale":"2023-10-18T08:57:18.091374+0000","last_undegraded":"2023-10-18T08:57:18.091374+0000","last_fullsized":"2023-10-18T08:57:18.091374+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:14.564913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.377083+0000","last_change":"2023-10-18T08:57:15.393582+0000","last_active":"2023-10-18T08:57:16.377083+0000","last_peered":"2023-10-18T08:57:16.377083+0000","last_clean":"2023-10-18T08:57:16.377083+0000","last_became_active":"2023-10-18T08:57:15.393072+0000","last_became_peered":"2023-10-18T08:57:15.393072+0000","last_unstale":"2023-10-18T08:57:16.377083+0000","last_undegraded":"2023-10-18T08:57:16.377083+0000","last_fullsized":"2023-10-18T08:57:16.377083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:29:38.697110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.091200+0000","last_change":"2023-10-18T08:57:15.399109+0000","last_active":"2023-10-18T08:57:18.091200+0000","last_peered":"2023-10-18T08:57:18.091200+0000","last_clean":"2023-10-18T08:57:18.091200+0000","last_became_active":"2023-10-18T08:57:15.397962+0000","last_became_peered":"2023-10-18T08:57:15.397962+0000","last_unstale":"2023-10-18T08:57:18.091200+0000","last_undegraded":"2023-10-18T08:57:18.091200+0000","last_fullsized":"2023-10-18T08:57:18.091200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:42.529724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.184883+0000","last_change":"2023-10-18T08:57:15.404013+0000","last_active":"2023-10-18T08:57:17.184883+0000","last_peered":"2023-10-18T08:57:17.184883+0000","last_clean":"2023-10-18T08:57:17.184883+0000","last_became_active":"2023-10-18T08:57:15.402116+0000","last_became_peered":"2023-10-18T08:57:15.402116+0000","last_unstale":"2023-10-18T08:57:17.184883+0000","last_undegraded":"2023-10-18T08:57:17.184883+0000","last_fullsized":"2023-10-18T08:57:17.184883+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:36.109411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.376966+0000","last_change":"2023-10-18T08:57:15.393254+0000","last_active":"2023-10-18T08:57:16.376966+0000","last_peered":"2023-10-18T08:57:16.376966+0000","last_clean":"2023-10-18T08:57:16.376966+0000","last_became_active":"2023-10-18T08:57:15.391546+0000","last_became_peered":"2023-10-18T08:57:15.391546+0000","last_unstale":"2023-10-18T08:57:16.376966+0000","last_undegraded":"2023-10-18T08:57:16.376966+0000","last_fullsized":"2023-10-18T08:57:16.376966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:42.529724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.373439+0000","last_change":"2023-10-18T08:57:15.378256+0000","last_active":"2023-10-18T08:57:16.373439+0000","last_peered":"2023-10-18T08:57:16.373439+0000","last_clean":"2023-10-18T08:57:16.373439+0000","last_became_active":"2023-10-18T08:57:15.377276+0000","last_became_peered":"2023-10-18T08:57:15.377276+0000","last_unstale":"2023-10-18T08:57:16.373439+0000","last_undegraded":"2023-10-18T08:57:16.373439+0000","last_fullsized":"2023-10-18T08:57:16.373439+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:36:22.196893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.091016+0000","last_change":"2023-10-18T08:57:15.400776+0000","last_active":"2023-10-18T08:57:18.091016+0000","last_peered":"2023-10-18T08:57:18.091016+0000","last_clean":"2023-10-18T08:57:18.091016+0000","last_became_active":"2023-10-18T08:57:15.400464+0000","last_became_peered":"2023-10-18T08:57:15.400464+0000","last_unstale":"2023-10-18T08:57:18.091016+0000","last_undegraded":"2023-10-18T08:57:18.091016+0000","last_fullsized":"2023-10-18T08:57:18.091016+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:15:31.459376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.409122+0000","last_change":"2023-10-18T08:57:15.401657+0000","last_active":"2023-10-18T08:57:16.409122+0000","last_peered":"2023-10-18T08:57:16.409122+0000","last_clean":"2023-10-18T08:57:16.409122+0000","last_became_active":"2023-10-18T08:57:15.401392+0000","last_became_peered":"2023-10-18T08:57:15.401392+0000","last_unstale":"2023-10-18T08:57:16.409122+0000","last_undegraded":"2023-10-18T08:57:16.409122+0000","last_fullsized":"2023-10-18T08:57:16.409122+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:08.507146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.414361+0000","last_change":"2023-10-18T08:57:14.371648+0000","last_active":"2023-10-18T08:57:15.414361+0000","last_peered":"2023-10-18T08:57:15.414361+0000","last_clean":"2023-10-18T08:57:15.414361+0000","last_became_active":"2023-10-18T08:57:14.371360+0000","last_became_peered":"2023-10-18T08:57:14.371360+0000","last_unstale":"2023-10-18T08:57:15.414361+0000","last_undegraded":"2023-10-18T08:57:15.414361+0000","last_fullsized":"2023-10-18T08:57:15.414361+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:27.276687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.184450+0000","last_change":"2023-10-18T08:57:15.410978+0000","last_active":"2023-10-18T08:57:17.184450+0000","last_peered":"2023-10-18T08:57:17.184450+0000","last_clean":"2023-10-18T08:57:17.184450+0000","last_became_active":"2023-10-18T08:57:15.408354+0000","last_became_peered":"2023-10-18T08:57:15.408354+0000","last_unstale":"2023-10-18T08:57:17.184450+0000","last_undegraded":"2023-10-18T08:57:17.184450+0000","last_fullsized":"2023-10-18T08:57:17.184450+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T08:57:15.293479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.413430+0000","last_change":"2023-10-18T08:57:14.371711+0000","last_active":"2023-10-18T08:57:15.413430+0000","last_peered":"2023-10-18T08:57:15.413430+0000","last_clean":"2023-10-18T08:57:15.413430+0000","last_became_active":"2023-10-18T08:57:14.371352+0000","last_became_peered":"2023-10-18T08:57:14.371352+0000","last_unstale":"2023-10-18T08:57:15.413430+0000","last_undegraded":"2023-10-18T08:57:15.413430+0000","last_fullsized":"2023-10-18T08:57:15.413430+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:28.358388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.091298+0000","last_change":"2023-10-18T08:57:15.378471+0000","last_active":"2023-10-18T08:57:18.091298+0000","last_peered":"2023-10-18T08:57:18.091298+0000","last_clean":"2023-10-18T08:57:18.091298+0000","last_became_active":"2023-10-18T08:57:15.376854+0000","last_became_peered":"2023-10-18T08:57:15.376854+0000","last_unstale":"2023-10-18T08:57:18.091298+0000","last_undegraded":"2023-10-18T08:57:18.091298+0000","last_fullsized":"2023-10-18T08:57:18.091298+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:29:38.697110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.412321+0000","last_change":"2023-10-18T08:57:14.373276+0000","last_active":"2023-10-18T08:57:15.412321+0000","last_peered":"2023-10-18T08:57:15.412321+0000","last_clean":"2023-10-18T08:57:15.412321+0000","last_became_active":"2023-10-18T08:57:14.373091+0000","last_became_peered":"2023-10-18T08:57:14.373091+0000","last_unstale":"2023-10-18T08:57:15.412321+0000","last_undegraded":"2023-10-18T08:57:15.412321+0000","last_fullsized":"2023-10-18T08:57:15.412321+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:35.097485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.408809+0000","last_change":"2023-10-18T08:57:15.405737+0000","last_active":"2023-10-18T08:57:16.408809+0000","last_peered":"2023-10-18T08:57:16.408809+0000","last_clean":"2023-10-18T08:57:16.408809+0000","last_became_active":"2023-10-18T08:57:15.405439+0000","last_became_peered":"2023-10-18T08:57:15.405439+0000","last_unstale":"2023-10-18T08:57:16.408809+0000","last_undegraded":"2023-10-18T08:57:16.408809+0000","last_fullsized":"2023-10-18T08:57:16.408809+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:09.244351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.409276+0000","last_change":"2023-10-18T08:57:14.373706+0000","last_active":"2023-10-18T08:57:15.409276+0000","last_peered":"2023-10-18T08:57:15.409276+0000","last_clean":"2023-10-18T08:57:15.409276+0000","last_became_active":"2023-10-18T08:57:14.373294+0000","last_became_peered":"2023-10-18T08:57:14.373294+0000","last_unstale":"2023-10-18T08:57:15.409276+0000","last_undegraded":"2023-10-18T08:57:15.409276+0000","last_fullsized":"2023-10-18T08:57:15.409276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:15.113701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.374872+0000","last_change":"2023-10-18T08:57:15.406143+0000","last_active":"2023-10-18T08:57:16.374872+0000","last_peered":"2023-10-18T08:57:16.374872+0000","last_clean":"2023-10-18T08:57:16.374872+0000","last_became_active":"2023-10-18T08:57:15.405879+0000","last_became_peered":"2023-10-18T08:57:15.405879+0000","last_unstale":"2023-10-18T08:57:16.374872+0000","last_undegraded":"2023-10-18T08:57:16.374872+0000","last_fullsized":"2023-10-18T08:57:16.374872+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:34.381463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.412507+0000","last_change":"2023-10-18T08:57:14.365008+0000","last_active":"2023-10-18T08:57:15.412507+0000","last_peered":"2023-10-18T08:57:15.412507+0000","last_clean":"2023-10-18T08:57:15.412507+0000","last_became_active":"2023-10-18T08:57:14.361849+0000","last_became_peered":"2023-10-18T08:57:14.361849+0000","last_unstale":"2023-10-18T08:57:15.412507+0000","last_undegraded":"2023-10-18T08:57:15.412507+0000","last_fullsized":"2023-10-18T08:57:15.412507+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:21.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302558+0000","last_change":"2023-10-18T08:57:15.402998+0000","last_active":"2023-10-18T08:57:17.302558+0000","last_peered":"2023-10-18T08:57:17.302558+0000","last_clean":"2023-10-18T08:57:17.302558+0000","last_became_active":"2023-10-18T08:57:15.402654+0000","last_became_peered":"2023-10-18T08:57:15.402654+0000","last_unstale":"2023-10-18T08:57:17.302558+0000","last_undegraded":"2023-10-18T08:57:17.302558+0000","last_fullsized":"2023-10-18T08:57:17.302558+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:42.529724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.413622+0000","last_change":"2023-10-18T08:57:14.372676+0000","last_active":"2023-10-18T08:57:15.413622+0000","last_peered":"2023-10-18T08:57:15.413622+0000","last_clean":"2023-10-18T08:57:15.413622+0000","last_became_active":"2023-10-18T08:57:14.372331+0000","last_became_peered":"2023-10-18T08:57:14.372331+0000","last_unstale":"2023-10-18T08:57:15.413622+0000","last_undegraded":"2023-10-18T08:57:15.413622+0000","last_fullsized":"2023-10-18T08:57:15.413622+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:00.621751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302286+0000","last_change":"2023-10-18T08:57:15.409908+0000","last_active":"2023-10-18T08:57:17.302286+0000","last_peered":"2023-10-18T08:57:17.302286+0000","last_clean":"2023-10-18T08:57:17.302286+0000","last_became_active":"2023-10-18T08:57:15.409185+0000","last_became_peered":"2023-10-18T08:57:15.409185+0000","last_unstale":"2023-10-18T08:57:17.302286+0000","last_undegraded":"2023-10-18T08:57:17.302286+0000","last_fullsized":"2023-10-18T08:57:17.302286+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:28.288612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.375123+0000","last_change":"2023-10-18T08:57:14.367951+0000","last_active":"2023-10-18T08:57:15.375123+0000","last_peered":"2023-10-18T08:57:15.375123+0000","last_clean":"2023-10-18T08:57:15.375123+0000","last_became_active":"2023-10-18T08:57:14.367667+0000","last_became_peered":"2023-10-18T08:57:14.367667+0000","last_unstale":"2023-10-18T08:57:15.375123+0000","last_undegraded":"2023-10-18T08:57:15.375123+0000","last_fullsized":"2023-10-18T08:57:15.375123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:00.621751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.185225+0000","last_change":"2023-10-18T08:57:15.405177+0000","last_active":"2023-10-18T08:57:17.185225+0000","last_peered":"2023-10-18T08:57:17.185225+0000","last_clean":"2023-10-18T08:57:17.185225+0000","last_became_active":"2023-10-18T08:57:15.403347+0000","last_became_peered":"2023-10-18T08:57:15.403347+0000","last_unstale":"2023-10-18T08:57:17.185225+0000","last_undegraded":"2023-10-18T08:57:17.185225+0000","last_fullsized":"2023-10-18T08:57:17.185225+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:59:53.559154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.405454+0000","last_change":"2023-10-18T08:57:14.371281+0000","last_active":"2023-10-18T08:57:15.405454+0000","last_peered":"2023-10-18T08:57:15.405454+0000","last_clean":"2023-10-18T08:57:15.405454+0000","last_became_active":"2023-10-18T08:57:14.370996+0000","last_became_peered":"2023-10-18T08:57:14.370996+0000","last_unstale":"2023-10-18T08:57:15.405454+0000","last_undegraded":"2023-10-18T08:57:15.405454+0000","last_fullsized":"2023-10-18T08:57:15.405454+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:08.232425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.377294+0000","last_change":"2023-10-18T08:57:15.409498+0000","last_active":"2023-10-18T08:57:16.377294+0000","last_peered":"2023-10-18T08:57:16.377294+0000","last_clean":"2023-10-18T08:57:16.377294+0000","last_became_active":"2023-10-18T08:57:15.408070+0000","last_became_peered":"2023-10-18T08:57:15.408070+0000","last_unstale":"2023-10-18T08:57:16.377294+0000","last_undegraded":"2023-10-18T08:57:16.377294+0000","last_fullsized":"2023-10-18T08:57:16.377294+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:10.615157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.385579+0000","last_change":"2023-10-18T08:57:14.363641+0000","last_active":"2023-10-18T08:57:15.385579+0000","last_peered":"2023-10-18T08:57:15.385579+0000","last_clean":"2023-10-18T08:57:15.385579+0000","last_became_active":"2023-10-18T08:57:14.359826+0000","last_became_peered":"2023-10-18T08:57:14.359826+0000","last_unstale":"2023-10-18T08:57:15.385579+0000","last_undegraded":"2023-10-18T08:57:15.385579+0000","last_fullsized":"2023-10-18T08:57:15.385579+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:08.465913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.375358+0000","last_change":"2023-10-18T08:57:15.380648+0000","last_active":"2023-10-18T08:57:16.375358+0000","last_peered":"2023-10-18T08:57:16.375358+0000","last_clean":"2023-10-18T08:57:16.375358+0000","last_became_active":"2023-10-18T08:57:15.380076+0000","last_became_peered":"2023-10-18T08:57:15.380076+0000","last_unstale":"2023-10-18T08:57:16.375358+0000","last_undegraded":"2023-10-18T08:57:16.375358+0000","last_fullsized":"2023-10-18T08:57:16.375358+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:10.032183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.398119+0000","last_change":"2023-10-18T08:57:14.369502+0000","last_active":"2023-10-18T08:57:15.398119+0000","last_peered":"2023-10-18T08:57:15.398119+0000","last_clean":"2023-10-18T08:57:15.398119+0000","last_became_active":"2023-10-18T08:57:14.369140+0000","last_became_peered":"2023-10-18T08:57:14.369140+0000","last_unstale":"2023-10-18T08:57:15.398119+0000","last_undegraded":"2023-10-18T08:57:15.398119+0000","last_fullsized":"2023-10-18T08:57:15.398119+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:08.465913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.372553+0000","last_change":"2023-10-18T08:57:15.395926+0000","last_active":"2023-10-18T08:57:16.372553+0000","last_peered":"2023-10-18T08:57:16.372553+0000","last_clean":"2023-10-18T08:57:16.372553+0000","last_became_active":"2023-10-18T08:57:15.394966+0000","last_became_peered":"2023-10-18T08:57:15.394966+0000","last_unstale":"2023-10-18T08:57:16.372553+0000","last_undegraded":"2023-10-18T08:57:16.372553+0000","last_fullsized":"2023-10-18T08:57:16.372553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:48:28.930263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.411915+0000","last_change":"2023-10-18T08:57:14.372079+0000","last_active":"2023-10-18T08:57:15.411915+0000","last_peered":"2023-10-18T08:57:15.411915+0000","last_clean":"2023-10-18T08:57:15.411915+0000","last_became_active":"2023-10-18T08:57:14.371832+0000","last_became_peered":"2023-10-18T08:57:14.371832+0000","last_unstale":"2023-10-18T08:57:15.411915+0000","last_undegraded":"2023-10-18T08:57:15.411915+0000","last_fullsized":"2023-10-18T08:57:15.411915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:00.621751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.375316+0000","last_change":"2023-10-18T08:57:15.401651+0000","last_active":"2023-10-18T08:57:16.375316+0000","last_peered":"2023-10-18T08:57:16.375316+0000","last_clean":"2023-10-18T08:57:16.375316+0000","last_became_active":"2023-10-18T08:57:15.400913+0000","last_became_peered":"2023-10-18T08:57:15.400913+0000","last_unstale":"2023-10-18T08:57:16.375316+0000","last_undegraded":"2023-10-18T08:57:16.375316+0000","last_fullsized":"2023-10-18T08:57:16.375316+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:32:51.933660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.418145+0000","last_change":"2023-10-18T08:57:14.358993+0000","last_active":"2023-10-18T08:57:15.418145+0000","last_peered":"2023-10-18T08:57:15.418145+0000","last_clean":"2023-10-18T08:57:15.418145+0000","last_became_active":"2023-10-18T08:57:14.358684+0000","last_became_peered":"2023-10-18T08:57:14.358684+0000","last_unstale":"2023-10-18T08:57:15.418145+0000","last_undegraded":"2023-10-18T08:57:15.418145+0000","last_fullsized":"2023-10-18T08:57:15.418145+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:31:16.160688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.184985+0000","last_change":"2023-10-18T08:57:15.402958+0000","last_active":"2023-10-18T08:57:17.184985+0000","last_peered":"2023-10-18T08:57:17.184985+0000","last_clean":"2023-10-18T08:57:17.184985+0000","last_became_active":"2023-10-18T08:57:15.402696+0000","last_became_peered":"2023-10-18T08:57:15.402696+0000","last_unstale":"2023-10-18T08:57:17.184985+0000","last_undegraded":"2023-10-18T08:57:17.184985+0000","last_fullsized":"2023-10-18T08:57:17.184985+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:09.477838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.412789+0000","last_change":"2023-10-18T08:57:14.363761+0000","last_active":"2023-10-18T08:57:15.412789+0000","last_peered":"2023-10-18T08:57:15.412789+0000","last_clean":"2023-10-18T08:57:15.412789+0000","last_became_active":"2023-10-18T08:57:14.363321+0000","last_became_peered":"2023-10-18T08:57:14.363321+0000","last_unstale":"2023-10-18T08:57:15.412789+0000","last_undegraded":"2023-10-18T08:57:15.412789+0000","last_fullsized":"2023-10-18T08:57:15.412789+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:50:33.631395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302695+0000","last_change":"2023-10-18T08:57:15.407663+0000","last_active":"2023-10-18T08:57:17.302695+0000","last_peered":"2023-10-18T08:57:17.302695+0000","last_clean":"2023-10-18T08:57:17.302695+0000","last_became_active":"2023-10-18T08:57:15.407129+0000","last_became_peered":"2023-10-18T08:57:15.407129+0000","last_unstale":"2023-10-18T08:57:17.302695+0000","last_undegraded":"2023-10-18T08:57:17.302695+0000","last_fullsized":"2023-10-18T08:57:17.302695+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:10.615157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.376766+0000","last_change":"2023-10-18T08:57:14.367626+0000","last_active":"2023-10-18T08:57:15.376766+0000","last_peered":"2023-10-18T08:57:15.376766+0000","last_clean":"2023-10-18T08:57:15.376766+0000","last_became_active":"2023-10-18T08:57:14.364913+0000","last_became_peered":"2023-10-18T08:57:14.364913+0000","last_unstale":"2023-10-18T08:57:15.376766+0000","last_undegraded":"2023-10-18T08:57:15.376766+0000","last_fullsized":"2023-10-18T08:57:15.376766+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:27.276687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.372408+0000","last_change":"2023-10-18T08:57:15.401548+0000","last_active":"2023-10-18T08:57:16.372408+0000","last_peered":"2023-10-18T08:57:16.372408+0000","last_clean":"2023-10-18T08:57:16.372408+0000","last_became_active":"2023-10-18T08:57:15.400108+0000","last_became_peered":"2023-10-18T08:57:15.400108+0000","last_unstale":"2023-10-18T08:57:16.372408+0000","last_undegraded":"2023-10-18T08:57:16.372408+0000","last_fullsized":"2023-10-18T08:57:16.372408+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:58.321274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.371185+0000","last_change":"2023-10-18T08:57:14.368186+0000","last_active":"2023-10-18T08:57:15.371185+0000","last_peered":"2023-10-18T08:57:15.371185+0000","last_clean":"2023-10-18T08:57:15.371185+0000","last_became_active":"2023-10-18T08:57:14.367105+0000","last_became_peered":"2023-10-18T08:57:14.367105+0000","last_unstale":"2023-10-18T08:57:15.371185+0000","last_undegraded":"2023-10-18T08:57:15.371185+0000","last_fullsized":"2023-10-18T08:57:15.371185+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:32:50.921734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.374771+0000","last_change":"2023-10-18T08:57:15.406911+0000","last_active":"2023-10-18T08:57:16.374771+0000","last_peered":"2023-10-18T08:57:16.374771+0000","last_clean":"2023-10-18T08:57:16.374771+0000","last_became_active":"2023-10-18T08:57:15.404857+0000","last_became_peered":"2023-10-18T08:57:15.404857+0000","last_unstale":"2023-10-18T08:57:16.374771+0000","last_undegraded":"2023-10-18T08:57:16.374771+0000","last_fullsized":"2023-10-18T08:57:16.374771+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:29:38.697110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.418242+0000","last_change":"2023-10-18T08:57:14.370165+0000","last_active":"2023-10-18T08:57:15.418242+0000","last_peered":"2023-10-18T08:57:15.418242+0000","last_clean":"2023-10-18T08:57:15.418242+0000","last_became_active":"2023-10-18T08:57:14.369906+0000","last_became_peered":"2023-10-18T08:57:14.369906+0000","last_unstale":"2023-10-18T08:57:15.418242+0000","last_undegraded":"2023-10-18T08:57:15.418242+0000","last_fullsized":"2023-10-18T08:57:15.418242+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:01.758247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.091164+0000","last_change":"2023-10-18T08:57:15.398384+0000","last_active":"2023-10-18T08:57:18.091164+0000","last_peered":"2023-10-18T08:57:18.091164+0000","last_clean":"2023-10-18T08:57:18.091164+0000","last_became_active":"2023-10-18T08:57:15.398048+0000","last_became_peered":"2023-10-18T08:57:15.398048+0000","last_unstale":"2023-10-18T08:57:18.091164+0000","last_undegraded":"2023-10-18T08:57:18.091164+0000","last_fullsized":"2023-10-18T08:57:18.091164+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:21:32.285864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.416385+0000","last_change":"2023-10-18T08:57:14.355217+0000","last_active":"2023-10-18T08:57:15.416385+0000","last_peered":"2023-10-18T08:57:15.416385+0000","last_clean":"2023-10-18T08:57:15.416385+0000","last_became_active":"2023-10-18T08:57:14.354864+0000","last_became_peered":"2023-10-18T08:57:14.354864+0000","last_unstale":"2023-10-18T08:57:15.416385+0000","last_undegraded":"2023-10-18T08:57:15.416385+0000","last_fullsized":"2023-10-18T08:57:15.416385+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:19.909678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302640+0000","last_change":"2023-10-18T08:57:15.391160+0000","last_active":"2023-10-18T08:57:17.302640+0000","last_peered":"2023-10-18T08:57:17.302640+0000","last_clean":"2023-10-18T08:57:17.302640+0000","last_became_active":"2023-10-18T08:57:15.390916+0000","last_became_peered":"2023-10-18T08:57:15.390916+0000","last_unstale":"2023-10-18T08:57:17.302640+0000","last_undegraded":"2023-10-18T08:57:17.302640+0000","last_fullsized":"2023-10-18T08:57:17.302640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:29:38.697110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.414515+0000","last_change":"2023-10-18T08:57:14.366838+0000","last_active":"2023-10-18T08:57:15.414515+0000","last_peered":"2023-10-18T08:57:15.414515+0000","last_clean":"2023-10-18T08:57:15.414515+0000","last_became_active":"2023-10-18T08:57:14.366429+0000","last_became_peered":"2023-10-18T08:57:14.366429+0000","last_unstale":"2023-10-18T08:57:15.414515+0000","last_undegraded":"2023-10-18T08:57:15.414515+0000","last_fullsized":"2023-10-18T08:57:15.414515+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:20:37.681848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.184345+0000","last_change":"2023-10-18T08:57:15.401580+0000","last_active":"2023-10-18T08:57:17.184345+0000","last_peered":"2023-10-18T08:57:17.184345+0000","last_clean":"2023-10-18T08:57:17.184345+0000","last_became_active":"2023-10-18T08:57:15.401353+0000","last_became_peered":"2023-10-18T08:57:15.401353+0000","last_unstale":"2023-10-18T08:57:17.184345+0000","last_undegraded":"2023-10-18T08:57:17.184345+0000","last_fullsized":"2023-10-18T08:57:17.184345+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:38.943057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.407906+0000","last_change":"2023-10-18T08:57:14.370663+0000","last_active":"2023-10-18T08:57:15.407906+0000","last_peered":"2023-10-18T08:57:15.407906+0000","last_clean":"2023-10-18T08:57:15.407906+0000","last_became_active":"2023-10-18T08:57:14.370383+0000","last_became_peered":"2023-10-18T08:57:14.370383+0000","last_unstale":"2023-10-18T08:57:15.407906+0000","last_undegraded":"2023-10-18T08:57:15.407906+0000","last_fullsized":"2023-10-18T08:57:15.407906+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:18.947640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.185278+0000","last_change":"2023-10-18T08:57:15.398017+0000","last_active":"2023-10-18T08:57:17.185278+0000","last_peered":"2023-10-18T08:57:17.185278+0000","last_clean":"2023-10-18T08:57:17.185278+0000","last_became_active":"2023-10-18T08:57:15.397794+0000","last_became_peered":"2023-10-18T08:57:15.397794+0000","last_unstale":"2023-10-18T08:57:17.185278+0000","last_undegraded":"2023-10-18T08:57:17.185278+0000","last_fullsized":"2023-10-18T08:57:17.185278+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:29.370314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.371526+0000","last_change":"2023-10-18T08:57:14.371953+0000","last_active":"2023-10-18T08:57:15.371526+0000","last_peered":"2023-10-18T08:57:15.371526+0000","last_clean":"2023-10-18T08:57:15.371526+0000","last_became_active":"2023-10-18T08:57:14.371639+0000","last_became_peered":"2023-10-18T08:57:14.371639+0000","last_unstale":"2023-10-18T08:57:15.371526+0000","last_undegraded":"2023-10-18T08:57:15.371526+0000","last_fullsized":"2023-10-18T08:57:15.371526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:09.020258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.372813+0000","last_change":"2023-10-18T08:57:15.395276+0000","last_active":"2023-10-18T08:57:16.372813+0000","last_peered":"2023-10-18T08:57:16.372813+0000","last_clean":"2023-10-18T08:57:16.372813+0000","last_became_active":"2023-10-18T08:57:15.394603+0000","last_became_peered":"2023-10-18T08:57:15.394603+0000","last_unstale":"2023-10-18T08:57:16.372813+0000","last_undegraded":"2023-10-18T08:57:16.372813+0000","last_fullsized":"2023-10-18T08:57:16.372813+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:34:00.421723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.413991+0000","last_change":"2023-10-18T08:57:14.356889+0000","last_active":"2023-10-18T08:57:15.413991+0000","last_peered":"2023-10-18T08:57:15.413991+0000","last_clean":"2023-10-18T08:57:15.413991+0000","last_became_active":"2023-10-18T08:57:14.355993+0000","last_became_peered":"2023-10-18T08:57:14.355993+0000","last_unstale":"2023-10-18T08:57:15.413991+0000","last_undegraded":"2023-10-18T08:57:15.413991+0000","last_fullsized":"2023-10-18T08:57:15.413991+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:05:07.672779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.408528+0000","last_change":"2023-10-18T08:57:15.399901+0000","last_active":"2023-10-18T08:57:16.408528+0000","last_peered":"2023-10-18T08:57:16.408528+0000","last_clean":"2023-10-18T08:57:16.408528+0000","last_became_active":"2023-10-18T08:57:15.399653+0000","last_became_peered":"2023-10-18T08:57:15.399653+0000","last_unstale":"2023-10-18T08:57:16.408528+0000","last_undegraded":"2023-10-18T08:57:16.408528+0000","last_fullsized":"2023-10-18T08:57:16.408528+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:29.370314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.375434+0000","last_change":"2023-10-18T08:57:14.371375+0000","last_active":"2023-10-18T08:57:15.375434+0000","last_peered":"2023-10-18T08:57:15.375434+0000","last_clean":"2023-10-18T08:57:15.375434+0000","last_became_active":"2023-10-18T08:57:14.371143+0000","last_became_peered":"2023-10-18T08:57:14.371143+0000","last_unstale":"2023-10-18T08:57:15.375434+0000","last_undegraded":"2023-10-18T08:57:15.375434+0000","last_fullsized":"2023-10-18T08:57:15.375434+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:33.369537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.090822+0000","last_change":"2023-10-18T08:57:15.397007+0000","last_active":"2023-10-18T08:57:18.090822+0000","last_peered":"2023-10-18T08:57:18.090822+0000","last_clean":"2023-10-18T08:57:18.090822+0000","last_became_active":"2023-10-18T08:57:15.396752+0000","last_became_peered":"2023-10-18T08:57:15.396752+0000","last_unstale":"2023-10-18T08:57:18.090822+0000","last_undegraded":"2023-10-18T08:57:18.090822+0000","last_fullsized":"2023-10-18T08:57:18.090822+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:45.669294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.405608+0000","last_change":"2023-10-18T08:57:14.369401+0000","last_active":"2023-10-18T08:57:15.405608+0000","last_peered":"2023-10-18T08:57:15.405608+0000","last_clean":"2023-10-18T08:57:15.405608+0000","last_became_active":"2023-10-18T08:57:14.369163+0000","last_became_peered":"2023-10-18T08:57:14.369163+0000","last_unstale":"2023-10-18T08:57:15.405608+0000","last_undegraded":"2023-10-18T08:57:15.405608+0000","last_fullsized":"2023-10-18T08:57:15.405608+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:07.495221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302907+0000","last_change":"2023-10-18T08:57:15.407547+0000","last_active":"2023-10-18T08:57:17.302907+0000","last_peered":"2023-10-18T08:57:17.302907+0000","last_clean":"2023-10-18T08:57:17.302907+0000","last_became_active":"2023-10-18T08:57:15.406795+0000","last_became_peered":"2023-10-18T08:57:15.406795+0000","last_unstale":"2023-10-18T08:57:17.302907+0000","last_undegraded":"2023-10-18T08:57:17.302907+0000","last_fullsized":"2023-10-18T08:57:17.302907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:32:51.933660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.377090+0000","last_change":"2023-10-18T08:57:14.368515+0000","last_active":"2023-10-18T08:57:15.377090+0000","last_peered":"2023-10-18T08:57:15.377090+0000","last_clean":"2023-10-18T08:57:15.377090+0000","last_became_active":"2023-10-18T08:57:14.368173+0000","last_became_peered":"2023-10-18T08:57:14.368173+0000","last_unstale":"2023-10-18T08:57:15.377090+0000","last_undegraded":"2023-10-18T08:57:15.377090+0000","last_fullsized":"2023-10-18T08:57:15.377090+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:05:07.672779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.184543+0000","last_change":"2023-10-18T08:57:15.409166+0000","last_active":"2023-10-18T08:57:17.184543+0000","last_peered":"2023-10-18T08:57:17.184543+0000","last_clean":"2023-10-18T08:57:17.184543+0000","last_became_active":"2023-10-18T08:57:15.408364+0000","last_became_peered":"2023-10-18T08:57:15.408364+0000","last_unstale":"2023-10-18T08:57:17.184543+0000","last_undegraded":"2023-10-18T08:57:17.184543+0000","last_fullsized":"2023-10-18T08:57:17.184543+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:08:15.503788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.373048+0000","last_change":"2023-10-18T08:57:14.366229+0000","last_active":"2023-10-18T08:57:16.373048+0000","last_peered":"2023-10-18T08:57:16.373048+0000","last_clean":"2023-10-18T08:57:16.373048+0000","last_became_active":"2023-10-18T08:57:14.365496+0000","last_became_peered":"2023-10-18T08:57:14.365496+0000","last_unstale":"2023-10-18T08:57:16.373048+0000","last_undegraded":"2023-10-18T08:57:16.373048+0000","last_fullsized":"2023-10-18T08:57:16.373048+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:09.603232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.303076+0000","last_change":"2023-10-18T08:57:15.407398+0000","last_active":"2023-10-18T08:57:17.303076+0000","last_peered":"2023-10-18T08:57:17.303076+0000","last_clean":"2023-10-18T08:57:17.303076+0000","last_became_active":"2023-10-18T08:57:15.406461+0000","last_became_peered":"2023-10-18T08:57:15.406461+0000","last_unstale":"2023-10-18T08:57:17.303076+0000","last_undegraded":"2023-10-18T08:57:17.303076+0000","last_fullsized":"2023-10-18T08:57:17.303076+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:10.032183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.373268+0000","last_change":"2023-10-18T08:57:14.373352+0000","last_active":"2023-10-18T08:57:15.373268+0000","last_peered":"2023-10-18T08:57:15.373268+0000","last_clean":"2023-10-18T08:57:15.373268+0000","last_became_active":"2023-10-18T08:57:14.373089+0000","last_became_peered":"2023-10-18T08:57:14.373089+0000","last_unstale":"2023-10-18T08:57:15.373268+0000","last_undegraded":"2023-10-18T08:57:15.373268+0000","last_fullsized":"2023-10-18T08:57:15.373268+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:07.495221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.376907+0000","last_change":"2023-10-18T08:57:15.395508+0000","last_active":"2023-10-18T08:57:16.376907+0000","last_peered":"2023-10-18T08:57:16.376907+0000","last_clean":"2023-10-18T08:57:16.376907+0000","last_became_active":"2023-10-18T08:57:15.395260+0000","last_became_peered":"2023-10-18T08:57:15.395260+0000","last_unstale":"2023-10-18T08:57:16.376907+0000","last_undegraded":"2023-10-18T08:57:16.376907+0000","last_fullsized":"2023-10-18T08:57:16.376907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:05:27.740292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.392360+0000","last_change":"2023-10-18T08:57:14.372847+0000","last_active":"2023-10-18T08:57:15.392360+0000","last_peered":"2023-10-18T08:57:15.392360+0000","last_clean":"2023-10-18T08:57:15.392360+0000","last_became_active":"2023-10-18T08:57:14.372531+0000","last_became_peered":"2023-10-18T08:57:14.372531+0000","last_unstale":"2023-10-18T08:57:15.392360+0000","last_undegraded":"2023-10-18T08:57:15.392360+0000","last_fullsized":"2023-10-18T08:57:15.392360+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:08.465913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.372564+0000","last_change":"2023-10-18T08:57:15.401027+0000","last_active":"2023-10-18T08:57:16.372564+0000","last_peered":"2023-10-18T08:57:16.372564+0000","last_clean":"2023-10-18T08:57:16.372564+0000","last_became_active":"2023-10-18T08:57:15.400790+0000","last_became_peered":"2023-10-18T08:57:15.400790+0000","last_unstale":"2023-10-18T08:57:16.372564+0000","last_undegraded":"2023-10-18T08:57:16.372564+0000","last_fullsized":"2023-10-18T08:57:16.372564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:59.252136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.377407+0000","last_change":"2023-10-18T08:57:14.373541+0000","last_active":"2023-10-18T08:57:15.377407+0000","last_peered":"2023-10-18T08:57:15.377407+0000","last_clean":"2023-10-18T08:57:15.377407+0000","last_became_active":"2023-10-18T08:57:14.372892+0000","last_became_peered":"2023-10-18T08:57:14.372892+0000","last_unstale":"2023-10-18T08:57:15.377407+0000","last_undegraded":"2023-10-18T08:57:15.377407+0000","last_fullsized":"2023-10-18T08:57:15.377407+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:21.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.373625+0000","last_change":"2023-10-18T08:57:15.395152+0000","last_active":"2023-10-18T08:57:16.373625+0000","last_peered":"2023-10-18T08:57:16.373625+0000","last_clean":"2023-10-18T08:57:16.373625+0000","last_became_active":"2023-10-18T08:57:15.394903+0000","last_became_peered":"2023-10-18T08:57:15.394903+0000","last_unstale":"2023-10-18T08:57:16.373625+0000","last_undegraded":"2023-10-18T08:57:16.373625+0000","last_fullsized":"2023-10-18T08:57:16.373625+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:30:16.102793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.387178+0000","last_change":"2023-10-18T08:57:14.358925+0000","last_active":"2023-10-18T08:57:15.387178+0000","last_peered":"2023-10-18T08:57:15.387178+0000","last_clean":"2023-10-18T08:57:15.387178+0000","last_became_active":"2023-10-18T08:57:14.355112+0000","last_became_peered":"2023-10-18T08:57:14.355112+0000","last_unstale":"2023-10-18T08:57:15.387178+0000","last_undegraded":"2023-10-18T08:57:15.387178+0000","last_fullsized":"2023-10-18T08:57:15.387178+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:30.531145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.090585+0000","last_change":"2023-10-18T08:57:15.399703+0000","last_active":"2023-10-18T08:57:18.090585+0000","last_peered":"2023-10-18T08:57:18.090585+0000","last_clean":"2023-10-18T08:57:18.090585+0000","last_became_active":"2023-10-18T08:57:15.399469+0000","last_became_peered":"2023-10-18T08:57:15.399469+0000","last_unstale":"2023-10-18T08:57:18.090585+0000","last_undegraded":"2023-10-18T08:57:18.090585+0000","last_fullsized":"2023-10-18T08:57:18.090585+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:16.125626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.387030+0000","last_change":"2023-10-18T08:57:14.374953+0000","last_active":"2023-10-18T08:57:15.387030+0000","last_peered":"2023-10-18T08:57:15.387030+0000","last_clean":"2023-10-18T08:57:15.387030+0000","last_became_active":"2023-10-18T08:57:14.374674+0000","last_became_peered":"2023-10-18T08:57:14.374674+0000","last_unstale":"2023-10-18T08:57:15.387030+0000","last_undegraded":"2023-10-18T08:57:15.387030+0000","last_fullsized":"2023-10-18T08:57:15.387030+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:05.351228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.376178+0000","last_change":"2023-10-18T08:57:15.409873+0000","last_active":"2023-10-18T08:57:16.376178+0000","last_peered":"2023-10-18T08:57:16.376178+0000","last_clean":"2023-10-18T08:57:16.376178+0000","last_became_active":"2023-10-18T08:57:15.408656+0000","last_became_peered":"2023-10-18T08:57:15.408656+0000","last_unstale":"2023-10-18T08:57:16.376178+0000","last_undegraded":"2023-10-18T08:57:16.376178+0000","last_fullsized":"2023-10-18T08:57:16.376178+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:28.288612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.390512+0000","last_change":"2023-10-18T08:57:14.367001+0000","last_active":"2023-10-18T08:57:15.390512+0000","last_peered":"2023-10-18T08:57:15.390512+0000","last_clean":"2023-10-18T08:57:15.390512+0000","last_became_active":"2023-10-18T08:57:14.366771+0000","last_became_peered":"2023-10-18T08:57:14.366771+0000","last_unstale":"2023-10-18T08:57:15.390512+0000","last_undegraded":"2023-10-18T08:57:15.390512+0000","last_fullsized":"2023-10-18T08:57:15.390512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:08:14.491862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302280+0000","last_change":"2023-10-18T08:57:15.392737+0000","last_active":"2023-10-18T08:57:17.302280+0000","last_peered":"2023-10-18T08:57:17.302280+0000","last_clean":"2023-10-18T08:57:17.302280+0000","last_became_active":"2023-10-18T08:57:15.392473+0000","last_became_peered":"2023-10-18T08:57:15.392473+0000","last_unstale":"2023-10-18T08:57:17.302280+0000","last_undegraded":"2023-10-18T08:57:17.302280+0000","last_fullsized":"2023-10-18T08:57:17.302280+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:16.125626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.412339+0000","last_change":"2023-10-18T08:57:14.372410+0000","last_active":"2023-10-18T08:57:15.412339+0000","last_peered":"2023-10-18T08:57:15.412339+0000","last_clean":"2023-10-18T08:57:15.412339+0000","last_became_active":"2023-10-18T08:57:14.372097+0000","last_became_peered":"2023-10-18T08:57:14.372097+0000","last_unstale":"2023-10-18T08:57:15.412339+0000","last_undegraded":"2023-10-18T08:57:15.412339+0000","last_fullsized":"2023-10-18T08:57:15.412339+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:08.232425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.376058+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:57:15.376058+0000","last_peered":"2023-10-18T08:57:15.376058+0000","last_clean":"2023-10-18T08:57:15.376058+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:57:15.376058+0000","last_undegraded":"2023-10-18T08:57:15.376058+0000","last_fullsized":"2023-10-18T08:57:15.376058+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:34.881744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":793,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194519,"statfs":{"total":200000000,"available":199187764,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.998,"15min":0.98999999999999999},"min":{"1min":0.67800000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3979999999999999,"5min":1.538,"15min":1.538},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.087,"5min":1.014,"15min":1.002},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":1.2689999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1399999999999999,"15min":1.1519999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.393,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.123,"15min":1.1160000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7849999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.484}]},{"osd":2,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.218,"15min":1.224},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.589,"5min":1.694,"15min":1.694},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1839999999999999,"15min":1.1859999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.1259999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.254,"15min":1.2609999999999999},"min":{"1min":0.91500000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.72,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.236},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3089999999999999,"15min":1.3129999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.8029999999999999,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.6040000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.1859999999999999,"15min":1.1579999999999999},"min":{"1min":0.92800000000000005,"5min":0.746,"15min":0.746},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.522},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.407,"15min":1.411},"min":{"1min":1.0369999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.885,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.095}]},{"osd":5,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3520000000000001,"15min":1.353},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.615,"5min":1.893,"15min":1.893},"last":1.319},{"interface":"front","average":{"1min":1.389,"5min":1.417,"15min":1.421},"min":{"1min":1.0940000000000001,"5min":0.874,"15min":0.874},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.764}]},{"osd":6,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.464,"15min":1.458},"min":{"1min":1.0169999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.9339999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.585},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.516,"15min":1.518},"min":{"1min":1.23,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.978,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.4410000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":217,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195095,"statfs":{"total":200000000,"available":199777550,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0800000000000001,"15min":1.073},"min":{"1min":0.65400000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.5529999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.651},{"interface":"front","average":{"1min":1.133,"5min":1.216,"15min":1.23},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.458}]},{"osd":1,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0780000000000001,"15min":1.081},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.3420000000000001,"5min":1.524,"15min":1.524},"last":1.55},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1659999999999999,"15min":1.1819999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.474,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":1.744}]},{"osd":2,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.1579999999999999,"15min":1.1719999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.1870000000000001,"15min":1.1910000000000001},"min":{"1min":0.82099999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5309999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.167}]},{"osd":3,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2509999999999999,"15min":1.2569999999999999},"min":{"1min":0.84799999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.502,"5min":1.784,"15min":1.784},"last":1.403},{"interface":"front","average":{"1min":1.222,"5min":1.2170000000000001,"15min":1.216},"min":{"1min":1.0329999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.494,"5min":1.5,"15min":1.5},"last":1.7889999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.1950000000000001,"15min":1.169},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.284,"5min":1.272,"15min":1.27},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6990000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.696}]},{"osd":5,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3560000000000001,"15min":1.3500000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.665,"5min":1.667,"15min":1.667},"last":1.599},{"interface":"front","average":{"1min":1.331,"5min":1.2330000000000001,"15min":1.2170000000000001},"min":{"1min":0.54900000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":1.216}]},{"osd":7,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4019999999999999,"15min":1.3959999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5249999999999999,"15min":1.53},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8500000000000001,"5min":2.794,"15min":2.794},"last":1.831}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":216,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195095,"statfs":{"total":200000000,"available":199778152,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2010000000000001,"15min":1.1850000000000001},"min":{"1min":0.77800000000000002,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.7,"5min":1.915,"15min":1.915},"last":9.3539999999999992},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.288,"15min":1.292},"min":{"1min":0.746,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7130000000000001,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":5.3029999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.2030000000000001,"15min":1.2210000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.613,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":2.7360000000000002},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.76000000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.581,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":2.367}]},{"osd":2,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2689999999999999,"15min":1.268},"min":{"1min":0.77700000000000002,"5min":0.755,"15min":0.755},"max":{"1min":1.7749999999999999,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.27,"5min":1.29,"15min":1.2929999999999999},"min":{"1min":0.96099999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.7210000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":2.3069999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.232,"15min":1.24},"min":{"1min":0.84799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5640000000000001,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":3.524},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.321,"15min":1.327},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.784,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.5629999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.417,"15min":1.4199999999999999},"min":{"1min":0.84899999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8799999999999999,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.881},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.5069999999999999,"15min":1.5369999999999999},"min":{"1min":0.90100000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7929999999999999,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":4.4429999999999996}]},{"osd":6,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.3839999999999999,"15min":1.4139999999999999},"min":{"1min":0.873,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6799999999999999,"5min":5.141,"15min":5.141},"last":4.5469999999999997},{"interface":"front","average":{"1min":1.448,"5min":1.5720000000000001,"15min":1.5920000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.931,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.611}]},{"osd":7,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.661,"15min":1.677},"min":{"1min":1.038,"5min":1.014,"15min":1.014},"max":{"1min":1.9990000000000001,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":4.2050000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.508,"15min":1.522},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.907,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":4.4690000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":217,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195095,"statfs":{"total":200000000,"available":199777629,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.105,"15min":1.109},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.081,"5min":1.1679999999999999,"15min":1.1819999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0659999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":0.60099999999999998}]},{"osd":1,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.0980000000000001,"15min":1.095},"min":{"1min":0.76100000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.583,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.2310000000000001,"15min":1.2410000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.929,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.72599999999999998}]},{"osd":2,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.1779999999999999,"15min":1.1759999999999999},"min":{"1min":0.75600000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.29,"15min":1.3100000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.601,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":0.58899999999999997}]},{"osd":3,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.292,"15min":1.302},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4159999999999999,"15min":1.413},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.0899999999999999,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":0.80500000000000005}]},{"osd":5,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3120000000000001,"15min":1.3},"min":{"1min":0.84099999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.8080000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.498},{"interface":"front","average":{"1min":1.323,"5min":1.365,"15min":1.3720000000000001},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.38400000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.397,"15min":1.3819999999999999},"min":{"1min":1.169,"5min":0.998,"15min":0.998},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.51600000000000001},{"interface":"front","average":{"1min":1.51,"5min":1.5680000000000001,"15min":1.5780000000000001},"min":{"1min":1.04,"5min":1.04,"15min":1.04},"max":{"1min":1.8939999999999999,"5min":3.6859999999999999,"15min":3.6859999999999999},"last":0.45400000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"max":{"1min":2.2759999999999998,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.438},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.5629999999999999,"15min":1.554},"min":{"1min":1.3089999999999999,"5min":1.23,"15min":1.23},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.747}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":217,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195095,"statfs":{"total":200000000,"available":199777584,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":0.82399999999999995,"15min":0.76500000000000001},"min":{"1min":0.52500000000000002,"5min":0.439,"15min":0.439},"max":{"1min":5.5730000000000004,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.365,"5min":0.93999999999999995,"15min":0.86899999999999999},"min":{"1min":0.53700000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.5599999999999996,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":0.72699999999999998}]},{"osd":1,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":0.98499999999999999,"15min":0.94399999999999995},"min":{"1min":0.59399999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":6.8099999999999996,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.238,"5min":1.1259999999999999,"15min":1.1080000000000001},"min":{"1min":0.69099999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.3959999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.0620000000000001,"15min":1.0169999999999999},"min":{"1min":0.61199999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":7.4580000000000002,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.0880000000000001,"15min":1.048},"min":{"1min":0.56699999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":3.5110000000000001,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":1.4730000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":2.3849999999999998,"5min":1.4570000000000001,"15min":1.3029999999999999},"min":{"1min":0.99099999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":18.664999999999999,"5min":18.664999999999999,"15min":18.664999999999999},"last":1.429},{"interface":"front","average":{"1min":1.47,"5min":1.1899999999999999,"15min":1.1439999999999999},"min":{"1min":0.83999999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":7.0700000000000003,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":1.1100000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.266,"15min":1.214},"min":{"1min":0.872,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":6.9139999999999997,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":1.161},{"interface":"front","average":{"1min":2.306,"5min":1.4690000000000001,"15min":1.329},"min":{"1min":1.0269999999999999,"5min":0.748,"15min":0.748},"max":{"1min":16.390999999999998,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.5980000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.8049999999999999,"5min":1.4750000000000001,"15min":1.4199999999999999},"min":{"1min":1.022,"5min":0.625,"15min":0.625},"max":{"1min":7.7270000000000003,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":1.536},{"interface":"front","average":{"1min":1.7190000000000001,"5min":1.3819999999999999,"15min":1.3260000000000001},"min":{"1min":0.96499999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":7.242,"5min":7.242,"15min":7.242},"last":1.355}]},{"osd":7,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.643,"5min":1.367,"15min":1.321},"min":{"1min":0.94299999999999995,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.7629999999999999},{"interface":"front","average":{"1min":2.1309999999999998,"5min":1.54,"15min":1.4419999999999999},"min":{"1min":0.84999999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":12.946,"5min":12.946,"15min":12.946},"last":1.6439999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574878,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195094,"statfs":{"total":200000000,"available":199776588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.92000000000000004,"15min":0.93999999999999995},"min":{"1min":0.433,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.395,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.024,"15min":1.0569999999999999},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3700000000000001,"5min":1.651,"15min":1.651},"last":0.71999999999999997}]},{"osd":1,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":1.04,"15min":1.0600000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3660000000000001,"5min":4.29,"15min":4.29},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.87,"5min":1.0580000000000001,"15min":1.089},"min":{"1min":0.57499999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3009999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.65700000000000003}]},{"osd":3,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.105,"15min":1.133},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.9139999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.111,"5min":1.4419999999999999,"15min":1.4970000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.609,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":0.57899999999999996}]},{"osd":4,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.47,"15min":1.502},"min":{"1min":0.90900000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.7,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.25,"5min":1.4239999999999999,"15min":1.4530000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.663,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.93799999999999994}]},{"osd":5,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.248,"15min":1.272},"min":{"1min":0.79400000000000004,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5389999999999999,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.256,"5min":1.542,"15min":1.5900000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.7569999999999999,"5min":7.657,"15min":7.657},"last":1.1639999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.4950000000000001,"15min":1.5389999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7170000000000001,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.244,"5min":1.4059999999999999,"15min":1.4330000000000001},"min":{"1min":0.94099999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.796,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.98899999999999999}]},{"osd":7,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.5449999999999999,"15min":1.5780000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7350000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.04},{"interface":"front","average":{"1min":1.3,"5min":1.4490000000000001,"15min":1.474},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.6899999999999999,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.026}]}]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":215,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195096,"statfs":{"total":200000000,"available":199779171,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.871,"15min":0.88400000000000001},"min":{"1min":0.54400000000000004,"5min":0.443,"15min":0.443},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.77200000000000002}]},{"osd":2,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.90000000000000002,"15min":0.91900000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.222,"5min":1.859,"15min":1.859},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.82599999999999996,"15min":0.83299999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.50800000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.98499999999999999,"15min":1.0089999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.024,"15min":1.048},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.7110000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":0.85299999999999998}]},{"osd":4,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.224,"15min":1.236},"min":{"1min":0.71499999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.53,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.1830000000000001,"15min":1.1890000000000001},"min":{"1min":0.85999999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.99,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.556}]},{"osd":5,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.262,"15min":1.2789999999999999},"min":{"1min":0.82899999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.6299999999999999,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.335,"15min":1.341},"min":{"1min":0.84999999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.8480000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.603}]},{"osd":6,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2529999999999999,"15min":1.2649999999999999},"min":{"1min":0.86199999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.2050000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.143,"5min":1.228,"15min":1.242},"min":{"1min":0.72999999999999998,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.853,"5min":1.853,"15min":1.853},"last":1.044}]},{"osd":7,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.3160000000000001,"15min":1.333},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.0419999999999998,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.522},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.333,"15min":1.3380000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.8169999999999999,"5min":1.994,"15min":1.994},"last":1.3360000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":792,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194520,"statfs":{"total":200000000,"available":199188839,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.92600000000000005,"15min":0.94599999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.25,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.504},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.077,"15min":1.083},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4990000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.58799999999999997}]},{"osd":2,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.339,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.621},{"interface":"front","average":{"1min":1.121,"5min":1.125,"15min":1.125},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.66000000000000003}]},{"osd":3,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.2849999999999999,"15min":1.327},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.294,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.323,"5min":1.5109999999999999,"15min":1.5429999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.8640000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.05}]},{"osd":4,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.2909999999999999,"15min":1.3260000000000001},"min":{"1min":0.72899999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.3460000000000001,"5min":4.492,"15min":4.492},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2509999999999999,"15min":1.2470000000000001},"min":{"1min":0.78500000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":1.2030000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.417,"15min":1.4630000000000001},"min":{"1min":0.79000000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.4770000000000001,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3879999999999999,"15min":1.3959999999999999},"min":{"1min":0.81299999999999994,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.758,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":0.92800000000000005}]},{"osd":6,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5980000000000001,"15min":1.6220000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.952,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.357,"5min":1.5069999999999999,"15min":1.532},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":2.0910000000000002,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.0309999999999999}]},{"osd":7,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.647,"15min":1.671},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.149,"5min":3.492,"15min":3.492},"last":1.363},{"interface":"front","average":{"1min":1.383,"5min":1.6830000000000001,"15min":1.7330000000000001},"min":{"1min":0.96799999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8859999999999999,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.2430000000000001}]}]}],"pool_statfs":[]}} 2023-10-18T08:57:25.054 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:57:25.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.292+0000 7f1bb9165640 1 -- 172.21.15.40:0/3506615030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40c99c0 msgr2=0x7f1bb40c9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:25.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.292+0000 7f1bb9165640 1 --2- 172.21.15.40:0/3506615030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40c99c0 0x7f1bb40c9e30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1bb406f590 tx=0x7f1bac02c210 comp rx=0 tx=0).stop 2023-10-18T08:57:25.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.292+0000 7f1bb9165640 1 -- 172.21.15.40:0/3506615030 shutdown_connections 2023-10-18T08:57:25.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.292+0000 7f1bb9165640 1 --2- 172.21.15.40:0/3506615030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40c99c0 0x7f1bb40c9e30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:25.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.292+0000 7f1bb9165640 1 --2- 172.21.15.40:0/3506615030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1bb40b15b0 0x7f1bb40b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:25.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.292+0000 7f1bb9165640 1 -- 172.21.15.40:0/3506615030 >> 172.21.15.40:0/3506615030 conn(0x7f1bb4069720 msgr2=0x7f1bb4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:25.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.292+0000 7f1bb9165640 1 -- 172.21.15.40:0/3506615030 shutdown_connections 2023-10-18T08:57:25.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.292+0000 7f1bb9165640 1 -- 172.21.15.40:0/3506615030 wait complete. 2023-10-18T08:57:25.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.294+0000 7f1bb9165640 1 Processor -- start 2023-10-18T08:57:25.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.294+0000 7f1bb9165640 1 -- start start 2023-10-18T08:57:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.295+0000 7f1bb9165640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1bb406d6f0 0x7f1bb4141450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.295+0000 7f1bb9165640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40b15b0 0x7f1bb4153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.295+0000 7f1bb9165640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1bb40c99c0 0x7f1bb41536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:25.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.295+0000 7f1bb9165640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40ab3d0 con 0x7f1bb40b15b0 2023-10-18T08:57:25.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.295+0000 7f1bb9165640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40aaa70 con 0x7f1bb40c99c0 2023-10-18T08:57:25.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.295+0000 7f1bb9165640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40aec20 con 0x7f1bb406d6f0 2023-10-18T08:57:25.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.295+0000 7f1bb37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40b15b0 0x7f1bb4153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:25.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.296+0000 7f1bb37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40b15b0 0x7f1bb4153190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38046/0 (socket says 172.21.15.40:38046) 2023-10-18T08:57:25.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.296+0000 7f1bb37fe640 1 -- 172.21.15.40:0/575222515 learned_addr learned my addr 172.21.15.40:0/575222515 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:25.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.296+0000 7f1bb8964640 1 --2- 172.21.15.40:0/575222515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1bb40c99c0 0x7f1bb41536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:25.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.296+0000 7f1bb3fff640 1 --2- 172.21.15.40:0/575222515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1bb406d6f0 0x7f1bb4141450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:25.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.296+0000 7f1bb37fe640 1 -- 172.21.15.40:0/575222515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1bb406d6f0 msgr2=0x7f1bb4141450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:25.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.296+0000 7f1bb37fe640 1 --2- 172.21.15.40:0/575222515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1bb406d6f0 0x7f1bb4141450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:25.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.296+0000 7f1bb37fe640 1 -- 172.21.15.40:0/575222515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1bb40c99c0 msgr2=0x7f1bb41536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:25.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.296+0000 7f1bb37fe640 1 --2- 172.21.15.40:0/575222515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1bb40c99c0 0x7f1bb41536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:25.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.296+0000 7f1bb37fe640 1 -- 172.21.15.40:0/575222515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac009900 con 0x7f1bb40b15b0 2023-10-18T08:57:25.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.296+0000 7f1bb37fe640 1 --2- 172.21.15.40:0/575222515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40b15b0 0x7f1bb4153190 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800c920 tx=0x7f1ba800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:25.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.297+0000 7f1bb0ff9640 1 -- 172.21.15.40:0/575222515 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8007c70 con 0x7f1bb40b15b0 2023-10-18T08:57:25.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.297+0000 7f1bb0ff9640 1 -- 172.21.15.40:0/575222515 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1ba800f450 con 0x7f1bb40b15b0 2023-10-18T08:57:25.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.297+0000 7f1bb0ff9640 1 -- 172.21.15.40:0/575222515 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba800f860 con 0x7f1bb40b15b0 2023-10-18T08:57:25.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.297+0000 7f1bb9165640 1 -- 172.21.15.40:0/575222515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb40aad10 con 0x7f1bb40b15b0 2023-10-18T08:57:25.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.297+0000 7f1bb9165640 1 -- 172.21.15.40:0/575222515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb40a92e0 con 0x7f1bb40b15b0 2023-10-18T08:57:25.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.298+0000 7f1bb9165640 1 -- 172.21.15.40:0/575222515 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb4066e00 con 0x7f1bb40b15b0 2023-10-18T08:57:25.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.299+0000 7f1bb0ff9640 1 -- 172.21.15.40:0/575222515 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1ba800f9c0 con 0x7f1bb40b15b0 2023-10-18T08:57:25.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.301+0000 7f1bb0ff9640 1 --2- 172.21.15.40:0/575222515 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1ba006ae40 0x7f1ba006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:25.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.301+0000 7f1bb0ff9640 1 -- 172.21.15.40:0/575222515 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ba8089f90 con 0x7f1bb40b15b0 2023-10-18T08:57:25.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.302+0000 7f1bb3fff640 1 --2- 172.21.15.40:0/575222515 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1ba006ae40 0x7f1ba006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:25.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.303+0000 7f1bb3fff640 1 --2- 172.21.15.40:0/575222515 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1ba006ae40 0x7f1ba006d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4004ea0 tx=0x7f1ba4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:25.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.308+0000 7f1bb0ff9640 1 -- 172.21.15.40:0/575222515 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ba8054fa0 con 0x7f1bb40b15b0 2023-10-18T08:57:25.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.502+0000 7f1bb9165640 1 -- 172.21.15.40:0/575222515 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1bb415bbb0 con 0x7f1ba006ae40 2023-10-18T08:57:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.524+0000 7f1bb0ff9640 1 -- 172.21.15.40:0/575222515 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282342 (secure 0 0 0) 0x7f1bb415bbb0 con 0x7f1ba006ae40 2023-10-18T08:57:25.530 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:25.532 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:57:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.531+0000 7f1bb9165640 1 -- 172.21.15.40:0/575222515 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1ba006ae40 msgr2=0x7f1ba006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.531+0000 7f1bb9165640 1 --2- 172.21.15.40:0/575222515 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1ba006ae40 0x7f1ba006d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4004ea0 tx=0x7f1ba4009040 comp rx=0 tx=0).stop 2023-10-18T08:57:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.531+0000 7f1bb9165640 1 -- 172.21.15.40:0/575222515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40b15b0 msgr2=0x7f1bb4153190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.531+0000 7f1bb9165640 1 --2- 172.21.15.40:0/575222515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40b15b0 0x7f1bb4153190 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800c920 tx=0x7f1ba800cde0 comp rx=0 tx=0).stop 2023-10-18T08:57:25.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.531+0000 7f1bb9165640 1 -- 172.21.15.40:0/575222515 shutdown_connections 2023-10-18T08:57:25.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.531+0000 7f1bb9165640 1 --2- 172.21.15.40:0/575222515 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1ba006ae40 0x7f1ba006d300 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:25.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.531+0000 7f1bb9165640 1 --2- 172.21.15.40:0/575222515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1bb40c99c0 0x7f1bb41536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:25.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.531+0000 7f1bb9165640 1 --2- 172.21.15.40:0/575222515 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1bb40b15b0 0x7f1bb4153190 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:25.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.531+0000 7f1bb9165640 1 --2- 172.21.15.40:0/575222515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1bb406d6f0 0x7f1bb4141450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:25.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.531+0000 7f1bb9165640 1 -- 172.21.15.40:0/575222515 >> 172.21.15.40:0/575222515 conn(0x7f1bb4069720 msgr2=0x7f1bb415a790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:25.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.531+0000 7f1bb9165640 1 -- 172.21.15.40:0/575222515 shutdown_connections 2023-10-18T08:57:25.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.531+0000 7f1bb9165640 1 -- 172.21.15.40:0/575222515 wait complete. 2023-10-18T08:57:25.555 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2023-10-18T08:57:24.940144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2887,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559612,"statfs":{"total":1600000000,"available":1597043277,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.569185"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.374986+0000","last_change":"2023-10-18T08:57:15.389269+0000","last_active":"2023-10-18T08:57:16.374986+0000","last_peered":"2023-10-18T08:57:16.374986+0000","last_clean":"2023-10-18T08:57:16.374986+0000","last_became_active":"2023-10-18T08:57:15.388586+0000","last_became_peered":"2023-10-18T08:57:15.388586+0000","last_unstale":"2023-10-18T08:57:16.374986+0000","last_undegraded":"2023-10-18T08:57:16.374986+0000","last_fullsized":"2023-10-18T08:57:16.374986+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:10.615157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.090744+0000","last_change":"2023-10-18T08:57:15.403603+0000","last_active":"2023-10-18T08:57:18.090744+0000","last_peered":"2023-10-18T08:57:18.090744+0000","last_clean":"2023-10-18T08:57:18.090744+0000","last_became_active":"2023-10-18T08:57:15.403352+0000","last_became_peered":"2023-10-18T08:57:15.403352+0000","last_unstale":"2023-10-18T08:57:18.090744+0000","last_undegraded":"2023-10-18T08:57:18.090744+0000","last_fullsized":"2023-10-18T08:57:18.090744+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:28.288612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.375196+0000","last_change":"2023-10-18T08:57:15.400440+0000","last_active":"2023-10-18T08:57:16.375196+0000","last_peered":"2023-10-18T08:57:16.375196+0000","last_clean":"2023-10-18T08:57:16.375196+0000","last_became_active":"2023-10-18T08:57:15.399898+0000","last_became_peered":"2023-10-18T08:57:15.399898+0000","last_unstale":"2023-10-18T08:57:16.375196+0000","last_undegraded":"2023-10-18T08:57:16.375196+0000","last_fullsized":"2023-10-18T08:57:16.375196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:16.125626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.301864+0000","last_change":"2023-10-18T08:57:15.404432+0000","last_active":"2023-10-18T08:57:17.301864+0000","last_peered":"2023-10-18T08:57:17.301864+0000","last_clean":"2023-10-18T08:57:17.301864+0000","last_became_active":"2023-10-18T08:57:15.404044+0000","last_became_peered":"2023-10-18T08:57:15.404044+0000","last_unstale":"2023-10-18T08:57:17.301864+0000","last_undegraded":"2023-10-18T08:57:17.301864+0000","last_fullsized":"2023-10-18T08:57:17.301864+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:31.543070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.091161+0000","last_change":"2023-10-18T08:57:15.408728+0000","last_active":"2023-10-18T08:57:18.091161+0000","last_peered":"2023-10-18T08:57:18.091161+0000","last_clean":"2023-10-18T08:57:18.091161+0000","last_became_active":"2023-10-18T08:57:15.408457+0000","last_became_peered":"2023-10-18T08:57:15.408457+0000","last_unstale":"2023-10-18T08:57:18.091161+0000","last_undegraded":"2023-10-18T08:57:18.091161+0000","last_fullsized":"2023-10-18T08:57:18.091161+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:22.712989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.373009+0000","last_change":"2023-10-18T08:57:15.379923+0000","last_active":"2023-10-18T08:57:16.373009+0000","last_peered":"2023-10-18T08:57:16.373009+0000","last_clean":"2023-10-18T08:57:16.373009+0000","last_became_active":"2023-10-18T08:57:15.378979+0000","last_became_peered":"2023-10-18T08:57:15.378979+0000","last_unstale":"2023-10-18T08:57:16.373009+0000","last_undegraded":"2023-10-18T08:57:16.373009+0000","last_fullsized":"2023-10-18T08:57:16.373009+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:29:20.959175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.374306+0000","last_change":"2023-10-18T08:57:15.401109+0000","last_active":"2023-10-18T08:57:16.374306+0000","last_peered":"2023-10-18T08:57:16.374306+0000","last_clean":"2023-10-18T08:57:16.374306+0000","last_became_active":"2023-10-18T08:57:15.400850+0000","last_became_peered":"2023-10-18T08:57:15.400850+0000","last_unstale":"2023-10-18T08:57:16.374306+0000","last_undegraded":"2023-10-18T08:57:16.374306+0000","last_fullsized":"2023-10-18T08:57:16.374306+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:42.529724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.301895+0000","last_change":"2023-10-18T08:57:15.404043+0000","last_active":"2023-10-18T08:57:17.301895+0000","last_peered":"2023-10-18T08:57:17.301895+0000","last_clean":"2023-10-18T08:57:17.301895+0000","last_became_active":"2023-10-18T08:57:15.403796+0000","last_became_peered":"2023-10-18T08:57:15.403796+0000","last_unstale":"2023-10-18T08:57:17.301895+0000","last_undegraded":"2023-10-18T08:57:17.301895+0000","last_fullsized":"2023-10-18T08:57:17.301895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:19.959565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302230+0000","last_change":"2023-10-18T08:57:15.404275+0000","last_active":"2023-10-18T08:57:17.302230+0000","last_peered":"2023-10-18T08:57:17.302230+0000","last_clean":"2023-10-18T08:57:17.302230+0000","last_became_active":"2023-10-18T08:57:15.403525+0000","last_became_peered":"2023-10-18T08:57:15.403525+0000","last_unstale":"2023-10-18T08:57:17.302230+0000","last_undegraded":"2023-10-18T08:57:17.302230+0000","last_fullsized":"2023-10-18T08:57:17.302230+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:34.381463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.301895+0000","last_change":"2023-10-18T08:57:15.410588+0000","last_active":"2023-10-18T08:57:17.301895+0000","last_peered":"2023-10-18T08:57:17.301895+0000","last_clean":"2023-10-18T08:57:17.301895+0000","last_became_active":"2023-10-18T08:57:15.410362+0000","last_became_peered":"2023-10-18T08:57:15.410362+0000","last_unstale":"2023-10-18T08:57:17.301895+0000","last_undegraded":"2023-10-18T08:57:17.301895+0000","last_fullsized":"2023-10-18T08:57:17.301895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:01.633676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302286+0000","last_change":"2023-10-18T08:57:15.400199+0000","last_active":"2023-10-18T08:57:17.302286+0000","last_peered":"2023-10-18T08:57:17.302286+0000","last_clean":"2023-10-18T08:57:17.302286+0000","last_became_active":"2023-10-18T08:57:15.399889+0000","last_became_peered":"2023-10-18T08:57:15.399889+0000","last_unstale":"2023-10-18T08:57:17.302286+0000","last_undegraded":"2023-10-18T08:57:17.302286+0000","last_fullsized":"2023-10-18T08:57:17.302286+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:05:08.684704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.090952+0000","last_change":"2023-10-18T08:57:15.398078+0000","last_active":"2023-10-18T08:57:18.090952+0000","last_peered":"2023-10-18T08:57:18.090952+0000","last_clean":"2023-10-18T08:57:18.090952+0000","last_became_active":"2023-10-18T08:57:15.397874+0000","last_became_peered":"2023-10-18T08:57:15.397874+0000","last_unstale":"2023-10-18T08:57:18.090952+0000","last_undegraded":"2023-10-18T08:57:18.090952+0000","last_fullsized":"2023-10-18T08:57:18.090952+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:05:08.684704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.091291+0000","last_change":"2023-10-18T08:57:15.405598+0000","last_active":"2023-10-18T08:57:18.091291+0000","last_peered":"2023-10-18T08:57:18.091291+0000","last_clean":"2023-10-18T08:57:18.091291+0000","last_became_active":"2023-10-18T08:57:15.405278+0000","last_became_peered":"2023-10-18T08:57:15.405278+0000","last_unstale":"2023-10-18T08:57:18.091291+0000","last_undegraded":"2023-10-18T08:57:18.091291+0000","last_fullsized":"2023-10-18T08:57:18.091291+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:19:38.945359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.303220+0000","last_change":"2023-10-18T08:57:15.403771+0000","last_active":"2023-10-18T08:57:17.303220+0000","last_peered":"2023-10-18T08:57:17.303220+0000","last_clean":"2023-10-18T08:57:17.303220+0000","last_became_active":"2023-10-18T08:57:15.402907+0000","last_became_peered":"2023-10-18T08:57:15.402907+0000","last_unstale":"2023-10-18T08:57:17.303220+0000","last_undegraded":"2023-10-18T08:57:17.303220+0000","last_fullsized":"2023-10-18T08:57:17.303220+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:05:27.740292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.372706+0000","last_change":"2023-10-18T08:57:15.390844+0000","last_active":"2023-10-18T08:57:16.372706+0000","last_peered":"2023-10-18T08:57:16.372706+0000","last_clean":"2023-10-18T08:57:16.372706+0000","last_became_active":"2023-10-18T08:57:15.390204+0000","last_became_peered":"2023-10-18T08:57:15.390204+0000","last_unstale":"2023-10-18T08:57:16.372706+0000","last_undegraded":"2023-10-18T08:57:16.372706+0000","last_fullsized":"2023-10-18T08:57:16.372706+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:06.663782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.184671+0000","last_change":"2023-10-18T08:57:15.406162+0000","last_active":"2023-10-18T08:57:17.184671+0000","last_peered":"2023-10-18T08:57:17.184671+0000","last_clean":"2023-10-18T08:57:17.184671+0000","last_became_active":"2023-10-18T08:57:15.405881+0000","last_became_peered":"2023-10-18T08:57:15.405881+0000","last_unstale":"2023-10-18T08:57:17.184671+0000","last_undegraded":"2023-10-18T08:57:17.184671+0000","last_fullsized":"2023-10-18T08:57:17.184671+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:06.363154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.376820+0000","last_change":"2023-10-18T08:57:15.399690+0000","last_active":"2023-10-18T08:57:16.376820+0000","last_peered":"2023-10-18T08:57:16.376820+0000","last_clean":"2023-10-18T08:57:16.376820+0000","last_became_active":"2023-10-18T08:57:15.397149+0000","last_became_peered":"2023-10-18T08:57:15.397149+0000","last_unstale":"2023-10-18T08:57:16.376820+0000","last_undegraded":"2023-10-18T08:57:16.376820+0000","last_fullsized":"2023-10-18T08:57:16.376820+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:22.712989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.376683+0000","last_change":"2023-10-18T08:57:15.400280+0000","last_active":"2023-10-18T08:57:16.376683+0000","last_peered":"2023-10-18T08:57:16.376683+0000","last_clean":"2023-10-18T08:57:16.376683+0000","last_became_active":"2023-10-18T08:57:15.399540+0000","last_became_peered":"2023-10-18T08:57:15.399540+0000","last_unstale":"2023-10-18T08:57:16.376683+0000","last_undegraded":"2023-10-18T08:57:16.376683+0000","last_fullsized":"2023-10-18T08:57:16.376683+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:05:08.684704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.185536+0000","last_change":"2023-10-18T08:57:15.406156+0000","last_active":"2023-10-18T08:57:17.185536+0000","last_peered":"2023-10-18T08:57:17.185536+0000","last_clean":"2023-10-18T08:57:17.185536+0000","last_became_active":"2023-10-18T08:57:15.405706+0000","last_became_peered":"2023-10-18T08:57:15.405706+0000","last_unstale":"2023-10-18T08:57:17.185536+0000","last_undegraded":"2023-10-18T08:57:17.185536+0000","last_fullsized":"2023-10-18T08:57:17.185536+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:08.507146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.090762+0000","last_change":"2023-10-18T08:57:15.408499+0000","last_active":"2023-10-18T08:57:18.090762+0000","last_peered":"2023-10-18T08:57:18.090762+0000","last_clean":"2023-10-18T08:57:18.090762+0000","last_became_active":"2023-10-18T08:57:15.408271+0000","last_became_peered":"2023-10-18T08:57:15.408271+0000","last_unstale":"2023-10-18T08:57:18.090762+0000","last_undegraded":"2023-10-18T08:57:18.090762+0000","last_fullsized":"2023-10-18T08:57:18.090762+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:40:23.738176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.373281+0000","last_change":"2023-10-18T08:57:15.400776+0000","last_active":"2023-10-18T08:57:16.373281+0000","last_peered":"2023-10-18T08:57:16.373281+0000","last_clean":"2023-10-18T08:57:16.373281+0000","last_became_active":"2023-10-18T08:57:15.400519+0000","last_became_peered":"2023-10-18T08:57:15.400519+0000","last_unstale":"2023-10-18T08:57:16.373281+0000","last_undegraded":"2023-10-18T08:57:16.373281+0000","last_fullsized":"2023-10-18T08:57:16.373281+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:02.402794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302344+0000","last_change":"2023-10-18T08:57:15.409255+0000","last_active":"2023-10-18T08:57:17.302344+0000","last_peered":"2023-10-18T08:57:17.302344+0000","last_clean":"2023-10-18T08:57:17.302344+0000","last_became_active":"2023-10-18T08:57:15.408910+0000","last_became_peered":"2023-10-18T08:57:15.408910+0000","last_unstale":"2023-10-18T08:57:17.302344+0000","last_undegraded":"2023-10-18T08:57:17.302344+0000","last_fullsized":"2023-10-18T08:57:17.302344+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:22.712989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.185285+0000","last_change":"2023-10-18T08:57:15.403446+0000","last_active":"2023-10-18T08:57:17.185285+0000","last_peered":"2023-10-18T08:57:17.185285+0000","last_clean":"2023-10-18T08:57:17.185285+0000","last_became_active":"2023-10-18T08:57:15.403176+0000","last_became_peered":"2023-10-18T08:57:15.403176+0000","last_unstale":"2023-10-18T08:57:17.185285+0000","last_undegraded":"2023-10-18T08:57:17.185285+0000","last_fullsized":"2023-10-18T08:57:17.185285+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:09.244351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.184459+0000","last_change":"2023-10-18T08:57:15.405352+0000","last_active":"2023-10-18T08:57:17.184459+0000","last_peered":"2023-10-18T08:57:17.184459+0000","last_clean":"2023-10-18T08:57:17.184459+0000","last_became_active":"2023-10-18T08:57:15.404942+0000","last_became_peered":"2023-10-18T08:57:15.404942+0000","last_unstale":"2023-10-18T08:57:17.184459+0000","last_undegraded":"2023-10-18T08:57:17.184459+0000","last_fullsized":"2023-10-18T08:57:17.184459+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:42:22.676749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.184750+0000","last_change":"2023-10-18T08:57:15.404895+0000","last_active":"2023-10-18T08:57:17.184750+0000","last_peered":"2023-10-18T08:57:17.184750+0000","last_clean":"2023-10-18T08:57:17.184750+0000","last_became_active":"2023-10-18T08:57:15.403675+0000","last_became_peered":"2023-10-18T08:57:15.403675+0000","last_unstale":"2023-10-18T08:57:17.184750+0000","last_undegraded":"2023-10-18T08:57:17.184750+0000","last_fullsized":"2023-10-18T08:57:17.184750+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:26:10.073233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.091374+0000","last_change":"2023-10-18T08:57:15.395124+0000","last_active":"2023-10-18T08:57:18.091374+0000","last_peered":"2023-10-18T08:57:18.091374+0000","last_clean":"2023-10-18T08:57:18.091374+0000","last_became_active":"2023-10-18T08:57:15.394746+0000","last_became_peered":"2023-10-18T08:57:15.394746+0000","last_unstale":"2023-10-18T08:57:18.091374+0000","last_undegraded":"2023-10-18T08:57:18.091374+0000","last_fullsized":"2023-10-18T08:57:18.091374+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:14.564913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.377083+0000","last_change":"2023-10-18T08:57:15.393582+0000","last_active":"2023-10-18T08:57:16.377083+0000","last_peered":"2023-10-18T08:57:16.377083+0000","last_clean":"2023-10-18T08:57:16.377083+0000","last_became_active":"2023-10-18T08:57:15.393072+0000","last_became_peered":"2023-10-18T08:57:15.393072+0000","last_unstale":"2023-10-18T08:57:16.377083+0000","last_undegraded":"2023-10-18T08:57:16.377083+0000","last_fullsized":"2023-10-18T08:57:16.377083+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:29:38.697110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.091200+0000","last_change":"2023-10-18T08:57:15.399109+0000","last_active":"2023-10-18T08:57:18.091200+0000","last_peered":"2023-10-18T08:57:18.091200+0000","last_clean":"2023-10-18T08:57:18.091200+0000","last_became_active":"2023-10-18T08:57:15.397962+0000","last_became_peered":"2023-10-18T08:57:15.397962+0000","last_unstale":"2023-10-18T08:57:18.091200+0000","last_undegraded":"2023-10-18T08:57:18.091200+0000","last_fullsized":"2023-10-18T08:57:18.091200+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:42.529724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.184883+0000","last_change":"2023-10-18T08:57:15.404013+0000","last_active":"2023-10-18T08:57:17.184883+0000","last_peered":"2023-10-18T08:57:17.184883+0000","last_clean":"2023-10-18T08:57:17.184883+0000","last_became_active":"2023-10-18T08:57:15.402116+0000","last_became_peered":"2023-10-18T08:57:15.402116+0000","last_unstale":"2023-10-18T08:57:17.184883+0000","last_undegraded":"2023-10-18T08:57:17.184883+0000","last_fullsized":"2023-10-18T08:57:17.184883+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:36.109411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.376966+0000","last_change":"2023-10-18T08:57:15.393254+0000","last_active":"2023-10-18T08:57:16.376966+0000","last_peered":"2023-10-18T08:57:16.376966+0000","last_clean":"2023-10-18T08:57:16.376966+0000","last_became_active":"2023-10-18T08:57:15.391546+0000","last_became_peered":"2023-10-18T08:57:15.391546+0000","last_unstale":"2023-10-18T08:57:16.376966+0000","last_undegraded":"2023-10-18T08:57:16.376966+0000","last_fullsized":"2023-10-18T08:57:16.376966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:42.529724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.373439+0000","last_change":"2023-10-18T08:57:15.378256+0000","last_active":"2023-10-18T08:57:16.373439+0000","last_peered":"2023-10-18T08:57:16.373439+0000","last_clean":"2023-10-18T08:57:16.373439+0000","last_became_active":"2023-10-18T08:57:15.377276+0000","last_became_peered":"2023-10-18T08:57:15.377276+0000","last_unstale":"2023-10-18T08:57:16.373439+0000","last_undegraded":"2023-10-18T08:57:16.373439+0000","last_fullsized":"2023-10-18T08:57:16.373439+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:36:22.196893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.091016+0000","last_change":"2023-10-18T08:57:15.400776+0000","last_active":"2023-10-18T08:57:18.091016+0000","last_peered":"2023-10-18T08:57:18.091016+0000","last_clean":"2023-10-18T08:57:18.091016+0000","last_became_active":"2023-10-18T08:57:15.400464+0000","last_became_peered":"2023-10-18T08:57:15.400464+0000","last_unstale":"2023-10-18T08:57:18.091016+0000","last_undegraded":"2023-10-18T08:57:18.091016+0000","last_fullsized":"2023-10-18T08:57:18.091016+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:15:31.459376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.409122+0000","last_change":"2023-10-18T08:57:15.401657+0000","last_active":"2023-10-18T08:57:16.409122+0000","last_peered":"2023-10-18T08:57:16.409122+0000","last_clean":"2023-10-18T08:57:16.409122+0000","last_became_active":"2023-10-18T08:57:15.401392+0000","last_became_peered":"2023-10-18T08:57:15.401392+0000","last_unstale":"2023-10-18T08:57:16.409122+0000","last_undegraded":"2023-10-18T08:57:16.409122+0000","last_fullsized":"2023-10-18T08:57:16.409122+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:08.507146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.414361+0000","last_change":"2023-10-18T08:57:14.371648+0000","last_active":"2023-10-18T08:57:15.414361+0000","last_peered":"2023-10-18T08:57:15.414361+0000","last_clean":"2023-10-18T08:57:15.414361+0000","last_became_active":"2023-10-18T08:57:14.371360+0000","last_became_peered":"2023-10-18T08:57:14.371360+0000","last_unstale":"2023-10-18T08:57:15.414361+0000","last_undegraded":"2023-10-18T08:57:15.414361+0000","last_fullsized":"2023-10-18T08:57:15.414361+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:27.276687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.184450+0000","last_change":"2023-10-18T08:57:15.410978+0000","last_active":"2023-10-18T08:57:17.184450+0000","last_peered":"2023-10-18T08:57:17.184450+0000","last_clean":"2023-10-18T08:57:17.184450+0000","last_became_active":"2023-10-18T08:57:15.408354+0000","last_became_peered":"2023-10-18T08:57:15.408354+0000","last_unstale":"2023-10-18T08:57:17.184450+0000","last_undegraded":"2023-10-18T08:57:17.184450+0000","last_fullsized":"2023-10-18T08:57:17.184450+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T08:57:15.293479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.413430+0000","last_change":"2023-10-18T08:57:14.371711+0000","last_active":"2023-10-18T08:57:15.413430+0000","last_peered":"2023-10-18T08:57:15.413430+0000","last_clean":"2023-10-18T08:57:15.413430+0000","last_became_active":"2023-10-18T08:57:14.371352+0000","last_became_peered":"2023-10-18T08:57:14.371352+0000","last_unstale":"2023-10-18T08:57:15.413430+0000","last_undegraded":"2023-10-18T08:57:15.413430+0000","last_fullsized":"2023-10-18T08:57:15.413430+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:28.358388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.091298+0000","last_change":"2023-10-18T08:57:15.378471+0000","last_active":"2023-10-18T08:57:18.091298+0000","last_peered":"2023-10-18T08:57:18.091298+0000","last_clean":"2023-10-18T08:57:18.091298+0000","last_became_active":"2023-10-18T08:57:15.376854+0000","last_became_peered":"2023-10-18T08:57:15.376854+0000","last_unstale":"2023-10-18T08:57:18.091298+0000","last_undegraded":"2023-10-18T08:57:18.091298+0000","last_fullsized":"2023-10-18T08:57:18.091298+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:29:38.697110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.412321+0000","last_change":"2023-10-18T08:57:14.373276+0000","last_active":"2023-10-18T08:57:15.412321+0000","last_peered":"2023-10-18T08:57:15.412321+0000","last_clean":"2023-10-18T08:57:15.412321+0000","last_became_active":"2023-10-18T08:57:14.373091+0000","last_became_peered":"2023-10-18T08:57:14.373091+0000","last_unstale":"2023-10-18T08:57:15.412321+0000","last_undegraded":"2023-10-18T08:57:15.412321+0000","last_fullsized":"2023-10-18T08:57:15.412321+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:59:35.097485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.408809+0000","last_change":"2023-10-18T08:57:15.405737+0000","last_active":"2023-10-18T08:57:16.408809+0000","last_peered":"2023-10-18T08:57:16.408809+0000","last_clean":"2023-10-18T08:57:16.408809+0000","last_became_active":"2023-10-18T08:57:15.405439+0000","last_became_peered":"2023-10-18T08:57:15.405439+0000","last_unstale":"2023-10-18T08:57:16.408809+0000","last_undegraded":"2023-10-18T08:57:16.408809+0000","last_fullsized":"2023-10-18T08:57:16.408809+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:09.244351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.409276+0000","last_change":"2023-10-18T08:57:14.373706+0000","last_active":"2023-10-18T08:57:15.409276+0000","last_peered":"2023-10-18T08:57:15.409276+0000","last_clean":"2023-10-18T08:57:15.409276+0000","last_became_active":"2023-10-18T08:57:14.373294+0000","last_became_peered":"2023-10-18T08:57:14.373294+0000","last_unstale":"2023-10-18T08:57:15.409276+0000","last_undegraded":"2023-10-18T08:57:15.409276+0000","last_fullsized":"2023-10-18T08:57:15.409276+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:15.113701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.374872+0000","last_change":"2023-10-18T08:57:15.406143+0000","last_active":"2023-10-18T08:57:16.374872+0000","last_peered":"2023-10-18T08:57:16.374872+0000","last_clean":"2023-10-18T08:57:16.374872+0000","last_became_active":"2023-10-18T08:57:15.405879+0000","last_became_peered":"2023-10-18T08:57:15.405879+0000","last_unstale":"2023-10-18T08:57:16.374872+0000","last_undegraded":"2023-10-18T08:57:16.374872+0000","last_fullsized":"2023-10-18T08:57:16.374872+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:34.381463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.412507+0000","last_change":"2023-10-18T08:57:14.365008+0000","last_active":"2023-10-18T08:57:15.412507+0000","last_peered":"2023-10-18T08:57:15.412507+0000","last_clean":"2023-10-18T08:57:15.412507+0000","last_became_active":"2023-10-18T08:57:14.361849+0000","last_became_peered":"2023-10-18T08:57:14.361849+0000","last_unstale":"2023-10-18T08:57:15.412507+0000","last_undegraded":"2023-10-18T08:57:15.412507+0000","last_fullsized":"2023-10-18T08:57:15.412507+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:21.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302558+0000","last_change":"2023-10-18T08:57:15.402998+0000","last_active":"2023-10-18T08:57:17.302558+0000","last_peered":"2023-10-18T08:57:17.302558+0000","last_clean":"2023-10-18T08:57:17.302558+0000","last_became_active":"2023-10-18T08:57:15.402654+0000","last_became_peered":"2023-10-18T08:57:15.402654+0000","last_unstale":"2023-10-18T08:57:17.302558+0000","last_undegraded":"2023-10-18T08:57:17.302558+0000","last_fullsized":"2023-10-18T08:57:17.302558+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:42.529724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.413622+0000","last_change":"2023-10-18T08:57:14.372676+0000","last_active":"2023-10-18T08:57:15.413622+0000","last_peered":"2023-10-18T08:57:15.413622+0000","last_clean":"2023-10-18T08:57:15.413622+0000","last_became_active":"2023-10-18T08:57:14.372331+0000","last_became_peered":"2023-10-18T08:57:14.372331+0000","last_unstale":"2023-10-18T08:57:15.413622+0000","last_undegraded":"2023-10-18T08:57:15.413622+0000","last_fullsized":"2023-10-18T08:57:15.413622+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:00.621751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302286+0000","last_change":"2023-10-18T08:57:15.409908+0000","last_active":"2023-10-18T08:57:17.302286+0000","last_peered":"2023-10-18T08:57:17.302286+0000","last_clean":"2023-10-18T08:57:17.302286+0000","last_became_active":"2023-10-18T08:57:15.409185+0000","last_became_peered":"2023-10-18T08:57:15.409185+0000","last_unstale":"2023-10-18T08:57:17.302286+0000","last_undegraded":"2023-10-18T08:57:17.302286+0000","last_fullsized":"2023-10-18T08:57:17.302286+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:28.288612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.375123+0000","last_change":"2023-10-18T08:57:14.367951+0000","last_active":"2023-10-18T08:57:15.375123+0000","last_peered":"2023-10-18T08:57:15.375123+0000","last_clean":"2023-10-18T08:57:15.375123+0000","last_became_active":"2023-10-18T08:57:14.367667+0000","last_became_peered":"2023-10-18T08:57:14.367667+0000","last_unstale":"2023-10-18T08:57:15.375123+0000","last_undegraded":"2023-10-18T08:57:15.375123+0000","last_fullsized":"2023-10-18T08:57:15.375123+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:00.621751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.185225+0000","last_change":"2023-10-18T08:57:15.405177+0000","last_active":"2023-10-18T08:57:17.185225+0000","last_peered":"2023-10-18T08:57:17.185225+0000","last_clean":"2023-10-18T08:57:17.185225+0000","last_became_active":"2023-10-18T08:57:15.403347+0000","last_became_peered":"2023-10-18T08:57:15.403347+0000","last_unstale":"2023-10-18T08:57:17.185225+0000","last_undegraded":"2023-10-18T08:57:17.185225+0000","last_fullsized":"2023-10-18T08:57:17.185225+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:59:53.559154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.405454+0000","last_change":"2023-10-18T08:57:14.371281+0000","last_active":"2023-10-18T08:57:15.405454+0000","last_peered":"2023-10-18T08:57:15.405454+0000","last_clean":"2023-10-18T08:57:15.405454+0000","last_became_active":"2023-10-18T08:57:14.370996+0000","last_became_peered":"2023-10-18T08:57:14.370996+0000","last_unstale":"2023-10-18T08:57:15.405454+0000","last_undegraded":"2023-10-18T08:57:15.405454+0000","last_fullsized":"2023-10-18T08:57:15.405454+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:08.232425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.377294+0000","last_change":"2023-10-18T08:57:15.409498+0000","last_active":"2023-10-18T08:57:16.377294+0000","last_peered":"2023-10-18T08:57:16.377294+0000","last_clean":"2023-10-18T08:57:16.377294+0000","last_became_active":"2023-10-18T08:57:15.408070+0000","last_became_peered":"2023-10-18T08:57:15.408070+0000","last_unstale":"2023-10-18T08:57:16.377294+0000","last_undegraded":"2023-10-18T08:57:16.377294+0000","last_fullsized":"2023-10-18T08:57:16.377294+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:10.615157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.385579+0000","last_change":"2023-10-18T08:57:14.363641+0000","last_active":"2023-10-18T08:57:15.385579+0000","last_peered":"2023-10-18T08:57:15.385579+0000","last_clean":"2023-10-18T08:57:15.385579+0000","last_became_active":"2023-10-18T08:57:14.359826+0000","last_became_peered":"2023-10-18T08:57:14.359826+0000","last_unstale":"2023-10-18T08:57:15.385579+0000","last_undegraded":"2023-10-18T08:57:15.385579+0000","last_fullsized":"2023-10-18T08:57:15.385579+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:08.465913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.375358+0000","last_change":"2023-10-18T08:57:15.380648+0000","last_active":"2023-10-18T08:57:16.375358+0000","last_peered":"2023-10-18T08:57:16.375358+0000","last_clean":"2023-10-18T08:57:16.375358+0000","last_became_active":"2023-10-18T08:57:15.380076+0000","last_became_peered":"2023-10-18T08:57:15.380076+0000","last_unstale":"2023-10-18T08:57:16.375358+0000","last_undegraded":"2023-10-18T08:57:16.375358+0000","last_fullsized":"2023-10-18T08:57:16.375358+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:10.032183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.398119+0000","last_change":"2023-10-18T08:57:14.369502+0000","last_active":"2023-10-18T08:57:15.398119+0000","last_peered":"2023-10-18T08:57:15.398119+0000","last_clean":"2023-10-18T08:57:15.398119+0000","last_became_active":"2023-10-18T08:57:14.369140+0000","last_became_peered":"2023-10-18T08:57:14.369140+0000","last_unstale":"2023-10-18T08:57:15.398119+0000","last_undegraded":"2023-10-18T08:57:15.398119+0000","last_fullsized":"2023-10-18T08:57:15.398119+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:08.465913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.372553+0000","last_change":"2023-10-18T08:57:15.395926+0000","last_active":"2023-10-18T08:57:16.372553+0000","last_peered":"2023-10-18T08:57:16.372553+0000","last_clean":"2023-10-18T08:57:16.372553+0000","last_became_active":"2023-10-18T08:57:15.394966+0000","last_became_peered":"2023-10-18T08:57:15.394966+0000","last_unstale":"2023-10-18T08:57:16.372553+0000","last_undegraded":"2023-10-18T08:57:16.372553+0000","last_fullsized":"2023-10-18T08:57:16.372553+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:48:28.930263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.411915+0000","last_change":"2023-10-18T08:57:14.372079+0000","last_active":"2023-10-18T08:57:15.411915+0000","last_peered":"2023-10-18T08:57:15.411915+0000","last_clean":"2023-10-18T08:57:15.411915+0000","last_became_active":"2023-10-18T08:57:14.371832+0000","last_became_peered":"2023-10-18T08:57:14.371832+0000","last_unstale":"2023-10-18T08:57:15.411915+0000","last_undegraded":"2023-10-18T08:57:15.411915+0000","last_fullsized":"2023-10-18T08:57:15.411915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:00.621751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.375316+0000","last_change":"2023-10-18T08:57:15.401651+0000","last_active":"2023-10-18T08:57:16.375316+0000","last_peered":"2023-10-18T08:57:16.375316+0000","last_clean":"2023-10-18T08:57:16.375316+0000","last_became_active":"2023-10-18T08:57:15.400913+0000","last_became_peered":"2023-10-18T08:57:15.400913+0000","last_unstale":"2023-10-18T08:57:16.375316+0000","last_undegraded":"2023-10-18T08:57:16.375316+0000","last_fullsized":"2023-10-18T08:57:16.375316+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:32:51.933660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.418145+0000","last_change":"2023-10-18T08:57:14.358993+0000","last_active":"2023-10-18T08:57:15.418145+0000","last_peered":"2023-10-18T08:57:15.418145+0000","last_clean":"2023-10-18T08:57:15.418145+0000","last_became_active":"2023-10-18T08:57:14.358684+0000","last_became_peered":"2023-10-18T08:57:14.358684+0000","last_unstale":"2023-10-18T08:57:15.418145+0000","last_undegraded":"2023-10-18T08:57:15.418145+0000","last_fullsized":"2023-10-18T08:57:15.418145+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:31:16.160688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.184985+0000","last_change":"2023-10-18T08:57:15.402958+0000","last_active":"2023-10-18T08:57:17.184985+0000","last_peered":"2023-10-18T08:57:17.184985+0000","last_clean":"2023-10-18T08:57:17.184985+0000","last_became_active":"2023-10-18T08:57:15.402696+0000","last_became_peered":"2023-10-18T08:57:15.402696+0000","last_unstale":"2023-10-18T08:57:17.184985+0000","last_undegraded":"2023-10-18T08:57:17.184985+0000","last_fullsized":"2023-10-18T08:57:17.184985+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:09.477838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.412789+0000","last_change":"2023-10-18T08:57:14.363761+0000","last_active":"2023-10-18T08:57:15.412789+0000","last_peered":"2023-10-18T08:57:15.412789+0000","last_clean":"2023-10-18T08:57:15.412789+0000","last_became_active":"2023-10-18T08:57:14.363321+0000","last_became_peered":"2023-10-18T08:57:14.363321+0000","last_unstale":"2023-10-18T08:57:15.412789+0000","last_undegraded":"2023-10-18T08:57:15.412789+0000","last_fullsized":"2023-10-18T08:57:15.412789+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:50:33.631395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302695+0000","last_change":"2023-10-18T08:57:15.407663+0000","last_active":"2023-10-18T08:57:17.302695+0000","last_peered":"2023-10-18T08:57:17.302695+0000","last_clean":"2023-10-18T08:57:17.302695+0000","last_became_active":"2023-10-18T08:57:15.407129+0000","last_became_peered":"2023-10-18T08:57:15.407129+0000","last_unstale":"2023-10-18T08:57:17.302695+0000","last_undegraded":"2023-10-18T08:57:17.302695+0000","last_fullsized":"2023-10-18T08:57:17.302695+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:10.615157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.376766+0000","last_change":"2023-10-18T08:57:14.367626+0000","last_active":"2023-10-18T08:57:15.376766+0000","last_peered":"2023-10-18T08:57:15.376766+0000","last_clean":"2023-10-18T08:57:15.376766+0000","last_became_active":"2023-10-18T08:57:14.364913+0000","last_became_peered":"2023-10-18T08:57:14.364913+0000","last_unstale":"2023-10-18T08:57:15.376766+0000","last_undegraded":"2023-10-18T08:57:15.376766+0000","last_fullsized":"2023-10-18T08:57:15.376766+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:27.276687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.372408+0000","last_change":"2023-10-18T08:57:15.401548+0000","last_active":"2023-10-18T08:57:16.372408+0000","last_peered":"2023-10-18T08:57:16.372408+0000","last_clean":"2023-10-18T08:57:16.372408+0000","last_became_active":"2023-10-18T08:57:15.400108+0000","last_became_peered":"2023-10-18T08:57:15.400108+0000","last_unstale":"2023-10-18T08:57:16.372408+0000","last_undegraded":"2023-10-18T08:57:16.372408+0000","last_fullsized":"2023-10-18T08:57:16.372408+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:58.321274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.371185+0000","last_change":"2023-10-18T08:57:14.368186+0000","last_active":"2023-10-18T08:57:15.371185+0000","last_peered":"2023-10-18T08:57:15.371185+0000","last_clean":"2023-10-18T08:57:15.371185+0000","last_became_active":"2023-10-18T08:57:14.367105+0000","last_became_peered":"2023-10-18T08:57:14.367105+0000","last_unstale":"2023-10-18T08:57:15.371185+0000","last_undegraded":"2023-10-18T08:57:15.371185+0000","last_fullsized":"2023-10-18T08:57:15.371185+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:32:50.921734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.374771+0000","last_change":"2023-10-18T08:57:15.406911+0000","last_active":"2023-10-18T08:57:16.374771+0000","last_peered":"2023-10-18T08:57:16.374771+0000","last_clean":"2023-10-18T08:57:16.374771+0000","last_became_active":"2023-10-18T08:57:15.404857+0000","last_became_peered":"2023-10-18T08:57:15.404857+0000","last_unstale":"2023-10-18T08:57:16.374771+0000","last_undegraded":"2023-10-18T08:57:16.374771+0000","last_fullsized":"2023-10-18T08:57:16.374771+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:29:38.697110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.418242+0000","last_change":"2023-10-18T08:57:14.370165+0000","last_active":"2023-10-18T08:57:15.418242+0000","last_peered":"2023-10-18T08:57:15.418242+0000","last_clean":"2023-10-18T08:57:15.418242+0000","last_became_active":"2023-10-18T08:57:14.369906+0000","last_became_peered":"2023-10-18T08:57:14.369906+0000","last_unstale":"2023-10-18T08:57:15.418242+0000","last_undegraded":"2023-10-18T08:57:15.418242+0000","last_fullsized":"2023-10-18T08:57:15.418242+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:01.758247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.091164+0000","last_change":"2023-10-18T08:57:15.398384+0000","last_active":"2023-10-18T08:57:18.091164+0000","last_peered":"2023-10-18T08:57:18.091164+0000","last_clean":"2023-10-18T08:57:18.091164+0000","last_became_active":"2023-10-18T08:57:15.398048+0000","last_became_peered":"2023-10-18T08:57:15.398048+0000","last_unstale":"2023-10-18T08:57:18.091164+0000","last_undegraded":"2023-10-18T08:57:18.091164+0000","last_fullsized":"2023-10-18T08:57:18.091164+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:21:32.285864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.416385+0000","last_change":"2023-10-18T08:57:14.355217+0000","last_active":"2023-10-18T08:57:15.416385+0000","last_peered":"2023-10-18T08:57:15.416385+0000","last_clean":"2023-10-18T08:57:15.416385+0000","last_became_active":"2023-10-18T08:57:14.354864+0000","last_became_peered":"2023-10-18T08:57:14.354864+0000","last_unstale":"2023-10-18T08:57:15.416385+0000","last_undegraded":"2023-10-18T08:57:15.416385+0000","last_fullsized":"2023-10-18T08:57:15.416385+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:19.909678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302640+0000","last_change":"2023-10-18T08:57:15.391160+0000","last_active":"2023-10-18T08:57:17.302640+0000","last_peered":"2023-10-18T08:57:17.302640+0000","last_clean":"2023-10-18T08:57:17.302640+0000","last_became_active":"2023-10-18T08:57:15.390916+0000","last_became_peered":"2023-10-18T08:57:15.390916+0000","last_unstale":"2023-10-18T08:57:17.302640+0000","last_undegraded":"2023-10-18T08:57:17.302640+0000","last_fullsized":"2023-10-18T08:57:17.302640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:29:38.697110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.414515+0000","last_change":"2023-10-18T08:57:14.366838+0000","last_active":"2023-10-18T08:57:15.414515+0000","last_peered":"2023-10-18T08:57:15.414515+0000","last_clean":"2023-10-18T08:57:15.414515+0000","last_became_active":"2023-10-18T08:57:14.366429+0000","last_became_peered":"2023-10-18T08:57:14.366429+0000","last_unstale":"2023-10-18T08:57:15.414515+0000","last_undegraded":"2023-10-18T08:57:15.414515+0000","last_fullsized":"2023-10-18T08:57:15.414515+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:20:37.681848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.184345+0000","last_change":"2023-10-18T08:57:15.401580+0000","last_active":"2023-10-18T08:57:17.184345+0000","last_peered":"2023-10-18T08:57:17.184345+0000","last_clean":"2023-10-18T08:57:17.184345+0000","last_became_active":"2023-10-18T08:57:15.401353+0000","last_became_peered":"2023-10-18T08:57:15.401353+0000","last_unstale":"2023-10-18T08:57:17.184345+0000","last_undegraded":"2023-10-18T08:57:17.184345+0000","last_fullsized":"2023-10-18T08:57:17.184345+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:38.943057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.407906+0000","last_change":"2023-10-18T08:57:14.370663+0000","last_active":"2023-10-18T08:57:15.407906+0000","last_peered":"2023-10-18T08:57:15.407906+0000","last_clean":"2023-10-18T08:57:15.407906+0000","last_became_active":"2023-10-18T08:57:14.370383+0000","last_became_peered":"2023-10-18T08:57:14.370383+0000","last_unstale":"2023-10-18T08:57:15.407906+0000","last_undegraded":"2023-10-18T08:57:15.407906+0000","last_fullsized":"2023-10-18T08:57:15.407906+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:34:18.947640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.185278+0000","last_change":"2023-10-18T08:57:15.398017+0000","last_active":"2023-10-18T08:57:17.185278+0000","last_peered":"2023-10-18T08:57:17.185278+0000","last_clean":"2023-10-18T08:57:17.185278+0000","last_became_active":"2023-10-18T08:57:15.397794+0000","last_became_peered":"2023-10-18T08:57:15.397794+0000","last_unstale":"2023-10-18T08:57:17.185278+0000","last_undegraded":"2023-10-18T08:57:17.185278+0000","last_fullsized":"2023-10-18T08:57:17.185278+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:29.370314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.371526+0000","last_change":"2023-10-18T08:57:14.371953+0000","last_active":"2023-10-18T08:57:15.371526+0000","last_peered":"2023-10-18T08:57:15.371526+0000","last_clean":"2023-10-18T08:57:15.371526+0000","last_became_active":"2023-10-18T08:57:14.371639+0000","last_became_peered":"2023-10-18T08:57:14.371639+0000","last_unstale":"2023-10-18T08:57:15.371526+0000","last_undegraded":"2023-10-18T08:57:15.371526+0000","last_fullsized":"2023-10-18T08:57:15.371526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:09.020258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.372813+0000","last_change":"2023-10-18T08:57:15.395276+0000","last_active":"2023-10-18T08:57:16.372813+0000","last_peered":"2023-10-18T08:57:16.372813+0000","last_clean":"2023-10-18T08:57:16.372813+0000","last_became_active":"2023-10-18T08:57:15.394603+0000","last_became_peered":"2023-10-18T08:57:15.394603+0000","last_unstale":"2023-10-18T08:57:16.372813+0000","last_undegraded":"2023-10-18T08:57:16.372813+0000","last_fullsized":"2023-10-18T08:57:16.372813+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:34:00.421723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.413991+0000","last_change":"2023-10-18T08:57:14.356889+0000","last_active":"2023-10-18T08:57:15.413991+0000","last_peered":"2023-10-18T08:57:15.413991+0000","last_clean":"2023-10-18T08:57:15.413991+0000","last_became_active":"2023-10-18T08:57:14.355993+0000","last_became_peered":"2023-10-18T08:57:14.355993+0000","last_unstale":"2023-10-18T08:57:15.413991+0000","last_undegraded":"2023-10-18T08:57:15.413991+0000","last_fullsized":"2023-10-18T08:57:15.413991+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:05:07.672779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.408528+0000","last_change":"2023-10-18T08:57:15.399901+0000","last_active":"2023-10-18T08:57:16.408528+0000","last_peered":"2023-10-18T08:57:16.408528+0000","last_clean":"2023-10-18T08:57:16.408528+0000","last_became_active":"2023-10-18T08:57:15.399653+0000","last_became_peered":"2023-10-18T08:57:15.399653+0000","last_unstale":"2023-10-18T08:57:16.408528+0000","last_undegraded":"2023-10-18T08:57:16.408528+0000","last_fullsized":"2023-10-18T08:57:16.408528+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:29.370314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.375434+0000","last_change":"2023-10-18T08:57:14.371375+0000","last_active":"2023-10-18T08:57:15.375434+0000","last_peered":"2023-10-18T08:57:15.375434+0000","last_clean":"2023-10-18T08:57:15.375434+0000","last_became_active":"2023-10-18T08:57:14.371143+0000","last_became_peered":"2023-10-18T08:57:14.371143+0000","last_unstale":"2023-10-18T08:57:15.375434+0000","last_undegraded":"2023-10-18T08:57:15.375434+0000","last_fullsized":"2023-10-18T08:57:15.375434+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:33.369537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.090822+0000","last_change":"2023-10-18T08:57:15.397007+0000","last_active":"2023-10-18T08:57:18.090822+0000","last_peered":"2023-10-18T08:57:18.090822+0000","last_clean":"2023-10-18T08:57:18.090822+0000","last_became_active":"2023-10-18T08:57:15.396752+0000","last_became_peered":"2023-10-18T08:57:15.396752+0000","last_unstale":"2023-10-18T08:57:18.090822+0000","last_undegraded":"2023-10-18T08:57:18.090822+0000","last_fullsized":"2023-10-18T08:57:18.090822+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:45.669294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.405608+0000","last_change":"2023-10-18T08:57:14.369401+0000","last_active":"2023-10-18T08:57:15.405608+0000","last_peered":"2023-10-18T08:57:15.405608+0000","last_clean":"2023-10-18T08:57:15.405608+0000","last_became_active":"2023-10-18T08:57:14.369163+0000","last_became_peered":"2023-10-18T08:57:14.369163+0000","last_unstale":"2023-10-18T08:57:15.405608+0000","last_undegraded":"2023-10-18T08:57:15.405608+0000","last_fullsized":"2023-10-18T08:57:15.405608+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:07.495221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302907+0000","last_change":"2023-10-18T08:57:15.407547+0000","last_active":"2023-10-18T08:57:17.302907+0000","last_peered":"2023-10-18T08:57:17.302907+0000","last_clean":"2023-10-18T08:57:17.302907+0000","last_became_active":"2023-10-18T08:57:15.406795+0000","last_became_peered":"2023-10-18T08:57:15.406795+0000","last_unstale":"2023-10-18T08:57:17.302907+0000","last_undegraded":"2023-10-18T08:57:17.302907+0000","last_fullsized":"2023-10-18T08:57:17.302907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:32:51.933660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.377090+0000","last_change":"2023-10-18T08:57:14.368515+0000","last_active":"2023-10-18T08:57:15.377090+0000","last_peered":"2023-10-18T08:57:15.377090+0000","last_clean":"2023-10-18T08:57:15.377090+0000","last_became_active":"2023-10-18T08:57:14.368173+0000","last_became_peered":"2023-10-18T08:57:14.368173+0000","last_unstale":"2023-10-18T08:57:15.377090+0000","last_undegraded":"2023-10-18T08:57:15.377090+0000","last_fullsized":"2023-10-18T08:57:15.377090+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:05:07.672779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.184543+0000","last_change":"2023-10-18T08:57:15.409166+0000","last_active":"2023-10-18T08:57:17.184543+0000","last_peered":"2023-10-18T08:57:17.184543+0000","last_clean":"2023-10-18T08:57:17.184543+0000","last_became_active":"2023-10-18T08:57:15.408364+0000","last_became_peered":"2023-10-18T08:57:15.408364+0000","last_unstale":"2023-10-18T08:57:17.184543+0000","last_undegraded":"2023-10-18T08:57:17.184543+0000","last_fullsized":"2023-10-18T08:57:17.184543+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:08:15.503788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.373048+0000","last_change":"2023-10-18T08:57:14.366229+0000","last_active":"2023-10-18T08:57:16.373048+0000","last_peered":"2023-10-18T08:57:16.373048+0000","last_clean":"2023-10-18T08:57:16.373048+0000","last_became_active":"2023-10-18T08:57:14.365496+0000","last_became_peered":"2023-10-18T08:57:14.365496+0000","last_unstale":"2023-10-18T08:57:16.373048+0000","last_undegraded":"2023-10-18T08:57:16.373048+0000","last_fullsized":"2023-10-18T08:57:16.373048+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:09.603232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.303076+0000","last_change":"2023-10-18T08:57:15.407398+0000","last_active":"2023-10-18T08:57:17.303076+0000","last_peered":"2023-10-18T08:57:17.303076+0000","last_clean":"2023-10-18T08:57:17.303076+0000","last_became_active":"2023-10-18T08:57:15.406461+0000","last_became_peered":"2023-10-18T08:57:15.406461+0000","last_unstale":"2023-10-18T08:57:17.303076+0000","last_undegraded":"2023-10-18T08:57:17.303076+0000","last_fullsized":"2023-10-18T08:57:17.303076+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:10.032183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.373268+0000","last_change":"2023-10-18T08:57:14.373352+0000","last_active":"2023-10-18T08:57:15.373268+0000","last_peered":"2023-10-18T08:57:15.373268+0000","last_clean":"2023-10-18T08:57:15.373268+0000","last_became_active":"2023-10-18T08:57:14.373089+0000","last_became_peered":"2023-10-18T08:57:14.373089+0000","last_unstale":"2023-10-18T08:57:15.373268+0000","last_undegraded":"2023-10-18T08:57:15.373268+0000","last_fullsized":"2023-10-18T08:57:15.373268+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:07.495221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.376907+0000","last_change":"2023-10-18T08:57:15.395508+0000","last_active":"2023-10-18T08:57:16.376907+0000","last_peered":"2023-10-18T08:57:16.376907+0000","last_clean":"2023-10-18T08:57:16.376907+0000","last_became_active":"2023-10-18T08:57:15.395260+0000","last_became_peered":"2023-10-18T08:57:15.395260+0000","last_unstale":"2023-10-18T08:57:16.376907+0000","last_undegraded":"2023-10-18T08:57:16.376907+0000","last_fullsized":"2023-10-18T08:57:16.376907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:05:27.740292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.392360+0000","last_change":"2023-10-18T08:57:14.372847+0000","last_active":"2023-10-18T08:57:15.392360+0000","last_peered":"2023-10-18T08:57:15.392360+0000","last_clean":"2023-10-18T08:57:15.392360+0000","last_became_active":"2023-10-18T08:57:14.372531+0000","last_became_peered":"2023-10-18T08:57:14.372531+0000","last_unstale":"2023-10-18T08:57:15.392360+0000","last_undegraded":"2023-10-18T08:57:15.392360+0000","last_fullsized":"2023-10-18T08:57:15.392360+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:58:08.465913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.372564+0000","last_change":"2023-10-18T08:57:15.401027+0000","last_active":"2023-10-18T08:57:16.372564+0000","last_peered":"2023-10-18T08:57:16.372564+0000","last_clean":"2023-10-18T08:57:16.372564+0000","last_became_active":"2023-10-18T08:57:15.400790+0000","last_became_peered":"2023-10-18T08:57:15.400790+0000","last_unstale":"2023-10-18T08:57:16.372564+0000","last_undegraded":"2023-10-18T08:57:16.372564+0000","last_fullsized":"2023-10-18T08:57:16.372564+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:59.252136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.377407+0000","last_change":"2023-10-18T08:57:14.373541+0000","last_active":"2023-10-18T08:57:15.377407+0000","last_peered":"2023-10-18T08:57:15.377407+0000","last_clean":"2023-10-18T08:57:15.377407+0000","last_became_active":"2023-10-18T08:57:14.372892+0000","last_became_peered":"2023-10-18T08:57:14.372892+0000","last_unstale":"2023-10-18T08:57:15.377407+0000","last_undegraded":"2023-10-18T08:57:15.377407+0000","last_fullsized":"2023-10-18T08:57:15.377407+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:21.701063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.373625+0000","last_change":"2023-10-18T08:57:15.395152+0000","last_active":"2023-10-18T08:57:16.373625+0000","last_peered":"2023-10-18T08:57:16.373625+0000","last_clean":"2023-10-18T08:57:16.373625+0000","last_became_active":"2023-10-18T08:57:15.394903+0000","last_became_peered":"2023-10-18T08:57:15.394903+0000","last_unstale":"2023-10-18T08:57:16.373625+0000","last_undegraded":"2023-10-18T08:57:16.373625+0000","last_fullsized":"2023-10-18T08:57:16.373625+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:30:16.102793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.387178+0000","last_change":"2023-10-18T08:57:14.358925+0000","last_active":"2023-10-18T08:57:15.387178+0000","last_peered":"2023-10-18T08:57:15.387178+0000","last_clean":"2023-10-18T08:57:15.387178+0000","last_became_active":"2023-10-18T08:57:14.355112+0000","last_became_peered":"2023-10-18T08:57:14.355112+0000","last_unstale":"2023-10-18T08:57:15.387178+0000","last_undegraded":"2023-10-18T08:57:15.387178+0000","last_fullsized":"2023-10-18T08:57:15.387178+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:30.531145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:18.090585+0000","last_change":"2023-10-18T08:57:15.399703+0000","last_active":"2023-10-18T08:57:18.090585+0000","last_peered":"2023-10-18T08:57:18.090585+0000","last_clean":"2023-10-18T08:57:18.090585+0000","last_became_active":"2023-10-18T08:57:15.399469+0000","last_became_peered":"2023-10-18T08:57:15.399469+0000","last_unstale":"2023-10-18T08:57:18.090585+0000","last_undegraded":"2023-10-18T08:57:18.090585+0000","last_fullsized":"2023-10-18T08:57:18.090585+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:16.125626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.387030+0000","last_change":"2023-10-18T08:57:14.374953+0000","last_active":"2023-10-18T08:57:15.387030+0000","last_peered":"2023-10-18T08:57:15.387030+0000","last_clean":"2023-10-18T08:57:15.387030+0000","last_became_active":"2023-10-18T08:57:14.374674+0000","last_became_peered":"2023-10-18T08:57:14.374674+0000","last_unstale":"2023-10-18T08:57:15.387030+0000","last_undegraded":"2023-10-18T08:57:15.387030+0000","last_fullsized":"2023-10-18T08:57:15.387030+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:05.351228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:16.376178+0000","last_change":"2023-10-18T08:57:15.409873+0000","last_active":"2023-10-18T08:57:16.376178+0000","last_peered":"2023-10-18T08:57:16.376178+0000","last_clean":"2023-10-18T08:57:16.376178+0000","last_became_active":"2023-10-18T08:57:15.408656+0000","last_became_peered":"2023-10-18T08:57:15.408656+0000","last_unstale":"2023-10-18T08:57:16.376178+0000","last_undegraded":"2023-10-18T08:57:16.376178+0000","last_fullsized":"2023-10-18T08:57:16.376178+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:28.288612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.390512+0000","last_change":"2023-10-18T08:57:14.367001+0000","last_active":"2023-10-18T08:57:15.390512+0000","last_peered":"2023-10-18T08:57:15.390512+0000","last_clean":"2023-10-18T08:57:15.390512+0000","last_became_active":"2023-10-18T08:57:14.366771+0000","last_became_peered":"2023-10-18T08:57:14.366771+0000","last_unstale":"2023-10-18T08:57:15.390512+0000","last_undegraded":"2023-10-18T08:57:15.390512+0000","last_fullsized":"2023-10-18T08:57:15.390512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:08:14.491862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-18T08:57:17.302280+0000","last_change":"2023-10-18T08:57:15.392737+0000","last_active":"2023-10-18T08:57:17.302280+0000","last_peered":"2023-10-18T08:57:17.302280+0000","last_clean":"2023-10-18T08:57:17.302280+0000","last_became_active":"2023-10-18T08:57:15.392473+0000","last_became_peered":"2023-10-18T08:57:15.392473+0000","last_unstale":"2023-10-18T08:57:17.302280+0000","last_undegraded":"2023-10-18T08:57:17.302280+0000","last_fullsized":"2023-10-18T08:57:17.302280+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:14.334221+0000","last_clean_scrub_stamp":"2023-10-18T08:57:14.334221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:16.125626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.412339+0000","last_change":"2023-10-18T08:57:14.372410+0000","last_active":"2023-10-18T08:57:15.412339+0000","last_peered":"2023-10-18T08:57:15.412339+0000","last_clean":"2023-10-18T08:57:15.412339+0000","last_became_active":"2023-10-18T08:57:14.372097+0000","last_became_peered":"2023-10-18T08:57:14.372097+0000","last_unstale":"2023-10-18T08:57:15.412339+0000","last_undegraded":"2023-10-18T08:57:15.412339+0000","last_fullsized":"2023-10-18T08:57:15.412339+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:57:13.322296+0000","last_clean_scrub_stamp":"2023-10-18T08:57:13.322296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:08.232425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":192,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-18T08:57:15.376058+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:57:15.376058+0000","last_peered":"2023-10-18T08:57:15.376058+0000","last_clean":"2023-10-18T08:57:15.376058+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:57:15.376058+0000","last_undegraded":"2023-10-18T08:57:15.376058+0000","last_fullsized":"2023-10-18T08:57:15.376058+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:34.881744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":793,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194519,"statfs":{"total":200000000,"available":199187764,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.998,"15min":0.98999999999999999},"min":{"1min":0.67800000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3979999999999999,"5min":1.538,"15min":1.538},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.087,"5min":1.014,"15min":1.002},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":1.2689999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1399999999999999,"15min":1.1519999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.393,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.123,"15min":1.1160000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7849999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.484}]},{"osd":2,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.218,"15min":1.224},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.589,"5min":1.694,"15min":1.694},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1839999999999999,"15min":1.1859999999999999},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.1259999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.254,"15min":1.2609999999999999},"min":{"1min":0.91500000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.72,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.236},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3089999999999999,"15min":1.3129999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.8029999999999999,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.6040000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.1859999999999999,"15min":1.1579999999999999},"min":{"1min":0.92800000000000005,"5min":0.746,"15min":0.746},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.522},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.407,"15min":1.411},"min":{"1min":1.0369999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.885,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.095}]},{"osd":5,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3520000000000001,"15min":1.353},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.615,"5min":1.893,"15min":1.893},"last":1.319},{"interface":"front","average":{"1min":1.389,"5min":1.417,"15min":1.421},"min":{"1min":1.0940000000000001,"5min":0.874,"15min":0.874},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.764}]},{"osd":6,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.464,"15min":1.458},"min":{"1min":1.0169999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.9339999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.585},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.516,"15min":1.518},"min":{"1min":1.23,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.978,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.4410000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":217,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195095,"statfs":{"total":200000000,"available":199777550,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0800000000000001,"15min":1.073},"min":{"1min":0.65400000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.5529999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.133,"5min":1.216,"15min":1.23},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":1.391}]},{"osd":1,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0780000000000001,"15min":1.081},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.3420000000000001,"5min":1.524,"15min":1.524},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.1659999999999999,"15min":1.1819999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.474,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":1.29}]},{"osd":2,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.1579999999999999,"15min":1.1719999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.23},{"interface":"front","average":{"1min":1.165,"5min":1.1870000000000001,"15min":1.1910000000000001},"min":{"1min":0.82099999999999995,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5309999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.629}]},{"osd":3,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2509999999999999,"15min":1.2569999999999999},"min":{"1min":0.84799999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.502,"5min":1.784,"15min":1.784},"last":1.669},{"interface":"front","average":{"1min":1.222,"5min":1.2170000000000001,"15min":1.216},"min":{"1min":1.0329999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.494,"5min":1.5,"15min":1.5},"last":1.347}]},{"osd":4,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.1950000000000001,"15min":1.169},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.272,"15min":1.27},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6990000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.4350000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3560000000000001,"15min":1.3500000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.665,"5min":1.667,"15min":1.667},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.2330000000000001,"15min":1.2170000000000001},"min":{"1min":0.54900000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":1.4910000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:57:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4019999999999999,"15min":1.3959999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.7050000000000001},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5249999999999999,"15min":1.53},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.8500000000000001,"5min":2.794,"15min":2.794},"last":1.742}]}]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":216,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195095,"statfs":{"total":200000000,"available":199778152,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2010000000000001,"15min":1.1850000000000001},"min":{"1min":0.77800000000000002,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.7,"5min":1.915,"15min":1.915},"last":1.002},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.288,"15min":1.292},"min":{"1min":0.746,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7130000000000001,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.05}]},{"osd":1,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.2030000000000001,"15min":1.2210000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.613,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.236},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.76000000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.581,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.95499999999999996}]},{"osd":2,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2689999999999999,"15min":1.268},"min":{"1min":0.77700000000000002,"5min":0.755,"15min":0.755},"max":{"1min":1.7749999999999999,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.29,"15min":1.2929999999999999},"min":{"1min":0.96099999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.7210000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.84099999999999997}]},{"osd":3,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.232,"15min":1.24},"min":{"1min":0.84799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5640000000000001,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.321,"15min":1.327},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.784,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.3859999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.417,"15min":1.4199999999999999},"min":{"1min":0.84899999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8799999999999999,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.659},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.5069999999999999,"15min":1.5369999999999999},"min":{"1min":0.90100000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7929999999999999,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":1.4359999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.3839999999999999,"15min":1.4139999999999999},"min":{"1min":0.873,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6799999999999999,"5min":5.141,"15min":5.141},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.5720000000000001,"15min":1.5920000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.931,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.7609999999999999}]},{"osd":7,"last update":"Wed Oct 18 08:57:08 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.661,"15min":1.677},"min":{"1min":1.038,"5min":1.014,"15min":1.014},"max":{"1min":1.9990000000000001,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":1.329},{"interface":"front","average":{"1min":1.423,"5min":1.508,"15min":1.522},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.907,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":1.4770000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":217,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195095,"statfs":{"total":200000000,"available":199777629,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.105,"15min":1.109},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.081,"5min":1.1679999999999999,"15min":1.1819999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0659999999999998,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":0.97199999999999998}]},{"osd":1,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.0980000000000001,"15min":1.095},"min":{"1min":0.76100000000000001,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.583,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.2310000000000001,"15min":1.2410000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.929,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.0389999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.1779999999999999,"15min":1.1759999999999999},"min":{"1min":0.75600000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.29,"15min":1.3100000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.601,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.4730000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.292,"15min":1.302},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.232},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4159999999999999,"15min":1.413},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.0899999999999999,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.5580000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3120000000000001,"15min":1.3},"min":{"1min":0.84099999999999997,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.8080000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.365,"15min":1.3720000000000001},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":1.3859999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.397,"15min":1.3819999999999999},"min":{"1min":1.169,"5min":0.998,"15min":0.998},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.51,"5min":1.5680000000000001,"15min":1.5780000000000001},"min":{"1min":1.04,"5min":1.04,"15min":1.04},"max":{"1min":1.8939999999999999,"5min":3.6859999999999999,"15min":3.6859999999999999},"last":1.6120000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"max":{"1min":2.2759999999999998,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.6459999999999999},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.5629999999999999,"15min":1.554},"min":{"1min":1.3089999999999999,"5min":1.23,"15min":1.23},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.4279999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":217,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195095,"statfs":{"total":200000000,"available":199777584,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":0.82399999999999995,"15min":0.76500000000000001},"min":{"1min":0.52500000000000002,"5min":0.439,"15min":0.439},"max":{"1min":5.5730000000000004,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.365,"5min":0.93999999999999995,"15min":0.86899999999999999},"min":{"1min":0.53700000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":6.5599999999999996,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":0.72699999999999998}]},{"osd":1,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":0.98499999999999999,"15min":0.94399999999999995},"min":{"1min":0.59399999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":6.8099999999999996,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.238,"5min":1.1259999999999999,"15min":1.1080000000000001},"min":{"1min":0.69099999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.3959999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.0620000000000001,"15min":1.0169999999999999},"min":{"1min":0.61199999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":7.4580000000000002,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.0880000000000001,"15min":1.048},"min":{"1min":0.56699999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":3.5110000000000001,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":1.4730000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":2.3849999999999998,"5min":1.4570000000000001,"15min":1.3029999999999999},"min":{"1min":0.99099999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":18.664999999999999,"5min":18.664999999999999,"15min":18.664999999999999},"last":1.429},{"interface":"front","average":{"1min":1.47,"5min":1.1899999999999999,"15min":1.1439999999999999},"min":{"1min":0.83999999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":7.0700000000000003,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":1.1100000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.266,"15min":1.214},"min":{"1min":0.872,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":6.9139999999999997,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":1.161},{"interface":"front","average":{"1min":2.306,"5min":1.4690000000000001,"15min":1.329},"min":{"1min":1.0269999999999999,"5min":0.748,"15min":0.748},"max":{"1min":16.390999999999998,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.5980000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.8049999999999999,"5min":1.4750000000000001,"15min":1.4199999999999999},"min":{"1min":1.022,"5min":0.625,"15min":0.625},"max":{"1min":7.7270000000000003,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":1.536},{"interface":"front","average":{"1min":1.7190000000000001,"5min":1.3819999999999999,"15min":1.3260000000000001},"min":{"1min":0.96499999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":7.242,"5min":7.242,"15min":7.242},"last":1.355}]},{"osd":7,"last update":"Wed Oct 18 08:57:06 2023","interfaces":[{"interface":"back","average":{"1min":1.643,"5min":1.367,"15min":1.321},"min":{"1min":0.94299999999999995,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.7629999999999999},{"interface":"front","average":{"1min":2.1309999999999998,"5min":1.54,"15min":1.4419999999999999},"min":{"1min":0.84999999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":12.946,"5min":12.946,"15min":12.946},"last":1.6439999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195094,"statfs":{"total":200000000,"available":199776588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.92000000000000004,"15min":0.93999999999999995},"min":{"1min":0.433,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.395,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.024,"15min":1.0569999999999999},"min":{"1min":0.48999999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3700000000000001,"5min":1.651,"15min":1.651},"last":0.95399999999999996}]},{"osd":1,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":1.04,"15min":1.0600000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3660000000000001,"5min":4.29,"15min":4.29},"last":1.1950000000000001},{"interface":"front","average":{"1min":0.87,"5min":1.0580000000000001,"15min":1.089},"min":{"1min":0.57499999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3009999999999999,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.173}]},{"osd":3,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.105,"15min":1.133},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.9139999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.4419999999999999,"15min":1.4970000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.609,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":1.3100000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.47,"15min":1.502},"min":{"1min":0.90900000000000003,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.7,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":1.538},{"interface":"front","average":{"1min":1.25,"5min":1.4239999999999999,"15min":1.4530000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.663,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.2370000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.248,"15min":1.272},"min":{"1min":0.79400000000000004,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5389999999999999,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.542,"15min":1.5900000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.7569999999999999,"5min":7.657,"15min":7.657},"last":1.4199999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.4950000000000001,"15min":1.5389999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7170000000000001,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.4059999999999999,"15min":1.4330000000000001},"min":{"1min":0.94099999999999995,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.796,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":1.395}]},{"osd":7,"last update":"Wed Oct 18 08:57:07 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.5449999999999999,"15min":1.5780000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7350000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.46},{"interface":"front","average":{"1min":1.3,"5min":1.4490000000000001,"15min":1.474},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.6899999999999999,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.496}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":215,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195096,"statfs":{"total":200000000,"available":199779171,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.871,"15min":0.88400000000000001},"min":{"1min":0.54400000000000004,"5min":0.443,"15min":0.443},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.075}]},{"osd":2,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.90000000000000002,"15min":0.91900000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.222,"5min":1.859,"15min":1.859},"last":1.014},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.82599999999999996,"15min":0.83299999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.133}]},{"osd":3,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.98499999999999999,"15min":1.0089999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":2.1059999999999999,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.5129999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.024,"15min":1.048},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.7110000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":1.2569999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.224,"15min":1.236},"min":{"1min":0.71499999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.53,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.1830000000000001,"15min":1.1890000000000001},"min":{"1min":0.85999999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.99,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.3380000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.262,"15min":1.2789999999999999},"min":{"1min":0.82899999999999996,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.6299999999999999,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.335,"15min":1.341},"min":{"1min":0.84999999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.8480000000000001,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.6759999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.2529999999999999,"15min":1.2649999999999999},"min":{"1min":0.86199999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.2050000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.143,"5min":1.228,"15min":1.242},"min":{"1min":0.72999999999999998,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.853,"5min":1.853,"15min":1.853},"last":1.569}]},{"osd":7,"last update":"Wed Oct 18 08:57:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.3160000000000001,"15min":1.333},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.0419999999999998,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.657},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.333,"15min":1.3380000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.8169999999999999,"5min":1.994,"15min":1.994},"last":1.758}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":792,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194520,"statfs":{"total":200000000,"available":199188839,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.92600000000000005,"15min":0.94599999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.25,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.077,"15min":1.083},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.4990000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.0860000000000001}]},{"osd":2,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.83199999999999996,"15min":0.83999999999999997},"min":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.339,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.121,"5min":1.125,"15min":1.125},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.5629999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.2849999999999999,"15min":1.327},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.294,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.5109999999999999,"15min":1.5429999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.8640000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.6990000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.2909999999999999,"15min":1.3260000000000001},"min":{"1min":0.72899999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.3460000000000001,"5min":4.492,"15min":4.492},"last":1.206},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.2509999999999999,"15min":1.2470000000000001},"min":{"1min":0.78500000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":1.2769999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.417,"15min":1.4630000000000001},"min":{"1min":0.79000000000000004,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.4770000000000001,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3879999999999999,"15min":1.3959999999999999},"min":{"1min":0.81299999999999994,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.758,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.405}]},{"osd":6,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5980000000000001,"15min":1.6220000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.952,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.633},{"interface":"front","average":{"1min":1.357,"5min":1.5069999999999999,"15min":1.532},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":2.0910000000000002,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.3560000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:57:09 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.647,"15min":1.671},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.149,"5min":3.492,"15min":3.492},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.6830000000000001,"15min":1.7330000000000001},"min":{"1min":0.96799999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.8859999999999999,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.512}]}]}],"pool_statfs":[]}} 2023-10-18T08:57:25.561 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T08:57:25.562 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T08:57:25.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.841+0000 7f435f335640 1 -- 172.21.15.40:0/2019020818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f435815cde0 msgr2=0x7f43580ca990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:25.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.841+0000 7f435f335640 1 --2- 172.21.15.40:0/2019020818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f435815cde0 0x7f43580ca990 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4354005fd0 tx=0x7f435402bd90 comp rx=0 tx=0).stop 2023-10-18T08:57:25.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.841+0000 7f435f335640 1 -- 172.21.15.40:0/2019020818 shutdown_connections 2023-10-18T08:57:25.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.841+0000 7f435f335640 1 --2- 172.21.15.40:0/2019020818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43580cd110 0x7f435815d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:25.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.841+0000 7f435f335640 1 --2- 172.21.15.40:0/2019020818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f435815cde0 0x7f43580ca990 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:25.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.841+0000 7f435f335640 1 -- 172.21.15.40:0/2019020818 >> 172.21.15.40:0/2019020818 conn(0x7f43580b0b60 msgr2=0x7f43580b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:25.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.841+0000 7f435f335640 1 -- 172.21.15.40:0/2019020818 shutdown_connections 2023-10-18T08:57:25.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.841+0000 7f435f335640 1 -- 172.21.15.40:0/2019020818 wait complete. 2023-10-18T08:57:25.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.843+0000 7f435f335640 1 Processor -- start 2023-10-18T08:57:25.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.843+0000 7f435f335640 1 -- start start 2023-10-18T08:57:25.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.844+0000 7f435f335640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43580cd110 0x7f4358152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:25.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.844+0000 7f435f335640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f435815cde0 0x7f4358155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:25.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.844+0000 7f435f335640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f435815d700 0x7f43581f0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:25.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.844+0000 7f435f335640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43581efbe0 con 0x7f43580cd110 2023-10-18T08:57:25.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.844+0000 7f435f335640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435815a210 con 0x7f435815d700 2023-10-18T08:57:25.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.844+0000 7f435f335640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435815a4f0 con 0x7f435815cde0 2023-10-18T08:57:25.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.844+0000 7f435e333640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43580cd110 0x7f4358152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:25.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.844+0000 7f435e333640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43580cd110 0x7f4358152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38068/0 (socket says 172.21.15.40:38068) 2023-10-18T08:57:25.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.844+0000 7f435e333640 1 -- 172.21.15.40:0/316359389 learned_addr learned my addr 172.21.15.40:0/316359389 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:25.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.845+0000 7f435e333640 1 -- 172.21.15.40:0/316359389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f435815cde0 msgr2=0x7f4358155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:25.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.845+0000 7f435db32640 1 --2- 172.21.15.40:0/316359389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f435815cde0 0x7f4358155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:25.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.845+0000 7f435e333640 1 --2- 172.21.15.40:0/316359389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f435815cde0 0x7f4358155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:25.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.845+0000 7f435e333640 1 -- 172.21.15.40:0/316359389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f435815d700 msgr2=0x7f43581f0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:25.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.845+0000 7f435eb34640 1 --2- 172.21.15.40:0/316359389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f435815d700 0x7f43581f0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:25.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.845+0000 7f435e333640 1 --2- 172.21.15.40:0/316359389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f435815d700 0x7f43581f0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:25.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.845+0000 7f435e333640 1 -- 172.21.15.40:0/316359389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4354009560 con 0x7f43580cd110 2023-10-18T08:57:25.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.845+0000 7f435eb34640 1 --2- 172.21.15.40:0/316359389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f435815d700 0x7f43581f0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:25.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.845+0000 7f435e333640 1 --2- 172.21.15.40:0/316359389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43580cd110 0x7f4358152540 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4354009850 tx=0x7f435402da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:25.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.846+0000 7f434effd640 1 -- 172.21.15.40:0/316359389 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435402de60 con 0x7f43580cd110 2023-10-18T08:57:25.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.846+0000 7f434effd640 1 -- 172.21.15.40:0/316359389 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f43540036e0 con 0x7f43580cd110 2023-10-18T08:57:25.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.846+0000 7f434effd640 1 -- 172.21.15.40:0/316359389 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354003a90 con 0x7f43580cd110 2023-10-18T08:57:25.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.846+0000 7f435f335640 1 -- 172.21.15.40:0/316359389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435815a7d0 con 0x7f43580cd110 2023-10-18T08:57:25.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.846+0000 7f435f335640 1 -- 172.21.15.40:0/316359389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4358155fc0 con 0x7f43580cd110 2023-10-18T08:57:25.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.847+0000 7f435f335640 1 -- 172.21.15.40:0/316359389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4358073570 con 0x7f43580cd110 2023-10-18T08:57:25.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.848+0000 7f434effd640 1 -- 172.21.15.40:0/316359389 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4354003bf0 con 0x7f43580cd110 2023-10-18T08:57:25.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.849+0000 7f434effd640 1 --2- 172.21.15.40:0/316359389 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f433c06ae40 0x7f433c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:25.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.849+0000 7f434effd640 1 -- 172.21.15.40:0/316359389 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43540ac280 con 0x7f43580cd110 2023-10-18T08:57:25.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.850+0000 7f435db32640 1 --2- 172.21.15.40:0/316359389 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f433c06ae40 0x7f433c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:25.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.851+0000 7f435db32640 1 --2- 172.21.15.40:0/316359389 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f433c06ae40 0x7f433c06d300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4348005fd0 tx=0x7f4348005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:25.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:25.853+0000 7f434effd640 1 -- 172.21.15.40:0/316359389 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4354077440 con 0x7f43580cd110 2023-10-18T08:57:26.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.121+0000 7f435f335640 1 -- 172.21.15.40:0/316359389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f43580b1740 con 0x7f43580cd110 2023-10-18T08:57:26.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.127+0000 7f434effd640 1 -- 172.21.15.40:0/316359389 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f435407aa60 con 0x7f43580cd110 2023-10-18T08:57:26.129 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2023-10-18T08:57:26.129 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w== 2023-10-18T08:57:26.129 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2023-10-18T08:57:26.130 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2023-10-18T08:57:26.130 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T08:57:26.130 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2023-10-18T08:57:26.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.129+0000 7f435f335640 1 -- 172.21.15.40:0/316359389 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f433c06ae40 msgr2=0x7f433c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:26.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.129+0000 7f435f335640 1 --2- 172.21.15.40:0/316359389 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f433c06ae40 0x7f433c06d300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4348005fd0 tx=0x7f4348005850 comp rx=0 tx=0).stop 2023-10-18T08:57:26.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.129+0000 7f435f335640 1 -- 172.21.15.40:0/316359389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43580cd110 msgr2=0x7f4358152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:26.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.129+0000 7f435f335640 1 --2- 172.21.15.40:0/316359389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43580cd110 0x7f4358152540 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4354009850 tx=0x7f435402da60 comp rx=0 tx=0).stop 2023-10-18T08:57:26.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.129+0000 7f435f335640 1 -- 172.21.15.40:0/316359389 shutdown_connections 2023-10-18T08:57:26.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.129+0000 7f435f335640 1 --2- 172.21.15.40:0/316359389 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f433c06ae40 0x7f433c06d300 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:26.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.129+0000 7f435f335640 1 --2- 172.21.15.40:0/316359389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f435815d700 0x7f43581f0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:26.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.129+0000 7f435f335640 1 --2- 172.21.15.40:0/316359389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f435815cde0 0x7f4358155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:26.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.129+0000 7f435f335640 1 --2- 172.21.15.40:0/316359389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43580cd110 0x7f4358152540 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:26.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.130+0000 7f435f335640 1 -- 172.21.15.40:0/316359389 >> 172.21.15.40:0/316359389 conn(0x7f43580b0b60 msgr2=0x7f43580c35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:26.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.130+0000 7f435f335640 1 -- 172.21.15.40:0/316359389 shutdown_connections 2023-10-18T08:57:26.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.130+0000 7f435f335640 1 -- 172.21.15.40:0/316359389 wait complete. 2023-10-18T08:57:26.150 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:57:26.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.374+0000 7f66bbfff640 1 -- 172.21.15.40:0/243684975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc0ca160 msgr2=0x7f66bc0b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:26.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.374+0000 7f66bbfff640 1 --2- 172.21.15.40:0/243684975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc0ca160 0x7f66bc0b0a80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f66a4005fd0 tx=0x7f66a402be20 comp rx=0 tx=0).stop 2023-10-18T08:57:26.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.374+0000 7f66bbfff640 1 -- 172.21.15.40:0/243684975 shutdown_connections 2023-10-18T08:57:26.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.374+0000 7f66bbfff640 1 --2- 172.21.15.40:0/243684975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc0ca160 0x7f66bc0b0a80 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:26.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.374+0000 7f66bbfff640 1 -- 172.21.15.40:0/243684975 >> 172.21.15.40:0/243684975 conn(0x7f66bc069720 msgr2=0x7f66bc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:26.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.374+0000 7f66bbfff640 1 -- 172.21.15.40:0/243684975 shutdown_connections 2023-10-18T08:57:26.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.374+0000 7f66bbfff640 1 -- 172.21.15.40:0/243684975 wait complete. 2023-10-18T08:57:26.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.376+0000 7f66bbfff640 1 Processor -- start 2023-10-18T08:57:26.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.376+0000 7f66bbfff640 1 -- start start 2023-10-18T08:57:26.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.377+0000 7f66bbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc0ca160 0x7f66bc158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:26.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.377+0000 7f66bbfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66bc1420f0 0x7f66bc0ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:26.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.377+0000 7f66bbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66bc1425a0 0x7f66bc159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:26.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.377+0000 7f66bbfff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc0afa10 con 0x7f66bc0ca160 2023-10-18T08:57:26.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.377+0000 7f66bbfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc159180 con 0x7f66bc1425a0 2023-10-18T08:57:26.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.377+0000 7f66bbfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc13bad0 con 0x7f66bc1420f0 2023-10-18T08:57:26.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.378+0000 7f66baffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc0ca160 0x7f66bc158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:26.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.378+0000 7f66baffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc0ca160 0x7f66bc158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38076/0 (socket says 172.21.15.40:38076) 2023-10-18T08:57:26.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.378+0000 7f66baffd640 1 -- 172.21.15.40:0/3254150015 learned_addr learned my addr 172.21.15.40:0/3254150015 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:26.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.378+0000 7f66baffd640 1 -- 172.21.15.40:0/3254150015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66bc1420f0 msgr2=0x7f66bc0ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:26.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.378+0000 7f66baffd640 1 --2- 172.21.15.40:0/3254150015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66bc1420f0 0x7f66bc0ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:26.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.378+0000 7f66baffd640 1 -- 172.21.15.40:0/3254150015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66bc1425a0 msgr2=0x7f66bc159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:26.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.378+0000 7f66baffd640 1 --2- 172.21.15.40:0/3254150015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66bc1425a0 0x7f66bc159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:26.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.378+0000 7f66baffd640 1 -- 172.21.15.40:0/3254150015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a4009560 con 0x7f66bc0ca160 2023-10-18T08:57:26.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.378+0000 7f66baffd640 1 --2- 172.21.15.40:0/3254150015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc0ca160 0x7f66bc158c40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00c790 tx=0x7f66ac00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:26.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.379+0000 7f66b37fe640 1 -- 172.21.15.40:0/3254150015 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac00cea0 con 0x7f66bc0ca160 2023-10-18T08:57:26.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.379+0000 7f66b37fe640 1 -- 172.21.15.40:0/3254150015 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f66ac006e40 con 0x7f66bc0ca160 2023-10-18T08:57:26.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.379+0000 7f66b37fe640 1 -- 172.21.15.40:0/3254150015 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac003bd0 con 0x7f66bc0ca160 2023-10-18T08:57:26.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.379+0000 7f66bbfff640 1 -- 172.21.15.40:0/3254150015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66bc0acb10 con 0x7f66bc0ca160 2023-10-18T08:57:26.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.379+0000 7f66bbfff640 1 -- 172.21.15.40:0/3254150015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66bc0ad0d0 con 0x7f66bc0ca160 2023-10-18T08:57:26.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.380+0000 7f66bbfff640 1 -- 172.21.15.40:0/3254150015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66bc066e00 con 0x7f66bc0ca160 2023-10-18T08:57:26.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.381+0000 7f66b37fe640 1 -- 172.21.15.40:0/3254150015 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f66ac003d30 con 0x7f66bc0ca160 2023-10-18T08:57:26.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.382+0000 7f66b37fe640 1 --2- 172.21.15.40:0/3254150015 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f669c06ae40 0x7f669c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:26.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.382+0000 7f66b37fe640 1 -- 172.21.15.40:0/3254150015 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66ac089500 con 0x7f66bc0ca160 2023-10-18T08:57:26.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.383+0000 7f66ba7fc640 1 --2- 172.21.15.40:0/3254150015 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f669c06ae40 0x7f669c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:26.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.384+0000 7f66ba7fc640 1 --2- 172.21.15.40:0/3254150015 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f669c06ae40 0x7f669c06d300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f66a4035000 tx=0x7f66a40046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:26.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.387+0000 7f66b37fe640 1 -- 172.21.15.40:0/3254150015 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66ac058030 con 0x7f66bc0ca160 2023-10-18T08:57:26.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.617+0000 7f66bbfff640 1 -- 172.21.15.40:0/3254150015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66bc13bf20 con 0x7f66bc0ca160 2023-10-18T08:57:26.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.618+0000 7f66b37fe640 1 -- 172.21.15.40:0/3254150015 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f66ac057bc0 con 0x7f66bc0ca160 2023-10-18T08:57:26.621 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:26.621 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3668356540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3668356540},{"type":"v1","addr":"172.21.15.144:6836","nonce":3668356540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4222679844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4222679844},{"type":"v1","addr":"172.21.15.40:6834","nonce":4222679844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2829399368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2829399368},{"type":"v1","addr":"172.21.15.40:6835","nonce":2829399368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:57:14.826066+0000","modified":"2023-10-18T08:57:17.903233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1438456692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1438456692},{"type":"v1","addr":"172.21.15.144:6837","nonce":1438456692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-18T08:57:26.621 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 24 2023-10-18T08:57:26.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.621+0000 7f66bbfff640 1 -- 172.21.15.40:0/3254150015 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f669c06ae40 msgr2=0x7f669c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:26.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.621+0000 7f66bbfff640 1 --2- 172.21.15.40:0/3254150015 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f669c06ae40 0x7f669c06d300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f66a4035000 tx=0x7f66a40046c0 comp rx=0 tx=0).stop 2023-10-18T08:57:26.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.621+0000 7f66bbfff640 1 -- 172.21.15.40:0/3254150015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc0ca160 msgr2=0x7f66bc158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:26.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.621+0000 7f66bbfff640 1 --2- 172.21.15.40:0/3254150015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc0ca160 0x7f66bc158c40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f66ac00c790 tx=0x7f66ac00cc50 comp rx=0 tx=0).stop 2023-10-18T08:57:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.621+0000 7f66bbfff640 1 -- 172.21.15.40:0/3254150015 shutdown_connections 2023-10-18T08:57:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.621+0000 7f66bbfff640 1 --2- 172.21.15.40:0/3254150015 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f669c06ae40 0x7f669c06d300 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.621+0000 7f66bbfff640 1 --2- 172.21.15.40:0/3254150015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66bc1425a0 0x7f66bc159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.621+0000 7f66bbfff640 1 --2- 172.21.15.40:0/3254150015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66bc1420f0 0x7f66bc0ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:26.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.621+0000 7f66bbfff640 1 --2- 172.21.15.40:0/3254150015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66bc0ca160 0x7f66bc158c40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.621+0000 7f66bbfff640 1 -- 172.21.15.40:0/3254150015 >> 172.21.15.40:0/3254150015 conn(0x7f66bc069720 msgr2=0x7f66bc06f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.622+0000 7f66bbfff640 1 -- 172.21.15.40:0/3254150015 shutdown_connections 2023-10-18T08:57:26.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:26.622+0000 7f66bbfff640 1 -- 172.21.15.40:0/3254150015 wait complete. 2023-10-18T08:57:27.043 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T08:57:14.826066+0000', 'modified': '2023-10-18T08:57:17.903233+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5655}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5655': {'gid': 5655, 'name': 'b', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.144:6837/1438456692', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6835', 'nonce': 1438456692}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 1438456692}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T08:57:27.044 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T08:57:27.044 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-18T08:57:27.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.286+0000 7f3583056640 1 -- 172.21.15.40:0/1678285348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35640051d0 msgr2=0x7f356400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:27.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.286+0000 7f3583056640 1 --2- 172.21.15.40:0/1678285348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35640051d0 0x7f356400b980 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f357c08a920 tx=0x7f35640057f0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.286+0000 7f3583056640 1 -- 172.21.15.40:0/1678285348 shutdown_connections 2023-10-18T08:57:27.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.286+0000 7f3583056640 1 --2- 172.21.15.40:0/1678285348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35640051d0 0x7f356400b980 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.286+0000 7f3583056640 1 -- 172.21.15.40:0/1678285348 >> 172.21.15.40:0/1678285348 conn(0x7f357c06a780 msgr2=0x7f357c06ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:27.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.286+0000 7f3583056640 1 -- 172.21.15.40:0/1678285348 shutdown_connections 2023-10-18T08:57:27.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.286+0000 7f3583056640 1 -- 172.21.15.40:0/1678285348 wait complete. 2023-10-18T08:57:27.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.288+0000 7f3583056640 1 Processor -- start 2023-10-18T08:57:27.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.288+0000 7f3583056640 1 -- start start 2023-10-18T08:57:27.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f3583056640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35640051d0 0x7f357c14c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f3583056640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f357c14cb40 0x7f357c1e65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f3583056640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f357c0cf950 0x7f357c0cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f3583056640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357c1e5ec0 con 0x7f357c0cf950 2023-10-18T08:57:27.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f3583056640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357c0d0310 con 0x7f357c14cb40 2023-10-18T08:57:27.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f3583056640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357c0c72e0 con 0x7f35640051d0 2023-10-18T08:57:27.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f35815f3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f357c0cf950 0x7f357c0cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f35815f3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f357c0cf950 0x7f357c0cfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38086/0 (socket says 172.21.15.40:38086) 2023-10-18T08:57:27.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f35815f3640 1 -- 172.21.15.40:0/2395751372 learned_addr learned my addr 172.21.15.40:0/2395751372 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:27.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f3580df2640 1 --2- 172.21.15.40:0/2395751372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35640051d0 0x7f357c14c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f3573fff640 1 --2- 172.21.15.40:0/2395751372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f357c14cb40 0x7f357c1e65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f35815f3640 1 -- 172.21.15.40:0/2395751372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35640051d0 msgr2=0x7f357c14c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f35815f3640 1 --2- 172.21.15.40:0/2395751372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35640051d0 0x7f357c14c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f35815f3640 1 -- 172.21.15.40:0/2395751372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f357c14cb40 msgr2=0x7f357c1e65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:27.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f35815f3640 1 --2- 172.21.15.40:0/2395751372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f357c14cb40 0x7f357c1e65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f35815f3640 1 -- 172.21.15.40:0/2395751372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357c0c7530 con 0x7f357c0cf950 2023-10-18T08:57:27.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.289+0000 7f3573fff640 1 --2- 172.21.15.40:0/2395751372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f357c14cb40 0x7f357c1e65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:27.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.290+0000 7f35815f3640 1 --2- 172.21.15.40:0/2395751372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f357c0cf950 0x7f357c0cfdd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3574009ee0 tx=0x7f3574005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:27.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.290+0000 7f35717fa640 1 -- 172.21.15.40:0/2395751372 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3574005920 con 0x7f357c0cf950 2023-10-18T08:57:27.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.290+0000 7f35717fa640 1 -- 172.21.15.40:0/2395751372 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3574003970 con 0x7f357c0cf950 2023-10-18T08:57:27.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.290+0000 7f35717fa640 1 -- 172.21.15.40:0/2395751372 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3574003d40 con 0x7f357c0cf950 2023-10-18T08:57:27.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.290+0000 7f3583056640 1 -- 172.21.15.40:0/2395751372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357c0c7810 con 0x7f357c0cf950 2023-10-18T08:57:27.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.290+0000 7f3583056640 1 -- 172.21.15.40:0/2395751372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357c0c7dd0 con 0x7f357c0cf950 2023-10-18T08:57:27.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.293+0000 7f35717fa640 1 -- 172.21.15.40:0/2395751372 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3574004030 con 0x7f357c0cf950 2023-10-18T08:57:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.293+0000 7f35717fa640 1 --2- 172.21.15.40:0/2395751372 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f356806ae40 0x7f356806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.293+0000 7f35717fa640 1 -- 172.21.15.40:0/2395751372 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35740ac0c0 con 0x7f357c0cf950 2023-10-18T08:57:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.294+0000 7f3580df2640 1 --2- 172.21.15.40:0/2395751372 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f356806ae40 0x7f356806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.294+0000 7f3585bed740 1 Processor -- start 2023-10-18T08:57:27.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.295+0000 7f3585bed740 1 -- start start 2023-10-18T08:57:27.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.295+0000 7f3580df2640 1 --2- 172.21.15.40:0/2395751372 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f356806ae40 0x7f356806d300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3578000c00 tx=0x7f35780069c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:27.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f3585bed740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c1557d0 0x556f5c128a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f3585bed740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c158560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f3585bed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c128f40 0x556f5c158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f3585bed740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f5c09b740 con 0x556f5c155bb0 2023-10-18T08:57:27.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f3585bed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f5c098e30 con 0x556f5c128f40 2023-10-18T08:57:27.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f3585bed740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f5c094720 con 0x556f5c1557d0 2023-10-18T08:57:27.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f3573fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f3573fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c158560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38090/0 (socket says 172.21.15.40:38090) 2023-10-18T08:57:27.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f3573fff640 1 -- 172.21.15.40:0/3953994139 learned_addr learned my addr 172.21.15.40:0/3953994139 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:27.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f3580df2640 1 --2- 172.21.15.40:0/3953994139 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c1557d0 0x556f5c128a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f35815f3640 1 --2- 172.21.15.40:0/3953994139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c128f40 0x556f5c158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f3573fff640 1 -- 172.21.15.40:0/3953994139 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c1557d0 msgr2=0x556f5c128a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:57:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f3573fff640 1 --2- 172.21.15.40:0/3953994139 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c1557d0 0x556f5c128a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f3573fff640 1 -- 172.21.15.40:0/3953994139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c128f40 msgr2=0x556f5c158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:57:27.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f3573fff640 1 --2- 172.21.15.40:0/3953994139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c128f40 0x556f5c158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f3573fff640 1 -- 172.21.15.40:0/3953994139 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f5c058910 con 0x556f5c155bb0 2023-10-18T08:57:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.296+0000 7f35815f3640 1 --2- 172.21.15.40:0/3953994139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c128f40 0x556f5c158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.297+0000 7f3573fff640 1 --2- 172.21.15.40:0/3953994139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c158560 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f356c004820 tx=0x7f356c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e97a9a253cf4dd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.297+0000 7f35627fc640 1 -- 172.21.15.40:0/3953994139 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00f8d0 con 0x556f5c155bb0 2023-10-18T08:57:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.297+0000 7f35627fc640 1 -- 172.21.15.40:0/3953994139 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f356c00fb20 con 0x556f5c155bb0 2023-10-18T08:57:27.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.297+0000 7f35627fc640 1 -- 172.21.15.40:0/3953994139 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c012470 con 0x556f5c155bb0 2023-10-18T08:57:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.298+0000 7f3585bed740 1 -- 172.21.15.40:0/3953994139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 msgr2=0x556f5c158560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.298+0000 7f3585bed740 1 --2- 172.21.15.40:0/3953994139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c158560 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f356c004820 tx=0x7f356c004ce0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.298+0000 7f3585bed740 1 -- 172.21.15.40:0/3953994139 shutdown_connections 2023-10-18T08:57:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.298+0000 7f3585bed740 1 --2- 172.21.15.40:0/3953994139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c128f40 0x556f5c158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.298+0000 7f3585bed740 1 --2- 172.21.15.40:0/3953994139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c158560 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.298+0000 7f3585bed740 1 --2- 172.21.15.40:0/3953994139 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c1557d0 0x556f5c128a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.298+0000 7f3585bed740 1 -- 172.21.15.40:0/3953994139 >> 172.21.15.40:0/3953994139 conn(0x556f5c12ac70 msgr2=0x556f5c173560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:27.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.298+0000 7f3585bed740 1 -- 172.21.15.40:0/3953994139 shutdown_connections 2023-10-18T08:57:27.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.298+0000 7f3585bed740 1 -- 172.21.15.40:0/3953994139 wait complete. 2023-10-18T08:57:27.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.299+0000 7f3585bed740 1 -- start start 2023-10-18T08:57:27.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.299+0000 7f3585bed740 1 Processor -- start 2023-10-18T08:57:27.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.300+0000 7f3585bed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c1557d0 0x556f5c15aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.300+0000 7f3585bed740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c142a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.300+0000 7f3585bed740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c142f40 0x556f5c16ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.300+0000 7f3585bed740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f5c032970 con 0x556f5c155bb0 2023-10-18T08:57:27.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.300+0000 7f3585bed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f5c032610 con 0x556f5c1557d0 2023-10-18T08:57:27.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.300+0000 7f3585bed740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f5bdf8f20 con 0x556f5c142f40 2023-10-18T08:57:27.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.300+0000 7f3585bed740 10 client.0 _ll_register_callbacks cb 0x556f5c0560c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T08:57:27.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.300+0000 7f3573fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.300+0000 7f3573fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c142a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38098/0 (socket says 172.21.15.40:38098) 2023-10-18T08:57:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.300+0000 7f3573fff640 1 -- 172.21.15.40:0/2816479390 learned_addr learned my addr 172.21.15.40:0/2816479390 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.300+0000 7f35815f3640 1 --2- 172.21.15.40:0/2816479390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c142f40 0x556f5c16ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.300+0000 7f3573fff640 1 -- 172.21.15.40:0/2816479390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c142f40 msgr2=0x556f5c16ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:57:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.301+0000 7f3573fff640 1 --2- 172.21.15.40:0/2816479390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c142f40 0x556f5c16ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.301+0000 7f3573fff640 1 -- 172.21.15.40:0/2816479390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c1557d0 msgr2=0x556f5c15aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:57:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.301+0000 7f3580df2640 1 --2- 172.21.15.40:0/2816479390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c1557d0 0x556f5c15aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.301+0000 7f3573fff640 1 --2- 172.21.15.40:0/2816479390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c1557d0 0x556f5c15aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.301+0000 7f3573fff640 1 -- 172.21.15.40:0/2816479390 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357c0cce70 con 0x556f5c155bb0 2023-10-18T08:57:27.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.301+0000 7f3573fff640 1 --2- 172.21.15.40:0/2816479390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c142a00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f356c00f6c0 tx=0x7f356c012680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a23f0c3d2d6c2f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:27.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.301+0000 7f355b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T08:57:27.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.301+0000 7f355b7fe640 1 -- 172.21.15.40:0/2816479390 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c012aa0 con 0x556f5c155bb0 2023-10-18T08:57:27.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.301+0000 7f355b7fe640 1 -- 172.21.15.40:0/2816479390 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f356c012c00 con 0x556f5c155bb0 2023-10-18T08:57:27.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.301+0000 7f3585bed740 1 -- 172.21.15.40:0/2816479390 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f5c08d700 con 0x556f5c155bb0 2023-10-18T08:57:27.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.301+0000 7f355b7fe640 1 -- 172.21.15.40:0/2816479390 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f356c012dc0 con 0x556f5c155bb0 2023-10-18T08:57:27.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.301+0000 7f3585bed740 10 client.5814 fetch_fsmap learned FSMap version 24 2023-10-18T08:57:27.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.301+0000 7f3585bed740 1 -- 172.21.15.40:0/2816479390 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x556f5c092da0 con 0x556f5c155bb0 2023-10-18T08:57:27.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.302+0000 7f355b7fe640 1 -- 172.21.15.40:0/2816479390 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x556f5c08d700 con 0x556f5c155bb0 2023-10-18T08:57:27.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.302+0000 7f355b7fe640 1 -- 172.21.15.40:0/2816479390 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f356c00d070 con 0x556f5c155bb0 2023-10-18T08:57:27.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.302+0000 7f3585bed740 10 client.5814 fetch_fsmap finished waiting for FSMap version 24 2023-10-18T08:57:27.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.302+0000 7f3585bed740 10 client.5814 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-18T08:57:27.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.302+0000 7f3585bed740 20 client.5814 populate_metadata read hostname 'smithi040' 2023-10-18T08:57:27.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.302+0000 7f3585bed740 1 --2- 172.21.15.40:0/2816479390 >> [v2:172.21.15.144:6835/1438456692,v1:172.21.15.144:6837/1438456692] conn(0x556f5c183200 0x556f5c187260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.302+0000 7f3585bed740 4 client.5814 mds_command: new command op to 5655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T08:57:27.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.302+0000 7f3585bed740 1 -- 172.21.15.40:0/2816479390 --> [v2:172.21.15.144:6835/1438456692,v1:172.21.15.144:6837/1438456692] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f5c08d420 con 0x556f5c183200 2023-10-18T08:57:27.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.303+0000 7f3580df2640 1 --2- 172.21.15.40:0/2816479390 >> [v2:172.21.15.144:6835/1438456692,v1:172.21.15.144:6837/1438456692] conn(0x556f5c183200 0x556f5c187260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.304+0000 7f3580df2640 1 --2- 172.21.15.40:0/2816479390 >> [v2:172.21.15.144:6835/1438456692,v1:172.21.15.144:6837/1438456692] conn(0x556f5c183200 0x556f5c187260 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=531853aa77d9ca9e server_cookie=6f11eb25c6842529 in_seq=0 out_seq=0 2023-10-18T08:57:27.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.304+0000 7f355b7fe640 10 client.5814 ms_handle_connect on v2:172.21.15.144:6835/1438456692 2023-10-18T08:57:27.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.306+0000 7f355b7fe640 1 -- 172.21.15.40:0/2816479390 <== mds.0 v2:172.21.15.144:6835/1438456692 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556f5c08d420 con 0x556f5c183200 2023-10-18T08:57:27.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.306+0000 7f355b7fe640 10 client.5814 handle_command_reply: tid=0 2023-10-18T08:57:27.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.306+0000 7f3585bed740 1 client.5814 shutdown 2023-10-18T08:57:27.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.306+0000 7f3585bed740 1 -- 172.21.15.40:0/2816479390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 msgr2=0x556f5c142a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:27.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.306+0000 7f3585bed740 1 --2- 172.21.15.40:0/2816479390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c142a00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f356c00f6c0 tx=0x7f356c012680 comp rx=0 tx=0).stop 2023-10-18T08:57:27.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.306+0000 7f3585bed740 1 -- 172.21.15.40:0/2816479390 shutdown_connections 2023-10-18T08:57:27.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.306+0000 7f3585bed740 1 --2- 172.21.15.40:0/2816479390 >> [v2:172.21.15.144:6835/1438456692,v1:172.21.15.144:6837/1438456692] conn(0x556f5c183200 0x556f5c187260 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.306+0000 7f3585bed740 1 --2- 172.21.15.40:0/2816479390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c142f40 0x556f5c16ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.306+0000 7f3585bed740 1 --2- 172.21.15.40:0/2816479390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c142a00 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.306+0000 7f3585bed740 1 --2- 172.21.15.40:0/2816479390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c1557d0 0x556f5c15aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.306+0000 7f355b7fe640 0 client.5814 ms_handle_reset on v2:172.21.15.144:6835/1438456692 2023-10-18T08:57:27.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.306+0000 7f3585bed740 1 -- 172.21.15.40:0/2816479390 >> 172.21.15.40:0/2816479390 conn(0x556f5c128f40 msgr2=0x556f5c12c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:27.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.307+0000 7f3585bed740 1 -- 172.21.15.40:0/2816479390 shutdown_connections 2023-10-18T08:57:27.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.307+0000 7f3585bed740 1 -- 172.21.15.40:0/2816479390 wait complete. 2023-10-18T08:57:27.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.307+0000 7f3585bed740 20 client.5814 trim_cache size 0 max 16384 2023-10-18T08:57:27.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.324+0000 7f3585bed740 1 Processor -- start 2023-10-18T08:57:27.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.324+0000 7f3585bed740 1 -- start start 2023-10-18T08:57:27.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.325+0000 7f3585bed740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c142f40 0x556f5c152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.325+0000 7f3585bed740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c1557d0 0x556f5c1534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.325+0000 7f3585bed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c155bb0 0x556f5c176a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.325+0000 7f3585bed740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f5c12cb80 con 0x556f5c142f40 2023-10-18T08:57:27.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.325+0000 7f3585bed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f5c154fb0 con 0x556f5c155bb0 2023-10-18T08:57:27.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.325+0000 7f3585bed740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f5c124680 con 0x556f5c1557d0 2023-10-18T08:57:27.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.325+0000 7f3580df2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c142f40 0x556f5c152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.325+0000 7f3580df2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c142f40 0x556f5c152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38106/0 (socket says 172.21.15.40:38106) 2023-10-18T08:57:27.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.325+0000 7f3580df2640 1 -- 172.21.15.40:0/3585878252 learned_addr learned my addr 172.21.15.40:0/3585878252 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:27.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.326+0000 7f3580df2640 1 -- 172.21.15.40:0/3585878252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c1557d0 msgr2=0x556f5c1534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:57:27.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.326+0000 7f3580df2640 1 --2- 172.21.15.40:0/3585878252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c1557d0 0x556f5c1534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.326+0000 7f3580df2640 1 -- 172.21.15.40:0/3585878252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c155bb0 msgr2=0x556f5c176a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:57:27.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.326+0000 7f3580df2640 1 --2- 172.21.15.40:0/3585878252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c155bb0 0x556f5c176a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.326+0000 7f3580df2640 1 -- 172.21.15.40:0/3585878252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f5c031b80 con 0x556f5c142f40 2023-10-18T08:57:27.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.326+0000 7f3580df2640 1 --2- 172.21.15.40:0/3585878252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c142f40 0x556f5c152fb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3578012690 tx=0x7f357800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487f093a7672eb5f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:27.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.326+0000 7f3562ffd640 1 -- 172.21.15.40:0/3585878252 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3578027070 con 0x556f5c142f40 2023-10-18T08:57:27.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.326+0000 7f3562ffd640 1 -- 172.21.15.40:0/3585878252 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3578012c10 con 0x556f5c142f40 2023-10-18T08:57:27.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.326+0000 7f3562ffd640 1 -- 172.21.15.40:0/3585878252 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3578027ea0 con 0x556f5c142f40 2023-10-18T08:57:27.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.327+0000 7f3585bed740 1 -- 172.21.15.40:0/3585878252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c142f40 msgr2=0x556f5c152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:27.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.327+0000 7f3585bed740 1 --2- 172.21.15.40:0/3585878252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c142f40 0x556f5c152fb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3578012690 tx=0x7f357800c000 comp rx=0 tx=0).stop 2023-10-18T08:57:27.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.327+0000 7f3585bed740 1 -- 172.21.15.40:0/3585878252 shutdown_connections 2023-10-18T08:57:27.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.327+0000 7f3585bed740 1 --2- 172.21.15.40:0/3585878252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c155bb0 0x556f5c176a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.327+0000 7f3585bed740 1 --2- 172.21.15.40:0/3585878252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c1557d0 0x556f5c1534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.327+0000 7f3585bed740 1 --2- 172.21.15.40:0/3585878252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c142f40 0x556f5c152fb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.327+0000 7f3585bed740 1 -- 172.21.15.40:0/3585878252 >> 172.21.15.40:0/3585878252 conn(0x556f5c128f40 msgr2=0x556f5c176290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.327+0000 7f3585bed740 1 -- 172.21.15.40:0/3585878252 shutdown_connections 2023-10-18T08:57:27.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.327+0000 7f3585bed740 1 -- 172.21.15.40:0/3585878252 wait complete. 2023-10-18T08:57:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.328+0000 7f3585bed740 1 -- start start 2023-10-18T08:57:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.328+0000 7f3585bed740 1 Processor -- start 2023-10-18T08:57:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.329+0000 7f3585bed740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c142f40 0x556f5c1dc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.329+0000 7f3585bed740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c1557d0 0x556f5c176a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.329+0000 7f3585bed740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c176fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.329+0000 7f3585bed740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f5c12c570 con 0x556f5c155bb0 2023-10-18T08:57:27.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.329+0000 7f3585bed740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f5c16c5a0 con 0x556f5c142f40 2023-10-18T08:57:27.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.329+0000 7f3585bed740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f5c1554e0 con 0x556f5c1557d0 2023-10-18T08:57:27.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.329+0000 7f3585bed740 10 client.0 _ll_register_callbacks cb 0x7f356c001dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T08:57:27.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.329+0000 7f35815f3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c176fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.329+0000 7f35815f3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c176fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38110/0 (socket says 172.21.15.40:38110) 2023-10-18T08:57:27.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.329+0000 7f35815f3640 1 -- 172.21.15.40:0/1905588281 learned_addr learned my addr 172.21.15.40:0/1905588281 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:27.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.329+0000 7f3573fff640 1 --2- 172.21.15.40:0/1905588281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c1557d0 0x556f5c176a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.329+0000 7f35815f3640 1 -- 172.21.15.40:0/1905588281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c1557d0 msgr2=0x556f5c176a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:57:27.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.329+0000 7f3580df2640 1 --2- 172.21.15.40:0/1905588281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c142f40 0x556f5c1dc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.329+0000 7f35815f3640 1 --2- 172.21.15.40:0/1905588281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c1557d0 0x556f5c176a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.329+0000 7f35815f3640 1 -- 172.21.15.40:0/1905588281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c142f40 msgr2=0x556f5c1dc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:57:27.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.329+0000 7f35815f3640 1 --2- 172.21.15.40:0/1905588281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c142f40 0x556f5c1dc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.330+0000 7f35815f3640 1 -- 172.21.15.40:0/1905588281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357c0ccb90 con 0x556f5c155bb0 2023-10-18T08:57:27.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.330+0000 7f3580df2640 1 --2- 172.21.15.40:0/1905588281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c142f40 0x556f5c1dc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:27.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.330+0000 7f35815f3640 1 --2- 172.21.15.40:0/1905588281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c176fb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f35740ac090 tx=0x7f357407aa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6c2f7ac7936411 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:27.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.330+0000 7f355b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T08:57:27.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.330+0000 7f355b7fe640 1 -- 172.21.15.40:0/1905588281 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357407ae30 con 0x556f5c155bb0 2023-10-18T08:57:27.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.330+0000 7f355b7fe640 1 -- 172.21.15.40:0/1905588281 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f357407ef80 con 0x556f5c155bb0 2023-10-18T08:57:27.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.330+0000 7f3585bed740 1 -- 172.21.15.40:0/1905588281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f356c00fb20 con 0x556f5c155bb0 2023-10-18T08:57:27.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.330+0000 7f355b7fe640 1 -- 172.21.15.40:0/1905588281 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3574083130 con 0x556f5c155bb0 2023-10-18T08:57:27.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.330+0000 7f3585bed740 10 client.5820 fetch_fsmap learned FSMap version 24 2023-10-18T08:57:27.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.330+0000 7f3585bed740 1 -- 172.21.15.40:0/1905588281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x556f5c057170 con 0x556f5c155bb0 2023-10-18T08:57:27.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.331+0000 7f355b7fe640 1 -- 172.21.15.40:0/1905588281 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f356c00fb20 con 0x556f5c155bb0 2023-10-18T08:57:27.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.331+0000 7f355b7fe640 1 -- 172.21.15.40:0/1905588281 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f35740aa250 con 0x556f5c155bb0 2023-10-18T08:57:27.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.331+0000 7f3585bed740 10 client.5820 fetch_fsmap finished waiting for FSMap version 24 2023-10-18T08:57:27.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.331+0000 7f3585bed740 10 client.5820 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-18T08:57:27.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.331+0000 7f3585bed740 20 client.5820 populate_metadata read hostname 'smithi040' 2023-10-18T08:57:27.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.331+0000 7f3585bed740 1 --2- 172.21.15.40:0/1905588281 >> [v2:172.21.15.144:6835/1438456692,v1:172.21.15.144:6837/1438456692] conn(0x556f5c153e70 0x556f5c154250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.331+0000 7f3585bed740 4 client.5820 mds_command: new command op to 5655 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T08:57:27.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.331+0000 7f3585bed740 1 -- 172.21.15.40:0/1905588281 --> [v2:172.21.15.144:6835/1438456692,v1:172.21.15.144:6837/1438456692] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556f5bd966c0 con 0x556f5c153e70 2023-10-18T08:57:27.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.332+0000 7f3580df2640 1 --2- 172.21.15.40:0/1905588281 >> [v2:172.21.15.144:6835/1438456692,v1:172.21.15.144:6837/1438456692] conn(0x556f5c153e70 0x556f5c154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.332+0000 7f3580df2640 1 --2- 172.21.15.40:0/1905588281 >> [v2:172.21.15.144:6835/1438456692,v1:172.21.15.144:6837/1438456692] conn(0x556f5c153e70 0x556f5c154250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eea48b8b2f0c660 server_cookie=e0fdd4daed4f0fa in_seq=0 out_seq=0 2023-10-18T08:57:27.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.332+0000 7f355b7fe640 10 client.5820 ms_handle_connect on v2:172.21.15.144:6835/1438456692 2023-10-18T08:57:27.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.333+0000 7f355b7fe640 1 -- 172.21.15.40:0/1905588281 <== mds.0 v2:172.21.15.144:6835/1438456692 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x556f5bd966c0 con 0x556f5c153e70 2023-10-18T08:57:27.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.333+0000 7f355b7fe640 10 client.5820 handle_command_reply: tid=0 2023-10-18T08:57:27.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.333+0000 7f3585bed740 1 client.5820 shutdown 2023-10-18T08:57:27.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.333+0000 7f3585bed740 1 -- 172.21.15.40:0/1905588281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 msgr2=0x556f5c176fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:27.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.333+0000 7f3585bed740 1 --2- 172.21.15.40:0/1905588281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c176fb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f35740ac090 tx=0x7f357407aa30 comp rx=0 tx=0).stop 2023-10-18T08:57:27.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.333+0000 7f3585bed740 1 -- 172.21.15.40:0/1905588281 shutdown_connections 2023-10-18T08:57:27.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.333+0000 7f3585bed740 1 --2- 172.21.15.40:0/1905588281 >> [v2:172.21.15.144:6835/1438456692,v1:172.21.15.144:6837/1438456692] conn(0x556f5c153e70 0x556f5c154250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.333+0000 7f3585bed740 1 --2- 172.21.15.40:0/1905588281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x556f5c155bb0 0x556f5c176fb0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.333+0000 7f3585bed740 1 --2- 172.21.15.40:0/1905588281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556f5c1557d0 0x556f5c176a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.333+0000 7f3585bed740 1 --2- 172.21.15.40:0/1905588281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556f5c142f40 0x556f5c1dc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.333+0000 7f355b7fe640 0 client.5820 ms_handle_reset on v2:172.21.15.144:6835/1438456692 2023-10-18T08:57:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.333+0000 7f3585bed740 1 -- 172.21.15.40:0/1905588281 >> 172.21.15.40:0/1905588281 conn(0x556f5c128f40 msgr2=0x556f5c152fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.334+0000 7f3585bed740 1 -- 172.21.15.40:0/1905588281 shutdown_connections 2023-10-18T08:57:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.334+0000 7f3585bed740 1 -- 172.21.15.40:0/1905588281 wait complete. 2023-10-18T08:57:27.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.334+0000 7f3585bed740 20 client.5820 trim_cache size 0 max 16384 2023-10-18T08:57:27.348 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T08:57:27.348 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T08:57:27.348 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-10-18T08:57:27.348 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 5655, 2023-10-18T08:57:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-10-18T08:57:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-10-18T08:57:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-10-18T08:57:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 11.942573154, 2023-10-18T08:57:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 24, 2023-10-18T08:57:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 46, 2023-10-18T08:57:27.350 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 46, 2023-10-18T08:57:27.350 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 17.773499425000001 2023-10-18T08:57:27.350 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T08:57:27.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.334+0000 7f3583056640 1 -- 172.21.15.40:0/2395751372 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f356806ae40 msgr2=0x7f356806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:27.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.334+0000 7f3583056640 1 --2- 172.21.15.40:0/2395751372 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f356806ae40 0x7f356806d300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3578000c00 tx=0x7f35780069c0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.334+0000 7f3583056640 1 -- 172.21.15.40:0/2395751372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f357c0cf950 msgr2=0x7f357c0cfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.334+0000 7f3583056640 1 --2- 172.21.15.40:0/2395751372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f357c0cf950 0x7f357c0cfdd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3574009ee0 tx=0x7f3574005320 comp rx=0 tx=0).stop 2023-10-18T08:57:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.334+0000 7f3583056640 1 -- 172.21.15.40:0/2395751372 shutdown_connections 2023-10-18T08:57:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.334+0000 7f3583056640 1 --2- 172.21.15.40:0/2395751372 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f356806ae40 0x7f356806d300 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.334+0000 7f3583056640 1 --2- 172.21.15.40:0/2395751372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f357c0cf950 0x7f357c0cfdd0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.334+0000 7f3583056640 1 --2- 172.21.15.40:0/2395751372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f357c14cb40 0x7f357c1e65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.334+0000 7f3583056640 1 --2- 172.21.15.40:0/2395751372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35640051d0 0x7f357c14c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.334+0000 7f3583056640 1 -- 172.21.15.40:0/2395751372 >> 172.21.15.40:0/2395751372 conn(0x7f357c06a780 msgr2=0x7f357c0b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:27.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.335+0000 7f3583056640 1 -- 172.21.15.40:0/2395751372 shutdown_connections 2023-10-18T08:57:27.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.335+0000 7f3583056640 1 -- 172.21.15.40:0/2395751372 wait complete. 2023-10-18T08:57:27.355 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T08:57:27.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.553+0000 7fbc1b59e640 1 -- 172.21.15.40:0/1851374811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc1c0b0dd0 msgr2=0x7fbc1c15a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.553+0000 7fbc1b59e640 1 --2- 172.21.15.40:0/1851374811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc1c0b0dd0 0x7fbc1c15a350 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c005fd0 tx=0x7fbc0c02bcf0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.553+0000 7fbc1b59e640 1 -- 172.21.15.40:0/1851374811 shutdown_connections 2023-10-18T08:57:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.553+0000 7fbc1b59e640 1 --2- 172.21.15.40:0/1851374811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc1c0b0070 0x7fbc1c06b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.553+0000 7fbc1b59e640 1 --2- 172.21.15.40:0/1851374811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc1c0b0dd0 0x7fbc1c15a350 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.553+0000 7fbc1b59e640 1 -- 172.21.15.40:0/1851374811 >> 172.21.15.40:0/1851374811 conn(0x7fbc1c0aed70 msgr2=0x7fbc1c0af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:27.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.553+0000 7fbc1b59e640 1 -- 172.21.15.40:0/1851374811 shutdown_connections 2023-10-18T08:57:27.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.553+0000 7fbc1b59e640 1 -- 172.21.15.40:0/1851374811 wait complete. 2023-10-18T08:57:27.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.555+0000 7fbc1b59e640 1 Processor -- start 2023-10-18T08:57:27.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.555+0000 7fbc1b59e640 1 -- start start 2023-10-18T08:57:27.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.556+0000 7fbc1b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc1c0b0070 0x7fbc1c138860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.556+0000 7fbc1b59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc1c0b0dd0 0x7fbc1c1f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.556+0000 7fbc1b59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc1c15a990 0x7fbc1c1f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.556+0000 7fbc1b59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c1f2370 con 0x7fbc1c0b0070 2023-10-18T08:57:27.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.556+0000 7fbc1b59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c13b090 con 0x7fbc1c0b0dd0 2023-10-18T08:57:27.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.556+0000 7fbc1b59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c13b370 con 0x7fbc1c15a990 2023-10-18T08:57:27.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.556+0000 7fbc1a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc1c0b0070 0x7fbc1c138860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.556+0000 7fbc1a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc1c0b0070 0x7fbc1c138860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38116/0 (socket says 172.21.15.40:38116) 2023-10-18T08:57:27.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.556+0000 7fbc1a59c640 1 -- 172.21.15.40:0/649845272 learned_addr learned my addr 172.21.15.40:0/649845272 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:27.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.557+0000 7fbc1a59c640 1 -- 172.21.15.40:0/649845272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc1c15a990 msgr2=0x7fbc1c1f1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:27.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.557+0000 7fbc1a59c640 1 --2- 172.21.15.40:0/649845272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc1c15a990 0x7fbc1c1f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.557+0000 7fbc1a59c640 1 -- 172.21.15.40:0/649845272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc1c0b0dd0 msgr2=0x7fbc1c1f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:27.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.557+0000 7fbc1a59c640 1 --2- 172.21.15.40:0/649845272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc1c0b0dd0 0x7fbc1c1f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.557+0000 7fbc1a59c640 1 -- 172.21.15.40:0/649845272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c009560 con 0x7fbc1c0b0070 2023-10-18T08:57:27.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.557+0000 7fbc1a59c640 1 --2- 172.21.15.40:0/649845272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc1c0b0070 0x7fbc1c138860 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009850 tx=0x7fbc0c02da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:27.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.557+0000 7fbc12ffd640 1 -- 172.21.15.40:0/649845272 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c02de70 con 0x7fbc1c0b0070 2023-10-18T08:57:27.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.557+0000 7fbc12ffd640 1 -- 172.21.15.40:0/649845272 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbc0c0036e0 con 0x7fbc1c0b0070 2023-10-18T08:57:27.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.557+0000 7fbc12ffd640 1 -- 172.21.15.40:0/649845272 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c003a90 con 0x7fbc1c0b0070 2023-10-18T08:57:27.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.558+0000 7fbc1b59e640 1 -- 172.21.15.40:0/649845272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1c13b650 con 0x7fbc1c0b0070 2023-10-18T08:57:27.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.558+0000 7fbc1b59e640 1 -- 172.21.15.40:0/649845272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1c13bc10 con 0x7fbc1c0b0070 2023-10-18T08:57:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.559+0000 7fbc1b59e640 1 -- 172.21.15.40:0/649845272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1c066e00 con 0x7fbc1c0b0070 2023-10-18T08:57:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.560+0000 7fbc12ffd640 1 -- 172.21.15.40:0/649845272 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbc0c003bf0 con 0x7fbc1c0b0070 2023-10-18T08:57:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.560+0000 7fbc12ffd640 1 --2- 172.21.15.40:0/649845272 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbbfc06ae40 0x7fbbfc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.561+0000 7fbc12ffd640 1 -- 172.21.15.40:0/649845272 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc0c0ac370 con 0x7fbc1c0b0070 2023-10-18T08:57:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.561+0000 7fbc13fff640 1 --2- 172.21.15.40:0/649845272 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbbfc06ae40 0x7fbbfc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.562+0000 7fbc13fff640 1 --2- 172.21.15.40:0/649845272 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbbfc06ae40 0x7fbbfc06d300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbc04006fd0 tx=0x7fbc04006bc0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.565+0000 7fbc12ffd640 1 -- 172.21.15.40:0/649845272 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc0c077550 con 0x7fbc1c0b0070 2023-10-18T08:57:27.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.801+0000 7fbc1b59e640 1 -- 172.21.15.40:0/649845272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbc1c0b0570 con 0x7fbc1c0b0070 2023-10-18T08:57:27.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.802+0000 7fbc12ffd640 1 -- 172.21.15.40:0/649845272 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbc0c043060 con 0x7fbc1c0b0070 2023-10-18T08:57:27.804 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T08:57:27.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.804+0000 7fbc1b59e640 1 -- 172.21.15.40:0/649845272 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbbfc06ae40 msgr2=0x7fbbfc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:27.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.804+0000 7fbc1b59e640 1 --2- 172.21.15.40:0/649845272 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbbfc06ae40 0x7fbbfc06d300 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbc04006fd0 tx=0x7fbc04006bc0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.804+0000 7fbc1b59e640 1 -- 172.21.15.40:0/649845272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc1c0b0070 msgr2=0x7fbc1c138860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:27.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.804+0000 7fbc1b59e640 1 --2- 172.21.15.40:0/649845272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc1c0b0070 0x7fbc1c138860 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009850 tx=0x7fbc0c02da70 comp rx=0 tx=0).stop 2023-10-18T08:57:27.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.805+0000 7fbc1b59e640 1 -- 172.21.15.40:0/649845272 shutdown_connections 2023-10-18T08:57:27.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.805+0000 7fbc1b59e640 1 --2- 172.21.15.40:0/649845272 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbbfc06ae40 0x7fbbfc06d300 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.805+0000 7fbc1b59e640 1 --2- 172.21.15.40:0/649845272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc1c15a990 0x7fbc1c1f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.805+0000 7fbc1b59e640 1 --2- 172.21.15.40:0/649845272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc1c0b0dd0 0x7fbc1c1f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.805+0000 7fbc1b59e640 1 --2- 172.21.15.40:0/649845272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc1c0b0070 0x7fbc1c138860 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:27.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.805+0000 7fbc1b59e640 1 -- 172.21.15.40:0/649845272 >> 172.21.15.40:0/649845272 conn(0x7fbc1c0aed70 msgr2=0x7fbc1c13fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:27.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.805+0000 7fbc1b59e640 1 -- 172.21.15.40:0/649845272 shutdown_connections 2023-10-18T08:57:27.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:27.805+0000 7fbc1b59e640 1 -- 172.21.15.40:0/649845272 wait complete. 2023-10-18T08:57:27.826 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T08:57:27.827 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T08:57:27.827 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-10-18T08:57:27.827 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T08:57:27.827 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T08:57:27.828 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T08:57:27.828 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T08:57:27.828 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T08:57:27.829 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T08:57:27.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:27.830 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-10-18T08:57:27.887 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T08:57:27.887 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T08:57:27.888 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T08:57:27.888 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:57:27.888 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-18T08:57:27.888 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:57:27.889 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:57:27.889 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:57:27.889 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-10-18T08:57:27.889 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:57:27.889 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:57:27.890 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-10-18T08:57:27.890 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T08:57:27.890 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-10-18T08:57:27.890 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-10-18T08:57:27.890 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-18T08:57:27.891 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42276sec preferred_lft 42276sec 2023-10-18T08:57:27.891 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2023-10-18T08:57:27.891 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:57:27.891 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:57:27.892 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-10-18T08:57:27.892 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-10-18T08:57:27.892 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T08:57:27.892 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:57:27.892 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T08:57:27.893 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:57:27.893 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2026:95ff:fea9:a463/64 scope link 2023-10-18T08:57:27.893 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:57:27.893 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T08:57:27.893 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T08:57:27.894 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::80bd:e5ff:fe0f:6f68/64 scope link 2023-10-18T08:57:27.894 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:57:27.894 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T08:57:27.894 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether ea:2b:21:f5:06:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T08:57:27.895 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::e82b:21ff:fef5:6af/64 scope link 2023-10-18T08:57:27.895 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:57:27.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:27.896 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T08:57:27.995 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-10-18T08:57:28.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:28.000 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-10-18T08:57:28.062 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-10-18T08:57:28.062 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T08:57:28.062 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-18T08:57:28.062 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-18T08:57:28.062 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T08:57:28.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:28.063 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T08:57:28.064 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T08:57:28.064 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-18T08:57:28.064 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-18T08:57:28.065 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-18T08:57:28.065 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T08:57:28.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:28.259 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-18T08:57:28.314 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T08:57:28.315 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T08:57:28.316 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T08:57:28.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:28.317 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-10-18T08:57:28.421 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:57:28.421 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T08:57:28.480 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T08:57:28.480 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:57:28.481 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T08:57:28.533 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-18T08:57:28.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:28.534 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:57:28.688 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:57:28.688 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:57:28.723 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:57:28.723 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:57:28.723 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T08:57:28.723 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T08:57:28.723 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T08:57:28.725 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-18T08:57:28.725 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2023-10-18T08:57:28.725 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T08:57:28.726 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T08:57:28.726 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T08:57:28.729 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T08:57:28.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:28.730 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T08:57:28.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:28.817 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:57:28.895 INFO:teuthology.orchestra.run.smithi040.stdout:client5221 2023-10-18T08:57:28.896 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:57:28.896 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:57:28.956 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T08:57:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.187+0000 7f24efa4b640 1 -- 172.21.15.40:0/798449318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24e815cf80 msgr2=0x7f24e80aed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.187+0000 7f24efa4b640 1 --2- 172.21.15.40:0/798449318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24e815cf80 0x7f24e80aed70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f24e80cc120 tx=0x7f24d802be20 comp rx=0 tx=0).stop 2023-10-18T08:57:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.187+0000 7f24efa4b640 1 -- 172.21.15.40:0/798449318 shutdown_connections 2023-10-18T08:57:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.187+0000 7f24efa4b640 1 --2- 172.21.15.40:0/798449318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24e815cf80 0x7f24e80aed70 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.187+0000 7f24efa4b640 1 -- 172.21.15.40:0/798449318 >> 172.21.15.40:0/798449318 conn(0x7f24e8069720 msgr2=0x7f24e8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.187+0000 7f24efa4b640 1 -- 172.21.15.40:0/798449318 shutdown_connections 2023-10-18T08:57:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.187+0000 7f24efa4b640 1 -- 172.21.15.40:0/798449318 wait complete. 2023-10-18T08:57:29.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.189+0000 7f24efa4b640 1 Processor -- start 2023-10-18T08:57:29.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.189+0000 7f24efa4b640 1 -- start start 2023-10-18T08:57:29.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.190+0000 7f24efa4b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24e8142cc0 0x7f24e80b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:29.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.190+0000 7f24efa4b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24e815cf80 0x7f24e8158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:29.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.190+0000 7f24efa4b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24e815d3f0 0x7f24e81f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:29.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.190+0000 7f24efa4b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e81f5e20 con 0x7f24e8142cc0 2023-10-18T08:57:29.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.190+0000 7f24efa4b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e80b1c00 con 0x7f24e815cf80 2023-10-18T08:57:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.190+0000 7f24efa4b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e80b0ff0 con 0x7f24e815d3f0 2023-10-18T08:57:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.190+0000 7f24ed7e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24e8142cc0 0x7f24e80b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.191+0000 7f24ed7e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24e8142cc0 0x7f24e80b0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38132/0 (socket says 172.21.15.40:38132) 2023-10-18T08:57:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.191+0000 7f24ed7e7640 1 -- 172.21.15.40:0/3447398524 learned_addr learned my addr 172.21.15.40:0/3447398524 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.191+0000 7f24ecfe6640 1 --2- 172.21.15.40:0/3447398524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24e815cf80 0x7f24e8158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.191+0000 7f24ed7e7640 1 -- 172.21.15.40:0/3447398524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24e815d3f0 msgr2=0x7f24e81f58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.191+0000 7f24ed7e7640 1 --2- 172.21.15.40:0/3447398524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24e815d3f0 0x7f24e81f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.191+0000 7f24ed7e7640 1 -- 172.21.15.40:0/3447398524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24e815cf80 msgr2=0x7f24e8158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.191+0000 7f24ed7e7640 1 --2- 172.21.15.40:0/3447398524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24e815cf80 0x7f24e8158b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.191+0000 7f24ed7e7640 1 -- 172.21.15.40:0/3447398524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d8009560 con 0x7f24e8142cc0 2023-10-18T08:57:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.191+0000 7f24ed7e7640 1 --2- 172.21.15.40:0/3447398524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24e8142cc0 0x7f24e80b0a10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f24e400c450 tx=0x7f24e400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.192+0000 7f24ddffb640 1 -- 172.21.15.40:0/3447398524 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e40052d0 con 0x7f24e8142cc0 2023-10-18T08:57:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.192+0000 7f24ddffb640 1 -- 172.21.15.40:0/3447398524 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f24e4005430 con 0x7f24e8142cc0 2023-10-18T08:57:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.192+0000 7f24ddffb640 1 -- 172.21.15.40:0/3447398524 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e4012890 con 0x7f24e8142cc0 2023-10-18T08:57:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.192+0000 7f24efa4b640 1 -- 172.21.15.40:0/3447398524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e80b12d0 con 0x7f24e8142cc0 2023-10-18T08:57:29.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.192+0000 7f24efa4b640 1 -- 172.21.15.40:0/3447398524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e80b1890 con 0x7f24e8142cc0 2023-10-18T08:57:29.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.193+0000 7f24efa4b640 1 -- 172.21.15.40:0/3447398524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e8066fd0 con 0x7f24e8142cc0 2023-10-18T08:57:29.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.194+0000 7f24ddffb640 1 -- 172.21.15.40:0/3447398524 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f24e40035f0 con 0x7f24e8142cc0 2023-10-18T08:57:29.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.195+0000 7f24ddffb640 1 --2- 172.21.15.40:0/3447398524 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f24cc06ae40 0x7f24cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:29.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.195+0000 7f24ddffb640 1 -- 172.21.15.40:0/3447398524 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24e40892d0 con 0x7f24e8142cc0 2023-10-18T08:57:29.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.195+0000 7f24ecfe6640 1 --2- 172.21.15.40:0/3447398524 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f24cc06ae40 0x7f24cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:29.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.197+0000 7f24ecfe6640 1 --2- 172.21.15.40:0/3447398524 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f24cc06ae40 0x7f24cc06d300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f24d8035000 tx=0x7f24d8004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:29.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.199+0000 7f24ddffb640 1 -- 172.21.15.40:0/3447398524 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f24e4054370 con 0x7f24e8142cc0 2023-10-18T08:57:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.431+0000 7f24efa4b640 1 -- 172.21.15.40:0/3447398524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f24e8070ac0 con 0x7f24e8142cc0 2023-10-18T08:57:29.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.432+0000 7f24ddffb640 1 -- 172.21.15.40:0/3447398524 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f24e4057990 con 0x7f24e8142cc0 2023-10-18T08:57:29.435 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T08:57:29.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.435+0000 7f24efa4b640 1 -- 172.21.15.40:0/3447398524 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f24cc06ae40 msgr2=0x7f24cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:29.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.435+0000 7f24efa4b640 1 --2- 172.21.15.40:0/3447398524 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f24cc06ae40 0x7f24cc06d300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f24d8035000 tx=0x7f24d8004730 comp rx=0 tx=0).stop 2023-10-18T08:57:29.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.435+0000 7f24efa4b640 1 -- 172.21.15.40:0/3447398524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24e8142cc0 msgr2=0x7f24e80b0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:29.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.435+0000 7f24efa4b640 1 --2- 172.21.15.40:0/3447398524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24e8142cc0 0x7f24e80b0a10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f24e400c450 tx=0x7f24e400c910 comp rx=0 tx=0).stop 2023-10-18T08:57:29.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.435+0000 7f24efa4b640 1 -- 172.21.15.40:0/3447398524 shutdown_connections 2023-10-18T08:57:29.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.435+0000 7f24efa4b640 1 --2- 172.21.15.40:0/3447398524 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f24cc06ae40 0x7f24cc06d300 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:29.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.435+0000 7f24efa4b640 1 --2- 172.21.15.40:0/3447398524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f24e815d3f0 0x7f24e81f58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:29.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.435+0000 7f24efa4b640 1 --2- 172.21.15.40:0/3447398524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24e815cf80 0x7f24e8158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:29.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.435+0000 7f24efa4b640 1 --2- 172.21.15.40:0/3447398524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24e8142cc0 0x7f24e80b0a10 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:29.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.435+0000 7f24efa4b640 1 -- 172.21.15.40:0/3447398524 >> 172.21.15.40:0/3447398524 conn(0x7f24e8069720 msgr2=0x7f24e813f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:29.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.435+0000 7f24efa4b640 1 -- 172.21.15.40:0/3447398524 shutdown_connections 2023-10-18T08:57:29.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:29.435+0000 7f24efa4b640 1 -- 172.21.15.40:0/3447398524 wait complete. 2023-10-18T08:57:29.456 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T08:57:29.456 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T08:57:29.457 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-10-18T08:57:29.457 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-18T08:57:29.458 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T08:57:29.458 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T08:57:29.459 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T08:57:29.459 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T08:57:29.459 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T08:57:29.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:29.460 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-10-18T08:57:29.522 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T08:57:29.522 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T08:57:29.523 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T08:57:29.523 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:57:29.523 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-18T08:57:29.523 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:57:29.524 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:57:29.524 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:57:29.524 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2023-10-18T08:57:29.524 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:57:29.524 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:57:29.525 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2023-10-18T08:57:29.525 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:57:29.525 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-10-18T08:57:29.525 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T08:57:29.525 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-10-18T08:57:29.526 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T08:57:29.526 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42268sec preferred_lft 42268sec 2023-10-18T08:57:29.526 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2023-10-18T08:57:29.526 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:57:29.526 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T08:57:29.527 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff 2023-10-18T08:57:29.527 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T08:57:29.527 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:57:29.527 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1493:a0ff:fed2:2c1/64 scope link 2023-10-18T08:57:29.527 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:57:29.528 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T08:57:29.528 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 4a:6e:a9:f0:92:62 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T08:57:29.528 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::486e:a9ff:fef0:9262/64 scope link 2023-10-18T08:57:29.528 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:57:29.529 INFO:teuthology.orchestra.run.smithi144.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T08:57:29.529 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T08:57:29.529 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::345a:c5ff:fe54:de6e/64 scope link 2023-10-18T08:57:29.529 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:57:29.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:29.531 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T08:57:29.623 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-10-18T08:57:29.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:29.628 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-10-18T08:57:29.685 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-10-18T08:57:29.685 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T08:57:29.685 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T08:57:29.685 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T08:57:29.686 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T08:57:29.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:29.686 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T08:57:29.687 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T08:57:29.687 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T08:57:29.688 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T08:57:29.688 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T08:57:29.688 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T08:57:29.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:29.870 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-18T08:57:29.919 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T08:57:29.920 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-18T08:57:29.920 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T08:57:29.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:29.921 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2023-10-18T08:57:30.054 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:57:30.055 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-18T08:57:30.105 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-18T08:57:30.105 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:57:30.106 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-18T08:57:30.203 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-18T08:57:30.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:30.204 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:57:30.333 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:57:30.333 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:57:30.365 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:57:30.366 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:57:30.366 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T08:57:30.366 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with new device syntax: 1@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T08:57:30.366 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T08:57:30.368 INFO:teuthology.orchestra.run.smithi144.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-18T08:57:30.368 INFO:teuthology.orchestra.run.smithi144.stdout: You just mounted a file system that supports labels which does not 2023-10-18T08:57:30.368 INFO:teuthology.orchestra.run.smithi144.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T08:57:30.368 INFO:teuthology.orchestra.run.smithi144.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T08:57:30.369 INFO:teuthology.orchestra.run.smithi144.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T08:57:30.372 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T08:57:30.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:30.373 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-18T08:57:30.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:30.462 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:57:30.521 INFO:teuthology.orchestra.run.smithi144.stdout:client5838 2023-10-18T08:57:30.522 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:57:30.522 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:57:30.572 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:57:30.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.807+0000 7f983c2e5640 1 -- 172.21.15.40:0/3344692115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f983413c470 msgr2=0x7f983413f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:30.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.807+0000 7f983c2e5640 1 --2- 172.21.15.40:0/3344692115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f983413c470 0x7f983413f5a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f983413fae0 tx=0x7f983002be40 comp rx=0 tx=0).stop 2023-10-18T08:57:30.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.807+0000 7f983c2e5640 1 -- 172.21.15.40:0/3344692115 shutdown_connections 2023-10-18T08:57:30.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.807+0000 7f983c2e5640 1 --2- 172.21.15.40:0/3344692115 >> v2:172.21.15.144:3300/0 conn(0x7f98340ca210 0x7f983415d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:30.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.807+0000 7f983c2e5640 1 --2- 172.21.15.40:0/3344692115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f983413c470 0x7f983413f5a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:30.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.807+0000 7f983c2e5640 1 -- 172.21.15.40:0/3344692115 >> 172.21.15.40:0/3344692115 conn(0x7f983406cee0 msgr2=0x7f9834069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:30.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.808+0000 7f983c2e5640 1 -- 172.21.15.40:0/3344692115 shutdown_connections 2023-10-18T08:57:30.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.808+0000 7f983c2e5640 1 -- 172.21.15.40:0/3344692115 wait complete. 2023-10-18T08:57:30.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.809+0000 7f983c2e5640 1 Processor -- start 2023-10-18T08:57:30.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.810+0000 7f983c2e5640 1 -- start start 2023-10-18T08:57:30.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.811+0000 7f983c2e5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98340ca210 0x7f9834159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:30.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.811+0000 7f983c2e5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f983413c470 0x7f9834131250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:30.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.811+0000 7f983c2e5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f983415cf30 0x7f9834141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:30.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.811+0000 7f983c2e5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98340ac930 con 0x7f983413c470 2023-10-18T08:57:30.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.811+0000 7f983c2e5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98340acc10 con 0x7f983415cf30 2023-10-18T08:57:30.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.811+0000 7f983c2e5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98340acef0 con 0x7f98340ca210 2023-10-18T08:57:30.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.811+0000 7f9839880640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f983413c470 0x7f9834131250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:30.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.811+0000 7f9839880640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f983413c470 0x7f9834131250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38156/0 (socket says 172.21.15.40:38156) 2023-10-18T08:57:30.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.811+0000 7f9839880640 1 -- 172.21.15.40:0/280242890 learned_addr learned my addr 172.21.15.40:0/280242890 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:30.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.811+0000 7f983a882640 1 --2- 172.21.15.40:0/280242890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f983415cf30 0x7f9834141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:30.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.811+0000 7f9839880640 1 -- 172.21.15.40:0/280242890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98340ca210 msgr2=0x7f9834159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:30.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.811+0000 7f9839880640 1 --2- 172.21.15.40:0/280242890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98340ca210 0x7f9834159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:30.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.811+0000 7f9839880640 1 -- 172.21.15.40:0/280242890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f983415cf30 msgr2=0x7f9834141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:30.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.811+0000 7f9839880640 1 --2- 172.21.15.40:0/280242890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f983415cf30 0x7f9834141d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:30.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.811+0000 7f9839880640 1 -- 172.21.15.40:0/280242890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9830009560 con 0x7f983413c470 2023-10-18T08:57:30.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.812+0000 7f9839880640 1 --2- 172.21.15.40:0/280242890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f983413c470 0x7f9834131250 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f982400c410 tx=0x7f982400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:30.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.812+0000 7f982affd640 1 -- 172.21.15.40:0/280242890 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98240035f0 con 0x7f983413c470 2023-10-18T08:57:30.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.812+0000 7f982affd640 1 -- 172.21.15.40:0/280242890 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9824003770 con 0x7f983413c470 2023-10-18T08:57:30.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.812+0000 7f983c2e5640 1 -- 172.21.15.40:0/280242890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98340ad1d0 con 0x7f983413c470 2023-10-18T08:57:30.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.812+0000 7f982affd640 1 -- 172.21.15.40:0/280242890 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824003b80 con 0x7f983413c470 2023-10-18T08:57:30.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.812+0000 7f983c2e5640 1 -- 172.21.15.40:0/280242890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9834142260 con 0x7f983413c470 2023-10-18T08:57:30.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.813+0000 7f983c2e5640 1 -- 172.21.15.40:0/280242890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98340ad6f0 con 0x7f983413c470 2023-10-18T08:57:30.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.815+0000 7f982affd640 1 -- 172.21.15.40:0/280242890 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9824003ce0 con 0x7f983413c470 2023-10-18T08:57:30.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.816+0000 7f982affd640 1 --2- 172.21.15.40:0/280242890 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f981806ae40 0x7f981806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:30.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.816+0000 7f982affd640 1 -- 172.21.15.40:0/280242890 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9824088e40 con 0x7f983413c470 2023-10-18T08:57:30.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.817+0000 7f983a081640 1 --2- 172.21.15.40:0/280242890 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f981806ae40 0x7f981806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:30.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.818+0000 7f983a081640 1 --2- 172.21.15.40:0/280242890 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f981806ae40 0x7f981806d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9830035000 tx=0x7f98300046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:30.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:30.821+0000 7f982affd640 1 -- 172.21.15.40:0/280242890 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f98340ad6f0 con 0x7f983413c470 2023-10-18T08:57:31.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:31.057+0000 7f983c2e5640 1 -- 172.21.15.40:0/280242890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98340ad6f0 con 0x7f983413c470 2023-10-18T08:57:31.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:31.058+0000 7f982affd640 1 -- 172.21.15.40:0/280242890 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f98340ad6f0 con 0x7f983413c470 2023-10-18T08:57:31.061 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:31.061 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3668356540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3668356540},{"type":"v1","addr":"172.21.15.144:6836","nonce":3668356540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4222679844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4222679844},{"type":"v1","addr":"172.21.15.40:6834","nonce":4222679844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2829399368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2829399368},{"type":"v1","addr":"172.21.15.40:6835","nonce":2829399368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:57:14.826066+0000","modified":"2023-10-18T08:57:17.903233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1438456692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1438456692},{"type":"v1","addr":"172.21.15.144:6837","nonce":1438456692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-18T08:57:31.061 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 24 2023-10-18T08:57:31.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:31.061+0000 7f983c2e5640 1 -- 172.21.15.40:0/280242890 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f981806ae40 msgr2=0x7f981806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:31.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:31.061+0000 7f983c2e5640 1 --2- 172.21.15.40:0/280242890 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f981806ae40 0x7f981806d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9830035000 tx=0x7f98300046c0 comp rx=0 tx=0).stop 2023-10-18T08:57:31.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:31.061+0000 7f983c2e5640 1 -- 172.21.15.40:0/280242890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f983413c470 msgr2=0x7f9834131250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:31.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:31.061+0000 7f983c2e5640 1 --2- 172.21.15.40:0/280242890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f983413c470 0x7f9834131250 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f982400c410 tx=0x7f982400c8d0 comp rx=0 tx=0).stop 2023-10-18T08:57:31.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:31.061+0000 7f983c2e5640 1 -- 172.21.15.40:0/280242890 shutdown_connections 2023-10-18T08:57:31.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:31.061+0000 7f983c2e5640 1 --2- 172.21.15.40:0/280242890 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f981806ae40 0x7f981806d300 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:31.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:31.061+0000 7f983c2e5640 1 --2- 172.21.15.40:0/280242890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f983415cf30 0x7f9834141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:31.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:31.061+0000 7f983c2e5640 1 --2- 172.21.15.40:0/280242890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f983413c470 0x7f9834131250 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:31.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:31.061+0000 7f983c2e5640 1 --2- 172.21.15.40:0/280242890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98340ca210 0x7f9834159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:31.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:31.061+0000 7f983c2e5640 1 -- 172.21.15.40:0/280242890 >> 172.21.15.40:0/280242890 conn(0x7f983406cee0 msgr2=0x7f983415aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:31.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:31.061+0000 7f983c2e5640 1 -- 172.21.15.40:0/280242890 shutdown_connections 2023-10-18T08:57:31.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:31.061+0000 7f983c2e5640 1 -- 172.21.15.40:0/280242890 wait complete. 2023-10-18T08:57:31.083 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-10-18T08:57:31.321 INFO:teuthology.orchestra.run.smithi144.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":5655,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.913374633,"mdsmap_epoch":24,"osdmap_epoch":46,"osdmap_epoch_barrier":46,"uptime":21.744300395} 2023-10-18T08:57:31.322 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 5655, "mdsmap_epoch": 24, "osdmap_epoch": 46, "osdmap_epoch_barrier": 46, "rank_uptime": 15.913374633, "state": "up:active", "uptime": 21.744300395, "want_state": "up:active", "whoami": 0 } 2023-10-18T08:57:31.323 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-18T08:57:31.575 INFO:teuthology.orchestra.run.smithi040.stdout:{"osd_mon_report_interval":"5"} 2023-10-18T08:57:31.575 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-18T08:57:31.576 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2023-10-18T08:57:31.576 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:57:31.577 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:57:31.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:31.633 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2023-10-18T08:57:33.555 INFO:teuthology.orchestra.run.smithi040.stderr:265+0 records in 2023-10-18T08:57:33.556 INFO:teuthology.orchestra.run.smithi040.stderr:265+0 records out 2023-10-18T08:57:33.556 INFO:teuthology.orchestra.run.smithi040.stderr:277872640 bytes (278 MB, 265 MiB) copied, 1.86029 s, 149 MB/s 2023-10-18T08:57:33.557 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:57:33.557 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:57:33.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:33.602 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2023-10-18T08:57:34.716 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:57:34.726 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:57:35.768 INFO:teuthology.orchestra.run.smithi040.stderr:292+0 records in 2023-10-18T08:57:35.769 INFO:teuthology.orchestra.run.smithi040.stderr:292+0 records out 2023-10-18T08:57:35.769 INFO:teuthology.orchestra.run.smithi040.stderr:306184192 bytes (306 MB, 292 MiB) copied, 2.0667 s, 148 MB/s 2023-10-18T08:57:35.770 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-18T08:57:35.771 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T08:57:35.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.958+0000 7ff81b55e640 1 -- 172.21.15.40:0/282986335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8141420f0 msgr2=0x7ff8140b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.958+0000 7ff81b55e640 1 --2- 172.21.15.40:0/282986335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8141420f0 0x7ff8140b0a80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff810009900 tx=0x7ff81002bd70 comp rx=0 tx=0).stop 2023-10-18T08:57:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.958+0000 7ff81b55e640 1 -- 172.21.15.40:0/282986335 shutdown_connections 2023-10-18T08:57:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.958+0000 7ff81b55e640 1 --2- 172.21.15.40:0/282986335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8141425a0 0x7ff8140c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:35.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.958+0000 7ff81b55e640 1 --2- 172.21.15.40:0/282986335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8141420f0 0x7ff8140b0a80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.958+0000 7ff81b55e640 1 -- 172.21.15.40:0/282986335 >> 172.21.15.40:0/282986335 conn(0x7ff814069720 msgr2=0x7ff814069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.958+0000 7ff81b55e640 1 -- 172.21.15.40:0/282986335 shutdown_connections 2023-10-18T08:57:35.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.958+0000 7ff81b55e640 1 -- 172.21.15.40:0/282986335 wait complete. 2023-10-18T08:57:35.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.960+0000 7ff81b55e640 1 Processor -- start 2023-10-18T08:57:35.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.960+0000 7ff81b55e640 1 -- start start 2023-10-18T08:57:35.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.961+0000 7ff81b55e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8140c9f90 0x7ff81413c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:35.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.961+0000 7ff81b55e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8141420f0 0x7ff8141f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:35.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.961+0000 7ff81b55e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8141425a0 0x7ff8140c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:35.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.961+0000 7ff81b55e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81415c960 con 0x7ff8140c9f90 2023-10-18T08:57:35.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.961+0000 7ff81b55e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8141f6470 con 0x7ff8141420f0 2023-10-18T08:57:35.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.961+0000 7ff81b55e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8140c8de0 con 0x7ff8141425a0 2023-10-18T08:57:35.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.961+0000 7ff81a55c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8140c9f90 0x7ff81413c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:35.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.961+0000 7ff81a55c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8140c9f90 0x7ff81413c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53020/0 (socket says 172.21.15.40:53020) 2023-10-18T08:57:35.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.961+0000 7ff81a55c640 1 -- 172.21.15.40:0/1560664912 learned_addr learned my addr 172.21.15.40:0/1560664912 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:35.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.961+0000 7ff81a55c640 1 -- 172.21.15.40:0/1560664912 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8141425a0 msgr2=0x7ff8140c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:35.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.961+0000 7ff81a55c640 1 --2- 172.21.15.40:0/1560664912 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8141425a0 0x7ff8140c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:35.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.961+0000 7ff81a55c640 1 -- 172.21.15.40:0/1560664912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8141420f0 msgr2=0x7ff8141f5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:35.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.961+0000 7ff81a55c640 1 --2- 172.21.15.40:0/1560664912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8141420f0 0x7ff8141f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:35.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.961+0000 7ff81a55c640 1 -- 172.21.15.40:0/1560664912 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff810009560 con 0x7ff8140c9f90 2023-10-18T08:57:35.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.961+0000 7ff81a55c640 1 --2- 172.21.15.40:0/1560664912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8140c9f90 0x7ff81413c760 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff810036040 tx=0x7ff81002daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:35.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.962+0000 7ff80affd640 1 -- 172.21.15.40:0/1560664912 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8100035f0 con 0x7ff8140c9f90 2023-10-18T08:57:35.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.962+0000 7ff80affd640 1 -- 172.21.15.40:0/1560664912 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff810003840 con 0x7ff8140c9f90 2023-10-18T08:57:35.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.962+0000 7ff81b55e640 1 -- 172.21.15.40:0/1560664912 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81413b6c0 con 0x7ff8140c9f90 2023-10-18T08:57:35.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.962+0000 7ff80affd640 1 -- 172.21.15.40:0/1560664912 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810003bf0 con 0x7ff8140c9f90 2023-10-18T08:57:35.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.962+0000 7ff81b55e640 1 -- 172.21.15.40:0/1560664912 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81413bc80 con 0x7ff8140c9f90 2023-10-18T08:57:35.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.963+0000 7ff81b55e640 1 -- 172.21.15.40:0/1560664912 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff814066e00 con 0x7ff8140c9f90 2023-10-18T08:57:35.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.964+0000 7ff80affd640 1 -- 172.21.15.40:0/1560664912 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff810003d50 con 0x7ff8140c9f90 2023-10-18T08:57:35.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.965+0000 7ff80affd640 1 --2- 172.21.15.40:0/1560664912 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7f806ae40 0x7ff7f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:35.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.966+0000 7ff80affd640 1 -- 172.21.15.40:0/1560664912 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8100ac390 con 0x7ff8140c9f90 2023-10-18T08:57:35.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.966+0000 7ff819d5b640 1 --2- 172.21.15.40:0/1560664912 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7f806ae40 0x7ff7f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:35.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.967+0000 7ff819d5b640 1 --2- 172.21.15.40:0/1560664912 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7f806ae40 0x7ff7f806d300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff814159280 tx=0x7ff804006c90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:35.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:35.970+0000 7ff80affd640 1 -- 172.21.15.40:0/1560664912 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff810077550 con 0x7ff8140c9f90 2023-10-18T08:57:36.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:36.159+0000 7ff81b55e640 1 -- 172.21.15.40:0/1560664912 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff8140c9c90 con 0x7ff8140c9f90 2023-10-18T08:57:36.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:36.162+0000 7ff80affd640 1 -- 172.21.15.40:0/1560664912 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v46) v1 ==== 81+0+33960 (secure 0 0 0) 0x7ff81007ab70 con 0x7ff8140c9f90 2023-10-18T08:57:36.165 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:36.165 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T08:57:36.165 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 46, 2023-10-18T08:57:36.165 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T08:57:36.165 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T08:57:36.166 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T08:57:16.366742+0000", 2023-10-18T08:57:36.166 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T08:57:36.166 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T08:57:36.166 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T08:57:36.166 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T08:57:36.167 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T08:57:36.167 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T08:57:36.167 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T08:57:36.167 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T08:57:36.167 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T08:57:36.168 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:36.168 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T08:57:36.168 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T08:57:36.168 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T08:57:36.168 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T08:57:36.168 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T08:57:36.169 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 7, 2023-10-18T08:57:36.169 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T08:57:36.169 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T08:57:36.169 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T08:57:36.169 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T08:57:36.170 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T08:57:36.170 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T08:57:36.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.170 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T08:57:36.170 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T08:57:36.170 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T08:57:36.171 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T08:57:36.171 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T08:57:36.171 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:36.171 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:36.171 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:36.171 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:36.172 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:36.172 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:36.172 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:36.172 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:36.172 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:36.173 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:36.173 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T08:57:36.173 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T08:57:36.173 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T08:57:36.173 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T08:57:36.174 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T08:57:36.174 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:36.174 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:36.174 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:36.174 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:36.174 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:36.175 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:36.175 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:36.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.175 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "16", 2023-10-18T08:57:36.175 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:36.176 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:36.176 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:36.176 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:36.176 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:36.176 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:36.176 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:36.177 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:36.177 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:36.177 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:36.177 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:36.177 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:36.177 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:36.178 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:36.178 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:36.178 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:36.178 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:36.178 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:36.179 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:36.179 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:36.179 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:36.179 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:36.179 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:36.179 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:36.180 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:36.180 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:36.180 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.180 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:36.180 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:36.180 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:36.181 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:36.181 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:36.181 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:36.181 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:36.181 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:36.182 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:36.182 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:36.182 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:36.182 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:36.183 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T08:57:36.183 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T08:57:36.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.183 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:36.183 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T08:57:36.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.184 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:36.184 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T08:57:36.184 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T08:57:36.184 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T08:57:36.184 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T08:57:36.185 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T08:57:36.185 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:36.185 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:36.185 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:36.185 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.186 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 6, 2023-10-18T08:57:36.186 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T08:57:36.186 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:57:12.765969+0000", 2023-10-18T08:57:36.186 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T08:57:36.186 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T08:57:36.187 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:36.187 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:36.187 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:36.187 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:36.187 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:36.188 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:36.188 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:36.188 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:36.188 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:36.188 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:36.189 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T08:57:36.189 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T08:57:36.189 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T08:57:36.189 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T08:57:36.189 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T08:57:36.189 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:36.190 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:36.190 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:36.190 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:36.190 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:36.190 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:36.190 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:36.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.191 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "45", 2023-10-18T08:57:36.191 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:36.191 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:36.191 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:36.191 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:36.192 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:36.192 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:36.192 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:36.192 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:36.192 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:36.193 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:36.193 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:36.193 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:36.193 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:36.193 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:36.194 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:36.194 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:36.194 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:36.194 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:36.194 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:36.195 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:36.195 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:36.195 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:36.195 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:36.196 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:36.196 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:36.196 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:36.196 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.196 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:36.196 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:36.197 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:36.197 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:36.197 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:36.197 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:36.197 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:36.198 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:36.198 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:36.198 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:36.198 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:36.198 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:36.199 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T08:57:36.199 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T08:57:36.199 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T08:57:36.199 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.199 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:36.200 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T08:57:36.200 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T08:57:36.200 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.200 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.200 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:36.200 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T08:57:36.201 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T08:57:36.201 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T08:57:36.201 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T08:57:36.201 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T08:57:36.201 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:36.202 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:36.202 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:36.202 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.202 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.202 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.202 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 7, 2023-10-18T08:57:36.203 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T08:57:36.203 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:57:13.851110+0000", 2023-10-18T08:57:36.203 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T08:57:36.203 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T08:57:36.203 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:36.204 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:36.204 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:36.204 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:36.204 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:36.204 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:36.204 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:36.205 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:36.205 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:36.205 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:36.205 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T08:57:36.205 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T08:57:36.205 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T08:57:36.206 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T08:57:36.206 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T08:57:36.206 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:36.206 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:36.206 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:36.206 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:36.207 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:36.207 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:36.207 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:36.207 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.207 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "46", 2023-10-18T08:57:36.207 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:36.208 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:36.208 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:36.208 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:36.208 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:36.208 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:36.208 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:36.209 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:36.209 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:36.209 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:36.209 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:36.209 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:36.209 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:36.210 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:36.210 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:36.210 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:36.210 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:36.210 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:36.210 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:36.211 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:36.211 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:36.211 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:36.211 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:36.211 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:36.211 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:36.212 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:36.212 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.212 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:36.212 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:36.212 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:36.212 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:36.213 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:36.213 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:36.213 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:36.213 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:36.213 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:36.213 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:36.214 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:36.214 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:36.214 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T08:57:36.214 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T08:57:36.214 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.214 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:36.215 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T08:57:36.215 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T08:57:36.215 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.215 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.215 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:36.215 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T08:57:36.216 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T08:57:36.216 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T08:57:36.216 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T08:57:36.216 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T08:57:36.216 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:36.216 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:36.217 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:36.217 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.217 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.217 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:36.217 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T08:57:36.218 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.218 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T08:57:36.218 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:36.218 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:36.218 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:36.219 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:36.219 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:36.219 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:36.219 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:36.219 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:36.219 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:36.220 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:36.220 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:36.220 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:36.220 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.220 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.220 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.221 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T08:57:36.221 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:36.221 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.221 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.221 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.222 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T08:57:36.222 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:36.222 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.222 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.222 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.223 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:36.223 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.223 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.223 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.223 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T08:57:36.223 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:36.224 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.224 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.224 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.224 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T08:57:36.224 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:36.225 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.225 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.225 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.225 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:36.225 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.225 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.226 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.226 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T08:57:36.226 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:36.226 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.226 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.227 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.227 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T08:57:36.227 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:36.227 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.227 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.227 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.228 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:36.228 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.228 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.228 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.228 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T08:57:36.229 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:36.229 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.229 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.229 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.229 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T08:57:36.230 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:36.230 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.230 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.230 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.230 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T08:57:36.230 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T08:57:36.231 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T08:57:36.231 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T08:57:36.231 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:36.231 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:36.231 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:36.231 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.232 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.232 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.232 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T08:57:36.232 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:36.232 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:36.233 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:36.233 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:36.233 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:36.233 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:36.233 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:36.233 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:36.234 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:36.234 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:36.234 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:36.234 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:36.234 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.235 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.235 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.235 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T08:57:36.235 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:36.235 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.236 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.236 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.236 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T08:57:36.236 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:36.236 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.237 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.237 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.237 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:36.237 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.237 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.237 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.238 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T08:57:36.238 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:36.238 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.238 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.238 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.238 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T08:57:36.239 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:36.239 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.239 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.239 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.239 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:36.240 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.240 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.240 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.240 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T08:57:36.240 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:36.240 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.241 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.241 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.241 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T08:57:36.241 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:36.241 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.241 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.241 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.242 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:36.242 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.242 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.242 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.242 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T08:57:36.242 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:36.243 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.243 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.243 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.243 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T08:57:36.243 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:36.243 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.244 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.244 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.244 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T08:57:36.244 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T08:57:36.244 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T08:57:36.244 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T08:57:36.245 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:36.245 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:36.245 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:36.245 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.245 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.245 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.246 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T08:57:36.246 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:36.246 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:36.246 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:36.246 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:36.247 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:36.247 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:36.247 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:36.247 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:36.247 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:36.247 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:36.247 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:36.248 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:36.248 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.248 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.248 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.248 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T08:57:36.248 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:36.249 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.249 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.249 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.249 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T08:57:36.249 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.250 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.251 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.251 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T08:57:36.251 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:36.251 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.251 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.251 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.252 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T08:57:36.252 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:36.252 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.252 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.252 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.253 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:36.253 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.253 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.253 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.253 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T08:57:36.253 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:36.254 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.254 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.254 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.254 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T08:57:36.254 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:36.255 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.255 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.255 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.255 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:36.255 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.256 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.256 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.256 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T08:57:36.256 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:36.256 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.256 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.257 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.257 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T08:57:36.257 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:36.257 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.257 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.258 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.258 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T08:57:36.258 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T08:57:36.258 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T08:57:36.258 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T08:57:36.259 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:36.259 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:36.259 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:36.259 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.259 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.260 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.260 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T08:57:36.260 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:36.260 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:36.260 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:36.260 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:36.261 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:36.261 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:36.261 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:36.261 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:36.261 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:36.262 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:36.262 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:36.262 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:36.262 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.262 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.262 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.263 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T08:57:36.263 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:36.263 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.263 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.263 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.264 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T08:57:36.264 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:36.264 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.264 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.264 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.264 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:36.265 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.265 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.265 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.265 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T08:57:36.265 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:36.266 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.266 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.266 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.266 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T08:57:36.266 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:36.267 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.267 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.267 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:36.267 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.268 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.268 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.268 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T08:57:36.268 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:36.268 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.268 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.269 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.269 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T08:57:36.269 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:36.269 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.269 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.270 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.270 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:36.270 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.270 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.270 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.270 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T08:57:36.271 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:36.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.271 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.271 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.271 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T08:57:36.272 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:36.272 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.272 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.272 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.272 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T08:57:36.273 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T08:57:36.273 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T08:57:36.273 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T08:57:36.273 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:36.273 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:36.274 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:36.274 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.274 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.274 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.274 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T08:57:36.275 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:36.275 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:36.275 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:36.275 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:36.275 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:36.276 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:36.276 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:36.276 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:36.276 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:36.276 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:36.277 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:36.277 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:36.277 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.277 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.277 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.277 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T08:57:36.278 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:36.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.278 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.278 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.278 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T08:57:36.279 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:36.279 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.279 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.279 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.279 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:36.280 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.280 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.280 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.280 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T08:57:36.280 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:36.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.281 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.281 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T08:57:36.281 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:36.281 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.281 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.282 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:36.282 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.282 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.283 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T08:57:36.283 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:36.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.283 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.283 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.283 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T08:57:36.284 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:36.284 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.284 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.284 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:36.284 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.285 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.285 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.285 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T08:57:36.285 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:36.285 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.285 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.286 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.286 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T08:57:36.286 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:36.286 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.286 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.287 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.287 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T08:57:36.287 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T08:57:36.287 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T08:57:36.287 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T08:57:36.287 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:36.288 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:36.288 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:36.288 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.288 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.288 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.288 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T08:57:36.289 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:36.289 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:36.289 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:36.289 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:36.289 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:36.290 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:36.290 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:36.290 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:36.290 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:36.290 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:36.290 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:36.291 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:36.291 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.291 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.291 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.291 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T08:57:36.291 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:36.292 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.292 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.292 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.292 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T08:57:36.292 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:36.293 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.293 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.293 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.293 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:36.293 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.294 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.294 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.294 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T08:57:36.294 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:36.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.294 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.295 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.295 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T08:57:36.295 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:36.295 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.295 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.296 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:36.296 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.296 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.296 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.297 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T08:57:36.297 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:36.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.297 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.298 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T08:57:36.298 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:36.298 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.298 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.299 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:36.299 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.299 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.299 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T08:57:36.300 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:36.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.300 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.300 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T08:57:36.301 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:36.301 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.301 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.302 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T08:57:36.302 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T08:57:36.302 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T08:57:36.302 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T08:57:36.302 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:36.303 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:36.303 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:36.303 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.303 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.304 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T08:57:36.304 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:36.304 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:36.304 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:36.304 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:36.304 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:36.305 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:36.305 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:36.305 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:36.305 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:36.305 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:36.306 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:36.306 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:36.306 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.306 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.306 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.306 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T08:57:36.307 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:36.307 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.307 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.307 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.307 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T08:57:36.308 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:36.308 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.308 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.308 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.308 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:36.308 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.309 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.309 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.309 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T08:57:36.309 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:36.309 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.309 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.310 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.310 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T08:57:36.310 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:36.310 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.310 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.311 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.311 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:36.311 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.311 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.311 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.311 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T08:57:36.312 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:36.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.312 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.312 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T08:57:36.312 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:36.313 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.313 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.313 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:36.313 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.313 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.314 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T08:57:36.314 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:36.314 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.314 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.314 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.315 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T08:57:36.315 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:36.315 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.315 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.315 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T08:57:36.316 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T08:57:36.316 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T08:57:36.316 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T08:57:36.316 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:36.316 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:36.316 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:36.317 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.317 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.317 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T08:57:36.317 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:36.317 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:36.318 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:36.318 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:36.318 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:36.318 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:36.318 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:36.319 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:36.319 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:36.319 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:36.319 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:36.319 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:36.319 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.319 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.320 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.320 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T08:57:36.320 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:36.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.321 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.321 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T08:57:36.321 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:36.321 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.321 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.322 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:36.322 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.322 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.322 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.322 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T08:57:36.323 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:36.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.323 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.323 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.323 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T08:57:36.323 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:36.324 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.324 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.324 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.324 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:36.324 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.325 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.325 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.325 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T08:57:36.325 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:36.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.325 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.326 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.326 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T08:57:36.326 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:36.326 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.326 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.327 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.327 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:36.327 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:36.327 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.327 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:36.327 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T08:57:36.328 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:36.328 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.328 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.328 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:36.328 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T08:57:36.329 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:36.329 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.329 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.329 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T08:57:36.329 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T08:57:36.330 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T08:57:36.330 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T08:57:36.330 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:36.330 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:36.330 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:36.330 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:36.331 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.331 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:36.331 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T08:57:36.331 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.331 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T08:57:36.332 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:36.332 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:36.332 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:36.332 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:36.332 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:36.333 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T08:57:36.333 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:36.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T08:57:36.333 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:36.334 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:36.334 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:36.334 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:36.334 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:36.334 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T08:57:36.334 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:36.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.335 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T08:57:36.335 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:36.335 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:36.335 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:36.336 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:36.336 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:36.336 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T08:57:36.336 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:36.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.337 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T08:57:36.337 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:36.337 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:36.337 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:36.337 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:36.338 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:36.338 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T08:57:36.338 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:36.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.339 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T08:57:36.339 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:36.339 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:36.339 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:36.339 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:36.339 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:36.340 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T08:57:36.340 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:36.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.340 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T08:57:36.340 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:36.341 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:36.341 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:36.341 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:36.341 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:36.341 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T08:57:36.341 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:36.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.342 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T08:57:36.342 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:36.342 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:36.342 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:36.343 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:36.343 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:36.343 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T08:57:36.343 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:36.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.343 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:36.344 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T08:57:36.344 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:36.344 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:36.344 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:36.344 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:36.344 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:36.345 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T08:57:36.345 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:36.345 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.345 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:36.345 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T08:57:36.345 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T08:57:36.346 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T08:57:36.346 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T08:57:36.346 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T08:57:36.346 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T08:57:36.346 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T08:57:36.347 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T08:57:36.347 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T08:57:36.347 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T08:57:36.347 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T08:57:36.347 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T08:57:36.347 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T08:57:36.348 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T08:57:36.348 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:36.348 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T08:57:36.348 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T08:57:36.348 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T08:57:36.349 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T08:57:36.349 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T08:57:36.349 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T08:57:36.349 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T08:57:36.349 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T08:57:36.350 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T08:57:36.350 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T08:57:36.350 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T08:57:36.350 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:36.350 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T08:57:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:36.164+0000 7ff81b55e640 1 -- 172.21.15.40:0/1560664912 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7f806ae40 msgr2=0x7ff7f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:36.164+0000 7ff81b55e640 1 --2- 172.21.15.40:0/1560664912 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7f806ae40 0x7ff7f806d300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff814159280 tx=0x7ff804006c90 comp rx=0 tx=0).stop 2023-10-18T08:57:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:36.164+0000 7ff81b55e640 1 -- 172.21.15.40:0/1560664912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8140c9f90 msgr2=0x7ff81413c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:36.164+0000 7ff81b55e640 1 --2- 172.21.15.40:0/1560664912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8140c9f90 0x7ff81413c760 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff810036040 tx=0x7ff81002daf0 comp rx=0 tx=0).stop 2023-10-18T08:57:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:36.165+0000 7ff81b55e640 1 -- 172.21.15.40:0/1560664912 shutdown_connections 2023-10-18T08:57:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:36.165+0000 7ff81b55e640 1 --2- 172.21.15.40:0/1560664912 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7f806ae40 0x7ff7f806d300 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:36.165+0000 7ff81b55e640 1 --2- 172.21.15.40:0/1560664912 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8141425a0 0x7ff8140c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:36.165+0000 7ff81b55e640 1 --2- 172.21.15.40:0/1560664912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8141420f0 0x7ff8141f5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:36.165+0000 7ff81b55e640 1 --2- 172.21.15.40:0/1560664912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8140c9f90 0x7ff81413c760 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:36.165+0000 7ff81b55e640 1 -- 172.21.15.40:0/1560664912 >> 172.21.15.40:0/1560664912 conn(0x7ff814069720 msgr2=0x7ff814072b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:36.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:36.165+0000 7ff81b55e640 1 -- 172.21.15.40:0/1560664912 shutdown_connections 2023-10-18T08:57:36.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:36.165+0000 7ff81b55e640 1 -- 172.21.15.40:0/1560664912 wait complete. 2023-10-18T08:57:36.354 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-18T08:57:37.010 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T08:57:37.008+0000 7f6a11f15640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2023-10-18T08:57:41.356 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T08:57:41.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.597+0000 7ff5626da640 1 -- 172.21.15.40:0/3139915248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff55c1586a0 msgr2=0x7ff55c0cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.597+0000 7ff5626da640 1 --2- 172.21.15.40:0/3139915248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff55c1586a0 0x7ff55c0cc990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff55c06c840 tx=0x7ff54402bd90 comp rx=0 tx=0).stop 2023-10-18T08:57:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.597+0000 7ff5626da640 1 -- 172.21.15.40:0/3139915248 shutdown_connections 2023-10-18T08:57:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.597+0000 7ff5626da640 1 --2- 172.21.15.40:0/3139915248 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff55c0cced0 0x7ff55c0cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.597+0000 7ff5626da640 1 --2- 172.21.15.40:0/3139915248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff55c1586a0 0x7ff55c0cc990 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.597+0000 7ff5626da640 1 -- 172.21.15.40:0/3139915248 >> 172.21.15.40:0/3139915248 conn(0x7ff55c06cee0 msgr2=0x7ff55c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.597+0000 7ff5626da640 1 -- 172.21.15.40:0/3139915248 shutdown_connections 2023-10-18T08:57:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.597+0000 7ff5626da640 1 -- 172.21.15.40:0/3139915248 wait complete. 2023-10-18T08:57:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.599+0000 7ff5626da640 1 Processor -- start 2023-10-18T08:57:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.599+0000 7ff5626da640 1 -- start start 2023-10-18T08:57:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.600+0000 7ff5626da640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff55c06d710 0x7ff55c0cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:41.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.600+0000 7ff5626da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff55c0cced0 0x7ff55c148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:41.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.600+0000 7ff5626da640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff55c1586a0 0x7ff55c14b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:41.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.600+0000 7ff5626da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c0cc310 con 0x7ff55c0cced0 2023-10-18T08:57:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.600+0000 7ff5626da640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c158dc0 con 0x7ff55c1586a0 2023-10-18T08:57:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.600+0000 7ff5626da640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c149280 con 0x7ff55c06d710 2023-10-18T08:57:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.601+0000 7ff5537fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff55c0cced0 0x7ff55c148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.601+0000 7ff5537fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff55c0cced0 0x7ff55c148d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53040/0 (socket says 172.21.15.40:53040) 2023-10-18T08:57:41.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.601+0000 7ff5537fe640 1 -- 172.21.15.40:0/732078045 learned_addr learned my addr 172.21.15.40:0/732078045 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.601+0000 7ff55bfff640 1 --2- 172.21.15.40:0/732078045 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff55c06d710 0x7ff55c0cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.601+0000 7ff560c77640 1 --2- 172.21.15.40:0/732078045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff55c1586a0 0x7ff55c14b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.601+0000 7ff5537fe640 1 -- 172.21.15.40:0/732078045 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff55c06d710 msgr2=0x7ff55c0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.601+0000 7ff5537fe640 1 --2- 172.21.15.40:0/732078045 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff55c06d710 0x7ff55c0cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.601+0000 7ff5537fe640 1 -- 172.21.15.40:0/732078045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff55c1586a0 msgr2=0x7ff55c14b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:41.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.601+0000 7ff5537fe640 1 --2- 172.21.15.40:0/732078045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff55c1586a0 0x7ff55c14b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.601+0000 7ff5537fe640 1 -- 172.21.15.40:0/732078045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff544009560 con 0x7ff55c0cced0 2023-10-18T08:57:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.601+0000 7ff55bfff640 1 --2- 172.21.15.40:0/732078045 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff55c06d710 0x7ff55c0cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.601+0000 7ff560c77640 1 --2- 172.21.15.40:0/732078045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff55c1586a0 0x7ff55c14b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:41.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.602+0000 7ff5537fe640 1 --2- 172.21.15.40:0/732078045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff55c0cced0 0x7ff55c148d40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff544036040 tx=0x7ff54402dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.602+0000 7ff5597fa640 1 -- 172.21.15.40:0/732078045 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544003800 con 0x7ff55c0cced0 2023-10-18T08:57:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.602+0000 7ff5597fa640 1 -- 172.21.15.40:0/732078045 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff544003a50 con 0x7ff55c0cced0 2023-10-18T08:57:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.602+0000 7ff5597fa640 1 -- 172.21.15.40:0/732078045 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544003e00 con 0x7ff55c0cced0 2023-10-18T08:57:41.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.602+0000 7ff5626da640 1 -- 172.21.15.40:0/732078045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55c159010 con 0x7ff55c0cced0 2023-10-18T08:57:41.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.602+0000 7ff5626da640 1 -- 172.21.15.40:0/732078045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55c1595d0 con 0x7ff55c0cced0 2023-10-18T08:57:41.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.603+0000 7ff5626da640 1 -- 172.21.15.40:0/732078045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff55c0cc900 con 0x7ff55c0cced0 2023-10-18T08:57:41.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.604+0000 7ff5597fa640 1 -- 172.21.15.40:0/732078045 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff54403e450 con 0x7ff55c0cced0 2023-10-18T08:57:41.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.605+0000 7ff5597fa640 1 --2- 172.21.15.40:0/732078045 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff53c06ae40 0x7ff53c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:41.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.605+0000 7ff5597fa640 1 -- 172.21.15.40:0/732078045 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5440abf40 con 0x7ff55c0cced0 2023-10-18T08:57:41.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.605+0000 7ff5597fa640 1 -- 172.21.15.40:0/732078045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=50}) v3 -- 0x7ff53c072970 con 0x7ff55c0cced0 2023-10-18T08:57:41.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.606+0000 7ff55bfff640 1 --2- 172.21.15.40:0/732078045 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff53c06ae40 0x7ff53c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:41.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.607+0000 7ff55bfff640 1 --2- 172.21.15.40:0/732078045 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff53c06ae40 0x7ff53c06d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0b06d0 tx=0x7ff54c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:41.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.610+0000 7ff5597fa640 1 -- 172.21.15.40:0/732078045 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff544042060 con 0x7ff55c0cced0 2023-10-18T08:57:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.807+0000 7ff5626da640 1 -- 172.21.15.40:0/732078045 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff55c066fd0 con 0x7ff55c0cced0 2023-10-18T08:57:41.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.810+0000 7ff5597fa640 1 -- 172.21.15.40:0/732078045 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v49) v1 ==== 81+0+34055 (secure 0 0 0) 0x7ff544077110 con 0x7ff55c0cced0 2023-10-18T08:57:41.814 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:41.814 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T08:57:41.815 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 49, 2023-10-18T08:57:41.815 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T08:57:41.815 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T08:57:41.815 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T08:57:40.026918+0000", 2023-10-18T08:57:41.815 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T08:57:41.816 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T08:57:41.816 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T08:57:41.816 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T08:57:41.816 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T08:57:41.816 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T08:57:41.817 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T08:57:41.817 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T08:57:41.817 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T08:57:41.817 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:41.817 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T08:57:41.817 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T08:57:41.818 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T08:57:41.818 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T08:57:41.818 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T08:57:41.818 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 7, 2023-10-18T08:57:41.818 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T08:57:41.819 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T08:57:41.819 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T08:57:41.819 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T08:57:41.819 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T08:57:41.819 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T08:57:41.820 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.820 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T08:57:41.820 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T08:57:41.820 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T08:57:41.820 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T08:57:41.821 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T08:57:41.821 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:41.821 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:41.821 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:41.821 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:41.821 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:41.822 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:41.822 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:41.822 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:41.822 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:41.822 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:41.823 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T08:57:41.823 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T08:57:41.823 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T08:57:41.823 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T08:57:41.823 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T08:57:41.824 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:41.824 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:41.824 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:41.824 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:41.824 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:41.825 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:41.825 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:41.825 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.825 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "47", 2023-10-18T08:57:41.825 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:41.826 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:41.826 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:41.826 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:41.826 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:41.826 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:41.826 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:41.827 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:41.827 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:41.827 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:41.827 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:41.827 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:41.828 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:41.828 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:41.828 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:41.828 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:41.828 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:41.828 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:41.829 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:41.829 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:41.829 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:41.829 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:41.829 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:41.830 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:41.830 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:41.830 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:41.830 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.830 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:41.831 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:41.831 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:41.831 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:41.831 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:41.831 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:41.832 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:41.832 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:41.832 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:41.832 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:41.832 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:41.832 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:41.833 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T08:57:41.833 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T08:57:41.833 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.833 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:41.833 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T08:57:41.834 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.834 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:41.834 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T08:57:41.834 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T08:57:41.834 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T08:57:41.835 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T08:57:41.835 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T08:57:41.835 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:41.835 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:41.835 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:41.836 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.836 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.836 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.836 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 6, 2023-10-18T08:57:41.836 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T08:57:41.836 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:57:12.765969+0000", 2023-10-18T08:57:41.837 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T08:57:41.837 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T08:57:41.837 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:41.837 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:41.837 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:41.838 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:41.838 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:41.838 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:41.838 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:41.838 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:41.838 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:41.839 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:41.839 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T08:57:41.839 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T08:57:41.839 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T08:57:41.839 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T08:57:41.840 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T08:57:41.840 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:41.840 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:41.840 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:41.840 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:41.840 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:41.841 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:41.841 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:41.841 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.841 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "47", 2023-10-18T08:57:41.841 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:41.842 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:41.842 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:41.842 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:41.842 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:41.842 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:41.843 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:41.843 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:41.843 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:41.843 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:41.843 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:41.843 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:41.844 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:41.844 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:41.844 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:41.844 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:41.844 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:41.844 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:41.845 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:41.845 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:41.845 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:41.845 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:41.845 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:41.846 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:41.846 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:41.846 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:41.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.846 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:41.847 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:41.847 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:41.847 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:41.847 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:41.847 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:41.847 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:41.848 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:41.848 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:41.848 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:41.848 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:41.848 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:41.849 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T08:57:41.849 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T08:57:41.849 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T08:57:41.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.850 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:41.850 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T08:57:41.850 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T08:57:41.850 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.851 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:41.851 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T08:57:41.851 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T08:57:41.851 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T08:57:41.851 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T08:57:41.852 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T08:57:41.852 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:41.852 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:41.852 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:41.852 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.853 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.853 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 7, 2023-10-18T08:57:41.853 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T08:57:41.853 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:57:13.851110+0000", 2023-10-18T08:57:41.853 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T08:57:41.854 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T08:57:41.854 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:41.854 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:41.854 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:41.854 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:41.855 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:41.855 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:41.855 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:41.855 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:41.855 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:41.855 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:41.856 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T08:57:41.856 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T08:57:41.856 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T08:57:41.856 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T08:57:41.857 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T08:57:41.857 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:41.857 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:41.857 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:41.857 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:41.857 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:41.858 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:41.858 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:41.858 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.858 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "47", 2023-10-18T08:57:41.858 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:41.859 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:41.859 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:41.859 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:41.859 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:41.859 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:41.860 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:41.860 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:41.860 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:41.860 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:41.860 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:41.861 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:41.861 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:41.861 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:41.861 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:41.861 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:41.862 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:41.862 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:41.862 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:41.862 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:41.862 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:41.863 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:41.863 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:41.863 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:41.863 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:41.863 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:41.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.864 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:41.864 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:41.864 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:41.864 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:41.865 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:41.865 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:41.865 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:41.865 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:41.865 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:41.865 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:41.866 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:41.866 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:41.866 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T08:57:41.866 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T08:57:41.866 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.867 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:41.867 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T08:57:41.867 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T08:57:41.867 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.867 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.868 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:41.868 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T08:57:41.868 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T08:57:41.868 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T08:57:41.868 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T08:57:41.869 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T08:57:41.869 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:41.869 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:41.869 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:41.869 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.870 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.870 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:41.870 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T08:57:41.870 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.870 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T08:57:41.871 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:41.871 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:41.871 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:41.871 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:41.871 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:41.871 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:41.872 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:41.872 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:41.872 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:41.872 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:41.872 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:41.873 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:41.873 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.873 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.873 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.873 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T08:57:41.873 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:41.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.874 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.874 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.874 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T08:57:41.874 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:41.875 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.875 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.875 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.875 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:41.875 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.875 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.876 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.876 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T08:57:41.876 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:41.876 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.876 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.877 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.877 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T08:57:41.877 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:41.877 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.877 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.878 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.878 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:41.878 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.878 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.878 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.879 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T08:57:41.879 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:41.879 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.879 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.879 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.880 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T08:57:41.880 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:41.880 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.880 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.880 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.881 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:41.881 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.881 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.881 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.881 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T08:57:41.882 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:41.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.882 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.882 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T08:57:41.883 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:41.883 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.883 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.884 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T08:57:41.884 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T08:57:41.884 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T08:57:41.884 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T08:57:41.884 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:41.885 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:41.885 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T08:57:41.885 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:41.885 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.885 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.886 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.886 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T08:57:41.886 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:41.886 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:41.886 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:41.887 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:41.887 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:41.887 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:41.887 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:41.887 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:41.888 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:41.888 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:41.888 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:41.888 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:41.888 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.888 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.889 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.889 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T08:57:41.889 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:41.889 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.889 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.890 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.890 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T08:57:41.890 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:41.890 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.890 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.891 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.891 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:41.891 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.891 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.891 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.891 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T08:57:41.892 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:41.892 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.892 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.892 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.892 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T08:57:41.893 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:41.893 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.893 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.893 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.893 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:41.894 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.894 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.894 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.894 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T08:57:41.894 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:41.895 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.895 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.895 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.895 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T08:57:41.895 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:41.896 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.896 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.896 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.896 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:41.896 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.897 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.897 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.897 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T08:57:41.897 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:41.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.898 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.898 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.898 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T08:57:41.898 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:41.898 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.899 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.899 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.899 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T08:57:41.899 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T08:57:41.899 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T08:57:41.900 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T08:57:41.900 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:41.900 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:41.900 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:41.900 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.900 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.901 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.901 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T08:57:41.901 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:41.901 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:41.902 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:41.902 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:41.902 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:41.902 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:41.902 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:41.902 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:41.903 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:41.903 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:41.903 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:41.903 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:41.903 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.904 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.904 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T08:57:41.904 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:41.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.905 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.905 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.905 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T08:57:41.905 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:41.905 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.906 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.906 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.906 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:41.906 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.906 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.907 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.907 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T08:57:41.907 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:41.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.907 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.908 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.908 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T08:57:41.908 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:41.908 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.908 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.909 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:41.909 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.909 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.909 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.909 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T08:57:41.910 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:41.910 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.910 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.910 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.910 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T08:57:41.911 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:41.911 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.911 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.911 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:41.911 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.912 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.912 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.912 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T08:57:41.912 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:41.912 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.912 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.913 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.913 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T08:57:41.913 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:41.913 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.913 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.913 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.914 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T08:57:41.914 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T08:57:41.914 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T08:57:41.914 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T08:57:41.914 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:41.915 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:41.915 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:41.915 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.915 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.915 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.915 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T08:57:41.916 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:41.916 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:41.916 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:41.916 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:41.916 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:41.917 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:41.917 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:41.917 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:41.917 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:41.917 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:41.918 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:41.918 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:41.918 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.918 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.918 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.918 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T08:57:41.919 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:41.919 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.919 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.919 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.919 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T08:57:41.920 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:41.920 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.920 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.920 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.920 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:41.920 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.921 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.921 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.921 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T08:57:41.921 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:41.921 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.922 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.922 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.922 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T08:57:41.922 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:41.922 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.923 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.923 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.923 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:41.923 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.923 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.924 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.924 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T08:57:41.924 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:41.924 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.924 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.925 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.925 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T08:57:41.925 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:41.925 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.925 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.926 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.926 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:41.926 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.926 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.926 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.927 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T08:57:41.927 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:41.927 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.927 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.928 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T08:57:41.928 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:41.928 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.928 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.929 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T08:57:41.929 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T08:57:41.929 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T08:57:41.929 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T08:57:41.929 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:41.930 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull", 2023-10-18T08:57:41.930 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:41.930 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:41.930 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.930 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.931 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.931 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T08:57:41.931 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:41.931 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:41.931 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:41.932 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:41.932 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:41.932 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:41.932 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:41.932 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:41.932 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:41.933 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:41.933 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:41.933 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:41.933 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.933 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.934 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.934 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T08:57:41.934 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:41.934 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.934 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.935 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.935 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T08:57:41.935 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:41.935 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.935 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.936 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.936 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:41.936 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.936 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.936 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.937 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T08:57:41.937 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:41.937 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.937 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.937 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.938 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T08:57:41.938 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:41.938 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.938 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.938 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.939 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:41.939 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.939 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.939 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.939 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T08:57:41.940 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:41.940 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.940 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.940 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.940 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T08:57:41.940 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:41.941 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.941 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.941 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.941 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:41.941 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.941 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.942 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.942 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T08:57:41.942 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:41.942 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.942 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.943 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.943 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T08:57:41.943 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:41.943 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.943 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.943 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.944 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T08:57:41.944 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T08:57:41.944 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T08:57:41.944 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T08:57:41.944 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:41.945 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:41.945 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:41.945 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.945 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.945 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.945 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T08:57:41.946 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:41.946 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:41.946 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:41.946 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:41.946 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:41.947 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:41.947 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:41.947 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:41.947 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:41.947 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:41.947 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:41.948 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:41.948 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.948 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.948 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.948 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T08:57:41.948 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:41.949 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.949 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.949 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.949 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T08:57:41.949 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:41.950 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.950 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.950 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.950 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:41.950 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.951 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.951 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.951 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T08:57:41.951 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:41.951 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.952 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.952 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.952 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T08:57:41.952 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:41.952 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.953 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.953 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.953 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:41.953 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.953 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.953 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.954 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T08:57:41.954 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:41.954 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.954 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.954 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.955 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T08:57:41.955 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:41.955 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.955 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.955 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.956 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:41.956 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.956 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.956 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.956 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T08:57:41.957 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:41.957 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.957 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.957 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.957 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T08:57:41.958 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:41.958 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.958 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.958 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.958 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T08:57:41.958 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T08:57:41.959 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T08:57:41.959 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T08:57:41.959 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:41.959 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:41.959 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:41.960 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.960 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.960 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.960 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T08:57:41.960 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:41.961 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:41.961 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:41.961 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:41.961 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:41.961 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:41.962 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:41.962 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:41.962 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:41.962 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:41.962 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:41.963 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:41.963 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.963 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.963 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.963 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T08:57:41.964 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:41.964 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.964 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.964 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.964 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T08:57:41.965 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:41.965 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.965 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.965 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.965 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:41.966 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.966 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.966 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.966 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T08:57:41.966 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:41.967 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.967 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.967 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.967 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T08:57:41.967 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:41.967 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.968 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.968 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.968 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:41.968 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.968 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.969 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.969 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T08:57:41.969 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:41.969 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.969 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.970 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.970 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T08:57:41.970 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:41.970 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.970 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.971 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.971 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:41.971 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.971 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.971 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T08:57:41.972 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:41.972 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.972 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.972 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.972 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T08:57:41.973 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:41.973 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.973 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.973 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T08:57:41.974 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T08:57:41.974 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T08:57:41.974 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T08:57:41.974 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:41.974 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:41.975 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T08:57:41.975 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:41.975 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.975 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.975 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.976 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T08:57:41.976 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:41.976 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:41.976 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:41.976 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:41.977 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:41.977 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:41.977 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:41.977 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:41.977 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:41.977 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:41.978 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:41.978 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:41.978 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.978 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.978 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.979 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T08:57:41.979 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:41.979 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.979 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.979 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.979 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T08:57:41.980 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:41.980 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.980 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.980 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.980 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:41.981 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.981 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.981 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.981 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T08:57:41.981 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:41.981 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.982 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.982 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.982 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T08:57:41.982 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:41.982 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.983 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.983 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.983 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:41.983 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.983 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.984 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.984 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T08:57:41.984 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:41.984 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.984 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.985 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.985 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T08:57:41.985 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:41.985 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.985 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.986 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.986 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:41.986 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:41.986 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.986 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:41.987 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T08:57:41.987 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:41.987 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.987 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.987 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:41.987 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T08:57:41.988 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:41.988 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.988 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.988 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.988 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T08:57:41.989 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T08:57:41.989 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T08:57:41.989 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T08:57:41.989 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:41.989 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:41.989 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:41.990 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:41.990 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:41.990 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:41.990 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T08:57:41.990 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.991 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T08:57:41.991 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:41.991 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:41.991 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:41.991 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:41.992 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:41.992 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T08:57:41.992 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:41.992 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.992 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.993 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T08:57:41.993 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:41.993 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:41.993 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:41.993 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:41.994 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:41.994 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T08:57:41.994 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:41.994 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.994 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.994 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T08:57:41.995 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:41.995 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:41.995 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:41.995 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:41.995 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:41.996 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T08:57:41.996 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:41.996 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.996 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.996 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T08:57:41.996 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:41.997 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:41.997 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:41.997 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:41.997 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:41.997 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T08:57:41.997 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:41.998 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:41.998 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:41.998 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T08:57:41.998 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:41.998 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:41.999 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:41.999 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:41.999 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:41.999 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T08:57:41.999 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:42.000 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:42.000 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:42.000 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T08:57:42.000 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:42.000 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:42.000 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:42.001 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:42.001 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:42.001 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T08:57:42.001 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:42.001 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:42.002 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:42.002 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T08:57:42.002 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:42.002 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:42.003 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:42.003 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:42.003 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:42.003 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T08:57:42.003 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:42.004 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:42.004 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:42.004 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T08:57:42.004 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:42.004 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:42.005 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:42.005 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:42.005 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:42.005 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T08:57:42.005 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:42.006 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:42.006 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:42.006 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T08:57:42.006 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T08:57:42.006 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T08:57:42.007 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T08:57:42.007 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T08:57:42.007 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T08:57:42.007 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T08:57:42.007 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T08:57:42.008 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T08:57:42.008 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T08:57:42.008 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T08:57:42.008 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T08:57:42.008 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T08:57:42.009 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T08:57:42.009 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:42.009 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:42.009 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T08:57:42.009 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T08:57:42.009 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T08:57:42.010 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T08:57:42.010 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T08:57:42.010 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T08:57:42.010 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T08:57:42.010 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T08:57:42.011 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T08:57:42.011 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T08:57:42.011 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T08:57:42.011 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:42.011 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T08:57:42.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.813+0000 7ff5626da640 1 -- 172.21.15.40:0/732078045 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff53c06ae40 msgr2=0x7ff53c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:42.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.813+0000 7ff5626da640 1 --2- 172.21.15.40:0/732078045 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff53c06ae40 0x7ff53c06d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0b06d0 tx=0x7ff54c009040 comp rx=0 tx=0).stop 2023-10-18T08:57:42.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.813+0000 7ff5626da640 1 -- 172.21.15.40:0/732078045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff55c0cced0 msgr2=0x7ff55c148d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:42.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.813+0000 7ff5626da640 1 --2- 172.21.15.40:0/732078045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff55c0cced0 0x7ff55c148d40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff544036040 tx=0x7ff54402dd60 comp rx=0 tx=0).stop 2023-10-18T08:57:42.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.813+0000 7ff5626da640 1 -- 172.21.15.40:0/732078045 shutdown_connections 2023-10-18T08:57:42.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.813+0000 7ff5626da640 1 --2- 172.21.15.40:0/732078045 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff53c06ae40 0x7ff53c06d300 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:42.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.813+0000 7ff5626da640 1 --2- 172.21.15.40:0/732078045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff55c1586a0 0x7ff55c14b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:42.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.813+0000 7ff5626da640 1 --2- 172.21.15.40:0/732078045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff55c0cced0 0x7ff55c148d40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:42.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.813+0000 7ff5626da640 1 --2- 172.21.15.40:0/732078045 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff55c06d710 0x7ff55c0cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:42.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.813+0000 7ff5626da640 1 -- 172.21.15.40:0/732078045 >> 172.21.15.40:0/732078045 conn(0x7ff55c06cee0 msgr2=0x7ff55c0709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:42.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.814+0000 7ff5626da640 1 -- 172.21.15.40:0/732078045 shutdown_connections 2023-10-18T08:57:42.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:41.814+0000 7ff5626da640 1 -- 172.21.15.40:0/732078045 wait complete. 2023-10-18T08:57:42.018 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-18T08:57:42.018 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:57:42.018 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:57:42.049 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T08:57:42.047+0000 7f6a11f15640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-10-18T08:57:42.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:42.080 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2023-10-18T08:57:42.531 INFO:teuthology.orchestra.run.smithi040.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2023-10-18T08:57:42.531 INFO:teuthology.orchestra.run.smithi040.stderr:50+0 records in 2023-10-18T08:57:42.532 INFO:teuthology.orchestra.run.smithi040.stderr:50+0 records out 2023-10-18T08:57:42.532 INFO:teuthology.orchestra.run.smithi040.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.381448 s, 137 MB/s 2023-10-18T08:57:42.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T08:57:42.533 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T08:57:42.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.759+0000 7fd4d9575640 1 -- 172.21.15.40:0/2484275676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4bc003fe0 msgr2=0x7fd4bc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:42.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.759+0000 7fd4d9575640 1 --2- 172.21.15.40:0/2484275676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4bc003fe0 0x7fd4bc0047c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8006b00 tx=0x7fd4bc0057f0 comp rx=0 tx=0).stop 2023-10-18T08:57:42.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.759+0000 7fd4d8d74640 1 -- 172.21.15.40:0/2484275676 reap_dead start 2023-10-18T08:57:42.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.759+0000 7fd4d9575640 1 -- 172.21.15.40:0/2484275676 shutdown_connections 2023-10-18T08:57:42.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.759+0000 7fd4d9575640 1 -- 172.21.15.40:0/2484275676 >> 172.21.15.40:0/2484275676 conn(0x7fd4d4069720 msgr2=0x7fd4d4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:42.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.759+0000 7fd4d9575640 1 -- 172.21.15.40:0/2484275676 shutdown_connections 2023-10-18T08:57:42.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.759+0000 7fd4d9575640 1 -- 172.21.15.40:0/2484275676 wait complete. 2023-10-18T08:57:42.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.761+0000 7fd4d9575640 1 Processor -- start 2023-10-18T08:57:42.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.761+0000 7fd4d9575640 1 -- start start 2023-10-18T08:57:42.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.762+0000 7fd4d9575640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4bc003fe0 0x7fd4d413a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:42.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.762+0000 7fd4d9575640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4d40adb30 0x7fd4d40af170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:42.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.762+0000 7fd4d9575640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4d41544e0 0x7fd4d41517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:42.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.762+0000 7fd4d9575640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40aece0 con 0x7fd4d41544e0 2023-10-18T08:57:42.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.762+0000 7fd4d9575640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40ae0d0 con 0x7fd4d40adb30 2023-10-18T08:57:42.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.762+0000 7fd4d9575640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40ae3b0 con 0x7fd4bc003fe0 2023-10-18T08:57:42.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.763+0000 7fd4d8d74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4d41544e0 0x7fd4d41517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:42.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.763+0000 7fd4d8d74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4d41544e0 0x7fd4d41517e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53048/0 (socket says 172.21.15.40:53048) 2023-10-18T08:57:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.763+0000 7fd4d8d74640 1 -- 172.21.15.40:0/2306511965 learned_addr learned my addr 172.21.15.40:0/2306511965 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.763+0000 7fd4d37fe640 1 --2- 172.21.15.40:0/2306511965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4d40adb30 0x7fd4d40af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.763+0000 7fd4d3fff640 1 --2- 172.21.15.40:0/2306511965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4bc003fe0 0x7fd4d413a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.763+0000 7fd4d8d74640 1 -- 172.21.15.40:0/2306511965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4bc003fe0 msgr2=0x7fd4d413a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:42.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.763+0000 7fd4d8d74640 1 --2- 172.21.15.40:0/2306511965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4bc003fe0 0x7fd4d413a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:42.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.763+0000 7fd4d8d74640 1 -- 172.21.15.40:0/2306511965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4d40adb30 msgr2=0x7fd4d40af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:42.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.763+0000 7fd4d8d74640 1 --2- 172.21.15.40:0/2306511965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4d40adb30 0x7fd4d40af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:42.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.763+0000 7fd4d8d74640 1 -- 172.21.15.40:0/2306511965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d40ae690 con 0x7fd4d41544e0 2023-10-18T08:57:42.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.763+0000 7fd4d8d74640 1 --2- 172.21.15.40:0/2306511965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4d41544e0 0x7fd4d41517e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40b26c0 tx=0x7fd4cc00eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:42.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.764+0000 7fd4d0ff9640 1 -- 172.21.15.40:0/2306511965 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00cc30 con 0x7fd4d41544e0 2023-10-18T08:57:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.764+0000 7fd4d0ff9640 1 -- 172.21.15.40:0/2306511965 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd4cc00ce80 con 0x7fd4d41544e0 2023-10-18T08:57:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.764+0000 7fd4d9575640 1 -- 172.21.15.40:0/2306511965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d40ae970 con 0x7fd4d41544e0 2023-10-18T08:57:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.764+0000 7fd4d0ff9640 1 -- 172.21.15.40:0/2306511965 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc003bd0 con 0x7fd4d41544e0 2023-10-18T08:57:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.764+0000 7fd4d9575640 1 -- 172.21.15.40:0/2306511965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d4154b40 con 0x7fd4d41544e0 2023-10-18T08:57:42.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.765+0000 7fd4d9575640 1 -- 172.21.15.40:0/2306511965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d4066e00 con 0x7fd4d41544e0 2023-10-18T08:57:42.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.766+0000 7fd4d0ff9640 1 -- 172.21.15.40:0/2306511965 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd4cc003d30 con 0x7fd4d41544e0 2023-10-18T08:57:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.767+0000 7fd4d0ff9640 1 --2- 172.21.15.40:0/2306511965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd4c006ae40 0x7fd4c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.767+0000 7fd4d0ff9640 1 -- 172.21.15.40:0/2306511965 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4cc089550 con 0x7fd4d41544e0 2023-10-18T08:57:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.767+0000 7fd4d0ff9640 1 -- 172.21.15.40:0/2306511965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=51}) v3 -- 0x7fd4c0072970 con 0x7fd4d41544e0 2023-10-18T08:57:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.768+0000 7fd4d3fff640 1 --2- 172.21.15.40:0/2306511965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd4c006ae40 0x7fd4c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:42.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.769+0000 7fd4d3fff640 1 --2- 172.21.15.40:0/2306511965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd4c006ae40 0x7fd4c006d300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4000c00 tx=0x7fd4c4006b00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.771+0000 7fd4d0ff9640 1 -- 172.21.15.40:0/2306511965 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4cc054560 con 0x7fd4d41544e0 2023-10-18T08:57:42.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.967+0000 7fd4d9575640 1 -- 172.21.15.40:0/2306511965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd4d40caf90 con 0x7fd4d41544e0 2023-10-18T08:57:42.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.969+0000 7fd4d0ff9640 1 -- 172.21.15.40:0/2306511965 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) v1 ==== 81+0+34103 (secure 0 0 0) 0x7fd4cc01a070 con 0x7fd4d41544e0 2023-10-18T08:57:42.972 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:42.972 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T08:57:42.973 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 50, 2023-10-18T08:57:42.973 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T08:57:42.973 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T08:57:42.973 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T08:57:42.047948+0000", 2023-10-18T08:57:42.973 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T08:57:42.974 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T08:57:42.974 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T08:57:42.974 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T08:57:42.974 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T08:57:42.974 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T08:57:42.975 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T08:57:42.975 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T08:57:42.975 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T08:57:42.975 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:42.975 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T08:57:42.975 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T08:57:42.976 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T08:57:42.976 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T08:57:42.976 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T08:57:42.976 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 7, 2023-10-18T08:57:42.976 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T08:57:42.976 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T08:57:42.977 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T08:57:42.977 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T08:57:42.977 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T08:57:42.977 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T08:57:42.977 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:42.977 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T08:57:42.978 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T08:57:42.978 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T08:57:42.978 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T08:57:42.978 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T08:57:42.978 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:42.979 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:42.979 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:42.979 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:42.979 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:42.979 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:42.980 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:42.980 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:42.980 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:42.980 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:42.980 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T08:57:42.980 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T08:57:42.981 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T08:57:42.981 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T08:57:42.981 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T08:57:42.981 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:42.981 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:42.981 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:42.982 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:42.982 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:42.982 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:42.982 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:42.982 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:42.983 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "47", 2023-10-18T08:57:42.983 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:42.983 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:42.983 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:42.983 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:42.984 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:42.984 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:42.984 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:42.984 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:42.984 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:42.985 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:42.985 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:42.985 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:42.985 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:42.985 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:42.986 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:42.986 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:42.986 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:42.986 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:42.986 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:42.986 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:42.987 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:42.987 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:42.987 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:42.987 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:42.987 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:42.987 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:42.988 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:42.988 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:42.988 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:42.988 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:42.988 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:42.988 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:42.989 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:42.989 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:42.989 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:42.989 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:42.989 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:42.989 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:42.990 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:42.990 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T08:57:42.990 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T08:57:42.990 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:42.990 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:42.991 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T08:57:42.991 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:42.991 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:42.991 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T08:57:42.991 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T08:57:42.991 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T08:57:42.992 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T08:57:42.992 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T08:57:42.992 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:42.992 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:42.992 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:42.993 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:42.993 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:42.993 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:42.993 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 6, 2023-10-18T08:57:42.993 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T08:57:42.993 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:57:12.765969+0000", 2023-10-18T08:57:42.994 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T08:57:42.994 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T08:57:42.994 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:42.994 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:42.994 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:42.994 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:42.995 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:42.995 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:42.995 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:42.995 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:42.995 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:42.995 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:42.996 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T08:57:42.996 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T08:57:42.996 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T08:57:42.996 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T08:57:42.996 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T08:57:42.996 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:42.997 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:42.997 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:42.997 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:42.997 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:42.997 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:42.998 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:42.998 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:42.998 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "47", 2023-10-18T08:57:42.999 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:42.999 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:42.999 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:42.999 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:42.999 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:43.000 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:43.000 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:43.000 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:43.000 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:43.000 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:43.000 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:43.001 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:43.001 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:43.001 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:43.001 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:43.001 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:43.002 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:43.002 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:43.002 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:43.002 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:43.002 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:43.002 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:43.003 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:43.003 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:43.003 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:43.003 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:43.003 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.004 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:43.004 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:43.004 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:43.004 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:43.004 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:43.005 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:43.005 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:43.005 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:43.005 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:43.005 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:43.005 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:43.006 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:43.006 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T08:57:43.006 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T08:57:43.006 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T08:57:43.006 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.007 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:43.007 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T08:57:43.007 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T08:57:43.007 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.007 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.007 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T08:57:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T08:57:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T08:57:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T08:57:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T08:57:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 7, 2023-10-18T08:57:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T08:57:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:57:13.851110+0000", 2023-10-18T08:57:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T08:57:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T08:57:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T08:57:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T08:57:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T08:57:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T08:57:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T08:57:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "47", 2023-10-18T08:57:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T08:57:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T08:57:43.023 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.023 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:43.023 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T08:57:43.023 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T08:57:43.023 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.024 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:43.024 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T08:57:43.024 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T08:57:43.024 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T08:57:43.025 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T08:57:43.025 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T08:57:43.025 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:43.025 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:43.025 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:43.025 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.026 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.026 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:43.026 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T08:57:43.026 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.026 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T08:57:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:43.029 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:43.029 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:43.029 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.029 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.029 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.029 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T08:57:43.030 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:43.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.030 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.030 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.030 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T08:57:43.031 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:43.031 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.031 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.031 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.031 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:43.032 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.032 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.032 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.032 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T08:57:43.032 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:43.032 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.033 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.033 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.033 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T08:57:43.033 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:43.033 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.033 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.034 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.034 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:43.034 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.034 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.034 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.035 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T08:57:43.035 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:43.035 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.035 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.037 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.037 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T08:57:43.037 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:43.037 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.037 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.037 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.038 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:43.038 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.038 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.038 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.038 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T08:57:43.039 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:43.039 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.039 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.039 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.039 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T08:57:43.039 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:43.040 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.040 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.040 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.040 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T08:57:43.040 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T08:57:43.041 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T08:57:43.041 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T08:57:43.041 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:43.041 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:43.041 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T08:57:43.042 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:43.042 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.042 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.042 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.042 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T08:57:43.043 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:43.043 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:43.043 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:43.043 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:43.043 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:43.043 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:43.044 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:43.044 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:43.044 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:43.044 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:43.044 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T08:57:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T08:57:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T08:57:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T08:57:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.050 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T08:57:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T08:57:43.051 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T08:57:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T08:57:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T08:57:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T08:57:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T08:57:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T08:57:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T08:57:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T08:57:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T08:57:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T08:57:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T08:57:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T08:57:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T08:57:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T08:57:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T08:57:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T08:57:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T08:57:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T08:57:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T08:57:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T08:57:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T08:57:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T08:57:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T08:57:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T08:57:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T08:57:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T08:57:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T08:57:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T08:57:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T08:57:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T08:57:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T08:57:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T08:57:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T08:57:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull", 2023-10-18T08:57:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T08:57:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T08:57:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.090 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.090 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.090 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T08:57:43.090 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:43.090 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.091 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.091 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.091 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:43.091 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.091 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.091 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.092 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T08:57:43.092 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:43.092 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.092 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.092 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.093 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T08:57:43.093 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:43.093 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.093 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.093 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.093 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:43.094 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.094 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.094 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T08:57:43.094 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:43.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.095 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.095 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.095 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T08:57:43.095 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:43.096 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.096 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.096 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:43.096 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.097 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.097 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T08:57:43.097 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:43.097 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.097 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.097 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.098 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T08:57:43.098 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:43.098 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.098 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.099 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T08:57:43.099 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T08:57:43.099 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T08:57:43.099 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T08:57:43.099 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:43.099 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:43.100 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:43.100 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.100 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.100 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T08:57:43.100 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:43.101 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:43.101 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:43.101 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:43.101 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:43.101 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:43.101 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:43.102 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:43.102 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:43.102 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:43.102 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:43.102 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:43.102 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.103 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.103 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.103 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T08:57:43.103 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:43.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.103 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.104 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.104 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T08:57:43.104 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:43.104 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.104 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.104 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.105 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:43.105 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.105 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.105 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T08:57:43.105 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:43.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.106 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.106 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.106 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T08:57:43.106 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:43.106 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.107 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.107 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:43.107 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.107 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.107 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T08:57:43.108 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:43.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.108 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.108 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T08:57:43.108 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:43.109 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.109 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.109 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:43.109 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.109 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.110 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.110 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T08:57:43.110 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:43.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.111 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.111 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T08:57:43.111 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:43.111 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.111 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.112 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T08:57:43.112 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T08:57:43.112 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T08:57:43.112 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T08:57:43.112 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:43.112 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:43.113 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:43.113 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.113 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.113 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T08:57:43.113 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:43.114 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:43.114 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:43.114 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:43.114 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:43.114 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:43.115 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:43.115 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:43.115 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:43.115 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:43.115 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:43.115 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:43.116 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.116 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.116 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T08:57:43.116 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:43.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.117 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.117 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T08:57:43.117 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:43.117 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.118 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.118 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:43.118 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.119 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.119 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T08:57:43.119 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:43.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.120 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.120 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T08:57:43.120 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:43.120 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.120 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.121 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:43.121 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.121 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.121 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T08:57:43.122 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:43.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.122 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.122 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T08:57:43.122 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:43.123 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.123 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.123 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:43.123 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.123 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.124 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T08:57:43.124 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:43.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.124 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.125 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T08:57:43.125 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:43.125 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.125 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.126 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T08:57:43.126 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T08:57:43.126 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T08:57:43.126 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T08:57:43.126 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:43.127 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:43.127 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T08:57:43.127 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:43.127 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.127 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.128 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T08:57:43.128 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:43.128 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:43.128 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:43.128 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:43.129 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:43.129 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:43.129 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:43.129 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:43.129 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:43.129 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:43.130 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:43.130 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:43.130 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.130 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.131 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T08:57:43.131 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:43.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.131 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.132 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T08:57:43.132 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:43.132 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.132 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.132 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:43.133 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.133 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.133 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.133 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T08:57:43.133 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:43.134 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.134 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.134 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T08:57:43.134 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:43.135 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.135 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.135 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:43.135 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.136 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.136 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T08:57:43.136 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:43.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.137 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.137 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T08:57:43.137 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:43.137 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.137 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.138 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:43.138 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:43.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.138 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:43.138 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T08:57:43.139 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:43.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.139 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:43.139 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T08:57:43.140 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:43.140 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.140 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.140 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T08:57:43.140 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T08:57:43.141 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T08:57:43.141 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T08:57:43.141 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:43.141 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:43.141 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T08:57:43.142 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:43.142 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:43.142 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.142 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:43.142 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T08:57:43.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.143 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T08:57:43.143 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:43.143 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:43.143 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:43.143 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:43.143 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:43.144 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T08:57:43.144 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:43.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.144 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T08:57:43.144 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:43.145 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:43.145 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:43.145 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:43.145 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:43.145 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T08:57:43.145 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:43.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.146 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.146 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T08:57:43.146 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:43.146 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:43.146 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:43.147 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:43.147 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:43.147 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T08:57:43.147 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:43.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.148 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T08:57:43.148 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:43.148 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:43.148 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:43.148 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:43.148 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:43.149 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T08:57:43.149 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:43.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.149 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T08:57:43.149 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:43.150 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:43.150 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:43.150 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:43.150 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:43.150 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T08:57:43.150 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:43.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.151 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T08:57:43.151 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:43.151 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:43.151 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:43.152 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:43.152 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:43.152 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T08:57:43.152 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:43.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.153 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T08:57:43.153 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:43.153 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:43.153 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:43.153 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:43.154 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:43.154 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T08:57:43.154 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:43.154 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:43.154 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T08:57:43.155 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:43.155 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:43.155 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:43.155 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:43.155 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:43.155 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T08:57:43.156 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:43.156 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.156 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:43.156 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T08:57:43.156 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T08:57:43.156 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T08:57:43.157 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T08:57:43.157 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T08:57:43.157 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T08:57:43.157 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T08:57:43.157 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T08:57:43.158 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T08:57:43.158 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T08:57:43.158 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T08:57:43.158 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T08:57:43.158 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T08:57:43.159 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T08:57:43.159 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:43.159 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T08:57:43.159 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T08:57:43.160 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T08:57:43.160 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T08:57:43.160 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T08:57:43.160 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T08:57:43.160 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T08:57:43.160 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T08:57:43.161 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T08:57:43.161 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T08:57:43.161 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T08:57:43.161 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:43.161 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T08:57:43.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.972+0000 7fd4d9575640 1 -- 172.21.15.40:0/2306511965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd4c006ae40 msgr2=0x7fd4c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:43.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.972+0000 7fd4d9575640 1 --2- 172.21.15.40:0/2306511965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd4c006ae40 0x7fd4c006d300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4000c00 tx=0x7fd4c4006b00 comp rx=0 tx=0).stop 2023-10-18T08:57:43.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.972+0000 7fd4d9575640 1 -- 172.21.15.40:0/2306511965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4d41544e0 msgr2=0x7fd4d41517e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:43.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.972+0000 7fd4d9575640 1 --2- 172.21.15.40:0/2306511965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4d41544e0 0x7fd4d41517e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40b26c0 tx=0x7fd4cc00eda0 comp rx=0 tx=0).stop 2023-10-18T08:57:43.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.973+0000 7fd4d9575640 1 -- 172.21.15.40:0/2306511965 shutdown_connections 2023-10-18T08:57:43.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.973+0000 7fd4d9575640 1 --2- 172.21.15.40:0/2306511965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd4c006ae40 0x7fd4c006d300 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:43.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.973+0000 7fd4d9575640 1 --2- 172.21.15.40:0/2306511965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4d41544e0 0x7fd4d41517e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:43.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.973+0000 7fd4d9575640 1 --2- 172.21.15.40:0/2306511965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4d40adb30 0x7fd4d40af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:43.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.973+0000 7fd4d9575640 1 --2- 172.21.15.40:0/2306511965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4bc003fe0 0x7fd4d413a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:43.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.973+0000 7fd4d9575640 1 -- 172.21.15.40:0/2306511965 >> 172.21.15.40:0/2306511965 conn(0x7fd4d4069720 msgr2=0x7fd4d415a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:43.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.973+0000 7fd4d9575640 1 -- 172.21.15.40:0/2306511965 shutdown_connections 2023-10-18T08:57:43.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:42.973+0000 7fd4d9575640 1 -- 172.21.15.40:0/2306511965 wait complete. 2023-10-18T08:57:43.166 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:57:43.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.407+0000 7f4246c13640 1 -- 172.21.15.40:0/2921658433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c003970 msgr2=0x7f422c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:43.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.407+0000 7f4246c13640 1 --2- 172.21.15.40:0/2921658433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c003970 0x7f422c003d90 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f423c00c260 tx=0x7f422c0057f0 comp rx=0 tx=0).stop 2023-10-18T08:57:43.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.408+0000 7f4246412640 1 -- 172.21.15.40:0/2921658433 reap_dead start 2023-10-18T08:57:43.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.408+0000 7f4246c13640 1 -- 172.21.15.40:0/2921658433 shutdown_connections 2023-10-18T08:57:43.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.408+0000 7f4246c13640 1 -- 172.21.15.40:0/2921658433 >> 172.21.15.40:0/2921658433 conn(0x7f424008acb0 msgr2=0x7f424009a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:43.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.408+0000 7f4246c13640 1 -- 172.21.15.40:0/2921658433 shutdown_connections 2023-10-18T08:57:43.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.408+0000 7f4246c13640 1 -- 172.21.15.40:0/2921658433 wait complete. 2023-10-18T08:57:43.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.410+0000 7f4246c13640 1 Processor -- start 2023-10-18T08:57:43.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.410+0000 7f4246c13640 1 -- start start 2023-10-18T08:57:43.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.411+0000 7f4246c13640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f422c003970 0x7f42400ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:43.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.411+0000 7f4246c13640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42401e3be0 0x7f42400c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.411+0000 7f4246c13640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4240153e70 0x7f42400af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.411+0000 7f4246c13640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4240157d20 con 0x7f4240153e70 2023-10-18T08:57:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.411+0000 7f4246c13640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42401e4060 con 0x7f422c003970 2023-10-18T08:57:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.411+0000 7f4246c13640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42400c88a0 con 0x7f42401e3be0 2023-10-18T08:57:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.411+0000 7f4246412640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4240153e70 0x7f42400af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.411+0000 7f4246412640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4240153e70 0x7f42400af2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53068/0 (socket says 172.21.15.40:53068) 2023-10-18T08:57:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.411+0000 7f4246412640 1 -- 172.21.15.40:0/605604646 learned_addr learned my addr 172.21.15.40:0/605604646 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:43.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.412+0000 7f4245410640 1 --2- 172.21.15.40:0/605604646 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42401e3be0 0x7f42400c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:43.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.412+0000 7f4245c11640 1 --2- 172.21.15.40:0/605604646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f422c003970 0x7f42400ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:43.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.412+0000 7f4246412640 1 -- 172.21.15.40:0/605604646 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42401e3be0 msgr2=0x7f42400c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:43.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.412+0000 7f4246412640 1 --2- 172.21.15.40:0/605604646 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42401e3be0 0x7f42400c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:43.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.412+0000 7f4246412640 1 -- 172.21.15.40:0/605604646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f422c003970 msgr2=0x7f42400ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:43.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.412+0000 7f4246412640 1 --2- 172.21.15.40:0/605604646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f422c003970 0x7f42400ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:43.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.412+0000 7f4246412640 1 -- 172.21.15.40:0/605604646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42400cf6a0 con 0x7f4240153e70 2023-10-18T08:57:43.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.412+0000 7f4246412640 1 --2- 172.21.15.40:0/605604646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4240153e70 0x7f42400af2f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f42400b3d90 tx=0x7f423800fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:43.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.413+0000 7f42367fc640 1 -- 172.21.15.40:0/605604646 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423800dda0 con 0x7f4240153e70 2023-10-18T08:57:43.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.413+0000 7f42367fc640 1 -- 172.21.15.40:0/605604646 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4238011440 con 0x7f4240153e70 2023-10-18T08:57:43.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.413+0000 7f42367fc640 1 -- 172.21.15.40:0/605604646 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238011850 con 0x7f4240153e70 2023-10-18T08:57:43.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.413+0000 7f4246c13640 1 -- 172.21.15.40:0/605604646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42400c8b30 con 0x7f4240153e70 2023-10-18T08:57:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.413+0000 7f4246c13640 1 -- 172.21.15.40:0/605604646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42400c90f0 con 0x7f4240153e70 2023-10-18T08:57:43.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.414+0000 7f4246c13640 1 -- 172.21.15.40:0/605604646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4240157e60 con 0x7f4240153e70 2023-10-18T08:57:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.415+0000 7f42367fc640 1 -- 172.21.15.40:0/605604646 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f42380119b0 con 0x7f4240153e70 2023-10-18T08:57:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.416+0000 7f42367fc640 1 --2- 172.21.15.40:0/605604646 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f422406ae40 0x7f422406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.417+0000 7f42367fc640 1 -- 172.21.15.40:0/605604646 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4238089a40 con 0x7f4240153e70 2023-10-18T08:57:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.417+0000 7f42367fc640 1 -- 172.21.15.40:0/605604646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=51}) v3 -- 0x7f4224072970 con 0x7f4240153e70 2023-10-18T08:57:43.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.417+0000 7f4245c11640 1 --2- 172.21.15.40:0/605604646 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f422406ae40 0x7f422406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:43.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.418+0000 7f4245c11640 1 --2- 172.21.15.40:0/605604646 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f422406ae40 0x7f422406d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f423c034000 tx=0x7f423c001a70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:43.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.421+0000 7f42367fc640 1 -- 172.21.15.40:0/605604646 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4238021060 con 0x7f4240153e70 2023-10-18T08:57:43.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.659+0000 7f4246c13640 1 -- 172.21.15.40:0/605604646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4240066e00 con 0x7f4240153e70 2023-10-18T08:57:43.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.660+0000 7f42367fc640 1 -- 172.21.15.40:0/605604646 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f4240157e60 con 0x7f4240153e70 2023-10-18T08:57:43.663 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:43.663 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3668356540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3668356540},{"type":"v1","addr":"172.21.15.144:6836","nonce":3668356540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4222679844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4222679844},{"type":"v1","addr":"172.21.15.40:6834","nonce":4222679844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2829399368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2829399368},{"type":"v1","addr":"172.21.15.40:6835","nonce":2829399368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:57:14.826066+0000","modified":"2023-10-18T08:57:17.903233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1438456692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1438456692},{"type":"v1","addr":"172.21.15.144:6837","nonce":1438456692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-18T08:57:43.663 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 24 2023-10-18T08:57:43.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.663+0000 7f4246c13640 1 -- 172.21.15.40:0/605604646 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f422406ae40 msgr2=0x7f422406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:43.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.663+0000 7f4246c13640 1 --2- 172.21.15.40:0/605604646 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f422406ae40 0x7f422406d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f423c034000 tx=0x7f423c001a70 comp rx=0 tx=0).stop 2023-10-18T08:57:43.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.663+0000 7f4246c13640 1 -- 172.21.15.40:0/605604646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4240153e70 msgr2=0x7f42400af2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:43.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.663+0000 7f4246c13640 1 --2- 172.21.15.40:0/605604646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4240153e70 0x7f42400af2f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f42400b3d90 tx=0x7f423800fe60 comp rx=0 tx=0).stop 2023-10-18T08:57:43.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.663+0000 7f4246c13640 1 -- 172.21.15.40:0/605604646 shutdown_connections 2023-10-18T08:57:43.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.663+0000 7f4246c13640 1 --2- 172.21.15.40:0/605604646 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f422406ae40 0x7f422406d300 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:43.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.663+0000 7f4246c13640 1 --2- 172.21.15.40:0/605604646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4240153e70 0x7f42400af2f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:43.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.663+0000 7f4246c13640 1 --2- 172.21.15.40:0/605604646 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42401e3be0 0x7f42400c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:43.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.663+0000 7f4246c13640 1 --2- 172.21.15.40:0/605604646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f422c003970 0x7f42400ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:43.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.663+0000 7f4246c13640 1 -- 172.21.15.40:0/605604646 >> 172.21.15.40:0/605604646 conn(0x7f424008acb0 msgr2=0x7f424015a640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.663+0000 7f4246c13640 1 -- 172.21.15.40:0/605604646 shutdown_connections 2023-10-18T08:57:43.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:43.664+0000 7f4246c13640 1 -- 172.21.15.40:0/605604646 wait complete. 2023-10-18T08:57:43.686 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-10-18T08:57:43.920 INFO:teuthology.orchestra.run.smithi144.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":5655,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":28.512555622000001,"mdsmap_epoch":24,"osdmap_epoch":50,"osdmap_epoch_barrier":50,"uptime":34.343483620000001} 2023-10-18T08:57:43.921 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 5655, "mdsmap_epoch": 24, "osdmap_epoch": 50, "osdmap_epoch_barrier": 50, "rank_uptime": 28.512555622, "state": "up:active", "uptime": 34.34348362, "want_state": "up:active", "whoami": 0 } 2023-10-18T08:57:43.921 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-18T08:57:43.921 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:57:43.922 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:57:43.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:43.984 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2023-10-18T08:57:44.042 INFO:teuthology.orchestra.run.smithi040.stderr:dd: failed to open 'small_file_1': Permission denied 2023-10-18T08:57:44.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T08:57:44.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:44.044 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-18T08:57:44.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:57:44.112 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-18T08:57:44.264 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T08:57:44.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.497+0000 7fc27acc3640 1 -- 172.21.15.40:0/4121950843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc274152560 msgr2=0x7fc274070f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:44.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.497+0000 7fc27acc3640 1 --2- 172.21.15.40:0/4121950843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc274152560 0x7fc274070f00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc27406d930 tx=0x7fc27002c2f0 comp rx=0 tx=0).stop 2023-10-18T08:57:44.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.497+0000 7fc27acc3640 1 -- 172.21.15.40:0/4121950843 shutdown_connections 2023-10-18T08:57:44.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.497+0000 7fc27acc3640 1 --2- 172.21.15.40:0/4121950843 >> v2:172.21.15.144:3300/0 conn(0x7fc2740ab0d0 0x7fc27406b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:44.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.497+0000 7fc27acc3640 1 --2- 172.21.15.40:0/4121950843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc274071440 0x7fc2740aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:44.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.497+0000 7fc27acc3640 1 --2- 172.21.15.40:0/4121950843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc274152560 0x7fc274070f00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:44.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.497+0000 7fc27acc3640 1 -- 172.21.15.40:0/4121950843 >> 172.21.15.40:0/4121950843 conn(0x7fc274069720 msgr2=0x7fc274069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:44.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.498+0000 7fc27acc3640 1 -- 172.21.15.40:0/4121950843 shutdown_connections 2023-10-18T08:57:44.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.498+0000 7fc27acc3640 1 -- 172.21.15.40:0/4121950843 wait complete. 2023-10-18T08:57:44.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.499+0000 7fc27acc3640 1 Processor -- start 2023-10-18T08:57:44.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.499+0000 7fc27acc3640 1 -- start start 2023-10-18T08:57:44.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.500+0000 7fc27acc3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc274071440 0x7fc274153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:44.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.500+0000 7fc27acc3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2740ab0d0 0x7fc274148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:44.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.500+0000 7fc27acc3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc274152560 0x7fc2740a9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:44.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.500+0000 7fc27acc3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27414ea60 con 0x7fc2740ab0d0 2023-10-18T08:57:44.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.500+0000 7fc27acc3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2741f6cc0 con 0x7fc274152560 2023-10-18T08:57:44.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.500+0000 7fc27acc3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27414ed00 con 0x7fc274071440 2023-10-18T08:57:44.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.501+0000 7fc2794c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2740ab0d0 0x7fc274148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:44.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.501+0000 7fc2794c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2740ab0d0 0x7fc274148800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53080/0 (socket says 172.21.15.40:53080) 2023-10-18T08:57:44.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.501+0000 7fc2794c0640 1 -- 172.21.15.40:0/994336448 learned_addr learned my addr 172.21.15.40:0/994336448 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:44.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.501+0000 7fc27a4c2640 1 --2- 172.21.15.40:0/994336448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc274152560 0x7fc2740a9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:44.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.501+0000 7fc279cc1640 1 --2- 172.21.15.40:0/994336448 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc274071440 0x7fc274153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:44.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.501+0000 7fc2794c0640 1 -- 172.21.15.40:0/994336448 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc274071440 msgr2=0x7fc274153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:44.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.501+0000 7fc2794c0640 1 --2- 172.21.15.40:0/994336448 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc274071440 0x7fc274153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:44.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.501+0000 7fc2794c0640 1 -- 172.21.15.40:0/994336448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc274152560 msgr2=0x7fc2740a9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.501+0000 7fc2794c0640 1 --2- 172.21.15.40:0/994336448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc274152560 0x7fc2740a9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.501+0000 7fc2794c0640 1 -- 172.21.15.40:0/994336448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc270009900 con 0x7fc2740ab0d0 2023-10-18T08:57:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.502+0000 7fc2794c0640 1 --2- 172.21.15.40:0/994336448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2740ab0d0 0x7fc274148800 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc264005940 tx=0x7fc26400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.502+0000 7fc26a7fc640 1 -- 172.21.15.40:0/994336448 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264007bb0 con 0x7fc2740ab0d0 2023-10-18T08:57:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.502+0000 7fc26a7fc640 1 -- 172.21.15.40:0/994336448 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc264007e00 con 0x7fc2740ab0d0 2023-10-18T08:57:44.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.502+0000 7fc26a7fc640 1 -- 172.21.15.40:0/994336448 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26400f6b0 con 0x7fc2740ab0d0 2023-10-18T08:57:44.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.502+0000 7fc27acc3640 1 -- 172.21.15.40:0/994336448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27414efe0 con 0x7fc2740ab0d0 2023-10-18T08:57:44.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.502+0000 7fc27acc3640 1 -- 172.21.15.40:0/994336448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc274148d40 con 0x7fc2740ab0d0 2023-10-18T08:57:44.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.503+0000 7fc27acc3640 1 -- 172.21.15.40:0/994336448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc274073250 con 0x7fc2740ab0d0 2023-10-18T08:57:44.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.504+0000 7fc26a7fc640 1 -- 172.21.15.40:0/994336448 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc26400f810 con 0x7fc2740ab0d0 2023-10-18T08:57:44.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.505+0000 7fc26a7fc640 1 --2- 172.21.15.40:0/994336448 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc25806ae40 0x7fc25806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:44.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.505+0000 7fc26a7fc640 1 -- 172.21.15.40:0/994336448 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc264089ee0 con 0x7fc2740ab0d0 2023-10-18T08:57:44.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.505+0000 7fc26a7fc640 1 -- 172.21.15.40:0/994336448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=51}) v3 -- 0x7fc258072970 con 0x7fc2740ab0d0 2023-10-18T08:57:44.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.506+0000 7fc279cc1640 1 --2- 172.21.15.40:0/994336448 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc25806ae40 0x7fc25806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:44.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.507+0000 7fc279cc1640 1 --2- 172.21.15.40:0/994336448 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc25806ae40 0x7fc25806d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc270036000 tx=0x7fc270004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:44.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.510+0000 7fc26a7fc640 1 -- 172.21.15.40:0/994336448 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc26401d060 con 0x7fc2740ab0d0 2023-10-18T08:57:44.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.717+0000 7fc27acc3640 1 -- 172.21.15.40:0/994336448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc274066e00 con 0x7fc2740ab0d0 2023-10-18T08:57:44.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.720+0000 7fc26a7fc640 1 -- 172.21.15.40:0/994336448 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) v1 ==== 81+0+34103 (secure 0 0 0) 0x7fc264054ef0 con 0x7fc2740ab0d0 2023-10-18T08:57:44.723 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:44.724 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T08:57:44.724 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 50, 2023-10-18T08:57:44.724 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T08:57:44.724 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T08:57:44.724 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T08:57:42.047948+0000", 2023-10-18T08:57:44.724 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T08:57:44.725 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T08:57:44.725 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T08:57:44.725 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T08:57:44.725 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T08:57:44.725 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T08:57:44.726 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T08:57:44.726 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T08:57:44.726 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T08:57:44.726 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:44.727 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T08:57:44.727 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T08:57:44.727 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T08:57:44.727 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T08:57:44.727 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T08:57:44.728 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 7, 2023-10-18T08:57:44.728 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T08:57:44.728 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T08:57:44.728 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T08:57:44.728 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T08:57:44.728 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T08:57:44.729 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T08:57:44.729 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.729 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T08:57:44.729 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T08:57:44.729 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T08:57:44.730 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T08:57:44.730 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T08:57:44.730 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:44.730 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:44.730 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:44.730 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:44.731 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:44.731 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:44.731 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:44.731 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:44.731 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:44.732 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:44.732 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T08:57:44.732 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T08:57:44.732 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T08:57:44.732 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T08:57:44.733 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T08:57:44.733 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:44.733 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:44.733 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:44.733 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:44.733 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:44.734 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:44.734 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:44.734 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.734 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "47", 2023-10-18T08:57:44.734 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:44.735 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:44.735 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:44.735 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:44.735 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:44.735 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:44.736 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:44.736 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:44.736 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:44.736 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:44.736 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:44.737 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:44.737 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:44.737 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:44.737 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:44.737 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:44.737 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:44.738 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:44.738 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:44.738 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:44.738 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:44.738 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:44.739 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:44.739 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:44.739 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:44.739 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:44.739 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.739 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:44.740 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:44.740 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:44.740 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:44.740 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:44.740 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:44.741 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:44.741 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:44.741 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:44.741 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:44.741 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:44.741 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:44.742 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T08:57:44.742 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T08:57:44.742 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.742 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:44.742 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T08:57:44.743 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.743 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:44.743 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T08:57:44.743 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T08:57:44.743 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T08:57:44.744 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T08:57:44.744 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T08:57:44.744 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:44.744 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:44.744 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:44.744 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.745 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.745 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.745 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 6, 2023-10-18T08:57:44.745 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T08:57:44.745 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:57:12.765969+0000", 2023-10-18T08:57:44.746 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T08:57:44.746 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T08:57:44.746 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:44.746 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:44.746 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:44.747 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:44.747 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:44.747 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:44.747 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:44.747 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:44.747 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:44.748 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:44.748 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T08:57:44.748 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T08:57:44.748 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T08:57:44.748 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T08:57:44.749 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T08:57:44.749 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:44.749 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:44.749 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:44.749 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:44.750 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:44.750 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:44.750 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:44.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.750 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "47", 2023-10-18T08:57:44.751 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:44.751 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:44.751 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:44.751 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:44.751 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:44.751 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:44.752 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:44.752 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:44.752 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:44.752 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:44.752 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:44.753 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:44.753 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:44.753 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:44.753 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:44.753 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:44.753 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:44.754 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:44.754 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:44.754 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:44.754 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:44.754 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:44.755 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:44.755 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:44.755 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:44.755 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:44.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.756 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:44.756 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:44.756 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:44.756 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:44.756 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:44.756 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:44.757 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:44.757 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:44.757 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:44.757 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:44.757 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:44.758 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:44.758 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T08:57:44.758 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T08:57:44.758 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T08:57:44.758 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.759 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:44.759 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T08:57:44.759 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T08:57:44.759 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.759 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.759 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:44.760 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T08:57:44.760 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T08:57:44.760 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T08:57:44.760 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T08:57:44.760 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T08:57:44.761 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:44.761 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:44.761 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:44.761 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.761 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.761 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.762 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 7, 2023-10-18T08:57:44.762 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T08:57:44.762 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:57:13.851110+0000", 2023-10-18T08:57:44.762 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T08:57:44.762 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T08:57:44.763 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:44.763 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:44.763 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:44.763 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:44.763 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:44.764 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:44.764 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:44.764 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:44.764 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:44.764 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:44.764 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T08:57:44.765 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T08:57:44.765 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T08:57:44.765 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T08:57:44.765 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T08:57:44.765 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:44.765 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:44.766 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:44.766 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:44.766 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:44.766 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:44.766 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:44.767 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.767 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "47", 2023-10-18T08:57:44.767 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:44.767 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:44.767 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:44.767 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:44.768 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:44.768 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:44.768 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:44.768 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:44.768 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:44.769 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:44.769 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:44.769 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:44.769 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:44.769 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:44.769 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:44.770 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:44.770 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:44.770 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:44.770 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:44.770 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:44.771 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:44.771 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:44.771 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:44.771 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:44.771 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:44.772 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:44.772 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.772 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:44.772 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:44.772 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:44.772 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:44.773 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:44.773 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:44.773 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:44.773 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:44.773 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:44.774 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:44.774 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:44.774 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:44.774 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T08:57:44.774 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T08:57:44.775 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.775 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:44.775 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T08:57:44.775 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T08:57:44.775 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.775 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.776 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:44.776 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T08:57:44.776 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T08:57:44.776 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T08:57:44.776 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T08:57:44.777 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T08:57:44.777 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:44.777 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:44.777 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:44.777 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.778 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.778 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:44.778 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T08:57:44.778 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.778 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T08:57:44.779 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:44.779 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:44.779 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:44.779 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:44.779 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:44.780 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:44.780 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:44.780 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:44.780 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:44.780 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:44.781 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:44.781 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:44.781 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.781 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.781 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.782 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T08:57:44.782 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:44.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.782 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.782 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.783 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T08:57:44.783 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:44.783 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.783 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.783 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.784 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:44.784 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.784 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.784 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.784 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T08:57:44.785 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:44.785 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.785 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.785 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.785 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T08:57:44.786 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:44.786 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.786 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.786 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:44.787 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.787 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.787 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T08:57:44.787 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:44.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.788 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.788 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T08:57:44.788 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:44.788 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.789 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.789 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:44.789 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.790 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.790 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T08:57:44.790 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:44.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.790 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.790 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.791 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T08:57:44.791 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:44.791 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.791 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.792 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T08:57:44.792 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T08:57:44.792 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T08:57:44.792 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T08:57:44.792 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:44.793 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:44.793 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T08:57:44.793 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:44.793 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.794 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T08:57:44.794 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:44.794 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:44.794 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:44.794 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:44.795 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:44.795 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:44.795 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:44.795 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:44.795 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:44.795 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:44.796 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:44.796 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:44.796 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.796 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.796 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.797 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T08:57:44.797 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:44.797 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.797 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.797 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.798 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T08:57:44.798 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:44.798 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.798 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.798 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.799 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:44.799 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.799 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.799 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.799 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T08:57:44.799 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:44.800 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.800 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.800 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.800 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T08:57:44.800 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:44.801 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.801 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.801 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.801 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:44.801 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.802 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.802 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.802 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T08:57:44.802 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:44.802 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.803 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.803 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.803 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T08:57:44.803 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:44.803 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.803 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.804 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.804 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:44.804 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.804 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.804 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.805 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T08:57:44.805 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:44.805 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.805 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.805 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.806 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T08:57:44.806 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:44.806 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.806 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.806 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.806 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T08:57:44.807 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T08:57:44.807 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T08:57:44.807 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T08:57:44.807 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:44.807 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:44.808 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:44.808 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.808 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.808 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T08:57:44.809 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:44.809 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:44.809 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:44.809 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:44.809 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:44.809 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:44.810 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:44.810 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:44.810 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:44.810 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:44.810 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:44.811 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:44.811 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.811 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.811 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.812 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T08:57:44.812 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:44.812 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.812 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.812 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.812 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T08:57:44.813 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:44.813 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.813 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.813 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:44.814 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.814 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.814 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.814 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T08:57:44.814 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:44.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.815 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.815 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.815 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T08:57:44.815 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:44.815 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.816 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.816 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.816 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:44.816 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.816 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.817 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.817 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T08:57:44.817 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:44.817 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.817 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.818 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.818 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T08:57:44.818 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:44.818 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.818 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.819 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:44.819 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.819 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.819 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.819 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T08:57:44.820 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:44.820 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.820 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.820 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.820 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T08:57:44.820 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:44.821 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.821 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.821 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.821 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T08:57:44.821 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T08:57:44.822 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T08:57:44.822 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T08:57:44.822 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:44.822 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:44.822 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T08:57:44.823 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:44.823 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.823 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.823 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.823 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T08:57:44.823 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:44.824 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:44.824 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:44.824 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:44.824 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:44.824 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:44.825 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:44.825 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:44.825 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:44.825 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:44.825 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:44.825 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:44.825 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.826 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.826 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.826 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T08:57:44.826 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:44.826 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.827 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.827 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.827 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T08:57:44.827 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:44.827 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.827 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.828 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.828 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:44.828 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.828 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.828 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.829 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T08:57:44.829 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:44.829 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.829 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.829 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.829 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T08:57:44.830 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:44.830 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.830 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.830 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.830 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:44.831 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.831 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.831 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.831 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T08:57:44.832 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:44.832 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.832 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.832 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.832 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T08:57:44.832 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:44.833 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.833 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.833 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.833 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:44.834 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.834 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.834 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.834 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T08:57:44.834 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:44.835 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.835 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.835 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.835 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T08:57:44.835 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:44.836 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.836 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.836 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.836 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T08:57:44.836 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T08:57:44.837 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T08:57:44.837 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T08:57:44.837 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:44.837 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull", 2023-10-18T08:57:44.837 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:44.837 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:44.838 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.838 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.838 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.838 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T08:57:44.838 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:44.839 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:44.839 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:44.839 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:44.839 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:44.839 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:44.839 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:44.840 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:44.840 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:44.840 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:44.840 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:44.840 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:44.841 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.841 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.841 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.841 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T08:57:44.841 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:44.842 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.842 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.842 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T08:57:44.842 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:44.843 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.843 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.843 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.843 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:44.843 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.844 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.844 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T08:57:44.844 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:44.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.844 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.845 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.845 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T08:57:44.845 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:44.845 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.845 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.846 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:44.846 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.846 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.846 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T08:57:44.847 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:44.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.847 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.847 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.847 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T08:57:44.848 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:44.848 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.848 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.848 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:44.848 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.849 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.849 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.849 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T08:57:44.849 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:44.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.850 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.850 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T08:57:44.850 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:44.850 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.850 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.851 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T08:57:44.851 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T08:57:44.851 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T08:57:44.851 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T08:57:44.852 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:44.852 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:44.852 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:44.852 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.852 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.852 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.853 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T08:57:44.853 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:44.853 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:44.853 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:44.854 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:44.854 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:44.854 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:44.854 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:44.854 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:44.854 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:44.855 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:44.855 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:44.855 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:44.855 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.855 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.856 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.856 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T08:57:44.856 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:44.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.857 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.857 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T08:57:44.857 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:44.857 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.857 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.858 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:44.858 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.858 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.858 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.858 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T08:57:44.859 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:44.859 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.859 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.859 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.859 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T08:57:44.860 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:44.860 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.860 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.860 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.860 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:44.861 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.861 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.861 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.861 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T08:57:44.861 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:44.862 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.862 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.862 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.862 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T08:57:44.862 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:44.863 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.863 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.863 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:44.863 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.864 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.864 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T08:57:44.864 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:44.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.865 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.865 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T08:57:44.865 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:44.865 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.865 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.866 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T08:57:44.866 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T08:57:44.866 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T08:57:44.866 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T08:57:44.867 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:44.867 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:44.867 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:44.867 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.867 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.867 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.868 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T08:57:44.868 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:44.868 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:44.868 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:44.869 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:44.869 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:44.869 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:44.869 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:44.869 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:44.870 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:44.870 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:44.870 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:44.870 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:44.870 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.871 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.871 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.871 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T08:57:44.871 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:44.871 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.872 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.872 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.872 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T08:57:44.872 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:44.872 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.872 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.873 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.873 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:44.873 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.873 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.873 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.874 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T08:57:44.874 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:44.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.874 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.874 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.874 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T08:57:44.875 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:44.875 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.875 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.875 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.875 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:44.876 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.876 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.876 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.876 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T08:57:44.876 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:44.876 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.877 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.877 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.877 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T08:57:44.877 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:44.877 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.878 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.878 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.878 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:44.878 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.878 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.881 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.881 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T08:57:44.881 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:44.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.882 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.882 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T08:57:44.882 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:44.882 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.883 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.883 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T08:57:44.883 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T08:57:44.883 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T08:57:44.884 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T08:57:44.884 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:44.884 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:44.884 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T08:57:44.884 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:44.884 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.885 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.885 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.885 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T08:57:44.885 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:44.885 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:44.886 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:44.886 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:44.886 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:44.886 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:44.886 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:44.887 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:44.887 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:44.887 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:44.887 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:44.887 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:44.888 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.888 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.888 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.888 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T08:57:44.888 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:44.888 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.889 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.889 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.889 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T08:57:44.889 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:44.889 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.890 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.890 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.890 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:44.890 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.890 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.891 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.891 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T08:57:44.891 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:44.891 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.891 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.892 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.892 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T08:57:44.892 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:44.892 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.892 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.893 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.893 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:44.893 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.893 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.893 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.894 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T08:57:44.894 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:44.894 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.894 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.894 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.894 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T08:57:44.895 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:44.895 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.895 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.895 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.895 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:44.896 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:44.896 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.896 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:44.896 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T08:57:44.896 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:44.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.897 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.897 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:44.897 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T08:57:44.897 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:44.897 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.898 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.898 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.898 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T08:57:44.898 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T08:57:44.898 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T08:57:44.899 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T08:57:44.899 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:44.899 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:44.899 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T08:57:44.899 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:44.900 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:44.900 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.900 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:44.900 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T08:57:44.900 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.900 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T08:57:44.901 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:44.901 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:44.901 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:44.901 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:44.901 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:44.902 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T08:57:44.902 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:44.902 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.902 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.902 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T08:57:44.903 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:44.903 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:44.903 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:44.903 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:44.903 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:44.904 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T08:57:44.904 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:44.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.904 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T08:57:44.905 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:44.905 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:44.905 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:44.905 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:44.905 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:44.905 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T08:57:44.906 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:44.906 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.906 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.906 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T08:57:44.906 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:44.907 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:44.907 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:44.907 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:44.907 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:44.907 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T08:57:44.908 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:44.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.908 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.908 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T08:57:44.908 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:44.908 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:44.909 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:44.909 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:44.909 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:44.909 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T08:57:44.909 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:44.909 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.910 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.910 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T08:57:44.910 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:44.910 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:44.910 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:44.911 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:44.911 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:44.911 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T08:57:44.911 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:44.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.912 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.912 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T08:57:44.912 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:44.912 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:44.912 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:44.913 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:44.913 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:44.913 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T08:57:44.913 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:44.913 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:44.914 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T08:57:44.914 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:44.914 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:44.914 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:44.914 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:44.915 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:44.915 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T08:57:44.915 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:44.915 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.915 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:44.916 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T08:57:44.916 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T08:57:44.916 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T08:57:44.916 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T08:57:44.916 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T08:57:44.916 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T08:57:44.917 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T08:57:44.917 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T08:57:44.917 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T08:57:44.917 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T08:57:44.917 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T08:57:44.918 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T08:57:44.918 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T08:57:44.918 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T08:57:44.918 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.919 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:44.919 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T08:57:44.919 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T08:57:44.919 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T08:57:44.919 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T08:57:44.919 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T08:57:44.920 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T08:57:44.920 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T08:57:44.920 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T08:57:44.920 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T08:57:44.920 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T08:57:44.921 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T08:57:44.921 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:44.921 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T08:57:44.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.723+0000 7fc27acc3640 1 -- 172.21.15.40:0/994336448 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc25806ae40 msgr2=0x7fc25806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:44.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.723+0000 7fc27acc3640 1 --2- 172.21.15.40:0/994336448 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc25806ae40 0x7fc25806d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc270036000 tx=0x7fc270004ab0 comp rx=0 tx=0).stop 2023-10-18T08:57:44.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.723+0000 7fc27acc3640 1 -- 172.21.15.40:0/994336448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2740ab0d0 msgr2=0x7fc274148800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:44.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.723+0000 7fc27acc3640 1 --2- 172.21.15.40:0/994336448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2740ab0d0 0x7fc274148800 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc264005940 tx=0x7fc26400cd60 comp rx=0 tx=0).stop 2023-10-18T08:57:44.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.724+0000 7fc27acc3640 1 -- 172.21.15.40:0/994336448 shutdown_connections 2023-10-18T08:57:44.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.724+0000 7fc27acc3640 1 --2- 172.21.15.40:0/994336448 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc25806ae40 0x7fc25806d300 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:44.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.724+0000 7fc27acc3640 1 --2- 172.21.15.40:0/994336448 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc274152560 0x7fc2740a9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:44.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.724+0000 7fc27acc3640 1 --2- 172.21.15.40:0/994336448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2740ab0d0 0x7fc274148800 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:44.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.724+0000 7fc27acc3640 1 --2- 172.21.15.40:0/994336448 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc274071440 0x7fc274153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:44.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.724+0000 7fc27acc3640 1 -- 172.21.15.40:0/994336448 >> 172.21.15.40:0/994336448 conn(0x7fc274069720 msgr2=0x7fc27414aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:44.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.724+0000 7fc27acc3640 1 -- 172.21.15.40:0/994336448 shutdown_connections 2023-10-18T08:57:44.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:44.724+0000 7fc27acc3640 1 -- 172.21.15.40:0/994336448 wait complete. 2023-10-18T08:57:44.925 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-18T08:57:49.927 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T08:57:50.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.170+0000 7f1debfff640 1 -- 172.21.15.40:0/120292390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dec0b01e0 msgr2=0x7f1dec0cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:50.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.170+0000 7f1debfff640 1 --2- 172.21.15.40:0/120292390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dec0b01e0 0x7f1dec0cf910 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4005fd0 tx=0x7f1dd402bc60 comp rx=0 tx=0).stop 2023-10-18T08:57:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.171+0000 7f1debfff640 1 -- 172.21.15.40:0/120292390 shutdown_connections 2023-10-18T08:57:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.171+0000 7f1debfff640 1 --2- 172.21.15.40:0/120292390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dec0b01e0 0x7f1dec0cf910 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.171+0000 7f1debfff640 1 --2- 172.21.15.40:0/120292390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dec15b640 0x7f1dec0d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.171+0000 7f1debfff640 1 -- 172.21.15.40:0/120292390 >> 172.21.15.40:0/120292390 conn(0x7f1dec08ad00 msgr2=0x7f1dec09a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.171+0000 7f1debfff640 1 -- 172.21.15.40:0/120292390 shutdown_connections 2023-10-18T08:57:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.171+0000 7f1debfff640 1 -- 172.21.15.40:0/120292390 wait complete. 2023-10-18T08:57:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.173+0000 7f1debfff640 1 Processor -- start 2023-10-18T08:57:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.173+0000 7f1debfff640 1 -- start start 2023-10-18T08:57:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.174+0000 7f1debfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dec0b01e0 0x7f1dec0b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.174+0000 7f1debfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dec0cfe50 0x7f1dec152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.174+0000 7f1debfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dec15b640 0x7f1dec0cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.174+0000 7f1debfff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec0b50e0 con 0x7f1dec15b640 2023-10-18T08:57:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.174+0000 7f1debfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec0b44d0 con 0x7f1dec0b01e0 2023-10-18T08:57:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.174+0000 7f1debfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec0b47b0 con 0x7f1dec0cfe50 2023-10-18T08:57:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.174+0000 7f1deb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dec15b640 0x7f1dec0cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.174+0000 7f1deb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dec15b640 0x7f1dec0cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60982/0 (socket says 172.21.15.40:60982) 2023-10-18T08:57:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.174+0000 7f1deb7fe640 1 -- 172.21.15.40:0/1370461925 learned_addr learned my addr 172.21.15.40:0/1370461925 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.175+0000 7f1deaffd640 1 --2- 172.21.15.40:0/1370461925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dec0b01e0 0x7f1dec0b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.175+0000 7f1dea7fc640 1 --2- 172.21.15.40:0/1370461925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dec0cfe50 0x7f1dec152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.175+0000 7f1deb7fe640 1 -- 172.21.15.40:0/1370461925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dec0cfe50 msgr2=0x7f1dec152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.175+0000 7f1deb7fe640 1 --2- 172.21.15.40:0/1370461925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dec0cfe50 0x7f1dec152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.175+0000 7f1deb7fe640 1 -- 172.21.15.40:0/1370461925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dec0b01e0 msgr2=0x7f1dec0b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.175+0000 7f1deb7fe640 1 --2- 172.21.15.40:0/1370461925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dec0b01e0 0x7f1dec0b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.175+0000 7f1deb7fe640 1 -- 172.21.15.40:0/1370461925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd4009560 con 0x7f1dec15b640 2023-10-18T08:57:50.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.175+0000 7f1deaffd640 1 --2- 172.21.15.40:0/1370461925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dec0b01e0 0x7f1dec0b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:57:50.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.175+0000 7f1deb7fe640 1 --2- 172.21.15.40:0/1370461925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dec15b640 0x7f1dec0cac10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1de400c410 tx=0x7f1de400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:50.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.175+0000 7f1de37fe640 1 -- 172.21.15.40:0/1370461925 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de4011790 con 0x7f1dec15b640 2023-10-18T08:57:50.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.175+0000 7f1de37fe640 1 -- 172.21.15.40:0/1370461925 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1de40119e0 con 0x7f1dec15b640 2023-10-18T08:57:50.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.175+0000 7f1debfff640 1 -- 172.21.15.40:0/1370461925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dec0b4a90 con 0x7f1dec15b640 2023-10-18T08:57:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.175+0000 7f1de37fe640 1 -- 172.21.15.40:0/1370461925 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de4011e30 con 0x7f1dec15b640 2023-10-18T08:57:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.176+0000 7f1debfff640 1 -- 172.21.15.40:0/1370461925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dec0cb150 con 0x7f1dec15b640 2023-10-18T08:57:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.177+0000 7f1debfff640 1 -- 172.21.15.40:0/1370461925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dec071da0 con 0x7f1dec15b640 2023-10-18T08:57:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.178+0000 7f1de37fe640 1 -- 172.21.15.40:0/1370461925 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1de401c460 con 0x7f1dec15b640 2023-10-18T08:57:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.179+0000 7f1de37fe640 1 --2- 172.21.15.40:0/1370461925 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1dcc06ae40 0x7f1dcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.179+0000 7f1de37fe640 1 -- 172.21.15.40:0/1370461925 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1de4089700 con 0x7f1dec15b640 2023-10-18T08:57:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.179+0000 7f1de37fe640 1 -- 172.21.15.40:0/1370461925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=51}) v3 -- 0x7f1dcc072970 con 0x7f1dec15b640 2023-10-18T08:57:50.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.179+0000 7f1deaffd640 1 --2- 172.21.15.40:0/1370461925 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1dcc06ae40 0x7f1dcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:50.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.181+0000 7f1deaffd640 1 --2- 172.21.15.40:0/1370461925 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1dcc06ae40 0x7f1dcc06d300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc005c40 tx=0x7f1ddc005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:50.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.185+0000 7f1de37fe640 1 -- 172.21.15.40:0/1370461925 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1de4054680 con 0x7f1dec15b640 2023-10-18T08:57:50.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.377+0000 7f1debfff640 1 -- 172.21.15.40:0/1370461925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f1dec066e00 con 0x7f1dec15b640 2023-10-18T08:57:50.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.380+0000 7f1de37fe640 1 -- 172.21.15.40:0/1370461925 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) v1 ==== 81+0+34103 (secure 0 0 0) 0x7f1de4057ca0 con 0x7f1dec15b640 2023-10-18T08:57:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:50.383 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T08:57:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 50, 2023-10-18T08:57:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T08:57:50.383 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T08:57:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T08:57:42.047948+0000", 2023-10-18T08:57:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T08:57:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T08:57:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T08:57:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T08:57:50.384 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T08:57:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T08:57:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T08:57:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T08:57:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T08:57:50.385 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T08:57:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T08:57:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T08:57:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T08:57:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T08:57:50.386 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 7, 2023-10-18T08:57:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T08:57:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T08:57:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T08:57:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T08:57:50.387 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T08:57:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T08:57:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T08:57:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T08:57:50.388 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T08:57:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T08:57:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T08:57:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:50.389 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:50.390 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T08:57:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T08:57:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T08:57:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T08:57:50.391 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T08:57:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:50.392 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "47", 2023-10-18T08:57:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:50.393 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:50.394 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:50.395 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:50.396 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:50.397 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:50.398 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:50.399 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T08:57:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T08:57:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.400 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T08:57:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T08:57:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T08:57:50.401 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T08:57:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T08:57:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T08:57:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:50.402 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 6, 2023-10-18T08:57:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T08:57:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:57:12.765969+0000", 2023-10-18T08:57:50.403 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T08:57:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T08:57:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:50.404 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:50.405 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T08:57:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T08:57:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T08:57:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T08:57:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T08:57:50.406 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:50.407 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "47", 2023-10-18T08:57:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:50.408 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:50.409 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:50.410 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:50.411 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.412 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:50.413 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:50.414 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T08:57:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T08:57:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T08:57:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:50.415 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T08:57:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T08:57:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T08:57:50.416 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T08:57:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T08:57:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T08:57:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T08:57:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:50.417 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 7, 2023-10-18T08:57:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T08:57:50.418 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:57:13.851110+0000", 2023-10-18T08:57:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T08:57:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T08:57:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:50.419 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:50.420 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T08:57:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T08:57:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T08:57:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T08:57:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T08:57:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:50.421 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:50.422 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "47", 2023-10-18T08:57:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:50.423 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:50.424 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:50.425 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:50.426 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.427 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:50.428 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:50.429 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T08:57:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T08:57:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T08:57:50.430 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T08:57:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T08:57:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T08:57:50.431 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T08:57:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T08:57:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T08:57:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:50.432 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T08:57:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.433 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T08:57:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:50.434 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:50.435 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T08:57:50.436 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T08:57:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:50.437 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.438 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T08:57:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.439 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T08:57:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.440 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T08:57:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:50.441 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T08:57:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.442 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.443 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T08:57:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T08:57:50.444 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T08:57:50.445 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T08:57:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T08:57:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T08:57:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T08:57:50.446 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T08:57:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:50.447 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:50.448 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:50.449 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T08:57:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T08:57:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T08:57:50.452 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T08:57:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:50.453 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.454 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T08:57:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T08:57:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:50.456 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T08:57:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T08:57:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T08:57:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T08:57:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T08:57:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T08:57:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:50.459 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T08:57:50.460 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:50.461 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:50.462 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T08:57:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T08:57:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.465 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T08:57:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T08:57:50.466 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.467 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T08:57:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.468 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T08:57:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.469 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T08:57:50.470 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T08:57:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:50.471 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T08:57:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T08:57:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T08:57:50.472 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T08:57:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T08:57:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.473 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T08:57:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:50.474 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:50.475 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.476 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T08:57:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.477 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T08:57:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.478 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T08:57:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:50.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T08:57:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.480 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.481 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T08:57:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T08:57:50.482 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:50.483 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T08:57:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.484 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T08:57:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.485 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T08:57:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T08:57:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T08:57:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T08:57:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:50.486 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull", 2023-10-18T08:57:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.487 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T08:57:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:50.488 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:50.489 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T08:57:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:50.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T08:57:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.491 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.492 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T08:57:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T08:57:50.493 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.494 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T08:57:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.495 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T08:57:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.496 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T08:57:50.497 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T08:57:50.498 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T08:57:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T08:57:50.499 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T08:57:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T08:57:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:50.500 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T08:57:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:50.501 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:50.502 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T08:57:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.504 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.505 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.505 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T08:57:50.505 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:50.505 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.505 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.505 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.506 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:50.506 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.506 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.506 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.506 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T08:57:50.507 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:50.507 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.507 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.507 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.507 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T08:57:50.507 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:50.508 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.508 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.508 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.508 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:50.508 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.508 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.509 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.509 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T08:57:50.509 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:50.509 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.509 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.510 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.510 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T08:57:50.510 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:50.510 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.510 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.511 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.511 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:50.511 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.511 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.511 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.511 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T08:57:50.512 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:50.512 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.512 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.512 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.512 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T08:57:50.512 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:50.513 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.513 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.513 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.513 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T08:57:50.513 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T08:57:50.513 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T08:57:50.514 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T08:57:50.514 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:50.514 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:50.514 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:50.514 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.515 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.515 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.515 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T08:57:50.515 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:50.515 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:50.515 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:50.516 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:50.516 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:50.516 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:50.516 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:50.516 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:50.516 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:50.517 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:50.517 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:50.517 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:50.517 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.517 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.517 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.518 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T08:57:50.518 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:50.518 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.518 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.518 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.518 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T08:57:50.519 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:50.519 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.519 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.519 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.519 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:50.519 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.520 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.520 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.520 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T08:57:50.520 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:50.520 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.520 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.521 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.521 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T08:57:50.521 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:50.521 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.521 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.521 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.522 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:50.522 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.522 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.522 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.522 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T08:57:50.523 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:50.523 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.523 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.523 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.523 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T08:57:50.523 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:50.523 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.524 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.524 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.524 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:50.524 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.524 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.524 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.525 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T08:57:50.525 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:50.525 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.525 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.525 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.525 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T08:57:50.526 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:50.526 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.526 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.526 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.526 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T08:57:50.527 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T08:57:50.527 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T08:57:50.527 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T08:57:50.527 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:50.527 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:50.527 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T08:57:50.528 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:50.528 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.528 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.528 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.528 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T08:57:50.528 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:50.529 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:50.529 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:50.529 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:50.529 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:50.529 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:50.529 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:50.530 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:50.530 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:50.530 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:50.530 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:50.530 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:50.531 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.531 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.531 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.531 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T08:57:50.531 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:50.531 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.531 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.532 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.532 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T08:57:50.532 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:50.532 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.532 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.532 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.533 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:50.533 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.533 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.533 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.533 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T08:57:50.533 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:50.534 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.534 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.534 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.534 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T08:57:50.534 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:50.535 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.535 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.535 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.535 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:50.535 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.535 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.535 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.536 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T08:57:50.536 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:50.536 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.536 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.536 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.536 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T08:57:50.537 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:50.537 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.537 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.537 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.537 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:50.537 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:50.538 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.538 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:50.538 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T08:57:50.538 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:50.538 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.538 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.539 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:50.539 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T08:57:50.539 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:50.539 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.539 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.539 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.540 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T08:57:50.540 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T08:57:50.540 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T08:57:50.540 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T08:57:50.540 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:50.540 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:50.541 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T08:57:50.541 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:50.541 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:50.541 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.541 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:50.541 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T08:57:50.542 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.542 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T08:57:50.542 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:50.542 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:50.542 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:50.543 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:50.543 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:50.543 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T08:57:50.543 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:50.543 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.543 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.544 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T08:57:50.544 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:50.544 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:50.544 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:50.544 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:50.544 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:50.544 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T08:57:50.545 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:50.545 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.545 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.545 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T08:57:50.545 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:50.545 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:50.546 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:50.546 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:50.546 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:50.546 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T08:57:50.546 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:50.546 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.547 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.547 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T08:57:50.547 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:50.547 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:50.547 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:50.548 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:50.548 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:50.548 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T08:57:50.548 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:50.548 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.548 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.549 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T08:57:50.549 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:50.549 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:50.549 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:50.549 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:50.549 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:50.550 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T08:57:50.550 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:50.550 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.550 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.550 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T08:57:50.551 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:50.551 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:50.551 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:50.551 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:50.551 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:50.551 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T08:57:50.552 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:50.552 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.552 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.552 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T08:57:50.552 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:50.552 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:50.553 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:50.553 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:50.553 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:50.553 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T08:57:50.553 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:50.553 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.554 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:50.554 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T08:57:50.554 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:50.554 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:50.554 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:50.554 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:50.555 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:50.555 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T08:57:50.555 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:50.555 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.555 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:50.555 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T08:57:50.556 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T08:57:50.556 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T08:57:50.556 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T08:57:50.556 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T08:57:50.556 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T08:57:50.556 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T08:57:50.557 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T08:57:50.557 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T08:57:50.557 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T08:57:50.557 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T08:57:50.557 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T08:57:50.557 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T08:57:50.558 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T08:57:50.558 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:50.558 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T08:57:50.558 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T08:57:50.558 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T08:57:50.559 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T08:57:50.559 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T08:57:50.559 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T08:57:50.559 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T08:57:50.559 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T08:57:50.559 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T08:57:50.560 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T08:57:50.560 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T08:57:50.560 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:50.560 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T08:57:50.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.382+0000 7f1debfff640 1 -- 172.21.15.40:0/1370461925 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1dcc06ae40 msgr2=0x7f1dcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:50.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.382+0000 7f1debfff640 1 --2- 172.21.15.40:0/1370461925 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1dcc06ae40 0x7f1dcc06d300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc005c40 tx=0x7f1ddc005f90 comp rx=0 tx=0).stop 2023-10-18T08:57:50.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.382+0000 7f1debfff640 1 -- 172.21.15.40:0/1370461925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dec15b640 msgr2=0x7f1dec0cac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:50.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.382+0000 7f1debfff640 1 --2- 172.21.15.40:0/1370461925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dec15b640 0x7f1dec0cac10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1de400c410 tx=0x7f1de400c8d0 comp rx=0 tx=0).stop 2023-10-18T08:57:50.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.383+0000 7f1debfff640 1 -- 172.21.15.40:0/1370461925 shutdown_connections 2023-10-18T08:57:50.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.383+0000 7f1debfff640 1 --2- 172.21.15.40:0/1370461925 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1dcc06ae40 0x7f1dcc06d300 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:50.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.383+0000 7f1debfff640 1 --2- 172.21.15.40:0/1370461925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1dec15b640 0x7f1dec0cac10 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:50.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.383+0000 7f1debfff640 1 --2- 172.21.15.40:0/1370461925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dec0cfe50 0x7f1dec152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:50.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.383+0000 7f1debfff640 1 --2- 172.21.15.40:0/1370461925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dec0b01e0 0x7f1dec0b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:50.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.383+0000 7f1debfff640 1 -- 172.21.15.40:0/1370461925 >> 172.21.15.40:0/1370461925 conn(0x7f1dec08ad00 msgr2=0x7f1dec06bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:50.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.383+0000 7f1debfff640 1 -- 172.21.15.40:0/1370461925 shutdown_connections 2023-10-18T08:57:50.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:50.383+0000 7f1debfff640 1 -- 172.21.15.40:0/1370461925 wait complete. 2023-10-18T08:57:50.564 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-18T08:57:53.024 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T08:57:53.021+0000 7f6a11f15640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2023-10-18T08:57:55.565 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T08:57:55.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.803+0000 7fe0703a9640 1 -- 172.21.15.40:0/1809362419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0540039a0 msgr2=0x7fe054003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:55.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.803+0000 7fe0703a9640 1 --2- 172.21.15.40:0/1809362419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0540039a0 0x7fe054003dc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe064006f00 tx=0x7fe0540057f0 comp rx=0 tx=0).stop 2023-10-18T08:57:55.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.803+0000 7fe06e946640 1 -- 172.21.15.40:0/1809362419 reap_dead start 2023-10-18T08:57:55.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.803+0000 7fe0703a9640 1 -- 172.21.15.40:0/1809362419 shutdown_connections 2023-10-18T08:57:55.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.803+0000 7fe0703a9640 1 -- 172.21.15.40:0/1809362419 >> 172.21.15.40:0/1809362419 conn(0x7fe06806cee0 msgr2=0x7fe068069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:55.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.803+0000 7fe0703a9640 1 -- 172.21.15.40:0/1809362419 shutdown_connections 2023-10-18T08:57:55.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.803+0000 7fe0703a9640 1 -- 172.21.15.40:0/1809362419 wait complete. 2023-10-18T08:57:55.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.805+0000 7fe0703a9640 1 Processor -- start 2023-10-18T08:57:55.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.805+0000 7fe0703a9640 1 -- start start 2023-10-18T08:57:55.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.806+0000 7fe0703a9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe06814bce0 0x7fe0680cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.806+0000 7fe0703a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0680cbc40 0x7fe06815cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.806+0000 7fe0703a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06815d470 0x7fe0681f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.806+0000 7fe0703a9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0680cc0a0 con 0x7fe06815d470 2023-10-18T08:57:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.806+0000 7fe0703a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06815dc60 con 0x7fe0680cbc40 2023-10-18T08:57:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.806+0000 7fe0703a9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06815ba30 con 0x7fe06814bce0 2023-10-18T08:57:55.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.806+0000 7fe06e946640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06815d470 0x7fe0681f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:55.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.807+0000 7fe06e946640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06815d470 0x7fe0681f5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41498/0 (socket says 172.21.15.40:41498) 2023-10-18T08:57:55.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.807+0000 7fe06e946640 1 -- 172.21.15.40:0/3861289641 learned_addr learned my addr 172.21.15.40:0/3861289641 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:57:55.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.807+0000 7fe06e145640 1 --2- 172.21.15.40:0/3861289641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe06814bce0 0x7fe0680cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:55.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.807+0000 7fe06d944640 1 --2- 172.21.15.40:0/3861289641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0680cbc40 0x7fe06815cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:55.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.807+0000 7fe06e946640 1 -- 172.21.15.40:0/3861289641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe06814bce0 msgr2=0x7fe0680cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:55.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.807+0000 7fe06e946640 1 --2- 172.21.15.40:0/3861289641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe06814bce0 0x7fe0680cb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:55.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.807+0000 7fe06e946640 1 -- 172.21.15.40:0/3861289641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0680cbc40 msgr2=0x7fe06815cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:55.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.807+0000 7fe06e946640 1 --2- 172.21.15.40:0/3861289641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0680cbc40 0x7fe06815cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:55.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.807+0000 7fe06e946640 1 -- 172.21.15.40:0/3861289641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06815b100 con 0x7fe06815d470 2023-10-18T08:57:55.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.807+0000 7fe06e946640 1 --2- 172.21.15.40:0/3861289641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06815d470 0x7fe0681f5110 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe060000f80 tx=0x7fe0600019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:55.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.807+0000 7fe05effd640 1 -- 172.21.15.40:0/3861289641 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060011870 con 0x7fe06815d470 2023-10-18T08:57:55.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.808+0000 7fe05effd640 1 -- 172.21.15.40:0/3861289641 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe060011ae0 con 0x7fe06815d470 2023-10-18T08:57:55.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.808+0000 7fe0703a9640 1 -- 172.21.15.40:0/3861289641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe06815b3e0 con 0x7fe06815d470 2023-10-18T08:57:55.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.808+0000 7fe05effd640 1 -- 172.21.15.40:0/3861289641 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0600103c0 con 0x7fe06815d470 2023-10-18T08:57:55.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.808+0000 7fe0703a9640 1 -- 172.21.15.40:0/3861289641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06815d8f0 con 0x7fe06815d470 2023-10-18T08:57:55.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.809+0000 7fe0703a9640 1 -- 172.21.15.40:0/3861289641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe068066fd0 con 0x7fe06815d470 2023-10-18T08:57:55.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.810+0000 7fe05effd640 1 -- 172.21.15.40:0/3861289641 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe060010520 con 0x7fe06815d470 2023-10-18T08:57:55.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.811+0000 7fe05effd640 1 --2- 172.21.15.40:0/3861289641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe04c06ae40 0x7fe04c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:57:55.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.811+0000 7fe05effd640 1 -- 172.21.15.40:0/3861289641 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe06008a410 con 0x7fe06815d470 2023-10-18T08:57:55.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.811+0000 7fe05effd640 1 -- 172.21.15.40:0/3861289641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=54}) v3 -- 0x7fe04c072970 con 0x7fe06815d470 2023-10-18T08:57:55.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.812+0000 7fe06e145640 1 --2- 172.21.15.40:0/3861289641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe04c06ae40 0x7fe04c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:57:55.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.813+0000 7fe06e145640 1 --2- 172.21.15.40:0/3861289641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe04c06ae40 0x7fe04c06d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe064034000 tx=0x7fe06402e070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:57:55.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:55.816+0000 7fe05effd640 1 -- 172.21.15.40:0/3861289641 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe060055430 con 0x7fe06815d470 2023-10-18T08:57:56.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:56.013+0000 7fe0703a9640 1 -- 172.21.15.40:0/3861289641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe0680aa950 con 0x7fe06815d470 2023-10-18T08:57:56.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:56.016+0000 7fe05effd640 1 -- 172.21.15.40:0/3861289641 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v53) v1 ==== 81+0+33999 (secure 0 0 0) 0x7fe060015030 con 0x7fe06815d470 2023-10-18T08:57:56.020 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:57:56.020 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T08:57:56.020 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 53, 2023-10-18T08:57:56.020 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T08:57:56.020 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T08:57:56.021 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T08:57:55.080728+0000", 2023-10-18T08:57:56.021 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T08:57:56.021 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T08:57:56.021 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T08:57:56.021 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T08:57:56.022 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T08:57:56.022 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T08:57:56.022 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T08:57:56.022 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T08:57:56.022 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T08:57:56.023 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:56.023 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T08:57:56.023 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T08:57:56.023 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T08:57:56.023 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T08:57:56.023 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T08:57:56.024 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 7, 2023-10-18T08:57:56.024 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T08:57:56.024 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T08:57:56.024 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T08:57:56.024 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T08:57:56.025 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T08:57:56.025 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T08:57:56.025 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.025 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T08:57:56.025 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T08:57:56.025 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T08:57:56.026 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T08:57:56.026 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T08:57:56.026 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:56.026 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:56.026 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:56.027 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:56.027 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:56.027 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:56.027 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:56.027 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:56.027 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:56.028 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:56.028 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T08:57:56.028 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T08:57:56.028 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T08:57:56.028 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T08:57:56.029 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T08:57:56.029 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:56.029 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:56.029 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:56.029 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:56.030 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:56.030 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:56.030 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:56.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.030 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "47", 2023-10-18T08:57:56.030 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:56.031 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:56.031 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:56.031 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:56.031 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:56.032 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:56.032 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:56.032 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:56.032 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:56.032 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:56.033 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:56.033 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:56.033 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:56.033 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:56.033 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:56.034 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:56.034 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:56.034 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:56.034 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:56.034 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:56.035 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:56.035 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:56.035 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:56.035 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:56.035 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:56.035 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:56.036 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.036 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:56.036 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:56.036 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:56.036 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:56.037 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:56.037 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:56.037 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:56.037 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:56.037 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:56.038 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:56.038 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:56.038 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:56.038 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T08:57:56.038 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T08:57:56.038 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.039 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:56.039 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T08:57:56.039 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.039 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:56.039 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T08:57:56.040 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T08:57:56.040 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T08:57:56.040 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T08:57:56.040 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T08:57:56.040 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:56.041 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:56.041 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:56.041 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.041 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.041 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.042 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 6, 2023-10-18T08:57:56.042 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T08:57:56.042 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:57:12.765969+0000", 2023-10-18T08:57:56.042 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T08:57:56.042 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T08:57:56.042 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:56.043 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:56.043 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:56.043 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:56.043 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:56.043 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:56.044 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:56.044 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:56.044 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:56.044 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:56.044 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T08:57:56.045 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T08:57:56.045 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T08:57:56.045 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T08:57:56.045 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T08:57:56.045 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:56.045 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:56.046 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:56.046 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:56.046 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:56.046 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:56.046 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:56.047 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.047 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "47", 2023-10-18T08:57:56.047 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:56.047 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:56.047 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:56.047 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:56.048 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:56.048 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:56.048 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:56.048 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:56.049 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:56.049 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:56.049 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:56.049 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:56.049 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:56.050 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:56.050 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:56.050 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:56.050 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:56.051 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:56.051 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:56.051 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:56.051 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:56.051 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:56.052 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:56.052 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:56.052 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:56.052 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:56.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.053 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:56.053 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:56.053 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:56.053 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:56.053 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:56.053 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:56.054 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:56.054 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:56.054 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:56.054 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:56.054 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:56.055 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:56.055 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T08:57:56.055 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T08:57:56.055 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T08:57:56.055 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.055 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:56.056 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T08:57:56.056 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T08:57:56.056 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.056 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.056 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:56.057 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T08:57:56.057 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T08:57:56.057 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T08:57:56.057 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T08:57:56.057 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T08:57:56.057 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:56.058 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:56.058 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:56.058 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.058 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.058 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.059 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 7, 2023-10-18T08:57:56.059 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T08:57:56.059 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:57:13.851110+0000", 2023-10-18T08:57:56.059 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T08:57:56.059 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T08:57:56.060 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:57:56.060 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:57:56.060 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:57:56.060 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:57:56.060 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:57:56.060 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:57:56.061 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:57:56.061 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:57:56.061 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:57:56.061 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:57:56.061 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T08:57:56.062 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T08:57:56.062 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T08:57:56.062 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T08:57:56.062 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T08:57:56.062 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:57:56.063 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:57:56.063 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:57:56.063 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:57:56.063 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:57:56.063 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:57:56.063 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:57:56.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.064 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "47", 2023-10-18T08:57:56.064 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:57:56.064 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:57:56.064 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:57:56.065 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:57:56.065 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:57:56.065 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:57:56.065 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:57:56.065 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:57:56.066 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:57:56.066 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:57:56.066 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:57:56.066 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:57:56.066 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:57:56.067 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:57:56.067 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:57:56.067 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:57:56.067 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:57:56.067 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:57:56.067 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:57:56.068 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:57:56.068 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:57:56.068 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:57:56.068 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:57:56.068 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:57:56.068 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:57:56.069 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:57:56.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.069 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:57:56.069 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:57:56.069 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:57:56.070 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:57:56.070 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:57:56.070 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:57:56.070 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:57:56.070 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:57:56.071 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:57:56.071 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:57:56.071 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:57:56.071 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:57:56.071 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T08:57:56.072 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T08:57:56.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.072 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:57:56.072 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T08:57:56.072 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T08:57:56.073 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.073 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:57:56.073 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T08:57:56.073 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T08:57:56.074 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T08:57:56.074 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T08:57:56.074 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T08:57:56.074 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:57:56.074 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:57:56.075 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:57:56.075 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.075 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.075 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:56.075 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T08:57:56.075 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.076 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T08:57:56.076 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:56.076 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:56.076 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:56.076 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:56.077 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:56.077 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:56.077 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:56.077 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:56.077 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:56.077 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:56.078 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:56.078 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:56.078 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.078 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.078 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.079 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T08:57:56.079 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:56.079 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.079 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.079 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.080 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T08:57:56.080 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:56.080 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.080 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.081 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:56.081 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.081 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.081 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.081 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T08:57:56.082 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:56.082 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.082 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.082 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.082 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T08:57:56.083 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:56.083 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.083 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.083 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.083 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:56.083 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.084 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.084 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.084 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T08:57:56.084 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:56.084 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.085 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.085 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.085 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T08:57:56.085 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:56.085 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.086 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.086 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.086 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:56.086 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.086 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.087 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.087 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T08:57:56.087 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:56.087 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.087 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.088 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.088 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T08:57:56.088 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:57:56.088 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.088 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.088 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.089 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T08:57:56.089 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T08:57:56.089 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T08:57:56.089 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T08:57:56.089 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:56.089 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:56.090 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T08:57:56.090 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:56.090 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.090 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.091 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.091 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T08:57:56.091 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:56.091 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:56.091 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:56.091 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:56.092 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:56.092 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:56.092 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:56.092 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:56.092 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.093 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T08:57:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.094 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T08:57:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T08:57:56.096 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T08:57:56.097 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.098 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T08:57:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.099 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T08:57:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.101 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T08:57:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.102 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T08:57:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:57:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T08:57:56.103 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T08:57:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T08:57:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T08:57:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:56.104 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T08:57:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:56.105 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:56.106 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:56.106 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:56.106 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:56.106 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:56.106 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:56.107 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:56.107 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:56.107 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:56.107 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:56.107 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:56.107 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.108 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.108 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T08:57:56.108 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:56.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.109 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.109 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T08:57:56.109 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:56.109 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.109 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.110 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:56.110 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.110 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T08:57:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.111 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T08:57:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.113 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:56.113 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.113 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.113 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.113 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T08:57:56.113 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:56.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.114 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.114 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T08:57:56.114 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:56.114 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.115 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.115 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:56.115 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.116 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.116 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T08:57:56.116 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:56.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.116 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.117 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T08:57:56.117 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:57:56.117 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.117 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.117 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T08:57:56.118 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T08:57:56.118 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T08:57:56.118 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T08:57:56.118 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:56.118 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:56.118 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:56.119 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.119 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T08:57:56.119 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:56.120 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:56.120 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:56.120 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:56.120 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:56.120 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:56.121 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:56.121 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:56.121 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:56.121 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:56.121 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:56.121 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:56.122 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.122 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.122 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T08:57:56.122 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:56.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.123 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.123 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T08:57:56.123 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:56.123 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.123 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.124 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:56.124 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.124 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.124 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T08:57:56.125 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:56.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.125 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.125 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T08:57:56.126 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:56.126 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.126 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.126 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:56.126 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.127 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.127 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.127 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T08:57:56.127 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:56.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.128 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.128 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T08:57:56.128 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:56.128 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.128 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.129 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:56.129 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.129 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.129 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T08:57:56.130 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:56.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.130 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.130 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T08:57:56.131 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:57:56.131 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.131 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.131 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T08:57:56.131 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T08:57:56.132 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T08:57:56.132 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T08:57:56.132 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:56.132 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:56.132 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T08:57:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T08:57:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.137 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.137 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T08:57:56.137 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:56.137 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.137 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T08:57:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T08:57:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T08:57:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:56.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.142 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.142 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T08:57:56.142 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:56.142 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T08:57:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.145 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T08:57:56.145 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:57:56.145 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.145 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.145 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T08:57:56.146 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T08:57:56.146 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T08:57:56.146 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T08:57:56.146 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:56.147 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:56.147 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:56.147 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.147 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T08:57:56.148 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:56.148 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:56.148 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:56.148 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:56.148 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:56.148 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:56.149 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:56.149 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:56.149 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:56.149 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:56.149 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:56.149 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:56.150 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.150 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.150 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T08:57:56.150 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:56.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.151 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.151 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T08:57:56.151 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:56.151 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.152 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.152 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:56.152 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.153 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.153 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T08:57:56.153 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:56.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.153 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.154 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T08:57:56.154 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:56.154 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.154 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.154 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.154 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:56.155 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.155 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.155 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T08:57:56.155 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:56.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.156 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.156 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.156 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T08:57:56.156 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:56.156 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.156 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.157 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:56.157 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.157 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.157 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.158 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T08:57:56.158 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:56.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.158 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.159 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T08:57:56.159 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:57:56.159 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.159 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.160 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T08:57:56.160 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T08:57:56.160 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T08:57:56.160 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T08:57:56.160 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:56.161 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:56.161 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:56.161 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.161 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T08:57:56.162 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:56.162 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:56.162 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:56.162 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:56.162 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:56.163 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:56.163 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:56.163 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:56.163 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:56.163 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:56.164 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:56.164 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:56.164 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.164 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.165 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T08:57:56.165 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:56.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.165 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.165 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.166 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T08:57:56.166 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:56.166 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.166 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.166 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.167 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:56.167 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.167 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.167 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T08:57:56.168 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:56.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.168 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.168 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.168 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T08:57:56.168 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:56.169 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.169 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.169 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:56.169 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.169 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.170 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.170 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T08:57:56.170 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:56.170 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.171 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.171 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T08:57:56.171 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:56.171 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.171 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.172 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:56.172 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.172 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.172 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T08:57:56.173 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:56.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.173 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.173 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T08:57:56.173 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:57:56.174 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.174 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.174 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T08:57:56.174 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T08:57:56.175 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T08:57:56.175 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T08:57:56.175 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:56.175 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:56.175 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:56.175 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.176 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.176 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T08:57:56.176 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:57:56.176 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:57:56.176 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:57:56.177 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:57:56.177 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:57:56.177 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:57:56.177 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:57:56.177 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:57:56.178 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:57:56.178 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:57:56.178 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:57:56.178 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:57:56.178 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.179 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.179 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.179 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T08:57:56.179 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:56.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.179 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.180 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.180 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T08:57:56.180 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:56.180 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.180 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.181 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:57:56.181 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.181 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.181 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T08:57:56.182 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:56.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.182 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.182 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.182 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T08:57:56.183 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:56.183 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.183 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.183 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:57:56.183 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.184 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.184 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T08:57:56.184 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:56.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.185 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.185 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T08:57:56.185 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:56.185 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.185 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.186 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:57:56.186 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:57:56.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.186 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:57:56.187 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T08:57:56.187 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:56.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.187 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:57:56.188 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T08:57:56.188 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:57:56.188 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.188 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.188 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T08:57:56.189 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T08:57:56.189 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T08:57:56.189 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T08:57:56.189 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:57:56.189 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:57:56.190 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:57:56.190 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:57:56.190 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.190 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:56.190 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T08:57:56.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.191 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T08:57:56.191 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:56.191 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:56.191 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:56.191 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:56.192 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:56.192 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T08:57:56.192 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:56.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.193 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T08:57:56.193 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:56.193 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:56.193 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:56.193 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:56.194 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:56.194 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T08:57:56.194 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:56.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.195 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T08:57:56.195 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:56.195 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:56.195 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:56.195 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:56.196 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:56.196 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T08:57:56.196 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:56.196 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.196 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.196 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T08:57:56.197 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:56.197 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:56.197 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:56.197 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:56.197 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:56.198 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T08:57:56.198 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:56.198 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.198 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.199 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T08:57:56.199 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:56.199 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:56.199 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:56.199 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:56.199 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:56.200 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T08:57:56.200 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:56.200 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.200 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.200 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T08:57:56.201 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:56.201 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:56.201 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:56.201 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:56.201 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:56.202 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T08:57:56.202 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:56.202 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.202 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.202 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T08:57:56.203 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:56.203 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:56.203 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:56.203 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:56.203 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:56.203 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T08:57:56.204 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:56.204 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.204 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:57:56.204 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T08:57:56.204 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:57:56.204 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:57:56.205 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:57:56.205 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:57:56.205 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:57:56.205 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T08:57:56.205 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:57:56.206 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.206 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:57:56.206 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T08:57:56.206 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T08:57:56.206 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T08:57:56.207 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T08:57:56.207 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T08:57:56.207 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T08:57:56.207 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T08:57:56.207 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T08:57:56.207 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T08:57:56.208 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T08:57:56.208 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T08:57:56.208 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T08:57:56.208 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T08:57:56.208 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T08:57:56.208 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.209 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:57:56.209 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T08:57:56.209 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T08:57:56.209 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T08:57:56.209 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T08:57:56.209 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T08:57:56.210 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T08:57:56.210 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T08:57:56.210 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T08:57:56.210 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T08:57:56.210 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T08:57:56.211 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T08:57:56.211 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:57:56.211 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T08:57:56.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:56.019+0000 7fe0703a9640 1 -- 172.21.15.40:0/3861289641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe04c06ae40 msgr2=0x7fe04c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:56.019+0000 7fe0703a9640 1 --2- 172.21.15.40:0/3861289641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe04c06ae40 0x7fe04c06d300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe064034000 tx=0x7fe06402e070 comp rx=0 tx=0).stop 2023-10-18T08:57:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:56.019+0000 7fe0703a9640 1 -- 172.21.15.40:0/3861289641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06815d470 msgr2=0x7fe0681f5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:57:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:56.019+0000 7fe0703a9640 1 --2- 172.21.15.40:0/3861289641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06815d470 0x7fe0681f5110 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe060000f80 tx=0x7fe0600019b0 comp rx=0 tx=0).stop 2023-10-18T08:57:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:56.019+0000 7fe0703a9640 1 -- 172.21.15.40:0/3861289641 shutdown_connections 2023-10-18T08:57:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:56.019+0000 7fe0703a9640 1 --2- 172.21.15.40:0/3861289641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe04c06ae40 0x7fe04c06d300 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:56.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:56.019+0000 7fe0703a9640 1 --2- 172.21.15.40:0/3861289641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06815d470 0x7fe0681f5110 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:56.019+0000 7fe0703a9640 1 --2- 172.21.15.40:0/3861289641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0680cbc40 0x7fe06815cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:56.019+0000 7fe0703a9640 1 --2- 172.21.15.40:0/3861289641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe06814bce0 0x7fe0680cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:57:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:56.019+0000 7fe0703a9640 1 -- 172.21.15.40:0/3861289641 >> 172.21.15.40:0/3861289641 conn(0x7fe06806cee0 msgr2=0x7fe0680adad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:57:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:56.020+0000 7fe0703a9640 1 -- 172.21.15.40:0/3861289641 shutdown_connections 2023-10-18T08:57:56.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:57:56.020+0000 7fe0703a9640 1 -- 172.21.15.40:0/3861289641 wait complete. 2023-10-18T08:57:56.215 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-18T08:58:01.216 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T08:58:01.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.466+0000 7f5bd48c1640 1 -- 172.21.15.40:0/2538915647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00b2270 msgr2=0x7f5bd00c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:01.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.466+0000 7f5bd48c1640 1 --2- 172.21.15.40:0/2538915647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00b2270 0x7f5bd00c9f00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009830 tx=0x7f5bc802bd60 comp rx=0 tx=0).stop 2023-10-18T08:58:01.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.466+0000 7f5bd48c1640 1 -- 172.21.15.40:0/2538915647 shutdown_connections 2023-10-18T08:58:01.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.466+0000 7f5bd48c1640 1 --2- 172.21.15.40:0/2538915647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00b2270 0x7f5bd00c9f00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:01.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.466+0000 7f5bd48c1640 1 --2- 172.21.15.40:0/2538915647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bd00b1e10 0x7f5bd00c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:01.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.466+0000 7f5bd48c1640 1 -- 172.21.15.40:0/2538915647 >> 172.21.15.40:0/2538915647 conn(0x7f5bd0069720 msgr2=0x7f5bd0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:01.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.466+0000 7f5bd48c1640 1 -- 172.21.15.40:0/2538915647 shutdown_connections 2023-10-18T08:58:01.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.466+0000 7f5bd48c1640 1 -- 172.21.15.40:0/2538915647 wait complete. 2023-10-18T08:58:01.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.468+0000 7f5bd48c1640 1 Processor -- start 2023-10-18T08:58:01.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.468+0000 7f5bd48c1640 1 -- start start 2023-10-18T08:58:01.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.469+0000 7f5bd48c1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bd00b15b0 0x7f5bd0145a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:01.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.469+0000 7f5bd48c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5bd00b1e10 0x7f5bd014ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:01.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.469+0000 7f5bd48c1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00b2270 0x7f5bd00acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:01.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.469+0000 7f5bd48c1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd0145590 con 0x7f5bd00b1e10 2023-10-18T08:58:01.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.469+0000 7f5bd48c1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd0145f60 con 0x7f5bd00b2270 2023-10-18T08:58:01.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.469+0000 7f5bd48c1640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd0152040 con 0x7f5bd00b15b0 2023-10-18T08:58:01.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.469+0000 7f5bc659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5bd00b1e10 0x7f5bd014ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:01.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.469+0000 7f5bc659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5bd00b1e10 0x7f5bd014ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41502/0 (socket says 172.21.15.40:41502) 2023-10-18T08:58:01.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.469+0000 7f5bc659c640 1 -- 172.21.15.40:0/356026380 learned_addr learned my addr 172.21.15.40:0/356026380 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:01.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.470+0000 7f5bcf59e640 1 --2- 172.21.15.40:0/356026380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00b2270 0x7f5bd00acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:01.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.470+0000 7f5bced9d640 1 --2- 172.21.15.40:0/356026380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bd00b15b0 0x7f5bd0145a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:01.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.470+0000 7f5bc659c640 1 -- 172.21.15.40:0/356026380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bd00b15b0 msgr2=0x7f5bd0145a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:01.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.470+0000 7f5bc659c640 1 --2- 172.21.15.40:0/356026380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bd00b15b0 0x7f5bd0145a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:01.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.470+0000 7f5bc659c640 1 -- 172.21.15.40:0/356026380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00b2270 msgr2=0x7f5bd00acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:01.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.470+0000 7f5bc659c640 1 --2- 172.21.15.40:0/356026380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00b2270 0x7f5bd00acae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:01.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.470+0000 7f5bc659c640 1 -- 172.21.15.40:0/356026380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb8009560 con 0x7f5bd00b1e10 2023-10-18T08:58:01.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.470+0000 7f5bcf59e640 1 --2- 172.21.15.40:0/356026380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00b2270 0x7f5bd00acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:01.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.470+0000 7f5bc659c640 1 --2- 172.21.15.40:0/356026380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5bd00b1e10 0x7f5bd014ba60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800e9c0 tx=0x7f5bb800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:01.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.470+0000 7f5bc7fff640 1 -- 172.21.15.40:0/356026380 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb800cde0 con 0x7f5bd00b1e10 2023-10-18T08:58:01.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.470+0000 7f5bc7fff640 1 -- 172.21.15.40:0/356026380 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5bb8004020 con 0x7f5bd00b1e10 2023-10-18T08:58:01.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.470+0000 7f5bd48c1640 1 -- 172.21.15.40:0/356026380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc8009490 con 0x7f5bd00b1e10 2023-10-18T08:58:01.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.471+0000 7f5bc7fff640 1 -- 172.21.15.40:0/356026380 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8004410 con 0x7f5bd00b1e10 2023-10-18T08:58:01.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.471+0000 7f5bd48c1640 1 -- 172.21.15.40:0/356026380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd0145220 con 0x7f5bd00b1e10 2023-10-18T08:58:01.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.472+0000 7f5bd48c1640 1 -- 172.21.15.40:0/356026380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd0145740 con 0x7f5bd00b1e10 2023-10-18T08:58:01.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.473+0000 7f5bc7fff640 1 -- 172.21.15.40:0/356026380 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5bb80035f0 con 0x7f5bd00b1e10 2023-10-18T08:58:01.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.474+0000 7f5bc7fff640 1 --2- 172.21.15.40:0/356026380 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5bb006ae40 0x7f5bb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:01.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.474+0000 7f5bc7fff640 1 -- 172.21.15.40:0/356026380 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bb8014070 con 0x7f5bd00b1e10 2023-10-18T08:58:01.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.474+0000 7f5bced9d640 1 --2- 172.21.15.40:0/356026380 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5bb006ae40 0x7f5bb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:01.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.475+0000 7f5bced9d640 1 --2- 172.21.15.40:0/356026380 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5bb006ae40 0x7f5bb006d300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00013b0 tx=0x7f5bc0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:01.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.478+0000 7f5bc7fff640 1 -- 172.21.15.40:0/356026380 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5bd0145740 con 0x7f5bd00b1e10 2023-10-18T08:58:01.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.678+0000 7f5bd48c1640 1 -- 172.21.15.40:0/356026380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5bd0145740 con 0x7f5bd00b1e10 2023-10-18T08:58:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.682+0000 7f5bc7fff640 1 -- 172.21.15.40:0/356026380 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v54) v1 ==== 81+0+33960 (secure 0 0 0) 0x7f5bd0145740 con 0x7f5bd00b1e10 2023-10-18T08:58:01.686 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:01.686 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T08:58:01.686 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 54, 2023-10-18T08:58:01.686 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T08:58:01.687 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T08:58:01.687 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T08:57:56.092763+0000", 2023-10-18T08:58:01.687 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T08:58:01.687 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T08:58:01.687 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T08:58:01.688 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T08:58:01.688 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T08:58:01.688 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T08:58:01.688 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T08:58:01.688 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T08:58:01.689 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T08:58:01.689 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:58:01.689 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T08:58:01.689 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T08:58:01.689 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T08:58:01.690 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T08:58:01.690 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T08:58:01.690 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 7, 2023-10-18T08:58:01.690 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T08:58:01.690 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T08:58:01.691 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T08:58:01.691 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T08:58:01.691 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T08:58:01.691 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T08:58:01.691 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.691 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T08:58:01.692 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T08:58:01.692 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T08:58:01.692 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T08:58:01.692 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T08:58:01.692 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:58:01.693 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:58:01.693 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:58:01.693 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:58:01.693 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:58:01.693 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:58:01.693 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:58:01.694 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:58:01.694 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:58:01.694 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:58:01.694 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T08:58:01.694 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T08:58:01.695 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T08:58:01.695 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T08:58:01.695 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T08:58:01.695 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:58:01.695 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:58:01.696 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:58:01.696 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:58:01.696 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:58:01.696 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:58:01.696 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:58:01.696 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.697 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "54", 2023-10-18T08:58:01.697 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:58:01.697 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:58:01.697 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:58:01.697 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:58:01.698 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:58:01.698 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:58:01.698 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:58:01.698 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:58:01.698 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:58:01.698 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:58:01.699 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:58:01.699 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:58:01.699 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:58:01.699 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:58:01.699 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:58:01.700 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:58:01.700 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:58:01.700 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:58:01.700 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:58:01.700 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:58:01.700 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:58:01.701 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:58:01.701 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:58:01.701 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:58:01.701 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:58:01.701 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:58:01.702 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.702 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:58:01.702 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:58:01.702 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:58:01.702 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:58:01.703 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:58:01.703 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:58:01.703 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:58:01.703 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:58:01.703 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:58:01.703 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:58:01.704 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:58:01.704 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:58:01.704 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T08:58:01.704 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T08:58:01.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.705 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:58:01.705 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T08:58:01.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.705 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:58:01.705 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T08:58:01.705 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T08:58:01.706 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T08:58:01.706 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T08:58:01.706 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T08:58:01.706 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:58:01.706 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:58:01.707 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:58:01.707 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.707 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.707 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.707 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 6, 2023-10-18T08:58:01.707 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T08:58:01.708 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:57:12.765969+0000", 2023-10-18T08:58:01.708 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T08:58:01.708 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T08:58:01.708 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:58:01.708 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:58:01.709 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:58:01.709 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:58:01.709 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:58:01.709 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:58:01.709 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:58:01.710 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:58:01.710 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:58:01.710 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:58:01.710 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T08:58:01.710 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T08:58:01.711 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T08:58:01.711 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T08:58:01.711 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T08:58:01.711 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:58:01.711 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:58:01.712 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:58:01.712 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:58:01.712 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:58:01.712 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:58:01.712 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:58:01.712 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.713 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "54", 2023-10-18T08:58:01.713 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:58:01.713 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:58:01.713 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:58:01.713 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:58:01.714 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:58:01.714 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:58:01.714 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:58:01.714 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:58:01.714 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:58:01.715 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:58:01.715 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:58:01.715 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:58:01.715 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:58:01.715 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:58:01.715 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:58:01.716 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:58:01.716 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:58:01.716 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:58:01.716 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:58:01.716 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:58:01.717 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:58:01.717 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:58:01.717 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:58:01.717 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:58:01.717 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:58:01.717 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:58:01.718 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.718 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:58:01.718 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:58:01.718 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:58:01.718 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:58:01.719 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:58:01.719 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:58:01.719 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:58:01.719 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:58:01.719 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:58:01.719 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:58:01.720 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:58:01.720 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:58:01.720 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T08:58:01.720 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T08:58:01.720 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T08:58:01.721 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.721 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:58:01.721 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T08:58:01.721 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T08:58:01.721 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.721 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.722 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:58:01.722 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T08:58:01.722 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T08:58:01.722 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T08:58:01.722 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T08:58:01.723 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T08:58:01.723 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:58:01.723 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:58:01.723 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:58:01.723 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.724 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.724 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.724 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 7, 2023-10-18T08:58:01.724 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T08:58:01.724 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:57:13.851110+0000", 2023-10-18T08:58:01.725 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T08:58:01.725 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T08:58:01.725 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:58:01.725 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:58:01.725 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:58:01.725 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:58:01.726 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:58:01.726 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:58:01.726 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:58:01.726 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:58:01.726 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:58:01.726 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:58:01.727 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T08:58:01.727 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T08:58:01.727 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T08:58:01.727 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T08:58:01.727 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T08:58:01.728 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:58:01.728 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:58:01.728 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:58:01.728 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:58:01.728 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:58:01.728 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:58:01.729 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:58:01.729 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.729 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "54", 2023-10-18T08:58:01.729 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:58:01.729 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:58:01.730 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:58:01.730 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:58:01.730 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:58:01.730 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:58:01.730 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:58:01.730 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:58:01.731 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:58:01.731 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:58:01.731 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:58:01.731 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:58:01.731 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:58:01.732 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:58:01.732 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:58:01.732 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:58:01.732 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:58:01.732 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:58:01.732 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:58:01.733 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:58:01.733 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:58:01.733 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:58:01.733 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:58:01.733 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:58:01.733 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:58:01.734 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:58:01.734 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.734 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:58:01.734 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:58:01.734 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:58:01.735 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:58:01.735 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:58:01.735 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:58:01.735 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:58:01.735 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:58:01.735 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:58:01.736 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:58:01.736 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:58:01.736 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:58:01.736 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T08:58:01.736 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T08:58:01.737 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.737 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:58:01.737 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T08:58:01.737 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T08:58:01.737 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.737 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.738 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:58:01.738 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T08:58:01.738 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T08:58:01.738 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T08:58:01.738 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T08:58:01.739 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T08:58:01.739 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:58:01.739 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:58:01.739 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:58:01.739 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.739 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.740 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:58:01.740 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T08:58:01.740 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.740 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T08:58:01.740 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:58:01.741 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:58:01.741 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:58:01.741 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:58:01.741 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:58:01.741 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:58:01.742 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:58:01.742 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:58:01.742 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:58:01.742 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:58:01.742 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:58:01.742 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:58:01.743 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.743 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.743 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.743 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T08:58:01.743 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:58:01.744 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.744 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.744 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.744 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T08:58:01.744 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:58:01.744 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.745 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.745 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.745 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:58:01.745 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.745 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.746 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.746 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T08:58:01.746 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:58:01.746 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.746 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.746 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.747 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T08:58:01.747 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:58:01.747 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.747 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.747 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.748 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:58:01.748 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.748 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.748 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.748 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T08:58:01.748 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:58:01.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.749 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.749 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T08:58:01.749 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:58:01.750 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.750 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.750 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:58:01.750 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.751 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.751 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T08:58:01.751 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:58:01.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.751 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.752 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.752 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T08:58:01.752 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:58:01.752 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.752 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.753 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.753 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T08:58:01.753 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T08:58:01.753 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T08:58:01.753 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T08:58:01.753 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:58:01.754 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:58:01.754 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:58:01.754 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.754 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.755 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T08:58:01.755 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:58:01.755 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:58:01.755 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:58:01.755 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:58:01.756 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:58:01.756 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:58:01.756 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:58:01.756 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:58:01.756 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:58:01.756 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:58:01.757 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:58:01.757 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:58:01.757 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.757 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.757 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.758 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T08:58:01.758 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:58:01.758 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.758 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.758 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.759 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T08:58:01.759 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:58:01.759 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.759 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.759 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.759 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:58:01.760 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.760 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.760 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.760 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T08:58:01.760 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:58:01.760 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.761 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.761 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.761 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T08:58:01.761 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:58:01.761 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.761 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.762 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.762 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:58:01.762 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.762 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.762 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.763 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T08:58:01.763 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:58:01.763 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.763 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.763 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.763 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T08:58:01.764 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:58:01.764 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.764 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.764 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.764 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:58:01.765 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.765 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.765 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.765 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T08:58:01.765 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:58:01.765 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.766 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.766 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.766 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T08:58:01.766 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:58:01.766 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.767 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.767 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.767 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T08:58:01.767 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T08:58:01.767 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T08:58:01.767 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T08:58:01.768 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:58:01.768 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:58:01.768 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:58:01.768 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.768 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.768 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.769 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T08:58:01.769 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:58:01.769 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:58:01.769 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:58:01.769 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:58:01.769 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:58:01.770 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:58:01.770 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:58:01.770 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:58:01.770 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:58:01.770 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:58:01.771 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:58:01.771 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:58:01.771 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.771 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.771 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.771 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T08:58:01.772 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:58:01.772 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.772 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.772 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.772 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T08:58:01.773 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:58:01.773 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.773 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.773 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.773 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:58:01.773 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.774 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.774 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.774 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T08:58:01.774 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:58:01.774 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.775 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.775 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.775 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T08:58:01.775 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:58:01.775 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.775 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.776 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.776 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:58:01.776 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.776 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.776 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.776 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T08:58:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:58:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T08:58:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:58:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:58:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.779 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.779 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.779 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T08:58:01.779 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:58:01.779 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.779 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.780 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.780 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T08:58:01.780 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:58:01.780 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.780 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.780 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.781 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T08:58:01.781 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T08:58:01.781 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T08:58:01.781 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T08:58:01.781 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:58:01.782 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:58:01.782 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:58:01.782 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.782 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.783 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T08:58:01.783 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:58:01.783 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:58:01.783 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:58:01.783 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:58:01.784 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:58:01.784 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:58:01.784 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:58:01.784 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:58:01.784 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:58:01.785 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:58:01.785 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:58:01.785 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:58:01.785 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.785 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.786 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.786 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T08:58:01.786 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:58:01.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.786 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.787 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.787 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T08:58:01.787 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:58:01.787 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.787 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.788 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:58:01.788 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.788 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.788 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T08:58:01.788 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:58:01.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.789 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.789 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T08:58:01.789 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:58:01.789 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.790 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.790 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:58:01.790 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.790 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T08:58:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:58:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T08:58:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:58:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:58:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T08:58:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:58:01.794 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.794 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.794 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T08:58:01.794 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:58:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T08:58:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T08:58:01.796 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T08:58:01.796 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T08:58:01.796 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:58:01.796 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:58:01.796 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:58:01.796 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.797 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.797 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.797 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T08:58:01.797 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:58:01.797 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:58:01.797 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:58:01.798 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:58:01.798 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:58:01.798 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:58:01.798 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:58:01.798 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:58:01.799 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:58:01.799 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:58:01.799 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:58:01.799 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:58:01.799 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.800 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.800 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.800 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T08:58:01.800 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:58:01.800 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.800 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.801 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.801 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T08:58:01.801 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:58:01.801 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.801 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.801 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.802 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:58:01.802 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.802 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.802 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.802 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T08:58:01.803 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:58:01.803 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.803 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.803 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.803 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T08:58:01.804 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:58:01.804 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.804 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.804 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.804 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:58:01.804 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.805 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.805 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.805 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T08:58:01.805 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:58:01.805 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.806 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.806 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.806 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T08:58:01.806 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:58:01.806 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.806 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.807 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.807 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:58:01.807 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.807 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.807 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.808 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T08:58:01.808 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:58:01.808 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.808 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.808 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.808 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T08:58:01.809 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:58:01.809 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.809 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.809 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.809 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T08:58:01.809 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T08:58:01.810 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T08:58:01.810 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T08:58:01.810 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:58:01.810 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:58:01.810 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:58:01.811 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.811 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.811 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.811 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T08:58:01.811 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:58:01.812 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:58:01.812 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:58:01.812 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:58:01.812 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:58:01.812 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:58:01.812 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:58:01.813 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:58:01.813 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:58:01.813 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:58:01.813 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:58:01.813 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:58:01.814 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.814 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.814 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.814 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T08:58:01.814 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:58:01.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.815 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.815 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.815 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T08:58:01.815 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:58:01.815 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.816 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.816 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.816 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:58:01.816 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.816 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.816 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.817 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T08:58:01.817 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:58:01.817 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.817 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.817 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.817 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T08:58:01.818 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:58:01.818 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.818 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.818 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:58:01.819 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.819 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.819 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.819 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T08:58:01.819 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:58:01.819 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.820 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.820 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.820 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T08:58:01.820 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:58:01.820 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.820 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.821 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.821 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:58:01.821 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.821 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.821 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.822 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T08:58:01.822 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:58:01.822 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.822 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.822 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.822 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T08:58:01.823 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:58:01.823 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.823 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.823 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.823 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T08:58:01.824 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T08:58:01.824 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T08:58:01.824 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T08:58:01.824 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:58:01.824 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:58:01.824 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:58:01.825 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.825 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.825 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.825 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T08:58:01.825 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:58:01.826 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:58:01.826 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:58:01.826 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:58:01.826 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:58:01.826 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:58:01.827 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:58:01.827 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:58:01.827 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:58:01.827 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:58:01.827 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:58:01.827 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:58:01.828 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.828 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.828 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.828 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T08:58:01.828 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:58:01.829 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.829 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.829 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.829 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T08:58:01.829 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:58:01.829 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.830 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.830 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.830 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:58:01.830 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.830 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.831 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.831 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T08:58:01.831 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:58:01.831 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.831 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.831 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.832 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T08:58:01.832 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:58:01.832 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.832 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.832 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.832 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:58:01.833 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.833 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.833 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.833 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T08:58:01.833 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:58:01.833 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.834 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.834 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.834 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T08:58:01.834 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:58:01.834 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.835 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.835 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.835 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:58:01.835 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.835 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.835 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.836 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T08:58:01.836 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:58:01.836 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.836 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.836 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.836 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T08:58:01.837 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:58:01.837 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.837 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.837 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.837 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T08:58:01.837 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T08:58:01.838 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T08:58:01.838 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T08:58:01.838 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:58:01.838 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:58:01.838 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:58:01.839 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.839 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.839 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.839 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T08:58:01.839 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:58:01.839 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:58:01.840 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:58:01.840 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:58:01.840 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:58:01.840 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:58:01.840 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:58:01.841 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:58:01.841 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:58:01.841 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:58:01.841 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:58:01.841 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:58:01.841 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.842 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.842 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T08:58:01.842 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:58:01.842 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.843 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.843 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T08:58:01.843 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:58:01.843 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.844 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.844 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:58:01.844 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.844 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.844 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.845 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T08:58:01.845 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:58:01.845 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.845 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.845 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.845 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T08:58:01.846 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:58:01.846 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.846 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.846 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:58:01.847 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.847 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.847 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.847 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T08:58:01.847 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:58:01.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.848 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.848 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T08:58:01.848 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:58:01.848 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.849 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.849 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:58:01.849 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:01.849 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.849 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:01.850 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T08:58:01.850 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:58:01.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.850 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:01.851 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T08:58:01.851 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:58:01.851 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.851 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.851 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T08:58:01.852 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T08:58:01.852 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T08:58:01.852 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T08:58:01.852 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:58:01.852 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:58:01.853 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:58:01.853 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:01.853 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.853 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:58:01.853 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T08:58:01.853 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.854 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T08:58:01.854 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:58:01.854 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:58:01.854 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:58:01.854 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:58:01.855 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:58:01.855 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T08:58:01.855 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:58:01.855 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.855 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.855 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T08:58:01.856 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:58:01.856 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:58:01.856 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:58:01.856 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:58:01.856 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:58:01.857 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T08:58:01.857 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:58:01.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.857 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.857 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T08:58:01.857 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:58:01.858 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:58:01.858 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:58:01.858 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:58:01.858 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:58:01.858 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T08:58:01.859 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:58:01.859 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.859 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.859 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T08:58:01.859 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:58:01.859 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:58:01.860 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:58:01.860 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:58:01.860 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:58:01.860 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T08:58:01.860 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:58:01.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.861 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.861 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T08:58:01.861 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:58:01.861 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:58:01.861 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:58:01.862 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:58:01.862 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:58:01.862 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T08:58:01.862 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:58:01.862 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.863 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T08:58:01.863 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:58:01.863 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:58:01.863 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:58:01.864 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:58:01.864 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:58:01.864 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T08:58:01.864 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:58:01.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.865 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T08:58:01.865 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:58:01.865 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:58:01.865 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:58:01.865 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:58:01.866 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:58:01.866 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T08:58:01.866 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:58:01.866 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:01.866 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T08:58:01.867 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:58:01.867 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:58:01.867 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:58:01.867 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:58:01.867 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:58:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T08:58:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:58:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:58:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T08:58:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T08:58:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T08:58:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T08:58:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T08:58:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T08:58:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T08:58:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T08:58:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T08:58:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T08:58:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T08:58:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T08:58:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T08:58:01.871 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T08:58:01.871 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.871 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:01.871 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T08:58:01.871 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T08:58:01.872 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T08:58:01.872 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T08:58:01.872 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T08:58:01.872 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T08:58:01.872 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T08:58:01.872 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T08:58:01.873 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T08:58:01.873 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T08:58:01.873 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T08:58:01.873 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:01.873 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T08:58:01.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.684+0000 7f5bd48c1640 1 -- 172.21.15.40:0/356026380 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5bb006ae40 msgr2=0x7f5bb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:01.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.684+0000 7f5bd48c1640 1 --2- 172.21.15.40:0/356026380 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5bb006ae40 0x7f5bb006d300 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00013b0 tx=0x7f5bc0009040 comp rx=0 tx=0).stop 2023-10-18T08:58:01.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.684+0000 7f5bd48c1640 1 -- 172.21.15.40:0/356026380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5bd00b1e10 msgr2=0x7f5bd014ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:01.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.684+0000 7f5bd48c1640 1 --2- 172.21.15.40:0/356026380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5bd00b1e10 0x7f5bd014ba60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800e9c0 tx=0x7f5bb800ee80 comp rx=0 tx=0).stop 2023-10-18T08:58:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.684+0000 7f5bd48c1640 1 -- 172.21.15.40:0/356026380 shutdown_connections 2023-10-18T08:58:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.684+0000 7f5bd48c1640 1 --2- 172.21.15.40:0/356026380 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5bb006ae40 0x7f5bb006d300 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.684+0000 7f5bd48c1640 1 --2- 172.21.15.40:0/356026380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00b2270 0x7f5bd00acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.684+0000 7f5bd48c1640 1 --2- 172.21.15.40:0/356026380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5bd00b1e10 0x7f5bd014ba60 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.684+0000 7f5bd48c1640 1 --2- 172.21.15.40:0/356026380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bd00b15b0 0x7f5bd0145a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.684+0000 7f5bd48c1640 1 -- 172.21.15.40:0/356026380 >> 172.21.15.40:0/356026380 conn(0x7f5bd0069720 msgr2=0x7f5bd0158d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.685+0000 7f5bd48c1640 1 -- 172.21.15.40:0/356026380 shutdown_connections 2023-10-18T08:58:01.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:01.685+0000 7f5bd48c1640 1 -- 172.21.15.40:0/356026380 wait complete. 2023-10-18T08:58:01.878 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-10-18T08:58:01.878 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T08:58:02.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.113+0000 7f13b9afc640 1 -- 172.21.15.40:0/4279712581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40b15b0 msgr2=0x7f13b40c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:02.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.113+0000 7f13b9afc640 1 --2- 172.21.15.40:0/4279712581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40b15b0 0x7f13b40c99c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f13b406b050 tx=0x7f13a402bd70 comp rx=0 tx=0).stop 2023-10-18T08:58:02.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.114+0000 7f13b9afc640 1 -- 172.21.15.40:0/4279712581 shutdown_connections 2023-10-18T08:58:02.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.114+0000 7f13b9afc640 1 --2- 172.21.15.40:0/4279712581 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13b40c9f00 0x7f13b4157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:02.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.114+0000 7f13b9afc640 1 --2- 172.21.15.40:0/4279712581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40b15b0 0x7f13b40c99c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:02.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.114+0000 7f13b9afc640 1 -- 172.21.15.40:0/4279712581 >> 172.21.15.40:0/4279712581 conn(0x7f13b4069720 msgr2=0x7f13b4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:02.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.114+0000 7f13b9afc640 1 -- 172.21.15.40:0/4279712581 shutdown_connections 2023-10-18T08:58:02.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.114+0000 7f13b9afc640 1 -- 172.21.15.40:0/4279712581 wait complete. 2023-10-18T08:58:02.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.116+0000 7f13b9afc640 1 Processor -- start 2023-10-18T08:58:02.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.116+0000 7f13b9afc640 1 -- start start 2023-10-18T08:58:02.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.117+0000 7f13b9afc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13b406d6f0 0x7f13b41f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:02.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.117+0000 7f13b9afc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13b40b15b0 0x7f13b41f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:02.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.117+0000 7f13b9afc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40c9f00 0x7f13b40aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:02.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.117+0000 7f13b9afc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b40af7f0 con 0x7f13b40c9f00 2023-10-18T08:58:02.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.117+0000 7f13b9afc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b40afad0 con 0x7f13b406d6f0 2023-10-18T08:58:02.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.117+0000 7f13b9afc640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b4157d50 con 0x7f13b40b15b0 2023-10-18T08:58:02.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.118+0000 7f13b92fb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40c9f00 0x7f13b40aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:02.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.118+0000 7f13b92fb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40c9f00 0x7f13b40aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41520/0 (socket says 172.21.15.40:41520) 2023-10-18T08:58:02.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.118+0000 7f13b92fb640 1 -- 172.21.15.40:0/766616659 learned_addr learned my addr 172.21.15.40:0/766616659 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:02.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.118+0000 7f13b8afa640 1 --2- 172.21.15.40:0/766616659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13b406d6f0 0x7f13b41f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:02.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.118+0000 7f13abfff640 1 --2- 172.21.15.40:0/766616659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13b40b15b0 0x7f13b41f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:02.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.118+0000 7f13b92fb640 1 -- 172.21.15.40:0/766616659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13b40b15b0 msgr2=0x7f13b41f5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:02.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.118+0000 7f13b92fb640 1 --2- 172.21.15.40:0/766616659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13b40b15b0 0x7f13b41f5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:02.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.118+0000 7f13b92fb640 1 -- 172.21.15.40:0/766616659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13b406d6f0 msgr2=0x7f13b41f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:02.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.118+0000 7f13b92fb640 1 --2- 172.21.15.40:0/766616659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13b406d6f0 0x7f13b41f54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:02.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.118+0000 7f13b92fb640 1 -- 172.21.15.40:0/766616659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a4009560 con 0x7f13b40c9f00 2023-10-18T08:58:02.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.119+0000 7f13b92fb640 1 --2- 172.21.15.40:0/766616659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40c9f00 0x7f13b40aeac0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00c8f0 tx=0x7f13ac00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:02.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.119+0000 7f13a97fa640 1 -- 172.21.15.40:0/766616659 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac007bb0 con 0x7f13b40c9f00 2023-10-18T08:58:02.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.119+0000 7f13a97fa640 1 -- 172.21.15.40:0/766616659 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f13ac007e00 con 0x7f13b40c9f00 2023-10-18T08:58:02.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.119+0000 7f13b9afc640 1 -- 172.21.15.40:0/766616659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b4158030 con 0x7f13b40c9f00 2023-10-18T08:58:02.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.119+0000 7f13a97fa640 1 -- 172.21.15.40:0/766616659 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac005280 con 0x7f13b40c9f00 2023-10-18T08:58:02.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.119+0000 7f13b9afc640 1 -- 172.21.15.40:0/766616659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b41f30b0 con 0x7f13b40c9f00 2023-10-18T08:58:02.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.121+0000 7f13b9afc640 1 -- 172.21.15.40:0/766616659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b4158550 con 0x7f13b40c9f00 2023-10-18T08:58:02.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.121+0000 7f13a97fa640 1 -- 172.21.15.40:0/766616659 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f13ac0053e0 con 0x7f13b40c9f00 2023-10-18T08:58:02.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.123+0000 7f13a97fa640 1 --2- 172.21.15.40:0/766616659 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f13a006ae40 0x7f13a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:02.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.123+0000 7f13a97fa640 1 -- 172.21.15.40:0/766616659 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f13ac089e50 con 0x7f13b40c9f00 2023-10-18T08:58:02.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.124+0000 7f13b8afa640 1 --2- 172.21.15.40:0/766616659 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f13a006ae40 0x7f13a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:02.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.125+0000 7f13b8afa640 1 --2- 172.21.15.40:0/766616659 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f13a006ae40 0x7f13a006d300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f13b41e0ff0 tx=0x7f13b0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:02.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.127+0000 7f13a97fa640 1 -- 172.21.15.40:0/766616659 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f13b4158550 con 0x7f13b40c9f00 2023-10-18T08:58:02.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.326+0000 7f13b9afc640 1 -- 172.21.15.40:0/766616659 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f13b4158550 con 0x7f13b40c9f00 2023-10-18T08:58:02.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.328+0000 7f13a97fa640 1 -- 172.21.15.40:0/766616659 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v55) v1 ==== 81+0+34334 (secure 0 0 0) 0x7f13b4158550 con 0x7f13b40c9f00 2023-10-18T08:58:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:02.332 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T08:58:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 55, 2023-10-18T08:58:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T08:58:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T08:58:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T08:58:01.908654+0000", 2023-10-18T08:58:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T08:58:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T08:58:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T08:58:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T08:58:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T08:58:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T08:58:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T08:58:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T08:58:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T08:58:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:58:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T08:58:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T08:58:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T08:58:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T08:58:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T08:58:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 7, 2023-10-18T08:58:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T08:58:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T08:58:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "luminous", 2023-10-18T08:58:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T08:58:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T08:58:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T08:58:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T08:58:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T08:58:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T08:58:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T08:58:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T08:58:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:58:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:58:02.339 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:58:02.339 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:58:02.339 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:58:02.339 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:58:02.339 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:58:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:58:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:58:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:58:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T08:58:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T08:58:02.341 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T08:58:02.341 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T08:58:02.341 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T08:58:02.341 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:58:02.341 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:58:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:58:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:58:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:58:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:58:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:58:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.343 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "54", 2023-10-18T08:58:02.343 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:58:02.343 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:58:02.343 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:58:02.343 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:58:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:58:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:58:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:58:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:58:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:58:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:58:02.345 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:58:02.345 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:58:02.345 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:58:02.345 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:58:02.345 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:58:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:58:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:58:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:58:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:58:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:58:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:58:02.347 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:58:02.347 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:58:02.347 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:58:02.347 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:58:02.347 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:58:02.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.348 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:58:02.348 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:58:02.348 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:58:02.348 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:58:02.348 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:58:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:58:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:58:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:58:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:58:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:58:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:58:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:58:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T08:58:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T08:58:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:58:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T08:58:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:58:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T08:58:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T08:58:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T08:58:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T08:58:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T08:58:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:58:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:58:02.353 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:58:02.353 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.353 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 6, 2023-10-18T08:58:02.353 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T08:58:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:57:12.765969+0000", 2023-10-18T08:58:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T08:58:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T08:58:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:58:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:58:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:58:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:58:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:58:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:58:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:58:02.356 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:58:02.356 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:58:02.356 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:58:02.356 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T08:58:02.356 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T08:58:02.356 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T08:58:02.357 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T08:58:02.357 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T08:58:02.357 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:58:02.357 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:58:02.357 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:58:02.357 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:58:02.358 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:58:02.358 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:58:02.358 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:58:02.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.358 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "54", 2023-10-18T08:58:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:58:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:58:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:58:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:58:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:58:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:58:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:58:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:58:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:58:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:58:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:58:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:58:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:58:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:58:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:58:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:58:02.362 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:58:02.362 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:58:02.362 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:58:02.362 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:58:02.362 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:58:02.363 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:58:02.363 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:58:02.363 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:58:02.363 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:58:02.363 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:58:02.364 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.364 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:58:02.364 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:58:02.364 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:58:02.364 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:58:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:58:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:58:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:58:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:58:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:58:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:58:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:58:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:58:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T08:58:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T08:58:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T08:58:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:58:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T08:58:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T08:58:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:58:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T08:58:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T08:58:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T08:58:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T08:58:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T08:58:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:58:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:58:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:58:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 7, 2023-10-18T08:58:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T08:58:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:57:13.851110+0000", 2023-10-18T08:58:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T08:58:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T08:58:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T08:58:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T08:58:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T08:58:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T08:58:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T08:58:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T08:58:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T08:58:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T08:58:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T08:58:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T08:58:02.373 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T08:58:02.373 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T08:58:02.373 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T08:58:02.373 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T08:58:02.373 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T08:58:02.374 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T08:58:02.374 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T08:58:02.374 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T08:58:02.374 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T08:58:02.374 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T08:58:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T08:58:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T08:58:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "54", 2023-10-18T08:58:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T08:58:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T08:58:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T08:58:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T08:58:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T08:58:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T08:58:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T08:58:02.377 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T08:58:02.377 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T08:58:02.377 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T08:58:02.377 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T08:58:02.377 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T08:58:02.378 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T08:58:02.378 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T08:58:02.378 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T08:58:02.378 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T08:58:02.378 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T08:58:02.379 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T08:58:02.379 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T08:58:02.379 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T08:58:02.379 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T08:58:02.379 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T08:58:02.379 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T08:58:02.380 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T08:58:02.380 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T08:58:02.380 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T08:58:02.380 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.380 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T08:58:02.381 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T08:58:02.381 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T08:58:02.381 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T08:58:02.381 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T08:58:02.381 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T08:58:02.381 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T08:58:02.382 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T08:58:02.382 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T08:58:02.382 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T08:58:02.382 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T08:58:02.382 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T08:58:02.383 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T08:58:02.383 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T08:58:02.383 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.383 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T08:58:02.383 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T08:58:02.383 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T08:58:02.384 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.384 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.384 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T08:58:02.384 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T08:58:02.384 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T08:58:02.385 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T08:58:02.385 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T08:58:02.385 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T08:58:02.385 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T08:58:02.385 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T08:58:02.385 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T08:58:02.386 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.386 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.386 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:58:02.386 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T08:58:02.386 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.387 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T08:58:02.387 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:58:02.387 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:58:02.387 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:58:02.387 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:58:02.388 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:58:02.388 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:58:02.388 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:58:02.388 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:58:02.388 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:58:02.389 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:58:02.389 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:58:02.389 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:58:02.389 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.389 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.390 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T08:58:02.390 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:58:02.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.390 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.390 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.391 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T08:58:02.391 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:58:02.391 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.391 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.392 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:58:02.392 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.392 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.392 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.392 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T08:58:02.392 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:58:02.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.393 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.393 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.393 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T08:58:02.393 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:58:02.394 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.394 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.394 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:58:02.394 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.395 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.395 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T08:58:02.395 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:58:02.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.396 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.396 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T08:58:02.396 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:58:02.396 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.396 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.397 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:58:02.397 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.397 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.397 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.397 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T08:58:02.398 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:58:02.398 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.398 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.398 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.398 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T08:58:02.398 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T08:58:02.399 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.399 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.399 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.399 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T08:58:02.399 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T08:58:02.400 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T08:58:02.400 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T08:58:02.400 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:58:02.400 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:58:02.400 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:58:02.401 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.401 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.401 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.401 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T08:58:02.401 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:58:02.401 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:58:02.402 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:58:02.402 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:58:02.402 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:58:02.402 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:58:02.402 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:58:02.403 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:58:02.403 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:58:02.403 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:58:02.403 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:58:02.403 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:58:02.404 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.404 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.404 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.404 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T08:58:02.404 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:58:02.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.405 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.405 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.405 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T08:58:02.405 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:58:02.405 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.406 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.406 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.406 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:58:02.406 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.406 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.407 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.407 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T08:58:02.407 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:58:02.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.407 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.407 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.408 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T08:58:02.408 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:58:02.408 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.408 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.408 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.409 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:58:02.409 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.409 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.409 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.409 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T08:58:02.409 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:58:02.410 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.410 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.410 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.410 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T08:58:02.410 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:58:02.411 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.411 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.411 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:58:02.411 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.411 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.412 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.412 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T08:58:02.412 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:58:02.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.412 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.413 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.413 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T08:58:02.413 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T08:58:02.413 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.413 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.413 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.414 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T08:58:02.414 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T08:58:02.414 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T08:58:02.414 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T08:58:02.414 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:58:02.414 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:58:02.415 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:58:02.415 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.415 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.415 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T08:58:02.416 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:58:02.416 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:58:02.416 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:58:02.416 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:58:02.416 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:58:02.417 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:58:02.417 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:58:02.417 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:58:02.417 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:58:02.417 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:58:02.417 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:58:02.418 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:58:02.418 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.418 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.418 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.418 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T08:58:02.419 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:58:02.419 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.419 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.419 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.419 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T08:58:02.420 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:58:02.420 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.420 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.420 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.420 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:58:02.421 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.421 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.421 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.421 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T08:58:02.421 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:58:02.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.422 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.422 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.422 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T08:58:02.422 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:58:02.422 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.423 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.423 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.423 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:58:02.423 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.423 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.423 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.424 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T08:58:02.424 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:58:02.424 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.424 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.424 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.425 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T08:58:02.425 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:58:02.425 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.425 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.425 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.425 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:58:02.426 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.426 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.426 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.426 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T08:58:02.427 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:58:02.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.427 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.427 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.428 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T08:58:02.428 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T08:58:02.428 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.428 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.428 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.429 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T08:58:02.429 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T08:58:02.429 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T08:58:02.429 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T08:58:02.430 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:58:02.430 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:58:02.430 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:58:02.430 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.430 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.431 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.431 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T08:58:02.431 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:58:02.431 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:58:02.431 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:58:02.432 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:58:02.432 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:58:02.432 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:58:02.432 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:58:02.433 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:58:02.433 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:58:02.433 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:58:02.433 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:58:02.433 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:58:02.434 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.434 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.434 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.434 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T08:58:02.434 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:58:02.435 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.435 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.435 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.435 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T08:58:02.435 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:58:02.436 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.436 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.436 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.436 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:58:02.436 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.437 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.437 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.437 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T08:58:02.437 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:58:02.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.438 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.438 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.438 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T08:58:02.438 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:58:02.438 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.439 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.439 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.439 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:58:02.439 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.439 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.440 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.440 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T08:58:02.440 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:58:02.440 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.440 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.441 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.441 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T08:58:02.441 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:58:02.441 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.441 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.442 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:58:02.442 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.442 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.443 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.443 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T08:58:02.443 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:58:02.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.443 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.444 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.444 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T08:58:02.444 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T08:58:02.444 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.444 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.445 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T08:58:02.445 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T08:58:02.445 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T08:58:02.445 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T08:58:02.446 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:58:02.446 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:58:02.446 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:58:02.446 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.447 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T08:58:02.447 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:58:02.447 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:58:02.448 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:58:02.448 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:58:02.448 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:58:02.448 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:58:02.448 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:58:02.448 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:58:02.449 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:58:02.449 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:58:02.449 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:58:02.449 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:58:02.449 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.450 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.450 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T08:58:02.450 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:58:02.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.451 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.451 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T08:58:02.451 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:58:02.451 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.452 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.452 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:58:02.452 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.453 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.453 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T08:58:02.453 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:58:02.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.454 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.454 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T08:58:02.454 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:58:02.454 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.454 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.455 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:58:02.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.455 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T08:58:02.456 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:58:02.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.456 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.456 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T08:58:02.457 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:58:02.457 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.457 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.457 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:58:02.458 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.458 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.458 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T08:58:02.458 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:58:02.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.459 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.459 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T08:58:02.459 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T08:58:02.459 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.460 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.460 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T08:58:02.460 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T08:58:02.460 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T08:58:02.461 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T08:58:02.461 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:58:02.461 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:58:02.461 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:58:02.461 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.462 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T08:58:02.462 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:58:02.462 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:58:02.463 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:58:02.463 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:58:02.463 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:58:02.463 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:58:02.463 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:58:02.465 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:58:02.465 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:58:02.465 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:58:02.465 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:58:02.466 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:58:02.466 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.466 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.466 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T08:58:02.467 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:58:02.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.467 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.467 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.467 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T08:58:02.468 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:58:02.468 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.468 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.468 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:58:02.469 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.469 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.469 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.469 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T08:58:02.469 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:58:02.470 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.470 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.470 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.470 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T08:58:02.470 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:58:02.471 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.471 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.471 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:58:02.471 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.472 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.472 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.472 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T08:58:02.472 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:58:02.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.472 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.473 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.473 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T08:58:02.473 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:58:02.473 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.473 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.474 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:58:02.474 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.474 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.474 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.475 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T08:58:02.475 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:58:02.475 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.475 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.475 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.476 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T08:58:02.476 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T08:58:02.476 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.476 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.477 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T08:58:02.477 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T08:58:02.477 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T08:58:02.477 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T08:58:02.477 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:58:02.478 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:58:02.478 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:58:02.478 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.478 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T08:58:02.479 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:58:02.479 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:58:02.479 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:58:02.479 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:58:02.479 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:58:02.480 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:58:02.480 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:58:02.480 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:58:02.480 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:58:02.481 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:58:02.481 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:58:02.481 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:58:02.481 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.481 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.482 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T08:58:02.482 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:58:02.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.482 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.482 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T08:58:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:58:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:58:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T08:58:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:58:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T08:58:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:58:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:58:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T08:58:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:58:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T08:58:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:58:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:58:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T08:58:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:58:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T08:58:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T08:58:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T08:58:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T08:58:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T08:58:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T08:58:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:58:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:58:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:58:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.494 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.494 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.494 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T08:58:02.494 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T08:58:02.494 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T08:58:02.495 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T08:58:02.495 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T08:58:02.495 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T08:58:02.496 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T08:58:02.496 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T08:58:02.496 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T08:58:02.496 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 44, 2023-10-18T08:58:02.496 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T08:58:02.497 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T08:58:02.497 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T08:58:02.497 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.497 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.497 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.498 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T08:58:02.498 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:58:02.498 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.498 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.498 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.499 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T08:58:02.499 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:58:02.499 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.499 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.499 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.499 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T08:58:02.500 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.500 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.500 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.500 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T08:58:02.500 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:58:02.501 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.501 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.501 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.501 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T08:58:02.501 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:58:02.501 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.502 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.502 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.502 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T08:58:02.502 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.502 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.503 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.503 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T08:58:02.503 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:58:02.503 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.504 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.504 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T08:58:02.504 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:58:02.504 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.504 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.505 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.505 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T08:58:02.505 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T08:58:02.505 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.505 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T08:58:02.506 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T08:58:02.506 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:58:02.506 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.506 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.506 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T08:58:02.507 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T08:58:02.507 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T08:58:02.507 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.507 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.507 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.508 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T08:58:02.508 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T08:58:02.508 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T08:58:02.508 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T08:58:02.508 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T08:58:02.508 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T08:58:02.509 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T08:58:02.509 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.509 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.509 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:58:02.509 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T08:58:02.510 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.510 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T08:58:02.510 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:58:02.510 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:58:02.511 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:58:02.511 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:58:02.511 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:58:02.511 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T08:58:02.511 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:58:02.512 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.512 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.512 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T08:58:02.512 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:58:02.512 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:58:02.512 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:58:02.513 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:58:02.513 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:58:02.513 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T08:58:02.513 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:58:02.513 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.514 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.514 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T08:58:02.514 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:58:02.514 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:58:02.514 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:58:02.515 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:58:02.515 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:58:02.515 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T08:58:02.515 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:58:02.515 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.516 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.516 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T08:58:02.516 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:58:02.516 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:58:02.516 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:58:02.517 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:58:02.517 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:58:02.517 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T08:58:02.517 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:58:02.517 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.518 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.518 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T08:58:02.518 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:58:02.518 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:58:02.519 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:58:02.519 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:58:02.519 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:58:02.519 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T08:58:02.519 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:58:02.520 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.520 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.520 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T08:58:02.520 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:58:02.520 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:58:02.521 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:58:02.521 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:58:02.521 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:58:02.521 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T08:58:02.521 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:58:02.522 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.522 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.522 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T08:58:02.522 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:58:02.522 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:58:02.523 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:58:02.523 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:58:02.523 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:58:02.523 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T08:58:02.523 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:58:02.524 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.524 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.524 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T08:58:02.524 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T08:58:02.524 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T08:58:02.524 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T08:58:02.525 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T08:58:02.525 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T08:58:02.525 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T08:58:02.525 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T08:58:02.525 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.526 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:58:02.526 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T08:58:02.526 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [ 2023-10-18T08:58:02.526 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.526 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "7.7", 2023-10-18T08:58:02.527 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T08:58:02.527 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.527 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 0, 2023-10-18T08:58:02.527 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 5 2023-10-18T08:58:02.527 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.528 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.528 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.528 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.528 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "7.3e", 2023-10-18T08:58:02.528 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T08:58:02.528 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:02.529 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 0, 2023-10-18T08:58:02.529 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 4 2023-10-18T08:58:02.529 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.529 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:02.529 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.530 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T08:58:02.530 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T08:58:02.530 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T08:58:02.530 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T08:58:02.530 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T08:58:02.531 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T08:58:02.531 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T08:58:02.531 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T08:58:02.531 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T08:58:02.531 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T08:58:02.532 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T08:58:02.532 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T08:58:02.532 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T08:58:02.532 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.532 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:02.533 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T08:58:02.533 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T08:58:02.533 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T08:58:02.533 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T08:58:02.533 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T08:58:02.533 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T08:58:02.534 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T08:58:02.534 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T08:58:02.534 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T08:58:02.534 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T08:58:02.534 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T08:58:02.535 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:02.535 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T08:58:02.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.331+0000 7f13b9afc640 1 -- 172.21.15.40:0/766616659 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f13a006ae40 msgr2=0x7f13a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:02.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.331+0000 7f13b9afc640 1 --2- 172.21.15.40:0/766616659 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f13a006ae40 0x7f13a006d300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f13b41e0ff0 tx=0x7f13b0009040 comp rx=0 tx=0).stop 2023-10-18T08:58:02.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.331+0000 7f13b9afc640 1 -- 172.21.15.40:0/766616659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40c9f00 msgr2=0x7f13b40aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:02.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.331+0000 7f13b9afc640 1 --2- 172.21.15.40:0/766616659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40c9f00 0x7f13b40aeac0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00c8f0 tx=0x7f13ac00cdb0 comp rx=0 tx=0).stop 2023-10-18T08:58:02.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.331+0000 7f13b9afc640 1 -- 172.21.15.40:0/766616659 shutdown_connections 2023-10-18T08:58:02.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.331+0000 7f13b9afc640 1 --2- 172.21.15.40:0/766616659 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f13a006ae40 0x7f13a006d300 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:02.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.331+0000 7f13b9afc640 1 --2- 172.21.15.40:0/766616659 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13b40c9f00 0x7f13b40aeac0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:02.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.331+0000 7f13b9afc640 1 --2- 172.21.15.40:0/766616659 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13b40b15b0 0x7f13b41f5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:02.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.331+0000 7f13b9afc640 1 --2- 172.21.15.40:0/766616659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13b406d6f0 0x7f13b41f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:02.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.331+0000 7f13b9afc640 1 -- 172.21.15.40:0/766616659 >> 172.21.15.40:0/766616659 conn(0x7f13b4069720 msgr2=0x7f13b40abf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:02.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.331+0000 7f13b9afc640 1 -- 172.21.15.40:0/766616659 shutdown_connections 2023-10-18T08:58:02.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.332+0000 7f13b9afc640 1 -- 172.21.15.40:0/766616659 wait complete. 2023-10-18T08:58:02.540 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:58:02.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.778+0000 7f7a5e955640 1 -- 172.21.15.40:0/2347630458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5815a2e0 msgr2=0x7f7a580cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:02.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.778+0000 7f7a5e955640 1 --2- 172.21.15.40:0/2347630458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5815a2e0 0x7f7a580cbc20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7a48005fd0 tx=0x7f7a4802be10 comp rx=0 tx=0).stop 2023-10-18T08:58:02.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.778+0000 7f7a5e955640 1 -- 172.21.15.40:0/2347630458 shutdown_connections 2023-10-18T08:58:02.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.778+0000 7f7a5e955640 1 --2- 172.21.15.40:0/2347630458 >> v2:172.21.15.144:3300/0 conn(0x7f7a5815a6c0 0x7f7a5806b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.778+0000 7f7a5e955640 1 --2- 172.21.15.40:0/2347630458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5815a2e0 0x7f7a580cbc20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.778+0000 7f7a5e955640 1 -- 172.21.15.40:0/2347630458 >> 172.21.15.40:0/2347630458 conn(0x7f7a5806cee0 msgr2=0x7f7a58069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.778+0000 7f7a5e955640 1 -- 172.21.15.40:0/2347630458 shutdown_connections 2023-10-18T08:58:02.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.778+0000 7f7a5e955640 1 -- 172.21.15.40:0/2347630458 wait complete. 2023-10-18T08:58:02.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.780+0000 7f7a5e955640 1 Processor -- start 2023-10-18T08:58:02.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.780+0000 7f7a5e955640 1 -- start start 2023-10-18T08:58:02.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.781+0000 7f7a5e955640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a5815a2e0 0x7f7a5814fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:02.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.781+0000 7f7a5e955640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a5815a6c0 0x7f7a58151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:02.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.781+0000 7f7a5e955640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5814d7b0 0x7f7a58158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:02.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.781+0000 7f7a5e955640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a581e4040 con 0x7f7a5815a6c0 2023-10-18T08:58:02.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.781+0000 7f7a5e955640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a581e6930 con 0x7f7a5815a2e0 2023-10-18T08:58:02.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.781+0000 7f7a5e955640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5814f210 con 0x7f7a5814d7b0 2023-10-18T08:58:02.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.781+0000 7f7a4f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a5815a6c0 0x7f7a58151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:02.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.781+0000 7f7a4f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a5815a6c0 0x7f7a58151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41522/0 (socket says 172.21.15.40:41522) 2023-10-18T08:58:02.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.781+0000 7f7a4f7fe640 1 -- 172.21.15.40:0/2888682175 learned_addr learned my addr 172.21.15.40:0/2888682175 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.782+0000 7f7a57fff640 1 --2- 172.21.15.40:0/2888682175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a5815a2e0 0x7f7a5814fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.782+0000 7f7a5cef2640 1 --2- 172.21.15.40:0/2888682175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5814d7b0 0x7f7a58158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.782+0000 7f7a4f7fe640 1 -- 172.21.15.40:0/2888682175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5814d7b0 msgr2=0x7f7a58158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.782+0000 7f7a4f7fe640 1 --2- 172.21.15.40:0/2888682175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5814d7b0 0x7f7a58158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:02.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.782+0000 7f7a4f7fe640 1 -- 172.21.15.40:0/2888682175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a5815a2e0 msgr2=0x7f7a5814fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:02.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.782+0000 7f7a4f7fe640 1 --2- 172.21.15.40:0/2888682175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a5815a2e0 0x7f7a5814fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:02.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.782+0000 7f7a4f7fe640 1 -- 172.21.15.40:0/2888682175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a50009840 con 0x7f7a5815a6c0 2023-10-18T08:58:02.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.782+0000 7f7a4f7fe640 1 --2- 172.21.15.40:0/2888682175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a5815a6c0 0x7f7a58151f90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000c550 tx=0x7f7a4000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:02.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.783+0000 7f7a557fa640 1 -- 172.21.15.40:0/2888682175 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a400035f0 con 0x7f7a5815a6c0 2023-10-18T08:58:02.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.783+0000 7f7a557fa640 1 -- 172.21.15.40:0/2888682175 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7a40003840 con 0x7f7a5815a6c0 2023-10-18T08:58:02.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.783+0000 7f7a5e955640 1 -- 172.21.15.40:0/2888682175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a48009560 con 0x7f7a5815a6c0 2023-10-18T08:58:02.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.783+0000 7f7a557fa640 1 -- 172.21.15.40:0/2888682175 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40003c50 con 0x7f7a5815a6c0 2023-10-18T08:58:02.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.783+0000 7f7a5e955640 1 -- 172.21.15.40:0/2888682175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a58152930 con 0x7f7a5815a6c0 2023-10-18T08:58:02.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.784+0000 7f7a5e955640 1 -- 172.21.15.40:0/2888682175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5806a5a0 con 0x7f7a5815a6c0 2023-10-18T08:58:02.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.785+0000 7f7a557fa640 1 -- 172.21.15.40:0/2888682175 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7a4001b3b0 con 0x7f7a5815a6c0 2023-10-18T08:58:02.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.786+0000 7f7a557fa640 1 --2- 172.21.15.40:0/2888682175 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7a3806ae70 0x7f7a3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:02.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.786+0000 7f7a557fa640 1 -- 172.21.15.40:0/2888682175 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7a40088ee0 con 0x7f7a5815a6c0 2023-10-18T08:58:02.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.786+0000 7f7a57fff640 1 --2- 172.21.15.40:0/2888682175 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7a3806ae70 0x7f7a3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:02.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.788+0000 7f7a57fff640 1 --2- 172.21.15.40:0/2888682175 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7a3806ae70 0x7f7a3806d330 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7a48035000 tx=0x7f7a48004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:02.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:02.790+0000 7f7a557fa640 1 -- 172.21.15.40:0/2888682175 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7a40053f60 con 0x7f7a5815a6c0 2023-10-18T08:58:03.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:03.027+0000 7f7a5e955640 1 -- 172.21.15.40:0/2888682175 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a580cb520 con 0x7f7a5815a6c0 2023-10-18T08:58:03.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:03.029+0000 7f7a557fa640 1 -- 172.21.15.40:0/2888682175 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f7a40017020 con 0x7f7a5815a6c0 2023-10-18T08:58:03.032 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:03.033 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3668356540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3668356540},{"type":"v1","addr":"172.21.15.144:6836","nonce":3668356540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4222679844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4222679844},{"type":"v1","addr":"172.21.15.40:6834","nonce":4222679844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2829399368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2829399368},{"type":"v1","addr":"172.21.15.40:6835","nonce":2829399368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:57:14.826066+0000","modified":"2023-10-18T08:57:17.903233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1438456692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1438456692},{"type":"v1","addr":"172.21.15.144:6837","nonce":1438456692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-18T08:58:03.033 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 24 2023-10-18T08:58:03.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:03.031+0000 7f7a5e955640 1 -- 172.21.15.40:0/2888682175 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7a3806ae70 msgr2=0x7f7a3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:03.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:03.031+0000 7f7a5e955640 1 --2- 172.21.15.40:0/2888682175 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7a3806ae70 0x7f7a3806d330 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7a48035000 tx=0x7f7a48004730 comp rx=0 tx=0).stop 2023-10-18T08:58:03.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:03.031+0000 7f7a5e955640 1 -- 172.21.15.40:0/2888682175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a5815a6c0 msgr2=0x7f7a58151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:03.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:03.031+0000 7f7a5e955640 1 --2- 172.21.15.40:0/2888682175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a5815a6c0 0x7f7a58151f90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000c550 tx=0x7f7a4000ca10 comp rx=0 tx=0).stop 2023-10-18T08:58:03.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:03.032+0000 7f7a5e955640 1 -- 172.21.15.40:0/2888682175 shutdown_connections 2023-10-18T08:58:03.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:03.032+0000 7f7a5e955640 1 --2- 172.21.15.40:0/2888682175 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7a3806ae70 0x7f7a3806d330 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:03.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:03.032+0000 7f7a5e955640 1 --2- 172.21.15.40:0/2888682175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5814d7b0 0x7f7a58158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:03.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:03.032+0000 7f7a5e955640 1 --2- 172.21.15.40:0/2888682175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a5815a6c0 0x7f7a58151f90 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:03.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:03.032+0000 7f7a5e955640 1 --2- 172.21.15.40:0/2888682175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a5815a2e0 0x7f7a5814fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:03.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:03.032+0000 7f7a5e955640 1 -- 172.21.15.40:0/2888682175 >> 172.21.15.40:0/2888682175 conn(0x7f7a5806cee0 msgr2=0x7f7a58153250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:03.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:03.032+0000 7f7a5e955640 1 -- 172.21.15.40:0/2888682175 shutdown_connections 2023-10-18T08:58:03.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:03.032+0000 7f7a5e955640 1 -- 172.21.15.40:0/2888682175 wait complete. 2023-10-18T08:58:03.054 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-10-18T08:58:03.297 INFO:teuthology.orchestra.run.smithi144.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":5655,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":47.889344876000003,"mdsmap_epoch":24,"osdmap_epoch":56,"osdmap_epoch_barrier":50,"uptime":53.720271662000002} 2023-10-18T08:58:03.298 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 5655, "mdsmap_epoch": 24, "osdmap_epoch": 56, "osdmap_epoch_barrier": 50, "rank_uptime": 47.889344876, "state": "up:active", "uptime": 53.720271662, "want_state": "up:active", "whoami": 0 } 2023-10-18T08:58:03.299 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-18T08:58:03.300 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:58:03.300 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:58:03.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:03.362 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-18T08:58:03.803 INFO:teuthology.orchestra.run.smithi040.stderr:50+0 records in 2023-10-18T08:58:03.803 INFO:teuthology.orchestra.run.smithi040.stderr:50+0 records out 2023-10-18T08:58:03.803 INFO:teuthology.orchestra.run.smithi040.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.37977 s, 138 MB/s 2023-10-18T08:58:03.804 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T08:58:03.805 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T08:58:03.912 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T08:58:04.030 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T08:58:04.031 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:58:04.031 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:58:04.096 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:58:04.096 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:58:04.155 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:58:04.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.383+0000 7ff309cca640 1 -- 172.21.15.40:0/3931061580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3040c9f00 msgr2=0x7ff304157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:04.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.383+0000 7ff309cca640 1 --2- 172.21.15.40:0/3931061580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3040c9f00 0x7ff304157d40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff30406af60 tx=0x7ff2fc02c250 comp rx=0 tx=0).stop 2023-10-18T08:58:04.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.383+0000 7ff309cca640 1 -- 172.21.15.40:0/3931061580 shutdown_connections 2023-10-18T08:58:04.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.383+0000 7ff309cca640 1 --2- 172.21.15.40:0/3931061580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3040c9f00 0x7ff304157d40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:04.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.383+0000 7ff309cca640 1 --2- 172.21.15.40:0/3931061580 >> v2:172.21.15.144:3300/0 conn(0x7ff3040b0630 0x7ff3040c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:04.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.383+0000 7ff309cca640 1 --2- 172.21.15.40:0/3931061580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff30406d6f0 0x7ff3040b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:04.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.383+0000 7ff309cca640 1 -- 172.21.15.40:0/3931061580 >> 172.21.15.40:0/3931061580 conn(0x7ff304069720 msgr2=0x7ff304069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:04.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.384+0000 7ff309cca640 1 -- 172.21.15.40:0/3931061580 shutdown_connections 2023-10-18T08:58:04.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.384+0000 7ff309cca640 1 -- 172.21.15.40:0/3931061580 wait complete. 2023-10-18T08:58:04.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.385+0000 7ff309cca640 1 Processor -- start 2023-10-18T08:58:04.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.386+0000 7ff309cca640 1 -- start start 2023-10-18T08:58:04.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.386+0000 7ff309cca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff30406d6f0 0x7ff304159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:04.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.386+0000 7ff309cca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3040b0630 0x7ff30414b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:04.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.386+0000 7ff309cca640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3040c9f00 0x7ff30414bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:04.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.387+0000 7ff309cca640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30414cf30 con 0x7ff30406d6f0 2023-10-18T08:58:04.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.387+0000 7ff309cca640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30414c3b0 con 0x7ff3040b0630 2023-10-18T08:58:04.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.387+0000 7ff309cca640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30414c690 con 0x7ff3040c9f00 2023-10-18T08:58:04.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.387+0000 7ff308cc8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff30406d6f0 0x7ff304159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:04.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.387+0000 7ff308cc8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff30406d6f0 0x7ff304159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41546/0 (socket says 172.21.15.40:41546) 2023-10-18T08:58:04.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.387+0000 7ff308cc8640 1 -- 172.21.15.40:0/3606857299 learned_addr learned my addr 172.21.15.40:0/3606857299 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:04.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.387+0000 7ff303fff640 1 --2- 172.21.15.40:0/3606857299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3040b0630 0x7ff30414b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:04.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.387+0000 7ff3094c9640 1 --2- 172.21.15.40:0/3606857299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3040c9f00 0x7ff30414bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:04.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.387+0000 7ff308cc8640 1 -- 172.21.15.40:0/3606857299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3040c9f00 msgr2=0x7ff30414bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:04.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.387+0000 7ff308cc8640 1 --2- 172.21.15.40:0/3606857299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3040c9f00 0x7ff30414bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:04.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.387+0000 7ff308cc8640 1 -- 172.21.15.40:0/3606857299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3040b0630 msgr2=0x7ff30414b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:04.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.387+0000 7ff308cc8640 1 --2- 172.21.15.40:0/3606857299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3040b0630 0x7ff30414b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:04.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.387+0000 7ff308cc8640 1 -- 172.21.15.40:0/3606857299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc009900 con 0x7ff30406d6f0 2023-10-18T08:58:04.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.387+0000 7ff303fff640 1 --2- 172.21.15.40:0/3606857299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3040b0630 0x7ff30414b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:04.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.388+0000 7ff308cc8640 1 --2- 172.21.15.40:0/3606857299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff30406d6f0 0x7ff304159a20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800d770 tx=0x7ff2f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:04.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.388+0000 7ff3017fa640 1 -- 172.21.15.40:0/3606857299 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8009c30 con 0x7ff30406d6f0 2023-10-18T08:58:04.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.388+0000 7ff3017fa640 1 -- 172.21.15.40:0/3606857299 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff2f800de80 con 0x7ff30406d6f0 2023-10-18T08:58:04.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.388+0000 7ff3017fa640 1 -- 172.21.15.40:0/3606857299 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8005280 con 0x7ff30406d6f0 2023-10-18T08:58:04.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.388+0000 7ff309cca640 1 -- 172.21.15.40:0/3606857299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30414c970 con 0x7ff30406d6f0 2023-10-18T08:58:04.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.388+0000 7ff309cca640 1 -- 172.21.15.40:0/3606857299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff304159f60 con 0x7ff30406d6f0 2023-10-18T08:58:04.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.389+0000 7ff309cca640 1 -- 172.21.15.40:0/3606857299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff304066e00 con 0x7ff30406d6f0 2023-10-18T08:58:04.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.390+0000 7ff3017fa640 1 -- 172.21.15.40:0/3606857299 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff2f80053e0 con 0x7ff30406d6f0 2023-10-18T08:58:04.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.391+0000 7ff3017fa640 1 --2- 172.21.15.40:0/3606857299 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2e806ae40 0x7ff2e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:04.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.391+0000 7ff3017fa640 1 -- 172.21.15.40:0/3606857299 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff2f8089950 con 0x7ff30406d6f0 2023-10-18T08:58:04.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.392+0000 7ff303fff640 1 --2- 172.21.15.40:0/3606857299 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2e806ae40 0x7ff2e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:04.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.393+0000 7ff303fff640 1 --2- 172.21.15.40:0/3606857299 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2e806ae40 0x7ff2e806d300 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0005fd0 tx=0x7ff2f00094d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:04.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.395+0000 7ff3017fa640 1 -- 172.21.15.40:0/3606857299 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff2f8054920 con 0x7ff30406d6f0 2023-10-18T08:58:04.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.591+0000 7ff309cca640 1 -- 172.21.15.40:0/3606857299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3040caf90 con 0x7ff30406d6f0 2023-10-18T08:58:04.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.594+0000 7ff3017fa640 1 -- 172.21.15.40:0/3606857299 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16620 (secure 0 0 0) 0x7ff2f8017020 con 0x7ff30406d6f0 2023-10-18T08:58:04.597 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:04.598 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":56,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:58:02.917888+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:57:12.765969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T08:57:13.851110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":5}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:58:04.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.596+0000 7ff309cca640 1 -- 172.21.15.40:0/3606857299 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2e806ae40 msgr2=0x7ff2e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:04.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.596+0000 7ff309cca640 1 --2- 172.21.15.40:0/3606857299 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2e806ae40 0x7ff2e806d300 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0005fd0 tx=0x7ff2f00094d0 comp rx=0 tx=0).stop 2023-10-18T08:58:04.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.596+0000 7ff309cca640 1 -- 172.21.15.40:0/3606857299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff30406d6f0 msgr2=0x7ff304159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:04.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.596+0000 7ff309cca640 1 --2- 172.21.15.40:0/3606857299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff30406d6f0 0x7ff304159a20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800d770 tx=0x7ff2f800dc30 comp rx=0 tx=0).stop 2023-10-18T08:58:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.597+0000 7ff309cca640 1 -- 172.21.15.40:0/3606857299 shutdown_connections 2023-10-18T08:58:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.597+0000 7ff309cca640 1 --2- 172.21.15.40:0/3606857299 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2e806ae40 0x7ff2e806d300 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.597+0000 7ff309cca640 1 --2- 172.21.15.40:0/3606857299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3040c9f00 0x7ff30414bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:04.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.597+0000 7ff309cca640 1 --2- 172.21.15.40:0/3606857299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3040b0630 0x7ff30414b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.597+0000 7ff309cca640 1 --2- 172.21.15.40:0/3606857299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff30406d6f0 0x7ff304159a20 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.597+0000 7ff309cca640 1 -- 172.21.15.40:0/3606857299 >> 172.21.15.40:0/3606857299 conn(0x7ff304069720 msgr2=0x7ff304156550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.597+0000 7ff309cca640 1 -- 172.21.15.40:0/3606857299 shutdown_connections 2023-10-18T08:58:04.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.597+0000 7ff309cca640 1 -- 172.21.15.40:0/3606857299 wait complete. 2023-10-18T08:58:04.619 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:57:12.765969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:57:13.851110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:58:04.620 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:58:04.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.849+0000 7fdce158e640 1 -- 172.21.15.40:0/3088098729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc0c98a0 msgr2=0x7fdcdc06f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:04.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.849+0000 7fdce158e640 1 --2- 172.21.15.40:0/3088098729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc0c98a0 0x7fdcdc06f510 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0ca5b0 tx=0x7fdccc02d0e0 comp rx=0 tx=0).stop 2023-10-18T08:58:04.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.849+0000 7fdce158e640 1 -- 172.21.15.40:0/3088098729 shutdown_connections 2023-10-18T08:58:04.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.849+0000 7fdce158e640 1 --2- 172.21.15.40:0/3088098729 >> v2:172.21.15.144:3300/0 conn(0x7fdcdc0b1900 0x7fdcdc06fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:04.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.849+0000 7fdce158e640 1 --2- 172.21.15.40:0/3088098729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc0c98a0 0x7fdcdc06f510 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:04.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.849+0000 7fdce158e640 1 -- 172.21.15.40:0/3088098729 >> 172.21.15.40:0/3088098729 conn(0x7fdcdc069720 msgr2=0x7fdcdc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:04.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.849+0000 7fdce158e640 1 -- 172.21.15.40:0/3088098729 shutdown_connections 2023-10-18T08:58:04.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.849+0000 7fdce158e640 1 -- 172.21.15.40:0/3088098729 wait complete. 2023-10-18T08:58:04.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.852+0000 7fdce158e640 1 Processor -- start 2023-10-18T08:58:04.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.852+0000 7fdce158e640 1 -- start start 2023-10-18T08:58:04.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.853+0000 7fdce158e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc0b1900 0x7fdcdc13db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:04.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.853+0000 7fdce158e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcdc1405a0 0x7fdcdc1e1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:04.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.853+0000 7fdce158e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcdc1e6450 0x7fdcdc1e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:04.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.853+0000 7fdce158e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc1e6e10 con 0x7fdcdc0b1900 2023-10-18T08:58:04.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.853+0000 7fdce158e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc1e3f90 con 0x7fdcdc1e6450 2023-10-18T08:58:04.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.853+0000 7fdce158e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc13e050 con 0x7fdcdc1405a0 2023-10-18T08:58:04.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.853+0000 7fdcdbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc0b1900 0x7fdcdc13db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:04.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.853+0000 7fdcdbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc0b1900 0x7fdcdc13db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41564/0 (socket says 172.21.15.40:41564) 2023-10-18T08:58:04.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.853+0000 7fdcdbfff640 1 -- 172.21.15.40:0/2958563850 learned_addr learned my addr 172.21.15.40:0/2958563850 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:04.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.854+0000 7fdcdbfff640 1 -- 172.21.15.40:0/2958563850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcdc1405a0 msgr2=0x7fdcdc1e1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:04.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.854+0000 7fdce0d8d640 1 --2- 172.21.15.40:0/2958563850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcdc1e6450 0x7fdcdc1e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:04.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.854+0000 7fdcdbfff640 1 --2- 172.21.15.40:0/2958563850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcdc1405a0 0x7fdcdc1e1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:04.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.854+0000 7fdcdbfff640 1 -- 172.21.15.40:0/2958563850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcdc1e6450 msgr2=0x7fdcdc1e68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:04.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.854+0000 7fdcdbfff640 1 --2- 172.21.15.40:0/2958563850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcdc1e6450 0x7fdcdc1e68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:04.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.854+0000 7fdcdbfff640 1 -- 172.21.15.40:0/2958563850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc00a670 con 0x7fdcdc0b1900 2023-10-18T08:58:04.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.854+0000 7fdcdbfff640 1 --2- 172.21.15.40:0/2958563850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc0b1900 0x7fdcdc13db10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00a960 tx=0x7fdccc004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:04.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.855+0000 7fdcd97fa640 1 -- 172.21.15.40:0/2958563850 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc0037f0 con 0x7fdcdc0b1900 2023-10-18T08:58:04.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.855+0000 7fdcd97fa640 1 -- 172.21.15.40:0/2958563850 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdccc003a40 con 0x7fdcdc0b1900 2023-10-18T08:58:04.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.855+0000 7fdcd97fa640 1 -- 172.21.15.40:0/2958563850 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc003df0 con 0x7fdcdc0b1900 2023-10-18T08:58:04.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.855+0000 7fdce158e640 1 -- 172.21.15.40:0/2958563850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcdc158730 con 0x7fdcdc0b1900 2023-10-18T08:58:04.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.855+0000 7fdce158e640 1 -- 172.21.15.40:0/2958563850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcdc1e44c0 con 0x7fdcdc0b1900 2023-10-18T08:58:04.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.856+0000 7fdce158e640 1 -- 172.21.15.40:0/2958563850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc066e00 con 0x7fdcdc0b1900 2023-10-18T08:58:04.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.857+0000 7fdcd97fa640 1 -- 172.21.15.40:0/2958563850 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdccc004e50 con 0x7fdcdc0b1900 2023-10-18T08:58:04.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.858+0000 7fdcd97fa640 1 --2- 172.21.15.40:0/2958563850 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdcbc06ae40 0x7fdcbc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:04.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.858+0000 7fdcd97fa640 1 -- 172.21.15.40:0/2958563850 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fdccc0ac950 con 0x7fdcdc0b1900 2023-10-18T08:58:04.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.859+0000 7fdcd3fff640 1 --2- 172.21.15.40:0/2958563850 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdcbc06ae40 0x7fdcbc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:04.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.860+0000 7fdcd3fff640 1 --2- 172.21.15.40:0/2958563850 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdcbc06ae40 0x7fdcbc06d300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4006fd0 tx=0x7fdcc4006bc0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:04.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:04.864+0000 7fdcd97fa640 1 -- 172.21.15.40:0/2958563850 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdccc0779b0 con 0x7fdcdc0b1900 2023-10-18T08:58:04.872 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:58:04.881 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:58:05.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.060+0000 7fdce158e640 1 -- 172.21.15.40:0/2958563850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdcdc066e00 con 0x7fdcdc0b1900 2023-10-18T08:58:05.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.061+0000 7fdcd97fa640 1 -- 172.21.15.40:0/2958563850 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdccc07afd0 con 0x7fdcdc0b1900 2023-10-18T08:58:05.064 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:58:05.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.063+0000 7fdce158e640 1 -- 172.21.15.40:0/2958563850 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdcbc06ae40 msgr2=0x7fdcbc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:05.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.063+0000 7fdce158e640 1 --2- 172.21.15.40:0/2958563850 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdcbc06ae40 0x7fdcbc06d300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4006fd0 tx=0x7fdcc4006bc0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.063+0000 7fdce158e640 1 -- 172.21.15.40:0/2958563850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc0b1900 msgr2=0x7fdcdc13db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:05.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.063+0000 7fdce158e640 1 --2- 172.21.15.40:0/2958563850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc0b1900 0x7fdcdc13db10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00a960 tx=0x7fdccc004a50 comp rx=0 tx=0).stop 2023-10-18T08:58:05.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.064+0000 7fdce158e640 1 -- 172.21.15.40:0/2958563850 shutdown_connections 2023-10-18T08:58:05.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.064+0000 7fdce158e640 1 --2- 172.21.15.40:0/2958563850 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdcbc06ae40 0x7fdcbc06d300 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.064+0000 7fdce158e640 1 --2- 172.21.15.40:0/2958563850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcdc1e6450 0x7fdcdc1e68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.064+0000 7fdce158e640 1 --2- 172.21.15.40:0/2958563850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcdc1405a0 0x7fdcdc1e1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.064+0000 7fdce158e640 1 --2- 172.21.15.40:0/2958563850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcdc0b1900 0x7fdcdc13db10 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.064+0000 7fdce158e640 1 -- 172.21.15.40:0/2958563850 >> 172.21.15.40:0/2958563850 conn(0x7fdcdc069720 msgr2=0x7fdcdc13f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:05.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.064+0000 7fdce158e640 1 -- 172.21.15.40:0/2958563850 shutdown_connections 2023-10-18T08:58:05.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.064+0000 7fdce158e640 1 -- 172.21.15.40:0/2958563850 wait complete. 2023-10-18T08:58:05.085 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:58:05.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.310+0000 7f4a454ea640 1 -- 172.21.15.40:0/945346174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a4015af70 msgr2=0x7f4a400c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:05.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.310+0000 7f4a454ea640 1 --2- 172.21.15.40:0/945346174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a4015af70 0x7f4a400c9620 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4a38005fd0 tx=0x7f4a3802c0f0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.310+0000 7f4a454ea640 1 -- 172.21.15.40:0/945346174 shutdown_connections 2023-10-18T08:58:05.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.310+0000 7f4a454ea640 1 --2- 172.21.15.40:0/945346174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a4015af70 0x7f4a400c9620 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.310+0000 7f4a454ea640 1 -- 172.21.15.40:0/945346174 >> 172.21.15.40:0/945346174 conn(0x7f4a4006ca10 msgr2=0x7f4a4006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.310+0000 7f4a454ea640 1 -- 172.21.15.40:0/945346174 shutdown_connections 2023-10-18T08:58:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.310+0000 7f4a454ea640 1 -- 172.21.15.40:0/945346174 wait complete. 2023-10-18T08:58:05.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.312+0000 7f4a454ea640 1 Processor -- start 2023-10-18T08:58:05.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.313+0000 7f4a454ea640 1 -- start start 2023-10-18T08:58:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.314+0000 7f4a454ea640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a4006d5d0 0x7f4a4013b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.314+0000 7f4a454ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a40071270 0x7f4a401533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.314+0000 7f4a454ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a4015af70 0x7f4a4013db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.314+0000 7f4a454ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a40152f40 con 0x7f4a4015af70 2023-10-18T08:58:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.314+0000 7f4a454ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a40152bf0 con 0x7f4a40071270 2023-10-18T08:58:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.314+0000 7f4a454ea640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a401f1810 con 0x7f4a4006d5d0 2023-10-18T08:58:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.314+0000 7f4a44ce9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a4015af70 0x7f4a4013db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.314+0000 7f4a44ce9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a4015af70 0x7f4a4013db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49556/0 (socket says 172.21.15.40:49556) 2023-10-18T08:58:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.314+0000 7f4a44ce9640 1 -- 172.21.15.40:0/1793274951 learned_addr learned my addr 172.21.15.40:0/1793274951 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.315+0000 7f4a3ffff640 1 --2- 172.21.15.40:0/1793274951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a4006d5d0 0x7f4a4013b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.315+0000 7f4a37fff640 1 --2- 172.21.15.40:0/1793274951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a40071270 0x7f4a401533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.315+0000 7f4a44ce9640 1 -- 172.21.15.40:0/1793274951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a4006d5d0 msgr2=0x7f4a4013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.315+0000 7f4a44ce9640 1 --2- 172.21.15.40:0/1793274951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a4006d5d0 0x7f4a4013b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.315+0000 7f4a44ce9640 1 -- 172.21.15.40:0/1793274951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a40071270 msgr2=0x7f4a401533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.315+0000 7f4a44ce9640 1 --2- 172.21.15.40:0/1793274951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a40071270 0x7f4a401533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.315+0000 7f4a44ce9640 1 -- 172.21.15.40:0/1793274951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a38009840 con 0x7f4a4015af70 2023-10-18T08:58:05.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.315+0000 7f4a37fff640 1 --2- 172.21.15.40:0/1793274951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a40071270 0x7f4a401533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.315+0000 7f4a44ce9640 1 --2- 172.21.15.40:0/1793274951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a4015af70 0x7f4a4013db80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4a38036040 tx=0x7f4a38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.316+0000 7f4a3d7fa640 1 -- 172.21.15.40:0/1793274951 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38004670 con 0x7f4a4015af70 2023-10-18T08:58:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.316+0000 7f4a3d7fa640 1 -- 172.21.15.40:0/1793274951 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4a380345a0 con 0x7f4a4015af70 2023-10-18T08:58:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.316+0000 7f4a3d7fa640 1 -- 172.21.15.40:0/1793274951 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a380349b0 con 0x7f4a4015af70 2023-10-18T08:58:05.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.316+0000 7f4a454ea640 1 -- 172.21.15.40:0/1793274951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4013e5b0 con 0x7f4a4015af70 2023-10-18T08:58:05.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.316+0000 7f4a454ea640 1 -- 172.21.15.40:0/1793274951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4014ed90 con 0x7f4a4015af70 2023-10-18T08:58:05.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.317+0000 7f4a454ea640 1 -- 172.21.15.40:0/1793274951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a401530f0 con 0x7f4a4015af70 2023-10-18T08:58:05.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.318+0000 7f4a3d7fa640 1 -- 172.21.15.40:0/1793274951 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4a38034b10 con 0x7f4a4015af70 2023-10-18T08:58:05.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.319+0000 7f4a3d7fa640 1 --2- 172.21.15.40:0/1793274951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a2006ae40 0x7f4a2006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:05.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.319+0000 7f4a3d7fa640 1 -- 172.21.15.40:0/1793274951 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4a380ac950 con 0x7f4a4015af70 2023-10-18T08:58:05.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.320+0000 7f4a3ffff640 1 --2- 172.21.15.40:0/1793274951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a2006ae40 0x7f4a2006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:05.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.321+0000 7f4a3ffff640 1 --2- 172.21.15.40:0/1793274951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a2006ae40 0x7f4a2006d300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4a30001270 tx=0x7f4a30009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:05.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.323+0000 7f4a3d7fa640 1 -- 172.21.15.40:0/1793274951 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a401530f0 con 0x7f4a4015af70 2023-10-18T08:58:05.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.519+0000 7f4a454ea640 1 -- 172.21.15.40:0/1793274951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4a40066e00 con 0x7f4a4015af70 2023-10-18T08:58:05.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.520+0000 7f4a3d7fa640 1 -- 172.21.15.40:0/1793274951 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4a38043020 con 0x7f4a4015af70 2023-10-18T08:58:05.523 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:58:05.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.522+0000 7f4a454ea640 1 -- 172.21.15.40:0/1793274951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a2006ae40 msgr2=0x7f4a2006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:05.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.522+0000 7f4a454ea640 1 --2- 172.21.15.40:0/1793274951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a2006ae40 0x7f4a2006d300 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4a30001270 tx=0x7f4a30009040 comp rx=0 tx=0).stop 2023-10-18T08:58:05.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.522+0000 7f4a454ea640 1 -- 172.21.15.40:0/1793274951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a4015af70 msgr2=0x7f4a4013db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:05.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.522+0000 7f4a454ea640 1 --2- 172.21.15.40:0/1793274951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a4015af70 0x7f4a4013db80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4a38036040 tx=0x7f4a38004270 comp rx=0 tx=0).stop 2023-10-18T08:58:05.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.522+0000 7f4a454ea640 1 -- 172.21.15.40:0/1793274951 shutdown_connections 2023-10-18T08:58:05.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.522+0000 7f4a454ea640 1 --2- 172.21.15.40:0/1793274951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a2006ae40 0x7f4a2006d300 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.522+0000 7f4a454ea640 1 --2- 172.21.15.40:0/1793274951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a4015af70 0x7f4a4013db80 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.522+0000 7f4a454ea640 1 --2- 172.21.15.40:0/1793274951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a40071270 0x7f4a401533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.522+0000 7f4a454ea640 1 --2- 172.21.15.40:0/1793274951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a4006d5d0 0x7f4a4013b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.522+0000 7f4a454ea640 1 -- 172.21.15.40:0/1793274951 >> 172.21.15.40:0/1793274951 conn(0x7f4a4006ca10 msgr2=0x7f4a40070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:05.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.522+0000 7f4a454ea640 1 -- 172.21.15.40:0/1793274951 shutdown_connections 2023-10-18T08:58:05.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.523+0000 7f4a454ea640 1 -- 172.21.15.40:0/1793274951 wait complete. 2023-10-18T08:58:05.546 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:58:05.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.781+0000 7f3bdffff640 1 -- 172.21.15.40:0/1357646961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3be00ac260 msgr2=0x7f3be0155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:05.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.781+0000 7f3bdffff640 1 --2- 172.21.15.40:0/1357646961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3be00ac260 0x7f3be0155330 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3be00c9b60 tx=0x7f3bd802c260 comp rx=0 tx=0).stop 2023-10-18T08:58:05.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.782+0000 7f3bdffff640 1 -- 172.21.15.40:0/1357646961 shutdown_connections 2023-10-18T08:58:05.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.782+0000 7f3bdffff640 1 --2- 172.21.15.40:0/1357646961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3be00ac260 0x7f3be0155330 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.782+0000 7f3bdffff640 1 --2- 172.21.15.40:0/1357646961 >> v2:172.21.15.144:3300/0 conn(0x7f3be006c200 0x7f3be00abd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.782+0000 7f3bdffff640 1 -- 172.21.15.40:0/1357646961 >> 172.21.15.40:0/1357646961 conn(0x7f3be006ca10 msgr2=0x7f3be006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:05.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.782+0000 7f3bdffff640 1 -- 172.21.15.40:0/1357646961 shutdown_connections 2023-10-18T08:58:05.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.782+0000 7f3bdffff640 1 -- 172.21.15.40:0/1357646961 wait complete. 2023-10-18T08:58:05.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.783+0000 7f3bdffff640 1 Processor -- start 2023-10-18T08:58:05.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.784+0000 7f3bdffff640 1 -- start start 2023-10-18T08:58:05.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.784+0000 7f3bdffff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3be006c200 0x7f3be00b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:05.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.784+0000 7f3bdffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be00ac260 0x7f3be00b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:05.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.784+0000 7f3bdffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3be00cc630 0x7f3be015e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:05.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.784+0000 7f3bdffff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be015dd50 con 0x7f3be00cc630 2023-10-18T08:58:05.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.784+0000 7f3bdffff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be015d140 con 0x7f3be00ac260 2023-10-18T08:58:05.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.784+0000 7f3bdffff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be015d420 con 0x7f3be006c200 2023-10-18T08:58:05.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.785+0000 7f3bdf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3be00cc630 0x7f3be015e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:05.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.785+0000 7f3bdf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3be00cc630 0x7f3be015e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49574/0 (socket says 172.21.15.40:49574) 2023-10-18T08:58:05.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.785+0000 7f3bdf7fe640 1 -- 172.21.15.40:0/1273596989 learned_addr learned my addr 172.21.15.40:0/1273596989 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:05.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.785+0000 7f3bd7fff640 1 --2- 172.21.15.40:0/1273596989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be00ac260 0x7f3be00b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:05.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.785+0000 7f3bdf7fe640 1 -- 172.21.15.40:0/1273596989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3be006c200 msgr2=0x7f3be00b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:05.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.785+0000 7f3bdeffd640 1 --2- 172.21.15.40:0/1273596989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3be006c200 0x7f3be00b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:05.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.785+0000 7f3bdf7fe640 1 --2- 172.21.15.40:0/1273596989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3be006c200 0x7f3be00b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.785+0000 7f3bdf7fe640 1 -- 172.21.15.40:0/1273596989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be00ac260 msgr2=0x7f3be00b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:05.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.785+0000 7f3bdf7fe640 1 --2- 172.21.15.40:0/1273596989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be00ac260 0x7f3be00b2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.785+0000 7f3bdf7fe640 1 -- 172.21.15.40:0/1273596989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd8009900 con 0x7f3be00cc630 2023-10-18T08:58:05.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.785+0000 7f3bd7fff640 1 --2- 172.21.15.40:0/1273596989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be00ac260 0x7f3be00b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:05.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.785+0000 7f3bdeffd640 1 --2- 172.21.15.40:0/1273596989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3be006c200 0x7f3be00b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:05.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.786+0000 7f3bdf7fe640 1 --2- 172.21.15.40:0/1273596989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3be00cc630 0x7f3be015e1e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009d60 tx=0x7f3bd8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:05.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.786+0000 7f3bdcff9640 1 -- 172.21.15.40:0/1273596989 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8004450 con 0x7f3be00cc630 2023-10-18T08:58:05.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.786+0000 7f3bdcff9640 1 -- 172.21.15.40:0/1273596989 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3bd80046a0 con 0x7f3be00cc630 2023-10-18T08:58:05.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.786+0000 7f3bdcff9640 1 -- 172.21.15.40:0/1273596989 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80406b0 con 0x7f3be00cc630 2023-10-18T08:58:05.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.786+0000 7f3bdffff640 1 -- 172.21.15.40:0/1273596989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be015d700 con 0x7f3be00cc630 2023-10-18T08:58:05.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.786+0000 7f3bdffff640 1 -- 172.21.15.40:0/1273596989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be00b0d80 con 0x7f3be00cc630 2023-10-18T08:58:05.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.788+0000 7f3bdffff640 1 -- 172.21.15.40:0/1273596989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be00b0f40 con 0x7f3be00cc630 2023-10-18T08:58:05.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.789+0000 7f3bdcff9640 1 -- 172.21.15.40:0/1273596989 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3bd800a040 con 0x7f3be00cc630 2023-10-18T08:58:05.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.789+0000 7f3bdcff9640 1 --2- 172.21.15.40:0/1273596989 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3bc006ae70 0x7f3bc006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:05.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.790+0000 7f3bdcff9640 1 -- 172.21.15.40:0/1273596989 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3bd80ac720 con 0x7f3be00cc630 2023-10-18T08:58:05.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.790+0000 7f3bdeffd640 1 --2- 172.21.15.40:0/1273596989 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3bc006ae70 0x7f3bc006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.791+0000 7f3bdeffd640 1 --2- 172.21.15.40:0/1273596989 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3bc006ae70 0x7f3bc006d330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000a830 tx=0x7f3bd0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.794+0000 7f3bdcff9640 1 -- 172.21.15.40:0/1273596989 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3be00b0f40 con 0x7f3be00cc630 2023-10-18T08:58:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.988+0000 7f3bdffff640 1 -- 172.21.15.40:0/1273596989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3be00b0f40 con 0x7f3be00cc630 2023-10-18T08:58:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.989+0000 7f3bdcff9640 1 -- 172.21.15.40:0/1273596989 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3be00b0f40 con 0x7f3be00cc630 2023-10-18T08:58:05.993 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:58:05.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.992+0000 7f3bdffff640 1 -- 172.21.15.40:0/1273596989 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3bc006ae70 msgr2=0x7f3bc006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.992+0000 7f3bdffff640 1 --2- 172.21.15.40:0/1273596989 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3bc006ae70 0x7f3bc006d330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000a830 tx=0x7f3bd0008040 comp rx=0 tx=0).stop 2023-10-18T08:58:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.992+0000 7f3bdffff640 1 -- 172.21.15.40:0/1273596989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3be00cc630 msgr2=0x7f3be015e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.992+0000 7f3bdffff640 1 --2- 172.21.15.40:0/1273596989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3be00cc630 0x7f3be015e1e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009d60 tx=0x7f3bd8003c20 comp rx=0 tx=0).stop 2023-10-18T08:58:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.992+0000 7f3bdffff640 1 -- 172.21.15.40:0/1273596989 shutdown_connections 2023-10-18T08:58:05.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.992+0000 7f3bdffff640 1 --2- 172.21.15.40:0/1273596989 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3bc006ae70 0x7f3bc006d330 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.992+0000 7f3bdffff640 1 --2- 172.21.15.40:0/1273596989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3be00cc630 0x7f3be015e1e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.992+0000 7f3bdffff640 1 --2- 172.21.15.40:0/1273596989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3be00ac260 0x7f3be00b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.992+0000 7f3bdffff640 1 --2- 172.21.15.40:0/1273596989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3be006c200 0x7f3be00b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.992+0000 7f3bdffff640 1 -- 172.21.15.40:0/1273596989 >> 172.21.15.40:0/1273596989 conn(0x7f3be006ca10 msgr2=0x7f3be015c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.992+0000 7f3bdffff640 1 -- 172.21.15.40:0/1273596989 shutdown_connections 2023-10-18T08:58:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:05.992+0000 7f3bdffff640 1 -- 172.21.15.40:0/1273596989 wait complete. 2023-10-18T08:58:06.013 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T08:58:06.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.237+0000 7fe87c883640 1 -- 172.21.15.40:0/1626612499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe87806b710 msgr2=0x7fe87806bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:06.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.237+0000 7fe87c883640 1 --2- 172.21.15.40:0/1626612499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe87806b710 0x7fe87806bb80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe870009ca0 tx=0x7fe87002c230 comp rx=0 tx=0).stop 2023-10-18T08:58:06.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.237+0000 7fe87c883640 1 -- 172.21.15.40:0/1626612499 shutdown_connections 2023-10-18T08:58:06.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.237+0000 7fe87c883640 1 --2- 172.21.15.40:0/1626612499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe87806b710 0x7fe87806bb80 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:06.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.237+0000 7fe87c883640 1 --2- 172.21.15.40:0/1626612499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8780cdbb0 0x7fe87815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:06.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.237+0000 7fe87c883640 1 -- 172.21.15.40:0/1626612499 >> 172.21.15.40:0/1626612499 conn(0x7fe87806cee0 msgr2=0x7fe878069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:06.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.237+0000 7fe87c883640 1 -- 172.21.15.40:0/1626612499 shutdown_connections 2023-10-18T08:58:06.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.237+0000 7fe87c883640 1 -- 172.21.15.40:0/1626612499 wait complete. 2023-10-18T08:58:06.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.239+0000 7fe87c883640 1 Processor -- start 2023-10-18T08:58:06.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.239+0000 7fe87c883640 1 -- start start 2023-10-18T08:58:06.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.240+0000 7fe87c883640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe87806b710 0x7fe87814c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:06.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.240+0000 7fe87c883640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8780cbc20 0x7fe8780ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:06.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.240+0000 7fe87c883640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8780cdbb0 0x7fe8780aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:06.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.240+0000 7fe87c883640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe878153d20 con 0x7fe87806b710 2023-10-18T08:58:06.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.240+0000 7fe87c883640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe878070fe0 con 0x7fe8780cbc20 2023-10-18T08:58:06.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.240+0000 7fe87c883640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8780709d0 con 0x7fe8780cdbb0 2023-10-18T08:58:06.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.240+0000 7fe87659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe87806b710 0x7fe87814c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:06.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.240+0000 7fe87659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe87806b710 0x7fe87814c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49596/0 (socket says 172.21.15.40:49596) 2023-10-18T08:58:06.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.240+0000 7fe87659c640 1 -- 172.21.15.40:0/175459334 learned_addr learned my addr 172.21.15.40:0/175459334 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:06.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.241+0000 7fe876d9d640 1 --2- 172.21.15.40:0/175459334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8780cdbb0 0x7fe8780aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:06.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.241+0000 7fe86ffff640 1 --2- 172.21.15.40:0/175459334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8780cbc20 0x7fe8780ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:06.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.241+0000 7fe87659c640 1 -- 172.21.15.40:0/175459334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8780cdbb0 msgr2=0x7fe8780aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:06.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.241+0000 7fe87659c640 1 --2- 172.21.15.40:0/175459334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8780cdbb0 0x7fe8780aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:06.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.241+0000 7fe87659c640 1 -- 172.21.15.40:0/175459334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8780cbc20 msgr2=0x7fe8780ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:06.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.241+0000 7fe87659c640 1 --2- 172.21.15.40:0/175459334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8780cbc20 0x7fe8780ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:06.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.241+0000 7fe87659c640 1 -- 172.21.15.40:0/175459334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe870009900 con 0x7fe87806b710 2023-10-18T08:58:06.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.241+0000 7fe87659c640 1 --2- 172.21.15.40:0/175459334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe87806b710 0x7fe87814c570 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe86800c970 tx=0x7fe86800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:06.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.241+0000 7fe86effd640 1 -- 172.21.15.40:0/175459334 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868007bb0 con 0x7fe87806b710 2023-10-18T08:58:06.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.241+0000 7fe86effd640 1 -- 172.21.15.40:0/175459334 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe868007e20 con 0x7fe87806b710 2023-10-18T08:58:06.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.241+0000 7fe86effd640 1 -- 172.21.15.40:0/175459334 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868005280 con 0x7fe87806b710 2023-10-18T08:58:06.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.241+0000 7fe87c883640 1 -- 172.21.15.40:0/175459334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8781534a0 con 0x7fe87806b710 2023-10-18T08:58:06.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.241+0000 7fe87c883640 1 -- 172.21.15.40:0/175459334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe878153a40 con 0x7fe87806b710 2023-10-18T08:58:06.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.243+0000 7fe87c883640 1 -- 172.21.15.40:0/175459334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe878066fd0 con 0x7fe87806b710 2023-10-18T08:58:06.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.244+0000 7fe86effd640 1 -- 172.21.15.40:0/175459334 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe8680053e0 con 0x7fe87806b710 2023-10-18T08:58:06.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.245+0000 7fe86effd640 1 --2- 172.21.15.40:0/175459334 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe85806ae40 0x7fe85806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:06.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.245+0000 7fe86effd640 1 -- 172.21.15.40:0/175459334 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe868089c50 con 0x7fe87806b710 2023-10-18T08:58:06.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.245+0000 7fe86ffff640 1 --2- 172.21.15.40:0/175459334 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe85806ae40 0x7fe85806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:06.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.246+0000 7fe86ffff640 1 --2- 172.21.15.40:0/175459334 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe85806ae40 0x7fe85806d300 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe8781573e0 tx=0x7fe860009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:06.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.249+0000 7fe86effd640 1 -- 172.21.15.40:0/175459334 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe868054c20 con 0x7fe87806b710 2023-10-18T08:58:06.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.457+0000 7fe87c883640 1 -- 172.21.15.40:0/175459334 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe8780c9ba0 con 0x7fe87806b710 2023-10-18T08:58:06.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.458+0000 7fe86effd640 1 -- 172.21.15.40:0/175459334 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v56) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe86801d070 con 0x7fe87806b710 2023-10-18T08:58:06.461 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T08:58:06.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.461+0000 7fe87c883640 1 -- 172.21.15.40:0/175459334 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe85806ae40 msgr2=0x7fe85806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:06.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.461+0000 7fe87c883640 1 --2- 172.21.15.40:0/175459334 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe85806ae40 0x7fe85806d300 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe8781573e0 tx=0x7fe860009040 comp rx=0 tx=0).stop 2023-10-18T08:58:06.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.461+0000 7fe87c883640 1 -- 172.21.15.40:0/175459334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe87806b710 msgr2=0x7fe87814c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:06.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.461+0000 7fe87c883640 1 --2- 172.21.15.40:0/175459334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe87806b710 0x7fe87814c570 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe86800c970 tx=0x7fe86800ce30 comp rx=0 tx=0).stop 2023-10-18T08:58:06.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.461+0000 7fe87c883640 1 -- 172.21.15.40:0/175459334 shutdown_connections 2023-10-18T08:58:06.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.461+0000 7fe87c883640 1 --2- 172.21.15.40:0/175459334 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe85806ae40 0x7fe85806d300 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:06.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.461+0000 7fe87c883640 1 --2- 172.21.15.40:0/175459334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8780cdbb0 0x7fe8780aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:06.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.461+0000 7fe87c883640 1 --2- 172.21.15.40:0/175459334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8780cbc20 0x7fe8780ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:06.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.461+0000 7fe87c883640 1 --2- 172.21.15.40:0/175459334 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe87806b710 0x7fe87814c570 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:06.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.461+0000 7fe87c883640 1 -- 172.21.15.40:0/175459334 >> 172.21.15.40:0/175459334 conn(0x7fe87806cee0 msgr2=0x7fe878158e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:06.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.461+0000 7fe87c883640 1 -- 172.21.15.40:0/175459334 shutdown_connections 2023-10-18T08:58:06.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:06.461+0000 7fe87c883640 1 -- 172.21.15.40:0/175459334 wait complete. 2023-10-18T08:58:06.483 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-18T08:58:06.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:06.484 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-18T08:58:06.658 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T08:58:06.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:06.659 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T08:58:06.702 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:58:06.703 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:58:06.749 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:58:06.750 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:58:06.844 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:58:07.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.084+0000 7f98a210f640 1 -- 172.21.15.40:0/2644473572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9888004d00 msgr2=0x7f9888005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:07.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.084+0000 7f98a210f640 1 --2- 172.21.15.40:0/2644473572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9888004d00 0x7f9888005180 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f989400c640 tx=0x7f98880057f0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.084+0000 7f989bfff640 1 -- 172.21.15.40:0/2644473572 reap_dead start 2023-10-18T08:58:07.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.084+0000 7f98a210f640 1 -- 172.21.15.40:0/2644473572 shutdown_connections 2023-10-18T08:58:07.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.084+0000 7f98a210f640 1 -- 172.21.15.40:0/2644473572 >> 172.21.15.40:0/2644473572 conn(0x7f989c06cee0 msgr2=0x7f989c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:07.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.084+0000 7f98a210f640 1 -- 172.21.15.40:0/2644473572 shutdown_connections 2023-10-18T08:58:07.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.084+0000 7f98a210f640 1 -- 172.21.15.40:0/2644473572 wait complete. 2023-10-18T08:58:07.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.086+0000 7f98a210f640 1 Processor -- start 2023-10-18T08:58:07.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.086+0000 7f98a210f640 1 -- start start 2023-10-18T08:58:07.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.087+0000 7f98a210f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9888004d00 0x7f989c13cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:07.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.087+0000 7f98a210f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f989c1e1070 0x7f989c1e14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:07.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.087+0000 7f98a210f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989c0cb010 0x7f989c0cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:07.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.087+0000 7f98a210f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c13d0c0 con 0x7f9888004d00 2023-10-18T08:58:07.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.087+0000 7f98a210f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989c0cb9d0 con 0x7f989c0cb010 2023-10-18T08:58:07.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.087+0000 7f98a210f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989c14b220 con 0x7f989c1e1070 2023-10-18T08:58:07.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.087+0000 7f989b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9888004d00 0x7f989c13cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:07.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.087+0000 7f989b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9888004d00 0x7f989c13cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49614/0 (socket says 172.21.15.40:49614) 2023-10-18T08:58:07.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.087+0000 7f989b7fe640 1 -- 172.21.15.40:0/253560147 learned_addr learned my addr 172.21.15.40:0/253560147 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:07.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.088+0000 7f989b7fe640 1 -- 172.21.15.40:0/253560147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f989c1e1070 msgr2=0x7f989c1e14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:07.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.088+0000 7f989b7fe640 1 --2- 172.21.15.40:0/253560147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f989c1e1070 0x7f989c1e14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.088+0000 7f989b7fe640 1 -- 172.21.15.40:0/253560147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989c0cb010 msgr2=0x7f989c0cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:07.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.088+0000 7f989b7fe640 1 --2- 172.21.15.40:0/253560147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989c0cb010 0x7f989c0cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.088+0000 7f989b7fe640 1 -- 172.21.15.40:0/253560147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c1e2480 con 0x7f9888004d00 2023-10-18T08:58:07.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.088+0000 7f989b7fe640 1 --2- 172.21.15.40:0/253560147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9888004d00 0x7f989c13cb80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f988c00fc40 tx=0x7f988c00d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:07.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.088+0000 7f9898ff9640 1 -- 172.21.15.40:0/253560147 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c00a710 con 0x7f9888004d00 2023-10-18T08:58:07.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.088+0000 7f9898ff9640 1 -- 172.21.15.40:0/253560147 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f988c005220 con 0x7f9888004d00 2023-10-18T08:58:07.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.088+0000 7f9898ff9640 1 -- 172.21.15.40:0/253560147 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c005610 con 0x7f9888004d00 2023-10-18T08:58:07.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.088+0000 7f98a210f640 1 -- 172.21.15.40:0/253560147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989c1e1be0 con 0x7f9888004d00 2023-10-18T08:58:07.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.089+0000 7f98a210f640 1 -- 172.21.15.40:0/253560147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989c1e21a0 con 0x7f9888004d00 2023-10-18T08:58:07.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.090+0000 7f98a210f640 1 -- 172.21.15.40:0/253560147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989c066fd0 con 0x7f9888004d00 2023-10-18T08:58:07.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.091+0000 7f9898ff9640 1 -- 172.21.15.40:0/253560147 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f988c0047a0 con 0x7f9888004d00 2023-10-18T08:58:07.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.092+0000 7f9898ff9640 1 --2- 172.21.15.40:0/253560147 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f987c06ae40 0x7f987c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:07.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.092+0000 7f9898ff9640 1 -- 172.21.15.40:0/253560147 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f988c08a750 con 0x7f9888004d00 2023-10-18T08:58:07.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.093+0000 7f9893fff640 1 --2- 172.21.15.40:0/253560147 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f987c06ae40 0x7f987c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:07.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.094+0000 7f9893fff640 1 --2- 172.21.15.40:0/253560147 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f987c06ae40 0x7f987c06d300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f989c069890 tx=0x7f98840068e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:07.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.096+0000 7f9898ff9640 1 -- 172.21.15.40:0/253560147 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f988c055720 con 0x7f9888004d00 2023-10-18T08:58:07.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.289+0000 7f98a210f640 1 -- 172.21.15.40:0/253560147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f989c06fb50 con 0x7f9888004d00 2023-10-18T08:58:07.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.292+0000 7f9898ff9640 1 -- 172.21.15.40:0/253560147 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16620 (secure 0 0 0) 0x7f988c058d40 con 0x7f9888004d00 2023-10-18T08:58:07.296 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:07.296 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":56,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:58:02.917888+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:57:12.765969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T08:57:13.851110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":5}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:58:07.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.294+0000 7f98a210f640 1 -- 172.21.15.40:0/253560147 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f987c06ae40 msgr2=0x7f987c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:07.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.294+0000 7f98a210f640 1 --2- 172.21.15.40:0/253560147 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f987c06ae40 0x7f987c06d300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f989c069890 tx=0x7f98840068e0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.295+0000 7f98a210f640 1 -- 172.21.15.40:0/253560147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9888004d00 msgr2=0x7f989c13cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:07.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.295+0000 7f98a210f640 1 --2- 172.21.15.40:0/253560147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9888004d00 0x7f989c13cb80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f988c00fc40 tx=0x7f988c00d7b0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.295+0000 7f98a210f640 1 -- 172.21.15.40:0/253560147 shutdown_connections 2023-10-18T08:58:07.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.295+0000 7f98a210f640 1 --2- 172.21.15.40:0/253560147 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f987c06ae40 0x7f987c06d300 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.295+0000 7f98a210f640 1 --2- 172.21.15.40:0/253560147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989c0cb010 0x7f989c0cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.295+0000 7f98a210f640 1 --2- 172.21.15.40:0/253560147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f989c1e1070 0x7f989c1e14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.295+0000 7f98a210f640 1 --2- 172.21.15.40:0/253560147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9888004d00 0x7f989c13cb80 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.295+0000 7f98a210f640 1 -- 172.21.15.40:0/253560147 >> 172.21.15.40:0/253560147 conn(0x7f989c06cee0 msgr2=0x7f989c0cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:07.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.295+0000 7f98a210f640 1 -- 172.21.15.40:0/253560147 shutdown_connections 2023-10-18T08:58:07.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.295+0000 7f98a210f640 1 -- 172.21.15.40:0/253560147 wait complete. 2023-10-18T08:58:07.317 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:57:12.765969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:57:13.851110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:58:07.318 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:58:07.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.550+0000 7fa923475640 1 -- 172.21.15.40:0/4233210464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c13c580 msgr2=0x7fa91c15a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:07.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.550+0000 7fa923475640 1 --2- 172.21.15.40:0/4233210464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c13c580 0x7fa91c15a270 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa918009830 tx=0x7fa91802bd60 comp rx=0 tx=0).stop 2023-10-18T08:58:07.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.550+0000 7fa923475640 1 -- 172.21.15.40:0/4233210464 shutdown_connections 2023-10-18T08:58:07.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.550+0000 7fa923475640 1 --2- 172.21.15.40:0/4233210464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa91c0aed00 0x7fa91c0af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.550+0000 7fa923475640 1 --2- 172.21.15.40:0/4233210464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c13c580 0x7fa91c15a270 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.550+0000 7fa923475640 1 -- 172.21.15.40:0/4233210464 >> 172.21.15.40:0/4233210464 conn(0x7fa91c069720 msgr2=0x7fa91c069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:07.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.550+0000 7fa923475640 1 -- 172.21.15.40:0/4233210464 shutdown_connections 2023-10-18T08:58:07.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.550+0000 7fa923475640 1 -- 172.21.15.40:0/4233210464 wait complete. 2023-10-18T08:58:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.552+0000 7fa923475640 1 Processor -- start 2023-10-18T08:58:07.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.552+0000 7fa923475640 1 -- start start 2023-10-18T08:58:07.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.553+0000 7fa923475640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa91c0708e0 0x7fa91c144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:07.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.553+0000 7fa923475640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa91c0aed00 0x7fa91c1391b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:07.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.553+0000 7fa923475640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c13c580 0x7fa91c142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:07.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.553+0000 7fa923475640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1387d0 con 0x7fa91c13c580 2023-10-18T08:58:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.553+0000 7fa923475640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1350e0 con 0x7fa91c0708e0 2023-10-18T08:58:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.553+0000 7fa923475640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c147f40 con 0x7fa91c0aed00 2023-10-18T08:58:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.554+0000 7fa921a12640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c13c580 0x7fa91c142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.554+0000 7fa921a12640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c13c580 0x7fa91c142090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49638/0 (socket says 172.21.15.40:49638) 2023-10-18T08:58:07.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.554+0000 7fa921a12640 1 -- 172.21.15.40:0/3179179793 learned_addr learned my addr 172.21.15.40:0/3179179793 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:07.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.554+0000 7fa921211640 1 --2- 172.21.15.40:0/3179179793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa91c0708e0 0x7fa91c144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:07.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.554+0000 7fa920a10640 1 --2- 172.21.15.40:0/3179179793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa91c0aed00 0x7fa91c1391b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:07.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.554+0000 7fa921a12640 1 -- 172.21.15.40:0/3179179793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa91c0aed00 msgr2=0x7fa91c1391b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:07.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.554+0000 7fa921a12640 1 --2- 172.21.15.40:0/3179179793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa91c0aed00 0x7fa91c1391b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.554+0000 7fa921a12640 1 -- 172.21.15.40:0/3179179793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa91c0708e0 msgr2=0x7fa91c144360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.554+0000 7fa921a12640 1 --2- 172.21.15.40:0/3179179793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa91c0708e0 0x7fa91c144360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.554+0000 7fa921a12640 1 -- 172.21.15.40:0/3179179793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa918009490 con 0x7fa91c13c580 2023-10-18T08:58:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.555+0000 7fa921a12640 1 --2- 172.21.15.40:0/3179179793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c13c580 0x7fa91c142090 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa91400cb10 tx=0x7fa91400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.555+0000 7fa911ffb640 1 -- 172.21.15.40:0/3179179793 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91400ac10 con 0x7fa91c13c580 2023-10-18T08:58:07.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.555+0000 7fa911ffb640 1 -- 172.21.15.40:0/3179179793 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa91400ae60 con 0x7fa91c13c580 2023-10-18T08:58:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.555+0000 7fa911ffb640 1 -- 172.21.15.40:0/3179179793 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914012660 con 0x7fa91c13c580 2023-10-18T08:58:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.555+0000 7fa923475640 1 -- 172.21.15.40:0/3179179793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91c1348a0 con 0x7fa91c13c580 2023-10-18T08:58:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.555+0000 7fa923475640 1 -- 172.21.15.40:0/3179179793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c1389b0 con 0x7fa91c13c580 2023-10-18T08:58:07.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.556+0000 7fa923475640 1 -- 172.21.15.40:0/3179179793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91c134bb0 con 0x7fa91c13c580 2023-10-18T08:58:07.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.557+0000 7fa911ffb640 1 -- 172.21.15.40:0/3179179793 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa91401a3b0 con 0x7fa91c13c580 2023-10-18T08:58:07.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.558+0000 7fa911ffb640 1 --2- 172.21.15.40:0/3179179793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa90006ae40 0x7fa90006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:07.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.558+0000 7fa911ffb640 1 -- 172.21.15.40:0/3179179793 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa914089f80 con 0x7fa91c13c580 2023-10-18T08:58:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.559+0000 7fa921211640 1 --2- 172.21.15.40:0/3179179793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa90006ae40 0x7fa90006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:07.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.560+0000 7fa921211640 1 --2- 172.21.15.40:0/3179179793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa90006ae40 0x7fa90006d300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa918036000 tx=0x7fa918004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:07.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.562+0000 7fa911ffb640 1 -- 172.21.15.40:0/3179179793 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa914054ec0 con 0x7fa91c13c580 2023-10-18T08:58:07.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.758+0000 7fa923475640 1 -- 172.21.15.40:0/3179179793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa91c066fd0 con 0x7fa91c13c580 2023-10-18T08:58:07.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.759+0000 7fa911ffb640 1 -- 172.21.15.40:0/3179179793 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa914016020 con 0x7fa91c13c580 2023-10-18T08:58:07.762 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:58:07.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.761+0000 7fa923475640 1 -- 172.21.15.40:0/3179179793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa90006ae40 msgr2=0x7fa90006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:07.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.761+0000 7fa923475640 1 --2- 172.21.15.40:0/3179179793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa90006ae40 0x7fa90006d300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa918036000 tx=0x7fa918004730 comp rx=0 tx=0).stop 2023-10-18T08:58:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.762+0000 7fa923475640 1 -- 172.21.15.40:0/3179179793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c13c580 msgr2=0x7fa91c142090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.762+0000 7fa923475640 1 --2- 172.21.15.40:0/3179179793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c13c580 0x7fa91c142090 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa91400cb10 tx=0x7fa91400a060 comp rx=0 tx=0).stop 2023-10-18T08:58:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.762+0000 7fa923475640 1 -- 172.21.15.40:0/3179179793 shutdown_connections 2023-10-18T08:58:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.762+0000 7fa923475640 1 --2- 172.21.15.40:0/3179179793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa90006ae40 0x7fa90006d300 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.762+0000 7fa923475640 1 --2- 172.21.15.40:0/3179179793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa91c13c580 0x7fa91c142090 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.762+0000 7fa923475640 1 --2- 172.21.15.40:0/3179179793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa91c0aed00 0x7fa91c1391b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.762+0000 7fa923475640 1 --2- 172.21.15.40:0/3179179793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa91c0708e0 0x7fa91c144360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:07.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.762+0000 7fa923475640 1 -- 172.21.15.40:0/3179179793 >> 172.21.15.40:0/3179179793 conn(0x7fa91c069720 msgr2=0x7fa91c13b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:07.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.762+0000 7fa923475640 1 -- 172.21.15.40:0/3179179793 shutdown_connections 2023-10-18T08:58:07.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:07.762+0000 7fa923475640 1 -- 172.21.15.40:0/3179179793 wait complete. 2023-10-18T08:58:07.784 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:58:08.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.003+0000 7f7b76a44640 1 -- 172.21.15.40:0/2333060979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b7015b4d0 msgr2=0x7f7b700ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:08.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.003+0000 7f7b76a44640 1 --2- 172.21.15.40:0/2333060979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b7015b4d0 0x7f7b700ca040 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7b60005fd0 tx=0x7f7b6002be20 comp rx=0 tx=0).stop 2023-10-18T08:58:08.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.003+0000 7f7b76a44640 1 -- 172.21.15.40:0/2333060979 shutdown_connections 2023-10-18T08:58:08.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.003+0000 7f7b76a44640 1 --2- 172.21.15.40:0/2333060979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b7015b4d0 0x7f7b700ca040 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.003+0000 7f7b76a44640 1 -- 172.21.15.40:0/2333060979 >> 172.21.15.40:0/2333060979 conn(0x7f7b70069720 msgr2=0x7f7b70069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:08.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.003+0000 7f7b76a44640 1 -- 172.21.15.40:0/2333060979 shutdown_connections 2023-10-18T08:58:08.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.003+0000 7f7b76a44640 1 -- 172.21.15.40:0/2333060979 wait complete. 2023-10-18T08:58:08.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.005+0000 7f7b76a44640 1 Processor -- start 2023-10-18T08:58:08.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.005+0000 7f7b76a44640 1 -- start start 2023-10-18T08:58:08.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.006+0000 7f7b76a44640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b700ca580 0x7f7b701483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:08.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.006+0000 7f7b76a44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b700cbf20 0x7f7b70070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:08.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.006+0000 7f7b76a44640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b7015b4d0 0x7f7b701417f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:08.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.006+0000 7f7b76a44640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b700af3d0 con 0x7f7b7015b4d0 2023-10-18T08:58:08.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.006+0000 7f7b76a44640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b70153d50 con 0x7f7b700cbf20 2023-10-18T08:58:08.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.006+0000 7f7b76a44640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b70148930 con 0x7f7b700ca580 2023-10-18T08:58:08.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.006+0000 7f7b74fe1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b7015b4d0 0x7f7b701417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:08.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.006+0000 7f7b74fe1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b7015b4d0 0x7f7b701417f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49656/0 (socket says 172.21.15.40:49656) 2023-10-18T08:58:08.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.006+0000 7f7b74fe1640 1 -- 172.21.15.40:0/2489964640 learned_addr learned my addr 172.21.15.40:0/2489964640 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:08.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.007+0000 7f7b74fe1640 1 -- 172.21.15.40:0/2489964640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b700ca580 msgr2=0x7f7b701483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.007+0000 7f7b74fe1640 1 --2- 172.21.15.40:0/2489964640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b700ca580 0x7f7b701483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.007+0000 7f7b74fe1640 1 -- 172.21.15.40:0/2489964640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b700cbf20 msgr2=0x7f7b70070aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.007+0000 7f7b74fe1640 1 --2- 172.21.15.40:0/2489964640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b700cbf20 0x7f7b70070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.007+0000 7f7b74fe1640 1 -- 172.21.15.40:0/2489964640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b60009560 con 0x7f7b7015b4d0 2023-10-18T08:58:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.007+0000 7f7b74fe1640 1 --2- 172.21.15.40:0/2489964640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b7015b4d0 0x7f7b701417f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800c790 tx=0x7f7b6800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.007+0000 7f7b6d7fa640 1 -- 172.21.15.40:0/2489964640 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6800cea0 con 0x7f7b7015b4d0 2023-10-18T08:58:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.007+0000 7f7b6d7fa640 1 -- 172.21.15.40:0/2489964640 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7b68003da0 con 0x7f7b7015b4d0 2023-10-18T08:58:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.007+0000 7f7b6d7fa640 1 -- 172.21.15.40:0/2489964640 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b680052d0 con 0x7f7b7015b4d0 2023-10-18T08:58:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.007+0000 7f7b76a44640 1 -- 172.21.15.40:0/2489964640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b700ae7c0 con 0x7f7b7015b4d0 2023-10-18T08:58:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.008+0000 7f7b76a44640 1 -- 172.21.15.40:0/2489964640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b700aed80 con 0x7f7b7015b4d0 2023-10-18T08:58:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.009+0000 7f7b76a44640 1 -- 172.21.15.40:0/2489964640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b70066fd0 con 0x7f7b7015b4d0 2023-10-18T08:58:08.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.010+0000 7f7b6d7fa640 1 -- 172.21.15.40:0/2489964640 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7b68005430 con 0x7f7b7015b4d0 2023-10-18T08:58:08.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.010+0000 7f7b6d7fa640 1 --2- 172.21.15.40:0/2489964640 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7b5006ae40 0x7f7b5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:08.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.011+0000 7f7b6d7fa640 1 -- 172.21.15.40:0/2489964640 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7b680896d0 con 0x7f7b7015b4d0 2023-10-18T08:58:08.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.011+0000 7f7b6ffff640 1 --2- 172.21.15.40:0/2489964640 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7b5006ae40 0x7f7b5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:08.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.012+0000 7f7b6ffff640 1 --2- 172.21.15.40:0/2489964640 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7b5006ae40 0x7f7b5006d300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7b60035000 tx=0x7f7b600046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.015+0000 7f7b6d7fa640 1 -- 172.21.15.40:0/2489964640 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b680546a0 con 0x7f7b7015b4d0 2023-10-18T08:58:08.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.209+0000 7f7b76a44640 1 -- 172.21.15.40:0/2489964640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7b700af510 con 0x7f7b7015b4d0 2023-10-18T08:58:08.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.210+0000 7f7b6d7fa640 1 -- 172.21.15.40:0/2489964640 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7b700af510 con 0x7f7b7015b4d0 2023-10-18T08:58:08.214 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:58:08.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.213+0000 7f7b76a44640 1 -- 172.21.15.40:0/2489964640 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7b5006ae40 msgr2=0x7f7b5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:08.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.213+0000 7f7b76a44640 1 --2- 172.21.15.40:0/2489964640 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7b5006ae40 0x7f7b5006d300 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7b60035000 tx=0x7f7b600046c0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.213+0000 7f7b76a44640 1 -- 172.21.15.40:0/2489964640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b7015b4d0 msgr2=0x7f7b701417f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:08.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.213+0000 7f7b76a44640 1 --2- 172.21.15.40:0/2489964640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b7015b4d0 0x7f7b701417f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800c790 tx=0x7f7b6800cc50 comp rx=0 tx=0).stop 2023-10-18T08:58:08.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.213+0000 7f7b76a44640 1 -- 172.21.15.40:0/2489964640 shutdown_connections 2023-10-18T08:58:08.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.213+0000 7f7b76a44640 1 --2- 172.21.15.40:0/2489964640 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7b5006ae40 0x7f7b5006d300 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.213+0000 7f7b76a44640 1 --2- 172.21.15.40:0/2489964640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b7015b4d0 0x7f7b701417f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.213+0000 7f7b76a44640 1 --2- 172.21.15.40:0/2489964640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b700cbf20 0x7f7b70070aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.213+0000 7f7b76a44640 1 --2- 172.21.15.40:0/2489964640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b700ca580 0x7f7b701483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.213+0000 7f7b76a44640 1 -- 172.21.15.40:0/2489964640 >> 172.21.15.40:0/2489964640 conn(0x7f7b70069720 msgr2=0x7f7b7015e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:08.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.214+0000 7f7b76a44640 1 -- 172.21.15.40:0/2489964640 shutdown_connections 2023-10-18T08:58:08.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.214+0000 7f7b76a44640 1 -- 172.21.15.40:0/2489964640 wait complete. 2023-10-18T08:58:08.236 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:58:08.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.465+0000 7f6695f9e640 1 -- 172.21.15.40:0/3300927174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66900cd260 msgr2=0x7f66900cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:08.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.465+0000 7f6695f9e640 1 --2- 172.21.15.40:0/3300927174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66900cd260 0x7f66900cd6f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f66900aea00 tx=0x7f668002bd10 comp rx=0 tx=0).stop 2023-10-18T08:58:08.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.465+0000 7f6695f9e640 1 -- 172.21.15.40:0/3300927174 shutdown_connections 2023-10-18T08:58:08.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.465+0000 7f6695f9e640 1 --2- 172.21.15.40:0/3300927174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6690097a10 0x7f6690097e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.465+0000 7f6695f9e640 1 --2- 172.21.15.40:0/3300927174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66900cd260 0x7f66900cd6f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.465+0000 7f6695f9e640 1 --2- 172.21.15.40:0/3300927174 >> v2:172.21.15.144:3300/0 conn(0x7f66901592e0 0x7f66900ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.465+0000 7f6695f9e640 1 -- 172.21.15.40:0/3300927174 >> 172.21.15.40:0/3300927174 conn(0x7f66900b0ac0 msgr2=0x7f66900b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:08.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.465+0000 7f6695f9e640 1 -- 172.21.15.40:0/3300927174 shutdown_connections 2023-10-18T08:58:08.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.465+0000 7f6695f9e640 1 -- 172.21.15.40:0/3300927174 wait complete. 2023-10-18T08:58:08.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.467+0000 7f6695f9e640 1 Processor -- start 2023-10-18T08:58:08.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.467+0000 7f6695f9e640 1 -- start start 2023-10-18T08:58:08.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.468+0000 7f6695f9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6690097a10 0x7f66900c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:08.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.468+0000 7f6695f9e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66900cd260 0x7f669015bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:08.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.468+0000 7f6695f9e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66901592e0 0x7f66900c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:08.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.468+0000 7f6695f9e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669015b8d0 con 0x7f6690097a10 2023-10-18T08:58:08.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.468+0000 7f6695f9e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669015aa00 con 0x7f66901592e0 2023-10-18T08:58:08.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.468+0000 7f6695f9e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6690156660 con 0x7f66900cd260 2023-10-18T08:58:08.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.468+0000 7f6694f9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6690097a10 0x7f66900c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:08.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.468+0000 7f6694f9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6690097a10 0x7f66900c5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49688/0 (socket says 172.21.15.40:49688) 2023-10-18T08:58:08.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.468+0000 7f6694f9c640 1 -- 172.21.15.40:0/1420277315 learned_addr learned my addr 172.21.15.40:0/1420277315 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:08.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.469+0000 7f668ffff640 1 --2- 172.21.15.40:0/1420277315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66900cd260 0x7f669015bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:08.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.469+0000 7f6694f9c640 1 -- 172.21.15.40:0/1420277315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66900cd260 msgr2=0x7f669015bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:08.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.469+0000 7f6694f9c640 1 --2- 172.21.15.40:0/1420277315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66900cd260 0x7f669015bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.469+0000 7f6694f9c640 1 -- 172.21.15.40:0/1420277315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66901592e0 msgr2=0x7f66900c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:08.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.469+0000 7f6694f9c640 1 --2- 172.21.15.40:0/1420277315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66901592e0 0x7f66900c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.469+0000 7f6694f9c640 1 -- 172.21.15.40:0/1420277315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6680009560 con 0x7f6690097a10 2023-10-18T08:58:08.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.469+0000 7f668ffff640 1 --2- 172.21.15.40:0/1420277315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66900cd260 0x7f669015bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:08.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.469+0000 7f6694f9c640 1 --2- 172.21.15.40:0/1420277315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6690097a10 0x7f66900c5870 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f668400d740 tx=0x7f668400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:08.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.469+0000 7f668d7fa640 1 -- 172.21.15.40:0/1420277315 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684009c30 con 0x7f6690097a10 2023-10-18T08:58:08.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.469+0000 7f668d7fa640 1 -- 172.21.15.40:0/1420277315 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6684009e80 con 0x7f6690097a10 2023-10-18T08:58:08.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.469+0000 7f668d7fa640 1 -- 172.21.15.40:0/1420277315 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66840106c0 con 0x7f6690097a10 2023-10-18T08:58:08.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.470+0000 7f6695f9e640 1 -- 172.21.15.40:0/1420277315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669014ad30 con 0x7f6690097a10 2023-10-18T08:58:08.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.470+0000 7f6695f9e640 1 -- 172.21.15.40:0/1420277315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669015af30 con 0x7f6690097a10 2023-10-18T08:58:08.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.471+0000 7f6695f9e640 1 -- 172.21.15.40:0/1420277315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669014aa90 con 0x7f6690097a10 2023-10-18T08:58:08.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.472+0000 7f668d7fa640 1 -- 172.21.15.40:0/1420277315 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6684010820 con 0x7f6690097a10 2023-10-18T08:58:08.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.473+0000 7f668d7fa640 1 --2- 172.21.15.40:0/1420277315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f667c06ae40 0x7f667c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:08.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.473+0000 7f668d7fa640 1 -- 172.21.15.40:0/1420277315 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6684089b70 con 0x7f6690097a10 2023-10-18T08:58:08.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.473+0000 7f668ffff640 1 --2- 172.21.15.40:0/1420277315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f667c06ae40 0x7f667c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:08.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.474+0000 7f668ffff640 1 --2- 172.21.15.40:0/1420277315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f667c06ae40 0x7f667c06d300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6680035000 tx=0x7f6680004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:08.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.477+0000 7f668d7fa640 1 -- 172.21.15.40:0/1420277315 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6684054b40 con 0x7f6690097a10 2023-10-18T08:58:08.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.670+0000 7f6695f9e640 1 -- 172.21.15.40:0/1420277315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6690073570 con 0x7f6690097a10 2023-10-18T08:58:08.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.670+0000 7f668d7fa640 1 -- 172.21.15.40:0/1420277315 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7f66840210a0 con 0x7f6690097a10 2023-10-18T08:58:08.674 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:58:08.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.673+0000 7f6695f9e640 1 -- 172.21.15.40:0/1420277315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f667c06ae40 msgr2=0x7f667c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:08.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.673+0000 7f6695f9e640 1 --2- 172.21.15.40:0/1420277315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f667c06ae40 0x7f667c06d300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6680035000 tx=0x7f6680004730 comp rx=0 tx=0).stop 2023-10-18T08:58:08.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.673+0000 7f6695f9e640 1 -- 172.21.15.40:0/1420277315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6690097a10 msgr2=0x7f66900c5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:08.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.673+0000 7f6695f9e640 1 --2- 172.21.15.40:0/1420277315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6690097a10 0x7f66900c5870 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f668400d740 tx=0x7f668400dc00 comp rx=0 tx=0).stop 2023-10-18T08:58:08.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.673+0000 7f6695f9e640 1 -- 172.21.15.40:0/1420277315 shutdown_connections 2023-10-18T08:58:08.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.673+0000 7f6695f9e640 1 --2- 172.21.15.40:0/1420277315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f667c06ae40 0x7f667c06d300 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.673+0000 7f6695f9e640 1 --2- 172.21.15.40:0/1420277315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66901592e0 0x7f66900c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.673+0000 7f6695f9e640 1 --2- 172.21.15.40:0/1420277315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66900cd260 0x7f669015bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.673+0000 7f6695f9e640 1 --2- 172.21.15.40:0/1420277315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6690097a10 0x7f66900c5870 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.673+0000 7f6695f9e640 1 -- 172.21.15.40:0/1420277315 >> 172.21.15.40:0/1420277315 conn(0x7f66900b0ac0 msgr2=0x7f6690158480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:08.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.673+0000 7f6695f9e640 1 -- 172.21.15.40:0/1420277315 shutdown_connections 2023-10-18T08:58:08.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.673+0000 7f6695f9e640 1 -- 172.21.15.40:0/1420277315 wait complete. 2023-10-18T08:58:08.696 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T08:58:08.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.928+0000 7fde5cb5e640 1 -- 172.21.15.40:0/4007392649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde44003970 msgr2=0x7fde44003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:08.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.928+0000 7fde5cb5e640 1 --2- 172.21.15.40:0/4007392649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde44003970 0x7fde44003d90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fde48001d50 tx=0x7fde440057f0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.928+0000 7fde5cb5e640 1 -- 172.21.15.40:0/4007392649 shutdown_connections 2023-10-18T08:58:08.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.928+0000 7fde5cb5e640 1 --2- 172.21.15.40:0/4007392649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde44003970 0x7fde44003d90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.928+0000 7fde5cb5e640 1 -- 172.21.15.40:0/4007392649 >> 172.21.15.40:0/4007392649 conn(0x7fde5808ad00 msgr2=0x7fde5809a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:08.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.928+0000 7fde5cb5e640 1 -- 172.21.15.40:0/4007392649 shutdown_connections 2023-10-18T08:58:08.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.928+0000 7fde5cb5e640 1 -- 172.21.15.40:0/4007392649 wait complete. 2023-10-18T08:58:08.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.930+0000 7fde5cb5e640 1 Processor -- start 2023-10-18T08:58:08.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.930+0000 7fde5cb5e640 1 -- start start 2023-10-18T08:58:08.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.931+0000 7fde5cb5e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde44003970 0x7fde581e53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.931+0000 7fde5cb5e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815d780 0x7fde5815dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.931+0000 7fde5cb5e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde5815bd10 0x7fde5815c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.931+0000 7fde5cb5e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde580c8400 con 0x7fde5815d780 2023-10-18T08:58:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.931+0000 7fde5cb5e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde580c8f80 con 0x7fde44003970 2023-10-18T08:58:08.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.931+0000 7fde5cb5e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde580c8650 con 0x7fde5815bd10 2023-10-18T08:58:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.931+0000 7fde56ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815d780 0x7fde5815dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.931+0000 7fde56ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815d780 0x7fde5815dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49710/0 (socket says 172.21.15.40:49710) 2023-10-18T08:58:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.931+0000 7fde56ffd640 1 -- 172.21.15.40:0/902835292 learned_addr learned my addr 172.21.15.40:0/902835292 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.932+0000 7fde57fff640 1 --2- 172.21.15.40:0/902835292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde5815bd10 0x7fde5815c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:08.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.932+0000 7fde56ffd640 1 -- 172.21.15.40:0/902835292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde5815bd10 msgr2=0x7fde5815c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:08.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.932+0000 7fde56ffd640 1 --2- 172.21.15.40:0/902835292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde5815bd10 0x7fde5815c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.932+0000 7fde56ffd640 1 -- 172.21.15.40:0/902835292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde44003970 msgr2=0x7fde581e53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:08.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.932+0000 7fde56ffd640 1 --2- 172.21.15.40:0/902835292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde44003970 0x7fde581e53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:08.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.932+0000 7fde56ffd640 1 -- 172.21.15.40:0/902835292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde580c8930 con 0x7fde5815d780 2023-10-18T08:58:08.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.932+0000 7fde57fff640 1 --2- 172.21.15.40:0/902835292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde5815bd10 0x7fde5815c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:08.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.932+0000 7fde56ffd640 1 --2- 172.21.15.40:0/902835292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815d780 0x7fde5815dc00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fde4000e9a0 tx=0x7fde4000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:08.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.932+0000 7fde4ffff640 1 -- 172.21.15.40:0/902835292 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4000ccc0 con 0x7fde5815d780 2023-10-18T08:58:08.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.932+0000 7fde4ffff640 1 -- 172.21.15.40:0/902835292 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fde4000ce20 con 0x7fde5815d780 2023-10-18T08:58:08.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.932+0000 7fde4ffff640 1 -- 172.21.15.40:0/902835292 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40005280 con 0x7fde5815d780 2023-10-18T08:58:08.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.932+0000 7fde5cb5e640 1 -- 172.21.15.40:0/902835292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde580c8c10 con 0x7fde5815d780 2023-10-18T08:58:08.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.933+0000 7fde5cb5e640 1 -- 172.21.15.40:0/902835292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde581e58e0 con 0x7fde5815d780 2023-10-18T08:58:08.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.934+0000 7fde5cb5e640 1 -- 172.21.15.40:0/902835292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde58071da0 con 0x7fde5815d780 2023-10-18T08:58:08.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.935+0000 7fde4ffff640 1 -- 172.21.15.40:0/902835292 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fde400053e0 con 0x7fde5815d780 2023-10-18T08:58:08.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.936+0000 7fde4ffff640 1 --2- 172.21.15.40:0/902835292 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde3806ae40 0x7fde3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:08.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.936+0000 7fde4ffff640 1 -- 172.21.15.40:0/902835292 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fde40014070 con 0x7fde5815d780 2023-10-18T08:58:08.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.936+0000 7fde577fe640 1 --2- 172.21.15.40:0/902835292 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde3806ae40 0x7fde3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:08.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.937+0000 7fde577fe640 1 --2- 172.21.15.40:0/902835292 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde3806ae40 0x7fde3806d300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fde48034000 tx=0x7fde480014a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:08.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:08.940+0000 7fde4ffff640 1 -- 172.21.15.40:0/902835292 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde40054ff0 con 0x7fde5815d780 2023-10-18T08:58:09.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.149+0000 7fde5cb5e640 1 -- 172.21.15.40:0/902835292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fde58066e00 con 0x7fde5815d780 2023-10-18T08:58:09.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.150+0000 7fde4ffff640 1 -- 172.21.15.40:0/902835292 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v56) v1 ==== 80+0+0 (secure 0 0 0) 0x7fde40058610 con 0x7fde5815d780 2023-10-18T08:58:09.153 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T08:58:09.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.152+0000 7fde5cb5e640 1 -- 172.21.15.40:0/902835292 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde3806ae40 msgr2=0x7fde3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:09.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.152+0000 7fde5cb5e640 1 --2- 172.21.15.40:0/902835292 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde3806ae40 0x7fde3806d300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fde48034000 tx=0x7fde480014a0 comp rx=0 tx=0).stop 2023-10-18T08:58:09.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.152+0000 7fde5cb5e640 1 -- 172.21.15.40:0/902835292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815d780 msgr2=0x7fde5815dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:09.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.152+0000 7fde5cb5e640 1 --2- 172.21.15.40:0/902835292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815d780 0x7fde5815dc00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fde4000e9a0 tx=0x7fde4000ee60 comp rx=0 tx=0).stop 2023-10-18T08:58:09.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.152+0000 7fde5cb5e640 1 -- 172.21.15.40:0/902835292 shutdown_connections 2023-10-18T08:58:09.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.152+0000 7fde5cb5e640 1 --2- 172.21.15.40:0/902835292 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde3806ae40 0x7fde3806d300 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:09.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.152+0000 7fde5cb5e640 1 --2- 172.21.15.40:0/902835292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde5815bd10 0x7fde5815c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:09.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.152+0000 7fde5cb5e640 1 --2- 172.21.15.40:0/902835292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5815d780 0x7fde5815dc00 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:09.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.152+0000 7fde5cb5e640 1 --2- 172.21.15.40:0/902835292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde44003970 0x7fde581e53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:09.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.152+0000 7fde5cb5e640 1 -- 172.21.15.40:0/902835292 >> 172.21.15.40:0/902835292 conn(0x7fde5808ad00 msgr2=0x7fde580cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:09.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.153+0000 7fde5cb5e640 1 -- 172.21.15.40:0/902835292 shutdown_connections 2023-10-18T08:58:09.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.153+0000 7fde5cb5e640 1 -- 172.21.15.40:0/902835292 wait complete. 2023-10-18T08:58:09.176 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-18T08:58:09.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:09.176 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-18T08:58:09.290 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T08:58:09.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:09.291 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T08:58:09.344 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:58:09.345 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:58:09.410 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T08:58:09.410 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:58:09.411 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:58:09.471 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T08:58:09.471 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:58:09.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.711+0000 7f91f0f80640 1 -- 172.21.15.40:0/3937205885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91ec0b0630 msgr2=0x7f91ec0c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:09.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.711+0000 7f91f0f80640 1 --2- 172.21.15.40:0/3937205885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91ec0b0630 0x7f91ec0c99c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f91ec06f590 tx=0x7f91d402bda0 comp rx=0 tx=0).stop 2023-10-18T08:58:09.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.711+0000 7f91f0f80640 1 -- 172.21.15.40:0/3937205885 shutdown_connections 2023-10-18T08:58:09.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.711+0000 7f91f0f80640 1 --2- 172.21.15.40:0/3937205885 >> v2:172.21.15.144:3300/0 conn(0x7f91ec0c9f00 0x7f91ec157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:09.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.711+0000 7f91f0f80640 1 --2- 172.21.15.40:0/3937205885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91ec0b0630 0x7f91ec0c99c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:09.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.711+0000 7f91f0f80640 1 --2- 172.21.15.40:0/3937205885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91ec06d6f0 0x7f91ec0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:09.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.711+0000 7f91f0f80640 1 -- 172.21.15.40:0/3937205885 >> 172.21.15.40:0/3937205885 conn(0x7f91ec069720 msgr2=0x7f91ec069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:09.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.711+0000 7f91f0f80640 1 -- 172.21.15.40:0/3937205885 shutdown_connections 2023-10-18T08:58:09.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.711+0000 7f91f0f80640 1 -- 172.21.15.40:0/3937205885 wait complete. 2023-10-18T08:58:09.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.713+0000 7f91f0f80640 1 Processor -- start 2023-10-18T08:58:09.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.713+0000 7f91f0f80640 1 -- start start 2023-10-18T08:58:09.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.714+0000 7f91f0f80640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91ec06d6f0 0x7f91ec143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:09.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.714+0000 7f91f0f80640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91ec0b0630 0x7f91ec1455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:09.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.714+0000 7f91f0f80640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91ec0c9f00 0x7f91ec0ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:09.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.714+0000 7f91f0f80640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec14a230 con 0x7f91ec0c9f00 2023-10-18T08:58:09.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.714+0000 7f91f0f80640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec0af260 con 0x7f91ec06d6f0 2023-10-18T08:58:09.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.714+0000 7f91f0f80640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec149ee0 con 0x7f91ec0b0630 2023-10-18T08:58:09.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.715+0000 7f91ebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91ec0c9f00 0x7f91ec0ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:09.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.715+0000 7f91ebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91ec0c9f00 0x7f91ec0ae4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49726/0 (socket says 172.21.15.40:49726) 2023-10-18T08:58:09.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.715+0000 7f91ebfff640 1 -- 172.21.15.40:0/2612074719 learned_addr learned my addr 172.21.15.40:0/2612074719 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:09.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.715+0000 7f91e2ffd640 1 --2- 172.21.15.40:0/2612074719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91ec0b0630 0x7f91ec1455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:09.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.715+0000 7f91eb7fe640 1 --2- 172.21.15.40:0/2612074719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91ec06d6f0 0x7f91ec143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:09.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.715+0000 7f91ebfff640 1 -- 172.21.15.40:0/2612074719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91ec0b0630 msgr2=0x7f91ec1455e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:09.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.715+0000 7f91ebfff640 1 --2- 172.21.15.40:0/2612074719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91ec0b0630 0x7f91ec1455e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:09.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.715+0000 7f91ebfff640 1 -- 172.21.15.40:0/2612074719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91ec06d6f0 msgr2=0x7f91ec143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:09.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.715+0000 7f91ebfff640 1 --2- 172.21.15.40:0/2612074719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91ec06d6f0 0x7f91ec143320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:09.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.715+0000 7f91ebfff640 1 -- 172.21.15.40:0/2612074719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d4009560 con 0x7f91ec0c9f00 2023-10-18T08:58:09.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.715+0000 7f91e2ffd640 1 --2- 172.21.15.40:0/2612074719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91ec0b0630 0x7f91ec1455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:09.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.715+0000 7f91ebfff640 1 --2- 172.21.15.40:0/2612074719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91ec0c9f00 0x7f91ec0ae4f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f91e400e960 tx=0x7f91e400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:09.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.716+0000 7f91e8ff9640 1 -- 172.21.15.40:0/2612074719 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4004d20 con 0x7f91ec0c9f00 2023-10-18T08:58:09.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.716+0000 7f91e8ff9640 1 -- 172.21.15.40:0/2612074719 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f91e400cb20 con 0x7f91ec0c9f00 2023-10-18T08:58:09.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.716+0000 7f91f0f80640 1 -- 172.21.15.40:0/2612074719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ec143ff0 con 0x7f91ec0c9f00 2023-10-18T08:58:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.716+0000 7f91e8ff9640 1 -- 172.21.15.40:0/2612074719 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4005020 con 0x7f91ec0c9f00 2023-10-18T08:58:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.716+0000 7f91f0f80640 1 -- 172.21.15.40:0/2612074719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ec0af790 con 0x7f91ec0c9f00 2023-10-18T08:58:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.717+0000 7f91f0f80640 1 -- 172.21.15.40:0/2612074719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91ec06a030 con 0x7f91ec0c9f00 2023-10-18T08:58:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.718+0000 7f91e8ff9640 1 -- 172.21.15.40:0/2612074719 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f91e4005180 con 0x7f91ec0c9f00 2023-10-18T08:58:09.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.719+0000 7f91e8ff9640 1 --2- 172.21.15.40:0/2612074719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f91cc06ae40 0x7f91cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:09.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.719+0000 7f91e8ff9640 1 -- 172.21.15.40:0/2612074719 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f91e4089ef0 con 0x7f91ec0c9f00 2023-10-18T08:58:09.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.720+0000 7f91eb7fe640 1 --2- 172.21.15.40:0/2612074719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f91cc06ae40 0x7f91cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:09.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.721+0000 7f91eb7fe640 1 --2- 172.21.15.40:0/2612074719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f91cc06ae40 0x7f91cc06d300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f91dc005fd0 tx=0x7f91dc005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:09.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.724+0000 7f91e8ff9640 1 -- 172.21.15.40:0/2612074719 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f91e4054f50 con 0x7f91ec0c9f00 2023-10-18T08:58:09.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.958+0000 7f91f0f80640 1 -- 172.21.15.40:0/2612074719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91ec066e00 con 0x7f91ec0c9f00 2023-10-18T08:58:09.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.959+0000 7f91e8ff9640 1 -- 172.21.15.40:0/2612074719 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f91e401f070 con 0x7f91ec0c9f00 2023-10-18T08:58:09.963 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:09.963 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3668356540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3668356540},{"type":"v1","addr":"172.21.15.144:6836","nonce":3668356540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4222679844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4222679844},{"type":"v1","addr":"172.21.15.40:6834","nonce":4222679844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2829399368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2829399368},{"type":"v1","addr":"172.21.15.40:6835","nonce":2829399368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:57:14.826066+0000","modified":"2023-10-18T08:57:17.903233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1438456692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1438456692},{"type":"v1","addr":"172.21.15.144:6837","nonce":1438456692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-18T08:58:09.964 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 24 2023-10-18T08:58:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.962+0000 7f91f0f80640 1 -- 172.21.15.40:0/2612074719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f91cc06ae40 msgr2=0x7f91cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:09.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.962+0000 7f91f0f80640 1 --2- 172.21.15.40:0/2612074719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f91cc06ae40 0x7f91cc06d300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f91dc005fd0 tx=0x7f91dc005850 comp rx=0 tx=0).stop 2023-10-18T08:58:09.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.962+0000 7f91f0f80640 1 -- 172.21.15.40:0/2612074719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91ec0c9f00 msgr2=0x7f91ec0ae4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:09.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.962+0000 7f91f0f80640 1 --2- 172.21.15.40:0/2612074719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91ec0c9f00 0x7f91ec0ae4f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f91e400e960 tx=0x7f91e400ee20 comp rx=0 tx=0).stop 2023-10-18T08:58:09.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.962+0000 7f91f0f80640 1 -- 172.21.15.40:0/2612074719 shutdown_connections 2023-10-18T08:58:09.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.962+0000 7f91f0f80640 1 --2- 172.21.15.40:0/2612074719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f91cc06ae40 0x7f91cc06d300 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.962+0000 7f91f0f80640 1 --2- 172.21.15.40:0/2612074719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91ec0c9f00 0x7f91ec0ae4f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.962+0000 7f91f0f80640 1 --2- 172.21.15.40:0/2612074719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91ec0b0630 0x7f91ec1455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.962+0000 7f91f0f80640 1 --2- 172.21.15.40:0/2612074719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91ec06d6f0 0x7f91ec143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.962+0000 7f91f0f80640 1 -- 172.21.15.40:0/2612074719 >> 172.21.15.40:0/2612074719 conn(0x7f91ec069720 msgr2=0x7f91ec06f690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.962+0000 7f91f0f80640 1 -- 172.21.15.40:0/2612074719 shutdown_connections 2023-10-18T08:58:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:09.962+0000 7f91f0f80640 1 -- 172.21.15.40:0/2612074719 wait complete. 2023-10-18T08:58:09.986 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:58:10.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.211+0000 7fd11f59e640 1 -- 172.21.15.40:0/1900120564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1200ca890 msgr2=0x7fd1200b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:10.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.211+0000 7fd11f59e640 1 --2- 172.21.15.40:0/1900120564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1200ca890 0x7fd1200b1d10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd114005fd0 tx=0x7fd11402bd90 comp rx=0 tx=0).stop 2023-10-18T08:58:10.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.212+0000 7fd11f59e640 1 -- 172.21.15.40:0/1900120564 shutdown_connections 2023-10-18T08:58:10.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.212+0000 7fd11f59e640 1 --2- 172.21.15.40:0/1900120564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1200ca890 0x7fd1200b1d10 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:10.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.212+0000 7fd11f59e640 1 --2- 172.21.15.40:0/1900120564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd120157870 0x7fd1200ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:10.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.212+0000 7fd11f59e640 1 -- 172.21.15.40:0/1900120564 >> 172.21.15.40:0/1900120564 conn(0x7fd120069720 msgr2=0x7fd120069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:10.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.212+0000 7fd11f59e640 1 -- 172.21.15.40:0/1900120564 shutdown_connections 2023-10-18T08:58:10.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.212+0000 7fd11f59e640 1 -- 172.21.15.40:0/1900120564 wait complete. 2023-10-18T08:58:10.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.213+0000 7fd11f59e640 1 Processor -- start 2023-10-18T08:58:10.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.214+0000 7fd11f59e640 1 -- start start 2023-10-18T08:58:10.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.215+0000 7fd11f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1200b2250 0x7fd12014c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:10.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.215+0000 7fd11f59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1200ca890 0x7fd12015b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:10.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.215+0000 7fd11f59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd120157870 0x7fd12015ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:10.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.215+0000 7fd11f59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12015c3a0 con 0x7fd1200b2250 2023-10-18T08:58:10.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.215+0000 7fd11f59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12015c0c0 con 0x7fd1200ca890 2023-10-18T08:58:10.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.215+0000 7fd11f59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1200ab960 con 0x7fd120157870 2023-10-18T08:58:10.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.215+0000 7fd11e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1200b2250 0x7fd12014c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:10.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.215+0000 7fd11e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1200b2250 0x7fd12014c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49740/0 (socket says 172.21.15.40:49740) 2023-10-18T08:58:10.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.215+0000 7fd11e59c640 1 -- 172.21.15.40:0/938442462 learned_addr learned my addr 172.21.15.40:0/938442462 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:10.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.215+0000 7fd11ed9d640 1 --2- 172.21.15.40:0/938442462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd120157870 0x7fd12015ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:10.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.215+0000 7fd11dd9b640 1 --2- 172.21.15.40:0/938442462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1200ca890 0x7fd12015b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:10.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.215+0000 7fd11e59c640 1 -- 172.21.15.40:0/938442462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd120157870 msgr2=0x7fd12015ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:10.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.215+0000 7fd11e59c640 1 --2- 172.21.15.40:0/938442462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd120157870 0x7fd12015ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:10.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.215+0000 7fd11e59c640 1 -- 172.21.15.40:0/938442462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1200ca890 msgr2=0x7fd12015b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:10.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.215+0000 7fd11e59c640 1 --2- 172.21.15.40:0/938442462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1200ca890 0x7fd12015b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:10.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.215+0000 7fd11e59c640 1 -- 172.21.15.40:0/938442462 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd114009560 con 0x7fd1200b2250 2023-10-18T08:58:10.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.216+0000 7fd11e59c640 1 --2- 172.21.15.40:0/938442462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1200b2250 0x7fd12014c410 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd11000d6d0 tx=0x7fd11000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:10.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.216+0000 7fd10effd640 1 -- 172.21.15.40:0/938442462 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd110005220 con 0x7fd1200b2250 2023-10-18T08:58:10.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.216+0000 7fd10effd640 1 -- 172.21.15.40:0/938442462 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd110005490 con 0x7fd1200b2250 2023-10-18T08:58:10.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.216+0000 7fd10effd640 1 -- 172.21.15.40:0/938442462 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd110009a80 con 0x7fd1200b2250 2023-10-18T08:58:10.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.216+0000 7fd11f59e640 1 -- 172.21.15.40:0/938442462 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1200abc40 con 0x7fd1200b2250 2023-10-18T08:58:10.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.216+0000 7fd11f59e640 1 -- 172.21.15.40:0/938442462 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12014cf10 con 0x7fd1200b2250 2023-10-18T08:58:10.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.217+0000 7fd11f59e640 1 -- 172.21.15.40:0/938442462 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1200ac160 con 0x7fd1200b2250 2023-10-18T08:58:10.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.218+0000 7fd10effd640 1 -- 172.21.15.40:0/938442462 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd1100047a0 con 0x7fd1200b2250 2023-10-18T08:58:10.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.219+0000 7fd10effd640 1 --2- 172.21.15.40:0/938442462 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd10006ae40 0x7fd10006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:10.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.219+0000 7fd10effd640 1 -- 172.21.15.40:0/938442462 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd110089a60 con 0x7fd1200b2250 2023-10-18T08:58:10.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.220+0000 7fd11dd9b640 1 --2- 172.21.15.40:0/938442462 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd10006ae40 0x7fd10006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:10.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.221+0000 7fd11dd9b640 1 --2- 172.21.15.40:0/938442462 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd10006ae40 0x7fd10006d300 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd114035000 tx=0x7fd114004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:10.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.223+0000 7fd10effd640 1 -- 172.21.15.40:0/938442462 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd110054ac0 con 0x7fd1200b2250 2023-10-18T08:58:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.421+0000 7fd11f59e640 1 -- 172.21.15.40:0/938442462 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd120066e00 con 0x7fd1200b2250 2023-10-18T08:58:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.424+0000 7fd10effd640 1 -- 172.21.15.40:0/938442462 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16620 (secure 0 0 0) 0x7fd110081e60 con 0x7fd1200b2250 2023-10-18T08:58:10.428 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:10.428 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":56,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:58:02.917888+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:57:12.765969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T08:57:13.851110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":5}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:58:10.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.427+0000 7fd11f59e640 1 -- 172.21.15.40:0/938442462 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd10006ae40 msgr2=0x7fd10006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:10.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.427+0000 7fd11f59e640 1 --2- 172.21.15.40:0/938442462 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd10006ae40 0x7fd10006d300 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd114035000 tx=0x7fd114004730 comp rx=0 tx=0).stop 2023-10-18T08:58:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.427+0000 7fd11f59e640 1 -- 172.21.15.40:0/938442462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1200b2250 msgr2=0x7fd12014c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.427+0000 7fd11f59e640 1 --2- 172.21.15.40:0/938442462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1200b2250 0x7fd12014c410 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd11000d6d0 tx=0x7fd11000db90 comp rx=0 tx=0).stop 2023-10-18T08:58:10.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.427+0000 7fd11f59e640 1 -- 172.21.15.40:0/938442462 shutdown_connections 2023-10-18T08:58:10.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.427+0000 7fd11f59e640 1 --2- 172.21.15.40:0/938442462 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd10006ae40 0x7fd10006d300 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:10.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.427+0000 7fd11f59e640 1 --2- 172.21.15.40:0/938442462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd120157870 0x7fd12015ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:10.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.427+0000 7fd11f59e640 1 --2- 172.21.15.40:0/938442462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1200ca890 0x7fd12015b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:10.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.427+0000 7fd11f59e640 1 --2- 172.21.15.40:0/938442462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1200b2250 0x7fd12014c410 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:10.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.427+0000 7fd11f59e640 1 -- 172.21.15.40:0/938442462 >> 172.21.15.40:0/938442462 conn(0x7fd120069720 msgr2=0x7fd1200aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:10.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.427+0000 7fd11f59e640 1 -- 172.21.15.40:0/938442462 shutdown_connections 2023-10-18T08:58:10.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.427+0000 7fd11f59e640 1 -- 172.21.15.40:0/938442462 wait complete. 2023-10-18T08:58:10.448 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:57:12.765969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:57:13.851110+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:58:10.449 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:58:10.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.681+0000 7fac96d21640 1 -- 172.21.15.40:0/1699844577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac900c98a0 msgr2=0x7fac9006f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:10.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.681+0000 7fac96d21640 1 --2- 172.21.15.40:0/1699844577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac900c98a0 0x7fac9006f510 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fac8c005fd0 tx=0x7fac8c02d150 comp rx=0 tx=0).stop 2023-10-18T08:58:10.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.681+0000 7fac96d21640 1 -- 172.21.15.40:0/1699844577 shutdown_connections 2023-10-18T08:58:10.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.681+0000 7fac96d21640 1 --2- 172.21.15.40:0/1699844577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac900c98a0 0x7fac9006f510 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:10.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.681+0000 7fac96d21640 1 -- 172.21.15.40:0/1699844577 >> 172.21.15.40:0/1699844577 conn(0x7fac90069720 msgr2=0x7fac90069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:10.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.681+0000 7fac96d21640 1 -- 172.21.15.40:0/1699844577 shutdown_connections 2023-10-18T08:58:10.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.681+0000 7fac96d21640 1 -- 172.21.15.40:0/1699844577 wait complete. 2023-10-18T08:58:10.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.683+0000 7fac96d21640 1 Processor -- start 2023-10-18T08:58:10.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.683+0000 7fac96d21640 1 -- start start 2023-10-18T08:58:10.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.684+0000 7fac96d21640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac900b1900 0x7fac90138e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:10.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.684+0000 7fac96d21640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac900c98a0 0x7fac901379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:10.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.684+0000 7fac96d21640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac901405a0 0x7fac901579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:10.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.684+0000 7fac96d21640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac901f6660 con 0x7fac900c98a0 2023-10-18T08:58:10.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.684+0000 7fac96d21640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac901393b0 con 0x7fac901405a0 2023-10-18T08:58:10.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.684+0000 7fac96d21640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac901f6380 con 0x7fac900b1900 2023-10-18T08:58:10.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.684+0000 7fac9551e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac900c98a0 0x7fac901379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:10.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.685+0000 7fac9551e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac900c98a0 0x7fac901379b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49762/0 (socket says 172.21.15.40:49762) 2023-10-18T08:58:10.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.685+0000 7fac9551e640 1 -- 172.21.15.40:0/1662913260 learned_addr learned my addr 172.21.15.40:0/1662913260 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:10.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.685+0000 7fac95d1f640 1 --2- 172.21.15.40:0/1662913260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac900b1900 0x7fac90138e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:10.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.685+0000 7fac9551e640 1 -- 172.21.15.40:0/1662913260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac900b1900 msgr2=0x7fac90138e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:10.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.685+0000 7fac96520640 1 --2- 172.21.15.40:0/1662913260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac901405a0 0x7fac901579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:10.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.685+0000 7fac9551e640 1 --2- 172.21.15.40:0/1662913260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac900b1900 0x7fac90138e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:10.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.685+0000 7fac9551e640 1 -- 172.21.15.40:0/1662913260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac901405a0 msgr2=0x7fac901579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:10.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.685+0000 7fac9551e640 1 --2- 172.21.15.40:0/1662913260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac901405a0 0x7fac901579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:10.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.685+0000 7fac9551e640 1 -- 172.21.15.40:0/1662913260 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac8c00a670 con 0x7fac900c98a0 2023-10-18T08:58:10.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.685+0000 7fac95d1f640 1 --2- 172.21.15.40:0/1662913260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac900b1900 0x7fac90138e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:10.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.685+0000 7fac9551e640 1 --2- 172.21.15.40:0/1662913260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac900c98a0 0x7fac901379b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fac8000c400 tx=0x7fac8000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:10.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.686+0000 7fac867fc640 1 -- 172.21.15.40:0/1662913260 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80011790 con 0x7fac900c98a0 2023-10-18T08:58:10.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.686+0000 7fac867fc640 1 -- 172.21.15.40:0/1662913260 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fac80011a20 con 0x7fac900c98a0 2023-10-18T08:58:10.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.686+0000 7fac867fc640 1 -- 172.21.15.40:0/1662913260 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80011e30 con 0x7fac900c98a0 2023-10-18T08:58:10.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.686+0000 7fac96d21640 1 -- 172.21.15.40:0/1662913260 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac900aaa20 con 0x7fac900c98a0 2023-10-18T08:58:10.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.686+0000 7fac96d21640 1 -- 172.21.15.40:0/1662913260 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac900aafe0 con 0x7fac900c98a0 2023-10-18T08:58:10.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.687+0000 7fac96d21640 1 -- 172.21.15.40:0/1662913260 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac900aa880 con 0x7fac900c98a0 2023-10-18T08:58:10.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.688+0000 7fac867fc640 1 -- 172.21.15.40:0/1662913260 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fac8001c460 con 0x7fac900c98a0 2023-10-18T08:58:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.689+0000 7fac867fc640 1 --2- 172.21.15.40:0/1662913260 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac7406ae40 0x7fac7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.689+0000 7fac867fc640 1 -- 172.21.15.40:0/1662913260 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fac80089930 con 0x7fac900c98a0 2023-10-18T08:58:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.690+0000 7fac95d1f640 1 --2- 172.21.15.40:0/1662913260 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac7406ae40 0x7fac7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.691+0000 7fac95d1f640 1 --2- 172.21.15.40:0/1662913260 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac7406ae40 0x7fac7406d300 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fac8c036000 tx=0x7fac8c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:10.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.693+0000 7fac867fc640 1 -- 172.21.15.40:0/1662913260 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac900aa880 con 0x7fac900c98a0 2023-10-18T08:58:10.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.883+0000 7fac96d21640 1 -- 172.21.15.40:0/1662913260 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac900aa880 con 0x7fac900c98a0 2023-10-18T08:58:10.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.884+0000 7fac867fc640 1 -- 172.21.15.40:0/1662913260 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac900aa880 con 0x7fac900c98a0 2023-10-18T08:58:10.888 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:58:10.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.887+0000 7fac96d21640 1 -- 172.21.15.40:0/1662913260 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac7406ae40 msgr2=0x7fac7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:10.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.887+0000 7fac96d21640 1 --2- 172.21.15.40:0/1662913260 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac7406ae40 0x7fac7406d300 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fac8c036000 tx=0x7fac8c004730 comp rx=0 tx=0).stop 2023-10-18T08:58:10.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.888+0000 7fac96d21640 1 -- 172.21.15.40:0/1662913260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac900c98a0 msgr2=0x7fac901379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:10.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.888+0000 7fac96d21640 1 --2- 172.21.15.40:0/1662913260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac900c98a0 0x7fac901379b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fac8000c400 tx=0x7fac8000c8c0 comp rx=0 tx=0).stop 2023-10-18T08:58:10.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.888+0000 7fac96d21640 1 -- 172.21.15.40:0/1662913260 shutdown_connections 2023-10-18T08:58:10.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.888+0000 7fac96d21640 1 --2- 172.21.15.40:0/1662913260 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac7406ae40 0x7fac7406d300 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:10.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.888+0000 7fac96d21640 1 --2- 172.21.15.40:0/1662913260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac901405a0 0x7fac901579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:10.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.888+0000 7fac96d21640 1 --2- 172.21.15.40:0/1662913260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac900c98a0 0x7fac901379b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:10.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.888+0000 7fac96d21640 1 --2- 172.21.15.40:0/1662913260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac900b1900 0x7fac90138e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:10.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.888+0000 7fac96d21640 1 -- 172.21.15.40:0/1662913260 >> 172.21.15.40:0/1662913260 conn(0x7fac90069720 msgr2=0x7fac900aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:10.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.888+0000 7fac96d21640 1 -- 172.21.15.40:0/1662913260 shutdown_connections 2023-10-18T08:58:10.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:10.888+0000 7fac96d21640 1 -- 172.21.15.40:0/1662913260 wait complete. 2023-10-18T08:58:10.909 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:58:11.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.142+0000 7feb5235e640 1 -- 172.21.15.40:0/2606084346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb4c0ae5b0 msgr2=0x7feb4c0cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:11.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.142+0000 7feb5235e640 1 --2- 172.21.15.40:0/2606084346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb4c0ae5b0 0x7feb4c0cda00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7feb3c005fd0 tx=0x7feb3c02bd80 comp rx=0 tx=0).stop 2023-10-18T08:58:11.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.142+0000 7feb5235e640 1 -- 172.21.15.40:0/2606084346 shutdown_connections 2023-10-18T08:58:11.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.142+0000 7feb5235e640 1 --2- 172.21.15.40:0/2606084346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb4c0ae5b0 0x7feb4c0cda00 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:11.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.142+0000 7feb5235e640 1 --2- 172.21.15.40:0/2606084346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb4c1594b0 0x7feb4c0cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:11.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.142+0000 7feb5235e640 1 -- 172.21.15.40:0/2606084346 >> 172.21.15.40:0/2606084346 conn(0x7feb4c0b0ad0 msgr2=0x7feb4c0b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:11.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.142+0000 7feb5235e640 1 -- 172.21.15.40:0/2606084346 shutdown_connections 2023-10-18T08:58:11.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.142+0000 7feb5235e640 1 -- 172.21.15.40:0/2606084346 wait complete. 2023-10-18T08:58:11.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.144+0000 7feb5235e640 1 Processor -- start 2023-10-18T08:58:11.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.144+0000 7feb5235e640 1 -- start start 2023-10-18T08:58:11.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.145+0000 7feb5235e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb4c0ae5b0 0x7feb4c09c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:11.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.145+0000 7feb5235e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb4c0ccdc0 0x7feb4c1e3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:11.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.145+0000 7feb5235e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb4c1594b0 0x7feb4c0c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:11.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.145+0000 7feb5235e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c09c510 con 0x7feb4c1594b0 2023-10-18T08:58:11.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.145+0000 7feb5235e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0b48d0 con 0x7feb4c0ccdc0 2023-10-18T08:58:11.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.145+0000 7feb5235e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0b4b90 con 0x7feb4c0ae5b0 2023-10-18T08:58:11.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.145+0000 7feb51b5d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb4c1594b0 0x7feb4c0c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:11.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.145+0000 7feb51b5d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb4c1594b0 0x7feb4c0c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49778/0 (socket says 172.21.15.40:49778) 2023-10-18T08:58:11.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.145+0000 7feb51b5d640 1 -- 172.21.15.40:0/2895426310 learned_addr learned my addr 172.21.15.40:0/2895426310 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:11.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.145+0000 7feb5135c640 1 --2- 172.21.15.40:0/2895426310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb4c0ae5b0 0x7feb4c09c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:11.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.145+0000 7feb50b5b640 1 --2- 172.21.15.40:0/2895426310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb4c0ccdc0 0x7feb4c1e3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:11.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.145+0000 7feb51b5d640 1 -- 172.21.15.40:0/2895426310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb4c0ae5b0 msgr2=0x7feb4c09c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:11.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.145+0000 7feb51b5d640 1 --2- 172.21.15.40:0/2895426310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb4c0ae5b0 0x7feb4c09c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:11.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.145+0000 7feb51b5d640 1 -- 172.21.15.40:0/2895426310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb4c0ccdc0 msgr2=0x7feb4c1e3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:11.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.145+0000 7feb51b5d640 1 --2- 172.21.15.40:0/2895426310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb4c0ccdc0 0x7feb4c1e3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:11.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.146+0000 7feb51b5d640 1 -- 172.21.15.40:0/2895426310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c009560 con 0x7feb4c1594b0 2023-10-18T08:58:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.146+0000 7feb51b5d640 1 --2- 172.21.15.40:0/2895426310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb4c1594b0 0x7feb4c0c8150 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7feb4400c7a0 tx=0x7feb4400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:11.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.146+0000 7feb41ffb640 1 -- 172.21.15.40:0/2895426310 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44003b70 con 0x7feb4c1594b0 2023-10-18T08:58:11.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.146+0000 7feb41ffb640 1 -- 172.21.15.40:0/2895426310 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feb44003cf0 con 0x7feb4c1594b0 2023-10-18T08:58:11.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.146+0000 7feb41ffb640 1 -- 172.21.15.40:0/2895426310 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4400aab0 con 0x7feb4c1594b0 2023-10-18T08:58:11.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.146+0000 7feb5235e640 1 -- 172.21.15.40:0/2895426310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4c09c1a0 con 0x7feb4c1594b0 2023-10-18T08:58:11.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.146+0000 7feb5235e640 1 -- 172.21.15.40:0/2895426310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4c0b45c0 con 0x7feb4c1594b0 2023-10-18T08:58:11.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.148+0000 7feb5235e640 1 -- 172.21.15.40:0/2895426310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4c09c680 con 0x7feb4c1594b0 2023-10-18T08:58:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.149+0000 7feb41ffb640 1 -- 172.21.15.40:0/2895426310 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feb4400ac10 con 0x7feb4c1594b0 2023-10-18T08:58:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.149+0000 7feb41ffb640 1 --2- 172.21.15.40:0/2895426310 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb3006ae40 0x7feb3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.150+0000 7feb41ffb640 1 -- 172.21.15.40:0/2895426310 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feb440895e0 con 0x7feb4c1594b0 2023-10-18T08:58:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.150+0000 7feb5135c640 1 --2- 172.21.15.40:0/2895426310 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb3006ae40 0x7feb3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.151+0000 7feb5135c640 1 --2- 172.21.15.40:0/2895426310 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb3006ae40 0x7feb3006d300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7feb48005d70 tx=0x7feb48005d00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.154+0000 7feb41ffb640 1 -- 172.21.15.40:0/2895426310 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb440546e0 con 0x7feb4c1594b0 2023-10-18T08:58:11.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.348+0000 7feb5235e640 1 -- 172.21.15.40:0/2895426310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb4c073570 con 0x7feb4c1594b0 2023-10-18T08:58:11.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.349+0000 7feb41ffb640 1 -- 172.21.15.40:0/2895426310 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb44057d00 con 0x7feb4c1594b0 2023-10-18T08:58:11.352 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:58:11.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.352+0000 7feb5235e640 1 -- 172.21.15.40:0/2895426310 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb3006ae40 msgr2=0x7feb3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:11.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.352+0000 7feb5235e640 1 --2- 172.21.15.40:0/2895426310 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb3006ae40 0x7feb3006d300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7feb48005d70 tx=0x7feb48005d00 comp rx=0 tx=0).stop 2023-10-18T08:58:11.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.352+0000 7feb5235e640 1 -- 172.21.15.40:0/2895426310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb4c1594b0 msgr2=0x7feb4c0c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:11.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.352+0000 7feb5235e640 1 --2- 172.21.15.40:0/2895426310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb4c1594b0 0x7feb4c0c8150 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7feb4400c7a0 tx=0x7feb4400cc60 comp rx=0 tx=0).stop 2023-10-18T08:58:11.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.352+0000 7feb5235e640 1 -- 172.21.15.40:0/2895426310 shutdown_connections 2023-10-18T08:58:11.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.352+0000 7feb5235e640 1 --2- 172.21.15.40:0/2895426310 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb3006ae40 0x7feb3006d300 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:11.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.352+0000 7feb5235e640 1 --2- 172.21.15.40:0/2895426310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb4c1594b0 0x7feb4c0c8150 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:11.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.352+0000 7feb5235e640 1 --2- 172.21.15.40:0/2895426310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb4c0ccdc0 0x7feb4c1e3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:11.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.352+0000 7feb5235e640 1 --2- 172.21.15.40:0/2895426310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb4c0ae5b0 0x7feb4c09c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:11.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.352+0000 7feb5235e640 1 -- 172.21.15.40:0/2895426310 >> 172.21.15.40:0/2895426310 conn(0x7feb4c0b0ad0 msgr2=0x7feb4c0b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:11.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.353+0000 7feb5235e640 1 -- 172.21.15.40:0/2895426310 shutdown_connections 2023-10-18T08:58:11.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.353+0000 7feb5235e640 1 -- 172.21.15.40:0/2895426310 wait complete. 2023-10-18T08:58:11.375 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:58:11.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.588+0000 7fbc4dd5e640 1 -- 172.21.15.40:0/1223667507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc4813c550 msgr2=0x7fbc4813ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:11.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.588+0000 7fbc4dd5e640 1 --2- 172.21.15.40:0/1223667507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc4813c550 0x7fbc4813ec50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009830 tx=0x7fbc3802bd60 comp rx=0 tx=0).stop 2023-10-18T08:58:11.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.589+0000 7fbc4dd5e640 1 -- 172.21.15.40:0/1223667507 shutdown_connections 2023-10-18T08:58:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.589+0000 7fbc4dd5e640 1 --2- 172.21.15.40:0/1223667507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc4813c550 0x7fbc4813ec50 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.589+0000 7fbc4dd5e640 1 -- 172.21.15.40:0/1223667507 >> 172.21.15.40:0/1223667507 conn(0x7fbc48069720 msgr2=0x7fbc48069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.589+0000 7fbc4dd5e640 1 -- 172.21.15.40:0/1223667507 shutdown_connections 2023-10-18T08:58:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.589+0000 7fbc4dd5e640 1 -- 172.21.15.40:0/1223667507 wait complete. 2023-10-18T08:58:11.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.590+0000 7fbc4dd5e640 1 Processor -- start 2023-10-18T08:58:11.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.591+0000 7fbc4dd5e640 1 -- start start 2023-10-18T08:58:11.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.592+0000 7fbc4dd5e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc48071410 0x7fbc48139a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:11.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.592+0000 7fbc4dd5e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc4813c550 0x7fbc480aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:11.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.592+0000 7fbc4dd5e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc4813f190 0x7fbc48158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:11.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.592+0000 7fbc4dd5e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc48155080 con 0x7fbc4813f190 2023-10-18T08:58:11.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.592+0000 7fbc4dd5e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc48155210 con 0x7fbc48071410 2023-10-18T08:58:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.592+0000 7fbc4dd5e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc48139fb0 con 0x7fbc4813c550 2023-10-18T08:58:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.592+0000 7fbc47fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc4813f190 0x7fbc48158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.592+0000 7fbc47fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc4813f190 0x7fbc48158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49792/0 (socket says 172.21.15.40:49792) 2023-10-18T08:58:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.592+0000 7fbc47fff640 1 -- 172.21.15.40:0/3381577073 learned_addr learned my addr 172.21.15.40:0/3381577073 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:11.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.592+0000 7fbc477fe640 1 --2- 172.21.15.40:0/3381577073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc48071410 0x7fbc48139a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:11.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.592+0000 7fbc47fff640 1 -- 172.21.15.40:0/3381577073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc4813c550 msgr2=0x7fbc480aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:11.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.592+0000 7fbc46ffd640 1 --2- 172.21.15.40:0/3381577073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc4813c550 0x7fbc480aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:11.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.593+0000 7fbc47fff640 1 --2- 172.21.15.40:0/3381577073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc4813c550 0x7fbc480aa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:11.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.593+0000 7fbc47fff640 1 -- 172.21.15.40:0/3381577073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc48071410 msgr2=0x7fbc48139a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:11.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.593+0000 7fbc47fff640 1 --2- 172.21.15.40:0/3381577073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc48071410 0x7fbc48139a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:11.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.593+0000 7fbc47fff640 1 -- 172.21.15.40:0/3381577073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc38009490 con 0x7fbc4813f190 2023-10-18T08:58:11.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.593+0000 7fbc47fff640 1 --2- 172.21.15.40:0/3381577073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc4813f190 0x7fbc48158710 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000c780 tx=0x7fbc4000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:11.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.593+0000 7fbc2ffff640 1 -- 172.21.15.40:0/3381577073 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40004b60 con 0x7fbc4813f190 2023-10-18T08:58:11.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.593+0000 7fbc2ffff640 1 -- 172.21.15.40:0/3381577073 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbc4000ce90 con 0x7fbc4813f190 2023-10-18T08:58:11.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.593+0000 7fbc2ffff640 1 -- 172.21.15.40:0/3381577073 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4000aab0 con 0x7fbc4813f190 2023-10-18T08:58:11.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.593+0000 7fbc4dd5e640 1 -- 172.21.15.40:0/3381577073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc480ab1f0 con 0x7fbc4813f190 2023-10-18T08:58:11.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.593+0000 7fbc4dd5e640 1 -- 172.21.15.40:0/3381577073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc480ab7b0 con 0x7fbc4813f190 2023-10-18T08:58:11.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.594+0000 7fbc4dd5e640 1 -- 172.21.15.40:0/3381577073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc481553a0 con 0x7fbc4813f190 2023-10-18T08:58:11.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.595+0000 7fbc2ffff640 1 -- 172.21.15.40:0/3381577073 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbc4000ac10 con 0x7fbc4813f190 2023-10-18T08:58:11.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.596+0000 7fbc2ffff640 1 --2- 172.21.15.40:0/3381577073 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc3406ae40 0x7fbc3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:11.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.596+0000 7fbc2ffff640 1 -- 172.21.15.40:0/3381577073 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbc40089b90 con 0x7fbc4813f190 2023-10-18T08:58:11.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.597+0000 7fbc477fe640 1 --2- 172.21.15.40:0/3381577073 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc3406ae40 0x7fbc3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:11.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.598+0000 7fbc477fe640 1 --2- 172.21.15.40:0/3381577073 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc3406ae40 0x7fbc3406d300 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbc38036000 tx=0x7fbc38004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:11.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.600+0000 7fbc2ffff640 1 -- 172.21.15.40:0/3381577073 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc40054b60 con 0x7fbc4813f190 2023-10-18T08:58:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.790+0000 7fbc4dd5e640 1 -- 172.21.15.40:0/3381577073 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbc48066fd0 con 0x7fbc4813f190 2023-10-18T08:58:11.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.790+0000 7fbc2ffff640 1 -- 172.21.15.40:0/3381577073 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbc40058180 con 0x7fbc4813f190 2023-10-18T08:58:11.794 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:58:11.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.793+0000 7fbc4dd5e640 1 -- 172.21.15.40:0/3381577073 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc3406ae40 msgr2=0x7fbc3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:11.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.793+0000 7fbc4dd5e640 1 --2- 172.21.15.40:0/3381577073 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc3406ae40 0x7fbc3406d300 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbc38036000 tx=0x7fbc38004730 comp rx=0 tx=0).stop 2023-10-18T08:58:11.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.793+0000 7fbc4dd5e640 1 -- 172.21.15.40:0/3381577073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc4813f190 msgr2=0x7fbc48158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:11.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.793+0000 7fbc4dd5e640 1 --2- 172.21.15.40:0/3381577073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc4813f190 0x7fbc48158710 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000c780 tx=0x7fbc4000cc40 comp rx=0 tx=0).stop 2023-10-18T08:58:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.794+0000 7fbc4dd5e640 1 -- 172.21.15.40:0/3381577073 shutdown_connections 2023-10-18T08:58:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.794+0000 7fbc4dd5e640 1 --2- 172.21.15.40:0/3381577073 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc3406ae40 0x7fbc3406d300 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.794+0000 7fbc4dd5e640 1 --2- 172.21.15.40:0/3381577073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc4813f190 0x7fbc48158710 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.794+0000 7fbc4dd5e640 1 --2- 172.21.15.40:0/3381577073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc4813c550 0x7fbc480aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:11.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.794+0000 7fbc4dd5e640 1 --2- 172.21.15.40:0/3381577073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc48071410 0x7fbc48139a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:11.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.794+0000 7fbc4dd5e640 1 -- 172.21.15.40:0/3381577073 >> 172.21.15.40:0/3381577073 conn(0x7fbc48069720 msgr2=0x7fbc4813b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:11.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.794+0000 7fbc4dd5e640 1 -- 172.21.15.40:0/3381577073 shutdown_connections 2023-10-18T08:58:11.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:11.794+0000 7fbc4dd5e640 1 -- 172.21.15.40:0/3381577073 wait complete. 2023-10-18T08:58:11.815 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:58:12.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.046+0000 7f43aa807640 1 -- 172.21.15.40:0/852615064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43a41586a0 msgr2=0x7f43a40b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:12.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.046+0000 7f43aa807640 1 --2- 172.21.15.40:0/852615064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43a41586a0 0x7f43a40b0bc0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f43a406a6d0 tx=0x7f439802bda0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.047+0000 7f43aa807640 1 -- 172.21.15.40:0/852615064 shutdown_connections 2023-10-18T08:58:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.047+0000 7f43aa807640 1 --2- 172.21.15.40:0/852615064 >> v2:172.21.15.144:3300/0 conn(0x7f43a40b1100 0x7f43a41625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.047+0000 7f43aa807640 1 --2- 172.21.15.40:0/852615064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43a41586a0 0x7f43a40b0bc0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.047+0000 7f43aa807640 1 -- 172.21.15.40:0/852615064 >> 172.21.15.40:0/852615064 conn(0x7f43a406cee0 msgr2=0x7f43a4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:12.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.047+0000 7f43aa807640 1 -- 172.21.15.40:0/852615064 shutdown_connections 2023-10-18T08:58:12.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.047+0000 7f43aa807640 1 -- 172.21.15.40:0/852615064 wait complete. 2023-10-18T08:58:12.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.048+0000 7f43aa807640 1 Processor -- start 2023-10-18T08:58:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.049+0000 7f43aa807640 1 -- start start 2023-10-18T08:58:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.049+0000 7f43aa807640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43a406d710 0x7f43a413bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.050+0000 7f43aa807640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a40b1100 0x7f43a41569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:12.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.050+0000 7f43aa807640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43a41586a0 0x7f43a40ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:12.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.050+0000 7f43aa807640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a413c000 con 0x7f43a41586a0 2023-10-18T08:58:12.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.050+0000 7f43aa807640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40ad420 con 0x7f43a406d710 2023-10-18T08:58:12.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.050+0000 7f43aa807640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40ad700 con 0x7f43a40b1100 2023-10-18T08:58:12.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.050+0000 7f43a8da4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43a41586a0 0x7f43a40ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:12.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.050+0000 7f43a8da4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43a41586a0 0x7f43a40ae4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49806/0 (socket says 172.21.15.40:49806) 2023-10-18T08:58:12.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.050+0000 7f43a8da4640 1 -- 172.21.15.40:0/1354736263 learned_addr learned my addr 172.21.15.40:0/1354736263 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:12.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.050+0000 7f43a3fff640 1 --2- 172.21.15.40:0/1354736263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43a406d710 0x7f43a413bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:12.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.050+0000 7f43a37fe640 1 --2- 172.21.15.40:0/1354736263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a40b1100 0x7f43a41569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:12.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.050+0000 7f43a8da4640 1 -- 172.21.15.40:0/1354736263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a40b1100 msgr2=0x7f43a41569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:12.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.050+0000 7f43a8da4640 1 --2- 172.21.15.40:0/1354736263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a40b1100 0x7f43a41569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.050+0000 7f43a8da4640 1 -- 172.21.15.40:0/1354736263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43a406d710 msgr2=0x7f43a413bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:12.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.050+0000 7f43a8da4640 1 --2- 172.21.15.40:0/1354736263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43a406d710 0x7f43a413bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.050+0000 7f43a8da4640 1 -- 172.21.15.40:0/1354736263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4398009560 con 0x7f43a41586a0 2023-10-18T08:58:12.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.050+0000 7f43a37fe640 1 --2- 172.21.15.40:0/1354736263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a40b1100 0x7f43a41569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:12.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.051+0000 7f43a8da4640 1 --2- 172.21.15.40:0/1354736263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43a41586a0 0x7f43a40ae4c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f439c009fd0 tx=0x7f439c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:12.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.051+0000 7f43a0ff9640 1 -- 172.21.15.40:0/1354736263 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c011830 con 0x7f43a41586a0 2023-10-18T08:58:12.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.051+0000 7f43a0ff9640 1 -- 172.21.15.40:0/1354736263 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f439c011a80 con 0x7f43a41586a0 2023-10-18T08:58:12.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.051+0000 7f43a0ff9640 1 -- 172.21.15.40:0/1354736263 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c011e70 con 0x7f43a41586a0 2023-10-18T08:58:12.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.051+0000 7f43aa807640 1 -- 172.21.15.40:0/1354736263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a40ad9e0 con 0x7f43a41586a0 2023-10-18T08:58:12.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.051+0000 7f43aa807640 1 -- 172.21.15.40:0/1354736263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a40adfa0 con 0x7f43a41586a0 2023-10-18T08:58:12.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.052+0000 7f43aa807640 1 -- 172.21.15.40:0/1354736263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a415c0a0 con 0x7f43a41586a0 2023-10-18T08:58:12.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.053+0000 7f43a0ff9640 1 -- 172.21.15.40:0/1354736263 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f439c01b460 con 0x7f43a41586a0 2023-10-18T08:58:12.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.054+0000 7f43a0ff9640 1 --2- 172.21.15.40:0/1354736263 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f439006ae40 0x7f439006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:12.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.054+0000 7f43a0ff9640 1 -- 172.21.15.40:0/1354736263 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f439c0897f0 con 0x7f43a41586a0 2023-10-18T08:58:12.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.055+0000 7f43a3fff640 1 --2- 172.21.15.40:0/1354736263 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f439006ae40 0x7f439006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:12.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.056+0000 7f43a3fff640 1 --2- 172.21.15.40:0/1354736263 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f439006ae40 0x7f439006d300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4394007fd0 tx=0x7f4394009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:12.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.059+0000 7f43a0ff9640 1 -- 172.21.15.40:0/1354736263 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f43a415c0a0 con 0x7f43a41586a0 2023-10-18T08:58:12.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.289+0000 7f43aa807640 1 -- 172.21.15.40:0/1354736263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43a415c0a0 con 0x7f43a41586a0 2023-10-18T08:58:12.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.291+0000 7f43a0ff9640 1 -- 172.21.15.40:0/1354736263 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f43a415c0a0 con 0x7f43a41586a0 2023-10-18T08:58:12.294 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:12.294 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3668356540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3668356540},{"type":"v1","addr":"172.21.15.144:6836","nonce":3668356540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4222679844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4222679844},{"type":"v1","addr":"172.21.15.40:6834","nonce":4222679844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2829399368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2829399368},{"type":"v1","addr":"172.21.15.40:6835","nonce":2829399368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:57:14.826066+0000","modified":"2023-10-18T08:57:17.903233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1438456692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1438456692},{"type":"v1","addr":"172.21.15.144:6837","nonce":1438456692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-18T08:58:12.295 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 24 2023-10-18T08:58:12.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.293+0000 7f43aa807640 1 -- 172.21.15.40:0/1354736263 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f439006ae40 msgr2=0x7f439006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:12.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.293+0000 7f43aa807640 1 --2- 172.21.15.40:0/1354736263 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f439006ae40 0x7f439006d300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4394007fd0 tx=0x7f4394009040 comp rx=0 tx=0).stop 2023-10-18T08:58:12.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.293+0000 7f43aa807640 1 -- 172.21.15.40:0/1354736263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43a41586a0 msgr2=0x7f43a40ae4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:12.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.293+0000 7f43aa807640 1 --2- 172.21.15.40:0/1354736263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43a41586a0 0x7f43a40ae4c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f439c009fd0 tx=0x7f439c00b8e0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.293+0000 7f43aa807640 1 -- 172.21.15.40:0/1354736263 shutdown_connections 2023-10-18T08:58:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.293+0000 7f43aa807640 1 --2- 172.21.15.40:0/1354736263 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f439006ae40 0x7f439006d300 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.293+0000 7f43aa807640 1 --2- 172.21.15.40:0/1354736263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43a41586a0 0x7f43a40ae4c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.293+0000 7f43aa807640 1 --2- 172.21.15.40:0/1354736263 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43a40b1100 0x7f43a41569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.293+0000 7f43aa807640 1 --2- 172.21.15.40:0/1354736263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43a406d710 0x7f43a413bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.293+0000 7f43aa807640 1 -- 172.21.15.40:0/1354736263 >> 172.21.15.40:0/1354736263 conn(0x7f43a406cee0 msgr2=0x7f43a4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.293+0000 7f43aa807640 1 -- 172.21.15.40:0/1354736263 shutdown_connections 2023-10-18T08:58:12.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.293+0000 7f43aa807640 1 -- 172.21.15.40:0/1354736263 wait complete. 2023-10-18T08:58:12.316 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:58:12.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.543+0000 7fcd61010640 1 -- 172.21.15.40:0/3727635284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd5c0cc990 msgr2=0x7fcd5c0ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:12.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.543+0000 7fcd61010640 1 --2- 172.21.15.40:0/3727635284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd5c0cc990 0x7fcd5c0ccde0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fcd54005fd0 tx=0x7fcd5402c0f0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.544+0000 7fcd61010640 1 -- 172.21.15.40:0/3727635284 shutdown_connections 2023-10-18T08:58:12.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.544+0000 7fcd61010640 1 --2- 172.21.15.40:0/3727635284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd5c0cc990 0x7fcd5c0ccde0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.544+0000 7fcd61010640 1 --2- 172.21.15.40:0/3727635284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd5c0b0bc0 0x7fcd5c0b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.544+0000 7fcd61010640 1 -- 172.21.15.40:0/3727635284 >> 172.21.15.40:0/3727635284 conn(0x7fcd5c06cee0 msgr2=0x7fcd5c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:12.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.544+0000 7fcd61010640 1 -- 172.21.15.40:0/3727635284 shutdown_connections 2023-10-18T08:58:12.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.544+0000 7fcd61010640 1 -- 172.21.15.40:0/3727635284 wait complete. 2023-10-18T08:58:12.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.546+0000 7fcd61010640 1 Processor -- start 2023-10-18T08:58:12.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.546+0000 7fcd61010640 1 -- start start 2023-10-18T08:58:12.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.547+0000 7fcd61010640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd5c0b0bc0 0x7fcd5c147dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:12.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.547+0000 7fcd61010640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd5c0cc990 0x7fcd5c13f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:12.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.547+0000 7fcd61010640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd5c151bd0 0x7fcd5c14bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:12.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.547+0000 7fcd61010640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c14ba60 con 0x7fcd5c151bd0 2023-10-18T08:58:12.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.547+0000 7fcd61010640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c14ae50 con 0x7fcd5c0cc990 2023-10-18T08:58:12.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.547+0000 7fcd61010640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c14b130 con 0x7fcd5c0b0bc0 2023-10-18T08:58:12.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.547+0000 7fcd5b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd5c151bd0 0x7fcd5c14bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:12.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.547+0000 7fcd5b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd5c151bd0 0x7fcd5c14bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49828/0 (socket says 172.21.15.40:49828) 2023-10-18T08:58:12.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.547+0000 7fcd5b59e640 1 -- 172.21.15.40:0/1653504218 learned_addr learned my addr 172.21.15.40:0/1653504218 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:12.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.547+0000 7fcd5ad9d640 1 --2- 172.21.15.40:0/1653504218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd5c0b0bc0 0x7fcd5c147dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:12.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.548+0000 7fcd5259c640 1 --2- 172.21.15.40:0/1653504218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd5c0cc990 0x7fcd5c13f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:12.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.548+0000 7fcd5b59e640 1 -- 172.21.15.40:0/1653504218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd5c0b0bc0 msgr2=0x7fcd5c147dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:12.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.548+0000 7fcd5b59e640 1 --2- 172.21.15.40:0/1653504218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd5c0b0bc0 0x7fcd5c147dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.548+0000 7fcd5b59e640 1 -- 172.21.15.40:0/1653504218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd5c0cc990 msgr2=0x7fcd5c13f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:12.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.548+0000 7fcd5b59e640 1 --2- 172.21.15.40:0/1653504218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd5c0cc990 0x7fcd5c13f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.548+0000 7fcd5b59e640 1 -- 172.21.15.40:0/1653504218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd54009840 con 0x7fcd5c151bd0 2023-10-18T08:58:12.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.548+0000 7fcd5259c640 1 --2- 172.21.15.40:0/1653504218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd5c0cc990 0x7fcd5c13f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:12.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.548+0000 7fcd5b59e640 1 --2- 172.21.15.40:0/1653504218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd5c151bd0 0x7fcd5c14bef0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcd54036040 tx=0x7fcd54004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:12.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.548+0000 7fcd53fff640 1 -- 172.21.15.40:0/1653504218 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd54004730 con 0x7fcd5c151bd0 2023-10-18T08:58:12.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.548+0000 7fcd53fff640 1 -- 172.21.15.40:0/1653504218 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcd540345a0 con 0x7fcd5c151bd0 2023-10-18T08:58:12.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.548+0000 7fcd53fff640 1 -- 172.21.15.40:0/1653504218 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd540349b0 con 0x7fcd5c151bd0 2023-10-18T08:58:12.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.548+0000 7fcd61010640 1 -- 172.21.15.40:0/1653504218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5c14b410 con 0x7fcd5c151bd0 2023-10-18T08:58:12.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.549+0000 7fcd61010640 1 -- 172.21.15.40:0/1653504218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5c14c430 con 0x7fcd5c151bd0 2023-10-18T08:58:12.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.550+0000 7fcd61010640 1 -- 172.21.15.40:0/1653504218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5c14bbd0 con 0x7fcd5c151bd0 2023-10-18T08:58:12.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.551+0000 7fcd53fff640 1 -- 172.21.15.40:0/1653504218 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcd54034b10 con 0x7fcd5c151bd0 2023-10-18T08:58:12.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.552+0000 7fcd53fff640 1 --2- 172.21.15.40:0/1653504218 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcd3c06ae70 0x7fcd3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:12.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.552+0000 7fcd53fff640 1 -- 172.21.15.40:0/1653504218 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcd540ac9e0 con 0x7fcd5c151bd0 2023-10-18T08:58:12.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.553+0000 7fcd5ad9d640 1 --2- 172.21.15.40:0/1653504218 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcd3c06ae70 0x7fcd3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:12.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.554+0000 7fcd5ad9d640 1 --2- 172.21.15.40:0/1653504218 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcd3c06ae70 0x7fcd3c06d330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c007fd0 tx=0x7fcd4c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:12.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.556+0000 7fcd53fff640 1 -- 172.21.15.40:0/1653504218 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd540779b0 con 0x7fcd5c151bd0 2023-10-18T08:58:12.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.748+0000 7fcd61010640 1 -- 172.21.15.40:0/1653504218 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd5c066fd0 con 0x7fcd5c151bd0 2023-10-18T08:58:12.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.751+0000 7fcd53fff640 1 -- 172.21.15.40:0/1653504218 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16620 (secure 0 0 0) 0x7fcd54043090 con 0x7fcd5c151bd0 2023-10-18T08:58:12.755 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:12.755 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":56,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:58:02.917888+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:57:12.765969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T08:57:13.851110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":5}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:58:12.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.754+0000 7fcd61010640 1 -- 172.21.15.40:0/1653504218 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcd3c06ae70 msgr2=0x7fcd3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:12.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.754+0000 7fcd61010640 1 --2- 172.21.15.40:0/1653504218 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcd3c06ae70 0x7fcd3c06d330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c007fd0 tx=0x7fcd4c009040 comp rx=0 tx=0).stop 2023-10-18T08:58:12.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.754+0000 7fcd61010640 1 -- 172.21.15.40:0/1653504218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd5c151bd0 msgr2=0x7fcd5c14bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:12.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.754+0000 7fcd61010640 1 --2- 172.21.15.40:0/1653504218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd5c151bd0 0x7fcd5c14bef0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcd54036040 tx=0x7fcd54004270 comp rx=0 tx=0).stop 2023-10-18T08:58:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.755+0000 7fcd61010640 1 -- 172.21.15.40:0/1653504218 shutdown_connections 2023-10-18T08:58:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.755+0000 7fcd61010640 1 --2- 172.21.15.40:0/1653504218 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcd3c06ae70 0x7fcd3c06d330 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.755+0000 7fcd61010640 1 --2- 172.21.15.40:0/1653504218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd5c151bd0 0x7fcd5c14bef0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.755+0000 7fcd61010640 1 --2- 172.21.15.40:0/1653504218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd5c0cc990 0x7fcd5c13f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.755+0000 7fcd61010640 1 --2- 172.21.15.40:0/1653504218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd5c0b0bc0 0x7fcd5c147dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:12.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.755+0000 7fcd61010640 1 -- 172.21.15.40:0/1653504218 >> 172.21.15.40:0/1653504218 conn(0x7fcd5c06cee0 msgr2=0x7fcd5c1488a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:12.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.755+0000 7fcd61010640 1 -- 172.21.15.40:0/1653504218 shutdown_connections 2023-10-18T08:58:12.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.755+0000 7fcd61010640 1 -- 172.21.15.40:0/1653504218 wait complete. 2023-10-18T08:58:12.779 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T08:58:12.779 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:58:13.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.997+0000 7f92970d7640 1 -- 172.21.15.40:0/1058764301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92900b15b0 msgr2=0x7f92900b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:13.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.997+0000 7f92970d7640 1 --2- 172.21.15.40:0/1058764301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92900b15b0 0x7f92900b0630 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f928c005fd0 tx=0x7f928c02be20 comp rx=0 tx=0).stop 2023-10-18T08:58:13.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.997+0000 7f92970d7640 1 -- 172.21.15.40:0/1058764301 shutdown_connections 2023-10-18T08:58:13.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.997+0000 7f92970d7640 1 --2- 172.21.15.40:0/1058764301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92900ca0d0 0x7f9290157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.997+0000 7f92970d7640 1 --2- 172.21.15.40:0/1058764301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92900b15b0 0x7f92900b0630 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.997+0000 7f92970d7640 1 -- 172.21.15.40:0/1058764301 >> 172.21.15.40:0/1058764301 conn(0x7f9290069720 msgr2=0x7f9290069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:13.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.997+0000 7f92970d7640 1 -- 172.21.15.40:0/1058764301 shutdown_connections 2023-10-18T08:58:13.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.997+0000 7f92970d7640 1 -- 172.21.15.40:0/1058764301 wait complete. 2023-10-18T08:58:13.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.998+0000 7f92970d7640 1 Processor -- start 2023-10-18T08:58:13.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.999+0000 7f92970d7640 1 -- start start 2023-10-18T08:58:13.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.999+0000 7f92970d7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92900b15b0 0x7f9290158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:13.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:12.999+0000 7f92970d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92900b1e10 0x7f9290159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:13.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.000+0000 7f92970d7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92900ca0d0 0x7f9290159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:13.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.000+0000 7f92970d7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9290157e30 con 0x7f92900b15b0 2023-10-18T08:58:13.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.000+0000 7f92970d7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9290158110 con 0x7f92900b1e10 2023-10-18T08:58:13.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.000+0000 7f92970d7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92901583f0 con 0x7f92900ca0d0 2023-10-18T08:58:13.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.000+0000 7f92960d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92900b15b0 0x7f9290158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:13.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.000+0000 7f92960d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92900b15b0 0x7f9290158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49840/0 (socket says 172.21.15.40:49840) 2023-10-18T08:58:13.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.000+0000 7f92960d5640 1 -- 172.21.15.40:0/2115821676 learned_addr learned my addr 172.21.15.40:0/2115821676 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:13.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.000+0000 7f92960d5640 1 -- 172.21.15.40:0/2115821676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92900ca0d0 msgr2=0x7f9290159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:13.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.000+0000 7f92960d5640 1 --2- 172.21.15.40:0/2115821676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92900ca0d0 0x7f9290159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.000+0000 7f92958d4640 1 --2- 172.21.15.40:0/2115821676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92900b1e10 0x7f9290159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:13.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.000+0000 7f92960d5640 1 -- 172.21.15.40:0/2115821676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92900b1e10 msgr2=0x7f9290159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:13.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.000+0000 7f92960d5640 1 --2- 172.21.15.40:0/2115821676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92900b1e10 0x7f9290159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.000+0000 7f92960d5640 1 -- 172.21.15.40:0/2115821676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c009560 con 0x7f92900b15b0 2023-10-18T08:58:13.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.000+0000 7f92958d4640 1 --2- 172.21.15.40:0/2115821676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92900b1e10 0x7f9290159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:13.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.001+0000 7f92960d5640 1 --2- 172.21.15.40:0/2115821676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92900b15b0 0x7f9290158bf0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f928c009850 tx=0x7f928c02dba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:13.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.001+0000 7f9286ffd640 1 -- 172.21.15.40:0/2115821676 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c0035f0 con 0x7f92900b15b0 2023-10-18T08:58:13.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.001+0000 7f9286ffd640 1 -- 172.21.15.40:0/2115821676 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f928c003840 con 0x7f92900b15b0 2023-10-18T08:58:13.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.001+0000 7f9286ffd640 1 -- 172.21.15.40:0/2115821676 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c003bf0 con 0x7f92900b15b0 2023-10-18T08:58:13.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.001+0000 7f92970d7640 1 -- 172.21.15.40:0/2115821676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92901586d0 con 0x7f92900b15b0 2023-10-18T08:58:13.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.001+0000 7f92970d7640 1 -- 172.21.15.40:0/2115821676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9290159d90 con 0x7f92900b15b0 2023-10-18T08:58:13.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.002+0000 7f92970d7640 1 -- 172.21.15.40:0/2115821676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9290159f20 con 0x7f92900b15b0 2023-10-18T08:58:13.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.003+0000 7f9286ffd640 1 -- 172.21.15.40:0/2115821676 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f928c003d50 con 0x7f92900b15b0 2023-10-18T08:58:13.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.004+0000 7f9286ffd640 1 --2- 172.21.15.40:0/2115821676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f927406ae40 0x7f927406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:13.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.004+0000 7f9286ffd640 1 -- 172.21.15.40:0/2115821676 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f928c0ac410 con 0x7f92900b15b0 2023-10-18T08:58:13.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.005+0000 7f92958d4640 1 --2- 172.21.15.40:0/2115821676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f927406ae40 0x7f927406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:13.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.006+0000 7f92958d4640 1 --2- 172.21.15.40:0/2115821676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f927406ae40 0x7f927406d300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f92900cac60 tx=0x7f9280009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:13.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.008+0000 7f9286ffd640 1 -- 172.21.15.40:0/2115821676 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9290159f20 con 0x7f92900b15b0 2023-10-18T08:58:13.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.238+0000 7f92970d7640 1 -- 172.21.15.40:0/2115821676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9290159f20 con 0x7f92900b15b0 2023-10-18T08:58:13.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.240+0000 7f9286ffd640 1 -- 172.21.15.40:0/2115821676 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f9290159f20 con 0x7f92900b15b0 2023-10-18T08:58:13.243 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:13.244 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3668356540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3668356540},{"type":"v1","addr":"172.21.15.144:6836","nonce":3668356540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4222679844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4222679844},{"type":"v1","addr":"172.21.15.40:6834","nonce":4222679844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2829399368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2829399368},{"type":"v1","addr":"172.21.15.40:6835","nonce":2829399368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:57:14.826066+0000","modified":"2023-10-18T08:57:17.903233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1438456692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1438456692},{"type":"v1","addr":"172.21.15.144:6837","nonce":1438456692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-18T08:58:13.244 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 24 2023-10-18T08:58:13.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.242+0000 7f92970d7640 1 -- 172.21.15.40:0/2115821676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f927406ae40 msgr2=0x7f927406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:13.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.242+0000 7f92970d7640 1 --2- 172.21.15.40:0/2115821676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f927406ae40 0x7f927406d300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f92900cac60 tx=0x7f9280009040 comp rx=0 tx=0).stop 2023-10-18T08:58:13.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.242+0000 7f92970d7640 1 -- 172.21.15.40:0/2115821676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92900b15b0 msgr2=0x7f9290158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:13.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.242+0000 7f92970d7640 1 --2- 172.21.15.40:0/2115821676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92900b15b0 0x7f9290158bf0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f928c009850 tx=0x7f928c02dba0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.242+0000 7f92970d7640 1 -- 172.21.15.40:0/2115821676 shutdown_connections 2023-10-18T08:58:13.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.242+0000 7f92970d7640 1 --2- 172.21.15.40:0/2115821676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f927406ae40 0x7f927406d300 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.242+0000 7f92970d7640 1 --2- 172.21.15.40:0/2115821676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92900ca0d0 0x7f9290159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.242+0000 7f92970d7640 1 --2- 172.21.15.40:0/2115821676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92900b1e10 0x7f9290159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.242+0000 7f92970d7640 1 --2- 172.21.15.40:0/2115821676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92900b15b0 0x7f9290158bf0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.242+0000 7f92970d7640 1 -- 172.21.15.40:0/2115821676 >> 172.21.15.40:0/2115821676 conn(0x7f9290069720 msgr2=0x7f9290155610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:13.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.243+0000 7f92970d7640 1 -- 172.21.15.40:0/2115821676 shutdown_connections 2023-10-18T08:58:13.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.243+0000 7f92970d7640 1 -- 172.21.15.40:0/2115821676 wait complete. 2023-10-18T08:58:13.263 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:58:13.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.486+0000 7f1edad4e640 1 -- 172.21.15.40:0/180270545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ed413c450 msgr2=0x7f1ed40aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:13.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.486+0000 7f1edad4e640 1 --2- 172.21.15.40:0/180270545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ed413c450 0x7f1ed40aed00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009830 tx=0x7f1ed002bdf0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.486+0000 7f1edad4e640 1 -- 172.21.15.40:0/180270545 shutdown_connections 2023-10-18T08:58:13.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.486+0000 7f1edad4e640 1 --2- 172.21.15.40:0/180270545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ed413c450 0x7f1ed40aed00 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.486+0000 7f1edad4e640 1 -- 172.21.15.40:0/180270545 >> 172.21.15.40:0/180270545 conn(0x7f1ed4069720 msgr2=0x7f1ed4069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:13.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.486+0000 7f1edad4e640 1 -- 172.21.15.40:0/180270545 shutdown_connections 2023-10-18T08:58:13.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.486+0000 7f1edad4e640 1 -- 172.21.15.40:0/180270545 wait complete. 2023-10-18T08:58:13.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.488+0000 7f1edad4e640 1 Processor -- start 2023-10-18T08:58:13.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.488+0000 7f1edad4e640 1 -- start start 2023-10-18T08:58:13.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.489+0000 7f1edad4e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ed40af240 0x7f1ed40b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:13.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.489+0000 7f1edad4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed413c450 0x7f1ed41f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:13.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.489+0000 7f1edad4e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ed4159e90 0x7f1ed41f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:13.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.489+0000 7f1edad4e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed40b1e60 con 0x7f1ed4159e90 2023-10-18T08:58:13.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.489+0000 7f1edad4e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed40b1250 con 0x7f1ed413c450 2023-10-18T08:58:13.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.489+0000 7f1edad4e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed40b1530 con 0x7f1ed40af240 2023-10-18T08:58:13.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.489+0000 7f1eda54d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ed4159e90 0x7f1ed41f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:13.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.489+0000 7f1eda54d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ed4159e90 0x7f1ed41f1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49862/0 (socket says 172.21.15.40:49862) 2023-10-18T08:58:13.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.489+0000 7f1eda54d640 1 -- 172.21.15.40:0/3706102795 learned_addr learned my addr 172.21.15.40:0/3706102795 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:13.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.489+0000 7f1ed954b640 1 --2- 172.21.15.40:0/3706102795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed413c450 0x7f1ed41f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:13.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.490+0000 7f1ed9d4c640 1 --2- 172.21.15.40:0/3706102795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ed40af240 0x7f1ed40b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:13.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.490+0000 7f1eda54d640 1 -- 172.21.15.40:0/3706102795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ed40af240 msgr2=0x7f1ed40b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:13.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.490+0000 7f1eda54d640 1 --2- 172.21.15.40:0/3706102795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ed40af240 0x7f1ed40b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.490+0000 7f1eda54d640 1 -- 172.21.15.40:0/3706102795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed413c450 msgr2=0x7f1ed41f18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:13.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.490+0000 7f1eda54d640 1 --2- 172.21.15.40:0/3706102795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed413c450 0x7f1ed41f18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.490+0000 7f1eda54d640 1 -- 172.21.15.40:0/3706102795 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed0009490 con 0x7f1ed4159e90 2023-10-18T08:58:13.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.490+0000 7f1eda54d640 1 --2- 172.21.15.40:0/3706102795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ed4159e90 0x7f1ed41f1e10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00c6b0 tx=0x7f1ecc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:13.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.490+0000 7f1eca7fc640 1 -- 172.21.15.40:0/3706102795 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc003b70 con 0x7f1ed4159e90 2023-10-18T08:58:13.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.490+0000 7f1eca7fc640 1 -- 172.21.15.40:0/3706102795 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1ecc003cf0 con 0x7f1ed4159e90 2023-10-18T08:58:13.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.490+0000 7f1eca7fc640 1 -- 172.21.15.40:0/3706102795 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc005330 con 0x7f1ed4159e90 2023-10-18T08:58:13.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.490+0000 7f1edad4e640 1 -- 172.21.15.40:0/3706102795 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed40b1810 con 0x7f1ed4159e90 2023-10-18T08:58:13.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.490+0000 7f1edad4e640 1 -- 172.21.15.40:0/3706102795 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed415c1f0 con 0x7f1ed4159e90 2023-10-18T08:58:13.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.491+0000 7f1edad4e640 1 -- 172.21.15.40:0/3706102795 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed4066cf0 con 0x7f1ed4159e90 2023-10-18T08:58:13.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.492+0000 7f1eca7fc640 1 -- 172.21.15.40:0/3706102795 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1ecc005490 con 0x7f1ed4159e90 2023-10-18T08:58:13.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.493+0000 7f1eca7fc640 1 --2- 172.21.15.40:0/3706102795 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1eb806ae40 0x7f1eb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:13.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.493+0000 7f1eca7fc640 1 -- 172.21.15.40:0/3706102795 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1ecc089620 con 0x7f1ed4159e90 2023-10-18T08:58:13.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.494+0000 7f1ed9d4c640 1 --2- 172.21.15.40:0/3706102795 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1eb806ae40 0x7f1eb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:13.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.495+0000 7f1ed9d4c640 1 --2- 172.21.15.40:0/3706102795 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1eb806ae40 0x7f1eb806d300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0036000 tx=0x7f1ed0004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:13.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.497+0000 7f1eca7fc640 1 -- 172.21.15.40:0/3706102795 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ecc0545f0 con 0x7f1ed4159e90 2023-10-18T08:58:13.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.726+0000 7f1edad4e640 1 -- 172.21.15.40:0/3706102795 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ed40b1fa0 con 0x7f1ed4159e90 2023-10-18T08:58:13.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.727+0000 7f1eca7fc640 1 -- 172.21.15.40:0/3706102795 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f1ed40b1fa0 con 0x7f1ed4159e90 2023-10-18T08:58:13.731 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:13.731 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3668356540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3668356540},{"type":"v1","addr":"172.21.15.144:6836","nonce":3668356540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4222679844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4222679844},{"type":"v1","addr":"172.21.15.40:6834","nonce":4222679844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2829399368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2829399368},{"type":"v1","addr":"172.21.15.40:6835","nonce":2829399368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:57:14.826066+0000","modified":"2023-10-18T08:57:17.903233+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"b","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1438456692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1438456692},{"type":"v1","addr":"172.21.15.144:6837","nonce":1438456692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-18T08:58:13.731 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 24 2023-10-18T08:58:13.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.731+0000 7f1edad4e640 1 -- 172.21.15.40:0/3706102795 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1eb806ae40 msgr2=0x7f1eb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:13.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.731+0000 7f1edad4e640 1 --2- 172.21.15.40:0/3706102795 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1eb806ae40 0x7f1eb806d300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0036000 tx=0x7f1ed0004730 comp rx=0 tx=0).stop 2023-10-18T08:58:13.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.731+0000 7f1edad4e640 1 -- 172.21.15.40:0/3706102795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ed4159e90 msgr2=0x7f1ed41f1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:13.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.731+0000 7f1edad4e640 1 --2- 172.21.15.40:0/3706102795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ed4159e90 0x7f1ed41f1e10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00c6b0 tx=0x7f1ecc00cb70 comp rx=0 tx=0).stop 2023-10-18T08:58:13.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.731+0000 7f1edad4e640 1 -- 172.21.15.40:0/3706102795 shutdown_connections 2023-10-18T08:58:13.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.731+0000 7f1edad4e640 1 --2- 172.21.15.40:0/3706102795 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1eb806ae40 0x7f1eb806d300 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.731+0000 7f1edad4e640 1 --2- 172.21.15.40:0/3706102795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ed4159e90 0x7f1ed41f1e10 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.731+0000 7f1edad4e640 1 --2- 172.21.15.40:0/3706102795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed413c450 0x7f1ed41f18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.731+0000 7f1edad4e640 1 --2- 172.21.15.40:0/3706102795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ed40af240 0x7f1ed40b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.731+0000 7f1edad4e640 1 -- 172.21.15.40:0/3706102795 >> 172.21.15.40:0/3706102795 conn(0x7f1ed4069720 msgr2=0x7f1ed4157380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:13.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.731+0000 7f1edad4e640 1 -- 172.21.15.40:0/3706102795 shutdown_connections 2023-10-18T08:58:13.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.731+0000 7f1edad4e640 1 -- 172.21.15.40:0/3706102795 wait complete. 2023-10-18T08:58:13.752 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:58:13.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.974+0000 7faca1f7d640 1 -- 172.21.15.40:0/2920737665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac9c1586a0 msgr2=0x7fac9c0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:13.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.974+0000 7faca1f7d640 1 --2- 172.21.15.40:0/2920737665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac9c1586a0 0x7fac9c0b0bc0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0acc20 tx=0x7fac8402be40 comp rx=0 tx=0).stop 2023-10-18T08:58:13.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.974+0000 7faca1f7d640 1 -- 172.21.15.40:0/2920737665 shutdown_connections 2023-10-18T08:58:13.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.974+0000 7faca1f7d640 1 --2- 172.21.15.40:0/2920737665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac9c1586a0 0x7fac9c0b0bc0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.974+0000 7faca1f7d640 1 --2- 172.21.15.40:0/2920737665 >> v2:172.21.15.144:3300/0 conn(0x7fac9c06d710 0x7fac9c151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.974+0000 7faca1f7d640 1 -- 172.21.15.40:0/2920737665 >> 172.21.15.40:0/2920737665 conn(0x7fac9c06cee0 msgr2=0x7fac9c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:13.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.975+0000 7faca1f7d640 1 -- 172.21.15.40:0/2920737665 shutdown_connections 2023-10-18T08:58:13.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.975+0000 7faca1f7d640 1 -- 172.21.15.40:0/2920737665 wait complete. 2023-10-18T08:58:13.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.976+0000 7faca1f7d640 1 Processor -- start 2023-10-18T08:58:13.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.976+0000 7faca1f7d640 1 -- start start 2023-10-18T08:58:13.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.977+0000 7faca1f7d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac9c06d710 0x7fac9c0c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:13.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.977+0000 7faca1f7d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9c0b1100 0x7fac9c0a7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:13.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.977+0000 7faca1f7d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac9c1586a0 0x7fac9c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:13.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.977+0000 7faca1f7d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c1f4ba0 con 0x7fac9c06d710 2023-10-18T08:58:13.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.977+0000 7faca1f7d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c1f59e0 con 0x7fac9c1586a0 2023-10-18T08:58:13.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.977+0000 7faca1f7d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c0a7e40 con 0x7fac9c0b1100 2023-10-18T08:58:13.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.978+0000 7fac9b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac9c06d710 0x7fac9c0c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:13.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.978+0000 7fac9b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac9c06d710 0x7fac9c0c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49876/0 (socket says 172.21.15.40:49876) 2023-10-18T08:58:13.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.978+0000 7fac9b7fe640 1 -- 172.21.15.40:0/4286830938 learned_addr learned my addr 172.21.15.40:0/4286830938 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:13.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.978+0000 7fac9bfff640 1 --2- 172.21.15.40:0/4286830938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac9c1586a0 0x7fac9c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:13.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.978+0000 7fac93fff640 1 --2- 172.21.15.40:0/4286830938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9c0b1100 0x7fac9c0a7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:13.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.978+0000 7fac9b7fe640 1 -- 172.21.15.40:0/4286830938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9c0b1100 msgr2=0x7fac9c0a7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:13.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.978+0000 7fac9b7fe640 1 --2- 172.21.15.40:0/4286830938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9c0b1100 0x7fac9c0a7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.978+0000 7fac9b7fe640 1 -- 172.21.15.40:0/4286830938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac9c1586a0 msgr2=0x7fac9c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:13.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.978+0000 7fac9b7fe640 1 --2- 172.21.15.40:0/4286830938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac9c1586a0 0x7fac9c0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:13.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.978+0000 7fac9b7fe640 1 -- 172.21.15.40:0/4286830938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac84009560 con 0x7fac9c06d710 2023-10-18T08:58:13.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.979+0000 7fac9b7fe640 1 --2- 172.21.15.40:0/4286830938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac9c06d710 0x7fac9c0c8870 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00d740 tx=0x7fac8c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:13.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.979+0000 7fac98ff9640 1 -- 172.21.15.40:0/4286830938 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c009c30 con 0x7fac9c06d710 2023-10-18T08:58:13.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.979+0000 7fac98ff9640 1 -- 172.21.15.40:0/4286830938 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fac8c009e80 con 0x7fac9c06d710 2023-10-18T08:58:13.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.979+0000 7fac98ff9640 1 -- 172.21.15.40:0/4286830938 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c0106c0 con 0x7fac9c06d710 2023-10-18T08:58:13.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.979+0000 7faca1f7d640 1 -- 172.21.15.40:0/4286830938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9c1f4dd0 con 0x7fac9c06d710 2023-10-18T08:58:13.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.979+0000 7faca1f7d640 1 -- 172.21.15.40:0/4286830938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9c1f5390 con 0x7fac9c06d710 2023-10-18T08:58:13.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.980+0000 7faca1f7d640 1 -- 172.21.15.40:0/4286830938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9c066fd0 con 0x7fac9c06d710 2023-10-18T08:58:13.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.981+0000 7fac98ff9640 1 -- 172.21.15.40:0/4286830938 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fac8c010820 con 0x7fac9c06d710 2023-10-18T08:58:13.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.982+0000 7fac98ff9640 1 --2- 172.21.15.40:0/4286830938 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac7c06ae70 0x7fac7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:13.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.982+0000 7fac98ff9640 1 -- 172.21.15.40:0/4286830938 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fac8c089ab0 con 0x7fac9c06d710 2023-10-18T08:58:13.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.983+0000 7fac93fff640 1 --2- 172.21.15.40:0/4286830938 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac7c06ae70 0x7fac7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:13.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.984+0000 7fac93fff640 1 --2- 172.21.15.40:0/4286830938 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac7c06ae70 0x7fac7c06d330 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fac84035000 tx=0x7fac84004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:13.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:13.986+0000 7fac98ff9640 1 -- 172.21.15.40:0/4286830938 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac8c054bb0 con 0x7fac9c06d710 2023-10-18T08:58:14.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.182+0000 7faca1f7d640 1 -- 172.21.15.40:0/4286830938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac9c06fb50 con 0x7fac9c06d710 2023-10-18T08:58:14.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.185+0000 7fac98ff9640 1 -- 172.21.15.40:0/4286830938 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16620 (secure 0 0 0) 0x7fac8c021090 con 0x7fac9c06d710 2023-10-18T08:58:14.189 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:14.189 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":56,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:58:02.917888+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:57:12.765969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-18T08:57:13.851110+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":5}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:58:14.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.187+0000 7faca1f7d640 1 -- 172.21.15.40:0/4286830938 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac7c06ae70 msgr2=0x7fac7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:14.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.187+0000 7faca1f7d640 1 --2- 172.21.15.40:0/4286830938 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac7c06ae70 0x7fac7c06d330 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fac84035000 tx=0x7fac84004730 comp rx=0 tx=0).stop 2023-10-18T08:58:14.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.187+0000 7faca1f7d640 1 -- 172.21.15.40:0/4286830938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac9c06d710 msgr2=0x7fac9c0c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:14.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.187+0000 7faca1f7d640 1 --2- 172.21.15.40:0/4286830938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac9c06d710 0x7fac9c0c8870 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00d740 tx=0x7fac8c00dc00 comp rx=0 tx=0).stop 2023-10-18T08:58:14.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.188+0000 7faca1f7d640 1 -- 172.21.15.40:0/4286830938 shutdown_connections 2023-10-18T08:58:14.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.188+0000 7faca1f7d640 1 --2- 172.21.15.40:0/4286830938 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac7c06ae70 0x7fac7c06d330 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.188+0000 7faca1f7d640 1 --2- 172.21.15.40:0/4286830938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac9c1586a0 0x7fac9c0ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.188+0000 7faca1f7d640 1 --2- 172.21.15.40:0/4286830938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac9c0b1100 0x7fac9c0a7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.188+0000 7faca1f7d640 1 --2- 172.21.15.40:0/4286830938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac9c06d710 0x7fac9c0c8870 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.188+0000 7faca1f7d640 1 -- 172.21.15.40:0/4286830938 >> 172.21.15.40:0/4286830938 conn(0x7fac9c06cee0 msgr2=0x7fac9c1566f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:14.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.188+0000 7faca1f7d640 1 -- 172.21.15.40:0/4286830938 shutdown_connections 2023-10-18T08:58:14.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.188+0000 7faca1f7d640 1 -- 172.21.15.40:0/4286830938 wait complete. 2023-10-18T08:58:14.211 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T08:58:14.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.433+0000 7f9bcb59e640 1 -- 172.21.15.40:0/1577394610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bcc06db80 msgr2=0x7f9bcc0cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:14.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.433+0000 7f9bcb59e640 1 --2- 172.21.15.40:0/1577394610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bcc06db80 0x7f9bcc0cb890 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0715d0 tx=0x7f9bb402bdc0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.433+0000 7f9bcb59e640 1 -- 172.21.15.40:0/1577394610 shutdown_connections 2023-10-18T08:58:14.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.433+0000 7f9bcb59e640 1 --2- 172.21.15.40:0/1577394610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bcc06db80 0x7f9bcc0cb890 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.433+0000 7f9bcb59e640 1 --2- 172.21.15.40:0/1577394610 >> v2:172.21.15.144:3300/0 conn(0x7f9bcc141aa0 0x7f9bcc15c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.433+0000 7f9bcb59e640 1 -- 172.21.15.40:0/1577394610 >> 172.21.15.40:0/1577394610 conn(0x7f9bcc0cdfd0 msgr2=0x7f9bcc0ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:14.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.433+0000 7f9bcb59e640 1 -- 172.21.15.40:0/1577394610 shutdown_connections 2023-10-18T08:58:14.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.433+0000 7f9bcb59e640 1 -- 172.21.15.40:0/1577394610 wait complete. 2023-10-18T08:58:14.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.435+0000 7f9bcb59e640 1 Processor -- start 2023-10-18T08:58:14.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.435+0000 7f9bcb59e640 1 -- start start 2023-10-18T08:58:14.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.436+0000 7f9bcb59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bcc06db80 0x7f9bcc157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:14.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.436+0000 7f9bcb59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9bcc070590 0x7f9bcc139160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:14.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.436+0000 7f9bcb59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bcc141aa0 0x7f9bcc1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:14.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.436+0000 7f9bcb59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc157f50 con 0x7f9bcc141aa0 2023-10-18T08:58:14.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.436+0000 7f9bcb59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc158750 con 0x7f9bcc06db80 2023-10-18T08:58:14.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.436+0000 7f9bcb59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc158a30 con 0x7f9bcc070590 2023-10-18T08:58:14.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.436+0000 7f9bcad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bcc141aa0 0x7f9bcc1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:14.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.436+0000 7f9bcad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bcc141aa0 0x7f9bcc1597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49906/0 (socket says 172.21.15.40:49906) 2023-10-18T08:58:14.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.436+0000 7f9bcad9d640 1 -- 172.21.15.40:0/753495553 learned_addr learned my addr 172.21.15.40:0/753495553 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:14.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.436+0000 7f9bca59c640 1 --2- 172.21.15.40:0/753495553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bcc06db80 0x7f9bcc157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:14.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.436+0000 7f9bc9d9b640 1 --2- 172.21.15.40:0/753495553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9bcc070590 0x7f9bcc139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:14.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.437+0000 7f9bcad9d640 1 -- 172.21.15.40:0/753495553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9bcc070590 msgr2=0x7f9bcc139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:14.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.437+0000 7f9bcad9d640 1 --2- 172.21.15.40:0/753495553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9bcc070590 0x7f9bcc139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.437+0000 7f9bcad9d640 1 -- 172.21.15.40:0/753495553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bcc06db80 msgr2=0x7f9bcc157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:14.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.437+0000 7f9bcad9d640 1 --2- 172.21.15.40:0/753495553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bcc06db80 0x7f9bcc157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.437+0000 7f9bcad9d640 1 -- 172.21.15.40:0/753495553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb4009560 con 0x7f9bcc141aa0 2023-10-18T08:58:14.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.437+0000 7f9bcad9d640 1 --2- 172.21.15.40:0/753495553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bcc141aa0 0x7f9bcc1597f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4001eb0 tx=0x7f9bc400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:14.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.437+0000 7f9bc2ffd640 1 -- 172.21.15.40:0/753495553 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40052d0 con 0x7f9bcc141aa0 2023-10-18T08:58:14.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.437+0000 7f9bc2ffd640 1 -- 172.21.15.40:0/753495553 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9bc40035f0 con 0x7f9bcc141aa0 2023-10-18T08:58:14.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.437+0000 7f9bc2ffd640 1 -- 172.21.15.40:0/753495553 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40039e0 con 0x7f9bcc141aa0 2023-10-18T08:58:14.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.437+0000 7f9bcb59e640 1 -- 172.21.15.40:0/753495553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc158d10 con 0x7f9bcc141aa0 2023-10-18T08:58:14.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.438+0000 7f9bcb59e640 1 -- 172.21.15.40:0/753495553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc1592d0 con 0x7f9bcc141aa0 2023-10-18T08:58:14.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.439+0000 7f9bcb59e640 1 -- 172.21.15.40:0/753495553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc066a90 con 0x7f9bcc141aa0 2023-10-18T08:58:14.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.440+0000 7f9bc2ffd640 1 -- 172.21.15.40:0/753495553 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9bc4003b40 con 0x7f9bcc141aa0 2023-10-18T08:58:14.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.441+0000 7f9bc2ffd640 1 --2- 172.21.15.40:0/753495553 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9bac06ae40 0x7f9bac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:14.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.441+0000 7f9bc2ffd640 1 -- 172.21.15.40:0/753495553 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9bc4088d50 con 0x7f9bcc141aa0 2023-10-18T08:58:14.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.441+0000 7f9bca59c640 1 --2- 172.21.15.40:0/753495553 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9bac06ae40 0x7f9bac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:14.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.442+0000 7f9bca59c640 1 --2- 172.21.15.40:0/753495553 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9bac06ae40 0x7f9bac06d300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00a2e0 tx=0x7f9bbc00a270 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:14.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.445+0000 7f9bc2ffd640 1 -- 172.21.15.40:0/753495553 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9bc4053d20 con 0x7f9bcc141aa0 2023-10-18T08:58:14.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.678+0000 7f9bcb59e640 1 -- 172.21.15.40:0/753495553 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9bcc06ebc0 con 0x7f9bcc141aa0 2023-10-18T08:58:14.682 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T08:58:14.679+0000 7f6a0f710640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T08:58:14.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.682+0000 7f9bc2ffd640 1 -- 172.21.15.40:0/753495553 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9bcc06ebc0 con 0x7f9bcc141aa0 2023-10-18T08:58:14.686 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T08:58:14.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.685+0000 7f9bcb59e640 1 -- 172.21.15.40:0/753495553 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9bac06ae40 msgr2=0x7f9bac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:14.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.686+0000 7f9bcb59e640 1 --2- 172.21.15.40:0/753495553 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9bac06ae40 0x7f9bac06d300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00a2e0 tx=0x7f9bbc00a270 comp rx=0 tx=0).stop 2023-10-18T08:58:14.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.686+0000 7f9bcb59e640 1 -- 172.21.15.40:0/753495553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bcc141aa0 msgr2=0x7f9bcc1597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:14.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.686+0000 7f9bcb59e640 1 --2- 172.21.15.40:0/753495553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bcc141aa0 0x7f9bcc1597f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4001eb0 tx=0x7f9bc400c890 comp rx=0 tx=0).stop 2023-10-18T08:58:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.686+0000 7f9bcb59e640 1 -- 172.21.15.40:0/753495553 shutdown_connections 2023-10-18T08:58:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.686+0000 7f9bcb59e640 1 --2- 172.21.15.40:0/753495553 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9bac06ae40 0x7f9bac06d300 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.686+0000 7f9bcb59e640 1 --2- 172.21.15.40:0/753495553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bcc141aa0 0x7f9bcc1597f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.686+0000 7f9bcb59e640 1 --2- 172.21.15.40:0/753495553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9bcc070590 0x7f9bcc139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.686+0000 7f9bcb59e640 1 --2- 172.21.15.40:0/753495553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bcc06db80 0x7f9bcc157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.686+0000 7f9bcb59e640 1 -- 172.21.15.40:0/753495553 >> 172.21.15.40:0/753495553 conn(0x7f9bcc0cdfd0 msgr2=0x7f9bcc140980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:14.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.686+0000 7f9bcb59e640 1 -- 172.21.15.40:0/753495553 shutdown_connections 2023-10-18T08:58:14.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.686+0000 7f9bcb59e640 1 -- 172.21.15.40:0/753495553 wait complete. 2023-10-18T08:58:14.707 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T08:58:14.806 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-18T08:58:14.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.939+0000 7fc67f59e640 1 -- 172.21.15.40:0/960251224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68015b380 msgr2=0x7fc6800b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:14.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.939+0000 7fc67f59e640 1 --2- 172.21.15.40:0/960251224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68015b380 0x7fc6800b1120 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc68008ab50 tx=0x7fc67002bd70 comp rx=0 tx=0).stop 2023-10-18T08:58:14.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.939+0000 7fc67f59e640 1 -- 172.21.15.40:0/960251224 shutdown_connections 2023-10-18T08:58:14.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.939+0000 7fc67f59e640 1 --2- 172.21.15.40:0/960251224 >> v2:172.21.15.144:3300/0 conn(0x7fc68015b760 0x7fc6800cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.939+0000 7fc67f59e640 1 --2- 172.21.15.40:0/960251224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68015b380 0x7fc6800b1120 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.939+0000 7fc67f59e640 1 -- 172.21.15.40:0/960251224 >> 172.21.15.40:0/960251224 conn(0x7fc68008acb0 msgr2=0x7fc68009a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:14.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.939+0000 7fc67f59e640 1 -- 172.21.15.40:0/960251224 shutdown_connections 2023-10-18T08:58:14.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.939+0000 7fc67f59e640 1 -- 172.21.15.40:0/960251224 wait complete. 2023-10-18T08:58:14.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.941+0000 7fc67f59e640 1 Processor -- start 2023-10-18T08:58:14.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.941+0000 7fc67f59e640 1 -- start start 2023-10-18T08:58:14.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.942+0000 7fc67f59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6800cdb60 0x7fc680155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:14.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.942+0000 7fc67f59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68015b380 0x7fc6800c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:14.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.942+0000 7fc67f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68015b760 0x7fc6800c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:14.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.942+0000 7fc67f59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68015cbc0 con 0x7fc68015b760 2023-10-18T08:58:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.942+0000 7fc67f59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6800c59c0 con 0x7fc6800cdb60 2023-10-18T08:58:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.942+0000 7fc67f59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc680155960 con 0x7fc68015b380 2023-10-18T08:58:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.942+0000 7fc67ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68015b760 0x7fc6800c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.942+0000 7fc67ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68015b760 0x7fc6800c88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60926/0 (socket says 172.21.15.40:60926) 2023-10-18T08:58:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.942+0000 7fc67ed9d640 1 -- 172.21.15.40:0/1976158517 learned_addr learned my addr 172.21.15.40:0/1976158517 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:14.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.943+0000 7fc677fff640 1 --2- 172.21.15.40:0/1976158517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68015b380 0x7fc6800c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:14.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.943+0000 7fc67e59c640 1 --2- 172.21.15.40:0/1976158517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6800cdb60 0x7fc680155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.943+0000 7fc67ed9d640 1 -- 172.21.15.40:0/1976158517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68015b380 msgr2=0x7fc6800c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.943+0000 7fc67ed9d640 1 --2- 172.21.15.40:0/1976158517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68015b380 0x7fc6800c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.943+0000 7fc67ed9d640 1 -- 172.21.15.40:0/1976158517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6800cdb60 msgr2=0x7fc680155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.943+0000 7fc67ed9d640 1 --2- 172.21.15.40:0/1976158517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6800cdb60 0x7fc680155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.943+0000 7fc67ed9d640 1 -- 172.21.15.40:0/1976158517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc670009490 con 0x7fc68015b760 2023-10-18T08:58:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.943+0000 7fc67ed9d640 1 --2- 172.21.15.40:0/1976158517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68015b760 0x7fc6800c88a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc67800cbc0 tx=0x7fc67800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:14.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.943+0000 7fc676ffd640 1 -- 172.21.15.40:0/1976158517 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67800ad40 con 0x7fc68015b760 2023-10-18T08:58:14.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.944+0000 7fc676ffd640 1 -- 172.21.15.40:0/1976158517 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc678012420 con 0x7fc68015b760 2023-10-18T08:58:14.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.944+0000 7fc67f59e640 1 -- 172.21.15.40:0/1976158517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc680156970 con 0x7fc68015b760 2023-10-18T08:58:14.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.944+0000 7fc676ffd640 1 -- 172.21.15.40:0/1976158517 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678012850 con 0x7fc68015b760 2023-10-18T08:58:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.944+0000 7fc67f59e640 1 -- 172.21.15.40:0/1976158517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68015cda0 con 0x7fc68015b760 2023-10-18T08:58:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.945+0000 7fc67f59e640 1 -- 172.21.15.40:0/1976158517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc680071da0 con 0x7fc68015b760 2023-10-18T08:58:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.946+0000 7fc676ffd640 1 -- 172.21.15.40:0/1976158517 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc67801a3b0 con 0x7fc68015b760 2023-10-18T08:58:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.947+0000 7fc676ffd640 1 --2- 172.21.15.40:0/1976158517 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc66006ae70 0x7fc66006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.947+0000 7fc676ffd640 1 -- 172.21.15.40:0/1976158517 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc678089f90 con 0x7fc68015b760 2023-10-18T08:58:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.947+0000 7fc67e59c640 1 --2- 172.21.15.40:0/1976158517 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc66006ae70 0x7fc66006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.949+0000 7fc67e59c640 1 --2- 172.21.15.40:0/1976158517 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc66006ae70 0x7fc66006d330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc670036000 tx=0x7fc670004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:14.951+0000 7fc676ffd640 1 -- 172.21.15.40:0/1976158517 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc678054f10 con 0x7fc68015b760 2023-10-18T08:58:15.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.191+0000 7fc67f59e640 1 -- 172.21.15.40:0/1976158517 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc680156e90 con 0x7fc68015b760 2023-10-18T08:58:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.693+0000 7fc676ffd640 1 -- 172.21.15.40:0/1976158517 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc680156e90 con 0x7fc68015b760 2023-10-18T08:58:15.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.696+0000 7fc67f59e640 1 -- 172.21.15.40:0/1976158517 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc66006ae70 msgr2=0x7fc66006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:15.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.696+0000 7fc67f59e640 1 --2- 172.21.15.40:0/1976158517 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc66006ae70 0x7fc66006d330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc670036000 tx=0x7fc670004730 comp rx=0 tx=0).stop 2023-10-18T08:58:15.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.696+0000 7fc67f59e640 1 -- 172.21.15.40:0/1976158517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68015b760 msgr2=0x7fc6800c88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:15.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.696+0000 7fc67f59e640 1 --2- 172.21.15.40:0/1976158517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68015b760 0x7fc6800c88a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc67800cbc0 tx=0x7fc67800a430 comp rx=0 tx=0).stop 2023-10-18T08:58:15.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.697+0000 7fc67f59e640 1 -- 172.21.15.40:0/1976158517 shutdown_connections 2023-10-18T08:58:15.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.697+0000 7fc67f59e640 1 --2- 172.21.15.40:0/1976158517 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc66006ae70 0x7fc66006d330 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:15.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.697+0000 7fc67f59e640 1 --2- 172.21.15.40:0/1976158517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc68015b760 0x7fc6800c88a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:15.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.697+0000 7fc67f59e640 1 --2- 172.21.15.40:0/1976158517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc68015b380 0x7fc6800c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:15.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.697+0000 7fc67f59e640 1 --2- 172.21.15.40:0/1976158517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6800cdb60 0x7fc680155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:15.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.697+0000 7fc67f59e640 1 -- 172.21.15.40:0/1976158517 >> 172.21.15.40:0/1976158517 conn(0x7fc68008acb0 msgr2=0x7fc6800ccb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:15.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.697+0000 7fc67f59e640 1 -- 172.21.15.40:0/1976158517 shutdown_connections 2023-10-18T08:58:15.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.697+0000 7fc67f59e640 1 -- 172.21.15.40:0/1976158517 wait complete. 2023-10-18T08:58:15.721 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T08:58:15.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.959+0000 7f79c66ca640 1 -- 172.21.15.40:0/3568884423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c013eaa0 msgr2=0x7f79c013eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:15.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.959+0000 7f79c66ca640 1 --2- 172.21.15.40:0/3568884423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c013eaa0 0x7f79c013eef0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f79c015a870 tx=0x7f79b802c260 comp rx=0 tx=0).stop 2023-10-18T08:58:15.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.959+0000 7f79c66ca640 1 -- 172.21.15.40:0/3568884423 shutdown_connections 2023-10-18T08:58:15.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.959+0000 7f79c66ca640 1 --2- 172.21.15.40:0/3568884423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c013eaa0 0x7f79c013eef0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:15.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.959+0000 7f79c66ca640 1 --2- 172.21.15.40:0/3568884423 >> v2:172.21.15.144:3300/0 conn(0x7f79c00c9f40 0x7f79c00cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:15.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.959+0000 7f79c66ca640 1 -- 172.21.15.40:0/3568884423 >> 172.21.15.40:0/3568884423 conn(0x7f79c00aeb90 msgr2=0x7f79c00aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:15.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.959+0000 7f79c66ca640 1 -- 172.21.15.40:0/3568884423 shutdown_connections 2023-10-18T08:58:15.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.959+0000 7f79c66ca640 1 -- 172.21.15.40:0/3568884423 wait complete. 2023-10-18T08:58:15.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.962+0000 7f79c66ca640 1 Processor -- start 2023-10-18T08:58:15.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.962+0000 7f79c66ca640 1 -- start start 2023-10-18T08:58:15.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.963+0000 7f79c66ca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c00c9f40 0x7f79c01f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:15.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.963+0000 7f79c66ca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79c013c220 0x7f79c01534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:15.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.963+0000 7f79c66ca640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79c013eaa0 0x7f79c00b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:15.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.963+0000 7f79c66ca640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c0153e50 con 0x7f79c00c9f40 2023-10-18T08:58:15.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.963+0000 7f79c66ca640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c0153a30 con 0x7f79c013c220 2023-10-18T08:58:15.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.963+0000 7f79c66ca640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c0157340 con 0x7f79c013eaa0 2023-10-18T08:58:15.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.964+0000 7f79c56c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c00c9f40 0x7f79c01f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:15.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.964+0000 7f79c56c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c00c9f40 0x7f79c01f2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60942/0 (socket says 172.21.15.40:60942) 2023-10-18T08:58:15.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.964+0000 7f79c56c8640 1 -- 172.21.15.40:0/2333738878 learned_addr learned my addr 172.21.15.40:0/2333738878 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:15.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.964+0000 7f79c5ec9640 1 --2- 172.21.15.40:0/2333738878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79c013eaa0 0x7f79c00b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:15.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.964+0000 7f79c4ec7640 1 --2- 172.21.15.40:0/2333738878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79c013c220 0x7f79c01534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:15.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.964+0000 7f79c56c8640 1 -- 172.21.15.40:0/2333738878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79c013eaa0 msgr2=0x7f79c00b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:15.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.964+0000 7f79c56c8640 1 --2- 172.21.15.40:0/2333738878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79c013eaa0 0x7f79c00b0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:15.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.964+0000 7f79c56c8640 1 -- 172.21.15.40:0/2333738878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79c013c220 msgr2=0x7f79c01534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:15.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.964+0000 7f79c56c8640 1 --2- 172.21.15.40:0/2333738878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79c013c220 0x7f79c01534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:15.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.964+0000 7f79c56c8640 1 -- 172.21.15.40:0/2333738878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b8009900 con 0x7f79c00c9f40 2023-10-18T08:58:15.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.964+0000 7f79c56c8640 1 --2- 172.21.15.40:0/2333738878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c00c9f40 0x7f79c01f2f00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00c430 tx=0x7f79bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:15.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.965+0000 7f79b5ffb640 1 -- 172.21.15.40:0/2333738878 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc011790 con 0x7f79c00c9f40 2023-10-18T08:58:15.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.965+0000 7f79b5ffb640 1 -- 172.21.15.40:0/2333738878 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f79bc0119e0 con 0x7f79c00c9f40 2023-10-18T08:58:15.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.965+0000 7f79c66ca640 1 -- 172.21.15.40:0/2333738878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c01575e0 con 0x7f79c00c9f40 2023-10-18T08:58:15.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.965+0000 7f79b5ffb640 1 -- 172.21.15.40:0/2333738878 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc011e30 con 0x7f79c00c9f40 2023-10-18T08:58:15.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.965+0000 7f79c66ca640 1 -- 172.21.15.40:0/2333738878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c0157ba0 con 0x7f79c00c9f40 2023-10-18T08:58:15.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.966+0000 7f79c66ca640 1 -- 172.21.15.40:0/2333738878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c0066e00 con 0x7f79c00c9f40 2023-10-18T08:58:15.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.967+0000 7f79b5ffb640 1 -- 172.21.15.40:0/2333738878 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f79bc01c460 con 0x7f79c00c9f40 2023-10-18T08:58:15.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.969+0000 7f79b5ffb640 1 --2- 172.21.15.40:0/2333738878 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79a406ae40 0x7f79a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:15.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.969+0000 7f79b5ffb640 1 -- 172.21.15.40:0/2333738878 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f79bc089900 con 0x7f79c00c9f40 2023-10-18T08:58:15.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.970+0000 7f79c4ec7640 1 --2- 172.21.15.40:0/2333738878 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79a406ae40 0x7f79a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:15.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.971+0000 7f79c4ec7640 1 --2- 172.21.15.40:0/2333738878 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79a406ae40 0x7f79a406d300 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f79b0005fd0 tx=0x7f79b0005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:15.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:15.974+0000 7f79b5ffb640 1 -- 172.21.15.40:0/2333738878 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79bc054880 con 0x7f79c00c9f40 2023-10-18T08:58:16.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:16.171+0000 7f79c66ca640 1 -- 172.21.15.40:0/2333738878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f79c01f1df0 con 0x7f79c00c9f40 2023-10-18T08:58:17.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.014+0000 7f79b5ffb640 1 -- 172.21.15.40:0/2333738878 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) v1 ==== 181+0+0 (secure 0 0 0) 0x7f79bc057ea0 con 0x7f79c00c9f40 2023-10-18T08:58:17.018 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-10-18T08:58:17.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.019+0000 7f79c66ca640 1 -- 172.21.15.40:0/2333738878 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79a406ae40 msgr2=0x7f79a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:17.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.019+0000 7f79c66ca640 1 --2- 172.21.15.40:0/2333738878 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79a406ae40 0x7f79a406d300 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f79b0005fd0 tx=0x7f79b0005850 comp rx=0 tx=0).stop 2023-10-18T08:58:17.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.019+0000 7f79c66ca640 1 -- 172.21.15.40:0/2333738878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c00c9f40 msgr2=0x7f79c01f2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:17.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.019+0000 7f79c66ca640 1 --2- 172.21.15.40:0/2333738878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c00c9f40 0x7f79c01f2f00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00c430 tx=0x7f79bc00c8f0 comp rx=0 tx=0).stop 2023-10-18T08:58:17.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.019+0000 7f79c66ca640 1 -- 172.21.15.40:0/2333738878 shutdown_connections 2023-10-18T08:58:17.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.019+0000 7f79c66ca640 1 --2- 172.21.15.40:0/2333738878 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f79a406ae40 0x7f79a406d300 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:17.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.019+0000 7f79c66ca640 1 --2- 172.21.15.40:0/2333738878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79c013eaa0 0x7f79c00b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:17.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.019+0000 7f79c66ca640 1 --2- 172.21.15.40:0/2333738878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79c013c220 0x7f79c01534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:17.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.019+0000 7f79c66ca640 1 --2- 172.21.15.40:0/2333738878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79c00c9f40 0x7f79c01f2f00 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:17.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.019+0000 7f79c66ca640 1 -- 172.21.15.40:0/2333738878 >> 172.21.15.40:0/2333738878 conn(0x7f79c00aeb90 msgr2=0x7f79c00ae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:17.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.020+0000 7f79c66ca640 1 -- 172.21.15.40:0/2333738878 shutdown_connections 2023-10-18T08:58:17.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.020+0000 7f79c66ca640 1 -- 172.21.15.40:0/2333738878 wait complete. 2023-10-18T08:58:17.049 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T08:58:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.280+0000 7f5f351bd640 1 -- 172.21.15.40:0/2148728772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1c0039a0 msgr2=0x7f5f1c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.280+0000 7f5f351bd640 1 --2- 172.21.15.40:0/2148728772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1c0039a0 0x7f5f1c003dc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f5f20001650 tx=0x7f5f1c0057f0 comp rx=0 tx=0).stop 2023-10-18T08:58:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.280+0000 7f5f349bc640 1 -- 172.21.15.40:0/2148728772 reap_dead start 2023-10-18T08:58:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.280+0000 7f5f351bd640 1 -- 172.21.15.40:0/2148728772 shutdown_connections 2023-10-18T08:58:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.280+0000 7f5f351bd640 1 -- 172.21.15.40:0/2148728772 >> 172.21.15.40:0/2148728772 conn(0x7f5f300aeb90 msgr2=0x7f5f300aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:17.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.280+0000 7f5f351bd640 1 -- 172.21.15.40:0/2148728772 shutdown_connections 2023-10-18T08:58:17.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.280+0000 7f5f351bd640 1 -- 172.21.15.40:0/2148728772 wait complete. 2023-10-18T08:58:17.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.282+0000 7f5f351bd640 1 Processor -- start 2023-10-18T08:58:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.282+0000 7f5f351bd640 1 -- start start 2023-10-18T08:58:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.283+0000 7f5f351bd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1c0039a0 0x7f5f301592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.283+0000 7f5f351bd640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f301553c0 0x7f5f30155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.283+0000 7f5f351bd640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f301e5c30 0x7f5f301e60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:17.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.283+0000 7f5f351bd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30156be0 con 0x7f5f1c0039a0 2023-10-18T08:58:17.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.283+0000 7f5f351bd640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30155d60 con 0x7f5f301e5c30 2023-10-18T08:58:17.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.283+0000 7f5f351bd640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f301e65f0 con 0x7f5f301553c0 2023-10-18T08:58:17.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.283+0000 7f5f2ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1c0039a0 0x7f5f301592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:17.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.283+0000 7f5f2ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1c0039a0 0x7f5f301592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60972/0 (socket says 172.21.15.40:60972) 2023-10-18T08:58:17.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.283+0000 7f5f2ffff640 1 -- 172.21.15.40:0/3742458967 learned_addr learned my addr 172.21.15.40:0/3742458967 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:17.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.284+0000 7f5f27fff640 1 --2- 172.21.15.40:0/3742458967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f301553c0 0x7f5f30155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:17.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.284+0000 7f5f349bc640 1 --2- 172.21.15.40:0/3742458967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f301e5c30 0x7f5f301e60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:17.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.284+0000 7f5f2ffff640 1 -- 172.21.15.40:0/3742458967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f301553c0 msgr2=0x7f5f30155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:17.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.284+0000 7f5f2ffff640 1 --2- 172.21.15.40:0/3742458967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f301553c0 0x7f5f30155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.284+0000 7f5f2ffff640 1 -- 172.21.15.40:0/3742458967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f301e5c30 msgr2=0x7f5f301e60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.284+0000 7f5f2ffff640 1 --2- 172.21.15.40:0/3742458967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f301e5c30 0x7f5f301e60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.284+0000 7f5f2ffff640 1 -- 172.21.15.40:0/3742458967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f30155f60 con 0x7f5f1c0039a0 2023-10-18T08:58:17.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.284+0000 7f5f2ffff640 1 --2- 172.21.15.40:0/3742458967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1c0039a0 0x7f5f301592b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5f2002c0b0 tx=0x7f5f200035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:17.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.284+0000 7f5f2d7fa640 1 -- 172.21.15.40:0/3742458967 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20003bf0 con 0x7f5f1c0039a0 2023-10-18T08:58:17.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.284+0000 7f5f2d7fa640 1 -- 172.21.15.40:0/3742458967 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5f20003e40 con 0x7f5f1c0039a0 2023-10-18T08:58:17.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.284+0000 7f5f2d7fa640 1 -- 172.21.15.40:0/3742458967 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2002c770 con 0x7f5f1c0039a0 2023-10-18T08:58:17.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.284+0000 7f5f351bd640 1 -- 172.21.15.40:0/3742458967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f30156240 con 0x7f5f1c0039a0 2023-10-18T08:58:17.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.285+0000 7f5f351bd640 1 -- 172.21.15.40:0/3742458967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f30156800 con 0x7f5f1c0039a0 2023-10-18T08:58:17.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.286+0000 7f5f351bd640 1 -- 172.21.15.40:0/3742458967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f30156990 con 0x7f5f1c0039a0 2023-10-18T08:58:17.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.287+0000 7f5f2d7fa640 1 -- 172.21.15.40:0/3742458967 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5f2002c8d0 con 0x7f5f1c0039a0 2023-10-18T08:58:17.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.288+0000 7f5f2d7fa640 1 --2- 172.21.15.40:0/3742458967 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f1006ae70 0x7f5f1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:17.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.288+0000 7f5f2d7fa640 1 -- 172.21.15.40:0/3742458967 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f5f200aa9f0 con 0x7f5f1c0039a0 2023-10-18T08:58:17.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.289+0000 7f5f27fff640 1 --2- 172.21.15.40:0/3742458967 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f1006ae70 0x7f5f1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:17.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.290+0000 7f5f27fff640 1 --2- 172.21.15.40:0/3742458967 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f1006ae70 0x7f5f1006d330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5f3013adb0 tx=0x7f5f18005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:17.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.292+0000 7f5f2d7fa640 1 -- 172.21.15.40:0/3742458967 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f20075aa0 con 0x7f5f1c0039a0 2023-10-18T08:58:17.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:17.484+0000 7f5f351bd640 1 -- 172.21.15.40:0/3742458967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5f30156be0 con 0x7f5f1c0039a0 2023-10-18T08:58:18.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.017+0000 7f5f2d7fa640 1 -- 172.21.15.40:0/3742458967 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5f200440d0 con 0x7f5f1c0039a0 2023-10-18T08:58:18.021 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-10-18T08:58:18.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.021+0000 7f5f351bd640 1 -- 172.21.15.40:0/3742458967 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f1006ae70 msgr2=0x7f5f1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:18.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.021+0000 7f5f351bd640 1 --2- 172.21.15.40:0/3742458967 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f1006ae70 0x7f5f1006d330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5f3013adb0 tx=0x7f5f18005f90 comp rx=0 tx=0).stop 2023-10-18T08:58:18.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.021+0000 7f5f351bd640 1 -- 172.21.15.40:0/3742458967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1c0039a0 msgr2=0x7f5f301592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:18.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.021+0000 7f5f351bd640 1 --2- 172.21.15.40:0/3742458967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1c0039a0 0x7f5f301592b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5f2002c0b0 tx=0x7f5f200035f0 comp rx=0 tx=0).stop 2023-10-18T08:58:18.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.022+0000 7f5f351bd640 1 -- 172.21.15.40:0/3742458967 shutdown_connections 2023-10-18T08:58:18.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.022+0000 7f5f351bd640 1 --2- 172.21.15.40:0/3742458967 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f1006ae70 0x7f5f1006d330 secure :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5f3013adb0 tx=0x7f5f18005f90 comp rx=0 tx=0).stop 2023-10-18T08:58:18.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.022+0000 7f5f351bd640 1 --2- 172.21.15.40:0/3742458967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f301e5c30 0x7f5f301e60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:18.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.022+0000 7f5f351bd640 1 --2- 172.21.15.40:0/3742458967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f301553c0 0x7f5f30155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:18.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.022+0000 7f5f351bd640 1 --2- 172.21.15.40:0/3742458967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1c0039a0 0x7f5f301592b0 secure :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5f2002c0b0 tx=0x7f5f200035f0 comp rx=0 tx=0).stop 2023-10-18T08:58:18.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.022+0000 7f5f351bd640 1 -- 172.21.15.40:0/3742458967 >> 172.21.15.40:0/3742458967 conn(0x7f5f300aeb90 msgr2=0x7f5f30139c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:18.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.024+0000 7f5f351bd640 1 -- 172.21.15.40:0/3742458967 shutdown_connections 2023-10-18T08:58:18.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.024+0000 7f5f351bd640 1 -- 172.21.15.40:0/3742458967 wait complete. 2023-10-18T08:58:18.052 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 22 2023-10-18T08:58:18.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.266+0000 7fd6a2554640 1 -- 172.21.15.40:0/2776669406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6880039c0 msgr2=0x7fd688003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:18.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.266+0000 7fd6a2554640 1 --2- 172.21.15.40:0/2776669406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6880039c0 0x7fd688003de0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd69800bbb0 tx=0x7fd6880057f0 comp rx=0 tx=0).stop 2023-10-18T08:58:18.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.266+0000 7fd6a2554640 1 -- 172.21.15.40:0/2776669406 shutdown_connections 2023-10-18T08:58:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.266+0000 7fd6a2554640 1 --2- 172.21.15.40:0/2776669406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6880039c0 0x7fd688003de0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.266+0000 7fd6a2554640 1 -- 172.21.15.40:0/2776669406 >> 172.21.15.40:0/2776669406 conn(0x7fd69c069720 msgr2=0x7fd69c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.267+0000 7fd6a2554640 1 -- 172.21.15.40:0/2776669406 shutdown_connections 2023-10-18T08:58:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.267+0000 7fd6a2554640 1 -- 172.21.15.40:0/2776669406 wait complete. 2023-10-18T08:58:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.268+0000 7fd6a2554640 1 Processor -- start 2023-10-18T08:58:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.268+0000 7fd6a2554640 1 -- start start 2023-10-18T08:58:18.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.269+0000 7fd6a2554640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6880039c0 0x7fd69c141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:18.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.269+0000 7fd6a2554640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd69c14f850 0x7fd69c14fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:18.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.269+0000 7fd6a2554640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd69c13f0e0 0x7fd69c13f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:18.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.269+0000 7fd6a2554640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c0ae7e0 con 0x7fd6880039c0 2023-10-18T08:58:18.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.269+0000 7fd6a2554640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c0ad4f0 con 0x7fd69c14f850 2023-10-18T08:58:18.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.269+0000 7fd6a2554640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c0ae350 con 0x7fd69c13f0e0 2023-10-18T08:58:18.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.270+0000 7fd6a1552640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6880039c0 0x7fd69c141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:18.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.270+0000 7fd6a1552640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6880039c0 0x7fd69c141a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60994/0 (socket says 172.21.15.40:60994) 2023-10-18T08:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.270+0000 7fd6a1552640 1 -- 172.21.15.40:0/1896025264 learned_addr learned my addr 172.21.15.40:0/1896025264 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.270+0000 7fd6a1d53640 1 --2- 172.21.15.40:0/1896025264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd69c13f0e0 0x7fd69c13f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.270+0000 7fd6a1552640 1 -- 172.21.15.40:0/1896025264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd69c13f0e0 msgr2=0x7fd69c13f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.270+0000 7fd6a1552640 1 --2- 172.21.15.40:0/1896025264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd69c13f0e0 0x7fd69c13f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.270+0000 7fd6a0d51640 1 --2- 172.21.15.40:0/1896025264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd69c14f850 0x7fd69c14fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:18.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.270+0000 7fd6a1552640 1 -- 172.21.15.40:0/1896025264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd69c14f850 msgr2=0x7fd69c14fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:18.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.270+0000 7fd6a1552640 1 --2- 172.21.15.40:0/1896025264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd69c14f850 0x7fd69c14fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:18.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.270+0000 7fd6a1552640 1 -- 172.21.15.40:0/1896025264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c1501d0 con 0x7fd6880039c0 2023-10-18T08:58:18.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.270+0000 7fd6a0d51640 1 --2- 172.21.15.40:0/1896025264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd69c14f850 0x7fd69c14fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:18.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.271+0000 7fd6a1552640 1 --2- 172.21.15.40:0/1896025264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6880039c0 0x7fd69c141a00 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd69800bcd0 tx=0x7fd698036780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:18.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.271+0000 7fd691ffb640 1 -- 172.21.15.40:0/1896025264 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698036c40 con 0x7fd6880039c0 2023-10-18T08:58:18.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.271+0000 7fd691ffb640 1 -- 172.21.15.40:0/1896025264 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd698036e90 con 0x7fd6880039c0 2023-10-18T08:58:18.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.271+0000 7fd6a2554640 1 -- 172.21.15.40:0/1896025264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69c0ac090 con 0x7fd6880039c0 2023-10-18T08:58:18.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.271+0000 7fd691ffb640 1 -- 172.21.15.40:0/1896025264 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698005280 con 0x7fd6880039c0 2023-10-18T08:58:18.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.271+0000 7fd6a2554640 1 -- 172.21.15.40:0/1896025264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69c0ad740 con 0x7fd6880039c0 2023-10-18T08:58:18.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.272+0000 7fd6a2554640 1 -- 172.21.15.40:0/1896025264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69c066e00 con 0x7fd6880039c0 2023-10-18T08:58:18.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.273+0000 7fd691ffb640 1 -- 172.21.15.40:0/1896025264 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd6980053e0 con 0x7fd6880039c0 2023-10-18T08:58:18.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.275+0000 7fd691ffb640 1 --2- 172.21.15.40:0/1896025264 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd68006ae40 0x7fd68006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:18.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.275+0000 7fd691ffb640 1 -- 172.21.15.40:0/1896025264 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd6980abc00 con 0x7fd6880039c0 2023-10-18T08:58:18.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.275+0000 7fd6a0d51640 1 --2- 172.21.15.40:0/1896025264 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd68006ae40 0x7fd68006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:18.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.277+0000 7fd6a0d51640 1 --2- 172.21.15.40:0/1896025264 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd68006ae40 0x7fd68006d300 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd68c001eb0 tx=0x7fd68c001560 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:18.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.279+0000 7fd691ffb640 1 -- 172.21.15.40:0/1896025264 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd698076e80 con 0x7fd6880039c0 2023-10-18T08:58:18.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.472+0000 7fd6a2554640 1 -- 172.21.15.40:0/1896025264 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config reset", "num": 22} v 0) v1 -- 0x7fd69c0caf90 con 0x7fd6880039c0 2023-10-18T08:58:18.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.479+0000 7fd691ffb640 1 -- 172.21.15.40:0/1896025264 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 22}]=0 v26) v1 ==== 71+0+0 (secure 0 0 0) 0x7fd698032020 con 0x7fd6880039c0 2023-10-18T08:58:18.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.482+0000 7fd6a2554640 1 -- 172.21.15.40:0/1896025264 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd68006ae40 msgr2=0x7fd68006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:18.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.482+0000 7fd6a2554640 1 --2- 172.21.15.40:0/1896025264 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd68006ae40 0x7fd68006d300 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd68c001eb0 tx=0x7fd68c001560 comp rx=0 tx=0).stop 2023-10-18T08:58:18.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.482+0000 7fd6a2554640 1 -- 172.21.15.40:0/1896025264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6880039c0 msgr2=0x7fd69c141a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:18.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.482+0000 7fd6a2554640 1 --2- 172.21.15.40:0/1896025264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6880039c0 0x7fd69c141a00 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd69800bcd0 tx=0x7fd698036780 comp rx=0 tx=0).stop 2023-10-18T08:58:18.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.482+0000 7fd6a2554640 1 -- 172.21.15.40:0/1896025264 shutdown_connections 2023-10-18T08:58:18.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.482+0000 7fd6a2554640 1 --2- 172.21.15.40:0/1896025264 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd68006ae40 0x7fd68006d300 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:18.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.482+0000 7fd6a2554640 1 --2- 172.21.15.40:0/1896025264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd69c13f0e0 0x7fd69c13f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.482+0000 7fd6a2554640 1 --2- 172.21.15.40:0/1896025264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd69c14f850 0x7fd69c14fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.482+0000 7fd6a2554640 1 --2- 172.21.15.40:0/1896025264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6880039c0 0x7fd69c141a00 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.482+0000 7fd6a2554640 1 -- 172.21.15.40:0/1896025264 >> 172.21.15.40:0/1896025264 conn(0x7fd69c069720 msgr2=0x7fd69c15cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.482+0000 7fd6a2554640 1 -- 172.21.15.40:0/1896025264 shutdown_connections 2023-10-18T08:58:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.482+0000 7fd6a2554640 1 -- 172.21.15.40:0/1896025264 wait complete. 2023-10-18T08:58:18.505 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2023-10-18T08:58:18.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.732+0000 7f554e26d640 1 -- 172.21.15.40:0/1908602963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f554815b640 msgr2=0x7f55480d1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:18.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.732+0000 7f554e26d640 1 --2- 172.21.15.40:0/1908602963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f554815b640 0x7f55480d1a90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5544009830 tx=0x7f554402bc30 comp rx=0 tx=0).stop 2023-10-18T08:58:18.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.732+0000 7f554e26d640 1 -- 172.21.15.40:0/1908602963 shutdown_connections 2023-10-18T08:58:18.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.732+0000 7f554e26d640 1 --2- 172.21.15.40:0/1908602963 >> v2:172.21.15.144:3300/0 conn(0x7f55480d0020 0x7f55480ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:18.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.732+0000 7f554e26d640 1 --2- 172.21.15.40:0/1908602963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f554815b640 0x7f55480d1a90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:18.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.732+0000 7f554e26d640 1 -- 172.21.15.40:0/1908602963 >> 172.21.15.40:0/1908602963 conn(0x7f554808ad00 msgr2=0x7f554809a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:18.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.732+0000 7f554e26d640 1 -- 172.21.15.40:0/1908602963 shutdown_connections 2023-10-18T08:58:18.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.732+0000 7f554e26d640 1 -- 172.21.15.40:0/1908602963 wait complete. 2023-10-18T08:58:18.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.735+0000 7f554e26d640 1 Processor -- start 2023-10-18T08:58:18.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.736+0000 7f554e26d640 1 -- start start 2023-10-18T08:58:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.737+0000 7f554e26d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55480b01e0 0x7f55481526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:18.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.737+0000 7f554e26d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55480d0020 0x7f55481f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:18.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.737+0000 7f554e26d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f554815b640 0x7f55480c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:18.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.737+0000 7f554e26d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55480c8400 con 0x7f554815b640 2023-10-18T08:58:18.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.737+0000 7f554e26d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55480c8980 con 0x7f55480d0020 2023-10-18T08:58:18.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.737+0000 7f554e26d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55481f5c60 con 0x7f55480b01e0 2023-10-18T08:58:18.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.737+0000 7f554da6c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f554815b640 0x7f55480c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:18.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.737+0000 7f554da6c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f554815b640 0x7f55480c8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32768/0 (socket says 172.21.15.40:32768) 2023-10-18T08:58:18.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.737+0000 7f554da6c640 1 -- 172.21.15.40:0/978069935 learned_addr learned my addr 172.21.15.40:0/978069935 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:18.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.738+0000 7f554d26b640 1 --2- 172.21.15.40:0/978069935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55480b01e0 0x7f55481526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.738+0000 7f554da6c640 1 -- 172.21.15.40:0/978069935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55480b01e0 msgr2=0x7f55481526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.738+0000 7f554da6c640 1 --2- 172.21.15.40:0/978069935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55480b01e0 0x7f55481526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.738+0000 7f554da6c640 1 -- 172.21.15.40:0/978069935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55480d0020 msgr2=0x7f55481f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.738+0000 7f554ca6a640 1 --2- 172.21.15.40:0/978069935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55480d0020 0x7f55481f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:18.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.738+0000 7f554da6c640 1 --2- 172.21.15.40:0/978069935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55480d0020 0x7f55481f5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.738+0000 7f554da6c640 1 -- 172.21.15.40:0/978069935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5544009490 con 0x7f554815b640 2023-10-18T08:58:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.738+0000 7f554d26b640 1 --2- 172.21.15.40:0/978069935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55480b01e0 0x7f55481526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.738+0000 7f554ca6a640 1 --2- 172.21.15.40:0/978069935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55480d0020 0x7f55481f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:18.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.738+0000 7f554da6c640 1 --2- 172.21.15.40:0/978069935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f554815b640 0x7f55480c8e10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f554000eb10 tx=0x7f554000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:18.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.739+0000 7f553dffb640 1 -- 172.21.15.40:0/978069935 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554000ce80 con 0x7f554815b640 2023-10-18T08:58:18.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.739+0000 7f553dffb640 1 -- 172.21.15.40:0/978069935 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5540004c10 con 0x7f554815b640 2023-10-18T08:58:18.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.739+0000 7f553dffb640 1 -- 172.21.15.40:0/978069935 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5540005020 con 0x7f554815b640 2023-10-18T08:58:18.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.739+0000 7f554e26d640 1 -- 172.21.15.40:0/978069935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55480cada0 con 0x7f554815b640 2023-10-18T08:58:18.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.739+0000 7f554e26d640 1 -- 172.21.15.40:0/978069935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55480c8610 con 0x7f554815b640 2023-10-18T08:58:18.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.740+0000 7f554e26d640 1 -- 172.21.15.40:0/978069935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5548071da0 con 0x7f554815b640 2023-10-18T08:58:18.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.741+0000 7f553dffb640 1 -- 172.21.15.40:0/978069935 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5540005180 con 0x7f554815b640 2023-10-18T08:58:18.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.742+0000 7f553dffb640 1 --2- 172.21.15.40:0/978069935 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f552c06ae40 0x7f552c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:18.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.742+0000 7f553dffb640 1 -- 172.21.15.40:0/978069935 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5540014070 con 0x7f554815b640 2023-10-18T08:58:18.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.743+0000 7f554d26b640 1 --2- 172.21.15.40:0/978069935 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f552c06ae40 0x7f552c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:18.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.744+0000 7f554d26b640 1 --2- 172.21.15.40:0/978069935 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f552c06ae40 0x7f552c06d300 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5544036000 tx=0x7f5544004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:18.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:18.746+0000 7f553dffb640 1 -- 172.21.15.40:0/978069935 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55400555d0 con 0x7f554815b640 2023-10-18T08:58:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.018+0000 7f554e26d640 1 -- 172.21.15.40:0/978069935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f5548066e00 con 0x7f554815b640 2023-10-18T08:58:19.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.031+0000 7f553dffb640 1 -- 172.21.15.40:0/978069935 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v184) v1 ==== 142+0+0 (secure 0 0 0) 0x7f554001d0d0 con 0x7f554815b640 2023-10-18T08:58:19.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.033+0000 7f554e26d640 1 -- 172.21.15.40:0/978069935 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f552c06ae40 msgr2=0x7f552c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:19.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.033+0000 7f554e26d640 1 --2- 172.21.15.40:0/978069935 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f552c06ae40 0x7f552c06d300 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5544036000 tx=0x7f5544004730 comp rx=0 tx=0).stop 2023-10-18T08:58:19.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.033+0000 7f554e26d640 1 -- 172.21.15.40:0/978069935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f554815b640 msgr2=0x7f55480c8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:19.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.033+0000 7f554e26d640 1 --2- 172.21.15.40:0/978069935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f554815b640 0x7f55480c8e10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f554000eb10 tx=0x7f554000c2d0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.034+0000 7f554e26d640 1 -- 172.21.15.40:0/978069935 shutdown_connections 2023-10-18T08:58:19.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.034+0000 7f554e26d640 1 --2- 172.21.15.40:0/978069935 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f552c06ae40 0x7f552c06d300 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.034+0000 7f554e26d640 1 --2- 172.21.15.40:0/978069935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f554815b640 0x7f55480c8e10 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.034+0000 7f554e26d640 1 --2- 172.21.15.40:0/978069935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55480d0020 0x7f55481f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.034+0000 7f554e26d640 1 --2- 172.21.15.40:0/978069935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f55480b01e0 0x7f55481526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.034+0000 7f554e26d640 1 -- 172.21.15.40:0/978069935 >> 172.21.15.40:0/978069935 conn(0x7f554808ad00 msgr2=0x7f55480cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:19.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.034+0000 7f554e26d640 1 -- 172.21.15.40:0/978069935 shutdown_connections 2023-10-18T08:58:19.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.034+0000 7f554e26d640 1 -- 172.21.15.40:0/978069935 wait complete. 2023-10-18T08:58:19.057 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-10-18T08:58:19.057 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) 2023-10-18T08:58:19.058 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:58:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.286+0000 7f08363a9640 1 -- 172.21.15.40:0/2997967366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f083013f5a0 msgr2=0x7f08300c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.286+0000 7f08363a9640 1 --2- 172.21.15.40:0/2997967366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f083013f5a0 0x7f08300c9b00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0830143080 tx=0x7f081802bd70 comp rx=0 tx=0).stop 2023-10-18T08:58:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.287+0000 7f08363a9640 1 -- 172.21.15.40:0/2997967366 shutdown_connections 2023-10-18T08:58:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.287+0000 7f08363a9640 1 --2- 172.21.15.40:0/2997967366 >> v2:172.21.15.144:3300/0 conn(0x7f08300ca040 0x7f08300cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.287+0000 7f08363a9640 1 --2- 172.21.15.40:0/2997967366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f083013f5a0 0x7f08300c9b00 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.287+0000 7f08363a9640 1 -- 172.21.15.40:0/2997967366 >> 172.21.15.40:0/2997967366 conn(0x7f083006cee0 msgr2=0x7f0830069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:19.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.287+0000 7f08363a9640 1 -- 172.21.15.40:0/2997967366 shutdown_connections 2023-10-18T08:58:19.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.287+0000 7f08363a9640 1 -- 172.21.15.40:0/2997967366 wait complete. 2023-10-18T08:58:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.288+0000 7f08363a9640 1 Processor -- start 2023-10-18T08:58:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.289+0000 7f08363a9640 1 -- start start 2023-10-18T08:58:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f08363a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08300ca040 0x7f08301f4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f08363a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f083013c470 0x7f08301e30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f08363a9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f083013f5a0 0x7f083013d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f08363a9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08300ae2a0 con 0x7f083013c470 2023-10-18T08:58:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f08363a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08301e3610 con 0x7f08300ca040 2023-10-18T08:58:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f08363a9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f083013c050 con 0x7f083013f5a0 2023-10-18T08:58:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f0827fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f083013c470 0x7f08301e30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:19.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f0827fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f083013c470 0x7f08301e30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32786/0 (socket says 172.21.15.40:32786) 2023-10-18T08:58:19.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f0827fff640 1 -- 172.21.15.40:0/2014663059 learned_addr learned my addr 172.21.15.40:0/2014663059 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:19.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f0834946640 1 --2- 172.21.15.40:0/2014663059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f083013f5a0 0x7f083013d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:19.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f082ffff640 1 --2- 172.21.15.40:0/2014663059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08300ca040 0x7f08301f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:19.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f0827fff640 1 -- 172.21.15.40:0/2014663059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f083013f5a0 msgr2=0x7f083013d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f0827fff640 1 --2- 172.21.15.40:0/2014663059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f083013f5a0 0x7f083013d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f0827fff640 1 -- 172.21.15.40:0/2014663059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08300ca040 msgr2=0x7f08301f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f0827fff640 1 --2- 172.21.15.40:0/2014663059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08300ca040 0x7f08301f4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f0827fff640 1 -- 172.21.15.40:0/2014663059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0818009490 con 0x7f083013c470 2023-10-18T08:58:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.290+0000 7f082ffff640 1 --2- 172.21.15.40:0/2014663059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08300ca040 0x7f08301f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.291+0000 7f0827fff640 1 --2- 172.21.15.40:0/2014663059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f083013c470 0x7f08301e30d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0818000c00 tx=0x7f08180038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:19.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.291+0000 7f082d7fa640 1 -- 172.21.15.40:0/2014663059 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818003ea0 con 0x7f083013c470 2023-10-18T08:58:19.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.291+0000 7f082d7fa640 1 -- 172.21.15.40:0/2014663059 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0818034550 con 0x7f083013c470 2023-10-18T08:58:19.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.291+0000 7f08363a9640 1 -- 172.21.15.40:0/2014663059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f083013f130 con 0x7f083013c470 2023-10-18T08:58:19.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.291+0000 7f082d7fa640 1 -- 172.21.15.40:0/2014663059 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818034960 con 0x7f083013c470 2023-10-18T08:58:19.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.291+0000 7f08363a9640 1 -- 172.21.15.40:0/2014663059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08300ade10 con 0x7f083013c470 2023-10-18T08:58:19.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.292+0000 7f08363a9640 1 -- 172.21.15.40:0/2014663059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0830066fd0 con 0x7f083013c470 2023-10-18T08:58:19.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.293+0000 7f082d7fa640 1 -- 172.21.15.40:0/2014663059 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f081803b040 con 0x7f083013c470 2023-10-18T08:58:19.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.294+0000 7f082d7fa640 1 --2- 172.21.15.40:0/2014663059 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f081006ae40 0x7f081006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:19.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.294+0000 7f082d7fa640 1 -- 172.21.15.40:0/2014663059 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f08180aba50 con 0x7f083013c470 2023-10-18T08:58:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.295+0000 7f082ffff640 1 --2- 172.21.15.40:0/2014663059 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f081006ae40 0x7f081006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.296+0000 7f082ffff640 1 --2- 172.21.15.40:0/2014663059 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f081006ae40 0x7f081006d300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0820000f60 tx=0x7f0820009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:19.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.298+0000 7f082d7fa640 1 -- 172.21.15.40:0/2014663059 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0818076cd0 con 0x7f083013c470 2023-10-18T08:58:19.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.497+0000 7f08363a9640 1 -- 172.21.15.40:0/2014663059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08301e3840 con 0x7f083013c470 2023-10-18T08:58:19.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.499+0000 7f082d7fa640 1 -- 172.21.15.40:0/2014663059 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+13000 (secure 0 0 0) 0x7f0818039090 con 0x7f083013c470 2023-10-18T08:58:19.503 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:19.503 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":59,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:58:18.015377+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:6837/1438456692":"2023-10-19T08:58:14.680242+0000","172.21.15.144:6835/1438456692":"2023-10-19T08:58:14.680242+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:58:19.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.502+0000 7f08363a9640 1 -- 172.21.15.40:0/2014663059 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f081006ae40 msgr2=0x7f081006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.502+0000 7f08363a9640 1 --2- 172.21.15.40:0/2014663059 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f081006ae40 0x7f081006d300 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0820000f60 tx=0x7f0820009040 comp rx=0 tx=0).stop 2023-10-18T08:58:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.502+0000 7f08363a9640 1 -- 172.21.15.40:0/2014663059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f083013c470 msgr2=0x7f08301e30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.502+0000 7f08363a9640 1 --2- 172.21.15.40:0/2014663059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f083013c470 0x7f08301e30d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0818000c00 tx=0x7f08180038a0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.502+0000 7f08363a9640 1 -- 172.21.15.40:0/2014663059 shutdown_connections 2023-10-18T08:58:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.502+0000 7f08363a9640 1 --2- 172.21.15.40:0/2014663059 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f081006ae40 0x7f081006d300 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.502+0000 7f08363a9640 1 --2- 172.21.15.40:0/2014663059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f083013f5a0 0x7f083013d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.502+0000 7f08363a9640 1 --2- 172.21.15.40:0/2014663059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f083013c470 0x7f08301e30d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.502+0000 7f08363a9640 1 --2- 172.21.15.40:0/2014663059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08300ca040 0x7f08301f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.502+0000 7f08363a9640 1 -- 172.21.15.40:0/2014663059 >> 172.21.15.40:0/2014663059 conn(0x7f083006cee0 msgr2=0x7f08300ab520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:19.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.502+0000 7f08363a9640 1 -- 172.21.15.40:0/2014663059 shutdown_connections 2023-10-18T08:58:19.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.502+0000 7f08363a9640 1 -- 172.21.15.40:0/2014663059 wait complete. 2023-10-18T08:58:19.526 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:58:19.526 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:58:19.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.750+0000 7fda7759e640 1 -- 172.21.15.40:0/2802498233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda780d1960 msgr2=0x7fda780cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:19.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.750+0000 7fda7759e640 1 --2- 172.21.15.40:0/2802498233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda780d1960 0x7fda780cc220 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fda70005fd0 tx=0x7fda7002bea0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.751+0000 7fda7759e640 1 -- 172.21.15.40:0/2802498233 shutdown_connections 2023-10-18T08:58:19.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.751+0000 7fda7759e640 1 --2- 172.21.15.40:0/2802498233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda780d1960 0x7fda780cc220 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.751+0000 7fda7759e640 1 --2- 172.21.15.40:0/2802498233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda7815b4d0 0x7fda780addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.751+0000 7fda7759e640 1 -- 172.21.15.40:0/2802498233 >> 172.21.15.40:0/2802498233 conn(0x7fda7808ad30 msgr2=0x7fda7809a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:19.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.751+0000 7fda7759e640 1 -- 172.21.15.40:0/2802498233 shutdown_connections 2023-10-18T08:58:19.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.751+0000 7fda7759e640 1 -- 172.21.15.40:0/2802498233 wait complete. 2023-10-18T08:58:19.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.753+0000 7fda7759e640 1 Processor -- start 2023-10-18T08:58:19.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.753+0000 7fda7759e640 1 -- start start 2023-10-18T08:58:19.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda7759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda780d1960 0x7fda78159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:19.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda7759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda7815b4d0 0x7fda780aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:19.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda7759e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda7815b8b0 0x7fda780af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:19.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda7759e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda780c6400 con 0x7fda780d1960 2023-10-18T08:58:19.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda7759e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda780c7fc0 con 0x7fda7815b4d0 2023-10-18T08:58:19.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda7759e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda780b69b0 con 0x7fda7815b8b0 2023-10-18T08:58:19.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda7659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda780d1960 0x7fda78159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda7659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda780d1960 0x7fda78159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32800/0 (socket says 172.21.15.40:32800) 2023-10-18T08:58:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda7659c640 1 -- 172.21.15.40:0/1687418903 learned_addr learned my addr 172.21.15.40:0/1687418903 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda6dd9b640 1 --2- 172.21.15.40:0/1687418903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda7815b4d0 0x7fda780aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda7659c640 1 -- 172.21.15.40:0/1687418903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda7815b8b0 msgr2=0x7fda780af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda76d9d640 1 --2- 172.21.15.40:0/1687418903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda7815b8b0 0x7fda780af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda7659c640 1 --2- 172.21.15.40:0/1687418903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda7815b8b0 0x7fda780af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda7659c640 1 -- 172.21.15.40:0/1687418903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda7815b4d0 msgr2=0x7fda780aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda7659c640 1 --2- 172.21.15.40:0/1687418903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda7815b4d0 0x7fda780aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda7659c640 1 -- 172.21.15.40:0/1687418903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda70009560 con 0x7fda780d1960 2023-10-18T08:58:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.754+0000 7fda6dd9b640 1 --2- 172.21.15.40:0/1687418903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda7815b4d0 0x7fda780aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.755+0000 7fda7659c640 1 --2- 172.21.15.40:0/1687418903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda780d1960 0x7fda78159200 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fda6800e9f0 tx=0x7fda6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.755+0000 7fda6f7fe640 1 -- 172.21.15.40:0/1687418903 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68009700 con 0x7fda780d1960 2023-10-18T08:58:19.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.755+0000 7fda6f7fe640 1 -- 172.21.15.40:0/1687418903 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fda6800ce70 con 0x7fda780d1960 2023-10-18T08:58:19.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.755+0000 7fda6f7fe640 1 -- 172.21.15.40:0/1687418903 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda68010670 con 0x7fda780d1960 2023-10-18T08:58:19.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.755+0000 7fda7759e640 1 -- 172.21.15.40:0/1687418903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda780b6c90 con 0x7fda780d1960 2023-10-18T08:58:19.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.755+0000 7fda7759e640 1 -- 172.21.15.40:0/1687418903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda780c7be0 con 0x7fda780d1960 2023-10-18T08:58:19.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.756+0000 7fda7759e640 1 -- 172.21.15.40:0/1687418903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda78071da0 con 0x7fda780d1960 2023-10-18T08:58:19.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.757+0000 7fda6f7fe640 1 -- 172.21.15.40:0/1687418903 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fda680107d0 con 0x7fda780d1960 2023-10-18T08:58:19.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.759+0000 7fda6f7fe640 1 --2- 172.21.15.40:0/1687418903 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fda5806ae40 0x7fda5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:19.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.759+0000 7fda6f7fe640 1 -- 172.21.15.40:0/1687418903 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fda68014070 con 0x7fda780d1960 2023-10-18T08:58:19.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.760+0000 7fda6dd9b640 1 --2- 172.21.15.40:0/1687418903 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fda5806ae40 0x7fda5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:19.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.761+0000 7fda6dd9b640 1 --2- 172.21.15.40:0/1687418903 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fda5806ae40 0x7fda5806d300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fda78071290 tx=0x7fda60009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:19.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.763+0000 7fda6f7fe640 1 -- 172.21.15.40:0/1687418903 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda68051ca0 con 0x7fda780d1960 2023-10-18T08:58:19.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.956+0000 7fda7759e640 1 -- 172.21.15.40:0/1687418903 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda78066e00 con 0x7fda780d1960 2023-10-18T08:58:19.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.957+0000 7fda6f7fe640 1 -- 172.21.15.40:0/1687418903 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda68058e40 con 0x7fda780d1960 2023-10-18T08:58:19.960 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:58:19.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.960+0000 7fda7759e640 1 -- 172.21.15.40:0/1687418903 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fda5806ae40 msgr2=0x7fda5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:19.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.960+0000 7fda7759e640 1 --2- 172.21.15.40:0/1687418903 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fda5806ae40 0x7fda5806d300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fda78071290 tx=0x7fda60009040 comp rx=0 tx=0).stop 2023-10-18T08:58:19.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.960+0000 7fda7759e640 1 -- 172.21.15.40:0/1687418903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda780d1960 msgr2=0x7fda78159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:19.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.960+0000 7fda7759e640 1 --2- 172.21.15.40:0/1687418903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda780d1960 0x7fda78159200 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fda6800e9f0 tx=0x7fda6800eeb0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.960+0000 7fda7759e640 1 -- 172.21.15.40:0/1687418903 shutdown_connections 2023-10-18T08:58:19.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.960+0000 7fda7759e640 1 --2- 172.21.15.40:0/1687418903 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fda5806ae40 0x7fda5806d300 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.960+0000 7fda7759e640 1 --2- 172.21.15.40:0/1687418903 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda7815b8b0 0x7fda780af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.960+0000 7fda7759e640 1 --2- 172.21.15.40:0/1687418903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda7815b4d0 0x7fda780aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.960+0000 7fda7759e640 1 --2- 172.21.15.40:0/1687418903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda780d1960 0x7fda78159200 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:19.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.960+0000 7fda7759e640 1 -- 172.21.15.40:0/1687418903 >> 172.21.15.40:0/1687418903 conn(0x7fda7808ad30 msgr2=0x7fda7815dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:19.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.960+0000 7fda7759e640 1 -- 172.21.15.40:0/1687418903 shutdown_connections 2023-10-18T08:58:19.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:19.960+0000 7fda7759e640 1 -- 172.21.15.40:0/1687418903 wait complete. 2023-10-18T08:58:19.983 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2023-10-18T08:58:20.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.210+0000 7fd25ffff640 1 -- 172.21.15.40:0/2719258656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd248003ff0 msgr2=0x7fd2480047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:20.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.210+0000 7fd25ffff640 1 --2- 172.21.15.40:0/2719258656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd248003ff0 0x7fd2480047c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd25400bb20 tx=0x7fd2480057f0 comp rx=0 tx=0).stop 2023-10-18T08:58:20.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.210+0000 7fd25f7fe640 1 -- 172.21.15.40:0/2719258656 reap_dead start 2023-10-18T08:58:20.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.210+0000 7fd25ffff640 1 -- 172.21.15.40:0/2719258656 shutdown_connections 2023-10-18T08:58:20.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.210+0000 7fd25ffff640 1 -- 172.21.15.40:0/2719258656 >> 172.21.15.40:0/2719258656 conn(0x7fd260069720 msgr2=0x7fd260069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:20.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.210+0000 7fd25ffff640 1 -- 172.21.15.40:0/2719258656 shutdown_connections 2023-10-18T08:58:20.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.210+0000 7fd25ffff640 1 -- 172.21.15.40:0/2719258656 wait complete. 2023-10-18T08:58:20.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.212+0000 7fd25ffff640 1 Processor -- start 2023-10-18T08:58:20.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.212+0000 7fd25ffff640 1 -- start start 2023-10-18T08:58:20.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.213+0000 7fd25ffff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd248003ff0 0x7fd2600ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:20.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.213+0000 7fd25ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2600ad8c0 0x7fd2601582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:20.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.213+0000 7fd25ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2601f1cd0 0x7fd2601e5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:20.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.213+0000 7fd25ffff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600aea20 con 0x7fd2601f1cd0 2023-10-18T08:58:20.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.213+0000 7fd25ffff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600adda0 con 0x7fd2600ad8c0 2023-10-18T08:58:20.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.213+0000 7fd25ffff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600ae080 con 0x7fd248003ff0 2023-10-18T08:58:20.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.213+0000 7fd25f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2601f1cd0 0x7fd2601e5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:20.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.213+0000 7fd25f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2601f1cd0 0x7fd2601e5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32828/0 (socket says 172.21.15.40:32828) 2023-10-18T08:58:20.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.213+0000 7fd25f7fe640 1 -- 172.21.15.40:0/3504042758 learned_addr learned my addr 172.21.15.40:0/3504042758 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:20.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.213+0000 7fd25effd640 1 --2- 172.21.15.40:0/3504042758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd248003ff0 0x7fd2600ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:20.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.214+0000 7fd25e7fc640 1 --2- 172.21.15.40:0/3504042758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2600ad8c0 0x7fd2601582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:20.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.214+0000 7fd25f7fe640 1 -- 172.21.15.40:0/3504042758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd248003ff0 msgr2=0x7fd2600ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:20.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.214+0000 7fd25f7fe640 1 --2- 172.21.15.40:0/3504042758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd248003ff0 0x7fd2600ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:20.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.214+0000 7fd25f7fe640 1 -- 172.21.15.40:0/3504042758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2600ad8c0 msgr2=0x7fd2601582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:20.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.214+0000 7fd25f7fe640 1 --2- 172.21.15.40:0/3504042758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2600ad8c0 0x7fd2601582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.214+0000 7fd25f7fe640 1 -- 172.21.15.40:0/3504042758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2600ae360 con 0x7fd2601f1cd0 2023-10-18T08:58:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.214+0000 7fd25effd640 1 --2- 172.21.15.40:0/3504042758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd248003ff0 0x7fd2600ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.214+0000 7fd25e7fc640 1 --2- 172.21.15.40:0/3504042758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2600ad8c0 0x7fd2601582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.214+0000 7fd25f7fe640 1 --2- 172.21.15.40:0/3504042758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2601f1cd0 0x7fd2601e5060 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd2600b20e0 tx=0x7fd258009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.214+0000 7fd2477fe640 1 -- 172.21.15.40:0/3504042758 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd258018070 con 0x7fd2601f1cd0 2023-10-18T08:58:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.214+0000 7fd2477fe640 1 -- 172.21.15.40:0/3504042758 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd258004b20 con 0x7fd2601f1cd0 2023-10-18T08:58:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.214+0000 7fd2477fe640 1 -- 172.21.15.40:0/3504042758 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd258011710 con 0x7fd2601f1cd0 2023-10-18T08:58:20.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.214+0000 7fd25ffff640 1 -- 172.21.15.40:0/3504042758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2600ae640 con 0x7fd2601f1cd0 2023-10-18T08:58:20.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.214+0000 7fd25ffff640 1 -- 172.21.15.40:0/3504042758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd260158800 con 0x7fd2601f1cd0 2023-10-18T08:58:20.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.216+0000 7fd25ffff640 1 -- 172.21.15.40:0/3504042758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2600aeb60 con 0x7fd2601f1cd0 2023-10-18T08:58:20.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.216+0000 7fd2477fe640 1 -- 172.21.15.40:0/3504042758 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd258011870 con 0x7fd2601f1cd0 2023-10-18T08:58:20.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.217+0000 7fd2477fe640 1 --2- 172.21.15.40:0/3504042758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd24c06ae40 0x7fd24c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.217+0000 7fd2477fe640 1 -- 172.21.15.40:0/3504042758 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd258089830 con 0x7fd2601f1cd0 2023-10-18T08:58:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.218+0000 7fd25effd640 1 --2- 172.21.15.40:0/3504042758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd24c06ae40 0x7fd24c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.219+0000 7fd25effd640 1 --2- 172.21.15.40:0/3504042758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd24c06ae40 0x7fd24c06d300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd250001860 tx=0x7fd250001c80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.221+0000 7fd2477fe640 1 -- 172.21.15.40:0/3504042758 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd2600aeb60 con 0x7fd2601f1cd0 2023-10-18T08:58:20.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:20.479+0000 7fd25ffff640 1 -- 172.21.15.40:0/3504042758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7fd2600aeb60 con 0x7fd2601f1cd0 2023-10-18T08:58:21.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.036+0000 7fd2477fe640 1 -- 172.21.15.40:0/3504042758 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v186) v1 ==== 137+0+0 (secure 0 0 0) 0x7fd2600aeb60 con 0x7fd2601f1cd0 2023-10-18T08:58:21.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.040+0000 7fd25ffff640 1 -- 172.21.15.40:0/3504042758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd24c06ae40 msgr2=0x7fd24c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.040+0000 7fd25ffff640 1 --2- 172.21.15.40:0/3504042758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd24c06ae40 0x7fd24c06d300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd250001860 tx=0x7fd250001c80 comp rx=0 tx=0).stop 2023-10-18T08:58:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.040+0000 7fd25ffff640 1 -- 172.21.15.40:0/3504042758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2601f1cd0 msgr2=0x7fd2601e5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.040+0000 7fd25ffff640 1 --2- 172.21.15.40:0/3504042758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2601f1cd0 0x7fd2601e5060 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd2600b20e0 tx=0x7fd258009a80 comp rx=0 tx=0).stop 2023-10-18T08:58:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.040+0000 7fd25ffff640 1 -- 172.21.15.40:0/3504042758 shutdown_connections 2023-10-18T08:58:21.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.040+0000 7fd25ffff640 1 --2- 172.21.15.40:0/3504042758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd24c06ae40 0x7fd24c06d300 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:21.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.040+0000 7fd25ffff640 1 --2- 172.21.15.40:0/3504042758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2601f1cd0 0x7fd2601e5060 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:21.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.040+0000 7fd25ffff640 1 --2- 172.21.15.40:0/3504042758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2600ad8c0 0x7fd2601582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:21.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.040+0000 7fd25ffff640 1 --2- 172.21.15.40:0/3504042758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd248003ff0 0x7fd2600ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:21.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.040+0000 7fd25ffff640 1 -- 172.21.15.40:0/3504042758 >> 172.21.15.40:0/3504042758 conn(0x7fd260069720 msgr2=0x7fd2600abc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:21.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.040+0000 7fd25ffff640 1 -- 172.21.15.40:0/3504042758 shutdown_connections 2023-10-18T08:58:21.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.040+0000 7fd25ffff640 1 -- 172.21.15.40:0/3504042758 wait complete. 2023-10-18T08:58:21.065 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-18T08:58:21.313 INFO:teuthology.orchestra.run.smithi040.stdout:{"osd_objectstore":"memstore"} 2023-10-18T08:58:21.314 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-18T08:58:21.314 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T08:58:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.542+0000 7ff30f7c0640 1 -- 172.21.15.40:0/1775523346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff308151cb0 msgr2=0x7ff308158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.542+0000 7ff30f7c0640 1 --2- 172.21.15.40:0/1775523346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff308151cb0 0x7ff308158ca0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff304009830 tx=0x7ff30402bd50 comp rx=0 tx=0).stop 2023-10-18T08:58:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.542+0000 7ff30f7c0640 1 -- 172.21.15.40:0/1775523346 shutdown_connections 2023-10-18T08:58:21.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.542+0000 7ff30f7c0640 1 --2- 172.21.15.40:0/1775523346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff30806fea0 0x7ff308070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.542+0000 7ff30f7c0640 1 --2- 172.21.15.40:0/1775523346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff308151cb0 0x7ff308158ca0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.542+0000 7ff30f7c0640 1 -- 172.21.15.40:0/1775523346 >> 172.21.15.40:0/1775523346 conn(0x7ff308069720 msgr2=0x7ff308069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.542+0000 7ff30f7c0640 1 -- 172.21.15.40:0/1775523346 shutdown_connections 2023-10-18T08:58:21.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.542+0000 7ff30f7c0640 1 -- 172.21.15.40:0/1775523346 wait complete. 2023-10-18T08:58:21.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.544+0000 7ff30f7c0640 1 Processor -- start 2023-10-18T08:58:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.544+0000 7ff30f7c0640 1 -- start start 2023-10-18T08:58:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.545+0000 7ff30f7c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff30806fea0 0x7ff308144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.545+0000 7ff30f7c0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff308151cb0 0x7ff3080ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:21.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.545+0000 7ff30f7c0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff308157c00 0x7ff3080aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.545+0000 7ff30f7c0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30814aea0 con 0x7ff30806fea0 2023-10-18T08:58:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.545+0000 7ff30f7c0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff308144c40 con 0x7ff308157c00 2023-10-18T08:58:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.545+0000 7ff30f7c0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30814a360 con 0x7ff308151cb0 2023-10-18T08:58:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.545+0000 7ff30d55c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff30806fea0 0x7ff308144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:21.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.546+0000 7ff30d55c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff30806fea0 0x7ff308144700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32840/0 (socket says 172.21.15.40:32840) 2023-10-18T08:58:21.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.546+0000 7ff30d55c640 1 -- 172.21.15.40:0/1304542458 learned_addr learned my addr 172.21.15.40:0/1304542458 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.546+0000 7ff30cd5b640 1 --2- 172.21.15.40:0/1304542458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff308151cb0 0x7ff3080ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:21.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.546+0000 7ff30d55c640 1 -- 172.21.15.40:0/1304542458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff308151cb0 msgr2=0x7ff3080ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.546+0000 7ff30d55c640 1 --2- 172.21.15.40:0/1304542458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff308151cb0 0x7ff3080ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:21.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.546+0000 7ff30d55c640 1 -- 172.21.15.40:0/1304542458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff308157c00 msgr2=0x7ff3080aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.546+0000 7ff30d55c640 1 --2- 172.21.15.40:0/1304542458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff308157c00 0x7ff3080aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.546+0000 7ff30d55c640 1 -- 172.21.15.40:0/1304542458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff304009490 con 0x7ff30806fea0 2023-10-18T08:58:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.546+0000 7ff30d55c640 1 --2- 172.21.15.40:0/1304542458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff30806fea0 0x7ff308144700 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff304000c00 tx=0x7ff3040038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.547+0000 7ff2fdffb640 1 -- 172.21.15.40:0/1304542458 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304003ea0 con 0x7ff30806fea0 2023-10-18T08:58:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.547+0000 7ff2fdffb640 1 -- 172.21.15.40:0/1304542458 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff304034440 con 0x7ff30806fea0 2023-10-18T08:58:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.547+0000 7ff2fdffb640 1 -- 172.21.15.40:0/1304542458 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304034850 con 0x7ff30806fea0 2023-10-18T08:58:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.547+0000 7ff30f7c0640 1 -- 172.21.15.40:0/1304542458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3080ae110 con 0x7ff30806fea0 2023-10-18T08:58:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.547+0000 7ff30f7c0640 1 -- 172.21.15.40:0/1304542458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30814a600 con 0x7ff30806fea0 2023-10-18T08:58:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.548+0000 7ff30f7c0640 1 -- 172.21.15.40:0/1304542458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3080ae590 con 0x7ff30806fea0 2023-10-18T08:58:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.549+0000 7ff2fdffb640 1 -- 172.21.15.40:0/1304542458 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff3040349b0 con 0x7ff30806fea0 2023-10-18T08:58:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.550+0000 7ff2fdffb640 1 --2- 172.21.15.40:0/1304542458 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2ec06ae40 0x7ff2ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:21.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.550+0000 7ff2fdffb640 1 -- 172.21.15.40:0/1304542458 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff3040abc30 con 0x7ff30806fea0 2023-10-18T08:58:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.551+0000 7ff30cd5b640 1 --2- 172.21.15.40:0/1304542458 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2ec06ae40 0x7ff2ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.552+0000 7ff30cd5b640 1 --2- 172.21.15.40:0/1304542458 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2ec06ae40 0x7ff2ec06d300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff30814a220 tx=0x7ff2f8005cc0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.555+0000 7ff2fdffb640 1 -- 172.21.15.40:0/1304542458 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff304076f40 con 0x7ff30806fea0 2023-10-18T08:58:21.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.750+0000 7ff30f7c0640 1 -- 172.21.15.40:0/1304542458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7ff308066fd0 con 0x7ff30806fea0 2023-10-18T08:58:21.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.751+0000 7ff2fdffb640 1 -- 172.21.15.40:0/1304542458 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v26) v1 ==== 59+0+1452 (secure 0 0 0) 0x7ff304039020 con 0x7ff30806fea0 2023-10-18T08:58:21.755 INFO:teuthology.orchestra.run.smithi040.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T08:58:21.755 INFO:teuthology.orchestra.run.smithi040.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T08:58:21.755 INFO:teuthology.orchestra.run.smithi040.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T08:58:21.756 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds 20/20 2023-10-18T08:58:21.756 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T08:58:21.756 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_ms 1/1 2023-10-18T08:58:21.756 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T08:58:21.756 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T08:58:21.757 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T08:58:21.757 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_size 100 2023-10-18T08:58:21.757 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_frag true 2023-10-18T08:58:21.757 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_scatterstat true 2023-10-18T08:58:21.757 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T08:58:21.758 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_verify_scatter true 2023-10-18T08:58:21.758 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T08:58:21.758 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T08:58:21.758 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T08:58:21.758 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced client_mount_timeout 600 2023-10-18T08:58:21.759 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_client 20/20 2023-10-18T08:58:21.759 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_ms 1/1 2023-10-18T08:58:21.759 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T08:58:21.759 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T08:58:21.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.754+0000 7ff30f7c0640 1 -- 172.21.15.40:0/1304542458 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2ec06ae40 msgr2=0x7ff2ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:21.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.754+0000 7ff30f7c0640 1 --2- 172.21.15.40:0/1304542458 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2ec06ae40 0x7ff2ec06d300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff30814a220 tx=0x7ff2f8005cc0 comp rx=0 tx=0).stop 2023-10-18T08:58:21.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.754+0000 7ff30f7c0640 1 -- 172.21.15.40:0/1304542458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff30806fea0 msgr2=0x7ff308144700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:21.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.754+0000 7ff30f7c0640 1 --2- 172.21.15.40:0/1304542458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff30806fea0 0x7ff308144700 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff304000c00 tx=0x7ff3040038a0 comp rx=0 tx=0).stop 2023-10-18T08:58:21.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.754+0000 7ff30f7c0640 1 -- 172.21.15.40:0/1304542458 shutdown_connections 2023-10-18T08:58:21.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.754+0000 7ff30f7c0640 1 --2- 172.21.15.40:0/1304542458 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2ec06ae40 0x7ff2ec06d300 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:21.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.754+0000 7ff30f7c0640 1 --2- 172.21.15.40:0/1304542458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff308157c00 0x7ff3080aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:21.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.754+0000 7ff30f7c0640 1 --2- 172.21.15.40:0/1304542458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff308151cb0 0x7ff3080ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:21.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.754+0000 7ff30f7c0640 1 --2- 172.21.15.40:0/1304542458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff30806fea0 0x7ff308144700 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:21.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.754+0000 7ff30f7c0640 1 -- 172.21.15.40:0/1304542458 >> 172.21.15.40:0/1304542458 conn(0x7ff308069720 msgr2=0x7ff308069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:21.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.755+0000 7ff30f7c0640 1 -- 172.21.15.40:0/1304542458 shutdown_connections 2023-10-18T08:58:21.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:21.755+0000 7ff30f7c0640 1 -- 172.21.15.40:0/1304542458 wait complete. 2023-10-18T08:58:21.776 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T08:58:22.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.007+0000 7efe415f5640 1 -- 172.21.15.40:0/1434512024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe280051d0 msgr2=0x7efe2800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.007+0000 7efe415f5640 1 --2- 172.21.15.40:0/1434512024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe280051d0 0x7efe2800b980 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7efe34001680 tx=0x7efe280057f0 comp rx=0 tx=0).stop 2023-10-18T08:58:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.007+0000 7efe415f5640 1 -- 172.21.15.40:0/1434512024 shutdown_connections 2023-10-18T08:58:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.007+0000 7efe415f5640 1 --2- 172.21.15.40:0/1434512024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe280051d0 0x7efe2800b980 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:22.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.007+0000 7efe415f5640 1 -- 172.21.15.40:0/1434512024 >> 172.21.15.40:0/1434512024 conn(0x7efe3c069720 msgr2=0x7efe3c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.007+0000 7efe415f5640 1 -- 172.21.15.40:0/1434512024 shutdown_connections 2023-10-18T08:58:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.007+0000 7efe415f5640 1 -- 172.21.15.40:0/1434512024 wait complete. 2023-10-18T08:58:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.009+0000 7efe415f5640 1 Processor -- start 2023-10-18T08:58:22.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.009+0000 7efe415f5640 1 -- start start 2023-10-18T08:58:22.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.010+0000 7efe415f5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe280051d0 0x7efe3c1f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:22.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.010+0000 7efe415f5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe3c13acc0 0x7efe3c13a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:22.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.010+0000 7efe415f5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe3c0b1360 0x7efe3c15bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:22.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.010+0000 7efe415f5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c1f4d40 con 0x7efe280051d0 2023-10-18T08:58:22.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.010+0000 7efe415f5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c13aa70 con 0x7efe3c0b1360 2023-10-18T08:58:22.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.010+0000 7efe415f5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0b1c60 con 0x7efe3c13acc0 2023-10-18T08:58:22.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.010+0000 7efe3affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe280051d0 0x7efe3c1f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:22.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.010+0000 7efe3affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe280051d0 0x7efe3c1f5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32852/0 (socket says 172.21.15.40:32852) 2023-10-18T08:58:22.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.010+0000 7efe3affd640 1 -- 172.21.15.40:0/3384345311 learned_addr learned my addr 172.21.15.40:0/3384345311 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:22.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.011+0000 7efe3b7fe640 1 --2- 172.21.15.40:0/3384345311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe3c0b1360 0x7efe3c15bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:22.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.011+0000 7efe3affd640 1 -- 172.21.15.40:0/3384345311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe3c13acc0 msgr2=0x7efe3c13a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:22.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.011+0000 7efe3a7fc640 1 --2- 172.21.15.40:0/3384345311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe3c13acc0 0x7efe3c13a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:22.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.011+0000 7efe3affd640 1 --2- 172.21.15.40:0/3384345311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe3c13acc0 0x7efe3c13a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:22.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.011+0000 7efe3affd640 1 -- 172.21.15.40:0/3384345311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe3c0b1360 msgr2=0x7efe3c15bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:22.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.011+0000 7efe3affd640 1 --2- 172.21.15.40:0/3384345311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe3c0b1360 0x7efe3c15bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:22.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.011+0000 7efe3affd640 1 -- 172.21.15.40:0/3384345311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c0b1f40 con 0x7efe280051d0 2023-10-18T08:58:22.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.011+0000 7efe3b7fe640 1 --2- 172.21.15.40:0/3384345311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe3c0b1360 0x7efe3c15bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:22.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.011+0000 7efe3affd640 1 --2- 172.21.15.40:0/3384345311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe280051d0 0x7efe3c1f5790 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00da70 tx=0x7efe2c00df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:22.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.011+0000 7efe337fe640 1 -- 172.21.15.40:0/3384345311 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c003b80 con 0x7efe280051d0 2023-10-18T08:58:22.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.012+0000 7efe337fe640 1 -- 172.21.15.40:0/3384345311 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efe2c003ce0 con 0x7efe280051d0 2023-10-18T08:58:22.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.012+0000 7efe337fe640 1 -- 172.21.15.40:0/3384345311 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c00baa0 con 0x7efe280051d0 2023-10-18T08:58:22.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.012+0000 7efe415f5640 1 -- 172.21.15.40:0/3384345311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3c1f4f90 con 0x7efe280051d0 2023-10-18T08:58:22.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.012+0000 7efe415f5640 1 -- 172.21.15.40:0/3384345311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c0b17c0 con 0x7efe280051d0 2023-10-18T08:58:22.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.013+0000 7efe415f5640 1 -- 172.21.15.40:0/3384345311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3c066fd0 con 0x7efe280051d0 2023-10-18T08:58:22.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.014+0000 7efe337fe640 1 -- 172.21.15.40:0/3384345311 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efe2c00bc00 con 0x7efe280051d0 2023-10-18T08:58:22.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.015+0000 7efe337fe640 1 --2- 172.21.15.40:0/3384345311 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe1c06ae40 0x7efe1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:22.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.015+0000 7efe337fe640 1 -- 172.21.15.40:0/3384345311 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efe2c088fd0 con 0x7efe280051d0 2023-10-18T08:58:22.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.015+0000 7efe3a7fc640 1 --2- 172.21.15.40:0/3384345311 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe1c06ae40 0x7efe1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:22.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.017+0000 7efe3a7fc640 1 --2- 172.21.15.40:0/3384345311 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe1c06ae40 0x7efe1c06d300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0ac9c0 tx=0x7efe24005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:22.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.019+0000 7efe337fe640 1 -- 172.21.15.40:0/3384345311 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe2c0542e0 con 0x7efe280051d0 2023-10-18T08:58:22.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.217+0000 7efe415f5640 1 -- 172.21.15.40:0/3384345311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7efe3c15a720 con 0x7efe280051d0 2023-10-18T08:58:22.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.224+0000 7efe337fe640 1 -- 172.21.15.40:0/3384345311 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v27)=0 v27) v1 ==== 122+0+0 (secure 0 0 0) 0x7efe2c0190c0 con 0x7efe280051d0 2023-10-18T08:58:22.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.227+0000 7efe415f5640 1 -- 172.21.15.40:0/3384345311 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe1c06ae40 msgr2=0x7efe1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:22.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.227+0000 7efe415f5640 1 --2- 172.21.15.40:0/3384345311 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe1c06ae40 0x7efe1c06d300 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0ac9c0 tx=0x7efe24005f90 comp rx=0 tx=0).stop 2023-10-18T08:58:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.227+0000 7efe415f5640 1 -- 172.21.15.40:0/3384345311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe280051d0 msgr2=0x7efe3c1f5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.227+0000 7efe415f5640 1 --2- 172.21.15.40:0/3384345311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe280051d0 0x7efe3c1f5790 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00da70 tx=0x7efe2c00df30 comp rx=0 tx=0).stop 2023-10-18T08:58:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.227+0000 7efe415f5640 1 -- 172.21.15.40:0/3384345311 shutdown_connections 2023-10-18T08:58:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.227+0000 7efe415f5640 1 --2- 172.21.15.40:0/3384345311 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe1c06ae40 0x7efe1c06d300 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:22.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.227+0000 7efe415f5640 1 --2- 172.21.15.40:0/3384345311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe3c0b1360 0x7efe3c15bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:22.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.227+0000 7efe415f5640 1 --2- 172.21.15.40:0/3384345311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe3c13acc0 0x7efe3c13a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:22.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.227+0000 7efe415f5640 1 --2- 172.21.15.40:0/3384345311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe280051d0 0x7efe3c1f5790 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:22.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.227+0000 7efe415f5640 1 -- 172.21.15.40:0/3384345311 >> 172.21.15.40:0/3384345311 conn(0x7efe3c069720 msgr2=0x7efe3c13b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:22.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.227+0000 7efe415f5640 1 -- 172.21.15.40:0/3384345311 shutdown_connections 2023-10-18T08:58:22.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.227+0000 7efe415f5640 1 -- 172.21.15.40:0/3384345311 wait complete. 2023-10-18T08:58:22.251 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T08:58:22.252 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T08:58:22.376 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T08:58:22.502 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T08:58:22.502 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:58:22.503 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:58:22.566 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T08:58:22.566 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:58:22.567 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:58:22.628 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T08:58:22.628 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:58:22.629 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:58:22.693 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T08:58:22.694 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:58:22.694 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:58:22.754 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T08:58:22.755 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T08:58:22.755 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:58:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.995+0000 7f2a2bfff640 1 -- 172.21.15.40:0/4128179620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18004a80 msgr2=0x7f2a18004e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:23.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.995+0000 7f2a2bfff640 1 --2- 172.21.15.40:0/4128179620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18004a80 0x7f2a18004e80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00baf0 tx=0x7f2a18006900 comp rx=0 tx=0).stop 2023-10-18T08:58:23.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.995+0000 7f2a2bfff640 1 -- 172.21.15.40:0/4128179620 shutdown_connections 2023-10-18T08:58:23.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.995+0000 7f2a2bfff640 1 --2- 172.21.15.40:0/4128179620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a18004a80 0x7f2a18004e80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:23.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.995+0000 7f2a2bfff640 1 -- 172.21.15.40:0/4128179620 >> 172.21.15.40:0/4128179620 conn(0x7f2a2c069720 msgr2=0x7f2a2c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:23.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.995+0000 7f2a2bfff640 1 -- 172.21.15.40:0/4128179620 shutdown_connections 2023-10-18T08:58:23.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.995+0000 7f2a2bfff640 1 -- 172.21.15.40:0/4128179620 wait complete. 2023-10-18T08:58:23.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.997+0000 7f2a2bfff640 1 Processor -- start 2023-10-18T08:58:23.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.997+0000 7f2a2bfff640 1 -- start start 2023-10-18T08:58:23.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.999+0000 7f2a2bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a18004a80 0x7f2a2c155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:23.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.999+0000 7f2a2bfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a2c144840 0x7f2a2c144cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:23.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.999+0000 7f2a2bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a2c1f5280 0x7f2a2c1f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:23.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.999+0000 7f2a2bfff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c156e40 con 0x7f2a2c1f5280 2023-10-18T08:58:23.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.999+0000 7f2a2bfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c157650 con 0x7f2a18004a80 2023-10-18T08:58:23.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.999+0000 7f2a2bfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c145200 con 0x7f2a2c144840 2023-10-18T08:58:23.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.999+0000 7f2a2b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a2c1f5280 0x7f2a2c1f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:23.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.999+0000 7f2a2b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a2c1f5280 0x7f2a2c1f5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32884/0 (socket says 172.21.15.40:32884) 2023-10-18T08:58:23.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.999+0000 7f2a2b7fe640 1 -- 172.21.15.40:0/3301565615 learned_addr learned my addr 172.21.15.40:0/3301565615 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:23.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.999+0000 7f2a2affd640 1 --2- 172.21.15.40:0/3301565615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a18004a80 0x7f2a2c155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:23.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:22.999+0000 7f2a23fff640 1 --2- 172.21.15.40:0/3301565615 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a2c144840 0x7f2a2c144cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:23.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.000+0000 7f2a2b7fe640 1 -- 172.21.15.40:0/3301565615 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a2c144840 msgr2=0x7f2a2c144cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:23.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.000+0000 7f2a2b7fe640 1 --2- 172.21.15.40:0/3301565615 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a2c144840 0x7f2a2c144cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:23.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.000+0000 7f2a2b7fe640 1 -- 172.21.15.40:0/3301565615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a18004a80 msgr2=0x7f2a2c155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:23.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.000+0000 7f2a2b7fe640 1 --2- 172.21.15.40:0/3301565615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a18004a80 0x7f2a2c155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:23.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.000+0000 7f2a2b7fe640 1 -- 172.21.15.40:0/3301565615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c15bef0 con 0x7f2a2c1f5280 2023-10-18T08:58:23.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.000+0000 7f2a2b7fe640 1 --2- 172.21.15.40:0/3301565615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a2c1f5280 0x7f2a2c1f5700 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0cad50 tx=0x7f2a2400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:23.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.000+0000 7f2a237fe640 1 -- 172.21.15.40:0/3301565615 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24003c50 con 0x7f2a2c1f5280 2023-10-18T08:58:23.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.000+0000 7f2a237fe640 1 -- 172.21.15.40:0/3301565615 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2a240107e0 con 0x7f2a2c1f5280 2023-10-18T08:58:23.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.000+0000 7f2a2bfff640 1 -- 172.21.15.40:0/3301565615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c157090 con 0x7f2a2c1f5280 2023-10-18T08:58:23.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.000+0000 7f2a237fe640 1 -- 172.21.15.40:0/3301565615 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24010c30 con 0x7f2a2c1f5280 2023-10-18T08:58:23.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.001+0000 7f2a2bfff640 1 -- 172.21.15.40:0/3301565615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c0aa990 con 0x7f2a2c1f5280 2023-10-18T08:58:23.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.002+0000 7f2a2bfff640 1 -- 172.21.15.40:0/3301565615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2c066e00 con 0x7f2a2c1f5280 2023-10-18T08:58:23.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.003+0000 7f2a237fe640 1 -- 172.21.15.40:0/3301565615 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2a2401c3b0 con 0x7f2a2c1f5280 2023-10-18T08:58:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.004+0000 7f2a237fe640 1 --2- 172.21.15.40:0/3301565615 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a0c06ae40 0x7f2a0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.004+0000 7f2a237fe640 1 -- 172.21.15.40:0/3301565615 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2a24088cc0 con 0x7f2a2c1f5280 2023-10-18T08:58:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.005+0000 7f2a2affd640 1 --2- 172.21.15.40:0/3301565615 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a0c06ae40 0x7f2a0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.006+0000 7f2a2affd640 1 --2- 172.21.15.40:0/3301565615 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a0c06ae40 0x7f2a0c06d300 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c037000 tx=0x7f2a1c02e040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:23.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.008+0000 7f2a237fe640 1 -- 172.21.15.40:0/3301565615 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a240541b0 con 0x7f2a2c1f5280 2023-10-18T08:58:23.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.250+0000 7f2a2bfff640 1 -- 172.21.15.40:0/3301565615 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a2c0caf90 con 0x7f2a2c1f5280 2023-10-18T08:58:23.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.251+0000 7f2a237fe640 1 -- 172.21.15.40:0/3301565615 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f2a240577d0 con 0x7f2a2c1f5280 2023-10-18T08:58:23.255 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:23.255 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/3668356540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3668356540},{"type":"v1","addr":"172.21.15.144:6836","nonce":3668356540}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5326,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/615577896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":615577896},{"type":"v1","addr":"172.21.15.144:6837","nonce":615577896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/4222679844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":4222679844},{"type":"v1","addr":"172.21.15.40:6834","nonce":4222679844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2829399368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2829399368},{"type":"v1","addr":"172.21.15.40:6835","nonce":2829399368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2023-10-18T08:58:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 26 2023-10-18T08:58:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.254+0000 7f2a2bfff640 1 -- 172.21.15.40:0/3301565615 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a0c06ae40 msgr2=0x7f2a0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.254+0000 7f2a2bfff640 1 --2- 172.21.15.40:0/3301565615 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a0c06ae40 0x7f2a0c06d300 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c037000 tx=0x7f2a1c02e040 comp rx=0 tx=0).stop 2023-10-18T08:58:23.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.254+0000 7f2a2bfff640 1 -- 172.21.15.40:0/3301565615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a2c1f5280 msgr2=0x7f2a2c1f5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:23.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.254+0000 7f2a2bfff640 1 --2- 172.21.15.40:0/3301565615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a2c1f5280 0x7f2a2c1f5700 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0cad50 tx=0x7f2a2400c870 comp rx=0 tx=0).stop 2023-10-18T08:58:23.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.254+0000 7f2a2bfff640 1 -- 172.21.15.40:0/3301565615 shutdown_connections 2023-10-18T08:58:23.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.254+0000 7f2a2bfff640 1 --2- 172.21.15.40:0/3301565615 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a0c06ae40 0x7f2a0c06d300 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:23.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.254+0000 7f2a2bfff640 1 --2- 172.21.15.40:0/3301565615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a2c1f5280 0x7f2a2c1f5700 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:23.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.254+0000 7f2a2bfff640 1 --2- 172.21.15.40:0/3301565615 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a2c144840 0x7f2a2c144cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:23.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.254+0000 7f2a2bfff640 1 --2- 172.21.15.40:0/3301565615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a18004a80 0x7f2a2c155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.254+0000 7f2a2bfff640 1 -- 172.21.15.40:0/3301565615 >> 172.21.15.40:0/3301565615 conn(0x7f2a2c069720 msgr2=0x7f2a2c153c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.255+0000 7f2a2bfff640 1 -- 172.21.15.40:0/3301565615 shutdown_connections 2023-10-18T08:58:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.255+0000 7f2a2bfff640 1 -- 172.21.15.40:0/3301565615 wait complete. 2023-10-18T08:58:23.277 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T08:58:23.277 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T08:58:23.278 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T08:58:23.278 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T08:58:23.278 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T08:58:23.279 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T08:58:23.279 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T08:58:23.279 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T08:58:23.280 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T08:58:23.280 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T08:58:23.280 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T08:58:23.280 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T08:58:23.281 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T08:58:23.281 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T08:58:23.281 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T08:58:23.282 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T08:58:23.298 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T08:58:23.298 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T08:58:23.300 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T08:58:23.300 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T08:58:23.304 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T08:58:23.304 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T08:58:23.306 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T08:58:23.307 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T08:58:23.348 INFO:tasks.ceph.mds.c:Started 2023-10-18T08:58:23.349 DEBUG:teuthology.parallel:result is None 2023-10-18T08:58:23.354 INFO:tasks.ceph.mds.b:Started 2023-10-18T08:58:23.354 DEBUG:teuthology.parallel:result is None 2023-10-18T08:58:23.356 INFO:tasks.ceph.mds.a:Started 2023-10-18T08:58:23.357 DEBUG:teuthology.parallel:result is None 2023-10-18T08:58:23.361 INFO:tasks.ceph.mds.d:Started 2023-10-18T08:58:23.362 DEBUG:teuthology.parallel:result is None 2023-10-18T08:58:23.362 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T08:58:23.515 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-18T08:58:23.517 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T08:58:23.518 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-18T08:58:23.520 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-10-18T08:58:23.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.564+0000 7f0ccbd65640 1 -- 172.21.15.40:0/3632971143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc40b1b90 msgr2=0x7f0cc40b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:23.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.564+0000 7f0ccbd65640 1 --2- 172.21.15.40:0/3632971143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc40b1b90 0x7f0cc40b07d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0cc406d720 tx=0x7f0cbc02c390 comp rx=0 tx=0).stop 2023-10-18T08:58:23.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.564+0000 7f0ccbd65640 1 -- 172.21.15.40:0/3632971143 shutdown_connections 2023-10-18T08:58:23.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.564+0000 7f0ccbd65640 1 --2- 172.21.15.40:0/3632971143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc40b1b90 0x7f0cc40b07d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:23.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.564+0000 7f0ccbd65640 1 --2- 172.21.15.40:0/3632971143 >> v2:172.21.15.144:3300/0 conn(0x7f0cc40cd4c0 0x7f0cc40b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:23.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.564+0000 7f0ccbd65640 1 -- 172.21.15.40:0/3632971143 >> 172.21.15.40:0/3632971143 conn(0x7f0cc406cee0 msgr2=0x7f0cc4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:23.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.564+0000 7f0ccbd65640 1 -- 172.21.15.40:0/3632971143 shutdown_connections 2023-10-18T08:58:23.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.564+0000 7f0ccbd65640 1 -- 172.21.15.40:0/3632971143 wait complete. 2023-10-18T08:58:23.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.566+0000 7f0ccbd65640 1 Processor -- start 2023-10-18T08:58:23.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.566+0000 7f0ccbd65640 1 -- start start 2023-10-18T08:58:23.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0ccbd65640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cc40b1b90 0x7f0cc4157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:23.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0ccbd65640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc40cd4c0 0x7f0cc40ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:23.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0ccbd65640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0cc415eaa0 0x7f0cc4155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:23.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0ccbd65640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc41591a0 con 0x7f0cc40cd4c0 2023-10-18T08:58:23.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0ccbd65640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4159480 con 0x7f0cc40b1b90 2023-10-18T08:58:23.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0ccbd65640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4159760 con 0x7f0cc415eaa0 2023-10-18T08:58:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0cc9300640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc40cd4c0 0x7f0cc40ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0cc9300640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc40cd4c0 0x7f0cc40ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32914/0 (socket says 172.21.15.40:32914) 2023-10-18T08:58:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0cc9300640 1 -- 172.21.15.40:0/2023118532 learned_addr learned my addr 172.21.15.40:0/2023118532 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0cca302640 1 --2- 172.21.15.40:0/2023118532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0cc415eaa0 0x7f0cc4155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0cc9b01640 1 --2- 172.21.15.40:0/2023118532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cc40b1b90 0x7f0cc4157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0cc9300640 1 -- 172.21.15.40:0/2023118532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0cc415eaa0 msgr2=0x7f0cc4155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0cc9300640 1 --2- 172.21.15.40:0/2023118532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0cc415eaa0 0x7f0cc4155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0cc9300640 1 -- 172.21.15.40:0/2023118532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cc40b1b90 msgr2=0x7f0cc4157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0cc9300640 1 --2- 172.21.15.40:0/2023118532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cc40b1b90 0x7f0cc4157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0cc9300640 1 -- 172.21.15.40:0/2023118532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc009900 con 0x7f0cc40cd4c0 2023-10-18T08:58:23.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.567+0000 7f0cca302640 1 --2- 172.21.15.40:0/2023118532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0cc415eaa0 0x7f0cc4155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:58:23.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.568+0000 7f0cc9300640 1 --2- 172.21.15.40:0/2023118532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc40cd4c0 0x7f0cc40ad870 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400e9f0 tx=0x7f0cb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:23.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.568+0000 7f0cba7fc640 1 -- 172.21.15.40:0/2023118532 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4009700 con 0x7f0cc40cd4c0 2023-10-18T08:58:23.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.568+0000 7f0cba7fc640 1 -- 172.21.15.40:0/2023118532 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0cb400ce70 con 0x7f0cc40cd4c0 2023-10-18T08:58:23.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.568+0000 7f0cba7fc640 1 -- 172.21.15.40:0/2023118532 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4010670 con 0x7f0cc40cd4c0 2023-10-18T08:58:23.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.568+0000 7f0ccbd65640 1 -- 172.21.15.40:0/2023118532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc4159a40 con 0x7f0cc40cd4c0 2023-10-18T08:58:23.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.568+0000 7f0ccbd65640 1 -- 172.21.15.40:0/2023118532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc4157790 con 0x7f0cc40cd4c0 2023-10-18T08:58:23.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.569+0000 7f0ccbd65640 1 -- 172.21.15.40:0/2023118532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc4066fd0 con 0x7f0cc40cd4c0 2023-10-18T08:58:23.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.570+0000 7f0cba7fc640 1 -- 172.21.15.40:0/2023118532 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0cb40107d0 con 0x7f0cc40cd4c0 2023-10-18T08:58:23.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.571+0000 7f0cba7fc640 1 --2- 172.21.15.40:0/2023118532 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ca806ae40 0x7f0ca806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:23.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.571+0000 7f0cba7fc640 1 -- 172.21.15.40:0/2023118532 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0cb4014070 con 0x7f0cc40cd4c0 2023-10-18T08:58:23.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.572+0000 7f0cc9b01640 1 --2- 172.21.15.40:0/2023118532 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ca806ae40 0x7f0ca806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:23.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.572+0000 7f0cc9b01640 1 --2- 172.21.15.40:0/2023118532 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ca806ae40 0x7f0ca806d300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0000f60 tx=0x7f0cc0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:23.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.575+0000 7f0cba7fc640 1 -- 172.21.15.40:0/2023118532 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0cb4051ca0 con 0x7f0cc40cd4c0 2023-10-18T08:58:23.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:23.782+0000 7f0ccbd65640 1 -- 172.21.15.40:0/2023118532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0cc40aa950 con 0x7f0cc40cd4c0 2023-10-18T08:58:24.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.251+0000 7f0cba7fc640 1 -- 172.21.15.40:0/2023118532 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v61) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0cb4058e40 con 0x7f0cc40cd4c0 2023-10-18T08:58:24.254 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2023-10-18T08:58:24.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.254+0000 7f0ccbd65640 1 -- 172.21.15.40:0/2023118532 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ca806ae40 msgr2=0x7f0ca806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:24.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.254+0000 7f0ccbd65640 1 --2- 172.21.15.40:0/2023118532 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ca806ae40 0x7f0ca806d300 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0000f60 tx=0x7f0cc0009040 comp rx=0 tx=0).stop 2023-10-18T08:58:24.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.254+0000 7f0ccbd65640 1 -- 172.21.15.40:0/2023118532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc40cd4c0 msgr2=0x7f0cc40ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:24.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.254+0000 7f0ccbd65640 1 --2- 172.21.15.40:0/2023118532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc40cd4c0 0x7f0cc40ad870 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400e9f0 tx=0x7f0cb400eeb0 comp rx=0 tx=0).stop 2023-10-18T08:58:24.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.254+0000 7f0ccbd65640 1 -- 172.21.15.40:0/2023118532 shutdown_connections 2023-10-18T08:58:24.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.254+0000 7f0ccbd65640 1 --2- 172.21.15.40:0/2023118532 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ca806ae40 0x7f0ca806d300 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:24.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.254+0000 7f0ccbd65640 1 --2- 172.21.15.40:0/2023118532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0cc415eaa0 0x7f0cc4155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:24.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.254+0000 7f0ccbd65640 1 --2- 172.21.15.40:0/2023118532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc40cd4c0 0x7f0cc40ad870 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:24.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.254+0000 7f0ccbd65640 1 --2- 172.21.15.40:0/2023118532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cc40b1b90 0x7f0cc4157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:24.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.254+0000 7f0ccbd65640 1 -- 172.21.15.40:0/2023118532 >> 172.21.15.40:0/2023118532 conn(0x7f0cc406cee0 msgr2=0x7f0cc40b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:24.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.255+0000 7f0ccbd65640 1 -- 172.21.15.40:0/2023118532 shutdown_connections 2023-10-18T08:58:24.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.255+0000 7f0ccbd65640 1 -- 172.21.15.40:0/2023118532 wait complete. 2023-10-18T08:58:24.279 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T08:58:24.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.497+0000 7fa7c0fe9640 1 -- 172.21.15.40:0/1416964297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc0ca890 msgr2=0x7fa7bc0b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:24.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.497+0000 7fa7c0fe9640 1 --2- 172.21.15.40:0/1416964297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc0ca890 0x7fa7bc0b1d10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc06a5a0 tx=0x7fa7a402bdb0 comp rx=0 tx=0).stop 2023-10-18T08:58:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.498+0000 7fa7c0fe9640 1 -- 172.21.15.40:0/1416964297 shutdown_connections 2023-10-18T08:58:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.498+0000 7fa7c0fe9640 1 --2- 172.21.15.40:0/1416964297 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc0ca890 0x7fa7bc0b1d10 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.498+0000 7fa7c0fe9640 1 --2- 172.21.15.40:0/1416964297 >> v2:172.21.15.144:3300/0 conn(0x7fa7bc157870 0x7fa7bc0ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.498+0000 7fa7c0fe9640 1 -- 172.21.15.40:0/1416964297 >> 172.21.15.40:0/1416964297 conn(0x7fa7bc069720 msgr2=0x7fa7bc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.498+0000 7fa7c0fe9640 1 -- 172.21.15.40:0/1416964297 shutdown_connections 2023-10-18T08:58:24.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.498+0000 7fa7c0fe9640 1 -- 172.21.15.40:0/1416964297 wait complete. 2023-10-18T08:58:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.500+0000 7fa7c0fe9640 1 Processor -- start 2023-10-18T08:58:24.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.500+0000 7fa7c0fe9640 1 -- start start 2023-10-18T08:58:24.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.501+0000 7fa7c0fe9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7bc0b2250 0x7fa7bc13ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.501+0000 7fa7c0fe9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc0ca890 0x7fa7bc0a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.501+0000 7fa7c0fe9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7bc157870 0x7fa7bc0a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.501+0000 7fa7c0fe9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc1e2c50 con 0x7fa7bc0ca890 2023-10-18T08:58:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.501+0000 7fa7c0fe9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc142820 con 0x7fa7bc0b2250 2023-10-18T08:58:24.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.501+0000 7fa7c0fe9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc1e2970 con 0x7fa7bc157870 2023-10-18T08:58:24.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.501+0000 7fa7b3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc0ca890 0x7fa7bc0a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:24.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.501+0000 7fa7b3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc0ca890 0x7fa7bc0a6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32928/0 (socket says 172.21.15.40:32928) 2023-10-18T08:58:24.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.501+0000 7fa7b3fff640 1 -- 172.21.15.40:0/1802735049 learned_addr learned my addr 172.21.15.40:0/1802735049 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:24.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.501+0000 7fa7bb7fe640 1 --2- 172.21.15.40:0/1802735049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7bc0b2250 0x7fa7bc13ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:24.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.502+0000 7fa7bbfff640 1 --2- 172.21.15.40:0/1802735049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7bc157870 0x7fa7bc0a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.502+0000 7fa7b3fff640 1 -- 172.21.15.40:0/1802735049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7bc157870 msgr2=0x7fa7bc0a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.502+0000 7fa7b3fff640 1 --2- 172.21.15.40:0/1802735049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7bc157870 0x7fa7bc0a6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.502+0000 7fa7b3fff640 1 -- 172.21.15.40:0/1802735049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7bc0b2250 msgr2=0x7fa7bc13ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.502+0000 7fa7b3fff640 1 --2- 172.21.15.40:0/1802735049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7bc0b2250 0x7fa7bc13ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.502+0000 7fa7b3fff640 1 -- 172.21.15.40:0/1802735049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a4009560 con 0x7fa7bc0ca890 2023-10-18T08:58:24.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.502+0000 7fa7b3fff640 1 --2- 172.21.15.40:0/1802735049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc0ca890 0x7fa7bc0a6590 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4036040 tx=0x7fa7a402dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:24.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.502+0000 7fa7b8ff9640 1 -- 172.21.15.40:0/1802735049 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4005650 con 0x7fa7bc0ca890 2023-10-18T08:58:24.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.502+0000 7fa7b8ff9640 1 -- 172.21.15.40:0/1802735049 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa7a40038f0 con 0x7fa7bc0ca890 2023-10-18T08:58:24.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.502+0000 7fa7b8ff9640 1 -- 172.21.15.40:0/1802735049 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4003ce0 con 0x7fa7bc0ca890 2023-10-18T08:58:24.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.502+0000 7fa7c0fe9640 1 -- 172.21.15.40:0/1802735049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc13f4b0 con 0x7fa7bc0ca890 2023-10-18T08:58:24.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.503+0000 7fa7c0fe9640 1 -- 172.21.15.40:0/1802735049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7bc13fa70 con 0x7fa7bc0ca890 2023-10-18T08:58:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.504+0000 7fa7c0fe9640 1 -- 172.21.15.40:0/1802735049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7bc13f310 con 0x7fa7bc0ca890 2023-10-18T08:58:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.505+0000 7fa7b8ff9640 1 -- 172.21.15.40:0/1802735049 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa7a403e450 con 0x7fa7bc0ca890 2023-10-18T08:58:24.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.506+0000 7fa7b8ff9640 1 --2- 172.21.15.40:0/1802735049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa79c06ae40 0x7fa79c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:24.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.506+0000 7fa7b8ff9640 1 -- 172.21.15.40:0/1802735049 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa7a40abc70 con 0x7fa7bc0ca890 2023-10-18T08:58:24.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.507+0000 7fa7bb7fe640 1 --2- 172.21.15.40:0/1802735049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa79c06ae40 0x7fa79c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:24.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.508+0000 7fa7bb7fe640 1 --2- 172.21.15.40:0/1802735049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa79c06ae40 0x7fa79c06d300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00b810 tx=0x7fa7ac008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:24.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.511+0000 7fa7b8ff9640 1 -- 172.21.15.40:0/1802735049 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa7bc13f310 con 0x7fa7bc0ca890 2023-10-18T08:58:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.775+0000 7fa7c0fe9640 1 -- 172.21.15.40:0/1802735049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa7bc13f310 con 0x7fa7bc0ca890 2023-10-18T08:58:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.776+0000 7fa7b8ff9640 1 -- 172.21.15.40:0/1802735049 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa7bc13f310 con 0x7fa7bc0ca890 2023-10-18T08:58:24.780 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:24.780 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T08:58:24.780 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2023-10-18T08:58:24.780 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.780 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2023-10-18T08:58:24.781 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lYsb5CxAAGButZ6+Usc62kJ0ksxZrYw==", 2023-10-18T08:58:24.781 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:58:24.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T08:58:24.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T08:58:24.782 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:58:24.782 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.782 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.783 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2023-10-18T08:58:24.783 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lAqlhMBAApAhxCYB9d9bz1GibADDRSQ==", 2023-10-18T08:58:24.783 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:58:24.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T08:58:24.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T08:58:24.784 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:58:24.784 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.785 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.785 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.785 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2023-10-18T08:58:24.785 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lPVh+HhAAzucSx+ksPDBkqJGa8+pyJw==", 2023-10-18T08:58:24.785 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:58:24.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T08:58:24.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T08:58:24.786 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:58:24.786 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.787 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2023-10-18T08:58:24.787 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lP7QCBhAAog7qDCgGVv0T7WKsRkATkA==", 2023-10-18T08:58:24.787 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:58:24.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T08:58:24.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T08:58:24.788 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:58:24.788 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.789 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2023-10-18T08:58:24.789 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lVx+hAhAA6ZpIyZKhbqy8tV84piGILA==", 2023-10-18T08:58:24.789 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:58:24.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:58:24.790 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:58:24.790 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.790 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.791 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2023-10-18T08:58:24.791 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lk9LwExAA7zFzQOn2hcvnck2cOZZ7AA==", 2023-10-18T08:58:24.791 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:58:24.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:58:24.792 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:58:24.792 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.792 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2023-10-18T08:58:24.792 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lK930JRAA1GyKRkIpXPHHGqY8owYWRQ==", 2023-10-18T08:58:24.793 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:58:24.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:58:24.793 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:58:24.793 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.794 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.794 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2023-10-18T08:58:24.794 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9liBWyNRAAMHBBdG5nbrGQupqvUtuKMA==", 2023-10-18T08:58:24.794 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:58:24.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:58:24.795 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:58:24.795 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.796 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.796 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2023-10-18T08:58:24.796 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9loDoOAhAAjHZub4wnB7dCr86rxAtI5g==", 2023-10-18T08:58:24.796 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.796 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:58:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:58:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:58:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2023-10-18T08:58:24.798 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lhqNfERAAw410g3DwSXrK23YkxVBFVw==", 2023-10-18T08:58:24.798 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.798 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:58:24.798 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:58:24.798 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:58:24.799 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.799 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.799 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.799 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2023-10-18T08:58:24.799 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lRRScIRAAg0pmfxCQBCPYodj1Lko+Lw==", 2023-10-18T08:58:24.800 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.800 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:58:24.800 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:58:24.800 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:58:24.800 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.801 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.801 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.801 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2023-10-18T08:58:24.801 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9ld1dXMhAAyovlrKOhVuynEiuPTgFcjQ==", 2023-10-18T08:58:24.801 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.802 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T08:58:24.802 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T08:58:24.802 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:58:24.802 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.802 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.803 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.803 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2023-10-18T08:58:24.803 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w==", 2023-10-18T08:58:24.803 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.803 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:58:24.804 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2023-10-18T08:58:24.804 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T08:58:24.804 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.804 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.804 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.805 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2023-10-18T08:58:24.805 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lAB3oKhAAD43WFdC23WGF6WUi4JKsvg==", 2023-10-18T08:58:24.805 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.805 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:58:24.805 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T08:58:24.806 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T08:58:24.806 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T08:58:24.806 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.806 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.806 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.807 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2023-10-18T08:58:24.807 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lu57fHxAAdzsVU2KKsWRkaARRS8TGjw==", 2023-10-18T08:58:24.807 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.807 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:58:24.807 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T08:58:24.807 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T08:58:24.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T08:58:24.808 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.808 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.808 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.808 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2023-10-18T08:58:24.809 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lygWsNRAA1OziXxk7hBtTw2Aq8OlSKQ==", 2023-10-18T08:58:24.809 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.809 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T08:58:24.809 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T08:58:24.809 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T08:58:24.809 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T08:58:24.810 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.810 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.810 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2023-10-18T08:58:24.810 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBMnS9lnrUjNhAAHP2eezRHF4/Jgox7iPwy7g==", 2023-10-18T08:58:24.811 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.811 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T08:58:24.811 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T08:58:24.811 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T08:58:24.811 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:58:24.812 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.812 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.812 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.812 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2023-10-18T08:58:24.812 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9lCf/xOhAAJHVoKIswWUcVGKQpiaJ12A==", 2023-10-18T08:58:24.812 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.813 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T08:58:24.813 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T08:58:24.813 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:58:24.813 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T08:58:24.814 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:24.814 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2023-10-18T08:58:24.814 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9loxRXLhAAQK1n1/4PB35umHxOKQzz7Q==", 2023-10-18T08:58:24.814 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T08:58:24.814 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T08:58:24.815 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T08:58:24.815 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T08:58:24.815 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.815 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:24.815 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T08:58:24.816 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T08:58:24.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.779+0000 7fa7c0fe9640 1 -- 172.21.15.40:0/1802735049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa79c06ae40 msgr2=0x7fa79c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:24.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.779+0000 7fa7c0fe9640 1 --2- 172.21.15.40:0/1802735049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa79c06ae40 0x7fa79c06d300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00b810 tx=0x7fa7ac008040 comp rx=0 tx=0).stop 2023-10-18T08:58:24.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.779+0000 7fa7c0fe9640 1 -- 172.21.15.40:0/1802735049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc0ca890 msgr2=0x7fa7bc0a6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:24.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.779+0000 7fa7c0fe9640 1 --2- 172.21.15.40:0/1802735049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc0ca890 0x7fa7bc0a6590 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4036040 tx=0x7fa7a402dd80 comp rx=0 tx=0).stop 2023-10-18T08:58:24.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.779+0000 7fa7c0fe9640 1 -- 172.21.15.40:0/1802735049 shutdown_connections 2023-10-18T08:58:24.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.779+0000 7fa7c0fe9640 1 --2- 172.21.15.40:0/1802735049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa79c06ae40 0x7fa79c06d300 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:24.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.779+0000 7fa7c0fe9640 1 --2- 172.21.15.40:0/1802735049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7bc157870 0x7fa7bc0a6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:24.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.779+0000 7fa7c0fe9640 1 --2- 172.21.15.40:0/1802735049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7bc0ca890 0x7fa7bc0a6590 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:24.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.779+0000 7fa7c0fe9640 1 --2- 172.21.15.40:0/1802735049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7bc0b2250 0x7fa7bc13ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:24.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.779+0000 7fa7c0fe9640 1 -- 172.21.15.40:0/1802735049 >> 172.21.15.40:0/1802735049 conn(0x7fa7bc069720 msgr2=0x7fa7bc0aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:24.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.779+0000 7fa7c0fe9640 1 -- 172.21.15.40:0/1802735049 shutdown_connections 2023-10-18T08:58:24.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:24.779+0000 7fa7c0fe9640 1 -- 172.21.15.40:0/1802735049 wait complete. 2023-10-18T08:58:24.820 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:58:25.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.051+0000 7f8036496640 1 -- 172.21.15.40:0/1832037599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80300cc990 msgr2=0x7f80300ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:25.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.051+0000 7f8036496640 1 --2- 172.21.15.40:0/1832037599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80300cc990 0x7f80300ccd90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f803006c840 tx=0x7f802802d1e0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.052+0000 7f8036496640 1 -- 172.21.15.40:0/1832037599 shutdown_connections 2023-10-18T08:58:25.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.052+0000 7f8036496640 1 --2- 172.21.15.40:0/1832037599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80300cc990 0x7f80300ccd90 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.052+0000 7f8036496640 1 --2- 172.21.15.40:0/1832037599 >> v2:172.21.15.144:3300/0 conn(0x7f80300b0bc0 0x7f80300b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.052+0000 7f8036496640 1 -- 172.21.15.40:0/1832037599 >> 172.21.15.40:0/1832037599 conn(0x7f803006cee0 msgr2=0x7f8030069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:25.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.052+0000 7f8036496640 1 -- 172.21.15.40:0/1832037599 shutdown_connections 2023-10-18T08:58:25.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.052+0000 7f8036496640 1 -- 172.21.15.40:0/1832037599 wait complete. 2023-10-18T08:58:25.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.054+0000 7f8036496640 1 Processor -- start 2023-10-18T08:58:25.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.054+0000 7f8036496640 1 -- start start 2023-10-18T08:58:25.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.055+0000 7f8036496640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80300b0bc0 0x7f8030159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:25.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.055+0000 7f8036496640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80300cc990 0x7f80301457e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:25.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.055+0000 7f8036496640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8030151bd0 0x7f8030145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:25.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.055+0000 7f8036496640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80301471d0 con 0x7f8030151bd0 2023-10-18T08:58:25.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.055+0000 7f8036496640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80301474b0 con 0x7f80300b0bc0 2023-10-18T08:58:25.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.055+0000 7f8036496640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803014ab80 con 0x7f80300cc990 2023-10-18T08:58:25.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.055+0000 7f8034a33640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8030151bd0 0x7f8030145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:25.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.055+0000 7f8034a33640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8030151bd0 0x7f8030145d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38294/0 (socket says 172.21.15.40:38294) 2023-10-18T08:58:25.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.055+0000 7f8034a33640 1 -- 172.21.15.40:0/4244657832 learned_addr learned my addr 172.21.15.40:0/4244657832 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:25.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.055+0000 7f802ffff640 1 --2- 172.21.15.40:0/4244657832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80300b0bc0 0x7f8030159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:25.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.055+0000 7f8027fff640 1 --2- 172.21.15.40:0/4244657832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80300cc990 0x7f80301457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:25.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.055+0000 7f8034a33640 1 -- 172.21.15.40:0/4244657832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80300cc990 msgr2=0x7f80301457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:25.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.056+0000 7f8034a33640 1 --2- 172.21.15.40:0/4244657832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80300cc990 0x7f80301457e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.056+0000 7f8034a33640 1 -- 172.21.15.40:0/4244657832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80300b0bc0 msgr2=0x7f8030159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:25.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.056+0000 7f8034a33640 1 --2- 172.21.15.40:0/4244657832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80300b0bc0 0x7f8030159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.056+0000 7f8034a33640 1 -- 172.21.15.40:0/4244657832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802800aa10 con 0x7f8030151bd0 2023-10-18T08:58:25.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.056+0000 7f8034a33640 1 --2- 172.21.15.40:0/4244657832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8030151bd0 0x7f8030145d20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8028038040 tx=0x7f8028036db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:25.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.056+0000 7f802d7fa640 1 -- 172.21.15.40:0/4244657832 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028035440 con 0x7f8030151bd0 2023-10-18T08:58:25.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.056+0000 7f802d7fa640 1 -- 172.21.15.40:0/4244657832 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8028004540 con 0x7f8030151bd0 2023-10-18T08:58:25.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.056+0000 7f8036496640 1 -- 172.21.15.40:0/4244657832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803014ae60 con 0x7f8030151bd0 2023-10-18T08:58:25.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.056+0000 7f802d7fa640 1 -- 172.21.15.40:0/4244657832 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028040450 con 0x7f8030151bd0 2023-10-18T08:58:25.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.056+0000 7f8036496640 1 -- 172.21.15.40:0/4244657832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8030159f60 con 0x7f8030151bd0 2023-10-18T08:58:25.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.058+0000 7f8036496640 1 -- 172.21.15.40:0/4244657832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803014b380 con 0x7f8030151bd0 2023-10-18T08:58:25.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.059+0000 7f802d7fa640 1 -- 172.21.15.40:0/4244657832 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f802803c070 con 0x7f8030151bd0 2023-10-18T08:58:25.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.059+0000 7f802d7fa640 1 --2- 172.21.15.40:0/4244657832 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f801006ae40 0x7f801006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:25.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.060+0000 7f802d7fa640 1 -- 172.21.15.40:0/4244657832 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f80280ac500 con 0x7f8030151bd0 2023-10-18T08:58:25.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.060+0000 7f802ffff640 1 --2- 172.21.15.40:0/4244657832 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f801006ae40 0x7f801006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:25.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.061+0000 7f802ffff640 1 --2- 172.21.15.40:0/4244657832 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f801006ae40 0x7f801006d300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8020000f60 tx=0x7f8020009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:25.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.064+0000 7f802d7fa640 1 -- 172.21.15.40:0/4244657832 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f803014b380 con 0x7f8030151bd0 2023-10-18T08:58:25.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.264+0000 7f8036496640 1 -- 172.21.15.40:0/4244657832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f803014b380 con 0x7f8030151bd0 2023-10-18T08:58:25.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.265+0000 7f802d7fa640 1 -- 172.21.15.40:0/4244657832 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+12869 (secure 0 0 0) 0x7f803014b380 con 0x7f8030151bd0 2023-10-18T08:58:25.268 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:25.269 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":63,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:58:24.266315+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:58:25.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.267+0000 7f8036496640 1 -- 172.21.15.40:0/4244657832 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f801006ae40 msgr2=0x7f801006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.267+0000 7f8036496640 1 --2- 172.21.15.40:0/4244657832 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f801006ae40 0x7f801006d300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8020000f60 tx=0x7f8020009040 comp rx=0 tx=0).stop 2023-10-18T08:58:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.267+0000 7f8036496640 1 -- 172.21.15.40:0/4244657832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8030151bd0 msgr2=0x7f8030145d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.267+0000 7f8036496640 1 --2- 172.21.15.40:0/4244657832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8030151bd0 0x7f8030145d20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8028038040 tx=0x7f8028036db0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.267+0000 7f8036496640 1 -- 172.21.15.40:0/4244657832 shutdown_connections 2023-10-18T08:58:25.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.267+0000 7f8036496640 1 --2- 172.21.15.40:0/4244657832 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f801006ae40 0x7f801006d300 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.267+0000 7f8036496640 1 --2- 172.21.15.40:0/4244657832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8030151bd0 0x7f8030145d20 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.267+0000 7f8036496640 1 --2- 172.21.15.40:0/4244657832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80300cc990 0x7f80301457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.267+0000 7f8036496640 1 --2- 172.21.15.40:0/4244657832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80300b0bc0 0x7f8030159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.267+0000 7f8036496640 1 -- 172.21.15.40:0/4244657832 >> 172.21.15.40:0/4244657832 conn(0x7f803006cee0 msgr2=0x7f80300712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:25.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.268+0000 7f8036496640 1 -- 172.21.15.40:0/4244657832 shutdown_connections 2023-10-18T08:58:25.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.268+0000 7f8036496640 1 -- 172.21.15.40:0/4244657832 wait complete. 2023-10-18T08:58:25.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:58:25.292 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:58:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.521+0000 7fc013fff640 1 -- 172.21.15.40:0/1442053839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc014144c10 msgr2=0x7fc01415b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:25.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.521+0000 7fc013fff640 1 --2- 172.21.15.40:0/1442053839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc014144c10 0x7fc01415b8d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc004005fd0 tx=0x7fc00402bd10 comp rx=0 tx=0).stop 2023-10-18T08:58:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.521+0000 7fc013fff640 1 -- 172.21.15.40:0/1442053839 shutdown_connections 2023-10-18T08:58:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.521+0000 7fc013fff640 1 --2- 172.21.15.40:0/1442053839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0140ca380 0x7fc01415f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.521+0000 7fc013fff640 1 --2- 172.21.15.40:0/1442053839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc014144c10 0x7fc01415b8d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.521+0000 7fc013fff640 1 -- 172.21.15.40:0/1442053839 >> 172.21.15.40:0/1442053839 conn(0x7fc014069720 msgr2=0x7fc014069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.522+0000 7fc013fff640 1 -- 172.21.15.40:0/1442053839 shutdown_connections 2023-10-18T08:58:25.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.522+0000 7fc013fff640 1 -- 172.21.15.40:0/1442053839 wait complete. 2023-10-18T08:58:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.523+0000 7fc013fff640 1 Processor -- start 2023-10-18T08:58:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.524+0000 7fc013fff640 1 -- start start 2023-10-18T08:58:25.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.524+0000 7fc013fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0140ca380 0x7fc0141f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:25.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.524+0000 7fc013fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc014144c10 0x7fc0141f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:25.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.525+0000 7fc013fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0140acba0 0x7fc01413e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:25.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.525+0000 7fc013fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0141f58e0 con 0x7fc0140acba0 2023-10-18T08:58:25.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.525+0000 7fc013fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0141f5570 con 0x7fc0140ca380 2023-10-18T08:58:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.525+0000 7fc013fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0141f2780 con 0x7fc014144c10 2023-10-18T08:58:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.525+0000 7fc0137fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0140acba0 0x7fc01413e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.525+0000 7fc0137fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0140acba0 0x7fc01413e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38310/0 (socket says 172.21.15.40:38310) 2023-10-18T08:58:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.525+0000 7fc0137fe640 1 -- 172.21.15.40:0/228809314 learned_addr learned my addr 172.21.15.40:0/228809314 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:25.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.525+0000 7fc012ffd640 1 --2- 172.21.15.40:0/228809314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0140ca380 0x7fc0141f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:25.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.525+0000 7fc00bfff640 1 --2- 172.21.15.40:0/228809314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc014144c10 0x7fc0141f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:25.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.525+0000 7fc0137fe640 1 -- 172.21.15.40:0/228809314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc014144c10 msgr2=0x7fc0141f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:25.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.525+0000 7fc0137fe640 1 --2- 172.21.15.40:0/228809314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc014144c10 0x7fc0141f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.525+0000 7fc0137fe640 1 -- 172.21.15.40:0/228809314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0140ca380 msgr2=0x7fc0141f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.525+0000 7fc0137fe640 1 --2- 172.21.15.40:0/228809314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0140ca380 0x7fc0141f1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.525+0000 7fc0137fe640 1 -- 172.21.15.40:0/228809314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc004009560 con 0x7fc0140acba0 2023-10-18T08:58:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.526+0000 7fc0137fe640 1 --2- 172.21.15.40:0/228809314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0140acba0 0x7fc01413e970 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00ba10 tx=0x7fc00c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:25.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.526+0000 7fc00b7fe640 1 -- 172.21.15.40:0/228809314 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c00caa0 con 0x7fc0140acba0 2023-10-18T08:58:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.526+0000 7fc00b7fe640 1 -- 172.21.15.40:0/228809314 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc00c00ccf0 con 0x7fc0140acba0 2023-10-18T08:58:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.526+0000 7fc00b7fe640 1 -- 172.21.15.40:0/228809314 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c00cea0 con 0x7fc0140acba0 2023-10-18T08:58:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.526+0000 7fc013fff640 1 -- 172.21.15.40:0/228809314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0141f2a60 con 0x7fc0140acba0 2023-10-18T08:58:25.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.526+0000 7fc013fff640 1 -- 172.21.15.40:0/228809314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0141f2d40 con 0x7fc0140acba0 2023-10-18T08:58:25.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.527+0000 7fc013fff640 1 -- 172.21.15.40:0/228809314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0141f5a20 con 0x7fc0140acba0 2023-10-18T08:58:25.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.529+0000 7fc00b7fe640 1 -- 172.21.15.40:0/228809314 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc00c01d3b0 con 0x7fc0140acba0 2023-10-18T08:58:25.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.529+0000 7fc00b7fe640 1 --2- 172.21.15.40:0/228809314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbff406ae40 0x7fbff406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:25.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.529+0000 7fc00b7fe640 1 -- 172.21.15.40:0/228809314 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc00c0893d0 con 0x7fc0140acba0 2023-10-18T08:58:25.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.530+0000 7fc012ffd640 1 --2- 172.21.15.40:0/228809314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbff406ae40 0x7fbff406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:25.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.531+0000 7fc012ffd640 1 --2- 172.21.15.40:0/228809314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbff406ae40 0x7fbff406d300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc004035000 tx=0x7fc004004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:25.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.534+0000 7fc00b7fe640 1 -- 172.21.15.40:0/228809314 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc0141f5a20 con 0x7fc0140acba0 2023-10-18T08:58:25.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.725+0000 7fc013fff640 1 -- 172.21.15.40:0/228809314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0141f5a20 con 0x7fc0140acba0 2023-10-18T08:58:25.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.726+0000 7fc00b7fe640 1 -- 172.21.15.40:0/228809314 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc00c019020 con 0x7fc0140acba0 2023-10-18T08:58:25.729 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:58:25.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.728+0000 7fc013fff640 1 -- 172.21.15.40:0/228809314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbff406ae40 msgr2=0x7fbff406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:25.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.728+0000 7fc013fff640 1 --2- 172.21.15.40:0/228809314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbff406ae40 0x7fbff406d300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc004035000 tx=0x7fc004004730 comp rx=0 tx=0).stop 2023-10-18T08:58:25.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.728+0000 7fc013fff640 1 -- 172.21.15.40:0/228809314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0140acba0 msgr2=0x7fc01413e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:25.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.728+0000 7fc013fff640 1 --2- 172.21.15.40:0/228809314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0140acba0 0x7fc01413e970 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00ba10 tx=0x7fc00c00bed0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.728+0000 7fc013fff640 1 -- 172.21.15.40:0/228809314 shutdown_connections 2023-10-18T08:58:25.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.728+0000 7fc013fff640 1 --2- 172.21.15.40:0/228809314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbff406ae40 0x7fbff406d300 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.728+0000 7fc013fff640 1 --2- 172.21.15.40:0/228809314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0140acba0 0x7fc01413e970 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.728+0000 7fc013fff640 1 --2- 172.21.15.40:0/228809314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc014144c10 0x7fc0141f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.728+0000 7fc013fff640 1 --2- 172.21.15.40:0/228809314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0140ca380 0x7fc0141f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.729+0000 7fc013fff640 1 -- 172.21.15.40:0/228809314 >> 172.21.15.40:0/228809314 conn(0x7fc014069720 msgr2=0x7fc014070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:25.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.729+0000 7fc013fff640 1 -- 172.21.15.40:0/228809314 shutdown_connections 2023-10-18T08:58:25.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.729+0000 7fc013fff640 1 -- 172.21.15.40:0/228809314 wait complete. 2023-10-18T08:58:25.751 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T08:58:25.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.985+0000 7f46107d0640 1 -- 172.21.15.40:0/3758889815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46080cc6d0 msgr2=0x7f46080ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.985+0000 7f46107d0640 1 --2- 172.21.15.40:0/3758889815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46080cc6d0 0x7f46080ccab0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4604005fd0 tx=0x7f460402d150 comp rx=0 tx=0).stop 2023-10-18T08:58:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.985+0000 7f46107d0640 1 -- 172.21.15.40:0/3758889815 shutdown_connections 2023-10-18T08:58:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.985+0000 7f46107d0640 1 --2- 172.21.15.40:0/3758889815 >> v2:172.21.15.144:3300/0 conn(0x7f46080b0db0 0x7f460806bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.985+0000 7f46107d0640 1 --2- 172.21.15.40:0/3758889815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46080cc6d0 0x7f46080ccab0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.985+0000 7f46107d0640 1 -- 172.21.15.40:0/3758889815 >> 172.21.15.40:0/3758889815 conn(0x7f460806cee0 msgr2=0x7f4608069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:25.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.985+0000 7f46107d0640 1 -- 172.21.15.40:0/3758889815 shutdown_connections 2023-10-18T08:58:25.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.985+0000 7f46107d0640 1 -- 172.21.15.40:0/3758889815 wait complete. 2023-10-18T08:58:25.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.987+0000 7f46107d0640 1 Processor -- start 2023-10-18T08:58:25.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.987+0000 7f46107d0640 1 -- start start 2023-10-18T08:58:25.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.988+0000 7f46107d0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f460806b710 0x7f460813bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:25.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.988+0000 7f46107d0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46080b0db0 0x7f4608149520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:25.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.988+0000 7f46107d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46080cc6d0 0x7f46081e3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:25.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.988+0000 7f46107d0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460814af30 con 0x7f46080b0db0 2023-10-18T08:58:25.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.988+0000 7f46107d0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460813c000 con 0x7f46080cc6d0 2023-10-18T08:58:25.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.988+0000 7f46107d0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4608149e90 con 0x7f460806b710 2023-10-18T08:58:25.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.988+0000 7f460dd6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46080b0db0 0x7f4608149520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:25.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.988+0000 7f460dd6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46080b0db0 0x7f4608149520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38314/0 (socket says 172.21.15.40:38314) 2023-10-18T08:58:25.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.988+0000 7f460dd6b640 1 -- 172.21.15.40:0/3107506116 learned_addr learned my addr 172.21.15.40:0/3107506116 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:25.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.989+0000 7f460e56c640 1 --2- 172.21.15.40:0/3107506116 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f460806b710 0x7f460813bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:25.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.989+0000 7f460ed6d640 1 --2- 172.21.15.40:0/3107506116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46080cc6d0 0x7f46081e3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:25.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.989+0000 7f460dd6b640 1 -- 172.21.15.40:0/3107506116 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f460806b710 msgr2=0x7f460813bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:25.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.989+0000 7f460dd6b640 1 --2- 172.21.15.40:0/3107506116 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f460806b710 0x7f460813bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.989+0000 7f460dd6b640 1 -- 172.21.15.40:0/3107506116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46080cc6d0 msgr2=0x7f46081e3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:25.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.989+0000 7f460dd6b640 1 --2- 172.21.15.40:0/3107506116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46080cc6d0 0x7f46081e3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:25.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.989+0000 7f460dd6b640 1 -- 172.21.15.40:0/3107506116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460400a670 con 0x7f46080b0db0 2023-10-18T08:58:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.989+0000 7f460dd6b640 1 --2- 172.21.15.40:0/3107506116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46080b0db0 0x7f4608149520 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f45f800c3c0 tx=0x7f45f800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.990+0000 7f45feffd640 1 -- 172.21.15.40:0/3107506116 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80035f0 con 0x7f46080b0db0 2023-10-18T08:58:25.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.990+0000 7f45feffd640 1 -- 172.21.15.40:0/3107506116 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f45f8003770 con 0x7f46080b0db0 2023-10-18T08:58:25.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.990+0000 7f45feffd640 1 -- 172.21.15.40:0/3107506116 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8003b80 con 0x7f46080b0db0 2023-10-18T08:58:25.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.990+0000 7f46107d0640 1 -- 172.21.15.40:0/3107506116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460814a170 con 0x7f46080b0db0 2023-10-18T08:58:25.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.990+0000 7f46107d0640 1 -- 172.21.15.40:0/3107506116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460814a730 con 0x7f46080b0db0 2023-10-18T08:58:25.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.991+0000 7f46107d0640 1 -- 172.21.15.40:0/3107506116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46080cd640 con 0x7f46080b0db0 2023-10-18T08:58:25.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.992+0000 7f45feffd640 1 -- 172.21.15.40:0/3107506116 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f45f8003ce0 con 0x7f46080b0db0 2023-10-18T08:58:25.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.993+0000 7f45feffd640 1 --2- 172.21.15.40:0/3107506116 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f45ec06ae40 0x7f45ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:25.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.994+0000 7f45feffd640 1 -- 172.21.15.40:0/3107506116 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f45f8088ad0 con 0x7f46080b0db0 2023-10-18T08:58:25.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.994+0000 7f460e56c640 1 --2- 172.21.15.40:0/3107506116 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f45ec06ae40 0x7f45ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:26.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.995+0000 7f460e56c640 1 --2- 172.21.15.40:0/3107506116 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f45ec06ae40 0x7f45ec06d300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4604036000 tx=0x7f4604004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:26.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:25.998+0000 7f45feffd640 1 -- 172.21.15.40:0/3107506116 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f45f8053da0 con 0x7f46080b0db0 2023-10-18T08:58:26.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.194+0000 7f46107d0640 1 -- 172.21.15.40:0/3107506116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4608066fd0 con 0x7f46080b0db0 2023-10-18T08:58:26.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.194+0000 7f45feffd640 1 -- 172.21.15.40:0/3107506116 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v63) v1 ==== 84+0+65 (secure 0 0 0) 0x7f45f8017020 con 0x7f46080b0db0 2023-10-18T08:58:26.198 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:26.198 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-10-18T08:58:26.199 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T08:58:26.199 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-10-18T08:58:26.199 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-10-18T08:58:26.199 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T08:58:26.199 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-10-18T08:58:26.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.197+0000 7f46107d0640 1 -- 172.21.15.40:0/3107506116 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f45ec06ae40 msgr2=0x7f45ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:26.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.197+0000 7f46107d0640 1 --2- 172.21.15.40:0/3107506116 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f45ec06ae40 0x7f45ec06d300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4604036000 tx=0x7f4604004730 comp rx=0 tx=0).stop 2023-10-18T08:58:26.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.197+0000 7f46107d0640 1 -- 172.21.15.40:0/3107506116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46080b0db0 msgr2=0x7f4608149520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:26.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.197+0000 7f46107d0640 1 --2- 172.21.15.40:0/3107506116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46080b0db0 0x7f4608149520 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f45f800c3c0 tx=0x7f45f800c880 comp rx=0 tx=0).stop 2023-10-18T08:58:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.197+0000 7f46107d0640 1 -- 172.21.15.40:0/3107506116 shutdown_connections 2023-10-18T08:58:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.197+0000 7f46107d0640 1 --2- 172.21.15.40:0/3107506116 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f45ec06ae40 0x7f45ec06d300 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.197+0000 7f46107d0640 1 --2- 172.21.15.40:0/3107506116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46080cc6d0 0x7f46081e3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.197+0000 7f46107d0640 1 --2- 172.21.15.40:0/3107506116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46080b0db0 0x7f4608149520 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:26.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.197+0000 7f46107d0640 1 --2- 172.21.15.40:0/3107506116 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f460806b710 0x7f460813bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:26.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.197+0000 7f46107d0640 1 -- 172.21.15.40:0/3107506116 >> 172.21.15.40:0/3107506116 conn(0x7f460806cee0 msgr2=0x7f460815a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:26.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.197+0000 7f46107d0640 1 -- 172.21.15.40:0/3107506116 shutdown_connections 2023-10-18T08:58:26.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.197+0000 7f46107d0640 1 -- 172.21.15.40:0/3107506116 wait complete. 2023-10-18T08:58:26.221 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T08:58:26.222 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T08:58:26.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.463+0000 7f1921ba9640 1 -- 172.21.15.40:0/3181052169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19080058d0 msgr2=0x7f1908005d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:26.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.463+0000 7f1921ba9640 1 --2- 172.21.15.40:0/3181052169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19080058d0 0x7f1908005d50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f190c001eb0 tx=0x7f1908006900 comp rx=0 tx=0).stop 2023-10-18T08:58:26.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.463+0000 7f191bfff640 1 -- 172.21.15.40:0/3181052169 reap_dead start 2023-10-18T08:58:26.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.463+0000 7f1921ba9640 1 -- 172.21.15.40:0/3181052169 shutdown_connections 2023-10-18T08:58:26.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.463+0000 7f1921ba9640 1 -- 172.21.15.40:0/3181052169 >> 172.21.15.40:0/3181052169 conn(0x7f191c0aec00 msgr2=0x7f191c0aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:26.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.463+0000 7f1921ba9640 1 -- 172.21.15.40:0/3181052169 shutdown_connections 2023-10-18T08:58:26.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.463+0000 7f1921ba9640 1 -- 172.21.15.40:0/3181052169 wait complete. 2023-10-18T08:58:26.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.465+0000 7f1921ba9640 1 Processor -- start 2023-10-18T08:58:26.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.465+0000 7f1921ba9640 1 -- start start 2023-10-18T08:58:26.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.466+0000 7f1921ba9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19080058d0 0x7f191c0c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:26.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.466+0000 7f1921ba9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f191c070bd0 0x7f191c071050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:26.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.466+0000 7f1921ba9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f191c153520 0x7f191c1539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:26.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.466+0000 7f1921ba9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c14daa0 con 0x7f191c070bd0 2023-10-18T08:58:26.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.466+0000 7f1921ba9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c071590 con 0x7f19080058d0 2023-10-18T08:58:26.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.466+0000 7f1921ba9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191c154290 con 0x7f191c153520 2023-10-18T08:58:26.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.466+0000 7f191affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f191c070bd0 0x7f191c071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:26.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.466+0000 7f191affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f191c070bd0 0x7f191c071050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38336/0 (socket says 172.21.15.40:38336) 2023-10-18T08:58:26.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.466+0000 7f191affd640 1 -- 172.21.15.40:0/2263518644 learned_addr learned my addr 172.21.15.40:0/2263518644 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:26.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.466+0000 7f191b7fe640 1 --2- 172.21.15.40:0/2263518644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19080058d0 0x7f191c0c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:26.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.466+0000 7f191bfff640 1 --2- 172.21.15.40:0/2263518644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f191c153520 0x7f191c1539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:26.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.466+0000 7f191affd640 1 -- 172.21.15.40:0/2263518644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f191c153520 msgr2=0x7f191c1539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:26.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.466+0000 7f191affd640 1 --2- 172.21.15.40:0/2263518644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f191c153520 0x7f191c1539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:26.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.466+0000 7f191affd640 1 -- 172.21.15.40:0/2263518644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19080058d0 msgr2=0x7f191c0c32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:26.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.466+0000 7f191affd640 1 --2- 172.21.15.40:0/2263518644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19080058d0 0x7f191c0c32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:26.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.466+0000 7f191affd640 1 -- 172.21.15.40:0/2263518644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c0aa9c0 con 0x7f191c070bd0 2023-10-18T08:58:26.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.467+0000 7f191bfff640 1 --2- 172.21.15.40:0/2263518644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f191c153520 0x7f191c1539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:26.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.467+0000 7f191affd640 1 --2- 172.21.15.40:0/2263518644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f191c070bd0 0x7f191c071050 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f190c00c350 tx=0x7f190c02dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:26.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.467+0000 7f1913fff640 1 -- 172.21.15.40:0/2263518644 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c03c040 con 0x7f191c070bd0 2023-10-18T08:58:26.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.467+0000 7f1913fff640 1 -- 172.21.15.40:0/2263518644 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f190c0037f0 con 0x7f191c070bd0 2023-10-18T08:58:26.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.467+0000 7f1921ba9640 1 -- 172.21.15.40:0/2263518644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f191c0aaca0 con 0x7f191c070bd0 2023-10-18T08:58:26.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.467+0000 7f1913fff640 1 -- 172.21.15.40:0/2263518644 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c003bc0 con 0x7f191c070bd0 2023-10-18T08:58:26.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.467+0000 7f1921ba9640 1 -- 172.21.15.40:0/2263518644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191c0ab260 con 0x7f191c070bd0 2023-10-18T08:58:26.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.468+0000 7f1921ba9640 1 -- 172.21.15.40:0/2263518644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191c066fd0 con 0x7f191c070bd0 2023-10-18T08:58:26.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.470+0000 7f1913fff640 1 -- 172.21.15.40:0/2263518644 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f190c032070 con 0x7f191c070bd0 2023-10-18T08:58:26.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.470+0000 7f1913fff640 1 --2- 172.21.15.40:0/2263518644 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18fc06ae40 0x7f18fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:26.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.471+0000 7f1913fff640 1 -- 172.21.15.40:0/2263518644 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f190c0aab30 con 0x7f191c070bd0 2023-10-18T08:58:26.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.471+0000 7f191b7fe640 1 --2- 172.21.15.40:0/2263518644 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18fc06ae40 0x7f18fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:26.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.472+0000 7f191b7fe640 1 --2- 172.21.15.40:0/2263518644 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18fc06ae40 0x7f18fc06d300 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f191c0adb60 tx=0x7f1904006ae0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:26.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.475+0000 7f1913fff640 1 -- 172.21.15.40:0/2263518644 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f190c075fc0 con 0x7f191c070bd0 2023-10-18T08:58:26.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:26.672+0000 7f1921ba9640 1 -- 172.21.15.40:0/2263518644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f191c0cca30 con 0x7f191c070bd0 2023-10-18T08:58:27.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.263+0000 7f1913fff640 1 -- 172.21.15.40:0/2263518644 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v64) v1 ==== 138+0+0 (secure 0 0 0) 0x7f190c0795e0 con 0x7f191c070bd0 2023-10-18T08:58:27.267 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-10-18T08:58:27.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.269+0000 7f1921ba9640 1 -- 172.21.15.40:0/2263518644 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18fc06ae40 msgr2=0x7f18fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:27.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.269+0000 7f1921ba9640 1 --2- 172.21.15.40:0/2263518644 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18fc06ae40 0x7f18fc06d300 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f191c0adb60 tx=0x7f1904006ae0 comp rx=0 tx=0).stop 2023-10-18T08:58:27.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.269+0000 7f1921ba9640 1 -- 172.21.15.40:0/2263518644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f191c070bd0 msgr2=0x7f191c071050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:27.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.269+0000 7f1921ba9640 1 --2- 172.21.15.40:0/2263518644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f191c070bd0 0x7f191c071050 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f190c00c350 tx=0x7f190c02dfa0 comp rx=0 tx=0).stop 2023-10-18T08:58:27.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.269+0000 7f1921ba9640 1 -- 172.21.15.40:0/2263518644 shutdown_connections 2023-10-18T08:58:27.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.269+0000 7f1921ba9640 1 --2- 172.21.15.40:0/2263518644 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18fc06ae40 0x7f18fc06d300 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:27.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.269+0000 7f1921ba9640 1 --2- 172.21.15.40:0/2263518644 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f191c153520 0x7f191c1539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:27.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.269+0000 7f1921ba9640 1 --2- 172.21.15.40:0/2263518644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f191c070bd0 0x7f191c071050 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:27.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.269+0000 7f1921ba9640 1 --2- 172.21.15.40:0/2263518644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19080058d0 0x7f191c0c32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:27.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.269+0000 7f1921ba9640 1 -- 172.21.15.40:0/2263518644 >> 172.21.15.40:0/2263518644 conn(0x7f191c0aec00 msgr2=0x7f191c0cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:27.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.271+0000 7f1921ba9640 1 -- 172.21.15.40:0/2263518644 shutdown_connections 2023-10-18T08:58:27.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.271+0000 7f1921ba9640 1 -- 172.21.15.40:0/2263518644 wait complete. 2023-10-18T08:58:27.296 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T08:58:27.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.531+0000 7f82dc982640 1 -- 172.21.15.40:0/3279306999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d81594b0 msgr2=0x7f82d80cd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:27.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.531+0000 7f82dc982640 1 --2- 172.21.15.40:0/3279306999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d81594b0 0x7f82d80cd4c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f82c8005fd0 tx=0x7f82c802c060 comp rx=0 tx=0).stop 2023-10-18T08:58:27.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.532+0000 7f82dc982640 1 -- 172.21.15.40:0/3279306999 shutdown_connections 2023-10-18T08:58:27.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.532+0000 7f82dc982640 1 --2- 172.21.15.40:0/3279306999 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d81594b0 0x7f82d80cd4c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:27.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.532+0000 7f82dc982640 1 -- 172.21.15.40:0/3279306999 >> 172.21.15.40:0/3279306999 conn(0x7f82d80b0ad0 msgr2=0x7f82d80b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:27.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.532+0000 7f82dc982640 1 -- 172.21.15.40:0/3279306999 shutdown_connections 2023-10-18T08:58:27.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.532+0000 7f82dc982640 1 -- 172.21.15.40:0/3279306999 wait complete. 2023-10-18T08:58:27.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.534+0000 7f82dc982640 1 Processor -- start 2023-10-18T08:58:27.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.534+0000 7f82dc982640 1 -- start start 2023-10-18T08:58:27.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.535+0000 7f82dc982640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d806eef0 0x7f82d81f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:27.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.535+0000 7f82dc982640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82d80ae5b0 0x7f82d8151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:27.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.535+0000 7f82dc982640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82d81594b0 0x7f82d8151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:27.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.535+0000 7f82dc982640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80c6880 con 0x7f82d806eef0 2023-10-18T08:58:27.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.535+0000 7f82dc982640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d8155a50 con 0x7f82d81594b0 2023-10-18T08:58:27.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.535+0000 7f82dc982640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d8155d30 con 0x7f82d80ae5b0 2023-10-18T08:58:27.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.535+0000 7f82d6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d806eef0 0x7f82d81f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:27.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.535+0000 7f82d6d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d806eef0 0x7f82d81f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38354/0 (socket says 172.21.15.40:38354) 2023-10-18T08:58:27.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.535+0000 7f82d6d9d640 1 -- 172.21.15.40:0/3196655338 learned_addr learned my addr 172.21.15.40:0/3196655338 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:27.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.535+0000 7f82d6d9d640 1 -- 172.21.15.40:0/3196655338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82d80ae5b0 msgr2=0x7f82d8151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:27.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.535+0000 7f82d6d9d640 1 --2- 172.21.15.40:0/3196655338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82d80ae5b0 0x7f82d8151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:27.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.535+0000 7f82d6d9d640 1 -- 172.21.15.40:0/3196655338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82d81594b0 msgr2=0x7f82d8151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:27.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.535+0000 7f82d6d9d640 1 --2- 172.21.15.40:0/3196655338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82d81594b0 0x7f82d8151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:27.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.535+0000 7f82d6d9d640 1 -- 172.21.15.40:0/3196655338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c8009840 con 0x7f82d806eef0 2023-10-18T08:58:27.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.536+0000 7f82d6d9d640 1 --2- 172.21.15.40:0/3196655338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d806eef0 0x7f82d81f4ba0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009b30 tx=0x7f82c8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:27.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.536+0000 7f82bf7fe640 1 -- 172.21.15.40:0/3196655338 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c803d630 con 0x7f82d806eef0 2023-10-18T08:58:27.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.536+0000 7f82bf7fe640 1 -- 172.21.15.40:0/3196655338 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f82c803d880 con 0x7f82d806eef0 2023-10-18T08:58:27.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.536+0000 7f82dc982640 1 -- 172.21.15.40:0/3196655338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d8156010 con 0x7f82d806eef0 2023-10-18T08:58:27.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.536+0000 7f82bf7fe640 1 -- 172.21.15.40:0/3196655338 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c803dc90 con 0x7f82d806eef0 2023-10-18T08:58:27.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.536+0000 7f82dc982640 1 -- 172.21.15.40:0/3196655338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d814db30 con 0x7f82d806eef0 2023-10-18T08:58:27.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.537+0000 7f82dc982640 1 -- 172.21.15.40:0/3196655338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d814dcc0 con 0x7f82d806eef0 2023-10-18T08:58:27.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.539+0000 7f82bf7fe640 1 -- 172.21.15.40:0/3196655338 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f82c8034430 con 0x7f82d806eef0 2023-10-18T08:58:27.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.540+0000 7f82bf7fe640 1 --2- 172.21.15.40:0/3196655338 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f82c406ae40 0x7f82c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:27.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.540+0000 7f82bf7fe640 1 -- 172.21.15.40:0/3196655338 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f82c80ac530 con 0x7f82d806eef0 2023-10-18T08:58:27.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.541+0000 7f82d659c640 1 --2- 172.21.15.40:0/3196655338 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f82c406ae40 0x7f82c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:27.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.542+0000 7f82d659c640 1 --2- 172.21.15.40:0/3196655338 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f82c406ae40 0x7f82c406d300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f82d80c90b0 tx=0x7f82cc0079b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:27.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.545+0000 7f82bf7fe640 1 -- 172.21.15.40:0/3196655338 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82d814dcc0 con 0x7f82d806eef0 2023-10-18T08:58:27.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:27.740+0000 7f82dc982640 1 -- 172.21.15.40:0/3196655338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f82d814dcc0 con 0x7f82d806eef0 2023-10-18T08:58:28.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.270+0000 7f82bf7fe640 1 -- 172.21.15.40:0/3196655338 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v65) v1 ==== 170+0+0 (secure 0 0 0) 0x7f82d814dcc0 con 0x7f82d806eef0 2023-10-18T08:58:28.274 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-10-18T08:58:28.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.275+0000 7f82dc982640 1 -- 172.21.15.40:0/3196655338 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f82c406ae40 msgr2=0x7f82c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:28.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.275+0000 7f82dc982640 1 --2- 172.21.15.40:0/3196655338 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f82c406ae40 0x7f82c406d300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f82d80c90b0 tx=0x7f82cc0079b0 comp rx=0 tx=0).stop 2023-10-18T08:58:28.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.275+0000 7f82dc982640 1 -- 172.21.15.40:0/3196655338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d806eef0 msgr2=0x7f82d81f4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:28.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.275+0000 7f82dc982640 1 --2- 172.21.15.40:0/3196655338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d806eef0 0x7f82d81f4ba0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009b30 tx=0x7f82c8004360 comp rx=0 tx=0).stop 2023-10-18T08:58:28.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.276+0000 7f82dc982640 1 -- 172.21.15.40:0/3196655338 shutdown_connections 2023-10-18T08:58:28.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.276+0000 7f82dc982640 1 --2- 172.21.15.40:0/3196655338 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f82c406ae40 0x7f82c406d300 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f82d80c90b0 tx=0x7f82cc0079b0 comp rx=0 tx=0).stop 2023-10-18T08:58:28.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.276+0000 7f82dc982640 1 --2- 172.21.15.40:0/3196655338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82d81594b0 0x7f82d8151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:28.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.276+0000 7f82dc982640 1 --2- 172.21.15.40:0/3196655338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82d80ae5b0 0x7f82d8151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:28.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.276+0000 7f82dc982640 1 --2- 172.21.15.40:0/3196655338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82d806eef0 0x7f82d81f4ba0 secure :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009b30 tx=0x7f82c8004360 comp rx=0 tx=0).stop 2023-10-18T08:58:28.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.276+0000 7f82dc982640 1 -- 172.21.15.40:0/3196655338 >> 172.21.15.40:0/3196655338 conn(0x7f82d80b0ad0 msgr2=0x7f82d81585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:28.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.277+0000 7f82dc982640 1 -- 172.21.15.40:0/3196655338 shutdown_connections 2023-10-18T08:58:28.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.277+0000 7f82dc982640 1 -- 172.21.15.40:0/3196655338 wait complete. 2023-10-18T08:58:28.309 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T08:58:28.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.508+0000 7f645641a640 1 -- 172.21.15.40:0/501575040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64500cab70 msgr2=0x7f645015d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:28.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.508+0000 7f645641a640 1 --2- 172.21.15.40:0/501575040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64500cab70 0x7f645015d390 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6448005fd0 tx=0x7f644802be20 comp rx=0 tx=0).stop 2023-10-18T08:58:28.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.508+0000 7f645641a640 1 -- 172.21.15.40:0/501575040 shutdown_connections 2023-10-18T08:58:28.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.508+0000 7f645641a640 1 --2- 172.21.15.40:0/501575040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64500cab70 0x7f645015d390 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:28.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.508+0000 7f645641a640 1 --2- 172.21.15.40:0/501575040 >> v2:172.21.15.144:3300/0 conn(0x7f64500ca6c0 0x7f645015ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:28.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.508+0000 7f645641a640 1 -- 172.21.15.40:0/501575040 >> 172.21.15.40:0/501575040 conn(0x7f64500b0ad0 msgr2=0x7f64500b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:28.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.508+0000 7f645641a640 1 -- 172.21.15.40:0/501575040 shutdown_connections 2023-10-18T08:58:28.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.508+0000 7f645641a640 1 -- 172.21.15.40:0/501575040 wait complete. 2023-10-18T08:58:28.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.510+0000 7f645641a640 1 Processor -- start 2023-10-18T08:58:28.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.510+0000 7f645641a640 1 -- start start 2023-10-18T08:58:28.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.511+0000 7f645641a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64500ca6c0 0x7f6450144280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:28.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.511+0000 7f645641a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64500cab70 0x7f6450159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:28.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.511+0000 7f645641a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6450159360 0x7f64500b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:28.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.511+0000 7f645641a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64500c3320 con 0x7f6450159360 2023-10-18T08:58:28.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.511+0000 7f645641a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645015cab0 con 0x7f64500ca6c0 2023-10-18T08:58:28.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.511+0000 7f645641a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64501f1de0 con 0x7f64500cab70 2023-10-18T08:58:28.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.512+0000 7f6455c19640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6450159360 0x7f64500b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:28.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.512+0000 7f6455c19640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6450159360 0x7f64500b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38378/0 (socket says 172.21.15.40:38378) 2023-10-18T08:58:28.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.512+0000 7f6455c19640 1 -- 172.21.15.40:0/1988913606 learned_addr learned my addr 172.21.15.40:0/1988913606 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:28.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.512+0000 7f6455418640 1 --2- 172.21.15.40:0/1988913606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64500ca6c0 0x7f6450144280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:28.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.512+0000 7f6454c17640 1 --2- 172.21.15.40:0/1988913606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64500cab70 0x7f6450159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:28.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.512+0000 7f6455c19640 1 -- 172.21.15.40:0/1988913606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64500cab70 msgr2=0x7f6450159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:28.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.512+0000 7f6455c19640 1 --2- 172.21.15.40:0/1988913606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64500cab70 0x7f6450159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:28.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.512+0000 7f6455c19640 1 -- 172.21.15.40:0/1988913606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64500ca6c0 msgr2=0x7f6450144280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:28.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.512+0000 7f6455c19640 1 --2- 172.21.15.40:0/1988913606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64500ca6c0 0x7f6450144280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:28.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.512+0000 7f6455c19640 1 -- 172.21.15.40:0/1988913606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6448009560 con 0x7f6450159360 2023-10-18T08:58:28.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.513+0000 7f6455c19640 1 --2- 172.21.15.40:0/1988913606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6450159360 0x7f64500b3900 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6448036040 tx=0x7f644802dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:28.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.513+0000 7f6445ffb640 1 -- 172.21.15.40:0/1988913606 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64480035f0 con 0x7f6450159360 2023-10-18T08:58:28.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.513+0000 7f6445ffb640 1 -- 172.21.15.40:0/1988913606 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6448003860 con 0x7f6450159360 2023-10-18T08:58:28.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.513+0000 7f645641a640 1 -- 172.21.15.40:0/1988913606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6450159e90 con 0x7f6450159360 2023-10-18T08:58:28.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.513+0000 7f6445ffb640 1 -- 172.21.15.40:0/1988913606 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6448003c70 con 0x7f6450159360 2023-10-18T08:58:28.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.513+0000 7f645641a640 1 -- 172.21.15.40:0/1988913606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64500c2c60 con 0x7f6450159360 2023-10-18T08:58:28.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.515+0000 7f645641a640 1 -- 172.21.15.40:0/1988913606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6450073570 con 0x7f6450159360 2023-10-18T08:58:28.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.516+0000 7f6445ffb640 1 -- 172.21.15.40:0/1988913606 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f644803e450 con 0x7f6450159360 2023-10-18T08:58:28.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.518+0000 7f6445ffb640 1 --2- 172.21.15.40:0/1988913606 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f643406ae40 0x7f643406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:28.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.518+0000 7f6445ffb640 1 -- 172.21.15.40:0/1988913606 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f64480ac580 con 0x7f6450159360 2023-10-18T08:58:28.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.518+0000 7f6455418640 1 --2- 172.21.15.40:0/1988913606 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f643406ae40 0x7f643406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:28.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.520+0000 7f6455418640 1 --2- 172.21.15.40:0/1988913606 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f643406ae40 0x7f643406d300 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f64500cef90 tx=0x7f644c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:28.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.522+0000 7f6445ffb640 1 -- 172.21.15.40:0/1988913606 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6448077820 con 0x7f6450159360 2023-10-18T08:58:28.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:28.757+0000 7f645641a640 1 -- 172.21.15.40:0/1988913606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6450072db0 con 0x7f6450159360 2023-10-18T08:58:29.325 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 8 and data pool 9 2023-10-18T08:58:29.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.316+0000 7f6445ffb640 1 -- 172.21.15.40:0/1988913606 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v29) v1 ==== 172+0+0 (secure 0 0 0) 0x7f64480430c0 con 0x7f6450159360 2023-10-18T08:58:29.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.326+0000 7f645641a640 1 -- 172.21.15.40:0/1988913606 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f643406ae40 msgr2=0x7f643406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:29.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.326+0000 7f645641a640 1 --2- 172.21.15.40:0/1988913606 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f643406ae40 0x7f643406d300 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f64500cef90 tx=0x7f644c009040 comp rx=0 tx=0).stop 2023-10-18T08:58:29.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.326+0000 7f645641a640 1 -- 172.21.15.40:0/1988913606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6450159360 msgr2=0x7f64500b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:29.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.326+0000 7f645641a640 1 --2- 172.21.15.40:0/1988913606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6450159360 0x7f64500b3900 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6448036040 tx=0x7f644802dad0 comp rx=0 tx=0).stop 2023-10-18T08:58:29.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.327+0000 7f645641a640 1 -- 172.21.15.40:0/1988913606 shutdown_connections 2023-10-18T08:58:29.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.327+0000 7f645641a640 1 --2- 172.21.15.40:0/1988913606 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f643406ae40 0x7f643406d300 secure :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f64500cef90 tx=0x7f644c009040 comp rx=0 tx=0).stop 2023-10-18T08:58:29.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.327+0000 7f645641a640 1 --2- 172.21.15.40:0/1988913606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6450159360 0x7f64500b3900 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:29.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.327+0000 7f645641a640 1 --2- 172.21.15.40:0/1988913606 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64500cab70 0x7f6450159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:29.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.327+0000 7f645641a640 1 --2- 172.21.15.40:0/1988913606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64500ca6c0 0x7f6450144280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:29.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.327+0000 7f645641a640 1 -- 172.21.15.40:0/1988913606 >> 172.21.15.40:0/1988913606 conn(0x7f64500b0ad0 msgr2=0x7f645015cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:29.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.328+0000 7f645641a640 1 -- 172.21.15.40:0/1988913606 shutdown_connections 2023-10-18T08:58:29.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.328+0000 7f645641a640 1 -- 172.21.15.40:0/1988913606 wait complete. 2023-10-18T08:58:29.355 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:58:29.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.547+0000 7fafa759e640 1 -- 172.21.15.40:0/3504662010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf94003f20 msgr2=0x7faf940047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:29.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.547+0000 7fafa759e640 1 --2- 172.21.15.40:0/3504662010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf94003f20 0x7faf940047c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7faf90001680 tx=0x7faf940057f0 comp rx=0 tx=0).stop 2023-10-18T08:58:29.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.547+0000 7fafa6d9d640 1 -- 172.21.15.40:0/3504662010 reap_dead start 2023-10-18T08:58:29.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.547+0000 7fafa759e640 1 -- 172.21.15.40:0/3504662010 shutdown_connections 2023-10-18T08:58:29.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.547+0000 7fafa759e640 1 -- 172.21.15.40:0/3504662010 >> 172.21.15.40:0/3504662010 conn(0x7fafa8069720 msgr2=0x7fafa8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:29.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.547+0000 7fafa759e640 1 -- 172.21.15.40:0/3504662010 shutdown_connections 2023-10-18T08:58:29.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.547+0000 7fafa759e640 1 -- 172.21.15.40:0/3504662010 wait complete. 2023-10-18T08:58:29.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.549+0000 7fafa759e640 1 Processor -- start 2023-10-18T08:58:29.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.549+0000 7fafa759e640 1 -- start start 2023-10-18T08:58:29.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.550+0000 7fafa759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf94003f20 0x7fafa81f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:29.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.550+0000 7fafa759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafa8150920 0x7fafa80af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:29.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.550+0000 7fafa759e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafa813c3a0 0x7fafa813c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:29.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.550+0000 7fafa759e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa8150500 con 0x7fafa8150920 2023-10-18T08:58:29.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.550+0000 7fafa759e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa81f1c40 con 0x7faf94003f20 2023-10-18T08:58:29.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.550+0000 7fafa759e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa813cd60 con 0x7fafa813c3a0 2023-10-18T08:58:29.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.550+0000 7faf9ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafa8150920 0x7fafa80af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:29.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.550+0000 7faf9ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafa8150920 0x7fafa80af3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38388/0 (socket says 172.21.15.40:38388) 2023-10-18T08:58:29.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.550+0000 7faf9ffff640 1 -- 172.21.15.40:0/394017938 learned_addr learned my addr 172.21.15.40:0/394017938 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:29.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.550+0000 7fafa659c640 1 --2- 172.21.15.40:0/394017938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf94003f20 0x7fafa81f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:29.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.551+0000 7fafa6d9d640 1 --2- 172.21.15.40:0/394017938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafa813c3a0 0x7fafa813c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:29.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.551+0000 7faf9ffff640 1 -- 172.21.15.40:0/394017938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafa813c3a0 msgr2=0x7fafa813c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:29.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.551+0000 7faf9ffff640 1 --2- 172.21.15.40:0/394017938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafa813c3a0 0x7fafa813c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:29.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.551+0000 7faf9ffff640 1 -- 172.21.15.40:0/394017938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf94003f20 msgr2=0x7fafa81f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:29.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.551+0000 7faf9ffff640 1 --2- 172.21.15.40:0/394017938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf94003f20 0x7fafa81f1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:29.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.551+0000 7faf9ffff640 1 -- 172.21.15.40:0/394017938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa814f880 con 0x7fafa8150920 2023-10-18T08:58:29.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.551+0000 7faf9ffff640 1 --2- 172.21.15.40:0/394017938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafa8150920 0x7fafa80af3a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7faf900017a0 tx=0x7faf90006360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:29.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.551+0000 7faf9effd640 1 -- 172.21.15.40:0/394017938 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf90003b80 con 0x7fafa8150920 2023-10-18T08:58:29.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.551+0000 7faf9effd640 1 -- 172.21.15.40:0/394017938 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faf90003dd0 con 0x7fafa8150920 2023-10-18T08:58:29.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.551+0000 7faf9effd640 1 -- 172.21.15.40:0/394017938 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9002c7b0 con 0x7fafa8150920 2023-10-18T08:58:29.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.551+0000 7fafa759e640 1 -- 172.21.15.40:0/394017938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa814fb60 con 0x7fafa8150920 2023-10-18T08:58:29.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.551+0000 7fafa759e640 1 -- 172.21.15.40:0/394017938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa8150120 con 0x7fafa8150920 2023-10-18T08:58:29.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.552+0000 7fafa759e640 1 -- 172.21.15.40:0/394017938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa8066e00 con 0x7fafa8150920 2023-10-18T08:58:29.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.553+0000 7faf9effd640 1 -- 172.21.15.40:0/394017938 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faf9002c910 con 0x7fafa8150920 2023-10-18T08:58:29.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.554+0000 7faf9effd640 1 --2- 172.21.15.40:0/394017938 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faf8806ae40 0x7faf8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:29.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.554+0000 7faf9effd640 1 -- 172.21.15.40:0/394017938 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf900aa910 con 0x7fafa8150920 2023-10-18T08:58:29.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.555+0000 7fafa659c640 1 --2- 172.21.15.40:0/394017938 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faf8806ae40 0x7faf8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:29.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.556+0000 7fafa659c640 1 --2- 172.21.15.40:0/394017938 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faf8806ae40 0x7faf8806d300 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7faf9800a2c0 tx=0x7faf9800a250 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:29.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.559+0000 7faf9effd640 1 -- 172.21.15.40:0/394017938 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faf900759b0 con 0x7fafa8150920 2023-10-18T08:58:29.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.750+0000 7fafa759e640 1 -- 172.21.15.40:0/394017938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafa80b24a0 con 0x7fafa8150920 2023-10-18T08:58:29.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.751+0000 7faf9effd640 1 -- 172.21.15.40:0/394017938 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16590 (secure 0 0 0) 0x7faf90078fd0 con 0x7fafa8150920 2023-10-18T08:58:29.755 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:29.755 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:58:28.759806+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:58:26.727620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-18T08:58:27.795303+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:58:29.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.754+0000 7fafa759e640 1 -- 172.21.15.40:0/394017938 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faf8806ae40 msgr2=0x7faf8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:29.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.754+0000 7fafa759e640 1 --2- 172.21.15.40:0/394017938 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faf8806ae40 0x7faf8806d300 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7faf9800a2c0 tx=0x7faf9800a250 comp rx=0 tx=0).stop 2023-10-18T08:58:29.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.754+0000 7fafa759e640 1 -- 172.21.15.40:0/394017938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafa8150920 msgr2=0x7fafa80af3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:29.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.754+0000 7fafa759e640 1 --2- 172.21.15.40:0/394017938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafa8150920 0x7fafa80af3a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7faf900017a0 tx=0x7faf90006360 comp rx=0 tx=0).stop 2023-10-18T08:58:29.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.754+0000 7fafa759e640 1 -- 172.21.15.40:0/394017938 shutdown_connections 2023-10-18T08:58:29.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.754+0000 7fafa759e640 1 --2- 172.21.15.40:0/394017938 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faf8806ae40 0x7faf8806d300 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:29.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.754+0000 7fafa759e640 1 --2- 172.21.15.40:0/394017938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafa813c3a0 0x7fafa813c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:29.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.754+0000 7fafa759e640 1 --2- 172.21.15.40:0/394017938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafa8150920 0x7fafa80af3a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:29.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.754+0000 7fafa759e640 1 --2- 172.21.15.40:0/394017938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf94003f20 0x7fafa81f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:29.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.754+0000 7fafa759e640 1 -- 172.21.15.40:0/394017938 >> 172.21.15.40:0/394017938 conn(0x7fafa8069720 msgr2=0x7fafa80ab9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:29.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.754+0000 7fafa759e640 1 -- 172.21.15.40:0/394017938 shutdown_connections 2023-10-18T08:58:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.754+0000 7fafa759e640 1 -- 172.21.15.40:0/394017938 wait complete. 2023-10-18T08:58:29.775 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:58:29.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.995+0000 7f154a002640 1 -- 172.21.15.40:0/2489504586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f154406b6f0 msgr2=0x7f1544097a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:29.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.995+0000 7f154a002640 1 --2- 172.21.15.40:0/2489504586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f154406b6f0 0x7f1544097a10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1544071350 tx=0x7f153c02c2e0 comp rx=0 tx=0).stop 2023-10-18T08:58:29.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.995+0000 7f154a002640 1 -- 172.21.15.40:0/2489504586 shutdown_connections 2023-10-18T08:58:29.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.995+0000 7f154a002640 1 --2- 172.21.15.40:0/2489504586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f154406b6f0 0x7f1544097a10 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:29.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.995+0000 7f154a002640 1 --2- 172.21.15.40:0/2489504586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f154406ac60 0x7f154406b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:29.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.995+0000 7f154a002640 1 --2- 172.21.15.40:0/2489504586 >> v2:172.21.15.144:3300/0 conn(0x7f15440b4a00 0x7f15440cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:30.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.995+0000 7f154a002640 1 -- 172.21.15.40:0/2489504586 >> 172.21.15.40:0/2489504586 conn(0x7f15440b0af0 msgr2=0x7f15440b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:30.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.995+0000 7f154a002640 1 -- 172.21.15.40:0/2489504586 shutdown_connections 2023-10-18T08:58:30.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.995+0000 7f154a002640 1 -- 172.21.15.40:0/2489504586 wait complete. 2023-10-18T08:58:30.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.997+0000 7f154a002640 1 Processor -- start 2023-10-18T08:58:30.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.997+0000 7f154a002640 1 -- start start 2023-10-18T08:58:30.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.998+0000 7f154a002640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f154406ac60 0x7f154415bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:30.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.998+0000 7f154a002640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f154406b6f0 0x7f154415a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:30.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.998+0000 7f154a002640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15440b4a00 0x7f154414d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:30.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.998+0000 7f154a002640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15440c89e0 con 0x7f15440b4a00 2023-10-18T08:58:30.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.998+0000 7f154a002640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154414da30 con 0x7f154406ac60 2023-10-18T08:58:30.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.998+0000 7f154a002640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154415ace0 con 0x7f154406b6f0 2023-10-18T08:58:30.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.998+0000 7f1549801640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15440b4a00 0x7f154414d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:30.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.998+0000 7f1549801640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15440b4a00 0x7f154414d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38412/0 (socket says 172.21.15.40:38412) 2023-10-18T08:58:30.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.998+0000 7f1549801640 1 -- 172.21.15.40:0/2693995869 learned_addr learned my addr 172.21.15.40:0/2693995869 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:30.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.999+0000 7f1549000640 1 --2- 172.21.15.40:0/2693995869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f154406ac60 0x7f154415bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:30.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.999+0000 7f1543fff640 1 --2- 172.21.15.40:0/2693995869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f154406b6f0 0x7f154415a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:30.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.999+0000 7f1549801640 1 -- 172.21.15.40:0/2693995869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f154406b6f0 msgr2=0x7f154415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:30.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.999+0000 7f1549801640 1 --2- 172.21.15.40:0/2693995869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f154406b6f0 0x7f154415a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:30.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.999+0000 7f1549801640 1 -- 172.21.15.40:0/2693995869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f154406ac60 msgr2=0x7f154415bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:30.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.999+0000 7f1549801640 1 --2- 172.21.15.40:0/2693995869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f154406ac60 0x7f154415bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:30.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.999+0000 7f1549801640 1 -- 172.21.15.40:0/2693995869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c009900 con 0x7f15440b4a00 2023-10-18T08:58:30.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.999+0000 7f1549000640 1 --2- 172.21.15.40:0/2693995869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f154406ac60 0x7f154415bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:30.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.999+0000 7f1549801640 1 --2- 172.21.15.40:0/2693995869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15440b4a00 0x7f154414d4f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f153c037040 tx=0x7f153c035ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:30.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.999+0000 7f15417fa640 1 -- 172.21.15.40:0/2693995869 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c004240 con 0x7f15440b4a00 2023-10-18T08:58:30.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.999+0000 7f15417fa640 1 -- 172.21.15.40:0/2693995869 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f153c004490 con 0x7f15440b4a00 2023-10-18T08:58:30.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.999+0000 7f15417fa640 1 -- 172.21.15.40:0/2693995869 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c004880 con 0x7f15440b4a00 2023-10-18T08:58:30.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:29.999+0000 7f154a002640 1 -- 172.21.15.40:0/2693995869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154415afc0 con 0x7f15440b4a00 2023-10-18T08:58:30.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.000+0000 7f154a002640 1 -- 172.21.15.40:0/2693995869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154415b580 con 0x7f15440b4a00 2023-10-18T08:58:30.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.001+0000 7f154a002640 1 -- 172.21.15.40:0/2693995869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1544066e00 con 0x7f15440b4a00 2023-10-18T08:58:30.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.002+0000 7f15417fa640 1 -- 172.21.15.40:0/2693995869 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f153c03b040 con 0x7f15440b4a00 2023-10-18T08:58:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.003+0000 7f15417fa640 1 --2- 172.21.15.40:0/2693995869 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f152806ae40 0x7f152806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.003+0000 7f15417fa640 1 -- 172.21.15.40:0/2693995869 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f153c0ac290 con 0x7f15440b4a00 2023-10-18T08:58:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.003+0000 7f1549000640 1 --2- 172.21.15.40:0/2693995869 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f152806ae40 0x7f152806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:30.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.004+0000 7f1549000640 1 --2- 172.21.15.40:0/2693995869 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f152806ae40 0x7f152806d300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1538009a20 tx=0x7f1538005d30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.008+0000 7f15417fa640 1 -- 172.21.15.40:0/2693995869 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f153c077340 con 0x7f15440b4a00 2023-10-18T08:58:30.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.204+0000 7f154a002640 1 -- 172.21.15.40:0/2693995869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1544072370 con 0x7f15440b4a00 2023-10-18T08:58:30.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.206+0000 7f15417fa640 1 -- 172.21.15.40:0/2693995869 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16590 (secure 0 0 0) 0x7f153c07a960 con 0x7f15440b4a00 2023-10-18T08:58:30.210 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:30.210 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:58:28.759806+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:58:26.727620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-18T08:58:27.795303+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:58:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.209+0000 7f154a002640 1 -- 172.21.15.40:0/2693995869 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f152806ae40 msgr2=0x7f152806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.209+0000 7f154a002640 1 --2- 172.21.15.40:0/2693995869 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f152806ae40 0x7f152806d300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1538009a20 tx=0x7f1538005d30 comp rx=0 tx=0).stop 2023-10-18T08:58:30.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.209+0000 7f154a002640 1 -- 172.21.15.40:0/2693995869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15440b4a00 msgr2=0x7f154414d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:30.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.209+0000 7f154a002640 1 --2- 172.21.15.40:0/2693995869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15440b4a00 0x7f154414d4f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f153c037040 tx=0x7f153c035ad0 comp rx=0 tx=0).stop 2023-10-18T08:58:30.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.209+0000 7f154a002640 1 -- 172.21.15.40:0/2693995869 shutdown_connections 2023-10-18T08:58:30.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.209+0000 7f154a002640 1 --2- 172.21.15.40:0/2693995869 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f152806ae40 0x7f152806d300 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:30.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.209+0000 7f154a002640 1 --2- 172.21.15.40:0/2693995869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15440b4a00 0x7f154414d4f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:30.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.209+0000 7f154a002640 1 --2- 172.21.15.40:0/2693995869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f154406b6f0 0x7f154415a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:30.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.209+0000 7f154a002640 1 --2- 172.21.15.40:0/2693995869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f154406ac60 0x7f154415bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:30.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.209+0000 7f154a002640 1 -- 172.21.15.40:0/2693995869 >> 172.21.15.40:0/2693995869 conn(0x7f15440b0af0 msgr2=0x7f154406ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:30.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.209+0000 7f154a002640 1 -- 172.21.15.40:0/2693995869 shutdown_connections 2023-10-18T08:58:30.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.209+0000 7f154a002640 1 -- 172.21.15.40:0/2693995869 wait complete. 2023-10-18T08:58:30.232 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T08:58:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.466+0000 7f6851c94640 1 -- 172.21.15.40:0/273664620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f684c0ccf10 msgr2=0x7f684c0cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:30.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.466+0000 7f6851c94640 1 --2- 172.21.15.40:0/273664620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f684c0ccf10 0x7f684c0cd3c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f684c0cdf10 tx=0x7f684402c260 comp rx=0 tx=0).stop 2023-10-18T08:58:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.466+0000 7f6851c94640 1 -- 172.21.15.40:0/273664620 shutdown_connections 2023-10-18T08:58:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.466+0000 7f6851c94640 1 --2- 172.21.15.40:0/273664620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f684c0ccf10 0x7f684c0cd3c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.466+0000 7f6851c94640 1 --2- 172.21.15.40:0/273664620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684c1586a0 0x7f684c15caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:30.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.466+0000 7f6851c94640 1 --2- 172.21.15.40:0/273664620 >> v2:172.21.15.144:3300/0 conn(0x7f684c1582c0 0x7f684c06f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:30.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.466+0000 7f6851c94640 1 -- 172.21.15.40:0/273664620 >> 172.21.15.40:0/273664620 conn(0x7f684c069720 msgr2=0x7f684c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:30.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.467+0000 7f6851c94640 1 -- 172.21.15.40:0/273664620 shutdown_connections 2023-10-18T08:58:30.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.467+0000 7f6851c94640 1 -- 172.21.15.40:0/273664620 wait complete. 2023-10-18T08:58:30.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.468+0000 7f6851c94640 1 Processor -- start 2023-10-18T08:58:30.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.469+0000 7f6851c94640 1 -- start start 2023-10-18T08:58:30.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.470+0000 7f6851c94640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c0ccf10 0x7f684c1f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:30.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.470+0000 7f6851c94640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684c1582c0 0x7f684c15a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:30.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.470+0000 7f6851c94640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f684c1586a0 0x7f684c0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:30.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.470+0000 7f6851c94640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0a75b0 con 0x7f684c1586a0 2023-10-18T08:58:30.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.470+0000 7f6851c94640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0a6930 con 0x7f684c0ccf10 2023-10-18T08:58:30.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.470+0000 7f6851c94640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0a6c10 con 0x7f684c1582c0 2023-10-18T08:58:30.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.470+0000 7f684bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f684c1586a0 0x7f684c0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:30.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.470+0000 7f684bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f684c1586a0 0x7f684c0c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38428/0 (socket says 172.21.15.40:38428) 2023-10-18T08:58:30.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.470+0000 7f684bfff640 1 -- 172.21.15.40:0/1316141216 learned_addr learned my addr 172.21.15.40:0/1316141216 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:30.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.470+0000 7f6843fff640 1 --2- 172.21.15.40:0/1316141216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684c1582c0 0x7f684c15a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:30.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.470+0000 7f684b7fe640 1 --2- 172.21.15.40:0/1316141216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c0ccf10 0x7f684c1f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:30.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.471+0000 7f684bfff640 1 -- 172.21.15.40:0/1316141216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684c1582c0 msgr2=0x7f684c15a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.471+0000 7f684bfff640 1 --2- 172.21.15.40:0/1316141216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684c1582c0 0x7f684c15a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.471+0000 7f684bfff640 1 -- 172.21.15.40:0/1316141216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c0ccf10 msgr2=0x7f684c1f13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.471+0000 7f684bfff640 1 --2- 172.21.15.40:0/1316141216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c0ccf10 0x7f684c1f13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.471+0000 7f684bfff640 1 -- 172.21.15.40:0/1316141216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6844009900 con 0x7f684c1586a0 2023-10-18T08:58:30.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.471+0000 7f684bfff640 1 --2- 172.21.15.40:0/1316141216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f684c1586a0 0x7f684c0c9c50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6844037040 tx=0x7f6844003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:30.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.471+0000 7f6848ff9640 1 -- 172.21.15.40:0/1316141216 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844004240 con 0x7f684c1586a0 2023-10-18T08:58:30.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.471+0000 7f6848ff9640 1 -- 172.21.15.40:0/1316141216 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6844004490 con 0x7f684c1586a0 2023-10-18T08:58:30.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.471+0000 7f6848ff9640 1 -- 172.21.15.40:0/1316141216 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844004880 con 0x7f684c1586a0 2023-10-18T08:58:30.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.471+0000 7f6851c94640 1 -- 172.21.15.40:0/1316141216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c0a6ef0 con 0x7f684c1586a0 2023-10-18T08:58:30.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.471+0000 7f6851c94640 1 -- 172.21.15.40:0/1316141216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c1f18e0 con 0x7f684c1586a0 2023-10-18T08:58:30.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.473+0000 7f6851c94640 1 -- 172.21.15.40:0/1316141216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684c0a76f0 con 0x7f684c1586a0 2023-10-18T08:58:30.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.474+0000 7f6848ff9640 1 -- 172.21.15.40:0/1316141216 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f684403b040 con 0x7f684c1586a0 2023-10-18T08:58:30.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.475+0000 7f6848ff9640 1 --2- 172.21.15.40:0/1316141216 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f682c06ae70 0x7f682c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:30.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.475+0000 7f6848ff9640 1 -- 172.21.15.40:0/1316141216 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68440ac430 con 0x7f684c1586a0 2023-10-18T08:58:30.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.475+0000 7f684b7fe640 1 --2- 172.21.15.40:0/1316141216 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f682c06ae70 0x7f682c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:30.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.476+0000 7f684b7fe640 1 --2- 172.21.15.40:0/1316141216 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f682c06ae70 0x7f682c06d330 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f683c00bb20 tx=0x7f683c005e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:30.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.479+0000 7f6848ff9640 1 -- 172.21.15.40:0/1316141216 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f684c0a76f0 con 0x7f684c1586a0 2023-10-18T08:58:30.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:30.713+0000 7f6851c94640 1 -- 172.21.15.40:0/1316141216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f684c0a76f0 con 0x7f684c1586a0 2023-10-18T08:58:31.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.330+0000 7f6848ff9640 1 -- 172.21.15.40:0/1316141216 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7f684c0a76f0 con 0x7f684c1586a0 2023-10-18T08:58:31.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.333+0000 7f6851c94640 1 -- 172.21.15.40:0/1316141216 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f682c06ae70 msgr2=0x7f682c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:31.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.333+0000 7f6851c94640 1 --2- 172.21.15.40:0/1316141216 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f682c06ae70 0x7f682c06d330 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f683c00bb20 tx=0x7f683c005e50 comp rx=0 tx=0).stop 2023-10-18T08:58:31.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.333+0000 7f6851c94640 1 -- 172.21.15.40:0/1316141216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f684c1586a0 msgr2=0x7f684c0c9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:31.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.333+0000 7f6851c94640 1 --2- 172.21.15.40:0/1316141216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f684c1586a0 0x7f684c0c9c50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6844037040 tx=0x7f6844003ba0 comp rx=0 tx=0).stop 2023-10-18T08:58:31.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.333+0000 7f6851c94640 1 -- 172.21.15.40:0/1316141216 shutdown_connections 2023-10-18T08:58:31.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.333+0000 7f6851c94640 1 --2- 172.21.15.40:0/1316141216 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f682c06ae70 0x7f682c06d330 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:31.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.333+0000 7f6851c94640 1 --2- 172.21.15.40:0/1316141216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f684c1586a0 0x7f684c0c9c50 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:31.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.333+0000 7f6851c94640 1 --2- 172.21.15.40:0/1316141216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f684c1582c0 0x7f684c15a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:31.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.333+0000 7f6851c94640 1 --2- 172.21.15.40:0/1316141216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c0ccf10 0x7f684c1f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:31.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.333+0000 7f6851c94640 1 -- 172.21.15.40:0/1316141216 >> 172.21.15.40:0/1316141216 conn(0x7f684c069720 msgr2=0x7f684c0cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:31.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.334+0000 7f6851c94640 1 -- 172.21.15.40:0/1316141216 shutdown_connections 2023-10-18T08:58:31.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.334+0000 7f6851c94640 1 -- 172.21.15.40:0/1316141216 wait complete. 2023-10-18T08:58:31.356 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T08:58:31.357 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T08:58:31.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.593+0000 7f0527475640 1 -- 172.21.15.40:0/1573939421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f052006c2d0 msgr2=0x7f05200c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:31.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.593+0000 7f0527475640 1 --2- 172.21.15.40:0/1573939421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f052006c2d0 0x7f05200c9620 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0510009830 tx=0x7f051002bcd0 comp rx=0 tx=0).stop 2023-10-18T08:58:31.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.594+0000 7f0527475640 1 -- 172.21.15.40:0/1573939421 shutdown_connections 2023-10-18T08:58:31.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.594+0000 7f0527475640 1 --2- 172.21.15.40:0/1573939421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f052015af70 0x7f0520070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:31.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.594+0000 7f0527475640 1 --2- 172.21.15.40:0/1573939421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f052006c2d0 0x7f05200c9620 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:31.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.594+0000 7f0527475640 1 -- 172.21.15.40:0/1573939421 >> 172.21.15.40:0/1573939421 conn(0x7f052006ca10 msgr2=0x7f052006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:31.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.594+0000 7f0527475640 1 -- 172.21.15.40:0/1573939421 shutdown_connections 2023-10-18T08:58:31.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.594+0000 7f0527475640 1 -- 172.21.15.40:0/1573939421 wait complete. 2023-10-18T08:58:31.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.596+0000 7f0527475640 1 Processor -- start 2023-10-18T08:58:31.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.596+0000 7f0527475640 1 -- start start 2023-10-18T08:58:31.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.597+0000 7f0527475640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f052006c2d0 0x7f05200b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:31.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.597+0000 7f0527475640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f052006d5d0 0x7f05201455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:31.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.597+0000 7f0527475640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f052015af70 0x7f0520158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:31.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.597+0000 7f0527475640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052015a700 con 0x7f052015af70 2023-10-18T08:58:31.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.597+0000 7f0527475640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05201587c0 con 0x7f052006c2d0 2023-10-18T08:58:31.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.597+0000 7f0527475640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05200b1b80 con 0x7f052006d5d0 2023-10-18T08:58:31.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.597+0000 7f0526c74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f052015af70 0x7f0520158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:31.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.597+0000 7f0526c74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f052015af70 0x7f0520158280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38444/0 (socket says 172.21.15.40:38444) 2023-10-18T08:58:31.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.597+0000 7f0526c74640 1 -- 172.21.15.40:0/3884708590 learned_addr learned my addr 172.21.15.40:0/3884708590 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:31.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.597+0000 7f0526473640 1 --2- 172.21.15.40:0/3884708590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f052006c2d0 0x7f05200b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.597+0000 7f0525c72640 1 --2- 172.21.15.40:0/3884708590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f052006d5d0 0x7f05201455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.598+0000 7f0526c74640 1 -- 172.21.15.40:0/3884708590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f052006d5d0 msgr2=0x7f05201455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.598+0000 7f0526c74640 1 --2- 172.21.15.40:0/3884708590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f052006d5d0 0x7f05201455c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:31.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.598+0000 7f0526c74640 1 -- 172.21.15.40:0/3884708590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f052006c2d0 msgr2=0x7f05200b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:31.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.598+0000 7f0526c74640 1 --2- 172.21.15.40:0/3884708590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f052006c2d0 0x7f05200b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:31.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.598+0000 7f0526c74640 1 -- 172.21.15.40:0/3884708590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0510009490 con 0x7f052015af70 2023-10-18T08:58:31.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.598+0000 7f0526c74640 1 --2- 172.21.15.40:0/3884708590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f052015af70 0x7f0520158280 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f051800e960 tx=0x7f051800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:31.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.598+0000 7f0516ffd640 1 -- 172.21.15.40:0/3884708590 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518004d20 con 0x7f052015af70 2023-10-18T08:58:31.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.598+0000 7f0516ffd640 1 -- 172.21.15.40:0/3884708590 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f051800cb20 con 0x7f052015af70 2023-10-18T08:58:31.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.598+0000 7f0527475640 1 -- 172.21.15.40:0/3884708590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0520159dd0 con 0x7f052015af70 2023-10-18T08:58:31.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.598+0000 7f0516ffd640 1 -- 172.21.15.40:0/3884708590 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518005020 con 0x7f052015af70 2023-10-18T08:58:31.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.598+0000 7f0527475640 1 -- 172.21.15.40:0/3884708590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052015a390 con 0x7f052015af70 2023-10-18T08:58:31.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.599+0000 7f0527475640 1 -- 172.21.15.40:0/3884708590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052015a870 con 0x7f052015af70 2023-10-18T08:58:31.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.600+0000 7f0516ffd640 1 -- 172.21.15.40:0/3884708590 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0518005180 con 0x7f052015af70 2023-10-18T08:58:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.601+0000 7f0516ffd640 1 --2- 172.21.15.40:0/3884708590 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f050406ae40 0x7f050406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.602+0000 7f0516ffd640 1 -- 172.21.15.40:0/3884708590 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0518089f80 con 0x7f052015af70 2023-10-18T08:58:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.602+0000 7f0526473640 1 --2- 172.21.15.40:0/3884708590 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f050406ae40 0x7f050406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:31.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.603+0000 7f0526473640 1 --2- 172.21.15.40:0/3884708590 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f050406ae40 0x7f050406d300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f05200ccdc0 tx=0x7f051c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:31.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.606+0000 7f0516ffd640 1 -- 172.21.15.40:0/3884708590 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f05180bcca0 con 0x7f052015af70 2023-10-18T08:58:31.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:31.843+0000 7f0527475640 1 -- 172.21.15.40:0/3884708590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0520066e00 con 0x7f052015af70 2023-10-18T08:58:32.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.333+0000 7f0516ffd640 1 -- 172.21.15.40:0/3884708590 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7f051801f020 con 0x7f052015af70 2023-10-18T08:58:32.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.336+0000 7f0527475640 1 -- 172.21.15.40:0/3884708590 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f050406ae40 msgr2=0x7f050406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.336+0000 7f0527475640 1 --2- 172.21.15.40:0/3884708590 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f050406ae40 0x7f050406d300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f05200ccdc0 tx=0x7f051c009040 comp rx=0 tx=0).stop 2023-10-18T08:58:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.336+0000 7f0527475640 1 -- 172.21.15.40:0/3884708590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f052015af70 msgr2=0x7f0520158280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.336+0000 7f0527475640 1 --2- 172.21.15.40:0/3884708590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f052015af70 0x7f0520158280 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f051800e960 tx=0x7f051800ee20 comp rx=0 tx=0).stop 2023-10-18T08:58:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.337+0000 7f0527475640 1 -- 172.21.15.40:0/3884708590 shutdown_connections 2023-10-18T08:58:32.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.337+0000 7f0527475640 1 --2- 172.21.15.40:0/3884708590 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f050406ae40 0x7f050406d300 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.337+0000 7f0527475640 1 --2- 172.21.15.40:0/3884708590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f052015af70 0x7f0520158280 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.337+0000 7f0527475640 1 --2- 172.21.15.40:0/3884708590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f052006d5d0 0x7f05201455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.337+0000 7f0527475640 1 --2- 172.21.15.40:0/3884708590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f052006c2d0 0x7f05200b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.337+0000 7f0527475640 1 -- 172.21.15.40:0/3884708590 >> 172.21.15.40:0/3884708590 conn(0x7f052006ca10 msgr2=0x7f052015c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:32.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.337+0000 7f0527475640 1 -- 172.21.15.40:0/3884708590 shutdown_connections 2023-10-18T08:58:32.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.337+0000 7f0527475640 1 -- 172.21.15.40:0/3884708590 wait complete. 2023-10-18T08:58:32.363 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T08:58:32.363 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:58:32.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.594+0000 7fc2b13a9640 1 -- 172.21.15.40:0/1265168851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ac0cced0 msgr2=0x7fc2ac0cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:32.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.594+0000 7fc2b13a9640 1 --2- 172.21.15.40:0/1265168851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ac0cced0 0x7fc2ac0cd320 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009830 tx=0x7fc2a402bd50 comp rx=0 tx=0).stop 2023-10-18T08:58:32.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.594+0000 7fc2b13a9640 1 -- 172.21.15.40:0/1265168851 shutdown_connections 2023-10-18T08:58:32.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.594+0000 7fc2b13a9640 1 --2- 172.21.15.40:0/1265168851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ac0cced0 0x7fc2ac0cd320 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:32.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.594+0000 7fc2b13a9640 1 --2- 172.21.15.40:0/1265168851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2ac1586a0 0x7fc2ac0cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:32.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.594+0000 7fc2b13a9640 1 -- 172.21.15.40:0/1265168851 >> 172.21.15.40:0/1265168851 conn(0x7fc2ac06cee0 msgr2=0x7fc2ac069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:32.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.594+0000 7fc2b13a9640 1 -- 172.21.15.40:0/1265168851 shutdown_connections 2023-10-18T08:58:32.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.594+0000 7fc2b13a9640 1 -- 172.21.15.40:0/1265168851 wait complete. 2023-10-18T08:58:32.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.596+0000 7fc2b13a9640 1 Processor -- start 2023-10-18T08:58:32.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.596+0000 7fc2b13a9640 1 -- start start 2023-10-18T08:58:32.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.597+0000 7fc2b13a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2ac06d710 0x7fc2ac142b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:32.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.597+0000 7fc2b13a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ac0cced0 0x7fc2ac1e4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:32.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.597+0000 7fc2b13a9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2ac1586a0 0x7fc2ac143370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:32.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.597+0000 7fc2b13a9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac1e48b0 con 0x7fc2ac0cced0 2023-10-18T08:58:32.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.597+0000 7fc2b13a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac143090 con 0x7fc2ac06d710 2023-10-18T08:58:32.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.597+0000 7fc2b13a9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac0cb520 con 0x7fc2ac1586a0 2023-10-18T08:58:32.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.597+0000 7fc2aa7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ac0cced0 0x7fc2ac1e4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:32.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.598+0000 7fc2aa7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ac0cced0 0x7fc2ac1e4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38464/0 (socket says 172.21.15.40:38464) 2023-10-18T08:58:32.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.598+0000 7fc2aa7fc640 1 -- 172.21.15.40:0/4173338468 learned_addr learned my addr 172.21.15.40:0/4173338468 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:32.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.598+0000 7fc2aaffd640 1 --2- 172.21.15.40:0/4173338468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2ac06d710 0x7fc2ac142b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:32.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.598+0000 7fc2aa7fc640 1 -- 172.21.15.40:0/4173338468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2ac1586a0 msgr2=0x7fc2ac143370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:32.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.598+0000 7fc2aa7fc640 1 --2- 172.21.15.40:0/4173338468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2ac1586a0 0x7fc2ac143370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:32.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.598+0000 7fc2aa7fc640 1 -- 172.21.15.40:0/4173338468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2ac06d710 msgr2=0x7fc2ac142b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:32.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.598+0000 7fc2aa7fc640 1 --2- 172.21.15.40:0/4173338468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2ac06d710 0x7fc2ac142b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:32.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.598+0000 7fc2aa7fc640 1 -- 172.21.15.40:0/4173338468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a4009490 con 0x7fc2ac0cced0 2023-10-18T08:58:32.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.598+0000 7fc2aaffd640 1 --2- 172.21.15.40:0/4173338468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2ac06d710 0x7fc2ac142b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:32.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.598+0000 7fc2aa7fc640 1 --2- 172.21.15.40:0/4173338468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ac0cced0 0x7fc2ac1e4cd0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000db90 tx=0x7fc2a000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:32.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.599+0000 7fc2937fe640 1 -- 172.21.15.40:0/4173338468 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a000e650 con 0x7fc2ac0cced0 2023-10-18T08:58:32.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.599+0000 7fc2937fe640 1 -- 172.21.15.40:0/4173338468 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc2a000e8a0 con 0x7fc2ac0cced0 2023-10-18T08:58:32.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.599+0000 7fc2b13a9640 1 -- 172.21.15.40:0/4173338468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ac1f6050 con 0x7fc2ac0cced0 2023-10-18T08:58:32.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.599+0000 7fc2937fe640 1 -- 172.21.15.40:0/4173338468 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a000ecb0 con 0x7fc2ac0cced0 2023-10-18T08:58:32.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.599+0000 7fc2b13a9640 1 -- 172.21.15.40:0/4173338468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ac1e44d0 con 0x7fc2ac0cced0 2023-10-18T08:58:32.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.600+0000 7fc2b13a9640 1 -- 172.21.15.40:0/4173338468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ac1e4b20 con 0x7fc2ac0cced0 2023-10-18T08:58:32.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.601+0000 7fc2937fe640 1 -- 172.21.15.40:0/4173338468 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc2a0016440 con 0x7fc2ac0cced0 2023-10-18T08:58:32.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.602+0000 7fc2937fe640 1 --2- 172.21.15.40:0/4173338468 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc29806ae40 0x7fc29806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:32.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.602+0000 7fc2937fe640 1 -- 172.21.15.40:0/4173338468 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2a0089de0 con 0x7fc2ac0cced0 2023-10-18T08:58:32.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.603+0000 7fc2aaffd640 1 --2- 172.21.15.40:0/4173338468 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc29806ae40 0x7fc29806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:32.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.604+0000 7fc2aaffd640 1 --2- 172.21.15.40:0/4173338468 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc29806ae40 0x7fc29806d300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac156b80 tx=0x7fc29c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:32.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.608+0000 7fc2937fe640 1 -- 172.21.15.40:0/4173338468 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2a0054f30 con 0x7fc2ac0cced0 2023-10-18T08:58:32.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.836+0000 7fc2b13a9640 1 -- 172.21.15.40:0/4173338468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2ac066fd0 con 0x7fc2ac0cced0 2023-10-18T08:58:32.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.837+0000 7fc2937fe640 1 -- 172.21.15.40:0/4173338468 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fc2a001b020 con 0x7fc2ac0cced0 2023-10-18T08:58:32.840 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:32.841 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5075,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2165596806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2165596806},{"type":"v1","addr":"172.21.15.144:6835","nonce":2165596806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/587522883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":587522883},{"type":"v1","addr":"172.21.15.40:6833","nonce":587522883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1174134524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1174134524},{"type":"v1","addr":"172.21.15.40:6835","nonce":1174134524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:58:28.760677+0000","modified":"2023-10-18T08:58:31.845684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5389":{"gid":5389,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1311290970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1311290970},{"type":"v1","addr":"172.21.15.144:6837","nonce":1311290970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-18T08:58:32.841 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 33 2023-10-18T08:58:32.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.840+0000 7fc2b13a9640 1 -- 172.21.15.40:0/4173338468 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc29806ae40 msgr2=0x7fc29806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:32.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.840+0000 7fc2b13a9640 1 --2- 172.21.15.40:0/4173338468 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc29806ae40 0x7fc29806d300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac156b80 tx=0x7fc29c009040 comp rx=0 tx=0).stop 2023-10-18T08:58:32.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.840+0000 7fc2b13a9640 1 -- 172.21.15.40:0/4173338468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ac0cced0 msgr2=0x7fc2ac1e4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:32.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.840+0000 7fc2b13a9640 1 --2- 172.21.15.40:0/4173338468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ac0cced0 0x7fc2ac1e4cd0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000db90 tx=0x7fc2a000a620 comp rx=0 tx=0).stop 2023-10-18T08:58:32.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.840+0000 7fc2b13a9640 1 -- 172.21.15.40:0/4173338468 shutdown_connections 2023-10-18T08:58:32.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.840+0000 7fc2b13a9640 1 --2- 172.21.15.40:0/4173338468 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc29806ae40 0x7fc29806d300 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:32.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.840+0000 7fc2b13a9640 1 --2- 172.21.15.40:0/4173338468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2ac1586a0 0x7fc2ac143370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:32.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.840+0000 7fc2b13a9640 1 --2- 172.21.15.40:0/4173338468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ac0cced0 0x7fc2ac1e4cd0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:32.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.840+0000 7fc2b13a9640 1 --2- 172.21.15.40:0/4173338468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2ac06d710 0x7fc2ac142b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:32.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.840+0000 7fc2b13a9640 1 -- 172.21.15.40:0/4173338468 >> 172.21.15.40:0/4173338468 conn(0x7fc2ac06cee0 msgr2=0x7fc2ac159c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:32.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.840+0000 7fc2b13a9640 1 -- 172.21.15.40:0/4173338468 shutdown_connections 2023-10-18T08:58:32.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:32.840+0000 7fc2b13a9640 1 -- 172.21.15.40:0/4173338468 wait complete. 2023-10-18T08:58:32.860 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:58:33.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.080+0000 7f963bdc0640 1 -- 172.21.15.40:0/2852879353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96200047c0 msgr2=0x7f9620004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:33.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.080+0000 7f963bdc0640 1 --2- 172.21.15.40:0/2852879353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96200047c0 0x7f9620004c40 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9624009d00 tx=0x7f96200057f0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.080+0000 7f963a35d640 1 -- 172.21.15.40:0/2852879353 reap_dead start 2023-10-18T08:58:33.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.080+0000 7f963bdc0640 1 -- 172.21.15.40:0/2852879353 shutdown_connections 2023-10-18T08:58:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.080+0000 7f963bdc0640 1 -- 172.21.15.40:0/2852879353 >> 172.21.15.40:0/2852879353 conn(0x7f963406cee0 msgr2=0x7f9634069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.080+0000 7f963bdc0640 1 -- 172.21.15.40:0/2852879353 shutdown_connections 2023-10-18T08:58:33.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.080+0000 7f963bdc0640 1 -- 172.21.15.40:0/2852879353 wait complete. 2023-10-18T08:58:33.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.082+0000 7f963bdc0640 1 Processor -- start 2023-10-18T08:58:33.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.082+0000 7f963bdc0640 1 -- start start 2023-10-18T08:58:33.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.083+0000 7f963bdc0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96200047c0 0x7f963413e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:33.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.083+0000 7f963bdc0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96341f5080 0x7f9634140e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:33.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.083+0000 7f963bdc0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9634154280 0x7f963413d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:33.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.083+0000 7f963bdc0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96341f6050 con 0x7f96200047c0 2023-10-18T08:58:33.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.083+0000 7f963bdc0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9634141380 con 0x7f96341f5080 2023-10-18T08:58:33.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.083+0000 7f963bdc0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96340ad460 con 0x7f9634154280 2023-10-18T08:58:33.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.084+0000 7f9639b5c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96200047c0 0x7f963413e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:33.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.084+0000 7f9639b5c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96200047c0 0x7f963413e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38480/0 (socket says 172.21.15.40:38480) 2023-10-18T08:58:33.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.084+0000 7f9639b5c640 1 -- 172.21.15.40:0/234688616 learned_addr learned my addr 172.21.15.40:0/234688616 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:33.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.084+0000 7f963935b640 1 --2- 172.21.15.40:0/234688616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96341f5080 0x7f9634140e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:33.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.084+0000 7f9639b5c640 1 -- 172.21.15.40:0/234688616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9634154280 msgr2=0x7f963413d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:33.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.084+0000 7f963a35d640 1 --2- 172.21.15.40:0/234688616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9634154280 0x7f963413d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:33.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.084+0000 7f9639b5c640 1 --2- 172.21.15.40:0/234688616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9634154280 0x7f963413d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.084+0000 7f9639b5c640 1 -- 172.21.15.40:0/234688616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96341f5080 msgr2=0x7f9634140e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:33.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.084+0000 7f9639b5c640 1 --2- 172.21.15.40:0/234688616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96341f5080 0x7f9634140e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.084+0000 7f9639b5c640 1 -- 172.21.15.40:0/234688616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96341f5500 con 0x7f96200047c0 2023-10-18T08:58:33.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.084+0000 7f963935b640 1 --2- 172.21.15.40:0/234688616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96341f5080 0x7f9634140e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:33.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.084+0000 7f9639b5c640 1 --2- 172.21.15.40:0/234688616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96200047c0 0x7f963413e320 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f963000e5c0 tx=0x7f963000ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:33.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.085+0000 7f962a7fc640 1 -- 172.21.15.40:0/234688616 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963000c8f0 con 0x7f96200047c0 2023-10-18T08:58:33.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.085+0000 7f962a7fc640 1 -- 172.21.15.40:0/234688616 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f963000cb60 con 0x7f96200047c0 2023-10-18T08:58:33.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.085+0000 7f963bdc0640 1 -- 172.21.15.40:0/234688616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96341f57b0 con 0x7f96200047c0 2023-10-18T08:58:33.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.085+0000 7f962a7fc640 1 -- 172.21.15.40:0/234688616 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630011460 con 0x7f96200047c0 2023-10-18T08:58:33.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.085+0000 7f963bdc0640 1 -- 172.21.15.40:0/234688616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96341f5d70 con 0x7f96200047c0 2023-10-18T08:58:33.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.086+0000 7f963bdc0640 1 -- 172.21.15.40:0/234688616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9634066fd0 con 0x7f96200047c0 2023-10-18T08:58:33.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.087+0000 7f962a7fc640 1 -- 172.21.15.40:0/234688616 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f96300115c0 con 0x7f96200047c0 2023-10-18T08:58:33.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.088+0000 7f962a7fc640 1 --2- 172.21.15.40:0/234688616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f961806ae70 0x7f961806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:33.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.088+0000 7f962a7fc640 1 -- 172.21.15.40:0/234688616 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9630089de0 con 0x7f96200047c0 2023-10-18T08:58:33.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.089+0000 7f963935b640 1 --2- 172.21.15.40:0/234688616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f961806ae70 0x7f961806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:33.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.090+0000 7f963935b640 1 --2- 172.21.15.40:0/234688616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f961806ae70 0x7f961806d330 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9624037000 tx=0x7f96240046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:33.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.092+0000 7f962a7fc640 1 -- 172.21.15.40:0/234688616 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9630054df0 con 0x7f96200047c0 2023-10-18T08:58:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.287+0000 7f963bdc0640 1 -- 172.21.15.40:0/234688616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96340aeb50 con 0x7f96200047c0 2023-10-18T08:58:33.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.289+0000 7f962a7fc640 1 -- 172.21.15.40:0/234688616 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16577 (secure 0 0 0) 0x7f963001f020 con 0x7f96200047c0 2023-10-18T08:58:33.293 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:33.293 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":67,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:58:30.294245+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:58:26.727620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-18T08:58:27.795303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:58:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.292+0000 7f963bdc0640 1 -- 172.21.15.40:0/234688616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f961806ae70 msgr2=0x7f961806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.292+0000 7f963bdc0640 1 --2- 172.21.15.40:0/234688616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f961806ae70 0x7f961806d330 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9624037000 tx=0x7f96240046c0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.292+0000 7f963bdc0640 1 -- 172.21.15.40:0/234688616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96200047c0 msgr2=0x7f963413e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.292+0000 7f963bdc0640 1 --2- 172.21.15.40:0/234688616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96200047c0 0x7f963413e320 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f963000e5c0 tx=0x7f963000ea80 comp rx=0 tx=0).stop 2023-10-18T08:58:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.292+0000 7f963bdc0640 1 -- 172.21.15.40:0/234688616 shutdown_connections 2023-10-18T08:58:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.292+0000 7f963bdc0640 1 --2- 172.21.15.40:0/234688616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f961806ae70 0x7f961806d330 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.292+0000 7f963bdc0640 1 --2- 172.21.15.40:0/234688616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9634154280 0x7f963413d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.292+0000 7f963bdc0640 1 --2- 172.21.15.40:0/234688616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96341f5080 0x7f9634140e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.292+0000 7f963bdc0640 1 --2- 172.21.15.40:0/234688616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96200047c0 0x7f963413e320 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.292+0000 7f963bdc0640 1 -- 172.21.15.40:0/234688616 >> 172.21.15.40:0/234688616 conn(0x7f963406cee0 msgr2=0x7f96341537a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:33.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.292+0000 7f963bdc0640 1 -- 172.21.15.40:0/234688616 shutdown_connections 2023-10-18T08:58:33.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.292+0000 7f963bdc0640 1 -- 172.21.15.40:0/234688616 wait complete. 2023-10-18T08:58:33.315 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T08:58:33.315 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:58:33.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.541+0000 7f3ee676c640 1 -- 172.21.15.40:0/2084642373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee00ca3a0 msgr2=0x7f3ee00ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:33.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.541+0000 7f3ee676c640 1 --2- 172.21.15.40:0/2084642373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee00ca3a0 0x7f3ee00ccf20 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0005fd0 tx=0x7f3ed002c2d0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.541+0000 7f3ee676c640 1 -- 172.21.15.40:0/2084642373 shutdown_connections 2023-10-18T08:58:33.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.541+0000 7f3ee676c640 1 --2- 172.21.15.40:0/2084642373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee00b15f0 0x7f3ee015c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.541+0000 7f3ee676c640 1 --2- 172.21.15.40:0/2084642373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee00ca3a0 0x7f3ee00ccf20 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.541+0000 7f3ee676c640 1 -- 172.21.15.40:0/2084642373 >> 172.21.15.40:0/2084642373 conn(0x7f3ee00cf3e0 msgr2=0x7f3ee00cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:33.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.541+0000 7f3ee676c640 1 -- 172.21.15.40:0/2084642373 shutdown_connections 2023-10-18T08:58:33.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.541+0000 7f3ee676c640 1 -- 172.21.15.40:0/2084642373 wait complete. 2023-10-18T08:58:33.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.543+0000 7f3ee676c640 1 Processor -- start 2023-10-18T08:58:33.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.543+0000 7f3ee676c640 1 -- start start 2023-10-18T08:58:33.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.544+0000 7f3ee676c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee00b15f0 0x7f3ee00b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:33.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.544+0000 7f3ee676c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee00ca3a0 0x7f3ee00c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:33.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.544+0000 7f3ee676c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee015f400 0x7f3ee0159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:33.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.544+0000 7f3ee676c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee00b13f0 con 0x7f3ee00b15f0 2023-10-18T08:58:33.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.544+0000 7f3ee676c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee01f57e0 con 0x7f3ee00ca3a0 2023-10-18T08:58:33.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.544+0000 7f3ee676c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee01f5ac0 con 0x7f3ee015f400 2023-10-18T08:58:33.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.544+0000 7f3edffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee00b15f0 0x7f3ee00b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:33.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.544+0000 7f3edffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee00b15f0 0x7f3ee00b0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38510/0 (socket says 172.21.15.40:38510) 2023-10-18T08:58:33.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.544+0000 7f3edffff640 1 -- 172.21.15.40:0/3129435823 learned_addr learned my addr 172.21.15.40:0/3129435823 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:33.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.544+0000 7f3edffff640 1 -- 172.21.15.40:0/3129435823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee015f400 msgr2=0x7f3ee0159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:33.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.544+0000 7f3ee4d09640 1 --2- 172.21.15.40:0/3129435823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee015f400 0x7f3ee0159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:33.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.545+0000 7f3edffff640 1 --2- 172.21.15.40:0/3129435823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee015f400 0x7f3ee0159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.545+0000 7f3edffff640 1 -- 172.21.15.40:0/3129435823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee00ca3a0 msgr2=0x7f3ee00c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:33.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.545+0000 7f3edf7fe640 1 --2- 172.21.15.40:0/3129435823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee00ca3a0 0x7f3ee00c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:33.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.545+0000 7f3edffff640 1 --2- 172.21.15.40:0/3129435823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee00ca3a0 0x7f3ee00c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.545+0000 7f3edffff640 1 -- 172.21.15.40:0/3129435823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed0009900 con 0x7f3ee00b15f0 2023-10-18T08:58:33.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.545+0000 7f3edf7fe640 1 --2- 172.21.15.40:0/3129435823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee00ca3a0 0x7f3ee00c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:33.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.545+0000 7f3edffff640 1 --2- 172.21.15.40:0/3129435823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee00b15f0 0x7f3ee00b0eb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0006010 tx=0x7f3ed00093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:33.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.545+0000 7f3edcff9640 1 -- 172.21.15.40:0/3129435823 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0004240 con 0x7f3ee00b15f0 2023-10-18T08:58:33.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.545+0000 7f3edcff9640 1 -- 172.21.15.40:0/3129435823 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3ed0004490 con 0x7f3ee00b15f0 2023-10-18T08:58:33.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.545+0000 7f3edcff9640 1 -- 172.21.15.40:0/3129435823 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0004840 con 0x7f3ee00b15f0 2023-10-18T08:58:33.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.545+0000 7f3ee676c640 1 -- 172.21.15.40:0/3129435823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee01f5da0 con 0x7f3ee00b15f0 2023-10-18T08:58:33.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.546+0000 7f3ee676c640 1 -- 172.21.15.40:0/3129435823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee00c5b60 con 0x7f3ee00b15f0 2023-10-18T08:58:33.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.548+0000 7f3ee676c640 1 -- 172.21.15.40:0/3129435823 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee0066d40 con 0x7f3ee00b15f0 2023-10-18T08:58:33.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.549+0000 7f3edcff9640 1 -- 172.21.15.40:0/3129435823 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3ed003b040 con 0x7f3ee00b15f0 2023-10-18T08:58:33.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.550+0000 7f3edcff9640 1 --2- 172.21.15.40:0/3129435823 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ec006ae40 0x7f3ec006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:33.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.550+0000 7f3edcff9640 1 -- 172.21.15.40:0/3129435823 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ed00ac530 con 0x7f3ee00b15f0 2023-10-18T08:58:33.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.550+0000 7f3edf7fe640 1 --2- 172.21.15.40:0/3129435823 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ec006ae40 0x7f3ec006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:33.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.551+0000 7f3edf7fe640 1 --2- 172.21.15.40:0/3129435823 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ec006ae40 0x7f3ec006d300 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00c5f80 tx=0x7f3ec8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:33.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.555+0000 7f3edcff9640 1 -- 172.21.15.40:0/3129435823 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ed00775e0 con 0x7f3ee00b15f0 2023-10-18T08:58:33.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.746+0000 7f3ee676c640 1 -- 172.21.15.40:0/3129435823 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ee01410e0 con 0x7f3ec006ae40 2023-10-18T08:58:33.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.774+0000 7f3edcff9640 1 -- 172.21.15.40:0/3129435823 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279218 (secure 0 0 0) 0x7f3ec80013e0 con 0x7f3ec006ae40 2023-10-18T08:58:33.779 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:33.781 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:58:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.779+0000 7f3ee676c640 1 -- 172.21.15.40:0/3129435823 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ec006ae40 msgr2=0x7f3ec006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.779+0000 7f3ee676c640 1 --2- 172.21.15.40:0/3129435823 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ec006ae40 0x7f3ec006d300 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00c5f80 tx=0x7f3ec8009040 comp rx=0 tx=0).stop 2023-10-18T08:58:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.779+0000 7f3ee676c640 1 -- 172.21.15.40:0/3129435823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee00b15f0 msgr2=0x7f3ee00b0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.779+0000 7f3ee676c640 1 --2- 172.21.15.40:0/3129435823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee00b15f0 0x7f3ee00b0eb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0006010 tx=0x7f3ed00093f0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.779+0000 7f3ee676c640 1 -- 172.21.15.40:0/3129435823 shutdown_connections 2023-10-18T08:58:33.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.779+0000 7f3ee676c640 1 --2- 172.21.15.40:0/3129435823 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ec006ae40 0x7f3ec006d300 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.779+0000 7f3ee676c640 1 --2- 172.21.15.40:0/3129435823 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee015f400 0x7f3ee0159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.779+0000 7f3ee676c640 1 --2- 172.21.15.40:0/3129435823 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee00ca3a0 0x7f3ee00c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.779+0000 7f3ee676c640 1 --2- 172.21.15.40:0/3129435823 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee00b15f0 0x7f3ee00b0eb0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:33.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.779+0000 7f3ee676c640 1 -- 172.21.15.40:0/3129435823 >> 172.21.15.40:0/3129435823 conn(0x7f3ee00cf3e0 msgr2=0x7f3ee006f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:33.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.779+0000 7f3ee676c640 1 -- 172.21.15.40:0/3129435823 shutdown_connections 2023-10-18T08:58:33.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:33.779+0000 7f3ee676c640 1 -- 172.21.15.40:0/3129435823 wait complete. 2023-10-18T08:58:33.801 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2023-10-18T08:58:32.964940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590514,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3746,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558753,"statfs":{"total":1600000000,"available":1596163471,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":146,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.697568"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.290178+0000","last_change":"2023-10-18T08:58:28.290178+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.290178+0000","last_undegraded":"2023-10-18T08:58:28.290178+0000","last_fullsized":"2023-10-18T08:58:28.290178+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.310859+0000","last_change":"2023-10-18T08:58:29.310661+0000","last_active":"2023-10-18T08:58:29.310859+0000","last_peered":"2023-10-18T08:58:29.310859+0000","last_clean":"2023-10-18T08:58:29.310859+0000","last_became_active":"2023-10-18T08:58:29.309557+0000","last_became_peered":"2023-10-18T08:58:29.309557+0000","last_unstale":"2023-10-18T08:58:29.310859+0000","last_undegraded":"2023-10-18T08:58:29.310859+0000","last_fullsized":"2023-10-18T08:58:29.310859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:46.399620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.323795+0000","last_change":"2023-10-18T08:58:29.321404+0000","last_active":"2023-10-18T08:58:29.323795+0000","last_peered":"2023-10-18T08:58:29.323795+0000","last_clean":"2023-10-18T08:58:29.323795+0000","last_became_active":"2023-10-18T08:58:29.320985+0000","last_became_peered":"2023-10-18T08:58:29.320985+0000","last_unstale":"2023-10-18T08:58:29.323795+0000","last_undegraded":"2023-10-18T08:58:29.323795+0000","last_fullsized":"2023-10-18T08:58:29.323795+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:10:04.504829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.293989+0000","last_change":"2023-10-18T08:58:28.293989+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.293989+0000","last_undegraded":"2023-10-18T08:58:28.293989+0000","last_fullsized":"2023-10-18T08:58:28.293989+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.291475+0000","last_change":"2023-10-18T08:58:28.291475+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.291475+0000","last_undegraded":"2023-10-18T08:58:28.291475+0000","last_fullsized":"2023-10-18T08:58:28.291475+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.287056+0000","last_change":"2023-10-18T08:58:28.287056+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.287056+0000","last_undegraded":"2023-10-18T08:58:28.287056+0000","last_fullsized":"2023-10-18T08:58:28.287056+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.290553+0000","last_change":"2023-10-18T08:58:28.290553+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.290553+0000","last_undegraded":"2023-10-18T08:58:28.290553+0000","last_fullsized":"2023-10-18T08:58:28.290553+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326822+0000","last_change":"2023-10-18T08:58:29.325559+0000","last_active":"2023-10-18T08:58:29.326822+0000","last_peered":"2023-10-18T08:58:29.326822+0000","last_clean":"2023-10-18T08:58:29.326822+0000","last_became_active":"2023-10-18T08:58:29.324359+0000","last_became_peered":"2023-10-18T08:58:29.324359+0000","last_unstale":"2023-10-18T08:58:29.326822+0000","last_undegraded":"2023-10-18T08:58:29.326822+0000","last_fullsized":"2023-10-18T08:58:29.326822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:45.688349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.294163+0000","last_change":"2023-10-18T08:58:28.294163+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.294163+0000","last_undegraded":"2023-10-18T08:58:28.294163+0000","last_fullsized":"2023-10-18T08:58:28.294163+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.278822+0000","last_change":"2023-10-18T08:58:28.278822+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.278822+0000","last_undegraded":"2023-10-18T08:58:28.278822+0000","last_fullsized":"2023-10-18T08:58:28.278822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.277474+0000","last_change":"2023-10-18T08:58:28.277474+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.277474+0000","last_undegraded":"2023-10-18T08:58:28.277474+0000","last_fullsized":"2023-10-18T08:58:28.277474+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.290401+0000","last_change":"2023-10-18T08:58:28.290401+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.290401+0000","last_undegraded":"2023-10-18T08:58:28.290401+0000","last_fullsized":"2023-10-18T08:58:28.290401+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.283202+0000","last_change":"2023-10-18T08:58:28.283202+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.283202+0000","last_undegraded":"2023-10-18T08:58:28.283202+0000","last_fullsized":"2023-10-18T08:58:28.283202+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.293634+0000","last_change":"2023-10-18T08:58:28.293634+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.293634+0000","last_undegraded":"2023-10-18T08:58:28.293634+0000","last_fullsized":"2023-10-18T08:58:28.293634+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289054+0000","last_change":"2023-10-18T08:58:28.289054+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289054+0000","last_undegraded":"2023-10-18T08:58:28.289054+0000","last_fullsized":"2023-10-18T08:58:28.289054+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.285983+0000","last_change":"2023-10-18T08:58:28.285983+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.285983+0000","last_undegraded":"2023-10-18T08:58:28.285983+0000","last_fullsized":"2023-10-18T08:58:28.285983+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326448+0000","last_change":"2023-10-18T08:58:29.325090+0000","last_active":"2023-10-18T08:58:29.326448+0000","last_peered":"2023-10-18T08:58:29.326448+0000","last_clean":"2023-10-18T08:58:29.326448+0000","last_became_active":"2023-10-18T08:58:29.323675+0000","last_became_peered":"2023-10-18T08:58:29.323675+0000","last_unstale":"2023-10-18T08:58:29.326448+0000","last_undegraded":"2023-10-18T08:58:29.326448+0000","last_fullsized":"2023-10-18T08:58:29.326448+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:01:55.373464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.329948+0000","last_change":"2023-10-18T08:58:29.329020+0000","last_active":"2023-10-18T08:58:29.329948+0000","last_peered":"2023-10-18T08:58:29.329948+0000","last_clean":"2023-10-18T08:58:29.329948+0000","last_became_active":"2023-10-18T08:58:29.328773+0000","last_became_peered":"2023-10-18T08:58:29.328773+0000","last_unstale":"2023-10-18T08:58:29.329948+0000","last_undegraded":"2023-10-18T08:58:29.329948+0000","last_fullsized":"2023-10-18T08:58:29.329948+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:57.274523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289622+0000","last_change":"2023-10-18T08:58:28.289622+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289622+0000","last_undegraded":"2023-10-18T08:58:28.289622+0000","last_fullsized":"2023-10-18T08:58:28.289622+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.330756+0000","last_change":"2023-10-18T08:58:29.330497+0000","last_active":"2023-10-18T08:58:29.330756+0000","last_peered":"2023-10-18T08:58:29.330756+0000","last_clean":"2023-10-18T08:58:29.330756+0000","last_became_active":"2023-10-18T08:58:29.330245+0000","last_became_peered":"2023-10-18T08:58:29.330245+0000","last_unstale":"2023-10-18T08:58:29.330756+0000","last_undegraded":"2023-10-18T08:58:29.330756+0000","last_fullsized":"2023-10-18T08:58:29.330756+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:23.135334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.323823+0000","last_change":"2023-10-18T08:58:29.323242+0000","last_active":"2023-10-18T08:58:29.323823+0000","last_peered":"2023-10-18T08:58:29.323823+0000","last_clean":"2023-10-18T08:58:29.323823+0000","last_became_active":"2023-10-18T08:58:29.322832+0000","last_became_peered":"2023-10-18T08:58:29.322832+0000","last_unstale":"2023-10-18T08:58:29.323823+0000","last_undegraded":"2023-10-18T08:58:29.323823+0000","last_fullsized":"2023-10-18T08:58:29.323823+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:10:00.029939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326366+0000","last_change":"2023-10-18T08:58:29.319011+0000","last_active":"2023-10-18T08:58:29.326366+0000","last_peered":"2023-10-18T08:58:29.326366+0000","last_clean":"2023-10-18T08:58:29.326366+0000","last_became_active":"2023-10-18T08:58:29.318536+0000","last_became_peered":"2023-10-18T08:58:29.318536+0000","last_unstale":"2023-10-18T08:58:29.326366+0000","last_undegraded":"2023-10-18T08:58:29.326366+0000","last_fullsized":"2023-10-18T08:58:29.326366+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:16.123758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.319916+0000","last_change":"2023-10-18T08:58:29.319576+0000","last_active":"2023-10-18T08:58:29.319916+0000","last_peered":"2023-10-18T08:58:29.319916+0000","last_clean":"2023-10-18T08:58:29.319916+0000","last_became_active":"2023-10-18T08:58:29.316842+0000","last_became_peered":"2023-10-18T08:58:29.316842+0000","last_unstale":"2023-10-18T08:58:29.319916+0000","last_undegraded":"2023-10-18T08:58:29.319916+0000","last_fullsized":"2023-10-18T08:58:29.319916+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:45.620650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.281482+0000","last_change":"2023-10-18T08:58:28.281482+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.281482+0000","last_undegraded":"2023-10-18T08:58:28.281482+0000","last_fullsized":"2023-10-18T08:58:28.281482+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.291034+0000","last_change":"2023-10-18T08:58:28.291034+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.291034+0000","last_undegraded":"2023-10-18T08:58:28.291034+0000","last_fullsized":"2023-10-18T08:58:28.291034+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.291241+0000","last_change":"2023-10-18T08:58:28.291241+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.291241+0000","last_undegraded":"2023-10-18T08:58:28.291241+0000","last_fullsized":"2023-10-18T08:58:28.291241+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.285161+0000","last_change":"2023-10-18T08:58:28.285161+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.285161+0000","last_undegraded":"2023-10-18T08:58:28.285161+0000","last_fullsized":"2023-10-18T08:58:28.285161+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289003+0000","last_change":"2023-10-18T08:58:28.289003+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289003+0000","last_undegraded":"2023-10-18T08:58:28.289003+0000","last_fullsized":"2023-10-18T08:58:28.289003+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.329742+0000","last_change":"2023-10-18T08:58:29.329658+0000","last_active":"2023-10-18T08:58:29.329742+0000","last_peered":"2023-10-18T08:58:29.329742+0000","last_clean":"2023-10-18T08:58:29.329742+0000","last_became_active":"2023-10-18T08:58:29.329393+0000","last_became_peered":"2023-10-18T08:58:29.329393+0000","last_unstale":"2023-10-18T08:58:29.329742+0000","last_undegraded":"2023-10-18T08:58:29.329742+0000","last_fullsized":"2023-10-18T08:58:29.329742+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:28:25.454956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326242+0000","last_change":"2023-10-18T08:58:29.326075+0000","last_active":"2023-10-18T08:58:29.326242+0000","last_peered":"2023-10-18T08:58:29.326242+0000","last_clean":"2023-10-18T08:58:29.326242+0000","last_became_active":"2023-10-18T08:58:29.324858+0000","last_became_peered":"2023-10-18T08:58:29.324858+0000","last_unstale":"2023-10-18T08:58:29.326242+0000","last_undegraded":"2023-10-18T08:58:29.326242+0000","last_fullsized":"2023-10-18T08:58:29.326242+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:49.015165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289440+0000","last_change":"2023-10-18T08:58:28.289440+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289440+0000","last_undegraded":"2023-10-18T08:58:28.289440+0000","last_fullsized":"2023-10-18T08:58:28.289440+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.284606+0000","last_change":"2023-10-18T08:58:28.284606+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.284606+0000","last_undegraded":"2023-10-18T08:58:28.284606+0000","last_fullsized":"2023-10-18T08:58:28.284606+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.288895+0000","last_change":"2023-10-18T08:58:28.288895+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.288895+0000","last_undegraded":"2023-10-18T08:58:28.288895+0000","last_fullsized":"2023-10-18T08:58:28.288895+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.301110+0000","last_change":"2023-10-18T08:58:28.300855+0000","last_active":"2023-10-18T08:58:28.301110+0000","last_peered":"2023-10-18T08:58:28.301110+0000","last_clean":"2023-10-18T08:58:28.301110+0000","last_became_active":"2023-10-18T08:58:28.300672+0000","last_became_peered":"2023-10-18T08:58:28.300672+0000","last_unstale":"2023-10-18T08:58:28.301110+0000","last_undegraded":"2023-10-18T08:58:28.301110+0000","last_fullsized":"2023-10-18T08:58:28.301110+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:57:22.719753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.277530+0000","last_change":"2023-10-18T08:58:28.277530+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.277530+0000","last_undegraded":"2023-10-18T08:58:28.277530+0000","last_fullsized":"2023-10-18T08:58:28.277530+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"66'1","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.342802+0000","last_change":"2023-10-18T08:58:28.295079+0000","last_active":"2023-10-18T08:58:29.342802+0000","last_peered":"2023-10-18T08:58:29.342802+0000","last_clean":"2023-10-18T08:58:29.342802+0000","last_became_active":"2023-10-18T08:58:28.294485+0000","last_became_peered":"2023-10-18T08:58:28.294485+0000","last_unstale":"2023-10-18T08:58:29.342802+0000","last_undegraded":"2023-10-18T08:58:29.342802+0000","last_fullsized":"2023-10-18T08:58:29.342802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:27.603595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.286466+0000","last_change":"2023-10-18T08:58:28.286466+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.286466+0000","last_undegraded":"2023-10-18T08:58:28.286466+0000","last_fullsized":"2023-10-18T08:58:28.286466+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.299117+0000","last_change":"2023-10-18T08:58:28.298521+0000","last_active":"2023-10-18T08:58:28.299117+0000","last_peered":"2023-10-18T08:58:28.299117+0000","last_clean":"2023-10-18T08:58:28.299117+0000","last_became_active":"2023-10-18T08:58:28.296841+0000","last_became_peered":"2023-10-18T08:58:28.296841+0000","last_unstale":"2023-10-18T08:58:28.299117+0000","last_undegraded":"2023-10-18T08:58:28.299117+0000","last_fullsized":"2023-10-18T08:58:28.299117+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:24.568573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289868+0000","last_change":"2023-10-18T08:58:28.289868+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289868+0000","last_undegraded":"2023-10-18T08:58:28.289868+0000","last_fullsized":"2023-10-18T08:58:28.289868+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.299261+0000","last_change":"2023-10-18T08:58:28.298876+0000","last_active":"2023-10-18T08:58:28.299261+0000","last_peered":"2023-10-18T08:58:28.299261+0000","last_clean":"2023-10-18T08:58:28.299261+0000","last_became_active":"2023-10-18T08:58:28.297696+0000","last_became_peered":"2023-10-18T08:58:28.297696+0000","last_unstale":"2023-10-18T08:58:28.299261+0000","last_undegraded":"2023-10-18T08:58:28.299261+0000","last_fullsized":"2023-10-18T08:58:28.299261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:07:47.505124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326799+0000","last_change":"2023-10-18T08:58:29.325300+0000","last_active":"2023-10-18T08:58:29.326799+0000","last_peered":"2023-10-18T08:58:29.326799+0000","last_clean":"2023-10-18T08:58:29.326799+0000","last_became_active":"2023-10-18T08:58:29.323662+0000","last_became_peered":"2023-10-18T08:58:29.323662+0000","last_unstale":"2023-10-18T08:58:29.326799+0000","last_undegraded":"2023-10-18T08:58:29.326799+0000","last_fullsized":"2023-10-18T08:58:29.326799+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:39:03.787381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300750+0000","last_change":"2023-10-18T08:58:28.299820+0000","last_active":"2023-10-18T08:58:28.300750+0000","last_peered":"2023-10-18T08:58:28.300750+0000","last_clean":"2023-10-18T08:58:28.300750+0000","last_became_active":"2023-10-18T08:58:28.299113+0000","last_became_peered":"2023-10-18T08:58:28.299113+0000","last_unstale":"2023-10-18T08:58:28.300750+0000","last_undegraded":"2023-10-18T08:58:28.300750+0000","last_fullsized":"2023-10-18T08:58:28.300750+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:59.489387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.332542+0000","last_change":"2023-10-18T08:58:29.332297+0000","last_active":"2023-10-18T08:58:29.332542+0000","last_peered":"2023-10-18T08:58:29.332542+0000","last_clean":"2023-10-18T08:58:29.332542+0000","last_became_active":"2023-10-18T08:58:29.332051+0000","last_became_peered":"2023-10-18T08:58:29.332051+0000","last_unstale":"2023-10-18T08:58:29.332542+0000","last_undegraded":"2023-10-18T08:58:29.332542+0000","last_fullsized":"2023-10-18T08:58:29.332542+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:30:53.581720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300168+0000","last_change":"2023-10-18T08:58:28.300048+0000","last_active":"2023-10-18T08:58:28.300168+0000","last_peered":"2023-10-18T08:58:28.300168+0000","last_clean":"2023-10-18T08:58:28.300168+0000","last_became_active":"2023-10-18T08:58:28.299438+0000","last_became_peered":"2023-10-18T08:58:28.299438+0000","last_unstale":"2023-10-18T08:58:28.300168+0000","last_undegraded":"2023-10-18T08:58:28.300168+0000","last_fullsized":"2023-10-18T08:58:28.300168+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:53.262221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.281849+0000","last_change":"2023-10-18T08:58:28.281849+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.281849+0000","last_undegraded":"2023-10-18T08:58:28.281849+0000","last_fullsized":"2023-10-18T08:58:28.281849+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.301494+0000","last_change":"2023-10-18T08:58:28.301417+0000","last_active":"2023-10-18T08:58:28.301494+0000","last_peered":"2023-10-18T08:58:28.301494+0000","last_clean":"2023-10-18T08:58:28.301494+0000","last_became_active":"2023-10-18T08:58:28.300999+0000","last_became_peered":"2023-10-18T08:58:28.300999+0000","last_unstale":"2023-10-18T08:58:28.301494+0000","last_undegraded":"2023-10-18T08:58:28.301494+0000","last_fullsized":"2023-10-18T08:58:28.301494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:30.254470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326387+0000","last_change":"2023-10-18T08:58:29.321607+0000","last_active":"2023-10-18T08:58:29.326387+0000","last_peered":"2023-10-18T08:58:29.326387+0000","last_clean":"2023-10-18T08:58:29.326387+0000","last_became_active":"2023-10-18T08:58:29.321062+0000","last_became_peered":"2023-10-18T08:58:29.321062+0000","last_unstale":"2023-10-18T08:58:29.326387+0000","last_undegraded":"2023-10-18T08:58:29.326387+0000","last_fullsized":"2023-10-18T08:58:29.326387+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:20.541018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.302195+0000","last_change":"2023-10-18T08:58:28.302109+0000","last_active":"2023-10-18T08:58:28.302195+0000","last_peered":"2023-10-18T08:58:28.302195+0000","last_clean":"2023-10-18T08:58:28.302195+0000","last_became_active":"2023-10-18T08:58:28.301703+0000","last_became_peered":"2023-10-18T08:58:28.301703+0000","last_unstale":"2023-10-18T08:58:28.302195+0000","last_undegraded":"2023-10-18T08:58:28.302195+0000","last_fullsized":"2023-10-18T08:58:28.302195+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:57.346205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289722+0000","last_change":"2023-10-18T08:58:28.289722+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289722+0000","last_undegraded":"2023-10-18T08:58:28.289722+0000","last_fullsized":"2023-10-18T08:58:28.289722+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.305138+0000","last_change":"2023-10-18T08:58:28.298815+0000","last_active":"2023-10-18T08:58:29.305138+0000","last_peered":"2023-10-18T08:58:29.305138+0000","last_clean":"2023-10-18T08:58:29.305138+0000","last_became_active":"2023-10-18T08:58:28.298503+0000","last_became_peered":"2023-10-18T08:58:28.298503+0000","last_unstale":"2023-10-18T08:58:29.305138+0000","last_undegraded":"2023-10-18T08:58:29.305138+0000","last_fullsized":"2023-10-18T08:58:29.305138+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:39.750254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313431+0000","last_change":"2023-10-18T08:58:29.312112+0000","last_active":"2023-10-18T08:58:29.313431+0000","last_peered":"2023-10-18T08:58:29.313431+0000","last_clean":"2023-10-18T08:58:29.313431+0000","last_became_active":"2023-10-18T08:58:29.310708+0000","last_became_peered":"2023-10-18T08:58:29.310708+0000","last_unstale":"2023-10-18T08:58:29.313431+0000","last_undegraded":"2023-10-18T08:58:29.313431+0000","last_fullsized":"2023-10-18T08:58:29.313431+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:24.996549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.306615+0000","last_change":"2023-10-18T08:58:28.298222+0000","last_active":"2023-10-18T08:58:29.306615+0000","last_peered":"2023-10-18T08:58:29.306615+0000","last_clean":"2023-10-18T08:58:29.306615+0000","last_became_active":"2023-10-18T08:58:28.297672+0000","last_became_peered":"2023-10-18T08:58:28.297672+0000","last_unstale":"2023-10-18T08:58:29.306615+0000","last_undegraded":"2023-10-18T08:58:29.306615+0000","last_fullsized":"2023-10-18T08:58:29.306615+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:59.505404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.277176+0000","last_change":"2023-10-18T08:58:28.277176+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.277176+0000","last_undegraded":"2023-10-18T08:58:28.277176+0000","last_fullsized":"2023-10-18T08:58:28.277176+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"66'3","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.344696+0000","last_change":"2023-10-18T08:58:28.290903+0000","last_active":"2023-10-18T08:58:29.344696+0000","last_peered":"2023-10-18T08:58:29.344696+0000","last_clean":"2023-10-18T08:58:29.344696+0000","last_became_active":"2023-10-18T08:58:28.290358+0000","last_became_peered":"2023-10-18T08:58:28.290358+0000","last_unstale":"2023-10-18T08:58:29.344696+0000","last_undegraded":"2023-10-18T08:58:29.344696+0000","last_fullsized":"2023-10-18T08:58:29.344696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:24.568573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.320694+0000","last_change":"2023-10-18T08:58:29.320149+0000","last_active":"2023-10-18T08:58:29.320694+0000","last_peered":"2023-10-18T08:58:29.320694+0000","last_clean":"2023-10-18T08:58:29.320694+0000","last_became_active":"2023-10-18T08:58:29.317356+0000","last_became_peered":"2023-10-18T08:58:29.317356+0000","last_unstale":"2023-10-18T08:58:29.320694+0000","last_undegraded":"2023-10-18T08:58:29.320694+0000","last_fullsized":"2023-10-18T08:58:29.320694+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:49.888147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.302112+0000","last_change":"2023-10-18T08:58:28.302001+0000","last_active":"2023-10-18T08:58:28.302112+0000","last_peered":"2023-10-18T08:58:28.302112+0000","last_clean":"2023-10-18T08:58:28.302112+0000","last_became_active":"2023-10-18T08:58:28.301382+0000","last_became_peered":"2023-10-18T08:58:28.301382+0000","last_unstale":"2023-10-18T08:58:28.302112+0000","last_undegraded":"2023-10-18T08:58:28.302112+0000","last_fullsized":"2023-10-18T08:58:28.302112+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:43:35.327448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.290401+0000","last_change":"2023-10-18T08:58:28.290401+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.290401+0000","last_undegraded":"2023-10-18T08:58:28.290401+0000","last_fullsized":"2023-10-18T08:58:28.290401+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"66'2","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341261+0000","last_change":"2023-10-18T08:58:28.296421+0000","last_active":"2023-10-18T08:58:29.341261+0000","last_peered":"2023-10-18T08:58:29.341261+0000","last_clean":"2023-10-18T08:58:29.341261+0000","last_became_active":"2023-10-18T08:58:28.296176+0000","last_became_peered":"2023-10-18T08:58:28.296176+0000","last_unstale":"2023-10-18T08:58:29.341261+0000","last_undegraded":"2023-10-18T08:58:29.341261+0000","last_fullsized":"2023-10-18T08:58:29.341261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:14:17.772896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.324023+0000","last_change":"2023-10-18T08:58:29.322440+0000","last_active":"2023-10-18T08:58:29.324023+0000","last_peered":"2023-10-18T08:58:29.324023+0000","last_clean":"2023-10-18T08:58:29.324023+0000","last_became_active":"2023-10-18T08:58:29.319010+0000","last_became_peered":"2023-10-18T08:58:29.319010+0000","last_unstale":"2023-10-18T08:58:29.324023+0000","last_undegraded":"2023-10-18T08:58:29.324023+0000","last_fullsized":"2023-10-18T08:58:29.324023+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:05.722853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.297280+0000","last_change":"2023-10-18T08:58:28.290878+0000","last_active":"2023-10-18T08:58:28.297280+0000","last_peered":"2023-10-18T08:58:28.297280+0000","last_clean":"2023-10-18T08:58:28.297280+0000","last_became_active":"2023-10-18T08:58:28.290244+0000","last_became_peered":"2023-10-18T08:58:28.290244+0000","last_unstale":"2023-10-18T08:58:28.297280+0000","last_undegraded":"2023-10-18T08:58:28.297280+0000","last_fullsized":"2023-10-18T08:58:28.297280+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:01.096478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.288004+0000","last_change":"2023-10-18T08:58:28.288004+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.288004+0000","last_undegraded":"2023-10-18T08:58:28.288004+0000","last_fullsized":"2023-10-18T08:58:28.288004+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300048+0000","last_change":"2023-10-18T08:58:28.299958+0000","last_active":"2023-10-18T08:58:28.300048+0000","last_peered":"2023-10-18T08:58:28.300048+0000","last_clean":"2023-10-18T08:58:28.300048+0000","last_became_active":"2023-10-18T08:58:28.299673+0000","last_became_peered":"2023-10-18T08:58:28.299673+0000","last_unstale":"2023-10-18T08:58:28.300048+0000","last_undegraded":"2023-10-18T08:58:28.300048+0000","last_fullsized":"2023-10-18T08:58:28.300048+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:08.788078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313474+0000","last_change":"2023-10-18T08:58:29.313209+0000","last_active":"2023-10-18T08:58:29.313474+0000","last_peered":"2023-10-18T08:58:29.313474+0000","last_clean":"2023-10-18T08:58:29.313474+0000","last_became_active":"2023-10-18T08:58:29.312599+0000","last_became_peered":"2023-10-18T08:58:29.312599+0000","last_unstale":"2023-10-18T08:58:29.313474+0000","last_undegraded":"2023-10-18T08:58:29.313474+0000","last_fullsized":"2023-10-18T08:58:29.313474+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:33:04.626358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"66'8","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.343659+0000","last_change":"2023-10-18T08:58:28.297447+0000","last_active":"2023-10-18T08:58:29.343659+0000","last_peered":"2023-10-18T08:58:29.343659+0000","last_clean":"2023-10-18T08:58:29.343659+0000","last_became_active":"2023-10-18T08:58:28.297055+0000","last_became_peered":"2023-10-18T08:58:28.297055+0000","last_unstale":"2023-10-18T08:58:29.343659+0000","last_undegraded":"2023-10-18T08:58:29.343659+0000","last_fullsized":"2023-10-18T08:58:29.343659+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:57:22.719753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326347+0000","last_change":"2023-10-18T08:58:29.326263+0000","last_active":"2023-10-18T08:58:29.326347+0000","last_peered":"2023-10-18T08:58:29.326347+0000","last_clean":"2023-10-18T08:58:29.326347+0000","last_became_active":"2023-10-18T08:58:29.325851+0000","last_became_peered":"2023-10-18T08:58:29.325851+0000","last_unstale":"2023-10-18T08:58:29.326347+0000","last_undegraded":"2023-10-18T08:58:29.326347+0000","last_fullsized":"2023-10-18T08:58:29.326347+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:42:38.047653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"66'1","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.339858+0000","last_change":"2023-10-18T08:58:28.300004+0000","last_active":"2023-10-18T08:58:29.339858+0000","last_peered":"2023-10-18T08:58:29.339858+0000","last_clean":"2023-10-18T08:58:29.339858+0000","last_became_active":"2023-10-18T08:58:28.294830+0000","last_became_peered":"2023-10-18T08:58:28.294830+0000","last_unstale":"2023-10-18T08:58:29.339858+0000","last_undegraded":"2023-10-18T08:58:29.339858+0000","last_fullsized":"2023-10-18T08:58:29.339858+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:16.750986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.293219+0000","last_change":"2023-10-18T08:58:28.293219+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.293219+0000","last_undegraded":"2023-10-18T08:58:28.293219+0000","last_fullsized":"2023-10-18T08:58:28.293219+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.298454+0000","last_change":"2023-10-18T08:58:28.298360+0000","last_active":"2023-10-18T08:58:28.298454+0000","last_peered":"2023-10-18T08:58:28.298454+0000","last_clean":"2023-10-18T08:58:28.298454+0000","last_became_active":"2023-10-18T08:58:28.296520+0000","last_became_peered":"2023-10-18T08:58:28.296520+0000","last_unstale":"2023-10-18T08:58:28.298454+0000","last_undegraded":"2023-10-18T08:58:28.298454+0000","last_fullsized":"2023-10-18T08:58:28.298454+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:48.874512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326429+0000","last_change":"2023-10-18T08:58:29.324872+0000","last_active":"2023-10-18T08:58:29.326429+0000","last_peered":"2023-10-18T08:58:29.326429+0000","last_clean":"2023-10-18T08:58:29.326429+0000","last_became_active":"2023-10-18T08:58:29.323103+0000","last_became_peered":"2023-10-18T08:58:29.323103+0000","last_unstale":"2023-10-18T08:58:29.326429+0000","last_undegraded":"2023-10-18T08:58:29.326429+0000","last_fullsized":"2023-10-18T08:58:29.326429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:11:33.988349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.299240+0000","last_change":"2023-10-18T08:58:28.298668+0000","last_active":"2023-10-18T08:58:28.299240+0000","last_peered":"2023-10-18T08:58:28.299240+0000","last_clean":"2023-10-18T08:58:28.299240+0000","last_became_active":"2023-10-18T08:58:28.297217+0000","last_became_peered":"2023-10-18T08:58:28.297217+0000","last_unstale":"2023-10-18T08:58:28.299240+0000","last_undegraded":"2023-10-18T08:58:28.299240+0000","last_fullsized":"2023-10-18T08:58:28.299240+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:37.673296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313511+0000","last_change":"2023-10-18T08:58:29.313231+0000","last_active":"2023-10-18T08:58:29.313511+0000","last_peered":"2023-10-18T08:58:29.313511+0000","last_clean":"2023-10-18T08:58:29.313511+0000","last_became_active":"2023-10-18T08:58:29.311497+0000","last_became_peered":"2023-10-18T08:58:29.311497+0000","last_unstale":"2023-10-18T08:58:29.313511+0000","last_undegraded":"2023-10-18T08:58:29.313511+0000","last_fullsized":"2023-10-18T08:58:29.313511+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:54.275856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.298061+0000","last_change":"2023-10-18T08:58:28.297688+0000","last_active":"2023-10-18T08:58:28.298061+0000","last_peered":"2023-10-18T08:58:28.298061+0000","last_clean":"2023-10-18T08:58:28.298061+0000","last_became_active":"2023-10-18T08:58:28.297401+0000","last_became_peered":"2023-10-18T08:58:28.297401+0000","last_unstale":"2023-10-18T08:58:28.298061+0000","last_undegraded":"2023-10-18T08:58:28.298061+0000","last_fullsized":"2023-10-18T08:58:28.298061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:44.607015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.292556+0000","last_change":"2023-10-18T08:58:28.292556+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.292556+0000","last_undegraded":"2023-10-18T08:58:28.292556+0000","last_fullsized":"2023-10-18T08:58:28.292556+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"66'2","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341374+0000","last_change":"2023-10-18T08:58:28.299131+0000","last_active":"2023-10-18T08:58:29.341374+0000","last_peered":"2023-10-18T08:58:29.341374+0000","last_clean":"2023-10-18T08:58:29.341374+0000","last_became_active":"2023-10-18T08:58:28.298514+0000","last_became_peered":"2023-10-18T08:58:28.298514+0000","last_unstale":"2023-10-18T08:58:29.341374+0000","last_undegraded":"2023-10-18T08:58:29.341374+0000","last_fullsized":"2023-10-18T08:58:29.341374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:42:24.333152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.294168+0000","last_change":"2023-10-18T08:58:28.294168+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.294168+0000","last_undegraded":"2023-10-18T08:58:28.294168+0000","last_fullsized":"2023-10-18T08:58:28.294168+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.295059+0000","last_change":"2023-10-18T08:58:28.294327+0000","last_active":"2023-10-18T08:58:28.295059+0000","last_peered":"2023-10-18T08:58:28.295059+0000","last_clean":"2023-10-18T08:58:28.295059+0000","last_became_active":"2023-10-18T08:58:28.293852+0000","last_became_peered":"2023-10-18T08:58:28.293852+0000","last_unstale":"2023-10-18T08:58:28.295059+0000","last_undegraded":"2023-10-18T08:58:28.295059+0000","last_fullsized":"2023-10-18T08:58:28.295059+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:17:54.342972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289360+0000","last_change":"2023-10-18T08:58:28.289360+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289360+0000","last_undegraded":"2023-10-18T08:58:28.289360+0000","last_fullsized":"2023-10-18T08:58:28.289360+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.302241+0000","last_change":"2023-10-18T08:58:28.302115+0000","last_active":"2023-10-18T08:58:28.302241+0000","last_peered":"2023-10-18T08:58:28.302241+0000","last_clean":"2023-10-18T08:58:28.302241+0000","last_became_active":"2023-10-18T08:58:28.300920+0000","last_became_peered":"2023-10-18T08:58:28.300920+0000","last_unstale":"2023-10-18T08:58:28.302241+0000","last_undegraded":"2023-10-18T08:58:28.302241+0000","last_fullsized":"2023-10-18T08:58:28.302241+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:26:14.774552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289377+0000","last_change":"2023-10-18T08:58:28.289377+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289377+0000","last_undegraded":"2023-10-18T08:58:28.289377+0000","last_fullsized":"2023-10-18T08:58:28.289377+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.280714+0000","last_change":"2023-10-18T08:58:28.280587+0000","last_active":"2023-10-18T08:58:28.280714+0000","last_peered":"2023-10-18T08:58:28.280714+0000","last_clean":"2023-10-18T08:58:28.280714+0000","last_became_active":"2023-10-18T08:58:28.280261+0000","last_became_peered":"2023-10-18T08:58:28.280261+0000","last_unstale":"2023-10-18T08:58:28.280714+0000","last_undegraded":"2023-10-18T08:58:28.280714+0000","last_fullsized":"2023-10-18T08:58:28.280714+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:56.134357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289491+0000","last_change":"2023-10-18T08:58:28.289491+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289491+0000","last_undegraded":"2023-10-18T08:58:28.289491+0000","last_fullsized":"2023-10-18T08:58:28.289491+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.299921+0000","last_change":"2023-10-18T08:58:28.299846+0000","last_active":"2023-10-18T08:58:28.299921+0000","last_peered":"2023-10-18T08:58:28.299921+0000","last_clean":"2023-10-18T08:58:28.299921+0000","last_became_active":"2023-10-18T08:58:28.299121+0000","last_became_peered":"2023-10-18T08:58:28.299121+0000","last_unstale":"2023-10-18T08:58:28.299921+0000","last_undegraded":"2023-10-18T08:58:28.299921+0000","last_fullsized":"2023-10-18T08:58:28.299921+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:23.982914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.277537+0000","last_change":"2023-10-18T08:58:28.277537+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.277537+0000","last_undegraded":"2023-10-18T08:58:28.277537+0000","last_fullsized":"2023-10-18T08:58:28.277537+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300837+0000","last_change":"2023-10-18T08:58:28.300674+0000","last_active":"2023-10-18T08:58:28.300837+0000","last_peered":"2023-10-18T08:58:28.300837+0000","last_clean":"2023-10-18T08:58:28.300837+0000","last_became_active":"2023-10-18T08:58:28.300119+0000","last_became_peered":"2023-10-18T08:58:28.300119+0000","last_unstale":"2023-10-18T08:58:28.300837+0000","last_undegraded":"2023-10-18T08:58:28.300837+0000","last_fullsized":"2023-10-18T08:58:28.300837+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:35.906463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.284498+0000","last_change":"2023-10-18T08:58:28.284498+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.284498+0000","last_undegraded":"2023-10-18T08:58:28.284498+0000","last_fullsized":"2023-10-18T08:58:28.284498+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.303285+0000","last_change":"2023-10-18T08:58:28.303209+0000","last_active":"2023-10-18T08:58:28.303285+0000","last_peered":"2023-10-18T08:58:28.303285+0000","last_clean":"2023-10-18T08:58:28.303285+0000","last_became_active":"2023-10-18T08:58:28.302960+0000","last_became_peered":"2023-10-18T08:58:28.302960+0000","last_unstale":"2023-10-18T08:58:28.303285+0000","last_undegraded":"2023-10-18T08:58:28.303285+0000","last_fullsized":"2023-10-18T08:58:28.303285+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:50.826443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.316386+0000","last_change":"2023-10-18T08:58:29.315776+0000","last_active":"2023-10-18T08:58:29.316386+0000","last_peered":"2023-10-18T08:58:29.316386+0000","last_clean":"2023-10-18T08:58:29.316386+0000","last_became_active":"2023-10-18T08:58:29.315154+0000","last_became_peered":"2023-10-18T08:58:29.315154+0000","last_unstale":"2023-10-18T08:58:29.316386+0000","last_undegraded":"2023-10-18T08:58:29.316386+0000","last_fullsized":"2023-10-18T08:58:29.316386+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:51.344108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.296149+0000","last_change":"2023-10-18T08:58:28.295800+0000","last_active":"2023-10-18T08:58:28.296149+0000","last_peered":"2023-10-18T08:58:28.296149+0000","last_clean":"2023-10-18T08:58:28.296149+0000","last_became_active":"2023-10-18T08:58:28.295520+0000","last_became_peered":"2023-10-18T08:58:28.295520+0000","last_unstale":"2023-10-18T08:58:28.296149+0000","last_undegraded":"2023-10-18T08:58:28.296149+0000","last_fullsized":"2023-10-18T08:58:28.296149+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:01.809009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313850+0000","last_change":"2023-10-18T08:58:29.313510+0000","last_active":"2023-10-18T08:58:29.313850+0000","last_peered":"2023-10-18T08:58:29.313850+0000","last_clean":"2023-10-18T08:58:29.313850+0000","last_became_active":"2023-10-18T08:58:29.312697+0000","last_became_peered":"2023-10-18T08:58:29.312697+0000","last_unstale":"2023-10-18T08:58:29.313850+0000","last_undegraded":"2023-10-18T08:58:29.313850+0000","last_fullsized":"2023-10-18T08:58:29.313850+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:47.848978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300768+0000","last_change":"2023-10-18T08:58:28.300006+0000","last_active":"2023-10-18T08:58:28.300768+0000","last_peered":"2023-10-18T08:58:28.300768+0000","last_clean":"2023-10-18T08:58:28.300768+0000","last_became_active":"2023-10-18T08:58:28.299510+0000","last_became_peered":"2023-10-18T08:58:28.299510+0000","last_unstale":"2023-10-18T08:58:28.300768+0000","last_undegraded":"2023-10-18T08:58:28.300768+0000","last_fullsized":"2023-10-18T08:58:28.300768+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:01.392655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.276744+0000","last_change":"2023-10-18T08:58:28.276744+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.276744+0000","last_undegraded":"2023-10-18T08:58:28.276744+0000","last_fullsized":"2023-10-18T08:58:28.276744+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.298217+0000","last_change":"2023-10-18T08:58:28.297980+0000","last_active":"2023-10-18T08:58:28.298217+0000","last_peered":"2023-10-18T08:58:28.298217+0000","last_clean":"2023-10-18T08:58:28.298217+0000","last_became_active":"2023-10-18T08:58:28.297743+0000","last_became_peered":"2023-10-18T08:58:28.297743+0000","last_unstale":"2023-10-18T08:58:28.298217+0000","last_undegraded":"2023-10-18T08:58:28.298217+0000","last_fullsized":"2023-10-18T08:58:28.298217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:01.186029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.318289+0000","last_change":"2023-10-18T08:58:29.318110+0000","last_active":"2023-10-18T08:58:29.318289+0000","last_peered":"2023-10-18T08:58:29.318289+0000","last_clean":"2023-10-18T08:58:29.318289+0000","last_became_active":"2023-10-18T08:58:29.317066+0000","last_became_peered":"2023-10-18T08:58:29.317066+0000","last_unstale":"2023-10-18T08:58:29.318289+0000","last_undegraded":"2023-10-18T08:58:29.318289+0000","last_fullsized":"2023-10-18T08:58:29.318289+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:54.004578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.299504+0000","last_change":"2023-10-18T08:58:28.300829+0000","last_active":"2023-10-18T08:58:29.299504+0000","last_peered":"2023-10-18T08:58:29.299504+0000","last_clean":"2023-10-18T08:58:29.299504+0000","last_became_active":"2023-10-18T08:58:28.300515+0000","last_became_peered":"2023-10-18T08:58:28.300515+0000","last_unstale":"2023-10-18T08:58:29.299504+0000","last_undegraded":"2023-10-18T08:58:29.299504+0000","last_fullsized":"2023-10-18T08:58:29.299504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:15:03.965062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.331823+0000","last_change":"2023-10-18T08:58:29.331231+0000","last_active":"2023-10-18T08:58:29.331823+0000","last_peered":"2023-10-18T08:58:29.331823+0000","last_clean":"2023-10-18T08:58:29.331823+0000","last_became_active":"2023-10-18T08:58:29.331003+0000","last_became_peered":"2023-10-18T08:58:29.331003+0000","last_unstale":"2023-10-18T08:58:29.331823+0000","last_undegraded":"2023-10-18T08:58:29.331823+0000","last_fullsized":"2023-10-18T08:58:29.331823+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:48.691583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300206+0000","last_change":"2023-10-18T08:58:28.300119+0000","last_active":"2023-10-18T08:58:28.300206+0000","last_peered":"2023-10-18T08:58:28.300206+0000","last_clean":"2023-10-18T08:58:28.300206+0000","last_became_active":"2023-10-18T08:58:28.299753+0000","last_became_peered":"2023-10-18T08:58:28.299753+0000","last_unstale":"2023-10-18T08:58:28.300206+0000","last_undegraded":"2023-10-18T08:58:28.300206+0000","last_fullsized":"2023-10-18T08:58:28.300206+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:27.836895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":214,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.275442+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:58:28.275442+0000","last_peered":"2023-10-18T08:58:28.275442+0000","last_clean":"2023-10-18T08:58:28.275442+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:58:28.275442+0000","last_undegraded":"2023-10-18T08:58:28.275442+0000","last_fullsized":"2023-10-18T08:58:28.275442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:53.386532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":146,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":902,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194409,"statfs":{"total":200000000,"available":199075709,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.99299999999999999,"15min":0.98899999999999999},"min":{"1min":0.70299999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3400000000000001,"5min":1.538,"15min":1.538},"last":1.222},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.034,"15min":1.008},"min":{"1min":0.69599999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4019999999999999,"5min":1.667,"15min":1.667},"last":1.048}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1240000000000001,"15min":1.147},"min":{"1min":0.80300000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5900000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.375},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1419999999999999,"15min":1.1220000000000001},"min":{"1min":0.748,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7450000000000001,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.5920000000000001}]},{"osd":2,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.2450000000000001,"15min":1.2330000000000001},"min":{"1min":0.80200000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.284},{"interface":"front","average":{"1min":1.276,"5min":1.2010000000000001,"15min":1.1910000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.798,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.3320000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2629999999999999,"15min":1.264},"min":{"1min":0.95999999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.702,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3140000000000001,"15min":1.3149999999999999},"min":{"1min":0.94199999999999995,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.873,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.478}]},{"osd":4,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1899999999999999,"15min":1.1599999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6799999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.3779999999999999,"15min":1.401},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.6279999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.534}]},{"osd":5,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.353,"15min":1.353},"min":{"1min":0.99399999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.111},{"interface":"front","average":{"1min":1.349,"5min":1.4019999999999999,"15min":1.4159999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.865,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.4430000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.464,"15min":1.458},"min":{"1min":1.0369999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.923,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.518},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.492,"15min":1.51},"min":{"1min":1.0669999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.8069999999999999,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.633}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":326,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194986,"statfs":{"total":200000000,"available":199666077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1020000000000001,"15min":1.0800000000000001},"min":{"1min":0.623,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.651,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.228,"15min":1.234},"min":{"1min":0.84499999999999997,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.613}]},{"osd":1,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.089,"15min":1.085},"min":{"1min":0.65600000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1799999999999999,"15min":1.1859999999999999},"min":{"1min":0.80600000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.962,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":1.2829999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.161,"15min":1.173},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5029999999999999,"5min":1.696,"15min":1.696},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.1970000000000001,"15min":1.194},"min":{"1min":0.92500000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.494}]},{"osd":3,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2509999999999999,"15min":1.2569999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.669,"5min":1.784,"15min":1.784},"last":1.345},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.228,"15min":1.22},"min":{"1min":0.84999999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.526}]},{"osd":4,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.212,"15min":1.175},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.129},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.26,"15min":1.266},"min":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.696,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.4690000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3520000000000001,"15min":1.3480000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.599,"5min":1.667,"15min":1.667},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.232,"5min":1.238,"15min":1.218},"min":{"1min":0.629,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.52,"5min":1.746,"15min":1.746},"last":1.403}]},{"osd":7,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.4039999999999999,"15min":1.3959999999999999},"min":{"1min":0.73999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.51,"15min":1.5249999999999999},"min":{"1min":0.84099999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.831,"5min":2.794,"15min":2.794},"last":1.6479999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":326,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194986,"statfs":{"total":200000000,"available":199666077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.7529999999999999,"5min":1.3160000000000001,"15min":1.224},"min":{"1min":0.68300000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":9.3539999999999992,"5min":9.3539999999999992,"15min":9.3539999999999992},"last":1.258},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.3220000000000001,"15min":1.3029999999999999},"min":{"1min":0.71999999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":5.3029999999999999,"5min":5.3029999999999999,"15min":5.3029999999999999},"last":1.4379999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2090000000000001,"15min":1.2230000000000001},"min":{"1min":0.755,"5min":0.749,"15min":0.749},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.222,"15min":1.248},"min":{"1min":0.82199999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.367,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":1.1739999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.268,"15min":1.268},"min":{"1min":0.78100000000000003,"5min":0.755,"15min":0.755},"max":{"1min":1.5309999999999999,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.292,"5min":1.2889999999999999,"15min":1.2929999999999999},"min":{"1min":0.78400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.3069999999999999,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.0549999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.25,"15min":1.246},"min":{"1min":0.93300000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.524,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.337,"15min":1.3320000000000001},"min":{"1min":0.94399999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.6379999999999999,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.999}]},{"osd":4,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4139999999999999,"15min":1.419},"min":{"1min":0.93400000000000005,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.881,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.4950000000000001,"15min":1.5329999999999999},"min":{"1min":0.90000000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":4.4429999999999996,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":1.349}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.377,"15min":1.4119999999999999},"min":{"1min":0.93899999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.5469999999999997,"5min":5.141,"15min":5.141},"last":1.202},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.556,"15min":1.587},"min":{"1min":1.151,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0590000000000002,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.117}]},{"osd":7,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.6419999999999999,"15min":1.6699999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":4.2050000000000001,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5149999999999999,"15min":1.5249999999999999},"min":{"1min":1.0249999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":4.4690000000000003,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":1.512}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":325,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194987,"statfs":{"total":200000000,"available":199667096,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.107,"15min":1.1100000000000001},"min":{"1min":0.745,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":0.996,"5min":1.129,"15min":1.169},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.6990000000000001,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.1439999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.4849999999999999,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":1.008},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2250000000000001,"15min":1.2390000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.651}]},{"osd":2,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1879999999999999,"15min":1.179},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.254},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.27,"15min":1.3029999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.994,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.3819999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2889999999999999,"15min":1.3009999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":1.286},{"interface":"front","average":{"1min":1.421,"5min":1.4179999999999999,"15min":1.4139999999999999},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.6880000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.1910000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.323,"15min":1.304},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.778,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.5},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.351,"15min":1.367},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.228}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.4339999999999999,"15min":1.3939999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.556,"5min":1.5629999999999999,"15min":1.5760000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":1.5549999999999999}]},{"osd":7,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.4950000000000001,"15min":1.514},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5529999999999999,"15min":1.5509999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.585,"5min":2.585,"15min":2.585},"last":1.4530000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":326,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194986,"statfs":{"total":200000000,"available":199666077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.85199999999999998,"15min":0.77400000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.304,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.95199999999999996,"15min":0.873},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.393,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":0.46600000000000003}]},{"osd":1,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.97799999999999998,"15min":0.94199999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.208,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.1419999999999999,"15min":1.113},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.5660000000000001,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.258}]},{"osd":2,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.0549999999999999,"15min":1.0149999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4330000000000001,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1080000000000001,"15min":1.054},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6459999999999999,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":1.3149999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.464,"15min":1.3049999999999999},"min":{"1min":0.83599999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.5649999999999999,"5min":18.664999999999999,"15min":18.664999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.21,"15min":1.1499999999999999},"min":{"1min":0.69099999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.756,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":1.0149999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.276,"15min":1.2170000000000001},"min":{"1min":0.72399999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.702,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":0.87},{"interface":"front","average":{"1min":1.425,"5min":1.502,"15min":1.3400000000000001},"min":{"1min":0.80800000000000005,"5min":0.748,"15min":0.748},"max":{"1min":1.885,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.375}]},{"osd":6,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4750000000000001,"15min":1.4199999999999999},"min":{"1min":0.82499999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.7649999999999999,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3959999999999999,"15min":1.3300000000000001},"min":{"1min":0.77200000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.774,"5min":7.242,"15min":7.242},"last":1.103}]},{"osd":7,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.4019999999999999,"15min":1.3320000000000001},"min":{"1min":0.80600000000000005,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.9970000000000001,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.431},{"interface":"front","average":{"1min":1.47,"5min":1.556,"15min":1.4470000000000001},"min":{"1min":0.93100000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.8580000000000001,"5min":12.946,"15min":12.946},"last":0.84199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":319,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194992,"statfs":{"total":200000000,"available":199672344,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90500000000000003,"15min":0.93500000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.55,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0029999999999999,"15min":1.05},"min":{"1min":0.63100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4870000000000001,"5min":1.651,"15min":1.651},"last":0.75900000000000001}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.0249999999999999,"15min":1.0549999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.427,"5min":4.29,"15min":4.29},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.198,"5min":1.0760000000000001,"15min":1.095},"min":{"1min":0.65700000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.583,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.8140000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0840000000000001,"15min":1.1259999999999999},"min":{"1min":0.54600000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.6419999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.256},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.391,"15min":1.48},"min":{"1min":0.57899999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8360000000000001,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":1.7709999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.448,"15min":1.4950000000000001},"min":{"1min":0.95699999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.7450000000000001,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":1.639},{"interface":"front","average":{"1min":1.421,"5min":1.415,"15min":1.45},"min":{"1min":0.93799999999999994,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.8029999999999999,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.5940000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.234,"15min":1.268},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6719999999999999,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":1.054},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.5,"15min":1.5760000000000001},"min":{"1min":0.96099999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.863,"5min":7.657,"15min":7.657},"last":1.2989999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.472,"15min":1.5309999999999999},"min":{"1min":0.82699999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8999999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.391},{"interface":"front","average":{"1min":3.536,"5min":1.8240000000000001,"15min":1.5720000000000001},"min":{"1min":0.98899999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":37.771999999999998,"5min":37.771999999999998,"15min":37.771999999999998},"last":1.718}]},{"osd":7,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.5309999999999999,"15min":1.573},"min":{"1min":0.98699999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.016,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.4410000000000001,"15min":1.4710000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1349999999999998,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.5269999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":325,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194987,"statfs":{"total":200000000,"available":199667096,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.79200000000000004,"15min":0.79800000000000004},"min":{"1min":0.44500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":1.327,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.999,"5min":0.89300000000000002,"15min":0.89100000000000001},"min":{"1min":0.501,"5min":0.443,"15min":0.443},"max":{"1min":1.5129999999999999,"5min":1.55,"15min":1.55},"last":0.96099999999999997}]},{"osd":2,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.91100000000000003,"15min":0.92300000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6850000000000001,"5min":1.859,"15min":1.859},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.83299999999999996,"15min":0.83499999999999996},"min":{"1min":0.47999999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.625,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.81000000000000005}]},{"osd":3,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.98199999999999998,"15min":1.008},"min":{"1min":0.52800000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.571,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.0229999999999999,"15min":1.0469999999999999},"min":{"1min":0.54500000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.8859999999999999,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":0.89000000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.22,"15min":1.2350000000000001},"min":{"1min":0.82099999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5760000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.216},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1970000000000001,"15min":1.1930000000000001},"min":{"1min":0.93799999999999994,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.556,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.6759999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.268,"15min":1.2809999999999999},"min":{"1min":0.86599999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.625,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.367},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.343,"15min":1.343},"min":{"1min":0.90500000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.6759999999999999,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.486}]},{"osd":6,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2529999999999999,"15min":1.2649999999999999},"min":{"1min":0.89100000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6240000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2450000000000001,"15min":1.248},"min":{"1min":0.78900000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.744,"5min":1.853,"15min":1.853},"last":1.1160000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.333,"15min":1.339},"min":{"1min":0.84399999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.716,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.288},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3380000000000001,"15min":1.3400000000000001},"min":{"1min":0.95099999999999996,"5min":0.746,"15min":0.746},"max":{"1min":1.758,"5min":1.994,"15min":1.994},"last":1.4179999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":895,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194416,"statfs":{"total":200000000,"available":199082995,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.89500000000000002,"15min":0.93600000000000005},"min":{"1min":0.371,"5min":0.37,"15min":0.37},"max":{"1min":1.0940000000000001,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":1.163},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0660000000000001,"15min":1.079},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.90300000000000002}]},{"osd":2,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.121,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.873},{"interface":"front","average":{"1min":1.121,"5min":1.1240000000000001,"15min":1.125},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.8109999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.0549999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.22,"15min":1.306},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6639999999999999,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.377,"5min":1.4750000000000001,"15min":1.5309999999999999},"min":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.9259999999999999,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.1180000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2629999999999999,"15min":1.3169999999999999},"min":{"1min":0.81200000000000006,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.5549999999999999,"5min":4.492,"15min":4.492},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.234,"5min":1.2490000000000001,"15min":1.2470000000000001},"min":{"1min":0.93200000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.629,"5min":1.726,"15min":1.726},"last":1.1459999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.377,"15min":1.4490000000000001},"min":{"1min":0.89500000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.944,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3580000000000001,"15min":1.3859999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8240000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.0900000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.5629999999999999,"15min":1.6100000000000001},"min":{"1min":0.877,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.0409999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.4670000000000001,"15min":1.5189999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.863,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.427}]},{"osd":7,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.619,"15min":1.661},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.1160000000000001,"5min":3.492,"15min":3.492},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.621,"15min":1.7130000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.0449999999999999,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.516}]}]}],"pool_statfs":[]}} 2023-10-18T08:58:33.807 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:58:34.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.078+0000 7f83efd68640 1 -- 172.21.15.40:0/2445556821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d40047c0 msgr2=0x7f83d4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:34.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.078+0000 7f83efd68640 1 --2- 172.21.15.40:0/2445556821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d40047c0 0x7f83d4004c40 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f83d8001680 tx=0x7f83d40057f0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.078+0000 7f83efd68640 1 -- 172.21.15.40:0/2445556821 shutdown_connections 2023-10-18T08:58:34.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.078+0000 7f83efd68640 1 --2- 172.21.15.40:0/2445556821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d40047c0 0x7f83d4004c40 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.078+0000 7f83efd68640 1 -- 172.21.15.40:0/2445556821 >> 172.21.15.40:0/2445556821 conn(0x7f83e80cd410 msgr2=0x7f83e80cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:34.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.078+0000 7f83efd68640 1 -- 172.21.15.40:0/2445556821 shutdown_connections 2023-10-18T08:58:34.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.078+0000 7f83efd68640 1 -- 172.21.15.40:0/2445556821 wait complete. 2023-10-18T08:58:34.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.080+0000 7f83efd68640 1 Processor -- start 2023-10-18T08:58:34.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.080+0000 7f83efd68640 1 -- start start 2023-10-18T08:58:34.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.081+0000 7f83efd68640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d40047c0 0x7f83e8096510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:34.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.081+0000 7f83efd68640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83e80c68d0 0x7f83e8151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:34.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.081+0000 7f83efd68640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83e8151c80 0x7f83e80b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:34.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.081+0000 7f83efd68640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e8152110 con 0x7f83d40047c0 2023-10-18T08:58:34.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.081+0000 7f83efd68640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e80ad430 con 0x7f83e80c68d0 2023-10-18T08:58:34.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.081+0000 7f83efd68640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e80c0f40 con 0x7f83e8151c80 2023-10-18T08:58:34.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.081+0000 7f83edb04640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d40047c0 0x7f83e8096510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:34.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.081+0000 7f83edb04640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d40047c0 0x7f83e8096510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38532/0 (socket says 172.21.15.40:38532) 2023-10-18T08:58:34.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.081+0000 7f83edb04640 1 -- 172.21.15.40:0/1284661692 learned_addr learned my addr 172.21.15.40:0/1284661692 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:34.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.082+0000 7f83ee305640 1 --2- 172.21.15.40:0/1284661692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83e8151c80 0x7f83e80b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:34.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.082+0000 7f83ed303640 1 --2- 172.21.15.40:0/1284661692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83e80c68d0 0x7f83e8151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:34.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.082+0000 7f83edb04640 1 -- 172.21.15.40:0/1284661692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83e8151c80 msgr2=0x7f83e80b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:34.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.082+0000 7f83edb04640 1 --2- 172.21.15.40:0/1284661692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83e8151c80 0x7f83e80b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.082+0000 7f83edb04640 1 -- 172.21.15.40:0/1284661692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83e80c68d0 msgr2=0x7f83e8151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:34.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.082+0000 7f83edb04640 1 --2- 172.21.15.40:0/1284661692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83e80c68d0 0x7f83e8151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.082+0000 7f83edb04640 1 -- 172.21.15.40:0/1284661692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e80c17e0 con 0x7f83d40047c0 2023-10-18T08:58:34.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.082+0000 7f83ee305640 1 --2- 172.21.15.40:0/1284661692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83e8151c80 0x7f83e80b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:34.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.082+0000 7f83ed303640 1 --2- 172.21.15.40:0/1284661692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83e80c68d0 0x7f83e8151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:34.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.082+0000 7f83edb04640 1 --2- 172.21.15.40:0/1284661692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d40047c0 0x7f83e8096510 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f83e400d730 tx=0x7f83e400dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:34.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.083+0000 7f83de7fc640 1 -- 172.21.15.40:0/1284661692 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e400de40 con 0x7f83d40047c0 2023-10-18T08:58:34.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.083+0000 7f83de7fc640 1 -- 172.21.15.40:0/1284661692 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f83e40052e0 con 0x7f83d40047c0 2023-10-18T08:58:34.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.083+0000 7f83de7fc640 1 -- 172.21.15.40:0/1284661692 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e400b920 con 0x7f83d40047c0 2023-10-18T08:58:34.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.083+0000 7f83efd68640 1 -- 172.21.15.40:0/1284661692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e80b0260 con 0x7f83d40047c0 2023-10-18T08:58:34.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.083+0000 7f83efd68640 1 -- 172.21.15.40:0/1284661692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e80ad150 con 0x7f83d40047c0 2023-10-18T08:58:34.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.084+0000 7f83efd68640 1 -- 172.21.15.40:0/1284661692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e80cc6c0 con 0x7f83d40047c0 2023-10-18T08:58:34.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.085+0000 7f83de7fc640 1 -- 172.21.15.40:0/1284661692 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f83e400ba80 con 0x7f83d40047c0 2023-10-18T08:58:34.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.086+0000 7f83de7fc640 1 --2- 172.21.15.40:0/1284661692 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83cc06ae70 0x7f83cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:34.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.086+0000 7f83de7fc640 1 -- 172.21.15.40:0/1284661692 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83e4088da0 con 0x7f83d40047c0 2023-10-18T08:58:34.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.087+0000 7f83ed303640 1 --2- 172.21.15.40:0/1284661692 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83cc06ae70 0x7f83cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:34.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.088+0000 7f83ed303640 1 --2- 172.21.15.40:0/1284661692 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83cc06ae70 0x7f83cc06d330 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f83d8034000 tx=0x7f83d8039040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:34.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.091+0000 7f83de7fc640 1 -- 172.21.15.40:0/1284661692 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f83e4057840 con 0x7f83d40047c0 2023-10-18T08:58:34.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.284+0000 7f83efd68640 1 -- 172.21.15.40:0/1284661692 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f83e815bcd0 con 0x7f83cc06ae70 2023-10-18T08:58:34.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.307+0000 7f83de7fc640 1 -- 172.21.15.40:0/1284661692 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279218 (secure 0 0 0) 0x7f83d80035f0 con 0x7f83cc06ae70 2023-10-18T08:58:34.313 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:34.314 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:58:34.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.313+0000 7f83efd68640 1 -- 172.21.15.40:0/1284661692 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83cc06ae70 msgr2=0x7f83cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:34.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.313+0000 7f83efd68640 1 --2- 172.21.15.40:0/1284661692 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83cc06ae70 0x7f83cc06d330 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f83d8034000 tx=0x7f83d8039040 comp rx=0 tx=0).stop 2023-10-18T08:58:34.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.313+0000 7f83efd68640 1 -- 172.21.15.40:0/1284661692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d40047c0 msgr2=0x7f83e8096510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.313+0000 7f83efd68640 1 --2- 172.21.15.40:0/1284661692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d40047c0 0x7f83e8096510 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f83e400d730 tx=0x7f83e400dbf0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.313+0000 7f83efd68640 1 -- 172.21.15.40:0/1284661692 shutdown_connections 2023-10-18T08:58:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.313+0000 7f83efd68640 1 --2- 172.21.15.40:0/1284661692 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83cc06ae70 0x7f83cc06d330 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.313+0000 7f83efd68640 1 --2- 172.21.15.40:0/1284661692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83e8151c80 0x7f83e80b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.313+0000 7f83efd68640 1 --2- 172.21.15.40:0/1284661692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83e80c68d0 0x7f83e8151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.313+0000 7f83efd68640 1 --2- 172.21.15.40:0/1284661692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83d40047c0 0x7f83e8096510 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.313+0000 7f83efd68640 1 -- 172.21.15.40:0/1284661692 >> 172.21.15.40:0/1284661692 conn(0x7f83e80cd410 msgr2=0x7f83e80c5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.313+0000 7f83efd68640 1 -- 172.21.15.40:0/1284661692 shutdown_connections 2023-10-18T08:58:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.314+0000 7f83efd68640 1 -- 172.21.15.40:0/1284661692 wait complete. 2023-10-18T08:58:34.336 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2023-10-18T08:58:32.964940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590514,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3746,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558753,"statfs":{"total":1600000000,"available":1596163471,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":146,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.697568"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.290178+0000","last_change":"2023-10-18T08:58:28.290178+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.290178+0000","last_undegraded":"2023-10-18T08:58:28.290178+0000","last_fullsized":"2023-10-18T08:58:28.290178+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.310859+0000","last_change":"2023-10-18T08:58:29.310661+0000","last_active":"2023-10-18T08:58:29.310859+0000","last_peered":"2023-10-18T08:58:29.310859+0000","last_clean":"2023-10-18T08:58:29.310859+0000","last_became_active":"2023-10-18T08:58:29.309557+0000","last_became_peered":"2023-10-18T08:58:29.309557+0000","last_unstale":"2023-10-18T08:58:29.310859+0000","last_undegraded":"2023-10-18T08:58:29.310859+0000","last_fullsized":"2023-10-18T08:58:29.310859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:46.399620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.323795+0000","last_change":"2023-10-18T08:58:29.321404+0000","last_active":"2023-10-18T08:58:29.323795+0000","last_peered":"2023-10-18T08:58:29.323795+0000","last_clean":"2023-10-18T08:58:29.323795+0000","last_became_active":"2023-10-18T08:58:29.320985+0000","last_became_peered":"2023-10-18T08:58:29.320985+0000","last_unstale":"2023-10-18T08:58:29.323795+0000","last_undegraded":"2023-10-18T08:58:29.323795+0000","last_fullsized":"2023-10-18T08:58:29.323795+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:10:04.504829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.293989+0000","last_change":"2023-10-18T08:58:28.293989+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.293989+0000","last_undegraded":"2023-10-18T08:58:28.293989+0000","last_fullsized":"2023-10-18T08:58:28.293989+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.291475+0000","last_change":"2023-10-18T08:58:28.291475+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.291475+0000","last_undegraded":"2023-10-18T08:58:28.291475+0000","last_fullsized":"2023-10-18T08:58:28.291475+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.287056+0000","last_change":"2023-10-18T08:58:28.287056+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.287056+0000","last_undegraded":"2023-10-18T08:58:28.287056+0000","last_fullsized":"2023-10-18T08:58:28.287056+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.290553+0000","last_change":"2023-10-18T08:58:28.290553+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.290553+0000","last_undegraded":"2023-10-18T08:58:28.290553+0000","last_fullsized":"2023-10-18T08:58:28.290553+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326822+0000","last_change":"2023-10-18T08:58:29.325559+0000","last_active":"2023-10-18T08:58:29.326822+0000","last_peered":"2023-10-18T08:58:29.326822+0000","last_clean":"2023-10-18T08:58:29.326822+0000","last_became_active":"2023-10-18T08:58:29.324359+0000","last_became_peered":"2023-10-18T08:58:29.324359+0000","last_unstale":"2023-10-18T08:58:29.326822+0000","last_undegraded":"2023-10-18T08:58:29.326822+0000","last_fullsized":"2023-10-18T08:58:29.326822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:45.688349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.294163+0000","last_change":"2023-10-18T08:58:28.294163+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.294163+0000","last_undegraded":"2023-10-18T08:58:28.294163+0000","last_fullsized":"2023-10-18T08:58:28.294163+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.278822+0000","last_change":"2023-10-18T08:58:28.278822+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.278822+0000","last_undegraded":"2023-10-18T08:58:28.278822+0000","last_fullsized":"2023-10-18T08:58:28.278822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.277474+0000","last_change":"2023-10-18T08:58:28.277474+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.277474+0000","last_undegraded":"2023-10-18T08:58:28.277474+0000","last_fullsized":"2023-10-18T08:58:28.277474+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.290401+0000","last_change":"2023-10-18T08:58:28.290401+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.290401+0000","last_undegraded":"2023-10-18T08:58:28.290401+0000","last_fullsized":"2023-10-18T08:58:28.290401+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.283202+0000","last_change":"2023-10-18T08:58:28.283202+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.283202+0000","last_undegraded":"2023-10-18T08:58:28.283202+0000","last_fullsized":"2023-10-18T08:58:28.283202+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.293634+0000","last_change":"2023-10-18T08:58:28.293634+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.293634+0000","last_undegraded":"2023-10-18T08:58:28.293634+0000","last_fullsized":"2023-10-18T08:58:28.293634+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289054+0000","last_change":"2023-10-18T08:58:28.289054+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289054+0000","last_undegraded":"2023-10-18T08:58:28.289054+0000","last_fullsized":"2023-10-18T08:58:28.289054+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.285983+0000","last_change":"2023-10-18T08:58:28.285983+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.285983+0000","last_undegraded":"2023-10-18T08:58:28.285983+0000","last_fullsized":"2023-10-18T08:58:28.285983+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326448+0000","last_change":"2023-10-18T08:58:29.325090+0000","last_active":"2023-10-18T08:58:29.326448+0000","last_peered":"2023-10-18T08:58:29.326448+0000","last_clean":"2023-10-18T08:58:29.326448+0000","last_became_active":"2023-10-18T08:58:29.323675+0000","last_became_peered":"2023-10-18T08:58:29.323675+0000","last_unstale":"2023-10-18T08:58:29.326448+0000","last_undegraded":"2023-10-18T08:58:29.326448+0000","last_fullsized":"2023-10-18T08:58:29.326448+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:01:55.373464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.329948+0000","last_change":"2023-10-18T08:58:29.329020+0000","last_active":"2023-10-18T08:58:29.329948+0000","last_peered":"2023-10-18T08:58:29.329948+0000","last_clean":"2023-10-18T08:58:29.329948+0000","last_became_active":"2023-10-18T08:58:29.328773+0000","last_became_peered":"2023-10-18T08:58:29.328773+0000","last_unstale":"2023-10-18T08:58:29.329948+0000","last_undegraded":"2023-10-18T08:58:29.329948+0000","last_fullsized":"2023-10-18T08:58:29.329948+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:57.274523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289622+0000","last_change":"2023-10-18T08:58:28.289622+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289622+0000","last_undegraded":"2023-10-18T08:58:28.289622+0000","last_fullsized":"2023-10-18T08:58:28.289622+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.330756+0000","last_change":"2023-10-18T08:58:29.330497+0000","last_active":"2023-10-18T08:58:29.330756+0000","last_peered":"2023-10-18T08:58:29.330756+0000","last_clean":"2023-10-18T08:58:29.330756+0000","last_became_active":"2023-10-18T08:58:29.330245+0000","last_became_peered":"2023-10-18T08:58:29.330245+0000","last_unstale":"2023-10-18T08:58:29.330756+0000","last_undegraded":"2023-10-18T08:58:29.330756+0000","last_fullsized":"2023-10-18T08:58:29.330756+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:23.135334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.323823+0000","last_change":"2023-10-18T08:58:29.323242+0000","last_active":"2023-10-18T08:58:29.323823+0000","last_peered":"2023-10-18T08:58:29.323823+0000","last_clean":"2023-10-18T08:58:29.323823+0000","last_became_active":"2023-10-18T08:58:29.322832+0000","last_became_peered":"2023-10-18T08:58:29.322832+0000","last_unstale":"2023-10-18T08:58:29.323823+0000","last_undegraded":"2023-10-18T08:58:29.323823+0000","last_fullsized":"2023-10-18T08:58:29.323823+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:10:00.029939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326366+0000","last_change":"2023-10-18T08:58:29.319011+0000","last_active":"2023-10-18T08:58:29.326366+0000","last_peered":"2023-10-18T08:58:29.326366+0000","last_clean":"2023-10-18T08:58:29.326366+0000","last_became_active":"2023-10-18T08:58:29.318536+0000","last_became_peered":"2023-10-18T08:58:29.318536+0000","last_unstale":"2023-10-18T08:58:29.326366+0000","last_undegraded":"2023-10-18T08:58:29.326366+0000","last_fullsized":"2023-10-18T08:58:29.326366+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:16.123758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.319916+0000","last_change":"2023-10-18T08:58:29.319576+0000","last_active":"2023-10-18T08:58:29.319916+0000","last_peered":"2023-10-18T08:58:29.319916+0000","last_clean":"2023-10-18T08:58:29.319916+0000","last_became_active":"2023-10-18T08:58:29.316842+0000","last_became_peered":"2023-10-18T08:58:29.316842+0000","last_unstale":"2023-10-18T08:58:29.319916+0000","last_undegraded":"2023-10-18T08:58:29.319916+0000","last_fullsized":"2023-10-18T08:58:29.319916+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:45.620650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.281482+0000","last_change":"2023-10-18T08:58:28.281482+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.281482+0000","last_undegraded":"2023-10-18T08:58:28.281482+0000","last_fullsized":"2023-10-18T08:58:28.281482+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.291034+0000","last_change":"2023-10-18T08:58:28.291034+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.291034+0000","last_undegraded":"2023-10-18T08:58:28.291034+0000","last_fullsized":"2023-10-18T08:58:28.291034+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.291241+0000","last_change":"2023-10-18T08:58:28.291241+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.291241+0000","last_undegraded":"2023-10-18T08:58:28.291241+0000","last_fullsized":"2023-10-18T08:58:28.291241+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.285161+0000","last_change":"2023-10-18T08:58:28.285161+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.285161+0000","last_undegraded":"2023-10-18T08:58:28.285161+0000","last_fullsized":"2023-10-18T08:58:28.285161+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289003+0000","last_change":"2023-10-18T08:58:28.289003+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289003+0000","last_undegraded":"2023-10-18T08:58:28.289003+0000","last_fullsized":"2023-10-18T08:58:28.289003+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.329742+0000","last_change":"2023-10-18T08:58:29.329658+0000","last_active":"2023-10-18T08:58:29.329742+0000","last_peered":"2023-10-18T08:58:29.329742+0000","last_clean":"2023-10-18T08:58:29.329742+0000","last_became_active":"2023-10-18T08:58:29.329393+0000","last_became_peered":"2023-10-18T08:58:29.329393+0000","last_unstale":"2023-10-18T08:58:29.329742+0000","last_undegraded":"2023-10-18T08:58:29.329742+0000","last_fullsized":"2023-10-18T08:58:29.329742+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:28:25.454956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326242+0000","last_change":"2023-10-18T08:58:29.326075+0000","last_active":"2023-10-18T08:58:29.326242+0000","last_peered":"2023-10-18T08:58:29.326242+0000","last_clean":"2023-10-18T08:58:29.326242+0000","last_became_active":"2023-10-18T08:58:29.324858+0000","last_became_peered":"2023-10-18T08:58:29.324858+0000","last_unstale":"2023-10-18T08:58:29.326242+0000","last_undegraded":"2023-10-18T08:58:29.326242+0000","last_fullsized":"2023-10-18T08:58:29.326242+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:49.015165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289440+0000","last_change":"2023-10-18T08:58:28.289440+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289440+0000","last_undegraded":"2023-10-18T08:58:28.289440+0000","last_fullsized":"2023-10-18T08:58:28.289440+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.284606+0000","last_change":"2023-10-18T08:58:28.284606+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.284606+0000","last_undegraded":"2023-10-18T08:58:28.284606+0000","last_fullsized":"2023-10-18T08:58:28.284606+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.288895+0000","last_change":"2023-10-18T08:58:28.288895+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.288895+0000","last_undegraded":"2023-10-18T08:58:28.288895+0000","last_fullsized":"2023-10-18T08:58:28.288895+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.301110+0000","last_change":"2023-10-18T08:58:28.300855+0000","last_active":"2023-10-18T08:58:28.301110+0000","last_peered":"2023-10-18T08:58:28.301110+0000","last_clean":"2023-10-18T08:58:28.301110+0000","last_became_active":"2023-10-18T08:58:28.300672+0000","last_became_peered":"2023-10-18T08:58:28.300672+0000","last_unstale":"2023-10-18T08:58:28.301110+0000","last_undegraded":"2023-10-18T08:58:28.301110+0000","last_fullsized":"2023-10-18T08:58:28.301110+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:57:22.719753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.277530+0000","last_change":"2023-10-18T08:58:28.277530+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.277530+0000","last_undegraded":"2023-10-18T08:58:28.277530+0000","last_fullsized":"2023-10-18T08:58:28.277530+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"66'1","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.342802+0000","last_change":"2023-10-18T08:58:28.295079+0000","last_active":"2023-10-18T08:58:29.342802+0000","last_peered":"2023-10-18T08:58:29.342802+0000","last_clean":"2023-10-18T08:58:29.342802+0000","last_became_active":"2023-10-18T08:58:28.294485+0000","last_became_peered":"2023-10-18T08:58:28.294485+0000","last_unstale":"2023-10-18T08:58:29.342802+0000","last_undegraded":"2023-10-18T08:58:29.342802+0000","last_fullsized":"2023-10-18T08:58:29.342802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:27.603595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.286466+0000","last_change":"2023-10-18T08:58:28.286466+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.286466+0000","last_undegraded":"2023-10-18T08:58:28.286466+0000","last_fullsized":"2023-10-18T08:58:28.286466+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.299117+0000","last_change":"2023-10-18T08:58:28.298521+0000","last_active":"2023-10-18T08:58:28.299117+0000","last_peered":"2023-10-18T08:58:28.299117+0000","last_clean":"2023-10-18T08:58:28.299117+0000","last_became_active":"2023-10-18T08:58:28.296841+0000","last_became_peered":"2023-10-18T08:58:28.296841+0000","last_unstale":"2023-10-18T08:58:28.299117+0000","last_undegraded":"2023-10-18T08:58:28.299117+0000","last_fullsized":"2023-10-18T08:58:28.299117+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:24.568573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289868+0000","last_change":"2023-10-18T08:58:28.289868+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289868+0000","last_undegraded":"2023-10-18T08:58:28.289868+0000","last_fullsized":"2023-10-18T08:58:28.289868+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.299261+0000","last_change":"2023-10-18T08:58:28.298876+0000","last_active":"2023-10-18T08:58:28.299261+0000","last_peered":"2023-10-18T08:58:28.299261+0000","last_clean":"2023-10-18T08:58:28.299261+0000","last_became_active":"2023-10-18T08:58:28.297696+0000","last_became_peered":"2023-10-18T08:58:28.297696+0000","last_unstale":"2023-10-18T08:58:28.299261+0000","last_undegraded":"2023-10-18T08:58:28.299261+0000","last_fullsized":"2023-10-18T08:58:28.299261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:07:47.505124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326799+0000","last_change":"2023-10-18T08:58:29.325300+0000","last_active":"2023-10-18T08:58:29.326799+0000","last_peered":"2023-10-18T08:58:29.326799+0000","last_clean":"2023-10-18T08:58:29.326799+0000","last_became_active":"2023-10-18T08:58:29.323662+0000","last_became_peered":"2023-10-18T08:58:29.323662+0000","last_unstale":"2023-10-18T08:58:29.326799+0000","last_undegraded":"2023-10-18T08:58:29.326799+0000","last_fullsized":"2023-10-18T08:58:29.326799+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:39:03.787381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300750+0000","last_change":"2023-10-18T08:58:28.299820+0000","last_active":"2023-10-18T08:58:28.300750+0000","last_peered":"2023-10-18T08:58:28.300750+0000","last_clean":"2023-10-18T08:58:28.300750+0000","last_became_active":"2023-10-18T08:58:28.299113+0000","last_became_peered":"2023-10-18T08:58:28.299113+0000","last_unstale":"2023-10-18T08:58:28.300750+0000","last_undegraded":"2023-10-18T08:58:28.300750+0000","last_fullsized":"2023-10-18T08:58:28.300750+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:59.489387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.332542+0000","last_change":"2023-10-18T08:58:29.332297+0000","last_active":"2023-10-18T08:58:29.332542+0000","last_peered":"2023-10-18T08:58:29.332542+0000","last_clean":"2023-10-18T08:58:29.332542+0000","last_became_active":"2023-10-18T08:58:29.332051+0000","last_became_peered":"2023-10-18T08:58:29.332051+0000","last_unstale":"2023-10-18T08:58:29.332542+0000","last_undegraded":"2023-10-18T08:58:29.332542+0000","last_fullsized":"2023-10-18T08:58:29.332542+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:30:53.581720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300168+0000","last_change":"2023-10-18T08:58:28.300048+0000","last_active":"2023-10-18T08:58:28.300168+0000","last_peered":"2023-10-18T08:58:28.300168+0000","last_clean":"2023-10-18T08:58:28.300168+0000","last_became_active":"2023-10-18T08:58:28.299438+0000","last_became_peered":"2023-10-18T08:58:28.299438+0000","last_unstale":"2023-10-18T08:58:28.300168+0000","last_undegraded":"2023-10-18T08:58:28.300168+0000","last_fullsized":"2023-10-18T08:58:28.300168+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:53.262221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.281849+0000","last_change":"2023-10-18T08:58:28.281849+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.281849+0000","last_undegraded":"2023-10-18T08:58:28.281849+0000","last_fullsized":"2023-10-18T08:58:28.281849+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.301494+0000","last_change":"2023-10-18T08:58:28.301417+0000","last_active":"2023-10-18T08:58:28.301494+0000","last_peered":"2023-10-18T08:58:28.301494+0000","last_clean":"2023-10-18T08:58:28.301494+0000","last_became_active":"2023-10-18T08:58:28.300999+0000","last_became_peered":"2023-10-18T08:58:28.300999+0000","last_unstale":"2023-10-18T08:58:28.301494+0000","last_undegraded":"2023-10-18T08:58:28.301494+0000","last_fullsized":"2023-10-18T08:58:28.301494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:30.254470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326387+0000","last_change":"2023-10-18T08:58:29.321607+0000","last_active":"2023-10-18T08:58:29.326387+0000","last_peered":"2023-10-18T08:58:29.326387+0000","last_clean":"2023-10-18T08:58:29.326387+0000","last_became_active":"2023-10-18T08:58:29.321062+0000","last_became_peered":"2023-10-18T08:58:29.321062+0000","last_unstale":"2023-10-18T08:58:29.326387+0000","last_undegraded":"2023-10-18T08:58:29.326387+0000","last_fullsized":"2023-10-18T08:58:29.326387+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:20.541018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.302195+0000","last_change":"2023-10-18T08:58:28.302109+0000","last_active":"2023-10-18T08:58:28.302195+0000","last_peered":"2023-10-18T08:58:28.302195+0000","last_clean":"2023-10-18T08:58:28.302195+0000","last_became_active":"2023-10-18T08:58:28.301703+0000","last_became_peered":"2023-10-18T08:58:28.301703+0000","last_unstale":"2023-10-18T08:58:28.302195+0000","last_undegraded":"2023-10-18T08:58:28.302195+0000","last_fullsized":"2023-10-18T08:58:28.302195+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:57.346205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289722+0000","last_change":"2023-10-18T08:58:28.289722+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289722+0000","last_undegraded":"2023-10-18T08:58:28.289722+0000","last_fullsized":"2023-10-18T08:58:28.289722+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.305138+0000","last_change":"2023-10-18T08:58:28.298815+0000","last_active":"2023-10-18T08:58:29.305138+0000","last_peered":"2023-10-18T08:58:29.305138+0000","last_clean":"2023-10-18T08:58:29.305138+0000","last_became_active":"2023-10-18T08:58:28.298503+0000","last_became_peered":"2023-10-18T08:58:28.298503+0000","last_unstale":"2023-10-18T08:58:29.305138+0000","last_undegraded":"2023-10-18T08:58:29.305138+0000","last_fullsized":"2023-10-18T08:58:29.305138+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:39.750254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313431+0000","last_change":"2023-10-18T08:58:29.312112+0000","last_active":"2023-10-18T08:58:29.313431+0000","last_peered":"2023-10-18T08:58:29.313431+0000","last_clean":"2023-10-18T08:58:29.313431+0000","last_became_active":"2023-10-18T08:58:29.310708+0000","last_became_peered":"2023-10-18T08:58:29.310708+0000","last_unstale":"2023-10-18T08:58:29.313431+0000","last_undegraded":"2023-10-18T08:58:29.313431+0000","last_fullsized":"2023-10-18T08:58:29.313431+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:24.996549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.306615+0000","last_change":"2023-10-18T08:58:28.298222+0000","last_active":"2023-10-18T08:58:29.306615+0000","last_peered":"2023-10-18T08:58:29.306615+0000","last_clean":"2023-10-18T08:58:29.306615+0000","last_became_active":"2023-10-18T08:58:28.297672+0000","last_became_peered":"2023-10-18T08:58:28.297672+0000","last_unstale":"2023-10-18T08:58:29.306615+0000","last_undegraded":"2023-10-18T08:58:29.306615+0000","last_fullsized":"2023-10-18T08:58:29.306615+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:59.505404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.277176+0000","last_change":"2023-10-18T08:58:28.277176+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.277176+0000","last_undegraded":"2023-10-18T08:58:28.277176+0000","last_fullsized":"2023-10-18T08:58:28.277176+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"66'3","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.344696+0000","last_change":"2023-10-18T08:58:28.290903+0000","last_active":"2023-10-18T08:58:29.344696+0000","last_peered":"2023-10-18T08:58:29.344696+0000","last_clean":"2023-10-18T08:58:29.344696+0000","last_became_active":"2023-10-18T08:58:28.290358+0000","last_became_peered":"2023-10-18T08:58:28.290358+0000","last_unstale":"2023-10-18T08:58:29.344696+0000","last_undegraded":"2023-10-18T08:58:29.344696+0000","last_fullsized":"2023-10-18T08:58:29.344696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:24.568573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.320694+0000","last_change":"2023-10-18T08:58:29.320149+0000","last_active":"2023-10-18T08:58:29.320694+0000","last_peered":"2023-10-18T08:58:29.320694+0000","last_clean":"2023-10-18T08:58:29.320694+0000","last_became_active":"2023-10-18T08:58:29.317356+0000","last_became_peered":"2023-10-18T08:58:29.317356+0000","last_unstale":"2023-10-18T08:58:29.320694+0000","last_undegraded":"2023-10-18T08:58:29.320694+0000","last_fullsized":"2023-10-18T08:58:29.320694+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:49.888147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.302112+0000","last_change":"2023-10-18T08:58:28.302001+0000","last_active":"2023-10-18T08:58:28.302112+0000","last_peered":"2023-10-18T08:58:28.302112+0000","last_clean":"2023-10-18T08:58:28.302112+0000","last_became_active":"2023-10-18T08:58:28.301382+0000","last_became_peered":"2023-10-18T08:58:28.301382+0000","last_unstale":"2023-10-18T08:58:28.302112+0000","last_undegraded":"2023-10-18T08:58:28.302112+0000","last_fullsized":"2023-10-18T08:58:28.302112+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:43:35.327448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.290401+0000","last_change":"2023-10-18T08:58:28.290401+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.290401+0000","last_undegraded":"2023-10-18T08:58:28.290401+0000","last_fullsized":"2023-10-18T08:58:28.290401+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"66'2","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341261+0000","last_change":"2023-10-18T08:58:28.296421+0000","last_active":"2023-10-18T08:58:29.341261+0000","last_peered":"2023-10-18T08:58:29.341261+0000","last_clean":"2023-10-18T08:58:29.341261+0000","last_became_active":"2023-10-18T08:58:28.296176+0000","last_became_peered":"2023-10-18T08:58:28.296176+0000","last_unstale":"2023-10-18T08:58:29.341261+0000","last_undegraded":"2023-10-18T08:58:29.341261+0000","last_fullsized":"2023-10-18T08:58:29.341261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:14:17.772896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.324023+0000","last_change":"2023-10-18T08:58:29.322440+0000","last_active":"2023-10-18T08:58:29.324023+0000","last_peered":"2023-10-18T08:58:29.324023+0000","last_clean":"2023-10-18T08:58:29.324023+0000","last_became_active":"2023-10-18T08:58:29.319010+0000","last_became_peered":"2023-10-18T08:58:29.319010+0000","last_unstale":"2023-10-18T08:58:29.324023+0000","last_undegraded":"2023-10-18T08:58:29.324023+0000","last_fullsized":"2023-10-18T08:58:29.324023+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:05.722853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.297280+0000","last_change":"2023-10-18T08:58:28.290878+0000","last_active":"2023-10-18T08:58:28.297280+0000","last_peered":"2023-10-18T08:58:28.297280+0000","last_clean":"2023-10-18T08:58:28.297280+0000","last_became_active":"2023-10-18T08:58:28.290244+0000","last_became_peered":"2023-10-18T08:58:28.290244+0000","last_unstale":"2023-10-18T08:58:28.297280+0000","last_undegraded":"2023-10-18T08:58:28.297280+0000","last_fullsized":"2023-10-18T08:58:28.297280+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:01.096478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.288004+0000","last_change":"2023-10-18T08:58:28.288004+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.288004+0000","last_undegraded":"2023-10-18T08:58:28.288004+0000","last_fullsized":"2023-10-18T08:58:28.288004+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300048+0000","last_change":"2023-10-18T08:58:28.299958+0000","last_active":"2023-10-18T08:58:28.300048+0000","last_peered":"2023-10-18T08:58:28.300048+0000","last_clean":"2023-10-18T08:58:28.300048+0000","last_became_active":"2023-10-18T08:58:28.299673+0000","last_became_peered":"2023-10-18T08:58:28.299673+0000","last_unstale":"2023-10-18T08:58:28.300048+0000","last_undegraded":"2023-10-18T08:58:28.300048+0000","last_fullsized":"2023-10-18T08:58:28.300048+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:08.788078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313474+0000","last_change":"2023-10-18T08:58:29.313209+0000","last_active":"2023-10-18T08:58:29.313474+0000","last_peered":"2023-10-18T08:58:29.313474+0000","last_clean":"2023-10-18T08:58:29.313474+0000","last_became_active":"2023-10-18T08:58:29.312599+0000","last_became_peered":"2023-10-18T08:58:29.312599+0000","last_unstale":"2023-10-18T08:58:29.313474+0000","last_undegraded":"2023-10-18T08:58:29.313474+0000","last_fullsized":"2023-10-18T08:58:29.313474+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:33:04.626358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"66'8","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.343659+0000","last_change":"2023-10-18T08:58:28.297447+0000","last_active":"2023-10-18T08:58:29.343659+0000","last_peered":"2023-10-18T08:58:29.343659+0000","last_clean":"2023-10-18T08:58:29.343659+0000","last_became_active":"2023-10-18T08:58:28.297055+0000","last_became_peered":"2023-10-18T08:58:28.297055+0000","last_unstale":"2023-10-18T08:58:29.343659+0000","last_undegraded":"2023-10-18T08:58:29.343659+0000","last_fullsized":"2023-10-18T08:58:29.343659+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:57:22.719753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326347+0000","last_change":"2023-10-18T08:58:29.326263+0000","last_active":"2023-10-18T08:58:29.326347+0000","last_peered":"2023-10-18T08:58:29.326347+0000","last_clean":"2023-10-18T08:58:29.326347+0000","last_became_active":"2023-10-18T08:58:29.325851+0000","last_became_peered":"2023-10-18T08:58:29.325851+0000","last_unstale":"2023-10-18T08:58:29.326347+0000","last_undegraded":"2023-10-18T08:58:29.326347+0000","last_fullsized":"2023-10-18T08:58:29.326347+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:42:38.047653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"66'1","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.339858+0000","last_change":"2023-10-18T08:58:28.300004+0000","last_active":"2023-10-18T08:58:29.339858+0000","last_peered":"2023-10-18T08:58:29.339858+0000","last_clean":"2023-10-18T08:58:29.339858+0000","last_became_active":"2023-10-18T08:58:28.294830+0000","last_became_peered":"2023-10-18T08:58:28.294830+0000","last_unstale":"2023-10-18T08:58:29.339858+0000","last_undegraded":"2023-10-18T08:58:29.339858+0000","last_fullsized":"2023-10-18T08:58:29.339858+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:16.750986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.293219+0000","last_change":"2023-10-18T08:58:28.293219+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.293219+0000","last_undegraded":"2023-10-18T08:58:28.293219+0000","last_fullsized":"2023-10-18T08:58:28.293219+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.298454+0000","last_change":"2023-10-18T08:58:28.298360+0000","last_active":"2023-10-18T08:58:28.298454+0000","last_peered":"2023-10-18T08:58:28.298454+0000","last_clean":"2023-10-18T08:58:28.298454+0000","last_became_active":"2023-10-18T08:58:28.296520+0000","last_became_peered":"2023-10-18T08:58:28.296520+0000","last_unstale":"2023-10-18T08:58:28.298454+0000","last_undegraded":"2023-10-18T08:58:28.298454+0000","last_fullsized":"2023-10-18T08:58:28.298454+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:48.874512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326429+0000","last_change":"2023-10-18T08:58:29.324872+0000","last_active":"2023-10-18T08:58:29.326429+0000","last_peered":"2023-10-18T08:58:29.326429+0000","last_clean":"2023-10-18T08:58:29.326429+0000","last_became_active":"2023-10-18T08:58:29.323103+0000","last_became_peered":"2023-10-18T08:58:29.323103+0000","last_unstale":"2023-10-18T08:58:29.326429+0000","last_undegraded":"2023-10-18T08:58:29.326429+0000","last_fullsized":"2023-10-18T08:58:29.326429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:11:33.988349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.299240+0000","last_change":"2023-10-18T08:58:28.298668+0000","last_active":"2023-10-18T08:58:28.299240+0000","last_peered":"2023-10-18T08:58:28.299240+0000","last_clean":"2023-10-18T08:58:28.299240+0000","last_became_active":"2023-10-18T08:58:28.297217+0000","last_became_peered":"2023-10-18T08:58:28.297217+0000","last_unstale":"2023-10-18T08:58:28.299240+0000","last_undegraded":"2023-10-18T08:58:28.299240+0000","last_fullsized":"2023-10-18T08:58:28.299240+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:37.673296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313511+0000","last_change":"2023-10-18T08:58:29.313231+0000","last_active":"2023-10-18T08:58:29.313511+0000","last_peered":"2023-10-18T08:58:29.313511+0000","last_clean":"2023-10-18T08:58:29.313511+0000","last_became_active":"2023-10-18T08:58:29.311497+0000","last_became_peered":"2023-10-18T08:58:29.311497+0000","last_unstale":"2023-10-18T08:58:29.313511+0000","last_undegraded":"2023-10-18T08:58:29.313511+0000","last_fullsized":"2023-10-18T08:58:29.313511+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:54.275856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.298061+0000","last_change":"2023-10-18T08:58:28.297688+0000","last_active":"2023-10-18T08:58:28.298061+0000","last_peered":"2023-10-18T08:58:28.298061+0000","last_clean":"2023-10-18T08:58:28.298061+0000","last_became_active":"2023-10-18T08:58:28.297401+0000","last_became_peered":"2023-10-18T08:58:28.297401+0000","last_unstale":"2023-10-18T08:58:28.298061+0000","last_undegraded":"2023-10-18T08:58:28.298061+0000","last_fullsized":"2023-10-18T08:58:28.298061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:44.607015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.292556+0000","last_change":"2023-10-18T08:58:28.292556+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.292556+0000","last_undegraded":"2023-10-18T08:58:28.292556+0000","last_fullsized":"2023-10-18T08:58:28.292556+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"66'2","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341374+0000","last_change":"2023-10-18T08:58:28.299131+0000","last_active":"2023-10-18T08:58:29.341374+0000","last_peered":"2023-10-18T08:58:29.341374+0000","last_clean":"2023-10-18T08:58:29.341374+0000","last_became_active":"2023-10-18T08:58:28.298514+0000","last_became_peered":"2023-10-18T08:58:28.298514+0000","last_unstale":"2023-10-18T08:58:29.341374+0000","last_undegraded":"2023-10-18T08:58:29.341374+0000","last_fullsized":"2023-10-18T08:58:29.341374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:42:24.333152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.294168+0000","last_change":"2023-10-18T08:58:28.294168+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.294168+0000","last_undegraded":"2023-10-18T08:58:28.294168+0000","last_fullsized":"2023-10-18T08:58:28.294168+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.295059+0000","last_change":"2023-10-18T08:58:28.294327+0000","last_active":"2023-10-18T08:58:28.295059+0000","last_peered":"2023-10-18T08:58:28.295059+0000","last_clean":"2023-10-18T08:58:28.295059+0000","last_became_active":"2023-10-18T08:58:28.293852+0000","last_became_peered":"2023-10-18T08:58:28.293852+0000","last_unstale":"2023-10-18T08:58:28.295059+0000","last_undegraded":"2023-10-18T08:58:28.295059+0000","last_fullsized":"2023-10-18T08:58:28.295059+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:17:54.342972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289360+0000","last_change":"2023-10-18T08:58:28.289360+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289360+0000","last_undegraded":"2023-10-18T08:58:28.289360+0000","last_fullsized":"2023-10-18T08:58:28.289360+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.302241+0000","last_change":"2023-10-18T08:58:28.302115+0000","last_active":"2023-10-18T08:58:28.302241+0000","last_peered":"2023-10-18T08:58:28.302241+0000","last_clean":"2023-10-18T08:58:28.302241+0000","last_became_active":"2023-10-18T08:58:28.300920+0000","last_became_peered":"2023-10-18T08:58:28.300920+0000","last_unstale":"2023-10-18T08:58:28.302241+0000","last_undegraded":"2023-10-18T08:58:28.302241+0000","last_fullsized":"2023-10-18T08:58:28.302241+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:26:14.774552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289377+0000","last_change":"2023-10-18T08:58:28.289377+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289377+0000","last_undegraded":"2023-10-18T08:58:28.289377+0000","last_fullsized":"2023-10-18T08:58:28.289377+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.280714+0000","last_change":"2023-10-18T08:58:28.280587+0000","last_active":"2023-10-18T08:58:28.280714+0000","last_peered":"2023-10-18T08:58:28.280714+0000","last_clean":"2023-10-18T08:58:28.280714+0000","last_became_active":"2023-10-18T08:58:28.280261+0000","last_became_peered":"2023-10-18T08:58:28.280261+0000","last_unstale":"2023-10-18T08:58:28.280714+0000","last_undegraded":"2023-10-18T08:58:28.280714+0000","last_fullsized":"2023-10-18T08:58:28.280714+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:56.134357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289491+0000","last_change":"2023-10-18T08:58:28.289491+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289491+0000","last_undegraded":"2023-10-18T08:58:28.289491+0000","last_fullsized":"2023-10-18T08:58:28.289491+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.299921+0000","last_change":"2023-10-18T08:58:28.299846+0000","last_active":"2023-10-18T08:58:28.299921+0000","last_peered":"2023-10-18T08:58:28.299921+0000","last_clean":"2023-10-18T08:58:28.299921+0000","last_became_active":"2023-10-18T08:58:28.299121+0000","last_became_peered":"2023-10-18T08:58:28.299121+0000","last_unstale":"2023-10-18T08:58:28.299921+0000","last_undegraded":"2023-10-18T08:58:28.299921+0000","last_fullsized":"2023-10-18T08:58:28.299921+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:23.982914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.277537+0000","last_change":"2023-10-18T08:58:28.277537+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.277537+0000","last_undegraded":"2023-10-18T08:58:28.277537+0000","last_fullsized":"2023-10-18T08:58:28.277537+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300837+0000","last_change":"2023-10-18T08:58:28.300674+0000","last_active":"2023-10-18T08:58:28.300837+0000","last_peered":"2023-10-18T08:58:28.300837+0000","last_clean":"2023-10-18T08:58:28.300837+0000","last_became_active":"2023-10-18T08:58:28.300119+0000","last_became_peered":"2023-10-18T08:58:28.300119+0000","last_unstale":"2023-10-18T08:58:28.300837+0000","last_undegraded":"2023-10-18T08:58:28.300837+0000","last_fullsized":"2023-10-18T08:58:28.300837+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:35.906463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.284498+0000","last_change":"2023-10-18T08:58:28.284498+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.284498+0000","last_undegraded":"2023-10-18T08:58:28.284498+0000","last_fullsized":"2023-10-18T08:58:28.284498+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.303285+0000","last_change":"2023-10-18T08:58:28.303209+0000","last_active":"2023-10-18T08:58:28.303285+0000","last_peered":"2023-10-18T08:58:28.303285+0000","last_clean":"2023-10-18T08:58:28.303285+0000","last_became_active":"2023-10-18T08:58:28.302960+0000","last_became_peered":"2023-10-18T08:58:28.302960+0000","last_unstale":"2023-10-18T08:58:28.303285+0000","last_undegraded":"2023-10-18T08:58:28.303285+0000","last_fullsized":"2023-10-18T08:58:28.303285+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:50.826443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.316386+0000","last_change":"2023-10-18T08:58:29.315776+0000","last_active":"2023-10-18T08:58:29.316386+0000","last_peered":"2023-10-18T08:58:29.316386+0000","last_clean":"2023-10-18T08:58:29.316386+0000","last_became_active":"2023-10-18T08:58:29.315154+0000","last_became_peered":"2023-10-18T08:58:29.315154+0000","last_unstale":"2023-10-18T08:58:29.316386+0000","last_undegraded":"2023-10-18T08:58:29.316386+0000","last_fullsized":"2023-10-18T08:58:29.316386+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:51.344108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.296149+0000","last_change":"2023-10-18T08:58:28.295800+0000","last_active":"2023-10-18T08:58:28.296149+0000","last_peered":"2023-10-18T08:58:28.296149+0000","last_clean":"2023-10-18T08:58:28.296149+0000","last_became_active":"2023-10-18T08:58:28.295520+0000","last_became_peered":"2023-10-18T08:58:28.295520+0000","last_unstale":"2023-10-18T08:58:28.296149+0000","last_undegraded":"2023-10-18T08:58:28.296149+0000","last_fullsized":"2023-10-18T08:58:28.296149+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:01.809009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313850+0000","last_change":"2023-10-18T08:58:29.313510+0000","last_active":"2023-10-18T08:58:29.313850+0000","last_peered":"2023-10-18T08:58:29.313850+0000","last_clean":"2023-10-18T08:58:29.313850+0000","last_became_active":"2023-10-18T08:58:29.312697+0000","last_became_peered":"2023-10-18T08:58:29.312697+0000","last_unstale":"2023-10-18T08:58:29.313850+0000","last_undegraded":"2023-10-18T08:58:29.313850+0000","last_fullsized":"2023-10-18T08:58:29.313850+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:47.848978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300768+0000","last_change":"2023-10-18T08:58:28.300006+0000","last_active":"2023-10-18T08:58:28.300768+0000","last_peered":"2023-10-18T08:58:28.300768+0000","last_clean":"2023-10-18T08:58:28.300768+0000","last_became_active":"2023-10-18T08:58:28.299510+0000","last_became_peered":"2023-10-18T08:58:28.299510+0000","last_unstale":"2023-10-18T08:58:28.300768+0000","last_undegraded":"2023-10-18T08:58:28.300768+0000","last_fullsized":"2023-10-18T08:58:28.300768+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:01.392655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.276744+0000","last_change":"2023-10-18T08:58:28.276744+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.276744+0000","last_undegraded":"2023-10-18T08:58:28.276744+0000","last_fullsized":"2023-10-18T08:58:28.276744+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.298217+0000","last_change":"2023-10-18T08:58:28.297980+0000","last_active":"2023-10-18T08:58:28.298217+0000","last_peered":"2023-10-18T08:58:28.298217+0000","last_clean":"2023-10-18T08:58:28.298217+0000","last_became_active":"2023-10-18T08:58:28.297743+0000","last_became_peered":"2023-10-18T08:58:28.297743+0000","last_unstale":"2023-10-18T08:58:28.298217+0000","last_undegraded":"2023-10-18T08:58:28.298217+0000","last_fullsized":"2023-10-18T08:58:28.298217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:01.186029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.318289+0000","last_change":"2023-10-18T08:58:29.318110+0000","last_active":"2023-10-18T08:58:29.318289+0000","last_peered":"2023-10-18T08:58:29.318289+0000","last_clean":"2023-10-18T08:58:29.318289+0000","last_became_active":"2023-10-18T08:58:29.317066+0000","last_became_peered":"2023-10-18T08:58:29.317066+0000","last_unstale":"2023-10-18T08:58:29.318289+0000","last_undegraded":"2023-10-18T08:58:29.318289+0000","last_fullsized":"2023-10-18T08:58:29.318289+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:54.004578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.299504+0000","last_change":"2023-10-18T08:58:28.300829+0000","last_active":"2023-10-18T08:58:29.299504+0000","last_peered":"2023-10-18T08:58:29.299504+0000","last_clean":"2023-10-18T08:58:29.299504+0000","last_became_active":"2023-10-18T08:58:28.300515+0000","last_became_peered":"2023-10-18T08:58:28.300515+0000","last_unstale":"2023-10-18T08:58:29.299504+0000","last_undegraded":"2023-10-18T08:58:29.299504+0000","last_fullsized":"2023-10-18T08:58:29.299504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:15:03.965062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.331823+0000","last_change":"2023-10-18T08:58:29.331231+0000","last_active":"2023-10-18T08:58:29.331823+0000","last_peered":"2023-10-18T08:58:29.331823+0000","last_clean":"2023-10-18T08:58:29.331823+0000","last_became_active":"2023-10-18T08:58:29.331003+0000","last_became_peered":"2023-10-18T08:58:29.331003+0000","last_unstale":"2023-10-18T08:58:29.331823+0000","last_undegraded":"2023-10-18T08:58:29.331823+0000","last_fullsized":"2023-10-18T08:58:29.331823+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:48.691583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300206+0000","last_change":"2023-10-18T08:58:28.300119+0000","last_active":"2023-10-18T08:58:28.300206+0000","last_peered":"2023-10-18T08:58:28.300206+0000","last_clean":"2023-10-18T08:58:28.300206+0000","last_became_active":"2023-10-18T08:58:28.299753+0000","last_became_peered":"2023-10-18T08:58:28.299753+0000","last_unstale":"2023-10-18T08:58:28.300206+0000","last_undegraded":"2023-10-18T08:58:28.300206+0000","last_fullsized":"2023-10-18T08:58:28.300206+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:27.836895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":214,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.275442+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:58:28.275442+0000","last_peered":"2023-10-18T08:58:28.275442+0000","last_clean":"2023-10-18T08:58:28.275442+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:58:28.275442+0000","last_undegraded":"2023-10-18T08:58:28.275442+0000","last_fullsized":"2023-10-18T08:58:28.275442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:53.386532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":146,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":902,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194409,"statfs":{"total":200000000,"available":199075709,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.99299999999999999,"15min":0.98899999999999999},"min":{"1min":0.70299999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3400000000000001,"5min":1.538,"15min":1.538},"last":1.222},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.034,"15min":1.008},"min":{"1min":0.69599999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4019999999999999,"5min":1.667,"15min":1.667},"last":1.048}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1240000000000001,"15min":1.147},"min":{"1min":0.80300000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5900000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.375},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1419999999999999,"15min":1.1220000000000001},"min":{"1min":0.748,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7450000000000001,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.5920000000000001}]},{"osd":2,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.2450000000000001,"15min":1.2330000000000001},"min":{"1min":0.80200000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.284},{"interface":"front","average":{"1min":1.276,"5min":1.2010000000000001,"15min":1.1910000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.798,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.3320000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2629999999999999,"15min":1.264},"min":{"1min":0.95999999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.702,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3140000000000001,"15min":1.3149999999999999},"min":{"1min":0.94199999999999995,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.873,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.478}]},{"osd":4,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1899999999999999,"15min":1.1599999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6799999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.3779999999999999,"15min":1.401},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.6279999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.534}]},{"osd":5,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.353,"15min":1.353},"min":{"1min":0.99399999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.111},{"interface":"front","average":{"1min":1.349,"5min":1.4019999999999999,"15min":1.4159999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.865,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.4430000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.464,"15min":1.458},"min":{"1min":1.0369999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.923,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.518},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.492,"15min":1.51},"min":{"1min":1.0669999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.8069999999999999,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.633}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":326,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194986,"statfs":{"total":200000000,"available":199666077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1020000000000001,"15min":1.0800000000000001},"min":{"1min":0.623,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.651,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.228,"15min":1.234},"min":{"1min":0.84499999999999997,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.613}]},{"osd":1,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.089,"15min":1.085},"min":{"1min":0.65600000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1799999999999999,"15min":1.1859999999999999},"min":{"1min":0.80600000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.962,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":1.2829999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.161,"15min":1.173},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5029999999999999,"5min":1.696,"15min":1.696},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.1970000000000001,"15min":1.194},"min":{"1min":0.92500000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.494}]},{"osd":3,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2509999999999999,"15min":1.2569999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.669,"5min":1.784,"15min":1.784},"last":1.345},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.228,"15min":1.22},"min":{"1min":0.84999999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.526}]},{"osd":4,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.212,"15min":1.175},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.129},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.26,"15min":1.266},"min":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.696,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.4690000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3520000000000001,"15min":1.3480000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.599,"5min":1.667,"15min":1.667},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.232,"5min":1.238,"15min":1.218},"min":{"1min":0.629,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.52,"5min":1.746,"15min":1.746},"last":1.403}]},{"osd":7,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.4039999999999999,"15min":1.3959999999999999},"min":{"1min":0.73999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.51,"15min":1.5249999999999999},"min":{"1min":0.84099999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.831,"5min":2.794,"15min":2.794},"last":1.6479999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":326,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194986,"statfs":{"total":200000000,"available":199666077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.7529999999999999,"5min":1.3160000000000001,"15min":1.224},"min":{"1min":0.68300000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":9.3539999999999992,"5min":9.3539999999999992,"15min":9.3539999999999992},"last":1.258},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.3220000000000001,"15min":1.3029999999999999},"min":{"1min":0.71999999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":5.3029999999999999,"5min":5.3029999999999999,"15min":5.3029999999999999},"last":1.4379999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2090000000000001,"15min":1.2230000000000001},"min":{"1min":0.755,"5min":0.749,"15min":0.749},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.222,"15min":1.248},"min":{"1min":0.82199999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.367,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":1.1739999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.268,"15min":1.268},"min":{"1min":0.78100000000000003,"5min":0.755,"15min":0.755},"max":{"1min":1.5309999999999999,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.292,"5min":1.2889999999999999,"15min":1.2929999999999999},"min":{"1min":0.78400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.3069999999999999,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.0549999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.25,"15min":1.246},"min":{"1min":0.93300000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.524,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.337,"15min":1.3320000000000001},"min":{"1min":0.94399999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.6379999999999999,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.999}]},{"osd":4,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4139999999999999,"15min":1.419},"min":{"1min":0.93400000000000005,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.881,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.4950000000000001,"15min":1.5329999999999999},"min":{"1min":0.90000000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":4.4429999999999996,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":1.349}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.377,"15min":1.4119999999999999},"min":{"1min":0.93899999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.5469999999999997,"5min":5.141,"15min":5.141},"last":1.202},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.556,"15min":1.587},"min":{"1min":1.151,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0590000000000002,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.117}]},{"osd":7,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.6419999999999999,"15min":1.6699999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":4.2050000000000001,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5149999999999999,"15min":1.5249999999999999},"min":{"1min":1.0249999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":4.4690000000000003,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":1.512}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":325,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194987,"statfs":{"total":200000000,"available":199667096,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.107,"15min":1.1100000000000001},"min":{"1min":0.745,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":0.996,"5min":1.129,"15min":1.169},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.6990000000000001,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.1439999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.4849999999999999,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":1.008},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2250000000000001,"15min":1.2390000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.651}]},{"osd":2,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1879999999999999,"15min":1.179},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.254},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.27,"15min":1.3029999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.994,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.3819999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2889999999999999,"15min":1.3009999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":1.286},{"interface":"front","average":{"1min":1.421,"5min":1.4179999999999999,"15min":1.4139999999999999},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.6880000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.1910000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.323,"15min":1.304},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.778,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.5},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.351,"15min":1.367},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.228}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.4339999999999999,"15min":1.3939999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.556,"5min":1.5629999999999999,"15min":1.5760000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":1.5549999999999999}]},{"osd":7,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.4950000000000001,"15min":1.514},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5529999999999999,"15min":1.5509999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.585,"5min":2.585,"15min":2.585},"last":1.4530000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":326,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194986,"statfs":{"total":200000000,"available":199666077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.85199999999999998,"15min":0.77400000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.304,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.95199999999999996,"15min":0.873},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.393,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":0.46600000000000003}]},{"osd":1,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.97799999999999998,"15min":0.94199999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.208,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.1419999999999999,"15min":1.113},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.5660000000000001,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.258}]},{"osd":2,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.0549999999999999,"15min":1.0149999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4330000000000001,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1080000000000001,"15min":1.054},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6459999999999999,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":1.3149999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.464,"15min":1.3049999999999999},"min":{"1min":0.83599999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.5649999999999999,"5min":18.664999999999999,"15min":18.664999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.21,"15min":1.1499999999999999},"min":{"1min":0.69099999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.756,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":1.0149999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.276,"15min":1.2170000000000001},"min":{"1min":0.72399999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.702,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":0.87},{"interface":"front","average":{"1min":1.425,"5min":1.502,"15min":1.3400000000000001},"min":{"1min":0.80800000000000005,"5min":0.748,"15min":0.748},"max":{"1min":1.885,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.375}]},{"osd":6,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4750000000000001,"15min":1.4199999999999999},"min":{"1min":0.82499999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.7649999999999999,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3959999999999999,"15min":1.3300000000000001},"min":{"1min":0.77200000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.774,"5min":7.242,"15min":7.242},"last":1.103}]},{"osd":7,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.4019999999999999,"15min":1.3320000000000001},"min":{"1min":0.80600000000000005,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.9970000000000001,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.431},{"interface":"front","average":{"1min":1.47,"5min":1.556,"15min":1.4470000000000001},"min":{"1min":0.93100000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.8580000000000001,"5min":12.946,"15min":12.946},"last":0.84199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":319,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194992,"statfs":{"total":200000000,"available":199672344,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90500000000000003,"15min":0.93500000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.55,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0029999999999999,"15min":1.05},"min":{"1min":0.63100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4870000000000001,"5min":1.651,"15min":1.651},"last":0.75900000000000001}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.0249999999999999,"15min":1.0549999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.427,"5min":4.29,"15min":4.29},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.198,"5min":1.0760000000000001,"15min":1.095},"min":{"1min":0.65700000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.583,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.8140000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0840000000000001,"15min":1.1259999999999999},"min":{"1min":0.54600000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.6419999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.256},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.391,"15min":1.48},"min":{"1min":0.57899999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8360000000000001,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":1.7709999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.448,"15min":1.4950000000000001},"min":{"1min":0.95699999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.7450000000000001,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":1.639},{"interface":"front","average":{"1min":1.421,"5min":1.415,"15min":1.45},"min":{"1min":0.93799999999999994,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.8029999999999999,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.5940000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.234,"15min":1.268},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6719999999999999,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":1.054},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.5,"15min":1.5760000000000001},"min":{"1min":0.96099999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.863,"5min":7.657,"15min":7.657},"last":1.2989999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.472,"15min":1.5309999999999999},"min":{"1min":0.82699999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8999999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.391},{"interface":"front","average":{"1min":3.536,"5min":1.8240000000000001,"15min":1.5720000000000001},"min":{"1min":0.98899999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":37.771999999999998,"5min":37.771999999999998,"15min":37.771999999999998},"last":1.718}]},{"osd":7,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.5309999999999999,"15min":1.573},"min":{"1min":0.98699999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.016,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.4410000000000001,"15min":1.4710000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1349999999999998,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.5269999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":325,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194987,"statfs":{"total":200000000,"available":199667096,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.79200000000000004,"15min":0.79800000000000004},"min":{"1min":0.44500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":1.327,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.999,"5min":0.89300000000000002,"15min":0.89100000000000001},"min":{"1min":0.501,"5min":0.443,"15min":0.443},"max":{"1min":1.5129999999999999,"5min":1.55,"15min":1.55},"last":0.96099999999999997}]},{"osd":2,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.91100000000000003,"15min":0.92300000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6850000000000001,"5min":1.859,"15min":1.859},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.83299999999999996,"15min":0.83499999999999996},"min":{"1min":0.47999999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.625,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.81000000000000005}]},{"osd":3,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.98199999999999998,"15min":1.008},"min":{"1min":0.52800000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.571,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.0229999999999999,"15min":1.0469999999999999},"min":{"1min":0.54500000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.8859999999999999,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":0.89000000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.22,"15min":1.2350000000000001},"min":{"1min":0.82099999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5760000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.216},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1970000000000001,"15min":1.1930000000000001},"min":{"1min":0.93799999999999994,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.556,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.6759999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.268,"15min":1.2809999999999999},"min":{"1min":0.86599999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.625,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.367},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.343,"15min":1.343},"min":{"1min":0.90500000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.6759999999999999,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.486}]},{"osd":6,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2529999999999999,"15min":1.2649999999999999},"min":{"1min":0.89100000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6240000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2450000000000001,"15min":1.248},"min":{"1min":0.78900000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.744,"5min":1.853,"15min":1.853},"last":1.1160000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.333,"15min":1.339},"min":{"1min":0.84399999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.716,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.288},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3380000000000001,"15min":1.3400000000000001},"min":{"1min":0.95099999999999996,"5min":0.746,"15min":0.746},"max":{"1min":1.758,"5min":1.994,"15min":1.994},"last":1.4179999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":895,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194416,"statfs":{"total":200000000,"available":199082995,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.89500000000000002,"15min":0.93600000000000005},"min":{"1min":0.371,"5min":0.37,"15min":0.37},"max":{"1min":1.0940000000000001,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":1.163},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0660000000000001,"15min":1.079},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.90300000000000002}]},{"osd":2,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.121,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.873},{"interface":"front","average":{"1min":1.121,"5min":1.1240000000000001,"15min":1.125},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.8109999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.0549999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.22,"15min":1.306},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6639999999999999,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.377,"5min":1.4750000000000001,"15min":1.5309999999999999},"min":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.9259999999999999,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.1180000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2629999999999999,"15min":1.3169999999999999},"min":{"1min":0.81200000000000006,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.5549999999999999,"5min":4.492,"15min":4.492},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.234,"5min":1.2490000000000001,"15min":1.2470000000000001},"min":{"1min":0.93200000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.629,"5min":1.726,"15min":1.726},"last":1.1459999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.377,"15min":1.4490000000000001},"min":{"1min":0.89500000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.944,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3580000000000001,"15min":1.3859999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8240000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.0900000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.5629999999999999,"15min":1.6100000000000001},"min":{"1min":0.877,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.0409999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.4670000000000001,"15min":1.5189999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.863,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.427}]},{"osd":7,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.619,"15min":1.661},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.1160000000000001,"5min":3.492,"15min":3.492},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.621,"15min":1.7130000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.0449999999999999,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.516}]}]}],"pool_statfs":[]}} 2023-10-18T08:58:34.342 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:58:34.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.613+0000 7f404527c640 1 -- 172.21.15.40:0/3722520208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40400d19c0 msgr2=0x7f40400cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:34.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.613+0000 7f404527c640 1 --2- 172.21.15.40:0/3722520208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40400d19c0 0x7f40400cdff0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f40400cf160 tx=0x7f403802be20 comp rx=0 tx=0).stop 2023-10-18T08:58:34.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.613+0000 7f404527c640 1 -- 172.21.15.40:0/3722520208 shutdown_connections 2023-10-18T08:58:34.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.613+0000 7f404527c640 1 --2- 172.21.15.40:0/3722520208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40400d19c0 0x7f40400cdff0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.613+0000 7f404527c640 1 --2- 172.21.15.40:0/3722520208 >> v2:172.21.15.144:3300/0 conn(0x7f40400d0100 0x7f40400d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.613+0000 7f404527c640 1 --2- 172.21.15.40:0/3722520208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f404015b760 0x7f40400cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.613+0000 7f404527c640 1 -- 172.21.15.40:0/3722520208 >> 172.21.15.40:0/3722520208 conn(0x7f404009a9a0 msgr2=0x7f4040088fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:34.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.613+0000 7f404527c640 1 -- 172.21.15.40:0/3722520208 shutdown_connections 2023-10-18T08:58:34.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.613+0000 7f404527c640 1 -- 172.21.15.40:0/3722520208 wait complete. 2023-10-18T08:58:34.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.615+0000 7f404527c640 1 Processor -- start 2023-10-18T08:58:34.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.615+0000 7f404527c640 1 -- start start 2023-10-18T08:58:34.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.616+0000 7f404527c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40400d0100 0x7f40401f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:34.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.616+0000 7f404527c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40400d19c0 0x7f40400b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:34.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.616+0000 7f404527c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f404015b760 0x7f40400c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:34.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.616+0000 7f404527c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40400c7950 con 0x7f40400d0100 2023-10-18T08:58:34.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.616+0000 7f404527c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40400c7c30 con 0x7f404015b760 2023-10-18T08:58:34.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.616+0000 7f404527c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40400c7f10 con 0x7f40400d19c0 2023-10-18T08:58:34.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.616+0000 7f403effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40400d0100 0x7f40401f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:34.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.616+0000 7f403effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40400d0100 0x7f40401f16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38552/0 (socket says 172.21.15.40:38552) 2023-10-18T08:58:34.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.616+0000 7f403effd640 1 -- 172.21.15.40:0/3933553459 learned_addr learned my addr 172.21.15.40:0/3933553459 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:34.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.617+0000 7f403effd640 1 -- 172.21.15.40:0/3933553459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40400d19c0 msgr2=0x7f40400b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:34.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.617+0000 7f403e7fc640 1 --2- 172.21.15.40:0/3933553459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40400d19c0 0x7f40400b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:34.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.617+0000 7f403effd640 1 --2- 172.21.15.40:0/3933553459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40400d19c0 0x7f40400b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.617+0000 7f403f7fe640 1 --2- 172.21.15.40:0/3933553459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f404015b760 0x7f40400c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:34.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.617+0000 7f403effd640 1 -- 172.21.15.40:0/3933553459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f404015b760 msgr2=0x7f40400c6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:34.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.617+0000 7f403effd640 1 --2- 172.21.15.40:0/3933553459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f404015b760 0x7f40400c6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.617+0000 7f403effd640 1 -- 172.21.15.40:0/3933553459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4038009560 con 0x7f40400d0100 2023-10-18T08:58:34.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.617+0000 7f403f7fe640 1 --2- 172.21.15.40:0/3933553459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f404015b760 0x7f40400c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:34.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.617+0000 7f403effd640 1 --2- 172.21.15.40:0/3933553459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40400d0100 0x7f40401f16a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f403000ba50 tx=0x7f403000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:34.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.617+0000 7f40377fe640 1 -- 172.21.15.40:0/3933553459 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403000cbe0 con 0x7f40400d0100 2023-10-18T08:58:34.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.617+0000 7f40377fe640 1 -- 172.21.15.40:0/3933553459 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f403000ce50 con 0x7f40400d0100 2023-10-18T08:58:34.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.617+0000 7f40377fe640 1 -- 172.21.15.40:0/3933553459 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4030014680 con 0x7f40400d0100 2023-10-18T08:58:34.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.618+0000 7f404527c640 1 -- 172.21.15.40:0/3933553459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4040157a90 con 0x7f40400d0100 2023-10-18T08:58:34.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.618+0000 7f404527c640 1 -- 172.21.15.40:0/3933553459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40401f1be0 con 0x7f40400d0100 2023-10-18T08:58:34.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.619+0000 7f404527c640 1 -- 172.21.15.40:0/3933553459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40400739c0 con 0x7f40400d0100 2023-10-18T08:58:34.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.620+0000 7f40377fe640 1 -- 172.21.15.40:0/3933553459 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f403001d3b0 con 0x7f40400d0100 2023-10-18T08:58:34.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.621+0000 7f40377fe640 1 --2- 172.21.15.40:0/3933553459 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f402006ae40 0x7f402006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:34.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.621+0000 7f40377fe640 1 -- 172.21.15.40:0/3933553459 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4030089850 con 0x7f40400d0100 2023-10-18T08:58:34.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.621+0000 7f403e7fc640 1 --2- 172.21.15.40:0/3933553459 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f402006ae40 0x7f402006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:34.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.622+0000 7f403e7fc640 1 --2- 172.21.15.40:0/3933553459 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f402006ae40 0x7f402006d300 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f404009f5e0 tx=0x7f4028008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:34.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.625+0000 7f40377fe640 1 -- 172.21.15.40:0/3933553459 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40300548f0 con 0x7f40400d0100 2023-10-18T08:58:34.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.815+0000 7f404527c640 1 -- 172.21.15.40:0/3933553459 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f404006b840 con 0x7f402006ae40 2023-10-18T08:58:34.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.837+0000 7f40377fe640 1 -- 172.21.15.40:0/3933553459 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279218 (secure 0 0 0) 0x7f404006b840 con 0x7f402006ae40 2023-10-18T08:58:34.841 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:58:34.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.844+0000 7f404527c640 1 -- 172.21.15.40:0/3933553459 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f402006ae40 msgr2=0x7f402006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:34.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.844+0000 7f404527c640 1 --2- 172.21.15.40:0/3933553459 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f402006ae40 0x7f402006d300 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f404009f5e0 tx=0x7f4028008040 comp rx=0 tx=0).stop 2023-10-18T08:58:34.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.844+0000 7f404527c640 1 -- 172.21.15.40:0/3933553459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40400d0100 msgr2=0x7f40401f16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.844+0000 7f404527c640 1 --2- 172.21.15.40:0/3933553459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40400d0100 0x7f40401f16a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f403000ba50 tx=0x7f403000bf10 comp rx=0 tx=0).stop 2023-10-18T08:58:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.844+0000 7f404527c640 1 -- 172.21.15.40:0/3933553459 shutdown_connections 2023-10-18T08:58:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.844+0000 7f404527c640 1 --2- 172.21.15.40:0/3933553459 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f402006ae40 0x7f402006d300 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.844+0000 7f404527c640 1 --2- 172.21.15.40:0/3933553459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f404015b760 0x7f40400c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.844+0000 7f404527c640 1 --2- 172.21.15.40:0/3933553459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40400d19c0 0x7f40400b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.844+0000 7f404527c640 1 --2- 172.21.15.40:0/3933553459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40400d0100 0x7f40401f16a0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.844+0000 7f404527c640 1 -- 172.21.15.40:0/3933553459 >> 172.21.15.40:0/3933553459 conn(0x7f404009a9a0 msgr2=0x7f404015aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.844+0000 7f404527c640 1 -- 172.21.15.40:0/3933553459 shutdown_connections 2023-10-18T08:58:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:34.844+0000 7f404527c640 1 -- 172.21.15.40:0/3933553459 wait complete. 2023-10-18T08:58:34.868 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2023-10-18T08:58:32.964940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590514,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":180,"num_write_kb":2425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3746,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558753,"statfs":{"total":1600000000,"available":1596163471,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":146,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.697568"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.290178+0000","last_change":"2023-10-18T08:58:28.290178+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.290178+0000","last_undegraded":"2023-10-18T08:58:28.290178+0000","last_fullsized":"2023-10-18T08:58:28.290178+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.310859+0000","last_change":"2023-10-18T08:58:29.310661+0000","last_active":"2023-10-18T08:58:29.310859+0000","last_peered":"2023-10-18T08:58:29.310859+0000","last_clean":"2023-10-18T08:58:29.310859+0000","last_became_active":"2023-10-18T08:58:29.309557+0000","last_became_peered":"2023-10-18T08:58:29.309557+0000","last_unstale":"2023-10-18T08:58:29.310859+0000","last_undegraded":"2023-10-18T08:58:29.310859+0000","last_fullsized":"2023-10-18T08:58:29.310859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:46.399620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.323795+0000","last_change":"2023-10-18T08:58:29.321404+0000","last_active":"2023-10-18T08:58:29.323795+0000","last_peered":"2023-10-18T08:58:29.323795+0000","last_clean":"2023-10-18T08:58:29.323795+0000","last_became_active":"2023-10-18T08:58:29.320985+0000","last_became_peered":"2023-10-18T08:58:29.320985+0000","last_unstale":"2023-10-18T08:58:29.323795+0000","last_undegraded":"2023-10-18T08:58:29.323795+0000","last_fullsized":"2023-10-18T08:58:29.323795+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:10:04.504829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.293989+0000","last_change":"2023-10-18T08:58:28.293989+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.293989+0000","last_undegraded":"2023-10-18T08:58:28.293989+0000","last_fullsized":"2023-10-18T08:58:28.293989+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.291475+0000","last_change":"2023-10-18T08:58:28.291475+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.291475+0000","last_undegraded":"2023-10-18T08:58:28.291475+0000","last_fullsized":"2023-10-18T08:58:28.291475+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.287056+0000","last_change":"2023-10-18T08:58:28.287056+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.287056+0000","last_undegraded":"2023-10-18T08:58:28.287056+0000","last_fullsized":"2023-10-18T08:58:28.287056+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.290553+0000","last_change":"2023-10-18T08:58:28.290553+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.290553+0000","last_undegraded":"2023-10-18T08:58:28.290553+0000","last_fullsized":"2023-10-18T08:58:28.290553+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326822+0000","last_change":"2023-10-18T08:58:29.325559+0000","last_active":"2023-10-18T08:58:29.326822+0000","last_peered":"2023-10-18T08:58:29.326822+0000","last_clean":"2023-10-18T08:58:29.326822+0000","last_became_active":"2023-10-18T08:58:29.324359+0000","last_became_peered":"2023-10-18T08:58:29.324359+0000","last_unstale":"2023-10-18T08:58:29.326822+0000","last_undegraded":"2023-10-18T08:58:29.326822+0000","last_fullsized":"2023-10-18T08:58:29.326822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:45.688349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.294163+0000","last_change":"2023-10-18T08:58:28.294163+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.294163+0000","last_undegraded":"2023-10-18T08:58:28.294163+0000","last_fullsized":"2023-10-18T08:58:28.294163+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.278822+0000","last_change":"2023-10-18T08:58:28.278822+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.278822+0000","last_undegraded":"2023-10-18T08:58:28.278822+0000","last_fullsized":"2023-10-18T08:58:28.278822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.277474+0000","last_change":"2023-10-18T08:58:28.277474+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.277474+0000","last_undegraded":"2023-10-18T08:58:28.277474+0000","last_fullsized":"2023-10-18T08:58:28.277474+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.290401+0000","last_change":"2023-10-18T08:58:28.290401+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.290401+0000","last_undegraded":"2023-10-18T08:58:28.290401+0000","last_fullsized":"2023-10-18T08:58:28.290401+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.283202+0000","last_change":"2023-10-18T08:58:28.283202+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.283202+0000","last_undegraded":"2023-10-18T08:58:28.283202+0000","last_fullsized":"2023-10-18T08:58:28.283202+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.293634+0000","last_change":"2023-10-18T08:58:28.293634+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.293634+0000","last_undegraded":"2023-10-18T08:58:28.293634+0000","last_fullsized":"2023-10-18T08:58:28.293634+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289054+0000","last_change":"2023-10-18T08:58:28.289054+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289054+0000","last_undegraded":"2023-10-18T08:58:28.289054+0000","last_fullsized":"2023-10-18T08:58:28.289054+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.285983+0000","last_change":"2023-10-18T08:58:28.285983+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.285983+0000","last_undegraded":"2023-10-18T08:58:28.285983+0000","last_fullsized":"2023-10-18T08:58:28.285983+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326448+0000","last_change":"2023-10-18T08:58:29.325090+0000","last_active":"2023-10-18T08:58:29.326448+0000","last_peered":"2023-10-18T08:58:29.326448+0000","last_clean":"2023-10-18T08:58:29.326448+0000","last_became_active":"2023-10-18T08:58:29.323675+0000","last_became_peered":"2023-10-18T08:58:29.323675+0000","last_unstale":"2023-10-18T08:58:29.326448+0000","last_undegraded":"2023-10-18T08:58:29.326448+0000","last_fullsized":"2023-10-18T08:58:29.326448+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:01:55.373464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.329948+0000","last_change":"2023-10-18T08:58:29.329020+0000","last_active":"2023-10-18T08:58:29.329948+0000","last_peered":"2023-10-18T08:58:29.329948+0000","last_clean":"2023-10-18T08:58:29.329948+0000","last_became_active":"2023-10-18T08:58:29.328773+0000","last_became_peered":"2023-10-18T08:58:29.328773+0000","last_unstale":"2023-10-18T08:58:29.329948+0000","last_undegraded":"2023-10-18T08:58:29.329948+0000","last_fullsized":"2023-10-18T08:58:29.329948+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:57.274523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289622+0000","last_change":"2023-10-18T08:58:28.289622+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289622+0000","last_undegraded":"2023-10-18T08:58:28.289622+0000","last_fullsized":"2023-10-18T08:58:28.289622+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.330756+0000","last_change":"2023-10-18T08:58:29.330497+0000","last_active":"2023-10-18T08:58:29.330756+0000","last_peered":"2023-10-18T08:58:29.330756+0000","last_clean":"2023-10-18T08:58:29.330756+0000","last_became_active":"2023-10-18T08:58:29.330245+0000","last_became_peered":"2023-10-18T08:58:29.330245+0000","last_unstale":"2023-10-18T08:58:29.330756+0000","last_undegraded":"2023-10-18T08:58:29.330756+0000","last_fullsized":"2023-10-18T08:58:29.330756+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:23.135334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.323823+0000","last_change":"2023-10-18T08:58:29.323242+0000","last_active":"2023-10-18T08:58:29.323823+0000","last_peered":"2023-10-18T08:58:29.323823+0000","last_clean":"2023-10-18T08:58:29.323823+0000","last_became_active":"2023-10-18T08:58:29.322832+0000","last_became_peered":"2023-10-18T08:58:29.322832+0000","last_unstale":"2023-10-18T08:58:29.323823+0000","last_undegraded":"2023-10-18T08:58:29.323823+0000","last_fullsized":"2023-10-18T08:58:29.323823+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:10:00.029939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326366+0000","last_change":"2023-10-18T08:58:29.319011+0000","last_active":"2023-10-18T08:58:29.326366+0000","last_peered":"2023-10-18T08:58:29.326366+0000","last_clean":"2023-10-18T08:58:29.326366+0000","last_became_active":"2023-10-18T08:58:29.318536+0000","last_became_peered":"2023-10-18T08:58:29.318536+0000","last_unstale":"2023-10-18T08:58:29.326366+0000","last_undegraded":"2023-10-18T08:58:29.326366+0000","last_fullsized":"2023-10-18T08:58:29.326366+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:16.123758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.319916+0000","last_change":"2023-10-18T08:58:29.319576+0000","last_active":"2023-10-18T08:58:29.319916+0000","last_peered":"2023-10-18T08:58:29.319916+0000","last_clean":"2023-10-18T08:58:29.319916+0000","last_became_active":"2023-10-18T08:58:29.316842+0000","last_became_peered":"2023-10-18T08:58:29.316842+0000","last_unstale":"2023-10-18T08:58:29.319916+0000","last_undegraded":"2023-10-18T08:58:29.319916+0000","last_fullsized":"2023-10-18T08:58:29.319916+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:45.620650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.281482+0000","last_change":"2023-10-18T08:58:28.281482+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.281482+0000","last_undegraded":"2023-10-18T08:58:28.281482+0000","last_fullsized":"2023-10-18T08:58:28.281482+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.291034+0000","last_change":"2023-10-18T08:58:28.291034+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.291034+0000","last_undegraded":"2023-10-18T08:58:28.291034+0000","last_fullsized":"2023-10-18T08:58:28.291034+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.291241+0000","last_change":"2023-10-18T08:58:28.291241+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.291241+0000","last_undegraded":"2023-10-18T08:58:28.291241+0000","last_fullsized":"2023-10-18T08:58:28.291241+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.285161+0000","last_change":"2023-10-18T08:58:28.285161+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.285161+0000","last_undegraded":"2023-10-18T08:58:28.285161+0000","last_fullsized":"2023-10-18T08:58:28.285161+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289003+0000","last_change":"2023-10-18T08:58:28.289003+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289003+0000","last_undegraded":"2023-10-18T08:58:28.289003+0000","last_fullsized":"2023-10-18T08:58:28.289003+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.329742+0000","last_change":"2023-10-18T08:58:29.329658+0000","last_active":"2023-10-18T08:58:29.329742+0000","last_peered":"2023-10-18T08:58:29.329742+0000","last_clean":"2023-10-18T08:58:29.329742+0000","last_became_active":"2023-10-18T08:58:29.329393+0000","last_became_peered":"2023-10-18T08:58:29.329393+0000","last_unstale":"2023-10-18T08:58:29.329742+0000","last_undegraded":"2023-10-18T08:58:29.329742+0000","last_fullsized":"2023-10-18T08:58:29.329742+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:28:25.454956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326242+0000","last_change":"2023-10-18T08:58:29.326075+0000","last_active":"2023-10-18T08:58:29.326242+0000","last_peered":"2023-10-18T08:58:29.326242+0000","last_clean":"2023-10-18T08:58:29.326242+0000","last_became_active":"2023-10-18T08:58:29.324858+0000","last_became_peered":"2023-10-18T08:58:29.324858+0000","last_unstale":"2023-10-18T08:58:29.326242+0000","last_undegraded":"2023-10-18T08:58:29.326242+0000","last_fullsized":"2023-10-18T08:58:29.326242+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:49.015165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289440+0000","last_change":"2023-10-18T08:58:28.289440+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289440+0000","last_undegraded":"2023-10-18T08:58:28.289440+0000","last_fullsized":"2023-10-18T08:58:28.289440+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.284606+0000","last_change":"2023-10-18T08:58:28.284606+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.284606+0000","last_undegraded":"2023-10-18T08:58:28.284606+0000","last_fullsized":"2023-10-18T08:58:28.284606+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.288895+0000","last_change":"2023-10-18T08:58:28.288895+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.288895+0000","last_undegraded":"2023-10-18T08:58:28.288895+0000","last_fullsized":"2023-10-18T08:58:28.288895+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.301110+0000","last_change":"2023-10-18T08:58:28.300855+0000","last_active":"2023-10-18T08:58:28.301110+0000","last_peered":"2023-10-18T08:58:28.301110+0000","last_clean":"2023-10-18T08:58:28.301110+0000","last_became_active":"2023-10-18T08:58:28.300672+0000","last_became_peered":"2023-10-18T08:58:28.300672+0000","last_unstale":"2023-10-18T08:58:28.301110+0000","last_undegraded":"2023-10-18T08:58:28.301110+0000","last_fullsized":"2023-10-18T08:58:28.301110+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:57:22.719753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.277530+0000","last_change":"2023-10-18T08:58:28.277530+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.277530+0000","last_undegraded":"2023-10-18T08:58:28.277530+0000","last_fullsized":"2023-10-18T08:58:28.277530+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"66'1","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.342802+0000","last_change":"2023-10-18T08:58:28.295079+0000","last_active":"2023-10-18T08:58:29.342802+0000","last_peered":"2023-10-18T08:58:29.342802+0000","last_clean":"2023-10-18T08:58:29.342802+0000","last_became_active":"2023-10-18T08:58:28.294485+0000","last_became_peered":"2023-10-18T08:58:28.294485+0000","last_unstale":"2023-10-18T08:58:29.342802+0000","last_undegraded":"2023-10-18T08:58:29.342802+0000","last_fullsized":"2023-10-18T08:58:29.342802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:27.603595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.286466+0000","last_change":"2023-10-18T08:58:28.286466+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.286466+0000","last_undegraded":"2023-10-18T08:58:28.286466+0000","last_fullsized":"2023-10-18T08:58:28.286466+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.299117+0000","last_change":"2023-10-18T08:58:28.298521+0000","last_active":"2023-10-18T08:58:28.299117+0000","last_peered":"2023-10-18T08:58:28.299117+0000","last_clean":"2023-10-18T08:58:28.299117+0000","last_became_active":"2023-10-18T08:58:28.296841+0000","last_became_peered":"2023-10-18T08:58:28.296841+0000","last_unstale":"2023-10-18T08:58:28.299117+0000","last_undegraded":"2023-10-18T08:58:28.299117+0000","last_fullsized":"2023-10-18T08:58:28.299117+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:24.568573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289868+0000","last_change":"2023-10-18T08:58:28.289868+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289868+0000","last_undegraded":"2023-10-18T08:58:28.289868+0000","last_fullsized":"2023-10-18T08:58:28.289868+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.299261+0000","last_change":"2023-10-18T08:58:28.298876+0000","last_active":"2023-10-18T08:58:28.299261+0000","last_peered":"2023-10-18T08:58:28.299261+0000","last_clean":"2023-10-18T08:58:28.299261+0000","last_became_active":"2023-10-18T08:58:28.297696+0000","last_became_peered":"2023-10-18T08:58:28.297696+0000","last_unstale":"2023-10-18T08:58:28.299261+0000","last_undegraded":"2023-10-18T08:58:28.299261+0000","last_fullsized":"2023-10-18T08:58:28.299261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:07:47.505124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326799+0000","last_change":"2023-10-18T08:58:29.325300+0000","last_active":"2023-10-18T08:58:29.326799+0000","last_peered":"2023-10-18T08:58:29.326799+0000","last_clean":"2023-10-18T08:58:29.326799+0000","last_became_active":"2023-10-18T08:58:29.323662+0000","last_became_peered":"2023-10-18T08:58:29.323662+0000","last_unstale":"2023-10-18T08:58:29.326799+0000","last_undegraded":"2023-10-18T08:58:29.326799+0000","last_fullsized":"2023-10-18T08:58:29.326799+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:39:03.787381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300750+0000","last_change":"2023-10-18T08:58:28.299820+0000","last_active":"2023-10-18T08:58:28.300750+0000","last_peered":"2023-10-18T08:58:28.300750+0000","last_clean":"2023-10-18T08:58:28.300750+0000","last_became_active":"2023-10-18T08:58:28.299113+0000","last_became_peered":"2023-10-18T08:58:28.299113+0000","last_unstale":"2023-10-18T08:58:28.300750+0000","last_undegraded":"2023-10-18T08:58:28.300750+0000","last_fullsized":"2023-10-18T08:58:28.300750+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:59.489387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.332542+0000","last_change":"2023-10-18T08:58:29.332297+0000","last_active":"2023-10-18T08:58:29.332542+0000","last_peered":"2023-10-18T08:58:29.332542+0000","last_clean":"2023-10-18T08:58:29.332542+0000","last_became_active":"2023-10-18T08:58:29.332051+0000","last_became_peered":"2023-10-18T08:58:29.332051+0000","last_unstale":"2023-10-18T08:58:29.332542+0000","last_undegraded":"2023-10-18T08:58:29.332542+0000","last_fullsized":"2023-10-18T08:58:29.332542+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:30:53.581720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300168+0000","last_change":"2023-10-18T08:58:28.300048+0000","last_active":"2023-10-18T08:58:28.300168+0000","last_peered":"2023-10-18T08:58:28.300168+0000","last_clean":"2023-10-18T08:58:28.300168+0000","last_became_active":"2023-10-18T08:58:28.299438+0000","last_became_peered":"2023-10-18T08:58:28.299438+0000","last_unstale":"2023-10-18T08:58:28.300168+0000","last_undegraded":"2023-10-18T08:58:28.300168+0000","last_fullsized":"2023-10-18T08:58:28.300168+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:53.262221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.281849+0000","last_change":"2023-10-18T08:58:28.281849+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.281849+0000","last_undegraded":"2023-10-18T08:58:28.281849+0000","last_fullsized":"2023-10-18T08:58:28.281849+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.301494+0000","last_change":"2023-10-18T08:58:28.301417+0000","last_active":"2023-10-18T08:58:28.301494+0000","last_peered":"2023-10-18T08:58:28.301494+0000","last_clean":"2023-10-18T08:58:28.301494+0000","last_became_active":"2023-10-18T08:58:28.300999+0000","last_became_peered":"2023-10-18T08:58:28.300999+0000","last_unstale":"2023-10-18T08:58:28.301494+0000","last_undegraded":"2023-10-18T08:58:28.301494+0000","last_fullsized":"2023-10-18T08:58:28.301494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:37:30.254470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326387+0000","last_change":"2023-10-18T08:58:29.321607+0000","last_active":"2023-10-18T08:58:29.326387+0000","last_peered":"2023-10-18T08:58:29.326387+0000","last_clean":"2023-10-18T08:58:29.326387+0000","last_became_active":"2023-10-18T08:58:29.321062+0000","last_became_peered":"2023-10-18T08:58:29.321062+0000","last_unstale":"2023-10-18T08:58:29.326387+0000","last_undegraded":"2023-10-18T08:58:29.326387+0000","last_fullsized":"2023-10-18T08:58:29.326387+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:20.541018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.302195+0000","last_change":"2023-10-18T08:58:28.302109+0000","last_active":"2023-10-18T08:58:28.302195+0000","last_peered":"2023-10-18T08:58:28.302195+0000","last_clean":"2023-10-18T08:58:28.302195+0000","last_became_active":"2023-10-18T08:58:28.301703+0000","last_became_peered":"2023-10-18T08:58:28.301703+0000","last_unstale":"2023-10-18T08:58:28.302195+0000","last_undegraded":"2023-10-18T08:58:28.302195+0000","last_fullsized":"2023-10-18T08:58:28.302195+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:57.346205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289722+0000","last_change":"2023-10-18T08:58:28.289722+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289722+0000","last_undegraded":"2023-10-18T08:58:28.289722+0000","last_fullsized":"2023-10-18T08:58:28.289722+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.305138+0000","last_change":"2023-10-18T08:58:28.298815+0000","last_active":"2023-10-18T08:58:29.305138+0000","last_peered":"2023-10-18T08:58:29.305138+0000","last_clean":"2023-10-18T08:58:29.305138+0000","last_became_active":"2023-10-18T08:58:28.298503+0000","last_became_peered":"2023-10-18T08:58:28.298503+0000","last_unstale":"2023-10-18T08:58:29.305138+0000","last_undegraded":"2023-10-18T08:58:29.305138+0000","last_fullsized":"2023-10-18T08:58:29.305138+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:39.750254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313431+0000","last_change":"2023-10-18T08:58:29.312112+0000","last_active":"2023-10-18T08:58:29.313431+0000","last_peered":"2023-10-18T08:58:29.313431+0000","last_clean":"2023-10-18T08:58:29.313431+0000","last_became_active":"2023-10-18T08:58:29.310708+0000","last_became_peered":"2023-10-18T08:58:29.310708+0000","last_unstale":"2023-10-18T08:58:29.313431+0000","last_undegraded":"2023-10-18T08:58:29.313431+0000","last_fullsized":"2023-10-18T08:58:29.313431+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:24.996549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.306615+0000","last_change":"2023-10-18T08:58:28.298222+0000","last_active":"2023-10-18T08:58:29.306615+0000","last_peered":"2023-10-18T08:58:29.306615+0000","last_clean":"2023-10-18T08:58:29.306615+0000","last_became_active":"2023-10-18T08:58:28.297672+0000","last_became_peered":"2023-10-18T08:58:28.297672+0000","last_unstale":"2023-10-18T08:58:29.306615+0000","last_undegraded":"2023-10-18T08:58:29.306615+0000","last_fullsized":"2023-10-18T08:58:29.306615+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:59.505404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.277176+0000","last_change":"2023-10-18T08:58:28.277176+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.277176+0000","last_undegraded":"2023-10-18T08:58:28.277176+0000","last_fullsized":"2023-10-18T08:58:28.277176+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"66'3","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.344696+0000","last_change":"2023-10-18T08:58:28.290903+0000","last_active":"2023-10-18T08:58:29.344696+0000","last_peered":"2023-10-18T08:58:29.344696+0000","last_clean":"2023-10-18T08:58:29.344696+0000","last_became_active":"2023-10-18T08:58:28.290358+0000","last_became_peered":"2023-10-18T08:58:28.290358+0000","last_unstale":"2023-10-18T08:58:29.344696+0000","last_undegraded":"2023-10-18T08:58:29.344696+0000","last_fullsized":"2023-10-18T08:58:29.344696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:24.568573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.320694+0000","last_change":"2023-10-18T08:58:29.320149+0000","last_active":"2023-10-18T08:58:29.320694+0000","last_peered":"2023-10-18T08:58:29.320694+0000","last_clean":"2023-10-18T08:58:29.320694+0000","last_became_active":"2023-10-18T08:58:29.317356+0000","last_became_peered":"2023-10-18T08:58:29.317356+0000","last_unstale":"2023-10-18T08:58:29.320694+0000","last_undegraded":"2023-10-18T08:58:29.320694+0000","last_fullsized":"2023-10-18T08:58:29.320694+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:49.888147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.302112+0000","last_change":"2023-10-18T08:58:28.302001+0000","last_active":"2023-10-18T08:58:28.302112+0000","last_peered":"2023-10-18T08:58:28.302112+0000","last_clean":"2023-10-18T08:58:28.302112+0000","last_became_active":"2023-10-18T08:58:28.301382+0000","last_became_peered":"2023-10-18T08:58:28.301382+0000","last_unstale":"2023-10-18T08:58:28.302112+0000","last_undegraded":"2023-10-18T08:58:28.302112+0000","last_fullsized":"2023-10-18T08:58:28.302112+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:43:35.327448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.290401+0000","last_change":"2023-10-18T08:58:28.290401+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.290401+0000","last_undegraded":"2023-10-18T08:58:28.290401+0000","last_fullsized":"2023-10-18T08:58:28.290401+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"66'2","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341261+0000","last_change":"2023-10-18T08:58:28.296421+0000","last_active":"2023-10-18T08:58:29.341261+0000","last_peered":"2023-10-18T08:58:29.341261+0000","last_clean":"2023-10-18T08:58:29.341261+0000","last_became_active":"2023-10-18T08:58:28.296176+0000","last_became_peered":"2023-10-18T08:58:28.296176+0000","last_unstale":"2023-10-18T08:58:29.341261+0000","last_undegraded":"2023-10-18T08:58:29.341261+0000","last_fullsized":"2023-10-18T08:58:29.341261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:14:17.772896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.324023+0000","last_change":"2023-10-18T08:58:29.322440+0000","last_active":"2023-10-18T08:58:29.324023+0000","last_peered":"2023-10-18T08:58:29.324023+0000","last_clean":"2023-10-18T08:58:29.324023+0000","last_became_active":"2023-10-18T08:58:29.319010+0000","last_became_peered":"2023-10-18T08:58:29.319010+0000","last_unstale":"2023-10-18T08:58:29.324023+0000","last_undegraded":"2023-10-18T08:58:29.324023+0000","last_fullsized":"2023-10-18T08:58:29.324023+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:05.722853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.297280+0000","last_change":"2023-10-18T08:58:28.290878+0000","last_active":"2023-10-18T08:58:28.297280+0000","last_peered":"2023-10-18T08:58:28.297280+0000","last_clean":"2023-10-18T08:58:28.297280+0000","last_became_active":"2023-10-18T08:58:28.290244+0000","last_became_peered":"2023-10-18T08:58:28.290244+0000","last_unstale":"2023-10-18T08:58:28.297280+0000","last_undegraded":"2023-10-18T08:58:28.297280+0000","last_fullsized":"2023-10-18T08:58:28.297280+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:01.096478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.288004+0000","last_change":"2023-10-18T08:58:28.288004+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.288004+0000","last_undegraded":"2023-10-18T08:58:28.288004+0000","last_fullsized":"2023-10-18T08:58:28.288004+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300048+0000","last_change":"2023-10-18T08:58:28.299958+0000","last_active":"2023-10-18T08:58:28.300048+0000","last_peered":"2023-10-18T08:58:28.300048+0000","last_clean":"2023-10-18T08:58:28.300048+0000","last_became_active":"2023-10-18T08:58:28.299673+0000","last_became_peered":"2023-10-18T08:58:28.299673+0000","last_unstale":"2023-10-18T08:58:28.300048+0000","last_undegraded":"2023-10-18T08:58:28.300048+0000","last_fullsized":"2023-10-18T08:58:28.300048+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:08.788078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313474+0000","last_change":"2023-10-18T08:58:29.313209+0000","last_active":"2023-10-18T08:58:29.313474+0000","last_peered":"2023-10-18T08:58:29.313474+0000","last_clean":"2023-10-18T08:58:29.313474+0000","last_became_active":"2023-10-18T08:58:29.312599+0000","last_became_peered":"2023-10-18T08:58:29.312599+0000","last_unstale":"2023-10-18T08:58:29.313474+0000","last_undegraded":"2023-10-18T08:58:29.313474+0000","last_fullsized":"2023-10-18T08:58:29.313474+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:33:04.626358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"66'8","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.343659+0000","last_change":"2023-10-18T08:58:28.297447+0000","last_active":"2023-10-18T08:58:29.343659+0000","last_peered":"2023-10-18T08:58:29.343659+0000","last_clean":"2023-10-18T08:58:29.343659+0000","last_became_active":"2023-10-18T08:58:28.297055+0000","last_became_peered":"2023-10-18T08:58:28.297055+0000","last_unstale":"2023-10-18T08:58:29.343659+0000","last_undegraded":"2023-10-18T08:58:29.343659+0000","last_fullsized":"2023-10-18T08:58:29.343659+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:57:22.719753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326347+0000","last_change":"2023-10-18T08:58:29.326263+0000","last_active":"2023-10-18T08:58:29.326347+0000","last_peered":"2023-10-18T08:58:29.326347+0000","last_clean":"2023-10-18T08:58:29.326347+0000","last_became_active":"2023-10-18T08:58:29.325851+0000","last_became_peered":"2023-10-18T08:58:29.325851+0000","last_unstale":"2023-10-18T08:58:29.326347+0000","last_undegraded":"2023-10-18T08:58:29.326347+0000","last_fullsized":"2023-10-18T08:58:29.326347+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:42:38.047653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"66'1","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.339858+0000","last_change":"2023-10-18T08:58:28.300004+0000","last_active":"2023-10-18T08:58:29.339858+0000","last_peered":"2023-10-18T08:58:29.339858+0000","last_clean":"2023-10-18T08:58:29.339858+0000","last_became_active":"2023-10-18T08:58:28.294830+0000","last_became_peered":"2023-10-18T08:58:28.294830+0000","last_unstale":"2023-10-18T08:58:29.339858+0000","last_undegraded":"2023-10-18T08:58:29.339858+0000","last_fullsized":"2023-10-18T08:58:29.339858+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:16.750986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.293219+0000","last_change":"2023-10-18T08:58:28.293219+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.293219+0000","last_undegraded":"2023-10-18T08:58:28.293219+0000","last_fullsized":"2023-10-18T08:58:28.293219+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.298454+0000","last_change":"2023-10-18T08:58:28.298360+0000","last_active":"2023-10-18T08:58:28.298454+0000","last_peered":"2023-10-18T08:58:28.298454+0000","last_clean":"2023-10-18T08:58:28.298454+0000","last_became_active":"2023-10-18T08:58:28.296520+0000","last_became_peered":"2023-10-18T08:58:28.296520+0000","last_unstale":"2023-10-18T08:58:28.298454+0000","last_undegraded":"2023-10-18T08:58:28.298454+0000","last_fullsized":"2023-10-18T08:58:28.298454+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:48.874512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326429+0000","last_change":"2023-10-18T08:58:29.324872+0000","last_active":"2023-10-18T08:58:29.326429+0000","last_peered":"2023-10-18T08:58:29.326429+0000","last_clean":"2023-10-18T08:58:29.326429+0000","last_became_active":"2023-10-18T08:58:29.323103+0000","last_became_peered":"2023-10-18T08:58:29.323103+0000","last_unstale":"2023-10-18T08:58:29.326429+0000","last_undegraded":"2023-10-18T08:58:29.326429+0000","last_fullsized":"2023-10-18T08:58:29.326429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:11:33.988349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.299240+0000","last_change":"2023-10-18T08:58:28.298668+0000","last_active":"2023-10-18T08:58:28.299240+0000","last_peered":"2023-10-18T08:58:28.299240+0000","last_clean":"2023-10-18T08:58:28.299240+0000","last_became_active":"2023-10-18T08:58:28.297217+0000","last_became_peered":"2023-10-18T08:58:28.297217+0000","last_unstale":"2023-10-18T08:58:28.299240+0000","last_undegraded":"2023-10-18T08:58:28.299240+0000","last_fullsized":"2023-10-18T08:58:28.299240+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:37.673296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313511+0000","last_change":"2023-10-18T08:58:29.313231+0000","last_active":"2023-10-18T08:58:29.313511+0000","last_peered":"2023-10-18T08:58:29.313511+0000","last_clean":"2023-10-18T08:58:29.313511+0000","last_became_active":"2023-10-18T08:58:29.311497+0000","last_became_peered":"2023-10-18T08:58:29.311497+0000","last_unstale":"2023-10-18T08:58:29.313511+0000","last_undegraded":"2023-10-18T08:58:29.313511+0000","last_fullsized":"2023-10-18T08:58:29.313511+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:54.275856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.298061+0000","last_change":"2023-10-18T08:58:28.297688+0000","last_active":"2023-10-18T08:58:28.298061+0000","last_peered":"2023-10-18T08:58:28.298061+0000","last_clean":"2023-10-18T08:58:28.298061+0000","last_became_active":"2023-10-18T08:58:28.297401+0000","last_became_peered":"2023-10-18T08:58:28.297401+0000","last_unstale":"2023-10-18T08:58:28.298061+0000","last_undegraded":"2023-10-18T08:58:28.298061+0000","last_fullsized":"2023-10-18T08:58:28.298061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:44.607015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.292556+0000","last_change":"2023-10-18T08:58:28.292556+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.292556+0000","last_undegraded":"2023-10-18T08:58:28.292556+0000","last_fullsized":"2023-10-18T08:58:28.292556+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"66'2","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341374+0000","last_change":"2023-10-18T08:58:28.299131+0000","last_active":"2023-10-18T08:58:29.341374+0000","last_peered":"2023-10-18T08:58:29.341374+0000","last_clean":"2023-10-18T08:58:29.341374+0000","last_became_active":"2023-10-18T08:58:28.298514+0000","last_became_peered":"2023-10-18T08:58:28.298514+0000","last_unstale":"2023-10-18T08:58:29.341374+0000","last_undegraded":"2023-10-18T08:58:29.341374+0000","last_fullsized":"2023-10-18T08:58:29.341374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:42:24.333152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.294168+0000","last_change":"2023-10-18T08:58:28.294168+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.294168+0000","last_undegraded":"2023-10-18T08:58:28.294168+0000","last_fullsized":"2023-10-18T08:58:28.294168+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.295059+0000","last_change":"2023-10-18T08:58:28.294327+0000","last_active":"2023-10-18T08:58:28.295059+0000","last_peered":"2023-10-18T08:58:28.295059+0000","last_clean":"2023-10-18T08:58:28.295059+0000","last_became_active":"2023-10-18T08:58:28.293852+0000","last_became_peered":"2023-10-18T08:58:28.293852+0000","last_unstale":"2023-10-18T08:58:28.295059+0000","last_undegraded":"2023-10-18T08:58:28.295059+0000","last_fullsized":"2023-10-18T08:58:28.295059+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:17:54.342972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289360+0000","last_change":"2023-10-18T08:58:28.289360+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289360+0000","last_undegraded":"2023-10-18T08:58:28.289360+0000","last_fullsized":"2023-10-18T08:58:28.289360+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.302241+0000","last_change":"2023-10-18T08:58:28.302115+0000","last_active":"2023-10-18T08:58:28.302241+0000","last_peered":"2023-10-18T08:58:28.302241+0000","last_clean":"2023-10-18T08:58:28.302241+0000","last_became_active":"2023-10-18T08:58:28.300920+0000","last_became_peered":"2023-10-18T08:58:28.300920+0000","last_unstale":"2023-10-18T08:58:28.302241+0000","last_undegraded":"2023-10-18T08:58:28.302241+0000","last_fullsized":"2023-10-18T08:58:28.302241+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:26:14.774552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289377+0000","last_change":"2023-10-18T08:58:28.289377+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289377+0000","last_undegraded":"2023-10-18T08:58:28.289377+0000","last_fullsized":"2023-10-18T08:58:28.289377+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.280714+0000","last_change":"2023-10-18T08:58:28.280587+0000","last_active":"2023-10-18T08:58:28.280714+0000","last_peered":"2023-10-18T08:58:28.280714+0000","last_clean":"2023-10-18T08:58:28.280714+0000","last_became_active":"2023-10-18T08:58:28.280261+0000","last_became_peered":"2023-10-18T08:58:28.280261+0000","last_unstale":"2023-10-18T08:58:28.280714+0000","last_undegraded":"2023-10-18T08:58:28.280714+0000","last_fullsized":"2023-10-18T08:58:28.280714+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:56.134357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.289491+0000","last_change":"2023-10-18T08:58:28.289491+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.289491+0000","last_undegraded":"2023-10-18T08:58:28.289491+0000","last_fullsized":"2023-10-18T08:58:28.289491+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.299921+0000","last_change":"2023-10-18T08:58:28.299846+0000","last_active":"2023-10-18T08:58:28.299921+0000","last_peered":"2023-10-18T08:58:28.299921+0000","last_clean":"2023-10-18T08:58:28.299921+0000","last_became_active":"2023-10-18T08:58:28.299121+0000","last_became_peered":"2023-10-18T08:58:28.299121+0000","last_unstale":"2023-10-18T08:58:28.299921+0000","last_undegraded":"2023-10-18T08:58:28.299921+0000","last_fullsized":"2023-10-18T08:58:28.299921+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:23.982914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.277537+0000","last_change":"2023-10-18T08:58:28.277537+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.277537+0000","last_undegraded":"2023-10-18T08:58:28.277537+0000","last_fullsized":"2023-10-18T08:58:28.277537+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300837+0000","last_change":"2023-10-18T08:58:28.300674+0000","last_active":"2023-10-18T08:58:28.300837+0000","last_peered":"2023-10-18T08:58:28.300837+0000","last_clean":"2023-10-18T08:58:28.300837+0000","last_became_active":"2023-10-18T08:58:28.300119+0000","last_became_peered":"2023-10-18T08:58:28.300119+0000","last_unstale":"2023-10-18T08:58:28.300837+0000","last_undegraded":"2023-10-18T08:58:28.300837+0000","last_fullsized":"2023-10-18T08:58:28.300837+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:35.906463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.284498+0000","last_change":"2023-10-18T08:58:28.284498+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.284498+0000","last_undegraded":"2023-10-18T08:58:28.284498+0000","last_fullsized":"2023-10-18T08:58:28.284498+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.303285+0000","last_change":"2023-10-18T08:58:28.303209+0000","last_active":"2023-10-18T08:58:28.303285+0000","last_peered":"2023-10-18T08:58:28.303285+0000","last_clean":"2023-10-18T08:58:28.303285+0000","last_became_active":"2023-10-18T08:58:28.302960+0000","last_became_peered":"2023-10-18T08:58:28.302960+0000","last_unstale":"2023-10-18T08:58:28.303285+0000","last_undegraded":"2023-10-18T08:58:28.303285+0000","last_fullsized":"2023-10-18T08:58:28.303285+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:50.826443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.316386+0000","last_change":"2023-10-18T08:58:29.315776+0000","last_active":"2023-10-18T08:58:29.316386+0000","last_peered":"2023-10-18T08:58:29.316386+0000","last_clean":"2023-10-18T08:58:29.316386+0000","last_became_active":"2023-10-18T08:58:29.315154+0000","last_became_peered":"2023-10-18T08:58:29.315154+0000","last_unstale":"2023-10-18T08:58:29.316386+0000","last_undegraded":"2023-10-18T08:58:29.316386+0000","last_fullsized":"2023-10-18T08:58:29.316386+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:51.344108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.296149+0000","last_change":"2023-10-18T08:58:28.295800+0000","last_active":"2023-10-18T08:58:28.296149+0000","last_peered":"2023-10-18T08:58:28.296149+0000","last_clean":"2023-10-18T08:58:28.296149+0000","last_became_active":"2023-10-18T08:58:28.295520+0000","last_became_peered":"2023-10-18T08:58:28.295520+0000","last_unstale":"2023-10-18T08:58:28.296149+0000","last_undegraded":"2023-10-18T08:58:28.296149+0000","last_fullsized":"2023-10-18T08:58:28.296149+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:01.809009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313850+0000","last_change":"2023-10-18T08:58:29.313510+0000","last_active":"2023-10-18T08:58:29.313850+0000","last_peered":"2023-10-18T08:58:29.313850+0000","last_clean":"2023-10-18T08:58:29.313850+0000","last_became_active":"2023-10-18T08:58:29.312697+0000","last_became_peered":"2023-10-18T08:58:29.312697+0000","last_unstale":"2023-10-18T08:58:29.313850+0000","last_undegraded":"2023-10-18T08:58:29.313850+0000","last_fullsized":"2023-10-18T08:58:29.313850+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:47.848978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300768+0000","last_change":"2023-10-18T08:58:28.300006+0000","last_active":"2023-10-18T08:58:28.300768+0000","last_peered":"2023-10-18T08:58:28.300768+0000","last_clean":"2023-10-18T08:58:28.300768+0000","last_became_active":"2023-10-18T08:58:28.299510+0000","last_became_peered":"2023-10-18T08:58:28.299510+0000","last_unstale":"2023-10-18T08:58:28.300768+0000","last_undegraded":"2023-10-18T08:58:28.300768+0000","last_fullsized":"2023-10-18T08:58:28.300768+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:01.392655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2023-10-18T08:58:28.276744+0000","last_change":"2023-10-18T08:58:28.276744+0000","last_active":"2023-10-18T08:58:28.264871+0000","last_peered":"2023-10-18T08:58:28.264871+0000","last_clean":"2023-10-18T08:58:28.264871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T08:58:28.276744+0000","last_undegraded":"2023-10-18T08:58:28.276744+0000","last_fullsized":"2023-10-18T08:58:28.276744+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.298217+0000","last_change":"2023-10-18T08:58:28.297980+0000","last_active":"2023-10-18T08:58:28.298217+0000","last_peered":"2023-10-18T08:58:28.298217+0000","last_clean":"2023-10-18T08:58:28.298217+0000","last_became_active":"2023-10-18T08:58:28.297743+0000","last_became_peered":"2023-10-18T08:58:28.297743+0000","last_unstale":"2023-10-18T08:58:28.298217+0000","last_undegraded":"2023-10-18T08:58:28.298217+0000","last_fullsized":"2023-10-18T08:58:28.298217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:40:01.186029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.318289+0000","last_change":"2023-10-18T08:58:29.318110+0000","last_active":"2023-10-18T08:58:29.318289+0000","last_peered":"2023-10-18T08:58:29.318289+0000","last_clean":"2023-10-18T08:58:29.318289+0000","last_became_active":"2023-10-18T08:58:29.317066+0000","last_became_peered":"2023-10-18T08:58:29.317066+0000","last_unstale":"2023-10-18T08:58:29.318289+0000","last_undegraded":"2023-10-18T08:58:29.318289+0000","last_fullsized":"2023-10-18T08:58:29.318289+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:54.004578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.299504+0000","last_change":"2023-10-18T08:58:28.300829+0000","last_active":"2023-10-18T08:58:29.299504+0000","last_peered":"2023-10-18T08:58:29.299504+0000","last_clean":"2023-10-18T08:58:29.299504+0000","last_became_active":"2023-10-18T08:58:28.300515+0000","last_became_peered":"2023-10-18T08:58:28.300515+0000","last_unstale":"2023-10-18T08:58:29.299504+0000","last_undegraded":"2023-10-18T08:58:29.299504+0000","last_fullsized":"2023-10-18T08:58:29.299504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:15:03.965062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.331823+0000","last_change":"2023-10-18T08:58:29.331231+0000","last_active":"2023-10-18T08:58:29.331823+0000","last_peered":"2023-10-18T08:58:29.331823+0000","last_clean":"2023-10-18T08:58:29.331823+0000","last_became_active":"2023-10-18T08:58:29.331003+0000","last_became_peered":"2023-10-18T08:58:29.331003+0000","last_unstale":"2023-10-18T08:58:29.331823+0000","last_undegraded":"2023-10-18T08:58:29.331823+0000","last_fullsized":"2023-10-18T08:58:29.331823+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:48.691583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.300206+0000","last_change":"2023-10-18T08:58:28.300119+0000","last_active":"2023-10-18T08:58:28.300206+0000","last_peered":"2023-10-18T08:58:28.300206+0000","last_clean":"2023-10-18T08:58:28.300206+0000","last_became_active":"2023-10-18T08:58:28.299753+0000","last_became_peered":"2023-10-18T08:58:28.299753+0000","last_unstale":"2023-10-18T08:58:28.300206+0000","last_undegraded":"2023-10-18T08:58:28.300206+0000","last_fullsized":"2023-10-18T08:58:28.300206+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:27.836895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":214,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.275442+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:58:28.275442+0000","last_peered":"2023-10-18T08:58:28.275442+0000","last_clean":"2023-10-18T08:58:28.275442+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:58:28.275442+0000","last_undegraded":"2023-10-18T08:58:28.275442+0000","last_fullsized":"2023-10-18T08:58:28.275442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:53.386532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":146,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":902,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194409,"statfs":{"total":200000000,"available":199075709,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.99299999999999999,"15min":0.98899999999999999},"min":{"1min":0.70299999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3400000000000001,"5min":1.538,"15min":1.538},"last":1.222},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.034,"15min":1.008},"min":{"1min":0.69599999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4019999999999999,"5min":1.667,"15min":1.667},"last":1.048}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1240000000000001,"15min":1.147},"min":{"1min":0.80300000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5900000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.375},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1419999999999999,"15min":1.1220000000000001},"min":{"1min":0.748,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7450000000000001,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.5920000000000001}]},{"osd":2,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.2450000000000001,"15min":1.2330000000000001},"min":{"1min":0.80200000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.284},{"interface":"front","average":{"1min":1.276,"5min":1.2010000000000001,"15min":1.1910000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.798,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.3320000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2629999999999999,"15min":1.264},"min":{"1min":0.95999999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.702,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3140000000000001,"15min":1.3149999999999999},"min":{"1min":0.94199999999999995,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.873,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.478}]},{"osd":4,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1899999999999999,"15min":1.1599999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6799999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.3779999999999999,"15min":1.401},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.6279999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.534}]},{"osd":5,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.353,"15min":1.353},"min":{"1min":0.99399999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.111},{"interface":"front","average":{"1min":1.349,"5min":1.4019999999999999,"15min":1.4159999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.865,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.4430000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.464,"15min":1.458},"min":{"1min":1.0369999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.923,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.518},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.492,"15min":1.51},"min":{"1min":1.0669999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.8069999999999999,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.633}]}]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":326,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194986,"statfs":{"total":200000000,"available":199666077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1020000000000001,"15min":1.0800000000000001},"min":{"1min":0.623,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.651,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.228,"15min":1.234},"min":{"1min":0.84499999999999997,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.613}]},{"osd":1,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.089,"15min":1.085},"min":{"1min":0.65600000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1799999999999999,"15min":1.1859999999999999},"min":{"1min":0.80600000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.962,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":1.2829999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.161,"15min":1.173},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5029999999999999,"5min":1.696,"15min":1.696},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.1970000000000001,"15min":1.194},"min":{"1min":0.92500000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.494}]},{"osd":3,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2509999999999999,"15min":1.2569999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.669,"5min":1.784,"15min":1.784},"last":1.345},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.228,"15min":1.22},"min":{"1min":0.84999999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.526}]},{"osd":4,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.212,"15min":1.175},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.129},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.26,"15min":1.266},"min":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.696,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.4690000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3520000000000001,"15min":1.3480000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.599,"5min":1.667,"15min":1.667},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.232,"5min":1.238,"15min":1.218},"min":{"1min":0.629,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.52,"5min":1.746,"15min":1.746},"last":1.403}]},{"osd":7,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.4039999999999999,"15min":1.3959999999999999},"min":{"1min":0.73999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.51,"15min":1.5249999999999999},"min":{"1min":0.84099999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.831,"5min":2.794,"15min":2.794},"last":1.6479999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":326,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194986,"statfs":{"total":200000000,"available":199666077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.7529999999999999,"5min":1.3160000000000001,"15min":1.224},"min":{"1min":0.68300000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":9.3539999999999992,"5min":9.3539999999999992,"15min":9.3539999999999992},"last":1.258},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.3220000000000001,"15min":1.3029999999999999},"min":{"1min":0.71999999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":5.3029999999999999,"5min":5.3029999999999999,"15min":5.3029999999999999},"last":1.4379999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2090000000000001,"15min":1.2230000000000001},"min":{"1min":0.755,"5min":0.749,"15min":0.749},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.222,"15min":1.248},"min":{"1min":0.82199999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.367,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":1.1739999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.268,"15min":1.268},"min":{"1min":0.78100000000000003,"5min":0.755,"15min":0.755},"max":{"1min":1.5309999999999999,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.292,"5min":1.2889999999999999,"15min":1.2929999999999999},"min":{"1min":0.78400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.3069999999999999,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.0549999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.25,"15min":1.246},"min":{"1min":0.93300000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.524,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.337,"15min":1.3320000000000001},"min":{"1min":0.94399999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.6379999999999999,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.999}]},{"osd":4,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4139999999999999,"15min":1.419},"min":{"1min":0.93400000000000005,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.881,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.4950000000000001,"15min":1.5329999999999999},"min":{"1min":0.90000000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":4.4429999999999996,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":1.349}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.377,"15min":1.4119999999999999},"min":{"1min":0.93899999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.5469999999999997,"5min":5.141,"15min":5.141},"last":1.202},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.556,"15min":1.587},"min":{"1min":1.151,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0590000000000002,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.117}]},{"osd":7,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.6419999999999999,"15min":1.6699999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":4.2050000000000001,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5149999999999999,"15min":1.5249999999999999},"min":{"1min":1.0249999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":4.4690000000000003,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":1.512}]}]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":325,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194987,"statfs":{"total":200000000,"available":199667096,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.107,"15min":1.1100000000000001},"min":{"1min":0.745,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":0.996,"5min":1.129,"15min":1.169},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.6990000000000001,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.1439999999999999}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.4849999999999999,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":1.008},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2250000000000001,"15min":1.2390000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.651}]},{"osd":2,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1879999999999999,"15min":1.179},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.254},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.27,"15min":1.3029999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.994,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.3819999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2889999999999999,"15min":1.3009999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":1.286},{"interface":"front","average":{"1min":1.421,"5min":1.4179999999999999,"15min":1.4139999999999999},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.6880000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.1910000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.323,"15min":1.304},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.778,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.5},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.351,"15min":1.367},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.228}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.4339999999999999,"15min":1.3939999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.556,"5min":1.5629999999999999,"15min":1.5760000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":1.5549999999999999}]},{"osd":7,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.4950000000000001,"15min":1.514},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5529999999999999,"15min":1.5509999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.585,"5min":2.585,"15min":2.585},"last":1.4530000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":326,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194986,"statfs":{"total":200000000,"available":199666077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.85199999999999998,"15min":0.77400000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.304,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.95199999999999996,"15min":0.873},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.393,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":0.46600000000000003}]},{"osd":1,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.97799999999999998,"15min":0.94199999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.208,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.1419999999999999,"15min":1.113},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.5660000000000001,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.258}]},{"osd":2,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.0549999999999999,"15min":1.0149999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4330000000000001,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1080000000000001,"15min":1.054},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6459999999999999,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":1.3149999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.464,"15min":1.3049999999999999},"min":{"1min":0.83599999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.5649999999999999,"5min":18.664999999999999,"15min":18.664999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.21,"15min":1.1499999999999999},"min":{"1min":0.69099999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.756,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":1.0149999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.276,"15min":1.2170000000000001},"min":{"1min":0.72399999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.702,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":0.87},{"interface":"front","average":{"1min":1.425,"5min":1.502,"15min":1.3400000000000001},"min":{"1min":0.80800000000000005,"5min":0.748,"15min":0.748},"max":{"1min":1.885,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.375}]},{"osd":6,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4750000000000001,"15min":1.4199999999999999},"min":{"1min":0.82499999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.7649999999999999,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3959999999999999,"15min":1.3300000000000001},"min":{"1min":0.77200000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.774,"5min":7.242,"15min":7.242},"last":1.103}]},{"osd":7,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.4019999999999999,"15min":1.3320000000000001},"min":{"1min":0.80600000000000005,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.9970000000000001,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.431},{"interface":"front","average":{"1min":1.47,"5min":1.556,"15min":1.4470000000000001},"min":{"1min":0.93100000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.8580000000000001,"5min":12.946,"15min":12.946},"last":0.84199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":319,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194992,"statfs":{"total":200000000,"available":199672344,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90500000000000003,"15min":0.93500000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.55,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":1.4330000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0029999999999999,"15min":1.05},"min":{"1min":0.63100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4870000000000001,"5min":1.651,"15min":1.651},"last":0.75900000000000001}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.0249999999999999,"15min":1.0549999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.427,"5min":4.29,"15min":4.29},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.198,"5min":1.0760000000000001,"15min":1.095},"min":{"1min":0.65700000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.583,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.8140000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0840000000000001,"15min":1.1259999999999999},"min":{"1min":0.54600000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.6419999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.256},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.391,"15min":1.48},"min":{"1min":0.57899999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8360000000000001,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":1.7709999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.448,"15min":1.4950000000000001},"min":{"1min":0.95699999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.7450000000000001,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":1.639},{"interface":"front","average":{"1min":1.421,"5min":1.415,"15min":1.45},"min":{"1min":0.93799999999999994,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.8029999999999999,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.5940000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.234,"15min":1.268},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6719999999999999,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":1.054},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.5,"15min":1.5760000000000001},"min":{"1min":0.96099999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.863,"5min":7.657,"15min":7.657},"last":1.2989999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.472,"15min":1.5309999999999999},"min":{"1min":0.82699999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8999999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.391},{"interface":"front","average":{"1min":3.536,"5min":1.8240000000000001,"15min":1.5720000000000001},"min":{"1min":0.98899999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":37.771999999999998,"5min":37.771999999999998,"15min":37.771999999999998},"last":1.718}]},{"osd":7,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.5309999999999999,"15min":1.573},"min":{"1min":0.98699999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.016,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.4410000000000001,"15min":1.4710000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1349999999999998,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.5269999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":325,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194987,"statfs":{"total":200000000,"available":199667096,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.79200000000000004,"15min":0.79800000000000004},"min":{"1min":0.44500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":1.327,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.999,"5min":0.89300000000000002,"15min":0.89100000000000001},"min":{"1min":0.501,"5min":0.443,"15min":0.443},"max":{"1min":1.5129999999999999,"5min":1.55,"15min":1.55},"last":0.96099999999999997}]},{"osd":2,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.91100000000000003,"15min":0.92300000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6850000000000001,"5min":1.859,"15min":1.859},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.83299999999999996,"15min":0.83499999999999996},"min":{"1min":0.47999999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.625,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.81000000000000005}]},{"osd":3,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.98199999999999998,"15min":1.008},"min":{"1min":0.52800000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.571,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.0229999999999999,"15min":1.0469999999999999},"min":{"1min":0.54500000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.8859999999999999,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":0.89000000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.22,"15min":1.2350000000000001},"min":{"1min":0.82099999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5760000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":1.216},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1970000000000001,"15min":1.1930000000000001},"min":{"1min":0.93799999999999994,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.556,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.6759999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.268,"15min":1.2809999999999999},"min":{"1min":0.86599999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.625,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.367},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.343,"15min":1.343},"min":{"1min":0.90500000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.6759999999999999,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.486}]},{"osd":6,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2529999999999999,"15min":1.2649999999999999},"min":{"1min":0.89100000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6240000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2450000000000001,"15min":1.248},"min":{"1min":0.78900000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.744,"5min":1.853,"15min":1.853},"last":1.1160000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.333,"15min":1.339},"min":{"1min":0.84399999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.716,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.288},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3380000000000001,"15min":1.3400000000000001},"min":{"1min":0.95099999999999996,"5min":0.746,"15min":0.746},"max":{"1min":1.758,"5min":1.994,"15min":1.994},"last":1.4179999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":895,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194416,"statfs":{"total":200000000,"available":199082995,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.89500000000000002,"15min":0.93600000000000005},"min":{"1min":0.371,"5min":0.37,"15min":0.37},"max":{"1min":1.0940000000000001,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":1.163},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0660000000000001,"15min":1.079},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.90300000000000002}]},{"osd":2,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.121,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.873},{"interface":"front","average":{"1min":1.121,"5min":1.1240000000000001,"15min":1.125},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.8109999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.0549999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.22,"15min":1.306},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6639999999999999,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.377,"5min":1.4750000000000001,"15min":1.5309999999999999},"min":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.9259999999999999,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.1180000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2629999999999999,"15min":1.3169999999999999},"min":{"1min":0.81200000000000006,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.5549999999999999,"5min":4.492,"15min":4.492},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.234,"5min":1.2490000000000001,"15min":1.2470000000000001},"min":{"1min":0.93200000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.629,"5min":1.726,"15min":1.726},"last":1.1459999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.377,"15min":1.4490000000000001},"min":{"1min":0.89500000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.944,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3580000000000001,"15min":1.3859999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8240000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.0900000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.5629999999999999,"15min":1.6100000000000001},"min":{"1min":0.877,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.0409999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.4670000000000001,"15min":1.5189999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.863,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.427}]},{"osd":7,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.619,"15min":1.661},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.1160000000000001,"5min":3.492,"15min":3.492},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.621,"15min":1.7130000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.0449999999999999,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.516}]}]}],"pool_statfs":[]}} 2023-10-18T08:58:34.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2023-10-18T08:58:34.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.290178+0000', 'last_change': '2023-10-18T08:58:28.290178+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.290178+0000', 'last_undegraded': '2023-10-18T08:58:28.290178+0000', 'last_fullsized': '2023-10-18T08:58:28.290178+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T08:58:34.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2023-10-18T08:58:34.875 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.293989+0000', 'last_change': '2023-10-18T08:58:28.293989+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.293989+0000', 'last_undegraded': '2023-10-18T08:58:28.293989+0000', 'last_fullsized': '2023-10-18T08:58:28.293989+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:58:34.875 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2023-10-18T08:58:34.876 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.291475+0000', 'last_change': '2023-10-18T08:58:28.291475+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.291475+0000', 'last_undegraded': '2023-10-18T08:58:28.291475+0000', 'last_fullsized': '2023-10-18T08:58:28.291475+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T08:58:34.876 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3a is not active+clean 2023-10-18T08:58:34.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.287056+0000', 'last_change': '2023-10-18T08:58:28.287056+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.287056+0000', 'last_undegraded': '2023-10-18T08:58:28.287056+0000', 'last_fullsized': '2023-10-18T08:58:28.287056+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:58:34.877 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.39 is not active+clean 2023-10-18T08:58:34.877 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.290553+0000', 'last_change': '2023-10-18T08:58:28.290553+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.290553+0000', 'last_undegraded': '2023-10-18T08:58:28.290553+0000', 'last_fullsized': '2023-10-18T08:58:28.290553+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T08:58:34.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2023-10-18T08:58:34.878 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.294163+0000', 'last_change': '2023-10-18T08:58:28.294163+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.294163+0000', 'last_undegraded': '2023-10-18T08:58:28.294163+0000', 'last_fullsized': '2023-10-18T08:58:28.294163+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T08:58:34.878 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2023-10-18T08:58:34.879 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.278822+0000', 'last_change': '2023-10-18T08:58:28.278822+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.278822+0000', 'last_undegraded': '2023-10-18T08:58:28.278822+0000', 'last_fullsized': '2023-10-18T08:58:28.278822+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T08:58:34.879 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.35 is not active+clean 2023-10-18T08:58:34.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.277474+0000', 'last_change': '2023-10-18T08:58:28.277474+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.277474+0000', 'last_undegraded': '2023-10-18T08:58:28.277474+0000', 'last_fullsized': '2023-10-18T08:58:28.277474+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T08:58:34.880 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2023-10-18T08:58:34.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.290401+0000', 'last_change': '2023-10-18T08:58:28.290401+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.290401+0000', 'last_undegraded': '2023-10-18T08:58:28.290401+0000', 'last_fullsized': '2023-10-18T08:58:28.290401+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:58:34.881 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2023-10-18T08:58:34.881 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.283202+0000', 'last_change': '2023-10-18T08:58:28.283202+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.283202+0000', 'last_undegraded': '2023-10-18T08:58:28.283202+0000', 'last_fullsized': '2023-10-18T08:58:28.283202+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T08:58:34.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2023-10-18T08:58:34.882 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.293634+0000', 'last_change': '2023-10-18T08:58:28.293634+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.293634+0000', 'last_undegraded': '2023-10-18T08:58:28.293634+0000', 'last_fullsized': '2023-10-18T08:58:28.293634+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T08:58:34.882 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2023-10-18T08:58:34.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.289054+0000', 'last_change': '2023-10-18T08:58:28.289054+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.289054+0000', 'last_undegraded': '2023-10-18T08:58:28.289054+0000', 'last_fullsized': '2023-10-18T08:58:28.289054+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T08:58:34.883 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.30 is not active+clean 2023-10-18T08:58:34.883 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.285983+0000', 'last_change': '2023-10-18T08:58:28.285983+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.285983+0000', 'last_undegraded': '2023-10-18T08:58:28.285983+0000', 'last_fullsized': '2023-10-18T08:58:28.285983+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:58:34.884 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2d is not active+clean 2023-10-18T08:58:34.884 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.289622+0000', 'last_change': '2023-10-18T08:58:28.289622+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.289622+0000', 'last_undegraded': '2023-10-18T08:58:28.289622+0000', 'last_fullsized': '2023-10-18T08:58:28.289622+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T08:58:34.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2023-10-18T08:58:34.885 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.281482+0000', 'last_change': '2023-10-18T08:58:28.281482+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.281482+0000', 'last_undegraded': '2023-10-18T08:58:28.281482+0000', 'last_fullsized': '2023-10-18T08:58:28.281482+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T08:58:34.885 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.27 is not active+clean 2023-10-18T08:58:34.886 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.291034+0000', 'last_change': '2023-10-18T08:58:28.291034+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.291034+0000', 'last_undegraded': '2023-10-18T08:58:28.291034+0000', 'last_fullsized': '2023-10-18T08:58:28.291034+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T08:58:34.886 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2023-10-18T08:58:34.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.291241+0000', 'last_change': '2023-10-18T08:58:28.291241+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.291241+0000', 'last_undegraded': '2023-10-18T08:58:28.291241+0000', 'last_fullsized': '2023-10-18T08:58:28.291241+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:58:34.887 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.25 is not active+clean 2023-10-18T08:58:34.887 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.285161+0000', 'last_change': '2023-10-18T08:58:28.285161+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.285161+0000', 'last_undegraded': '2023-10-18T08:58:28.285161+0000', 'last_fullsized': '2023-10-18T08:58:28.285161+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:58:34.888 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2023-10-18T08:58:34.888 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.289003+0000', 'last_change': '2023-10-18T08:58:28.289003+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.289003+0000', 'last_undegraded': '2023-10-18T08:58:28.289003+0000', 'last_fullsized': '2023-10-18T08:58:28.289003+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T08:58:34.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.21 is not active+clean 2023-10-18T08:58:34.889 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.289440+0000', 'last_change': '2023-10-18T08:58:28.289440+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.289440+0000', 'last_undegraded': '2023-10-18T08:58:28.289440+0000', 'last_fullsized': '2023-10-18T08:58:28.289440+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T08:58:34.889 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.20 is not active+clean 2023-10-18T08:58:34.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.284606+0000', 'last_change': '2023-10-18T08:58:28.284606+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.284606+0000', 'last_undegraded': '2023-10-18T08:58:28.284606+0000', 'last_fullsized': '2023-10-18T08:58:28.284606+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:58:34.890 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2023-10-18T08:58:34.890 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.288895+0000', 'last_change': '2023-10-18T08:58:28.288895+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.288895+0000', 'last_undegraded': '2023-10-18T08:58:28.288895+0000', 'last_fullsized': '2023-10-18T08:58:28.288895+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:58:34.891 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2023-10-18T08:58:34.891 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.277530+0000', 'last_change': '2023-10-18T08:58:28.277530+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.277530+0000', 'last_undegraded': '2023-10-18T08:58:28.277530+0000', 'last_fullsized': '2023-10-18T08:58:28.277530+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T08:58:34.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2023-10-18T08:58:34.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.286466+0000', 'last_change': '2023-10-18T08:58:28.286466+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.286466+0000', 'last_undegraded': '2023-10-18T08:58:28.286466+0000', 'last_fullsized': '2023-10-18T08:58:28.286466+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:58:34.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2023-10-18T08:58:34.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.289868+0000', 'last_change': '2023-10-18T08:58:28.289868+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.289868+0000', 'last_undegraded': '2023-10-18T08:58:28.289868+0000', 'last_fullsized': '2023-10-18T08:58:28.289868+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:58:34.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-10-18T08:58:34.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.281849+0000', 'last_change': '2023-10-18T08:58:28.281849+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.281849+0000', 'last_undegraded': '2023-10-18T08:58:28.281849+0000', 'last_fullsized': '2023-10-18T08:58:28.281849+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T08:58:34.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2023-10-18T08:58:34.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.289722+0000', 'last_change': '2023-10-18T08:58:28.289722+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.289722+0000', 'last_undegraded': '2023-10-18T08:58:28.289722+0000', 'last_fullsized': '2023-10-18T08:58:28.289722+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T08:58:34.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2023-10-18T08:58:34.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.277176+0000', 'last_change': '2023-10-18T08:58:28.277176+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.277176+0000', 'last_undegraded': '2023-10-18T08:58:28.277176+0000', 'last_fullsized': '2023-10-18T08:58:28.277176+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T08:58:34.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-10-18T08:58:34.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.290401+0000', 'last_change': '2023-10-18T08:58:28.290401+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.290401+0000', 'last_undegraded': '2023-10-18T08:58:28.290401+0000', 'last_fullsized': '2023-10-18T08:58:28.290401+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:58:34.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2023-10-18T08:58:34.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.288004+0000', 'last_change': '2023-10-18T08:58:28.288004+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.288004+0000', 'last_undegraded': '2023-10-18T08:58:28.288004+0000', 'last_fullsized': '2023-10-18T08:58:28.288004+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T08:58:34.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-10-18T08:58:34.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.293219+0000', 'last_change': '2023-10-18T08:58:28.293219+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.293219+0000', 'last_undegraded': '2023-10-18T08:58:28.293219+0000', 'last_fullsized': '2023-10-18T08:58:28.293219+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:58:34.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-10-18T08:58:34.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.292556+0000', 'last_change': '2023-10-18T08:58:28.292556+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.292556+0000', 'last_undegraded': '2023-10-18T08:58:28.292556+0000', 'last_fullsized': '2023-10-18T08:58:28.292556+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:58:34.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2023-10-18T08:58:34.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.294168+0000', 'last_change': '2023-10-18T08:58:28.294168+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.294168+0000', 'last_undegraded': '2023-10-18T08:58:28.294168+0000', 'last_fullsized': '2023-10-18T08:58:28.294168+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:58:34.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2023-10-18T08:58:34.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.289360+0000', 'last_change': '2023-10-18T08:58:28.289360+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.289360+0000', 'last_undegraded': '2023-10-18T08:58:28.289360+0000', 'last_fullsized': '2023-10-18T08:58:28.289360+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:58:34.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-10-18T08:58:34.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.289377+0000', 'last_change': '2023-10-18T08:58:28.289377+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.289377+0000', 'last_undegraded': '2023-10-18T08:58:28.289377+0000', 'last_fullsized': '2023-10-18T08:58:28.289377+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:58:34.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-10-18T08:58:34.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.289491+0000', 'last_change': '2023-10-18T08:58:28.289491+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.289491+0000', 'last_undegraded': '2023-10-18T08:58:28.289491+0000', 'last_fullsized': '2023-10-18T08:58:28.289491+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T08:58:34.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2023-10-18T08:58:34.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.277537+0000', 'last_change': '2023-10-18T08:58:28.277537+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.277537+0000', 'last_undegraded': '2023-10-18T08:58:28.277537+0000', 'last_fullsized': '2023-10-18T08:58:28.277537+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T08:58:34.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2023-10-18T08:58:34.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.284498+0000', 'last_change': '2023-10-18T08:58:28.284498+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.284498+0000', 'last_undegraded': '2023-10-18T08:58:28.284498+0000', 'last_fullsized': '2023-10-18T08:58:28.284498+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T08:58:34.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2023-10-18T08:58:34.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2023-10-18T08:58:28.276744+0000', 'last_change': '2023-10-18T08:58:28.276744+0000', 'last_active': '2023-10-18T08:58:28.264871+0000', 'last_peered': '2023-10-18T08:58:28.264871+0000', 'last_clean': '2023-10-18T08:58:28.264871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T08:58:28.276744+0000', 'last_undegraded': '2023-10-18T08:58:28.276744+0000', 'last_fullsized': '2023-10-18T08:58:28.276744+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'last_clean_scrub_stamp': '2023-10-18T08:58:28.264871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T08:58:34.905 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T08:58:35.019 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:58:35.026 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:58:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.146+0000 7f1fb97d7640 1 -- 172.21.15.40:0/610862075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fa00039a0 msgr2=0x7f1fa0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.146+0000 7f1fb97d7640 1 --2- 172.21.15.40:0/610862075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fa00039a0 0x7f1fa0003dc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4006ed0 tx=0x7f1fa00057f0 comp rx=0 tx=0).stop 2023-10-18T08:58:35.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.146+0000 7f1fb37fe640 1 -- 172.21.15.40:0/610862075 reap_dead start 2023-10-18T08:58:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.146+0000 7f1fb97d7640 1 -- 172.21.15.40:0/610862075 shutdown_connections 2023-10-18T08:58:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.146+0000 7f1fb97d7640 1 -- 172.21.15.40:0/610862075 >> 172.21.15.40:0/610862075 conn(0x7f1fb4069720 msgr2=0x7f1fb4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.146+0000 7f1fb97d7640 1 -- 172.21.15.40:0/610862075 shutdown_connections 2023-10-18T08:58:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.146+0000 7f1fb97d7640 1 -- 172.21.15.40:0/610862075 wait complete. 2023-10-18T08:58:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.148+0000 7f1fb97d7640 1 Processor -- start 2023-10-18T08:58:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.148+0000 7f1fb97d7640 1 -- start start 2023-10-18T08:58:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.149+0000 7f1fb97d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fa00039a0 0x7f1fb413d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.149+0000 7f1fb97d7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb41f3930 0x7f1fb4158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:35.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.149+0000 7f1fb97d7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fb415b700 0x7f1fb415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.149+0000 7f1fb97d7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb413de50 con 0x7f1fb41f3930 2023-10-18T08:58:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.149+0000 7f1fb97d7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb415c0f0 con 0x7f1fa00039a0 2023-10-18T08:58:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.149+0000 7f1fb97d7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb40abef0 con 0x7f1fb415b700 2023-10-18T08:58:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.149+0000 7f1fabfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb41f3930 0x7f1fb4158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.149+0000 7f1fabfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb41f3930 0x7f1fb4158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43324/0 (socket says 172.21.15.40:43324) 2023-10-18T08:58:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.149+0000 7f1fabfff640 1 -- 172.21.15.40:0/2521644809 learned_addr learned my addr 172.21.15.40:0/2521644809 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.150+0000 7f1fb37fe640 1 --2- 172.21.15.40:0/2521644809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fb415b700 0x7f1fb415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.150+0000 7f1fb2ffd640 1 --2- 172.21.15.40:0/2521644809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fa00039a0 0x7f1fb413d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.150+0000 7f1fabfff640 1 -- 172.21.15.40:0/2521644809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fb415b700 msgr2=0x7f1fb415bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:35.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.150+0000 7f1fabfff640 1 --2- 172.21.15.40:0/2521644809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fb415b700 0x7f1fb415bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:35.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.150+0000 7f1fabfff640 1 -- 172.21.15.40:0/2521644809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fa00039a0 msgr2=0x7f1fb413d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:35.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.150+0000 7f1fabfff640 1 --2- 172.21.15.40:0/2521644809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fa00039a0 0x7f1fb413d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:35.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.150+0000 7f1fabfff640 1 -- 172.21.15.40:0/2521644809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb4158b80 con 0x7f1fb41f3930 2023-10-18T08:58:35.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.150+0000 7f1fb2ffd640 1 --2- 172.21.15.40:0/2521644809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fa00039a0 0x7f1fb413d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.150+0000 7f1fb37fe640 1 --2- 172.21.15.40:0/2521644809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fb415b700 0x7f1fb415bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.150+0000 7f1fabfff640 1 --2- 172.21.15.40:0/2521644809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb41f3930 0x7f1fb4158640 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c004e20 tx=0x7f1f9c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.150+0000 7f1fab7fe640 1 -- 172.21.15.40:0/2521644809 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0107d0 con 0x7f1fb41f3930 2023-10-18T08:58:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.150+0000 7f1fab7fe640 1 -- 172.21.15.40:0/2521644809 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1f9c010a40 con 0x7f1fb41f3930 2023-10-18T08:58:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.150+0000 7f1fb97d7640 1 -- 172.21.15.40:0/2521644809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb41f3db0 con 0x7f1fb41f3930 2023-10-18T08:58:35.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.150+0000 7f1fab7fe640 1 -- 172.21.15.40:0/2521644809 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c010e50 con 0x7f1fb41f3930 2023-10-18T08:58:35.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.150+0000 7f1fb97d7640 1 -- 172.21.15.40:0/2521644809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb413cc30 con 0x7f1fb41f3930 2023-10-18T08:58:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.152+0000 7f1fb97d7640 1 -- 172.21.15.40:0/2521644809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb4066fd0 con 0x7f1fb41f3930 2023-10-18T08:58:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.153+0000 7f1fab7fe640 1 -- 172.21.15.40:0/2521644809 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1f9c01b460 con 0x7f1fb41f3930 2023-10-18T08:58:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.154+0000 7f1fab7fe640 1 --2- 172.21.15.40:0/2521644809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f9406ae70 0x7f1f9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.154+0000 7f1fab7fe640 1 -- 172.21.15.40:0/2521644809 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f9c08a120 con 0x7f1fb41f3930 2023-10-18T08:58:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.155+0000 7f1fb2ffd640 1 --2- 172.21.15.40:0/2521644809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f9406ae70 0x7f1f9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.156+0000 7f1fb2ffd640 1 --2- 172.21.15.40:0/2521644809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f9406ae70 0x7f1f9406d330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4034000 tx=0x7f1fa402e070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:35.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.159+0000 7f1fab7fe640 1 -- 172.21.15.40:0/2521644809 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f9c0551c0 con 0x7f1fb41f3930 2023-10-18T08:58:35.265 INFO:tasks.ceph.mds.d.smithi144.stderr:2023-10-18T08:58:35.262+0000 7f737cefc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T08:58:35.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.429+0000 7f1fb97d7640 1 -- 172.21.15.40:0/2521644809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1fb40aed70 con 0x7f1fb41f3930 2023-10-18T08:58:35.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.429+0000 7f1fab7fe640 1 -- 172.21.15.40:0/2521644809 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1007 (secure 0 0 0) 0x7f1f9c0587e0 con 0x7f1fb41f3930 2023-10-18T08:58:35.432 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:35.432 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":219,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":67,"num_osds":8,"num_up_osds":8,"osd_up_since":1697619304,"num_in_osds":8,"osd_in_since":1697619300,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":3932117,"bytes_avail":1596067883,"bytes_total":1600000000,"write_bytes_sec":2142,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":33,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"d","status":"up:active","gid":5389}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-10-18T08:58:14.959370+0000","services":{}},"progress_events":{}} 2023-10-18T08:58:35.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.432+0000 7f1fb97d7640 1 -- 172.21.15.40:0/2521644809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f9406ae70 msgr2=0x7f1f9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:35.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.432+0000 7f1fb97d7640 1 --2- 172.21.15.40:0/2521644809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f9406ae70 0x7f1f9406d330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4034000 tx=0x7f1fa402e070 comp rx=0 tx=0).stop 2023-10-18T08:58:35.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.432+0000 7f1fb97d7640 1 -- 172.21.15.40:0/2521644809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb41f3930 msgr2=0x7f1fb4158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:35.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.432+0000 7f1fb97d7640 1 --2- 172.21.15.40:0/2521644809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb41f3930 0x7f1fb4158640 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c004e20 tx=0x7f1f9c001630 comp rx=0 tx=0).stop 2023-10-18T08:58:35.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.432+0000 7f1fb97d7640 1 -- 172.21.15.40:0/2521644809 shutdown_connections 2023-10-18T08:58:35.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.432+0000 7f1fb97d7640 1 --2- 172.21.15.40:0/2521644809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f9406ae70 0x7f1f9406d330 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:35.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.432+0000 7f1fb97d7640 1 --2- 172.21.15.40:0/2521644809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fb415b700 0x7f1fb415bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:35.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.432+0000 7f1fb97d7640 1 --2- 172.21.15.40:0/2521644809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fb41f3930 0x7f1fb4158640 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:35.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.432+0000 7f1fb97d7640 1 --2- 172.21.15.40:0/2521644809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fa00039a0 0x7f1fb413d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:35.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.432+0000 7f1fb97d7640 1 -- 172.21.15.40:0/2521644809 >> 172.21.15.40:0/2521644809 conn(0x7f1fb4069720 msgr2=0x7f1fb4141a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:35.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.432+0000 7f1fb97d7640 1 -- 172.21.15.40:0/2521644809 shutdown_connections 2023-10-18T08:58:35.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.432+0000 7f1fb97d7640 1 -- 172.21.15.40:0/2521644809 wait complete. 2023-10-18T08:58:35.452 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T08:58:35.453 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:58:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.679+0000 7f9ddb59e640 1 -- 172.21.15.40:0/2605100453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ddc06edf0 msgr2=0x7f9ddc069720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.679+0000 7f9ddb59e640 1 --2- 172.21.15.40:0/2605100453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ddc06edf0 0x7f9ddc069720 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0005fd0 tx=0x7f9dd002be20 comp rx=0 tx=0).stop 2023-10-18T08:58:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.680+0000 7f9ddb59e640 1 -- 172.21.15.40:0/2605100453 shutdown_connections 2023-10-18T08:58:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.680+0000 7f9ddb59e640 1 --2- 172.21.15.40:0/2605100453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ddc06edf0 0x7f9ddc069720 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.680+0000 7f9ddb59e640 1 -- 172.21.15.40:0/2605100453 >> 172.21.15.40:0/2605100453 conn(0x7f9ddc0afdf0 msgr2=0x7f9ddc0b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:35.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.680+0000 7f9ddb59e640 1 -- 172.21.15.40:0/2605100453 shutdown_connections 2023-10-18T08:58:35.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.680+0000 7f9ddb59e640 1 -- 172.21.15.40:0/2605100453 wait complete. 2023-10-18T08:58:35.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.682+0000 7f9ddb59e640 1 Processor -- start 2023-10-18T08:58:35.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.682+0000 7f9ddb59e640 1 -- start start 2023-10-18T08:58:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.683+0000 7f9ddb59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ddc15b3d0 0x7f9ddc1503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.684+0000 7f9ddb59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ddc15c640 0x7f9ddc1f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.684+0000 7f9ddb59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ddc1f53a0 0x7f9ddc0caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.684+0000 7f9ddb59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc0aae70 con 0x7f9ddc15b3d0 2023-10-18T08:58:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.684+0000 7f9ddb59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc0aa880 con 0x7f9ddc1f53a0 2023-10-18T08:58:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.684+0000 7f9ddb59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc1f5920 con 0x7f9ddc15c640 2023-10-18T08:58:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.684+0000 7f9dda59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ddc15b3d0 0x7f9ddc1503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.684+0000 7f9dda59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ddc15b3d0 0x7f9ddc1503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43354/0 (socket says 172.21.15.40:43354) 2023-10-18T08:58:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.684+0000 7f9dda59c640 1 -- 172.21.15.40:0/203258132 learned_addr learned my addr 172.21.15.40:0/203258132 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.684+0000 7f9dda59c640 1 -- 172.21.15.40:0/203258132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ddc15c640 msgr2=0x7f9ddc1f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:35.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.684+0000 7f9dd9d9b640 1 --2- 172.21.15.40:0/203258132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ddc15c640 0x7f9ddc1f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:35.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.684+0000 7f9ddad9d640 1 --2- 172.21.15.40:0/203258132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ddc1f53a0 0x7f9ddc0caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:35.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.684+0000 7f9dda59c640 1 --2- 172.21.15.40:0/203258132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ddc15c640 0x7f9ddc1f4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:35.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.684+0000 7f9dda59c640 1 -- 172.21.15.40:0/203258132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ddc1f53a0 msgr2=0x7f9ddc0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:35.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.684+0000 7f9dda59c640 1 --2- 172.21.15.40:0/203258132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ddc1f53a0 0x7f9ddc0caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:35.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.684+0000 7f9dda59c640 1 -- 172.21.15.40:0/203258132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd0009560 con 0x7f9ddc15b3d0 2023-10-18T08:58:35.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.685+0000 7f9ddad9d640 1 --2- 172.21.15.40:0/203258132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ddc1f53a0 0x7f9ddc0caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:35.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.685+0000 7f9dda59c640 1 --2- 172.21.15.40:0/203258132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ddc15b3d0 0x7f9ddc1503b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00c450 tx=0x7f9dcc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:35.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.685+0000 7f9dcaffd640 1 -- 172.21.15.40:0/203258132 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc0052d0 con 0x7f9ddc15b3d0 2023-10-18T08:58:35.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.685+0000 7f9dcaffd640 1 -- 172.21.15.40:0/203258132 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9dcc005430 con 0x7f9ddc15b3d0 2023-10-18T08:58:35.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.685+0000 7f9dcaffd640 1 -- 172.21.15.40:0/203258132 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc012890 con 0x7f9ddc15b3d0 2023-10-18T08:58:35.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.685+0000 7f9ddb59e640 1 -- 172.21.15.40:0/203258132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ddc06a0c0 con 0x7f9ddc15b3d0 2023-10-18T08:58:35.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.685+0000 7f9ddb59e640 1 -- 172.21.15.40:0/203258132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ddc0aaa90 con 0x7f9ddc15b3d0 2023-10-18T08:58:35.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.687+0000 7f9ddb59e640 1 -- 172.21.15.40:0/203258132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ddc066e00 con 0x7f9ddc15b3d0 2023-10-18T08:58:35.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.688+0000 7f9dcaffd640 1 -- 172.21.15.40:0/203258132 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9dcc0035f0 con 0x7f9ddc15b3d0 2023-10-18T08:58:35.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.688+0000 7f9dcaffd640 1 --2- 172.21.15.40:0/203258132 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9dbc06ae40 0x7f9dbc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:35.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.688+0000 7f9dcaffd640 1 -- 172.21.15.40:0/203258132 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9dcc0892d0 con 0x7f9ddc15b3d0 2023-10-18T08:58:35.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.689+0000 7f9dd9d9b640 1 --2- 172.21.15.40:0/203258132 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9dbc06ae40 0x7f9dbc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:35.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.690+0000 7f9dd9d9b640 1 --2- 172.21.15.40:0/203258132 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9dbc06ae40 0x7f9dbc06d300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0035000 tx=0x7f9dd0004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:35.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.693+0000 7f9dcaffd640 1 -- 172.21.15.40:0/203258132 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9dcc054370 con 0x7f9ddc15b3d0 2023-10-18T08:58:35.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.884+0000 7f9ddb59e640 1 -- 172.21.15.40:0/203258132 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ddc0cc0e0 con 0x7f9dbc06ae40 2023-10-18T08:58:35.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.908+0000 7f9dcaffd640 1 -- 172.21.15.40:0/203258132 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282442 (secure 0 0 0) 0x7f9ddc0cc0e0 con 0x7f9dbc06ae40 2023-10-18T08:58:35.912 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:35.915 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:58:35.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.914+0000 7f9ddb59e640 1 -- 172.21.15.40:0/203258132 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9dbc06ae40 msgr2=0x7f9dbc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:35.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.914+0000 7f9ddb59e640 1 --2- 172.21.15.40:0/203258132 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9dbc06ae40 0x7f9dbc06d300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0035000 tx=0x7f9dd0004730 comp rx=0 tx=0).stop 2023-10-18T08:58:35.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.914+0000 7f9ddb59e640 1 -- 172.21.15.40:0/203258132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ddc15b3d0 msgr2=0x7f9ddc1503b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:35.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.914+0000 7f9ddb59e640 1 --2- 172.21.15.40:0/203258132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ddc15b3d0 0x7f9ddc1503b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00c450 tx=0x7f9dcc00c910 comp rx=0 tx=0).stop 2023-10-18T08:58:35.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.915+0000 7f9ddb59e640 1 -- 172.21.15.40:0/203258132 shutdown_connections 2023-10-18T08:58:35.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.915+0000 7f9ddb59e640 1 --2- 172.21.15.40:0/203258132 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9dbc06ae40 0x7f9dbc06d300 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:35.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.915+0000 7f9ddb59e640 1 --2- 172.21.15.40:0/203258132 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ddc1f53a0 0x7f9ddc0caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:35.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.915+0000 7f9ddb59e640 1 --2- 172.21.15.40:0/203258132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ddc15c640 0x7f9ddc1f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:35.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.915+0000 7f9ddb59e640 1 --2- 172.21.15.40:0/203258132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ddc15b3d0 0x7f9ddc1503b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.915+0000 7f9ddb59e640 1 -- 172.21.15.40:0/203258132 >> 172.21.15.40:0/203258132 conn(0x7f9ddc0afdf0 msgr2=0x7f9ddc0ca760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.915+0000 7f9ddb59e640 1 -- 172.21.15.40:0/203258132 shutdown_connections 2023-10-18T08:58:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:35.915+0000 7f9ddb59e640 1 -- 172.21.15.40:0/203258132 wait complete. 2023-10-18T08:58:35.939 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2023-10-18T08:58:34.965520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3839,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558660,"statfs":{"total":1600000000,"available":1596067883,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.690859"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.301202+0000","last_change":"2023-10-18T08:58:29.324418+0000","last_active":"2023-10-18T08:58:30.301202+0000","last_peered":"2023-10-18T08:58:30.301202+0000","last_clean":"2023-10-18T08:58:30.301202+0000","last_became_active":"2023-10-18T08:58:29.322273+0000","last_became_peered":"2023-10-18T08:58:29.322273+0000","last_unstale":"2023-10-18T08:58:30.301202+0000","last_undegraded":"2023-10-18T08:58:30.301202+0000","last_fullsized":"2023-10-18T08:58:30.301202+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:20:15.523189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.310859+0000","last_change":"2023-10-18T08:58:29.310661+0000","last_active":"2023-10-18T08:58:29.310859+0000","last_peered":"2023-10-18T08:58:29.310859+0000","last_clean":"2023-10-18T08:58:29.310859+0000","last_became_active":"2023-10-18T08:58:29.309557+0000","last_became_peered":"2023-10-18T08:58:29.309557+0000","last_unstale":"2023-10-18T08:58:29.310859+0000","last_undegraded":"2023-10-18T08:58:29.310859+0000","last_fullsized":"2023-10-18T08:58:29.310859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:46.399620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.323795+0000","last_change":"2023-10-18T08:58:29.321404+0000","last_active":"2023-10-18T08:58:29.323795+0000","last_peered":"2023-10-18T08:58:29.323795+0000","last_clean":"2023-10-18T08:58:29.323795+0000","last_became_active":"2023-10-18T08:58:29.320985+0000","last_became_peered":"2023-10-18T08:58:29.320985+0000","last_unstale":"2023-10-18T08:58:29.323795+0000","last_undegraded":"2023-10-18T08:58:29.323795+0000","last_fullsized":"2023-10-18T08:58:29.323795+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:10:04.504829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.486351+0000","last_change":"2023-10-18T08:58:29.332571+0000","last_active":"2023-10-18T08:58:30.486351+0000","last_peered":"2023-10-18T08:58:30.486351+0000","last_clean":"2023-10-18T08:58:30.486351+0000","last_became_active":"2023-10-18T08:58:29.330694+0000","last_became_peered":"2023-10-18T08:58:29.330694+0000","last_unstale":"2023-10-18T08:58:30.486351+0000","last_undegraded":"2023-10-18T08:58:30.486351+0000","last_fullsized":"2023-10-18T08:58:30.486351+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:49.888147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.301195+0000","last_change":"2023-10-18T08:58:29.327743+0000","last_active":"2023-10-18T08:58:30.301195+0000","last_peered":"2023-10-18T08:58:30.301195+0000","last_clean":"2023-10-18T08:58:30.301195+0000","last_became_active":"2023-10-18T08:58:29.327430+0000","last_became_peered":"2023-10-18T08:58:29.327430+0000","last_unstale":"2023-10-18T08:58:30.301195+0000","last_undegraded":"2023-10-18T08:58:30.301195+0000","last_fullsized":"2023-10-18T08:58:30.301195+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:40:49.731282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.526723+0000","last_change":"2023-10-18T08:58:29.331173+0000","last_active":"2023-10-18T08:58:31.526723+0000","last_peered":"2023-10-18T08:58:31.526723+0000","last_clean":"2023-10-18T08:58:31.526723+0000","last_became_active":"2023-10-18T08:58:29.330450+0000","last_became_peered":"2023-10-18T08:58:29.330450+0000","last_unstale":"2023-10-18T08:58:31.526723+0000","last_undegraded":"2023-10-18T08:58:31.526723+0000","last_fullsized":"2023-10-18T08:58:31.526723+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:17.764621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.487514+0000","last_change":"2023-10-18T08:58:29.331909+0000","last_active":"2023-10-18T08:58:30.487514+0000","last_peered":"2023-10-18T08:58:30.487514+0000","last_clean":"2023-10-18T08:58:30.487514+0000","last_became_active":"2023-10-18T08:58:29.331625+0000","last_became_peered":"2023-10-18T08:58:29.331625+0000","last_unstale":"2023-10-18T08:58:30.487514+0000","last_undegraded":"2023-10-18T08:58:30.487514+0000","last_fullsized":"2023-10-18T08:58:30.487514+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:49.888147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326822+0000","last_change":"2023-10-18T08:58:29.325559+0000","last_active":"2023-10-18T08:58:29.326822+0000","last_peered":"2023-10-18T08:58:29.326822+0000","last_clean":"2023-10-18T08:58:29.326822+0000","last_became_active":"2023-10-18T08:58:29.324359+0000","last_became_peered":"2023-10-18T08:58:29.324359+0000","last_unstale":"2023-10-18T08:58:29.326822+0000","last_undegraded":"2023-10-18T08:58:29.326822+0000","last_fullsized":"2023-10-18T08:58:29.326822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:45.688349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.487715+0000","last_change":"2023-10-18T08:58:29.333697+0000","last_active":"2023-10-18T08:58:30.487715+0000","last_peered":"2023-10-18T08:58:30.487715+0000","last_clean":"2023-10-18T08:58:30.487715+0000","last_became_active":"2023-10-18T08:58:29.330251+0000","last_became_peered":"2023-10-18T08:58:29.330251+0000","last_unstale":"2023-10-18T08:58:30.487715+0000","last_undegraded":"2023-10-18T08:58:30.487715+0000","last_fullsized":"2023-10-18T08:58:30.487715+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:54.275856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.525455+0000","last_change":"2023-10-18T08:58:29.319001+0000","last_active":"2023-10-18T08:58:31.525455+0000","last_peered":"2023-10-18T08:58:31.525455+0000","last_clean":"2023-10-18T08:58:31.525455+0000","last_became_active":"2023-10-18T08:58:29.318606+0000","last_became_peered":"2023-10-18T08:58:29.318606+0000","last_unstale":"2023-10-18T08:58:31.525455+0000","last_undegraded":"2023-10-18T08:58:31.525455+0000","last_fullsized":"2023-10-18T08:58:31.525455+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:48:54.439674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.525671+0000","last_change":"2023-10-18T08:58:29.307513+0000","last_active":"2023-10-18T08:58:31.525671+0000","last_peered":"2023-10-18T08:58:31.525671+0000","last_clean":"2023-10-18T08:58:31.525671+0000","last_became_active":"2023-10-18T08:58:29.306861+0000","last_became_peered":"2023-10-18T08:58:29.306861+0000","last_unstale":"2023-10-18T08:58:31.525671+0000","last_undegraded":"2023-10-18T08:58:31.525671+0000","last_fullsized":"2023-10-18T08:58:31.525671+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:07.925495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.486472+0000","last_change":"2023-10-18T08:58:29.326923+0000","last_active":"2023-10-18T08:58:30.486472+0000","last_peered":"2023-10-18T08:58:30.486472+0000","last_clean":"2023-10-18T08:58:30.486472+0000","last_became_active":"2023-10-18T08:58:29.324619+0000","last_became_peered":"2023-10-18T08:58:29.324619+0000","last_unstale":"2023-10-18T08:58:30.486472+0000","last_undegraded":"2023-10-18T08:58:30.486472+0000","last_fullsized":"2023-10-18T08:58:30.486472+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:54.275856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.526053+0000","last_change":"2023-10-18T08:58:29.329776+0000","last_active":"2023-10-18T08:58:31.526053+0000","last_peered":"2023-10-18T08:58:31.526053+0000","last_clean":"2023-10-18T08:58:31.526053+0000","last_became_active":"2023-10-18T08:58:29.329526+0000","last_became_peered":"2023-10-18T08:58:29.329526+0000","last_unstale":"2023-10-18T08:58:31.526053+0000","last_undegraded":"2023-10-18T08:58:31.526053+0000","last_fullsized":"2023-10-18T08:58:31.526053+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:09.801713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.487921+0000","last_change":"2023-10-18T08:58:29.328265+0000","last_active":"2023-10-18T08:58:30.487921+0000","last_peered":"2023-10-18T08:58:30.487921+0000","last_clean":"2023-10-18T08:58:30.487921+0000","last_became_active":"2023-10-18T08:58:29.327575+0000","last_became_peered":"2023-10-18T08:58:29.327575+0000","last_unstale":"2023-10-18T08:58:30.487921+0000","last_undegraded":"2023-10-18T08:58:30.487921+0000","last_fullsized":"2023-10-18T08:58:30.487921+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:25.582208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.488833+0000","last_change":"2023-10-18T08:58:29.331412+0000","last_active":"2023-10-18T08:58:30.488833+0000","last_peered":"2023-10-18T08:58:30.488833+0000","last_clean":"2023-10-18T08:58:30.488833+0000","last_became_active":"2023-10-18T08:58:29.330792+0000","last_became_peered":"2023-10-18T08:58:29.330792+0000","last_unstale":"2023-10-18T08:58:30.488833+0000","last_undegraded":"2023-10-18T08:58:30.488833+0000","last_fullsized":"2023-10-18T08:58:30.488833+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:40.763889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.527429+0000","last_change":"2023-10-18T08:58:29.330739+0000","last_active":"2023-10-18T08:58:31.527429+0000","last_peered":"2023-10-18T08:58:31.527429+0000","last_clean":"2023-10-18T08:58:31.527429+0000","last_became_active":"2023-10-18T08:58:29.328465+0000","last_became_peered":"2023-10-18T08:58:29.328465+0000","last_unstale":"2023-10-18T08:58:31.527429+0000","last_undegraded":"2023-10-18T08:58:31.527429+0000","last_fullsized":"2023-10-18T08:58:31.527429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:51.344108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326448+0000","last_change":"2023-10-18T08:58:29.325090+0000","last_active":"2023-10-18T08:58:29.326448+0000","last_peered":"2023-10-18T08:58:29.326448+0000","last_clean":"2023-10-18T08:58:29.326448+0000","last_became_active":"2023-10-18T08:58:29.323675+0000","last_became_peered":"2023-10-18T08:58:29.323675+0000","last_unstale":"2023-10-18T08:58:29.326448+0000","last_undegraded":"2023-10-18T08:58:29.326448+0000","last_fullsized":"2023-10-18T08:58:29.326448+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:01:55.373464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.329948+0000","last_change":"2023-10-18T08:58:29.329020+0000","last_active":"2023-10-18T08:58:29.329948+0000","last_peered":"2023-10-18T08:58:29.329948+0000","last_clean":"2023-10-18T08:58:29.329948+0000","last_became_active":"2023-10-18T08:58:29.328773+0000","last_became_peered":"2023-10-18T08:58:29.328773+0000","last_unstale":"2023-10-18T08:58:29.329948+0000","last_undegraded":"2023-10-18T08:58:29.329948+0000","last_fullsized":"2023-10-18T08:58:29.329948+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:57.274523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.300662+0000","last_change":"2023-10-18T08:58:29.326282+0000","last_active":"2023-10-18T08:58:30.300662+0000","last_peered":"2023-10-18T08:58:30.300662+0000","last_clean":"2023-10-18T08:58:30.300662+0000","last_became_active":"2023-10-18T08:58:29.325556+0000","last_became_peered":"2023-10-18T08:58:29.325556+0000","last_unstale":"2023-10-18T08:58:30.300662+0000","last_undegraded":"2023-10-18T08:58:30.300662+0000","last_fullsized":"2023-10-18T08:58:30.300662+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:56:51.163636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.330756+0000","last_change":"2023-10-18T08:58:29.330497+0000","last_active":"2023-10-18T08:58:29.330756+0000","last_peered":"2023-10-18T08:58:29.330756+0000","last_clean":"2023-10-18T08:58:29.330756+0000","last_became_active":"2023-10-18T08:58:29.330245+0000","last_became_peered":"2023-10-18T08:58:29.330245+0000","last_unstale":"2023-10-18T08:58:29.330756+0000","last_undegraded":"2023-10-18T08:58:29.330756+0000","last_fullsized":"2023-10-18T08:58:29.330756+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:23.135334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.323823+0000","last_change":"2023-10-18T08:58:29.323242+0000","last_active":"2023-10-18T08:58:29.323823+0000","last_peered":"2023-10-18T08:58:29.323823+0000","last_clean":"2023-10-18T08:58:29.323823+0000","last_became_active":"2023-10-18T08:58:29.322832+0000","last_became_peered":"2023-10-18T08:58:29.322832+0000","last_unstale":"2023-10-18T08:58:29.323823+0000","last_undegraded":"2023-10-18T08:58:29.323823+0000","last_fullsized":"2023-10-18T08:58:29.323823+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:10:00.029939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326366+0000","last_change":"2023-10-18T08:58:29.319011+0000","last_active":"2023-10-18T08:58:29.326366+0000","last_peered":"2023-10-18T08:58:29.326366+0000","last_clean":"2023-10-18T08:58:29.326366+0000","last_became_active":"2023-10-18T08:58:29.318536+0000","last_became_peered":"2023-10-18T08:58:29.318536+0000","last_unstale":"2023-10-18T08:58:29.326366+0000","last_undegraded":"2023-10-18T08:58:29.326366+0000","last_fullsized":"2023-10-18T08:58:29.326366+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:16.123758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.319916+0000","last_change":"2023-10-18T08:58:29.319576+0000","last_active":"2023-10-18T08:58:29.319916+0000","last_peered":"2023-10-18T08:58:29.319916+0000","last_clean":"2023-10-18T08:58:29.319916+0000","last_became_active":"2023-10-18T08:58:29.316842+0000","last_became_peered":"2023-10-18T08:58:29.316842+0000","last_unstale":"2023-10-18T08:58:29.319916+0000","last_undegraded":"2023-10-18T08:58:29.319916+0000","last_fullsized":"2023-10-18T08:58:29.319916+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:45.620650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.525555+0000","last_change":"2023-10-18T08:58:29.331123+0000","last_active":"2023-10-18T08:58:31.525555+0000","last_peered":"2023-10-18T08:58:31.525555+0000","last_clean":"2023-10-18T08:58:31.525555+0000","last_became_active":"2023-10-18T08:58:29.330917+0000","last_became_peered":"2023-10-18T08:58:29.330917+0000","last_unstale":"2023-10-18T08:58:31.525555+0000","last_undegraded":"2023-10-18T08:58:31.525555+0000","last_fullsized":"2023-10-18T08:58:31.525555+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:26:15.788187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.301380+0000","last_change":"2023-10-18T08:58:29.325403+0000","last_active":"2023-10-18T08:58:30.301380+0000","last_peered":"2023-10-18T08:58:30.301380+0000","last_clean":"2023-10-18T08:58:30.301380+0000","last_became_active":"2023-10-18T08:58:29.325064+0000","last_became_peered":"2023-10-18T08:58:29.325064+0000","last_unstale":"2023-10-18T08:58:30.301380+0000","last_undegraded":"2023-10-18T08:58:30.301380+0000","last_fullsized":"2023-10-18T08:58:30.301380+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:52:42.152402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.486747+0000","last_change":"2023-10-18T08:58:29.323694+0000","last_active":"2023-10-18T08:58:30.486747+0000","last_peered":"2023-10-18T08:58:30.486747+0000","last_clean":"2023-10-18T08:58:30.486747+0000","last_became_active":"2023-10-18T08:58:29.323083+0000","last_became_peered":"2023-10-18T08:58:29.323083+0000","last_unstale":"2023-10-18T08:58:30.486747+0000","last_undegraded":"2023-10-18T08:58:30.486747+0000","last_fullsized":"2023-10-18T08:58:30.486747+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:57:23.733388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.526798+0000","last_change":"2023-10-18T08:58:29.332412+0000","last_active":"2023-10-18T08:58:31.526798+0000","last_peered":"2023-10-18T08:58:31.526798+0000","last_clean":"2023-10-18T08:58:31.526798+0000","last_became_active":"2023-10-18T08:58:29.330172+0000","last_became_peered":"2023-10-18T08:58:29.330172+0000","last_unstale":"2023-10-18T08:58:31.526798+0000","last_undegraded":"2023-10-18T08:58:31.526798+0000","last_fullsized":"2023-10-18T08:58:31.526798+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:15:04.978697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.301125+0000","last_change":"2023-10-18T08:58:29.326391+0000","last_active":"2023-10-18T08:58:30.301125+0000","last_peered":"2023-10-18T08:58:30.301125+0000","last_clean":"2023-10-18T08:58:30.301125+0000","last_became_active":"2023-10-18T08:58:29.326036+0000","last_became_peered":"2023-10-18T08:58:29.326036+0000","last_unstale":"2023-10-18T08:58:30.301125+0000","last_undegraded":"2023-10-18T08:58:30.301125+0000","last_fullsized":"2023-10-18T08:58:30.301125+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:39:24.943573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.329742+0000","last_change":"2023-10-18T08:58:29.329658+0000","last_active":"2023-10-18T08:58:29.329742+0000","last_peered":"2023-10-18T08:58:29.329742+0000","last_clean":"2023-10-18T08:58:29.329742+0000","last_became_active":"2023-10-18T08:58:29.329393+0000","last_became_peered":"2023-10-18T08:58:29.329393+0000","last_unstale":"2023-10-18T08:58:29.329742+0000","last_undegraded":"2023-10-18T08:58:29.329742+0000","last_fullsized":"2023-10-18T08:58:29.329742+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:28:25.454956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326242+0000","last_change":"2023-10-18T08:58:29.326075+0000","last_active":"2023-10-18T08:58:29.326242+0000","last_peered":"2023-10-18T08:58:29.326242+0000","last_clean":"2023-10-18T08:58:29.326242+0000","last_became_active":"2023-10-18T08:58:29.324858+0000","last_became_peered":"2023-10-18T08:58:29.324858+0000","last_unstale":"2023-10-18T08:58:29.326242+0000","last_undegraded":"2023-10-18T08:58:29.326242+0000","last_fullsized":"2023-10-18T08:58:29.326242+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:49.015165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.301546+0000","last_change":"2023-10-18T08:58:29.325277+0000","last_active":"2023-10-18T08:58:30.301546+0000","last_peered":"2023-10-18T08:58:30.301546+0000","last_clean":"2023-10-18T08:58:30.301546+0000","last_became_active":"2023-10-18T08:58:29.324019+0000","last_became_peered":"2023-10-18T08:58:29.324019+0000","last_unstale":"2023-10-18T08:58:30.301546+0000","last_undegraded":"2023-10-18T08:58:30.301546+0000","last_fullsized":"2023-10-18T08:58:30.301546+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:17:55.356607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.527125+0000","last_change":"2023-10-18T08:58:29.331792+0000","last_active":"2023-10-18T08:58:31.527125+0000","last_peered":"2023-10-18T08:58:31.527125+0000","last_clean":"2023-10-18T08:58:31.527125+0000","last_became_active":"2023-10-18T08:58:29.329338+0000","last_became_peered":"2023-10-18T08:58:29.329338+0000","last_unstale":"2023-10-18T08:58:31.527125+0000","last_undegraded":"2023-10-18T08:58:31.527125+0000","last_fullsized":"2023-10-18T08:58:31.527125+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:39:03.787381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.526782+0000","last_change":"2023-10-18T08:58:29.329602+0000","last_active":"2023-10-18T08:58:31.526782+0000","last_peered":"2023-10-18T08:58:31.526782+0000","last_clean":"2023-10-18T08:58:31.526782+0000","last_became_active":"2023-10-18T08:58:29.328496+0000","last_became_peered":"2023-10-18T08:58:29.328496+0000","last_unstale":"2023-10-18T08:58:31.526782+0000","last_undegraded":"2023-10-18T08:58:31.526782+0000","last_fullsized":"2023-10-18T08:58:31.526782+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:16.123758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"66'4","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341720+0000","last_change":"2023-10-18T08:58:28.300855+0000","last_active":"2023-10-18T08:58:29.341720+0000","last_peered":"2023-10-18T08:58:29.341720+0000","last_clean":"2023-10-18T08:58:29.341720+0000","last_became_active":"2023-10-18T08:58:28.300672+0000","last_became_peered":"2023-10-18T08:58:28.300672+0000","last_unstale":"2023-10-18T08:58:29.341720+0000","last_undegraded":"2023-10-18T08:58:29.341720+0000","last_fullsized":"2023-10-18T08:58:29.341720+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:03.137892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.524804+0000","last_change":"2023-10-18T08:58:29.310840+0000","last_active":"2023-10-18T08:58:31.524804+0000","last_peered":"2023-10-18T08:58:31.524804+0000","last_clean":"2023-10-18T08:58:31.524804+0000","last_became_active":"2023-10-18T08:58:29.310085+0000","last_became_peered":"2023-10-18T08:58:29.310085+0000","last_unstale":"2023-10-18T08:58:31.524804+0000","last_undegraded":"2023-10-18T08:58:31.524804+0000","last_fullsized":"2023-10-18T08:58:31.524804+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:10:21.837993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"66'1","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.342802+0000","last_change":"2023-10-18T08:58:28.295079+0000","last_active":"2023-10-18T08:58:29.342802+0000","last_peered":"2023-10-18T08:58:29.342802+0000","last_clean":"2023-10-18T08:58:29.342802+0000","last_became_active":"2023-10-18T08:58:28.294485+0000","last_became_peered":"2023-10-18T08:58:28.294485+0000","last_unstale":"2023-10-18T08:58:29.342802+0000","last_undegraded":"2023-10-18T08:58:29.342802+0000","last_fullsized":"2023-10-18T08:58:29.342802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:27.603595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.527080+0000","last_change":"2023-10-18T08:58:29.330968+0000","last_active":"2023-10-18T08:58:31.527080+0000","last_peered":"2023-10-18T08:58:31.527080+0000","last_clean":"2023-10-18T08:58:31.527080+0000","last_became_active":"2023-10-18T08:58:29.329041+0000","last_became_peered":"2023-10-18T08:58:29.329041+0000","last_unstale":"2023-10-18T08:58:31.527080+0000","last_undegraded":"2023-10-18T08:58:31.527080+0000","last_fullsized":"2023-10-18T08:58:31.527080+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:30:53.581720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.342059+0000","last_change":"2023-10-18T08:58:28.298521+0000","last_active":"2023-10-18T08:58:29.342059+0000","last_peered":"2023-10-18T08:58:29.342059+0000","last_clean":"2023-10-18T08:58:29.342059+0000","last_became_active":"2023-10-18T08:58:28.296841+0000","last_became_peered":"2023-10-18T08:58:28.296841+0000","last_unstale":"2023-10-18T08:58:29.342059+0000","last_undegraded":"2023-10-18T08:58:29.342059+0000","last_fullsized":"2023-10-18T08:58:29.342059+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:22.121699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.526860+0000","last_change":"2023-10-18T08:58:29.329828+0000","last_active":"2023-10-18T08:58:31.526860+0000","last_peered":"2023-10-18T08:58:31.526860+0000","last_clean":"2023-10-18T08:58:31.526860+0000","last_became_active":"2023-10-18T08:58:29.329029+0000","last_became_peered":"2023-10-18T08:58:29.329029+0000","last_unstale":"2023-10-18T08:58:31.526860+0000","last_undegraded":"2023-10-18T08:58:31.526860+0000","last_fullsized":"2023-10-18T08:58:31.526860+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:54.004578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341158+0000","last_change":"2023-10-18T08:58:28.298876+0000","last_active":"2023-10-18T08:58:29.341158+0000","last_peered":"2023-10-18T08:58:29.341158+0000","last_clean":"2023-10-18T08:58:29.341158+0000","last_became_active":"2023-10-18T08:58:28.297696+0000","last_became_peered":"2023-10-18T08:58:28.297696+0000","last_unstale":"2023-10-18T08:58:29.341158+0000","last_undegraded":"2023-10-18T08:58:29.341158+0000","last_fullsized":"2023-10-18T08:58:29.341158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:59.489387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326799+0000","last_change":"2023-10-18T08:58:29.325300+0000","last_active":"2023-10-18T08:58:29.326799+0000","last_peered":"2023-10-18T08:58:29.326799+0000","last_clean":"2023-10-18T08:58:29.326799+0000","last_became_active":"2023-10-18T08:58:29.323662+0000","last_became_peered":"2023-10-18T08:58:29.323662+0000","last_unstale":"2023-10-18T08:58:29.326799+0000","last_undegraded":"2023-10-18T08:58:29.326799+0000","last_fullsized":"2023-10-18T08:58:29.326799+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:39:03.787381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.339629+0000","last_change":"2023-10-18T08:58:28.299820+0000","last_active":"2023-10-18T08:58:29.339629+0000","last_peered":"2023-10-18T08:58:29.339629+0000","last_clean":"2023-10-18T08:58:29.339629+0000","last_became_active":"2023-10-18T08:58:28.299113+0000","last_became_peered":"2023-10-18T08:58:28.299113+0000","last_unstale":"2023-10-18T08:58:29.339629+0000","last_undegraded":"2023-10-18T08:58:29.339629+0000","last_fullsized":"2023-10-18T08:58:29.339629+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:04.709218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.332542+0000","last_change":"2023-10-18T08:58:29.332297+0000","last_active":"2023-10-18T08:58:29.332542+0000","last_peered":"2023-10-18T08:58:29.332542+0000","last_clean":"2023-10-18T08:58:29.332542+0000","last_became_active":"2023-10-18T08:58:29.332051+0000","last_became_peered":"2023-10-18T08:58:29.332051+0000","last_unstale":"2023-10-18T08:58:29.332542+0000","last_undegraded":"2023-10-18T08:58:29.332542+0000","last_fullsized":"2023-10-18T08:58:29.332542+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:30:53.581720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341445+0000","last_change":"2023-10-18T08:58:28.300048+0000","last_active":"2023-10-18T08:58:29.341445+0000","last_peered":"2023-10-18T08:58:29.341445+0000","last_clean":"2023-10-18T08:58:29.341445+0000","last_became_active":"2023-10-18T08:58:28.299438+0000","last_became_peered":"2023-10-18T08:58:28.299438+0000","last_unstale":"2023-10-18T08:58:29.341445+0000","last_undegraded":"2023-10-18T08:58:29.341445+0000","last_fullsized":"2023-10-18T08:58:29.341445+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:14.019261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.525822+0000","last_change":"2023-10-18T08:58:29.327702+0000","last_active":"2023-10-18T08:58:31.525822+0000","last_peered":"2023-10-18T08:58:31.525822+0000","last_clean":"2023-10-18T08:58:31.525822+0000","last_became_active":"2023-10-18T08:58:29.327462+0000","last_became_peered":"2023-10-18T08:58:29.327462+0000","last_unstale":"2023-10-18T08:58:31.525822+0000","last_undegraded":"2023-10-18T08:58:31.525822+0000","last_fullsized":"2023-10-18T08:58:31.525822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:51.840078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326626+0000","last_change":"2023-10-18T08:58:28.301417+0000","last_active":"2023-10-18T08:58:29.326626+0000","last_peered":"2023-10-18T08:58:29.326626+0000","last_clean":"2023-10-18T08:58:29.326626+0000","last_became_active":"2023-10-18T08:58:28.300999+0000","last_became_peered":"2023-10-18T08:58:28.300999+0000","last_unstale":"2023-10-18T08:58:29.326626+0000","last_undegraded":"2023-10-18T08:58:29.326626+0000","last_fullsized":"2023-10-18T08:58:29.326626+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:44.607015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326387+0000","last_change":"2023-10-18T08:58:29.321607+0000","last_active":"2023-10-18T08:58:29.326387+0000","last_peered":"2023-10-18T08:58:29.326387+0000","last_clean":"2023-10-18T08:58:29.326387+0000","last_became_active":"2023-10-18T08:58:29.321062+0000","last_became_peered":"2023-10-18T08:58:29.321062+0000","last_unstale":"2023-10-18T08:58:29.326387+0000","last_undegraded":"2023-10-18T08:58:29.326387+0000","last_fullsized":"2023-10-18T08:58:29.326387+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:20.541018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.319303+0000","last_change":"2023-10-18T08:58:28.302109+0000","last_active":"2023-10-18T08:58:29.319303+0000","last_peered":"2023-10-18T08:58:29.319303+0000","last_clean":"2023-10-18T08:58:29.319303+0000","last_became_active":"2023-10-18T08:58:28.301703+0000","last_became_peered":"2023-10-18T08:58:28.301703+0000","last_unstale":"2023-10-18T08:58:29.319303+0000","last_undegraded":"2023-10-18T08:58:29.319303+0000","last_fullsized":"2023-10-18T08:58:29.319303+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:23.982914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.488091+0000","last_change":"2023-10-18T08:58:29.331075+0000","last_active":"2023-10-18T08:58:30.488091+0000","last_peered":"2023-10-18T08:58:30.488091+0000","last_clean":"2023-10-18T08:58:30.488091+0000","last_became_active":"2023-10-18T08:58:29.326251+0000","last_became_peered":"2023-10-18T08:58:29.326251+0000","last_unstale":"2023-10-18T08:58:30.488091+0000","last_undegraded":"2023-10-18T08:58:30.488091+0000","last_fullsized":"2023-10-18T08:58:30.488091+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:47.848978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.305138+0000","last_change":"2023-10-18T08:58:28.298815+0000","last_active":"2023-10-18T08:58:29.305138+0000","last_peered":"2023-10-18T08:58:29.305138+0000","last_clean":"2023-10-18T08:58:29.305138+0000","last_became_active":"2023-10-18T08:58:28.298503+0000","last_became_peered":"2023-10-18T08:58:28.298503+0000","last_unstale":"2023-10-18T08:58:29.305138+0000","last_undegraded":"2023-10-18T08:58:29.305138+0000","last_fullsized":"2023-10-18T08:58:29.305138+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:39.750254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313431+0000","last_change":"2023-10-18T08:58:29.312112+0000","last_active":"2023-10-18T08:58:29.313431+0000","last_peered":"2023-10-18T08:58:29.313431+0000","last_clean":"2023-10-18T08:58:29.313431+0000","last_became_active":"2023-10-18T08:58:29.310708+0000","last_became_peered":"2023-10-18T08:58:29.310708+0000","last_unstale":"2023-10-18T08:58:29.313431+0000","last_undegraded":"2023-10-18T08:58:29.313431+0000","last_fullsized":"2023-10-18T08:58:29.313431+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:24.996549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.306615+0000","last_change":"2023-10-18T08:58:28.298222+0000","last_active":"2023-10-18T08:58:29.306615+0000","last_peered":"2023-10-18T08:58:29.306615+0000","last_clean":"2023-10-18T08:58:29.306615+0000","last_became_active":"2023-10-18T08:58:28.297672+0000","last_became_peered":"2023-10-18T08:58:28.297672+0000","last_unstale":"2023-10-18T08:58:29.306615+0000","last_undegraded":"2023-10-18T08:58:29.306615+0000","last_fullsized":"2023-10-18T08:58:29.306615+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:59.505404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.524505+0000","last_change":"2023-10-18T08:58:29.321498+0000","last_active":"2023-10-18T08:58:31.524505+0000","last_peered":"2023-10-18T08:58:31.524505+0000","last_clean":"2023-10-18T08:58:31.524505+0000","last_became_active":"2023-10-18T08:58:29.321153+0000","last_became_peered":"2023-10-18T08:58:29.321153+0000","last_unstale":"2023-10-18T08:58:31.524505+0000","last_undegraded":"2023-10-18T08:58:31.524505+0000","last_fullsized":"2023-10-18T08:58:31.524505+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:02.110113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"66'3","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.344696+0000","last_change":"2023-10-18T08:58:28.290903+0000","last_active":"2023-10-18T08:58:29.344696+0000","last_peered":"2023-10-18T08:58:29.344696+0000","last_clean":"2023-10-18T08:58:29.344696+0000","last_became_active":"2023-10-18T08:58:28.290358+0000","last_became_peered":"2023-10-18T08:58:28.290358+0000","last_unstale":"2023-10-18T08:58:29.344696+0000","last_undegraded":"2023-10-18T08:58:29.344696+0000","last_fullsized":"2023-10-18T08:58:29.344696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:24.568573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.320694+0000","last_change":"2023-10-18T08:58:29.320149+0000","last_active":"2023-10-18T08:58:29.320694+0000","last_peered":"2023-10-18T08:58:29.320694+0000","last_clean":"2023-10-18T08:58:29.320694+0000","last_became_active":"2023-10-18T08:58:29.317356+0000","last_became_peered":"2023-10-18T08:58:29.317356+0000","last_unstale":"2023-10-18T08:58:29.320694+0000","last_undegraded":"2023-10-18T08:58:29.320694+0000","last_fullsized":"2023-10-18T08:58:29.320694+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:49.888147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"66'6","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.343024+0000","last_change":"2023-10-18T08:58:28.302001+0000","last_active":"2023-10-18T08:58:29.343024+0000","last_peered":"2023-10-18T08:58:29.343024+0000","last_clean":"2023-10-18T08:58:29.343024+0000","last_became_active":"2023-10-18T08:58:28.301382+0000","last_became_peered":"2023-10-18T08:58:28.301382+0000","last_unstale":"2023-10-18T08:58:29.343024+0000","last_undegraded":"2023-10-18T08:58:29.343024+0000","last_fullsized":"2023-10-18T08:58:29.343024+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:44.607015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.487088+0000","last_change":"2023-10-18T08:58:29.325972+0000","last_active":"2023-10-18T08:58:30.487088+0000","last_peered":"2023-10-18T08:58:30.487088+0000","last_clean":"2023-10-18T08:58:30.487088+0000","last_became_active":"2023-10-18T08:58:29.325717+0000","last_became_peered":"2023-10-18T08:58:29.325717+0000","last_unstale":"2023-10-18T08:58:30.487088+0000","last_undegraded":"2023-10-18T08:58:30.487088+0000","last_fullsized":"2023-10-18T08:58:30.487088+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:49.908171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"66'2","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341261+0000","last_change":"2023-10-18T08:58:28.296421+0000","last_active":"2023-10-18T08:58:29.341261+0000","last_peered":"2023-10-18T08:58:29.341261+0000","last_clean":"2023-10-18T08:58:29.341261+0000","last_became_active":"2023-10-18T08:58:28.296176+0000","last_became_peered":"2023-10-18T08:58:28.296176+0000","last_unstale":"2023-10-18T08:58:29.341261+0000","last_undegraded":"2023-10-18T08:58:29.341261+0000","last_fullsized":"2023-10-18T08:58:29.341261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:14:17.772896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.324023+0000","last_change":"2023-10-18T08:58:29.322440+0000","last_active":"2023-10-18T08:58:29.324023+0000","last_peered":"2023-10-18T08:58:29.324023+0000","last_clean":"2023-10-18T08:58:29.324023+0000","last_became_active":"2023-10-18T08:58:29.319010+0000","last_became_peered":"2023-10-18T08:58:29.319010+0000","last_unstale":"2023-10-18T08:58:29.324023+0000","last_undegraded":"2023-10-18T08:58:29.324023+0000","last_fullsized":"2023-10-18T08:58:29.324023+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:05.722853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.315431+0000","last_change":"2023-10-18T08:58:28.290878+0000","last_active":"2023-10-18T08:58:29.315431+0000","last_peered":"2023-10-18T08:58:29.315431+0000","last_clean":"2023-10-18T08:58:29.315431+0000","last_became_active":"2023-10-18T08:58:28.290244+0000","last_became_peered":"2023-10-18T08:58:28.290244+0000","last_unstale":"2023-10-18T08:58:29.315431+0000","last_undegraded":"2023-10-18T08:58:29.315431+0000","last_fullsized":"2023-10-18T08:58:29.315431+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:41.494592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.487931+0000","last_change":"2023-10-18T08:58:29.331835+0000","last_active":"2023-10-18T08:58:30.487931+0000","last_peered":"2023-10-18T08:58:30.487931+0000","last_clean":"2023-10-18T08:58:30.487931+0000","last_became_active":"2023-10-18T08:58:29.326532+0000","last_became_peered":"2023-10-18T08:58:29.326532+0000","last_unstale":"2023-10-18T08:58:30.487931+0000","last_undegraded":"2023-10-18T08:58:30.487931+0000","last_fullsized":"2023-10-18T08:58:30.487931+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:57:23.733388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.317704+0000","last_change":"2023-10-18T08:58:28.299958+0000","last_active":"2023-10-18T08:58:29.317704+0000","last_peered":"2023-10-18T08:58:29.317704+0000","last_clean":"2023-10-18T08:58:29.317704+0000","last_became_active":"2023-10-18T08:58:28.299673+0000","last_became_peered":"2023-10-18T08:58:28.299673+0000","last_unstale":"2023-10-18T08:58:29.317704+0000","last_undegraded":"2023-10-18T08:58:29.317704+0000","last_fullsized":"2023-10-18T08:58:29.317704+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:04.709218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313474+0000","last_change":"2023-10-18T08:58:29.313209+0000","last_active":"2023-10-18T08:58:29.313474+0000","last_peered":"2023-10-18T08:58:29.313474+0000","last_clean":"2023-10-18T08:58:29.313474+0000","last_became_active":"2023-10-18T08:58:29.312599+0000","last_became_peered":"2023-10-18T08:58:29.312599+0000","last_unstale":"2023-10-18T08:58:29.313474+0000","last_undegraded":"2023-10-18T08:58:29.313474+0000","last_fullsized":"2023-10-18T08:58:29.313474+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:33:04.626358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"66'8","reported_seq":19,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.343659+0000","last_change":"2023-10-18T08:58:28.297447+0000","last_active":"2023-10-18T08:58:29.343659+0000","last_peered":"2023-10-18T08:58:29.343659+0000","last_clean":"2023-10-18T08:58:29.343659+0000","last_became_active":"2023-10-18T08:58:28.297055+0000","last_became_peered":"2023-10-18T08:58:28.297055+0000","last_unstale":"2023-10-18T08:58:29.343659+0000","last_undegraded":"2023-10-18T08:58:29.343659+0000","last_fullsized":"2023-10-18T08:58:29.343659+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:57:22.719753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326347+0000","last_change":"2023-10-18T08:58:29.326263+0000","last_active":"2023-10-18T08:58:29.326347+0000","last_peered":"2023-10-18T08:58:29.326347+0000","last_clean":"2023-10-18T08:58:29.326347+0000","last_became_active":"2023-10-18T08:58:29.325851+0000","last_became_peered":"2023-10-18T08:58:29.325851+0000","last_unstale":"2023-10-18T08:58:29.326347+0000","last_undegraded":"2023-10-18T08:58:29.326347+0000","last_fullsized":"2023-10-18T08:58:29.326347+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:42:38.047653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"66'1","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.339858+0000","last_change":"2023-10-18T08:58:28.300004+0000","last_active":"2023-10-18T08:58:29.339858+0000","last_peered":"2023-10-18T08:58:29.339858+0000","last_clean":"2023-10-18T08:58:29.339858+0000","last_became_active":"2023-10-18T08:58:28.294830+0000","last_became_peered":"2023-10-18T08:58:28.294830+0000","last_unstale":"2023-10-18T08:58:29.339858+0000","last_undegraded":"2023-10-18T08:58:29.339858+0000","last_fullsized":"2023-10-18T08:58:29.339858+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:16.750986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.486751+0000","last_change":"2023-10-18T08:58:29.332382+0000","last_active":"2023-10-18T08:58:30.486751+0000","last_peered":"2023-10-18T08:58:30.486751+0000","last_clean":"2023-10-18T08:58:30.486751+0000","last_became_active":"2023-10-18T08:58:29.330341+0000","last_became_peered":"2023-10-18T08:58:29.330341+0000","last_unstale":"2023-10-18T08:58:30.486751+0000","last_undegraded":"2023-10-18T08:58:30.486751+0000","last_fullsized":"2023-10-18T08:58:30.486751+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:47.848978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341932+0000","last_change":"2023-10-18T08:58:28.298360+0000","last_active":"2023-10-18T08:58:29.341932+0000","last_peered":"2023-10-18T08:58:29.341932+0000","last_clean":"2023-10-18T08:58:29.341932+0000","last_became_active":"2023-10-18T08:58:28.296520+0000","last_became_peered":"2023-10-18T08:58:28.296520+0000","last_unstale":"2023-10-18T08:58:29.341932+0000","last_undegraded":"2023-10-18T08:58:29.341932+0000","last_fullsized":"2023-10-18T08:58:29.341932+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:01:54.359829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326429+0000","last_change":"2023-10-18T08:58:29.324872+0000","last_active":"2023-10-18T08:58:29.326429+0000","last_peered":"2023-10-18T08:58:29.326429+0000","last_clean":"2023-10-18T08:58:29.326429+0000","last_became_active":"2023-10-18T08:58:29.323103+0000","last_became_peered":"2023-10-18T08:58:29.323103+0000","last_unstale":"2023-10-18T08:58:29.326429+0000","last_undegraded":"2023-10-18T08:58:29.326429+0000","last_fullsized":"2023-10-18T08:58:29.326429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:11:33.988349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341292+0000","last_change":"2023-10-18T08:58:28.298668+0000","last_active":"2023-10-18T08:58:29.341292+0000","last_peered":"2023-10-18T08:58:29.341292+0000","last_clean":"2023-10-18T08:58:29.341292+0000","last_became_active":"2023-10-18T08:58:28.297217+0000","last_became_peered":"2023-10-18T08:58:28.297217+0000","last_unstale":"2023-10-18T08:58:29.341292+0000","last_undegraded":"2023-10-18T08:58:29.341292+0000","last_fullsized":"2023-10-18T08:58:29.341292+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:43:35.327448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313511+0000","last_change":"2023-10-18T08:58:29.313231+0000","last_active":"2023-10-18T08:58:29.313511+0000","last_peered":"2023-10-18T08:58:29.313511+0000","last_clean":"2023-10-18T08:58:29.313511+0000","last_became_active":"2023-10-18T08:58:29.311497+0000","last_became_peered":"2023-10-18T08:58:29.311497+0000","last_unstale":"2023-10-18T08:58:29.313511+0000","last_undegraded":"2023-10-18T08:58:29.313511+0000","last_fullsized":"2023-10-18T08:58:29.313511+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:54.275856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.319166+0000","last_change":"2023-10-18T08:58:28.297688+0000","last_active":"2023-10-18T08:58:29.319166+0000","last_peered":"2023-10-18T08:58:29.319166+0000","last_clean":"2023-10-18T08:58:29.319166+0000","last_became_active":"2023-10-18T08:58:28.297401+0000","last_became_peered":"2023-10-18T08:58:28.297401+0000","last_unstale":"2023-10-18T08:58:29.319166+0000","last_undegraded":"2023-10-18T08:58:29.319166+0000","last_fullsized":"2023-10-18T08:58:29.319166+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:19.527383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.487313+0000","last_change":"2023-10-18T08:58:29.332231+0000","last_active":"2023-10-18T08:58:30.487313+0000","last_peered":"2023-10-18T08:58:30.487313+0000","last_clean":"2023-10-18T08:58:30.487313+0000","last_became_active":"2023-10-18T08:58:29.329812+0000","last_became_peered":"2023-10-18T08:58:29.329812+0000","last_unstale":"2023-10-18T08:58:30.487313+0000","last_undegraded":"2023-10-18T08:58:30.487313+0000","last_fullsized":"2023-10-18T08:58:30.487313+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:34:27.509601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"66'2","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341374+0000","last_change":"2023-10-18T08:58:28.299131+0000","last_active":"2023-10-18T08:58:29.341374+0000","last_peered":"2023-10-18T08:58:29.341374+0000","last_clean":"2023-10-18T08:58:29.341374+0000","last_became_active":"2023-10-18T08:58:28.298514+0000","last_became_peered":"2023-10-18T08:58:28.298514+0000","last_unstale":"2023-10-18T08:58:29.341374+0000","last_undegraded":"2023-10-18T08:58:29.341374+0000","last_fullsized":"2023-10-18T08:58:29.341374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:42:24.333152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.526355+0000","last_change":"2023-10-18T08:58:29.324824+0000","last_active":"2023-10-18T08:58:31.526355+0000","last_peered":"2023-10-18T08:58:31.526355+0000","last_clean":"2023-10-18T08:58:31.526355+0000","last_became_active":"2023-10-18T08:58:29.324548+0000","last_became_peered":"2023-10-18T08:58:29.324548+0000","last_unstale":"2023-10-18T08:58:31.526355+0000","last_undegraded":"2023-10-18T08:58:31.526355+0000","last_fullsized":"2023-10-18T08:58:31.526355+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:28.617230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.302848+0000","last_change":"2023-10-18T08:58:28.294327+0000","last_active":"2023-10-18T08:58:29.302848+0000","last_peered":"2023-10-18T08:58:29.302848+0000","last_clean":"2023-10-18T08:58:29.302848+0000","last_became_active":"2023-10-18T08:58:28.293852+0000","last_became_peered":"2023-10-18T08:58:28.293852+0000","last_unstale":"2023-10-18T08:58:29.302848+0000","last_undegraded":"2023-10-18T08:58:29.302848+0000","last_fullsized":"2023-10-18T08:58:29.302848+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:35.906463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.527170+0000","last_change":"2023-10-18T08:58:29.329694+0000","last_active":"2023-10-18T08:58:31.527170+0000","last_peered":"2023-10-18T08:58:31.527170+0000","last_clean":"2023-10-18T08:58:31.527170+0000","last_became_active":"2023-10-18T08:58:29.328351+0000","last_became_peered":"2023-10-18T08:58:29.328351+0000","last_unstale":"2023-10-18T08:58:31.527170+0000","last_undegraded":"2023-10-18T08:58:31.527170+0000","last_fullsized":"2023-10-18T08:58:31.527170+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:42:25.346787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.320402+0000","last_change":"2023-10-18T08:58:28.302115+0000","last_active":"2023-10-18T08:58:29.320402+0000","last_peered":"2023-10-18T08:58:29.320402+0000","last_clean":"2023-10-18T08:58:29.320402+0000","last_became_active":"2023-10-18T08:58:28.300920+0000","last_became_peered":"2023-10-18T08:58:28.300920+0000","last_unstale":"2023-10-18T08:58:29.320402+0000","last_undegraded":"2023-10-18T08:58:29.320402+0000","last_fullsized":"2023-10-18T08:58:29.320402+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:56.260888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.486648+0000","last_change":"2023-10-18T08:58:29.328074+0000","last_active":"2023-10-18T08:58:30.486648+0000","last_peered":"2023-10-18T08:58:30.486648+0000","last_clean":"2023-10-18T08:58:30.486648+0000","last_became_active":"2023-10-18T08:58:29.327486+0000","last_became_peered":"2023-10-18T08:58:29.327486+0000","last_unstale":"2023-10-18T08:58:30.486648+0000","last_undegraded":"2023-10-18T08:58:30.486648+0000","last_fullsized":"2023-10-18T08:58:30.486648+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:25.582208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341494+0000","last_change":"2023-10-18T08:58:28.280587+0000","last_active":"2023-10-18T08:58:29.341494+0000","last_peered":"2023-10-18T08:58:29.341494+0000","last_clean":"2023-10-18T08:58:29.341494+0000","last_became_active":"2023-10-18T08:58:28.280261+0000","last_became_peered":"2023-10-18T08:58:28.280261+0000","last_unstale":"2023-10-18T08:58:29.341494+0000","last_undegraded":"2023-10-18T08:58:29.341494+0000","last_fullsized":"2023-10-18T08:58:29.341494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:00.794061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.486903+0000","last_change":"2023-10-18T08:58:29.326708+0000","last_active":"2023-10-18T08:58:30.486903+0000","last_peered":"2023-10-18T08:58:30.486903+0000","last_clean":"2023-10-18T08:58:30.486903+0000","last_became_active":"2023-10-18T08:58:29.324290+0000","last_became_peered":"2023-10-18T08:58:29.324290+0000","last_unstale":"2023-10-18T08:58:30.486903+0000","last_undegraded":"2023-10-18T08:58:30.486903+0000","last_fullsized":"2023-10-18T08:58:30.486903+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:40.763889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"67'1","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.526790+0000","last_change":"2023-10-18T08:58:28.299846+0000","last_active":"2023-10-18T08:58:31.526790+0000","last_peered":"2023-10-18T08:58:31.526790+0000","last_clean":"2023-10-18T08:58:31.526790+0000","last_became_active":"2023-10-18T08:58:28.299121+0000","last_became_peered":"2023-10-18T08:58:28.299121+0000","last_unstale":"2023-10-18T08:58:31.526790+0000","last_undegraded":"2023-10-18T08:58:31.526790+0000","last_fullsized":"2023-10-18T08:58:31.526790+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:25.132096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.525751+0000","last_change":"2023-10-18T08:58:29.326192+0000","last_active":"2023-10-18T08:58:31.525751+0000","last_peered":"2023-10-18T08:58:31.525751+0000","last_clean":"2023-10-18T08:58:31.525751+0000","last_became_active":"2023-10-18T08:58:29.325922+0000","last_became_peered":"2023-10-18T08:58:29.325922+0000","last_unstale":"2023-10-18T08:58:31.525751+0000","last_undegraded":"2023-10-18T08:58:31.525751+0000","last_fullsized":"2023-10-18T08:58:31.525751+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:53:50.071506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.314770+0000","last_change":"2023-10-18T08:58:28.300674+0000","last_active":"2023-10-18T08:58:29.314770+0000","last_peered":"2023-10-18T08:58:29.314770+0000","last_clean":"2023-10-18T08:58:29.314770+0000","last_became_active":"2023-10-18T08:58:28.300119+0000","last_became_peered":"2023-10-18T08:58:28.300119+0000","last_unstale":"2023-10-18T08:58:29.314770+0000","last_undegraded":"2023-10-18T08:58:29.314770+0000","last_fullsized":"2023-10-18T08:58:29.314770+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:06.911860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.527116+0000","last_change":"2023-10-18T08:58:29.333041+0000","last_active":"2023-10-18T08:58:31.527116+0000","last_peered":"2023-10-18T08:58:31.527116+0000","last_clean":"2023-10-18T08:58:31.527116+0000","last_became_active":"2023-10-18T08:58:29.332765+0000","last_became_peered":"2023-10-18T08:58:29.332765+0000","last_unstale":"2023-10-18T08:58:31.527116+0000","last_undegraded":"2023-10-18T08:58:31.527116+0000","last_fullsized":"2023-10-18T08:58:31.527116+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:11:33.988349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.315366+0000","last_change":"2023-10-18T08:58:28.303209+0000","last_active":"2023-10-18T08:58:29.315366+0000","last_peered":"2023-10-18T08:58:29.315366+0000","last_clean":"2023-10-18T08:58:29.315366+0000","last_became_active":"2023-10-18T08:58:28.302960+0000","last_became_peered":"2023-10-18T08:58:28.302960+0000","last_unstale":"2023-10-18T08:58:29.315366+0000","last_undegraded":"2023-10-18T08:58:29.315366+0000","last_fullsized":"2023-10-18T08:58:29.315366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:48.001530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.316386+0000","last_change":"2023-10-18T08:58:29.315776+0000","last_active":"2023-10-18T08:58:29.316386+0000","last_peered":"2023-10-18T08:58:29.316386+0000","last_clean":"2023-10-18T08:58:29.316386+0000","last_became_active":"2023-10-18T08:58:29.315154+0000","last_became_peered":"2023-10-18T08:58:29.315154+0000","last_unstale":"2023-10-18T08:58:29.316386+0000","last_undegraded":"2023-10-18T08:58:29.316386+0000","last_fullsized":"2023-10-18T08:58:29.316386+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:51.344108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.301965+0000","last_change":"2023-10-18T08:58:28.295800+0000","last_active":"2023-10-18T08:58:29.301965+0000","last_peered":"2023-10-18T08:58:29.301965+0000","last_clean":"2023-10-18T08:58:29.301965+0000","last_became_active":"2023-10-18T08:58:28.295520+0000","last_became_peered":"2023-10-18T08:58:28.295520+0000","last_unstale":"2023-10-18T08:58:29.301965+0000","last_undegraded":"2023-10-18T08:58:29.301965+0000","last_fullsized":"2023-10-18T08:58:29.301965+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:39:14.200132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313850+0000","last_change":"2023-10-18T08:58:29.313510+0000","last_active":"2023-10-18T08:58:29.313850+0000","last_peered":"2023-10-18T08:58:29.313850+0000","last_clean":"2023-10-18T08:58:29.313850+0000","last_became_active":"2023-10-18T08:58:29.312697+0000","last_became_peered":"2023-10-18T08:58:29.312697+0000","last_unstale":"2023-10-18T08:58:29.313850+0000","last_undegraded":"2023-10-18T08:58:29.313850+0000","last_fullsized":"2023-10-18T08:58:29.313850+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:47.848978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313207+0000","last_change":"2023-10-18T08:58:28.300006+0000","last_active":"2023-10-18T08:58:29.313207+0000","last_peered":"2023-10-18T08:58:29.313207+0000","last_clean":"2023-10-18T08:58:29.313207+0000","last_became_active":"2023-10-18T08:58:28.299510+0000","last_became_peered":"2023-10-18T08:58:28.299510+0000","last_unstale":"2023-10-18T08:58:29.313207+0000","last_undegraded":"2023-10-18T08:58:29.313207+0000","last_fullsized":"2023-10-18T08:58:29.313207+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:56.260888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.524868+0000","last_change":"2023-10-18T08:58:29.319656+0000","last_active":"2023-10-18T08:58:31.524868+0000","last_peered":"2023-10-18T08:58:31.524868+0000","last_clean":"2023-10-18T08:58:31.524868+0000","last_became_active":"2023-10-18T08:58:29.319370+0000","last_became_peered":"2023-10-18T08:58:29.319370+0000","last_unstale":"2023-10-18T08:58:31.524868+0000","last_undegraded":"2023-10-18T08:58:31.524868+0000","last_fullsized":"2023-10-18T08:58:31.524868+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:43.446552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.300554+0000","last_change":"2023-10-18T08:58:28.297980+0000","last_active":"2023-10-18T08:58:29.300554+0000","last_peered":"2023-10-18T08:58:29.300554+0000","last_clean":"2023-10-18T08:58:29.300554+0000","last_became_active":"2023-10-18T08:58:28.297743+0000","last_became_peered":"2023-10-18T08:58:28.297743+0000","last_unstale":"2023-10-18T08:58:29.300554+0000","last_undegraded":"2023-10-18T08:58:29.300554+0000","last_fullsized":"2023-10-18T08:58:29.300554+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:04.513124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.318289+0000","last_change":"2023-10-18T08:58:29.318110+0000","last_active":"2023-10-18T08:58:29.318289+0000","last_peered":"2023-10-18T08:58:29.318289+0000","last_clean":"2023-10-18T08:58:29.318289+0000","last_became_active":"2023-10-18T08:58:29.317066+0000","last_became_peered":"2023-10-18T08:58:29.317066+0000","last_unstale":"2023-10-18T08:58:29.318289+0000","last_undegraded":"2023-10-18T08:58:29.318289+0000","last_fullsized":"2023-10-18T08:58:29.318289+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:54.004578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.299504+0000","last_change":"2023-10-18T08:58:28.300829+0000","last_active":"2023-10-18T08:58:29.299504+0000","last_peered":"2023-10-18T08:58:29.299504+0000","last_clean":"2023-10-18T08:58:29.299504+0000","last_became_active":"2023-10-18T08:58:28.300515+0000","last_became_peered":"2023-10-18T08:58:28.300515+0000","last_unstale":"2023-10-18T08:58:29.299504+0000","last_undegraded":"2023-10-18T08:58:29.299504+0000","last_fullsized":"2023-10-18T08:58:29.299504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:15:03.965062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.331823+0000","last_change":"2023-10-18T08:58:29.331231+0000","last_active":"2023-10-18T08:58:29.331823+0000","last_peered":"2023-10-18T08:58:29.331823+0000","last_clean":"2023-10-18T08:58:29.331823+0000","last_became_active":"2023-10-18T08:58:29.331003+0000","last_became_peered":"2023-10-18T08:58:29.331003+0000","last_unstale":"2023-10-18T08:58:29.331823+0000","last_undegraded":"2023-10-18T08:58:29.331823+0000","last_fullsized":"2023-10-18T08:58:29.331823+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:48.691583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.315179+0000","last_change":"2023-10-18T08:58:28.300119+0000","last_active":"2023-10-18T08:58:29.315179+0000","last_peered":"2023-10-18T08:58:29.315179+0000","last_clean":"2023-10-18T08:58:29.315179+0000","last_became_active":"2023-10-18T08:58:28.299753+0000","last_became_peered":"2023-10-18T08:58:28.299753+0000","last_unstale":"2023-10-18T08:58:29.315179+0000","last_undegraded":"2023-10-18T08:58:29.315179+0000","last_fullsized":"2023-10-18T08:58:29.315179+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:53:49.057871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":214,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-10-18T08:58:28.275442+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:58:28.275442+0000","last_peered":"2023-10-18T08:58:28.275442+0000","last_clean":"2023-10-18T08:58:28.275442+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:58:28.275442+0000","last_undegraded":"2023-10-18T08:58:28.275442+0000","last_fullsized":"2023-10-18T08:58:28.275442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:53.386532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":902,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194409,"statfs":{"total":200000000,"available":199075709,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.99299999999999999,"15min":0.98899999999999999},"min":{"1min":0.70299999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3400000000000001,"5min":1.538,"15min":1.538},"last":1.222},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.034,"15min":1.008},"min":{"1min":0.69599999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4019999999999999,"5min":1.667,"15min":1.667},"last":1.048}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1240000000000001,"15min":1.147},"min":{"1min":0.80300000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5900000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.375},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1419999999999999,"15min":1.1220000000000001},"min":{"1min":0.748,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7450000000000001,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.5920000000000001}]},{"osd":2,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.2450000000000001,"15min":1.2330000000000001},"min":{"1min":0.80200000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.284},{"interface":"front","average":{"1min":1.276,"5min":1.2010000000000001,"15min":1.1910000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.798,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.3320000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2629999999999999,"15min":1.264},"min":{"1min":0.95999999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.702,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3140000000000001,"15min":1.3149999999999999},"min":{"1min":0.94199999999999995,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.873,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.478}]},{"osd":4,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1899999999999999,"15min":1.1599999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6799999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.3779999999999999,"15min":1.401},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.6279999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.534}]},{"osd":5,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.353,"15min":1.353},"min":{"1min":0.99399999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.111},{"interface":"front","average":{"1min":1.349,"5min":1.4019999999999999,"15min":1.4159999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.865,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.4430000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.464,"15min":1.458},"min":{"1min":1.0369999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.923,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.518},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.492,"15min":1.51},"min":{"1min":1.0669999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.8069999999999999,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.633}]}]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":338,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194973,"statfs":{"total":200000000,"available":199653002,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1020000000000001,"15min":1.0800000000000001},"min":{"1min":0.623,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.651,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.298,"5min":1.228,"15min":1.234},"min":{"1min":0.84499999999999997,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.244}]},{"osd":1,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.089,"15min":1.085},"min":{"1min":0.65600000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1799999999999999,"15min":1.1859999999999999},"min":{"1min":0.80600000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.962,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":1.5940000000000001}]},{"osd":2,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.161,"15min":1.173},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5029999999999999,"5min":1.696,"15min":1.696},"last":1.365},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.1970000000000001,"15min":1.194},"min":{"1min":0.92500000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.512}]},{"osd":3,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2509999999999999,"15min":1.2569999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.669,"5min":1.784,"15min":1.784},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.228,"15min":1.22},"min":{"1min":0.84999999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.095}]},{"osd":4,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.212,"15min":1.175},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.26,"15min":1.266},"min":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.696,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.1679999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3520000000000001,"15min":1.3480000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.599,"5min":1.667,"15min":1.667},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.238,"15min":1.218},"min":{"1min":0.629,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.52,"5min":1.746,"15min":1.746},"last":1.032}]},{"osd":7,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.4039999999999999,"15min":1.3959999999999999},"min":{"1min":0.73999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.51,"15min":1.5249999999999999},"min":{"1min":0.84099999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.831,"5min":2.794,"15min":2.794},"last":1.538}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":339,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194972,"statfs":{"total":200000000,"available":199652350,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.7529999999999999,"5min":1.3160000000000001,"15min":1.224},"min":{"1min":0.68300000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":9.3539999999999992,"5min":9.3539999999999992,"15min":9.3539999999999992},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.3220000000000001,"15min":1.3029999999999999},"min":{"1min":0.71999999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":5.3029999999999999,"5min":5.3029999999999999,"15min":5.3029999999999999},"last":1.4630000000000001}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2090000000000001,"15min":1.2230000000000001},"min":{"1min":0.755,"5min":0.749,"15min":0.749},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.222,"15min":1.248},"min":{"1min":0.82199999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.367,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.88500000000000001}]},{"osd":2,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.268,"15min":1.268},"min":{"1min":0.78100000000000003,"5min":0.755,"15min":0.755},"max":{"1min":1.5309999999999999,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.2889999999999999,"15min":1.2929999999999999},"min":{"1min":0.78400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.3069999999999999,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.139}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.25,"15min":1.246},"min":{"1min":0.93300000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.524,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":1.349},{"interface":"front","average":{"1min":1.409,"5min":1.337,"15min":1.3320000000000001},"min":{"1min":0.94399999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.6379999999999999,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.407}]},{"osd":4,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4139999999999999,"15min":1.419},"min":{"1min":0.93400000000000005,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.881,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.7869999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.4950000000000001,"15min":1.5329999999999999},"min":{"1min":0.90000000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":4.4429999999999996,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":1.3169999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.377,"15min":1.4119999999999999},"min":{"1min":0.93899999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.5469999999999997,"5min":5.141,"15min":5.141},"last":1.496},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.556,"15min":1.587},"min":{"1min":1.151,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0590000000000002,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.6930000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.6419999999999999,"15min":1.6699999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":4.2050000000000001,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":1.591},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5149999999999999,"15min":1.5249999999999999},"min":{"1min":1.0249999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":4.4690000000000003,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":1.633}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":339,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194973,"statfs":{"total":200000000,"available":199652409,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.107,"15min":1.1100000000000001},"min":{"1min":0.745,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.996,"5min":1.129,"15min":1.169},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.6990000000000001,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.2050000000000001}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.4849999999999999,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2250000000000001,"15min":1.2390000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.91100000000000003}]},{"osd":2,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1879999999999999,"15min":1.179},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.27,"15min":1.3029999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.994,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.04}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2889999999999999,"15min":1.3009999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.4179999999999999,"15min":1.4139999999999999},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.6880000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.575}]},{"osd":5,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.323,"15min":1.304},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.778,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.351,"15min":1.367},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.359}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.4339999999999999,"15min":1.3939999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.556,"5min":1.5629999999999999,"15min":1.5760000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":1.3959999999999999}]},{"osd":7,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.4950000000000001,"15min":1.514},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5529999999999999,"15min":1.5509999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.585,"5min":2.585,"15min":2.585},"last":1.7250000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":326,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194986,"statfs":{"total":200000000,"available":199666077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.85199999999999998,"15min":0.77400000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.304,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.95199999999999996,"15min":0.873},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.393,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":0.46600000000000003}]},{"osd":1,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.97799999999999998,"15min":0.94199999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.208,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.1419999999999999,"15min":1.113},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.5660000000000001,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.258}]},{"osd":2,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.0549999999999999,"15min":1.0149999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4330000000000001,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1080000000000001,"15min":1.054},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6459999999999999,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":1.3149999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.464,"15min":1.3049999999999999},"min":{"1min":0.83599999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.5649999999999999,"5min":18.664999999999999,"15min":18.664999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.21,"15min":1.1499999999999999},"min":{"1min":0.69099999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.756,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":1.0149999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.276,"15min":1.2170000000000001},"min":{"1min":0.72399999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.702,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":0.87},{"interface":"front","average":{"1min":1.425,"5min":1.502,"15min":1.3400000000000001},"min":{"1min":0.80800000000000005,"5min":0.748,"15min":0.748},"max":{"1min":1.885,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.375}]},{"osd":6,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4750000000000001,"15min":1.4199999999999999},"min":{"1min":0.82499999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.7649999999999999,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3959999999999999,"15min":1.3300000000000001},"min":{"1min":0.77200000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.774,"5min":7.242,"15min":7.242},"last":1.103}]},{"osd":7,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.4019999999999999,"15min":1.3320000000000001},"min":{"1min":0.80600000000000005,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.9970000000000001,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.431},{"interface":"front","average":{"1min":1.47,"5min":1.556,"15min":1.4470000000000001},"min":{"1min":0.93100000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.8580000000000001,"5min":12.946,"15min":12.946},"last":0.84199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":338,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194973,"statfs":{"total":200000000,"available":199652946,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90500000000000003,"15min":0.93500000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.55,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":1.25},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0029999999999999,"15min":1.05},"min":{"1min":0.63100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4870000000000001,"5min":1.651,"15min":1.651},"last":0.80400000000000005}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.0249999999999999,"15min":1.0549999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.427,"5min":4.29,"15min":4.29},"last":1.085},{"interface":"front","average":{"1min":1.198,"5min":1.0760000000000001,"15min":1.095},"min":{"1min":0.65700000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.583,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.0589999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0840000000000001,"15min":1.1259999999999999},"min":{"1min":0.54600000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.6419999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.391,"15min":1.48},"min":{"1min":0.57899999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8360000000000001,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":1.373}]},{"osd":4,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.448,"15min":1.4950000000000001},"min":{"1min":0.95699999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.7450000000000001,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.415,"15min":1.45},"min":{"1min":0.93799999999999994,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.8029999999999999,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.536}]},{"osd":5,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.234,"15min":1.268},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6719999999999999,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.5,"15min":1.5760000000000001},"min":{"1min":0.96099999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.863,"5min":7.657,"15min":7.657},"last":1.6060000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.472,"15min":1.5309999999999999},"min":{"1min":0.82699999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8999999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.171},{"interface":"front","average":{"1min":3.536,"5min":1.8240000000000001,"15min":1.5720000000000001},"min":{"1min":0.98899999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":37.771999999999998,"5min":37.771999999999998,"15min":37.771999999999998},"last":1.4790000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.5309999999999999,"15min":1.573},"min":{"1min":0.98699999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.016,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.4410000000000001,"15min":1.4710000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1349999999999998,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.575}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":337,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194974,"statfs":{"total":200000000,"available":199653931,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.79200000000000004,"15min":0.79800000000000004},"min":{"1min":0.44500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":1.327,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.165},{"interface":"front","average":{"1min":0.999,"5min":0.89300000000000002,"15min":0.89100000000000001},"min":{"1min":0.501,"5min":0.443,"15min":0.443},"max":{"1min":1.5129999999999999,"5min":1.55,"15min":1.55},"last":1.6830000000000001}]},{"osd":2,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.91100000000000003,"15min":0.92300000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6850000000000001,"5min":1.859,"15min":1.859},"last":1.6060000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.83299999999999996,"15min":0.83499999999999996},"min":{"1min":0.47999999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.625,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.3140000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.98199999999999998,"15min":1.008},"min":{"1min":0.52800000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.571,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.016},{"interface":"front","average":{"1min":1.052,"5min":1.0229999999999999,"15min":1.0469999999999999},"min":{"1min":0.54500000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.8859999999999999,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":1.4339999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.22,"15min":1.2350000000000001},"min":{"1min":0.82099999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5760000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":2.0489999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1970000000000001,"15min":1.1930000000000001},"min":{"1min":0.93799999999999994,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.556,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.8280000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.268,"15min":1.2809999999999999},"min":{"1min":0.86599999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.625,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.948},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.343,"15min":1.343},"min":{"1min":0.90500000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.6759999999999999,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.881}]},{"osd":6,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2529999999999999,"15min":1.2649999999999999},"min":{"1min":0.89100000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6240000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2450000000000001,"15min":1.248},"min":{"1min":0.78900000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.744,"5min":1.853,"15min":1.853},"last":2.1070000000000002}]},{"osd":7,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.333,"15min":1.339},"min":{"1min":0.84399999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.716,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":2.2229999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3380000000000001,"15min":1.3400000000000001},"min":{"1min":0.95099999999999996,"5min":0.746,"15min":0.746},"max":{"1min":1.758,"5min":1.994,"15min":1.994},"last":2.4180000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":916,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194395,"statfs":{"total":200000000,"available":199061459,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.89500000000000002,"15min":0.93600000000000005},"min":{"1min":0.371,"5min":0.37,"15min":0.37},"max":{"1min":1.0940000000000001,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0660000000000001,"15min":1.079},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.181}]},{"osd":2,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.121,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.1240000000000001,"15min":1.125},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.8109999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.01}]},{"osd":3,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.22,"15min":1.306},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6639999999999999,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.4750000000000001,"15min":1.5309999999999999},"min":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.9259999999999999,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.1180000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2629999999999999,"15min":1.3169999999999999},"min":{"1min":0.81200000000000006,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.5549999999999999,"5min":4.492,"15min":4.492},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.234,"5min":1.2490000000000001,"15min":1.2470000000000001},"min":{"1min":0.93200000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.629,"5min":1.726,"15min":1.726},"last":1.552}]},{"osd":5,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.377,"15min":1.4490000000000001},"min":{"1min":0.89500000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.944,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.071},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3580000000000001,"15min":1.3859999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8240000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.6559999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.5629999999999999,"15min":1.6100000000000001},"min":{"1min":0.877,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.0409999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.4670000000000001,"15min":1.5189999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.863,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.2769999999999999}]},{"osd":7,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.619,"15min":1.661},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.1160000000000001,"5min":3.492,"15min":3.492},"last":1.359},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.621,"15min":1.7130000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.0449999999999999,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.2290000000000001}]}]}],"pool_statfs":[]}} 2023-10-18T08:58:38.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T08:58:39.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.185+0000 7fb6bb59e640 1 -- 172.21.15.40:0/3764718324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6bc152560 msgr2=0x7fb6bc070f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:39.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.185+0000 7fb6bb59e640 1 --2- 172.21.15.40:0/3764718324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6bc152560 0x7fb6bc070f00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc06d930 tx=0x7fb6ac02be20 comp rx=0 tx=0).stop 2023-10-18T08:58:39.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.185+0000 7fb6bb59e640 1 -- 172.21.15.40:0/3764718324 shutdown_connections 2023-10-18T08:58:39.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.185+0000 7fb6bb59e640 1 --2- 172.21.15.40:0/3764718324 >> v2:172.21.15.144:3300/0 conn(0x7fb6bc0ab0d0 0x7fb6bc06b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:39.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.185+0000 7fb6bb59e640 1 --2- 172.21.15.40:0/3764718324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6bc071440 0x7fb6bc0aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:39.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.185+0000 7fb6bb59e640 1 --2- 172.21.15.40:0/3764718324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6bc152560 0x7fb6bc070f00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:39.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.185+0000 7fb6bb59e640 1 -- 172.21.15.40:0/3764718324 >> 172.21.15.40:0/3764718324 conn(0x7fb6bc069720 msgr2=0x7fb6bc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:39.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.185+0000 7fb6bb59e640 1 -- 172.21.15.40:0/3764718324 shutdown_connections 2023-10-18T08:58:39.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.185+0000 7fb6bb59e640 1 -- 172.21.15.40:0/3764718324 wait complete. 2023-10-18T08:58:39.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.187+0000 7fb6bb59e640 1 Processor -- start 2023-10-18T08:58:39.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.187+0000 7fb6bb59e640 1 -- start start 2023-10-18T08:58:39.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.188+0000 7fb6bb59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6bc071440 0x7fb6bc14a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:39.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.188+0000 7fb6bb59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6bc0ab0d0 0x7fb6bc14f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:39.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.188+0000 7fb6bb59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6bc152560 0x7fb6bc14fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:39.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.188+0000 7fb6bb59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc154480 con 0x7fb6bc152560 2023-10-18T08:58:39.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.188+0000 7fb6bb59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc154760 con 0x7fb6bc071440 2023-10-18T08:58:39.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.188+0000 7fb6bb59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc154a40 con 0x7fb6bc0ab0d0 2023-10-18T08:58:39.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.188+0000 7fb6bad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6bc152560 0x7fb6bc14fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:39.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.188+0000 7fb6bad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6bc152560 0x7fb6bc14fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43362/0 (socket says 172.21.15.40:43362) 2023-10-18T08:58:39.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.188+0000 7fb6bad9d640 1 -- 172.21.15.40:0/3062216625 learned_addr learned my addr 172.21.15.40:0/3062216625 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:39.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.189+0000 7fb6ba59c640 1 --2- 172.21.15.40:0/3062216625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6bc071440 0x7fb6bc14a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:39.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.189+0000 7fb6bad9d640 1 -- 172.21.15.40:0/3062216625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6bc0ab0d0 msgr2=0x7fb6bc14f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:39.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.189+0000 7fb6b3fff640 1 --2- 172.21.15.40:0/3062216625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6bc0ab0d0 0x7fb6bc14f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:39.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.189+0000 7fb6bad9d640 1 --2- 172.21.15.40:0/3062216625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6bc0ab0d0 0x7fb6bc14f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:39.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.189+0000 7fb6bad9d640 1 -- 172.21.15.40:0/3062216625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6bc071440 msgr2=0x7fb6bc14a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:39.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.189+0000 7fb6bad9d640 1 --2- 172.21.15.40:0/3062216625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6bc071440 0x7fb6bc14a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:39.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.189+0000 7fb6bad9d640 1 -- 172.21.15.40:0/3062216625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac009560 con 0x7fb6bc152560 2023-10-18T08:58:39.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.189+0000 7fb6ba59c640 1 --2- 172.21.15.40:0/3062216625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6bc071440 0x7fb6bc14a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:39.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.189+0000 7fb6bad9d640 1 --2- 172.21.15.40:0/3062216625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6bc152560 0x7fb6bc14fa90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400eaa0 tx=0x7fb6b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:39.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.189+0000 7fb6b2ffd640 1 -- 172.21.15.40:0/3062216625 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b400cdb0 con 0x7fb6bc152560 2023-10-18T08:58:39.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.190+0000 7fb6b2ffd640 1 -- 172.21.15.40:0/3062216625 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb6b4004b20 con 0x7fb6bc152560 2023-10-18T08:58:39.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.190+0000 7fb6bb59e640 1 -- 172.21.15.40:0/3062216625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6bc154d20 con 0x7fb6bc152560 2023-10-18T08:58:39.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.190+0000 7fb6b2ffd640 1 -- 172.21.15.40:0/3062216625 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4005020 con 0x7fb6bc152560 2023-10-18T08:58:39.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.190+0000 7fb6bb59e640 1 -- 172.21.15.40:0/3062216625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6bc155520 con 0x7fb6bc152560 2023-10-18T08:58:39.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.191+0000 7fb6bb59e640 1 -- 172.21.15.40:0/3062216625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6bc066e00 con 0x7fb6bc152560 2023-10-18T08:58:39.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.192+0000 7fb6b2ffd640 1 -- 172.21.15.40:0/3062216625 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb6b4005180 con 0x7fb6bc152560 2023-10-18T08:58:39.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.193+0000 7fb6b2ffd640 1 --2- 172.21.15.40:0/3062216625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb69c06ae40 0x7fb69c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:39.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.193+0000 7fb6b2ffd640 1 -- 172.21.15.40:0/3062216625 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6b4089ff0 con 0x7fb6bc152560 2023-10-18T08:58:39.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.194+0000 7fb6ba59c640 1 --2- 172.21.15.40:0/3062216625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb69c06ae40 0x7fb69c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:39.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.195+0000 7fb6ba59c640 1 --2- 172.21.15.40:0/3062216625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb69c06ae40 0x7fb69c06d300 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac035000 tx=0x7fb6ac004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:39.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.198+0000 7fb6b2ffd640 1 -- 172.21.15.40:0/3062216625 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb6b40550a0 con 0x7fb6bc152560 2023-10-18T08:58:39.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.389+0000 7fb6bb59e640 1 -- 172.21.15.40:0/3062216625 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6bc0ab500 con 0x7fb69c06ae40 2023-10-18T08:58:39.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.413+0000 7fb6b2ffd640 1 -- 172.21.15.40:0/3062216625 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282536 (secure 0 0 0) 0x7fb6ac004280 con 0x7fb69c06ae40 2023-10-18T08:58:39.417 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:39.420 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T08:58:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.419+0000 7fb6bb59e640 1 -- 172.21.15.40:0/3062216625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb69c06ae40 msgr2=0x7fb69c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.419+0000 7fb6bb59e640 1 --2- 172.21.15.40:0/3062216625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb69c06ae40 0x7fb69c06d300 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac035000 tx=0x7fb6ac004730 comp rx=0 tx=0).stop 2023-10-18T08:58:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.419+0000 7fb6bb59e640 1 -- 172.21.15.40:0/3062216625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6bc152560 msgr2=0x7fb6bc14fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.419+0000 7fb6bb59e640 1 --2- 172.21.15.40:0/3062216625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6bc152560 0x7fb6bc14fa90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400eaa0 tx=0x7fb6b400ef60 comp rx=0 tx=0).stop 2023-10-18T08:58:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.419+0000 7fb6bb59e640 1 -- 172.21.15.40:0/3062216625 shutdown_connections 2023-10-18T08:58:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.419+0000 7fb6bb59e640 1 --2- 172.21.15.40:0/3062216625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb69c06ae40 0x7fb69c06d300 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.419+0000 7fb6bb59e640 1 --2- 172.21.15.40:0/3062216625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6bc152560 0x7fb6bc14fa90 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.419+0000 7fb6bb59e640 1 --2- 172.21.15.40:0/3062216625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6bc0ab0d0 0x7fb6bc14f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.419+0000 7fb6bb59e640 1 --2- 172.21.15.40:0/3062216625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6bc071440 0x7fb6bc14a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.419+0000 7fb6bb59e640 1 -- 172.21.15.40:0/3062216625 >> 172.21.15.40:0/3062216625 conn(0x7fb6bc069720 msgr2=0x7fb6bc15d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.419+0000 7fb6bb59e640 1 -- 172.21.15.40:0/3062216625 shutdown_connections 2023-10-18T08:58:39.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.419+0000 7fb6bb59e640 1 -- 172.21.15.40:0/3062216625 wait complete. 2023-10-18T08:58:39.441 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2023-10-18T08:58:38.967233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3865,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558634,"statfs":{"total":1600000000,"available":1596041517,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.670067"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.301202+0000","last_change":"2023-10-18T08:58:29.324418+0000","last_active":"2023-10-18T08:58:30.301202+0000","last_peered":"2023-10-18T08:58:30.301202+0000","last_clean":"2023-10-18T08:58:30.301202+0000","last_became_active":"2023-10-18T08:58:29.322273+0000","last_became_peered":"2023-10-18T08:58:29.322273+0000","last_unstale":"2023-10-18T08:58:30.301202+0000","last_undegraded":"2023-10-18T08:58:30.301202+0000","last_fullsized":"2023-10-18T08:58:30.301202+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:20:15.523189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.299429+0000","last_change":"2023-10-18T08:58:29.310661+0000","last_active":"2023-10-18T08:58:30.299429+0000","last_peered":"2023-10-18T08:58:30.299429+0000","last_clean":"2023-10-18T08:58:30.299429+0000","last_became_active":"2023-10-18T08:58:29.309557+0000","last_became_peered":"2023-10-18T08:58:29.309557+0000","last_unstale":"2023-10-18T08:58:30.299429+0000","last_undegraded":"2023-10-18T08:58:30.299429+0000","last_fullsized":"2023-10-18T08:58:30.299429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:23:36.939199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.301643+0000","last_change":"2023-10-18T08:58:29.321404+0000","last_active":"2023-10-18T08:58:30.301643+0000","last_peered":"2023-10-18T08:58:30.301643+0000","last_clean":"2023-10-18T08:58:30.301643+0000","last_became_active":"2023-10-18T08:58:29.320985+0000","last_became_peered":"2023-10-18T08:58:29.320985+0000","last_unstale":"2023-10-18T08:58:30.301643+0000","last_undegraded":"2023-10-18T08:58:30.301643+0000","last_fullsized":"2023-10-18T08:58:30.301643+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:49.908171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.486351+0000","last_change":"2023-10-18T08:58:29.332571+0000","last_active":"2023-10-18T08:58:30.486351+0000","last_peered":"2023-10-18T08:58:30.486351+0000","last_clean":"2023-10-18T08:58:30.486351+0000","last_became_active":"2023-10-18T08:58:29.330694+0000","last_became_peered":"2023-10-18T08:58:29.330694+0000","last_unstale":"2023-10-18T08:58:30.486351+0000","last_undegraded":"2023-10-18T08:58:30.486351+0000","last_fullsized":"2023-10-18T08:58:30.486351+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:49.888147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.301195+0000","last_change":"2023-10-18T08:58:29.327743+0000","last_active":"2023-10-18T08:58:30.301195+0000","last_peered":"2023-10-18T08:58:30.301195+0000","last_clean":"2023-10-18T08:58:30.301195+0000","last_became_active":"2023-10-18T08:58:29.327430+0000","last_became_peered":"2023-10-18T08:58:29.327430+0000","last_unstale":"2023-10-18T08:58:30.301195+0000","last_undegraded":"2023-10-18T08:58:30.301195+0000","last_fullsized":"2023-10-18T08:58:30.301195+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:40:49.731282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.526723+0000","last_change":"2023-10-18T08:58:29.331173+0000","last_active":"2023-10-18T08:58:31.526723+0000","last_peered":"2023-10-18T08:58:31.526723+0000","last_clean":"2023-10-18T08:58:31.526723+0000","last_became_active":"2023-10-18T08:58:29.330450+0000","last_became_peered":"2023-10-18T08:58:29.330450+0000","last_unstale":"2023-10-18T08:58:31.526723+0000","last_undegraded":"2023-10-18T08:58:31.526723+0000","last_fullsized":"2023-10-18T08:58:31.526723+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:17.764621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.487514+0000","last_change":"2023-10-18T08:58:29.331909+0000","last_active":"2023-10-18T08:58:30.487514+0000","last_peered":"2023-10-18T08:58:30.487514+0000","last_clean":"2023-10-18T08:58:30.487514+0000","last_became_active":"2023-10-18T08:58:29.331625+0000","last_became_peered":"2023-10-18T08:58:29.331625+0000","last_unstale":"2023-10-18T08:58:30.487514+0000","last_undegraded":"2023-10-18T08:58:30.487514+0000","last_fullsized":"2023-10-18T08:58:30.487514+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:39:49.888147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.299248+0000","last_change":"2023-10-18T08:58:29.325559+0000","last_active":"2023-10-18T08:58:30.299248+0000","last_peered":"2023-10-18T08:58:30.299248+0000","last_clean":"2023-10-18T08:58:30.299248+0000","last_became_active":"2023-10-18T08:58:29.324359+0000","last_became_peered":"2023-10-18T08:58:29.324359+0000","last_unstale":"2023-10-18T08:58:30.299248+0000","last_undegraded":"2023-10-18T08:58:30.299248+0000","last_fullsized":"2023-10-18T08:58:30.299248+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:57.836356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.487715+0000","last_change":"2023-10-18T08:58:29.333697+0000","last_active":"2023-10-18T08:58:30.487715+0000","last_peered":"2023-10-18T08:58:30.487715+0000","last_clean":"2023-10-18T08:58:30.487715+0000","last_became_active":"2023-10-18T08:58:29.330251+0000","last_became_peered":"2023-10-18T08:58:29.330251+0000","last_unstale":"2023-10-18T08:58:30.487715+0000","last_undegraded":"2023-10-18T08:58:30.487715+0000","last_fullsized":"2023-10-18T08:58:30.487715+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:54.275856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.525455+0000","last_change":"2023-10-18T08:58:29.319001+0000","last_active":"2023-10-18T08:58:31.525455+0000","last_peered":"2023-10-18T08:58:31.525455+0000","last_clean":"2023-10-18T08:58:31.525455+0000","last_became_active":"2023-10-18T08:58:29.318606+0000","last_became_peered":"2023-10-18T08:58:29.318606+0000","last_unstale":"2023-10-18T08:58:31.525455+0000","last_undegraded":"2023-10-18T08:58:31.525455+0000","last_fullsized":"2023-10-18T08:58:31.525455+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:48:54.439674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.525671+0000","last_change":"2023-10-18T08:58:29.307513+0000","last_active":"2023-10-18T08:58:31.525671+0000","last_peered":"2023-10-18T08:58:31.525671+0000","last_clean":"2023-10-18T08:58:31.525671+0000","last_became_active":"2023-10-18T08:58:29.306861+0000","last_became_peered":"2023-10-18T08:58:29.306861+0000","last_unstale":"2023-10-18T08:58:31.525671+0000","last_undegraded":"2023-10-18T08:58:31.525671+0000","last_fullsized":"2023-10-18T08:58:31.525671+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:07.925495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.486472+0000","last_change":"2023-10-18T08:58:29.326923+0000","last_active":"2023-10-18T08:58:30.486472+0000","last_peered":"2023-10-18T08:58:30.486472+0000","last_clean":"2023-10-18T08:58:30.486472+0000","last_became_active":"2023-10-18T08:58:29.324619+0000","last_became_peered":"2023-10-18T08:58:29.324619+0000","last_unstale":"2023-10-18T08:58:30.486472+0000","last_undegraded":"2023-10-18T08:58:30.486472+0000","last_fullsized":"2023-10-18T08:58:30.486472+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:54.275856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.526053+0000","last_change":"2023-10-18T08:58:29.329776+0000","last_active":"2023-10-18T08:58:31.526053+0000","last_peered":"2023-10-18T08:58:31.526053+0000","last_clean":"2023-10-18T08:58:31.526053+0000","last_became_active":"2023-10-18T08:58:29.329526+0000","last_became_peered":"2023-10-18T08:58:29.329526+0000","last_unstale":"2023-10-18T08:58:31.526053+0000","last_undegraded":"2023-10-18T08:58:31.526053+0000","last_fullsized":"2023-10-18T08:58:31.526053+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:09.801713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.487921+0000","last_change":"2023-10-18T08:58:29.328265+0000","last_active":"2023-10-18T08:58:30.487921+0000","last_peered":"2023-10-18T08:58:30.487921+0000","last_clean":"2023-10-18T08:58:30.487921+0000","last_became_active":"2023-10-18T08:58:29.327575+0000","last_became_peered":"2023-10-18T08:58:29.327575+0000","last_unstale":"2023-10-18T08:58:30.487921+0000","last_undegraded":"2023-10-18T08:58:30.487921+0000","last_fullsized":"2023-10-18T08:58:30.487921+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:25.582208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.488833+0000","last_change":"2023-10-18T08:58:29.331412+0000","last_active":"2023-10-18T08:58:30.488833+0000","last_peered":"2023-10-18T08:58:30.488833+0000","last_clean":"2023-10-18T08:58:30.488833+0000","last_became_active":"2023-10-18T08:58:29.330792+0000","last_became_peered":"2023-10-18T08:58:29.330792+0000","last_unstale":"2023-10-18T08:58:30.488833+0000","last_undegraded":"2023-10-18T08:58:30.488833+0000","last_fullsized":"2023-10-18T08:58:30.488833+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:40.763889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.527429+0000","last_change":"2023-10-18T08:58:29.330739+0000","last_active":"2023-10-18T08:58:31.527429+0000","last_peered":"2023-10-18T08:58:31.527429+0000","last_clean":"2023-10-18T08:58:31.527429+0000","last_became_active":"2023-10-18T08:58:29.328465+0000","last_became_peered":"2023-10-18T08:58:29.328465+0000","last_unstale":"2023-10-18T08:58:31.527429+0000","last_undegraded":"2023-10-18T08:58:31.527429+0000","last_fullsized":"2023-10-18T08:58:31.527429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:51.344108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.299826+0000","last_change":"2023-10-18T08:58:29.325090+0000","last_active":"2023-10-18T08:58:30.299826+0000","last_peered":"2023-10-18T08:58:30.299826+0000","last_clean":"2023-10-18T08:58:30.299826+0000","last_became_active":"2023-10-18T08:58:29.323675+0000","last_became_peered":"2023-10-18T08:58:29.323675+0000","last_unstale":"2023-10-18T08:58:30.299826+0000","last_undegraded":"2023-10-18T08:58:30.299826+0000","last_fullsized":"2023-10-18T08:58:30.299826+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:36.183909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.302031+0000","last_change":"2023-10-18T08:58:29.329020+0000","last_active":"2023-10-18T08:58:30.302031+0000","last_peered":"2023-10-18T08:58:30.302031+0000","last_clean":"2023-10-18T08:58:30.302031+0000","last_became_active":"2023-10-18T08:58:29.328773+0000","last_became_peered":"2023-10-18T08:58:29.328773+0000","last_unstale":"2023-10-18T08:58:30.302031+0000","last_undegraded":"2023-10-18T08:58:30.302031+0000","last_fullsized":"2023-10-18T08:58:30.302031+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:04.151527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.300662+0000","last_change":"2023-10-18T08:58:29.326282+0000","last_active":"2023-10-18T08:58:30.300662+0000","last_peered":"2023-10-18T08:58:30.300662+0000","last_clean":"2023-10-18T08:58:30.300662+0000","last_became_active":"2023-10-18T08:58:29.325556+0000","last_became_peered":"2023-10-18T08:58:29.325556+0000","last_unstale":"2023-10-18T08:58:30.300662+0000","last_undegraded":"2023-10-18T08:58:30.300662+0000","last_fullsized":"2023-10-18T08:58:30.300662+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:56:51.163636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.299994+0000","last_change":"2023-10-18T08:58:29.330497+0000","last_active":"2023-10-18T08:58:30.299994+0000","last_peered":"2023-10-18T08:58:30.299994+0000","last_clean":"2023-10-18T08:58:30.299994+0000","last_became_active":"2023-10-18T08:58:29.330245+0000","last_became_peered":"2023-10-18T08:58:29.330245+0000","last_unstale":"2023-10-18T08:58:30.299994+0000","last_undegraded":"2023-10-18T08:58:30.299994+0000","last_fullsized":"2023-10-18T08:58:30.299994+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:53:39.968475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.301736+0000","last_change":"2023-10-18T08:58:29.323242+0000","last_active":"2023-10-18T08:58:30.301736+0000","last_peered":"2023-10-18T08:58:30.301736+0000","last_clean":"2023-10-18T08:58:30.301736+0000","last_became_active":"2023-10-18T08:58:29.322832+0000","last_became_peered":"2023-10-18T08:58:29.322832+0000","last_unstale":"2023-10-18T08:58:30.301736+0000","last_undegraded":"2023-10-18T08:58:30.301736+0000","last_fullsized":"2023-10-18T08:58:30.301736+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:34:27.509601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.300206+0000","last_change":"2023-10-18T08:58:29.319011+0000","last_active":"2023-10-18T08:58:30.300206+0000","last_peered":"2023-10-18T08:58:30.300206+0000","last_clean":"2023-10-18T08:58:30.300206+0000","last_became_active":"2023-10-18T08:58:29.318536+0000","last_became_peered":"2023-10-18T08:58:29.318536+0000","last_unstale":"2023-10-18T08:58:30.300206+0000","last_undegraded":"2023-10-18T08:58:30.300206+0000","last_fullsized":"2023-10-18T08:58:30.300206+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:03:22.951934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.302340+0000","last_change":"2023-10-18T08:58:29.319576+0000","last_active":"2023-10-18T08:58:30.302340+0000","last_peered":"2023-10-18T08:58:30.302340+0000","last_clean":"2023-10-18T08:58:30.302340+0000","last_became_active":"2023-10-18T08:58:29.316842+0000","last_became_peered":"2023-10-18T08:58:29.316842+0000","last_unstale":"2023-10-18T08:58:30.302340+0000","last_undegraded":"2023-10-18T08:58:30.302340+0000","last_fullsized":"2023-10-18T08:58:30.302340+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:28:25.454956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.525555+0000","last_change":"2023-10-18T08:58:29.331123+0000","last_active":"2023-10-18T08:58:31.525555+0000","last_peered":"2023-10-18T08:58:31.525555+0000","last_clean":"2023-10-18T08:58:31.525555+0000","last_became_active":"2023-10-18T08:58:29.330917+0000","last_became_peered":"2023-10-18T08:58:29.330917+0000","last_unstale":"2023-10-18T08:58:31.525555+0000","last_undegraded":"2023-10-18T08:58:31.525555+0000","last_fullsized":"2023-10-18T08:58:31.525555+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:26:15.788187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.301380+0000","last_change":"2023-10-18T08:58:29.325403+0000","last_active":"2023-10-18T08:58:30.301380+0000","last_peered":"2023-10-18T08:58:30.301380+0000","last_clean":"2023-10-18T08:58:30.301380+0000","last_became_active":"2023-10-18T08:58:29.325064+0000","last_became_peered":"2023-10-18T08:58:29.325064+0000","last_unstale":"2023-10-18T08:58:30.301380+0000","last_undegraded":"2023-10-18T08:58:30.301380+0000","last_fullsized":"2023-10-18T08:58:30.301380+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:52:42.152402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.486747+0000","last_change":"2023-10-18T08:58:29.323694+0000","last_active":"2023-10-18T08:58:30.486747+0000","last_peered":"2023-10-18T08:58:30.486747+0000","last_clean":"2023-10-18T08:58:30.486747+0000","last_became_active":"2023-10-18T08:58:29.323083+0000","last_became_peered":"2023-10-18T08:58:29.323083+0000","last_unstale":"2023-10-18T08:58:30.486747+0000","last_undegraded":"2023-10-18T08:58:30.486747+0000","last_fullsized":"2023-10-18T08:58:30.486747+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:57:23.733388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.526798+0000","last_change":"2023-10-18T08:58:29.332412+0000","last_active":"2023-10-18T08:58:31.526798+0000","last_peered":"2023-10-18T08:58:31.526798+0000","last_clean":"2023-10-18T08:58:31.526798+0000","last_became_active":"2023-10-18T08:58:29.330172+0000","last_became_peered":"2023-10-18T08:58:29.330172+0000","last_unstale":"2023-10-18T08:58:31.526798+0000","last_undegraded":"2023-10-18T08:58:31.526798+0000","last_fullsized":"2023-10-18T08:58:31.526798+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:15:04.978697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.301125+0000","last_change":"2023-10-18T08:58:29.326391+0000","last_active":"2023-10-18T08:58:30.301125+0000","last_peered":"2023-10-18T08:58:30.301125+0000","last_clean":"2023-10-18T08:58:30.301125+0000","last_became_active":"2023-10-18T08:58:29.326036+0000","last_became_peered":"2023-10-18T08:58:29.326036+0000","last_unstale":"2023-10-18T08:58:30.301125+0000","last_undegraded":"2023-10-18T08:58:30.301125+0000","last_fullsized":"2023-10-18T08:58:30.301125+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:39:24.943573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.299191+0000","last_change":"2023-10-18T08:58:29.329658+0000","last_active":"2023-10-18T08:58:30.299191+0000","last_peered":"2023-10-18T08:58:30.299191+0000","last_clean":"2023-10-18T08:58:30.299191+0000","last_became_active":"2023-10-18T08:58:29.329393+0000","last_became_peered":"2023-10-18T08:58:29.329393+0000","last_unstale":"2023-10-18T08:58:30.299191+0000","last_undegraded":"2023-10-18T08:58:30.299191+0000","last_fullsized":"2023-10-18T08:58:30.299191+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:36:17.355093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.302090+0000","last_change":"2023-10-18T08:58:29.326075+0000","last_active":"2023-10-18T08:58:30.302090+0000","last_peered":"2023-10-18T08:58:30.302090+0000","last_clean":"2023-10-18T08:58:30.302090+0000","last_became_active":"2023-10-18T08:58:29.324858+0000","last_became_peered":"2023-10-18T08:58:29.324858+0000","last_unstale":"2023-10-18T08:58:30.302090+0000","last_undegraded":"2023-10-18T08:58:30.302090+0000","last_fullsized":"2023-10-18T08:58:30.302090+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:15.032896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.301546+0000","last_change":"2023-10-18T08:58:29.325277+0000","last_active":"2023-10-18T08:58:30.301546+0000","last_peered":"2023-10-18T08:58:30.301546+0000","last_clean":"2023-10-18T08:58:30.301546+0000","last_became_active":"2023-10-18T08:58:29.324019+0000","last_became_peered":"2023-10-18T08:58:29.324019+0000","last_unstale":"2023-10-18T08:58:30.301546+0000","last_undegraded":"2023-10-18T08:58:30.301546+0000","last_fullsized":"2023-10-18T08:58:30.301546+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:17:55.356607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.527125+0000","last_change":"2023-10-18T08:58:29.331792+0000","last_active":"2023-10-18T08:58:31.527125+0000","last_peered":"2023-10-18T08:58:31.527125+0000","last_clean":"2023-10-18T08:58:31.527125+0000","last_became_active":"2023-10-18T08:58:29.329338+0000","last_became_peered":"2023-10-18T08:58:29.329338+0000","last_unstale":"2023-10-18T08:58:31.527125+0000","last_undegraded":"2023-10-18T08:58:31.527125+0000","last_fullsized":"2023-10-18T08:58:31.527125+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:39:03.787381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.526782+0000","last_change":"2023-10-18T08:58:29.329602+0000","last_active":"2023-10-18T08:58:31.526782+0000","last_peered":"2023-10-18T08:58:31.526782+0000","last_clean":"2023-10-18T08:58:31.526782+0000","last_became_active":"2023-10-18T08:58:29.328496+0000","last_became_peered":"2023-10-18T08:58:29.328496+0000","last_unstale":"2023-10-18T08:58:31.526782+0000","last_undegraded":"2023-10-18T08:58:31.526782+0000","last_fullsized":"2023-10-18T08:58:31.526782+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:16.123758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"66'4","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341720+0000","last_change":"2023-10-18T08:58:28.300855+0000","last_active":"2023-10-18T08:58:29.341720+0000","last_peered":"2023-10-18T08:58:29.341720+0000","last_clean":"2023-10-18T08:58:29.341720+0000","last_became_active":"2023-10-18T08:58:28.300672+0000","last_became_peered":"2023-10-18T08:58:28.300672+0000","last_unstale":"2023-10-18T08:58:29.341720+0000","last_undegraded":"2023-10-18T08:58:29.341720+0000","last_fullsized":"2023-10-18T08:58:29.341720+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:03.137892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.524804+0000","last_change":"2023-10-18T08:58:29.310840+0000","last_active":"2023-10-18T08:58:31.524804+0000","last_peered":"2023-10-18T08:58:31.524804+0000","last_clean":"2023-10-18T08:58:31.524804+0000","last_became_active":"2023-10-18T08:58:29.310085+0000","last_became_peered":"2023-10-18T08:58:29.310085+0000","last_unstale":"2023-10-18T08:58:31.524804+0000","last_undegraded":"2023-10-18T08:58:31.524804+0000","last_fullsized":"2023-10-18T08:58:31.524804+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:10:21.837993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.046858+0000","last_change":"2023-10-18T08:58:28.295079+0000","last_active":"2023-10-18T08:58:30.046858+0000","last_peered":"2023-10-18T08:58:30.046858+0000","last_clean":"2023-10-18T08:58:30.046858+0000","last_became_active":"2023-10-18T08:58:28.294485+0000","last_became_peered":"2023-10-18T08:58:28.294485+0000","last_unstale":"2023-10-18T08:58:30.046858+0000","last_undegraded":"2023-10-18T08:58:30.046858+0000","last_fullsized":"2023-10-18T08:58:30.046858+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:27.603595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.527080+0000","last_change":"2023-10-18T08:58:29.330968+0000","last_active":"2023-10-18T08:58:31.527080+0000","last_peered":"2023-10-18T08:58:31.527080+0000","last_clean":"2023-10-18T08:58:31.527080+0000","last_became_active":"2023-10-18T08:58:29.329041+0000","last_became_peered":"2023-10-18T08:58:29.329041+0000","last_unstale":"2023-10-18T08:58:31.527080+0000","last_undegraded":"2023-10-18T08:58:31.527080+0000","last_fullsized":"2023-10-18T08:58:31.527080+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:30:53.581720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.342059+0000","last_change":"2023-10-18T08:58:28.298521+0000","last_active":"2023-10-18T08:58:29.342059+0000","last_peered":"2023-10-18T08:58:29.342059+0000","last_clean":"2023-10-18T08:58:29.342059+0000","last_became_active":"2023-10-18T08:58:28.296841+0000","last_became_peered":"2023-10-18T08:58:28.296841+0000","last_unstale":"2023-10-18T08:58:29.342059+0000","last_undegraded":"2023-10-18T08:58:29.342059+0000","last_fullsized":"2023-10-18T08:58:29.342059+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:22.121699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.526860+0000","last_change":"2023-10-18T08:58:29.329828+0000","last_active":"2023-10-18T08:58:31.526860+0000","last_peered":"2023-10-18T08:58:31.526860+0000","last_clean":"2023-10-18T08:58:31.526860+0000","last_became_active":"2023-10-18T08:58:29.329029+0000","last_became_peered":"2023-10-18T08:58:29.329029+0000","last_unstale":"2023-10-18T08:58:31.526860+0000","last_undegraded":"2023-10-18T08:58:31.526860+0000","last_fullsized":"2023-10-18T08:58:31.526860+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:54.004578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341158+0000","last_change":"2023-10-18T08:58:28.298876+0000","last_active":"2023-10-18T08:58:29.341158+0000","last_peered":"2023-10-18T08:58:29.341158+0000","last_clean":"2023-10-18T08:58:29.341158+0000","last_became_active":"2023-10-18T08:58:28.297696+0000","last_became_peered":"2023-10-18T08:58:28.297696+0000","last_unstale":"2023-10-18T08:58:29.341158+0000","last_undegraded":"2023-10-18T08:58:29.341158+0000","last_fullsized":"2023-10-18T08:58:29.341158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:59.489387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.299836+0000","last_change":"2023-10-18T08:58:29.325300+0000","last_active":"2023-10-18T08:58:30.299836+0000","last_peered":"2023-10-18T08:58:30.299836+0000","last_clean":"2023-10-18T08:58:30.299836+0000","last_became_active":"2023-10-18T08:58:29.323662+0000","last_became_peered":"2023-10-18T08:58:29.323662+0000","last_unstale":"2023-10-18T08:58:30.299836+0000","last_undegraded":"2023-10-18T08:58:30.299836+0000","last_fullsized":"2023-10-18T08:58:30.299836+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:04.081839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.339629+0000","last_change":"2023-10-18T08:58:28.299820+0000","last_active":"2023-10-18T08:58:29.339629+0000","last_peered":"2023-10-18T08:58:29.339629+0000","last_clean":"2023-10-18T08:58:29.339629+0000","last_became_active":"2023-10-18T08:58:28.299113+0000","last_became_peered":"2023-10-18T08:58:28.299113+0000","last_unstale":"2023-10-18T08:58:29.339629+0000","last_undegraded":"2023-10-18T08:58:29.339629+0000","last_fullsized":"2023-10-18T08:58:29.339629+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:04.709218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.300519+0000","last_change":"2023-10-18T08:58:29.332297+0000","last_active":"2023-10-18T08:58:30.300519+0000","last_peered":"2023-10-18T08:58:30.300519+0000","last_clean":"2023-10-18T08:58:30.300519+0000","last_became_active":"2023-10-18T08:58:29.332051+0000","last_became_peered":"2023-10-18T08:58:29.332051+0000","last_unstale":"2023-10-18T08:58:30.300519+0000","last_undegraded":"2023-10-18T08:58:30.300519+0000","last_fullsized":"2023-10-18T08:58:30.300519+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:20:42.521847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341445+0000","last_change":"2023-10-18T08:58:28.300048+0000","last_active":"2023-10-18T08:58:29.341445+0000","last_peered":"2023-10-18T08:58:29.341445+0000","last_clean":"2023-10-18T08:58:29.341445+0000","last_became_active":"2023-10-18T08:58:28.299438+0000","last_became_peered":"2023-10-18T08:58:28.299438+0000","last_unstale":"2023-10-18T08:58:29.341445+0000","last_undegraded":"2023-10-18T08:58:29.341445+0000","last_fullsized":"2023-10-18T08:58:29.341445+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:14.019261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.525822+0000","last_change":"2023-10-18T08:58:29.327702+0000","last_active":"2023-10-18T08:58:31.525822+0000","last_peered":"2023-10-18T08:58:31.525822+0000","last_clean":"2023-10-18T08:58:31.525822+0000","last_became_active":"2023-10-18T08:58:29.327462+0000","last_became_peered":"2023-10-18T08:58:29.327462+0000","last_unstale":"2023-10-18T08:58:31.525822+0000","last_undegraded":"2023-10-18T08:58:31.525822+0000","last_fullsized":"2023-10-18T08:58:31.525822+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:51.840078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.326626+0000","last_change":"2023-10-18T08:58:28.301417+0000","last_active":"2023-10-18T08:58:29.326626+0000","last_peered":"2023-10-18T08:58:29.326626+0000","last_clean":"2023-10-18T08:58:29.326626+0000","last_became_active":"2023-10-18T08:58:28.300999+0000","last_became_peered":"2023-10-18T08:58:28.300999+0000","last_unstale":"2023-10-18T08:58:29.326626+0000","last_undegraded":"2023-10-18T08:58:29.326626+0000","last_fullsized":"2023-10-18T08:58:29.326626+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:44.607015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.299309+0000","last_change":"2023-10-18T08:58:29.321607+0000","last_active":"2023-10-18T08:58:30.299309+0000","last_peered":"2023-10-18T08:58:30.299309+0000","last_clean":"2023-10-18T08:58:30.299309+0000","last_became_active":"2023-10-18T08:58:29.321062+0000","last_became_peered":"2023-10-18T08:58:29.321062+0000","last_unstale":"2023-10-18T08:58:30.299309+0000","last_undegraded":"2023-10-18T08:58:30.299309+0000","last_fullsized":"2023-10-18T08:58:30.299309+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:42.438559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.319303+0000","last_change":"2023-10-18T08:58:28.302109+0000","last_active":"2023-10-18T08:58:29.319303+0000","last_peered":"2023-10-18T08:58:29.319303+0000","last_clean":"2023-10-18T08:58:29.319303+0000","last_became_active":"2023-10-18T08:58:28.301703+0000","last_became_peered":"2023-10-18T08:58:28.301703+0000","last_unstale":"2023-10-18T08:58:29.319303+0000","last_undegraded":"2023-10-18T08:58:29.319303+0000","last_fullsized":"2023-10-18T08:58:29.319303+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:23.982914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.488091+0000","last_change":"2023-10-18T08:58:29.331075+0000","last_active":"2023-10-18T08:58:30.488091+0000","last_peered":"2023-10-18T08:58:30.488091+0000","last_clean":"2023-10-18T08:58:30.488091+0000","last_became_active":"2023-10-18T08:58:29.326251+0000","last_became_peered":"2023-10-18T08:58:29.326251+0000","last_unstale":"2023-10-18T08:58:30.488091+0000","last_undegraded":"2023-10-18T08:58:30.488091+0000","last_fullsized":"2023-10-18T08:58:30.488091+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:47.848978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.053703+0000","last_change":"2023-10-18T08:58:28.298815+0000","last_active":"2023-10-18T08:58:30.053703+0000","last_peered":"2023-10-18T08:58:30.053703+0000","last_clean":"2023-10-18T08:58:30.053703+0000","last_became_active":"2023-10-18T08:58:28.298503+0000","last_became_peered":"2023-10-18T08:58:28.298503+0000","last_unstale":"2023-10-18T08:58:30.053703+0000","last_undegraded":"2023-10-18T08:58:30.053703+0000","last_fullsized":"2023-10-18T08:58:30.053703+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:39.750254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.302018+0000","last_change":"2023-10-18T08:58:29.312112+0000","last_active":"2023-10-18T08:58:30.302018+0000","last_peered":"2023-10-18T08:58:30.302018+0000","last_clean":"2023-10-18T08:58:30.302018+0000","last_became_active":"2023-10-18T08:58:29.310708+0000","last_became_peered":"2023-10-18T08:58:29.310708+0000","last_unstale":"2023-10-18T08:58:30.302018+0000","last_undegraded":"2023-10-18T08:58:30.302018+0000","last_fullsized":"2023-10-18T08:58:30.302018+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:14:02.570524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.047151+0000","last_change":"2023-10-18T08:58:28.298222+0000","last_active":"2023-10-18T08:58:30.047151+0000","last_peered":"2023-10-18T08:58:30.047151+0000","last_clean":"2023-10-18T08:58:30.047151+0000","last_became_active":"2023-10-18T08:58:28.297672+0000","last_became_peered":"2023-10-18T08:58:28.297672+0000","last_unstale":"2023-10-18T08:58:30.047151+0000","last_undegraded":"2023-10-18T08:58:30.047151+0000","last_fullsized":"2023-10-18T08:58:30.047151+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:59.505404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.524505+0000","last_change":"2023-10-18T08:58:29.321498+0000","last_active":"2023-10-18T08:58:31.524505+0000","last_peered":"2023-10-18T08:58:31.524505+0000","last_clean":"2023-10-18T08:58:31.524505+0000","last_became_active":"2023-10-18T08:58:29.321153+0000","last_became_peered":"2023-10-18T08:58:29.321153+0000","last_unstale":"2023-10-18T08:58:31.524505+0000","last_undegraded":"2023-10-18T08:58:31.524505+0000","last_fullsized":"2023-10-18T08:58:31.524505+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:02.110113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"66'3","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.053503+0000","last_change":"2023-10-18T08:58:28.290903+0000","last_active":"2023-10-18T08:58:30.053503+0000","last_peered":"2023-10-18T08:58:30.053503+0000","last_clean":"2023-10-18T08:58:30.053503+0000","last_became_active":"2023-10-18T08:58:28.290358+0000","last_became_peered":"2023-10-18T08:58:28.290358+0000","last_unstale":"2023-10-18T08:58:30.053503+0000","last_undegraded":"2023-10-18T08:58:30.053503+0000","last_fullsized":"2023-10-18T08:58:30.053503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:24.568573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.302673+0000","last_change":"2023-10-18T08:58:29.320149+0000","last_active":"2023-10-18T08:58:30.302673+0000","last_peered":"2023-10-18T08:58:30.302673+0000","last_clean":"2023-10-18T08:58:30.302673+0000","last_became_active":"2023-10-18T08:58:29.317356+0000","last_became_peered":"2023-10-18T08:58:29.317356+0000","last_unstale":"2023-10-18T08:58:30.302673+0000","last_undegraded":"2023-10-18T08:58:30.302673+0000","last_fullsized":"2023-10-18T08:58:30.302673+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:15:48.691583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"66'6","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.343024+0000","last_change":"2023-10-18T08:58:28.302001+0000","last_active":"2023-10-18T08:58:29.343024+0000","last_peered":"2023-10-18T08:58:29.343024+0000","last_clean":"2023-10-18T08:58:29.343024+0000","last_became_active":"2023-10-18T08:58:28.301382+0000","last_became_peered":"2023-10-18T08:58:28.301382+0000","last_unstale":"2023-10-18T08:58:29.343024+0000","last_undegraded":"2023-10-18T08:58:29.343024+0000","last_fullsized":"2023-10-18T08:58:29.343024+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:47:44.607015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.487088+0000","last_change":"2023-10-18T08:58:29.325972+0000","last_active":"2023-10-18T08:58:30.487088+0000","last_peered":"2023-10-18T08:58:30.487088+0000","last_clean":"2023-10-18T08:58:30.487088+0000","last_became_active":"2023-10-18T08:58:29.325717+0000","last_became_peered":"2023-10-18T08:58:29.325717+0000","last_unstale":"2023-10-18T08:58:30.487088+0000","last_undegraded":"2023-10-18T08:58:30.487088+0000","last_fullsized":"2023-10-18T08:58:30.487088+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:49.908171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.047219+0000","last_change":"2023-10-18T08:58:28.296421+0000","last_active":"2023-10-18T08:58:30.047219+0000","last_peered":"2023-10-18T08:58:30.047219+0000","last_clean":"2023-10-18T08:58:30.047219+0000","last_became_active":"2023-10-18T08:58:28.296176+0000","last_became_peered":"2023-10-18T08:58:28.296176+0000","last_unstale":"2023-10-18T08:58:30.047219+0000","last_undegraded":"2023-10-18T08:58:30.047219+0000","last_fullsized":"2023-10-18T08:58:30.047219+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:14:17.772896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.302810+0000","last_change":"2023-10-18T08:58:29.322440+0000","last_active":"2023-10-18T08:58:30.302810+0000","last_peered":"2023-10-18T08:58:30.302810+0000","last_clean":"2023-10-18T08:58:30.302810+0000","last_became_active":"2023-10-18T08:58:29.319010+0000","last_became_peered":"2023-10-18T08:58:29.319010+0000","last_unstale":"2023-10-18T08:58:30.302810+0000","last_undegraded":"2023-10-18T08:58:30.302810+0000","last_fullsized":"2023-10-18T08:58:30.302810+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:23.135334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.315431+0000","last_change":"2023-10-18T08:58:28.290878+0000","last_active":"2023-10-18T08:58:29.315431+0000","last_peered":"2023-10-18T08:58:29.315431+0000","last_clean":"2023-10-18T08:58:29.315431+0000","last_became_active":"2023-10-18T08:58:28.290244+0000","last_became_peered":"2023-10-18T08:58:28.290244+0000","last_unstale":"2023-10-18T08:58:29.315431+0000","last_undegraded":"2023-10-18T08:58:29.315431+0000","last_fullsized":"2023-10-18T08:58:29.315431+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:41.494592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.487931+0000","last_change":"2023-10-18T08:58:29.331835+0000","last_active":"2023-10-18T08:58:30.487931+0000","last_peered":"2023-10-18T08:58:30.487931+0000","last_clean":"2023-10-18T08:58:30.487931+0000","last_became_active":"2023-10-18T08:58:29.326532+0000","last_became_peered":"2023-10-18T08:58:29.326532+0000","last_unstale":"2023-10-18T08:58:30.487931+0000","last_undegraded":"2023-10-18T08:58:30.487931+0000","last_fullsized":"2023-10-18T08:58:30.487931+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:57:23.733388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.317704+0000","last_change":"2023-10-18T08:58:28.299958+0000","last_active":"2023-10-18T08:58:29.317704+0000","last_peered":"2023-10-18T08:58:29.317704+0000","last_clean":"2023-10-18T08:58:29.317704+0000","last_became_active":"2023-10-18T08:58:28.299673+0000","last_became_peered":"2023-10-18T08:58:28.299673+0000","last_unstale":"2023-10-18T08:58:29.317704+0000","last_undegraded":"2023-10-18T08:58:29.317704+0000","last_fullsized":"2023-10-18T08:58:29.317704+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:04.709218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.302312+0000","last_change":"2023-10-18T08:58:29.313209+0000","last_active":"2023-10-18T08:58:30.302312+0000","last_peered":"2023-10-18T08:58:30.302312+0000","last_clean":"2023-10-18T08:58:30.302312+0000","last_became_active":"2023-10-18T08:58:29.312599+0000","last_became_peered":"2023-10-18T08:58:29.312599+0000","last_unstale":"2023-10-18T08:58:30.302312+0000","last_undegraded":"2023-10-18T08:58:30.302312+0000","last_fullsized":"2023-10-18T08:58:30.302312+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:45.688349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"66'8","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.053526+0000","last_change":"2023-10-18T08:58:28.297447+0000","last_active":"2023-10-18T08:58:30.053526+0000","last_peered":"2023-10-18T08:58:30.053526+0000","last_clean":"2023-10-18T08:58:30.053526+0000","last_became_active":"2023-10-18T08:58:28.297055+0000","last_became_peered":"2023-10-18T08:58:28.297055+0000","last_unstale":"2023-10-18T08:58:30.053526+0000","last_undegraded":"2023-10-18T08:58:30.053526+0000","last_fullsized":"2023-10-18T08:58:30.053526+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:57:22.719753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.303148+0000","last_change":"2023-10-18T08:58:29.326263+0000","last_active":"2023-10-18T08:58:30.303148+0000","last_peered":"2023-10-18T08:58:30.303148+0000","last_clean":"2023-10-18T08:58:30.303148+0000","last_became_active":"2023-10-18T08:58:29.325851+0000","last_became_peered":"2023-10-18T08:58:29.325851+0000","last_unstale":"2023-10-18T08:58:30.303148+0000","last_undegraded":"2023-10-18T08:58:30.303148+0000","last_fullsized":"2023-10-18T08:58:30.303148+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:20.541018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.046885+0000","last_change":"2023-10-18T08:58:28.300004+0000","last_active":"2023-10-18T08:58:30.046885+0000","last_peered":"2023-10-18T08:58:30.046885+0000","last_clean":"2023-10-18T08:58:30.046885+0000","last_became_active":"2023-10-18T08:58:28.294830+0000","last_became_peered":"2023-10-18T08:58:28.294830+0000","last_unstale":"2023-10-18T08:58:30.046885+0000","last_undegraded":"2023-10-18T08:58:30.046885+0000","last_fullsized":"2023-10-18T08:58:30.046885+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:43:16.750986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.486751+0000","last_change":"2023-10-18T08:58:29.332382+0000","last_active":"2023-10-18T08:58:30.486751+0000","last_peered":"2023-10-18T08:58:30.486751+0000","last_clean":"2023-10-18T08:58:30.486751+0000","last_became_active":"2023-10-18T08:58:29.330341+0000","last_became_peered":"2023-10-18T08:58:29.330341+0000","last_unstale":"2023-10-18T08:58:30.486751+0000","last_undegraded":"2023-10-18T08:58:30.486751+0000","last_fullsized":"2023-10-18T08:58:30.486751+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:47.848978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341932+0000","last_change":"2023-10-18T08:58:28.298360+0000","last_active":"2023-10-18T08:58:29.341932+0000","last_peered":"2023-10-18T08:58:29.341932+0000","last_clean":"2023-10-18T08:58:29.341932+0000","last_became_active":"2023-10-18T08:58:28.296520+0000","last_became_peered":"2023-10-18T08:58:28.296520+0000","last_unstale":"2023-10-18T08:58:29.341932+0000","last_undegraded":"2023-10-18T08:58:29.341932+0000","last_fullsized":"2023-10-18T08:58:29.341932+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:01:54.359829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.301449+0000","last_change":"2023-10-18T08:58:29.324872+0000","last_active":"2023-10-18T08:58:30.301449+0000","last_peered":"2023-10-18T08:58:30.301449+0000","last_clean":"2023-10-18T08:58:30.301449+0000","last_became_active":"2023-10-18T08:58:29.323103+0000","last_became_peered":"2023-10-18T08:58:29.323103+0000","last_unstale":"2023-10-18T08:58:30.301449+0000","last_undegraded":"2023-10-18T08:58:30.301449+0000","last_fullsized":"2023-10-18T08:58:30.301449+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:57.391953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341292+0000","last_change":"2023-10-18T08:58:28.298668+0000","last_active":"2023-10-18T08:58:29.341292+0000","last_peered":"2023-10-18T08:58:29.341292+0000","last_clean":"2023-10-18T08:58:29.341292+0000","last_became_active":"2023-10-18T08:58:28.297217+0000","last_became_peered":"2023-10-18T08:58:28.297217+0000","last_unstale":"2023-10-18T08:58:29.341292+0000","last_undegraded":"2023-10-18T08:58:29.341292+0000","last_fullsized":"2023-10-18T08:58:29.341292+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:43:35.327448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.302188+0000","last_change":"2023-10-18T08:58:29.313231+0000","last_active":"2023-10-18T08:58:30.302188+0000","last_peered":"2023-10-18T08:58:30.302188+0000","last_clean":"2023-10-18T08:58:30.302188+0000","last_became_active":"2023-10-18T08:58:29.311497+0000","last_became_peered":"2023-10-18T08:58:29.311497+0000","last_unstale":"2023-10-18T08:58:30.302188+0000","last_undegraded":"2023-10-18T08:58:30.302188+0000","last_fullsized":"2023-10-18T08:58:30.302188+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:26.145731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.319166+0000","last_change":"2023-10-18T08:58:28.297688+0000","last_active":"2023-10-18T08:58:29.319166+0000","last_peered":"2023-10-18T08:58:29.319166+0000","last_clean":"2023-10-18T08:58:29.319166+0000","last_became_active":"2023-10-18T08:58:28.297401+0000","last_became_peered":"2023-10-18T08:58:28.297401+0000","last_unstale":"2023-10-18T08:58:29.319166+0000","last_undegraded":"2023-10-18T08:58:29.319166+0000","last_fullsized":"2023-10-18T08:58:29.319166+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:55:19.527383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.487313+0000","last_change":"2023-10-18T08:58:29.332231+0000","last_active":"2023-10-18T08:58:30.487313+0000","last_peered":"2023-10-18T08:58:30.487313+0000","last_clean":"2023-10-18T08:58:30.487313+0000","last_became_active":"2023-10-18T08:58:29.329812+0000","last_became_peered":"2023-10-18T08:58:29.329812+0000","last_unstale":"2023-10-18T08:58:30.487313+0000","last_undegraded":"2023-10-18T08:58:30.487313+0000","last_fullsized":"2023-10-18T08:58:30.487313+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:34:27.509601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.046993+0000","last_change":"2023-10-18T08:58:28.299131+0000","last_active":"2023-10-18T08:58:30.046993+0000","last_peered":"2023-10-18T08:58:30.046993+0000","last_clean":"2023-10-18T08:58:30.046993+0000","last_became_active":"2023-10-18T08:58:28.298514+0000","last_became_peered":"2023-10-18T08:58:28.298514+0000","last_unstale":"2023-10-18T08:58:30.046993+0000","last_undegraded":"2023-10-18T08:58:30.046993+0000","last_fullsized":"2023-10-18T08:58:30.046993+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:42:24.333152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.526355+0000","last_change":"2023-10-18T08:58:29.324824+0000","last_active":"2023-10-18T08:58:31.526355+0000","last_peered":"2023-10-18T08:58:31.526355+0000","last_clean":"2023-10-18T08:58:31.526355+0000","last_became_active":"2023-10-18T08:58:29.324548+0000","last_became_peered":"2023-10-18T08:58:29.324548+0000","last_unstale":"2023-10-18T08:58:31.526355+0000","last_undegraded":"2023-10-18T08:58:31.526355+0000","last_fullsized":"2023-10-18T08:58:31.526355+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:28.617230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.302848+0000","last_change":"2023-10-18T08:58:28.294327+0000","last_active":"2023-10-18T08:58:29.302848+0000","last_peered":"2023-10-18T08:58:29.302848+0000","last_clean":"2023-10-18T08:58:29.302848+0000","last_became_active":"2023-10-18T08:58:28.293852+0000","last_became_peered":"2023-10-18T08:58:28.293852+0000","last_unstale":"2023-10-18T08:58:29.302848+0000","last_undegraded":"2023-10-18T08:58:29.302848+0000","last_fullsized":"2023-10-18T08:58:29.302848+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:35.906463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.527170+0000","last_change":"2023-10-18T08:58:29.329694+0000","last_active":"2023-10-18T08:58:31.527170+0000","last_peered":"2023-10-18T08:58:31.527170+0000","last_clean":"2023-10-18T08:58:31.527170+0000","last_became_active":"2023-10-18T08:58:29.328351+0000","last_became_peered":"2023-10-18T08:58:29.328351+0000","last_unstale":"2023-10-18T08:58:31.527170+0000","last_undegraded":"2023-10-18T08:58:31.527170+0000","last_fullsized":"2023-10-18T08:58:31.527170+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:42:25.346787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.320402+0000","last_change":"2023-10-18T08:58:28.302115+0000","last_active":"2023-10-18T08:58:29.320402+0000","last_peered":"2023-10-18T08:58:29.320402+0000","last_clean":"2023-10-18T08:58:29.320402+0000","last_became_active":"2023-10-18T08:58:28.300920+0000","last_became_peered":"2023-10-18T08:58:28.300920+0000","last_unstale":"2023-10-18T08:58:29.320402+0000","last_undegraded":"2023-10-18T08:58:29.320402+0000","last_fullsized":"2023-10-18T08:58:29.320402+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:56.260888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.486648+0000","last_change":"2023-10-18T08:58:29.328074+0000","last_active":"2023-10-18T08:58:30.486648+0000","last_peered":"2023-10-18T08:58:30.486648+0000","last_clean":"2023-10-18T08:58:30.486648+0000","last_became_active":"2023-10-18T08:58:29.327486+0000","last_became_peered":"2023-10-18T08:58:29.327486+0000","last_unstale":"2023-10-18T08:58:30.486648+0000","last_undegraded":"2023-10-18T08:58:30.486648+0000","last_fullsized":"2023-10-18T08:58:30.486648+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:25.582208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.341494+0000","last_change":"2023-10-18T08:58:28.280587+0000","last_active":"2023-10-18T08:58:29.341494+0000","last_peered":"2023-10-18T08:58:29.341494+0000","last_clean":"2023-10-18T08:58:29.341494+0000","last_became_active":"2023-10-18T08:58:28.280261+0000","last_became_peered":"2023-10-18T08:58:28.280261+0000","last_unstale":"2023-10-18T08:58:29.341494+0000","last_undegraded":"2023-10-18T08:58:29.341494+0000","last_fullsized":"2023-10-18T08:58:29.341494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:31:00.794061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.486903+0000","last_change":"2023-10-18T08:58:29.326708+0000","last_active":"2023-10-18T08:58:30.486903+0000","last_peered":"2023-10-18T08:58:30.486903+0000","last_clean":"2023-10-18T08:58:30.486903+0000","last_became_active":"2023-10-18T08:58:29.324290+0000","last_became_peered":"2023-10-18T08:58:29.324290+0000","last_unstale":"2023-10-18T08:58:30.486903+0000","last_undegraded":"2023-10-18T08:58:30.486903+0000","last_fullsized":"2023-10-18T08:58:30.486903+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:40.763889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"67'1","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.526790+0000","last_change":"2023-10-18T08:58:28.299846+0000","last_active":"2023-10-18T08:58:31.526790+0000","last_peered":"2023-10-18T08:58:31.526790+0000","last_clean":"2023-10-18T08:58:31.526790+0000","last_became_active":"2023-10-18T08:58:28.299121+0000","last_became_peered":"2023-10-18T08:58:28.299121+0000","last_unstale":"2023-10-18T08:58:31.526790+0000","last_undegraded":"2023-10-18T08:58:31.526790+0000","last_fullsized":"2023-10-18T08:58:31.526790+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:25.132096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.525751+0000","last_change":"2023-10-18T08:58:29.326192+0000","last_active":"2023-10-18T08:58:31.525751+0000","last_peered":"2023-10-18T08:58:31.525751+0000","last_clean":"2023-10-18T08:58:31.525751+0000","last_became_active":"2023-10-18T08:58:29.325922+0000","last_became_peered":"2023-10-18T08:58:29.325922+0000","last_unstale":"2023-10-18T08:58:31.525751+0000","last_undegraded":"2023-10-18T08:58:31.525751+0000","last_fullsized":"2023-10-18T08:58:31.525751+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:53:50.071506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.314770+0000","last_change":"2023-10-18T08:58:28.300674+0000","last_active":"2023-10-18T08:58:29.314770+0000","last_peered":"2023-10-18T08:58:29.314770+0000","last_clean":"2023-10-18T08:58:29.314770+0000","last_became_active":"2023-10-18T08:58:28.300119+0000","last_became_peered":"2023-10-18T08:58:28.300119+0000","last_unstale":"2023-10-18T08:58:29.314770+0000","last_undegraded":"2023-10-18T08:58:29.314770+0000","last_fullsized":"2023-10-18T08:58:29.314770+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:47:06.911860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.527116+0000","last_change":"2023-10-18T08:58:29.333041+0000","last_active":"2023-10-18T08:58:31.527116+0000","last_peered":"2023-10-18T08:58:31.527116+0000","last_clean":"2023-10-18T08:58:31.527116+0000","last_became_active":"2023-10-18T08:58:29.332765+0000","last_became_peered":"2023-10-18T08:58:29.332765+0000","last_unstale":"2023-10-18T08:58:31.527116+0000","last_undegraded":"2023-10-18T08:58:31.527116+0000","last_fullsized":"2023-10-18T08:58:31.527116+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:11:33.988349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.315366+0000","last_change":"2023-10-18T08:58:28.303209+0000","last_active":"2023-10-18T08:58:29.315366+0000","last_peered":"2023-10-18T08:58:29.315366+0000","last_clean":"2023-10-18T08:58:29.315366+0000","last_became_active":"2023-10-18T08:58:28.302960+0000","last_became_peered":"2023-10-18T08:58:28.302960+0000","last_unstale":"2023-10-18T08:58:29.315366+0000","last_undegraded":"2023-10-18T08:58:29.315366+0000","last_fullsized":"2023-10-18T08:58:29.315366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:48.001530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.302006+0000","last_change":"2023-10-18T08:58:29.315776+0000","last_active":"2023-10-18T08:58:30.302006+0000","last_peered":"2023-10-18T08:58:30.302006+0000","last_clean":"2023-10-18T08:58:30.302006+0000","last_became_active":"2023-10-18T08:58:29.315154+0000","last_became_peered":"2023-10-18T08:58:29.315154+0000","last_unstale":"2023-10-18T08:58:30.302006+0000","last_undegraded":"2023-10-18T08:58:30.302006+0000","last_fullsized":"2023-10-18T08:58:30.302006+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:33:20.141999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.301965+0000","last_change":"2023-10-18T08:58:28.295800+0000","last_active":"2023-10-18T08:58:29.301965+0000","last_peered":"2023-10-18T08:58:29.301965+0000","last_clean":"2023-10-18T08:58:29.301965+0000","last_became_active":"2023-10-18T08:58:28.295520+0000","last_became_peered":"2023-10-18T08:58:28.295520+0000","last_unstale":"2023-10-18T08:58:29.301965+0000","last_undegraded":"2023-10-18T08:58:29.301965+0000","last_fullsized":"2023-10-18T08:58:29.301965+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:39:14.200132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.302428+0000","last_change":"2023-10-18T08:58:29.313510+0000","last_active":"2023-10-18T08:58:30.302428+0000","last_peered":"2023-10-18T08:58:30.302428+0000","last_clean":"2023-10-18T08:58:30.302428+0000","last_became_active":"2023-10-18T08:58:29.312697+0000","last_became_peered":"2023-10-18T08:58:29.312697+0000","last_unstale":"2023-10-18T08:58:30.302428+0000","last_undegraded":"2023-10-18T08:58:30.302428+0000","last_fullsized":"2023-10-18T08:58:30.302428+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:46.399620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.313207+0000","last_change":"2023-10-18T08:58:28.300006+0000","last_active":"2023-10-18T08:58:29.313207+0000","last_peered":"2023-10-18T08:58:29.313207+0000","last_clean":"2023-10-18T08:58:29.313207+0000","last_became_active":"2023-10-18T08:58:28.299510+0000","last_became_peered":"2023-10-18T08:58:28.299510+0000","last_unstale":"2023-10-18T08:58:29.313207+0000","last_undegraded":"2023-10-18T08:58:29.313207+0000","last_fullsized":"2023-10-18T08:58:29.313207+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:56.260888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:31.524868+0000","last_change":"2023-10-18T08:58:29.319656+0000","last_active":"2023-10-18T08:58:31.524868+0000","last_peered":"2023-10-18T08:58:31.524868+0000","last_clean":"2023-10-18T08:58:31.524868+0000","last_became_active":"2023-10-18T08:58:29.319370+0000","last_became_peered":"2023-10-18T08:58:29.319370+0000","last_unstale":"2023-10-18T08:58:31.524868+0000","last_undegraded":"2023-10-18T08:58:31.524868+0000","last_fullsized":"2023-10-18T08:58:31.524868+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:21:43.446552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.300554+0000","last_change":"2023-10-18T08:58:28.297980+0000","last_active":"2023-10-18T08:58:29.300554+0000","last_peered":"2023-10-18T08:58:29.300554+0000","last_clean":"2023-10-18T08:58:29.300554+0000","last_became_active":"2023-10-18T08:58:28.297743+0000","last_became_peered":"2023-10-18T08:58:28.297743+0000","last_unstale":"2023-10-18T08:58:29.300554+0000","last_undegraded":"2023-10-18T08:58:29.300554+0000","last_fullsized":"2023-10-18T08:58:29.300554+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:04.513124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.299450+0000","last_change":"2023-10-18T08:58:29.318110+0000","last_active":"2023-10-18T08:58:30.299450+0000","last_peered":"2023-10-18T08:58:30.299450+0000","last_clean":"2023-10-18T08:58:30.299450+0000","last_became_active":"2023-10-18T08:58:29.317066+0000","last_became_peered":"2023-10-18T08:58:29.317066+0000","last_unstale":"2023-10-18T08:58:30.299450+0000","last_undegraded":"2023-10-18T08:58:30.299450+0000","last_fullsized":"2023-10-18T08:58:30.299450+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:21:10.335374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.046908+0000","last_change":"2023-10-18T08:58:28.300829+0000","last_active":"2023-10-18T08:58:30.046908+0000","last_peered":"2023-10-18T08:58:30.046908+0000","last_clean":"2023-10-18T08:58:30.046908+0000","last_became_active":"2023-10-18T08:58:28.300515+0000","last_became_peered":"2023-10-18T08:58:28.300515+0000","last_unstale":"2023-10-18T08:58:30.046908+0000","last_undegraded":"2023-10-18T08:58:30.046908+0000","last_fullsized":"2023-10-18T08:58:30.046908+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:15:03.965062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-18T08:58:30.299932+0000","last_change":"2023-10-18T08:58:29.331231+0000","last_active":"2023-10-18T08:58:30.299932+0000","last_peered":"2023-10-18T08:58:30.299932+0000","last_clean":"2023-10-18T08:58:30.299932+0000","last_became_active":"2023-10-18T08:58:29.331003+0000","last_became_peered":"2023-10-18T08:58:29.331003+0000","last_unstale":"2023-10-18T08:58:30.299932+0000","last_undegraded":"2023-10-18T08:58:30.299932+0000","last_fullsized":"2023-10-18T08:58:30.299932+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:28.264871+0000","last_clean_scrub_stamp":"2023-10-18T08:58:28.264871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:36:44.641007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.315179+0000","last_change":"2023-10-18T08:58:28.300119+0000","last_active":"2023-10-18T08:58:29.315179+0000","last_peered":"2023-10-18T08:58:29.315179+0000","last_clean":"2023-10-18T08:58:29.315179+0000","last_became_active":"2023-10-18T08:58:28.299753+0000","last_became_peered":"2023-10-18T08:58:28.299753+0000","last_unstale":"2023-10-18T08:58:29.315179+0000","last_undegraded":"2023-10-18T08:58:29.315179+0000","last_fullsized":"2023-10-18T08:58:29.315179+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:58:27.251236+0000","last_clean_scrub_stamp":"2023-10-18T08:58:27.251236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:53:49.057871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":215,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-10-18T08:58:29.299519+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T08:58:29.299519+0000","last_peered":"2023-10-18T08:58:29.299519+0000","last_clean":"2023-10-18T08:58:29.299519+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T08:58:29.299519+0000","last_undegraded":"2023-10-18T08:58:29.299519+0000","last_fullsized":"2023-10-18T08:58:29.299519+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:53.386532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":915,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194396,"statfs":{"total":200000000,"available":199062474,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.99299999999999999,"15min":0.98899999999999999},"min":{"1min":0.70299999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3400000000000001,"5min":1.538,"15min":1.538},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.034,"15min":1.008},"min":{"1min":0.69599999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4019999999999999,"5min":1.667,"15min":1.667},"last":0.71899999999999997}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.1240000000000001,"15min":1.147},"min":{"1min":0.80300000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5900000000000001,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.874},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1419999999999999,"15min":1.1220000000000001},"min":{"1min":0.748,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7450000000000001,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.89800000000000002}]},{"osd":2,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.2450000000000001,"15min":1.2330000000000001},"min":{"1min":0.80200000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.232},{"interface":"front","average":{"1min":1.276,"5min":1.2010000000000001,"15min":1.1910000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.798,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":0.94199999999999995}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2629999999999999,"15min":1.264},"min":{"1min":0.95999999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.702,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.3140000000000001,"15min":1.3149999999999999},"min":{"1min":0.94199999999999995,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.873,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.3260000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1899999999999999,"15min":1.1599999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6799999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.022},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.3779999999999999,"15min":1.401},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.6279999999999999,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.1719999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.353,"15min":1.353},"min":{"1min":0.99399999999999999,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.466},{"interface":"front","average":{"1min":1.349,"5min":1.4019999999999999,"15min":1.4159999999999999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.865,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.1379999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.464,"15min":1.458},"min":{"1min":1.0369999999999999,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.923,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.492,"15min":1.51},"min":{"1min":1.0669999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.8069999999999999,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.5029999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":338,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194973,"statfs":{"total":200000000,"available":199653002,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1020000000000001,"15min":1.0800000000000001},"min":{"1min":0.623,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.651,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.298,"5min":1.228,"15min":1.234},"min":{"1min":0.84499999999999997,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":1.244}]},{"osd":1,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.089,"15min":1.085},"min":{"1min":0.65600000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1799999999999999,"15min":1.1859999999999999},"min":{"1min":0.80600000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.962,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":1.5940000000000001}]},{"osd":2,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.161,"15min":1.173},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5029999999999999,"5min":1.696,"15min":1.696},"last":1.365},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.1970000000000001,"15min":1.194},"min":{"1min":0.92500000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.512}]},{"osd":3,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2509999999999999,"15min":1.2569999999999999},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.669,"5min":1.784,"15min":1.784},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.228,"15min":1.22},"min":{"1min":0.84999999999999998,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.095}]},{"osd":4,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.212,"15min":1.175},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.26,"15min":1.266},"min":{"1min":0.60299999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.696,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.1679999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3520000000000001,"15min":1.3480000000000001},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.599,"5min":1.667,"15min":1.667},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.232,"5min":1.238,"15min":1.218},"min":{"1min":0.629,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.52,"5min":1.746,"15min":1.746},"last":1.032}]},{"osd":7,"last update":"Wed Oct 18 08:58:05 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.4039999999999999,"15min":1.3959999999999999},"min":{"1min":0.73999999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.51,"15min":1.5249999999999999},"min":{"1min":0.84099999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.831,"5min":2.794,"15min":2.794},"last":1.538}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":339,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194972,"statfs":{"total":200000000,"available":199652350,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.7529999999999999,"5min":1.3160000000000001,"15min":1.224},"min":{"1min":0.68300000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":9.3539999999999992,"5min":9.3539999999999992,"15min":9.3539999999999992},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.3220000000000001,"15min":1.3029999999999999},"min":{"1min":0.71999999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":5.3029999999999999,"5min":5.3029999999999999,"15min":5.3029999999999999},"last":1.4630000000000001}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2090000000000001,"15min":1.2230000000000001},"min":{"1min":0.755,"5min":0.749,"15min":0.749},"max":{"1min":2.7360000000000002,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.222,"15min":1.248},"min":{"1min":0.82199999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.367,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":0.88500000000000001}]},{"osd":2,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.268,"15min":1.268},"min":{"1min":0.78100000000000003,"5min":0.755,"15min":0.755},"max":{"1min":1.5309999999999999,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.2889999999999999,"15min":1.2929999999999999},"min":{"1min":0.78400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.3069999999999999,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.139}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.25,"15min":1.246},"min":{"1min":0.93300000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.524,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":1.349},{"interface":"front","average":{"1min":1.409,"5min":1.337,"15min":1.3320000000000001},"min":{"1min":0.94399999999999995,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.6379999999999999,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.407}]},{"osd":4,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4139999999999999,"15min":1.419},"min":{"1min":0.93400000000000005,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.881,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.7869999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.4950000000000001,"15min":1.5329999999999999},"min":{"1min":0.90000000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":4.4429999999999996,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":1.3169999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.377,"15min":1.4119999999999999},"min":{"1min":0.93899999999999995,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.5469999999999997,"5min":5.141,"15min":5.141},"last":1.496},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.556,"15min":1.587},"min":{"1min":1.151,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0590000000000002,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.6930000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.6419999999999999,"15min":1.6699999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":4.2050000000000001,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":1.591},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5149999999999999,"15min":1.5249999999999999},"min":{"1min":1.0249999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":4.4690000000000003,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":1.633}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":339,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194973,"statfs":{"total":200000000,"available":199652409,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.107,"15min":1.1100000000000001},"min":{"1min":0.745,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.996,"5min":1.129,"15min":1.169},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.6990000000000001,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.2050000000000001}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.4849999999999999,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2250000000000001,"15min":1.2390000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.91100000000000003}]},{"osd":2,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1879999999999999,"15min":1.179},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.27,"15min":1.3029999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.994,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.04}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2889999999999999,"15min":1.3009999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.4179999999999999,"15min":1.4139999999999999},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.6880000000000002,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.575}]},{"osd":5,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.323,"15min":1.304},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.778,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.351,"15min":1.367},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.359}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.4339999999999999,"15min":1.3939999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.556,"5min":1.5629999999999999,"15min":1.5760000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":1.3959999999999999}]},{"osd":7,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.4950000000000001,"15min":1.514},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":2.552,"5min":2.552,"15min":2.552},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.5529999999999999,"15min":1.5509999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.585,"5min":2.585,"15min":2.585},"last":1.7250000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":338,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194973,"statfs":{"total":200000000,"available":199652946,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.85199999999999998,"15min":0.77400000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.304,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.95199999999999996,"15min":0.873},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.393,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":0.46600000000000003}]},{"osd":1,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.97799999999999998,"15min":0.94199999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.208,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.1419999999999999,"15min":1.113},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.5660000000000001,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.258}]},{"osd":2,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.0549999999999999,"15min":1.0149999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4330000000000001,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1080000000000001,"15min":1.054},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6459999999999999,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":1.3149999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.464,"15min":1.3049999999999999},"min":{"1min":0.83599999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.5649999999999999,"5min":18.664999999999999,"15min":18.664999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.21,"15min":1.1499999999999999},"min":{"1min":0.69099999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.756,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":1.0149999999999999}]},{"osd":5,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.276,"15min":1.2170000000000001},"min":{"1min":0.72399999999999998,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.702,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":0.87},{"interface":"front","average":{"1min":1.425,"5min":1.502,"15min":1.3400000000000001},"min":{"1min":0.80800000000000005,"5min":0.748,"15min":0.748},"max":{"1min":1.885,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.375}]},{"osd":6,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4750000000000001,"15min":1.4199999999999999},"min":{"1min":0.82499999999999996,"5min":0.625,"15min":0.625},"max":{"1min":1.7649999999999999,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3959999999999999,"15min":1.3300000000000001},"min":{"1min":0.77200000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.774,"5min":7.242,"15min":7.242},"last":1.103}]},{"osd":7,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.4019999999999999,"15min":1.3320000000000001},"min":{"1min":0.80600000000000005,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.9970000000000001,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.431},{"interface":"front","average":{"1min":1.47,"5min":1.556,"15min":1.4470000000000001},"min":{"1min":0.93100000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.8580000000000001,"5min":12.946,"15min":12.946},"last":0.84199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":338,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194973,"statfs":{"total":200000000,"available":199652946,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90500000000000003,"15min":0.93500000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.55,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":1.25},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0029999999999999,"15min":1.05},"min":{"1min":0.63100000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.4870000000000001,"5min":1.651,"15min":1.651},"last":0.80400000000000005}]},{"osd":1,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.0249999999999999,"15min":1.0549999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.427,"5min":4.29,"15min":4.29},"last":1.085},{"interface":"front","average":{"1min":1.198,"5min":1.0760000000000001,"15min":1.095},"min":{"1min":0.65700000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.583,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.0589999999999999}]},{"osd":3,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0840000000000001,"15min":1.1259999999999999},"min":{"1min":0.54600000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.6419999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.391,"15min":1.48},"min":{"1min":0.57899999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8360000000000001,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":1.373}]},{"osd":4,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.448,"15min":1.4950000000000001},"min":{"1min":0.95699999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.7450000000000001,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.415,"15min":1.45},"min":{"1min":0.93799999999999994,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.8029999999999999,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.536}]},{"osd":5,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.234,"15min":1.268},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6719999999999999,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.5,"15min":1.5760000000000001},"min":{"1min":0.96099999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.863,"5min":7.657,"15min":7.657},"last":1.6060000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.472,"15min":1.5309999999999999},"min":{"1min":0.82699999999999996,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8999999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.171},{"interface":"front","average":{"1min":3.536,"5min":1.8240000000000001,"15min":1.5720000000000001},"min":{"1min":0.98899999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":37.771999999999998,"5min":37.771999999999998,"15min":37.771999999999998},"last":1.4790000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:58:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.5309999999999999,"15min":1.573},"min":{"1min":0.98699999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.016,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.4410000000000001,"15min":1.4710000000000001},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1349999999999998,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.575}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":337,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194974,"statfs":{"total":200000000,"available":199653931,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.79200000000000004,"15min":0.79800000000000004},"min":{"1min":0.44500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":1.327,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.165},{"interface":"front","average":{"1min":0.999,"5min":0.89300000000000002,"15min":0.89100000000000001},"min":{"1min":0.501,"5min":0.443,"15min":0.443},"max":{"1min":1.5129999999999999,"5min":1.55,"15min":1.55},"last":1.6830000000000001}]},{"osd":2,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.91100000000000003,"15min":0.92300000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6850000000000001,"5min":1.859,"15min":1.859},"last":1.6060000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.83299999999999996,"15min":0.83499999999999996},"min":{"1min":0.47999999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.625,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.3140000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.98199999999999998,"15min":1.008},"min":{"1min":0.52800000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.571,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.016},{"interface":"front","average":{"1min":1.052,"5min":1.0229999999999999,"15min":1.0469999999999999},"min":{"1min":0.54500000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.8859999999999999,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":1.4339999999999999}]},{"osd":4,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.22,"15min":1.2350000000000001},"min":{"1min":0.82099999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.5760000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":2.0489999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1970000000000001,"15min":1.1930000000000001},"min":{"1min":0.93799999999999994,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.556,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.8280000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.268,"15min":1.2809999999999999},"min":{"1min":0.86599999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.625,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.948},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.343,"15min":1.343},"min":{"1min":0.90500000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.6759999999999999,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.881}]},{"osd":6,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2529999999999999,"15min":1.2649999999999999},"min":{"1min":0.89100000000000001,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6240000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.2450000000000001,"15min":1.248},"min":{"1min":0.78900000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.744,"5min":1.853,"15min":1.853},"last":2.1070000000000002}]},{"osd":7,"last update":"Wed Oct 18 08:58:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.333,"15min":1.339},"min":{"1min":0.84399999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.716,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":2.2229999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3380000000000001,"15min":1.3400000000000001},"min":{"1min":0.95099999999999996,"5min":0.746,"15min":0.746},"max":{"1min":1.758,"5min":1.994,"15min":1.994},"last":2.4180000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574892,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":916,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194395,"statfs":{"total":200000000,"available":199061459,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.89500000000000002,"15min":0.93600000000000005},"min":{"1min":0.371,"5min":0.37,"15min":0.37},"max":{"1min":1.0940000000000001,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0660000000000001,"15min":1.079},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.181}]},{"osd":2,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.121,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.1240000000000001,"15min":1.125},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.8109999999999999,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.01}]},{"osd":3,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.22,"15min":1.306},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6639999999999999,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.4750000000000001,"15min":1.5309999999999999},"min":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.9259999999999999,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.1180000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2629999999999999,"15min":1.3169999999999999},"min":{"1min":0.81200000000000006,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.5549999999999999,"5min":4.492,"15min":4.492},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.234,"5min":1.2490000000000001,"15min":1.2470000000000001},"min":{"1min":0.93200000000000005,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.629,"5min":1.726,"15min":1.726},"last":1.552}]},{"osd":5,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.377,"15min":1.4490000000000001},"min":{"1min":0.89500000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.944,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.071},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3580000000000001,"15min":1.3859999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8240000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.6559999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.5629999999999999,"15min":1.6100000000000001},"min":{"1min":0.877,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.0409999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.4670000000000001,"15min":1.5189999999999999},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.863,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.2769999999999999}]},{"osd":7,"last update":"Wed Oct 18 08:58:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.619,"15min":1.661},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.1160000000000001,"5min":3.492,"15min":3.492},"last":1.359},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.621,"15min":1.7130000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.0449999999999999,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.2290000000000001}]}]}],"pool_statfs":[]}} 2023-10-18T08:58:39.447 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T08:58:39.447 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T08:58:39.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.724+0000 7fea92da9640 1 -- 172.21.15.40:0/2597679543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea8c0d1aa0 msgr2=0x7fea8c0cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:39.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.724+0000 7fea92da9640 1 --2- 172.21.15.40:0/2597679543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea8c0d1aa0 0x7fea8c0cdf80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fea8c08b9f0 tx=0x7fea8402c260 comp rx=0 tx=0).stop 2023-10-18T08:58:39.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.724+0000 7fea92da9640 1 -- 172.21.15.40:0/2597679543 shutdown_connections 2023-10-18T08:58:39.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.724+0000 7fea92da9640 1 --2- 172.21.15.40:0/2597679543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea8c0d1aa0 0x7fea8c0cdf80 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.724+0000 7fea92da9640 1 --2- 172.21.15.40:0/2597679543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c0cff00 0x7fea8c0d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.724+0000 7fea92da9640 1 --2- 172.21.15.40:0/2597679543 >> v2:172.21.15.144:3300/0 conn(0x7fea8c15b6c0 0x7fea8c0b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.724+0000 7fea92da9640 1 -- 172.21.15.40:0/2597679543 >> 172.21.15.40:0/2597679543 conn(0x7fea8c09a960 msgr2=0x7fea8c088fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.724+0000 7fea92da9640 1 -- 172.21.15.40:0/2597679543 shutdown_connections 2023-10-18T08:58:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.724+0000 7fea92da9640 1 -- 172.21.15.40:0/2597679543 wait complete. 2023-10-18T08:58:39.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.726+0000 7fea92da9640 1 Processor -- start 2023-10-18T08:58:39.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.726+0000 7fea92da9640 1 -- start start 2023-10-18T08:58:39.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.727+0000 7fea92da9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea8c0cff00 0x7fea8c14dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:39.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.727+0000 7fea92da9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea8c0d1aa0 0x7fea8c0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.727+0000 7fea92da9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c15b6c0 0x7fea8c15df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.727+0000 7fea92da9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c0c7370 con 0x7fea8c0d1aa0 2023-10-18T08:58:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.727+0000 7fea92da9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c0c6760 con 0x7fea8c0cff00 2023-10-18T08:58:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.727+0000 7fea92da9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c0c6a40 con 0x7fea8c15b6c0 2023-10-18T08:58:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.728+0000 7fea8bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea8c0d1aa0 0x7fea8c0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:39.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.728+0000 7fea8bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea8c0d1aa0 0x7fea8c0c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43384/0 (socket says 172.21.15.40:43384) 2023-10-18T08:58:39.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.728+0000 7fea8bfff640 1 -- 172.21.15.40:0/1008216013 learned_addr learned my addr 172.21.15.40:0/1008216013 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:39.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.728+0000 7fea90b45640 1 --2- 172.21.15.40:0/1008216013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea8c0cff00 0x7fea8c14dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:39.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.728+0000 7fea91346640 1 --2- 172.21.15.40:0/1008216013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c15b6c0 0x7fea8c15df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:39.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.728+0000 7fea8bfff640 1 -- 172.21.15.40:0/1008216013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c15b6c0 msgr2=0x7fea8c15df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:39.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.728+0000 7fea8bfff640 1 --2- 172.21.15.40:0/1008216013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c15b6c0 0x7fea8c15df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:39.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.728+0000 7fea8bfff640 1 -- 172.21.15.40:0/1008216013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea8c0cff00 msgr2=0x7fea8c14dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:39.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.728+0000 7fea8bfff640 1 --2- 172.21.15.40:0/1008216013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea8c0cff00 0x7fea8c14dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.728+0000 7fea8bfff640 1 -- 172.21.15.40:0/1008216013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea84009900 con 0x7fea8c0d1aa0 2023-10-18T08:58:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.728+0000 7fea91346640 1 --2- 172.21.15.40:0/1008216013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c15b6c0 0x7fea8c15df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.728+0000 7fea90b45640 1 --2- 172.21.15.40:0/1008216013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea8c0cff00 0x7fea8c14dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:58:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.729+0000 7fea8bfff640 1 --2- 172.21.15.40:0/1008216013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea8c0d1aa0 0x7fea8c0c8830 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fea7800c940 tx=0x7fea7800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.729+0000 7fea897fa640 1 -- 172.21.15.40:0/1008216013 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78007bb0 con 0x7fea8c0d1aa0 2023-10-18T08:58:39.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.729+0000 7fea897fa640 1 -- 172.21.15.40:0/1008216013 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fea78007e00 con 0x7fea8c0d1aa0 2023-10-18T08:58:39.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.729+0000 7fea92da9640 1 -- 172.21.15.40:0/1008216013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea8c0c6d20 con 0x7fea8c0d1aa0 2023-10-18T08:58:39.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.729+0000 7fea897fa640 1 -- 172.21.15.40:0/1008216013 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78005280 con 0x7fea8c0d1aa0 2023-10-18T08:58:39.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.729+0000 7fea92da9640 1 -- 172.21.15.40:0/1008216013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea8c14e200 con 0x7fea8c0d1aa0 2023-10-18T08:58:39.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.730+0000 7fea92da9640 1 -- 172.21.15.40:0/1008216013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea8c0739c0 con 0x7fea8c0d1aa0 2023-10-18T08:58:39.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.731+0000 7fea897fa640 1 -- 172.21.15.40:0/1008216013 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fea780054a0 con 0x7fea8c0d1aa0 2023-10-18T08:58:39.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.732+0000 7fea897fa640 1 --2- 172.21.15.40:0/1008216013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea7006ae40 0x7fea7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:39.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.732+0000 7fea897fa640 1 -- 172.21.15.40:0/1008216013 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea78089e50 con 0x7fea8c0d1aa0 2023-10-18T08:58:39.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.733+0000 7fea90b45640 1 --2- 172.21.15.40:0/1008216013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea7006ae40 0x7fea7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:39.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.734+0000 7fea90b45640 1 --2- 172.21.15.40:0/1008216013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea7006ae40 0x7fea7006d300 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fea80005940 tx=0x7fea80005e30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:39.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.737+0000 7fea897fa640 1 -- 172.21.15.40:0/1008216013 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea78054ef0 con 0x7fea8c0d1aa0 2023-10-18T08:58:39.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:39.997+0000 7fea92da9640 1 -- 172.21.15.40:0/1008216013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fea8c072b00 con 0x7fea8c0d1aa0 2023-10-18T08:58:40.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.003+0000 7fea897fa640 1 -- 172.21.15.40:0/1008216013 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fea7801d070 con 0x7fea8c0d1aa0 2023-10-18T08:58:40.005 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2023-10-18T08:58:40.005 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w== 2023-10-18T08:58:40.006 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2023-10-18T08:58:40.006 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2023-10-18T08:58:40.006 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T08:58:40.006 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2023-10-18T08:58:40.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.005+0000 7fea92da9640 1 -- 172.21.15.40:0/1008216013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea7006ae40 msgr2=0x7fea7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:40.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.005+0000 7fea92da9640 1 --2- 172.21.15.40:0/1008216013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea7006ae40 0x7fea7006d300 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fea80005940 tx=0x7fea80005e30 comp rx=0 tx=0).stop 2023-10-18T08:58:40.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.005+0000 7fea92da9640 1 -- 172.21.15.40:0/1008216013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea8c0d1aa0 msgr2=0x7fea8c0c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:40.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.005+0000 7fea92da9640 1 --2- 172.21.15.40:0/1008216013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea8c0d1aa0 0x7fea8c0c8830 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fea7800c940 tx=0x7fea7800ce00 comp rx=0 tx=0).stop 2023-10-18T08:58:40.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.005+0000 7fea92da9640 1 -- 172.21.15.40:0/1008216013 shutdown_connections 2023-10-18T08:58:40.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.005+0000 7fea92da9640 1 --2- 172.21.15.40:0/1008216013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea7006ae40 0x7fea7006d300 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:40.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.005+0000 7fea92da9640 1 --2- 172.21.15.40:0/1008216013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea8c15b6c0 0x7fea8c15df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:40.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.005+0000 7fea92da9640 1 --2- 172.21.15.40:0/1008216013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea8c0d1aa0 0x7fea8c0c8830 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:40.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.005+0000 7fea92da9640 1 --2- 172.21.15.40:0/1008216013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea8c0cff00 0x7fea8c14dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:40.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.005+0000 7fea92da9640 1 -- 172.21.15.40:0/1008216013 >> 172.21.15.40:0/1008216013 conn(0x7fea8c09a960 msgr2=0x7fea8c15a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:40.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.005+0000 7fea92da9640 1 -- 172.21.15.40:0/1008216013 shutdown_connections 2023-10-18T08:58:40.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.005+0000 7fea92da9640 1 -- 172.21.15.40:0/1008216013 wait complete. 2023-10-18T08:58:40.027 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:58:40.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.248+0000 7f191359e640 1 -- 172.21.15.40:0/2582795821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19141420f0 msgr2=0x7f19140b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.248+0000 7f191359e640 1 --2- 172.21.15.40:0/2582795821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19141420f0 0x7f19140b0a80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1904005fd0 tx=0x7f190402be20 comp rx=0 tx=0).stop 2023-10-18T08:58:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.248+0000 7f191359e640 1 -- 172.21.15.40:0/2582795821 shutdown_connections 2023-10-18T08:58:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.248+0000 7f191359e640 1 --2- 172.21.15.40:0/2582795821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19141420f0 0x7f19140b0a80 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:40.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.248+0000 7f191359e640 1 -- 172.21.15.40:0/2582795821 >> 172.21.15.40:0/2582795821 conn(0x7f1914069720 msgr2=0x7f1914069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:40.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.248+0000 7f191359e640 1 -- 172.21.15.40:0/2582795821 shutdown_connections 2023-10-18T08:58:40.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.248+0000 7f191359e640 1 -- 172.21.15.40:0/2582795821 wait complete. 2023-10-18T08:58:40.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.250+0000 7f191359e640 1 Processor -- start 2023-10-18T08:58:40.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.250+0000 7f191359e640 1 -- start start 2023-10-18T08:58:40.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.251+0000 7f191359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19140ca160 0x7f1914158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:40.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.251+0000 7f191359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19141420f0 0x7f191415b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:40.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.251+0000 7f191359e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19141425a0 0x7f191415bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:40.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.251+0000 7f191359e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19141f6350 con 0x7f19141420f0 2023-10-18T08:58:40.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.251+0000 7f191359e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19140ace80 con 0x7f19140ca160 2023-10-18T08:58:40.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.251+0000 7f191359e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19140ad160 con 0x7f19141425a0 2023-10-18T08:58:40.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.251+0000 7f1909d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19141420f0 0x7f191415b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:40.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.252+0000 7f1909d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19141420f0 0x7f191415b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43404/0 (socket says 172.21.15.40:43404) 2023-10-18T08:58:40.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.252+0000 7f1909d9b640 1 -- 172.21.15.40:0/2483802209 learned_addr learned my addr 172.21.15.40:0/2483802209 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:40.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.252+0000 7f1912d9d640 1 --2- 172.21.15.40:0/2483802209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19141425a0 0x7f191415bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:40.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.252+0000 7f191259c640 1 --2- 172.21.15.40:0/2483802209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19140ca160 0x7f1914158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:40.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.252+0000 7f1909d9b640 1 -- 172.21.15.40:0/2483802209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19141425a0 msgr2=0x7f191415bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:40.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.252+0000 7f1909d9b640 1 --2- 172.21.15.40:0/2483802209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19141425a0 0x7f191415bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:40.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.252+0000 7f1909d9b640 1 -- 172.21.15.40:0/2483802209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19140ca160 msgr2=0x7f1914158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:40.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.252+0000 7f1909d9b640 1 --2- 172.21.15.40:0/2483802209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19140ca160 0x7f1914158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:40.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.252+0000 7f1909d9b640 1 -- 172.21.15.40:0/2483802209 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1904009560 con 0x7f19141420f0 2023-10-18T08:58:40.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.253+0000 7f1909d9b640 1 --2- 172.21.15.40:0/2483802209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19141420f0 0x7f191415b750 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00b3f0 tx=0x7f18fc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:40.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.253+0000 7f190b7fe640 1 -- 172.21.15.40:0/2483802209 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc011830 con 0x7f19141420f0 2023-10-18T08:58:40.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.253+0000 7f190b7fe640 1 -- 172.21.15.40:0/2483802209 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f18fc011a80 con 0x7f19141420f0 2023-10-18T08:58:40.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.253+0000 7f190b7fe640 1 -- 172.21.15.40:0/2483802209 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc011e90 con 0x7f19141420f0 2023-10-18T08:58:40.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.253+0000 7f191359e640 1 -- 172.21.15.40:0/2483802209 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19140ad440 con 0x7f19141420f0 2023-10-18T08:58:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.253+0000 7f191359e640 1 -- 172.21.15.40:0/2483802209 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19140adc40 con 0x7f19141420f0 2023-10-18T08:58:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.254+0000 7f191359e640 1 -- 172.21.15.40:0/2483802209 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19140ad960 con 0x7f19141420f0 2023-10-18T08:58:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.255+0000 7f190b7fe640 1 -- 172.21.15.40:0/2483802209 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f18fc01a400 con 0x7f19141420f0 2023-10-18T08:58:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.256+0000 7f190b7fe640 1 --2- 172.21.15.40:0/2483802209 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18f406ae40 0x7f18f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:40.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.256+0000 7f190b7fe640 1 -- 172.21.15.40:0/2483802209 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18fc0897c0 con 0x7f19141420f0 2023-10-18T08:58:40.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.257+0000 7f191259c640 1 --2- 172.21.15.40:0/2483802209 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18f406ae40 0x7f18f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:40.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.258+0000 7f191259c640 1 --2- 172.21.15.40:0/2483802209 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18f406ae40 0x7f18f406d300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1904035000 tx=0x7f1904004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:40.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.260+0000 7f190b7fe640 1 -- 172.21.15.40:0/2483802209 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f19140ad960 con 0x7f19141420f0 2023-10-18T08:58:40.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.489+0000 7f191359e640 1 -- 172.21.15.40:0/2483802209 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19140ad960 con 0x7f19141420f0 2023-10-18T08:58:40.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.490+0000 7f190b7fe640 1 -- 172.21.15.40:0/2483802209 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f19140ad960 con 0x7f19141420f0 2023-10-18T08:58:40.493 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:40.493 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5075,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2165596806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2165596806},{"type":"v1","addr":"172.21.15.144:6835","nonce":2165596806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/587522883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":587522883},{"type":"v1","addr":"172.21.15.40:6833","nonce":587522883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1174134524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1174134524},{"type":"v1","addr":"172.21.15.40:6835","nonce":1174134524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:58:28.760677+0000","modified":"2023-10-18T08:58:31.845684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5389":{"gid":5389,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1311290970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1311290970},{"type":"v1","addr":"172.21.15.144:6837","nonce":1311290970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-18T08:58:40.494 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 33 2023-10-18T08:58:40.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.493+0000 7f191359e640 1 -- 172.21.15.40:0/2483802209 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18f406ae40 msgr2=0x7f18f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:40.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.493+0000 7f191359e640 1 --2- 172.21.15.40:0/2483802209 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18f406ae40 0x7f18f406d300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1904035000 tx=0x7f1904004730 comp rx=0 tx=0).stop 2023-10-18T08:58:40.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.493+0000 7f191359e640 1 -- 172.21.15.40:0/2483802209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19141420f0 msgr2=0x7f191415b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:40.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.493+0000 7f191359e640 1 --2- 172.21.15.40:0/2483802209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19141420f0 0x7f191415b750 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00b3f0 tx=0x7f18fc00b8b0 comp rx=0 tx=0).stop 2023-10-18T08:58:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.494+0000 7f191359e640 1 -- 172.21.15.40:0/2483802209 shutdown_connections 2023-10-18T08:58:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.494+0000 7f191359e640 1 --2- 172.21.15.40:0/2483802209 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18f406ae40 0x7f18f406d300 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.494+0000 7f191359e640 1 --2- 172.21.15.40:0/2483802209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19141425a0 0x7f191415bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.494+0000 7f191359e640 1 --2- 172.21.15.40:0/2483802209 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19141420f0 0x7f191415b750 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.494+0000 7f191359e640 1 --2- 172.21.15.40:0/2483802209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19140ca160 0x7f1914158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:40.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.494+0000 7f191359e640 1 -- 172.21.15.40:0/2483802209 >> 172.21.15.40:0/2483802209 conn(0x7f1914069720 msgr2=0x7f1914070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:40.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.494+0000 7f191359e640 1 -- 172.21.15.40:0/2483802209 shutdown_connections 2023-10-18T08:58:40.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:40.494+0000 7f191359e640 1 -- 172.21.15.40:0/2483802209 wait complete. 2023-10-18T08:58:40.919 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T08:58:28.760677+0000', 'modified': '2023-10-18T08:58:31.845684+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5389}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5389': {'gid': 5389, 'name': 'd', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.144:6837/1311290970', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6836', 'nonce': 1311290970}, {'type': 'v1', 'addr': '172.21.15.144:6837', 'nonce': 1311290970}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T08:58:40.919 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T08:58:40.920 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-18T08:58:41.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.160+0000 7f29d9867640 1 -- 172.21.15.40:0/763040311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d41425a0 msgr2=0x7f29d40c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:41.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.160+0000 7f29d9867640 1 --2- 172.21.15.40:0/763040311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d41425a0 0x7f29d40c9a50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f29d415dfb0 tx=0x7f29c002bd90 comp rx=0 tx=0).stop 2023-10-18T08:58:41.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.160+0000 7f29d9867640 1 -- 172.21.15.40:0/763040311 shutdown_connections 2023-10-18T08:58:41.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.160+0000 7f29d9867640 1 --2- 172.21.15.40:0/763040311 >> v2:172.21.15.144:3300/0 conn(0x7f29d40c9f90 0x7f29d406b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.160+0000 7f29d9867640 1 --2- 172.21.15.40:0/763040311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d41425a0 0x7f29d40c9a50 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.160+0000 7f29d9867640 1 --2- 172.21.15.40:0/763040311 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29d41420f0 0x7f29d40b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.160+0000 7f29d9867640 1 -- 172.21.15.40:0/763040311 >> 172.21.15.40:0/763040311 conn(0x7f29d4069720 msgr2=0x7f29d4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.160+0000 7f29d9867640 1 -- 172.21.15.40:0/763040311 shutdown_connections 2023-10-18T08:58:41.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.160+0000 7f29d9867640 1 -- 172.21.15.40:0/763040311 wait complete. 2023-10-18T08:58:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.162+0000 7f29d9867640 1 Processor -- start 2023-10-18T08:58:41.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.162+0000 7f29d9867640 1 -- start start 2023-10-18T08:58:41.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.163+0000 7f29d9867640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d40c9f90 0x7f29d415b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.163+0000 7f29d9867640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29d41420f0 0x7f29d4158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.163+0000 7f29d9867640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29d41425a0 0x7f29d4140730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.163+0000 7f29d9867640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d415b500 con 0x7f29d40c9f90 2023-10-18T08:58:41.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.163+0000 7f29d9867640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d415a8f0 con 0x7f29d41425a0 2023-10-18T08:58:41.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.163+0000 7f29d9867640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d415abd0 con 0x7f29d41420f0 2023-10-18T08:58:41.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.164+0000 7f29d8865640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d40c9f90 0x7f29d415b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.164+0000 7f29d8865640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d40c9f90 0x7f29d415b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43430/0 (socket says 172.21.15.40:43430) 2023-10-18T08:58:41.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.164+0000 7f29d8865640 1 -- 172.21.15.40:0/3188798767 learned_addr learned my addr 172.21.15.40:0/3188798767 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:41.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.164+0000 7f29d3fff640 1 --2- 172.21.15.40:0/3188798767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29d41420f0 0x7f29d4158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.164+0000 7f29d9066640 1 --2- 172.21.15.40:0/3188798767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29d41425a0 0x7f29d4140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.164+0000 7f29d8865640 1 -- 172.21.15.40:0/3188798767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29d41420f0 msgr2=0x7f29d4158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.164+0000 7f29d8865640 1 --2- 172.21.15.40:0/3188798767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29d41420f0 0x7f29d4158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.164+0000 7f29d8865640 1 -- 172.21.15.40:0/3188798767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29d41425a0 msgr2=0x7f29d4140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:41.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.164+0000 7f29d8865640 1 --2- 172.21.15.40:0/3188798767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29d41425a0 0x7f29d4140730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.164+0000 7f29d8865640 1 -- 172.21.15.40:0/3188798767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c0009560 con 0x7f29d40c9f90 2023-10-18T08:58:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.164+0000 7f29d8865640 1 --2- 172.21.15.40:0/3188798767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d40c9f90 0x7f29d415b990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f29c800b4a0 tx=0x7f29c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.165+0000 7f29d17fa640 1 -- 172.21.15.40:0/3188798767 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c8010610 con 0x7f29d40c9f90 2023-10-18T08:58:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.165+0000 7f29d17fa640 1 -- 172.21.15.40:0/3188798767 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29c8010860 con 0x7f29d40c9f90 2023-10-18T08:58:41.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.165+0000 7f29d17fa640 1 -- 172.21.15.40:0/3188798767 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c8010c70 con 0x7f29d40c9f90 2023-10-18T08:58:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.165+0000 7f29d9867640 1 -- 172.21.15.40:0/3188798767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d415aeb0 con 0x7f29d40c9f90 2023-10-18T08:58:41.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.165+0000 7f29d9867640 1 -- 172.21.15.40:0/3188798767 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d4159080 con 0x7f29d40c9f90 2023-10-18T08:58:41.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.167+0000 7f29d17fa640 1 -- 172.21.15.40:0/3188798767 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f29c8017360 con 0x7f29d40c9f90 2023-10-18T08:58:41.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.168+0000 7f29d17fa640 1 --2- 172.21.15.40:0/3188798767 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29b806ae40 0x7f29b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.168+0000 7f29d17fa640 1 -- 172.21.15.40:0/3188798767 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29c80894d0 con 0x7f29d40c9f90 2023-10-18T08:58:41.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.169+0000 7f29dd658740 1 Processor -- start 2023-10-18T08:58:41.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.169+0000 7f29d3fff640 1 --2- 172.21.15.40:0/3188798767 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29b806ae40 0x7f29b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.169+0000 7f29dd658740 1 -- start start 2023-10-18T08:58:41.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.170+0000 7f29dd658740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0970 0x5648c3c93ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.170+0000 7f29dd658740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc0d50 0x5648c3cc3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.170+0000 7f29dd658740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3c940e0 0x5648c3cc3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.170+0000 7f29dd658740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648c3c06740 con 0x5648c3cc0d50 2023-10-18T08:58:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.170+0000 7f29dd658740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648c3c03e30 con 0x5648c3c940e0 2023-10-18T08:58:41.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.170+0000 7f29dd658740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648c3bff720 con 0x5648c3cc0970 2023-10-18T08:58:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.170+0000 7f29d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc0d50 0x5648c3cc3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.170+0000 7f29d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc0d50 0x5648c3cc3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43434/0 (socket says 172.21.15.40:43434) 2023-10-18T08:58:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.170+0000 7f29d3fff640 1 -- 172.21.15.40:0/4267804551 learned_addr learned my addr 172.21.15.40:0/4267804551 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.170+0000 7f29d3fff640 1 --2- 172.21.15.40:0/3188798767 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29b806ae40 0x7f29b806d300 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f29c0035000 tx=0x7f29c0004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:41.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.170+0000 7f29d8865640 1 --2- 172.21.15.40:0/4267804551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0970 0x5648c3c93ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.170+0000 7f29d9066640 1 --2- 172.21.15.40:0/4267804551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3c940e0 0x5648c3cc3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.171+0000 7f29d3fff640 1 -- 172.21.15.40:0/4267804551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0970 msgr2=0x5648c3c93ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:58:41.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.171+0000 7f29d3fff640 1 --2- 172.21.15.40:0/4267804551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0970 0x5648c3c93ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.171+0000 7f29d3fff640 1 -- 172.21.15.40:0/4267804551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3c940e0 msgr2=0x5648c3cc3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:58:41.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.171+0000 7f29d3fff640 1 --2- 172.21.15.40:0/4267804551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3c940e0 0x5648c3cc3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.171+0000 7f29d3fff640 1 -- 172.21.15.40:0/4267804551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648c3bc3910 con 0x5648c3cc0d50 2023-10-18T08:58:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.171+0000 7f29d9066640 1 --2- 172.21.15.40:0/4267804551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3c940e0 0x5648c3cc3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:58:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.171+0000 7f29d3fff640 1 --2- 172.21.15.40:0/4267804551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc0d50 0x5648c3cc3700 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f29c002c3a0 tx=0x7f29c003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c60fba0f4d64d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.171+0000 7f29be7fc640 1 -- 172.21.15.40:0/4267804551 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c002cc80 con 0x5648c3cc0d50 2023-10-18T08:58:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.171+0000 7f29be7fc640 1 -- 172.21.15.40:0/4267804551 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29c0045070 con 0x5648c3cc0d50 2023-10-18T08:58:41.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.171+0000 7f29be7fc640 1 -- 172.21.15.40:0/4267804551 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c00037f0 con 0x5648c3cc0d50 2023-10-18T08:58:41.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.172+0000 7f29dd658740 1 -- 172.21.15.40:0/4267804551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc0d50 msgr2=0x5648c3cc3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:41.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.172+0000 7f29dd658740 1 --2- 172.21.15.40:0/4267804551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc0d50 0x5648c3cc3700 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f29c002c3a0 tx=0x7f29c003e000 comp rx=0 tx=0).stop 2023-10-18T08:58:41.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.172+0000 7f29dd658740 1 -- 172.21.15.40:0/4267804551 shutdown_connections 2023-10-18T08:58:41.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.172+0000 7f29dd658740 1 --2- 172.21.15.40:0/4267804551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3c940e0 0x5648c3cc3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.172+0000 7f29dd658740 1 --2- 172.21.15.40:0/4267804551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc0d50 0x5648c3cc3700 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.172+0000 7f29dd658740 1 --2- 172.21.15.40:0/4267804551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0970 0x5648c3c93ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.172+0000 7f29dd658740 1 -- 172.21.15.40:0/4267804551 >> 172.21.15.40:0/4267804551 conn(0x5648c3c95e10 msgr2=0x5648c3cde700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:41.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.172+0000 7f29dd658740 1 -- 172.21.15.40:0/4267804551 shutdown_connections 2023-10-18T08:58:41.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.172+0000 7f29dd658740 1 -- 172.21.15.40:0/4267804551 wait complete. 2023-10-18T08:58:41.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.173+0000 7f29dd658740 1 -- start start 2023-10-18T08:58:41.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.173+0000 7f29dd658740 1 Processor -- start 2023-10-18T08:58:41.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.174+0000 7f29dd658740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0970 0x5648c3cadd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.174+0000 7f29dd658740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc0d50 0x5648c3cc8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.174+0000 7f29dd658740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3cc3420 0x5648c3cb0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.174+0000 7f29dd658740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648c3b9d970 con 0x5648c3cc0d50 2023-10-18T08:58:41.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.174+0000 7f29dd658740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648c3b9d610 con 0x5648c3cc3420 2023-10-18T08:58:41.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.174+0000 7f29dd658740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648c3963f20 con 0x5648c3cc0970 2023-10-18T08:58:41.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.174+0000 7f29dd658740 10 client.0 _ll_register_callbacks cb 0x5648c3bc10c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T08:58:41.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.175+0000 7f29d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc0d50 0x5648c3cc8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.175+0000 7f29d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc0d50 0x5648c3cc8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43450/0 (socket says 172.21.15.40:43450) 2023-10-18T08:58:41.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.175+0000 7f29d3fff640 1 -- 172.21.15.40:0/3868397231 learned_addr learned my addr 172.21.15.40:0/3868397231 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:41.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.175+0000 7f29d8865640 1 --2- 172.21.15.40:0/3868397231 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0970 0x5648c3cadd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.175+0000 7f29d3fff640 1 -- 172.21.15.40:0/3868397231 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0970 msgr2=0x5648c3cadd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:58:41.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.175+0000 7f29d3fff640 1 --2- 172.21.15.40:0/3868397231 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0970 0x5648c3cadd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.175+0000 7f29d3fff640 1 -- 172.21.15.40:0/3868397231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3cc3420 msgr2=0x5648c3cb0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:58:41.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.175+0000 7f29d3fff640 1 --2- 172.21.15.40:0/3868397231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3cc3420 0x5648c3cb0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.175+0000 7f29d3fff640 1 -- 172.21.15.40:0/3868397231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d415d330 con 0x5648c3cc0d50 2023-10-18T08:58:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.175+0000 7f29d3fff640 1 --2- 172.21.15.40:0/3868397231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc0d50 0x5648c3cc8430 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f29c0036040 tx=0x7f29c0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6db78eb8a542eeb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.175+0000 7f29b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T08:58:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.176+0000 7f29b37fe640 1 -- 172.21.15.40:0/3868397231 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c002cc80 con 0x5648c3cc0d50 2023-10-18T08:58:41.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.176+0000 7f29b37fe640 1 -- 172.21.15.40:0/3868397231 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29c0003f30 con 0x5648c3cc0d50 2023-10-18T08:58:41.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.176+0000 7f29dd658740 1 -- 172.21.15.40:0/3868397231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648c3bf8700 con 0x5648c3cc0d50 2023-10-18T08:58:41.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.176+0000 7f29b37fe640 1 -- 172.21.15.40:0/3868397231 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29c0041680 con 0x5648c3cc0d50 2023-10-18T08:58:41.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.176+0000 7f29dd658740 10 client.6318 fetch_fsmap learned FSMap version 33 2023-10-18T08:58:41.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.176+0000 7f29dd658740 1 -- 172.21.15.40:0/3868397231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5648c3bfdda0 con 0x5648c3cc0d50 2023-10-18T08:58:41.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.176+0000 7f29b37fe640 1 -- 172.21.15.40:0/3868397231 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x5648c3bf8700 con 0x5648c3cc0d50 2023-10-18T08:58:41.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.176+0000 7f29b37fe640 1 -- 172.21.15.40:0/3868397231 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29c00417e0 con 0x5648c3cc0d50 2023-10-18T08:58:41.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.177+0000 7f29dd658740 10 client.6318 fetch_fsmap finished waiting for FSMap version 33 2023-10-18T08:58:41.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.177+0000 7f29dd658740 10 client.6318 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-18T08:58:41.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.177+0000 7f29dd658740 20 client.6318 populate_metadata read hostname 'smithi040' 2023-10-18T08:58:41.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.177+0000 7f29dd658740 1 --2- 172.21.15.40:0/3868397231 >> [v2:172.21.15.144:6836/1311290970,v1:172.21.15.144:6837/1311290970] conn(0x5648c3cab5d0 0x5648c3caa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.177+0000 7f29dd658740 4 client.6318 mds_command: new command op to 5389 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T08:58:41.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.177+0000 7f29dd658740 1 -- 172.21.15.40:0/3868397231 --> [v2:172.21.15.144:6836/1311290970,v1:172.21.15.144:6837/1311290970] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648c3bf8420 con 0x5648c3cab5d0 2023-10-18T08:58:41.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.177+0000 7f29d8865640 1 --2- 172.21.15.40:0/3868397231 >> [v2:172.21.15.144:6836/1311290970,v1:172.21.15.144:6837/1311290970] conn(0x5648c3cab5d0 0x5648c3caa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.178+0000 7f29d8865640 1 --2- 172.21.15.40:0/3868397231 >> [v2:172.21.15.144:6836/1311290970,v1:172.21.15.144:6837/1311290970] conn(0x5648c3cab5d0 0x5648c3caa240 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=746d25b401d9c1b3 server_cookie=a7d74a7faf5d80b5 in_seq=0 out_seq=0 2023-10-18T08:58:41.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.178+0000 7f29b37fe640 10 client.6318 ms_handle_connect on v2:172.21.15.144:6836/1311290970 2023-10-18T08:58:41.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.180+0000 7f29b37fe640 1 -- 172.21.15.40:0/3868397231 <== mds.0 v2:172.21.15.144:6836/1311290970 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5648c3bf8420 con 0x5648c3cab5d0 2023-10-18T08:58:41.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.180+0000 7f29b37fe640 10 client.6318 handle_command_reply: tid=0 2023-10-18T08:58:41.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.180+0000 7f29dd658740 1 client.6318 shutdown 2023-10-18T08:58:41.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.181+0000 7f29dd658740 1 -- 172.21.15.40:0/3868397231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc0d50 msgr2=0x5648c3cc8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:41.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.181+0000 7f29dd658740 1 --2- 172.21.15.40:0/3868397231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc0d50 0x5648c3cc8430 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f29c0036040 tx=0x7f29c0003d40 comp rx=0 tx=0).stop 2023-10-18T08:58:41.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.181+0000 7f29dd658740 1 -- 172.21.15.40:0/3868397231 shutdown_connections 2023-10-18T08:58:41.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.181+0000 7f29dd658740 1 --2- 172.21.15.40:0/3868397231 >> [v2:172.21.15.144:6836/1311290970,v1:172.21.15.144:6837/1311290970] conn(0x5648c3cab5d0 0x5648c3caa240 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.181+0000 7f29dd658740 1 --2- 172.21.15.40:0/3868397231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3cc3420 0x5648c3cb0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.181+0000 7f29dd658740 1 --2- 172.21.15.40:0/3868397231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc0d50 0x5648c3cc8430 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.181+0000 7f29dd658740 1 --2- 172.21.15.40:0/3868397231 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0970 0x5648c3cadd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.181+0000 7f29b37fe640 0 client.6318 ms_handle_reset on v2:172.21.15.144:6836/1311290970 2023-10-18T08:58:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.181+0000 7f29dd658740 1 -- 172.21.15.40:0/3868397231 >> 172.21.15.40:0/3868397231 conn(0x5648c3c940e0 msgr2=0x5648c3c97780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.181+0000 7f29dd658740 1 -- 172.21.15.40:0/3868397231 shutdown_connections 2023-10-18T08:58:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.181+0000 7f29dd658740 1 -- 172.21.15.40:0/3868397231 wait complete. 2023-10-18T08:58:41.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.181+0000 7f29dd658740 20 client.6318 trim_cache size 0 max 16384 2023-10-18T08:58:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.198+0000 7f29dd658740 1 Processor -- start 2023-10-18T08:58:41.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.198+0000 7f29dd658740 1 -- start start 2023-10-18T08:58:41.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.198+0000 7f29dd658740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3cc0970 0x5648c3cbe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.198+0000 7f29dd658740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0d50 0x5648c3caaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.198+0000 7f29dd658740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc3420 0x5648c3d3d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.199+0000 7f29dd658740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648c3cc4be0 con 0x5648c3cc3420 2023-10-18T08:58:41.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.199+0000 7f29dd658740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648c3cc3820 con 0x5648c3cc0970 2023-10-18T08:58:41.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.199+0000 7f29dd658740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648c3c01600 con 0x5648c3cc0d50 2023-10-18T08:58:41.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.199+0000 7f29d9066640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc3420 0x5648c3d3d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.199+0000 7f29d9066640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc3420 0x5648c3d3d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43458/0 (socket says 172.21.15.40:43458) 2023-10-18T08:58:41.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.199+0000 7f29d9066640 1 -- 172.21.15.40:0/963101802 learned_addr learned my addr 172.21.15.40:0/963101802 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:41.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.199+0000 7f29d9066640 1 -- 172.21.15.40:0/963101802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0d50 msgr2=0x5648c3caaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:58:41.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.199+0000 7f29d9066640 1 --2- 172.21.15.40:0/963101802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0d50 0x5648c3caaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.199+0000 7f29d9066640 1 -- 172.21.15.40:0/963101802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3cc0970 msgr2=0x5648c3cbe9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:58:41.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.199+0000 7f29d9066640 1 --2- 172.21.15.40:0/963101802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3cc0970 0x5648c3cbe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.199+0000 7f29d9066640 1 -- 172.21.15.40:0/963101802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648c3b9cb80 con 0x5648c3cc3420 2023-10-18T08:58:41.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.200+0000 7f29d9066640 1 --2- 172.21.15.40:0/963101802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc3420 0x5648c3d3d1e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00eb30 tx=0x7f29cc0103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aeb619c5aff02a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:41.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.200+0000 7f29beffd640 1 -- 172.21.15.40:0/963101802 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc010e00 con 0x5648c3cc3420 2023-10-18T08:58:41.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.200+0000 7f29beffd640 1 -- 172.21.15.40:0/963101802 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29cc015070 con 0x5648c3cc3420 2023-10-18T08:58:41.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.200+0000 7f29beffd640 1 -- 172.21.15.40:0/963101802 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc003970 con 0x5648c3cc3420 2023-10-18T08:58:41.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.200+0000 7f29dd658740 1 -- 172.21.15.40:0/963101802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc3420 msgr2=0x5648c3d3d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:41.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.200+0000 7f29dd658740 1 --2- 172.21.15.40:0/963101802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc3420 0x5648c3d3d1e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00eb30 tx=0x7f29cc0103d0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.200+0000 7f29dd658740 1 -- 172.21.15.40:0/963101802 shutdown_connections 2023-10-18T08:58:41.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.200+0000 7f29dd658740 1 --2- 172.21.15.40:0/963101802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc3420 0x5648c3d3d1e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.200+0000 7f29dd658740 1 --2- 172.21.15.40:0/963101802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0d50 0x5648c3caaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.200+0000 7f29dd658740 1 --2- 172.21.15.40:0/963101802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3cc0970 0x5648c3cbe9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.200+0000 7f29dd658740 1 -- 172.21.15.40:0/963101802 >> 172.21.15.40:0/963101802 conn(0x5648c3c940e0 msgr2=0x5648c3c8f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:41.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.201+0000 7f29dd658740 1 -- 172.21.15.40:0/963101802 shutdown_connections 2023-10-18T08:58:41.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.201+0000 7f29dd658740 1 -- 172.21.15.40:0/963101802 wait complete. 2023-10-18T08:58:41.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.201+0000 7f29dd658740 1 -- start start 2023-10-18T08:58:41.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.201+0000 7f29dd658740 1 Processor -- start 2023-10-18T08:58:41.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.202+0000 7f29dd658740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0970 0x5648c3cde130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.202+0000 7f29dd658740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3cc0d50 0x5648c3d480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.202+0000 7f29dd658740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc3420 0x5648c3d485f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.202+0000 7f29dd658740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648c3d48e40 con 0x5648c3cc3420 2023-10-18T08:58:41.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.202+0000 7f29dd658740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648c3cbbc30 con 0x5648c3cc0d50 2023-10-18T08:58:41.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.202+0000 7f29dd658740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648c3cbbf10 con 0x5648c3cc0970 2023-10-18T08:58:41.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.202+0000 7f29dd658740 10 client.0 _ll_register_callbacks cb 0x7f29c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T08:58:41.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.203+0000 7f29d9066640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc3420 0x5648c3d485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.203+0000 7f29d9066640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc3420 0x5648c3d485f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43468/0 (socket says 172.21.15.40:43468) 2023-10-18T08:58:41.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.203+0000 7f29d9066640 1 -- 172.21.15.40:0/204372916 learned_addr learned my addr 172.21.15.40:0/204372916 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:41.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.203+0000 7f29d9066640 1 -- 172.21.15.40:0/204372916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0970 msgr2=0x5648c3cde130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:58:41.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.203+0000 7f29d9066640 1 --2- 172.21.15.40:0/204372916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0970 0x5648c3cde130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.203+0000 7f29d9066640 1 -- 172.21.15.40:0/204372916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3cc0d50 msgr2=0x5648c3d480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T08:58:41.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.203+0000 7f29d9066640 1 --2- 172.21.15.40:0/204372916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3cc0d50 0x5648c3d480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.203+0000 7f29d9066640 1 -- 172.21.15.40:0/204372916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29cc00e790 con 0x5648c3cc3420 2023-10-18T08:58:41.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.203+0000 7f29d9066640 1 --2- 172.21.15.40:0/204372916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc3420 0x5648c3d485f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x5648c3cc4d20 tx=0x7f29cc010e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe28e153e322f51f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:41.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.203+0000 7f29b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T08:58:41.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.204+0000 7f29b37fe640 1 -- 172.21.15.40:0/204372916 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc005900 con 0x5648c3cc3420 2023-10-18T08:58:41.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.204+0000 7f29b37fe640 1 -- 172.21.15.40:0/204372916 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29cc003970 con 0x5648c3cc3420 2023-10-18T08:58:41.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.204+0000 7f29dd658740 1 -- 172.21.15.40:0/204372916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29c0045070 con 0x5648c3cc3420 2023-10-18T08:58:41.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.204+0000 7f29b37fe640 1 -- 172.21.15.40:0/204372916 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29cc010ea0 con 0x5648c3cc3420 2023-10-18T08:58:41.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.204+0000 7f29dd658740 10 client.6324 fetch_fsmap learned FSMap version 33 2023-10-18T08:58:41.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.204+0000 7f29dd658740 1 -- 172.21.15.40:0/204372916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5648c3bc2170 con 0x5648c3cc3420 2023-10-18T08:58:41.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.204+0000 7f29b37fe640 1 -- 172.21.15.40:0/204372916 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29c0045070 con 0x5648c3cc3420 2023-10-18T08:58:41.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.204+0000 7f29b37fe640 1 -- 172.21.15.40:0/204372916 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29cc003b30 con 0x5648c3cc3420 2023-10-18T08:58:41.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.204+0000 7f29dd658740 10 client.6324 fetch_fsmap finished waiting for FSMap version 33 2023-10-18T08:58:41.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.204+0000 7f29dd658740 10 client.6324 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-18T08:58:41.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.204+0000 7f29dd658740 20 client.6324 populate_metadata read hostname 'smithi040' 2023-10-18T08:58:41.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.204+0000 7f29dd658740 1 --2- 172.21.15.40:0/204372916 >> [v2:172.21.15.144:6836/1311290970,v1:172.21.15.144:6837/1311290970] conn(0x5648c3c97560 0x5648c3c97940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.204+0000 7f29dd658740 4 client.6324 mds_command: new command op to 5389 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T08:58:41.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.204+0000 7f29dd658740 1 -- 172.21.15.40:0/204372916 --> [v2:172.21.15.144:6836/1311290970,v1:172.21.15.144:6837/1311290970] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5648c39016c0 con 0x5648c3c97560 2023-10-18T08:58:41.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.205+0000 7f29d8865640 1 --2- 172.21.15.40:0/204372916 >> [v2:172.21.15.144:6836/1311290970,v1:172.21.15.144:6837/1311290970] conn(0x5648c3c97560 0x5648c3c97940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.206+0000 7f29d8865640 1 --2- 172.21.15.40:0/204372916 >> [v2:172.21.15.144:6836/1311290970,v1:172.21.15.144:6837/1311290970] conn(0x5648c3c97560 0x5648c3c97940 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=434a033dd30eb3e5 server_cookie=be6153c7d6046049 in_seq=0 out_seq=0 2023-10-18T08:58:41.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.206+0000 7f29b37fe640 10 client.6324 ms_handle_connect on v2:172.21.15.144:6836/1311290970 2023-10-18T08:58:41.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.207+0000 7f29b37fe640 1 -- 172.21.15.40:0/204372916 <== mds.0 v2:172.21.15.144:6836/1311290970 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5648c39016c0 con 0x5648c3c97560 2023-10-18T08:58:41.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.207+0000 7f29b37fe640 10 client.6324 handle_command_reply: tid=0 2023-10-18T08:58:41.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.207+0000 7f29dd658740 1 client.6324 shutdown 2023-10-18T08:58:41.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.207+0000 7f29dd658740 1 -- 172.21.15.40:0/204372916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc3420 msgr2=0x5648c3d485f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:41.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.207+0000 7f29dd658740 1 --2- 172.21.15.40:0/204372916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc3420 0x5648c3d485f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x5648c3cc4d20 tx=0x7f29cc010e30 comp rx=0 tx=0).stop 2023-10-18T08:58:41.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.207+0000 7f29dd658740 1 -- 172.21.15.40:0/204372916 shutdown_connections 2023-10-18T08:58:41.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.207+0000 7f29dd658740 1 --2- 172.21.15.40:0/204372916 >> [v2:172.21.15.144:6836/1311290970,v1:172.21.15.144:6837/1311290970] conn(0x5648c3c97560 0x5648c3c97940 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.207+0000 7f29dd658740 1 --2- 172.21.15.40:0/204372916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648c3cc3420 0x5648c3d485f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.207+0000 7f29dd658740 1 --2- 172.21.15.40:0/204372916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648c3cc0d50 0x5648c3d480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.207+0000 7f29dd658740 1 --2- 172.21.15.40:0/204372916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648c3cc0970 0x5648c3cde130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.207+0000 7f29b37fe640 0 client.6324 ms_handle_reset on v2:172.21.15.144:6836/1311290970 2023-10-18T08:58:41.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.207+0000 7f29dd658740 1 -- 172.21.15.40:0/204372916 >> 172.21.15.40:0/204372916 conn(0x5648c3c940e0 msgr2=0x5648c3c8f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:41.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.207+0000 7f29dd658740 1 -- 172.21.15.40:0/204372916 shutdown_connections 2023-10-18T08:58:41.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.207+0000 7f29dd658740 1 -- 172.21.15.40:0/204372916 wait complete. 2023-10-18T08:58:41.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.208+0000 7f29dd658740 20 client.6324 trim_cache size 0 max 16384 2023-10-18T08:58:41.220 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T08:58:41.220 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T08:58:41.220 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-10-18T08:58:41.220 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 5389, 2023-10-18T08:58:41.221 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-10-18T08:58:41.221 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-10-18T08:58:41.221 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-10-18T08:58:41.221 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 11.879713884999999, 2023-10-18T08:58:41.221 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 33, 2023-10-18T08:58:41.222 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 67, 2023-10-18T08:58:41.222 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 67, 2023-10-18T08:58:41.222 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 17.685032404000001 2023-10-18T08:58:41.222 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T08:58:41.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.208+0000 7f29d9867640 1 -- 172.21.15.40:0/3188798767 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29b806ae40 msgr2=0x7f29b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:41.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.208+0000 7f29d9867640 1 --2- 172.21.15.40:0/3188798767 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29b806ae40 0x7f29b806d300 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f29c0035000 tx=0x7f29c0004730 comp rx=0 tx=0).stop 2023-10-18T08:58:41.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.208+0000 7f29d9867640 1 -- 172.21.15.40:0/3188798767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d40c9f90 msgr2=0x7f29d415b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:41.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.208+0000 7f29d9867640 1 --2- 172.21.15.40:0/3188798767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d40c9f90 0x7f29d415b990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f29c800b4a0 tx=0x7f29c800b960 comp rx=0 tx=0).stop 2023-10-18T08:58:41.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.208+0000 7f29d9867640 1 -- 172.21.15.40:0/3188798767 shutdown_connections 2023-10-18T08:58:41.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.208+0000 7f29d9867640 1 --2- 172.21.15.40:0/3188798767 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29b806ae40 0x7f29b806d300 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.208+0000 7f29d9867640 1 --2- 172.21.15.40:0/3188798767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29d41425a0 0x7f29d4140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.208+0000 7f29d9867640 1 --2- 172.21.15.40:0/3188798767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29d41420f0 0x7f29d4158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.208+0000 7f29d9867640 1 --2- 172.21.15.40:0/3188798767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d40c9f90 0x7f29d415b990 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.208+0000 7f29d9867640 1 -- 172.21.15.40:0/3188798767 >> 172.21.15.40:0/3188798767 conn(0x7f29d4069720 msgr2=0x7f29d40ad550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:41.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.208+0000 7f29d9867640 1 -- 172.21.15.40:0/3188798767 shutdown_connections 2023-10-18T08:58:41.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.208+0000 7f29d9867640 1 -- 172.21.15.40:0/3188798767 wait complete. 2023-10-18T08:58:41.230 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T08:58:41.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.455+0000 7f775b59e640 1 -- 172.21.15.40:0/283945264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f775c06bbe0 msgr2=0x7f775c15cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:41.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.455+0000 7f775b59e640 1 --2- 172.21.15.40:0/283945264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f775c06bbe0 0x7f775c15cee0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7754005fd0 tx=0x7f775402c2e0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.455+0000 7f775b59e640 1 -- 172.21.15.40:0/283945264 shutdown_connections 2023-10-18T08:58:41.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.455+0000 7f775b59e640 1 --2- 172.21.15.40:0/283945264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f775c06bbe0 0x7f775c15cee0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.455+0000 7f775b59e640 1 -- 172.21.15.40:0/283945264 >> 172.21.15.40:0/283945264 conn(0x7f775c0aed70 msgr2=0x7f775c0af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:41.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.456+0000 7f775b59e640 1 -- 172.21.15.40:0/283945264 shutdown_connections 2023-10-18T08:58:41.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.456+0000 7f775b59e640 1 -- 172.21.15.40:0/283945264 wait complete. 2023-10-18T08:58:41.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.457+0000 7f775b59e640 1 Processor -- start 2023-10-18T08:58:41.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.457+0000 7f775b59e640 1 -- start start 2023-10-18T08:58:41.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.458+0000 7f775b59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f775c06bbe0 0x7f775c1f1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.458+0000 7f775b59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f775c0aa8f0 0x7f775c1f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.458+0000 7f775b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f775c0ac070 0x7f775c13e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.458+0000 7f775b59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c141e90 con 0x7f775c0ac070 2023-10-18T08:58:41.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.458+0000 7f775b59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c13c300 con 0x7f775c06bbe0 2023-10-18T08:58:41.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.458+0000 7f775b59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775c13c5e0 con 0x7f775c0aa8f0 2023-10-18T08:58:41.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.459+0000 7f775ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f775c0ac070 0x7f775c13e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.459+0000 7f775ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f775c0ac070 0x7f775c13e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43496/0 (socket says 172.21.15.40:43496) 2023-10-18T08:58:41.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.459+0000 7f775ad9d640 1 -- 172.21.15.40:0/2388000225 learned_addr learned my addr 172.21.15.40:0/2388000225 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:41.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.459+0000 7f7753fff640 1 --2- 172.21.15.40:0/2388000225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f775c0aa8f0 0x7f775c1f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.459+0000 7f775a59c640 1 --2- 172.21.15.40:0/2388000225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f775c06bbe0 0x7f775c1f1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.459+0000 7f775ad9d640 1 -- 172.21.15.40:0/2388000225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f775c0aa8f0 msgr2=0x7f775c1f2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:41.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.459+0000 7f775ad9d640 1 --2- 172.21.15.40:0/2388000225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f775c0aa8f0 0x7f775c1f2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.459+0000 7f775ad9d640 1 -- 172.21.15.40:0/2388000225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f775c06bbe0 msgr2=0x7f775c1f1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.459+0000 7f775ad9d640 1 --2- 172.21.15.40:0/2388000225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f775c06bbe0 0x7f775c1f1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.459+0000 7f775ad9d640 1 -- 172.21.15.40:0/2388000225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7754009900 con 0x7f775c0ac070 2023-10-18T08:58:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.459+0000 7f7753fff640 1 --2- 172.21.15.40:0/2388000225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f775c0aa8f0 0x7f775c1f2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.459+0000 7f775a59c640 1 --2- 172.21.15.40:0/2388000225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f775c06bbe0 0x7f775c1f1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:58:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.460+0000 7f775ad9d640 1 --2- 172.21.15.40:0/2388000225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f775c0ac070 0x7f775c13e610 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7754037040 tx=0x7f7754035ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:41.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.460+0000 7f7752ffd640 1 -- 172.21.15.40:0/2388000225 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754004240 con 0x7f775c0ac070 2023-10-18T08:58:41.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.460+0000 7f7752ffd640 1 -- 172.21.15.40:0/2388000225 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7754004490 con 0x7f775c0ac070 2023-10-18T08:58:41.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.460+0000 7f775b59e640 1 -- 172.21.15.40:0/2388000225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775c13c8c0 con 0x7f775c0ac070 2023-10-18T08:58:41.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.460+0000 7f7752ffd640 1 -- 172.21.15.40:0/2388000225 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754004840 con 0x7f775c0ac070 2023-10-18T08:58:41.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.460+0000 7f775b59e640 1 -- 172.21.15.40:0/2388000225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775c1f19c0 con 0x7f775c0ac070 2023-10-18T08:58:41.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.461+0000 7f775b59e640 1 -- 172.21.15.40:0/2388000225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775c066e00 con 0x7f775c0ac070 2023-10-18T08:58:41.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.462+0000 7f7752ffd640 1 -- 172.21.15.40:0/2388000225 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f775403b040 con 0x7f775c0ac070 2023-10-18T08:58:41.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.463+0000 7f7752ffd640 1 --2- 172.21.15.40:0/2388000225 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f773c06ae40 0x7f773c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:41.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.463+0000 7f7752ffd640 1 -- 172.21.15.40:0/2388000225 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77540ac210 con 0x7f775c0ac070 2023-10-18T08:58:41.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.464+0000 7f775a59c640 1 --2- 172.21.15.40:0/2388000225 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f773c06ae40 0x7f773c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:41.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.465+0000 7f775a59c640 1 --2- 172.21.15.40:0/2388000225 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f773c06ae40 0x7f773c06d300 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f775c13e3b0 tx=0x7f774c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:41.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.468+0000 7f7752ffd640 1 -- 172.21.15.40:0/2388000225 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7754077370 con 0x7f775c0ac070 2023-10-18T08:58:41.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.702+0000 7f775b59e640 1 -- 172.21.15.40:0/2388000225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f775c06c0d0 con 0x7f775c0ac070 2023-10-18T08:58:41.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.703+0000 7f7752ffd640 1 -- 172.21.15.40:0/2388000225 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7f775407a990 con 0x7f775c0ac070 2023-10-18T08:58:41.706 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T08:58:41.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.706+0000 7f775b59e640 1 -- 172.21.15.40:0/2388000225 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f773c06ae40 msgr2=0x7f773c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:41.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.706+0000 7f775b59e640 1 --2- 172.21.15.40:0/2388000225 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f773c06ae40 0x7f773c06d300 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f775c13e3b0 tx=0x7f774c008040 comp rx=0 tx=0).stop 2023-10-18T08:58:41.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.706+0000 7f775b59e640 1 -- 172.21.15.40:0/2388000225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f775c0ac070 msgr2=0x7f775c13e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:41.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.706+0000 7f775b59e640 1 --2- 172.21.15.40:0/2388000225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f775c0ac070 0x7f775c13e610 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7754037040 tx=0x7f7754035ad0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.706+0000 7f775b59e640 1 -- 172.21.15.40:0/2388000225 shutdown_connections 2023-10-18T08:58:41.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.706+0000 7f775b59e640 1 --2- 172.21.15.40:0/2388000225 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f773c06ae40 0x7f773c06d300 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.706+0000 7f775b59e640 1 --2- 172.21.15.40:0/2388000225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f775c0ac070 0x7f775c13e610 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.706+0000 7f775b59e640 1 --2- 172.21.15.40:0/2388000225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f775c0aa8f0 0x7f775c1f2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.706+0000 7f775b59e640 1 --2- 172.21.15.40:0/2388000225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f775c06bbe0 0x7f775c1f1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:41.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.706+0000 7f775b59e640 1 -- 172.21.15.40:0/2388000225 >> 172.21.15.40:0/2388000225 conn(0x7f775c0aed70 msgr2=0x7f775c0cc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:41.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.706+0000 7f775b59e640 1 -- 172.21.15.40:0/2388000225 shutdown_connections 2023-10-18T08:58:41.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:41.706+0000 7f775b59e640 1 -- 172.21.15.40:0/2388000225 wait complete. 2023-10-18T08:58:41.729 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T08:58:41.729 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T08:58:41.729 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-10-18T08:58:41.730 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T08:58:41.730 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T08:58:41.730 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T08:58:41.731 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T08:58:41.731 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T08:58:41.731 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T08:58:41.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:41.732 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-10-18T08:58:41.786 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T08:58:41.786 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T08:58:41.786 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T08:58:41.787 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:58:41.787 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-18T08:58:41.787 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:58:41.787 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:58:41.787 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:58:41.788 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-10-18T08:58:41.788 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:58:41.788 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:58:41.788 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-10-18T08:58:41.788 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T08:58:41.789 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-10-18T08:58:41.789 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-10-18T08:58:41.789 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-18T08:58:41.789 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42203sec preferred_lft 42203sec 2023-10-18T08:58:41.789 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2023-10-18T08:58:41.790 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:58:41.790 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:58:41.790 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-10-18T08:58:41.790 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-10-18T08:58:41.790 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T08:58:41.791 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:58:41.791 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T08:58:41.791 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:58:41.791 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2026:95ff:fea9:a463/64 scope link 2023-10-18T08:58:41.791 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:58:41.792 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T08:58:41.792 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T08:58:41.792 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::80bd:e5ff:fe0f:6f68/64 scope link 2023-10-18T08:58:41.792 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:58:41.792 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T08:58:41.792 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether ea:2b:21:f5:06:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T08:58:41.793 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::e82b:21ff:fef5:6af/64 scope link 2023-10-18T08:58:41.793 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:58:41.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:41.794 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T08:58:41.887 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-10-18T08:58:41.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:41.892 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-10-18T08:58:41.949 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-10-18T08:58:41.949 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T08:58:41.950 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-18T08:58:41.950 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-18T08:58:41.950 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T08:58:41.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:41.952 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T08:58:41.952 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T08:58:41.952 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-18T08:58:41.953 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-18T08:58:41.953 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-18T08:58:41.953 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T08:58:42.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:42.155 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-18T08:58:42.209 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T08:58:42.210 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T08:58:42.211 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T08:58:42.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:42.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-10-18T08:58:42.312 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:58:42.313 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T08:58:42.374 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T08:58:42.374 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:58:42.375 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T08:58:42.431 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-18T08:58:42.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:42.432 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:58:42.587 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:58:42.587 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:58:42.622 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:58:42.622 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:58:42.623 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T08:58:42.623 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T08:58:42.623 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T08:58:42.624 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-18T08:58:42.624 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2023-10-18T08:58:42.625 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T08:58:42.625 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T08:58:42.625 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T08:58:42.628 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T08:58:42.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:42.629 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T08:58:42.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:42.733 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:58:42.820 INFO:teuthology.orchestra.run.smithi040.stdout:client5138 2023-10-18T08:58:42.821 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:58:42.821 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:58:42.885 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T08:58:43.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.124+0000 7f0ab7fff640 1 -- 172.21.15.40:0/853511286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa4003970 msgr2=0x7f0aa4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:43.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.124+0000 7f0ab7fff640 1 --2- 172.21.15.40:0/853511286 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa4003970 0x7f0aa4003d90 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009ed0 tx=0x7f0aa40057f0 comp rx=0 tx=0).stop 2023-10-18T08:58:43.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.125+0000 7f0ab77fe640 1 -- 172.21.15.40:0/853511286 reap_dead start 2023-10-18T08:58:43.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.125+0000 7f0ab7fff640 1 -- 172.21.15.40:0/853511286 shutdown_connections 2023-10-18T08:58:43.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.125+0000 7f0ab7fff640 1 -- 172.21.15.40:0/853511286 >> 172.21.15.40:0/853511286 conn(0x7f0ab808ac80 msgr2=0x7f0ab809a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:43.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.125+0000 7f0ab7fff640 1 -- 172.21.15.40:0/853511286 shutdown_connections 2023-10-18T08:58:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.125+0000 7f0ab7fff640 1 -- 172.21.15.40:0/853511286 wait complete. 2023-10-18T08:58:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.126+0000 7f0ab7fff640 1 Processor -- start 2023-10-18T08:58:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.127+0000 7f0ab7fff640 1 -- start start 2023-10-18T08:58:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.128+0000 7f0ab7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa4003970 0x7f0ab80b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.128+0000 7f0ab7fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ab80c68a0 0x7f0ab80c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:43.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.128+0000 7f0ab7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ab80c7330 0x7f0ab80c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:43.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.128+0000 7f0ab7fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80c77e0 con 0x7f0aa4003970 2023-10-18T08:58:43.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.128+0000 7f0ab7fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80c7d70 con 0x7f0ab80c7330 2023-10-18T08:58:43.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.128+0000 7f0ab7fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab81e67a0 con 0x7f0ab80c68a0 2023-10-18T08:58:43.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.128+0000 7f0ab6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa4003970 0x7f0ab80b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:43.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.128+0000 7f0ab6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa4003970 0x7f0ab80b6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43510/0 (socket says 172.21.15.40:43510) 2023-10-18T08:58:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.128+0000 7f0ab6ffd640 1 -- 172.21.15.40:0/2026083410 learned_addr learned my addr 172.21.15.40:0/2026083410 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.128+0000 7f0ab6ffd640 1 -- 172.21.15.40:0/2026083410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ab80c68a0 msgr2=0x7f0ab80c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.128+0000 7f0ab6ffd640 1 --2- 172.21.15.40:0/2026083410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ab80c68a0 0x7f0ab80c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.128+0000 7f0ab6ffd640 1 -- 172.21.15.40:0/2026083410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ab80c7330 msgr2=0x7f0ab80c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:43.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.128+0000 7f0ab6ffd640 1 --2- 172.21.15.40:0/2026083410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ab80c7330 0x7f0ab80c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.128+0000 7f0ab6ffd640 1 -- 172.21.15.40:0/2026083410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab81e6a60 con 0x7f0aa4003970 2023-10-18T08:58:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.129+0000 7f0ab6ffd640 1 --2- 172.21.15.40:0/2026083410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa4003970 0x7f0ab80b6a80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009e20 tx=0x7f0aa8031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.129+0000 7f0aaffff640 1 -- 172.21.15.40:0/2026083410 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa803f070 con 0x7f0aa4003970 2023-10-18T08:58:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.129+0000 7f0aaffff640 1 -- 172.21.15.40:0/2026083410 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0aa803a7a0 con 0x7f0aa4003970 2023-10-18T08:58:43.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.129+0000 7f0aaffff640 1 -- 172.21.15.40:0/2026083410 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa803abb0 con 0x7f0aa4003970 2023-10-18T08:58:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.129+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2026083410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab80b5da0 con 0x7f0aa4003970 2023-10-18T08:58:43.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.129+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2026083410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab80c65e0 con 0x7f0aa4003970 2023-10-18T08:58:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.130+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2026083410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab8071da0 con 0x7f0aa4003970 2023-10-18T08:58:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.131+0000 7f0aaffff640 1 -- 172.21.15.40:0/2026083410 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0aa803ad10 con 0x7f0aa4003970 2023-10-18T08:58:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.132+0000 7f0aaffff640 1 --2- 172.21.15.40:0/2026083410 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a9806ae70 0x7f0a9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:43.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.132+0000 7f0aaffff640 1 -- 172.21.15.40:0/2026083410 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0aa80abfe0 con 0x7f0aa4003970 2023-10-18T08:58:43.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.133+0000 7f0ab67fc640 1 --2- 172.21.15.40:0/2026083410 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a9806ae70 0x7f0a9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:43.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.134+0000 7f0ab67fc640 1 --2- 172.21.15.40:0/2026083410 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a9806ae70 0x7f0a9806d330 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00017a0 tx=0x7f0aa0000f70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:43.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.137+0000 7f0aaffff640 1 -- 172.21.15.40:0/2026083410 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0aa8076ff0 con 0x7f0aa4003970 2023-10-18T08:58:43.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.373+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2026083410 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0ab80c7780 con 0x7f0aa4003970 2023-10-18T08:58:43.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.374+0000 7f0aaffff640 1 -- 172.21.15.40:0/2026083410 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0ab80c7780 con 0x7f0aa4003970 2023-10-18T08:58:43.376 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T08:58:43.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.377+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2026083410 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a9806ae70 msgr2=0x7f0a9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:43.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.377+0000 7f0ab7fff640 1 --2- 172.21.15.40:0/2026083410 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a9806ae70 0x7f0a9806d330 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00017a0 tx=0x7f0aa0000f70 comp rx=0 tx=0).stop 2023-10-18T08:58:43.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.377+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2026083410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa4003970 msgr2=0x7f0ab80b6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:43.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.377+0000 7f0ab7fff640 1 --2- 172.21.15.40:0/2026083410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa4003970 0x7f0ab80b6a80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009e20 tx=0x7f0aa8031750 comp rx=0 tx=0).stop 2023-10-18T08:58:43.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.377+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2026083410 shutdown_connections 2023-10-18T08:58:43.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.377+0000 7f0ab7fff640 1 --2- 172.21.15.40:0/2026083410 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a9806ae70 0x7f0a9806d330 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:43.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.377+0000 7f0ab7fff640 1 --2- 172.21.15.40:0/2026083410 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ab80c7330 0x7f0ab80c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:43.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.377+0000 7f0ab7fff640 1 --2- 172.21.15.40:0/2026083410 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ab80c68a0 0x7f0ab80c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:43.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.377+0000 7f0ab7fff640 1 --2- 172.21.15.40:0/2026083410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0aa4003970 0x7f0ab80b6a80 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:43.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.377+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2026083410 >> 172.21.15.40:0/2026083410 conn(0x7f0ab808ac80 msgr2=0x7f0ab80cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:43.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.377+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2026083410 shutdown_connections 2023-10-18T08:58:43.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:43.377+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2026083410 wait complete. 2023-10-18T08:58:43.398 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T08:58:43.398 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T08:58:43.398 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-10-18T08:58:43.399 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-18T08:58:43.399 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T08:58:43.399 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T08:58:43.400 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T08:58:43.400 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T08:58:43.400 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T08:58:43.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:43.401 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-10-18T08:58:43.460 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T08:58:43.460 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T08:58:43.460 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T08:58:43.461 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:58:43.461 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-18T08:58:43.461 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:58:43.461 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:58:43.462 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:58:43.462 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2023-10-18T08:58:43.462 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:58:43.462 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-10-18T08:58:43.462 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2023-10-18T08:58:43.463 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T08:58:43.463 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-10-18T08:58:43.463 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T08:58:43.463 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-10-18T08:58:43.463 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T08:58:43.464 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42194sec preferred_lft 42194sec 2023-10-18T08:58:43.464 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2023-10-18T08:58:43.464 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:58:43.464 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T08:58:43.464 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff 2023-10-18T08:58:43.464 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T08:58:43.465 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:58:43.465 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1493:a0ff:fed2:2c1/64 scope link 2023-10-18T08:58:43.465 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:58:43.465 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T08:58:43.466 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 4a:6e:a9:f0:92:62 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T08:58:43.466 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::486e:a9ff:fef0:9262/64 scope link 2023-10-18T08:58:43.466 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:58:43.466 INFO:teuthology.orchestra.run.smithi144.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T08:58:43.467 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T08:58:43.467 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::345a:c5ff:fe54:de6e/64 scope link 2023-10-18T08:58:43.467 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T08:58:43.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:43.468 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T08:58:43.562 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-10-18T08:58:43.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:43.567 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-10-18T08:58:43.619 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-10-18T08:58:43.619 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T08:58:43.620 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T08:58:43.620 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T08:58:43.620 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T08:58:43.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:43.621 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T08:58:43.621 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T08:58:43.622 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T08:58:43.622 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T08:58:43.623 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T08:58:43.623 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T08:58:43.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:43.806 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-18T08:58:43.860 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T08:58:43.860 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-18T08:58:43.860 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T08:58:43.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:43.861 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2023-10-18T08:58:43.957 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:58:43.958 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-18T08:58:44.014 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-18T08:58:44.015 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T08:58:44.015 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-18T08:58:44.066 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-18T08:58:44.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:44.067 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:58:44.206 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:58:44.206 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:58:44.238 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T08:58:44.238 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T08:58:44.238 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T08:58:44.238 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with new device syntax: 1@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T08:58:44.239 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T08:58:44.241 INFO:teuthology.orchestra.run.smithi144.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-18T08:58:44.241 INFO:teuthology.orchestra.run.smithi144.stdout: You just mounted a file system that supports labels which does not 2023-10-18T08:58:44.241 INFO:teuthology.orchestra.run.smithi144.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T08:58:44.242 INFO:teuthology.orchestra.run.smithi144.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T08:58:44.242 INFO:teuthology.orchestra.run.smithi144.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T08:58:44.245 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T08:58:44.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:44.246 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-18T08:58:44.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:58:44.331 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T08:58:44.386 INFO:teuthology.orchestra.run.smithi144.stdout:client6342 2023-10-18T08:58:44.387 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:58:44.387 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:58:44.436 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:58:44.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.679+0000 7f1763fff640 1 -- 172.21.15.40:0/1644177149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17640ca160 msgr2=0x7f17640cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:44.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.679+0000 7f1763fff640 1 --2- 172.21.15.40:0/1644177149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17640ca160 0x7f17640cafc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f175c005fd0 tx=0x7f175c02c0e0 comp rx=0 tx=0).stop 2023-10-18T08:58:44.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.679+0000 7f1763fff640 1 -- 172.21.15.40:0/1644177149 shutdown_connections 2023-10-18T08:58:44.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.679+0000 7f1763fff640 1 --2- 172.21.15.40:0/1644177149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17640ca160 0x7f17640cafc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:44.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.679+0000 7f1763fff640 1 --2- 172.21.15.40:0/1644177149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17641420f0 0x7f17640b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.679+0000 7f1763fff640 1 -- 172.21.15.40:0/1644177149 >> 172.21.15.40:0/1644177149 conn(0x7f1764069720 msgr2=0x7f1764069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.679+0000 7f1763fff640 1 -- 172.21.15.40:0/1644177149 shutdown_connections 2023-10-18T08:58:44.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.679+0000 7f1763fff640 1 -- 172.21.15.40:0/1644177149 wait complete. 2023-10-18T08:58:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.681+0000 7f1763fff640 1 Processor -- start 2023-10-18T08:58:44.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.682+0000 7f1763fff640 1 -- start start 2023-10-18T08:58:44.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.683+0000 7f1763fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17640ca160 0x7f1764152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.683+0000 7f1763fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17641420f0 0x7f176413a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.683+0000 7f1763fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17641425a0 0x7f1764154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.683+0000 7f1763fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1764070b10 con 0x7f17640ca160 2023-10-18T08:58:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.683+0000 7f1763fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176406fc60 con 0x7f17641420f0 2023-10-18T08:58:44.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.683+0000 7f1763fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1764154bd0 con 0x7f17641425a0 2023-10-18T08:58:44.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.683+0000 7f1762ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17640ca160 0x7f1764152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:44.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.683+0000 7f1762ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17640ca160 0x7f1764152e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43522/0 (socket says 172.21.15.40:43522) 2023-10-18T08:58:44.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.683+0000 7f1762ffd640 1 -- 172.21.15.40:0/1068239907 learned_addr learned my addr 172.21.15.40:0/1068239907 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:58:44.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.684+0000 7f17637fe640 1 --2- 172.21.15.40:0/1068239907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17641425a0 0x7f1764154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.684+0000 7f175bfff640 1 --2- 172.21.15.40:0/1068239907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17641420f0 0x7f176413a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.684+0000 7f1762ffd640 1 -- 172.21.15.40:0/1068239907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17641425a0 msgr2=0x7f1764154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.684+0000 7f1762ffd640 1 --2- 172.21.15.40:0/1068239907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17641425a0 0x7f1764154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.684+0000 7f1762ffd640 1 -- 172.21.15.40:0/1068239907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17641420f0 msgr2=0x7f176413a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:44.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.684+0000 7f1762ffd640 1 --2- 172.21.15.40:0/1068239907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17641420f0 0x7f176413a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.684+0000 7f1762ffd640 1 -- 172.21.15.40:0/1068239907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c009840 con 0x7f17640ca160 2023-10-18T08:58:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.684+0000 7f1762ffd640 1 --2- 172.21.15.40:0/1068239907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17640ca160 0x7f1764152e50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f175400e9f0 tx=0x7f175400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.685+0000 7f175b7fe640 1 -- 172.21.15.40:0/1068239907 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754009700 con 0x7f17640ca160 2023-10-18T08:58:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.685+0000 7f175b7fe640 1 -- 172.21.15.40:0/1068239907 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f175400ce70 con 0x7f17640ca160 2023-10-18T08:58:44.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.685+0000 7f1763fff640 1 -- 172.21.15.40:0/1068239907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176406ff00 con 0x7f17640ca160 2023-10-18T08:58:44.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.685+0000 7f175b7fe640 1 -- 172.21.15.40:0/1068239907 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754010670 con 0x7f17640ca160 2023-10-18T08:58:44.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.685+0000 7f1763fff640 1 -- 172.21.15.40:0/1068239907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17640704c0 con 0x7f17640ca160 2023-10-18T08:58:44.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.686+0000 7f1763fff640 1 -- 172.21.15.40:0/1068239907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1764066e00 con 0x7f17640ca160 2023-10-18T08:58:44.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.687+0000 7f175b7fe640 1 -- 172.21.15.40:0/1068239907 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f17540107d0 con 0x7f17640ca160 2023-10-18T08:58:44.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.688+0000 7f175b7fe640 1 --2- 172.21.15.40:0/1068239907 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f174406ae40 0x7f174406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:58:44.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.688+0000 7f175b7fe640 1 -- 172.21.15.40:0/1068239907 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1754014070 con 0x7f17640ca160 2023-10-18T08:58:44.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.689+0000 7f175bfff640 1 --2- 172.21.15.40:0/1068239907 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f174406ae40 0x7f174406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:58:44.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.690+0000 7f175bfff640 1 --2- 172.21.15.40:0/1068239907 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f174406ae40 0x7f174406d300 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1764151ac0 tx=0x7f174c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:58:44.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.692+0000 7f175b7fe640 1 -- 172.21.15.40:0/1068239907 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1754051ca0 con 0x7f17640ca160 2023-10-18T08:58:44.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.920+0000 7f1763fff640 1 -- 172.21.15.40:0/1068239907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f176406d920 con 0x7f17640ca160 2023-10-18T08:58:44.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.921+0000 7f175b7fe640 1 -- 172.21.15.40:0/1068239907 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f1754058e60 con 0x7f17640ca160 2023-10-18T08:58:44.924 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:58:44.924 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5075,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2165596806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2165596806},{"type":"v1","addr":"172.21.15.144:6835","nonce":2165596806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/587522883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":587522883},{"type":"v1","addr":"172.21.15.40:6833","nonce":587522883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1174134524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1174134524},{"type":"v1","addr":"172.21.15.40:6835","nonce":1174134524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:58:28.760677+0000","modified":"2023-10-18T08:58:31.845684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5389":{"gid":5389,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1311290970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1311290970},{"type":"v1","addr":"172.21.15.144:6837","nonce":1311290970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-18T08:58:44.924 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 33 2023-10-18T08:58:44.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.924+0000 7f1763fff640 1 -- 172.21.15.40:0/1068239907 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f174406ae40 msgr2=0x7f174406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:44.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.924+0000 7f1763fff640 1 --2- 172.21.15.40:0/1068239907 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f174406ae40 0x7f174406d300 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1764151ac0 tx=0x7f174c009040 comp rx=0 tx=0).stop 2023-10-18T08:58:44.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.924+0000 7f1763fff640 1 -- 172.21.15.40:0/1068239907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17640ca160 msgr2=0x7f1764152e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:58:44.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.924+0000 7f1763fff640 1 --2- 172.21.15.40:0/1068239907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17640ca160 0x7f1764152e50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f175400e9f0 tx=0x7f175400eeb0 comp rx=0 tx=0).stop 2023-10-18T08:58:44.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.924+0000 7f1763fff640 1 -- 172.21.15.40:0/1068239907 shutdown_connections 2023-10-18T08:58:44.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.924+0000 7f1763fff640 1 --2- 172.21.15.40:0/1068239907 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f174406ae40 0x7f174406d300 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:44.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.924+0000 7f1763fff640 1 --2- 172.21.15.40:0/1068239907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17641425a0 0x7f1764154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:44.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.924+0000 7f1763fff640 1 --2- 172.21.15.40:0/1068239907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17641420f0 0x7f176413a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:44.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.924+0000 7f1763fff640 1 --2- 172.21.15.40:0/1068239907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17640ca160 0x7f1764152e50 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:58:44.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.924+0000 7f1763fff640 1 -- 172.21.15.40:0/1068239907 >> 172.21.15.40:0/1068239907 conn(0x7f1764069720 msgr2=0x7f17640c9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:58:44.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.924+0000 7f1763fff640 1 -- 172.21.15.40:0/1068239907 shutdown_connections 2023-10-18T08:58:44.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:58:44.924+0000 7f1763fff640 1 -- 172.21.15.40:0/1068239907 wait complete. 2023-10-18T08:58:44.946 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-10-18T08:58:45.185 INFO:teuthology.orchestra.run.smithi144.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":5389,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.842504290999999,"mdsmap_epoch":33,"osdmap_epoch":67,"osdmap_epoch_barrier":67,"uptime":21.647816407000001} 2023-10-18T08:58:45.186 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 5389, "mdsmap_epoch": 33, "osdmap_epoch": 67, "osdmap_epoch_barrier": 67, "rank_uptime": 15.842504291, "state": "up:active", "uptime": 21.647816407, "want_state": "up:active", "whoami": 0 } 2023-10-18T08:58:45.187 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-18T08:58:45.443 INFO:teuthology.orchestra.run.smithi040.stdout:{"osd_mon_report_interval":"5"} 2023-10-18T08:58:45.443 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-18T08:58:45.444 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-18T08:58:45.674 INFO:teuthology.orchestra.run.smithi040.stdout:{"mon_tick_interval":"5"} 2023-10-18T08:58:45.674 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-18T08:58:45.675 INFO:tasks.cephfs.test_full:pool capacity 557049408, 531MB should be enough to fill it 2023-10-18T08:58:45.675 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2023-10-18T08:58:45.675 DEBUG:teuthology.orchestra.run.smithi040:> import time 2023-10-18T08:58:45.676 DEBUG:teuthology.orchestra.run.smithi040:> import datetime 2023-10-18T08:58:45.676 DEBUG:teuthology.orchestra.run.smithi040:> import subprocess 2023-10-18T08:58:45.677 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-10-18T08:58:45.677 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T08:58:45.677 DEBUG:teuthology.orchestra.run.smithi040:> # Write some buffered data through before going full, all should be well 2023-10-18T08:58:45.678 DEBUG:teuthology.orchestra.run.smithi040:> print("writing some data through which we expect to succeed") 2023-10-18T08:58:45.678 DEBUG:teuthology.orchestra.run.smithi040:> bytes = 0 2023-10-18T08:58:45.678 DEBUG:teuthology.orchestra.run.smithi040:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-18T08:58:45.679 DEBUG:teuthology.orchestra.run.smithi040:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-18T08:58:45.679 DEBUG:teuthology.orchestra.run.smithi040:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2023-10-18T08:58:45.679 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(f) 2023-10-18T08:58:45.680 DEBUG:teuthology.orchestra.run.smithi040:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-18T08:58:45.680 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T08:58:45.680 DEBUG:teuthology.orchestra.run.smithi040:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-18T08:58:45.681 DEBUG:teuthology.orchestra.run.smithi040:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-18T08:58:45.681 DEBUG:teuthology.orchestra.run.smithi040:> # buffered IO, the error should always only appear from fsync and not 2023-10-18T08:58:45.681 DEBUG:teuthology.orchestra.run.smithi040:> # from write 2023-10-18T08:58:45.682 DEBUG:teuthology.orchestra.run.smithi040:> full = False 2023-10-18T08:58:45.682 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T08:58:45.682 DEBUG:teuthology.orchestra.run.smithi040:> for n in range(0, int(531 * 0.9)): 2023-10-18T08:58:45.683 DEBUG:teuthology.orchestra.run.smithi040:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-18T08:58:45.683 DEBUG:teuthology.orchestra.run.smithi040:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2023-10-18T08:58:45.683 DEBUG:teuthology.orchestra.run.smithi040:> print("done writing {0} bytes".format(bytes)) 2023-10-18T08:58:45.684 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T08:58:45.684 DEBUG:teuthology.orchestra.run.smithi040:> # OK, now we should sneak in under the full condition 2023-10-18T08:58:45.684 DEBUG:teuthology.orchestra.run.smithi040:> # due to the time it takes the OSDs to report to the 2023-10-18T08:58:45.685 DEBUG:teuthology.orchestra.run.smithi040:> # mons, and get a successful fsync on our full-making data 2023-10-18T08:58:45.685 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(f) 2023-10-18T08:58:45.685 DEBUG:teuthology.orchestra.run.smithi040:> print("successfully fsync'"'"'ed prior to getting full state reported") 2023-10-18T08:58:45.686 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T08:58:45.686 DEBUG:teuthology.orchestra.run.smithi040:> # buffered write, add more dirty data to the buffer 2023-10-18T08:58:45.686 DEBUG:teuthology.orchestra.run.smithi040:> print("starting buffered write") 2023-10-18T08:58:45.687 DEBUG:teuthology.orchestra.run.smithi040:> try: 2023-10-18T08:58:45.687 DEBUG:teuthology.orchestra.run.smithi040:> for n in range(0, int(531 * 0.2)): 2023-10-18T08:58:45.687 DEBUG:teuthology.orchestra.run.smithi040:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-18T08:58:45.688 DEBUG:teuthology.orchestra.run.smithi040:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-18T08:58:45.688 DEBUG:teuthology.orchestra.run.smithi040:> time.sleep(15.0) 2023-10-18T08:58:45.688 DEBUG:teuthology.orchestra.run.smithi040:> except OSError: 2023-10-18T08:58:45.689 DEBUG:teuthology.orchestra.run.smithi040:> pass; 2023-10-18T08:58:45.689 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T08:58:45.689 DEBUG:teuthology.orchestra.run.smithi040:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2023-10-18T08:58:45.690 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T08:58:45.690 DEBUG:teuthology.orchestra.run.smithi040:> # Wait long enough for a background flush that should fail 2023-10-18T08:58:45.691 DEBUG:teuthology.orchestra.run.smithi040:> time.sleep(30) 2023-10-18T08:58:45.691 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T08:58:45.691 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2023-10-18T08:58:45.692 DEBUG:teuthology.orchestra.run.smithi040:> # ...and check that the failed background flush is reflected in fclose 2023-10-18T08:58:45.692 DEBUG:teuthology.orchestra.run.smithi040:> try: 2023-10-18T08:58:45.692 DEBUG:teuthology.orchestra.run.smithi040:> os.close(f) 2023-10-18T08:58:45.693 DEBUG:teuthology.orchestra.run.smithi040:> except OSError: 2023-10-18T08:58:45.693 DEBUG:teuthology.orchestra.run.smithi040:> print("close() returned an error as expected") 2023-10-18T08:58:45.693 DEBUG:teuthology.orchestra.run.smithi040:> else: 2023-10-18T08:58:45.694 DEBUG:teuthology.orchestra.run.smithi040:> raise RuntimeError("close() failed to raise error") 2023-10-18T08:58:45.694 DEBUG:teuthology.orchestra.run.smithi040:> else: 2023-10-18T08:58:45.694 DEBUG:teuthology.orchestra.run.smithi040:> # The kernel cephfs client does not raise errors on fclose 2023-10-18T08:58:45.695 DEBUG:teuthology.orchestra.run.smithi040:> os.close(f) 2023-10-18T08:58:45.695 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T08:58:45.695 DEBUG:teuthology.orchestra.run.smithi040:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-18T08:58:45.695 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-10-18T08:58:48.781 INFO:teuthology.orchestra.run.smithi040.stdout:writing some data through which we expect to succeed 2023-10-18T08:58:48.781 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-18T08:58:48.782 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 1572864 bytes via buffered write, may repeat 2023-10-18T08:58:48.782 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 2621440 bytes via buffered write, may repeat 2023-10-18T08:58:48.782 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 3670016 bytes via buffered write, may repeat 2023-10-18T08:58:48.782 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 4718592 bytes via buffered write, may repeat 2023-10-18T08:58:48.782 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 5767168 bytes via buffered write, may repeat 2023-10-18T08:58:48.783 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 6815744 bytes via buffered write, may repeat 2023-10-18T08:58:48.783 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 7864320 bytes via buffered write, may repeat 2023-10-18T08:58:48.783 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 8912896 bytes via buffered write, may repeat 2023-10-18T08:58:48.783 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 9961472 bytes via buffered write, may repeat 2023-10-18T08:58:48.783 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 11010048 bytes via buffered write, may repeat 2023-10-18T08:58:48.784 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 12058624 bytes via buffered write, may repeat 2023-10-18T08:58:48.784 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 13107200 bytes via buffered write, may repeat 2023-10-18T08:58:48.784 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 14155776 bytes via buffered write, may repeat 2023-10-18T08:58:48.784 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 15204352 bytes via buffered write, may repeat 2023-10-18T08:58:48.784 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 16252928 bytes via buffered write, may repeat 2023-10-18T08:58:48.785 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 17301504 bytes via buffered write, may repeat 2023-10-18T08:58:48.785 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 18350080 bytes via buffered write, may repeat 2023-10-18T08:58:48.785 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 19398656 bytes via buffered write, may repeat 2023-10-18T08:58:48.785 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 20447232 bytes via buffered write, may repeat 2023-10-18T08:58:48.785 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 21495808 bytes via buffered write, may repeat 2023-10-18T08:58:48.785 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 22544384 bytes via buffered write, may repeat 2023-10-18T08:58:48.786 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 23592960 bytes via buffered write, may repeat 2023-10-18T08:58:48.786 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 24641536 bytes via buffered write, may repeat 2023-10-18T08:58:48.786 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 25690112 bytes via buffered write, may repeat 2023-10-18T08:58:48.786 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 26738688 bytes via buffered write, may repeat 2023-10-18T08:58:48.786 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 27787264 bytes via buffered write, may repeat 2023-10-18T08:58:48.787 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 28835840 bytes via buffered write, may repeat 2023-10-18T08:58:48.787 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 29884416 bytes via buffered write, may repeat 2023-10-18T08:58:48.787 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 30932992 bytes via buffered write, may repeat 2023-10-18T08:58:48.787 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 31981568 bytes via buffered write, may repeat 2023-10-18T08:58:48.787 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 33030144 bytes via buffered write, may repeat 2023-10-18T08:58:48.788 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 34078720 bytes via buffered write, may repeat 2023-10-18T08:58:48.788 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 35127296 bytes via buffered write, may repeat 2023-10-18T08:58:48.788 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 36175872 bytes via buffered write, may repeat 2023-10-18T08:58:48.788 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 37224448 bytes via buffered write, may repeat 2023-10-18T08:58:48.788 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 38273024 bytes via buffered write, may repeat 2023-10-18T08:58:48.788 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 39321600 bytes via buffered write, may repeat 2023-10-18T08:58:48.789 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 40370176 bytes via buffered write, may repeat 2023-10-18T08:58:48.789 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 41418752 bytes via buffered write, may repeat 2023-10-18T08:58:48.789 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 42467328 bytes via buffered write, may repeat 2023-10-18T08:58:48.789 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 43515904 bytes via buffered write, may repeat 2023-10-18T08:58:48.789 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 44564480 bytes via buffered write, may repeat 2023-10-18T08:58:48.790 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 45613056 bytes via buffered write, may repeat 2023-10-18T08:58:48.790 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 46661632 bytes via buffered write, may repeat 2023-10-18T08:58:48.790 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 47710208 bytes via buffered write, may repeat 2023-10-18T08:58:48.790 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 48758784 bytes via buffered write, may repeat 2023-10-18T08:58:48.790 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 49807360 bytes via buffered write, may repeat 2023-10-18T08:58:48.790 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 50855936 bytes via buffered write, may repeat 2023-10-18T08:58:48.791 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 51904512 bytes via buffered write, may repeat 2023-10-18T08:58:48.791 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 52953088 bytes via buffered write, may repeat 2023-10-18T08:58:48.791 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 54001664 bytes via buffered write, may repeat 2023-10-18T08:58:48.791 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 55050240 bytes via buffered write, may repeat 2023-10-18T08:58:48.792 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 56098816 bytes via buffered write, may repeat 2023-10-18T08:58:48.792 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 57147392 bytes via buffered write, may repeat 2023-10-18T08:58:48.792 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 58195968 bytes via buffered write, may repeat 2023-10-18T08:58:48.792 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 59244544 bytes via buffered write, may repeat 2023-10-18T08:58:48.792 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 60293120 bytes via buffered write, may repeat 2023-10-18T08:58:48.792 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 61341696 bytes via buffered write, may repeat 2023-10-18T08:58:48.793 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 62390272 bytes via buffered write, may repeat 2023-10-18T08:58:48.793 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 63438848 bytes via buffered write, may repeat 2023-10-18T08:58:48.793 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 64487424 bytes via buffered write, may repeat 2023-10-18T08:58:48.793 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 65536000 bytes via buffered write, may repeat 2023-10-18T08:58:48.793 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 66584576 bytes via buffered write, may repeat 2023-10-18T08:58:48.794 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 67633152 bytes via buffered write, may repeat 2023-10-18T08:58:48.794 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 68681728 bytes via buffered write, may repeat 2023-10-18T08:58:48.794 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 69730304 bytes via buffered write, may repeat 2023-10-18T08:58:48.794 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 70778880 bytes via buffered write, may repeat 2023-10-18T08:58:48.794 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 71827456 bytes via buffered write, may repeat 2023-10-18T08:58:48.795 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 72876032 bytes via buffered write, may repeat 2023-10-18T08:58:48.795 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 73924608 bytes via buffered write, may repeat 2023-10-18T08:58:48.795 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 74973184 bytes via buffered write, may repeat 2023-10-18T08:58:48.795 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 76021760 bytes via buffered write, may repeat 2023-10-18T08:58:48.795 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 77070336 bytes via buffered write, may repeat 2023-10-18T08:58:48.796 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 78118912 bytes via buffered write, may repeat 2023-10-18T08:58:48.796 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 79167488 bytes via buffered write, may repeat 2023-10-18T08:58:48.796 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 80216064 bytes via buffered write, may repeat 2023-10-18T08:58:48.796 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 81264640 bytes via buffered write, may repeat 2023-10-18T08:58:48.796 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 82313216 bytes via buffered write, may repeat 2023-10-18T08:58:48.797 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 83361792 bytes via buffered write, may repeat 2023-10-18T08:58:48.797 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 84410368 bytes via buffered write, may repeat 2023-10-18T08:58:48.797 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 85458944 bytes via buffered write, may repeat 2023-10-18T08:58:48.797 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 86507520 bytes via buffered write, may repeat 2023-10-18T08:58:48.798 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 87556096 bytes via buffered write, may repeat 2023-10-18T08:58:48.798 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 88604672 bytes via buffered write, may repeat 2023-10-18T08:58:48.798 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 89653248 bytes via buffered write, may repeat 2023-10-18T08:58:48.798 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 90701824 bytes via buffered write, may repeat 2023-10-18T08:58:48.798 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 91750400 bytes via buffered write, may repeat 2023-10-18T08:58:48.798 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 92798976 bytes via buffered write, may repeat 2023-10-18T08:58:48.799 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 93847552 bytes via buffered write, may repeat 2023-10-18T08:58:48.799 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 94896128 bytes via buffered write, may repeat 2023-10-18T08:58:48.799 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 95944704 bytes via buffered write, may repeat 2023-10-18T08:58:48.799 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 96993280 bytes via buffered write, may repeat 2023-10-18T08:58:48.799 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 98041856 bytes via buffered write, may repeat 2023-10-18T08:58:48.800 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 99090432 bytes via buffered write, may repeat 2023-10-18T08:58:48.800 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 100139008 bytes via buffered write, may repeat 2023-10-18T08:58:48.800 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 101187584 bytes via buffered write, may repeat 2023-10-18T08:58:48.800 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 102236160 bytes via buffered write, may repeat 2023-10-18T08:58:48.800 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 103284736 bytes via buffered write, may repeat 2023-10-18T08:58:48.801 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 104333312 bytes via buffered write, may repeat 2023-10-18T08:58:48.801 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 105381888 bytes via buffered write, may repeat 2023-10-18T08:58:48.801 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 106430464 bytes via buffered write, may repeat 2023-10-18T08:58:48.801 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 107479040 bytes via buffered write, may repeat 2023-10-18T08:58:48.801 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 108527616 bytes via buffered write, may repeat 2023-10-18T08:58:48.801 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 109576192 bytes via buffered write, may repeat 2023-10-18T08:58:48.802 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 110624768 bytes via buffered write, may repeat 2023-10-18T08:58:48.802 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 111673344 bytes via buffered write, may repeat 2023-10-18T08:58:48.802 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 112721920 bytes via buffered write, may repeat 2023-10-18T08:58:48.802 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 113770496 bytes via buffered write, may repeat 2023-10-18T08:58:48.802 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 114819072 bytes via buffered write, may repeat 2023-10-18T08:58:48.803 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 115867648 bytes via buffered write, may repeat 2023-10-18T08:58:48.803 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 116916224 bytes via buffered write, may repeat 2023-10-18T08:58:48.803 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 117964800 bytes via buffered write, may repeat 2023-10-18T08:58:48.803 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 119013376 bytes via buffered write, may repeat 2023-10-18T08:58:48.803 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 120061952 bytes via buffered write, may repeat 2023-10-18T08:58:48.803 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 121110528 bytes via buffered write, may repeat 2023-10-18T08:58:48.804 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 122159104 bytes via buffered write, may repeat 2023-10-18T08:58:48.804 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 123207680 bytes via buffered write, may repeat 2023-10-18T08:58:48.804 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 124256256 bytes via buffered write, may repeat 2023-10-18T08:58:48.804 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 125304832 bytes via buffered write, may repeat 2023-10-18T08:58:48.804 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 126353408 bytes via buffered write, may repeat 2023-10-18T08:58:48.805 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 127401984 bytes via buffered write, may repeat 2023-10-18T08:58:48.805 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 128450560 bytes via buffered write, may repeat 2023-10-18T08:58:48.805 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 129499136 bytes via buffered write, may repeat 2023-10-18T08:58:48.805 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 130547712 bytes via buffered write, may repeat 2023-10-18T08:58:48.805 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 131596288 bytes via buffered write, may repeat 2023-10-18T08:58:48.806 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 132644864 bytes via buffered write, may repeat 2023-10-18T08:58:48.806 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 133693440 bytes via buffered write, may repeat 2023-10-18T08:58:48.806 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 134742016 bytes via buffered write, may repeat 2023-10-18T08:58:48.806 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 135790592 bytes via buffered write, may repeat 2023-10-18T08:58:48.806 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 136839168 bytes via buffered write, may repeat 2023-10-18T08:58:48.806 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 137887744 bytes via buffered write, may repeat 2023-10-18T08:58:48.807 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 138936320 bytes via buffered write, may repeat 2023-10-18T08:58:48.807 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 139984896 bytes via buffered write, may repeat 2023-10-18T08:58:48.807 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 141033472 bytes via buffered write, may repeat 2023-10-18T08:58:48.807 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 142082048 bytes via buffered write, may repeat 2023-10-18T08:58:48.807 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 143130624 bytes via buffered write, may repeat 2023-10-18T08:58:48.808 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 144179200 bytes via buffered write, may repeat 2023-10-18T08:58:48.808 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 145227776 bytes via buffered write, may repeat 2023-10-18T08:58:48.808 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 146276352 bytes via buffered write, may repeat 2023-10-18T08:58:48.808 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 147324928 bytes via buffered write, may repeat 2023-10-18T08:58:48.808 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 148373504 bytes via buffered write, may repeat 2023-10-18T08:58:48.809 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 149422080 bytes via buffered write, may repeat 2023-10-18T08:58:48.809 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 150470656 bytes via buffered write, may repeat 2023-10-18T08:58:48.809 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 151519232 bytes via buffered write, may repeat 2023-10-18T08:58:48.809 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 152567808 bytes via buffered write, may repeat 2023-10-18T08:58:48.809 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 153616384 bytes via buffered write, may repeat 2023-10-18T08:58:48.809 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 154664960 bytes via buffered write, may repeat 2023-10-18T08:58:48.810 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 155713536 bytes via buffered write, may repeat 2023-10-18T08:58:48.810 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 156762112 bytes via buffered write, may repeat 2023-10-18T08:58:48.810 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 157810688 bytes via buffered write, may repeat 2023-10-18T08:58:48.810 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 158859264 bytes via buffered write, may repeat 2023-10-18T08:58:48.810 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 159907840 bytes via buffered write, may repeat 2023-10-18T08:58:48.811 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 160956416 bytes via buffered write, may repeat 2023-10-18T08:58:48.811 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 162004992 bytes via buffered write, may repeat 2023-10-18T08:58:48.964 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 163053568 bytes via buffered write, may repeat 2023-10-18T08:58:48.964 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 164102144 bytes via buffered write, may repeat 2023-10-18T08:58:48.965 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 165150720 bytes via buffered write, may repeat 2023-10-18T08:58:48.965 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 166199296 bytes via buffered write, may repeat 2023-10-18T08:58:48.965 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 167247872 bytes via buffered write, may repeat 2023-10-18T08:58:48.966 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 168296448 bytes via buffered write, may repeat 2023-10-18T08:58:48.966 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 169345024 bytes via buffered write, may repeat 2023-10-18T08:58:48.966 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 170393600 bytes via buffered write, may repeat 2023-10-18T08:58:48.967 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 171442176 bytes via buffered write, may repeat 2023-10-18T08:58:48.967 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 172490752 bytes via buffered write, may repeat 2023-10-18T08:58:48.967 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 173539328 bytes via buffered write, may repeat 2023-10-18T08:58:48.968 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 174587904 bytes via buffered write, may repeat 2023-10-18T08:58:48.968 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 175636480 bytes via buffered write, may repeat 2023-10-18T08:58:48.968 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 176685056 bytes via buffered write, may repeat 2023-10-18T08:58:48.969 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 177733632 bytes via buffered write, may repeat 2023-10-18T08:58:48.969 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 178782208 bytes via buffered write, may repeat 2023-10-18T08:58:48.970 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 179830784 bytes via buffered write, may repeat 2023-10-18T08:58:48.970 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 180879360 bytes via buffered write, may repeat 2023-10-18T08:58:48.970 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 181927936 bytes via buffered write, may repeat 2023-10-18T08:58:48.970 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 182976512 bytes via buffered write, may repeat 2023-10-18T08:58:48.971 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 184025088 bytes via buffered write, may repeat 2023-10-18T08:58:48.971 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 185073664 bytes via buffered write, may repeat 2023-10-18T08:58:48.972 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 186122240 bytes via buffered write, may repeat 2023-10-18T08:58:48.972 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 187170816 bytes via buffered write, may repeat 2023-10-18T08:58:48.972 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 188219392 bytes via buffered write, may repeat 2023-10-18T08:58:48.973 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 189267968 bytes via buffered write, may repeat 2023-10-18T08:58:48.973 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 190316544 bytes via buffered write, may repeat 2023-10-18T08:58:48.973 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 191365120 bytes via buffered write, may repeat 2023-10-18T08:58:48.974 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 192413696 bytes via buffered write, may repeat 2023-10-18T08:58:48.974 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 193462272 bytes via buffered write, may repeat 2023-10-18T08:58:48.974 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 194510848 bytes via buffered write, may repeat 2023-10-18T08:58:48.975 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 195559424 bytes via buffered write, may repeat 2023-10-18T08:58:48.975 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 196608000 bytes via buffered write, may repeat 2023-10-18T08:58:48.975 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 197656576 bytes via buffered write, may repeat 2023-10-18T08:58:48.976 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 198705152 bytes via buffered write, may repeat 2023-10-18T08:58:48.976 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 199753728 bytes via buffered write, may repeat 2023-10-18T08:58:48.977 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 200802304 bytes via buffered write, may repeat 2023-10-18T08:58:48.977 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 201850880 bytes via buffered write, may repeat 2023-10-18T08:58:48.977 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 202899456 bytes via buffered write, may repeat 2023-10-18T08:58:48.977 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 203948032 bytes via buffered write, may repeat 2023-10-18T08:58:48.978 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 204996608 bytes via buffered write, may repeat 2023-10-18T08:58:48.978 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 206045184 bytes via buffered write, may repeat 2023-10-18T08:58:48.978 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 207093760 bytes via buffered write, may repeat 2023-10-18T08:58:48.978 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 208142336 bytes via buffered write, may repeat 2023-10-18T08:58:48.979 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 209190912 bytes via buffered write, may repeat 2023-10-18T08:58:48.979 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 210239488 bytes via buffered write, may repeat 2023-10-18T08:58:48.979 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 211288064 bytes via buffered write, may repeat 2023-10-18T08:58:48.979 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 212336640 bytes via buffered write, may repeat 2023-10-18T08:58:48.979 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 213385216 bytes via buffered write, may repeat 2023-10-18T08:58:48.980 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 214433792 bytes via buffered write, may repeat 2023-10-18T08:58:48.980 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 215482368 bytes via buffered write, may repeat 2023-10-18T08:58:48.980 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 216530944 bytes via buffered write, may repeat 2023-10-18T08:58:48.981 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 217579520 bytes via buffered write, may repeat 2023-10-18T08:58:48.981 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 218628096 bytes via buffered write, may repeat 2023-10-18T08:58:48.981 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 219676672 bytes via buffered write, may repeat 2023-10-18T08:58:48.981 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 220725248 bytes via buffered write, may repeat 2023-10-18T08:58:48.981 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 221773824 bytes via buffered write, may repeat 2023-10-18T08:58:48.982 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 222822400 bytes via buffered write, may repeat 2023-10-18T08:58:48.982 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 223870976 bytes via buffered write, may repeat 2023-10-18T08:58:48.982 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 224919552 bytes via buffered write, may repeat 2023-10-18T08:58:48.983 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 225968128 bytes via buffered write, may repeat 2023-10-18T08:58:48.983 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 227016704 bytes via buffered write, may repeat 2023-10-18T08:58:48.983 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 228065280 bytes via buffered write, may repeat 2023-10-18T08:58:48.983 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 229113856 bytes via buffered write, may repeat 2023-10-18T08:58:48.984 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 230162432 bytes via buffered write, may repeat 2023-10-18T08:58:48.984 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 231211008 bytes via buffered write, may repeat 2023-10-18T08:58:48.984 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 232259584 bytes via buffered write, may repeat 2023-10-18T08:58:48.984 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 233308160 bytes via buffered write, may repeat 2023-10-18T08:58:48.985 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 234356736 bytes via buffered write, may repeat 2023-10-18T08:58:48.985 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 235405312 bytes via buffered write, may repeat 2023-10-18T08:58:48.985 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 236453888 bytes via buffered write, may repeat 2023-10-18T08:58:48.985 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 237502464 bytes via buffered write, may repeat 2023-10-18T08:58:48.986 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 238551040 bytes via buffered write, may repeat 2023-10-18T08:58:48.986 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 239599616 bytes via buffered write, may repeat 2023-10-18T08:58:48.986 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 240648192 bytes via buffered write, may repeat 2023-10-18T08:58:48.987 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 241696768 bytes via buffered write, may repeat 2023-10-18T08:58:48.987 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 242745344 bytes via buffered write, may repeat 2023-10-18T08:58:48.987 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 243793920 bytes via buffered write, may repeat 2023-10-18T08:58:48.987 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 244842496 bytes via buffered write, may repeat 2023-10-18T08:58:48.987 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 245891072 bytes via buffered write, may repeat 2023-10-18T08:58:48.987 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 246939648 bytes via buffered write, may repeat 2023-10-18T08:58:48.988 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 247988224 bytes via buffered write, may repeat 2023-10-18T08:58:48.988 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 249036800 bytes via buffered write, may repeat 2023-10-18T08:58:48.988 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 250085376 bytes via buffered write, may repeat 2023-10-18T08:58:48.989 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 251133952 bytes via buffered write, may repeat 2023-10-18T08:58:48.989 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 252182528 bytes via buffered write, may repeat 2023-10-18T08:58:48.989 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 253231104 bytes via buffered write, may repeat 2023-10-18T08:58:48.989 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 254279680 bytes via buffered write, may repeat 2023-10-18T08:58:48.990 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 255328256 bytes via buffered write, may repeat 2023-10-18T08:58:48.990 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 256376832 bytes via buffered write, may repeat 2023-10-18T08:58:48.990 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 257425408 bytes via buffered write, may repeat 2023-10-18T08:58:48.991 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 258473984 bytes via buffered write, may repeat 2023-10-18T08:58:48.991 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 259522560 bytes via buffered write, may repeat 2023-10-18T08:58:48.991 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 260571136 bytes via buffered write, may repeat 2023-10-18T08:58:48.991 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 261619712 bytes via buffered write, may repeat 2023-10-18T08:58:48.991 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 262668288 bytes via buffered write, may repeat 2023-10-18T08:58:48.991 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 263716864 bytes via buffered write, may repeat 2023-10-18T08:58:48.992 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 264765440 bytes via buffered write, may repeat 2023-10-18T08:58:48.992 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 265814016 bytes via buffered write, may repeat 2023-10-18T08:58:48.992 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 266862592 bytes via buffered write, may repeat 2023-10-18T08:58:48.993 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 267911168 bytes via buffered write, may repeat 2023-10-18T08:58:48.993 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 268959744 bytes via buffered write, may repeat 2023-10-18T08:58:48.993 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 270008320 bytes via buffered write, may repeat 2023-10-18T08:58:48.993 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 271056896 bytes via buffered write, may repeat 2023-10-18T08:58:48.994 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 272105472 bytes via buffered write, may repeat 2023-10-18T08:58:48.994 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 273154048 bytes via buffered write, may repeat 2023-10-18T08:58:48.994 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 274202624 bytes via buffered write, may repeat 2023-10-18T08:58:48.995 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 275251200 bytes via buffered write, may repeat 2023-10-18T08:58:48.995 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 276299776 bytes via buffered write, may repeat 2023-10-18T08:58:48.995 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 277348352 bytes via buffered write, may repeat 2023-10-18T08:58:48.995 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 278396928 bytes via buffered write, may repeat 2023-10-18T08:58:48.995 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 279445504 bytes via buffered write, may repeat 2023-10-18T08:58:48.996 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 280494080 bytes via buffered write, may repeat 2023-10-18T08:58:48.996 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 281542656 bytes via buffered write, may repeat 2023-10-18T08:58:48.996 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 282591232 bytes via buffered write, may repeat 2023-10-18T08:58:48.997 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 283639808 bytes via buffered write, may repeat 2023-10-18T08:58:48.997 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 284688384 bytes via buffered write, may repeat 2023-10-18T08:58:48.997 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 285736960 bytes via buffered write, may repeat 2023-10-18T08:58:48.997 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 286785536 bytes via buffered write, may repeat 2023-10-18T08:58:48.998 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 287834112 bytes via buffered write, may repeat 2023-10-18T08:58:48.998 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 288882688 bytes via buffered write, may repeat 2023-10-18T08:58:48.998 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 289931264 bytes via buffered write, may repeat 2023-10-18T08:58:48.998 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 290979840 bytes via buffered write, may repeat 2023-10-18T08:58:48.999 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 292028416 bytes via buffered write, may repeat 2023-10-18T08:58:48.999 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 293076992 bytes via buffered write, may repeat 2023-10-18T08:58:48.999 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 294125568 bytes via buffered write, may repeat 2023-10-18T08:58:48.999 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 295174144 bytes via buffered write, may repeat 2023-10-18T08:58:49.000 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 296222720 bytes via buffered write, may repeat 2023-10-18T08:58:49.000 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 297271296 bytes via buffered write, may repeat 2023-10-18T08:58:49.000 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 298319872 bytes via buffered write, may repeat 2023-10-18T08:58:49.000 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 299368448 bytes via buffered write, may repeat 2023-10-18T08:58:49.000 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 300417024 bytes via buffered write, may repeat 2023-10-18T08:58:49.001 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 301465600 bytes via buffered write, may repeat 2023-10-18T08:58:49.001 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 302514176 bytes via buffered write, may repeat 2023-10-18T08:58:49.001 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 303562752 bytes via buffered write, may repeat 2023-10-18T08:58:49.001 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 304611328 bytes via buffered write, may repeat 2023-10-18T08:58:49.001 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 305659904 bytes via buffered write, may repeat 2023-10-18T08:58:49.002 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 306708480 bytes via buffered write, may repeat 2023-10-18T08:58:49.002 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 307757056 bytes via buffered write, may repeat 2023-10-18T08:58:49.002 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 308805632 bytes via buffered write, may repeat 2023-10-18T08:58:49.002 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 309854208 bytes via buffered write, may repeat 2023-10-18T08:58:49.002 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 310902784 bytes via buffered write, may repeat 2023-10-18T08:58:49.003 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 311951360 bytes via buffered write, may repeat 2023-10-18T08:58:49.003 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 312999936 bytes via buffered write, may repeat 2023-10-18T08:58:49.003 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 314048512 bytes via buffered write, may repeat 2023-10-18T08:58:49.003 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 315097088 bytes via buffered write, may repeat 2023-10-18T08:58:49.003 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 316145664 bytes via buffered write, may repeat 2023-10-18T08:58:49.004 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 317194240 bytes via buffered write, may repeat 2023-10-18T08:58:49.004 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 318242816 bytes via buffered write, may repeat 2023-10-18T08:58:49.004 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 319291392 bytes via buffered write, may repeat 2023-10-18T08:58:49.004 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 320339968 bytes via buffered write, may repeat 2023-10-18T08:58:49.004 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 321388544 bytes via buffered write, may repeat 2023-10-18T08:58:49.004 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 322437120 bytes via buffered write, may repeat 2023-10-18T08:58:49.005 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 323485696 bytes via buffered write, may repeat 2023-10-18T08:58:49.147 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 324534272 bytes via buffered write, may repeat 2023-10-18T08:58:49.148 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 325582848 bytes via buffered write, may repeat 2023-10-18T08:58:49.148 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 326631424 bytes via buffered write, may repeat 2023-10-18T08:58:49.148 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 327680000 bytes via buffered write, may repeat 2023-10-18T08:58:49.148 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 328728576 bytes via buffered write, may repeat 2023-10-18T08:58:49.148 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 329777152 bytes via buffered write, may repeat 2023-10-18T08:58:49.149 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 330825728 bytes via buffered write, may repeat 2023-10-18T08:58:49.149 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 331874304 bytes via buffered write, may repeat 2023-10-18T08:58:49.149 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 332922880 bytes via buffered write, may repeat 2023-10-18T08:58:49.149 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 333971456 bytes via buffered write, may repeat 2023-10-18T08:58:49.150 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 335020032 bytes via buffered write, may repeat 2023-10-18T08:58:49.150 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 336068608 bytes via buffered write, may repeat 2023-10-18T08:58:49.150 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 337117184 bytes via buffered write, may repeat 2023-10-18T08:58:49.150 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 338165760 bytes via buffered write, may repeat 2023-10-18T08:58:49.150 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 339214336 bytes via buffered write, may repeat 2023-10-18T08:58:49.151 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 340262912 bytes via buffered write, may repeat 2023-10-18T08:58:49.151 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 341311488 bytes via buffered write, may repeat 2023-10-18T08:58:49.151 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 342360064 bytes via buffered write, may repeat 2023-10-18T08:58:49.151 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 343408640 bytes via buffered write, may repeat 2023-10-18T08:58:49.151 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 344457216 bytes via buffered write, may repeat 2023-10-18T08:58:49.152 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 345505792 bytes via buffered write, may repeat 2023-10-18T08:58:49.152 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 346554368 bytes via buffered write, may repeat 2023-10-18T08:58:49.152 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 347602944 bytes via buffered write, may repeat 2023-10-18T08:58:49.152 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 348651520 bytes via buffered write, may repeat 2023-10-18T08:58:49.152 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 349700096 bytes via buffered write, may repeat 2023-10-18T08:58:49.153 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 350748672 bytes via buffered write, may repeat 2023-10-18T08:58:49.153 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 351797248 bytes via buffered write, may repeat 2023-10-18T08:58:49.153 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 352845824 bytes via buffered write, may repeat 2023-10-18T08:58:49.153 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 353894400 bytes via buffered write, may repeat 2023-10-18T08:58:49.153 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 354942976 bytes via buffered write, may repeat 2023-10-18T08:58:49.154 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 355991552 bytes via buffered write, may repeat 2023-10-18T08:58:49.154 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 357040128 bytes via buffered write, may repeat 2023-10-18T08:58:49.154 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 358088704 bytes via buffered write, may repeat 2023-10-18T08:58:49.154 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 359137280 bytes via buffered write, may repeat 2023-10-18T08:58:49.155 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 360185856 bytes via buffered write, may repeat 2023-10-18T08:58:49.155 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 361234432 bytes via buffered write, may repeat 2023-10-18T08:58:49.155 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 362283008 bytes via buffered write, may repeat 2023-10-18T08:58:49.156 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 363331584 bytes via buffered write, may repeat 2023-10-18T08:58:49.156 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 364380160 bytes via buffered write, may repeat 2023-10-18T08:58:49.156 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 365428736 bytes via buffered write, may repeat 2023-10-18T08:58:49.156 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 366477312 bytes via buffered write, may repeat 2023-10-18T08:58:49.157 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 367525888 bytes via buffered write, may repeat 2023-10-18T08:58:49.157 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 368574464 bytes via buffered write, may repeat 2023-10-18T08:58:49.157 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 369623040 bytes via buffered write, may repeat 2023-10-18T08:58:49.157 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 370671616 bytes via buffered write, may repeat 2023-10-18T08:58:49.157 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 371720192 bytes via buffered write, may repeat 2023-10-18T08:58:49.158 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 372768768 bytes via buffered write, may repeat 2023-10-18T08:58:49.158 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 373817344 bytes via buffered write, may repeat 2023-10-18T08:58:49.158 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 374865920 bytes via buffered write, may repeat 2023-10-18T08:58:49.158 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 375914496 bytes via buffered write, may repeat 2023-10-18T08:58:49.159 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 376963072 bytes via buffered write, may repeat 2023-10-18T08:58:49.159 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 378011648 bytes via buffered write, may repeat 2023-10-18T08:58:49.159 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 379060224 bytes via buffered write, may repeat 2023-10-18T08:58:49.159 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 380108800 bytes via buffered write, may repeat 2023-10-18T08:58:49.159 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 381157376 bytes via buffered write, may repeat 2023-10-18T08:58:49.160 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 382205952 bytes via buffered write, may repeat 2023-10-18T08:58:49.160 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 383254528 bytes via buffered write, may repeat 2023-10-18T08:58:49.160 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 384303104 bytes via buffered write, may repeat 2023-10-18T08:58:49.160 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 385351680 bytes via buffered write, may repeat 2023-10-18T08:58:49.160 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 386400256 bytes via buffered write, may repeat 2023-10-18T08:58:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 387448832 bytes via buffered write, may repeat 2023-10-18T08:58:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 388497408 bytes via buffered write, may repeat 2023-10-18T08:58:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 389545984 bytes via buffered write, may repeat 2023-10-18T08:58:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 390594560 bytes via buffered write, may repeat 2023-10-18T08:58:49.161 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 391643136 bytes via buffered write, may repeat 2023-10-18T08:58:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 392691712 bytes via buffered write, may repeat 2023-10-18T08:58:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 393740288 bytes via buffered write, may repeat 2023-10-18T08:58:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 394788864 bytes via buffered write, may repeat 2023-10-18T08:58:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 395837440 bytes via buffered write, may repeat 2023-10-18T08:58:49.162 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 396886016 bytes via buffered write, may repeat 2023-10-18T08:58:49.163 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 397934592 bytes via buffered write, may repeat 2023-10-18T08:58:49.163 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 398983168 bytes via buffered write, may repeat 2023-10-18T08:58:49.163 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 400031744 bytes via buffered write, may repeat 2023-10-18T08:58:49.163 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 401080320 bytes via buffered write, may repeat 2023-10-18T08:58:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 402128896 bytes via buffered write, may repeat 2023-10-18T08:58:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 403177472 bytes via buffered write, may repeat 2023-10-18T08:58:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 404226048 bytes via buffered write, may repeat 2023-10-18T08:58:49.164 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 405274624 bytes via buffered write, may repeat 2023-10-18T08:58:49.165 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 406323200 bytes via buffered write, may repeat 2023-10-18T08:58:49.165 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 407371776 bytes via buffered write, may repeat 2023-10-18T08:58:49.165 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 408420352 bytes via buffered write, may repeat 2023-10-18T08:58:49.165 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 409468928 bytes via buffered write, may repeat 2023-10-18T08:58:49.165 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 410517504 bytes via buffered write, may repeat 2023-10-18T08:58:49.166 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 411566080 bytes via buffered write, may repeat 2023-10-18T08:58:49.166 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 412614656 bytes via buffered write, may repeat 2023-10-18T08:58:49.166 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 413663232 bytes via buffered write, may repeat 2023-10-18T08:58:49.167 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 414711808 bytes via buffered write, may repeat 2023-10-18T08:58:49.167 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 415760384 bytes via buffered write, may repeat 2023-10-18T08:58:49.167 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 416808960 bytes via buffered write, may repeat 2023-10-18T08:58:49.167 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 417857536 bytes via buffered write, may repeat 2023-10-18T08:58:49.168 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 418906112 bytes via buffered write, may repeat 2023-10-18T08:58:49.168 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 419954688 bytes via buffered write, may repeat 2023-10-18T08:58:49.168 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 421003264 bytes via buffered write, may repeat 2023-10-18T08:58:49.168 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 422051840 bytes via buffered write, may repeat 2023-10-18T08:58:49.168 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 423100416 bytes via buffered write, may repeat 2023-10-18T08:58:49.169 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 424148992 bytes via buffered write, may repeat 2023-10-18T08:58:49.169 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 425197568 bytes via buffered write, may repeat 2023-10-18T08:58:49.169 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 426246144 bytes via buffered write, may repeat 2023-10-18T08:58:49.169 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 427294720 bytes via buffered write, may repeat 2023-10-18T08:58:49.169 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 428343296 bytes via buffered write, may repeat 2023-10-18T08:58:49.170 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 429391872 bytes via buffered write, may repeat 2023-10-18T08:58:49.170 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 430440448 bytes via buffered write, may repeat 2023-10-18T08:58:49.170 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 431489024 bytes via buffered write, may repeat 2023-10-18T08:58:49.170 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 432537600 bytes via buffered write, may repeat 2023-10-18T08:58:49.170 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 433586176 bytes via buffered write, may repeat 2023-10-18T08:58:49.171 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 434634752 bytes via buffered write, may repeat 2023-10-18T08:58:49.171 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 435683328 bytes via buffered write, may repeat 2023-10-18T08:58:49.171 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 436731904 bytes via buffered write, may repeat 2023-10-18T08:58:49.171 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 437780480 bytes via buffered write, may repeat 2023-10-18T08:58:49.171 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 438829056 bytes via buffered write, may repeat 2023-10-18T08:58:49.172 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 439877632 bytes via buffered write, may repeat 2023-10-18T08:58:49.172 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 440926208 bytes via buffered write, may repeat 2023-10-18T08:58:49.172 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 441974784 bytes via buffered write, may repeat 2023-10-18T08:58:49.172 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 443023360 bytes via buffered write, may repeat 2023-10-18T08:58:49.172 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 444071936 bytes via buffered write, may repeat 2023-10-18T08:58:49.173 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 445120512 bytes via buffered write, may repeat 2023-10-18T08:58:49.173 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 446169088 bytes via buffered write, may repeat 2023-10-18T08:58:49.173 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 447217664 bytes via buffered write, may repeat 2023-10-18T08:58:49.173 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 448266240 bytes via buffered write, may repeat 2023-10-18T08:58:49.174 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 449314816 bytes via buffered write, may repeat 2023-10-18T08:58:49.174 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 450363392 bytes via buffered write, may repeat 2023-10-18T08:58:49.174 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 451411968 bytes via buffered write, may repeat 2023-10-18T08:58:49.174 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 452460544 bytes via buffered write, may repeat 2023-10-18T08:58:49.175 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 453509120 bytes via buffered write, may repeat 2023-10-18T08:58:49.175 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 454557696 bytes via buffered write, may repeat 2023-10-18T08:58:49.175 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 455606272 bytes via buffered write, may repeat 2023-10-18T08:58:49.175 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 456654848 bytes via buffered write, may repeat 2023-10-18T08:58:49.175 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 457703424 bytes via buffered write, may repeat 2023-10-18T08:58:49.176 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 458752000 bytes via buffered write, may repeat 2023-10-18T08:58:49.176 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 459800576 bytes via buffered write, may repeat 2023-10-18T08:58:49.176 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 460849152 bytes via buffered write, may repeat 2023-10-18T08:58:49.176 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 461897728 bytes via buffered write, may repeat 2023-10-18T08:58:49.177 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 462946304 bytes via buffered write, may repeat 2023-10-18T08:58:49.177 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 463994880 bytes via buffered write, may repeat 2023-10-18T08:58:49.177 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 465043456 bytes via buffered write, may repeat 2023-10-18T08:58:49.177 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 466092032 bytes via buffered write, may repeat 2023-10-18T08:58:49.178 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 467140608 bytes via buffered write, may repeat 2023-10-18T08:58:49.178 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 468189184 bytes via buffered write, may repeat 2023-10-18T08:58:49.178 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 469237760 bytes via buffered write, may repeat 2023-10-18T08:58:49.178 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 470286336 bytes via buffered write, may repeat 2023-10-18T08:58:49.179 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 471334912 bytes via buffered write, may repeat 2023-10-18T08:58:49.179 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 472383488 bytes via buffered write, may repeat 2023-10-18T08:58:49.179 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 473432064 bytes via buffered write, may repeat 2023-10-18T08:58:49.179 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 474480640 bytes via buffered write, may repeat 2023-10-18T08:58:49.179 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 475529216 bytes via buffered write, may repeat 2023-10-18T08:58:49.179 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 476577792 bytes via buffered write, may repeat 2023-10-18T08:58:49.180 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 477626368 bytes via buffered write, may repeat 2023-10-18T08:58:49.180 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 478674944 bytes via buffered write, may repeat 2023-10-18T08:58:49.180 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 479723520 bytes via buffered write, may repeat 2023-10-18T08:58:49.180 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 480772096 bytes via buffered write, may repeat 2023-10-18T08:58:49.181 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 481820672 bytes via buffered write, may repeat 2023-10-18T08:58:49.181 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 482869248 bytes via buffered write, may repeat 2023-10-18T08:58:49.181 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 483917824 bytes via buffered write, may repeat 2023-10-18T08:58:49.181 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 484966400 bytes via buffered write, may repeat 2023-10-18T08:58:51.928 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T08:58:51.926+0000 7f6a11f15640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2023-10-18T08:58:58.030 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T08:58:58.027+0000 7f6a11f15640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-10-18T08:59:05.162 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:59:05.173 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:59:35.330 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:59:35.341 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T08:59:50.512 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 486014976 bytes via buffered write, may repeat 2023-10-18T08:59:50.513 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 487063552 bytes via buffered write, may repeat 2023-10-18T08:59:50.513 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 488112128 bytes via buffered write, may repeat 2023-10-18T08:59:50.513 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 489160704 bytes via buffered write, may repeat 2023-10-18T08:59:50.513 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 490209280 bytes via buffered write, may repeat 2023-10-18T08:59:50.513 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 491257856 bytes via buffered write, may repeat 2023-10-18T08:59:50.514 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 492306432 bytes via buffered write, may repeat 2023-10-18T08:59:50.514 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 493355008 bytes via buffered write, may repeat 2023-10-18T08:59:50.514 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 494403584 bytes via buffered write, may repeat 2023-10-18T08:59:50.514 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 495452160 bytes via buffered write, may repeat 2023-10-18T08:59:50.514 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 496500736 bytes via buffered write, may repeat 2023-10-18T08:59:50.515 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 497549312 bytes via buffered write, may repeat 2023-10-18T08:59:50.515 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 498597888 bytes via buffered write, may repeat 2023-10-18T08:59:50.515 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 499646464 bytes via buffered write, may repeat 2023-10-18T08:59:50.515 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 500695040 bytes via buffered write, may repeat 2023-10-18T08:59:50.515 INFO:teuthology.orchestra.run.smithi040.stdout:done writing 500695040 bytes 2023-10-18T08:59:50.515 INFO:teuthology.orchestra.run.smithi040.stdout:successfully fsync'ed prior to getting full state reported 2023-10-18T08:59:50.516 INFO:teuthology.orchestra.run.smithi040.stdout:starting buffered write 2023-10-18T08:59:50.516 INFO:teuthology.orchestra.run.smithi040.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-18T08:59:50.516 INFO:teuthology.orchestra.run.smithi040.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-18T08:59:50.516 INFO:teuthology.orchestra.run.smithi040.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2023-10-18T08:59:50.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T08:59:50.538 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T08:59:50.539 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T08:59:50.663 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T08:59:50.791 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T08:59:50.792 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:59:50.792 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:59:50.858 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:59:50.859 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:59:50.917 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:59:51.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.156+0000 7fa95b444640 1 -- 172.21.15.40:0/3394629469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9400039c0 msgr2=0x7fa940003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:51.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.156+0000 7fa95b444640 1 --2- 172.21.15.40:0/3394629469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9400039c0 0x7fa940003de0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa95000c610 tx=0x7fa9400057f0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.156+0000 7fa95b444640 1 -- 172.21.15.40:0/3394629469 shutdown_connections 2023-10-18T08:59:51.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.156+0000 7fa95b444640 1 --2- 172.21.15.40:0/3394629469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9400039c0 0x7fa940003de0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.156+0000 7fa95b444640 1 -- 172.21.15.40:0/3394629469 >> 172.21.15.40:0/3394629469 conn(0x7fa9540655f0 msgr2=0x7fa9540659d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:51.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.156+0000 7fa95b444640 1 -- 172.21.15.40:0/3394629469 shutdown_connections 2023-10-18T08:59:51.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.156+0000 7fa95b444640 1 -- 172.21.15.40:0/3394629469 wait complete. 2023-10-18T08:59:51.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.158+0000 7fa95b444640 1 Processor -- start 2023-10-18T08:59:51.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.158+0000 7fa95b444640 1 -- start start 2023-10-18T08:59:51.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.160+0000 7fa95b444640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9400039c0 0x7fa9541f1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:51.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.160+0000 7fa95b444640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9540af130 0x7fa9540cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:51.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.160+0000 7fa95b444640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9540ca600 0x7fa9540caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:51.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.160+0000 7fa95b444640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9540ce600 con 0x7fa9540af130 2023-10-18T08:59:51.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.160+0000 7fa95b444640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9540cafc0 con 0x7fa9540ca600 2023-10-18T08:59:51.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.160+0000 7fa95b444640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9540cea20 con 0x7fa9400039c0 2023-10-18T08:59:51.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.160+0000 7fa9589df640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9540af130 0x7fa9540cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:51.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.160+0000 7fa9589df640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9540af130 0x7fa9540cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34952/0 (socket says 172.21.15.40:34952) 2023-10-18T08:59:51.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.160+0000 7fa9589df640 1 -- 172.21.15.40:0/788689804 learned_addr learned my addr 172.21.15.40:0/788689804 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:51.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.160+0000 7fa9591e0640 1 --2- 172.21.15.40:0/788689804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9400039c0 0x7fa9541f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:51.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.161+0000 7fa9599e1640 1 --2- 172.21.15.40:0/788689804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9540ca600 0x7fa9540caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:51.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.161+0000 7fa9589df640 1 -- 172.21.15.40:0/788689804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9400039c0 msgr2=0x7fa9541f1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:51.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.161+0000 7fa9589df640 1 --2- 172.21.15.40:0/788689804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9400039c0 0x7fa9541f1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.161+0000 7fa9589df640 1 -- 172.21.15.40:0/788689804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9540ca600 msgr2=0x7fa9540caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:51.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.161+0000 7fa9589df640 1 --2- 172.21.15.40:0/788689804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9540ca600 0x7fa9540caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.161+0000 7fa9589df640 1 -- 172.21.15.40:0/788689804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9540cd980 con 0x7fa9540af130 2023-10-18T08:59:51.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.161+0000 7fa9589df640 1 --2- 172.21.15.40:0/788689804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9540af130 0x7fa9540cd080 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa94400c530 tx=0x7fa94400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:51.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.161+0000 7fa949ffb640 1 -- 172.21.15.40:0/788689804 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9440052d0 con 0x7fa9540af130 2023-10-18T08:59:51.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.161+0000 7fa949ffb640 1 -- 172.21.15.40:0/788689804 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa944005430 con 0x7fa9540af130 2023-10-18T08:59:51.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.161+0000 7fa949ffb640 1 -- 172.21.15.40:0/788689804 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944012890 con 0x7fa9540af130 2023-10-18T08:59:51.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.161+0000 7fa95b444640 1 -- 172.21.15.40:0/788689804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9540cdc60 con 0x7fa9540af130 2023-10-18T08:59:51.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.162+0000 7fa95b444640 1 -- 172.21.15.40:0/788689804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9540ce220 con 0x7fa9540af130 2023-10-18T08:59:51.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.163+0000 7fa95b444640 1 -- 172.21.15.40:0/788689804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9540ce3e0 con 0x7fa9540af130 2023-10-18T08:59:51.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.164+0000 7fa949ffb640 1 -- 172.21.15.40:0/788689804 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa9440129f0 con 0x7fa9540af130 2023-10-18T08:59:51.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.165+0000 7fa949ffb640 1 --2- 172.21.15.40:0/788689804 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa93806ae40 0x7fa93806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:51.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.165+0000 7fa949ffb640 1 -- 172.21.15.40:0/788689804 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa944089380 con 0x7fa9540af130 2023-10-18T08:59:51.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.165+0000 7fa949ffb640 1 -- 172.21.15.40:0/788689804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7fa938072970 con 0x7fa9540af130 2023-10-18T08:59:51.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.165+0000 7fa9591e0640 1 --2- 172.21.15.40:0/788689804 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa93806ae40 0x7fa93806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:51.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.167+0000 7fa9591e0640 1 --2- 172.21.15.40:0/788689804 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa93806ae40 0x7fa93806d300 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa950034000 tx=0x7fa95002e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:51.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.170+0000 7fa949ffb640 1 -- 172.21.15.40:0/788689804 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa944057eb0 con 0x7fa9540af130 2023-10-18T08:59:51.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.361+0000 7fa95b444640 1 -- 172.21.15.40:0/788689804 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9540663a0 con 0x7fa9540af130 2023-10-18T08:59:51.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.364+0000 7fa949ffb640 1 -- 172.21.15.40:0/788689804 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16635 (secure 0 0 0) 0x7fa944057a40 con 0x7fa9540af130 2023-10-18T08:59:51.367 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:59:51.368 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":72,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:58:58.028245+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:58:26.727620+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-18T08:58:27.795303+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:59:51.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.366+0000 7fa95b444640 1 -- 172.21.15.40:0/788689804 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa93806ae40 msgr2=0x7fa93806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:51.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.366+0000 7fa95b444640 1 --2- 172.21.15.40:0/788689804 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa93806ae40 0x7fa93806d300 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa950034000 tx=0x7fa95002e000 comp rx=0 tx=0).stop 2023-10-18T08:59:51.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.366+0000 7fa95b444640 1 -- 172.21.15.40:0/788689804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9540af130 msgr2=0x7fa9540cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:51.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.366+0000 7fa95b444640 1 --2- 172.21.15.40:0/788689804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9540af130 0x7fa9540cd080 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa94400c530 tx=0x7fa94400c9f0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.366+0000 7fa95b444640 1 -- 172.21.15.40:0/788689804 shutdown_connections 2023-10-18T08:59:51.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.366+0000 7fa95b444640 1 --2- 172.21.15.40:0/788689804 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa93806ae40 0x7fa93806d300 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.366+0000 7fa95b444640 1 --2- 172.21.15.40:0/788689804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9540ca600 0x7fa9540caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.366+0000 7fa95b444640 1 --2- 172.21.15.40:0/788689804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9540af130 0x7fa9540cd080 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.366+0000 7fa95b444640 1 --2- 172.21.15.40:0/788689804 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa9400039c0 0x7fa9541f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.366+0000 7fa95b444640 1 -- 172.21.15.40:0/788689804 >> 172.21.15.40:0/788689804 conn(0x7fa9540655f0 msgr2=0x7fa95415a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:51.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.366+0000 7fa95b444640 1 -- 172.21.15.40:0/788689804 shutdown_connections 2023-10-18T08:59:51.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.366+0000 7fa95b444640 1 -- 172.21.15.40:0/788689804 wait complete. 2023-10-18T08:59:51.390 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:58:26.727620+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:58:27.795303+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:59:51.391 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:59:51.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.623+0000 7fe926a39640 1 -- 172.21.15.40:0/331016597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe920155cd0 msgr2=0x7fe9200cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:51.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.623+0000 7fe926a39640 1 --2- 172.21.15.40:0/331016597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe920155cd0 0x7fe9200cc1f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe910009860 tx=0x7fe91002bd40 comp rx=0 tx=0).stop 2023-10-18T08:59:51.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.623+0000 7fe926a39640 1 -- 172.21.15.40:0/331016597 shutdown_connections 2023-10-18T08:59:51.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.623+0000 7fe926a39640 1 --2- 172.21.15.40:0/331016597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9200aed00 0x7fe9200af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.623+0000 7fe926a39640 1 --2- 172.21.15.40:0/331016597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe920155cd0 0x7fe9200cc1f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.623+0000 7fe926a39640 1 -- 172.21.15.40:0/331016597 >> 172.21.15.40:0/331016597 conn(0x7fe920069720 msgr2=0x7fe920069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:51.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.623+0000 7fe926a39640 1 -- 172.21.15.40:0/331016597 shutdown_connections 2023-10-18T08:59:51.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.623+0000 7fe926a39640 1 -- 172.21.15.40:0/331016597 wait complete. 2023-10-18T08:59:51.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.625+0000 7fe926a39640 1 Processor -- start 2023-10-18T08:59:51.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.625+0000 7fe926a39640 1 -- start start 2023-10-18T08:59:51.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.626+0000 7fe926a39640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9200ac7b0 0x7fe920152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:51.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.626+0000 7fe926a39640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9200aed00 0x7fe9201562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:51.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.626+0000 7fe926a39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe920155cd0 0x7fe9200acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:51.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.626+0000 7fe926a39640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9201f1ef0 con 0x7fe920155cd0 2023-10-18T08:59:51.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.626+0000 7fe926a39640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe920156bc0 con 0x7fe9200aed00 2023-10-18T08:59:51.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.626+0000 7fe926a39640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe920156ea0 con 0x7fe9200ac7b0 2023-10-18T08:59:51.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.626+0000 7fe924fd6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe920155cd0 0x7fe9200acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:51.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.627+0000 7fe924fd6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe920155cd0 0x7fe9200acc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34956/0 (socket says 172.21.15.40:34956) 2023-10-18T08:59:51.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.627+0000 7fe924fd6640 1 -- 172.21.15.40:0/1444146572 learned_addr learned my addr 172.21.15.40:0/1444146572 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:51.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.627+0000 7fe91f7fe640 1 --2- 172.21.15.40:0/1444146572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9200aed00 0x7fe9201562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:51.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.627+0000 7fe924fd6640 1 -- 172.21.15.40:0/1444146572 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9200ac7b0 msgr2=0x7fe920152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:51.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.627+0000 7fe924fd6640 1 --2- 172.21.15.40:0/1444146572 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9200ac7b0 0x7fe920152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.627+0000 7fe924fd6640 1 -- 172.21.15.40:0/1444146572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9200aed00 msgr2=0x7fe9201562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:51.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.627+0000 7fe924fd6640 1 --2- 172.21.15.40:0/1444146572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9200aed00 0x7fe9201562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.627+0000 7fe924fd6640 1 -- 172.21.15.40:0/1444146572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe910009490 con 0x7fe920155cd0 2023-10-18T08:59:51.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.627+0000 7fe91f7fe640 1 --2- 172.21.15.40:0/1444146572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9200aed00 0x7fe9201562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:59:51.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.627+0000 7fe924fd6640 1 --2- 172.21.15.40:0/1444146572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe920155cd0 0x7fe9200acc90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe91800cce0 tx=0x7fe91800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:51.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.628+0000 7fe91cff9640 1 -- 172.21.15.40:0/1444146572 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91800f450 con 0x7fe920155cd0 2023-10-18T08:59:51.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.628+0000 7fe91cff9640 1 -- 172.21.15.40:0/1444146572 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe91800f6c0 con 0x7fe920155cd0 2023-10-18T08:59:51.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.628+0000 7fe91cff9640 1 -- 172.21.15.40:0/1444146572 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91800fad0 con 0x7fe920155cd0 2023-10-18T08:59:51.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.628+0000 7fe926a39640 1 -- 172.21.15.40:0/1444146572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe920157180 con 0x7fe920155cd0 2023-10-18T08:59:51.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.628+0000 7fe926a39640 1 -- 172.21.15.40:0/1444146572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe920153420 con 0x7fe920155cd0 2023-10-18T08:59:51.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.629+0000 7fe926a39640 1 -- 172.21.15.40:0/1444146572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9200cacb0 con 0x7fe920155cd0 2023-10-18T08:59:51.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.630+0000 7fe91cff9640 1 -- 172.21.15.40:0/1444146572 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe91800fc30 con 0x7fe920155cd0 2023-10-18T08:59:51.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.631+0000 7fe91cff9640 1 --2- 172.21.15.40:0/1444146572 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe90c06ae40 0x7fe90c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:51.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.631+0000 7fe91cff9640 1 -- 172.21.15.40:0/1444146572 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe91808a2a0 con 0x7fe920155cd0 2023-10-18T08:59:51.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.631+0000 7fe91cff9640 1 -- 172.21.15.40:0/1444146572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7fe90c072970 con 0x7fe920155cd0 2023-10-18T08:59:51.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.632+0000 7fe91ffff640 1 --2- 172.21.15.40:0/1444146572 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe90c06ae40 0x7fe90c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:51.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.633+0000 7fe91ffff640 1 --2- 172.21.15.40:0/1444146572 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe90c06ae40 0x7fe90c06d300 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe910036000 tx=0x7fe9100046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:51.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.636+0000 7fe91cff9640 1 -- 172.21.15.40:0/1444146572 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe9180552b0 con 0x7fe920155cd0 2023-10-18T08:59:51.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.832+0000 7fe926a39640 1 -- 172.21.15.40:0/1444146572 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe920066fd0 con 0x7fe920155cd0 2023-10-18T08:59:51.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.833+0000 7fe91cff9640 1 -- 172.21.15.40:0/1444146572 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe91801d070 con 0x7fe920155cd0 2023-10-18T08:59:51.836 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:59:51.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.836+0000 7fe926a39640 1 -- 172.21.15.40:0/1444146572 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe90c06ae40 msgr2=0x7fe90c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:51.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.836+0000 7fe926a39640 1 --2- 172.21.15.40:0/1444146572 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe90c06ae40 0x7fe90c06d300 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe910036000 tx=0x7fe9100046c0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.836+0000 7fe926a39640 1 -- 172.21.15.40:0/1444146572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe920155cd0 msgr2=0x7fe9200acc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:51.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.836+0000 7fe926a39640 1 --2- 172.21.15.40:0/1444146572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe920155cd0 0x7fe9200acc90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe91800cce0 tx=0x7fe91800a620 comp rx=0 tx=0).stop 2023-10-18T08:59:51.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.836+0000 7fe926a39640 1 -- 172.21.15.40:0/1444146572 shutdown_connections 2023-10-18T08:59:51.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.836+0000 7fe926a39640 1 --2- 172.21.15.40:0/1444146572 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe90c06ae40 0x7fe90c06d300 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.836+0000 7fe926a39640 1 --2- 172.21.15.40:0/1444146572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe920155cd0 0x7fe9200acc90 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.836+0000 7fe926a39640 1 --2- 172.21.15.40:0/1444146572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9200aed00 0x7fe9201562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.836+0000 7fe926a39640 1 --2- 172.21.15.40:0/1444146572 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9200ac7b0 0x7fe920152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:51.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.836+0000 7fe926a39640 1 -- 172.21.15.40:0/1444146572 >> 172.21.15.40:0/1444146572 conn(0x7fe920069720 msgr2=0x7fe9200cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:51.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.836+0000 7fe926a39640 1 -- 172.21.15.40:0/1444146572 shutdown_connections 2023-10-18T08:59:51.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:51.836+0000 7fe926a39640 1 -- 172.21.15.40:0/1444146572 wait complete. 2023-10-18T08:59:51.859 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:59:52.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.091+0000 7f0f7c835640 1 -- 172.21.15.40:0/3365364212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f7815eea0 msgr2=0x7f0f780b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:52.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.091+0000 7f0f7c835640 1 --2- 172.21.15.40:0/3365364212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f7815eea0 0x7f0f780b0be0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0f7808a740 tx=0x7f0f6002bd90 comp rx=0 tx=0).stop 2023-10-18T08:59:52.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.092+0000 7f0f7c835640 1 -- 172.21.15.40:0/3365364212 shutdown_connections 2023-10-18T08:59:52.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.092+0000 7f0f7c835640 1 --2- 172.21.15.40:0/3365364212 >> v2:172.21.15.144:3300/0 conn(0x7f0f780d19b0 0x7f0f780ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.092+0000 7f0f7c835640 1 --2- 172.21.15.40:0/3365364212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f7815eea0 0x7f0f780b0be0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.092+0000 7f0f7c835640 1 --2- 172.21.15.40:0/3365364212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f7815eac0 0x7f0f780cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.092+0000 7f0f7c835640 1 -- 172.21.15.40:0/3365364212 >> 172.21.15.40:0/3365364212 conn(0x7f0f7809a610 msgr2=0x7f0f780d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:52.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.092+0000 7f0f7c835640 1 -- 172.21.15.40:0/3365364212 shutdown_connections 2023-10-18T08:59:52.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.092+0000 7f0f7c835640 1 -- 172.21.15.40:0/3365364212 wait complete. 2023-10-18T08:59:52.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.093+0000 7f0f7c835640 1 Processor -- start 2023-10-18T08:59:52.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.094+0000 7f0f7c835640 1 -- start start 2023-10-18T08:59:52.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.094+0000 7f0f7c835640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f780d19b0 0x7f0f781f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:52.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.094+0000 7f0f7c835640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f7815eac0 0x7f0f781f3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:52.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.094+0000 7f0f7c835640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f7815eea0 0x7f0f780c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:52.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.095+0000 7f0f7c835640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f780b51e0 con 0x7f0f780d19b0 2023-10-18T08:59:52.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.095+0000 7f0f7c835640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f781f2e00 con 0x7f0f7815eea0 2023-10-18T08:59:52.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.095+0000 7f0f7c835640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f780b5760 con 0x7f0f7815eac0 2023-10-18T08:59:52.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.095+0000 7f0f777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f780d19b0 0x7f0f781f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:52.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.095+0000 7f0f777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f780d19b0 0x7f0f781f1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34970/0 (socket says 172.21.15.40:34970) 2023-10-18T08:59:52.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.095+0000 7f0f777fe640 1 -- 172.21.15.40:0/334665762 learned_addr learned my addr 172.21.15.40:0/334665762 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:52.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.095+0000 7f0f76ffd640 1 --2- 172.21.15.40:0/334665762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f7815eac0 0x7f0f781f3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:52.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.095+0000 7f0f77fff640 1 --2- 172.21.15.40:0/334665762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f7815eea0 0x7f0f780c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:52.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.096+0000 7f0f777fe640 1 -- 172.21.15.40:0/334665762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f7815eac0 msgr2=0x7f0f781f3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:52.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.096+0000 7f0f777fe640 1 --2- 172.21.15.40:0/334665762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f7815eac0 0x7f0f781f3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.096+0000 7f0f777fe640 1 -- 172.21.15.40:0/334665762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f7815eea0 msgr2=0x7f0f780c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.096+0000 7f0f777fe640 1 --2- 172.21.15.40:0/334665762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f7815eea0 0x7f0f780c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.096+0000 7f0f777fe640 1 -- 172.21.15.40:0/334665762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f60009560 con 0x7f0f780d19b0 2023-10-18T08:59:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.096+0000 7f0f77fff640 1 --2- 172.21.15.40:0/334665762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f7815eea0 0x7f0f780c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:59:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.096+0000 7f0f777fe640 1 --2- 172.21.15.40:0/334665762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f780d19b0 0x7f0f781f1ef0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800cce0 tx=0x7f0f6800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.096+0000 7f0f6ffff640 1 -- 172.21.15.40:0/334665762 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68004b20 con 0x7f0f780d19b0 2023-10-18T08:59:52.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.096+0000 7f0f6ffff640 1 -- 172.21.15.40:0/334665762 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0f68004d70 con 0x7f0f780d19b0 2023-10-18T08:59:52.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.096+0000 7f0f6ffff640 1 -- 172.21.15.40:0/334665762 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68005280 con 0x7f0f780d19b0 2023-10-18T08:59:52.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.096+0000 7f0f7c835640 1 -- 172.21.15.40:0/334665762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f781f2840 con 0x7f0f780d19b0 2023-10-18T08:59:52.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.096+0000 7f0f7c835640 1 -- 172.21.15.40:0/334665762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f780b53f0 con 0x7f0f780d19b0 2023-10-18T08:59:52.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.097+0000 7f0f7c835640 1 -- 172.21.15.40:0/334665762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f78071da0 con 0x7f0f780d19b0 2023-10-18T08:59:52.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.099+0000 7f0f6ffff640 1 -- 172.21.15.40:0/334665762 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0f680053e0 con 0x7f0f780d19b0 2023-10-18T08:59:52.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.100+0000 7f0f6ffff640 1 --2- 172.21.15.40:0/334665762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f5806ae70 0x7f0f5806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:52.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.100+0000 7f0f6ffff640 1 -- 172.21.15.40:0/334665762 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f68089ed0 con 0x7f0f780d19b0 2023-10-18T08:59:52.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.101+0000 7f0f6ffff640 1 -- 172.21.15.40:0/334665762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7f0f580729a0 con 0x7f0f780d19b0 2023-10-18T08:59:52.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.101+0000 7f0f76ffd640 1 --2- 172.21.15.40:0/334665762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f5806ae70 0x7f0f5806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:52.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.102+0000 7f0f76ffd640 1 --2- 172.21.15.40:0/334665762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f5806ae70 0x7f0f5806d330 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0f60035000 tx=0x7f0f60004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:52.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.105+0000 7f0f6ffff640 1 -- 172.21.15.40:0/334665762 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0f6801d070 con 0x7f0f780d19b0 2023-10-18T08:59:52.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.299+0000 7f0f7c835640 1 -- 172.21.15.40:0/334665762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0f78066e00 con 0x7f0f780d19b0 2023-10-18T08:59:52.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.300+0000 7f0f6ffff640 1 -- 172.21.15.40:0/334665762 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0f68054f90 con 0x7f0f780d19b0 2023-10-18T08:59:52.304 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:59:52.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.303+0000 7f0f7c835640 1 -- 172.21.15.40:0/334665762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f5806ae70 msgr2=0x7f0f5806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:52.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.303+0000 7f0f7c835640 1 --2- 172.21.15.40:0/334665762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f5806ae70 0x7f0f5806d330 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0f60035000 tx=0x7f0f60004730 comp rx=0 tx=0).stop 2023-10-18T08:59:52.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.303+0000 7f0f7c835640 1 -- 172.21.15.40:0/334665762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f780d19b0 msgr2=0x7f0f781f1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:52.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.303+0000 7f0f7c835640 1 --2- 172.21.15.40:0/334665762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f780d19b0 0x7f0f781f1ef0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800cce0 tx=0x7f0f6800a620 comp rx=0 tx=0).stop 2023-10-18T08:59:52.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.303+0000 7f0f7c835640 1 -- 172.21.15.40:0/334665762 shutdown_connections 2023-10-18T08:59:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.303+0000 7f0f7c835640 1 --2- 172.21.15.40:0/334665762 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f5806ae70 0x7f0f5806d330 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.303+0000 7f0f7c835640 1 --2- 172.21.15.40:0/334665762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f7815eea0 0x7f0f780c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.303+0000 7f0f7c835640 1 --2- 172.21.15.40:0/334665762 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f7815eac0 0x7f0f781f3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.303+0000 7f0f7c835640 1 --2- 172.21.15.40:0/334665762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f780d19b0 0x7f0f781f1ef0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.303+0000 7f0f7c835640 1 -- 172.21.15.40:0/334665762 >> 172.21.15.40:0/334665762 conn(0x7f0f7809a610 msgr2=0x7f0f780d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:52.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.303+0000 7f0f7c835640 1 -- 172.21.15.40:0/334665762 shutdown_connections 2023-10-18T08:59:52.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.303+0000 7f0f7c835640 1 -- 172.21.15.40:0/334665762 wait complete. 2023-10-18T08:59:52.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:59:52.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.551+0000 7fafe290f640 1 -- 172.21.15.40:0/2657098022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafdc06e420 msgr2=0x7fafdc06e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:52.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.551+0000 7fafe290f640 1 --2- 172.21.15.40:0/2657098022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafdc06e420 0x7fafdc06e8b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fafd4005fd0 tx=0x7fafd402c1b0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.551+0000 7fafe290f640 1 -- 172.21.15.40:0/2657098022 shutdown_connections 2023-10-18T08:59:52.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.551+0000 7fafe290f640 1 --2- 172.21.15.40:0/2657098022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafdc06e420 0x7fafdc06e8b0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.551+0000 7fafe290f640 1 --2- 172.21.15.40:0/2657098022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafdc0cfc50 0x7fafdc0ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.551+0000 7fafe290f640 1 -- 172.21.15.40:0/2657098022 >> 172.21.15.40:0/2657098022 conn(0x7fafdc088fa0 msgr2=0x7fafdc0cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.551+0000 7fafe290f640 1 -- 172.21.15.40:0/2657098022 shutdown_connections 2023-10-18T08:59:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.551+0000 7fafe290f640 1 -- 172.21.15.40:0/2657098022 wait complete. 2023-10-18T08:59:52.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.553+0000 7fafe290f640 1 Processor -- start 2023-10-18T08:59:52.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.553+0000 7fafe290f640 1 -- start start 2023-10-18T08:59:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.554+0000 7fafe290f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafdc06e420 0x7fafdc14f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.554+0000 7fafe290f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafdc0cfc50 0x7fafdc0c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.554+0000 7fafe290f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafdc15f110 0x7fafdc0c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.554+0000 7fafe290f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc14f1b0 con 0x7fafdc15f110 2023-10-18T08:59:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.554+0000 7fafe290f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0c91d0 con 0x7fafdc06e420 2023-10-18T08:59:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.554+0000 7fafe290f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc14af30 con 0x7fafdc0cfc50 2023-10-18T08:59:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.555+0000 7fafe0eac640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafdc15f110 0x7fafdc0c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.555+0000 7fafe0eac640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafdc15f110 0x7fafdc0c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34988/0 (socket says 172.21.15.40:34988) 2023-10-18T08:59:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.555+0000 7fafe0eac640 1 -- 172.21.15.40:0/3137564373 learned_addr learned my addr 172.21.15.40:0/3137564373 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.555+0000 7fafdb7fe640 1 --2- 172.21.15.40:0/3137564373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafdc0cfc50 0x7fafdc0c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.555+0000 7fafe0eac640 1 -- 172.21.15.40:0/3137564373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafdc0cfc50 msgr2=0x7fafdc0c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.555+0000 7fafe0eac640 1 --2- 172.21.15.40:0/3137564373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafdc0cfc50 0x7fafdc0c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.555+0000 7fafdbfff640 1 --2- 172.21.15.40:0/3137564373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafdc06e420 0x7fafdc14f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.555+0000 7fafe0eac640 1 -- 172.21.15.40:0/3137564373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafdc06e420 msgr2=0x7fafdc14f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.555+0000 7fafe0eac640 1 --2- 172.21.15.40:0/3137564373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafdc06e420 0x7fafdc14f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.555+0000 7fafe0eac640 1 -- 172.21.15.40:0/3137564373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd4009900 con 0x7fafdc15f110 2023-10-18T08:59:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.555+0000 7fafdbfff640 1 --2- 172.21.15.40:0/3137564373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafdc06e420 0x7fafdc14f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:59:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.556+0000 7fafe0eac640 1 --2- 172.21.15.40:0/3137564373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafdc15f110 0x7fafdc0c4fe0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009bf0 tx=0x7fafd4035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.556+0000 7fafd8ff9640 1 -- 172.21.15.40:0/3137564373 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4034440 con 0x7fafdc15f110 2023-10-18T08:59:52.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.556+0000 7fafd8ff9640 1 -- 172.21.15.40:0/3137564373 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fafd4004540 con 0x7fafdc15f110 2023-10-18T08:59:52.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.556+0000 7fafd8ff9640 1 -- 172.21.15.40:0/3137564373 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4040450 con 0x7fafdc15f110 2023-10-18T08:59:52.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.556+0000 7fafe290f640 1 -- 172.21.15.40:0/3137564373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafdc06dfb0 con 0x7fafdc15f110 2023-10-18T08:59:52.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.556+0000 7fafe290f640 1 -- 172.21.15.40:0/3137564373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafdc14eaf0 con 0x7fafdc15f110 2023-10-18T08:59:52.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.557+0000 7fafe290f640 1 -- 172.21.15.40:0/3137564373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafdc066fd0 con 0x7fafdc15f110 2023-10-18T08:59:52.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.558+0000 7fafd8ff9640 1 -- 172.21.15.40:0/3137564373 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fafd400a040 con 0x7fafdc15f110 2023-10-18T08:59:52.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.559+0000 7fafd8ff9640 1 --2- 172.21.15.40:0/3137564373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fafc806ae40 0x7fafc806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:52.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.559+0000 7fafd8ff9640 1 -- 172.21.15.40:0/3137564373 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fafd40ac700 con 0x7fafdc15f110 2023-10-18T08:59:52.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.559+0000 7fafd8ff9640 1 -- 172.21.15.40:0/3137564373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7fafc8072970 con 0x7fafdc15f110 2023-10-18T08:59:52.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.560+0000 7fafdbfff640 1 --2- 172.21.15.40:0/3137564373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fafc806ae40 0x7fafc806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:52.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.561+0000 7fafdbfff640 1 --2- 172.21.15.40:0/3137564373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fafc806ae40 0x7fafc806d300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00a830 tx=0x7fafcc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:52.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.563+0000 7fafd8ff9640 1 -- 172.21.15.40:0/3137564373 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fafd4039060 con 0x7fafdc15f110 2023-10-18T08:59:52.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.764+0000 7fafe290f640 1 -- 172.21.15.40:0/3137564373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fafdc0cc970 con 0x7fafdc15f110 2023-10-18T08:59:52.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.765+0000 7fafd8ff9640 1 -- 172.21.15.40:0/3137564373 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) v1 ==== 100+0+11 (secure 0 0 0) 0x7fafd4044020 con 0x7fafdc15f110 2023-10-18T08:59:52.768 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:59:52.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.767+0000 7fafe290f640 1 -- 172.21.15.40:0/3137564373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fafc806ae40 msgr2=0x7fafc806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:52.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.767+0000 7fafe290f640 1 --2- 172.21.15.40:0/3137564373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fafc806ae40 0x7fafc806d300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00a830 tx=0x7fafcc008040 comp rx=0 tx=0).stop 2023-10-18T08:59:52.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.767+0000 7fafe290f640 1 -- 172.21.15.40:0/3137564373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafdc15f110 msgr2=0x7fafdc0c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:52.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.767+0000 7fafe290f640 1 --2- 172.21.15.40:0/3137564373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafdc15f110 0x7fafdc0c4fe0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009bf0 tx=0x7fafd4035e30 comp rx=0 tx=0).stop 2023-10-18T08:59:52.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.768+0000 7fafe290f640 1 -- 172.21.15.40:0/3137564373 shutdown_connections 2023-10-18T08:59:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.768+0000 7fafe290f640 1 --2- 172.21.15.40:0/3137564373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fafc806ae40 0x7fafc806d300 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.768+0000 7fafe290f640 1 --2- 172.21.15.40:0/3137564373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafdc15f110 0x7fafdc0c4fe0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.768+0000 7fafe290f640 1 --2- 172.21.15.40:0/3137564373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fafdc0cfc50 0x7fafdc0c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.768+0000 7fafe290f640 1 --2- 172.21.15.40:0/3137564373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fafdc06e420 0x7fafdc14f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.768+0000 7fafe290f640 1 -- 172.21.15.40:0/3137564373 >> 172.21.15.40:0/3137564373 conn(0x7fafdc088fa0 msgr2=0x7fafdc0ac040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.768+0000 7fafe290f640 1 -- 172.21.15.40:0/3137564373 shutdown_connections 2023-10-18T08:59:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:52.768+0000 7fafe290f640 1 -- 172.21.15.40:0/3137564373 wait complete. 2023-10-18T08:59:52.789 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T08:59:53.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.023+0000 7f6b23700640 1 -- 172.21.15.40:0/3212266358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b08003970 msgr2=0x7f6b08003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:53.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.023+0000 7f6b23700640 1 --2- 172.21.15.40:0/3212266358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b08003970 0x7f6b08003d90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6b18006f10 tx=0x7f6b080057f0 comp rx=0 tx=0).stop 2023-10-18T08:59:53.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.023+0000 7f6b21c9d640 1 -- 172.21.15.40:0/3212266358 reap_dead start 2023-10-18T08:59:53.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.023+0000 7f6b23700640 1 -- 172.21.15.40:0/3212266358 shutdown_connections 2023-10-18T08:59:53.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.023+0000 7f6b23700640 1 -- 172.21.15.40:0/3212266358 >> 172.21.15.40:0/3212266358 conn(0x7f6b1c06cee0 msgr2=0x7f6b1c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:53.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.023+0000 7f6b23700640 1 -- 172.21.15.40:0/3212266358 shutdown_connections 2023-10-18T08:59:53.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.023+0000 7f6b23700640 1 -- 172.21.15.40:0/3212266358 wait complete. 2023-10-18T08:59:53.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.025+0000 7f6b23700640 1 Processor -- start 2023-10-18T08:59:53.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.025+0000 7f6b23700640 1 -- start start 2023-10-18T08:59:53.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.026+0000 7f6b23700640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b08003970 0x7f6b1c145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:53.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.026+0000 7f6b23700640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b1c1e6840 0x7f6b1c14a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:53.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.026+0000 7f6b23700640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b1c1e6cc0 0x7f6b1c14aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:53.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.026+0000 7f6b23700640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1e63b0 con 0x7f6b1c1e6cc0 2023-10-18T08:59:53.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.026+0000 7f6b23700640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1e57a0 con 0x7f6b08003970 2023-10-18T08:59:53.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.026+0000 7f6b23700640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1e5a80 con 0x7f6b1c1e6840 2023-10-18T08:59:53.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.027+0000 7f6b21c9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b1c1e6cc0 0x7f6b1c14aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:53.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.027+0000 7f6b21c9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b1c1e6cc0 0x7f6b1c14aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35014/0 (socket says 172.21.15.40:35014) 2023-10-18T08:59:53.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.027+0000 7f6b21c9d640 1 -- 172.21.15.40:0/2937786351 learned_addr learned my addr 172.21.15.40:0/2937786351 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:53.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.027+0000 7f6b2149c640 1 --2- 172.21.15.40:0/2937786351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b08003970 0x7f6b1c145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:53.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.027+0000 7f6b21c9d640 1 -- 172.21.15.40:0/2937786351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b1c1e6840 msgr2=0x7f6b1c14a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:53.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.027+0000 7f6b21c9d640 1 --2- 172.21.15.40:0/2937786351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b1c1e6840 0x7f6b1c14a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:53.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.027+0000 7f6b21c9d640 1 -- 172.21.15.40:0/2937786351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b08003970 msgr2=0x7f6b1c145c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:53.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.027+0000 7f6b21c9d640 1 --2- 172.21.15.40:0/2937786351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b08003970 0x7f6b1c145c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:53.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.027+0000 7f6b21c9d640 1 -- 172.21.15.40:0/2937786351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c15c1d0 con 0x7f6b1c1e6cc0 2023-10-18T08:59:53.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.027+0000 7f6b2149c640 1 --2- 172.21.15.40:0/2937786351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b08003970 0x7f6b1c145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:59:53.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.028+0000 7f6b21c9d640 1 --2- 172.21.15.40:0/2937786351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b1c1e6cc0 0x7f6b1c14aed0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c06a700 tx=0x7f6b1400ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:53.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.028+0000 7f6b11ffb640 1 -- 172.21.15.40:0/2937786351 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14018070 con 0x7f6b1c1e6cc0 2023-10-18T08:59:53.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.028+0000 7f6b11ffb640 1 -- 172.21.15.40:0/2937786351 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6b14004130 con 0x7f6b1c1e6cc0 2023-10-18T08:59:53.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.028+0000 7f6b11ffb640 1 -- 172.21.15.40:0/2937786351 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14005020 con 0x7f6b1c1e6cc0 2023-10-18T08:59:53.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.028+0000 7f6b23700640 1 -- 172.21.15.40:0/2937786351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1c1e5d60 con 0x7f6b1c1e6cc0 2023-10-18T08:59:53.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.028+0000 7f6b23700640 1 -- 172.21.15.40:0/2937786351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1c146150 con 0x7f6b1c1e6cc0 2023-10-18T08:59:53.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.029+0000 7f6b23700640 1 -- 172.21.15.40:0/2937786351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1c1e6520 con 0x7f6b1c1e6cc0 2023-10-18T08:59:53.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.030+0000 7f6b11ffb640 1 -- 172.21.15.40:0/2937786351 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6b14005180 con 0x7f6b1c1e6cc0 2023-10-18T08:59:53.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.031+0000 7f6b11ffb640 1 --2- 172.21.15.40:0/2937786351 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b0006ae40 0x7f6b0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:53.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.031+0000 7f6b11ffb640 1 -- 172.21.15.40:0/2937786351 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b14089100 con 0x7f6b1c1e6cc0 2023-10-18T08:59:53.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.032+0000 7f6b11ffb640 1 -- 172.21.15.40:0/2937786351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7f6b00072970 con 0x7f6b1c1e6cc0 2023-10-18T08:59:53.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.032+0000 7f6b2149c640 1 --2- 172.21.15.40:0/2937786351 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b0006ae40 0x7f6b0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:53.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.033+0000 7f6b2149c640 1 --2- 172.21.15.40:0/2937786351 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b0006ae40 0x7f6b0006d300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6b18034000 tx=0x7f6b18039040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:53.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.037+0000 7f6b11ffb640 1 -- 172.21.15.40:0/2937786351 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b140541b0 con 0x7f6b1c1e6cc0 2023-10-18T08:59:53.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.240+0000 7f6b23700640 1 -- 172.21.15.40:0/2937786351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6b1c066fd0 con 0x7f6b1c1e6cc0 2023-10-18T08:59:53.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.240+0000 7f6b11ffb640 1 -- 172.21.15.40:0/2937786351 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v72) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6b1405e0d0 con 0x7f6b1c1e6cc0 2023-10-18T08:59:53.244 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T08:59:53.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.243+0000 7f6b23700640 1 -- 172.21.15.40:0/2937786351 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b0006ae40 msgr2=0x7f6b0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.243+0000 7f6b23700640 1 --2- 172.21.15.40:0/2937786351 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b0006ae40 0x7f6b0006d300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6b18034000 tx=0x7f6b18039040 comp rx=0 tx=0).stop 2023-10-18T08:59:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.243+0000 7f6b23700640 1 -- 172.21.15.40:0/2937786351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b1c1e6cc0 msgr2=0x7f6b1c14aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.243+0000 7f6b23700640 1 --2- 172.21.15.40:0/2937786351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b1c1e6cc0 0x7f6b1c14aed0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c06a700 tx=0x7f6b1400ea60 comp rx=0 tx=0).stop 2023-10-18T08:59:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.243+0000 7f6b23700640 1 -- 172.21.15.40:0/2937786351 shutdown_connections 2023-10-18T08:59:53.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.243+0000 7f6b23700640 1 --2- 172.21.15.40:0/2937786351 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b0006ae40 0x7f6b0006d300 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.243+0000 7f6b23700640 1 --2- 172.21.15.40:0/2937786351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b1c1e6cc0 0x7f6b1c14aed0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.243+0000 7f6b23700640 1 --2- 172.21.15.40:0/2937786351 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b1c1e6840 0x7f6b1c14a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.243+0000 7f6b23700640 1 --2- 172.21.15.40:0/2937786351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b08003970 0x7f6b1c145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.243+0000 7f6b23700640 1 -- 172.21.15.40:0/2937786351 >> 172.21.15.40:0/2937786351 conn(0x7f6b1c06cee0 msgr2=0x7f6b1c150a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.243+0000 7f6b23700640 1 -- 172.21.15.40:0/2937786351 shutdown_connections 2023-10-18T08:59:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.243+0000 7f6b23700640 1 -- 172.21.15.40:0/2937786351 wait complete. 2023-10-18T08:59:53.266 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-18T08:59:53.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:59:53.267 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-18T08:59:53.384 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T08:59:53.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:59:53.385 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T08:59:53.437 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:59:53.438 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:59:53.505 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:59:53.505 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:59:53.567 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:59:53.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.797+0000 7f7215064640 1 -- 172.21.15.40:0/702283709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f721015eaa0 msgr2=0x7f72100ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:53.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.798+0000 7f7215064640 1 --2- 172.21.15.40:0/702283709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f721015eaa0 0x7f72100ccdb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7200009830 tx=0x7f720002bd60 comp rx=0 tx=0).stop 2023-10-18T08:59:53.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.798+0000 7f7215064640 1 -- 172.21.15.40:0/702283709 shutdown_connections 2023-10-18T08:59:53.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.798+0000 7f7215064640 1 --2- 172.21.15.40:0/702283709 >> v2:172.21.15.144:3300/0 conn(0x7f72100b1b90 0x7f72100cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:53.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.798+0000 7f7215064640 1 --2- 172.21.15.40:0/702283709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f721015eaa0 0x7f72100ccdb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:53.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.798+0000 7f7215064640 1 -- 172.21.15.40:0/702283709 >> 172.21.15.40:0/702283709 conn(0x7f721006cee0 msgr2=0x7f7210069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:53.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.798+0000 7f7215064640 1 -- 172.21.15.40:0/702283709 shutdown_connections 2023-10-18T08:59:53.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.798+0000 7f7215064640 1 -- 172.21.15.40:0/702283709 wait complete. 2023-10-18T08:59:53.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.799+0000 7f7215064640 1 Processor -- start 2023-10-18T08:59:53.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.799+0000 7f7215064640 1 -- start start 2023-10-18T08:59:53.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.800+0000 7f7215064640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f721006ffe0 0x7f72100ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:53.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.800+0000 7f7215064640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72100b1b90 0x7f7210142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:53.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.800+0000 7f7215064640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f721015eaa0 0x7f72101426c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:53.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.800+0000 7f7215064640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72101410e0 con 0x7f721006ffe0 2023-10-18T08:59:53.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.800+0000 7f7215064640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72101413c0 con 0x7f721015eaa0 2023-10-18T08:59:53.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.800+0000 7f7215064640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72101416a0 con 0x7f72100b1b90 2023-10-18T08:59:53.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.801+0000 7f720ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f721006ffe0 0x7f72100ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:53.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.801+0000 7f720ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f721006ffe0 0x7f72100ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35024/0 (socket says 172.21.15.40:35024) 2023-10-18T08:59:53.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.801+0000 7f720ed9d640 1 -- 172.21.15.40:0/978528065 learned_addr learned my addr 172.21.15.40:0/978528065 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:53.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.801+0000 7f720f59e640 1 --2- 172.21.15.40:0/978528065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f721015eaa0 0x7f72101426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:53.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.801+0000 7f720659c640 1 --2- 172.21.15.40:0/978528065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72100b1b90 0x7f7210142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:53.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.801+0000 7f720ed9d640 1 -- 172.21.15.40:0/978528065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72100b1b90 msgr2=0x7f7210142180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:53.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.801+0000 7f720ed9d640 1 --2- 172.21.15.40:0/978528065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72100b1b90 0x7f7210142180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:53.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.801+0000 7f720ed9d640 1 -- 172.21.15.40:0/978528065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f721015eaa0 msgr2=0x7f72101426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:53.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.801+0000 7f720ed9d640 1 --2- 172.21.15.40:0/978528065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f721015eaa0 0x7f72101426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:53.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.801+0000 7f720ed9d640 1 -- 172.21.15.40:0/978528065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f8009840 con 0x7f721006ffe0 2023-10-18T08:59:53.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.801+0000 7f720f59e640 1 --2- 172.21.15.40:0/978528065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f721015eaa0 0x7f72101426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:59:53.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.802+0000 7f720ed9d640 1 --2- 172.21.15.40:0/978528065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f721006ffe0 0x7f72100ca010 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7200000c00 tx=0x7f72000038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:53.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.802+0000 7f7207fff640 1 -- 172.21.15.40:0/978528065 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200003ea0 con 0x7f721006ffe0 2023-10-18T08:59:53.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.802+0000 7f7207fff640 1 -- 172.21.15.40:0/978528065 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7200034440 con 0x7f721006ffe0 2023-10-18T08:59:53.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.802+0000 7f7215064640 1 -- 172.21.15.40:0/978528065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7200009490 con 0x7f721006ffe0 2023-10-18T08:59:53.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.802+0000 7f7207fff640 1 -- 172.21.15.40:0/978528065 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200034850 con 0x7f721006ffe0 2023-10-18T08:59:53.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.802+0000 7f7215064640 1 -- 172.21.15.40:0/978528065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7210141c60 con 0x7f721006ffe0 2023-10-18T08:59:53.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.803+0000 7f7215064640 1 -- 172.21.15.40:0/978528065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7210066fd0 con 0x7f721006ffe0 2023-10-18T08:59:53.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.804+0000 7f7207fff640 1 -- 172.21.15.40:0/978528065 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f720003b040 con 0x7f721006ffe0 2023-10-18T08:59:53.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.805+0000 7f7207fff640 1 --2- 172.21.15.40:0/978528065 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f71f006aec0 0x7f71f006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:53.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.805+0000 7f7207fff640 1 -- 172.21.15.40:0/978528065 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72000abaa0 con 0x7f721006ffe0 2023-10-18T08:59:53.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.807+0000 7f7207fff640 1 -- 172.21.15.40:0/978528065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7f71f00729f0 con 0x7f721006ffe0 2023-10-18T08:59:53.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.807+0000 7f720659c640 1 --2- 172.21.15.40:0/978528065 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f71f006aec0 0x7f71f006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:53.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.808+0000 7f720659c640 1 --2- 172.21.15.40:0/978528065 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f71f006aec0 0x7f71f006d380 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f72100cdfd0 tx=0x7f71f8009500 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:53.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:53.811+0000 7f7207fff640 1 -- 172.21.15.40:0/978528065 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7200039060 con 0x7f721006ffe0 2023-10-18T08:59:54.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.006+0000 7f7215064640 1 -- 172.21.15.40:0/978528065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72100aa950 con 0x7f721006ffe0 2023-10-18T08:59:54.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.009+0000 7f7207fff640 1 -- 172.21.15.40:0/978528065 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16635 (secure 0 0 0) 0x7f7200041070 con 0x7f721006ffe0 2023-10-18T08:59:54.012 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:59:54.012 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":72,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:58:58.028245+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:58:26.727620+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-18T08:58:27.795303+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:59:54.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.011+0000 7f7215064640 1 -- 172.21.15.40:0/978528065 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f71f006aec0 msgr2=0x7f71f006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:54.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.011+0000 7f7215064640 1 --2- 172.21.15.40:0/978528065 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f71f006aec0 0x7f71f006d380 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f72100cdfd0 tx=0x7f71f8009500 comp rx=0 tx=0).stop 2023-10-18T08:59:54.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.011+0000 7f7215064640 1 -- 172.21.15.40:0/978528065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f721006ffe0 msgr2=0x7f72100ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:54.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.011+0000 7f7215064640 1 --2- 172.21.15.40:0/978528065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f721006ffe0 0x7f72100ca010 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7200000c00 tx=0x7f72000038a0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.012+0000 7f7215064640 1 -- 172.21.15.40:0/978528065 shutdown_connections 2023-10-18T08:59:54.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.012+0000 7f7215064640 1 --2- 172.21.15.40:0/978528065 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f71f006aec0 0x7f71f006d380 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.012+0000 7f7215064640 1 --2- 172.21.15.40:0/978528065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f721015eaa0 0x7f72101426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.012+0000 7f7215064640 1 --2- 172.21.15.40:0/978528065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72100b1b90 0x7f7210142180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.012+0000 7f7215064640 1 --2- 172.21.15.40:0/978528065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f721006ffe0 0x7f72100ca010 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.012+0000 7f7215064640 1 -- 172.21.15.40:0/978528065 >> 172.21.15.40:0/978528065 conn(0x7f721006cee0 msgr2=0x7f721015d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:54.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.012+0000 7f7215064640 1 -- 172.21.15.40:0/978528065 shutdown_connections 2023-10-18T08:59:54.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.012+0000 7f7215064640 1 -- 172.21.15.40:0/978528065 wait complete. 2023-10-18T08:59:54.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:58:26.727620+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:58:27.795303+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:59:54.035 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:59:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.272+0000 7f5396456640 1 -- 172.21.15.40:0/1912297192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f537c004d00 msgr2=0x7f537c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:54.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.272+0000 7f5396456640 1 --2- 172.21.15.40:0/1912297192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f537c004d00 0x7f537c005180 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f539006f3e0 tx=0x7f537c0057f0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.272+0000 7f53949f3640 1 -- 172.21.15.40:0/1912297192 reap_dead start 2023-10-18T08:59:54.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.272+0000 7f5396456640 1 -- 172.21.15.40:0/1912297192 shutdown_connections 2023-10-18T08:59:54.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.272+0000 7f5396456640 1 -- 172.21.15.40:0/1912297192 >> 172.21.15.40:0/1912297192 conn(0x7f5390069720 msgr2=0x7f5390069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:54.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.272+0000 7f5396456640 1 -- 172.21.15.40:0/1912297192 shutdown_connections 2023-10-18T08:59:54.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.272+0000 7f5396456640 1 -- 172.21.15.40:0/1912297192 wait complete. 2023-10-18T08:59:54.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.274+0000 7f5396456640 1 Processor -- start 2023-10-18T08:59:54.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.274+0000 7f5396456640 1 -- start start 2023-10-18T08:59:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.275+0000 7f5396456640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c004d00 0x7f53901486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.275+0000 7f5396456640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539015a230 0x7f53900b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.275+0000 7f5396456640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53900b1430 0x7f5390152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:54.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.275+0000 7f5396456640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53900b18b0 con 0x7f53900b1430 2023-10-18T08:59:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.275+0000 7f5396456640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5390148c10 con 0x7f539015a230 2023-10-18T08:59:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.275+0000 7f5396456640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53901e5410 con 0x7f537c004d00 2023-10-18T08:59:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.276+0000 7f53949f3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53900b1430 0x7f5390152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.276+0000 7f53949f3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53900b1430 0x7f5390152630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35048/0 (socket says 172.21.15.40:35048) 2023-10-18T08:59:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.276+0000 7f53949f3640 1 -- 172.21.15.40:0/3397660098 learned_addr learned my addr 172.21.15.40:0/3397660098 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.276+0000 7f538ffff640 1 --2- 172.21.15.40:0/3397660098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c004d00 0x7f53901486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.276+0000 7f5387fff640 1 --2- 172.21.15.40:0/3397660098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539015a230 0x7f53900b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:54.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.276+0000 7f53949f3640 1 -- 172.21.15.40:0/3397660098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c004d00 msgr2=0x7f53901486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:54.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.276+0000 7f53949f3640 1 --2- 172.21.15.40:0/3397660098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c004d00 0x7f53901486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.276+0000 7f53949f3640 1 -- 172.21.15.40:0/3397660098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539015a230 msgr2=0x7f53900b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.276+0000 7f53949f3640 1 --2- 172.21.15.40:0/3397660098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539015a230 0x7f53900b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.276+0000 7f53949f3640 1 -- 172.21.15.40:0/3397660098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5390151210 con 0x7f53900b1430 2023-10-18T08:59:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.276+0000 7f538ffff640 1 --2- 172.21.15.40:0/3397660098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c004d00 0x7f53901486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:59:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.276+0000 7f5387fff640 1 --2- 172.21.15.40:0/3397660098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539015a230 0x7f53900b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:59:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.277+0000 7f53949f3640 1 --2- 172.21.15.40:0/3397660098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53900b1430 0x7f5390152630 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5388009f90 tx=0x7f5388033f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.277+0000 7f538d7fa640 1 -- 172.21.15.40:0/3397660098 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538803f070 con 0x7f53900b1430 2023-10-18T08:59:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.277+0000 7f538d7fa640 1 -- 172.21.15.40:0/3397660098 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5388031be0 con 0x7f53900b1430 2023-10-18T08:59:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.277+0000 7f538d7fa640 1 -- 172.21.15.40:0/3397660098 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388003970 con 0x7f53900b1430 2023-10-18T08:59:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.277+0000 7f5396456640 1 -- 172.21.15.40:0/3397660098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53901514f0 con 0x7f53900b1430 2023-10-18T08:59:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.277+0000 7f5396456640 1 -- 172.21.15.40:0/3397660098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5390151ab0 con 0x7f53900b1430 2023-10-18T08:59:54.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.278+0000 7f5396456640 1 -- 172.21.15.40:0/3397660098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5390066fd0 con 0x7f53900b1430 2023-10-18T08:59:54.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.279+0000 7f538d7fa640 1 -- 172.21.15.40:0/3397660098 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f538802f070 con 0x7f53900b1430 2023-10-18T08:59:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.280+0000 7f538d7fa640 1 --2- 172.21.15.40:0/3397660098 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f537006ae40 0x7f537006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.280+0000 7f538d7fa640 1 -- 172.21.15.40:0/3397660098 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53880abd10 con 0x7f53900b1430 2023-10-18T08:59:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.281+0000 7f538d7fa640 1 -- 172.21.15.40:0/3397660098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7f5370072970 con 0x7f53900b1430 2023-10-18T08:59:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.281+0000 7f538ffff640 1 --2- 172.21.15.40:0/3397660098 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f537006ae40 0x7f537006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.282+0000 7f538ffff640 1 --2- 172.21.15.40:0/3397660098 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f537006ae40 0x7f537006d300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5380001eb0 tx=0x7f5380001d20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:54.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.284+0000 7f538d7fa640 1 -- 172.21.15.40:0/3397660098 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5388049070 con 0x7f53900b1430 2023-10-18T08:59:54.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.478+0000 7f5396456640 1 -- 172.21.15.40:0/3397660098 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f539015bca0 con 0x7f53900b1430 2023-10-18T08:59:54.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.479+0000 7f538d7fa640 1 -- 172.21.15.40:0/3397660098 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5388035030 con 0x7f53900b1430 2023-10-18T08:59:54.481 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:59:54.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.482+0000 7f5396456640 1 -- 172.21.15.40:0/3397660098 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f537006ae40 msgr2=0x7f537006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:54.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.482+0000 7f5396456640 1 --2- 172.21.15.40:0/3397660098 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f537006ae40 0x7f537006d300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5380001eb0 tx=0x7f5380001d20 comp rx=0 tx=0).stop 2023-10-18T08:59:54.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.482+0000 7f5396456640 1 -- 172.21.15.40:0/3397660098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53900b1430 msgr2=0x7f5390152630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:54.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.482+0000 7f5396456640 1 --2- 172.21.15.40:0/3397660098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53900b1430 0x7f5390152630 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5388009f90 tx=0x7f5388033f90 comp rx=0 tx=0).stop 2023-10-18T08:59:54.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.482+0000 7f5396456640 1 -- 172.21.15.40:0/3397660098 shutdown_connections 2023-10-18T08:59:54.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.482+0000 7f5396456640 1 --2- 172.21.15.40:0/3397660098 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f537006ae40 0x7f537006d300 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.482+0000 7f5396456640 1 --2- 172.21.15.40:0/3397660098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53900b1430 0x7f5390152630 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.482+0000 7f5396456640 1 --2- 172.21.15.40:0/3397660098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f539015a230 0x7f53900b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.482+0000 7f5396456640 1 --2- 172.21.15.40:0/3397660098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f537c004d00 0x7f53901486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.482+0000 7f5396456640 1 -- 172.21.15.40:0/3397660098 >> 172.21.15.40:0/3397660098 conn(0x7f5390069720 msgr2=0x7f5390154ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:54.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.482+0000 7f5396456640 1 -- 172.21.15.40:0/3397660098 shutdown_connections 2023-10-18T08:59:54.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.482+0000 7f5396456640 1 -- 172.21.15.40:0/3397660098 wait complete. 2023-10-18T08:59:54.505 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:59:54.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.741+0000 7fcdfce4b640 1 -- 172.21.15.40:0/3050148105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf80cae80 msgr2=0x7fcdf80cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:54.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.741+0000 7fcdfce4b640 1 --2- 172.21.15.40:0/3050148105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf80cae80 0x7fcdf80cb2f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcde0005fd0 tx=0x7fcde002be20 comp rx=0 tx=0).stop 2023-10-18T08:59:54.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.741+0000 7fcdfce4b640 1 -- 172.21.15.40:0/3050148105 shutdown_connections 2023-10-18T08:59:54.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.741+0000 7fcdfce4b640 1 --2- 172.21.15.40:0/3050148105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdf80cb830 0x7fcdf806f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.741+0000 7fcdfce4b640 1 --2- 172.21.15.40:0/3050148105 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf80cae80 0x7fcdf80cb2f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.741+0000 7fcdfce4b640 1 -- 172.21.15.40:0/3050148105 >> 172.21.15.40:0/3050148105 conn(0x7fcdf8069720 msgr2=0x7fcdf8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.742+0000 7fcdfce4b640 1 -- 172.21.15.40:0/3050148105 shutdown_connections 2023-10-18T08:59:54.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.742+0000 7fcdfce4b640 1 -- 172.21.15.40:0/3050148105 wait complete. 2023-10-18T08:59:54.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.743+0000 7fcdfce4b640 1 Processor -- start 2023-10-18T08:59:54.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.743+0000 7fcdfce4b640 1 -- start start 2023-10-18T08:59:54.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.744+0000 7fcdfce4b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf80cae80 0x7fcdf8156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.744+0000 7fcdfce4b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdf80cb830 0x7fcdf8159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.744+0000 7fcdfce4b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdf8157800 0x7fcdf80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:54.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.744+0000 7fcdfce4b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf8156910 con 0x7fcdf80cae80 2023-10-18T08:59:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.744+0000 7fcdfce4b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf81548a0 con 0x7fcdf80cb830 2023-10-18T08:59:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.744+0000 7fcdfce4b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf8154b60 con 0x7fcdf8157800 2023-10-18T08:59:54.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.745+0000 7fcdf659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf80cae80 0x7fcdf8156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:54.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.745+0000 7fcdf659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf80cae80 0x7fcdf8156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35066/0 (socket says 172.21.15.40:35066) 2023-10-18T08:59:54.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.745+0000 7fcdf659c640 1 -- 172.21.15.40:0/2239010548 learned_addr learned my addr 172.21.15.40:0/2239010548 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.745+0000 7fcdf6d9d640 1 --2- 172.21.15.40:0/2239010548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdf8157800 0x7fcdf80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:54.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.745+0000 7fcdf659c640 1 -- 172.21.15.40:0/2239010548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdf8157800 msgr2=0x7fcdf80705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.745+0000 7fcdf659c640 1 --2- 172.21.15.40:0/2239010548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdf8157800 0x7fcdf80705f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.745+0000 7fcdf659c640 1 -- 172.21.15.40:0/2239010548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdf80cb830 msgr2=0x7fcdf8159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.745+0000 7fcdf659c640 1 --2- 172.21.15.40:0/2239010548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdf80cb830 0x7fcdf8159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.745+0000 7fcdf659c640 1 -- 172.21.15.40:0/2239010548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde0009560 con 0x7fcdf80cae80 2023-10-18T08:59:54.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.745+0000 7fcdf659c640 1 --2- 172.21.15.40:0/2239010548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf80cae80 0x7fcdf8156da0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcde800c400 tx=0x7fcde800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:54.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.746+0000 7fcdef7fe640 1 -- 172.21.15.40:0/2239010548 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde80052d0 con 0x7fcdf80cae80 2023-10-18T08:59:54.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.746+0000 7fcdef7fe640 1 -- 172.21.15.40:0/2239010548 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcde80035f0 con 0x7fcdf80cae80 2023-10-18T08:59:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.746+0000 7fcdef7fe640 1 -- 172.21.15.40:0/2239010548 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8003a00 con 0x7fcdf80cae80 2023-10-18T08:59:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.746+0000 7fcdfce4b640 1 -- 172.21.15.40:0/2239010548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf8154e40 con 0x7fcdf80cae80 2023-10-18T08:59:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.746+0000 7fcdfce4b640 1 -- 172.21.15.40:0/2239010548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf81564a0 con 0x7fcdf80cae80 2023-10-18T08:59:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.747+0000 7fcdfce4b640 1 -- 172.21.15.40:0/2239010548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf8156690 con 0x7fcdf80cae80 2023-10-18T08:59:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.748+0000 7fcdef7fe640 1 -- 172.21.15.40:0/2239010548 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcde8003b60 con 0x7fcdf80cae80 2023-10-18T08:59:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.749+0000 7fcdef7fe640 1 --2- 172.21.15.40:0/2239010548 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcddc06ae40 0x7fcddc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.749+0000 7fcdef7fe640 1 -- 172.21.15.40:0/2239010548 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcde8088d30 con 0x7fcdf80cae80 2023-10-18T08:59:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.749+0000 7fcdef7fe640 1 -- 172.21.15.40:0/2239010548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=73}) v3 -- 0x7fcddc072970 con 0x7fcdf80cae80 2023-10-18T08:59:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.750+0000 7fcdedd9b640 1 --2- 172.21.15.40:0/2239010548 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcddc06ae40 0x7fcddc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.751+0000 7fcdedd9b640 1 --2- 172.21.15.40:0/2239010548 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcddc06ae40 0x7fcddc06d300 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcde0035000 tx=0x7fcde00046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.753+0000 7fcdef7fe640 1 -- 172.21.15.40:0/2239010548 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcde8017060 con 0x7fcdf80cae80 2023-10-18T08:59:54.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.944+0000 7fcdfce4b640 1 -- 172.21.15.40:0/2239010548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcdf80cc450 con 0x7fcdf80cae80 2023-10-18T08:59:54.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.945+0000 7fcdef7fe640 1 -- 172.21.15.40:0/2239010548 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcdf80cc450 con 0x7fcdf80cae80 2023-10-18T08:59:54.948 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:59:54.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.948+0000 7fcdfce4b640 1 -- 172.21.15.40:0/2239010548 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcddc06ae40 msgr2=0x7fcddc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.948+0000 7fcdfce4b640 1 --2- 172.21.15.40:0/2239010548 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcddc06ae40 0x7fcddc06d300 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcde0035000 tx=0x7fcde00046c0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.948+0000 7fcdfce4b640 1 -- 172.21.15.40:0/2239010548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf80cae80 msgr2=0x7fcdf8156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.948+0000 7fcdfce4b640 1 --2- 172.21.15.40:0/2239010548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf80cae80 0x7fcdf8156da0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcde800c400 tx=0x7fcde800c8c0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.948+0000 7fcdfce4b640 1 -- 172.21.15.40:0/2239010548 shutdown_connections 2023-10-18T08:59:54.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.948+0000 7fcdfce4b640 1 --2- 172.21.15.40:0/2239010548 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcddc06ae40 0x7fcddc06d300 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.948+0000 7fcdfce4b640 1 --2- 172.21.15.40:0/2239010548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdf8157800 0x7fcdf80705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.948+0000 7fcdfce4b640 1 --2- 172.21.15.40:0/2239010548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdf80cb830 0x7fcdf8159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.948+0000 7fcdfce4b640 1 --2- 172.21.15.40:0/2239010548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf80cae80 0x7fcdf8156da0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:54.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.948+0000 7fcdfce4b640 1 -- 172.21.15.40:0/2239010548 >> 172.21.15.40:0/2239010548 conn(0x7fcdf8069720 msgr2=0x7fcdf8159f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:54.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.948+0000 7fcdfce4b640 1 -- 172.21.15.40:0/2239010548 shutdown_connections 2023-10-18T08:59:54.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:54.948+0000 7fcdfce4b640 1 -- 172.21.15.40:0/2239010548 wait complete. 2023-10-18T08:59:54.970 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:59:54.972 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T08:59:54.970+0000 7f6a11f15640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2023-10-18T08:59:55.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.191+0000 7f8415528640 1 -- 172.21.15.40:0/1002850624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f841015af70 msgr2=0x7f8410070b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:55.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.191+0000 7f8415528640 1 --2- 172.21.15.40:0/1002850624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f841015af70 0x7f8410070b60 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f84100aef60 tx=0x7f840402d150 comp rx=0 tx=0).stop 2023-10-18T08:59:55.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.192+0000 7f8415528640 1 -- 172.21.15.40:0/1002850624 shutdown_connections 2023-10-18T08:59:55.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.192+0000 7f8415528640 1 --2- 172.21.15.40:0/1002850624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f841015af70 0x7f8410070b60 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.192+0000 7f8415528640 1 --2- 172.21.15.40:0/1002850624 >> v2:172.21.15.144:3300/0 conn(0x7f841006d5d0 0x7f84100c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.192+0000 7f8415528640 1 -- 172.21.15.40:0/1002850624 >> 172.21.15.40:0/1002850624 conn(0x7f841006ca10 msgr2=0x7f841006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:55.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.192+0000 7f8415528640 1 -- 172.21.15.40:0/1002850624 shutdown_connections 2023-10-18T08:59:55.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.192+0000 7f8415528640 1 -- 172.21.15.40:0/1002850624 wait complete. 2023-10-18T08:59:55.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.194+0000 7f8415528640 1 Processor -- start 2023-10-18T08:59:55.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.194+0000 7f8415528640 1 -- start start 2023-10-18T08:59:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.195+0000 7f8415528640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f841006d5d0 0x7f84101f0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.195+0000 7f8415528640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84100710a0 0x7f841015b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.195+0000 7f8415528640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f841015af70 0x7f84101565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.195+0000 7f8415528640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84101ef660 con 0x7f841006d5d0 2023-10-18T08:59:55.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.195+0000 7f8415528640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8410157060 con 0x7f841015af70 2023-10-18T08:59:55.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.195+0000 7f8415528640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84101ef900 con 0x7f84100710a0 2023-10-18T08:59:55.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.196+0000 7f840ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f841006d5d0 0x7f84101f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:55.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.196+0000 7f840ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f841006d5d0 0x7f84101f0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47662/0 (socket says 172.21.15.40:47662) 2023-10-18T08:59:55.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.196+0000 7f840ffff640 1 -- 172.21.15.40:0/2277540039 learned_addr learned my addr 172.21.15.40:0/2277540039 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:55.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.196+0000 7f840f7fe640 1 --2- 172.21.15.40:0/2277540039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84100710a0 0x7f841015b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.196+0000 7f8414d27640 1 --2- 172.21.15.40:0/2277540039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f841015af70 0x7f84101565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.196+0000 7f840ffff640 1 -- 172.21.15.40:0/2277540039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84100710a0 msgr2=0x7f841015b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.196+0000 7f840ffff640 1 --2- 172.21.15.40:0/2277540039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84100710a0 0x7f841015b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.196+0000 7f840ffff640 1 -- 172.21.15.40:0/2277540039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f841015af70 msgr2=0x7f84101565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:55.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.196+0000 7f840ffff640 1 --2- 172.21.15.40:0/2277540039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f841015af70 0x7f84101565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.196+0000 7f840ffff640 1 -- 172.21.15.40:0/2277540039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840400aa10 con 0x7f841006d5d0 2023-10-18T08:59:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.196+0000 7f840f7fe640 1 --2- 172.21.15.40:0/2277540039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84100710a0 0x7f841015b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:59:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.196+0000 7f8414d27640 1 --2- 172.21.15.40:0/2277540039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f841015af70 0x7f84101565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:59:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.197+0000 7f840ffff640 1 --2- 172.21.15.40:0/2277540039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f841006d5d0 0x7f84101f0100 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f840000c970 tx=0x7f840000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:55.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.197+0000 7f840cff9640 1 -- 172.21.15.40:0/2277540039 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400007bb0 con 0x7f841006d5d0 2023-10-18T08:59:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.197+0000 7f840cff9640 1 -- 172.21.15.40:0/2277540039 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8400007e00 con 0x7f841006d5d0 2023-10-18T08:59:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.197+0000 7f8415528640 1 -- 172.21.15.40:0/2277540039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84101efbe0 con 0x7f841006d5d0 2023-10-18T08:59:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.197+0000 7f840cff9640 1 -- 172.21.15.40:0/2277540039 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840000f6b0 con 0x7f841006d5d0 2023-10-18T08:59:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.197+0000 7f8415528640 1 -- 172.21.15.40:0/2277540039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841014dba0 con 0x7f841006d5d0 2023-10-18T08:59:55.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.198+0000 7f8415528640 1 -- 172.21.15.40:0/2277540039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8410066e00 con 0x7f841006d5d0 2023-10-18T08:59:55.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.199+0000 7f840cff9640 1 -- 172.21.15.40:0/2277540039 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f840000f810 con 0x7f841006d5d0 2023-10-18T08:59:55.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.200+0000 7f840cff9640 1 --2- 172.21.15.40:0/2277540039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83fc06ae40 0x7f83fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:55.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.200+0000 7f840cff9640 1 -- 172.21.15.40:0/2277540039 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8400089ec0 con 0x7f841006d5d0 2023-10-18T08:59:55.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.201+0000 7f840cff9640 1 -- 172.21.15.40:0/2277540039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7f83fc072970 con 0x7f841006d5d0 2023-10-18T08:59:55.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.201+0000 7f840f7fe640 1 --2- 172.21.15.40:0/2277540039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83fc06ae40 0x7f83fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:55.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.202+0000 7f840f7fe640 1 --2- 172.21.15.40:0/2277540039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83fc06ae40 0x7f83fc06d300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8404037000 tx=0x7f8404004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:55.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.204+0000 7f840cff9640 1 -- 172.21.15.40:0/2277540039 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f840001d060 con 0x7f841006d5d0 2023-10-18T08:59:55.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.399+0000 7f8415528640 1 -- 172.21.15.40:0/2277540039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f84100c9190 con 0x7f841006d5d0 2023-10-18T08:59:55.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.400+0000 7f840cff9640 1 -- 172.21.15.40:0/2277540039 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8400054ed0 con 0x7f841006d5d0 2023-10-18T08:59:55.402 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:59:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.403+0000 7f8415528640 1 -- 172.21.15.40:0/2277540039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83fc06ae40 msgr2=0x7f83fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.403+0000 7f8415528640 1 --2- 172.21.15.40:0/2277540039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83fc06ae40 0x7f83fc06d300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8404037000 tx=0x7f8404004ab0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.403+0000 7f8415528640 1 -- 172.21.15.40:0/2277540039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f841006d5d0 msgr2=0x7f84101f0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.403+0000 7f8415528640 1 --2- 172.21.15.40:0/2277540039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f841006d5d0 0x7f84101f0100 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f840000c970 tx=0x7f840000ce30 comp rx=0 tx=0).stop 2023-10-18T08:59:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.403+0000 7f8415528640 1 -- 172.21.15.40:0/2277540039 shutdown_connections 2023-10-18T08:59:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.403+0000 7f8415528640 1 --2- 172.21.15.40:0/2277540039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83fc06ae40 0x7f83fc06d300 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.403+0000 7f8415528640 1 --2- 172.21.15.40:0/2277540039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f841015af70 0x7f84101565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.403+0000 7f8415528640 1 --2- 172.21.15.40:0/2277540039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84100710a0 0x7f841015b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.403+0000 7f8415528640 1 --2- 172.21.15.40:0/2277540039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f841006d5d0 0x7f84101f0100 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.403+0000 7f8415528640 1 -- 172.21.15.40:0/2277540039 >> 172.21.15.40:0/2277540039 conn(0x7f841006ca10 msgr2=0x7f8410153d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:55.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.403+0000 7f8415528640 1 -- 172.21.15.40:0/2277540039 shutdown_connections 2023-10-18T08:59:55.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.403+0000 7f8415528640 1 -- 172.21.15.40:0/2277540039 wait complete. 2023-10-18T08:59:55.424 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T08:59:55.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.656+0000 7f79af59e640 1 -- 172.21.15.40:0/2969826336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b00b1e10 msgr2=0x7f79b00c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:55.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.656+0000 7f79af59e640 1 --2- 172.21.15.40:0/2969826336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b00b1e10 0x7f79b00c99c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f7998009830 tx=0x7f799802bd60 comp rx=0 tx=0).stop 2023-10-18T08:59:55.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.656+0000 7f79af59e640 1 -- 172.21.15.40:0/2969826336 shutdown_connections 2023-10-18T08:59:55.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.656+0000 7f79af59e640 1 --2- 172.21.15.40:0/2969826336 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79b00b2270 0x7f79b00c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.656+0000 7f79af59e640 1 --2- 172.21.15.40:0/2969826336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b00b1e10 0x7f79b00c99c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.656+0000 7f79af59e640 1 -- 172.21.15.40:0/2969826336 >> 172.21.15.40:0/2969826336 conn(0x7f79b0069720 msgr2=0x7f79b0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:55.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.656+0000 7f79af59e640 1 -- 172.21.15.40:0/2969826336 shutdown_connections 2023-10-18T08:59:55.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.656+0000 7f79af59e640 1 -- 172.21.15.40:0/2969826336 wait complete. 2023-10-18T08:59:55.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.658+0000 7f79af59e640 1 Processor -- start 2023-10-18T08:59:55.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.658+0000 7f79af59e640 1 -- start start 2023-10-18T08:59:55.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.659+0000 7f79af59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79b00b15b0 0x7f79b014b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:55.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.659+0000 7f79af59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b00b1e10 0x7f79b014ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:55.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.659+0000 7f79af59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79b00b2270 0x7f79b0143bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:55.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.659+0000 7f79af59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b0145e50 con 0x7f79b00b1e10 2023-10-18T08:59:55.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.659+0000 7f79af59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01451c0 con 0x7f79b00b15b0 2023-10-18T08:59:55.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.659+0000 7f79af59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b014a230 con 0x7f79b00b2270 2023-10-18T08:59:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.660+0000 7f79add9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b00b1e10 0x7f79b014ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.660+0000 7f79add9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b00b1e10 0x7f79b014ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47686/0 (socket says 172.21.15.40:47686) 2023-10-18T08:59:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.660+0000 7f79add9b640 1 -- 172.21.15.40:0/56528067 learned_addr learned my addr 172.21.15.40:0/56528067 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:55.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.660+0000 7f79aed9d640 1 --2- 172.21.15.40:0/56528067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79b00b2270 0x7f79b0143bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:55.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.660+0000 7f79ae59c640 1 --2- 172.21.15.40:0/56528067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79b00b15b0 0x7f79b014b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:55.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.660+0000 7f79add9b640 1 -- 172.21.15.40:0/56528067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79b00b2270 msgr2=0x7f79b0143bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:55.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.660+0000 7f79add9b640 1 --2- 172.21.15.40:0/56528067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79b00b2270 0x7f79b0143bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.660+0000 7f79add9b640 1 -- 172.21.15.40:0/56528067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79b00b15b0 msgr2=0x7f79b014b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.660+0000 7f79add9b640 1 --2- 172.21.15.40:0/56528067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79b00b15b0 0x7f79b014b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.660+0000 7f79add9b640 1 -- 172.21.15.40:0/56528067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7998009490 con 0x7f79b00b1e10 2023-10-18T08:59:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.660+0000 7f79add9b640 1 --2- 172.21.15.40:0/56528067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b00b1e10 0x7f79b014ac50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7998000c00 tx=0x7f79980038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.661+0000 7f79a6ffd640 1 -- 172.21.15.40:0/56528067 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998003ea0 con 0x7f79b00b1e10 2023-10-18T08:59:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.661+0000 7f79a6ffd640 1 -- 172.21.15.40:0/56528067 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7998034440 con 0x7f79b00b1e10 2023-10-18T08:59:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.661+0000 7f79a6ffd640 1 -- 172.21.15.40:0/56528067 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998034850 con 0x7f79b00b1e10 2023-10-18T08:59:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.661+0000 7f79af59e640 1 -- 172.21.15.40:0/56528067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b0146060 con 0x7f79b00b1e10 2023-10-18T08:59:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.661+0000 7f79af59e640 1 -- 172.21.15.40:0/56528067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b0144100 con 0x7f79b00b1e10 2023-10-18T08:59:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.662+0000 7f79af59e640 1 -- 172.21.15.40:0/56528067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b014aa10 con 0x7f79b00b1e10 2023-10-18T08:59:55.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.663+0000 7f79a6ffd640 1 -- 172.21.15.40:0/56528067 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f79980349b0 con 0x7f79b00b1e10 2023-10-18T08:59:55.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.664+0000 7f79a6ffd640 1 --2- 172.21.15.40:0/56528067 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f799006ae40 0x7f799006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:55.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.665+0000 7f79a6ffd640 1 -- 172.21.15.40:0/56528067 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79980abe00 con 0x7f79b00b1e10 2023-10-18T08:59:55.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.665+0000 7f79a6ffd640 1 -- 172.21.15.40:0/56528067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7f7990072970 con 0x7f79b00b1e10 2023-10-18T08:59:55.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.665+0000 7f79ae59c640 1 --2- 172.21.15.40:0/56528067 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f799006ae40 0x7f799006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:55.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.666+0000 7f79ae59c640 1 --2- 172.21.15.40:0/56528067 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f799006ae40 0x7f799006d300 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f79a0001460 tx=0x7f79a0001ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:55.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.669+0000 7f79a6ffd640 1 -- 172.21.15.40:0/56528067 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7998042020 con 0x7f79b00b1e10 2023-10-18T08:59:55.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.880+0000 7f79af59e640 1 -- 172.21.15.40:0/56528067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f79b0066e00 con 0x7f79b00b1e10 2023-10-18T08:59:55.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.880+0000 7f79a6ffd640 1 -- 172.21.15.40:0/56528067 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v73) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7998039070 con 0x7f79b00b1e10 2023-10-18T08:59:55.883 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T08:59:55.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.883+0000 7f79af59e640 1 -- 172.21.15.40:0/56528067 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f799006ae40 msgr2=0x7f799006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:55.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.883+0000 7f79af59e640 1 --2- 172.21.15.40:0/56528067 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f799006ae40 0x7f799006d300 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f79a0001460 tx=0x7f79a0001ab0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.883+0000 7f79af59e640 1 -- 172.21.15.40:0/56528067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b00b1e10 msgr2=0x7f79b014ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:55.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.883+0000 7f79af59e640 1 --2- 172.21.15.40:0/56528067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b00b1e10 0x7f79b014ac50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7998000c00 tx=0x7f79980038a0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.883+0000 7f79af59e640 1 -- 172.21.15.40:0/56528067 shutdown_connections 2023-10-18T08:59:55.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.883+0000 7f79af59e640 1 --2- 172.21.15.40:0/56528067 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f799006ae40 0x7f799006d300 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.883+0000 7f79af59e640 1 --2- 172.21.15.40:0/56528067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f79b00b2270 0x7f79b0143bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.883+0000 7f79af59e640 1 --2- 172.21.15.40:0/56528067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79b00b1e10 0x7f79b014ac50 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.883+0000 7f79af59e640 1 --2- 172.21.15.40:0/56528067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79b00b15b0 0x7f79b014b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:55.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.883+0000 7f79af59e640 1 -- 172.21.15.40:0/56528067 >> 172.21.15.40:0/56528067 conn(0x7f79b0069720 msgr2=0x7f79b0148a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:55.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.884+0000 7f79af59e640 1 -- 172.21.15.40:0/56528067 shutdown_connections 2023-10-18T08:59:55.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:55.884+0000 7f79af59e640 1 -- 172.21.15.40:0/56528067 wait complete. 2023-10-18T08:59:55.904 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-18T08:59:55.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:59:55.905 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-18T08:59:56.022 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T08:59:56.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T08:59:56.023 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T08:59:56.073 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T08:59:56.073 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:59:56.136 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T08:59:56.137 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T08:59:56.137 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T08:59:56.197 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T08:59:56.197 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:59:56.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.438+0000 7ff0b8b1d640 1 -- 172.21.15.40:0/2539309001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b40cdbb0 msgr2=0x7ff0b415a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:56.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.438+0000 7ff0b8b1d640 1 --2- 172.21.15.40:0/2539309001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b40cdbb0 0x7ff0b415a470 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff0b406c300 tx=0x7ff0a402bda0 comp rx=0 tx=0).stop 2023-10-18T08:59:56.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.438+0000 7ff0b8b1d640 1 -- 172.21.15.40:0/2539309001 shutdown_connections 2023-10-18T08:59:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.438+0000 7ff0b8b1d640 1 --2- 172.21.15.40:0/2539309001 >> v2:172.21.15.144:3300/0 conn(0x7ff0b40cbc20 0x7ff0b40c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.438+0000 7ff0b8b1d640 1 --2- 172.21.15.40:0/2539309001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b40cdbb0 0x7ff0b415a470 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.438+0000 7ff0b8b1d640 1 -- 172.21.15.40:0/2539309001 >> 172.21.15.40:0/2539309001 conn(0x7ff0b406cee0 msgr2=0x7ff0b4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.438+0000 7ff0b8b1d640 1 -- 172.21.15.40:0/2539309001 shutdown_connections 2023-10-18T08:59:56.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.438+0000 7ff0b8b1d640 1 -- 172.21.15.40:0/2539309001 wait complete. 2023-10-18T08:59:56.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.440+0000 7ff0b8b1d640 1 Processor -- start 2023-10-18T08:59:56.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.440+0000 7ff0b8b1d640 1 -- start start 2023-10-18T08:59:56.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.441+0000 7ff0b8b1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b406b710 0x7ff0b4154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:56.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.441+0000 7ff0b8b1d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0b40cbc20 0x7ff0b4147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:56.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.441+0000 7ff0b8b1d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0b40cdbb0 0x7ff0b41526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:56.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.441+0000 7ff0b8b1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b415b490 con 0x7ff0b406b710 2023-10-18T08:59:56.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.441+0000 7ff0b8b1d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b4155310 con 0x7ff0b40cdbb0 2023-10-18T08:59:56.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.441+0000 7ff0b8b1d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b415ab80 con 0x7ff0b40cbc20 2023-10-18T08:59:56.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.441+0000 7ff0b259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b406b710 0x7ff0b4154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:56.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.442+0000 7ff0b259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b406b710 0x7ff0b4154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47690/0 (socket says 172.21.15.40:47690) 2023-10-18T08:59:56.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.442+0000 7ff0b259c640 1 -- 172.21.15.40:0/1718681708 learned_addr learned my addr 172.21.15.40:0/1718681708 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:56.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.442+0000 7ff0b259c640 1 -- 172.21.15.40:0/1718681708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0b40cbc20 msgr2=0x7ff0b4147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:56.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.442+0000 7ff0a9d9b640 1 --2- 172.21.15.40:0/1718681708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0b40cbc20 0x7ff0b4147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:56.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.442+0000 7ff0b259c640 1 --2- 172.21.15.40:0/1718681708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0b40cbc20 0x7ff0b4147b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:56.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.442+0000 7ff0b259c640 1 -- 172.21.15.40:0/1718681708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0b40cdbb0 msgr2=0x7ff0b41526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:56.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.442+0000 7ff0b2d9d640 1 --2- 172.21.15.40:0/1718681708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0b40cdbb0 0x7ff0b41526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:56.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.442+0000 7ff0b259c640 1 --2- 172.21.15.40:0/1718681708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0b40cdbb0 0x7ff0b41526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:56.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.442+0000 7ff0b259c640 1 -- 172.21.15.40:0/1718681708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a4009560 con 0x7ff0b406b710 2023-10-18T08:59:56.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.442+0000 7ff0b2d9d640 1 --2- 172.21.15.40:0/1718681708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0b40cdbb0 0x7ff0b41526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:59:56.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.443+0000 7ff0b259c640 1 --2- 172.21.15.40:0/1718681708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b406b710 0x7ff0b4154dd0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009850 tx=0x7ff0a402dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:56.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.443+0000 7ff0ab7fe640 1 -- 172.21.15.40:0/1718681708 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4005650 con 0x7ff0b406b710 2023-10-18T08:59:56.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.443+0000 7ff0ab7fe640 1 -- 172.21.15.40:0/1718681708 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff0a40038f0 con 0x7ff0b406b710 2023-10-18T08:59:56.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.443+0000 7ff0ab7fe640 1 -- 172.21.15.40:0/1718681708 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4003ce0 con 0x7ff0b406b710 2023-10-18T08:59:56.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.443+0000 7ff0b8b1d640 1 -- 172.21.15.40:0/1718681708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b415ae40 con 0x7ff0b406b710 2023-10-18T08:59:56.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.443+0000 7ff0b8b1d640 1 -- 172.21.15.40:0/1718681708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b4148050 con 0x7ff0b406b710 2023-10-18T08:59:56.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.444+0000 7ff0b8b1d640 1 -- 172.21.15.40:0/1718681708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b40b1b60 con 0x7ff0b406b710 2023-10-18T08:59:56.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.445+0000 7ff0ab7fe640 1 -- 172.21.15.40:0/1718681708 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff0a403e450 con 0x7ff0b406b710 2023-10-18T08:59:56.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.446+0000 7ff0ab7fe640 1 --2- 172.21.15.40:0/1718681708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff09406ae40 0x7ff09406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:56.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.446+0000 7ff0ab7fe640 1 -- 172.21.15.40:0/1718681708 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0a40abfa0 con 0x7ff0b406b710 2023-10-18T08:59:56.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.446+0000 7ff0ab7fe640 1 -- 172.21.15.40:0/1718681708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7ff094072970 con 0x7ff0b406b710 2023-10-18T08:59:56.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.447+0000 7ff0a9d9b640 1 --2- 172.21.15.40:0/1718681708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff09406ae40 0x7ff09406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:56.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.448+0000 7ff0a9d9b640 1 --2- 172.21.15.40:0/1718681708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff09406ae40 0x7ff09406d300 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff0b415da00 tx=0x7ff09c0072f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:56.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.451+0000 7ff0ab7fe640 1 -- 172.21.15.40:0/1718681708 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff0a4042060 con 0x7ff0b406b710 2023-10-18T08:59:56.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.684+0000 7ff0b8b1d640 1 -- 172.21.15.40:0/1718681708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0b4066fd0 con 0x7ff0b406b710 2023-10-18T08:59:56.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.686+0000 7ff0ab7fe640 1 -- 172.21.15.40:0/1718681708 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4916 (secure 0 0 0) 0x7ff0a4077040 con 0x7ff0b406b710 2023-10-18T08:59:56.688 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:59:56.689 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5075,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2165596806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2165596806},{"type":"v1","addr":"172.21.15.144:6835","nonce":2165596806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/587522883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":587522883},{"type":"v1","addr":"172.21.15.40:6833","nonce":587522883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1174134524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1174134524},{"type":"v1","addr":"172.21.15.40:6835","nonce":1174134524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:58:28.760677+0000","modified":"2023-10-18T08:58:31.845684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5389":{"gid":5389,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1311290970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1311290970},{"type":"v1","addr":"172.21.15.144:6837","nonce":1311290970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-18T08:59:56.689 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 33 2023-10-18T08:59:56.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.689+0000 7ff0b8b1d640 1 -- 172.21.15.40:0/1718681708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff09406ae40 msgr2=0x7ff09406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:56.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.689+0000 7ff0b8b1d640 1 --2- 172.21.15.40:0/1718681708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff09406ae40 0x7ff09406d300 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff0b415da00 tx=0x7ff09c0072f0 comp rx=0 tx=0).stop 2023-10-18T08:59:56.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.689+0000 7ff0b8b1d640 1 -- 172.21.15.40:0/1718681708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b406b710 msgr2=0x7ff0b4154dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:56.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.689+0000 7ff0b8b1d640 1 --2- 172.21.15.40:0/1718681708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b406b710 0x7ff0b4154dd0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009850 tx=0x7ff0a402dd70 comp rx=0 tx=0).stop 2023-10-18T08:59:56.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.689+0000 7ff0b8b1d640 1 -- 172.21.15.40:0/1718681708 shutdown_connections 2023-10-18T08:59:56.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.689+0000 7ff0b8b1d640 1 --2- 172.21.15.40:0/1718681708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff09406ae40 0x7ff09406d300 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:56.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.689+0000 7ff0b8b1d640 1 --2- 172.21.15.40:0/1718681708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0b40cdbb0 0x7ff0b41526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:56.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.689+0000 7ff0b8b1d640 1 --2- 172.21.15.40:0/1718681708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0b40cbc20 0x7ff0b4147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:56.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.689+0000 7ff0b8b1d640 1 --2- 172.21.15.40:0/1718681708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0b406b710 0x7ff0b4154dd0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:56.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.689+0000 7ff0b8b1d640 1 -- 172.21.15.40:0/1718681708 >> 172.21.15.40:0/1718681708 conn(0x7ff0b406cee0 msgr2=0x7ff0b4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:56.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.689+0000 7ff0b8b1d640 1 -- 172.21.15.40:0/1718681708 shutdown_connections 2023-10-18T08:59:56.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.689+0000 7ff0b8b1d640 1 -- 172.21.15.40:0/1718681708 wait complete. 2023-10-18T08:59:56.711 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:59:56.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.949+0000 7f3e0c305640 1 -- 172.21.15.40:0/2715466223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df00051d0 msgr2=0x7f3df000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:56.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.949+0000 7f3e0c305640 1 --2- 172.21.15.40:0/2715466223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df00051d0 0x7f3df000b980 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00c640 tx=0x7f3df00057f0 comp rx=0 tx=0).stop 2023-10-18T08:59:56.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.949+0000 7f3e0c305640 1 -- 172.21.15.40:0/2715466223 shutdown_connections 2023-10-18T08:59:56.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.949+0000 7f3e0c305640 1 --2- 172.21.15.40:0/2715466223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df00051d0 0x7f3df000b980 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:56.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.949+0000 7f3e0c305640 1 -- 172.21.15.40:0/2715466223 >> 172.21.15.40:0/2715466223 conn(0x7f3e0406cee0 msgr2=0x7f3e04069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:56.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.949+0000 7f3e0c305640 1 -- 172.21.15.40:0/2715466223 shutdown_connections 2023-10-18T08:59:56.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.949+0000 7f3e0c305640 1 -- 172.21.15.40:0/2715466223 wait complete. 2023-10-18T08:59:56.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.951+0000 7f3e0c305640 1 Processor -- start 2023-10-18T08:59:56.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.951+0000 7f3e0c305640 1 -- start start 2023-10-18T08:59:56.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.952+0000 7f3e0c305640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df00051d0 0x7f3e04149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:56.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.952+0000 7f3e0c305640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e0414f810 0x7f3e0414fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:56.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.952+0000 7f3e0c305640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e0414b390 0x7f3e0414b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:56.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.952+0000 7f3e0c305640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e041501d0 con 0x7f3e0414f810 2023-10-18T08:59:56.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.952+0000 7f3e0c305640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0414d160 con 0x7f3df00051d0 2023-10-18T08:59:56.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.952+0000 7f3e0c305640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0414c0c0 con 0x7f3e0414b390 2023-10-18T08:59:56.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.953+0000 7f3e098a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e0414f810 0x7f3e0414fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:56.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.953+0000 7f3e098a0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e0414f810 0x7f3e0414fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47710/0 (socket says 172.21.15.40:47710) 2023-10-18T08:59:56.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.953+0000 7f3e098a0640 1 -- 172.21.15.40:0/156399049 learned_addr learned my addr 172.21.15.40:0/156399049 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:56.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.953+0000 7f3e0a0a1640 1 --2- 172.21.15.40:0/156399049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df00051d0 0x7f3e04149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:56.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.953+0000 7f3e0a8a2640 1 --2- 172.21.15.40:0/156399049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e0414b390 0x7f3e0414b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:56.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.953+0000 7f3e098a0640 1 -- 172.21.15.40:0/156399049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e0414b390 msgr2=0x7f3e0414b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:56.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.953+0000 7f3e098a0640 1 --2- 172.21.15.40:0/156399049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e0414b390 0x7f3e0414b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:56.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.953+0000 7f3e098a0640 1 -- 172.21.15.40:0/156399049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df00051d0 msgr2=0x7f3e04149a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:56.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.953+0000 7f3e098a0640 1 --2- 172.21.15.40:0/156399049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df00051d0 0x7f3e04149a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:56.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.953+0000 7f3e098a0640 1 -- 172.21.15.40:0/156399049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0414c3a0 con 0x7f3e0414f810 2023-10-18T08:59:56.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.953+0000 7f3e0a8a2640 1 --2- 172.21.15.40:0/156399049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e0414b390 0x7f3e0414b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:59:56.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.954+0000 7f3e098a0640 1 --2- 172.21.15.40:0/156399049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e0414f810 0x7f3e0414fc90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3df400da70 tx=0x7f3df400df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.954+0000 7f3dfaffd640 1 -- 172.21.15.40:0/156399049 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df400adc0 con 0x7f3e0414f810 2023-10-18T08:59:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.954+0000 7f3dfaffd640 1 -- 172.21.15.40:0/156399049 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3df400e450 con 0x7f3e0414f810 2023-10-18T08:59:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.954+0000 7f3e0c305640 1 -- 172.21.15.40:0/156399049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0414c680 con 0x7f3e0414f810 2023-10-18T08:59:56.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.954+0000 7f3dfaffd640 1 -- 172.21.15.40:0/156399049 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df400e840 con 0x7f3e0414f810 2023-10-18T08:59:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.954+0000 7f3e0c305640 1 -- 172.21.15.40:0/156399049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0414cc40 con 0x7f3e0414f810 2023-10-18T08:59:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.955+0000 7f3e0c305640 1 -- 172.21.15.40:0/156399049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e04066fd0 con 0x7f3e0414f810 2023-10-18T08:59:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.956+0000 7f3dfaffd640 1 -- 172.21.15.40:0/156399049 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3df400e9a0 con 0x7f3e0414f810 2023-10-18T08:59:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.957+0000 7f3dfaffd640 1 --2- 172.21.15.40:0/156399049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3de806ae70 0x7f3de806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:56.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.958+0000 7f3dfaffd640 1 -- 172.21.15.40:0/156399049 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3df4089ef0 con 0x7f3e0414f810 2023-10-18T08:59:56.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.958+0000 7f3dfaffd640 1 -- 172.21.15.40:0/156399049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7f3de80729a0 con 0x7f3e0414f810 2023-10-18T08:59:56.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.958+0000 7f3e0a0a1640 1 --2- 172.21.15.40:0/156399049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3de806ae70 0x7f3de806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:56.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.959+0000 7f3e0a0a1640 1 --2- 172.21.15.40:0/156399049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3de806ae70 0x7f3de806d330 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3e000016c0 tx=0x7f3e00006a60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:56.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.962+0000 7f3dfaffd640 1 -- 172.21.15.40:0/156399049 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3df401b070 con 0x7f3e0414f810 2023-10-18T08:59:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.993+0000 7f3dfaffd640 1 -- 172.21.15.40:0/156399049 <== mon.0 v2:172.21.15.40:3300/0 7 ==== osd_map(75..75 src has 1..75) v4 ==== 296+0+0 (secure 0 0 0) 0x7f3df40592c0 con 0x7f3e0414f810 2023-10-18T08:59:56.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:56.993+0000 7f3dfaffd640 1 -- 172.21.15.40:0/156399049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=76}) v3 -- 0x7f3de80731f0 con 0x7f3e0414f810 2023-10-18T08:59:57.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.166+0000 7f3e0c305640 1 -- 172.21.15.40:0/156399049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e0406f850 con 0x7f3e0414f810 2023-10-18T08:59:57.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.167+0000 7f3dfaffd640 1 -- 172.21.15.40:0/156399049 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16599 (secure 0 0 0) 0x7f3df4058990 con 0x7f3e0414f810 2023-10-18T08:59:57.170 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:59:57.170 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":75,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:59:56.989945+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:58:26.727620+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-18T08:58:27.795303+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:59:57.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.170+0000 7f3e0c305640 1 -- 172.21.15.40:0/156399049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3de806ae70 msgr2=0x7f3de806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:57.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.170+0000 7f3e0c305640 1 --2- 172.21.15.40:0/156399049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3de806ae70 0x7f3de806d330 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3e000016c0 tx=0x7f3e00006a60 comp rx=0 tx=0).stop 2023-10-18T08:59:57.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.170+0000 7f3e0c305640 1 -- 172.21.15.40:0/156399049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e0414f810 msgr2=0x7f3e0414fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:57.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.170+0000 7f3e0c305640 1 --2- 172.21.15.40:0/156399049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e0414f810 0x7f3e0414fc90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3df400da70 tx=0x7f3df400df30 comp rx=0 tx=0).stop 2023-10-18T08:59:57.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.170+0000 7f3e0c305640 1 -- 172.21.15.40:0/156399049 shutdown_connections 2023-10-18T08:59:57.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.170+0000 7f3e0c305640 1 --2- 172.21.15.40:0/156399049 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3de806ae70 0x7f3de806d330 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.170+0000 7f3e0c305640 1 --2- 172.21.15.40:0/156399049 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e0414b390 0x7f3e0414b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.170+0000 7f3e0c305640 1 --2- 172.21.15.40:0/156399049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e0414f810 0x7f3e0414fc90 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.170+0000 7f3e0c305640 1 --2- 172.21.15.40:0/156399049 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3df00051d0 0x7f3e04149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.170+0000 7f3e0c305640 1 -- 172.21.15.40:0/156399049 >> 172.21.15.40:0/156399049 conn(0x7f3e0406cee0 msgr2=0x7f3e04070580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:57.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.170+0000 7f3e0c305640 1 -- 172.21.15.40:0/156399049 shutdown_connections 2023-10-18T08:59:57.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.170+0000 7f3e0c305640 1 -- 172.21.15.40:0/156399049 wait complete. 2023-10-18T08:59:57.193 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T08:58:26.727620+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T08:58:27.795303+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T08:59:57.194 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T08:59:57.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.436+0000 7fe0fb59e640 1 -- 172.21.15.40:0/412322090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc0b1900 msgr2=0x7fe0fc0c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:57.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.436+0000 7fe0fb59e640 1 --2- 172.21.15.40:0/412322090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc0b1900 0x7fe0fc0c98a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4005fd0 tx=0x7fe0f402c0f0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.436+0000 7fe0fb59e640 1 -- 172.21.15.40:0/412322090 shutdown_connections 2023-10-18T08:59:57.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.436+0000 7fe0fb59e640 1 --2- 172.21.15.40:0/412322090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc0b1900 0x7fe0fc0c98a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.436+0000 7fe0fb59e640 1 -- 172.21.15.40:0/412322090 >> 172.21.15.40:0/412322090 conn(0x7fe0fc069720 msgr2=0x7fe0fc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:57.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.436+0000 7fe0fb59e640 1 -- 172.21.15.40:0/412322090 shutdown_connections 2023-10-18T08:59:57.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.436+0000 7fe0fb59e640 1 -- 172.21.15.40:0/412322090 wait complete. 2023-10-18T08:59:57.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.438+0000 7fe0fb59e640 1 Processor -- start 2023-10-18T08:59:57.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.438+0000 7fe0fb59e640 1 -- start start 2023-10-18T08:59:57.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.439+0000 7fe0fb59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc06d600 0x7fe0fc1f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:57.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.439+0000 7fe0fb59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0fc0b1900 0x7fe0fc0cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:57.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.439+0000 7fe0fb59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0fc1405a0 0x7fe0fc0cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:57.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.439+0000 7fe0fb59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc1fa280 con 0x7fe0fc06d600 2023-10-18T08:59:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.439+0000 7fe0fb59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc0acce0 con 0x7fe0fc1405a0 2023-10-18T08:59:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.439+0000 7fe0fb59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc0acfc0 con 0x7fe0fc0b1900 2023-10-18T08:59:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.439+0000 7fe0fa59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc06d600 0x7fe0fc1f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.440+0000 7fe0fa59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc06d600 0x7fe0fc1f5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47724/0 (socket says 172.21.15.40:47724) 2023-10-18T08:59:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.440+0000 7fe0fa59c640 1 -- 172.21.15.40:0/4053026889 learned_addr learned my addr 172.21.15.40:0/4053026889 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.440+0000 7fe0f9d9b640 1 --2- 172.21.15.40:0/4053026889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0fc0b1900 0x7fe0fc0cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.440+0000 7fe0fa59c640 1 -- 172.21.15.40:0/4053026889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0fc0b1900 msgr2=0x7fe0fc0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.440+0000 7fe0fa59c640 1 --2- 172.21.15.40:0/4053026889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0fc0b1900 0x7fe0fc0cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.440+0000 7fe0fa59c640 1 -- 172.21.15.40:0/4053026889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0fc1405a0 msgr2=0x7fe0fc0cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.440+0000 7fe0fa59c640 1 --2- 172.21.15.40:0/4053026889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0fc1405a0 0x7fe0fc0cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.440+0000 7fe0fa59c640 1 -- 172.21.15.40:0/4053026889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f4009840 con 0x7fe0fc06d600 2023-10-18T08:59:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.440+0000 7fe0fa59c640 1 --2- 172.21.15.40:0/4053026889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc06d600 0x7fe0fc1f5690 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00c490 tx=0x7fe0ec00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.441+0000 7fe0eaffd640 1 -- 172.21.15.40:0/4053026889 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec012830 con 0x7fe0fc06d600 2023-10-18T08:59:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.441+0000 7fe0eaffd640 1 -- 172.21.15.40:0/4053026889 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe0ec012a80 con 0x7fe0fc06d600 2023-10-18T08:59:57.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.441+0000 7fe0eaffd640 1 -- 172.21.15.40:0/4053026889 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec011380 con 0x7fe0fc06d600 2023-10-18T08:59:57.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.441+0000 7fe0fb59e640 1 -- 172.21.15.40:0/4053026889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0fc0ad2a0 con 0x7fe0fc06d600 2023-10-18T08:59:57.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.441+0000 7fe0fb59e640 1 -- 172.21.15.40:0/4053026889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0fc1f5bd0 con 0x7fe0fc06d600 2023-10-18T08:59:57.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.442+0000 7fe0fb59e640 1 -- 172.21.15.40:0/4053026889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0fc0ad7c0 con 0x7fe0fc06d600 2023-10-18T08:59:57.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.443+0000 7fe0eaffd640 1 -- 172.21.15.40:0/4053026889 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe0ec0114e0 con 0x7fe0fc06d600 2023-10-18T08:59:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.444+0000 7fe0eaffd640 1 --2- 172.21.15.40:0/4053026889 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe0dc06ae70 0x7fe0dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.444+0000 7fe0eaffd640 1 -- 172.21.15.40:0/4053026889 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0ec089510 con 0x7fe0fc06d600 2023-10-18T08:59:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.444+0000 7fe0eaffd640 1 -- 172.21.15.40:0/4053026889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=76}) v3 -- 0x7fe0dc0729a0 con 0x7fe0fc06d600 2023-10-18T08:59:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.445+0000 7fe0f9d9b640 1 --2- 172.21.15.40:0/4053026889 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe0dc06ae70 0x7fe0dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.446+0000 7fe0f9d9b640 1 --2- 172.21.15.40:0/4053026889 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe0dc06ae70 0x7fe0dc06d330 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0006fd0 tx=0x7fe0f0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:57.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.448+0000 7fe0eaffd640 1 -- 172.21.15.40:0/4053026889 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe0fc0ad7c0 con 0x7fe0fc06d600 2023-10-18T08:59:57.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.639+0000 7fe0fb59e640 1 -- 172.21.15.40:0/4053026889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0fc0ad7c0 con 0x7fe0fc06d600 2023-10-18T08:59:57.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.640+0000 7fe0eaffd640 1 -- 172.21.15.40:0/4053026889 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0fc0ad7c0 con 0x7fe0fc06d600 2023-10-18T08:59:57.642 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T08:59:57.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.642+0000 7fe0fb59e640 1 -- 172.21.15.40:0/4053026889 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe0dc06ae70 msgr2=0x7fe0dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:57.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.642+0000 7fe0fb59e640 1 --2- 172.21.15.40:0/4053026889 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe0dc06ae70 0x7fe0dc06d330 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0006fd0 tx=0x7fe0f0008040 comp rx=0 tx=0).stop 2023-10-18T08:59:57.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.642+0000 7fe0fb59e640 1 -- 172.21.15.40:0/4053026889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc06d600 msgr2=0x7fe0fc1f5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:57.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.642+0000 7fe0fb59e640 1 --2- 172.21.15.40:0/4053026889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc06d600 0x7fe0fc1f5690 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00c490 tx=0x7fe0ec00c950 comp rx=0 tx=0).stop 2023-10-18T08:59:57.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.643+0000 7fe0fb59e640 1 -- 172.21.15.40:0/4053026889 shutdown_connections 2023-10-18T08:59:57.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.643+0000 7fe0fb59e640 1 --2- 172.21.15.40:0/4053026889 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe0dc06ae70 0x7fe0dc06d330 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.643+0000 7fe0fb59e640 1 --2- 172.21.15.40:0/4053026889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0fc1405a0 0x7fe0fc0cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.643+0000 7fe0fb59e640 1 --2- 172.21.15.40:0/4053026889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0fc0b1900 0x7fe0fc0cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.643+0000 7fe0fb59e640 1 --2- 172.21.15.40:0/4053026889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0fc06d600 0x7fe0fc1f5690 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.643+0000 7fe0fb59e640 1 -- 172.21.15.40:0/4053026889 >> 172.21.15.40:0/4053026889 conn(0x7fe0fc069720 msgr2=0x7fe0fc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:57.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.643+0000 7fe0fb59e640 1 -- 172.21.15.40:0/4053026889 shutdown_connections 2023-10-18T08:59:57.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.643+0000 7fe0fb59e640 1 -- 172.21.15.40:0/4053026889 wait complete. 2023-10-18T08:59:57.663 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T08:59:57.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.898+0000 7fdcb7b39640 1 -- 172.21.15.40:0/2029495737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb00ac930 msgr2=0x7fdcb015f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:57.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.898+0000 7fdcb7b39640 1 --2- 172.21.15.40:0/2029495737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb00ac930 0x7fdcb015f280 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009830 tx=0x7fdca802bcd0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.898+0000 7fdcb7b39640 1 -- 172.21.15.40:0/2029495737 shutdown_connections 2023-10-18T08:59:57.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.898+0000 7fdcb7b39640 1 --2- 172.21.15.40:0/2029495737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb00ac930 0x7fdcb015f280 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.898+0000 7fdcb7b39640 1 --2- 172.21.15.40:0/2029495737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcb006d7b0 0x7fdcb00ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.898+0000 7fdcb7b39640 1 -- 172.21.15.40:0/2029495737 >> 172.21.15.40:0/2029495737 conn(0x7fdcb00cd510 msgr2=0x7fdcb00cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:57.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.898+0000 7fdcb7b39640 1 -- 172.21.15.40:0/2029495737 shutdown_connections 2023-10-18T08:59:57.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.899+0000 7fdcb7b39640 1 -- 172.21.15.40:0/2029495737 wait complete. 2023-10-18T08:59:57.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.900+0000 7fdcb7b39640 1 Processor -- start 2023-10-18T08:59:57.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.901+0000 7fdcb7b39640 1 -- start start 2023-10-18T08:59:57.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.901+0000 7fdcb7b39640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcb006d7b0 0x7fdcb0157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:57.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.901+0000 7fdcb7b39640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcb00ac930 0x7fdcb01f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:57.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.902+0000 7fdcb7b39640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb015edd0 0x7fdcb015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:57.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.902+0000 7fdcb7b39640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb00977b0 con 0x7fdcb015edd0 2023-10-18T08:59:57.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.902+0000 7fdcb7b39640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb009a960 con 0x7fdcb006d7b0 2023-10-18T08:59:57.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.902+0000 7fdcb7b39640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb009ac40 con 0x7fdcb00ac930 2023-10-18T08:59:57.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.902+0000 7fdcb60d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb015edd0 0x7fdcb015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:57.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.902+0000 7fdcb60d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb015edd0 0x7fdcb015ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47744/0 (socket says 172.21.15.40:47744) 2023-10-18T08:59:57.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.902+0000 7fdcb60d6640 1 -- 172.21.15.40:0/3001056961 learned_addr learned my addr 172.21.15.40:0/3001056961 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:57.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.902+0000 7fdcb60d6640 1 -- 172.21.15.40:0/3001056961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcb00ac930 msgr2=0x7fdcb01f50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:57.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.902+0000 7fdcb60d6640 1 --2- 172.21.15.40:0/3001056961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcb00ac930 0x7fdcb01f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.902+0000 7fdcb60d6640 1 -- 172.21.15.40:0/3001056961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcb006d7b0 msgr2=0x7fdcb0157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:57.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.902+0000 7fdcb58d5640 1 --2- 172.21.15.40:0/3001056961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcb006d7b0 0x7fdcb0157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:57.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.902+0000 7fdcb60d6640 1 --2- 172.21.15.40:0/3001056961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcb006d7b0 0x7fdcb0157fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:57.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.902+0000 7fdcb60d6640 1 -- 172.21.15.40:0/3001056961 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca8009490 con 0x7fdcb015edd0 2023-10-18T08:59:57.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.903+0000 7fdcb60d6640 1 --2- 172.21.15.40:0/3001056961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb015edd0 0x7fdcb015ab40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009780 tx=0x7fdca8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:57.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.903+0000 7fdcae7fc640 1 -- 172.21.15.40:0/3001056961 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8003e20 con 0x7fdcb015edd0 2023-10-18T08:59:57.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.903+0000 7fdcae7fc640 1 -- 172.21.15.40:0/3001056961 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdca8034530 con 0x7fdcb015edd0 2023-10-18T08:59:57.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.903+0000 7fdcb7b39640 1 -- 172.21.15.40:0/3001056961 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb009af20 con 0x7fdcb015edd0 2023-10-18T08:59:57.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.903+0000 7fdcae7fc640 1 -- 172.21.15.40:0/3001056961 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca8034940 con 0x7fdcb015edd0 2023-10-18T08:59:57.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.903+0000 7fdcb7b39640 1 -- 172.21.15.40:0/3001056961 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb01f55e0 con 0x7fdcb015edd0 2023-10-18T08:59:57.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.905+0000 7fdcb7b39640 1 -- 172.21.15.40:0/3001056961 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb00ccf70 con 0x7fdcb015edd0 2023-10-18T08:59:57.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.905+0000 7fdcae7fc640 1 -- 172.21.15.40:0/3001056961 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdca803b040 con 0x7fdcb015edd0 2023-10-18T08:59:57.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.906+0000 7fdcae7fc640 1 --2- 172.21.15.40:0/3001056961 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdc9406ae40 0x7fdc9406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:57.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.906+0000 7fdcae7fc640 1 -- 172.21.15.40:0/3001056961 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdca80abc90 con 0x7fdcb015edd0 2023-10-18T08:59:57.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.907+0000 7fdcae7fc640 1 -- 172.21.15.40:0/3001056961 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=76}) v3 -- 0x7fdc94072970 con 0x7fdcb015edd0 2023-10-18T08:59:57.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.907+0000 7fdcb58d5640 1 --2- 172.21.15.40:0/3001056961 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdc9406ae40 0x7fdc9406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:57.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.908+0000 7fdcb58d5640 1 --2- 172.21.15.40:0/3001056961 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdc9406ae40 0x7fdc9406d300 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdca4004a30 tx=0x7fdca4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:57.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:57.911+0000 7fdcae7fc640 1 -- 172.21.15.40:0/3001056961 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdca8041060 con 0x7fdcb015edd0 2023-10-18T08:59:58.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.003+0000 7fdcae7fc640 1 -- 172.21.15.40:0/3001056961 <== mon.0 v2:172.21.15.40:3300/0 7 ==== osd_map(76..76 src has 1..76) v4 ==== 1468+0+0 (secure 0 0 0) 0x7fdca807b060 con 0x7fdcb015edd0 2023-10-18T08:59:58.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.003+0000 7fdcae7fc640 1 -- 172.21.15.40:0/3001056961 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=77}) v3 -- 0x7fdc94073b80 con 0x7fdcb015edd0 2023-10-18T08:59:58.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.115+0000 7fdcb7b39640 1 -- 172.21.15.40:0/3001056961 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdcb00cc7b0 con 0x7fdcb015edd0 2023-10-18T08:59:58.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.116+0000 7fdcae7fc640 1 -- 172.21.15.40:0/3001056961 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdca8039070 con 0x7fdcb015edd0 2023-10-18T08:59:58.118 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T08:59:58.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.119+0000 7fdcb7b39640 1 -- 172.21.15.40:0/3001056961 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdc9406ae40 msgr2=0x7fdc9406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:58.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.119+0000 7fdcb7b39640 1 --2- 172.21.15.40:0/3001056961 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdc9406ae40 0x7fdc9406d300 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdca4004a30 tx=0x7fdca4009040 comp rx=0 tx=0).stop 2023-10-18T08:59:58.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.119+0000 7fdcb7b39640 1 -- 172.21.15.40:0/3001056961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb015edd0 msgr2=0x7fdcb015ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:58.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.119+0000 7fdcb7b39640 1 --2- 172.21.15.40:0/3001056961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb015edd0 0x7fdcb015ab40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009780 tx=0x7fdca8003820 comp rx=0 tx=0).stop 2023-10-18T08:59:58.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.119+0000 7fdcb7b39640 1 -- 172.21.15.40:0/3001056961 shutdown_connections 2023-10-18T08:59:58.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.119+0000 7fdcb7b39640 1 --2- 172.21.15.40:0/3001056961 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdc9406ae40 0x7fdc9406d300 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:58.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.119+0000 7fdcb7b39640 1 --2- 172.21.15.40:0/3001056961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcb015edd0 0x7fdcb015ab40 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:58.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.119+0000 7fdcb7b39640 1 --2- 172.21.15.40:0/3001056961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcb00ac930 0x7fdcb01f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:58.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.119+0000 7fdcb7b39640 1 --2- 172.21.15.40:0/3001056961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcb006d7b0 0x7fdcb0157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:58.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.119+0000 7fdcb7b39640 1 -- 172.21.15.40:0/3001056961 >> 172.21.15.40:0/3001056961 conn(0x7fdcb00cd510 msgr2=0x7fdcb00c1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:58.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.119+0000 7fdcb7b39640 1 -- 172.21.15.40:0/3001056961 shutdown_connections 2023-10-18T08:59:58.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.119+0000 7fdcb7b39640 1 -- 172.21.15.40:0/3001056961 wait complete. 2023-10-18T08:59:58.140 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T08:59:58.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.369+0000 7f1f6ddc9640 1 -- 172.21.15.40:0/808231169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f54004d00 msgr2=0x7f1f54005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:58.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.369+0000 7f1f6ddc9640 1 --2- 172.21.15.40:0/808231169 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f54004d00 0x7f1f54005180 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009f00 tx=0x7f1f540057f0 comp rx=0 tx=0).stop 2023-10-18T08:59:58.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.369+0000 7f1f67fff640 1 -- 172.21.15.40:0/808231169 reap_dead start 2023-10-18T08:59:58.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.370+0000 7f1f6ddc9640 1 -- 172.21.15.40:0/808231169 shutdown_connections 2023-10-18T08:59:58.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.370+0000 7f1f6ddc9640 1 -- 172.21.15.40:0/808231169 >> 172.21.15.40:0/808231169 conn(0x7f1f68069720 msgr2=0x7f1f68069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:58.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.370+0000 7f1f6ddc9640 1 -- 172.21.15.40:0/808231169 shutdown_connections 2023-10-18T08:59:58.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.370+0000 7f1f6ddc9640 1 -- 172.21.15.40:0/808231169 wait complete. 2023-10-18T08:59:58.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.372+0000 7f1f6ddc9640 1 Processor -- start 2023-10-18T08:59:58.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.372+0000 7f1f6ddc9640 1 -- start start 2023-10-18T08:59:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.373+0000 7f1f6ddc9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f54004d00 0x7f1f6813c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.373+0000 7f1f6ddc9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f6813ccd0 0x7f1f681f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:58.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.373+0000 7f1f6ddc9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f681f2080 0x7f1f6814b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:58.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.373+0000 7f1f6ddc9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6813ca90 con 0x7f1f681f2080 2023-10-18T08:59:58.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.373+0000 7f1f6ddc9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6814a300 con 0x7f1f6813ccd0 2023-10-18T08:59:58.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.373+0000 7f1f6ddc9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6814a5a0 con 0x7f1f54004d00 2023-10-18T08:59:58.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.373+0000 7f1f67fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f681f2080 0x7f1f6814b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:58.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.373+0000 7f1f67fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f681f2080 0x7f1f6814b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47758/0 (socket says 172.21.15.40:47758) 2023-10-18T08:59:58.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.373+0000 7f1f67fff640 1 -- 172.21.15.40:0/80410729 learned_addr learned my addr 172.21.15.40:0/80410729 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:58.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.374+0000 7f1f677fe640 1 --2- 172.21.15.40:0/80410729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f54004d00 0x7f1f6813c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:58.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.374+0000 7f1f5ffff640 1 --2- 172.21.15.40:0/80410729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f6813ccd0 0x7f1f681f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:58.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.374+0000 7f1f67fff640 1 -- 172.21.15.40:0/80410729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f54004d00 msgr2=0x7f1f6813c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:58.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.374+0000 7f1f67fff640 1 --2- 172.21.15.40:0/80410729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f54004d00 0x7f1f6813c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:58.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.374+0000 7f1f67fff640 1 -- 172.21.15.40:0/80410729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f6813ccd0 msgr2=0x7f1f681f1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:58.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.374+0000 7f1f67fff640 1 --2- 172.21.15.40:0/80410729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f6813ccd0 0x7f1f681f1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:58.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.374+0000 7f1f67fff640 1 -- 172.21.15.40:0/80410729 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6814a880 con 0x7f1f681f2080 2023-10-18T08:59:58.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.374+0000 7f1f677fe640 1 --2- 172.21.15.40:0/80410729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f54004d00 0x7f1f6813c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:59:58.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.374+0000 7f1f5ffff640 1 --2- 172.21.15.40:0/80410729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f6813ccd0 0x7f1f681f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:59:58.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.374+0000 7f1f67fff640 1 --2- 172.21.15.40:0/80410729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f681f2080 0x7f1f6814b360 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1f60002230 tx=0x7f1f60031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:58.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.374+0000 7f1f64ff9640 1 -- 172.21.15.40:0/80410729 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6003f070 con 0x7f1f681f2080 2023-10-18T08:59:58.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.374+0000 7f1f64ff9640 1 -- 172.21.15.40:0/80410729 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1f60031bf0 con 0x7f1f681f2080 2023-10-18T08:59:58.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.374+0000 7f1f6ddc9640 1 -- 172.21.15.40:0/80410729 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6814ab60 con 0x7f1f681f2080 2023-10-18T08:59:58.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.374+0000 7f1f64ff9640 1 -- 172.21.15.40:0/80410729 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60003970 con 0x7f1f681f2080 2023-10-18T08:59:58.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.375+0000 7f1f6ddc9640 1 -- 172.21.15.40:0/80410729 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6814b8a0 con 0x7f1f681f2080 2023-10-18T08:59:58.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.376+0000 7f1f6ddc9640 1 -- 172.21.15.40:0/80410729 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6814b080 con 0x7f1f681f2080 2023-10-18T08:59:58.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.377+0000 7f1f64ff9640 1 -- 172.21.15.40:0/80410729 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1f60003ad0 con 0x7f1f681f2080 2023-10-18T08:59:58.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.378+0000 7f1f64ff9640 1 --2- 172.21.15.40:0/80410729 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f4806ae40 0x7f1f4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:58.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.378+0000 7f1f64ff9640 1 -- 172.21.15.40:0/80410729 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f600ac090 con 0x7f1f681f2080 2023-10-18T08:59:58.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.378+0000 7f1f677fe640 1 --2- 172.21.15.40:0/80410729 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f4806ae40 0x7f1f4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:58.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.379+0000 7f1f677fe640 1 --2- 172.21.15.40:0/80410729 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f4806ae40 0x7f1f4806d300 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009b10 tx=0x7f1f58005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:58.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.382+0000 7f1f64ff9640 1 -- 172.21.15.40:0/80410729 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f6814b080 con 0x7f1f681f2080 2023-10-18T08:59:58.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.576+0000 7f1f6ddc9640 1 -- 172.21.15.40:0/80410729 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1f6813ca90 con 0x7f1f681f2080 2023-10-18T08:59:58.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.577+0000 7f1f64ff9640 1 -- 172.21.15.40:0/80410729 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1f600770a0 con 0x7f1f681f2080 2023-10-18T08:59:58.579 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T08:59:58.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.579+0000 7f1f6ddc9640 1 -- 172.21.15.40:0/80410729 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f4806ae40 msgr2=0x7f1f4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:58.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.579+0000 7f1f6ddc9640 1 --2- 172.21.15.40:0/80410729 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f4806ae40 0x7f1f4806d300 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009b10 tx=0x7f1f58005f90 comp rx=0 tx=0).stop 2023-10-18T08:59:58.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.580+0000 7f1f6ddc9640 1 -- 172.21.15.40:0/80410729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f681f2080 msgr2=0x7f1f6814b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:58.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.580+0000 7f1f6ddc9640 1 --2- 172.21.15.40:0/80410729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f681f2080 0x7f1f6814b360 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1f60002230 tx=0x7f1f60031740 comp rx=0 tx=0).stop 2023-10-18T08:59:58.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.580+0000 7f1f6ddc9640 1 -- 172.21.15.40:0/80410729 shutdown_connections 2023-10-18T08:59:58.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.580+0000 7f1f6ddc9640 1 --2- 172.21.15.40:0/80410729 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f4806ae40 0x7f1f4806d300 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:58.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.580+0000 7f1f6ddc9640 1 --2- 172.21.15.40:0/80410729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f681f2080 0x7f1f6814b360 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:58.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.580+0000 7f1f6ddc9640 1 --2- 172.21.15.40:0/80410729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f6813ccd0 0x7f1f681f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:58.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.580+0000 7f1f6ddc9640 1 --2- 172.21.15.40:0/80410729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f54004d00 0x7f1f6813c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:58.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.580+0000 7f1f6ddc9640 1 -- 172.21.15.40:0/80410729 >> 172.21.15.40:0/80410729 conn(0x7f1f68069720 msgr2=0x7f1f680cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:58.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.580+0000 7f1f6ddc9640 1 -- 172.21.15.40:0/80410729 shutdown_connections 2023-10-18T08:59:58.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.580+0000 7f1f6ddc9640 1 -- 172.21.15.40:0/80410729 wait complete. 2023-10-18T08:59:58.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:59:58.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.828+0000 7fd7f532c640 1 -- 172.21.15.40:0/1677409583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7dc005180 msgr2=0x7fd7dc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.828+0000 7fd7f532c640 1 --2- 172.21.15.40:0/1677409583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7dc005180 0x7fd7dc00b980 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8001d80 tx=0x7fd7dc0057f0 comp rx=0 tx=0).stop 2023-10-18T08:59:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.828+0000 7fd7f4b2b640 1 -- 172.21.15.40:0/1677409583 reap_dead start 2023-10-18T08:59:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.828+0000 7fd7f532c640 1 -- 172.21.15.40:0/1677409583 shutdown_connections 2023-10-18T08:59:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.828+0000 7fd7f532c640 1 -- 172.21.15.40:0/1677409583 >> 172.21.15.40:0/1677409583 conn(0x7fd7f0069720 msgr2=0x7fd7f0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:58.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.828+0000 7fd7f532c640 1 -- 172.21.15.40:0/1677409583 shutdown_connections 2023-10-18T08:59:58.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.828+0000 7fd7f532c640 1 -- 172.21.15.40:0/1677409583 wait complete. 2023-10-18T08:59:58.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.830+0000 7fd7f532c640 1 Processor -- start 2023-10-18T08:59:58.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.830+0000 7fd7f532c640 1 -- start start 2023-10-18T08:59:58.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.831+0000 7fd7f532c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7dc005180 0x7fd7f0150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:58.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.831+0000 7fd7f532c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7f01499d0 0x7fd7f01f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:58.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.831+0000 7fd7f532c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7f01e5df0 0x7fd7f01e6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:58.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.831+0000 7fd7f532c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f01e67b0 con 0x7fd7f01499d0 2023-10-18T08:59:58.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.831+0000 7fd7f532c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f01e46b0 con 0x7fd7f01e5df0 2023-10-18T08:59:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.831+0000 7fd7f532c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f014a190 con 0x7fd7dc005180 2023-10-18T08:59:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.831+0000 7fd7e7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7f01499d0 0x7fd7f01f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.831+0000 7fd7e7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7f01499d0 0x7fd7f01f4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47788/0 (socket says 172.21.15.40:47788) 2023-10-18T08:59:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.831+0000 7fd7e7fff640 1 -- 172.21.15.40:0/2467319907 learned_addr learned my addr 172.21.15.40:0/2467319907 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.832+0000 7fd7effff640 1 --2- 172.21.15.40:0/2467319907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7dc005180 0x7fd7f0150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.832+0000 7fd7f4b2b640 1 --2- 172.21.15.40:0/2467319907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7f01e5df0 0x7fd7f01e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.832+0000 7fd7e7fff640 1 -- 172.21.15.40:0/2467319907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7dc005180 msgr2=0x7fd7f0150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.832+0000 7fd7e7fff640 1 --2- 172.21.15.40:0/2467319907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7dc005180 0x7fd7f0150c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.832+0000 7fd7e7fff640 1 -- 172.21.15.40:0/2467319907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7f01e5df0 msgr2=0x7fd7f01e6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.832+0000 7fd7e7fff640 1 --2- 172.21.15.40:0/2467319907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7f01e5df0 0x7fd7f01e6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.832+0000 7fd7e7fff640 1 -- 172.21.15.40:0/2467319907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f015c5c0 con 0x7fd7f01499d0 2023-10-18T08:59:58.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.832+0000 7fd7f4b2b640 1 --2- 172.21.15.40:0/2467319907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7f01e5df0 0x7fd7f01e6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:59:58.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.832+0000 7fd7e7fff640 1 --2- 172.21.15.40:0/2467319907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7f01499d0 0x7fd7f01f4bf0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800d9a0 tx=0x7fd7d800de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:58.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.832+0000 7fd7ed7fa640 1 -- 172.21.15.40:0/2467319907 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8006360 con 0x7fd7f01499d0 2023-10-18T08:59:58.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.832+0000 7fd7ed7fa640 1 -- 172.21.15.40:0/2467319907 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd7d800b8f0 con 0x7fd7f01499d0 2023-10-18T08:59:58.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.832+0000 7fd7f532c640 1 -- 172.21.15.40:0/2467319907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f01e3d80 con 0x7fd7f01499d0 2023-10-18T08:59:58.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.833+0000 7fd7ed7fa640 1 -- 172.21.15.40:0/2467319907 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d800bc60 con 0x7fd7f01499d0 2023-10-18T08:59:58.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.833+0000 7fd7f532c640 1 -- 172.21.15.40:0/2467319907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f01e4340 con 0x7fd7f01499d0 2023-10-18T08:59:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.834+0000 7fd7f532c640 1 -- 172.21.15.40:0/2467319907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f00ca610 con 0x7fd7f01499d0 2023-10-18T08:59:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.835+0000 7fd7ed7fa640 1 -- 172.21.15.40:0/2467319907 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd7d80035f0 con 0x7fd7f01499d0 2023-10-18T08:59:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.836+0000 7fd7ed7fa640 1 --2- 172.21.15.40:0/2467319907 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd7d006ae70 0x7fd7d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:58.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.836+0000 7fd7ed7fa640 1 -- 172.21.15.40:0/2467319907 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7d808a180 con 0x7fd7f01499d0 2023-10-18T08:59:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.836+0000 7fd7effff640 1 --2- 172.21.15.40:0/2467319907 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd7d006ae70 0x7fd7d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.837+0000 7fd7effff640 1 --2- 172.21.15.40:0/2467319907 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd7d006ae70 0x7fd7d006d330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0005fd0 tx=0x7fd7e0005c70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:58.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:58.840+0000 7fd7ed7fa640 1 -- 172.21.15.40:0/2467319907 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd7d8058cb0 con 0x7fd7f01499d0 2023-10-18T08:59:59.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.068+0000 7fd7f532c640 1 -- 172.21.15.40:0/2467319907 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7f0066e00 con 0x7fd7f01499d0 2023-10-18T08:59:59.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.070+0000 7fd7ed7fa640 1 -- 172.21.15.40:0/2467319907 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fd7d8058840 con 0x7fd7f01499d0 2023-10-18T08:59:59.072 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:59:59.073 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5075,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2165596806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2165596806},{"type":"v1","addr":"172.21.15.144:6835","nonce":2165596806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/587522883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":587522883},{"type":"v1","addr":"172.21.15.40:6833","nonce":587522883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1174134524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1174134524},{"type":"v1","addr":"172.21.15.40:6835","nonce":1174134524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:58:28.760677+0000","modified":"2023-10-18T08:58:31.845684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5389":{"gid":5389,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1311290970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1311290970},{"type":"v1","addr":"172.21.15.144:6837","nonce":1311290970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-18T08:59:59.073 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 33 2023-10-18T08:59:59.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.072+0000 7fd7f532c640 1 -- 172.21.15.40:0/2467319907 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd7d006ae70 msgr2=0x7fd7d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:59.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.072+0000 7fd7f532c640 1 --2- 172.21.15.40:0/2467319907 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd7d006ae70 0x7fd7d006d330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0005fd0 tx=0x7fd7e0005c70 comp rx=0 tx=0).stop 2023-10-18T08:59:59.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.072+0000 7fd7f532c640 1 -- 172.21.15.40:0/2467319907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7f01499d0 msgr2=0x7fd7f01f4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:59.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.072+0000 7fd7f532c640 1 --2- 172.21.15.40:0/2467319907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7f01499d0 0x7fd7f01f4bf0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800d9a0 tx=0x7fd7d800de60 comp rx=0 tx=0).stop 2023-10-18T08:59:59.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.073+0000 7fd7f532c640 1 -- 172.21.15.40:0/2467319907 shutdown_connections 2023-10-18T08:59:59.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.073+0000 7fd7f532c640 1 --2- 172.21.15.40:0/2467319907 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd7d006ae70 0x7fd7d006d330 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.073+0000 7fd7f532c640 1 --2- 172.21.15.40:0/2467319907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7f01e5df0 0x7fd7f01e6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.073+0000 7fd7f532c640 1 --2- 172.21.15.40:0/2467319907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7f01499d0 0x7fd7f01f4bf0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.073+0000 7fd7f532c640 1 --2- 172.21.15.40:0/2467319907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7dc005180 0x7fd7f0150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.073+0000 7fd7f532c640 1 -- 172.21.15.40:0/2467319907 >> 172.21.15.40:0/2467319907 conn(0x7fd7f0069720 msgr2=0x7fd7f014bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:59.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.073+0000 7fd7f532c640 1 -- 172.21.15.40:0/2467319907 shutdown_connections 2023-10-18T08:59:59.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.073+0000 7fd7f532c640 1 -- 172.21.15.40:0/2467319907 wait complete. 2023-10-18T08:59:59.094 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T08:59:59.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.327+0000 7fcb63d97640 1 -- 172.21.15.40:0/137211198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb5c0cc990 msgr2=0x7fcb5c0ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:59.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.327+0000 7fcb63d97640 1 --2- 172.21.15.40:0/137211198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb5c0cc990 0x7fcb5c0ccde0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcb54005fd0 tx=0x7fcb5402c2e0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.327+0000 7fcb63d97640 1 -- 172.21.15.40:0/137211198 shutdown_connections 2023-10-18T08:59:59.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.327+0000 7fcb63d97640 1 --2- 172.21.15.40:0/137211198 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb5c0cc990 0x7fcb5c0ccde0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.327+0000 7fcb63d97640 1 --2- 172.21.15.40:0/137211198 >> v2:172.21.15.144:3300/0 conn(0x7fcb5c0b0bc0 0x7fcb5c0b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.327+0000 7fcb63d97640 1 -- 172.21.15.40:0/137211198 >> 172.21.15.40:0/137211198 conn(0x7fcb5c06cee0 msgr2=0x7fcb5c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:59.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.328+0000 7fcb63d97640 1 -- 172.21.15.40:0/137211198 shutdown_connections 2023-10-18T08:59:59.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.328+0000 7fcb63d97640 1 -- 172.21.15.40:0/137211198 wait complete. 2023-10-18T08:59:59.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.330+0000 7fcb63d97640 1 Processor -- start 2023-10-18T08:59:59.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.330+0000 7fcb63d97640 1 -- start start 2023-10-18T08:59:59.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.332+0000 7fcb63d97640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb5c0b0bc0 0x7fcb5c1455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:59.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.332+0000 7fcb63d97640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5c0cc990 0x7fcb5c156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:59.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.332+0000 7fcb63d97640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb5c151bd0 0x7fcb5c156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:59.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.332+0000 7fcb63d97640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c159010 con 0x7fcb5c0cc990 2023-10-18T08:59:59.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.332+0000 7fcb63d97640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c158d10 con 0x7fcb5c0b0bc0 2023-10-18T08:59:59.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.332+0000 7fcb63d97640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c1594a0 con 0x7fcb5c151bd0 2023-10-18T08:59:59.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.332+0000 7fcb61332640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5c0cc990 0x7fcb5c156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:59.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.332+0000 7fcb61332640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5c0cc990 0x7fcb5c156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47800/0 (socket says 172.21.15.40:47800) 2023-10-18T08:59:59.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.332+0000 7fcb61332640 1 -- 172.21.15.40:0/3544824009 learned_addr learned my addr 172.21.15.40:0/3544824009 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:59.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.332+0000 7fcb62334640 1 --2- 172.21.15.40:0/3544824009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb5c151bd0 0x7fcb5c156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:59.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.332+0000 7fcb61b33640 1 --2- 172.21.15.40:0/3544824009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb5c0b0bc0 0x7fcb5c1455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:59.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.333+0000 7fcb61332640 1 -- 172.21.15.40:0/3544824009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb5c151bd0 msgr2=0x7fcb5c156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:59.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.333+0000 7fcb61332640 1 --2- 172.21.15.40:0/3544824009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb5c151bd0 0x7fcb5c156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.333+0000 7fcb61332640 1 -- 172.21.15.40:0/3544824009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb5c0b0bc0 msgr2=0x7fcb5c1455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:59.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.333+0000 7fcb61332640 1 --2- 172.21.15.40:0/3544824009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb5c0b0bc0 0x7fcb5c1455c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.333+0000 7fcb61332640 1 -- 172.21.15.40:0/3544824009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb54009900 con 0x7fcb5c0cc990 2023-10-18T08:59:59.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.333+0000 7fcb62334640 1 --2- 172.21.15.40:0/3544824009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb5c151bd0 0x7fcb5c156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T08:59:59.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.333+0000 7fcb61332640 1 --2- 172.21.15.40:0/3544824009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5c0cc990 0x7fcb5c156600 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00ea20 tx=0x7fcb4c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:59.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.334+0000 7fcb527fc640 1 -- 172.21.15.40:0/3544824009 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c009700 con 0x7fcb5c0cc990 2023-10-18T08:59:59.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.334+0000 7fcb527fc640 1 -- 172.21.15.40:0/3544824009 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb4c00ce70 con 0x7fcb5c0cc990 2023-10-18T08:59:59.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.334+0000 7fcb527fc640 1 -- 172.21.15.40:0/3544824009 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c010690 con 0x7fcb5c0cc990 2023-10-18T08:59:59.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.334+0000 7fcb63d97640 1 -- 172.21.15.40:0/3544824009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5c0cb270 con 0x7fcb5c0cc990 2023-10-18T08:59:59.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.334+0000 7fcb63d97640 1 -- 172.21.15.40:0/3544824009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c140400 con 0x7fcb5c0cc990 2023-10-18T08:59:59.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.335+0000 7fcb63d97640 1 -- 172.21.15.40:0/3544824009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb5c066fd0 con 0x7fcb5c0cc990 2023-10-18T08:59:59.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.336+0000 7fcb527fc640 1 -- 172.21.15.40:0/3544824009 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcb4c0107f0 con 0x7fcb5c0cc990 2023-10-18T08:59:59.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.337+0000 7fcb527fc640 1 --2- 172.21.15.40:0/3544824009 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcb4006ae70 0x7fcb4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:59.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.338+0000 7fcb527fc640 1 -- 172.21.15.40:0/3544824009 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb4c014070 con 0x7fcb5c0cc990 2023-10-18T08:59:59.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.338+0000 7fcb61b33640 1 --2- 172.21.15.40:0/3544824009 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcb4006ae70 0x7fcb4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:59.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.339+0000 7fcb61b33640 1 --2- 172.21.15.40:0/3544824009 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcb4006ae70 0x7fcb4006d330 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcb58000f60 tx=0x7fcb58009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:59.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.342+0000 7fcb527fc640 1 -- 172.21.15.40:0/3544824009 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb4c051ca0 con 0x7fcb5c0cc990 2023-10-18T08:59:59.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.536+0000 7fcb63d97640 1 -- 172.21.15.40:0/3544824009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb5c158e50 con 0x7fcb5c0cc990 2023-10-18T08:59:59.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.539+0000 7fcb527fc640 1 -- 172.21.15.40:0/3544824009 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16577 (secure 0 0 0) 0x7fcb5c158e50 con 0x7fcb5c0cc990 2023-10-18T08:59:59.541 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T08:59:59.542 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":76,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:59:57.993496+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:58:26.727620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-18T08:58:27.795303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T08:59:59.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.541+0000 7fcb63d97640 1 -- 172.21.15.40:0/3544824009 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcb4006ae70 msgr2=0x7fcb4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:59.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.541+0000 7fcb63d97640 1 --2- 172.21.15.40:0/3544824009 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcb4006ae70 0x7fcb4006d330 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcb58000f60 tx=0x7fcb58009040 comp rx=0 tx=0).stop 2023-10-18T08:59:59.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.541+0000 7fcb63d97640 1 -- 172.21.15.40:0/3544824009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5c0cc990 msgr2=0x7fcb5c156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:59.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.541+0000 7fcb63d97640 1 --2- 172.21.15.40:0/3544824009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5c0cc990 0x7fcb5c156600 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00ea20 tx=0x7fcb4c00eee0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.541+0000 7fcb63d97640 1 -- 172.21.15.40:0/3544824009 shutdown_connections 2023-10-18T08:59:59.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.541+0000 7fcb63d97640 1 --2- 172.21.15.40:0/3544824009 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcb4006ae70 0x7fcb4006d330 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.541+0000 7fcb63d97640 1 --2- 172.21.15.40:0/3544824009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb5c151bd0 0x7fcb5c156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.541+0000 7fcb63d97640 1 --2- 172.21.15.40:0/3544824009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb5c0cc990 0x7fcb5c156600 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.541+0000 7fcb63d97640 1 --2- 172.21.15.40:0/3544824009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb5c0b0bc0 0x7fcb5c1455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.541+0000 7fcb63d97640 1 -- 172.21.15.40:0/3544824009 >> 172.21.15.40:0/3544824009 conn(0x7fcb5c06cee0 msgr2=0x7fcb5c0ae0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:59.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.542+0000 7fcb63d97640 1 -- 172.21.15.40:0/3544824009 shutdown_connections 2023-10-18T08:59:59.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.542+0000 7fcb63d97640 1 -- 172.21.15.40:0/3544824009 wait complete. 2023-10-18T08:59:59.562 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T08:59:59.562 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T08:59:59.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.776+0000 7fee233de640 1 -- 172.21.15.40:0/3423985438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee08003970 msgr2=0x7fee08003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:59.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.776+0000 7fee233de640 1 --2- 172.21.15.40:0/3423985438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee08003970 0x7fee08003d70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fee1c1555e0 tx=0x7fee080057f0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.776+0000 7fee233de640 1 -- 172.21.15.40:0/3423985438 shutdown_connections 2023-10-18T08:59:59.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.776+0000 7fee233de640 1 --2- 172.21.15.40:0/3423985438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee08003970 0x7fee08003d70 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.776+0000 7fee233de640 1 -- 172.21.15.40:0/3423985438 >> 172.21.15.40:0/3423985438 conn(0x7fee1c069720 msgr2=0x7fee1c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T08:59:59.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.776+0000 7fee233de640 1 -- 172.21.15.40:0/3423985438 shutdown_connections 2023-10-18T08:59:59.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.776+0000 7fee233de640 1 -- 172.21.15.40:0/3423985438 wait complete. 2023-10-18T08:59:59.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.778+0000 7fee233de640 1 Processor -- start 2023-10-18T08:59:59.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.778+0000 7fee233de640 1 -- start start 2023-10-18T08:59:59.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.779+0000 7fee233de640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee08003970 0x7fee1c14dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:59.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.779+0000 7fee233de640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee1c0acb20 0x7fee1c0cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:59.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.779+0000 7fee233de640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee1c0cbec0 0x7fee1c150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:59.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.779+0000 7fee233de640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c0adbd0 con 0x7fee1c0acb20 2023-10-18T08:59:59.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.779+0000 7fee233de640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c0cc340 con 0x7fee08003970 2023-10-18T08:59:59.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.779+0000 7fee233de640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c14e0f0 con 0x7fee1c0cbec0 2023-10-18T08:59:59.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.779+0000 7fee21bdb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee1c0acb20 0x7fee1c0cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:59.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.779+0000 7fee21bdb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee1c0acb20 0x7fee1c0cb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47818/0 (socket says 172.21.15.40:47818) 2023-10-18T08:59:59.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.779+0000 7fee21bdb640 1 -- 172.21.15.40:0/104519488 learned_addr learned my addr 172.21.15.40:0/104519488 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T08:59:59.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.780+0000 7fee223dc640 1 --2- 172.21.15.40:0/104519488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee08003970 0x7fee1c14dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:59.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.780+0000 7fee22bdd640 1 --2- 172.21.15.40:0/104519488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee1c0cbec0 0x7fee1c150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:59.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.780+0000 7fee21bdb640 1 -- 172.21.15.40:0/104519488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee1c0cbec0 msgr2=0x7fee1c150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:59.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.780+0000 7fee21bdb640 1 --2- 172.21.15.40:0/104519488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee1c0cbec0 0x7fee1c150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.780+0000 7fee21bdb640 1 -- 172.21.15.40:0/104519488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee08003970 msgr2=0x7fee1c14dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T08:59:59.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.780+0000 7fee21bdb640 1 --2- 172.21.15.40:0/104519488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee08003970 0x7fee1c14dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T08:59:59.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.780+0000 7fee21bdb640 1 -- 172.21.15.40:0/104519488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c153750 con 0x7fee1c0acb20 2023-10-18T08:59:59.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.780+0000 7fee22bdd640 1 --2- 172.21.15.40:0/104519488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee1c0cbec0 0x7fee1c150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T08:59:59.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.780+0000 7fee21bdb640 1 --2- 172.21.15.40:0/104519488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee1c0acb20 0x7fee1c0cb980 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00b530 tx=0x7fee0c00b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:59.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.780+0000 7fee12ffd640 1 -- 172.21.15.40:0/104519488 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c006300 con 0x7fee1c0acb20 2023-10-18T08:59:59.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.780+0000 7fee12ffd640 1 -- 172.21.15.40:0/104519488 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fee0c0038b0 con 0x7fee1c0acb20 2023-10-18T08:59:59.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.780+0000 7fee233de640 1 -- 172.21.15.40:0/104519488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1c153470 con 0x7fee1c0acb20 2023-10-18T08:59:59.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.781+0000 7fee12ffd640 1 -- 172.21.15.40:0/104519488 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c003cc0 con 0x7fee1c0acb20 2023-10-18T08:59:59.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.781+0000 7fee233de640 1 -- 172.21.15.40:0/104519488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1c0ad7f0 con 0x7fee1c0acb20 2023-10-18T08:59:59.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.782+0000 7fee233de640 1 -- 172.21.15.40:0/104519488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee1c0ad9b0 con 0x7fee1c0acb20 2023-10-18T08:59:59.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.783+0000 7fee12ffd640 1 -- 172.21.15.40:0/104519488 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fee0c00f470 con 0x7fee1c0acb20 2023-10-18T08:59:59.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.784+0000 7fee12ffd640 1 --2- 172.21.15.40:0/104519488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fee0006ae70 0x7fee0006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T08:59:59.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.784+0000 7fee12ffd640 1 -- 172.21.15.40:0/104519488 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee0c089cd0 con 0x7fee1c0acb20 2023-10-18T08:59:59.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.785+0000 7fee223dc640 1 --2- 172.21.15.40:0/104519488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fee0006ae70 0x7fee0006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T08:59:59.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.786+0000 7fee223dc640 1 --2- 172.21.15.40:0/104519488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fee0006ae70 0x7fee0006d330 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fee18034000 tx=0x7fee1802e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T08:59:59.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T08:59:59.788+0000 7fee12ffd640 1 -- 172.21.15.40:0/104519488 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee0c054db0 con 0x7fee1c0acb20 2023-10-18T09:00:00.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.022+0000 7fee233de640 1 -- 172.21.15.40:0/104519488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee1c066e00 con 0x7fee1c0acb20 2023-10-18T09:00:00.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.023+0000 7fee12ffd640 1 -- 172.21.15.40:0/104519488 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fee0c0583d0 con 0x7fee1c0acb20 2023-10-18T09:00:00.026 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:00.026 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5075,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2165596806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2165596806},{"type":"v1","addr":"172.21.15.144:6835","nonce":2165596806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/587522883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":587522883},{"type":"v1","addr":"172.21.15.40:6833","nonce":587522883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1174134524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1174134524},{"type":"v1","addr":"172.21.15.40:6835","nonce":1174134524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:58:28.760677+0000","modified":"2023-10-18T08:58:31.845684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5389":{"gid":5389,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1311290970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1311290970},{"type":"v1","addr":"172.21.15.144:6837","nonce":1311290970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-18T09:00:00.027 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 33 2023-10-18T09:00:00.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.026+0000 7fee233de640 1 -- 172.21.15.40:0/104519488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fee0006ae70 msgr2=0x7fee0006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:00.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.026+0000 7fee233de640 1 --2- 172.21.15.40:0/104519488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fee0006ae70 0x7fee0006d330 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fee18034000 tx=0x7fee1802e000 comp rx=0 tx=0).stop 2023-10-18T09:00:00.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.026+0000 7fee233de640 1 -- 172.21.15.40:0/104519488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee1c0acb20 msgr2=0x7fee1c0cb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:00.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.026+0000 7fee233de640 1 --2- 172.21.15.40:0/104519488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee1c0acb20 0x7fee1c0cb980 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00b530 tx=0x7fee0c00b9f0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.026+0000 7fee233de640 1 -- 172.21.15.40:0/104519488 shutdown_connections 2023-10-18T09:00:00.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.026+0000 7fee233de640 1 --2- 172.21.15.40:0/104519488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fee0006ae70 0x7fee0006d330 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.026+0000 7fee233de640 1 --2- 172.21.15.40:0/104519488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee1c0cbec0 0x7fee1c150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.026+0000 7fee233de640 1 --2- 172.21.15.40:0/104519488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee1c0acb20 0x7fee1c0cb980 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.026+0000 7fee233de640 1 --2- 172.21.15.40:0/104519488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee08003970 0x7fee1c14dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.026+0000 7fee233de640 1 -- 172.21.15.40:0/104519488 >> 172.21.15.40:0/104519488 conn(0x7fee1c069720 msgr2=0x7fee1c159f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:00.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.027+0000 7fee233de640 1 -- 172.21.15.40:0/104519488 shutdown_connections 2023-10-18T09:00:00.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.027+0000 7fee233de640 1 -- 172.21.15.40:0/104519488 wait complete. 2023-10-18T09:00:00.049 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:00:00.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.276+0000 7fa006ac2640 1 -- 172.21.15.40:0/240527408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0000b12d0 msgr2=0x7fa0001625a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:00.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.276+0000 7fa006ac2640 1 --2- 172.21.15.40:0/240527408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0000b12d0 0x7fa0001625a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8005fd0 tx=0x7f9ff802c2d0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.276+0000 7fa006ac2640 1 -- 172.21.15.40:0/240527408 shutdown_connections 2023-10-18T09:00:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.276+0000 7fa006ac2640 1 --2- 172.21.15.40:0/240527408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0000b12d0 0x7fa0001625a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.276+0000 7fa006ac2640 1 --2- 172.21.15.40:0/240527408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa00006d710 0x7fa000151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.276+0000 7fa006ac2640 1 -- 172.21.15.40:0/240527408 >> 172.21.15.40:0/240527408 conn(0x7fa00006cee0 msgr2=0x7fa000069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.277+0000 7fa006ac2640 1 -- 172.21.15.40:0/240527408 shutdown_connections 2023-10-18T09:00:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.277+0000 7fa006ac2640 1 -- 172.21.15.40:0/240527408 wait complete. 2023-10-18T09:00:00.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.278+0000 7fa006ac2640 1 Processor -- start 2023-10-18T09:00:00.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.278+0000 7fa006ac2640 1 -- start start 2023-10-18T09:00:00.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.279+0000 7fa006ac2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa00006d710 0x7fa00015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:00.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.279+0000 7fa006ac2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0000b12d0 0x7fa000139990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:00.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.279+0000 7fa006ac2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0001586a0 0x7fa00014eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:00.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.279+0000 7fa006ac2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa000139ed0 con 0x7fa00006d710 2023-10-18T09:00:00.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.279+0000 7fa006ac2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00014a260 con 0x7fa0000b12d0 2023-10-18T09:00:00.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.279+0000 7fa006ac2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00014a540 con 0x7fa0001586a0 2023-10-18T09:00:00.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.280+0000 7fa00485e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa00006d710 0x7fa00015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:00.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.280+0000 7fa00485e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa00006d710 0x7fa00015a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47842/0 (socket says 172.21.15.40:47842) 2023-10-18T09:00:00.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.280+0000 7fa00485e640 1 -- 172.21.15.40:0/86117763 learned_addr learned my addr 172.21.15.40:0/86117763 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:00.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.280+0000 7f9ffffff640 1 --2- 172.21.15.40:0/86117763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0000b12d0 0x7fa000139990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:00.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.280+0000 7fa00505f640 1 --2- 172.21.15.40:0/86117763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0001586a0 0x7fa00014eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:00.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.280+0000 7fa00485e640 1 -- 172.21.15.40:0/86117763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0001586a0 msgr2=0x7fa00014eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:00.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.280+0000 7fa00485e640 1 --2- 172.21.15.40:0/86117763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0001586a0 0x7fa00014eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.280+0000 7fa00485e640 1 -- 172.21.15.40:0/86117763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0000b12d0 msgr2=0x7fa000139990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:00.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.280+0000 7fa00485e640 1 --2- 172.21.15.40:0/86117763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0000b12d0 0x7fa000139990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.280+0000 7fa00485e640 1 -- 172.21.15.40:0/86117763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff8009900 con 0x7fa00006d710 2023-10-18T09:00:00.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.280+0000 7fa00505f640 1 --2- 172.21.15.40:0/86117763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0001586a0 0x7fa00014eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.280+0000 7f9ffffff640 1 --2- 172.21.15.40:0/86117763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0000b12d0 0x7fa000139990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.281+0000 7fa00485e640 1 --2- 172.21.15.40:0/86117763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa00006d710 0x7fa00015a970 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400d770 tx=0x7f9ff400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.281+0000 7f9ffd7fa640 1 -- 172.21.15.40:0/86117763 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4009c30 con 0x7fa00006d710 2023-10-18T09:00:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.281+0000 7f9ffd7fa640 1 -- 172.21.15.40:0/86117763 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9ff400de80 con 0x7fa00006d710 2023-10-18T09:00:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.281+0000 7f9ffd7fa640 1 -- 172.21.15.40:0/86117763 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4005280 con 0x7fa00006d710 2023-10-18T09:00:00.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.281+0000 7fa006ac2640 1 -- 172.21.15.40:0/86117763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa00014a820 con 0x7fa00006d710 2023-10-18T09:00:00.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.281+0000 7fa006ac2640 1 -- 172.21.15.40:0/86117763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa00014b020 con 0x7fa00006d710 2023-10-18T09:00:00.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.282+0000 7fa006ac2640 1 -- 172.21.15.40:0/86117763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa000066fd0 con 0x7fa00006d710 2023-10-18T09:00:00.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.283+0000 7f9ffd7fa640 1 -- 172.21.15.40:0/86117763 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9ff40053e0 con 0x7fa00006d710 2023-10-18T09:00:00.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.284+0000 7f9ffd7fa640 1 --2- 172.21.15.40:0/86117763 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9fe406ae40 0x7f9fe406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:00.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.285+0000 7f9ffd7fa640 1 -- 172.21.15.40:0/86117763 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ff4089910 con 0x7fa00006d710 2023-10-18T09:00:00.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.285+0000 7f9ffffff640 1 --2- 172.21.15.40:0/86117763 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9fe406ae40 0x7f9fe406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.286+0000 7f9ffffff640 1 --2- 172.21.15.40:0/86117763 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9fe406ae40 0x7f9fe406d300 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa00006bd40 tx=0x7f9fec009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:00.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.289+0000 7f9ffd7fa640 1 -- 172.21.15.40:0/86117763 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ff4054920 con 0x7fa00006d710 2023-10-18T09:00:00.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.512+0000 7fa006ac2640 1 -- 172.21.15.40:0/86117763 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa00006fb50 con 0x7fa00006d710 2023-10-18T09:00:00.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.514+0000 7f9ffd7fa640 1 -- 172.21.15.40:0/86117763 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f9ff4017020 con 0x7fa00006d710 2023-10-18T09:00:00.516 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:00.516 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5075,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2165596806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2165596806},{"type":"v1","addr":"172.21.15.144:6835","nonce":2165596806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/587522883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":587522883},{"type":"v1","addr":"172.21.15.40:6833","nonce":587522883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1174134524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1174134524},{"type":"v1","addr":"172.21.15.40:6835","nonce":1174134524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T08:58:28.760677+0000","modified":"2023-10-18T08:58:31.845684+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5389},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5389":{"gid":5389,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6837/1311290970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":1311290970},{"type":"v1","addr":"172.21.15.144:6837","nonce":1311290970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-18T09:00:00.517 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 33 2023-10-18T09:00:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.516+0000 7fa006ac2640 1 -- 172.21.15.40:0/86117763 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9fe406ae40 msgr2=0x7f9fe406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.516+0000 7fa006ac2640 1 --2- 172.21.15.40:0/86117763 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9fe406ae40 0x7f9fe406d300 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa00006bd40 tx=0x7f9fec009040 comp rx=0 tx=0).stop 2023-10-18T09:00:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.516+0000 7fa006ac2640 1 -- 172.21.15.40:0/86117763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa00006d710 msgr2=0x7fa00015a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:00.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.516+0000 7fa006ac2640 1 --2- 172.21.15.40:0/86117763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa00006d710 0x7fa00015a970 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400d770 tx=0x7f9ff400dc30 comp rx=0 tx=0).stop 2023-10-18T09:00:00.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.516+0000 7fa006ac2640 1 -- 172.21.15.40:0/86117763 shutdown_connections 2023-10-18T09:00:00.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.516+0000 7fa006ac2640 1 --2- 172.21.15.40:0/86117763 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9fe406ae40 0x7f9fe406d300 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.516+0000 7fa006ac2640 1 --2- 172.21.15.40:0/86117763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0001586a0 0x7fa00014eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.516+0000 7fa006ac2640 1 --2- 172.21.15.40:0/86117763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0000b12d0 0x7fa000139990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.516+0000 7fa006ac2640 1 --2- 172.21.15.40:0/86117763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa00006d710 0x7fa00015a970 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.516+0000 7fa006ac2640 1 -- 172.21.15.40:0/86117763 >> 172.21.15.40:0/86117763 conn(0x7fa00006cee0 msgr2=0x7fa0000cd220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:00.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.516+0000 7fa006ac2640 1 -- 172.21.15.40:0/86117763 shutdown_connections 2023-10-18T09:00:00.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.516+0000 7fa006ac2640 1 -- 172.21.15.40:0/86117763 wait complete. 2023-10-18T09:00:00.535 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:00:00.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.748+0000 7f23e744d640 1 -- 172.21.15.40:0/3353182613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23cc0039a0 msgr2=0x7f23cc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:00.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.748+0000 7f23e744d640 1 --2- 172.21.15.40:0/3353182613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23cc0039a0 0x7f23cc003dc0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f23dc006fa0 tx=0x7f23cc0057f0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.748+0000 7f23e6c4c640 1 -- 172.21.15.40:0/3353182613 reap_dead start 2023-10-18T09:00:00.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.748+0000 7f23e744d640 1 -- 172.21.15.40:0/3353182613 shutdown_connections 2023-10-18T09:00:00.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.748+0000 7f23e744d640 1 -- 172.21.15.40:0/3353182613 >> 172.21.15.40:0/3353182613 conn(0x7f23e0069720 msgr2=0x7f23e0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:00.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.748+0000 7f23e744d640 1 -- 172.21.15.40:0/3353182613 shutdown_connections 2023-10-18T09:00:00.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.748+0000 7f23e744d640 1 -- 172.21.15.40:0/3353182613 wait complete. 2023-10-18T09:00:00.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.750+0000 7f23e744d640 1 Processor -- start 2023-10-18T09:00:00.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.750+0000 7f23e744d640 1 -- start start 2023-10-18T09:00:00.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.751+0000 7f23e744d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23cc0039a0 0x7f23e01f2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:00.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.751+0000 7f23e744d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23e013dc60 0x7f23e013d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:00.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.751+0000 7f23e744d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23e00ac970 0x7f23e0159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:00.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.751+0000 7f23e744d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e01f2cc0 con 0x7f23e013dc60 2023-10-18T09:00:00.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.751+0000 7f23e744d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e013dac0 con 0x7f23cc0039a0 2023-10-18T09:00:00.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.751+0000 7f23e744d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00af5f0 con 0x7f23e00ac970 2023-10-18T09:00:00.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.752+0000 7f23e5c4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23e013dc60 0x7f23e013d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:00.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.752+0000 7f23e5c4a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23e013dc60 0x7f23e013d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47862/0 (socket says 172.21.15.40:47862) 2023-10-18T09:00:00.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.752+0000 7f23e5c4a640 1 -- 172.21.15.40:0/3301022290 learned_addr learned my addr 172.21.15.40:0/3301022290 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:00.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.752+0000 7f23e6c4c640 1 --2- 172.21.15.40:0/3301022290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23e00ac970 0x7f23e0159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:00.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.752+0000 7f23e644b640 1 --2- 172.21.15.40:0/3301022290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23cc0039a0 0x7f23e01f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:00.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.752+0000 7f23e5c4a640 1 -- 172.21.15.40:0/3301022290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23e00ac970 msgr2=0x7f23e0159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:00.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.752+0000 7f23e5c4a640 1 --2- 172.21.15.40:0/3301022290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23e00ac970 0x7f23e0159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.752+0000 7f23e5c4a640 1 -- 172.21.15.40:0/3301022290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23cc0039a0 msgr2=0x7f23e01f2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:00.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.752+0000 7f23e5c4a640 1 --2- 172.21.15.40:0/3301022290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23cc0039a0 0x7f23e01f2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.752+0000 7f23e5c4a640 1 -- 172.21.15.40:0/3301022290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e00af8d0 con 0x7f23e013dc60 2023-10-18T09:00:00.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.753+0000 7f23e5c4a640 1 --2- 172.21.15.40:0/3301022290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23e013dc60 0x7f23e013d580 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f23d0004e50 tx=0x7f23d0001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:00.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.753+0000 7f23d6ffd640 1 -- 172.21.15.40:0/3301022290 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d00107d0 con 0x7f23e013dc60 2023-10-18T09:00:00.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.753+0000 7f23d6ffd640 1 -- 172.21.15.40:0/3301022290 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f23d0010a40 con 0x7f23e013dc60 2023-10-18T09:00:00.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.753+0000 7f23d6ffd640 1 -- 172.21.15.40:0/3301022290 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0010e50 con 0x7f23e013dc60 2023-10-18T09:00:00.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.753+0000 7f23e744d640 1 -- 172.21.15.40:0/3301022290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e015b420 con 0x7f23e013dc60 2023-10-18T09:00:00.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.753+0000 7f23e744d640 1 -- 172.21.15.40:0/3301022290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e015b9e0 con 0x7f23e013dc60 2023-10-18T09:00:00.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.754+0000 7f23e744d640 1 -- 172.21.15.40:0/3301022290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e015bba0 con 0x7f23e013dc60 2023-10-18T09:00:00.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.755+0000 7f23d6ffd640 1 -- 172.21.15.40:0/3301022290 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f23d001b460 con 0x7f23e013dc60 2023-10-18T09:00:00.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.756+0000 7f23d6ffd640 1 --2- 172.21.15.40:0/3301022290 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f23c406ae70 0x7f23c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:00.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.756+0000 7f23d6ffd640 1 -- 172.21.15.40:0/3301022290 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23d008a0d0 con 0x7f23e013dc60 2023-10-18T09:00:00.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.757+0000 7f23e644b640 1 --2- 172.21.15.40:0/3301022290 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f23c406ae70 0x7f23c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:00.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.758+0000 7f23e644b640 1 --2- 172.21.15.40:0/3301022290 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f23c406ae70 0x7f23c406d330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f23dc034000 tx=0x7f23dc039040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:00.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.761+0000 7f23d6ffd640 1 -- 172.21.15.40:0/3301022290 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f23d00550e0 con 0x7f23e013dc60 2023-10-18T09:00:00.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.955+0000 7f23e744d640 1 -- 172.21.15.40:0/3301022290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23e015dc20 con 0x7f23e013dc60 2023-10-18T09:00:00.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.957+0000 7f23d6ffd640 1 -- 172.21.15.40:0/3301022290 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16577 (secure 0 0 0) 0x7f23d0001d40 con 0x7f23e013dc60 2023-10-18T09:00:00.960 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:00.960 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":76,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T08:59:57.993496+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-18T08:58:26.727620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-18T08:58:27.795303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:00:00.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.960+0000 7f23e744d640 1 -- 172.21.15.40:0/3301022290 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f23c406ae70 msgr2=0x7f23c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.960+0000 7f23e744d640 1 --2- 172.21.15.40:0/3301022290 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f23c406ae70 0x7f23c406d330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f23dc034000 tx=0x7f23dc039040 comp rx=0 tx=0).stop 2023-10-18T09:00:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.960+0000 7f23e744d640 1 -- 172.21.15.40:0/3301022290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23e013dc60 msgr2=0x7f23e013d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.960+0000 7f23e744d640 1 --2- 172.21.15.40:0/3301022290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23e013dc60 0x7f23e013d580 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f23d0004e50 tx=0x7f23d0001630 comp rx=0 tx=0).stop 2023-10-18T09:00:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.960+0000 7f23e744d640 1 -- 172.21.15.40:0/3301022290 shutdown_connections 2023-10-18T09:00:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.960+0000 7f23e744d640 1 --2- 172.21.15.40:0/3301022290 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f23c406ae70 0x7f23c406d330 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.960+0000 7f23e744d640 1 --2- 172.21.15.40:0/3301022290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23e00ac970 0x7f23e0159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.960+0000 7f23e744d640 1 --2- 172.21.15.40:0/3301022290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23e013dc60 0x7f23e013d580 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.960+0000 7f23e744d640 1 --2- 172.21.15.40:0/3301022290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23cc0039a0 0x7f23e01f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:00.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.960+0000 7f23e744d640 1 -- 172.21.15.40:0/3301022290 >> 172.21.15.40:0/3301022290 conn(0x7f23e0069720 msgr2=0x7f23e0070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:00.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.960+0000 7f23e744d640 1 -- 172.21.15.40:0/3301022290 shutdown_connections 2023-10-18T09:00:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:00.960+0000 7f23e744d640 1 -- 172.21.15.40:0/3301022290 wait complete. 2023-10-18T09:00:00.982 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T09:00:01.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.208+0000 7f8b99267640 1 -- 172.21.15.40:0/1567392228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b940b01e0 msgr2=0x7f8b940cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:01.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.208+0000 7f8b99267640 1 --2- 172.21.15.40:0/1567392228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b940b01e0 0x7f8b940cf910 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009830 tx=0x7f8b7c02bc30 comp rx=0 tx=0).stop 2023-10-18T09:00:01.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.208+0000 7f8b99267640 1 -- 172.21.15.40:0/1567392228 shutdown_connections 2023-10-18T09:00:01.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.208+0000 7f8b99267640 1 --2- 172.21.15.40:0/1567392228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b940b01e0 0x7f8b940cf910 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:01.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.208+0000 7f8b99267640 1 --2- 172.21.15.40:0/1567392228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b9415b640 0x7f8b940d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:01.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.208+0000 7f8b99267640 1 -- 172.21.15.40:0/1567392228 >> 172.21.15.40:0/1567392228 conn(0x7f8b9408ad00 msgr2=0x7f8b9409a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:01.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.208+0000 7f8b99267640 1 -- 172.21.15.40:0/1567392228 shutdown_connections 2023-10-18T09:00:01.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.208+0000 7f8b99267640 1 -- 172.21.15.40:0/1567392228 wait complete. 2023-10-18T09:00:01.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.210+0000 7f8b99267640 1 Processor -- start 2023-10-18T09:00:01.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.210+0000 7f8b99267640 1 -- start start 2023-10-18T09:00:01.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.211+0000 7f8b99267640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b940b01e0 0x7f8b94149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:01.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.211+0000 7f8b99267640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b940cfe50 0x7f8b9415bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:01.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.211+0000 7f8b99267640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9415b640 0x7f8b9415d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:01.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.211+0000 7f8b99267640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b940caa20 con 0x7f8b9415b640 2023-10-18T09:00:01.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.211+0000 7f8b99267640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b94149b60 con 0x7f8b940cfe50 2023-10-18T09:00:01.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.211+0000 7f8b99267640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9415d100 con 0x7f8b940b01e0 2023-10-18T09:00:01.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.211+0000 7f8b98a66640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9415b640 0x7f8b9415d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:01.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.211+0000 7f8b98a66640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9415b640 0x7f8b9415d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47866/0 (socket says 172.21.15.40:47866) 2023-10-18T09:00:01.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.211+0000 7f8b98a66640 1 -- 172.21.15.40:0/4054186630 learned_addr learned my addr 172.21.15.40:0/4054186630 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:01.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.212+0000 7f8b93fff640 1 --2- 172.21.15.40:0/4054186630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b940b01e0 0x7f8b94149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:01.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.212+0000 7f8b937fe640 1 --2- 172.21.15.40:0/4054186630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b940cfe50 0x7f8b9415bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:01.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.212+0000 7f8b98a66640 1 -- 172.21.15.40:0/4054186630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b940b01e0 msgr2=0x7f8b94149620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:01.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.212+0000 7f8b98a66640 1 --2- 172.21.15.40:0/4054186630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b940b01e0 0x7f8b94149620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:01.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.212+0000 7f8b98a66640 1 -- 172.21.15.40:0/4054186630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b940cfe50 msgr2=0x7f8b9415bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:01.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.212+0000 7f8b98a66640 1 --2- 172.21.15.40:0/4054186630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b940cfe50 0x7f8b9415bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:01.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.212+0000 7f8b98a66640 1 -- 172.21.15.40:0/4054186630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c009490 con 0x7f8b9415b640 2023-10-18T09:00:01.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.212+0000 7f8b98a66640 1 --2- 172.21.15.40:0/4054186630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9415b640 0x7f8b9415d590 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c001eb0 tx=0x7f8b8c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:01.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.213+0000 7f8b90ff9640 1 -- 172.21.15.40:0/4054186630 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0052d0 con 0x7f8b9415b640 2023-10-18T09:00:01.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.213+0000 7f8b90ff9640 1 -- 172.21.15.40:0/4054186630 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8b8c0035f0 con 0x7f8b9415b640 2023-10-18T09:00:01.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.213+0000 7f8b99267640 1 -- 172.21.15.40:0/4054186630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9415c250 con 0x7f8b9415b640 2023-10-18T09:00:01.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.213+0000 7f8b90ff9640 1 -- 172.21.15.40:0/4054186630 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0039e0 con 0x7f8b9415b640 2023-10-18T09:00:01.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.213+0000 7f8b99267640 1 -- 172.21.15.40:0/4054186630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9415c7d0 con 0x7f8b9415b640 2023-10-18T09:00:01.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.214+0000 7f8b99267640 1 -- 172.21.15.40:0/4054186630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b94071da0 con 0x7f8b9415b640 2023-10-18T09:00:01.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.215+0000 7f8b90ff9640 1 -- 172.21.15.40:0/4054186630 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8b8c003b40 con 0x7f8b9415b640 2023-10-18T09:00:01.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.216+0000 7f8b90ff9640 1 --2- 172.21.15.40:0/4054186630 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b7406ae40 0x7f8b7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:01.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.216+0000 7f8b90ff9640 1 -- 172.21.15.40:0/4054186630 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b8c088ec0 con 0x7f8b9415b640 2023-10-18T09:00:01.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.216+0000 7f8b93fff640 1 --2- 172.21.15.40:0/4054186630 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b7406ae40 0x7f8b7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:01.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.218+0000 7f8b93fff640 1 --2- 172.21.15.40:0/4054186630 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b7406ae40 0x7f8b7406d300 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8b84005fd0 tx=0x7f8b840095c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:01.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.220+0000 7f8b90ff9640 1 -- 172.21.15.40:0/4054186630 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8b8c053e50 con 0x7f8b9415b640 2023-10-18T09:00:01.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.454+0000 7f8b99267640 1 -- 172.21.15.40:0/4054186630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8b94066e00 con 0x7f8b9415b640 2023-10-18T09:00:01.457 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:00:01.455+0000 7f6a0f710640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T09:00:01.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.458+0000 7f8b90ff9640 1 -- 172.21.15.40:0/4054186630 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8b8c057470 con 0x7f8b9415b640 2023-10-18T09:00:01.460 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T09:00:01.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.462+0000 7f8b99267640 1 -- 172.21.15.40:0/4054186630 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b7406ae40 msgr2=0x7f8b7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:01.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.462+0000 7f8b99267640 1 --2- 172.21.15.40:0/4054186630 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b7406ae40 0x7f8b7406d300 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8b84005fd0 tx=0x7f8b840095c0 comp rx=0 tx=0).stop 2023-10-18T09:00:01.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.462+0000 7f8b99267640 1 -- 172.21.15.40:0/4054186630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9415b640 msgr2=0x7f8b9415d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:01.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.462+0000 7f8b99267640 1 --2- 172.21.15.40:0/4054186630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9415b640 0x7f8b9415d590 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c001eb0 tx=0x7f8b8c00c890 comp rx=0 tx=0).stop 2023-10-18T09:00:01.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.465+0000 7f8b99267640 1 -- 172.21.15.40:0/4054186630 shutdown_connections 2023-10-18T09:00:01.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.465+0000 7f8b99267640 1 --2- 172.21.15.40:0/4054186630 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b7406ae40 0x7f8b7406d300 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:01.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.465+0000 7f8b99267640 1 --2- 172.21.15.40:0/4054186630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b9415b640 0x7f8b9415d590 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:01.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.465+0000 7f8b99267640 1 --2- 172.21.15.40:0/4054186630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b940cfe50 0x7f8b9415bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:01.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.465+0000 7f8b99267640 1 --2- 172.21.15.40:0/4054186630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b940b01e0 0x7f8b94149620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:01.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.465+0000 7f8b99267640 1 -- 172.21.15.40:0/4054186630 >> 172.21.15.40:0/4054186630 conn(0x7f8b9408ad00 msgr2=0x7f8b940b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:01.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.466+0000 7f8b99267640 1 -- 172.21.15.40:0/4054186630 shutdown_connections 2023-10-18T09:00:01.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.466+0000 7f8b99267640 1 -- 172.21.15.40:0/4054186630 wait complete. 2023-10-18T09:00:01.487 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T09:00:01.580 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-18T09:00:01.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.714+0000 7f67e9375640 1 -- 172.21.15.40:0/3084708004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e415af70 msgr2=0x7f67e4070b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:01.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.714+0000 7f67e9375640 1 --2- 172.21.15.40:0/3084708004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e415af70 0x7f67e4070b60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f67e40aef60 tx=0x7f67cc02bd10 comp rx=0 tx=0).stop 2023-10-18T09:00:01.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.714+0000 7f67e9375640 1 -- 172.21.15.40:0/3084708004 shutdown_connections 2023-10-18T09:00:01.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.714+0000 7f67e9375640 1 --2- 172.21.15.40:0/3084708004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67e40710a0 0x7f67e4157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:01.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.714+0000 7f67e9375640 1 --2- 172.21.15.40:0/3084708004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e415af70 0x7f67e4070b60 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:01.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.714+0000 7f67e9375640 1 --2- 172.21.15.40:0/3084708004 >> v2:172.21.15.144:3300/0 conn(0x7f67e406d5d0 0x7f67e40c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:01.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.714+0000 7f67e9375640 1 -- 172.21.15.40:0/3084708004 >> 172.21.15.40:0/3084708004 conn(0x7f67e406ca10 msgr2=0x7f67e406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:01.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.714+0000 7f67e9375640 1 -- 172.21.15.40:0/3084708004 shutdown_connections 2023-10-18T09:00:01.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.714+0000 7f67e9375640 1 -- 172.21.15.40:0/3084708004 wait complete. 2023-10-18T09:00:01.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.716+0000 7f67e9375640 1 Processor -- start 2023-10-18T09:00:01.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.716+0000 7f67e9375640 1 -- start start 2023-10-18T09:00:01.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.717+0000 7f67e9375640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e406d5d0 0x7f67e40addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.717+0000 7f67e9375640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67e40710a0 0x7f67e41517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.717+0000 7f67e9375640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e415af70 0x7f67e41546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.717+0000 7f67e9375640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4144830 con 0x7f67e415af70 2023-10-18T09:00:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.717+0000 7f67e9375640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4143c20 con 0x7f67e406d5d0 2023-10-18T09:00:01.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.717+0000 7f67e9375640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e4143f00 con 0x7f67e40710a0 2023-10-18T09:00:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.717+0000 7f67e8b74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e415af70 0x7f67e41546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.717+0000 7f67e8b74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e415af70 0x7f67e41546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47886/0 (socket says 172.21.15.40:47886) 2023-10-18T09:00:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.717+0000 7f67e8b74640 1 -- 172.21.15.40:0/2089935130 learned_addr learned my addr 172.21.15.40:0/2089935130 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.718+0000 7f67dbfff640 1 --2- 172.21.15.40:0/2089935130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67e40710a0 0x7f67e41517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:01.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.718+0000 7f67e3fff640 1 --2- 172.21.15.40:0/2089935130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e406d5d0 0x7f67e40addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.718+0000 7f67e8b74640 1 -- 172.21.15.40:0/2089935130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67e40710a0 msgr2=0x7f67e41517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.718+0000 7f67e8b74640 1 --2- 172.21.15.40:0/2089935130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67e40710a0 0x7f67e41517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.718+0000 7f67e8b74640 1 -- 172.21.15.40:0/2089935130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e406d5d0 msgr2=0x7f67e40addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.718+0000 7f67e8b74640 1 --2- 172.21.15.40:0/2089935130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e406d5d0 0x7f67e40addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:01.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.718+0000 7f67e8b74640 1 -- 172.21.15.40:0/2089935130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc009560 con 0x7f67e415af70 2023-10-18T09:00:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.718+0000 7f67e8b74640 1 --2- 172.21.15.40:0/2089935130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e415af70 0x7f67e41546d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00eaa0 tx=0x7f67dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.718+0000 7f67e17fa640 1 -- 172.21.15.40:0/2089935130 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00cdb0 con 0x7f67e415af70 2023-10-18T09:00:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.718+0000 7f67e17fa640 1 -- 172.21.15.40:0/2089935130 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f67dc004b20 con 0x7f67e415af70 2023-10-18T09:00:01.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.718+0000 7f67e9375640 1 -- 172.21.15.40:0/2089935130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e41441e0 con 0x7f67e415af70 2023-10-18T09:00:01.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.718+0000 7f67e17fa640 1 -- 172.21.15.40:0/2089935130 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc005020 con 0x7f67e415af70 2023-10-18T09:00:01.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.719+0000 7f67e9375640 1 -- 172.21.15.40:0/2089935130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e4144cc0 con 0x7f67e415af70 2023-10-18T09:00:01.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.720+0000 7f67e9375640 1 -- 172.21.15.40:0/2089935130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e41449a0 con 0x7f67e415af70 2023-10-18T09:00:01.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.721+0000 7f67e17fa640 1 -- 172.21.15.40:0/2089935130 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f67dc005180 con 0x7f67e415af70 2023-10-18T09:00:01.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.722+0000 7f67e17fa640 1 --2- 172.21.15.40:0/2089935130 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f67c406ae70 0x7f67c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:01.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.722+0000 7f67e17fa640 1 -- 172.21.15.40:0/2089935130 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f67dc08a080 con 0x7f67e415af70 2023-10-18T09:00:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.722+0000 7f67e3fff640 1 --2- 172.21.15.40:0/2089935130 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f67c406ae70 0x7f67c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.724+0000 7f67e3fff640 1 --2- 172.21.15.40:0/2089935130 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f67c406ae70 0x7f67c406d330 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f67d4005fd0 tx=0x7f67d4005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:01.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.726+0000 7f67e17fa640 1 -- 172.21.15.40:0/2089935130 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f67dc0bcca0 con 0x7f67e415af70 2023-10-18T09:00:01.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:01.956+0000 7f67e9375640 1 -- 172.21.15.40:0/2089935130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f67e4066e00 con 0x7f67e415af70 2023-10-18T09:00:02.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.461+0000 7f67e17fa640 1 -- 172.21.15.40:0/2089935130 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7f67dc01f0a0 con 0x7f67e415af70 2023-10-18T09:00:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.465+0000 7f67e9375640 1 -- 172.21.15.40:0/2089935130 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f67c406ae70 msgr2=0x7f67c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.465+0000 7f67e9375640 1 --2- 172.21.15.40:0/2089935130 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f67c406ae70 0x7f67c406d330 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f67d4005fd0 tx=0x7f67d4005850 comp rx=0 tx=0).stop 2023-10-18T09:00:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.465+0000 7f67e9375640 1 -- 172.21.15.40:0/2089935130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e415af70 msgr2=0x7f67e41546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.465+0000 7f67e9375640 1 --2- 172.21.15.40:0/2089935130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e415af70 0x7f67e41546d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00eaa0 tx=0x7f67dc00ef60 comp rx=0 tx=0).stop 2023-10-18T09:00:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.465+0000 7f67e9375640 1 -- 172.21.15.40:0/2089935130 shutdown_connections 2023-10-18T09:00:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.465+0000 7f67e9375640 1 --2- 172.21.15.40:0/2089935130 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f67c406ae70 0x7f67c406d330 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.465+0000 7f67e9375640 1 --2- 172.21.15.40:0/2089935130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67e415af70 0x7f67e41546d0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.465+0000 7f67e9375640 1 --2- 172.21.15.40:0/2089935130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67e40710a0 0x7f67e41517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.465+0000 7f67e9375640 1 --2- 172.21.15.40:0/2089935130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e406d5d0 0x7f67e40addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.465+0000 7f67e9375640 1 -- 172.21.15.40:0/2089935130 >> 172.21.15.40:0/2089935130 conn(0x7f67e406ca10 msgr2=0x7f67e406bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.466+0000 7f67e9375640 1 -- 172.21.15.40:0/2089935130 shutdown_connections 2023-10-18T09:00:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.466+0000 7f67e9375640 1 -- 172.21.15.40:0/2089935130 wait complete. 2023-10-18T09:00:02.491 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T09:00:02.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.726+0000 7ffb56061640 1 -- 172.21.15.40:0/2883071251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb3c004d00 msgr2=0x7ffb3c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:02.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.726+0000 7ffb56061640 1 --2- 172.21.15.40:0/2883071251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb3c004d00 0x7ffb3c005180 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800c260 tx=0x7ffb3c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:00:02.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.726+0000 7ffb4ffff640 1 -- 172.21.15.40:0/2883071251 reap_dead start 2023-10-18T09:00:02.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.726+0000 7ffb56061640 1 -- 172.21.15.40:0/2883071251 shutdown_connections 2023-10-18T09:00:02.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.726+0000 7ffb56061640 1 -- 172.21.15.40:0/2883071251 >> 172.21.15.40:0/2883071251 conn(0x7ffb500aec00 msgr2=0x7ffb500aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:02.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.726+0000 7ffb56061640 1 -- 172.21.15.40:0/2883071251 shutdown_connections 2023-10-18T09:00:02.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.726+0000 7ffb56061640 1 -- 172.21.15.40:0/2883071251 wait complete. 2023-10-18T09:00:02.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.728+0000 7ffb56061640 1 Processor -- start 2023-10-18T09:00:02.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.728+0000 7ffb56061640 1 -- start start 2023-10-18T09:00:02.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.729+0000 7ffb56061640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb3c004d00 0x7ffb501586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:02.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.729+0000 7ffb56061640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb50157ff0 0x7ffb5015a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:02.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.729+0000 7ffb56061640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb501f4fa0 0x7ffb501f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:02.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.729+0000 7ffb56061640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501f5960 con 0x7ffb50157ff0 2023-10-18T09:00:02.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.729+0000 7ffb56061640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb500adc20 con 0x7ffb501f4fa0 2023-10-18T09:00:02.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.729+0000 7ffb56061640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb500ae480 con 0x7ffb3c004d00 2023-10-18T09:00:02.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.729+0000 7ffb46ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb50157ff0 0x7ffb5015a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:02.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.729+0000 7ffb46ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb50157ff0 0x7ffb5015a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47902/0 (socket says 172.21.15.40:47902) 2023-10-18T09:00:02.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.729+0000 7ffb46ffd640 1 -- 172.21.15.40:0/2076701072 learned_addr learned my addr 172.21.15.40:0/2076701072 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:02.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.730+0000 7ffb4f7fe640 1 --2- 172.21.15.40:0/2076701072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb3c004d00 0x7ffb501586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:02.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.730+0000 7ffb46ffd640 1 -- 172.21.15.40:0/2076701072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb3c004d00 msgr2=0x7ffb501586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:02.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.730+0000 7ffb46ffd640 1 --2- 172.21.15.40:0/2076701072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb3c004d00 0x7ffb501586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:02.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.730+0000 7ffb46ffd640 1 -- 172.21.15.40:0/2076701072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb501f4fa0 msgr2=0x7ffb501f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:02.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.730+0000 7ffb4ffff640 1 --2- 172.21.15.40:0/2076701072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb501f4fa0 0x7ffb501f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:02.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.730+0000 7ffb46ffd640 1 --2- 172.21.15.40:0/2076701072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb501f4fa0 0x7ffb501f5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:02.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.730+0000 7ffb46ffd640 1 -- 172.21.15.40:0/2076701072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb500adec0 con 0x7ffb50157ff0 2023-10-18T09:00:02.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.730+0000 7ffb46ffd640 1 --2- 172.21.15.40:0/2076701072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb50157ff0 0x7ffb5015a530 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009fd0 tx=0x7ffb38011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:02.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.730+0000 7ffb4cff9640 1 -- 172.21.15.40:0/2076701072 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38004460 con 0x7ffb50157ff0 2023-10-18T09:00:02.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.730+0000 7ffb4cff9640 1 -- 172.21.15.40:0/2076701072 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffb3800f940 con 0x7ffb50157ff0 2023-10-18T09:00:02.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.730+0000 7ffb4cff9640 1 -- 172.21.15.40:0/2076701072 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3800fd50 con 0x7ffb50157ff0 2023-10-18T09:00:02.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.730+0000 7ffb56061640 1 -- 172.21.15.40:0/2076701072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb500ae1a0 con 0x7ffb50157ff0 2023-10-18T09:00:02.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.731+0000 7ffb56061640 1 -- 172.21.15.40:0/2076701072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5015aa70 con 0x7ffb50157ff0 2023-10-18T09:00:02.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.732+0000 7ffb56061640 1 -- 172.21.15.40:0/2076701072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb50066fd0 con 0x7ffb50157ff0 2023-10-18T09:00:02.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.733+0000 7ffb4cff9640 1 -- 172.21.15.40:0/2076701072 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffb380035f0 con 0x7ffb50157ff0 2023-10-18T09:00:02.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.734+0000 7ffb4cff9640 1 --2- 172.21.15.40:0/2076701072 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffb3006ae70 0x7ffb3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:02.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.734+0000 7ffb4cff9640 1 -- 172.21.15.40:0/2076701072 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ffb3808a430 con 0x7ffb50157ff0 2023-10-18T09:00:02.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.734+0000 7ffb4f7fe640 1 --2- 172.21.15.40:0/2076701072 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffb3006ae70 0x7ffb3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:02.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.735+0000 7ffb4f7fe640 1 --2- 172.21.15.40:0/2076701072 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffb3006ae70 0x7ffb3006d330 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ffb40005fd0 tx=0x7ffb40009500 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:02.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.738+0000 7ffb4cff9640 1 -- 172.21.15.40:0/2076701072 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffb38055480 con 0x7ffb50157ff0 2023-10-18T09:00:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:02.935+0000 7ffb56061640 1 -- 172.21.15.40:0/2076701072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffb500cacb0 con 0x7ffb50157ff0 2023-10-18T09:00:03.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.471+0000 7ffb4cff9640 1 -- 172.21.15.40:0/2076701072 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v78) v1 ==== 181+0+0 (secure 0 0 0) 0x7ffb38058aa0 con 0x7ffb50157ff0 2023-10-18T09:00:03.475 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-10-18T09:00:03.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.476+0000 7ffb56061640 1 -- 172.21.15.40:0/2076701072 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffb3006ae70 msgr2=0x7ffb3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:03.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.476+0000 7ffb56061640 1 --2- 172.21.15.40:0/2076701072 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffb3006ae70 0x7ffb3006d330 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ffb40005fd0 tx=0x7ffb40009500 comp rx=0 tx=0).stop 2023-10-18T09:00:03.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.476+0000 7ffb56061640 1 -- 172.21.15.40:0/2076701072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb50157ff0 msgr2=0x7ffb5015a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:03.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.476+0000 7ffb56061640 1 --2- 172.21.15.40:0/2076701072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb50157ff0 0x7ffb5015a530 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009fd0 tx=0x7ffb38011cb0 comp rx=0 tx=0).stop 2023-10-18T09:00:03.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.476+0000 7ffb56061640 1 -- 172.21.15.40:0/2076701072 shutdown_connections 2023-10-18T09:00:03.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.476+0000 7ffb56061640 1 --2- 172.21.15.40:0/2076701072 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffb3006ae70 0x7ffb3006d330 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:03.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.476+0000 7ffb56061640 1 --2- 172.21.15.40:0/2076701072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb501f4fa0 0x7ffb501f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:03.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.476+0000 7ffb56061640 1 --2- 172.21.15.40:0/2076701072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb50157ff0 0x7ffb5015a530 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:03.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.476+0000 7ffb56061640 1 --2- 172.21.15.40:0/2076701072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb3c004d00 0x7ffb501586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:03.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.476+0000 7ffb56061640 1 -- 172.21.15.40:0/2076701072 >> 172.21.15.40:0/2076701072 conn(0x7ffb500aec00 msgr2=0x7ffb50156fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:03.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.477+0000 7ffb56061640 1 -- 172.21.15.40:0/2076701072 shutdown_connections 2023-10-18T09:00:03.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.477+0000 7ffb56061640 1 -- 172.21.15.40:0/2076701072 wait complete. 2023-10-18T09:00:03.501 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T09:00:03.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.724+0000 7f3d49bc7640 1 -- 172.21.15.40:0/1680463231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4415b460 msgr2=0x7f3d440cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:03.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.724+0000 7f3d49bc7640 1 --2- 172.21.15.40:0/1680463231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4415b460 0x7f3d440cae80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3d34005fd0 tx=0x7f3d3402beb0 comp rx=0 tx=0).stop 2023-10-18T09:00:03.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.724+0000 7f3d49bc7640 1 -- 172.21.15.40:0/1680463231 shutdown_connections 2023-10-18T09:00:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.724+0000 7f3d49bc7640 1 --2- 172.21.15.40:0/1680463231 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d440c9a20 0x7f3d440c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.724+0000 7f3d49bc7640 1 --2- 172.21.15.40:0/1680463231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4415b460 0x7f3d440cae80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.724+0000 7f3d49bc7640 1 -- 172.21.15.40:0/1680463231 >> 172.21.15.40:0/1680463231 conn(0x7f3d440aec00 msgr2=0x7f3d440aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:03.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.724+0000 7f3d49bc7640 1 -- 172.21.15.40:0/1680463231 shutdown_connections 2023-10-18T09:00:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.724+0000 7f3d49bc7640 1 -- 172.21.15.40:0/1680463231 wait complete. 2023-10-18T09:00:03.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.727+0000 7f3d49bc7640 1 Processor -- start 2023-10-18T09:00:03.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.727+0000 7f3d49bc7640 1 -- start start 2023-10-18T09:00:03.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.728+0000 7f3d49bc7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d440c9a20 0x7f3d440c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:03.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.728+0000 7f3d49bc7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4415b460 0x7f3d4414f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:03.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.728+0000 7f3d49bc7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4415fd90 0x7f3d440c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:03.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.728+0000 7f3d49bc7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d441f26e0 con 0x7f3d440c9a20 2023-10-18T09:00:03.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.728+0000 7f3d49bc7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d441f2b70 con 0x7f3d4415b460 2023-10-18T09:00:03.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.728+0000 7f3d49bc7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4415ea10 con 0x7f3d4415fd90 2023-10-18T09:00:03.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.728+0000 7f3d437fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d440c9a20 0x7f3d440c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:03.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.728+0000 7f3d437fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d440c9a20 0x7f3d440c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47908/0 (socket says 172.21.15.40:47908) 2023-10-18T09:00:03.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.728+0000 7f3d437fe640 1 -- 172.21.15.40:0/3356580554 learned_addr learned my addr 172.21.15.40:0/3356580554 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:03.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.729+0000 7f3d42ffd640 1 --2- 172.21.15.40:0/3356580554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4415b460 0x7f3d4414f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:03.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.729+0000 7f3d43fff640 1 --2- 172.21.15.40:0/3356580554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4415fd90 0x7f3d440c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:03.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.729+0000 7f3d437fe640 1 -- 172.21.15.40:0/3356580554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4415fd90 msgr2=0x7f3d440c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:03.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.729+0000 7f3d437fe640 1 --2- 172.21.15.40:0/3356580554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4415fd90 0x7f3d440c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:03.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.729+0000 7f3d437fe640 1 -- 172.21.15.40:0/3356580554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4415b460 msgr2=0x7f3d4414f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:03.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.729+0000 7f3d437fe640 1 --2- 172.21.15.40:0/3356580554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4415b460 0x7f3d4414f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:03.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.729+0000 7f3d437fe640 1 -- 172.21.15.40:0/3356580554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d34009560 con 0x7f3d440c9a20 2023-10-18T09:00:03.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.729+0000 7f3d43fff640 1 --2- 172.21.15.40:0/3356580554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4415fd90 0x7f3d440c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:03.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.729+0000 7f3d437fe640 1 --2- 172.21.15.40:0/3356580554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d440c9a20 0x7f3d440c6b90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3d34009850 tx=0x7f3d340038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:03.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.730+0000 7f3d2bfff640 1 -- 172.21.15.40:0/3356580554 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34003ea0 con 0x7f3d440c9a20 2023-10-18T09:00:03.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.730+0000 7f3d2bfff640 1 -- 172.21.15.40:0/3356580554 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3d3403d440 con 0x7f3d440c9a20 2023-10-18T09:00:03.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.730+0000 7f3d2bfff640 1 -- 172.21.15.40:0/3356580554 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3403d850 con 0x7f3d440c9a20 2023-10-18T09:00:03.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.730+0000 7f3d49bc7640 1 -- 172.21.15.40:0/3356580554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d440c70d0 con 0x7f3d440c9a20 2023-10-18T09:00:03.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.730+0000 7f3d49bc7640 1 -- 172.21.15.40:0/3356580554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d441f1db0 con 0x7f3d440c9a20 2023-10-18T09:00:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.731+0000 7f3d49bc7640 1 -- 172.21.15.40:0/3356580554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d44066fd0 con 0x7f3d440c9a20 2023-10-18T09:00:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.732+0000 7f3d2bfff640 1 -- 172.21.15.40:0/3356580554 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3d3403d9b0 con 0x7f3d440c9a20 2023-10-18T09:00:03.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.733+0000 7f3d2bfff640 1 --2- 172.21.15.40:0/3356580554 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d3006ae40 0x7f3d3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:03.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.733+0000 7f3d2bfff640 1 -- 172.21.15.40:0/3356580554 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f3d340ac1d0 con 0x7f3d440c9a20 2023-10-18T09:00:03.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.734+0000 7f3d42ffd640 1 --2- 172.21.15.40:0/3356580554 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d3006ae40 0x7f3d3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:03.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.735+0000 7f3d42ffd640 1 --2- 172.21.15.40:0/3356580554 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d3006ae40 0x7f3d3006d300 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3d440719a0 tx=0x7f3d38009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:03.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.739+0000 7f3d2bfff640 1 -- 172.21.15.40:0/3356580554 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3d340773e0 con 0x7f3d440c9a20 2023-10-18T09:00:03.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:03.937+0000 7f3d49bc7640 1 -- 172.21.15.40:0/3356580554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3d440c95b0 con 0x7f3d440c9a20 2023-10-18T09:00:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.475+0000 7f3d2bfff640 1 -- 172.21.15.40:0/3356580554 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v79) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3d3407aa00 con 0x7f3d440c9a20 2023-10-18T09:00:04.478 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-10-18T09:00:04.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.480+0000 7f3d49bc7640 1 -- 172.21.15.40:0/3356580554 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d3006ae40 msgr2=0x7f3d3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:04.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.480+0000 7f3d49bc7640 1 --2- 172.21.15.40:0/3356580554 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d3006ae40 0x7f3d3006d300 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3d440719a0 tx=0x7f3d38009040 comp rx=0 tx=0).stop 2023-10-18T09:00:04.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.480+0000 7f3d49bc7640 1 -- 172.21.15.40:0/3356580554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d440c9a20 msgr2=0x7f3d440c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:04.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.480+0000 7f3d49bc7640 1 --2- 172.21.15.40:0/3356580554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d440c9a20 0x7f3d440c6b90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3d34009850 tx=0x7f3d340038a0 comp rx=0 tx=0).stop 2023-10-18T09:00:04.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.480+0000 7f3d49bc7640 1 -- 172.21.15.40:0/3356580554 shutdown_connections 2023-10-18T09:00:04.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.480+0000 7f3d49bc7640 1 --2- 172.21.15.40:0/3356580554 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d3006ae40 0x7f3d3006d300 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:04.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.480+0000 7f3d49bc7640 1 --2- 172.21.15.40:0/3356580554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d4415fd90 0x7f3d440c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:04.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.480+0000 7f3d49bc7640 1 --2- 172.21.15.40:0/3356580554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d4415b460 0x7f3d4414f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:04.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.480+0000 7f3d49bc7640 1 --2- 172.21.15.40:0/3356580554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d440c9a20 0x7f3d440c6b90 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:04.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.480+0000 7f3d49bc7640 1 -- 172.21.15.40:0/3356580554 >> 172.21.15.40:0/3356580554 conn(0x7f3d440aec00 msgr2=0x7f3d4415a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:04.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.480+0000 7f3d49bc7640 1 -- 172.21.15.40:0/3356580554 shutdown_connections 2023-10-18T09:00:04.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.480+0000 7f3d49bc7640 1 -- 172.21.15.40:0/3356580554 wait complete. 2023-10-18T09:00:04.504 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 22 2023-10-18T09:00:04.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.724+0000 7fb8c08e5640 1 -- 172.21.15.40:0/959361897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc0b7000 msgr2=0x7fb8bc0cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:04.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.724+0000 7fb8c08e5640 1 --2- 172.21.15.40:0/959361897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc0b7000 0x7fb8bc0cdef0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0005fd0 tx=0x7fb8b002be20 comp rx=0 tx=0).stop 2023-10-18T09:00:04.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.724+0000 7fb8c08e5640 1 -- 172.21.15.40:0/959361897 shutdown_connections 2023-10-18T09:00:04.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.724+0000 7fb8c08e5640 1 --2- 172.21.15.40:0/959361897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8bc0ce430 0x7fb8bc0b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:04.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.724+0000 7fb8c08e5640 1 --2- 172.21.15.40:0/959361897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc0b7000 0x7fb8bc0cdef0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.724+0000 7fb8c08e5640 1 -- 172.21.15.40:0/959361897 >> 172.21.15.40:0/959361897 conn(0x7fb8bc06a780 msgr2=0x7fb8bc06ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.724+0000 7fb8c08e5640 1 -- 172.21.15.40:0/959361897 shutdown_connections 2023-10-18T09:00:04.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.724+0000 7fb8c08e5640 1 -- 172.21.15.40:0/959361897 wait complete. 2023-10-18T09:00:04.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.726+0000 7fb8c08e5640 1 Processor -- start 2023-10-18T09:00:04.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.726+0000 7fb8c08e5640 1 -- start start 2023-10-18T09:00:04.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.727+0000 7fb8c08e5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8bc0ce430 0x7fb8bc151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:04.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.727+0000 7fb8c08e5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc0c5f30 0x7fb8bc0c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:04.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.727+0000 7fb8c08e5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8bc1e40a0 0x7fb8bc1e4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:04.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.727+0000 7fb8c08e5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc0caad0 con 0x7fb8bc0ce430 2023-10-18T09:00:04.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.727+0000 7fb8c08e5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc151f50 con 0x7fb8bc0c5f30 2023-10-18T09:00:04.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.727+0000 7fb8c08e5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc0ca1a0 con 0x7fb8bc1e40a0 2023-10-18T09:00:04.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.727+0000 7fb8ba59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8bc0ce430 0x7fb8bc151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:04.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.727+0000 7fb8ba59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8bc0ce430 0x7fb8bc151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47918/0 (socket says 172.21.15.40:47918) 2023-10-18T09:00:04.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.727+0000 7fb8ba59c640 1 -- 172.21.15.40:0/2089283998 learned_addr learned my addr 172.21.15.40:0/2089283998 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:04.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.728+0000 7fb8b9d9b640 1 --2- 172.21.15.40:0/2089283998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc0c5f30 0x7fb8bc0c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:04.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.728+0000 7fb8bad9d640 1 --2- 172.21.15.40:0/2089283998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8bc1e40a0 0x7fb8bc1e4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:04.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.728+0000 7fb8ba59c640 1 -- 172.21.15.40:0/2089283998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8bc1e40a0 msgr2=0x7fb8bc1e4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:04.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.728+0000 7fb8ba59c640 1 --2- 172.21.15.40:0/2089283998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8bc1e40a0 0x7fb8bc1e4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:04.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.728+0000 7fb8ba59c640 1 -- 172.21.15.40:0/2089283998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc0c5f30 msgr2=0x7fb8bc0c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:04.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.728+0000 7fb8ba59c640 1 --2- 172.21.15.40:0/2089283998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc0c5f30 0x7fb8bc0c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:04.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.728+0000 7fb8ba59c640 1 -- 172.21.15.40:0/2089283998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b0009560 con 0x7fb8bc0ce430 2023-10-18T09:00:04.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.728+0000 7fb8bad9d640 1 --2- 172.21.15.40:0/2089283998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8bc1e40a0 0x7fb8bc1e4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:04.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.728+0000 7fb8b9d9b640 1 --2- 172.21.15.40:0/2089283998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc0c5f30 0x7fb8bc0c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:04.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.728+0000 7fb8ba59c640 1 --2- 172.21.15.40:0/2089283998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8bc0ce430 0x7fb8bc151a10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00c400 tx=0x7fb8ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:04.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.728+0000 7fb8aaffd640 1 -- 172.21.15.40:0/2089283998 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac0052d0 con 0x7fb8bc0ce430 2023-10-18T09:00:04.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.728+0000 7fb8aaffd640 1 -- 172.21.15.40:0/2089283998 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb8ac0035f0 con 0x7fb8bc0ce430 2023-10-18T09:00:04.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.728+0000 7fb8aaffd640 1 -- 172.21.15.40:0/2089283998 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac003a00 con 0x7fb8bc0ce430 2023-10-18T09:00:04.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.729+0000 7fb8c08e5640 1 -- 172.21.15.40:0/2089283998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8bc0ca480 con 0x7fb8bc0ce430 2023-10-18T09:00:04.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.729+0000 7fb8c08e5640 1 -- 172.21.15.40:0/2089283998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8bc0c51c0 con 0x7fb8bc0ce430 2023-10-18T09:00:04.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.730+0000 7fb8c08e5640 1 -- 172.21.15.40:0/2089283998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8bc066fd0 con 0x7fb8bc0ce430 2023-10-18T09:00:04.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.731+0000 7fb8aaffd640 1 -- 172.21.15.40:0/2089283998 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb8ac003b60 con 0x7fb8bc0ce430 2023-10-18T09:00:04.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.732+0000 7fb8aaffd640 1 --2- 172.21.15.40:0/2089283998 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb89c06ae40 0x7fb89c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:04.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.732+0000 7fb8aaffd640 1 -- 172.21.15.40:0/2089283998 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb8ac088b30 con 0x7fb8bc0ce430 2023-10-18T09:00:04.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.733+0000 7fb8b9d9b640 1 --2- 172.21.15.40:0/2089283998 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb89c06ae40 0x7fb89c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:04.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.734+0000 7fb8b9d9b640 1 --2- 172.21.15.40:0/2089283998 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb89c06ae40 0x7fb89c06d300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0035000 tx=0x7fb8b0004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:04.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.736+0000 7fb8aaffd640 1 -- 172.21.15.40:0/2089283998 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb8ac053ed0 con 0x7fb8bc0ce430 2023-10-18T09:00:04.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.937+0000 7fb8c08e5640 1 -- 172.21.15.40:0/2089283998 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config reset", "num": 22} v 0) v1 -- 0x7fb8bc15db60 con 0x7fb8bc0ce430 2023-10-18T09:00:04.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.945+0000 7fb8aaffd640 1 -- 172.21.15.40:0/2089283998 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 22}]=0 v28) v1 ==== 71+0+0 (secure 0 0 0) 0x7fb8ac017080 con 0x7fb8bc0ce430 2023-10-18T09:00:04.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.948+0000 7fb8c08e5640 1 -- 172.21.15.40:0/2089283998 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb89c06ae40 msgr2=0x7fb89c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:04.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.948+0000 7fb8c08e5640 1 --2- 172.21.15.40:0/2089283998 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb89c06ae40 0x7fb89c06d300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0035000 tx=0x7fb8b0004730 comp rx=0 tx=0).stop 2023-10-18T09:00:04.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.948+0000 7fb8c08e5640 1 -- 172.21.15.40:0/2089283998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8bc0ce430 msgr2=0x7fb8bc151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:04.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.948+0000 7fb8c08e5640 1 --2- 172.21.15.40:0/2089283998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8bc0ce430 0x7fb8bc151a10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00c400 tx=0x7fb8ac00c8c0 comp rx=0 tx=0).stop 2023-10-18T09:00:04.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.948+0000 7fb8c08e5640 1 -- 172.21.15.40:0/2089283998 shutdown_connections 2023-10-18T09:00:04.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.948+0000 7fb8c08e5640 1 --2- 172.21.15.40:0/2089283998 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb89c06ae40 0x7fb89c06d300 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:04.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.948+0000 7fb8c08e5640 1 --2- 172.21.15.40:0/2089283998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8bc1e40a0 0x7fb8bc1e4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:04.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.948+0000 7fb8c08e5640 1 --2- 172.21.15.40:0/2089283998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc0c5f30 0x7fb8bc0c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:04.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.948+0000 7fb8c08e5640 1 --2- 172.21.15.40:0/2089283998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8bc0ce430 0x7fb8bc151a10 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:04.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.948+0000 7fb8c08e5640 1 -- 172.21.15.40:0/2089283998 >> 172.21.15.40:0/2089283998 conn(0x7fb8bc06a780 msgr2=0x7fb8bc0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:04.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.949+0000 7fb8c08e5640 1 -- 172.21.15.40:0/2089283998 shutdown_connections 2023-10-18T09:00:04.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:04.949+0000 7fb8c08e5640 1 -- 172.21.15.40:0/2089283998 wait complete. 2023-10-18T09:00:04.970 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2023-10-18T09:00:05.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.193+0000 7fd18359e640 1 -- 172.21.15.40:0/2995004349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd18406d6f0 msgr2=0x7fd1840b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:05.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.193+0000 7fd18359e640 1 --2- 172.21.15.40:0/2995004349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd18406d6f0 0x7fd1840b15b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd174009830 tx=0x7fd17402bd60 comp rx=0 tx=0).stop 2023-10-18T09:00:05.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.193+0000 7fd18359e640 1 -- 172.21.15.40:0/2995004349 shutdown_connections 2023-10-18T09:00:05.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.193+0000 7fd18359e640 1 --2- 172.21.15.40:0/2995004349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1840ca0d0 0x7fd184157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:05.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.193+0000 7fd18359e640 1 --2- 172.21.15.40:0/2995004349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd18406d6f0 0x7fd1840b15b0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:05.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.193+0000 7fd18359e640 1 -- 172.21.15.40:0/2995004349 >> 172.21.15.40:0/2995004349 conn(0x7fd184069720 msgr2=0x7fd184069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:05.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.193+0000 7fd18359e640 1 -- 172.21.15.40:0/2995004349 shutdown_connections 2023-10-18T09:00:05.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.193+0000 7fd18359e640 1 -- 172.21.15.40:0/2995004349 wait complete. 2023-10-18T09:00:05.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.195+0000 7fd18359e640 1 Processor -- start 2023-10-18T09:00:05.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.195+0000 7fd18359e640 1 -- start start 2023-10-18T09:00:05.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.196+0000 7fd18359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd18406d6f0 0x7fd184151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:05.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.196+0000 7fd18359e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1840b0630 0x7fd184144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:05.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.196+0000 7fd18359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1840ca0d0 0x7fd184152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:05.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.196+0000 7fd18359e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd184151dd0 con 0x7fd1840ca0d0 2023-10-18T09:00:05.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.196+0000 7fd18359e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1841f0b50 con 0x7fd18406d6f0 2023-10-18T09:00:05.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.196+0000 7fd18359e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1841f0870 con 0x7fd1840b0630 2023-10-18T09:00:05.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.196+0000 7fd182d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1840ca0d0 0x7fd184152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:05.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.196+0000 7fd182d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1840ca0d0 0x7fd184152000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44134/0 (socket says 172.21.15.40:44134) 2023-10-18T09:00:05.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.196+0000 7fd182d9d640 1 -- 172.21.15.40:0/3271039011 learned_addr learned my addr 172.21.15.40:0/3271039011 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:05.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.197+0000 7fd18259c640 1 --2- 172.21.15.40:0/3271039011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd18406d6f0 0x7fd184151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:05.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.197+0000 7fd181d9b640 1 --2- 172.21.15.40:0/3271039011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1840b0630 0x7fd184144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:05.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.197+0000 7fd182d9d640 1 -- 172.21.15.40:0/3271039011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1840b0630 msgr2=0x7fd184144c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:05.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.197+0000 7fd182d9d640 1 --2- 172.21.15.40:0/3271039011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1840b0630 0x7fd184144c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:05.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.197+0000 7fd182d9d640 1 -- 172.21.15.40:0/3271039011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd18406d6f0 msgr2=0x7fd184151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:05.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.197+0000 7fd182d9d640 1 --2- 172.21.15.40:0/3271039011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd18406d6f0 0x7fd184151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:05.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.197+0000 7fd182d9d640 1 -- 172.21.15.40:0/3271039011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd174009490 con 0x7fd1840ca0d0 2023-10-18T09:00:05.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.197+0000 7fd18259c640 1 --2- 172.21.15.40:0/3271039011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd18406d6f0 0x7fd184151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.197+0000 7fd181d9b640 1 --2- 172.21.15.40:0/3271039011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1840b0630 0x7fd184144c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.197+0000 7fd182d9d640 1 --2- 172.21.15.40:0/3271039011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1840ca0d0 0x7fd184152000 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e990 tx=0x7fd17c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:05.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.198+0000 7fd172ffd640 1 -- 172.21.15.40:0/3271039011 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c004d20 con 0x7fd1840ca0d0 2023-10-18T09:00:05.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.198+0000 7fd172ffd640 1 -- 172.21.15.40:0/3271039011 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd17c00caa0 con 0x7fd1840ca0d0 2023-10-18T09:00:05.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.198+0000 7fd18359e640 1 -- 172.21.15.40:0/3271039011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd184150750 con 0x7fd1840ca0d0 2023-10-18T09:00:05.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.198+0000 7fd172ffd640 1 -- 172.21.15.40:0/3271039011 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c005020 con 0x7fd1840ca0d0 2023-10-18T09:00:05.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.198+0000 7fd18359e640 1 -- 172.21.15.40:0/3271039011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd184150d10 con 0x7fd1840ca0d0 2023-10-18T09:00:05.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.199+0000 7fd18359e640 1 -- 172.21.15.40:0/3271039011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd184150f30 con 0x7fd1840ca0d0 2023-10-18T09:00:05.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.200+0000 7fd172ffd640 1 -- 172.21.15.40:0/3271039011 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd17c005180 con 0x7fd1840ca0d0 2023-10-18T09:00:05.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.201+0000 7fd172ffd640 1 --2- 172.21.15.40:0/3271039011 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd16406ae40 0x7fd16406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:05.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.201+0000 7fd172ffd640 1 -- 172.21.15.40:0/3271039011 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd17c089c40 con 0x7fd1840ca0d0 2023-10-18T09:00:05.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.202+0000 7fd18259c640 1 --2- 172.21.15.40:0/3271039011 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd16406ae40 0x7fd16406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:05.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.203+0000 7fd18259c640 1 --2- 172.21.15.40:0/3271039011 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd16406ae40 0x7fd16406d300 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd174036000 tx=0x7fd174004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:05.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.205+0000 7fd172ffd640 1 -- 172.21.15.40:0/3271039011 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd17c054fe0 con 0x7fd1840ca0d0 2023-10-18T09:00:05.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.470+0000 7fd18359e640 1 -- 172.21.15.40:0/3271039011 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7fd184066e00 con 0x7fd1840ca0d0 2023-10-18T09:00:05.478 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:00:05.488 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:00:05.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.478+0000 7fd172ffd640 1 -- 172.21.15.40:0/3271039011 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v257) v1 ==== 134+0+0 (secure 0 0 0) 0x7fd17c01f070 con 0x7fd1840ca0d0 2023-10-18T09:00:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.480+0000 7fd18359e640 1 -- 172.21.15.40:0/3271039011 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd16406ae40 msgr2=0x7fd16406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.480+0000 7fd18359e640 1 --2- 172.21.15.40:0/3271039011 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd16406ae40 0x7fd16406d300 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd174036000 tx=0x7fd174004730 comp rx=0 tx=0).stop 2023-10-18T09:00:05.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.480+0000 7fd18359e640 1 -- 172.21.15.40:0/3271039011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1840ca0d0 msgr2=0x7fd184152000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:05.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.480+0000 7fd18359e640 1 --2- 172.21.15.40:0/3271039011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1840ca0d0 0x7fd184152000 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e990 tx=0x7fd17c00ee50 comp rx=0 tx=0).stop 2023-10-18T09:00:05.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.481+0000 7fd18359e640 1 -- 172.21.15.40:0/3271039011 shutdown_connections 2023-10-18T09:00:05.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.481+0000 7fd18359e640 1 --2- 172.21.15.40:0/3271039011 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd16406ae40 0x7fd16406d300 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:05.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.481+0000 7fd18359e640 1 --2- 172.21.15.40:0/3271039011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1840ca0d0 0x7fd184152000 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:05.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.481+0000 7fd18359e640 1 --2- 172.21.15.40:0/3271039011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1840b0630 0x7fd184144c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:05.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.481+0000 7fd18359e640 1 --2- 172.21.15.40:0/3271039011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd18406d6f0 0x7fd184151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:05.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.481+0000 7fd18359e640 1 -- 172.21.15.40:0/3271039011 >> 172.21.15.40:0/3271039011 conn(0x7fd184069720 msgr2=0x7fd184158970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.481+0000 7fd18359e640 1 -- 172.21.15.40:0/3271039011 shutdown_connections 2023-10-18T09:00:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.481+0000 7fd18359e640 1 -- 172.21.15.40:0/3271039011 wait complete. 2023-10-18T09:00:05.502 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-10-18T09:00:05.502 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) That when the full flag is encountered during asynchronous 2023-10-18T09:00:05.503 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2023-10-18T09:00:05.504 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:00:05.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.749+0000 7fa5e941a640 1 -- 172.21.15.40:0/2667805700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e40ca160 msgr2=0x7fa5e40b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:05.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.749+0000 7fa5e941a640 1 --2- 172.21.15.40:0/2667805700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e40ca160 0x7fa5e40b0a80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009830 tx=0x7fa5d802bb90 comp rx=0 tx=0).stop 2023-10-18T09:00:05.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.749+0000 7fa5e941a640 1 -- 172.21.15.40:0/2667805700 shutdown_connections 2023-10-18T09:00:05.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.749+0000 7fa5e941a640 1 --2- 172.21.15.40:0/2667805700 >> v2:172.21.15.144:3300/0 conn(0x7fa5e41425a0 0x7fa5e40cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:05.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.749+0000 7fa5e941a640 1 --2- 172.21.15.40:0/2667805700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e40ca160 0x7fa5e40b0a80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:05.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.749+0000 7fa5e941a640 1 -- 172.21.15.40:0/2667805700 >> 172.21.15.40:0/2667805700 conn(0x7fa5e4069720 msgr2=0x7fa5e4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:05.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.749+0000 7fa5e941a640 1 -- 172.21.15.40:0/2667805700 shutdown_connections 2023-10-18T09:00:05.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.749+0000 7fa5e941a640 1 -- 172.21.15.40:0/2667805700 wait complete. 2023-10-18T09:00:05.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.751+0000 7fa5e941a640 1 Processor -- start 2023-10-18T09:00:05.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.751+0000 7fa5e941a640 1 -- start start 2023-10-18T09:00:05.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.752+0000 7fa5e941a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5e40ca160 0x7fa5e40abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:05.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.752+0000 7fa5e941a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e41420f0 0x7fa5e415be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:05.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.752+0000 7fa5e941a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e41425a0 0x7fa5e415c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:05.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.752+0000 7fa5e941a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e4128620 con 0x7fa5e40ca160 2023-10-18T09:00:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.752+0000 7fa5e941a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e4128010 con 0x7fa5e41425a0 2023-10-18T09:00:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.752+0000 7fa5e941a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e41e46b0 con 0x7fa5e41420f0 2023-10-18T09:00:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.753+0000 7fa5e3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5e40ca160 0x7fa5e40abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.753+0000 7fa5e3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5e40ca160 0x7fa5e40abde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44136/0 (socket says 172.21.15.40:44136) 2023-10-18T09:00:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.753+0000 7fa5e3fff640 1 -- 172.21.15.40:0/2653066130 learned_addr learned my addr 172.21.15.40:0/2653066130 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.753+0000 7fa5e37fe640 1 --2- 172.21.15.40:0/2653066130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e41420f0 0x7fa5e415be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.753+0000 7fa5e8c19640 1 --2- 172.21.15.40:0/2653066130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e41425a0 0x7fa5e415c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.753+0000 7fa5e3fff640 1 -- 172.21.15.40:0/2653066130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e41420f0 msgr2=0x7fa5e415be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:05.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.753+0000 7fa5e3fff640 1 --2- 172.21.15.40:0/2653066130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e41420f0 0x7fa5e415be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.753+0000 7fa5e3fff640 1 -- 172.21.15.40:0/2653066130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e41425a0 msgr2=0x7fa5e415c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.753+0000 7fa5e3fff640 1 --2- 172.21.15.40:0/2653066130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e41425a0 0x7fa5e415c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.753+0000 7fa5e3fff640 1 -- 172.21.15.40:0/2653066130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d8009490 con 0x7fa5e40ca160 2023-10-18T09:00:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.754+0000 7fa5e3fff640 1 --2- 172.21.15.40:0/2653066130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5e40ca160 0x7fa5e40abde0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400c770 tx=0x7fa5d400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.754+0000 7fa5e0ff9640 1 -- 172.21.15.40:0/2653066130 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d400a840 con 0x7fa5e40ca160 2023-10-18T09:00:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.754+0000 7fa5e0ff9640 1 -- 172.21.15.40:0/2653066130 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa5d400aab0 con 0x7fa5e40ca160 2023-10-18T09:00:05.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.754+0000 7fa5e0ff9640 1 -- 172.21.15.40:0/2653066130 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4012420 con 0x7fa5e40ca160 2023-10-18T09:00:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.754+0000 7fa5e941a640 1 -- 172.21.15.40:0/2653066130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e41282b0 con 0x7fa5e40ca160 2023-10-18T09:00:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.754+0000 7fa5e941a640 1 -- 172.21.15.40:0/2653066130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e41e3930 con 0x7fa5e40ca160 2023-10-18T09:00:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.755+0000 7fa5e941a640 1 -- 172.21.15.40:0/2653066130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e40c98c0 con 0x7fa5e40ca160 2023-10-18T09:00:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.756+0000 7fa5e0ff9640 1 -- 172.21.15.40:0/2653066130 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa5d401a3b0 con 0x7fa5e40ca160 2023-10-18T09:00:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.758+0000 7fa5e0ff9640 1 --2- 172.21.15.40:0/2653066130 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa5d006ae40 0x7fa5d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.758+0000 7fa5e0ff9640 1 -- 172.21.15.40:0/2653066130 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa5d40897e0 con 0x7fa5e40ca160 2023-10-18T09:00:05.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.758+0000 7fa5e37fe640 1 --2- 172.21.15.40:0/2653066130 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa5d006ae40 0x7fa5d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.760+0000 7fa5e37fe640 1 --2- 172.21.15.40:0/2653066130 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa5d006ae40 0x7fa5d006d300 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8036000 tx=0x7fa5d8004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:05.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.762+0000 7fa5e0ff9640 1 -- 172.21.15.40:0/2653066130 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5d4054a60 con 0x7fa5e40ca160 2023-10-18T09:00:05.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.958+0000 7fa5e941a640 1 -- 172.21.15.40:0/2653066130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5e4066e00 con 0x7fa5e40ca160 2023-10-18T09:00:05.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.960+0000 7fa5e0ff9640 1 -- 172.21.15.40:0/2653066130 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+13000 (secure 0 0 0) 0x7fa5d4016020 con 0x7fa5e40ca160 2023-10-18T09:00:05.962 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:05.962 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":79,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:00:04.472067+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:6837/1311290970":"2023-10-19T09:00:01.455779+0000","172.21.15.144:6836/1311290970":"2023-10-19T09:00:01.455779+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:00:05.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.963+0000 7fa5e941a640 1 -- 172.21.15.40:0/2653066130 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa5d006ae40 msgr2=0x7fa5d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:05.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.963+0000 7fa5e941a640 1 --2- 172.21.15.40:0/2653066130 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa5d006ae40 0x7fa5d006d300 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8036000 tx=0x7fa5d8004730 comp rx=0 tx=0).stop 2023-10-18T09:00:05.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.963+0000 7fa5e941a640 1 -- 172.21.15.40:0/2653066130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5e40ca160 msgr2=0x7fa5e40abde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:05.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.963+0000 7fa5e941a640 1 --2- 172.21.15.40:0/2653066130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5e40ca160 0x7fa5e40abde0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400c770 tx=0x7fa5d400cc30 comp rx=0 tx=0).stop 2023-10-18T09:00:05.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.963+0000 7fa5e941a640 1 -- 172.21.15.40:0/2653066130 shutdown_connections 2023-10-18T09:00:05.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.963+0000 7fa5e941a640 1 --2- 172.21.15.40:0/2653066130 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa5d006ae40 0x7fa5d006d300 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.963+0000 7fa5e941a640 1 --2- 172.21.15.40:0/2653066130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5e41425a0 0x7fa5e415c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.963+0000 7fa5e941a640 1 --2- 172.21.15.40:0/2653066130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5e41420f0 0x7fa5e415be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.963+0000 7fa5e941a640 1 --2- 172.21.15.40:0/2653066130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5e40ca160 0x7fa5e40abde0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.963+0000 7fa5e941a640 1 -- 172.21.15.40:0/2653066130 >> 172.21.15.40:0/2653066130 conn(0x7fa5e4069720 msgr2=0x7fa5e413b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:05.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.963+0000 7fa5e941a640 1 -- 172.21.15.40:0/2653066130 shutdown_connections 2023-10-18T09:00:05.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:05.963+0000 7fa5e941a640 1 -- 172.21.15.40:0/2653066130 wait complete. 2023-10-18T09:00:05.985 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:00:05.986 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:00:06.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.209+0000 7f482bb97640 1 -- 172.21.15.40:0/2583896961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48240aed00 msgr2=0x7f48240af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:06.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.209+0000 7f482bb97640 1 --2- 172.21.15.40:0/2583896961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48240aed00 0x7f48240af170 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f481c009830 tx=0x7f481c02bd60 comp rx=0 tx=0).stop 2023-10-18T09:00:06.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.209+0000 7f482bb97640 1 -- 172.21.15.40:0/2583896961 shutdown_connections 2023-10-18T09:00:06.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.209+0000 7f482bb97640 1 --2- 172.21.15.40:0/2583896961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48240aed00 0x7f48240af170 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:06.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.209+0000 7f482bb97640 1 --2- 172.21.15.40:0/2583896961 >> v2:172.21.15.144:3300/0 conn(0x7f48240ac7b0 0x7f48240cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:06.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.209+0000 7f482bb97640 1 -- 172.21.15.40:0/2583896961 >> 172.21.15.40:0/2583896961 conn(0x7f4824069720 msgr2=0x7f4824069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:06.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.209+0000 7f482bb97640 1 -- 172.21.15.40:0/2583896961 shutdown_connections 2023-10-18T09:00:06.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.209+0000 7f482bb97640 1 -- 172.21.15.40:0/2583896961 wait complete. 2023-10-18T09:00:06.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.211+0000 7f482bb97640 1 Processor -- start 2023-10-18T09:00:06.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.211+0000 7f482bb97640 1 -- start start 2023-10-18T09:00:06.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.212+0000 7f482bb97640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48240ac7b0 0x7f4824157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:06.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.212+0000 7f482bb97640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48240aed00 0x7f48241562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:06.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.212+0000 7f482bb97640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4824155cd0 0x7f4824149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:06.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.212+0000 7f482bb97640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48241f62d0 con 0x7f48240aed00 2023-10-18T09:00:06.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.212+0000 7f482bb97640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4824152ff0 con 0x7f48240ac7b0 2023-10-18T09:00:06.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.212+0000 7f482bb97640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48241582f0 con 0x7f4824155cd0 2023-10-18T09:00:06.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.213+0000 7f4829132640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48240aed00 0x7f48241562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:06.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.213+0000 7f4829132640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48240aed00 0x7f48241562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44152/0 (socket says 172.21.15.40:44152) 2023-10-18T09:00:06.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.213+0000 7f4829132640 1 -- 172.21.15.40:0/957638018 learned_addr learned my addr 172.21.15.40:0/957638018 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:06.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.213+0000 7f4829933640 1 --2- 172.21.15.40:0/957638018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48240ac7b0 0x7f4824157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:06.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.213+0000 7f482a134640 1 --2- 172.21.15.40:0/957638018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4824155cd0 0x7f4824149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:06.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.213+0000 7f4829132640 1 -- 172.21.15.40:0/957638018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4824155cd0 msgr2=0x7f4824149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:06.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.213+0000 7f4829132640 1 --2- 172.21.15.40:0/957638018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4824155cd0 0x7f4824149900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:06.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.213+0000 7f4829132640 1 -- 172.21.15.40:0/957638018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48240ac7b0 msgr2=0x7f4824157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:06.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.213+0000 7f4829132640 1 --2- 172.21.15.40:0/957638018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48240ac7b0 0x7f4824157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:06.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.213+0000 7f4829132640 1 -- 172.21.15.40:0/957638018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c009490 con 0x7f48240aed00 2023-10-18T09:00:06.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.213+0000 7f4829933640 1 --2- 172.21.15.40:0/957638018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48240ac7b0 0x7f4824157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:06.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.213+0000 7f4829132640 1 --2- 172.21.15.40:0/957638018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48240aed00 0x7f48241562c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f481400d7a0 tx=0x7f481400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:06.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.214+0000 7f481a7fc640 1 -- 172.21.15.40:0/957638018 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814009c30 con 0x7f48240aed00 2023-10-18T09:00:06.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.214+0000 7f481a7fc640 1 -- 172.21.15.40:0/957638018 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4814009e80 con 0x7f48240aed00 2023-10-18T09:00:06.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.214+0000 7f482bb97640 1 -- 172.21.15.40:0/957638018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48241f5750 con 0x7f48240aed00 2023-10-18T09:00:06.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.214+0000 7f481a7fc640 1 -- 172.21.15.40:0/957638018 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814004260 con 0x7f48240aed00 2023-10-18T09:00:06.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.214+0000 7f482bb97640 1 -- 172.21.15.40:0/957638018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48241f5d10 con 0x7f48240aed00 2023-10-18T09:00:06.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.215+0000 7f482bb97640 1 -- 172.21.15.40:0/957638018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4824151010 con 0x7f48240aed00 2023-10-18T09:00:06.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.216+0000 7f481a7fc640 1 -- 172.21.15.40:0/957638018 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f48140035f0 con 0x7f48240aed00 2023-10-18T09:00:06.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.217+0000 7f481a7fc640 1 --2- 172.21.15.40:0/957638018 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f480806ae40 0x7f480806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:06.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.217+0000 7f481a7fc640 1 -- 172.21.15.40:0/957638018 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4814089590 con 0x7f48240aed00 2023-10-18T09:00:06.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.218+0000 7f4829933640 1 --2- 172.21.15.40:0/957638018 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f480806ae40 0x7f480806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:06.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.219+0000 7f4829933640 1 --2- 172.21.15.40:0/957638018 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f480806ae40 0x7f480806d300 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4820001560 tx=0x7f4820009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:06.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.222+0000 7f481a7fc640 1 -- 172.21.15.40:0/957638018 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f48140548a0 con 0x7f48240aed00 2023-10-18T09:00:06.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.414+0000 7f482bb97640 1 -- 172.21.15.40:0/957638018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4824066fd0 con 0x7f48240aed00 2023-10-18T09:00:06.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.415+0000 7f481a7fc640 1 -- 172.21.15.40:0/957638018 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48140170a0 con 0x7f48240aed00 2023-10-18T09:00:06.418 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:00:06.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.418+0000 7f482bb97640 1 -- 172.21.15.40:0/957638018 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f480806ae40 msgr2=0x7f480806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:06.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.418+0000 7f482bb97640 1 --2- 172.21.15.40:0/957638018 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f480806ae40 0x7f480806d300 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4820001560 tx=0x7f4820009040 comp rx=0 tx=0).stop 2023-10-18T09:00:06.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.418+0000 7f482bb97640 1 -- 172.21.15.40:0/957638018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48240aed00 msgr2=0x7f48241562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:06.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.418+0000 7f482bb97640 1 --2- 172.21.15.40:0/957638018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48240aed00 0x7f48241562c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f481400d7a0 tx=0x7f481400dc60 comp rx=0 tx=0).stop 2023-10-18T09:00:06.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.418+0000 7f482bb97640 1 -- 172.21.15.40:0/957638018 shutdown_connections 2023-10-18T09:00:06.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.418+0000 7f482bb97640 1 --2- 172.21.15.40:0/957638018 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f480806ae40 0x7f480806d300 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:06.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.418+0000 7f482bb97640 1 --2- 172.21.15.40:0/957638018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4824155cd0 0x7f4824149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:06.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.418+0000 7f482bb97640 1 --2- 172.21.15.40:0/957638018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48240aed00 0x7f48241562c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:06.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.418+0000 7f482bb97640 1 --2- 172.21.15.40:0/957638018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48240ac7b0 0x7f4824157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:06.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.418+0000 7f482bb97640 1 -- 172.21.15.40:0/957638018 >> 172.21.15.40:0/957638018 conn(0x7f4824069720 msgr2=0x7f4824154ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:06.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.418+0000 7f482bb97640 1 -- 172.21.15.40:0/957638018 shutdown_connections 2023-10-18T09:00:06.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.418+0000 7f482bb97640 1 -- 172.21.15.40:0/957638018 wait complete. 2023-10-18T09:00:06.439 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2023-10-18T09:00:06.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.650+0000 7f2a19cc3640 1 -- 172.21.15.40:0/1227891219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a1406d6f0 msgr2=0x7f2a140b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:06.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.650+0000 7f2a19cc3640 1 --2- 172.21.15.40:0/1227891219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a1406d6f0 0x7f2a140b15b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2a1406b050 tx=0x7f2a0802bd90 comp rx=0 tx=0).stop 2023-10-18T09:00:06.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.650+0000 7f2a19cc3640 1 -- 172.21.15.40:0/1227891219 shutdown_connections 2023-10-18T09:00:06.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.650+0000 7f2a19cc3640 1 --2- 172.21.15.40:0/1227891219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a140c9f00 0x7f2a14157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:06.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.650+0000 7f2a19cc3640 1 --2- 172.21.15.40:0/1227891219 >> v2:172.21.15.144:3300/0 conn(0x7f2a140b0630 0x7f2a140c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:06.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.650+0000 7f2a19cc3640 1 --2- 172.21.15.40:0/1227891219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a1406d6f0 0x7f2a140b15b0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:06.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.650+0000 7f2a19cc3640 1 -- 172.21.15.40:0/1227891219 >> 172.21.15.40:0/1227891219 conn(0x7f2a14069720 msgr2=0x7f2a14069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:06.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.650+0000 7f2a19cc3640 1 -- 172.21.15.40:0/1227891219 shutdown_connections 2023-10-18T09:00:06.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.650+0000 7f2a19cc3640 1 -- 172.21.15.40:0/1227891219 wait complete. 2023-10-18T09:00:06.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.652+0000 7f2a19cc3640 1 Processor -- start 2023-10-18T09:00:06.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.652+0000 7f2a19cc3640 1 -- start start 2023-10-18T09:00:06.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.653+0000 7f2a19cc3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a1406d6f0 0x7f2a14142a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:06.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.653+0000 7f2a19cc3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a140b0630 0x7f2a141482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:06.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.653+0000 7f2a19cc3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a140c9f00 0x7f2a14159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:06.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.653+0000 7f2a19cc3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a14142f40 con 0x7f2a140b0630 2023-10-18T09:00:06.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.653+0000 7f2a19cc3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140ab870 con 0x7f2a140c9f00 2023-10-18T09:00:06.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.653+0000 7f2a19cc3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140ab590 con 0x7f2a1406d6f0 2023-10-18T09:00:06.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.653+0000 7f2a13fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a140b0630 0x7f2a141482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:06.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.653+0000 7f2a13fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a140b0630 0x7f2a141482c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44166/0 (socket says 172.21.15.40:44166) 2023-10-18T09:00:06.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.653+0000 7f2a13fff640 1 -- 172.21.15.40:0/2427836676 learned_addr learned my addr 172.21.15.40:0/2427836676 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:06.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.654+0000 7f2a194c2640 1 --2- 172.21.15.40:0/2427836676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a140c9f00 0x7f2a14159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:06.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.654+0000 7f2a18cc1640 1 --2- 172.21.15.40:0/2427836676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a1406d6f0 0x7f2a14142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:06.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.654+0000 7f2a13fff640 1 -- 172.21.15.40:0/2427836676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a1406d6f0 msgr2=0x7f2a14142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:06.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.654+0000 7f2a13fff640 1 --2- 172.21.15.40:0/2427836676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a1406d6f0 0x7f2a14142a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:06.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.654+0000 7f2a13fff640 1 -- 172.21.15.40:0/2427836676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a140c9f00 msgr2=0x7f2a14159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:06.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.654+0000 7f2a13fff640 1 --2- 172.21.15.40:0/2427836676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a140c9f00 0x7f2a14159c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:06.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.654+0000 7f2a13fff640 1 -- 172.21.15.40:0/2427836676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a08009560 con 0x7f2a140b0630 2023-10-18T09:00:06.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.654+0000 7f2a13fff640 1 --2- 172.21.15.40:0/2427836676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a140b0630 0x7f2a141482c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000c920 tx=0x7f2a0000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:06.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.654+0000 7f2a117fa640 1 -- 172.21.15.40:0/2427836676 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a00007c70 con 0x7f2a140b0630 2023-10-18T09:00:06.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.654+0000 7f2a117fa640 1 -- 172.21.15.40:0/2427836676 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2a0000f450 con 0x7f2a140b0630 2023-10-18T09:00:06.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.654+0000 7f2a19cc3640 1 -- 172.21.15.40:0/2427836676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a140acb80 con 0x7f2a140b0630 2023-10-18T09:00:06.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.654+0000 7f2a117fa640 1 -- 172.21.15.40:0/2427836676 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0000f860 con 0x7f2a140b0630 2023-10-18T09:00:06.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.654+0000 7f2a19cc3640 1 -- 172.21.15.40:0/2427836676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a140ad140 con 0x7f2a140b0630 2023-10-18T09:00:06.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.656+0000 7f2a19cc3640 1 -- 172.21.15.40:0/2427836676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a14066e00 con 0x7f2a140b0630 2023-10-18T09:00:06.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.657+0000 7f2a117fa640 1 -- 172.21.15.40:0/2427836676 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2a0000f9c0 con 0x7f2a140b0630 2023-10-18T09:00:06.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.658+0000 7f2a117fa640 1 --2- 172.21.15.40:0/2427836676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29f806ae70 0x7f29f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:06.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.658+0000 7f2a117fa640 1 -- 172.21.15.40:0/2427836676 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2a00089bf0 con 0x7f2a140b0630 2023-10-18T09:00:06.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.658+0000 7f2a18cc1640 1 --2- 172.21.15.40:0/2427836676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29f806ae70 0x7f29f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:06.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.659+0000 7f2a18cc1640 1 --2- 172.21.15.40:0/2427836676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29f806ae70 0x7f29f806d330 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2a08035000 tx=0x7f2a08004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:06.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.662+0000 7f2a117fa640 1 -- 172.21.15.40:0/2427836676 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a00054fa0 con 0x7f2a140b0630 2023-10-18T09:00:06.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:06.921+0000 7f2a19cc3640 1 -- 172.21.15.40:0/2427836676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f2a140cc110 con 0x7f2a140b0630 2023-10-18T09:00:07.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.490+0000 7f2a117fa640 1 -- 172.21.15.40:0/2427836676 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v259) v1 ==== 136+0+0 (secure 0 0 0) 0x7f2a000585c0 con 0x7f2a140b0630 2023-10-18T09:00:07.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.494+0000 7f2a19cc3640 1 -- 172.21.15.40:0/2427836676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29f806ae70 msgr2=0x7f29f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:07.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.494+0000 7f2a19cc3640 1 --2- 172.21.15.40:0/2427836676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29f806ae70 0x7f29f806d330 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2a08035000 tx=0x7f2a08004730 comp rx=0 tx=0).stop 2023-10-18T09:00:07.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.494+0000 7f2a19cc3640 1 -- 172.21.15.40:0/2427836676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a140b0630 msgr2=0x7f2a141482c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:07.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.494+0000 7f2a19cc3640 1 --2- 172.21.15.40:0/2427836676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a140b0630 0x7f2a141482c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000c920 tx=0x7f2a0000cde0 comp rx=0 tx=0).stop 2023-10-18T09:00:07.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.494+0000 7f2a19cc3640 1 -- 172.21.15.40:0/2427836676 shutdown_connections 2023-10-18T09:00:07.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.494+0000 7f2a19cc3640 1 --2- 172.21.15.40:0/2427836676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29f806ae70 0x7f29f806d330 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:07.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.494+0000 7f2a19cc3640 1 --2- 172.21.15.40:0/2427836676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a140c9f00 0x7f2a14159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:07.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.494+0000 7f2a19cc3640 1 --2- 172.21.15.40:0/2427836676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a140b0630 0x7f2a141482c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:07.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.494+0000 7f2a19cc3640 1 --2- 172.21.15.40:0/2427836676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a1406d6f0 0x7f2a14142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:07.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.494+0000 7f2a19cc3640 1 -- 172.21.15.40:0/2427836676 >> 172.21.15.40:0/2427836676 conn(0x7f2a14069720 msgr2=0x7f2a14155610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:07.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.494+0000 7f2a19cc3640 1 -- 172.21.15.40:0/2427836676 shutdown_connections 2023-10-18T09:00:07.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.494+0000 7f2a19cc3640 1 -- 172.21.15.40:0/2427836676 wait complete. 2023-10-18T09:00:07.518 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-18T09:00:07.746 INFO:teuthology.orchestra.run.smithi040.stdout:{"osd_objectstore":"memstore"} 2023-10-18T09:00:07.746 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-18T09:00:07.747 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T09:00:07.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.974+0000 7f2b363d0640 1 -- 172.21.15.40:0/297728443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b1c0047c0 msgr2=0x7f2b1c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.974+0000 7f2b363d0640 1 --2- 172.21.15.40:0/297728443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b1c0047c0 0x7f2b1c004c40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2b180014e0 tx=0x7f2b1c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:00:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.974+0000 7f2b3496d640 1 -- 172.21.15.40:0/297728443 reap_dead start 2023-10-18T09:00:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.974+0000 7f2b363d0640 1 -- 172.21.15.40:0/297728443 shutdown_connections 2023-10-18T09:00:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.974+0000 7f2b363d0640 1 -- 172.21.15.40:0/297728443 >> 172.21.15.40:0/297728443 conn(0x7f2b30069720 msgr2=0x7f2b30069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:07.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.974+0000 7f2b363d0640 1 -- 172.21.15.40:0/297728443 shutdown_connections 2023-10-18T09:00:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.974+0000 7f2b363d0640 1 -- 172.21.15.40:0/297728443 wait complete. 2023-10-18T09:00:07.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.976+0000 7f2b363d0640 1 Processor -- start 2023-10-18T09:00:07.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.976+0000 7f2b363d0640 1 -- start start 2023-10-18T09:00:07.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.977+0000 7f2b363d0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b1c0047c0 0x7f2b30156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:07.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.977+0000 7f2b363d0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b30150ee0 0x7f2b3015ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:07.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.977+0000 7f2b363d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b30151360 0x7f2b30156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:07.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.977+0000 7f2b363d0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3014fe40 con 0x7f2b1c0047c0 2023-10-18T09:00:07.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.977+0000 7f2b363d0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b30150120 con 0x7f2b30151360 2023-10-18T09:00:07.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.977+0000 7f2b363d0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b30150400 con 0x7f2b30150ee0 2023-10-18T09:00:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.977+0000 7f2b2ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b1c0047c0 0x7f2b30156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.977+0000 7f2b2ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b1c0047c0 0x7f2b30156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44196/0 (socket says 172.21.15.40:44196) 2023-10-18T09:00:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.977+0000 7f2b2ffff640 1 -- 172.21.15.40:0/3411127338 learned_addr learned my addr 172.21.15.40:0/3411127338 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.978+0000 7f2b2ffff640 1 -- 172.21.15.40:0/3411127338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b30150ee0 msgr2=0x7f2b3015ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:07.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.978+0000 7f2b2ffff640 1 --2- 172.21.15.40:0/3411127338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b30150ee0 0x7f2b3015ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.978+0000 7f2b2ffff640 1 -- 172.21.15.40:0/3411127338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b30151360 msgr2=0x7f2b30156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.978+0000 7f2b2ffff640 1 --2- 172.21.15.40:0/3411127338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b30151360 0x7f2b30156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.978+0000 7f2b2ffff640 1 -- 172.21.15.40:0/3411127338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b301506e0 con 0x7f2b1c0047c0 2023-10-18T09:00:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.978+0000 7f2b2ffff640 1 --2- 172.21.15.40:0/3411127338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b1c0047c0 0x7f2b30156250 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2b200097e0 tx=0x7f2b20009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:07.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.978+0000 7f2b2cff9640 1 -- 172.21.15.40:0/3411127338 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20004d20 con 0x7f2b1c0047c0 2023-10-18T09:00:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.978+0000 7f2b2cff9640 1 -- 172.21.15.40:0/3411127338 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2b2000fb50 con 0x7f2b1c0047c0 2023-10-18T09:00:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.978+0000 7f2b363d0640 1 -- 172.21.15.40:0/3411127338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b301509c0 con 0x7f2b1c0047c0 2023-10-18T09:00:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.978+0000 7f2b2cff9640 1 -- 172.21.15.40:0/3411127338 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20005020 con 0x7f2b1c0047c0 2023-10-18T09:00:07.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.978+0000 7f2b363d0640 1 -- 172.21.15.40:0/3411127338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b30156790 con 0x7f2b1c0047c0 2023-10-18T09:00:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.979+0000 7f2b363d0640 1 -- 172.21.15.40:0/3411127338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b30066fd0 con 0x7f2b1c0047c0 2023-10-18T09:00:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.980+0000 7f2b2cff9640 1 -- 172.21.15.40:0/3411127338 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2b20005180 con 0x7f2b1c0047c0 2023-10-18T09:00:07.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.982+0000 7f2b2cff9640 1 --2- 172.21.15.40:0/3411127338 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2b1006ae40 0x7f2b1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:07.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.982+0000 7f2b2cff9640 1 -- 172.21.15.40:0/3411127338 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2b20015070 con 0x7f2b1c0047c0 2023-10-18T09:00:07.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.983+0000 7f2b2f7fe640 1 --2- 172.21.15.40:0/3411127338 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2b1006ae40 0x7f2b1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:07.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.984+0000 7f2b2f7fe640 1 --2- 172.21.15.40:0/3411127338 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2b1006ae40 0x7f2b1006d300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2b18034000 tx=0x7f2b18001120 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:07.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:07.986+0000 7f2b2cff9640 1 -- 172.21.15.40:0/3411127338 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2b200555d0 con 0x7f2b1c0047c0 2023-10-18T09:00:08.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.176+0000 7f2b363d0640 1 -- 172.21.15.40:0/3411127338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f2b300ab740 con 0x7f2b1c0047c0 2023-10-18T09:00:08.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.177+0000 7f2b2cff9640 1 -- 172.21.15.40:0/3411127338 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v28) v1 ==== 59+0+1452 (secure 0 0 0) 0x7f2b20058bf0 con 0x7f2b1c0047c0 2023-10-18T09:00:08.179 INFO:teuthology.orchestra.run.smithi040.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T09:00:08.179 INFO:teuthology.orchestra.run.smithi040.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T09:00:08.180 INFO:teuthology.orchestra.run.smithi040.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T09:00:08.180 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds 20/20 2023-10-18T09:00:08.180 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T09:00:08.180 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_ms 1/1 2023-10-18T09:00:08.180 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T09:00:08.181 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T09:00:08.181 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T09:00:08.181 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_size 100 2023-10-18T09:00:08.181 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_frag true 2023-10-18T09:00:08.182 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_scatterstat true 2023-10-18T09:00:08.182 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T09:00:08.182 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_verify_scatter true 2023-10-18T09:00:08.182 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T09:00:08.182 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T09:00:08.183 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T09:00:08.183 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced client_mount_timeout 600 2023-10-18T09:00:08.183 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_client 20/20 2023-10-18T09:00:08.183 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_ms 1/1 2023-10-18T09:00:08.183 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T09:00:08.184 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T09:00:08.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.179+0000 7f2b363d0640 1 -- 172.21.15.40:0/3411127338 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2b1006ae40 msgr2=0x7f2b1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:08.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.179+0000 7f2b363d0640 1 --2- 172.21.15.40:0/3411127338 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2b1006ae40 0x7f2b1006d300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2b18034000 tx=0x7f2b18001120 comp rx=0 tx=0).stop 2023-10-18T09:00:08.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.179+0000 7f2b363d0640 1 -- 172.21.15.40:0/3411127338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b1c0047c0 msgr2=0x7f2b30156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:08.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.179+0000 7f2b363d0640 1 --2- 172.21.15.40:0/3411127338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b1c0047c0 0x7f2b30156250 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2b200097e0 tx=0x7f2b20009ca0 comp rx=0 tx=0).stop 2023-10-18T09:00:08.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.179+0000 7f2b363d0640 1 -- 172.21.15.40:0/3411127338 shutdown_connections 2023-10-18T09:00:08.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.179+0000 7f2b363d0640 1 --2- 172.21.15.40:0/3411127338 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2b1006ae40 0x7f2b1006d300 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:08.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.179+0000 7f2b363d0640 1 --2- 172.21.15.40:0/3411127338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b30151360 0x7f2b30156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:08.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.179+0000 7f2b363d0640 1 --2- 172.21.15.40:0/3411127338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b30150ee0 0x7f2b3015ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:08.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.179+0000 7f2b363d0640 1 --2- 172.21.15.40:0/3411127338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b1c0047c0 0x7f2b30156250 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:08.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.179+0000 7f2b363d0640 1 -- 172.21.15.40:0/3411127338 >> 172.21.15.40:0/3411127338 conn(0x7f2b30069720 msgr2=0x7f2b300cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:08.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.179+0000 7f2b363d0640 1 -- 172.21.15.40:0/3411127338 shutdown_connections 2023-10-18T09:00:08.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.180+0000 7f2b363d0640 1 -- 172.21.15.40:0/3411127338 wait complete. 2023-10-18T09:00:08.199 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T09:00:08.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.428+0000 7f2a3cb75640 1 -- 172.21.15.40:0/3399696198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a24004a80 msgr2=0x7f2a24004ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:08.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.428+0000 7f2a3cb75640 1 --2- 172.21.15.40:0/3399696198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a24004a80 0x7f2a24004ea0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2a28001680 tx=0x7f2a24006900 comp rx=0 tx=0).stop 2023-10-18T09:00:08.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.428+0000 7f2a3cb75640 1 -- 172.21.15.40:0/3399696198 shutdown_connections 2023-10-18T09:00:08.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.428+0000 7f2a3cb75640 1 --2- 172.21.15.40:0/3399696198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a24004a80 0x7f2a24004ea0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:08.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.428+0000 7f2a3cb75640 1 -- 172.21.15.40:0/3399696198 >> 172.21.15.40:0/3399696198 conn(0x7f2a3806ca10 msgr2=0x7f2a3806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:08.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.428+0000 7f2a3cb75640 1 -- 172.21.15.40:0/3399696198 shutdown_connections 2023-10-18T09:00:08.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.428+0000 7f2a3cb75640 1 -- 172.21.15.40:0/3399696198 wait complete. 2023-10-18T09:00:08.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.430+0000 7f2a3cb75640 1 Processor -- start 2023-10-18T09:00:08.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.430+0000 7f2a3cb75640 1 -- start start 2023-10-18T09:00:08.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.431+0000 7f2a3cb75640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a24004a80 0x7f2a38152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:08.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.431+0000 7f2a3cb75640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a381e65b0 0x7f2a380cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:08.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.431+0000 7f2a3cb75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a380cd740 0x7f2a38158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:08.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.431+0000 7f2a3cb75640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a380cdbc0 con 0x7f2a380cd740 2023-10-18T09:00:08.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.431+0000 7f2a3cb75640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3813e570 con 0x7f2a381e65b0 2023-10-18T09:00:08.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.431+0000 7f2a3cb75640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3813e850 con 0x7f2a24004a80 2023-10-18T09:00:08.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.431+0000 7f2a37fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a380cd740 0x7f2a38158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:08.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.431+0000 7f2a37fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a380cd740 0x7f2a38158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44210/0 (socket says 172.21.15.40:44210) 2023-10-18T09:00:08.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.431+0000 7f2a37fff640 1 -- 172.21.15.40:0/4220523329 learned_addr learned my addr 172.21.15.40:0/4220523329 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:08.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.431+0000 7f2a37fff640 1 -- 172.21.15.40:0/4220523329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a24004a80 msgr2=0x7f2a38152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:08.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.431+0000 7f2a37fff640 1 --2- 172.21.15.40:0/4220523329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a24004a80 0x7f2a38152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:08.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.431+0000 7f2a37fff640 1 -- 172.21.15.40:0/4220523329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a381e65b0 msgr2=0x7f2a380cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:08.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.431+0000 7f2a37fff640 1 --2- 172.21.15.40:0/4220523329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a381e65b0 0x7f2a380cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:08.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.432+0000 7f2a37fff640 1 -- 172.21.15.40:0/4220523329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3813eb30 con 0x7f2a380cd740 2023-10-18T09:00:08.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.432+0000 7f2a37fff640 1 --2- 172.21.15.40:0/4220523329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a380cd740 0x7f2a38158b10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2a380afcb0 tx=0x7f2a3000ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:08.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.432+0000 7f2a34ff9640 1 -- 172.21.15.40:0/4220523329 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30003da0 con 0x7f2a380cd740 2023-10-18T09:00:08.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.432+0000 7f2a34ff9640 1 -- 172.21.15.40:0/4220523329 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2a3000c860 con 0x7f2a380cd740 2023-10-18T09:00:08.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.432+0000 7f2a34ff9640 1 -- 172.21.15.40:0/4220523329 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000cc30 con 0x7f2a380cd740 2023-10-18T09:00:08.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.432+0000 7f2a3cb75640 1 -- 172.21.15.40:0/4220523329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3813ee10 con 0x7f2a380cd740 2023-10-18T09:00:08.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.432+0000 7f2a3cb75640 1 -- 172.21.15.40:0/4220523329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a381e6a10 con 0x7f2a380cd740 2023-10-18T09:00:08.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.433+0000 7f2a3cb75640 1 -- 172.21.15.40:0/4220523329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a381e6bd0 con 0x7f2a380cd740 2023-10-18T09:00:08.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.434+0000 7f2a34ff9640 1 -- 172.21.15.40:0/4220523329 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2a3000cd90 con 0x7f2a380cd740 2023-10-18T09:00:08.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.436+0000 7f2a34ff9640 1 --2- 172.21.15.40:0/4220523329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a1806ae40 0x7f2a1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:08.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.436+0000 7f2a34ff9640 1 -- 172.21.15.40:0/4220523329 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2a30089b00 con 0x7f2a380cd740 2023-10-18T09:00:08.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.436+0000 7f2a377fe640 1 --2- 172.21.15.40:0/4220523329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a1806ae40 0x7f2a1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:08.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.437+0000 7f2a377fe640 1 --2- 172.21.15.40:0/4220523329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a1806ae40 0x7f2a1806d300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2a28034000 tx=0x7f2a28001120 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:08.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.441+0000 7f2a34ff9640 1 -- 172.21.15.40:0/4220523329 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a3008be60 con 0x7f2a380cd740 2023-10-18T09:00:08.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.633+0000 7f2a3cb75640 1 -- 172.21.15.40:0/4220523329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2a38066e00 con 0x7f2a380cd740 2023-10-18T09:00:08.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.640+0000 7f2a34ff9640 1 -- 172.21.15.40:0/4220523329 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v29)=0 v29) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2a30054d80 con 0x7f2a380cd740 2023-10-18T09:00:08.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.643+0000 7f2a3cb75640 1 -- 172.21.15.40:0/4220523329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a1806ae40 msgr2=0x7f2a1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:08.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.643+0000 7f2a3cb75640 1 --2- 172.21.15.40:0/4220523329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a1806ae40 0x7f2a1806d300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2a28034000 tx=0x7f2a28001120 comp rx=0 tx=0).stop 2023-10-18T09:00:08.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.643+0000 7f2a3cb75640 1 -- 172.21.15.40:0/4220523329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a380cd740 msgr2=0x7f2a38158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:08.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.643+0000 7f2a3cb75640 1 --2- 172.21.15.40:0/4220523329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a380cd740 0x7f2a38158b10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2a380afcb0 tx=0x7f2a3000ee10 comp rx=0 tx=0).stop 2023-10-18T09:00:08.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.643+0000 7f2a3cb75640 1 -- 172.21.15.40:0/4220523329 shutdown_connections 2023-10-18T09:00:08.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.643+0000 7f2a3cb75640 1 --2- 172.21.15.40:0/4220523329 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a1806ae40 0x7f2a1806d300 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:08.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.643+0000 7f2a3cb75640 1 --2- 172.21.15.40:0/4220523329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a380cd740 0x7f2a38158b10 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:08.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.643+0000 7f2a3cb75640 1 --2- 172.21.15.40:0/4220523329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a381e65b0 0x7f2a380cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:08.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.643+0000 7f2a3cb75640 1 --2- 172.21.15.40:0/4220523329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a24004a80 0x7f2a38152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:08.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.643+0000 7f2a3cb75640 1 -- 172.21.15.40:0/4220523329 >> 172.21.15.40:0/4220523329 conn(0x7f2a3806ca10 msgr2=0x7f2a38151d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:08.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.643+0000 7f2a3cb75640 1 -- 172.21.15.40:0/4220523329 shutdown_connections 2023-10-18T09:00:08.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:08.643+0000 7f2a3cb75640 1 -- 172.21.15.40:0/4220523329 wait complete. 2023-10-18T09:00:08.664 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T09:00:08.665 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:00:08.785 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:00:08.920 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T09:00:08.920 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:00:08.921 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:00:08.985 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:00:08.986 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:00:08.986 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:00:09.046 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:00:09.047 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:00:09.047 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:00:09.113 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:00:09.114 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:00:09.114 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:00:09.174 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:00:09.175 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T09:00:09.175 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:00:09.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.412+0000 7f3011202640 1 -- 172.21.15.40:0/702872960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff4004ab0 msgr2=0x7f2ff4004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:09.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.412+0000 7f3011202640 1 --2- 172.21.15.40:0/702872960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff4004ab0 0x7f2ff4004ed0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009b20 tx=0x7f2ff4006900 comp rx=0 tx=0).stop 2023-10-18T09:00:09.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.412+0000 7f3010a01640 1 -- 172.21.15.40:0/702872960 reap_dead start 2023-10-18T09:00:09.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.412+0000 7f3011202640 1 -- 172.21.15.40:0/702872960 shutdown_connections 2023-10-18T09:00:09.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.412+0000 7f3011202640 1 -- 172.21.15.40:0/702872960 >> 172.21.15.40:0/702872960 conn(0x7f300c069720 msgr2=0x7f300c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:09.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.412+0000 7f3011202640 1 -- 172.21.15.40:0/702872960 shutdown_connections 2023-10-18T09:00:09.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.412+0000 7f3011202640 1 -- 172.21.15.40:0/702872960 wait complete. 2023-10-18T09:00:09.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.414+0000 7f3011202640 1 Processor -- start 2023-10-18T09:00:09.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.414+0000 7f3011202640 1 -- start start 2023-10-18T09:00:09.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.415+0000 7f3011202640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff4004ab0 0x7f300c1411d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:09.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.415+0000 7f3011202640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f300c0a6470 0x7f300c0a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:09.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.415+0000 7f3011202640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f300c0ae700 0x7f300c0aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:09.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.415+0000 7f3011202640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c13f310 con 0x7f2ff4004ab0 2023-10-18T09:00:09.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.415+0000 7f3011202640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c0a6e30 con 0x7f300c0a6470 2023-10-18T09:00:09.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.415+0000 7f3011202640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300c141710 con 0x7f300c0ae700 2023-10-18T09:00:09.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.415+0000 7f300bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff4004ab0 0x7f300c1411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:09.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.415+0000 7f300bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff4004ab0 0x7f300c1411d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44238/0 (socket says 172.21.15.40:44238) 2023-10-18T09:00:09.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.415+0000 7f300bfff640 1 -- 172.21.15.40:0/3441404407 learned_addr learned my addr 172.21.15.40:0/3441404407 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:09.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.416+0000 7f3010a01640 1 --2- 172.21.15.40:0/3441404407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f300c0ae700 0x7f300c0aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:09.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.416+0000 7f300b7fe640 1 --2- 172.21.15.40:0/3441404407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f300c0a6470 0x7f300c0a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:09.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.416+0000 7f300bfff640 1 -- 172.21.15.40:0/3441404407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f300c0ae700 msgr2=0x7f300c0aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:09.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.416+0000 7f300bfff640 1 --2- 172.21.15.40:0/3441404407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f300c0ae700 0x7f300c0aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:09.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.416+0000 7f300bfff640 1 -- 172.21.15.40:0/3441404407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f300c0a6470 msgr2=0x7f300c0a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:09.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.416+0000 7f300bfff640 1 --2- 172.21.15.40:0/3441404407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f300c0a6470 0x7f300c0a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:09.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.416+0000 7f300bfff640 1 -- 172.21.15.40:0/3441404407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300c0af430 con 0x7f2ff4004ab0 2023-10-18T09:00:09.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.416+0000 7f300b7fe640 1 --2- 172.21.15.40:0/3441404407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f300c0a6470 0x7f300c0a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:09.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.416+0000 7f300bfff640 1 --2- 172.21.15.40:0/3441404407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff4004ab0 0x7f300c1411d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009b20 tx=0x7f2ffc033c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:09.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.417+0000 7f3008ff9640 1 -- 172.21.15.40:0/3441404407 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc004460 con 0x7f2ff4004ab0 2023-10-18T09:00:09.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.417+0000 7f3008ff9640 1 -- 172.21.15.40:0/3441404407 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2ffc033e60 con 0x7f2ff4004ab0 2023-10-18T09:00:09.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.417+0000 7f3008ff9640 1 -- 172.21.15.40:0/3441404407 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc003830 con 0x7f2ff4004ab0 2023-10-18T09:00:09.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.417+0000 7f3011202640 1 -- 172.21.15.40:0/3441404407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300c0af710 con 0x7f2ff4004ab0 2023-10-18T09:00:09.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.417+0000 7f3011202640 1 -- 172.21.15.40:0/3441404407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300c0afcd0 con 0x7f2ff4004ab0 2023-10-18T09:00:09.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.418+0000 7f3011202640 1 -- 172.21.15.40:0/3441404407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300c066e00 con 0x7f2ff4004ab0 2023-10-18T09:00:09.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.419+0000 7f3008ff9640 1 -- 172.21.15.40:0/3441404407 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2ffc003990 con 0x7f2ff4004ab0 2023-10-18T09:00:09.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.420+0000 7f3008ff9640 1 --2- 172.21.15.40:0/3441404407 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ff806ae40 0x7f2ff806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:09.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.420+0000 7f3008ff9640 1 -- 172.21.15.40:0/3441404407 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2ffc0ac7f0 con 0x7f2ff4004ab0 2023-10-18T09:00:09.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.421+0000 7f300b7fe640 1 --2- 172.21.15.40:0/3441404407 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ff806ae40 0x7f2ff806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:09.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.422+0000 7f300b7fe640 1 --2- 172.21.15.40:0/3441404407 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ff806ae40 0x7f2ff806d300 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f300c14d5c0 tx=0x7f3000009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:09.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.424+0000 7f3008ff9640 1 -- 172.21.15.40:0/3441404407 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ffc077cc0 con 0x7f2ff4004ab0 2023-10-18T09:00:09.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.662+0000 7f3011202640 1 -- 172.21.15.40:0/3441404407 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f300c0caf90 con 0x7f2ff4004ab0 2023-10-18T09:00:09.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.663+0000 7f3008ff9640 1 -- 172.21.15.40:0/3441404407 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f2ffc04b070 con 0x7f2ff4004ab0 2023-10-18T09:00:09.665 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:09.666 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":35,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5075,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2165596806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2165596806},{"type":"v1","addr":"172.21.15.144:6835","nonce":2165596806}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":5078,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/587522883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":587522883},{"type":"v1","addr":"172.21.15.40:6833","nonce":587522883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1174134524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1174134524},{"type":"v1","addr":"172.21.15.40:6835","nonce":1174134524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5222,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3423671126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":3423671126},{"type":"v1","addr":"172.21.15.144:6837","nonce":3423671126}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[]} 2023-10-18T09:00:09.666 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 35 2023-10-18T09:00:09.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.666+0000 7f3011202640 1 -- 172.21.15.40:0/3441404407 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ff806ae40 msgr2=0x7f2ff806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:09.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.666+0000 7f3011202640 1 --2- 172.21.15.40:0/3441404407 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ff806ae40 0x7f2ff806d300 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f300c14d5c0 tx=0x7f3000009040 comp rx=0 tx=0).stop 2023-10-18T09:00:09.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.666+0000 7f3011202640 1 -- 172.21.15.40:0/3441404407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff4004ab0 msgr2=0x7f300c1411d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:09.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.666+0000 7f3011202640 1 --2- 172.21.15.40:0/3441404407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff4004ab0 0x7f300c1411d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009b20 tx=0x7f2ffc033c30 comp rx=0 tx=0).stop 2023-10-18T09:00:09.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.666+0000 7f3011202640 1 -- 172.21.15.40:0/3441404407 shutdown_connections 2023-10-18T09:00:09.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.666+0000 7f3011202640 1 --2- 172.21.15.40:0/3441404407 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ff806ae40 0x7f2ff806d300 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:09.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.666+0000 7f3011202640 1 --2- 172.21.15.40:0/3441404407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f300c0ae700 0x7f300c0aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:09.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.666+0000 7f3011202640 1 --2- 172.21.15.40:0/3441404407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f300c0a6470 0x7f300c0a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:09.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.666+0000 7f3011202640 1 --2- 172.21.15.40:0/3441404407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ff4004ab0 0x7f300c1411d0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:09.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.666+0000 7f3011202640 1 -- 172.21.15.40:0/3441404407 >> 172.21.15.40:0/3441404407 conn(0x7f300c069720 msgr2=0x7f300c154660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:09.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.666+0000 7f3011202640 1 -- 172.21.15.40:0/3441404407 shutdown_connections 2023-10-18T09:00:09.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.667+0000 7f3011202640 1 -- 172.21.15.40:0/3441404407 wait complete. 2023-10-18T09:00:09.689 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T09:00:09.690 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T09:00:09.690 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T09:00:09.690 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:00:09.691 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T09:00:09.691 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T09:00:09.691 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T09:00:09.691 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:00:09.692 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T09:00:09.692 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T09:00:09.692 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T09:00:09.693 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:00:09.693 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T09:00:09.693 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T09:00:09.693 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T09:00:09.694 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:00:09.708 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T09:00:09.708 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T09:00:09.709 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T09:00:09.710 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T09:00:09.711 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T09:00:09.712 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T09:00:09.713 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T09:00:09.713 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T09:00:09.757 INFO:tasks.ceph.mds.a:Started 2023-10-18T09:00:09.757 DEBUG:teuthology.parallel:result is None 2023-10-18T09:00:09.761 INFO:tasks.ceph.mds.d:Started 2023-10-18T09:00:09.762 DEBUG:teuthology.parallel:result is None 2023-10-18T09:00:09.764 INFO:tasks.ceph.mds.c:Started 2023-10-18T09:00:09.764 DEBUG:teuthology.parallel:result is None 2023-10-18T09:00:09.769 INFO:tasks.ceph.mds.b:Started 2023-10-18T09:00:09.769 DEBUG:teuthology.parallel:result is None 2023-10-18T09:00:09.770 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T09:00:09.921 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-10-18T09:00:09.924 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T09:00:09.928 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-18T09:00:09.930 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-18T09:00:09.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.969+0000 7f8ef5783640 1 -- 172.21.15.40:0/4026878161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef00b0bc0 msgr2=0x7f8ef00b1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:09.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.969+0000 7f8ef5783640 1 --2- 172.21.15.40:0/4026878161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef00b0bc0 0x7f8ef00b1040 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009830 tx=0x7f8ed802bd50 comp rx=0 tx=0).stop 2023-10-18T09:00:09.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.969+0000 7f8ef5783640 1 -- 172.21.15.40:0/4026878161 shutdown_connections 2023-10-18T09:00:09.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.969+0000 7f8ef5783640 1 --2- 172.21.15.40:0/4026878161 >> v2:172.21.15.144:3300/0 conn(0x7f8ef00cc990 0x7f8ef00ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.969+0000 7f8ef5783640 1 --2- 172.21.15.40:0/4026878161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef00b0bc0 0x7f8ef00b1040 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.969+0000 7f8ef5783640 1 -- 172.21.15.40:0/4026878161 >> 172.21.15.40:0/4026878161 conn(0x7f8ef006cee0 msgr2=0x7f8ef0069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.970+0000 7f8ef5783640 1 -- 172.21.15.40:0/4026878161 shutdown_connections 2023-10-18T09:00:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.970+0000 7f8ef5783640 1 -- 172.21.15.40:0/4026878161 wait complete. 2023-10-18T09:00:09.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.971+0000 7f8ef5783640 1 Processor -- start 2023-10-18T09:00:09.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.972+0000 7f8ef5783640 1 -- start start 2023-10-18T09:00:09.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.972+0000 7f8ef5783640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef00b0bc0 0x7f8ef00abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:09.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.972+0000 7f8ef5783640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ef00cc990 0x7f8ef013e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:09.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.973+0000 7f8ef5783640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ef0151bd0 0x7f8ef01566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:09.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.973+0000 7f8ef5783640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef00ac3c0 con 0x7f8ef0151bd0 2023-10-18T09:00:09.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.973+0000 7f8ef5783640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef00a9da0 con 0x7f8ef00cc990 2023-10-18T09:00:09.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.973+0000 7f8ef5783640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef00aa040 con 0x7f8ef00b0bc0 2023-10-18T09:00:09.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.973+0000 7f8eef7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ef0151bd0 0x7f8ef01566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:09.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.973+0000 7f8eef7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ef0151bd0 0x7f8ef01566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44260/0 (socket says 172.21.15.40:44260) 2023-10-18T09:00:09.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.973+0000 7f8eef7fe640 1 -- 172.21.15.40:0/2036819643 learned_addr learned my addr 172.21.15.40:0/2036819643 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:09.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.973+0000 7f8ee7fff640 1 --2- 172.21.15.40:0/2036819643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ef00cc990 0x7f8ef013e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:09.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.973+0000 7f8eeeffd640 1 --2- 172.21.15.40:0/2036819643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef00b0bc0 0x7f8ef00abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:09.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.973+0000 7f8eef7fe640 1 -- 172.21.15.40:0/2036819643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef00b0bc0 msgr2=0x7f8ef00abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:09.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.973+0000 7f8eef7fe640 1 --2- 172.21.15.40:0/2036819643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef00b0bc0 0x7f8ef00abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:09.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.973+0000 7f8eef7fe640 1 -- 172.21.15.40:0/2036819643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ef00cc990 msgr2=0x7f8ef013e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:09.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.973+0000 7f8eef7fe640 1 --2- 172.21.15.40:0/2036819643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ef00cc990 0x7f8ef013e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:09.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.973+0000 7f8eef7fe640 1 -- 172.21.15.40:0/2036819643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee8009840 con 0x7f8ef0151bd0 2023-10-18T09:00:09.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.973+0000 7f8ee7fff640 1 --2- 172.21.15.40:0/2036819643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ef00cc990 0x7f8ef013e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:00:09.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.974+0000 7f8eef7fe640 1 --2- 172.21.15.40:0/2036819643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ef0151bd0 0x7f8ef01566f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800efd0 tx=0x7f8ee800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:09.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.974+0000 7f8ee77fe640 1 -- 172.21.15.40:0/2036819643 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8009b40 con 0x7f8ef0151bd0 2023-10-18T09:00:09.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.974+0000 7f8ee77fe640 1 -- 172.21.15.40:0/2036819643 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8ee8003c60 con 0x7f8ef0151bd0 2023-10-18T09:00:09.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.974+0000 7f8ee77fe640 1 -- 172.21.15.40:0/2036819643 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8004020 con 0x7f8ef0151bd0 2023-10-18T09:00:09.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.974+0000 7f8ef5783640 1 -- 172.21.15.40:0/2036819643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed8009490 con 0x7f8ef0151bd0 2023-10-18T09:00:09.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.974+0000 7f8ef5783640 1 -- 172.21.15.40:0/2036819643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef00aa600 con 0x7f8ef0151bd0 2023-10-18T09:00:09.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.975+0000 7f8ef5783640 1 -- 172.21.15.40:0/2036819643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef00aa840 con 0x7f8ef0151bd0 2023-10-18T09:00:09.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.976+0000 7f8ee77fe640 1 -- 172.21.15.40:0/2036819643 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8ee8004180 con 0x7f8ef0151bd0 2023-10-18T09:00:09.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.977+0000 7f8ee77fe640 1 --2- 172.21.15.40:0/2036819643 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ed006ae40 0x7f8ed006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:09.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.977+0000 7f8ee77fe640 1 -- 172.21.15.40:0/2036819643 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8ee8014070 con 0x7f8ef0151bd0 2023-10-18T09:00:09.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.978+0000 7f8eeeffd640 1 --2- 172.21.15.40:0/2036819643 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ed006ae40 0x7f8ed006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:09.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.979+0000 7f8eeeffd640 1 --2- 172.21.15.40:0/2036819643 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ed006ae40 0x7f8ed006d300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0001710 tx=0x7f8ee0007cc0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:09.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:09.982+0000 7f8ee77fe640 1 -- 172.21.15.40:0/2036819643 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ef00aa840 con 0x7f8ef0151bd0 2023-10-18T09:00:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.187+0000 7f8ef5783640 1 -- 172.21.15.40:0/2036819643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8ef00aa840 con 0x7f8ef0151bd0 2023-10-18T09:00:10.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.670+0000 7f8ee77fe640 1 -- 172.21.15.40:0/2036819643 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v81) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8ef00aa840 con 0x7f8ef0151bd0 2023-10-18T09:00:10.672 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2023-10-18T09:00:10.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.674+0000 7f8ef5783640 1 -- 172.21.15.40:0/2036819643 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ed006ae40 msgr2=0x7f8ed006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:10.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.674+0000 7f8ef5783640 1 --2- 172.21.15.40:0/2036819643 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ed006ae40 0x7f8ed006d300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0001710 tx=0x7f8ee0007cc0 comp rx=0 tx=0).stop 2023-10-18T09:00:10.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.674+0000 7f8ef5783640 1 -- 172.21.15.40:0/2036819643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ef0151bd0 msgr2=0x7f8ef01566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:10.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.674+0000 7f8ef5783640 1 --2- 172.21.15.40:0/2036819643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ef0151bd0 0x7f8ef01566f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800efd0 tx=0x7f8ee800c6a0 comp rx=0 tx=0).stop 2023-10-18T09:00:10.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.675+0000 7f8ef5783640 1 -- 172.21.15.40:0/2036819643 shutdown_connections 2023-10-18T09:00:10.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.675+0000 7f8ef5783640 1 --2- 172.21.15.40:0/2036819643 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ed006ae40 0x7f8ed006d300 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:10.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.675+0000 7f8ef5783640 1 --2- 172.21.15.40:0/2036819643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ef0151bd0 0x7f8ef01566f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.675+0000 7f8ef5783640 1 --2- 172.21.15.40:0/2036819643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ef00cc990 0x7f8ef013e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.675+0000 7f8ef5783640 1 --2- 172.21.15.40:0/2036819643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ef00b0bc0 0x7f8ef00abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.675+0000 7f8ef5783640 1 -- 172.21.15.40:0/2036819643 >> 172.21.15.40:0/2036819643 conn(0x7f8ef006cee0 msgr2=0x7f8ef0069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.675+0000 7f8ef5783640 1 -- 172.21.15.40:0/2036819643 shutdown_connections 2023-10-18T09:00:10.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.675+0000 7f8ef5783640 1 -- 172.21.15.40:0/2036819643 wait complete. 2023-10-18T09:00:10.701 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T09:00:10.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.935+0000 7f13872ea640 1 -- 172.21.15.40:0/4017529072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f136c004d00 msgr2=0x7f136c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:10.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.935+0000 7f13872ea640 1 --2- 172.21.15.40:0/4017529072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f136c004d00 0x7f136c005180 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1378005fd0 tx=0x7f136c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:00:10.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.935+0000 7f1386ae9640 1 -- 172.21.15.40:0/4017529072 reap_dead start 2023-10-18T09:00:10.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.936+0000 7f13872ea640 1 -- 172.21.15.40:0/4017529072 shutdown_connections 2023-10-18T09:00:10.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.936+0000 7f13872ea640 1 -- 172.21.15.40:0/4017529072 >> 172.21.15.40:0/4017529072 conn(0x7f1380069720 msgr2=0x7f1380069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:10.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.936+0000 7f13872ea640 1 -- 172.21.15.40:0/4017529072 shutdown_connections 2023-10-18T09:00:10.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.936+0000 7f13872ea640 1 -- 172.21.15.40:0/4017529072 wait complete. 2023-10-18T09:00:10.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.938+0000 7f13872ea640 1 Processor -- start 2023-10-18T09:00:10.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.938+0000 7f13872ea640 1 -- start start 2023-10-18T09:00:10.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.939+0000 7f13872ea640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f136c004d00 0x7f13800af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:10.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.939+0000 7f13872ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1380159880 0x7f1380159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:10.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.939+0000 7f13872ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1380150900 0x7f1380150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:10.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.939+0000 7f13872ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13800ad460 con 0x7f1380159880 2023-10-18T09:00:10.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.939+0000 7f13872ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138015a220 con 0x7f1380150900 2023-10-18T09:00:10.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.939+0000 7f13872ea640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13801512c0 con 0x7f136c004d00 2023-10-18T09:00:10.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.939+0000 7f1385ae7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1380159880 0x7f1380159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:10.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.939+0000 7f1385ae7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1380159880 0x7f1380159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44282/0 (socket says 172.21.15.40:44282) 2023-10-18T09:00:10.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.939+0000 7f1385ae7640 1 -- 172.21.15.40:0/3837097358 learned_addr learned my addr 172.21.15.40:0/3837097358 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:10.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.939+0000 7f13862e8640 1 --2- 172.21.15.40:0/3837097358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f136c004d00 0x7f13800af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:10.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.940+0000 7f1386ae9640 1 --2- 172.21.15.40:0/3837097358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1380150900 0x7f1380150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:10.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.940+0000 7f1385ae7640 1 -- 172.21.15.40:0/3837097358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f136c004d00 msgr2=0x7f13800af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:10.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.940+0000 7f1385ae7640 1 --2- 172.21.15.40:0/3837097358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f136c004d00 0x7f13800af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:10.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.940+0000 7f1385ae7640 1 -- 172.21.15.40:0/3837097358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1380150900 msgr2=0x7f1380150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:10.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.940+0000 7f1385ae7640 1 --2- 172.21.15.40:0/3837097358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1380150900 0x7f1380150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:10.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.940+0000 7f1385ae7640 1 -- 172.21.15.40:0/3837097358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13800acae0 con 0x7f1380159880 2023-10-18T09:00:10.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.940+0000 7f1385ae7640 1 --2- 172.21.15.40:0/3837097358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1380159880 0x7f1380159ce0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f137000d640 tx=0x7f137000db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:10.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.940+0000 7f1376ffd640 1 -- 172.21.15.40:0/3837097358 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13700062e0 con 0x7f1380159880 2023-10-18T09:00:10.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.940+0000 7f1376ffd640 1 -- 172.21.15.40:0/3837097358 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1370006530 con 0x7f1380159880 2023-10-18T09:00:10.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.940+0000 7f1376ffd640 1 -- 172.21.15.40:0/3837097358 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1370003870 con 0x7f1380159880 2023-10-18T09:00:10.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.940+0000 7f13872ea640 1 -- 172.21.15.40:0/3837097358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138014dcf0 con 0x7f1380159880 2023-10-18T09:00:10.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.940+0000 7f13872ea640 1 -- 172.21.15.40:0/3837097358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13800acda0 con 0x7f1380159880 2023-10-18T09:00:10.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.942+0000 7f13872ea640 1 -- 172.21.15.40:0/3837097358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1380066e00 con 0x7f1380159880 2023-10-18T09:00:10.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.943+0000 7f1376ffd640 1 -- 172.21.15.40:0/3837097358 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f13700039d0 con 0x7f1380159880 2023-10-18T09:00:10.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.944+0000 7f1376ffd640 1 --2- 172.21.15.40:0/3837097358 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f136406ae40 0x7f136406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:10.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.944+0000 7f1376ffd640 1 -- 172.21.15.40:0/3837097358 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1370089830 con 0x7f1380159880 2023-10-18T09:00:10.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.944+0000 7f13862e8640 1 --2- 172.21.15.40:0/3837097358 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f136406ae40 0x7f136406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:10.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.946+0000 7f13862e8640 1 --2- 172.21.15.40:0/3837097358 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f136406ae40 0x7f136406d300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f137c001050 tx=0x7f137c000fa0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:10.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:10.948+0000 7f1376ffd640 1 -- 172.21.15.40:0/3837097358 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1370054cf0 con 0x7f1380159880 2023-10-18T09:00:11.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.215+0000 7f13872ea640 1 -- 172.21.15.40:0/3837097358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f13800caf90 con 0x7f1380159880 2023-10-18T09:00:11.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.216+0000 7f1376ffd640 1 -- 172.21.15.40:0/3837097358 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1370014070 con 0x7f1380159880 2023-10-18T09:00:11.218 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:11.218 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:00:11.218 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2023-10-18T09:00:11.219 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.219 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2023-10-18T09:00:11.219 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lYsb5CxAAGButZ6+Usc62kJ0ksxZrYw==", 2023-10-18T09:00:11.219 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.219 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:00:11.220 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:00:11.220 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:00:11.220 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:00:11.220 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.220 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.221 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.221 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2023-10-18T09:00:11.221 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lAqlhMBAApAhxCYB9d9bz1GibADDRSQ==", 2023-10-18T09:00:11.221 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.221 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:00:11.222 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:00:11.222 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:00:11.222 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:00:11.222 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.222 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.223 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.223 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2023-10-18T09:00:11.223 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lPVh+HhAAzucSx+ksPDBkqJGa8+pyJw==", 2023-10-18T09:00:11.223 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.223 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:00:11.224 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:00:11.224 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:00:11.224 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:00:11.224 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.224 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.224 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.225 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2023-10-18T09:00:11.225 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lP7QCBhAAog7qDCgGVv0T7WKsRkATkA==", 2023-10-18T09:00:11.225 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.225 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:00:11.225 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:00:11.226 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:00:11.226 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:00:11.226 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.226 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.227 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.227 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2023-10-18T09:00:11.227 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lVx+hAhAA6ZpIyZKhbqy8tV84piGILA==", 2023-10-18T09:00:11.227 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.227 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:00:11.228 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:00:11.228 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:00:11.228 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.228 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.228 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.228 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2023-10-18T09:00:11.229 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lk9LwExAA7zFzQOn2hcvnck2cOZZ7AA==", 2023-10-18T09:00:11.229 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.229 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:00:11.229 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:00:11.229 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:00:11.229 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.230 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.230 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.230 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2023-10-18T09:00:11.230 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lK930JRAA1GyKRkIpXPHHGqY8owYWRQ==", 2023-10-18T09:00:11.230 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.231 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:00:11.231 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:00:11.231 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:00:11.231 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.231 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.232 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.232 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2023-10-18T09:00:11.232 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9liBWyNRAAMHBBdG5nbrGQupqvUtuKMA==", 2023-10-18T09:00:11.232 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.232 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:00:11.232 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:00:11.233 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:00:11.233 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.233 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.233 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.233 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2023-10-18T09:00:11.233 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9loDoOAhAAjHZub4wnB7dCr86rxAtI5g==", 2023-10-18T09:00:11.234 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.234 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:00:11.234 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:00:11.234 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:00:11.234 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.235 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.235 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.235 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2023-10-18T09:00:11.235 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lhqNfERAAw410g3DwSXrK23YkxVBFVw==", 2023-10-18T09:00:11.235 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.236 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:00:11.236 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:00:11.236 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:00:11.236 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.236 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.237 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.237 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2023-10-18T09:00:11.237 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lRRScIRAAg0pmfxCQBCPYodj1Lko+Lw==", 2023-10-18T09:00:11.237 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.237 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:00:11.238 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:00:11.238 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:00:11.238 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.238 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.238 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.239 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2023-10-18T09:00:11.239 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9ld1dXMhAAyovlrKOhVuynEiuPTgFcjQ==", 2023-10-18T09:00:11.239 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.239 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:00:11.239 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:00:11.239 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:00:11.240 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.240 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.240 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.240 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2023-10-18T09:00:11.240 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w==", 2023-10-18T09:00:11.241 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.241 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:00:11.241 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2023-10-18T09:00:11.241 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T09:00:11.241 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.242 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.242 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.242 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2023-10-18T09:00:11.242 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lAB3oKhAAD43WFdC23WGF6WUi4JKsvg==", 2023-10-18T09:00:11.242 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.243 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:00:11.243 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:00:11.243 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:00:11.243 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:00:11.243 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.243 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.244 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.244 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2023-10-18T09:00:11.244 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lu57fHxAAdzsVU2KKsWRkaARRS8TGjw==", 2023-10-18T09:00:11.244 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.245 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:00:11.245 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:00:11.245 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:00:11.245 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:00:11.245 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.246 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.246 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.246 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2023-10-18T09:00:11.246 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lygWsNRAA1OziXxk7hBtTw2Aq8OlSKQ==", 2023-10-18T09:00:11.246 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.246 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:00:11.247 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:00:11.247 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:00:11.247 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:00:11.247 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.247 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.248 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.248 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2023-10-18T09:00:11.248 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBMnS9lnrUjNhAAHP2eezRHF4/Jgox7iPwy7g==", 2023-10-18T09:00:11.248 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.248 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:00:11.249 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:00:11.249 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:00:11.249 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:00:11.249 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.249 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.249 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.250 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2023-10-18T09:00:11.250 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9lCf/xOhAAJHVoKIswWUcVGKQpiaJ12A==", 2023-10-18T09:00:11.250 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.250 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:00:11.250 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T09:00:11.251 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:00:11.251 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.251 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:00:11.251 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:11.251 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2023-10-18T09:00:11.252 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9loxRXLhAAQK1n1/4PB35umHxOKQzz7Q==", 2023-10-18T09:00:11.252 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:00:11.252 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:00:11.252 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T09:00:11.252 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:00:11.253 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.253 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:11.253 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:00:11.253 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:00:11.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.218+0000 7f13872ea640 1 -- 172.21.15.40:0/3837097358 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f136406ae40 msgr2=0x7f136406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:11.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.218+0000 7f13872ea640 1 --2- 172.21.15.40:0/3837097358 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f136406ae40 0x7f136406d300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f137c001050 tx=0x7f137c000fa0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.218+0000 7f13872ea640 1 -- 172.21.15.40:0/3837097358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1380159880 msgr2=0x7f1380159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.218+0000 7f13872ea640 1 --2- 172.21.15.40:0/3837097358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1380159880 0x7f1380159ce0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f137000d640 tx=0x7f137000db00 comp rx=0 tx=0).stop 2023-10-18T09:00:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.218+0000 7f13872ea640 1 -- 172.21.15.40:0/3837097358 shutdown_connections 2023-10-18T09:00:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.218+0000 7f13872ea640 1 --2- 172.21.15.40:0/3837097358 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f136406ae40 0x7f136406d300 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.218+0000 7f13872ea640 1 --2- 172.21.15.40:0/3837097358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1380150900 0x7f1380150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.218+0000 7f13872ea640 1 --2- 172.21.15.40:0/3837097358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1380159880 0x7f1380159ce0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.218+0000 7f13872ea640 1 --2- 172.21.15.40:0/3837097358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f136c004d00 0x7f13800af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.218+0000 7f13872ea640 1 -- 172.21.15.40:0/3837097358 >> 172.21.15.40:0/3837097358 conn(0x7f1380069720 msgr2=0x7f13801555f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:11.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.218+0000 7f13872ea640 1 -- 172.21.15.40:0/3837097358 shutdown_connections 2023-10-18T09:00:11.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.219+0000 7f13872ea640 1 -- 172.21.15.40:0/3837097358 wait complete. 2023-10-18T09:00:11.258 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:00:11.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.495+0000 7f38c1296640 1 -- 172.21.15.40:0/3632469517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc0cc6d0 msgr2=0x7f38bc0ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:11.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.495+0000 7f38c1296640 1 --2- 172.21.15.40:0/3632469517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc0cc6d0 0x7f38bc0ccab0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f38bc0c9b70 tx=0x7f38ac02bda0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.496+0000 7f38c1296640 1 -- 172.21.15.40:0/3632469517 shutdown_connections 2023-10-18T09:00:11.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.496+0000 7f38c1296640 1 --2- 172.21.15.40:0/3632469517 >> v2:172.21.15.144:3300/0 conn(0x7f38bc06b710 0x7f38bc06bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.496+0000 7f38c1296640 1 --2- 172.21.15.40:0/3632469517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc0cc6d0 0x7f38bc0ccab0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.496+0000 7f38c1296640 1 -- 172.21.15.40:0/3632469517 >> 172.21.15.40:0/3632469517 conn(0x7f38bc06cee0 msgr2=0x7f38bc069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:11.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.496+0000 7f38c1296640 1 -- 172.21.15.40:0/3632469517 shutdown_connections 2023-10-18T09:00:11.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.496+0000 7f38c1296640 1 -- 172.21.15.40:0/3632469517 wait complete. 2023-10-18T09:00:11.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.497+0000 7f38c1296640 1 Processor -- start 2023-10-18T09:00:11.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.498+0000 7f38c1296640 1 -- start start 2023-10-18T09:00:11.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.498+0000 7f38c1296640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc06b710 0x7f38bc150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:11.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.498+0000 7f38c1296640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38bc0b06a0 0x7f38bc1f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:11.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.499+0000 7f38c1296640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38bc0cc6d0 0x7f38bc149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:11.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.499+0000 7f38c1296640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc0c2050 con 0x7f38bc06b710 2023-10-18T09:00:11.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.499+0000 7f38c1296640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc1f5990 con 0x7f38bc0b06a0 2023-10-18T09:00:11.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.499+0000 7f38c1296640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc1f5c30 con 0x7f38bc0cc6d0 2023-10-18T09:00:11.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.499+0000 7f38baffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc06b710 0x7f38bc150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:11.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.499+0000 7f38baffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc06b710 0x7f38bc150bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44304/0 (socket says 172.21.15.40:44304) 2023-10-18T09:00:11.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.499+0000 7f38baffd640 1 -- 172.21.15.40:0/1431805898 learned_addr learned my addr 172.21.15.40:0/1431805898 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:11.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.499+0000 7f38b3fff640 1 --2- 172.21.15.40:0/1431805898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38bc0b06a0 0x7f38bc1f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:11.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.499+0000 7f38bb7fe640 1 --2- 172.21.15.40:0/1431805898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38bc0cc6d0 0x7f38bc149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:11.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.499+0000 7f38baffd640 1 -- 172.21.15.40:0/1431805898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38bc0cc6d0 msgr2=0x7f38bc149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:11.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.499+0000 7f38baffd640 1 --2- 172.21.15.40:0/1431805898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38bc0cc6d0 0x7f38bc149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.499+0000 7f38baffd640 1 -- 172.21.15.40:0/1431805898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38bc0b06a0 msgr2=0x7f38bc1f5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:11.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.499+0000 7f38baffd640 1 --2- 172.21.15.40:0/1431805898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38bc0b06a0 0x7f38bc1f5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.499+0000 7f38baffd640 1 -- 172.21.15.40:0/1431805898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ac009560 con 0x7f38bc06b710 2023-10-18T09:00:11.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.499+0000 7f38b3fff640 1 --2- 172.21.15.40:0/1431805898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38bc0b06a0 0x7f38bc1f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:11.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.499+0000 7f38bb7fe640 1 --2- 172.21.15.40:0/1431805898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38bc0cc6d0 0x7f38bc149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:11.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.500+0000 7f38baffd640 1 --2- 172.21.15.40:0/1431805898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc06b710 0x7f38bc150bf0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f38ac036040 tx=0x7f38ac02db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:11.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.500+0000 7f38b37fe640 1 -- 172.21.15.40:0/1431805898 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac003800 con 0x7f38bc06b710 2023-10-18T09:00:11.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.500+0000 7f38b37fe640 1 -- 172.21.15.40:0/1431805898 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f38ac003a50 con 0x7f38bc06b710 2023-10-18T09:00:11.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.500+0000 7f38b37fe640 1 -- 172.21.15.40:0/1431805898 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac003de0 con 0x7f38bc06b710 2023-10-18T09:00:11.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.500+0000 7f38c1296640 1 -- 172.21.15.40:0/1431805898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38bc1f5f10 con 0x7f38bc06b710 2023-10-18T09:00:11.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.500+0000 7f38c1296640 1 -- 172.21.15.40:0/1431805898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38bc0c22a0 con 0x7f38bc06b710 2023-10-18T09:00:11.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.501+0000 7f38c1296640 1 -- 172.21.15.40:0/1431805898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38bc0cd670 con 0x7f38bc06b710 2023-10-18T09:00:11.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.502+0000 7f38b37fe640 1 -- 172.21.15.40:0/1431805898 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f38ac03e450 con 0x7f38bc06b710 2023-10-18T09:00:11.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.504+0000 7f38b37fe640 1 --2- 172.21.15.40:0/1431805898 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f389c06ae40 0x7f389c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:11.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.504+0000 7f38b37fe640 1 -- 172.21.15.40:0/1431805898 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f38ac0abe30 con 0x7f38bc06b710 2023-10-18T09:00:11.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.504+0000 7f38b3fff640 1 --2- 172.21.15.40:0/1431805898 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f389c06ae40 0x7f389c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:11.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.506+0000 7f38b3fff640 1 --2- 172.21.15.40:0/1431805898 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f389c06ae40 0x7f389c06d300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f38bc1f53b0 tx=0x7f38a4005db0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:11.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.509+0000 7f38b37fe640 1 -- 172.21.15.40:0/1431805898 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38ac077220 con 0x7f38bc06b710 2023-10-18T09:00:11.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.714+0000 7f38c1296640 1 -- 172.21.15.40:0/1431805898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38bc14b9b0 con 0x7f38bc06b710 2023-10-18T09:00:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.716+0000 7f38b37fe640 1 -- 172.21.15.40:0/1431805898 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+12869 (secure 0 0 0) 0x7f38ac042020 con 0x7f38bc06b710 2023-10-18T09:00:11.719 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:11.719 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":83,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:00:10.683428+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:00:11.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.719+0000 7f38c1296640 1 -- 172.21.15.40:0/1431805898 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f389c06ae40 msgr2=0x7f389c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:11.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.719+0000 7f38c1296640 1 --2- 172.21.15.40:0/1431805898 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f389c06ae40 0x7f389c06d300 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f38bc1f53b0 tx=0x7f38a4005db0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.719+0000 7f38c1296640 1 -- 172.21.15.40:0/1431805898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc06b710 msgr2=0x7f38bc150bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:11.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.719+0000 7f38c1296640 1 --2- 172.21.15.40:0/1431805898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc06b710 0x7f38bc150bf0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f38ac036040 tx=0x7f38ac02db20 comp rx=0 tx=0).stop 2023-10-18T09:00:11.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.720+0000 7f38c1296640 1 -- 172.21.15.40:0/1431805898 shutdown_connections 2023-10-18T09:00:11.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.720+0000 7f38c1296640 1 --2- 172.21.15.40:0/1431805898 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f389c06ae40 0x7f389c06d300 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.720+0000 7f38c1296640 1 --2- 172.21.15.40:0/1431805898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38bc0cc6d0 0x7f38bc149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.720+0000 7f38c1296640 1 --2- 172.21.15.40:0/1431805898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38bc0b06a0 0x7f38bc1f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.720+0000 7f38c1296640 1 --2- 172.21.15.40:0/1431805898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc06b710 0x7f38bc150bf0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.720+0000 7f38c1296640 1 -- 172.21.15.40:0/1431805898 >> 172.21.15.40:0/1431805898 conn(0x7f38bc06cee0 msgr2=0x7f38bc0b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:11.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.720+0000 7f38c1296640 1 -- 172.21.15.40:0/1431805898 shutdown_connections 2023-10-18T09:00:11.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.720+0000 7f38c1296640 1 -- 172.21.15.40:0/1431805898 wait complete. 2023-10-18T09:00:11.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:00:11.743 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:00:11.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.977+0000 7fefed30c640 1 -- 172.21.15.40:0/4218571599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd00039a0 msgr2=0x7fefd0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:11.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.977+0000 7fefed30c640 1 --2- 172.21.15.40:0/4218571599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd00039a0 0x7fefd0003da0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009e50 tx=0x7fefd00057f0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.977+0000 7fefecb0b640 1 -- 172.21.15.40:0/4218571599 reap_dead start 2023-10-18T09:00:11.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.978+0000 7fefed30c640 1 -- 172.21.15.40:0/4218571599 shutdown_connections 2023-10-18T09:00:11.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.978+0000 7fefed30c640 1 -- 172.21.15.40:0/4218571599 >> 172.21.15.40:0/4218571599 conn(0x7fefe80cdef0 msgr2=0x7fefe80ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:11.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.978+0000 7fefed30c640 1 -- 172.21.15.40:0/4218571599 shutdown_connections 2023-10-18T09:00:11.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.978+0000 7fefed30c640 1 -- 172.21.15.40:0/4218571599 wait complete. 2023-10-18T09:00:11.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.979+0000 7fefed30c640 1 Processor -- start 2023-10-18T09:00:11.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.980+0000 7fefed30c640 1 -- start start 2023-10-18T09:00:11.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.981+0000 7fefed30c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd00039a0 0x7fefe8148900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:11.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.981+0000 7fefed30c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefe80b2060 0x7fefe806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:11.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.981+0000 7fefed30c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe80b24c0 0x7fefe806d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:11.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.981+0000 7fefed30c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80b35a0 con 0x7fefd00039a0 2023-10-18T09:00:11.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.981+0000 7fefed30c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe8099bc0 con 0x7fefe80b2060 2023-10-18T09:00:11.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.981+0000 7fefed30c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80b2a20 con 0x7fefe80b24c0 2023-10-18T09:00:11.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.981+0000 7fefe7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd00039a0 0x7fefe8148900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:11.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.981+0000 7fefe7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd00039a0 0x7fefe8148900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44310/0 (socket says 172.21.15.40:44310) 2023-10-18T09:00:11.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.981+0000 7fefe7fff640 1 -- 172.21.15.40:0/2644541251 learned_addr learned my addr 172.21.15.40:0/2644541251 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:11.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.981+0000 7fefe77fe640 1 --2- 172.21.15.40:0/2644541251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefe80b2060 0x7fefe806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:11.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.981+0000 7fefe7fff640 1 -- 172.21.15.40:0/2644541251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe80b24c0 msgr2=0x7fefe806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:11.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.981+0000 7fefe7fff640 1 --2- 172.21.15.40:0/2644541251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe80b24c0 0x7fefe806d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.981+0000 7fefe7fff640 1 -- 172.21.15.40:0/2644541251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefe80b2060 msgr2=0x7fefe806d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:11.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.981+0000 7fefe7fff640 1 --2- 172.21.15.40:0/2644541251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefe80b2060 0x7fefe806d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:11.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.981+0000 7fefe7fff640 1 -- 172.21.15.40:0/2644541251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe80b2d00 con 0x7fefd00039a0 2023-10-18T09:00:11.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.982+0000 7fefe7fff640 1 --2- 172.21.15.40:0/2644541251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd00039a0 0x7fefe8148900 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009d70 tx=0x7fefd8033c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:11.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.982+0000 7fefe4ff9640 1 -- 172.21.15.40:0/2644541251 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd803d820 con 0x7fefd00039a0 2023-10-18T09:00:11.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.982+0000 7fefe4ff9640 1 -- 172.21.15.40:0/2644541251 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fefd803d980 con 0x7fefd00039a0 2023-10-18T09:00:11.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.982+0000 7fefe4ff9640 1 -- 172.21.15.40:0/2644541251 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd803dd50 con 0x7fefd00039a0 2023-10-18T09:00:11.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.982+0000 7fefed30c640 1 -- 172.21.15.40:0/2644541251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe80b2fe0 con 0x7fefd00039a0 2023-10-18T09:00:11.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.982+0000 7fefed30c640 1 -- 172.21.15.40:0/2644541251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe8148e40 con 0x7fefd00039a0 2023-10-18T09:00:11.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.983+0000 7fefed30c640 1 -- 172.21.15.40:0/2644541251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe80b3500 con 0x7fefd00039a0 2023-10-18T09:00:11.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.984+0000 7fefe4ff9640 1 -- 172.21.15.40:0/2644541251 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fefd8045400 con 0x7fefd00039a0 2023-10-18T09:00:11.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.985+0000 7fefe4ff9640 1 --2- 172.21.15.40:0/2644541251 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fefd406ae40 0x7fefd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:11.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.986+0000 7fefe4ff9640 1 -- 172.21.15.40:0/2644541251 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fefd80ac020 con 0x7fefd00039a0 2023-10-18T09:00:11.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.986+0000 7fefe77fe640 1 --2- 172.21.15.40:0/2644541251 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fefd406ae40 0x7fefd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:11.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.987+0000 7fefe77fe640 1 --2- 172.21.15.40:0/2644541251 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fefd406ae40 0x7fefd406d300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0017a0 tx=0x7fefdc0014b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:11.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:11.990+0000 7fefe4ff9640 1 -- 172.21.15.40:0/2644541251 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fefd80772f0 con 0x7fefd00039a0 2023-10-18T09:00:12.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.187+0000 7fefed30c640 1 -- 172.21.15.40:0/2644541251 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fefe80ac0e0 con 0x7fefd00039a0 2023-10-18T09:00:12.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.188+0000 7fefe4ff9640 1 -- 172.21.15.40:0/2644541251 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fefd8033e60 con 0x7fefd00039a0 2023-10-18T09:00:12.190 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:00:12.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.190+0000 7fefed30c640 1 -- 172.21.15.40:0/2644541251 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fefd406ae40 msgr2=0x7fefd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:12.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.190+0000 7fefed30c640 1 --2- 172.21.15.40:0/2644541251 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fefd406ae40 0x7fefd406d300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fefdc0017a0 tx=0x7fefdc0014b0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.190+0000 7fefed30c640 1 -- 172.21.15.40:0/2644541251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd00039a0 msgr2=0x7fefe8148900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:12.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.190+0000 7fefed30c640 1 --2- 172.21.15.40:0/2644541251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd00039a0 0x7fefe8148900 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009d70 tx=0x7fefd8033c70 comp rx=0 tx=0).stop 2023-10-18T09:00:12.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.190+0000 7fefed30c640 1 -- 172.21.15.40:0/2644541251 shutdown_connections 2023-10-18T09:00:12.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.190+0000 7fefed30c640 1 --2- 172.21.15.40:0/2644541251 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fefd406ae40 0x7fefd406d300 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.190+0000 7fefed30c640 1 --2- 172.21.15.40:0/2644541251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fefe80b24c0 0x7fefe806d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.190+0000 7fefed30c640 1 --2- 172.21.15.40:0/2644541251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefe80b2060 0x7fefe806d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.191+0000 7fefed30c640 1 --2- 172.21.15.40:0/2644541251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefd00039a0 0x7fefe8148900 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.191+0000 7fefed30c640 1 -- 172.21.15.40:0/2644541251 >> 172.21.15.40:0/2644541251 conn(0x7fefe80cdef0 msgr2=0x7fefe80cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:12.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.191+0000 7fefed30c640 1 -- 172.21.15.40:0/2644541251 shutdown_connections 2023-10-18T09:00:12.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.191+0000 7fefed30c640 1 -- 172.21.15.40:0/2644541251 wait complete. 2023-10-18T09:00:12.213 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T09:00:12.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.442+0000 7f20ef59e640 1 -- 172.21.15.40:0/2312078309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f00c9f00 msgr2=0x7f20f0157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:12.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.442+0000 7f20ef59e640 1 --2- 172.21.15.40:0/2312078309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f00c9f00 0x7f20f0157d40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f20f006af60 tx=0x7f20e802c260 comp rx=0 tx=0).stop 2023-10-18T09:00:12.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.442+0000 7f20ef59e640 1 -- 172.21.15.40:0/2312078309 shutdown_connections 2023-10-18T09:00:12.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.442+0000 7f20ef59e640 1 --2- 172.21.15.40:0/2312078309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f00c9f00 0x7f20f0157d40 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.442+0000 7f20ef59e640 1 --2- 172.21.15.40:0/2312078309 >> v2:172.21.15.144:3300/0 conn(0x7f20f00b0630 0x7f20f00c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.442+0000 7f20ef59e640 1 --2- 172.21.15.40:0/2312078309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f006d6f0 0x7f20f00b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.442+0000 7f20ef59e640 1 -- 172.21.15.40:0/2312078309 >> 172.21.15.40:0/2312078309 conn(0x7f20f0069720 msgr2=0x7f20f0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:12.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.442+0000 7f20ef59e640 1 -- 172.21.15.40:0/2312078309 shutdown_connections 2023-10-18T09:00:12.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.442+0000 7f20ef59e640 1 -- 172.21.15.40:0/2312078309 wait complete. 2023-10-18T09:00:12.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.444+0000 7f20ef59e640 1 Processor -- start 2023-10-18T09:00:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.444+0000 7f20ef59e640 1 -- start start 2023-10-18T09:00:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.445+0000 7f20ef59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f006d6f0 0x7f20f0145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.445+0000 7f20ef59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f00b0630 0x7f20f01569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.445+0000 7f20ef59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f00c9f00 0x7f20f0142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:12.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.445+0000 7f20ef59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f0156f00 con 0x7f20f00c9f00 2023-10-18T09:00:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.445+0000 7f20ef59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f015ac70 con 0x7f20f006d6f0 2023-10-18T09:00:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.445+0000 7f20ef59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f014a550 con 0x7f20f00b0630 2023-10-18T09:00:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.446+0000 7f20eed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f00c9f00 0x7f20f0142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.446+0000 7f20eed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f00c9f00 0x7f20f0142de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44326/0 (socket says 172.21.15.40:44326) 2023-10-18T09:00:12.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.446+0000 7f20eed9d640 1 -- 172.21.15.40:0/3054742599 learned_addr learned my addr 172.21.15.40:0/3054742599 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:12.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.446+0000 7f20edd9b640 1 --2- 172.21.15.40:0/3054742599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f00b0630 0x7f20f01569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:12.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.446+0000 7f20eed9d640 1 -- 172.21.15.40:0/3054742599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f00b0630 msgr2=0x7f20f01569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:12.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.446+0000 7f20ee59c640 1 --2- 172.21.15.40:0/3054742599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f006d6f0 0x7f20f0145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:12.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.446+0000 7f20eed9d640 1 --2- 172.21.15.40:0/3054742599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f00b0630 0x7f20f01569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.446+0000 7f20eed9d640 1 -- 172.21.15.40:0/3054742599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f006d6f0 msgr2=0x7f20f0145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.446+0000 7f20eed9d640 1 --2- 172.21.15.40:0/3054742599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f006d6f0 0x7f20f0145ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.446+0000 7f20eed9d640 1 -- 172.21.15.40:0/3054742599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e8009900 con 0x7f20f00c9f00 2023-10-18T09:00:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.446+0000 7f20ee59c640 1 --2- 172.21.15.40:0/3054742599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f006d6f0 0x7f20f0145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.446+0000 7f20edd9b640 1 --2- 172.21.15.40:0/3054742599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f00b0630 0x7f20f01569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.446+0000 7f20eed9d640 1 --2- 172.21.15.40:0/3054742599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f00c9f00 0x7f20f0142de0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f20e8037040 tx=0x7f20e8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.447+0000 7f20deffd640 1 -- 172.21.15.40:0/3054742599 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8004250 con 0x7f20f00c9f00 2023-10-18T09:00:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.447+0000 7f20deffd640 1 -- 172.21.15.40:0/3054742599 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f20e80044a0 con 0x7f20f00c9f00 2023-10-18T09:00:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.447+0000 7f20deffd640 1 -- 172.21.15.40:0/3054742599 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8004890 con 0x7f20f00c9f00 2023-10-18T09:00:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.447+0000 7f20ef59e640 1 -- 172.21.15.40:0/3054742599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f014a830 con 0x7f20f00c9f00 2023-10-18T09:00:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.447+0000 7f20ef59e640 1 -- 172.21.15.40:0/3054742599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f01564a0 con 0x7f20f00c9f00 2023-10-18T09:00:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.448+0000 7f20ef59e640 1 -- 172.21.15.40:0/3054742599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f0155ff0 con 0x7f20f00c9f00 2023-10-18T09:00:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.449+0000 7f20deffd640 1 -- 172.21.15.40:0/3054742599 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f20e803b040 con 0x7f20f00c9f00 2023-10-18T09:00:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.450+0000 7f20deffd640 1 --2- 172.21.15.40:0/3054742599 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f20d006ae40 0x7f20d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.450+0000 7f20deffd640 1 -- 172.21.15.40:0/3054742599 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f20e80ac200 con 0x7f20f00c9f00 2023-10-18T09:00:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.451+0000 7f20ee59c640 1 --2- 172.21.15.40:0/3054742599 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f20d006ae40 0x7f20d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.452+0000 7f20ee59c640 1 --2- 172.21.15.40:0/3054742599 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f20d006ae40 0x7f20d006d300 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f20e00058c0 tx=0x7f20e0005d50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:12.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.454+0000 7f20deffd640 1 -- 172.21.15.40:0/3054742599 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f20e8077560 con 0x7f20f00c9f00 2023-10-18T09:00:12.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.648+0000 7f20ef59e640 1 -- 172.21.15.40:0/3054742599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f20f0156690 con 0x7f20f00c9f00 2023-10-18T09:00:12.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.648+0000 7f20deffd640 1 -- 172.21.15.40:0/3054742599 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) v1 ==== 84+0+65 (secure 0 0 0) 0x7f20e807ab80 con 0x7f20f00c9f00 2023-10-18T09:00:12.651 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:12.651 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-10-18T09:00:12.651 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:00:12.652 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-10-18T09:00:12.652 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-10-18T09:00:12.652 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:00:12.652 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-10-18T09:00:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.651+0000 7f20ef59e640 1 -- 172.21.15.40:0/3054742599 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f20d006ae40 msgr2=0x7f20d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.651+0000 7f20ef59e640 1 --2- 172.21.15.40:0/3054742599 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f20d006ae40 0x7f20d006d300 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f20e00058c0 tx=0x7f20e0005d50 comp rx=0 tx=0).stop 2023-10-18T09:00:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.651+0000 7f20ef59e640 1 -- 172.21.15.40:0/3054742599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f00c9f00 msgr2=0x7f20f0142de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:12.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.651+0000 7f20ef59e640 1 --2- 172.21.15.40:0/3054742599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f00c9f00 0x7f20f0142de0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f20e8037040 tx=0x7f20e8003c10 comp rx=0 tx=0).stop 2023-10-18T09:00:12.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.651+0000 7f20ef59e640 1 -- 172.21.15.40:0/3054742599 shutdown_connections 2023-10-18T09:00:12.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.651+0000 7f20ef59e640 1 --2- 172.21.15.40:0/3054742599 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f20d006ae40 0x7f20d006d300 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.651+0000 7f20ef59e640 1 --2- 172.21.15.40:0/3054742599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20f00c9f00 0x7f20f0142de0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.651+0000 7f20ef59e640 1 --2- 172.21.15.40:0/3054742599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20f00b0630 0x7f20f01569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.651+0000 7f20ef59e640 1 --2- 172.21.15.40:0/3054742599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20f006d6f0 0x7f20f0145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.651+0000 7f20ef59e640 1 -- 172.21.15.40:0/3054742599 >> 172.21.15.40:0/3054742599 conn(0x7f20f0069720 msgr2=0x7f20f01542b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.651+0000 7f20ef59e640 1 -- 172.21.15.40:0/3054742599 shutdown_connections 2023-10-18T09:00:12.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.651+0000 7f20ef59e640 1 -- 172.21.15.40:0/3054742599 wait complete. 2023-10-18T09:00:12.673 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T09:00:12.674 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T09:00:12.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.911+0000 7feb8359e640 1 -- 172.21.15.40:0/1819662781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb841585e0 msgr2=0x7feb84158a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:12.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.911+0000 7feb8359e640 1 --2- 172.21.15.40:0/1819662781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb841585e0 0x7feb84158a30 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7feb8414f6d0 tx=0x7feb7c02c2f0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.912+0000 7feb8359e640 1 -- 172.21.15.40:0/1819662781 shutdown_connections 2023-10-18T09:00:12.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.912+0000 7feb8359e640 1 --2- 172.21.15.40:0/1819662781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb841585e0 0x7feb84158a30 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.912+0000 7feb8359e640 1 --2- 172.21.15.40:0/1819662781 >> v2:172.21.15.144:3300/0 conn(0x7feb84070e20 0x7feb840712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.912+0000 7feb8359e640 1 -- 172.21.15.40:0/1819662781 >> 172.21.15.40:0/1819662781 conn(0x7feb8406b7d0 msgr2=0x7feb8406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:12.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.912+0000 7feb8359e640 1 -- 172.21.15.40:0/1819662781 shutdown_connections 2023-10-18T09:00:12.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.912+0000 7feb8359e640 1 -- 172.21.15.40:0/1819662781 wait complete. 2023-10-18T09:00:12.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.913+0000 7feb8359e640 1 Processor -- start 2023-10-18T09:00:12.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.914+0000 7feb8359e640 1 -- start start 2023-10-18T09:00:12.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.914+0000 7feb8359e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb84070e20 0x7feb8413e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:12.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.914+0000 7feb8359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb8413af60 0x7feb8414ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:12.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.915+0000 7feb8359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb841585e0 0x7feb84136ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:12.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.915+0000 7feb8359e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb84150aa0 con 0x7feb8413af60 2023-10-18T09:00:12.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.915+0000 7feb8359e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb840b1530 con 0x7feb841585e0 2023-10-18T09:00:12.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.915+0000 7feb8359e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8413ecd0 con 0x7feb84070e20 2023-10-18T09:00:12.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.915+0000 7feb7bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb8413af60 0x7feb8414ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:12.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.915+0000 7feb7bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb8413af60 0x7feb8414ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44336/0 (socket says 172.21.15.40:44336) 2023-10-18T09:00:12.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.915+0000 7feb7bfff640 1 -- 172.21.15.40:0/741613456 learned_addr learned my addr 172.21.15.40:0/741613456 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:12.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.915+0000 7feb8259c640 1 --2- 172.21.15.40:0/741613456 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb84070e20 0x7feb8413e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:12.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.915+0000 7feb82d9d640 1 --2- 172.21.15.40:0/741613456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb841585e0 0x7feb84136ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:12.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.915+0000 7feb7bfff640 1 -- 172.21.15.40:0/741613456 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb84070e20 msgr2=0x7feb8413e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:12.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.915+0000 7feb7bfff640 1 --2- 172.21.15.40:0/741613456 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb84070e20 0x7feb8413e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.915+0000 7feb7bfff640 1 -- 172.21.15.40:0/741613456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb841585e0 msgr2=0x7feb84136ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:12.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.915+0000 7feb7bfff640 1 --2- 172.21.15.40:0/741613456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb841585e0 0x7feb84136ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:12.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.915+0000 7feb7bfff640 1 -- 172.21.15.40:0/741613456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c009900 con 0x7feb8413af60 2023-10-18T09:00:12.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.916+0000 7feb7bfff640 1 --2- 172.21.15.40:0/741613456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb8413af60 0x7feb8414ffe0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00d980 tx=0x7feb6c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:12.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.916+0000 7feb7affd640 1 -- 172.21.15.40:0/741613456 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c009c30 con 0x7feb8413af60 2023-10-18T09:00:12.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.916+0000 7feb7affd640 1 -- 172.21.15.40:0/741613456 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feb6c009e80 con 0x7feb8413af60 2023-10-18T09:00:12.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.916+0000 7feb7affd640 1 -- 172.21.15.40:0/741613456 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c0106a0 con 0x7feb8413af60 2023-10-18T09:00:12.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.916+0000 7feb8359e640 1 -- 172.21.15.40:0/741613456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb84150520 con 0x7feb8413af60 2023-10-18T09:00:12.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.916+0000 7feb8359e640 1 -- 172.21.15.40:0/741613456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb840b0e70 con 0x7feb8413af60 2023-10-18T09:00:12.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.917+0000 7feb8359e640 1 -- 172.21.15.40:0/741613456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb840b1670 con 0x7feb8413af60 2023-10-18T09:00:12.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.918+0000 7feb7affd640 1 -- 172.21.15.40:0/741613456 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feb6c010800 con 0x7feb8413af60 2023-10-18T09:00:12.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.919+0000 7feb7affd640 1 --2- 172.21.15.40:0/741613456 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb6406ae40 0x7feb6406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:12.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.919+0000 7feb7affd640 1 -- 172.21.15.40:0/741613456 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feb6c0897b0 con 0x7feb8413af60 2023-10-18T09:00:12.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.920+0000 7feb8259c640 1 --2- 172.21.15.40:0/741613456 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb6406ae40 0x7feb6406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:12.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.921+0000 7feb8259c640 1 --2- 172.21.15.40:0/741613456 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb6406ae40 0x7feb6406d300 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7feb74000f60 tx=0x7feb74009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:12.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:12.924+0000 7feb7affd640 1 -- 172.21.15.40:0/741613456 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb840b1670 con 0x7feb8413af60 2023-10-18T09:00:13.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.116+0000 7feb8359e640 1 -- 172.21.15.40:0/741613456 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7feb840b1670 con 0x7feb8413af60 2023-10-18T09:00:13.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.691+0000 7feb7affd640 1 -- 172.21.15.40:0/741613456 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) v1 ==== 138+0+0 (secure 0 0 0) 0x7feb840b1670 con 0x7feb8413af60 2023-10-18T09:00:13.693 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-10-18T09:00:13.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.694+0000 7feb8359e640 1 -- 172.21.15.40:0/741613456 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb6406ae40 msgr2=0x7feb6406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:13.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.694+0000 7feb8359e640 1 --2- 172.21.15.40:0/741613456 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb6406ae40 0x7feb6406d300 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7feb74000f60 tx=0x7feb74009040 comp rx=0 tx=0).stop 2023-10-18T09:00:13.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.694+0000 7feb8359e640 1 -- 172.21.15.40:0/741613456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb8413af60 msgr2=0x7feb8414ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:13.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.694+0000 7feb8359e640 1 --2- 172.21.15.40:0/741613456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb8413af60 0x7feb8414ffe0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7feb6c00d980 tx=0x7feb6c00de40 comp rx=0 tx=0).stop 2023-10-18T09:00:13.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.695+0000 7feb8359e640 1 -- 172.21.15.40:0/741613456 shutdown_connections 2023-10-18T09:00:13.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.695+0000 7feb8359e640 1 --2- 172.21.15.40:0/741613456 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb6406ae40 0x7feb6406d300 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:13.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.695+0000 7feb8359e640 1 --2- 172.21.15.40:0/741613456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb841585e0 0x7feb84136ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.695+0000 7feb8359e640 1 --2- 172.21.15.40:0/741613456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb8413af60 0x7feb8414ffe0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.695+0000 7feb8359e640 1 --2- 172.21.15.40:0/741613456 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb84070e20 0x7feb8413e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.695+0000 7feb8359e640 1 -- 172.21.15.40:0/741613456 >> 172.21.15.40:0/741613456 conn(0x7feb8406b7d0 msgr2=0x7feb8415a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.695+0000 7feb8359e640 1 -- 172.21.15.40:0/741613456 shutdown_connections 2023-10-18T09:00:13.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.696+0000 7feb8359e640 1 -- 172.21.15.40:0/741613456 wait complete. 2023-10-18T09:00:13.723 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T09:00:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.948+0000 7f06bb060640 1 -- 172.21.15.40:0/32747285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b406bb70 msgr2=0x7f06b406bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.948+0000 7f06bb060640 1 --2- 172.21.15.40:0/32747285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b406bb70 0x7f06b406bf50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f06b40af420 tx=0x7f06b002bd10 comp rx=0 tx=0).stop 2023-10-18T09:00:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.948+0000 7f06bb060640 1 -- 172.21.15.40:0/32747285 shutdown_connections 2023-10-18T09:00:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.948+0000 7f06bb060640 1 --2- 172.21.15.40:0/32747285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06b40b1550 0x7f06b40ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.948+0000 7f06bb060640 1 --2- 172.21.15.40:0/32747285 >> v2:172.21.15.144:3300/0 conn(0x7f06b41598e0 0x7f06b40b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.948+0000 7f06bb060640 1 --2- 172.21.15.40:0/32747285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b406bb70 0x7f06b406bf50 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.948+0000 7f06bb060640 1 -- 172.21.15.40:0/32747285 >> 172.21.15.40:0/32747285 conn(0x7f06b406ca10 msgr2=0x7f06b406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.948+0000 7f06bb060640 1 -- 172.21.15.40:0/32747285 shutdown_connections 2023-10-18T09:00:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.948+0000 7f06bb060640 1 -- 172.21.15.40:0/32747285 wait complete. 2023-10-18T09:00:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.951+0000 7f06bb060640 1 Processor -- start 2023-10-18T09:00:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.951+0000 7f06bb060640 1 -- start start 2023-10-18T09:00:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.952+0000 7f06bb060640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b406bb70 0x7f06b41f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.952+0000 7f06bb060640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06b40b1550 0x7f06b41f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.952+0000 7f06bb060640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06b41598e0 0x7f06b40abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.952+0000 7f06bb060640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b41f2b90 con 0x7f06b406bb70 2023-10-18T09:00:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.952+0000 7f06bb060640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40ac890 con 0x7f06b40b1550 2023-10-18T09:00:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.952+0000 7f06bb060640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40ac5b0 con 0x7f06b41598e0 2023-10-18T09:00:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.953+0000 7f06ba05e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b406bb70 0x7f06b41f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.953+0000 7f06ba05e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b406bb70 0x7f06b41f2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44352/0 (socket says 172.21.15.40:44352) 2023-10-18T09:00:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.953+0000 7f06ba05e640 1 -- 172.21.15.40:0/2210687809 learned_addr learned my addr 172.21.15.40:0/2210687809 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.953+0000 7f06ba85f640 1 --2- 172.21.15.40:0/2210687809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06b41598e0 0x7f06b40abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.953+0000 7f06b985d640 1 --2- 172.21.15.40:0/2210687809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06b40b1550 0x7f06b41f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.953+0000 7f06ba05e640 1 -- 172.21.15.40:0/2210687809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06b41598e0 msgr2=0x7f06b40abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.953+0000 7f06ba05e640 1 --2- 172.21.15.40:0/2210687809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06b41598e0 0x7f06b40abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.953+0000 7f06ba05e640 1 -- 172.21.15.40:0/2210687809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06b40b1550 msgr2=0x7f06b41f11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.953+0000 7f06ba05e640 1 --2- 172.21.15.40:0/2210687809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06b40b1550 0x7f06b41f11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.953+0000 7f06ba05e640 1 -- 172.21.15.40:0/2210687809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b0009560 con 0x7f06b406bb70 2023-10-18T09:00:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.953+0000 7f06b985d640 1 --2- 172.21.15.40:0/2210687809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06b40b1550 0x7f06b41f11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.954+0000 7f06ba05e640 1 --2- 172.21.15.40:0/2210687809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b406bb70 0x7f06b41f2650 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009850 tx=0x7f06b002da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.954+0000 7f06aaffd640 1 -- 172.21.15.40:0/2210687809 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0003800 con 0x7f06b406bb70 2023-10-18T09:00:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.954+0000 7f06aaffd640 1 -- 172.21.15.40:0/2210687809 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f06b0003a50 con 0x7f06b406bb70 2023-10-18T09:00:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.954+0000 7f06bb060640 1 -- 172.21.15.40:0/2210687809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b41f1b70 con 0x7f06b406bb70 2023-10-18T09:00:13.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.954+0000 7f06aaffd640 1 -- 172.21.15.40:0/2210687809 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0003e20 con 0x7f06b406bb70 2023-10-18T09:00:13.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.954+0000 7f06bb060640 1 -- 172.21.15.40:0/2210687809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b41f2130 con 0x7f06b406bb70 2023-10-18T09:00:13.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.955+0000 7f06bb060640 1 -- 172.21.15.40:0/2210687809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b4066e00 con 0x7f06b406bb70 2023-10-18T09:00:13.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.956+0000 7f06aaffd640 1 -- 172.21.15.40:0/2210687809 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f06b003e450 con 0x7f06b406bb70 2023-10-18T09:00:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.957+0000 7f06aaffd640 1 --2- 172.21.15.40:0/2210687809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f069806ae40 0x7f069806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.957+0000 7f06aaffd640 1 -- 172.21.15.40:0/2210687809 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f06b00ac030 con 0x7f06b406bb70 2023-10-18T09:00:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.958+0000 7f06b985d640 1 --2- 172.21.15.40:0/2210687809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f069806ae40 0x7f069806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:13.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.959+0000 7f06b985d640 1 --2- 172.21.15.40:0/2210687809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f069806ae40 0x7f069806d300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f06b41e34f0 tx=0x7f06a4005d30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:13.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:13.963+0000 7f06aaffd640 1 -- 172.21.15.40:0/2210687809 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f06b00deca0 con 0x7f06b406bb70 2023-10-18T09:00:14.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.151+0000 7f06bb060640 1 -- 172.21.15.40:0/2210687809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f06b40ccf80 con 0x7f06b406bb70 2023-10-18T09:00:14.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.698+0000 7f06aaffd640 1 -- 172.21.15.40:0/2210687809 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) v1 ==== 170+0+0 (secure 0 0 0) 0x7f06b00420d0 con 0x7f06b406bb70 2023-10-18T09:00:14.700 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-10-18T09:00:14.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.701+0000 7f06bb060640 1 -- 172.21.15.40:0/2210687809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f069806ae40 msgr2=0x7f069806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:14.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.701+0000 7f06bb060640 1 --2- 172.21.15.40:0/2210687809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f069806ae40 0x7f069806d300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f06b41e34f0 tx=0x7f06a4005d30 comp rx=0 tx=0).stop 2023-10-18T09:00:14.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.701+0000 7f06bb060640 1 -- 172.21.15.40:0/2210687809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b406bb70 msgr2=0x7f06b41f2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:14.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.701+0000 7f06bb060640 1 --2- 172.21.15.40:0/2210687809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b406bb70 0x7f06b41f2650 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009850 tx=0x7f06b002da40 comp rx=0 tx=0).stop 2023-10-18T09:00:14.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.702+0000 7f06bb060640 1 -- 172.21.15.40:0/2210687809 shutdown_connections 2023-10-18T09:00:14.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.702+0000 7f06bb060640 1 --2- 172.21.15.40:0/2210687809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f069806ae40 0x7f069806d300 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:14.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.702+0000 7f06bb060640 1 --2- 172.21.15.40:0/2210687809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06b41598e0 0x7f06b40abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:14.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.702+0000 7f06bb060640 1 --2- 172.21.15.40:0/2210687809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06b40b1550 0x7f06b41f11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:14.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.702+0000 7f06bb060640 1 --2- 172.21.15.40:0/2210687809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06b406bb70 0x7f06b41f2650 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:14.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.702+0000 7f06bb060640 1 -- 172.21.15.40:0/2210687809 >> 172.21.15.40:0/2210687809 conn(0x7f06b406ca10 msgr2=0x7f06b41538d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:14.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.702+0000 7f06bb060640 1 -- 172.21.15.40:0/2210687809 shutdown_connections 2023-10-18T09:00:14.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.702+0000 7f06bb060640 1 -- 172.21.15.40:0/2210687809 wait complete. 2023-10-18T09:00:14.735 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T09:00:14.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.941+0000 7ff0d4f9e640 1 -- 172.21.15.40:0/1846276620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0bc005030 msgr2=0x7ff0bc0058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:14.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.941+0000 7ff0d4f9e640 1 --2- 172.21.15.40:0/1846276620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0bc005030 0x7ff0bc0058d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8009b50 tx=0x7ff0bc006900 comp rx=0 tx=0).stop 2023-10-18T09:00:14.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.942+0000 7ff0ced9d640 1 -- 172.21.15.40:0/1846276620 reap_dead start 2023-10-18T09:00:14.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.942+0000 7ff0d4f9e640 1 -- 172.21.15.40:0/1846276620 shutdown_connections 2023-10-18T09:00:14.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.942+0000 7ff0d4f9e640 1 -- 172.21.15.40:0/1846276620 >> 172.21.15.40:0/1846276620 conn(0x7ff0d0069720 msgr2=0x7ff0d0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:14.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.942+0000 7ff0d4f9e640 1 -- 172.21.15.40:0/1846276620 shutdown_connections 2023-10-18T09:00:14.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.942+0000 7ff0d4f9e640 1 -- 172.21.15.40:0/1846276620 wait complete. 2023-10-18T09:00:14.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.944+0000 7ff0d4f9e640 1 Processor -- start 2023-10-18T09:00:14.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.944+0000 7ff0d4f9e640 1 -- start start 2023-10-18T09:00:14.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0d4f9e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0bc005030 0x7ff0d013e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:14.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0d4f9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0d01554d0 0x7ff0d0155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:14.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0d4f9e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0d0151190 0x7ff0d0151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:14.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0d4f9e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d013e880 con 0x7ff0d01554d0 2023-10-18T09:00:14.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0d4f9e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d0155e50 con 0x7ff0bc005030 2023-10-18T09:00:14.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0d4f9e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d00ac300 con 0x7ff0d0151190 2023-10-18T09:00:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0cdd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0d01554d0 0x7ff0d0155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0cdd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0d01554d0 0x7ff0d0155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48368/0 (socket says 172.21.15.40:48368) 2023-10-18T09:00:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0cdd9b640 1 -- 172.21.15.40:0/2205507958 learned_addr learned my addr 172.21.15.40:0/2205507958 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0ce59c640 1 --2- 172.21.15.40:0/2205507958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0bc005030 0x7ff0d013e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0ced9d640 1 --2- 172.21.15.40:0/2205507958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0d0151190 0x7ff0d0151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0cdd9b640 1 -- 172.21.15.40:0/2205507958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0d0151190 msgr2=0x7ff0d0151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0cdd9b640 1 --2- 172.21.15.40:0/2205507958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0d0151190 0x7ff0d0151610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0cdd9b640 1 -- 172.21.15.40:0/2205507958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0bc005030 msgr2=0x7ff0d013e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0cdd9b640 1 --2- 172.21.15.40:0/2205507958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0bc005030 0x7ff0d013e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0cdd9b640 1 -- 172.21.15.40:0/2205507958 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d0152160 con 0x7ff0d01554d0 2023-10-18T09:00:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.945+0000 7ff0ce59c640 1 --2- 172.21.15.40:0/2205507958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0bc005030 0x7ff0d013e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.946+0000 7ff0cdd9b640 1 --2- 172.21.15.40:0/2205507958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0d01554d0 0x7ff0d0155910 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff0b802cb70 tx=0x7ff0b8033c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.946+0000 7ff0c6ffd640 1 -- 172.21.15.40:0/2205507958 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8004220 con 0x7ff0d01554d0 2023-10-18T09:00:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.946+0000 7ff0c6ffd640 1 -- 172.21.15.40:0/2205507958 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff0b8004380 con 0x7ff0d01554d0 2023-10-18T09:00:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.946+0000 7ff0c6ffd640 1 -- 172.21.15.40:0/2205507958 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b803d940 con 0x7ff0d01554d0 2023-10-18T09:00:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.946+0000 7ff0d4f9e640 1 -- 172.21.15.40:0/2205507958 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d0151e80 con 0x7ff0d01554d0 2023-10-18T09:00:14.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.946+0000 7ff0d4f9e640 1 -- 172.21.15.40:0/2205507958 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d013eef0 con 0x7ff0d01554d0 2023-10-18T09:00:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.947+0000 7ff0d4f9e640 1 -- 172.21.15.40:0/2205507958 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d00b0fc0 con 0x7ff0d01554d0 2023-10-18T09:00:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.949+0000 7ff0c6ffd640 1 -- 172.21.15.40:0/2205507958 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff0b803daa0 con 0x7ff0d01554d0 2023-10-18T09:00:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.950+0000 7ff0c6ffd640 1 --2- 172.21.15.40:0/2205507958 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff0b006ae70 0x7ff0b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.950+0000 7ff0c6ffd640 1 -- 172.21.15.40:0/2205507958 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5609+0+0 (secure 0 0 0) 0x7ff0b8033e40 con 0x7ff0d01554d0 2023-10-18T09:00:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.950+0000 7ff0ce59c640 1 --2- 172.21.15.40:0/2205507958 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff0b006ae70 0x7ff0b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.951+0000 7ff0ce59c640 1 --2- 172.21.15.40:0/2205507958 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff0b006ae70 0x7ff0b006d330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00013b0 tx=0x7ff0c0006d90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:14.954+0000 7ff0c6ffd640 1 -- 172.21.15.40:0/2205507958 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff0b8077430 con 0x7ff0d01554d0 2023-10-18T09:00:15.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.195+0000 7ff0d4f9e640 1 -- 172.21.15.40:0/2205507958 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff0d00b2190 con 0x7ff0d01554d0 2023-10-18T09:00:15.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.750+0000 7ff0c6ffd640 1 -- 172.21.15.40:0/2205507958 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v38) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff0b807aa50 con 0x7ff0d01554d0 2023-10-18T09:00:15.758 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 10 and data pool 11 2023-10-18T09:00:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.760+0000 7ff0d4f9e640 1 -- 172.21.15.40:0/2205507958 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff0b006ae70 msgr2=0x7ff0b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.760+0000 7ff0d4f9e640 1 --2- 172.21.15.40:0/2205507958 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff0b006ae70 0x7ff0b006d330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00013b0 tx=0x7ff0c0006d90 comp rx=0 tx=0).stop 2023-10-18T09:00:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.760+0000 7ff0d4f9e640 1 -- 172.21.15.40:0/2205507958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0d01554d0 msgr2=0x7ff0d0155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.760+0000 7ff0d4f9e640 1 --2- 172.21.15.40:0/2205507958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0d01554d0 0x7ff0d0155910 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff0b802cb70 tx=0x7ff0b8033c50 comp rx=0 tx=0).stop 2023-10-18T09:00:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.760+0000 7ff0d4f9e640 1 -- 172.21.15.40:0/2205507958 shutdown_connections 2023-10-18T09:00:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.760+0000 7ff0d4f9e640 1 --2- 172.21.15.40:0/2205507958 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff0b006ae70 0x7ff0b006d330 secure :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00013b0 tx=0x7ff0c0006d90 comp rx=0 tx=0).stop 2023-10-18T09:00:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.760+0000 7ff0d4f9e640 1 --2- 172.21.15.40:0/2205507958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0d0151190 0x7ff0d0151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.760+0000 7ff0d4f9e640 1 --2- 172.21.15.40:0/2205507958 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0d01554d0 0x7ff0d0155910 secure :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff0b802cb70 tx=0x7ff0b8033c50 comp rx=0 tx=0).stop 2023-10-18T09:00:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.760+0000 7ff0d4f9e640 1 --2- 172.21.15.40:0/2205507958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0bc005030 0x7ff0d013e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:15.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.760+0000 7ff0d4f9e640 1 -- 172.21.15.40:0/2205507958 >> 172.21.15.40:0/2205507958 conn(0x7ff0d0069720 msgr2=0x7ff0d00aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.761+0000 7ff0d4f9e640 1 -- 172.21.15.40:0/2205507958 shutdown_connections 2023-10-18T09:00:15.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.761+0000 7ff0d4f9e640 1 -- 172.21.15.40:0/2205507958 wait complete. 2023-10-18T09:00:15.789 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:00:15.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.981+0000 7fd3c6f8e640 1 -- 172.21.15.40:0/1371792753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3c015a2e0 msgr2=0x7fd3c00cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:15.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.981+0000 7fd3c6f8e640 1 --2- 172.21.15.40:0/1371792753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3c015a2e0 0x7fd3c00cbc20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009830 tx=0x7fd3b402bd50 comp rx=0 tx=0).stop 2023-10-18T09:00:15.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.981+0000 7fd3c6f8e640 1 -- 172.21.15.40:0/1371792753 shutdown_connections 2023-10-18T09:00:15.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.981+0000 7fd3c6f8e640 1 --2- 172.21.15.40:0/1371792753 >> v2:172.21.15.144:3300/0 conn(0x7fd3c015a6c0 0x7fd3c006b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:15.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.981+0000 7fd3c6f8e640 1 --2- 172.21.15.40:0/1371792753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3c015a2e0 0x7fd3c00cbc20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:15.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.981+0000 7fd3c6f8e640 1 -- 172.21.15.40:0/1371792753 >> 172.21.15.40:0/1371792753 conn(0x7fd3c006cee0 msgr2=0x7fd3c0069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:15.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.982+0000 7fd3c6f8e640 1 -- 172.21.15.40:0/1371792753 shutdown_connections 2023-10-18T09:00:15.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.982+0000 7fd3c6f8e640 1 -- 172.21.15.40:0/1371792753 wait complete. 2023-10-18T09:00:15.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.983+0000 7fd3c6f8e640 1 Processor -- start 2023-10-18T09:00:15.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.983+0000 7fd3c6f8e640 1 -- start start 2023-10-18T09:00:15.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.984+0000 7fd3c6f8e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3c015a2e0 0x7fd3c015b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:15.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.984+0000 7fd3c6f8e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c015a6c0 0x7fd3c0143270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:15.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.984+0000 7fd3c6f8e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3c01437b0 0x7fd3c00a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:15.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.984+0000 7fd3c6f8e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c00a7440 con 0x7fd3c015a6c0 2023-10-18T09:00:15.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.984+0000 7fd3c6f8e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c0143c60 con 0x7fd3c01437b0 2023-10-18T09:00:15.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.984+0000 7fd3c6f8e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c00a7800 con 0x7fd3c015a2e0 2023-10-18T09:00:15.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.985+0000 7fd3bffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c015a6c0 0x7fd3c0143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:15.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.985+0000 7fd3bffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c015a6c0 0x7fd3c0143270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48372/0 (socket says 172.21.15.40:48372) 2023-10-18T09:00:15.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.985+0000 7fd3bffff640 1 -- 172.21.15.40:0/1201137882 learned_addr learned my addr 172.21.15.40:0/1201137882 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:15.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.985+0000 7fd3c4d2a640 1 --2- 172.21.15.40:0/1201137882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3c015a2e0 0x7fd3c015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:15.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.985+0000 7fd3c552b640 1 --2- 172.21.15.40:0/1201137882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3c01437b0 0x7fd3c00a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:15.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.985+0000 7fd3bffff640 1 -- 172.21.15.40:0/1201137882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3c015a2e0 msgr2=0x7fd3c015b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:15.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.985+0000 7fd3bffff640 1 --2- 172.21.15.40:0/1201137882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3c015a2e0 0x7fd3c015b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:15.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.985+0000 7fd3bffff640 1 -- 172.21.15.40:0/1201137882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3c01437b0 msgr2=0x7fd3c00a6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:15.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.985+0000 7fd3bffff640 1 --2- 172.21.15.40:0/1201137882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3c01437b0 0x7fd3c00a6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:15.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.985+0000 7fd3bffff640 1 -- 172.21.15.40:0/1201137882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b8009900 con 0x7fd3c015a6c0 2023-10-18T09:00:15.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.986+0000 7fd3bffff640 1 --2- 172.21.15.40:0/1201137882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c015a6c0 0x7fd3c0143270 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00c400 tx=0x7fd3ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:15.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.986+0000 7fd3bd7fa640 1 -- 172.21.15.40:0/1201137882 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0052d0 con 0x7fd3c015a6c0 2023-10-18T09:00:15.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.986+0000 7fd3c6f8e640 1 -- 172.21.15.40:0/1201137882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b4009490 con 0x7fd3c015a6c0 2023-10-18T09:00:15.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.986+0000 7fd3bd7fa640 1 -- 172.21.15.40:0/1201137882 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd3ac0035f0 con 0x7fd3c015a6c0 2023-10-18T09:00:15.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.986+0000 7fd3bd7fa640 1 -- 172.21.15.40:0/1201137882 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac003a00 con 0x7fd3c015a6c0 2023-10-18T09:00:15.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.986+0000 7fd3c6f8e640 1 -- 172.21.15.40:0/1201137882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c0144060 con 0x7fd3c015a6c0 2023-10-18T09:00:15.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.987+0000 7fd3c6f8e640 1 -- 172.21.15.40:0/1201137882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c006a5a0 con 0x7fd3c015a6c0 2023-10-18T09:00:15.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.988+0000 7fd3bd7fa640 1 -- 172.21.15.40:0/1201137882 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd3ac003b60 con 0x7fd3c015a6c0 2023-10-18T09:00:15.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.989+0000 7fd3bd7fa640 1 --2- 172.21.15.40:0/1201137882 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd3a406ae40 0x7fd3a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:15.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.989+0000 7fd3bd7fa640 1 -- 172.21.15.40:0/1201137882 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3ac088ff0 con 0x7fd3c015a6c0 2023-10-18T09:00:15.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.990+0000 7fd3c4d2a640 1 --2- 172.21.15.40:0/1201137882 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd3a406ae40 0x7fd3a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:15.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.991+0000 7fd3c4d2a640 1 --2- 172.21.15.40:0/1201137882 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd3a406ae40 0x7fd3a406d300 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4036000 tx=0x7fd3b4004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:15.994+0000 7fd3bd7fa640 1 -- 172.21.15.40:0/1201137882 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd3ac053f50 con 0x7fd3c015a6c0 2023-10-18T09:00:16.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.189+0000 7fd3c6f8e640 1 -- 172.21.15.40:0/1201137882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3c0066fd0 con 0x7fd3c015a6c0 2023-10-18T09:00:16.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.192+0000 7fd3bd7fa640 1 -- 172.21.15.40:0/1201137882 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16593 (secure 0 0 0) 0x7fd3ac057570 con 0x7fd3c015a6c0 2023-10-18T09:00:16.195 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:16.195 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":86,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:00:15.197176+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:00:13.170937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-18T09:00:14.205418+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:00:16.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.194+0000 7fd3c6f8e640 1 -- 172.21.15.40:0/1201137882 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd3a406ae40 msgr2=0x7fd3a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:16.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.194+0000 7fd3c6f8e640 1 --2- 172.21.15.40:0/1201137882 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd3a406ae40 0x7fd3a406d300 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4036000 tx=0x7fd3b4004730 comp rx=0 tx=0).stop 2023-10-18T09:00:16.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.194+0000 7fd3c6f8e640 1 -- 172.21.15.40:0/1201137882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c015a6c0 msgr2=0x7fd3c0143270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:16.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.194+0000 7fd3c6f8e640 1 --2- 172.21.15.40:0/1201137882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c015a6c0 0x7fd3c0143270 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00c400 tx=0x7fd3ac00c8c0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.195+0000 7fd3c6f8e640 1 -- 172.21.15.40:0/1201137882 shutdown_connections 2023-10-18T09:00:16.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.195+0000 7fd3c6f8e640 1 --2- 172.21.15.40:0/1201137882 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd3a406ae40 0x7fd3a406d300 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.195+0000 7fd3c6f8e640 1 --2- 172.21.15.40:0/1201137882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3c01437b0 0x7fd3c00a6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.195+0000 7fd3c6f8e640 1 --2- 172.21.15.40:0/1201137882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd3c015a6c0 0x7fd3c0143270 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.195+0000 7fd3c6f8e640 1 --2- 172.21.15.40:0/1201137882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3c015a2e0 0x7fd3c015b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.195+0000 7fd3c6f8e640 1 -- 172.21.15.40:0/1201137882 >> 172.21.15.40:0/1201137882 conn(0x7fd3c006cee0 msgr2=0x7fd3c00c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:16.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.195+0000 7fd3c6f8e640 1 -- 172.21.15.40:0/1201137882 shutdown_connections 2023-10-18T09:00:16.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.195+0000 7fd3c6f8e640 1 -- 172.21.15.40:0/1201137882 wait complete. 2023-10-18T09:00:16.215 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:00:16.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.441+0000 7f5b7f59e640 1 -- 172.21.15.40:0/121677100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b800cdca0 msgr2=0x7f5b800ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:16.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.441+0000 7f5b7f59e640 1 --2- 172.21.15.40:0/121677100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b800cdca0 0x7f5b800ce110 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f5b78005fd0 tx=0x7f5b7802be10 comp rx=0 tx=0).stop 2023-10-18T09:00:16.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.441+0000 7f5b7f59e640 1 -- 172.21.15.40:0/121677100 shutdown_connections 2023-10-18T09:00:16.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.441+0000 7f5b7f59e640 1 --2- 172.21.15.40:0/121677100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b800cdca0 0x7f5b800ce110 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.441+0000 7f5b7f59e640 1 --2- 172.21.15.40:0/121677100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b8015b880 0x7f5b800cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.441+0000 7f5b7f59e640 1 -- 172.21.15.40:0/121677100 >> 172.21.15.40:0/121677100 conn(0x7f5b8008aca0 msgr2=0x7f5b8009a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.441+0000 7f5b7f59e640 1 -- 172.21.15.40:0/121677100 shutdown_connections 2023-10-18T09:00:16.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.441+0000 7f5b7f59e640 1 -- 172.21.15.40:0/121677100 wait complete. 2023-10-18T09:00:16.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.443+0000 7f5b7f59e640 1 Processor -- start 2023-10-18T09:00:16.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.444+0000 7f5b7f59e640 1 -- start start 2023-10-18T09:00:16.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.445+0000 7f5b7f59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b800cdca0 0x7f5b800cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:16.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.445+0000 7f5b7f59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b8015b4a0 0x7f5b801f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:16.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.445+0000 7f5b7f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b800c6bd0 0x7f5b801f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:16.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.445+0000 7f5b7f59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b80159070 con 0x7f5b800c6bd0 2023-10-18T09:00:16.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.445+0000 7f5b7f59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800cb270 con 0x7f5b800cdca0 2023-10-18T09:00:16.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.445+0000 7f5b7f59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b801f2d20 con 0x7f5b8015b4a0 2023-10-18T09:00:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.445+0000 7f5b7ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b800c6bd0 0x7f5b801f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.445+0000 7f5b7ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b800c6bd0 0x7f5b801f2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48380/0 (socket says 172.21.15.40:48380) 2023-10-18T09:00:16.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.445+0000 7f5b7ed9d640 1 -- 172.21.15.40:0/3277893472 learned_addr learned my addr 172.21.15.40:0/3277893472 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.446+0000 7f5b7e59c640 1 --2- 172.21.15.40:0/3277893472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b800cdca0 0x7f5b800cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.446+0000 7f5b7dd9b640 1 --2- 172.21.15.40:0/3277893472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b8015b4a0 0x7f5b801f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:16.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.446+0000 7f5b7ed9d640 1 -- 172.21.15.40:0/3277893472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b8015b4a0 msgr2=0x7f5b801f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.446+0000 7f5b7ed9d640 1 --2- 172.21.15.40:0/3277893472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b8015b4a0 0x7f5b801f27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.446+0000 7f5b7ed9d640 1 -- 172.21.15.40:0/3277893472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b800cdca0 msgr2=0x7f5b800cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.446+0000 7f5b7ed9d640 1 --2- 172.21.15.40:0/3277893472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b800cdca0 0x7f5b800cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.446+0000 7f5b7ed9d640 1 -- 172.21.15.40:0/3277893472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b78009560 con 0x7f5b800c6bd0 2023-10-18T09:00:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.446+0000 7f5b7e59c640 1 --2- 172.21.15.40:0/3277893472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b800cdca0 0x7f5b800cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:00:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.446+0000 7f5b7ed9d640 1 --2- 172.21.15.40:0/3277893472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b800c6bd0 0x7f5b801f2f20 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009230 tx=0x7f5b7802dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.447+0000 7f5b76ffd640 1 -- 172.21.15.40:0/3277893472 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78003800 con 0x7f5b800c6bd0 2023-10-18T09:00:16.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.447+0000 7f5b76ffd640 1 -- 172.21.15.40:0/3277893472 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5b78003a50 con 0x7f5b800c6bd0 2023-10-18T09:00:16.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.447+0000 7f5b76ffd640 1 -- 172.21.15.40:0/3277893472 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78003e60 con 0x7f5b800c6bd0 2023-10-18T09:00:16.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.447+0000 7f5b7f59e640 1 -- 172.21.15.40:0/3277893472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b800b3fc0 con 0x7f5b800c6bd0 2023-10-18T09:00:16.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.447+0000 7f5b7f59e640 1 -- 172.21.15.40:0/3277893472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b800b4580 con 0x7f5b800c6bd0 2023-10-18T09:00:16.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.448+0000 7f5b7f59e640 1 -- 172.21.15.40:0/3277893472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b80071da0 con 0x7f5b800c6bd0 2023-10-18T09:00:16.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.449+0000 7f5b76ffd640 1 -- 172.21.15.40:0/3277893472 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5b7803e450 con 0x7f5b800c6bd0 2023-10-18T09:00:16.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.450+0000 7f5b76ffd640 1 --2- 172.21.15.40:0/3277893472 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5b6006ae40 0x7f5b6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:16.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.451+0000 7f5b76ffd640 1 -- 172.21.15.40:0/3277893472 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b780ac200 con 0x7f5b800c6bd0 2023-10-18T09:00:16.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.451+0000 7f5b7e59c640 1 --2- 172.21.15.40:0/3277893472 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5b6006ae40 0x7f5b6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:16.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.452+0000 7f5b7e59c640 1 --2- 172.21.15.40:0/3277893472 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5b6006ae40 0x7f5b6006d300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5b700012e0 tx=0x7f5b70009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:16.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.455+0000 7f5b76ffd640 1 -- 172.21.15.40:0/3277893472 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5b78077330 con 0x7f5b800c6bd0 2023-10-18T09:00:16.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.651+0000 7f5b7f59e640 1 -- 172.21.15.40:0/3277893472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b80066e00 con 0x7f5b800c6bd0 2023-10-18T09:00:16.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.654+0000 7f5b76ffd640 1 -- 172.21.15.40:0/3277893472 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16593 (secure 0 0 0) 0x7f5b7807a950 con 0x7f5b800c6bd0 2023-10-18T09:00:16.656 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:16.657 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":86,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:00:15.197176+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:00:13.170937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-18T09:00:14.205418+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:00:16.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.656+0000 7f5b7f59e640 1 -- 172.21.15.40:0/3277893472 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5b6006ae40 msgr2=0x7f5b6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:16.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.656+0000 7f5b7f59e640 1 --2- 172.21.15.40:0/3277893472 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5b6006ae40 0x7f5b6006d300 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5b700012e0 tx=0x7f5b70009040 comp rx=0 tx=0).stop 2023-10-18T09:00:16.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.656+0000 7f5b7f59e640 1 -- 172.21.15.40:0/3277893472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b800c6bd0 msgr2=0x7f5b801f2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:16.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.656+0000 7f5b7f59e640 1 --2- 172.21.15.40:0/3277893472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b800c6bd0 0x7f5b801f2f20 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009230 tx=0x7f5b7802dac0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.657+0000 7f5b7f59e640 1 -- 172.21.15.40:0/3277893472 shutdown_connections 2023-10-18T09:00:16.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.657+0000 7f5b7f59e640 1 --2- 172.21.15.40:0/3277893472 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5b6006ae40 0x7f5b6006d300 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.657+0000 7f5b7f59e640 1 --2- 172.21.15.40:0/3277893472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b800c6bd0 0x7f5b801f2f20 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.657+0000 7f5b7f59e640 1 --2- 172.21.15.40:0/3277893472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b8015b4a0 0x7f5b801f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.657+0000 7f5b7f59e640 1 --2- 172.21.15.40:0/3277893472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b800cdca0 0x7f5b800cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.657+0000 7f5b7f59e640 1 -- 172.21.15.40:0/3277893472 >> 172.21.15.40:0/3277893472 conn(0x7f5b8008aca0 msgr2=0x7f5b80072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:16.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.657+0000 7f5b7f59e640 1 -- 172.21.15.40:0/3277893472 shutdown_connections 2023-10-18T09:00:16.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.657+0000 7f5b7f59e640 1 -- 172.21.15.40:0/3277893472 wait complete. 2023-10-18T09:00:16.678 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T09:00:16.689 INFO:tasks.ceph.mds.c.smithi040.stderr:2023-10-18T09:00:16.687+0000 7f45fb56c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T09:00:16.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.908+0000 7f7ed236e640 1 -- 172.21.15.40:0/3682413876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7eb40039a0 msgr2=0x7f7eb4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:16.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.908+0000 7f7ed236e640 1 --2- 172.21.15.40:0/3682413876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7eb40039a0 0x7f7eb4003dc0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc001680 tx=0x7f7eb40057f0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.908+0000 7f7ed236e640 1 -- 172.21.15.40:0/3682413876 shutdown_connections 2023-10-18T09:00:16.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.908+0000 7f7ed236e640 1 --2- 172.21.15.40:0/3682413876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7eb40039a0 0x7f7eb4003dc0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.908+0000 7f7ed236e640 1 -- 172.21.15.40:0/3682413876 >> 172.21.15.40:0/3682413876 conn(0x7f7ecc069720 msgr2=0x7f7ecc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:16.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.909+0000 7f7ed236e640 1 -- 172.21.15.40:0/3682413876 shutdown_connections 2023-10-18T09:00:16.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.909+0000 7f7ed236e640 1 -- 172.21.15.40:0/3682413876 wait complete. 2023-10-18T09:00:16.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.910+0000 7f7ed236e640 1 Processor -- start 2023-10-18T09:00:16.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.911+0000 7f7ed236e640 1 -- start start 2023-10-18T09:00:16.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.911+0000 7f7ed236e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7eb40039a0 0x7f7ecc0aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:16.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.911+0000 7f7ed236e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ecc0b10f0 0x7f7ecc15bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:16.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.912+0000 7f7ed236e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ecc15c1e0 0x7f7ecc15c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:16.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.912+0000 7f7ed236e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc13a170 con 0x7f7ecc15c1e0 2023-10-18T09:00:16.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.912+0000 7f7ed236e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0ab180 con 0x7f7ecc0b10f0 2023-10-18T09:00:16.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.912+0000 7f7ed236e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0ab460 con 0x7f7eb40039a0 2023-10-18T09:00:16.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.912+0000 7f7ed090b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ecc15c1e0 0x7f7ecc15c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:16.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.912+0000 7f7ed090b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ecc15c1e0 0x7f7ecc15c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48392/0 (socket says 172.21.15.40:48392) 2023-10-18T09:00:16.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.912+0000 7f7ed090b640 1 -- 172.21.15.40:0/398569866 learned_addr learned my addr 172.21.15.40:0/398569866 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:16.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.912+0000 7f7ecbfff640 1 --2- 172.21.15.40:0/398569866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7eb40039a0 0x7f7ecc0aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:16.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.912+0000 7f7ed090b640 1 -- 172.21.15.40:0/398569866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7eb40039a0 msgr2=0x7f7ecc0aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:16.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.912+0000 7f7ed090b640 1 --2- 172.21.15.40:0/398569866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7eb40039a0 0x7f7ecc0aa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.912+0000 7f7ecb7fe640 1 --2- 172.21.15.40:0/398569866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ecc0b10f0 0x7f7ecc15bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:16.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.912+0000 7f7ed090b640 1 -- 172.21.15.40:0/398569866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ecc0b10f0 msgr2=0x7f7ecc15bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:16.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.912+0000 7f7ed090b640 1 --2- 172.21.15.40:0/398569866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ecc0b10f0 0x7f7ecc15bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:16.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.912+0000 7f7ed090b640 1 -- 172.21.15.40:0/398569866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc0ab740 con 0x7f7ecc15c1e0 2023-10-18T09:00:16.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.913+0000 7f7ed090b640 1 --2- 172.21.15.40:0/398569866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ecc15c1e0 0x7f7ecc15c660 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0cdc40 tx=0x7f7ec400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:16.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.913+0000 7f7ec8ff9640 1 -- 172.21.15.40:0/398569866 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4003b80 con 0x7f7ecc15c1e0 2023-10-18T09:00:16.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.913+0000 7f7ec8ff9640 1 -- 172.21.15.40:0/398569866 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7ec4003ce0 con 0x7f7ecc15c1e0 2023-10-18T09:00:16.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.913+0000 7f7ec8ff9640 1 -- 172.21.15.40:0/398569866 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4005330 con 0x7f7ecc15c1e0 2023-10-18T09:00:16.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.913+0000 7f7ed236e640 1 -- 172.21.15.40:0/398569866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ecc0aba20 con 0x7f7ecc15c1e0 2023-10-18T09:00:16.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.913+0000 7f7ed236e640 1 -- 172.21.15.40:0/398569866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ecc1522c0 con 0x7f7ecc15c1e0 2023-10-18T09:00:16.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.914+0000 7f7ed236e640 1 -- 172.21.15.40:0/398569866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ecc0abbb0 con 0x7f7ecc15c1e0 2023-10-18T09:00:16.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.916+0000 7f7ec8ff9640 1 -- 172.21.15.40:0/398569866 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7ec4005490 con 0x7f7ecc15c1e0 2023-10-18T09:00:16.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.917+0000 7f7ec8ff9640 1 --2- 172.21.15.40:0/398569866 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7eb806ae40 0x7f7eb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:16.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.917+0000 7f7ec8ff9640 1 -- 172.21.15.40:0/398569866 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ec4089210 con 0x7f7ecc15c1e0 2023-10-18T09:00:16.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.917+0000 7f7ecbfff640 1 --2- 172.21.15.40:0/398569866 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7eb806ae40 0x7f7eb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:16.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.918+0000 7f7ecbfff640 1 --2- 172.21.15.40:0/398569866 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7eb806ae40 0x7f7eb806d300 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc034000 tx=0x7f7ebc039040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:16.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:16.921+0000 7f7ec8ff9640 1 -- 172.21.15.40:0/398569866 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ec4054350 con 0x7f7ecc15c1e0 2023-10-18T09:00:17.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:17.154+0000 7f7ed236e640 1 -- 172.21.15.40:0/398569866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7ecc066fd0 con 0x7f7ecc15c1e0 2023-10-18T09:00:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:17.745+0000 7f7ec8ff9640 1 -- 172.21.15.40:0/398569866 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7ec4057970 con 0x7f7ecc15c1e0 2023-10-18T09:00:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:17.749+0000 7f7ed236e640 1 -- 172.21.15.40:0/398569866 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7eb806ae40 msgr2=0x7f7eb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:17.749+0000 7f7ed236e640 1 --2- 172.21.15.40:0/398569866 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7eb806ae40 0x7f7eb806d300 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc034000 tx=0x7f7ebc039040 comp rx=0 tx=0).stop 2023-10-18T09:00:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:17.749+0000 7f7ed236e640 1 -- 172.21.15.40:0/398569866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ecc15c1e0 msgr2=0x7f7ecc15c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:17.749+0000 7f7ed236e640 1 --2- 172.21.15.40:0/398569866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ecc15c1e0 0x7f7ecc15c660 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0cdc40 tx=0x7f7ec400dcf0 comp rx=0 tx=0).stop 2023-10-18T09:00:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:17.749+0000 7f7ed236e640 1 -- 172.21.15.40:0/398569866 shutdown_connections 2023-10-18T09:00:17.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:17.749+0000 7f7ed236e640 1 --2- 172.21.15.40:0/398569866 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7eb806ae40 0x7f7eb806d300 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:17.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:17.749+0000 7f7ed236e640 1 --2- 172.21.15.40:0/398569866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ecc15c1e0 0x7f7ecc15c660 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:17.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:17.749+0000 7f7ed236e640 1 --2- 172.21.15.40:0/398569866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ecc0b10f0 0x7f7ecc15bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:17.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:17.749+0000 7f7ed236e640 1 --2- 172.21.15.40:0/398569866 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7eb40039a0 0x7f7ecc0aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:17.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:17.749+0000 7f7ed236e640 1 -- 172.21.15.40:0/398569866 >> 172.21.15.40:0/398569866 conn(0x7f7ecc069720 msgr2=0x7f7ecc0ad5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:17.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:17.749+0000 7f7ed236e640 1 -- 172.21.15.40:0/398569866 shutdown_connections 2023-10-18T09:00:17.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:17.749+0000 7f7ed236e640 1 -- 172.21.15.40:0/398569866 wait complete. 2023-10-18T09:00:17.773 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T09:00:17.774 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T09:00:18.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.058+0000 7f405deda640 1 -- 172.21.15.40:0/3610268337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4048005180 msgr2=0x7f404800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:18.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.058+0000 7f405deda640 1 --2- 172.21.15.40:0/3610268337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4048005180 0x7f404800b980 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f405000c640 tx=0x7f40480057f0 comp rx=0 tx=0).stop 2023-10-18T09:00:18.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.058+0000 7f405deda640 1 -- 172.21.15.40:0/3610268337 shutdown_connections 2023-10-18T09:00:18.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.058+0000 7f405deda640 1 --2- 172.21.15.40:0/3610268337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4048005180 0x7f404800b980 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:18.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.058+0000 7f405deda640 1 -- 172.21.15.40:0/3610268337 >> 172.21.15.40:0/3610268337 conn(0x7f405806ca10 msgr2=0x7f405806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:18.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.059+0000 7f405deda640 1 -- 172.21.15.40:0/3610268337 shutdown_connections 2023-10-18T09:00:18.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.059+0000 7f405deda640 1 -- 172.21.15.40:0/3610268337 wait complete. 2023-10-18T09:00:18.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.060+0000 7f405deda640 1 Processor -- start 2023-10-18T09:00:18.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.061+0000 7f405deda640 1 -- start start 2023-10-18T09:00:18.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.061+0000 7f405deda640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4048005180 0x7f40581e6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:18.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.061+0000 7f405deda640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40580c6800 0x7f40580c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:18.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.061+0000 7f405deda640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40581e2af0 0x7f40581e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:18.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.061+0000 7f405deda640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40581e3cf0 con 0x7f40581e2af0 2023-10-18T09:00:18.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.061+0000 7f405deda640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40581e34b0 con 0x7f4048005180 2023-10-18T09:00:18.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.061+0000 7f405deda640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40580c9370 con 0x7f40580c6800 2023-10-18T09:00:18.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.062+0000 7f405d6d9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40581e2af0 0x7f40581e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:18.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.062+0000 7f405d6d9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40581e2af0 0x7f40581e2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48412/0 (socket says 172.21.15.40:48412) 2023-10-18T09:00:18.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.062+0000 7f405d6d9640 1 -- 172.21.15.40:0/3651097036 learned_addr learned my addr 172.21.15.40:0/3651097036 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:18.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.062+0000 7f405ced8640 1 --2- 172.21.15.40:0/3651097036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4048005180 0x7f40581e6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:18.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.062+0000 7f4057fff640 1 --2- 172.21.15.40:0/3651097036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40580c6800 0x7f40580c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:18.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.062+0000 7f405d6d9640 1 -- 172.21.15.40:0/3651097036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40580c6800 msgr2=0x7f40580c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:18.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.062+0000 7f405d6d9640 1 --2- 172.21.15.40:0/3651097036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40580c6800 0x7f40580c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:18.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.062+0000 7f405d6d9640 1 -- 172.21.15.40:0/3651097036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4048005180 msgr2=0x7f40581e6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.062+0000 7f405d6d9640 1 --2- 172.21.15.40:0/3651097036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4048005180 0x7f40581e6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.062+0000 7f405d6d9640 1 -- 172.21.15.40:0/3651097036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40580c9650 con 0x7f40581e2af0 2023-10-18T09:00:18.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.062+0000 7f4057fff640 1 --2- 172.21.15.40:0/3651097036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40580c6800 0x7f40580c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:18.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.063+0000 7f405d6d9640 1 --2- 172.21.15.40:0/3651097036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40581e2af0 0x7f40581e2f70 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f405000c760 tx=0x7f4050001830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:18.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.063+0000 7f40557fa640 1 -- 172.21.15.40:0/3651097036 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4050001ae0 con 0x7f40581e2af0 2023-10-18T09:00:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.063+0000 7f40557fa640 1 -- 172.21.15.40:0/3651097036 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4050003970 con 0x7f40581e2af0 2023-10-18T09:00:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.063+0000 7f405deda640 1 -- 172.21.15.40:0/3651097036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40581e3f00 con 0x7f40581e2af0 2023-10-18T09:00:18.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.063+0000 7f40557fa640 1 -- 172.21.15.40:0/3651097036 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4050003d80 con 0x7f40581e2af0 2023-10-18T09:00:18.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.063+0000 7f405deda640 1 -- 172.21.15.40:0/3651097036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40581e6760 con 0x7f40581e2af0 2023-10-18T09:00:18.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.064+0000 7f405deda640 1 -- 172.21.15.40:0/3651097036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40581e4420 con 0x7f40581e2af0 2023-10-18T09:00:18.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.065+0000 7f40557fa640 1 -- 172.21.15.40:0/3651097036 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4050031030 con 0x7f40581e2af0 2023-10-18T09:00:18.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.066+0000 7f40557fa640 1 --2- 172.21.15.40:0/3651097036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f403c06ae70 0x7f403c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:18.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.066+0000 7f40557fa640 1 -- 172.21.15.40:0/3651097036 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40500aa120 con 0x7f40581e2af0 2023-10-18T09:00:18.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.067+0000 7f405ced8640 1 --2- 172.21.15.40:0/3651097036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f403c06ae70 0x7f403c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:18.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.068+0000 7f405ced8640 1 --2- 172.21.15.40:0/3651097036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f403c06ae70 0x7f403c06d330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f404c007fd0 tx=0x7f404c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:18.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.070+0000 7f40557fa640 1 -- 172.21.15.40:0/3651097036 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40581e4420 con 0x7f40581e2af0 2023-10-18T09:00:18.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.304+0000 7f405deda640 1 -- 172.21.15.40:0/3651097036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f40581e4420 con 0x7f40581e2af0 2023-10-18T09:00:18.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.747+0000 7f40557fa640 1 -- 172.21.15.40:0/3651097036 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v41) v1 ==== 122+0+0 (secure 0 0 0) 0x7f40581e4420 con 0x7f40581e2af0 2023-10-18T09:00:18.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.751+0000 7f405deda640 1 -- 172.21.15.40:0/3651097036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f403c06ae70 msgr2=0x7f403c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:18.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.751+0000 7f405deda640 1 --2- 172.21.15.40:0/3651097036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f403c06ae70 0x7f403c06d330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f404c007fd0 tx=0x7f404c009040 comp rx=0 tx=0).stop 2023-10-18T09:00:18.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.751+0000 7f405deda640 1 -- 172.21.15.40:0/3651097036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40581e2af0 msgr2=0x7f40581e2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:18.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.751+0000 7f405deda640 1 --2- 172.21.15.40:0/3651097036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40581e2af0 0x7f40581e2f70 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f405000c760 tx=0x7f4050001830 comp rx=0 tx=0).stop 2023-10-18T09:00:18.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.751+0000 7f405deda640 1 -- 172.21.15.40:0/3651097036 shutdown_connections 2023-10-18T09:00:18.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.751+0000 7f405deda640 1 --2- 172.21.15.40:0/3651097036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f403c06ae70 0x7f403c06d330 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:18.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.751+0000 7f405deda640 1 --2- 172.21.15.40:0/3651097036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40581e2af0 0x7f40581e2f70 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:18.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.751+0000 7f405deda640 1 --2- 172.21.15.40:0/3651097036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40580c6800 0x7f40580c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:18.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.751+0000 7f405deda640 1 --2- 172.21.15.40:0/3651097036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4048005180 0x7f40581e6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:18.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.751+0000 7f405deda640 1 -- 172.21.15.40:0/3651097036 >> 172.21.15.40:0/3651097036 conn(0x7f405806ca10 msgr2=0x7f405815c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:18.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.752+0000 7f405deda640 1 -- 172.21.15.40:0/3651097036 shutdown_connections 2023-10-18T09:00:18.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:18.752+0000 7f405deda640 1 -- 172.21.15.40:0/3651097036 wait complete. 2023-10-18T09:00:18.776 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T09:00:18.777 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:00:19.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.011+0000 7fae851e9640 1 -- 172.21.15.40:0/3331863719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae801425a0 msgr2=0x7fae800c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.011+0000 7fae851e9640 1 --2- 172.21.15.40:0/3331863719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae801425a0 0x7fae800c9a50 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fae800ccea0 tx=0x7fae6802be40 comp rx=0 tx=0).stop 2023-10-18T09:00:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.011+0000 7fae851e9640 1 -- 172.21.15.40:0/3331863719 shutdown_connections 2023-10-18T09:00:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.011+0000 7fae851e9640 1 --2- 172.21.15.40:0/3331863719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae801425a0 0x7fae800c9a50 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.011+0000 7fae851e9640 1 --2- 172.21.15.40:0/3331863719 >> v2:172.21.15.144:3300/0 conn(0x7fae801420f0 0x7fae800b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.011+0000 7fae851e9640 1 -- 172.21.15.40:0/3331863719 >> 172.21.15.40:0/3331863719 conn(0x7fae80069720 msgr2=0x7fae80069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:19.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.011+0000 7fae851e9640 1 -- 172.21.15.40:0/3331863719 shutdown_connections 2023-10-18T09:00:19.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.011+0000 7fae851e9640 1 -- 172.21.15.40:0/3331863719 wait complete. 2023-10-18T09:00:19.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.013+0000 7fae851e9640 1 Processor -- start 2023-10-18T09:00:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.013+0000 7fae851e9640 1 -- start start 2023-10-18T09:00:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.014+0000 7fae851e9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae800c9f90 0x7fae801395b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.014+0000 7fae851e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae801420f0 0x7fae8015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.014+0000 7fae851e9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae801425a0 0x7fae80139de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.014+0000 7fae851e9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae80153490 con 0x7fae801420f0 2023-10-18T09:00:19.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.014+0000 7fae851e9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8015b150 con 0x7fae800c9f90 2023-10-18T09:00:19.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.014+0000 7fae851e9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8015a580 con 0x7fae801425a0 2023-10-18T09:00:19.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.014+0000 7fae77fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae801420f0 0x7fae8015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:19.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.014+0000 7fae77fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae801420f0 0x7fae8015b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48426/0 (socket says 172.21.15.40:48426) 2023-10-18T09:00:19.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.014+0000 7fae77fff640 1 -- 172.21.15.40:0/3994521621 learned_addr learned my addr 172.21.15.40:0/3994521621 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:19.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.015+0000 7fae7ffff640 1 --2- 172.21.15.40:0/3994521621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae800c9f90 0x7fae801395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:19.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.015+0000 7fae849e8640 1 --2- 172.21.15.40:0/3994521621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae801425a0 0x7fae80139de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:19.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.015+0000 7fae77fff640 1 -- 172.21.15.40:0/3994521621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae801425a0 msgr2=0x7fae80139de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:19.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.015+0000 7fae77fff640 1 --2- 172.21.15.40:0/3994521621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae801425a0 0x7fae80139de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.015+0000 7fae77fff640 1 -- 172.21.15.40:0/3994521621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae800c9f90 msgr2=0x7fae801395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.015+0000 7fae77fff640 1 --2- 172.21.15.40:0/3994521621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae800c9f90 0x7fae801395b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.015+0000 7fae77fff640 1 -- 172.21.15.40:0/3994521621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae68009560 con 0x7fae801420f0 2023-10-18T09:00:19.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.015+0000 7fae849e8640 1 --2- 172.21.15.40:0/3994521621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae801425a0 0x7fae80139de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:19.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.015+0000 7fae77fff640 1 --2- 172.21.15.40:0/3994521621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae801420f0 0x7fae8015b5e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fae68009850 tx=0x7fae6802db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:19.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.016+0000 7fae7d7fa640 1 -- 172.21.15.40:0/3994521621 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68003800 con 0x7fae801420f0 2023-10-18T09:00:19.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.016+0000 7fae7d7fa640 1 -- 172.21.15.40:0/3994521621 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fae68003a50 con 0x7fae801420f0 2023-10-18T09:00:19.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.016+0000 7fae7d7fa640 1 -- 172.21.15.40:0/3994521621 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68003e20 con 0x7fae801420f0 2023-10-18T09:00:19.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.016+0000 7fae851e9640 1 -- 172.21.15.40:0/3994521621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8015d9a0 con 0x7fae801420f0 2023-10-18T09:00:19.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.016+0000 7fae851e9640 1 -- 172.21.15.40:0/3994521621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8015a820 con 0x7fae801420f0 2023-10-18T09:00:19.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.017+0000 7fae851e9640 1 -- 172.21.15.40:0/3994521621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8015b2c0 con 0x7fae801420f0 2023-10-18T09:00:19.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.018+0000 7fae7d7fa640 1 -- 172.21.15.40:0/3994521621 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fae6803e450 con 0x7fae801420f0 2023-10-18T09:00:19.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.019+0000 7fae7d7fa640 1 --2- 172.21.15.40:0/3994521621 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fae6006ae40 0x7fae6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:19.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.019+0000 7fae7d7fa640 1 -- 172.21.15.40:0/3994521621 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae680ac1e0 con 0x7fae801420f0 2023-10-18T09:00:19.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.020+0000 7fae7ffff640 1 --2- 172.21.15.40:0/3994521621 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fae6006ae40 0x7fae6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:19.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.021+0000 7fae7ffff640 1 --2- 172.21.15.40:0/3994521621 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fae6006ae40 0x7fae6006d300 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fae70005fd0 tx=0x7fae70005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:19.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.024+0000 7fae7d7fa640 1 -- 172.21.15.40:0/3994521621 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fae680771f0 con 0x7fae801420f0 2023-10-18T09:00:19.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.262+0000 7fae851e9640 1 -- 172.21.15.40:0/3994521621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae80066e00 con 0x7fae801420f0 2023-10-18T09:00:19.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.263+0000 7fae7d7fa640 1 -- 172.21.15.40:0/3994521621 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fae6807a810 con 0x7fae801420f0 2023-10-18T09:00:19.266 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:19.266 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1341351808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1341351808},{"type":"v1","addr":"172.21.15.144:6836","nonce":1341351808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1187670073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1187670073},{"type":"v1","addr":"172.21.15.40:6833","nonce":1187670073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5626,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1615448513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1615448513},{"type":"v1","addr":"172.21.15.144:6837","nonce":1615448513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:00:15.197720+0000","modified":"2023-10-18T09:00:18.306206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1926421699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1926421699},{"type":"v1","addr":"172.21.15.40:6835","nonce":1926421699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-18T09:00:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 41 2023-10-18T09:00:19.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.266+0000 7fae851e9640 1 -- 172.21.15.40:0/3994521621 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fae6006ae40 msgr2=0x7fae6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:19.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.266+0000 7fae851e9640 1 --2- 172.21.15.40:0/3994521621 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fae6006ae40 0x7fae6006d300 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fae70005fd0 tx=0x7fae70005850 comp rx=0 tx=0).stop 2023-10-18T09:00:19.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.266+0000 7fae851e9640 1 -- 172.21.15.40:0/3994521621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae801420f0 msgr2=0x7fae8015b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:19.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.266+0000 7fae851e9640 1 --2- 172.21.15.40:0/3994521621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae801420f0 0x7fae8015b5e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fae68009850 tx=0x7fae6802db70 comp rx=0 tx=0).stop 2023-10-18T09:00:19.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.266+0000 7fae851e9640 1 -- 172.21.15.40:0/3994521621 shutdown_connections 2023-10-18T09:00:19.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.266+0000 7fae851e9640 1 --2- 172.21.15.40:0/3994521621 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fae6006ae40 0x7fae6006d300 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.266+0000 7fae851e9640 1 --2- 172.21.15.40:0/3994521621 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae801425a0 0x7fae80139de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.266+0000 7fae851e9640 1 --2- 172.21.15.40:0/3994521621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae801420f0 0x7fae8015b5e0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.266+0000 7fae851e9640 1 --2- 172.21.15.40:0/3994521621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae800c9f90 0x7fae801395b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.266+0000 7fae851e9640 1 -- 172.21.15.40:0/3994521621 >> 172.21.15.40:0/3994521621 conn(0x7fae80069720 msgr2=0x7fae800b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:19.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.267+0000 7fae851e9640 1 -- 172.21.15.40:0/3994521621 shutdown_connections 2023-10-18T09:00:19.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.267+0000 7fae851e9640 1 -- 172.21.15.40:0/3994521621 wait complete. 2023-10-18T09:00:19.289 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:00:19.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.523+0000 7f78f1c21640 1 -- 172.21.15.40:0/2860654091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec0cd9a0 msgr2=0x7f78ec0b1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:19.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.523+0000 7f78f1c21640 1 --2- 172.21.15.40:0/2860654091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec0cd9a0 0x7f78ec0b1070 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f78ec0acc20 tx=0x7f78e402bda0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.523+0000 7f78f1c21640 1 -- 172.21.15.40:0/2860654091 shutdown_connections 2023-10-18T09:00:19.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.523+0000 7f78f1c21640 1 --2- 172.21.15.40:0/2860654091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec0cd9a0 0x7f78ec0b1070 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.523+0000 7f78f1c21640 1 --2- 172.21.15.40:0/2860654091 >> v2:172.21.15.144:3300/0 conn(0x7f78ec151bd0 0x7f78ec1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.524+0000 7f78f1c21640 1 -- 172.21.15.40:0/2860654091 >> 172.21.15.40:0/2860654091 conn(0x7f78ec06cee0 msgr2=0x7f78ec069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:19.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.524+0000 7f78f1c21640 1 -- 172.21.15.40:0/2860654091 shutdown_connections 2023-10-18T09:00:19.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.524+0000 7f78f1c21640 1 -- 172.21.15.40:0/2860654091 wait complete. 2023-10-18T09:00:19.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.525+0000 7f78f1c21640 1 Processor -- start 2023-10-18T09:00:19.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.526+0000 7f78f1c21640 1 -- start start 2023-10-18T09:00:19.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.526+0000 7f78f1c21640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec0b0bc0 0x7f78ec14cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:19.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.527+0000 7f78f1c21640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec0cd9a0 0x7f78ec14b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:19.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.527+0000 7f78f1c21640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78ec151bd0 0x7f78ec0ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.527+0000 7f78f1c21640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec14ca00 con 0x7f78ec0b0bc0 2023-10-18T09:00:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.527+0000 7f78f1c21640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec0ca0f0 con 0x7f78ec0cd9a0 2023-10-18T09:00:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.527+0000 7f78f1c21640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec14c160 con 0x7f78ec151bd0 2023-10-18T09:00:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.527+0000 7f78eb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec0b0bc0 0x7f78ec14cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.527+0000 7f78eb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec0b0bc0 0x7f78ec14cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48442/0 (socket says 172.21.15.40:48442) 2023-10-18T09:00:19.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.527+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3307383356 learned_addr learned my addr 172.21.15.40:0/3307383356 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:19.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.527+0000 7f78ebfff640 1 --2- 172.21.15.40:0/3307383356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78ec151bd0 0x7f78ec0ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:19.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.527+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3307383356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78ec151bd0 msgr2=0x7f78ec0ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:19.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.527+0000 7f78eb7fe640 1 --2- 172.21.15.40:0/3307383356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78ec151bd0 0x7f78ec0ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.527+0000 7f78e2ffd640 1 --2- 172.21.15.40:0/3307383356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec0cd9a0 0x7f78ec14b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:19.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.527+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3307383356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec0cd9a0 msgr2=0x7f78ec14b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:19.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.527+0000 7f78eb7fe640 1 --2- 172.21.15.40:0/3307383356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec0cd9a0 0x7f78ec14b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.527+0000 7f78eb7fe640 1 -- 172.21.15.40:0/3307383356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e4009560 con 0x7f78ec0b0bc0 2023-10-18T09:00:19.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.527+0000 7f78ebfff640 1 --2- 172.21.15.40:0/3307383356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78ec151bd0 0x7f78ec0ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:19.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.528+0000 7f78eb7fe640 1 --2- 172.21.15.40:0/3307383356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec0b0bc0 0x7f78ec14cc10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00dd80 tx=0x7f78dc00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:19.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.528+0000 7f78e8ff9640 1 -- 172.21.15.40:0/3307383356 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc00e860 con 0x7f78ec0b0bc0 2023-10-18T09:00:19.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.528+0000 7f78e8ff9640 1 -- 172.21.15.40:0/3307383356 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f78dc00eab0 con 0x7f78ec0b0bc0 2023-10-18T09:00:19.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.528+0000 7f78f1c21640 1 -- 172.21.15.40:0/3307383356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ec14c440 con 0x7f78ec0b0bc0 2023-10-18T09:00:19.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.528+0000 7f78e8ff9640 1 -- 172.21.15.40:0/3307383356 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78dc00ee80 con 0x7f78ec0b0bc0 2023-10-18T09:00:19.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.528+0000 7f78f1c21640 1 -- 172.21.15.40:0/3307383356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ec14d150 con 0x7f78ec0b0bc0 2023-10-18T09:00:19.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.529+0000 7f78f1c21640 1 -- 172.21.15.40:0/3307383356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78ec066fd0 con 0x7f78ec0b0bc0 2023-10-18T09:00:19.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.530+0000 7f78e8ff9640 1 -- 172.21.15.40:0/3307383356 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f78dc016430 con 0x7f78ec0b0bc0 2023-10-18T09:00:19.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.532+0000 7f78e8ff9640 1 --2- 172.21.15.40:0/3307383356 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78cc06ae70 0x7f78cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:19.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.532+0000 7f78e8ff9640 1 -- 172.21.15.40:0/3307383356 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78dc08a310 con 0x7f78ec0b0bc0 2023-10-18T09:00:19.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.532+0000 7f78e2ffd640 1 --2- 172.21.15.40:0/3307383356 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78cc06ae70 0x7f78cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:19.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.533+0000 7f78e2ffd640 1 --2- 172.21.15.40:0/3307383356 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78cc06ae70 0x7f78cc06d330 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f78ec0cd880 tx=0x7f78d4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:19.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.536+0000 7f78e8ff9640 1 -- 172.21.15.40:0/3307383356 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78dc058ee0 con 0x7f78ec0b0bc0 2023-10-18T09:00:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.730+0000 7f78f1c21640 1 -- 172.21.15.40:0/3307383356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78ec0ca260 con 0x7f78ec0b0bc0 2023-10-18T09:00:19.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.733+0000 7f78e8ff9640 1 -- 172.21.15.40:0/3307383356 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16580 (secure 0 0 0) 0x7f78ec0ca260 con 0x7f78ec0b0bc0 2023-10-18T09:00:19.736 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:19.736 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":87,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:00:16.722867+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:00:13.170937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-18T09:00:14.205418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:00:19.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.736+0000 7f78f1c21640 1 -- 172.21.15.40:0/3307383356 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78cc06ae70 msgr2=0x7f78cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:19.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.736+0000 7f78f1c21640 1 --2- 172.21.15.40:0/3307383356 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78cc06ae70 0x7f78cc06d330 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f78ec0cd880 tx=0x7f78d4009040 comp rx=0 tx=0).stop 2023-10-18T09:00:19.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.736+0000 7f78f1c21640 1 -- 172.21.15.40:0/3307383356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec0b0bc0 msgr2=0x7f78ec14cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:19.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.736+0000 7f78f1c21640 1 --2- 172.21.15.40:0/3307383356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec0b0bc0 0x7f78ec14cc10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00dd80 tx=0x7f78dc00b730 comp rx=0 tx=0).stop 2023-10-18T09:00:19.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.736+0000 7f78f1c21640 1 -- 172.21.15.40:0/3307383356 shutdown_connections 2023-10-18T09:00:19.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.736+0000 7f78f1c21640 1 --2- 172.21.15.40:0/3307383356 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78cc06ae70 0x7f78cc06d330 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.736+0000 7f78f1c21640 1 --2- 172.21.15.40:0/3307383356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78ec151bd0 0x7f78ec0ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.736+0000 7f78f1c21640 1 --2- 172.21.15.40:0/3307383356 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78ec0cd9a0 0x7f78ec14b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.736+0000 7f78f1c21640 1 --2- 172.21.15.40:0/3307383356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78ec0b0bc0 0x7f78ec14cc10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.736+0000 7f78f1c21640 1 -- 172.21.15.40:0/3307383356 >> 172.21.15.40:0/3307383356 conn(0x7f78ec06cee0 msgr2=0x7f78ec0acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:19.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.737+0000 7f78f1c21640 1 -- 172.21.15.40:0/3307383356 shutdown_connections 2023-10-18T09:00:19.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.737+0000 7f78f1c21640 1 -- 172.21.15.40:0/3307383356 wait complete. 2023-10-18T09:00:19.757 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T09:00:19.758 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:00:19.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.986+0000 7f18c3fff640 1 -- 172.21.15.40:0/3194799212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18c40b1d10 msgr2=0x7f18c40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:19.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.986+0000 7f18c3fff640 1 --2- 172.21.15.40:0/3194799212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18c40b1d10 0x7f18c40b2170 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f18ac005fd0 tx=0x7f18ac02be20 comp rx=0 tx=0).stop 2023-10-18T09:00:19.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.987+0000 7f18c3fff640 1 -- 172.21.15.40:0/3194799212 shutdown_connections 2023-10-18T09:00:19.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.987+0000 7f18c3fff640 1 --2- 172.21.15.40:0/3194799212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18c40b1d10 0x7f18c40b2170 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.987+0000 7f18c3fff640 1 -- 172.21.15.40:0/3194799212 >> 172.21.15.40:0/3194799212 conn(0x7f18c4069720 msgr2=0x7f18c4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:19.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.987+0000 7f18c3fff640 1 -- 172.21.15.40:0/3194799212 shutdown_connections 2023-10-18T09:00:19.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.987+0000 7f18c3fff640 1 -- 172.21.15.40:0/3194799212 wait complete. 2023-10-18T09:00:19.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.988+0000 7f18c3fff640 1 Processor -- start 2023-10-18T09:00:19.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.989+0000 7f18c3fff640 1 -- start start 2023-10-18T09:00:19.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.990+0000 7f18c3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18c40b1d10 0x7f18c413e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:19.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.990+0000 7f18c3fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18c40caec0 0x7f18c4153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:19.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.990+0000 7f18c3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18c4157870 0x7f18c41402b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:19.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.990+0000 7f18c3fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c413fe20 con 0x7f18c40b1d10 2023-10-18T09:00:19.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.990+0000 7f18c3fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c413f210 con 0x7f18c4157870 2023-10-18T09:00:19.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.990+0000 7f18c3fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c413f4f0 con 0x7f18c40caec0 2023-10-18T09:00:19.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.990+0000 7f18c2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18c40b1d10 0x7f18c413e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:19.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.990+0000 7f18c2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18c40b1d10 0x7f18c413e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48460/0 (socket says 172.21.15.40:48460) 2023-10-18T09:00:19.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.990+0000 7f18c2ffd640 1 -- 172.21.15.40:0/3575307516 learned_addr learned my addr 172.21.15.40:0/3575307516 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:19.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.990+0000 7f18c37fe640 1 --2- 172.21.15.40:0/3575307516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18c4157870 0x7f18c41402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:19.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.990+0000 7f18bbfff640 1 --2- 172.21.15.40:0/3575307516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18c40caec0 0x7f18c4153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:19.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.990+0000 7f18c2ffd640 1 -- 172.21.15.40:0/3575307516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18c40caec0 msgr2=0x7f18c4153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:19.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.991+0000 7f18c2ffd640 1 --2- 172.21.15.40:0/3575307516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18c40caec0 0x7f18c4153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.991+0000 7f18c2ffd640 1 -- 172.21.15.40:0/3575307516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18c4157870 msgr2=0x7f18c41402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:19.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.991+0000 7f18c2ffd640 1 --2- 172.21.15.40:0/3575307516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18c4157870 0x7f18c41402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:19.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.991+0000 7f18c2ffd640 1 -- 172.21.15.40:0/3575307516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ac009560 con 0x7f18c40b1d10 2023-10-18T09:00:19.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.991+0000 7f18bbfff640 1 --2- 172.21.15.40:0/3575307516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18c40caec0 0x7f18c4153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:19.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.991+0000 7f18c2ffd640 1 --2- 172.21.15.40:0/3575307516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18c40b1d10 0x7f18c413e470 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f18b400b3e0 tx=0x7f18b400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:19.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.991+0000 7f18bb7fe640 1 -- 172.21.15.40:0/3575307516 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b4011830 con 0x7f18c40b1d10 2023-10-18T09:00:19.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.991+0000 7f18bb7fe640 1 -- 172.21.15.40:0/3575307516 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f18b4011aa0 con 0x7f18c40b1d10 2023-10-18T09:00:20.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.991+0000 7f18c3fff640 1 -- 172.21.15.40:0/3575307516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c413f7d0 con 0x7f18c40b1d10 2023-10-18T09:00:20.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.991+0000 7f18bb7fe640 1 -- 172.21.15.40:0/3575307516 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b4010380 con 0x7f18c40b1d10 2023-10-18T09:00:20.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.991+0000 7f18c3fff640 1 -- 172.21.15.40:0/3575307516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c413e9b0 con 0x7f18c40b1d10 2023-10-18T09:00:20.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.992+0000 7f18c3fff640 1 -- 172.21.15.40:0/3575307516 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c4066e00 con 0x7f18c40b1d10 2023-10-18T09:00:20.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.993+0000 7f18bb7fe640 1 -- 172.21.15.40:0/3575307516 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f18b40104e0 con 0x7f18c40b1d10 2023-10-18T09:00:20.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.994+0000 7f18bb7fe640 1 --2- 172.21.15.40:0/3575307516 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18a406ae40 0x7f18a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:20.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.994+0000 7f18bb7fe640 1 -- 172.21.15.40:0/3575307516 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18b4089b50 con 0x7f18c40b1d10 2023-10-18T09:00:20.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.995+0000 7f18bbfff640 1 --2- 172.21.15.40:0/3575307516 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18a406ae40 0x7f18a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:20.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.996+0000 7f18bbfff640 1 --2- 172.21.15.40:0/3575307516 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18a406ae40 0x7f18a406d300 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f18ac035000 tx=0x7f18ac004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:20.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:19.999+0000 7f18bb7fe640 1 -- 172.21.15.40:0/3575307516 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18b4054b60 con 0x7f18c40b1d10 2023-10-18T09:00:20.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.191+0000 7f18c3fff640 1 -- 172.21.15.40:0/3575307516 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f18c40caa60 con 0x7f18a406ae40 2023-10-18T09:00:20.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.217+0000 7f18bb7fe640 1 -- 172.21.15.40:0/3575307516 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+273449 (secure 0 0 0) 0x7f18ac004340 con 0x7f18a406ae40 2023-10-18T09:00:20.223 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:20.225 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:00:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.223+0000 7f18c3fff640 1 -- 172.21.15.40:0/3575307516 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18a406ae40 msgr2=0x7f18a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:20.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.223+0000 7f18c3fff640 1 --2- 172.21.15.40:0/3575307516 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18a406ae40 0x7f18a406d300 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f18ac035000 tx=0x7f18ac004730 comp rx=0 tx=0).stop 2023-10-18T09:00:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.223+0000 7f18c3fff640 1 -- 172.21.15.40:0/3575307516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18c40b1d10 msgr2=0x7f18c413e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.223+0000 7f18c3fff640 1 --2- 172.21.15.40:0/3575307516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18c40b1d10 0x7f18c413e470 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f18b400b3e0 tx=0x7f18b400b8a0 comp rx=0 tx=0).stop 2023-10-18T09:00:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.223+0000 7f18c3fff640 1 -- 172.21.15.40:0/3575307516 shutdown_connections 2023-10-18T09:00:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.223+0000 7f18c3fff640 1 --2- 172.21.15.40:0/3575307516 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18a406ae40 0x7f18a406d300 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:20.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.223+0000 7f18c3fff640 1 --2- 172.21.15.40:0/3575307516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18c4157870 0x7f18c41402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.223+0000 7f18c3fff640 1 --2- 172.21.15.40:0/3575307516 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18c40caec0 0x7f18c4153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.223+0000 7f18c3fff640 1 --2- 172.21.15.40:0/3575307516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18c40b1d10 0x7f18c413e470 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.223+0000 7f18c3fff640 1 -- 172.21.15.40:0/3575307516 >> 172.21.15.40:0/3575307516 conn(0x7f18c4069720 msgr2=0x7f18c4153340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.223+0000 7f18c3fff640 1 -- 172.21.15.40:0/3575307516 shutdown_connections 2023-10-18T09:00:20.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.223+0000 7f18c3fff640 1 -- 172.21.15.40:0/3575307516 wait complete. 2023-10-18T09:00:20.245 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2023-10-18T09:00:19.004899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":102,"acting":102,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557890,"statfs":{"total":1600000000,"available":1595279782,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.310184"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.708745+0000","last_change":"2023-10-18T09:00:14.708745+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.708745+0000","last_undegraded":"2023-10-18T09:00:14.708745+0000","last_fullsized":"2023-10-18T09:00:14.708745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.709438+0000","last_change":"2023-10-18T09:00:14.709438+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.709438+0000","last_undegraded":"2023-10-18T09:00:14.709438+0000","last_fullsized":"2023-10-18T09:00:14.709438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.707926+0000","last_change":"2023-10-18T09:00:14.707926+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.707926+0000","last_undegraded":"2023-10-18T09:00:14.707926+0000","last_fullsized":"2023-10-18T09:00:14.707926+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.703500+0000","last_change":"2023-10-18T09:00:14.703500+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.703500+0000","last_undegraded":"2023-10-18T09:00:14.703500+0000","last_fullsized":"2023-10-18T09:00:14.703500+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.711597+0000","last_change":"2023-10-18T09:00:14.711597+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.711597+0000","last_undegraded":"2023-10-18T09:00:14.711597+0000","last_fullsized":"2023-10-18T09:00:14.711597+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.709270+0000","last_change":"2023-10-18T09:00:14.709270+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.709270+0000","last_undegraded":"2023-10-18T09:00:14.709270+0000","last_fullsized":"2023-10-18T09:00:14.709270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.709112+0000","last_change":"2023-10-18T09:00:14.709112+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.709112+0000","last_undegraded":"2023-10-18T09:00:14.709112+0000","last_fullsized":"2023-10-18T09:00:14.709112+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.710603+0000","last_change":"2023-10-18T09:00:14.710603+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.710603+0000","last_undegraded":"2023-10-18T09:00:14.710603+0000","last_fullsized":"2023-10-18T09:00:14.710603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-18T09:00:14.722080+0000","last_change":"2023-10-18T09:00:14.719832+0000","last_active":"2023-10-18T09:00:14.722080+0000","last_peered":"2023-10-18T09:00:14.722080+0000","last_clean":"2023-10-18T09:00:14.722080+0000","last_became_active":"2023-10-18T09:00:14.719074+0000","last_became_peered":"2023-10-18T09:00:14.719074+0000","last_unstale":"2023-10-18T09:00:14.722080+0000","last_undegraded":"2023-10-18T09:00:14.722080+0000","last_fullsized":"2023-10-18T09:00:14.722080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:21.838152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.698383+0000","last_change":"2023-10-18T09:00:13.698383+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.698383+0000","last_undegraded":"2023-10-18T09:00:13.698383+0000","last_fullsized":"2023-10-18T09:00:13.698383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.700897+0000","last_change":"2023-10-18T09:00:13.700897+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.700897+0000","last_undegraded":"2023-10-18T09:00:13.700897+0000","last_fullsized":"2023-10-18T09:00:13.700897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.705500+0000","last_change":"2023-10-18T09:00:14.705500+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.705500+0000","last_undegraded":"2023-10-18T09:00:14.705500+0000","last_fullsized":"2023-10-18T09:00:14.705500+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.705976+0000","last_change":"2023-10-18T09:00:13.705976+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.705976+0000","last_undegraded":"2023-10-18T09:00:13.705976+0000","last_fullsized":"2023-10-18T09:00:13.705976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-18T09:00:14.721125+0000","last_change":"2023-10-18T09:00:14.720994+0000","last_active":"2023-10-18T09:00:14.721125+0000","last_peered":"2023-10-18T09:00:14.721125+0000","last_clean":"2023-10-18T09:00:14.721125+0000","last_became_active":"2023-10-18T09:00:14.720661+0000","last_became_peered":"2023-10-18T09:00:14.720661+0000","last_unstale":"2023-10-18T09:00:14.721125+0000","last_undegraded":"2023-10-18T09:00:14.721125+0000","last_fullsized":"2023-10-18T09:00:14.721125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:03:33.089526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.706980+0000","last_change":"2023-10-18T09:00:13.706980+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.706980+0000","last_undegraded":"2023-10-18T09:00:13.706980+0000","last_fullsized":"2023-10-18T09:00:13.706980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.698989+0000","last_change":"2023-10-18T09:00:13.698989+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.698989+0000","last_undegraded":"2023-10-18T09:00:13.698989+0000","last_fullsized":"2023-10-18T09:00:13.698989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.696228+0000","last_change":"2023-10-18T09:00:13.696228+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.696228+0000","last_undegraded":"2023-10-18T09:00:13.696228+0000","last_fullsized":"2023-10-18T09:00:13.696228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.699199+0000","last_change":"2023-10-18T09:00:13.699199+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.699199+0000","last_undegraded":"2023-10-18T09:00:13.699199+0000","last_fullsized":"2023-10-18T09:00:13.699199+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.707158+0000","last_change":"2023-10-18T09:00:14.707158+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.707158+0000","last_undegraded":"2023-10-18T09:00:14.707158+0000","last_fullsized":"2023-10-18T09:00:14.707158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.706160+0000","last_change":"2023-10-18T09:00:13.706160+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.706160+0000","last_undegraded":"2023-10-18T09:00:13.706160+0000","last_fullsized":"2023-10-18T09:00:13.706160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-18T09:00:14.723533+0000","last_change":"2023-10-18T09:00:14.723428+0000","last_active":"2023-10-18T09:00:14.723533+0000","last_peered":"2023-10-18T09:00:14.723533+0000","last_clean":"2023-10-18T09:00:14.723533+0000","last_became_active":"2023-10-18T09:00:14.723174+0000","last_became_peered":"2023-10-18T09:00:14.723174+0000","last_unstale":"2023-10-18T09:00:14.723533+0000","last_undegraded":"2023-10-18T09:00:14.723533+0000","last_fullsized":"2023-10-18T09:00:14.723533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:09:08.852692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.698316+0000","last_change":"2023-10-18T09:00:13.698316+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.698316+0000","last_undegraded":"2023-10-18T09:00:13.698316+0000","last_fullsized":"2023-10-18T09:00:13.698316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.706323+0000","last_change":"2023-10-18T09:00:14.706323+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.706323+0000","last_undegraded":"2023-10-18T09:00:14.706323+0000","last_fullsized":"2023-10-18T09:00:14.706323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.697251+0000","last_change":"2023-10-18T09:00:13.697251+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.697251+0000","last_undegraded":"2023-10-18T09:00:13.697251+0000","last_fullsized":"2023-10-18T09:00:13.697251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.705085+0000","last_change":"2023-10-18T09:00:13.705085+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.705085+0000","last_undegraded":"2023-10-18T09:00:13.705085+0000","last_fullsized":"2023-10-18T09:00:13.705085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.700521+0000","last_change":"2023-10-18T09:00:13.700521+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.700521+0000","last_undegraded":"2023-10-18T09:00:13.700521+0000","last_fullsized":"2023-10-18T09:00:13.700521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.701145+0000","last_change":"2023-10-18T09:00:13.701145+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.701145+0000","last_undegraded":"2023-10-18T09:00:13.701145+0000","last_fullsized":"2023-10-18T09:00:13.701145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.706157+0000","last_change":"2023-10-18T09:00:13.706157+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.706157+0000","last_undegraded":"2023-10-18T09:00:13.706157+0000","last_fullsized":"2023-10-18T09:00:13.706157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.709724+0000","last_change":"2023-10-18T09:00:14.709724+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.709724+0000","last_undegraded":"2023-10-18T09:00:14.709724+0000","last_fullsized":"2023-10-18T09:00:14.709724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.699049+0000","last_change":"2023-10-18T09:00:13.699049+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.699049+0000","last_undegraded":"2023-10-18T09:00:13.699049+0000","last_fullsized":"2023-10-18T09:00:13.699049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.704226+0000","last_change":"2023-10-18T09:00:13.704226+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.704226+0000","last_undegraded":"2023-10-18T09:00:13.704226+0000","last_fullsized":"2023-10-18T09:00:13.704226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-18T09:00:14.724961+0000","last_change":"2023-10-18T09:00:14.723114+0000","last_active":"2023-10-18T09:00:14.724961+0000","last_peered":"2023-10-18T09:00:14.724961+0000","last_clean":"2023-10-18T09:00:14.724961+0000","last_became_active":"2023-10-18T09:00:14.722764+0000","last_became_peered":"2023-10-18T09:00:14.722764+0000","last_unstale":"2023-10-18T09:00:14.724961+0000","last_undegraded":"2023-10-18T09:00:14.724961+0000","last_fullsized":"2023-10-18T09:00:14.724961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:05.813755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.708813+0000","last_change":"2023-10-18T09:00:14.708813+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.708813+0000","last_undegraded":"2023-10-18T09:00:14.708813+0000","last_fullsized":"2023-10-18T09:00:14.708813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.705311+0000","last_change":"2023-10-18T09:00:13.705311+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.705311+0000","last_undegraded":"2023-10-18T09:00:13.705311+0000","last_fullsized":"2023-10-18T09:00:13.705311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.708614+0000","last_change":"2023-10-18T09:00:14.708614+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.708614+0000","last_undegraded":"2023-10-18T09:00:14.708614+0000","last_fullsized":"2023-10-18T09:00:14.708614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.698536+0000","last_change":"2023-10-18T09:00:13.698536+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.698536+0000","last_undegraded":"2023-10-18T09:00:13.698536+0000","last_fullsized":"2023-10-18T09:00:13.698536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.708619+0000","last_change":"2023-10-18T09:00:14.708619+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.708619+0000","last_undegraded":"2023-10-18T09:00:14.708619+0000","last_fullsized":"2023-10-18T09:00:14.708619+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.697743+0000","last_change":"2023-10-18T09:00:13.697743+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.697743+0000","last_undegraded":"2023-10-18T09:00:13.697743+0000","last_fullsized":"2023-10-18T09:00:13.697743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.705179+0000","last_change":"2023-10-18T09:00:13.705179+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.705179+0000","last_undegraded":"2023-10-18T09:00:13.705179+0000","last_fullsized":"2023-10-18T09:00:13.705179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.698859+0000","last_change":"2023-10-18T09:00:13.698859+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.698859+0000","last_undegraded":"2023-10-18T09:00:13.698859+0000","last_fullsized":"2023-10-18T09:00:13.698859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.704694+0000","last_change":"2023-10-18T09:00:14.704694+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.704694+0000","last_undegraded":"2023-10-18T09:00:14.704694+0000","last_fullsized":"2023-10-18T09:00:14.704694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.696051+0000","last_change":"2023-10-18T09:00:13.696051+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.696051+0000","last_undegraded":"2023-10-18T09:00:13.696051+0000","last_fullsized":"2023-10-18T09:00:13.696051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.706477+0000","last_change":"2023-10-18T09:00:14.706477+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.706477+0000","last_undegraded":"2023-10-18T09:00:14.706477+0000","last_fullsized":"2023-10-18T09:00:14.706477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.697921+0000","last_change":"2023-10-18T09:00:13.697921+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.697921+0000","last_undegraded":"2023-10-18T09:00:13.697921+0000","last_fullsized":"2023-10-18T09:00:13.697921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.696049+0000","last_change":"2023-10-18T09:00:13.696049+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.696049+0000","last_undegraded":"2023-10-18T09:00:13.696049+0000","last_fullsized":"2023-10-18T09:00:13.696049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.697231+0000","last_change":"2023-10-18T09:00:13.697231+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.697231+0000","last_undegraded":"2023-10-18T09:00:13.697231+0000","last_fullsized":"2023-10-18T09:00:13.697231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.700302+0000","last_change":"2023-10-18T09:00:13.700302+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.700302+0000","last_undegraded":"2023-10-18T09:00:13.700302+0000","last_fullsized":"2023-10-18T09:00:13.700302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.704099+0000","last_change":"2023-10-18T09:00:13.704099+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.704099+0000","last_undegraded":"2023-10-18T09:00:13.704099+0000","last_fullsized":"2023-10-18T09:00:13.704099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.708806+0000","last_change":"2023-10-18T09:00:14.708806+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.708806+0000","last_undegraded":"2023-10-18T09:00:14.708806+0000","last_fullsized":"2023-10-18T09:00:14.708806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.706147+0000","last_change":"2023-10-18T09:00:13.706147+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.706147+0000","last_undegraded":"2023-10-18T09:00:13.706147+0000","last_fullsized":"2023-10-18T09:00:13.706147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":236,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T09:00:13.696161+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:00:13.696161+0000","last_peered":"2023-10-18T09:00:13.696161+0000","last_clean":"2023-10-18T09:00:13.696161+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:00:13.696161+0000","last_undegraded":"2023-10-18T09:00:13.696161+0000","last_fullsized":"2023-10-18T09:00:13.696161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:36.150056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1012,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194299,"statfs":{"total":200000000,"available":198963179,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.071,"15min":1.0149999999999999},"min":{"1min":0.72599999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.575,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.165,"5min":1.1000000000000001,"15min":1.03},"min":{"1min":0.84399999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.536,"5min":1.97,"15min":1.97},"last":1.5740000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.125,"15min":1.147},"min":{"1min":0.88200000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5169999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1659999999999999,"15min":1.1299999999999999},"min":{"1min":0.91800000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.083}]},{"osd":2,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.284,"15min":1.246},"min":{"1min":0.78600000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.522,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.524},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.236,"15min":1.2030000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.637,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.202}]},{"osd":3,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.2609999999999999,"15min":1.2629999999999999},"min":{"1min":0.80300000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.845,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3500000000000001,"15min":1.327},"min":{"1min":1.0069999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.736,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.389}]},{"osd":4,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.238,"15min":1.1759999999999999},"min":{"1min":0.94599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.825,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.393,"5min":1.351,"15min":1.3919999999999999},"min":{"1min":1.101,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.627,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.131}]},{"osd":5,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3580000000000001,"15min":1.355},"min":{"1min":0.89600000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7470000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.415,"15min":1.421},"min":{"1min":1.079,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7130000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.45}]},{"osd":6,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.482,"15min":1.464},"min":{"1min":1.355,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.8169999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.306},{"interface":"front","average":{"1min":1.512,"5min":1.484,"15min":1.5069999999999999},"min":{"1min":1.214,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.768,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.421}]}]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":430,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194882,"statfs":{"total":200000000,"available":199559364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.1759999999999999,"15min":1.105},"min":{"1min":0.71799999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.627,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.2709999999999999,"15min":1.248},"min":{"1min":0.86699999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7629999999999999,"5min":1.855,"15min":1.855},"last":2.0990000000000002}]},{"osd":1,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1040000000000001,"15min":1.0900000000000001},"min":{"1min":0.68500000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.488,"5min":1.633,"15min":1.633},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.198,"15min":1.1919999999999999},"min":{"1min":0.72599999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5409999999999999,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":1.8560000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.879,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5720000000000001,"5min":1.696,"15min":1.696},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2170000000000001,"15min":1.2010000000000001},"min":{"1min":0.90000000000000002,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.667,"5min":2.137,"15min":2.137},"last":1.032}]},{"osd":3,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.256,"15min":1.2589999999999999},"min":{"1min":0.91900000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.6719999999999999,"5min":1.784,"15min":1.784},"last":2.1589999999999998},{"interface":"front","average":{"1min":1.274,"5min":1.248,"15min":1.226},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.6479999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.4299999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.2170000000000001,"15min":1.177},"min":{"1min":0.748,"5min":0.625,"15min":0.625},"max":{"1min":1.389,"5min":2.048,"15min":2.048},"last":2.444},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.262,"15min":1.2669999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":2.4039999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3819999999999999,"15min":1.3580000000000001},"min":{"1min":0.625,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":2.032},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2589999999999999,"15min":1.2250000000000001},"min":{"1min":0.71899999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.5289999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.391,"15min":1.3919999999999999},"min":{"1min":0.93500000000000005,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5900000000000001,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":2.3540000000000001},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4910000000000001,"15min":1.5189999999999999},"min":{"1min":0.83099999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.944,"5min":2.794,"15min":2.794},"last":2.3140000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":430,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194882,"statfs":{"total":200000000,"available":199559364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.345,"15min":1.2330000000000001},"min":{"1min":0.71999999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2480000000000002,"5min":9.3539999999999992,"15min":9.3539999999999992},"last":1.163},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3260000000000001,"15min":1.304},"min":{"1min":0.77000000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.8260000000000001,"5min":5.3029999999999999,"15min":5.3029999999999999},"last":1.2410000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.194,"15min":1.218},"min":{"1min":0.85099999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.1499999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.272,"5min":1.1910000000000001,"15min":1.2370000000000001},"min":{"1min":0.78500000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.2410000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":1.018}]},{"osd":2,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2450000000000001,"15min":1.26},"min":{"1min":0.88900000000000001,"5min":0.755,"15min":0.755},"max":{"1min":2.0350000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.288},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.2629999999999999,"15min":1.284},"min":{"1min":0.95499999999999996,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.153,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.75900000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2230000000000001,"15min":1.2370000000000001},"min":{"1min":0.86899999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.613,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3440000000000001,"15min":1.3340000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.5}]},{"osd":4,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.393,"15min":1.4119999999999999},"min":{"1min":0.93899999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.4449999999999998,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3919999999999999,"15min":1.498},"min":{"1min":0.91700000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.4049999999999998,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":1.135}]},{"osd":6,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2989999999999999,"15min":1.3859999999999999},"min":{"1min":0.91900000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.72,"5min":5.141,"15min":5.141},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.508,"15min":1.571},"min":{"1min":0.95299999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":1.609}]},{"osd":7,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.55,"15min":1.6399999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.5819999999999999,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":1.468},{"interface":"front","average":{"1min":1.385,"5min":1.4610000000000001,"15min":1.5069999999999999},"min":{"1min":1.091,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8240000000000001,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":1.6419999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":429,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194883,"statfs":{"total":200000000,"available":199560383,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.80900000000000005,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7669999999999999,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.125,"15min":1.1679999999999999},"min":{"1min":0.71199999999999997,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.7749999999999999,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.6439999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.089,"15min":1.0920000000000001},"min":{"1min":0.749,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4990000000000001,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.23,"5min":1.1930000000000001,"15min":1.2290000000000001},"min":{"1min":0.70399999999999996,"5min":0.627,"15min":0.627},"max":{"1min":1.3740000000000001,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.3149999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1890000000000001,"15min":1.179},"min":{"1min":0.81999999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4350000000000001,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.383,"5min":1.2709999999999999,"15min":1.3029999999999999},"min":{"1min":0.85899999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6930000000000001,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.5880000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.89700000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9179999999999999,"5min":2.649,"15min":2.649},"last":1.522},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.403,"15min":1.409},"min":{"1min":0.86799999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6399999999999999,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.0640000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.319,"15min":1.3029999999999999},"min":{"1min":0.96399999999999997,"5min":0.498,"15min":0.498},"max":{"1min":1.7649999999999999,"5min":1.923,"15min":1.923},"last":1.7649999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.3180000000000001,"15min":1.3560000000000001},"min":{"1min":0.92100000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.0720000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":2.04}]},{"osd":6,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4379999999999999,"15min":1.3959999999999999},"min":{"1min":0.96999999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.1150000000000002,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":2.1150000000000002},{"interface":"front","average":{"1min":1.454,"5min":1.508,"15min":1.5580000000000001},"min":{"1min":1.2250000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6839999999999999,"5min":4.407,"15min":4.407},"last":1.6839999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.46,"15min":1.502},"min":{"1min":1.0169999999999999,"5min":0.438,"15min":0.438},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.8700000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.54,"15min":1.546},"min":{"1min":1.196,"5min":0.747,"15min":0.747},"max":{"1min":1.7270000000000001,"5min":2.585,"15min":2.585},"last":1.3720000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":435,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194876,"statfs":{"total":200000000,"available":199553547,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.90600000000000003,"15min":0.79200000000000004},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2969999999999999,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.012,"15min":0.89300000000000002},"min":{"1min":0.48999999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.524,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":0.78600000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0309999999999999,"15min":0.95899999999999996},"min":{"1min":0.60099999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.633,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.2250000000000001,"15min":1.141},"min":{"1min":0.622,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.786,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.95699999999999996}]},{"osd":2,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.079,"15min":1.0229999999999999},"min":{"1min":0.55200000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.675,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.163,"15min":1.073},"min":{"1min":0.59899999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.708,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":1.1040000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.5029999999999999,"15min":1.3180000000000001},"min":{"1min":0.91000000000000003,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.593,"5min":18.664999999999999,"15min":18.664999999999999},"last":1.016},{"interface":"front","average":{"1min":1.248,"5min":1.274,"15min":1.1719999999999999},"min":{"1min":0.93899999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.746,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":1.46}]},{"osd":5,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.3129999999999999,"15min":1.2290000000000001},"min":{"1min":1.014,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5700000000000001,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":1.042},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.5700000000000001,"15min":1.363},"min":{"1min":0.78300000000000003,"5min":0.748,"15min":0.748},"max":{"1min":1.9179999999999999,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.5329999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.5209999999999999,"15min":1.4350000000000001},"min":{"1min":1.115,"5min":0.625,"15min":0.625},"max":{"1min":1.7949999999999999,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":1.585},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4650000000000001,"15min":1.353},"min":{"1min":1.1339999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.772,"5min":7.242,"15min":7.242},"last":1.2569999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4870000000000001,"15min":1.361},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9950000000000001,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.6060000000000001,"15min":1.464},"min":{"1min":1.014,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0329999999999999,"5min":12.946,"15min":12.946},"last":1.415}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":430,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194882,"statfs":{"total":200000000,"available":199559364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.87,"15min":0.92400000000000004},"min":{"1min":0.48599999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93400000000000005,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.96599999999999997,"15min":1.038},"min":{"1min":0.623,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3560000000000001,"5min":1.651,"15min":1.651},"last":0.55900000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.996,"15min":1.0449999999999999},"min":{"1min":0.67500000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3120000000000001,"5min":4.29,"15min":4.29},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.1020000000000001,"15min":1.1040000000000001},"min":{"1min":0.75,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5820000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.70799999999999996}]},{"osd":3,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.0940000000000001,"15min":1.129},"min":{"1min":0.72599999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8109999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.345,"15min":1.4650000000000001},"min":{"1min":0.73399999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.7929999999999999,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":1.0329999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.403,"15min":1.48},"min":{"1min":0.95299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.6479999999999999,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":1.069},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.407,"15min":1.4470000000000001},"min":{"1min":1.1319999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.766,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.448}]},{"osd":5,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.244,"15min":1.2709999999999999},"min":{"1min":1.119,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.786,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.446,"15min":1.5580000000000001},"min":{"1min":1.204,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.794,"5min":7.657,"15min":7.657},"last":1.151}]},{"osd":6,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.4430000000000001,"15min":1.5209999999999999},"min":{"1min":1.2070000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0649999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.47},{"interface":"front","average":{"1min":1.51,"5min":1.8420000000000001,"15min":1.5780000000000001},"min":{"1min":1.0069999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.9350000000000001,"5min":37.771999999999998,"15min":37.771999999999998},"last":1.49}]},{"osd":7,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.506,"15min":1.5649999999999999},"min":{"1min":1.1890000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.1930000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.587,"5min":1.4670000000000001,"15min":1.48},"min":{"1min":1.056,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1139999999999999,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.2290000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":434,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194877,"statfs":{"total":200000000,"available":199554566,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.81599999999999995,"15min":0.80600000000000005},"min":{"1min":0.52800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":1.1990000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.92900000000000005,"15min":0.90300000000000002},"min":{"1min":0.72699999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.2470000000000001}]},{"osd":2,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":0.93400000000000005,"15min":0.93000000000000005},"min":{"1min":0.51700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6060000000000001,"5min":1.859,"15min":1.859},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.86799999999999999,"15min":0.84699999999999998},"min":{"1min":0.58399999999999996,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.355,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.391}]},{"osd":3,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.008,"15min":1.016},"min":{"1min":0.68500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.407,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.45},{"interface":"front","average":{"1min":1.159,"5min":1.0429999999999999,"15min":1.054},"min":{"1min":0.63700000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.6299999999999999,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":1.4690000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.252,"15min":1.246},"min":{"1min":1.095,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.2370000000000001,"15min":1.2070000000000001},"min":{"1min":0.90900000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.903,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.708}]},{"osd":5,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.296,"15min":1.29},"min":{"1min":0.874,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.948,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":2.161},{"interface":"front","average":{"1min":1.504,"5min":1.375,"15min":1.3540000000000001},"min":{"1min":0.95599999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.964,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.7549999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.294,"15min":1.2789999999999999},"min":{"1min":0.84699999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.0230000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.504},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.3,"15min":1.266},"min":{"1min":0.95499999999999996,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":1.6120000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.3819999999999999,"15min":1.355},"min":{"1min":1.022,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.992},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.3819999999999999,"15min":1.3540000000000001},"min":{"1min":1.1379999999999999,"5min":0.746,"15min":0.746},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":2.0569999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1005,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198970015,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.86599999999999999,"15min":0.92600000000000005},"min":{"1min":0.437,"5min":0.37,"15min":0.37},"max":{"1min":1.163,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":1.026},{"interface":"front","average":{"1min":1.048,"5min":1.0580000000000001,"15min":1.0760000000000001},"min":{"1min":0.58699999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.514,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.6379999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.83599999999999997,"15min":0.84199999999999997},"min":{"1min":0.56000000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.262,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.1140000000000001,"15min":1.1220000000000001},"min":{"1min":0.52500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.462,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.3700000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1830000000000001,"15min":1.2929999999999999},"min":{"1min":0.67500000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6919999999999999,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.431,"15min":1.516},"min":{"1min":0.75800000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.8560000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.7330000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.2290000000000001,"15min":1.3049999999999999},"min":{"1min":0.78100000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.6519999999999999,"5min":4.492,"15min":4.492},"last":1.857},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.95699999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6339999999999999,"5min":1.726,"15min":1.726},"last":1.4770000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.3129999999999999,"15min":1.4279999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.53,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.268},{"interface":"front","average":{"1min":1.27,"5min":1.3320000000000001,"15min":1.377},"min":{"1min":0.88900000000000001,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6559999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.5820000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.5169999999999999,"15min":1.595},"min":{"1min":1.0640000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.6819999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.796},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.4259999999999999,"15min":1.5049999999999999},"min":{"1min":0.92400000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.7869999999999999,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.532}]},{"osd":7,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.579,"15min":1.6479999999999999},"min":{"1min":1.004,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.7909999999999999,"5min":3.492,"15min":3.492},"last":1.9279999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.556,"15min":1.6910000000000001},"min":{"1min":0.94599999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.8660000000000001,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.681}]}]}],"pool_statfs":[]}} 2023-10-18T09:00:20.251 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:00:20.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.522+0000 7f82d2b3c640 1 -- 172.21.15.40:0/3581292242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b8005180 msgr2=0x7f82b800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:20.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.522+0000 7f82d2b3c640 1 --2- 172.21.15.40:0/3581292242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b8005180 0x7f82b800b980 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f82c4005fd0 tx=0x7f82b80057f0 comp rx=0 tx=0).stop 2023-10-18T09:00:20.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.523+0000 7f82d2b3c640 1 -- 172.21.15.40:0/3581292242 shutdown_connections 2023-10-18T09:00:20.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.523+0000 7f82d2b3c640 1 --2- 172.21.15.40:0/3581292242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b8005180 0x7f82b800b980 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:20.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.523+0000 7f82d2b3c640 1 -- 172.21.15.40:0/3581292242 >> 172.21.15.40:0/3581292242 conn(0x7f82cc069720 msgr2=0x7f82cc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:20.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.523+0000 7f82d2b3c640 1 -- 172.21.15.40:0/3581292242 shutdown_connections 2023-10-18T09:00:20.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.523+0000 7f82d2b3c640 1 -- 172.21.15.40:0/3581292242 wait complete. 2023-10-18T09:00:20.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.524+0000 7f82d2b3c640 1 Processor -- start 2023-10-18T09:00:20.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.525+0000 7f82d2b3c640 1 -- start start 2023-10-18T09:00:20.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.525+0000 7f82d2b3c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82b8005180 0x7f82cc0ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:20.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.525+0000 7f82d2b3c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82cc149a50 0x7f82cc14ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:20.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.526+0000 7f82d2b3c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82cc14b320 0x7f82cc1e1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:20.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.526+0000 7f82d2b3c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc0ae340 con 0x7f82cc149a50 2023-10-18T09:00:20.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.526+0000 7f82d2b3c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc14cb00 con 0x7f82cc14b320 2023-10-18T09:00:20.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.526+0000 7f82d2b3c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc14b7c0 con 0x7f82b8005180 2023-10-18T09:00:20.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.526+0000 7f82d1339640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82cc149a50 0x7f82cc14ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:20.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.526+0000 7f82d1339640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82cc149a50 0x7f82cc14ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48466/0 (socket says 172.21.15.40:48466) 2023-10-18T09:00:20.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.526+0000 7f82d1339640 1 -- 172.21.15.40:0/791480062 learned_addr learned my addr 172.21.15.40:0/791480062 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:20.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.526+0000 7f82d233b640 1 --2- 172.21.15.40:0/791480062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82cc14b320 0x7f82cc1e1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:20.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.526+0000 7f82d1339640 1 -- 172.21.15.40:0/791480062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82b8005180 msgr2=0x7f82cc0ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:20.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.526+0000 7f82d1b3a640 1 --2- 172.21.15.40:0/791480062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82b8005180 0x7f82cc0ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:20.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.526+0000 7f82d1339640 1 --2- 172.21.15.40:0/791480062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82b8005180 0x7f82cc0ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:20.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.526+0000 7f82d1339640 1 -- 172.21.15.40:0/791480062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82cc14b320 msgr2=0x7f82cc1e1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:20.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.526+0000 7f82d1339640 1 --2- 172.21.15.40:0/791480062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82cc14b320 0x7f82cc1e1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:20.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.526+0000 7f82d1339640 1 -- 172.21.15.40:0/791480062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc14baa0 con 0x7f82cc149a50 2023-10-18T09:00:20.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.526+0000 7f82d233b640 1 --2- 172.21.15.40:0/791480062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82cc14b320 0x7f82cc1e1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:20.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.526+0000 7f82d1b3a640 1 --2- 172.21.15.40:0/791480062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82b8005180 0x7f82cc0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:20.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.527+0000 7f82d1339640 1 --2- 172.21.15.40:0/791480062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82cc149a50 0x7f82cc14ade0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c760 tx=0x7f82bc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:20.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.527+0000 7f82c27fc640 1 -- 172.21.15.40:0/791480062 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc005270 con 0x7f82cc149a50 2023-10-18T09:00:20.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.527+0000 7f82c27fc640 1 -- 172.21.15.40:0/791480062 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f82bc0035f0 con 0x7f82cc149a50 2023-10-18T09:00:20.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.527+0000 7f82c27fc640 1 -- 172.21.15.40:0/791480062 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc003a00 con 0x7f82cc149a50 2023-10-18T09:00:20.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.527+0000 7f82d2b3c640 1 -- 172.21.15.40:0/791480062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82cc14bd80 con 0x7f82cc149a50 2023-10-18T09:00:20.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.527+0000 7f82d2b3c640 1 -- 172.21.15.40:0/791480062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82cc149ed0 con 0x7f82cc149a50 2023-10-18T09:00:20.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.528+0000 7f82d2b3c640 1 -- 172.21.15.40:0/791480062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82cc066e00 con 0x7f82cc149a50 2023-10-18T09:00:20.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.529+0000 7f82c27fc640 1 -- 172.21.15.40:0/791480062 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f82bc003b60 con 0x7f82cc149a50 2023-10-18T09:00:20.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.530+0000 7f82c27fc640 1 --2- 172.21.15.40:0/791480062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f82b006ae40 0x7f82b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:20.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.530+0000 7f82c27fc640 1 -- 172.21.15.40:0/791480062 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82bc089070 con 0x7f82cc149a50 2023-10-18T09:00:20.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.531+0000 7f82d1b3a640 1 --2- 172.21.15.40:0/791480062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f82b006ae40 0x7f82b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:20.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.532+0000 7f82d1b3a640 1 --2- 172.21.15.40:0/791480062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f82b006ae40 0x7f82b006d300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f82c8000f20 tx=0x7f82c8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:20.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.534+0000 7f82c27fc640 1 -- 172.21.15.40:0/791480062 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82bc0bbca0 con 0x7f82cc149a50 2023-10-18T09:00:20.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.726+0000 7f82d2b3c640 1 -- 172.21.15.40:0/791480062 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82cc0b2270 con 0x7f82b006ae40 2023-10-18T09:00:20.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.748+0000 7f82c27fc640 1 -- 172.21.15.40:0/791480062 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+273449 (secure 0 0 0) 0x7f82c800d4a0 con 0x7f82b006ae40 2023-10-18T09:00:20.754 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:20.756 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:00:20.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.753+0000 7f82d2b3c640 1 -- 172.21.15.40:0/791480062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f82b006ae40 msgr2=0x7f82b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:20.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.753+0000 7f82d2b3c640 1 --2- 172.21.15.40:0/791480062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f82b006ae40 0x7f82b006d300 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f82c8000f20 tx=0x7f82c8009040 comp rx=0 tx=0).stop 2023-10-18T09:00:20.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.753+0000 7f82d2b3c640 1 -- 172.21.15.40:0/791480062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82cc149a50 msgr2=0x7f82cc14ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:20.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.753+0000 7f82d2b3c640 1 --2- 172.21.15.40:0/791480062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82cc149a50 0x7f82cc14ade0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c760 tx=0x7f82bc00cc20 comp rx=0 tx=0).stop 2023-10-18T09:00:20.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.753+0000 7f82d2b3c640 1 -- 172.21.15.40:0/791480062 shutdown_connections 2023-10-18T09:00:20.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.753+0000 7f82d2b3c640 1 --2- 172.21.15.40:0/791480062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f82b006ae40 0x7f82b006d300 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:20.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.753+0000 7f82d2b3c640 1 --2- 172.21.15.40:0/791480062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82cc14b320 0x7f82cc1e1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:20.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.753+0000 7f82d2b3c640 1 --2- 172.21.15.40:0/791480062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82cc149a50 0x7f82cc14ade0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:20.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.753+0000 7f82d2b3c640 1 --2- 172.21.15.40:0/791480062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82b8005180 0x7f82cc0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:20.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.753+0000 7f82d2b3c640 1 -- 172.21.15.40:0/791480062 >> 172.21.15.40:0/791480062 conn(0x7f82cc069720 msgr2=0x7f82cc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:20.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.753+0000 7f82d2b3c640 1 -- 172.21.15.40:0/791480062 shutdown_connections 2023-10-18T09:00:20.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:20.753+0000 7f82d2b3c640 1 -- 172.21.15.40:0/791480062 wait complete. 2023-10-18T09:00:20.777 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2023-10-18T09:00:19.004899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":102,"acting":102,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557890,"statfs":{"total":1600000000,"available":1595279782,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.310184"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.708745+0000","last_change":"2023-10-18T09:00:14.708745+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.708745+0000","last_undegraded":"2023-10-18T09:00:14.708745+0000","last_fullsized":"2023-10-18T09:00:14.708745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.709438+0000","last_change":"2023-10-18T09:00:14.709438+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.709438+0000","last_undegraded":"2023-10-18T09:00:14.709438+0000","last_fullsized":"2023-10-18T09:00:14.709438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.707926+0000","last_change":"2023-10-18T09:00:14.707926+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.707926+0000","last_undegraded":"2023-10-18T09:00:14.707926+0000","last_fullsized":"2023-10-18T09:00:14.707926+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.703500+0000","last_change":"2023-10-18T09:00:14.703500+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.703500+0000","last_undegraded":"2023-10-18T09:00:14.703500+0000","last_fullsized":"2023-10-18T09:00:14.703500+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.711597+0000","last_change":"2023-10-18T09:00:14.711597+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.711597+0000","last_undegraded":"2023-10-18T09:00:14.711597+0000","last_fullsized":"2023-10-18T09:00:14.711597+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.709270+0000","last_change":"2023-10-18T09:00:14.709270+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.709270+0000","last_undegraded":"2023-10-18T09:00:14.709270+0000","last_fullsized":"2023-10-18T09:00:14.709270+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.709112+0000","last_change":"2023-10-18T09:00:14.709112+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.709112+0000","last_undegraded":"2023-10-18T09:00:14.709112+0000","last_fullsized":"2023-10-18T09:00:14.709112+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.710603+0000","last_change":"2023-10-18T09:00:14.710603+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.710603+0000","last_undegraded":"2023-10-18T09:00:14.710603+0000","last_fullsized":"2023-10-18T09:00:14.710603+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-18T09:00:14.722080+0000","last_change":"2023-10-18T09:00:14.719832+0000","last_active":"2023-10-18T09:00:14.722080+0000","last_peered":"2023-10-18T09:00:14.722080+0000","last_clean":"2023-10-18T09:00:14.722080+0000","last_became_active":"2023-10-18T09:00:14.719074+0000","last_became_peered":"2023-10-18T09:00:14.719074+0000","last_unstale":"2023-10-18T09:00:14.722080+0000","last_undegraded":"2023-10-18T09:00:14.722080+0000","last_fullsized":"2023-10-18T09:00:14.722080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:21.838152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.698383+0000","last_change":"2023-10-18T09:00:13.698383+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.698383+0000","last_undegraded":"2023-10-18T09:00:13.698383+0000","last_fullsized":"2023-10-18T09:00:13.698383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.700897+0000","last_change":"2023-10-18T09:00:13.700897+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.700897+0000","last_undegraded":"2023-10-18T09:00:13.700897+0000","last_fullsized":"2023-10-18T09:00:13.700897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.705500+0000","last_change":"2023-10-18T09:00:14.705500+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.705500+0000","last_undegraded":"2023-10-18T09:00:14.705500+0000","last_fullsized":"2023-10-18T09:00:14.705500+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.705976+0000","last_change":"2023-10-18T09:00:13.705976+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.705976+0000","last_undegraded":"2023-10-18T09:00:13.705976+0000","last_fullsized":"2023-10-18T09:00:13.705976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-18T09:00:14.721125+0000","last_change":"2023-10-18T09:00:14.720994+0000","last_active":"2023-10-18T09:00:14.721125+0000","last_peered":"2023-10-18T09:00:14.721125+0000","last_clean":"2023-10-18T09:00:14.721125+0000","last_became_active":"2023-10-18T09:00:14.720661+0000","last_became_peered":"2023-10-18T09:00:14.720661+0000","last_unstale":"2023-10-18T09:00:14.721125+0000","last_undegraded":"2023-10-18T09:00:14.721125+0000","last_fullsized":"2023-10-18T09:00:14.721125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:03:33.089526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.706980+0000","last_change":"2023-10-18T09:00:13.706980+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.706980+0000","last_undegraded":"2023-10-18T09:00:13.706980+0000","last_fullsized":"2023-10-18T09:00:13.706980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.698989+0000","last_change":"2023-10-18T09:00:13.698989+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.698989+0000","last_undegraded":"2023-10-18T09:00:13.698989+0000","last_fullsized":"2023-10-18T09:00:13.698989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.696228+0000","last_change":"2023-10-18T09:00:13.696228+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.696228+0000","last_undegraded":"2023-10-18T09:00:13.696228+0000","last_fullsized":"2023-10-18T09:00:13.696228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.699199+0000","last_change":"2023-10-18T09:00:13.699199+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.699199+0000","last_undegraded":"2023-10-18T09:00:13.699199+0000","last_fullsized":"2023-10-18T09:00:13.699199+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.707158+0000","last_change":"2023-10-18T09:00:14.707158+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.707158+0000","last_undegraded":"2023-10-18T09:00:14.707158+0000","last_fullsized":"2023-10-18T09:00:14.707158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.706160+0000","last_change":"2023-10-18T09:00:13.706160+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.706160+0000","last_undegraded":"2023-10-18T09:00:13.706160+0000","last_fullsized":"2023-10-18T09:00:13.706160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-18T09:00:14.723533+0000","last_change":"2023-10-18T09:00:14.723428+0000","last_active":"2023-10-18T09:00:14.723533+0000","last_peered":"2023-10-18T09:00:14.723533+0000","last_clean":"2023-10-18T09:00:14.723533+0000","last_became_active":"2023-10-18T09:00:14.723174+0000","last_became_peered":"2023-10-18T09:00:14.723174+0000","last_unstale":"2023-10-18T09:00:14.723533+0000","last_undegraded":"2023-10-18T09:00:14.723533+0000","last_fullsized":"2023-10-18T09:00:14.723533+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:09:08.852692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.698316+0000","last_change":"2023-10-18T09:00:13.698316+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.698316+0000","last_undegraded":"2023-10-18T09:00:13.698316+0000","last_fullsized":"2023-10-18T09:00:13.698316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.706323+0000","last_change":"2023-10-18T09:00:14.706323+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.706323+0000","last_undegraded":"2023-10-18T09:00:14.706323+0000","last_fullsized":"2023-10-18T09:00:14.706323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.697251+0000","last_change":"2023-10-18T09:00:13.697251+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.697251+0000","last_undegraded":"2023-10-18T09:00:13.697251+0000","last_fullsized":"2023-10-18T09:00:13.697251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.705085+0000","last_change":"2023-10-18T09:00:13.705085+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.705085+0000","last_undegraded":"2023-10-18T09:00:13.705085+0000","last_fullsized":"2023-10-18T09:00:13.705085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.700521+0000","last_change":"2023-10-18T09:00:13.700521+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.700521+0000","last_undegraded":"2023-10-18T09:00:13.700521+0000","last_fullsized":"2023-10-18T09:00:13.700521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.701145+0000","last_change":"2023-10-18T09:00:13.701145+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.701145+0000","last_undegraded":"2023-10-18T09:00:13.701145+0000","last_fullsized":"2023-10-18T09:00:13.701145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.706157+0000","last_change":"2023-10-18T09:00:13.706157+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.706157+0000","last_undegraded":"2023-10-18T09:00:13.706157+0000","last_fullsized":"2023-10-18T09:00:13.706157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.709724+0000","last_change":"2023-10-18T09:00:14.709724+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.709724+0000","last_undegraded":"2023-10-18T09:00:14.709724+0000","last_fullsized":"2023-10-18T09:00:14.709724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.699049+0000","last_change":"2023-10-18T09:00:13.699049+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.699049+0000","last_undegraded":"2023-10-18T09:00:13.699049+0000","last_fullsized":"2023-10-18T09:00:13.699049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.704226+0000","last_change":"2023-10-18T09:00:13.704226+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.704226+0000","last_undegraded":"2023-10-18T09:00:13.704226+0000","last_fullsized":"2023-10-18T09:00:13.704226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-18T09:00:14.724961+0000","last_change":"2023-10-18T09:00:14.723114+0000","last_active":"2023-10-18T09:00:14.724961+0000","last_peered":"2023-10-18T09:00:14.724961+0000","last_clean":"2023-10-18T09:00:14.724961+0000","last_became_active":"2023-10-18T09:00:14.722764+0000","last_became_peered":"2023-10-18T09:00:14.722764+0000","last_unstale":"2023-10-18T09:00:14.724961+0000","last_undegraded":"2023-10-18T09:00:14.724961+0000","last_fullsized":"2023-10-18T09:00:14.724961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:05.813755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.708813+0000","last_change":"2023-10-18T09:00:14.708813+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.708813+0000","last_undegraded":"2023-10-18T09:00:14.708813+0000","last_fullsized":"2023-10-18T09:00:14.708813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.705311+0000","last_change":"2023-10-18T09:00:13.705311+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.705311+0000","last_undegraded":"2023-10-18T09:00:13.705311+0000","last_fullsized":"2023-10-18T09:00:13.705311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.708614+0000","last_change":"2023-10-18T09:00:14.708614+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.708614+0000","last_undegraded":"2023-10-18T09:00:14.708614+0000","last_fullsized":"2023-10-18T09:00:14.708614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.698536+0000","last_change":"2023-10-18T09:00:13.698536+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.698536+0000","last_undegraded":"2023-10-18T09:00:13.698536+0000","last_fullsized":"2023-10-18T09:00:13.698536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.708619+0000","last_change":"2023-10-18T09:00:14.708619+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.708619+0000","last_undegraded":"2023-10-18T09:00:14.708619+0000","last_fullsized":"2023-10-18T09:00:14.708619+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.697743+0000","last_change":"2023-10-18T09:00:13.697743+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.697743+0000","last_undegraded":"2023-10-18T09:00:13.697743+0000","last_fullsized":"2023-10-18T09:00:13.697743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.705179+0000","last_change":"2023-10-18T09:00:13.705179+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.705179+0000","last_undegraded":"2023-10-18T09:00:13.705179+0000","last_fullsized":"2023-10-18T09:00:13.705179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.698859+0000","last_change":"2023-10-18T09:00:13.698859+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.698859+0000","last_undegraded":"2023-10-18T09:00:13.698859+0000","last_fullsized":"2023-10-18T09:00:13.698859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.704694+0000","last_change":"2023-10-18T09:00:14.704694+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.704694+0000","last_undegraded":"2023-10-18T09:00:14.704694+0000","last_fullsized":"2023-10-18T09:00:14.704694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.696051+0000","last_change":"2023-10-18T09:00:13.696051+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.696051+0000","last_undegraded":"2023-10-18T09:00:13.696051+0000","last_fullsized":"2023-10-18T09:00:13.696051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.706477+0000","last_change":"2023-10-18T09:00:14.706477+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.706477+0000","last_undegraded":"2023-10-18T09:00:14.706477+0000","last_fullsized":"2023-10-18T09:00:14.706477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.697921+0000","last_change":"2023-10-18T09:00:13.697921+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.697921+0000","last_undegraded":"2023-10-18T09:00:13.697921+0000","last_fullsized":"2023-10-18T09:00:13.697921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.696049+0000","last_change":"2023-10-18T09:00:13.696049+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.696049+0000","last_undegraded":"2023-10-18T09:00:13.696049+0000","last_fullsized":"2023-10-18T09:00:13.696049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.697231+0000","last_change":"2023-10-18T09:00:13.697231+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.697231+0000","last_undegraded":"2023-10-18T09:00:13.697231+0000","last_fullsized":"2023-10-18T09:00:13.697231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.700302+0000","last_change":"2023-10-18T09:00:13.700302+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.700302+0000","last_undegraded":"2023-10-18T09:00:13.700302+0000","last_fullsized":"2023-10-18T09:00:13.700302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:00:14.691350+0000","last_change":"2023-10-18T09:00:14.691350+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.691350+0000","last_undegraded":"2023-10-18T09:00:14.691350+0000","last_fullsized":"2023-10-18T09:00:14.691350+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.704099+0000","last_change":"2023-10-18T09:00:13.704099+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.704099+0000","last_undegraded":"2023-10-18T09:00:13.704099+0000","last_fullsized":"2023-10-18T09:00:13.704099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2023-10-18T09:00:14.708806+0000","last_change":"2023-10-18T09:00:14.708806+0000","last_active":"2023-10-18T09:00:14.691350+0000","last_peered":"2023-10-18T09:00:14.691350+0000","last_clean":"2023-10-18T09:00:14.691350+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:14.708806+0000","last_undegraded":"2023-10-18T09:00:14.708806+0000","last_fullsized":"2023-10-18T09:00:14.708806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2023-10-18T09:00:13.706147+0000","last_change":"2023-10-18T09:00:13.706147+0000","last_active":"2023-10-18T09:00:13.678464+0000","last_peered":"2023-10-18T09:00:13.678464+0000","last_clean":"2023-10-18T09:00:13.678464+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:00:13.706147+0000","last_undegraded":"2023-10-18T09:00:13.706147+0000","last_fullsized":"2023-10-18T09:00:13.706147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":236,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-10-18T09:00:13.696161+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:00:13.696161+0000","last_peered":"2023-10-18T09:00:13.696161+0000","last_clean":"2023-10-18T09:00:13.696161+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:00:13.696161+0000","last_undegraded":"2023-10-18T09:00:13.696161+0000","last_fullsized":"2023-10-18T09:00:13.696161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:36.150056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":36,"acting":36,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1012,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194299,"statfs":{"total":200000000,"available":198963179,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.071,"15min":1.0149999999999999},"min":{"1min":0.72599999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.575,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.165,"5min":1.1000000000000001,"15min":1.03},"min":{"1min":0.84399999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.536,"5min":1.97,"15min":1.97},"last":1.5740000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.125,"15min":1.147},"min":{"1min":0.88200000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5169999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1659999999999999,"15min":1.1299999999999999},"min":{"1min":0.91800000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.083}]},{"osd":2,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.284,"15min":1.246},"min":{"1min":0.78600000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.522,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.524},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.236,"15min":1.2030000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.637,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.202}]},{"osd":3,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.2609999999999999,"15min":1.2629999999999999},"min":{"1min":0.80300000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.845,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3500000000000001,"15min":1.327},"min":{"1min":1.0069999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.736,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.389}]},{"osd":4,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.238,"15min":1.1759999999999999},"min":{"1min":0.94599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.825,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.393,"5min":1.351,"15min":1.3919999999999999},"min":{"1min":1.101,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.627,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.131}]},{"osd":5,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3580000000000001,"15min":1.355},"min":{"1min":0.89600000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7470000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.415,"15min":1.421},"min":{"1min":1.079,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7130000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.45}]},{"osd":6,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.482,"15min":1.464},"min":{"1min":1.355,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.8169999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.306},{"interface":"front","average":{"1min":1.512,"5min":1.484,"15min":1.5069999999999999},"min":{"1min":1.214,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.768,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.421}]}]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":430,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194882,"statfs":{"total":200000000,"available":199559364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.1759999999999999,"15min":1.105},"min":{"1min":0.71799999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.627,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.2709999999999999,"15min":1.248},"min":{"1min":0.86699999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7629999999999999,"5min":1.855,"15min":1.855},"last":2.0990000000000002}]},{"osd":1,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1040000000000001,"15min":1.0900000000000001},"min":{"1min":0.68500000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.488,"5min":1.633,"15min":1.633},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.198,"15min":1.1919999999999999},"min":{"1min":0.72599999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5409999999999999,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":1.8560000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.879,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5720000000000001,"5min":1.696,"15min":1.696},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2170000000000001,"15min":1.2010000000000001},"min":{"1min":0.90000000000000002,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.667,"5min":2.137,"15min":2.137},"last":1.032}]},{"osd":3,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.256,"15min":1.2589999999999999},"min":{"1min":0.91900000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.6719999999999999,"5min":1.784,"15min":1.784},"last":2.1589999999999998},{"interface":"front","average":{"1min":1.274,"5min":1.248,"15min":1.226},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.6479999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.4299999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.2170000000000001,"15min":1.177},"min":{"1min":0.748,"5min":0.625,"15min":0.625},"max":{"1min":1.389,"5min":2.048,"15min":2.048},"last":2.444},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.262,"15min":1.2669999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":2.4039999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3819999999999999,"15min":1.3580000000000001},"min":{"1min":0.625,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":2.032},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2589999999999999,"15min":1.2250000000000001},"min":{"1min":0.71899999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.5289999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.391,"15min":1.3919999999999999},"min":{"1min":0.93500000000000005,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5900000000000001,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":2.3540000000000001},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4910000000000001,"15min":1.5189999999999999},"min":{"1min":0.83099999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.944,"5min":2.794,"15min":2.794},"last":2.3140000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":430,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194882,"statfs":{"total":200000000,"available":199559364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.345,"15min":1.2330000000000001},"min":{"1min":0.71999999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2480000000000002,"5min":9.3539999999999992,"15min":9.3539999999999992},"last":1.163},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3260000000000001,"15min":1.304},"min":{"1min":0.77000000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.8260000000000001,"5min":5.3029999999999999,"15min":5.3029999999999999},"last":1.2410000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.194,"15min":1.218},"min":{"1min":0.85099999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.1499999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.272,"5min":1.1910000000000001,"15min":1.2370000000000001},"min":{"1min":0.78500000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.2410000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":1.018}]},{"osd":2,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2450000000000001,"15min":1.26},"min":{"1min":0.88900000000000001,"5min":0.755,"15min":0.755},"max":{"1min":2.0350000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.288},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.2629999999999999,"15min":1.284},"min":{"1min":0.95499999999999996,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.153,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.75900000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2230000000000001,"15min":1.2370000000000001},"min":{"1min":0.86899999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.613,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3440000000000001,"15min":1.3340000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.5}]},{"osd":4,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.393,"15min":1.4119999999999999},"min":{"1min":0.93899999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.4449999999999998,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3919999999999999,"15min":1.498},"min":{"1min":0.91700000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.4049999999999998,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":1.135}]},{"osd":6,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2989999999999999,"15min":1.3859999999999999},"min":{"1min":0.91900000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.72,"5min":5.141,"15min":5.141},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.508,"15min":1.571},"min":{"1min":0.95299999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":1.609}]},{"osd":7,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.55,"15min":1.6399999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.5819999999999999,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":1.468},{"interface":"front","average":{"1min":1.385,"5min":1.4610000000000001,"15min":1.5069999999999999},"min":{"1min":1.091,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8240000000000001,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":1.6419999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":429,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194883,"statfs":{"total":200000000,"available":199560383,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.80900000000000005,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7669999999999999,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.125,"15min":1.1679999999999999},"min":{"1min":0.71199999999999997,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.7749999999999999,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.6439999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.089,"15min":1.0920000000000001},"min":{"1min":0.749,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4990000000000001,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.23,"5min":1.1930000000000001,"15min":1.2290000000000001},"min":{"1min":0.70399999999999996,"5min":0.627,"15min":0.627},"max":{"1min":1.3740000000000001,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.3149999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1890000000000001,"15min":1.179},"min":{"1min":0.81999999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4350000000000001,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.383,"5min":1.2709999999999999,"15min":1.3029999999999999},"min":{"1min":0.85899999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6930000000000001,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.5880000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.89700000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9179999999999999,"5min":2.649,"15min":2.649},"last":1.522},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.403,"15min":1.409},"min":{"1min":0.86799999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6399999999999999,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.0640000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.319,"15min":1.3029999999999999},"min":{"1min":0.96399999999999997,"5min":0.498,"15min":0.498},"max":{"1min":1.7649999999999999,"5min":1.923,"15min":1.923},"last":1.7649999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.3180000000000001,"15min":1.3560000000000001},"min":{"1min":0.92100000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.0720000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":2.04}]},{"osd":6,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4379999999999999,"15min":1.3959999999999999},"min":{"1min":0.96999999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.1150000000000002,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":2.1150000000000002},{"interface":"front","average":{"1min":1.454,"5min":1.508,"15min":1.5580000000000001},"min":{"1min":1.2250000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6839999999999999,"5min":4.407,"15min":4.407},"last":1.6839999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.46,"15min":1.502},"min":{"1min":1.0169999999999999,"5min":0.438,"15min":0.438},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.8700000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.54,"15min":1.546},"min":{"1min":1.196,"5min":0.747,"15min":0.747},"max":{"1min":1.7270000000000001,"5min":2.585,"15min":2.585},"last":1.3720000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":435,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194876,"statfs":{"total":200000000,"available":199553547,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.90600000000000003,"15min":0.79200000000000004},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2969999999999999,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.012,"15min":0.89300000000000002},"min":{"1min":0.48999999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.524,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":0.78600000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0309999999999999,"15min":0.95899999999999996},"min":{"1min":0.60099999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.633,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.2250000000000001,"15min":1.141},"min":{"1min":0.622,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.786,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":0.95699999999999996}]},{"osd":2,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.079,"15min":1.0229999999999999},"min":{"1min":0.55200000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.675,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.163,"15min":1.073},"min":{"1min":0.59899999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.708,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":1.1040000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.5029999999999999,"15min":1.3180000000000001},"min":{"1min":0.91000000000000003,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.593,"5min":18.664999999999999,"15min":18.664999999999999},"last":1.016},{"interface":"front","average":{"1min":1.248,"5min":1.274,"15min":1.1719999999999999},"min":{"1min":0.93899999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.746,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":1.46}]},{"osd":5,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.3129999999999999,"15min":1.2290000000000001},"min":{"1min":1.014,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5700000000000001,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":1.042},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.5700000000000001,"15min":1.363},"min":{"1min":0.78300000000000003,"5min":0.748,"15min":0.748},"max":{"1min":1.9179999999999999,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.5329999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.5209999999999999,"15min":1.4350000000000001},"min":{"1min":1.115,"5min":0.625,"15min":0.625},"max":{"1min":1.7949999999999999,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":1.585},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4650000000000001,"15min":1.353},"min":{"1min":1.1339999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.772,"5min":7.242,"15min":7.242},"last":1.2569999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4870000000000001,"15min":1.361},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9950000000000001,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.6060000000000001,"15min":1.464},"min":{"1min":1.014,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0329999999999999,"5min":12.946,"15min":12.946},"last":1.415}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":430,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194882,"statfs":{"total":200000000,"available":199559364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.87,"15min":0.92400000000000004},"min":{"1min":0.48599999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93400000000000005,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.96599999999999997,"15min":1.038},"min":{"1min":0.623,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3560000000000001,"5min":1.651,"15min":1.651},"last":0.55900000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.996,"15min":1.0449999999999999},"min":{"1min":0.67500000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3120000000000001,"5min":4.29,"15min":4.29},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.1020000000000001,"15min":1.1040000000000001},"min":{"1min":0.75,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5820000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.70799999999999996}]},{"osd":3,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.0940000000000001,"15min":1.129},"min":{"1min":0.72599999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8109999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.345,"15min":1.4650000000000001},"min":{"1min":0.73399999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.7929999999999999,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":1.0329999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.403,"15min":1.48},"min":{"1min":0.95299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.6479999999999999,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":1.069},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.407,"15min":1.4470000000000001},"min":{"1min":1.1319999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.766,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.448}]},{"osd":5,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.244,"15min":1.2709999999999999},"min":{"1min":1.119,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.786,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.446,"15min":1.5580000000000001},"min":{"1min":1.204,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.794,"5min":7.657,"15min":7.657},"last":1.151}]},{"osd":6,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.4430000000000001,"15min":1.5209999999999999},"min":{"1min":1.2070000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0649999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.47},{"interface":"front","average":{"1min":1.51,"5min":1.8420000000000001,"15min":1.5780000000000001},"min":{"1min":1.0069999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.9350000000000001,"5min":37.771999999999998,"15min":37.771999999999998},"last":1.49}]},{"osd":7,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.506,"15min":1.5649999999999999},"min":{"1min":1.1890000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.1930000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.587,"5min":1.4670000000000001,"15min":1.48},"min":{"1min":1.056,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1139999999999999,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.2290000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":434,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194877,"statfs":{"total":200000000,"available":199554566,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.81599999999999995,"15min":0.80600000000000005},"min":{"1min":0.52800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":1.1990000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.92900000000000005,"15min":0.90300000000000002},"min":{"1min":0.72699999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.2470000000000001}]},{"osd":2,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":0.93400000000000005,"15min":0.93000000000000005},"min":{"1min":0.51700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6060000000000001,"5min":1.859,"15min":1.859},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.86799999999999999,"15min":0.84699999999999998},"min":{"1min":0.58399999999999996,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.355,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.391}]},{"osd":3,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.008,"15min":1.016},"min":{"1min":0.68500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.407,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.45},{"interface":"front","average":{"1min":1.159,"5min":1.0429999999999999,"15min":1.054},"min":{"1min":0.63700000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.6299999999999999,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":1.4690000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.252,"15min":1.246},"min":{"1min":1.095,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.2370000000000001,"15min":1.2070000000000001},"min":{"1min":0.90900000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.903,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.708}]},{"osd":5,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.296,"15min":1.29},"min":{"1min":0.874,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.948,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":2.161},{"interface":"front","average":{"1min":1.504,"5min":1.375,"15min":1.3540000000000001},"min":{"1min":0.95599999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.964,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.7549999999999999}]},{"osd":6,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.294,"15min":1.2789999999999999},"min":{"1min":0.84699999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.0230000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.504},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.3,"15min":1.266},"min":{"1min":0.95499999999999996,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":1.6120000000000001}]},{"osd":7,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.3819999999999999,"15min":1.355},"min":{"1min":1.022,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.992},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.3819999999999999,"15min":1.3540000000000001},"min":{"1min":1.1379999999999999,"5min":0.746,"15min":0.746},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":2.0569999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1005,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194306,"statfs":{"total":200000000,"available":198970015,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.86599999999999999,"15min":0.92600000000000005},"min":{"1min":0.437,"5min":0.37,"15min":0.37},"max":{"1min":1.163,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":1.026},{"interface":"front","average":{"1min":1.048,"5min":1.0580000000000001,"15min":1.0760000000000001},"min":{"1min":0.58699999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.514,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.6379999999999999}]},{"osd":2,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.83599999999999997,"15min":0.84199999999999997},"min":{"1min":0.56000000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.262,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.1140000000000001,"15min":1.1220000000000001},"min":{"1min":0.52500000000000002,"5min":0.437,"15min":0.437},"max":{"1min":1.462,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":1.3700000000000001}]},{"osd":3,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1830000000000001,"15min":1.2929999999999999},"min":{"1min":0.67500000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6919999999999999,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.431,"15min":1.516},"min":{"1min":0.75800000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.8560000000000001,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.7330000000000001}]},{"osd":4,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.2290000000000001,"15min":1.3049999999999999},"min":{"1min":0.78100000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.6519999999999999,"5min":4.492,"15min":4.492},"last":1.857},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.95699999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6339999999999999,"5min":1.726,"15min":1.726},"last":1.4770000000000001}]},{"osd":5,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.3129999999999999,"15min":1.4279999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.53,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.268},{"interface":"front","average":{"1min":1.27,"5min":1.3320000000000001,"15min":1.377},"min":{"1min":0.88900000000000001,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.6559999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.5820000000000001}]},{"osd":6,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.5169999999999999,"15min":1.595},"min":{"1min":1.0640000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.6819999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.796},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.4259999999999999,"15min":1.5049999999999999},"min":{"1min":0.92400000000000004,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.7869999999999999,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.532}]},{"osd":7,"last update":"Wed Oct 18 08:59:12 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.579,"15min":1.6479999999999999},"min":{"1min":1.004,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.7909999999999999,"5min":3.492,"15min":3.492},"last":1.9279999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.556,"15min":1.6910000000000001},"min":{"1min":0.94599999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.8660000000000001,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.681}]}]}],"pool_statfs":[]}} 2023-10-18T09:00:20.783 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:00:21.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.057+0000 7f0553d6e640 1 -- 172.21.15.40:0/588386186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05380039c0 msgr2=0x7f0538003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:21.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.057+0000 7f0553d6e640 1 --2- 172.21.15.40:0/588386186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05380039c0 0x7f0538003dc0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f054800c6e0 tx=0x7f05380057f0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.057+0000 7f0553d6e640 1 -- 172.21.15.40:0/588386186 shutdown_connections 2023-10-18T09:00:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.057+0000 7f0553d6e640 1 --2- 172.21.15.40:0/588386186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05380039c0 0x7f0538003dc0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.057+0000 7f0553d6e640 1 -- 172.21.15.40:0/588386186 >> 172.21.15.40:0/588386186 conn(0x7f054c06cee0 msgr2=0x7f054c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.057+0000 7f0553d6e640 1 -- 172.21.15.40:0/588386186 shutdown_connections 2023-10-18T09:00:21.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.057+0000 7f0553d6e640 1 -- 172.21.15.40:0/588386186 wait complete. 2023-10-18T09:00:21.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.059+0000 7f0553d6e640 1 Processor -- start 2023-10-18T09:00:21.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.060+0000 7f0553d6e640 1 -- start start 2023-10-18T09:00:21.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.061+0000 7f0553d6e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05380039c0 0x7f054c13ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:21.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.061+0000 7f0553d6e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f054c158520 0x7f054c158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:21.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.061+0000 7f0553d6e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c15a5e0 0x7f054c15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:21.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.061+0000 7f0553d6e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c1f2880 con 0x7f054c15a5e0 2023-10-18T09:00:21.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.061+0000 7f0553d6e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c158ec0 con 0x7f05380039c0 2023-10-18T09:00:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.061+0000 7f0553d6e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c13b1a0 con 0x7f054c158520 2023-10-18T09:00:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.061+0000 7f055230b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c15a5e0 0x7f054c15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.061+0000 7f055230b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c15a5e0 0x7f054c15aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48484/0 (socket says 172.21.15.40:48484) 2023-10-18T09:00:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.061+0000 7f055230b640 1 -- 172.21.15.40:0/1396077698 learned_addr learned my addr 172.21.15.40:0/1396077698 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.062+0000 7f0551b0a640 1 --2- 172.21.15.40:0/1396077698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05380039c0 0x7f054c13ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:21.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.062+0000 7f0551309640 1 --2- 172.21.15.40:0/1396077698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f054c158520 0x7f054c158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.062+0000 7f055230b640 1 -- 172.21.15.40:0/1396077698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f054c158520 msgr2=0x7f054c158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.062+0000 7f055230b640 1 --2- 172.21.15.40:0/1396077698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f054c158520 0x7f054c158980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.062+0000 7f055230b640 1 -- 172.21.15.40:0/1396077698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05380039c0 msgr2=0x7f054c13ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.062+0000 7f055230b640 1 --2- 172.21.15.40:0/1396077698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05380039c0 0x7f054c13ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.062+0000 7f055230b640 1 -- 172.21.15.40:0/1396077698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c1f1c00 con 0x7f054c15a5e0 2023-10-18T09:00:21.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.062+0000 7f055230b640 1 --2- 172.21.15.40:0/1396077698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c15a5e0 0x7f054c15aa60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0544001330 tx=0x7f054400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:21.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.063+0000 7f05427fc640 1 -- 172.21.15.40:0/1396077698 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544003d80 con 0x7f054c15a5e0 2023-10-18T09:00:21.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.063+0000 7f05427fc640 1 -- 172.21.15.40:0/1396077698 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0544005070 con 0x7f054c15a5e0 2023-10-18T09:00:21.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.063+0000 7f05427fc640 1 -- 172.21.15.40:0/1396077698 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544005480 con 0x7f054c15a5e0 2023-10-18T09:00:21.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.063+0000 7f0553d6e640 1 -- 172.21.15.40:0/1396077698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054c1f1ee0 con 0x7f054c15a5e0 2023-10-18T09:00:21.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.063+0000 7f0553d6e640 1 -- 172.21.15.40:0/1396077698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054c1f24a0 con 0x7f054c15a5e0 2023-10-18T09:00:21.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.064+0000 7f0553d6e640 1 -- 172.21.15.40:0/1396077698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054c1f29c0 con 0x7f054c15a5e0 2023-10-18T09:00:21.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.065+0000 7f05427fc640 1 -- 172.21.15.40:0/1396077698 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f05440055e0 con 0x7f054c15a5e0 2023-10-18T09:00:21.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.067+0000 7f05427fc640 1 --2- 172.21.15.40:0/1396077698 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f053006ae40 0x7f053006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:21.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.067+0000 7f05427fc640 1 -- 172.21.15.40:0/1396077698 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0544089080 con 0x7f054c15a5e0 2023-10-18T09:00:21.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.068+0000 7f0551b0a640 1 --2- 172.21.15.40:0/1396077698 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f053006ae40 0x7f053006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:21.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.069+0000 7f0551b0a640 1 --2- 172.21.15.40:0/1396077698 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f053006ae40 0x7f053006d300 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0548034000 tx=0x7f0548003760 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:21.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.072+0000 7f05427fc640 1 -- 172.21.15.40:0/1396077698 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f054c1f29c0 con 0x7f054c15a5e0 2023-10-18T09:00:21.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.266+0000 7f0553d6e640 1 -- 172.21.15.40:0/1396077698 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f054c1f29c0 con 0x7f053006ae40 2023-10-18T09:00:21.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.289+0000 7f05427fc640 1 -- 172.21.15.40:0/1396077698 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282230 (secure 0 0 0) 0x7f054c1f29c0 con 0x7f053006ae40 2023-10-18T09:00:21.295 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:21.296 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:00:21.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.295+0000 7f0553d6e640 1 -- 172.21.15.40:0/1396077698 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f053006ae40 msgr2=0x7f053006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:21.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.295+0000 7f0553d6e640 1 --2- 172.21.15.40:0/1396077698 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f053006ae40 0x7f053006d300 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0548034000 tx=0x7f0548003760 comp rx=0 tx=0).stop 2023-10-18T09:00:21.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.295+0000 7f0553d6e640 1 -- 172.21.15.40:0/1396077698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c15a5e0 msgr2=0x7f054c15aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:21.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.295+0000 7f0553d6e640 1 --2- 172.21.15.40:0/1396077698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c15a5e0 0x7f054c15aa60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0544001330 tx=0x7f054400ccd0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.296+0000 7f0553d6e640 1 -- 172.21.15.40:0/1396077698 shutdown_connections 2023-10-18T09:00:21.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.296+0000 7f0553d6e640 1 --2- 172.21.15.40:0/1396077698 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f053006ae40 0x7f053006d300 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.296+0000 7f0553d6e640 1 --2- 172.21.15.40:0/1396077698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f054c15a5e0 0x7f054c15aa60 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.296+0000 7f0553d6e640 1 --2- 172.21.15.40:0/1396077698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f054c158520 0x7f054c158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.296+0000 7f0553d6e640 1 --2- 172.21.15.40:0/1396077698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05380039c0 0x7f054c13ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.296+0000 7f0553d6e640 1 -- 172.21.15.40:0/1396077698 >> 172.21.15.40:0/1396077698 conn(0x7f054c06cee0 msgr2=0x7f054c0ae1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:21.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.296+0000 7f0553d6e640 1 -- 172.21.15.40:0/1396077698 shutdown_connections 2023-10-18T09:00:21.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.296+0000 7f0553d6e640 1 -- 172.21.15.40:0/1396077698 wait complete. 2023-10-18T09:00:21.318 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2023-10-18T09:00:21.005517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4793,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557706,"statfs":{"total":1600000000,"available":1595091077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.304691"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736891+0000","last_change":"2023-10-18T09:00:15.763191+0000","last_active":"2023-10-18T09:00:16.736891+0000","last_peered":"2023-10-18T09:00:16.736891+0000","last_clean":"2023-10-18T09:00:16.736891+0000","last_became_active":"2023-10-18T09:00:15.758469+0000","last_became_peered":"2023-10-18T09:00:15.758469+0000","last_unstale":"2023-10-18T09:00:16.736891+0000","last_undegraded":"2023-10-18T09:00:16.736891+0000","last_fullsized":"2023-10-18T09:00:16.736891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:03:56.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866602+0000","last_change":"2023-10-18T09:00:15.757911+0000","last_active":"2023-10-18T09:00:16.866602+0000","last_peered":"2023-10-18T09:00:16.866602+0000","last_clean":"2023-10-18T09:00:16.866602+0000","last_became_active":"2023-10-18T09:00:15.757452+0000","last_became_peered":"2023-10-18T09:00:15.757452+0000","last_unstale":"2023-10-18T09:00:16.866602+0000","last_undegraded":"2023-10-18T09:00:16.866602+0000","last_fullsized":"2023-10-18T09:00:16.866602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:19:41.723289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.734969+0000","last_change":"2023-10-18T09:00:15.761182+0000","last_active":"2023-10-18T09:00:16.734969+0000","last_peered":"2023-10-18T09:00:16.734969+0000","last_clean":"2023-10-18T09:00:16.734969+0000","last_became_active":"2023-10-18T09:00:15.760952+0000","last_became_peered":"2023-10-18T09:00:15.760952+0000","last_unstale":"2023-10-18T09:00:16.734969+0000","last_undegraded":"2023-10-18T09:00:16.734969+0000","last_fullsized":"2023-10-18T09:00:16.734969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:54:55.101341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.865987+0000","last_change":"2023-10-18T09:00:15.762841+0000","last_active":"2023-10-18T09:00:16.865987+0000","last_peered":"2023-10-18T09:00:16.865987+0000","last_clean":"2023-10-18T09:00:16.865987+0000","last_became_active":"2023-10-18T09:00:15.762393+0000","last_became_peered":"2023-10-18T09:00:15.762393+0000","last_unstale":"2023-10-18T09:00:16.865987+0000","last_undegraded":"2023-10-18T09:00:16.865987+0000","last_fullsized":"2023-10-18T09:00:16.865987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:51:04.245123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736606+0000","last_change":"2023-10-18T09:00:15.755929+0000","last_active":"2023-10-18T09:00:16.736606+0000","last_peered":"2023-10-18T09:00:16.736606+0000","last_clean":"2023-10-18T09:00:16.736606+0000","last_became_active":"2023-10-18T09:00:15.755639+0000","last_became_peered":"2023-10-18T09:00:15.755639+0000","last_unstale":"2023-10-18T09:00:16.736606+0000","last_undegraded":"2023-10-18T09:00:16.736606+0000","last_fullsized":"2023-10-18T09:00:16.736606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:50.213859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.869497+0000","last_change":"2023-10-18T09:00:15.765807+0000","last_active":"2023-10-18T09:00:16.869497+0000","last_peered":"2023-10-18T09:00:16.869497+0000","last_clean":"2023-10-18T09:00:16.869497+0000","last_became_active":"2023-10-18T09:00:15.765306+0000","last_became_peered":"2023-10-18T09:00:15.765306+0000","last_unstale":"2023-10-18T09:00:16.869497+0000","last_undegraded":"2023-10-18T09:00:16.869497+0000","last_fullsized":"2023-10-18T09:00:16.869497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:28:57.510395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.870713+0000","last_change":"2023-10-18T09:00:15.758255+0000","last_active":"2023-10-18T09:00:16.870713+0000","last_peered":"2023-10-18T09:00:16.870713+0000","last_clean":"2023-10-18T09:00:16.870713+0000","last_became_active":"2023-10-18T09:00:15.756071+0000","last_became_peered":"2023-10-18T09:00:15.756071+0000","last_unstale":"2023-10-18T09:00:16.870713+0000","last_undegraded":"2023-10-18T09:00:16.870713+0000","last_fullsized":"2023-10-18T09:00:16.870713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:46:39.794722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866761+0000","last_change":"2023-10-18T09:00:15.760360+0000","last_active":"2023-10-18T09:00:16.866761+0000","last_peered":"2023-10-18T09:00:16.866761+0000","last_clean":"2023-10-18T09:00:16.866761+0000","last_became_active":"2023-10-18T09:00:15.755866+0000","last_became_peered":"2023-10-18T09:00:15.755866+0000","last_unstale":"2023-10-18T09:00:16.866761+0000","last_undegraded":"2023-10-18T09:00:16.866761+0000","last_fullsized":"2023-10-18T09:00:16.866761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:24:22.148863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736100+0000","last_change":"2023-10-18T09:00:15.758789+0000","last_active":"2023-10-18T09:00:16.736100+0000","last_peered":"2023-10-18T09:00:16.736100+0000","last_clean":"2023-10-18T09:00:16.736100+0000","last_became_active":"2023-10-18T09:00:15.758487+0000","last_became_peered":"2023-10-18T09:00:15.758487+0000","last_unstale":"2023-10-18T09:00:16.736100+0000","last_undegraded":"2023-10-18T09:00:16.736100+0000","last_fullsized":"2023-10-18T09:00:16.736100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:51:02.550237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867399+0000","last_change":"2023-10-18T09:00:15.758617+0000","last_active":"2023-10-18T09:00:16.867399+0000","last_peered":"2023-10-18T09:00:16.867399+0000","last_clean":"2023-10-18T09:00:16.867399+0000","last_became_active":"2023-10-18T09:00:15.758324+0000","last_became_peered":"2023-10-18T09:00:15.758324+0000","last_unstale":"2023-10-18T09:00:16.867399+0000","last_undegraded":"2023-10-18T09:00:16.867399+0000","last_fullsized":"2023-10-18T09:00:16.867399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:03:24.223447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737441+0000","last_change":"2023-10-18T09:00:15.736563+0000","last_active":"2023-10-18T09:00:16.737441+0000","last_peered":"2023-10-18T09:00:16.737441+0000","last_clean":"2023-10-18T09:00:16.737441+0000","last_became_active":"2023-10-18T09:00:15.736013+0000","last_became_peered":"2023-10-18T09:00:15.736013+0000","last_unstale":"2023-10-18T09:00:16.737441+0000","last_undegraded":"2023-10-18T09:00:16.737441+0000","last_fullsized":"2023-10-18T09:00:16.737441+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:14:19.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866898+0000","last_change":"2023-10-18T09:00:15.763124+0000","last_active":"2023-10-18T09:00:16.866898+0000","last_peered":"2023-10-18T09:00:16.866898+0000","last_clean":"2023-10-18T09:00:16.866898+0000","last_became_active":"2023-10-18T09:00:15.762786+0000","last_became_peered":"2023-10-18T09:00:15.762786+0000","last_unstale":"2023-10-18T09:00:16.866898+0000","last_undegraded":"2023-10-18T09:00:16.866898+0000","last_fullsized":"2023-10-18T09:00:16.866898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:12:39.359221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.865876+0000","last_change":"2023-10-18T09:00:15.768782+0000","last_active":"2023-10-18T09:00:16.865876+0000","last_peered":"2023-10-18T09:00:16.865876+0000","last_clean":"2023-10-18T09:00:16.865876+0000","last_became_active":"2023-10-18T09:00:15.767657+0000","last_became_peered":"2023-10-18T09:00:15.767657+0000","last_unstale":"2023-10-18T09:00:16.865876+0000","last_undegraded":"2023-10-18T09:00:16.865876+0000","last_fullsized":"2023-10-18T09:00:16.865876+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:17:29.238634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736280+0000","last_change":"2023-10-18T09:00:15.752025+0000","last_active":"2023-10-18T09:00:16.736280+0000","last_peered":"2023-10-18T09:00:16.736280+0000","last_clean":"2023-10-18T09:00:16.736280+0000","last_became_active":"2023-10-18T09:00:15.751249+0000","last_became_peered":"2023-10-18T09:00:15.751249+0000","last_unstale":"2023-10-18T09:00:16.736280+0000","last_undegraded":"2023-10-18T09:00:16.736280+0000","last_fullsized":"2023-10-18T09:00:16.736280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:56:40.431057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.865758+0000","last_change":"2023-10-18T09:00:15.761180+0000","last_active":"2023-10-18T09:00:16.865758+0000","last_peered":"2023-10-18T09:00:16.865758+0000","last_clean":"2023-10-18T09:00:16.865758+0000","last_became_active":"2023-10-18T09:00:15.760931+0000","last_became_peered":"2023-10-18T09:00:15.760931+0000","last_unstale":"2023-10-18T09:00:16.865758+0000","last_undegraded":"2023-10-18T09:00:16.865758+0000","last_fullsized":"2023-10-18T09:00:16.865758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:44.909930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866905+0000","last_change":"2023-10-18T09:00:15.755241+0000","last_active":"2023-10-18T09:00:16.866905+0000","last_peered":"2023-10-18T09:00:16.866905+0000","last_clean":"2023-10-18T09:00:16.866905+0000","last_became_active":"2023-10-18T09:00:15.754219+0000","last_became_peered":"2023-10-18T09:00:15.754219+0000","last_unstale":"2023-10-18T09:00:16.866905+0000","last_undegraded":"2023-10-18T09:00:16.866905+0000","last_fullsized":"2023-10-18T09:00:16.866905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:10:37.475877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736469+0000","last_change":"2023-10-18T09:00:15.762556+0000","last_active":"2023-10-18T09:00:16.736469+0000","last_peered":"2023-10-18T09:00:16.736469+0000","last_clean":"2023-10-18T09:00:16.736469+0000","last_became_active":"2023-10-18T09:00:15.762213+0000","last_became_peered":"2023-10-18T09:00:15.762213+0000","last_unstale":"2023-10-18T09:00:16.736469+0000","last_undegraded":"2023-10-18T09:00:16.736469+0000","last_fullsized":"2023-10-18T09:00:16.736469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:13:20.414827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.865832+0000","last_change":"2023-10-18T09:00:15.756905+0000","last_active":"2023-10-18T09:00:16.865832+0000","last_peered":"2023-10-18T09:00:16.865832+0000","last_clean":"2023-10-18T09:00:16.865832+0000","last_became_active":"2023-10-18T09:00:15.753779+0000","last_became_peered":"2023-10-18T09:00:15.753779+0000","last_unstale":"2023-10-18T09:00:16.865832+0000","last_undegraded":"2023-10-18T09:00:16.865832+0000","last_fullsized":"2023-10-18T09:00:16.865832+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:15:47.083054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866960+0000","last_change":"2023-10-18T09:00:15.765485+0000","last_active":"2023-10-18T09:00:16.866960+0000","last_peered":"2023-10-18T09:00:16.866960+0000","last_clean":"2023-10-18T09:00:16.866960+0000","last_became_active":"2023-10-18T09:00:15.764978+0000","last_became_peered":"2023-10-18T09:00:15.764978+0000","last_unstale":"2023-10-18T09:00:16.866960+0000","last_undegraded":"2023-10-18T09:00:16.866960+0000","last_fullsized":"2023-10-18T09:00:16.866960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:28:45.528730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866110+0000","last_change":"2023-10-18T09:00:15.757707+0000","last_active":"2023-10-18T09:00:16.866110+0000","last_peered":"2023-10-18T09:00:16.866110+0000","last_clean":"2023-10-18T09:00:16.866110+0000","last_became_active":"2023-10-18T09:00:15.757478+0000","last_became_peered":"2023-10-18T09:00:15.757478+0000","last_unstale":"2023-10-18T09:00:16.866110+0000","last_undegraded":"2023-10-18T09:00:16.866110+0000","last_fullsized":"2023-10-18T09:00:16.866110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:00.837466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867390+0000","last_change":"2023-10-18T09:00:15.762884+0000","last_active":"2023-10-18T09:00:16.867390+0000","last_peered":"2023-10-18T09:00:16.867390+0000","last_clean":"2023-10-18T09:00:16.867390+0000","last_became_active":"2023-10-18T09:00:15.762479+0000","last_became_peered":"2023-10-18T09:00:15.762479+0000","last_unstale":"2023-10-18T09:00:16.867390+0000","last_undegraded":"2023-10-18T09:00:16.867390+0000","last_fullsized":"2023-10-18T09:00:16.867390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:26:17.885550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867242+0000","last_change":"2023-10-18T09:00:15.753122+0000","last_active":"2023-10-18T09:00:16.867242+0000","last_peered":"2023-10-18T09:00:16.867242+0000","last_clean":"2023-10-18T09:00:16.867242+0000","last_became_active":"2023-10-18T09:00:15.752653+0000","last_became_peered":"2023-10-18T09:00:15.752653+0000","last_unstale":"2023-10-18T09:00:16.867242+0000","last_undegraded":"2023-10-18T09:00:16.867242+0000","last_fullsized":"2023-10-18T09:00:16.867242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:15:32.449721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736879+0000","last_change":"2023-10-18T09:00:15.749937+0000","last_active":"2023-10-18T09:00:16.736879+0000","last_peered":"2023-10-18T09:00:16.736879+0000","last_clean":"2023-10-18T09:00:16.736879+0000","last_became_active":"2023-10-18T09:00:15.749226+0000","last_became_peered":"2023-10-18T09:00:15.749226+0000","last_unstale":"2023-10-18T09:00:16.736879+0000","last_undegraded":"2023-10-18T09:00:16.736879+0000","last_fullsized":"2023-10-18T09:00:16.736879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:28.075342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736586+0000","last_change":"2023-10-18T09:00:15.762946+0000","last_active":"2023-10-18T09:00:16.736586+0000","last_peered":"2023-10-18T09:00:16.736586+0000","last_clean":"2023-10-18T09:00:16.736586+0000","last_became_active":"2023-10-18T09:00:15.762276+0000","last_became_peered":"2023-10-18T09:00:15.762276+0000","last_unstale":"2023-10-18T09:00:16.736586+0000","last_undegraded":"2023-10-18T09:00:16.736586+0000","last_fullsized":"2023-10-18T09:00:16.736586+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:28:15.286205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867778+0000","last_change":"2023-10-18T09:00:15.737169+0000","last_active":"2023-10-18T09:00:16.867778+0000","last_peered":"2023-10-18T09:00:16.867778+0000","last_clean":"2023-10-18T09:00:16.867778+0000","last_became_active":"2023-10-18T09:00:15.736508+0000","last_became_peered":"2023-10-18T09:00:15.736508+0000","last_unstale":"2023-10-18T09:00:16.867778+0000","last_undegraded":"2023-10-18T09:00:16.867778+0000","last_fullsized":"2023-10-18T09:00:16.867778+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:32:57.960964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.868854+0000","last_change":"2023-10-18T09:00:15.757071+0000","last_active":"2023-10-18T09:00:16.868854+0000","last_peered":"2023-10-18T09:00:16.868854+0000","last_clean":"2023-10-18T09:00:16.868854+0000","last_became_active":"2023-10-18T09:00:15.754963+0000","last_became_peered":"2023-10-18T09:00:15.754963+0000","last_unstale":"2023-10-18T09:00:16.868854+0000","last_undegraded":"2023-10-18T09:00:16.868854+0000","last_fullsized":"2023-10-18T09:00:16.868854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:38.079651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.735091+0000","last_change":"2023-10-18T09:00:15.734271+0000","last_active":"2023-10-18T09:00:16.735091+0000","last_peered":"2023-10-18T09:00:16.735091+0000","last_clean":"2023-10-18T09:00:16.735091+0000","last_became_active":"2023-10-18T09:00:15.733788+0000","last_became_peered":"2023-10-18T09:00:15.733788+0000","last_unstale":"2023-10-18T09:00:16.735091+0000","last_undegraded":"2023-10-18T09:00:16.735091+0000","last_fullsized":"2023-10-18T09:00:16.735091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:24:56.735813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867924+0000","last_change":"2023-10-18T09:00:15.756508+0000","last_active":"2023-10-18T09:00:16.867924+0000","last_peered":"2023-10-18T09:00:16.867924+0000","last_clean":"2023-10-18T09:00:16.867924+0000","last_became_active":"2023-10-18T09:00:15.756155+0000","last_became_peered":"2023-10-18T09:00:15.756155+0000","last_unstale":"2023-10-18T09:00:16.867924+0000","last_undegraded":"2023-10-18T09:00:16.867924+0000","last_fullsized":"2023-10-18T09:00:16.867924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:38.079651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.734779+0000","last_change":"2023-10-18T09:00:15.754144+0000","last_active":"2023-10-18T09:00:16.734779+0000","last_peered":"2023-10-18T09:00:16.734779+0000","last_clean":"2023-10-18T09:00:16.734779+0000","last_became_active":"2023-10-18T09:00:15.748159+0000","last_became_peered":"2023-10-18T09:00:15.748159+0000","last_unstale":"2023-10-18T09:00:16.734779+0000","last_undegraded":"2023-10-18T09:00:16.734779+0000","last_fullsized":"2023-10-18T09:00:16.734779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:26:54.506506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.871708+0000","last_change":"2023-10-18T09:00:15.755928+0000","last_active":"2023-10-18T09:00:16.871708+0000","last_peered":"2023-10-18T09:00:16.871708+0000","last_clean":"2023-10-18T09:00:16.871708+0000","last_became_active":"2023-10-18T09:00:15.755233+0000","last_became_peered":"2023-10-18T09:00:15.755233+0000","last_unstale":"2023-10-18T09:00:16.871708+0000","last_undegraded":"2023-10-18T09:00:16.871708+0000","last_fullsized":"2023-10-18T09:00:16.871708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:06.826641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.869159+0000","last_change":"2023-10-18T09:00:15.760173+0000","last_active":"2023-10-18T09:00:16.869159+0000","last_peered":"2023-10-18T09:00:16.869159+0000","last_clean":"2023-10-18T09:00:16.869159+0000","last_became_active":"2023-10-18T09:00:15.759923+0000","last_became_peered":"2023-10-18T09:00:15.759923+0000","last_unstale":"2023-10-18T09:00:16.869159+0000","last_undegraded":"2023-10-18T09:00:16.869159+0000","last_fullsized":"2023-10-18T09:00:16.869159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:22.137803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866984+0000","last_change":"2023-10-18T09:00:15.765266+0000","last_active":"2023-10-18T09:00:16.866984+0000","last_peered":"2023-10-18T09:00:16.866984+0000","last_clean":"2023-10-18T09:00:16.866984+0000","last_became_active":"2023-10-18T09:00:15.764999+0000","last_became_peered":"2023-10-18T09:00:15.764999+0000","last_unstale":"2023-10-18T09:00:16.866984+0000","last_undegraded":"2023-10-18T09:00:16.866984+0000","last_fullsized":"2023-10-18T09:00:16.866984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:38.805225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737031+0000","last_change":"2023-10-18T09:00:15.762805+0000","last_active":"2023-10-18T09:00:16.737031+0000","last_peered":"2023-10-18T09:00:16.737031+0000","last_clean":"2023-10-18T09:00:16.737031+0000","last_became_active":"2023-10-18T09:00:15.761946+0000","last_became_peered":"2023-10-18T09:00:15.761946+0000","last_unstale":"2023-10-18T09:00:16.737031+0000","last_undegraded":"2023-10-18T09:00:16.737031+0000","last_fullsized":"2023-10-18T09:00:16.737031+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:28:00.652852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"86'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773766+0000","last_change":"2023-10-18T09:00:14.719832+0000","last_active":"2023-10-18T09:00:15.773766+0000","last_peered":"2023-10-18T09:00:15.773766+0000","last_clean":"2023-10-18T09:00:15.773766+0000","last_became_active":"2023-10-18T09:00:14.719074+0000","last_became_peered":"2023-10-18T09:00:14.719074+0000","last_unstale":"2023-10-18T09:00:15.773766+0000","last_undegraded":"2023-10-18T09:00:15.773766+0000","last_fullsized":"2023-10-18T09:00:15.773766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:31:09.739409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736246+0000","last_change":"2023-10-18T09:00:15.761074+0000","last_active":"2023-10-18T09:00:16.736246+0000","last_peered":"2023-10-18T09:00:16.736246+0000","last_clean":"2023-10-18T09:00:16.736246+0000","last_became_active":"2023-10-18T09:00:15.757708+0000","last_became_peered":"2023-10-18T09:00:15.757708+0000","last_unstale":"2023-10-18T09:00:16.736246+0000","last_undegraded":"2023-10-18T09:00:16.736246+0000","last_fullsized":"2023-10-18T09:00:16.736246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:16:51.405175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.772773+0000","last_change":"2023-10-18T09:00:14.722227+0000","last_active":"2023-10-18T09:00:15.772773+0000","last_peered":"2023-10-18T09:00:15.772773+0000","last_clean":"2023-10-18T09:00:15.772773+0000","last_became_active":"2023-10-18T09:00:14.720313+0000","last_became_peered":"2023-10-18T09:00:14.720313+0000","last_unstale":"2023-10-18T09:00:15.772773+0000","last_undegraded":"2023-10-18T09:00:15.772773+0000","last_fullsized":"2023-10-18T09:00:15.772773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:28:25.001764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.869848+0000","last_change":"2023-10-18T09:00:15.753770+0000","last_active":"2023-10-18T09:00:16.869848+0000","last_peered":"2023-10-18T09:00:16.869848+0000","last_clean":"2023-10-18T09:00:16.869848+0000","last_became_active":"2023-10-18T09:00:15.752239+0000","last_became_peered":"2023-10-18T09:00:15.752239+0000","last_unstale":"2023-10-18T09:00:16.869848+0000","last_undegraded":"2023-10-18T09:00:16.869848+0000","last_fullsized":"2023-10-18T09:00:16.869848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:04:34.318321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773255+0000","last_change":"2023-10-18T09:00:14.724618+0000","last_active":"2023-10-18T09:00:15.773255+0000","last_peered":"2023-10-18T09:00:15.773255+0000","last_clean":"2023-10-18T09:00:15.773255+0000","last_became_active":"2023-10-18T09:00:14.721500+0000","last_became_peered":"2023-10-18T09:00:14.721500+0000","last_unstale":"2023-10-18T09:00:15.773255+0000","last_undegraded":"2023-10-18T09:00:15.773255+0000","last_fullsized":"2023-10-18T09:00:15.773255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:59.045131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737858+0000","last_change":"2023-10-18T09:00:15.759339+0000","last_active":"2023-10-18T09:00:16.737858+0000","last_peered":"2023-10-18T09:00:16.737858+0000","last_clean":"2023-10-18T09:00:16.737858+0000","last_became_active":"2023-10-18T09:00:15.758140+0000","last_became_peered":"2023-10-18T09:00:15.758140+0000","last_unstale":"2023-10-18T09:00:16.737858+0000","last_undegraded":"2023-10-18T09:00:16.737858+0000","last_fullsized":"2023-10-18T09:00:16.737858+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:17:29.238634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773868+0000","last_change":"2023-10-18T09:00:14.729343+0000","last_active":"2023-10-18T09:00:15.773868+0000","last_peered":"2023-10-18T09:00:15.773868+0000","last_clean":"2023-10-18T09:00:15.773868+0000","last_became_active":"2023-10-18T09:00:14.728957+0000","last_became_peered":"2023-10-18T09:00:14.728957+0000","last_unstale":"2023-10-18T09:00:15.773868+0000","last_undegraded":"2023-10-18T09:00:15.773868+0000","last_fullsized":"2023-10-18T09:00:15.773868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:12:38.346335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.869564+0000","last_change":"2023-10-18T09:00:15.758912+0000","last_active":"2023-10-18T09:00:16.869564+0000","last_peered":"2023-10-18T09:00:16.869564+0000","last_clean":"2023-10-18T09:00:16.869564+0000","last_became_active":"2023-10-18T09:00:15.756824+0000","last_became_peered":"2023-10-18T09:00:15.756824+0000","last_unstale":"2023-10-18T09:00:16.869564+0000","last_undegraded":"2023-10-18T09:00:16.869564+0000","last_fullsized":"2023-10-18T09:00:16.869564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:24:27.536167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.772736+0000","last_change":"2023-10-18T09:00:14.720994+0000","last_active":"2023-10-18T09:00:15.772736+0000","last_peered":"2023-10-18T09:00:15.772736+0000","last_clean":"2023-10-18T09:00:15.772736+0000","last_became_active":"2023-10-18T09:00:14.720661+0000","last_became_peered":"2023-10-18T09:00:14.720661+0000","last_unstale":"2023-10-18T09:00:15.772736+0000","last_undegraded":"2023-10-18T09:00:15.772736+0000","last_fullsized":"2023-10-18T09:00:15.772736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:55:27.065983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867552+0000","last_change":"2023-10-18T09:00:15.737040+0000","last_active":"2023-10-18T09:00:16.867552+0000","last_peered":"2023-10-18T09:00:16.867552+0000","last_clean":"2023-10-18T09:00:16.867552+0000","last_became_active":"2023-10-18T09:00:15.736400+0000","last_became_peered":"2023-10-18T09:00:15.736400+0000","last_unstale":"2023-10-18T09:00:16.867552+0000","last_undegraded":"2023-10-18T09:00:16.867552+0000","last_fullsized":"2023-10-18T09:00:16.867552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:27:57.117593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773337+0000","last_change":"2023-10-18T09:00:14.726650+0000","last_active":"2023-10-18T09:00:15.773337+0000","last_peered":"2023-10-18T09:00:15.773337+0000","last_clean":"2023-10-18T09:00:15.773337+0000","last_became_active":"2023-10-18T09:00:14.726047+0000","last_became_peered":"2023-10-18T09:00:14.726047+0000","last_unstale":"2023-10-18T09:00:15.773337+0000","last_undegraded":"2023-10-18T09:00:15.773337+0000","last_fullsized":"2023-10-18T09:00:15.773337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:55:25.382068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736921+0000","last_change":"2023-10-18T09:00:15.759622+0000","last_active":"2023-10-18T09:00:16.736921+0000","last_peered":"2023-10-18T09:00:16.736921+0000","last_clean":"2023-10-18T09:00:16.736921+0000","last_became_active":"2023-10-18T09:00:15.759302+0000","last_became_peered":"2023-10-18T09:00:15.759302+0000","last_unstale":"2023-10-18T09:00:16.736921+0000","last_undegraded":"2023-10-18T09:00:16.736921+0000","last_fullsized":"2023-10-18T09:00:16.736921+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:25:33.396541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.749958+0000","last_change":"2023-10-18T09:00:14.722001+0000","last_active":"2023-10-18T09:00:15.749958+0000","last_peered":"2023-10-18T09:00:15.749958+0000","last_clean":"2023-10-18T09:00:15.749958+0000","last_became_active":"2023-10-18T09:00:14.721325+0000","last_became_peered":"2023-10-18T09:00:14.721325+0000","last_unstale":"2023-10-18T09:00:15.749958+0000","last_undegraded":"2023-10-18T09:00:15.749958+0000","last_fullsized":"2023-10-18T09:00:15.749958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:05:08.365526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.865724+0000","last_change":"2023-10-18T09:00:15.765115+0000","last_active":"2023-10-18T09:00:16.865724+0000","last_peered":"2023-10-18T09:00:16.865724+0000","last_clean":"2023-10-18T09:00:16.865724+0000","last_became_active":"2023-10-18T09:00:15.764828+0000","last_became_peered":"2023-10-18T09:00:15.764828+0000","last_unstale":"2023-10-18T09:00:16.865724+0000","last_undegraded":"2023-10-18T09:00:16.865724+0000","last_fullsized":"2023-10-18T09:00:16.865724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:43:18.411223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.751377+0000","last_change":"2023-10-18T09:00:14.719395+0000","last_active":"2023-10-18T09:00:15.751377+0000","last_peered":"2023-10-18T09:00:15.751377+0000","last_clean":"2023-10-18T09:00:15.751377+0000","last_became_active":"2023-10-18T09:00:14.718426+0000","last_became_peered":"2023-10-18T09:00:14.718426+0000","last_unstale":"2023-10-18T09:00:15.751377+0000","last_undegraded":"2023-10-18T09:00:15.751377+0000","last_fullsized":"2023-10-18T09:00:15.751377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:42:14.030822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.871025+0000","last_change":"2023-10-18T09:00:15.758761+0000","last_active":"2023-10-18T09:00:16.871025+0000","last_peered":"2023-10-18T09:00:16.871025+0000","last_clean":"2023-10-18T09:00:16.871025+0000","last_became_active":"2023-10-18T09:00:15.758524+0000","last_became_peered":"2023-10-18T09:00:15.758524+0000","last_unstale":"2023-10-18T09:00:16.871025+0000","last_undegraded":"2023-10-18T09:00:16.871025+0000","last_fullsized":"2023-10-18T09:00:16.871025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:23.871839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.730864+0000","last_change":"2023-10-18T09:00:14.713528+0000","last_active":"2023-10-18T09:00:15.730864+0000","last_peered":"2023-10-18T09:00:15.730864+0000","last_clean":"2023-10-18T09:00:15.730864+0000","last_became_active":"2023-10-18T09:00:14.712766+0000","last_became_peered":"2023-10-18T09:00:14.712766+0000","last_unstale":"2023-10-18T09:00:15.730864+0000","last_undegraded":"2023-10-18T09:00:15.730864+0000","last_fullsized":"2023-10-18T09:00:15.730864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:37:44.788135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737694+0000","last_change":"2023-10-18T09:00:15.758755+0000","last_active":"2023-10-18T09:00:16.737694+0000","last_peered":"2023-10-18T09:00:16.737694+0000","last_clean":"2023-10-18T09:00:16.737694+0000","last_became_active":"2023-10-18T09:00:15.757506+0000","last_became_peered":"2023-10-18T09:00:15.757506+0000","last_unstale":"2023-10-18T09:00:16.737694+0000","last_undegraded":"2023-10-18T09:00:16.737694+0000","last_fullsized":"2023-10-18T09:00:16.737694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:43:18.411223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.748766+0000","last_change":"2023-10-18T09:00:14.728500+0000","last_active":"2023-10-18T09:00:15.748766+0000","last_peered":"2023-10-18T09:00:15.748766+0000","last_clean":"2023-10-18T09:00:15.748766+0000","last_became_active":"2023-10-18T09:00:14.728243+0000","last_became_peered":"2023-10-18T09:00:14.728243+0000","last_unstale":"2023-10-18T09:00:15.748766+0000","last_undegraded":"2023-10-18T09:00:15.748766+0000","last_fullsized":"2023-10-18T09:00:15.748766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:51:03.232237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867311+0000","last_change":"2023-10-18T09:00:15.759536+0000","last_active":"2023-10-18T09:00:16.867311+0000","last_peered":"2023-10-18T09:00:16.867311+0000","last_clean":"2023-10-18T09:00:16.867311+0000","last_became_active":"2023-10-18T09:00:15.759116+0000","last_became_peered":"2023-10-18T09:00:15.759116+0000","last_unstale":"2023-10-18T09:00:16.867311+0000","last_undegraded":"2023-10-18T09:00:16.867311+0000","last_fullsized":"2023-10-18T09:00:16.867311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:34:17.782897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"86'3","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.774146+0000","last_change":"2023-10-18T09:00:14.723428+0000","last_active":"2023-10-18T09:00:15.774146+0000","last_peered":"2023-10-18T09:00:15.774146+0000","last_clean":"2023-10-18T09:00:15.774146+0000","last_became_active":"2023-10-18T09:00:14.723174+0000","last_became_peered":"2023-10-18T09:00:14.723174+0000","last_unstale":"2023-10-18T09:00:15.774146+0000","last_undegraded":"2023-10-18T09:00:15.774146+0000","last_fullsized":"2023-10-18T09:00:15.774146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:53:02.168967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736447+0000","last_change":"2023-10-18T09:00:15.755629+0000","last_active":"2023-10-18T09:00:16.736447+0000","last_peered":"2023-10-18T09:00:16.736447+0000","last_clean":"2023-10-18T09:00:16.736447+0000","last_became_active":"2023-10-18T09:00:15.755325+0000","last_became_peered":"2023-10-18T09:00:15.755325+0000","last_unstale":"2023-10-18T09:00:16.736447+0000","last_undegraded":"2023-10-18T09:00:16.736447+0000","last_fullsized":"2023-10-18T09:00:16.736447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:32:40.008198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"86'6","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.776851+0000","last_change":"2023-10-18T09:00:14.720054+0000","last_active":"2023-10-18T09:00:15.776851+0000","last_peered":"2023-10-18T09:00:15.776851+0000","last_clean":"2023-10-18T09:00:15.776851+0000","last_became_active":"2023-10-18T09:00:14.719704+0000","last_became_peered":"2023-10-18T09:00:14.719704+0000","last_unstale":"2023-10-18T09:00:15.776851+0000","last_undegraded":"2023-10-18T09:00:15.776851+0000","last_fullsized":"2023-10-18T09:00:15.776851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:22:27.935440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737925+0000","last_change":"2023-10-18T09:00:15.758943+0000","last_active":"2023-10-18T09:00:16.737925+0000","last_peered":"2023-10-18T09:00:16.737925+0000","last_clean":"2023-10-18T09:00:16.737925+0000","last_became_active":"2023-10-18T09:00:15.758060+0000","last_became_peered":"2023-10-18T09:00:15.758060+0000","last_unstale":"2023-10-18T09:00:16.737925+0000","last_undegraded":"2023-10-18T09:00:16.737925+0000","last_fullsized":"2023-10-18T09:00:16.737925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:00.837466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.774849+0000","last_change":"2023-10-18T09:00:14.719224+0000","last_active":"2023-10-18T09:00:15.774849+0000","last_peered":"2023-10-18T09:00:15.774849+0000","last_clean":"2023-10-18T09:00:15.774849+0000","last_became_active":"2023-10-18T09:00:14.718827+0000","last_became_peered":"2023-10-18T09:00:14.718827+0000","last_unstale":"2023-10-18T09:00:15.774849+0000","last_undegraded":"2023-10-18T09:00:15.774849+0000","last_fullsized":"2023-10-18T09:00:15.774849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:55:25.382068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.872019+0000","last_change":"2023-10-18T09:00:15.753951+0000","last_active":"2023-10-18T09:00:16.872019+0000","last_peered":"2023-10-18T09:00:16.872019+0000","last_clean":"2023-10-18T09:00:16.872019+0000","last_became_active":"2023-10-18T09:00:15.753614+0000","last_became_peered":"2023-10-18T09:00:15.753614+0000","last_unstale":"2023-10-18T09:00:16.872019+0000","last_undegraded":"2023-10-18T09:00:16.872019+0000","last_fullsized":"2023-10-18T09:00:16.872019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:31.329353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.750279+0000","last_change":"2023-10-18T09:00:14.724705+0000","last_active":"2023-10-18T09:00:15.750279+0000","last_peered":"2023-10-18T09:00:15.750279+0000","last_clean":"2023-10-18T09:00:15.750279+0000","last_became_active":"2023-10-18T09:00:14.724414+0000","last_became_peered":"2023-10-18T09:00:14.724414+0000","last_unstale":"2023-10-18T09:00:15.750279+0000","last_undegraded":"2023-10-18T09:00:15.750279+0000","last_fullsized":"2023-10-18T09:00:15.750279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:53.753897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866195+0000","last_change":"2023-10-18T09:00:15.767934+0000","last_active":"2023-10-18T09:00:16.866195+0000","last_peered":"2023-10-18T09:00:16.866195+0000","last_clean":"2023-10-18T09:00:16.866195+0000","last_became_active":"2023-10-18T09:00:15.767341+0000","last_became_peered":"2023-10-18T09:00:15.767341+0000","last_unstale":"2023-10-18T09:00:16.866195+0000","last_undegraded":"2023-10-18T09:00:16.866195+0000","last_fullsized":"2023-10-18T09:00:16.866195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:28.179544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.745451+0000","last_change":"2023-10-18T09:00:14.721694+0000","last_active":"2023-10-18T09:00:15.745451+0000","last_peered":"2023-10-18T09:00:15.745451+0000","last_clean":"2023-10-18T09:00:15.745451+0000","last_became_active":"2023-10-18T09:00:14.720757+0000","last_became_peered":"2023-10-18T09:00:14.720757+0000","last_unstale":"2023-10-18T09:00:15.745451+0000","last_undegraded":"2023-10-18T09:00:15.745451+0000","last_fullsized":"2023-10-18T09:00:15.745451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:26:16.872665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736651+0000","last_change":"2023-10-18T09:00:15.759417+0000","last_active":"2023-10-18T09:00:16.736651+0000","last_peered":"2023-10-18T09:00:16.736651+0000","last_clean":"2023-10-18T09:00:16.736651+0000","last_became_active":"2023-10-18T09:00:15.757457+0000","last_became_peered":"2023-10-18T09:00:15.757457+0000","last_unstale":"2023-10-18T09:00:16.736651+0000","last_undegraded":"2023-10-18T09:00:16.736651+0000","last_fullsized":"2023-10-18T09:00:16.736651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:44:11.773265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"86'8","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.776052+0000","last_change":"2023-10-18T09:00:14.724785+0000","last_active":"2023-10-18T09:00:15.776052+0000","last_peered":"2023-10-18T09:00:15.776052+0000","last_clean":"2023-10-18T09:00:15.776052+0000","last_became_active":"2023-10-18T09:00:14.724371+0000","last_became_peered":"2023-10-18T09:00:14.724371+0000","last_unstale":"2023-10-18T09:00:15.776052+0000","last_undegraded":"2023-10-18T09:00:15.776052+0000","last_fullsized":"2023-10-18T09:00:15.776052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:46.341496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.868082+0000","last_change":"2023-10-18T09:00:15.751116+0000","last_active":"2023-10-18T09:00:16.868082+0000","last_peered":"2023-10-18T09:00:16.868082+0000","last_clean":"2023-10-18T09:00:16.868082+0000","last_became_active":"2023-10-18T09:00:15.750596+0000","last_became_peered":"2023-10-18T09:00:15.750596+0000","last_unstale":"2023-10-18T09:00:16.868082+0000","last_undegraded":"2023-10-18T09:00:16.868082+0000","last_fullsized":"2023-10-18T09:00:16.868082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:22.137803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.772684+0000","last_change":"2023-10-18T09:00:14.728865+0000","last_active":"2023-10-18T09:00:15.772684+0000","last_peered":"2023-10-18T09:00:15.772684+0000","last_clean":"2023-10-18T09:00:15.772684+0000","last_became_active":"2023-10-18T09:00:14.728369+0000","last_became_peered":"2023-10-18T09:00:14.728369+0000","last_unstale":"2023-10-18T09:00:15.772684+0000","last_undegraded":"2023-10-18T09:00:15.772684+0000","last_fullsized":"2023-10-18T09:00:15.772684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:51:43.253647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737387+0000","last_change":"2023-10-18T09:00:15.762625+0000","last_active":"2023-10-18T09:00:16.737387+0000","last_peered":"2023-10-18T09:00:16.737387+0000","last_clean":"2023-10-18T09:00:16.737387+0000","last_became_active":"2023-10-18T09:00:15.761420+0000","last_became_peered":"2023-10-18T09:00:15.761420+0000","last_unstale":"2023-10-18T09:00:16.737387+0000","last_undegraded":"2023-10-18T09:00:16.737387+0000","last_fullsized":"2023-10-18T09:00:16.737387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:14:41.715749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773990+0000","last_change":"2023-10-18T09:00:14.722184+0000","last_active":"2023-10-18T09:00:15.773990+0000","last_peered":"2023-10-18T09:00:15.773990+0000","last_clean":"2023-10-18T09:00:15.773990+0000","last_became_active":"2023-10-18T09:00:14.721073+0000","last_became_peered":"2023-10-18T09:00:14.721073+0000","last_unstale":"2023-10-18T09:00:15.773990+0000","last_undegraded":"2023-10-18T09:00:15.773990+0000","last_fullsized":"2023-10-18T09:00:15.773990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:35:27.318871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867237+0000","last_change":"2023-10-18T09:00:15.758090+0000","last_active":"2023-10-18T09:00:16.867237+0000","last_peered":"2023-10-18T09:00:16.867237+0000","last_clean":"2023-10-18T09:00:16.867237+0000","last_became_active":"2023-10-18T09:00:15.757750+0000","last_became_peered":"2023-10-18T09:00:15.757750+0000","last_unstale":"2023-10-18T09:00:16.867237+0000","last_undegraded":"2023-10-18T09:00:16.867237+0000","last_fullsized":"2023-10-18T09:00:16.867237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:54.766783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.772911+0000","last_change":"2023-10-18T09:00:14.729164+0000","last_active":"2023-10-18T09:00:15.772911+0000","last_peered":"2023-10-18T09:00:15.772911+0000","last_clean":"2023-10-18T09:00:15.772911+0000","last_became_active":"2023-10-18T09:00:14.728269+0000","last_became_peered":"2023-10-18T09:00:14.728269+0000","last_unstale":"2023-10-18T09:00:15.772911+0000","last_undegraded":"2023-10-18T09:00:15.772911+0000","last_fullsized":"2023-10-18T09:00:15.772911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:27.534647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736706+0000","last_change":"2023-10-18T09:00:15.761518+0000","last_active":"2023-10-18T09:00:16.736706+0000","last_peered":"2023-10-18T09:00:16.736706+0000","last_clean":"2023-10-18T09:00:16.736706+0000","last_became_active":"2023-10-18T09:00:15.760762+0000","last_became_peered":"2023-10-18T09:00:15.760762+0000","last_unstale":"2023-10-18T09:00:16.736706+0000","last_undegraded":"2023-10-18T09:00:16.736706+0000","last_fullsized":"2023-10-18T09:00:16.736706+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:02:37.770586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.730935+0000","last_change":"2023-10-18T09:00:14.723114+0000","last_active":"2023-10-18T09:00:15.730935+0000","last_peered":"2023-10-18T09:00:15.730935+0000","last_clean":"2023-10-18T09:00:15.730935+0000","last_became_active":"2023-10-18T09:00:14.722764+0000","last_became_peered":"2023-10-18T09:00:14.722764+0000","last_unstale":"2023-10-18T09:00:15.730935+0000","last_undegraded":"2023-10-18T09:00:15.730935+0000","last_fullsized":"2023-10-18T09:00:15.730935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:24:00.853217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.735158+0000","last_change":"2023-10-18T09:00:15.755724+0000","last_active":"2023-10-18T09:00:16.735158+0000","last_peered":"2023-10-18T09:00:16.735158+0000","last_clean":"2023-10-18T09:00:16.735158+0000","last_became_active":"2023-10-18T09:00:15.755472+0000","last_became_peered":"2023-10-18T09:00:15.755472+0000","last_unstale":"2023-10-18T09:00:16.735158+0000","last_undegraded":"2023-10-18T09:00:16.735158+0000","last_fullsized":"2023-10-18T09:00:16.735158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:22:41.062843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.758241+0000","last_change":"2023-10-18T09:00:14.727319+0000","last_active":"2023-10-18T09:00:15.758241+0000","last_peered":"2023-10-18T09:00:15.758241+0000","last_clean":"2023-10-18T09:00:15.758241+0000","last_became_active":"2023-10-18T09:00:14.726864+0000","last_became_peered":"2023-10-18T09:00:14.726864+0000","last_unstale":"2023-10-18T09:00:15.758241+0000","last_undegraded":"2023-10-18T09:00:15.758241+0000","last_fullsized":"2023-10-18T09:00:15.758241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:15.393555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.734848+0000","last_change":"2023-10-18T09:00:15.756743+0000","last_active":"2023-10-18T09:00:16.734848+0000","last_peered":"2023-10-18T09:00:16.734848+0000","last_clean":"2023-10-18T09:00:16.734848+0000","last_became_active":"2023-10-18T09:00:15.756462+0000","last_became_peered":"2023-10-18T09:00:15.756462+0000","last_unstale":"2023-10-18T09:00:16.734848+0000","last_undegraded":"2023-10-18T09:00:16.734848+0000","last_fullsized":"2023-10-18T09:00:16.734848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:14.505133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.752571+0000","last_change":"2023-10-18T09:00:14.723338+0000","last_active":"2023-10-18T09:00:15.752571+0000","last_peered":"2023-10-18T09:00:15.752571+0000","last_clean":"2023-10-18T09:00:15.752571+0000","last_became_active":"2023-10-18T09:00:14.723084+0000","last_became_peered":"2023-10-18T09:00:14.723084+0000","last_unstale":"2023-10-18T09:00:15.752571+0000","last_undegraded":"2023-10-18T09:00:15.752571+0000","last_fullsized":"2023-10-18T09:00:15.752571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:32:56.948078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.734768+0000","last_change":"2023-10-18T09:00:15.758267+0000","last_active":"2023-10-18T09:00:16.734768+0000","last_peered":"2023-10-18T09:00:16.734768+0000","last_clean":"2023-10-18T09:00:16.734768+0000","last_became_active":"2023-10-18T09:00:15.758033+0000","last_became_peered":"2023-10-18T09:00:15.758033+0000","last_unstale":"2023-10-18T09:00:16.734768+0000","last_undegraded":"2023-10-18T09:00:16.734768+0000","last_fullsized":"2023-10-18T09:00:16.734768+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:04:50.114527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.754653+0000","last_change":"2023-10-18T09:00:14.717552+0000","last_active":"2023-10-18T09:00:15.754653+0000","last_peered":"2023-10-18T09:00:15.754653+0000","last_clean":"2023-10-18T09:00:15.754653+0000","last_became_active":"2023-10-18T09:00:14.717311+0000","last_became_peered":"2023-10-18T09:00:14.717311+0000","last_unstale":"2023-10-18T09:00:15.754653+0000","last_undegraded":"2023-10-18T09:00:15.754653+0000","last_fullsized":"2023-10-18T09:00:15.754653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:27:59.639966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866525+0000","last_change":"2023-10-18T09:00:15.766855+0000","last_active":"2023-10-18T09:00:16.866525+0000","last_peered":"2023-10-18T09:00:16.866525+0000","last_clean":"2023-10-18T09:00:16.866525+0000","last_became_active":"2023-10-18T09:00:15.763490+0000","last_became_peered":"2023-10-18T09:00:15.763490+0000","last_unstale":"2023-10-18T09:00:16.866525+0000","last_undegraded":"2023-10-18T09:00:16.866525+0000","last_fullsized":"2023-10-18T09:00:16.866525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:16:52.626915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.772821+0000","last_change":"2023-10-18T09:00:14.725676+0000","last_active":"2023-10-18T09:00:15.772821+0000","last_peered":"2023-10-18T09:00:15.772821+0000","last_clean":"2023-10-18T09:00:15.772821+0000","last_became_active":"2023-10-18T09:00:14.725396+0000","last_became_peered":"2023-10-18T09:00:14.725396+0000","last_unstale":"2023-10-18T09:00:15.772821+0000","last_undegraded":"2023-10-18T09:00:15.772821+0000","last_fullsized":"2023-10-18T09:00:15.772821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:00.426054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.871368+0000","last_change":"2023-10-18T09:00:15.754403+0000","last_active":"2023-10-18T09:00:16.871368+0000","last_peered":"2023-10-18T09:00:16.871368+0000","last_clean":"2023-10-18T09:00:16.871368+0000","last_became_active":"2023-10-18T09:00:15.753953+0000","last_became_peered":"2023-10-18T09:00:15.753953+0000","last_unstale":"2023-10-18T09:00:16.871368+0000","last_undegraded":"2023-10-18T09:00:16.871368+0000","last_fullsized":"2023-10-18T09:00:16.871368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:22.851038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.742969+0000","last_change":"2023-10-18T09:00:14.719262+0000","last_active":"2023-10-18T09:00:15.742969+0000","last_peered":"2023-10-18T09:00:15.742969+0000","last_clean":"2023-10-18T09:00:15.742969+0000","last_became_active":"2023-10-18T09:00:14.718856+0000","last_became_peered":"2023-10-18T09:00:14.718856+0000","last_unstale":"2023-10-18T09:00:15.742969+0000","last_undegraded":"2023-10-18T09:00:15.742969+0000","last_fullsized":"2023-10-18T09:00:15.742969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:59.045131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.738154+0000","last_change":"2023-10-18T09:00:15.758945+0000","last_active":"2023-10-18T09:00:16.738154+0000","last_peered":"2023-10-18T09:00:16.738154+0000","last_clean":"2023-10-18T09:00:16.738154+0000","last_became_active":"2023-10-18T09:00:15.757454+0000","last_became_peered":"2023-10-18T09:00:15.757454+0000","last_unstale":"2023-10-18T09:00:16.738154+0000","last_undegraded":"2023-10-18T09:00:16.738154+0000","last_fullsized":"2023-10-18T09:00:16.738154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:28.179544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.744492+0000","last_change":"2023-10-18T09:00:14.719287+0000","last_active":"2023-10-18T09:00:15.744492+0000","last_peered":"2023-10-18T09:00:15.744492+0000","last_clean":"2023-10-18T09:00:15.744492+0000","last_became_active":"2023-10-18T09:00:14.719023+0000","last_became_peered":"2023-10-18T09:00:14.719023+0000","last_unstale":"2023-10-18T09:00:15.744492+0000","last_undegraded":"2023-10-18T09:00:15.744492+0000","last_fullsized":"2023-10-18T09:00:15.744492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:08.548927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737731+0000","last_change":"2023-10-18T09:00:15.748118+0000","last_active":"2023-10-18T09:00:16.737731+0000","last_peered":"2023-10-18T09:00:16.737731+0000","last_clean":"2023-10-18T09:00:16.737731+0000","last_became_active":"2023-10-18T09:00:15.747642+0000","last_became_peered":"2023-10-18T09:00:15.747642+0000","last_unstale":"2023-10-18T09:00:16.737731+0000","last_undegraded":"2023-10-18T09:00:16.737731+0000","last_fullsized":"2023-10-18T09:00:16.737731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:44.909930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.755422+0000","last_change":"2023-10-18T09:00:14.727210+0000","last_active":"2023-10-18T09:00:15.755422+0000","last_peered":"2023-10-18T09:00:15.755422+0000","last_clean":"2023-10-18T09:00:15.755422+0000","last_became_active":"2023-10-18T09:00:14.727034+0000","last_became_peered":"2023-10-18T09:00:14.727034+0000","last_unstale":"2023-10-18T09:00:15.755422+0000","last_undegraded":"2023-10-18T09:00:15.755422+0000","last_fullsized":"2023-10-18T09:00:15.755422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:14:40.702864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736824+0000","last_change":"2023-10-18T09:00:15.746384+0000","last_active":"2023-10-18T09:00:16.736824+0000","last_peered":"2023-10-18T09:00:16.736824+0000","last_clean":"2023-10-18T09:00:16.736824+0000","last_became_active":"2023-10-18T09:00:15.745760+0000","last_became_peered":"2023-10-18T09:00:15.745760+0000","last_unstale":"2023-10-18T09:00:16.736824+0000","last_undegraded":"2023-10-18T09:00:16.736824+0000","last_fullsized":"2023-10-18T09:00:16.736824+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:05:46.945517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.753907+0000","last_change":"2023-10-18T09:00:14.723490+0000","last_active":"2023-10-18T09:00:15.753907+0000","last_peered":"2023-10-18T09:00:15.753907+0000","last_clean":"2023-10-18T09:00:15.753907+0000","last_became_active":"2023-10-18T09:00:14.723220+0000","last_became_peered":"2023-10-18T09:00:14.723220+0000","last_unstale":"2023-10-18T09:00:15.753907+0000","last_undegraded":"2023-10-18T09:00:15.753907+0000","last_fullsized":"2023-10-18T09:00:15.753907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:03.178214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.870157+0000","last_change":"2023-10-18T09:00:15.753547+0000","last_active":"2023-10-18T09:00:16.870157+0000","last_peered":"2023-10-18T09:00:16.870157+0000","last_clean":"2023-10-18T09:00:16.870157+0000","last_became_active":"2023-10-18T09:00:15.751658+0000","last_became_peered":"2023-10-18T09:00:15.751658+0000","last_unstale":"2023-10-18T09:00:16.870157+0000","last_undegraded":"2023-10-18T09:00:16.870157+0000","last_fullsized":"2023-10-18T09:00:16.870157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:47.354382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.752338+0000","last_change":"2023-10-18T09:00:14.716730+0000","last_active":"2023-10-18T09:00:15.752338+0000","last_peered":"2023-10-18T09:00:15.752338+0000","last_clean":"2023-10-18T09:00:15.752338+0000","last_became_active":"2023-10-18T09:00:14.716479+0000","last_became_peered":"2023-10-18T09:00:14.716479+0000","last_unstale":"2023-10-18T09:00:15.752338+0000","last_undegraded":"2023-10-18T09:00:15.752338+0000","last_fullsized":"2023-10-18T09:00:15.752338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:42.805546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866334+0000","last_change":"2023-10-18T09:00:15.760295+0000","last_active":"2023-10-18T09:00:16.866334+0000","last_peered":"2023-10-18T09:00:16.866334+0000","last_clean":"2023-10-18T09:00:16.866334+0000","last_became_active":"2023-10-18T09:00:15.760058+0000","last_became_peered":"2023-10-18T09:00:15.760058+0000","last_unstale":"2023-10-18T09:00:16.866334+0000","last_undegraded":"2023-10-18T09:00:16.866334+0000","last_fullsized":"2023-10-18T09:00:16.866334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:01.438940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.728164+0000","last_change":"2023-10-18T09:00:14.724630+0000","last_active":"2023-10-18T09:00:15.728164+0000","last_peered":"2023-10-18T09:00:15.728164+0000","last_clean":"2023-10-18T09:00:15.728164+0000","last_became_active":"2023-10-18T09:00:14.724028+0000","last_became_peered":"2023-10-18T09:00:14.724028+0000","last_unstale":"2023-10-18T09:00:15.728164+0000","last_undegraded":"2023-10-18T09:00:15.728164+0000","last_fullsized":"2023-10-18T09:00:15.728164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:53:10.671218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867111+0000","last_change":"2023-10-18T09:00:15.766128+0000","last_active":"2023-10-18T09:00:16.867111+0000","last_peered":"2023-10-18T09:00:16.867111+0000","last_clean":"2023-10-18T09:00:16.867111+0000","last_became_active":"2023-10-18T09:00:15.765583+0000","last_became_peered":"2023-10-18T09:00:15.765583+0000","last_unstale":"2023-10-18T09:00:16.867111+0000","last_undegraded":"2023-10-18T09:00:16.867111+0000","last_fullsized":"2023-10-18T09:00:16.867111+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:26:45.228100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.758442+0000","last_change":"2023-10-18T09:00:14.727160+0000","last_active":"2023-10-18T09:00:15.758442+0000","last_peered":"2023-10-18T09:00:15.758442+0000","last_clean":"2023-10-18T09:00:15.758442+0000","last_became_active":"2023-10-18T09:00:14.726471+0000","last_became_peered":"2023-10-18T09:00:14.726471+0000","last_unstale":"2023-10-18T09:00:15.758442+0000","last_undegraded":"2023-10-18T09:00:15.758442+0000","last_fullsized":"2023-10-18T09:00:15.758442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:11:15.412520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.735001+0000","last_change":"2023-10-18T09:00:15.765588+0000","last_active":"2023-10-18T09:00:16.735001+0000","last_peered":"2023-10-18T09:00:16.735001+0000","last_clean":"2023-10-18T09:00:16.735001+0000","last_became_active":"2023-10-18T09:00:15.765331+0000","last_became_peered":"2023-10-18T09:00:15.765331+0000","last_unstale":"2023-10-18T09:00:16.735001+0000","last_undegraded":"2023-10-18T09:00:16.735001+0000","last_fullsized":"2023-10-18T09:00:16.735001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:44:03.498540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773229+0000","last_change":"2023-10-18T09:00:14.726818+0000","last_active":"2023-10-18T09:00:15.773229+0000","last_peered":"2023-10-18T09:00:15.773229+0000","last_clean":"2023-10-18T09:00:15.773229+0000","last_became_active":"2023-10-18T09:00:14.726391+0000","last_became_peered":"2023-10-18T09:00:14.726391+0000","last_unstale":"2023-10-18T09:00:15.773229+0000","last_undegraded":"2023-10-18T09:00:15.773229+0000","last_fullsized":"2023-10-18T09:00:15.773229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:05:08.365526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":238,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.734462+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:00:15.734462+0000","last_peered":"2023-10-18T09:00:15.734462+0000","last_clean":"2023-10-18T09:00:15.734462+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:00:15.734462+0000","last_undegraded":"2023-10-18T09:00:15.734462+0000","last_fullsized":"2023-10-18T09:00:15.734462+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:36.150056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1032,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194280,"statfs":{"total":200000000,"available":198943213,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.071,"15min":1.0149999999999999},"min":{"1min":0.72599999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.575,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.165,"5min":1.1000000000000001,"15min":1.03},"min":{"1min":0.84399999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.536,"5min":1.97,"15min":1.97},"last":1.5740000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.125,"15min":1.147},"min":{"1min":0.88200000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5169999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1659999999999999,"15min":1.1299999999999999},"min":{"1min":0.91800000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.083}]},{"osd":2,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.284,"15min":1.246},"min":{"1min":0.78600000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.522,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.524},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.236,"15min":1.2030000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.637,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.202}]},{"osd":3,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.2609999999999999,"15min":1.2629999999999999},"min":{"1min":0.80300000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.845,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3500000000000001,"15min":1.327},"min":{"1min":1.0069999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.736,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.389}]},{"osd":4,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.238,"15min":1.1759999999999999},"min":{"1min":0.94599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.825,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.393,"5min":1.351,"15min":1.3919999999999999},"min":{"1min":1.101,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.627,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.131}]},{"osd":5,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3580000000000001,"15min":1.355},"min":{"1min":0.89600000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7470000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.415,"15min":1.421},"min":{"1min":1.079,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7130000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.45}]},{"osd":6,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.482,"15min":1.464},"min":{"1min":1.355,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.8169999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.306},{"interface":"front","average":{"1min":1.512,"5min":1.484,"15min":1.5069999999999999},"min":{"1min":1.214,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.768,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.421}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":454,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194857,"statfs":{"total":200000000,"available":199534105,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.1759999999999999,"15min":1.105},"min":{"1min":0.71799999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.627,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.411},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.2709999999999999,"15min":1.248},"min":{"1min":0.86699999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7629999999999999,"5min":1.855,"15min":1.855},"last":1.6319999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1040000000000001,"15min":1.0900000000000001},"min":{"1min":0.68500000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.488,"5min":1.633,"15min":1.633},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.198,"15min":1.1919999999999999},"min":{"1min":0.72599999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5409999999999999,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":1.2430000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.879,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5720000000000001,"5min":1.696,"15min":1.696},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2170000000000001,"15min":1.2010000000000001},"min":{"1min":0.90000000000000002,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.667,"5min":2.137,"15min":2.137},"last":1.5660000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.256,"15min":1.2589999999999999},"min":{"1min":0.91900000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.6719999999999999,"5min":1.784,"15min":1.784},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.248,"15min":1.226},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.6479999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.006}]},{"osd":4,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.2170000000000001,"15min":1.177},"min":{"1min":0.748,"5min":0.625,"15min":0.625},"max":{"1min":1.389,"5min":2.048,"15min":2.048},"last":1.367},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.262,"15min":1.2669999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.135}]},{"osd":5,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3819999999999999,"15min":1.3580000000000001},"min":{"1min":0.625,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.534},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2589999999999999,"15min":1.2250000000000001},"min":{"1min":0.71899999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.51}]},{"osd":7,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.391,"15min":1.3919999999999999},"min":{"1min":0.93500000000000005,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5900000000000001,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4910000000000001,"15min":1.5189999999999999},"min":{"1min":0.83099999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.944,"5min":2.794,"15min":2.794},"last":1.5880000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":454,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194857,"statfs":{"total":200000000,"available":199534107,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.345,"15min":1.2330000000000001},"min":{"1min":0.71999999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2480000000000002,"5min":9.3539999999999992,"15min":9.3539999999999992},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3260000000000001,"15min":1.304},"min":{"1min":0.77000000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.8260000000000001,"5min":5.3029999999999999,"15min":5.3029999999999999},"last":0.91900000000000004}]},{"osd":1,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.194,"15min":1.218},"min":{"1min":0.85099999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.1499999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.524},{"interface":"front","average":{"1min":1.272,"5min":1.1910000000000001,"15min":1.2370000000000001},"min":{"1min":0.78500000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.2410000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":1.1259999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2450000000000001,"15min":1.26},"min":{"1min":0.88900000000000001,"5min":0.755,"15min":0.755},"max":{"1min":2.0350000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.2629999999999999,"15min":1.284},"min":{"1min":0.95499999999999996,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.153,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.026}]},{"osd":3,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2230000000000001,"15min":1.2370000000000001},"min":{"1min":0.86899999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.613,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":1.169},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3440000000000001,"15min":1.3340000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.633}]},{"osd":4,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.393,"15min":1.4119999999999999},"min":{"1min":0.93899999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.4449999999999998,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.778},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3919999999999999,"15min":1.498},"min":{"1min":0.91700000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.4049999999999998,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":1.224}]},{"osd":6,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2989999999999999,"15min":1.3859999999999999},"min":{"1min":0.91900000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.72,"5min":5.141,"15min":5.141},"last":1.268},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.508,"15min":1.571},"min":{"1min":0.95299999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":1.718}]},{"osd":7,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.55,"15min":1.6399999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.5819999999999999,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":1.347},{"interface":"front","average":{"1min":1.385,"5min":1.4610000000000001,"15min":1.5069999999999999},"min":{"1min":1.091,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8240000000000001,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":1.476}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":453,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194858,"statfs":{"total":200000000,"available":199535146,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.80900000000000005,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7669999999999999,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.01},{"interface":"front","average":{"1min":1.25,"5min":1.125,"15min":1.1679999999999999},"min":{"1min":0.71199999999999997,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.7749999999999999,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.2809999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.089,"15min":1.0920000000000001},"min":{"1min":0.749,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4990000000000001,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":1.331},{"interface":"front","average":{"1min":1.23,"5min":1.1930000000000001,"15min":1.2290000000000001},"min":{"1min":0.70399999999999996,"5min":0.627,"15min":0.627},"max":{"1min":1.3740000000000001,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.123}]},{"osd":2,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1890000000000001,"15min":1.179},"min":{"1min":0.81999999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4350000000000001,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.2709999999999999,"15min":1.3029999999999999},"min":{"1min":0.85899999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6930000000000001,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.224}]},{"osd":3,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.89700000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9179999999999999,"5min":2.649,"15min":2.649},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.403,"15min":1.409},"min":{"1min":0.86799999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6399999999999999,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.4650000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.319,"15min":1.3029999999999999},"min":{"1min":0.96399999999999997,"5min":0.498,"15min":0.498},"max":{"1min":1.7649999999999999,"5min":1.923,"15min":1.923},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.359,"5min":1.3180000000000001,"15min":1.3560000000000001},"min":{"1min":0.92100000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.0720000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":0.94499999999999995}]},{"osd":6,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4379999999999999,"15min":1.3959999999999999},"min":{"1min":0.96999999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.1150000000000002,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.508,"15min":1.5580000000000001},"min":{"1min":1.2250000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6839999999999999,"5min":4.407,"15min":4.407},"last":1.385}]},{"osd":7,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.46,"15min":1.502},"min":{"1min":1.0169999999999999,"5min":0.438,"15min":0.438},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.508,"5min":1.54,"15min":1.546},"min":{"1min":1.196,"5min":0.747,"15min":0.747},"max":{"1min":1.7270000000000001,"5min":2.585,"15min":2.585},"last":1.6259999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":454,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194857,"statfs":{"total":200000000,"available":199534085,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.90600000000000003,"15min":0.79200000000000004},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2969999999999999,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.012,"15min":0.89300000000000002},"min":{"1min":0.48999999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.524,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":0.99099999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0309999999999999,"15min":0.95899999999999996},"min":{"1min":0.60099999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.633,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.2250000000000001,"15min":1.141},"min":{"1min":0.622,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.786,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.1379999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.079,"15min":1.0229999999999999},"min":{"1min":0.55200000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.675,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":0.755},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.163,"15min":1.073},"min":{"1min":0.59899999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.708,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":1.2569999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.5029999999999999,"15min":1.3180000000000001},"min":{"1min":0.91000000000000003,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.593,"5min":18.664999999999999,"15min":18.664999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.274,"15min":1.1719999999999999},"min":{"1min":0.93899999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.746,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":0.877}]},{"osd":5,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.3129999999999999,"15min":1.2290000000000001},"min":{"1min":1.014,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5700000000000001,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.5700000000000001,"15min":1.363},"min":{"1min":0.78300000000000003,"5min":0.748,"15min":0.748},"max":{"1min":1.9179999999999999,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.478}]},{"osd":6,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.5209999999999999,"15min":1.4350000000000001},"min":{"1min":1.115,"5min":0.625,"15min":0.625},"max":{"1min":1.7949999999999999,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4650000000000001,"15min":1.353},"min":{"1min":1.1339999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.772,"5min":7.242,"15min":7.242},"last":1.3959999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4870000000000001,"15min":1.361},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9950000000000001,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.379},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.6060000000000001,"15min":1.464},"min":{"1min":1.014,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0329999999999999,"5min":12.946,"15min":12.946},"last":1.4319999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":456,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194856,"statfs":{"total":200000000,"available":199532585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.87,"15min":0.92400000000000004},"min":{"1min":0.48599999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93400000000000005,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":0.873},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.96599999999999997,"15min":1.038},"min":{"1min":0.623,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3560000000000001,"5min":1.651,"15min":1.651},"last":1.123}]},{"osd":1,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.996,"15min":1.0449999999999999},"min":{"1min":0.67500000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3120000000000001,"5min":4.29,"15min":4.29},"last":1.04},{"interface":"front","average":{"1min":1.226,"5min":1.1020000000000001,"15min":1.1040000000000001},"min":{"1min":0.75,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5820000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.3320000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.0940000000000001,"15min":1.129},"min":{"1min":0.72599999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8109999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.345,"15min":1.4650000000000001},"min":{"1min":0.73399999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.7929999999999999,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":1.383}]},{"osd":4,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.403,"15min":1.48},"min":{"1min":0.95299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.6479999999999999,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.407,"15min":1.4470000000000001},"min":{"1min":1.1319999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.766,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.454}]},{"osd":5,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.244,"15min":1.2709999999999999},"min":{"1min":1.119,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.786,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.446,"15min":1.5580000000000001},"min":{"1min":1.204,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.794,"5min":7.657,"15min":7.657},"last":1.6299999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.4430000000000001,"15min":1.5209999999999999},"min":{"1min":1.2070000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0649999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.51,"5min":1.8420000000000001,"15min":1.5780000000000001},"min":{"1min":1.0069999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.9350000000000001,"5min":37.771999999999998,"15min":37.771999999999998},"last":1.8089999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.506,"15min":1.5649999999999999},"min":{"1min":1.1890000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.1930000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.587,"5min":1.4670000000000001,"15min":1.48},"min":{"1min":1.056,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1139999999999999,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.905}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":456,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194856,"statfs":{"total":200000000,"available":199532988,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.83599999999999997,"15min":0.81200000000000006},"min":{"1min":0.51000000000000001,"5min":0.315,"15min":0.315},"max":{"1min":1.343,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.302},{"interface":"front","average":{"1min":1.145,"5min":0.97999999999999998,"15min":0.92000000000000004},"min":{"1min":0.66600000000000004,"5min":0.443,"15min":0.443},"max":{"1min":1.399,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.2370000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":0.96399999999999997,"15min":0.93999999999999995},"min":{"1min":0.54900000000000004,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.456,"5min":1.859,"15min":1.859},"last":1.165},{"interface":"front","average":{"1min":1.028,"5min":0.90700000000000003,"15min":0.85999999999999999},"min":{"1min":0.70099999999999996,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.522,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.89200000000000002}]},{"osd":3,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.048,"15min":1.03},"min":{"1min":0.68500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.617,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.052,"15min":1.0569999999999999},"min":{"1min":0.82499999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5640000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":1.024}]},{"osd":4,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.2709999999999999,"15min":1.252},"min":{"1min":1.089,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.716,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.2749999999999999,"15min":1.2190000000000001},"min":{"1min":0.99199999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.774,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.276}]},{"osd":5,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.3320000000000001,"15min":1.302},"min":{"1min":1.111,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.161,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.345},{"interface":"front","average":{"1min":1.456,"5min":1.3979999999999999,"15min":1.3620000000000001},"min":{"1min":1.0820000000000001,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.7549999999999999,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.397}]},{"osd":6,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.3360000000000001,"15min":1.2929999999999999},"min":{"1min":1.079,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.857,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.3380000000000001,"15min":1.2789999999999999},"min":{"1min":1.0980000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.75,"5min":2.121,"15min":2.121},"last":1.423}]},{"osd":7,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.419,"15min":1.367},"min":{"1min":1.175,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.992,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.423,"15min":1.3680000000000001},"min":{"1min":1.256,"5min":0.746,"15min":0.746},"max":{"1min":2.0569999999999999,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.49}]}]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1030,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194282,"statfs":{"total":200000000,"available":198944848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84299999999999997,"15min":0.91900000000000004},"min":{"1min":0.65900000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.026,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":1.006},{"interface":"front","average":{"1min":1.171,"5min":1.075,"15min":1.0820000000000001},"min":{"1min":0.70199999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.70199999999999996}]},{"osd":2,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.86199999999999999,"15min":0.84999999999999998},"min":{"1min":0.77800000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.4159999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.143,"15min":1.131},"min":{"1min":0.63700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":1.24}]},{"osd":3,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.163,"15min":1.2869999999999999},"min":{"1min":0.69599999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.8200000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.494,"5min":1.4179999999999999,"15min":1.512},"min":{"1min":0.70099999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.3639999999999999,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.369}]},{"osd":4,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2330000000000001,"15min":1.3069999999999999},"min":{"1min":0.97999999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.8819999999999999,"5min":4.492,"15min":4.492},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.2809999999999999,"15min":1.2569999999999999},"min":{"1min":1.0449999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.5960000000000001,"5min":1.726,"15min":1.726},"last":1.526}]},{"osd":5,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.296,"15min":1.4219999999999999},"min":{"1min":0.85299999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":2.4510000000000001,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.284},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.3380000000000001,"15min":1.379},"min":{"1min":0.98999999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.784,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.464}]},{"osd":6,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.4970000000000001,"15min":1.5880000000000001},"min":{"1min":1.2509999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.0419999999999998,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4139999999999999,"15min":1.5009999999999999},"min":{"1min":1.159,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.1059999999999999,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.6879999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.631,"5min":1.569,"15min":1.645},"min":{"1min":1.2729999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.2000000000000002,"5min":3.492,"15min":3.492},"last":1.581},{"interface":"front","average":{"1min":1.536,"5min":1.512,"15min":1.6759999999999999},"min":{"1min":1.069,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.0009999999999999,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.425}]}]}],"pool_statfs":[]}} 2023-10-18T09:00:21.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T09:00:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.593+0000 7fa8a66e5640 1 -- 172.21.15.40:0/4293071820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a006d710 msgr2=0x7fa8a0158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.593+0000 7fa8a66e5640 1 --2- 172.21.15.40:0/4293071820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a006d710 0x7fa8a0158ca0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa8a006b290 tx=0x7fa89002bd90 comp rx=0 tx=0).stop 2023-10-18T09:00:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.594+0000 7fa8a66e5640 1 -- 172.21.15.40:0/4293071820 shutdown_connections 2023-10-18T09:00:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.594+0000 7fa8a66e5640 1 --2- 172.21.15.40:0/4293071820 >> v2:172.21.15.144:3300/0 conn(0x7fa8a0158140 0x7fa8a00b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.594+0000 7fa8a66e5640 1 --2- 172.21.15.40:0/4293071820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8a0151cb0 0x7fa8a0157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.594+0000 7fa8a66e5640 1 --2- 172.21.15.40:0/4293071820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a006d710 0x7fa8a0158ca0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.594+0000 7fa8a66e5640 1 -- 172.21.15.40:0/4293071820 >> 172.21.15.40:0/4293071820 conn(0x7fa8a0069720 msgr2=0x7fa8a0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:21.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.594+0000 7fa8a66e5640 1 -- 172.21.15.40:0/4293071820 shutdown_connections 2023-10-18T09:00:21.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.594+0000 7fa8a66e5640 1 -- 172.21.15.40:0/4293071820 wait complete. 2023-10-18T09:00:21.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.595+0000 7fa8a66e5640 1 Processor -- start 2023-10-18T09:00:21.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.596+0000 7fa8a66e5640 1 -- start start 2023-10-18T09:00:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.597+0000 7fa8a66e5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8a006d710 0x7fa8a0152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.597+0000 7fa8a66e5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a0151cb0 0x7fa8a0141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.597+0000 7fa8a66e5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8a0158140 0x7fa8a0163570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.597+0000 7fa8a66e5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00ad9a0 con 0x7fa8a0151cb0 2023-10-18T09:00:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.597+0000 7fa8a66e5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0142410 con 0x7fa8a006d710 2023-10-18T09:00:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.597+0000 7fa8a66e5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0142d70 con 0x7fa8a0158140 2023-10-18T09:00:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.597+0000 7fa897fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a0151cb0 0x7fa8a0141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.597+0000 7fa897fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a0151cb0 0x7fa8a0141ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48506/0 (socket says 172.21.15.40:48506) 2023-10-18T09:00:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.597+0000 7fa897fff640 1 -- 172.21.15.40:0/1005624592 learned_addr learned my addr 172.21.15.40:0/1005624592 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.597+0000 7fa89ffff640 1 --2- 172.21.15.40:0/1005624592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8a006d710 0x7fa8a0152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.597+0000 7fa897fff640 1 -- 172.21.15.40:0/1005624592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8a0158140 msgr2=0x7fa8a0163570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.597+0000 7fa897fff640 1 --2- 172.21.15.40:0/1005624592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8a0158140 0x7fa8a0163570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.597+0000 7fa897fff640 1 -- 172.21.15.40:0/1005624592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8a006d710 msgr2=0x7fa8a0152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.597+0000 7fa897fff640 1 --2- 172.21.15.40:0/1005624592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8a006d710 0x7fa8a0152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.597+0000 7fa897fff640 1 -- 172.21.15.40:0/1005624592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa890009560 con 0x7fa8a0151cb0 2023-10-18T09:00:21.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.597+0000 7fa89ffff640 1 --2- 172.21.15.40:0/1005624592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8a006d710 0x7fa8a0152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:21.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.598+0000 7fa897fff640 1 --2- 172.21.15.40:0/1005624592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a0151cb0 0x7fa8a0141ed0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa88800d780 tx=0x7fa88800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:21.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.598+0000 7fa89d7fa640 1 -- 172.21.15.40:0/1005624592 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888009bd0 con 0x7fa8a0151cb0 2023-10-18T09:00:21.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.598+0000 7fa89d7fa640 1 -- 172.21.15.40:0/1005624592 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa888009e20 con 0x7fa8a0151cb0 2023-10-18T09:00:21.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.598+0000 7fa89d7fa640 1 -- 172.21.15.40:0/1005624592 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888005280 con 0x7fa8a0151cb0 2023-10-18T09:00:21.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.598+0000 7fa8a66e5640 1 -- 172.21.15.40:0/1005624592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a0142a90 con 0x7fa8a0151cb0 2023-10-18T09:00:21.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.598+0000 7fa8a66e5640 1 -- 172.21.15.40:0/1005624592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a00ad5c0 con 0x7fa8a0151cb0 2023-10-18T09:00:21.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.599+0000 7fa8a66e5640 1 -- 172.21.15.40:0/1005624592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a00ad140 con 0x7fa8a0151cb0 2023-10-18T09:00:21.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.600+0000 7fa89d7fa640 1 -- 172.21.15.40:0/1005624592 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa8880053e0 con 0x7fa8a0151cb0 2023-10-18T09:00:21.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.601+0000 7fa89d7fa640 1 --2- 172.21.15.40:0/1005624592 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa88006ae40 0x7fa88006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:21.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.601+0000 7fa89d7fa640 1 -- 172.21.15.40:0/1005624592 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8880897e0 con 0x7fa8a0151cb0 2023-10-18T09:00:21.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.602+0000 7fa89ffff640 1 --2- 172.21.15.40:0/1005624592 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa88006ae40 0x7fa88006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:21.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.603+0000 7fa89ffff640 1 --2- 172.21.15.40:0/1005624592 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa88006ae40 0x7fa88006d300 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa890035000 tx=0x7fa890004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:21.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.606+0000 7fa89d7fa640 1 -- 172.21.15.40:0/1005624592 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa8a00ad140 con 0x7fa8a0151cb0 2023-10-18T09:00:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.875+0000 7fa8a66e5640 1 -- 172.21.15.40:0/1005624592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa8a00adae0 con 0x7fa8a0151cb0 2023-10-18T09:00:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.876+0000 7fa89d7fa640 1 -- 172.21.15.40:0/1005624592 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1103 (secure 0 0 0) 0x7fa888017460 con 0x7fa8a0151cb0 2023-10-18T09:00:21.878 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:21.878 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":326,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":87,"num_osds":8,"num_up_osds":8,"osd_up_since":1697619304,"num_in_osds":8,"osd_in_since":1697619300,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":46},{"state_name":"creating+peering","count":46},{"state_name":"active+clean","count":5}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":4720218,"bytes_avail":1595279782,"bytes_total":1600000000,"unknown_pgs_ratio":0.47422680258750916,"inactive_pgs_ratio":0.47422680258750916},"fsmap":{"epoch":41,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":6573}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2023-10-18T08:59:36.990753+0000","services":{}},"progress_events":{}} 2023-10-18T09:00:21.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.879+0000 7fa8a66e5640 1 -- 172.21.15.40:0/1005624592 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa88006ae40 msgr2=0x7fa88006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:21.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.879+0000 7fa8a66e5640 1 --2- 172.21.15.40:0/1005624592 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa88006ae40 0x7fa88006d300 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa890035000 tx=0x7fa890004730 comp rx=0 tx=0).stop 2023-10-18T09:00:21.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.879+0000 7fa8a66e5640 1 -- 172.21.15.40:0/1005624592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a0151cb0 msgr2=0x7fa8a0141ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:21.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.879+0000 7fa8a66e5640 1 --2- 172.21.15.40:0/1005624592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a0151cb0 0x7fa8a0141ed0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa88800d780 tx=0x7fa88800dc40 comp rx=0 tx=0).stop 2023-10-18T09:00:21.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.879+0000 7fa8a66e5640 1 -- 172.21.15.40:0/1005624592 shutdown_connections 2023-10-18T09:00:21.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.879+0000 7fa8a66e5640 1 --2- 172.21.15.40:0/1005624592 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa88006ae40 0x7fa88006d300 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.879+0000 7fa8a66e5640 1 --2- 172.21.15.40:0/1005624592 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8a0158140 0x7fa8a0163570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.879+0000 7fa8a66e5640 1 --2- 172.21.15.40:0/1005624592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8a0151cb0 0x7fa8a0141ed0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.879+0000 7fa8a66e5640 1 --2- 172.21.15.40:0/1005624592 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8a006d710 0x7fa8a0152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:21.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.879+0000 7fa8a66e5640 1 -- 172.21.15.40:0/1005624592 >> 172.21.15.40:0/1005624592 conn(0x7fa8a0069720 msgr2=0x7fa8a0156e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:21.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.879+0000 7fa8a66e5640 1 -- 172.21.15.40:0/1005624592 shutdown_connections 2023-10-18T09:00:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:21.879+0000 7fa8a66e5640 1 -- 172.21.15.40:0/1005624592 wait complete. 2023-10-18T09:00:21.902 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T09:00:21.902 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:00:22.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.126+0000 7f83aa1a5640 1 -- 172.21.15.40:0/71314367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a40b15b0 msgr2=0x7f83a40c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:22.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.126+0000 7f83aa1a5640 1 --2- 172.21.15.40:0/71314367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a40b15b0 0x7f83a40c99c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8394009830 tx=0x7f839402bd60 comp rx=0 tx=0).stop 2023-10-18T09:00:22.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.126+0000 7f83aa1a5640 1 -- 172.21.15.40:0/71314367 shutdown_connections 2023-10-18T09:00:22.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.126+0000 7f83aa1a5640 1 --2- 172.21.15.40:0/71314367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a40b15b0 0x7f83a40c99c0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:22.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.126+0000 7f83aa1a5640 1 -- 172.21.15.40:0/71314367 >> 172.21.15.40:0/71314367 conn(0x7f83a4069720 msgr2=0x7f83a4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:22.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.126+0000 7f83aa1a5640 1 -- 172.21.15.40:0/71314367 shutdown_connections 2023-10-18T09:00:22.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.126+0000 7f83aa1a5640 1 -- 172.21.15.40:0/71314367 wait complete. 2023-10-18T09:00:22.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.128+0000 7f83aa1a5640 1 Processor -- start 2023-10-18T09:00:22.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.128+0000 7f83aa1a5640 1 -- start start 2023-10-18T09:00:22.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.129+0000 7f83aa1a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a406d6f0 0x7f83a40c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:22.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.129+0000 7f83aa1a5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83a40b15b0 0x7f83a4154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:22.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.129+0000 7f83aa1a5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83a40c9f00 0x7f83a40acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:22.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.129+0000 7f83aa1a5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a4151930 con 0x7f83a406d6f0 2023-10-18T09:00:22.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.129+0000 7f83aa1a5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a40af360 con 0x7f83a40c9f00 2023-10-18T09:00:22.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.129+0000 7f83aa1a5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a40af640 con 0x7f83a40b15b0 2023-10-18T09:00:22.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.130+0000 7f83a91a3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a406d6f0 0x7f83a40c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:22.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.130+0000 7f83a91a3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a406d6f0 0x7f83a40c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48520/0 (socket says 172.21.15.40:48520) 2023-10-18T09:00:22.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.130+0000 7f83a91a3640 1 -- 172.21.15.40:0/3646677694 learned_addr learned my addr 172.21.15.40:0/3646677694 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:22.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.130+0000 7f83a89a2640 1 --2- 172.21.15.40:0/3646677694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83a40b15b0 0x7f83a4154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:22.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.130+0000 7f83a91a3640 1 -- 172.21.15.40:0/3646677694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83a40b15b0 msgr2=0x7f83a4154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:22.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.130+0000 7f83a91a3640 1 --2- 172.21.15.40:0/3646677694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83a40b15b0 0x7f83a4154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:22.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.130+0000 7f83a91a3640 1 -- 172.21.15.40:0/3646677694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83a40c9f00 msgr2=0x7f83a40acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:22.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.130+0000 7f83a91a3640 1 --2- 172.21.15.40:0/3646677694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83a40c9f00 0x7f83a40acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:22.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.130+0000 7f83a91a3640 1 -- 172.21.15.40:0/3646677694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8394009490 con 0x7f83a406d6f0 2023-10-18T09:00:22.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.130+0000 7f83a91a3640 1 --2- 172.21.15.40:0/3646677694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a406d6f0 0x7f83a40c9040 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f83a000c7c0 tx=0x7f83a000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:22.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.131+0000 7f8399ffb640 1 -- 172.21.15.40:0/3646677694 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a000aa50 con 0x7f83a406d6f0 2023-10-18T09:00:22.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.131+0000 7f8399ffb640 1 -- 172.21.15.40:0/3646677694 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f83a000aca0 con 0x7f83a406d6f0 2023-10-18T09:00:22.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.131+0000 7f8399ffb640 1 -- 172.21.15.40:0/3646677694 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a0012680 con 0x7f83a406d6f0 2023-10-18T09:00:22.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.131+0000 7f83aa1a5640 1 -- 172.21.15.40:0/3646677694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a40c1fe0 con 0x7f83a406d6f0 2023-10-18T09:00:22.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.131+0000 7f83aa1a5640 1 -- 172.21.15.40:0/3646677694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a40c25a0 con 0x7f83a406d6f0 2023-10-18T09:00:22.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.132+0000 7f83aa1a5640 1 -- 172.21.15.40:0/3646677694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a4066e00 con 0x7f83a406d6f0 2023-10-18T09:00:22.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.133+0000 7f8399ffb640 1 -- 172.21.15.40:0/3646677694 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f83a001a3b0 con 0x7f83a406d6f0 2023-10-18T09:00:22.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.134+0000 7f8399ffb640 1 --2- 172.21.15.40:0/3646677694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f838806ae40 0x7f838806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:22.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.134+0000 7f8399ffb640 1 -- 172.21.15.40:0/3646677694 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83a00897e0 con 0x7f83a406d6f0 2023-10-18T09:00:22.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.135+0000 7f83a89a2640 1 --2- 172.21.15.40:0/3646677694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f838806ae40 0x7f838806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:22.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.136+0000 7f83a89a2640 1 --2- 172.21.15.40:0/3646677694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f838806ae40 0x7f838806d300 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8394036000 tx=0x7f8394004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:22.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.138+0000 7f8399ffb640 1 -- 172.21.15.40:0/3646677694 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f83a0054880 con 0x7f83a406d6f0 2023-10-18T09:00:22.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.329+0000 7f83aa1a5640 1 -- 172.21.15.40:0/3646677694 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f83a415bec0 con 0x7f838806ae40 2023-10-18T09:00:22.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.352+0000 7f8399ffb640 1 -- 172.21.15.40:0/3646677694 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282230 (secure 0 0 0) 0x7f8394004040 con 0x7f838806ae40 2023-10-18T09:00:22.356 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:22.360 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:00:22.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.360+0000 7f83aa1a5640 1 -- 172.21.15.40:0/3646677694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f838806ae40 msgr2=0x7f838806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:22.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.360+0000 7f83aa1a5640 1 --2- 172.21.15.40:0/3646677694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f838806ae40 0x7f838806d300 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8394036000 tx=0x7f8394004730 comp rx=0 tx=0).stop 2023-10-18T09:00:22.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.360+0000 7f83aa1a5640 1 -- 172.21.15.40:0/3646677694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a406d6f0 msgr2=0x7f83a40c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:22.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.360+0000 7f83aa1a5640 1 --2- 172.21.15.40:0/3646677694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a406d6f0 0x7f83a40c9040 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f83a000c7c0 tx=0x7f83a000cc80 comp rx=0 tx=0).stop 2023-10-18T09:00:22.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.360+0000 7f83aa1a5640 1 -- 172.21.15.40:0/3646677694 shutdown_connections 2023-10-18T09:00:22.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.360+0000 7f83aa1a5640 1 --2- 172.21.15.40:0/3646677694 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f838806ae40 0x7f838806d300 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:22.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.360+0000 7f83aa1a5640 1 --2- 172.21.15.40:0/3646677694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83a40c9f00 0x7f83a40acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:22.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.360+0000 7f83aa1a5640 1 --2- 172.21.15.40:0/3646677694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83a40b15b0 0x7f83a4154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:22.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.360+0000 7f83aa1a5640 1 --2- 172.21.15.40:0/3646677694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a406d6f0 0x7f83a40c9040 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.360+0000 7f83aa1a5640 1 -- 172.21.15.40:0/3646677694 >> 172.21.15.40:0/3646677694 conn(0x7f83a4069720 msgr2=0x7f83a40abd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.361+0000 7f83aa1a5640 1 -- 172.21.15.40:0/3646677694 shutdown_connections 2023-10-18T09:00:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:22.361+0000 7f83aa1a5640 1 -- 172.21.15.40:0/3646677694 wait complete. 2023-10-18T09:00:22.385 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2023-10-18T09:00:21.005517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4793,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557706,"statfs":{"total":1600000000,"available":1595091077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.304691"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736891+0000","last_change":"2023-10-18T09:00:15.763191+0000","last_active":"2023-10-18T09:00:16.736891+0000","last_peered":"2023-10-18T09:00:16.736891+0000","last_clean":"2023-10-18T09:00:16.736891+0000","last_became_active":"2023-10-18T09:00:15.758469+0000","last_became_peered":"2023-10-18T09:00:15.758469+0000","last_unstale":"2023-10-18T09:00:16.736891+0000","last_undegraded":"2023-10-18T09:00:16.736891+0000","last_fullsized":"2023-10-18T09:00:16.736891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:03:56.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866602+0000","last_change":"2023-10-18T09:00:15.757911+0000","last_active":"2023-10-18T09:00:16.866602+0000","last_peered":"2023-10-18T09:00:16.866602+0000","last_clean":"2023-10-18T09:00:16.866602+0000","last_became_active":"2023-10-18T09:00:15.757452+0000","last_became_peered":"2023-10-18T09:00:15.757452+0000","last_unstale":"2023-10-18T09:00:16.866602+0000","last_undegraded":"2023-10-18T09:00:16.866602+0000","last_fullsized":"2023-10-18T09:00:16.866602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:19:41.723289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.734969+0000","last_change":"2023-10-18T09:00:15.761182+0000","last_active":"2023-10-18T09:00:16.734969+0000","last_peered":"2023-10-18T09:00:16.734969+0000","last_clean":"2023-10-18T09:00:16.734969+0000","last_became_active":"2023-10-18T09:00:15.760952+0000","last_became_peered":"2023-10-18T09:00:15.760952+0000","last_unstale":"2023-10-18T09:00:16.734969+0000","last_undegraded":"2023-10-18T09:00:16.734969+0000","last_fullsized":"2023-10-18T09:00:16.734969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:54:55.101341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.865987+0000","last_change":"2023-10-18T09:00:15.762841+0000","last_active":"2023-10-18T09:00:16.865987+0000","last_peered":"2023-10-18T09:00:16.865987+0000","last_clean":"2023-10-18T09:00:16.865987+0000","last_became_active":"2023-10-18T09:00:15.762393+0000","last_became_peered":"2023-10-18T09:00:15.762393+0000","last_unstale":"2023-10-18T09:00:16.865987+0000","last_undegraded":"2023-10-18T09:00:16.865987+0000","last_fullsized":"2023-10-18T09:00:16.865987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:51:04.245123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736606+0000","last_change":"2023-10-18T09:00:15.755929+0000","last_active":"2023-10-18T09:00:16.736606+0000","last_peered":"2023-10-18T09:00:16.736606+0000","last_clean":"2023-10-18T09:00:16.736606+0000","last_became_active":"2023-10-18T09:00:15.755639+0000","last_became_peered":"2023-10-18T09:00:15.755639+0000","last_unstale":"2023-10-18T09:00:16.736606+0000","last_undegraded":"2023-10-18T09:00:16.736606+0000","last_fullsized":"2023-10-18T09:00:16.736606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:50.213859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.869497+0000","last_change":"2023-10-18T09:00:15.765807+0000","last_active":"2023-10-18T09:00:16.869497+0000","last_peered":"2023-10-18T09:00:16.869497+0000","last_clean":"2023-10-18T09:00:16.869497+0000","last_became_active":"2023-10-18T09:00:15.765306+0000","last_became_peered":"2023-10-18T09:00:15.765306+0000","last_unstale":"2023-10-18T09:00:16.869497+0000","last_undegraded":"2023-10-18T09:00:16.869497+0000","last_fullsized":"2023-10-18T09:00:16.869497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:28:57.510395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.870713+0000","last_change":"2023-10-18T09:00:15.758255+0000","last_active":"2023-10-18T09:00:16.870713+0000","last_peered":"2023-10-18T09:00:16.870713+0000","last_clean":"2023-10-18T09:00:16.870713+0000","last_became_active":"2023-10-18T09:00:15.756071+0000","last_became_peered":"2023-10-18T09:00:15.756071+0000","last_unstale":"2023-10-18T09:00:16.870713+0000","last_undegraded":"2023-10-18T09:00:16.870713+0000","last_fullsized":"2023-10-18T09:00:16.870713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:46:39.794722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866761+0000","last_change":"2023-10-18T09:00:15.760360+0000","last_active":"2023-10-18T09:00:16.866761+0000","last_peered":"2023-10-18T09:00:16.866761+0000","last_clean":"2023-10-18T09:00:16.866761+0000","last_became_active":"2023-10-18T09:00:15.755866+0000","last_became_peered":"2023-10-18T09:00:15.755866+0000","last_unstale":"2023-10-18T09:00:16.866761+0000","last_undegraded":"2023-10-18T09:00:16.866761+0000","last_fullsized":"2023-10-18T09:00:16.866761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:24:22.148863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736100+0000","last_change":"2023-10-18T09:00:15.758789+0000","last_active":"2023-10-18T09:00:16.736100+0000","last_peered":"2023-10-18T09:00:16.736100+0000","last_clean":"2023-10-18T09:00:16.736100+0000","last_became_active":"2023-10-18T09:00:15.758487+0000","last_became_peered":"2023-10-18T09:00:15.758487+0000","last_unstale":"2023-10-18T09:00:16.736100+0000","last_undegraded":"2023-10-18T09:00:16.736100+0000","last_fullsized":"2023-10-18T09:00:16.736100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:51:02.550237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867399+0000","last_change":"2023-10-18T09:00:15.758617+0000","last_active":"2023-10-18T09:00:16.867399+0000","last_peered":"2023-10-18T09:00:16.867399+0000","last_clean":"2023-10-18T09:00:16.867399+0000","last_became_active":"2023-10-18T09:00:15.758324+0000","last_became_peered":"2023-10-18T09:00:15.758324+0000","last_unstale":"2023-10-18T09:00:16.867399+0000","last_undegraded":"2023-10-18T09:00:16.867399+0000","last_fullsized":"2023-10-18T09:00:16.867399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:03:24.223447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737441+0000","last_change":"2023-10-18T09:00:15.736563+0000","last_active":"2023-10-18T09:00:16.737441+0000","last_peered":"2023-10-18T09:00:16.737441+0000","last_clean":"2023-10-18T09:00:16.737441+0000","last_became_active":"2023-10-18T09:00:15.736013+0000","last_became_peered":"2023-10-18T09:00:15.736013+0000","last_unstale":"2023-10-18T09:00:16.737441+0000","last_undegraded":"2023-10-18T09:00:16.737441+0000","last_fullsized":"2023-10-18T09:00:16.737441+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:14:19.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866898+0000","last_change":"2023-10-18T09:00:15.763124+0000","last_active":"2023-10-18T09:00:16.866898+0000","last_peered":"2023-10-18T09:00:16.866898+0000","last_clean":"2023-10-18T09:00:16.866898+0000","last_became_active":"2023-10-18T09:00:15.762786+0000","last_became_peered":"2023-10-18T09:00:15.762786+0000","last_unstale":"2023-10-18T09:00:16.866898+0000","last_undegraded":"2023-10-18T09:00:16.866898+0000","last_fullsized":"2023-10-18T09:00:16.866898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:12:39.359221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.865876+0000","last_change":"2023-10-18T09:00:15.768782+0000","last_active":"2023-10-18T09:00:16.865876+0000","last_peered":"2023-10-18T09:00:16.865876+0000","last_clean":"2023-10-18T09:00:16.865876+0000","last_became_active":"2023-10-18T09:00:15.767657+0000","last_became_peered":"2023-10-18T09:00:15.767657+0000","last_unstale":"2023-10-18T09:00:16.865876+0000","last_undegraded":"2023-10-18T09:00:16.865876+0000","last_fullsized":"2023-10-18T09:00:16.865876+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:17:29.238634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736280+0000","last_change":"2023-10-18T09:00:15.752025+0000","last_active":"2023-10-18T09:00:16.736280+0000","last_peered":"2023-10-18T09:00:16.736280+0000","last_clean":"2023-10-18T09:00:16.736280+0000","last_became_active":"2023-10-18T09:00:15.751249+0000","last_became_peered":"2023-10-18T09:00:15.751249+0000","last_unstale":"2023-10-18T09:00:16.736280+0000","last_undegraded":"2023-10-18T09:00:16.736280+0000","last_fullsized":"2023-10-18T09:00:16.736280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:56:40.431057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.865758+0000","last_change":"2023-10-18T09:00:15.761180+0000","last_active":"2023-10-18T09:00:16.865758+0000","last_peered":"2023-10-18T09:00:16.865758+0000","last_clean":"2023-10-18T09:00:16.865758+0000","last_became_active":"2023-10-18T09:00:15.760931+0000","last_became_peered":"2023-10-18T09:00:15.760931+0000","last_unstale":"2023-10-18T09:00:16.865758+0000","last_undegraded":"2023-10-18T09:00:16.865758+0000","last_fullsized":"2023-10-18T09:00:16.865758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:44.909930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866905+0000","last_change":"2023-10-18T09:00:15.755241+0000","last_active":"2023-10-18T09:00:16.866905+0000","last_peered":"2023-10-18T09:00:16.866905+0000","last_clean":"2023-10-18T09:00:16.866905+0000","last_became_active":"2023-10-18T09:00:15.754219+0000","last_became_peered":"2023-10-18T09:00:15.754219+0000","last_unstale":"2023-10-18T09:00:16.866905+0000","last_undegraded":"2023-10-18T09:00:16.866905+0000","last_fullsized":"2023-10-18T09:00:16.866905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:10:37.475877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736469+0000","last_change":"2023-10-18T09:00:15.762556+0000","last_active":"2023-10-18T09:00:16.736469+0000","last_peered":"2023-10-18T09:00:16.736469+0000","last_clean":"2023-10-18T09:00:16.736469+0000","last_became_active":"2023-10-18T09:00:15.762213+0000","last_became_peered":"2023-10-18T09:00:15.762213+0000","last_unstale":"2023-10-18T09:00:16.736469+0000","last_undegraded":"2023-10-18T09:00:16.736469+0000","last_fullsized":"2023-10-18T09:00:16.736469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:13:20.414827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.865832+0000","last_change":"2023-10-18T09:00:15.756905+0000","last_active":"2023-10-18T09:00:16.865832+0000","last_peered":"2023-10-18T09:00:16.865832+0000","last_clean":"2023-10-18T09:00:16.865832+0000","last_became_active":"2023-10-18T09:00:15.753779+0000","last_became_peered":"2023-10-18T09:00:15.753779+0000","last_unstale":"2023-10-18T09:00:16.865832+0000","last_undegraded":"2023-10-18T09:00:16.865832+0000","last_fullsized":"2023-10-18T09:00:16.865832+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:15:47.083054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866960+0000","last_change":"2023-10-18T09:00:15.765485+0000","last_active":"2023-10-18T09:00:16.866960+0000","last_peered":"2023-10-18T09:00:16.866960+0000","last_clean":"2023-10-18T09:00:16.866960+0000","last_became_active":"2023-10-18T09:00:15.764978+0000","last_became_peered":"2023-10-18T09:00:15.764978+0000","last_unstale":"2023-10-18T09:00:16.866960+0000","last_undegraded":"2023-10-18T09:00:16.866960+0000","last_fullsized":"2023-10-18T09:00:16.866960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:28:45.528730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866110+0000","last_change":"2023-10-18T09:00:15.757707+0000","last_active":"2023-10-18T09:00:16.866110+0000","last_peered":"2023-10-18T09:00:16.866110+0000","last_clean":"2023-10-18T09:00:16.866110+0000","last_became_active":"2023-10-18T09:00:15.757478+0000","last_became_peered":"2023-10-18T09:00:15.757478+0000","last_unstale":"2023-10-18T09:00:16.866110+0000","last_undegraded":"2023-10-18T09:00:16.866110+0000","last_fullsized":"2023-10-18T09:00:16.866110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:00.837466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867390+0000","last_change":"2023-10-18T09:00:15.762884+0000","last_active":"2023-10-18T09:00:16.867390+0000","last_peered":"2023-10-18T09:00:16.867390+0000","last_clean":"2023-10-18T09:00:16.867390+0000","last_became_active":"2023-10-18T09:00:15.762479+0000","last_became_peered":"2023-10-18T09:00:15.762479+0000","last_unstale":"2023-10-18T09:00:16.867390+0000","last_undegraded":"2023-10-18T09:00:16.867390+0000","last_fullsized":"2023-10-18T09:00:16.867390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:26:17.885550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867242+0000","last_change":"2023-10-18T09:00:15.753122+0000","last_active":"2023-10-18T09:00:16.867242+0000","last_peered":"2023-10-18T09:00:16.867242+0000","last_clean":"2023-10-18T09:00:16.867242+0000","last_became_active":"2023-10-18T09:00:15.752653+0000","last_became_peered":"2023-10-18T09:00:15.752653+0000","last_unstale":"2023-10-18T09:00:16.867242+0000","last_undegraded":"2023-10-18T09:00:16.867242+0000","last_fullsized":"2023-10-18T09:00:16.867242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:15:32.449721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736879+0000","last_change":"2023-10-18T09:00:15.749937+0000","last_active":"2023-10-18T09:00:16.736879+0000","last_peered":"2023-10-18T09:00:16.736879+0000","last_clean":"2023-10-18T09:00:16.736879+0000","last_became_active":"2023-10-18T09:00:15.749226+0000","last_became_peered":"2023-10-18T09:00:15.749226+0000","last_unstale":"2023-10-18T09:00:16.736879+0000","last_undegraded":"2023-10-18T09:00:16.736879+0000","last_fullsized":"2023-10-18T09:00:16.736879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:28.075342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736586+0000","last_change":"2023-10-18T09:00:15.762946+0000","last_active":"2023-10-18T09:00:16.736586+0000","last_peered":"2023-10-18T09:00:16.736586+0000","last_clean":"2023-10-18T09:00:16.736586+0000","last_became_active":"2023-10-18T09:00:15.762276+0000","last_became_peered":"2023-10-18T09:00:15.762276+0000","last_unstale":"2023-10-18T09:00:16.736586+0000","last_undegraded":"2023-10-18T09:00:16.736586+0000","last_fullsized":"2023-10-18T09:00:16.736586+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:28:15.286205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867778+0000","last_change":"2023-10-18T09:00:15.737169+0000","last_active":"2023-10-18T09:00:16.867778+0000","last_peered":"2023-10-18T09:00:16.867778+0000","last_clean":"2023-10-18T09:00:16.867778+0000","last_became_active":"2023-10-18T09:00:15.736508+0000","last_became_peered":"2023-10-18T09:00:15.736508+0000","last_unstale":"2023-10-18T09:00:16.867778+0000","last_undegraded":"2023-10-18T09:00:16.867778+0000","last_fullsized":"2023-10-18T09:00:16.867778+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:32:57.960964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.868854+0000","last_change":"2023-10-18T09:00:15.757071+0000","last_active":"2023-10-18T09:00:16.868854+0000","last_peered":"2023-10-18T09:00:16.868854+0000","last_clean":"2023-10-18T09:00:16.868854+0000","last_became_active":"2023-10-18T09:00:15.754963+0000","last_became_peered":"2023-10-18T09:00:15.754963+0000","last_unstale":"2023-10-18T09:00:16.868854+0000","last_undegraded":"2023-10-18T09:00:16.868854+0000","last_fullsized":"2023-10-18T09:00:16.868854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:38.079651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.735091+0000","last_change":"2023-10-18T09:00:15.734271+0000","last_active":"2023-10-18T09:00:16.735091+0000","last_peered":"2023-10-18T09:00:16.735091+0000","last_clean":"2023-10-18T09:00:16.735091+0000","last_became_active":"2023-10-18T09:00:15.733788+0000","last_became_peered":"2023-10-18T09:00:15.733788+0000","last_unstale":"2023-10-18T09:00:16.735091+0000","last_undegraded":"2023-10-18T09:00:16.735091+0000","last_fullsized":"2023-10-18T09:00:16.735091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:24:56.735813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867924+0000","last_change":"2023-10-18T09:00:15.756508+0000","last_active":"2023-10-18T09:00:16.867924+0000","last_peered":"2023-10-18T09:00:16.867924+0000","last_clean":"2023-10-18T09:00:16.867924+0000","last_became_active":"2023-10-18T09:00:15.756155+0000","last_became_peered":"2023-10-18T09:00:15.756155+0000","last_unstale":"2023-10-18T09:00:16.867924+0000","last_undegraded":"2023-10-18T09:00:16.867924+0000","last_fullsized":"2023-10-18T09:00:16.867924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:38.079651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.734779+0000","last_change":"2023-10-18T09:00:15.754144+0000","last_active":"2023-10-18T09:00:16.734779+0000","last_peered":"2023-10-18T09:00:16.734779+0000","last_clean":"2023-10-18T09:00:16.734779+0000","last_became_active":"2023-10-18T09:00:15.748159+0000","last_became_peered":"2023-10-18T09:00:15.748159+0000","last_unstale":"2023-10-18T09:00:16.734779+0000","last_undegraded":"2023-10-18T09:00:16.734779+0000","last_fullsized":"2023-10-18T09:00:16.734779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:26:54.506506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.871708+0000","last_change":"2023-10-18T09:00:15.755928+0000","last_active":"2023-10-18T09:00:16.871708+0000","last_peered":"2023-10-18T09:00:16.871708+0000","last_clean":"2023-10-18T09:00:16.871708+0000","last_became_active":"2023-10-18T09:00:15.755233+0000","last_became_peered":"2023-10-18T09:00:15.755233+0000","last_unstale":"2023-10-18T09:00:16.871708+0000","last_undegraded":"2023-10-18T09:00:16.871708+0000","last_fullsized":"2023-10-18T09:00:16.871708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:06.826641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.869159+0000","last_change":"2023-10-18T09:00:15.760173+0000","last_active":"2023-10-18T09:00:16.869159+0000","last_peered":"2023-10-18T09:00:16.869159+0000","last_clean":"2023-10-18T09:00:16.869159+0000","last_became_active":"2023-10-18T09:00:15.759923+0000","last_became_peered":"2023-10-18T09:00:15.759923+0000","last_unstale":"2023-10-18T09:00:16.869159+0000","last_undegraded":"2023-10-18T09:00:16.869159+0000","last_fullsized":"2023-10-18T09:00:16.869159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:22.137803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866984+0000","last_change":"2023-10-18T09:00:15.765266+0000","last_active":"2023-10-18T09:00:16.866984+0000","last_peered":"2023-10-18T09:00:16.866984+0000","last_clean":"2023-10-18T09:00:16.866984+0000","last_became_active":"2023-10-18T09:00:15.764999+0000","last_became_peered":"2023-10-18T09:00:15.764999+0000","last_unstale":"2023-10-18T09:00:16.866984+0000","last_undegraded":"2023-10-18T09:00:16.866984+0000","last_fullsized":"2023-10-18T09:00:16.866984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:38.805225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737031+0000","last_change":"2023-10-18T09:00:15.762805+0000","last_active":"2023-10-18T09:00:16.737031+0000","last_peered":"2023-10-18T09:00:16.737031+0000","last_clean":"2023-10-18T09:00:16.737031+0000","last_became_active":"2023-10-18T09:00:15.761946+0000","last_became_peered":"2023-10-18T09:00:15.761946+0000","last_unstale":"2023-10-18T09:00:16.737031+0000","last_undegraded":"2023-10-18T09:00:16.737031+0000","last_fullsized":"2023-10-18T09:00:16.737031+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:28:00.652852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"86'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773766+0000","last_change":"2023-10-18T09:00:14.719832+0000","last_active":"2023-10-18T09:00:15.773766+0000","last_peered":"2023-10-18T09:00:15.773766+0000","last_clean":"2023-10-18T09:00:15.773766+0000","last_became_active":"2023-10-18T09:00:14.719074+0000","last_became_peered":"2023-10-18T09:00:14.719074+0000","last_unstale":"2023-10-18T09:00:15.773766+0000","last_undegraded":"2023-10-18T09:00:15.773766+0000","last_fullsized":"2023-10-18T09:00:15.773766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:31:09.739409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736246+0000","last_change":"2023-10-18T09:00:15.761074+0000","last_active":"2023-10-18T09:00:16.736246+0000","last_peered":"2023-10-18T09:00:16.736246+0000","last_clean":"2023-10-18T09:00:16.736246+0000","last_became_active":"2023-10-18T09:00:15.757708+0000","last_became_peered":"2023-10-18T09:00:15.757708+0000","last_unstale":"2023-10-18T09:00:16.736246+0000","last_undegraded":"2023-10-18T09:00:16.736246+0000","last_fullsized":"2023-10-18T09:00:16.736246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:16:51.405175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.772773+0000","last_change":"2023-10-18T09:00:14.722227+0000","last_active":"2023-10-18T09:00:15.772773+0000","last_peered":"2023-10-18T09:00:15.772773+0000","last_clean":"2023-10-18T09:00:15.772773+0000","last_became_active":"2023-10-18T09:00:14.720313+0000","last_became_peered":"2023-10-18T09:00:14.720313+0000","last_unstale":"2023-10-18T09:00:15.772773+0000","last_undegraded":"2023-10-18T09:00:15.772773+0000","last_fullsized":"2023-10-18T09:00:15.772773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:28:25.001764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.869848+0000","last_change":"2023-10-18T09:00:15.753770+0000","last_active":"2023-10-18T09:00:16.869848+0000","last_peered":"2023-10-18T09:00:16.869848+0000","last_clean":"2023-10-18T09:00:16.869848+0000","last_became_active":"2023-10-18T09:00:15.752239+0000","last_became_peered":"2023-10-18T09:00:15.752239+0000","last_unstale":"2023-10-18T09:00:16.869848+0000","last_undegraded":"2023-10-18T09:00:16.869848+0000","last_fullsized":"2023-10-18T09:00:16.869848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:04:34.318321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773255+0000","last_change":"2023-10-18T09:00:14.724618+0000","last_active":"2023-10-18T09:00:15.773255+0000","last_peered":"2023-10-18T09:00:15.773255+0000","last_clean":"2023-10-18T09:00:15.773255+0000","last_became_active":"2023-10-18T09:00:14.721500+0000","last_became_peered":"2023-10-18T09:00:14.721500+0000","last_unstale":"2023-10-18T09:00:15.773255+0000","last_undegraded":"2023-10-18T09:00:15.773255+0000","last_fullsized":"2023-10-18T09:00:15.773255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:59.045131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737858+0000","last_change":"2023-10-18T09:00:15.759339+0000","last_active":"2023-10-18T09:00:16.737858+0000","last_peered":"2023-10-18T09:00:16.737858+0000","last_clean":"2023-10-18T09:00:16.737858+0000","last_became_active":"2023-10-18T09:00:15.758140+0000","last_became_peered":"2023-10-18T09:00:15.758140+0000","last_unstale":"2023-10-18T09:00:16.737858+0000","last_undegraded":"2023-10-18T09:00:16.737858+0000","last_fullsized":"2023-10-18T09:00:16.737858+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:17:29.238634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773868+0000","last_change":"2023-10-18T09:00:14.729343+0000","last_active":"2023-10-18T09:00:15.773868+0000","last_peered":"2023-10-18T09:00:15.773868+0000","last_clean":"2023-10-18T09:00:15.773868+0000","last_became_active":"2023-10-18T09:00:14.728957+0000","last_became_peered":"2023-10-18T09:00:14.728957+0000","last_unstale":"2023-10-18T09:00:15.773868+0000","last_undegraded":"2023-10-18T09:00:15.773868+0000","last_fullsized":"2023-10-18T09:00:15.773868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:12:38.346335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.869564+0000","last_change":"2023-10-18T09:00:15.758912+0000","last_active":"2023-10-18T09:00:16.869564+0000","last_peered":"2023-10-18T09:00:16.869564+0000","last_clean":"2023-10-18T09:00:16.869564+0000","last_became_active":"2023-10-18T09:00:15.756824+0000","last_became_peered":"2023-10-18T09:00:15.756824+0000","last_unstale":"2023-10-18T09:00:16.869564+0000","last_undegraded":"2023-10-18T09:00:16.869564+0000","last_fullsized":"2023-10-18T09:00:16.869564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:24:27.536167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.772736+0000","last_change":"2023-10-18T09:00:14.720994+0000","last_active":"2023-10-18T09:00:15.772736+0000","last_peered":"2023-10-18T09:00:15.772736+0000","last_clean":"2023-10-18T09:00:15.772736+0000","last_became_active":"2023-10-18T09:00:14.720661+0000","last_became_peered":"2023-10-18T09:00:14.720661+0000","last_unstale":"2023-10-18T09:00:15.772736+0000","last_undegraded":"2023-10-18T09:00:15.772736+0000","last_fullsized":"2023-10-18T09:00:15.772736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:55:27.065983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867552+0000","last_change":"2023-10-18T09:00:15.737040+0000","last_active":"2023-10-18T09:00:16.867552+0000","last_peered":"2023-10-18T09:00:16.867552+0000","last_clean":"2023-10-18T09:00:16.867552+0000","last_became_active":"2023-10-18T09:00:15.736400+0000","last_became_peered":"2023-10-18T09:00:15.736400+0000","last_unstale":"2023-10-18T09:00:16.867552+0000","last_undegraded":"2023-10-18T09:00:16.867552+0000","last_fullsized":"2023-10-18T09:00:16.867552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:27:57.117593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773337+0000","last_change":"2023-10-18T09:00:14.726650+0000","last_active":"2023-10-18T09:00:15.773337+0000","last_peered":"2023-10-18T09:00:15.773337+0000","last_clean":"2023-10-18T09:00:15.773337+0000","last_became_active":"2023-10-18T09:00:14.726047+0000","last_became_peered":"2023-10-18T09:00:14.726047+0000","last_unstale":"2023-10-18T09:00:15.773337+0000","last_undegraded":"2023-10-18T09:00:15.773337+0000","last_fullsized":"2023-10-18T09:00:15.773337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:55:25.382068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736921+0000","last_change":"2023-10-18T09:00:15.759622+0000","last_active":"2023-10-18T09:00:16.736921+0000","last_peered":"2023-10-18T09:00:16.736921+0000","last_clean":"2023-10-18T09:00:16.736921+0000","last_became_active":"2023-10-18T09:00:15.759302+0000","last_became_peered":"2023-10-18T09:00:15.759302+0000","last_unstale":"2023-10-18T09:00:16.736921+0000","last_undegraded":"2023-10-18T09:00:16.736921+0000","last_fullsized":"2023-10-18T09:00:16.736921+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:25:33.396541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.749958+0000","last_change":"2023-10-18T09:00:14.722001+0000","last_active":"2023-10-18T09:00:15.749958+0000","last_peered":"2023-10-18T09:00:15.749958+0000","last_clean":"2023-10-18T09:00:15.749958+0000","last_became_active":"2023-10-18T09:00:14.721325+0000","last_became_peered":"2023-10-18T09:00:14.721325+0000","last_unstale":"2023-10-18T09:00:15.749958+0000","last_undegraded":"2023-10-18T09:00:15.749958+0000","last_fullsized":"2023-10-18T09:00:15.749958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:05:08.365526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.865724+0000","last_change":"2023-10-18T09:00:15.765115+0000","last_active":"2023-10-18T09:00:16.865724+0000","last_peered":"2023-10-18T09:00:16.865724+0000","last_clean":"2023-10-18T09:00:16.865724+0000","last_became_active":"2023-10-18T09:00:15.764828+0000","last_became_peered":"2023-10-18T09:00:15.764828+0000","last_unstale":"2023-10-18T09:00:16.865724+0000","last_undegraded":"2023-10-18T09:00:16.865724+0000","last_fullsized":"2023-10-18T09:00:16.865724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:43:18.411223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.751377+0000","last_change":"2023-10-18T09:00:14.719395+0000","last_active":"2023-10-18T09:00:15.751377+0000","last_peered":"2023-10-18T09:00:15.751377+0000","last_clean":"2023-10-18T09:00:15.751377+0000","last_became_active":"2023-10-18T09:00:14.718426+0000","last_became_peered":"2023-10-18T09:00:14.718426+0000","last_unstale":"2023-10-18T09:00:15.751377+0000","last_undegraded":"2023-10-18T09:00:15.751377+0000","last_fullsized":"2023-10-18T09:00:15.751377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:42:14.030822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.871025+0000","last_change":"2023-10-18T09:00:15.758761+0000","last_active":"2023-10-18T09:00:16.871025+0000","last_peered":"2023-10-18T09:00:16.871025+0000","last_clean":"2023-10-18T09:00:16.871025+0000","last_became_active":"2023-10-18T09:00:15.758524+0000","last_became_peered":"2023-10-18T09:00:15.758524+0000","last_unstale":"2023-10-18T09:00:16.871025+0000","last_undegraded":"2023-10-18T09:00:16.871025+0000","last_fullsized":"2023-10-18T09:00:16.871025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:23.871839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.730864+0000","last_change":"2023-10-18T09:00:14.713528+0000","last_active":"2023-10-18T09:00:15.730864+0000","last_peered":"2023-10-18T09:00:15.730864+0000","last_clean":"2023-10-18T09:00:15.730864+0000","last_became_active":"2023-10-18T09:00:14.712766+0000","last_became_peered":"2023-10-18T09:00:14.712766+0000","last_unstale":"2023-10-18T09:00:15.730864+0000","last_undegraded":"2023-10-18T09:00:15.730864+0000","last_fullsized":"2023-10-18T09:00:15.730864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:37:44.788135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737694+0000","last_change":"2023-10-18T09:00:15.758755+0000","last_active":"2023-10-18T09:00:16.737694+0000","last_peered":"2023-10-18T09:00:16.737694+0000","last_clean":"2023-10-18T09:00:16.737694+0000","last_became_active":"2023-10-18T09:00:15.757506+0000","last_became_peered":"2023-10-18T09:00:15.757506+0000","last_unstale":"2023-10-18T09:00:16.737694+0000","last_undegraded":"2023-10-18T09:00:16.737694+0000","last_fullsized":"2023-10-18T09:00:16.737694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:43:18.411223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.748766+0000","last_change":"2023-10-18T09:00:14.728500+0000","last_active":"2023-10-18T09:00:15.748766+0000","last_peered":"2023-10-18T09:00:15.748766+0000","last_clean":"2023-10-18T09:00:15.748766+0000","last_became_active":"2023-10-18T09:00:14.728243+0000","last_became_peered":"2023-10-18T09:00:14.728243+0000","last_unstale":"2023-10-18T09:00:15.748766+0000","last_undegraded":"2023-10-18T09:00:15.748766+0000","last_fullsized":"2023-10-18T09:00:15.748766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:51:03.232237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867311+0000","last_change":"2023-10-18T09:00:15.759536+0000","last_active":"2023-10-18T09:00:16.867311+0000","last_peered":"2023-10-18T09:00:16.867311+0000","last_clean":"2023-10-18T09:00:16.867311+0000","last_became_active":"2023-10-18T09:00:15.759116+0000","last_became_peered":"2023-10-18T09:00:15.759116+0000","last_unstale":"2023-10-18T09:00:16.867311+0000","last_undegraded":"2023-10-18T09:00:16.867311+0000","last_fullsized":"2023-10-18T09:00:16.867311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:34:17.782897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"86'3","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.774146+0000","last_change":"2023-10-18T09:00:14.723428+0000","last_active":"2023-10-18T09:00:15.774146+0000","last_peered":"2023-10-18T09:00:15.774146+0000","last_clean":"2023-10-18T09:00:15.774146+0000","last_became_active":"2023-10-18T09:00:14.723174+0000","last_became_peered":"2023-10-18T09:00:14.723174+0000","last_unstale":"2023-10-18T09:00:15.774146+0000","last_undegraded":"2023-10-18T09:00:15.774146+0000","last_fullsized":"2023-10-18T09:00:15.774146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:53:02.168967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736447+0000","last_change":"2023-10-18T09:00:15.755629+0000","last_active":"2023-10-18T09:00:16.736447+0000","last_peered":"2023-10-18T09:00:16.736447+0000","last_clean":"2023-10-18T09:00:16.736447+0000","last_became_active":"2023-10-18T09:00:15.755325+0000","last_became_peered":"2023-10-18T09:00:15.755325+0000","last_unstale":"2023-10-18T09:00:16.736447+0000","last_undegraded":"2023-10-18T09:00:16.736447+0000","last_fullsized":"2023-10-18T09:00:16.736447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:32:40.008198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"86'6","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.776851+0000","last_change":"2023-10-18T09:00:14.720054+0000","last_active":"2023-10-18T09:00:15.776851+0000","last_peered":"2023-10-18T09:00:15.776851+0000","last_clean":"2023-10-18T09:00:15.776851+0000","last_became_active":"2023-10-18T09:00:14.719704+0000","last_became_peered":"2023-10-18T09:00:14.719704+0000","last_unstale":"2023-10-18T09:00:15.776851+0000","last_undegraded":"2023-10-18T09:00:15.776851+0000","last_fullsized":"2023-10-18T09:00:15.776851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:22:27.935440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737925+0000","last_change":"2023-10-18T09:00:15.758943+0000","last_active":"2023-10-18T09:00:16.737925+0000","last_peered":"2023-10-18T09:00:16.737925+0000","last_clean":"2023-10-18T09:00:16.737925+0000","last_became_active":"2023-10-18T09:00:15.758060+0000","last_became_peered":"2023-10-18T09:00:15.758060+0000","last_unstale":"2023-10-18T09:00:16.737925+0000","last_undegraded":"2023-10-18T09:00:16.737925+0000","last_fullsized":"2023-10-18T09:00:16.737925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:00.837466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.774849+0000","last_change":"2023-10-18T09:00:14.719224+0000","last_active":"2023-10-18T09:00:15.774849+0000","last_peered":"2023-10-18T09:00:15.774849+0000","last_clean":"2023-10-18T09:00:15.774849+0000","last_became_active":"2023-10-18T09:00:14.718827+0000","last_became_peered":"2023-10-18T09:00:14.718827+0000","last_unstale":"2023-10-18T09:00:15.774849+0000","last_undegraded":"2023-10-18T09:00:15.774849+0000","last_fullsized":"2023-10-18T09:00:15.774849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:55:25.382068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.872019+0000","last_change":"2023-10-18T09:00:15.753951+0000","last_active":"2023-10-18T09:00:16.872019+0000","last_peered":"2023-10-18T09:00:16.872019+0000","last_clean":"2023-10-18T09:00:16.872019+0000","last_became_active":"2023-10-18T09:00:15.753614+0000","last_became_peered":"2023-10-18T09:00:15.753614+0000","last_unstale":"2023-10-18T09:00:16.872019+0000","last_undegraded":"2023-10-18T09:00:16.872019+0000","last_fullsized":"2023-10-18T09:00:16.872019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:31.329353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.750279+0000","last_change":"2023-10-18T09:00:14.724705+0000","last_active":"2023-10-18T09:00:15.750279+0000","last_peered":"2023-10-18T09:00:15.750279+0000","last_clean":"2023-10-18T09:00:15.750279+0000","last_became_active":"2023-10-18T09:00:14.724414+0000","last_became_peered":"2023-10-18T09:00:14.724414+0000","last_unstale":"2023-10-18T09:00:15.750279+0000","last_undegraded":"2023-10-18T09:00:15.750279+0000","last_fullsized":"2023-10-18T09:00:15.750279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:53.753897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866195+0000","last_change":"2023-10-18T09:00:15.767934+0000","last_active":"2023-10-18T09:00:16.866195+0000","last_peered":"2023-10-18T09:00:16.866195+0000","last_clean":"2023-10-18T09:00:16.866195+0000","last_became_active":"2023-10-18T09:00:15.767341+0000","last_became_peered":"2023-10-18T09:00:15.767341+0000","last_unstale":"2023-10-18T09:00:16.866195+0000","last_undegraded":"2023-10-18T09:00:16.866195+0000","last_fullsized":"2023-10-18T09:00:16.866195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:28.179544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.745451+0000","last_change":"2023-10-18T09:00:14.721694+0000","last_active":"2023-10-18T09:00:15.745451+0000","last_peered":"2023-10-18T09:00:15.745451+0000","last_clean":"2023-10-18T09:00:15.745451+0000","last_became_active":"2023-10-18T09:00:14.720757+0000","last_became_peered":"2023-10-18T09:00:14.720757+0000","last_unstale":"2023-10-18T09:00:15.745451+0000","last_undegraded":"2023-10-18T09:00:15.745451+0000","last_fullsized":"2023-10-18T09:00:15.745451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:26:16.872665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736651+0000","last_change":"2023-10-18T09:00:15.759417+0000","last_active":"2023-10-18T09:00:16.736651+0000","last_peered":"2023-10-18T09:00:16.736651+0000","last_clean":"2023-10-18T09:00:16.736651+0000","last_became_active":"2023-10-18T09:00:15.757457+0000","last_became_peered":"2023-10-18T09:00:15.757457+0000","last_unstale":"2023-10-18T09:00:16.736651+0000","last_undegraded":"2023-10-18T09:00:16.736651+0000","last_fullsized":"2023-10-18T09:00:16.736651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:44:11.773265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"86'8","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.776052+0000","last_change":"2023-10-18T09:00:14.724785+0000","last_active":"2023-10-18T09:00:15.776052+0000","last_peered":"2023-10-18T09:00:15.776052+0000","last_clean":"2023-10-18T09:00:15.776052+0000","last_became_active":"2023-10-18T09:00:14.724371+0000","last_became_peered":"2023-10-18T09:00:14.724371+0000","last_unstale":"2023-10-18T09:00:15.776052+0000","last_undegraded":"2023-10-18T09:00:15.776052+0000","last_fullsized":"2023-10-18T09:00:15.776052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:46.341496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.868082+0000","last_change":"2023-10-18T09:00:15.751116+0000","last_active":"2023-10-18T09:00:16.868082+0000","last_peered":"2023-10-18T09:00:16.868082+0000","last_clean":"2023-10-18T09:00:16.868082+0000","last_became_active":"2023-10-18T09:00:15.750596+0000","last_became_peered":"2023-10-18T09:00:15.750596+0000","last_unstale":"2023-10-18T09:00:16.868082+0000","last_undegraded":"2023-10-18T09:00:16.868082+0000","last_fullsized":"2023-10-18T09:00:16.868082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:22.137803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.772684+0000","last_change":"2023-10-18T09:00:14.728865+0000","last_active":"2023-10-18T09:00:15.772684+0000","last_peered":"2023-10-18T09:00:15.772684+0000","last_clean":"2023-10-18T09:00:15.772684+0000","last_became_active":"2023-10-18T09:00:14.728369+0000","last_became_peered":"2023-10-18T09:00:14.728369+0000","last_unstale":"2023-10-18T09:00:15.772684+0000","last_undegraded":"2023-10-18T09:00:15.772684+0000","last_fullsized":"2023-10-18T09:00:15.772684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:51:43.253647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737387+0000","last_change":"2023-10-18T09:00:15.762625+0000","last_active":"2023-10-18T09:00:16.737387+0000","last_peered":"2023-10-18T09:00:16.737387+0000","last_clean":"2023-10-18T09:00:16.737387+0000","last_became_active":"2023-10-18T09:00:15.761420+0000","last_became_peered":"2023-10-18T09:00:15.761420+0000","last_unstale":"2023-10-18T09:00:16.737387+0000","last_undegraded":"2023-10-18T09:00:16.737387+0000","last_fullsized":"2023-10-18T09:00:16.737387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:14:41.715749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773990+0000","last_change":"2023-10-18T09:00:14.722184+0000","last_active":"2023-10-18T09:00:15.773990+0000","last_peered":"2023-10-18T09:00:15.773990+0000","last_clean":"2023-10-18T09:00:15.773990+0000","last_became_active":"2023-10-18T09:00:14.721073+0000","last_became_peered":"2023-10-18T09:00:14.721073+0000","last_unstale":"2023-10-18T09:00:15.773990+0000","last_undegraded":"2023-10-18T09:00:15.773990+0000","last_fullsized":"2023-10-18T09:00:15.773990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:35:27.318871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867237+0000","last_change":"2023-10-18T09:00:15.758090+0000","last_active":"2023-10-18T09:00:16.867237+0000","last_peered":"2023-10-18T09:00:16.867237+0000","last_clean":"2023-10-18T09:00:16.867237+0000","last_became_active":"2023-10-18T09:00:15.757750+0000","last_became_peered":"2023-10-18T09:00:15.757750+0000","last_unstale":"2023-10-18T09:00:16.867237+0000","last_undegraded":"2023-10-18T09:00:16.867237+0000","last_fullsized":"2023-10-18T09:00:16.867237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:54.766783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.772911+0000","last_change":"2023-10-18T09:00:14.729164+0000","last_active":"2023-10-18T09:00:15.772911+0000","last_peered":"2023-10-18T09:00:15.772911+0000","last_clean":"2023-10-18T09:00:15.772911+0000","last_became_active":"2023-10-18T09:00:14.728269+0000","last_became_peered":"2023-10-18T09:00:14.728269+0000","last_unstale":"2023-10-18T09:00:15.772911+0000","last_undegraded":"2023-10-18T09:00:15.772911+0000","last_fullsized":"2023-10-18T09:00:15.772911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:27.534647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736706+0000","last_change":"2023-10-18T09:00:15.761518+0000","last_active":"2023-10-18T09:00:16.736706+0000","last_peered":"2023-10-18T09:00:16.736706+0000","last_clean":"2023-10-18T09:00:16.736706+0000","last_became_active":"2023-10-18T09:00:15.760762+0000","last_became_peered":"2023-10-18T09:00:15.760762+0000","last_unstale":"2023-10-18T09:00:16.736706+0000","last_undegraded":"2023-10-18T09:00:16.736706+0000","last_fullsized":"2023-10-18T09:00:16.736706+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:02:37.770586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.730935+0000","last_change":"2023-10-18T09:00:14.723114+0000","last_active":"2023-10-18T09:00:15.730935+0000","last_peered":"2023-10-18T09:00:15.730935+0000","last_clean":"2023-10-18T09:00:15.730935+0000","last_became_active":"2023-10-18T09:00:14.722764+0000","last_became_peered":"2023-10-18T09:00:14.722764+0000","last_unstale":"2023-10-18T09:00:15.730935+0000","last_undegraded":"2023-10-18T09:00:15.730935+0000","last_fullsized":"2023-10-18T09:00:15.730935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:24:00.853217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.735158+0000","last_change":"2023-10-18T09:00:15.755724+0000","last_active":"2023-10-18T09:00:16.735158+0000","last_peered":"2023-10-18T09:00:16.735158+0000","last_clean":"2023-10-18T09:00:16.735158+0000","last_became_active":"2023-10-18T09:00:15.755472+0000","last_became_peered":"2023-10-18T09:00:15.755472+0000","last_unstale":"2023-10-18T09:00:16.735158+0000","last_undegraded":"2023-10-18T09:00:16.735158+0000","last_fullsized":"2023-10-18T09:00:16.735158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:22:41.062843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.758241+0000","last_change":"2023-10-18T09:00:14.727319+0000","last_active":"2023-10-18T09:00:15.758241+0000","last_peered":"2023-10-18T09:00:15.758241+0000","last_clean":"2023-10-18T09:00:15.758241+0000","last_became_active":"2023-10-18T09:00:14.726864+0000","last_became_peered":"2023-10-18T09:00:14.726864+0000","last_unstale":"2023-10-18T09:00:15.758241+0000","last_undegraded":"2023-10-18T09:00:15.758241+0000","last_fullsized":"2023-10-18T09:00:15.758241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:15.393555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.734848+0000","last_change":"2023-10-18T09:00:15.756743+0000","last_active":"2023-10-18T09:00:16.734848+0000","last_peered":"2023-10-18T09:00:16.734848+0000","last_clean":"2023-10-18T09:00:16.734848+0000","last_became_active":"2023-10-18T09:00:15.756462+0000","last_became_peered":"2023-10-18T09:00:15.756462+0000","last_unstale":"2023-10-18T09:00:16.734848+0000","last_undegraded":"2023-10-18T09:00:16.734848+0000","last_fullsized":"2023-10-18T09:00:16.734848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:14.505133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.752571+0000","last_change":"2023-10-18T09:00:14.723338+0000","last_active":"2023-10-18T09:00:15.752571+0000","last_peered":"2023-10-18T09:00:15.752571+0000","last_clean":"2023-10-18T09:00:15.752571+0000","last_became_active":"2023-10-18T09:00:14.723084+0000","last_became_peered":"2023-10-18T09:00:14.723084+0000","last_unstale":"2023-10-18T09:00:15.752571+0000","last_undegraded":"2023-10-18T09:00:15.752571+0000","last_fullsized":"2023-10-18T09:00:15.752571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:32:56.948078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.734768+0000","last_change":"2023-10-18T09:00:15.758267+0000","last_active":"2023-10-18T09:00:16.734768+0000","last_peered":"2023-10-18T09:00:16.734768+0000","last_clean":"2023-10-18T09:00:16.734768+0000","last_became_active":"2023-10-18T09:00:15.758033+0000","last_became_peered":"2023-10-18T09:00:15.758033+0000","last_unstale":"2023-10-18T09:00:16.734768+0000","last_undegraded":"2023-10-18T09:00:16.734768+0000","last_fullsized":"2023-10-18T09:00:16.734768+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:04:50.114527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.754653+0000","last_change":"2023-10-18T09:00:14.717552+0000","last_active":"2023-10-18T09:00:15.754653+0000","last_peered":"2023-10-18T09:00:15.754653+0000","last_clean":"2023-10-18T09:00:15.754653+0000","last_became_active":"2023-10-18T09:00:14.717311+0000","last_became_peered":"2023-10-18T09:00:14.717311+0000","last_unstale":"2023-10-18T09:00:15.754653+0000","last_undegraded":"2023-10-18T09:00:15.754653+0000","last_fullsized":"2023-10-18T09:00:15.754653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:27:59.639966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866525+0000","last_change":"2023-10-18T09:00:15.766855+0000","last_active":"2023-10-18T09:00:16.866525+0000","last_peered":"2023-10-18T09:00:16.866525+0000","last_clean":"2023-10-18T09:00:16.866525+0000","last_became_active":"2023-10-18T09:00:15.763490+0000","last_became_peered":"2023-10-18T09:00:15.763490+0000","last_unstale":"2023-10-18T09:00:16.866525+0000","last_undegraded":"2023-10-18T09:00:16.866525+0000","last_fullsized":"2023-10-18T09:00:16.866525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:16:52.626915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.772821+0000","last_change":"2023-10-18T09:00:14.725676+0000","last_active":"2023-10-18T09:00:15.772821+0000","last_peered":"2023-10-18T09:00:15.772821+0000","last_clean":"2023-10-18T09:00:15.772821+0000","last_became_active":"2023-10-18T09:00:14.725396+0000","last_became_peered":"2023-10-18T09:00:14.725396+0000","last_unstale":"2023-10-18T09:00:15.772821+0000","last_undegraded":"2023-10-18T09:00:15.772821+0000","last_fullsized":"2023-10-18T09:00:15.772821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:00.426054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.871368+0000","last_change":"2023-10-18T09:00:15.754403+0000","last_active":"2023-10-18T09:00:16.871368+0000","last_peered":"2023-10-18T09:00:16.871368+0000","last_clean":"2023-10-18T09:00:16.871368+0000","last_became_active":"2023-10-18T09:00:15.753953+0000","last_became_peered":"2023-10-18T09:00:15.753953+0000","last_unstale":"2023-10-18T09:00:16.871368+0000","last_undegraded":"2023-10-18T09:00:16.871368+0000","last_fullsized":"2023-10-18T09:00:16.871368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:22.851038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.742969+0000","last_change":"2023-10-18T09:00:14.719262+0000","last_active":"2023-10-18T09:00:15.742969+0000","last_peered":"2023-10-18T09:00:15.742969+0000","last_clean":"2023-10-18T09:00:15.742969+0000","last_became_active":"2023-10-18T09:00:14.718856+0000","last_became_peered":"2023-10-18T09:00:14.718856+0000","last_unstale":"2023-10-18T09:00:15.742969+0000","last_undegraded":"2023-10-18T09:00:15.742969+0000","last_fullsized":"2023-10-18T09:00:15.742969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:59.045131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.738154+0000","last_change":"2023-10-18T09:00:15.758945+0000","last_active":"2023-10-18T09:00:16.738154+0000","last_peered":"2023-10-18T09:00:16.738154+0000","last_clean":"2023-10-18T09:00:16.738154+0000","last_became_active":"2023-10-18T09:00:15.757454+0000","last_became_peered":"2023-10-18T09:00:15.757454+0000","last_unstale":"2023-10-18T09:00:16.738154+0000","last_undegraded":"2023-10-18T09:00:16.738154+0000","last_fullsized":"2023-10-18T09:00:16.738154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:28.179544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.744492+0000","last_change":"2023-10-18T09:00:14.719287+0000","last_active":"2023-10-18T09:00:15.744492+0000","last_peered":"2023-10-18T09:00:15.744492+0000","last_clean":"2023-10-18T09:00:15.744492+0000","last_became_active":"2023-10-18T09:00:14.719023+0000","last_became_peered":"2023-10-18T09:00:14.719023+0000","last_unstale":"2023-10-18T09:00:15.744492+0000","last_undegraded":"2023-10-18T09:00:15.744492+0000","last_fullsized":"2023-10-18T09:00:15.744492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:08.548927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737731+0000","last_change":"2023-10-18T09:00:15.748118+0000","last_active":"2023-10-18T09:00:16.737731+0000","last_peered":"2023-10-18T09:00:16.737731+0000","last_clean":"2023-10-18T09:00:16.737731+0000","last_became_active":"2023-10-18T09:00:15.747642+0000","last_became_peered":"2023-10-18T09:00:15.747642+0000","last_unstale":"2023-10-18T09:00:16.737731+0000","last_undegraded":"2023-10-18T09:00:16.737731+0000","last_fullsized":"2023-10-18T09:00:16.737731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:44.909930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.755422+0000","last_change":"2023-10-18T09:00:14.727210+0000","last_active":"2023-10-18T09:00:15.755422+0000","last_peered":"2023-10-18T09:00:15.755422+0000","last_clean":"2023-10-18T09:00:15.755422+0000","last_became_active":"2023-10-18T09:00:14.727034+0000","last_became_peered":"2023-10-18T09:00:14.727034+0000","last_unstale":"2023-10-18T09:00:15.755422+0000","last_undegraded":"2023-10-18T09:00:15.755422+0000","last_fullsized":"2023-10-18T09:00:15.755422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:14:40.702864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736824+0000","last_change":"2023-10-18T09:00:15.746384+0000","last_active":"2023-10-18T09:00:16.736824+0000","last_peered":"2023-10-18T09:00:16.736824+0000","last_clean":"2023-10-18T09:00:16.736824+0000","last_became_active":"2023-10-18T09:00:15.745760+0000","last_became_peered":"2023-10-18T09:00:15.745760+0000","last_unstale":"2023-10-18T09:00:16.736824+0000","last_undegraded":"2023-10-18T09:00:16.736824+0000","last_fullsized":"2023-10-18T09:00:16.736824+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:05:46.945517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.753907+0000","last_change":"2023-10-18T09:00:14.723490+0000","last_active":"2023-10-18T09:00:15.753907+0000","last_peered":"2023-10-18T09:00:15.753907+0000","last_clean":"2023-10-18T09:00:15.753907+0000","last_became_active":"2023-10-18T09:00:14.723220+0000","last_became_peered":"2023-10-18T09:00:14.723220+0000","last_unstale":"2023-10-18T09:00:15.753907+0000","last_undegraded":"2023-10-18T09:00:15.753907+0000","last_fullsized":"2023-10-18T09:00:15.753907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:03.178214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.870157+0000","last_change":"2023-10-18T09:00:15.753547+0000","last_active":"2023-10-18T09:00:16.870157+0000","last_peered":"2023-10-18T09:00:16.870157+0000","last_clean":"2023-10-18T09:00:16.870157+0000","last_became_active":"2023-10-18T09:00:15.751658+0000","last_became_peered":"2023-10-18T09:00:15.751658+0000","last_unstale":"2023-10-18T09:00:16.870157+0000","last_undegraded":"2023-10-18T09:00:16.870157+0000","last_fullsized":"2023-10-18T09:00:16.870157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:47.354382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.752338+0000","last_change":"2023-10-18T09:00:14.716730+0000","last_active":"2023-10-18T09:00:15.752338+0000","last_peered":"2023-10-18T09:00:15.752338+0000","last_clean":"2023-10-18T09:00:15.752338+0000","last_became_active":"2023-10-18T09:00:14.716479+0000","last_became_peered":"2023-10-18T09:00:14.716479+0000","last_unstale":"2023-10-18T09:00:15.752338+0000","last_undegraded":"2023-10-18T09:00:15.752338+0000","last_fullsized":"2023-10-18T09:00:15.752338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:42.805546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866334+0000","last_change":"2023-10-18T09:00:15.760295+0000","last_active":"2023-10-18T09:00:16.866334+0000","last_peered":"2023-10-18T09:00:16.866334+0000","last_clean":"2023-10-18T09:00:16.866334+0000","last_became_active":"2023-10-18T09:00:15.760058+0000","last_became_peered":"2023-10-18T09:00:15.760058+0000","last_unstale":"2023-10-18T09:00:16.866334+0000","last_undegraded":"2023-10-18T09:00:16.866334+0000","last_fullsized":"2023-10-18T09:00:16.866334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:01.438940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.728164+0000","last_change":"2023-10-18T09:00:14.724630+0000","last_active":"2023-10-18T09:00:15.728164+0000","last_peered":"2023-10-18T09:00:15.728164+0000","last_clean":"2023-10-18T09:00:15.728164+0000","last_became_active":"2023-10-18T09:00:14.724028+0000","last_became_peered":"2023-10-18T09:00:14.724028+0000","last_unstale":"2023-10-18T09:00:15.728164+0000","last_undegraded":"2023-10-18T09:00:15.728164+0000","last_fullsized":"2023-10-18T09:00:15.728164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:53:10.671218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867111+0000","last_change":"2023-10-18T09:00:15.766128+0000","last_active":"2023-10-18T09:00:16.867111+0000","last_peered":"2023-10-18T09:00:16.867111+0000","last_clean":"2023-10-18T09:00:16.867111+0000","last_became_active":"2023-10-18T09:00:15.765583+0000","last_became_peered":"2023-10-18T09:00:15.765583+0000","last_unstale":"2023-10-18T09:00:16.867111+0000","last_undegraded":"2023-10-18T09:00:16.867111+0000","last_fullsized":"2023-10-18T09:00:16.867111+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:26:45.228100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.758442+0000","last_change":"2023-10-18T09:00:14.727160+0000","last_active":"2023-10-18T09:00:15.758442+0000","last_peered":"2023-10-18T09:00:15.758442+0000","last_clean":"2023-10-18T09:00:15.758442+0000","last_became_active":"2023-10-18T09:00:14.726471+0000","last_became_peered":"2023-10-18T09:00:14.726471+0000","last_unstale":"2023-10-18T09:00:15.758442+0000","last_undegraded":"2023-10-18T09:00:15.758442+0000","last_fullsized":"2023-10-18T09:00:15.758442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:11:15.412520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.735001+0000","last_change":"2023-10-18T09:00:15.765588+0000","last_active":"2023-10-18T09:00:16.735001+0000","last_peered":"2023-10-18T09:00:16.735001+0000","last_clean":"2023-10-18T09:00:16.735001+0000","last_became_active":"2023-10-18T09:00:15.765331+0000","last_became_peered":"2023-10-18T09:00:15.765331+0000","last_unstale":"2023-10-18T09:00:16.735001+0000","last_undegraded":"2023-10-18T09:00:16.735001+0000","last_fullsized":"2023-10-18T09:00:16.735001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:44:03.498540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773229+0000","last_change":"2023-10-18T09:00:14.726818+0000","last_active":"2023-10-18T09:00:15.773229+0000","last_peered":"2023-10-18T09:00:15.773229+0000","last_clean":"2023-10-18T09:00:15.773229+0000","last_became_active":"2023-10-18T09:00:14.726391+0000","last_became_peered":"2023-10-18T09:00:14.726391+0000","last_unstale":"2023-10-18T09:00:15.773229+0000","last_undegraded":"2023-10-18T09:00:15.773229+0000","last_fullsized":"2023-10-18T09:00:15.773229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:05:08.365526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":238,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.734462+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:00:15.734462+0000","last_peered":"2023-10-18T09:00:15.734462+0000","last_clean":"2023-10-18T09:00:15.734462+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:00:15.734462+0000","last_undegraded":"2023-10-18T09:00:15.734462+0000","last_fullsized":"2023-10-18T09:00:15.734462+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:36.150056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1032,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194280,"statfs":{"total":200000000,"available":198943213,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.071,"15min":1.0149999999999999},"min":{"1min":0.72599999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.575,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.165,"5min":1.1000000000000001,"15min":1.03},"min":{"1min":0.84399999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.536,"5min":1.97,"15min":1.97},"last":1.5740000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.125,"15min":1.147},"min":{"1min":0.88200000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5169999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1659999999999999,"15min":1.1299999999999999},"min":{"1min":0.91800000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.083}]},{"osd":2,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.284,"15min":1.246},"min":{"1min":0.78600000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.522,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.524},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.236,"15min":1.2030000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.637,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.202}]},{"osd":3,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.2609999999999999,"15min":1.2629999999999999},"min":{"1min":0.80300000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.845,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3500000000000001,"15min":1.327},"min":{"1min":1.0069999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.736,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.389}]},{"osd":4,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.238,"15min":1.1759999999999999},"min":{"1min":0.94599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.825,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.393,"5min":1.351,"15min":1.3919999999999999},"min":{"1min":1.101,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.627,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.131}]},{"osd":5,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3580000000000001,"15min":1.355},"min":{"1min":0.89600000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7470000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.415,"15min":1.421},"min":{"1min":1.079,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7130000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.45}]},{"osd":6,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.482,"15min":1.464},"min":{"1min":1.355,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.8169999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.306},{"interface":"front","average":{"1min":1.512,"5min":1.484,"15min":1.5069999999999999},"min":{"1min":1.214,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.768,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.421}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":454,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194857,"statfs":{"total":200000000,"available":199534105,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.1759999999999999,"15min":1.105},"min":{"1min":0.71799999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.627,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.411},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.2709999999999999,"15min":1.248},"min":{"1min":0.86699999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7629999999999999,"5min":1.855,"15min":1.855},"last":1.6319999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1040000000000001,"15min":1.0900000000000001},"min":{"1min":0.68500000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.488,"5min":1.633,"15min":1.633},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.198,"15min":1.1919999999999999},"min":{"1min":0.72599999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5409999999999999,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":1.2430000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.879,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5720000000000001,"5min":1.696,"15min":1.696},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2170000000000001,"15min":1.2010000000000001},"min":{"1min":0.90000000000000002,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.667,"5min":2.137,"15min":2.137},"last":1.5660000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.256,"15min":1.2589999999999999},"min":{"1min":0.91900000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.6719999999999999,"5min":1.784,"15min":1.784},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.248,"15min":1.226},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.6479999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.006}]},{"osd":4,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.2170000000000001,"15min":1.177},"min":{"1min":0.748,"5min":0.625,"15min":0.625},"max":{"1min":1.389,"5min":2.048,"15min":2.048},"last":1.367},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.262,"15min":1.2669999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.135}]},{"osd":5,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3819999999999999,"15min":1.3580000000000001},"min":{"1min":0.625,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.534},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2589999999999999,"15min":1.2250000000000001},"min":{"1min":0.71899999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.51}]},{"osd":7,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.391,"15min":1.3919999999999999},"min":{"1min":0.93500000000000005,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5900000000000001,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4910000000000001,"15min":1.5189999999999999},"min":{"1min":0.83099999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.944,"5min":2.794,"15min":2.794},"last":1.5880000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":454,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194857,"statfs":{"total":200000000,"available":199534107,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.345,"15min":1.2330000000000001},"min":{"1min":0.71999999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2480000000000002,"5min":9.3539999999999992,"15min":9.3539999999999992},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3260000000000001,"15min":1.304},"min":{"1min":0.77000000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.8260000000000001,"5min":5.3029999999999999,"15min":5.3029999999999999},"last":0.91900000000000004}]},{"osd":1,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.194,"15min":1.218},"min":{"1min":0.85099999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.1499999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.524},{"interface":"front","average":{"1min":1.272,"5min":1.1910000000000001,"15min":1.2370000000000001},"min":{"1min":0.78500000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.2410000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":1.1259999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2450000000000001,"15min":1.26},"min":{"1min":0.88900000000000001,"5min":0.755,"15min":0.755},"max":{"1min":2.0350000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.2629999999999999,"15min":1.284},"min":{"1min":0.95499999999999996,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.153,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":1.026}]},{"osd":3,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2230000000000001,"15min":1.2370000000000001},"min":{"1min":0.86899999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.613,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":1.169},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3440000000000001,"15min":1.3340000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.633}]},{"osd":4,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.393,"15min":1.4119999999999999},"min":{"1min":0.93899999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.4449999999999998,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.778},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3919999999999999,"15min":1.498},"min":{"1min":0.91700000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.4049999999999998,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":1.224}]},{"osd":6,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2989999999999999,"15min":1.3859999999999999},"min":{"1min":0.91900000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.72,"5min":5.141,"15min":5.141},"last":1.268},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.508,"15min":1.571},"min":{"1min":0.95299999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":1.718}]},{"osd":7,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.55,"15min":1.6399999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.5819999999999999,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":1.347},{"interface":"front","average":{"1min":1.385,"5min":1.4610000000000001,"15min":1.5069999999999999},"min":{"1min":1.091,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8240000000000001,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":1.476}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":453,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194858,"statfs":{"total":200000000,"available":199535146,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.80900000000000005,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7669999999999999,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.01},{"interface":"front","average":{"1min":1.25,"5min":1.125,"15min":1.1679999999999999},"min":{"1min":0.71199999999999997,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.7749999999999999,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.2809999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.089,"15min":1.0920000000000001},"min":{"1min":0.749,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4990000000000001,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":1.331},{"interface":"front","average":{"1min":1.23,"5min":1.1930000000000001,"15min":1.2290000000000001},"min":{"1min":0.70399999999999996,"5min":0.627,"15min":0.627},"max":{"1min":1.3740000000000001,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.123}]},{"osd":2,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1890000000000001,"15min":1.179},"min":{"1min":0.81999999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4350000000000001,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.2709999999999999,"15min":1.3029999999999999},"min":{"1min":0.85899999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6930000000000001,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.224}]},{"osd":3,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.89700000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9179999999999999,"5min":2.649,"15min":2.649},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.403,"15min":1.409},"min":{"1min":0.86799999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6399999999999999,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.4650000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.319,"15min":1.3029999999999999},"min":{"1min":0.96399999999999997,"5min":0.498,"15min":0.498},"max":{"1min":1.7649999999999999,"5min":1.923,"15min":1.923},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.359,"5min":1.3180000000000001,"15min":1.3560000000000001},"min":{"1min":0.92100000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.0720000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":0.94499999999999995}]},{"osd":6,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4379999999999999,"15min":1.3959999999999999},"min":{"1min":0.96999999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.1150000000000002,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.508,"15min":1.5580000000000001},"min":{"1min":1.2250000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6839999999999999,"5min":4.407,"15min":4.407},"last":1.385}]},{"osd":7,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.46,"15min":1.502},"min":{"1min":1.0169999999999999,"5min":0.438,"15min":0.438},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.508,"5min":1.54,"15min":1.546},"min":{"1min":1.196,"5min":0.747,"15min":0.747},"max":{"1min":1.7270000000000001,"5min":2.585,"15min":2.585},"last":1.6259999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":454,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194857,"statfs":{"total":200000000,"available":199534085,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.90600000000000003,"15min":0.79200000000000004},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2969999999999999,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.012,"15min":0.89300000000000002},"min":{"1min":0.48999999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.524,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":0.99099999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0309999999999999,"15min":0.95899999999999996},"min":{"1min":0.60099999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.633,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.2250000000000001,"15min":1.141},"min":{"1min":0.622,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.786,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.1379999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.079,"15min":1.0229999999999999},"min":{"1min":0.55200000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.675,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":0.755},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.163,"15min":1.073},"min":{"1min":0.59899999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.708,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":1.2569999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.5029999999999999,"15min":1.3180000000000001},"min":{"1min":0.91000000000000003,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.593,"5min":18.664999999999999,"15min":18.664999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.274,"15min":1.1719999999999999},"min":{"1min":0.93899999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.746,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":0.877}]},{"osd":5,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.3129999999999999,"15min":1.2290000000000001},"min":{"1min":1.014,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5700000000000001,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.5700000000000001,"15min":1.363},"min":{"1min":0.78300000000000003,"5min":0.748,"15min":0.748},"max":{"1min":1.9179999999999999,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.478}]},{"osd":6,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.5209999999999999,"15min":1.4350000000000001},"min":{"1min":1.115,"5min":0.625,"15min":0.625},"max":{"1min":1.7949999999999999,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4650000000000001,"15min":1.353},"min":{"1min":1.1339999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.772,"5min":7.242,"15min":7.242},"last":1.3959999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4870000000000001,"15min":1.361},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9950000000000001,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.379},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.6060000000000001,"15min":1.464},"min":{"1min":1.014,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0329999999999999,"5min":12.946,"15min":12.946},"last":1.4319999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":456,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194856,"statfs":{"total":200000000,"available":199532585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.87,"15min":0.92400000000000004},"min":{"1min":0.48599999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93400000000000005,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":0.873},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.96599999999999997,"15min":1.038},"min":{"1min":0.623,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3560000000000001,"5min":1.651,"15min":1.651},"last":1.123}]},{"osd":1,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.996,"15min":1.0449999999999999},"min":{"1min":0.67500000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3120000000000001,"5min":4.29,"15min":4.29},"last":1.04},{"interface":"front","average":{"1min":1.226,"5min":1.1020000000000001,"15min":1.1040000000000001},"min":{"1min":0.75,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5820000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.3320000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.0940000000000001,"15min":1.129},"min":{"1min":0.72599999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8109999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.345,"15min":1.4650000000000001},"min":{"1min":0.73399999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.7929999999999999,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":1.383}]},{"osd":4,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.403,"15min":1.48},"min":{"1min":0.95299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.6479999999999999,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.407,"15min":1.4470000000000001},"min":{"1min":1.1319999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.766,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.454}]},{"osd":5,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.244,"15min":1.2709999999999999},"min":{"1min":1.119,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.786,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.446,"15min":1.5580000000000001},"min":{"1min":1.204,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.794,"5min":7.657,"15min":7.657},"last":1.6299999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.4430000000000001,"15min":1.5209999999999999},"min":{"1min":1.2070000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0649999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.51,"5min":1.8420000000000001,"15min":1.5780000000000001},"min":{"1min":1.0069999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.9350000000000001,"5min":37.771999999999998,"15min":37.771999999999998},"last":1.8089999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.506,"15min":1.5649999999999999},"min":{"1min":1.1890000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.1930000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.587,"5min":1.4670000000000001,"15min":1.48},"min":{"1min":1.056,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1139999999999999,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.905}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":456,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194856,"statfs":{"total":200000000,"available":199532988,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.83599999999999997,"15min":0.81200000000000006},"min":{"1min":0.51000000000000001,"5min":0.315,"15min":0.315},"max":{"1min":1.343,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.302},{"interface":"front","average":{"1min":1.145,"5min":0.97999999999999998,"15min":0.92000000000000004},"min":{"1min":0.66600000000000004,"5min":0.443,"15min":0.443},"max":{"1min":1.399,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.2370000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":0.96399999999999997,"15min":0.93999999999999995},"min":{"1min":0.54900000000000004,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.456,"5min":1.859,"15min":1.859},"last":1.165},{"interface":"front","average":{"1min":1.028,"5min":0.90700000000000003,"15min":0.85999999999999999},"min":{"1min":0.70099999999999996,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.522,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.89200000000000002}]},{"osd":3,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.048,"15min":1.03},"min":{"1min":0.68500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.617,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.052,"15min":1.0569999999999999},"min":{"1min":0.82499999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5640000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":1.024}]},{"osd":4,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.2709999999999999,"15min":1.252},"min":{"1min":1.089,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.716,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.2749999999999999,"15min":1.2190000000000001},"min":{"1min":0.99199999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.774,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.276}]},{"osd":5,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.3320000000000001,"15min":1.302},"min":{"1min":1.111,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.161,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.345},{"interface":"front","average":{"1min":1.456,"5min":1.3979999999999999,"15min":1.3620000000000001},"min":{"1min":1.0820000000000001,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.7549999999999999,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.397}]},{"osd":6,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.3360000000000001,"15min":1.2929999999999999},"min":{"1min":1.079,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.857,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.3380000000000001,"15min":1.2789999999999999},"min":{"1min":1.0980000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.75,"5min":2.121,"15min":2.121},"last":1.423}]},{"osd":7,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.419,"15min":1.367},"min":{"1min":1.175,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.992,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.423,"15min":1.3680000000000001},"min":{"1min":1.256,"5min":0.746,"15min":0.746},"max":{"1min":2.0569999999999999,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.49}]}]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1030,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194282,"statfs":{"total":200000000,"available":198944848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84299999999999997,"15min":0.91900000000000004},"min":{"1min":0.65900000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.026,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":1.006},{"interface":"front","average":{"1min":1.171,"5min":1.075,"15min":1.0820000000000001},"min":{"1min":0.70199999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.70199999999999996}]},{"osd":2,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.86199999999999999,"15min":0.84999999999999998},"min":{"1min":0.77800000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.4159999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.143,"15min":1.131},"min":{"1min":0.63700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":1.24}]},{"osd":3,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.163,"15min":1.2869999999999999},"min":{"1min":0.69599999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.8200000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.494,"5min":1.4179999999999999,"15min":1.512},"min":{"1min":0.70099999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.3639999999999999,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":1.369}]},{"osd":4,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2330000000000001,"15min":1.3069999999999999},"min":{"1min":0.97999999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.8819999999999999,"5min":4.492,"15min":4.492},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.2809999999999999,"15min":1.2569999999999999},"min":{"1min":1.0449999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.5960000000000001,"5min":1.726,"15min":1.726},"last":1.526}]},{"osd":5,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.296,"15min":1.4219999999999999},"min":{"1min":0.85299999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":2.4510000000000001,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.284},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.3380000000000001,"15min":1.379},"min":{"1min":0.98999999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.784,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.464}]},{"osd":6,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.4970000000000001,"15min":1.5880000000000001},"min":{"1min":1.2509999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.0419999999999998,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4139999999999999,"15min":1.5009999999999999},"min":{"1min":1.159,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.1059999999999999,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.6879999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.631,"5min":1.569,"15min":1.645},"min":{"1min":1.2729999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.2000000000000002,"5min":3.492,"15min":3.492},"last":1.581},{"interface":"front","average":{"1min":1.536,"5min":1.512,"15min":1.6759999999999999},"min":{"1min":1.069,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.0009999999999999,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.425}]}]}],"pool_statfs":[]}} 2023-10-18T09:00:25.393 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:00:25.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.631+0000 7fc91759e640 1 -- 172.21.15.40:0/2476713237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc91806d6f0 msgr2=0x7fc9180b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:25.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.631+0000 7fc91759e640 1 --2- 172.21.15.40:0/2476713237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc91806d6f0 0x7fc9180b15b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fc91806aeb0 tx=0x7fc90802bd90 comp rx=0 tx=0).stop 2023-10-18T09:00:25.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.631+0000 7fc91759e640 1 -- 172.21.15.40:0/2476713237 shutdown_connections 2023-10-18T09:00:25.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.631+0000 7fc91759e640 1 --2- 172.21.15.40:0/2476713237 >> v2:172.21.15.144:3300/0 conn(0x7fc9180b0630 0x7fc9180b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:25.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.631+0000 7fc91759e640 1 --2- 172.21.15.40:0/2476713237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc91806d6f0 0x7fc9180b15b0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:25.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.631+0000 7fc91759e640 1 -- 172.21.15.40:0/2476713237 >> 172.21.15.40:0/2476713237 conn(0x7fc918069720 msgr2=0x7fc918069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:25.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.631+0000 7fc91759e640 1 -- 172.21.15.40:0/2476713237 shutdown_connections 2023-10-18T09:00:25.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.631+0000 7fc91759e640 1 -- 172.21.15.40:0/2476713237 wait complete. 2023-10-18T09:00:25.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.633+0000 7fc91759e640 1 Processor -- start 2023-10-18T09:00:25.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.633+0000 7fc91759e640 1 -- start start 2023-10-18T09:00:25.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.634+0000 7fc91759e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc91806d6f0 0x7fc91814cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:25.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.634+0000 7fc91759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9180b0630 0x7fc91814f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:25.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.634+0000 7fc91759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9180c99c0 0x7fc9180aba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:25.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.634+0000 7fc91759e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91814e890 con 0x7fc9180c99c0 2023-10-18T09:00:25.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.634+0000 7fc91759e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91814eb70 con 0x7fc9180b0630 2023-10-18T09:00:25.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.634+0000 7fc91759e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91814ee50 con 0x7fc91806d6f0 2023-10-18T09:00:25.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.635+0000 7fc916d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9180c99c0 0x7fc9180aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:25.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.635+0000 7fc916d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9180c99c0 0x7fc9180aba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40170/0 (socket says 172.21.15.40:40170) 2023-10-18T09:00:25.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.635+0000 7fc916d9d640 1 -- 172.21.15.40:0/1318368964 learned_addr learned my addr 172.21.15.40:0/1318368964 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:25.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.635+0000 7fc91659c640 1 --2- 172.21.15.40:0/1318368964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc91806d6f0 0x7fc91814cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:25.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.635+0000 7fc90ffff640 1 --2- 172.21.15.40:0/1318368964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9180b0630 0x7fc91814f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:25.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.635+0000 7fc916d9d640 1 -- 172.21.15.40:0/1318368964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc91806d6f0 msgr2=0x7fc91814cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:25.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.635+0000 7fc916d9d640 1 --2- 172.21.15.40:0/1318368964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc91806d6f0 0x7fc91814cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:25.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.635+0000 7fc916d9d640 1 -- 172.21.15.40:0/1318368964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9180b0630 msgr2=0x7fc91814f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:25.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.635+0000 7fc916d9d640 1 --2- 172.21.15.40:0/1318368964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9180b0630 0x7fc91814f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:25.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.635+0000 7fc916d9d640 1 -- 172.21.15.40:0/1318368964 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc908009560 con 0x7fc9180c99c0 2023-10-18T09:00:25.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.636+0000 7fc916d9d640 1 --2- 172.21.15.40:0/1318368964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9180c99c0 0x7fc9180aba00 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fc91000b990 tx=0x7fc91000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:25.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.636+0000 7fc90effd640 1 -- 172.21.15.40:0/1318368964 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91000cba0 con 0x7fc9180c99c0 2023-10-18T09:00:25.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.636+0000 7fc90effd640 1 -- 172.21.15.40:0/1318368964 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc91000cdf0 con 0x7fc9180c99c0 2023-10-18T09:00:25.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.636+0000 7fc90effd640 1 -- 172.21.15.40:0/1318368964 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9100146a0 con 0x7fc9180c99c0 2023-10-18T09:00:25.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.636+0000 7fc91759e640 1 -- 172.21.15.40:0/1318368964 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91815bef0 con 0x7fc9180c99c0 2023-10-18T09:00:25.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.636+0000 7fc91759e640 1 -- 172.21.15.40:0/1318368964 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91814f410 con 0x7fc9180c99c0 2023-10-18T09:00:25.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.637+0000 7fc91759e640 1 -- 172.21.15.40:0/1318368964 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91814f600 con 0x7fc9180c99c0 2023-10-18T09:00:25.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.638+0000 7fc90effd640 1 -- 172.21.15.40:0/1318368964 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc9100148c0 con 0x7fc9180c99c0 2023-10-18T09:00:25.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.639+0000 7fc90effd640 1 --2- 172.21.15.40:0/1318368964 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8f806ae40 0x7fc8f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:25.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.639+0000 7fc90effd640 1 -- 172.21.15.40:0/1318368964 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc910089810 con 0x7fc9180c99c0 2023-10-18T09:00:25.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.640+0000 7fc91659c640 1 --2- 172.21.15.40:0/1318368964 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8f806ae40 0x7fc8f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:25.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.641+0000 7fc91659c640 1 --2- 172.21.15.40:0/1318368964 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8f806ae40 0x7fc8f806d300 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc908035000 tx=0x7fc908004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:25.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.644+0000 7fc90effd640 1 -- 172.21.15.40:0/1318368964 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9100548b0 con 0x7fc9180c99c0 2023-10-18T09:00:25.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.832+0000 7fc91759e640 1 -- 172.21.15.40:0/1318368964 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc91814efc0 con 0x7fc8f806ae40 2023-10-18T09:00:25.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.857+0000 7fc90effd640 1 -- 172.21.15.40:0/1318368964 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282464 (secure 0 0 0) 0x7fc91814efc0 con 0x7fc8f806ae40 2023-10-18T09:00:25.866 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:25.868 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:00:25.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.863+0000 7fc91759e640 1 -- 172.21.15.40:0/1318368964 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8f806ae40 msgr2=0x7fc8f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:25.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.863+0000 7fc91759e640 1 --2- 172.21.15.40:0/1318368964 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8f806ae40 0x7fc8f806d300 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc908035000 tx=0x7fc908004730 comp rx=0 tx=0).stop 2023-10-18T09:00:25.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.863+0000 7fc91759e640 1 -- 172.21.15.40:0/1318368964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9180c99c0 msgr2=0x7fc9180aba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:25.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.863+0000 7fc91759e640 1 --2- 172.21.15.40:0/1318368964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9180c99c0 0x7fc9180aba00 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fc91000b990 tx=0x7fc91000be50 comp rx=0 tx=0).stop 2023-10-18T09:00:25.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.863+0000 7fc91759e640 1 -- 172.21.15.40:0/1318368964 shutdown_connections 2023-10-18T09:00:25.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.863+0000 7fc91759e640 1 --2- 172.21.15.40:0/1318368964 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8f806ae40 0x7fc8f806d300 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:25.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.863+0000 7fc91759e640 1 --2- 172.21.15.40:0/1318368964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9180c99c0 0x7fc9180aba00 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:25.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.863+0000 7fc91759e640 1 --2- 172.21.15.40:0/1318368964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9180b0630 0x7fc91814f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:25.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.863+0000 7fc91759e640 1 --2- 172.21.15.40:0/1318368964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc91806d6f0 0x7fc91814cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:25.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.863+0000 7fc91759e640 1 -- 172.21.15.40:0/1318368964 >> 172.21.15.40:0/1318368964 conn(0x7fc918069720 msgr2=0x7fc91806f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:25.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.863+0000 7fc91759e640 1 -- 172.21.15.40:0/1318368964 shutdown_connections 2023-10-18T09:00:25.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:25.864+0000 7fc91759e640 1 -- 172.21.15.40:0/1318368964 wait complete. 2023-10-18T09:00:25.887 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2023-10-18T09:00:25.007166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4793,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557706,"statfs":{"total":1600000000,"available":1595091077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.281960"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736891+0000","last_change":"2023-10-18T09:00:15.763191+0000","last_active":"2023-10-18T09:00:16.736891+0000","last_peered":"2023-10-18T09:00:16.736891+0000","last_clean":"2023-10-18T09:00:16.736891+0000","last_became_active":"2023-10-18T09:00:15.758469+0000","last_became_peered":"2023-10-18T09:00:15.758469+0000","last_unstale":"2023-10-18T09:00:16.736891+0000","last_undegraded":"2023-10-18T09:00:16.736891+0000","last_fullsized":"2023-10-18T09:00:16.736891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:03:56.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866602+0000","last_change":"2023-10-18T09:00:15.757911+0000","last_active":"2023-10-18T09:00:16.866602+0000","last_peered":"2023-10-18T09:00:16.866602+0000","last_clean":"2023-10-18T09:00:16.866602+0000","last_became_active":"2023-10-18T09:00:15.757452+0000","last_became_peered":"2023-10-18T09:00:15.757452+0000","last_unstale":"2023-10-18T09:00:16.866602+0000","last_undegraded":"2023-10-18T09:00:16.866602+0000","last_fullsized":"2023-10-18T09:00:16.866602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:19:41.723289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.734969+0000","last_change":"2023-10-18T09:00:15.761182+0000","last_active":"2023-10-18T09:00:16.734969+0000","last_peered":"2023-10-18T09:00:16.734969+0000","last_clean":"2023-10-18T09:00:16.734969+0000","last_became_active":"2023-10-18T09:00:15.760952+0000","last_became_peered":"2023-10-18T09:00:15.760952+0000","last_unstale":"2023-10-18T09:00:16.734969+0000","last_undegraded":"2023-10-18T09:00:16.734969+0000","last_fullsized":"2023-10-18T09:00:16.734969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:54:55.101341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.865987+0000","last_change":"2023-10-18T09:00:15.762841+0000","last_active":"2023-10-18T09:00:16.865987+0000","last_peered":"2023-10-18T09:00:16.865987+0000","last_clean":"2023-10-18T09:00:16.865987+0000","last_became_active":"2023-10-18T09:00:15.762393+0000","last_became_peered":"2023-10-18T09:00:15.762393+0000","last_unstale":"2023-10-18T09:00:16.865987+0000","last_undegraded":"2023-10-18T09:00:16.865987+0000","last_fullsized":"2023-10-18T09:00:16.865987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:51:04.245123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736606+0000","last_change":"2023-10-18T09:00:15.755929+0000","last_active":"2023-10-18T09:00:16.736606+0000","last_peered":"2023-10-18T09:00:16.736606+0000","last_clean":"2023-10-18T09:00:16.736606+0000","last_became_active":"2023-10-18T09:00:15.755639+0000","last_became_peered":"2023-10-18T09:00:15.755639+0000","last_unstale":"2023-10-18T09:00:16.736606+0000","last_undegraded":"2023-10-18T09:00:16.736606+0000","last_fullsized":"2023-10-18T09:00:16.736606+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:50.213859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.869497+0000","last_change":"2023-10-18T09:00:15.765807+0000","last_active":"2023-10-18T09:00:16.869497+0000","last_peered":"2023-10-18T09:00:16.869497+0000","last_clean":"2023-10-18T09:00:16.869497+0000","last_became_active":"2023-10-18T09:00:15.765306+0000","last_became_peered":"2023-10-18T09:00:15.765306+0000","last_unstale":"2023-10-18T09:00:16.869497+0000","last_undegraded":"2023-10-18T09:00:16.869497+0000","last_fullsized":"2023-10-18T09:00:16.869497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:28:57.510395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.870713+0000","last_change":"2023-10-18T09:00:15.758255+0000","last_active":"2023-10-18T09:00:16.870713+0000","last_peered":"2023-10-18T09:00:16.870713+0000","last_clean":"2023-10-18T09:00:16.870713+0000","last_became_active":"2023-10-18T09:00:15.756071+0000","last_became_peered":"2023-10-18T09:00:15.756071+0000","last_unstale":"2023-10-18T09:00:16.870713+0000","last_undegraded":"2023-10-18T09:00:16.870713+0000","last_fullsized":"2023-10-18T09:00:16.870713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:46:39.794722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866761+0000","last_change":"2023-10-18T09:00:15.760360+0000","last_active":"2023-10-18T09:00:16.866761+0000","last_peered":"2023-10-18T09:00:16.866761+0000","last_clean":"2023-10-18T09:00:16.866761+0000","last_became_active":"2023-10-18T09:00:15.755866+0000","last_became_peered":"2023-10-18T09:00:15.755866+0000","last_unstale":"2023-10-18T09:00:16.866761+0000","last_undegraded":"2023-10-18T09:00:16.866761+0000","last_fullsized":"2023-10-18T09:00:16.866761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:24:22.148863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736100+0000","last_change":"2023-10-18T09:00:15.758789+0000","last_active":"2023-10-18T09:00:16.736100+0000","last_peered":"2023-10-18T09:00:16.736100+0000","last_clean":"2023-10-18T09:00:16.736100+0000","last_became_active":"2023-10-18T09:00:15.758487+0000","last_became_peered":"2023-10-18T09:00:15.758487+0000","last_unstale":"2023-10-18T09:00:16.736100+0000","last_undegraded":"2023-10-18T09:00:16.736100+0000","last_fullsized":"2023-10-18T09:00:16.736100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:51:02.550237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867399+0000","last_change":"2023-10-18T09:00:15.758617+0000","last_active":"2023-10-18T09:00:16.867399+0000","last_peered":"2023-10-18T09:00:16.867399+0000","last_clean":"2023-10-18T09:00:16.867399+0000","last_became_active":"2023-10-18T09:00:15.758324+0000","last_became_peered":"2023-10-18T09:00:15.758324+0000","last_unstale":"2023-10-18T09:00:16.867399+0000","last_undegraded":"2023-10-18T09:00:16.867399+0000","last_fullsized":"2023-10-18T09:00:16.867399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:03:24.223447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737441+0000","last_change":"2023-10-18T09:00:15.736563+0000","last_active":"2023-10-18T09:00:16.737441+0000","last_peered":"2023-10-18T09:00:16.737441+0000","last_clean":"2023-10-18T09:00:16.737441+0000","last_became_active":"2023-10-18T09:00:15.736013+0000","last_became_peered":"2023-10-18T09:00:15.736013+0000","last_unstale":"2023-10-18T09:00:16.737441+0000","last_undegraded":"2023-10-18T09:00:16.737441+0000","last_fullsized":"2023-10-18T09:00:16.737441+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:14:19.706536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866898+0000","last_change":"2023-10-18T09:00:15.763124+0000","last_active":"2023-10-18T09:00:16.866898+0000","last_peered":"2023-10-18T09:00:16.866898+0000","last_clean":"2023-10-18T09:00:16.866898+0000","last_became_active":"2023-10-18T09:00:15.762786+0000","last_became_peered":"2023-10-18T09:00:15.762786+0000","last_unstale":"2023-10-18T09:00:16.866898+0000","last_undegraded":"2023-10-18T09:00:16.866898+0000","last_fullsized":"2023-10-18T09:00:16.866898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:12:39.359221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.865876+0000","last_change":"2023-10-18T09:00:15.768782+0000","last_active":"2023-10-18T09:00:16.865876+0000","last_peered":"2023-10-18T09:00:16.865876+0000","last_clean":"2023-10-18T09:00:16.865876+0000","last_became_active":"2023-10-18T09:00:15.767657+0000","last_became_peered":"2023-10-18T09:00:15.767657+0000","last_unstale":"2023-10-18T09:00:16.865876+0000","last_undegraded":"2023-10-18T09:00:16.865876+0000","last_fullsized":"2023-10-18T09:00:16.865876+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:17:29.238634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736280+0000","last_change":"2023-10-18T09:00:15.752025+0000","last_active":"2023-10-18T09:00:16.736280+0000","last_peered":"2023-10-18T09:00:16.736280+0000","last_clean":"2023-10-18T09:00:16.736280+0000","last_became_active":"2023-10-18T09:00:15.751249+0000","last_became_peered":"2023-10-18T09:00:15.751249+0000","last_unstale":"2023-10-18T09:00:16.736280+0000","last_undegraded":"2023-10-18T09:00:16.736280+0000","last_fullsized":"2023-10-18T09:00:16.736280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:56:40.431057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.865758+0000","last_change":"2023-10-18T09:00:15.761180+0000","last_active":"2023-10-18T09:00:16.865758+0000","last_peered":"2023-10-18T09:00:16.865758+0000","last_clean":"2023-10-18T09:00:16.865758+0000","last_became_active":"2023-10-18T09:00:15.760931+0000","last_became_peered":"2023-10-18T09:00:15.760931+0000","last_unstale":"2023-10-18T09:00:16.865758+0000","last_undegraded":"2023-10-18T09:00:16.865758+0000","last_fullsized":"2023-10-18T09:00:16.865758+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:44.909930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866905+0000","last_change":"2023-10-18T09:00:15.755241+0000","last_active":"2023-10-18T09:00:16.866905+0000","last_peered":"2023-10-18T09:00:16.866905+0000","last_clean":"2023-10-18T09:00:16.866905+0000","last_became_active":"2023-10-18T09:00:15.754219+0000","last_became_peered":"2023-10-18T09:00:15.754219+0000","last_unstale":"2023-10-18T09:00:16.866905+0000","last_undegraded":"2023-10-18T09:00:16.866905+0000","last_fullsized":"2023-10-18T09:00:16.866905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:10:37.475877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736469+0000","last_change":"2023-10-18T09:00:15.762556+0000","last_active":"2023-10-18T09:00:16.736469+0000","last_peered":"2023-10-18T09:00:16.736469+0000","last_clean":"2023-10-18T09:00:16.736469+0000","last_became_active":"2023-10-18T09:00:15.762213+0000","last_became_peered":"2023-10-18T09:00:15.762213+0000","last_unstale":"2023-10-18T09:00:16.736469+0000","last_undegraded":"2023-10-18T09:00:16.736469+0000","last_fullsized":"2023-10-18T09:00:16.736469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:13:20.414827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.865832+0000","last_change":"2023-10-18T09:00:15.756905+0000","last_active":"2023-10-18T09:00:16.865832+0000","last_peered":"2023-10-18T09:00:16.865832+0000","last_clean":"2023-10-18T09:00:16.865832+0000","last_became_active":"2023-10-18T09:00:15.753779+0000","last_became_peered":"2023-10-18T09:00:15.753779+0000","last_unstale":"2023-10-18T09:00:16.865832+0000","last_undegraded":"2023-10-18T09:00:16.865832+0000","last_fullsized":"2023-10-18T09:00:16.865832+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:15:47.083054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866960+0000","last_change":"2023-10-18T09:00:15.765485+0000","last_active":"2023-10-18T09:00:16.866960+0000","last_peered":"2023-10-18T09:00:16.866960+0000","last_clean":"2023-10-18T09:00:16.866960+0000","last_became_active":"2023-10-18T09:00:15.764978+0000","last_became_peered":"2023-10-18T09:00:15.764978+0000","last_unstale":"2023-10-18T09:00:16.866960+0000","last_undegraded":"2023-10-18T09:00:16.866960+0000","last_fullsized":"2023-10-18T09:00:16.866960+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:28:45.528730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866110+0000","last_change":"2023-10-18T09:00:15.757707+0000","last_active":"2023-10-18T09:00:16.866110+0000","last_peered":"2023-10-18T09:00:16.866110+0000","last_clean":"2023-10-18T09:00:16.866110+0000","last_became_active":"2023-10-18T09:00:15.757478+0000","last_became_peered":"2023-10-18T09:00:15.757478+0000","last_unstale":"2023-10-18T09:00:16.866110+0000","last_undegraded":"2023-10-18T09:00:16.866110+0000","last_fullsized":"2023-10-18T09:00:16.866110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:00.837466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867390+0000","last_change":"2023-10-18T09:00:15.762884+0000","last_active":"2023-10-18T09:00:16.867390+0000","last_peered":"2023-10-18T09:00:16.867390+0000","last_clean":"2023-10-18T09:00:16.867390+0000","last_became_active":"2023-10-18T09:00:15.762479+0000","last_became_peered":"2023-10-18T09:00:15.762479+0000","last_unstale":"2023-10-18T09:00:16.867390+0000","last_undegraded":"2023-10-18T09:00:16.867390+0000","last_fullsized":"2023-10-18T09:00:16.867390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:26:17.885550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867242+0000","last_change":"2023-10-18T09:00:15.753122+0000","last_active":"2023-10-18T09:00:16.867242+0000","last_peered":"2023-10-18T09:00:16.867242+0000","last_clean":"2023-10-18T09:00:16.867242+0000","last_became_active":"2023-10-18T09:00:15.752653+0000","last_became_peered":"2023-10-18T09:00:15.752653+0000","last_unstale":"2023-10-18T09:00:16.867242+0000","last_undegraded":"2023-10-18T09:00:16.867242+0000","last_fullsized":"2023-10-18T09:00:16.867242+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:15:32.449721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736879+0000","last_change":"2023-10-18T09:00:15.749937+0000","last_active":"2023-10-18T09:00:16.736879+0000","last_peered":"2023-10-18T09:00:16.736879+0000","last_clean":"2023-10-18T09:00:16.736879+0000","last_became_active":"2023-10-18T09:00:15.749226+0000","last_became_peered":"2023-10-18T09:00:15.749226+0000","last_unstale":"2023-10-18T09:00:16.736879+0000","last_undegraded":"2023-10-18T09:00:16.736879+0000","last_fullsized":"2023-10-18T09:00:16.736879+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:28.075342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736586+0000","last_change":"2023-10-18T09:00:15.762946+0000","last_active":"2023-10-18T09:00:16.736586+0000","last_peered":"2023-10-18T09:00:16.736586+0000","last_clean":"2023-10-18T09:00:16.736586+0000","last_became_active":"2023-10-18T09:00:15.762276+0000","last_became_peered":"2023-10-18T09:00:15.762276+0000","last_unstale":"2023-10-18T09:00:16.736586+0000","last_undegraded":"2023-10-18T09:00:16.736586+0000","last_fullsized":"2023-10-18T09:00:16.736586+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:28:15.286205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867778+0000","last_change":"2023-10-18T09:00:15.737169+0000","last_active":"2023-10-18T09:00:16.867778+0000","last_peered":"2023-10-18T09:00:16.867778+0000","last_clean":"2023-10-18T09:00:16.867778+0000","last_became_active":"2023-10-18T09:00:15.736508+0000","last_became_peered":"2023-10-18T09:00:15.736508+0000","last_unstale":"2023-10-18T09:00:16.867778+0000","last_undegraded":"2023-10-18T09:00:16.867778+0000","last_fullsized":"2023-10-18T09:00:16.867778+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:32:57.960964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.868854+0000","last_change":"2023-10-18T09:00:15.757071+0000","last_active":"2023-10-18T09:00:16.868854+0000","last_peered":"2023-10-18T09:00:16.868854+0000","last_clean":"2023-10-18T09:00:16.868854+0000","last_became_active":"2023-10-18T09:00:15.754963+0000","last_became_peered":"2023-10-18T09:00:15.754963+0000","last_unstale":"2023-10-18T09:00:16.868854+0000","last_undegraded":"2023-10-18T09:00:16.868854+0000","last_fullsized":"2023-10-18T09:00:16.868854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:38.079651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.735091+0000","last_change":"2023-10-18T09:00:15.734271+0000","last_active":"2023-10-18T09:00:16.735091+0000","last_peered":"2023-10-18T09:00:16.735091+0000","last_clean":"2023-10-18T09:00:16.735091+0000","last_became_active":"2023-10-18T09:00:15.733788+0000","last_became_peered":"2023-10-18T09:00:15.733788+0000","last_unstale":"2023-10-18T09:00:16.735091+0000","last_undegraded":"2023-10-18T09:00:16.735091+0000","last_fullsized":"2023-10-18T09:00:16.735091+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:24:56.735813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867924+0000","last_change":"2023-10-18T09:00:15.756508+0000","last_active":"2023-10-18T09:00:16.867924+0000","last_peered":"2023-10-18T09:00:16.867924+0000","last_clean":"2023-10-18T09:00:16.867924+0000","last_became_active":"2023-10-18T09:00:15.756155+0000","last_became_peered":"2023-10-18T09:00:15.756155+0000","last_unstale":"2023-10-18T09:00:16.867924+0000","last_undegraded":"2023-10-18T09:00:16.867924+0000","last_fullsized":"2023-10-18T09:00:16.867924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:59:38.079651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.734779+0000","last_change":"2023-10-18T09:00:15.754144+0000","last_active":"2023-10-18T09:00:16.734779+0000","last_peered":"2023-10-18T09:00:16.734779+0000","last_clean":"2023-10-18T09:00:16.734779+0000","last_became_active":"2023-10-18T09:00:15.748159+0000","last_became_peered":"2023-10-18T09:00:15.748159+0000","last_unstale":"2023-10-18T09:00:16.734779+0000","last_undegraded":"2023-10-18T09:00:16.734779+0000","last_fullsized":"2023-10-18T09:00:16.734779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:26:54.506506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.871708+0000","last_change":"2023-10-18T09:00:15.755928+0000","last_active":"2023-10-18T09:00:16.871708+0000","last_peered":"2023-10-18T09:00:16.871708+0000","last_clean":"2023-10-18T09:00:16.871708+0000","last_became_active":"2023-10-18T09:00:15.755233+0000","last_became_peered":"2023-10-18T09:00:15.755233+0000","last_unstale":"2023-10-18T09:00:16.871708+0000","last_undegraded":"2023-10-18T09:00:16.871708+0000","last_fullsized":"2023-10-18T09:00:16.871708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:06.826641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.869159+0000","last_change":"2023-10-18T09:00:15.760173+0000","last_active":"2023-10-18T09:00:16.869159+0000","last_peered":"2023-10-18T09:00:16.869159+0000","last_clean":"2023-10-18T09:00:16.869159+0000","last_became_active":"2023-10-18T09:00:15.759923+0000","last_became_peered":"2023-10-18T09:00:15.759923+0000","last_unstale":"2023-10-18T09:00:16.869159+0000","last_undegraded":"2023-10-18T09:00:16.869159+0000","last_fullsized":"2023-10-18T09:00:16.869159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:22.137803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866984+0000","last_change":"2023-10-18T09:00:15.765266+0000","last_active":"2023-10-18T09:00:16.866984+0000","last_peered":"2023-10-18T09:00:16.866984+0000","last_clean":"2023-10-18T09:00:16.866984+0000","last_became_active":"2023-10-18T09:00:15.764999+0000","last_became_peered":"2023-10-18T09:00:15.764999+0000","last_unstale":"2023-10-18T09:00:16.866984+0000","last_undegraded":"2023-10-18T09:00:16.866984+0000","last_fullsized":"2023-10-18T09:00:16.866984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:38.805225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737031+0000","last_change":"2023-10-18T09:00:15.762805+0000","last_active":"2023-10-18T09:00:16.737031+0000","last_peered":"2023-10-18T09:00:16.737031+0000","last_clean":"2023-10-18T09:00:16.737031+0000","last_became_active":"2023-10-18T09:00:15.761946+0000","last_became_peered":"2023-10-18T09:00:15.761946+0000","last_unstale":"2023-10-18T09:00:16.737031+0000","last_undegraded":"2023-10-18T09:00:16.737031+0000","last_fullsized":"2023-10-18T09:00:16.737031+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:28:00.652852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"86'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773766+0000","last_change":"2023-10-18T09:00:14.719832+0000","last_active":"2023-10-18T09:00:15.773766+0000","last_peered":"2023-10-18T09:00:15.773766+0000","last_clean":"2023-10-18T09:00:15.773766+0000","last_became_active":"2023-10-18T09:00:14.719074+0000","last_became_peered":"2023-10-18T09:00:14.719074+0000","last_unstale":"2023-10-18T09:00:15.773766+0000","last_undegraded":"2023-10-18T09:00:15.773766+0000","last_fullsized":"2023-10-18T09:00:15.773766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:31:09.739409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736246+0000","last_change":"2023-10-18T09:00:15.761074+0000","last_active":"2023-10-18T09:00:16.736246+0000","last_peered":"2023-10-18T09:00:16.736246+0000","last_clean":"2023-10-18T09:00:16.736246+0000","last_became_active":"2023-10-18T09:00:15.757708+0000","last_became_peered":"2023-10-18T09:00:15.757708+0000","last_unstale":"2023-10-18T09:00:16.736246+0000","last_undegraded":"2023-10-18T09:00:16.736246+0000","last_fullsized":"2023-10-18T09:00:16.736246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:16:51.405175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.772773+0000","last_change":"2023-10-18T09:00:14.722227+0000","last_active":"2023-10-18T09:00:15.772773+0000","last_peered":"2023-10-18T09:00:15.772773+0000","last_clean":"2023-10-18T09:00:15.772773+0000","last_became_active":"2023-10-18T09:00:14.720313+0000","last_became_peered":"2023-10-18T09:00:14.720313+0000","last_unstale":"2023-10-18T09:00:15.772773+0000","last_undegraded":"2023-10-18T09:00:15.772773+0000","last_fullsized":"2023-10-18T09:00:15.772773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:28:25.001764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.869848+0000","last_change":"2023-10-18T09:00:15.753770+0000","last_active":"2023-10-18T09:00:16.869848+0000","last_peered":"2023-10-18T09:00:16.869848+0000","last_clean":"2023-10-18T09:00:16.869848+0000","last_became_active":"2023-10-18T09:00:15.752239+0000","last_became_peered":"2023-10-18T09:00:15.752239+0000","last_unstale":"2023-10-18T09:00:16.869848+0000","last_undegraded":"2023-10-18T09:00:16.869848+0000","last_fullsized":"2023-10-18T09:00:16.869848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:04:34.318321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773255+0000","last_change":"2023-10-18T09:00:14.724618+0000","last_active":"2023-10-18T09:00:15.773255+0000","last_peered":"2023-10-18T09:00:15.773255+0000","last_clean":"2023-10-18T09:00:15.773255+0000","last_became_active":"2023-10-18T09:00:14.721500+0000","last_became_peered":"2023-10-18T09:00:14.721500+0000","last_unstale":"2023-10-18T09:00:15.773255+0000","last_undegraded":"2023-10-18T09:00:15.773255+0000","last_fullsized":"2023-10-18T09:00:15.773255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:59.045131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737858+0000","last_change":"2023-10-18T09:00:15.759339+0000","last_active":"2023-10-18T09:00:16.737858+0000","last_peered":"2023-10-18T09:00:16.737858+0000","last_clean":"2023-10-18T09:00:16.737858+0000","last_became_active":"2023-10-18T09:00:15.758140+0000","last_became_peered":"2023-10-18T09:00:15.758140+0000","last_unstale":"2023-10-18T09:00:16.737858+0000","last_undegraded":"2023-10-18T09:00:16.737858+0000","last_fullsized":"2023-10-18T09:00:16.737858+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:17:29.238634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773868+0000","last_change":"2023-10-18T09:00:14.729343+0000","last_active":"2023-10-18T09:00:15.773868+0000","last_peered":"2023-10-18T09:00:15.773868+0000","last_clean":"2023-10-18T09:00:15.773868+0000","last_became_active":"2023-10-18T09:00:14.728957+0000","last_became_peered":"2023-10-18T09:00:14.728957+0000","last_unstale":"2023-10-18T09:00:15.773868+0000","last_undegraded":"2023-10-18T09:00:15.773868+0000","last_fullsized":"2023-10-18T09:00:15.773868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:12:38.346335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.869564+0000","last_change":"2023-10-18T09:00:15.758912+0000","last_active":"2023-10-18T09:00:16.869564+0000","last_peered":"2023-10-18T09:00:16.869564+0000","last_clean":"2023-10-18T09:00:16.869564+0000","last_became_active":"2023-10-18T09:00:15.756824+0000","last_became_peered":"2023-10-18T09:00:15.756824+0000","last_unstale":"2023-10-18T09:00:16.869564+0000","last_undegraded":"2023-10-18T09:00:16.869564+0000","last_fullsized":"2023-10-18T09:00:16.869564+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:24:27.536167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.772736+0000","last_change":"2023-10-18T09:00:14.720994+0000","last_active":"2023-10-18T09:00:15.772736+0000","last_peered":"2023-10-18T09:00:15.772736+0000","last_clean":"2023-10-18T09:00:15.772736+0000","last_became_active":"2023-10-18T09:00:14.720661+0000","last_became_peered":"2023-10-18T09:00:14.720661+0000","last_unstale":"2023-10-18T09:00:15.772736+0000","last_undegraded":"2023-10-18T09:00:15.772736+0000","last_fullsized":"2023-10-18T09:00:15.772736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:55:27.065983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867552+0000","last_change":"2023-10-18T09:00:15.737040+0000","last_active":"2023-10-18T09:00:16.867552+0000","last_peered":"2023-10-18T09:00:16.867552+0000","last_clean":"2023-10-18T09:00:16.867552+0000","last_became_active":"2023-10-18T09:00:15.736400+0000","last_became_peered":"2023-10-18T09:00:15.736400+0000","last_unstale":"2023-10-18T09:00:16.867552+0000","last_undegraded":"2023-10-18T09:00:16.867552+0000","last_fullsized":"2023-10-18T09:00:16.867552+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:27:57.117593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773337+0000","last_change":"2023-10-18T09:00:14.726650+0000","last_active":"2023-10-18T09:00:15.773337+0000","last_peered":"2023-10-18T09:00:15.773337+0000","last_clean":"2023-10-18T09:00:15.773337+0000","last_became_active":"2023-10-18T09:00:14.726047+0000","last_became_peered":"2023-10-18T09:00:14.726047+0000","last_unstale":"2023-10-18T09:00:15.773337+0000","last_undegraded":"2023-10-18T09:00:15.773337+0000","last_fullsized":"2023-10-18T09:00:15.773337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:55:25.382068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736921+0000","last_change":"2023-10-18T09:00:15.759622+0000","last_active":"2023-10-18T09:00:16.736921+0000","last_peered":"2023-10-18T09:00:16.736921+0000","last_clean":"2023-10-18T09:00:16.736921+0000","last_became_active":"2023-10-18T09:00:15.759302+0000","last_became_peered":"2023-10-18T09:00:15.759302+0000","last_unstale":"2023-10-18T09:00:16.736921+0000","last_undegraded":"2023-10-18T09:00:16.736921+0000","last_fullsized":"2023-10-18T09:00:16.736921+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:25:33.396541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.749958+0000","last_change":"2023-10-18T09:00:14.722001+0000","last_active":"2023-10-18T09:00:15.749958+0000","last_peered":"2023-10-18T09:00:15.749958+0000","last_clean":"2023-10-18T09:00:15.749958+0000","last_became_active":"2023-10-18T09:00:14.721325+0000","last_became_peered":"2023-10-18T09:00:14.721325+0000","last_unstale":"2023-10-18T09:00:15.749958+0000","last_undegraded":"2023-10-18T09:00:15.749958+0000","last_fullsized":"2023-10-18T09:00:15.749958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:05:08.365526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.865724+0000","last_change":"2023-10-18T09:00:15.765115+0000","last_active":"2023-10-18T09:00:16.865724+0000","last_peered":"2023-10-18T09:00:16.865724+0000","last_clean":"2023-10-18T09:00:16.865724+0000","last_became_active":"2023-10-18T09:00:15.764828+0000","last_became_peered":"2023-10-18T09:00:15.764828+0000","last_unstale":"2023-10-18T09:00:16.865724+0000","last_undegraded":"2023-10-18T09:00:16.865724+0000","last_fullsized":"2023-10-18T09:00:16.865724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:43:18.411223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.751377+0000","last_change":"2023-10-18T09:00:14.719395+0000","last_active":"2023-10-18T09:00:15.751377+0000","last_peered":"2023-10-18T09:00:15.751377+0000","last_clean":"2023-10-18T09:00:15.751377+0000","last_became_active":"2023-10-18T09:00:14.718426+0000","last_became_peered":"2023-10-18T09:00:14.718426+0000","last_unstale":"2023-10-18T09:00:15.751377+0000","last_undegraded":"2023-10-18T09:00:15.751377+0000","last_fullsized":"2023-10-18T09:00:15.751377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:42:14.030822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.871025+0000","last_change":"2023-10-18T09:00:15.758761+0000","last_active":"2023-10-18T09:00:16.871025+0000","last_peered":"2023-10-18T09:00:16.871025+0000","last_clean":"2023-10-18T09:00:16.871025+0000","last_became_active":"2023-10-18T09:00:15.758524+0000","last_became_peered":"2023-10-18T09:00:15.758524+0000","last_unstale":"2023-10-18T09:00:16.871025+0000","last_undegraded":"2023-10-18T09:00:16.871025+0000","last_fullsized":"2023-10-18T09:00:16.871025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:23.871839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.730864+0000","last_change":"2023-10-18T09:00:14.713528+0000","last_active":"2023-10-18T09:00:15.730864+0000","last_peered":"2023-10-18T09:00:15.730864+0000","last_clean":"2023-10-18T09:00:15.730864+0000","last_became_active":"2023-10-18T09:00:14.712766+0000","last_became_peered":"2023-10-18T09:00:14.712766+0000","last_unstale":"2023-10-18T09:00:15.730864+0000","last_undegraded":"2023-10-18T09:00:15.730864+0000","last_fullsized":"2023-10-18T09:00:15.730864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:37:44.788135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737694+0000","last_change":"2023-10-18T09:00:15.758755+0000","last_active":"2023-10-18T09:00:16.737694+0000","last_peered":"2023-10-18T09:00:16.737694+0000","last_clean":"2023-10-18T09:00:16.737694+0000","last_became_active":"2023-10-18T09:00:15.757506+0000","last_became_peered":"2023-10-18T09:00:15.757506+0000","last_unstale":"2023-10-18T09:00:16.737694+0000","last_undegraded":"2023-10-18T09:00:16.737694+0000","last_fullsized":"2023-10-18T09:00:16.737694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:43:18.411223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.748766+0000","last_change":"2023-10-18T09:00:14.728500+0000","last_active":"2023-10-18T09:00:15.748766+0000","last_peered":"2023-10-18T09:00:15.748766+0000","last_clean":"2023-10-18T09:00:15.748766+0000","last_became_active":"2023-10-18T09:00:14.728243+0000","last_became_peered":"2023-10-18T09:00:14.728243+0000","last_unstale":"2023-10-18T09:00:15.748766+0000","last_undegraded":"2023-10-18T09:00:15.748766+0000","last_fullsized":"2023-10-18T09:00:15.748766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:51:03.232237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867311+0000","last_change":"2023-10-18T09:00:15.759536+0000","last_active":"2023-10-18T09:00:16.867311+0000","last_peered":"2023-10-18T09:00:16.867311+0000","last_clean":"2023-10-18T09:00:16.867311+0000","last_became_active":"2023-10-18T09:00:15.759116+0000","last_became_peered":"2023-10-18T09:00:15.759116+0000","last_unstale":"2023-10-18T09:00:16.867311+0000","last_undegraded":"2023-10-18T09:00:16.867311+0000","last_fullsized":"2023-10-18T09:00:16.867311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:34:17.782897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"86'3","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.774146+0000","last_change":"2023-10-18T09:00:14.723428+0000","last_active":"2023-10-18T09:00:15.774146+0000","last_peered":"2023-10-18T09:00:15.774146+0000","last_clean":"2023-10-18T09:00:15.774146+0000","last_became_active":"2023-10-18T09:00:14.723174+0000","last_became_peered":"2023-10-18T09:00:14.723174+0000","last_unstale":"2023-10-18T09:00:15.774146+0000","last_undegraded":"2023-10-18T09:00:15.774146+0000","last_fullsized":"2023-10-18T09:00:15.774146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:53:02.168967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736447+0000","last_change":"2023-10-18T09:00:15.755629+0000","last_active":"2023-10-18T09:00:16.736447+0000","last_peered":"2023-10-18T09:00:16.736447+0000","last_clean":"2023-10-18T09:00:16.736447+0000","last_became_active":"2023-10-18T09:00:15.755325+0000","last_became_peered":"2023-10-18T09:00:15.755325+0000","last_unstale":"2023-10-18T09:00:16.736447+0000","last_undegraded":"2023-10-18T09:00:16.736447+0000","last_fullsized":"2023-10-18T09:00:16.736447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:32:40.008198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"86'6","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.776851+0000","last_change":"2023-10-18T09:00:14.720054+0000","last_active":"2023-10-18T09:00:15.776851+0000","last_peered":"2023-10-18T09:00:15.776851+0000","last_clean":"2023-10-18T09:00:15.776851+0000","last_became_active":"2023-10-18T09:00:14.719704+0000","last_became_peered":"2023-10-18T09:00:14.719704+0000","last_unstale":"2023-10-18T09:00:15.776851+0000","last_undegraded":"2023-10-18T09:00:15.776851+0000","last_fullsized":"2023-10-18T09:00:15.776851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:22:27.935440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737925+0000","last_change":"2023-10-18T09:00:15.758943+0000","last_active":"2023-10-18T09:00:16.737925+0000","last_peered":"2023-10-18T09:00:16.737925+0000","last_clean":"2023-10-18T09:00:16.737925+0000","last_became_active":"2023-10-18T09:00:15.758060+0000","last_became_peered":"2023-10-18T09:00:15.758060+0000","last_unstale":"2023-10-18T09:00:16.737925+0000","last_undegraded":"2023-10-18T09:00:16.737925+0000","last_fullsized":"2023-10-18T09:00:16.737925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:11:00.837466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.774849+0000","last_change":"2023-10-18T09:00:14.719224+0000","last_active":"2023-10-18T09:00:15.774849+0000","last_peered":"2023-10-18T09:00:15.774849+0000","last_clean":"2023-10-18T09:00:15.774849+0000","last_became_active":"2023-10-18T09:00:14.718827+0000","last_became_peered":"2023-10-18T09:00:14.718827+0000","last_unstale":"2023-10-18T09:00:15.774849+0000","last_undegraded":"2023-10-18T09:00:15.774849+0000","last_fullsized":"2023-10-18T09:00:15.774849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:55:25.382068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.872019+0000","last_change":"2023-10-18T09:00:15.753951+0000","last_active":"2023-10-18T09:00:16.872019+0000","last_peered":"2023-10-18T09:00:16.872019+0000","last_clean":"2023-10-18T09:00:16.872019+0000","last_became_active":"2023-10-18T09:00:15.753614+0000","last_became_peered":"2023-10-18T09:00:15.753614+0000","last_unstale":"2023-10-18T09:00:16.872019+0000","last_undegraded":"2023-10-18T09:00:16.872019+0000","last_fullsized":"2023-10-18T09:00:16.872019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:31.329353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.750279+0000","last_change":"2023-10-18T09:00:14.724705+0000","last_active":"2023-10-18T09:00:15.750279+0000","last_peered":"2023-10-18T09:00:15.750279+0000","last_clean":"2023-10-18T09:00:15.750279+0000","last_became_active":"2023-10-18T09:00:14.724414+0000","last_became_peered":"2023-10-18T09:00:14.724414+0000","last_unstale":"2023-10-18T09:00:15.750279+0000","last_undegraded":"2023-10-18T09:00:15.750279+0000","last_fullsized":"2023-10-18T09:00:15.750279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:53.753897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866195+0000","last_change":"2023-10-18T09:00:15.767934+0000","last_active":"2023-10-18T09:00:16.866195+0000","last_peered":"2023-10-18T09:00:16.866195+0000","last_clean":"2023-10-18T09:00:16.866195+0000","last_became_active":"2023-10-18T09:00:15.767341+0000","last_became_peered":"2023-10-18T09:00:15.767341+0000","last_unstale":"2023-10-18T09:00:16.866195+0000","last_undegraded":"2023-10-18T09:00:16.866195+0000","last_fullsized":"2023-10-18T09:00:16.866195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:28.179544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.745451+0000","last_change":"2023-10-18T09:00:14.721694+0000","last_active":"2023-10-18T09:00:15.745451+0000","last_peered":"2023-10-18T09:00:15.745451+0000","last_clean":"2023-10-18T09:00:15.745451+0000","last_became_active":"2023-10-18T09:00:14.720757+0000","last_became_peered":"2023-10-18T09:00:14.720757+0000","last_unstale":"2023-10-18T09:00:15.745451+0000","last_undegraded":"2023-10-18T09:00:15.745451+0000","last_fullsized":"2023-10-18T09:00:15.745451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:26:16.872665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736651+0000","last_change":"2023-10-18T09:00:15.759417+0000","last_active":"2023-10-18T09:00:16.736651+0000","last_peered":"2023-10-18T09:00:16.736651+0000","last_clean":"2023-10-18T09:00:16.736651+0000","last_became_active":"2023-10-18T09:00:15.757457+0000","last_became_peered":"2023-10-18T09:00:15.757457+0000","last_unstale":"2023-10-18T09:00:16.736651+0000","last_undegraded":"2023-10-18T09:00:16.736651+0000","last_fullsized":"2023-10-18T09:00:16.736651+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:44:11.773265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"86'8","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.776052+0000","last_change":"2023-10-18T09:00:14.724785+0000","last_active":"2023-10-18T09:00:15.776052+0000","last_peered":"2023-10-18T09:00:15.776052+0000","last_clean":"2023-10-18T09:00:15.776052+0000","last_became_active":"2023-10-18T09:00:14.724371+0000","last_became_peered":"2023-10-18T09:00:14.724371+0000","last_unstale":"2023-10-18T09:00:15.776052+0000","last_undegraded":"2023-10-18T09:00:15.776052+0000","last_fullsized":"2023-10-18T09:00:15.776052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:46.341496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.868082+0000","last_change":"2023-10-18T09:00:15.751116+0000","last_active":"2023-10-18T09:00:16.868082+0000","last_peered":"2023-10-18T09:00:16.868082+0000","last_clean":"2023-10-18T09:00:16.868082+0000","last_became_active":"2023-10-18T09:00:15.750596+0000","last_became_peered":"2023-10-18T09:00:15.750596+0000","last_unstale":"2023-10-18T09:00:16.868082+0000","last_undegraded":"2023-10-18T09:00:16.868082+0000","last_fullsized":"2023-10-18T09:00:16.868082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:22.137803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.772684+0000","last_change":"2023-10-18T09:00:14.728865+0000","last_active":"2023-10-18T09:00:15.772684+0000","last_peered":"2023-10-18T09:00:15.772684+0000","last_clean":"2023-10-18T09:00:15.772684+0000","last_became_active":"2023-10-18T09:00:14.728369+0000","last_became_peered":"2023-10-18T09:00:14.728369+0000","last_unstale":"2023-10-18T09:00:15.772684+0000","last_undegraded":"2023-10-18T09:00:15.772684+0000","last_fullsized":"2023-10-18T09:00:15.772684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:51:43.253647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737387+0000","last_change":"2023-10-18T09:00:15.762625+0000","last_active":"2023-10-18T09:00:16.737387+0000","last_peered":"2023-10-18T09:00:16.737387+0000","last_clean":"2023-10-18T09:00:16.737387+0000","last_became_active":"2023-10-18T09:00:15.761420+0000","last_became_peered":"2023-10-18T09:00:15.761420+0000","last_unstale":"2023-10-18T09:00:16.737387+0000","last_undegraded":"2023-10-18T09:00:16.737387+0000","last_fullsized":"2023-10-18T09:00:16.737387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:14:41.715749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773990+0000","last_change":"2023-10-18T09:00:14.722184+0000","last_active":"2023-10-18T09:00:15.773990+0000","last_peered":"2023-10-18T09:00:15.773990+0000","last_clean":"2023-10-18T09:00:15.773990+0000","last_became_active":"2023-10-18T09:00:14.721073+0000","last_became_peered":"2023-10-18T09:00:14.721073+0000","last_unstale":"2023-10-18T09:00:15.773990+0000","last_undegraded":"2023-10-18T09:00:15.773990+0000","last_fullsized":"2023-10-18T09:00:15.773990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:35:27.318871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867237+0000","last_change":"2023-10-18T09:00:15.758090+0000","last_active":"2023-10-18T09:00:16.867237+0000","last_peered":"2023-10-18T09:00:16.867237+0000","last_clean":"2023-10-18T09:00:16.867237+0000","last_became_active":"2023-10-18T09:00:15.757750+0000","last_became_peered":"2023-10-18T09:00:15.757750+0000","last_unstale":"2023-10-18T09:00:16.867237+0000","last_undegraded":"2023-10-18T09:00:16.867237+0000","last_fullsized":"2023-10-18T09:00:16.867237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:06:54.766783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.772911+0000","last_change":"2023-10-18T09:00:14.729164+0000","last_active":"2023-10-18T09:00:15.772911+0000","last_peered":"2023-10-18T09:00:15.772911+0000","last_clean":"2023-10-18T09:00:15.772911+0000","last_became_active":"2023-10-18T09:00:14.728269+0000","last_became_peered":"2023-10-18T09:00:14.728269+0000","last_unstale":"2023-10-18T09:00:15.772911+0000","last_undegraded":"2023-10-18T09:00:15.772911+0000","last_fullsized":"2023-10-18T09:00:15.772911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:56:27.534647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736706+0000","last_change":"2023-10-18T09:00:15.761518+0000","last_active":"2023-10-18T09:00:16.736706+0000","last_peered":"2023-10-18T09:00:16.736706+0000","last_clean":"2023-10-18T09:00:16.736706+0000","last_became_active":"2023-10-18T09:00:15.760762+0000","last_became_peered":"2023-10-18T09:00:15.760762+0000","last_unstale":"2023-10-18T09:00:16.736706+0000","last_undegraded":"2023-10-18T09:00:16.736706+0000","last_fullsized":"2023-10-18T09:00:16.736706+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:02:37.770586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.730935+0000","last_change":"2023-10-18T09:00:14.723114+0000","last_active":"2023-10-18T09:00:15.730935+0000","last_peered":"2023-10-18T09:00:15.730935+0000","last_clean":"2023-10-18T09:00:15.730935+0000","last_became_active":"2023-10-18T09:00:14.722764+0000","last_became_peered":"2023-10-18T09:00:14.722764+0000","last_unstale":"2023-10-18T09:00:15.730935+0000","last_undegraded":"2023-10-18T09:00:15.730935+0000","last_fullsized":"2023-10-18T09:00:15.730935+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:24:00.853217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.735158+0000","last_change":"2023-10-18T09:00:15.755724+0000","last_active":"2023-10-18T09:00:16.735158+0000","last_peered":"2023-10-18T09:00:16.735158+0000","last_clean":"2023-10-18T09:00:16.735158+0000","last_became_active":"2023-10-18T09:00:15.755472+0000","last_became_peered":"2023-10-18T09:00:15.755472+0000","last_unstale":"2023-10-18T09:00:16.735158+0000","last_undegraded":"2023-10-18T09:00:16.735158+0000","last_fullsized":"2023-10-18T09:00:16.735158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:22:41.062843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.758241+0000","last_change":"2023-10-18T09:00:14.727319+0000","last_active":"2023-10-18T09:00:15.758241+0000","last_peered":"2023-10-18T09:00:15.758241+0000","last_clean":"2023-10-18T09:00:15.758241+0000","last_became_active":"2023-10-18T09:00:14.726864+0000","last_became_peered":"2023-10-18T09:00:14.726864+0000","last_unstale":"2023-10-18T09:00:15.758241+0000","last_undegraded":"2023-10-18T09:00:15.758241+0000","last_fullsized":"2023-10-18T09:00:15.758241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:15.393555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.734848+0000","last_change":"2023-10-18T09:00:15.756743+0000","last_active":"2023-10-18T09:00:16.734848+0000","last_peered":"2023-10-18T09:00:16.734848+0000","last_clean":"2023-10-18T09:00:16.734848+0000","last_became_active":"2023-10-18T09:00:15.756462+0000","last_became_peered":"2023-10-18T09:00:15.756462+0000","last_unstale":"2023-10-18T09:00:16.734848+0000","last_undegraded":"2023-10-18T09:00:16.734848+0000","last_fullsized":"2023-10-18T09:00:16.734848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:14.505133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.752571+0000","last_change":"2023-10-18T09:00:14.723338+0000","last_active":"2023-10-18T09:00:15.752571+0000","last_peered":"2023-10-18T09:00:15.752571+0000","last_clean":"2023-10-18T09:00:15.752571+0000","last_became_active":"2023-10-18T09:00:14.723084+0000","last_became_peered":"2023-10-18T09:00:14.723084+0000","last_unstale":"2023-10-18T09:00:15.752571+0000","last_undegraded":"2023-10-18T09:00:15.752571+0000","last_fullsized":"2023-10-18T09:00:15.752571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:32:56.948078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.734768+0000","last_change":"2023-10-18T09:00:15.758267+0000","last_active":"2023-10-18T09:00:16.734768+0000","last_peered":"2023-10-18T09:00:16.734768+0000","last_clean":"2023-10-18T09:00:16.734768+0000","last_became_active":"2023-10-18T09:00:15.758033+0000","last_became_peered":"2023-10-18T09:00:15.758033+0000","last_unstale":"2023-10-18T09:00:16.734768+0000","last_undegraded":"2023-10-18T09:00:16.734768+0000","last_fullsized":"2023-10-18T09:00:16.734768+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:04:50.114527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.754653+0000","last_change":"2023-10-18T09:00:14.717552+0000","last_active":"2023-10-18T09:00:15.754653+0000","last_peered":"2023-10-18T09:00:15.754653+0000","last_clean":"2023-10-18T09:00:15.754653+0000","last_became_active":"2023-10-18T09:00:14.717311+0000","last_became_peered":"2023-10-18T09:00:14.717311+0000","last_unstale":"2023-10-18T09:00:15.754653+0000","last_undegraded":"2023-10-18T09:00:15.754653+0000","last_fullsized":"2023-10-18T09:00:15.754653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:27:59.639966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866525+0000","last_change":"2023-10-18T09:00:15.766855+0000","last_active":"2023-10-18T09:00:16.866525+0000","last_peered":"2023-10-18T09:00:16.866525+0000","last_clean":"2023-10-18T09:00:16.866525+0000","last_became_active":"2023-10-18T09:00:15.763490+0000","last_became_peered":"2023-10-18T09:00:15.763490+0000","last_unstale":"2023-10-18T09:00:16.866525+0000","last_undegraded":"2023-10-18T09:00:16.866525+0000","last_fullsized":"2023-10-18T09:00:16.866525+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:16:52.626915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.772821+0000","last_change":"2023-10-18T09:00:14.725676+0000","last_active":"2023-10-18T09:00:15.772821+0000","last_peered":"2023-10-18T09:00:15.772821+0000","last_clean":"2023-10-18T09:00:15.772821+0000","last_became_active":"2023-10-18T09:00:14.725396+0000","last_became_peered":"2023-10-18T09:00:14.725396+0000","last_unstale":"2023-10-18T09:00:15.772821+0000","last_undegraded":"2023-10-18T09:00:15.772821+0000","last_fullsized":"2023-10-18T09:00:15.772821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:00.426054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.871368+0000","last_change":"2023-10-18T09:00:15.754403+0000","last_active":"2023-10-18T09:00:16.871368+0000","last_peered":"2023-10-18T09:00:16.871368+0000","last_clean":"2023-10-18T09:00:16.871368+0000","last_became_active":"2023-10-18T09:00:15.753953+0000","last_became_peered":"2023-10-18T09:00:15.753953+0000","last_unstale":"2023-10-18T09:00:16.871368+0000","last_undegraded":"2023-10-18T09:00:16.871368+0000","last_fullsized":"2023-10-18T09:00:16.871368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:22.851038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"87'1","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.870154+0000","last_change":"2023-10-18T09:00:14.719262+0000","last_active":"2023-10-18T09:00:16.870154+0000","last_peered":"2023-10-18T09:00:16.870154+0000","last_clean":"2023-10-18T09:00:16.870154+0000","last_became_active":"2023-10-18T09:00:14.718856+0000","last_became_peered":"2023-10-18T09:00:14.718856+0000","last_unstale":"2023-10-18T09:00:16.870154+0000","last_undegraded":"2023-10-18T09:00:16.870154+0000","last_fullsized":"2023-10-18T09:00:16.870154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:59:59.045131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.738154+0000","last_change":"2023-10-18T09:00:15.758945+0000","last_active":"2023-10-18T09:00:16.738154+0000","last_peered":"2023-10-18T09:00:16.738154+0000","last_clean":"2023-10-18T09:00:16.738154+0000","last_became_active":"2023-10-18T09:00:15.757454+0000","last_became_peered":"2023-10-18T09:00:15.757454+0000","last_unstale":"2023-10-18T09:00:16.738154+0000","last_undegraded":"2023-10-18T09:00:16.738154+0000","last_fullsized":"2023-10-18T09:00:16.738154+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:28.179544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.744492+0000","last_change":"2023-10-18T09:00:14.719287+0000","last_active":"2023-10-18T09:00:15.744492+0000","last_peered":"2023-10-18T09:00:15.744492+0000","last_clean":"2023-10-18T09:00:15.744492+0000","last_became_active":"2023-10-18T09:00:14.719023+0000","last_became_peered":"2023-10-18T09:00:14.719023+0000","last_unstale":"2023-10-18T09:00:15.744492+0000","last_undegraded":"2023-10-18T09:00:15.744492+0000","last_fullsized":"2023-10-18T09:00:15.744492+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:08.548927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.737731+0000","last_change":"2023-10-18T09:00:15.748118+0000","last_active":"2023-10-18T09:00:16.737731+0000","last_peered":"2023-10-18T09:00:16.737731+0000","last_clean":"2023-10-18T09:00:16.737731+0000","last_became_active":"2023-10-18T09:00:15.747642+0000","last_became_peered":"2023-10-18T09:00:15.747642+0000","last_unstale":"2023-10-18T09:00:16.737731+0000","last_undegraded":"2023-10-18T09:00:16.737731+0000","last_fullsized":"2023-10-18T09:00:16.737731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:44.909930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.755422+0000","last_change":"2023-10-18T09:00:14.727210+0000","last_active":"2023-10-18T09:00:15.755422+0000","last_peered":"2023-10-18T09:00:15.755422+0000","last_clean":"2023-10-18T09:00:15.755422+0000","last_became_active":"2023-10-18T09:00:14.727034+0000","last_became_peered":"2023-10-18T09:00:14.727034+0000","last_unstale":"2023-10-18T09:00:15.755422+0000","last_undegraded":"2023-10-18T09:00:15.755422+0000","last_fullsized":"2023-10-18T09:00:15.755422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:14:40.702864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.736824+0000","last_change":"2023-10-18T09:00:15.746384+0000","last_active":"2023-10-18T09:00:16.736824+0000","last_peered":"2023-10-18T09:00:16.736824+0000","last_clean":"2023-10-18T09:00:16.736824+0000","last_became_active":"2023-10-18T09:00:15.745760+0000","last_became_peered":"2023-10-18T09:00:15.745760+0000","last_unstale":"2023-10-18T09:00:16.736824+0000","last_undegraded":"2023-10-18T09:00:16.736824+0000","last_fullsized":"2023-10-18T09:00:16.736824+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:05:46.945517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.753907+0000","last_change":"2023-10-18T09:00:14.723490+0000","last_active":"2023-10-18T09:00:15.753907+0000","last_peered":"2023-10-18T09:00:15.753907+0000","last_clean":"2023-10-18T09:00:15.753907+0000","last_became_active":"2023-10-18T09:00:14.723220+0000","last_became_peered":"2023-10-18T09:00:14.723220+0000","last_unstale":"2023-10-18T09:00:15.753907+0000","last_undegraded":"2023-10-18T09:00:15.753907+0000","last_fullsized":"2023-10-18T09:00:15.753907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:45:03.178214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.870157+0000","last_change":"2023-10-18T09:00:15.753547+0000","last_active":"2023-10-18T09:00:16.870157+0000","last_peered":"2023-10-18T09:00:16.870157+0000","last_clean":"2023-10-18T09:00:16.870157+0000","last_became_active":"2023-10-18T09:00:15.751658+0000","last_became_peered":"2023-10-18T09:00:15.751658+0000","last_unstale":"2023-10-18T09:00:16.870157+0000","last_undegraded":"2023-10-18T09:00:16.870157+0000","last_fullsized":"2023-10-18T09:00:16.870157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:36:47.354382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.752338+0000","last_change":"2023-10-18T09:00:14.716730+0000","last_active":"2023-10-18T09:00:15.752338+0000","last_peered":"2023-10-18T09:00:15.752338+0000","last_clean":"2023-10-18T09:00:15.752338+0000","last_became_active":"2023-10-18T09:00:14.716479+0000","last_became_peered":"2023-10-18T09:00:14.716479+0000","last_unstale":"2023-10-18T09:00:15.752338+0000","last_undegraded":"2023-10-18T09:00:15.752338+0000","last_fullsized":"2023-10-18T09:00:15.752338+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:42.805546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.866334+0000","last_change":"2023-10-18T09:00:15.760295+0000","last_active":"2023-10-18T09:00:16.866334+0000","last_peered":"2023-10-18T09:00:16.866334+0000","last_clean":"2023-10-18T09:00:16.866334+0000","last_became_active":"2023-10-18T09:00:15.760058+0000","last_became_peered":"2023-10-18T09:00:15.760058+0000","last_unstale":"2023-10-18T09:00:16.866334+0000","last_undegraded":"2023-10-18T09:00:16.866334+0000","last_fullsized":"2023-10-18T09:00:16.866334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:01.438940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.728164+0000","last_change":"2023-10-18T09:00:14.724630+0000","last_active":"2023-10-18T09:00:15.728164+0000","last_peered":"2023-10-18T09:00:15.728164+0000","last_clean":"2023-10-18T09:00:15.728164+0000","last_became_active":"2023-10-18T09:00:14.724028+0000","last_became_peered":"2023-10-18T09:00:14.724028+0000","last_unstale":"2023-10-18T09:00:15.728164+0000","last_undegraded":"2023-10-18T09:00:15.728164+0000","last_fullsized":"2023-10-18T09:00:15.728164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:53:10.671218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.867111+0000","last_change":"2023-10-18T09:00:15.766128+0000","last_active":"2023-10-18T09:00:16.867111+0000","last_peered":"2023-10-18T09:00:16.867111+0000","last_clean":"2023-10-18T09:00:16.867111+0000","last_became_active":"2023-10-18T09:00:15.765583+0000","last_became_peered":"2023-10-18T09:00:15.765583+0000","last_unstale":"2023-10-18T09:00:16.867111+0000","last_undegraded":"2023-10-18T09:00:16.867111+0000","last_fullsized":"2023-10-18T09:00:16.867111+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:26:45.228100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.758442+0000","last_change":"2023-10-18T09:00:14.727160+0000","last_active":"2023-10-18T09:00:15.758442+0000","last_peered":"2023-10-18T09:00:15.758442+0000","last_clean":"2023-10-18T09:00:15.758442+0000","last_became_active":"2023-10-18T09:00:14.726471+0000","last_became_peered":"2023-10-18T09:00:14.726471+0000","last_unstale":"2023-10-18T09:00:15.758442+0000","last_undegraded":"2023-10-18T09:00:15.758442+0000","last_fullsized":"2023-10-18T09:00:15.758442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:11:15.412520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:00:16.735001+0000","last_change":"2023-10-18T09:00:15.765588+0000","last_active":"2023-10-18T09:00:16.735001+0000","last_peered":"2023-10-18T09:00:16.735001+0000","last_clean":"2023-10-18T09:00:16.735001+0000","last_became_active":"2023-10-18T09:00:15.765331+0000","last_became_peered":"2023-10-18T09:00:15.765331+0000","last_unstale":"2023-10-18T09:00:16.735001+0000","last_undegraded":"2023-10-18T09:00:16.735001+0000","last_fullsized":"2023-10-18T09:00:16.735001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:14.691350+0000","last_clean_scrub_stamp":"2023-10-18T09:00:14.691350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:44:03.498540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.773229+0000","last_change":"2023-10-18T09:00:14.726818+0000","last_active":"2023-10-18T09:00:15.773229+0000","last_peered":"2023-10-18T09:00:15.773229+0000","last_clean":"2023-10-18T09:00:15.773229+0000","last_became_active":"2023-10-18T09:00:14.726391+0000","last_became_peered":"2023-10-18T09:00:14.726391+0000","last_unstale":"2023-10-18T09:00:15.773229+0000","last_undegraded":"2023-10-18T09:00:15.773229+0000","last_fullsized":"2023-10-18T09:00:15.773229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:00:13.678464+0000","last_clean_scrub_stamp":"2023-10-18T09:00:13.678464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:05:08.365526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":238,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-18T09:00:15.734462+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:00:15.734462+0000","last_peered":"2023-10-18T09:00:15.734462+0000","last_clean":"2023-10-18T09:00:15.734462+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:00:15.734462+0000","last_undegraded":"2023-10-18T09:00:15.734462+0000","last_fullsized":"2023-10-18T09:00:15.734462+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:32:36.150056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1032,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194280,"statfs":{"total":200000000,"available":198943213,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.071,"15min":1.0149999999999999},"min":{"1min":0.72599999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.575,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.165,"5min":1.1000000000000001,"15min":1.03},"min":{"1min":0.84399999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.536,"5min":1.97,"15min":1.97},"last":1.5740000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.125,"15min":1.147},"min":{"1min":0.88200000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5169999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1659999999999999,"15min":1.1299999999999999},"min":{"1min":0.91800000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.083}]},{"osd":2,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.284,"15min":1.246},"min":{"1min":0.78600000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.522,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.524},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.236,"15min":1.2030000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.637,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.202}]},{"osd":3,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.2609999999999999,"15min":1.2629999999999999},"min":{"1min":0.80300000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.845,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3500000000000001,"15min":1.327},"min":{"1min":1.0069999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.736,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.389}]},{"osd":4,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.238,"15min":1.1759999999999999},"min":{"1min":0.94599999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.825,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.393,"5min":1.351,"15min":1.3919999999999999},"min":{"1min":1.101,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.627,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.131}]},{"osd":5,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3580000000000001,"15min":1.355},"min":{"1min":0.89600000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7470000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.415,"15min":1.421},"min":{"1min":1.079,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7130000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.45}]},{"osd":6,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.482,"15min":1.464},"min":{"1min":1.355,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.8169999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.306},{"interface":"front","average":{"1min":1.512,"5min":1.484,"15min":1.5069999999999999},"min":{"1min":1.214,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.768,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.421}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":454,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194857,"statfs":{"total":200000000,"available":199534105,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.1759999999999999,"15min":1.105},"min":{"1min":0.71799999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.627,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.2709999999999999,"15min":1.248},"min":{"1min":0.86699999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7629999999999999,"5min":1.855,"15min":1.855},"last":1.494}]},{"osd":1,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1040000000000001,"15min":1.0900000000000001},"min":{"1min":0.68500000000000005,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.488,"5min":1.633,"15min":1.633},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.198,"15min":1.1919999999999999},"min":{"1min":0.72599999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.5409999999999999,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":1.2130000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.879,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5720000000000001,"5min":1.696,"15min":1.696},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2170000000000001,"15min":1.2010000000000001},"min":{"1min":0.90000000000000002,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.667,"5min":2.137,"15min":2.137},"last":1.3440000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.256,"15min":1.2589999999999999},"min":{"1min":0.91900000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.6719999999999999,"5min":1.784,"15min":1.784},"last":1.391},{"interface":"front","average":{"1min":1.274,"5min":1.248,"15min":1.226},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.6479999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.3029999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.2170000000000001,"15min":1.177},"min":{"1min":0.748,"5min":0.625,"15min":0.625},"max":{"1min":1.389,"5min":2.048,"15min":2.048},"last":1.109},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.262,"15min":1.2669999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.524}]},{"osd":5,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3819999999999999,"15min":1.3580000000000001},"min":{"1min":0.625,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2589999999999999,"15min":1.2250000000000001},"min":{"1min":0.71899999999999997,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.1599999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:00:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.391,"15min":1.3919999999999999},"min":{"1min":0.93500000000000005,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5900000000000001,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.4910000000000001,"15min":1.5189999999999999},"min":{"1min":0.83099999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.944,"5min":2.794,"15min":2.794},"last":1.4730000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":454,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194857,"statfs":{"total":200000000,"available":199534107,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.345,"15min":1.2330000000000001},"min":{"1min":0.71999999999999997,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.2480000000000002,"5min":9.3539999999999992,"15min":9.3539999999999992},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3260000000000001,"15min":1.304},"min":{"1min":0.77000000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.8260000000000001,"5min":5.3029999999999999,"15min":5.3029999999999999},"last":1.371}]},{"osd":1,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.194,"15min":1.218},"min":{"1min":0.85099999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.1499999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":1.202},{"interface":"front","average":{"1min":1.272,"5min":1.1910000000000001,"15min":1.2370000000000001},"min":{"1min":0.78500000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.2410000000000001,"5min":3.7050000000000001,"15min":3.7050000000000001},"last":1.4790000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2450000000000001,"15min":1.26},"min":{"1min":0.88900000000000001,"5min":0.755,"15min":0.755},"max":{"1min":2.0350000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.087},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.2629999999999999,"15min":1.284},"min":{"1min":0.95499999999999996,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.153,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.91000000000000003}]},{"osd":3,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2230000000000001,"15min":1.2370000000000001},"min":{"1min":0.86899999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.613,"5min":4.7119999999999997,"15min":4.7119999999999997},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3440000000000001,"15min":1.3340000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.5249999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.393,"15min":1.4119999999999999},"min":{"1min":0.93899999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.4449999999999998,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3919999999999999,"15min":1.498},"min":{"1min":0.91700000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.4049999999999998,"5min":5.4580000000000002,"15min":5.4580000000000002},"last":1.226}]},{"osd":6,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2989999999999999,"15min":1.3859999999999999},"min":{"1min":0.91900000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.72,"5min":5.141,"15min":5.141},"last":1.274},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.508,"15min":1.571},"min":{"1min":0.95299999999999996,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":1.423}]},{"osd":7,"last update":"Wed Oct 18 09:00:10 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.55,"15min":1.6399999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.5819999999999999,"5min":5.5490000000000004,"15min":5.5490000000000004},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.4610000000000001,"15min":1.5069999999999999},"min":{"1min":1.091,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.8240000000000001,"5min":4.8419999999999996,"15min":4.8419999999999996},"last":1.456}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":453,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194858,"statfs":{"total":200000000,"available":199535146,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.80900000000000005,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7669999999999999,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.125,"15min":1.1679999999999999},"min":{"1min":0.71199999999999997,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.7749999999999999,"5min":2.2599999999999998,"15min":2.2599999999999998},"last":1.385}]},{"osd":1,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.089,"15min":1.0920000000000001},"min":{"1min":0.749,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.4990000000000001,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":1.214},{"interface":"front","average":{"1min":1.23,"5min":1.1930000000000001,"15min":1.2290000000000001},"min":{"1min":0.70399999999999996,"5min":0.627,"15min":0.627},"max":{"1min":1.3740000000000001,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.109}]},{"osd":2,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1890000000000001,"15min":1.179},"min":{"1min":0.81999999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4350000000000001,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.383,"5min":1.2709999999999999,"15min":1.3029999999999999},"min":{"1min":0.85899999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.6930000000000001,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.468}]},{"osd":3,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.2869999999999999,"15min":1.3009999999999999},"min":{"1min":0.89700000000000002,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.9179999999999999,"5min":2.649,"15min":2.649},"last":1.643},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.403,"15min":1.409},"min":{"1min":0.86799999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6399999999999999,"5min":2.8010000000000002,"15min":2.8010000000000002},"last":1.4330000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.319,"15min":1.3029999999999999},"min":{"1min":0.96399999999999997,"5min":0.498,"15min":0.498},"max":{"1min":1.7649999999999999,"5min":1.923,"15min":1.923},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.3180000000000001,"15min":1.3560000000000001},"min":{"1min":0.92100000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.0720000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.5009999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4379999999999999,"15min":1.3959999999999999},"min":{"1min":0.96999999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.1150000000000002,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.329},{"interface":"front","average":{"1min":1.454,"5min":1.508,"15min":1.5580000000000001},"min":{"1min":1.2250000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6839999999999999,"5min":4.407,"15min":4.407},"last":1.6000000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:00:13 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.46,"15min":1.502},"min":{"1min":1.0169999999999999,"5min":0.438,"15min":0.438},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.54,"15min":1.546},"min":{"1min":1.196,"5min":0.747,"15min":0.747},"max":{"1min":1.7270000000000001,"5min":2.585,"15min":2.585},"last":1.5549999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":454,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194857,"statfs":{"total":200000000,"available":199534085,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.90600000000000003,"15min":0.79200000000000004},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2969999999999999,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.012,"15min":0.89300000000000002},"min":{"1min":0.48999999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.524,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":0.99099999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0309999999999999,"15min":0.95899999999999996},"min":{"1min":0.60099999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.633,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.2250000000000001,"15min":1.141},"min":{"1min":0.622,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.786,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.1379999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.079,"15min":1.0229999999999999},"min":{"1min":0.55200000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.675,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":0.755},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.163,"15min":1.073},"min":{"1min":0.59899999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.708,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":1.2569999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.5029999999999999,"15min":1.3180000000000001},"min":{"1min":0.91000000000000003,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.593,"5min":18.664999999999999,"15min":18.664999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.274,"15min":1.1719999999999999},"min":{"1min":0.93899999999999995,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.746,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":0.877}]},{"osd":5,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.3129999999999999,"15min":1.2290000000000001},"min":{"1min":1.014,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5700000000000001,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.5700000000000001,"15min":1.363},"min":{"1min":0.78300000000000003,"5min":0.748,"15min":0.748},"max":{"1min":1.9179999999999999,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.478}]},{"osd":6,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.5209999999999999,"15min":1.4350000000000001},"min":{"1min":1.115,"5min":0.625,"15min":0.625},"max":{"1min":1.7949999999999999,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4650000000000001,"15min":1.353},"min":{"1min":1.1339999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.772,"5min":7.242,"15min":7.242},"last":1.3959999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:00:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4870000000000001,"15min":1.361},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9950000000000001,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":1.379},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.6060000000000001,"15min":1.464},"min":{"1min":1.014,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0329999999999999,"5min":12.946,"15min":12.946},"last":1.4319999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":456,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194856,"statfs":{"total":200000000,"available":199532585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.87,"15min":0.92400000000000004},"min":{"1min":0.48599999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93400000000000005,"5min":4.2210000000000001,"15min":4.2210000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.96599999999999997,"15min":1.038},"min":{"1min":0.623,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3560000000000001,"5min":1.651,"15min":1.651},"last":0.83499999999999996}]},{"osd":1,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.996,"15min":1.0449999999999999},"min":{"1min":0.67500000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3120000000000001,"5min":4.29,"15min":4.29},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.1020000000000001,"15min":1.1040000000000001},"min":{"1min":0.75,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5820000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.145}]},{"osd":3,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.0940000000000001,"15min":1.129},"min":{"1min":0.72599999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8109999999999999,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.345,"15min":1.4650000000000001},"min":{"1min":0.73399999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.7929999999999999,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":1.2769999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.403,"15min":1.48},"min":{"1min":0.95299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.6479999999999999,"5min":7.6189999999999998,"15min":7.6189999999999998},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.407,"15min":1.4470000000000001},"min":{"1min":1.1319999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.766,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.615}]},{"osd":5,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.244,"15min":1.2709999999999999},"min":{"1min":1.119,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.786,"5min":4.4320000000000004,"15min":4.4320000000000004},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.446,"15min":1.5580000000000001},"min":{"1min":1.204,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.794,"5min":7.657,"15min":7.657},"last":1.4890000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.4430000000000001,"15min":1.5209999999999999},"min":{"1min":1.2070000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0649999999999999,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.51,"5min":1.8420000000000001,"15min":1.5780000000000001},"min":{"1min":1.0069999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.9350000000000001,"5min":37.771999999999998,"15min":37.771999999999998},"last":1.4430000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:00:09 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.506,"15min":1.5649999999999999},"min":{"1min":1.1890000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.1930000000000001,"5min":3.3620000000000001,"15min":3.3620000000000001},"last":1.524},{"interface":"front","average":{"1min":1.587,"5min":1.4670000000000001,"15min":1.48},"min":{"1min":1.056,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.1139999999999999,"5min":4.3390000000000004,"15min":4.3390000000000004},"last":1.5509999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":456,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194856,"statfs":{"total":200000000,"available":199532988,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.83599999999999997,"15min":0.81200000000000006},"min":{"1min":0.51000000000000001,"5min":0.315,"15min":0.315},"max":{"1min":1.343,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.004},{"interface":"front","average":{"1min":1.145,"5min":0.97999999999999998,"15min":0.92000000000000004},"min":{"1min":0.66600000000000004,"5min":0.443,"15min":0.443},"max":{"1min":1.399,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.1140000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":0.96399999999999997,"15min":0.93999999999999995},"min":{"1min":0.54900000000000004,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.456,"5min":1.859,"15min":1.859},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.028,"5min":0.90700000000000003,"15min":0.85999999999999999},"min":{"1min":0.70099999999999996,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.522,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.91200000000000003}]},{"osd":3,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.048,"15min":1.03},"min":{"1min":0.68500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.617,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.052,"15min":1.0569999999999999},"min":{"1min":0.82499999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5640000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":1.5249999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.2709999999999999,"15min":1.252},"min":{"1min":1.089,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.716,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.421},{"interface":"front","average":{"1min":1.379,"5min":1.2749999999999999,"15min":1.2190000000000001},"min":{"1min":0.99199999999999999,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.774,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.3049999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.3320000000000001,"15min":1.302},"min":{"1min":1.111,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.161,"5min":3.5750000000000002,"15min":3.5750000000000002},"last":1.677},{"interface":"front","average":{"1min":1.456,"5min":1.3979999999999999,"15min":1.3620000000000001},"min":{"1min":1.0820000000000001,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.7549999999999999,"5min":2.3769999999999998,"15min":2.3769999999999998},"last":1.6339999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.3360000000000001,"15min":1.2929999999999999},"min":{"1min":1.079,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.857,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.3380000000000001,"15min":1.2789999999999999},"min":{"1min":1.0980000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.75,"5min":2.121,"15min":2.121},"last":1.2589999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.419,"15min":1.367},"min":{"1min":1.175,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.992,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.359},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.423,"15min":1.3680000000000001},"min":{"1min":1.256,"5min":0.746,"15min":0.746},"max":{"1min":2.0569999999999999,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.5620000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1030,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194282,"statfs":{"total":200000000,"available":198944848,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84299999999999997,"15min":0.91900000000000004},"min":{"1min":0.65900000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.026,"5min":3.4420000000000002,"15min":3.4420000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":1.171,"5min":1.075,"15min":1.0820000000000001},"min":{"1min":0.70199999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.0660000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.86199999999999999,"15min":0.84999999999999998},"min":{"1min":0.77800000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.4159999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.143,"15min":1.131},"min":{"1min":0.63700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.73299999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.163,"15min":1.2869999999999999},"min":{"1min":0.69599999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.8200000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.494,"5min":1.4179999999999999,"15min":1.512},"min":{"1min":0.70099999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.3639999999999999,"5min":3.5720000000000001,"15min":3.5720000000000001},"last":0.84199999999999997}]},{"osd":4,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2330000000000001,"15min":1.3069999999999999},"min":{"1min":0.97999999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.8819999999999999,"5min":4.492,"15min":4.492},"last":1.117},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.2809999999999999,"15min":1.2569999999999999},"min":{"1min":1.0449999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.5960000000000001,"5min":1.726,"15min":1.726},"last":0.96499999999999997}]},{"osd":5,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.296,"15min":1.4219999999999999},"min":{"1min":0.85299999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":2.4510000000000001,"5min":4.2779999999999996,"15min":4.2779999999999996},"last":1.038},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.3380000000000001,"15min":1.379},"min":{"1min":0.98999999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.784,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.4450000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.4970000000000001,"15min":1.5880000000000001},"min":{"1min":1.2509999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":2.0419999999999998,"5min":3.5299999999999998,"15min":3.5299999999999998},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4139999999999999,"15min":1.5009999999999999},"min":{"1min":1.159,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.1059999999999999,"5min":4.4349999999999996,"15min":4.4349999999999996},"last":1.1559999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:00:16 2023","interfaces":[{"interface":"back","average":{"1min":1.631,"5min":1.569,"15min":1.645},"min":{"1min":1.2729999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.2000000000000002,"5min":3.492,"15min":3.492},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.536,"5min":1.512,"15min":1.6759999999999999},"min":{"1min":1.069,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.0009999999999999,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":1.1839999999999999}]}]}],"pool_statfs":[]}} 2023-10-18T09:00:25.897 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T09:00:25.898 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T09:00:26.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.171+0000 7fb092a40640 1 -- 172.21.15.40:0/700634232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08c071270 msgr2=0x7fb08c0c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.171+0000 7fb092a40640 1 --2- 172.21.15.40:0/700634232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08c071270 0x7fb08c0c9620 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb07c005fd0 tx=0x7fb07c02d0c0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.171+0000 7fb092a40640 1 -- 172.21.15.40:0/700634232 shutdown_connections 2023-10-18T09:00:26.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.171+0000 7fb092a40640 1 --2- 172.21.15.40:0/700634232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08c071270 0x7fb08c0c9620 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.171+0000 7fb092a40640 1 -- 172.21.15.40:0/700634232 >> 172.21.15.40:0/700634232 conn(0x7fb08c06ca10 msgr2=0x7fb08c06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.171+0000 7fb092a40640 1 -- 172.21.15.40:0/700634232 shutdown_connections 2023-10-18T09:00:26.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.171+0000 7fb092a40640 1 -- 172.21.15.40:0/700634232 wait complete. 2023-10-18T09:00:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.173+0000 7fb092a40640 1 Processor -- start 2023-10-18T09:00:26.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.173+0000 7fb092a40640 1 -- start start 2023-10-18T09:00:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.174+0000 7fb092a40640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb08c06d5d0 0x7fb08c14c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:26.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.174+0000 7fb092a40640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08c071270 0x7fb08c159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:26.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.174+0000 7fb092a40640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb08c15af70 0x7fb08c1599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:26.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.174+0000 7fb092a40640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c1f1ac0 con 0x7fb08c071270 2023-10-18T09:00:26.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.174+0000 7fb092a40640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c155c80 con 0x7fb08c06d5d0 2023-10-18T09:00:26.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.174+0000 7fb092a40640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c14d010 con 0x7fb08c15af70 2023-10-18T09:00:26.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.175+0000 7fb09123d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08c071270 0x7fb08c159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:26.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.175+0000 7fb09123d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08c071270 0x7fb08c159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40196/0 (socket says 172.21.15.40:40196) 2023-10-18T09:00:26.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.175+0000 7fb09123d640 1 -- 172.21.15.40:0/629347492 learned_addr learned my addr 172.21.15.40:0/629347492 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:26.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.175+0000 7fb09123d640 1 -- 172.21.15.40:0/629347492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb08c15af70 msgr2=0x7fb08c1599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:26.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.175+0000 7fb09123d640 1 --2- 172.21.15.40:0/629347492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb08c15af70 0x7fb08c1599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.175+0000 7fb09123d640 1 -- 172.21.15.40:0/629347492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb08c06d5d0 msgr2=0x7fb08c14c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:26.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.175+0000 7fb09123d640 1 --2- 172.21.15.40:0/629347492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb08c06d5d0 0x7fb08c14c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.175+0000 7fb09123d640 1 -- 172.21.15.40:0/629347492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c00a670 con 0x7fb08c071270 2023-10-18T09:00:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.175+0000 7fb09123d640 1 --2- 172.21.15.40:0/629347492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08c071270 0x7fb08c159460 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00a960 tx=0x7fb07c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.176+0000 7fb0827fc640 1 -- 172.21.15.40:0/629347492 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c0044f0 con 0x7fb08c071270 2023-10-18T09:00:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.176+0000 7fb0827fc640 1 -- 172.21.15.40:0/629347492 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb07c0036e0 con 0x7fb08c071270 2023-10-18T09:00:26.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.176+0000 7fb0827fc640 1 -- 172.21.15.40:0/629347492 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c003a90 con 0x7fb08c071270 2023-10-18T09:00:26.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.176+0000 7fb092a40640 1 -- 172.21.15.40:0/629347492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08c159fd0 con 0x7fb08c071270 2023-10-18T09:00:26.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.176+0000 7fb092a40640 1 -- 172.21.15.40:0/629347492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08c155630 con 0x7fb08c071270 2023-10-18T09:00:26.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.177+0000 7fb092a40640 1 -- 172.21.15.40:0/629347492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08c066e00 con 0x7fb08c071270 2023-10-18T09:00:26.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.178+0000 7fb0827fc640 1 -- 172.21.15.40:0/629347492 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb07c003bf0 con 0x7fb08c071270 2023-10-18T09:00:26.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.179+0000 7fb0827fc640 1 --2- 172.21.15.40:0/629347492 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb07006ae40 0x7fb07006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:26.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.179+0000 7fb0827fc640 1 -- 172.21.15.40:0/629347492 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb07c0acab0 con 0x7fb08c071270 2023-10-18T09:00:26.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.180+0000 7fb091a3e640 1 --2- 172.21.15.40:0/629347492 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb07006ae40 0x7fb07006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:26.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.181+0000 7fb091a3e640 1 --2- 172.21.15.40:0/629347492 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb07006ae40 0x7fb07006d300 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb088003cc0 tx=0x7fb0880079b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:26.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.184+0000 7fb0827fc640 1 -- 172.21.15.40:0/629347492 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb07c077a40 con 0x7fb08c071270 2023-10-18T09:00:26.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.447+0000 7fb092a40640 1 -- 172.21.15.40:0/629347492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb08c0c96b0 con 0x7fb08c071270 2023-10-18T09:00:26.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.452+0000 7fb0827fc640 1 -- 172.21.15.40:0/629347492 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7fb07c07b060 con 0x7fb08c071270 2023-10-18T09:00:26.454 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2023-10-18T09:00:26.455 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w== 2023-10-18T09:00:26.455 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2023-10-18T09:00:26.455 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2023-10-18T09:00:26.455 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T09:00:26.456 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2023-10-18T09:00:26.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.454+0000 7fb092a40640 1 -- 172.21.15.40:0/629347492 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb07006ae40 msgr2=0x7fb07006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:26.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.454+0000 7fb092a40640 1 --2- 172.21.15.40:0/629347492 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb07006ae40 0x7fb07006d300 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb088003cc0 tx=0x7fb0880079b0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.454+0000 7fb092a40640 1 -- 172.21.15.40:0/629347492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08c071270 msgr2=0x7fb08c159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:26.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.454+0000 7fb092a40640 1 --2- 172.21.15.40:0/629347492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08c071270 0x7fb08c159460 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00a960 tx=0x7fb07c0049d0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.455+0000 7fb092a40640 1 -- 172.21.15.40:0/629347492 shutdown_connections 2023-10-18T09:00:26.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.455+0000 7fb092a40640 1 --2- 172.21.15.40:0/629347492 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb07006ae40 0x7fb07006d300 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.455+0000 7fb092a40640 1 --2- 172.21.15.40:0/629347492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb08c15af70 0x7fb08c1599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.455+0000 7fb092a40640 1 --2- 172.21.15.40:0/629347492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb08c071270 0x7fb08c159460 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.455+0000 7fb092a40640 1 --2- 172.21.15.40:0/629347492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb08c06d5d0 0x7fb08c14c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.455+0000 7fb092a40640 1 -- 172.21.15.40:0/629347492 >> 172.21.15.40:0/629347492 conn(0x7fb08c06ca10 msgr2=0x7fb08c070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:26.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.455+0000 7fb092a40640 1 -- 172.21.15.40:0/629347492 shutdown_connections 2023-10-18T09:00:26.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.455+0000 7fb092a40640 1 -- 172.21.15.40:0/629347492 wait complete. 2023-10-18T09:00:26.476 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:00:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.706+0000 7ff1fb2a3640 1 -- 172.21.15.40:0/3308749263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f4159410 msgr2=0x7ff1f40b1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.706+0000 7ff1fb2a3640 1 --2- 172.21.15.40:0/3308749263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f4159410 0x7ff1f40b1610 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4070c50 tx=0x7ff1f002c110 comp rx=0 tx=0).stop 2023-10-18T09:00:26.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.706+0000 7ff1fb2a3640 1 -- 172.21.15.40:0/3308749263 shutdown_connections 2023-10-18T09:00:26.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.706+0000 7ff1fb2a3640 1 --2- 172.21.15.40:0/3308749263 >> v2:172.21.15.144:3300/0 conn(0x7ff1f40ca620 0x7ff1f40cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.706+0000 7ff1fb2a3640 1 --2- 172.21.15.40:0/3308749263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f4159410 0x7ff1f40b1610 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.706+0000 7ff1fb2a3640 1 -- 172.21.15.40:0/3308749263 >> 172.21.15.40:0/3308749263 conn(0x7ff1f40b0af0 msgr2=0x7ff1f40b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:26.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.706+0000 7ff1fb2a3640 1 -- 172.21.15.40:0/3308749263 shutdown_connections 2023-10-18T09:00:26.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.707+0000 7ff1fb2a3640 1 -- 172.21.15.40:0/3308749263 wait complete. 2023-10-18T09:00:26.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.709+0000 7ff1fb2a3640 1 Processor -- start 2023-10-18T09:00:26.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.709+0000 7ff1fb2a3640 1 -- start start 2023-10-18T09:00:26.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.710+0000 7ff1fb2a3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f40ca620 0x7ff1f406a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:26.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.710+0000 7ff1fb2a3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f40cdb20 0x7ff1f40b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:26.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.710+0000 7ff1fb2a3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1f4159410 0x7ff1f40991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:26.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.710+0000 7ff1fb2a3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f409b1e0 con 0x7ff1f40cdb20 2023-10-18T09:00:26.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.710+0000 7ff1fb2a3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f409b460 con 0x7ff1f40ca620 2023-10-18T09:00:26.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.710+0000 7ff1fb2a3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f409b740 con 0x7ff1f4159410 2023-10-18T09:00:26.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.710+0000 7ff1f9aa0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f40cdb20 0x7ff1f40b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:26.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.710+0000 7ff1f9aa0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f40cdb20 0x7ff1f40b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40210/0 (socket says 172.21.15.40:40210) 2023-10-18T09:00:26.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.710+0000 7ff1f9aa0640 1 -- 172.21.15.40:0/3478999087 learned_addr learned my addr 172.21.15.40:0/3478999087 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:26.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.711+0000 7ff1fa2a1640 1 --2- 172.21.15.40:0/3478999087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f40ca620 0x7ff1f406a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:26.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.711+0000 7ff1f9aa0640 1 -- 172.21.15.40:0/3478999087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1f4159410 msgr2=0x7ff1f40991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:26.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.711+0000 7ff1faaa2640 1 --2- 172.21.15.40:0/3478999087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1f4159410 0x7ff1f40991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:26.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.711+0000 7ff1f9aa0640 1 --2- 172.21.15.40:0/3478999087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1f4159410 0x7ff1f40991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.711+0000 7ff1f9aa0640 1 -- 172.21.15.40:0/3478999087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f40ca620 msgr2=0x7ff1f406a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:26.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.711+0000 7ff1f9aa0640 1 --2- 172.21.15.40:0/3478999087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f40ca620 0x7ff1f406a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.711+0000 7ff1f9aa0640 1 -- 172.21.15.40:0/3478999087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f0009840 con 0x7ff1f40cdb20 2023-10-18T09:00:26.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.711+0000 7ff1fa2a1640 1 --2- 172.21.15.40:0/3478999087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f40ca620 0x7ff1f406a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:26.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.711+0000 7ff1f9aa0640 1 --2- 172.21.15.40:0/3478999087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f40cdb20 0x7ff1f40b3450 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400d950 tx=0x7ff1e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:26.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.711+0000 7ff1eaffd640 1 -- 172.21.15.40:0/3478999087 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4009c30 con 0x7ff1f40cdb20 2023-10-18T09:00:26.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.711+0000 7ff1eaffd640 1 -- 172.21.15.40:0/3478999087 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff1e4009e80 con 0x7ff1f40cdb20 2023-10-18T09:00:26.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.711+0000 7ff1eaffd640 1 -- 172.21.15.40:0/3478999087 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40106c0 con 0x7ff1f40cdb20 2023-10-18T09:00:26.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.711+0000 7ff1fb2a3640 1 -- 172.21.15.40:0/3478999087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f409ba20 con 0x7ff1f40cdb20 2023-10-18T09:00:26.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.712+0000 7ff1fb2a3640 1 -- 172.21.15.40:0/3478999087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f40b2680 con 0x7ff1f40cdb20 2023-10-18T09:00:26.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.713+0000 7ff1fb2a3640 1 -- 172.21.15.40:0/3478999087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f40b28a0 con 0x7ff1f40cdb20 2023-10-18T09:00:26.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.714+0000 7ff1eaffd640 1 -- 172.21.15.40:0/3478999087 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff1e4010820 con 0x7ff1f40cdb20 2023-10-18T09:00:26.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.715+0000 7ff1eaffd640 1 --2- 172.21.15.40:0/3478999087 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1d806ae40 0x7ff1d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:26.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.715+0000 7ff1eaffd640 1 -- 172.21.15.40:0/3478999087 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1e4089ba0 con 0x7ff1f40cdb20 2023-10-18T09:00:26.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.716+0000 7ff1fa2a1640 1 --2- 172.21.15.40:0/3478999087 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1d806ae40 0x7ff1d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:26.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.717+0000 7ff1fa2a1640 1 --2- 172.21.15.40:0/3478999087 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1d806ae40 0x7ff1d806d300 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0035000 tx=0x7ff1f0004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:26.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.720+0000 7ff1eaffd640 1 -- 172.21.15.40:0/3478999087 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff1f40b28a0 con 0x7ff1f40cdb20 2023-10-18T09:00:26.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.950+0000 7ff1fb2a3640 1 -- 172.21.15.40:0/3478999087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1f40b28a0 con 0x7ff1f40cdb20 2023-10-18T09:00:26.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.952+0000 7ff1eaffd640 1 -- 172.21.15.40:0/3478999087 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4921 (secure 0 0 0) 0x7ff1f40b28a0 con 0x7ff1f40cdb20 2023-10-18T09:00:26.954 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:26.954 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1341351808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1341351808},{"type":"v1","addr":"172.21.15.144:6836","nonce":1341351808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1187670073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1187670073},{"type":"v1","addr":"172.21.15.40:6833","nonce":1187670073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5626,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1615448513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1615448513},{"type":"v1","addr":"172.21.15.144:6837","nonce":1615448513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:00:15.197720+0000","modified":"2023-10-18T09:00:18.306206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1926421699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1926421699},{"type":"v1","addr":"172.21.15.40:6835","nonce":1926421699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-18T09:00:26.955 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 41 2023-10-18T09:00:26.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.955+0000 7ff1fb2a3640 1 -- 172.21.15.40:0/3478999087 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1d806ae40 msgr2=0x7ff1d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:26.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.955+0000 7ff1fb2a3640 1 --2- 172.21.15.40:0/3478999087 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1d806ae40 0x7ff1d806d300 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0035000 tx=0x7ff1f0004ab0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.955+0000 7ff1fb2a3640 1 -- 172.21.15.40:0/3478999087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f40cdb20 msgr2=0x7ff1f40b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:26.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.955+0000 7ff1fb2a3640 1 --2- 172.21.15.40:0/3478999087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f40cdb20 0x7ff1f40b3450 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400d950 tx=0x7ff1e400de10 comp rx=0 tx=0).stop 2023-10-18T09:00:26.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.955+0000 7ff1fb2a3640 1 -- 172.21.15.40:0/3478999087 shutdown_connections 2023-10-18T09:00:26.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.955+0000 7ff1fb2a3640 1 --2- 172.21.15.40:0/3478999087 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1d806ae40 0x7ff1d806d300 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.955+0000 7ff1fb2a3640 1 --2- 172.21.15.40:0/3478999087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1f4159410 0x7ff1f40991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.955+0000 7ff1fb2a3640 1 --2- 172.21.15.40:0/3478999087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1f40cdb20 0x7ff1f40b3450 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.955+0000 7ff1fb2a3640 1 --2- 172.21.15.40:0/3478999087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1f40ca620 0x7ff1f406a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:26.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.955+0000 7ff1fb2a3640 1 -- 172.21.15.40:0/3478999087 >> 172.21.15.40:0/3478999087 conn(0x7ff1f40b0af0 msgr2=0x7ff1f406ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:26.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.955+0000 7ff1fb2a3640 1 -- 172.21.15.40:0/3478999087 shutdown_connections 2023-10-18T09:00:26.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:26.955+0000 7ff1fb2a3640 1 -- 172.21.15.40:0/3478999087 wait complete. 2023-10-18T09:00:27.381 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T09:00:15.197720+0000', 'modified': '2023-10-18T09:00:18.306206+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6573}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6573': {'gid': 6573, 'name': 'c', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/1926421699', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 1926421699}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 1926421699}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T09:00:27.382 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T09:00:27.382 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-18T09:00:27.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.624+0000 7f8bc9ac9640 1 -- 172.21.15.40:0/3341519504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bc40ae700 msgr2=0x7f8bc406dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:27.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.624+0000 7f8bc9ac9640 1 --2- 172.21.15.40:0/3341519504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bc40ae700 0x7f8bc406dd50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8bc406fcb0 tx=0x7f8bac02bd70 comp rx=0 tx=0).stop 2023-10-18T09:00:27.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.625+0000 7f8bc9ac9640 1 -- 172.21.15.40:0/3341519504 shutdown_connections 2023-10-18T09:00:27.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.625+0000 7f8bc9ac9640 1 --2- 172.21.15.40:0/3341519504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bc40ae700 0x7f8bc406dd50 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.625+0000 7f8bc9ac9640 1 --2- 172.21.15.40:0/3341519504 >> v2:172.21.15.144:3300/0 conn(0x7f8bc4142200 0x7f8bc40b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.625+0000 7f8bc9ac9640 1 -- 172.21.15.40:0/3341519504 >> 172.21.15.40:0/3341519504 conn(0x7f8bc40655f0 msgr2=0x7f8bc40659d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:27.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.625+0000 7f8bc9ac9640 1 -- 172.21.15.40:0/3341519504 shutdown_connections 2023-10-18T09:00:27.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.625+0000 7f8bc9ac9640 1 -- 172.21.15.40:0/3341519504 wait complete. 2023-10-18T09:00:27.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.626+0000 7f8bc9ac9640 1 Processor -- start 2023-10-18T09:00:27.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.627+0000 7f8bc9ac9640 1 -- start start 2023-10-18T09:00:27.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.628+0000 7f8bc9ac9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bc406e290 0x7f8bc415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.628+0000 7f8bc9ac9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40ae700 0x7f8bc4158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.628+0000 7f8bc9ac9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4142200 0x7f8bc413cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.628+0000 7f8bc9ac9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40ab530 con 0x7f8bc406e290 2023-10-18T09:00:27.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.628+0000 7f8bc9ac9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40aa920 con 0x7f8bc40ae700 2023-10-18T09:00:27.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.628+0000 7f8bc9ac9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40aac00 con 0x7f8bc4142200 2023-10-18T09:00:27.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.628+0000 7f8bc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bc406e290 0x7f8bc415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.628+0000 7f8bc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bc406e290 0x7f8bc415c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40236/0 (socket says 172.21.15.40:40236) 2023-10-18T09:00:27.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.628+0000 7f8bc37fe640 1 -- 172.21.15.40:0/2521342641 learned_addr learned my addr 172.21.15.40:0/2521342641 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:27.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.628+0000 7f8bbbfff640 1 --2- 172.21.15.40:0/2521342641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40ae700 0x7f8bc4158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.628+0000 7f8bc37fe640 1 -- 172.21.15.40:0/2521342641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4142200 msgr2=0x7f8bc413cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:27.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.628+0000 7f8bc3fff640 1 --2- 172.21.15.40:0/2521342641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4142200 0x7f8bc413cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.628+0000 7f8bc37fe640 1 --2- 172.21.15.40:0/2521342641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4142200 0x7f8bc413cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.629+0000 7f8bc37fe640 1 -- 172.21.15.40:0/2521342641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40ae700 msgr2=0x7f8bc4158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:27.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.629+0000 7f8bc37fe640 1 --2- 172.21.15.40:0/2521342641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40ae700 0x7f8bc4158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.629+0000 7f8bc37fe640 1 -- 172.21.15.40:0/2521342641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac009490 con 0x7f8bc406e290 2023-10-18T09:00:27.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.629+0000 7f8bbbfff640 1 --2- 172.21.15.40:0/2521342641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40ae700 0x7f8bc4158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:27.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.629+0000 7f8bc3fff640 1 --2- 172.21.15.40:0/2521342641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4142200 0x7f8bc413cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:27.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.629+0000 7f8bc37fe640 1 --2- 172.21.15.40:0/2521342641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bc406e290 0x7f8bc415c3f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400d950 tx=0x7f8bb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:27.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.629+0000 7f8bc0ff9640 1 -- 172.21.15.40:0/2521342641 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4009c30 con 0x7f8bc406e290 2023-10-18T09:00:27.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.629+0000 7f8bc0ff9640 1 -- 172.21.15.40:0/2521342641 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8bb4009e80 con 0x7f8bc406e290 2023-10-18T09:00:27.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.629+0000 7f8bc0ff9640 1 -- 172.21.15.40:0/2521342641 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40106c0 con 0x7f8bc406e290 2023-10-18T09:00:27.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.629+0000 7f8bc9ac9640 1 -- 172.21.15.40:0/2521342641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc40aaee0 con 0x7f8bc406e290 2023-10-18T09:00:27.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.629+0000 7f8bc9ac9640 1 -- 172.21.15.40:0/2521342641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc4159020 con 0x7f8bc406e290 2023-10-18T09:00:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.632+0000 7f8bc0ff9640 1 -- 172.21.15.40:0/2521342641 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8bb4010820 con 0x7f8bc406e290 2023-10-18T09:00:27.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.633+0000 7f8bc0ff9640 1 --2- 172.21.15.40:0/2521342641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ba406ae40 0x7f8ba406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.633+0000 7f8bc0ff9640 1 -- 172.21.15.40:0/2521342641 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bb4089ba0 con 0x7f8bc406e290 2023-10-18T09:00:27.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.633+0000 7f8bcc65c740 1 Processor -- start 2023-10-18T09:00:27.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.633+0000 7f8bbbfff640 1 --2- 172.21.15.40:0/2521342641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ba406ae40 0x7f8ba406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.633+0000 7f8bcc65c740 1 -- start start 2023-10-18T09:00:27.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.634+0000 7f8bcc65c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611392407d0 0x561139213a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.634+0000 7f8bcc65c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561139240bb0 0x561139243560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.634+0000 7f8bcc65c740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139213f40 0x561139243aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.634+0000 7f8bcc65c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x561139186740 con 0x561139240bb0 2023-10-18T09:00:27.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.634+0000 7f8bcc65c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561139183e30 con 0x5611392407d0 2023-10-18T09:00:27.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.634+0000 7f8bcc65c740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56113917f720 con 0x561139213f40 2023-10-18T09:00:27.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.635+0000 7f8bbbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561139240bb0 0x561139243560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.635+0000 7f8bbbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561139240bb0 0x561139243560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40238/0 (socket says 172.21.15.40:40238) 2023-10-18T09:00:27.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.635+0000 7f8bbbfff640 1 -- 172.21.15.40:0/28307656 learned_addr learned my addr 172.21.15.40:0/28307656 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:27.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.635+0000 7f8bc3fff640 1 --2- 172.21.15.40:0/28307656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139213f40 0x561139243aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.635+0000 7f8bc37fe640 1 --2- 172.21.15.40:0/28307656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611392407d0 0x561139213a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.635+0000 7f8bbbfff640 1 --2- 172.21.15.40:0/2521342641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ba406ae40 0x7f8ba406d300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8bac036000 tx=0x7f8bac004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:27.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.635+0000 7f8bbbfff640 1 -- 172.21.15.40:0/28307656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139213f40 msgr2=0x561139243aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:00:27.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.635+0000 7f8bbbfff640 1 --2- 172.21.15.40:0/28307656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139213f40 0x561139243aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.635+0000 7f8bbbfff640 1 -- 172.21.15.40:0/28307656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611392407d0 msgr2=0x561139213a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:00:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.635+0000 7f8bbbfff640 1 --2- 172.21.15.40:0/28307656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611392407d0 0x561139213a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.635+0000 7f8bbbfff640 1 -- 172.21.15.40:0/28307656 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561139143910 con 0x561139240bb0 2023-10-18T09:00:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.636+0000 7f8bbbfff640 1 --2- 172.21.15.40:0/28307656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561139240bb0 0x561139243560 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0098f0 tx=0x7f8bac03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6db16fa581a5133 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.636+0000 7f8bb97fa640 1 -- 172.21.15.40:0/28307656 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac0037f0 con 0x561139240bb0 2023-10-18T09:00:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.636+0000 7f8bb97fa640 1 -- 172.21.15.40:0/28307656 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8bac003a40 con 0x561139240bb0 2023-10-18T09:00:27.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.636+0000 7f8bb97fa640 1 -- 172.21.15.40:0/28307656 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac003db0 con 0x561139240bb0 2023-10-18T09:00:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.637+0000 7f8bcc65c740 1 -- 172.21.15.40:0/28307656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561139240bb0 msgr2=0x561139243560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.637+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/28307656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561139240bb0 0x561139243560 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0098f0 tx=0x7f8bac03d000 comp rx=0 tx=0).stop 2023-10-18T09:00:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.637+0000 7f8bcc65c740 1 -- 172.21.15.40:0/28307656 shutdown_connections 2023-10-18T09:00:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.637+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/28307656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139213f40 0x561139243aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.637+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/28307656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561139240bb0 0x561139243560 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.637+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/28307656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611392407d0 0x561139213a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.637+0000 7f8bcc65c740 1 -- 172.21.15.40:0/28307656 >> 172.21.15.40:0/28307656 conn(0x561139215c70 msgr2=0x56113925e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:27.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.637+0000 7f8bcc65c740 1 -- 172.21.15.40:0/28307656 shutdown_connections 2023-10-18T09:00:27.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.637+0000 7f8bcc65c740 1 -- 172.21.15.40:0/28307656 wait complete. 2023-10-18T09:00:27.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.638+0000 7f8bcc65c740 1 -- start start 2023-10-18T09:00:27.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.638+0000 7f8bcc65c740 1 Processor -- start 2023-10-18T09:00:27.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.639+0000 7f8bcc65c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5611392407d0 0x561139246c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.639+0000 7f8bcc65c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561139240bb0 0x561139258b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.639+0000 7f8bcc65c740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139271c00 0x561139272080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.639+0000 7f8bcc65c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56113911d970 con 0x5611392407d0 2023-10-18T09:00:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.639+0000 7f8bcc65c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x56113911d610 con 0x561139240bb0 2023-10-18T09:00:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.639+0000 7f8bcc65c740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x561138ee3f20 con 0x561139271c00 2023-10-18T09:00:27.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.639+0000 7f8bcc65c740 10 client.0 _ll_register_callbacks cb 0x5611391410c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:00:27.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.639+0000 7f8bc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5611392407d0 0x561139246c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.640+0000 7f8bc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5611392407d0 0x561139246c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40250/0 (socket says 172.21.15.40:40250) 2023-10-18T09:00:27.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.640+0000 7f8bc37fe640 1 -- 172.21.15.40:0/2192361400 learned_addr learned my addr 172.21.15.40:0/2192361400 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:27.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.640+0000 7f8bbbfff640 1 --2- 172.21.15.40:0/2192361400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561139240bb0 0x561139258b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.640+0000 7f8bc3fff640 1 --2- 172.21.15.40:0/2192361400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139271c00 0x561139272080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.640+0000 7f8bc37fe640 1 -- 172.21.15.40:0/2192361400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139271c00 msgr2=0x561139272080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:00:27.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.640+0000 7f8bc37fe640 1 --2- 172.21.15.40:0/2192361400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139271c00 0x561139272080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.640+0000 7f8bc37fe640 1 -- 172.21.15.40:0/2192361400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561139240bb0 msgr2=0x561139258b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:00:27.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.640+0000 7f8bc37fe640 1 --2- 172.21.15.40:0/2192361400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561139240bb0 0x561139258b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.640+0000 7f8bc37fe640 1 -- 172.21.15.40:0/2192361400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc40b3320 con 0x5611392407d0 2023-10-18T09:00:27.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.640+0000 7f8bc3fff640 1 --2- 172.21.15.40:0/2192361400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139271c00 0x561139272080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:27.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.640+0000 7f8bbbfff640 1 --2- 172.21.15.40:0/2192361400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561139240bb0 0x561139258b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:00:27.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.640+0000 7f8bc37fe640 1 --2- 172.21.15.40:0/2192361400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5611392407d0 0x561139246c00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4088030 tx=0x7f8bb405c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab97174dea9c63bf server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:27.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.640+0000 7f8ba27fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T09:00:27.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.640+0000 7f8ba27fc640 1 -- 172.21.15.40:0/2192361400 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40608e0 con 0x5611392407d0 2023-10-18T09:00:27.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.640+0000 7f8ba27fc640 1 -- 172.21.15.40:0/2192361400 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bb4056cd0 con 0x5611392407d0 2023-10-18T09:00:27.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.641+0000 7f8bcc65c740 1 -- 172.21.15.40:0/2192361400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561139178700 con 0x5611392407d0 2023-10-18T09:00:27.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.641+0000 7f8ba27fc640 1 -- 172.21.15.40:0/2192361400 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bb40701c0 con 0x5611392407d0 2023-10-18T09:00:27.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.641+0000 7f8bcc65c740 10 client.6741 fetch_fsmap learned FSMap version 41 2023-10-18T09:00:27.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.641+0000 7f8bcc65c740 1 -- 172.21.15.40:0/2192361400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x56113917dda0 con 0x5611392407d0 2023-10-18T09:00:27.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.641+0000 7f8ba27fc640 1 -- 172.21.15.40:0/2192361400 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8bb4075080 con 0x5611392407d0 2023-10-18T09:00:27.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.641+0000 7f8ba27fc640 1 -- 172.21.15.40:0/2192361400 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8bb4059ba0 con 0x5611392407d0 2023-10-18T09:00:27.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.641+0000 7f8bcc65c740 10 client.6741 fetch_fsmap finished waiting for FSMap version 41 2023-10-18T09:00:27.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.641+0000 7f8bcc65c740 10 client.6741 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T09:00:27.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.641+0000 7f8bcc65c740 20 client.6741 populate_metadata read hostname 'smithi040' 2023-10-18T09:00:27.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.641+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/2192361400 >> [v2:172.21.15.40:6834/1926421699,v1:172.21.15.40:6835/1926421699] conn(0x5611392461a0 0x56113927bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.641+0000 7f8bcc65c740 4 client.6741 mds_command: new command op to 6573 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:00:27.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.641+0000 7f8bcc65c740 1 -- 172.21.15.40:0/2192361400 --> [v2:172.21.15.40:6834/1926421699,v1:172.21.15.40:6835/1926421699] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561139178420 con 0x5611392461a0 2023-10-18T09:00:27.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.642+0000 7f8bbbfff640 1 --2- 172.21.15.40:0/2192361400 >> [v2:172.21.15.40:6834/1926421699,v1:172.21.15.40:6835/1926421699] conn(0x5611392461a0 0x56113927bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.642+0000 7f8bbbfff640 1 --2- 172.21.15.40:0/2192361400 >> [v2:172.21.15.40:6834/1926421699,v1:172.21.15.40:6835/1926421699] conn(0x5611392461a0 0x56113927bf10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=123e3ecca02120fa server_cookie=3e9342d4f1c5140e in_seq=0 out_seq=0 2023-10-18T09:00:27.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.642+0000 7f8ba27fc640 10 client.6741 ms_handle_connect on v2:172.21.15.40:6834/1926421699 2023-10-18T09:00:27.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.643+0000 7f8ba27fc640 1 -- 172.21.15.40:0/2192361400 <== mds.0 v2:172.21.15.40:6834/1926421699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561139178420 con 0x5611392461a0 2023-10-18T09:00:27.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.643+0000 7f8ba27fc640 10 client.6741 handle_command_reply: tid=0 2023-10-18T09:00:27.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.643+0000 7f8bcc65c740 1 client.6741 shutdown 2023-10-18T09:00:27.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.643+0000 7f8bcc65c740 1 -- 172.21.15.40:0/2192361400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5611392407d0 msgr2=0x561139246c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:27.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.643+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/2192361400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5611392407d0 0x561139246c00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4088030 tx=0x7f8bb405c9c0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.644+0000 7f8bcc65c740 1 -- 172.21.15.40:0/2192361400 shutdown_connections 2023-10-18T09:00:27.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.644+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/2192361400 >> [v2:172.21.15.40:6834/1926421699,v1:172.21.15.40:6835/1926421699] conn(0x5611392461a0 0x56113927bf10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.644+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/2192361400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139271c00 0x561139272080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.644+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/2192361400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561139240bb0 0x561139258b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.644+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/2192361400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5611392407d0 0x561139246c00 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.644+0000 7f8ba27fc640 0 client.6741 ms_handle_reset on v2:172.21.15.40:6834/1926421699 2023-10-18T09:00:27.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.644+0000 7f8bcc65c740 1 -- 172.21.15.40:0/2192361400 >> 172.21.15.40:0/2192361400 conn(0x561139213f40 msgr2=0x561139217680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:27.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.644+0000 7f8bcc65c740 1 -- 172.21.15.40:0/2192361400 shutdown_connections 2023-10-18T09:00:27.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.644+0000 7f8bcc65c740 1 -- 172.21.15.40:0/2192361400 wait complete. 2023-10-18T09:00:27.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.644+0000 7f8bcc65c740 20 client.6741 trim_cache size 0 max 16384 2023-10-18T09:00:27.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.661+0000 7f8bcc65c740 1 Processor -- start 2023-10-18T09:00:27.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.661+0000 7f8bcc65c740 1 -- start start 2023-10-18T09:00:27.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.662+0000 7f8bcc65c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611392407d0 0x561139280410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.662+0000 7f8bcc65c740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139240bb0 0x5611392439c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.662+0000 7f8bcc65c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561139271c00 0x561139253ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.662+0000 7f8bcc65c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56113926e380 con 0x561139271c00 2023-10-18T09:00:27.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.662+0000 7f8bcc65c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x561139258b20 con 0x5611392407d0 2023-10-18T09:00:27.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.662+0000 7f8bcc65c740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56113926e130 con 0x561139240bb0 2023-10-18T09:00:27.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.662+0000 7f8bc3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561139271c00 0x561139253ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.662+0000 7f8bc3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561139271c00 0x561139253ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40264/0 (socket says 172.21.15.40:40264) 2023-10-18T09:00:27.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.662+0000 7f8bc3fff640 1 -- 172.21.15.40:0/623784444 learned_addr learned my addr 172.21.15.40:0/623784444 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:27.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.663+0000 7f8bc37fe640 1 --2- 172.21.15.40:0/623784444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611392407d0 0x561139280410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.663+0000 7f8bc3fff640 1 -- 172.21.15.40:0/623784444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139240bb0 msgr2=0x5611392439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:00:27.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.663+0000 7f8bc3fff640 1 --2- 172.21.15.40:0/623784444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139240bb0 0x5611392439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.663+0000 7f8bc3fff640 1 -- 172.21.15.40:0/623784444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611392407d0 msgr2=0x561139280410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:00:27.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.663+0000 7f8bc3fff640 1 --2- 172.21.15.40:0/623784444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611392407d0 0x561139280410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.663+0000 7f8bc3fff640 1 -- 172.21.15.40:0/623784444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56113911cb80 con 0x561139271c00 2023-10-18T09:00:27.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.663+0000 7f8bc3fff640 1 --2- 172.21.15.40:0/623784444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561139271c00 0x561139253ef0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc001d70 tx=0x7f8bbc010240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86bd86e9ef07983 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:27.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.663+0000 7f8bb9ffb640 1 -- 172.21.15.40:0/623784444 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc010e50 con 0x561139271c00 2023-10-18T09:00:27.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.663+0000 7f8bb9ffb640 1 -- 172.21.15.40:0/623784444 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8bbc00d040 con 0x561139271c00 2023-10-18T09:00:27.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.663+0000 7f8bb9ffb640 1 -- 172.21.15.40:0/623784444 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc003bb0 con 0x561139271c00 2023-10-18T09:00:27.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.664+0000 7f8bcc65c740 1 -- 172.21.15.40:0/623784444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561139271c00 msgr2=0x561139253ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:27.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.664+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/623784444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561139271c00 0x561139253ef0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc001d70 tx=0x7f8bbc010240 comp rx=0 tx=0).stop 2023-10-18T09:00:27.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.664+0000 7f8bcc65c740 1 -- 172.21.15.40:0/623784444 shutdown_connections 2023-10-18T09:00:27.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.664+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/623784444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x561139271c00 0x561139253ef0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.664+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/623784444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139240bb0 0x5611392439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.664+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/623784444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5611392407d0 0x561139280410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.664+0000 7f8bcc65c740 1 -- 172.21.15.40:0/623784444 >> 172.21.15.40:0/623784444 conn(0x561139213f40 msgr2=0x56113923f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:27.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.664+0000 7f8bcc65c740 1 -- 172.21.15.40:0/623784444 shutdown_connections 2023-10-18T09:00:27.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.664+0000 7f8bcc65c740 1 -- 172.21.15.40:0/623784444 wait complete. 2023-10-18T09:00:27.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.665+0000 7f8bcc65c740 1 -- start start 2023-10-18T09:00:27.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.665+0000 7f8bcc65c740 1 Processor -- start 2023-10-18T09:00:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.666+0000 7f8bcc65c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5611392407d0 0x561139244e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.666+0000 7f8bcc65c740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139240bb0 0x56113925c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.666+0000 7f8bcc65c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561139271c00 0x56113925dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.666+0000 7f8bcc65c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56113926ddc0 con 0x5611392407d0 2023-10-18T09:00:27.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.666+0000 7f8bcc65c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611392c1e20 con 0x561139271c00 2023-10-18T09:00:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.666+0000 7f8bcc65c740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x56113925ea80 con 0x561139240bb0 2023-10-18T09:00:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.666+0000 7f8bcc65c740 10 client.0 _ll_register_callbacks cb 0x7f8bac037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:00:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.667+0000 7f8bc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5611392407d0 0x561139244e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.667+0000 7f8bc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5611392407d0 0x561139244e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40280/0 (socket says 172.21.15.40:40280) 2023-10-18T09:00:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.667+0000 7f8bc37fe640 1 -- 172.21.15.40:0/2170768935 learned_addr learned my addr 172.21.15.40:0/2170768935 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.667+0000 7f8bc3fff640 1 --2- 172.21.15.40:0/2170768935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561139271c00 0x56113925dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.667+0000 7f8bbbfff640 1 --2- 172.21.15.40:0/2170768935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139240bb0 0x56113925c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.667+0000 7f8bc37fe640 1 -- 172.21.15.40:0/2170768935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139240bb0 msgr2=0x56113925c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:00:27.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.667+0000 7f8bc37fe640 1 --2- 172.21.15.40:0/2170768935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139240bb0 0x56113925c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.667+0000 7f8bc37fe640 1 -- 172.21.15.40:0/2170768935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561139271c00 msgr2=0x56113925dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:00:27.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.667+0000 7f8bc37fe640 1 --2- 172.21.15.40:0/2170768935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561139271c00 0x56113925dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.667+0000 7f8bc37fe640 1 -- 172.21.15.40:0/2170768935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bbc0019d0 con 0x5611392407d0 2023-10-18T09:00:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.667+0000 7f8bc37fe640 1 --2- 172.21.15.40:0/2170768935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5611392407d0 0x561139244e10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4058800 tx=0x7f8bb400b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b9a2b1f9eac980 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.667+0000 7f8ba27fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T09:00:27.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.667+0000 7f8ba27fc640 1 -- 172.21.15.40:0/2170768935 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40701c0 con 0x5611392407d0 2023-10-18T09:00:27.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.667+0000 7f8ba27fc640 1 -- 172.21.15.40:0/2170768935 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bb4056cd0 con 0x5611392407d0 2023-10-18T09:00:27.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.667+0000 7f8bcc65c740 1 -- 172.21.15.40:0/2170768935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bac003a40 con 0x5611392407d0 2023-10-18T09:00:27.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.668+0000 7f8ba27fc640 1 -- 172.21.15.40:0/2170768935 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bb4058f70 con 0x5611392407d0 2023-10-18T09:00:27.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.668+0000 7f8bcc65c740 10 client.6747 fetch_fsmap learned FSMap version 41 2023-10-18T09:00:27.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.668+0000 7f8bcc65c740 1 -- 172.21.15.40:0/2170768935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x561139142170 con 0x5611392407d0 2023-10-18T09:00:27.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.668+0000 7f8ba27fc640 1 -- 172.21.15.40:0/2170768935 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8bac003a40 con 0x5611392407d0 2023-10-18T09:00:27.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.668+0000 7f8ba27fc640 1 -- 172.21.15.40:0/2170768935 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8bb4010ad0 con 0x5611392407d0 2023-10-18T09:00:27.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.668+0000 7f8bcc65c740 10 client.6747 fetch_fsmap finished waiting for FSMap version 41 2023-10-18T09:00:27.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.668+0000 7f8bcc65c740 10 client.6747 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T09:00:27.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.668+0000 7f8bcc65c740 20 client.6747 populate_metadata read hostname 'smithi040' 2023-10-18T09:00:27.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.668+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/2170768935 >> [v2:172.21.15.40:6834/1926421699,v1:172.21.15.40:6835/1926421699] conn(0x56113925e210 0x5611392bfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.668+0000 7f8bcc65c740 4 client.6747 mds_command: new command op to 6573 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:00:27.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.668+0000 7f8bcc65c740 1 -- 172.21.15.40:0/2170768935 --> [v2:172.21.15.40:6834/1926421699,v1:172.21.15.40:6835/1926421699] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561138e816c0 con 0x56113925e210 2023-10-18T09:00:27.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.668+0000 7f8bbbfff640 1 --2- 172.21.15.40:0/2170768935 >> [v2:172.21.15.40:6834/1926421699,v1:172.21.15.40:6835/1926421699] conn(0x56113925e210 0x5611392bfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.669+0000 7f8bbbfff640 1 --2- 172.21.15.40:0/2170768935 >> [v2:172.21.15.40:6834/1926421699,v1:172.21.15.40:6835/1926421699] conn(0x56113925e210 0x5611392bfdf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b48d47c27ca85b9 server_cookie=71fdac8cc2a1c4ee in_seq=0 out_seq=0 2023-10-18T09:00:27.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.669+0000 7f8ba27fc640 10 client.6747 ms_handle_connect on v2:172.21.15.40:6834/1926421699 2023-10-18T09:00:27.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.669+0000 7f8ba27fc640 1 -- 172.21.15.40:0/2170768935 <== mds.0 v2:172.21.15.40:6834/1926421699 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x561138e816c0 con 0x56113925e210 2023-10-18T09:00:27.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.669+0000 7f8ba27fc640 10 client.6747 handle_command_reply: tid=0 2023-10-18T09:00:27.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.669+0000 7f8bcc65c740 1 client.6747 shutdown 2023-10-18T09:00:27.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.669+0000 7f8bcc65c740 1 -- 172.21.15.40:0/2170768935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5611392407d0 msgr2=0x561139244e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:27.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.669+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/2170768935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5611392407d0 0x561139244e10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4058800 tx=0x7f8bb400b530 comp rx=0 tx=0).stop 2023-10-18T09:00:27.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.669+0000 7f8bcc65c740 1 -- 172.21.15.40:0/2170768935 shutdown_connections 2023-10-18T09:00:27.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.669+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/2170768935 >> [v2:172.21.15.40:6834/1926421699,v1:172.21.15.40:6835/1926421699] conn(0x56113925e210 0x5611392bfdf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.669+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/2170768935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561139271c00 0x56113925dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.669+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/2170768935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561139240bb0 0x56113925c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.669+0000 7f8bcc65c740 1 --2- 172.21.15.40:0/2170768935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5611392407d0 0x561139244e10 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.669+0000 7f8ba27fc640 0 client.6747 ms_handle_reset on v2:172.21.15.40:6834/1926421699 2023-10-18T09:00:27.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.669+0000 7f8bcc65c740 1 -- 172.21.15.40:0/2170768935 >> 172.21.15.40:0/2170768935 conn(0x561139213f40 msgr2=0x561139253ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:27.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.670+0000 7f8bcc65c740 1 -- 172.21.15.40:0/2170768935 shutdown_connections 2023-10-18T09:00:27.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.670+0000 7f8bcc65c740 1 -- 172.21.15.40:0/2170768935 wait complete. 2023-10-18T09:00:27.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.670+0000 7f8bcc65c740 20 client.6747 trim_cache size 0 max 16384 2023-10-18T09:00:27.690 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:00:27.691 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:00:27.691 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-10-18T09:00:27.691 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 6573, 2023-10-18T09:00:27.691 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-10-18T09:00:27.692 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-10-18T09:00:27.692 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-10-18T09:00:27.692 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 11.916556865, 2023-10-18T09:00:27.692 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 41, 2023-10-18T09:00:27.692 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 87, 2023-10-18T09:00:27.693 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 87, 2023-10-18T09:00:27.693 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 17.741448635000001 2023-10-18T09:00:27.693 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:00:27.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.670+0000 7f8bc9ac9640 1 -- 172.21.15.40:0/2521342641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ba406ae40 msgr2=0x7f8ba406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:27.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.670+0000 7f8bc9ac9640 1 --2- 172.21.15.40:0/2521342641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ba406ae40 0x7f8ba406d300 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8bac036000 tx=0x7f8bac004730 comp rx=0 tx=0).stop 2023-10-18T09:00:27.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.670+0000 7f8bc9ac9640 1 -- 172.21.15.40:0/2521342641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bc406e290 msgr2=0x7f8bc415c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:27.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.670+0000 7f8bc9ac9640 1 --2- 172.21.15.40:0/2521342641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bc406e290 0x7f8bc415c3f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400d950 tx=0x7f8bb400de10 comp rx=0 tx=0).stop 2023-10-18T09:00:27.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.670+0000 7f8bc9ac9640 1 -- 172.21.15.40:0/2521342641 shutdown_connections 2023-10-18T09:00:27.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.670+0000 7f8bc9ac9640 1 --2- 172.21.15.40:0/2521342641 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ba406ae40 0x7f8ba406d300 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.670+0000 7f8bc9ac9640 1 --2- 172.21.15.40:0/2521342641 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bc4142200 0x7f8bc413cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.670+0000 7f8bc9ac9640 1 --2- 172.21.15.40:0/2521342641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bc40ae700 0x7f8bc4158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.670+0000 7f8bc9ac9640 1 --2- 172.21.15.40:0/2521342641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bc406e290 0x7f8bc415c3f0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.670+0000 7f8bc9ac9640 1 -- 172.21.15.40:0/2521342641 >> 172.21.15.40:0/2521342641 conn(0x7f8bc40655f0 msgr2=0x7f8bc415cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:27.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.671+0000 7f8bc9ac9640 1 -- 172.21.15.40:0/2521342641 shutdown_connections 2023-10-18T09:00:27.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.671+0000 7f8bc9ac9640 1 -- 172.21.15.40:0/2521342641 wait complete. 2023-10-18T09:00:27.697 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T09:00:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.912+0000 7fb880813640 1 -- 172.21.15.40:0/776033213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0aa8f0 msgr2=0x7fb87c15cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.912+0000 7fb880813640 1 --2- 172.21.15.40:0/776033213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0aa8f0 0x7fb87c15cee0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb870005fd0 tx=0x7fb87002be20 comp rx=0 tx=0).stop 2023-10-18T09:00:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.912+0000 7fb880813640 1 -- 172.21.15.40:0/776033213 shutdown_connections 2023-10-18T09:00:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.912+0000 7fb880813640 1 --2- 172.21.15.40:0/776033213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0aa8f0 0x7fb87c15cee0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.912+0000 7fb880813640 1 -- 172.21.15.40:0/776033213 >> 172.21.15.40:0/776033213 conn(0x7fb87c0aed70 msgr2=0x7fb87c0af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:27.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.912+0000 7fb880813640 1 -- 172.21.15.40:0/776033213 shutdown_connections 2023-10-18T09:00:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.912+0000 7fb880813640 1 -- 172.21.15.40:0/776033213 wait complete. 2023-10-18T09:00:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.914+0000 7fb880813640 1 Processor -- start 2023-10-18T09:00:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.914+0000 7fb880813640 1 -- start start 2023-10-18T09:00:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.916+0000 7fb880813640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb87c06b4d0 0x7fb87c1f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.916+0000 7fb880813640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb87c0aa8f0 0x7fb87c1446f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.916+0000 7fb880813640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0ac070 0x7fb87c0c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.916+0000 7fb880813640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c0c4ca0 con 0x7fb87c0ac070 2023-10-18T09:00:27.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.916+0000 7fb880813640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c1f2bf0 con 0x7fb87c06b4d0 2023-10-18T09:00:27.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.916+0000 7fb880813640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c0c3df0 con 0x7fb87c0aa8f0 2023-10-18T09:00:27.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.916+0000 7fb87bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0ac070 0x7fb87c0c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.916+0000 7fb87bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0ac070 0x7fb87c0c24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40296/0 (socket says 172.21.15.40:40296) 2023-10-18T09:00:27.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.916+0000 7fb87bfff640 1 -- 172.21.15.40:0/2405273306 learned_addr learned my addr 172.21.15.40:0/2405273306 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:27.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.916+0000 7fb87b7fe640 1 --2- 172.21.15.40:0/2405273306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb87c06b4d0 0x7fb87c1f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.916+0000 7fb87affd640 1 --2- 172.21.15.40:0/2405273306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb87c0aa8f0 0x7fb87c1446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.916+0000 7fb87bfff640 1 -- 172.21.15.40:0/2405273306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb87c0aa8f0 msgr2=0x7fb87c1446f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:27.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.916+0000 7fb87bfff640 1 --2- 172.21.15.40:0/2405273306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb87c0aa8f0 0x7fb87c1446f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.916+0000 7fb87bfff640 1 -- 172.21.15.40:0/2405273306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb87c06b4d0 msgr2=0x7fb87c1f26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:27.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.917+0000 7fb87bfff640 1 --2- 172.21.15.40:0/2405273306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb87c06b4d0 0x7fb87c1f26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:27.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.917+0000 7fb87bfff640 1 -- 172.21.15.40:0/2405273306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb870009560 con 0x7fb87c0ac070 2023-10-18T09:00:27.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.917+0000 7fb87bfff640 1 --2- 172.21.15.40:0/2405273306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0ac070 0x7fb87c0c24b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb87400c7d0 tx=0x7fb87400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.917+0000 7fb863fff640 1 -- 172.21.15.40:0/2405273306 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874003b70 con 0x7fb87c0ac070 2023-10-18T09:00:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.917+0000 7fb863fff640 1 -- 172.21.15.40:0/2405273306 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb874003cf0 con 0x7fb87c0ac070 2023-10-18T09:00:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.917+0000 7fb863fff640 1 -- 172.21.15.40:0/2405273306 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874003ea0 con 0x7fb87c0ac070 2023-10-18T09:00:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.917+0000 7fb880813640 1 -- 172.21.15.40:0/2405273306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87c06be70 con 0x7fb87c0ac070 2023-10-18T09:00:27.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.917+0000 7fb880813640 1 -- 172.21.15.40:0/2405273306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87c0c4370 con 0x7fb87c0ac070 2023-10-18T09:00:27.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.919+0000 7fb880813640 1 -- 172.21.15.40:0/2405273306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87c0c4e50 con 0x7fb87c0ac070 2023-10-18T09:00:27.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.920+0000 7fb863fff640 1 -- 172.21.15.40:0/2405273306 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb87400aab0 con 0x7fb87c0ac070 2023-10-18T09:00:27.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.921+0000 7fb863fff640 1 --2- 172.21.15.40:0/2405273306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb86806ae40 0x7fb86806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:27.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.922+0000 7fb863fff640 1 -- 172.21.15.40:0/2405273306 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb874089690 con 0x7fb87c0ac070 2023-10-18T09:00:27.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.922+0000 7fb87b7fe640 1 --2- 172.21.15.40:0/2405273306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb86806ae40 0x7fb86806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:27.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.923+0000 7fb87b7fe640 1 --2- 172.21.15.40:0/2405273306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb86806ae40 0x7fb86806d300 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb86c006fd0 tx=0x7fb86c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:27.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:27.926+0000 7fb863fff640 1 -- 172.21.15.40:0/2405273306 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb87c0c4e50 con 0x7fb87c0ac070 2023-10-18T09:00:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:28.164+0000 7fb880813640 1 -- 172.21.15.40:0/2405273306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb87c0c4e50 con 0x7fb87c0ac070 2023-10-18T09:00:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:28.165+0000 7fb863fff640 1 -- 172.21.15.40:0/2405273306 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb87c0c4e50 con 0x7fb87c0ac070 2023-10-18T09:00:28.167 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T09:00:28.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:28.167+0000 7fb880813640 1 -- 172.21.15.40:0/2405273306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb86806ae40 msgr2=0x7fb86806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:28.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:28.167+0000 7fb880813640 1 --2- 172.21.15.40:0/2405273306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb86806ae40 0x7fb86806d300 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb86c006fd0 tx=0x7fb86c008040 comp rx=0 tx=0).stop 2023-10-18T09:00:28.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:28.167+0000 7fb880813640 1 -- 172.21.15.40:0/2405273306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0ac070 msgr2=0x7fb87c0c24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:28.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:28.167+0000 7fb880813640 1 --2- 172.21.15.40:0/2405273306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0ac070 0x7fb87c0c24b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb87400c7d0 tx=0x7fb87400cc90 comp rx=0 tx=0).stop 2023-10-18T09:00:28.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:28.167+0000 7fb880813640 1 -- 172.21.15.40:0/2405273306 shutdown_connections 2023-10-18T09:00:28.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:28.167+0000 7fb880813640 1 --2- 172.21.15.40:0/2405273306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb86806ae40 0x7fb86806d300 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:28.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:28.167+0000 7fb880813640 1 --2- 172.21.15.40:0/2405273306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb87c0ac070 0x7fb87c0c24b0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:28.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:28.167+0000 7fb880813640 1 --2- 172.21.15.40:0/2405273306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb87c0aa8f0 0x7fb87c1446f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:28.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:28.167+0000 7fb880813640 1 --2- 172.21.15.40:0/2405273306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb87c06b4d0 0x7fb87c1f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:28.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:28.168+0000 7fb880813640 1 -- 172.21.15.40:0/2405273306 >> 172.21.15.40:0/2405273306 conn(0x7fb87c0aed70 msgr2=0x7fb87c0caec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:28.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:28.168+0000 7fb880813640 1 -- 172.21.15.40:0/2405273306 shutdown_connections 2023-10-18T09:00:28.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:28.168+0000 7fb880813640 1 -- 172.21.15.40:0/2405273306 wait complete. 2023-10-18T09:00:28.189 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T09:00:28.190 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T09:00:28.190 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-10-18T09:00:28.190 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T09:00:28.191 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T09:00:28.191 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T09:00:28.191 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T09:00:28.192 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T09:00:28.192 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:00:28.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:28.193 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-10-18T09:00:28.250 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T09:00:28.251 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T09:00:28.251 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T09:00:28.251 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:00:28.251 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-18T09:00:28.251 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:00:28.252 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:00:28.252 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:00:28.252 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-10-18T09:00:28.252 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:00:28.252 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:00:28.252 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-10-18T09:00:28.253 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T09:00:28.253 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-10-18T09:00:28.253 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-10-18T09:00:28.253 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-18T09:00:28.253 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42096sec preferred_lft 42096sec 2023-10-18T09:00:28.254 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2023-10-18T09:00:28.254 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:00:28.254 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:00:28.254 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-10-18T09:00:28.254 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-10-18T09:00:28.255 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T09:00:28.255 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:00:28.255 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T09:00:28.255 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:00:28.255 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2026:95ff:fea9:a463/64 scope link 2023-10-18T09:00:28.255 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:00:28.256 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:00:28.256 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T09:00:28.256 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::80bd:e5ff:fe0f:6f68/64 scope link 2023-10-18T09:00:28.256 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:00:28.256 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:00:28.256 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether ea:2b:21:f5:06:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T09:00:28.257 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::e82b:21ff:fef5:6af/64 scope link 2023-10-18T09:00:28.257 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:00:28.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:28.258 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T09:00:28.360 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-10-18T09:00:28.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:28.365 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-10-18T09:00:28.425 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-10-18T09:00:28.426 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T09:00:28.426 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-18T09:00:28.426 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-18T09:00:28.427 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T09:00:28.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:28.427 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T09:00:28.428 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T09:00:28.428 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-18T09:00:28.429 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-18T09:00:28.429 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-18T09:00:28.429 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T09:00:28.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:28.627 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-18T09:00:28.681 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T09:00:28.682 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T09:00:28.683 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:00:28.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:28.684 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-10-18T09:00:28.786 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:00:28.787 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T09:00:28.848 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:00:28.849 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:00:28.849 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T09:00:28.906 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-18T09:00:28.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:28.907 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:00:29.061 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:00:29.062 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:00:29.099 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:00:29.099 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:00:29.099 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T09:00:29.100 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T09:00:29.100 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T09:00:29.101 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-18T09:00:29.101 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2023-10-18T09:00:29.101 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T09:00:29.102 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T09:00:29.102 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T09:00:29.106 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T09:00:29.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:29.107 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T09:00:29.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:29.191 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:00:29.265 INFO:teuthology.orchestra.run.smithi040.stdout:client5722 2023-10-18T09:00:29.265 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:00:29.266 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:00:29.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T09:00:29.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.553+0000 7f7f69859640 1 -- 172.21.15.40:0/1857539168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f640cceb0 msgr2=0x7f7f6406cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:29.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.553+0000 7f7f69859640 1 --2- 172.21.15.40:0/1857539168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f640cceb0 0x7f7f6406cbe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7f58005fd0 tx=0x7f7f5802bd90 comp rx=0 tx=0).stop 2023-10-18T09:00:29.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.553+0000 7f7f69859640 1 -- 172.21.15.40:0/1857539168 shutdown_connections 2023-10-18T09:00:29.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.553+0000 7f7f69859640 1 --2- 172.21.15.40:0/1857539168 >> v2:172.21.15.144:3300/0 conn(0x7f7f64070b90 0x7f7f64070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:29.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.553+0000 7f7f69859640 1 --2- 172.21.15.40:0/1857539168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f640cceb0 0x7f7f6406cbe0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:29.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.553+0000 7f7f69859640 1 -- 172.21.15.40:0/1857539168 >> 172.21.15.40:0/1857539168 conn(0x7f7f640afcf0 msgr2=0x7f7f640b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:29.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.553+0000 7f7f69859640 1 -- 172.21.15.40:0/1857539168 shutdown_connections 2023-10-18T09:00:29.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.553+0000 7f7f69859640 1 -- 172.21.15.40:0/1857539168 wait complete. 2023-10-18T09:00:29.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.555+0000 7f7f69859640 1 Processor -- start 2023-10-18T09:00:29.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.556+0000 7f7f69859640 1 -- start start 2023-10-18T09:00:29.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.557+0000 7f7f69859640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f64070b90 0x7f7f641437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:29.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.557+0000 7f7f69859640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f640cceb0 0x7f7f641e5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:29.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.557+0000 7f7f69859640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f6415ef60 0x7f7f641e6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:29.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.557+0000 7f7f69859640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640b2360 con 0x7f7f6415ef60 2023-10-18T09:00:29.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.557+0000 7f7f69859640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640b2640 con 0x7f7f640cceb0 2023-10-18T09:00:29.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.557+0000 7f7f69859640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640b2920 con 0x7f7f64070b90 2023-10-18T09:00:29.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.557+0000 7f7f69058640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f6415ef60 0x7f7f641e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:29.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.557+0000 7f7f69058640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f6415ef60 0x7f7f641e6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40306/0 (socket says 172.21.15.40:40306) 2023-10-18T09:00:29.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.557+0000 7f7f69058640 1 -- 172.21.15.40:0/2132240865 learned_addr learned my addr 172.21.15.40:0/2132240865 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:29.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.558+0000 7f7f68857640 1 --2- 172.21.15.40:0/2132240865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f64070b90 0x7f7f641437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:29.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.558+0000 7f7f63fff640 1 --2- 172.21.15.40:0/2132240865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f640cceb0 0x7f7f641e5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:29.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.558+0000 7f7f69058640 1 -- 172.21.15.40:0/2132240865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f64070b90 msgr2=0x7f7f641437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:29.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.558+0000 7f7f69058640 1 --2- 172.21.15.40:0/2132240865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f64070b90 0x7f7f641437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:29.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.558+0000 7f7f69058640 1 -- 172.21.15.40:0/2132240865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f640cceb0 msgr2=0x7f7f641e5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:29.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.558+0000 7f7f69058640 1 --2- 172.21.15.40:0/2132240865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f640cceb0 0x7f7f641e5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:29.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.558+0000 7f7f69058640 1 -- 172.21.15.40:0/2132240865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f58009560 con 0x7f7f6415ef60 2023-10-18T09:00:29.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.558+0000 7f7f69058640 1 --2- 172.21.15.40:0/2132240865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f6415ef60 0x7f7f641e6440 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00eb10 tx=0x7f7f5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:29.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.559+0000 7f7f617fa640 1 -- 172.21.15.40:0/2132240865 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c004b20 con 0x7f7f6415ef60 2023-10-18T09:00:29.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.559+0000 7f7f617fa640 1 -- 172.21.15.40:0/2132240865 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7f5c004ca0 con 0x7f7f6415ef60 2023-10-18T09:00:29.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.559+0000 7f7f617fa640 1 -- 172.21.15.40:0/2132240865 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c005280 con 0x7f7f6415ef60 2023-10-18T09:00:29.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.559+0000 7f7f69859640 1 -- 172.21.15.40:0/2132240865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f640b2c00 con 0x7f7f6415ef60 2023-10-18T09:00:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.559+0000 7f7f69859640 1 -- 172.21.15.40:0/2132240865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f640b31c0 con 0x7f7f6415ef60 2023-10-18T09:00:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.560+0000 7f7f69859640 1 -- 172.21.15.40:0/2132240865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f64066a90 con 0x7f7f6415ef60 2023-10-18T09:00:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.561+0000 7f7f617fa640 1 -- 172.21.15.40:0/2132240865 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7f5c0053e0 con 0x7f7f6415ef60 2023-10-18T09:00:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.562+0000 7f7f617fa640 1 --2- 172.21.15.40:0/2132240865 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7f4806ae40 0x7f7f4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.562+0000 7f7f617fa640 1 -- 172.21.15.40:0/2132240865 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f5c014070 con 0x7f7f6415ef60 2023-10-18T09:00:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.563+0000 7f7f68857640 1 --2- 172.21.15.40:0/2132240865 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7f4806ae40 0x7f7f4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.564+0000 7f7f68857640 1 --2- 172.21.15.40:0/2132240865 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7f4806ae40 0x7f7f4806d300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7f58035000 tx=0x7f7f58004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:29.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.566+0000 7f7f617fa640 1 -- 172.21.15.40:0/2132240865 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f5c055430 con 0x7f7f6415ef60 2023-10-18T09:00:29.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.802+0000 7f7f69859640 1 -- 172.21.15.40:0/2132240865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7f641f14f0 con 0x7f7f6415ef60 2023-10-18T09:00:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.803+0000 7f7f617fa640 1 -- 172.21.15.40:0/2132240865 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7f641f14f0 con 0x7f7f6415ef60 2023-10-18T09:00:29.805 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T09:00:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.805+0000 7f7f69859640 1 -- 172.21.15.40:0/2132240865 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7f4806ae40 msgr2=0x7f7f4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.805+0000 7f7f69859640 1 --2- 172.21.15.40:0/2132240865 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7f4806ae40 0x7f7f4806d300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7f58035000 tx=0x7f7f58004730 comp rx=0 tx=0).stop 2023-10-18T09:00:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.805+0000 7f7f69859640 1 -- 172.21.15.40:0/2132240865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f6415ef60 msgr2=0x7f7f641e6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.805+0000 7f7f69859640 1 --2- 172.21.15.40:0/2132240865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f6415ef60 0x7f7f641e6440 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00eb10 tx=0x7f7f5c00c2d0 comp rx=0 tx=0).stop 2023-10-18T09:00:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.805+0000 7f7f69859640 1 -- 172.21.15.40:0/2132240865 shutdown_connections 2023-10-18T09:00:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.806+0000 7f7f69859640 1 --2- 172.21.15.40:0/2132240865 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7f4806ae40 0x7f7f4806d300 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.806+0000 7f7f69859640 1 --2- 172.21.15.40:0/2132240865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f6415ef60 0x7f7f641e6440 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.806+0000 7f7f69859640 1 --2- 172.21.15.40:0/2132240865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f640cceb0 0x7f7f641e5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.806+0000 7f7f69859640 1 --2- 172.21.15.40:0/2132240865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f64070b90 0x7f7f641437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:29.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.806+0000 7f7f69859640 1 -- 172.21.15.40:0/2132240865 >> 172.21.15.40:0/2132240865 conn(0x7f7f640afcf0 msgr2=0x7f7f640cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:29.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.806+0000 7f7f69859640 1 -- 172.21.15.40:0/2132240865 shutdown_connections 2023-10-18T09:00:29.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:29.806+0000 7f7f69859640 1 -- 172.21.15.40:0/2132240865 wait complete. 2023-10-18T09:00:29.827 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T09:00:29.828 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T09:00:29.828 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-10-18T09:00:29.828 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-18T09:00:29.829 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T09:00:29.829 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T09:00:29.830 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T09:00:29.830 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T09:00:29.830 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:00:29.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:29.831 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-10-18T09:00:29.891 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T09:00:29.892 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T09:00:29.892 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T09:00:29.893 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:00:29.893 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-18T09:00:29.893 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:00:29.893 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:00:29.894 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:00:29.894 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2023-10-18T09:00:29.894 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:00:29.894 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:00:29.895 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2023-10-18T09:00:29.895 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:00:29.895 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-10-18T09:00:29.895 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T09:00:29.895 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-10-18T09:00:29.896 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T09:00:29.896 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42088sec preferred_lft 42088sec 2023-10-18T09:00:29.896 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2023-10-18T09:00:29.896 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:00:29.897 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T09:00:29.897 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff 2023-10-18T09:00:29.897 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T09:00:29.897 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:00:29.897 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1493:a0ff:fed2:2c1/64 scope link 2023-10-18T09:00:29.898 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:00:29.898 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:00:29.898 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 4a:6e:a9:f0:92:62 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T09:00:29.899 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::486e:a9ff:fef0:9262/64 scope link 2023-10-18T09:00:29.899 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:00:29.899 INFO:teuthology.orchestra.run.smithi144.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:00:29.899 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T09:00:29.899 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::345a:c5ff:fe54:de6e/64 scope link 2023-10-18T09:00:29.900 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:00:29.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:29.901 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T09:00:29.995 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-10-18T09:00:29.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:30.000 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-10-18T09:00:30.055 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-10-18T09:00:30.056 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T09:00:30.056 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T09:00:30.056 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T09:00:30.056 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T09:00:30.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:30.057 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T09:00:30.058 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T09:00:30.058 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T09:00:30.058 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T09:00:30.059 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T09:00:30.059 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T09:00:30.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:30.239 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-18T09:00:30.290 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T09:00:30.290 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-18T09:00:30.291 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:00:30.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:30.292 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2023-10-18T09:00:30.389 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:00:30.390 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-18T09:00:30.444 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:00:30.444 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:00:30.445 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-18T09:00:30.495 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-18T09:00:30.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:30.496 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:00:30.634 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:00:30.634 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:00:30.667 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:00:30.667 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:00:30.667 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T09:00:30.667 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with new device syntax: 1@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T09:00:30.668 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T09:00:30.669 INFO:teuthology.orchestra.run.smithi144.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-18T09:00:30.669 INFO:teuthology.orchestra.run.smithi144.stdout: You just mounted a file system that supports labels which does not 2023-10-18T09:00:30.669 INFO:teuthology.orchestra.run.smithi144.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T09:00:30.670 INFO:teuthology.orchestra.run.smithi144.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T09:00:30.670 INFO:teuthology.orchestra.run.smithi144.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T09:00:30.673 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T09:00:30.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:30.674 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-18T09:00:30.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:00:30.764 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:00:30.820 INFO:teuthology.orchestra.run.smithi144.stdout:client5734 2023-10-18T09:00:30.821 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:00:30.821 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:00:30.873 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:00:31.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.113+0000 7f7b3d62e640 1 -- 172.21.15.40:0/1168582923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b380b15b0 msgr2=0x7f7b380b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:31.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.113+0000 7f7b3d62e640 1 --2- 172.21.15.40:0/1168582923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b380b15b0 0x7f7b380b0630 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7b3806aeb0 tx=0x7f7b2802bd60 comp rx=0 tx=0).stop 2023-10-18T09:00:31.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.113+0000 7f7b3d62e640 1 -- 172.21.15.40:0/1168582923 shutdown_connections 2023-10-18T09:00:31.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.113+0000 7f7b3d62e640 1 --2- 172.21.15.40:0/1168582923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b380ca0d0 0x7f7b38157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:31.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.113+0000 7f7b3d62e640 1 --2- 172.21.15.40:0/1168582923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b380b15b0 0x7f7b380b0630 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:31.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.113+0000 7f7b3d62e640 1 -- 172.21.15.40:0/1168582923 >> 172.21.15.40:0/1168582923 conn(0x7f7b38069720 msgr2=0x7f7b38069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:31.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.114+0000 7f7b3d62e640 1 -- 172.21.15.40:0/1168582923 shutdown_connections 2023-10-18T09:00:31.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.114+0000 7f7b3d62e640 1 -- 172.21.15.40:0/1168582923 wait complete. 2023-10-18T09:00:31.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.115+0000 7f7b3d62e640 1 Processor -- start 2023-10-18T09:00:31.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.115+0000 7f7b3d62e640 1 -- start start 2023-10-18T09:00:31.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.116+0000 7f7b3d62e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b380b15b0 0x7f7b3814efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:31.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.116+0000 7f7b3d62e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b380b1e10 0x7f7b380af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:31.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.116+0000 7f7b3d62e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b380ca0d0 0x7f7b381e0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:31.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.116+0000 7f7b3d62e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b381e2b80 con 0x7f7b380b1e10 2023-10-18T09:00:31.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.116+0000 7f7b3d62e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b380afe40 con 0x7f7b380ca0d0 2023-10-18T09:00:31.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.116+0000 7f7b3d62e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3814f5c0 con 0x7f7b380b15b0 2023-10-18T09:00:31.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.117+0000 7f7b2f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b380b1e10 0x7f7b380af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:31.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.117+0000 7f7b2f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b380b1e10 0x7f7b380af900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40316/0 (socket says 172.21.15.40:40316) 2023-10-18T09:00:31.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.117+0000 7f7b2f7fe640 1 -- 172.21.15.40:0/4242237884 learned_addr learned my addr 172.21.15.40:0/4242237884 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:00:31.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.117+0000 7f7b37fff640 1 --2- 172.21.15.40:0/4242237884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b380b15b0 0x7f7b3814efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:31.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.117+0000 7f7b3ce2d640 1 --2- 172.21.15.40:0/4242237884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b380ca0d0 0x7f7b381e0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.117+0000 7f7b2f7fe640 1 -- 172.21.15.40:0/4242237884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b380b15b0 msgr2=0x7f7b3814efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.117+0000 7f7b2f7fe640 1 --2- 172.21.15.40:0/4242237884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b380b15b0 0x7f7b3814efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.117+0000 7f7b2f7fe640 1 -- 172.21.15.40:0/4242237884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b380ca0d0 msgr2=0x7f7b381e0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.117+0000 7f7b2f7fe640 1 --2- 172.21.15.40:0/4242237884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b380ca0d0 0x7f7b381e0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:31.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.117+0000 7f7b2f7fe640 1 -- 172.21.15.40:0/4242237884 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b28009df0 con 0x7f7b380b1e10 2023-10-18T09:00:31.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.118+0000 7f7b2f7fe640 1 --2- 172.21.15.40:0/4242237884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b380b1e10 0x7f7b380af900 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000c7c0 tx=0x7f7b2000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:31.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.118+0000 7f7b357fa640 1 -- 172.21.15.40:0/4242237884 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000aa50 con 0x7f7b380b1e10 2023-10-18T09:00:31.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.118+0000 7f7b357fa640 1 -- 172.21.15.40:0/4242237884 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7b2000aca0 con 0x7f7b380b1e10 2023-10-18T09:00:31.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.118+0000 7f7b357fa640 1 -- 172.21.15.40:0/4242237884 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20012680 con 0x7f7b380b1e10 2023-10-18T09:00:31.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.118+0000 7f7b3d62e640 1 -- 172.21.15.40:0/4242237884 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3814f8a0 con 0x7f7b380b1e10 2023-10-18T09:00:31.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.118+0000 7f7b3d62e640 1 -- 172.21.15.40:0/4242237884 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3814fe60 con 0x7f7b380b1e10 2023-10-18T09:00:31.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.119+0000 7f7b3d62e640 1 -- 172.21.15.40:0/4242237884 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b381e29c0 con 0x7f7b380b1e10 2023-10-18T09:00:31.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.120+0000 7f7b357fa640 1 -- 172.21.15.40:0/4242237884 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7b2001b400 con 0x7f7b380b1e10 2023-10-18T09:00:31.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.121+0000 7f7b357fa640 1 --2- 172.21.15.40:0/4242237884 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7b1806ae70 0x7f7b1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:00:31.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.121+0000 7f7b357fa640 1 -- 172.21.15.40:0/4242237884 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b20089b50 con 0x7f7b380b1e10 2023-10-18T09:00:31.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.122+0000 7f7b37fff640 1 --2- 172.21.15.40:0/4242237884 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7b1806ae70 0x7f7b1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:00:31.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.123+0000 7f7b37fff640 1 --2- 172.21.15.40:0/4242237884 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7b1806ae70 0x7f7b1806d330 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7b28035000 tx=0x7f7b28004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:00:31.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.126+0000 7f7b357fa640 1 -- 172.21.15.40:0/4242237884 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b381e29c0 con 0x7f7b380b1e10 2023-10-18T09:00:31.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.360+0000 7f7b3d62e640 1 -- 172.21.15.40:0/4242237884 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b381e29c0 con 0x7f7b380b1e10 2023-10-18T09:00:31.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.361+0000 7f7b357fa640 1 -- 172.21.15.40:0/4242237884 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f7b381e29c0 con 0x7f7b380b1e10 2023-10-18T09:00:31.363 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:00:31.364 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1341351808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1341351808},{"type":"v1","addr":"172.21.15.144:6836","nonce":1341351808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1187670073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1187670073},{"type":"v1","addr":"172.21.15.40:6833","nonce":1187670073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5626,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1615448513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1615448513},{"type":"v1","addr":"172.21.15.144:6837","nonce":1615448513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:00:15.197720+0000","modified":"2023-10-18T09:00:18.306206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1926421699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1926421699},{"type":"v1","addr":"172.21.15.40:6835","nonce":1926421699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-18T09:00:31.364 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 41 2023-10-18T09:00:31.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.363+0000 7f7b3d62e640 1 -- 172.21.15.40:0/4242237884 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7b1806ae70 msgr2=0x7f7b1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:31.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.363+0000 7f7b3d62e640 1 --2- 172.21.15.40:0/4242237884 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7b1806ae70 0x7f7b1806d330 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7b28035000 tx=0x7f7b28004730 comp rx=0 tx=0).stop 2023-10-18T09:00:31.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.364+0000 7f7b3d62e640 1 -- 172.21.15.40:0/4242237884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b380b1e10 msgr2=0x7f7b380af900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:00:31.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.364+0000 7f7b3d62e640 1 --2- 172.21.15.40:0/4242237884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b380b1e10 0x7f7b380af900 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000c7c0 tx=0x7f7b2000cc80 comp rx=0 tx=0).stop 2023-10-18T09:00:31.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.364+0000 7f7b3d62e640 1 -- 172.21.15.40:0/4242237884 shutdown_connections 2023-10-18T09:00:31.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.364+0000 7f7b3d62e640 1 --2- 172.21.15.40:0/4242237884 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7b1806ae70 0x7f7b1806d330 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:31.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.364+0000 7f7b3d62e640 1 --2- 172.21.15.40:0/4242237884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b380ca0d0 0x7f7b381e0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:31.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.364+0000 7f7b3d62e640 1 --2- 172.21.15.40:0/4242237884 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b380b1e10 0x7f7b380af900 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:31.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.364+0000 7f7b3d62e640 1 --2- 172.21.15.40:0/4242237884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b380b15b0 0x7f7b3814efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:00:31.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.364+0000 7f7b3d62e640 1 -- 172.21.15.40:0/4242237884 >> 172.21.15.40:0/4242237884 conn(0x7f7b38069720 msgr2=0x7f7b381545d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:00:31.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.364+0000 7f7b3d62e640 1 -- 172.21.15.40:0/4242237884 shutdown_connections 2023-10-18T09:00:31.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:00:31.364+0000 7f7b3d62e640 1 -- 172.21.15.40:0/4242237884 wait complete. 2023-10-18T09:00:31.386 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-18T09:00:31.631 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":6573,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.859975705,"mdsmap_epoch":41,"osdmap_epoch":87,"osdmap_epoch_barrier":87,"uptime":21.684880237000002} 2023-10-18T09:00:31.631 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 6573, "mdsmap_epoch": 41, "osdmap_epoch": 87, "osdmap_epoch_barrier": 87, "rank_uptime": 15.859975705, "state": "up:active", "uptime": 21.684880237, "want_state": "up:active", "whoami": 0 } 2023-10-18T09:00:31.632 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-18T09:00:31.877 INFO:teuthology.orchestra.run.smithi040.stdout:{"osd_mon_report_interval":"5"} 2023-10-18T09:00:31.877 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-18T09:00:31.878 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-18T09:00:32.114 INFO:teuthology.orchestra.run.smithi040.stdout:{"mon_tick_interval":"5"} 2023-10-18T09:00:32.115 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-18T09:00:32.115 INFO:tasks.cephfs.test_full:pool capacity 557049408, 531MB should be enough to fill it 2023-10-18T09:00:32.115 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2023-10-18T09:00:32.116 DEBUG:teuthology.orchestra.run.smithi040:> import time 2023-10-18T09:00:32.116 DEBUG:teuthology.orchestra.run.smithi040:> import datetime 2023-10-18T09:00:32.116 DEBUG:teuthology.orchestra.run.smithi040:> import subprocess 2023-10-18T09:00:32.117 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-10-18T09:00:32.117 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:00:32.117 DEBUG:teuthology.orchestra.run.smithi040:> # Write some buffered data through before going full, all should be well 2023-10-18T09:00:32.118 DEBUG:teuthology.orchestra.run.smithi040:> print("writing some data through which we expect to succeed") 2023-10-18T09:00:32.118 DEBUG:teuthology.orchestra.run.smithi040:> bytes = 0 2023-10-18T09:00:32.118 DEBUG:teuthology.orchestra.run.smithi040:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-18T09:00:32.119 DEBUG:teuthology.orchestra.run.smithi040:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-18T09:00:32.119 DEBUG:teuthology.orchestra.run.smithi040:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2023-10-18T09:00:32.119 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(f) 2023-10-18T09:00:32.120 DEBUG:teuthology.orchestra.run.smithi040:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-18T09:00:32.120 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:00:32.120 DEBUG:teuthology.orchestra.run.smithi040:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-18T09:00:32.121 DEBUG:teuthology.orchestra.run.smithi040:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-18T09:00:32.121 DEBUG:teuthology.orchestra.run.smithi040:> # buffered IO, the error should always only appear from fsync and not 2023-10-18T09:00:32.121 DEBUG:teuthology.orchestra.run.smithi040:> # from write 2023-10-18T09:00:32.122 DEBUG:teuthology.orchestra.run.smithi040:> full = False 2023-10-18T09:00:32.122 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:00:32.122 DEBUG:teuthology.orchestra.run.smithi040:> for n in range(0, int(531 * 1.1)): 2023-10-18T09:00:32.123 DEBUG:teuthology.orchestra.run.smithi040:> try: 2023-10-18T09:00:32.123 DEBUG:teuthology.orchestra.run.smithi040:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-18T09:00:32.123 DEBUG:teuthology.orchestra.run.smithi040:> print("wrote bytes via buffered write, moving on to fsync") 2023-10-18T09:00:32.124 DEBUG:teuthology.orchestra.run.smithi040:> except OSError as e: 2023-10-18T09:00:32.124 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2023-10-18T09:00:32.124 DEBUG:teuthology.orchestra.run.smithi040:> print("Unexpected error %s from write() instead of fsync()" % e) 2023-10-18T09:00:32.125 DEBUG:teuthology.orchestra.run.smithi040:> raise 2023-10-18T09:00:32.125 DEBUG:teuthology.orchestra.run.smithi040:> else: 2023-10-18T09:00:32.125 DEBUG:teuthology.orchestra.run.smithi040:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-18T09:00:32.126 DEBUG:teuthology.orchestra.run.smithi040:> full = True 2023-10-18T09:00:32.126 DEBUG:teuthology.orchestra.run.smithi040:> break 2023-10-18T09:00:32.126 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:00:32.127 DEBUG:teuthology.orchestra.run.smithi040:> try: 2023-10-18T09:00:32.127 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(f) 2023-10-18T09:00:32.127 DEBUG:teuthology.orchestra.run.smithi040:> print("fsync'"'"'ed successfully") 2023-10-18T09:00:32.128 DEBUG:teuthology.orchestra.run.smithi040:> except OSError as e: 2023-10-18T09:00:32.128 DEBUG:teuthology.orchestra.run.smithi040:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-18T09:00:32.128 DEBUG:teuthology.orchestra.run.smithi040:> full = True 2023-10-18T09:00:32.129 DEBUG:teuthology.orchestra.run.smithi040:> break 2023-10-18T09:00:32.129 DEBUG:teuthology.orchestra.run.smithi040:> else: 2023-10-18T09:00:32.129 DEBUG:teuthology.orchestra.run.smithi040:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-18T09:00:32.130 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:00:32.130 DEBUG:teuthology.orchestra.run.smithi040:> if n > 531 * 0.9: 2023-10-18T09:00:32.130 DEBUG:teuthology.orchestra.run.smithi040:> # Be cautious in the last region where we expect to hit 2023-10-18T09:00:32.131 DEBUG:teuthology.orchestra.run.smithi040:> # the full condition, so that we don'"'"'t overshoot too dramatically 2023-10-18T09:00:32.131 DEBUG:teuthology.orchestra.run.smithi040:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-18T09:00:32.131 DEBUG:teuthology.orchestra.run.smithi040:> time.sleep(15.0) 2023-10-18T09:00:32.132 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:00:32.132 DEBUG:teuthology.orchestra.run.smithi040:> if not full: 2023-10-18T09:00:32.132 DEBUG:teuthology.orchestra.run.smithi040:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2023-10-18T09:00:32.133 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:00:32.133 DEBUG:teuthology.orchestra.run.smithi040:> # close() should not raise an error because we already caught it in 2023-10-18T09:00:32.133 DEBUG:teuthology.orchestra.run.smithi040:> # fsync. There shouldn'"'"'t have been any more writeback errors 2023-10-18T09:00:32.133 DEBUG:teuthology.orchestra.run.smithi040:> # since then because all IOs got cancelled on the full flag. 2023-10-18T09:00:32.134 DEBUG:teuthology.orchestra.run.smithi040:> print("calling close") 2023-10-18T09:00:32.134 DEBUG:teuthology.orchestra.run.smithi040:> os.close(f) 2023-10-18T09:00:32.134 DEBUG:teuthology.orchestra.run.smithi040:> print("close() did not raise error") 2023-10-18T09:00:32.135 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:00:32.135 DEBUG:teuthology.orchestra.run.smithi040:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-18T09:00:32.135 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-10-18T09:00:35.623 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:00:35.636 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:00:35.955 INFO:teuthology.orchestra.run.smithi040.stdout:writing some data through which we expect to succeed 2023-10-18T09:00:35.956 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-18T09:00:35.956 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.956 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.956 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 1.00 MB 2023-10-18T09:00:35.957 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.957 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.957 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 2.00 MB 2023-10-18T09:00:35.957 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.957 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.958 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 3.00 MB 2023-10-18T09:00:35.958 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.958 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.958 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 4.00 MB 2023-10-18T09:00:35.958 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.959 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.959 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 5.00 MB 2023-10-18T09:00:35.959 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.959 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.959 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 6.00 MB 2023-10-18T09:00:35.960 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.960 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.960 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 7.00 MB 2023-10-18T09:00:35.960 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.960 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.960 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 8.00 MB 2023-10-18T09:00:35.961 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.961 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.961 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 9.00 MB 2023-10-18T09:00:35.961 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.961 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.962 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 10.00 MB 2023-10-18T09:00:35.962 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.962 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.962 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 11.00 MB 2023-10-18T09:00:35.962 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.963 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.963 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 12.00 MB 2023-10-18T09:00:35.963 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.963 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.963 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 13.00 MB 2023-10-18T09:00:35.964 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.964 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.964 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 14.00 MB 2023-10-18T09:00:35.964 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.964 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.965 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 15.00 MB 2023-10-18T09:00:35.965 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.965 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.965 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 16.00 MB 2023-10-18T09:00:35.965 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.966 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.966 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 17.00 MB 2023-10-18T09:00:35.966 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.966 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.966 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 18.00 MB 2023-10-18T09:00:35.966 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.967 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.967 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 19.00 MB 2023-10-18T09:00:35.967 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.967 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.967 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 20.00 MB 2023-10-18T09:00:35.968 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.968 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.968 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 21.00 MB 2023-10-18T09:00:35.968 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.968 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.969 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 22.00 MB 2023-10-18T09:00:35.969 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.969 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.969 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 23.00 MB 2023-10-18T09:00:35.969 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.969 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.970 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 24.00 MB 2023-10-18T09:00:35.970 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.970 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.970 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 25.00 MB 2023-10-18T09:00:35.970 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.971 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.971 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 26.00 MB 2023-10-18T09:00:35.971 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.971 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.971 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 27.00 MB 2023-10-18T09:00:35.972 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.972 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.972 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 28.00 MB 2023-10-18T09:00:35.972 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.972 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.973 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 29.00 MB 2023-10-18T09:00:35.973 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.973 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.973 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 30.00 MB 2023-10-18T09:00:35.973 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.973 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.974 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 31.00 MB 2023-10-18T09:00:35.974 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.974 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.974 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 32.00 MB 2023-10-18T09:00:35.974 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.975 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.975 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 33.00 MB 2023-10-18T09:00:35.975 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.975 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.975 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 34.00 MB 2023-10-18T09:00:35.976 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.976 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.976 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 35.00 MB 2023-10-18T09:00:35.976 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.976 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.977 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 36.00 MB 2023-10-18T09:00:35.977 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.977 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.977 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 37.00 MB 2023-10-18T09:00:35.977 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.978 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.978 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 38.00 MB 2023-10-18T09:00:35.978 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.978 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.978 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 39.00 MB 2023-10-18T09:00:35.979 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.979 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.979 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 40.00 MB 2023-10-18T09:00:35.979 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.979 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.979 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 41.00 MB 2023-10-18T09:00:35.980 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.980 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.980 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 42.00 MB 2023-10-18T09:00:35.980 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.980 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.980 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 43.00 MB 2023-10-18T09:00:35.981 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.981 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.981 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 44.00 MB 2023-10-18T09:00:35.981 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.981 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.982 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 45.00 MB 2023-10-18T09:00:35.982 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.982 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.982 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 46.00 MB 2023-10-18T09:00:35.982 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.983 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.983 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 47.00 MB 2023-10-18T09:00:35.983 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.983 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.983 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 48.00 MB 2023-10-18T09:00:35.983 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.984 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.984 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 49.00 MB 2023-10-18T09:00:35.984 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.984 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.984 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 50.00 MB 2023-10-18T09:00:35.985 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.985 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.985 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 51.00 MB 2023-10-18T09:00:35.985 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.985 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.985 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 52.00 MB 2023-10-18T09:00:35.986 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.986 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.986 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 53.00 MB 2023-10-18T09:00:35.986 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.986 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.987 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 54.00 MB 2023-10-18T09:00:35.987 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.987 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.987 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 55.00 MB 2023-10-18T09:00:35.987 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.988 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.988 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 56.00 MB 2023-10-18T09:00:35.988 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.988 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.988 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 57.00 MB 2023-10-18T09:00:35.989 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.989 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.989 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 58.00 MB 2023-10-18T09:00:35.989 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.989 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.989 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 59.00 MB 2023-10-18T09:00:35.990 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.990 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.990 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 60.00 MB 2023-10-18T09:00:35.990 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.990 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.991 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 61.00 MB 2023-10-18T09:00:35.991 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.991 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.991 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 62.00 MB 2023-10-18T09:00:35.991 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.991 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.992 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 63.00 MB 2023-10-18T09:00:35.992 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.992 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.992 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 64.00 MB 2023-10-18T09:00:35.992 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.992 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.993 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 65.00 MB 2023-10-18T09:00:35.993 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.993 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.993 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 66.00 MB 2023-10-18T09:00:35.993 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.993 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.994 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 67.00 MB 2023-10-18T09:00:35.994 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.994 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.994 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 68.00 MB 2023-10-18T09:00:35.994 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.995 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.995 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 69.00 MB 2023-10-18T09:00:35.995 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.995 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.995 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 70.00 MB 2023-10-18T09:00:35.996 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.996 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.996 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 71.00 MB 2023-10-18T09:00:35.996 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.996 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.996 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 72.00 MB 2023-10-18T09:00:35.997 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.997 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.997 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 73.00 MB 2023-10-18T09:00:35.997 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.997 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.997 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 74.00 MB 2023-10-18T09:00:35.998 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.998 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.998 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 75.00 MB 2023-10-18T09:00:35.998 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.998 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.998 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 76.00 MB 2023-10-18T09:00:35.999 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.999 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.999 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 77.00 MB 2023-10-18T09:00:35.999 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:35.999 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:35.999 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 78.00 MB 2023-10-18T09:00:36.000 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.000 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.000 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 79.00 MB 2023-10-18T09:00:36.000 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.000 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.001 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 80.00 MB 2023-10-18T09:00:36.966 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.966 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.966 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 81.00 MB 2023-10-18T09:00:36.967 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.967 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.967 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 82.00 MB 2023-10-18T09:00:36.967 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.967 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.968 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 83.00 MB 2023-10-18T09:00:36.968 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.968 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.968 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 84.00 MB 2023-10-18T09:00:36.968 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.969 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.969 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 85.00 MB 2023-10-18T09:00:36.969 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.969 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.969 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 86.00 MB 2023-10-18T09:00:36.970 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.970 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.970 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 87.00 MB 2023-10-18T09:00:36.970 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.970 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.971 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 88.00 MB 2023-10-18T09:00:36.971 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.971 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.971 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 89.00 MB 2023-10-18T09:00:36.971 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.972 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.972 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 90.00 MB 2023-10-18T09:00:36.972 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.972 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.972 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 91.00 MB 2023-10-18T09:00:36.973 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.973 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.973 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 92.00 MB 2023-10-18T09:00:36.973 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.973 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.974 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 93.00 MB 2023-10-18T09:00:36.974 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.974 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.974 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 94.00 MB 2023-10-18T09:00:36.974 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.975 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.975 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 95.00 MB 2023-10-18T09:00:36.975 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.975 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.975 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 96.00 MB 2023-10-18T09:00:36.976 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.976 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.976 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 97.00 MB 2023-10-18T09:00:36.976 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.976 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.976 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 98.00 MB 2023-10-18T09:00:36.977 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.977 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.977 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 99.00 MB 2023-10-18T09:00:36.977 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.977 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.978 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 100.00 MB 2023-10-18T09:00:36.978 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.978 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.978 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 101.00 MB 2023-10-18T09:00:36.978 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.979 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.979 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 102.00 MB 2023-10-18T09:00:36.979 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.979 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.979 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 103.00 MB 2023-10-18T09:00:36.980 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.980 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.980 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 104.00 MB 2023-10-18T09:00:36.980 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.980 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.981 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 105.00 MB 2023-10-18T09:00:36.981 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.981 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.981 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 106.00 MB 2023-10-18T09:00:36.981 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.982 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.982 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 107.00 MB 2023-10-18T09:00:36.982 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.982 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.982 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 108.00 MB 2023-10-18T09:00:36.983 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.983 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.983 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 109.00 MB 2023-10-18T09:00:36.983 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.983 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.984 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 110.00 MB 2023-10-18T09:00:36.984 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.984 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.984 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 111.00 MB 2023-10-18T09:00:36.984 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.985 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.985 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 112.00 MB 2023-10-18T09:00:36.985 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.985 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.985 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 113.00 MB 2023-10-18T09:00:36.986 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.986 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.986 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 114.00 MB 2023-10-18T09:00:36.986 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.986 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.987 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 115.00 MB 2023-10-18T09:00:36.987 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.987 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.987 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 116.00 MB 2023-10-18T09:00:36.987 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.987 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.988 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 117.00 MB 2023-10-18T09:00:36.988 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.988 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.988 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 118.00 MB 2023-10-18T09:00:36.989 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.989 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.989 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 119.00 MB 2023-10-18T09:00:36.989 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.989 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.990 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 120.00 MB 2023-10-18T09:00:36.990 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.990 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.990 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 121.00 MB 2023-10-18T09:00:36.990 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.991 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.991 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 122.00 MB 2023-10-18T09:00:36.991 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.991 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.991 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 123.00 MB 2023-10-18T09:00:36.991 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.992 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.992 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 124.00 MB 2023-10-18T09:00:36.992 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.992 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.992 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 125.00 MB 2023-10-18T09:00:36.993 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.993 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.993 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 126.00 MB 2023-10-18T09:00:36.993 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.993 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.994 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 127.00 MB 2023-10-18T09:00:36.994 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.994 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.994 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 128.00 MB 2023-10-18T09:00:36.994 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.995 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.995 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 129.00 MB 2023-10-18T09:00:36.995 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.995 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.995 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 130.00 MB 2023-10-18T09:00:36.996 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.996 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.996 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 131.00 MB 2023-10-18T09:00:36.996 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.996 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.997 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 132.00 MB 2023-10-18T09:00:36.997 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.997 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.997 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 133.00 MB 2023-10-18T09:00:36.998 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.998 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.998 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 134.00 MB 2023-10-18T09:00:36.998 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.998 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.999 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 135.00 MB 2023-10-18T09:00:36.999 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:36.999 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:36.999 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 136.00 MB 2023-10-18T09:00:36.999 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.000 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.000 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 137.00 MB 2023-10-18T09:00:37.000 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.000 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.000 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 138.00 MB 2023-10-18T09:00:37.001 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.001 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.001 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 139.00 MB 2023-10-18T09:00:37.001 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.001 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.002 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 140.00 MB 2023-10-18T09:00:37.002 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.002 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.002 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 141.00 MB 2023-10-18T09:00:37.002 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.003 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.003 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 142.00 MB 2023-10-18T09:00:37.003 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.003 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.003 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 143.00 MB 2023-10-18T09:00:37.004 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.004 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.004 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 144.00 MB 2023-10-18T09:00:37.004 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.004 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.005 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 145.00 MB 2023-10-18T09:00:37.005 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.005 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.005 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 146.00 MB 2023-10-18T09:00:37.005 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.006 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.006 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 147.00 MB 2023-10-18T09:00:37.006 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.006 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.006 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 148.00 MB 2023-10-18T09:00:37.007 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.007 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.007 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 149.00 MB 2023-10-18T09:00:37.007 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.008 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.008 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 150.00 MB 2023-10-18T09:00:37.008 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.008 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.008 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 151.00 MB 2023-10-18T09:00:37.009 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.009 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.009 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 152.00 MB 2023-10-18T09:00:37.009 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.009 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.010 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 153.00 MB 2023-10-18T09:00:37.010 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.010 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.010 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 154.00 MB 2023-10-18T09:00:37.010 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.011 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.011 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 155.00 MB 2023-10-18T09:00:37.011 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.011 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.011 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 156.00 MB 2023-10-18T09:00:37.011 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.012 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.012 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 157.00 MB 2023-10-18T09:00:37.012 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.012 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.012 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 158.00 MB 2023-10-18T09:00:37.013 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.013 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.013 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 159.00 MB 2023-10-18T09:00:37.013 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.013 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.014 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 160.00 MB 2023-10-18T09:00:37.014 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.838 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.839 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 161.00 MB 2023-10-18T09:00:37.839 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.840 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.840 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 162.00 MB 2023-10-18T09:00:37.840 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.840 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.841 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 163.00 MB 2023-10-18T09:00:37.841 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.842 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.842 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 164.00 MB 2023-10-18T09:00:37.842 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.843 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.843 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 165.00 MB 2023-10-18T09:00:37.844 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.844 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.844 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 166.00 MB 2023-10-18T09:00:37.845 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.845 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.845 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 167.00 MB 2023-10-18T09:00:37.846 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.846 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.846 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 168.00 MB 2023-10-18T09:00:37.847 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.847 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.848 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 169.00 MB 2023-10-18T09:00:37.848 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.848 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.848 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 170.00 MB 2023-10-18T09:00:37.849 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.849 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.849 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 171.00 MB 2023-10-18T09:00:37.849 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.849 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.849 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 172.00 MB 2023-10-18T09:00:37.850 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.850 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.850 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 173.00 MB 2023-10-18T09:00:37.850 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.851 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.851 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 174.00 MB 2023-10-18T09:00:37.851 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.851 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.851 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 175.00 MB 2023-10-18T09:00:37.852 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.852 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.852 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 176.00 MB 2023-10-18T09:00:37.853 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.853 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.854 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 177.00 MB 2023-10-18T09:00:37.854 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.854 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.855 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 178.00 MB 2023-10-18T09:00:37.855 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.855 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.856 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 179.00 MB 2023-10-18T09:00:37.856 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.856 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.857 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 180.00 MB 2023-10-18T09:00:37.859 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.859 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.859 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 181.00 MB 2023-10-18T09:00:37.859 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.860 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.860 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 182.00 MB 2023-10-18T09:00:37.860 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.860 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.860 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 183.00 MB 2023-10-18T09:00:37.861 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.861 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.861 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 184.00 MB 2023-10-18T09:00:37.862 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.862 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.862 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 185.00 MB 2023-10-18T09:00:37.862 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.862 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.863 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 186.00 MB 2023-10-18T09:00:37.863 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.863 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.864 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 187.00 MB 2023-10-18T09:00:37.864 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.864 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.864 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 188.00 MB 2023-10-18T09:00:37.865 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.866 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.866 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 189.00 MB 2023-10-18T09:00:37.867 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.867 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.867 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 190.00 MB 2023-10-18T09:00:37.867 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.867 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.868 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 191.00 MB 2023-10-18T09:00:37.868 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.868 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.868 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 192.00 MB 2023-10-18T09:00:37.869 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.869 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.869 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 193.00 MB 2023-10-18T09:00:37.870 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.870 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.870 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 194.00 MB 2023-10-18T09:00:37.870 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.871 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.871 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 195.00 MB 2023-10-18T09:00:37.871 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.871 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.871 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 196.00 MB 2023-10-18T09:00:37.872 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.872 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.872 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 197.00 MB 2023-10-18T09:00:37.872 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.872 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.873 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 198.00 MB 2023-10-18T09:00:37.873 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.873 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.873 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 199.00 MB 2023-10-18T09:00:37.873 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.874 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.874 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 200.00 MB 2023-10-18T09:00:37.874 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.874 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.874 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 201.00 MB 2023-10-18T09:00:37.875 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.875 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.875 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 202.00 MB 2023-10-18T09:00:37.875 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.875 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.875 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 203.00 MB 2023-10-18T09:00:37.876 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.876 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.876 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 204.00 MB 2023-10-18T09:00:37.876 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.877 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.877 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 205.00 MB 2023-10-18T09:00:37.877 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.877 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.877 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 206.00 MB 2023-10-18T09:00:37.878 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.878 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.878 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 207.00 MB 2023-10-18T09:00:37.878 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.878 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.879 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 208.00 MB 2023-10-18T09:00:37.879 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.879 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.879 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 209.00 MB 2023-10-18T09:00:37.880 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.880 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.880 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 210.00 MB 2023-10-18T09:00:37.880 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.880 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.881 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 211.00 MB 2023-10-18T09:00:37.881 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.881 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.881 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 212.00 MB 2023-10-18T09:00:37.881 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.882 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.882 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 213.00 MB 2023-10-18T09:00:37.882 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.883 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.883 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 214.00 MB 2023-10-18T09:00:37.883 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.883 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.883 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 215.00 MB 2023-10-18T09:00:37.884 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.884 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.884 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 216.00 MB 2023-10-18T09:00:37.884 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.885 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.885 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 217.00 MB 2023-10-18T09:00:37.885 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.885 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.885 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 218.00 MB 2023-10-18T09:00:37.886 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.886 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.886 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 219.00 MB 2023-10-18T09:00:37.886 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.886 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.887 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 220.00 MB 2023-10-18T09:00:37.887 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.887 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.887 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 221.00 MB 2023-10-18T09:00:37.887 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.888 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.888 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 222.00 MB 2023-10-18T09:00:37.888 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.888 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.888 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 223.00 MB 2023-10-18T09:00:37.889 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.889 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.889 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 224.00 MB 2023-10-18T09:00:37.889 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.889 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.890 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 225.00 MB 2023-10-18T09:00:37.890 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.890 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.890 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 226.00 MB 2023-10-18T09:00:37.891 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.891 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.891 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 227.00 MB 2023-10-18T09:00:37.891 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.891 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.891 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 228.00 MB 2023-10-18T09:00:37.892 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.892 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.892 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 229.00 MB 2023-10-18T09:00:37.892 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.893 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.893 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 230.00 MB 2023-10-18T09:00:37.893 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.893 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.893 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 231.00 MB 2023-10-18T09:00:37.893 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.894 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.894 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 232.00 MB 2023-10-18T09:00:37.894 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.894 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.894 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 233.00 MB 2023-10-18T09:00:37.895 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.895 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.895 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 234.00 MB 2023-10-18T09:00:37.895 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.895 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.896 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 235.00 MB 2023-10-18T09:00:37.896 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.896 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.896 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 236.00 MB 2023-10-18T09:00:37.896 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.897 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.897 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 237.00 MB 2023-10-18T09:00:37.897 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.897 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.897 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 238.00 MB 2023-10-18T09:00:37.898 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.898 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.898 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 239.00 MB 2023-10-18T09:00:37.898 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.898 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:37.899 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 240.00 MB 2023-10-18T09:00:37.899 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:37.899 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.726 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 241.00 MB 2023-10-18T09:00:38.726 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.726 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.726 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 242.00 MB 2023-10-18T09:00:38.727 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.727 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.727 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 243.00 MB 2023-10-18T09:00:38.727 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.727 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.728 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 244.00 MB 2023-10-18T09:00:38.728 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.728 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.728 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 245.00 MB 2023-10-18T09:00:38.729 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.729 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.729 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 246.00 MB 2023-10-18T09:00:38.729 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.729 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.729 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 247.00 MB 2023-10-18T09:00:38.730 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.730 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.730 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 248.00 MB 2023-10-18T09:00:38.730 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.731 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.731 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 249.00 MB 2023-10-18T09:00:38.731 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.731 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.731 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 250.00 MB 2023-10-18T09:00:38.732 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.732 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.732 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 251.00 MB 2023-10-18T09:00:38.732 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.732 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.732 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 252.00 MB 2023-10-18T09:00:38.733 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.733 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.733 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 253.00 MB 2023-10-18T09:00:38.733 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.733 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.734 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 254.00 MB 2023-10-18T09:00:38.734 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.734 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.734 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 255.00 MB 2023-10-18T09:00:38.734 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.735 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.735 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 256.00 MB 2023-10-18T09:00:38.735 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.735 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.735 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 257.00 MB 2023-10-18T09:00:38.735 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.736 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.736 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 258.00 MB 2023-10-18T09:00:38.736 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.736 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.736 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 259.00 MB 2023-10-18T09:00:38.737 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.737 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.737 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 260.00 MB 2023-10-18T09:00:38.737 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.737 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.738 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 261.00 MB 2023-10-18T09:00:38.738 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.738 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.738 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 262.00 MB 2023-10-18T09:00:38.738 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.739 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.739 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 263.00 MB 2023-10-18T09:00:38.739 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.739 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.739 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 264.00 MB 2023-10-18T09:00:38.740 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.740 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.740 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 265.00 MB 2023-10-18T09:00:38.740 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.740 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.740 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 266.00 MB 2023-10-18T09:00:38.741 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.741 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.741 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 267.00 MB 2023-10-18T09:00:38.741 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.741 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.742 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 268.00 MB 2023-10-18T09:00:38.742 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.742 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.742 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 269.00 MB 2023-10-18T09:00:38.742 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.743 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.743 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 270.00 MB 2023-10-18T09:00:38.743 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.743 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.743 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 271.00 MB 2023-10-18T09:00:38.743 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.744 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.744 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 272.00 MB 2023-10-18T09:00:38.744 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.744 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.744 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 273.00 MB 2023-10-18T09:00:38.745 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.745 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.745 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 274.00 MB 2023-10-18T09:00:38.745 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.745 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.745 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 275.00 MB 2023-10-18T09:00:38.746 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.746 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.746 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 276.00 MB 2023-10-18T09:00:38.747 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.747 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.747 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 277.00 MB 2023-10-18T09:00:38.747 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.747 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.748 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 278.00 MB 2023-10-18T09:00:38.748 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.748 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.748 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 279.00 MB 2023-10-18T09:00:38.748 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.748 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.749 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 280.00 MB 2023-10-18T09:00:38.749 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.749 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.749 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 281.00 MB 2023-10-18T09:00:38.750 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.750 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.750 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 282.00 MB 2023-10-18T09:00:38.750 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.751 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.751 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 283.00 MB 2023-10-18T09:00:38.751 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.751 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.751 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 284.00 MB 2023-10-18T09:00:38.752 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.752 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.752 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 285.00 MB 2023-10-18T09:00:38.752 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.752 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.753 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 286.00 MB 2023-10-18T09:00:38.753 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.753 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.753 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 287.00 MB 2023-10-18T09:00:38.753 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.754 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.754 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 288.00 MB 2023-10-18T09:00:38.754 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.754 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.754 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 289.00 MB 2023-10-18T09:00:38.755 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.755 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.755 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 290.00 MB 2023-10-18T09:00:38.755 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.755 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.756 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 291.00 MB 2023-10-18T09:00:38.756 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.756 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.756 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 292.00 MB 2023-10-18T09:00:38.756 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.757 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.757 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 293.00 MB 2023-10-18T09:00:38.757 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.757 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.757 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 294.00 MB 2023-10-18T09:00:38.758 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.758 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.758 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 295.00 MB 2023-10-18T09:00:38.758 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.759 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.759 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 296.00 MB 2023-10-18T09:00:38.759 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.759 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.759 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 297.00 MB 2023-10-18T09:00:38.760 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.760 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.760 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 298.00 MB 2023-10-18T09:00:38.760 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.760 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.761 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 299.00 MB 2023-10-18T09:00:38.761 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.761 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.761 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 300.00 MB 2023-10-18T09:00:38.761 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.762 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.762 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 301.00 MB 2023-10-18T09:00:38.762 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.762 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.762 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 302.00 MB 2023-10-18T09:00:38.763 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.763 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.763 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 303.00 MB 2023-10-18T09:00:38.763 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.763 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.764 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 304.00 MB 2023-10-18T09:00:38.764 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.764 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.764 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 305.00 MB 2023-10-18T09:00:38.765 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.765 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.765 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 306.00 MB 2023-10-18T09:00:38.765 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.766 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.766 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 307.00 MB 2023-10-18T09:00:38.766 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.766 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.766 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 308.00 MB 2023-10-18T09:00:38.767 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.767 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.767 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 309.00 MB 2023-10-18T09:00:38.767 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.768 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.768 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 310.00 MB 2023-10-18T09:00:38.768 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.768 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.768 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 311.00 MB 2023-10-18T09:00:38.769 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.769 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.769 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 312.00 MB 2023-10-18T09:00:38.769 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.769 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.769 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 313.00 MB 2023-10-18T09:00:38.770 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.770 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.770 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 314.00 MB 2023-10-18T09:00:38.770 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.770 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.771 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 315.00 MB 2023-10-18T09:00:38.771 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.771 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.771 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 316.00 MB 2023-10-18T09:00:38.771 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.772 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.772 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 317.00 MB 2023-10-18T09:00:38.772 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.772 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.772 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 318.00 MB 2023-10-18T09:00:38.773 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.773 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.773 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 319.00 MB 2023-10-18T09:00:38.773 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.773 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.774 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 320.00 MB 2023-10-18T09:00:38.774 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:38.774 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:38.774 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 321.00 MB 2023-10-18T09:00:39.621 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.621 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.621 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 322.00 MB 2023-10-18T09:00:39.621 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 323.00 MB 2023-10-18T09:00:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.622 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 324.00 MB 2023-10-18T09:00:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 325.00 MB 2023-10-18T09:00:39.623 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.624 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.624 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 326.00 MB 2023-10-18T09:00:39.624 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.624 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.624 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 327.00 MB 2023-10-18T09:00:39.624 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.625 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.625 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 328.00 MB 2023-10-18T09:00:39.625 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.625 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.625 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 329.00 MB 2023-10-18T09:00:39.626 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.626 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.626 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 330.00 MB 2023-10-18T09:00:39.626 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.626 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.627 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 331.00 MB 2023-10-18T09:00:39.627 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.627 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.627 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 332.00 MB 2023-10-18T09:00:39.628 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.628 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.628 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 333.00 MB 2023-10-18T09:00:39.628 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.628 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.628 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 334.00 MB 2023-10-18T09:00:39.629 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.629 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.629 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 335.00 MB 2023-10-18T09:00:39.629 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.629 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.629 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 336.00 MB 2023-10-18T09:00:39.630 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.630 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.630 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 337.00 MB 2023-10-18T09:00:39.630 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.630 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.631 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 338.00 MB 2023-10-18T09:00:39.631 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.631 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.631 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 339.00 MB 2023-10-18T09:00:39.631 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.632 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.632 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 340.00 MB 2023-10-18T09:00:39.632 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.632 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.633 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 341.00 MB 2023-10-18T09:00:39.633 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.633 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.633 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 342.00 MB 2023-10-18T09:00:39.633 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.634 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.634 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 343.00 MB 2023-10-18T09:00:39.634 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.634 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.635 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 344.00 MB 2023-10-18T09:00:39.635 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.635 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.635 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 345.00 MB 2023-10-18T09:00:39.635 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.636 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.636 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 346.00 MB 2023-10-18T09:00:39.636 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.636 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.636 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 347.00 MB 2023-10-18T09:00:39.637 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.637 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.637 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 348.00 MB 2023-10-18T09:00:39.637 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.637 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.638 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 349.00 MB 2023-10-18T09:00:39.638 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.638 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.638 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 350.00 MB 2023-10-18T09:00:39.639 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.639 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.639 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 351.00 MB 2023-10-18T09:00:39.639 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.639 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.640 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 352.00 MB 2023-10-18T09:00:39.640 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.640 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.640 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 353.00 MB 2023-10-18T09:00:39.640 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.641 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.641 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 354.00 MB 2023-10-18T09:00:39.641 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.641 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.641 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 355.00 MB 2023-10-18T09:00:39.642 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.642 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.642 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 356.00 MB 2023-10-18T09:00:39.642 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.642 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.643 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 357.00 MB 2023-10-18T09:00:39.643 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.643 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.643 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 358.00 MB 2023-10-18T09:00:39.643 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.644 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.644 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 359.00 MB 2023-10-18T09:00:39.644 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.644 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.644 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 360.00 MB 2023-10-18T09:00:39.645 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.645 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.645 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 361.00 MB 2023-10-18T09:00:39.645 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.645 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.645 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 362.00 MB 2023-10-18T09:00:39.646 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.646 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.646 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 363.00 MB 2023-10-18T09:00:39.646 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.646 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.647 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 364.00 MB 2023-10-18T09:00:39.647 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.647 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.647 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 365.00 MB 2023-10-18T09:00:39.647 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.648 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.648 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 366.00 MB 2023-10-18T09:00:39.648 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.648 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.648 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 367.00 MB 2023-10-18T09:00:39.649 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.649 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.649 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 368.00 MB 2023-10-18T09:00:39.649 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.649 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.649 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 369.00 MB 2023-10-18T09:00:39.650 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.650 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.650 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 370.00 MB 2023-10-18T09:00:39.650 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.650 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.651 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 371.00 MB 2023-10-18T09:00:39.651 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.651 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.651 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 372.00 MB 2023-10-18T09:00:39.651 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.652 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.652 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 373.00 MB 2023-10-18T09:00:39.652 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.652 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.652 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 374.00 MB 2023-10-18T09:00:39.653 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.653 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.653 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 375.00 MB 2023-10-18T09:00:39.653 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.653 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.653 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 376.00 MB 2023-10-18T09:00:39.654 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.654 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.654 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 377.00 MB 2023-10-18T09:00:39.654 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.654 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.655 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 378.00 MB 2023-10-18T09:00:39.655 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.655 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.655 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 379.00 MB 2023-10-18T09:00:39.655 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.655 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.656 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 380.00 MB 2023-10-18T09:00:39.656 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.656 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.656 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 381.00 MB 2023-10-18T09:00:39.656 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.657 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.657 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 382.00 MB 2023-10-18T09:00:39.657 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.657 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.657 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 383.00 MB 2023-10-18T09:00:39.658 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.658 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.658 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 384.00 MB 2023-10-18T09:00:39.658 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.658 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.659 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 385.00 MB 2023-10-18T09:00:39.659 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.659 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.659 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 386.00 MB 2023-10-18T09:00:39.659 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.659 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.660 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 387.00 MB 2023-10-18T09:00:39.660 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.660 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.660 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 388.00 MB 2023-10-18T09:00:39.660 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.661 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.661 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 389.00 MB 2023-10-18T09:00:39.661 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.661 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.661 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 390.00 MB 2023-10-18T09:00:39.661 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.662 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.662 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 391.00 MB 2023-10-18T09:00:39.662 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.662 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.662 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 392.00 MB 2023-10-18T09:00:39.663 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.663 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.663 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 393.00 MB 2023-10-18T09:00:39.663 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.663 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.665 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 394.00 MB 2023-10-18T09:00:39.665 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.665 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.665 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 395.00 MB 2023-10-18T09:00:39.666 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.666 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.666 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 396.00 MB 2023-10-18T09:00:39.666 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.666 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.667 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 397.00 MB 2023-10-18T09:00:39.667 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.667 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.667 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 398.00 MB 2023-10-18T09:00:39.667 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.667 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.668 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 399.00 MB 2023-10-18T09:00:39.668 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.668 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.668 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 400.00 MB 2023-10-18T09:00:39.668 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:00:39.668 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:00:39.669 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 401.00 MB 2023-10-18T09:00:42.981 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:00:42.977+0000 7f6a11f15640 -1 log_channel(cluster) log [ERR] : Health check failed: 3 full osd(s) (OSD_FULL) 2023-10-18T09:00:50.967 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:00:50.964+0000 7f6a11f15640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-10-18T09:01:05.078 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:01:05.075+0000 7f6a11f15640 -1 log_channel(cluster) log [ERR] : Health check failed: Full OSDs blocking recovery: 1 pg recovery_toofull (PG_RECOVERY_FULL) 2023-10-18T09:01:05.770 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:01:05.781 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:01:10.566 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.567 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.567 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 402.00 MB 2023-10-18T09:01:10.567 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.567 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.567 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 403.00 MB 2023-10-18T09:01:10.568 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.568 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.568 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 404.00 MB 2023-10-18T09:01:10.568 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.568 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.569 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 405.00 MB 2023-10-18T09:01:10.569 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.569 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.569 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 406.00 MB 2023-10-18T09:01:10.569 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.569 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.570 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 407.00 MB 2023-10-18T09:01:10.570 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.570 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.570 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 408.00 MB 2023-10-18T09:01:10.570 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.571 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.571 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 409.00 MB 2023-10-18T09:01:10.571 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.571 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.571 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 410.00 MB 2023-10-18T09:01:10.572 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.572 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.572 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 411.00 MB 2023-10-18T09:01:10.572 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.572 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.573 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 412.00 MB 2023-10-18T09:01:10.573 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.573 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.573 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 413.00 MB 2023-10-18T09:01:10.573 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.573 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.574 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 414.00 MB 2023-10-18T09:01:10.574 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.574 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.574 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 415.00 MB 2023-10-18T09:01:10.574 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.575 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.575 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 416.00 MB 2023-10-18T09:01:10.575 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.575 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.575 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 417.00 MB 2023-10-18T09:01:10.576 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.576 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.576 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 418.00 MB 2023-10-18T09:01:10.576 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.576 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.577 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 419.00 MB 2023-10-18T09:01:10.577 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.577 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.577 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 420.00 MB 2023-10-18T09:01:10.577 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.578 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.578 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 421.00 MB 2023-10-18T09:01:10.578 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.578 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.578 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 422.00 MB 2023-10-18T09:01:10.579 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.579 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.579 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 423.00 MB 2023-10-18T09:01:10.579 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.579 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.580 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 424.00 MB 2023-10-18T09:01:10.580 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.580 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.580 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 425.00 MB 2023-10-18T09:01:10.580 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.581 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.581 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 426.00 MB 2023-10-18T09:01:10.581 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.581 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.581 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 427.00 MB 2023-10-18T09:01:10.581 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.582 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.582 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 428.00 MB 2023-10-18T09:01:10.582 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.582 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.582 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 429.00 MB 2023-10-18T09:01:10.583 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.583 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.583 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 430.00 MB 2023-10-18T09:01:10.583 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.583 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.583 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 431.00 MB 2023-10-18T09:01:10.584 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.584 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.584 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 432.00 MB 2023-10-18T09:01:10.584 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.584 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.585 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 433.00 MB 2023-10-18T09:01:10.585 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.585 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.585 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 434.00 MB 2023-10-18T09:01:10.585 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.586 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.586 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 435.00 MB 2023-10-18T09:01:10.586 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.586 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.586 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 436.00 MB 2023-10-18T09:01:10.586 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.587 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.587 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 437.00 MB 2023-10-18T09:01:10.587 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.587 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.587 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 438.00 MB 2023-10-18T09:01:10.587 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.588 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.588 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 439.00 MB 2023-10-18T09:01:10.588 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.588 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.588 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 440.00 MB 2023-10-18T09:01:10.589 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.589 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.589 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 441.00 MB 2023-10-18T09:01:10.589 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.589 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.590 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 442.00 MB 2023-10-18T09:01:10.590 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.590 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.590 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 443.00 MB 2023-10-18T09:01:10.590 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.591 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.591 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 444.00 MB 2023-10-18T09:01:10.591 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.591 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.591 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 445.00 MB 2023-10-18T09:01:10.591 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.592 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.592 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 446.00 MB 2023-10-18T09:01:10.592 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.592 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.592 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 447.00 MB 2023-10-18T09:01:10.593 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.593 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.593 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 448.00 MB 2023-10-18T09:01:10.593 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.593 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.594 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 449.00 MB 2023-10-18T09:01:10.594 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.594 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.594 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 450.00 MB 2023-10-18T09:01:10.594 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.594 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.595 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 451.00 MB 2023-10-18T09:01:10.595 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.595 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.595 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 452.00 MB 2023-10-18T09:01:10.595 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.596 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.596 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 453.00 MB 2023-10-18T09:01:10.596 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.596 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.596 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 454.00 MB 2023-10-18T09:01:10.597 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.597 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.597 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 455.00 MB 2023-10-18T09:01:10.597 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.597 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.598 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 456.00 MB 2023-10-18T09:01:10.598 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.598 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.598 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 457.00 MB 2023-10-18T09:01:10.598 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.599 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.599 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 458.00 MB 2023-10-18T09:01:10.599 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.599 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.599 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 459.00 MB 2023-10-18T09:01:10.599 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.600 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.600 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 460.00 MB 2023-10-18T09:01:10.600 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.600 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.600 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 461.00 MB 2023-10-18T09:01:10.601 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.601 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.601 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 462.00 MB 2023-10-18T09:01:10.601 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.601 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.602 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 463.00 MB 2023-10-18T09:01:10.602 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.602 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.602 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 464.00 MB 2023-10-18T09:01:10.602 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.603 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.603 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 465.00 MB 2023-10-18T09:01:10.603 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.603 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.603 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 466.00 MB 2023-10-18T09:01:10.603 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.604 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.604 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 467.00 MB 2023-10-18T09:01:10.604 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.604 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.604 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 468.00 MB 2023-10-18T09:01:10.604 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.605 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.605 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 469.00 MB 2023-10-18T09:01:10.605 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.605 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.605 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 470.00 MB 2023-10-18T09:01:10.605 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.606 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.606 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 471.00 MB 2023-10-18T09:01:10.606 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.606 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.607 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 472.00 MB 2023-10-18T09:01:10.607 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.607 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.607 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 473.00 MB 2023-10-18T09:01:10.607 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.608 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.608 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 474.00 MB 2023-10-18T09:01:10.608 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.608 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.608 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 475.00 MB 2023-10-18T09:01:10.609 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.609 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.609 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 476.00 MB 2023-10-18T09:01:10.609 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.609 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.610 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 477.00 MB 2023-10-18T09:01:10.610 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.610 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.610 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 478.00 MB 2023-10-18T09:01:10.611 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.611 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.611 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 479.00 MB 2023-10-18T09:01:10.611 INFO:teuthology.orchestra.run.smithi040.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-18T09:01:10.611 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:01:10.612 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:01:10.612 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 480.00 MB 2023-10-18T09:01:10.612 INFO:teuthology.orchestra.run.smithi040.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-18T09:01:10.706 INFO:teuthology.orchestra.run.smithi040.stdout:Reached fullness after 480.00 MB 2023-10-18T09:01:10.706 INFO:teuthology.orchestra.run.smithi040.stdout:calling close 2023-10-18T09:01:10.706 INFO:teuthology.orchestra.run.smithi040.stdout:close() did not raise error 2023-10-18T09:01:10.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T09:01:10.734 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T09:01:10.735 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:01:10.853 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:01:10.974 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:01:10.972+0000 7f6a11f15640 -1 log_channel(cluster) log [ERR] : Health check update: Full OSDs blocking recovery: 2 pgs recovery_toofull (PG_RECOVERY_FULL) 2023-10-18T09:01:10.976 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T09:01:10.976 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:01:10.977 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:01:11.040 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:01:11.041 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:01:11.096 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:01:11.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.326+0000 7f95597b6640 1 -- 172.21.15.40:0/1347780192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9540003f40 msgr2=0x7f95400047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:11.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.326+0000 7f95597b6640 1 --2- 172.21.15.40:0/1347780192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9540003f40 0x7f95400047c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f953c00ac90 tx=0x7f95400057f0 comp rx=0 tx=0).stop 2023-10-18T09:01:11.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.326+0000 7f95537fe640 1 -- 172.21.15.40:0/1347780192 reap_dead start 2023-10-18T09:01:11.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.326+0000 7f95597b6640 1 -- 172.21.15.40:0/1347780192 shutdown_connections 2023-10-18T09:01:11.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.326+0000 7f95597b6640 1 -- 172.21.15.40:0/1347780192 >> 172.21.15.40:0/1347780192 conn(0x7f955406cee0 msgr2=0x7f9554069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:11.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.326+0000 7f95597b6640 1 -- 172.21.15.40:0/1347780192 shutdown_connections 2023-10-18T09:01:11.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.326+0000 7f95597b6640 1 -- 172.21.15.40:0/1347780192 wait complete. 2023-10-18T09:01:11.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.328+0000 7f95597b6640 1 Processor -- start 2023-10-18T09:01:11.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.328+0000 7f95597b6640 1 -- start start 2023-10-18T09:01:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.329+0000 7f95597b6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9540003f40 0x7f95540cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:11.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.329+0000 7f95597b6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f955414a1c0 0x7f9554149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:11.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.329+0000 7f95597b6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9554070a80 0x7f9554070f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:11.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.329+0000 7f95597b6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9554071440 con 0x7f9540003f40 2023-10-18T09:01:11.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.329+0000 7f95597b6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9554155fe0 con 0x7f9554070a80 2023-10-18T09:01:11.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.329+0000 7f95597b6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9554156230 con 0x7f955414a1c0 2023-10-18T09:01:11.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.329+0000 7f9552ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9540003f40 0x7f95540cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:11.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.329+0000 7f9552ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9540003f40 0x7f95540cbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60880/0 (socket says 172.21.15.40:60880) 2023-10-18T09:01:11.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.329+0000 7f9552ffd640 1 -- 172.21.15.40:0/1439336755 learned_addr learned my addr 172.21.15.40:0/1439336755 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:11.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.330+0000 7f9552ffd640 1 -- 172.21.15.40:0/1439336755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f955414a1c0 msgr2=0x7f9554149790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.330+0000 7f9552ffd640 1 --2- 172.21.15.40:0/1439336755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f955414a1c0 0x7f9554149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.330+0000 7f9552ffd640 1 -- 172.21.15.40:0/1439336755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9554070a80 msgr2=0x7f9554070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.330+0000 7f9552ffd640 1 --2- 172.21.15.40:0/1439336755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9554070a80 0x7f9554070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.330+0000 7f9552ffd640 1 -- 172.21.15.40:0/1439336755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9554156510 con 0x7f9540003f40 2023-10-18T09:01:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.330+0000 7f9552ffd640 1 --2- 172.21.15.40:0/1439336755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9540003f40 0x7f95540cbbe0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f954400e850 tx=0x7f954400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.330+0000 7f954b7fe640 1 -- 172.21.15.40:0/1439336755 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954400cbd0 con 0x7f9540003f40 2023-10-18T09:01:11.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.330+0000 7f954b7fe640 1 -- 172.21.15.40:0/1439336755 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f954400ce40 con 0x7f9540003f40 2023-10-18T09:01:11.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.330+0000 7f954b7fe640 1 -- 172.21.15.40:0/1439336755 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95440116c0 con 0x7f9540003f40 2023-10-18T09:01:11.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.330+0000 7f95597b6640 1 -- 172.21.15.40:0/1439336755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95541567f0 con 0x7f9540003f40 2023-10-18T09:01:11.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.331+0000 7f95597b6640 1 -- 172.21.15.40:0/1439336755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9554149cd0 con 0x7f9540003f40 2023-10-18T09:01:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.332+0000 7f95597b6640 1 -- 172.21.15.40:0/1439336755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955406ff20 con 0x7f9540003f40 2023-10-18T09:01:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.333+0000 7f954b7fe640 1 -- 172.21.15.40:0/1439336755 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9544011820 con 0x7f9540003f40 2023-10-18T09:01:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.334+0000 7f954b7fe640 1 --2- 172.21.15.40:0/1439336755 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f953406ae40 0x7f953406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.334+0000 7f954b7fe640 1 -- 172.21.15.40:0/1439336755 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f954408a1b0 con 0x7f9540003f40 2023-10-18T09:01:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.334+0000 7f954b7fe640 1 -- 172.21.15.40:0/1439336755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=93}) v3 -- 0x7f9534072b50 con 0x7f9540003f40 2023-10-18T09:01:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.334+0000 7f95527fc640 1 --2- 172.21.15.40:0/1439336755 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f953406ae40 0x7f953406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:11.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.336+0000 7f95527fc640 1 --2- 172.21.15.40:0/1439336755 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f953406ae40 0x7f953406d300 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f953c038000 tx=0x7f953c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:11.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.338+0000 7f954b7fe640 1 -- 172.21.15.40:0/1439336755 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9544015060 con 0x7f9540003f40 2023-10-18T09:01:11.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.538+0000 7f95597b6640 1 -- 172.21.15.40:0/1439336755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f955415b010 con 0x7f9540003f40 2023-10-18T09:01:11.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.540+0000 7f954b7fe640 1 -- 172.21.15.40:0/1439336755 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16863 (secure 0 0 0) 0x7f954401f030 con 0x7f9540003f40 2023-10-18T09:01:11.544 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:11.545 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":92,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:01:02.983030+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:00:13.170937+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-18T09:00:14.205418+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1","mappings":[{"from":6,"to":0}]},{"pgid":"11.8","mappings":[{"from":6,"to":1}]},{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.18","mappings":[{"from":2,"to":0}]},{"pgid":"11.33","mappings":[{"from":6,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:01:11.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.543+0000 7f95597b6640 1 -- 172.21.15.40:0/1439336755 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f953406ae40 msgr2=0x7f953406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:11.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.543+0000 7f95597b6640 1 --2- 172.21.15.40:0/1439336755 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f953406ae40 0x7f953406d300 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f953c038000 tx=0x7f953c004730 comp rx=0 tx=0).stop 2023-10-18T09:01:11.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.543+0000 7f95597b6640 1 -- 172.21.15.40:0/1439336755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9540003f40 msgr2=0x7f95540cbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:11.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.543+0000 7f95597b6640 1 --2- 172.21.15.40:0/1439336755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9540003f40 0x7f95540cbbe0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f954400e850 tx=0x7f954400ed10 comp rx=0 tx=0).stop 2023-10-18T09:01:11.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.543+0000 7f95597b6640 1 -- 172.21.15.40:0/1439336755 shutdown_connections 2023-10-18T09:01:11.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.543+0000 7f95597b6640 1 --2- 172.21.15.40:0/1439336755 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f953406ae40 0x7f953406d300 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:11.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.543+0000 7f95597b6640 1 --2- 172.21.15.40:0/1439336755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9554070a80 0x7f9554070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:11.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.543+0000 7f95597b6640 1 --2- 172.21.15.40:0/1439336755 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f955414a1c0 0x7f9554149790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:11.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.543+0000 7f95597b6640 1 --2- 172.21.15.40:0/1439336755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9540003f40 0x7f95540cbbe0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:11.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.543+0000 7f95597b6640 1 -- 172.21.15.40:0/1439336755 >> 172.21.15.40:0/1439336755 conn(0x7f955406cee0 msgr2=0x7f9554070150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:11.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.543+0000 7f95597b6640 1 -- 172.21.15.40:0/1439336755 shutdown_connections 2023-10-18T09:01:11.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.543+0000 7f95597b6640 1 -- 172.21.15.40:0/1439336755 wait complete. 2023-10-18T09:01:11.566 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:00:13.170937+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:00:14.205418+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:01:11.567 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:01:11.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.806+0000 7f391ee7c640 1 -- 172.21.15.40:0/2283450546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39180715c0 msgr2=0x7f39180b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:11.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.806+0000 7f391ee7c640 1 --2- 172.21.15.40:0/2283450546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39180715c0 0x7f39180b1950 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f391806a480 tx=0x7f390402bd90 comp rx=0 tx=0).stop 2023-10-18T09:01:11.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.806+0000 7f391ee7c640 1 -- 172.21.15.40:0/2283450546 shutdown_connections 2023-10-18T09:01:11.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.806+0000 7f391ee7c640 1 --2- 172.21.15.40:0/2283450546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39180715c0 0x7f39180b1950 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:11.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.806+0000 7f391ee7c640 1 --2- 172.21.15.40:0/2283450546 >> v2:172.21.15.144:3300/0 conn(0x7f391806d6f0 0x7f391814e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:11.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.806+0000 7f391ee7c640 1 -- 172.21.15.40:0/2283450546 >> 172.21.15.40:0/2283450546 conn(0x7f3918069720 msgr2=0x7f3918069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:11.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.806+0000 7f391ee7c640 1 -- 172.21.15.40:0/2283450546 shutdown_connections 2023-10-18T09:01:11.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.806+0000 7f391ee7c640 1 -- 172.21.15.40:0/2283450546 wait complete. 2023-10-18T09:01:11.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.808+0000 7f391ee7c640 1 Processor -- start 2023-10-18T09:01:11.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.808+0000 7f391ee7c640 1 -- start start 2023-10-18T09:01:11.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.809+0000 7f391ee7c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f391806d6f0 0x7f39181e3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:11.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.809+0000 7f391ee7c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39180715c0 0x7f3918151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:11.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.809+0000 7f391ee7c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39180aed00 0x7f391814cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:11.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.809+0000 7f391ee7c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39181e3ff0 con 0x7f391806d6f0 2023-10-18T09:01:11.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.809+0000 7f391ee7c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391814bca0 con 0x7f39180715c0 2023-10-18T09:01:11.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.809+0000 7f391ee7c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391814bf80 con 0x7f39180aed00 2023-10-18T09:01:11.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.809+0000 7f391cc18640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f391806d6f0 0x7f39181e3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:11.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.809+0000 7f391cc18640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f391806d6f0 0x7f39181e3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60904/0 (socket says 172.21.15.40:60904) 2023-10-18T09:01:11.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.809+0000 7f391cc18640 1 -- 172.21.15.40:0/2132714567 learned_addr learned my addr 172.21.15.40:0/2132714567 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:11.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.810+0000 7f391d419640 1 --2- 172.21.15.40:0/2132714567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39180aed00 0x7f391814cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:11.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.810+0000 7f3917fff640 1 --2- 172.21.15.40:0/2132714567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39180715c0 0x7f3918151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:11.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.810+0000 7f391cc18640 1 -- 172.21.15.40:0/2132714567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39180aed00 msgr2=0x7f391814cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:11.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.810+0000 7f391cc18640 1 --2- 172.21.15.40:0/2132714567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39180aed00 0x7f391814cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:11.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.810+0000 7f391cc18640 1 -- 172.21.15.40:0/2132714567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39180715c0 msgr2=0x7f3918151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:11.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.810+0000 7f391cc18640 1 --2- 172.21.15.40:0/2132714567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39180715c0 0x7f3918151690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:11.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.810+0000 7f391cc18640 1 -- 172.21.15.40:0/2132714567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3904009560 con 0x7f391806d6f0 2023-10-18T09:01:11.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.810+0000 7f391d419640 1 --2- 172.21.15.40:0/2132714567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39180aed00 0x7f391814cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:11.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.810+0000 7f391cc18640 1 --2- 172.21.15.40:0/2132714567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f391806d6f0 0x7f39181e3ab0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f390c00b4a0 tx=0x7f390c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:11.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.810+0000 7f39157fa640 1 -- 172.21.15.40:0/2132714567 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c010610 con 0x7f391806d6f0 2023-10-18T09:01:11.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.810+0000 7f39157fa640 1 -- 172.21.15.40:0/2132714567 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f390c010860 con 0x7f391806d6f0 2023-10-18T09:01:11.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.810+0000 7f391ee7c640 1 -- 172.21.15.40:0/2132714567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391814c260 con 0x7f391806d6f0 2023-10-18T09:01:11.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.810+0000 7f39157fa640 1 -- 172.21.15.40:0/2132714567 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c010c70 con 0x7f391806d6f0 2023-10-18T09:01:11.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.811+0000 7f391ee7c640 1 -- 172.21.15.40:0/2132714567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391814c820 con 0x7f391806d6f0 2023-10-18T09:01:11.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.812+0000 7f391ee7c640 1 -- 172.21.15.40:0/2132714567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3918066fd0 con 0x7f391806d6f0 2023-10-18T09:01:11.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.813+0000 7f39157fa640 1 -- 172.21.15.40:0/2132714567 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f390c017360 con 0x7f391806d6f0 2023-10-18T09:01:11.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.814+0000 7f39157fa640 1 --2- 172.21.15.40:0/2132714567 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f38fc06ae40 0x7f38fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:11.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.814+0000 7f39157fa640 1 -- 172.21.15.40:0/2132714567 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f390c0895a0 con 0x7f391806d6f0 2023-10-18T09:01:11.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.814+0000 7f39157fa640 1 -- 172.21.15.40:0/2132714567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=93}) v3 -- 0x7f38fc072b50 con 0x7f391806d6f0 2023-10-18T09:01:11.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.814+0000 7f3917fff640 1 --2- 172.21.15.40:0/2132714567 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f38fc06ae40 0x7f38fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:11.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.816+0000 7f3917fff640 1 --2- 172.21.15.40:0/2132714567 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f38fc06ae40 0x7f38fc06d300 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3904035000 tx=0x7f3904004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:11.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:11.819+0000 7f39157fa640 1 -- 172.21.15.40:0/2132714567 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f390c0545b0 con 0x7f391806d6f0 2023-10-18T09:01:12.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.014+0000 7f391ee7c640 1 -- 172.21.15.40:0/2132714567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39180b2350 con 0x7f391806d6f0 2023-10-18T09:01:12.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.015+0000 7f39157fa640 1 -- 172.21.15.40:0/2132714567 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f390c0140a0 con 0x7f391806d6f0 2023-10-18T09:01:12.019 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:01:12.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.018+0000 7f391ee7c640 1 -- 172.21.15.40:0/2132714567 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f38fc06ae40 msgr2=0x7f38fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:12.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.018+0000 7f391ee7c640 1 --2- 172.21.15.40:0/2132714567 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f38fc06ae40 0x7f38fc06d300 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3904035000 tx=0x7f3904004730 comp rx=0 tx=0).stop 2023-10-18T09:01:12.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.018+0000 7f391ee7c640 1 -- 172.21.15.40:0/2132714567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f391806d6f0 msgr2=0x7f39181e3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:12.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.018+0000 7f391ee7c640 1 --2- 172.21.15.40:0/2132714567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f391806d6f0 0x7f39181e3ab0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f390c00b4a0 tx=0x7f390c00b960 comp rx=0 tx=0).stop 2023-10-18T09:01:12.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.019+0000 7f391ee7c640 1 -- 172.21.15.40:0/2132714567 shutdown_connections 2023-10-18T09:01:12.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.019+0000 7f391ee7c640 1 --2- 172.21.15.40:0/2132714567 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f38fc06ae40 0x7f38fc06d300 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.019+0000 7f391ee7c640 1 --2- 172.21.15.40:0/2132714567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39180aed00 0x7f391814cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.019+0000 7f391ee7c640 1 --2- 172.21.15.40:0/2132714567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39180715c0 0x7f3918151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.019+0000 7f391ee7c640 1 --2- 172.21.15.40:0/2132714567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f391806d6f0 0x7f39181e3ab0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.019+0000 7f391ee7c640 1 -- 172.21.15.40:0/2132714567 >> 172.21.15.40:0/2132714567 conn(0x7f3918069720 msgr2=0x7f39180ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:12.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.019+0000 7f391ee7c640 1 -- 172.21.15.40:0/2132714567 shutdown_connections 2023-10-18T09:01:12.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.019+0000 7f391ee7c640 1 -- 172.21.15.40:0/2132714567 wait complete. 2023-10-18T09:01:12.042 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:01:12.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.275+0000 7f6e1cbfc640 1 -- 172.21.15.40:0/918749876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e1815a620 msgr2=0x7f6e180cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:12.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.275+0000 7f6e1cbfc640 1 --2- 172.21.15.40:0/918749876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e1815a620 0x7f6e180cae80 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f6e180caa30 tx=0x7f6e0002bd90 comp rx=0 tx=0).stop 2023-10-18T09:01:12.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.276+0000 7f6e1cbfc640 1 -- 172.21.15.40:0/918749876 shutdown_connections 2023-10-18T09:01:12.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.276+0000 7f6e1cbfc640 1 --2- 172.21.15.40:0/918749876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e180cb3c0 0x7f6e1815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.276+0000 7f6e1cbfc640 1 --2- 172.21.15.40:0/918749876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e1815a620 0x7f6e180cae80 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.276+0000 7f6e1cbfc640 1 --2- 172.21.15.40:0/918749876 >> v2:172.21.15.144:3300/0 conn(0x7f6e1813c320 0x7f6e1815a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.276+0000 7f6e1cbfc640 1 -- 172.21.15.40:0/918749876 >> 172.21.15.40:0/918749876 conn(0x7f6e18069720 msgr2=0x7f6e18069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:12.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.276+0000 7f6e1cbfc640 1 -- 172.21.15.40:0/918749876 shutdown_connections 2023-10-18T09:01:12.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.276+0000 7f6e1cbfc640 1 -- 172.21.15.40:0/918749876 wait complete. 2023-10-18T09:01:12.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.278+0000 7f6e1cbfc640 1 Processor -- start 2023-10-18T09:01:12.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.279+0000 7f6e1cbfc640 1 -- start start 2023-10-18T09:01:12.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.280+0000 7f6e1cbfc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e180cb3c0 0x7f6e180cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:12.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.280+0000 7f6e1cbfc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e1813c320 0x7f6e1813e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:12.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.280+0000 7f6e1cbfc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e1815a620 0x7f6e1813ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:12.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.280+0000 7f6e1cbfc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e181337d0 con 0x7f6e180cb3c0 2023-10-18T09:01:12.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.280+0000 7f6e1cbfc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e180adb10 con 0x7f6e1815a620 2023-10-18T09:01:12.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.280+0000 7f6e1cbfc640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e180addf0 con 0x7f6e1813c320 2023-10-18T09:01:12.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.280+0000 7f6e1659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e180cb3c0 0x7f6e180cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:12.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.280+0000 7f6e1659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e180cb3c0 0x7f6e180cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60912/0 (socket says 172.21.15.40:60912) 2023-10-18T09:01:12.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.280+0000 7f6e1659c640 1 -- 172.21.15.40:0/1107185889 learned_addr learned my addr 172.21.15.40:0/1107185889 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:12.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.281+0000 7f6e15d9b640 1 --2- 172.21.15.40:0/1107185889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e1813c320 0x7f6e1813e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:12.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.281+0000 7f6e16d9d640 1 --2- 172.21.15.40:0/1107185889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e1815a620 0x7f6e1813ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:12.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.281+0000 7f6e1659c640 1 -- 172.21.15.40:0/1107185889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e1813c320 msgr2=0x7f6e1813e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:12.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.281+0000 7f6e1659c640 1 --2- 172.21.15.40:0/1107185889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e1813c320 0x7f6e1813e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.281+0000 7f6e1659c640 1 -- 172.21.15.40:0/1107185889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e1815a620 msgr2=0x7f6e1813ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:12.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.281+0000 7f6e1659c640 1 --2- 172.21.15.40:0/1107185889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e1815a620 0x7f6e1813ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.281+0000 7f6e1659c640 1 -- 172.21.15.40:0/1107185889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e00009560 con 0x7f6e180cb3c0 2023-10-18T09:01:12.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.281+0000 7f6e16d9d640 1 --2- 172.21.15.40:0/1107185889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e1815a620 0x7f6e1813ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:12.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.281+0000 7f6e1659c640 1 --2- 172.21.15.40:0/1107185889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e180cb3c0 0x7f6e180cc3b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800eab0 tx=0x7f6e0800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:12.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.281+0000 7f6e0effd640 1 -- 172.21.15.40:0/1107185889 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800ce40 con 0x7f6e180cb3c0 2023-10-18T09:01:12.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.282+0000 7f6e0effd640 1 -- 172.21.15.40:0/1107185889 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6e08011560 con 0x7f6e180cb3c0 2023-10-18T09:01:12.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.282+0000 7f6e0effd640 1 -- 172.21.15.40:0/1107185889 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e08011910 con 0x7f6e180cb3c0 2023-10-18T09:01:12.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.282+0000 7f6e1cbfc640 1 -- 172.21.15.40:0/1107185889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e180ae0d0 con 0x7f6e180cb3c0 2023-10-18T09:01:12.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.282+0000 7f6e1cbfc640 1 -- 172.21.15.40:0/1107185889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1813f230 con 0x7f6e180cb3c0 2023-10-18T09:01:12.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.283+0000 7f6e1cbfc640 1 -- 172.21.15.40:0/1107185889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e18066fd0 con 0x7f6e180cb3c0 2023-10-18T09:01:12.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.284+0000 7f6e0effd640 1 -- 172.21.15.40:0/1107185889 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6e08011a70 con 0x7f6e180cb3c0 2023-10-18T09:01:12.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.287+0000 7f6e0effd640 1 --2- 172.21.15.40:0/1107185889 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6df806ae40 0x7f6df806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:12.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.287+0000 7f6e0effd640 1 -- 172.21.15.40:0/1107185889 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6e0808a3a0 con 0x7f6e180cb3c0 2023-10-18T09:01:12.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.287+0000 7f6e0effd640 1 -- 172.21.15.40:0/1107185889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=93}) v3 -- 0x7f6df8072b50 con 0x7f6e180cb3c0 2023-10-18T09:01:12.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.287+0000 7f6e15d9b640 1 --2- 172.21.15.40:0/1107185889 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6df806ae40 0x7f6df806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:12.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.288+0000 7f6e15d9b640 1 --2- 172.21.15.40:0/1107185889 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6df806ae40 0x7f6df806d300 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6e00035000 tx=0x7f6e00004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:12.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.292+0000 7f6e0effd640 1 -- 172.21.15.40:0/1107185889 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e080553b0 con 0x7f6e180cb3c0 2023-10-18T09:01:12.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.488+0000 7f6e1cbfc640 1 -- 172.21.15.40:0/1107185889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e1813c750 con 0x7f6e180cb3c0 2023-10-18T09:01:12.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.489+0000 7f6e0effd640 1 -- 172.21.15.40:0/1107185889 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6e0801f020 con 0x7f6e180cb3c0 2023-10-18T09:01:12.492 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:01:12.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.493+0000 7f6e1cbfc640 1 -- 172.21.15.40:0/1107185889 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6df806ae40 msgr2=0x7f6df806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:12.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.493+0000 7f6e1cbfc640 1 --2- 172.21.15.40:0/1107185889 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6df806ae40 0x7f6df806d300 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6e00035000 tx=0x7f6e00004730 comp rx=0 tx=0).stop 2023-10-18T09:01:12.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.493+0000 7f6e1cbfc640 1 -- 172.21.15.40:0/1107185889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e180cb3c0 msgr2=0x7f6e180cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:12.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.493+0000 7f6e1cbfc640 1 --2- 172.21.15.40:0/1107185889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e180cb3c0 0x7f6e180cc3b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800eab0 tx=0x7f6e0800ef70 comp rx=0 tx=0).stop 2023-10-18T09:01:12.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.493+0000 7f6e1cbfc640 1 -- 172.21.15.40:0/1107185889 shutdown_connections 2023-10-18T09:01:12.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.493+0000 7f6e1cbfc640 1 --2- 172.21.15.40:0/1107185889 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6df806ae40 0x7f6df806d300 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.493+0000 7f6e1cbfc640 1 --2- 172.21.15.40:0/1107185889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e1815a620 0x7f6e1813ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.493+0000 7f6e1cbfc640 1 --2- 172.21.15.40:0/1107185889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e1813c320 0x7f6e1813e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.493+0000 7f6e1cbfc640 1 --2- 172.21.15.40:0/1107185889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e180cb3c0 0x7f6e180cc3b0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.493+0000 7f6e1cbfc640 1 -- 172.21.15.40:0/1107185889 >> 172.21.15.40:0/1107185889 conn(0x7f6e18069720 msgr2=0x7f6e1815cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.493+0000 7f6e1cbfc640 1 -- 172.21.15.40:0/1107185889 shutdown_connections 2023-10-18T09:01:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.493+0000 7f6e1cbfc640 1 -- 172.21.15.40:0/1107185889 wait complete. 2023-10-18T09:01:12.513 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:01:12.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.745+0000 7f6594ec3640 1 -- 172.21.15.40:0/898528392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65900b1de0 msgr2=0x7f65900cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:12.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.745+0000 7f6594ec3640 1 --2- 172.21.15.40:0/898528392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65900b1de0 0x7f65900cbe00 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f659008b520 tx=0x7f658802bd10 comp rx=0 tx=0).stop 2023-10-18T09:01:12.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.745+0000 7f6594ec3640 1 -- 172.21.15.40:0/898528392 shutdown_connections 2023-10-18T09:01:12.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.745+0000 7f6594ec3640 1 --2- 172.21.15.40:0/898528392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65900b1de0 0x7f65900cbe00 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.745+0000 7f6594ec3640 1 --2- 172.21.15.40:0/898528392 >> v2:172.21.15.144:3300/0 conn(0x7f65900ce2d0 0x7f65900affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.745+0000 7f6594ec3640 1 -- 172.21.15.40:0/898528392 >> 172.21.15.40:0/898528392 conn(0x7f659009a580 msgr2=0x7f65900d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:12.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.745+0000 7f6594ec3640 1 -- 172.21.15.40:0/898528392 shutdown_connections 2023-10-18T09:01:12.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.745+0000 7f6594ec3640 1 -- 172.21.15.40:0/898528392 wait complete. 2023-10-18T09:01:12.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.747+0000 7f6594ec3640 1 Processor -- start 2023-10-18T09:01:12.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.747+0000 7f6594ec3640 1 -- start start 2023-10-18T09:01:12.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.748+0000 7f6594ec3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900b1de0 0x7f6590155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:12.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.748+0000 7f6594ec3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65900ce2d0 0x7f65900b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:12.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.748+0000 7f6594ec3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659015ed60 0x7f65901f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:12.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.748+0000 7f6594ec3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6590151180 con 0x7f659015ed60 2023-10-18T09:01:12.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.748+0000 7f6594ec3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65900b42f0 con 0x7f65900ce2d0 2023-10-18T09:01:12.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.748+0000 7f6594ec3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65901f5640 con 0x7f65900b1de0 2023-10-18T09:01:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.749+0000 7f658ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659015ed60 0x7f65901f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.749+0000 7f658ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659015ed60 0x7f65901f5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60930/0 (socket says 172.21.15.40:60930) 2023-10-18T09:01:12.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.749+0000 7f658ffff640 1 -- 172.21.15.40:0/3971358405 learned_addr learned my addr 172.21.15.40:0/3971358405 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:12.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.749+0000 7f658f7fe640 1 --2- 172.21.15.40:0/3971358405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900b1de0 0x7f6590155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:12.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.749+0000 7f658effd640 1 --2- 172.21.15.40:0/3971358405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65900ce2d0 0x7f65900b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:12.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.749+0000 7f658ffff640 1 -- 172.21.15.40:0/3971358405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900b1de0 msgr2=0x7f6590155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:12.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.749+0000 7f658ffff640 1 --2- 172.21.15.40:0/3971358405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900b1de0 0x7f6590155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.749+0000 7f658ffff640 1 -- 172.21.15.40:0/3971358405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65900ce2d0 msgr2=0x7f65900b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:12.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.749+0000 7f658ffff640 1 --2- 172.21.15.40:0/3971358405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65900ce2d0 0x7f65900b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.749+0000 7f658ffff640 1 -- 172.21.15.40:0/3971358405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6588009560 con 0x7f659015ed60 2023-10-18T09:01:12.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.750+0000 7f658ffff640 1 --2- 172.21.15.40:0/3971358405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659015ed60 0x7f65901f5100 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f6588009230 tx=0x7f658802dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:12.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.750+0000 7f6577fff640 1 -- 172.21.15.40:0/3971358405 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588005690 con 0x7f659015ed60 2023-10-18T09:01:12.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.750+0000 7f6577fff640 1 -- 172.21.15.40:0/3971358405 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f65880038f0 con 0x7f659015ed60 2023-10-18T09:01:12.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.750+0000 7f6577fff640 1 -- 172.21.15.40:0/3971358405 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588003d00 con 0x7f659015ed60 2023-10-18T09:01:12.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.750+0000 7f6594ec3640 1 -- 172.21.15.40:0/3971358405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65901f58e0 con 0x7f659015ed60 2023-10-18T09:01:12.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.750+0000 7f6594ec3640 1 -- 172.21.15.40:0/3971358405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65901f5ea0 con 0x7f659015ed60 2023-10-18T09:01:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.751+0000 7f6594ec3640 1 -- 172.21.15.40:0/3971358405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6590073570 con 0x7f659015ed60 2023-10-18T09:01:12.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.752+0000 7f6577fff640 1 -- 172.21.15.40:0/3971358405 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f658803e450 con 0x7f659015ed60 2023-10-18T09:01:12.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.753+0000 7f6577fff640 1 --2- 172.21.15.40:0/3971358405 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f657c06ae40 0x7f657c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:12.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.753+0000 7f6577fff640 1 -- 172.21.15.40:0/3971358405 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f65880ac0c0 con 0x7f659015ed60 2023-10-18T09:01:12.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.754+0000 7f6577fff640 1 -- 172.21.15.40:0/3971358405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=93}) v3 -- 0x7f657c072b50 con 0x7f659015ed60 2023-10-18T09:01:12.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.754+0000 7f658f7fe640 1 --2- 172.21.15.40:0/3971358405 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f657c06ae40 0x7f657c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:12.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.755+0000 7f658f7fe640 1 --2- 172.21.15.40:0/3971358405 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f657c06ae40 0x7f657c06d300 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6580009f60 tx=0x7f65800094f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:12.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.758+0000 7f6577fff640 1 -- 172.21.15.40:0/3971358405 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6588042070 con 0x7f659015ed60 2023-10-18T09:01:12.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.953+0000 7f6594ec3640 1 -- 172.21.15.40:0/3971358405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6590152c30 con 0x7f659015ed60 2023-10-18T09:01:12.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.954+0000 7f6577fff640 1 -- 172.21.15.40:0/3971358405 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65880771f0 con 0x7f659015ed60 2023-10-18T09:01:12.958 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:01:12.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.957+0000 7f6594ec3640 1 -- 172.21.15.40:0/3971358405 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f657c06ae40 msgr2=0x7f657c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:12.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.957+0000 7f6594ec3640 1 --2- 172.21.15.40:0/3971358405 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f657c06ae40 0x7f657c06d300 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6580009f60 tx=0x7f65800094f0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.957+0000 7f6594ec3640 1 -- 172.21.15.40:0/3971358405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659015ed60 msgr2=0x7f65901f5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:12.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.957+0000 7f6594ec3640 1 --2- 172.21.15.40:0/3971358405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659015ed60 0x7f65901f5100 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f6588009230 tx=0x7f658802dc60 comp rx=0 tx=0).stop 2023-10-18T09:01:12.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.957+0000 7f6594ec3640 1 -- 172.21.15.40:0/3971358405 shutdown_connections 2023-10-18T09:01:12.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.957+0000 7f6594ec3640 1 --2- 172.21.15.40:0/3971358405 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f657c06ae40 0x7f657c06d300 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.957+0000 7f6594ec3640 1 --2- 172.21.15.40:0/3971358405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f659015ed60 0x7f65901f5100 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.957+0000 7f6594ec3640 1 --2- 172.21.15.40:0/3971358405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65900ce2d0 0x7f65900b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.957+0000 7f6594ec3640 1 --2- 172.21.15.40:0/3971358405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65900b1de0 0x7f6590155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:12.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.957+0000 7f6594ec3640 1 -- 172.21.15.40:0/3971358405 >> 172.21.15.40:0/3971358405 conn(0x7f659009a580 msgr2=0x7f659015c440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:12.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.957+0000 7f6594ec3640 1 -- 172.21.15.40:0/3971358405 shutdown_connections 2023-10-18T09:01:12.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:12.957+0000 7f6594ec3640 1 -- 172.21.15.40:0/3971358405 wait complete. 2023-10-18T09:01:12.980 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:01:13.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.210+0000 7fc707fff640 1 -- 172.21.15.40:0/731754830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f4004d00 msgr2=0x7fc6f4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:13.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.210+0000 7fc707fff640 1 --2- 172.21.15.40:0/731754830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f4004d00 0x7fc6f4005180 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc70806c860 tx=0x7fc6f40057f0 comp rx=0 tx=0).stop 2023-10-18T09:01:13.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.210+0000 7fc7077fe640 1 -- 172.21.15.40:0/731754830 reap_dead start 2023-10-18T09:01:13.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.210+0000 7fc707fff640 1 -- 172.21.15.40:0/731754830 shutdown_connections 2023-10-18T09:01:13.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.210+0000 7fc707fff640 1 -- 172.21.15.40:0/731754830 >> 172.21.15.40:0/731754830 conn(0x7fc708069720 msgr2=0x7fc708069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:13.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.210+0000 7fc707fff640 1 -- 172.21.15.40:0/731754830 shutdown_connections 2023-10-18T09:01:13.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.210+0000 7fc707fff640 1 -- 172.21.15.40:0/731754830 wait complete. 2023-10-18T09:01:13.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.212+0000 7fc707fff640 1 Processor -- start 2023-10-18T09:01:13.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.212+0000 7fc707fff640 1 -- start start 2023-10-18T09:01:13.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.213+0000 7fc707fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6f4004d00 0x7fc70815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:13.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.213+0000 7fc707fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7080aede0 0x7fc7080af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:13.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.213+0000 7fc707fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70813c850 0x7fc70813ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:13.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.213+0000 7fc707fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081f5c00 con 0x7fc70813c850 2023-10-18T09:01:13.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.213+0000 7fc707fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081f2710 con 0x7fc7080aede0 2023-10-18T09:01:13.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.213+0000 7fc707fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081f4d30 con 0x7fc6f4004d00 2023-10-18T09:01:13.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.214+0000 7fc7077fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70813c850 0x7fc70813ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:13.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.214+0000 7fc7077fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70813c850 0x7fc70813ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60944/0 (socket says 172.21.15.40:60944) 2023-10-18T09:01:13.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.214+0000 7fc7077fe640 1 -- 172.21.15.40:0/1242772680 learned_addr learned my addr 172.21.15.40:0/1242772680 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:13.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.214+0000 7fc706ffd640 1 --2- 172.21.15.40:0/1242772680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6f4004d00 0x7fc70815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:13.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.214+0000 7fc6fe7fc640 1 --2- 172.21.15.40:0/1242772680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7080aede0 0x7fc7080af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:13.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.214+0000 7fc7077fe640 1 -- 172.21.15.40:0/1242772680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6f4004d00 msgr2=0x7fc70815ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:13.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.214+0000 7fc7077fe640 1 --2- 172.21.15.40:0/1242772680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6f4004d00 0x7fc70815ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:13.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.214+0000 7fc7077fe640 1 -- 172.21.15.40:0/1242772680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7080aede0 msgr2=0x7fc7080af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:13.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.214+0000 7fc7077fe640 1 --2- 172.21.15.40:0/1242772680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7080aede0 0x7fc7080af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:13.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.214+0000 7fc7077fe640 1 -- 172.21.15.40:0/1242772680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7081f4f80 con 0x7fc70813c850 2023-10-18T09:01:13.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.214+0000 7fc7077fe640 1 --2- 172.21.15.40:0/1242772680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70813c850 0x7fc70813ccd0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc700001930 tx=0x7fc70002df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:13.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.215+0000 7fc6fffff640 1 -- 172.21.15.40:0/1242772680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70003b070 con 0x7fc70813c850 2023-10-18T09:01:13.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.215+0000 7fc6fffff640 1 -- 172.21.15.40:0/1242772680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc700001680 con 0x7fc70813c850 2023-10-18T09:01:13.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.215+0000 7fc6fffff640 1 -- 172.21.15.40:0/1242772680 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc700003850 con 0x7fc70813c850 2023-10-18T09:01:13.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.215+0000 7fc707fff640 1 -- 172.21.15.40:0/1242772680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7081f5260 con 0x7fc70813c850 2023-10-18T09:01:13.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.215+0000 7fc707fff640 1 -- 172.21.15.40:0/1242772680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7081f5820 con 0x7fc70813c850 2023-10-18T09:01:13.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.216+0000 7fc707fff640 1 -- 172.21.15.40:0/1242772680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7080c98c0 con 0x7fc70813c850 2023-10-18T09:01:13.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.217+0000 7fc6fffff640 1 -- 172.21.15.40:0/1242772680 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc700032070 con 0x7fc70813c850 2023-10-18T09:01:13.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.218+0000 7fc6fffff640 1 --2- 172.21.15.40:0/1242772680 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc6e806ae70 0x7fc6e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:13.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.219+0000 7fc6fffff640 1 -- 172.21.15.40:0/1242772680 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fc7000aa330 con 0x7fc70813c850 2023-10-18T09:01:13.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.219+0000 7fc6fffff640 1 -- 172.21.15.40:0/1242772680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=93}) v3 -- 0x7fc6e8072b80 con 0x7fc70813c850 2023-10-18T09:01:13.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.219+0000 7fc706ffd640 1 --2- 172.21.15.40:0/1242772680 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc6e806ae70 0x7fc6e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:13.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.220+0000 7fc706ffd640 1 --2- 172.21.15.40:0/1242772680 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc6e806ae70 0x7fc6e806d330 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8006510 tx=0x7fc6f8006ae0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:13.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.223+0000 7fc6fffff640 1 -- 172.21.15.40:0/1242772680 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc700075340 con 0x7fc70813c850 2023-10-18T09:01:13.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.433+0000 7fc707fff640 1 -- 172.21.15.40:0/1242772680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc7081e5040 con 0x7fc70813c850 2023-10-18T09:01:13.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.434+0000 7fc6fffff640 1 -- 172.21.15.40:0/1242772680 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v92) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc700078960 con 0x7fc70813c850 2023-10-18T09:01:13.437 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T09:01:13.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.437+0000 7fc707fff640 1 -- 172.21.15.40:0/1242772680 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc6e806ae70 msgr2=0x7fc6e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:13.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.437+0000 7fc707fff640 1 --2- 172.21.15.40:0/1242772680 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc6e806ae70 0x7fc6e806d330 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8006510 tx=0x7fc6f8006ae0 comp rx=0 tx=0).stop 2023-10-18T09:01:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.437+0000 7fc707fff640 1 -- 172.21.15.40:0/1242772680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70813c850 msgr2=0x7fc70813ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.437+0000 7fc707fff640 1 --2- 172.21.15.40:0/1242772680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70813c850 0x7fc70813ccd0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc700001930 tx=0x7fc70002df20 comp rx=0 tx=0).stop 2023-10-18T09:01:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.437+0000 7fc707fff640 1 -- 172.21.15.40:0/1242772680 shutdown_connections 2023-10-18T09:01:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.437+0000 7fc707fff640 1 --2- 172.21.15.40:0/1242772680 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc6e806ae70 0x7fc6e806d330 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:13.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.437+0000 7fc707fff640 1 --2- 172.21.15.40:0/1242772680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70813c850 0x7fc70813ccd0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.437+0000 7fc707fff640 1 --2- 172.21.15.40:0/1242772680 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7080aede0 0x7fc7080af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.437+0000 7fc707fff640 1 --2- 172.21.15.40:0/1242772680 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6f4004d00 0x7fc70815ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.437+0000 7fc707fff640 1 -- 172.21.15.40:0/1242772680 >> 172.21.15.40:0/1242772680 conn(0x7fc708069720 msgr2=0x7fc70806f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.437+0000 7fc707fff640 1 -- 172.21.15.40:0/1242772680 shutdown_connections 2023-10-18T09:01:13.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:13.437+0000 7fc707fff640 1 -- 172.21.15.40:0/1242772680 wait complete. 2023-10-18T09:01:13.460 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-18T09:01:13.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:13.461 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-18T09:01:13.570 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T09:01:13.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:13.571 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T09:01:13.625 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:01:13.626 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:01:13.686 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:01:13.687 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:01:13.744 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:01:14.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.058+0000 7ff211925640 1 -- 172.21.15.40:0/1675138971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff20c0ce430 msgr2=0x7ff20c0b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:14.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.058+0000 7ff211925640 1 --2- 172.21.15.40:0/1675138971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff20c0ce430 0x7ff20c0b4e00 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff204005fd0 tx=0x7ff20402be60 comp rx=0 tx=0).stop 2023-10-18T09:01:14.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.059+0000 7ff211925640 1 -- 172.21.15.40:0/1675138971 shutdown_connections 2023-10-18T09:01:14.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.059+0000 7ff211925640 1 --2- 172.21.15.40:0/1675138971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff20c0ce430 0x7ff20c0b4e00 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.059+0000 7ff211925640 1 --2- 172.21.15.40:0/1675138971 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff20c0b7000 0x7ff20c0cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.059+0000 7ff211925640 1 -- 172.21.15.40:0/1675138971 >> 172.21.15.40:0/1675138971 conn(0x7ff20c06a780 msgr2=0x7ff20c06ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:14.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.060+0000 7ff211925640 1 -- 172.21.15.40:0/1675138971 shutdown_connections 2023-10-18T09:01:14.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.060+0000 7ff211925640 1 -- 172.21.15.40:0/1675138971 wait complete. 2023-10-18T09:01:14.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.063+0000 7ff211925640 1 Processor -- start 2023-10-18T09:01:14.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.063+0000 7ff211925640 1 -- start start 2023-10-18T09:01:14.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.065+0000 7ff211925640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff20c0b7000 0x7ff20c1e5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:14.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.065+0000 7ff211925640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff20c0ce430 0x7ff20c14f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:14.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.065+0000 7ff211925640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff20c0c54d0 0x7ff20c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:14.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.065+0000 7ff211925640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c0c7ec0 con 0x7ff20c0ce430 2023-10-18T09:01:14.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.065+0000 7ff211925640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c14ff30 con 0x7ff20c0b7000 2023-10-18T09:01:14.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.065+0000 7ff211925640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c1e6070 con 0x7ff20c0c54d0 2023-10-18T09:01:14.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.065+0000 7ff20b7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff20c0c54d0 0x7ff20c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:14.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.065+0000 7ff20b7fe640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff20c0c54d0 0x7ff20c0c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:41680/0 (socket says 172.21.15.40:41680) 2023-10-18T09:01:14.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.066+0000 7ff20b7fe640 1 -- 172.21.15.40:0/1320002722 learned_addr learned my addr 172.21.15.40:0/1320002722 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:14.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.066+0000 7ff20a7fc640 1 --2- 172.21.15.40:0/1320002722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff20c0ce430 0x7ff20c14f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:14.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.066+0000 7ff20affd640 1 --2- 172.21.15.40:0/1320002722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff20c0b7000 0x7ff20c1e5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:14.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.067+0000 7ff20b7fe640 1 -- 172.21.15.40:0/1320002722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff20c0b7000 msgr2=0x7ff20c1e5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:14.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.067+0000 7ff20b7fe640 1 --2- 172.21.15.40:0/1320002722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff20c0b7000 0x7ff20c1e5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.067+0000 7ff20b7fe640 1 -- 172.21.15.40:0/1320002722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff20c0ce430 msgr2=0x7ff20c14f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:14.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.067+0000 7ff20b7fe640 1 --2- 172.21.15.40:0/1320002722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff20c0ce430 0x7ff20c14f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.067+0000 7ff20b7fe640 1 -- 172.21.15.40:0/1320002722 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff204009560 con 0x7ff20c0c54d0 2023-10-18T09:01:14.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.067+0000 7ff20a7fc640 1 --2- 172.21.15.40:0/1320002722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff20c0ce430 0x7ff20c14f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:01:14.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.068+0000 7ff20b7fe640 1 --2- 172.21.15.40:0/1320002722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff20c0c54d0 0x7ff20c0c5980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff204036040 tx=0x7ff20402db80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:14.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.068+0000 7ff1f3fff640 1 -- 172.21.15.40:0/1320002722 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2040037f0 con 0x7ff20c0c54d0 2023-10-18T09:01:14.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.068+0000 7ff1f3fff640 1 -- 172.21.15.40:0/1320002722 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff204003a40 con 0x7ff20c0c54d0 2023-10-18T09:01:14.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.068+0000 7ff1f3fff640 1 -- 172.21.15.40:0/1320002722 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff204003df0 con 0x7ff20c0c54d0 2023-10-18T09:01:14.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.068+0000 7ff211925640 1 -- 172.21.15.40:0/1320002722 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff20c1e6350 con 0x7ff20c0c54d0 2023-10-18T09:01:14.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.069+0000 7ff211925640 1 -- 172.21.15.40:0/1320002722 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff20c0b1a90 con 0x7ff20c0c54d0 2023-10-18T09:01:14.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.071+0000 7ff211925640 1 -- 172.21.15.40:0/1320002722 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff20c066fd0 con 0x7ff20c0c54d0 2023-10-18T09:01:14.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.073+0000 7ff1f3fff640 1 -- 172.21.15.40:0/1320002722 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff20403f470 con 0x7ff20c0c54d0 2023-10-18T09:01:14.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.074+0000 7ff1f3fff640 1 --2- 172.21.15.40:0/1320002722 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1f806aec0 0x7ff1f806d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:14.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.074+0000 7ff1f3fff640 1 -- 172.21.15.40:0/1320002722 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff2040ac4a0 con 0x7ff20c0c54d0 2023-10-18T09:01:14.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.074+0000 7ff1f3fff640 1 -- 172.21.15.40:0/1320002722 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=93}) v3 -- 0x7ff1f8072bd0 con 0x7ff20c0c54d0 2023-10-18T09:01:14.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.074+0000 7ff20affd640 1 --2- 172.21.15.40:0/1320002722 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1f806aec0 0x7ff1f806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:14.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.075+0000 7ff20affd640 1 --2- 172.21.15.40:0/1320002722 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1f806aec0 0x7ff1f806d380 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc0012e0 tx=0x7ff1fc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:14.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.080+0000 7ff1f3fff640 1 -- 172.21.15.40:0/1320002722 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff204043060 con 0x7ff20c0c54d0 2023-10-18T09:01:14.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.278+0000 7ff211925640 1 -- 172.21.15.40:0/1320002722 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff20c15c630 con 0x7ff20c0c54d0 2023-10-18T09:01:14.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.283+0000 7ff1f3fff640 1 -- 172.21.15.40:0/1320002722 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16863 (secure 0 0 0) 0x7ff204043030 con 0x7ff20c0c54d0 2023-10-18T09:01:14.286 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:14.286 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":92,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:01:02.983030+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:00:13.170937+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-18T09:00:14.205418+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1","mappings":[{"from":6,"to":0}]},{"pgid":"11.8","mappings":[{"from":6,"to":1}]},{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.18","mappings":[{"from":2,"to":0}]},{"pgid":"11.33","mappings":[{"from":6,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:01:14.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.285+0000 7ff211925640 1 -- 172.21.15.40:0/1320002722 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1f806aec0 msgr2=0x7ff1f806d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:14.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.285+0000 7ff211925640 1 --2- 172.21.15.40:0/1320002722 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1f806aec0 0x7ff1f806d380 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc0012e0 tx=0x7ff1fc009040 comp rx=0 tx=0).stop 2023-10-18T09:01:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.285+0000 7ff211925640 1 -- 172.21.15.40:0/1320002722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff20c0c54d0 msgr2=0x7ff20c0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.285+0000 7ff211925640 1 --2- 172.21.15.40:0/1320002722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff20c0c54d0 0x7ff20c0c5980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff204036040 tx=0x7ff20402db80 comp rx=0 tx=0).stop 2023-10-18T09:01:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.285+0000 7ff211925640 1 -- 172.21.15.40:0/1320002722 shutdown_connections 2023-10-18T09:01:14.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.285+0000 7ff211925640 1 --2- 172.21.15.40:0/1320002722 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1f806aec0 0x7ff1f806d380 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.285+0000 7ff211925640 1 --2- 172.21.15.40:0/1320002722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff20c0c54d0 0x7ff20c0c5980 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.285+0000 7ff211925640 1 --2- 172.21.15.40:0/1320002722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff20c0ce430 0x7ff20c14f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.285+0000 7ff211925640 1 --2- 172.21.15.40:0/1320002722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff20c0b7000 0x7ff20c1e5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.285+0000 7ff211925640 1 -- 172.21.15.40:0/1320002722 >> 172.21.15.40:0/1320002722 conn(0x7ff20c06a780 msgr2=0x7ff20c151bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:14.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.286+0000 7ff211925640 1 -- 172.21.15.40:0/1320002722 shutdown_connections 2023-10-18T09:01:14.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.286+0000 7ff211925640 1 -- 172.21.15.40:0/1320002722 wait complete. 2023-10-18T09:01:14.308 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:00:13.170937+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:00:14.205418+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:01:14.309 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:01:14.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.531+0000 7f31e7fff640 1 -- 172.21.15.40:0/3451701681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31d00058d0 msgr2=0x7f31d0005d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:14.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.531+0000 7f31e7fff640 1 --2- 172.21.15.40:0/3451701681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31d00058d0 0x7f31d0005d50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f31e81409f0 tx=0x7f31d0006900 comp rx=0 tx=0).stop 2023-10-18T09:01:14.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.531+0000 7f31e7fff640 1 -- 172.21.15.40:0/3451701681 shutdown_connections 2023-10-18T09:01:14.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.531+0000 7f31e7fff640 1 --2- 172.21.15.40:0/3451701681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31d00058d0 0x7f31d0005d50 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.531+0000 7f31e7fff640 1 -- 172.21.15.40:0/3451701681 >> 172.21.15.40:0/3451701681 conn(0x7f31e8069720 msgr2=0x7f31e8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:14.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.531+0000 7f31e7fff640 1 -- 172.21.15.40:0/3451701681 shutdown_connections 2023-10-18T09:01:14.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.531+0000 7f31e7fff640 1 -- 172.21.15.40:0/3451701681 wait complete. 2023-10-18T09:01:14.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.533+0000 7f31e7fff640 1 Processor -- start 2023-10-18T09:01:14.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.533+0000 7f31e7fff640 1 -- start start 2023-10-18T09:01:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.534+0000 7f31e7fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31d00058d0 0x7f31e8156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.534+0000 7f31e7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31e81e5130 0x7f31e81e55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.534+0000 7f31e7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e81e1c70 0x7f31e81e20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:14.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.534+0000 7f31e7fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e81e5af0 con 0x7f31e81e1c70 2023-10-18T09:01:14.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.534+0000 7f31e7fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e81e10d0 con 0x7f31e81e5130 2023-10-18T09:01:14.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.534+0000 7f31e7fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e81e2630 con 0x7f31d00058d0 2023-10-18T09:01:14.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.535+0000 7f31e77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e81e1c70 0x7f31e81e20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:14.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.535+0000 7f31e77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e81e1c70 0x7f31e81e20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60980/0 (socket says 172.21.15.40:60980) 2023-10-18T09:01:14.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.535+0000 7f31e77fe640 1 -- 172.21.15.40:0/1558813146 learned_addr learned my addr 172.21.15.40:0/1558813146 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:14.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.535+0000 7f31e6ffd640 1 --2- 172.21.15.40:0/1558813146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31d00058d0 0x7f31e8156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:14.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.535+0000 7f31e67fc640 1 --2- 172.21.15.40:0/1558813146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31e81e5130 0x7f31e81e55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:14.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.535+0000 7f31e77fe640 1 -- 172.21.15.40:0/1558813146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31d00058d0 msgr2=0x7f31e8156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.535+0000 7f31e77fe640 1 --2- 172.21.15.40:0/1558813146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31d00058d0 0x7f31e8156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.535+0000 7f31e77fe640 1 -- 172.21.15.40:0/1558813146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31e81e5130 msgr2=0x7f31e81e55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.535+0000 7f31e77fe640 1 --2- 172.21.15.40:0/1558813146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31e81e5130 0x7f31e81e55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.535+0000 7f31e77fe640 1 -- 172.21.15.40:0/1558813146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e80b0480 con 0x7f31e81e1c70 2023-10-18T09:01:14.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.536+0000 7f31e77fe640 1 --2- 172.21.15.40:0/1558813146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e81e1c70 0x7f31e81e20f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f31e80ca290 tx=0x7f31e000d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.536+0000 7f31cffff640 1 -- 172.21.15.40:0/1558813146 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e000f880 con 0x7f31e81e1c70 2023-10-18T09:01:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.536+0000 7f31cffff640 1 -- 172.21.15.40:0/1558813146 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f31e000fad0 con 0x7f31e81e1c70 2023-10-18T09:01:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.536+0000 7f31e7fff640 1 -- 172.21.15.40:0/1558813146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e8157d10 con 0x7f31e81e1c70 2023-10-18T09:01:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.536+0000 7f31cffff640 1 -- 172.21.15.40:0/1558813146 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0010450 con 0x7f31e81e1c70 2023-10-18T09:01:14.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.536+0000 7f31e7fff640 1 -- 172.21.15.40:0/1558813146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e81e2b60 con 0x7f31e81e1c70 2023-10-18T09:01:14.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.537+0000 7f31e7fff640 1 -- 172.21.15.40:0/1558813146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e80af2e0 con 0x7f31e81e1c70 2023-10-18T09:01:14.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.538+0000 7f31cffff640 1 -- 172.21.15.40:0/1558813146 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f31e00105b0 con 0x7f31e81e1c70 2023-10-18T09:01:14.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.540+0000 7f31cffff640 1 --2- 172.21.15.40:0/1558813146 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31d406ae40 0x7f31d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:14.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.540+0000 7f31cffff640 1 -- 172.21.15.40:0/1558813146 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f31e0089de0 con 0x7f31e81e1c70 2023-10-18T09:01:14.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.540+0000 7f31cffff640 1 -- 172.21.15.40:0/1558813146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=93}) v3 -- 0x7f31d4072b50 con 0x7f31e81e1c70 2023-10-18T09:01:14.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.540+0000 7f31e6ffd640 1 --2- 172.21.15.40:0/1558813146 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31d406ae40 0x7f31d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:14.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.541+0000 7f31e6ffd640 1 --2- 172.21.15.40:0/1558813146 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31d406ae40 0x7f31d406d300 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f31d8005d00 tx=0x7f31d8005c70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:14.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.545+0000 7f31cffff640 1 -- 172.21.15.40:0/1558813146 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f31e0014020 con 0x7f31e81e1c70 2023-10-18T09:01:14.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.736+0000 7f31e7fff640 1 -- 172.21.15.40:0/1558813146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31e8066e00 con 0x7f31e81e1c70 2023-10-18T09:01:14.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.738+0000 7f31cffff640 1 -- 172.21.15.40:0/1558813146 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31e0054df0 con 0x7f31e81e1c70 2023-10-18T09:01:14.741 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:01:14.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.740+0000 7f31e7fff640 1 -- 172.21.15.40:0/1558813146 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31d406ae40 msgr2=0x7f31d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:14.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.740+0000 7f31e7fff640 1 --2- 172.21.15.40:0/1558813146 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31d406ae40 0x7f31d406d300 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f31d8005d00 tx=0x7f31d8005c70 comp rx=0 tx=0).stop 2023-10-18T09:01:14.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.740+0000 7f31e7fff640 1 -- 172.21.15.40:0/1558813146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e81e1c70 msgr2=0x7f31e81e20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:14.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.740+0000 7f31e7fff640 1 --2- 172.21.15.40:0/1558813146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e81e1c70 0x7f31e81e20f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f31e80ca290 tx=0x7f31e000d970 comp rx=0 tx=0).stop 2023-10-18T09:01:14.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.740+0000 7f31e7fff640 1 -- 172.21.15.40:0/1558813146 shutdown_connections 2023-10-18T09:01:14.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.740+0000 7f31e7fff640 1 --2- 172.21.15.40:0/1558813146 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31d406ae40 0x7f31d406d300 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.740+0000 7f31e7fff640 1 --2- 172.21.15.40:0/1558813146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e81e1c70 0x7f31e81e20f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.740+0000 7f31e7fff640 1 --2- 172.21.15.40:0/1558813146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31e81e5130 0x7f31e81e55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.740+0000 7f31e7fff640 1 --2- 172.21.15.40:0/1558813146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31d00058d0 0x7f31e8156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.740+0000 7f31e7fff640 1 -- 172.21.15.40:0/1558813146 >> 172.21.15.40:0/1558813146 conn(0x7f31e8069720 msgr2=0x7f31e80aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:14.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.740+0000 7f31e7fff640 1 -- 172.21.15.40:0/1558813146 shutdown_connections 2023-10-18T09:01:14.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.740+0000 7f31e7fff640 1 -- 172.21.15.40:0/1558813146 wait complete. 2023-10-18T09:01:14.763 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:01:14.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.995+0000 7f4c77fff640 1 -- 172.21.15.40:0/3337479485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c600047c0 msgr2=0x7f4c60004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:14.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.995+0000 7f4c77fff640 1 --2- 172.21.15.40:0/3337479485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c600047c0 0x7f4c60004c40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c001680 tx=0x7f4c600057f0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.995+0000 7f4c77fff640 1 -- 172.21.15.40:0/3337479485 shutdown_connections 2023-10-18T09:01:14.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.995+0000 7f4c77fff640 1 --2- 172.21.15.40:0/3337479485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c600047c0 0x7f4c60004c40 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:14.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.995+0000 7f4c77fff640 1 -- 172.21.15.40:0/3337479485 >> 172.21.15.40:0/3337479485 conn(0x7f4c7806ca10 msgr2=0x7f4c7806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:14.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.995+0000 7f4c77fff640 1 -- 172.21.15.40:0/3337479485 shutdown_connections 2023-10-18T09:01:14.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.995+0000 7f4c77fff640 1 -- 172.21.15.40:0/3337479485 wait complete. 2023-10-18T09:01:15.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.997+0000 7f4c77fff640 1 Processor -- start 2023-10-18T09:01:15.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.997+0000 7f4c77fff640 1 -- start start 2023-10-18T09:01:15.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.998+0000 7f4c77fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c600047c0 0x7f4c7814e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:15.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.998+0000 7f4c77fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c78159350 0x7f4c7813ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:15.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.998+0000 7f4c77fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c78157a60 0x7f4c781567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:15.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.998+0000 7f4c77fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c78158ec0 con 0x7f4c78157a60 2023-10-18T09:01:15.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.998+0000 7f4c77fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c781582b0 con 0x7f4c78159350 2023-10-18T09:01:15.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.998+0000 7f4c77fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c78158590 con 0x7f4c600047c0 2023-10-18T09:01:15.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.999+0000 7f4c777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c78157a60 0x7f4c781567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:15.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.999+0000 7f4c777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c78157a60 0x7f4c781567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57678/0 (socket says 172.21.15.40:57678) 2023-10-18T09:01:15.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.999+0000 7f4c777fe640 1 -- 172.21.15.40:0/2681474214 learned_addr learned my addr 172.21.15.40:0/2681474214 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:15.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.999+0000 7f4c76ffd640 1 --2- 172.21.15.40:0/2681474214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c600047c0 0x7f4c7814e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:15.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.999+0000 7f4c767fc640 1 --2- 172.21.15.40:0/2681474214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c78159350 0x7f4c7813ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:15.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.999+0000 7f4c777fe640 1 -- 172.21.15.40:0/2681474214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c600047c0 msgr2=0x7f4c7814e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:15.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.999+0000 7f4c777fe640 1 --2- 172.21.15.40:0/2681474214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c600047c0 0x7f4c7814e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.999+0000 7f4c777fe640 1 -- 172.21.15.40:0/2681474214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c78159350 msgr2=0x7f4c7813ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:15.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.999+0000 7f4c777fe640 1 --2- 172.21.15.40:0/2681474214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c78159350 0x7f4c7813ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:14.999+0000 7f4c777fe640 1 -- 172.21.15.40:0/2681474214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c78158870 con 0x7f4c78157a60 2023-10-18T09:01:15.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.000+0000 7f4c777fe640 1 --2- 172.21.15.40:0/2681474214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c78157a60 0x7f4c781567e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4c70001370 tx=0x7f4c7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:15.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.000+0000 7f4c5f7fe640 1 -- 172.21.15.40:0/2681474214 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c70015030 con 0x7f4c78157a60 2023-10-18T09:01:15.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.000+0000 7f4c5f7fe640 1 -- 172.21.15.40:0/2681474214 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4c70003e70 con 0x7f4c78157a60 2023-10-18T09:01:15.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.000+0000 7f4c5f7fe640 1 -- 172.21.15.40:0/2681474214 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7000bae0 con 0x7f4c78157a60 2023-10-18T09:01:15.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.000+0000 7f4c77fff640 1 -- 172.21.15.40:0/2681474214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c78158b50 con 0x7f4c78157a60 2023-10-18T09:01:15.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.000+0000 7f4c77fff640 1 -- 172.21.15.40:0/2681474214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c78156d20 con 0x7f4c78157a60 2023-10-18T09:01:15.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.001+0000 7f4c77fff640 1 -- 172.21.15.40:0/2681474214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c781586d0 con 0x7f4c78157a60 2023-10-18T09:01:15.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.002+0000 7f4c5f7fe640 1 -- 172.21.15.40:0/2681474214 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4c70018070 con 0x7f4c78157a60 2023-10-18T09:01:15.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.003+0000 7f4c5f7fe640 1 --2- 172.21.15.40:0/2681474214 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4c6406ae40 0x7f4c6406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:15.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.003+0000 7f4c5f7fe640 1 -- 172.21.15.40:0/2681474214 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f4c70016070 con 0x7f4c78157a60 2023-10-18T09:01:15.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.003+0000 7f4c5f7fe640 1 -- 172.21.15.40:0/2681474214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=93}) v3 -- 0x7f4c64072b50 con 0x7f4c78157a60 2023-10-18T09:01:15.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.004+0000 7f4c76ffd640 1 --2- 172.21.15.40:0/2681474214 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4c6406ae40 0x7f4c6406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:15.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.005+0000 7f4c76ffd640 1 --2- 172.21.15.40:0/2681474214 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4c6406ae40 0x7f4c6406d300 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4c781f4dc0 tx=0x7f4c68005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:15.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.008+0000 7f4c5f7fe640 1 -- 172.21.15.40:0/2681474214 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c781586d0 con 0x7f4c78157a60 2023-10-18T09:01:15.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.201+0000 7f4c77fff640 1 -- 172.21.15.40:0/2681474214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4c781586d0 con 0x7f4c78157a60 2023-10-18T09:01:15.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.202+0000 7f4c5f7fe640 1 -- 172.21.15.40:0/2681474214 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4c781586d0 con 0x7f4c78157a60 2023-10-18T09:01:15.205 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:01:15.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.205+0000 7f4c77fff640 1 -- 172.21.15.40:0/2681474214 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4c6406ae40 msgr2=0x7f4c6406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:15.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.205+0000 7f4c77fff640 1 --2- 172.21.15.40:0/2681474214 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4c6406ae40 0x7f4c6406d300 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4c781f4dc0 tx=0x7f4c68005f90 comp rx=0 tx=0).stop 2023-10-18T09:01:15.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.205+0000 7f4c77fff640 1 -- 172.21.15.40:0/2681474214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c78157a60 msgr2=0x7f4c781567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:15.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.205+0000 7f4c77fff640 1 --2- 172.21.15.40:0/2681474214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c78157a60 0x7f4c781567e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4c70001370 tx=0x7f4c7000de10 comp rx=0 tx=0).stop 2023-10-18T09:01:15.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.205+0000 7f4c77fff640 1 -- 172.21.15.40:0/2681474214 shutdown_connections 2023-10-18T09:01:15.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.205+0000 7f4c77fff640 1 --2- 172.21.15.40:0/2681474214 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4c6406ae40 0x7f4c6406d300 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.205+0000 7f4c77fff640 1 --2- 172.21.15.40:0/2681474214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c78157a60 0x7f4c781567e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.205+0000 7f4c77fff640 1 --2- 172.21.15.40:0/2681474214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c78159350 0x7f4c7813ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.205+0000 7f4c77fff640 1 --2- 172.21.15.40:0/2681474214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c600047c0 0x7f4c7814e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.205+0000 7f4c77fff640 1 -- 172.21.15.40:0/2681474214 >> 172.21.15.40:0/2681474214 conn(0x7f4c7806ca10 msgr2=0x7f4c780ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:15.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.205+0000 7f4c77fff640 1 -- 172.21.15.40:0/2681474214 shutdown_connections 2023-10-18T09:01:15.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.205+0000 7f4c77fff640 1 -- 172.21.15.40:0/2681474214 wait complete. 2023-10-18T09:01:15.227 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:01:15.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.462+0000 7f523cb8f640 1 -- 172.21.15.40:0/2999024 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52340339e0 con 0x7f5224004de0 2023-10-18T09:01:15.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.463+0000 7f523eb93640 1 -- 172.21.15.40:0/2999024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5224004de0 msgr2=0x7f52240071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:15.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.463+0000 7f523eb93640 1 --2- 172.21.15.40:0/2999024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5224004de0 0x7f52240071d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f5234009b50 tx=0x7f52240037d0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.463+0000 7f523eb93640 1 -- 172.21.15.40:0/2999024 shutdown_connections 2023-10-18T09:01:15.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.463+0000 7f523eb93640 1 --2- 172.21.15.40:0/2999024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5224004de0 0x7f52240071d0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.463+0000 7f523eb93640 1 -- 172.21.15.40:0/2999024 >> 172.21.15.40:0/2999024 conn(0x7f523806ca10 msgr2=0x7f523806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:15.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.463+0000 7f523eb93640 1 -- 172.21.15.40:0/2999024 shutdown_connections 2023-10-18T09:01:15.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.463+0000 7f523eb93640 1 -- 172.21.15.40:0/2999024 wait complete. 2023-10-18T09:01:15.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.465+0000 7f523eb93640 1 Processor -- start 2023-10-18T09:01:15.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.465+0000 7f523eb93640 1 -- start start 2023-10-18T09:01:15.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.466+0000 7f523eb93640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5224004de0 0x7f523813fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:15.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.466+0000 7f523eb93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5238156290 0x7f5238153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:15.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.466+0000 7f523eb93640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5238156710 0x7f5238154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:15.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.466+0000 7f523eb93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52381554f0 con 0x7f5238156290 2023-10-18T09:01:15.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.466+0000 7f523eb93640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523815a800 con 0x7f5224004de0 2023-10-18T09:01:15.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.466+0000 7f523eb93640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52381557b0 con 0x7f5238156710 2023-10-18T09:01:15.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.466+0000 7f523d390640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5238156290 0x7f5238153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:15.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.466+0000 7f523d390640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5238156290 0x7f5238153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57698/0 (socket says 172.21.15.40:57698) 2023-10-18T09:01:15.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.466+0000 7f523d390640 1 -- 172.21.15.40:0/3592571462 learned_addr learned my addr 172.21.15.40:0/3592571462 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:15.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.467+0000 7f523db91640 1 --2- 172.21.15.40:0/3592571462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5224004de0 0x7f523813fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:15.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.467+0000 7f523e392640 1 --2- 172.21.15.40:0/3592571462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5238156710 0x7f5238154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:15.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.467+0000 7f523d390640 1 -- 172.21.15.40:0/3592571462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5238156710 msgr2=0x7f5238154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:15.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.467+0000 7f523d390640 1 --2- 172.21.15.40:0/3592571462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5238156710 0x7f5238154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.467+0000 7f523d390640 1 -- 172.21.15.40:0/3592571462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5224004de0 msgr2=0x7f523813fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:15.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.467+0000 7f523d390640 1 --2- 172.21.15.40:0/3592571462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5224004de0 0x7f523813fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.467+0000 7f523d390640 1 -- 172.21.15.40:0/3592571462 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5238155a90 con 0x7f5238156290 2023-10-18T09:01:15.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.467+0000 7f523e392640 1 --2- 172.21.15.40:0/3592571462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5238156710 0x7f5238154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:15.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.467+0000 7f523db91640 1 --2- 172.21.15.40:0/3592571462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5224004de0 0x7f523813fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:15.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.467+0000 7f523d390640 1 --2- 172.21.15.40:0/3592571462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5238156290 0x7f5238153ff0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f522800c730 tx=0x7f522800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:15.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.467+0000 7f522e7fc640 1 -- 172.21.15.40:0/3592571462 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228005270 con 0x7f5238156290 2023-10-18T09:01:15.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.467+0000 7f522e7fc640 1 -- 172.21.15.40:0/3592571462 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f52280053f0 con 0x7f5238156290 2023-10-18T09:01:15.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.467+0000 7f523eb93640 1 -- 172.21.15.40:0/3592571462 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5238155d70 con 0x7f5238156290 2023-10-18T09:01:15.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.467+0000 7f522e7fc640 1 -- 172.21.15.40:0/3592571462 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522800a8a0 con 0x7f5238156290 2023-10-18T09:01:15.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.468+0000 7f523eb93640 1 -- 172.21.15.40:0/3592571462 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52381f20f0 con 0x7f5238156290 2023-10-18T09:01:15.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.469+0000 7f523eb93640 1 -- 172.21.15.40:0/3592571462 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5238070b20 con 0x7f5238156290 2023-10-18T09:01:15.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.470+0000 7f522e7fc640 1 -- 172.21.15.40:0/3592571462 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f522800aa00 con 0x7f5238156290 2023-10-18T09:01:15.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.471+0000 7f522e7fc640 1 --2- 172.21.15.40:0/3592571462 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f521c06ae70 0x7f521c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:15.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.471+0000 7f522e7fc640 1 -- 172.21.15.40:0/3592571462 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f52280894e0 con 0x7f5238156290 2023-10-18T09:01:15.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.471+0000 7f522e7fc640 1 -- 172.21.15.40:0/3592571462 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=93}) v3 -- 0x7f521c072b80 con 0x7f5238156290 2023-10-18T09:01:15.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.471+0000 7f523db91640 1 --2- 172.21.15.40:0/3592571462 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f521c06ae70 0x7f521c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:15.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.473+0000 7f523db91640 1 --2- 172.21.15.40:0/3592571462 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f521c06ae70 0x7f521c06d330 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5234036000 tx=0x7f5234035040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:15.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.475+0000 7f522e7fc640 1 -- 172.21.15.40:0/3592571462 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f52280544f0 con 0x7f5238156290 2023-10-18T09:01:15.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.667+0000 7f523eb93640 1 -- 172.21.15.40:0/3592571462 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5238066e00 con 0x7f5238156290 2023-10-18T09:01:15.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.668+0000 7f522e7fc640 1 -- 172.21.15.40:0/3592571462 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5228057b10 con 0x7f5238156290 2023-10-18T09:01:15.671 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:01:15.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.671+0000 7f523eb93640 1 -- 172.21.15.40:0/3592571462 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f521c06ae70 msgr2=0x7f521c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:15.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.671+0000 7f523eb93640 1 --2- 172.21.15.40:0/3592571462 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f521c06ae70 0x7f521c06d330 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5234036000 tx=0x7f5234035040 comp rx=0 tx=0).stop 2023-10-18T09:01:15.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.671+0000 7f523eb93640 1 -- 172.21.15.40:0/3592571462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5238156290 msgr2=0x7f5238153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:15.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.671+0000 7f523eb93640 1 --2- 172.21.15.40:0/3592571462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5238156290 0x7f5238153ff0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f522800c730 tx=0x7f522800cbf0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.671+0000 7f523eb93640 1 -- 172.21.15.40:0/3592571462 shutdown_connections 2023-10-18T09:01:15.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.671+0000 7f523eb93640 1 --2- 172.21.15.40:0/3592571462 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f521c06ae70 0x7f521c06d330 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.671+0000 7f523eb93640 1 --2- 172.21.15.40:0/3592571462 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5238156710 0x7f5238154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.671+0000 7f523eb93640 1 --2- 172.21.15.40:0/3592571462 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5238156290 0x7f5238153ff0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.671+0000 7f523eb93640 1 --2- 172.21.15.40:0/3592571462 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5224004de0 0x7f523813fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.671+0000 7f523eb93640 1 -- 172.21.15.40:0/3592571462 >> 172.21.15.40:0/3592571462 conn(0x7f523806ca10 msgr2=0x7f523814c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:15.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.671+0000 7f523eb93640 1 -- 172.21.15.40:0/3592571462 shutdown_connections 2023-10-18T09:01:15.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.671+0000 7f523eb93640 1 -- 172.21.15.40:0/3592571462 wait complete. 2023-10-18T09:01:15.694 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:01:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.923+0000 7f69bd95e640 1 -- 172.21.15.40:0/1570345495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69a8003970 msgr2=0x7f69a8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.923+0000 7f69bd95e640 1 --2- 172.21.15.40:0/1570345495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69a8003970 0x7f69a8003d90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f69ac006f10 tx=0x7f69a80057f0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.923+0000 7f69bd15d640 1 -- 172.21.15.40:0/1570345495 reap_dead start 2023-10-18T09:01:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.923+0000 7f69bd95e640 1 -- 172.21.15.40:0/1570345495 shutdown_connections 2023-10-18T09:01:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.923+0000 7f69bd95e640 1 -- 172.21.15.40:0/1570345495 >> 172.21.15.40:0/1570345495 conn(0x7f69b8069720 msgr2=0x7f69b8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.923+0000 7f69bd95e640 1 -- 172.21.15.40:0/1570345495 shutdown_connections 2023-10-18T09:01:15.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.923+0000 7f69bd95e640 1 -- 172.21.15.40:0/1570345495 wait complete. 2023-10-18T09:01:15.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.925+0000 7f69bd95e640 1 Processor -- start 2023-10-18T09:01:15.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.925+0000 7f69bd95e640 1 -- start start 2023-10-18T09:01:15.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.926+0000 7f69bd95e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69a8003970 0x7f69b8158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:15.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.926+0000 7f69bd95e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b80aca50 0x7f69b80af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.926+0000 7f69bd95e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b80af7d0 0x7f69b8130200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.926+0000 7f69bd95e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b813ed60 con 0x7f69b80aca50 2023-10-18T09:01:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.926+0000 7f69bd95e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b813f040 con 0x7f69b80af7d0 2023-10-18T09:01:15.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.926+0000 7f69bd95e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b813f320 con 0x7f69a8003970 2023-10-18T09:01:15.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.927+0000 7f69b7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b80aca50 0x7f69b80af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:15.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.927+0000 7f69b7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b80aca50 0x7f69b80af290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57708/0 (socket says 172.21.15.40:57708) 2023-10-18T09:01:15.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.927+0000 7f69b7fff640 1 -- 172.21.15.40:0/800826957 learned_addr learned my addr 172.21.15.40:0/800826957 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:15.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.927+0000 7f69bd15d640 1 --2- 172.21.15.40:0/800826957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b80af7d0 0x7f69b8130200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:15.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.927+0000 7f69b7fff640 1 -- 172.21.15.40:0/800826957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69a8003970 msgr2=0x7f69b8158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:15.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.927+0000 7f69b7fff640 1 --2- 172.21.15.40:0/800826957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69a8003970 0x7f69b8158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.927+0000 7f69b7fff640 1 -- 172.21.15.40:0/800826957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b80af7d0 msgr2=0x7f69b8130200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:15.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.927+0000 7f69b7fff640 1 --2- 172.21.15.40:0/800826957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b80af7d0 0x7f69b8130200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:15.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.927+0000 7f69b7fff640 1 -- 172.21.15.40:0/800826957 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b813f600 con 0x7f69b80aca50 2023-10-18T09:01:15.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.928+0000 7f69b7fff640 1 --2- 172.21.15.40:0/800826957 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b80aca50 0x7f69b80af290 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f69a400f9d0 tx=0x7f69a400fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:15.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.928+0000 7f69b57fa640 1 -- 172.21.15.40:0/800826957 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a400bd70 con 0x7f69b80aca50 2023-10-18T09:01:15.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.928+0000 7f69b57fa640 1 -- 172.21.15.40:0/800826957 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f69a4004020 con 0x7f69b80aca50 2023-10-18T09:01:15.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.928+0000 7f69b57fa640 1 -- 172.21.15.40:0/800826957 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a40043d0 con 0x7f69b80aca50 2023-10-18T09:01:15.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.928+0000 7f69bd95e640 1 -- 172.21.15.40:0/800826957 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b813f8e0 con 0x7f69b80aca50 2023-10-18T09:01:15.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.928+0000 7f69bd95e640 1 -- 172.21.15.40:0/800826957 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b8130740 con 0x7f69b80aca50 2023-10-18T09:01:15.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.929+0000 7f69bd95e640 1 -- 172.21.15.40:0/800826957 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b8130a20 con 0x7f69b80aca50 2023-10-18T09:01:15.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.931+0000 7f69b57fa640 1 -- 172.21.15.40:0/800826957 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f69a40035f0 con 0x7f69b80aca50 2023-10-18T09:01:15.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.932+0000 7f69b57fa640 1 --2- 172.21.15.40:0/800826957 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f699c06ae40 0x7f699c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:15.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.932+0000 7f69b57fa640 1 -- 172.21.15.40:0/800826957 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f69a408a3c0 con 0x7f69b80aca50 2023-10-18T09:01:15.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.932+0000 7f69b57fa640 1 -- 172.21.15.40:0/800826957 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=93}) v3 -- 0x7f699c072b50 con 0x7f69b80aca50 2023-10-18T09:01:15.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.932+0000 7f69bc95c640 1 --2- 172.21.15.40:0/800826957 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f699c06ae40 0x7f699c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:15.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.934+0000 7f69bc95c640 1 --2- 172.21.15.40:0/800826957 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f699c06ae40 0x7f699c06d300 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f69ac034000 tx=0x7f69ac001930 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:15.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:15.936+0000 7f69b57fa640 1 -- 172.21.15.40:0/800826957 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f69a40553d0 con 0x7f69b80aca50 2023-10-18T09:01:16.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.144+0000 7f69bd95e640 1 -- 172.21.15.40:0/800826957 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f69b806d920 con 0x7f69b80aca50 2023-10-18T09:01:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.145+0000 7f69b57fa640 1 -- 172.21.15.40:0/800826957 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v92) v1 ==== 80+0+0 (secure 0 0 0) 0x7f69a401e070 con 0x7f69b80aca50 2023-10-18T09:01:16.147 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T09:01:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.148+0000 7f69bd95e640 1 -- 172.21.15.40:0/800826957 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f699c06ae40 msgr2=0x7f699c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:16.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.148+0000 7f69bd95e640 1 --2- 172.21.15.40:0/800826957 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f699c06ae40 0x7f699c06d300 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f69ac034000 tx=0x7f69ac001930 comp rx=0 tx=0).stop 2023-10-18T09:01:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.148+0000 7f69bd95e640 1 -- 172.21.15.40:0/800826957 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b80aca50 msgr2=0x7f69b80af290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.148+0000 7f69bd95e640 1 --2- 172.21.15.40:0/800826957 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b80aca50 0x7f69b80af290 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f69a400f9d0 tx=0x7f69a400fe90 comp rx=0 tx=0).stop 2023-10-18T09:01:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.148+0000 7f69bd95e640 1 -- 172.21.15.40:0/800826957 shutdown_connections 2023-10-18T09:01:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.148+0000 7f69bd95e640 1 --2- 172.21.15.40:0/800826957 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f699c06ae40 0x7f699c06d300 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:16.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.148+0000 7f69bd95e640 1 --2- 172.21.15.40:0/800826957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b80af7d0 0x7f69b8130200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:16.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.148+0000 7f69bd95e640 1 --2- 172.21.15.40:0/800826957 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b80aca50 0x7f69b80af290 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:16.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.148+0000 7f69bd95e640 1 --2- 172.21.15.40:0/800826957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69a8003970 0x7f69b8158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:16.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.148+0000 7f69bd95e640 1 -- 172.21.15.40:0/800826957 >> 172.21.15.40:0/800826957 conn(0x7f69b8069720 msgr2=0x7f69b80ac010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:16.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.148+0000 7f69bd95e640 1 -- 172.21.15.40:0/800826957 shutdown_connections 2023-10-18T09:01:16.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.148+0000 7f69bd95e640 1 -- 172.21.15.40:0/800826957 wait complete. 2023-10-18T09:01:16.168 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-18T09:01:16.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:16.169 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-18T09:01:16.283 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T09:01:16.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:16.284 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T09:01:16.336 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:01:16.336 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:01:16.400 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:01:16.401 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:01:16.401 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:01:16.461 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:01:16.461 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:01:16.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.701+0000 7f350bed7640 1 -- 172.21.15.40:0/3555455818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35041590f0 msgr2=0x7f350406dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:16.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.701+0000 7f350bed7640 1 --2- 172.21.15.40:0/3555455818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35041590f0 0x7f350406dca0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f3500005fd0 tx=0x7f350002bd90 comp rx=0 tx=0).stop 2023-10-18T09:01:16.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.701+0000 7f350bed7640 1 -- 172.21.15.40:0/3555455818 shutdown_connections 2023-10-18T09:01:16.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.701+0000 7f350bed7640 1 --2- 172.21.15.40:0/3555455818 >> v2:172.21.15.144:3300/0 conn(0x7f350415c0f0 0x7f350406e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:16.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.701+0000 7f350bed7640 1 --2- 172.21.15.40:0/3555455818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35041590f0 0x7f350406dca0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:16.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.702+0000 7f350bed7640 1 -- 172.21.15.40:0/3555455818 >> 172.21.15.40:0/3555455818 conn(0x7f35040655f0 msgr2=0x7f35040659d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:16.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.702+0000 7f350bed7640 1 -- 172.21.15.40:0/3555455818 shutdown_connections 2023-10-18T09:01:16.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.702+0000 7f350bed7640 1 -- 172.21.15.40:0/3555455818 wait complete. 2023-10-18T09:01:16.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.703+0000 7f350bed7640 1 Processor -- start 2023-10-18T09:01:16.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.704+0000 7f350bed7640 1 -- start start 2023-10-18T09:01:16.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.705+0000 7f350bed7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35040cfd70 0x7f35040b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:16.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.705+0000 7f350bed7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35041590f0 0x7f3504145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:16.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.705+0000 7f350bed7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f350415c0f0 0x7f3504145da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:16.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.705+0000 7f350bed7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35040b2240 con 0x7f350415c0f0 2023-10-18T09:01:16.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.705+0000 7f350bed7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3504144620 con 0x7f35041590f0 2023-10-18T09:01:16.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.705+0000 7f350bed7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3504144900 con 0x7f35040cfd70 2023-10-18T09:01:16.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.705+0000 7f350a474640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f350415c0f0 0x7f3504145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:16.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.705+0000 7f350a474640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f350415c0f0 0x7f3504145da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57730/0 (socket says 172.21.15.40:57730) 2023-10-18T09:01:16.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.705+0000 7f350a474640 1 -- 172.21.15.40:0/2353769896 learned_addr learned my addr 172.21.15.40:0/2353769896 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:16.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.705+0000 7f3509472640 1 --2- 172.21.15.40:0/2353769896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35041590f0 0x7f3504145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:16.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.705+0000 7f350a474640 1 -- 172.21.15.40:0/2353769896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35040cfd70 msgr2=0x7f35040b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:16.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.705+0000 7f350a474640 1 --2- 172.21.15.40:0/2353769896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35040cfd70 0x7f35040b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:16.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.705+0000 7f350a474640 1 -- 172.21.15.40:0/2353769896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35041590f0 msgr2=0x7f3504145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:16.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.705+0000 7f350a474640 1 --2- 172.21.15.40:0/2353769896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35041590f0 0x7f3504145860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:16.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.705+0000 7f350a474640 1 -- 172.21.15.40:0/2353769896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3500009560 con 0x7f350415c0f0 2023-10-18T09:01:16.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.706+0000 7f350a474640 1 --2- 172.21.15.40:0/2353769896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f350415c0f0 0x7f3504145da0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00cad0 tx=0x7f34fc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:16.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.706+0000 7f34fa7fc640 1 -- 172.21.15.40:0/2353769896 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc00ae20 con 0x7f350415c0f0 2023-10-18T09:01:16.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.706+0000 7f34fa7fc640 1 -- 172.21.15.40:0/2353769896 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f34fc00f450 con 0x7f350415c0f0 2023-10-18T09:01:16.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.706+0000 7f34fa7fc640 1 -- 172.21.15.40:0/2353769896 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc00f8c0 con 0x7f350415c0f0 2023-10-18T09:01:16.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.706+0000 7f350bed7640 1 -- 172.21.15.40:0/2353769896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3504144be0 con 0x7f350415c0f0 2023-10-18T09:01:16.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.706+0000 7f350bed7640 1 -- 172.21.15.40:0/2353769896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35041451a0 con 0x7f350415c0f0 2023-10-18T09:01:16.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.707+0000 7f350bed7640 1 -- 172.21.15.40:0/2353769896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35040663a0 con 0x7f350415c0f0 2023-10-18T09:01:16.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.708+0000 7f34fa7fc640 1 -- 172.21.15.40:0/2353769896 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f34fc00fa20 con 0x7f350415c0f0 2023-10-18T09:01:16.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.709+0000 7f34fa7fc640 1 --2- 172.21.15.40:0/2353769896 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f34e806ae40 0x7f34e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:16.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.709+0000 7f34fa7fc640 1 -- 172.21.15.40:0/2353769896 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f34fc08a430 con 0x7f350415c0f0 2023-10-18T09:01:16.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.710+0000 7f34fa7fc640 1 -- 172.21.15.40:0/2353769896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=93}) v3 -- 0x7f34e8072b50 con 0x7f350415c0f0 2023-10-18T09:01:16.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.710+0000 7f3509c73640 1 --2- 172.21.15.40:0/2353769896 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f34e806ae40 0x7f34e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:16.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.711+0000 7f3509c73640 1 --2- 172.21.15.40:0/2353769896 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f34e806ae40 0x7f34e806d300 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3500035000 tx=0x7f35000046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:16.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.714+0000 7f34fa7fc640 1 -- 172.21.15.40:0/2353769896 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f34fc01d020 con 0x7f350415c0f0 2023-10-18T09:01:16.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.940+0000 7f350bed7640 1 -- 172.21.15.40:0/2353769896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35041453c0 con 0x7f350415c0f0 2023-10-18T09:01:16.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.941+0000 7f34fa7fc640 1 -- 172.21.15.40:0/2353769896 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f34fc058db0 con 0x7f350415c0f0 2023-10-18T09:01:16.944 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:16.944 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1341351808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1341351808},{"type":"v1","addr":"172.21.15.144:6836","nonce":1341351808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1187670073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1187670073},{"type":"v1","addr":"172.21.15.40:6833","nonce":1187670073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5626,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1615448513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1615448513},{"type":"v1","addr":"172.21.15.144:6837","nonce":1615448513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:00:15.197720+0000","modified":"2023-10-18T09:00:18.306206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1926421699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1926421699},{"type":"v1","addr":"172.21.15.40:6835","nonce":1926421699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-18T09:01:16.945 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 41 2023-10-18T09:01:16.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.944+0000 7f350bed7640 1 -- 172.21.15.40:0/2353769896 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f34e806ae40 msgr2=0x7f34e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:16.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.944+0000 7f350bed7640 1 --2- 172.21.15.40:0/2353769896 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f34e806ae40 0x7f34e806d300 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3500035000 tx=0x7f35000046c0 comp rx=0 tx=0).stop 2023-10-18T09:01:16.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.944+0000 7f350bed7640 1 -- 172.21.15.40:0/2353769896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f350415c0f0 msgr2=0x7f3504145da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:16.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.944+0000 7f350bed7640 1 --2- 172.21.15.40:0/2353769896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f350415c0f0 0x7f3504145da0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00cad0 tx=0x7f34fc00cf90 comp rx=0 tx=0).stop 2023-10-18T09:01:16.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.944+0000 7f350bed7640 1 -- 172.21.15.40:0/2353769896 shutdown_connections 2023-10-18T09:01:16.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.944+0000 7f350bed7640 1 --2- 172.21.15.40:0/2353769896 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f34e806ae40 0x7f34e806d300 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:16.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.944+0000 7f350bed7640 1 --2- 172.21.15.40:0/2353769896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f350415c0f0 0x7f3504145da0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:16.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.944+0000 7f350bed7640 1 --2- 172.21.15.40:0/2353769896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35041590f0 0x7f3504145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:16.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.944+0000 7f350bed7640 1 --2- 172.21.15.40:0/2353769896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35040cfd70 0x7f35040b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:16.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.944+0000 7f350bed7640 1 -- 172.21.15.40:0/2353769896 >> 172.21.15.40:0/2353769896 conn(0x7f35040655f0 msgr2=0x7f350415d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:16.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.944+0000 7f350bed7640 1 -- 172.21.15.40:0/2353769896 shutdown_connections 2023-10-18T09:01:16.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:16.945+0000 7f350bed7640 1 -- 172.21.15.40:0/2353769896 wait complete. 2023-10-18T09:01:16.966 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:01:16.979 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:01:16.977+0000 7f6a11f15640 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2023-10-18T09:01:17.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.174+0000 7f88409ba640 1 -- 172.21.15.40:0/319366241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8828005100 msgr2=0x7f88280058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:17.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.174+0000 7f88409ba640 1 --2- 172.21.15.40:0/319366241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8828005100 0x7f88280058d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f882400bb20 tx=0x7f8828006900 comp rx=0 tx=0).stop 2023-10-18T09:01:17.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.174+0000 7f883b59e640 1 -- 172.21.15.40:0/319366241 reap_dead start 2023-10-18T09:01:17.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.175+0000 7f88409ba640 1 -- 172.21.15.40:0/319366241 shutdown_connections 2023-10-18T09:01:17.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.175+0000 7f88409ba640 1 -- 172.21.15.40:0/319366241 >> 172.21.15.40:0/319366241 conn(0x7f883c069720 msgr2=0x7f883c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:17.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.175+0000 7f88409ba640 1 -- 172.21.15.40:0/319366241 shutdown_connections 2023-10-18T09:01:17.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.175+0000 7f88409ba640 1 -- 172.21.15.40:0/319366241 wait complete. 2023-10-18T09:01:17.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.176+0000 7f88409ba640 1 Processor -- start 2023-10-18T09:01:17.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.177+0000 7f88409ba640 1 -- start start 2023-10-18T09:01:17.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.177+0000 7f88409ba640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8828005100 0x7f883c1381e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:17.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.177+0000 7f88409ba640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883c159d20 0x7f883c13b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:17.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.177+0000 7f88409ba640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f883c1e2f00 0x7f883c1e3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.177+0000 7f88409ba640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c1e38c0 con 0x7f883c1e2f00 2023-10-18T09:01:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.177+0000 7f88409ba640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c1efad0 con 0x7f883c159d20 2023-10-18T09:01:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.177+0000 7f88409ba640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883c13b8a0 con 0x7f8828005100 2023-10-18T09:01:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.178+0000 7f883b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f883c1e2f00 0x7f883c1e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.178+0000 7f883b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f883c1e2f00 0x7f883c1e3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57758/0 (socket says 172.21.15.40:57758) 2023-10-18T09:01:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.178+0000 7f883b59e640 1 -- 172.21.15.40:0/4025983337 learned_addr learned my addr 172.21.15.40:0/4025983337 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.178+0000 7f883a59c640 1 --2- 172.21.15.40:0/4025983337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883c159d20 0x7f883c13b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.178+0000 7f883ad9d640 1 --2- 172.21.15.40:0/4025983337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8828005100 0x7f883c1381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.178+0000 7f883b59e640 1 -- 172.21.15.40:0/4025983337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8828005100 msgr2=0x7f883c1381e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.178+0000 7f883b59e640 1 --2- 172.21.15.40:0/4025983337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8828005100 0x7f883c1381e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:17.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.178+0000 7f883b59e640 1 -- 172.21.15.40:0/4025983337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883c159d20 msgr2=0x7f883c13b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:17.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.178+0000 7f883b59e640 1 --2- 172.21.15.40:0/4025983337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883c159d20 0x7f883c13b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:17.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.178+0000 7f883b59e640 1 -- 172.21.15.40:0/4025983337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c158c80 con 0x7f883c1e2f00 2023-10-18T09:01:17.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.178+0000 7f883a59c640 1 --2- 172.21.15.40:0/4025983337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883c159d20 0x7f883c13b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:17.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.178+0000 7f883ad9d640 1 --2- 172.21.15.40:0/4025983337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8828005100 0x7f883c1381e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.179+0000 7f883b59e640 1 --2- 172.21.15.40:0/4025983337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f883c1e2f00 0x7f883c1e3380 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f883c0caf60 tx=0x7f883400ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.179+0000 7f88337fe640 1 -- 172.21.15.40:0/4025983337 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834018070 con 0x7f883c1e2f00 2023-10-18T09:01:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.179+0000 7f88337fe640 1 -- 172.21.15.40:0/4025983337 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8834004110 con 0x7f883c1e2f00 2023-10-18T09:01:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.179+0000 7f88337fe640 1 -- 172.21.15.40:0/4025983337 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8834005020 con 0x7f883c1e2f00 2023-10-18T09:01:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.179+0000 7f88409ba640 1 -- 172.21.15.40:0/4025983337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f883c158f60 con 0x7f883c1e2f00 2023-10-18T09:01:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.179+0000 7f88409ba640 1 -- 172.21.15.40:0/4025983337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883c159520 con 0x7f883c1e2f00 2023-10-18T09:01:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.180+0000 7f88409ba640 1 -- 172.21.15.40:0/4025983337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f883c066e00 con 0x7f883c1e2f00 2023-10-18T09:01:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.181+0000 7f88337fe640 1 -- 172.21.15.40:0/4025983337 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8834005180 con 0x7f883c1e2f00 2023-10-18T09:01:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.182+0000 7f88337fe640 1 --2- 172.21.15.40:0/4025983337 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f881c06ae70 0x7f881c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.182+0000 7f88337fe640 1 -- 172.21.15.40:0/4025983337 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f88340892b0 con 0x7f883c1e2f00 2023-10-18T09:01:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.182+0000 7f88337fe640 1 -- 172.21.15.40:0/4025983337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=94}) v3 -- 0x7f881c072b80 con 0x7f883c1e2f00 2023-10-18T09:01:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.183+0000 7f883ad9d640 1 --2- 172.21.15.40:0/4025983337 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f881c06ae70 0x7f881c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.184+0000 7f883ad9d640 1 --2- 172.21.15.40:0/4025983337 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f881c06ae70 0x7f881c06d330 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f882c007fd0 tx=0x7f882c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.187+0000 7f88337fe640 1 -- 172.21.15.40:0/4025983337 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8834054360 con 0x7f883c1e2f00 2023-10-18T09:01:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.379+0000 7f88409ba640 1 -- 172.21.15.40:0/4025983337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f883c0cb750 con 0x7f883c1e2f00 2023-10-18T09:01:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.381+0000 7f88337fe640 1 -- 172.21.15.40:0/4025983337 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+16849 (secure 0 0 0) 0x7f8834057980 con 0x7f883c1e2f00 2023-10-18T09:01:17.384 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:17.384 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":93,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:01:16.978273+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:00:13.170937+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-18T09:00:14.205418+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1","mappings":[{"from":6,"to":0}]},{"pgid":"11.8","mappings":[{"from":6,"to":1}]},{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.18","mappings":[{"from":2,"to":0}]},{"pgid":"11.33","mappings":[{"from":6,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:01:17.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.384+0000 7f88409ba640 1 -- 172.21.15.40:0/4025983337 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f881c06ae70 msgr2=0x7f881c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:17.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.384+0000 7f88409ba640 1 --2- 172.21.15.40:0/4025983337 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f881c06ae70 0x7f881c06d330 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f882c007fd0 tx=0x7f882c009040 comp rx=0 tx=0).stop 2023-10-18T09:01:17.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.384+0000 7f88409ba640 1 -- 172.21.15.40:0/4025983337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f883c1e2f00 msgr2=0x7f883c1e3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:17.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.384+0000 7f88409ba640 1 --2- 172.21.15.40:0/4025983337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f883c1e2f00 0x7f883c1e3380 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f883c0caf60 tx=0x7f883400ea30 comp rx=0 tx=0).stop 2023-10-18T09:01:17.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.384+0000 7f88409ba640 1 -- 172.21.15.40:0/4025983337 shutdown_connections 2023-10-18T09:01:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.384+0000 7f88409ba640 1 --2- 172.21.15.40:0/4025983337 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f881c06ae70 0x7f881c06d330 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.384+0000 7f88409ba640 1 --2- 172.21.15.40:0/4025983337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f883c1e2f00 0x7f883c1e3380 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.384+0000 7f88409ba640 1 --2- 172.21.15.40:0/4025983337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f883c159d20 0x7f883c13b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.384+0000 7f88409ba640 1 --2- 172.21.15.40:0/4025983337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8828005100 0x7f883c1381e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.384+0000 7f88409ba640 1 -- 172.21.15.40:0/4025983337 >> 172.21.15.40:0/4025983337 conn(0x7f883c069720 msgr2=0x7f883c0cc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:17.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.385+0000 7f88409ba640 1 -- 172.21.15.40:0/4025983337 shutdown_connections 2023-10-18T09:01:17.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.385+0000 7f88409ba640 1 -- 172.21.15.40:0/4025983337 wait complete. 2023-10-18T09:01:17.406 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:00:13.170937+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:00:14.205418+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:01:17.407 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:01:17.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.687+0000 7fd643cb4640 1 -- 172.21.15.40:0/961113853 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c0af410 msgr2=0x7fd63c0cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:17.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.687+0000 7fd643cb4640 1 --2- 172.21.15.40:0/961113853 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c0af410 0x7fd63c0cd200 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd634009ca0 tx=0x7fd63402c2c0 comp rx=0 tx=0).stop 2023-10-18T09:01:17.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.687+0000 7fd643cb4640 1 -- 172.21.15.40:0/961113853 shutdown_connections 2023-10-18T09:01:17.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.687+0000 7fd643cb4640 1 --2- 172.21.15.40:0/961113853 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c0af410 0x7fd63c0cd200 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:17.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.687+0000 7fd643cb4640 1 --2- 172.21.15.40:0/961113853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd63c06d6f0 0x7fd63c14e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:17.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.687+0000 7fd643cb4640 1 -- 172.21.15.40:0/961113853 >> 172.21.15.40:0/961113853 conn(0x7fd63c069720 msgr2=0x7fd63c069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:17.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.687+0000 7fd643cb4640 1 -- 172.21.15.40:0/961113853 shutdown_connections 2023-10-18T09:01:17.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.687+0000 7fd643cb4640 1 -- 172.21.15.40:0/961113853 wait complete. 2023-10-18T09:01:17.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.689+0000 7fd643cb4640 1 Processor -- start 2023-10-18T09:01:17.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.689+0000 7fd643cb4640 1 -- start start 2023-10-18T09:01:17.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.690+0000 7fd643cb4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c06d6f0 0x7fd63c151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:17.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.690+0000 7fd643cb4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd63c0715c0 0x7fd63c0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:17.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.690+0000 7fd643cb4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c0af410 0x7fd63c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:17.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.690+0000 7fd643cb4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c14edf0 con 0x7fd63c0af410 2023-10-18T09:01:17.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.690+0000 7fd643cb4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c1414e0 con 0x7fd63c06d6f0 2023-10-18T09:01:17.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.690+0000 7fd643cb4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c152180 con 0x7fd63c0715c0 2023-10-18T09:01:17.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.690+0000 7fd642251640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c0af410 0x7fd63c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:17.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.690+0000 7fd642251640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c0af410 0x7fd63c0cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57766/0 (socket says 172.21.15.40:57766) 2023-10-18T09:01:17.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.690+0000 7fd642251640 1 -- 172.21.15.40:0/4259963953 learned_addr learned my addr 172.21.15.40:0/4259963953 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:17.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.690+0000 7fd642251640 1 -- 172.21.15.40:0/4259963953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd63c0715c0 msgr2=0x7fd63c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:17.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.690+0000 7fd642251640 1 --2- 172.21.15.40:0/4259963953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd63c0715c0 0x7fd63c0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:17.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.690+0000 7fd641a50640 1 --2- 172.21.15.40:0/4259963953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c06d6f0 0x7fd63c151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:17.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.690+0000 7fd642251640 1 -- 172.21.15.40:0/4259963953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c06d6f0 msgr2=0x7fd63c151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:17.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.691+0000 7fd642251640 1 --2- 172.21.15.40:0/4259963953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c06d6f0 0x7fd63c151970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:17.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.691+0000 7fd642251640 1 -- 172.21.15.40:0/4259963953 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd634009900 con 0x7fd63c0af410 2023-10-18T09:01:17.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.691+0000 7fd642251640 1 --2- 172.21.15.40:0/4259963953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c0af410 0x7fd63c0cd780 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd634009ca0 tx=0x7fd634003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:17.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.691+0000 7fd6327fc640 1 -- 172.21.15.40:0/4259963953 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634004240 con 0x7fd63c0af410 2023-10-18T09:01:17.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.691+0000 7fd6327fc640 1 -- 172.21.15.40:0/4259963953 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd634004490 con 0x7fd63c0af410 2023-10-18T09:01:17.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.691+0000 7fd6327fc640 1 -- 172.21.15.40:0/4259963953 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634004840 con 0x7fd63c0af410 2023-10-18T09:01:17.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.691+0000 7fd643cb4640 1 -- 172.21.15.40:0/4259963953 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c140bf0 con 0x7fd63c0af410 2023-10-18T09:01:17.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.691+0000 7fd643cb4640 1 -- 172.21.15.40:0/4259963953 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c140e90 con 0x7fd63c0af410 2023-10-18T09:01:17.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.693+0000 7fd643cb4640 1 -- 172.21.15.40:0/4259963953 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c066fd0 con 0x7fd63c0af410 2023-10-18T09:01:17.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.694+0000 7fd6327fc640 1 -- 172.21.15.40:0/4259963953 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd63403b040 con 0x7fd63c0af410 2023-10-18T09:01:17.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.694+0000 7fd6327fc640 1 --2- 172.21.15.40:0/4259963953 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd62006ae40 0x7fd62006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:17.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.695+0000 7fd6327fc640 1 -- 172.21.15.40:0/4259963953 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd6340ac4d0 con 0x7fd63c0af410 2023-10-18T09:01:17.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.695+0000 7fd6327fc640 1 -- 172.21.15.40:0/4259963953 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=94}) v3 -- 0x7fd620072b50 con 0x7fd63c0af410 2023-10-18T09:01:17.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.695+0000 7fd641a50640 1 --2- 172.21.15.40:0/4259963953 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd62006ae40 0x7fd62006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:17.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.696+0000 7fd641a50640 1 --2- 172.21.15.40:0/4259963953 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd62006ae40 0x7fd62006d300 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd6380058c0 tx=0x7fd638005c60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:17.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.699+0000 7fd6327fc640 1 -- 172.21.15.40:0/4259963953 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd634077610 con 0x7fd63c0af410 2023-10-18T09:01:17.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.936+0000 7fd643cb4640 1 -- 172.21.15.40:0/4259963953 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd63c141620 con 0x7fd63c0af410 2023-10-18T09:01:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.936+0000 7fd6327fc640 1 -- 172.21.15.40:0/4259963953 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6340430a0 con 0x7fd63c0af410 2023-10-18T09:01:17.939 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:01:17.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.939+0000 7fd643cb4640 1 -- 172.21.15.40:0/4259963953 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd62006ae40 msgr2=0x7fd62006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:17.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.939+0000 7fd643cb4640 1 --2- 172.21.15.40:0/4259963953 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd62006ae40 0x7fd62006d300 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd6380058c0 tx=0x7fd638005c60 comp rx=0 tx=0).stop 2023-10-18T09:01:17.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.939+0000 7fd643cb4640 1 -- 172.21.15.40:0/4259963953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c0af410 msgr2=0x7fd63c0cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:17.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.939+0000 7fd643cb4640 1 --2- 172.21.15.40:0/4259963953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c0af410 0x7fd63c0cd780 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd634009ca0 tx=0x7fd634003ba0 comp rx=0 tx=0).stop 2023-10-18T09:01:17.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.940+0000 7fd643cb4640 1 -- 172.21.15.40:0/4259963953 shutdown_connections 2023-10-18T09:01:17.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.940+0000 7fd643cb4640 1 --2- 172.21.15.40:0/4259963953 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd62006ae40 0x7fd62006d300 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:17.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.940+0000 7fd643cb4640 1 --2- 172.21.15.40:0/4259963953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd63c0af410 0x7fd63c0cd780 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:17.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.940+0000 7fd643cb4640 1 --2- 172.21.15.40:0/4259963953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd63c0715c0 0x7fd63c0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:17.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.940+0000 7fd643cb4640 1 --2- 172.21.15.40:0/4259963953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd63c06d6f0 0x7fd63c151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:17.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.940+0000 7fd643cb4640 1 -- 172.21.15.40:0/4259963953 >> 172.21.15.40:0/4259963953 conn(0x7fd63c069720 msgr2=0x7fd63c1567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:17.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.940+0000 7fd643cb4640 1 -- 172.21.15.40:0/4259963953 shutdown_connections 2023-10-18T09:01:17.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:17.940+0000 7fd643cb4640 1 -- 172.21.15.40:0/4259963953 wait complete. 2023-10-18T09:01:17.960 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:01:18.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.194+0000 7f66f22f1640 1 -- 172.21.15.40:0/3248133024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66ec15af70 msgr2=0x7f66ec070b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:18.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.194+0000 7f66f22f1640 1 --2- 172.21.15.40:0/3248133024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66ec15af70 0x7f66ec070b60 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f66dc005fd0 tx=0x7f66dc02bd10 comp rx=0 tx=0).stop 2023-10-18T09:01:18.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.194+0000 7f66f22f1640 1 -- 172.21.15.40:0/3248133024 shutdown_connections 2023-10-18T09:01:18.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.194+0000 7f66f22f1640 1 --2- 172.21.15.40:0/3248133024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66ec15af70 0x7f66ec070b60 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:18.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.194+0000 7f66f22f1640 1 --2- 172.21.15.40:0/3248133024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66ec06d5d0 0x7f66ec0c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.194+0000 7f66f22f1640 1 -- 172.21.15.40:0/3248133024 >> 172.21.15.40:0/3248133024 conn(0x7f66ec06ca10 msgr2=0x7f66ec06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.194+0000 7f66f22f1640 1 -- 172.21.15.40:0/3248133024 shutdown_connections 2023-10-18T09:01:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.194+0000 7f66f22f1640 1 -- 172.21.15.40:0/3248133024 wait complete. 2023-10-18T09:01:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.196+0000 7f66f22f1640 1 Processor -- start 2023-10-18T09:01:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.196+0000 7f66f22f1640 1 -- start start 2023-10-18T09:01:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.197+0000 7f66f22f1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66ec06d5d0 0x7f66ec13fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.197+0000 7f66f22f1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66ec0710a0 0x7f66ec155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.197+0000 7f66f22f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66ec15af70 0x7f66ec14d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.197+0000 7f66f22f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec1559f0 con 0x7f66ec15af70 2023-10-18T09:01:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.197+0000 7f66f22f1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec140000 con 0x7f66ec06d5d0 2023-10-18T09:01:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.197+0000 7f66f22f1640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec14e580 con 0x7f66ec0710a0 2023-10-18T09:01:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.197+0000 7f66f1af0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66ec15af70 0x7f66ec14d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.197+0000 7f66f1af0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66ec15af70 0x7f66ec14d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57796/0 (socket says 172.21.15.40:57796) 2023-10-18T09:01:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.197+0000 7f66f1af0640 1 -- 172.21.15.40:0/1761342586 learned_addr learned my addr 172.21.15.40:0/1761342586 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:18.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.198+0000 7f66f0aee640 1 --2- 172.21.15.40:0/1761342586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66ec0710a0 0x7f66ec155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.198+0000 7f66f12ef640 1 --2- 172.21.15.40:0/1761342586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66ec06d5d0 0x7f66ec13fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.198+0000 7f66f1af0640 1 -- 172.21.15.40:0/1761342586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66ec0710a0 msgr2=0x7f66ec155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.198+0000 7f66f1af0640 1 --2- 172.21.15.40:0/1761342586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66ec0710a0 0x7f66ec155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.198+0000 7f66f1af0640 1 -- 172.21.15.40:0/1761342586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66ec06d5d0 msgr2=0x7f66ec13fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.198+0000 7f66f1af0640 1 --2- 172.21.15.40:0/1761342586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66ec06d5d0 0x7f66ec13fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.198+0000 7f66f1af0640 1 -- 172.21.15.40:0/1761342586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc009560 con 0x7f66ec15af70 2023-10-18T09:01:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.198+0000 7f66f12ef640 1 --2- 172.21.15.40:0/1761342586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66ec06d5d0 0x7f66ec13fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.198+0000 7f66f1af0640 1 --2- 172.21.15.40:0/1761342586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66ec15af70 0x7f66ec14d470 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f66e400c780 tx=0x7f66e400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.198+0000 7f66e1ffb640 1 -- 172.21.15.40:0/1761342586 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4003d80 con 0x7f66ec15af70 2023-10-18T09:01:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.198+0000 7f66e1ffb640 1 -- 172.21.15.40:0/1761342586 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f66e400ce90 con 0x7f66ec15af70 2023-10-18T09:01:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.198+0000 7f66e1ffb640 1 -- 172.21.15.40:0/1761342586 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e40052d0 con 0x7f66ec15af70 2023-10-18T09:01:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.198+0000 7f66f22f1640 1 -- 172.21.15.40:0/1761342586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66ec14dfc0 con 0x7f66ec15af70 2023-10-18T09:01:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.198+0000 7f66f22f1640 1 -- 172.21.15.40:0/1761342586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66ec155610 con 0x7f66ec15af70 2023-10-18T09:01:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.200+0000 7f66f22f1640 1 -- 172.21.15.40:0/1761342586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66ec155b30 con 0x7f66ec15af70 2023-10-18T09:01:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.201+0000 7f66e1ffb640 1 -- 172.21.15.40:0/1761342586 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f66e4005430 con 0x7f66ec15af70 2023-10-18T09:01:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.202+0000 7f66e1ffb640 1 --2- 172.21.15.40:0/1761342586 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f66d006ae40 0x7f66d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.202+0000 7f66e1ffb640 1 -- 172.21.15.40:0/1761342586 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f66e4088de0 con 0x7f66ec15af70 2023-10-18T09:01:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.202+0000 7f66e1ffb640 1 -- 172.21.15.40:0/1761342586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=95}) v3 -- 0x7f66d0072b50 con 0x7f66ec15af70 2023-10-18T09:01:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.202+0000 7f66f12ef640 1 --2- 172.21.15.40:0/1761342586 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f66d006ae40 0x7f66d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:18.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.203+0000 7f66f12ef640 1 --2- 172.21.15.40:0/1761342586 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f66d006ae40 0x7f66d006d300 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f66e8006fd0 tx=0x7f66e8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:18.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.206+0000 7f66e1ffb640 1 -- 172.21.15.40:0/1761342586 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66e4017020 con 0x7f66ec15af70 2023-10-18T09:01:18.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.396+0000 7f66f22f1640 1 -- 172.21.15.40:0/1761342586 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f66ec066e00 con 0x7f66ec15af70 2023-10-18T09:01:18.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.397+0000 7f66e1ffb640 1 -- 172.21.15.40:0/1761342586 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f66ec155b30 con 0x7f66ec15af70 2023-10-18T09:01:18.400 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:01:18.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.400+0000 7f66f22f1640 1 -- 172.21.15.40:0/1761342586 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f66d006ae40 msgr2=0x7f66d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:18.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.400+0000 7f66f22f1640 1 --2- 172.21.15.40:0/1761342586 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f66d006ae40 0x7f66d006d300 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f66e8006fd0 tx=0x7f66e8008040 comp rx=0 tx=0).stop 2023-10-18T09:01:18.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.400+0000 7f66f22f1640 1 -- 172.21.15.40:0/1761342586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66ec15af70 msgr2=0x7f66ec14d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:18.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.400+0000 7f66f22f1640 1 --2- 172.21.15.40:0/1761342586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66ec15af70 0x7f66ec14d470 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f66e400c780 tx=0x7f66e400cc40 comp rx=0 tx=0).stop 2023-10-18T09:01:18.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.400+0000 7f66f22f1640 1 -- 172.21.15.40:0/1761342586 shutdown_connections 2023-10-18T09:01:18.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.400+0000 7f66f22f1640 1 --2- 172.21.15.40:0/1761342586 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f66d006ae40 0x7f66d006d300 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:18.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.400+0000 7f66f22f1640 1 --2- 172.21.15.40:0/1761342586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66ec15af70 0x7f66ec14d470 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:18.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.400+0000 7f66f22f1640 1 --2- 172.21.15.40:0/1761342586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66ec0710a0 0x7f66ec155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:18.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.400+0000 7f66f22f1640 1 --2- 172.21.15.40:0/1761342586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66ec06d5d0 0x7f66ec13fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:18.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.400+0000 7f66f22f1640 1 -- 172.21.15.40:0/1761342586 >> 172.21.15.40:0/1761342586 conn(0x7f66ec06ca10 msgr2=0x7f66ec0b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:18.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.400+0000 7f66f22f1640 1 -- 172.21.15.40:0/1761342586 shutdown_connections 2023-10-18T09:01:18.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.400+0000 7f66f22f1640 1 -- 172.21.15.40:0/1761342586 wait complete. 2023-10-18T09:01:18.421 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:01:18.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.652+0000 7f21eef3c640 1 -- 172.21.15.40:0/1598513502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e815b380 msgr2=0x7f21e80b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:18.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.652+0000 7f21eef3c640 1 --2- 172.21.15.40:0/1598513502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e815b380 0x7f21e80b1120 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f21e4005fd0 tx=0x7f21e402be20 comp rx=0 tx=0).stop 2023-10-18T09:01:18.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.652+0000 7f21eef3c640 1 -- 172.21.15.40:0/1598513502 shutdown_connections 2023-10-18T09:01:18.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.652+0000 7f21eef3c640 1 --2- 172.21.15.40:0/1598513502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e80d1580 0x7f21e80cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:18.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.652+0000 7f21eef3c640 1 --2- 172.21.15.40:0/1598513502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e815b380 0x7f21e80b1120 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:18.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.652+0000 7f21eef3c640 1 -- 172.21.15.40:0/1598513502 >> 172.21.15.40:0/1598513502 conn(0x7f21e808acb0 msgr2=0x7f21e809a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:18.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.653+0000 7f21eef3c640 1 -- 172.21.15.40:0/1598513502 shutdown_connections 2023-10-18T09:01:18.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.653+0000 7f21eef3c640 1 -- 172.21.15.40:0/1598513502 wait complete. 2023-10-18T09:01:18.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.654+0000 7f21eef3c640 1 Processor -- start 2023-10-18T09:01:18.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.655+0000 7f21eef3c640 1 -- start start 2023-10-18T09:01:18.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.655+0000 7f21eef3c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e80d1580 0x7f21e80c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:18.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.656+0000 7f21eef3c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e815b380 0x7f21e81f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:18.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.656+0000 7f21eef3c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e815b760 0x7f21e81562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:18.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.656+0000 7f21eef3c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e81f1ad0 con 0x7f21e815b760 2023-10-18T09:01:18.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.656+0000 7f21eef3c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e81f5600 con 0x7f21e815b380 2023-10-18T09:01:18.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.656+0000 7f21eef3c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e81f58d0 con 0x7f21e80d1580 2023-10-18T09:01:18.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.656+0000 7f21edf3a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e80d1580 0x7f21e80c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:18.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.656+0000 7f21ee73b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e815b760 0x7f21e81562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:18.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.656+0000 7f21ee73b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e815b760 0x7f21e81562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57826/0 (socket says 172.21.15.40:57826) 2023-10-18T09:01:18.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.656+0000 7f21ee73b640 1 -- 172.21.15.40:0/2481992179 learned_addr learned my addr 172.21.15.40:0/2481992179 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:18.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.656+0000 7f21ed739640 1 --2- 172.21.15.40:0/2481992179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e815b380 0x7f21e81f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:18.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.657+0000 7f21ee73b640 1 -- 172.21.15.40:0/2481992179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e80d1580 msgr2=0x7f21e80c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:18.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.657+0000 7f21ee73b640 1 --2- 172.21.15.40:0/2481992179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e80d1580 0x7f21e80c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:18.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.657+0000 7f21ee73b640 1 -- 172.21.15.40:0/2481992179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e815b380 msgr2=0x7f21e81f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:18.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.657+0000 7f21ee73b640 1 --2- 172.21.15.40:0/2481992179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e815b380 0x7f21e81f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:18.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.657+0000 7f21ee73b640 1 -- 172.21.15.40:0/2481992179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e4009560 con 0x7f21e815b760 2023-10-18T09:01:18.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.657+0000 7f21ed739640 1 --2- 172.21.15.40:0/2481992179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e815b380 0x7f21e81f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:01:18.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.657+0000 7f21ee73b640 1 --2- 172.21.15.40:0/2481992179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e815b760 0x7f21e81562a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f21e000cce0 tx=0x7f21e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:18.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.657+0000 7f21de7fc640 1 -- 172.21.15.40:0/2481992179 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e000f450 con 0x7f21e815b760 2023-10-18T09:01:18.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.657+0000 7f21de7fc640 1 -- 172.21.15.40:0/2481992179 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f21e000f6c0 con 0x7f21e815b760 2023-10-18T09:01:18.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.657+0000 7f21de7fc640 1 -- 172.21.15.40:0/2481992179 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e000fad0 con 0x7f21e815b760 2023-10-18T09:01:18.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.657+0000 7f21eef3c640 1 -- 172.21.15.40:0/2481992179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e81f5bb0 con 0x7f21e815b760 2023-10-18T09:01:18.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.657+0000 7f21eef3c640 1 -- 172.21.15.40:0/2481992179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e81f1d90 con 0x7f21e815b760 2023-10-18T09:01:18.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.658+0000 7f21eef3c640 1 -- 172.21.15.40:0/2481992179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e8071da0 con 0x7f21e815b760 2023-10-18T09:01:18.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.659+0000 7f21de7fc640 1 -- 172.21.15.40:0/2481992179 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f21e000fc30 con 0x7f21e815b760 2023-10-18T09:01:18.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.660+0000 7f21de7fc640 1 --2- 172.21.15.40:0/2481992179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f21cc06ae70 0x7f21cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:18.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.660+0000 7f21de7fc640 1 -- 172.21.15.40:0/2481992179 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f21e008a2a0 con 0x7f21e815b760 2023-10-18T09:01:18.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.660+0000 7f21de7fc640 1 -- 172.21.15.40:0/2481992179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=95}) v3 -- 0x7f21cc072b80 con 0x7f21e815b760 2023-10-18T09:01:18.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.661+0000 7f21edf3a640 1 --2- 172.21.15.40:0/2481992179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f21cc06ae70 0x7f21cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:18.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.662+0000 7f21edf3a640 1 --2- 172.21.15.40:0/2481992179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f21cc06ae70 0x7f21cc06d330 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f21e4035000 tx=0x7f21e4004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:18.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.664+0000 7f21de7fc640 1 -- 172.21.15.40:0/2481992179 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21e00552b0 con 0x7f21e815b760 2023-10-18T09:01:18.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.858+0000 7f21eef3c640 1 -- 172.21.15.40:0/2481992179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f21e81f1fb0 con 0x7f21e815b760 2023-10-18T09:01:18.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.859+0000 7f21de7fc640 1 -- 172.21.15.40:0/2481992179 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f21e001d070 con 0x7f21e815b760 2023-10-18T09:01:18.862 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:01:18.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.861+0000 7f21eef3c640 1 -- 172.21.15.40:0/2481992179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f21cc06ae70 msgr2=0x7f21cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:18.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.861+0000 7f21eef3c640 1 --2- 172.21.15.40:0/2481992179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f21cc06ae70 0x7f21cc06d330 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f21e4035000 tx=0x7f21e4004730 comp rx=0 tx=0).stop 2023-10-18T09:01:18.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.861+0000 7f21eef3c640 1 -- 172.21.15.40:0/2481992179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e815b760 msgr2=0x7f21e81562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:18.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.861+0000 7f21eef3c640 1 --2- 172.21.15.40:0/2481992179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e815b760 0x7f21e81562a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f21e000cce0 tx=0x7f21e000a620 comp rx=0 tx=0).stop 2023-10-18T09:01:18.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.861+0000 7f21eef3c640 1 -- 172.21.15.40:0/2481992179 shutdown_connections 2023-10-18T09:01:18.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.861+0000 7f21eef3c640 1 --2- 172.21.15.40:0/2481992179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f21cc06ae70 0x7f21cc06d330 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:18.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.861+0000 7f21eef3c640 1 --2- 172.21.15.40:0/2481992179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21e815b760 0x7f21e81562a0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:18.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.861+0000 7f21eef3c640 1 --2- 172.21.15.40:0/2481992179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e815b380 0x7f21e81f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:18.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.861+0000 7f21eef3c640 1 --2- 172.21.15.40:0/2481992179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e80d1580 0x7f21e80c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:18.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.861+0000 7f21eef3c640 1 -- 172.21.15.40:0/2481992179 >> 172.21.15.40:0/2481992179 conn(0x7f21e808acb0 msgr2=0x7f21e80ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:18.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.862+0000 7f21eef3c640 1 -- 172.21.15.40:0/2481992179 shutdown_connections 2023-10-18T09:01:18.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:18.862+0000 7f21eef3c640 1 -- 172.21.15.40:0/2481992179 wait complete. 2023-10-18T09:01:18.884 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:01:19.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.103+0000 7f7875683640 1 -- 172.21.15.40:0/3979985871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78700b15b0 msgr2=0x7f78700c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:19.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.103+0000 7f7875683640 1 --2- 172.21.15.40:0/3979985871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78700b15b0 0x7f78700c99c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f785800a940 tx=0x7f785802d090 comp rx=0 tx=0).stop 2023-10-18T09:01:19.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.103+0000 7f7875683640 1 -- 172.21.15.40:0/3979985871 shutdown_connections 2023-10-18T09:01:19.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.103+0000 7f7875683640 1 --2- 172.21.15.40:0/3979985871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78700b15b0 0x7f78700c99c0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:19.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.103+0000 7f7875683640 1 -- 172.21.15.40:0/3979985871 >> 172.21.15.40:0/3979985871 conn(0x7f7870069720 msgr2=0x7f7870069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:19.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.103+0000 7f7875683640 1 -- 172.21.15.40:0/3979985871 shutdown_connections 2023-10-18T09:01:19.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.103+0000 7f7875683640 1 -- 172.21.15.40:0/3979985871 wait complete. 2023-10-18T09:01:19.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.105+0000 7f7875683640 1 Processor -- start 2023-10-18T09:01:19.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.105+0000 7f7875683640 1 -- start start 2023-10-18T09:01:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.106+0000 7f7875683640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f787006d6f0 0x7f78701e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.106+0000 7f7875683640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78700b15b0 0x7f7870149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.106+0000 7f7875683640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78700c9f00 0x7f78700cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.106+0000 7f7875683640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7870149b80 con 0x7f787006d6f0 2023-10-18T09:01:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.106+0000 7f7875683640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78701e6cc0 con 0x7f78700b15b0 2023-10-18T09:01:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.106+0000 7f7875683640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78701e5c20 con 0x7f78700c9f00 2023-10-18T09:01:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.106+0000 7f786ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f787006d6f0 0x7f78701e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.106+0000 7f786ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f787006d6f0 0x7f78701e4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57846/0 (socket says 172.21.15.40:57846) 2023-10-18T09:01:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.106+0000 7f786ffff640 1 -- 172.21.15.40:0/738288505 learned_addr learned my addr 172.21.15.40:0/738288505 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.107+0000 7f786ffff640 1 -- 172.21.15.40:0/738288505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78700c9f00 msgr2=0x7f78700cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:19.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.107+0000 7f786ffff640 1 --2- 172.21.15.40:0/738288505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78700c9f00 0x7f78700cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:19.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.107+0000 7f786ffff640 1 -- 172.21.15.40:0/738288505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78700b15b0 msgr2=0x7f7870149640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:19.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.107+0000 7f786ffff640 1 --2- 172.21.15.40:0/738288505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78700b15b0 0x7f7870149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:19.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.107+0000 7f786ffff640 1 -- 172.21.15.40:0/738288505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785800a5a0 con 0x7f787006d6f0 2023-10-18T09:01:19.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.107+0000 7f786ffff640 1 --2- 172.21.15.40:0/738288505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f787006d6f0 0x7f78701e4d40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f786000c4a0 tx=0x7f786000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:19.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.107+0000 7f786d7fa640 1 -- 172.21.15.40:0/738288505 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860012830 con 0x7f787006d6f0 2023-10-18T09:01:19.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.107+0000 7f786d7fa640 1 -- 172.21.15.40:0/738288505 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7860012a80 con 0x7f787006d6f0 2023-10-18T09:01:19.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.107+0000 7f786d7fa640 1 -- 172.21.15.40:0/738288505 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7860012e90 con 0x7f787006d6f0 2023-10-18T09:01:19.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.108+0000 7f7875683640 1 -- 172.21.15.40:0/738288505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78701e5f00 con 0x7f787006d6f0 2023-10-18T09:01:19.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.108+0000 7f7875683640 1 -- 172.21.15.40:0/738288505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78701e64c0 con 0x7f787006d6f0 2023-10-18T09:01:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.109+0000 7f7875683640 1 -- 172.21.15.40:0/738288505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787015bea0 con 0x7f787006d6f0 2023-10-18T09:01:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.110+0000 7f786d7fa640 1 -- 172.21.15.40:0/738288505 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f786001c460 con 0x7f787006d6f0 2023-10-18T09:01:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.111+0000 7f786d7fa640 1 --2- 172.21.15.40:0/738288505 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f785006ae40 0x7f785006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.111+0000 7f786d7fa640 1 -- 172.21.15.40:0/738288505 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f78600897f0 con 0x7f787006d6f0 2023-10-18T09:01:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.111+0000 7f786d7fa640 1 -- 172.21.15.40:0/738288505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=95}) v3 -- 0x7f7850072b50 con 0x7f787006d6f0 2023-10-18T09:01:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.112+0000 7f7867fff640 1 --2- 172.21.15.40:0/738288505 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f785006ae40 0x7f785006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.113+0000 7f7867fff640 1 --2- 172.21.15.40:0/738288505 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f785006ae40 0x7f785006d300 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7858037000 tx=0x7f78580046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.115+0000 7f786d7fa640 1 -- 172.21.15.40:0/738288505 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f787015bea0 con 0x7f787006d6f0 2023-10-18T09:01:19.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.340+0000 7f7875683640 1 -- 172.21.15.40:0/738288505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f787015bea0 con 0x7f787006d6f0 2023-10-18T09:01:19.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.342+0000 7f786d7fa640 1 -- 172.21.15.40:0/738288505 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f787015bea0 con 0x7f787006d6f0 2023-10-18T09:01:19.344 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:19.345 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1341351808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1341351808},{"type":"v1","addr":"172.21.15.144:6836","nonce":1341351808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1187670073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1187670073},{"type":"v1","addr":"172.21.15.40:6833","nonce":1187670073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5626,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1615448513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1615448513},{"type":"v1","addr":"172.21.15.144:6837","nonce":1615448513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:00:15.197720+0000","modified":"2023-10-18T09:00:18.306206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1926421699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1926421699},{"type":"v1","addr":"172.21.15.40:6835","nonce":1926421699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-18T09:01:19.345 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 41 2023-10-18T09:01:19.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.344+0000 7f7875683640 1 -- 172.21.15.40:0/738288505 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f785006ae40 msgr2=0x7f785006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:19.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.344+0000 7f7875683640 1 --2- 172.21.15.40:0/738288505 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f785006ae40 0x7f785006d300 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7858037000 tx=0x7f78580046c0 comp rx=0 tx=0).stop 2023-10-18T09:01:19.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.344+0000 7f7875683640 1 -- 172.21.15.40:0/738288505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f787006d6f0 msgr2=0x7f78701e4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:19.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.344+0000 7f7875683640 1 --2- 172.21.15.40:0/738288505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f787006d6f0 0x7f78701e4d40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f786000c4a0 tx=0x7f786000c960 comp rx=0 tx=0).stop 2023-10-18T09:01:19.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.344+0000 7f7875683640 1 -- 172.21.15.40:0/738288505 shutdown_connections 2023-10-18T09:01:19.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.344+0000 7f7875683640 1 --2- 172.21.15.40:0/738288505 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f785006ae40 0x7f785006d300 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:19.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.344+0000 7f7875683640 1 --2- 172.21.15.40:0/738288505 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78700c9f00 0x7f78700cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:19.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.344+0000 7f7875683640 1 --2- 172.21.15.40:0/738288505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78700b15b0 0x7f7870149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:19.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.344+0000 7f7875683640 1 --2- 172.21.15.40:0/738288505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f787006d6f0 0x7f78701e4d40 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:19.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.344+0000 7f7875683640 1 -- 172.21.15.40:0/738288505 >> 172.21.15.40:0/738288505 conn(0x7f7870069720 msgr2=0x7f7870154810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:19.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.345+0000 7f7875683640 1 -- 172.21.15.40:0/738288505 shutdown_connections 2023-10-18T09:01:19.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.345+0000 7f7875683640 1 -- 172.21.15.40:0/738288505 wait complete. 2023-10-18T09:01:19.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:01:19.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.587+0000 7fabb51b0640 1 -- 172.21.15.40:0/3714469705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabb015a470 msgr2=0x7fabb00c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:19.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.587+0000 7fabb51b0640 1 --2- 172.21.15.40:0/3714469705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabb015a470 0x7fabb00c9b00 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fab98005fd0 tx=0x7fab9802be20 comp rx=0 tx=0).stop 2023-10-18T09:01:19.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.588+0000 7fabb51b0640 1 -- 172.21.15.40:0/3714469705 shutdown_connections 2023-10-18T09:01:19.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.588+0000 7fabb51b0640 1 --2- 172.21.15.40:0/3714469705 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabb015a470 0x7fabb00c9b00 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:19.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.588+0000 7fabb51b0640 1 -- 172.21.15.40:0/3714469705 >> 172.21.15.40:0/3714469705 conn(0x7fabb006cee0 msgr2=0x7fabb0069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:19.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.588+0000 7fabb51b0640 1 -- 172.21.15.40:0/3714469705 shutdown_connections 2023-10-18T09:01:19.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.588+0000 7fabb51b0640 1 -- 172.21.15.40:0/3714469705 wait complete. 2023-10-18T09:01:19.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.590+0000 7fabb51b0640 1 Processor -- start 2023-10-18T09:01:19.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.590+0000 7fabb51b0640 1 -- start start 2023-10-18T09:01:19.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.591+0000 7fabb51b0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabb006b710 0x7fabb0144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.591+0000 7fabb51b0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb00cdbb0 0x7fabb01518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.591+0000 7fabb51b0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabb015a470 0x7fabb00cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.591+0000 7fabb51b0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb014e0a0 con 0x7fabb015a470 2023-10-18T09:01:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.591+0000 7fabb51b0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb014d490 con 0x7fabb00cdbb0 2023-10-18T09:01:19.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.591+0000 7fabb51b0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb014d770 con 0x7fabb006b710 2023-10-18T09:01:19.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.591+0000 7fabaf59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabb015a470 0x7fabb00cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:19.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.591+0000 7fabaf59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabb015a470 0x7fabb00cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57866/0 (socket says 172.21.15.40:57866) 2023-10-18T09:01:19.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.591+0000 7fabaf59e640 1 -- 172.21.15.40:0/4216367375 learned_addr learned my addr 172.21.15.40:0/4216367375 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:19.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.591+0000 7fabaed9d640 1 --2- 172.21.15.40:0/4216367375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabb006b710 0x7fabb0144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:19.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.591+0000 7faba659c640 1 --2- 172.21.15.40:0/4216367375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb00cdbb0 0x7fabb01518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:19.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.592+0000 7fabaf59e640 1 -- 172.21.15.40:0/4216367375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabb006b710 msgr2=0x7fabb0144d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.592+0000 7fabaf59e640 1 --2- 172.21.15.40:0/4216367375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabb006b710 0x7fabb0144d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.592+0000 7fabaf59e640 1 -- 172.21.15.40:0/4216367375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb00cdbb0 msgr2=0x7fabb01518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.592+0000 7fabaf59e640 1 --2- 172.21.15.40:0/4216367375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb00cdbb0 0x7fabb01518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.592+0000 7fabaf59e640 1 -- 172.21.15.40:0/4216367375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab98009560 con 0x7fabb015a470 2023-10-18T09:01:19.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.592+0000 7faba659c640 1 --2- 172.21.15.40:0/4216367375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb00cdbb0 0x7fabb01518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.592+0000 7fabaf59e640 1 --2- 172.21.15.40:0/4216367375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabb015a470 0x7fabb00cbc20 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7faba8001560 tx=0x7faba800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.592+0000 7faba7fff640 1 -- 172.21.15.40:0/4216367375 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8003da0 con 0x7fabb015a470 2023-10-18T09:01:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.592+0000 7faba7fff640 1 -- 172.21.15.40:0/4216367375 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faba8005070 con 0x7fabb015a470 2023-10-18T09:01:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.592+0000 7faba7fff640 1 -- 172.21.15.40:0/4216367375 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8005460 con 0x7fabb015a470 2023-10-18T09:01:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.592+0000 7fabb51b0640 1 -- 172.21.15.40:0/4216367375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb014da50 con 0x7fabb015a470 2023-10-18T09:01:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.592+0000 7fabb51b0640 1 -- 172.21.15.40:0/4216367375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb0145240 con 0x7fabb015a470 2023-10-18T09:01:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.593+0000 7fabb51b0640 1 -- 172.21.15.40:0/4216367375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb0066fd0 con 0x7fabb015a470 2023-10-18T09:01:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.595+0000 7faba7fff640 1 -- 172.21.15.40:0/4216367375 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faba80055c0 con 0x7fabb015a470 2023-10-18T09:01:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.596+0000 7faba7fff640 1 --2- 172.21.15.40:0/4216367375 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fab9006ae70 0x7fab9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.596+0000 7faba7fff640 1 -- 172.21.15.40:0/4216367375 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5854+0+0 (secure 0 0 0) 0x7faba8089110 con 0x7fabb015a470 2023-10-18T09:01:19.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.596+0000 7faba7fff640 1 -- 172.21.15.40:0/4216367375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=95}) v3 -- 0x7fab90072b80 con 0x7fabb015a470 2023-10-18T09:01:19.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.596+0000 7fabaed9d640 1 --2- 172.21.15.40:0/4216367375 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fab9006ae70 0x7fab9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:19.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.597+0000 7fabaed9d640 1 --2- 172.21.15.40:0/4216367375 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fab9006ae70 0x7fab9006d330 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fabb00c9390 tx=0x7faba0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:19.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.600+0000 7faba7fff640 1 -- 172.21.15.40:0/4216367375 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faba8017070 con 0x7fabb015a470 2023-10-18T09:01:19.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.798+0000 7fabb51b0640 1 -- 172.21.15.40:0/4216367375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabb0159af0 con 0x7fabb015a470 2023-10-18T09:01:19.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.801+0000 7faba7fff640 1 -- 172.21.15.40:0/4216367375 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16842 (secure 0 0 0) 0x7faba80541c0 con 0x7fabb015a470 2023-10-18T09:01:19.804 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:19.804 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":94,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:01:18.125513+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:00:13.170937+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-18T09:00:14.205418+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1","mappings":[{"from":6,"to":0}]},{"pgid":"11.8","mappings":[{"from":6,"to":1}]},{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.18","mappings":[{"from":2,"to":0}]},{"pgid":"11.33","mappings":[{"from":6,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:01:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.803+0000 7fabb51b0640 1 -- 172.21.15.40:0/4216367375 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fab9006ae70 msgr2=0x7fab9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.803+0000 7fabb51b0640 1 --2- 172.21.15.40:0/4216367375 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fab9006ae70 0x7fab9006d330 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fabb00c9390 tx=0x7faba0009040 comp rx=0 tx=0).stop 2023-10-18T09:01:19.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.804+0000 7fabb51b0640 1 -- 172.21.15.40:0/4216367375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabb015a470 msgr2=0x7fabb00cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:19.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.804+0000 7fabb51b0640 1 --2- 172.21.15.40:0/4216367375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabb015a470 0x7fabb00cbc20 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7faba8001560 tx=0x7faba800cc70 comp rx=0 tx=0).stop 2023-10-18T09:01:19.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.804+0000 7fabb51b0640 1 -- 172.21.15.40:0/4216367375 shutdown_connections 2023-10-18T09:01:19.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.804+0000 7fabb51b0640 1 --2- 172.21.15.40:0/4216367375 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fab9006ae70 0x7fab9006d330 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:19.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.804+0000 7fabb51b0640 1 --2- 172.21.15.40:0/4216367375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabb015a470 0x7fabb00cbc20 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:19.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.804+0000 7fabb51b0640 1 --2- 172.21.15.40:0/4216367375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabb00cdbb0 0x7fabb01518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:19.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.804+0000 7fabb51b0640 1 --2- 172.21.15.40:0/4216367375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabb006b710 0x7fabb0144d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:19.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.804+0000 7fabb51b0640 1 -- 172.21.15.40:0/4216367375 >> 172.21.15.40:0/4216367375 conn(0x7fabb006cee0 msgr2=0x7fabb0069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:19.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.804+0000 7fabb51b0640 1 -- 172.21.15.40:0/4216367375 shutdown_connections 2023-10-18T09:01:19.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:19.804+0000 7fabb51b0640 1 -- 172.21.15.40:0/4216367375 wait complete. 2023-10-18T09:01:19.826 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T09:01:19.827 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:01:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.057+0000 7f9739b75640 1 -- 172.21.15.40:0/2707583016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97340b1f60 msgr2=0x7f9734157460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.057+0000 7f9739b75640 1 --2- 172.21.15.40:0/2707583016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97340b1f60 0x7f9734157460 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f972c005fd0 tx=0x7f972c02d130 comp rx=0 tx=0).stop 2023-10-18T09:01:20.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.057+0000 7f9739b75640 1 -- 172.21.15.40:0/2707583016 shutdown_connections 2023-10-18T09:01:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.057+0000 7f9739b75640 1 --2- 172.21.15.40:0/2707583016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97340b1f60 0x7f9734157460 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.058+0000 7f9739b75640 1 --2- 172.21.15.40:0/2707583016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97340cc6b0 0x7f9734156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.058+0000 7f9739b75640 1 -- 172.21.15.40:0/2707583016 >> 172.21.15.40:0/2707583016 conn(0x7f9734069720 msgr2=0x7f9734069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.058+0000 7f9739b75640 1 -- 172.21.15.40:0/2707583016 shutdown_connections 2023-10-18T09:01:20.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.058+0000 7f9739b75640 1 -- 172.21.15.40:0/2707583016 wait complete. 2023-10-18T09:01:20.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.059+0000 7f9739b75640 1 Processor -- start 2023-10-18T09:01:20.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.060+0000 7f9739b75640 1 -- start start 2023-10-18T09:01:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.060+0000 7f9739b75640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97340b1b80 0x7f9734151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.060+0000 7f9739b75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97340b1f60 0x7f9734149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.060+0000 7f9739b75640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97340cc6b0 0x7f9734153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.060+0000 7f9739b75640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973414f550 con 0x7f97340b1f60 2023-10-18T09:01:20.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.060+0000 7f9739b75640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973414f830 con 0x7f97340b1b80 2023-10-18T09:01:20.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.060+0000 7f9739b75640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973414fb10 con 0x7f97340cc6b0 2023-10-18T09:01:20.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.061+0000 7f972bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97340b1f60 0x7f9734149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:20.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.061+0000 7f972bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97340b1f60 0x7f9734149240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57886/0 (socket says 172.21.15.40:57886) 2023-10-18T09:01:20.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.061+0000 7f972bfff640 1 -- 172.21.15.40:0/3475544637 learned_addr learned my addr 172.21.15.40:0/3475544637 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:20.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.061+0000 7f9738b73640 1 --2- 172.21.15.40:0/3475544637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97340b1b80 0x7f9734151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:20.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.061+0000 7f9739374640 1 --2- 172.21.15.40:0/3475544637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97340cc6b0 0x7f9734153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:20.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.061+0000 7f972bfff640 1 -- 172.21.15.40:0/3475544637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97340cc6b0 msgr2=0x7f9734153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:20.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.061+0000 7f972bfff640 1 --2- 172.21.15.40:0/3475544637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97340cc6b0 0x7f9734153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.061+0000 7f972bfff640 1 -- 172.21.15.40:0/3475544637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97340b1b80 msgr2=0x7f9734151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:20.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.061+0000 7f972bfff640 1 --2- 172.21.15.40:0/3475544637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97340b1b80 0x7f9734151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.061+0000 7f972bfff640 1 -- 172.21.15.40:0/3475544637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c00aa10 con 0x7f97340b1f60 2023-10-18T09:01:20.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.062+0000 7f972bfff640 1 --2- 172.21.15.40:0/3475544637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97340b1f60 0x7f9734149240 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f972400d6a0 tx=0x7f972400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:20.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.062+0000 7f97297fa640 1 -- 172.21.15.40:0/3475544637 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9724005220 con 0x7f97340b1f60 2023-10-18T09:01:20.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.062+0000 7f97297fa640 1 -- 172.21.15.40:0/3475544637 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9724005470 con 0x7f97340b1f60 2023-10-18T09:01:20.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.062+0000 7f97297fa640 1 -- 172.21.15.40:0/3475544637 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9724009a80 con 0x7f97340b1f60 2023-10-18T09:01:20.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.062+0000 7f9739b75640 1 -- 172.21.15.40:0/3475544637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973414fdf0 con 0x7f97340b1f60 2023-10-18T09:01:20.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.062+0000 7f9739b75640 1 -- 172.21.15.40:0/3475544637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9734149780 con 0x7f97340b1f60 2023-10-18T09:01:20.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.063+0000 7f9739b75640 1 -- 172.21.15.40:0/3475544637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9734150310 con 0x7f97340b1f60 2023-10-18T09:01:20.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.064+0000 7f97297fa640 1 -- 172.21.15.40:0/3475544637 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9724009be0 con 0x7f97340b1f60 2023-10-18T09:01:20.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.065+0000 7f97297fa640 1 --2- 172.21.15.40:0/3475544637 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f972006ae40 0x7f972006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:20.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.065+0000 7f97297fa640 1 -- 172.21.15.40:0/3475544637 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9724089860 con 0x7f97340b1f60 2023-10-18T09:01:20.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.066+0000 7f97297fa640 1 -- 172.21.15.40:0/3475544637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=95}) v3 -- 0x7f9720072b50 con 0x7f97340b1f60 2023-10-18T09:01:20.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.066+0000 7f9738b73640 1 --2- 172.21.15.40:0/3475544637 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f972006ae40 0x7f972006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:20.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.067+0000 7f9738b73640 1 --2- 172.21.15.40:0/3475544637 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f972006ae40 0x7f972006d300 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9730000f60 tx=0x7f9730009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:20.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.069+0000 7f97297fa640 1 -- 172.21.15.40:0/3475544637 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9724017020 con 0x7f97340b1f60 2023-10-18T09:01:20.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.149+0000 7f97297fa640 1 -- 172.21.15.40:0/3475544637 <== mon.0 v2:172.21.15.40:3300/0 7 ==== osd_map(95..95 src has 1..95) v4 ==== 1613+0+0 (secure 0 0 0) 0x7f9724058c30 con 0x7f97340b1f60 2023-10-18T09:01:20.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.149+0000 7f97297fa640 1 -- 172.21.15.40:0/3475544637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=96}) v3 -- 0x7f9720073fc0 con 0x7f97340b1f60 2023-10-18T09:01:20.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.312+0000 7f9739b75640 1 -- 172.21.15.40:0/3475544637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9734066cf0 con 0x7f97340b1f60 2023-10-18T09:01:20.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.313+0000 7f97297fa640 1 -- 172.21.15.40:0/3475544637 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f9724020070 con 0x7f97340b1f60 2023-10-18T09:01:20.315 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:20.315 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1341351808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1341351808},{"type":"v1","addr":"172.21.15.144:6836","nonce":1341351808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1187670073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1187670073},{"type":"v1","addr":"172.21.15.40:6833","nonce":1187670073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5626,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1615448513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1615448513},{"type":"v1","addr":"172.21.15.144:6837","nonce":1615448513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:00:15.197720+0000","modified":"2023-10-18T09:00:18.306206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1926421699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1926421699},{"type":"v1","addr":"172.21.15.40:6835","nonce":1926421699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-18T09:01:20.316 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 41 2023-10-18T09:01:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.315+0000 7f9739b75640 1 -- 172.21.15.40:0/3475544637 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f972006ae40 msgr2=0x7f972006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.315+0000 7f9739b75640 1 --2- 172.21.15.40:0/3475544637 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f972006ae40 0x7f972006d300 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9730000f60 tx=0x7f9730009040 comp rx=0 tx=0).stop 2023-10-18T09:01:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.315+0000 7f9739b75640 1 -- 172.21.15.40:0/3475544637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97340b1f60 msgr2=0x7f9734149240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.315+0000 7f9739b75640 1 --2- 172.21.15.40:0/3475544637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97340b1f60 0x7f9734149240 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f972400d6a0 tx=0x7f972400db60 comp rx=0 tx=0).stop 2023-10-18T09:01:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.315+0000 7f9739b75640 1 -- 172.21.15.40:0/3475544637 shutdown_connections 2023-10-18T09:01:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.315+0000 7f9739b75640 1 --2- 172.21.15.40:0/3475544637 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f972006ae40 0x7f972006d300 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.315+0000 7f9739b75640 1 --2- 172.21.15.40:0/3475544637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97340cc6b0 0x7f9734153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.315+0000 7f9739b75640 1 --2- 172.21.15.40:0/3475544637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97340b1f60 0x7f9734149240 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.315+0000 7f9739b75640 1 --2- 172.21.15.40:0/3475544637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97340b1b80 0x7f9734151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.315+0000 7f9739b75640 1 -- 172.21.15.40:0/3475544637 >> 172.21.15.40:0/3475544637 conn(0x7f9734069720 msgr2=0x7f97340b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.316+0000 7f9739b75640 1 -- 172.21.15.40:0/3475544637 shutdown_connections 2023-10-18T09:01:20.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.316+0000 7f9739b75640 1 -- 172.21.15.40:0/3475544637 wait complete. 2023-10-18T09:01:20.338 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:01:20.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.572+0000 7fdf90840640 1 -- 172.21.15.40:0/3814876587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf8c070590 msgr2=0x7fdf8c070a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.572+0000 7fdf90840640 1 --2- 172.21.15.40:0/3814876587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf8c070590 0x7fdf8c070a20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0b0d40 tx=0x7fdf8402bd80 comp rx=0 tx=0).stop 2023-10-18T09:01:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.572+0000 7fdf90840640 1 -- 172.21.15.40:0/3814876587 shutdown_connections 2023-10-18T09:01:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.572+0000 7fdf90840640 1 --2- 172.21.15.40:0/3814876587 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf8c070590 0x7fdf8c070a20 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.572+0000 7fdf90840640 1 --2- 172.21.15.40:0/3814876587 >> v2:172.21.15.144:3300/0 conn(0x7fdf8c06db80 0x7fdf8c0cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.572+0000 7fdf90840640 1 --2- 172.21.15.40:0/3814876587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf8c141aa0 0x7fdf8c15c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.572+0000 7fdf90840640 1 -- 172.21.15.40:0/3814876587 >> 172.21.15.40:0/3814876587 conn(0x7fdf8c0cdfd0 msgr2=0x7fdf8c0ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:20.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.572+0000 7fdf90840640 1 -- 172.21.15.40:0/3814876587 shutdown_connections 2023-10-18T09:01:20.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.572+0000 7fdf90840640 1 -- 172.21.15.40:0/3814876587 wait complete. 2023-10-18T09:01:20.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.574+0000 7fdf90840640 1 Processor -- start 2023-10-18T09:01:20.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.574+0000 7fdf90840640 1 -- start start 2023-10-18T09:01:20.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.575+0000 7fdf90840640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf8c06db80 0x7fdf8c158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:20.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.575+0000 7fdf90840640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf8c070590 0x7fdf8c1307b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:20.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.575+0000 7fdf90840640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf8c141aa0 0x7fdf8c0b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:20.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.575+0000 7fdf90840640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c130cf0 con 0x7fdf8c141aa0 2023-10-18T09:01:20.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.575+0000 7fdf90840640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c0aef90 con 0x7fdf8c06db80 2023-10-18T09:01:20.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.575+0000 7fdf90840640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c0af270 con 0x7fdf8c070590 2023-10-18T09:01:20.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.575+0000 7fdf8bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf8c141aa0 0x7fdf8c0b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:20.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.575+0000 7fdf8bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf8c141aa0 0x7fdf8c0b0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57900/0 (socket says 172.21.15.40:57900) 2023-10-18T09:01:20.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.575+0000 7fdf8bfff640 1 -- 172.21.15.40:0/2455031365 learned_addr learned my addr 172.21.15.40:0/2455031365 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:20.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.576+0000 7fdf8affd640 1 --2- 172.21.15.40:0/2455031365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf8c070590 0x7fdf8c1307b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:20.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.576+0000 7fdf8bfff640 1 -- 172.21.15.40:0/2455031365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf8c070590 msgr2=0x7fdf8c1307b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:20.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.576+0000 7fdf8b7fe640 1 --2- 172.21.15.40:0/2455031365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf8c06db80 0x7fdf8c158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:20.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.576+0000 7fdf8bfff640 1 --2- 172.21.15.40:0/2455031365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf8c070590 0x7fdf8c1307b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.576+0000 7fdf8bfff640 1 -- 172.21.15.40:0/2455031365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf8c06db80 msgr2=0x7fdf8c158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:20.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.576+0000 7fdf8bfff640 1 --2- 172.21.15.40:0/2455031365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf8c06db80 0x7fdf8c158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.576+0000 7fdf8bfff640 1 -- 172.21.15.40:0/2455031365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf84009560 con 0x7fdf8c141aa0 2023-10-18T09:01:20.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.576+0000 7fdf8bfff640 1 --2- 172.21.15.40:0/2455031365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf8c141aa0 0x7fdf8c0b0030 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdf84036040 tx=0x7fdf8402da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:20.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.576+0000 7fdf73fff640 1 -- 172.21.15.40:0/2455031365 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8402de30 con 0x7fdf8c141aa0 2023-10-18T09:01:20.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.577+0000 7fdf73fff640 1 -- 172.21.15.40:0/2455031365 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdf840036e0 con 0x7fdf8c141aa0 2023-10-18T09:01:20.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.577+0000 7fdf90840640 1 -- 172.21.15.40:0/2455031365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8c0af550 con 0x7fdf8c141aa0 2023-10-18T09:01:20.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.577+0000 7fdf73fff640 1 -- 172.21.15.40:0/2455031365 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf84003af0 con 0x7fdf8c141aa0 2023-10-18T09:01:20.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.577+0000 7fdf90840640 1 -- 172.21.15.40:0/2455031365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8c0afb10 con 0x7fdf8c141aa0 2023-10-18T09:01:20.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.578+0000 7fdf90840640 1 -- 172.21.15.40:0/2455031365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8c066a90 con 0x7fdf8c141aa0 2023-10-18T09:01:20.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.579+0000 7fdf73fff640 1 -- 172.21.15.40:0/2455031365 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdf84003c50 con 0x7fdf8c141aa0 2023-10-18T09:01:20.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.580+0000 7fdf73fff640 1 --2- 172.21.15.40:0/2455031365 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf7806ae40 0x7fdf7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:20.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.580+0000 7fdf73fff640 1 -- 172.21.15.40:0/2455031365 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fdf840ac3f0 con 0x7fdf8c141aa0 2023-10-18T09:01:20.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.581+0000 7fdf8b7fe640 1 --2- 172.21.15.40:0/2455031365 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf7806ae40 0x7fdf7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:20.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.582+0000 7fdf8b7fe640 1 --2- 172.21.15.40:0/2455031365 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf7806ae40 0x7fdf7806d300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c005c40 tx=0x7fdf7c005bd0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:20.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.584+0000 7fdf73fff640 1 -- 172.21.15.40:0/2455031365 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf84077400 con 0x7fdf8c141aa0 2023-10-18T09:01:20.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.815+0000 7fdf90840640 1 -- 172.21.15.40:0/2455031365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf8c0afd30 con 0x7fdf8c141aa0 2023-10-18T09:01:20.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.817+0000 7fdf73fff640 1 -- 172.21.15.40:0/2455031365 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fdf84043090 con 0x7fdf8c141aa0 2023-10-18T09:01:20.819 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:20.819 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1341351808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1341351808},{"type":"v1","addr":"172.21.15.144:6836","nonce":1341351808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1187670073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1187670073},{"type":"v1","addr":"172.21.15.40:6833","nonce":1187670073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5626,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1615448513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1615448513},{"type":"v1","addr":"172.21.15.144:6837","nonce":1615448513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:00:15.197720+0000","modified":"2023-10-18T09:00:18.306206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1926421699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1926421699},{"type":"v1","addr":"172.21.15.40:6835","nonce":1926421699}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-18T09:01:20.820 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 41 2023-10-18T09:01:20.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.819+0000 7fdf90840640 1 -- 172.21.15.40:0/2455031365 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf7806ae40 msgr2=0x7fdf7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:20.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.819+0000 7fdf90840640 1 --2- 172.21.15.40:0/2455031365 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf7806ae40 0x7fdf7806d300 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c005c40 tx=0x7fdf7c005bd0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.819+0000 7fdf90840640 1 -- 172.21.15.40:0/2455031365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf8c141aa0 msgr2=0x7fdf8c0b0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:20.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.819+0000 7fdf90840640 1 --2- 172.21.15.40:0/2455031365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf8c141aa0 0x7fdf8c0b0030 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdf84036040 tx=0x7fdf8402da30 comp rx=0 tx=0).stop 2023-10-18T09:01:20.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.820+0000 7fdf90840640 1 -- 172.21.15.40:0/2455031365 shutdown_connections 2023-10-18T09:01:20.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.820+0000 7fdf90840640 1 --2- 172.21.15.40:0/2455031365 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf7806ae40 0x7fdf7806d300 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.820+0000 7fdf90840640 1 --2- 172.21.15.40:0/2455031365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf8c141aa0 0x7fdf8c0b0030 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.820+0000 7fdf90840640 1 --2- 172.21.15.40:0/2455031365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf8c070590 0x7fdf8c1307b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.820+0000 7fdf90840640 1 --2- 172.21.15.40:0/2455031365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf8c06db80 0x7fdf8c158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:20.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.820+0000 7fdf90840640 1 -- 172.21.15.40:0/2455031365 >> 172.21.15.40:0/2455031365 conn(0x7fdf8c0cdfd0 msgr2=0x7fdf8c140a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:20.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.820+0000 7fdf90840640 1 -- 172.21.15.40:0/2455031365 shutdown_connections 2023-10-18T09:01:20.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:20.820+0000 7fdf90840640 1 -- 172.21.15.40:0/2455031365 wait complete. 2023-10-18T09:01:20.840 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:01:21.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.073+0000 7fedd2890640 1 -- 172.21.15.40:0/2378168211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedcc0b0630 msgr2=0x7fedcc0c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:21.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.073+0000 7fedd2890640 1 --2- 172.21.15.40:0/2378168211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedcc0b0630 0x7fedcc0c99c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fedcc06aeb0 tx=0x7fedbc02bda0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.073+0000 7fedd2890640 1 -- 172.21.15.40:0/2378168211 shutdown_connections 2023-10-18T09:01:21.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.073+0000 7fedd2890640 1 --2- 172.21.15.40:0/2378168211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedcc0c9f00 0x7fedcc157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.073+0000 7fedd2890640 1 --2- 172.21.15.40:0/2378168211 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedcc0b0630 0x7fedcc0c99c0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.073+0000 7fedd2890640 1 --2- 172.21.15.40:0/2378168211 >> v2:172.21.15.144:3300/0 conn(0x7fedcc06d6f0 0x7fedcc0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.073+0000 7fedd2890640 1 -- 172.21.15.40:0/2378168211 >> 172.21.15.40:0/2378168211 conn(0x7fedcc069720 msgr2=0x7fedcc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:21.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.073+0000 7fedd2890640 1 -- 172.21.15.40:0/2378168211 shutdown_connections 2023-10-18T09:01:21.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.073+0000 7fedd2890640 1 -- 172.21.15.40:0/2378168211 wait complete. 2023-10-18T09:01:21.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.075+0000 7fedd2890640 1 Processor -- start 2023-10-18T09:01:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.075+0000 7fedd2890640 1 -- start start 2023-10-18T09:01:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.076+0000 7fedd2890640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedcc06d6f0 0x7fedcc148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.076+0000 7fedd2890640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedcc0b0630 0x7fedcc148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.076+0000 7fedd2890640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedcc0c9f00 0x7fedcc0acc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.076+0000 7fedd2890640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0aea40 con 0x7fedcc0c9f00 2023-10-18T09:01:21.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.076+0000 7fedd2890640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0ad9a0 con 0x7fedcc0b0630 2023-10-18T09:01:21.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.076+0000 7fedd2890640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0adc80 con 0x7fedcc06d6f0 2023-10-18T09:01:21.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.076+0000 7fedd208f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedcc0c9f00 0x7fedcc0acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:21.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.076+0000 7fedd208f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedcc0c9f00 0x7fedcc0acc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57916/0 (socket says 172.21.15.40:57916) 2023-10-18T09:01:21.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.077+0000 7fedd208f640 1 -- 172.21.15.40:0/314692437 learned_addr learned my addr 172.21.15.40:0/314692437 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:21.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.077+0000 7fedd188e640 1 --2- 172.21.15.40:0/314692437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedcc06d6f0 0x7fedcc148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:21.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.077+0000 7fedd108d640 1 --2- 172.21.15.40:0/314692437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedcc0b0630 0x7fedcc148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:21.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.077+0000 7fedd208f640 1 -- 172.21.15.40:0/314692437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedcc06d6f0 msgr2=0x7fedcc148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:21.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.077+0000 7fedd208f640 1 --2- 172.21.15.40:0/314692437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedcc06d6f0 0x7fedcc148920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.077+0000 7fedd208f640 1 -- 172.21.15.40:0/314692437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedcc0b0630 msgr2=0x7fedcc148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:21.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.077+0000 7fedd208f640 1 --2- 172.21.15.40:0/314692437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedcc0b0630 0x7fedcc148e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.077+0000 7fedd208f640 1 -- 172.21.15.40:0/314692437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc009560 con 0x7fedcc0c9f00 2023-10-18T09:01:21.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.077+0000 7fedd208f640 1 --2- 172.21.15.40:0/314692437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedcc0c9f00 0x7fedcc0acc00 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fedc400cad0 tx=0x7fedc400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:21.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.078+0000 7fedc27fc640 1 -- 172.21.15.40:0/314692437 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc400ae20 con 0x7fedcc0c9f00 2023-10-18T09:01:21.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.078+0000 7fedc27fc640 1 -- 172.21.15.40:0/314692437 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fedc400f450 con 0x7fedcc0c9f00 2023-10-18T09:01:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.078+0000 7fedd2890640 1 -- 172.21.15.40:0/314692437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc0adf60 con 0x7fedcc0c9f00 2023-10-18T09:01:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.078+0000 7fedc27fc640 1 -- 172.21.15.40:0/314692437 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc400f8c0 con 0x7fedcc0c9f00 2023-10-18T09:01:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.078+0000 7fedd2890640 1 -- 172.21.15.40:0/314692437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc0ae520 con 0x7fedcc0c9f00 2023-10-18T09:01:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.079+0000 7fedd2890640 1 -- 172.21.15.40:0/314692437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc073250 con 0x7fedcc0c9f00 2023-10-18T09:01:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.080+0000 7fedc27fc640 1 -- 172.21.15.40:0/314692437 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fedc400fa20 con 0x7fedcc0c9f00 2023-10-18T09:01:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.082+0000 7fedc27fc640 1 --2- 172.21.15.40:0/314692437 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fedb006ae40 0x7fedb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.082+0000 7fedc27fc640 1 -- 172.21.15.40:0/314692437 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fedc408a1d0 con 0x7fedcc0c9f00 2023-10-18T09:01:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.082+0000 7fedd188e640 1 --2- 172.21.15.40:0/314692437 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fedb006ae40 0x7fedb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:21.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.083+0000 7fedd188e640 1 --2- 172.21.15.40:0/314692437 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fedb006ae40 0x7fedb006d300 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fedc8005fd0 tx=0x7fedc8005e40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:21.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.087+0000 7fedc27fc640 1 -- 172.21.15.40:0/314692437 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fedc4058c70 con 0x7fedcc0c9f00 2023-10-18T09:01:21.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.287+0000 7fedd2890640 1 -- 172.21.15.40:0/314692437 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fedcc0ae710 con 0x7fedcc0c9f00 2023-10-18T09:01:21.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.290+0000 7fedc27fc640 1 -- 172.21.15.40:0/314692437 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16820 (secure 0 0 0) 0x7fedc4058800 con 0x7fedcc0c9f00 2023-10-18T09:01:21.293 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:21.293 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":95,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:01:20.144954+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:00:13.170937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-18T09:00:14.205418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1","mappings":[{"from":6,"to":0}]},{"pgid":"11.8","mappings":[{"from":6,"to":1}]},{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.18","mappings":[{"from":2,"to":0}]},{"pgid":"11.33","mappings":[{"from":6,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:01:21.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.293+0000 7fedd2890640 1 -- 172.21.15.40:0/314692437 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fedb006ae40 msgr2=0x7fedb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:21.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.293+0000 7fedd2890640 1 --2- 172.21.15.40:0/314692437 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fedb006ae40 0x7fedb006d300 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fedc8005fd0 tx=0x7fedc8005e40 comp rx=0 tx=0).stop 2023-10-18T09:01:21.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.293+0000 7fedd2890640 1 -- 172.21.15.40:0/314692437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedcc0c9f00 msgr2=0x7fedcc0acc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:21.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.293+0000 7fedd2890640 1 --2- 172.21.15.40:0/314692437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedcc0c9f00 0x7fedcc0acc00 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fedc400cad0 tx=0x7fedc400cf90 comp rx=0 tx=0).stop 2023-10-18T09:01:21.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.293+0000 7fedd2890640 1 -- 172.21.15.40:0/314692437 shutdown_connections 2023-10-18T09:01:21.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.293+0000 7fedd2890640 1 --2- 172.21.15.40:0/314692437 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fedb006ae40 0x7fedb006d300 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.293+0000 7fedd2890640 1 --2- 172.21.15.40:0/314692437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fedcc0c9f00 0x7fedcc0acc00 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.293+0000 7fedd2890640 1 --2- 172.21.15.40:0/314692437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedcc0b0630 0x7fedcc148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.293+0000 7fedd2890640 1 --2- 172.21.15.40:0/314692437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fedcc06d6f0 0x7fedcc148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.293+0000 7fedd2890640 1 -- 172.21.15.40:0/314692437 >> 172.21.15.40:0/314692437 conn(0x7fedcc069720 msgr2=0x7fedcc157020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:21.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.294+0000 7fedd2890640 1 -- 172.21.15.40:0/314692437 shutdown_connections 2023-10-18T09:01:21.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.294+0000 7fedd2890640 1 -- 172.21.15.40:0/314692437 wait complete. 2023-10-18T09:01:21.317 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T09:01:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.551+0000 7f1d34c32640 1 -- 172.21.15.40:0/1618910111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d3015b4a0 msgr2=0x7f1d300cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.551+0000 7f1d34c32640 1 --2- 172.21.15.40:0/1618910111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d3015b4a0 0x7f1d300cfe60 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1d20005fd0 tx=0x7f1d2002bd90 comp rx=0 tx=0).stop 2023-10-18T09:01:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.551+0000 7f1d34c32640 1 -- 172.21.15.40:0/1618910111 shutdown_connections 2023-10-18T09:01:21.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.551+0000 7f1d34c32640 1 --2- 172.21.15.40:0/1618910111 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d3015b880 0x7f1d300cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.551+0000 7f1d34c32640 1 --2- 172.21.15.40:0/1618910111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d3015b4a0 0x7f1d300cfe60 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.551+0000 7f1d34c32640 1 -- 172.21.15.40:0/1618910111 >> 172.21.15.40:0/1618910111 conn(0x7f1d3008aca0 msgr2=0x7f1d3009a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.551+0000 7f1d34c32640 1 -- 172.21.15.40:0/1618910111 shutdown_connections 2023-10-18T09:01:21.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.551+0000 7f1d34c32640 1 -- 172.21.15.40:0/1618910111 wait complete. 2023-10-18T09:01:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.553+0000 7f1d34c32640 1 Processor -- start 2023-10-18T09:01:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.553+0000 7f1d34c32640 1 -- start start 2023-10-18T09:01:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.554+0000 7f1d34c32640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d300cdca0 0x7f1d30151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:21.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.554+0000 7f1d34c32640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d3015b4a0 0x7f1d3015c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.554+0000 7f1d34c32640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d3015df70 0x7f1d300b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.554+0000 7f1d34c32640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3015db50 con 0x7f1d3015df70 2023-10-18T09:01:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.554+0000 7f1d34c32640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d30151940 con 0x7f1d3015b4a0 2023-10-18T09:01:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.554+0000 7f1d34c32640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3015ced0 con 0x7f1d300cdca0 2023-10-18T09:01:21.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.555+0000 7f1d2ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d3015df70 0x7f1d300b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:21.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.555+0000 7f1d2ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d3015df70 0x7f1d300b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57936/0 (socket says 172.21.15.40:57936) 2023-10-18T09:01:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.555+0000 7f1d2ffff640 1 -- 172.21.15.40:0/582712579 learned_addr learned my addr 172.21.15.40:0/582712579 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.555+0000 7f1d2f7fe640 1 --2- 172.21.15.40:0/582712579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d300cdca0 0x7f1d30151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:21.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.555+0000 7f1d2effd640 1 --2- 172.21.15.40:0/582712579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d3015b4a0 0x7f1d3015c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:21.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.555+0000 7f1d2ffff640 1 -- 172.21.15.40:0/582712579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d300cdca0 msgr2=0x7f1d30151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:21.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.555+0000 7f1d2ffff640 1 --2- 172.21.15.40:0/582712579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d300cdca0 0x7f1d30151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.555+0000 7f1d2ffff640 1 -- 172.21.15.40:0/582712579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d3015b4a0 msgr2=0x7f1d3015c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.555+0000 7f1d2ffff640 1 --2- 172.21.15.40:0/582712579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d3015b4a0 0x7f1d3015c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.555+0000 7f1d2ffff640 1 -- 172.21.15.40:0/582712579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d20009560 con 0x7f1d3015df70 2023-10-18T09:01:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.556+0000 7f1d2ffff640 1 --2- 172.21.15.40:0/582712579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d3015df70 0x7f1d300b4fd0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1d2800c410 tx=0x7f1d2800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.556+0000 7f1d27fff640 1 -- 172.21.15.40:0/582712579 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28011790 con 0x7f1d3015df70 2023-10-18T09:01:21.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.556+0000 7f1d27fff640 1 -- 172.21.15.40:0/582712579 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1d280119e0 con 0x7f1d3015df70 2023-10-18T09:01:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.556+0000 7f1d27fff640 1 -- 172.21.15.40:0/582712579 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28011e30 con 0x7f1d3015df70 2023-10-18T09:01:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.556+0000 7f1d34c32640 1 -- 172.21.15.40:0/582712579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3015d1b0 con 0x7f1d3015df70 2023-10-18T09:01:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.556+0000 7f1d34c32640 1 -- 172.21.15.40:0/582712579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3015d770 con 0x7f1d3015df70 2023-10-18T09:01:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.557+0000 7f1d34c32640 1 -- 172.21.15.40:0/582712579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d30071da0 con 0x7f1d3015df70 2023-10-18T09:01:21.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.558+0000 7f1d27fff640 1 -- 172.21.15.40:0/582712579 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1d2801c460 con 0x7f1d3015df70 2023-10-18T09:01:21.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.560+0000 7f1d27fff640 1 --2- 172.21.15.40:0/582712579 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1d1006aec0 0x7f1d1006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:21.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.560+0000 7f1d27fff640 1 -- 172.21.15.40:0/582712579 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f1d28089590 con 0x7f1d3015df70 2023-10-18T09:01:21.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.560+0000 7f1d2f7fe640 1 --2- 172.21.15.40:0/582712579 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1d1006aec0 0x7f1d1006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:21.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.562+0000 7f1d2f7fe640 1 --2- 172.21.15.40:0/582712579 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1d1006aec0 0x7f1d1006d380 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1d20035000 tx=0x7f1d20004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:21.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.564+0000 7f1d27fff640 1 -- 172.21.15.40:0/582712579 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d28054590 con 0x7f1d3015df70 2023-10-18T09:01:21.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.795+0000 7f1d34c32640 1 -- 172.21.15.40:0/582712579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1d300667d0 con 0x7f1d3015df70 2023-10-18T09:01:21.798 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:01:21.796+0000 7f6a0f710640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T09:01:21.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.799+0000 7f1d27fff640 1 -- 172.21.15.40:0/582712579 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1d28057bb0 con 0x7f1d3015df70 2023-10-18T09:01:21.802 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T09:01:21.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.802+0000 7f1d34c32640 1 -- 172.21.15.40:0/582712579 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1d1006aec0 msgr2=0x7f1d1006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:21.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.802+0000 7f1d34c32640 1 --2- 172.21.15.40:0/582712579 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1d1006aec0 0x7f1d1006d380 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1d20035000 tx=0x7f1d20004730 comp rx=0 tx=0).stop 2023-10-18T09:01:21.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.802+0000 7f1d34c32640 1 -- 172.21.15.40:0/582712579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d3015df70 msgr2=0x7f1d300b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:21.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.802+0000 7f1d34c32640 1 --2- 172.21.15.40:0/582712579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d3015df70 0x7f1d300b4fd0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1d2800c410 tx=0x7f1d2800c8d0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.803+0000 7f1d34c32640 1 -- 172.21.15.40:0/582712579 shutdown_connections 2023-10-18T09:01:21.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.803+0000 7f1d34c32640 1 --2- 172.21.15.40:0/582712579 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1d1006aec0 0x7f1d1006d380 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.803+0000 7f1d34c32640 1 --2- 172.21.15.40:0/582712579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d3015df70 0x7f1d300b4fd0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.803+0000 7f1d34c32640 1 --2- 172.21.15.40:0/582712579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d3015b4a0 0x7f1d3015c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.803+0000 7f1d34c32640 1 --2- 172.21.15.40:0/582712579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d300cdca0 0x7f1d30151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:21.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.803+0000 7f1d34c32640 1 -- 172.21.15.40:0/582712579 >> 172.21.15.40:0/582712579 conn(0x7f1d3008aca0 msgr2=0x7f1d30158da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:21.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.804+0000 7f1d34c32640 1 -- 172.21.15.40:0/582712579 shutdown_connections 2023-10-18T09:01:21.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:21.804+0000 7f1d34c32640 1 -- 172.21.15.40:0/582712579 wait complete. 2023-10-18T09:01:21.827 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T09:01:21.901 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T09:01:22.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.033+0000 7f4bd4690640 1 -- 172.21.15.40:0/267867186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bb80039c0 msgr2=0x7f4bb8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.033+0000 7f4bd4690640 1 --2- 172.21.15.40:0/267867186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bb80039c0 0x7f4bb8003de0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800bb20 tx=0x7f4bb80057f0 comp rx=0 tx=0).stop 2023-10-18T09:01:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.033+0000 7f4bd4690640 1 -- 172.21.15.40:0/267867186 shutdown_connections 2023-10-18T09:01:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.033+0000 7f4bd4690640 1 --2- 172.21.15.40:0/267867186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bb80039c0 0x7f4bb8003de0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:22.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.033+0000 7f4bd4690640 1 -- 172.21.15.40:0/267867186 >> 172.21.15.40:0/267867186 conn(0x7f4bcc069720 msgr2=0x7f4bcc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:22.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.033+0000 7f4bd4690640 1 -- 172.21.15.40:0/267867186 shutdown_connections 2023-10-18T09:01:22.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.033+0000 7f4bd4690640 1 -- 172.21.15.40:0/267867186 wait complete. 2023-10-18T09:01:22.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.035+0000 7f4bd4690640 1 Processor -- start 2023-10-18T09:01:22.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.035+0000 7f4bd4690640 1 -- start start 2023-10-18T09:01:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.036+0000 7f4bd4690640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bb80039c0 0x7f4bcc149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.036+0000 7f4bd4690640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bcc1f1e10 0x7f4bcc1f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.036+0000 7f4bd4690640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bcc1553a0 0x7f4bcc155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.036+0000 7f4bd4690640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc1f2fd0 con 0x7f4bcc1553a0 2023-10-18T09:01:22.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.036+0000 7f4bd4690640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc1f2940 con 0x7f4bb80039c0 2023-10-18T09:01:22.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.036+0000 7f4bd4690640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc1f2c20 con 0x7f4bcc1f1e10 2023-10-18T09:01:22.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.037+0000 7f4bd2c2d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bcc1553a0 0x7f4bcc155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:22.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.037+0000 7f4bd2c2d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bcc1553a0 0x7f4bcc155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57974/0 (socket says 172.21.15.40:57974) 2023-10-18T09:01:22.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.037+0000 7f4bd2c2d640 1 -- 172.21.15.40:0/3494818277 learned_addr learned my addr 172.21.15.40:0/3494818277 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:22.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.037+0000 7f4bd2c2d640 1 -- 172.21.15.40:0/3494818277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bcc1f1e10 msgr2=0x7f4bcc1f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.037+0000 7f4bd2c2d640 1 --2- 172.21.15.40:0/3494818277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bcc1f1e10 0x7f4bcc1f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.037+0000 7f4bd242c640 1 --2- 172.21.15.40:0/3494818277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bb80039c0 0x7f4bcc149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.037+0000 7f4bd2c2d640 1 -- 172.21.15.40:0/3494818277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bb80039c0 msgr2=0x7f4bcc149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.037+0000 7f4bd2c2d640 1 --2- 172.21.15.40:0/3494818277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bb80039c0 0x7f4bcc149940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:22.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.037+0000 7f4bd2c2d640 1 -- 172.21.15.40:0/3494818277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc06f780 con 0x7f4bcc1553a0 2023-10-18T09:01:22.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.037+0000 7f4bd2c2d640 1 --2- 172.21.15.40:0/3494818277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bcc1553a0 0x7f4bcc155820 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc06f4b0 tx=0x7f4bc400eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:22.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.038+0000 7f4bc2ffd640 1 -- 172.21.15.40:0/3494818277 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc400c8f0 con 0x7f4bcc1553a0 2023-10-18T09:01:22.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.038+0000 7f4bc2ffd640 1 -- 172.21.15.40:0/3494818277 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4bc400cb40 con 0x7f4bcc1553a0 2023-10-18T09:01:22.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.038+0000 7f4bc2ffd640 1 -- 172.21.15.40:0/3494818277 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4011460 con 0x7f4bcc1553a0 2023-10-18T09:01:22.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.038+0000 7f4bd4690640 1 -- 172.21.15.40:0/3494818277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bcc1f31e0 con 0x7f4bcc1553a0 2023-10-18T09:01:22.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.038+0000 7f4bd4690640 1 -- 172.21.15.40:0/3494818277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bcc149e80 con 0x7f4bcc1553a0 2023-10-18T09:01:22.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.039+0000 7f4bd4690640 1 -- 172.21.15.40:0/3494818277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bcc1f3700 con 0x7f4bcc1553a0 2023-10-18T09:01:22.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.040+0000 7f4bc2ffd640 1 -- 172.21.15.40:0/3494818277 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4bc40115c0 con 0x7f4bcc1553a0 2023-10-18T09:01:22.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.041+0000 7f4bc2ffd640 1 --2- 172.21.15.40:0/3494818277 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4bb006ae40 0x7f4bb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:22.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.041+0000 7f4bc2ffd640 1 -- 172.21.15.40:0/3494818277 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f4bc4089e50 con 0x7f4bcc1553a0 2023-10-18T09:01:22.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.042+0000 7f4bd242c640 1 --2- 172.21.15.40:0/3494818277 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4bb006ae40 0x7f4bb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:22.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.043+0000 7f4bd242c640 1 --2- 172.21.15.40:0/3494818277 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4bb006ae40 0x7f4bb006d300 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8037000 tx=0x7f4bc802e040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:22.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.045+0000 7f4bc2ffd640 1 -- 172.21.15.40:0/3494818277 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4bcc1f3700 con 0x7f4bcc1553a0 2023-10-18T09:01:22.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.274+0000 7f4bd4690640 1 -- 172.21.15.40:0/3494818277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4bcc1f3700 con 0x7f4bcc1553a0 2023-10-18T09:01:22.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.803+0000 7f4bc2ffd640 1 -- 172.21.15.40:0/3494818277 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4bcc1f3700 con 0x7f4bcc1553a0 2023-10-18T09:01:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.807+0000 7f4bd4690640 1 -- 172.21.15.40:0/3494818277 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4bb006ae40 msgr2=0x7f4bb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.807+0000 7f4bd4690640 1 --2- 172.21.15.40:0/3494818277 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4bb006ae40 0x7f4bb006d300 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8037000 tx=0x7f4bc802e040 comp rx=0 tx=0).stop 2023-10-18T09:01:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.807+0000 7f4bd4690640 1 -- 172.21.15.40:0/3494818277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bcc1553a0 msgr2=0x7f4bcc155820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.807+0000 7f4bd4690640 1 --2- 172.21.15.40:0/3494818277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bcc1553a0 0x7f4bcc155820 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc06f4b0 tx=0x7f4bc400eae0 comp rx=0 tx=0).stop 2023-10-18T09:01:22.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.807+0000 7f4bd4690640 1 -- 172.21.15.40:0/3494818277 shutdown_connections 2023-10-18T09:01:22.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.807+0000 7f4bd4690640 1 --2- 172.21.15.40:0/3494818277 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4bb006ae40 0x7f4bb006d300 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:22.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.807+0000 7f4bd4690640 1 --2- 172.21.15.40:0/3494818277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bcc1553a0 0x7f4bcc155820 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:22.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.807+0000 7f4bd4690640 1 --2- 172.21.15.40:0/3494818277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4bcc1f1e10 0x7f4bcc1f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:22.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.807+0000 7f4bd4690640 1 --2- 172.21.15.40:0/3494818277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4bb80039c0 0x7f4bcc149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:22.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.807+0000 7f4bd4690640 1 -- 172.21.15.40:0/3494818277 >> 172.21.15.40:0/3494818277 conn(0x7f4bcc069720 msgr2=0x7f4bcc0b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:22.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.807+0000 7f4bd4690640 1 -- 172.21.15.40:0/3494818277 shutdown_connections 2023-10-18T09:01:22.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:22.807+0000 7f4bd4690640 1 -- 172.21.15.40:0/3494818277 wait complete. 2023-10-18T09:01:22.831 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T09:01:23.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.069+0000 7f9efb347640 1 -- 172.21.15.40:0/2867880343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef414e350 msgr2=0x7f9ef406b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:23.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.069+0000 7f9efb347640 1 --2- 172.21.15.40:0/2867880343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef414e350 0x7f9ef406b710 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0005fd0 tx=0x7f9ef002be10 comp rx=0 tx=0).stop 2023-10-18T09:01:23.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.069+0000 7f9efb347640 1 -- 172.21.15.40:0/2867880343 shutdown_connections 2023-10-18T09:01:23.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.069+0000 7f9efb347640 1 --2- 172.21.15.40:0/2867880343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ef40cc950 0x7f9ef40b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:23.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.069+0000 7f9efb347640 1 --2- 172.21.15.40:0/2867880343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef414e350 0x7f9ef406b710 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:23.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.069+0000 7f9efb347640 1 -- 172.21.15.40:0/2867880343 >> 172.21.15.40:0/2867880343 conn(0x7f9ef40aeb90 msgr2=0x7f9ef40aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:23.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.069+0000 7f9efb347640 1 -- 172.21.15.40:0/2867880343 shutdown_connections 2023-10-18T09:01:23.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.069+0000 7f9efb347640 1 -- 172.21.15.40:0/2867880343 wait complete. 2023-10-18T09:01:23.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.071+0000 7f9efb347640 1 Processor -- start 2023-10-18T09:01:23.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.071+0000 7f9efb347640 1 -- start start 2023-10-18T09:01:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.072+0000 7f9efb347640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef406fbc0 0x7f9ef4141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.072+0000 7f9efb347640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ef40cc950 0x7f9ef4145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.072+0000 7f9efb347640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ef414e350 0x7f9ef4145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.072+0000 7f9efb347640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef4158a00 con 0x7f9ef414e350 2023-10-18T09:01:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.072+0000 7f9efb347640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef40ab100 con 0x7f9ef406fbc0 2023-10-18T09:01:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.072+0000 7f9efb347640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef41507b0 con 0x7f9ef40cc950 2023-10-18T09:01:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.073+0000 7f9efab46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ef414e350 0x7f9ef4145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.073+0000 7f9efab46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ef414e350 0x7f9ef4145f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57980/0 (socket says 172.21.15.40:57980) 2023-10-18T09:01:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.073+0000 7f9efab46640 1 -- 172.21.15.40:0/167733000 learned_addr learned my addr 172.21.15.40:0/167733000 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.073+0000 7f9efa345640 1 --2- 172.21.15.40:0/167733000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef406fbc0 0x7f9ef4141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.073+0000 7f9ef9b44640 1 --2- 172.21.15.40:0/167733000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ef40cc950 0x7f9ef4145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.073+0000 7f9efab46640 1 -- 172.21.15.40:0/167733000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ef40cc950 msgr2=0x7f9ef4145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.073+0000 7f9efab46640 1 --2- 172.21.15.40:0/167733000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ef40cc950 0x7f9ef4145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.073+0000 7f9efab46640 1 -- 172.21.15.40:0/167733000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef406fbc0 msgr2=0x7f9ef4141a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.073+0000 7f9efab46640 1 --2- 172.21.15.40:0/167733000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef406fbc0 0x7f9ef4141a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.073+0000 7f9efab46640 1 -- 172.21.15.40:0/167733000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef0009560 con 0x7f9ef414e350 2023-10-18T09:01:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.074+0000 7f9efab46640 1 --2- 172.21.15.40:0/167733000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ef414e350 0x7f9ef4145f50 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00eb10 tx=0x7f9eec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.074+0000 7f9eeaffd640 1 -- 172.21.15.40:0/167733000 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec004b20 con 0x7f9ef414e350 2023-10-18T09:01:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.074+0000 7f9eeaffd640 1 -- 172.21.15.40:0/167733000 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9eec004ca0 con 0x7f9ef414e350 2023-10-18T09:01:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.074+0000 7f9efb347640 1 -- 172.21.15.40:0/167733000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef4150a90 con 0x7f9ef414e350 2023-10-18T09:01:23.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.074+0000 7f9eeaffd640 1 -- 172.21.15.40:0/167733000 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec005280 con 0x7f9ef414e350 2023-10-18T09:01:23.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.074+0000 7f9efb347640 1 -- 172.21.15.40:0/167733000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef4151050 con 0x7f9ef414e350 2023-10-18T09:01:23.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.075+0000 7f9efb347640 1 -- 172.21.15.40:0/167733000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef4158b70 con 0x7f9ef414e350 2023-10-18T09:01:23.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.076+0000 7f9eeaffd640 1 -- 172.21.15.40:0/167733000 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9eec0053e0 con 0x7f9ef414e350 2023-10-18T09:01:23.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.078+0000 7f9eeaffd640 1 --2- 172.21.15.40:0/167733000 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9ed806ae40 0x7f9ed806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:23.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.078+0000 7f9eeaffd640 1 -- 172.21.15.40:0/167733000 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f9eec014070 con 0x7f9ef414e350 2023-10-18T09:01:23.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.079+0000 7f9efa345640 1 --2- 172.21.15.40:0/167733000 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9ed806ae40 0x7f9ed806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:23.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.080+0000 7f9efa345640 1 --2- 172.21.15.40:0/167733000 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9ed806ae40 0x7f9ed806d300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0035000 tx=0x7f9ef0004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:23.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.082+0000 7f9eeaffd640 1 -- 172.21.15.40:0/167733000 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9eec0554a0 con 0x7f9ef414e350 2023-10-18T09:01:23.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:23.277+0000 7f9efb347640 1 -- 172.21.15.40:0/167733000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9ef4066e00 con 0x7f9ef414e350 2023-10-18T09:01:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.184+0000 7f9eeaffd640 1 -- 172.21.15.40:0/167733000 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9eec058ac0 con 0x7f9ef414e350 2023-10-18T09:01:24.187 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-10-18T09:01:24.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.190+0000 7f9efb347640 1 -- 172.21.15.40:0/167733000 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9ed806ae40 msgr2=0x7f9ed806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:24.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.190+0000 7f9efb347640 1 --2- 172.21.15.40:0/167733000 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9ed806ae40 0x7f9ed806d300 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0035000 tx=0x7f9ef0004730 comp rx=0 tx=0).stop 2023-10-18T09:01:24.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.190+0000 7f9efb347640 1 -- 172.21.15.40:0/167733000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ef414e350 msgr2=0x7f9ef4145f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:24.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.190+0000 7f9efb347640 1 --2- 172.21.15.40:0/167733000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ef414e350 0x7f9ef4145f50 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00eb10 tx=0x7f9eec00c2d0 comp rx=0 tx=0).stop 2023-10-18T09:01:24.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.190+0000 7f9efb347640 1 -- 172.21.15.40:0/167733000 shutdown_connections 2023-10-18T09:01:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.190+0000 7f9efb347640 1 --2- 172.21.15.40:0/167733000 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9ed806ae40 0x7f9ed806d300 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.190+0000 7f9efb347640 1 --2- 172.21.15.40:0/167733000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ef414e350 0x7f9ef4145f50 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.190+0000 7f9efb347640 1 --2- 172.21.15.40:0/167733000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ef40cc950 0x7f9ef4145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.190+0000 7f9efb347640 1 --2- 172.21.15.40:0/167733000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ef406fbc0 0x7f9ef4141a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.190+0000 7f9efb347640 1 -- 172.21.15.40:0/167733000 >> 172.21.15.40:0/167733000 conn(0x7f9ef40aeb90 msgr2=0x7f9ef4149ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.191+0000 7f9efb347640 1 -- 172.21.15.40:0/167733000 shutdown_connections 2023-10-18T09:01:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.192+0000 7f9efb347640 1 -- 172.21.15.40:0/167733000 wait complete. 2023-10-18T09:01:24.216 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T09:01:24.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.448+0000 7f33edcca640 1 -- 172.21.15.40:0/1485195987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33e814e350 msgr2=0x7f33e806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:24.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.448+0000 7f33edcca640 1 --2- 172.21.15.40:0/1485195987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33e814e350 0x7f33e806fbc0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009860 tx=0x7f33d802bd60 comp rx=0 tx=0).stop 2023-10-18T09:01:24.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.448+0000 7f33edcca640 1 -- 172.21.15.40:0/1485195987 shutdown_connections 2023-10-18T09:01:24.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.448+0000 7f33edcca640 1 --2- 172.21.15.40:0/1485195987 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33e814e350 0x7f33e806fbc0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:24.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.448+0000 7f33edcca640 1 -- 172.21.15.40:0/1485195987 >> 172.21.15.40:0/1485195987 conn(0x7f33e80aeb90 msgr2=0x7f33e80aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:24.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.449+0000 7f33edcca640 1 -- 172.21.15.40:0/1485195987 shutdown_connections 2023-10-18T09:01:24.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.449+0000 7f33edcca640 1 -- 172.21.15.40:0/1485195987 wait complete. 2023-10-18T09:01:24.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.450+0000 7f33edcca640 1 Processor -- start 2023-10-18T09:01:24.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.450+0000 7f33edcca640 1 -- start start 2023-10-18T09:01:24.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.451+0000 7f33edcca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33e806d550 0x7f33e80cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:24.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.451+0000 7f33edcca640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33e8070100 0x7f33e80cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:24.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.451+0000 7f33edcca640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33e814e350 0x7f33e80aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:24.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.451+0000 7f33edcca640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e80ae990 con 0x7f33e806d550 2023-10-18T09:01:24.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.451+0000 7f33edcca640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e81f1770 con 0x7f33e8070100 2023-10-18T09:01:24.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.451+0000 7f33edcca640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e81f2590 con 0x7f33e814e350 2023-10-18T09:01:24.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.452+0000 7f33eccc8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33e806d550 0x7f33e80cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:24.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.452+0000 7f33eccc8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33e806d550 0x7f33e80cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57992/0 (socket says 172.21.15.40:57992) 2023-10-18T09:01:24.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.452+0000 7f33eccc8640 1 -- 172.21.15.40:0/1077776455 learned_addr learned my addr 172.21.15.40:0/1077776455 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:24.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.452+0000 7f33dffff640 1 --2- 172.21.15.40:0/1077776455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33e8070100 0x7f33e80cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:24.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.452+0000 7f33ed4c9640 1 --2- 172.21.15.40:0/1077776455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33e814e350 0x7f33e80aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:24.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.452+0000 7f33eccc8640 1 -- 172.21.15.40:0/1077776455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33e814e350 msgr2=0x7f33e80aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:24.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.452+0000 7f33eccc8640 1 --2- 172.21.15.40:0/1077776455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33e814e350 0x7f33e80aca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:24.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.452+0000 7f33eccc8640 1 -- 172.21.15.40:0/1077776455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33e8070100 msgr2=0x7f33e80cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:24.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.452+0000 7f33eccc8640 1 --2- 172.21.15.40:0/1077776455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33e8070100 0x7f33e80cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:24.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.452+0000 7f33eccc8640 1 -- 172.21.15.40:0/1077776455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d8009490 con 0x7f33e806d550 2023-10-18T09:01:24.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.452+0000 7f33dffff640 1 --2- 172.21.15.40:0/1077776455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33e8070100 0x7f33e80cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:24.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.453+0000 7f33eccc8640 1 --2- 172.21.15.40:0/1077776455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33e806d550 0x7f33e80cab40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f33e400c4b0 tx=0x7f33e400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:24.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.453+0000 7f33dd7fa640 1 -- 172.21.15.40:0/1077776455 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4005480 con 0x7f33e806d550 2023-10-18T09:01:24.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.453+0000 7f33dd7fa640 1 -- 172.21.15.40:0/1077776455 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f33e40036e0 con 0x7f33e806d550 2023-10-18T09:01:24.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.453+0000 7f33edcca640 1 -- 172.21.15.40:0/1077776455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e81f1980 con 0x7f33e806d550 2023-10-18T09:01:24.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.453+0000 7f33dd7fa640 1 -- 172.21.15.40:0/1077776455 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4003b30 con 0x7f33e806d550 2023-10-18T09:01:24.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.453+0000 7f33edcca640 1 -- 172.21.15.40:0/1077776455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e81f1f40 con 0x7f33e806d550 2023-10-18T09:01:24.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.454+0000 7f33edcca640 1 -- 172.21.15.40:0/1077776455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e8066e00 con 0x7f33e806d550 2023-10-18T09:01:24.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.455+0000 7f33dd7fa640 1 -- 172.21.15.40:0/1077776455 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f33e4003c90 con 0x7f33e806d550 2023-10-18T09:01:24.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.457+0000 7f33dd7fa640 1 --2- 172.21.15.40:0/1077776455 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f33d406ae40 0x7f33d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:24.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.457+0000 7f33dd7fa640 1 -- 172.21.15.40:0/1077776455 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f33e4088d70 con 0x7f33e806d550 2023-10-18T09:01:24.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.457+0000 7f33dffff640 1 --2- 172.21.15.40:0/1077776455 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f33d406ae40 0x7f33d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:24.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.458+0000 7f33dffff640 1 --2- 172.21.15.40:0/1077776455 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f33d406ae40 0x7f33d406d300 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f33d8036000 tx=0x7f33d8004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:24.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.461+0000 7f33dd7fa640 1 -- 172.21.15.40:0/1077776455 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33e4053e50 con 0x7f33e806d550 2023-10-18T09:01:24.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:24.656+0000 7f33edcca640 1 -- 172.21.15.40:0/1077776455 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f33e80b0610 con 0x7f33e806d550 2023-10-18T09:01:25.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.194+0000 7f33dd7fa640 1 -- 172.21.15.40:0/1077776455 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) v1 ==== 169+0+0 (secure 0 0 0) 0x7f33e4017070 con 0x7f33e806d550 2023-10-18T09:01:25.197 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-10-18T09:01:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.199+0000 7f33edcca640 1 -- 172.21.15.40:0/1077776455 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f33d406ae40 msgr2=0x7f33d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.199+0000 7f33edcca640 1 --2- 172.21.15.40:0/1077776455 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f33d406ae40 0x7f33d406d300 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f33d8036000 tx=0x7f33d8004730 comp rx=0 tx=0).stop 2023-10-18T09:01:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.199+0000 7f33edcca640 1 -- 172.21.15.40:0/1077776455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33e806d550 msgr2=0x7f33e80cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.199+0000 7f33edcca640 1 --2- 172.21.15.40:0/1077776455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33e806d550 0x7f33e80cab40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f33e400c4b0 tx=0x7f33e400c970 comp rx=0 tx=0).stop 2023-10-18T09:01:25.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.199+0000 7f33edcca640 1 -- 172.21.15.40:0/1077776455 shutdown_connections 2023-10-18T09:01:25.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.199+0000 7f33edcca640 1 --2- 172.21.15.40:0/1077776455 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f33d406ae40 0x7f33d406d300 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:25.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.199+0000 7f33edcca640 1 --2- 172.21.15.40:0/1077776455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33e814e350 0x7f33e80aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:25.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.199+0000 7f33edcca640 1 --2- 172.21.15.40:0/1077776455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33e8070100 0x7f33e80cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:25.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.199+0000 7f33edcca640 1 --2- 172.21.15.40:0/1077776455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33e806d550 0x7f33e80cab40 secure :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f33e400c4b0 tx=0x7f33e400c970 comp rx=0 tx=0).stop 2023-10-18T09:01:25.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.199+0000 7f33edcca640 1 -- 172.21.15.40:0/1077776455 >> 172.21.15.40:0/1077776455 conn(0x7f33e80aeb90 msgr2=0x7f33e80b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:25.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.203+0000 7f33edcca640 1 -- 172.21.15.40:0/1077776455 shutdown_connections 2023-10-18T09:01:25.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.203+0000 7f33edcca640 1 -- 172.21.15.40:0/1077776455 wait complete. 2023-10-18T09:01:25.230 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 22 2023-10-18T09:01:25.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.462+0000 7f5d618c9640 1 -- 172.21.15.40:0/22441701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c06d710 msgr2=0x7f5d5c151bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:25.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.462+0000 7f5d618c9640 1 --2- 172.21.15.40:0/22441701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c06d710 0x7f5d5c151bd0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c06c840 tx=0x7f5d4c02c100 comp rx=0 tx=0).stop 2023-10-18T09:01:25.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.462+0000 7f5d618c9640 1 -- 172.21.15.40:0/22441701 shutdown_connections 2023-10-18T09:01:25.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.462+0000 7f5d618c9640 1 --2- 172.21.15.40:0/22441701 >> v2:172.21.15.144:3300/0 conn(0x7f5d5c1586a0 0x7f5d5c0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:25.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.462+0000 7f5d618c9640 1 --2- 172.21.15.40:0/22441701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c06d710 0x7f5d5c151bd0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:25.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.462+0000 7f5d618c9640 1 -- 172.21.15.40:0/22441701 >> 172.21.15.40:0/22441701 conn(0x7f5d5c06cee0 msgr2=0x7f5d5c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:25.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.462+0000 7f5d618c9640 1 -- 172.21.15.40:0/22441701 shutdown_connections 2023-10-18T09:01:25.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.462+0000 7f5d618c9640 1 -- 172.21.15.40:0/22441701 wait complete. 2023-10-18T09:01:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.464+0000 7f5d618c9640 1 Processor -- start 2023-10-18T09:01:25.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.464+0000 7f5d618c9640 1 -- start start 2023-10-18T09:01:25.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.465+0000 7f5d618c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c06d710 0x7f5d5c149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:25.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.465+0000 7f5d618c9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d5c0b1100 0x7f5d5c14a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:25.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.465+0000 7f5d618c9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d5c1586a0 0x7f5d5c159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:25.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.465+0000 7f5d618c9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c14b260 con 0x7f5d5c06d710 2023-10-18T09:01:25.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.465+0000 7f5d618c9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c14a490 con 0x7f5d5c1586a0 2023-10-18T09:01:25.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.465+0000 7f5d618c9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c14af80 con 0x7f5d5c0b1100 2023-10-18T09:01:25.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.465+0000 7f5d5affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c06d710 0x7f5d5c149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:25.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.465+0000 7f5d5affd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c06d710 0x7f5d5c149f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41222/0 (socket says 172.21.15.40:41222) 2023-10-18T09:01:25.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.465+0000 7f5d5affd640 1 -- 172.21.15.40:0/2536344986 learned_addr learned my addr 172.21.15.40:0/2536344986 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:25.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.466+0000 7f5d5affd640 1 -- 172.21.15.40:0/2536344986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d5c0b1100 msgr2=0x7f5d5c14a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:25.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.466+0000 7f5d5affd640 1 --2- 172.21.15.40:0/2536344986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d5c0b1100 0x7f5d5c14a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:25.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.466+0000 7f5d5affd640 1 -- 172.21.15.40:0/2536344986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d5c1586a0 msgr2=0x7f5d5c159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:25.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.466+0000 7f5d5affd640 1 --2- 172.21.15.40:0/2536344986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d5c1586a0 0x7f5d5c159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:25.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.466+0000 7f5d5affd640 1 -- 172.21.15.40:0/2536344986 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c009840 con 0x7f5d5c06d710 2023-10-18T09:01:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.466+0000 7f5d5affd640 1 --2- 172.21.15.40:0/2536344986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c06d710 0x7f5d5c149f50 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c02c710 tx=0x7f5d4c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.466+0000 7f5d53fff640 1 -- 172.21.15.40:0/2536344986 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c004760 con 0x7f5d5c06d710 2023-10-18T09:01:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.467+0000 7f5d53fff640 1 -- 172.21.15.40:0/2536344986 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5d4c0345a0 con 0x7f5d5c06d710 2023-10-18T09:01:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.467+0000 7f5d618c9640 1 -- 172.21.15.40:0/2536344986 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c14c2d0 con 0x7f5d5c06d710 2023-10-18T09:01:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.467+0000 7f5d53fff640 1 -- 172.21.15.40:0/2536344986 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c034930 con 0x7f5d5c06d710 2023-10-18T09:01:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.467+0000 7f5d618c9640 1 -- 172.21.15.40:0/2536344986 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c14c890 con 0x7f5d5c06d710 2023-10-18T09:01:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.468+0000 7f5d618c9640 1 -- 172.21.15.40:0/2536344986 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5c14c130 con 0x7f5d5c06d710 2023-10-18T09:01:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.469+0000 7f5d53fff640 1 -- 172.21.15.40:0/2536344986 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5d4c034a90 con 0x7f5d5c06d710 2023-10-18T09:01:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.470+0000 7f5d53fff640 1 --2- 172.21.15.40:0/2536344986 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d3c06ae40 0x7f5d3c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.470+0000 7f5d53fff640 1 -- 172.21.15.40:0/2536344986 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5d4c0ac920 con 0x7f5d5c06d710 2023-10-18T09:01:25.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.470+0000 7f5d5a7fc640 1 --2- 172.21.15.40:0/2536344986 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d3c06ae40 0x7f5d3c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:25.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.472+0000 7f5d5a7fc640 1 --2- 172.21.15.40:0/2536344986 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d3c06ae40 0x7f5d3c06d300 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0cd590 tx=0x7f5d44005bf0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:25.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.475+0000 7f5d53fff640 1 -- 172.21.15.40:0/2536344986 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d5c14c130 con 0x7f5d5c06d710 2023-10-18T09:01:25.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.670+0000 7f5d618c9640 1 -- 172.21.15.40:0/2536344986 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config reset", "num": 22} v 0) v1 -- 0x7f5d5c14c130 con 0x7f5d5c06d710 2023-10-18T09:01:25.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.678+0000 7f5d53fff640 1 -- 172.21.15.40:0/2536344986 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 22}]=0 v30) v1 ==== 71+0+0 (secure 0 0 0) 0x7f5d5c14c130 con 0x7f5d5c06d710 2023-10-18T09:01:25.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.681+0000 7f5d618c9640 1 -- 172.21.15.40:0/2536344986 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d3c06ae40 msgr2=0x7f5d3c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:25.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.681+0000 7f5d618c9640 1 --2- 172.21.15.40:0/2536344986 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d3c06ae40 0x7f5d3c06d300 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c0cd590 tx=0x7f5d44005bf0 comp rx=0 tx=0).stop 2023-10-18T09:01:25.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.681+0000 7f5d618c9640 1 -- 172.21.15.40:0/2536344986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c06d710 msgr2=0x7f5d5c149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:25.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.681+0000 7f5d618c9640 1 --2- 172.21.15.40:0/2536344986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c06d710 0x7f5d5c149f50 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c02c710 tx=0x7f5d4c004360 comp rx=0 tx=0).stop 2023-10-18T09:01:25.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.681+0000 7f5d618c9640 1 -- 172.21.15.40:0/2536344986 shutdown_connections 2023-10-18T09:01:25.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.681+0000 7f5d618c9640 1 --2- 172.21.15.40:0/2536344986 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d3c06ae40 0x7f5d3c06d300 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:25.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.681+0000 7f5d618c9640 1 --2- 172.21.15.40:0/2536344986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d5c1586a0 0x7f5d5c159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:25.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.681+0000 7f5d618c9640 1 --2- 172.21.15.40:0/2536344986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d5c0b1100 0x7f5d5c14a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:25.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.681+0000 7f5d618c9640 1 --2- 172.21.15.40:0/2536344986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d5c06d710 0x7f5d5c149f50 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:25.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.681+0000 7f5d618c9640 1 -- 172.21.15.40:0/2536344986 >> 172.21.15.40:0/2536344986 conn(0x7f5d5c06cee0 msgr2=0x7f5d5c0af650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:25.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.681+0000 7f5d618c9640 1 -- 172.21.15.40:0/2536344986 shutdown_connections 2023-10-18T09:01:25.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.681+0000 7f5d618c9640 1 -- 172.21.15.40:0/2536344986 wait complete. 2023-10-18T09:01:25.702 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2023-10-18T09:01:25.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.933+0000 7f9b2b59e640 1 -- 172.21.15.40:0/3027922578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b2c0ab2a0 msgr2=0x7f9b2c0cb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:25.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.933+0000 7f9b2b59e640 1 --2- 172.21.15.40:0/3027922578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b2c0ab2a0 0x7f9b2c0cb3d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009ca0 tx=0x7f9b2402c350 comp rx=0 tx=0).stop 2023-10-18T09:01:25.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.934+0000 7f9b2b59e640 1 -- 172.21.15.40:0/3027922578 shutdown_connections 2023-10-18T09:01:25.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.934+0000 7f9b2b59e640 1 --2- 172.21.15.40:0/3027922578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b2c0ab2a0 0x7f9b2c0cb3d0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:25.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.934+0000 7f9b2b59e640 1 --2- 172.21.15.40:0/3027922578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b2c152560 0x7f9b2c070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:25.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.934+0000 7f9b2b59e640 1 -- 172.21.15.40:0/3027922578 >> 172.21.15.40:0/3027922578 conn(0x7f9b2c069720 msgr2=0x7f9b2c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:25.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.934+0000 7f9b2b59e640 1 -- 172.21.15.40:0/3027922578 shutdown_connections 2023-10-18T09:01:25.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.934+0000 7f9b2b59e640 1 -- 172.21.15.40:0/3027922578 wait complete. 2023-10-18T09:01:25.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.936+0000 7f9b2b59e640 1 Processor -- start 2023-10-18T09:01:25.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.936+0000 7f9b2b59e640 1 -- start start 2023-10-18T09:01:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.937+0000 7f9b2b59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b2c071440 0x7f9b2c15cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.937+0000 7f9b2b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b2c0ab2a0 0x7f9b2c15d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:25.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.937+0000 7f9b2b59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b2c152560 0x7f9b2c1500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.937+0000 7f9b2b59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c14f810 con 0x7f9b2c0ab2a0 2023-10-18T09:01:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.937+0000 7f9b2b59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c14ec00 con 0x7f9b2c071440 2023-10-18T09:01:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.937+0000 7f9b2b59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c14eee0 con 0x7f9b2c152560 2023-10-18T09:01:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.937+0000 7f9b23fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b2c0ab2a0 0x7f9b2c15d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:25.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.937+0000 7f9b23fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b2c0ab2a0 0x7f9b2c15d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41230/0 (socket says 172.21.15.40:41230) 2023-10-18T09:01:25.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.937+0000 7f9b23fff640 1 -- 172.21.15.40:0/1856477421 learned_addr learned my addr 172.21.15.40:0/1856477421 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:25.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.938+0000 7f9b23fff640 1 -- 172.21.15.40:0/1856477421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b2c152560 msgr2=0x7f9b2c1500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:25.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.938+0000 7f9b2ad9d640 1 --2- 172.21.15.40:0/1856477421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b2c152560 0x7f9b2c1500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:25.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.938+0000 7f9b23fff640 1 --2- 172.21.15.40:0/1856477421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b2c152560 0x7f9b2c1500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:25.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.938+0000 7f9b23fff640 1 -- 172.21.15.40:0/1856477421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b2c071440 msgr2=0x7f9b2c15cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:25.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.938+0000 7f9b2a59c640 1 --2- 172.21.15.40:0/1856477421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b2c071440 0x7f9b2c15cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:25.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.938+0000 7f9b23fff640 1 --2- 172.21.15.40:0/1856477421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b2c071440 0x7f9b2c15cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:25.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.938+0000 7f9b23fff640 1 -- 172.21.15.40:0/1856477421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b24009900 con 0x7f9b2c0ab2a0 2023-10-18T09:01:25.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.938+0000 7f9b23fff640 1 --2- 172.21.15.40:0/1856477421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b2c0ab2a0 0x7f9b2c15d0f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400c450 tx=0x7f9b1400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:25.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.938+0000 7f9b22ffd640 1 -- 172.21.15.40:0/1856477421 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b140052d0 con 0x7f9b2c0ab2a0 2023-10-18T09:01:25.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.938+0000 7f9b22ffd640 1 -- 172.21.15.40:0/1856477421 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9b14005430 con 0x7f9b2c0ab2a0 2023-10-18T09:01:25.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.938+0000 7f9b22ffd640 1 -- 172.21.15.40:0/1856477421 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b14012890 con 0x7f9b2c0ab2a0 2023-10-18T09:01:25.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.938+0000 7f9b2b59e640 1 -- 172.21.15.40:0/1856477421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2c14f1c0 con 0x7f9b2c0ab2a0 2023-10-18T09:01:25.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.939+0000 7f9b2b59e640 1 -- 172.21.15.40:0/1856477421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2c14fca0 con 0x7f9b2c0ab2a0 2023-10-18T09:01:25.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.940+0000 7f9b2b59e640 1 -- 172.21.15.40:0/1856477421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2c066e00 con 0x7f9b2c0ab2a0 2023-10-18T09:01:25.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.941+0000 7f9b22ffd640 1 -- 172.21.15.40:0/1856477421 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9b140035f0 con 0x7f9b2c0ab2a0 2023-10-18T09:01:25.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.942+0000 7f9b22ffd640 1 --2- 172.21.15.40:0/1856477421 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9b0c06ae40 0x7f9b0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:25.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.942+0000 7f9b22ffd640 1 -- 172.21.15.40:0/1856477421 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9b14088fc0 con 0x7f9b2c0ab2a0 2023-10-18T09:01:25.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.942+0000 7f9b2a59c640 1 --2- 172.21.15.40:0/1856477421 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9b0c06ae40 0x7f9b0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:25.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.944+0000 7f9b2a59c640 1 --2- 172.21.15.40:0/1856477421 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9b0c06ae40 0x7f9b0c06d300 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0058c0 tx=0x7f9b1c005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:25.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:25.946+0000 7f9b22ffd640 1 -- 172.21.15.40:0/1856477421 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9b14054360 con 0x7f9b2c0ab2a0 2023-10-18T09:01:26.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:26.218+0000 7f9b2b59e640 1 -- 172.21.15.40:0/1856477421 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f9b2c1552b0 con 0x7f9b2c0ab2a0 2023-10-18T09:01:27.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.200+0000 7f9b22ffd640 1 -- 172.21.15.40:0/1856477421 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v319) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9b14057980 con 0x7f9b2c0ab2a0 2023-10-18T09:01:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.204+0000 7f9b2b59e640 1 -- 172.21.15.40:0/1856477421 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9b0c06ae40 msgr2=0x7f9b0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.204+0000 7f9b2b59e640 1 --2- 172.21.15.40:0/1856477421 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9b0c06ae40 0x7f9b0c06d300 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0058c0 tx=0x7f9b1c005850 comp rx=0 tx=0).stop 2023-10-18T09:01:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.204+0000 7f9b2b59e640 1 -- 172.21.15.40:0/1856477421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b2c0ab2a0 msgr2=0x7f9b2c15d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.204+0000 7f9b2b59e640 1 --2- 172.21.15.40:0/1856477421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b2c0ab2a0 0x7f9b2c15d0f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f9b1400c450 tx=0x7f9b1400c910 comp rx=0 tx=0).stop 2023-10-18T09:01:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.205+0000 7f9b2b59e640 1 -- 172.21.15.40:0/1856477421 shutdown_connections 2023-10-18T09:01:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.205+0000 7f9b2b59e640 1 --2- 172.21.15.40:0/1856477421 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9b0c06ae40 0x7f9b0c06d300 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.205+0000 7f9b2b59e640 1 --2- 172.21.15.40:0/1856477421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b2c152560 0x7f9b2c1500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.205+0000 7f9b2b59e640 1 --2- 172.21.15.40:0/1856477421 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b2c0ab2a0 0x7f9b2c15d0f0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.205+0000 7f9b2b59e640 1 --2- 172.21.15.40:0/1856477421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b2c071440 0x7f9b2c15cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.205+0000 7f9b2b59e640 1 -- 172.21.15.40:0/1856477421 >> 172.21.15.40:0/1856477421 conn(0x7f9b2c069720 msgr2=0x7f9b2c15e060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.205+0000 7f9b2b59e640 1 -- 172.21.15.40:0/1856477421 shutdown_connections 2023-10-18T09:01:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.205+0000 7f9b2b59e640 1 -- 172.21.15.40:0/1856477421 wait complete. 2023-10-18T09:01:27.230 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2023-10-18T09:01:27.230 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestClusterFull) 2023-10-18T09:01:27.231 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:01:27.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.464+0000 7fe072a9a640 1 -- 172.21.15.40:0/2843156971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe058004d00 msgr2=0x7fe058005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:27.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.464+0000 7fe072a9a640 1 --2- 172.21.15.40:0/2843156971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe058004d00 0x7fe058005180 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fe064009ed0 tx=0x7fe0580057f0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.464+0000 7fe072299640 1 -- 172.21.15.40:0/2843156971 reap_dead start 2023-10-18T09:01:27.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.465+0000 7fe072a9a640 1 -- 172.21.15.40:0/2843156971 shutdown_connections 2023-10-18T09:01:27.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.465+0000 7fe072a9a640 1 -- 172.21.15.40:0/2843156971 >> 172.21.15.40:0/2843156971 conn(0x7fe06c069720 msgr2=0x7fe06c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:27.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.465+0000 7fe072a9a640 1 -- 172.21.15.40:0/2843156971 shutdown_connections 2023-10-18T09:01:27.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.465+0000 7fe072a9a640 1 -- 172.21.15.40:0/2843156971 wait complete. 2023-10-18T09:01:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.466+0000 7fe072a9a640 1 Processor -- start 2023-10-18T09:01:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.467+0000 7fe072a9a640 1 -- start start 2023-10-18T09:01:27.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.468+0000 7fe072a9a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe058004d00 0x7fe06c14bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.468+0000 7fe072a9a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe06c0ab4c0 0x7fe06c0ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.468+0000 7fe072a9a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c1e41c0 0x7fe06c1e4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.468+0000 7fe072a9a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c0adc60 con 0x7fe06c1e41c0 2023-10-18T09:01:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.468+0000 7fe072a9a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c1e4bb0 con 0x7fe06c0ab4c0 2023-10-18T09:01:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.468+0000 7fe072a9a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c0ab940 con 0x7fe058004d00 2023-10-18T09:01:27.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.468+0000 7fe072299640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c1e41c0 0x7fe06c1e4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:27.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.468+0000 7fe072299640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c1e41c0 0x7fe06c1e4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41242/0 (socket says 172.21.15.40:41242) 2023-10-18T09:01:27.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.468+0000 7fe072299640 1 -- 172.21.15.40:0/1743437176 learned_addr learned my addr 172.21.15.40:0/1743437176 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:27.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.468+0000 7fe071297640 1 --2- 172.21.15.40:0/1743437176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe06c0ab4c0 0x7fe06c0ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:27.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.468+0000 7fe072299640 1 -- 172.21.15.40:0/1743437176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe058004d00 msgr2=0x7fe06c14bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:27.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.468+0000 7fe072299640 1 --2- 172.21.15.40:0/1743437176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe058004d00 0x7fe06c14bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.468+0000 7fe072299640 1 -- 172.21.15.40:0/1743437176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe06c0ab4c0 msgr2=0x7fe06c0ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:27.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.468+0000 7fe072299640 1 --2- 172.21.15.40:0/1743437176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe06c0ab4c0 0x7fe06c0ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.468+0000 7fe072299640 1 -- 172.21.15.40:0/1743437176 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06c15bc10 con 0x7fe06c1e41c0 2023-10-18T09:01:27.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.469+0000 7fe072299640 1 --2- 172.21.15.40:0/1743437176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c1e41c0 0x7fe06c1e4670 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fe064009ed0 tx=0x7fe0640306e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:27.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.469+0000 7fe0627fc640 1 -- 172.21.15.40:0/1743437176 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064003b70 con 0x7fe06c1e41c0 2023-10-18T09:01:27.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.469+0000 7fe0627fc640 1 -- 172.21.15.40:0/1743437176 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe064003dc0 con 0x7fe06c1e41c0 2023-10-18T09:01:27.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.469+0000 7fe0627fc640 1 -- 172.21.15.40:0/1743437176 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064004250 con 0x7fe06c1e41c0 2023-10-18T09:01:27.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.469+0000 7fe072a9a640 1 -- 172.21.15.40:0/1743437176 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe06c0ada30 con 0x7fe06c1e41c0 2023-10-18T09:01:27.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.469+0000 7fe072a9a640 1 -- 172.21.15.40:0/1743437176 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06c0af950 con 0x7fe06c1e41c0 2023-10-18T09:01:27.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.470+0000 7fe072a9a640 1 -- 172.21.15.40:0/1743437176 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe06c066e00 con 0x7fe06c1e41c0 2023-10-18T09:01:27.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.471+0000 7fe0627fc640 1 -- 172.21.15.40:0/1743437176 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe0640043b0 con 0x7fe06c1e41c0 2023-10-18T09:01:27.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.472+0000 7fe0627fc640 1 --2- 172.21.15.40:0/1743437176 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe05006ae70 0x7fe05006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:27.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.472+0000 7fe0627fc640 1 -- 172.21.15.40:0/1743437176 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe0640abaa0 con 0x7fe06c1e41c0 2023-10-18T09:01:27.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.473+0000 7fe071a98640 1 --2- 172.21.15.40:0/1743437176 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe05006ae70 0x7fe05006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:27.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.474+0000 7fe071a98640 1 --2- 172.21.15.40:0/1743437176 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe05006ae70 0x7fe05006d330 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe068001eb0 tx=0x7fe068000f50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:27.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.478+0000 7fe0627fc640 1 -- 172.21.15.40:0/1743437176 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe064076d20 con 0x7fe06c1e41c0 2023-10-18T09:01:27.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.671+0000 7fe072a9a640 1 -- 172.21.15.40:0/1743437176 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe06c0ad2d0 con 0x7fe06c1e41c0 2023-10-18T09:01:27.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.672+0000 7fe0627fc640 1 -- 172.21.15.40:0/1743437176 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+12999 (secure 0 0 0) 0x7fe064030e60 con 0x7fe06c1e41c0 2023-10-18T09:01:27.675 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:27.675 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":98,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:01:25.185355+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/1926421699":"2023-10-19T09:01:21.796842+0000","172.21.15.40:6834/1926421699":"2023-10-19T09:01:21.796842+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:01:27.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.675+0000 7fe072a9a640 1 -- 172.21.15.40:0/1743437176 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe05006ae70 msgr2=0x7fe05006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:27.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.675+0000 7fe072a9a640 1 --2- 172.21.15.40:0/1743437176 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe05006ae70 0x7fe05006d330 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe068001eb0 tx=0x7fe068000f50 comp rx=0 tx=0).stop 2023-10-18T09:01:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.675+0000 7fe072a9a640 1 -- 172.21.15.40:0/1743437176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c1e41c0 msgr2=0x7fe06c1e4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.675+0000 7fe072a9a640 1 --2- 172.21.15.40:0/1743437176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c1e41c0 0x7fe06c1e4670 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fe064009ed0 tx=0x7fe0640306e0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.675+0000 7fe072a9a640 1 -- 172.21.15.40:0/1743437176 shutdown_connections 2023-10-18T09:01:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.675+0000 7fe072a9a640 1 --2- 172.21.15.40:0/1743437176 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe05006ae70 0x7fe05006d330 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.675+0000 7fe072a9a640 1 --2- 172.21.15.40:0/1743437176 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe06c1e41c0 0x7fe06c1e4670 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.675+0000 7fe072a9a640 1 --2- 172.21.15.40:0/1743437176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe06c0ab4c0 0x7fe06c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.675+0000 7fe072a9a640 1 --2- 172.21.15.40:0/1743437176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe058004d00 0x7fe06c14bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.675+0000 7fe072a9a640 1 -- 172.21.15.40:0/1743437176 >> 172.21.15.40:0/1743437176 conn(0x7fe06c069720 msgr2=0x7fe06c0cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.675+0000 7fe072a9a640 1 -- 172.21.15.40:0/1743437176 shutdown_connections 2023-10-18T09:01:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.675+0000 7fe072a9a640 1 -- 172.21.15.40:0/1743437176 wait complete. 2023-10-18T09:01:27.697 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:01:27.698 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:01:27.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.930+0000 7fbb7b59e640 1 -- 172.21.15.40:0/1847695435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c1405a0 msgr2=0x7fbb7c0b0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:27.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.930+0000 7fbb7b59e640 1 --2- 172.21.15.40:0/1847695435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c1405a0 0x7fbb7c0b0e30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0ca340 tx=0x7fbb7402bdb0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.931+0000 7fbb7b59e640 1 -- 172.21.15.40:0/1847695435 shutdown_connections 2023-10-18T09:01:27.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.931+0000 7fbb7b59e640 1 --2- 172.21.15.40:0/1847695435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c1405a0 0x7fbb7c0b0e30 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.931+0000 7fbb7b59e640 1 --2- 172.21.15.40:0/1847695435 >> v2:172.21.15.144:3300/0 conn(0x7fbb7c06fc20 0x7fbb7c0b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.931+0000 7fbb7b59e640 1 -- 172.21.15.40:0/1847695435 >> 172.21.15.40:0/1847695435 conn(0x7fbb7c069720 msgr2=0x7fbb7c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:27.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.931+0000 7fbb7b59e640 1 -- 172.21.15.40:0/1847695435 shutdown_connections 2023-10-18T09:01:27.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.931+0000 7fbb7b59e640 1 -- 172.21.15.40:0/1847695435 wait complete. 2023-10-18T09:01:27.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.932+0000 7fbb7b59e640 1 Processor -- start 2023-10-18T09:01:27.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.933+0000 7fbb7b59e640 1 -- start start 2023-10-18T09:01:27.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.934+0000 7fbb7b59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb7c06fc20 0x7fbb7c0a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:27.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.934+0000 7fbb7b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c0c98a0 0x7fbb7c13f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:27.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.934+0000 7fbb7b59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb7c1405a0 0x7fbb7c0ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:27.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.934+0000 7fbb7b59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c141610 con 0x7fbb7c0c98a0 2023-10-18T09:01:27.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.934+0000 7fbb7b59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c140030 con 0x7fbb7c1405a0 2023-10-18T09:01:27.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.934+0000 7fbb7b59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c13fa70 con 0x7fbb7c06fc20 2023-10-18T09:01:27.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.934+0000 7fbb79d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c0c98a0 0x7fbb7c13f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:27.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.934+0000 7fbb79d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c0c98a0 0x7fbb7c13f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41260/0 (socket says 172.21.15.40:41260) 2023-10-18T09:01:27.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.934+0000 7fbb79d9b640 1 -- 172.21.15.40:0/1339142187 learned_addr learned my addr 172.21.15.40:0/1339142187 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:27.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.934+0000 7fbb7ad9d640 1 --2- 172.21.15.40:0/1339142187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb7c1405a0 0x7fbb7c0ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:27.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.934+0000 7fbb79d9b640 1 -- 172.21.15.40:0/1339142187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb7c06fc20 msgr2=0x7fbb7c0a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:27.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.934+0000 7fbb79d9b640 1 --2- 172.21.15.40:0/1339142187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb7c06fc20 0x7fbb7c0a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.934+0000 7fbb79d9b640 1 -- 172.21.15.40:0/1339142187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb7c1405a0 msgr2=0x7fbb7c0ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:27.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.934+0000 7fbb79d9b640 1 --2- 172.21.15.40:0/1339142187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb7c1405a0 0x7fbb7c0ab940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:27.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.934+0000 7fbb79d9b640 1 -- 172.21.15.40:0/1339142187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb74009560 con 0x7fbb7c0c98a0 2023-10-18T09:01:27.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.934+0000 7fbb7ad9d640 1 --2- 172.21.15.40:0/1339142187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb7c1405a0 0x7fbb7c0ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:27.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.935+0000 7fbb79d9b640 1 --2- 172.21.15.40:0/1339142187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c0c98a0 0x7fbb7c13f080 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400eab0 tx=0x7fbb6400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:27.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.935+0000 7fbb72ffd640 1 -- 172.21.15.40:0/1339142187 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6400ce40 con 0x7fbb7c0c98a0 2023-10-18T09:01:27.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.935+0000 7fbb72ffd640 1 -- 172.21.15.40:0/1339142187 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbb64004c10 con 0x7fbb7c0c98a0 2023-10-18T09:01:27.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.935+0000 7fbb72ffd640 1 -- 172.21.15.40:0/1339142187 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64005020 con 0x7fbb7c0c98a0 2023-10-18T09:01:27.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.935+0000 7fbb7b59e640 1 -- 172.21.15.40:0/1339142187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7c13fd50 con 0x7fbb7c0c98a0 2023-10-18T09:01:27.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.935+0000 7fbb7b59e640 1 -- 172.21.15.40:0/1339142187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c0a6c20 con 0x7fbb7c0c98a0 2023-10-18T09:01:27.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.936+0000 7fbb7b59e640 1 -- 172.21.15.40:0/1339142187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7c066e00 con 0x7fbb7c0c98a0 2023-10-18T09:01:27.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.937+0000 7fbb72ffd640 1 -- 172.21.15.40:0/1339142187 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbb64005180 con 0x7fbb7c0c98a0 2023-10-18T09:01:27.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.938+0000 7fbb72ffd640 1 --2- 172.21.15.40:0/1339142187 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb5c06ae40 0x7fbb5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:27.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.938+0000 7fbb72ffd640 1 -- 172.21.15.40:0/1339142187 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbb64089ed0 con 0x7fbb7c0c98a0 2023-10-18T09:01:27.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.939+0000 7fbb7a59c640 1 --2- 172.21.15.40:0/1339142187 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb5c06ae40 0x7fbb5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:27.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.940+0000 7fbb7a59c640 1 --2- 172.21.15.40:0/1339142187 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb5c06ae40 0x7fbb5c06d300 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c003cc0 tx=0x7fbb6c0079b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:27.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:27.942+0000 7fbb72ffd640 1 -- 172.21.15.40:0/1339142187 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb640551e0 con 0x7fbb7c0c98a0 2023-10-18T09:01:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.134+0000 7fbb7b59e640 1 -- 172.21.15.40:0/1339142187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb7c070c60 con 0x7fbb7c0c98a0 2023-10-18T09:01:28.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.134+0000 7fbb72ffd640 1 -- 172.21.15.40:0/1339142187 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb6401f020 con 0x7fbb7c0c98a0 2023-10-18T09:01:28.137 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:01:28.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.137+0000 7fbb7b59e640 1 -- 172.21.15.40:0/1339142187 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb5c06ae40 msgr2=0x7fbb5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:28.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.137+0000 7fbb7b59e640 1 --2- 172.21.15.40:0/1339142187 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb5c06ae40 0x7fbb5c06d300 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c003cc0 tx=0x7fbb6c0079b0 comp rx=0 tx=0).stop 2023-10-18T09:01:28.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.137+0000 7fbb7b59e640 1 -- 172.21.15.40:0/1339142187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c0c98a0 msgr2=0x7fbb7c13f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:28.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.137+0000 7fbb7b59e640 1 --2- 172.21.15.40:0/1339142187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c0c98a0 0x7fbb7c13f080 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400eab0 tx=0x7fbb6400ef70 comp rx=0 tx=0).stop 2023-10-18T09:01:28.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.137+0000 7fbb7b59e640 1 -- 172.21.15.40:0/1339142187 shutdown_connections 2023-10-18T09:01:28.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.137+0000 7fbb7b59e640 1 --2- 172.21.15.40:0/1339142187 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbb5c06ae40 0x7fbb5c06d300 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:28.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.137+0000 7fbb7b59e640 1 --2- 172.21.15.40:0/1339142187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb7c1405a0 0x7fbb7c0ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:28.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.137+0000 7fbb7b59e640 1 --2- 172.21.15.40:0/1339142187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c0c98a0 0x7fbb7c13f080 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:28.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.137+0000 7fbb7b59e640 1 --2- 172.21.15.40:0/1339142187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb7c06fc20 0x7fbb7c0a66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:28.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.137+0000 7fbb7b59e640 1 -- 172.21.15.40:0/1339142187 >> 172.21.15.40:0/1339142187 conn(0x7fbb7c069720 msgr2=0x7fbb7c159330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:28.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.137+0000 7fbb7b59e640 1 -- 172.21.15.40:0/1339142187 shutdown_connections 2023-10-18T09:01:28.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.137+0000 7fbb7b59e640 1 -- 172.21.15.40:0/1339142187 wait complete. 2023-10-18T09:01:28.160 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2023-10-18T09:01:28.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.392+0000 7efe62b9e640 1 -- 172.21.15.40:0/1745481293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c0d1740 msgr2=0x7efe5c06a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:28.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.392+0000 7efe62b9e640 1 --2- 172.21.15.40:0/1745481293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c0d1740 0x7efe5c06a5a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efe4c005fd0 tx=0x7efe4c02be20 comp rx=0 tx=0).stop 2023-10-18T09:01:28.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.393+0000 7efe62b9e640 1 -- 172.21.15.40:0/1745481293 shutdown_connections 2023-10-18T09:01:28.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.393+0000 7efe62b9e640 1 --2- 172.21.15.40:0/1745481293 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c0d1740 0x7efe5c06a5a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:28.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.393+0000 7efe62b9e640 1 --2- 172.21.15.40:0/1745481293 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe5c15b530 0x7efe5c0d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:28.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.393+0000 7efe62b9e640 1 -- 172.21.15.40:0/1745481293 >> 172.21.15.40:0/1745481293 conn(0x7efe5c08ac60 msgr2=0x7efe5c09a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:28.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.393+0000 7efe62b9e640 1 -- 172.21.15.40:0/1745481293 shutdown_connections 2023-10-18T09:01:28.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.393+0000 7efe62b9e640 1 -- 172.21.15.40:0/1745481293 wait complete. 2023-10-18T09:01:28.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.395+0000 7efe62b9e640 1 Processor -- start 2023-10-18T09:01:28.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.395+0000 7efe62b9e640 1 -- start start 2023-10-18T09:01:28.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.396+0000 7efe62b9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe5c06aae0 0x7efe5c15e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:28.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.397+0000 7efe62b9e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe5c0d1740 0x7efe5c148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:28.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.397+0000 7efe62b9e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c15b530 0x7efe5c0c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:28.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.397+0000 7efe62b9e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c15ec70 con 0x7efe5c06aae0 2023-10-18T09:01:28.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.397+0000 7efe62b9e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c0b4420 con 0x7efe5c15b530 2023-10-18T09:01:28.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.397+0000 7efe62b9e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c0b3e60 con 0x7efe5c0d1740 2023-10-18T09:01:28.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.397+0000 7efe61b9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe5c06aae0 0x7efe5c15e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:28.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.397+0000 7efe61b9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe5c06aae0 0x7efe5c15e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41266/0 (socket says 172.21.15.40:41266) 2023-10-18T09:01:28.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.397+0000 7efe61b9c640 1 -- 172.21.15.40:0/211653265 learned_addr learned my addr 172.21.15.40:0/211653265 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:28.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.397+0000 7efe6139b640 1 --2- 172.21.15.40:0/211653265 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe5c0d1740 0x7efe5c148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:28.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.397+0000 7efe6239d640 1 --2- 172.21.15.40:0/211653265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c15b530 0x7efe5c0c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:28.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.397+0000 7efe61b9c640 1 -- 172.21.15.40:0/211653265 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe5c0d1740 msgr2=0x7efe5c148320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:28.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.397+0000 7efe61b9c640 1 --2- 172.21.15.40:0/211653265 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe5c0d1740 0x7efe5c148320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:28.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.397+0000 7efe61b9c640 1 -- 172.21.15.40:0/211653265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c15b530 msgr2=0x7efe5c0c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:28.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.397+0000 7efe61b9c640 1 --2- 172.21.15.40:0/211653265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c15b530 0x7efe5c0c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:28.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.397+0000 7efe61b9c640 1 -- 172.21.15.40:0/211653265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c009560 con 0x7efe5c06aae0 2023-10-18T09:01:28.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.398+0000 7efe61b9c640 1 --2- 172.21.15.40:0/211653265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe5c06aae0 0x7efe5c15e730 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7efe5800e790 tx=0x7efe5800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:28.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.398+0000 7efe527fc640 1 -- 172.21.15.40:0/211653265 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58009580 con 0x7efe5c06aae0 2023-10-18T09:01:28.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.398+0000 7efe527fc640 1 -- 172.21.15.40:0/211653265 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efe5800cdb0 con 0x7efe5c06aae0 2023-10-18T09:01:28.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.398+0000 7efe62b9e640 1 -- 172.21.15.40:0/211653265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5c0b4140 con 0x7efe5c06aae0 2023-10-18T09:01:28.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.398+0000 7efe527fc640 1 -- 172.21.15.40:0/211653265 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58010690 con 0x7efe5c06aae0 2023-10-18T09:01:28.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.398+0000 7efe62b9e640 1 -- 172.21.15.40:0/211653265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5c0b6e70 con 0x7efe5c06aae0 2023-10-18T09:01:28.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.399+0000 7efe62b9e640 1 -- 172.21.15.40:0/211653265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe5c0c7eb0 con 0x7efe5c06aae0 2023-10-18T09:01:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.400+0000 7efe527fc640 1 -- 172.21.15.40:0/211653265 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efe580107f0 con 0x7efe5c06aae0 2023-10-18T09:01:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.401+0000 7efe527fc640 1 --2- 172.21.15.40:0/211653265 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe4006ae40 0x7efe4006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:28.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.401+0000 7efe527fc640 1 -- 172.21.15.40:0/211653265 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efe58014070 con 0x7efe5c06aae0 2023-10-18T09:01:28.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.402+0000 7efe6139b640 1 --2- 172.21.15.40:0/211653265 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe4006ae40 0x7efe4006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:28.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.403+0000 7efe6139b640 1 --2- 172.21.15.40:0/211653265 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe4006ae40 0x7efe4006d300 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efe4c035000 tx=0x7efe4c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:28.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.406+0000 7efe527fc640 1 -- 172.21.15.40:0/211653265 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe58051ca0 con 0x7efe5c06aae0 2023-10-18T09:01:28.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:28.670+0000 7efe62b9e640 1 -- 172.21.15.40:0/211653265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7efe5c066e00 con 0x7efe5c06aae0 2023-10-18T09:01:29.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.213+0000 7efe527fc640 1 -- 172.21.15.40:0/211653265 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v321) v1 ==== 140+0+0 (secure 0 0 0) 0x7efe58055170 con 0x7efe5c06aae0 2023-10-18T09:01:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.216+0000 7efe62b9e640 1 -- 172.21.15.40:0/211653265 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe4006ae40 msgr2=0x7efe4006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.216+0000 7efe62b9e640 1 --2- 172.21.15.40:0/211653265 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe4006ae40 0x7efe4006d300 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efe4c035000 tx=0x7efe4c004730 comp rx=0 tx=0).stop 2023-10-18T09:01:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.216+0000 7efe62b9e640 1 -- 172.21.15.40:0/211653265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe5c06aae0 msgr2=0x7efe5c15e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.216+0000 7efe62b9e640 1 --2- 172.21.15.40:0/211653265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe5c06aae0 0x7efe5c15e730 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7efe5800e790 tx=0x7efe5800ec50 comp rx=0 tx=0).stop 2023-10-18T09:01:29.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.216+0000 7efe62b9e640 1 -- 172.21.15.40:0/211653265 shutdown_connections 2023-10-18T09:01:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.216+0000 7efe62b9e640 1 --2- 172.21.15.40:0/211653265 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe4006ae40 0x7efe4006d300 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.216+0000 7efe62b9e640 1 --2- 172.21.15.40:0/211653265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe5c15b530 0x7efe5c0c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.216+0000 7efe62b9e640 1 --2- 172.21.15.40:0/211653265 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe5c0d1740 0x7efe5c148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.216+0000 7efe62b9e640 1 --2- 172.21.15.40:0/211653265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe5c06aae0 0x7efe5c15e730 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.216+0000 7efe62b9e640 1 -- 172.21.15.40:0/211653265 >> 172.21.15.40:0/211653265 conn(0x7efe5c08ac60 msgr2=0x7efe5c0b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.216+0000 7efe62b9e640 1 -- 172.21.15.40:0/211653265 shutdown_connections 2023-10-18T09:01:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.217+0000 7efe62b9e640 1 -- 172.21.15.40:0/211653265 wait complete. 2023-10-18T09:01:29.241 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-18T09:01:29.481 INFO:teuthology.orchestra.run.smithi040.stdout:{"osd_objectstore":"memstore"} 2023-10-18T09:01:29.482 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-18T09:01:29.482 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T09:01:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.711+0000 7fb013fff640 1 -- 172.21.15.40:0/1515790142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faffc003970 msgr2=0x7faffc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.711+0000 7fb013fff640 1 --2- 172.21.15.40:0/1515790142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faffc003970 0x7faffc003d90 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb004009b80 tx=0x7faffc0057f0 comp rx=0 tx=0).stop 2023-10-18T09:01:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.711+0000 7fb0137fe640 1 -- 172.21.15.40:0/1515790142 reap_dead start 2023-10-18T09:01:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.711+0000 7fb013fff640 1 -- 172.21.15.40:0/1515790142 shutdown_connections 2023-10-18T09:01:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.711+0000 7fb013fff640 1 -- 172.21.15.40:0/1515790142 >> 172.21.15.40:0/1515790142 conn(0x7fb0140aed70 msgr2=0x7fb0140af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.711+0000 7fb013fff640 1 -- 172.21.15.40:0/1515790142 shutdown_connections 2023-10-18T09:01:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.711+0000 7fb013fff640 1 -- 172.21.15.40:0/1515790142 wait complete. 2023-10-18T09:01:29.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.713+0000 7fb013fff640 1 Processor -- start 2023-10-18T09:01:29.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.713+0000 7fb013fff640 1 -- start start 2023-10-18T09:01:29.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.714+0000 7fb013fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0140b22a0 0x7fb0141f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:29.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.714+0000 7fb013fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb014142560 0x7fb0141f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:29.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.714+0000 7fb013fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0140b2680 0x7fb01413a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:29.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.714+0000 7fb013fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0141f5c70 con 0x7fb0140b22a0 2023-10-18T09:01:29.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.714+0000 7fb013fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0141f5f50 con 0x7fb014142560 2023-10-18T09:01:29.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.714+0000 7fb013fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0141f6230 con 0x7fb0140b2680 2023-10-18T09:01:29.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.714+0000 7fb012ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0140b22a0 0x7fb0141f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:29.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.714+0000 7fb012ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0140b22a0 0x7fb0141f5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41278/0 (socket says 172.21.15.40:41278) 2023-10-18T09:01:29.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.714+0000 7fb012ffd640 1 -- 172.21.15.40:0/2393917123 learned_addr learned my addr 172.21.15.40:0/2393917123 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:29.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.715+0000 7fb0137fe640 1 --2- 172.21.15.40:0/2393917123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0140b2680 0x7fb01413a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:29.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.715+0000 7fb012ffd640 1 -- 172.21.15.40:0/2393917123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0140b2680 msgr2=0x7fb01413a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:29.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.715+0000 7fb0127fc640 1 --2- 172.21.15.40:0/2393917123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb014142560 0x7fb0141f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:29.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.715+0000 7fb012ffd640 1 --2- 172.21.15.40:0/2393917123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0140b2680 0x7fb01413a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.715+0000 7fb012ffd640 1 -- 172.21.15.40:0/2393917123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb014142560 msgr2=0x7fb0141f2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.715+0000 7fb012ffd640 1 --2- 172.21.15.40:0/2393917123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb014142560 0x7fb0141f2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.715+0000 7fb012ffd640 1 -- 172.21.15.40:0/2393917123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0141f1e90 con 0x7fb0140b22a0 2023-10-18T09:01:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.715+0000 7fb0127fc640 1 --2- 172.21.15.40:0/2393917123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb014142560 0x7fb0141f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.715+0000 7fb0137fe640 1 --2- 172.21.15.40:0/2393917123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0140b2680 0x7fb01413a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.715+0000 7fb012ffd640 1 --2- 172.21.15.40:0/2393917123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0140b22a0 0x7fb0141f5370 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb004009ca0 tx=0x7fb004033c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.716+0000 7faffbfff640 1 -- 172.21.15.40:0/2393917123 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00403e830 con 0x7fb0140b22a0 2023-10-18T09:01:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.716+0000 7faffbfff640 1 -- 172.21.15.40:0/2393917123 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb00403e990 con 0x7fb0140b22a0 2023-10-18T09:01:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.716+0000 7faffbfff640 1 -- 172.21.15.40:0/2393917123 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00403eda0 con 0x7fb0140b22a0 2023-10-18T09:01:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.716+0000 7fb013fff640 1 -- 172.21.15.40:0/2393917123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0141f2170 con 0x7fb0140b22a0 2023-10-18T09:01:29.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.716+0000 7fb013fff640 1 -- 172.21.15.40:0/2393917123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0141f2bd0 con 0x7fb0140b22a0 2023-10-18T09:01:29.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.717+0000 7fb013fff640 1 -- 172.21.15.40:0/2393917123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb014066e00 con 0x7fb0140b22a0 2023-10-18T09:01:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.718+0000 7faffbfff640 1 -- 172.21.15.40:0/2393917123 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb004046400 con 0x7fb0140b22a0 2023-10-18T09:01:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.719+0000 7faffbfff640 1 --2- 172.21.15.40:0/2393917123 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb00006ae70 0x7fb00006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.719+0000 7faffbfff640 1 -- 172.21.15.40:0/2393917123 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb004033e40 con 0x7fb0140b22a0 2023-10-18T09:01:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.720+0000 7fb0127fc640 1 --2- 172.21.15.40:0/2393917123 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb00006ae70 0x7fb00006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.721+0000 7fb0127fc640 1 --2- 172.21.15.40:0/2393917123 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb00006ae70 0x7fb00006d330 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb014142c80 tx=0x7fb008009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.724+0000 7faffbfff640 1 -- 172.21.15.40:0/2393917123 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb004077810 con 0x7fb0140b22a0 2023-10-18T09:01:29.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.920+0000 7fb013fff640 1 -- 172.21.15.40:0/2393917123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7fb01406c0d0 con 0x7fb0140b22a0 2023-10-18T09:01:29.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.921+0000 7faffbfff640 1 -- 172.21.15.40:0/2393917123 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v30) v1 ==== 59+0+1452 (secure 0 0 0) 0x7fb00404b5c0 con 0x7fb0140b22a0 2023-10-18T09:01:29.923 INFO:teuthology.orchestra.run.smithi040.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T09:01:29.924 INFO:teuthology.orchestra.run.smithi040.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T09:01:29.924 INFO:teuthology.orchestra.run.smithi040.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T09:01:29.924 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds 20/20 2023-10-18T09:01:29.924 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T09:01:29.924 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_ms 1/1 2023-10-18T09:01:29.925 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T09:01:29.925 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T09:01:29.925 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T09:01:29.925 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_size 100 2023-10-18T09:01:29.925 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_frag true 2023-10-18T09:01:29.926 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_scatterstat true 2023-10-18T09:01:29.926 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T09:01:29.926 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_verify_scatter true 2023-10-18T09:01:29.926 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T09:01:29.926 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T09:01:29.927 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T09:01:29.927 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced client_mount_timeout 600 2023-10-18T09:01:29.927 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_client 20/20 2023-10-18T09:01:29.927 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_ms 1/1 2023-10-18T09:01:29.927 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T09:01:29.928 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T09:01:29.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.923+0000 7fb013fff640 1 -- 172.21.15.40:0/2393917123 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb00006ae70 msgr2=0x7fb00006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:29.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.924+0000 7fb013fff640 1 --2- 172.21.15.40:0/2393917123 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb00006ae70 0x7fb00006d330 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb014142c80 tx=0x7fb008009040 comp rx=0 tx=0).stop 2023-10-18T09:01:29.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.924+0000 7fb013fff640 1 -- 172.21.15.40:0/2393917123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0140b22a0 msgr2=0x7fb0141f5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:29.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.924+0000 7fb013fff640 1 --2- 172.21.15.40:0/2393917123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0140b22a0 0x7fb0141f5370 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb004009ca0 tx=0x7fb004033c50 comp rx=0 tx=0).stop 2023-10-18T09:01:29.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.924+0000 7fb013fff640 1 -- 172.21.15.40:0/2393917123 shutdown_connections 2023-10-18T09:01:29.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.924+0000 7fb013fff640 1 --2- 172.21.15.40:0/2393917123 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb00006ae70 0x7fb00006d330 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:29.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.924+0000 7fb013fff640 1 --2- 172.21.15.40:0/2393917123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0140b2680 0x7fb01413a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:29.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.924+0000 7fb013fff640 1 --2- 172.21.15.40:0/2393917123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb014142560 0x7fb0141f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:29.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.924+0000 7fb013fff640 1 --2- 172.21.15.40:0/2393917123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0140b22a0 0x7fb0141f5370 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:29.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.924+0000 7fb013fff640 1 -- 172.21.15.40:0/2393917123 >> 172.21.15.40:0/2393917123 conn(0x7fb0140aed70 msgr2=0x7fb0140af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:29.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.924+0000 7fb013fff640 1 -- 172.21.15.40:0/2393917123 shutdown_connections 2023-10-18T09:01:29.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:29.924+0000 7fb013fff640 1 -- 172.21.15.40:0/2393917123 wait complete. 2023-10-18T09:01:29.947 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T09:01:30.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.184+0000 7f0ecbfff640 1 -- 172.21.15.40:0/2468106122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb80058d0 msgr2=0x7f0eb8005d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:30.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.184+0000 7f0ecbfff640 1 --2- 172.21.15.40:0/2468106122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0eb80058d0 0x7f0eb8005d50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400bb20 tx=0x7f0eb8006900 comp rx=0 tx=0).stop 2023-10-18T09:01:30.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.184+0000 7f0ecb7fe640 1 -- 172.21.15.40:0/2468106122 reap_dead start 2023-10-18T09:01:30.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.184+0000 7f0ecbfff640 1 -- 172.21.15.40:0/2468106122 shutdown_connections 2023-10-18T09:01:30.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.184+0000 7f0ecbfff640 1 -- 172.21.15.40:0/2468106122 >> 172.21.15.40:0/2468106122 conn(0x7f0ecc06ca10 msgr2=0x7f0ecc06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:30.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.184+0000 7f0ecbfff640 1 -- 172.21.15.40:0/2468106122 shutdown_connections 2023-10-18T09:01:30.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.184+0000 7f0ecbfff640 1 -- 172.21.15.40:0/2468106122 wait complete. 2023-10-18T09:01:30.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.186+0000 7f0ecbfff640 1 Processor -- start 2023-10-18T09:01:30.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.186+0000 7f0ecbfff640 1 -- start start 2023-10-18T09:01:30.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.187+0000 7f0ecbfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0eb80058d0 0x7f0ecc14df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:30.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.187+0000 7f0ecbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ecc0b1c60 0x7f0ecc1514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:30.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.187+0000 7f0ecbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ecc0b0db0 0x7f0ecc0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:30.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.187+0000 7f0ecbfff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0b1770 con 0x7f0ecc0b0db0 2023-10-18T09:01:30.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.187+0000 7f0ecbfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0c73a0 con 0x7f0ecc0b1c60 2023-10-18T09:01:30.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.187+0000 7f0ecbfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc151e00 con 0x7f0eb80058d0 2023-10-18T09:01:30.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.188+0000 7f0ecb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ecc0b0db0 0x7f0ecc0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:30.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.188+0000 7f0ecb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ecc0b0db0 0x7f0ecc0b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41294/0 (socket says 172.21.15.40:41294) 2023-10-18T09:01:30.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.188+0000 7f0ecb7fe640 1 -- 172.21.15.40:0/2279336714 learned_addr learned my addr 172.21.15.40:0/2279336714 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:30.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.188+0000 7f0ecaffd640 1 --2- 172.21.15.40:0/2279336714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0eb80058d0 0x7f0ecc14df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:30.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.188+0000 7f0ec3fff640 1 --2- 172.21.15.40:0/2279336714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ecc0b1c60 0x7f0ecc1514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:30.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.188+0000 7f0ecb7fe640 1 -- 172.21.15.40:0/2279336714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0eb80058d0 msgr2=0x7f0ecc14df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:30.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.188+0000 7f0ecb7fe640 1 --2- 172.21.15.40:0/2279336714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0eb80058d0 0x7f0ecc14df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:30.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.188+0000 7f0ecb7fe640 1 -- 172.21.15.40:0/2279336714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ecc0b1c60 msgr2=0x7f0ecc1514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:30.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.188+0000 7f0ecb7fe640 1 --2- 172.21.15.40:0/2279336714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ecc0b1c60 0x7f0ecc1514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:30.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.188+0000 7f0ecb7fe640 1 -- 172.21.15.40:0/2279336714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc0c7610 con 0x7f0ecc0b0db0 2023-10-18T09:01:30.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.189+0000 7f0ecb7fe640 1 --2- 172.21.15.40:0/2279336714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ecc0b0db0 0x7f0ecc0b1230 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0afcb0 tx=0x7f0ec4010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:30.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.189+0000 7f0ec37fe640 1 -- 172.21.15.40:0/2279336714 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4003970 con 0x7f0ecc0b0db0 2023-10-18T09:01:30.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.189+0000 7f0ec37fe640 1 -- 172.21.15.40:0/2279336714 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0ec4003bc0 con 0x7f0ecc0b0db0 2023-10-18T09:01:30.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.189+0000 7f0ec37fe640 1 -- 172.21.15.40:0/2279336714 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4004030 con 0x7f0ecc0b0db0 2023-10-18T09:01:30.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.189+0000 7f0ecbfff640 1 -- 172.21.15.40:0/2279336714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc0c78f0 con 0x7f0ecc0b0db0 2023-10-18T09:01:30.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.189+0000 7f0ecbfff640 1 -- 172.21.15.40:0/2279336714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc1519f0 con 0x7f0ecc0b0db0 2023-10-18T09:01:30.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.190+0000 7f0ecbfff640 1 -- 172.21.15.40:0/2279336714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ecc066e00 con 0x7f0ecc0b0db0 2023-10-18T09:01:30.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.191+0000 7f0ec37fe640 1 -- 172.21.15.40:0/2279336714 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0ec4019030 con 0x7f0ecc0b0db0 2023-10-18T09:01:30.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.193+0000 7f0ec37fe640 1 --2- 172.21.15.40:0/2279336714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0eac06ae40 0x7f0eac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:30.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.193+0000 7f0ec37fe640 1 -- 172.21.15.40:0/2279336714 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0ec4017070 con 0x7f0ecc0b0db0 2023-10-18T09:01:30.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.193+0000 7f0ecaffd640 1 --2- 172.21.15.40:0/2279336714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0eac06ae40 0x7f0eac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:30.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.194+0000 7f0ecaffd640 1 --2- 172.21.15.40:0/2279336714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0eac06ae40 0x7f0eac06d300 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc000c00 tx=0x7f0ebc0016f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:30.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.197+0000 7f0ec37fe640 1 -- 172.21.15.40:0/2279336714 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ec4055290 con 0x7f0ecc0b0db0 2023-10-18T09:01:30.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.391+0000 7f0ecbfff640 1 -- 172.21.15.40:0/2279336714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0ecc0cdbf0 con 0x7f0ecc0b0db0 2023-10-18T09:01:30.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.398+0000 7f0ec37fe640 1 -- 172.21.15.40:0/2279336714 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0ec40588b0 con 0x7f0ecc0b0db0 2023-10-18T09:01:30.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.401+0000 7f0ecbfff640 1 -- 172.21.15.40:0/2279336714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0eac06ae40 msgr2=0x7f0eac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:30.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.401+0000 7f0ecbfff640 1 --2- 172.21.15.40:0/2279336714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0eac06ae40 0x7f0eac06d300 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc000c00 tx=0x7f0ebc0016f0 comp rx=0 tx=0).stop 2023-10-18T09:01:30.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.401+0000 7f0ecbfff640 1 -- 172.21.15.40:0/2279336714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ecc0b0db0 msgr2=0x7f0ecc0b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:30.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.401+0000 7f0ecbfff640 1 --2- 172.21.15.40:0/2279336714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ecc0b0db0 0x7f0ecc0b1230 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc0afcb0 tx=0x7f0ec4010f40 comp rx=0 tx=0).stop 2023-10-18T09:01:30.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.402+0000 7f0ecbfff640 1 -- 172.21.15.40:0/2279336714 shutdown_connections 2023-10-18T09:01:30.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.402+0000 7f0ecbfff640 1 --2- 172.21.15.40:0/2279336714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0eac06ae40 0x7f0eac06d300 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:30.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.402+0000 7f0ecbfff640 1 --2- 172.21.15.40:0/2279336714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ecc0b0db0 0x7f0ecc0b1230 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:30.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.402+0000 7f0ecbfff640 1 --2- 172.21.15.40:0/2279336714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ecc0b1c60 0x7f0ecc1514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:30.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.402+0000 7f0ecbfff640 1 --2- 172.21.15.40:0/2279336714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0eb80058d0 0x7f0ecc14df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:30.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.402+0000 7f0ecbfff640 1 -- 172.21.15.40:0/2279336714 >> 172.21.15.40:0/2279336714 conn(0x7f0ecc06ca10 msgr2=0x7f0ecc14e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:30.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.402+0000 7f0ecbfff640 1 -- 172.21.15.40:0/2279336714 shutdown_connections 2023-10-18T09:01:30.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:30.402+0000 7f0ecbfff640 1 -- 172.21.15.40:0/2279336714 wait complete. 2023-10-18T09:01:30.422 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T09:01:30.423 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:01:30.543 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:01:30.671 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T09:01:30.672 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:01:30.672 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:01:30.739 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:01:30.740 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:01:30.740 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:01:30.800 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:01:30.801 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:01:30.802 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:01:30.864 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:01:30.864 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:01:30.864 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:01:30.924 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:01:30.925 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T09:01:30.925 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:01:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.166+0000 7ffa0bfff640 1 -- 172.21.15.40:0/4204845541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f8003970 msgr2=0x7ff9f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.166+0000 7ffa0bfff640 1 --2- 172.21.15.40:0/4204845541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f8003970 0x7ff9f8003d90 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc005fd0 tx=0x7ff9f80057f0 comp rx=0 tx=0).stop 2023-10-18T09:01:31.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.166+0000 7ffa0b7fe640 1 -- 172.21.15.40:0/4204845541 reap_dead start 2023-10-18T09:01:31.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.166+0000 7ffa0bfff640 1 -- 172.21.15.40:0/4204845541 shutdown_connections 2023-10-18T09:01:31.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.166+0000 7ffa0bfff640 1 -- 172.21.15.40:0/4204845541 >> 172.21.15.40:0/4204845541 conn(0x7ffa0c069720 msgr2=0x7ffa0c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:31.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.167+0000 7ffa0bfff640 1 -- 172.21.15.40:0/4204845541 shutdown_connections 2023-10-18T09:01:31.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.167+0000 7ffa0bfff640 1 -- 172.21.15.40:0/4204845541 wait complete. 2023-10-18T09:01:31.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.168+0000 7ffa0bfff640 1 Processor -- start 2023-10-18T09:01:31.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.168+0000 7ffa0bfff640 1 -- start start 2023-10-18T09:01:31.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.169+0000 7ffa0bfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f8003970 0x7ffa0c145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:31.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.169+0000 7ffa0bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa0c153610 0x7ffa0c14b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:31.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.169+0000 7ffa0bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c153a90 0x7ffa0c14bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:31.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.169+0000 7ffa0bfff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c14e990 con 0x7ffa0c153a90 2023-10-18T09:01:31.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.169+0000 7ffa0bfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c14ec70 con 0x7ffa0c153610 2023-10-18T09:01:31.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.169+0000 7ffa0bfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c152b30 con 0x7ff9f8003970 2023-10-18T09:01:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.170+0000 7ffa0b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c153a90 0x7ffa0c14bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.170+0000 7ffa0b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c153a90 0x7ffa0c14bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41312/0 (socket says 172.21.15.40:41312) 2023-10-18T09:01:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.170+0000 7ffa0b7fe640 1 -- 172.21.15.40:0/3265974375 learned_addr learned my addr 172.21.15.40:0/3265974375 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.170+0000 7ffa027fc640 1 --2- 172.21.15.40:0/3265974375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa0c153610 0x7ffa0c14b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.170+0000 7ffa0affd640 1 --2- 172.21.15.40:0/3265974375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f8003970 0x7ffa0c145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:31.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.170+0000 7ffa0b7fe640 1 -- 172.21.15.40:0/3265974375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f8003970 msgr2=0x7ffa0c145cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.170+0000 7ffa0b7fe640 1 --2- 172.21.15.40:0/3265974375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f8003970 0x7ffa0c145cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.170+0000 7ffa0b7fe640 1 -- 172.21.15.40:0/3265974375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa0c153610 msgr2=0x7ffa0c14b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.170+0000 7ffa0b7fe640 1 --2- 172.21.15.40:0/3265974375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa0c153610 0x7ffa0c14b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.170+0000 7ffa0b7fe640 1 -- 172.21.15.40:0/3265974375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c152e10 con 0x7ffa0c153a90 2023-10-18T09:01:31.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.171+0000 7ffa0b7fe640 1 --2- 172.21.15.40:0/3265974375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c153a90 0x7ffa0c14bd20 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0cad50 tx=0x7ffa0400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.171+0000 7ffa03fff640 1 -- 172.21.15.40:0/3265974375 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400ac00 con 0x7ffa0c153a90 2023-10-18T09:01:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.171+0000 7ffa03fff640 1 -- 172.21.15.40:0/3265974375 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffa0400ae50 con 0x7ffa0c153a90 2023-10-18T09:01:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.171+0000 7ffa03fff640 1 -- 172.21.15.40:0/3265974375 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa040072d0 con 0x7ffa0c153a90 2023-10-18T09:01:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.171+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3265974375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0c1530f0 con 0x7ffa0c153a90 2023-10-18T09:01:31.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.171+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3265974375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c14dd10 con 0x7ffa0c153a90 2023-10-18T09:01:31.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.172+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3265974375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0c073250 con 0x7ffa0c153a90 2023-10-18T09:01:31.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.173+0000 7ffa03fff640 1 -- 172.21.15.40:0/3265974375 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffa04007430 con 0x7ffa0c153a90 2023-10-18T09:01:31.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.174+0000 7ffa03fff640 1 --2- 172.21.15.40:0/3265974375 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9ec06ae70 0x7ff9ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:31.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.174+0000 7ffa03fff640 1 -- 172.21.15.40:0/3265974375 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffa040894d0 con 0x7ffa0c153a90 2023-10-18T09:01:31.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.175+0000 7ffa0affd640 1 --2- 172.21.15.40:0/3265974375 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9ec06ae70 0x7ff9ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:31.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.176+0000 7ffa0affd640 1 --2- 172.21.15.40:0/3265974375 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9ec06ae70 0x7ff9ec06d330 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80057a0 tx=0x7ff9fc00e040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:31.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.179+0000 7ffa03fff640 1 -- 172.21.15.40:0/3265974375 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa040547e0 con 0x7ffa0c153a90 2023-10-18T09:01:31.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.412+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3265974375 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa0c066e00 con 0x7ffa0c153a90 2023-10-18T09:01:31.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.413+0000 7ffa03fff640 1 -- 172.21.15.40:0/3265974375 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3705 (secure 0 0 0) 0x7ffa04017090 con 0x7ffa0c153a90 2023-10-18T09:01:31.416 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:31.416 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":43,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1341351808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1341351808},{"type":"v1","addr":"172.21.15.144:6836","nonce":1341351808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1187670073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1187670073},{"type":"v1","addr":"172.21.15.40:6833","nonce":1187670073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5626,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/1615448513","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":1615448513},{"type":"v1","addr":"172.21.15.144:6837","nonce":1615448513}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6915,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3635490395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3635490395},{"type":"v1","addr":"172.21.15.40:6835","nonce":3635490395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[]} 2023-10-18T09:01:31.416 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 43 2023-10-18T09:01:31.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.416+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3265974375 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9ec06ae70 msgr2=0x7ff9ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:31.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.416+0000 7ffa0bfff640 1 --2- 172.21.15.40:0/3265974375 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9ec06ae70 0x7ff9ec06d330 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80057a0 tx=0x7ff9fc00e040 comp rx=0 tx=0).stop 2023-10-18T09:01:31.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.416+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3265974375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c153a90 msgr2=0x7ffa0c14bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:31.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.416+0000 7ffa0bfff640 1 --2- 172.21.15.40:0/3265974375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c153a90 0x7ffa0c14bd20 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0cad50 tx=0x7ffa0400cf80 comp rx=0 tx=0).stop 2023-10-18T09:01:31.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.416+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3265974375 shutdown_connections 2023-10-18T09:01:31.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.416+0000 7ffa0bfff640 1 --2- 172.21.15.40:0/3265974375 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9ec06ae70 0x7ff9ec06d330 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:31.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.416+0000 7ffa0bfff640 1 --2- 172.21.15.40:0/3265974375 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa0c153a90 0x7ffa0c14bd20 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:31.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.416+0000 7ffa0bfff640 1 --2- 172.21.15.40:0/3265974375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa0c153610 0x7ffa0c14b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:31.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.416+0000 7ffa0bfff640 1 --2- 172.21.15.40:0/3265974375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f8003970 0x7ffa0c145cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:31.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.416+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3265974375 >> 172.21.15.40:0/3265974375 conn(0x7ffa0c069720 msgr2=0x7ffa0c156ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:31.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.417+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3265974375 shutdown_connections 2023-10-18T09:01:31.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.417+0000 7ffa0bfff640 1 -- 172.21.15.40:0/3265974375 wait complete. 2023-10-18T09:01:31.440 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T09:01:31.441 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T09:01:31.441 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T09:01:31.441 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:01:31.442 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T09:01:31.442 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T09:01:31.442 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T09:01:31.442 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:01:31.443 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T09:01:31.443 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T09:01:31.443 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T09:01:31.444 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:01:31.444 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T09:01:31.444 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T09:01:31.444 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T09:01:31.445 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:01:31.457 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T09:01:31.457 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T09:01:31.459 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T09:01:31.459 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T09:01:31.470 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T09:01:31.471 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T09:01:31.472 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T09:01:31.472 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T09:01:31.509 INFO:tasks.ceph.mds.a:Started 2023-10-18T09:01:31.510 DEBUG:teuthology.parallel:result is None 2023-10-18T09:01:31.517 INFO:tasks.ceph.mds.c:Started 2023-10-18T09:01:31.517 DEBUG:teuthology.parallel:result is None 2023-10-18T09:01:31.520 INFO:tasks.ceph.mds.b:Started 2023-10-18T09:01:31.520 DEBUG:teuthology.parallel:result is None 2023-10-18T09:01:31.527 INFO:tasks.ceph.mds.d:Started 2023-10-18T09:01:31.527 DEBUG:teuthology.parallel:result is None 2023-10-18T09:01:31.528 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T09:01:31.671 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-10-18T09:01:31.680 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T09:01:31.692 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-18T09:01:31.692 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-18T09:01:31.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.727+0000 7f239f59e640 1 -- 172.21.15.40:0/4172031464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23a00d0020 msgr2=0x7f23a00ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:31.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.727+0000 7f239f59e640 1 --2- 172.21.15.40:0/4172031464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23a00d0020 0x7f23a00ae960 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f23a00b0f20 tx=0x7f239802c140 comp rx=0 tx=0).stop 2023-10-18T09:01:31.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.727+0000 7f239f59e640 1 -- 172.21.15.40:0/4172031464 shutdown_connections 2023-10-18T09:01:31.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.727+0000 7f239f59e640 1 --2- 172.21.15.40:0/4172031464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23a00d0020 0x7f23a00ae960 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:31.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.727+0000 7f239f59e640 1 --2- 172.21.15.40:0/4172031464 >> v2:172.21.15.144:3300/0 conn(0x7f23a015b640 0x7f23a00d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:31.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.727+0000 7f239f59e640 1 -- 172.21.15.40:0/4172031464 >> 172.21.15.40:0/4172031464 conn(0x7f23a008ad00 msgr2=0x7f23a009a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:31.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.727+0000 7f239f59e640 1 -- 172.21.15.40:0/4172031464 shutdown_connections 2023-10-18T09:01:31.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.727+0000 7f239f59e640 1 -- 172.21.15.40:0/4172031464 wait complete. 2023-10-18T09:01:31.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.729+0000 7f239f59e640 1 Processor -- start 2023-10-18T09:01:31.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.729+0000 7f239f59e640 1 -- start start 2023-10-18T09:01:31.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.730+0000 7f239f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23a00b01e0 0x7f23a009ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:31.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.730+0000 7f239f59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23a00d0020 0x7f23a006a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:31.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.730+0000 7f239f59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23a015b640 0x7f23a0158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:31.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.730+0000 7f239f59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a01f5bb0 con 0x7f23a00b01e0 2023-10-18T09:01:31.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.730+0000 7f239f59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a009e210 con 0x7f23a015b640 2023-10-18T09:01:31.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.730+0000 7f239f59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a009e4f0 con 0x7f23a00d0020 2023-10-18T09:01:31.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.731+0000 7f239dd9b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23a00d0020 0x7f23a006a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:31.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.731+0000 7f239ed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23a015b640 0x7f23a0158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:31.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.731+0000 7f239dd9b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23a00d0020 0x7f23a006a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.40:38246/0 (socket says 172.21.15.40:38246) 2023-10-18T09:01:31.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.731+0000 7f239e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23a00b01e0 0x7f23a009ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:31.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.731+0000 7f239ed9d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23a015b640 0x7f23a0158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.40:41140/0 (socket says 172.21.15.40:41140) 2023-10-18T09:01:31.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.731+0000 7f239dd9b640 1 -- 172.21.15.40:0/3309519013 learned_addr learned my addr 172.21.15.40:0/3309519013 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:31.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.731+0000 7f239e59c640 1 -- 172.21.15.40:0/3309519013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23a00d0020 msgr2=0x7f23a006a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:31.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.731+0000 7f239e59c640 1 --2- 172.21.15.40:0/3309519013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23a00d0020 0x7f23a006a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:31.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.731+0000 7f239e59c640 1 -- 172.21.15.40:0/3309519013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23a015b640 msgr2=0x7f23a0158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:31.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.731+0000 7f239e59c640 1 --2- 172.21.15.40:0/3309519013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23a015b640 0x7f23a0158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:31.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.731+0000 7f239e59c640 1 -- 172.21.15.40:0/3309519013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2398009900 con 0x7f23a00b01e0 2023-10-18T09:01:31.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.732+0000 7f239e59c640 1 --2- 172.21.15.40:0/3309519013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23a00b01e0 0x7f23a009ecf0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f239000d680 tx=0x7f239000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:31.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.732+0000 7f238effd640 1 -- 172.21.15.40:0/3309519013 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390005220 con 0x7f23a00b01e0 2023-10-18T09:01:31.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.732+0000 7f238effd640 1 -- 172.21.15.40:0/3309519013 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2390005470 con 0x7f23a00b01e0 2023-10-18T09:01:31.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.732+0000 7f238effd640 1 -- 172.21.15.40:0/3309519013 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390009a80 con 0x7f23a00b01e0 2023-10-18T09:01:31.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.732+0000 7f239f59e640 1 -- 172.21.15.40:0/3309519013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a009e7d0 con 0x7f23a00b01e0 2023-10-18T09:01:31.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.732+0000 7f239f59e640 1 -- 172.21.15.40:0/3309519013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a006ac20 con 0x7f23a00b01e0 2023-10-18T09:01:31.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.733+0000 7f239f59e640 1 -- 172.21.15.40:0/3309519013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a006ae70 con 0x7f23a00b01e0 2023-10-18T09:01:31.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.734+0000 7f238effd640 1 -- 172.21.15.40:0/3309519013 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2390009be0 con 0x7f23a00b01e0 2023-10-18T09:01:31.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.735+0000 7f238effd640 1 --2- 172.21.15.40:0/3309519013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f238006ae40 0x7f238006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:31.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.735+0000 7f238effd640 1 -- 172.21.15.40:0/3309519013 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2390089360 con 0x7f23a00b01e0 2023-10-18T09:01:31.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.736+0000 7f239dd9b640 1 --2- 172.21.15.40:0/3309519013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f238006ae40 0x7f238006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:31.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.737+0000 7f239dd9b640 1 --2- 172.21.15.40:0/3309519013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f238006ae40 0x7f238006d300 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f23a00cd990 tx=0x7f2394007ef0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:31.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.739+0000 7f238effd640 1 -- 172.21.15.40:0/3309519013 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f23900547b0 con 0x7f23a00b01e0 2023-10-18T09:01:31.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:31.947+0000 7f239f59e640 1 -- 172.21.15.40:0/3309519013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f23a0071da0 con 0x7f23a00b01e0 2023-10-18T09:01:33.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.026+0000 7f238effd640 1 -- 172.21.15.40:0/3309519013 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v103) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2390020080 con 0x7f23a00b01e0 2023-10-18T09:01:33.029 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2023-10-18T09:01:33.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.030+0000 7f239f59e640 1 -- 172.21.15.40:0/3309519013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f238006ae40 msgr2=0x7f238006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:33.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.030+0000 7f239f59e640 1 --2- 172.21.15.40:0/3309519013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f238006ae40 0x7f238006d300 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f23a00cd990 tx=0x7f2394007ef0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.030+0000 7f239f59e640 1 -- 172.21.15.40:0/3309519013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23a00b01e0 msgr2=0x7f23a009ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:33.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.030+0000 7f239f59e640 1 --2- 172.21.15.40:0/3309519013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23a00b01e0 0x7f23a009ecf0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f239000d680 tx=0x7f239000db40 comp rx=0 tx=0).stop 2023-10-18T09:01:33.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.030+0000 7f239f59e640 1 -- 172.21.15.40:0/3309519013 shutdown_connections 2023-10-18T09:01:33.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.031+0000 7f239f59e640 1 --2- 172.21.15.40:0/3309519013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f238006ae40 0x7f238006d300 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.031+0000 7f239f59e640 1 --2- 172.21.15.40:0/3309519013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23a015b640 0x7f23a0158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.031+0000 7f239f59e640 1 --2- 172.21.15.40:0/3309519013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23a00d0020 0x7f23a006a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.031+0000 7f239f59e640 1 --2- 172.21.15.40:0/3309519013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23a00b01e0 0x7f23a009ecf0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.031+0000 7f239f59e640 1 -- 172.21.15.40:0/3309519013 >> 172.21.15.40:0/3309519013 conn(0x7f23a008ad00 msgr2=0x7f23a0072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:33.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.031+0000 7f239f59e640 1 -- 172.21.15.40:0/3309519013 shutdown_connections 2023-10-18T09:01:33.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.031+0000 7f239f59e640 1 -- 172.21.15.40:0/3309519013 wait complete. 2023-10-18T09:01:33.056 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T09:01:33.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.289+0000 7f1b9e9b8640 1 -- 172.21.15.40:0/3970524291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b980c9f00 msgr2=0x7f1b98157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:33.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.289+0000 7f1b9e9b8640 1 --2- 172.21.15.40:0/3970524291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b980c9f00 0x7f1b98157d40 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1b9806aeb0 tx=0x7f1b9002c260 comp rx=0 tx=0).stop 2023-10-18T09:01:33.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.290+0000 7f1b9e9b8640 1 -- 172.21.15.40:0/3970524291 shutdown_connections 2023-10-18T09:01:33.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.290+0000 7f1b9e9b8640 1 --2- 172.21.15.40:0/3970524291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b980c9f00 0x7f1b98157d40 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.290+0000 7f1b9e9b8640 1 --2- 172.21.15.40:0/3970524291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b980b0630 0x7f1b980c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.290+0000 7f1b9e9b8640 1 --2- 172.21.15.40:0/3970524291 >> v2:172.21.15.144:3300/0 conn(0x7f1b9806d6f0 0x7f1b980b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.290+0000 7f1b9e9b8640 1 -- 172.21.15.40:0/3970524291 >> 172.21.15.40:0/3970524291 conn(0x7f1b98069720 msgr2=0x7f1b98069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:33.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.290+0000 7f1b9e9b8640 1 -- 172.21.15.40:0/3970524291 shutdown_connections 2023-10-18T09:01:33.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.290+0000 7f1b9e9b8640 1 -- 172.21.15.40:0/3970524291 wait complete. 2023-10-18T09:01:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.292+0000 7f1b9e9b8640 1 Processor -- start 2023-10-18T09:01:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.292+0000 7f1b9e9b8640 1 -- start start 2023-10-18T09:01:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.293+0000 7f1b9e9b8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b9806d6f0 0x7f1b980c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.293+0000 7f1b9e9b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b980b0630 0x7f1b98151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.293+0000 7f1b9e9b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b980c9f00 0x7f1b980ad340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.293+0000 7f1b9e9b8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b98153080 con 0x7f1b980c9f00 2023-10-18T09:01:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.293+0000 7f1b9e9b8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b98151f60 con 0x7f1b9806d6f0 2023-10-18T09:01:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.293+0000 7f1b9e9b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b981519a0 con 0x7f1b980b0630 2023-10-18T09:01:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.293+0000 7f1b9e1b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b980c9f00 0x7f1b980ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.293+0000 7f1b9e1b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b980c9f00 0x7f1b980ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41380/0 (socket says 172.21.15.40:41380) 2023-10-18T09:01:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.293+0000 7f1b9e1b7640 1 -- 172.21.15.40:0/1825749881 learned_addr learned my addr 172.21.15.40:0/1825749881 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.293+0000 7f1b9d9b6640 1 --2- 172.21.15.40:0/1825749881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b9806d6f0 0x7f1b980c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:33.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.293+0000 7f1b9d1b5640 1 --2- 172.21.15.40:0/1825749881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b980b0630 0x7f1b98151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:33.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.294+0000 7f1b9e1b7640 1 -- 172.21.15.40:0/1825749881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b980b0630 msgr2=0x7f1b98151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:33.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.294+0000 7f1b9e1b7640 1 --2- 172.21.15.40:0/1825749881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b980b0630 0x7f1b98151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.294+0000 7f1b9e1b7640 1 -- 172.21.15.40:0/1825749881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b9806d6f0 msgr2=0x7f1b980c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:33.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.294+0000 7f1b9e1b7640 1 --2- 172.21.15.40:0/1825749881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b9806d6f0 0x7f1b980c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.294+0000 7f1b9e1b7640 1 -- 172.21.15.40:0/1825749881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b90009900 con 0x7f1b980c9f00 2023-10-18T09:01:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.294+0000 7f1b9e1b7640 1 --2- 172.21.15.40:0/1825749881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b980c9f00 0x7f1b980ad340 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009d60 tx=0x7f1b900093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.294+0000 7f1b8e7fc640 1 -- 172.21.15.40:0/1825749881 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b90004240 con 0x7f1b980c9f00 2023-10-18T09:01:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.294+0000 7f1b8e7fc640 1 -- 172.21.15.40:0/1825749881 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1b90004490 con 0x7f1b980c9f00 2023-10-18T09:01:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.294+0000 7f1b8e7fc640 1 -- 172.21.15.40:0/1825749881 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b900048a0 con 0x7f1b980c9f00 2023-10-18T09:01:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.294+0000 7f1b9e9b8640 1 -- 172.21.15.40:0/1825749881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b98151c80 con 0x7f1b980c9f00 2023-10-18T09:01:33.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.295+0000 7f1b9e9b8640 1 -- 172.21.15.40:0/1825749881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b980c8cb0 con 0x7f1b980c9f00 2023-10-18T09:01:33.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.296+0000 7f1b9e9b8640 1 -- 172.21.15.40:0/1825749881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b98066e00 con 0x7f1b980c9f00 2023-10-18T09:01:33.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.297+0000 7f1b8e7fc640 1 -- 172.21.15.40:0/1825749881 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1b9000a040 con 0x7f1b980c9f00 2023-10-18T09:01:33.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.297+0000 7f1b8e7fc640 1 --2- 172.21.15.40:0/1825749881 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b7c06ae70 0x7f1b7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:33.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.298+0000 7f1b8e7fc640 1 -- 172.21.15.40:0/1825749881 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1b900ac4b0 con 0x7f1b980c9f00 2023-10-18T09:01:33.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.298+0000 7f1b9d9b6640 1 --2- 172.21.15.40:0/1825749881 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b7c06ae70 0x7f1b7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:33.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.299+0000 7f1b9d9b6640 1 --2- 172.21.15.40:0/1825749881 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b7c06ae70 0x7f1b7c06d330 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1b94005fd0 tx=0x7f1b94005e40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:33.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.302+0000 7f1b8e7fc640 1 -- 172.21.15.40:0/1825749881 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b90077810 con 0x7f1b980c9f00 2023-10-18T09:01:33.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.564+0000 7f1b9e9b8640 1 -- 172.21.15.40:0/1825749881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1b980caf90 con 0x7f1b980c9f00 2023-10-18T09:01:33.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.565+0000 7f1b8e7fc640 1 -- 172.21.15.40:0/1825749881 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1b90044090 con 0x7f1b980c9f00 2023-10-18T09:01:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:33.568 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:01:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2023-10-18T09:01:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2023-10-18T09:01:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lYsb5CxAAGButZ6+Usc62kJ0ksxZrYw==", 2023-10-18T09:01:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:01:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:01:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:01:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:01:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2023-10-18T09:01:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lAqlhMBAApAhxCYB9d9bz1GibADDRSQ==", 2023-10-18T09:01:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:01:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:01:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:01:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:01:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2023-10-18T09:01:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lPVh+HhAAzucSx+ksPDBkqJGa8+pyJw==", 2023-10-18T09:01:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:01:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:01:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:01:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:01:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2023-10-18T09:01:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lP7QCBhAAog7qDCgGVv0T7WKsRkATkA==", 2023-10-18T09:01:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:01:33.576 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:01:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:01:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:01:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2023-10-18T09:01:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lVx+hAhAA6ZpIyZKhbqy8tV84piGILA==", 2023-10-18T09:01:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:01:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:01:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:01:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2023-10-18T09:01:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lk9LwExAA7zFzQOn2hcvnck2cOZZ7AA==", 2023-10-18T09:01:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:01:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:01:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:01:33.581 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.581 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2023-10-18T09:01:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lK930JRAA1GyKRkIpXPHHGqY8owYWRQ==", 2023-10-18T09:01:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:01:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:01:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:01:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2023-10-18T09:01:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9liBWyNRAAMHBBdG5nbrGQupqvUtuKMA==", 2023-10-18T09:01:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:01:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:01:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:01:33.589 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.589 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.589 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.589 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2023-10-18T09:01:33.589 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9loDoOAhAAjHZub4wnB7dCr86rxAtI5g==", 2023-10-18T09:01:33.589 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.590 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:01:33.590 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:01:33.590 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:01:33.590 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2023-10-18T09:01:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lhqNfERAAw410g3DwSXrK23YkxVBFVw==", 2023-10-18T09:01:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:01:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:01:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:01:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2023-10-18T09:01:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lRRScIRAAg0pmfxCQBCPYodj1Lko+Lw==", 2023-10-18T09:01:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:01:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:01:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:01:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2023-10-18T09:01:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9ld1dXMhAAyovlrKOhVuynEiuPTgFcjQ==", 2023-10-18T09:01:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:01:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:01:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:01:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2023-10-18T09:01:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w==", 2023-10-18T09:01:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:01:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2023-10-18T09:01:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T09:01:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.598 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2023-10-18T09:01:33.598 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lAB3oKhAAD43WFdC23WGF6WUi4JKsvg==", 2023-10-18T09:01:33.598 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.598 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:01:33.599 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:01:33.599 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:01:33.599 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:01:33.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2023-10-18T09:01:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lu57fHxAAdzsVU2KKsWRkaARRS8TGjw==", 2023-10-18T09:01:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:01:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:01:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:01:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:01:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2023-10-18T09:01:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lygWsNRAA1OziXxk7hBtTw2Aq8OlSKQ==", 2023-10-18T09:01:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:01:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:01:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:01:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:01:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2023-10-18T09:01:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBMnS9lnrUjNhAAHP2eezRHF4/Jgox7iPwy7g==", 2023-10-18T09:01:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:01:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:01:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:01:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:01:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2023-10-18T09:01:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9lCf/xOhAAJHVoKIswWUcVGKQpiaJ12A==", 2023-10-18T09:01:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:01:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T09:01:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:01:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2023-10-18T09:01:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9loxRXLhAAQK1n1/4PB35umHxOKQzz7Q==", 2023-10-18T09:01:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:01:33.608 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:01:33.608 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T09:01:33.608 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:01:33.608 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.608 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:33.609 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:33.609 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:01:33.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.568+0000 7f1b9e9b8640 1 -- 172.21.15.40:0/1825749881 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b7c06ae70 msgr2=0x7f1b7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:33.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.568+0000 7f1b9e9b8640 1 --2- 172.21.15.40:0/1825749881 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b7c06ae70 0x7f1b7c06d330 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1b94005fd0 tx=0x7f1b94005e40 comp rx=0 tx=0).stop 2023-10-18T09:01:33.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.568+0000 7f1b9e9b8640 1 -- 172.21.15.40:0/1825749881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b980c9f00 msgr2=0x7f1b980ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:33.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.568+0000 7f1b9e9b8640 1 --2- 172.21.15.40:0/1825749881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b980c9f00 0x7f1b980ad340 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1b90009d60 tx=0x7f1b900093f0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.568+0000 7f1b9e9b8640 1 -- 172.21.15.40:0/1825749881 shutdown_connections 2023-10-18T09:01:33.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.568+0000 7f1b9e9b8640 1 --2- 172.21.15.40:0/1825749881 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b7c06ae70 0x7f1b7c06d330 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.568+0000 7f1b9e9b8640 1 --2- 172.21.15.40:0/1825749881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b980c9f00 0x7f1b980ad340 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.568+0000 7f1b9e9b8640 1 --2- 172.21.15.40:0/1825749881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b980b0630 0x7f1b98151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.568+0000 7f1b9e9b8640 1 --2- 172.21.15.40:0/1825749881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b9806d6f0 0x7f1b980c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.568+0000 7f1b9e9b8640 1 -- 172.21.15.40:0/1825749881 >> 172.21.15.40:0/1825749881 conn(0x7f1b98069720 msgr2=0x7f1b980abee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:33.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.569+0000 7f1b9e9b8640 1 -- 172.21.15.40:0/1825749881 shutdown_connections 2023-10-18T09:01:33.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.569+0000 7f1b9e9b8640 1 -- 172.21.15.40:0/1825749881 wait complete. 2023-10-18T09:01:33.614 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:01:33.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.844+0000 7fa3348d6640 1 -- 172.21.15.40:0/1562701880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa318004d00 msgr2=0x7fa318005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:33.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.844+0000 7fa3348d6640 1 --2- 172.21.15.40:0/1562701880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa318004d00 0x7fa318005180 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa32800c670 tx=0x7fa3180057f0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.844+0000 7fa32f59e640 1 -- 172.21.15.40:0/1562701880 reap_dead start 2023-10-18T09:01:33.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.845+0000 7fa3348d6640 1 -- 172.21.15.40:0/1562701880 shutdown_connections 2023-10-18T09:01:33.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.845+0000 7fa3348d6640 1 -- 172.21.15.40:0/1562701880 >> 172.21.15.40:0/1562701880 conn(0x7fa330069720 msgr2=0x7fa330069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:33.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.845+0000 7fa3348d6640 1 -- 172.21.15.40:0/1562701880 shutdown_connections 2023-10-18T09:01:33.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.845+0000 7fa3348d6640 1 -- 172.21.15.40:0/1562701880 wait complete. 2023-10-18T09:01:33.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.846+0000 7fa3348d6640 1 Processor -- start 2023-10-18T09:01:33.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.847+0000 7fa3348d6640 1 -- start start 2023-10-18T09:01:33.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.847+0000 7fa3348d6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa318004d00 0x7fa3300abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:33.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.847+0000 7fa3348d6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa330071210 0x7fa33013f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:33.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.848+0000 7fa3348d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa330158db0 0x7fa33013f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:33.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.848+0000 7fa3348d6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3301f1b40 con 0x7fa330158db0 2023-10-18T09:01:33.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.848+0000 7fa3348d6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3301f1d30 con 0x7fa318004d00 2023-10-18T09:01:33.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.848+0000 7fa3348d6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3301f2010 con 0x7fa330071210 2023-10-18T09:01:33.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.848+0000 7fa32f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa330158db0 0x7fa33013f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:33.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.848+0000 7fa32f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa330158db0 0x7fa33013f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41398/0 (socket says 172.21.15.40:41398) 2023-10-18T09:01:33.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.848+0000 7fa32f59e640 1 -- 172.21.15.40:0/45040271 learned_addr learned my addr 172.21.15.40:0/45040271 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:33.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.848+0000 7fa32ed9d640 1 --2- 172.21.15.40:0/45040271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa318004d00 0x7fa3300abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:33.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.848+0000 7fa32e59c640 1 --2- 172.21.15.40:0/45040271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa330071210 0x7fa33013f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:33.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.848+0000 7fa32f59e640 1 -- 172.21.15.40:0/45040271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa330071210 msgr2=0x7fa33013f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:33.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.848+0000 7fa32f59e640 1 --2- 172.21.15.40:0/45040271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa330071210 0x7fa33013f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.848+0000 7fa32f59e640 1 -- 172.21.15.40:0/45040271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa318004d00 msgr2=0x7fa3300abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:33.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.848+0000 7fa32f59e640 1 --2- 172.21.15.40:0/45040271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa318004d00 0x7fa3300abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:33.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.848+0000 7fa32f59e640 1 -- 172.21.15.40:0/45040271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3301f22f0 con 0x7fa330158db0 2023-10-18T09:01:33.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.849+0000 7fa32f59e640 1 --2- 172.21.15.40:0/45040271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa330158db0 0x7fa33013f540 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa32800c670 tx=0x7fa328001290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:33.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.849+0000 7fa3177fe640 1 -- 172.21.15.40:0/45040271 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328001870 con 0x7fa330158db0 2023-10-18T09:01:33.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.849+0000 7fa3177fe640 1 -- 172.21.15.40:0/45040271 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa3280019d0 con 0x7fa330158db0 2023-10-18T09:01:33.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.849+0000 7fa3348d6640 1 -- 172.21.15.40:0/45040271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3301f25d0 con 0x7fa330158db0 2023-10-18T09:01:33.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.849+0000 7fa3177fe640 1 -- 172.21.15.40:0/45040271 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328003bb0 con 0x7fa330158db0 2023-10-18T09:01:33.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.849+0000 7fa3348d6640 1 -- 172.21.15.40:0/45040271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3300ac430 con 0x7fa330158db0 2023-10-18T09:01:33.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.850+0000 7fa3348d6640 1 -- 172.21.15.40:0/45040271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa330066e00 con 0x7fa330158db0 2023-10-18T09:01:33.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.851+0000 7fa3177fe640 1 -- 172.21.15.40:0/45040271 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa328031030 con 0x7fa330158db0 2023-10-18T09:01:33.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.852+0000 7fa3177fe640 1 --2- 172.21.15.40:0/45040271 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa31c06ae70 0x7fa31c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:33.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.852+0000 7fa3177fe640 1 -- 172.21.15.40:0/45040271 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa3280aa2b0 con 0x7fa330158db0 2023-10-18T09:01:33.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.853+0000 7fa32ed9d640 1 --2- 172.21.15.40:0/45040271 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa31c06ae70 0x7fa31c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:33.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.854+0000 7fa32ed9d640 1 --2- 172.21.15.40:0/45040271 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa31c06ae70 0x7fa31c06d330 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa320009a20 tx=0x7fa320005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:33.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:33.857+0000 7fa3177fe640 1 -- 172.21.15.40:0/45040271 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa3280790a0 con 0x7fa330158db0 2023-10-18T09:01:34.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.047+0000 7fa3348d6640 1 -- 172.21.15.40:0/45040271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3300b0860 con 0x7fa330158db0 2023-10-18T09:01:34.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.049+0000 7fa3177fe640 1 -- 172.21.15.40:0/45040271 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+12871 (secure 0 0 0) 0x7fa328043090 con 0x7fa330158db0 2023-10-18T09:01:34.052 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:34.052 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":103,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:01:33.023474+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:01:34.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.052+0000 7fa3348d6640 1 -- 172.21.15.40:0/45040271 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa31c06ae70 msgr2=0x7fa31c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:34.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.052+0000 7fa3348d6640 1 --2- 172.21.15.40:0/45040271 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa31c06ae70 0x7fa31c06d330 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa320009a20 tx=0x7fa320005f90 comp rx=0 tx=0).stop 2023-10-18T09:01:34.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.052+0000 7fa3348d6640 1 -- 172.21.15.40:0/45040271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa330158db0 msgr2=0x7fa33013f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:34.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.052+0000 7fa3348d6640 1 --2- 172.21.15.40:0/45040271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa330158db0 0x7fa33013f540 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa32800c670 tx=0x7fa328001290 comp rx=0 tx=0).stop 2023-10-18T09:01:34.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.052+0000 7fa3348d6640 1 -- 172.21.15.40:0/45040271 shutdown_connections 2023-10-18T09:01:34.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.052+0000 7fa3348d6640 1 --2- 172.21.15.40:0/45040271 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa31c06ae70 0x7fa31c06d330 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.052+0000 7fa3348d6640 1 --2- 172.21.15.40:0/45040271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa330158db0 0x7fa33013f540 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.052+0000 7fa3348d6640 1 --2- 172.21.15.40:0/45040271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa330071210 0x7fa33013f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.052+0000 7fa3348d6640 1 --2- 172.21.15.40:0/45040271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa318004d00 0x7fa3300abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.052+0000 7fa3348d6640 1 -- 172.21.15.40:0/45040271 >> 172.21.15.40:0/45040271 conn(0x7fa330069720 msgr2=0x7fa33013e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:34.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.052+0000 7fa3348d6640 1 -- 172.21.15.40:0/45040271 shutdown_connections 2023-10-18T09:01:34.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.052+0000 7fa3348d6640 1 -- 172.21.15.40:0/45040271 wait complete. 2023-10-18T09:01:34.072 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:01:34.072 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:01:34.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.286+0000 7f6b74a83640 1 -- 172.21.15.40:0/3579226842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b700b2c20 msgr2=0x7f6b700ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:34.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.286+0000 7f6b74a83640 1 --2- 172.21.15.40:0/3579226842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b700b2c20 0x7f6b700ca5f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6b7006d8f0 tx=0x7f6b6002bcf0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.286+0000 7f6b74a83640 1 -- 172.21.15.40:0/3579226842 shutdown_connections 2023-10-18T09:01:34.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.286+0000 7f6b74a83640 1 --2- 172.21.15.40:0/3579226842 >> v2:172.21.15.144:3300/0 conn(0x7f6b700aec00 0x7f6b700af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.286+0000 7f6b74a83640 1 --2- 172.21.15.40:0/3579226842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b700b2c20 0x7f6b700ca5f0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.286+0000 7f6b74a83640 1 -- 172.21.15.40:0/3579226842 >> 172.21.15.40:0/3579226842 conn(0x7f6b70069720 msgr2=0x7f6b70069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:34.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.286+0000 7f6b74a83640 1 -- 172.21.15.40:0/3579226842 shutdown_connections 2023-10-18T09:01:34.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.286+0000 7f6b74a83640 1 -- 172.21.15.40:0/3579226842 wait complete. 2023-10-18T09:01:34.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.288+0000 7f6b74a83640 1 Processor -- start 2023-10-18T09:01:34.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.288+0000 7f6b74a83640 1 -- start start 2023-10-18T09:01:34.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.289+0000 7f6b74a83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b700aec00 0x7f6b700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:34.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.289+0000 7f6b74a83640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b700b2c20 0x7f6b7014da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:34.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.289+0000 7f6b74a83640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b7015a200 0x7f6b70154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:34.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.289+0000 7f6b74a83640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b70070b30 con 0x7f6b700aec00 2023-10-18T09:01:34.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.289+0000 7f6b74a83640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b70070d60 con 0x7f6b7015a200 2023-10-18T09:01:34.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.289+0000 7f6b74a83640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b70071040 con 0x7f6b700b2c20 2023-10-18T09:01:34.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.289+0000 7f6b6f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b700aec00 0x7f6b700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:34.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.289+0000 7f6b6f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b700aec00 0x7f6b700705f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41408/0 (socket says 172.21.15.40:41408) 2023-10-18T09:01:34.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.289+0000 7f6b6f7fe640 1 -- 172.21.15.40:0/147824474 learned_addr learned my addr 172.21.15.40:0/147824474 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:34.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.290+0000 7f6b6f7fe640 1 -- 172.21.15.40:0/147824474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b700b2c20 msgr2=0x7f6b7014da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:34.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.290+0000 7f6b6f7fe640 1 --2- 172.21.15.40:0/147824474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b700b2c20 0x7f6b7014da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.290+0000 7f6b6f7fe640 1 -- 172.21.15.40:0/147824474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b7015a200 msgr2=0x7f6b70154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:34.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.290+0000 7f6b6f7fe640 1 --2- 172.21.15.40:0/147824474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b7015a200 0x7f6b70154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.290+0000 7f6b6f7fe640 1 -- 172.21.15.40:0/147824474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b60009490 con 0x7f6b700aec00 2023-10-18T09:01:34.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.290+0000 7f6b6f7fe640 1 --2- 172.21.15.40:0/147824474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b700aec00 0x7f6b700705f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6b60000c00 tx=0x7f6b6002c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:34.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.290+0000 7f6b57fff640 1 -- 172.21.15.40:0/147824474 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60003a80 con 0x7f6b700aec00 2023-10-18T09:01:34.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.290+0000 7f6b57fff640 1 -- 172.21.15.40:0/147824474 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6b60003cf0 con 0x7f6b700aec00 2023-10-18T09:01:34.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.290+0000 7f6b57fff640 1 -- 172.21.15.40:0/147824474 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b600346a0 con 0x7f6b700aec00 2023-10-18T09:01:34.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.290+0000 7f6b74a83640 1 -- 172.21.15.40:0/147824474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b70071320 con 0x7f6b700aec00 2023-10-18T09:01:34.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.291+0000 7f6b74a83640 1 -- 172.21.15.40:0/147824474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b70155800 con 0x7f6b700aec00 2023-10-18T09:01:34.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.292+0000 7f6b74a83640 1 -- 172.21.15.40:0/147824474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b700aac40 con 0x7f6b700aec00 2023-10-18T09:01:34.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.292+0000 7f6b57fff640 1 -- 172.21.15.40:0/147824474 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6b6003b040 con 0x7f6b700aec00 2023-10-18T09:01:34.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.293+0000 7f6b57fff640 1 --2- 172.21.15.40:0/147824474 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b5c06ae40 0x7f6b5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:34.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.293+0000 7f6b57fff640 1 -- 172.21.15.40:0/147824474 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6b600ab500 con 0x7f6b700aec00 2023-10-18T09:01:34.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.294+0000 7f6b6effd640 1 --2- 172.21.15.40:0/147824474 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b5c06ae40 0x7f6b5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:34.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.295+0000 7f6b6effd640 1 --2- 172.21.15.40:0/147824474 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b5c06ae40 0x7f6b5c06d300 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6b64000f60 tx=0x7f6b64007a30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:34.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.297+0000 7f6b57fff640 1 -- 172.21.15.40:0/147824474 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b700aac40 con 0x7f6b700aec00 2023-10-18T09:01:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.491+0000 7f6b74a83640 1 -- 172.21.15.40:0/147824474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b700aac40 con 0x7f6b700aec00 2023-10-18T09:01:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.492+0000 7f6b57fff640 1 -- 172.21.15.40:0/147824474 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b700aac40 con 0x7f6b700aec00 2023-10-18T09:01:34.495 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:01:34.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.495+0000 7f6b74a83640 1 -- 172.21.15.40:0/147824474 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b5c06ae40 msgr2=0x7f6b5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:34.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.495+0000 7f6b74a83640 1 --2- 172.21.15.40:0/147824474 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b5c06ae40 0x7f6b5c06d300 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6b64000f60 tx=0x7f6b64007a30 comp rx=0 tx=0).stop 2023-10-18T09:01:34.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.495+0000 7f6b74a83640 1 -- 172.21.15.40:0/147824474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b700aec00 msgr2=0x7f6b700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:34.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.495+0000 7f6b74a83640 1 --2- 172.21.15.40:0/147824474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b700aec00 0x7f6b700705f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6b60000c00 tx=0x7f6b6002c880 comp rx=0 tx=0).stop 2023-10-18T09:01:34.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.496+0000 7f6b74a83640 1 -- 172.21.15.40:0/147824474 shutdown_connections 2023-10-18T09:01:34.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.496+0000 7f6b74a83640 1 --2- 172.21.15.40:0/147824474 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b5c06ae40 0x7f6b5c06d300 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.496+0000 7f6b74a83640 1 --2- 172.21.15.40:0/147824474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b7015a200 0x7f6b70154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.496+0000 7f6b74a83640 1 --2- 172.21.15.40:0/147824474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b700b2c20 0x7f6b7014da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.496+0000 7f6b74a83640 1 --2- 172.21.15.40:0/147824474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b700aec00 0x7f6b700705f0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.496+0000 7f6b74a83640 1 -- 172.21.15.40:0/147824474 >> 172.21.15.40:0/147824474 conn(0x7f6b70069720 msgr2=0x7f6b70072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.496+0000 7f6b74a83640 1 -- 172.21.15.40:0/147824474 shutdown_connections 2023-10-18T09:01:34.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.496+0000 7f6b74a83640 1 -- 172.21.15.40:0/147824474 wait complete. 2023-10-18T09:01:34.518 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T09:01:34.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.735+0000 7f583a71d640 1 -- 172.21.15.40:0/1948155822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58340cc990 msgr2=0x7f58340ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:34.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.735+0000 7f583a71d640 1 --2- 172.21.15.40:0/1948155822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58340cc990 0x7f58340ccd90 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f583406c840 tx=0x7f582c02c260 comp rx=0 tx=0).stop 2023-10-18T09:01:34.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.735+0000 7f583a71d640 1 -- 172.21.15.40:0/1948155822 shutdown_connections 2023-10-18T09:01:34.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.735+0000 7f583a71d640 1 --2- 172.21.15.40:0/1948155822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58340cc990 0x7f58340ccd90 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.735+0000 7f583a71d640 1 --2- 172.21.15.40:0/1948155822 >> v2:172.21.15.144:3300/0 conn(0x7f58340b0bc0 0x7f58340b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.735+0000 7f583a71d640 1 -- 172.21.15.40:0/1948155822 >> 172.21.15.40:0/1948155822 conn(0x7f583406cee0 msgr2=0x7f5834069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:34.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.735+0000 7f583a71d640 1 -- 172.21.15.40:0/1948155822 shutdown_connections 2023-10-18T09:01:34.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.735+0000 7f583a71d640 1 -- 172.21.15.40:0/1948155822 wait complete. 2023-10-18T09:01:34.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.737+0000 7f583a71d640 1 Processor -- start 2023-10-18T09:01:34.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.737+0000 7f583a71d640 1 -- start start 2023-10-18T09:01:34.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.739+0000 7f583a71d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58340b0bc0 0x7f58340cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:34.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.739+0000 7f583a71d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58340cc990 0x7f583414fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:34.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.739+0000 7f583a71d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5834151bd0 0x7f583413a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:34.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.739+0000 7f583a71d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5834150690 con 0x7f5834151bd0 2023-10-18T09:01:34.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.739+0000 7f583a71d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583414d3a0 con 0x7f58340cc990 2023-10-18T09:01:34.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.739+0000 7f583a71d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583414c180 con 0x7f58340b0bc0 2023-10-18T09:01:34.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.739+0000 7f5838cba640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5834151bd0 0x7f583413a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:34.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.739+0000 7f5838cba640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5834151bd0 0x7f583413a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41426/0 (socket says 172.21.15.40:41426) 2023-10-18T09:01:34.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.739+0000 7f5838cba640 1 -- 172.21.15.40:0/2125934278 learned_addr learned my addr 172.21.15.40:0/2125934278 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:34.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.739+0000 7f5833fff640 1 --2- 172.21.15.40:0/2125934278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58340b0bc0 0x7f58340cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:34.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.739+0000 7f582bfff640 1 --2- 172.21.15.40:0/2125934278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58340cc990 0x7f583414fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:34.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.740+0000 7f5838cba640 1 -- 172.21.15.40:0/2125934278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58340b0bc0 msgr2=0x7f58340cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:34.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.740+0000 7f5838cba640 1 --2- 172.21.15.40:0/2125934278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58340b0bc0 0x7f58340cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.740+0000 7f5838cba640 1 -- 172.21.15.40:0/2125934278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58340cc990 msgr2=0x7f583414fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:34.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.740+0000 7f5838cba640 1 --2- 172.21.15.40:0/2125934278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58340cc990 0x7f583414fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.740+0000 7f5838cba640 1 -- 172.21.15.40:0/2125934278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c009900 con 0x7f5834151bd0 2023-10-18T09:01:34.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.740+0000 7f5838cba640 1 --2- 172.21.15.40:0/2125934278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5834151bd0 0x7f583413a930 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f582c037040 tx=0x7f582c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:34.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.740+0000 7f58317fa640 1 -- 172.21.15.40:0/2125934278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c004250 con 0x7f5834151bd0 2023-10-18T09:01:34.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.740+0000 7f58317fa640 1 -- 172.21.15.40:0/2125934278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f582c0044a0 con 0x7f5834151bd0 2023-10-18T09:01:34.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.740+0000 7f58317fa640 1 -- 172.21.15.40:0/2125934278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c004890 con 0x7f5834151bd0 2023-10-18T09:01:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.740+0000 7f583a71d640 1 -- 172.21.15.40:0/2125934278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f583414c460 con 0x7f5834151bd0 2023-10-18T09:01:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.741+0000 7f583a71d640 1 -- 172.21.15.40:0/2125934278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58341503a0 con 0x7f5834151bd0 2023-10-18T09:01:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.742+0000 7f583a71d640 1 -- 172.21.15.40:0/2125934278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5834066fd0 con 0x7f5834151bd0 2023-10-18T09:01:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.743+0000 7f58317fa640 1 -- 172.21.15.40:0/2125934278 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f582c03b040 con 0x7f5834151bd0 2023-10-18T09:01:34.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.744+0000 7f58317fa640 1 --2- 172.21.15.40:0/2125934278 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f581406ae40 0x7f581406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:34.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.744+0000 7f58317fa640 1 -- 172.21.15.40:0/2125934278 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f582c0ac1b0 con 0x7f5834151bd0 2023-10-18T09:01:34.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.744+0000 7f5833fff640 1 --2- 172.21.15.40:0/2125934278 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f581406ae40 0x7f581406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:34.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.745+0000 7f5833fff640 1 --2- 172.21.15.40:0/2125934278 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f581406ae40 0x7f581406d300 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5824006fd0 tx=0x7f5824008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:34.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.747+0000 7f58317fa640 1 -- 172.21.15.40:0/2125934278 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f582c0775a0 con 0x7f5834151bd0 2023-10-18T09:01:34.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.942+0000 7f583a71d640 1 -- 172.21.15.40:0/2125934278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f583406fb50 con 0x7f5834151bd0 2023-10-18T09:01:34.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.943+0000 7f58317fa640 1 -- 172.21.15.40:0/2125934278 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v103) v1 ==== 84+0+65 (secure 0 0 0) 0x7f582c03d3f0 con 0x7f5834151bd0 2023-10-18T09:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-10-18T09:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-10-18T09:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-10-18T09:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-10-18T09:01:34.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.946+0000 7f583a71d640 1 -- 172.21.15.40:0/2125934278 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f581406ae40 msgr2=0x7f581406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:34.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.946+0000 7f583a71d640 1 --2- 172.21.15.40:0/2125934278 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f581406ae40 0x7f581406d300 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5824006fd0 tx=0x7f5824008040 comp rx=0 tx=0).stop 2023-10-18T09:01:34.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.946+0000 7f583a71d640 1 -- 172.21.15.40:0/2125934278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5834151bd0 msgr2=0x7f583413a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:34.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.946+0000 7f583a71d640 1 --2- 172.21.15.40:0/2125934278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5834151bd0 0x7f583413a930 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f582c037040 tx=0x7f582c003c10 comp rx=0 tx=0).stop 2023-10-18T09:01:34.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.946+0000 7f583a71d640 1 -- 172.21.15.40:0/2125934278 shutdown_connections 2023-10-18T09:01:34.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.946+0000 7f583a71d640 1 --2- 172.21.15.40:0/2125934278 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f581406ae40 0x7f581406d300 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.946+0000 7f583a71d640 1 --2- 172.21.15.40:0/2125934278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5834151bd0 0x7f583413a930 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.946+0000 7f583a71d640 1 --2- 172.21.15.40:0/2125934278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58340cc990 0x7f583414fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.946+0000 7f583a71d640 1 --2- 172.21.15.40:0/2125934278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58340b0bc0 0x7f58340cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:34.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.946+0000 7f583a71d640 1 -- 172.21.15.40:0/2125934278 >> 172.21.15.40:0/2125934278 conn(0x7f583406cee0 msgr2=0x7f583414cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:34.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.946+0000 7f583a71d640 1 -- 172.21.15.40:0/2125934278 shutdown_connections 2023-10-18T09:01:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:34.947+0000 7f583a71d640 1 -- 172.21.15.40:0/2125934278 wait complete. 2023-10-18T09:01:34.967 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T09:01:34.967 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T09:01:35.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.200+0000 7f8bf938e640 1 -- 172.21.15.40:0/3308462556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bf415b750 msgr2=0x7f8bf40d1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.200+0000 7f8bf938e640 1 --2- 172.21.15.40:0/3308462556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bf415b750 0x7f8bf40d1610 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009830 tx=0x7f8bdc02bcd0 comp rx=0 tx=0).stop 2023-10-18T09:01:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.200+0000 7f8bf938e640 1 -- 172.21.15.40:0/3308462556 shutdown_connections 2023-10-18T09:01:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.200+0000 7f8bf938e640 1 --2- 172.21.15.40:0/3308462556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bf415b750 0x7f8bf40d1610 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:35.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.200+0000 7f8bf938e640 1 -- 172.21.15.40:0/3308462556 >> 172.21.15.40:0/3308462556 conn(0x7f8bf408acf0 msgr2=0x7f8bf409a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:35.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.201+0000 7f8bf938e640 1 -- 172.21.15.40:0/3308462556 shutdown_connections 2023-10-18T09:01:35.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.201+0000 7f8bf938e640 1 -- 172.21.15.40:0/3308462556 wait complete. 2023-10-18T09:01:35.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.202+0000 7f8bf938e640 1 Processor -- start 2023-10-18T09:01:35.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.203+0000 7f8bf938e640 1 -- start start 2023-10-18T09:01:35.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.203+0000 7f8bf938e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bf40cdb70 0x7f8bf41510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:35.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.204+0000 7f8bf938e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf40d1230 0x7f8bf4151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:35.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.204+0000 7f8bf938e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bf415b750 0x7f8bf4152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:35.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.204+0000 7f8bf938e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf40c9680 con 0x7f8bf415b750 2023-10-18T09:01:35.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.204+0000 7f8bf938e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf40c8a00 con 0x7f8bf40d1230 2023-10-18T09:01:35.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.204+0000 7f8bf938e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf40c8ce0 con 0x7f8bf40cdb70 2023-10-18T09:01:35.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.204+0000 7f8bf8b8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bf415b750 0x7f8bf4152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:35.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.204+0000 7f8bf8b8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bf415b750 0x7f8bf4152f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50820/0 (socket says 172.21.15.40:50820) 2023-10-18T09:01:35.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.204+0000 7f8bf8b8d640 1 -- 172.21.15.40:0/952242567 learned_addr learned my addr 172.21.15.40:0/952242567 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:35.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.204+0000 7f8beb7fe640 1 --2- 172.21.15.40:0/952242567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf40d1230 0x7f8bf4151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:35.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.204+0000 7f8bf3fff640 1 --2- 172.21.15.40:0/952242567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bf40cdb70 0x7f8bf41510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:35.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.204+0000 7f8bf8b8d640 1 -- 172.21.15.40:0/952242567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bf40cdb70 msgr2=0x7f8bf41510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:35.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.204+0000 7f8bf8b8d640 1 --2- 172.21.15.40:0/952242567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bf40cdb70 0x7f8bf41510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:35.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.204+0000 7f8bf8b8d640 1 -- 172.21.15.40:0/952242567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf40d1230 msgr2=0x7f8bf4151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:35.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.204+0000 7f8bf8b8d640 1 --2- 172.21.15.40:0/952242567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf40d1230 0x7f8bf4151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:35.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.204+0000 7f8bf8b8d640 1 -- 172.21.15.40:0/952242567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bdc009490 con 0x7f8bf415b750 2023-10-18T09:01:35.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.205+0000 7f8bf8b8d640 1 --2- 172.21.15.40:0/952242567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bf415b750 0x7f8bf4152f50 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00c780 tx=0x7f8bec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:35.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.205+0000 7f8bf17fa640 1 -- 172.21.15.40:0/952242567 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec004b60 con 0x7f8bf415b750 2023-10-18T09:01:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.205+0000 7f8bf17fa640 1 -- 172.21.15.40:0/952242567 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8bec00ce90 con 0x7f8bf415b750 2023-10-18T09:01:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.205+0000 7f8bf938e640 1 -- 172.21.15.40:0/952242567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf40c8fc0 con 0x7f8bf415b750 2023-10-18T09:01:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.205+0000 7f8bf17fa640 1 -- 172.21.15.40:0/952242567 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec00aab0 con 0x7f8bf415b750 2023-10-18T09:01:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.205+0000 7f8bf938e640 1 -- 172.21.15.40:0/952242567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf4153490 con 0x7f8bf415b750 2023-10-18T09:01:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.206+0000 7f8bf938e640 1 -- 172.21.15.40:0/952242567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf4071da0 con 0x7f8bf415b750 2023-10-18T09:01:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.208+0000 7f8bf17fa640 1 -- 172.21.15.40:0/952242567 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8bec00ac10 con 0x7f8bf415b750 2023-10-18T09:01:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.209+0000 7f8bf17fa640 1 --2- 172.21.15.40:0/952242567 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8bd406ae70 0x7f8bd406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.210+0000 7f8bf17fa640 1 -- 172.21.15.40:0/952242567 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8bec089710 con 0x7f8bf415b750 2023-10-18T09:01:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.210+0000 7f8bf3fff640 1 --2- 172.21.15.40:0/952242567 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8bd406ae70 0x7f8bd406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.211+0000 7f8bf3fff640 1 --2- 172.21.15.40:0/952242567 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8bd406ae70 0x7f8bd406d330 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8be4007fd0 tx=0x7f8be4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:35.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.215+0000 7f8bf17fa640 1 -- 172.21.15.40:0/952242567 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8bec054b00 con 0x7f8bf415b750 2023-10-18T09:01:35.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.399+0000 7f8bf938e640 1 -- 172.21.15.40:0/952242567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8bf40c97c0 con 0x7f8bf415b750 2023-10-18T09:01:35.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.507+0000 7f8bf17fa640 1 -- 172.21.15.40:0/952242567 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v104) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8bf40c97c0 con 0x7f8bf415b750 2023-10-18T09:01:35.510 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-10-18T09:01:35.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.511+0000 7f8bf938e640 1 -- 172.21.15.40:0/952242567 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8bd406ae70 msgr2=0x7f8bd406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:35.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.511+0000 7f8bf938e640 1 --2- 172.21.15.40:0/952242567 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8bd406ae70 0x7f8bd406d330 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8be4007fd0 tx=0x7f8be4009040 comp rx=0 tx=0).stop 2023-10-18T09:01:35.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.511+0000 7f8bf938e640 1 -- 172.21.15.40:0/952242567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bf415b750 msgr2=0x7f8bf4152f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:35.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.511+0000 7f8bf938e640 1 --2- 172.21.15.40:0/952242567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bf415b750 0x7f8bf4152f50 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00c780 tx=0x7f8bec00cc40 comp rx=0 tx=0).stop 2023-10-18T09:01:35.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.511+0000 7f8bf938e640 1 -- 172.21.15.40:0/952242567 shutdown_connections 2023-10-18T09:01:35.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.511+0000 7f8bf938e640 1 --2- 172.21.15.40:0/952242567 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8bd406ae70 0x7f8bd406d330 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:35.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.511+0000 7f8bf938e640 1 --2- 172.21.15.40:0/952242567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bf415b750 0x7f8bf4152f50 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:35.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.511+0000 7f8bf938e640 1 --2- 172.21.15.40:0/952242567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bf40d1230 0x7f8bf4151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:35.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.511+0000 7f8bf938e640 1 --2- 172.21.15.40:0/952242567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bf40cdb70 0x7f8bf41510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:35.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.511+0000 7f8bf938e640 1 -- 172.21.15.40:0/952242567 >> 172.21.15.40:0/952242567 conn(0x7f8bf408acf0 msgr2=0x7f8bf40b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:35.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.511+0000 7f8bf938e640 1 -- 172.21.15.40:0/952242567 shutdown_connections 2023-10-18T09:01:35.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.511+0000 7f8bf938e640 1 -- 172.21.15.40:0/952242567 wait complete. 2023-10-18T09:01:35.537 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T09:01:35.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.771+0000 7efd33fff640 1 -- 172.21.15.40:0/389733022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd340d19b0 msgr2=0x7efd340ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:35.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.771+0000 7efd33fff640 1 --2- 172.21.15.40:0/389733022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd340d19b0 0x7efd340ccbf0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7efd2c005fd0 tx=0x7efd2c02c2d0 comp rx=0 tx=0).stop 2023-10-18T09:01:35.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.771+0000 7efd33fff640 1 -- 172.21.15.40:0/389733022 shutdown_connections 2023-10-18T09:01:35.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.771+0000 7efd33fff640 1 --2- 172.21.15.40:0/389733022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd340d19b0 0x7efd340ccbf0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:35.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.771+0000 7efd33fff640 1 --2- 172.21.15.40:0/389733022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd3415eea0 0x7efd340cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:35.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.771+0000 7efd33fff640 1 -- 172.21.15.40:0/389733022 >> 172.21.15.40:0/389733022 conn(0x7efd3409a610 msgr2=0x7efd340d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:35.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.771+0000 7efd33fff640 1 -- 172.21.15.40:0/389733022 shutdown_connections 2023-10-18T09:01:35.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.771+0000 7efd33fff640 1 -- 172.21.15.40:0/389733022 wait complete. 2023-10-18T09:01:35.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.773+0000 7efd33fff640 1 Processor -- start 2023-10-18T09:01:35.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.773+0000 7efd33fff640 1 -- start start 2023-10-18T09:01:35.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.774+0000 7efd33fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd340d19b0 0x7efd3415c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:35.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.774+0000 7efd33fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd3415eac0 0x7efd340c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.774+0000 7efd33fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd3415eea0 0x7efd34151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.774+0000 7efd33fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3415c8f0 con 0x7efd3415eac0 2023-10-18T09:01:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.774+0000 7efd33fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd340c9da0 con 0x7efd3415eea0 2023-10-18T09:01:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.774+0000 7efd33fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3415cb00 con 0x7efd340d19b0 2023-10-18T09:01:35.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.774+0000 7efd327fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd3415eac0 0x7efd340c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:35.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.774+0000 7efd327fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd3415eac0 0x7efd340c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50836/0 (socket says 172.21.15.40:50836) 2023-10-18T09:01:35.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.775+0000 7efd327fc640 1 -- 172.21.15.40:0/2319023488 learned_addr learned my addr 172.21.15.40:0/2319023488 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:35.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.775+0000 7efd327fc640 1 -- 172.21.15.40:0/2319023488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd340d19b0 msgr2=0x7efd3415c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.775+0000 7efd337fe640 1 --2- 172.21.15.40:0/2319023488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd3415eea0 0x7efd34151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.775+0000 7efd32ffd640 1 --2- 172.21.15.40:0/2319023488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd340d19b0 0x7efd3415c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.775+0000 7efd327fc640 1 --2- 172.21.15.40:0/2319023488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd340d19b0 0x7efd3415c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.775+0000 7efd327fc640 1 -- 172.21.15.40:0/2319023488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd3415eea0 msgr2=0x7efd34151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.775+0000 7efd327fc640 1 --2- 172.21.15.40:0/2319023488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd3415eea0 0x7efd34151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.775+0000 7efd327fc640 1 -- 172.21.15.40:0/2319023488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2c009900 con 0x7efd3415eac0 2023-10-18T09:01:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.775+0000 7efd337fe640 1 --2- 172.21.15.40:0/2319023488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd3415eea0 0x7efd34151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.775+0000 7efd32ffd640 1 --2- 172.21.15.40:0/2319023488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd340d19b0 0x7efd3415c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.775+0000 7efd327fc640 1 --2- 172.21.15.40:0/2319023488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd3415eac0 0x7efd340c9860 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7efd2800d6a0 tx=0x7efd2800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.776+0000 7efd1b7fe640 1 -- 172.21.15.40:0/2319023488 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28005220 con 0x7efd3415eac0 2023-10-18T09:01:35.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.776+0000 7efd1b7fe640 1 -- 172.21.15.40:0/2319023488 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efd28005470 con 0x7efd3415eac0 2023-10-18T09:01:35.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.776+0000 7efd1b7fe640 1 -- 172.21.15.40:0/2319023488 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28009a80 con 0x7efd3415eac0 2023-10-18T09:01:35.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.776+0000 7efd33fff640 1 -- 172.21.15.40:0/2319023488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3415cde0 con 0x7efd3415eac0 2023-10-18T09:01:35.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.776+0000 7efd33fff640 1 -- 172.21.15.40:0/2319023488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3415d3a0 con 0x7efd3415eac0 2023-10-18T09:01:35.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.777+0000 7efd33fff640 1 -- 172.21.15.40:0/2319023488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd34071da0 con 0x7efd3415eac0 2023-10-18T09:01:35.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.778+0000 7efd1b7fe640 1 -- 172.21.15.40:0/2319023488 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efd28009be0 con 0x7efd3415eac0 2023-10-18T09:01:35.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.779+0000 7efd1b7fe640 1 --2- 172.21.15.40:0/2319023488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efd2006ae40 0x7efd2006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:35.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.779+0000 7efd1b7fe640 1 -- 172.21.15.40:0/2319023488 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5227+0+0 (secure 0 0 0) 0x7efd28089ac0 con 0x7efd3415eac0 2023-10-18T09:01:35.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.780+0000 7efd32ffd640 1 --2- 172.21.15.40:0/2319023488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efd2006ae40 0x7efd2006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:35.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.781+0000 7efd32ffd640 1 --2- 172.21.15.40:0/2319023488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efd2006ae40 0x7efd2006d300 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efd24000f60 tx=0x7efd24009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:35.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.783+0000 7efd1b7fe640 1 -- 172.21.15.40:0/2319023488 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efd28054d30 con 0x7efd3415eac0 2023-10-18T09:01:35.944 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:01:35.954 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:01:35.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:35.968+0000 7efd33fff640 1 -- 172.21.15.40:0/2319023488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efd34066e00 con 0x7efd3415eac0 2023-10-18T09:01:36.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.513+0000 7efd1b7fe640 1 -- 172.21.15.40:0/2319023488 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v105) v1 ==== 170+0+0 (secure 0 0 0) 0x7efd28020020 con 0x7efd3415eac0 2023-10-18T09:01:36.516 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-10-18T09:01:36.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.517+0000 7efd33fff640 1 -- 172.21.15.40:0/2319023488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efd2006ae40 msgr2=0x7efd2006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:36.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.517+0000 7efd33fff640 1 --2- 172.21.15.40:0/2319023488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efd2006ae40 0x7efd2006d300 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efd24000f60 tx=0x7efd24009040 comp rx=0 tx=0).stop 2023-10-18T09:01:36.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.517+0000 7efd33fff640 1 -- 172.21.15.40:0/2319023488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd3415eac0 msgr2=0x7efd340c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:36.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.517+0000 7efd33fff640 1 --2- 172.21.15.40:0/2319023488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd3415eac0 0x7efd340c9860 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7efd2800d6a0 tx=0x7efd2800db60 comp rx=0 tx=0).stop 2023-10-18T09:01:36.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.517+0000 7efd33fff640 1 -- 172.21.15.40:0/2319023488 shutdown_connections 2023-10-18T09:01:36.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.517+0000 7efd33fff640 1 --2- 172.21.15.40:0/2319023488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efd2006ae40 0x7efd2006d300 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:36.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.517+0000 7efd33fff640 1 --2- 172.21.15.40:0/2319023488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd3415eea0 0x7efd34151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:36.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.517+0000 7efd33fff640 1 --2- 172.21.15.40:0/2319023488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd3415eac0 0x7efd340c9860 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:36.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.517+0000 7efd33fff640 1 --2- 172.21.15.40:0/2319023488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd340d19b0 0x7efd3415c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:36.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.517+0000 7efd33fff640 1 -- 172.21.15.40:0/2319023488 >> 172.21.15.40:0/2319023488 conn(0x7efd3409a610 msgr2=0x7efd34072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:36.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.518+0000 7efd33fff640 1 -- 172.21.15.40:0/2319023488 shutdown_connections 2023-10-18T09:01:36.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.521+0000 7efd33fff640 1 -- 172.21.15.40:0/2319023488 wait complete. 2023-10-18T09:01:36.553 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T09:01:36.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.756+0000 7fe564d5e640 1 -- 172.21.15.40:0/2124828781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe54c003970 msgr2=0x7fe54c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:36.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.756+0000 7fe564d5e640 1 --2- 172.21.15.40:0/2124828781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe54c003970 0x7fe54c003d90 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe550005fd0 tx=0x7fe54c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:01:36.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.756+0000 7fe564d5e640 1 -- 172.21.15.40:0/2124828781 shutdown_connections 2023-10-18T09:01:36.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.756+0000 7fe564d5e640 1 --2- 172.21.15.40:0/2124828781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe54c003970 0x7fe54c003d90 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:36.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.756+0000 7fe564d5e640 1 -- 172.21.15.40:0/2124828781 >> 172.21.15.40:0/2124828781 conn(0x7fe56006ca10 msgr2=0x7fe56006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:36.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.756+0000 7fe564d5e640 1 -- 172.21.15.40:0/2124828781 shutdown_connections 2023-10-18T09:01:36.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.756+0000 7fe564d5e640 1 -- 172.21.15.40:0/2124828781 wait complete. 2023-10-18T09:01:36.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.758+0000 7fe564d5e640 1 Processor -- start 2023-10-18T09:01:36.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.758+0000 7fe564d5e640 1 -- start start 2023-10-18T09:01:36.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.759+0000 7fe564d5e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe54c003970 0x7fe560158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:36.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.759+0000 7fe564d5e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5601439f0 0x7fe560143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:36.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.759+0000 7fe564d5e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5600aa880 0x7fe56014fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:36.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.759+0000 7fe564d5e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe560158f10 con 0x7fe5601439f0 2023-10-18T09:01:36.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.759+0000 7fe564d5e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe560158c30 con 0x7fe5600aa880 2023-10-18T09:01:36.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.759+0000 7fe564d5e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56014ef40 con 0x7fe54c003970 2023-10-18T09:01:36.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.759+0000 7fe55effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5601439f0 0x7fe560143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:36.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.759+0000 7fe55effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5601439f0 0x7fe560143240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50858/0 (socket says 172.21.15.40:50858) 2023-10-18T09:01:36.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.759+0000 7fe55effd640 1 -- 172.21.15.40:0/2406752520 learned_addr learned my addr 172.21.15.40:0/2406752520 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:36.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.759+0000 7fe55ffff640 1 --2- 172.21.15.40:0/2406752520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5600aa880 0x7fe56014fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:36.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.759+0000 7fe55f7fe640 1 --2- 172.21.15.40:0/2406752520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe54c003970 0x7fe560158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:36.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.760+0000 7fe55effd640 1 -- 172.21.15.40:0/2406752520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe54c003970 msgr2=0x7fe560158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:36.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.760+0000 7fe55effd640 1 --2- 172.21.15.40:0/2406752520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe54c003970 0x7fe560158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:36.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.760+0000 7fe55effd640 1 -- 172.21.15.40:0/2406752520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5600aa880 msgr2=0x7fe56014fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:36.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.760+0000 7fe55effd640 1 --2- 172.21.15.40:0/2406752520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5600aa880 0x7fe56014fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:36.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.760+0000 7fe55effd640 1 -- 172.21.15.40:0/2406752520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56014f220 con 0x7fe5601439f0 2023-10-18T09:01:36.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.760+0000 7fe55effd640 1 --2- 172.21.15.40:0/2406752520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5601439f0 0x7fe560143240 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe5480066a0 tx=0x7fe548003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:36.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.760+0000 7fe557fff640 1 -- 172.21.15.40:0/2406752520 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54800f980 con 0x7fe5601439f0 2023-10-18T09:01:36.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.760+0000 7fe557fff640 1 -- 172.21.15.40:0/2406752520 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe54800fae0 con 0x7fe5601439f0 2023-10-18T09:01:36.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.760+0000 7fe557fff640 1 -- 172.21.15.40:0/2406752520 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548011430 con 0x7fe5601439f0 2023-10-18T09:01:36.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.760+0000 7fe564d5e640 1 -- 172.21.15.40:0/2406752520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe56014f500 con 0x7fe5601439f0 2023-10-18T09:01:36.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.760+0000 7fe564d5e640 1 -- 172.21.15.40:0/2406752520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe560143780 con 0x7fe5601439f0 2023-10-18T09:01:36.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.761+0000 7fe564d5e640 1 -- 172.21.15.40:0/2406752520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56014f740 con 0x7fe5601439f0 2023-10-18T09:01:36.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.763+0000 7fe557fff640 1 -- 172.21.15.40:0/2406752520 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe548011590 con 0x7fe5601439f0 2023-10-18T09:01:36.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.764+0000 7fe557fff640 1 --2- 172.21.15.40:0/2406752520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe54006ae70 0x7fe54006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:36.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.764+0000 7fe557fff640 1 -- 172.21.15.40:0/2406752520 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe54808a190 con 0x7fe5601439f0 2023-10-18T09:01:36.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.764+0000 7fe55f7fe640 1 --2- 172.21.15.40:0/2406752520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe54006ae70 0x7fe54006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:36.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.765+0000 7fe55f7fe640 1 --2- 172.21.15.40:0/2406752520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe54006ae70 0x7fe54006d330 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe550036000 tx=0x7fe550035040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:36.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.769+0000 7fe557fff640 1 -- 172.21.15.40:0/2406752520 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe56014f740 con 0x7fe5601439f0 2023-10-18T09:01:36.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:36.993+0000 7fe564d5e640 1 -- 172.21.15.40:0/2406752520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe5600c9b60 con 0x7fe5601439f0 2023-10-18T09:01:37.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.536+0000 7fe557fff640 1 -- 172.21.15.40:0/2406752520 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v45) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe5600c9b60 con 0x7fe5601439f0 2023-10-18T09:01:37.544 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 12 and data pool 13 2023-10-18T09:01:37.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.549+0000 7fe564d5e640 1 -- 172.21.15.40:0/2406752520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe54006ae70 msgr2=0x7fe54006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:37.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.549+0000 7fe564d5e640 1 --2- 172.21.15.40:0/2406752520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe54006ae70 0x7fe54006d330 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe550036000 tx=0x7fe550035040 comp rx=0 tx=0).stop 2023-10-18T09:01:37.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.549+0000 7fe564d5e640 1 -- 172.21.15.40:0/2406752520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5601439f0 msgr2=0x7fe560143240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:37.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.549+0000 7fe564d5e640 1 --2- 172.21.15.40:0/2406752520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5601439f0 0x7fe560143240 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe5480066a0 tx=0x7fe548003a80 comp rx=0 tx=0).stop 2023-10-18T09:01:37.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.550+0000 7fe564d5e640 1 -- 172.21.15.40:0/2406752520 shutdown_connections 2023-10-18T09:01:37.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.550+0000 7fe564d5e640 1 --2- 172.21.15.40:0/2406752520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe54006ae70 0x7fe54006d330 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:37.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.550+0000 7fe564d5e640 1 --2- 172.21.15.40:0/2406752520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5600aa880 0x7fe56014fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:37.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.550+0000 7fe564d5e640 1 --2- 172.21.15.40:0/2406752520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5601439f0 0x7fe560143240 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:37.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.550+0000 7fe564d5e640 1 --2- 172.21.15.40:0/2406752520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe54c003970 0x7fe560158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:37.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.550+0000 7fe564d5e640 1 -- 172.21.15.40:0/2406752520 >> 172.21.15.40:0/2406752520 conn(0x7fe56006ca10 msgr2=0x7fe5600cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:37.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.552+0000 7fe564d5e640 1 -- 172.21.15.40:0/2406752520 shutdown_connections 2023-10-18T09:01:37.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.554+0000 7fe564d5e640 1 -- 172.21.15.40:0/2406752520 wait complete. 2023-10-18T09:01:37.581 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:01:37.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.774+0000 7f78dd347640 1 -- 172.21.15.40:0/479652842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78d80b15b0 msgr2=0x7f78d80c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:37.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.774+0000 7f78dd347640 1 --2- 172.21.15.40:0/479652842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78d80b15b0 0x7f78d80c99c0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009890 tx=0x7f78c002e020 comp rx=0 tx=0).stop 2023-10-18T09:01:37.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.774+0000 7f78dd347640 1 -- 172.21.15.40:0/479652842 shutdown_connections 2023-10-18T09:01:37.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.774+0000 7f78dd347640 1 --2- 172.21.15.40:0/479652842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78d80b15b0 0x7f78d80c99c0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:37.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.774+0000 7f78dd347640 1 -- 172.21.15.40:0/479652842 >> 172.21.15.40:0/479652842 conn(0x7f78d8069720 msgr2=0x7f78d8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:37.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.774+0000 7f78dd347640 1 -- 172.21.15.40:0/479652842 shutdown_connections 2023-10-18T09:01:37.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.774+0000 7f78dd347640 1 -- 172.21.15.40:0/479652842 wait complete. 2023-10-18T09:01:37.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.776+0000 7f78dd347640 1 Processor -- start 2023-10-18T09:01:37.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.776+0000 7f78dd347640 1 -- start start 2023-10-18T09:01:37.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.777+0000 7f78dd347640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78d806d6f0 0x7f78d814ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:37.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.777+0000 7f78dd347640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78d80b15b0 0x7f78d814c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:37.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.777+0000 7f78dd347640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78d80c9f00 0x7f78d81e0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:37.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.777+0000 7f78dd347640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d81e1290 con 0x7f78d80c9f00 2023-10-18T09:01:37.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.777+0000 7f78dd347640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d8156020 con 0x7f78d80b15b0 2023-10-18T09:01:37.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.777+0000 7f78dd347640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d8156300 con 0x7f78d806d6f0 2023-10-18T09:01:37.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.777+0000 7f78dcb46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78d80c9f00 0x7f78d81e0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:37.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.777+0000 7f78dcb46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78d80c9f00 0x7f78d81e0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50870/0 (socket says 172.21.15.40:50870) 2023-10-18T09:01:37.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.777+0000 7f78dcb46640 1 -- 172.21.15.40:0/903501064 learned_addr learned my addr 172.21.15.40:0/903501064 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:37.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.777+0000 7f78d7fff640 1 --2- 172.21.15.40:0/903501064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78d806d6f0 0x7f78d814ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:37.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.777+0000 7f78cffff640 1 --2- 172.21.15.40:0/903501064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78d80b15b0 0x7f78d814c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:37.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.778+0000 7f78dcb46640 1 -- 172.21.15.40:0/903501064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78d806d6f0 msgr2=0x7f78d814ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:37.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.778+0000 7f78dcb46640 1 --2- 172.21.15.40:0/903501064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78d806d6f0 0x7f78d814ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:37.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.778+0000 7f78dcb46640 1 -- 172.21.15.40:0/903501064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78d80b15b0 msgr2=0x7f78d814c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:37.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.778+0000 7f78dcb46640 1 --2- 172.21.15.40:0/903501064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78d80b15b0 0x7f78d814c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:37.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.778+0000 7f78dcb46640 1 -- 172.21.15.40:0/903501064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c0009490 con 0x7f78d80c9f00 2023-10-18T09:01:37.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.778+0000 7f78dcb46640 1 --2- 172.21.15.40:0/903501064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78d80c9f00 0x7f78d81e0d50 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f78d000c6b0 tx=0x7f78d000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:37.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.778+0000 7f78d57fa640 1 -- 172.21.15.40:0/903501064 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0003b70 con 0x7f78d80c9f00 2023-10-18T09:01:37.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.778+0000 7f78d57fa640 1 -- 172.21.15.40:0/903501064 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f78d0003cf0 con 0x7f78d80c9f00 2023-10-18T09:01:37.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.778+0000 7f78d57fa640 1 -- 172.21.15.40:0/903501064 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0005330 con 0x7f78d80c9f00 2023-10-18T09:01:37.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.778+0000 7f78dd347640 1 -- 172.21.15.40:0/903501064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d81565e0 con 0x7f78d80c9f00 2023-10-18T09:01:37.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.779+0000 7f78dd347640 1 -- 172.21.15.40:0/903501064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d8156ba0 con 0x7f78d80c9f00 2023-10-18T09:01:37.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.780+0000 7f78dd347640 1 -- 172.21.15.40:0/903501064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d8066e00 con 0x7f78d80c9f00 2023-10-18T09:01:37.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.781+0000 7f78d57fa640 1 -- 172.21.15.40:0/903501064 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f78d0005490 con 0x7f78d80c9f00 2023-10-18T09:01:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.782+0000 7f78d57fa640 1 --2- 172.21.15.40:0/903501064 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78b806ae40 0x7f78b806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.782+0000 7f78d57fa640 1 -- 172.21.15.40:0/903501064 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78d0089550 con 0x7f78d80c9f00 2023-10-18T09:01:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.782+0000 7f78d7fff640 1 --2- 172.21.15.40:0/903501064 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78b806ae40 0x7f78b806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:37.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.783+0000 7f78d7fff640 1 --2- 172.21.15.40:0/903501064 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78b806ae40 0x7f78b806d300 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f78c80046b0 tx=0x7f78c8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:37.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.786+0000 7f78d57fa640 1 -- 172.21.15.40:0/903501064 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78d0054600 con 0x7f78d80c9f00 2023-10-18T09:01:37.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.980+0000 7f78dd347640 1 -- 172.21.15.40:0/903501064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78d80caf90 con 0x7f78d80c9f00 2023-10-18T09:01:37.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.983+0000 7f78d57fa640 1 -- 172.21.15.40:0/903501064 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16596 (secure 0 0 0) 0x7f78d0057c20 con 0x7f78d80c9f00 2023-10-18T09:01:37.986 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:37.986 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":106,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:01:36.994770+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:01:35.453236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-18T09:01:36.012460+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:01:37.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.985+0000 7f78dd347640 1 -- 172.21.15.40:0/903501064 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78b806ae40 msgr2=0x7f78b806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:37.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.985+0000 7f78dd347640 1 --2- 172.21.15.40:0/903501064 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78b806ae40 0x7f78b806d300 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f78c80046b0 tx=0x7f78c8009040 comp rx=0 tx=0).stop 2023-10-18T09:01:37.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.985+0000 7f78dd347640 1 -- 172.21.15.40:0/903501064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78d80c9f00 msgr2=0x7f78d81e0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:37.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.985+0000 7f78dd347640 1 --2- 172.21.15.40:0/903501064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78d80c9f00 0x7f78d81e0d50 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f78d000c6b0 tx=0x7f78d000cb70 comp rx=0 tx=0).stop 2023-10-18T09:01:37.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.985+0000 7f78dd347640 1 -- 172.21.15.40:0/903501064 shutdown_connections 2023-10-18T09:01:37.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.985+0000 7f78dd347640 1 --2- 172.21.15.40:0/903501064 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f78b806ae40 0x7f78b806d300 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:37.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.985+0000 7f78dd347640 1 --2- 172.21.15.40:0/903501064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78d80c9f00 0x7f78d81e0d50 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:37.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.985+0000 7f78dd347640 1 --2- 172.21.15.40:0/903501064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78d80b15b0 0x7f78d814c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:37.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.985+0000 7f78dd347640 1 --2- 172.21.15.40:0/903501064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78d806d6f0 0x7f78d814ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:37.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.985+0000 7f78dd347640 1 -- 172.21.15.40:0/903501064 >> 172.21.15.40:0/903501064 conn(0x7f78d8069720 msgr2=0x7f78d80cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:37.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.985+0000 7f78dd347640 1 -- 172.21.15.40:0/903501064 shutdown_connections 2023-10-18T09:01:37.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:37.985+0000 7f78dd347640 1 -- 172.21.15.40:0/903501064 wait complete. 2023-10-18T09:01:38.007 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:01:38.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.231+0000 7f90e759e640 1 -- 172.21.15.40:0/3786929344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e80ca0d0 msgr2=0x7f90e8157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:38.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.231+0000 7f90e759e640 1 --2- 172.21.15.40:0/3786929344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e80ca0d0 0x7f90e8157d40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f90e806aeb0 tx=0x7f90e002c110 comp rx=0 tx=0).stop 2023-10-18T09:01:38.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.231+0000 7f90e759e640 1 -- 172.21.15.40:0/3786929344 shutdown_connections 2023-10-18T09:01:38.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.231+0000 7f90e759e640 1 --2- 172.21.15.40:0/3786929344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e80ca0d0 0x7f90e8157d40 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:38.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.231+0000 7f90e759e640 1 --2- 172.21.15.40:0/3786929344 >> v2:172.21.15.144:3300/0 conn(0x7f90e806d6f0 0x7f90e80b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:38.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.231+0000 7f90e759e640 1 -- 172.21.15.40:0/3786929344 >> 172.21.15.40:0/3786929344 conn(0x7f90e8069720 msgr2=0x7f90e8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:38.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.231+0000 7f90e759e640 1 -- 172.21.15.40:0/3786929344 shutdown_connections 2023-10-18T09:01:38.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.231+0000 7f90e759e640 1 -- 172.21.15.40:0/3786929344 wait complete. 2023-10-18T09:01:38.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.233+0000 7f90e759e640 1 Processor -- start 2023-10-18T09:01:38.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.233+0000 7f90e759e640 1 -- start start 2023-10-18T09:01:38.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.234+0000 7f90e759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e806d6f0 0x7f90e813dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:38.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.234+0000 7f90e759e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90e80b0630 0x7f90e8140a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:38.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.234+0000 7f90e759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90e80ca0d0 0x7f90e8140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:38.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.234+0000 7f90e759e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e80af4b0 con 0x7f90e806d6f0 2023-10-18T09:01:38.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.234+0000 7f90e759e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e813eb10 con 0x7f90e80ca0d0 2023-10-18T09:01:38.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.234+0000 7f90e759e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e813edf0 con 0x7f90e80b0630 2023-10-18T09:01:38.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.234+0000 7f90e659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e806d6f0 0x7f90e813dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:38.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.235+0000 7f90e659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e806d6f0 0x7f90e813dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50886/0 (socket says 172.21.15.40:50886) 2023-10-18T09:01:38.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.235+0000 7f90e659c640 1 -- 172.21.15.40:0/1903636213 learned_addr learned my addr 172.21.15.40:0/1903636213 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:38.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.235+0000 7f90e6d9d640 1 --2- 172.21.15.40:0/1903636213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90e80ca0d0 0x7f90e8140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:38.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.235+0000 7f90dffff640 1 --2- 172.21.15.40:0/1903636213 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90e80b0630 0x7f90e8140a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:38.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.235+0000 7f90e659c640 1 -- 172.21.15.40:0/1903636213 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90e80b0630 msgr2=0x7f90e8140a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:38.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.235+0000 7f90e659c640 1 --2- 172.21.15.40:0/1903636213 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90e80b0630 0x7f90e8140a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:38.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.235+0000 7f90e659c640 1 -- 172.21.15.40:0/1903636213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90e80ca0d0 msgr2=0x7f90e8140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:38.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.235+0000 7f90e659c640 1 --2- 172.21.15.40:0/1903636213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90e80ca0d0 0x7f90e8140fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:38.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.235+0000 7f90e659c640 1 -- 172.21.15.40:0/1903636213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e0009840 con 0x7f90e806d6f0 2023-10-18T09:01:38.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.235+0000 7f90e659c640 1 --2- 172.21.15.40:0/1903636213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e806d6f0 0x7f90e813dfd0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f90d800ddc0 tx=0x7f90d800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:38.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.236+0000 7f90deffd640 1 -- 172.21.15.40:0/1903636213 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d800e650 con 0x7f90e806d6f0 2023-10-18T09:01:38.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.236+0000 7f90deffd640 1 -- 172.21.15.40:0/1903636213 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f90d800e8a0 con 0x7f90e806d6f0 2023-10-18T09:01:38.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.236+0000 7f90deffd640 1 -- 172.21.15.40:0/1903636213 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d800ecb0 con 0x7f90e806d6f0 2023-10-18T09:01:38.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.236+0000 7f90e759e640 1 -- 172.21.15.40:0/1903636213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e813f0d0 con 0x7f90e806d6f0 2023-10-18T09:01:38.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.236+0000 7f90e759e640 1 -- 172.21.15.40:0/1903636213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e814d320 con 0x7f90e806d6f0 2023-10-18T09:01:38.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.237+0000 7f90e759e640 1 -- 172.21.15.40:0/1903636213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e8066e00 con 0x7f90e806d6f0 2023-10-18T09:01:38.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.238+0000 7f90deffd640 1 -- 172.21.15.40:0/1903636213 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f90d8016440 con 0x7f90e806d6f0 2023-10-18T09:01:38.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.239+0000 7f90deffd640 1 --2- 172.21.15.40:0/1903636213 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f90c806ae40 0x7f90c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:38.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.239+0000 7f90deffd640 1 -- 172.21.15.40:0/1903636213 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90d8089ee0 con 0x7f90e806d6f0 2023-10-18T09:01:38.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.240+0000 7f90dffff640 1 --2- 172.21.15.40:0/1903636213 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f90c806ae40 0x7f90c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:38.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.241+0000 7f90dffff640 1 --2- 172.21.15.40:0/1903636213 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f90c806ae40 0x7f90c806d300 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f90e81507a0 tx=0x7f90d0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:38.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.243+0000 7f90deffd640 1 -- 172.21.15.40:0/1903636213 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f90d8054fa0 con 0x7f90e806d6f0 2023-10-18T09:01:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.441+0000 7f90e759e640 1 -- 172.21.15.40:0/1903636213 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90e80caf90 con 0x7f90e806d6f0 2023-10-18T09:01:38.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.443+0000 7f90deffd640 1 -- 172.21.15.40:0/1903636213 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16596 (secure 0 0 0) 0x7f90d801b020 con 0x7f90e806d6f0 2023-10-18T09:01:38.446 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:38.447 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":106,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:01:36.994770+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:01:35.453236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-18T09:01:36.012460+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:01:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.446+0000 7f90e759e640 1 -- 172.21.15.40:0/1903636213 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f90c806ae40 msgr2=0x7f90c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.446+0000 7f90e759e640 1 --2- 172.21.15.40:0/1903636213 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f90c806ae40 0x7f90c806d300 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f90e81507a0 tx=0x7f90d0009040 comp rx=0 tx=0).stop 2023-10-18T09:01:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.446+0000 7f90e759e640 1 -- 172.21.15.40:0/1903636213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e806d6f0 msgr2=0x7f90e813dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.446+0000 7f90e759e640 1 --2- 172.21.15.40:0/1903636213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e806d6f0 0x7f90e813dfd0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f90d800ddc0 tx=0x7f90d800a620 comp rx=0 tx=0).stop 2023-10-18T09:01:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.447+0000 7f90e759e640 1 -- 172.21.15.40:0/1903636213 shutdown_connections 2023-10-18T09:01:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.447+0000 7f90e759e640 1 --2- 172.21.15.40:0/1903636213 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f90c806ae40 0x7f90c806d300 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.447+0000 7f90e759e640 1 --2- 172.21.15.40:0/1903636213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90e80ca0d0 0x7f90e8140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.447+0000 7f90e759e640 1 --2- 172.21.15.40:0/1903636213 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90e80b0630 0x7f90e8140a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.447+0000 7f90e759e640 1 --2- 172.21.15.40:0/1903636213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90e806d6f0 0x7f90e813dfd0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.447+0000 7f90e759e640 1 -- 172.21.15.40:0/1903636213 >> 172.21.15.40:0/1903636213 conn(0x7f90e8069720 msgr2=0x7f90e81542c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.447+0000 7f90e759e640 1 -- 172.21.15.40:0/1903636213 shutdown_connections 2023-10-18T09:01:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.447+0000 7f90e759e640 1 -- 172.21.15.40:0/1903636213 wait complete. 2023-10-18T09:01:38.468 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T09:01:38.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.703+0000 7f7572b2c640 1 -- 172.21.15.40:0/1273726081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f755c005180 msgr2=0x7f755c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:38.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.703+0000 7f7572b2c640 1 --2- 172.21.15.40:0/1273726081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f755c005180 0x7f755c00b980 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7564009e20 tx=0x7f755c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:01:38.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.703+0000 7f75710c9640 1 -- 172.21.15.40:0/1273726081 reap_dead start 2023-10-18T09:01:38.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.703+0000 7f7572b2c640 1 -- 172.21.15.40:0/1273726081 shutdown_connections 2023-10-18T09:01:38.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.703+0000 7f7572b2c640 1 -- 172.21.15.40:0/1273726081 >> 172.21.15.40:0/1273726081 conn(0x7f756c069720 msgr2=0x7f756c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:38.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.703+0000 7f7572b2c640 1 -- 172.21.15.40:0/1273726081 shutdown_connections 2023-10-18T09:01:38.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.703+0000 7f7572b2c640 1 -- 172.21.15.40:0/1273726081 wait complete. 2023-10-18T09:01:38.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.705+0000 7f7572b2c640 1 Processor -- start 2023-10-18T09:01:38.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.705+0000 7f7572b2c640 1 -- start start 2023-10-18T09:01:38.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.706+0000 7f7572b2c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f755c005180 0x7f756c1419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:38.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.706+0000 7f7572b2c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f756c0ae780 0x7f756c0aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:38.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.706+0000 7f7572b2c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f756c15d140 0x7f756c15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:38.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.706+0000 7f7572b2c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c0af140 con 0x7f755c005180 2023-10-18T09:01:38.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.706+0000 7f7572b2c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c15ea10 con 0x7f756c0ae780 2023-10-18T09:01:38.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.706+0000 7f7572b2c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756c141f10 con 0x7f756c15d140 2023-10-18T09:01:38.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.706+0000 7f75708c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f755c005180 0x7f756c1419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:38.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.706+0000 7f75708c8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f755c005180 0x7f756c1419d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50904/0 (socket says 172.21.15.40:50904) 2023-10-18T09:01:38.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.707+0000 7f75708c8640 1 -- 172.21.15.40:0/667891788 learned_addr learned my addr 172.21.15.40:0/667891788 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:38.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.707+0000 7f756bfff640 1 --2- 172.21.15.40:0/667891788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f756c0ae780 0x7f756c0aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:38.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.707+0000 7f75708c8640 1 -- 172.21.15.40:0/667891788 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f756c15d140 msgr2=0x7f756c15d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:38.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.707+0000 7f75708c8640 1 --2- 172.21.15.40:0/667891788 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f756c15d140 0x7f756c15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:38.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.707+0000 7f75708c8640 1 -- 172.21.15.40:0/667891788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f756c0ae780 msgr2=0x7f756c0aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:38.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.707+0000 7f75708c8640 1 --2- 172.21.15.40:0/667891788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f756c0ae780 0x7f756c0aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:38.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.707+0000 7f75708c8640 1 -- 172.21.15.40:0/667891788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c15de00 con 0x7f755c005180 2023-10-18T09:01:38.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.707+0000 7f75708c8640 1 --2- 172.21.15.40:0/667891788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f755c005180 0x7f756c1419d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f756000d6d0 tx=0x7f756000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:38.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.708+0000 7f75697fa640 1 -- 172.21.15.40:0/667891788 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560004220 con 0x7f755c005180 2023-10-18T09:01:38.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.708+0000 7f75697fa640 1 -- 172.21.15.40:0/667891788 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f75600043c0 con 0x7f755c005180 2023-10-18T09:01:38.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.708+0000 7f75697fa640 1 -- 172.21.15.40:0/667891788 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75600099a0 con 0x7f755c005180 2023-10-18T09:01:38.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.708+0000 7f7572b2c640 1 -- 172.21.15.40:0/667891788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756c15e0e0 con 0x7f755c005180 2023-10-18T09:01:38.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.708+0000 7f7572b2c640 1 -- 172.21.15.40:0/667891788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c15e6a0 con 0x7f755c005180 2023-10-18T09:01:38.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.709+0000 7f7572b2c640 1 -- 172.21.15.40:0/667891788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756c15eb50 con 0x7f755c005180 2023-10-18T09:01:38.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.710+0000 7f75697fa640 1 -- 172.21.15.40:0/667891788 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7560009b00 con 0x7f755c005180 2023-10-18T09:01:38.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.711+0000 7f75697fa640 1 --2- 172.21.15.40:0/667891788 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f755006ae40 0x7f755006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:38.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.711+0000 7f75697fa640 1 -- 172.21.15.40:0/667891788 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75600894b0 con 0x7f755c005180 2023-10-18T09:01:38.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.712+0000 7f756bfff640 1 --2- 172.21.15.40:0/667891788 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f755006ae40 0x7f755006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:38.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.713+0000 7f756bfff640 1 --2- 172.21.15.40:0/667891788 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f755006ae40 0x7f755006d300 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f75580017a0 tx=0x7f7558001d70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:38.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.716+0000 7f75697fa640 1 -- 172.21.15.40:0/667891788 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7560054560 con 0x7f755c005180 2023-10-18T09:01:38.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:38.949+0000 7f7572b2c640 1 -- 172.21.15.40:0/667891788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f756c15e860 con 0x7f755c005180 2023-10-18T09:01:39.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.560+0000 7f75697fa640 1 -- 172.21.15.40:0/667891788 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v48) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7560009db0 con 0x7f755c005180 2023-10-18T09:01:39.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.563+0000 7f7572b2c640 1 -- 172.21.15.40:0/667891788 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f755006ae40 msgr2=0x7f755006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:39.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.563+0000 7f7572b2c640 1 --2- 172.21.15.40:0/667891788 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f755006ae40 0x7f755006d300 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f75580017a0 tx=0x7f7558001d70 comp rx=0 tx=0).stop 2023-10-18T09:01:39.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.563+0000 7f7572b2c640 1 -- 172.21.15.40:0/667891788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f755c005180 msgr2=0x7f756c1419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:39.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.563+0000 7f7572b2c640 1 --2- 172.21.15.40:0/667891788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f755c005180 0x7f756c1419d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f756000d6d0 tx=0x7f756000db90 comp rx=0 tx=0).stop 2023-10-18T09:01:39.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.563+0000 7f7572b2c640 1 -- 172.21.15.40:0/667891788 shutdown_connections 2023-10-18T09:01:39.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.563+0000 7f7572b2c640 1 --2- 172.21.15.40:0/667891788 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f755006ae40 0x7f755006d300 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:39.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.563+0000 7f7572b2c640 1 --2- 172.21.15.40:0/667891788 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f756c15d140 0x7f756c15d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:39.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.563+0000 7f7572b2c640 1 --2- 172.21.15.40:0/667891788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f756c0ae780 0x7f756c0aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:39.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.563+0000 7f7572b2c640 1 --2- 172.21.15.40:0/667891788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f755c005180 0x7f756c1419d0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:39.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.563+0000 7f7572b2c640 1 -- 172.21.15.40:0/667891788 >> 172.21.15.40:0/667891788 conn(0x7f756c069720 msgr2=0x7f756c153d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:39.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.564+0000 7f7572b2c640 1 -- 172.21.15.40:0/667891788 shutdown_connections 2023-10-18T09:01:39.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.564+0000 7f7572b2c640 1 -- 172.21.15.40:0/667891788 wait complete. 2023-10-18T09:01:39.586 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T09:01:39.587 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T09:01:39.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.818+0000 7fb81c5a5640 1 -- 172.21.15.40:0/776349918 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8140cca30 msgr2=0x7fb8140cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:39.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.818+0000 7fb81c5a5640 1 --2- 172.21.15.40:0/776349918 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8140cca30 0x7fb8140cce70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb804009830 tx=0x7fb80402bd60 comp rx=0 tx=0).stop 2023-10-18T09:01:39.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.818+0000 7fb81c5a5640 1 -- 172.21.15.40:0/776349918 shutdown_connections 2023-10-18T09:01:39.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.818+0000 7fb81c5a5640 1 --2- 172.21.15.40:0/776349918 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb8140cca30 0x7fb8140cce70 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:39.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.818+0000 7fb81c5a5640 1 --2- 172.21.15.40:0/776349918 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb814157800 0x7fb8140b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:39.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.818+0000 7fb81c5a5640 1 -- 172.21.15.40:0/776349918 >> 172.21.15.40:0/776349918 conn(0x7fb814069720 msgr2=0x7fb814069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:39.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.818+0000 7fb81c5a5640 1 -- 172.21.15.40:0/776349918 shutdown_connections 2023-10-18T09:01:39.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.818+0000 7fb81c5a5640 1 -- 172.21.15.40:0/776349918 wait complete. 2023-10-18T09:01:39.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.820+0000 7fb81c5a5640 1 Processor -- start 2023-10-18T09:01:39.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.820+0000 7fb81c5a5640 1 -- start start 2023-10-18T09:01:39.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.821+0000 7fb81c5a5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8140cca30 0x7fb814161b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:39.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.821+0000 7fb81c5a5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8140cd3b0 0x7fb814150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:39.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.821+0000 7fb81c5a5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb814157800 0x7fb8141558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:39.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.821+0000 7fb81c5a5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81413c690 con 0x7fb814157800 2023-10-18T09:01:39.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.821+0000 7fb81c5a5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb814151400 con 0x7fb8140cd3b0 2023-10-18T09:01:39.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.821+0000 7fb81c5a5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8141516e0 con 0x7fb8140cca30 2023-10-18T09:01:39.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.821+0000 7fb81ab42640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb814157800 0x7fb8141558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:39.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.821+0000 7fb81ab42640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb814157800 0x7fb8141558f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50932/0 (socket says 172.21.15.40:50932) 2023-10-18T09:01:39.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.821+0000 7fb81ab42640 1 -- 172.21.15.40:0/222497434 learned_addr learned my addr 172.21.15.40:0/222497434 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:39.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.821+0000 7fb81a341640 1 --2- 172.21.15.40:0/222497434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8140cca30 0x7fb814161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:39.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.821+0000 7fb819b40640 1 --2- 172.21.15.40:0/222497434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8140cd3b0 0x7fb814150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:39.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.822+0000 7fb81ab42640 1 -- 172.21.15.40:0/222497434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8140cca30 msgr2=0x7fb814161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:39.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.822+0000 7fb81ab42640 1 --2- 172.21.15.40:0/222497434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8140cca30 0x7fb814161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.822+0000 7fb81ab42640 1 -- 172.21.15.40:0/222497434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8140cd3b0 msgr2=0x7fb814150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.822+0000 7fb81ab42640 1 --2- 172.21.15.40:0/222497434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8140cd3b0 0x7fb814150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.822+0000 7fb81ab42640 1 -- 172.21.15.40:0/222497434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb804009490 con 0x7fb814157800 2023-10-18T09:01:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.822+0000 7fb81ab42640 1 --2- 172.21.15.40:0/222497434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb814157800 0x7fb8141558f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00c700 tx=0x7fb80c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:39.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.822+0000 7fb80affd640 1 -- 172.21.15.40:0/222497434 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c00a640 con 0x7fb814157800 2023-10-18T09:01:39.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.822+0000 7fb80affd640 1 -- 172.21.15.40:0/222497434 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb80c00a890 con 0x7fb814157800 2023-10-18T09:01:39.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.822+0000 7fb80affd640 1 -- 172.21.15.40:0/222497434 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c00ace0 con 0x7fb814157800 2023-10-18T09:01:39.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.822+0000 7fb81c5a5640 1 -- 172.21.15.40:0/222497434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8141519c0 con 0x7fb814157800 2023-10-18T09:01:39.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.823+0000 7fb81c5a5640 1 -- 172.21.15.40:0/222497434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb814150fc0 con 0x7fb814157800 2023-10-18T09:01:39.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.824+0000 7fb81c5a5640 1 -- 172.21.15.40:0/222497434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb814066fd0 con 0x7fb814157800 2023-10-18T09:01:39.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.825+0000 7fb80affd640 1 -- 172.21.15.40:0/222497434 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb80c01a3b0 con 0x7fb814157800 2023-10-18T09:01:39.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.826+0000 7fb80affd640 1 --2- 172.21.15.40:0/222497434 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7f806ae40 0x7fb7f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:39.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.826+0000 7fb80affd640 1 -- 172.21.15.40:0/222497434 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb80c0896e0 con 0x7fb814157800 2023-10-18T09:01:39.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.826+0000 7fb81a341640 1 --2- 172.21.15.40:0/222497434 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7f806ae40 0x7fb7f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:39.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.828+0000 7fb81a341640 1 --2- 172.21.15.40:0/222497434 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7f806ae40 0x7fb7f806d300 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb810005fd0 tx=0x7fb8100097d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:39.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:39.830+0000 7fb80affd640 1 -- 172.21.15.40:0/222497434 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb80c054660 con 0x7fb814157800 2023-10-18T09:01:40.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.067+0000 7fb81c5a5640 1 -- 172.21.15.40:0/222497434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb8140cc450 con 0x7fb814157800 2023-10-18T09:01:40.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.570+0000 7fb80affd640 1 -- 172.21.15.40:0/222497434 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v49) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb8140cc450 con 0x7fb814157800 2023-10-18T09:01:40.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.574+0000 7fb81c5a5640 1 -- 172.21.15.40:0/222497434 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7f806ae40 msgr2=0x7fb7f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:40.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.574+0000 7fb81c5a5640 1 --2- 172.21.15.40:0/222497434 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7f806ae40 0x7fb7f806d300 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb810005fd0 tx=0x7fb8100097d0 comp rx=0 tx=0).stop 2023-10-18T09:01:40.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.574+0000 7fb81c5a5640 1 -- 172.21.15.40:0/222497434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb814157800 msgr2=0x7fb8141558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:40.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.574+0000 7fb81c5a5640 1 --2- 172.21.15.40:0/222497434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb814157800 0x7fb8141558f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00c700 tx=0x7fb80c00cbc0 comp rx=0 tx=0).stop 2023-10-18T09:01:40.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.574+0000 7fb81c5a5640 1 -- 172.21.15.40:0/222497434 shutdown_connections 2023-10-18T09:01:40.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.574+0000 7fb81c5a5640 1 --2- 172.21.15.40:0/222497434 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb7f806ae40 0x7fb7f806d300 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:40.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.574+0000 7fb81c5a5640 1 --2- 172.21.15.40:0/222497434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb814157800 0x7fb8141558f0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:40.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.574+0000 7fb81c5a5640 1 --2- 172.21.15.40:0/222497434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8140cd3b0 0x7fb814150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:40.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.574+0000 7fb81c5a5640 1 --2- 172.21.15.40:0/222497434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8140cca30 0x7fb814161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:40.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.574+0000 7fb81c5a5640 1 -- 172.21.15.40:0/222497434 >> 172.21.15.40:0/222497434 conn(0x7fb814069720 msgr2=0x7fb8140cb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:40.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.574+0000 7fb81c5a5640 1 -- 172.21.15.40:0/222497434 shutdown_connections 2023-10-18T09:01:40.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.574+0000 7fb81c5a5640 1 -- 172.21.15.40:0/222497434 wait complete. 2023-10-18T09:01:40.599 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T09:01:40.599 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:01:40.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.831+0000 7fbcbafe9640 1 -- 172.21.15.40:0/2811494660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca0003970 msgr2=0x7fbca0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:40.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.831+0000 7fbcbafe9640 1 --2- 172.21.15.40:0/2811494660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca0003970 0x7fbca0003d90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009b50 tx=0x7fbca00057f0 comp rx=0 tx=0).stop 2023-10-18T09:01:40.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.831+0000 7fbcba7e8640 1 -- 172.21.15.40:0/2811494660 reap_dead start 2023-10-18T09:01:40.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.831+0000 7fbcbafe9640 1 -- 172.21.15.40:0/2811494660 shutdown_connections 2023-10-18T09:01:40.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.831+0000 7fbcbafe9640 1 -- 172.21.15.40:0/2811494660 >> 172.21.15.40:0/2811494660 conn(0x7fbcb4069720 msgr2=0x7fbcb4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:40.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.831+0000 7fbcbafe9640 1 -- 172.21.15.40:0/2811494660 shutdown_connections 2023-10-18T09:01:40.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.831+0000 7fbcbafe9640 1 -- 172.21.15.40:0/2811494660 wait complete. 2023-10-18T09:01:40.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.833+0000 7fbcbafe9640 1 Processor -- start 2023-10-18T09:01:40.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.833+0000 7fbcbafe9640 1 -- start start 2023-10-18T09:01:40.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.834+0000 7fbcbafe9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbca0003970 0x7fbcb413b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:40.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.834+0000 7fbcbafe9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb413bbf0 0x7fbcb40acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:40.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.834+0000 7fbcbafe9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb413c070 0x7fbcb40ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:40.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.834+0000 7fbcbafe9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb40aea60 con 0x7fbcb413bbf0 2023-10-18T09:01:40.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.834+0000 7fbcbafe9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb414e030 con 0x7fbcb413c070 2023-10-18T09:01:40.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.834+0000 7fbcbafe9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb413b960 con 0x7fbca0003970 2023-10-18T09:01:40.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.834+0000 7fbcb97e6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb413bbf0 0x7fbcb40acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:40.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.834+0000 7fbcb97e6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb413bbf0 0x7fbcb40acae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50952/0 (socket says 172.21.15.40:50952) 2023-10-18T09:01:40.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.834+0000 7fbcb97e6640 1 -- 172.21.15.40:0/2474114689 learned_addr learned my addr 172.21.15.40:0/2474114689 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:40.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.834+0000 7fbcb9fe7640 1 --2- 172.21.15.40:0/2474114689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbca0003970 0x7fbcb413b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:40.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.834+0000 7fbcba7e8640 1 --2- 172.21.15.40:0/2474114689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb413c070 0x7fbcb40ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:40.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.834+0000 7fbcb97e6640 1 -- 172.21.15.40:0/2474114689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbca0003970 msgr2=0x7fbcb413b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:40.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.835+0000 7fbcb97e6640 1 --2- 172.21.15.40:0/2474114689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbca0003970 0x7fbcb413b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:40.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.835+0000 7fbcb97e6640 1 -- 172.21.15.40:0/2474114689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb413c070 msgr2=0x7fbcb40ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:40.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.835+0000 7fbcb97e6640 1 --2- 172.21.15.40:0/2474114689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb413c070 0x7fbcb40ad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:40.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.835+0000 7fbcb97e6640 1 -- 172.21.15.40:0/2474114689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb40aefe0 con 0x7fbcb413bbf0 2023-10-18T09:01:40.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.835+0000 7fbcba7e8640 1 --2- 172.21.15.40:0/2474114689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb413c070 0x7fbcb40ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:40.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.835+0000 7fbcb97e6640 1 --2- 172.21.15.40:0/2474114689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb413bbf0 0x7fbcb40acae0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fbca400c6b0 tx=0x7fbca400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:40.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.835+0000 7fbcaa7fc640 1 -- 172.21.15.40:0/2474114689 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca4005270 con 0x7fbcb413bbf0 2023-10-18T09:01:40.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.835+0000 7fbcaa7fc640 1 -- 172.21.15.40:0/2474114689 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbca400a640 con 0x7fbcb413bbf0 2023-10-18T09:01:40.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.835+0000 7fbcaa7fc640 1 -- 172.21.15.40:0/2474114689 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca400aa50 con 0x7fbcb413bbf0 2023-10-18T09:01:40.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.835+0000 7fbcbafe9640 1 -- 172.21.15.40:0/2474114689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb41503a0 con 0x7fbcb413bbf0 2023-10-18T09:01:40.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.835+0000 7fbcbafe9640 1 -- 172.21.15.40:0/2474114689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb41500c0 con 0x7fbcb413bbf0 2023-10-18T09:01:40.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.837+0000 7fbcbafe9640 1 -- 172.21.15.40:0/2474114689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb40af170 con 0x7fbcb413bbf0 2023-10-18T09:01:40.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.838+0000 7fbcaa7fc640 1 -- 172.21.15.40:0/2474114689 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbca400abb0 con 0x7fbcb413bbf0 2023-10-18T09:01:40.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.839+0000 7fbcaa7fc640 1 --2- 172.21.15.40:0/2474114689 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc9806ae70 0x7fbc9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:40.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.839+0000 7fbcaa7fc640 1 -- 172.21.15.40:0/2474114689 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbca40890f0 con 0x7fbcb413bbf0 2023-10-18T09:01:40.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.839+0000 7fbcb9fe7640 1 --2- 172.21.15.40:0/2474114689 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc9806ae70 0x7fbc9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:40.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.841+0000 7fbcb9fe7640 1 --2- 172.21.15.40:0/2474114689 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc9806ae70 0x7fbc9806d330 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0036000 tx=0x7fbcb0035040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:40.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:40.843+0000 7fbcaa7fc640 1 -- 172.21.15.40:0/2474114689 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbca4054100 con 0x7fbcb413bbf0 2023-10-18T09:01:41.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.079+0000 7fbcbafe9640 1 -- 172.21.15.40:0/2474114689 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcb4066e00 con 0x7fbcb413bbf0 2023-10-18T09:01:41.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.080+0000 7fbcaa7fc640 1 -- 172.21.15.40:0/2474114689 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fbca4017090 con 0x7fbcb413bbf0 2023-10-18T09:01:41.083 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:41.083 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2565609773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2565609773},{"type":"v1","addr":"172.21.15.144:6836","nonce":2565609773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":5872,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3371548374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3371548374},{"type":"v1","addr":"172.21.15.144:6837","nonce":3371548374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3328648628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3328648628},{"type":"v1","addr":"172.21.15.40:6833","nonce":3328648628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:01:36.995669+0000","modified":"2023-10-18T09:01:40.068484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/891188043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":891188043},{"type":"v1","addr":"172.21.15.40:6835","nonce":891188043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-18T09:01:41.083 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 49 2023-10-18T09:01:41.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.083+0000 7fbcbafe9640 1 -- 172.21.15.40:0/2474114689 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc9806ae70 msgr2=0x7fbc9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:41.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.083+0000 7fbcbafe9640 1 --2- 172.21.15.40:0/2474114689 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc9806ae70 0x7fbc9806d330 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0036000 tx=0x7fbcb0035040 comp rx=0 tx=0).stop 2023-10-18T09:01:41.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.083+0000 7fbcbafe9640 1 -- 172.21.15.40:0/2474114689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb413bbf0 msgr2=0x7fbcb40acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:41.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.083+0000 7fbcbafe9640 1 --2- 172.21.15.40:0/2474114689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb413bbf0 0x7fbcb40acae0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fbca400c6b0 tx=0x7fbca400cb70 comp rx=0 tx=0).stop 2023-10-18T09:01:41.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.083+0000 7fbcbafe9640 1 -- 172.21.15.40:0/2474114689 shutdown_connections 2023-10-18T09:01:41.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.083+0000 7fbcbafe9640 1 --2- 172.21.15.40:0/2474114689 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbc9806ae70 0x7fbc9806d330 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:41.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.083+0000 7fbcbafe9640 1 --2- 172.21.15.40:0/2474114689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbcb413c070 0x7fbcb40ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:41.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.083+0000 7fbcbafe9640 1 --2- 172.21.15.40:0/2474114689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbcb413bbf0 0x7fbcb40acae0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:41.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.083+0000 7fbcbafe9640 1 --2- 172.21.15.40:0/2474114689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbca0003970 0x7fbcb413b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:41.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.083+0000 7fbcbafe9640 1 -- 172.21.15.40:0/2474114689 >> 172.21.15.40:0/2474114689 conn(0x7fbcb4069720 msgr2=0x7fbcb40aa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:41.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.083+0000 7fbcbafe9640 1 -- 172.21.15.40:0/2474114689 shutdown_connections 2023-10-18T09:01:41.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.083+0000 7fbcbafe9640 1 -- 172.21.15.40:0/2474114689 wait complete. 2023-10-18T09:01:41.106 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:01:41.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.334+0000 7fbe12554640 1 -- 172.21.15.40:0/1388220854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c1425a0 msgr2=0x7fbe0c0cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:41.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.334+0000 7fbe12554640 1 --2- 172.21.15.40:0/1388220854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c1425a0 0x7fbe0c0cafc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbe04005fd0 tx=0x7fbe0402bd90 comp rx=0 tx=0).stop 2023-10-18T09:01:41.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.334+0000 7fbe12554640 1 -- 172.21.15.40:0/1388220854 shutdown_connections 2023-10-18T09:01:41.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.334+0000 7fbe12554640 1 --2- 172.21.15.40:0/1388220854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c1425a0 0x7fbe0c0cafc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:41.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.334+0000 7fbe12554640 1 --2- 172.21.15.40:0/1388220854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe0c0ca160 0x7fbe0c0b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:41.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.334+0000 7fbe12554640 1 -- 172.21.15.40:0/1388220854 >> 172.21.15.40:0/1388220854 conn(0x7fbe0c069720 msgr2=0x7fbe0c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:41.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.335+0000 7fbe12554640 1 -- 172.21.15.40:0/1388220854 shutdown_connections 2023-10-18T09:01:41.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.335+0000 7fbe12554640 1 -- 172.21.15.40:0/1388220854 wait complete. 2023-10-18T09:01:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.336+0000 7fbe12554640 1 Processor -- start 2023-10-18T09:01:41.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.337+0000 7fbe12554640 1 -- start start 2023-10-18T09:01:41.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.338+0000 7fbe12554640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe0c0ca160 0x7fbe0c13aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:41.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.338+0000 7fbe12554640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c1420f0 0x7fbe0c0a71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:41.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.338+0000 7fbe12554640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe0c1425a0 0x7fbe0c1402a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:41.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.338+0000 7fbe12554640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c1efe80 con 0x7fbe0c1425a0 2023-10-18T09:01:41.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.338+0000 7fbe12554640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c13afe0 con 0x7fbe0c1420f0 2023-10-18T09:01:41.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.338+0000 7fbe12554640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c1f0070 con 0x7fbe0c0ca160 2023-10-18T09:01:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.338+0000 7fbe11d53640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe0c1425a0 0x7fbe0c1402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.338+0000 7fbe11d53640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe0c1425a0 0x7fbe0c1402a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50962/0 (socket says 172.21.15.40:50962) 2023-10-18T09:01:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.338+0000 7fbe11d53640 1 -- 172.21.15.40:0/3001577016 learned_addr learned my addr 172.21.15.40:0/3001577016 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.338+0000 7fbe11552640 1 --2- 172.21.15.40:0/3001577016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe0c0ca160 0x7fbe0c13aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:41.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.338+0000 7fbe10d51640 1 --2- 172.21.15.40:0/3001577016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c1420f0 0x7fbe0c0a71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:41.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.338+0000 7fbe11d53640 1 -- 172.21.15.40:0/3001577016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe0c0ca160 msgr2=0x7fbe0c13aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:41.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.339+0000 7fbe11d53640 1 --2- 172.21.15.40:0/3001577016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe0c0ca160 0x7fbe0c13aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:41.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.339+0000 7fbe11d53640 1 -- 172.21.15.40:0/3001577016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c1420f0 msgr2=0x7fbe0c0a71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:41.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.339+0000 7fbe11d53640 1 --2- 172.21.15.40:0/3001577016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c1420f0 0x7fbe0c0a71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:41.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.339+0000 7fbe11d53640 1 -- 172.21.15.40:0/3001577016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe04009560 con 0x7fbe0c1425a0 2023-10-18T09:01:41.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.339+0000 7fbe11552640 1 --2- 172.21.15.40:0/3001577016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe0c0ca160 0x7fbe0c13aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:01:41.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.339+0000 7fbe11d53640 1 --2- 172.21.15.40:0/3001577016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe0c1425a0 0x7fbe0c1402a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fbe04036040 tx=0x7fbe0402d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:41.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.339+0000 7fbdfdffb640 1 -- 172.21.15.40:0/3001577016 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0402dd70 con 0x7fbe0c1425a0 2023-10-18T09:01:41.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.339+0000 7fbdfdffb640 1 -- 172.21.15.40:0/3001577016 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbe040035f0 con 0x7fbe0c1425a0 2023-10-18T09:01:41.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.339+0000 7fbe12554640 1 -- 172.21.15.40:0/3001577016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0c1f0350 con 0x7fbe0c1425a0 2023-10-18T09:01:41.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.339+0000 7fbdfdffb640 1 -- 172.21.15.40:0/3001577016 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe04003a00 con 0x7fbe0c1425a0 2023-10-18T09:01:41.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.339+0000 7fbe12554640 1 -- 172.21.15.40:0/3001577016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0c1f0910 con 0x7fbe0c1425a0 2023-10-18T09:01:41.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.341+0000 7fbe12554640 1 -- 172.21.15.40:0/3001577016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0c0c98c0 con 0x7fbe0c1425a0 2023-10-18T09:01:41.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.341+0000 7fbdfdffb640 1 -- 172.21.15.40:0/3001577016 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbe04003b60 con 0x7fbe0c1425a0 2023-10-18T09:01:41.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.343+0000 7fbdfdffb640 1 --2- 172.21.15.40:0/3001577016 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbdf006ae40 0x7fbdf006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:41.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.343+0000 7fbdfdffb640 1 -- 172.21.15.40:0/3001577016 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe040ac180 con 0x7fbe0c1425a0 2023-10-18T09:01:41.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.344+0000 7fbe11552640 1 --2- 172.21.15.40:0/3001577016 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbdf006ae40 0x7fbdf006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:41.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.345+0000 7fbe11552640 1 --2- 172.21.15.40:0/3001577016 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbdf006ae40 0x7fbdf006d300 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbe00007fd0 tx=0x7fbe00009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:41.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.348+0000 7fbdfdffb640 1 -- 172.21.15.40:0/3001577016 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbe040772b0 con 0x7fbe0c1425a0 2023-10-18T09:01:41.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.549+0000 7fbe12554640 1 -- 172.21.15.40:0/3001577016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe0c066e00 con 0x7fbe0c1425a0 2023-10-18T09:01:41.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.552+0000 7fbdfdffb640 1 -- 172.21.15.40:0/3001577016 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16583 (secure 0 0 0) 0x7fbe0407a8d0 con 0x7fbe0c1425a0 2023-10-18T09:01:41.555 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:41.555 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":107,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:01:38.517370+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:01:35.453236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-18T09:01:36.012460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:01:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.555+0000 7fbe12554640 1 -- 172.21.15.40:0/3001577016 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbdf006ae40 msgr2=0x7fbdf006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.555+0000 7fbe12554640 1 --2- 172.21.15.40:0/3001577016 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbdf006ae40 0x7fbdf006d300 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbe00007fd0 tx=0x7fbe00009040 comp rx=0 tx=0).stop 2023-10-18T09:01:41.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.555+0000 7fbe12554640 1 -- 172.21.15.40:0/3001577016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe0c1425a0 msgr2=0x7fbe0c1402a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:41.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.555+0000 7fbe12554640 1 --2- 172.21.15.40:0/3001577016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe0c1425a0 0x7fbe0c1402a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fbe04036040 tx=0x7fbe0402d970 comp rx=0 tx=0).stop 2023-10-18T09:01:41.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.555+0000 7fbe12554640 1 -- 172.21.15.40:0/3001577016 shutdown_connections 2023-10-18T09:01:41.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.555+0000 7fbe12554640 1 --2- 172.21.15.40:0/3001577016 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbdf006ae40 0x7fbdf006d300 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:41.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.555+0000 7fbe12554640 1 --2- 172.21.15.40:0/3001577016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe0c1425a0 0x7fbe0c1402a0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:41.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.555+0000 7fbe12554640 1 --2- 172.21.15.40:0/3001577016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe0c1420f0 0x7fbe0c0a71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:41.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.555+0000 7fbe12554640 1 --2- 172.21.15.40:0/3001577016 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe0c0ca160 0x7fbe0c13aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:41.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.555+0000 7fbe12554640 1 -- 172.21.15.40:0/3001577016 >> 172.21.15.40:0/3001577016 conn(0x7fbe0c069720 msgr2=0x7fbe0c0cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:41.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.555+0000 7fbe12554640 1 -- 172.21.15.40:0/3001577016 shutdown_connections 2023-10-18T09:01:41.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.555+0000 7fbe12554640 1 -- 172.21.15.40:0/3001577016 wait complete. 2023-10-18T09:01:41.578 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T09:01:41.579 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:01:41.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.808+0000 7f35aee87640 1 -- 172.21.15.40:0/2958914803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a806bec0 msgr2=0x7f35a806dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:41.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.808+0000 7f35aee87640 1 --2- 172.21.15.40:0/2958914803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a806bec0 0x7f35a806dcd0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f35a80ce8d0 tx=0x7f35a002bd10 comp rx=0 tx=0).stop 2023-10-18T09:01:41.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.808+0000 7f35aee87640 1 -- 172.21.15.40:0/2958914803 shutdown_connections 2023-10-18T09:01:41.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.808+0000 7f35aee87640 1 --2- 172.21.15.40:0/2958914803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a806bec0 0x7f35a806dcd0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:41.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.808+0000 7f35aee87640 1 --2- 172.21.15.40:0/2958914803 >> v2:172.21.15.144:3300/0 conn(0x7f35a80ca8c0 0x7f35a806d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:41.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.808+0000 7f35aee87640 1 -- 172.21.15.40:0/2958914803 >> 172.21.15.40:0/2958914803 conn(0x7f35a80af530 msgr2=0x7f35a80af910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:41.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.809+0000 7f35aee87640 1 -- 172.21.15.40:0/2958914803 shutdown_connections 2023-10-18T09:01:41.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.809+0000 7f35aee87640 1 -- 172.21.15.40:0/2958914803 wait complete. 2023-10-18T09:01:41.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.810+0000 7f35aee87640 1 Processor -- start 2023-10-18T09:01:41.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.810+0000 7f35aee87640 1 -- start start 2023-10-18T09:01:41.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.811+0000 7f35aee87640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a806bec0 0x7f35a813a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:41.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.811+0000 7f35aee87640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35a80ca8c0 0x7f35a815b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:41.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.811+0000 7f35aee87640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a81594d0 0x7f35a815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:41.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.811+0000 7f35aee87640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a813aa60 con 0x7f35a81594d0 2023-10-18T09:01:41.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.811+0000 7f35aee87640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a814ee60 con 0x7f35a806bec0 2023-10-18T09:01:41.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.811+0000 7f35aee87640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a815ba20 con 0x7f35a80ca8c0 2023-10-18T09:01:41.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.812+0000 7f35ad424640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a81594d0 0x7f35a815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:41.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.812+0000 7f35ad424640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a81594d0 0x7f35a815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50990/0 (socket says 172.21.15.40:50990) 2023-10-18T09:01:41.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.812+0000 7f35ad424640 1 -- 172.21.15.40:0/3724570818 learned_addr learned my addr 172.21.15.40:0/3724570818 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:41.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.812+0000 7f35a7fff640 1 --2- 172.21.15.40:0/3724570818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35a80ca8c0 0x7f35a815b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:41.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.812+0000 7f35ad424640 1 -- 172.21.15.40:0/3724570818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35a80ca8c0 msgr2=0x7f35a815b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:41.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.812+0000 7f35ad424640 1 --2- 172.21.15.40:0/3724570818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35a80ca8c0 0x7f35a815b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:41.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.812+0000 7f35ad424640 1 -- 172.21.15.40:0/3724570818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a806bec0 msgr2=0x7f35a813a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:41.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.812+0000 7f35acc23640 1 --2- 172.21.15.40:0/3724570818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a806bec0 0x7f35a813a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:41.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.812+0000 7f35ad424640 1 --2- 172.21.15.40:0/3724570818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a806bec0 0x7f35a813a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:41.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.812+0000 7f35ad424640 1 -- 172.21.15.40:0/3724570818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a0009560 con 0x7f35a81594d0 2023-10-18T09:01:41.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.812+0000 7f35a7fff640 1 --2- 172.21.15.40:0/3724570818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35a80ca8c0 0x7f35a815b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:41.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.813+0000 7f35ad424640 1 --2- 172.21.15.40:0/3724570818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a81594d0 0x7f35a815d4a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f35a0009230 tx=0x7f35a002dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:41.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.813+0000 7f35a57fa640 1 -- 172.21.15.40:0/3724570818 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0003800 con 0x7f35a81594d0 2023-10-18T09:01:41.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.813+0000 7f35a57fa640 1 -- 172.21.15.40:0/3724570818 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f35a0003a50 con 0x7f35a81594d0 2023-10-18T09:01:41.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.813+0000 7f35a57fa640 1 -- 172.21.15.40:0/3724570818 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a0003e60 con 0x7f35a81594d0 2023-10-18T09:01:41.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.813+0000 7f35aee87640 1 -- 172.21.15.40:0/3724570818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a815bd00 con 0x7f35a81594d0 2023-10-18T09:01:41.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.813+0000 7f35aee87640 1 -- 172.21.15.40:0/3724570818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a814f390 con 0x7f35a81594d0 2023-10-18T09:01:41.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.814+0000 7f35aee87640 1 -- 172.21.15.40:0/3724570818 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a814f5b0 con 0x7f35a81594d0 2023-10-18T09:01:41.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.817+0000 7f35a57fa640 1 -- 172.21.15.40:0/3724570818 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f35a003e450 con 0x7f35a81594d0 2023-10-18T09:01:41.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.818+0000 7f35a57fa640 1 --2- 172.21.15.40:0/3724570818 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f358806ae90 0x7f358806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:41.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.818+0000 7f35a57fa640 1 -- 172.21.15.40:0/3724570818 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35a00ac410 con 0x7f35a81594d0 2023-10-18T09:01:41.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.819+0000 7f35acc23640 1 --2- 172.21.15.40:0/3724570818 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f358806ae90 0x7f358806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:41.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.820+0000 7f35acc23640 1 --2- 172.21.15.40:0/3724570818 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f358806ae90 0x7f358806d350 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3598007dc0 tx=0x7f35980072f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:41.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:41.821+0000 7f35a57fa640 1 -- 172.21.15.40:0/3724570818 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35a00775c0 con 0x7f35a81594d0 2023-10-18T09:01:42.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.012+0000 7f35aee87640 1 -- 172.21.15.40:0/3724570818 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f35a8065530 con 0x7f358806ae90 2023-10-18T09:01:42.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.036+0000 7f35a57fa640 1 -- 172.21.15.40:0/3724570818 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282810 (secure 0 0 0) 0x7f35980079b0 con 0x7f358806ae90 2023-10-18T09:01:42.042 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:42.044 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:01:42.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.042+0000 7f35aee87640 1 -- 172.21.15.40:0/3724570818 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f358806ae90 msgr2=0x7f358806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:42.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.042+0000 7f35aee87640 1 --2- 172.21.15.40:0/3724570818 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f358806ae90 0x7f358806d350 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3598007dc0 tx=0x7f35980072f0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.042+0000 7f35aee87640 1 -- 172.21.15.40:0/3724570818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a81594d0 msgr2=0x7f35a815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:42.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.042+0000 7f35aee87640 1 --2- 172.21.15.40:0/3724570818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a81594d0 0x7f35a815d4a0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f35a0009230 tx=0x7f35a002dc60 comp rx=0 tx=0).stop 2023-10-18T09:01:42.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.042+0000 7f35aee87640 1 -- 172.21.15.40:0/3724570818 shutdown_connections 2023-10-18T09:01:42.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.042+0000 7f35aee87640 1 --2- 172.21.15.40:0/3724570818 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f358806ae90 0x7f358806d350 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.042+0000 7f35aee87640 1 --2- 172.21.15.40:0/3724570818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35a81594d0 0x7f35a815d4a0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.042+0000 7f35aee87640 1 --2- 172.21.15.40:0/3724570818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35a80ca8c0 0x7f35a815b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.042+0000 7f35aee87640 1 --2- 172.21.15.40:0/3724570818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35a806bec0 0x7f35a813a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.042+0000 7f35aee87640 1 -- 172.21.15.40:0/3724570818 >> 172.21.15.40:0/3724570818 conn(0x7f35a80af530 msgr2=0x7f35a80ae6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.042+0000 7f35aee87640 1 -- 172.21.15.40:0/3724570818 shutdown_connections 2023-10-18T09:01:42.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.043+0000 7f35aee87640 1 -- 172.21.15.40:0/3724570818 wait complete. 2023-10-18T09:01:42.067 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2023-10-18T09:01:41.035449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5718,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556781,"statfs":{"total":1600000000,"available":1594144229,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.524102"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868397+0000","last_change":"2023-10-18T09:01:37.552785+0000","last_active":"2023-10-18T09:01:38.868397+0000","last_peered":"2023-10-18T09:01:38.868397+0000","last_clean":"2023-10-18T09:01:38.868397+0000","last_became_active":"2023-10-18T09:01:37.551859+0000","last_became_peered":"2023-10-18T09:01:37.551859+0000","last_unstale":"2023-10-18T09:01:38.868397+0000","last_undegraded":"2023-10-18T09:01:38.868397+0000","last_fullsized":"2023-10-18T09:01:38.868397+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:01:51.628601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524125+0000","last_change":"2023-10-18T09:01:37.545250+0000","last_active":"2023-10-18T09:01:38.524125+0000","last_peered":"2023-10-18T09:01:38.524125+0000","last_clean":"2023-10-18T09:01:38.524125+0000","last_became_active":"2023-10-18T09:01:37.544495+0000","last_became_peered":"2023-10-18T09:01:37.544495+0000","last_unstale":"2023-10-18T09:01:38.524125+0000","last_undegraded":"2023-10-18T09:01:38.524125+0000","last_fullsized":"2023-10-18T09:01:38.524125+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:19:47.931295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524363+0000","last_change":"2023-10-18T09:01:37.547103+0000","last_active":"2023-10-18T09:01:38.524363+0000","last_peered":"2023-10-18T09:01:38.524363+0000","last_clean":"2023-10-18T09:01:38.524363+0000","last_became_active":"2023-10-18T09:01:37.546795+0000","last_became_peered":"2023-10-18T09:01:37.546795+0000","last_unstale":"2023-10-18T09:01:38.524363+0000","last_undegraded":"2023-10-18T09:01:38.524363+0000","last_fullsized":"2023-10-18T09:01:38.524363+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:07:24.109273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865399+0000","last_change":"2023-10-18T09:01:37.558780+0000","last_active":"2023-10-18T09:01:38.865399+0000","last_peered":"2023-10-18T09:01:38.865399+0000","last_clean":"2023-10-18T09:01:38.865399+0000","last_became_active":"2023-10-18T09:01:37.557141+0000","last_became_peered":"2023-10-18T09:01:37.557141+0000","last_unstale":"2023-10-18T09:01:38.865399+0000","last_undegraded":"2023-10-18T09:01:38.865399+0000","last_fullsized":"2023-10-18T09:01:38.865399+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:44:40.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868999+0000","last_change":"2023-10-18T09:01:37.553214+0000","last_active":"2023-10-18T09:01:38.868999+0000","last_peered":"2023-10-18T09:01:38.868999+0000","last_clean":"2023-10-18T09:01:38.868999+0000","last_became_active":"2023-10-18T09:01:37.553004+0000","last_became_peered":"2023-10-18T09:01:37.553004+0000","last_unstale":"2023-10-18T09:01:38.868999+0000","last_undegraded":"2023-10-18T09:01:38.868999+0000","last_fullsized":"2023-10-18T09:01:38.868999+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:47.390225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867364+0000","last_change":"2023-10-18T09:01:37.548740+0000","last_active":"2023-10-18T09:01:38.867364+0000","last_peered":"2023-10-18T09:01:38.867364+0000","last_clean":"2023-10-18T09:01:38.867364+0000","last_became_active":"2023-10-18T09:01:37.548435+0000","last_became_peered":"2023-10-18T09:01:37.548435+0000","last_unstale":"2023-10-18T09:01:38.867364+0000","last_undegraded":"2023-10-18T09:01:38.867364+0000","last_fullsized":"2023-10-18T09:01:38.867364+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:36:16.799678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866108+0000","last_change":"2023-10-18T09:01:37.545636+0000","last_active":"2023-10-18T09:01:38.866108+0000","last_peered":"2023-10-18T09:01:38.866108+0000","last_clean":"2023-10-18T09:01:38.866108+0000","last_became_active":"2023-10-18T09:01:37.542287+0000","last_became_peered":"2023-10-18T09:01:37.542287+0000","last_unstale":"2023-10-18T09:01:38.866108+0000","last_undegraded":"2023-10-18T09:01:38.866108+0000","last_fullsized":"2023-10-18T09:01:38.866108+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:14:49.996096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865833+0000","last_change":"2023-10-18T09:01:37.552855+0000","last_active":"2023-10-18T09:01:38.865833+0000","last_peered":"2023-10-18T09:01:38.865833+0000","last_clean":"2023-10-18T09:01:38.865833+0000","last_became_active":"2023-10-18T09:01:37.552315+0000","last_became_peered":"2023-10-18T09:01:37.552315+0000","last_unstale":"2023-10-18T09:01:38.865833+0000","last_undegraded":"2023-10-18T09:01:38.865833+0000","last_fullsized":"2023-10-18T09:01:38.865833+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:07:42.731036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868149+0000","last_change":"2023-10-18T09:01:37.548169+0000","last_active":"2023-10-18T09:01:38.868149+0000","last_peered":"2023-10-18T09:01:38.868149+0000","last_clean":"2023-10-18T09:01:38.868149+0000","last_became_active":"2023-10-18T09:01:37.547901+0000","last_became_peered":"2023-10-18T09:01:37.547901+0000","last_unstale":"2023-10-18T09:01:38.868149+0000","last_undegraded":"2023-10-18T09:01:38.868149+0000","last_fullsized":"2023-10-18T09:01:38.868149+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:56:06.961383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868510+0000","last_change":"2023-10-18T09:01:37.545882+0000","last_active":"2023-10-18T09:01:38.868510+0000","last_peered":"2023-10-18T09:01:38.868510+0000","last_clean":"2023-10-18T09:01:38.868510+0000","last_became_active":"2023-10-18T09:01:37.545324+0000","last_became_peered":"2023-10-18T09:01:37.545324+0000","last_unstale":"2023-10-18T09:01:38.868510+0000","last_undegraded":"2023-10-18T09:01:38.868510+0000","last_fullsized":"2023-10-18T09:01:38.868510+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:34.616762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524394+0000","last_change":"2023-10-18T09:01:37.530225+0000","last_active":"2023-10-18T09:01:38.524394+0000","last_peered":"2023-10-18T09:01:38.524394+0000","last_clean":"2023-10-18T09:01:38.524394+0000","last_became_active":"2023-10-18T09:01:37.529247+0000","last_became_peered":"2023-10-18T09:01:37.529247+0000","last_unstale":"2023-10-18T09:01:38.524394+0000","last_undegraded":"2023-10-18T09:01:38.524394+0000","last_fullsized":"2023-10-18T09:01:38.524394+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:56:49.895421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866495+0000","last_change":"2023-10-18T09:01:37.544915+0000","last_active":"2023-10-18T09:01:38.866495+0000","last_peered":"2023-10-18T09:01:38.866495+0000","last_clean":"2023-10-18T09:01:38.866495+0000","last_became_active":"2023-10-18T09:01:37.544688+0000","last_became_peered":"2023-10-18T09:01:37.544688+0000","last_unstale":"2023-10-18T09:01:38.866495+0000","last_undegraded":"2023-10-18T09:01:38.866495+0000","last_fullsized":"2023-10-18T09:01:38.866495+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:29.195475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524740+0000","last_change":"2023-10-18T09:01:37.544219+0000","last_active":"2023-10-18T09:01:38.524740+0000","last_peered":"2023-10-18T09:01:38.524740+0000","last_clean":"2023-10-18T09:01:38.524740+0000","last_became_active":"2023-10-18T09:01:37.543994+0000","last_became_peered":"2023-10-18T09:01:37.543994+0000","last_unstale":"2023-10-18T09:01:38.524740+0000","last_undegraded":"2023-10-18T09:01:38.524740+0000","last_fullsized":"2023-10-18T09:01:38.524740+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:34:37.336226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865538+0000","last_change":"2023-10-18T09:01:37.558598+0000","last_active":"2023-10-18T09:01:38.865538+0000","last_peered":"2023-10-18T09:01:38.865538+0000","last_clean":"2023-10-18T09:01:38.865538+0000","last_became_active":"2023-10-18T09:01:37.556828+0000","last_became_peered":"2023-10-18T09:01:37.556828+0000","last_unstale":"2023-10-18T09:01:38.865538+0000","last_undegraded":"2023-10-18T09:01:38.865538+0000","last_fullsized":"2023-10-18T09:01:38.865538+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:42:06.726482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.525418+0000","last_change":"2023-10-18T09:01:37.533631+0000","last_active":"2023-10-18T09:01:38.525418+0000","last_peered":"2023-10-18T09:01:38.525418+0000","last_clean":"2023-10-18T09:01:38.525418+0000","last_became_active":"2023-10-18T09:01:37.532137+0000","last_became_peered":"2023-10-18T09:01:37.532137+0000","last_unstale":"2023-10-18T09:01:38.525418+0000","last_undegraded":"2023-10-18T09:01:38.525418+0000","last_fullsized":"2023-10-18T09:01:38.525418+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:46.732096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865882+0000","last_change":"2023-10-18T09:01:37.551281+0000","last_active":"2023-10-18T09:01:38.865882+0000","last_peered":"2023-10-18T09:01:38.865882+0000","last_clean":"2023-10-18T09:01:38.865882+0000","last_became_active":"2023-10-18T09:01:37.548874+0000","last_became_peered":"2023-10-18T09:01:37.548874+0000","last_unstale":"2023-10-18T09:01:38.865882+0000","last_undegraded":"2023-10-18T09:01:38.865882+0000","last_fullsized":"2023-10-18T09:01:38.865882+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:27.053204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866477+0000","last_change":"2023-10-18T09:01:37.548422+0000","last_active":"2023-10-18T09:01:38.866477+0000","last_peered":"2023-10-18T09:01:38.866477+0000","last_clean":"2023-10-18T09:01:38.866477+0000","last_became_active":"2023-10-18T09:01:37.548207+0000","last_became_peered":"2023-10-18T09:01:37.548207+0000","last_unstale":"2023-10-18T09:01:38.866477+0000","last_undegraded":"2023-10-18T09:01:38.866477+0000","last_fullsized":"2023-10-18T09:01:38.866477+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:54:34.252765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868615+0000","last_change":"2023-10-18T09:01:37.559103+0000","last_active":"2023-10-18T09:01:38.868615+0000","last_peered":"2023-10-18T09:01:38.868615+0000","last_clean":"2023-10-18T09:01:38.868615+0000","last_became_active":"2023-10-18T09:01:37.558024+0000","last_became_peered":"2023-10-18T09:01:37.558024+0000","last_unstale":"2023-10-18T09:01:38.868615+0000","last_undegraded":"2023-10-18T09:01:38.868615+0000","last_fullsized":"2023-10-18T09:01:38.868615+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:04:39.546838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865554+0000","last_change":"2023-10-18T09:01:37.545473+0000","last_active":"2023-10-18T09:01:38.865554+0000","last_peered":"2023-10-18T09:01:38.865554+0000","last_clean":"2023-10-18T09:01:38.865554+0000","last_became_active":"2023-10-18T09:01:37.544854+0000","last_became_peered":"2023-10-18T09:01:37.544854+0000","last_unstale":"2023-10-18T09:01:38.865554+0000","last_undegraded":"2023-10-18T09:01:38.865554+0000","last_fullsized":"2023-10-18T09:01:38.865554+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:34.708210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866365+0000","last_change":"2023-10-18T09:01:37.547736+0000","last_active":"2023-10-18T09:01:38.866365+0000","last_peered":"2023-10-18T09:01:38.866365+0000","last_clean":"2023-10-18T09:01:38.866365+0000","last_became_active":"2023-10-18T09:01:37.545779+0000","last_became_peered":"2023-10-18T09:01:37.545779+0000","last_unstale":"2023-10-18T09:01:38.866365+0000","last_undegraded":"2023-10-18T09:01:38.866365+0000","last_fullsized":"2023-10-18T09:01:38.866365+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:18:14.443467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866707+0000","last_change":"2023-10-18T09:01:37.556332+0000","last_active":"2023-10-18T09:01:38.866707+0000","last_peered":"2023-10-18T09:01:38.866707+0000","last_clean":"2023-10-18T09:01:38.866707+0000","last_became_active":"2023-10-18T09:01:37.555835+0000","last_became_peered":"2023-10-18T09:01:37.555835+0000","last_unstale":"2023-10-18T09:01:38.866707+0000","last_undegraded":"2023-10-18T09:01:38.866707+0000","last_fullsized":"2023-10-18T09:01:38.866707+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:34:30.100471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524298+0000","last_change":"2023-10-18T09:01:37.550010+0000","last_active":"2023-10-18T09:01:38.524298+0000","last_peered":"2023-10-18T09:01:38.524298+0000","last_clean":"2023-10-18T09:01:38.524298+0000","last_became_active":"2023-10-18T09:01:37.549749+0000","last_became_peered":"2023-10-18T09:01:37.549749+0000","last_unstale":"2023-10-18T09:01:38.524298+0000","last_undegraded":"2023-10-18T09:01:38.524298+0000","last_fullsized":"2023-10-18T09:01:38.524298+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:46.999487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867427+0000","last_change":"2023-10-18T09:01:37.554476+0000","last_active":"2023-10-18T09:01:38.867427+0000","last_peered":"2023-10-18T09:01:38.867427+0000","last_clean":"2023-10-18T09:01:38.867427+0000","last_became_active":"2023-10-18T09:01:37.553393+0000","last_became_peered":"2023-10-18T09:01:37.553393+0000","last_unstale":"2023-10-18T09:01:38.867427+0000","last_undegraded":"2023-10-18T09:01:38.867427+0000","last_fullsized":"2023-10-18T09:01:38.867427+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:39.025489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865688+0000","last_change":"2023-10-18T09:01:37.545951+0000","last_active":"2023-10-18T09:01:38.865688+0000","last_peered":"2023-10-18T09:01:38.865688+0000","last_clean":"2023-10-18T09:01:38.865688+0000","last_became_active":"2023-10-18T09:01:37.545721+0000","last_became_peered":"2023-10-18T09:01:37.545721+0000","last_unstale":"2023-10-18T09:01:38.865688+0000","last_undegraded":"2023-10-18T09:01:38.865688+0000","last_fullsized":"2023-10-18T09:01:38.865688+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:18:51.055185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524105+0000","last_change":"2023-10-18T09:01:37.541983+0000","last_active":"2023-10-18T09:01:38.524105+0000","last_peered":"2023-10-18T09:01:38.524105+0000","last_clean":"2023-10-18T09:01:38.524105+0000","last_became_active":"2023-10-18T09:01:37.541725+0000","last_became_peered":"2023-10-18T09:01:37.541725+0000","last_unstale":"2023-10-18T09:01:38.524105+0000","last_undegraded":"2023-10-18T09:01:38.524105+0000","last_fullsized":"2023-10-18T09:01:38.524105+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:49.997650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524685+0000","last_change":"2023-10-18T09:01:37.531929+0000","last_active":"2023-10-18T09:01:38.524685+0000","last_peered":"2023-10-18T09:01:38.524685+0000","last_clean":"2023-10-18T09:01:38.524685+0000","last_became_active":"2023-10-18T09:01:37.531684+0000","last_became_peered":"2023-10-18T09:01:37.531684+0000","last_unstale":"2023-10-18T09:01:38.524685+0000","last_undegraded":"2023-10-18T09:01:38.524685+0000","last_fullsized":"2023-10-18T09:01:38.524685+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:54.562735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867701+0000","last_change":"2023-10-18T09:01:37.543073+0000","last_active":"2023-10-18T09:01:38.867701+0000","last_peered":"2023-10-18T09:01:38.867701+0000","last_clean":"2023-10-18T09:01:38.867701+0000","last_became_active":"2023-10-18T09:01:37.542040+0000","last_became_peered":"2023-10-18T09:01:37.542040+0000","last_unstale":"2023-10-18T09:01:38.867701+0000","last_undegraded":"2023-10-18T09:01:38.867701+0000","last_fullsized":"2023-10-18T09:01:38.867701+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:34.616762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867754+0000","last_change":"2023-10-18T09:01:37.554245+0000","last_active":"2023-10-18T09:01:38.867754+0000","last_peered":"2023-10-18T09:01:38.867754+0000","last_clean":"2023-10-18T09:01:38.867754+0000","last_became_active":"2023-10-18T09:01:37.550345+0000","last_became_peered":"2023-10-18T09:01:37.550345+0000","last_unstale":"2023-10-18T09:01:38.867754+0000","last_undegraded":"2023-10-18T09:01:38.867754+0000","last_fullsized":"2023-10-18T09:01:38.867754+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:54:34.252765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524187+0000","last_change":"2023-10-18T09:01:37.540038+0000","last_active":"2023-10-18T09:01:38.524187+0000","last_peered":"2023-10-18T09:01:38.524187+0000","last_clean":"2023-10-18T09:01:38.524187+0000","last_became_active":"2023-10-18T09:01:37.539791+0000","last_became_peered":"2023-10-18T09:01:37.539791+0000","last_unstale":"2023-10-18T09:01:38.524187+0000","last_undegraded":"2023-10-18T09:01:38.524187+0000","last_fullsized":"2023-10-18T09:01:38.524187+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:28:43.093717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524294+0000","last_change":"2023-10-18T09:01:37.546371+0000","last_active":"2023-10-18T09:01:38.524294+0000","last_peered":"2023-10-18T09:01:38.524294+0000","last_clean":"2023-10-18T09:01:38.524294+0000","last_became_active":"2023-10-18T09:01:37.544888+0000","last_became_peered":"2023-10-18T09:01:37.544888+0000","last_unstale":"2023-10-18T09:01:38.524294+0000","last_undegraded":"2023-10-18T09:01:38.524294+0000","last_fullsized":"2023-10-18T09:01:38.524294+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:46:32.086246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866400+0000","last_change":"2023-10-18T09:01:37.551724+0000","last_active":"2023-10-18T09:01:38.866400+0000","last_peered":"2023-10-18T09:01:38.866400+0000","last_clean":"2023-10-18T09:01:38.866400+0000","last_became_active":"2023-10-18T09:01:37.551031+0000","last_became_peered":"2023-10-18T09:01:37.551031+0000","last_unstale":"2023-10-18T09:01:38.866400+0000","last_undegraded":"2023-10-18T09:01:38.866400+0000","last_fullsized":"2023-10-18T09:01:38.866400+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:39.025489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866862+0000","last_change":"2023-10-18T09:01:37.551638+0000","last_active":"2023-10-18T09:01:38.866862+0000","last_peered":"2023-10-18T09:01:38.866862+0000","last_clean":"2023-10-18T09:01:38.866862+0000","last_became_active":"2023-10-18T09:01:37.551394+0000","last_became_peered":"2023-10-18T09:01:37.551394+0000","last_unstale":"2023-10-18T09:01:38.866862+0000","last_undegraded":"2023-10-18T09:01:38.866862+0000","last_fullsized":"2023-10-18T09:01:38.866862+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:08.381428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865005+0000","last_change":"2023-10-18T09:01:37.549886+0000","last_active":"2023-10-18T09:01:38.865005+0000","last_peered":"2023-10-18T09:01:38.865005+0000","last_clean":"2023-10-18T09:01:38.865005+0000","last_became_active":"2023-10-18T09:01:37.549538+0000","last_became_peered":"2023-10-18T09:01:37.549538+0000","last_unstale":"2023-10-18T09:01:38.865005+0000","last_undegraded":"2023-10-18T09:01:38.865005+0000","last_fullsized":"2023-10-18T09:01:38.865005+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:08.843083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"106'4","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868280+0000","last_change":"2023-10-18T09:01:36.541071+0000","last_active":"2023-10-18T09:01:38.868280+0000","last_peered":"2023-10-18T09:01:38.868280+0000","last_clean":"2023-10-18T09:01:38.868280+0000","last_became_active":"2023-10-18T09:01:36.540811+0000","last_became_peered":"2023-10-18T09:01:36.540811+0000","last_unstale":"2023-10-18T09:01:38.868280+0000","last_undegraded":"2023-10-18T09:01:38.868280+0000","last_fullsized":"2023-10-18T09:01:38.868280+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:11.953586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868060+0000","last_change":"2023-10-18T09:01:37.554100+0000","last_active":"2023-10-18T09:01:38.868060+0000","last_peered":"2023-10-18T09:01:38.868060+0000","last_clean":"2023-10-18T09:01:38.868060+0000","last_became_active":"2023-10-18T09:01:37.549932+0000","last_became_peered":"2023-10-18T09:01:37.549932+0000","last_unstale":"2023-10-18T09:01:38.868060+0000","last_undegraded":"2023-10-18T09:01:38.868060+0000","last_fullsized":"2023-10-18T09:01:38.868060+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:09.634600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"106'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867252+0000","last_change":"2023-10-18T09:01:36.543509+0000","last_active":"2023-10-18T09:01:38.867252+0000","last_peered":"2023-10-18T09:01:38.867252+0000","last_clean":"2023-10-18T09:01:38.867252+0000","last_became_active":"2023-10-18T09:01:36.543069+0000","last_became_peered":"2023-10-18T09:01:36.543069+0000","last_unstale":"2023-10-18T09:01:38.867252+0000","last_undegraded":"2023-10-18T09:01:38.867252+0000","last_fullsized":"2023-10-18T09:01:38.867252+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:11:55.265579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868774+0000","last_change":"2023-10-18T09:01:37.548336+0000","last_active":"2023-10-18T09:01:38.868774+0000","last_peered":"2023-10-18T09:01:38.868774+0000","last_clean":"2023-10-18T09:01:38.868774+0000","last_became_active":"2023-10-18T09:01:37.548088+0000","last_became_peered":"2023-10-18T09:01:37.548088+0000","last_unstale":"2023-10-18T09:01:38.868774+0000","last_undegraded":"2023-10-18T09:01:38.868774+0000","last_fullsized":"2023-10-18T09:01:38.868774+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:23.642776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"106'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866222+0000","last_change":"2023-10-18T09:01:36.540493+0000","last_active":"2023-10-18T09:01:38.866222+0000","last_peered":"2023-10-18T09:01:38.866222+0000","last_clean":"2023-10-18T09:01:38.866222+0000","last_became_active":"2023-10-18T09:01:36.540255+0000","last_became_peered":"2023-10-18T09:01:36.540255+0000","last_unstale":"2023-10-18T09:01:38.866222+0000","last_undegraded":"2023-10-18T09:01:38.866222+0000","last_fullsized":"2023-10-18T09:01:38.866222+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:47:16.660316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.525525+0000","last_change":"2023-10-18T09:01:37.541887+0000","last_active":"2023-10-18T09:01:38.525525+0000","last_peered":"2023-10-18T09:01:38.525525+0000","last_clean":"2023-10-18T09:01:38.525525+0000","last_became_active":"2023-10-18T09:01:37.541669+0000","last_became_peered":"2023-10-18T09:01:37.541669+0000","last_unstale":"2023-10-18T09:01:38.525525+0000","last_undegraded":"2023-10-18T09:01:38.525525+0000","last_fullsized":"2023-10-18T09:01:38.525525+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:46.732096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"106'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867811+0000","last_change":"2023-10-18T09:01:36.539895+0000","last_active":"2023-10-18T09:01:38.867811+0000","last_peered":"2023-10-18T09:01:38.867811+0000","last_clean":"2023-10-18T09:01:38.867811+0000","last_became_active":"2023-10-18T09:01:36.539615+0000","last_became_peered":"2023-10-18T09:01:36.539615+0000","last_unstale":"2023-10-18T09:01:38.867811+0000","last_undegraded":"2023-10-18T09:01:38.867811+0000","last_fullsized":"2023-10-18T09:01:38.867811+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:50:35.235441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524960+0000","last_change":"2023-10-18T09:01:37.550215+0000","last_active":"2023-10-18T09:01:38.524960+0000","last_peered":"2023-10-18T09:01:38.524960+0000","last_clean":"2023-10-18T09:01:38.524960+0000","last_became_active":"2023-10-18T09:01:37.547491+0000","last_became_peered":"2023-10-18T09:01:37.547491+0000","last_unstale":"2023-10-18T09:01:38.524960+0000","last_undegraded":"2023-10-18T09:01:38.524960+0000","last_fullsized":"2023-10-18T09:01:38.524960+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:28:16.323058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"106'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867346+0000","last_change":"2023-10-18T09:01:36.543614+0000","last_active":"2023-10-18T09:01:38.867346+0000","last_peered":"2023-10-18T09:01:38.867346+0000","last_clean":"2023-10-18T09:01:38.867346+0000","last_became_active":"2023-10-18T09:01:36.543130+0000","last_became_peered":"2023-10-18T09:01:36.543130+0000","last_unstale":"2023-10-18T09:01:38.867346+0000","last_undegraded":"2023-10-18T09:01:38.867346+0000","last_fullsized":"2023-10-18T09:01:38.867346+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:12:17.149808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866820+0000","last_change":"2023-10-18T09:01:37.547888+0000","last_active":"2023-10-18T09:01:38.866820+0000","last_peered":"2023-10-18T09:01:38.866820+0000","last_clean":"2023-10-18T09:01:38.866820+0000","last_became_active":"2023-10-18T09:01:37.547666+0000","last_became_peered":"2023-10-18T09:01:37.547666+0000","last_unstale":"2023-10-18T09:01:38.866820+0000","last_undegraded":"2023-10-18T09:01:38.866820+0000","last_fullsized":"2023-10-18T09:01:38.866820+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:01:51.628601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"106'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866110+0000","last_change":"2023-10-18T09:01:36.542738+0000","last_active":"2023-10-18T09:01:38.866110+0000","last_peered":"2023-10-18T09:01:38.866110+0000","last_clean":"2023-10-18T09:01:38.866110+0000","last_became_active":"2023-10-18T09:01:36.542450+0000","last_became_peered":"2023-10-18T09:01:36.542450+0000","last_unstale":"2023-10-18T09:01:38.866110+0000","last_undegraded":"2023-10-18T09:01:38.866110+0000","last_fullsized":"2023-10-18T09:01:38.866110+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:11.953586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524847+0000","last_change":"2023-10-18T09:01:37.546633+0000","last_active":"2023-10-18T09:01:38.524847+0000","last_peered":"2023-10-18T09:01:38.524847+0000","last_clean":"2023-10-18T09:01:38.524847+0000","last_became_active":"2023-10-18T09:01:37.546228+0000","last_became_peered":"2023-10-18T09:01:37.546228+0000","last_unstale":"2023-10-18T09:01:38.524847+0000","last_undegraded":"2023-10-18T09:01:38.524847+0000","last_fullsized":"2023-10-18T09:01:38.524847+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:50:03.587917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866317+0000","last_change":"2023-10-18T09:01:36.540596+0000","last_active":"2023-10-18T09:01:38.866317+0000","last_peered":"2023-10-18T09:01:38.866317+0000","last_clean":"2023-10-18T09:01:38.866317+0000","last_became_active":"2023-10-18T09:01:36.540320+0000","last_became_peered":"2023-10-18T09:01:36.540320+0000","last_unstale":"2023-10-18T09:01:38.866317+0000","last_undegraded":"2023-10-18T09:01:38.866317+0000","last_fullsized":"2023-10-18T09:01:38.866317+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:26:01.238341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.525078+0000","last_change":"2023-10-18T09:01:37.535100+0000","last_active":"2023-10-18T09:01:38.525078+0000","last_peered":"2023-10-18T09:01:38.525078+0000","last_clean":"2023-10-18T09:01:38.525078+0000","last_became_active":"2023-10-18T09:01:37.534088+0000","last_became_peered":"2023-10-18T09:01:37.534088+0000","last_unstale":"2023-10-18T09:01:38.525078+0000","last_undegraded":"2023-10-18T09:01:38.525078+0000","last_fullsized":"2023-10-18T09:01:38.525078+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:50:03.587917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524418+0000","last_change":"2023-10-18T09:01:36.543284+0000","last_active":"2023-10-18T09:01:38.524418+0000","last_peered":"2023-10-18T09:01:38.524418+0000","last_clean":"2023-10-18T09:01:38.524418+0000","last_became_active":"2023-10-18T09:01:36.543011+0000","last_became_peered":"2023-10-18T09:01:36.543011+0000","last_unstale":"2023-10-18T09:01:38.524418+0000","last_undegraded":"2023-10-18T09:01:38.524418+0000","last_fullsized":"2023-10-18T09:01:38.524418+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:39:01.836488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.525397+0000","last_change":"2023-10-18T09:01:37.537846+0000","last_active":"2023-10-18T09:01:38.525397+0000","last_peered":"2023-10-18T09:01:38.525397+0000","last_clean":"2023-10-18T09:01:38.525397+0000","last_became_active":"2023-10-18T09:01:37.537285+0000","last_became_peered":"2023-10-18T09:01:37.537285+0000","last_unstale":"2023-10-18T09:01:38.525397+0000","last_undegraded":"2023-10-18T09:01:38.525397+0000","last_fullsized":"2023-10-18T09:01:38.525397+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:54:40.847787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868914+0000","last_change":"2023-10-18T09:01:36.531255+0000","last_active":"2023-10-18T09:01:38.868914+0000","last_peered":"2023-10-18T09:01:38.868914+0000","last_clean":"2023-10-18T09:01:38.868914+0000","last_became_active":"2023-10-18T09:01:36.530793+0000","last_became_peered":"2023-10-18T09:01:36.530793+0000","last_unstale":"2023-10-18T09:01:38.868914+0000","last_undegraded":"2023-10-18T09:01:38.868914+0000","last_fullsized":"2023-10-18T09:01:38.868914+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:07:41.726630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867709+0000","last_change":"2023-10-18T09:01:37.545288+0000","last_active":"2023-10-18T09:01:38.867709+0000","last_peered":"2023-10-18T09:01:38.867709+0000","last_clean":"2023-10-18T09:01:38.867709+0000","last_became_active":"2023-10-18T09:01:37.545069+0000","last_became_peered":"2023-10-18T09:01:37.545069+0000","last_unstale":"2023-10-18T09:01:38.867709+0000","last_undegraded":"2023-10-18T09:01:38.867709+0000","last_fullsized":"2023-10-18T09:01:38.867709+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:16.495132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867088+0000","last_change":"2023-10-18T09:01:36.544296+0000","last_active":"2023-10-18T09:01:38.867088+0000","last_peered":"2023-10-18T09:01:38.867088+0000","last_clean":"2023-10-18T09:01:38.867088+0000","last_became_active":"2023-10-18T09:01:36.543995+0000","last_became_peered":"2023-10-18T09:01:36.543995+0000","last_unstale":"2023-10-18T09:01:38.867088+0000","last_undegraded":"2023-10-18T09:01:38.867088+0000","last_fullsized":"2023-10-18T09:01:38.867088+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:41.862220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865922+0000","last_change":"2023-10-18T09:01:37.547316+0000","last_active":"2023-10-18T09:01:38.865922+0000","last_peered":"2023-10-18T09:01:38.865922+0000","last_clean":"2023-10-18T09:01:38.865922+0000","last_became_active":"2023-10-18T09:01:37.546568+0000","last_became_peered":"2023-10-18T09:01:37.546568+0000","last_unstale":"2023-10-18T09:01:38.865922+0000","last_undegraded":"2023-10-18T09:01:38.865922+0000","last_fullsized":"2023-10-18T09:01:38.865922+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:17.241941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"106'3","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868936+0000","last_change":"2023-10-18T09:01:36.535945+0000","last_active":"2023-10-18T09:01:38.868936+0000","last_peered":"2023-10-18T09:01:38.868936+0000","last_clean":"2023-10-18T09:01:38.868936+0000","last_became_active":"2023-10-18T09:01:36.535683+0000","last_became_peered":"2023-10-18T09:01:36.535683+0000","last_unstale":"2023-10-18T09:01:38.868936+0000","last_undegraded":"2023-10-18T09:01:38.868936+0000","last_fullsized":"2023-10-18T09:01:38.868936+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:07.838678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865499+0000","last_change":"2023-10-18T09:01:37.550449+0000","last_active":"2023-10-18T09:01:38.865499+0000","last_peered":"2023-10-18T09:01:38.865499+0000","last_clean":"2023-10-18T09:01:38.865499+0000","last_became_active":"2023-10-18T09:01:37.550089+0000","last_became_peered":"2023-10-18T09:01:37.550089+0000","last_unstale":"2023-10-18T09:01:38.865499+0000","last_undegraded":"2023-10-18T09:01:38.865499+0000","last_fullsized":"2023-10-18T09:01:38.865499+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:29.195475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"106'6","reported_seq":18,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868528+0000","last_change":"2023-10-18T09:01:36.538146+0000","last_active":"2023-10-18T09:01:38.868528+0000","last_peered":"2023-10-18T09:01:38.868528+0000","last_clean":"2023-10-18T09:01:38.868528+0000","last_became_active":"2023-10-18T09:01:36.537921+0000","last_became_peered":"2023-10-18T09:01:36.537921+0000","last_unstale":"2023-10-18T09:01:38.868528+0000","last_undegraded":"2023-10-18T09:01:38.868528+0000","last_fullsized":"2023-10-18T09:01:38.868528+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:27:14.103486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867256+0000","last_change":"2023-10-18T09:01:37.548459+0000","last_active":"2023-10-18T09:01:38.867256+0000","last_peered":"2023-10-18T09:01:38.867256+0000","last_clean":"2023-10-18T09:01:38.867256+0000","last_became_active":"2023-10-18T09:01:37.547761+0000","last_became_peered":"2023-10-18T09:01:37.547761+0000","last_unstale":"2023-10-18T09:01:38.867256+0000","last_undegraded":"2023-10-18T09:01:38.867256+0000","last_fullsized":"2023-10-18T09:01:38.867256+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:07:24.109273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"106'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524549+0000","last_change":"2023-10-18T09:01:36.529426+0000","last_active":"2023-10-18T09:01:38.524549+0000","last_peered":"2023-10-18T09:01:38.524549+0000","last_clean":"2023-10-18T09:01:38.524549+0000","last_became_active":"2023-10-18T09:01:36.528864+0000","last_became_peered":"2023-10-18T09:01:36.528864+0000","last_unstale":"2023-10-18T09:01:38.524549+0000","last_undegraded":"2023-10-18T09:01:38.524549+0000","last_fullsized":"2023-10-18T09:01:38.524549+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:04:38.542433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.525222+0000","last_change":"2023-10-18T09:01:37.546472+0000","last_active":"2023-10-18T09:01:38.525222+0000","last_peered":"2023-10-18T09:01:38.525222+0000","last_clean":"2023-10-18T09:01:38.525222+0000","last_became_active":"2023-10-18T09:01:37.545891+0000","last_became_peered":"2023-10-18T09:01:37.545891+0000","last_unstale":"2023-10-18T09:01:38.525222+0000","last_undegraded":"2023-10-18T09:01:38.525222+0000","last_fullsized":"2023-10-18T09:01:38.525222+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:54:40.847787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524475+0000","last_change":"2023-10-18T09:01:36.524365+0000","last_active":"2023-10-18T09:01:38.524475+0000","last_peered":"2023-10-18T09:01:38.524475+0000","last_clean":"2023-10-18T09:01:38.524475+0000","last_became_active":"2023-10-18T09:01:36.523666+0000","last_became_peered":"2023-10-18T09:01:36.523666+0000","last_unstale":"2023-10-18T09:01:38.524475+0000","last_undegraded":"2023-10-18T09:01:38.524475+0000","last_fullsized":"2023-10-18T09:01:38.524475+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:53.558330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524783+0000","last_change":"2023-10-18T09:01:37.547821+0000","last_active":"2023-10-18T09:01:38.524783+0000","last_peered":"2023-10-18T09:01:38.524783+0000","last_clean":"2023-10-18T09:01:38.524783+0000","last_became_active":"2023-10-18T09:01:37.547086+0000","last_became_peered":"2023-10-18T09:01:37.547086+0000","last_unstale":"2023-10-18T09:01:38.524783+0000","last_undegraded":"2023-10-18T09:01:38.524783+0000","last_fullsized":"2023-10-18T09:01:38.524783+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:28:43.060432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866434+0000","last_change":"2023-10-18T09:01:36.544368+0000","last_active":"2023-10-18T09:01:38.866434+0000","last_peered":"2023-10-18T09:01:38.866434+0000","last_clean":"2023-10-18T09:01:38.866434+0000","last_became_active":"2023-10-18T09:01:36.544115+0000","last_became_peered":"2023-10-18T09:01:36.544115+0000","last_unstale":"2023-10-18T09:01:38.866434+0000","last_undegraded":"2023-10-18T09:01:38.866434+0000","last_fullsized":"2023-10-18T09:01:38.866434+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:11:55.265579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524476+0000","last_change":"2023-10-18T09:01:37.545077+0000","last_active":"2023-10-18T09:01:38.524476+0000","last_peered":"2023-10-18T09:01:38.524476+0000","last_clean":"2023-10-18T09:01:38.524476+0000","last_became_active":"2023-10-18T09:01:37.544152+0000","last_became_peered":"2023-10-18T09:01:37.544152+0000","last_unstale":"2023-10-18T09:01:38.524476+0000","last_undegraded":"2023-10-18T09:01:38.524476+0000","last_fullsized":"2023-10-18T09:01:38.524476+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:46.837894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"106'8","reported_seq":20,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865186+0000","last_change":"2023-10-18T09:01:36.541137+0000","last_active":"2023-10-18T09:01:38.865186+0000","last_peered":"2023-10-18T09:01:38.865186+0000","last_clean":"2023-10-18T09:01:38.865186+0000","last_became_active":"2023-10-18T09:01:36.540762+0000","last_became_peered":"2023-10-18T09:01:36.540762+0000","last_unstale":"2023-10-18T09:01:38.865186+0000","last_undegraded":"2023-10-18T09:01:38.865186+0000","last_fullsized":"2023-10-18T09:01:38.865186+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:46.385819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.525843+0000","last_change":"2023-10-18T09:01:37.543318+0000","last_active":"2023-10-18T09:01:38.525843+0000","last_peered":"2023-10-18T09:01:38.525843+0000","last_clean":"2023-10-18T09:01:38.525843+0000","last_became_active":"2023-10-18T09:01:37.543015+0000","last_became_peered":"2023-10-18T09:01:37.543015+0000","last_unstale":"2023-10-18T09:01:38.525843+0000","last_undegraded":"2023-10-18T09:01:38.525843+0000","last_fullsized":"2023-10-18T09:01:38.525843+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:40.037987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"106'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867658+0000","last_change":"2023-10-18T09:01:36.541151+0000","last_active":"2023-10-18T09:01:38.867658+0000","last_peered":"2023-10-18T09:01:38.867658+0000","last_clean":"2023-10-18T09:01:38.867658+0000","last_became_active":"2023-10-18T09:01:36.540752+0000","last_became_peered":"2023-10-18T09:01:36.540752+0000","last_unstale":"2023-10-18T09:01:38.867658+0000","last_undegraded":"2023-10-18T09:01:38.867658+0000","last_fullsized":"2023-10-18T09:01:38.867658+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:54:21.788547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867412+0000","last_change":"2023-10-18T09:01:37.545042+0000","last_active":"2023-10-18T09:01:38.867412+0000","last_peered":"2023-10-18T09:01:38.867412+0000","last_clean":"2023-10-18T09:01:38.867412+0000","last_became_active":"2023-10-18T09:01:37.544672+0000","last_became_peered":"2023-10-18T09:01:37.544672+0000","last_unstale":"2023-10-18T09:01:38.867412+0000","last_undegraded":"2023-10-18T09:01:38.867412+0000","last_fullsized":"2023-10-18T09:01:38.867412+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:27.053204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"106'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866468+0000","last_change":"2023-10-18T09:01:36.543711+0000","last_active":"2023-10-18T09:01:38.866468+0000","last_peered":"2023-10-18T09:01:38.866468+0000","last_clean":"2023-10-18T09:01:38.866468+0000","last_became_active":"2023-10-18T09:01:36.543476+0000","last_became_peered":"2023-10-18T09:01:36.543476+0000","last_unstale":"2023-10-18T09:01:38.866468+0000","last_undegraded":"2023-10-18T09:01:38.866468+0000","last_fullsized":"2023-10-18T09:01:38.866468+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:56:41.458555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866767+0000","last_change":"2023-10-18T09:01:37.551901+0000","last_active":"2023-10-18T09:01:38.866767+0000","last_peered":"2023-10-18T09:01:38.866767+0000","last_clean":"2023-10-18T09:01:38.866767+0000","last_became_active":"2023-10-18T09:01:37.551513+0000","last_became_peered":"2023-10-18T09:01:37.551513+0000","last_unstale":"2023-10-18T09:01:38.866767+0000","last_undegraded":"2023-10-18T09:01:38.866767+0000","last_fullsized":"2023-10-18T09:01:38.866767+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:09.634600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865210+0000","last_change":"2023-10-18T09:01:36.535030+0000","last_active":"2023-10-18T09:01:38.865210+0000","last_peered":"2023-10-18T09:01:38.865210+0000","last_clean":"2023-10-18T09:01:38.865210+0000","last_became_active":"2023-10-18T09:01:36.534718+0000","last_became_peered":"2023-10-18T09:01:36.534718+0000","last_unstale":"2023-10-18T09:01:38.865210+0000","last_undegraded":"2023-10-18T09:01:38.865210+0000","last_fullsized":"2023-10-18T09:01:38.865210+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:22.638371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865818+0000","last_change":"2023-10-18T09:01:37.557396+0000","last_active":"2023-10-18T09:01:38.865818+0000","last_peered":"2023-10-18T09:01:38.865818+0000","last_clean":"2023-10-18T09:01:38.865818+0000","last_became_active":"2023-10-18T09:01:37.556462+0000","last_became_peered":"2023-10-18T09:01:37.556462+0000","last_unstale":"2023-10-18T09:01:38.865818+0000","last_undegraded":"2023-10-18T09:01:38.865818+0000","last_fullsized":"2023-10-18T09:01:38.865818+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:12:22.654018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"106'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524263+0000","last_change":"2023-10-18T09:01:36.543114+0000","last_active":"2023-10-18T09:01:38.524263+0000","last_peered":"2023-10-18T09:01:38.524263+0000","last_clean":"2023-10-18T09:01:38.524263+0000","last_became_active":"2023-10-18T09:01:36.542876+0000","last_became_peered":"2023-10-18T09:01:36.542876+0000","last_unstale":"2023-10-18T09:01:38.524263+0000","last_undegraded":"2023-10-18T09:01:38.524263+0000","last_fullsized":"2023-10-18T09:01:38.524263+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:13.995601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866123+0000","last_change":"2023-10-18T09:01:37.552671+0000","last_active":"2023-10-18T09:01:38.866123+0000","last_peered":"2023-10-18T09:01:38.866123+0000","last_clean":"2023-10-18T09:01:38.866123+0000","last_became_active":"2023-10-18T09:01:37.551974+0000","last_became_peered":"2023-10-18T09:01:37.551974+0000","last_unstale":"2023-10-18T09:01:38.866123+0000","last_undegraded":"2023-10-18T09:01:38.866123+0000","last_fullsized":"2023-10-18T09:01:38.866123+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:38:36.085141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865976+0000","last_change":"2023-10-18T09:01:36.537605+0000","last_active":"2023-10-18T09:01:38.865976+0000","last_peered":"2023-10-18T09:01:38.865976+0000","last_clean":"2023-10-18T09:01:38.865976+0000","last_became_active":"2023-10-18T09:01:36.537190+0000","last_became_peered":"2023-10-18T09:01:36.537190+0000","last_unstale":"2023-10-18T09:01:38.865976+0000","last_undegraded":"2023-10-18T09:01:38.865976+0000","last_fullsized":"2023-10-18T09:01:38.865976+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:02.527895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867030+0000","last_change":"2023-10-18T09:01:37.551826+0000","last_active":"2023-10-18T09:01:38.867030+0000","last_peered":"2023-10-18T09:01:38.867030+0000","last_clean":"2023-10-18T09:01:38.867030+0000","last_became_active":"2023-10-18T09:01:37.548572+0000","last_became_peered":"2023-10-18T09:01:37.548572+0000","last_unstale":"2023-10-18T09:01:38.867030+0000","last_undegraded":"2023-10-18T09:01:38.867030+0000","last_fullsized":"2023-10-18T09:01:38.867030+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:39:02.840893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866743+0000","last_change":"2023-10-18T09:01:36.537476+0000","last_active":"2023-10-18T09:01:38.866743+0000","last_peered":"2023-10-18T09:01:38.866743+0000","last_clean":"2023-10-18T09:01:38.866743+0000","last_became_active":"2023-10-18T09:01:36.537161+0000","last_became_peered":"2023-10-18T09:01:36.537161+0000","last_unstale":"2023-10-18T09:01:38.866743+0000","last_undegraded":"2023-10-18T09:01:38.866743+0000","last_fullsized":"2023-10-18T09:01:38.866743+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:05:17.734155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866645+0000","last_change":"2023-10-18T09:01:37.546947+0000","last_active":"2023-10-18T09:01:38.866645+0000","last_peered":"2023-10-18T09:01:38.866645+0000","last_clean":"2023-10-18T09:01:38.866645+0000","last_became_active":"2023-10-18T09:01:37.546020+0000","last_became_peered":"2023-10-18T09:01:37.546020+0000","last_unstale":"2023-10-18T09:01:38.866645+0000","last_undegraded":"2023-10-18T09:01:38.866645+0000","last_fullsized":"2023-10-18T09:01:38.866645+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:12:18.154213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524611+0000","last_change":"2023-10-18T09:01:36.540049+0000","last_active":"2023-10-18T09:01:38.524611+0000","last_peered":"2023-10-18T09:01:38.524611+0000","last_clean":"2023-10-18T09:01:38.524611+0000","last_became_active":"2023-10-18T09:01:36.538963+0000","last_became_peered":"2023-10-18T09:01:36.538963+0000","last_unstale":"2023-10-18T09:01:38.524611+0000","last_undegraded":"2023-10-18T09:01:38.524611+0000","last_fullsized":"2023-10-18T09:01:38.524611+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:34:36.331821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865868+0000","last_change":"2023-10-18T09:01:37.550302+0000","last_active":"2023-10-18T09:01:38.865868+0000","last_peered":"2023-10-18T09:01:38.865868+0000","last_clean":"2023-10-18T09:01:38.865868+0000","last_became_active":"2023-10-18T09:01:37.549458+0000","last_became_peered":"2023-10-18T09:01:37.549458+0000","last_unstale":"2023-10-18T09:01:38.865868+0000","last_undegraded":"2023-10-18T09:01:38.865868+0000","last_fullsized":"2023-10-18T09:01:38.865868+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:08.381428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"106'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524906+0000","last_change":"2023-10-18T09:01:36.542371+0000","last_active":"2023-10-18T09:01:38.524906+0000","last_peered":"2023-10-18T09:01:38.524906+0000","last_clean":"2023-10-18T09:01:38.524906+0000","last_became_active":"2023-10-18T09:01:36.542052+0000","last_became_peered":"2023-10-18T09:01:36.542052+0000","last_unstale":"2023-10-18T09:01:38.524906+0000","last_undegraded":"2023-10-18T09:01:38.524906+0000","last_fullsized":"2023-10-18T09:01:38.524906+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:04:54.914558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524596+0000","last_change":"2023-10-18T09:01:37.541636+0000","last_active":"2023-10-18T09:01:38.524596+0000","last_peered":"2023-10-18T09:01:38.524596+0000","last_clean":"2023-10-18T09:01:38.524596+0000","last_became_active":"2023-10-18T09:01:37.541280+0000","last_became_peered":"2023-10-18T09:01:37.541280+0000","last_unstale":"2023-10-18T09:01:38.524596+0000","last_undegraded":"2023-10-18T09:01:38.524596+0000","last_fullsized":"2023-10-18T09:01:38.524596+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:15.000006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.525078+0000","last_change":"2023-10-18T09:01:36.530909+0000","last_active":"2023-10-18T09:01:38.525078+0000","last_peered":"2023-10-18T09:01:38.525078+0000","last_clean":"2023-10-18T09:01:38.525078+0000","last_became_active":"2023-10-18T09:01:36.530299+0000","last_became_peered":"2023-10-18T09:01:36.530299+0000","last_unstale":"2023-10-18T09:01:38.525078+0000","last_undegraded":"2023-10-18T09:01:38.525078+0000","last_fullsized":"2023-10-18T09:01:38.525078+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:51:09.996176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868903+0000","last_change":"2023-10-18T09:01:37.547142+0000","last_active":"2023-10-18T09:01:38.868903+0000","last_peered":"2023-10-18T09:01:38.868903+0000","last_clean":"2023-10-18T09:01:38.868903+0000","last_became_active":"2023-10-18T09:01:37.546805+0000","last_became_peered":"2023-10-18T09:01:37.546805+0000","last_unstale":"2023-10-18T09:01:38.868903+0000","last_undegraded":"2023-10-18T09:01:38.868903+0000","last_fullsized":"2023-10-18T09:01:38.868903+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:02:48.470528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867868+0000","last_change":"2023-10-18T09:01:36.540982+0000","last_active":"2023-10-18T09:01:38.867868+0000","last_peered":"2023-10-18T09:01:38.867868+0000","last_clean":"2023-10-18T09:01:38.867868+0000","last_became_active":"2023-10-18T09:01:36.539000+0000","last_became_peered":"2023-10-18T09:01:36.539000+0000","last_unstale":"2023-10-18T09:01:38.867868+0000","last_undegraded":"2023-10-18T09:01:38.867868+0000","last_fullsized":"2023-10-18T09:01:38.867868+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:39:44.899385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865375+0000","last_change":"2023-10-18T09:01:37.549748+0000","last_active":"2023-10-18T09:01:38.865375+0000","last_peered":"2023-10-18T09:01:38.865375+0000","last_clean":"2023-10-18T09:01:38.865375+0000","last_became_active":"2023-10-18T09:01:37.549202+0000","last_became_peered":"2023-10-18T09:01:37.549202+0000","last_unstale":"2023-10-18T09:01:38.865375+0000","last_undegraded":"2023-10-18T09:01:38.865375+0000","last_fullsized":"2023-10-18T09:01:38.865375+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:35:47.510705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.869040+0000","last_change":"2023-10-18T09:01:36.533876+0000","last_active":"2023-10-18T09:01:38.869040+0000","last_peered":"2023-10-18T09:01:38.869040+0000","last_clean":"2023-10-18T09:01:38.869040+0000","last_became_active":"2023-10-18T09:01:36.533485+0000","last_became_peered":"2023-10-18T09:01:36.533485+0000","last_unstale":"2023-10-18T09:01:38.869040+0000","last_undegraded":"2023-10-18T09:01:38.869040+0000","last_fullsized":"2023-10-18T09:01:38.869040+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:56:05.956978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866084+0000","last_change":"2023-10-18T09:01:37.549720+0000","last_active":"2023-10-18T09:01:38.866084+0000","last_peered":"2023-10-18T09:01:38.866084+0000","last_clean":"2023-10-18T09:01:38.866084+0000","last_became_active":"2023-10-18T09:01:37.549372+0000","last_became_peered":"2023-10-18T09:01:37.549372+0000","last_unstale":"2023-10-18T09:01:38.866084+0000","last_undegraded":"2023-10-18T09:01:38.866084+0000","last_fullsized":"2023-10-18T09:01:38.866084+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:15:16.985894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868017+0000","last_change":"2023-10-18T09:01:36.542502+0000","last_active":"2023-10-18T09:01:38.868017+0000","last_peered":"2023-10-18T09:01:38.868017+0000","last_clean":"2023-10-18T09:01:38.868017+0000","last_became_active":"2023-10-18T09:01:36.542275+0000","last_became_peered":"2023-10-18T09:01:36.542275+0000","last_unstale":"2023-10-18T09:01:38.868017+0000","last_undegraded":"2023-10-18T09:01:38.868017+0000","last_fullsized":"2023-10-18T09:01:38.868017+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:46:53.225811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867560+0000","last_change":"2023-10-18T09:01:37.548270+0000","last_active":"2023-10-18T09:01:38.867560+0000","last_peered":"2023-10-18T09:01:38.867560+0000","last_clean":"2023-10-18T09:01:38.867560+0000","last_became_active":"2023-10-18T09:01:37.547247+0000","last_became_peered":"2023-10-18T09:01:37.547247+0000","last_unstale":"2023-10-18T09:01:38.867560+0000","last_undegraded":"2023-10-18T09:01:38.867560+0000","last_fullsized":"2023-10-18T09:01:38.867560+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:15.000006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867906+0000","last_change":"2023-10-18T09:01:36.543486+0000","last_active":"2023-10-18T09:01:38.867906+0000","last_peered":"2023-10-18T09:01:38.867906+0000","last_clean":"2023-10-18T09:01:38.867906+0000","last_became_active":"2023-10-18T09:01:36.542827+0000","last_became_peered":"2023-10-18T09:01:36.542827+0000","last_unstale":"2023-10-18T09:01:38.867906+0000","last_undegraded":"2023-10-18T09:01:38.867906+0000","last_fullsized":"2023-10-18T09:01:38.867906+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:41.862220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524675+0000","last_change":"2023-10-18T09:01:37.527193+0000","last_active":"2023-10-18T09:01:38.524675+0000","last_peered":"2023-10-18T09:01:38.524675+0000","last_clean":"2023-10-18T09:01:38.524675+0000","last_became_active":"2023-10-18T09:01:37.526570+0000","last_became_peered":"2023-10-18T09:01:37.526570+0000","last_unstale":"2023-10-18T09:01:38.524675+0000","last_undegraded":"2023-10-18T09:01:38.524675+0000","last_fullsized":"2023-10-18T09:01:38.524675+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:57.229192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.869163+0000","last_change":"2023-10-18T09:01:36.531999+0000","last_active":"2023-10-18T09:01:38.869163+0000","last_peered":"2023-10-18T09:01:38.869163+0000","last_clean":"2023-10-18T09:01:38.869163+0000","last_became_active":"2023-10-18T09:01:36.531378+0000","last_became_peered":"2023-10-18T09:01:36.531378+0000","last_unstale":"2023-10-18T09:01:38.869163+0000","last_undegraded":"2023-10-18T09:01:38.869163+0000","last_fullsized":"2023-10-18T09:01:38.869163+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:35:46.506299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868685+0000","last_change":"2023-10-18T09:01:37.553633+0000","last_active":"2023-10-18T09:01:38.868685+0000","last_peered":"2023-10-18T09:01:38.868685+0000","last_clean":"2023-10-18T09:01:38.868685+0000","last_became_active":"2023-10-18T09:01:37.549651+0000","last_became_peered":"2023-10-18T09:01:37.549651+0000","last_unstale":"2023-10-18T09:01:38.868685+0000","last_undegraded":"2023-10-18T09:01:38.868685+0000","last_fullsized":"2023-10-18T09:01:38.868685+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:47.390225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867501+0000","last_change":"2023-10-18T09:01:36.543327+0000","last_active":"2023-10-18T09:01:38.867501+0000","last_peered":"2023-10-18T09:01:38.867501+0000","last_clean":"2023-10-18T09:01:38.867501+0000","last_became_active":"2023-10-18T09:01:36.542413+0000","last_became_peered":"2023-10-18T09:01:36.542413+0000","last_unstale":"2023-10-18T09:01:38.867501+0000","last_undegraded":"2023-10-18T09:01:38.867501+0000","last_fullsized":"2023-10-18T09:01:38.867501+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:15:15.981489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524782+0000","last_change":"2023-10-18T09:01:37.545392+0000","last_active":"2023-10-18T09:01:38.524782+0000","last_peered":"2023-10-18T09:01:38.524782+0000","last_clean":"2023-10-18T09:01:38.524782+0000","last_became_active":"2023-10-18T09:01:37.544830+0000","last_became_peered":"2023-10-18T09:01:37.544830+0000","last_unstale":"2023-10-18T09:01:38.524782+0000","last_undegraded":"2023-10-18T09:01:38.524782+0000","last_fullsized":"2023-10-18T09:01:38.524782+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:54:24.998405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"106'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.525082+0000","last_change":"2023-10-18T09:01:36.544488+0000","last_active":"2023-10-18T09:01:38.525082+0000","last_peered":"2023-10-18T09:01:38.525082+0000","last_clean":"2023-10-18T09:01:38.525082+0000","last_became_active":"2023-10-18T09:01:36.544271+0000","last_became_peered":"2023-10-18T09:01:36.544271+0000","last_unstale":"2023-10-18T09:01:38.525082+0000","last_undegraded":"2023-10-18T09:01:38.525082+0000","last_fullsized":"2023-10-18T09:01:38.525082+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:46:53.225811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":262,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-18T09:01:37.522739+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:01:37.522739+0000","last_peered":"2023-10-18T09:01:37.522739+0000","last_clean":"2023-10-18T09:01:37.522739+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:01:37.522739+0000","last_undegraded":"2023-10-18T09:01:37.522739+0000","last_fullsized":"2023-10-18T09:01:37.522739+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:18:36.913912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1150,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194162,"statfs":{"total":200000000,"available":198822253,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.101,"15min":1.0249999999999999},"min":{"1min":0.77800000000000002,"5min":0.67800000000000005,"15min":0.60999999999999999},"max":{"1min":1.5669999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.135,"15min":1.042},"min":{"1min":0.79200000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6439999999999999,"5min":1.97,"15min":1.97},"last":0.63100000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.145,"15min":1.1539999999999999},"min":{"1min":0.79100000000000004,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5189999999999999,"5min":1.5900000000000001,"15min":1.8109999999999999},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.163,"15min":1.129},"min":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5469999999999999,"5min":1.7849999999999999,"15min":1.7989999999999999},"last":1.0720000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.302,"15min":1.252},"min":{"1min":0.93500000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.7050000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.264,"15min":1.212},"min":{"1min":0.98599999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.8049999999999999,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.0169999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.27,"15min":1.266},"min":{"1min":0.96599999999999997,"5min":0.80300000000000005,"15min":0.76700000000000002},"max":{"1min":1.744,"5min":1.845,"15min":1.9430000000000001},"last":1.153},{"interface":"front","average":{"1min":1.359,"5min":1.3580000000000001,"15min":1.3300000000000001},"min":{"1min":0.998,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.756,"5min":1.873,"15min":1.9930000000000001},"last":1.365}]},{"osd":4,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.27,"15min":1.1859999999999999},"min":{"1min":0.90400000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.8939999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3300000000000001,"15min":1.385},"min":{"1min":0.84499999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.659,"5min":1.885,"15min":2.5169999999999999},"last":1.425}]},{"osd":5,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.361,"15min":1.3560000000000001},"min":{"1min":0.94499999999999995,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.831,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.429,"15min":1.425},"min":{"1min":1.1200000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.0259999999999998,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.2869999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.502,"15min":1.47},"min":{"1min":1.224,"5min":0.97899999999999998,"15min":0.95899999999999996},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.3959999999999999},"last":1.454},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.4830000000000001,"15min":1.5069999999999999},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.81599999999999995},"max":{"1min":1.9610000000000001,"5min":1.978,"15min":2.0590000000000002},"last":1.651}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":568,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194744,"statfs":{"total":200000000,"available":199418297,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:01:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2110000000000001,"15min":1.117},"min":{"1min":0.64700000000000002,"5min":0.623,"15min":0.60299999999999998},"max":{"1min":1.6240000000000001,"5min":1.651,"15min":1.6850000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.2989999999999999,"15min":1.2569999999999999},"min":{"1min":0.93200000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.018}]},{"osd":1,"last update":"Wed Oct 18 09:01:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.109,"15min":1.091},"min":{"1min":0.81999999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.4950000000000001,"5min":1.633,"15min":1.633},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.2190000000000001,"15min":1.1990000000000001},"min":{"1min":0.96599999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.8560000000000001,"5min":1.962,"15min":2.9609999999999999},"last":1.139}]},{"osd":2,"last update":"Wed Oct 18 09:01:09 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.1930000000000001,"15min":1.1839999999999999},"min":{"1min":0.75800000000000001,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.347,"5min":1.248,"15min":1.2110000000000001},"min":{"1min":0.98699999999999999,"5min":0.82099999999999995,"15min":0.66300000000000003},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.873}]},{"osd":3,"last update":"Wed Oct 18 09:01:09 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.2749999999999999,"15min":1.2649999999999999},"min":{"1min":1.0009999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.097},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.2669999999999999,"15min":1.2330000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.6679999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.587}]},{"osd":4,"last update":"Wed Oct 18 09:01:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.238,"15min":1.1839999999999999},"min":{"1min":0.86199999999999999,"5min":0.625,"15min":0.625},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.2809999999999999,"15min":1.2729999999999999},"min":{"1min":0.89700000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.97399999999999998}]},{"osd":5,"last update":"Wed Oct 18 09:01:09 2023","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.4179999999999999,"15min":1.3700000000000001},"min":{"1min":1.171,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.294,"15min":1.2370000000000001},"min":{"1min":0.90300000000000002,"5min":0.54900000000000004,"15min":0.49199999999999999},"max":{"1min":1.806,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.5349999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:01:09 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.4119999999999999,"15min":1.399},"min":{"1min":1.0289999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.633,"5min":1.5109999999999999,"15min":1.5249999999999999},"min":{"1min":1.373,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.794},"last":1.24}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":568,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194743,"statfs":{"total":200000000,"available":199417357,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:01:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3720000000000001,"15min":1.242},"min":{"1min":0.73899999999999999,"5min":0.68300000000000005,"15min":0.63400000000000001},"max":{"1min":1.6970000000000001,"5min":9.3539999999999992,"15min":9.3539999999999992},"last":9.609},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3300000000000001,"15min":1.306},"min":{"1min":0.751,"5min":0.71999999999999997,"15min":0.60799999999999998},"max":{"1min":1.623,"5min":5.3029999999999999,"15min":5.3029999999999999},"last":1.4399999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:01:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1919999999999999,"15min":1.218},"min":{"1min":0.82699999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.7669999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":11.68},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.1890000000000001,"15min":1.2370000000000001},"min":{"1min":0.94699999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6639999999999999,"5min":2.367,"15min":3.7050000000000001},"last":1.2629999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:01:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2490000000000001,"15min":1.2609999999999999},"min":{"1min":0.93999999999999995,"5min":0.77700000000000002,"15min":0.755},"max":{"1min":1.7749999999999999,"5min":2.0350000000000001,"15min":2.2090000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.232,"5min":1.2509999999999999,"15min":1.28},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.72399999999999998},"max":{"1min":1.72,"5min":2.3069999999999999,"15min":3.8180000000000001},"last":1.3200000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:01:14 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.234,"15min":1.2410000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.65400000000000003},"max":{"1min":2.6139999999999999,"5min":3.524,"15min":4.7119999999999997},"last":11.797000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.365,"15min":1.341},"min":{"1min":1.0900000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.633,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.747}]},{"osd":4,"last update":"Wed Oct 18 09:01:14 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3879999999999999,"15min":1.4099999999999999},"min":{"1min":1.0429999999999999,"5min":0.84899999999999998,"15min":0.75900000000000001},"max":{"1min":1.778,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":2.0680000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.3200000000000001,"15min":1.474},"min":{"1min":0.91500000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.7130000000000001,"5min":4.4429999999999996,"15min":5.4580000000000002},"last":1.1639999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:01:14 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.2869999999999999,"15min":1.3819999999999999},"min":{"1min":0.80300000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.5190000000000001,"5min":4.5469999999999997,"15min":5.141},"last":1.96},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4870000000000001,"15min":1.5640000000000001},"min":{"1min":1.0940000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.718,"5min":2.657,"15min":2.657},"last":0.82499999999999996}]},{"osd":7,"last update":"Wed Oct 18 09:01:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.4890000000000001,"15min":1.619},"min":{"1min":0.90300000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.762,"5min":4.2050000000000001,"15min":5.5490000000000004},"last":11.829000000000001},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.448,"15min":1.502},"min":{"1min":0.998,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.9420000000000002,"5min":4.4690000000000003,"15min":4.8419999999999996},"last":1.1220000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":566,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194746,"statfs":{"total":200000000,"available":199420010,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.129,"15min":1.117},"min":{"1min":0.66700000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.353},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.109,"15min":1.163},"min":{"1min":0.72499999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.4970000000000001,"5min":2.0659999999999998,"15min":2.2599999999999998},"last":1.252}]},{"osd":1,"last update":"Wed Oct 18 09:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.1120000000000001,"15min":1.1000000000000001},"min":{"1min":0.72299999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.5269999999999999,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":1.552},{"interface":"front","average":{"1min":1.149,"5min":1.173,"15min":1.222},"min":{"1min":0.77800000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.6259999999999999,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.204}]},{"osd":2,"last update":"Wed Oct 18 09:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1930000000000001,"15min":1.181},"min":{"1min":0.89000000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.468,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.496},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.2669999999999999,"15min":1.302},"min":{"1min":0.88200000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.837,"5min":1.994,"15min":2.3650000000000002},"last":1.4690000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.26,"15min":1.292},"min":{"1min":0.78800000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.669,"5min":2.649,"15min":2.649},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.397,"15min":1.407},"min":{"1min":1.05,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.728,"5min":2.6880000000000002,"15min":2.8010000000000002},"last":1.3140000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.296,"15min":1.2949999999999999},"min":{"1min":0.85099999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.978,"5min":1.978,"15min":1.978},"last":0.995},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.286,"15min":1.3460000000000001},"min":{"1min":0.83899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.5369999999999999,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.391}]},{"osd":6,"last update":"Wed Oct 18 09:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.427,"15min":1.3919999999999999},"min":{"1min":0.95399999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.788,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.145},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.472,"15min":1.546},"min":{"1min":1.167,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.085,"5min":4.407,"15min":4.407},"last":1.5269999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.4359999999999999,"15min":1.494},"min":{"1min":1.147,"5min":0.438,"15min":0.438},"max":{"1min":1.7090000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.484,"5min":1.526,"15min":1.542},"min":{"1min":1.0980000000000001,"5min":0.747,"15min":0.747},"max":{"1min":1.952,"5min":2.585,"15min":2.585},"last":1.4370000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":575,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194737,"statfs":{"total":200000000,"available":199411079,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.95599999999999996,"15min":0.80900000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.466,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0509999999999999,"15min":0.90600000000000003},"min":{"1min":0.48599999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.4569999999999999,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":0.56000000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0640000000000001,"15min":0.96999999999999997},"min":{"1min":0.59699999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.645,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":1.369,"5min":1.2789999999999999,"15min":1.159},"min":{"1min":0.80600000000000005,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.698,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.0549999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1259999999999999,"15min":1.038},"min":{"1min":0.63100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.0600000000000001,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":0.623},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.216,"15min":1.0900000000000001},"min":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.51400000000000001},"max":{"1min":3.169,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.53,"15min":1.327},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.897,"5min":18.664999999999999,"15min":18.664999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.421,"5min":1.3340000000000001,"15min":1.1919999999999999},"min":{"1min":0.877,"5min":0.69099999999999995,"15min":0.64000000000000001},"max":{"1min":1.99,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":1.008}]},{"osd":5,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3540000000000001,"15min":1.2430000000000001},"min":{"1min":0.83199999999999996,"5min":0.72399999999999998,"15min":0.67700000000000005},"max":{"1min":1.998,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.53,"5min":1.6240000000000001,"15min":1.381},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.0179999999999998,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.087}]},{"osd":6,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.5449999999999999,"15min":1.4430000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.129,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.5269999999999999,"15min":1.3740000000000001},"min":{"1min":1.2669999999999999,"5min":0.77200000000000002,"15min":0.60199999999999998},"max":{"1min":2.3759999999999999,"5min":7.242,"15min":7.242},"last":0.91400000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.5449999999999999,"15min":1.3799999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.899,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.645,"15min":1.4770000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.319,"5min":12.946,"15min":12.946},"last":1.1379999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":567,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194744,"statfs":{"total":200000000,"available":199418409,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.86799999999999999,"15min":0.92300000000000004},"min":{"1min":0.58899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.478,"5min":1.706,"15min":4.2210000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":0.96399999999999997,"15min":1.0369999999999999},"min":{"1min":0.55900000000000005,"5min":0.48999999999999999,"15min":0.44500000000000001},"max":{"1min":1.637,"5min":1.637,"15min":1.651},"last":1.494}]},{"osd":1,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.038,"15min":1.0589999999999999},"min":{"1min":0.64100000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":4.29},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1180000000000001,"15min":1.109},"min":{"1min":0.65000000000000002,"5min":0.57499999999999996,"15min":0.46200000000000002},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.5350000000000001},"last":1.1930000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.0980000000000001,"15min":1.1299999999999999},"min":{"1min":0.63600000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.915,"5min":1.915,"15min":4.0030000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.496,"5min":1.339,"15min":1.4630000000000001},"min":{"1min":0.69199999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":7.5570000000000004},"last":1.1439999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.409,"15min":1.482},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.4119999999999999,"5min":2.4119999999999999,"15min":7.6189999999999998},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.4419999999999999,"15min":1.4590000000000001},"min":{"1min":1.1919999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":1.3}]},{"osd":5,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.272,"15min":1.28},"min":{"1min":1.087,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":4.4320000000000004},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.4550000000000001,"15min":1.5609999999999999},"min":{"1min":0.997,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.536,"5min":2.536,"15min":7.657},"last":1.4750000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.651,"5min":1.4610000000000001,"15min":1.5269999999999999},"min":{"1min":1.0960000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.8780000000000001,"5min":2.8780000000000001,"15min":3.5299999999999998},"last":1.8480000000000001},{"interface":"front","average":{"1min":1.603,"5min":1.873,"15min":1.589},"min":{"1min":1.0720000000000001,"5min":0.94099999999999995,"15min":0.77600000000000002},"max":{"1min":2.4830000000000001,"5min":37.771999999999998,"15min":37.771999999999998},"last":1.4359999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.704,"5min":1.528,"15min":1.5720000000000001},"min":{"1min":1.262,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.5979999999999999,"5min":2.5979999999999999,"15min":3.3620000000000001},"last":1.9119999999999999},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.5029999999999999,"15min":1.492},"min":{"1min":1.147,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.6960000000000002,"5min":2.6960000000000002,"15min":4.3390000000000004},"last":1.5620000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":572,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194739,"statfs":{"total":200000000,"available":199413516,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.86899999999999999,"15min":0.82299999999999995},"min":{"1min":0.34499999999999997,"5min":0.315,"15min":0.315},"max":{"1min":1.5589999999999999,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.622},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0229999999999999,"15min":0.93500000000000005},"min":{"1min":0.52800000000000002,"5min":0.501,"15min":0.443},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.86799999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:01:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.98899999999999999,"15min":0.94899999999999995},"min":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.859},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.92900000000000005,"15min":0.86699999999999999},"min":{"1min":0.44800000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.601,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.3879999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:01:18 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0529999999999999,"15min":1.0309999999999999},"min":{"1min":0.49199999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.6040000000000001,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":3.7010000000000001},"last":1.2150000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:01:18 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.2989999999999999,"15min":1.2609999999999999},"min":{"1min":0.91900000000000004,"5min":0.71499999999999997,"15min":0.59999999999999998},"max":{"1min":1.909,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.296,"15min":1.226},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.8520000000000001,"5min":1.99,"15min":2.0470000000000002},"last":1.681}]},{"osd":5,"last update":"Wed Oct 18 09:01:18 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.3560000000000001,"15min":1.3100000000000001},"min":{"1min":0.93700000000000006,"5min":0.82899999999999996,"15min":0.65400000000000003},"max":{"1min":1.7230000000000001,"5min":2.161,"15min":3.5750000000000002},"last":1.504},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4239999999999999,"15min":1.3700000000000001},"min":{"1min":0.97799999999999998,"5min":0.84999999999999998,"15min":0.83899999999999997},"max":{"1min":1.907,"5min":1.964,"15min":2.3769999999999998},"last":1.45}]},{"osd":6,"last update":"Wed Oct 18 09:01:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.363,"15min":1.302},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.8280000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.373,"15min":1.2909999999999999},"min":{"1min":0.99099999999999999,"5min":0.72999999999999998,"15min":0.63100000000000001},"max":{"1min":1.859,"5min":2.121,"15min":2.121},"last":1.631}]},{"osd":7,"last update":"Wed Oct 18 09:01:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4450000000000001,"15min":1.3759999999999999},"min":{"1min":0.90900000000000003,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.0579999999999998,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.728},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.468,"15min":1.383},"min":{"1min":0.82299999999999995,"5min":0.746,"15min":0.746},"max":{"1min":2.2290000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.5489999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1149,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194163,"statfs":{"total":200000000,"available":198823308,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81599999999999995,"15min":0.91000000000000003},"min":{"1min":0.51700000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.1890000000000001,"5min":1.25,"15min":3.4420000000000002},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.0920000000000001,"15min":1.0880000000000001},"min":{"1min":0.90000000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.262}]},{"osd":2,"last update":"Wed Oct 18 09:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86499999999999999,"15min":0.85099999999999998},"min":{"1min":0.58499999999999996,"5min":0.436,"15min":0.436},"max":{"1min":1.3069999999999999,"5min":1.4159999999999999,"15min":1.5029999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.181,"5min":1.1539999999999999,"15min":1.135},"min":{"1min":0.73299999999999998,"5min":0.437,"15min":0.437},"max":{"1min":1.55,"5min":1.929,"15min":1.929},"last":0.94999999999999996}]},{"osd":3,"last update":"Wed Oct 18 09:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1399999999999999,"15min":1.2789999999999999},"min":{"1min":0.80000000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.788,"5min":1.8200000000000001,"15min":2.7639999999999998},"last":1.113},{"interface":"front","average":{"1min":1.488,"5min":1.4039999999999999,"15min":1.5069999999999999},"min":{"1min":0.84199999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.8779999999999999,"5min":2.3639999999999999,"15min":3.5720000000000001},"last":1.7250000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2230000000000001,"15min":1.3029999999999999},"min":{"1min":0.86599999999999999,"5min":0.72899999999999998,"15min":0.72399999999999998},"max":{"1min":1.8109999999999999,"5min":1.8819999999999999,"15min":4.492},"last":1.671},{"interface":"front","average":{"1min":1.288,"5min":1.2889999999999999,"15min":1.26},"min":{"1min":0.95599999999999996,"5min":0.78500000000000003,"15min":0.72599999999999998},"max":{"1min":1.585,"5min":1.726,"15min":1.726},"last":1.4079999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2569999999999999,"15min":1.409},"min":{"1min":0.93799999999999994,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.675,"5min":2.4510000000000001,"15min":4.2779999999999996},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3200000000000001,"15min":1.373},"min":{"1min":0.94499999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.722,"5min":1.8240000000000001,"15min":1.8740000000000001},"last":1.5900000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.464,"15min":1.577},"min":{"1min":1.2729999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.881,"5min":2.0419999999999998,"15min":3.5299999999999998},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.393,"15min":1.494},"min":{"1min":0.95699999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.73,"5min":2.1059999999999999,"15min":4.4349999999999996},"last":1.1799999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.5509999999999999,"15min":1.639},"min":{"1min":1.036,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.956,"5min":2.2000000000000002,"15min":3.492},"last":1.7929999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4490000000000001,"15min":1.655},"min":{"1min":0.995,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.853,"5min":2.0449999999999999,"15min":4.5419999999999998},"last":1.4830000000000001}]}]}],"pool_statfs":[]}} 2023-10-18T09:01:42.074 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:01:42.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.347+0000 7f931d316640 1 -- 172.21.15.40:0/2874865019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9304004d00 msgr2=0x7f9304005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:42.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.347+0000 7f931d316640 1 --2- 172.21.15.40:0/2874865019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9304004d00 0x7f9304005180 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9310009b80 tx=0x7f93040057f0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.347+0000 7f93177fe640 1 -- 172.21.15.40:0/2874865019 reap_dead start 2023-10-18T09:01:42.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.347+0000 7f931d316640 1 -- 172.21.15.40:0/2874865019 shutdown_connections 2023-10-18T09:01:42.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.347+0000 7f931d316640 1 -- 172.21.15.40:0/2874865019 >> 172.21.15.40:0/2874865019 conn(0x7f931806a780 msgr2=0x7f931806ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:42.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.347+0000 7f931d316640 1 -- 172.21.15.40:0/2874865019 shutdown_connections 2023-10-18T09:01:42.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.347+0000 7f931d316640 1 -- 172.21.15.40:0/2874865019 wait complete. 2023-10-18T09:01:42.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.349+0000 7f931d316640 1 Processor -- start 2023-10-18T09:01:42.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.349+0000 7f931d316640 1 -- start start 2023-10-18T09:01:42.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.350+0000 7f931d316640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9304004d00 0x7f93180c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:42.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.350+0000 7f931d316640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9318150e30 0x7f931815dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:42.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.350+0000 7f931d316640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f931815e1b0 0x7f93180c5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:42.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.350+0000 7f931d316640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931815e630 con 0x7f9304004d00 2023-10-18T09:01:42.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.350+0000 7f931d316640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9318155470 con 0x7f931815e1b0 2023-10-18T09:01:42.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.350+0000 7f931d316640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93181512b0 con 0x7f9318150e30 2023-10-18T09:01:42.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.350+0000 7f9316ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9304004d00 0x7f93180c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:42.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.351+0000 7f9316ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9304004d00 0x7f93180c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51010/0 (socket says 172.21.15.40:51010) 2023-10-18T09:01:42.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.351+0000 7f9316ffd640 1 -- 172.21.15.40:0/3611872836 learned_addr learned my addr 172.21.15.40:0/3611872836 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:42.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.351+0000 7f9316ffd640 1 -- 172.21.15.40:0/3611872836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9318150e30 msgr2=0x7f931815dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:42.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.351+0000 7f9316ffd640 1 --2- 172.21.15.40:0/3611872836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9318150e30 0x7f931815dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.351+0000 7f9316ffd640 1 -- 172.21.15.40:0/3611872836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f931815e1b0 msgr2=0x7f93180c5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:42.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.351+0000 7f9316ffd640 1 --2- 172.21.15.40:0/3611872836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f931815e1b0 0x7f93180c5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.351+0000 7f9316ffd640 1 -- 172.21.15.40:0/3611872836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93180c5bf0 con 0x7f9304004d00 2023-10-18T09:01:42.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.351+0000 7f9316ffd640 1 --2- 172.21.15.40:0/3611872836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9304004d00 0x7f93180c6c90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f930800dcc0 tx=0x7f930800b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:42.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.351+0000 7f930f7fe640 1 -- 172.21.15.40:0/3611872836 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930800de70 con 0x7f9304004d00 2023-10-18T09:01:42.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.352+0000 7f930f7fe640 1 -- 172.21.15.40:0/3611872836 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9308003970 con 0x7f9304004d00 2023-10-18T09:01:42.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.352+0000 7f930f7fe640 1 -- 172.21.15.40:0/3611872836 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308003d80 con 0x7f9304004d00 2023-10-18T09:01:42.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.352+0000 7f931d316640 1 -- 172.21.15.40:0/3611872836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93180c5ed0 con 0x7f9304004d00 2023-10-18T09:01:42.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.352+0000 7f931d316640 1 -- 172.21.15.40:0/3611872836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93180c6490 con 0x7f9304004d00 2023-10-18T09:01:42.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.353+0000 7f931d316640 1 -- 172.21.15.40:0/3611872836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9318066fd0 con 0x7f9304004d00 2023-10-18T09:01:42.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.354+0000 7f930f7fe640 1 -- 172.21.15.40:0/3611872836 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f93080060d0 con 0x7f9304004d00 2023-10-18T09:01:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.355+0000 7f930f7fe640 1 --2- 172.21.15.40:0/3611872836 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f92f806ae70 0x7f92f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.355+0000 7f930f7fe640 1 -- 172.21.15.40:0/3611872836 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f930808a500 con 0x7f9304004d00 2023-10-18T09:01:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.356+0000 7f93167fc640 1 --2- 172.21.15.40:0/3611872836 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f92f806ae70 0x7f92f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:42.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.357+0000 7f93167fc640 1 --2- 172.21.15.40:0/3611872836 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f92f806ae70 0x7f92f806d330 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f93181f51c0 tx=0x7f93000095d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:42.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.360+0000 7f930f7fe640 1 -- 172.21.15.40:0/3611872836 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9308055480 con 0x7f9304004d00 2023-10-18T09:01:42.424 INFO:tasks.ceph.mds.c.smithi040.stderr:2023-10-18T09:01:42.422+0000 7fd431497640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T09:01:42.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.548+0000 7f931d316640 1 -- 172.21.15.40:0/3611872836 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93180b5ba0 con 0x7f92f806ae70 2023-10-18T09:01:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.570+0000 7f930f7fe640 1 -- 172.21.15.40:0/3611872836 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282810 (secure 0 0 0) 0x7f93180b5ba0 con 0x7f92f806ae70 2023-10-18T09:01:42.579 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:42.581 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:01:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.576+0000 7f931d316640 1 -- 172.21.15.40:0/3611872836 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f92f806ae70 msgr2=0x7f92f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:42.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.576+0000 7f931d316640 1 --2- 172.21.15.40:0/3611872836 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f92f806ae70 0x7f92f806d330 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f93181f51c0 tx=0x7f93000095d0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.576+0000 7f931d316640 1 -- 172.21.15.40:0/3611872836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9304004d00 msgr2=0x7f93180c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:42.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.576+0000 7f931d316640 1 --2- 172.21.15.40:0/3611872836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9304004d00 0x7f93180c6c90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f930800dcc0 tx=0x7f930800b620 comp rx=0 tx=0).stop 2023-10-18T09:01:42.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.576+0000 7f931d316640 1 -- 172.21.15.40:0/3611872836 shutdown_connections 2023-10-18T09:01:42.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.576+0000 7f931d316640 1 --2- 172.21.15.40:0/3611872836 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f92f806ae70 0x7f92f806d330 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.576+0000 7f931d316640 1 --2- 172.21.15.40:0/3611872836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f931815e1b0 0x7f93180c5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.576+0000 7f931d316640 1 --2- 172.21.15.40:0/3611872836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9318150e30 0x7f931815dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.576+0000 7f931d316640 1 --2- 172.21.15.40:0/3611872836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9304004d00 0x7f93180c6c90 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.576+0000 7f931d316640 1 -- 172.21.15.40:0/3611872836 >> 172.21.15.40:0/3611872836 conn(0x7f931806a780 msgr2=0x7f93180b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.576+0000 7f931d316640 1 -- 172.21.15.40:0/3611872836 shutdown_connections 2023-10-18T09:01:42.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.576+0000 7f931d316640 1 -- 172.21.15.40:0/3611872836 wait complete. 2023-10-18T09:01:42.597 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2023-10-18T09:01:41.035449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5718,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556781,"statfs":{"total":1600000000,"available":1594144229,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.524102"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868397+0000","last_change":"2023-10-18T09:01:37.552785+0000","last_active":"2023-10-18T09:01:38.868397+0000","last_peered":"2023-10-18T09:01:38.868397+0000","last_clean":"2023-10-18T09:01:38.868397+0000","last_became_active":"2023-10-18T09:01:37.551859+0000","last_became_peered":"2023-10-18T09:01:37.551859+0000","last_unstale":"2023-10-18T09:01:38.868397+0000","last_undegraded":"2023-10-18T09:01:38.868397+0000","last_fullsized":"2023-10-18T09:01:38.868397+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:01:51.628601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524125+0000","last_change":"2023-10-18T09:01:37.545250+0000","last_active":"2023-10-18T09:01:38.524125+0000","last_peered":"2023-10-18T09:01:38.524125+0000","last_clean":"2023-10-18T09:01:38.524125+0000","last_became_active":"2023-10-18T09:01:37.544495+0000","last_became_peered":"2023-10-18T09:01:37.544495+0000","last_unstale":"2023-10-18T09:01:38.524125+0000","last_undegraded":"2023-10-18T09:01:38.524125+0000","last_fullsized":"2023-10-18T09:01:38.524125+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:19:47.931295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524363+0000","last_change":"2023-10-18T09:01:37.547103+0000","last_active":"2023-10-18T09:01:38.524363+0000","last_peered":"2023-10-18T09:01:38.524363+0000","last_clean":"2023-10-18T09:01:38.524363+0000","last_became_active":"2023-10-18T09:01:37.546795+0000","last_became_peered":"2023-10-18T09:01:37.546795+0000","last_unstale":"2023-10-18T09:01:38.524363+0000","last_undegraded":"2023-10-18T09:01:38.524363+0000","last_fullsized":"2023-10-18T09:01:38.524363+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:07:24.109273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865399+0000","last_change":"2023-10-18T09:01:37.558780+0000","last_active":"2023-10-18T09:01:38.865399+0000","last_peered":"2023-10-18T09:01:38.865399+0000","last_clean":"2023-10-18T09:01:38.865399+0000","last_became_active":"2023-10-18T09:01:37.557141+0000","last_became_peered":"2023-10-18T09:01:37.557141+0000","last_unstale":"2023-10-18T09:01:38.865399+0000","last_undegraded":"2023-10-18T09:01:38.865399+0000","last_fullsized":"2023-10-18T09:01:38.865399+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:44:40.227775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868999+0000","last_change":"2023-10-18T09:01:37.553214+0000","last_active":"2023-10-18T09:01:38.868999+0000","last_peered":"2023-10-18T09:01:38.868999+0000","last_clean":"2023-10-18T09:01:38.868999+0000","last_became_active":"2023-10-18T09:01:37.553004+0000","last_became_peered":"2023-10-18T09:01:37.553004+0000","last_unstale":"2023-10-18T09:01:38.868999+0000","last_undegraded":"2023-10-18T09:01:38.868999+0000","last_fullsized":"2023-10-18T09:01:38.868999+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:47.390225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867364+0000","last_change":"2023-10-18T09:01:37.548740+0000","last_active":"2023-10-18T09:01:38.867364+0000","last_peered":"2023-10-18T09:01:38.867364+0000","last_clean":"2023-10-18T09:01:38.867364+0000","last_became_active":"2023-10-18T09:01:37.548435+0000","last_became_peered":"2023-10-18T09:01:37.548435+0000","last_unstale":"2023-10-18T09:01:38.867364+0000","last_undegraded":"2023-10-18T09:01:38.867364+0000","last_fullsized":"2023-10-18T09:01:38.867364+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:36:16.799678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866108+0000","last_change":"2023-10-18T09:01:37.545636+0000","last_active":"2023-10-18T09:01:38.866108+0000","last_peered":"2023-10-18T09:01:38.866108+0000","last_clean":"2023-10-18T09:01:38.866108+0000","last_became_active":"2023-10-18T09:01:37.542287+0000","last_became_peered":"2023-10-18T09:01:37.542287+0000","last_unstale":"2023-10-18T09:01:38.866108+0000","last_undegraded":"2023-10-18T09:01:38.866108+0000","last_fullsized":"2023-10-18T09:01:38.866108+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:14:49.996096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865833+0000","last_change":"2023-10-18T09:01:37.552855+0000","last_active":"2023-10-18T09:01:38.865833+0000","last_peered":"2023-10-18T09:01:38.865833+0000","last_clean":"2023-10-18T09:01:38.865833+0000","last_became_active":"2023-10-18T09:01:37.552315+0000","last_became_peered":"2023-10-18T09:01:37.552315+0000","last_unstale":"2023-10-18T09:01:38.865833+0000","last_undegraded":"2023-10-18T09:01:38.865833+0000","last_fullsized":"2023-10-18T09:01:38.865833+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:07:42.731036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868149+0000","last_change":"2023-10-18T09:01:37.548169+0000","last_active":"2023-10-18T09:01:38.868149+0000","last_peered":"2023-10-18T09:01:38.868149+0000","last_clean":"2023-10-18T09:01:38.868149+0000","last_became_active":"2023-10-18T09:01:37.547901+0000","last_became_peered":"2023-10-18T09:01:37.547901+0000","last_unstale":"2023-10-18T09:01:38.868149+0000","last_undegraded":"2023-10-18T09:01:38.868149+0000","last_fullsized":"2023-10-18T09:01:38.868149+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:56:06.961383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868510+0000","last_change":"2023-10-18T09:01:37.545882+0000","last_active":"2023-10-18T09:01:38.868510+0000","last_peered":"2023-10-18T09:01:38.868510+0000","last_clean":"2023-10-18T09:01:38.868510+0000","last_became_active":"2023-10-18T09:01:37.545324+0000","last_became_peered":"2023-10-18T09:01:37.545324+0000","last_unstale":"2023-10-18T09:01:38.868510+0000","last_undegraded":"2023-10-18T09:01:38.868510+0000","last_fullsized":"2023-10-18T09:01:38.868510+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:34.616762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524394+0000","last_change":"2023-10-18T09:01:37.530225+0000","last_active":"2023-10-18T09:01:38.524394+0000","last_peered":"2023-10-18T09:01:38.524394+0000","last_clean":"2023-10-18T09:01:38.524394+0000","last_became_active":"2023-10-18T09:01:37.529247+0000","last_became_peered":"2023-10-18T09:01:37.529247+0000","last_unstale":"2023-10-18T09:01:38.524394+0000","last_undegraded":"2023-10-18T09:01:38.524394+0000","last_fullsized":"2023-10-18T09:01:38.524394+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:56:49.895421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866495+0000","last_change":"2023-10-18T09:01:37.544915+0000","last_active":"2023-10-18T09:01:38.866495+0000","last_peered":"2023-10-18T09:01:38.866495+0000","last_clean":"2023-10-18T09:01:38.866495+0000","last_became_active":"2023-10-18T09:01:37.544688+0000","last_became_peered":"2023-10-18T09:01:37.544688+0000","last_unstale":"2023-10-18T09:01:38.866495+0000","last_undegraded":"2023-10-18T09:01:38.866495+0000","last_fullsized":"2023-10-18T09:01:38.866495+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:29.195475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524740+0000","last_change":"2023-10-18T09:01:37.544219+0000","last_active":"2023-10-18T09:01:38.524740+0000","last_peered":"2023-10-18T09:01:38.524740+0000","last_clean":"2023-10-18T09:01:38.524740+0000","last_became_active":"2023-10-18T09:01:37.543994+0000","last_became_peered":"2023-10-18T09:01:37.543994+0000","last_unstale":"2023-10-18T09:01:38.524740+0000","last_undegraded":"2023-10-18T09:01:38.524740+0000","last_fullsized":"2023-10-18T09:01:38.524740+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:34:37.336226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865538+0000","last_change":"2023-10-18T09:01:37.558598+0000","last_active":"2023-10-18T09:01:38.865538+0000","last_peered":"2023-10-18T09:01:38.865538+0000","last_clean":"2023-10-18T09:01:38.865538+0000","last_became_active":"2023-10-18T09:01:37.556828+0000","last_became_peered":"2023-10-18T09:01:37.556828+0000","last_unstale":"2023-10-18T09:01:38.865538+0000","last_undegraded":"2023-10-18T09:01:38.865538+0000","last_fullsized":"2023-10-18T09:01:38.865538+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:42:06.726482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.525418+0000","last_change":"2023-10-18T09:01:37.533631+0000","last_active":"2023-10-18T09:01:38.525418+0000","last_peered":"2023-10-18T09:01:38.525418+0000","last_clean":"2023-10-18T09:01:38.525418+0000","last_became_active":"2023-10-18T09:01:37.532137+0000","last_became_peered":"2023-10-18T09:01:37.532137+0000","last_unstale":"2023-10-18T09:01:38.525418+0000","last_undegraded":"2023-10-18T09:01:38.525418+0000","last_fullsized":"2023-10-18T09:01:38.525418+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:46.732096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865882+0000","last_change":"2023-10-18T09:01:37.551281+0000","last_active":"2023-10-18T09:01:38.865882+0000","last_peered":"2023-10-18T09:01:38.865882+0000","last_clean":"2023-10-18T09:01:38.865882+0000","last_became_active":"2023-10-18T09:01:37.548874+0000","last_became_peered":"2023-10-18T09:01:37.548874+0000","last_unstale":"2023-10-18T09:01:38.865882+0000","last_undegraded":"2023-10-18T09:01:38.865882+0000","last_fullsized":"2023-10-18T09:01:38.865882+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:27.053204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866477+0000","last_change":"2023-10-18T09:01:37.548422+0000","last_active":"2023-10-18T09:01:38.866477+0000","last_peered":"2023-10-18T09:01:38.866477+0000","last_clean":"2023-10-18T09:01:38.866477+0000","last_became_active":"2023-10-18T09:01:37.548207+0000","last_became_peered":"2023-10-18T09:01:37.548207+0000","last_unstale":"2023-10-18T09:01:38.866477+0000","last_undegraded":"2023-10-18T09:01:38.866477+0000","last_fullsized":"2023-10-18T09:01:38.866477+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:54:34.252765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868615+0000","last_change":"2023-10-18T09:01:37.559103+0000","last_active":"2023-10-18T09:01:38.868615+0000","last_peered":"2023-10-18T09:01:38.868615+0000","last_clean":"2023-10-18T09:01:38.868615+0000","last_became_active":"2023-10-18T09:01:37.558024+0000","last_became_peered":"2023-10-18T09:01:37.558024+0000","last_unstale":"2023-10-18T09:01:38.868615+0000","last_undegraded":"2023-10-18T09:01:38.868615+0000","last_fullsized":"2023-10-18T09:01:38.868615+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:04:39.546838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865554+0000","last_change":"2023-10-18T09:01:37.545473+0000","last_active":"2023-10-18T09:01:38.865554+0000","last_peered":"2023-10-18T09:01:38.865554+0000","last_clean":"2023-10-18T09:01:38.865554+0000","last_became_active":"2023-10-18T09:01:37.544854+0000","last_became_peered":"2023-10-18T09:01:37.544854+0000","last_unstale":"2023-10-18T09:01:38.865554+0000","last_undegraded":"2023-10-18T09:01:38.865554+0000","last_fullsized":"2023-10-18T09:01:38.865554+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:34.708210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866365+0000","last_change":"2023-10-18T09:01:37.547736+0000","last_active":"2023-10-18T09:01:38.866365+0000","last_peered":"2023-10-18T09:01:38.866365+0000","last_clean":"2023-10-18T09:01:38.866365+0000","last_became_active":"2023-10-18T09:01:37.545779+0000","last_became_peered":"2023-10-18T09:01:37.545779+0000","last_unstale":"2023-10-18T09:01:38.866365+0000","last_undegraded":"2023-10-18T09:01:38.866365+0000","last_fullsized":"2023-10-18T09:01:38.866365+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:18:14.443467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866707+0000","last_change":"2023-10-18T09:01:37.556332+0000","last_active":"2023-10-18T09:01:38.866707+0000","last_peered":"2023-10-18T09:01:38.866707+0000","last_clean":"2023-10-18T09:01:38.866707+0000","last_became_active":"2023-10-18T09:01:37.555835+0000","last_became_peered":"2023-10-18T09:01:37.555835+0000","last_unstale":"2023-10-18T09:01:38.866707+0000","last_undegraded":"2023-10-18T09:01:38.866707+0000","last_fullsized":"2023-10-18T09:01:38.866707+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:34:30.100471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524298+0000","last_change":"2023-10-18T09:01:37.550010+0000","last_active":"2023-10-18T09:01:38.524298+0000","last_peered":"2023-10-18T09:01:38.524298+0000","last_clean":"2023-10-18T09:01:38.524298+0000","last_became_active":"2023-10-18T09:01:37.549749+0000","last_became_peered":"2023-10-18T09:01:37.549749+0000","last_unstale":"2023-10-18T09:01:38.524298+0000","last_undegraded":"2023-10-18T09:01:38.524298+0000","last_fullsized":"2023-10-18T09:01:38.524298+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:46.999487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867427+0000","last_change":"2023-10-18T09:01:37.554476+0000","last_active":"2023-10-18T09:01:38.867427+0000","last_peered":"2023-10-18T09:01:38.867427+0000","last_clean":"2023-10-18T09:01:38.867427+0000","last_became_active":"2023-10-18T09:01:37.553393+0000","last_became_peered":"2023-10-18T09:01:37.553393+0000","last_unstale":"2023-10-18T09:01:38.867427+0000","last_undegraded":"2023-10-18T09:01:38.867427+0000","last_fullsized":"2023-10-18T09:01:38.867427+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:39.025489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865688+0000","last_change":"2023-10-18T09:01:37.545951+0000","last_active":"2023-10-18T09:01:38.865688+0000","last_peered":"2023-10-18T09:01:38.865688+0000","last_clean":"2023-10-18T09:01:38.865688+0000","last_became_active":"2023-10-18T09:01:37.545721+0000","last_became_peered":"2023-10-18T09:01:37.545721+0000","last_unstale":"2023-10-18T09:01:38.865688+0000","last_undegraded":"2023-10-18T09:01:38.865688+0000","last_fullsized":"2023-10-18T09:01:38.865688+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:18:51.055185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524105+0000","last_change":"2023-10-18T09:01:37.541983+0000","last_active":"2023-10-18T09:01:38.524105+0000","last_peered":"2023-10-18T09:01:38.524105+0000","last_clean":"2023-10-18T09:01:38.524105+0000","last_became_active":"2023-10-18T09:01:37.541725+0000","last_became_peered":"2023-10-18T09:01:37.541725+0000","last_unstale":"2023-10-18T09:01:38.524105+0000","last_undegraded":"2023-10-18T09:01:38.524105+0000","last_fullsized":"2023-10-18T09:01:38.524105+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:49.997650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524685+0000","last_change":"2023-10-18T09:01:37.531929+0000","last_active":"2023-10-18T09:01:38.524685+0000","last_peered":"2023-10-18T09:01:38.524685+0000","last_clean":"2023-10-18T09:01:38.524685+0000","last_became_active":"2023-10-18T09:01:37.531684+0000","last_became_peered":"2023-10-18T09:01:37.531684+0000","last_unstale":"2023-10-18T09:01:38.524685+0000","last_undegraded":"2023-10-18T09:01:38.524685+0000","last_fullsized":"2023-10-18T09:01:38.524685+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:54.562735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867701+0000","last_change":"2023-10-18T09:01:37.543073+0000","last_active":"2023-10-18T09:01:38.867701+0000","last_peered":"2023-10-18T09:01:38.867701+0000","last_clean":"2023-10-18T09:01:38.867701+0000","last_became_active":"2023-10-18T09:01:37.542040+0000","last_became_peered":"2023-10-18T09:01:37.542040+0000","last_unstale":"2023-10-18T09:01:38.867701+0000","last_undegraded":"2023-10-18T09:01:38.867701+0000","last_fullsized":"2023-10-18T09:01:38.867701+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:13:34.616762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867754+0000","last_change":"2023-10-18T09:01:37.554245+0000","last_active":"2023-10-18T09:01:38.867754+0000","last_peered":"2023-10-18T09:01:38.867754+0000","last_clean":"2023-10-18T09:01:38.867754+0000","last_became_active":"2023-10-18T09:01:37.550345+0000","last_became_peered":"2023-10-18T09:01:37.550345+0000","last_unstale":"2023-10-18T09:01:38.867754+0000","last_undegraded":"2023-10-18T09:01:38.867754+0000","last_fullsized":"2023-10-18T09:01:38.867754+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:54:34.252765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524187+0000","last_change":"2023-10-18T09:01:37.540038+0000","last_active":"2023-10-18T09:01:38.524187+0000","last_peered":"2023-10-18T09:01:38.524187+0000","last_clean":"2023-10-18T09:01:38.524187+0000","last_became_active":"2023-10-18T09:01:37.539791+0000","last_became_peered":"2023-10-18T09:01:37.539791+0000","last_unstale":"2023-10-18T09:01:38.524187+0000","last_undegraded":"2023-10-18T09:01:38.524187+0000","last_fullsized":"2023-10-18T09:01:38.524187+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:28:43.093717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524294+0000","last_change":"2023-10-18T09:01:37.546371+0000","last_active":"2023-10-18T09:01:38.524294+0000","last_peered":"2023-10-18T09:01:38.524294+0000","last_clean":"2023-10-18T09:01:38.524294+0000","last_became_active":"2023-10-18T09:01:37.544888+0000","last_became_peered":"2023-10-18T09:01:37.544888+0000","last_unstale":"2023-10-18T09:01:38.524294+0000","last_undegraded":"2023-10-18T09:01:38.524294+0000","last_fullsized":"2023-10-18T09:01:38.524294+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:46:32.086246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866400+0000","last_change":"2023-10-18T09:01:37.551724+0000","last_active":"2023-10-18T09:01:38.866400+0000","last_peered":"2023-10-18T09:01:38.866400+0000","last_clean":"2023-10-18T09:01:38.866400+0000","last_became_active":"2023-10-18T09:01:37.551031+0000","last_became_peered":"2023-10-18T09:01:37.551031+0000","last_unstale":"2023-10-18T09:01:38.866400+0000","last_undegraded":"2023-10-18T09:01:38.866400+0000","last_fullsized":"2023-10-18T09:01:38.866400+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:39.025489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866862+0000","last_change":"2023-10-18T09:01:37.551638+0000","last_active":"2023-10-18T09:01:38.866862+0000","last_peered":"2023-10-18T09:01:38.866862+0000","last_clean":"2023-10-18T09:01:38.866862+0000","last_became_active":"2023-10-18T09:01:37.551394+0000","last_became_peered":"2023-10-18T09:01:37.551394+0000","last_unstale":"2023-10-18T09:01:38.866862+0000","last_undegraded":"2023-10-18T09:01:38.866862+0000","last_fullsized":"2023-10-18T09:01:38.866862+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:08.381428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865005+0000","last_change":"2023-10-18T09:01:37.549886+0000","last_active":"2023-10-18T09:01:38.865005+0000","last_peered":"2023-10-18T09:01:38.865005+0000","last_clean":"2023-10-18T09:01:38.865005+0000","last_became_active":"2023-10-18T09:01:37.549538+0000","last_became_peered":"2023-10-18T09:01:37.549538+0000","last_unstale":"2023-10-18T09:01:38.865005+0000","last_undegraded":"2023-10-18T09:01:38.865005+0000","last_fullsized":"2023-10-18T09:01:38.865005+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:08.843083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"106'4","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868280+0000","last_change":"2023-10-18T09:01:36.541071+0000","last_active":"2023-10-18T09:01:38.868280+0000","last_peered":"2023-10-18T09:01:38.868280+0000","last_clean":"2023-10-18T09:01:38.868280+0000","last_became_active":"2023-10-18T09:01:36.540811+0000","last_became_peered":"2023-10-18T09:01:36.540811+0000","last_unstale":"2023-10-18T09:01:38.868280+0000","last_undegraded":"2023-10-18T09:01:38.868280+0000","last_fullsized":"2023-10-18T09:01:38.868280+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:11.953586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868060+0000","last_change":"2023-10-18T09:01:37.554100+0000","last_active":"2023-10-18T09:01:38.868060+0000","last_peered":"2023-10-18T09:01:38.868060+0000","last_clean":"2023-10-18T09:01:38.868060+0000","last_became_active":"2023-10-18T09:01:37.549932+0000","last_became_peered":"2023-10-18T09:01:37.549932+0000","last_unstale":"2023-10-18T09:01:38.868060+0000","last_undegraded":"2023-10-18T09:01:38.868060+0000","last_fullsized":"2023-10-18T09:01:38.868060+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:09.634600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"106'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867252+0000","last_change":"2023-10-18T09:01:36.543509+0000","last_active":"2023-10-18T09:01:38.867252+0000","last_peered":"2023-10-18T09:01:38.867252+0000","last_clean":"2023-10-18T09:01:38.867252+0000","last_became_active":"2023-10-18T09:01:36.543069+0000","last_became_peered":"2023-10-18T09:01:36.543069+0000","last_unstale":"2023-10-18T09:01:38.867252+0000","last_undegraded":"2023-10-18T09:01:38.867252+0000","last_fullsized":"2023-10-18T09:01:38.867252+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:11:55.265579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868774+0000","last_change":"2023-10-18T09:01:37.548336+0000","last_active":"2023-10-18T09:01:38.868774+0000","last_peered":"2023-10-18T09:01:38.868774+0000","last_clean":"2023-10-18T09:01:38.868774+0000","last_became_active":"2023-10-18T09:01:37.548088+0000","last_became_peered":"2023-10-18T09:01:37.548088+0000","last_unstale":"2023-10-18T09:01:38.868774+0000","last_undegraded":"2023-10-18T09:01:38.868774+0000","last_fullsized":"2023-10-18T09:01:38.868774+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:23.642776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"106'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866222+0000","last_change":"2023-10-18T09:01:36.540493+0000","last_active":"2023-10-18T09:01:38.866222+0000","last_peered":"2023-10-18T09:01:38.866222+0000","last_clean":"2023-10-18T09:01:38.866222+0000","last_became_active":"2023-10-18T09:01:36.540255+0000","last_became_peered":"2023-10-18T09:01:36.540255+0000","last_unstale":"2023-10-18T09:01:38.866222+0000","last_undegraded":"2023-10-18T09:01:38.866222+0000","last_fullsized":"2023-10-18T09:01:38.866222+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:47:16.660316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.525525+0000","last_change":"2023-10-18T09:01:37.541887+0000","last_active":"2023-10-18T09:01:38.525525+0000","last_peered":"2023-10-18T09:01:38.525525+0000","last_clean":"2023-10-18T09:01:38.525525+0000","last_became_active":"2023-10-18T09:01:37.541669+0000","last_became_peered":"2023-10-18T09:01:37.541669+0000","last_unstale":"2023-10-18T09:01:38.525525+0000","last_undegraded":"2023-10-18T09:01:38.525525+0000","last_fullsized":"2023-10-18T09:01:38.525525+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:12:46.732096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"106'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867811+0000","last_change":"2023-10-18T09:01:36.539895+0000","last_active":"2023-10-18T09:01:38.867811+0000","last_peered":"2023-10-18T09:01:38.867811+0000","last_clean":"2023-10-18T09:01:38.867811+0000","last_became_active":"2023-10-18T09:01:36.539615+0000","last_became_peered":"2023-10-18T09:01:36.539615+0000","last_unstale":"2023-10-18T09:01:38.867811+0000","last_undegraded":"2023-10-18T09:01:38.867811+0000","last_fullsized":"2023-10-18T09:01:38.867811+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:50:35.235441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524960+0000","last_change":"2023-10-18T09:01:37.550215+0000","last_active":"2023-10-18T09:01:38.524960+0000","last_peered":"2023-10-18T09:01:38.524960+0000","last_clean":"2023-10-18T09:01:38.524960+0000","last_became_active":"2023-10-18T09:01:37.547491+0000","last_became_peered":"2023-10-18T09:01:37.547491+0000","last_unstale":"2023-10-18T09:01:38.524960+0000","last_undegraded":"2023-10-18T09:01:38.524960+0000","last_fullsized":"2023-10-18T09:01:38.524960+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:28:16.323058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"106'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867346+0000","last_change":"2023-10-18T09:01:36.543614+0000","last_active":"2023-10-18T09:01:38.867346+0000","last_peered":"2023-10-18T09:01:38.867346+0000","last_clean":"2023-10-18T09:01:38.867346+0000","last_became_active":"2023-10-18T09:01:36.543130+0000","last_became_peered":"2023-10-18T09:01:36.543130+0000","last_unstale":"2023-10-18T09:01:38.867346+0000","last_undegraded":"2023-10-18T09:01:38.867346+0000","last_fullsized":"2023-10-18T09:01:38.867346+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:12:17.149808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866820+0000","last_change":"2023-10-18T09:01:37.547888+0000","last_active":"2023-10-18T09:01:38.866820+0000","last_peered":"2023-10-18T09:01:38.866820+0000","last_clean":"2023-10-18T09:01:38.866820+0000","last_became_active":"2023-10-18T09:01:37.547666+0000","last_became_peered":"2023-10-18T09:01:37.547666+0000","last_unstale":"2023-10-18T09:01:38.866820+0000","last_undegraded":"2023-10-18T09:01:38.866820+0000","last_fullsized":"2023-10-18T09:01:38.866820+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:01:51.628601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"106'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866110+0000","last_change":"2023-10-18T09:01:36.542738+0000","last_active":"2023-10-18T09:01:38.866110+0000","last_peered":"2023-10-18T09:01:38.866110+0000","last_clean":"2023-10-18T09:01:38.866110+0000","last_became_active":"2023-10-18T09:01:36.542450+0000","last_became_peered":"2023-10-18T09:01:36.542450+0000","last_unstale":"2023-10-18T09:01:38.866110+0000","last_undegraded":"2023-10-18T09:01:38.866110+0000","last_fullsized":"2023-10-18T09:01:38.866110+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:11.953586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524847+0000","last_change":"2023-10-18T09:01:37.546633+0000","last_active":"2023-10-18T09:01:38.524847+0000","last_peered":"2023-10-18T09:01:38.524847+0000","last_clean":"2023-10-18T09:01:38.524847+0000","last_became_active":"2023-10-18T09:01:37.546228+0000","last_became_peered":"2023-10-18T09:01:37.546228+0000","last_unstale":"2023-10-18T09:01:38.524847+0000","last_undegraded":"2023-10-18T09:01:38.524847+0000","last_fullsized":"2023-10-18T09:01:38.524847+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:50:03.587917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866317+0000","last_change":"2023-10-18T09:01:36.540596+0000","last_active":"2023-10-18T09:01:38.866317+0000","last_peered":"2023-10-18T09:01:38.866317+0000","last_clean":"2023-10-18T09:01:38.866317+0000","last_became_active":"2023-10-18T09:01:36.540320+0000","last_became_peered":"2023-10-18T09:01:36.540320+0000","last_unstale":"2023-10-18T09:01:38.866317+0000","last_undegraded":"2023-10-18T09:01:38.866317+0000","last_fullsized":"2023-10-18T09:01:38.866317+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:26:01.238341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.525078+0000","last_change":"2023-10-18T09:01:37.535100+0000","last_active":"2023-10-18T09:01:38.525078+0000","last_peered":"2023-10-18T09:01:38.525078+0000","last_clean":"2023-10-18T09:01:38.525078+0000","last_became_active":"2023-10-18T09:01:37.534088+0000","last_became_peered":"2023-10-18T09:01:37.534088+0000","last_unstale":"2023-10-18T09:01:38.525078+0000","last_undegraded":"2023-10-18T09:01:38.525078+0000","last_fullsized":"2023-10-18T09:01:38.525078+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:50:03.587917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524418+0000","last_change":"2023-10-18T09:01:36.543284+0000","last_active":"2023-10-18T09:01:38.524418+0000","last_peered":"2023-10-18T09:01:38.524418+0000","last_clean":"2023-10-18T09:01:38.524418+0000","last_became_active":"2023-10-18T09:01:36.543011+0000","last_became_peered":"2023-10-18T09:01:36.543011+0000","last_unstale":"2023-10-18T09:01:38.524418+0000","last_undegraded":"2023-10-18T09:01:38.524418+0000","last_fullsized":"2023-10-18T09:01:38.524418+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:39:01.836488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.525397+0000","last_change":"2023-10-18T09:01:37.537846+0000","last_active":"2023-10-18T09:01:38.525397+0000","last_peered":"2023-10-18T09:01:38.525397+0000","last_clean":"2023-10-18T09:01:38.525397+0000","last_became_active":"2023-10-18T09:01:37.537285+0000","last_became_peered":"2023-10-18T09:01:37.537285+0000","last_unstale":"2023-10-18T09:01:38.525397+0000","last_undegraded":"2023-10-18T09:01:38.525397+0000","last_fullsized":"2023-10-18T09:01:38.525397+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:54:40.847787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868914+0000","last_change":"2023-10-18T09:01:36.531255+0000","last_active":"2023-10-18T09:01:38.868914+0000","last_peered":"2023-10-18T09:01:38.868914+0000","last_clean":"2023-10-18T09:01:38.868914+0000","last_became_active":"2023-10-18T09:01:36.530793+0000","last_became_peered":"2023-10-18T09:01:36.530793+0000","last_unstale":"2023-10-18T09:01:38.868914+0000","last_undegraded":"2023-10-18T09:01:38.868914+0000","last_fullsized":"2023-10-18T09:01:38.868914+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:07:41.726630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867709+0000","last_change":"2023-10-18T09:01:37.545288+0000","last_active":"2023-10-18T09:01:38.867709+0000","last_peered":"2023-10-18T09:01:38.867709+0000","last_clean":"2023-10-18T09:01:38.867709+0000","last_became_active":"2023-10-18T09:01:37.545069+0000","last_became_peered":"2023-10-18T09:01:37.545069+0000","last_unstale":"2023-10-18T09:01:38.867709+0000","last_undegraded":"2023-10-18T09:01:38.867709+0000","last_fullsized":"2023-10-18T09:01:38.867709+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:16.495132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867088+0000","last_change":"2023-10-18T09:01:36.544296+0000","last_active":"2023-10-18T09:01:38.867088+0000","last_peered":"2023-10-18T09:01:38.867088+0000","last_clean":"2023-10-18T09:01:38.867088+0000","last_became_active":"2023-10-18T09:01:36.543995+0000","last_became_peered":"2023-10-18T09:01:36.543995+0000","last_unstale":"2023-10-18T09:01:38.867088+0000","last_undegraded":"2023-10-18T09:01:38.867088+0000","last_fullsized":"2023-10-18T09:01:38.867088+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:41.862220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865922+0000","last_change":"2023-10-18T09:01:37.547316+0000","last_active":"2023-10-18T09:01:38.865922+0000","last_peered":"2023-10-18T09:01:38.865922+0000","last_clean":"2023-10-18T09:01:38.865922+0000","last_became_active":"2023-10-18T09:01:37.546568+0000","last_became_peered":"2023-10-18T09:01:37.546568+0000","last_unstale":"2023-10-18T09:01:38.865922+0000","last_undegraded":"2023-10-18T09:01:38.865922+0000","last_fullsized":"2023-10-18T09:01:38.865922+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:17.241941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"106'3","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868936+0000","last_change":"2023-10-18T09:01:36.535945+0000","last_active":"2023-10-18T09:01:38.868936+0000","last_peered":"2023-10-18T09:01:38.868936+0000","last_clean":"2023-10-18T09:01:38.868936+0000","last_became_active":"2023-10-18T09:01:36.535683+0000","last_became_peered":"2023-10-18T09:01:36.535683+0000","last_unstale":"2023-10-18T09:01:38.868936+0000","last_undegraded":"2023-10-18T09:01:38.868936+0000","last_fullsized":"2023-10-18T09:01:38.868936+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:07.838678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865499+0000","last_change":"2023-10-18T09:01:37.550449+0000","last_active":"2023-10-18T09:01:38.865499+0000","last_peered":"2023-10-18T09:01:38.865499+0000","last_clean":"2023-10-18T09:01:38.865499+0000","last_became_active":"2023-10-18T09:01:37.550089+0000","last_became_peered":"2023-10-18T09:01:37.550089+0000","last_unstale":"2023-10-18T09:01:38.865499+0000","last_undegraded":"2023-10-18T09:01:38.865499+0000","last_fullsized":"2023-10-18T09:01:38.865499+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:29.195475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"106'6","reported_seq":18,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868528+0000","last_change":"2023-10-18T09:01:36.538146+0000","last_active":"2023-10-18T09:01:38.868528+0000","last_peered":"2023-10-18T09:01:38.868528+0000","last_clean":"2023-10-18T09:01:38.868528+0000","last_became_active":"2023-10-18T09:01:36.537921+0000","last_became_peered":"2023-10-18T09:01:36.537921+0000","last_unstale":"2023-10-18T09:01:38.868528+0000","last_undegraded":"2023-10-18T09:01:38.868528+0000","last_fullsized":"2023-10-18T09:01:38.868528+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:27:14.103486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867256+0000","last_change":"2023-10-18T09:01:37.548459+0000","last_active":"2023-10-18T09:01:38.867256+0000","last_peered":"2023-10-18T09:01:38.867256+0000","last_clean":"2023-10-18T09:01:38.867256+0000","last_became_active":"2023-10-18T09:01:37.547761+0000","last_became_peered":"2023-10-18T09:01:37.547761+0000","last_unstale":"2023-10-18T09:01:38.867256+0000","last_undegraded":"2023-10-18T09:01:38.867256+0000","last_fullsized":"2023-10-18T09:01:38.867256+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:07:24.109273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"106'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524549+0000","last_change":"2023-10-18T09:01:36.529426+0000","last_active":"2023-10-18T09:01:38.524549+0000","last_peered":"2023-10-18T09:01:38.524549+0000","last_clean":"2023-10-18T09:01:38.524549+0000","last_became_active":"2023-10-18T09:01:36.528864+0000","last_became_peered":"2023-10-18T09:01:36.528864+0000","last_unstale":"2023-10-18T09:01:38.524549+0000","last_undegraded":"2023-10-18T09:01:38.524549+0000","last_fullsized":"2023-10-18T09:01:38.524549+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:04:38.542433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.525222+0000","last_change":"2023-10-18T09:01:37.546472+0000","last_active":"2023-10-18T09:01:38.525222+0000","last_peered":"2023-10-18T09:01:38.525222+0000","last_clean":"2023-10-18T09:01:38.525222+0000","last_became_active":"2023-10-18T09:01:37.545891+0000","last_became_peered":"2023-10-18T09:01:37.545891+0000","last_unstale":"2023-10-18T09:01:38.525222+0000","last_undegraded":"2023-10-18T09:01:38.525222+0000","last_fullsized":"2023-10-18T09:01:38.525222+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:54:40.847787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524475+0000","last_change":"2023-10-18T09:01:36.524365+0000","last_active":"2023-10-18T09:01:38.524475+0000","last_peered":"2023-10-18T09:01:38.524475+0000","last_clean":"2023-10-18T09:01:38.524475+0000","last_became_active":"2023-10-18T09:01:36.523666+0000","last_became_peered":"2023-10-18T09:01:36.523666+0000","last_unstale":"2023-10-18T09:01:38.524475+0000","last_undegraded":"2023-10-18T09:01:38.524475+0000","last_fullsized":"2023-10-18T09:01:38.524475+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:53.558330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524783+0000","last_change":"2023-10-18T09:01:37.547821+0000","last_active":"2023-10-18T09:01:38.524783+0000","last_peered":"2023-10-18T09:01:38.524783+0000","last_clean":"2023-10-18T09:01:38.524783+0000","last_became_active":"2023-10-18T09:01:37.547086+0000","last_became_peered":"2023-10-18T09:01:37.547086+0000","last_unstale":"2023-10-18T09:01:38.524783+0000","last_undegraded":"2023-10-18T09:01:38.524783+0000","last_fullsized":"2023-10-18T09:01:38.524783+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:28:43.060432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866434+0000","last_change":"2023-10-18T09:01:36.544368+0000","last_active":"2023-10-18T09:01:38.866434+0000","last_peered":"2023-10-18T09:01:38.866434+0000","last_clean":"2023-10-18T09:01:38.866434+0000","last_became_active":"2023-10-18T09:01:36.544115+0000","last_became_peered":"2023-10-18T09:01:36.544115+0000","last_unstale":"2023-10-18T09:01:38.866434+0000","last_undegraded":"2023-10-18T09:01:38.866434+0000","last_fullsized":"2023-10-18T09:01:38.866434+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:11:55.265579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524476+0000","last_change":"2023-10-18T09:01:37.545077+0000","last_active":"2023-10-18T09:01:38.524476+0000","last_peered":"2023-10-18T09:01:38.524476+0000","last_clean":"2023-10-18T09:01:38.524476+0000","last_became_active":"2023-10-18T09:01:37.544152+0000","last_became_peered":"2023-10-18T09:01:37.544152+0000","last_unstale":"2023-10-18T09:01:38.524476+0000","last_undegraded":"2023-10-18T09:01:38.524476+0000","last_fullsized":"2023-10-18T09:01:38.524476+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:37:46.837894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"106'8","reported_seq":20,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865186+0000","last_change":"2023-10-18T09:01:36.541137+0000","last_active":"2023-10-18T09:01:38.865186+0000","last_peered":"2023-10-18T09:01:38.865186+0000","last_clean":"2023-10-18T09:01:38.865186+0000","last_became_active":"2023-10-18T09:01:36.540762+0000","last_became_peered":"2023-10-18T09:01:36.540762+0000","last_unstale":"2023-10-18T09:01:38.865186+0000","last_undegraded":"2023-10-18T09:01:38.865186+0000","last_fullsized":"2023-10-18T09:01:38.865186+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:46.385819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.525843+0000","last_change":"2023-10-18T09:01:37.543318+0000","last_active":"2023-10-18T09:01:38.525843+0000","last_peered":"2023-10-18T09:01:38.525843+0000","last_clean":"2023-10-18T09:01:38.525843+0000","last_became_active":"2023-10-18T09:01:37.543015+0000","last_became_peered":"2023-10-18T09:01:37.543015+0000","last_unstale":"2023-10-18T09:01:38.525843+0000","last_undegraded":"2023-10-18T09:01:38.525843+0000","last_fullsized":"2023-10-18T09:01:38.525843+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:40.037987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"106'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867658+0000","last_change":"2023-10-18T09:01:36.541151+0000","last_active":"2023-10-18T09:01:38.867658+0000","last_peered":"2023-10-18T09:01:38.867658+0000","last_clean":"2023-10-18T09:01:38.867658+0000","last_became_active":"2023-10-18T09:01:36.540752+0000","last_became_peered":"2023-10-18T09:01:36.540752+0000","last_unstale":"2023-10-18T09:01:38.867658+0000","last_undegraded":"2023-10-18T09:01:38.867658+0000","last_fullsized":"2023-10-18T09:01:38.867658+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:54:21.788547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867412+0000","last_change":"2023-10-18T09:01:37.545042+0000","last_active":"2023-10-18T09:01:38.867412+0000","last_peered":"2023-10-18T09:01:38.867412+0000","last_clean":"2023-10-18T09:01:38.867412+0000","last_became_active":"2023-10-18T09:01:37.544672+0000","last_became_peered":"2023-10-18T09:01:37.544672+0000","last_unstale":"2023-10-18T09:01:38.867412+0000","last_undegraded":"2023-10-18T09:01:38.867412+0000","last_fullsized":"2023-10-18T09:01:38.867412+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:27.053204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"106'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866468+0000","last_change":"2023-10-18T09:01:36.543711+0000","last_active":"2023-10-18T09:01:38.866468+0000","last_peered":"2023-10-18T09:01:38.866468+0000","last_clean":"2023-10-18T09:01:38.866468+0000","last_became_active":"2023-10-18T09:01:36.543476+0000","last_became_peered":"2023-10-18T09:01:36.543476+0000","last_unstale":"2023-10-18T09:01:38.866468+0000","last_undegraded":"2023-10-18T09:01:38.866468+0000","last_fullsized":"2023-10-18T09:01:38.866468+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:56:41.458555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866767+0000","last_change":"2023-10-18T09:01:37.551901+0000","last_active":"2023-10-18T09:01:38.866767+0000","last_peered":"2023-10-18T09:01:38.866767+0000","last_clean":"2023-10-18T09:01:38.866767+0000","last_became_active":"2023-10-18T09:01:37.551513+0000","last_became_peered":"2023-10-18T09:01:37.551513+0000","last_unstale":"2023-10-18T09:01:38.866767+0000","last_undegraded":"2023-10-18T09:01:38.866767+0000","last_fullsized":"2023-10-18T09:01:38.866767+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:09.634600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865210+0000","last_change":"2023-10-18T09:01:36.535030+0000","last_active":"2023-10-18T09:01:38.865210+0000","last_peered":"2023-10-18T09:01:38.865210+0000","last_clean":"2023-10-18T09:01:38.865210+0000","last_became_active":"2023-10-18T09:01:36.534718+0000","last_became_peered":"2023-10-18T09:01:36.534718+0000","last_unstale":"2023-10-18T09:01:38.865210+0000","last_undegraded":"2023-10-18T09:01:38.865210+0000","last_fullsized":"2023-10-18T09:01:38.865210+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:22.638371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865818+0000","last_change":"2023-10-18T09:01:37.557396+0000","last_active":"2023-10-18T09:01:38.865818+0000","last_peered":"2023-10-18T09:01:38.865818+0000","last_clean":"2023-10-18T09:01:38.865818+0000","last_became_active":"2023-10-18T09:01:37.556462+0000","last_became_peered":"2023-10-18T09:01:37.556462+0000","last_unstale":"2023-10-18T09:01:38.865818+0000","last_undegraded":"2023-10-18T09:01:38.865818+0000","last_fullsized":"2023-10-18T09:01:38.865818+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:12:22.654018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"106'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524263+0000","last_change":"2023-10-18T09:01:36.543114+0000","last_active":"2023-10-18T09:01:38.524263+0000","last_peered":"2023-10-18T09:01:38.524263+0000","last_clean":"2023-10-18T09:01:38.524263+0000","last_became_active":"2023-10-18T09:01:36.542876+0000","last_became_peered":"2023-10-18T09:01:36.542876+0000","last_unstale":"2023-10-18T09:01:38.524263+0000","last_undegraded":"2023-10-18T09:01:38.524263+0000","last_fullsized":"2023-10-18T09:01:38.524263+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:13.995601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866123+0000","last_change":"2023-10-18T09:01:37.552671+0000","last_active":"2023-10-18T09:01:38.866123+0000","last_peered":"2023-10-18T09:01:38.866123+0000","last_clean":"2023-10-18T09:01:38.866123+0000","last_became_active":"2023-10-18T09:01:37.551974+0000","last_became_peered":"2023-10-18T09:01:37.551974+0000","last_unstale":"2023-10-18T09:01:38.866123+0000","last_undegraded":"2023-10-18T09:01:38.866123+0000","last_fullsized":"2023-10-18T09:01:38.866123+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:38:36.085141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865976+0000","last_change":"2023-10-18T09:01:36.537605+0000","last_active":"2023-10-18T09:01:38.865976+0000","last_peered":"2023-10-18T09:01:38.865976+0000","last_clean":"2023-10-18T09:01:38.865976+0000","last_became_active":"2023-10-18T09:01:36.537190+0000","last_became_peered":"2023-10-18T09:01:36.537190+0000","last_unstale":"2023-10-18T09:01:38.865976+0000","last_undegraded":"2023-10-18T09:01:38.865976+0000","last_fullsized":"2023-10-18T09:01:38.865976+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:16:02.527895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867030+0000","last_change":"2023-10-18T09:01:37.551826+0000","last_active":"2023-10-18T09:01:38.867030+0000","last_peered":"2023-10-18T09:01:38.867030+0000","last_clean":"2023-10-18T09:01:38.867030+0000","last_became_active":"2023-10-18T09:01:37.548572+0000","last_became_peered":"2023-10-18T09:01:37.548572+0000","last_unstale":"2023-10-18T09:01:38.867030+0000","last_undegraded":"2023-10-18T09:01:38.867030+0000","last_fullsized":"2023-10-18T09:01:38.867030+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:39:02.840893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866743+0000","last_change":"2023-10-18T09:01:36.537476+0000","last_active":"2023-10-18T09:01:38.866743+0000","last_peered":"2023-10-18T09:01:38.866743+0000","last_clean":"2023-10-18T09:01:38.866743+0000","last_became_active":"2023-10-18T09:01:36.537161+0000","last_became_peered":"2023-10-18T09:01:36.537161+0000","last_unstale":"2023-10-18T09:01:38.866743+0000","last_undegraded":"2023-10-18T09:01:38.866743+0000","last_fullsized":"2023-10-18T09:01:38.866743+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:05:17.734155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866645+0000","last_change":"2023-10-18T09:01:37.546947+0000","last_active":"2023-10-18T09:01:38.866645+0000","last_peered":"2023-10-18T09:01:38.866645+0000","last_clean":"2023-10-18T09:01:38.866645+0000","last_became_active":"2023-10-18T09:01:37.546020+0000","last_became_peered":"2023-10-18T09:01:37.546020+0000","last_unstale":"2023-10-18T09:01:38.866645+0000","last_undegraded":"2023-10-18T09:01:38.866645+0000","last_fullsized":"2023-10-18T09:01:38.866645+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:12:18.154213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524611+0000","last_change":"2023-10-18T09:01:36.540049+0000","last_active":"2023-10-18T09:01:38.524611+0000","last_peered":"2023-10-18T09:01:38.524611+0000","last_clean":"2023-10-18T09:01:38.524611+0000","last_became_active":"2023-10-18T09:01:36.538963+0000","last_became_peered":"2023-10-18T09:01:36.538963+0000","last_unstale":"2023-10-18T09:01:38.524611+0000","last_undegraded":"2023-10-18T09:01:38.524611+0000","last_fullsized":"2023-10-18T09:01:38.524611+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:34:36.331821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865868+0000","last_change":"2023-10-18T09:01:37.550302+0000","last_active":"2023-10-18T09:01:38.865868+0000","last_peered":"2023-10-18T09:01:38.865868+0000","last_clean":"2023-10-18T09:01:38.865868+0000","last_became_active":"2023-10-18T09:01:37.549458+0000","last_became_peered":"2023-10-18T09:01:37.549458+0000","last_unstale":"2023-10-18T09:01:38.865868+0000","last_undegraded":"2023-10-18T09:01:38.865868+0000","last_fullsized":"2023-10-18T09:01:38.865868+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:08.381428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"106'2","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524906+0000","last_change":"2023-10-18T09:01:36.542371+0000","last_active":"2023-10-18T09:01:38.524906+0000","last_peered":"2023-10-18T09:01:38.524906+0000","last_clean":"2023-10-18T09:01:38.524906+0000","last_became_active":"2023-10-18T09:01:36.542052+0000","last_became_peered":"2023-10-18T09:01:36.542052+0000","last_unstale":"2023-10-18T09:01:38.524906+0000","last_undegraded":"2023-10-18T09:01:38.524906+0000","last_fullsized":"2023-10-18T09:01:38.524906+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:04:54.914558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524596+0000","last_change":"2023-10-18T09:01:37.541636+0000","last_active":"2023-10-18T09:01:38.524596+0000","last_peered":"2023-10-18T09:01:38.524596+0000","last_clean":"2023-10-18T09:01:38.524596+0000","last_became_active":"2023-10-18T09:01:37.541280+0000","last_became_peered":"2023-10-18T09:01:37.541280+0000","last_unstale":"2023-10-18T09:01:38.524596+0000","last_undegraded":"2023-10-18T09:01:38.524596+0000","last_fullsized":"2023-10-18T09:01:38.524596+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:15.000006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.525078+0000","last_change":"2023-10-18T09:01:36.530909+0000","last_active":"2023-10-18T09:01:38.525078+0000","last_peered":"2023-10-18T09:01:38.525078+0000","last_clean":"2023-10-18T09:01:38.525078+0000","last_became_active":"2023-10-18T09:01:36.530299+0000","last_became_peered":"2023-10-18T09:01:36.530299+0000","last_unstale":"2023-10-18T09:01:38.525078+0000","last_undegraded":"2023-10-18T09:01:38.525078+0000","last_fullsized":"2023-10-18T09:01:38.525078+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:51:09.996176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868903+0000","last_change":"2023-10-18T09:01:37.547142+0000","last_active":"2023-10-18T09:01:38.868903+0000","last_peered":"2023-10-18T09:01:38.868903+0000","last_clean":"2023-10-18T09:01:38.868903+0000","last_became_active":"2023-10-18T09:01:37.546805+0000","last_became_peered":"2023-10-18T09:01:37.546805+0000","last_unstale":"2023-10-18T09:01:38.868903+0000","last_undegraded":"2023-10-18T09:01:38.868903+0000","last_fullsized":"2023-10-18T09:01:38.868903+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:02:48.470528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867868+0000","last_change":"2023-10-18T09:01:36.540982+0000","last_active":"2023-10-18T09:01:38.867868+0000","last_peered":"2023-10-18T09:01:38.867868+0000","last_clean":"2023-10-18T09:01:38.867868+0000","last_became_active":"2023-10-18T09:01:36.539000+0000","last_became_peered":"2023-10-18T09:01:36.539000+0000","last_unstale":"2023-10-18T09:01:38.867868+0000","last_undegraded":"2023-10-18T09:01:38.867868+0000","last_fullsized":"2023-10-18T09:01:38.867868+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:39:44.899385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.865375+0000","last_change":"2023-10-18T09:01:37.549748+0000","last_active":"2023-10-18T09:01:38.865375+0000","last_peered":"2023-10-18T09:01:38.865375+0000","last_clean":"2023-10-18T09:01:38.865375+0000","last_became_active":"2023-10-18T09:01:37.549202+0000","last_became_peered":"2023-10-18T09:01:37.549202+0000","last_unstale":"2023-10-18T09:01:38.865375+0000","last_undegraded":"2023-10-18T09:01:38.865375+0000","last_fullsized":"2023-10-18T09:01:38.865375+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:35:47.510705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.869040+0000","last_change":"2023-10-18T09:01:36.533876+0000","last_active":"2023-10-18T09:01:38.869040+0000","last_peered":"2023-10-18T09:01:38.869040+0000","last_clean":"2023-10-18T09:01:38.869040+0000","last_became_active":"2023-10-18T09:01:36.533485+0000","last_became_peered":"2023-10-18T09:01:36.533485+0000","last_unstale":"2023-10-18T09:01:38.869040+0000","last_undegraded":"2023-10-18T09:01:38.869040+0000","last_fullsized":"2023-10-18T09:01:38.869040+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:56:05.956978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.866084+0000","last_change":"2023-10-18T09:01:37.549720+0000","last_active":"2023-10-18T09:01:38.866084+0000","last_peered":"2023-10-18T09:01:38.866084+0000","last_clean":"2023-10-18T09:01:38.866084+0000","last_became_active":"2023-10-18T09:01:37.549372+0000","last_became_peered":"2023-10-18T09:01:37.549372+0000","last_unstale":"2023-10-18T09:01:38.866084+0000","last_undegraded":"2023-10-18T09:01:38.866084+0000","last_fullsized":"2023-10-18T09:01:38.866084+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:15:16.985894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868017+0000","last_change":"2023-10-18T09:01:36.542502+0000","last_active":"2023-10-18T09:01:38.868017+0000","last_peered":"2023-10-18T09:01:38.868017+0000","last_clean":"2023-10-18T09:01:38.868017+0000","last_became_active":"2023-10-18T09:01:36.542275+0000","last_became_peered":"2023-10-18T09:01:36.542275+0000","last_unstale":"2023-10-18T09:01:38.868017+0000","last_undegraded":"2023-10-18T09:01:38.868017+0000","last_fullsized":"2023-10-18T09:01:38.868017+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:46:53.225811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867560+0000","last_change":"2023-10-18T09:01:37.548270+0000","last_active":"2023-10-18T09:01:38.867560+0000","last_peered":"2023-10-18T09:01:38.867560+0000","last_clean":"2023-10-18T09:01:38.867560+0000","last_became_active":"2023-10-18T09:01:37.547247+0000","last_became_peered":"2023-10-18T09:01:37.547247+0000","last_unstale":"2023-10-18T09:01:38.867560+0000","last_undegraded":"2023-10-18T09:01:38.867560+0000","last_fullsized":"2023-10-18T09:01:38.867560+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:15.000006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867906+0000","last_change":"2023-10-18T09:01:36.543486+0000","last_active":"2023-10-18T09:01:38.867906+0000","last_peered":"2023-10-18T09:01:38.867906+0000","last_clean":"2023-10-18T09:01:38.867906+0000","last_became_active":"2023-10-18T09:01:36.542827+0000","last_became_peered":"2023-10-18T09:01:36.542827+0000","last_unstale":"2023-10-18T09:01:38.867906+0000","last_undegraded":"2023-10-18T09:01:38.867906+0000","last_fullsized":"2023-10-18T09:01:38.867906+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:41.862220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524675+0000","last_change":"2023-10-18T09:01:37.527193+0000","last_active":"2023-10-18T09:01:38.524675+0000","last_peered":"2023-10-18T09:01:38.524675+0000","last_clean":"2023-10-18T09:01:38.524675+0000","last_became_active":"2023-10-18T09:01:37.526570+0000","last_became_peered":"2023-10-18T09:01:37.526570+0000","last_unstale":"2023-10-18T09:01:38.524675+0000","last_undegraded":"2023-10-18T09:01:38.524675+0000","last_fullsized":"2023-10-18T09:01:38.524675+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:57.229192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.869163+0000","last_change":"2023-10-18T09:01:36.531999+0000","last_active":"2023-10-18T09:01:38.869163+0000","last_peered":"2023-10-18T09:01:38.869163+0000","last_clean":"2023-10-18T09:01:38.869163+0000","last_became_active":"2023-10-18T09:01:36.531378+0000","last_became_peered":"2023-10-18T09:01:36.531378+0000","last_unstale":"2023-10-18T09:01:38.869163+0000","last_undegraded":"2023-10-18T09:01:38.869163+0000","last_fullsized":"2023-10-18T09:01:38.869163+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:35:46.506299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.868685+0000","last_change":"2023-10-18T09:01:37.553633+0000","last_active":"2023-10-18T09:01:38.868685+0000","last_peered":"2023-10-18T09:01:38.868685+0000","last_clean":"2023-10-18T09:01:38.868685+0000","last_became_active":"2023-10-18T09:01:37.549651+0000","last_became_peered":"2023-10-18T09:01:37.549651+0000","last_unstale":"2023-10-18T09:01:38.868685+0000","last_undegraded":"2023-10-18T09:01:38.868685+0000","last_fullsized":"2023-10-18T09:01:38.868685+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:47.390225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.867501+0000","last_change":"2023-10-18T09:01:36.543327+0000","last_active":"2023-10-18T09:01:38.867501+0000","last_peered":"2023-10-18T09:01:38.867501+0000","last_clean":"2023-10-18T09:01:38.867501+0000","last_became_active":"2023-10-18T09:01:36.542413+0000","last_became_peered":"2023-10-18T09:01:36.542413+0000","last_unstale":"2023-10-18T09:01:38.867501+0000","last_undegraded":"2023-10-18T09:01:38.867501+0000","last_fullsized":"2023-10-18T09:01:38.867501+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:15:15.981489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.524782+0000","last_change":"2023-10-18T09:01:37.545392+0000","last_active":"2023-10-18T09:01:38.524782+0000","last_peered":"2023-10-18T09:01:38.524782+0000","last_clean":"2023-10-18T09:01:38.524782+0000","last_became_active":"2023-10-18T09:01:37.544830+0000","last_became_peered":"2023-10-18T09:01:37.544830+0000","last_unstale":"2023-10-18T09:01:38.524782+0000","last_undegraded":"2023-10-18T09:01:38.524782+0000","last_fullsized":"2023-10-18T09:01:38.524782+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:36.507901+0000","last_clean_scrub_stamp":"2023-10-18T09:01:36.507901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:54:24.998405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"106'1","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-10-18T09:01:38.525082+0000","last_change":"2023-10-18T09:01:36.544488+0000","last_active":"2023-10-18T09:01:38.525082+0000","last_peered":"2023-10-18T09:01:38.525082+0000","last_clean":"2023-10-18T09:01:38.525082+0000","last_became_active":"2023-10-18T09:01:36.544271+0000","last_became_peered":"2023-10-18T09:01:36.544271+0000","last_unstale":"2023-10-18T09:01:38.525082+0000","last_undegraded":"2023-10-18T09:01:38.525082+0000","last_fullsized":"2023-10-18T09:01:38.525082+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:01:35.503496+0000","last_clean_scrub_stamp":"2023-10-18T09:01:35.503496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:46:53.225811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":262,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-18T09:01:37.522739+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:01:37.522739+0000","last_peered":"2023-10-18T09:01:37.522739+0000","last_clean":"2023-10-18T09:01:37.522739+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:01:37.522739+0000","last_undegraded":"2023-10-18T09:01:37.522739+0000","last_fullsized":"2023-10-18T09:01:37.522739+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:18:36.913912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1150,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194162,"statfs":{"total":200000000,"available":198822253,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.101,"15min":1.0249999999999999},"min":{"1min":0.77800000000000002,"5min":0.67800000000000005,"15min":0.60999999999999999},"max":{"1min":1.5669999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.135,"15min":1.042},"min":{"1min":0.79200000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6439999999999999,"5min":1.97,"15min":1.97},"last":0.63100000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.145,"15min":1.1539999999999999},"min":{"1min":0.79100000000000004,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5189999999999999,"5min":1.5900000000000001,"15min":1.8109999999999999},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.163,"15min":1.129},"min":{"1min":0.61099999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5469999999999999,"5min":1.7849999999999999,"15min":1.7989999999999999},"last":1.0720000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.302,"15min":1.252},"min":{"1min":0.93500000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.7050000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.264,"15min":1.212},"min":{"1min":0.98599999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.8049999999999999,"5min":1.8320000000000001,"15min":1.8320000000000001},"last":1.0169999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.27,"15min":1.266},"min":{"1min":0.96599999999999997,"5min":0.80300000000000005,"15min":0.76700000000000002},"max":{"1min":1.744,"5min":1.845,"15min":1.9430000000000001},"last":1.153},{"interface":"front","average":{"1min":1.359,"5min":1.3580000000000001,"15min":1.3300000000000001},"min":{"1min":0.998,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.756,"5min":1.873,"15min":1.9930000000000001},"last":1.365}]},{"osd":4,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.27,"15min":1.1859999999999999},"min":{"1min":0.90400000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.8939999999999999,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3300000000000001,"15min":1.385},"min":{"1min":0.84499999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.659,"5min":1.885,"15min":2.5169999999999999},"last":1.425}]},{"osd":5,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.361,"15min":1.3560000000000001},"min":{"1min":0.94499999999999995,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.831,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.429,"15min":1.425},"min":{"1min":1.1200000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.0259999999999998,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.2869999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.502,"15min":1.47},"min":{"1min":1.224,"5min":0.97899999999999998,"15min":0.95899999999999996},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.3959999999999999},"last":1.454},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.4830000000000001,"15min":1.5069999999999999},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.81599999999999995},"max":{"1min":1.9610000000000001,"5min":1.978,"15min":2.0590000000000002},"last":1.651}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":568,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194744,"statfs":{"total":200000000,"available":199418297,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:01:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2110000000000001,"15min":1.117},"min":{"1min":0.64700000000000002,"5min":0.623,"15min":0.60299999999999998},"max":{"1min":1.6240000000000001,"5min":1.651,"15min":1.6850000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.2989999999999999,"15min":1.2569999999999999},"min":{"1min":0.93200000000000005,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.018}]},{"osd":1,"last update":"Wed Oct 18 09:01:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.109,"15min":1.091},"min":{"1min":0.81999999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.4950000000000001,"5min":1.633,"15min":1.633},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.2190000000000001,"15min":1.1990000000000001},"min":{"1min":0.96599999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.8560000000000001,"5min":1.962,"15min":2.9609999999999999},"last":1.139}]},{"osd":2,"last update":"Wed Oct 18 09:01:09 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.1930000000000001,"15min":1.1839999999999999},"min":{"1min":0.75800000000000001,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.347,"5min":1.248,"15min":1.2110000000000001},"min":{"1min":0.98699999999999999,"5min":0.82099999999999995,"15min":0.66300000000000003},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.873}]},{"osd":3,"last update":"Wed Oct 18 09:01:09 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.2749999999999999,"15min":1.2649999999999999},"min":{"1min":1.0009999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.097},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.2669999999999999,"15min":1.2330000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.6679999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.587}]},{"osd":4,"last update":"Wed Oct 18 09:01:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.238,"15min":1.1839999999999999},"min":{"1min":0.86199999999999999,"5min":0.625,"15min":0.625},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.2809999999999999,"15min":1.2729999999999999},"min":{"1min":0.89700000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":0.97399999999999998}]},{"osd":5,"last update":"Wed Oct 18 09:01:09 2023","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.4179999999999999,"15min":1.3700000000000001},"min":{"1min":1.171,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.294,"15min":1.2370000000000001},"min":{"1min":0.90300000000000002,"5min":0.54900000000000004,"15min":0.49199999999999999},"max":{"1min":1.806,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.5349999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:01:09 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.4119999999999999,"15min":1.399},"min":{"1min":1.0289999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.3540000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.633,"5min":1.5109999999999999,"15min":1.5249999999999999},"min":{"1min":1.373,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.794},"last":1.24}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":568,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194743,"statfs":{"total":200000000,"available":199417357,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:01:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.3720000000000001,"15min":1.242},"min":{"1min":0.73899999999999999,"5min":0.68300000000000005,"15min":0.63400000000000001},"max":{"1min":1.6970000000000001,"5min":9.3539999999999992,"15min":9.3539999999999992},"last":9.609},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3300000000000001,"15min":1.306},"min":{"1min":0.751,"5min":0.71999999999999997,"15min":0.60799999999999998},"max":{"1min":1.623,"5min":5.3029999999999999,"15min":5.3029999999999999},"last":1.4399999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:01:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.1919999999999999,"15min":1.218},"min":{"1min":0.82699999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.7669999999999999,"5min":2.7360000000000002,"15min":2.7360000000000002},"last":11.68},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.1890000000000001,"15min":1.2370000000000001},"min":{"1min":0.94699999999999995,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6639999999999999,"5min":2.367,"15min":3.7050000000000001},"last":1.2629999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:01:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2490000000000001,"15min":1.2609999999999999},"min":{"1min":0.93999999999999995,"5min":0.77700000000000002,"15min":0.755},"max":{"1min":1.7749999999999999,"5min":2.0350000000000001,"15min":2.2090000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.232,"5min":1.2509999999999999,"15min":1.28},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.72399999999999998},"max":{"1min":1.72,"5min":2.3069999999999999,"15min":3.8180000000000001},"last":1.3200000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:01:14 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.234,"15min":1.2410000000000001},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.65400000000000003},"max":{"1min":2.6139999999999999,"5min":3.524,"15min":4.7119999999999997},"last":11.797000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.365,"15min":1.341},"min":{"1min":1.0900000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.633,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.747}]},{"osd":4,"last update":"Wed Oct 18 09:01:14 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3879999999999999,"15min":1.4099999999999999},"min":{"1min":1.0429999999999999,"5min":0.84899999999999998,"15min":0.75900000000000001},"max":{"1min":1.778,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":2.0680000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.3200000000000001,"15min":1.474},"min":{"1min":0.91500000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.7130000000000001,"5min":4.4429999999999996,"15min":5.4580000000000002},"last":1.1639999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:01:14 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.2869999999999999,"15min":1.3819999999999999},"min":{"1min":0.80300000000000005,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.5190000000000001,"5min":4.5469999999999997,"15min":5.141},"last":1.96},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4870000000000001,"15min":1.5640000000000001},"min":{"1min":1.0940000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.718,"5min":2.657,"15min":2.657},"last":0.82499999999999996}]},{"osd":7,"last update":"Wed Oct 18 09:01:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.4890000000000001,"15min":1.619},"min":{"1min":0.90300000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.762,"5min":4.2050000000000001,"15min":5.5490000000000004},"last":11.829000000000001},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.448,"15min":1.502},"min":{"1min":0.998,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.9420000000000002,"5min":4.4690000000000003,"15min":4.8419999999999996},"last":1.1220000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":566,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194746,"statfs":{"total":200000000,"available":199420010,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.129,"15min":1.117},"min":{"1min":0.66700000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.353},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.109,"15min":1.163},"min":{"1min":0.72499999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.4970000000000001,"5min":2.0659999999999998,"15min":2.2599999999999998},"last":1.252}]},{"osd":1,"last update":"Wed Oct 18 09:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.1120000000000001,"15min":1.1000000000000001},"min":{"1min":0.72299999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.5269999999999999,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":1.552},{"interface":"front","average":{"1min":1.149,"5min":1.173,"15min":1.222},"min":{"1min":0.77800000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.6259999999999999,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.204}]},{"osd":2,"last update":"Wed Oct 18 09:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1930000000000001,"15min":1.181},"min":{"1min":0.89000000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.468,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":1.496},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.2669999999999999,"15min":1.302},"min":{"1min":0.88200000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.837,"5min":1.994,"15min":2.3650000000000002},"last":1.4690000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.26,"15min":1.292},"min":{"1min":0.78800000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.669,"5min":2.649,"15min":2.649},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.397,"15min":1.407},"min":{"1min":1.05,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.728,"5min":2.6880000000000002,"15min":2.8010000000000002},"last":1.3140000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.296,"15min":1.2949999999999999},"min":{"1min":0.85099999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.978,"5min":1.978,"15min":1.978},"last":0.995},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.286,"15min":1.3460000000000001},"min":{"1min":0.83899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.5369999999999999,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.391}]},{"osd":6,"last update":"Wed Oct 18 09:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.427,"15min":1.3919999999999999},"min":{"1min":0.95399999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.788,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.145},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.472,"15min":1.546},"min":{"1min":1.167,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2.085,"5min":4.407,"15min":4.407},"last":1.5269999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:01:15 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.4359999999999999,"15min":1.494},"min":{"1min":1.147,"5min":0.438,"15min":0.438},"max":{"1min":1.7090000000000001,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.484,"5min":1.526,"15min":1.542},"min":{"1min":1.0980000000000001,"5min":0.747,"15min":0.747},"max":{"1min":1.952,"5min":2.585,"15min":2.585},"last":1.4370000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":575,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194737,"statfs":{"total":200000000,"available":199411079,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.95599999999999996,"15min":0.80900000000000005},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.466,"5min":5.5730000000000004,"15min":5.5730000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0509999999999999,"15min":0.90600000000000003},"min":{"1min":0.48599999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.4569999999999999,"5min":6.5599999999999996,"15min":6.5599999999999996},"last":0.56000000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0640000000000001,"15min":0.96999999999999997},"min":{"1min":0.59699999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.645,"5min":6.8099999999999996,"15min":6.8099999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":1.369,"5min":1.2789999999999999,"15min":1.159},"min":{"1min":0.80600000000000005,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.698,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.0549999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1259999999999999,"15min":1.038},"min":{"1min":0.63100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.0600000000000001,"5min":7.4580000000000002,"15min":7.4580000000000002},"last":0.623},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.216,"15min":1.0900000000000001},"min":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.51400000000000001},"max":{"1min":3.169,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.53,"15min":1.327},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.897,"5min":18.664999999999999,"15min":18.664999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.421,"5min":1.3340000000000001,"15min":1.1919999999999999},"min":{"1min":0.877,"5min":0.69099999999999995,"15min":0.64000000000000001},"max":{"1min":1.99,"5min":7.0700000000000003,"15min":7.0700000000000003},"last":1.008}]},{"osd":5,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3540000000000001,"15min":1.2430000000000001},"min":{"1min":0.83199999999999996,"5min":0.72399999999999998,"15min":0.67700000000000005},"max":{"1min":1.998,"5min":6.9139999999999997,"15min":6.9139999999999997},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.53,"5min":1.6240000000000001,"15min":1.381},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.0179999999999998,"5min":16.390999999999998,"15min":16.390999999999998},"last":1.087}]},{"osd":6,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.5449999999999999,"15min":1.4430000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.129,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.5269999999999999,"15min":1.3740000000000001},"min":{"1min":1.2669999999999999,"5min":0.77200000000000002,"15min":0.60199999999999998},"max":{"1min":2.3759999999999999,"5min":7.242,"15min":7.242},"last":0.91400000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.5449999999999999,"15min":1.3799999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.899,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.645,"15min":1.4770000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.319,"5min":12.946,"15min":12.946},"last":1.1379999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":567,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194744,"statfs":{"total":200000000,"available":199418409,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.86799999999999999,"15min":0.92300000000000004},"min":{"1min":0.58899999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.478,"5min":1.706,"15min":4.2210000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":0.96399999999999997,"15min":1.0369999999999999},"min":{"1min":0.55900000000000005,"5min":0.48999999999999999,"15min":0.44500000000000001},"max":{"1min":1.637,"5min":1.637,"15min":1.651},"last":1.494}]},{"osd":1,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.038,"15min":1.0589999999999999},"min":{"1min":0.64100000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":4.29},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1180000000000001,"15min":1.109},"min":{"1min":0.65000000000000002,"5min":0.57499999999999996,"15min":0.46200000000000002},"max":{"1min":2.1000000000000001,"5min":2.1000000000000001,"15min":2.5350000000000001},"last":1.1930000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.0980000000000001,"15min":1.1299999999999999},"min":{"1min":0.63600000000000001,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.915,"5min":1.915,"15min":4.0030000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.496,"5min":1.339,"15min":1.4630000000000001},"min":{"1min":0.69199999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":7.5570000000000004},"last":1.1439999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":1.409,"15min":1.482},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.4119999999999999,"5min":2.4119999999999999,"15min":7.6189999999999998},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.4419999999999999,"15min":1.4590000000000001},"min":{"1min":1.1919999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":2.7749999999999999,"5min":2.7749999999999999,"15min":2.7749999999999999},"last":1.3}]},{"osd":5,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.272,"15min":1.28},"min":{"1min":1.087,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":4.4320000000000004},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.6579999999999999,"5min":1.4550000000000001,"15min":1.5609999999999999},"min":{"1min":0.997,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.536,"5min":2.536,"15min":7.657},"last":1.4750000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.651,"5min":1.4610000000000001,"15min":1.5269999999999999},"min":{"1min":1.0960000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.8780000000000001,"5min":2.8780000000000001,"15min":3.5299999999999998},"last":1.8480000000000001},{"interface":"front","average":{"1min":1.603,"5min":1.873,"15min":1.589},"min":{"1min":1.0720000000000001,"5min":0.94099999999999995,"15min":0.77600000000000002},"max":{"1min":2.4830000000000001,"5min":37.771999999999998,"15min":37.771999999999998},"last":1.4359999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:01:13 2023","interfaces":[{"interface":"back","average":{"1min":1.704,"5min":1.528,"15min":1.5720000000000001},"min":{"1min":1.262,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.5979999999999999,"5min":2.5979999999999999,"15min":3.3620000000000001},"last":1.9119999999999999},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.5029999999999999,"15min":1.492},"min":{"1min":1.147,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.6960000000000002,"5min":2.6960000000000002,"15min":4.3390000000000004},"last":1.5620000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":572,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194739,"statfs":{"total":200000000,"available":199413516,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:01:18 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.86899999999999999,"15min":0.82299999999999995},"min":{"1min":0.34499999999999997,"5min":0.315,"15min":0.315},"max":{"1min":1.5589999999999999,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.622},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0229999999999999,"15min":0.93500000000000005},"min":{"1min":0.52800000000000002,"5min":0.501,"15min":0.443},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.86799999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:01:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.98899999999999999,"15min":0.94899999999999995},"min":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.859},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.92900000000000005,"15min":0.86699999999999999},"min":{"1min":0.44800000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.601,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.3879999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:01:18 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.0529999999999999,"15min":1.0309999999999999},"min":{"1min":0.49199999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.6040000000000001,"5min":2.1059999999999999,"15min":2.1059999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":3.7010000000000001},"last":1.2150000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:01:18 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.2989999999999999,"15min":1.2609999999999999},"min":{"1min":0.91900000000000004,"5min":0.71499999999999997,"15min":0.59999999999999998},"max":{"1min":1.909,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.296,"15min":1.226},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.8520000000000001,"5min":1.99,"15min":2.0470000000000002},"last":1.681}]},{"osd":5,"last update":"Wed Oct 18 09:01:18 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.3560000000000001,"15min":1.3100000000000001},"min":{"1min":0.93700000000000006,"5min":0.82899999999999996,"15min":0.65400000000000003},"max":{"1min":1.7230000000000001,"5min":2.161,"15min":3.5750000000000002},"last":1.504},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4239999999999999,"15min":1.3700000000000001},"min":{"1min":0.97799999999999998,"5min":0.84999999999999998,"15min":0.83899999999999997},"max":{"1min":1.907,"5min":1.964,"15min":2.3769999999999998},"last":1.45}]},{"osd":6,"last update":"Wed Oct 18 09:01:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.363,"15min":1.302},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":1.8280000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.373,"15min":1.2909999999999999},"min":{"1min":0.99099999999999999,"5min":0.72999999999999998,"15min":0.63100000000000001},"max":{"1min":1.859,"5min":2.121,"15min":2.121},"last":1.631}]},{"osd":7,"last update":"Wed Oct 18 09:01:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4450000000000001,"15min":1.3759999999999999},"min":{"1min":0.90900000000000003,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.0579999999999998,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.728},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.468,"15min":1.383},"min":{"1min":0.82299999999999995,"5min":0.746,"15min":0.746},"max":{"1min":2.2290000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.5489999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1149,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194163,"statfs":{"total":200000000,"available":198823308,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81599999999999995,"15min":0.91000000000000003},"min":{"1min":0.51700000000000002,"5min":0.37,"15min":0.37},"max":{"1min":1.1890000000000001,"5min":1.25,"15min":3.4420000000000002},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.0920000000000001,"15min":1.0880000000000001},"min":{"1min":0.90000000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.262}]},{"osd":2,"last update":"Wed Oct 18 09:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86499999999999999,"15min":0.85099999999999998},"min":{"1min":0.58499999999999996,"5min":0.436,"15min":0.436},"max":{"1min":1.3069999999999999,"5min":1.4159999999999999,"15min":1.5029999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.181,"5min":1.1539999999999999,"15min":1.135},"min":{"1min":0.73299999999999998,"5min":0.437,"15min":0.437},"max":{"1min":1.55,"5min":1.929,"15min":1.929},"last":0.94999999999999996}]},{"osd":3,"last update":"Wed Oct 18 09:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1399999999999999,"15min":1.2789999999999999},"min":{"1min":0.80000000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.788,"5min":1.8200000000000001,"15min":2.7639999999999998},"last":1.113},{"interface":"front","average":{"1min":1.488,"5min":1.4039999999999999,"15min":1.5069999999999999},"min":{"1min":0.84199999999999997,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.8779999999999999,"5min":2.3639999999999999,"15min":3.5720000000000001},"last":1.7250000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2230000000000001,"15min":1.3029999999999999},"min":{"1min":0.86599999999999999,"5min":0.72899999999999998,"15min":0.72399999999999998},"max":{"1min":1.8109999999999999,"5min":1.8819999999999999,"15min":4.492},"last":1.671},{"interface":"front","average":{"1min":1.288,"5min":1.2889999999999999,"15min":1.26},"min":{"1min":0.95599999999999996,"5min":0.78500000000000003,"15min":0.72599999999999998},"max":{"1min":1.585,"5min":1.726,"15min":1.726},"last":1.4079999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2569999999999999,"15min":1.409},"min":{"1min":0.93799999999999994,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.675,"5min":2.4510000000000001,"15min":4.2779999999999996},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3200000000000001,"15min":1.373},"min":{"1min":0.94499999999999995,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.722,"5min":1.8240000000000001,"15min":1.8740000000000001},"last":1.5900000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.464,"15min":1.577},"min":{"1min":1.2729999999999999,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.881,"5min":2.0419999999999998,"15min":3.5299999999999998},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.393,"15min":1.494},"min":{"1min":0.95699999999999996,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.73,"5min":2.1059999999999999,"15min":4.4349999999999996},"last":1.1799999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:01:16 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.5509999999999999,"15min":1.639},"min":{"1min":1.036,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.956,"5min":2.2000000000000002,"15min":3.492},"last":1.7929999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4490000000000001,"15min":1.655},"min":{"1min":0.995,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.853,"5min":2.0449999999999999,"15min":4.5419999999999998},"last":1.4830000000000001}]}]}],"pool_statfs":[]}} 2023-10-18T09:01:42.603 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T09:01:42.603 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T09:01:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.883+0000 7f8b3d213640 1 -- 172.21.15.40:0/348960954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b3815ed60 msgr2=0x7f8b380affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.883+0000 7f8b3d213640 1 --2- 172.21.15.40:0/348960954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b3815ed60 0x7f8b380affb0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f8b3808b590 tx=0x7f8b2802c1d0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.883+0000 7f8b3d213640 1 -- 172.21.15.40:0/348960954 shutdown_connections 2023-10-18T09:01:42.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.883+0000 7f8b3d213640 1 --2- 172.21.15.40:0/348960954 >> v2:172.21.15.144:3300/0 conn(0x7f8b380ce040 0x7f8b380cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.883+0000 7f8b3d213640 1 --2- 172.21.15.40:0/348960954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b3815ed60 0x7f8b380affb0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.883+0000 7f8b3d213640 1 -- 172.21.15.40:0/348960954 >> 172.21.15.40:0/348960954 conn(0x7f8b3809a580 msgr2=0x7f8b380d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:42.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.883+0000 7f8b3d213640 1 -- 172.21.15.40:0/348960954 shutdown_connections 2023-10-18T09:01:42.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.883+0000 7f8b3d213640 1 -- 172.21.15.40:0/348960954 wait complete. 2023-10-18T09:01:42.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.885+0000 7f8b3d213640 1 Processor -- start 2023-10-18T09:01:42.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.885+0000 7f8b3d213640 1 -- start start 2023-10-18T09:01:42.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.886+0000 7f8b3d213640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b380cdbc0 0x7f8b380c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:42.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.886+0000 7f8b3d213640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b380ce040 0x7f8b3815a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:42.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.886+0000 7f8b3d213640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b3815ed60 0x7f8b380aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:42.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.886+0000 7f8b3d213640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b380c6ce0 con 0x7f8b3815ed60 2023-10-18T09:01:42.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.886+0000 7f8b3d213640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b380c74c0 con 0x7f8b380ce040 2023-10-18T09:01:42.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.886+0000 7f8b3d213640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b380b54b0 con 0x7f8b380cdbc0 2023-10-18T09:01:42.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.887+0000 7f8b3ca12640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b3815ed60 0x7f8b380aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:42.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.887+0000 7f8b3ca12640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b3815ed60 0x7f8b380aec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51030/0 (socket says 172.21.15.40:51030) 2023-10-18T09:01:42.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.887+0000 7f8b3ca12640 1 -- 172.21.15.40:0/4122197668 learned_addr learned my addr 172.21.15.40:0/4122197668 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:42.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.887+0000 7f8b377fe640 1 --2- 172.21.15.40:0/4122197668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b380ce040 0x7f8b3815a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:42.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.887+0000 7f8b37fff640 1 --2- 172.21.15.40:0/4122197668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b380cdbc0 0x7f8b380c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:42.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.887+0000 7f8b3ca12640 1 -- 172.21.15.40:0/4122197668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b380cdbc0 msgr2=0x7f8b380c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:42.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.887+0000 7f8b3ca12640 1 --2- 172.21.15.40:0/4122197668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b380cdbc0 0x7f8b380c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.887+0000 7f8b3ca12640 1 -- 172.21.15.40:0/4122197668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b380ce040 msgr2=0x7f8b3815a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:42.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.887+0000 7f8b3ca12640 1 --2- 172.21.15.40:0/4122197668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b380ce040 0x7f8b3815a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:42.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.887+0000 7f8b3ca12640 1 -- 172.21.15.40:0/4122197668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b28009900 con 0x7f8b3815ed60 2023-10-18T09:01:42.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.887+0000 7f8b377fe640 1 --2- 172.21.15.40:0/4122197668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b380ce040 0x7f8b3815a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:42.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.887+0000 7f8b37fff640 1 --2- 172.21.15.40:0/4122197668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b380cdbc0 0x7f8b380c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:42.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.887+0000 7f8b3ca12640 1 --2- 172.21.15.40:0/4122197668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b3815ed60 0x7f8b380aec80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000c970 tx=0x7f8b3000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:42.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.888+0000 7f8b34ff9640 1 -- 172.21.15.40:0/4122197668 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30007bb0 con 0x7f8b3815ed60 2023-10-18T09:01:42.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.888+0000 7f8b34ff9640 1 -- 172.21.15.40:0/4122197668 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8b30007e00 con 0x7f8b3815ed60 2023-10-18T09:01:42.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.888+0000 7f8b3d213640 1 -- 172.21.15.40:0/4122197668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b380b5790 con 0x7f8b3815ed60 2023-10-18T09:01:42.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.888+0000 7f8b34ff9640 1 -- 172.21.15.40:0/4122197668 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30005280 con 0x7f8b3815ed60 2023-10-18T09:01:42.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.888+0000 7f8b3d213640 1 -- 172.21.15.40:0/4122197668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b380b5d50 con 0x7f8b3815ed60 2023-10-18T09:01:42.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.889+0000 7f8b3d213640 1 -- 172.21.15.40:0/4122197668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b38073570 con 0x7f8b3815ed60 2023-10-18T09:01:42.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.890+0000 7f8b34ff9640 1 -- 172.21.15.40:0/4122197668 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8b300053e0 con 0x7f8b3815ed60 2023-10-18T09:01:42.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.891+0000 7f8b34ff9640 1 --2- 172.21.15.40:0/4122197668 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b2406ae40 0x7f8b2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:42.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.891+0000 7f8b34ff9640 1 -- 172.21.15.40:0/4122197668 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b30089c50 con 0x7f8b3815ed60 2023-10-18T09:01:42.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.892+0000 7f8b37fff640 1 --2- 172.21.15.40:0/4122197668 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b2406ae40 0x7f8b2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:42.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.893+0000 7f8b37fff640 1 --2- 172.21.15.40:0/4122197668 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b2406ae40 0x7f8b2406d300 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8b28036000 tx=0x7f8b28004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:42.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:42.895+0000 7f8b34ff9640 1 -- 172.21.15.40:0/4122197668 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8b30054c60 con 0x7f8b3815ed60 2023-10-18T09:01:43.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.161+0000 7f8b3d213640 1 -- 172.21.15.40:0/4122197668 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8b380cd560 con 0x7f8b3815ed60 2023-10-18T09:01:43.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.167+0000 7f8b34ff9640 1 -- 172.21.15.40:0/4122197668 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8b3001d070 con 0x7f8b3815ed60 2023-10-18T09:01:43.170 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2023-10-18T09:01:43.170 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w== 2023-10-18T09:01:43.170 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2023-10-18T09:01:43.170 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2023-10-18T09:01:43.171 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T09:01:43.171 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2023-10-18T09:01:43.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.170+0000 7f8b3d213640 1 -- 172.21.15.40:0/4122197668 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b2406ae40 msgr2=0x7f8b2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:43.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.170+0000 7f8b3d213640 1 --2- 172.21.15.40:0/4122197668 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b2406ae40 0x7f8b2406d300 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8b28036000 tx=0x7f8b28004ab0 comp rx=0 tx=0).stop 2023-10-18T09:01:43.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.170+0000 7f8b3d213640 1 -- 172.21.15.40:0/4122197668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b3815ed60 msgr2=0x7f8b380aec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:43.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.170+0000 7f8b3d213640 1 --2- 172.21.15.40:0/4122197668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b3815ed60 0x7f8b380aec80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000c970 tx=0x7f8b3000ce30 comp rx=0 tx=0).stop 2023-10-18T09:01:43.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.170+0000 7f8b3d213640 1 -- 172.21.15.40:0/4122197668 shutdown_connections 2023-10-18T09:01:43.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.170+0000 7f8b3d213640 1 --2- 172.21.15.40:0/4122197668 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b2406ae40 0x7f8b2406d300 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:43.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.170+0000 7f8b3d213640 1 --2- 172.21.15.40:0/4122197668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b3815ed60 0x7f8b380aec80 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:43.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.170+0000 7f8b3d213640 1 --2- 172.21.15.40:0/4122197668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b380ce040 0x7f8b3815a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:43.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.170+0000 7f8b3d213640 1 --2- 172.21.15.40:0/4122197668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b380cdbc0 0x7f8b380c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:43.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.170+0000 7f8b3d213640 1 -- 172.21.15.40:0/4122197668 >> 172.21.15.40:0/4122197668 conn(0x7f8b3809a580 msgr2=0x7f8b3806bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:43.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.170+0000 7f8b3d213640 1 -- 172.21.15.40:0/4122197668 shutdown_connections 2023-10-18T09:01:43.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.170+0000 7f8b3d213640 1 -- 172.21.15.40:0/4122197668 wait complete. 2023-10-18T09:01:43.191 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:01:43.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.401+0000 7f90b21a9640 1 -- 172.21.15.40:0/870294758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9098004d00 msgr2=0x7f9098005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:43.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.401+0000 7f90b21a9640 1 --2- 172.21.15.40:0/870294758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9098004d00 0x7f9098005180 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0b16b0 tx=0x7f90980057f0 comp rx=0 tx=0).stop 2023-10-18T09:01:43.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.401+0000 7f90abfff640 1 -- 172.21.15.40:0/870294758 reap_dead start 2023-10-18T09:01:43.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.402+0000 7f90b21a9640 1 -- 172.21.15.40:0/870294758 shutdown_connections 2023-10-18T09:01:43.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.402+0000 7f90b21a9640 1 -- 172.21.15.40:0/870294758 >> 172.21.15.40:0/870294758 conn(0x7f90ac06cee0 msgr2=0x7f90ac069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:43.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.402+0000 7f90b21a9640 1 -- 172.21.15.40:0/870294758 shutdown_connections 2023-10-18T09:01:43.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.402+0000 7f90b21a9640 1 -- 172.21.15.40:0/870294758 wait complete. 2023-10-18T09:01:43.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.404+0000 7f90b21a9640 1 Processor -- start 2023-10-18T09:01:43.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.404+0000 7f90b21a9640 1 -- start start 2023-10-18T09:01:43.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.405+0000 7f90b21a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9098004d00 0x7f90ac0cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:43.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.405+0000 7f90b21a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90ac0ac200 0x7f90ac147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:43.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.405+0000 7f90b21a9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90ac0ac680 0x7f90ac147cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:43.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.405+0000 7f90b21a9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac15b4b0 con 0x7f9098004d00 2023-10-18T09:01:43.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.405+0000 7f90b21a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac0cca20 con 0x7f90ac0ac200 2023-10-18T09:01:43.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.405+0000 7f90b21a9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac151e20 con 0x7f90ac0ac680 2023-10-18T09:01:43.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.405+0000 7f90ab7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9098004d00 0x7f90ac0cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:43.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.405+0000 7f90ab7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9098004d00 0x7f90ac0cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51044/0 (socket says 172.21.15.40:51044) 2023-10-18T09:01:43.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.405+0000 7f90ab7fe640 1 -- 172.21.15.40:0/574737813 learned_addr learned my addr 172.21.15.40:0/574737813 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:43.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.405+0000 7f90abfff640 1 --2- 172.21.15.40:0/574737813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90ac0ac680 0x7f90ac147cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:43.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.405+0000 7f90aaffd640 1 --2- 172.21.15.40:0/574737813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90ac0ac200 0x7f90ac147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:43.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.405+0000 7f90ab7fe640 1 -- 172.21.15.40:0/574737813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90ac0ac680 msgr2=0x7f90ac147cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:43.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.405+0000 7f90ab7fe640 1 --2- 172.21.15.40:0/574737813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90ac0ac680 0x7f90ac147cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:43.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.405+0000 7f90ab7fe640 1 -- 172.21.15.40:0/574737813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90ac0ac200 msgr2=0x7f90ac147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:43.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.405+0000 7f90ab7fe640 1 --2- 172.21.15.40:0/574737813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90ac0ac200 0x7f90ac147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:43.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.406+0000 7f90ab7fe640 1 -- 172.21.15.40:0/574737813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ac152100 con 0x7f9098004d00 2023-10-18T09:01:43.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.406+0000 7f90aaffd640 1 --2- 172.21.15.40:0/574737813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90ac0ac200 0x7f90ac147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:43.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.406+0000 7f90ab7fe640 1 --2- 172.21.15.40:0/574737813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9098004d00 0x7f90ac0cc4e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f909c010bf0 tx=0x7f909c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:43.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.406+0000 7f90a3fff640 1 -- 172.21.15.40:0/574737813 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c003b70 con 0x7f9098004d00 2023-10-18T09:01:43.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.406+0000 7f90a3fff640 1 -- 172.21.15.40:0/574737813 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f909c015070 con 0x7f9098004d00 2023-10-18T09:01:43.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.406+0000 7f90a3fff640 1 -- 172.21.15.40:0/574737813 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c0042b0 con 0x7f9098004d00 2023-10-18T09:01:43.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.406+0000 7f90b21a9640 1 -- 172.21.15.40:0/574737813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90ac1523e0 con 0x7f9098004d00 2023-10-18T09:01:43.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.406+0000 7f90b21a9640 1 -- 172.21.15.40:0/574737813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90ac1529a0 con 0x7f9098004d00 2023-10-18T09:01:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.407+0000 7f90b21a9640 1 -- 172.21.15.40:0/574737813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90ac066fd0 con 0x7f9098004d00 2023-10-18T09:01:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.408+0000 7f90a3fff640 1 -- 172.21.15.40:0/574737813 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f909c004410 con 0x7f9098004d00 2023-10-18T09:01:43.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.409+0000 7f90a3fff640 1 --2- 172.21.15.40:0/574737813 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f908c06ae70 0x7f908c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.410+0000 7f90a3fff640 1 -- 172.21.15.40:0/574737813 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f909c017070 con 0x7f9098004d00 2023-10-18T09:01:43.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.410+0000 7f90aaffd640 1 --2- 172.21.15.40:0/574737813 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f908c06ae70 0x7f908c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:43.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.411+0000 7f90aaffd640 1 --2- 172.21.15.40:0/574737813 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f908c06ae70 0x7f908c06d330 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9094001ce0 tx=0x7f9094000ed0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:43.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.413+0000 7f90a3fff640 1 -- 172.21.15.40:0/574737813 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f909c0557f0 con 0x7f9098004d00 2023-10-18T09:01:43.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.642+0000 7f90b21a9640 1 -- 172.21.15.40:0/574737813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90ac0aa950 con 0x7f9098004d00 2023-10-18T09:01:43.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.643+0000 7f90a3fff640 1 -- 172.21.15.40:0/574737813 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f909c058e10 con 0x7f9098004d00 2023-10-18T09:01:43.646 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:43.646 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2565609773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2565609773},{"type":"v1","addr":"172.21.15.144:6836","nonce":2565609773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":5872,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3371548374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3371548374},{"type":"v1","addr":"172.21.15.144:6837","nonce":3371548374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3328648628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3328648628},{"type":"v1","addr":"172.21.15.40:6833","nonce":3328648628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:01:36.995669+0000","modified":"2023-10-18T09:01:40.068484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/891188043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":891188043},{"type":"v1","addr":"172.21.15.40:6835","nonce":891188043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-18T09:01:43.647 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 49 2023-10-18T09:01:43.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.646+0000 7f90b21a9640 1 -- 172.21.15.40:0/574737813 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f908c06ae70 msgr2=0x7f908c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:43.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.646+0000 7f90b21a9640 1 --2- 172.21.15.40:0/574737813 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f908c06ae70 0x7f908c06d330 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9094001ce0 tx=0x7f9094000ed0 comp rx=0 tx=0).stop 2023-10-18T09:01:43.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.646+0000 7f90b21a9640 1 -- 172.21.15.40:0/574737813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9098004d00 msgr2=0x7f90ac0cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:43.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.646+0000 7f90b21a9640 1 --2- 172.21.15.40:0/574737813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9098004d00 0x7f90ac0cc4e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f909c010bf0 tx=0x7f909c00e720 comp rx=0 tx=0).stop 2023-10-18T09:01:43.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.646+0000 7f90b21a9640 1 -- 172.21.15.40:0/574737813 shutdown_connections 2023-10-18T09:01:43.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.646+0000 7f90b21a9640 1 --2- 172.21.15.40:0/574737813 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f908c06ae70 0x7f908c06d330 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:43.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.646+0000 7f90b21a9640 1 --2- 172.21.15.40:0/574737813 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f90ac0ac680 0x7f90ac147cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:43.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.646+0000 7f90b21a9640 1 --2- 172.21.15.40:0/574737813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f90ac0ac200 0x7f90ac147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:43.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.646+0000 7f90b21a9640 1 --2- 172.21.15.40:0/574737813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9098004d00 0x7f90ac0cc4e0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:43.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.646+0000 7f90b21a9640 1 -- 172.21.15.40:0/574737813 >> 172.21.15.40:0/574737813 conn(0x7f90ac06cee0 msgr2=0x7f90ac159b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:43.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.647+0000 7f90b21a9640 1 -- 172.21.15.40:0/574737813 shutdown_connections 2023-10-18T09:01:43.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:43.647+0000 7f90b21a9640 1 -- 172.21.15.40:0/574737813 wait complete. 2023-10-18T09:01:44.070 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T09:01:36.995669+0000', 'modified': '2023-10-18T09:01:40.068484+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6996}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6996': {'gid': 6996, 'name': 'c', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/891188043', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 891188043}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 891188043}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T09:01:44.071 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T09:01:44.071 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-18T09:01:44.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.309+0000 7f3ba20ea640 1 -- 172.21.15.40:0/3025014898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b880047c0 msgr2=0x7f3b88004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:44.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.309+0000 7f3ba20ea640 1 --2- 172.21.15.40:0/3025014898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b880047c0 0x7f3b88004c40 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009b50 tx=0x7f3b880057f0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.309+0000 7f3ba18e9640 1 -- 172.21.15.40:0/3025014898 reap_dead start 2023-10-18T09:01:44.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.309+0000 7f3ba20ea640 1 -- 172.21.15.40:0/3025014898 shutdown_connections 2023-10-18T09:01:44.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.309+0000 7f3ba20ea640 1 -- 172.21.15.40:0/3025014898 >> 172.21.15.40:0/3025014898 conn(0x7f3b9c069720 msgr2=0x7f3b9c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:44.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.309+0000 7f3ba20ea640 1 -- 172.21.15.40:0/3025014898 shutdown_connections 2023-10-18T09:01:44.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.309+0000 7f3ba20ea640 1 -- 172.21.15.40:0/3025014898 wait complete. 2023-10-18T09:01:44.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.311+0000 7f3ba20ea640 1 Processor -- start 2023-10-18T09:01:44.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.311+0000 7f3ba20ea640 1 -- start start 2023-10-18T09:01:44.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.312+0000 7f3ba20ea640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b9c13ec00 0x7f3b9c138ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.312+0000 7f3ba20ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b9c13efe0 0x7f3b9c1393e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.312+0000 7f3ba20ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b9c1587a0 0x7f3b9c158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.312+0000 7f3ba20ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c15a8b0 con 0x7f3b9c13efe0 2023-10-18T09:01:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.312+0000 7f3ba20ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c159160 con 0x7f3b9c1587a0 2023-10-18T09:01:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.312+0000 7f3ba20ea640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c159950 con 0x7f3b9c13ec00 2023-10-18T09:01:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.313+0000 7f3ba08e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b9c13efe0 0x7f3b9c1393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.313+0000 7f3ba08e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b9c13efe0 0x7f3b9c1393e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51074/0 (socket says 172.21.15.40:51074) 2023-10-18T09:01:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.313+0000 7f3ba08e7640 1 -- 172.21.15.40:0/2451377145 learned_addr learned my addr 172.21.15.40:0/2451377145 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.313+0000 7f3ba10e8640 1 --2- 172.21.15.40:0/2451377145 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b9c13ec00 0x7f3b9c138ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.313+0000 7f3ba18e9640 1 --2- 172.21.15.40:0/2451377145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b9c1587a0 0x7f3b9c158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.313+0000 7f3ba08e7640 1 -- 172.21.15.40:0/2451377145 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b9c13ec00 msgr2=0x7f3b9c138ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.313+0000 7f3ba08e7640 1 --2- 172.21.15.40:0/2451377145 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b9c13ec00 0x7f3b9c138ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.313+0000 7f3ba08e7640 1 -- 172.21.15.40:0/2451377145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b9c1587a0 msgr2=0x7f3b9c158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.313+0000 7f3ba08e7640 1 --2- 172.21.15.40:0/2451377145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b9c1587a0 0x7f3b9c158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.313+0000 7f3ba08e7640 1 -- 172.21.15.40:0/2451377145 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c159c30 con 0x7f3b9c13efe0 2023-10-18T09:01:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.313+0000 7f3ba18e9640 1 --2- 172.21.15.40:0/2451377145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b9c1587a0 0x7f3b9c158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.313+0000 7f3ba08e7640 1 --2- 172.21.15.40:0/2451377145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b9c13efe0 0x7f3b9c1393e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c02cb70 tx=0x7f3b8c033c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:44.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.314+0000 7f3b91ffb640 1 -- 172.21.15.40:0/2451377145 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c004220 con 0x7f3b9c13efe0 2023-10-18T09:01:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.314+0000 7f3b91ffb640 1 -- 172.21.15.40:0/2451377145 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b8c004380 con 0x7f3b9c13efe0 2023-10-18T09:01:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.314+0000 7f3b91ffb640 1 -- 172.21.15.40:0/2451377145 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c03e950 con 0x7f3b9c13efe0 2023-10-18T09:01:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.314+0000 7f3ba20ea640 1 -- 172.21.15.40:0/2451377145 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b9c159f10 con 0x7f3b9c13efe0 2023-10-18T09:01:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.314+0000 7f3ba20ea640 1 -- 172.21.15.40:0/2451377145 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9c15a4d0 con 0x7f3b9c13efe0 2023-10-18T09:01:44.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.316+0000 7f3b91ffb640 1 -- 172.21.15.40:0/2451377145 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3b8c03eab0 con 0x7f3b9c13efe0 2023-10-18T09:01:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.317+0000 7f3b91ffb640 1 --2- 172.21.15.40:0/2451377145 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3b8006ae40 0x7f3b8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.317+0000 7f3b91ffb640 1 -- 172.21.15.40:0/2451377145 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b8c033e40 con 0x7f3b9c13efe0 2023-10-18T09:01:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.318+0000 7f3ba5fb5740 1 Processor -- start 2023-10-18T09:01:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.318+0000 7f3ba10e8640 1 --2- 172.21.15.40:0/2451377145 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3b8006ae40 0x7f3b8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.318+0000 7f3ba5fb5740 1 -- start start 2023-10-18T09:01:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.319+0000 7f3ba5fb5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec5a970 0x5648eec2dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.319+0000 7f3ba5fb5740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5ad50 0x5648eec5d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.319+0000 7f3ba5fb5740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec2e0e0 0x5648eec5dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.319+0000 7f3ba5fb5740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648eeba0740 con 0x5648eec5ad50 2023-10-18T09:01:44.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.319+0000 7f3ba5fb5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648eeb9de30 con 0x5648eec5a970 2023-10-18T09:01:44.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.319+0000 7f3ba5fb5740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648eeb99720 con 0x5648eec2e0e0 2023-10-18T09:01:44.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.319+0000 7f3ba10e8640 1 --2- 172.21.15.40:0/2451377145 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3b8006ae40 0x7f3b8006d300 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3b98000ff0 tx=0x7f3b98001c10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:44.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.319+0000 7f3ba08e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5ad50 0x5648eec5d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.319+0000 7f3ba08e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5ad50 0x5648eec5d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51084/0 (socket says 172.21.15.40:51084) 2023-10-18T09:01:44.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.319+0000 7f3ba08e7640 1 -- 172.21.15.40:0/2003299727 learned_addr learned my addr 172.21.15.40:0/2003299727 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:44.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.320+0000 7f3ba10e8640 1 --2- 172.21.15.40:0/2003299727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec5a970 0x5648eec2dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.320+0000 7f3ba08e7640 1 -- 172.21.15.40:0/2003299727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec2e0e0 msgr2=0x5648eec5dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:01:44.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.320+0000 7f3ba08e7640 1 --2- 172.21.15.40:0/2003299727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec2e0e0 0x5648eec5dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.320+0000 7f3ba08e7640 1 -- 172.21.15.40:0/2003299727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec5a970 msgr2=0x5648eec2dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:01:44.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.320+0000 7f3ba08e7640 1 --2- 172.21.15.40:0/2003299727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec5a970 0x5648eec2dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.320+0000 7f3ba08e7640 1 -- 172.21.15.40:0/2003299727 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648eeb5d910 con 0x5648eec5ad50 2023-10-18T09:01:44.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.320+0000 7f3ba08e7640 1 --2- 172.21.15.40:0/2003299727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5ad50 0x5648eec5d700 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c083340 tx=0x7f3b8c07bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa734d6f45f6eb1d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:44.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.320+0000 7f3b7effd640 1 -- 172.21.15.40:0/2003299727 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c07f6c0 con 0x5648eec5ad50 2023-10-18T09:01:44.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.320+0000 7f3b7effd640 1 -- 172.21.15.40:0/2003299727 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b8c079910 con 0x5648eec5ad50 2023-10-18T09:01:44.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.320+0000 7f3b7effd640 1 -- 172.21.15.40:0/2003299727 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c03ed60 con 0x5648eec5ad50 2023-10-18T09:01:44.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.321+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/2003299727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5ad50 msgr2=0x5648eec5d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:44.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.321+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/2003299727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5ad50 0x5648eec5d700 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c083340 tx=0x7f3b8c07bc70 comp rx=0 tx=0).stop 2023-10-18T09:01:44.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.321+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/2003299727 shutdown_connections 2023-10-18T09:01:44.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.321+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/2003299727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec2e0e0 0x5648eec5dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.321+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/2003299727 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5ad50 0x5648eec5d700 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.321+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/2003299727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec5a970 0x5648eec2dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.321+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/2003299727 >> 172.21.15.40:0/2003299727 conn(0x5648eec2fe10 msgr2=0x5648eec78700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:44.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.321+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/2003299727 shutdown_connections 2023-10-18T09:01:44.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.321+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/2003299727 wait complete. 2023-10-18T09:01:44.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.322+0000 7f3ba5fb5740 1 -- start start 2023-10-18T09:01:44.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.322+0000 7f3ba5fb5740 1 Processor -- start 2023-10-18T09:01:44.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.323+0000 7f3ba5fb5740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5a970 0x5648eec72d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.323+0000 7f3ba5fb5740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec5ad50 0x5648eec73270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.323+0000 7f3ba5fb5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec49c60 0x5648eec4a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.323+0000 7f3ba5fb5740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648eeb37970 con 0x5648eec5a970 2023-10-18T09:01:44.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.323+0000 7f3ba5fb5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648eeb37610 con 0x5648eec49c60 2023-10-18T09:01:44.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.323+0000 7f3ba5fb5740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648ee8fdf20 con 0x5648eec5ad50 2023-10-18T09:01:44.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.323+0000 7f3ba5fb5740 10 client.0 _ll_register_callbacks cb 0x5648eeb5b0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:01:44.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.323+0000 7f3ba10e8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5a970 0x5648eec72d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.323+0000 7f3ba10e8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5a970 0x5648eec72d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51088/0 (socket says 172.21.15.40:51088) 2023-10-18T09:01:44.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.323+0000 7f3ba10e8640 1 -- 172.21.15.40:0/2971730509 learned_addr learned my addr 172.21.15.40:0/2971730509 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:44.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.324+0000 7f3ba10e8640 1 -- 172.21.15.40:0/2971730509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec5ad50 msgr2=0x5648eec73270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:01:44.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.324+0000 7f3ba08e7640 1 --2- 172.21.15.40:0/2971730509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec5ad50 0x5648eec73270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.324+0000 7f3ba18e9640 1 --2- 172.21.15.40:0/2971730509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec49c60 0x5648eec4a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.324+0000 7f3ba10e8640 1 --2- 172.21.15.40:0/2971730509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec5ad50 0x5648eec73270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.324+0000 7f3ba10e8640 1 -- 172.21.15.40:0/2971730509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec49c60 msgr2=0x5648eec4a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:01:44.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.324+0000 7f3ba10e8640 1 --2- 172.21.15.40:0/2971730509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec49c60 0x5648eec4a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.324+0000 7f3ba10e8640 1 -- 172.21.15.40:0/2971730509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9c15a4d0 con 0x5648eec5a970 2023-10-18T09:01:44.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.324+0000 7f3ba10e8640 1 --2- 172.21.15.40:0/2971730509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5a970 0x5648eec72d30 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f3b98011a10 tx=0x7f3b980035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141cc5d0898b360d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:44.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.324+0000 7f3b77fff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T09:01:44.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.325+0000 7f3b77fff640 1 -- 172.21.15.40:0/2971730509 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800feb0 con 0x5648eec5a970 2023-10-18T09:01:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.325+0000 7f3b77fff640 1 -- 172.21.15.40:0/2971730509 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b98010ad0 con 0x5648eec5a970 2023-10-18T09:01:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.325+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/2971730509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648eeb92700 con 0x5648eec5a970 2023-10-18T09:01:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.325+0000 7f3b77fff640 1 -- 172.21.15.40:0/2971730509 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b98010c90 con 0x5648eec5a970 2023-10-18T09:01:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.325+0000 7f3ba5fb5740 10 client.7137 fetch_fsmap learned FSMap version 49 2023-10-18T09:01:44.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.325+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/2971730509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x5648eeb97da0 con 0x5648eec5a970 2023-10-18T09:01:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.325+0000 7f3b77fff640 1 -- 172.21.15.40:0/2971730509 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x5648eeb92700 con 0x5648eec5a970 2023-10-18T09:01:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.325+0000 7f3b77fff640 1 -- 172.21.15.40:0/2971730509 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3b98019070 con 0x5648eec5a970 2023-10-18T09:01:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.325+0000 7f3ba5fb5740 10 client.7137 fetch_fsmap finished waiting for FSMap version 49 2023-10-18T09:01:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.325+0000 7f3ba5fb5740 10 client.7137 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T09:01:44.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.325+0000 7f3ba5fb5740 20 client.7137 populate_metadata read hostname 'smithi040' 2023-10-18T09:01:44.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.325+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/2971730509 >> [v2:172.21.15.40:6834/891188043,v1:172.21.15.40:6835/891188043] conn(0x5648eec60ed0 0x5648eec4abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.325+0000 7f3ba5fb5740 4 client.7137 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:01:44.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.325+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/2971730509 --> [v2:172.21.15.40:6834/891188043,v1:172.21.15.40:6835/891188043] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648eeb92420 con 0x5648eec60ed0 2023-10-18T09:01:44.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.326+0000 7f3ba08e7640 1 --2- 172.21.15.40:0/2971730509 >> [v2:172.21.15.40:6834/891188043,v1:172.21.15.40:6835/891188043] conn(0x5648eec60ed0 0x5648eec4abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.326+0000 7f3ba08e7640 1 --2- 172.21.15.40:0/2971730509 >> [v2:172.21.15.40:6834/891188043,v1:172.21.15.40:6835/891188043] conn(0x5648eec60ed0 0x5648eec4abb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1c5fb91b2f8f019 server_cookie=9ea8dca645be7dc in_seq=0 out_seq=0 2023-10-18T09:01:44.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.326+0000 7f3b77fff640 10 client.7137 ms_handle_connect on v2:172.21.15.40:6834/891188043 2023-10-18T09:01:44.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.327+0000 7f3b77fff640 1 -- 172.21.15.40:0/2971730509 <== mds.0 v2:172.21.15.40:6834/891188043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5648eeb92420 con 0x5648eec60ed0 2023-10-18T09:01:44.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.327+0000 7f3b77fff640 10 client.7137 handle_command_reply: tid=0 2023-10-18T09:01:44.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.327+0000 7f3ba5fb5740 1 client.7137 shutdown 2023-10-18T09:01:44.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.327+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/2971730509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5a970 msgr2=0x5648eec72d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:44.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.327+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/2971730509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5a970 0x5648eec72d30 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f3b98011a10 tx=0x7f3b980035f0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.327+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/2971730509 shutdown_connections 2023-10-18T09:01:44.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.327+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/2971730509 >> [v2:172.21.15.40:6834/891188043,v1:172.21.15.40:6835/891188043] conn(0x5648eec60ed0 0x5648eec4abb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.327+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/2971730509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec49c60 0x5648eec4a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.327+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/2971730509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec5ad50 0x5648eec73270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.327+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/2971730509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5a970 0x5648eec72d30 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.327+0000 7f3b77fff640 0 client.7137 ms_handle_reset on v2:172.21.15.40:6834/891188043 2023-10-18T09:01:44.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.327+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/2971730509 >> 172.21.15.40:0/2971730509 conn(0x5648eec2e0e0 msgr2=0x5648eec5d870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:44.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.327+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/2971730509 shutdown_connections 2023-10-18T09:01:44.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.328+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/2971730509 wait complete. 2023-10-18T09:01:44.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.328+0000 7f3ba5fb5740 20 client.7137 trim_cache size 0 max 16384 2023-10-18T09:01:44.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.345+0000 7f3ba5fb5740 1 Processor -- start 2023-10-18T09:01:44.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.345+0000 7f3ba5fb5740 1 -- start start 2023-10-18T09:01:44.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.346+0000 7f3ba5fb5740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec49c60 0x5648eec339a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.346+0000 7f3ba5fb5740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5a970 0x5648eec7bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.346+0000 7f3ba5fb5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec5ad50 0x5648eec7c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.346+0000 7f3ba5fb5740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648eec6fda0 con 0x5648eec5a970 2023-10-18T09:01:44.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.346+0000 7f3ba5fb5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648eec28980 con 0x5648eec5ad50 2023-10-18T09:01:44.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.346+0000 7f3ba5fb5740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648eec72850 con 0x5648eec49c60 2023-10-18T09:01:44.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.346+0000 7f3ba08e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5a970 0x5648eec7bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.346+0000 7f3ba08e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5a970 0x5648eec7bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51090/0 (socket says 172.21.15.40:51090) 2023-10-18T09:01:44.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.346+0000 7f3ba08e7640 1 -- 172.21.15.40:0/1422689078 learned_addr learned my addr 172.21.15.40:0/1422689078 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:44.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.346+0000 7f3ba08e7640 1 -- 172.21.15.40:0/1422689078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec49c60 msgr2=0x5648eec339a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:01:44.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.347+0000 7f3ba10e8640 1 --2- 172.21.15.40:0/1422689078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec49c60 0x5648eec339a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.347+0000 7f3ba18e9640 1 --2- 172.21.15.40:0/1422689078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec5ad50 0x5648eec7c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.347+0000 7f3ba08e7640 1 --2- 172.21.15.40:0/1422689078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec49c60 0x5648eec339a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.347+0000 7f3ba08e7640 1 -- 172.21.15.40:0/1422689078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec5ad50 msgr2=0x5648eec7c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:01:44.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.347+0000 7f3ba08e7640 1 --2- 172.21.15.40:0/1422689078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec5ad50 0x5648eec7c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.347+0000 7f3ba08e7640 1 -- 172.21.15.40:0/1422689078 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648eeb36b80 con 0x5648eec5a970 2023-10-18T09:01:44.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.347+0000 7f3ba08e7640 1 --2- 172.21.15.40:0/1422689078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5a970 0x5648eec7bc10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c07b730 tx=0x7f3b8c07e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b27ef2668f2288 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:44.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.347+0000 7f3b7f7fe640 1 -- 172.21.15.40:0/1422689078 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c095030 con 0x5648eec5a970 2023-10-18T09:01:44.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.347+0000 7f3b7f7fe640 1 -- 172.21.15.40:0/1422689078 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b8c097030 con 0x5648eec5a970 2023-10-18T09:01:44.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.347+0000 7f3b7f7fe640 1 -- 172.21.15.40:0/1422689078 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c009720 con 0x5648eec5a970 2023-10-18T09:01:44.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.348+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/1422689078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5a970 msgr2=0x5648eec7bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:44.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.348+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/1422689078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5a970 0x5648eec7bc10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c07b730 tx=0x7f3b8c07e040 comp rx=0 tx=0).stop 2023-10-18T09:01:44.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.348+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/1422689078 shutdown_connections 2023-10-18T09:01:44.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.348+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/1422689078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec5ad50 0x5648eec7c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.348+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/1422689078 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5a970 0x5648eec7bc10 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.348+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/1422689078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec49c60 0x5648eec339a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.348+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/1422689078 >> 172.21.15.40:0/1422689078 conn(0x5648eec2e0e0 msgr2=0x5648eec72d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:44.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.348+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/1422689078 shutdown_connections 2023-10-18T09:01:44.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.348+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/1422689078 wait complete. 2023-10-18T09:01:44.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.349+0000 7f3ba5fb5740 1 -- start start 2023-10-18T09:01:44.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.349+0000 7f3ba5fb5740 1 Processor -- start 2023-10-18T09:01:44.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.350+0000 7f3ba5fb5740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec49c60 0x5648eec72d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.350+0000 7f3ba5fb5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec5a970 0x5648eec73270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.350+0000 7f3ba5fb5740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5ad50 0x5648eeccf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.350+0000 7f3ba5fb5740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648eecd0810 con 0x5648eec5ad50 2023-10-18T09:01:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.350+0000 7f3ba5fb5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648eec6f2b0 con 0x5648eec5a970 2023-10-18T09:01:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.350+0000 7f3ba5fb5740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648eec6e210 con 0x5648eec49c60 2023-10-18T09:01:44.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.350+0000 7f3ba5fb5740 10 client.0 _ll_register_callbacks cb 0x7f3b8c0045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:01:44.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.350+0000 7f3ba18e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5ad50 0x5648eeccf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.350+0000 7f3ba18e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5ad50 0x5648eeccf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51096/0 (socket says 172.21.15.40:51096) 2023-10-18T09:01:44.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.350+0000 7f3ba18e9640 1 -- 172.21.15.40:0/3456356764 learned_addr learned my addr 172.21.15.40:0/3456356764 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:44.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.351+0000 7f3ba08e7640 1 --2- 172.21.15.40:0/3456356764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec5a970 0x5648eec73270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.351+0000 7f3ba10e8640 1 --2- 172.21.15.40:0/3456356764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec49c60 0x5648eec72d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.351+0000 7f3ba18e9640 1 -- 172.21.15.40:0/3456356764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec49c60 msgr2=0x5648eec72d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:01:44.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.351+0000 7f3ba18e9640 1 --2- 172.21.15.40:0/3456356764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec49c60 0x5648eec72d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.351+0000 7f3ba18e9640 1 -- 172.21.15.40:0/3456356764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec5a970 msgr2=0x5648eec73270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:01:44.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.351+0000 7f3ba18e9640 1 --2- 172.21.15.40:0/3456356764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec5a970 0x5648eec73270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.351+0000 7f3ba18e9640 1 -- 172.21.15.40:0/3456356764 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648eeb5d910 con 0x5648eec5ad50 2023-10-18T09:01:44.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.351+0000 7f3ba18e9640 1 --2- 172.21.15.40:0/3456356764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5ad50 0x5648eeccf650 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0b20e0 tx=0x7f3b94011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5748624b453a6c74 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:44.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.351+0000 7f3b77fff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T09:01:44.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.352+0000 7f3b77fff640 1 -- 172.21.15.40:0/3456356764 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94014640 con 0x5648eec5ad50 2023-10-18T09:01:44.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.352+0000 7f3b77fff640 1 -- 172.21.15.40:0/3456356764 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b940147a0 con 0x5648eec5ad50 2023-10-18T09:01:44.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.352+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/3456356764 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b8c079910 con 0x5648eec5ad50 2023-10-18T09:01:44.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.352+0000 7f3b77fff640 1 -- 172.21.15.40:0/3456356764 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b940149a0 con 0x5648eec5ad50 2023-10-18T09:01:44.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.352+0000 7f3ba5fb5740 10 client.7143 fetch_fsmap learned FSMap version 49 2023-10-18T09:01:44.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.352+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/3456356764 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x5648eeb5c170 con 0x5648eec5ad50 2023-10-18T09:01:44.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.352+0000 7f3b77fff640 1 -- 172.21.15.40:0/3456356764 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b8c079910 con 0x5648eec5ad50 2023-10-18T09:01:44.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.352+0000 7f3b77fff640 1 -- 172.21.15.40:0/3456356764 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3b94014b00 con 0x5648eec5ad50 2023-10-18T09:01:44.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.352+0000 7f3ba5fb5740 10 client.7143 fetch_fsmap finished waiting for FSMap version 49 2023-10-18T09:01:44.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.352+0000 7f3ba5fb5740 10 client.7143 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T09:01:44.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.352+0000 7f3ba5fb5740 20 client.7143 populate_metadata read hostname 'smithi040' 2023-10-18T09:01:44.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.352+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/3456356764 >> [v2:172.21.15.40:6834/891188043,v1:172.21.15.40:6835/891188043] conn(0x5648eeccfbe0 0x5648eeccffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.352+0000 7f3ba5fb5740 4 client.7143 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:01:44.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.352+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/3456356764 --> [v2:172.21.15.40:6834/891188043,v1:172.21.15.40:6835/891188043] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5648ee89b6c0 con 0x5648eeccfbe0 2023-10-18T09:01:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.353+0000 7f3ba10e8640 1 --2- 172.21.15.40:0/3456356764 >> [v2:172.21.15.40:6834/891188043,v1:172.21.15.40:6835/891188043] conn(0x5648eeccfbe0 0x5648eeccffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.353+0000 7f3ba10e8640 1 --2- 172.21.15.40:0/3456356764 >> [v2:172.21.15.40:6834/891188043,v1:172.21.15.40:6835/891188043] conn(0x5648eeccfbe0 0x5648eeccffc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8e58acbe8634dd1 server_cookie=c2b1ba1fb9a143cd in_seq=0 out_seq=0 2023-10-18T09:01:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.353+0000 7f3b77fff640 10 client.7143 ms_handle_connect on v2:172.21.15.40:6834/891188043 2023-10-18T09:01:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.354+0000 7f3b77fff640 1 -- 172.21.15.40:0/3456356764 <== mds.0 v2:172.21.15.40:6834/891188043 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5648ee89b6c0 con 0x5648eeccfbe0 2023-10-18T09:01:44.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.354+0000 7f3b77fff640 10 client.7143 handle_command_reply: tid=0 2023-10-18T09:01:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.354+0000 7f3ba5fb5740 1 client.7143 shutdown 2023-10-18T09:01:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.354+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/3456356764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5ad50 msgr2=0x5648eeccf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.354+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/3456356764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5ad50 0x5648eeccf650 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0b20e0 tx=0x7f3b94011690 comp rx=0 tx=0).stop 2023-10-18T09:01:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.354+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/3456356764 shutdown_connections 2023-10-18T09:01:44.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.354+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/3456356764 >> [v2:172.21.15.40:6834/891188043,v1:172.21.15.40:6835/891188043] conn(0x5648eeccfbe0 0x5648eeccffc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.354+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/3456356764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x5648eec5ad50 0x5648eeccf650 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.354+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/3456356764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5648eec5a970 0x5648eec73270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.354+0000 7f3ba5fb5740 1 --2- 172.21.15.40:0/3456356764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5648eec49c60 0x5648eec72d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.354+0000 7f3b77fff640 0 client.7143 ms_handle_reset on v2:172.21.15.40:6834/891188043 2023-10-18T09:01:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.354+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/3456356764 >> 172.21.15.40:0/3456356764 conn(0x5648eec2e0e0 msgr2=0x5648eec7bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:44.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.354+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/3456356764 shutdown_connections 2023-10-18T09:01:44.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.354+0000 7f3ba5fb5740 1 -- 172.21.15.40:0/3456356764 wait complete. 2023-10-18T09:01:44.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.354+0000 7f3ba5fb5740 20 client.7143 trim_cache size 0 max 16384 2023-10-18T09:01:44.369 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:01:44.369 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:01:44.370 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-10-18T09:01:44.370 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 6996, 2023-10-18T09:01:44.370 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-10-18T09:01:44.370 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-10-18T09:01:44.370 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-10-18T09:01:44.371 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.8156134389999998, 2023-10-18T09:01:44.371 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 49, 2023-10-18T09:01:44.371 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 107, 2023-10-18T09:01:44.371 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 107, 2023-10-18T09:01:44.371 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 12.671489016000001 2023-10-18T09:01:44.372 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:01:44.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.355+0000 7f3ba20ea640 1 -- 172.21.15.40:0/2451377145 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3b8006ae40 msgr2=0x7f3b8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.355+0000 7f3ba20ea640 1 --2- 172.21.15.40:0/2451377145 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3b8006ae40 0x7f3b8006d300 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3b98000ff0 tx=0x7f3b98001c10 comp rx=0 tx=0).stop 2023-10-18T09:01:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.355+0000 7f3ba20ea640 1 -- 172.21.15.40:0/2451377145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b9c13efe0 msgr2=0x7f3b9c1393e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.355+0000 7f3ba20ea640 1 --2- 172.21.15.40:0/2451377145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b9c13efe0 0x7f3b9c1393e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c02cb70 tx=0x7f3b8c033c50 comp rx=0 tx=0).stop 2023-10-18T09:01:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.355+0000 7f3ba20ea640 1 -- 172.21.15.40:0/2451377145 shutdown_connections 2023-10-18T09:01:44.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.355+0000 7f3ba20ea640 1 --2- 172.21.15.40:0/2451377145 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3b8006ae40 0x7f3b8006d300 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.355+0000 7f3ba20ea640 1 --2- 172.21.15.40:0/2451377145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b9c1587a0 0x7f3b9c158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.355+0000 7f3ba20ea640 1 --2- 172.21.15.40:0/2451377145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b9c13efe0 0x7f3b9c1393e0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.355+0000 7f3ba20ea640 1 --2- 172.21.15.40:0/2451377145 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3b9c13ec00 0x7f3b9c138ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.355+0000 7f3ba20ea640 1 -- 172.21.15.40:0/2451377145 >> 172.21.15.40:0/2451377145 conn(0x7f3b9c069720 msgr2=0x7f3b9c13f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:44.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.355+0000 7f3ba20ea640 1 -- 172.21.15.40:0/2451377145 shutdown_connections 2023-10-18T09:01:44.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.355+0000 7f3ba20ea640 1 -- 172.21.15.40:0/2451377145 wait complete. 2023-10-18T09:01:44.376 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T09:01:44.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.602+0000 7faabac75640 1 -- 172.21.15.40:0/854923352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab40d1910 msgr2=0x7faab40cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:44.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.602+0000 7faabac75640 1 --2- 172.21.15.40:0/854923352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab40d1910 0x7faab40cf380 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7faab40d11d0 tx=0x7faaac02c1d0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.602+0000 7faabac75640 1 -- 172.21.15.40:0/854923352 shutdown_connections 2023-10-18T09:01:44.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.602+0000 7faabac75640 1 --2- 172.21.15.40:0/854923352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab40d1910 0x7faab40cf380 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.602+0000 7faabac75640 1 --2- 172.21.15.40:0/854923352 >> v2:172.21.15.144:3300/0 conn(0x7faab415ecc0 0x7faab40cd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.602+0000 7faabac75640 1 -- 172.21.15.40:0/854923352 >> 172.21.15.40:0/854923352 conn(0x7faab408ac80 msgr2=0x7faab409a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:44.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.602+0000 7faabac75640 1 -- 172.21.15.40:0/854923352 shutdown_connections 2023-10-18T09:01:44.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.602+0000 7faabac75640 1 -- 172.21.15.40:0/854923352 wait complete. 2023-10-18T09:01:44.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.604+0000 7faabac75640 1 Processor -- start 2023-10-18T09:01:44.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.604+0000 7faabac75640 1 -- start start 2023-10-18T09:01:44.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.605+0000 7faabac75640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faab40b2810 0x7faab406b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.605+0000 7faabac75640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faab40d1910 0x7faab414b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.605+0000 7faabac75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab415ecc0 0x7faab414bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.605+0000 7faabac75640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab40c4da0 con 0x7faab415ecc0 2023-10-18T09:01:44.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.605+0000 7faabac75640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab40c5600 con 0x7faab40b2810 2023-10-18T09:01:44.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.605+0000 7faabac75640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab40b6c90 con 0x7faab40d1910 2023-10-18T09:01:44.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.605+0000 7faaba474640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab415ecc0 0x7faab414bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.605+0000 7faaba474640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab415ecc0 0x7faab414bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51122/0 (socket says 172.21.15.40:51122) 2023-10-18T09:01:44.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.605+0000 7faaba474640 1 -- 172.21.15.40:0/561659947 learned_addr learned my addr 172.21.15.40:0/561659947 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:44.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.606+0000 7faab9c73640 1 --2- 172.21.15.40:0/561659947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faab40b2810 0x7faab406b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.606+0000 7faaba474640 1 -- 172.21.15.40:0/561659947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faab40d1910 msgr2=0x7faab414b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:44.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.606+0000 7faaba474640 1 --2- 172.21.15.40:0/561659947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faab40d1910 0x7faab414b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.606+0000 7faaba474640 1 -- 172.21.15.40:0/561659947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faab40b2810 msgr2=0x7faab406b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:44.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.606+0000 7faaba474640 1 --2- 172.21.15.40:0/561659947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faab40b2810 0x7faab406b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.606+0000 7faaba474640 1 -- 172.21.15.40:0/561659947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac009900 con 0x7faab415ecc0 2023-10-18T09:01:44.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.606+0000 7faab9c73640 1 --2- 172.21.15.40:0/561659947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faab40b2810 0x7faab406b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:44.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.606+0000 7faaba474640 1 --2- 172.21.15.40:0/561659947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab415ecc0 0x7faab414bd10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7faaac037040 tx=0x7faaac035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:44.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.606+0000 7faaaa7fc640 1 -- 172.21.15.40:0/561659947 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac034440 con 0x7faab415ecc0 2023-10-18T09:01:44.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.607+0000 7faaaa7fc640 1 -- 172.21.15.40:0/561659947 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faaac004540 con 0x7faab415ecc0 2023-10-18T09:01:44.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.607+0000 7faaaa7fc640 1 -- 172.21.15.40:0/561659947 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac03f440 con 0x7faab415ecc0 2023-10-18T09:01:44.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.607+0000 7faabac75640 1 -- 172.21.15.40:0/561659947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab40c4fb0 con 0x7faab415ecc0 2023-10-18T09:01:44.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.607+0000 7faabac75640 1 -- 172.21.15.40:0/561659947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab406b6f0 con 0x7faab415ecc0 2023-10-18T09:01:44.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.608+0000 7faabac75640 1 -- 172.21.15.40:0/561659947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab4071da0 con 0x7faab415ecc0 2023-10-18T09:01:44.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.609+0000 7faaaa7fc640 1 -- 172.21.15.40:0/561659947 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faaac03b040 con 0x7faab415ecc0 2023-10-18T09:01:44.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.610+0000 7faaaa7fc640 1 --2- 172.21.15.40:0/561659947 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa9806ae70 0x7faa9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:44.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.610+0000 7faaaa7fc640 1 -- 172.21.15.40:0/561659947 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faaac0ac4b0 con 0x7faab415ecc0 2023-10-18T09:01:44.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.611+0000 7faab9c73640 1 --2- 172.21.15.40:0/561659947 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa9806ae70 0x7faa9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:44.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.612+0000 7faab9c73640 1 --2- 172.21.15.40:0/561659947 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa9806ae70 0x7faa9806d330 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7faab00058c0 tx=0x7faab0005c90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:44.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.614+0000 7faaaa7fc640 1 -- 172.21.15.40:0/561659947 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faaac077550 con 0x7faab415ecc0 2023-10-18T09:01:44.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.846+0000 7faabac75640 1 -- 172.21.15.40:0/561659947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faab40c5770 con 0x7faab415ecc0 2023-10-18T09:01:44.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.847+0000 7faaaa7fc640 1 -- 172.21.15.40:0/561659947 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7faaac07ab70 con 0x7faab415ecc0 2023-10-18T09:01:44.849 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T09:01:44.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.849+0000 7faabac75640 1 -- 172.21.15.40:0/561659947 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa9806ae70 msgr2=0x7faa9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:44.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.849+0000 7faabac75640 1 --2- 172.21.15.40:0/561659947 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa9806ae70 0x7faa9806d330 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7faab00058c0 tx=0x7faab0005c90 comp rx=0 tx=0).stop 2023-10-18T09:01:44.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.849+0000 7faabac75640 1 -- 172.21.15.40:0/561659947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab415ecc0 msgr2=0x7faab414bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:44.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.849+0000 7faabac75640 1 --2- 172.21.15.40:0/561659947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab415ecc0 0x7faab414bd10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7faaac037040 tx=0x7faaac035db0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.849+0000 7faabac75640 1 -- 172.21.15.40:0/561659947 shutdown_connections 2023-10-18T09:01:44.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.849+0000 7faabac75640 1 --2- 172.21.15.40:0/561659947 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa9806ae70 0x7faa9806d330 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.849+0000 7faabac75640 1 --2- 172.21.15.40:0/561659947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab415ecc0 0x7faab414bd10 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.849+0000 7faabac75640 1 --2- 172.21.15.40:0/561659947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faab40d1910 0x7faab414b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.849+0000 7faabac75640 1 --2- 172.21.15.40:0/561659947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faab40b2810 0x7faab406b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:44.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.849+0000 7faabac75640 1 -- 172.21.15.40:0/561659947 >> 172.21.15.40:0/561659947 conn(0x7faab408ac80 msgr2=0x7faab415b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.849+0000 7faabac75640 1 -- 172.21.15.40:0/561659947 shutdown_connections 2023-10-18T09:01:44.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:44.849+0000 7faabac75640 1 -- 172.21.15.40:0/561659947 wait complete. 2023-10-18T09:01:44.871 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T09:01:44.872 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T09:01:44.872 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-10-18T09:01:44.872 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T09:01:44.872 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T09:01:44.873 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T09:01:44.873 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T09:01:44.873 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T09:01:44.874 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:01:44.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:44.876 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-10-18T09:01:44.932 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T09:01:44.933 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T09:01:44.933 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T09:01:44.933 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:01:44.934 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-18T09:01:44.934 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:01:44.934 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:01:44.934 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:01:44.934 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-10-18T09:01:44.935 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:01:44.935 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:01:44.935 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-10-18T09:01:44.935 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T09:01:44.935 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-10-18T09:01:44.936 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-10-18T09:01:44.936 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-18T09:01:44.936 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42019sec preferred_lft 42019sec 2023-10-18T09:01:44.936 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2023-10-18T09:01:44.936 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:01:44.937 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:01:44.937 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-10-18T09:01:44.937 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-10-18T09:01:44.937 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T09:01:44.938 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:01:44.938 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T09:01:44.938 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:01:44.939 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2026:95ff:fea9:a463/64 scope link 2023-10-18T09:01:44.939 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:01:44.939 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:01:44.939 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T09:01:44.939 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::80bd:e5ff:fe0f:6f68/64 scope link 2023-10-18T09:01:44.940 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:01:44.940 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:01:44.940 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether ea:2b:21:f5:06:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T09:01:44.940 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::e82b:21ff:fef5:6af/64 scope link 2023-10-18T09:01:44.940 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:01:44.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:44.942 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T09:01:45.042 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-10-18T09:01:45.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:45.047 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-10-18T09:01:45.109 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-10-18T09:01:45.109 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T09:01:45.110 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-18T09:01:45.110 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-18T09:01:45.110 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T09:01:45.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:45.111 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T09:01:45.112 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T09:01:45.112 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-18T09:01:45.112 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-18T09:01:45.113 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-18T09:01:45.113 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T09:01:45.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:45.310 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-18T09:01:45.369 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T09:01:45.369 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T09:01:45.370 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:01:45.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:45.371 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-10-18T09:01:45.477 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:01:45.478 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T09:01:45.539 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:01:45.539 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:01:45.540 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T09:01:45.598 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-18T09:01:45.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:45.599 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:01:45.744 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:01:45.744 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:01:45.780 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:01:45.780 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:01:45.780 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T09:01:45.780 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T09:01:45.781 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T09:01:45.782 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-18T09:01:45.783 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2023-10-18T09:01:45.783 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T09:01:45.783 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T09:01:45.783 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T09:01:45.787 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T09:01:45.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:45.787 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T09:01:45.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:45.880 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:01:45.953 INFO:teuthology.orchestra.run.smithi040.stdout:client5950 2023-10-18T09:01:45.953 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:01:45.954 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:01:46.012 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T09:01:46.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.244+0000 7fbdef06d640 1 -- 172.21.15.40:0/2542042710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde806d6f0 msgr2=0x7fbde8071310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:46.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.244+0000 7fbdef06d640 1 --2- 172.21.15.40:0/2542042710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde806d6f0 0x7fbde8071310 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbde8066660 tx=0x7fbde402bd70 comp rx=0 tx=0).stop 2023-10-18T09:01:46.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.244+0000 7fbdef06d640 1 -- 172.21.15.40:0/2542042710 shutdown_connections 2023-10-18T09:01:46.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.244+0000 7fbdef06d640 1 --2- 172.21.15.40:0/2542042710 >> v2:172.21.15.144:3300/0 conn(0x7fbde8157cd0 0x7fbde8158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:46.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.244+0000 7fbdef06d640 1 --2- 172.21.15.40:0/2542042710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde806d6f0 0x7fbde8071310 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:46.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.244+0000 7fbdef06d640 1 -- 172.21.15.40:0/2542042710 >> 172.21.15.40:0/2542042710 conn(0x7fbde80aed70 msgr2=0x7fbde80af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.244+0000 7fbdef06d640 1 -- 172.21.15.40:0/2542042710 shutdown_connections 2023-10-18T09:01:46.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.244+0000 7fbdef06d640 1 -- 172.21.15.40:0/2542042710 wait complete. 2023-10-18T09:01:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.246+0000 7fbdef06d640 1 Processor -- start 2023-10-18T09:01:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.246+0000 7fbdef06d640 1 -- start start 2023-10-18T09:01:46.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.247+0000 7fbdef06d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbde806d6f0 0x7fbde8139c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.247+0000 7fbdef06d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde8157cd0 0x7fbde8153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.247+0000 7fbdef06d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbde815af90 0x7fbde8156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:46.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.247+0000 7fbdef06d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde80adfc0 con 0x7fbde8157cd0 2023-10-18T09:01:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.247+0000 7fbdef06d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde80ccdc0 con 0x7fbde806d6f0 2023-10-18T09:01:46.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.247+0000 7fbdef06d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde8153d80 con 0x7fbde815af90 2023-10-18T09:01:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.247+0000 7fbded86a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde8157cd0 0x7fbde8153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.247+0000 7fbded86a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde8157cd0 0x7fbde8153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40226/0 (socket says 172.21.15.40:40226) 2023-10-18T09:01:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.247+0000 7fbded86a640 1 -- 172.21.15.40:0/3508662217 learned_addr learned my addr 172.21.15.40:0/3508662217 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:46.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.247+0000 7fbdee06b640 1 --2- 172.21.15.40:0/3508662217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbde806d6f0 0x7fbde8139c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:46.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.248+0000 7fbdee86c640 1 --2- 172.21.15.40:0/3508662217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbde815af90 0x7fbde8156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:46.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.248+0000 7fbded86a640 1 -- 172.21.15.40:0/3508662217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbde815af90 msgr2=0x7fbde8156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.248+0000 7fbded86a640 1 --2- 172.21.15.40:0/3508662217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbde815af90 0x7fbde8156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.248+0000 7fbded86a640 1 -- 172.21.15.40:0/3508662217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbde806d6f0 msgr2=0x7fbde8139c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:46.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.248+0000 7fbded86a640 1 --2- 172.21.15.40:0/3508662217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbde806d6f0 0x7fbde8139c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:46.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.248+0000 7fbded86a640 1 -- 172.21.15.40:0/3508662217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde4009490 con 0x7fbde8157cd0 2023-10-18T09:01:46.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.248+0000 7fbdee06b640 1 --2- 172.21.15.40:0/3508662217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbde806d6f0 0x7fbde8139c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.248+0000 7fbdee86c640 1 --2- 172.21.15.40:0/3508662217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbde815af90 0x7fbde8156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.248+0000 7fbded86a640 1 --2- 172.21.15.40:0/3508662217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde8157cd0 0x7fbde8153840 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800d770 tx=0x7fbdd800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.248+0000 7fbddeffd640 1 -- 172.21.15.40:0/3508662217 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8009c30 con 0x7fbde8157cd0 2023-10-18T09:01:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.248+0000 7fbddeffd640 1 -- 172.21.15.40:0/3508662217 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbdd800de80 con 0x7fbde8157cd0 2023-10-18T09:01:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.248+0000 7fbddeffd640 1 -- 172.21.15.40:0/3508662217 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8004260 con 0x7fbde8157cd0 2023-10-18T09:01:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.248+0000 7fbdef06d640 1 -- 172.21.15.40:0/3508662217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde80adce0 con 0x7fbde8157cd0 2023-10-18T09:01:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.249+0000 7fbdef06d640 1 -- 172.21.15.40:0/3508662217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde80cca50 con 0x7fbde8157cd0 2023-10-18T09:01:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.250+0000 7fbdef06d640 1 -- 172.21.15.40:0/3508662217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde8066e00 con 0x7fbde8157cd0 2023-10-18T09:01:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.251+0000 7fbddeffd640 1 -- 172.21.15.40:0/3508662217 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbdd80035f0 con 0x7fbde8157cd0 2023-10-18T09:01:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.251+0000 7fbddeffd640 1 --2- 172.21.15.40:0/3508662217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbdcc06ae40 0x7fbdcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.252+0000 7fbddeffd640 1 -- 172.21.15.40:0/3508662217 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbdd8089820 con 0x7fbde8157cd0 2023-10-18T09:01:46.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.252+0000 7fbdee06b640 1 --2- 172.21.15.40:0/3508662217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbdcc06ae40 0x7fbdcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:46.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.253+0000 7fbdee06b640 1 --2- 172.21.15.40:0/3508662217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbdcc06ae40 0x7fbdcc06d300 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbde4036000 tx=0x7fbde4004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:46.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.256+0000 7fbddeffd640 1 -- 172.21.15.40:0/3508662217 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbdd80548c0 con 0x7fbde8157cd0 2023-10-18T09:01:46.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.491+0000 7fbdef06d640 1 -- 172.21.15.40:0/3508662217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbde80706b0 con 0x7fbde8157cd0 2023-10-18T09:01:46.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.492+0000 7fbddeffd640 1 -- 172.21.15.40:0/3508662217 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbdd8017070 con 0x7fbde8157cd0 2023-10-18T09:01:46.494 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T09:01:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.494+0000 7fbdef06d640 1 -- 172.21.15.40:0/3508662217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbdcc06ae40 msgr2=0x7fbdcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.494+0000 7fbdef06d640 1 --2- 172.21.15.40:0/3508662217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbdcc06ae40 0x7fbdcc06d300 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbde4036000 tx=0x7fbde4004730 comp rx=0 tx=0).stop 2023-10-18T09:01:46.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.494+0000 7fbdef06d640 1 -- 172.21.15.40:0/3508662217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde8157cd0 msgr2=0x7fbde8153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:46.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.495+0000 7fbdef06d640 1 --2- 172.21.15.40:0/3508662217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde8157cd0 0x7fbde8153840 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800d770 tx=0x7fbdd800dc30 comp rx=0 tx=0).stop 2023-10-18T09:01:46.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.495+0000 7fbdef06d640 1 -- 172.21.15.40:0/3508662217 shutdown_connections 2023-10-18T09:01:46.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.495+0000 7fbdef06d640 1 --2- 172.21.15.40:0/3508662217 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbdcc06ae40 0x7fbdcc06d300 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:46.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.495+0000 7fbdef06d640 1 --2- 172.21.15.40:0/3508662217 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbde815af90 0x7fbde8156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:46.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.495+0000 7fbdef06d640 1 --2- 172.21.15.40:0/3508662217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbde8157cd0 0x7fbde8153840 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:46.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.495+0000 7fbdef06d640 1 --2- 172.21.15.40:0/3508662217 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbde806d6f0 0x7fbde8139c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:46.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.495+0000 7fbdef06d640 1 -- 172.21.15.40:0/3508662217 >> 172.21.15.40:0/3508662217 conn(0x7fbde80aed70 msgr2=0x7fbde80c9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:46.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.495+0000 7fbdef06d640 1 -- 172.21.15.40:0/3508662217 shutdown_connections 2023-10-18T09:01:46.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:46.495+0000 7fbdef06d640 1 -- 172.21.15.40:0/3508662217 wait complete. 2023-10-18T09:01:46.516 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T09:01:46.517 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T09:01:46.517 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-10-18T09:01:46.518 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-18T09:01:46.518 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T09:01:46.518 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T09:01:46.519 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T09:01:46.519 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T09:01:46.519 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:01:46.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:46.520 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-10-18T09:01:46.581 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T09:01:46.581 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T09:01:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T09:01:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:01:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-18T09:01:46.582 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:01:46.583 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:01:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:01:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2023-10-18T09:01:46.583 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:01:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:01:46.583 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2023-10-18T09:01:46.584 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:01:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-10-18T09:01:46.584 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T09:01:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-10-18T09:01:46.584 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T09:01:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42011sec preferred_lft 42011sec 2023-10-18T09:01:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2023-10-18T09:01:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:01:46.585 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T09:01:46.585 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff 2023-10-18T09:01:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T09:01:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:01:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1493:a0ff:fed2:2c1/64 scope link 2023-10-18T09:01:46.586 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:01:46.586 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:01:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 4a:6e:a9:f0:92:62 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T09:01:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::486e:a9ff:fef0:9262/64 scope link 2023-10-18T09:01:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:01:46.587 INFO:teuthology.orchestra.run.smithi144.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:01:46.587 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T09:01:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::345a:c5ff:fe54:de6e/64 scope link 2023-10-18T09:01:46.588 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:01:46.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:46.589 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T09:01:46.678 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-10-18T09:01:46.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:46.682 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-10-18T09:01:46.737 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-10-18T09:01:46.737 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T09:01:46.737 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T09:01:46.737 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T09:01:46.738 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T09:01:46.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:46.739 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T09:01:46.739 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T09:01:46.739 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T09:01:46.740 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T09:01:46.740 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T09:01:46.740 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T09:01:46.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:46.918 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-18T09:01:46.968 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T09:01:46.968 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-18T09:01:46.969 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:01:46.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:46.969 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2023-10-18T09:01:47.063 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:01:47.064 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-18T09:01:47.116 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:01:47.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:01:47.117 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-18T09:01:47.165 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-18T09:01:47.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:47.166 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:01:47.302 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:01:47.303 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:01:47.336 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:01:47.336 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:01:47.336 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T09:01:47.336 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with new device syntax: 1@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T09:01:47.337 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T09:01:47.338 INFO:teuthology.orchestra.run.smithi144.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-18T09:01:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: You just mounted a file system that supports labels which does not 2023-10-18T09:01:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T09:01:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T09:01:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T09:01:47.342 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T09:01:47.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:47.343 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-18T09:01:47.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:47.424 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:01:47.477 INFO:teuthology.orchestra.run.smithi144.stdout:client5513 2023-10-18T09:01:47.477 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:01:47.478 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:01:47.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:01:47.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.763+0000 7f16c0987640 1 -- 172.21.15.40:0/3893214940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a8004a80 msgr2=0x7f16a8004ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:47.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.763+0000 7f16c0987640 1 --2- 172.21.15.40:0/3893214940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16a8004a80 0x7f16a8004ea0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009b50 tx=0x7f16a8006900 comp rx=0 tx=0).stop 2023-10-18T09:01:47.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.763+0000 7f16bbfff640 1 -- 172.21.15.40:0/3893214940 reap_dead start 2023-10-18T09:01:47.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.763+0000 7f16c0987640 1 -- 172.21.15.40:0/3893214940 shutdown_connections 2023-10-18T09:01:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.763+0000 7f16c0987640 1 -- 172.21.15.40:0/3893214940 >> 172.21.15.40:0/3893214940 conn(0x7f16bc069720 msgr2=0x7f16bc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.763+0000 7f16c0987640 1 -- 172.21.15.40:0/3893214940 shutdown_connections 2023-10-18T09:01:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.763+0000 7f16c0987640 1 -- 172.21.15.40:0/3893214940 wait complete. 2023-10-18T09:01:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.765+0000 7f16c0987640 1 Processor -- start 2023-10-18T09:01:47.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.765+0000 7f16c0987640 1 -- start start 2023-10-18T09:01:47.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.766+0000 7f16c0987640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a8004a80 0x7f16bc0ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:47.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.766+0000 7f16c0987640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16bc0aa880 0x7f16bc142240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:47.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.766+0000 7f16c0987640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16bc0aace0 0x7f16bc0af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.766+0000 7f16c0987640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc0aebf0 con 0x7f16bc0aa880 2023-10-18T09:01:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.766+0000 7f16c0987640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc0ab7e0 con 0x7f16a8004a80 2023-10-18T09:01:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.766+0000 7f16c0987640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc0ad780 con 0x7f16bc0aace0 2023-10-18T09:01:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.767+0000 7f16b3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16bc0aa880 0x7f16bc142240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:47.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.767+0000 7f16b3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16bc0aa880 0x7f16bc142240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40246/0 (socket says 172.21.15.40:40246) 2023-10-18T09:01:47.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.767+0000 7f16b3fff640 1 -- 172.21.15.40:0/1817632290 learned_addr learned my addr 172.21.15.40:0/1817632290 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:47.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.767+0000 7f16bb7fe640 1 --2- 172.21.15.40:0/1817632290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a8004a80 0x7f16bc0ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:47.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.767+0000 7f16bbfff640 1 --2- 172.21.15.40:0/1817632290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16bc0aace0 0x7f16bc0af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:47.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.767+0000 7f16b3fff640 1 -- 172.21.15.40:0/1817632290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16bc0aace0 msgr2=0x7f16bc0af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:47.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.767+0000 7f16b3fff640 1 --2- 172.21.15.40:0/1817632290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16bc0aace0 0x7f16bc0af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:47.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.767+0000 7f16b3fff640 1 -- 172.21.15.40:0/1817632290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a8004a80 msgr2=0x7f16bc0ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:47.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.767+0000 7f16b3fff640 1 --2- 172.21.15.40:0/1817632290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a8004a80 0x7f16bc0ad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:47.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.767+0000 7f16b3fff640 1 -- 172.21.15.40:0/1817632290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc0ab220 con 0x7f16bc0aa880 2023-10-18T09:01:47.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.767+0000 7f16bbfff640 1 --2- 172.21.15.40:0/1817632290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16bc0aace0 0x7f16bc0af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:47.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.768+0000 7f16b3fff640 1 --2- 172.21.15.40:0/1817632290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16bc0aa880 0x7f16bc142240 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f16a400d7a0 tx=0x7f16a400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:47.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.768+0000 7f16b8ff9640 1 -- 172.21.15.40:0/1817632290 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a40050d0 con 0x7f16bc0aa880 2023-10-18T09:01:47.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.768+0000 7f16b8ff9640 1 -- 172.21.15.40:0/1817632290 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f16a4005230 con 0x7f16bc0aa880 2023-10-18T09:01:47.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.768+0000 7f16b8ff9640 1 -- 172.21.15.40:0/1817632290 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a400b6c0 con 0x7f16bc0aa880 2023-10-18T09:01:47.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.768+0000 7f16c0987640 1 -- 172.21.15.40:0/1817632290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16bc0ab500 con 0x7f16bc0aa880 2023-10-18T09:01:47.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.768+0000 7f16c0987640 1 -- 172.21.15.40:0/1817632290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc0af100 con 0x7f16bc0aa880 2023-10-18T09:01:47.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.769+0000 7f16c0987640 1 -- 172.21.15.40:0/1817632290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16bc066e00 con 0x7f16bc0aa880 2023-10-18T09:01:47.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.770+0000 7f16b8ff9640 1 -- 172.21.15.40:0/1817632290 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f16a400b820 con 0x7f16bc0aa880 2023-10-18T09:01:47.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.771+0000 7f16b8ff9640 1 --2- 172.21.15.40:0/1817632290 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f169c06ae40 0x7f169c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:47.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.771+0000 7f16b8ff9640 1 -- 172.21.15.40:0/1817632290 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16a4088f40 con 0x7f16bc0aa880 2023-10-18T09:01:47.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.772+0000 7f16bb7fe640 1 --2- 172.21.15.40:0/1817632290 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f169c06ae40 0x7f169c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:47.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.773+0000 7f16bb7fe640 1 --2- 172.21.15.40:0/1817632290 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f169c06ae40 0x7f169c06d300 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f16ac036000 tx=0x7f16ac035040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:47.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:47.776+0000 7f16b8ff9640 1 -- 172.21.15.40:0/1817632290 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f16a4053ff0 con 0x7f16bc0aa880 2023-10-18T09:01:48.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:48.006+0000 7f16c0987640 1 -- 172.21.15.40:0/1817632290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16bc0caf90 con 0x7f16bc0aa880 2023-10-18T09:01:48.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:48.007+0000 7f16b8ff9640 1 -- 172.21.15.40:0/1817632290 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f16a4019090 con 0x7f16bc0aa880 2023-10-18T09:01:48.009 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:48.010 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2565609773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2565609773},{"type":"v1","addr":"172.21.15.144:6836","nonce":2565609773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":5872,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3371548374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3371548374},{"type":"v1","addr":"172.21.15.144:6837","nonce":3371548374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3328648628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3328648628},{"type":"v1","addr":"172.21.15.40:6833","nonce":3328648628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:01:36.995669+0000","modified":"2023-10-18T09:01:40.068484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/891188043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":891188043},{"type":"v1","addr":"172.21.15.40:6835","nonce":891188043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-18T09:01:48.010 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 49 2023-10-18T09:01:48.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:48.009+0000 7f16c0987640 1 -- 172.21.15.40:0/1817632290 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f169c06ae40 msgr2=0x7f169c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:48.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:48.009+0000 7f16c0987640 1 --2- 172.21.15.40:0/1817632290 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f169c06ae40 0x7f169c06d300 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f16ac036000 tx=0x7f16ac035040 comp rx=0 tx=0).stop 2023-10-18T09:01:48.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:48.009+0000 7f16c0987640 1 -- 172.21.15.40:0/1817632290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16bc0aa880 msgr2=0x7f16bc142240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:48.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:48.009+0000 7f16c0987640 1 --2- 172.21.15.40:0/1817632290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16bc0aa880 0x7f16bc142240 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f16a400d7a0 tx=0x7f16a400dc60 comp rx=0 tx=0).stop 2023-10-18T09:01:48.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:48.010+0000 7f16c0987640 1 -- 172.21.15.40:0/1817632290 shutdown_connections 2023-10-18T09:01:48.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:48.010+0000 7f16c0987640 1 --2- 172.21.15.40:0/1817632290 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f169c06ae40 0x7f169c06d300 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:48.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:48.010+0000 7f16c0987640 1 --2- 172.21.15.40:0/1817632290 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16bc0aace0 0x7f16bc0af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:48.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:48.010+0000 7f16c0987640 1 --2- 172.21.15.40:0/1817632290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16bc0aa880 0x7f16bc142240 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:48.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:48.010+0000 7f16c0987640 1 --2- 172.21.15.40:0/1817632290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16a8004a80 0x7f16bc0ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:48.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:48.010+0000 7f16c0987640 1 -- 172.21.15.40:0/1817632290 >> 172.21.15.40:0/1817632290 conn(0x7f16bc069720 msgr2=0x7f16bc0cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:48.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:48.010+0000 7f16c0987640 1 -- 172.21.15.40:0/1817632290 shutdown_connections 2023-10-18T09:01:48.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:48.010+0000 7f16c0987640 1 -- 172.21.15.40:0/1817632290 wait complete. 2023-10-18T09:01:48.032 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-18T09:01:48.259 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":6996,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.703601876,"mdsmap_epoch":49,"osdmap_epoch":107,"osdmap_epoch_barrier":107,"uptime":16.559491387000001} 2023-10-18T09:01:48.260 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 6996, "mdsmap_epoch": 49, "osdmap_epoch": 107, "osdmap_epoch_barrier": 107, "rank_uptime": 10.703601876, "state": "up:active", "uptime": 16.559491387, "want_state": "up:active", "whoami": 0 } 2023-10-18T09:01:48.261 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-18T09:01:48.503 INFO:teuthology.orchestra.run.smithi040.stdout:{"osd_mon_report_interval":"5"} 2023-10-18T09:01:48.503 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-18T09:01:48.504 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2023-10-18T09:01:48.504 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:01:48.504 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:01:48.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:48.561 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2023-10-18T09:01:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:265+0 records in 2023-10-18T09:01:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:265+0 records out 2023-10-18T09:01:50.577 INFO:teuthology.orchestra.run.smithi040.stderr:277872640 bytes (278 MB, 265 MiB) copied, 1.95175 s, 142 MB/s 2023-10-18T09:01:50.578 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:01:50.578 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:01:50.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:50.617 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2023-10-18T09:01:52.851 INFO:teuthology.orchestra.run.smithi040.stderr:292+0 records in 2023-10-18T09:01:52.852 INFO:teuthology.orchestra.run.smithi040.stderr:292+0 records out 2023-10-18T09:01:52.852 INFO:teuthology.orchestra.run.smithi040.stderr:306184192 bytes (306 MB, 292 MiB) copied, 2.13927 s, 143 MB/s 2023-10-18T09:01:52.853 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-18T09:01:52.853 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:01:53.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.055+0000 7f635ffff640 1 -- 172.21.15.40:0/1289592897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63600b1e10 msgr2=0x7f63600c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:53.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.055+0000 7f635ffff640 1 --2- 172.21.15.40:0/1289592897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63600b1e10 0x7f63600c99c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6354005fd0 tx=0x7f635402be20 comp rx=0 tx=0).stop 2023-10-18T09:01:53.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.055+0000 7f635ffff640 1 -- 172.21.15.40:0/1289592897 shutdown_connections 2023-10-18T09:01:53.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.055+0000 7f635ffff640 1 --2- 172.21.15.40:0/1289592897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63600b2270 0x7f63600c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:53.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.055+0000 7f635ffff640 1 --2- 172.21.15.40:0/1289592897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63600b1e10 0x7f63600c99c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:53.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.055+0000 7f635ffff640 1 -- 172.21.15.40:0/1289592897 >> 172.21.15.40:0/1289592897 conn(0x7f6360069720 msgr2=0x7f6360069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:53.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.055+0000 7f635ffff640 1 -- 172.21.15.40:0/1289592897 shutdown_connections 2023-10-18T09:01:53.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.055+0000 7f635ffff640 1 -- 172.21.15.40:0/1289592897 wait complete. 2023-10-18T09:01:53.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.057+0000 7f635ffff640 1 Processor -- start 2023-10-18T09:01:53.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.057+0000 7f635ffff640 1 -- start start 2023-10-18T09:01:53.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.058+0000 7f635ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63600b15b0 0x7f6360147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:53.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.058+0000 7f635ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63600b1e10 0x7f6360150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:53.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.058+0000 7f635ffff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63600b2270 0x7f6360149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:53.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.058+0000 7f635ffff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63600af4d0 con 0x7f63600b1e10 2023-10-18T09:01:53.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.058+0000 7f635ffff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6360148c90 con 0x7f63600b15b0 2023-10-18T09:01:53.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.058+0000 7f635ffff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6360148f70 con 0x7f63600b2270 2023-10-18T09:01:53.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.058+0000 7f635e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63600b1e10 0x7f6360150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:53.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.058+0000 7f635e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63600b1e10 0x7f6360150130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40254/0 (socket says 172.21.15.40:40254) 2023-10-18T09:01:53.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.058+0000 7f635e7fc640 1 -- 172.21.15.40:0/223405885 learned_addr learned my addr 172.21.15.40:0/223405885 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:53.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.059+0000 7f635effd640 1 --2- 172.21.15.40:0/223405885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63600b15b0 0x7f6360147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:53.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.059+0000 7f635f7fe640 1 --2- 172.21.15.40:0/223405885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63600b2270 0x7f6360149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:53.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.059+0000 7f635e7fc640 1 -- 172.21.15.40:0/223405885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63600b2270 msgr2=0x7f6360149d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:53.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.059+0000 7f635e7fc640 1 --2- 172.21.15.40:0/223405885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63600b2270 0x7f6360149d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:53.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.059+0000 7f635e7fc640 1 -- 172.21.15.40:0/223405885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63600b15b0 msgr2=0x7f6360147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:53.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.059+0000 7f635e7fc640 1 --2- 172.21.15.40:0/223405885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63600b15b0 0x7f6360147c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:53.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.059+0000 7f635e7fc640 1 -- 172.21.15.40:0/223405885 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6354009560 con 0x7f63600b1e10 2023-10-18T09:01:53.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.059+0000 7f635f7fe640 1 --2- 172.21.15.40:0/223405885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63600b2270 0x7f6360149d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:01:53.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.059+0000 7f635e7fc640 1 --2- 172.21.15.40:0/223405885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63600b1e10 0x7f6360150130 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6354009850 tx=0x7f635402da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:53.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.059+0000 7f63477fe640 1 -- 172.21.15.40:0/223405885 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635402de60 con 0x7f63600b1e10 2023-10-18T09:01:53.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.060+0000 7f63477fe640 1 -- 172.21.15.40:0/223405885 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f63540036e0 con 0x7f63600b1e10 2023-10-18T09:01:53.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.060+0000 7f63477fe640 1 -- 172.21.15.40:0/223405885 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354003a90 con 0x7f63600b1e10 2023-10-18T09:01:53.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.060+0000 7f635ffff640 1 -- 172.21.15.40:0/223405885 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6360149250 con 0x7f63600b1e10 2023-10-18T09:01:53.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.060+0000 7f635ffff640 1 -- 172.21.15.40:0/223405885 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6360149810 con 0x7f63600b1e10 2023-10-18T09:01:53.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.061+0000 7f635ffff640 1 -- 172.21.15.40:0/223405885 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6360066e00 con 0x7f63600b1e10 2023-10-18T09:01:53.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.062+0000 7f63477fe640 1 -- 172.21.15.40:0/223405885 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6354003bf0 con 0x7f63600b1e10 2023-10-18T09:01:53.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.063+0000 7f63477fe640 1 --2- 172.21.15.40:0/223405885 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f634c06ae40 0x7f634c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:53.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.063+0000 7f63477fe640 1 -- 172.21.15.40:0/223405885 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63540ac230 con 0x7f63600b1e10 2023-10-18T09:01:53.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.063+0000 7f635effd640 1 --2- 172.21.15.40:0/223405885 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f634c06ae40 0x7f634c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:53.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.065+0000 7f635effd640 1 --2- 172.21.15.40:0/223405885 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f634c06ae40 0x7f634c06d300 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6350007fd0 tx=0x7f6350009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:53.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.067+0000 7f63477fe640 1 -- 172.21.15.40:0/223405885 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6354077360 con 0x7f63600b1e10 2023-10-18T09:01:53.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.263+0000 7f635ffff640 1 -- 172.21.15.40:0/223405885 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6360070f40 con 0x7f63600b1e10 2023-10-18T09:01:53.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.266+0000 7f63477fe640 1 -- 172.21.15.40:0/223405885 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v107) v1 ==== 81+0+34018 (secure 0 0 0) 0x7f635407a980 con 0x7f63600b1e10 2023-10-18T09:01:53.269 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:53.269 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:01:53.269 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 107, 2023-10-18T09:01:53.270 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:01:53.270 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:01:53.270 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:01:38.517370+0000", 2023-10-18T09:01:53.270 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:01:53.271 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:01:53.271 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:01:53.271 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:01:53.271 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:01:53.272 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:01:53.272 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:01:53.272 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:01:53.272 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:01:53.272 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:01:53.273 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:01:53.273 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:01:53.273 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:01:53.273 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:01:53.273 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:01:53.274 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 13, 2023-10-18T09:01:53.274 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:01:53.274 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:01:53.274 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:01:53.274 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:01:53.275 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:01:53.275 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:01:53.275 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.275 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:01:53.275 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:01:53.275 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:01:53.276 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:01:53.276 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:01:53.276 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:01:53.276 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:01:53.277 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:01:53.277 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:01:53.277 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:01:53.277 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:01:53.277 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:01:53.278 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:01:53.278 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:01:53.278 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:01:53.278 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:01:53.278 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:01:53.279 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:01:53.279 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:01:53.279 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:01:53.279 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:01:53.279 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:01:53.280 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:01:53.280 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:01:53.280 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:01:53.280 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:01:53.280 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:01:53.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.281 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "95", 2023-10-18T09:01:53.281 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:01:53.281 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:01:53.282 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:01:53.282 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:01:53.282 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:01:53.282 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:01:53.282 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:01:53.283 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:01:53.283 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:01:53.283 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:01:53.283 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:01:53.283 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:01:53.284 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:01:53.284 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:01:53.284 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:01:53.284 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:01:53.284 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:01:53.285 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:01:53.285 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:01:53.285 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:01:53.285 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:01:53.285 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:01:53.285 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:01:53.286 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:01:53.286 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:01:53.286 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:01:53.286 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.286 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:01:53.287 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:01:53.287 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:01:53.287 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:01:53.287 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:01:53.288 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:01:53.288 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:01:53.288 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:01:53.288 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:01:53.288 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:01:53.289 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:01:53.289 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:01:53.289 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:01:53.289 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:01:53.289 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.289 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:01:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:01:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:01:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:01:53.290 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:01:53.291 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:01:53.291 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:01:53.291 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:01:53.291 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:01:53.292 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:01:53.292 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:01:53.292 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.292 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.293 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.293 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 12, 2023-10-18T09:01:53.293 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:01:53.293 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:01:35.453236+0000", 2023-10-18T09:01:53.293 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:01:53.294 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:01:53.294 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:01:53.294 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:01:53.294 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:01:53.294 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:01:53.295 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:01:53.295 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:01:53.295 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:01:53.295 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:01:53.295 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:01:53.295 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:01:53.296 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:01:53.296 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:01:53.296 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:01:53.296 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:01:53.296 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:01:53.297 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:01:53.297 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:01:53.297 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:01:53.297 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:01:53.298 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:01:53.298 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:01:53.298 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:01:53.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.298 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "106", 2023-10-18T09:01:53.299 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:01:53.299 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:01:53.299 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:01:53.299 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:01:53.299 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:01:53.299 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:01:53.300 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:01:53.300 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:01:53.300 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:01:53.300 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:01:53.300 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:01:53.301 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:01:53.301 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:01:53.301 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:01:53.301 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:01:53.301 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:01:53.301 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:01:53.302 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:01:53.302 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:01:53.302 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:01:53.302 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:01:53.303 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:01:53.303 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:01:53.303 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:01:53.303 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:01:53.304 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:01:53.304 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.304 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:01:53.304 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:01:53.304 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:01:53.305 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:01:53.305 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:01:53.305 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:01:53.305 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:01:53.305 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:01:53.305 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:01:53.306 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:01:53.306 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:01:53.306 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:01:53.306 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:01:53.306 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:01:53.307 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:01:53.307 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.307 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:01:53.308 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:01:53.308 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:01:53.308 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.308 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.308 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:01:53.309 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 2, 2023-10-18T09:01:53.309 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 2, 2023-10-18T09:01:53.309 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:01:53.309 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:01:53.309 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:01:53.309 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:01:53.310 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:01:53.310 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:01:53.310 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.310 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.311 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 13, 2023-10-18T09:01:53.311 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:01:53.311 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:01:36.012460+0000", 2023-10-18T09:01:53.311 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:01:53.311 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:01:53.312 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:01:53.312 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:01:53.312 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:01:53.312 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:01:53.312 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:01:53.312 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:01:53.313 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:01:53.313 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:01:53.313 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:01:53.314 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:01:53.314 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:01:53.314 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:01:53.314 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:01:53.314 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:01:53.315 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:01:53.315 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:01:53.315 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:01:53.315 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:01:53.315 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:01:53.316 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:01:53.316 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:01:53.316 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:01:53.316 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.316 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "107", 2023-10-18T09:01:53.317 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:01:53.317 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:01:53.317 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:01:53.317 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:01:53.317 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:01:53.317 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:01:53.318 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:01:53.318 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:01:53.318 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:01:53.319 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:01:53.319 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:01:53.319 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:01:53.319 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:01:53.319 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:01:53.319 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:01:53.320 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:01:53.320 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:01:53.320 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:01:53.320 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:01:53.320 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:01:53.321 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:01:53.321 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:01:53.321 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:01:53.321 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:01:53.321 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:01:53.322 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:01:53.322 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.322 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:01:53.322 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:01:53.322 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:01:53.323 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:01:53.323 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:01:53.323 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:01:53.323 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:01:53.323 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:01:53.324 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:01:53.324 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:01:53.324 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:01:53.324 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:01:53.325 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:01:53.325 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:01:53.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.325 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:01:53.325 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:01:53.326 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:01:53.326 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.326 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.326 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:01:53.326 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.3799999952316284, 2023-10-18T09:01:53.327 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.3799999952316284, 2023-10-18T09:01:53.327 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:01:53.327 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-18T09:01:53.327 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-18T09:01:53.327 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:01:53.328 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:01:53.328 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:01:53.328 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.328 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.328 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:01:53.329 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:01:53.329 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.329 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:01:53.329 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:01:53.329 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:01:53.330 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:01:53.330 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:01:53.330 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:01:53.330 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:01:53.330 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:01:53.331 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:01:53.331 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:01:53.331 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:01:53.331 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:01:53.331 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:01:53.332 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.332 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.332 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.332 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:01:53.332 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:01:53.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.333 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.333 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:01:53.333 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:01:53.333 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.334 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.334 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:01:53.335 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.335 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.336 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:01:53.336 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:01:53.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.336 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.337 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:01:53.337 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:01:53.337 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.337 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.338 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:01:53.338 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.338 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.338 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:01:53.339 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:01:53.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.339 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.340 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:01:53.340 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:01:53.340 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.340 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.341 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:01:53.341 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.341 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.341 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:01:53.342 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:01:53.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.343 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.343 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:01:53.343 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:01:53.344 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.344 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.345 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.345 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:01:53.345 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:01:53.346 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:01:53.346 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:01:53.346 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:01:53.347 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:01:53.347 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:01:53.348 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.349 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:01:53.349 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:01:53.349 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:01:53.350 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:01:53.350 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:01:53.351 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:01:53.351 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:01:53.351 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:01:53.352 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:01:53.352 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:01:53.352 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:01:53.353 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:01:53.353 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:01:53.354 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.354 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.355 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:01:53.355 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:01:53.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.356 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.357 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:01:53.357 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:01:53.357 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.358 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.359 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:01:53.359 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.360 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.360 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:01:53.360 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:01:53.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.361 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.361 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.362 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:01:53.362 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:01:53.363 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.363 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.363 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.363 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:01:53.364 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.364 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.364 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.365 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:01:53.365 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:01:53.365 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.365 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.366 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.366 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:01:53.366 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:01:53.367 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.367 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.367 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.367 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:01:53.368 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.368 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.368 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.368 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:01:53.368 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:01:53.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.369 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.369 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.369 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:01:53.369 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:01:53.369 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.370 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.370 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.370 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:01:53.370 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:01:53.370 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:01:53.371 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:01:53.371 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:01:53.371 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:01:53.371 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:01:53.371 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.372 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.372 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.372 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:01:53.372 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:01:53.372 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:01:53.372 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:01:53.373 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:01:53.373 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:01:53.373 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:01:53.373 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:01:53.373 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:01:53.373 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:01:53.374 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:01:53.374 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:01:53.374 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:01:53.374 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.374 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.375 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.375 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:01:53.375 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:01:53.376 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.376 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.376 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:01:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:01:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.377 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:01:53.378 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.379 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.379 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.379 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:01:53.379 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:01:53.380 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.380 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.380 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.380 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:01:53.380 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:01:53.381 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.381 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.381 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.381 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:01:53.382 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.382 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.382 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.382 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:01:53.382 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:01:53.383 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.383 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.383 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.383 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:01:53.383 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:01:53.384 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.384 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.384 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.384 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:01:53.384 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.385 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.385 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.385 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:01:53.385 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:01:53.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.386 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.386 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.386 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:01:53.386 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:01:53.387 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.387 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.387 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.387 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:01:53.387 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:01:53.388 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:01:53.388 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:01:53.388 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:01:53.388 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:01:53.388 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:01:53.389 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.389 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:01:53.389 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:01:53.390 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:01:53.390 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:01:53.390 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:01:53.390 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:01:53.391 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:01:53.391 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:01:53.391 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:01:53.391 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:01:53.391 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:01:53.392 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:01:53.392 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:01:53.392 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.392 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.392 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.393 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:01:53.393 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:01:53.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.393 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.393 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.394 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:01:53.394 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:01:53.394 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.394 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.395 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:01:53.395 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.395 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.396 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:01:53.396 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:01:53.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.396 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.396 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.397 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:01:53.397 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:01:53.397 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.397 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.397 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:01:53.398 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.398 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.398 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.398 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:01:53.398 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:01:53.399 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.399 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.399 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.399 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:01:53.399 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:01:53.400 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.400 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.400 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:01:53.400 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.400 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.401 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.401 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:01:53.401 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:01:53.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.402 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.402 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.402 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:01:53.403 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:01:53.403 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.403 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.403 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.403 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:01:53.404 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:01:53.404 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:01:53.404 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:01:53.404 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:01:53.404 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:01:53.405 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:01:53.405 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.405 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.405 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:01:53.406 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:01:53.406 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:01:53.406 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:01:53.407 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:01:53.407 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:01:53.407 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:01:53.407 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:01:53.407 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:01:53.408 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:01:53.408 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:01:53.408 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:01:53.408 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:01:53.408 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.409 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.409 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.409 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:01:53.409 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:01:53.409 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.410 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.410 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.410 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:01:53.410 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:01:53.410 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.411 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.411 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:01:53.411 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.411 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.412 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.412 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:01:53.412 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:01:53.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.413 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.413 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.413 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:01:53.413 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:01:53.413 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.414 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.414 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.414 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:01:53.414 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.414 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.415 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.415 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:01:53.415 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:01:53.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.416 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.416 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.416 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:01:53.416 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:01:53.416 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.417 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.417 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.417 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:01:53.417 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.417 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.417 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.418 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:01:53.418 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:01:53.418 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.418 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.418 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.419 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:01:53.419 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:01:53.419 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.419 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.419 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.420 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:01:53.420 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:01:53.420 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:01:53.420 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:01:53.420 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:01:53.421 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:01:53.421 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:01:53.421 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.421 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.421 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:01:53.422 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:01:53.422 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:01:53.422 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:01:53.423 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:01:53.423 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:01:53.423 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:01:53.423 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:01:53.424 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:01:53.424 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:01:53.424 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:01:53.424 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:01:53.425 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:01:53.425 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.425 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.425 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.425 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:01:53.425 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:01:53.426 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.426 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.426 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.426 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:01:53.427 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:01:53.427 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.427 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.427 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:01:53.427 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.428 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.428 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.428 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:01:53.428 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:01:53.428 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.429 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.429 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.429 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:01:53.429 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:01:53.429 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.429 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.430 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.430 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:01:53.430 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.430 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.430 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.431 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:01:53.431 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:01:53.431 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.431 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.431 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.432 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:01:53.432 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:01:53.432 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.432 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.432 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.433 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:01:53.433 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.433 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.433 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.434 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:01:53.434 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:01:53.434 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.434 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.434 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.435 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:01:53.435 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:01:53.435 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.435 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.435 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.436 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:01:53.436 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:01:53.436 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:01:53.436 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:01:53.436 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:01:53.437 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:01:53.437 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:01:53.437 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.437 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.438 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:01:53.438 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:01:53.438 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:01:53.438 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:01:53.439 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:01:53.439 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:01:53.439 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:01:53.439 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:01:53.440 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:01:53.440 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:01:53.440 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:01:53.440 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:01:53.440 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:01:53.441 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.441 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.441 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.441 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:01:53.441 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:01:53.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.442 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.442 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.442 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:01:53.442 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:01:53.443 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.443 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.443 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:01:53.443 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.444 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.444 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.444 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:01:53.444 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:01:53.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.445 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.445 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.445 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:01:53.445 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:01:53.445 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.446 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.446 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:01:53.446 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.447 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.447 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:01:53.447 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:01:53.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.448 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.448 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:01:53.448 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:01:53.448 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.449 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.449 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:01:53.449 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.449 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.449 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.450 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:01:53.450 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:01:53.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.451 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.451 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:01:53.451 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:01:53.451 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.452 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.452 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:01:53.452 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:01:53.452 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:01:53.452 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:01:53.453 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:01:53.453 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:01:53.453 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:01:53.453 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.454 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.454 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:01:53.454 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:01:53.454 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:01:53.454 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:01:53.455 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:01:53.455 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:01:53.455 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:01:53.455 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:01:53.455 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:01:53.456 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:01:53.456 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:01:53.456 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:01:53.456 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:01:53.456 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:01:53.457 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:01:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:01:53.458 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:01:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:01:53.459 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:01:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:01:53.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:01:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:01:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.461 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:01:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.462 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:01:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:01:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.463 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:01:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:01:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:01:53.464 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:01:53.465 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:01:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:01:53.466 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:01:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.467 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:01:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:01:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:01:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:01:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:01:53.468 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:01:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:01:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:01:53.469 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:01:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:01:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:01:53.470 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:01:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:01:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:01:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:01:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:01:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:01:53.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:01:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:01:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:01:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:01:53.472 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:01:53.473 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:01:53.473 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:01:53.473 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:01:53.473 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.473 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.473 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:01:53.474 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:01:53.474 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:01:53.474 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:01:53.474 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:01:53.474 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:01:53.475 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:01:53.475 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:01:53.475 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.475 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.476 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:01:53.476 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:01:53.476 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:01:53.476 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:01:53.476 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:01:53.476 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:01:53.477 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:01:53.477 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:01:53.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.478 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:01:53.478 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:01:53.479 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:01:53.479 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:01:53.479 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:01:53.479 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:01:53.480 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:01:53.480 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:01:53.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.480 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.481 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:01:53.481 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:01:53.481 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:01:53.482 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:01:53.482 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:01:53.482 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:01:53.483 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:01:53.483 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:01:53.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.483 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.483 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:01:53.484 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:01:53.484 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:01:53.484 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:01:53.484 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:01:53.485 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:01:53.485 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:01:53.485 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:01:53.485 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.486 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:53.486 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:01:53.486 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:01:53.486 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:01:53.487 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:01:53.487 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:01:53.487 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:01:53.487 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:01:53.487 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:01:53.488 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.488 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:01:53.488 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:01:53.488 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:01:53.488 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:01:53.489 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:01:53.489 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:01:53.489 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:01:53.489 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:01:53.489 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:01:53.489 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:01:53.490 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:01:53.490 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:01:53.490 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:01:53.490 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:01:53.491 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:01:53.491 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.491 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:53.491 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:01:53.491 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:01:53.492 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:01:53.492 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:01:53.492 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:01:53.492 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:01:53.492 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:01:53.493 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:01:53.493 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:01:53.493 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:01:53.493 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:01:53.494 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:53.494 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:01:53.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.269+0000 7f635ffff640 1 -- 172.21.15.40:0/223405885 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f634c06ae40 msgr2=0x7f634c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:53.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.269+0000 7f635ffff640 1 --2- 172.21.15.40:0/223405885 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f634c06ae40 0x7f634c06d300 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6350007fd0 tx=0x7f6350009040 comp rx=0 tx=0).stop 2023-10-18T09:01:53.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.269+0000 7f635ffff640 1 -- 172.21.15.40:0/223405885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63600b1e10 msgr2=0x7f6360150130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:53.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.269+0000 7f635ffff640 1 --2- 172.21.15.40:0/223405885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63600b1e10 0x7f6360150130 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6354009850 tx=0x7f635402da60 comp rx=0 tx=0).stop 2023-10-18T09:01:53.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.269+0000 7f635ffff640 1 -- 172.21.15.40:0/223405885 shutdown_connections 2023-10-18T09:01:53.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.269+0000 7f635ffff640 1 --2- 172.21.15.40:0/223405885 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f634c06ae40 0x7f634c06d300 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:53.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.269+0000 7f635ffff640 1 --2- 172.21.15.40:0/223405885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63600b2270 0x7f6360149d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:53.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.269+0000 7f635ffff640 1 --2- 172.21.15.40:0/223405885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63600b1e10 0x7f6360150130 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:53.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.269+0000 7f635ffff640 1 --2- 172.21.15.40:0/223405885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63600b15b0 0x7f6360147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:53.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.269+0000 7f635ffff640 1 -- 172.21.15.40:0/223405885 >> 172.21.15.40:0/223405885 conn(0x7f6360069720 msgr2=0x7f6360159920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:53.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.269+0000 7f635ffff640 1 -- 172.21.15.40:0/223405885 shutdown_connections 2023-10-18T09:01:53.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:53.269+0000 7f635ffff640 1 -- 172.21.15.40:0/223405885 wait complete. 2023-10-18T09:01:53.499 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-18T09:01:54.098 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:01:54.095+0000 7f6a11f15640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2023-10-18T09:01:58.500 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:01:58.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.735+0000 7fc8d8a1a640 1 -- 172.21.15.40:0/3768546894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d406d6f0 msgr2=0x7fc8d40b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:58.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.735+0000 7fc8d8a1a640 1 --2- 172.21.15.40:0/3768546894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d406d6f0 0x7fc8d40b15b0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc8d406aeb0 tx=0x7fc8c402bd90 comp rx=0 tx=0).stop 2023-10-18T09:01:58.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.736+0000 7fc8d8a1a640 1 -- 172.21.15.40:0/3768546894 shutdown_connections 2023-10-18T09:01:58.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.736+0000 7fc8d8a1a640 1 --2- 172.21.15.40:0/3768546894 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8d40ca0d0 0x7fc8d4157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:58.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.736+0000 7fc8d8a1a640 1 --2- 172.21.15.40:0/3768546894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d406d6f0 0x7fc8d40b15b0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:58.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.736+0000 7fc8d8a1a640 1 -- 172.21.15.40:0/3768546894 >> 172.21.15.40:0/3768546894 conn(0x7fc8d4069720 msgr2=0x7fc8d4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:58.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.736+0000 7fc8d8a1a640 1 -- 172.21.15.40:0/3768546894 shutdown_connections 2023-10-18T09:01:58.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.736+0000 7fc8d8a1a640 1 -- 172.21.15.40:0/3768546894 wait complete. 2023-10-18T09:01:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.737+0000 7fc8d8a1a640 1 Processor -- start 2023-10-18T09:01:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.738+0000 7fc8d8a1a640 1 -- start start 2023-10-18T09:01:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.738+0000 7fc8d8a1a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d406d6f0 0x7fc8d414b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.738+0000 7fc8d8a1a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d40b0630 0x7fc8d4153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.739+0000 7fc8d8a1a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8d40ca0d0 0x7fc8d4153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.739+0000 7fc8d8a1a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d414c1f0 con 0x7fc8d40b0630 2023-10-18T09:01:58.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.739+0000 7fc8d8a1a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d40aa920 con 0x7fc8d406d6f0 2023-10-18T09:01:58.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.739+0000 7fc8d8a1a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d40aac00 con 0x7fc8d40ca0d0 2023-10-18T09:01:58.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.739+0000 7fc8cbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d40b0630 0x7fc8d4153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:58.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.739+0000 7fc8cbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d40b0630 0x7fc8d4153440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34810/0 (socket says 172.21.15.40:34810) 2023-10-18T09:01:58.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.739+0000 7fc8cbfff640 1 -- 172.21.15.40:0/3776832057 learned_addr learned my addr 172.21.15.40:0/3776832057 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:58.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.739+0000 7fc8d37fe640 1 --2- 172.21.15.40:0/3776832057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d406d6f0 0x7fc8d414b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:58.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.739+0000 7fc8d3fff640 1 --2- 172.21.15.40:0/3776832057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8d40ca0d0 0x7fc8d4153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:58.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.739+0000 7fc8cbfff640 1 -- 172.21.15.40:0/3776832057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8d40ca0d0 msgr2=0x7fc8d4153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:58.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.739+0000 7fc8cbfff640 1 --2- 172.21.15.40:0/3776832057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8d40ca0d0 0x7fc8d4153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:58.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.739+0000 7fc8cbfff640 1 -- 172.21.15.40:0/3776832057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d406d6f0 msgr2=0x7fc8d414b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:58.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.739+0000 7fc8cbfff640 1 --2- 172.21.15.40:0/3776832057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d406d6f0 0x7fc8d414b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:58.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.739+0000 7fc8cbfff640 1 -- 172.21.15.40:0/3776832057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c4009560 con 0x7fc8d40b0630 2023-10-18T09:01:58.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.739+0000 7fc8d37fe640 1 --2- 172.21.15.40:0/3776832057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d406d6f0 0x7fc8d414b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:01:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.740+0000 7fc8cbfff640 1 --2- 172.21.15.40:0/3776832057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d40b0630 0x7fc8d4153440 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00c410 tx=0x7fc8bc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.740+0000 7fc8d0ff9640 1 -- 172.21.15.40:0/3776832057 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc0035f0 con 0x7fc8d40b0630 2023-10-18T09:01:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.740+0000 7fc8d0ff9640 1 -- 172.21.15.40:0/3776832057 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc8bc003750 con 0x7fc8d40b0630 2023-10-18T09:01:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.740+0000 7fc8d0ff9640 1 -- 172.21.15.40:0/3776832057 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc003ba0 con 0x7fc8d40b0630 2023-10-18T09:01:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.740+0000 7fc8d8a1a640 1 -- 172.21.15.40:0/3776832057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d40aaee0 con 0x7fc8d40b0630 2023-10-18T09:01:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.740+0000 7fc8d8a1a640 1 -- 172.21.15.40:0/3776832057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d40ab6e0 con 0x7fc8d40b0630 2023-10-18T09:01:58.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.741+0000 7fc8d8a1a640 1 -- 172.21.15.40:0/3776832057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d40ab400 con 0x7fc8d40b0630 2023-10-18T09:01:58.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.742+0000 7fc8d0ff9640 1 -- 172.21.15.40:0/3776832057 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc8bc003d00 con 0x7fc8d40b0630 2023-10-18T09:01:58.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.744+0000 7fc8d0ff9640 1 --2- 172.21.15.40:0/3776832057 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8b406ae40 0x7fc8b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:58.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.744+0000 7fc8d0ff9640 1 -- 172.21.15.40:0/3776832057 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8bc088a90 con 0x7fc8d40b0630 2023-10-18T09:01:58.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.744+0000 7fc8d0ff9640 1 -- 172.21.15.40:0/3776832057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=111}) v3 -- 0x7fc8b4072970 con 0x7fc8d40b0630 2023-10-18T09:01:58.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.745+0000 7fc8d37fe640 1 --2- 172.21.15.40:0/3776832057 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8b406ae40 0x7fc8b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:58.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.746+0000 7fc8d37fe640 1 --2- 172.21.15.40:0/3776832057 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8b406ae40 0x7fc8b406d300 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4035000 tx=0x7fc8c4004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:58.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.749+0000 7fc8d0ff9640 1 -- 172.21.15.40:0/3776832057 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc8bc017060 con 0x7fc8d40b0630 2023-10-18T09:01:58.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.944+0000 7fc8d8a1a640 1 -- 172.21.15.40:0/3776832057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc8d4066e00 con 0x7fc8d40b0630 2023-10-18T09:01:58.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.947+0000 7fc8d0ff9640 1 -- 172.21.15.40:0/3776832057 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v110) v1 ==== 81+0+34162 (secure 0 0 0) 0x7fc8d40ab400 con 0x7fc8d40b0630 2023-10-18T09:01:58.950 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:01:58.951 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:01:58.951 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 110, 2023-10-18T09:01:58.951 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:01:58.951 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:01:58.951 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:01:58.134654+0000", 2023-10-18T09:01:58.952 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:01:58.952 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:01:58.952 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:01:58.952 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:01:58.952 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:01:58.953 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:01:58.953 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:01:58.953 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:01:58.953 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:01:58.953 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:01:58.954 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:01:58.954 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:01:58.954 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:01:58.954 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:01:58.954 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:01:58.955 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 13, 2023-10-18T09:01:58.955 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:01:58.955 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:01:58.955 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:01:58.955 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:01:58.957 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:01:58.957 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:01:58.957 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:58.957 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:01:58.957 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:01:58.958 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:01:58.958 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T09:01:58.958 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T09:01:58.958 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:01:58.958 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:01:58.959 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:01:58.959 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:01:58.959 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:01:58.959 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:01:58.959 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:01:58.960 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:01:58.960 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:01:58.960 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:01:58.960 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:01:58.960 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:01:58.960 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:01:58.961 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:01:58.961 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:01:58.961 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:01:58.961 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:01:58.961 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:01:58.962 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:01:58.962 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:01:58.962 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:01:58.962 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:01:58.962 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:58.962 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "108", 2023-10-18T09:01:58.963 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:01:58.963 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:01:58.963 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:01:58.963 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:01:58.963 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:01:58.964 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:01:58.964 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:01:58.964 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:01:58.964 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:01:58.964 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:01:58.965 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:01:58.965 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:01:58.965 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:01:58.965 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:01:58.965 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:01:58.965 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:01:58.966 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:01:58.966 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:01:58.966 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:01:58.966 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:01:58.966 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:01:58.967 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:01:58.967 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:01:58.967 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:01:58.967 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:01:58.967 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:01:58.968 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:58.968 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:01:58.968 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:01:58.968 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:01:58.968 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:01:58.968 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:01:58.969 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:01:58.969 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:01:58.969 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:01:58.969 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:01:58.969 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:01:58.969 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:01:58.970 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:01:58.970 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:01:58.970 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:01:58.970 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:58.970 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:01:58.971 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:01:58.971 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:58.971 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:01:58.971 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:01:58.971 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:01:58.972 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:01:58.972 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:01:58.972 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:01:58.972 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:01:58.972 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:01:58.972 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:01:58.973 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:58.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:58.973 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:58.973 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 12, 2023-10-18T09:01:58.973 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:01:58.973 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:01:35.453236+0000", 2023-10-18T09:01:58.974 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T09:01:58.974 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T09:01:58.974 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:01:58.974 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:01:58.974 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:01:58.975 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:01:58.975 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:01:58.975 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:01:58.975 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:01:58.975 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:01:58.976 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:01:58.976 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:01:58.976 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:01:58.976 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:01:58.976 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:01:58.977 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:01:58.977 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:01:58.977 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:01:58.977 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:01:58.977 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:01:58.977 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:01:58.978 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:01:58.978 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:01:58.978 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:01:58.978 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:58.978 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "108", 2023-10-18T09:01:58.979 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:01:58.979 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:01:58.979 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:01:58.979 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:01:58.979 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:01:58.980 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:01:58.980 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:01:58.980 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:01:58.980 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:01:58.980 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:01:58.981 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:01:58.981 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:01:58.981 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:01:58.981 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:01:58.981 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:01:58.981 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:01:58.982 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:01:58.982 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:01:58.982 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:01:58.982 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:01:58.982 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:01:58.983 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:01:58.983 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:01:58.983 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:01:58.983 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:01:58.983 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:01:58.984 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:58.984 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:01:58.984 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:01:58.984 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:01:58.984 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:01:58.984 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:01:58.985 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:01:58.985 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:01:58.985 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:01:58.985 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:01:58.985 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:01:58.986 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:01:58.986 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:01:58.986 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:01:58.986 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:01:58.986 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:01:58.987 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:58.987 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:01:58.987 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:01:58.987 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:01:58.987 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:58.988 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:58.988 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:01:58.988 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 2, 2023-10-18T09:01:58.988 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 2, 2023-10-18T09:01:58.988 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:01:58.988 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:01:58.989 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:01:58.989 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:01:58.989 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:01:58.989 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:01:58.990 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:58.990 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:58.990 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:58.990 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 13, 2023-10-18T09:01:58.990 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:01:58.990 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:01:36.012460+0000", 2023-10-18T09:01:58.991 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T09:01:58.991 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T09:01:58.991 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:01:58.991 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:01:58.991 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:01:58.992 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:01:58.992 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:01:58.992 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:01:58.992 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:01:58.992 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:01:58.992 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:01:58.993 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:01:58.993 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:01:58.993 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:01:58.993 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:01:58.993 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:01:58.994 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:01:58.994 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:01:58.994 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:01:58.994 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:01:58.994 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:01:58.995 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:01:58.995 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:01:58.995 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:01:58.995 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:58.995 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "108", 2023-10-18T09:01:58.996 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:01:58.996 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:01:58.996 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:01:58.996 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:01:58.997 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:01:58.997 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:01:58.997 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:01:58.997 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:01:58.997 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:01:58.997 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:01:58.998 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:01:58.998 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:01:58.998 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:01:58.998 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:01:58.998 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:01:58.999 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:01:58.999 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:01:58.999 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:01:58.999 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:01:58.999 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:01:59.000 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:01:59.000 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:01:59.000 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:01:59.000 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:01:59.000 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:01:59.001 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:01:59.001 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.001 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:01:59.001 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:01:59.001 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:01:59.002 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:01:59.002 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:01:59.002 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:01:59.002 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:01:59.002 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:01:59.003 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:01:59.003 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:01:59.003 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:01:59.003 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:01:59.003 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:01:59.003 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:01:59.004 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.004 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:01:59.004 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:01:59.004 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:01:59.004 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.004 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.005 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:01:59.005 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.3799999952316284, 2023-10-18T09:01:59.005 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.3799999952316284, 2023-10-18T09:01:59.005 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:01:59.005 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-18T09:01:59.006 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-18T09:01:59.006 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:01:59.006 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:01:59.006 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:01:59.006 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.007 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.007 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:01:59.007 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:01:59.007 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.007 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:01:59.007 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:01:59.008 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:01:59.008 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:01:59.008 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:01:59.008 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:01:59.008 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:01:59.008 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:01:59.009 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:01:59.009 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:01:59.009 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:01:59.009 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:01:59.009 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:01:59.010 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.010 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.010 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.010 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:01:59.010 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:01:59.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.011 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.011 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.011 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:01:59.012 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:01:59.012 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.012 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.012 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.012 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:01:59.013 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.013 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.013 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.013 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:01:59.013 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:01:59.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.014 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.014 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.014 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:01:59.014 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:01:59.015 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.015 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.015 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:01:59.015 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.016 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.016 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.016 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:01:59.016 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:01:59.016 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.016 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.017 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.017 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:01:59.017 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:01:59.017 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.017 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.018 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:01:59.018 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.018 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.018 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.019 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:01:59.019 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:01:59.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.019 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.019 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.019 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:01:59.020 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:01:59.020 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.020 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.020 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:01:59.021 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:01:59.021 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:01:59.021 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:01:59.021 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:01:59.021 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:01:59.022 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T09:01:59.022 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:01:59.022 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.022 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.022 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.023 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:01:59.023 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:01:59.023 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:01:59.023 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:01:59.023 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:01:59.024 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:01:59.024 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:01:59.024 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:01:59.024 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:01:59.024 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:01:59.024 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:01:59.025 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:01:59.025 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:01:59.025 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.025 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.025 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.026 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:01:59.026 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:01:59.026 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.026 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.026 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.026 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:01:59.027 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:01:59.027 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.027 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.027 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:01:59.028 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.028 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.028 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.028 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:01:59.028 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:01:59.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.029 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.029 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.029 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:01:59.029 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:01:59.029 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.030 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.030 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:01:59.030 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.030 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.031 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.031 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:01:59.031 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:01:59.031 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.031 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.031 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.032 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:01:59.032 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:01:59.032 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.032 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.032 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.033 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:01:59.033 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.033 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.033 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.033 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:01:59.033 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:01:59.034 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.034 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.034 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.034 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:01:59.035 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:01:59.035 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.035 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.035 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.035 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:01:59.035 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:01:59.036 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:01:59.036 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:01:59.036 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:01:59.036 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:01:59.036 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T09:01:59.037 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:01:59.037 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.037 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.037 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.037 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:01:59.037 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:01:59.038 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:01:59.038 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:01:59.038 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:01:59.038 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:01:59.038 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:01:59.039 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:01:59.039 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:01:59.039 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:01:59.039 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:01:59.039 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:01:59.040 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:01:59.040 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.040 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.040 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.040 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:01:59.041 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:01:59.041 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.041 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.041 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.041 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:01:59.041 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:01:59.042 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.042 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.042 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.042 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:01:59.042 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.043 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.043 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.043 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:01:59.043 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:01:59.043 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.044 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.044 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.044 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:01:59.044 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:01:59.044 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.045 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.045 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.045 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:01:59.045 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.045 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.045 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.046 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:01:59.046 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:01:59.046 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.046 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.046 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.047 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:01:59.047 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:01:59.047 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.047 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.047 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.048 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:01:59.048 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.048 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.048 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.048 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:01:59.048 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:01:59.049 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.049 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.049 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.049 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:01:59.049 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:01:59.050 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.050 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.050 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:01:59.050 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:01:59.051 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:01:59.051 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:01:59.051 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:01:59.051 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:01:59.051 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:01:59.051 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.052 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.052 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:01:59.052 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:01:59.052 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:01:59.053 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:01:59.053 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:01:59.053 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:01:59.053 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:01:59.053 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:01:59.054 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:01:59.054 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:01:59.054 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:01:59.054 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:01:59.054 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:01:59.054 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:01:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:01:59.055 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:01:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:01:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.056 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:01:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.057 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:01:59.058 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:01:59.058 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.058 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.058 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.058 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:01:59.059 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:01:59.059 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.059 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.059 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.059 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:01:59.060 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.060 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.060 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.060 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:01:59.060 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:01:59.060 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.061 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.061 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.061 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:01:59.061 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:01:59.061 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.062 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.062 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.062 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:01:59.062 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.062 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.063 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.063 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:01:59.063 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:01:59.063 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.063 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.063 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.064 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:01:59.064 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:01:59.064 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.064 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.065 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:01:59.065 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:01:59.065 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:01:59.065 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:01:59.065 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:01:59.066 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:01:59.066 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:01:59.066 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.066 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.066 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.067 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:01:59.067 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:01:59.067 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:01:59.067 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:01:59.067 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:01:59.067 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:01:59.068 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:01:59.068 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:01:59.068 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:01:59.068 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:01:59.068 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:01:59.069 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:01:59.069 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:01:59.069 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.069 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.069 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.070 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:01:59.070 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:01:59.070 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.070 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.070 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.070 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:01:59.071 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:01:59.071 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.071 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.071 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:01:59.072 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.072 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.072 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.072 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:01:59.072 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:01:59.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.073 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.073 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.073 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:01:59.073 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:01:59.074 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.074 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.074 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.074 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:01:59.074 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.075 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.075 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.075 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:01:59.075 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:01:59.075 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.076 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.076 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.076 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:01:59.076 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:01:59.076 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.076 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.077 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.077 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:01:59.077 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.077 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.077 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.078 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:01:59.078 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:01:59.078 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.078 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.079 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.079 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:01:59.079 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:01:59.079 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.079 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.080 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:01:59.080 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:01:59.080 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:01:59.080 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:01:59.080 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:01:59.081 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:01:59.081 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T09:01:59.081 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:01:59.081 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.081 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.082 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.082 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:01:59.082 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:01:59.082 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:01:59.082 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:01:59.083 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:01:59.083 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:01:59.083 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:01:59.083 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:01:59.083 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:01:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:01:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:01:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:01:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:01:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.084 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.085 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.085 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:01:59.085 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:01:59.085 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.085 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.086 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.086 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:01:59.086 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:01:59.086 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.086 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.086 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.087 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:01:59.087 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.087 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.087 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.087 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:01:59.088 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:01:59.088 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.088 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.088 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.088 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:01:59.088 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:01:59.089 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.089 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.089 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.089 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:01:59.089 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.090 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.090 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.090 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:01:59.090 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:01:59.090 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.091 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.091 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.091 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:01:59.091 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:01:59.091 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.092 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.092 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.092 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:01:59.092 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.092 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.092 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.093 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:01:59.093 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:01:59.093 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.093 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.093 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.094 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:01:59.094 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:01:59.094 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.094 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.095 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:01:59.095 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:01:59.095 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:01:59.095 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:01:59.095 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:01:59.096 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:01:59.096 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:01:59.096 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.097 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:01:59.097 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:01:59.097 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:01:59.097 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:01:59.097 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:01:59.097 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:01:59.098 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:01:59.098 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:01:59.098 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:01:59.098 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:01:59.098 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:01:59.099 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:01:59.099 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:01:59.099 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.099 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.099 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.100 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:01:59.100 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:01:59.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.100 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.100 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.100 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:01:59.101 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:01:59.101 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.101 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.101 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.101 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:01:59.101 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.102 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.102 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.102 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:01:59.102 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:01:59.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.102 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.103 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.103 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:01:59.103 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:01:59.103 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.103 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.104 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.104 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:01:59.104 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.104 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.104 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.104 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:01:59.105 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:01:59.105 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.105 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.105 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:01:59.105 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:01:59.106 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.106 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.106 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:01:59.106 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.107 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.107 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:01:59.107 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:01:59.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.108 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.108 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:01:59.108 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:01:59.108 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.108 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.109 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:01:59.109 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:01:59.109 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:01:59.109 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:01:59.110 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:01:59.110 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:01:59.110 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T09:01:59.110 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:01:59.110 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.111 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.111 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:01:59.111 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:01:59.111 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:01:59.112 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:01:59.112 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:01:59.112 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:01:59.112 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:01:59.112 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:01:59.113 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:01:59.113 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:01:59.113 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:01:59.113 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:01:59.113 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:01:59.113 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.114 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.114 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:01:59.114 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:01:59.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.115 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.115 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:01:59.115 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:01:59.115 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.116 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.116 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:01:59.116 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.117 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.117 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:01:59.117 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:01:59.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.117 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.118 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:01:59.118 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:01:59.118 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.118 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.119 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:01:59.119 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.119 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.119 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:01:59.119 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:01:59.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.120 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.120 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:01:59.120 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:01:59.121 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.121 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.121 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:01:59.121 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:01:59.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.122 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:01:59.122 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:01:59.122 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:01:59.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.123 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:01:59.123 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:01:59.123 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:01:59.123 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.123 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.124 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:01:59.124 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:01:59.124 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:01:59.124 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:01:59.124 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:01:59.125 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull", 2023-10-18T09:01:59.125 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:01:59.125 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:01:59.125 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:01:59.125 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.126 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:01:59.126 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:01:59.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.126 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:01:59.126 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:01:59.127 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:01:59.127 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:01:59.127 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:01:59.127 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:01:59.127 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:01:59.128 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:01:59.128 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.128 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:01:59.128 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:01:59.129 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:01:59.129 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:01:59.129 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:01:59.129 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:01:59.129 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:01:59.129 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:01:59.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.130 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:01:59.130 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:01:59.131 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:01:59.131 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:01:59.131 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:01:59.131 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:01:59.131 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:01:59.131 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:01:59.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.132 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:01:59.132 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:01:59.132 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:01:59.132 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:01:59.133 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:01:59.133 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:01:59.133 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:01:59.133 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:01:59.134 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.134 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:01:59.134 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:01:59.134 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:01:59.135 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:01:59.135 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:01:59.135 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:01:59.135 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:01:59.135 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:01:59.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.136 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:01:59.136 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:01:59.136 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:01:59.137 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:01:59.137 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:01:59.137 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:01:59.137 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:01:59.138 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:01:59.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.138 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:01:59.138 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:01:59.139 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:01:59.139 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:01:59.139 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:01:59.139 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:01:59.140 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:01:59.140 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:01:59.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:01:59.140 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:01:59.141 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:01:59.141 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:01:59.141 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:01:59.141 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:01:59.141 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:01:59.142 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:01:59.142 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:01:59.142 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.142 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:01:59.142 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:01:59.143 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:01:59.143 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:01:59.143 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:01:59.143 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:01:59.143 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:01:59.144 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:01:59.144 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:01:59.144 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:01:59.144 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:01:59.144 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:01:59.145 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:01:59.145 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:01:59.145 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:01:59.145 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:01:59.146 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:01:59.146 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:01:59.146 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:01:59.146 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:01:59.147 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:01:59.147 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:01:59.147 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:01:59.147 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:01:59.147 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:01:59.147 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:01:59.148 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:01:59.148 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:01:59.148 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:01:59.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.950+0000 7fc8d8a1a640 1 -- 172.21.15.40:0/3776832057 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8b406ae40 msgr2=0x7fc8b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:59.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.950+0000 7fc8d8a1a640 1 --2- 172.21.15.40:0/3776832057 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8b406ae40 0x7fc8b406d300 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4035000 tx=0x7fc8c4004730 comp rx=0 tx=0).stop 2023-10-18T09:01:59.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.950+0000 7fc8d8a1a640 1 -- 172.21.15.40:0/3776832057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d40b0630 msgr2=0x7fc8d4153440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:59.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.950+0000 7fc8d8a1a640 1 --2- 172.21.15.40:0/3776832057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d40b0630 0x7fc8d4153440 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00c410 tx=0x7fc8bc00c8d0 comp rx=0 tx=0).stop 2023-10-18T09:01:59.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.950+0000 7fc8d8a1a640 1 -- 172.21.15.40:0/3776832057 shutdown_connections 2023-10-18T09:01:59.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.950+0000 7fc8d8a1a640 1 --2- 172.21.15.40:0/3776832057 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8b406ae40 0x7fc8b406d300 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:59.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.950+0000 7fc8d8a1a640 1 --2- 172.21.15.40:0/3776832057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8d40ca0d0 0x7fc8d4153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:59.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.950+0000 7fc8d8a1a640 1 --2- 172.21.15.40:0/3776832057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8d40b0630 0x7fc8d4153440 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:59.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.950+0000 7fc8d8a1a640 1 --2- 172.21.15.40:0/3776832057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d406d6f0 0x7fc8d414b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:59.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.950+0000 7fc8d8a1a640 1 -- 172.21.15.40:0/3776832057 >> 172.21.15.40:0/3776832057 conn(0x7fc8d4069720 msgr2=0x7fc8d41545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:59.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.950+0000 7fc8d8a1a640 1 -- 172.21.15.40:0/3776832057 shutdown_connections 2023-10-18T09:01:59.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:58.950+0000 7fc8d8a1a640 1 -- 172.21.15.40:0/3776832057 wait complete. 2023-10-18T09:01:59.153 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-18T09:01:59.154 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:01:59.154 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:01:59.155 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:01:59.138+0000 7f6a11f15640 -1 log_channel(cluster) log [ERR] : Health check update: 5 full osd(s) (OSD_FULL) 2023-10-18T09:01:59.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:01:59.219 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2023-10-18T09:01:59.655 INFO:teuthology.orchestra.run.smithi040.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2023-10-18T09:01:59.656 INFO:teuthology.orchestra.run.smithi040.stderr:50+0 records in 2023-10-18T09:01:59.656 INFO:teuthology.orchestra.run.smithi040.stderr:50+0 records out 2023-10-18T09:01:59.656 INFO:teuthology.orchestra.run.smithi040.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.367162 s, 143 MB/s 2023-10-18T09:01:59.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:01:59.658 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:01:59.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.875+0000 7f5e9b067640 1 -- 172.21.15.40:0/3512449843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e940c9f00 msgr2=0x7f5e94157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:59.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.875+0000 7f5e9b067640 1 --2- 172.21.15.40:0/3512449843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e940c9f00 0x7f5e94157d40 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f5e940caf30 tx=0x7f5e8c02c230 comp rx=0 tx=0).stop 2023-10-18T09:01:59.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.876+0000 7f5e9b067640 1 -- 172.21.15.40:0/3512449843 shutdown_connections 2023-10-18T09:01:59.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.876+0000 7f5e9b067640 1 --2- 172.21.15.40:0/3512449843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e940c9f00 0x7f5e94157d40 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:59.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.876+0000 7f5e9b067640 1 --2- 172.21.15.40:0/3512449843 >> v2:172.21.15.144:3300/0 conn(0x7f5e940b15b0 0x7f5e940c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:59.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.876+0000 7f5e9b067640 1 -- 172.21.15.40:0/3512449843 >> 172.21.15.40:0/3512449843 conn(0x7f5e94069720 msgr2=0x7f5e94069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:01:59.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.876+0000 7f5e9b067640 1 -- 172.21.15.40:0/3512449843 shutdown_connections 2023-10-18T09:01:59.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.876+0000 7f5e9b067640 1 -- 172.21.15.40:0/3512449843 wait complete. 2023-10-18T09:01:59.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.878+0000 7f5e9b067640 1 Processor -- start 2023-10-18T09:01:59.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.878+0000 7f5e9b067640 1 -- start start 2023-10-18T09:01:59.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.880+0000 7f5e9b067640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e9406d6f0 0x7f5e94145870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:59.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.880+0000 7f5e9b067640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e940b15b0 0x7f5e94144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:59.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.880+0000 7f5e9b067640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e940c9f00 0x7f5e940ab220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:59.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.880+0000 7f5e9b067640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e94156710 con 0x7f5e940c9f00 2023-10-18T09:01:59.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.880+0000 7f5e9b067640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e940ace30 con 0x7f5e940b15b0 2023-10-18T09:01:59.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.880+0000 7f5e9b067640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e94156150 con 0x7f5e9406d6f0 2023-10-18T09:01:59.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.880+0000 7f5e9a866640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e940c9f00 0x7f5e940ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:59.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.880+0000 7f5e9a866640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e940c9f00 0x7f5e940ab220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34822/0 (socket says 172.21.15.40:34822) 2023-10-18T09:01:59.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.880+0000 7f5e9a866640 1 -- 172.21.15.40:0/4832557 learned_addr learned my addr 172.21.15.40:0/4832557 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:01:59.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.880+0000 7f5e99864640 1 --2- 172.21.15.40:0/4832557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e940b15b0 0x7f5e94144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:59.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.880+0000 7f5e9a065640 1 --2- 172.21.15.40:0/4832557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e9406d6f0 0x7f5e94145870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:59.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.880+0000 7f5e9a866640 1 -- 172.21.15.40:0/4832557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e9406d6f0 msgr2=0x7f5e94145870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:59.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.880+0000 7f5e9a866640 1 --2- 172.21.15.40:0/4832557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e9406d6f0 0x7f5e94145870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:59.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.881+0000 7f5e9a866640 1 -- 172.21.15.40:0/4832557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e940b15b0 msgr2=0x7f5e94144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:01:59.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.881+0000 7f5e9a866640 1 --2- 172.21.15.40:0/4832557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e940b15b0 0x7f5e94144ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:01:59.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.881+0000 7f5e9a866640 1 -- 172.21.15.40:0/4832557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c009900 con 0x7f5e940c9f00 2023-10-18T09:01:59.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.881+0000 7f5e9a866640 1 --2- 172.21.15.40:0/4832557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e940c9f00 0x7f5e940ab220 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c005fd0 tx=0x7f5e8c035ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:59.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.881+0000 7f5e8affd640 1 -- 172.21.15.40:0/4832557 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c034440 con 0x7f5e940c9f00 2023-10-18T09:01:59.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.881+0000 7f5e8affd640 1 -- 172.21.15.40:0/4832557 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5e8c004330 con 0x7f5e940c9f00 2023-10-18T09:01:59.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.881+0000 7f5e9b067640 1 -- 172.21.15.40:0/4832557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e94156430 con 0x7f5e940c9f00 2023-10-18T09:01:59.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.881+0000 7f5e8affd640 1 -- 172.21.15.40:0/4832557 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8c004740 con 0x7f5e940c9f00 2023-10-18T09:01:59.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.882+0000 7f5e9b067640 1 -- 172.21.15.40:0/4832557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e940aad00 con 0x7f5e940c9f00 2023-10-18T09:01:59.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.883+0000 7f5e9b067640 1 -- 172.21.15.40:0/4832557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e940aa880 con 0x7f5e940c9f00 2023-10-18T09:01:59.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.884+0000 7f5e8affd640 1 -- 172.21.15.40:0/4832557 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5e8c00a040 con 0x7f5e940c9f00 2023-10-18T09:01:59.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.885+0000 7f5e8affd640 1 --2- 172.21.15.40:0/4832557 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5e7806ae40 0x7f5e7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:01:59.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.885+0000 7f5e8affd640 1 -- 172.21.15.40:0/4832557 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e8c0ac8f0 con 0x7f5e940c9f00 2023-10-18T09:01:59.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.885+0000 7f5e8affd640 1 -- 172.21.15.40:0/4832557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=112}) v3 -- 0x7f5e78072970 con 0x7f5e940c9f00 2023-10-18T09:01:59.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.885+0000 7f5e9a065640 1 --2- 172.21.15.40:0/4832557 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5e7806ae40 0x7f5e7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:01:59.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.887+0000 7f5e9a065640 1 --2- 172.21.15.40:0/4832557 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5e7806ae40 0x7f5e7806d300 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5e90003d90 tx=0x7f5e90009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:01:59.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:01:59.890+0000 7f5e8affd640 1 -- 172.21.15.40:0/4832557 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e8c039060 con 0x7f5e940c9f00 2023-10-18T09:02:00.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.083+0000 7f5e9b067640 1 -- 172.21.15.40:0/4832557 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5e940aaf20 con 0x7f5e940c9f00 2023-10-18T09:02:00.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.087+0000 7f5e8affd640 1 -- 172.21.15.40:0/4832557 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v111) v1 ==== 81+0+34186 (secure 0 0 0) 0x7f5e8c044020 con 0x7f5e940c9f00 2023-10-18T09:02:00.089 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:00.090 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:02:00.090 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 111, 2023-10-18T09:02:00.090 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:02:00.090 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:02:00.091 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:01:59.138988+0000", 2023-10-18T09:02:00.091 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:02:00.091 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:02:00.091 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:02:00.091 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:02:00.091 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:02:00.092 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:02:00.092 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:02:00.092 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:02:00.092 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:02:00.092 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:00.093 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:02:00.093 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:02:00.093 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:02:00.093 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:02:00.093 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:02:00.094 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 13, 2023-10-18T09:02:00.094 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:02:00.094 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:02:00.094 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:02:00.094 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:02:00.095 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:02:00.095 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:02:00.095 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.095 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:02:00.095 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:02:00.096 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:02:00.096 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T09:02:00.096 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T09:02:00.096 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:02:00.096 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:02:00.097 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:02:00.097 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:02:00.097 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:02:00.097 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:02:00.097 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:02:00.097 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:02:00.098 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:02:00.098 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:02:00.098 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:02:00.098 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:02:00.098 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:02:00.099 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:02:00.099 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:02:00.099 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:02:00.099 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:02:00.099 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:02:00.100 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:02:00.100 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:02:00.100 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:02:00.100 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:02:00.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.101 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "108", 2023-10-18T09:02:00.101 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:02:00.101 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:02:00.101 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:02:00.101 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:02:00.101 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:02:00.102 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:02:00.102 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:02:00.102 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:02:00.102 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:02:00.102 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:02:00.103 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:02:00.103 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:02:00.103 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:02:00.103 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:02:00.103 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:02:00.104 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:02:00.104 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:02:00.104 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:02:00.104 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:02:00.104 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:02:00.104 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:02:00.105 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:02:00.105 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:02:00.105 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:02:00.105 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:02:00.105 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:02:00.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.106 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:02:00.106 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:02:00.106 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:02:00.106 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:02:00.107 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:02:00.107 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:02:00.107 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:02:00.107 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:02:00.107 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:02:00.107 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:02:00.108 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:02:00.108 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:02:00.108 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:02:00.108 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:02:00.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.109 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:02:00.109 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:02:00.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.109 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:02:00.109 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:02:00.110 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:02:00.110 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:02:00.110 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:02:00.110 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:02:00.110 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:02:00.111 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:02:00.111 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:02:00.111 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.111 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.111 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 12, 2023-10-18T09:02:00.112 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:02:00.112 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:01:35.453236+0000", 2023-10-18T09:02:00.112 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T09:02:00.112 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T09:02:00.112 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:02:00.113 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:02:00.113 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:02:00.113 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:02:00.113 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:02:00.113 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:02:00.113 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:02:00.114 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:02:00.114 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:02:00.114 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:02:00.114 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:02:00.114 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:02:00.115 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:02:00.115 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:02:00.115 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:02:00.115 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:02:00.115 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:02:00.115 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:02:00.116 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:02:00.116 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:02:00.116 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:02:00.116 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:02:00.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.117 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "108", 2023-10-18T09:02:00.117 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:02:00.117 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:02:00.117 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:02:00.117 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:02:00.117 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:02:00.118 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:02:00.118 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:02:00.118 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:02:00.118 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:02:00.118 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:02:00.119 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:02:00.119 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:02:00.119 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:02:00.119 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:02:00.119 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:02:00.120 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:02:00.120 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:02:00.120 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:02:00.120 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:02:00.120 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:02:00.120 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:02:00.121 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:02:00.121 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:02:00.121 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:02:00.121 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:02:00.121 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:02:00.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.122 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:02:00.122 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:02:00.122 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:02:00.122 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:02:00.123 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:02:00.123 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:02:00.123 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:02:00.123 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:02:00.123 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:02:00.124 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:02:00.124 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:02:00.124 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:02:00.124 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:02:00.124 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:02:00.124 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:02:00.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.125 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:02:00.125 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:02:00.125 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:02:00.125 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.126 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:02:00.126 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 2, 2023-10-18T09:02:00.126 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 2, 2023-10-18T09:02:00.126 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:02:00.127 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:02:00.127 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:02:00.127 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:02:00.127 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:02:00.127 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:02:00.127 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.128 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.128 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 13, 2023-10-18T09:02:00.128 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:02:00.128 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:01:36.012460+0000", 2023-10-18T09:02:00.129 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T09:02:00.129 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T09:02:00.129 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:02:00.129 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:02:00.129 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:02:00.129 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:02:00.130 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:02:00.130 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:02:00.130 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:02:00.130 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:02:00.130 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:02:00.131 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:02:00.131 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:02:00.131 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:02:00.131 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:02:00.131 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:02:00.132 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:02:00.132 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:02:00.132 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:02:00.132 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:02:00.132 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:02:00.132 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:02:00.133 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:02:00.133 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:02:00.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.133 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "108", 2023-10-18T09:02:00.133 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:02:00.134 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:02:00.134 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:02:00.134 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:02:00.134 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:02:00.134 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:02:00.135 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:02:00.135 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:02:00.135 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:02:00.135 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:02:00.135 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:02:00.136 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:02:00.136 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:02:00.136 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:02:00.136 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:02:00.136 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:02:00.136 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:02:00.137 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:02:00.137 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:02:00.137 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:02:00.137 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:02:00.137 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:02:00.138 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:02:00.138 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:02:00.138 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:02:00.138 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:02:00.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.138 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:02:00.139 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:02:00.139 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:02:00.139 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:02:00.139 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:02:00.139 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:02:00.140 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:02:00.140 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:02:00.140 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:02:00.140 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:02:00.140 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:02:00.140 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:02:00.141 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:02:00.141 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:02:00.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.141 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:02:00.141 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:02:00.142 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:02:00.142 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.142 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:02:00.142 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.3799999952316284, 2023-10-18T09:02:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.3799999952316284, 2023-10-18T09:02:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:02:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-18T09:02:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-18T09:02:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:02:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:02:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:02:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:02:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:02:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:02:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:02:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:02:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:02:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:02:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:02:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.155 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.155 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:00.155 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.155 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.156 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:02:00.156 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:00.156 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.156 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.156 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.157 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:02:00.157 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:00.157 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.157 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.158 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:02:00.158 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:02:00.158 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:02:00.158 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:02:00.158 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:00.159 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:00.159 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T09:02:00.159 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:00.159 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.159 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.160 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:02:00.160 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:00.160 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:00.160 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:00.160 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:00.161 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:00.161 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:00.161 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:00.161 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:00.161 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:00.161 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:00.162 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:00.162 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:00.162 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.162 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.162 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.163 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:02:00.163 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:00.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.163 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.164 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:02:00.164 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:00.164 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.164 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.164 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:00.165 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.165 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.165 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.165 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:02:00.165 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:00.166 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.166 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.166 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.166 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:02:00.166 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:00.167 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.167 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.167 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:00.167 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.168 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.168 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:02:00.168 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:00.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.168 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.169 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.169 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:02:00.169 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:00.169 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.169 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.170 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:00.170 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.170 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.170 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:02:00.171 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:00.171 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.171 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.171 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.171 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:02:00.172 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:00.172 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.172 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.172 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:02:00.172 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:02:00.173 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:02:00.173 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:02:00.173 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:00.173 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:00.173 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T09:02:00.174 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:00.174 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.174 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.174 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:02:00.175 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:00.175 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:00.175 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:00.175 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:00.175 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:00.175 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:00.176 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:00.176 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:00.176 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:00.176 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:00.176 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:00.176 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:00.177 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.177 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.177 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.177 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:02:00.177 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:00.178 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.178 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.178 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.178 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:02:00.178 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:00.178 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.179 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.179 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:00.179 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.179 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.180 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.180 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:02:00.180 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:00.180 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.180 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.181 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.181 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:02:00.181 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:00.181 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.181 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.182 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:00.182 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.182 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.182 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.183 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:02:00.183 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:00.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.183 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.183 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.183 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:02:00.184 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:00.184 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.184 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.184 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:00.185 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.185 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.185 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:02:00.185 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:00.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.186 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.186 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:02:00.186 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:00.187 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.187 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.187 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:02:00.187 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:02:00.188 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:02:00.188 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:02:00.188 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:00.188 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:00.188 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T09:02:00.188 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:00.189 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.189 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:02:00.189 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:00.190 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:00.190 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:00.190 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:00.190 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:00.190 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:00.191 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:00.191 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:00.191 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:00.191 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:00.191 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:00.192 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:00.192 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.192 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.192 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:02:00.192 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:00.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.193 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.193 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:02:00.193 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:00.194 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.194 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.194 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:00.194 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.195 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.195 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:02:00.195 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:00.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.195 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.196 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.196 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:02:00.196 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:00.196 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.196 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.197 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.197 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:00.197 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.197 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.197 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.198 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:02:00.198 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:00.198 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.198 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.198 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.198 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:02:00.199 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:00.199 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.199 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.199 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.199 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:00.200 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.200 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.200 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.200 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:02:00.200 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:00.201 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.201 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.201 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.201 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:02:00.201 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:00.201 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.202 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.202 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.202 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:02:00.202 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:02:00.202 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:02:00.203 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:02:00.203 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:00.203 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:00.203 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:00.203 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.204 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.204 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.204 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:02:00.204 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:00.204 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:00.204 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:00.205 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:00.205 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:00.205 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:00.205 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:00.205 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:00.206 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:00.206 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:00.206 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:00.206 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:00.206 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.207 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.207 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.207 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:02:00.207 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:00.207 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.207 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.208 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.208 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:02:00.208 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:00.208 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.208 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.209 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.209 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:00.209 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.209 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.209 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.210 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:02:00.210 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:00.210 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.210 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.210 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.211 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:02:00.211 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:00.211 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.211 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.212 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.212 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:00.212 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.212 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.212 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.212 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:02:00.213 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:00.213 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.213 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.213 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.213 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:02:00.214 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:00.214 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.214 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.214 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.214 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:00.215 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.215 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.215 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.215 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:02:00.215 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:00.216 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.216 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.216 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.216 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:02:00.216 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:00.216 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.217 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.217 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.217 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:02:00.217 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:02:00.217 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:02:00.218 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:02:00.218 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:00.218 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:00.218 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T09:02:00.218 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:00.219 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.219 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.219 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.219 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:02:00.219 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:00.220 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:00.220 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:00.220 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:00.220 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:00.220 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:00.220 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:00.221 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:00.221 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:00.221 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:00.221 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:00.221 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:00.222 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.222 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.222 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.222 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:02:00.222 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:00.223 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.223 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.223 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.223 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:02:00.223 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:00.224 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.224 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.224 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.224 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:00.224 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.224 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.225 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.225 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:02:00.225 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:00.225 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.226 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.226 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.226 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:02:00.226 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:00.226 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.227 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.227 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.227 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:00.227 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.227 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.227 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.228 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:02:00.228 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:00.228 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.228 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.228 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.229 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:02:00.229 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:00.229 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.229 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.229 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.229 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:00.230 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.230 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.230 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.230 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:02:00.230 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:00.231 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.231 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.231 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.231 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:02:00.231 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:00.232 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.232 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.232 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.232 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:02:00.232 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:02:00.232 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:02:00.233 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:02:00.233 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:00.233 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:00.233 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:00.233 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.234 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.234 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.234 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:02:00.234 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:00.234 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:00.235 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:00.235 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:00.235 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:00.235 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:00.235 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:00.235 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:00.236 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:00.236 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:00.236 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:00.236 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:00.236 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.237 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.237 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.237 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:02:00.237 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:00.237 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.237 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.238 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.238 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:02:00.238 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:00.238 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.238 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.239 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.239 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:00.239 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.239 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.239 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.239 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:02:00.240 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:00.240 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.240 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.240 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.240 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:02:00.241 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:00.241 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.241 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.241 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.241 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:00.242 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.242 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.242 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.242 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:02:00.242 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:00.242 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.243 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.243 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.243 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:02:00.243 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:00.243 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.244 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.244 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.244 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:00.244 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.244 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.245 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.245 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:02:00.245 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:00.245 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.245 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.245 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.246 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:02:00.246 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:00.246 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.246 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.246 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.247 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:02:00.247 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:02:00.247 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:02:00.247 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:02:00.247 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:00.248 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:00.248 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T09:02:00.248 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:00.248 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.248 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.249 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.249 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:02:00.249 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:00.249 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:00.249 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:00.250 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:00.250 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:00.250 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:00.250 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:00.250 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:00.250 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:00.251 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:00.251 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:00.251 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:00.251 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.251 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.252 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.252 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:02:00.252 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:00.252 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.252 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.253 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.253 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:02:00.253 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:00.253 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.253 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.253 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.254 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:00.254 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.254 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.254 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.254 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:02:00.255 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:00.255 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.255 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.255 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.255 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:02:00.256 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:00.256 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.256 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.256 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.256 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:00.256 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.257 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.257 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.257 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:02:00.257 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:00.257 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.258 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.258 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.258 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:02:00.258 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:00.258 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.258 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.259 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.259 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:00.259 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:00.259 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.259 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:00.260 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:02:00.260 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:00.260 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.260 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.260 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:00.260 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:02:00.261 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:00.261 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.261 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.261 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.261 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:02:00.262 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:02:00.262 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:02:00.262 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:02:00.262 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:00.262 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull", 2023-10-18T09:02:00.263 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:00.263 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:00.263 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:00.263 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.263 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:00.263 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:02:00.264 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.264 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:02:00.264 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:00.264 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:00.264 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:00.265 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:00.265 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:00.265 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:02:00.265 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:00.265 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.266 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.266 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:02:00.266 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:00.266 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:00.266 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:00.267 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:00.267 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:00.267 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:02:00.267 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:00.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.268 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.268 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:02:00.268 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:00.268 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:00.268 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:00.268 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:00.269 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:00.269 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:02:00.269 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:00.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.269 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.270 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:02:00.270 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:00.270 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:00.270 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:00.270 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:00.271 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:00.271 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:02:00.271 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:00.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.271 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.272 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:02:00.272 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:00.272 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:00.272 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:00.272 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:00.273 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:00.273 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:02:00.273 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:00.273 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.273 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.273 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:02:00.274 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:00.274 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:00.274 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:00.274 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:00.274 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:00.275 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:02:00.275 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:00.275 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.275 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.275 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:02:00.276 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:00.276 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:00.276 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:00.276 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:00.276 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:00.277 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:02:00.277 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:00.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.277 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:00.277 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:02:00.277 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:00.278 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:00.278 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:00.278 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:00.278 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:00.278 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:02:00.279 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:00.279 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.279 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:00.279 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:02:00.279 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:02:00.280 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:02:00.280 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:02:00.280 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:02:00.280 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:02:00.280 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:02:00.281 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:02:00.281 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:02:00.281 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:02:00.281 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:02:00.281 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:02:00.281 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:02:00.282 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:02:00.282 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:00.282 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:02:00.282 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:02:00.283 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:02:00.283 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:02:00.283 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:02:00.283 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:02:00.283 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:02:00.284 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:02:00.284 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:02:00.284 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:02:00.284 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:02:00.284 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:00.284 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:02:00.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.089+0000 7f5e9b067640 1 -- 172.21.15.40:0/4832557 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5e7806ae40 msgr2=0x7f5e7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:00.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.089+0000 7f5e9b067640 1 --2- 172.21.15.40:0/4832557 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5e7806ae40 0x7f5e7806d300 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5e90003d90 tx=0x7f5e90009040 comp rx=0 tx=0).stop 2023-10-18T09:02:00.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.089+0000 7f5e9b067640 1 -- 172.21.15.40:0/4832557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e940c9f00 msgr2=0x7f5e940ab220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.089+0000 7f5e9b067640 1 --2- 172.21.15.40:0/4832557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e940c9f00 0x7f5e940ab220 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c005fd0 tx=0x7f5e8c035ac0 comp rx=0 tx=0).stop 2023-10-18T09:02:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.089+0000 7f5e9b067640 1 -- 172.21.15.40:0/4832557 shutdown_connections 2023-10-18T09:02:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.089+0000 7f5e9b067640 1 --2- 172.21.15.40:0/4832557 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5e7806ae40 0x7f5e7806d300 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.089+0000 7f5e9b067640 1 --2- 172.21.15.40:0/4832557 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e940c9f00 0x7f5e940ab220 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.089+0000 7f5e9b067640 1 --2- 172.21.15.40:0/4832557 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e940b15b0 0x7f5e94144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:00.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.089+0000 7f5e9b067640 1 --2- 172.21.15.40:0/4832557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e9406d6f0 0x7f5e94145870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:00.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.089+0000 7f5e9b067640 1 -- 172.21.15.40:0/4832557 >> 172.21.15.40:0/4832557 conn(0x7f5e94069720 msgr2=0x7f5e94154a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:00.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.090+0000 7f5e9b067640 1 -- 172.21.15.40:0/4832557 shutdown_connections 2023-10-18T09:02:00.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.090+0000 7f5e9b067640 1 -- 172.21.15.40:0/4832557 wait complete. 2023-10-18T09:02:00.289 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:02:00.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.535+0000 7febab59e640 1 -- 172.21.15.40:0/2445033362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febac157870 msgr2=0x7febac0ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:00.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.535+0000 7febab59e640 1 --2- 172.21.15.40:0/2445033362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febac157870 0x7febac0ca350 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7febac15c4d0 tx=0x7feb9c02bd70 comp rx=0 tx=0).stop 2023-10-18T09:02:00.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.535+0000 7febab59e640 1 -- 172.21.15.40:0/2445033362 shutdown_connections 2023-10-18T09:02:00.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.535+0000 7febab59e640 1 --2- 172.21.15.40:0/2445033362 >> v2:172.21.15.144:3300/0 conn(0x7febac0caec0 0x7febac0cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:00.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.535+0000 7febab59e640 1 --2- 172.21.15.40:0/2445033362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febac157870 0x7febac0ca350 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:00.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.535+0000 7febab59e640 1 -- 172.21.15.40:0/2445033362 >> 172.21.15.40:0/2445033362 conn(0x7febac069720 msgr2=0x7febac069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:00.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.535+0000 7febab59e640 1 -- 172.21.15.40:0/2445033362 shutdown_connections 2023-10-18T09:02:00.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.535+0000 7febab59e640 1 -- 172.21.15.40:0/2445033362 wait complete. 2023-10-18T09:02:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.537+0000 7febab59e640 1 Processor -- start 2023-10-18T09:02:00.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.537+0000 7febab59e640 1 -- start start 2023-10-18T09:02:00.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.538+0000 7febab59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febac0ca890 0x7febac0ad350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:00.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.538+0000 7febab59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febac0caec0 0x7febac0a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.538+0000 7febab59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febac157870 0x7febac0701e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.538+0000 7febab59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0a8640 con 0x7febac157870 2023-10-18T09:02:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.539+0000 7febab59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0a8850 con 0x7febac0ca890 2023-10-18T09:02:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.539+0000 7febab59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac0a8b30 con 0x7febac0caec0 2023-10-18T09:02:00.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.539+0000 7febaad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febac157870 0x7febac0701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:00.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.539+0000 7febaad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febac157870 0x7febac0701e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34848/0 (socket says 172.21.15.40:34848) 2023-10-18T09:02:00.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.539+0000 7febaad9d640 1 -- 172.21.15.40:0/1536657896 learned_addr learned my addr 172.21.15.40:0/1536657896 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:00.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.539+0000 7febaa59c640 1 --2- 172.21.15.40:0/1536657896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febac0ca890 0x7febac0ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:00.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.539+0000 7feba3fff640 1 --2- 172.21.15.40:0/1536657896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febac0caec0 0x7febac0a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:00.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.539+0000 7febaad9d640 1 -- 172.21.15.40:0/1536657896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febac0caec0 msgr2=0x7febac0a98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:00.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.539+0000 7febaad9d640 1 --2- 172.21.15.40:0/1536657896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febac0caec0 0x7febac0a98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:00.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.539+0000 7febaad9d640 1 -- 172.21.15.40:0/1536657896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febac0ca890 msgr2=0x7febac0ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:00.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.539+0000 7febaad9d640 1 --2- 172.21.15.40:0/1536657896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febac0ca890 0x7febac0ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:00.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.539+0000 7febaad9d640 1 -- 172.21.15.40:0/1536657896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c009490 con 0x7febac157870 2023-10-18T09:02:00.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.540+0000 7febaad9d640 1 --2- 172.21.15.40:0/1536657896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febac157870 0x7febac0701e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7feba400ec60 tx=0x7feba400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:00.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.540+0000 7feba2ffd640 1 -- 172.21.15.40:0/1536657896 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4004d20 con 0x7febac157870 2023-10-18T09:02:00.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.540+0000 7feba2ffd640 1 -- 172.21.15.40:0/1536657896 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feba4004e80 con 0x7febac157870 2023-10-18T09:02:00.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.540+0000 7feba2ffd640 1 -- 172.21.15.40:0/1536657896 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba400b7d0 con 0x7febac157870 2023-10-18T09:02:00.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.540+0000 7febab59e640 1 -- 172.21.15.40:0/1536657896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac0a8e10 con 0x7febac157870 2023-10-18T09:02:00.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.540+0000 7febab59e640 1 -- 172.21.15.40:0/1536657896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac0a93d0 con 0x7febac157870 2023-10-18T09:02:00.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.542+0000 7febab59e640 1 -- 172.21.15.40:0/1536657896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febac0a95f0 con 0x7febac157870 2023-10-18T09:02:00.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.543+0000 7feba2ffd640 1 -- 172.21.15.40:0/1536657896 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feba400ed50 con 0x7febac157870 2023-10-18T09:02:00.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.544+0000 7feba2ffd640 1 --2- 172.21.15.40:0/1536657896 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb8c06ae40 0x7feb8c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:00.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.544+0000 7feba2ffd640 1 -- 172.21.15.40:0/1536657896 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feba408a100 con 0x7febac157870 2023-10-18T09:02:00.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.544+0000 7feba2ffd640 1 -- 172.21.15.40:0/1536657896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=112}) v3 -- 0x7feb8c072970 con 0x7febac157870 2023-10-18T09:02:00.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.545+0000 7febaa59c640 1 --2- 172.21.15.40:0/1536657896 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb8c06ae40 0x7feb8c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:00.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.546+0000 7febaa59c640 1 --2- 172.21.15.40:0/1536657896 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb8c06ae40 0x7feb8c06d300 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7feb9c036000 tx=0x7feb9c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:00.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.548+0000 7feba2ffd640 1 -- 172.21.15.40:0/1536657896 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feba4055080 con 0x7febac157870 2023-10-18T09:02:00.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.779+0000 7febab59e640 1 -- 172.21.15.40:0/1536657896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febac066e00 con 0x7febac157870 2023-10-18T09:02:00.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.781+0000 7feba2ffd640 1 -- 172.21.15.40:0/1536657896 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4918 (secure 0 0 0) 0x7feba40586a0 con 0x7febac157870 2023-10-18T09:02:00.784 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:00.784 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2565609773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2565609773},{"type":"v1","addr":"172.21.15.144:6836","nonce":2565609773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":5872,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3371548374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3371548374},{"type":"v1","addr":"172.21.15.144:6837","nonce":3371548374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3328648628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3328648628},{"type":"v1","addr":"172.21.15.40:6833","nonce":3328648628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:01:36.995669+0000","modified":"2023-10-18T09:01:40.068484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/891188043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":891188043},{"type":"v1","addr":"172.21.15.40:6835","nonce":891188043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-18T09:02:00.785 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 49 2023-10-18T09:02:00.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.783+0000 7febab59e640 1 -- 172.21.15.40:0/1536657896 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb8c06ae40 msgr2=0x7feb8c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:00.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.783+0000 7febab59e640 1 --2- 172.21.15.40:0/1536657896 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb8c06ae40 0x7feb8c06d300 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7feb9c036000 tx=0x7feb9c004730 comp rx=0 tx=0).stop 2023-10-18T09:02:00.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.783+0000 7febab59e640 1 -- 172.21.15.40:0/1536657896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febac157870 msgr2=0x7febac0701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:00.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.783+0000 7febab59e640 1 --2- 172.21.15.40:0/1536657896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febac157870 0x7febac0701e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7feba400ec60 tx=0x7feba400c7a0 comp rx=0 tx=0).stop 2023-10-18T09:02:00.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.784+0000 7febab59e640 1 -- 172.21.15.40:0/1536657896 shutdown_connections 2023-10-18T09:02:00.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.784+0000 7febab59e640 1 --2- 172.21.15.40:0/1536657896 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb8c06ae40 0x7feb8c06d300 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:00.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.784+0000 7febab59e640 1 --2- 172.21.15.40:0/1536657896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febac157870 0x7febac0701e0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:00.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.784+0000 7febab59e640 1 --2- 172.21.15.40:0/1536657896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febac0caec0 0x7febac0a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:00.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.784+0000 7febab59e640 1 --2- 172.21.15.40:0/1536657896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febac0ca890 0x7febac0ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:00.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.784+0000 7febab59e640 1 -- 172.21.15.40:0/1536657896 >> 172.21.15.40:0/1536657896 conn(0x7febac069720 msgr2=0x7febac0afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:00.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.784+0000 7febab59e640 1 -- 172.21.15.40:0/1536657896 shutdown_connections 2023-10-18T09:02:00.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:00.784+0000 7febab59e640 1 -- 172.21.15.40:0/1536657896 wait complete. 2023-10-18T09:02:00.806 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-18T09:02:01.042 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":6996,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":23.485814562000002,"mdsmap_epoch":49,"osdmap_epoch":111,"osdmap_epoch_barrier":111,"uptime":29.341713732999999} 2023-10-18T09:02:01.042 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 6996, "mdsmap_epoch": 49, "osdmap_epoch": 111, "osdmap_epoch_barrier": 111, "rank_uptime": 23.485814562, "state": "up:active", "uptime": 29.341713733, "want_state": "up:active", "whoami": 0 } 2023-10-18T09:02:01.043 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-18T09:02:01.043 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:02:01.044 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:02:01.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:01.097 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2023-10-18T09:02:01.155 INFO:teuthology.orchestra.run.smithi040.stderr:dd: failed to open 'small_file_1': Permission denied 2023-10-18T09:02:01.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:02:01.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:01.157 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-18T09:02:01.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:01.219 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-18T09:02:01.366 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:02:01.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.601+0000 7f172d7d0640 1 -- 172.21.15.40:0/1607603728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17280b3c20 msgr2=0x7f17280ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:01.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.601+0000 7f172d7d0640 1 --2- 172.21.15.40:0/1607603728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17280b3c20 0x7f17280ccc90 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1710005fd0 tx=0x7f171002bd90 comp rx=0 tx=0).stop 2023-10-18T09:02:01.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.601+0000 7f172d7d0640 1 -- 172.21.15.40:0/1607603728 shutdown_connections 2023-10-18T09:02:01.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.601+0000 7f172d7d0640 1 --2- 172.21.15.40:0/1607603728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1728071140 0x7f1728159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:01.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.601+0000 7f172d7d0640 1 --2- 172.21.15.40:0/1607603728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17280b3c20 0x7f17280ccc90 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:01.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.601+0000 7f172d7d0640 1 -- 172.21.15.40:0/1607603728 >> 172.21.15.40:0/1607603728 conn(0x7f17280aff00 msgr2=0x7f17280b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:01.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.601+0000 7f172d7d0640 1 -- 172.21.15.40:0/1607603728 shutdown_connections 2023-10-18T09:02:01.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.601+0000 7f172d7d0640 1 -- 172.21.15.40:0/1607603728 wait complete. 2023-10-18T09:02:01.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.603+0000 7f172d7d0640 1 Processor -- start 2023-10-18T09:02:01.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.604+0000 7f172d7d0640 1 -- start start 2023-10-18T09:02:01.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.605+0000 7f172d7d0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1728071140 0x7f17281f0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:01.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.605+0000 7f172d7d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17280b3c20 0x7f1728069990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:01.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.605+0000 7f172d7d0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1728142be0 0x7f172813ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:01.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.605+0000 7f172d7d0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17281f07a0 con 0x7f1728142be0 2023-10-18T09:02:01.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.605+0000 7f172d7d0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17280c9980 con 0x7f17280b3c20 2023-10-18T09:02:01.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.605+0000 7f172d7d0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17280a8000 con 0x7f1728071140 2023-10-18T09:02:01.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.606+0000 7f17277fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1728142be0 0x7f172813ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:01.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.606+0000 7f17277fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1728142be0 0x7f172813ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34854/0 (socket says 172.21.15.40:34854) 2023-10-18T09:02:01.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.606+0000 7f17277fe640 1 -- 172.21.15.40:0/3634384013 learned_addr learned my addr 172.21.15.40:0/3634384013 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:01.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.606+0000 7f1726ffd640 1 --2- 172.21.15.40:0/3634384013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1728071140 0x7f17281f0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:01.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.606+0000 7f17277fe640 1 -- 172.21.15.40:0/3634384013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1728071140 msgr2=0x7f17281f0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:01.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.606+0000 7f17277fe640 1 --2- 172.21.15.40:0/3634384013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1728071140 0x7f17281f0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:01.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.606+0000 7f17277fe640 1 -- 172.21.15.40:0/3634384013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17280b3c20 msgr2=0x7f1728069990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:01.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.606+0000 7f17277fe640 1 --2- 172.21.15.40:0/3634384013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17280b3c20 0x7f1728069990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:01.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.606+0000 7f17277fe640 1 -- 172.21.15.40:0/3634384013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1710009560 con 0x7f1728142be0 2023-10-18T09:02:01.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.607+0000 7f17277fe640 1 --2- 172.21.15.40:0/3634384013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1728142be0 0x7f172813ef10 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f172000cad0 tx=0x7f172000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:01.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.607+0000 7f171f7fe640 1 -- 172.21.15.40:0/3634384013 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172000ae20 con 0x7f1728142be0 2023-10-18T09:02:01.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.608+0000 7f171f7fe640 1 -- 172.21.15.40:0/3634384013 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1720004b20 con 0x7f1728142be0 2023-10-18T09:02:01.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.608+0000 7f171f7fe640 1 -- 172.21.15.40:0/3634384013 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720005020 con 0x7f1728142be0 2023-10-18T09:02:01.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.608+0000 7f172d7d0640 1 -- 172.21.15.40:0/3634384013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17280c8d70 con 0x7f1728142be0 2023-10-18T09:02:01.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.608+0000 7f172d7d0640 1 -- 172.21.15.40:0/3634384013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17280c9330 con 0x7f1728142be0 2023-10-18T09:02:01.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.610+0000 7f172d7d0640 1 -- 172.21.15.40:0/3634384013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1728066fd0 con 0x7f1728142be0 2023-10-18T09:02:01.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.611+0000 7f171f7fe640 1 -- 172.21.15.40:0/3634384013 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1720005180 con 0x7f1728142be0 2023-10-18T09:02:01.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.613+0000 7f171f7fe640 1 --2- 172.21.15.40:0/3634384013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f170c06ae90 0x7f170c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:01.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.613+0000 7f171f7fe640 1 -- 172.21.15.40:0/3634384013 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f172008a200 con 0x7f1728142be0 2023-10-18T09:02:01.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.613+0000 7f171f7fe640 1 -- 172.21.15.40:0/3634384013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=112}) v3 -- 0x7f170c0729c0 con 0x7f1728142be0 2023-10-18T09:02:01.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.614+0000 7f1726ffd640 1 --2- 172.21.15.40:0/3634384013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f170c06ae90 0x7f170c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.616+0000 7f1726ffd640 1 --2- 172.21.15.40:0/3634384013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f170c06ae90 0x7f170c06d350 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1718006fd0 tx=0x7f1718008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:01.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.620+0000 7f171f7fe640 1 -- 172.21.15.40:0/3634384013 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f172001d060 con 0x7f1728142be0 2023-10-18T09:02:01.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.827+0000 7f172d7d0640 1 -- 172.21.15.40:0/3634384013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f172806d080 con 0x7f1728142be0 2023-10-18T09:02:01.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.831+0000 7f171f7fe640 1 -- 172.21.15.40:0/3634384013 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v111) v1 ==== 81+0+34186 (secure 0 0 0) 0x7f1720055290 con 0x7f1728142be0 2023-10-18T09:02:01.834 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:01.835 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:02:01.835 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 111, 2023-10-18T09:02:01.835 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:02:01.835 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:02:01.835 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:01:59.138988+0000", 2023-10-18T09:02:01.836 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:02:01.836 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:02:01.836 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:02:01.836 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:02:01.836 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:02:01.837 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:02:01.837 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:02:01.837 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:02:01.837 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:02:01.837 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:01.837 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:02:01.838 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:02:01.838 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:02:01.838 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:02:01.838 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:02:01.838 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 13, 2023-10-18T09:02:01.839 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:02:01.839 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:02:01.839 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:02:01.839 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:02:01.839 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:02:01.840 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:02:01.840 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.840 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:02:01.840 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:02:01.840 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:02:01.841 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T09:02:01.841 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T09:02:01.841 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:02:01.841 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:02:01.841 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:02:01.841 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:02:01.842 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:02:01.842 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:02:01.842 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:02:01.842 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:02:01.842 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:02:01.843 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:02:01.843 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:02:01.843 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:02:01.843 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:02:01.843 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:02:01.844 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:02:01.844 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:02:01.844 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:02:01.844 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:02:01.844 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:02:01.844 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:02:01.845 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:02:01.845 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:02:01.845 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.845 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "108", 2023-10-18T09:02:01.845 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:02:01.845 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:02:01.846 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:02:01.846 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:02:01.846 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:02:01.846 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:02:01.846 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:02:01.847 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:02:01.847 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:02:01.847 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:02:01.847 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:02:01.847 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:02:01.848 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:02:01.848 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:02:01.848 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:02:01.848 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:02:01.848 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:02:01.849 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:02:01.849 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:02:01.849 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:02:01.849 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:02:01.849 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:02:01.850 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:02:01.850 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:02:01.850 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:02:01.850 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:02:01.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.850 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:02:01.851 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:02:01.851 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:02:01.851 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:02:01.851 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:02:01.851 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:02:01.852 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:02:01.852 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:02:01.852 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:02:01.852 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:02:01.852 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:02:01.853 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:02:01.853 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:02:01.853 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:02:01.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.853 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:02:01.854 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:02:01.854 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.854 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:02:01.854 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:02:01.854 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:02:01.854 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:02:01.855 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:02:01.855 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:02:01.855 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:02:01.855 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:02:01.855 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:02:01.856 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.856 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 12, 2023-10-18T09:02:01.856 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:02:01.856 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:01:35.453236+0000", 2023-10-18T09:02:01.857 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T09:02:01.857 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T09:02:01.857 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:02:01.857 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:02:01.857 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:02:01.858 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:02:01.858 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:02:01.858 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:02:01.858 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:02:01.858 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:02:01.858 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:02:01.859 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:02:01.859 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:02:01.859 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:02:01.859 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:02:01.859 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:02:01.860 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:02:01.860 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:02:01.860 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:02:01.860 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:02:01.860 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:02:01.861 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:02:01.861 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:02:01.861 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:02:01.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.861 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "108", 2023-10-18T09:02:01.861 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:02:01.862 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:02:01.862 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:02:01.862 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:02:01.862 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:02:01.862 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:02:01.863 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:02:01.863 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:02:01.863 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:02:01.863 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:02:01.863 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:02:01.864 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:02:01.864 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:02:01.864 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:02:01.864 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:02:01.864 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:02:01.865 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:02:01.865 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:02:01.865 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:02:01.865 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:02:01.865 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:02:01.865 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:02:01.866 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:02:01.866 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:02:01.866 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:02:01.866 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:02:01.866 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.867 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:02:01.867 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:02:01.867 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:02:01.867 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:02:01.867 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:02:01.867 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:02:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:02:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:02:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:02:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:02:01.868 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:02:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:02:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:02:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:02:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:02:01.869 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:02:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:02:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:02:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.870 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.871 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:02:01.871 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 2, 2023-10-18T09:02:01.871 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 2, 2023-10-18T09:02:01.871 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:02:01.871 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:02:01.871 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:02:01.872 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:02:01.872 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:02:01.872 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:02:01.872 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.873 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.873 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 13, 2023-10-18T09:02:01.873 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:02:01.873 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:01:36.012460+0000", 2023-10-18T09:02:01.873 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T09:02:01.874 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T09:02:01.874 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:02:01.874 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:02:01.874 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:02:01.874 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:02:01.874 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:02:01.875 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:02:01.875 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:02:01.875 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:02:01.875 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:02:01.875 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:02:01.876 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:02:01.876 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:02:01.876 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:02:01.876 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:02:01.876 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:02:01.876 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:02:01.877 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:02:01.877 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:02:01.877 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:02:01.877 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:02:01.877 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:02:01.877 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:02:01.878 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.878 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "108", 2023-10-18T09:02:01.878 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:02:01.878 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:02:01.878 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:02:01.879 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:02:01.879 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:02:01.879 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:02:01.879 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:02:01.879 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:02:01.880 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:02:01.880 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:02:01.880 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:02:01.880 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:02:01.880 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:02:01.880 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:02:01.881 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:02:01.881 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:02:01.881 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:02:01.881 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:02:01.881 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:02:01.882 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:02:01.882 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:02:01.882 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:02:01.882 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:02:01.882 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:02:01.883 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:02:01.883 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:02:01.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.883 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:02:01.883 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:02:01.883 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:02:01.884 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:02:01.884 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:02:01.884 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:02:01.884 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:02:01.884 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:02:01.885 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:02:01.885 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:02:01.885 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:02:01.885 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:02:01.885 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:02:01.886 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:02:01.886 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.886 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:02:01.886 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:02:01.886 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:02:01.886 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.887 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.887 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:02:01.887 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.3799999952316284, 2023-10-18T09:02:01.887 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.3799999952316284, 2023-10-18T09:02:01.887 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:02:01.888 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-18T09:02:01.888 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-18T09:02:01.888 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:02:01.888 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:02:01.888 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:02:01.888 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.889 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.889 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:01.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:02:01.889 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:02:01.890 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:01.890 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:01.890 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:01.890 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:01.890 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:01.891 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:01.891 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:01.891 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:01.891 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:01.891 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:01.892 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:01.892 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:01.892 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.892 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.892 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.893 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:02:01.893 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:01.893 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.893 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.893 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.893 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:02:01.894 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:01.894 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.894 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.894 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.894 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:01.895 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.895 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.895 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.895 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:02:01.895 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:01.896 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.896 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.896 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.896 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:02:01.896 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:01.897 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.897 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.897 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:01.897 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.898 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.898 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.898 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:02:01.898 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:01.899 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.899 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.899 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.899 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:02:01.899 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:01.900 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.900 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.900 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.900 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:01.900 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.901 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.901 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.901 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:02:01.901 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:01.902 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.902 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.902 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.902 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:02:01.902 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:01.903 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.903 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.903 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:02:01.903 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:02:01.904 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:02:01.904 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:02:01.904 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:01.904 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:01.904 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T09:02:01.904 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:01.905 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.905 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.905 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.905 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:02:01.905 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:01.906 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:01.906 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:01.906 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:01.906 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:01.906 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:01.906 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:01.907 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:01.907 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:01.907 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:01.907 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:01.907 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:01.908 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.908 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.908 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.908 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:02:01.908 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:01.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.909 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.909 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.909 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:02:01.909 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:01.909 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.910 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.910 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.910 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:01.910 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.910 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.911 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.911 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:02:01.911 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:01.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.911 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.911 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.912 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:02:01.912 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:01.912 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.912 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.912 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.913 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:01.913 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.913 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.913 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:02:01.914 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:01.914 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.914 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.914 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.915 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:02:01.915 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:01.915 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.915 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.915 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.916 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:01.916 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.916 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.916 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.916 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:02:01.916 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:01.917 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.917 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.917 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.917 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:02:01.917 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:01.918 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.918 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.918 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.918 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:02:01.918 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:02:01.918 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:02:01.919 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:02:01.919 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:01.919 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:01.919 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T09:02:01.919 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:01.919 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.920 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.920 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.920 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:02:01.920 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:01.920 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:01.921 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:01.921 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:01.921 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:01.921 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:01.921 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:01.922 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:01.922 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:01.922 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:01.922 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:01.922 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:01.923 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.923 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.923 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.923 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:02:01.923 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:01.924 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.924 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.924 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.924 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:02:01.924 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:01.925 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.925 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.925 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.925 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:01.925 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.925 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.926 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.926 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:02:01.926 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:01.926 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.926 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.927 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.927 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:02:01.927 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:01.927 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.927 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.928 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:01.928 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.928 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.928 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.928 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:02:01.929 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:01.929 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.929 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.929 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.929 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:02:01.930 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:01.930 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.930 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.930 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.930 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:01.931 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.931 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.931 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.931 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:02:01.931 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:01.932 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.932 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.932 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.932 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:02:01.932 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:01.932 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.933 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.933 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.933 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:02:01.933 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:02:01.933 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:02:01.934 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:02:01.934 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:01.934 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:01.934 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T09:02:01.934 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:01.934 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.935 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.935 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.935 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:02:01.935 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:01.935 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:01.936 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:01.936 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:01.936 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:01.936 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:01.936 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:01.936 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:01.937 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:01.937 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:01.937 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:01.937 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:01.937 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.938 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.938 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.938 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:02:01.938 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:01.938 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.939 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.939 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.939 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:02:01.939 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:01.939 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.940 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.940 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.940 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:01.940 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.940 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.940 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.941 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:02:01.941 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:01.941 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.941 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.941 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.942 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:02:01.942 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:01.942 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.942 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.942 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.943 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:01.943 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.943 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.943 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.943 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:02:01.944 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:01.944 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.944 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.944 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.944 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:02:01.945 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:01.945 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.945 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.945 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.945 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:01.946 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.946 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.946 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.946 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:02:01.946 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:01.947 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.947 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.947 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.947 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:02:01.947 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:01.948 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.948 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.948 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.948 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:02:01.948 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:02:01.948 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:02:01.949 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:02:01.949 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:01.949 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:01.949 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:01.949 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.950 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.950 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.950 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:02:01.950 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:01.950 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:02:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.954 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.954 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:02:01.954 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:01.954 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.954 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:02:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:02:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:02:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:02:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:02:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:02:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:01.962 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.962 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.962 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.962 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:02:01.962 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:02:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:02:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:02:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T09:02:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:01.964 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.964 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.964 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.964 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:02:01.964 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:01.965 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:01.965 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:01.965 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:01.965 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:01.965 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:01.965 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:01.966 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:01.966 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:01.966 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:01.966 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:01.966 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:01.967 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.967 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.967 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.967 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:02:01.967 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:01.967 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.968 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.968 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.968 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:02:01.968 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:01.968 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.969 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.969 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.969 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:01.969 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.969 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.969 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.970 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:02:01.970 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:01.970 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.970 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.970 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.971 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:02:01.971 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:01.971 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.971 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.971 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.972 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:01.972 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.972 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.972 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.972 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:02:01.973 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:01.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.973 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.973 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.973 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:02:01.973 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:01.974 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.974 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.974 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.974 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:01.975 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.975 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.975 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.975 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:02:01.975 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:01.975 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.976 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.976 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.976 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:02:01.976 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:01.976 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.977 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.977 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.977 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:02:01.977 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:02:01.977 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:02:01.978 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:02:01.978 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:01.978 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:01.978 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:01.978 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.979 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.979 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.979 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:02:01.979 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:01.979 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:01.980 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:01.980 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:01.980 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:01.980 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:01.980 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:01.980 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:01.981 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:01.981 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:01.981 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:01.981 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:01.981 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.982 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.982 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.982 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:02:01.982 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:01.982 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.983 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.983 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.983 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:02:01.983 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:01.983 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.983 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.984 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.984 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:01.984 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.984 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.984 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.985 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:02:01.985 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:01.985 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.985 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.985 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.986 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:02:01.986 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:01.986 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.986 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.986 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.987 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:01.987 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.987 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.987 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.987 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:02:01.988 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:01.988 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.988 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.988 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.988 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:02:01.988 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:01.989 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.989 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.989 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.989 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:01.989 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.990 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.990 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.990 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:02:01.990 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:01.990 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.990 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.991 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.991 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:02:01.991 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:01.991 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.991 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.992 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.992 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:02:01.992 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:02:01.992 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:02:01.992 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:02:01.992 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:01.993 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:01.993 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T09:02:01.993 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:01.993 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.993 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.994 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.994 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:02:01.994 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:01.994 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:01.994 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:01.995 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:01.995 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:01.995 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:01.995 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:01.995 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:01.995 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:01.996 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:01.996 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:01.996 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:01.996 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.997 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.997 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:01.997 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:02:01.997 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:01.997 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.998 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.998 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:01.998 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:02:01.998 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:01.998 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:01.999 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:01.999 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:01.999 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:01.999 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:01.999 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:01.999 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:02.000 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:02:02.000 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:02.000 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:02.000 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:02.000 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:02.001 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:02:02.001 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:02.001 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:02.001 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:02.001 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:02.002 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:02.002 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:02.002 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:02.002 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:02.002 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:02:02.003 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:02.003 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:02.003 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:02.003 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:02.003 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:02:02.003 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:02.004 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:02.004 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:02.004 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:02.004 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:02.004 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:02.004 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:02.005 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:02.005 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:02:02.005 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:02.005 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:02.005 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:02.006 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:02.006 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:02:02.006 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:02.006 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:02.006 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:02.006 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:02.007 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:02:02.007 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:02:02.007 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:02:02.007 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:02:02.007 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:02.008 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull", 2023-10-18T09:02:02.008 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:02.008 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:02.008 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:02.008 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:02.008 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:02.009 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:02:02.009 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:02.009 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:02:02.009 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:02.009 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:02.009 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:02.010 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:02.010 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:02.010 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:02:02.010 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:02.010 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:02.011 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:02.011 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:02:02.011 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:02.011 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:02.011 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:02.011 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:02.012 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:02.012 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:02:02.012 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:02.012 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:02.012 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:02.013 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:02:02.013 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:02.013 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:02.013 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:02.013 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:02.014 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:02.014 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:02:02.014 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:02.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:02.014 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:02.015 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:02:02.015 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:02.015 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:02.015 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:02.015 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:02.015 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:02.016 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:02:02.016 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:02.016 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:02.016 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:02.016 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:02:02.017 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:02.017 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:02.017 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:02.017 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:02.017 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:02.018 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:02:02.018 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:02.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:02.018 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:02.018 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:02:02.019 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:02.019 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:02.019 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:02.019 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:02.019 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:02.020 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:02:02.020 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:02.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:02.020 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:02.020 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:02:02.021 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:02.021 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:02.021 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:02.021 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:02.021 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:02.022 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:02:02.022 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:02.022 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:02.022 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:02.022 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:02:02.023 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:02.023 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:02.023 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:02.023 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:02.023 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:02.023 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:02:02.024 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:02.024 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:02.024 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:02.024 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:02:02.024 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:02:02.025 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:02:02.025 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:02:02.025 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:02:02.025 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:02:02.025 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:02:02.025 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:02:02.026 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:02:02.026 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:02:02.026 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:02:02.026 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:02:02.026 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:02:02.027 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:02:02.027 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:02.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:02.027 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:02:02.027 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:02:02.028 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:02:02.028 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:02:02.028 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:02:02.028 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:02:02.028 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:02:02.028 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:02:02.029 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:02:02.029 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:02:02.029 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:02:02.029 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:02.029 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:02:02.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.833+0000 7f172d7d0640 1 -- 172.21.15.40:0/3634384013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f170c06ae90 msgr2=0x7f170c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:02.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.833+0000 7f172d7d0640 1 --2- 172.21.15.40:0/3634384013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f170c06ae90 0x7f170c06d350 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1718006fd0 tx=0x7f1718008040 comp rx=0 tx=0).stop 2023-10-18T09:02:02.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.833+0000 7f172d7d0640 1 -- 172.21.15.40:0/3634384013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1728142be0 msgr2=0x7f172813ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:02.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.833+0000 7f172d7d0640 1 --2- 172.21.15.40:0/3634384013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1728142be0 0x7f172813ef10 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f172000cad0 tx=0x7f172000cf90 comp rx=0 tx=0).stop 2023-10-18T09:02:02.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.833+0000 7f172d7d0640 1 -- 172.21.15.40:0/3634384013 shutdown_connections 2023-10-18T09:02:02.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.833+0000 7f172d7d0640 1 --2- 172.21.15.40:0/3634384013 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f170c06ae90 0x7f170c06d350 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:02.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.833+0000 7f172d7d0640 1 --2- 172.21.15.40:0/3634384013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1728142be0 0x7f172813ef10 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:02.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.833+0000 7f172d7d0640 1 --2- 172.21.15.40:0/3634384013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17280b3c20 0x7f1728069990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:02.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.833+0000 7f172d7d0640 1 --2- 172.21.15.40:0/3634384013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1728071140 0x7f17281f0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:02.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.833+0000 7f172d7d0640 1 -- 172.21.15.40:0/3634384013 >> 172.21.15.40:0/3634384013 conn(0x7f17280aff00 msgr2=0x7f17280c3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:02.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.833+0000 7f172d7d0640 1 -- 172.21.15.40:0/3634384013 shutdown_connections 2023-10-18T09:02:02.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:01.833+0000 7f172d7d0640 1 -- 172.21.15.40:0/3634384013 wait complete. 2023-10-18T09:02:02.033 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-18T09:02:04.183 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:02:04.180+0000 7f6a11f15640 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2023-10-18T09:02:05.238 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:02:05.235+0000 7f6a11f15640 -1 log_channel(cluster) log [ERR] : Health check failed: Full OSDs blocking recovery: 3 pgs recovery_toofull (PG_RECOVERY_FULL) 2023-10-18T09:02:06.114 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:02:06.124 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:02:07.035 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:02:07.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.286+0000 7fdc1f06d640 1 -- 172.21.15.40:0/4198897121 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1815b760 msgr2=0x7fdc180b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:07.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.286+0000 7fdc1f06d640 1 --2- 172.21.15.40:0/4198897121 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1815b760 0x7fdc180b1120 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009860 tx=0x7fdc0802bde0 comp rx=0 tx=0).stop 2023-10-18T09:02:07.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.286+0000 7fdc1f06d640 1 -- 172.21.15.40:0/4198897121 shutdown_connections 2023-10-18T09:02:07.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.286+0000 7fdc1f06d640 1 --2- 172.21.15.40:0/4198897121 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1815b760 0x7fdc180b1120 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:07.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.286+0000 7fdc1f06d640 1 -- 172.21.15.40:0/4198897121 >> 172.21.15.40:0/4198897121 conn(0x7fdc1808acb0 msgr2=0x7fdc1809a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:07.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.286+0000 7fdc1f06d640 1 -- 172.21.15.40:0/4198897121 shutdown_connections 2023-10-18T09:02:07.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.286+0000 7fdc1f06d640 1 -- 172.21.15.40:0/4198897121 wait complete. 2023-10-18T09:02:07.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.288+0000 7fdc1f06d640 1 Processor -- start 2023-10-18T09:02:07.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.289+0000 7fdc1f06d640 1 -- start start 2023-10-18T09:02:07.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.290+0000 7fdc1f06d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc180d1580 0x7fdc181f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:07.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.290+0000 7fdc1f06d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1815b380 0x7fdc181e4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:07.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.290+0000 7fdc1f06d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc1815b760 0x7fdc181e5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:07.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.290+0000 7fdc1f06d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180c5800 con 0x7fdc1815b760 2023-10-18T09:02:07.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.290+0000 7fdc1f06d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc181e57a0 con 0x7fdc180d1580 2023-10-18T09:02:07.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.290+0000 7fdc1f06d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1815d7d0 con 0x7fdc1815b380 2023-10-18T09:02:07.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.291+0000 7fdc1e86c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc1815b760 0x7fdc181e5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:07.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.291+0000 7fdc1e86c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc1815b760 0x7fdc181e5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40972/0 (socket says 172.21.15.40:40972) 2023-10-18T09:02:07.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.291+0000 7fdc1e86c640 1 -- 172.21.15.40:0/1905499323 learned_addr learned my addr 172.21.15.40:0/1905499323 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:07.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.291+0000 7fdc1e06b640 1 --2- 172.21.15.40:0/1905499323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc180d1580 0x7fdc181f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:07.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.291+0000 7fdc1d86a640 1 --2- 172.21.15.40:0/1905499323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1815b380 0x7fdc181e4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:07.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.291+0000 7fdc1e86c640 1 -- 172.21.15.40:0/1905499323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1815b380 msgr2=0x7fdc181e4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.291+0000 7fdc1e86c640 1 --2- 172.21.15.40:0/1905499323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1815b380 0x7fdc181e4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.291+0000 7fdc1e86c640 1 -- 172.21.15.40:0/1905499323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc180d1580 msgr2=0x7fdc181f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.291+0000 7fdc1e86c640 1 --2- 172.21.15.40:0/1905499323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc180d1580 0x7fdc181f1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.291+0000 7fdc1e86c640 1 -- 172.21.15.40:0/1905499323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc08009490 con 0x7fdc1815b760 2023-10-18T09:02:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.291+0000 7fdc1d86a640 1 --2- 172.21.15.40:0/1905499323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1815b380 0x7fdc181e4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:02:07.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.292+0000 7fdc1e86c640 1 --2- 172.21.15.40:0/1905499323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc1815b760 0x7fdc181e5260 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000cb70 tx=0x7fdc1000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:07.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.292+0000 7fdc0effd640 1 -- 172.21.15.40:0/1905499323 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1000ad40 con 0x7fdc1815b760 2023-10-18T09:02:07.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.292+0000 7fdc0effd640 1 -- 172.21.15.40:0/1905499323 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdc10012420 con 0x7fdc1815b760 2023-10-18T09:02:07.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.292+0000 7fdc0effd640 1 -- 172.21.15.40:0/1905499323 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10012810 con 0x7fdc1815b760 2023-10-18T09:02:07.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.292+0000 7fdc1f06d640 1 -- 172.21.15.40:0/1905499323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc181f1d40 con 0x7fdc1815b760 2023-10-18T09:02:07.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.292+0000 7fdc1f06d640 1 -- 172.21.15.40:0/1905499323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc180c51b0 con 0x7fdc1815b760 2023-10-18T09:02:07.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.294+0000 7fdc1f06d640 1 -- 172.21.15.40:0/1905499323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc180c59b0 con 0x7fdc1815b760 2023-10-18T09:02:07.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.295+0000 7fdc0effd640 1 -- 172.21.15.40:0/1905499323 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdc1001a3b0 con 0x7fdc1815b760 2023-10-18T09:02:07.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.296+0000 7fdc0effd640 1 --2- 172.21.15.40:0/1905499323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdbfc06ae40 0x7fdbfc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:07.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.296+0000 7fdc0effd640 1 -- 172.21.15.40:0/1905499323 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fdc10089df0 con 0x7fdc1815b760 2023-10-18T09:02:07.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.296+0000 7fdc0effd640 1 -- 172.21.15.40:0/1905499323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=117}) v3 -- 0x7fdbfc072bd0 con 0x7fdc1815b760 2023-10-18T09:02:07.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.296+0000 7fdc1e06b640 1 --2- 172.21.15.40:0/1905499323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdbfc06ae40 0x7fdbfc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:07.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.298+0000 7fdc1e06b640 1 --2- 172.21.15.40:0/1905499323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdbfc06ae40 0x7fdbfc06d300 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdc14000f60 tx=0x7fdc14009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:07.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.300+0000 7fdc0effd640 1 -- 172.21.15.40:0/1905499323 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc180c59b0 con 0x7fdc1815b760 2023-10-18T09:02:07.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.495+0000 7fdc1f06d640 1 -- 172.21.15.40:0/1905499323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fdc180c59b0 con 0x7fdc1815b760 2023-10-18T09:02:07.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.499+0000 7fdc0effd640 1 -- 172.21.15.40:0/1905499323 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v116) v1 ==== 81+0+35206 (secure 0 0 0) 0x7fdc180c59b0 con 0x7fdc1815b760 2023-10-18T09:02:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:07.503 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:02:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 116, 2023-10-18T09:02:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:02:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:02:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:02:07.261448+0000", 2023-10-18T09:02:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:02:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:02:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:02:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:02:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:02:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:02:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:02:07.506 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:02:07.506 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:02:07.506 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:07.506 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:02:07.506 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:02:07.507 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:02:07.507 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:02:07.507 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:02:07.507 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 13, 2023-10-18T09:02:07.507 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:02:07.507 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:02:07.508 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "luminous", 2023-10-18T09:02:07.508 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:02:07.508 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:02:07.508 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:02:07.508 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.509 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:02:07.509 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:02:07.509 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:02:07.509 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T09:02:07.509 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T09:02:07.509 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:02:07.510 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:02:07.510 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:02:07.510 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:02:07.510 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:02:07.510 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:02:07.511 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:02:07.511 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:02:07.511 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:02:07.511 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:02:07.511 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:02:07.512 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:02:07.512 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:02:07.512 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:02:07.512 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:02:07.512 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:02:07.512 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:02:07.513 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:02:07.513 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:02:07.513 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:02:07.513 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:02:07.513 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:02:07.513 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.514 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "108", 2023-10-18T09:02:07.514 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:02:07.514 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:02:07.514 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:02:07.514 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:02:07.515 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:02:07.515 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:02:07.515 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:02:07.515 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:02:07.515 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:02:07.516 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:02:07.516 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:02:07.516 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:02:07.516 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:02:07.516 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:02:07.516 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:02:07.517 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:02:07.517 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:02:07.517 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:02:07.517 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:02:07.517 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:02:07.517 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:02:07.518 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:02:07.518 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:02:07.518 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:02:07.518 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:02:07.518 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:02:07.519 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.519 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:02:07.519 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:02:07.519 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:02:07.519 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:02:07.519 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:02:07.520 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:02:07.520 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:02:07.520 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:02:07.520 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:02:07.520 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:02:07.521 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:02:07.521 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:02:07.521 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:02:07.521 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:02:07.521 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.521 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:02:07.522 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:02:07.522 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.522 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:02:07.522 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:02:07.522 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:02:07.523 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:02:07.523 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:02:07.523 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:02:07.523 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:02:07.523 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:02:07.523 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:02:07.524 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.524 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.524 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.524 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 12, 2023-10-18T09:02:07.524 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:02:07.525 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:01:35.453236+0000", 2023-10-18T09:02:07.525 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T09:02:07.525 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T09:02:07.525 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:02:07.525 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:02:07.525 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:02:07.526 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:02:07.526 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:02:07.526 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:02:07.526 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:02:07.526 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:02:07.527 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:02:07.527 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:02:07.527 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:02:07.527 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:02:07.527 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:02:07.528 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:02:07.528 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:02:07.528 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:02:07.528 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:02:07.528 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:02:07.528 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:02:07.529 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:02:07.529 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:02:07.529 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:02:07.529 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.529 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "108", 2023-10-18T09:02:07.530 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:02:07.530 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:02:07.530 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:02:07.530 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:02:07.530 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:02:07.530 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:02:07.531 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:02:07.531 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:02:07.531 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:02:07.531 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:02:07.531 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:02:07.532 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:02:07.532 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:02:07.532 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:02:07.532 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:02:07.532 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:02:07.532 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:02:07.533 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:02:07.533 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:02:07.533 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:02:07.533 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:02:07.533 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:02:07.534 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:02:07.534 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:02:07.534 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:02:07.534 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:02:07.534 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.534 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:02:07.535 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:02:07.535 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:02:07.535 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:02:07.535 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:02:07.535 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:02:07.536 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:02:07.536 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:02:07.536 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:02:07.536 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:02:07.536 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:02:07.536 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:02:07.537 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:02:07.537 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:02:07.537 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:02:07.537 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.537 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:02:07.538 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:02:07.538 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:02:07.538 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.538 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.538 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:02:07.539 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 2, 2023-10-18T09:02:07.539 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 2, 2023-10-18T09:02:07.539 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:02:07.539 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:02:07.539 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:02:07.539 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:02:07.540 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:02:07.540 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:02:07.540 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.540 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.540 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.541 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 13, 2023-10-18T09:02:07.541 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:02:07.541 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:01:36.012460+0000", 2023-10-18T09:02:07.541 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 3, 2023-10-18T09:02:07.542 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full", 2023-10-18T09:02:07.542 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:02:07.542 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:02:07.542 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:02:07.542 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:02:07.542 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:02:07.543 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:02:07.543 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:02:07.543 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:02:07.543 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:02:07.543 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:02:07.544 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:02:07.544 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:02:07.544 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:02:07.544 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:02:07.544 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:02:07.545 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:02:07.545 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:02:07.545 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:02:07.545 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:02:07.545 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:02:07.546 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:02:07.546 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:02:07.546 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.546 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "108", 2023-10-18T09:02:07.546 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:02:07.547 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:02:07.547 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:02:07.547 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:02:07.547 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:02:07.547 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:02:07.548 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:02:07.548 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:02:07.548 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:02:07.548 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:02:07.548 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:02:07.548 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:02:07.549 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:02:07.549 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:02:07.549 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:02:07.549 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:02:07.549 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:02:07.550 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:02:07.550 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:02:07.550 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:02:07.550 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:02:07.550 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:02:07.551 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:02:07.551 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:02:07.551 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:02:07.551 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:02:07.551 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.551 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:02:07.552 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:02:07.552 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:02:07.552 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:02:07.552 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:02:07.552 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:02:07.553 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:02:07.553 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:02:07.553 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:02:07.553 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:02:07.553 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:02:07.554 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:02:07.554 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:02:07.554 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:02:07.554 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.554 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:02:07.554 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:02:07.555 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:02:07.555 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.555 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.555 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:02:07.555 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.3799999952316284, 2023-10-18T09:02:07.556 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.3799999952316284, 2023-10-18T09:02:07.557 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:02:07.557 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-18T09:02:07.557 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-18T09:02:07.558 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:02:07.558 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:02:07.558 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:02:07.558 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.558 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.559 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:07.559 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:02:07.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.559 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:02:07.559 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:07.560 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:07.560 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:07.560 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:07.560 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:07.560 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:07.560 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:07.561 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:07.561 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 112, 2023-10-18T09:02:07.561 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:07.561 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:07.561 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:07.561 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.562 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.562 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.562 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:02:07.562 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:07.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:02:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.564 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.564 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.564 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:07.564 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.564 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.565 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.565 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:02:07.565 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:07.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.565 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.566 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.566 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:02:07.566 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:07.566 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.566 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.566 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.567 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:07.567 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.567 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.567 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.567 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:02:07.568 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:07.568 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.568 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.568 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.568 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:02:07.568 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:07.569 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.569 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.569 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:07.569 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.569 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.570 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.570 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:02:07.570 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:07.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.570 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.571 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.571 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:02:07.571 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:07.571 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.571 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.572 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:02:07.572 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:02:07.572 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:02:07.572 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:02:07.572 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:07.573 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:07.573 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:07.573 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.573 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.574 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:02:07.574 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:07.574 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:07.574 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:07.574 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:07.575 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:07.575 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:07.575 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:07.575 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:07.575 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:07.575 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:07.576 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:07.576 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:07.576 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.576 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.576 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.576 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:02:07.577 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:07.577 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.577 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.577 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.577 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:02:07.578 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:07.578 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.578 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.578 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:07.579 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.579 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.579 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:02:07.579 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:07.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.580 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.580 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.580 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:02:07.580 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:07.580 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.581 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.581 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.581 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:07.581 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.581 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.582 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.582 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:02:07.582 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:07.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.582 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.583 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.583 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:02:07.583 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:07.583 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.583 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.584 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:07.584 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.584 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.584 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.584 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:02:07.585 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:07.585 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.585 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.585 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.585 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:02:07.585 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:07.586 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.586 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.586 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.586 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:02:07.586 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:02:07.587 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:02:07.587 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:02:07.587 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:07.587 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:07.587 INFO:teuthology.orchestra.run.smithi040.stdout: "full", 2023-10-18T09:02:07.588 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:07.588 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.588 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.588 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:02:07.588 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:07.589 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:07.589 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:07.589 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:07.589 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:07.589 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:07.590 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:07.590 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:07.590 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:07.590 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:07.590 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:07.590 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:07.591 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.591 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.591 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:02:07.591 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:07.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.592 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.592 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.592 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:02:07.592 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:07.592 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.593 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.593 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:07.593 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.593 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.594 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:02:07.594 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:07.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.594 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.594 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.595 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:02:07.595 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:07.595 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.595 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.596 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:07.596 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.596 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.596 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:02:07.596 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:07.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.597 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.597 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:02:07.597 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:07.598 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.598 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.598 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:07.598 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.599 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.599 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:02:07.599 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:07.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.600 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.600 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:02:07.600 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:07.600 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.600 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.600 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.601 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:02:07.601 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:02:07.601 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:02:07.601 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:02:07.601 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:07.602 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:07.602 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:07.602 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.603 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:02:07.603 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:07.603 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:07.603 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:07.603 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:07.603 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:07.604 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:07.604 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:07.604 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:07.604 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:07.604 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:07.605 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:07.605 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:07.605 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.605 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.606 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:02:07.606 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:07.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.606 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.606 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:02:07.607 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:07.607 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.607 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.607 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:07.608 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.608 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.608 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:02:07.608 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:07.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.609 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.609 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.609 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:02:07.609 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:07.609 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.610 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.610 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:07.610 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.611 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.611 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.611 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:02:07.611 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:07.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.611 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.612 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.612 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:02:07.612 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:07.612 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.612 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.613 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:07.613 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.613 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.614 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:02:07.614 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:07.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.614 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.614 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:02:07.615 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:07.615 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.615 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.615 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:02:07.616 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:02:07.616 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:02:07.616 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:02:07.616 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:07.616 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:07.616 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:07.617 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.617 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:02:07.617 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:07.618 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:07.618 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:07.618 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:07.618 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:07.618 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:07.619 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:07.619 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:07.619 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:07.619 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:07.619 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:07.620 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:07.620 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.620 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.620 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:02:07.620 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:07.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.621 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.621 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:02:07.621 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:07.622 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.622 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.622 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:07.622 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.623 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.623 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:02:07.623 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:07.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.624 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.624 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:02:07.624 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:07.624 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.624 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.625 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:07.625 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.625 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.626 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:02:07.626 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:07.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.626 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.626 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.627 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:02:07.627 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:07.627 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.627 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.627 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:07.628 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.628 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.628 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.628 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:02:07.628 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:07.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.629 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.629 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.629 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:02:07.629 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:07.629 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.630 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.630 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:02:07.630 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:02:07.630 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:02:07.631 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:02:07.631 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:07.631 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull", 2023-10-18T09:02:07.631 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:07.631 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:07.631 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.632 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.632 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:02:07.632 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:07.632 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:07.633 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:07.633 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:07.633 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:07.633 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:07.633 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:07.633 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:07.634 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 112, 2023-10-18T09:02:07.634 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:07.634 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:07.634 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:07.634 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.635 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.635 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:02:07.635 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:07.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.635 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.636 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.636 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:02:07.636 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:07.636 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.636 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.637 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:07.637 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.637 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.637 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:02:07.638 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:07.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.638 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.638 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.638 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:02:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:02:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:02:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:02:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:02:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:02:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:02:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:02:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:02:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:02:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 112, 2023-10-18T09:02:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:02:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:02:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:02:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:02:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:02:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:02:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:02:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:02:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:02:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:02:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:02:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:02:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:02:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:02:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:02:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:02:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:02:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:02:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:02:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:02:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:02:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:02:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:02:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:02:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:02:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:02:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:02:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:02:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:02:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:02:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:02:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:02:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:02:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:02:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:02:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:02:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:02:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:02:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:02:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:02:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:02:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:02:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:02:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [ 2023-10-18T09:02:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.1", 2023-10-18T09:02:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 2, 2023-10-18T09:02:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 7 2023-10-18T09:02:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.14", 2023-10-18T09:02:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 6, 2023-10-18T09:02:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 5 2023-10-18T09:02:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.1c", 2023-10-18T09:02:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 1, 2023-10-18T09:02:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 5 2023-10-18T09:02:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.20", 2023-10-18T09:02:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 1, 2023-10-18T09:02:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 5 2023-10-18T09:02:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.34", 2023-10-18T09:02:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 6, 2023-10-18T09:02:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 5 2023-10-18T09:02:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.38", 2023-10-18T09:02:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 1, 2023-10-18T09:02:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 3 2023-10-18T09:02:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:02:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:02:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:02:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:02:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:02:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:02:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:02:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:02:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:02:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:02:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:02:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:02:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:02:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:02:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:02:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:02:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:02:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:02:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:02:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:02:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:02:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:02:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:02:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:07.706 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:02:07.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.502+0000 7fdc1f06d640 1 -- 172.21.15.40:0/1905499323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdbfc06ae40 msgr2=0x7fdbfc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:07.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.502+0000 7fdc1f06d640 1 --2- 172.21.15.40:0/1905499323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdbfc06ae40 0x7fdbfc06d300 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fdc14000f60 tx=0x7fdc14009040 comp rx=0 tx=0).stop 2023-10-18T09:02:07.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.502+0000 7fdc1f06d640 1 -- 172.21.15.40:0/1905499323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc1815b760 msgr2=0x7fdc181e5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:07.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.502+0000 7fdc1f06d640 1 --2- 172.21.15.40:0/1905499323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc1815b760 0x7fdc181e5260 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000cb70 tx=0x7fdc1000a430 comp rx=0 tx=0).stop 2023-10-18T09:02:07.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.502+0000 7fdc1f06d640 1 -- 172.21.15.40:0/1905499323 shutdown_connections 2023-10-18T09:02:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.502+0000 7fdc1f06d640 1 --2- 172.21.15.40:0/1905499323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdbfc06ae40 0x7fdbfc06d300 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.502+0000 7fdc1f06d640 1 --2- 172.21.15.40:0/1905499323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc1815b760 0x7fdc181e5260 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.502+0000 7fdc1f06d640 1 --2- 172.21.15.40:0/1905499323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc1815b380 0x7fdc181e4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.502+0000 7fdc1f06d640 1 --2- 172.21.15.40:0/1905499323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc180d1580 0x7fdc181f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.502+0000 7fdc1f06d640 1 -- 172.21.15.40:0/1905499323 >> 172.21.15.40:0/1905499323 conn(0x7fdc1808acb0 msgr2=0x7fdc1806bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.502+0000 7fdc1f06d640 1 -- 172.21.15.40:0/1905499323 shutdown_connections 2023-10-18T09:02:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:07.502+0000 7fdc1f06d640 1 -- 172.21.15.40:0/1905499323 wait complete. 2023-10-18T09:02:07.710 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-18T09:02:12.712 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:02:12.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.945+0000 7f2bc93bf640 1 -- 172.21.15.40:0/1320394588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc40ca0d0 msgr2=0x7f2bc4157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:12.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.945+0000 7f2bc93bf640 1 --2- 172.21.15.40:0/1320394588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc40ca0d0 0x7f2bc4157d40 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f2bc406aeb0 tx=0x7f2bbc02c2f0 comp rx=0 tx=0).stop 2023-10-18T09:02:12.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.945+0000 7f2bc93bf640 1 -- 172.21.15.40:0/1320394588 shutdown_connections 2023-10-18T09:02:12.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.945+0000 7f2bc93bf640 1 --2- 172.21.15.40:0/1320394588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc40ca0d0 0x7f2bc4157d40 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:12.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.945+0000 7f2bc93bf640 1 --2- 172.21.15.40:0/1320394588 >> v2:172.21.15.144:3300/0 conn(0x7f2bc40b15b0 0x7f2bc40b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:12.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.945+0000 7f2bc93bf640 1 -- 172.21.15.40:0/1320394588 >> 172.21.15.40:0/1320394588 conn(0x7f2bc4069720 msgr2=0x7f2bc4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:12.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.946+0000 7f2bc93bf640 1 -- 172.21.15.40:0/1320394588 shutdown_connections 2023-10-18T09:02:12.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.946+0000 7f2bc93bf640 1 -- 172.21.15.40:0/1320394588 wait complete. 2023-10-18T09:02:12.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.947+0000 7f2bc93bf640 1 Processor -- start 2023-10-18T09:02:12.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.947+0000 7f2bc93bf640 1 -- start start 2023-10-18T09:02:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.948+0000 7f2bc93bf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bc40b15b0 0x7f2bc4145930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.948+0000 7f2bc93bf640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bc40b1e10 0x7f2bc4157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.948+0000 7f2bc93bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc40ca0d0 0x7f2bc4159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.949+0000 7f2bc93bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc41463f0 con 0x7f2bc40ca0d0 2023-10-18T09:02:12.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.949+0000 7f2bc93bf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4158770 con 0x7f2bc40b15b0 2023-10-18T09:02:12.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.949+0000 7f2bc93bf640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4158a50 con 0x7f2bc40b1e10 2023-10-18T09:02:12.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.949+0000 7f2bc8bbe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc40ca0d0 0x7f2bc4159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:12.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.949+0000 7f2bc8bbe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc40ca0d0 0x7f2bc4159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40988/0 (socket says 172.21.15.40:40988) 2023-10-18T09:02:12.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.949+0000 7f2bc8bbe640 1 -- 172.21.15.40:0/622491637 learned_addr learned my addr 172.21.15.40:0/622491637 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:12.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.949+0000 7f2bc3fff640 1 --2- 172.21.15.40:0/622491637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bc40b15b0 0x7f2bc4145930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:12.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.949+0000 7f2bbbfff640 1 --2- 172.21.15.40:0/622491637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bc40b1e10 0x7f2bc4157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:12.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.949+0000 7f2bc8bbe640 1 -- 172.21.15.40:0/622491637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bc40b1e10 msgr2=0x7f2bc4157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:12.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.949+0000 7f2bc8bbe640 1 --2- 172.21.15.40:0/622491637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bc40b1e10 0x7f2bc4157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:12.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.949+0000 7f2bc8bbe640 1 -- 172.21.15.40:0/622491637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bc40b15b0 msgr2=0x7f2bc4145930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:12.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.949+0000 7f2bc8bbe640 1 --2- 172.21.15.40:0/622491637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bc40b15b0 0x7f2bc4145930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:12.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.949+0000 7f2bc8bbe640 1 -- 172.21.15.40:0/622491637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc009900 con 0x7f2bc40ca0d0 2023-10-18T09:02:12.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.950+0000 7f2bc8bbe640 1 --2- 172.21.15.40:0/622491637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc40ca0d0 0x7f2bc4159080 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc037040 tx=0x7f2bbc035e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:12.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.950+0000 7f2bc17fa640 1 -- 172.21.15.40:0/622491637 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc02cea0 con 0x7f2bc40ca0d0 2023-10-18T09:02:12.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.950+0000 7f2bc17fa640 1 -- 172.21.15.40:0/622491637 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2bbc004250 con 0x7f2bc40ca0d0 2023-10-18T09:02:12.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.950+0000 7f2bc17fa640 1 -- 172.21.15.40:0/622491637 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc004640 con 0x7f2bc40ca0d0 2023-10-18T09:02:12.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.950+0000 7f2bc93bf640 1 -- 172.21.15.40:0/622491637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc415bef0 con 0x7f2bc40ca0d0 2023-10-18T09:02:12.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.950+0000 7f2bc93bf640 1 -- 172.21.15.40:0/622491637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc40cb1b0 con 0x7f2bc40ca0d0 2023-10-18T09:02:12.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.951+0000 7f2bc93bf640 1 -- 172.21.15.40:0/622491637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc40ca890 con 0x7f2bc40ca0d0 2023-10-18T09:02:12.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.953+0000 7f2bc17fa640 1 -- 172.21.15.40:0/622491637 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2bbc03b040 con 0x7f2bc40ca0d0 2023-10-18T09:02:12.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.953+0000 7f2bc17fa640 1 --2- 172.21.15.40:0/622491637 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ba406ae40 0x7f2ba406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:12.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.954+0000 7f2bc17fa640 1 -- 172.21.15.40:0/622491637 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f2bbc0ac360 con 0x7f2bc40ca0d0 2023-10-18T09:02:12.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.954+0000 7f2bc3fff640 1 --2- 172.21.15.40:0/622491637 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ba406ae40 0x7f2ba406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:12.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.955+0000 7f2bc3fff640 1 --2- 172.21.15.40:0/622491637 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ba406ae40 0x7f2ba406d300 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4005fd0 tx=0x7f2bb4005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:12.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:12.958+0000 7f2bc17fa640 1 -- 172.21.15.40:0/622491637 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2bbc077350 con 0x7f2bc40ca0d0 2023-10-18T09:02:13.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.153+0000 7f2bc93bf640 1 -- 172.21.15.40:0/622491637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2bc4066e00 con 0x7f2bc40ca0d0 2023-10-18T09:02:13.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.155+0000 7f2bc17fa640 1 -- 172.21.15.40:0/622491637 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v118) v1 ==== 81+0+35135 (secure 0 0 0) 0x7f2bbc07a970 con 0x7f2bc40ca0d0 2023-10-18T09:02:13.159 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:13.159 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:02:13.159 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 118, 2023-10-18T09:02:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:02:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:02:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:02:09.285724+0000", 2023-10-18T09:02:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:02:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:02:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:02:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:02:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:02:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:02:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:02:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:02:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:02:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:02:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:02:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:02:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:02:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:02:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 13, 2023-10-18T09:02:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:02:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:02:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "luminous", 2023-10-18T09:02:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:02:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:02:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:02:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:02:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:02:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:02:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:02:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:02:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:02:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:02:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:02:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:02:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:02:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:02:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:02:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:02:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:02:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:02:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:02:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:02:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:02:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:02:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:02:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:02:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:02:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:02:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:02:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:02:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:02:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:02:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:02:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:02:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:02:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:02:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:02:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:02:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:02:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:02:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:02:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:02:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:02:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:02:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:02:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:02:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:02:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:02:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:02:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:02:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:02:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:02:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:02:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:02:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:02:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:02:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:02:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:02:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:02:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:02:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:02:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:02:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:02:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:02:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:02:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:02:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:02:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:02:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:02:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:02:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:02:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:02:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:02:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:02:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:02:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:02:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:02:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:02:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:02:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:02:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:02:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:02:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:02:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:02:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 12, 2023-10-18T09:02:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:02:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:01:35.453236+0000", 2023-10-18T09:02:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:02:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:02:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:02:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:02:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:02:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:02:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:02:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:02:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:02:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:02:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:02:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:02:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:02:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:02:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:02:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:02:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:02:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:02:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:02:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:02:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:02:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:02:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:02:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:02:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:02:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:02:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:02:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:02:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:02:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:02:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:02:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:02:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:02:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:02:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:02:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:02:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:02:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:02:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:02:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:02:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:02:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:02:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:02:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:02:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:02:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:02:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:02:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:02:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:02:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:02:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:02:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:02:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:02:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:02:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:02:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:02:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:02:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:02:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:02:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:02:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:02:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:02:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:02:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:02:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:02:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:02:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:02:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:02:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:02:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:02:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 2, 2023-10-18T09:02:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 2, 2023-10-18T09:02:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:02:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:02:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:02:13.196 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:02:13.196 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:02:13.196 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:02:13.196 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.196 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.197 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.197 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 13, 2023-10-18T09:02:13.197 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:02:13.197 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:01:36.012460+0000", 2023-10-18T09:02:13.197 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:02:13.197 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:02:13.198 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:02:13.198 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:02:13.198 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:02:13.198 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:02:13.198 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:02:13.199 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:02:13.199 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:02:13.199 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:02:13.199 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:02:13.199 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:02:13.199 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:02:13.200 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:02:13.200 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:02:13.200 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:02:13.200 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:02:13.200 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:02:13.201 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:02:13.201 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:02:13.201 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:02:13.201 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:02:13.201 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:02:13.201 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:02:13.202 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.202 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:02:13.202 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:02:13.202 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:02:13.202 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:02:13.203 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:02:13.203 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:02:13.203 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:02:13.203 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:02:13.203 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:02:13.203 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:02:13.204 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:02:13.204 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:02:13.204 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:02:13.204 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:02:13.204 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:02:13.205 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:02:13.205 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:02:13.205 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:02:13.205 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:02:13.205 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:02:13.205 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:02:13.206 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:02:13.206 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:02:13.206 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:02:13.206 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:02:13.206 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:02:13.207 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:02:13.207 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.207 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:02:13.207 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:02:13.207 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:02:13.207 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:02:13.208 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:02:13.208 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:02:13.208 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:02:13.208 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:02:13.208 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:02:13.209 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:02:13.209 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:02:13.209 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:02:13.209 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:02:13.209 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:02:13.210 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.210 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:02:13.210 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:02:13.210 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:02:13.210 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.210 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.211 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:02:13.211 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.3799999952316284, 2023-10-18T09:02:13.211 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.3799999952316284, 2023-10-18T09:02:13.211 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:02:13.211 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-18T09:02:13.212 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-18T09:02:13.212 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:02:13.212 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:02:13.212 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:02:13.212 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.213 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.213 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:13.213 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:02:13.213 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.213 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:02:13.213 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:13.214 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:13.214 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:13.214 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:13.214 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:13.214 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:13.215 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:13.215 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:13.215 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 112, 2023-10-18T09:02:13.215 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:13.215 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:13.215 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:13.216 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.216 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.216 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.216 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:02:13.216 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:13.217 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.217 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.217 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.217 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:02:13.217 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:13.217 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.218 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.218 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.218 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:13.218 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.218 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.219 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.219 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:02:13.219 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:13.219 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.219 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.220 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.220 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:02:13.220 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:13.220 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.220 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.220 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.221 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:13.221 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.221 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.221 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.221 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:02:13.222 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:13.222 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.222 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.222 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.222 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:02:13.222 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:13.223 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.223 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.223 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.223 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:13.223 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.224 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.224 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.224 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:02:13.224 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:13.224 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.225 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.225 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.225 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:02:13.225 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:13.225 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.225 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.226 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.226 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:02:13.226 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:02:13.226 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:02:13.226 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:02:13.227 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:13.227 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:13.227 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:13.227 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.227 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.227 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.228 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:02:13.228 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:13.228 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:13.228 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:13.228 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:13.228 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:13.229 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:13.229 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:13.229 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:13.229 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:13.229 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:13.230 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:13.230 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:13.230 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.230 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.230 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.231 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:02:13.231 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:13.231 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.231 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.231 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.231 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:02:13.232 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:13.232 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.232 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.232 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.232 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:13.233 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.233 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.233 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.233 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:02:13.233 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:13.234 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.234 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.234 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.234 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:02:13.234 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:13.234 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.235 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.235 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.235 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:13.235 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.235 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.236 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.236 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:02:13.236 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:13.236 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.236 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.236 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.237 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:02:13.237 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:13.237 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.237 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.237 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.238 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:13.238 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.238 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.238 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.238 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:02:13.238 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:13.239 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.239 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.239 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.239 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:02:13.239 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:13.240 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.240 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.240 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.240 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:02:13.240 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:02:13.240 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:02:13.241 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:02:13.241 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:13.241 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:13.241 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:13.241 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.242 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.242 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.242 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:02:13.242 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:13.242 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:13.242 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:13.243 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:13.243 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:13.243 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:13.243 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:13.243 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:13.244 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:13.244 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:13.244 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:13.244 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:13.244 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.245 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.245 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.245 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:02:13.245 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:13.245 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.245 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.246 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.246 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:02:13.246 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:13.246 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.246 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.247 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.247 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:13.247 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.247 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.247 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.248 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:02:13.248 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:13.248 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.248 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.248 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.248 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:02:13.249 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:13.249 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.249 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.249 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.249 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:13.250 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.250 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.250 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.250 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:02:13.250 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:13.250 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.251 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.251 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.251 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:02:13.251 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:13.251 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.252 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.252 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.252 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:13.252 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.252 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.252 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.253 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:02:13.253 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:13.253 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.253 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.253 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.254 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:02:13.254 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:13.254 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.254 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.254 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.255 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:02:13.255 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:02:13.255 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:02:13.255 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:02:13.255 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:13.255 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:13.256 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:13.256 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.256 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.256 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.256 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:02:13.256 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:13.257 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:13.257 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:13.257 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:13.257 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:13.257 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:13.257 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:13.258 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:13.258 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:13.258 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:13.258 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:13.258 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:13.259 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.259 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.259 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.259 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:02:13.259 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:13.260 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.260 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.260 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.260 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:02:13.260 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:13.260 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.261 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.261 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.261 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:13.261 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.261 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.262 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.262 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:02:13.262 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:13.262 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.262 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.262 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.263 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:02:13.263 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:13.263 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.263 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.263 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.264 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:13.264 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.264 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.264 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.264 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:02:13.265 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:13.265 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.265 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.265 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.265 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:02:13.265 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:13.266 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.266 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.266 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.266 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:13.266 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.267 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.267 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.267 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:02:13.267 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:13.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.267 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.268 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.268 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:02:13.268 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:13.268 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.268 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.269 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:02:13.269 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:02:13.269 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:02:13.269 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:02:13.269 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:13.270 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:13.270 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:13.270 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.270 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.270 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.271 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:02:13.271 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:13.271 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:13.271 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:13.271 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:13.271 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:13.272 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:13.272 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:13.272 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:13.272 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:13.272 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:13.273 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:13.273 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:13.273 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.273 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.273 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.273 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:02:13.274 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:13.274 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.274 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.274 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.274 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:02:13.275 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:13.275 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.275 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.275 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.275 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:13.277 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.277 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.277 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.277 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:02:13.277 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:13.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.278 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.278 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.278 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:02:13.278 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:13.279 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.279 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.279 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.279 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:13.279 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.280 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.280 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.280 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:02:13.280 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:13.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.281 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.281 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:02:13.281 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:13.281 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.281 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.282 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:13.282 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.282 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.283 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:02:13.283 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:13.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.283 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.283 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.283 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:02:13.284 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:13.284 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.284 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.284 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:02:13.285 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:02:13.285 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:02:13.285 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:02:13.285 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:13.285 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:13.286 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:13.286 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.286 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.286 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.286 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:02:13.286 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:13.287 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:13.287 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:13.287 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:13.287 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:13.287 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:13.288 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:13.288 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:13.288 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 112, 2023-10-18T09:02:13.288 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:13.288 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:13.288 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:13.289 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.289 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.289 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.289 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:02:13.289 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:13.290 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.290 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.290 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.290 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:02:13.290 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:13.291 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.291 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.291 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:13.291 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.291 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.292 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.292 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:02:13.292 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:13.292 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.292 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.293 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.293 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:02:13.293 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:13.293 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.293 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.294 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:13.294 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.294 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.294 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.294 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:02:13.295 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:13.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.295 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.295 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:02:13.296 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:13.296 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.296 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.296 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:13.296 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.297 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.297 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:02:13.297 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:13.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.298 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.298 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:02:13.298 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:13.298 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.299 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.299 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:02:13.299 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:02:13.299 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:02:13.299 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:02:13.300 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:13.300 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:13.300 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:13.300 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.301 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:02:13.301 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:13.301 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:13.301 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:13.301 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:13.302 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:13.302 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:13.302 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:13.302 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:13.302 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 112, 2023-10-18T09:02:13.303 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:13.303 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:13.303 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:13.303 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.303 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.303 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.304 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:02:13.304 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:13.304 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.304 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.305 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:02:13.305 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:13.305 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.305 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.305 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.305 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:13.306 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.306 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.306 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.306 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:02:13.306 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:13.307 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.307 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.307 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.307 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:02:13.307 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:13.307 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.308 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.308 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.308 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:13.308 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.308 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.309 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.309 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:02:13.309 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:13.309 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.309 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.309 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.310 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:02:13.310 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:13.310 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.310 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.311 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:13.311 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.311 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.311 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.311 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:02:13.311 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:13.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.312 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.312 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:02:13.312 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:13.313 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.313 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.313 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:02:13.313 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:02:13.313 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:02:13.314 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:02:13.314 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:13.314 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:13.314 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:13.314 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.315 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.315 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:02:13.315 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:13.315 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:13.315 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:13.316 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:13.316 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:13.316 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:13.316 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:13.316 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:13.317 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:13.317 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:13.317 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:13.317 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:13.317 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.318 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.318 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:02:13.318 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:13.318 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.318 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.319 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.319 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:02:13.319 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:13.319 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.319 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.320 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:13.320 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.320 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.320 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:02:13.321 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:13.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.321 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.321 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.321 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:02:13.321 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:13.322 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.322 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.322 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.322 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:13.322 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.323 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.323 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.323 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:02:13.323 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:13.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.324 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.324 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.324 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:02:13.324 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:13.324 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.324 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.325 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:13.325 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.325 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.325 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.326 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:02:13.326 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:13.326 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.326 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.326 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.326 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:02:13.327 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:13.327 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.327 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.327 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.327 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:02:13.328 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:02:13.328 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:02:13.328 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:02:13.328 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:13.328 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:13.328 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:13.329 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.329 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.329 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:13.329 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:02:13.329 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.329 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:02:13.330 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:13.330 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:13.330 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:13.330 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:13.330 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:13.331 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:02:13.331 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:13.331 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.331 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.331 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:02:13.332 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:13.332 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:13.332 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:13.332 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:13.332 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:13.332 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:02:13.333 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:13.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:02:13.333 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:13.333 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:13.334 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:13.334 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:13.334 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:13.334 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:02:13.334 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:13.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.335 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:02:13.335 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:13.335 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:13.335 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:13.336 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:13.336 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:13.336 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:02:13.336 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:13.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.337 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:02:13.337 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:13.337 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:13.337 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:13.337 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:13.338 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:13.338 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:02:13.338 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:13.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.338 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:02:13.339 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:13.339 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:13.339 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:13.339 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:13.339 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:13.340 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:02:13.340 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:13.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.340 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:02:13.340 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:13.341 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:13.341 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:13.341 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:13.341 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:13.341 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:02:13.342 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:13.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.342 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:02:13.342 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:13.342 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:13.343 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:13.343 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:13.343 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:13.343 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:02:13.343 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:13.344 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.344 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:13.344 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:02:13.344 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [ 2023-10-18T09:02:13.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.345 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.1", 2023-10-18T09:02:13.345 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:13.345 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.345 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 2, 2023-10-18T09:02:13.345 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 7 2023-10-18T09:02:13.345 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.346 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.346 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.14", 2023-10-18T09:02:13.346 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:13.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.347 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 6, 2023-10-18T09:02:13.347 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 5 2023-10-18T09:02:13.347 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.347 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.348 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.1c", 2023-10-18T09:02:13.348 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:13.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.348 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 1, 2023-10-18T09:02:13.349 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 5 2023-10-18T09:02:13.349 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.349 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.350 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.20", 2023-10-18T09:02:13.350 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:13.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.350 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 1, 2023-10-18T09:02:13.350 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 5 2023-10-18T09:02:13.350 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.351 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.351 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.34", 2023-10-18T09:02:13.351 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:13.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.352 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 6, 2023-10-18T09:02:13.352 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 5 2023-10-18T09:02:13.352 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.352 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.353 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.38", 2023-10-18T09:02:13.353 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:13.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.353 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 1, 2023-10-18T09:02:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 3 2023-10-18T09:02:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:02:13.355 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:02:13.355 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:02:13.355 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:02:13.355 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:02:13.355 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:02:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:02:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:02:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:02:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:02:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:02:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:02:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:02:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:02:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:02:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:02:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:02:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:02:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:02:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:02:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:02:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:02:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:02:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.359 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:02:13.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.158+0000 7f2bc93bf640 1 -- 172.21.15.40:0/622491637 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ba406ae40 msgr2=0x7f2ba406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:13.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.158+0000 7f2bc93bf640 1 --2- 172.21.15.40:0/622491637 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ba406ae40 0x7f2ba406d300 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4005fd0 tx=0x7f2bb4005850 comp rx=0 tx=0).stop 2023-10-18T09:02:13.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.158+0000 7f2bc93bf640 1 -- 172.21.15.40:0/622491637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc40ca0d0 msgr2=0x7f2bc4159080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.158+0000 7f2bc93bf640 1 --2- 172.21.15.40:0/622491637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc40ca0d0 0x7f2bc4159080 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc037040 tx=0x7f2bbc035e50 comp rx=0 tx=0).stop 2023-10-18T09:02:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.158+0000 7f2bc93bf640 1 -- 172.21.15.40:0/622491637 shutdown_connections 2023-10-18T09:02:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.158+0000 7f2bc93bf640 1 --2- 172.21.15.40:0/622491637 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ba406ae40 0x7f2ba406d300 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.158+0000 7f2bc93bf640 1 --2- 172.21.15.40:0/622491637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bc40ca0d0 0x7f2bc4159080 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:13.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.158+0000 7f2bc93bf640 1 --2- 172.21.15.40:0/622491637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bc40b1e10 0x7f2bc4157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:13.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.158+0000 7f2bc93bf640 1 --2- 172.21.15.40:0/622491637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bc40b15b0 0x7f2bc4145930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:13.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.158+0000 7f2bc93bf640 1 -- 172.21.15.40:0/622491637 >> 172.21.15.40:0/622491637 conn(0x7f2bc4069720 msgr2=0x7f2bc4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:13.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.158+0000 7f2bc93bf640 1 -- 172.21.15.40:0/622491637 shutdown_connections 2023-10-18T09:02:13.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.158+0000 7f2bc93bf640 1 -- 172.21.15.40:0/622491637 wait complete. 2023-10-18T09:02:13.364 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-18T09:02:13.364 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:02:13.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.589+0000 7feaca90c640 1 -- 172.21.15.40:0/625725111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40cc190 msgr2=0x7feac40b1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:13.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.589+0000 7feaca90c640 1 --2- 172.21.15.40:0/625725111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40cc190 0x7feac40b1de0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7feac406a2a0 tx=0x7feab402bd70 comp rx=0 tx=0).stop 2023-10-18T09:02:13.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.589+0000 7feaca90c640 1 -- 172.21.15.40:0/625725111 shutdown_connections 2023-10-18T09:02:13.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.589+0000 7feaca90c640 1 --2- 172.21.15.40:0/625725111 >> v2:172.21.15.144:3300/0 conn(0x7feac40b2320 0x7feac406f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:13.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.589+0000 7feaca90c640 1 --2- 172.21.15.40:0/625725111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40cc190 0x7feac40b1de0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.589+0000 7feaca90c640 1 -- 172.21.15.40:0/625725111 >> 172.21.15.40:0/625725111 conn(0x7feac4069720 msgr2=0x7feac4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.589+0000 7feaca90c640 1 -- 172.21.15.40:0/625725111 shutdown_connections 2023-10-18T09:02:13.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.589+0000 7feaca90c640 1 -- 172.21.15.40:0/625725111 wait complete. 2023-10-18T09:02:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.591+0000 7feaca90c640 1 Processor -- start 2023-10-18T09:02:13.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.591+0000 7feaca90c640 1 -- start start 2023-10-18T09:02:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.592+0000 7feaca90c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40cc190 0x7feac4157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.592+0000 7feaca90c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feac4141c80 0x7feac40af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:13.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.592+0000 7feaca90c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feac40af8f0 0x7feac406b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.592+0000 7feaca90c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac40aef50 con 0x7feac40cc190 2023-10-18T09:02:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.592+0000 7feaca90c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac40adeb0 con 0x7feac40af8f0 2023-10-18T09:02:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.592+0000 7feaca90c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac40ae190 con 0x7feac4141c80 2023-10-18T09:02:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.593+0000 7feac990a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40cc190 0x7feac4157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:13.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.593+0000 7feac990a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40cc190 0x7feac4157420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41014/0 (socket says 172.21.15.40:41014) 2023-10-18T09:02:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.593+0000 7feac990a640 1 -- 172.21.15.40:0/600884703 learned_addr learned my addr 172.21.15.40:0/600884703 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.593+0000 7feaca10b640 1 --2- 172.21.15.40:0/600884703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feac40af8f0 0x7feac406b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.593+0000 7feac9109640 1 --2- 172.21.15.40:0/600884703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feac4141c80 0x7feac40af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.593+0000 7feac990a640 1 -- 172.21.15.40:0/600884703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feac4141c80 msgr2=0x7feac40af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:13.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.593+0000 7feac990a640 1 --2- 172.21.15.40:0/600884703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feac4141c80 0x7feac40af3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.593+0000 7feac990a640 1 -- 172.21.15.40:0/600884703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feac40af8f0 msgr2=0x7feac406b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.593+0000 7feac990a640 1 --2- 172.21.15.40:0/600884703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feac40af8f0 0x7feac406b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.593+0000 7feac990a640 1 -- 172.21.15.40:0/600884703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab4009490 con 0x7feac40cc190 2023-10-18T09:02:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.593+0000 7feac9109640 1 --2- 172.21.15.40:0/600884703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feac4141c80 0x7feac40af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:13.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.593+0000 7feaca10b640 1 --2- 172.21.15.40:0/600884703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feac40af8f0 0x7feac406b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.593+0000 7feac990a640 1 --2- 172.21.15.40:0/600884703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40cc190 0x7feac4157420 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7feac000c390 tx=0x7feac000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.594+0000 7feaba7fc640 1 -- 172.21.15.40:0/600884703 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac0004230 con 0x7feac40cc190 2023-10-18T09:02:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.594+0000 7feaba7fc640 1 -- 172.21.15.40:0/600884703 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feac00044a0 con 0x7feac40cc190 2023-10-18T09:02:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.594+0000 7feaba7fc640 1 -- 172.21.15.40:0/600884703 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac0003850 con 0x7feac40cc190 2023-10-18T09:02:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.594+0000 7feaca90c640 1 -- 172.21.15.40:0/600884703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac40ae470 con 0x7feac40cc190 2023-10-18T09:02:13.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.594+0000 7feaca90c640 1 -- 172.21.15.40:0/600884703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac40aea30 con 0x7feac40cc190 2023-10-18T09:02:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.595+0000 7feaca90c640 1 -- 172.21.15.40:0/600884703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac40b0f40 con 0x7feac40cc190 2023-10-18T09:02:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.596+0000 7feaba7fc640 1 -- 172.21.15.40:0/600884703 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feac00039b0 con 0x7feac40cc190 2023-10-18T09:02:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.597+0000 7feaba7fc640 1 --2- 172.21.15.40:0/600884703 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feaa806ae40 0x7feaa806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.597+0000 7feaba7fc640 1 -- 172.21.15.40:0/600884703 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7feac0088ef0 con 0x7feac40cc190 2023-10-18T09:02:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.598+0000 7feac9109640 1 --2- 172.21.15.40:0/600884703 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feaa806ae40 0x7feaa806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.599+0000 7feac9109640 1 --2- 172.21.15.40:0/600884703 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feaa806ae40 0x7feaa806d300 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7feab4036000 tx=0x7feab4004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:13.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.601+0000 7feaba7fc640 1 -- 172.21.15.40:0/600884703 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feac0053fb0 con 0x7feac40cc190 2023-10-18T09:02:13.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.798+0000 7feaca90c640 1 -- 172.21.15.40:0/600884703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7feac4066e00 con 0x7feac40cc190 2023-10-18T09:02:13.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.801+0000 7feaba7fc640 1 -- 172.21.15.40:0/600884703 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v118) v1 ==== 81+0+35135 (secure 0 0 0) 0x7feac0017020 con 0x7feac40cc190 2023-10-18T09:02:13.805 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:13.805 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:02:13.805 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 118, 2023-10-18T09:02:13.805 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:02:13.805 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:02:13.806 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:02:09.285724+0000", 2023-10-18T09:02:13.806 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:02:13.806 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:02:13.806 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:02:13.806 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:02:13.807 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:02:13.807 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:02:13.807 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:02:13.807 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:02:13.807 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:02:13.807 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:13.808 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:02:13.808 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:02:13.808 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:02:13.808 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:02:13.808 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:02:13.809 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 13, 2023-10-18T09:02:13.809 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:02:13.809 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:02:13.809 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "luminous", 2023-10-18T09:02:13.809 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:02:13.809 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:02:13.810 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:02:13.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.810 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:02:13.810 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:02:13.810 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:02:13.811 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:02:13.811 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:02:13.811 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:02:13.811 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:02:13.811 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:02:13.812 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:02:13.812 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:02:13.812 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:02:13.812 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:02:13.812 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:02:13.813 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:02:13.813 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:02:13.813 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:02:13.813 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:02:13.813 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:02:13.813 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:02:13.814 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:02:13.814 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:02:13.814 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:02:13.814 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:02:13.814 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:02:13.815 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:02:13.815 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:02:13.815 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:02:13.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.815 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:02:13.816 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:02:13.816 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:02:13.816 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:02:13.816 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:02:13.816 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:02:13.816 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:02:13.817 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:02:13.817 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:02:13.817 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:02:13.817 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:02:13.817 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:02:13.817 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:02:13.818 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:02:13.818 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:02:13.818 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:02:13.818 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:02:13.818 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:02:13.819 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:02:13.819 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:02:13.819 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:02:13.819 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:02:13.819 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:02:13.820 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:02:13.820 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:02:13.820 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:02:13.820 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:02:13.820 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.821 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:02:13.821 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:02:13.821 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:02:13.821 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:02:13.821 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:02:13.821 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:02:13.822 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:02:13.822 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:02:13.822 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:02:13.822 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:02:13.822 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:02:13.823 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:02:13.823 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:02:13.823 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:02:13.823 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.823 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:02:13.824 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:02:13.824 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.824 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:02:13.824 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:02:13.824 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:02:13.825 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:02:13.825 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:02:13.825 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:02:13.825 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:02:13.825 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:02:13.826 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:02:13.826 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.826 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.826 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.826 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 12, 2023-10-18T09:02:13.827 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:02:13.827 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:01:35.453236+0000", 2023-10-18T09:02:13.827 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:02:13.827 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:02:13.827 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:02:13.827 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:02:13.828 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:02:13.828 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:02:13.828 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:02:13.828 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:02:13.828 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:02:13.828 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:02:13.829 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:02:13.829 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:02:13.829 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:02:13.829 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:02:13.829 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:02:13.830 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:02:13.830 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:02:13.830 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:02:13.830 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:02:13.830 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:02:13.831 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:02:13.831 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:02:13.831 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:02:13.831 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:02:13.831 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.832 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:02:13.832 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:02:13.832 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:02:13.832 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:02:13.832 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:02:13.832 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:02:13.833 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:02:13.833 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:02:13.833 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:02:13.833 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:02:13.833 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:02:13.834 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:02:13.834 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:02:13.834 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:02:13.834 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:02:13.834 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:02:13.835 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:02:13.835 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:02:13.835 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:02:13.835 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:02:13.835 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:02:13.836 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:02:13.836 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:02:13.836 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:02:13.836 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:02:13.836 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:02:13.836 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:02:13.837 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.837 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:02:13.837 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:02:13.837 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:02:13.837 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:02:13.838 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:02:13.838 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:02:13.838 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:02:13.838 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:02:13.838 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:02:13.839 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:02:13.839 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:02:13.839 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:02:13.839 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:02:13.839 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:02:13.839 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:02:13.840 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.840 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:02:13.840 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:02:13.840 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:02:13.840 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.841 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.841 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:02:13.841 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 2, 2023-10-18T09:02:13.841 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 2, 2023-10-18T09:02:13.841 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:02:13.842 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:02:13.842 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:02:13.842 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:02:13.842 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:02:13.842 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:02:13.843 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.843 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.843 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 13, 2023-10-18T09:02:13.843 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:02:13.843 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:01:36.012460+0000", 2023-10-18T09:02:13.844 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:02:13.844 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:02:13.844 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:02:13.844 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:02:13.844 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:02:13.845 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:02:13.845 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:02:13.845 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:02:13.845 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:02:13.845 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:02:13.845 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:02:13.846 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:02:13.846 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:02:13.846 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:02:13.846 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:02:13.846 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:02:13.847 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:02:13.847 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:02:13.847 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:02:13.847 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:02:13.847 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:02:13.848 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:02:13.848 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:02:13.848 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:02:13.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.848 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:02:13.848 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:02:13.849 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:02:13.849 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:02:13.849 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:02:13.849 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:02:13.849 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:02:13.850 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:02:13.850 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:02:13.850 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:02:13.850 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:02:13.850 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:02:13.850 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:02:13.851 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:02:13.851 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:02:13.851 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:02:13.851 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:02:13.851 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:02:13.852 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:02:13.852 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:02:13.852 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:02:13.852 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:02:13.852 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:02:13.852 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:02:13.853 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:02:13.853 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:02:13.853 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:02:13.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.853 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:02:13.854 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:02:13.854 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:02:13.854 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:02:13.854 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:02:13.854 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:02:13.854 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:02:13.855 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:02:13.855 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:02:13.855 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:02:13.855 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:02:13.855 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:02:13.856 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:02:13.856 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:02:13.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.856 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:02:13.856 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:02:13.856 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:02:13.857 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.857 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:02:13.857 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.3799999952316284, 2023-10-18T09:02:13.857 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.3799999952316284, 2023-10-18T09:02:13.858 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:02:13.858 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-18T09:02:13.858 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-18T09:02:13.858 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:02:13.858 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:02:13.859 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:02:13.859 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.859 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.859 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:13.859 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:02:13.860 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.860 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:02:13.860 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:13.860 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:13.860 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:13.860 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:13.861 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:13.861 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:13.861 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:13.861 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:13.861 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 112, 2023-10-18T09:02:13.861 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:13.862 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:13.862 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:13.862 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.862 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.862 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.863 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:02:13.863 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:13.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.863 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.863 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:02:13.864 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:13.864 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.864 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.864 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:13.865 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.865 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.865 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.865 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:02:13.865 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:13.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.866 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.866 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:02:13.866 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:13.866 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.867 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.867 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.867 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:13.867 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.867 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.867 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.868 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:02:13.868 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:13.868 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.868 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.868 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.869 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:02:13.869 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:13.869 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.869 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.869 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.869 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:13.870 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.870 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.870 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.870 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:02:13.870 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:13.871 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.871 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.871 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.871 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:02:13.871 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:02:13.871 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.872 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.872 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:02:13.872 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:02:13.872 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:02:13.873 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:02:13.873 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:13.873 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:13.873 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:13.873 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.873 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.874 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.874 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:02:13.874 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:13.874 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:13.874 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:13.875 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:13.875 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:13.875 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:13.875 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:13.875 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:13.875 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:13.876 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:13.876 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:13.876 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:13.876 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.876 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.876 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.877 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:02:13.877 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:13.877 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.877 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.877 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.877 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:02:13.878 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:13.878 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.878 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.878 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.878 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:13.879 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.879 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.879 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.879 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:02:13.879 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:13.879 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.880 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.880 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.880 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:02:13.880 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:13.880 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.881 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.881 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:13.881 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.881 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.882 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.882 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:02:13.882 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:13.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.883 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.883 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:02:13.883 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:13.883 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.883 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.884 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.884 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:13.884 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.884 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.884 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.884 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:02:13.885 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:13.885 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.885 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.885 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.885 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:02:13.886 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:02:13.886 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.886 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.886 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.886 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:02:13.887 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:02:13.887 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:02:13.887 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:02:13.887 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:13.887 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:13.888 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:13.888 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.888 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.888 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.888 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:02:13.889 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:13.889 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:13.889 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:13.889 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:13.889 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:13.889 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:13.890 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:13.890 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:13.890 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:13.890 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:13.890 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:13.891 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:13.891 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.891 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.891 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.891 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:02:13.891 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:13.892 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.892 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.892 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.892 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:02:13.892 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:13.892 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.893 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.893 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.893 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:13.893 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.893 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.893 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.894 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:02:13.894 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:13.894 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.894 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.895 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.895 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:02:13.895 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:13.895 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.895 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.895 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.896 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:13.896 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.896 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.896 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.896 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:02:13.897 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:13.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.897 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.897 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.897 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:02:13.897 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:13.898 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.898 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.898 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.898 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:13.898 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.899 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.899 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.899 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:02:13.899 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:13.899 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.899 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.900 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.900 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:02:13.900 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:02:13.900 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.900 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.901 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.901 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:02:13.901 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:02:13.901 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:02:13.901 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:02:13.901 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:13.902 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:13.902 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:13.902 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.902 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.902 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.903 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:02:13.903 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:13.903 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:13.903 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:13.903 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:13.904 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:13.904 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:13.904 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:13.904 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:13.904 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:13.904 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:13.905 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:13.905 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:13.905 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.905 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.905 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.906 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:02:13.906 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:13.906 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.906 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.906 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.906 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:02:13.907 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:13.907 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.907 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.907 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:13.908 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.908 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.908 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.908 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:02:13.908 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:13.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.909 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.909 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.909 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:02:13.909 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:13.909 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.910 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.910 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.910 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:13.910 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.910 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.910 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.911 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:02:13.911 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:13.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.911 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.911 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.912 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:02:13.912 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:13.912 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.912 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.912 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.912 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:13.913 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.913 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.913 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:02:13.913 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:13.914 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.914 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.914 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.914 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:02:13.914 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:02:13.914 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.915 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.915 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.915 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:02:13.915 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:02:13.915 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:02:13.916 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:02:13.916 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:13.916 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:13.916 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:13.916 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.917 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.917 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.917 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:02:13.917 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:13.917 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:13.917 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:13.918 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:13.918 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:13.918 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:13.918 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:13.918 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:13.919 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:13.919 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:13.919 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:13.919 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:13.919 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.919 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.920 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.920 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:02:13.920 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:13.920 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.920 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.921 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.921 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:02:13.921 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:13.921 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.921 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:13.933 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:13.933 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:13.933 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:13.933 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 112, 2023-10-18T09:02:13.933 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:13.934 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:13.934 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:13.934 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.934 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.934 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.935 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:02:13.935 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:13.935 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.935 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.935 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.936 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:02:13.936 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:13.936 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.936 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.936 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.937 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:13.937 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.937 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.937 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.937 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:02:13.937 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:13.938 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.938 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.938 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.938 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:02:13.938 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:13.939 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.939 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.939 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.939 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:13.939 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.940 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.940 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.940 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:02:13.940 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:13.940 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.941 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.941 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.941 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:02:13.941 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:13.941 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.942 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.942 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.942 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:13.942 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.942 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.943 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.943 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:02:13.943 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:13.943 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.943 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.944 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.944 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:02:13.944 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:02:13.944 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.944 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.944 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.945 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:02:13.945 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:02:13.945 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:02:13.945 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:02:13.945 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:13.946 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:13.946 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:13.946 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.946 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.946 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.947 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:02:13.947 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:13.947 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:13.947 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:13.947 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:13.947 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:13.948 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:13.948 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:13.948 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:13.948 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 112, 2023-10-18T09:02:13.948 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:13.949 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:13.949 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:13.949 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.949 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.949 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.950 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:02:13.950 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:13.950 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.950 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.950 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.951 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:02:13.951 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:13.951 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.951 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.951 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.951 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:13.952 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.952 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.952 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.952 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:02:13.952 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:13.953 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.953 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.953 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.953 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:02:13.953 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:13.954 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.954 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.954 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.954 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:13.954 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.955 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.955 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.955 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:02:13.955 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:13.955 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.955 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.956 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.956 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:02:13.956 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:13.956 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.956 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.957 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.957 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:13.957 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.957 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.957 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.957 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:02:13.958 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:13.958 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.958 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.958 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.958 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:02:13.959 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:02:13.959 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.959 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.959 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.959 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:02:13.959 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:02:13.960 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:02:13.960 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:02:13.960 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:13.960 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:13.960 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:13.961 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.961 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.961 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.961 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:02:13.961 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:02:13.961 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:02:13.962 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:02:13.962 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:02:13.962 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:02:13.962 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:02:13.962 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:02:13.963 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:02:13.963 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 105, 2023-10-18T09:02:13.963 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:02:13.963 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:02:13.963 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:02:13.964 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.964 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.964 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.964 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:02:13.964 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:13.964 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.965 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.965 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.965 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:02:13.965 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:13.965 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.966 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.966 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.966 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:02:13.966 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.966 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.967 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.967 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:02:13.967 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:13.967 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.967 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.967 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.968 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:02:13.968 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:13.968 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.968 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.968 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.969 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:02:13.969 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.969 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.969 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.969 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:02:13.970 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:13.970 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.970 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.970 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.970 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:02:13.971 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:13.971 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.971 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.971 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.971 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:02:13.972 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:02:13.972 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.972 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:02:13.972 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:02:13.972 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:13.972 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.973 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.973 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:02:13.973 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:02:13.973 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:02:13.973 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.974 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.974 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.974 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:02:13.974 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:02:13.974 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:02:13.975 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:02:13.975 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:02:13.975 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:02:13.975 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:02:13.975 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.975 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.976 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:13.976 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:02:13.976 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.976 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:02:13.976 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:13.977 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:13.977 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:13.977 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:13.977 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:13.977 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:02:13.978 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:13.978 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.978 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.978 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:02:13.978 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:13.978 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:13.979 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:13.979 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:13.979 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:13.979 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:02:13.979 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:13.980 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.980 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.980 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:02:13.980 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:13.980 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:13.981 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:13.981 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:13.981 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:13.981 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:02:13.981 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:13.981 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.982 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.982 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:02:13.982 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:13.982 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:13.982 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:13.983 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:13.983 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:13.983 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:02:13.983 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:13.983 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.984 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.984 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:02:13.984 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:13.984 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:13.984 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:13.985 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:13.985 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:13.985 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:02:13.985 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:13.985 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.985 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.986 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:02:13.986 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:13.986 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:13.986 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:13.986 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:13.987 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:13.987 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:02:13.987 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:13.987 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.987 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.988 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:02:13.988 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:13.988 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:13.988 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:13.988 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:13.988 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:13.989 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:02:13.989 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:13.989 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.989 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.989 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:02:13.990 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:02:13.990 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:02:13.990 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:02:13.990 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:02:13.990 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:02:13.991 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:02:13.991 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:02:13.991 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.991 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:13.991 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:02:13.992 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [ 2023-10-18T09:02:13.992 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.992 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.1", 2023-10-18T09:02:13.992 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:13.992 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.992 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 2, 2023-10-18T09:02:13.993 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 7 2023-10-18T09:02:13.993 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.993 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.993 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.993 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.994 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.14", 2023-10-18T09:02:13.994 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:13.994 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.994 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 6, 2023-10-18T09:02:13.994 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 5 2023-10-18T09:02:13.995 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.995 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.995 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.995 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.995 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.1c", 2023-10-18T09:02:13.995 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:13.996 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.996 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 1, 2023-10-18T09:02:13.996 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 5 2023-10-18T09:02:13.996 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.996 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.997 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.997 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.997 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.20", 2023-10-18T09:02:13.997 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:13.997 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.998 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 1, 2023-10-18T09:02:13.998 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 5 2023-10-18T09:02:13.998 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:13.998 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:13.998 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:13.998 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.999 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.34", 2023-10-18T09:02:13.999 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:13.999 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:13.999 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 6, 2023-10-18T09:02:13.999 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 5 2023-10-18T09:02:14.000 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:14.000 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:14.000 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:14.000 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:14.000 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "13.38", 2023-10-18T09:02:14.001 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:02:14.001 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:14.001 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 1, 2023-10-18T09:02:14.001 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 3 2023-10-18T09:02:14.001 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:14.001 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:14.002 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:14.002 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:02:14.002 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:02:14.002 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:02:14.002 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:02:14.003 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:02:14.003 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:02:14.003 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:02:14.003 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:02:14.003 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:02:14.004 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:02:14.004 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:02:14.004 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:02:14.004 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:02:14.004 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:14.005 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:14.005 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:02:14.005 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:02:14.005 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:02:14.005 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:02:14.005 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:02:14.006 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:02:14.006 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:02:14.006 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:02:14.006 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:02:14.006 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:02:14.007 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:02:14.007 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:14.007 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:02:14.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.803+0000 7feaca90c640 1 -- 172.21.15.40:0/600884703 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feaa806ae40 msgr2=0x7feaa806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:14.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.804+0000 7feaca90c640 1 --2- 172.21.15.40:0/600884703 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feaa806ae40 0x7feaa806d300 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7feab4036000 tx=0x7feab4004730 comp rx=0 tx=0).stop 2023-10-18T09:02:14.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.804+0000 7feaca90c640 1 -- 172.21.15.40:0/600884703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40cc190 msgr2=0x7feac4157420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:14.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.804+0000 7feaca90c640 1 --2- 172.21.15.40:0/600884703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40cc190 0x7feac4157420 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7feac000c390 tx=0x7feac000c850 comp rx=0 tx=0).stop 2023-10-18T09:02:14.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.804+0000 7feaca90c640 1 -- 172.21.15.40:0/600884703 shutdown_connections 2023-10-18T09:02:14.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.804+0000 7feaca90c640 1 --2- 172.21.15.40:0/600884703 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feaa806ae40 0x7feaa806d300 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:14.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.804+0000 7feaca90c640 1 --2- 172.21.15.40:0/600884703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feac40af8f0 0x7feac406b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:14.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.804+0000 7feaca90c640 1 --2- 172.21.15.40:0/600884703 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feac4141c80 0x7feac40af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:14.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.804+0000 7feaca90c640 1 --2- 172.21.15.40:0/600884703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feac40cc190 0x7feac4157420 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:14.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.804+0000 7feaca90c640 1 -- 172.21.15.40:0/600884703 >> 172.21.15.40:0/600884703 conn(0x7feac4069720 msgr2=0x7feac413d610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:14.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.804+0000 7feaca90c640 1 -- 172.21.15.40:0/600884703 shutdown_connections 2023-10-18T09:02:14.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:13.804+0000 7feaca90c640 1 -- 172.21.15.40:0/600884703 wait complete. 2023-10-18T09:02:14.012 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:02:14.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.249+0000 7fc5ed96e640 1 -- 172.21.15.40:0/3960541080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5d40062e0 msgr2=0x7fc5d400ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:14.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.249+0000 7fc5ed96e640 1 --2- 172.21.15.40:0/3960541080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5d40062e0 0x7fc5d400ca90 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000c640 tx=0x7fc5d4006900 comp rx=0 tx=0).stop 2023-10-18T09:02:14.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.249+0000 7fc5ed96e640 1 -- 172.21.15.40:0/3960541080 shutdown_connections 2023-10-18T09:02:14.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.249+0000 7fc5ed96e640 1 --2- 172.21.15.40:0/3960541080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5d40062e0 0x7fc5d400ca90 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:14.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.249+0000 7fc5ed96e640 1 -- 172.21.15.40:0/3960541080 >> 172.21.15.40:0/3960541080 conn(0x7fc5e806cee0 msgr2=0x7fc5e8069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:14.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.249+0000 7fc5ed96e640 1 -- 172.21.15.40:0/3960541080 shutdown_connections 2023-10-18T09:02:14.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.249+0000 7fc5ed96e640 1 -- 172.21.15.40:0/3960541080 wait complete. 2023-10-18T09:02:14.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.251+0000 7fc5ed96e640 1 Processor -- start 2023-10-18T09:02:14.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.251+0000 7fc5ed96e640 1 -- start start 2023-10-18T09:02:14.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.252+0000 7fc5ed96e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5d40062e0 0x7fc5e814f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:14.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.252+0000 7fc5ed96e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5e814bbf0 0x7fc5e813aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:14.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.252+0000 7fc5ed96e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e8138430 0x7fc5e813b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:14.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.252+0000 7fc5ed96e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e80a6a30 con 0x7fc5e8138430 2023-10-18T09:02:14.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.252+0000 7fc5ed96e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e8138a60 con 0x7fc5d40062e0 2023-10-18T09:02:14.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.252+0000 7fc5ed96e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e813b900 con 0x7fc5e814bbf0 2023-10-18T09:02:14.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.252+0000 7fc5e77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e8138430 0x7fc5e813b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:14.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.252+0000 7fc5e77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e8138430 0x7fc5e813b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41038/0 (socket says 172.21.15.40:41038) 2023-10-18T09:02:14.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.252+0000 7fc5e77fe640 1 -- 172.21.15.40:0/1616627843 learned_addr learned my addr 172.21.15.40:0/1616627843 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:14.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.253+0000 7fc5e6ffd640 1 --2- 172.21.15.40:0/1616627843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5d40062e0 0x7fc5e814f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:14.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.253+0000 7fc5dffff640 1 --2- 172.21.15.40:0/1616627843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5e814bbf0 0x7fc5e813aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:14.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.253+0000 7fc5e77fe640 1 -- 172.21.15.40:0/1616627843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5e814bbf0 msgr2=0x7fc5e813aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:14.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.253+0000 7fc5e77fe640 1 --2- 172.21.15.40:0/1616627843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5e814bbf0 0x7fc5e813aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:14.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.253+0000 7fc5e77fe640 1 -- 172.21.15.40:0/1616627843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5d40062e0 msgr2=0x7fc5e814f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:14.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.253+0000 7fc5e77fe640 1 --2- 172.21.15.40:0/1616627843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5d40062e0 0x7fc5e814f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:14.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.253+0000 7fc5e77fe640 1 -- 172.21.15.40:0/1616627843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e80a6470 con 0x7fc5e8138430 2023-10-18T09:02:14.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.253+0000 7fc5e77fe640 1 --2- 172.21.15.40:0/1616627843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e8138430 0x7fc5e813b030 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0002010 tx=0x7fc5e0004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:14.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.253+0000 7fc5e4ff9640 1 -- 172.21.15.40:0/1616627843 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0001730 con 0x7fc5e8138430 2023-10-18T09:02:14.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.253+0000 7fc5e4ff9640 1 -- 172.21.15.40:0/1616627843 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc5e00018b0 con 0x7fc5e8138430 2023-10-18T09:02:14.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.253+0000 7fc5e4ff9640 1 -- 172.21.15.40:0/1616627843 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e00062d0 con 0x7fc5e8138430 2023-10-18T09:02:14.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.253+0000 7fc5ed96e640 1 -- 172.21.15.40:0/1616627843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e80a6750 con 0x7fc5e8138430 2023-10-18T09:02:14.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.253+0000 7fc5ed96e640 1 -- 172.21.15.40:0/1616627843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e813b570 con 0x7fc5e8138430 2023-10-18T09:02:14.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.255+0000 7fc5ed96e640 1 -- 172.21.15.40:0/1616627843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e8138ef0 con 0x7fc5e8138430 2023-10-18T09:02:14.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.256+0000 7fc5e4ff9640 1 -- 172.21.15.40:0/1616627843 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc5e0031030 con 0x7fc5e8138430 2023-10-18T09:02:14.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.257+0000 7fc5e4ff9640 1 --2- 172.21.15.40:0/1616627843 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc5c806ae70 0x7fc5c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:14.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.257+0000 7fc5e4ff9640 1 -- 172.21.15.40:0/1616627843 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fc5e00aa3c0 con 0x7fc5e8138430 2023-10-18T09:02:14.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.258+0000 7fc5e6ffd640 1 --2- 172.21.15.40:0/1616627843 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc5c806ae70 0x7fc5c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:14.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.259+0000 7fc5e6ffd640 1 --2- 172.21.15.40:0/1616627843 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc5c806ae70 0x7fc5c806d330 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800e9a0 tx=0x7fc5d800e380 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:14.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.261+0000 7fc5e4ff9640 1 -- 172.21.15.40:0/1616627843 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc5e8138ef0 con 0x7fc5e8138430 2023-10-18T09:02:14.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.493+0000 7fc5ed96e640 1 -- 172.21.15.40:0/1616627843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5e8138ef0 con 0x7fc5e8138430 2023-10-18T09:02:14.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.495+0000 7fc5e4ff9640 1 -- 172.21.15.40:0/1616627843 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fc5e8138ef0 con 0x7fc5e8138430 2023-10-18T09:02:14.498 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:14.499 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2565609773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2565609773},{"type":"v1","addr":"172.21.15.144:6836","nonce":2565609773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":5872,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3371548374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3371548374},{"type":"v1","addr":"172.21.15.144:6837","nonce":3371548374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3328648628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3328648628},{"type":"v1","addr":"172.21.15.40:6833","nonce":3328648628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:01:36.995669+0000","modified":"2023-10-18T09:01:40.068484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/891188043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":891188043},{"type":"v1","addr":"172.21.15.40:6835","nonce":891188043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-18T09:02:14.499 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 49 2023-10-18T09:02:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.497+0000 7fc5ed96e640 1 -- 172.21.15.40:0/1616627843 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc5c806ae70 msgr2=0x7fc5c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.497+0000 7fc5ed96e640 1 --2- 172.21.15.40:0/1616627843 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc5c806ae70 0x7fc5c806d330 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800e9a0 tx=0x7fc5d800e380 comp rx=0 tx=0).stop 2023-10-18T09:02:14.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.497+0000 7fc5ed96e640 1 -- 172.21.15.40:0/1616627843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e8138430 msgr2=0x7fc5e813b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:14.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.497+0000 7fc5ed96e640 1 --2- 172.21.15.40:0/1616627843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e8138430 0x7fc5e813b030 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0002010 tx=0x7fc5e0004a90 comp rx=0 tx=0).stop 2023-10-18T09:02:14.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.497+0000 7fc5ed96e640 1 -- 172.21.15.40:0/1616627843 shutdown_connections 2023-10-18T09:02:14.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.497+0000 7fc5ed96e640 1 --2- 172.21.15.40:0/1616627843 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc5c806ae70 0x7fc5c806d330 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:14.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.497+0000 7fc5ed96e640 1 --2- 172.21.15.40:0/1616627843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5e8138430 0x7fc5e813b030 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:14.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.497+0000 7fc5ed96e640 1 --2- 172.21.15.40:0/1616627843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5e814bbf0 0x7fc5e813aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:14.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.497+0000 7fc5ed96e640 1 --2- 172.21.15.40:0/1616627843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5d40062e0 0x7fc5e814f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:14.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.497+0000 7fc5ed96e640 1 -- 172.21.15.40:0/1616627843 >> 172.21.15.40:0/1616627843 conn(0x7fc5e806cee0 msgr2=0x7fc5e8070820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:14.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.498+0000 7fc5ed96e640 1 -- 172.21.15.40:0/1616627843 shutdown_connections 2023-10-18T09:02:14.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:14.498+0000 7fc5ed96e640 1 -- 172.21.15.40:0/1616627843 wait complete. 2023-10-18T09:02:14.521 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-18T09:02:14.768 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":6996,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":37.211154972999999,"mdsmap_epoch":49,"osdmap_epoch":118,"osdmap_epoch_barrier":111,"uptime":43.067045063000002} 2023-10-18T09:02:14.768 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 6996, "mdsmap_epoch": 49, "osdmap_epoch": 118, "osdmap_epoch_barrier": 111, "rank_uptime": 37.211154973, "state": "up:active", "uptime": 43.067045063, "want_state": "up:active", "whoami": 0 } 2023-10-18T09:02:14.769 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-18T09:02:14.769 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:02:14.770 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:02:14.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:14.828 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-18T09:02:15.267 INFO:teuthology.orchestra.run.smithi040.stderr:50+0 records in 2023-10-18T09:02:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:50+0 records out 2023-10-18T09:02:15.268 INFO:teuthology.orchestra.run.smithi040.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.37917 s, 138 MB/s 2023-10-18T09:02:15.269 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T09:02:15.269 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:02:15.367 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:02:15.481 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T09:02:15.482 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:02:15.482 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:02:15.547 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:02:15.547 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:02:15.607 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:02:15.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.848+0000 7fdf6e834640 1 -- 172.21.15.40:0/1632925093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680b15b0 msgr2=0x7fdf680b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:15.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.848+0000 7fdf6e834640 1 --2- 172.21.15.40:0/1632925093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680b15b0 0x7fdf680b0630 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fdf6806aeb0 tx=0x7fdf6402bd60 comp rx=0 tx=0).stop 2023-10-18T09:02:15.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.848+0000 7fdf6e834640 1 -- 172.21.15.40:0/1632925093 shutdown_connections 2023-10-18T09:02:15.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.848+0000 7fdf6e834640 1 --2- 172.21.15.40:0/1632925093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf680ca0d0 0x7fdf68157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:15.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.848+0000 7fdf6e834640 1 --2- 172.21.15.40:0/1632925093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680b15b0 0x7fdf680b0630 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:15.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.848+0000 7fdf6e834640 1 -- 172.21.15.40:0/1632925093 >> 172.21.15.40:0/1632925093 conn(0x7fdf68069720 msgr2=0x7fdf68069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:15.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.848+0000 7fdf6e834640 1 -- 172.21.15.40:0/1632925093 shutdown_connections 2023-10-18T09:02:15.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.848+0000 7fdf6e834640 1 -- 172.21.15.40:0/1632925093 wait complete. 2023-10-18T09:02:15.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.850+0000 7fdf6e834640 1 Processor -- start 2023-10-18T09:02:15.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.850+0000 7fdf6e834640 1 -- start start 2023-10-18T09:02:15.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.852+0000 7fdf6e834640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf680b15b0 0x7fdf680accb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:15.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.852+0000 7fdf6e834640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf680b1e10 0x7fdf68149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:15.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.852+0000 7fdf6e834640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680ca0d0 0x7fdf681387d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:15.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.852+0000 7fdf6e834640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6814a620 con 0x7fdf680ca0d0 2023-10-18T09:02:15.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.852+0000 7fdf6e834640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6814a900 con 0x7fdf680b15b0 2023-10-18T09:02:15.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.852+0000 7fdf6e834640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6814abe0 con 0x7fdf680b1e10 2023-10-18T09:02:15.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.852+0000 7fdf6e033640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680ca0d0 0x7fdf681387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:15.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.852+0000 7fdf6e033640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680ca0d0 0x7fdf681387d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34168/0 (socket says 172.21.15.40:34168) 2023-10-18T09:02:15.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.852+0000 7fdf6e033640 1 -- 172.21.15.40:0/989058773 learned_addr learned my addr 172.21.15.40:0/989058773 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:15.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.853+0000 7fdf6d031640 1 --2- 172.21.15.40:0/989058773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf680b1e10 0x7fdf68149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:15.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.853+0000 7fdf6d832640 1 --2- 172.21.15.40:0/989058773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf680b15b0 0x7fdf680accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:15.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.853+0000 7fdf6e033640 1 -- 172.21.15.40:0/989058773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf680b1e10 msgr2=0x7fdf68149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:15.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.853+0000 7fdf6e033640 1 --2- 172.21.15.40:0/989058773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf680b1e10 0x7fdf68149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:15.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.853+0000 7fdf6e033640 1 -- 172.21.15.40:0/989058773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf680b15b0 msgr2=0x7fdf680accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:15.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.853+0000 7fdf6e033640 1 --2- 172.21.15.40:0/989058773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf680b15b0 0x7fdf680accb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:15.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.853+0000 7fdf6e033640 1 -- 172.21.15.40:0/989058773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf64009560 con 0x7fdf680ca0d0 2023-10-18T09:02:15.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.853+0000 7fdf6e033640 1 --2- 172.21.15.40:0/989058773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680ca0d0 0x7fdf681387d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000cad0 tx=0x7fdf6000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:15.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.853+0000 7fdf5e7fc640 1 -- 172.21.15.40:0/989058773 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6000ae20 con 0x7fdf680ca0d0 2023-10-18T09:02:15.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.854+0000 7fdf5e7fc640 1 -- 172.21.15.40:0/989058773 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdf6000f450 con 0x7fdf680ca0d0 2023-10-18T09:02:15.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.854+0000 7fdf5e7fc640 1 -- 172.21.15.40:0/989058773 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6000f8c0 con 0x7fdf680ca0d0 2023-10-18T09:02:15.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.854+0000 7fdf6e834640 1 -- 172.21.15.40:0/989058773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf6814aec0 con 0x7fdf680ca0d0 2023-10-18T09:02:15.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.854+0000 7fdf6e834640 1 -- 172.21.15.40:0/989058773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf68138d10 con 0x7fdf680ca0d0 2023-10-18T09:02:15.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.855+0000 7fdf6e834640 1 -- 172.21.15.40:0/989058773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf68066e00 con 0x7fdf680ca0d0 2023-10-18T09:02:15.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.856+0000 7fdf5e7fc640 1 -- 172.21.15.40:0/989058773 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdf6000fa20 con 0x7fdf680ca0d0 2023-10-18T09:02:15.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.857+0000 7fdf5e7fc640 1 --2- 172.21.15.40:0/989058773 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf4c06ae40 0x7fdf4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.857+0000 7fdf5e7fc640 1 -- 172.21.15.40:0/989058773 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fdf6008a330 con 0x7fdf680ca0d0 2023-10-18T09:02:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.858+0000 7fdf6d832640 1 --2- 172.21.15.40:0/989058773 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf4c06ae40 0x7fdf4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:15.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.859+0000 7fdf6d832640 1 --2- 172.21.15.40:0/989058773 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf4c06ae40 0x7fdf4c06d300 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdf64035000 tx=0x7fdf64004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:15.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:15.861+0000 7fdf5e7fc640 1 -- 172.21.15.40:0/989058773 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf60058d20 con 0x7fdf680ca0d0 2023-10-18T09:02:16.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.055+0000 7fdf6e834640 1 -- 172.21.15.40:0/989058773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf6815bed0 con 0x7fdf680ca0d0 2023-10-18T09:02:16.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.057+0000 7fdf5e7fc640 1 -- 172.21.15.40:0/989058773 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16873 (secure 0 0 0) 0x7fdf600588b0 con 0x7fdf680ca0d0 2023-10-18T09:02:16.060 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:16.061 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":118,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:02:09.285724+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:01:35.453236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-18T09:01:36.012460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1","mappings":[{"from":2,"to":7}]},{"pgid":"13.14","mappings":[{"from":6,"to":5}]},{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.20","mappings":[{"from":1,"to":5}]},{"pgid":"13.34","mappings":[{"from":6,"to":5}]},{"pgid":"13.38","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:02:16.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.059+0000 7fdf6e834640 1 -- 172.21.15.40:0/989058773 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf4c06ae40 msgr2=0x7fdf4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:16.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.059+0000 7fdf6e834640 1 --2- 172.21.15.40:0/989058773 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf4c06ae40 0x7fdf4c06d300 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdf64035000 tx=0x7fdf64004730 comp rx=0 tx=0).stop 2023-10-18T09:02:16.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.060+0000 7fdf6e834640 1 -- 172.21.15.40:0/989058773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680ca0d0 msgr2=0x7fdf681387d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:16.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.060+0000 7fdf6e834640 1 --2- 172.21.15.40:0/989058773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680ca0d0 0x7fdf681387d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000cad0 tx=0x7fdf6000cf90 comp rx=0 tx=0).stop 2023-10-18T09:02:16.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.060+0000 7fdf6e834640 1 -- 172.21.15.40:0/989058773 shutdown_connections 2023-10-18T09:02:16.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.060+0000 7fdf6e834640 1 --2- 172.21.15.40:0/989058773 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf4c06ae40 0x7fdf4c06d300 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:16.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.060+0000 7fdf6e834640 1 --2- 172.21.15.40:0/989058773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680ca0d0 0x7fdf681387d0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:16.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.060+0000 7fdf6e834640 1 --2- 172.21.15.40:0/989058773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf680b1e10 0x7fdf68149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:16.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.060+0000 7fdf6e834640 1 --2- 172.21.15.40:0/989058773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf680b15b0 0x7fdf680accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:16.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.060+0000 7fdf6e834640 1 -- 172.21.15.40:0/989058773 >> 172.21.15.40:0/989058773 conn(0x7fdf68069720 msgr2=0x7fdf68155b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:16.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.060+0000 7fdf6e834640 1 -- 172.21.15.40:0/989058773 shutdown_connections 2023-10-18T09:02:16.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.060+0000 7fdf6e834640 1 -- 172.21.15.40:0/989058773 wait complete. 2023-10-18T09:02:16.083 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:01:35.453236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:01:36.012460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:02:16.084 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:02:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.316+0000 7fc5ccde9640 1 -- 172.21.15.40:0/3186497504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5c80af410 msgr2=0x7fc5c814e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.316+0000 7fc5ccde9640 1 --2- 172.21.15.40:0/3186497504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5c80af410 0x7fc5c814e730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0009830 tx=0x7fc5b002bd50 comp rx=0 tx=0).stop 2023-10-18T09:02:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.316+0000 7fc5ccde9640 1 -- 172.21.15.40:0/3186497504 shutdown_connections 2023-10-18T09:02:16.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.316+0000 7fc5ccde9640 1 --2- 172.21.15.40:0/3186497504 >> v2:172.21.15.144:3300/0 conn(0x7fc5c80715c0 0x7fc5c80cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:16.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.316+0000 7fc5ccde9640 1 --2- 172.21.15.40:0/3186497504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5c80af410 0x7fc5c814e730 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:16.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.316+0000 7fc5ccde9640 1 -- 172.21.15.40:0/3186497504 >> 172.21.15.40:0/3186497504 conn(0x7fc5c8069720 msgr2=0x7fc5c8069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:16.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.316+0000 7fc5ccde9640 1 -- 172.21.15.40:0/3186497504 shutdown_connections 2023-10-18T09:02:16.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.316+0000 7fc5ccde9640 1 -- 172.21.15.40:0/3186497504 wait complete. 2023-10-18T09:02:16.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.319+0000 7fc5ccde9640 1 Processor -- start 2023-10-18T09:02:16.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.319+0000 7fc5ccde9640 1 -- start start 2023-10-18T09:02:16.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.320+0000 7fc5ccde9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5c806d6f0 0x7fc5c80cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:16.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.320+0000 7fc5ccde9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5c80715c0 0x7fc5c8163c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:16.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.320+0000 7fc5ccde9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c80af410 0x7fc5c81335c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:16.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.320+0000 7fc5ccde9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c8157750 con 0x7fc5c80715c0 2023-10-18T09:02:16.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.320+0000 7fc5ccde9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c8156b40 con 0x7fc5c80af410 2023-10-18T09:02:16.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.320+0000 7fc5ccde9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c8156e20 con 0x7fc5c806d6f0 2023-10-18T09:02:16.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.321+0000 7fc5c5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5c80715c0 0x7fc5c8163c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:16.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.321+0000 7fc5c5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5c80715c0 0x7fc5c8163c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34184/0 (socket says 172.21.15.40:34184) 2023-10-18T09:02:16.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.321+0000 7fc5c5d9b640 1 -- 172.21.15.40:0/3971791081 learned_addr learned my addr 172.21.15.40:0/3971791081 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:16.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.321+0000 7fc5c6d9d640 1 --2- 172.21.15.40:0/3971791081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c80af410 0x7fc5c81335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:16.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.321+0000 7fc5c659c640 1 --2- 172.21.15.40:0/3971791081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5c806d6f0 0x7fc5c80cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:16.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.321+0000 7fc5c5d9b640 1 -- 172.21.15.40:0/3971791081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5c806d6f0 msgr2=0x7fc5c80cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:16.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.321+0000 7fc5c5d9b640 1 --2- 172.21.15.40:0/3971791081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5c806d6f0 0x7fc5c80cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:16.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.321+0000 7fc5c5d9b640 1 -- 172.21.15.40:0/3971791081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c80af410 msgr2=0x7fc5c81335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:16.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.321+0000 7fc5c5d9b640 1 --2- 172.21.15.40:0/3971791081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c80af410 0x7fc5c81335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:16.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.321+0000 7fc5c5d9b640 1 -- 172.21.15.40:0/3971791081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c0009560 con 0x7fc5c80715c0 2023-10-18T09:02:16.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.322+0000 7fc5c5d9b640 1 --2- 172.21.15.40:0/3971791081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5c80715c0 0x7fc5c8163c40 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0000c00 tx=0x7fc5b0003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:16.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.322+0000 7fc5beffd640 1 -- 172.21.15.40:0/3971791081 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0003e00 con 0x7fc5c80715c0 2023-10-18T09:02:16.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.322+0000 7fc5beffd640 1 -- 172.21.15.40:0/3971791081 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc5b003e530 con 0x7fc5c80715c0 2023-10-18T09:02:16.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.322+0000 7fc5beffd640 1 -- 172.21.15.40:0/3971791081 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b003e920 con 0x7fc5c80715c0 2023-10-18T09:02:16.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.322+0000 7fc5ccde9640 1 -- 172.21.15.40:0/3971791081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b0009490 con 0x7fc5c80715c0 2023-10-18T09:02:16.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.322+0000 7fc5ccde9640 1 -- 172.21.15.40:0/3971791081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c81573e0 con 0x7fc5c80715c0 2023-10-18T09:02:16.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.324+0000 7fc5ccde9640 1 -- 172.21.15.40:0/3971791081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5c8157570 con 0x7fc5c80715c0 2023-10-18T09:02:16.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.325+0000 7fc5beffd640 1 -- 172.21.15.40:0/3971791081 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc5b003ea80 con 0x7fc5c80715c0 2023-10-18T09:02:16.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.326+0000 7fc5beffd640 1 --2- 172.21.15.40:0/3971791081 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc5a806ae40 0x7fc5a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:16.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.326+0000 7fc5beffd640 1 -- 172.21.15.40:0/3971791081 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fc5b00abda0 con 0x7fc5c80715c0 2023-10-18T09:02:16.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.326+0000 7fc5c659c640 1 --2- 172.21.15.40:0/3971791081 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc5a806ae40 0x7fc5a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:16.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.328+0000 7fc5c659c640 1 --2- 172.21.15.40:0/3971791081 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc5a806ae40 0x7fc5a806d300 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80ae780 tx=0x7fc5b8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:16.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.331+0000 7fc5beffd640 1 -- 172.21.15.40:0/3971791081 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc5b0076da0 con 0x7fc5c80715c0 2023-10-18T09:02:16.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.526+0000 7fc5ccde9640 1 -- 172.21.15.40:0/3971791081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5c8066fd0 con 0x7fc5c80715c0 2023-10-18T09:02:16.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.527+0000 7fc5beffd640 1 -- 172.21.15.40:0/3971791081 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5b00390a0 con 0x7fc5c80715c0 2023-10-18T09:02:16.531 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:02:16.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.530+0000 7fc5ccde9640 1 -- 172.21.15.40:0/3971791081 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc5a806ae40 msgr2=0x7fc5a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:16.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.530+0000 7fc5ccde9640 1 --2- 172.21.15.40:0/3971791081 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc5a806ae40 0x7fc5a806d300 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80ae780 tx=0x7fc5b8009040 comp rx=0 tx=0).stop 2023-10-18T09:02:16.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.530+0000 7fc5ccde9640 1 -- 172.21.15.40:0/3971791081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5c80715c0 msgr2=0x7fc5c8163c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:16.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.530+0000 7fc5ccde9640 1 --2- 172.21.15.40:0/3971791081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5c80715c0 0x7fc5c8163c40 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0000c00 tx=0x7fc5b0003800 comp rx=0 tx=0).stop 2023-10-18T09:02:16.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.530+0000 7fc5ccde9640 1 -- 172.21.15.40:0/3971791081 shutdown_connections 2023-10-18T09:02:16.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.530+0000 7fc5ccde9640 1 --2- 172.21.15.40:0/3971791081 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc5a806ae40 0x7fc5a806d300 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:16.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.530+0000 7fc5ccde9640 1 --2- 172.21.15.40:0/3971791081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5c80af410 0x7fc5c81335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:16.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.530+0000 7fc5ccde9640 1 --2- 172.21.15.40:0/3971791081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc5c80715c0 0x7fc5c8163c40 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:16.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.530+0000 7fc5ccde9640 1 --2- 172.21.15.40:0/3971791081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5c806d6f0 0x7fc5c80cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:16.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.530+0000 7fc5ccde9640 1 -- 172.21.15.40:0/3971791081 >> 172.21.15.40:0/3971791081 conn(0x7fc5c8069720 msgr2=0x7fc5c8069c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:16.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.531+0000 7fc5ccde9640 1 -- 172.21.15.40:0/3971791081 shutdown_connections 2023-10-18T09:02:16.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.531+0000 7fc5ccde9640 1 -- 172.21.15.40:0/3971791081 wait complete. 2023-10-18T09:02:16.555 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:02:16.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.790+0000 7f1fe73e2640 1 -- 172.21.15.40:0/2318282992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fe006d5d0 msgr2=0x7f1fe00c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:16.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.790+0000 7f1fe73e2640 1 --2- 172.21.15.40:0/2318282992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fe006d5d0 0x7f1fe00c9620 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8005fd0 tx=0x7f1fd802c060 comp rx=0 tx=0).stop 2023-10-18T09:02:16.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.790+0000 7f1fe73e2640 1 -- 172.21.15.40:0/2318282992 shutdown_connections 2023-10-18T09:02:16.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.790+0000 7f1fe73e2640 1 --2- 172.21.15.40:0/2318282992 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fe0071270 0x7f1fe0157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:16.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.790+0000 7f1fe73e2640 1 --2- 172.21.15.40:0/2318282992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fe006d5d0 0x7f1fe00c9620 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:16.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.790+0000 7f1fe73e2640 1 -- 172.21.15.40:0/2318282992 >> 172.21.15.40:0/2318282992 conn(0x7f1fe006ca10 msgr2=0x7f1fe006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:16.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.790+0000 7f1fe73e2640 1 -- 172.21.15.40:0/2318282992 shutdown_connections 2023-10-18T09:02:16.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.790+0000 7f1fe73e2640 1 -- 172.21.15.40:0/2318282992 wait complete. 2023-10-18T09:02:16.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.792+0000 7f1fe73e2640 1 Processor -- start 2023-10-18T09:02:16.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.792+0000 7f1fe73e2640 1 -- start start 2023-10-18T09:02:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.793+0000 7f1fe73e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fe006d5d0 0x7f1fe0144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.793+0000 7f1fe73e2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe0071270 0x7f1fe0155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.793+0000 7f1fe73e2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fe015af70 0x7f1fe00afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.793+0000 7f1fe73e2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe0156860 con 0x7f1fe006d5d0 2023-10-18T09:02:16.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.793+0000 7f1fe73e2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe0156b40 con 0x7f1fe0071270 2023-10-18T09:02:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.793+0000 7f1fe73e2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe0156e20 con 0x7f1fe015af70 2023-10-18T09:02:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.793+0000 7f1fe63e0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fe006d5d0 0x7f1fe0144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.793+0000 7f1fe63e0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fe006d5d0 0x7f1fe0144a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34208/0 (socket says 172.21.15.40:34208) 2023-10-18T09:02:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.793+0000 7f1fe63e0640 1 -- 172.21.15.40:0/3383991226 learned_addr learned my addr 172.21.15.40:0/3383991226 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.794+0000 7f1fe63e0640 1 -- 172.21.15.40:0/3383991226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fe015af70 msgr2=0x7f1fe00afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:16.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.794+0000 7f1fe63e0640 1 --2- 172.21.15.40:0/3383991226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fe015af70 0x7f1fe00afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.794+0000 7f1fe63e0640 1 -- 172.21.15.40:0/3383991226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe0071270 msgr2=0x7f1fe0155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.794+0000 7f1fe63e0640 1 --2- 172.21.15.40:0/3383991226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe0071270 0x7f1fe0155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:16.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.794+0000 7f1fe63e0640 1 -- 172.21.15.40:0/3383991226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd8009840 con 0x7f1fe006d5d0 2023-10-18T09:02:16.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.794+0000 7f1fe63e0640 1 --2- 172.21.15.40:0/3383991226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fe006d5d0 0x7f1fe0144a80 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8036040 tx=0x7f1fd8009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:16.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.794+0000 7f1fdeffd640 1 -- 172.21.15.40:0/3383991226 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd803d430 con 0x7f1fe006d5d0 2023-10-18T09:02:16.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.794+0000 7f1fdeffd640 1 -- 172.21.15.40:0/3383991226 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1fd803d6c0 con 0x7f1fe006d5d0 2023-10-18T09:02:16.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.795+0000 7f1fe73e2640 1 -- 172.21.15.40:0/3383991226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe0157100 con 0x7f1fe006d5d0 2023-10-18T09:02:16.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.795+0000 7f1fdeffd640 1 -- 172.21.15.40:0/3383991226 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd803dad0 con 0x7f1fe006d5d0 2023-10-18T09:02:16.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.795+0000 7f1fe73e2640 1 -- 172.21.15.40:0/3383991226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe0152880 con 0x7f1fe006d5d0 2023-10-18T09:02:16.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.796+0000 7f1fe73e2640 1 -- 172.21.15.40:0/3383991226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe0157750 con 0x7f1fe006d5d0 2023-10-18T09:02:16.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.797+0000 7f1fdeffd640 1 -- 172.21.15.40:0/3383991226 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1fd803dc30 con 0x7f1fe006d5d0 2023-10-18T09:02:16.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.798+0000 7f1fdeffd640 1 --2- 172.21.15.40:0/3383991226 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1fcc06ae40 0x7f1fcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:16.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.798+0000 7f1fdeffd640 1 -- 172.21.15.40:0/3383991226 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f1fd80ac600 con 0x7f1fe006d5d0 2023-10-18T09:02:16.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.799+0000 7f1fe5bdf640 1 --2- 172.21.15.40:0/3383991226 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1fcc06ae40 0x7f1fcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:16.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.800+0000 7f1fe5bdf640 1 --2- 172.21.15.40:0/3383991226 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1fcc06ae40 0x7f1fcc06d300 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1fe014e300 tx=0x7f1fd4007930 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:16.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.802+0000 7f1fdeffd640 1 -- 172.21.15.40:0/3383991226 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1fd8077600 con 0x7f1fe006d5d0 2023-10-18T09:02:16.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.993+0000 7f1fe73e2640 1 -- 172.21.15.40:0/3383991226 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1fe0157290 con 0x7f1fe006d5d0 2023-10-18T09:02:16.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.994+0000 7f1fdeffd640 1 -- 172.21.15.40:0/3383991226 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1fd807ac20 con 0x7f1fe006d5d0 2023-10-18T09:02:16.998 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:02:17.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.997+0000 7f1fe73e2640 1 -- 172.21.15.40:0/3383991226 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1fcc06ae40 msgr2=0x7f1fcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:17.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.997+0000 7f1fe73e2640 1 --2- 172.21.15.40:0/3383991226 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1fcc06ae40 0x7f1fcc06d300 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1fe014e300 tx=0x7f1fd4007930 comp rx=0 tx=0).stop 2023-10-18T09:02:17.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.997+0000 7f1fe73e2640 1 -- 172.21.15.40:0/3383991226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fe006d5d0 msgr2=0x7f1fe0144a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:17.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.997+0000 7f1fe73e2640 1 --2- 172.21.15.40:0/3383991226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fe006d5d0 0x7f1fe0144a80 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8036040 tx=0x7f1fd8009330 comp rx=0 tx=0).stop 2023-10-18T09:02:17.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.997+0000 7f1fe73e2640 1 -- 172.21.15.40:0/3383991226 shutdown_connections 2023-10-18T09:02:17.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.997+0000 7f1fe73e2640 1 --2- 172.21.15.40:0/3383991226 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1fcc06ae40 0x7f1fcc06d300 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.997+0000 7f1fe73e2640 1 --2- 172.21.15.40:0/3383991226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fe015af70 0x7f1fe00afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.997+0000 7f1fe73e2640 1 --2- 172.21.15.40:0/3383991226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe0071270 0x7f1fe0155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.997+0000 7f1fe73e2640 1 --2- 172.21.15.40:0/3383991226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fe006d5d0 0x7f1fe0144a80 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.997+0000 7f1fe73e2640 1 -- 172.21.15.40:0/3383991226 >> 172.21.15.40:0/3383991226 conn(0x7f1fe006ca10 msgr2=0x7f1fe00ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:17.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.997+0000 7f1fe73e2640 1 -- 172.21.15.40:0/3383991226 shutdown_connections 2023-10-18T09:02:17.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:16.997+0000 7f1fe73e2640 1 -- 172.21.15.40:0/3383991226 wait complete. 2023-10-18T09:02:17.018 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:02:17.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.244+0000 7fa059733640 1 -- 172.21.15.40:0/1788541404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040005380 msgr2=0x7fa04000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:17.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.244+0000 7fa059733640 1 --2- 172.21.15.40:0/1788541404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040005380 0x7fa04000b980 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa04c006f00 tx=0x7fa0400057f0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.245+0000 7fa059733640 1 -- 172.21.15.40:0/1788541404 shutdown_connections 2023-10-18T09:02:17.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.245+0000 7fa059733640 1 --2- 172.21.15.40:0/1788541404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040005380 0x7fa04000b980 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.245+0000 7fa059733640 1 -- 172.21.15.40:0/1788541404 >> 172.21.15.40:0/1788541404 conn(0x7fa054069720 msgr2=0x7fa054069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:17.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.245+0000 7fa059733640 1 -- 172.21.15.40:0/1788541404 shutdown_connections 2023-10-18T09:02:17.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.245+0000 7fa059733640 1 -- 172.21.15.40:0/1788541404 wait complete. 2023-10-18T09:02:17.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.246+0000 7fa059733640 1 Processor -- start 2023-10-18T09:02:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.247+0000 7fa059733640 1 -- start start 2023-10-18T09:02:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.247+0000 7fa059733640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040005380 0x7fa05414e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.248+0000 7fa059733640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa05414a860 0x7fa05414ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:17.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.248+0000 7fa059733640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa054146fb0 0x7fa054147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:17.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.248+0000 7fa059733640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa054152740 con 0x7fa040005380 2023-10-18T09:02:17.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.248+0000 7fa059733640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa054147970 con 0x7fa05414a860 2023-10-18T09:02:17.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.248+0000 7fa059733640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa054147df0 con 0x7fa054146fb0 2023-10-18T09:02:17.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.248+0000 7fa052ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040005380 0x7fa05414e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:17.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.248+0000 7fa052ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040005380 0x7fa05414e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34216/0 (socket says 172.21.15.40:34216) 2023-10-18T09:02:17.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.248+0000 7fa052ffd640 1 -- 172.21.15.40:0/4178837520 learned_addr learned my addr 172.21.15.40:0/4178837520 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:17.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.248+0000 7fa0537fe640 1 --2- 172.21.15.40:0/4178837520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa054146fb0 0x7fa054147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:17.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.248+0000 7fa04a7fc640 1 --2- 172.21.15.40:0/4178837520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa05414a860 0x7fa05414ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:17.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.248+0000 7fa052ffd640 1 -- 172.21.15.40:0/4178837520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa054146fb0 msgr2=0x7fa054147430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:17.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.248+0000 7fa052ffd640 1 --2- 172.21.15.40:0/4178837520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa054146fb0 0x7fa054147430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.248+0000 7fa052ffd640 1 -- 172.21.15.40:0/4178837520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa05414a860 msgr2=0x7fa05414ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.248+0000 7fa052ffd640 1 --2- 172.21.15.40:0/4178837520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa05414a860 0x7fa05414ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.249+0000 7fa052ffd640 1 -- 172.21.15.40:0/4178837520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05415bd70 con 0x7fa040005380 2023-10-18T09:02:17.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.249+0000 7fa052ffd640 1 --2- 172.21.15.40:0/4178837520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040005380 0x7fa05414e810 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa04400c480 tx=0x7fa04400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.249+0000 7fa04bfff640 1 -- 172.21.15.40:0/4178837520 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0440052d0 con 0x7fa040005380 2023-10-18T09:02:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.249+0000 7fa04bfff640 1 -- 172.21.15.40:0/4178837520 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa044005450 con 0x7fa040005380 2023-10-18T09:02:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.249+0000 7fa059733640 1 -- 172.21.15.40:0/4178837520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa054156a30 con 0x7fa040005380 2023-10-18T09:02:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.249+0000 7fa04bfff640 1 -- 172.21.15.40:0/4178837520 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044012890 con 0x7fa040005380 2023-10-18T09:02:17.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.249+0000 7fa059733640 1 -- 172.21.15.40:0/4178837520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa054156ff0 con 0x7fa040005380 2023-10-18T09:02:17.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.251+0000 7fa059733640 1 -- 172.21.15.40:0/4178837520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa054066fd0 con 0x7fa040005380 2023-10-18T09:02:17.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.251+0000 7fa04bfff640 1 -- 172.21.15.40:0/4178837520 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa0440035f0 con 0x7fa040005380 2023-10-18T09:02:17.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.252+0000 7fa04bfff640 1 --2- 172.21.15.40:0/4178837520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa03406ae40 0x7fa03406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:17.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.252+0000 7fa04bfff640 1 -- 172.21.15.40:0/4178837520 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fa0440894a0 con 0x7fa040005380 2023-10-18T09:02:17.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.253+0000 7fa04a7fc640 1 --2- 172.21.15.40:0/4178837520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa03406ae40 0x7fa03406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:17.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.255+0000 7fa04a7fc640 1 --2- 172.21.15.40:0/4178837520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa03406ae40 0x7fa03406d300 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa054155b70 tx=0x7fa03c009410 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:17.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.257+0000 7fa04bfff640 1 -- 172.21.15.40:0/4178837520 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa044054410 con 0x7fa040005380 2023-10-18T09:02:17.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.453+0000 7fa059733640 1 -- 172.21.15.40:0/4178837520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0540cc810 con 0x7fa040005380 2023-10-18T09:02:17.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.454+0000 7fa04bfff640 1 -- 172.21.15.40:0/4178837520 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa044057a30 con 0x7fa040005380 2023-10-18T09:02:17.457 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:02:17.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.456+0000 7fa059733640 1 -- 172.21.15.40:0/4178837520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa03406ae40 msgr2=0x7fa03406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:17.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.456+0000 7fa059733640 1 --2- 172.21.15.40:0/4178837520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa03406ae40 0x7fa03406d300 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa054155b70 tx=0x7fa03c009410 comp rx=0 tx=0).stop 2023-10-18T09:02:17.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.456+0000 7fa059733640 1 -- 172.21.15.40:0/4178837520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040005380 msgr2=0x7fa05414e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:17.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.456+0000 7fa059733640 1 --2- 172.21.15.40:0/4178837520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040005380 0x7fa05414e810 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa04400c480 tx=0x7fa04400c940 comp rx=0 tx=0).stop 2023-10-18T09:02:17.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.456+0000 7fa059733640 1 -- 172.21.15.40:0/4178837520 shutdown_connections 2023-10-18T09:02:17.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.456+0000 7fa059733640 1 --2- 172.21.15.40:0/4178837520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa03406ae40 0x7fa03406d300 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.456+0000 7fa059733640 1 --2- 172.21.15.40:0/4178837520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa054146fb0 0x7fa054147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.456+0000 7fa059733640 1 --2- 172.21.15.40:0/4178837520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa05414a860 0x7fa05414ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.456+0000 7fa059733640 1 --2- 172.21.15.40:0/4178837520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa040005380 0x7fa05414e810 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.456+0000 7fa059733640 1 -- 172.21.15.40:0/4178837520 >> 172.21.15.40:0/4178837520 conn(0x7fa054069720 msgr2=0x7fa054070fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:17.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.456+0000 7fa059733640 1 -- 172.21.15.40:0/4178837520 shutdown_connections 2023-10-18T09:02:17.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.456+0000 7fa059733640 1 -- 172.21.15.40:0/4178837520 wait complete. 2023-10-18T09:02:17.479 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:02:17.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.713+0000 7fef99a67640 1 -- 172.21.15.40:0/584373669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef840047c0 msgr2=0x7fef84004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:17.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.713+0000 7fef99a67640 1 --2- 172.21.15.40:0/584373669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef840047c0 0x7fef84004c40 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fef8000c6e0 tx=0x7fef840057f0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.713+0000 7fef99a67640 1 -- 172.21.15.40:0/584373669 shutdown_connections 2023-10-18T09:02:17.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.713+0000 7fef99a67640 1 --2- 172.21.15.40:0/584373669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef840047c0 0x7fef84004c40 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.713+0000 7fef99a67640 1 -- 172.21.15.40:0/584373669 >> 172.21.15.40:0/584373669 conn(0x7fef94069720 msgr2=0x7fef94069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:17.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.713+0000 7fef99a67640 1 -- 172.21.15.40:0/584373669 shutdown_connections 2023-10-18T09:02:17.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.713+0000 7fef99a67640 1 -- 172.21.15.40:0/584373669 wait complete. 2023-10-18T09:02:17.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.715+0000 7fef99a67640 1 Processor -- start 2023-10-18T09:02:17.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.715+0000 7fef99a67640 1 -- start start 2023-10-18T09:02:17.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.716+0000 7fef99a67640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef840047c0 0x7fef94155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:17.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.716+0000 7fef99a67640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef941596f0 0x7fef94157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.716+0000 7fef99a67640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef94147820 0x7fef9414e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.716+0000 7fef99a67640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef941592d0 con 0x7fef941596f0 2023-10-18T09:02:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.716+0000 7fef99a67640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef941583f0 con 0x7fef94147820 2023-10-18T09:02:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.716+0000 7fef99a67640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef94159b70 con 0x7fef840047c0 2023-10-18T09:02:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.717+0000 7fef93fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef941596f0 0x7fef94157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:17.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.717+0000 7fef93fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef941596f0 0x7fef94157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34238/0 (socket says 172.21.15.40:34238) 2023-10-18T09:02:17.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.717+0000 7fef93fff640 1 -- 172.21.15.40:0/1436692532 learned_addr learned my addr 172.21.15.40:0/1436692532 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:17.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.717+0000 7fef98a65640 1 --2- 172.21.15.40:0/1436692532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef840047c0 0x7fef94155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:17.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.717+0000 7fef99266640 1 --2- 172.21.15.40:0/1436692532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef94147820 0x7fef9414e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:17.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.717+0000 7fef93fff640 1 -- 172.21.15.40:0/1436692532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef840047c0 msgr2=0x7fef94155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:17.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.717+0000 7fef93fff640 1 --2- 172.21.15.40:0/1436692532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef840047c0 0x7fef94155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.717+0000 7fef93fff640 1 -- 172.21.15.40:0/1436692532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef94147820 msgr2=0x7fef9414e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:17.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.717+0000 7fef93fff640 1 --2- 172.21.15.40:0/1436692532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef94147820 0x7fef9414e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.717+0000 7fef93fff640 1 -- 172.21.15.40:0/1436692532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef94158650 con 0x7fef941596f0 2023-10-18T09:02:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.717+0000 7fef99266640 1 --2- 172.21.15.40:0/1436692532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef94147820 0x7fef9414e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.718+0000 7fef93fff640 1 --2- 172.21.15.40:0/1436692532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef941596f0 0x7fef94157eb0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fef8000c630 tx=0x7fef80004e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.718+0000 7fef917fa640 1 -- 172.21.15.40:0/1436692532 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8003d070 con 0x7fef941596f0 2023-10-18T09:02:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.718+0000 7fef917fa640 1 -- 172.21.15.40:0/1436692532 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fef80006260 con 0x7fef941596f0 2023-10-18T09:02:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.718+0000 7fef99a67640 1 -- 172.21.15.40:0/1436692532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef94158930 con 0x7fef941596f0 2023-10-18T09:02:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.718+0000 7fef917fa640 1 -- 172.21.15.40:0/1436692532 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80037690 con 0x7fef941596f0 2023-10-18T09:02:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.718+0000 7fef99a67640 1 -- 172.21.15.40:0/1436692532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef94158ef0 con 0x7fef941596f0 2023-10-18T09:02:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.720+0000 7fef99a67640 1 -- 172.21.15.40:0/1436692532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef94066e00 con 0x7fef941596f0 2023-10-18T09:02:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.720+0000 7fef917fa640 1 -- 172.21.15.40:0/1436692532 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fef800377f0 con 0x7fef941596f0 2023-10-18T09:02:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.722+0000 7fef917fa640 1 --2- 172.21.15.40:0/1436692532 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fef7806ae40 0x7fef7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.722+0000 7fef917fa640 1 -- 172.21.15.40:0/1436692532 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fef800ab120 con 0x7fef941596f0 2023-10-18T09:02:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.723+0000 7fef98a65640 1 --2- 172.21.15.40:0/1436692532 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fef7806ae40 0x7fef7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.724+0000 7fef98a65640 1 --2- 172.21.15.40:0/1436692532 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fef7806ae40 0x7fef7806d300 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fef880017a0 tx=0x7fef88009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.726+0000 7fef917fa640 1 -- 172.21.15.40:0/1436692532 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef80076110 con 0x7fef941596f0 2023-10-18T09:02:17.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.932+0000 7fef99a67640 1 -- 172.21.15.40:0/1436692532 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fef94152b20 con 0x7fef941596f0 2023-10-18T09:02:17.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.933+0000 7fef917fa640 1 -- 172.21.15.40:0/1436692532 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v118) v1 ==== 80+0+0 (secure 0 0 0) 0x7fef80032020 con 0x7fef941596f0 2023-10-18T09:02:17.936 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T09:02:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.935+0000 7fef99a67640 1 -- 172.21.15.40:0/1436692532 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fef7806ae40 msgr2=0x7fef7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.935+0000 7fef99a67640 1 --2- 172.21.15.40:0/1436692532 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fef7806ae40 0x7fef7806d300 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fef880017a0 tx=0x7fef88009040 comp rx=0 tx=0).stop 2023-10-18T09:02:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.935+0000 7fef99a67640 1 -- 172.21.15.40:0/1436692532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef941596f0 msgr2=0x7fef94157eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.935+0000 7fef99a67640 1 --2- 172.21.15.40:0/1436692532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef941596f0 0x7fef94157eb0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fef8000c630 tx=0x7fef80004e10 comp rx=0 tx=0).stop 2023-10-18T09:02:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.936+0000 7fef99a67640 1 -- 172.21.15.40:0/1436692532 shutdown_connections 2023-10-18T09:02:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.936+0000 7fef99a67640 1 --2- 172.21.15.40:0/1436692532 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fef7806ae40 0x7fef7806d300 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.936+0000 7fef99a67640 1 --2- 172.21.15.40:0/1436692532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef94147820 0x7fef9414e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.936+0000 7fef99a67640 1 --2- 172.21.15.40:0/1436692532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef941596f0 0x7fef94157eb0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.936+0000 7fef99a67640 1 --2- 172.21.15.40:0/1436692532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef840047c0 0x7fef94155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:17.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.936+0000 7fef99a67640 1 -- 172.21.15.40:0/1436692532 >> 172.21.15.40:0/1436692532 conn(0x7fef94069720 msgr2=0x7fef940cb920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:17.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.936+0000 7fef99a67640 1 -- 172.21.15.40:0/1436692532 shutdown_connections 2023-10-18T09:02:17.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:17.936+0000 7fef99a67640 1 -- 172.21.15.40:0/1436692532 wait complete. 2023-10-18T09:02:17.956 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-18T09:02:17.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:17.957 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-18T09:02:18.119 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T09:02:18.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:18.120 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T09:02:18.168 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:02:18.168 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:02:18.211 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:02:18.212 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:02:18.307 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:02:18.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.547+0000 7fa21bfff640 1 -- 172.21.15.40:0/3426987129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa204004d00 msgr2=0x7fa204005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:18.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.547+0000 7fa21bfff640 1 --2- 172.21.15.40:0/3426987129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa204004d00 0x7fa204005180 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0b26c0 tx=0x7fa2040057f0 comp rx=0 tx=0).stop 2023-10-18T09:02:18.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.547+0000 7fa21b7fe640 1 -- 172.21.15.40:0/3426987129 reap_dead start 2023-10-18T09:02:18.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.548+0000 7fa21bfff640 1 -- 172.21.15.40:0/3426987129 shutdown_connections 2023-10-18T09:02:18.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.548+0000 7fa21bfff640 1 -- 172.21.15.40:0/3426987129 >> 172.21.15.40:0/3426987129 conn(0x7fa21c069720 msgr2=0x7fa21c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:18.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.548+0000 7fa21bfff640 1 -- 172.21.15.40:0/3426987129 shutdown_connections 2023-10-18T09:02:18.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.548+0000 7fa21bfff640 1 -- 172.21.15.40:0/3426987129 wait complete. 2023-10-18T09:02:18.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.549+0000 7fa21bfff640 1 Processor -- start 2023-10-18T09:02:18.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.550+0000 7fa21bfff640 1 -- start start 2023-10-18T09:02:18.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.551+0000 7fa21bfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa204004d00 0x7fa21c13f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:18.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.551+0000 7fa21bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21c1400f0 0x7fa21c150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:18.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.551+0000 7fa21bfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa21c140570 0x7fa21c0aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:18.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.551+0000 7fa21bfff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c152a40 con 0x7fa21c1400f0 2023-10-18T09:02:18.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.551+0000 7fa21bfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c151e70 con 0x7fa204004d00 2023-10-18T09:02:18.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.551+0000 7fa21bfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c152110 con 0x7fa21c140570 2023-10-18T09:02:18.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.551+0000 7fa21a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21c1400f0 0x7fa21c150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:18.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.551+0000 7fa21a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21c1400f0 0x7fa21c150ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34260/0 (socket says 172.21.15.40:34260) 2023-10-18T09:02:18.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.551+0000 7fa21a7fc640 1 -- 172.21.15.40:0/1452969704 learned_addr learned my addr 172.21.15.40:0/1452969704 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:18.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.551+0000 7fa21affd640 1 --2- 172.21.15.40:0/1452969704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa204004d00 0x7fa21c13f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:18.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.552+0000 7fa21b7fe640 1 --2- 172.21.15.40:0/1452969704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa21c140570 0x7fa21c0aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:18.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.552+0000 7fa21a7fc640 1 -- 172.21.15.40:0/1452969704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa21c140570 msgr2=0x7fa21c0aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:18.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.552+0000 7fa21a7fc640 1 --2- 172.21.15.40:0/1452969704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa21c140570 0x7fa21c0aecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:18.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.552+0000 7fa21a7fc640 1 -- 172.21.15.40:0/1452969704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa204004d00 msgr2=0x7fa21c13f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:18.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.552+0000 7fa21a7fc640 1 --2- 172.21.15.40:0/1452969704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa204004d00 0x7fa21c13f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:18.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.552+0000 7fa21a7fc640 1 -- 172.21.15.40:0/1452969704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c1523f0 con 0x7fa21c1400f0 2023-10-18T09:02:18.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.552+0000 7fa21a7fc640 1 --2- 172.21.15.40:0/1452969704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21c1400f0 0x7fa21c150ef0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa21000c500 tx=0x7fa21000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:18.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.552+0000 7fa2037fe640 1 -- 172.21.15.40:0/1452969704 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2100052d0 con 0x7fa21c1400f0 2023-10-18T09:02:18.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.552+0000 7fa2037fe640 1 -- 172.21.15.40:0/1452969704 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa210005430 con 0x7fa21c1400f0 2023-10-18T09:02:18.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.552+0000 7fa2037fe640 1 -- 172.21.15.40:0/1452969704 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210012890 con 0x7fa21c1400f0 2023-10-18T09:02:18.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.552+0000 7fa21bfff640 1 -- 172.21.15.40:0/1452969704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c1526d0 con 0x7fa21c1400f0 2023-10-18T09:02:18.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.553+0000 7fa21bfff640 1 -- 172.21.15.40:0/1452969704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c14dea0 con 0x7fa21c1400f0 2023-10-18T09:02:18.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.554+0000 7fa21bfff640 1 -- 172.21.15.40:0/1452969704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21c152bb0 con 0x7fa21c1400f0 2023-10-18T09:02:18.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.555+0000 7fa2037fe640 1 -- 172.21.15.40:0/1452969704 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa2100129f0 con 0x7fa21c1400f0 2023-10-18T09:02:18.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.556+0000 7fa2037fe640 1 --2- 172.21.15.40:0/1452969704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa20806ae70 0x7fa20806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:18.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.556+0000 7fa2037fe640 1 -- 172.21.15.40:0/1452969704 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fa210089460 con 0x7fa21c1400f0 2023-10-18T09:02:18.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.556+0000 7fa21affd640 1 --2- 172.21.15.40:0/1452969704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa20806ae70 0x7fa20806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:18.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.557+0000 7fa21affd640 1 --2- 172.21.15.40:0/1452969704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa20806ae70 0x7fa20806d330 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa20c005fd0 tx=0x7fa20c00a4d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:18.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.560+0000 7fa2037fe640 1 -- 172.21.15.40:0/1452969704 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa210054450 con 0x7fa21c1400f0 2023-10-18T09:02:18.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.753+0000 7fa21bfff640 1 -- 172.21.15.40:0/1452969704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa21c066e00 con 0x7fa21c1400f0 2023-10-18T09:02:18.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.755+0000 7fa2037fe640 1 -- 172.21.15.40:0/1452969704 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16873 (secure 0 0 0) 0x7fa210057a70 con 0x7fa21c1400f0 2023-10-18T09:02:18.759 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:18.759 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":118,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:02:09.285724+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:01:35.453236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-18T09:01:36.012460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1","mappings":[{"from":2,"to":7}]},{"pgid":"13.14","mappings":[{"from":6,"to":5}]},{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.20","mappings":[{"from":1,"to":5}]},{"pgid":"13.34","mappings":[{"from":6,"to":5}]},{"pgid":"13.38","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:02:18.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.758+0000 7fa21bfff640 1 -- 172.21.15.40:0/1452969704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa20806ae70 msgr2=0x7fa20806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:18.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.758+0000 7fa21bfff640 1 --2- 172.21.15.40:0/1452969704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa20806ae70 0x7fa20806d330 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa20c005fd0 tx=0x7fa20c00a4d0 comp rx=0 tx=0).stop 2023-10-18T09:02:18.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.758+0000 7fa21bfff640 1 -- 172.21.15.40:0/1452969704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21c1400f0 msgr2=0x7fa21c150ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:18.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.758+0000 7fa21bfff640 1 --2- 172.21.15.40:0/1452969704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21c1400f0 0x7fa21c150ef0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fa21000c500 tx=0x7fa21000c9c0 comp rx=0 tx=0).stop 2023-10-18T09:02:18.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.758+0000 7fa21bfff640 1 -- 172.21.15.40:0/1452969704 shutdown_connections 2023-10-18T09:02:18.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.758+0000 7fa21bfff640 1 --2- 172.21.15.40:0/1452969704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa20806ae70 0x7fa20806d330 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:18.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.758+0000 7fa21bfff640 1 --2- 172.21.15.40:0/1452969704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa21c140570 0x7fa21c0aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:18.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.758+0000 7fa21bfff640 1 --2- 172.21.15.40:0/1452969704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa21c1400f0 0x7fa21c150ef0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:18.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.758+0000 7fa21bfff640 1 --2- 172.21.15.40:0/1452969704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa204004d00 0x7fa21c13f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:18.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.758+0000 7fa21bfff640 1 -- 172.21.15.40:0/1452969704 >> 172.21.15.40:0/1452969704 conn(0x7fa21c069720 msgr2=0x7fa21c15a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:18.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.759+0000 7fa21bfff640 1 -- 172.21.15.40:0/1452969704 shutdown_connections 2023-10-18T09:02:18.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:18.759+0000 7fa21bfff640 1 -- 172.21.15.40:0/1452969704 wait complete. 2023-10-18T09:02:18.780 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:01:35.453236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:01:36.012460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:02:18.780 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:02:19.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.010+0000 7fcf6a2f2640 1 -- 172.21.15.40:0/374469547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6414e730 msgr2=0x7fcf640b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.010+0000 7fcf6a2f2640 1 --2- 172.21.15.40:0/374469547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6414e730 0x7fcf640b1950 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fcf64071be0 tx=0x7fcf5402bd80 comp rx=0 tx=0).stop 2023-10-18T09:02:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.010+0000 7fcf6a2f2640 1 -- 172.21.15.40:0/374469547 shutdown_connections 2023-10-18T09:02:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.010+0000 7fcf6a2f2640 1 --2- 172.21.15.40:0/374469547 >> v2:172.21.15.144:3300/0 conn(0x7fcf640af240 0x7fcf640cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.010+0000 7fcf6a2f2640 1 --2- 172.21.15.40:0/374469547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf640715c0 0x7fcf640aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.010+0000 7fcf6a2f2640 1 --2- 172.21.15.40:0/374469547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6414e730 0x7fcf640b1950 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.010+0000 7fcf6a2f2640 1 -- 172.21.15.40:0/374469547 >> 172.21.15.40:0/374469547 conn(0x7fcf64069720 msgr2=0x7fcf64069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:19.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.010+0000 7fcf6a2f2640 1 -- 172.21.15.40:0/374469547 shutdown_connections 2023-10-18T09:02:19.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.010+0000 7fcf6a2f2640 1 -- 172.21.15.40:0/374469547 wait complete. 2023-10-18T09:02:19.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.012+0000 7fcf6a2f2640 1 Processor -- start 2023-10-18T09:02:19.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.013+0000 7fcf6a2f2640 1 -- start start 2023-10-18T09:02:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.014+0000 7fcf6a2f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf640715c0 0x7fcf64140af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.014+0000 7fcf6a2f2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf640af240 0x7fcf641567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:19.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.014+0000 7fcf6a2f2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf6414e730 0x7fcf64156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:19.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.014+0000 7fcf6a2f2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf64150ea0 con 0x7fcf640715c0 2023-10-18T09:02:19.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.014+0000 7fcf6a2f2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf641508e0 con 0x7fcf6414e730 2023-10-18T09:02:19.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.014+0000 7fcf6a2f2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf64150bc0 con 0x7fcf640af240 2023-10-18T09:02:19.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.014+0000 7fcf63fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf640715c0 0x7fcf64140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:19.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.014+0000 7fcf63fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf640715c0 0x7fcf64140af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34286/0 (socket says 172.21.15.40:34286) 2023-10-18T09:02:19.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.014+0000 7fcf63fff640 1 -- 172.21.15.40:0/289241639 learned_addr learned my addr 172.21.15.40:0/289241639 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:19.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.015+0000 7fcf63fff640 1 -- 172.21.15.40:0/289241639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf640af240 msgr2=0x7fcf641567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:19.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.015+0000 7fcf5bfff640 1 --2- 172.21.15.40:0/289241639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf640af240 0x7fcf641567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:19.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.015+0000 7fcf6888f640 1 --2- 172.21.15.40:0/289241639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf6414e730 0x7fcf64156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:19.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.015+0000 7fcf63fff640 1 --2- 172.21.15.40:0/289241639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf640af240 0x7fcf641567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.015+0000 7fcf63fff640 1 -- 172.21.15.40:0/289241639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf6414e730 msgr2=0x7fcf64156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:19.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.015+0000 7fcf63fff640 1 --2- 172.21.15.40:0/289241639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf6414e730 0x7fcf64156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.015+0000 7fcf63fff640 1 -- 172.21.15.40:0/289241639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf54009df0 con 0x7fcf640715c0 2023-10-18T09:02:19.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.015+0000 7fcf6888f640 1 --2- 172.21.15.40:0/289241639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf6414e730 0x7fcf64156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:19.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.015+0000 7fcf63fff640 1 --2- 172.21.15.40:0/289241639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf640715c0 0x7fcf64140af0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009340 tx=0x7fcf5402ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:19.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.016+0000 7fcf617fa640 1 -- 172.21.15.40:0/289241639 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5402e270 con 0x7fcf640715c0 2023-10-18T09:02:19.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.016+0000 7fcf617fa640 1 -- 172.21.15.40:0/289241639 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcf5402e4c0 con 0x7fcf640715c0 2023-10-18T09:02:19.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.016+0000 7fcf617fa640 1 -- 172.21.15.40:0/289241639 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5402c940 con 0x7fcf640715c0 2023-10-18T09:02:19.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.016+0000 7fcf6a2f2640 1 -- 172.21.15.40:0/289241639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf640c2180 con 0x7fcf640715c0 2023-10-18T09:02:19.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.016+0000 7fcf6a2f2640 1 -- 172.21.15.40:0/289241639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf64141030 con 0x7fcf640715c0 2023-10-18T09:02:19.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.017+0000 7fcf6a2f2640 1 -- 172.21.15.40:0/289241639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf640c1fe0 con 0x7fcf640715c0 2023-10-18T09:02:19.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.018+0000 7fcf617fa640 1 -- 172.21.15.40:0/289241639 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcf5402caa0 con 0x7fcf640715c0 2023-10-18T09:02:19.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.019+0000 7fcf617fa640 1 --2- 172.21.15.40:0/289241639 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf4406ae40 0x7fcf4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:19.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.019+0000 7fcf617fa640 1 -- 172.21.15.40:0/289241639 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fcf540ac180 con 0x7fcf640715c0 2023-10-18T09:02:19.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.020+0000 7fcf5bfff640 1 --2- 172.21.15.40:0/289241639 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf4406ae40 0x7fcf4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:19.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.021+0000 7fcf5bfff640 1 --2- 172.21.15.40:0/289241639 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf4406ae40 0x7fcf4406d300 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c005fd0 tx=0x7fcf4c005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:19.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.023+0000 7fcf617fa640 1 -- 172.21.15.40:0/289241639 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcf640c1fe0 con 0x7fcf640715c0 2023-10-18T09:02:19.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.225+0000 7fcf6a2f2640 1 -- 172.21.15.40:0/289241639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf640c1fe0 con 0x7fcf640715c0 2023-10-18T09:02:19.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.225+0000 7fcf617fa640 1 -- 172.21.15.40:0/289241639 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf54042020 con 0x7fcf640715c0 2023-10-18T09:02:19.229 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:02:19.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.228+0000 7fcf6a2f2640 1 -- 172.21.15.40:0/289241639 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf4406ae40 msgr2=0x7fcf4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:19.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.228+0000 7fcf6a2f2640 1 --2- 172.21.15.40:0/289241639 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf4406ae40 0x7fcf4406d300 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c005fd0 tx=0x7fcf4c005850 comp rx=0 tx=0).stop 2023-10-18T09:02:19.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.228+0000 7fcf6a2f2640 1 -- 172.21.15.40:0/289241639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf640715c0 msgr2=0x7fcf64140af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:19.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.228+0000 7fcf6a2f2640 1 --2- 172.21.15.40:0/289241639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf640715c0 0x7fcf64140af0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009340 tx=0x7fcf5402ea60 comp rx=0 tx=0).stop 2023-10-18T09:02:19.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.228+0000 7fcf6a2f2640 1 -- 172.21.15.40:0/289241639 shutdown_connections 2023-10-18T09:02:19.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.228+0000 7fcf6a2f2640 1 --2- 172.21.15.40:0/289241639 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf4406ae40 0x7fcf4406d300 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.228+0000 7fcf6a2f2640 1 --2- 172.21.15.40:0/289241639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf6414e730 0x7fcf64156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.228+0000 7fcf6a2f2640 1 --2- 172.21.15.40:0/289241639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf640af240 0x7fcf641567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.228+0000 7fcf6a2f2640 1 --2- 172.21.15.40:0/289241639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf640715c0 0x7fcf64140af0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.228+0000 7fcf6a2f2640 1 -- 172.21.15.40:0/289241639 >> 172.21.15.40:0/289241639 conn(0x7fcf64069720 msgr2=0x7fcf64151000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:19.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.228+0000 7fcf6a2f2640 1 -- 172.21.15.40:0/289241639 shutdown_connections 2023-10-18T09:02:19.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.228+0000 7fcf6a2f2640 1 -- 172.21.15.40:0/289241639 wait complete. 2023-10-18T09:02:19.251 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:02:19.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.484+0000 7facb6e78640 1 -- 172.21.15.40:0/1408917602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb006c240 msgr2=0x7facb015faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:19.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.484+0000 7facb6e78640 1 --2- 172.21.15.40:0/1408917602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb006c240 0x7facb015faf0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7faca8005fd0 tx=0x7faca802be20 comp rx=0 tx=0).stop 2023-10-18T09:02:19.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.485+0000 7facb6e78640 1 -- 172.21.15.40:0/1408917602 shutdown_connections 2023-10-18T09:02:19.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.485+0000 7facb6e78640 1 --2- 172.21.15.40:0/1408917602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb006c240 0x7facb015faf0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.485+0000 7facb6e78640 1 --2- 172.21.15.40:0/1408917602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facb015b5c0 0x7facb00cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.485+0000 7facb6e78640 1 -- 172.21.15.40:0/1408917602 >> 172.21.15.40:0/1408917602 conn(0x7facb006ca10 msgr2=0x7facb006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:19.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.485+0000 7facb6e78640 1 -- 172.21.15.40:0/1408917602 shutdown_connections 2023-10-18T09:02:19.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.485+0000 7facb6e78640 1 -- 172.21.15.40:0/1408917602 wait complete. 2023-10-18T09:02:19.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.487+0000 7facb6e78640 1 Processor -- start 2023-10-18T09:02:19.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.487+0000 7facb6e78640 1 -- start start 2023-10-18T09:02:19.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb6e78640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb006c240 0x7facb0150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:19.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb6e78640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb0070e40 0x7facb015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:19.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb6e78640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facb015b5c0 0x7facb00ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:19.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb6e78640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb0151710 con 0x7facb006c240 2023-10-18T09:02:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb6e78640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb015e7e0 con 0x7facb0070e40 2023-10-18T09:02:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb6e78640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb015d3d0 con 0x7facb015b5c0 2023-10-18T09:02:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb5e76640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb006c240 0x7facb0150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb5e76640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb006c240 0x7facb0150cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34298/0 (socket says 172.21.15.40:34298) 2023-10-18T09:02:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb5e76640 1 -- 172.21.15.40:0/154257323 learned_addr learned my addr 172.21.15.40:0/154257323 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:19.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb5675640 1 --2- 172.21.15.40:0/154257323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb0070e40 0x7facb015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:19.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb6677640 1 --2- 172.21.15.40:0/154257323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facb015b5c0 0x7facb00ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:19.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb5e76640 1 -- 172.21.15.40:0/154257323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facb015b5c0 msgr2=0x7facb00ad3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:19.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb5e76640 1 --2- 172.21.15.40:0/154257323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facb015b5c0 0x7facb00ad3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb5e76640 1 -- 172.21.15.40:0/154257323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb0070e40 msgr2=0x7facb015ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:19.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb5e76640 1 --2- 172.21.15.40:0/154257323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb0070e40 0x7facb015ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb5e76640 1 -- 172.21.15.40:0/154257323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca8009560 con 0x7facb006c240 2023-10-18T09:02:19.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.488+0000 7facb5675640 1 --2- 172.21.15.40:0/154257323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb0070e40 0x7facb015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:19.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.489+0000 7facb5e76640 1 --2- 172.21.15.40:0/154257323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb006c240 0x7facb0150cb0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7facac00eab0 tx=0x7facac00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:19.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.489+0000 7faca67fc640 1 -- 172.21.15.40:0/154257323 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac00ce40 con 0x7facb006c240 2023-10-18T09:02:19.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.489+0000 7faca67fc640 1 -- 172.21.15.40:0/154257323 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7facac004c10 con 0x7facb006c240 2023-10-18T09:02:19.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.489+0000 7faca67fc640 1 -- 172.21.15.40:0/154257323 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac005020 con 0x7facb006c240 2023-10-18T09:02:19.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.489+0000 7facb6e78640 1 -- 172.21.15.40:0/154257323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb015dbd0 con 0x7facb006c240 2023-10-18T09:02:19.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.489+0000 7facb6e78640 1 -- 172.21.15.40:0/154257323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb015e190 con 0x7facb006c240 2023-10-18T09:02:19.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.490+0000 7facb6e78640 1 -- 172.21.15.40:0/154257323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb0066e00 con 0x7facb006c240 2023-10-18T09:02:19.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.491+0000 7faca67fc640 1 -- 172.21.15.40:0/154257323 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7facac005180 con 0x7facb006c240 2023-10-18T09:02:19.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.492+0000 7faca67fc640 1 --2- 172.21.15.40:0/154257323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac9406ae40 0x7fac9406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:19.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.492+0000 7faca67fc640 1 -- 172.21.15.40:0/154257323 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7facac08a270 con 0x7facb006c240 2023-10-18T09:02:19.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.493+0000 7facb5675640 1 --2- 172.21.15.40:0/154257323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac9406ae40 0x7fac9406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:19.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.494+0000 7facb5675640 1 --2- 172.21.15.40:0/154257323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac9406ae40 0x7fac9406d300 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7facb00af9a0 tx=0x7faca0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:19.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.496+0000 7faca67fc640 1 -- 172.21.15.40:0/154257323 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7facac0551e0 con 0x7facb006c240 2023-10-18T09:02:19.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.691+0000 7facb6e78640 1 -- 172.21.15.40:0/154257323 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7facb015f660 con 0x7facb006c240 2023-10-18T09:02:19.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.692+0000 7faca67fc640 1 -- 172.21.15.40:0/154257323 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+11 (secure 0 0 0) 0x7facac01f0a0 con 0x7facb006c240 2023-10-18T09:02:19.696 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:02:19.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.695+0000 7facb6e78640 1 -- 172.21.15.40:0/154257323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac9406ae40 msgr2=0x7fac9406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:19.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.695+0000 7facb6e78640 1 --2- 172.21.15.40:0/154257323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac9406ae40 0x7fac9406d300 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7facb00af9a0 tx=0x7faca0009040 comp rx=0 tx=0).stop 2023-10-18T09:02:19.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.695+0000 7facb6e78640 1 -- 172.21.15.40:0/154257323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb006c240 msgr2=0x7facb0150cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:19.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.695+0000 7facb6e78640 1 --2- 172.21.15.40:0/154257323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb006c240 0x7facb0150cb0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7facac00eab0 tx=0x7facac00ef70 comp rx=0 tx=0).stop 2023-10-18T09:02:19.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.695+0000 7facb6e78640 1 -- 172.21.15.40:0/154257323 shutdown_connections 2023-10-18T09:02:19.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.695+0000 7facb6e78640 1 --2- 172.21.15.40:0/154257323 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac9406ae40 0x7fac9406d300 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.695+0000 7facb6e78640 1 --2- 172.21.15.40:0/154257323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facb015b5c0 0x7facb00ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.695+0000 7facb6e78640 1 --2- 172.21.15.40:0/154257323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facb0070e40 0x7facb015ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.695+0000 7facb6e78640 1 --2- 172.21.15.40:0/154257323 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facb006c240 0x7facb0150cb0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.695+0000 7facb6e78640 1 -- 172.21.15.40:0/154257323 >> 172.21.15.40:0/154257323 conn(0x7facb006ca10 msgr2=0x7facb0070290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:19.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.695+0000 7facb6e78640 1 -- 172.21.15.40:0/154257323 shutdown_connections 2023-10-18T09:02:19.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.695+0000 7facb6e78640 1 -- 172.21.15.40:0/154257323 wait complete. 2023-10-18T09:02:19.718 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:02:19.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.953+0000 7fd46d147640 1 -- 172.21.15.40:0/304402929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4680c9620 msgr2=0x7fd46806bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:19.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.953+0000 7fd46d147640 1 --2- 172.21.15.40:0/304402929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4680c9620 0x7fd46806bbc0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fd4680b1770 tx=0x7fd45802d0c0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.953+0000 7fd46d147640 1 -- 172.21.15.40:0/304402929 shutdown_connections 2023-10-18T09:02:19.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.953+0000 7fd46d147640 1 --2- 172.21.15.40:0/304402929 >> v2:172.21.15.144:3300/0 conn(0x7fd468070b60 0x7fd468070f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.953+0000 7fd46d147640 1 --2- 172.21.15.40:0/304402929 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd46815af70 0x7fd46806c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.953+0000 7fd46d147640 1 --2- 172.21.15.40:0/304402929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4680c9620 0x7fd46806bbc0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.953+0000 7fd46d147640 1 -- 172.21.15.40:0/304402929 >> 172.21.15.40:0/304402929 conn(0x7fd46806ca10 msgr2=0x7fd46806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:19.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.953+0000 7fd46d147640 1 -- 172.21.15.40:0/304402929 shutdown_connections 2023-10-18T09:02:19.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.953+0000 7fd46d147640 1 -- 172.21.15.40:0/304402929 wait complete. 2023-10-18T09:02:19.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.955+0000 7fd46d147640 1 Processor -- start 2023-10-18T09:02:19.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.955+0000 7fd46d147640 1 -- start start 2023-10-18T09:02:19.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.956+0000 7fd46d147640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd468070b60 0x7fd468158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:19.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.956+0000 7fd46d147640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4680c9620 0x7fd4681528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:19.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.956+0000 7fd46d147640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd46815af70 0x7fd4681563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:19.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.956+0000 7fd46d147640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd468155660 con 0x7fd4680c9620 2023-10-18T09:02:19.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.956+0000 7fd46d147640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4681592a0 con 0x7fd468070b60 2023-10-18T09:02:19.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.956+0000 7fd46d147640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd468155900 con 0x7fd46815af70 2023-10-18T09:02:19.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.957+0000 7fd45f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4680c9620 0x7fd4681528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:19.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.957+0000 7fd45f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4680c9620 0x7fd4681528a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34320/0 (socket says 172.21.15.40:34320) 2023-10-18T09:02:19.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.957+0000 7fd45f7fe640 1 -- 172.21.15.40:0/3577025538 learned_addr learned my addr 172.21.15.40:0/3577025538 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:19.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.957+0000 7fd46c946640 1 --2- 172.21.15.40:0/3577025538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd46815af70 0x7fd4681563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:19.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.957+0000 7fd467fff640 1 --2- 172.21.15.40:0/3577025538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd468070b60 0x7fd468158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:19.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.957+0000 7fd45f7fe640 1 -- 172.21.15.40:0/3577025538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd46815af70 msgr2=0x7fd4681563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:19.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.957+0000 7fd45f7fe640 1 --2- 172.21.15.40:0/3577025538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd46815af70 0x7fd4681563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.957+0000 7fd45f7fe640 1 -- 172.21.15.40:0/3577025538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd468070b60 msgr2=0x7fd468158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:19.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.957+0000 7fd45f7fe640 1 --2- 172.21.15.40:0/3577025538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd468070b60 0x7fd468158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:19.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.957+0000 7fd45f7fe640 1 -- 172.21.15.40:0/3577025538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45800aa10 con 0x7fd4680c9620 2023-10-18T09:02:19.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.958+0000 7fd45f7fe640 1 --2- 172.21.15.40:0/3577025538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4680c9620 0x7fd4681528a0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd45000e7c0 tx=0x7fd45000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:19.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.958+0000 7fd4657fa640 1 -- 172.21.15.40:0/3577025538 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4500095a0 con 0x7fd4680c9620 2023-10-18T09:02:19.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.958+0000 7fd4657fa640 1 -- 172.21.15.40:0/3577025538 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd45000eed0 con 0x7fd4680c9620 2023-10-18T09:02:19.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.958+0000 7fd4657fa640 1 -- 172.21.15.40:0/3577025538 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450005020 con 0x7fd4680c9620 2023-10-18T09:02:19.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.958+0000 7fd46d147640 1 -- 172.21.15.40:0/3577025538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd468155be0 con 0x7fd4680c9620 2023-10-18T09:02:19.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.958+0000 7fd46d147640 1 -- 172.21.15.40:0/3577025538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd468158e70 con 0x7fd4680c9620 2023-10-18T09:02:19.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.959+0000 7fd46d147640 1 -- 172.21.15.40:0/3577025538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd468066e00 con 0x7fd4680c9620 2023-10-18T09:02:19.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.960+0000 7fd4657fa640 1 -- 172.21.15.40:0/3577025538 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd450005180 con 0x7fd4680c9620 2023-10-18T09:02:19.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.961+0000 7fd4657fa640 1 --2- 172.21.15.40:0/3577025538 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd44806ae40 0x7fd44806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:19.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.961+0000 7fd4657fa640 1 -- 172.21.15.40:0/3577025538 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fd450014070 con 0x7fd4680c9620 2023-10-18T09:02:19.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.962+0000 7fd467fff640 1 --2- 172.21.15.40:0/3577025538 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd44806ae40 0x7fd44806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:19.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.963+0000 7fd467fff640 1 --2- 172.21.15.40:0/3577025538 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd44806ae40 0x7fd44806d300 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd458037000 tx=0x7fd458004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:19.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:19.965+0000 7fd4657fa640 1 -- 172.21.15.40:0/3577025538 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd450058b60 con 0x7fd4680c9620 2023-10-18T09:02:20.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.161+0000 7fd46d147640 1 -- 172.21.15.40:0/3577025538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd46815a9d0 con 0x7fd4680c9620 2023-10-18T09:02:20.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.162+0000 7fd4657fa640 1 -- 172.21.15.40:0/3577025538 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd4500586f0 con 0x7fd4680c9620 2023-10-18T09:02:20.166 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:02:20.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.164+0000 7fd46d147640 1 -- 172.21.15.40:0/3577025538 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd44806ae40 msgr2=0x7fd44806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:20.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.164+0000 7fd46d147640 1 --2- 172.21.15.40:0/3577025538 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd44806ae40 0x7fd44806d300 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd458037000 tx=0x7fd458004ab0 comp rx=0 tx=0).stop 2023-10-18T09:02:20.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.164+0000 7fd46d147640 1 -- 172.21.15.40:0/3577025538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4680c9620 msgr2=0x7fd4681528a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.164+0000 7fd46d147640 1 --2- 172.21.15.40:0/3577025538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4680c9620 0x7fd4681528a0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd45000e7c0 tx=0x7fd45000ec80 comp rx=0 tx=0).stop 2023-10-18T09:02:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.164+0000 7fd46d147640 1 -- 172.21.15.40:0/3577025538 shutdown_connections 2023-10-18T09:02:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.164+0000 7fd46d147640 1 --2- 172.21.15.40:0/3577025538 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd44806ae40 0x7fd44806d300 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.165+0000 7fd46d147640 1 --2- 172.21.15.40:0/3577025538 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd46815af70 0x7fd4681563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:20.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.165+0000 7fd46d147640 1 --2- 172.21.15.40:0/3577025538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4680c9620 0x7fd4681528a0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:20.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.165+0000 7fd46d147640 1 --2- 172.21.15.40:0/3577025538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd468070b60 0x7fd468158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:20.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.165+0000 7fd46d147640 1 -- 172.21.15.40:0/3577025538 >> 172.21.15.40:0/3577025538 conn(0x7fd46806ca10 msgr2=0x7fd468151eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:20.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.165+0000 7fd46d147640 1 -- 172.21.15.40:0/3577025538 shutdown_connections 2023-10-18T09:02:20.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.165+0000 7fd46d147640 1 -- 172.21.15.40:0/3577025538 wait complete. 2023-10-18T09:02:20.187 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:02:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.411+0000 7f3d608ea640 1 -- 172.21.15.40:0/1451679964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d48003970 msgr2=0x7f3d48003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.411+0000 7f3d608ea640 1 --2- 172.21.15.40:0/1451679964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d48003970 0x7f3d48003d90 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00c640 tx=0x7f3d480057f0 comp rx=0 tx=0).stop 2023-10-18T09:02:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.411+0000 7f3d608ea640 1 -- 172.21.15.40:0/1451679964 shutdown_connections 2023-10-18T09:02:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.411+0000 7f3d608ea640 1 --2- 172.21.15.40:0/1451679964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d48003970 0x7f3d48003d90 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.411+0000 7f3d608ea640 1 -- 172.21.15.40:0/1451679964 >> 172.21.15.40:0/1451679964 conn(0x7f3d5c08acb0 msgr2=0x7f3d5c09a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.411+0000 7f3d608ea640 1 -- 172.21.15.40:0/1451679964 shutdown_connections 2023-10-18T09:02:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.411+0000 7f3d608ea640 1 -- 172.21.15.40:0/1451679964 wait complete. 2023-10-18T09:02:20.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.413+0000 7f3d608ea640 1 Processor -- start 2023-10-18T09:02:20.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.413+0000 7f3d608ea640 1 -- start start 2023-10-18T09:02:20.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.414+0000 7f3d608ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d48003970 0x7f3d5c1e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:20.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.414+0000 7f3d608ea640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c0b4120 0x7f3d5c149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:20.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.414+0000 7f3d608ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d5c0b4580 0x7f3d5c1497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:20.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.414+0000 7f3d608ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c1e60c0 con 0x7f3d48003970 2023-10-18T09:02:20.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.414+0000 7f3d608ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c1e5440 con 0x7f3d5c0b4580 2023-10-18T09:02:20.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.414+0000 7f3d608ea640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c1e5720 con 0x7f3d5c0b4120 2023-10-18T09:02:20.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.415+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d48003970 0x7f3d5c1e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:20.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.415+0000 7f3d5b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d48003970 0x7f3d5c1e64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34338/0 (socket says 172.21.15.40:34338) 2023-10-18T09:02:20.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.415+0000 7f3d5b7fe640 1 -- 172.21.15.40:0/3555112520 learned_addr learned my addr 172.21.15.40:0/3555112520 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:20.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.415+0000 7f3d5b7fe640 1 -- 172.21.15.40:0/3555112520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c0b4120 msgr2=0x7f3d5c149260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:20.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.415+0000 7f3d5b7fe640 1 --2- 172.21.15.40:0/3555112520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c0b4120 0x7f3d5c149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:20.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.415+0000 7f3d5b7fe640 1 -- 172.21.15.40:0/3555112520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d5c0b4580 msgr2=0x7f3d5c1497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:20.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.415+0000 7f3d5bfff640 1 --2- 172.21.15.40:0/3555112520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d5c0b4580 0x7f3d5c1497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:20.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.415+0000 7f3d5b7fe640 1 --2- 172.21.15.40:0/3555112520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d5c0b4580 0x7f3d5c1497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:20.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.415+0000 7f3d5b7fe640 1 -- 172.21.15.40:0/3555112520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c1e5a00 con 0x7f3d48003970 2023-10-18T09:02:20.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.415+0000 7f3d5bfff640 1 --2- 172.21.15.40:0/3555112520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d5c0b4580 0x7f3d5c1497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:20.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.416+0000 7f3d5b7fe640 1 --2- 172.21.15.40:0/3555112520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d48003970 0x7f3d5c1e64e0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0012e0 tx=0x7f3d4c0019d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:20.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.416+0000 7f3d58ff9640 1 -- 172.21.15.40:0/3555112520 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c003b70 con 0x7f3d48003970 2023-10-18T09:02:20.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.416+0000 7f3d58ff9640 1 -- 172.21.15.40:0/3555112520 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3d4c003cd0 con 0x7f3d48003970 2023-10-18T09:02:20.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.416+0000 7f3d58ff9640 1 -- 172.21.15.40:0/3555112520 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c006320 con 0x7f3d48003970 2023-10-18T09:02:20.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.416+0000 7f3d608ea640 1 -- 172.21.15.40:0/3555112520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5c1e5ce0 con 0x7f3d48003970 2023-10-18T09:02:20.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.416+0000 7f3d608ea640 1 -- 172.21.15.40:0/3555112520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5c0b4ef0 con 0x7f3d48003970 2023-10-18T09:02:20.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.417+0000 7f3d608ea640 1 -- 172.21.15.40:0/3555112520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5c066e00 con 0x7f3d48003970 2023-10-18T09:02:20.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.418+0000 7f3d58ff9640 1 -- 172.21.15.40:0/3555112520 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3d4c031030 con 0x7f3d48003970 2023-10-18T09:02:20.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.419+0000 7f3d58ff9640 1 --2- 172.21.15.40:0/3555112520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d3c06ae40 0x7f3d3c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:20.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.419+0000 7f3d58ff9640 1 -- 172.21.15.40:0/3555112520 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f3d4c0aa610 con 0x7f3d48003970 2023-10-18T09:02:20.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.420+0000 7f3d52ffd640 1 --2- 172.21.15.40:0/3555112520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d3c06ae40 0x7f3d3c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:20.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.421+0000 7f3d52ffd640 1 --2- 172.21.15.40:0/3555112520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d3c06ae40 0x7f3d3c06d300 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c14d040 tx=0x7f3d44006d50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:20.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.423+0000 7f3d58ff9640 1 -- 172.21.15.40:0/3555112520 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3d4c075600 con 0x7f3d48003970 2023-10-18T09:02:20.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.631+0000 7f3d608ea640 1 -- 172.21.15.40:0/3555112520 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3d5c0d2180 con 0x7f3d48003970 2023-10-18T09:02:20.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.631+0000 7f3d58ff9640 1 -- 172.21.15.40:0/3555112520 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v118) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3d4c078c20 con 0x7f3d48003970 2023-10-18T09:02:20.635 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T09:02:20.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.634+0000 7f3d608ea640 1 -- 172.21.15.40:0/3555112520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d3c06ae40 msgr2=0x7f3d3c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:20.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.634+0000 7f3d608ea640 1 --2- 172.21.15.40:0/3555112520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d3c06ae40 0x7f3d3c06d300 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c14d040 tx=0x7f3d44006d50 comp rx=0 tx=0).stop 2023-10-18T09:02:20.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.634+0000 7f3d608ea640 1 -- 172.21.15.40:0/3555112520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d48003970 msgr2=0x7f3d5c1e64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:20.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.634+0000 7f3d608ea640 1 --2- 172.21.15.40:0/3555112520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d48003970 0x7f3d5c1e64e0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0012e0 tx=0x7f3d4c0019d0 comp rx=0 tx=0).stop 2023-10-18T09:02:20.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.635+0000 7f3d608ea640 1 -- 172.21.15.40:0/3555112520 shutdown_connections 2023-10-18T09:02:20.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.635+0000 7f3d608ea640 1 --2- 172.21.15.40:0/3555112520 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3d3c06ae40 0x7f3d3c06d300 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:20.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.635+0000 7f3d608ea640 1 --2- 172.21.15.40:0/3555112520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d5c0b4580 0x7f3d5c1497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:20.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.635+0000 7f3d608ea640 1 --2- 172.21.15.40:0/3555112520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d5c0b4120 0x7f3d5c149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:20.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.635+0000 7f3d608ea640 1 --2- 172.21.15.40:0/3555112520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d48003970 0x7f3d5c1e64e0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:20.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.635+0000 7f3d608ea640 1 -- 172.21.15.40:0/3555112520 >> 172.21.15.40:0/3555112520 conn(0x7f3d5c08acb0 msgr2=0x7f3d5c15a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:20.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.635+0000 7f3d608ea640 1 -- 172.21.15.40:0/3555112520 shutdown_connections 2023-10-18T09:02:20.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:20.635+0000 7f3d608ea640 1 -- 172.21.15.40:0/3555112520 wait complete. 2023-10-18T09:02:20.658 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-18T09:02:20.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:20.658 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-18T09:02:20.768 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T09:02:20.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:20.769 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T09:02:20.825 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:02:20.825 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:02:20.888 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:02:20.889 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:02:20.889 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:02:20.948 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:02:20.949 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:02:21.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.195+0000 7f29527e9640 1 -- 172.21.15.40:0/3640539125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2938005180 msgr2=0x7f293800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:21.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.195+0000 7f29527e9640 1 --2- 172.21.15.40:0/3640539125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2938005180 0x7f293800b980 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f294c15af40 tx=0x7f29380057f0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.195+0000 7f2951fe8640 1 -- 172.21.15.40:0/3640539125 reap_dead start 2023-10-18T09:02:21.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.195+0000 7f29527e9640 1 -- 172.21.15.40:0/3640539125 shutdown_connections 2023-10-18T09:02:21.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.195+0000 7f29527e9640 1 -- 172.21.15.40:0/3640539125 >> 172.21.15.40:0/3640539125 conn(0x7f294c069720 msgr2=0x7f294c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:21.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.195+0000 7f29527e9640 1 -- 172.21.15.40:0/3640539125 shutdown_connections 2023-10-18T09:02:21.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.195+0000 7f29527e9640 1 -- 172.21.15.40:0/3640539125 wait complete. 2023-10-18T09:02:21.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.197+0000 7f29527e9640 1 Processor -- start 2023-10-18T09:02:21.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.197+0000 7f29527e9640 1 -- start start 2023-10-18T09:02:21.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.198+0000 7f29527e9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2938005180 0x7f294c13d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:21.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.198+0000 7f29527e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f294c14b7e0 0x7f294c14bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:21.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.198+0000 7f29527e9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f294c156620 0x7f294c156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:21.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.198+0000 7f29527e9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c14cbb0 con 0x7f294c14b7e0 2023-10-18T09:02:21.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.198+0000 7f29527e9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c13de60 con 0x7f294c156620 2023-10-18T09:02:21.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.198+0000 7f29527e9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c156fe0 con 0x7f2938005180 2023-10-18T09:02:21.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.198+0000 7f2950fe6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f294c14b7e0 0x7f294c14bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:21.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.198+0000 7f2950fe6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f294c14b7e0 0x7f294c14bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34360/0 (socket says 172.21.15.40:34360) 2023-10-18T09:02:21.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.198+0000 7f2950fe6640 1 -- 172.21.15.40:0/2619956020 learned_addr learned my addr 172.21.15.40:0/2619956020 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:21.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.199+0000 7f2951fe8640 1 --2- 172.21.15.40:0/2619956020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f294c156620 0x7f294c156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:21.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.199+0000 7f2950fe6640 1 -- 172.21.15.40:0/2619956020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2938005180 msgr2=0x7f294c13d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:21.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.199+0000 7f2950fe6640 1 --2- 172.21.15.40:0/2619956020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2938005180 0x7f294c13d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.199+0000 7f2950fe6640 1 -- 172.21.15.40:0/2619956020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f294c156620 msgr2=0x7f294c156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:21.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.199+0000 7f2950fe6640 1 --2- 172.21.15.40:0/2619956020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f294c156620 0x7f294c156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.199+0000 7f2950fe6640 1 -- 172.21.15.40:0/2619956020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c14cf70 con 0x7f294c14b7e0 2023-10-18T09:02:21.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.199+0000 7f2950fe6640 1 --2- 172.21.15.40:0/2619956020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f294c14b7e0 0x7f294c14bc60 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f293c00fa00 tx=0x7f293c00fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:21.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.199+0000 7f2941ffb640 1 -- 172.21.15.40:0/2619956020 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c00bc50 con 0x7f294c14b7e0 2023-10-18T09:02:21.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.199+0000 7f2941ffb640 1 -- 172.21.15.40:0/2619956020 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f293c004030 con 0x7f294c14b7e0 2023-10-18T09:02:21.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.199+0000 7f2941ffb640 1 -- 172.21.15.40:0/2619956020 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c00bea0 con 0x7f294c14b7e0 2023-10-18T09:02:21.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.199+0000 7f29527e9640 1 -- 172.21.15.40:0/2619956020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c154c10 con 0x7f294c14b7e0 2023-10-18T09:02:21.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.200+0000 7f29527e9640 1 -- 172.21.15.40:0/2619956020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294c14c5f0 con 0x7f294c14b7e0 2023-10-18T09:02:21.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.201+0000 7f29527e9640 1 -- 172.21.15.40:0/2619956020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294c066e00 con 0x7f294c14b7e0 2023-10-18T09:02:21.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.202+0000 7f2941ffb640 1 -- 172.21.15.40:0/2619956020 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f293c0035f0 con 0x7f294c14b7e0 2023-10-18T09:02:21.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.203+0000 7f2941ffb640 1 --2- 172.21.15.40:0/2619956020 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f293006ae70 0x7f293006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:21.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.203+0000 7f2941ffb640 1 -- 172.21.15.40:0/2619956020 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f293c08a150 con 0x7f294c14b7e0 2023-10-18T09:02:21.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.203+0000 7f29517e7640 1 --2- 172.21.15.40:0/2619956020 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f293006ae70 0x7f293006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:21.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.205+0000 7f29517e7640 1 --2- 172.21.15.40:0/2619956020 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f293006ae70 0x7f293006d330 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2948005d00 tx=0x7f2948009530 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:21.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.207+0000 7f2941ffb640 1 -- 172.21.15.40:0/2619956020 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f293c055160 con 0x7f294c14b7e0 2023-10-18T09:02:21.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.442+0000 7f29527e9640 1 -- 172.21.15.40:0/2619956020 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f294c0caf90 con 0x7f294c14b7e0 2023-10-18T09:02:21.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.444+0000 7f2941ffb640 1 -- 172.21.15.40:0/2619956020 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f293c058780 con 0x7f294c14b7e0 2023-10-18T09:02:21.447 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:21.447 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2565609773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2565609773},{"type":"v1","addr":"172.21.15.144:6836","nonce":2565609773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":5872,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3371548374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3371548374},{"type":"v1","addr":"172.21.15.144:6837","nonce":3371548374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3328648628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3328648628},{"type":"v1","addr":"172.21.15.40:6833","nonce":3328648628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:01:36.995669+0000","modified":"2023-10-18T09:01:40.068484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/891188043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":891188043},{"type":"v1","addr":"172.21.15.40:6835","nonce":891188043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-18T09:02:21.448 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 49 2023-10-18T09:02:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.446+0000 7f29527e9640 1 -- 172.21.15.40:0/2619956020 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f293006ae70 msgr2=0x7f293006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.446+0000 7f29527e9640 1 --2- 172.21.15.40:0/2619956020 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f293006ae70 0x7f293006d330 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2948005d00 tx=0x7f2948009530 comp rx=0 tx=0).stop 2023-10-18T09:02:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.446+0000 7f29527e9640 1 -- 172.21.15.40:0/2619956020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f294c14b7e0 msgr2=0x7f294c14bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.446+0000 7f29527e9640 1 --2- 172.21.15.40:0/2619956020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f294c14b7e0 0x7f294c14bc60 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f293c00fa00 tx=0x7f293c00fec0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.447+0000 7f29527e9640 1 -- 172.21.15.40:0/2619956020 shutdown_connections 2023-10-18T09:02:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.447+0000 7f29527e9640 1 --2- 172.21.15.40:0/2619956020 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f293006ae70 0x7f293006d330 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.447+0000 7f29527e9640 1 --2- 172.21.15.40:0/2619956020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f294c156620 0x7f294c156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.447+0000 7f29527e9640 1 --2- 172.21.15.40:0/2619956020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f294c14b7e0 0x7f294c14bc60 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.447+0000 7f29527e9640 1 --2- 172.21.15.40:0/2619956020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2938005180 0x7f294c13d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.447+0000 7f29527e9640 1 -- 172.21.15.40:0/2619956020 >> 172.21.15.40:0/2619956020 conn(0x7f294c069720 msgr2=0x7f294c0ab4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.447+0000 7f29527e9640 1 -- 172.21.15.40:0/2619956020 shutdown_connections 2023-10-18T09:02:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.447+0000 7f29527e9640 1 -- 172.21.15.40:0/2619956020 wait complete. 2023-10-18T09:02:21.471 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:02:21.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.702+0000 7fc5344e5640 1 -- 172.21.15.40:0/2539828311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc52c1586a0 msgr2=0x7fc52c0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:21.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.702+0000 7fc5344e5640 1 --2- 172.21.15.40:0/2539828311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc52c1586a0 0x7fc52c0b0bc0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fc52c06a6d0 tx=0x7fc51c02bd70 comp rx=0 tx=0).stop 2023-10-18T09:02:21.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.702+0000 7fc5344e5640 1 -- 172.21.15.40:0/2539828311 shutdown_connections 2023-10-18T09:02:21.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.702+0000 7fc5344e5640 1 --2- 172.21.15.40:0/2539828311 >> v2:172.21.15.144:3300/0 conn(0x7fc52c0b1100 0x7fc52c1625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.702+0000 7fc5344e5640 1 --2- 172.21.15.40:0/2539828311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc52c1586a0 0x7fc52c0b0bc0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.702+0000 7fc5344e5640 1 -- 172.21.15.40:0/2539828311 >> 172.21.15.40:0/2539828311 conn(0x7fc52c06cee0 msgr2=0x7fc52c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:21.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.703+0000 7fc5344e5640 1 -- 172.21.15.40:0/2539828311 shutdown_connections 2023-10-18T09:02:21.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.703+0000 7fc5344e5640 1 -- 172.21.15.40:0/2539828311 wait complete. 2023-10-18T09:02:21.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.704+0000 7fc5344e5640 1 Processor -- start 2023-10-18T09:02:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.704+0000 7fc5344e5640 1 -- start start 2023-10-18T09:02:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.705+0000 7fc5344e5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc52c06d710 0x7fc52c13d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.705+0000 7fc5344e5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc52c0b1100 0x7fc52c14c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.705+0000 7fc5344e5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc52c1586a0 0x7fc52c14cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:21.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.705+0000 7fc5344e5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c141080 con 0x7fc52c0b1100 2023-10-18T09:02:21.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.705+0000 7fc5344e5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c140140 con 0x7fc52c1586a0 2023-10-18T09:02:21.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.705+0000 7fc5344e5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c140400 con 0x7fc52c06d710 2023-10-18T09:02:21.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.706+0000 7fc531a80640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc52c0b1100 0x7fc52c14c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:21.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.706+0000 7fc531a80640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc52c0b1100 0x7fc52c14c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34384/0 (socket says 172.21.15.40:34384) 2023-10-18T09:02:21.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.706+0000 7fc531a80640 1 -- 172.21.15.40:0/3644642315 learned_addr learned my addr 172.21.15.40:0/3644642315 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.706+0000 7fc532281640 1 --2- 172.21.15.40:0/3644642315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc52c06d710 0x7fc52c13d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.706+0000 7fc532a82640 1 --2- 172.21.15.40:0/3644642315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc52c1586a0 0x7fc52c14cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.706+0000 7fc531a80640 1 -- 172.21.15.40:0/3644642315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc52c06d710 msgr2=0x7fc52c13d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.706+0000 7fc531a80640 1 --2- 172.21.15.40:0/3644642315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc52c06d710 0x7fc52c13d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.706+0000 7fc531a80640 1 -- 172.21.15.40:0/3644642315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc52c1586a0 msgr2=0x7fc52c14cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.706+0000 7fc531a80640 1 --2- 172.21.15.40:0/3644642315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc52c1586a0 0x7fc52c14cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.706+0000 7fc531a80640 1 -- 172.21.15.40:0/3644642315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c009490 con 0x7fc52c0b1100 2023-10-18T09:02:21.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.707+0000 7fc531a80640 1 --2- 172.21.15.40:0/3644642315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc52c0b1100 0x7fc52c14c710 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fc51c000c00 tx=0x7fc51c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:21.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.707+0000 7fc522ffd640 1 -- 172.21.15.40:0/3644642315 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c003ea0 con 0x7fc52c0b1100 2023-10-18T09:02:21.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.707+0000 7fc522ffd640 1 -- 172.21.15.40:0/3644642315 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc51c034550 con 0x7fc52c0b1100 2023-10-18T09:02:21.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.707+0000 7fc522ffd640 1 -- 172.21.15.40:0/3644642315 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c034960 con 0x7fc52c0b1100 2023-10-18T09:02:21.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.707+0000 7fc5344e5640 1 -- 172.21.15.40:0/3644642315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52c1406e0 con 0x7fc52c0b1100 2023-10-18T09:02:21.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.707+0000 7fc5344e5640 1 -- 172.21.15.40:0/3644642315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52c140ca0 con 0x7fc52c0b1100 2023-10-18T09:02:21.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.708+0000 7fc5344e5640 1 -- 172.21.15.40:0/3644642315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52c066fd0 con 0x7fc52c0b1100 2023-10-18T09:02:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.709+0000 7fc522ffd640 1 -- 172.21.15.40:0/3644642315 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc51c03b040 con 0x7fc52c0b1100 2023-10-18T09:02:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.710+0000 7fc522ffd640 1 --2- 172.21.15.40:0/3644642315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc51006aec0 0x7fc51006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.710+0000 7fc522ffd640 1 -- 172.21.15.40:0/3644642315 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fc51c0abc60 con 0x7fc52c0b1100 2023-10-18T09:02:21.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.711+0000 7fc532281640 1 --2- 172.21.15.40:0/3644642315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc51006aec0 0x7fc51006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:21.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.712+0000 7fc532281640 1 --2- 172.21.15.40:0/3644642315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc51006aec0 0x7fc51006d380 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc5280046b0 tx=0x7fc528009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:21.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.714+0000 7fc522ffd640 1 -- 172.21.15.40:0/3644642315 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc51c076c40 con 0x7fc52c0b1100 2023-10-18T09:02:21.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.907+0000 7fc5344e5640 1 -- 172.21.15.40:0/3644642315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc52c15b2d0 con 0x7fc52c0b1100 2023-10-18T09:02:21.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.910+0000 7fc522ffd640 1 -- 172.21.15.40:0/3644642315 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16873 (secure 0 0 0) 0x7fc51c041090 con 0x7fc52c0b1100 2023-10-18T09:02:21.913 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:21.913 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":118,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:02:09.285724+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:01:35.453236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-18T09:01:36.012460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1","mappings":[{"from":2,"to":7}]},{"pgid":"13.14","mappings":[{"from":6,"to":5}]},{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.20","mappings":[{"from":1,"to":5}]},{"pgid":"13.34","mappings":[{"from":6,"to":5}]},{"pgid":"13.38","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:02:21.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.912+0000 7fc5344e5640 1 -- 172.21.15.40:0/3644642315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc51006aec0 msgr2=0x7fc51006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:21.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.912+0000 7fc5344e5640 1 --2- 172.21.15.40:0/3644642315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc51006aec0 0x7fc51006d380 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc5280046b0 tx=0x7fc528009040 comp rx=0 tx=0).stop 2023-10-18T09:02:21.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.912+0000 7fc5344e5640 1 -- 172.21.15.40:0/3644642315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc52c0b1100 msgr2=0x7fc52c14c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:21.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.912+0000 7fc5344e5640 1 --2- 172.21.15.40:0/3644642315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc52c0b1100 0x7fc52c14c710 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fc51c000c00 tx=0x7fc51c0038a0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.913+0000 7fc5344e5640 1 -- 172.21.15.40:0/3644642315 shutdown_connections 2023-10-18T09:02:21.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.913+0000 7fc5344e5640 1 --2- 172.21.15.40:0/3644642315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc51006aec0 0x7fc51006d380 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.913+0000 7fc5344e5640 1 --2- 172.21.15.40:0/3644642315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc52c1586a0 0x7fc52c14cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.913+0000 7fc5344e5640 1 --2- 172.21.15.40:0/3644642315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc52c0b1100 0x7fc52c14c710 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.913+0000 7fc5344e5640 1 --2- 172.21.15.40:0/3644642315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc52c06d710 0x7fc52c13d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:21.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.913+0000 7fc5344e5640 1 -- 172.21.15.40:0/3644642315 >> 172.21.15.40:0/3644642315 conn(0x7fc52c06cee0 msgr2=0x7fc52c070590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:21.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.913+0000 7fc5344e5640 1 -- 172.21.15.40:0/3644642315 shutdown_connections 2023-10-18T09:02:21.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:21.913+0000 7fc5344e5640 1 -- 172.21.15.40:0/3644642315 wait complete. 2023-10-18T09:02:21.934 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:01:35.453236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:01:36.012460+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:02:21.935 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:02:22.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.175+0000 7f174cd97640 1 -- 172.21.15.40:0/3181757627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17480aed00 msgr2=0x7f17480af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:22.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.175+0000 7f174cd97640 1 --2- 172.21.15.40:0/3181757627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17480aed00 0x7f17480af170 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1740009be0 tx=0x7f174002c0d0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.175+0000 7f174cd97640 1 -- 172.21.15.40:0/3181757627 shutdown_connections 2023-10-18T09:02:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.175+0000 7f174cd97640 1 --2- 172.21.15.40:0/3181757627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17480aed00 0x7f17480af170 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.175+0000 7f174cd97640 1 --2- 172.21.15.40:0/3181757627 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17480715c0 0x7f174814e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.175+0000 7f174cd97640 1 -- 172.21.15.40:0/3181757627 >> 172.21.15.40:0/3181757627 conn(0x7f1748069720 msgr2=0x7f1748069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.176+0000 7f174cd97640 1 -- 172.21.15.40:0/3181757627 shutdown_connections 2023-10-18T09:02:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.176+0000 7f174cd97640 1 -- 172.21.15.40:0/3181757627 wait complete. 2023-10-18T09:02:22.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.177+0000 7f174cd97640 1 Processor -- start 2023-10-18T09:02:22.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.177+0000 7f174cd97640 1 -- start start 2023-10-18T09:02:22.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.178+0000 7f174cd97640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f174806d6f0 0x7f17480ab960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:22.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.178+0000 7f174cd97640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17480715c0 0x7f1748150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:22.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.178+0000 7f174cd97640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17480aed00 0x7f17481567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:22.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.178+0000 7f174cd97640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1748151d00 con 0x7f174806d6f0 2023-10-18T09:02:22.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.178+0000 7f174cd97640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17481592e0 con 0x7f17480715c0 2023-10-18T09:02:22.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.178+0000 7f174cd97640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174814f820 con 0x7f17480aed00 2023-10-18T09:02:22.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.179+0000 7f174659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f174806d6f0 0x7f17480ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:22.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.179+0000 7f174659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f174806d6f0 0x7f17480ab960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34402/0 (socket says 172.21.15.40:34402) 2023-10-18T09:02:22.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.179+0000 7f174659c640 1 -- 172.21.15.40:0/3353853042 learned_addr learned my addr 172.21.15.40:0/3353853042 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.179+0000 7f173ffff640 1 --2- 172.21.15.40:0/3353853042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17480715c0 0x7f1748150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.179+0000 7f1746d9d640 1 --2- 172.21.15.40:0/3353853042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17480aed00 0x7f17481567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.179+0000 7f174659c640 1 -- 172.21.15.40:0/3353853042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17480aed00 msgr2=0x7f17481567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.179+0000 7f174659c640 1 --2- 172.21.15.40:0/3353853042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17480aed00 0x7f17481567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.179+0000 7f174659c640 1 -- 172.21.15.40:0/3353853042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17480715c0 msgr2=0x7f1748150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.179+0000 7f174659c640 1 --2- 172.21.15.40:0/3353853042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17480715c0 0x7f1748150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.179+0000 7f174659c640 1 -- 172.21.15.40:0/3353853042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1740009840 con 0x7f174806d6f0 2023-10-18T09:02:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.179+0000 7f173ffff640 1 --2- 172.21.15.40:0/3353853042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17480715c0 0x7f1748150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:22.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.180+0000 7f174659c640 1 --2- 172.21.15.40:0/3353853042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f174806d6f0 0x7f17480ab960 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f173800c730 tx=0x7f173800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:22.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.180+0000 7f173effd640 1 -- 172.21.15.40:0/3353853042 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738004230 con 0x7f174806d6f0 2023-10-18T09:02:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.180+0000 7f173effd640 1 -- 172.21.15.40:0/3353853042 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f17380044a0 con 0x7f174806d6f0 2023-10-18T09:02:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.180+0000 7f173effd640 1 -- 172.21.15.40:0/3353853042 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173800a8a0 con 0x7f174806d6f0 2023-10-18T09:02:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.180+0000 7f174cd97640 1 -- 172.21.15.40:0/3353853042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17480abea0 con 0x7f174806d6f0 2023-10-18T09:02:22.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.180+0000 7f174cd97640 1 -- 172.21.15.40:0/3353853042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1748158f70 con 0x7f174806d6f0 2023-10-18T09:02:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.181+0000 7f174cd97640 1 -- 172.21.15.40:0/3353853042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17481517a0 con 0x7f174806d6f0 2023-10-18T09:02:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.182+0000 7f173effd640 1 -- 172.21.15.40:0/3353853042 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f173800aa00 con 0x7f174806d6f0 2023-10-18T09:02:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.183+0000 7f173effd640 1 --2- 172.21.15.40:0/3353853042 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f172806ae40 0x7f172806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.183+0000 7f173effd640 1 -- 172.21.15.40:0/3353853042 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f1738089910 con 0x7f174806d6f0 2023-10-18T09:02:22.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.184+0000 7f173ffff640 1 --2- 172.21.15.40:0/3353853042 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f172806ae40 0x7f172806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.185+0000 7f173ffff640 1 --2- 172.21.15.40:0/3353853042 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f172806ae40 0x7f172806d300 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f174814def0 tx=0x7f1730005e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:22.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.187+0000 7f173effd640 1 -- 172.21.15.40:0/3353853042 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1738054900 con 0x7f174806d6f0 2023-10-18T09:02:22.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.383+0000 7f174cd97640 1 -- 172.21.15.40:0/3353853042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f17480b2060 con 0x7f174806d6f0 2023-10-18T09:02:22.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.384+0000 7f173effd640 1 -- 172.21.15.40:0/3353853042 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17480b2060 con 0x7f174806d6f0 2023-10-18T09:02:22.387 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:02:22.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.386+0000 7f174cd97640 1 -- 172.21.15.40:0/3353853042 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f172806ae40 msgr2=0x7f172806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:22.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.386+0000 7f174cd97640 1 --2- 172.21.15.40:0/3353853042 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f172806ae40 0x7f172806d300 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f174814def0 tx=0x7f1730005e50 comp rx=0 tx=0).stop 2023-10-18T09:02:22.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.386+0000 7f174cd97640 1 -- 172.21.15.40:0/3353853042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f174806d6f0 msgr2=0x7f17480ab960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:22.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.386+0000 7f174cd97640 1 --2- 172.21.15.40:0/3353853042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f174806d6f0 0x7f17480ab960 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f173800c730 tx=0x7f173800cbf0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.386+0000 7f174cd97640 1 -- 172.21.15.40:0/3353853042 shutdown_connections 2023-10-18T09:02:22.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.386+0000 7f174cd97640 1 --2- 172.21.15.40:0/3353853042 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f172806ae40 0x7f172806d300 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.386+0000 7f174cd97640 1 --2- 172.21.15.40:0/3353853042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17480aed00 0x7f17481567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.386+0000 7f174cd97640 1 --2- 172.21.15.40:0/3353853042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17480715c0 0x7f1748150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.386+0000 7f174cd97640 1 --2- 172.21.15.40:0/3353853042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f174806d6f0 0x7f17480ab960 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.386+0000 7f174cd97640 1 -- 172.21.15.40:0/3353853042 >> 172.21.15.40:0/3353853042 conn(0x7f1748069720 msgr2=0x7f17480acab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:22.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.386+0000 7f174cd97640 1 -- 172.21.15.40:0/3353853042 shutdown_connections 2023-10-18T09:02:22.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.386+0000 7f174cd97640 1 -- 172.21.15.40:0/3353853042 wait complete. 2023-10-18T09:02:22.408 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:02:22.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.640+0000 7f99741e9640 1 -- 172.21.15.40:0/4041280360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f996c0cb590 msgr2=0x7f996c163620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:22.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.640+0000 7f99741e9640 1 --2- 172.21.15.40:0/4041280360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f996c0cb590 0x7f996c163620 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f9964005fd0 tx=0x7f996402c050 comp rx=0 tx=0).stop 2023-10-18T09:02:22.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.641+0000 7f99741e9640 1 -- 172.21.15.40:0/4041280360 shutdown_connections 2023-10-18T09:02:22.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.641+0000 7f99741e9640 1 --2- 172.21.15.40:0/4041280360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f996c0cb590 0x7f996c163620 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.641+0000 7f99741e9640 1 -- 172.21.15.40:0/4041280360 >> 172.21.15.40:0/4041280360 conn(0x7f996c069720 msgr2=0x7f996c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:22.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.641+0000 7f99741e9640 1 -- 172.21.15.40:0/4041280360 shutdown_connections 2023-10-18T09:02:22.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.641+0000 7f99741e9640 1 -- 172.21.15.40:0/4041280360 wait complete. 2023-10-18T09:02:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.642+0000 7f99741e9640 1 Processor -- start 2023-10-18T09:02:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.643+0000 7f99741e9640 1 -- start start 2023-10-18T09:02:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.643+0000 7f99741e9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c06d660 0x7f996c14fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.643+0000 7f99741e9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c0cb590 0x7f996c149ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.644+0000 7f99741e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f996c157800 0x7f996c14a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.644+0000 7f99741e9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c151c40 con 0x7f996c157800 2023-10-18T09:02:22.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.644+0000 7f99741e9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c150ba0 con 0x7f996c0cb590 2023-10-18T09:02:22.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.644+0000 7f99741e9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996c150e80 con 0x7f996c06d660 2023-10-18T09:02:22.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.644+0000 7f9972786640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f996c157800 0x7f996c14a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:22.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.644+0000 7f9972786640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f996c157800 0x7f996c14a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34420/0 (socket says 172.21.15.40:34420) 2023-10-18T09:02:22.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.644+0000 7f9972786640 1 -- 172.21.15.40:0/2739944401 learned_addr learned my addr 172.21.15.40:0/2739944401 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:22.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.644+0000 7f9971f85640 1 --2- 172.21.15.40:0/2739944401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c06d660 0x7f996c14fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:22.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.644+0000 7f9971784640 1 --2- 172.21.15.40:0/2739944401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c0cb590 0x7f996c149ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:22.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.644+0000 7f9972786640 1 -- 172.21.15.40:0/2739944401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c06d660 msgr2=0x7f996c14fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:22.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.644+0000 7f9972786640 1 --2- 172.21.15.40:0/2739944401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c06d660 0x7f996c14fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.645+0000 7f9972786640 1 -- 172.21.15.40:0/2739944401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c0cb590 msgr2=0x7f996c149ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:22.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.645+0000 7f9972786640 1 --2- 172.21.15.40:0/2739944401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c0cb590 0x7f996c149ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.645+0000 7f9972786640 1 -- 172.21.15.40:0/2739944401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9964009840 con 0x7f996c157800 2023-10-18T09:02:22.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.645+0000 7f9972786640 1 --2- 172.21.15.40:0/2739944401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f996c157800 0x7f996c14a220 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f9964009b30 tx=0x7f99640042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:22.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.645+0000 7f99627fc640 1 -- 172.21.15.40:0/2739944401 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996403e460 con 0x7f996c157800 2023-10-18T09:02:22.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.645+0000 7f99627fc640 1 -- 172.21.15.40:0/2739944401 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f996403e6b0 con 0x7f996c157800 2023-10-18T09:02:22.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.645+0000 7f99627fc640 1 -- 172.21.15.40:0/2739944401 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996403ea40 con 0x7f996c157800 2023-10-18T09:02:22.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.645+0000 7f99741e9640 1 -- 172.21.15.40:0/2739944401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996c151160 con 0x7f996c157800 2023-10-18T09:02:22.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.645+0000 7f99741e9640 1 -- 172.21.15.40:0/2739944401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996c151720 con 0x7f996c157800 2023-10-18T09:02:22.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.646+0000 7f99741e9640 1 -- 172.21.15.40:0/2739944401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996c151910 con 0x7f996c157800 2023-10-18T09:02:22.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.648+0000 7f99627fc640 1 -- 172.21.15.40:0/2739944401 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f996403eba0 con 0x7f996c157800 2023-10-18T09:02:22.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.648+0000 7f99627fc640 1 --2- 172.21.15.40:0/2739944401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f995006ae40 0x7f995006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.649+0000 7f99627fc640 1 -- 172.21.15.40:0/2739944401 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f99640ac930 con 0x7f996c157800 2023-10-18T09:02:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.649+0000 7f9971f85640 1 --2- 172.21.15.40:0/2739944401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f995006ae40 0x7f995006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:22.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.650+0000 7f9971f85640 1 --2- 172.21.15.40:0/2739944401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f995006ae40 0x7f995006d300 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9968004ea0 tx=0x7f9968009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:22.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.653+0000 7f99627fc640 1 -- 172.21.15.40:0/2739944401 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9964077890 con 0x7f996c157800 2023-10-18T09:02:22.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.846+0000 7f99741e9640 1 -- 172.21.15.40:0/2739944401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f996c066fd0 con 0x7f996c157800 2023-10-18T09:02:22.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.847+0000 7f99627fc640 1 -- 172.21.15.40:0/2739944401 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+11 (secure 0 0 0) 0x7f99640420a0 con 0x7f996c157800 2023-10-18T09:02:22.850 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:02:22.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.849+0000 7f99741e9640 1 -- 172.21.15.40:0/2739944401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f995006ae40 msgr2=0x7f995006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:22.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.849+0000 7f99741e9640 1 --2- 172.21.15.40:0/2739944401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f995006ae40 0x7f995006d300 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9968004ea0 tx=0x7f9968009040 comp rx=0 tx=0).stop 2023-10-18T09:02:22.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.850+0000 7f99741e9640 1 -- 172.21.15.40:0/2739944401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f996c157800 msgr2=0x7f996c14a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:22.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.850+0000 7f99741e9640 1 --2- 172.21.15.40:0/2739944401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f996c157800 0x7f996c14a220 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f9964009b30 tx=0x7f99640042e0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.850+0000 7f99741e9640 1 -- 172.21.15.40:0/2739944401 shutdown_connections 2023-10-18T09:02:22.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.850+0000 7f99741e9640 1 --2- 172.21.15.40:0/2739944401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f995006ae40 0x7f995006d300 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.850+0000 7f99741e9640 1 --2- 172.21.15.40:0/2739944401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f996c157800 0x7f996c14a220 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.850+0000 7f99741e9640 1 --2- 172.21.15.40:0/2739944401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c0cb590 0x7f996c149ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.850+0000 7f99741e9640 1 --2- 172.21.15.40:0/2739944401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f996c06d660 0x7f996c14fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:22.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.850+0000 7f99741e9640 1 -- 172.21.15.40:0/2739944401 >> 172.21.15.40:0/2739944401 conn(0x7f996c069720 msgr2=0x7f996c1548f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:22.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.850+0000 7f99741e9640 1 -- 172.21.15.40:0/2739944401 shutdown_connections 2023-10-18T09:02:22.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:22.850+0000 7f99741e9640 1 -- 172.21.15.40:0/2739944401 wait complete. 2023-10-18T09:02:22.873 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:02:23.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.093+0000 7f6b52b3c640 1 -- 172.21.15.40:0/2114156566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b4c157800 msgr2=0x7f6b4c0b1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:23.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.093+0000 7f6b52b3c640 1 --2- 172.21.15.40:0/2114156566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b4c157800 0x7f6b4c0b1cc0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0acb90 tx=0x7f6b3c02c260 comp rx=0 tx=0).stop 2023-10-18T09:02:23.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.093+0000 7f6b52b3c640 1 -- 172.21.15.40:0/2114156566 shutdown_connections 2023-10-18T09:02:23.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.093+0000 7f6b52b3c640 1 --2- 172.21.15.40:0/2114156566 >> v2:172.21.15.144:3300/0 conn(0x7f6b4c0cca30 0x7f6b4c0ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.093+0000 7f6b52b3c640 1 --2- 172.21.15.40:0/2114156566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b4c157800 0x7f6b4c0b1cc0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.093+0000 7f6b52b3c640 1 -- 172.21.15.40:0/2114156566 >> 172.21.15.40:0/2114156566 conn(0x7f6b4c069720 msgr2=0x7f6b4c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:23.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.093+0000 7f6b52b3c640 1 -- 172.21.15.40:0/2114156566 shutdown_connections 2023-10-18T09:02:23.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.093+0000 7f6b52b3c640 1 -- 172.21.15.40:0/2114156566 wait complete. 2023-10-18T09:02:23.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.096+0000 7f6b52b3c640 1 Processor -- start 2023-10-18T09:02:23.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.096+0000 7f6b52b3c640 1 -- start start 2023-10-18T09:02:23.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.097+0000 7f6b52b3c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b4c0cca30 0x7f6b4c0ac420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:23.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.097+0000 7f6b52b3c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c0cd3e0 0x7f6b4c14df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:23.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.097+0000 7f6b52b3c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b4c157800 0x7f6b4c148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:23.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.097+0000 7f6b52b3c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c1526d0 con 0x7f6b4c0cca30 2023-10-18T09:02:23.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.097+0000 7f6b52b3c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c1511b0 con 0x7f6b4c0cd3e0 2023-10-18T09:02:23.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.097+0000 7f6b52b3c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c152f50 con 0x7f6b4c157800 2023-10-18T09:02:23.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.098+0000 7f6b508d8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b4c0cca30 0x7f6b4c0ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:23.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.098+0000 7f6b508d8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b4c0cca30 0x7f6b4c0ac420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34426/0 (socket says 172.21.15.40:34426) 2023-10-18T09:02:23.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.098+0000 7f6b508d8640 1 -- 172.21.15.40:0/1130256508 learned_addr learned my addr 172.21.15.40:0/1130256508 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:23.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.098+0000 7f6b508d8640 1 -- 172.21.15.40:0/1130256508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b4c157800 msgr2=0x7f6b4c148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:23.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.098+0000 7f6b4bfff640 1 --2- 172.21.15.40:0/1130256508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c0cd3e0 0x7f6b4c14df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:23.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.098+0000 7f6b508d8640 1 --2- 172.21.15.40:0/1130256508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b4c157800 0x7f6b4c148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.098+0000 7f6b508d8640 1 -- 172.21.15.40:0/1130256508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c0cd3e0 msgr2=0x7f6b4c14df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:23.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.098+0000 7f6b508d8640 1 --2- 172.21.15.40:0/1130256508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c0cd3e0 0x7f6b4c14df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.098+0000 7f6b508d8640 1 -- 172.21.15.40:0/1130256508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c009900 con 0x7f6b4c0cca30 2023-10-18T09:02:23.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.098+0000 7f6b508d8640 1 --2- 172.21.15.40:0/1130256508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b4c0cca30 0x7f6b4c0ac420 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009bf0 tx=0x7f6b3c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:23.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.099+0000 7f6b497fa640 1 -- 172.21.15.40:0/1130256508 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c004450 con 0x7f6b4c0cca30 2023-10-18T09:02:23.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.099+0000 7f6b497fa640 1 -- 172.21.15.40:0/1130256508 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6b3c0046a0 con 0x7f6b4c0cca30 2023-10-18T09:02:23.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.099+0000 7f6b497fa640 1 -- 172.21.15.40:0/1130256508 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b3c03d650 con 0x7f6b4c0cca30 2023-10-18T09:02:23.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.099+0000 7f6b52b3c640 1 -- 172.21.15.40:0/1130256508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4c0ac960 con 0x7f6b4c0cca30 2023-10-18T09:02:23.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.099+0000 7f6b52b3c640 1 -- 172.21.15.40:0/1130256508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4c152900 con 0x7f6b4c0cca30 2023-10-18T09:02:23.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.100+0000 7f6b52b3c640 1 -- 172.21.15.40:0/1130256508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4c066fd0 con 0x7f6b4c0cca30 2023-10-18T09:02:23.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.101+0000 7f6b497fa640 1 -- 172.21.15.40:0/1130256508 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6b3c03b040 con 0x7f6b4c0cca30 2023-10-18T09:02:23.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.102+0000 7f6b497fa640 1 --2- 172.21.15.40:0/1130256508 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b2c06ae40 0x7f6b2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:23.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.102+0000 7f6b497fa640 1 -- 172.21.15.40:0/1130256508 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f6b3c0ac1b0 con 0x7f6b4c0cca30 2023-10-18T09:02:23.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.103+0000 7f6b4bfff640 1 --2- 172.21.15.40:0/1130256508 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b2c06ae40 0x7f6b2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:23.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.104+0000 7f6b4bfff640 1 --2- 172.21.15.40:0/1130256508 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b2c06ae40 0x7f6b2c06d300 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6b340058c0 tx=0x7f6b34005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:23.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.106+0000 7f6b497fa640 1 -- 172.21.15.40:0/1130256508 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b3c077110 con 0x7f6b4c0cca30 2023-10-18T09:02:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.301+0000 7f6b52b3c640 1 -- 172.21.15.40:0/1130256508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b4c0cc450 con 0x7f6b4c0cca30 2023-10-18T09:02:23.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.302+0000 7f6b497fa640 1 -- 172.21.15.40:0/1130256508 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b4c0cc450 con 0x7f6b4c0cca30 2023-10-18T09:02:23.305 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:02:23.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.305+0000 7f6b52b3c640 1 -- 172.21.15.40:0/1130256508 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b2c06ae40 msgr2=0x7f6b2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:23.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.305+0000 7f6b52b3c640 1 --2- 172.21.15.40:0/1130256508 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b2c06ae40 0x7f6b2c06d300 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6b340058c0 tx=0x7f6b34005850 comp rx=0 tx=0).stop 2023-10-18T09:02:23.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.305+0000 7f6b52b3c640 1 -- 172.21.15.40:0/1130256508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b4c0cca30 msgr2=0x7f6b4c0ac420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:23.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.305+0000 7f6b52b3c640 1 --2- 172.21.15.40:0/1130256508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b4c0cca30 0x7f6b4c0ac420 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009bf0 tx=0x7f6b3c003c20 comp rx=0 tx=0).stop 2023-10-18T09:02:23.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.305+0000 7f6b52b3c640 1 -- 172.21.15.40:0/1130256508 shutdown_connections 2023-10-18T09:02:23.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.305+0000 7f6b52b3c640 1 --2- 172.21.15.40:0/1130256508 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b2c06ae40 0x7f6b2c06d300 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.305+0000 7f6b52b3c640 1 --2- 172.21.15.40:0/1130256508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b4c157800 0x7f6b4c148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.305+0000 7f6b52b3c640 1 --2- 172.21.15.40:0/1130256508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c0cd3e0 0x7f6b4c14df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.305+0000 7f6b52b3c640 1 --2- 172.21.15.40:0/1130256508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b4c0cca30 0x7f6b4c0ac420 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.305+0000 7f6b52b3c640 1 -- 172.21.15.40:0/1130256508 >> 172.21.15.40:0/1130256508 conn(0x7f6b4c069720 msgr2=0x7f6b4c0cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:23.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.305+0000 7f6b52b3c640 1 -- 172.21.15.40:0/1130256508 shutdown_connections 2023-10-18T09:02:23.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.305+0000 7f6b52b3c640 1 -- 172.21.15.40:0/1130256508 wait complete. 2023-10-18T09:02:23.328 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:02:23.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.562+0000 7f76a69e2640 1 -- 172.21.15.40:0/2731080374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a00ca160 msgr2=0x7f76a00cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:23.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.562+0000 7f76a69e2640 1 --2- 172.21.15.40:0/2731080374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a00ca160 0x7f76a00cafc0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f7698005fd0 tx=0x7f769802c2d0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.562+0000 7f76a69e2640 1 -- 172.21.15.40:0/2731080374 shutdown_connections 2023-10-18T09:02:23.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.562+0000 7f76a69e2640 1 --2- 172.21.15.40:0/2731080374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a00ca160 0x7f76a00cafc0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.562+0000 7f76a69e2640 1 --2- 172.21.15.40:0/2731080374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76a01420f0 0x7f76a00b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.562+0000 7f76a69e2640 1 -- 172.21.15.40:0/2731080374 >> 172.21.15.40:0/2731080374 conn(0x7f76a0069720 msgr2=0x7f76a0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:23.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.562+0000 7f76a69e2640 1 -- 172.21.15.40:0/2731080374 shutdown_connections 2023-10-18T09:02:23.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.562+0000 7f76a69e2640 1 -- 172.21.15.40:0/2731080374 wait complete. 2023-10-18T09:02:23.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.564+0000 7f76a69e2640 1 Processor -- start 2023-10-18T09:02:23.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.564+0000 7f76a69e2640 1 -- start start 2023-10-18T09:02:23.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.565+0000 7f76a69e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a00ca160 0x7f76a01f0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:23.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.565+0000 7f76a69e2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76a01420f0 0x7f76a00a6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:23.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.565+0000 7f76a69e2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a01425a0 0x7f76a013ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:23.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.565+0000 7f76a69e2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a01f3e30 con 0x7f76a00ca160 2023-10-18T09:02:23.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.565+0000 7f76a69e2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a01efd60 con 0x7f76a01425a0 2023-10-18T09:02:23.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.565+0000 7f76a69e2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a01f0040 con 0x7f76a01420f0 2023-10-18T09:02:23.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.565+0000 7f76a59e0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a00ca160 0x7f76a01f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:23.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.565+0000 7f76a59e0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a00ca160 0x7f76a01f0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34436/0 (socket says 172.21.15.40:34436) 2023-10-18T09:02:23.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.565+0000 7f76a59e0640 1 -- 172.21.15.40:0/4014802256 learned_addr learned my addr 172.21.15.40:0/4014802256 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:23.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.566+0000 7f76a61e1640 1 --2- 172.21.15.40:0/4014802256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a01425a0 0x7f76a013ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:23.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.566+0000 7f76a59e0640 1 -- 172.21.15.40:0/4014802256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76a01420f0 msgr2=0x7f76a00a6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.566+0000 7f76a51df640 1 --2- 172.21.15.40:0/4014802256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76a01420f0 0x7f76a00a6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.566+0000 7f76a59e0640 1 --2- 172.21.15.40:0/4014802256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76a01420f0 0x7f76a00a6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.566+0000 7f76a59e0640 1 -- 172.21.15.40:0/4014802256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a01425a0 msgr2=0x7f76a013ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.566+0000 7f76a59e0640 1 --2- 172.21.15.40:0/4014802256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a01425a0 0x7f76a013ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.566+0000 7f76a59e0640 1 -- 172.21.15.40:0/4014802256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7698009900 con 0x7f76a00ca160 2023-10-18T09:02:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.566+0000 7f76a61e1640 1 --2- 172.21.15.40:0/4014802256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a01425a0 0x7f76a013ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.566+0000 7f76a51df640 1 --2- 172.21.15.40:0/4014802256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76a01420f0 0x7f76a00a6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:23.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.566+0000 7f76a59e0640 1 --2- 172.21.15.40:0/4014802256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a00ca160 0x7f76a01f0840 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f769c00d950 tx=0x7f769c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:23.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.566+0000 7f76967fc640 1 -- 172.21.15.40:0/4014802256 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c009c30 con 0x7f76a00ca160 2023-10-18T09:02:23.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.567+0000 7f76967fc640 1 -- 172.21.15.40:0/4014802256 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f769c009e80 con 0x7f76a00ca160 2023-10-18T09:02:23.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.567+0000 7f76967fc640 1 -- 172.21.15.40:0/4014802256 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c0106c0 con 0x7f76a00ca160 2023-10-18T09:02:23.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.567+0000 7f76a69e2640 1 -- 172.21.15.40:0/4014802256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a01f0320 con 0x7f76a00ca160 2023-10-18T09:02:23.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.567+0000 7f76a69e2640 1 -- 172.21.15.40:0/4014802256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a00a7380 con 0x7f76a00ca160 2023-10-18T09:02:23.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.568+0000 7f76a69e2640 1 -- 172.21.15.40:0/4014802256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a0066e00 con 0x7f76a00ca160 2023-10-18T09:02:23.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.569+0000 7f76967fc640 1 -- 172.21.15.40:0/4014802256 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f769c010820 con 0x7f76a00ca160 2023-10-18T09:02:23.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.570+0000 7f76967fc640 1 --2- 172.21.15.40:0/4014802256 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f768406ae40 0x7f768406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:23.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.570+0000 7f76967fc640 1 -- 172.21.15.40:0/4014802256 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f769c089c50 con 0x7f76a00ca160 2023-10-18T09:02:23.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.571+0000 7f76a51df640 1 --2- 172.21.15.40:0/4014802256 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f768406ae40 0x7f768406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:23.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.572+0000 7f76a51df640 1 --2- 172.21.15.40:0/4014802256 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f768406ae40 0x7f768406d300 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f76a013d530 tx=0x7f7690009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:23.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.574+0000 7f76967fc640 1 -- 172.21.15.40:0/4014802256 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f769c054bb0 con 0x7f76a00ca160 2023-10-18T09:02:23.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.807+0000 7f76a69e2640 1 -- 172.21.15.40:0/4014802256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76a006a260 con 0x7f76a00ca160 2023-10-18T09:02:23.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.808+0000 7f76967fc640 1 -- 172.21.15.40:0/4014802256 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f769c017090 con 0x7f76a00ca160 2023-10-18T09:02:23.812 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:23.812 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2565609773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2565609773},{"type":"v1","addr":"172.21.15.144:6836","nonce":2565609773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":5872,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3371548374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3371548374},{"type":"v1","addr":"172.21.15.144:6837","nonce":3371548374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3328648628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3328648628},{"type":"v1","addr":"172.21.15.40:6833","nonce":3328648628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:01:36.995669+0000","modified":"2023-10-18T09:01:40.068484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/891188043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":891188043},{"type":"v1","addr":"172.21.15.40:6835","nonce":891188043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-18T09:02:23.812 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 49 2023-10-18T09:02:23.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.811+0000 7f76a69e2640 1 -- 172.21.15.40:0/4014802256 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f768406ae40 msgr2=0x7f768406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:23.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.811+0000 7f76a69e2640 1 --2- 172.21.15.40:0/4014802256 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f768406ae40 0x7f768406d300 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f76a013d530 tx=0x7f7690009040 comp rx=0 tx=0).stop 2023-10-18T09:02:23.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.811+0000 7f76a69e2640 1 -- 172.21.15.40:0/4014802256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a00ca160 msgr2=0x7f76a01f0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:23.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.811+0000 7f76a69e2640 1 --2- 172.21.15.40:0/4014802256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a00ca160 0x7f76a01f0840 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f769c00d950 tx=0x7f769c00de10 comp rx=0 tx=0).stop 2023-10-18T09:02:23.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.811+0000 7f76a69e2640 1 -- 172.21.15.40:0/4014802256 shutdown_connections 2023-10-18T09:02:23.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.811+0000 7f76a69e2640 1 --2- 172.21.15.40:0/4014802256 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f768406ae40 0x7f768406d300 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.811+0000 7f76a69e2640 1 --2- 172.21.15.40:0/4014802256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76a01425a0 0x7f76a013ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.811+0000 7f76a69e2640 1 --2- 172.21.15.40:0/4014802256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f76a01420f0 0x7f76a00a6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.811+0000 7f76a69e2640 1 --2- 172.21.15.40:0/4014802256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76a00ca160 0x7f76a01f0840 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:23.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.811+0000 7f76a69e2640 1 -- 172.21.15.40:0/4014802256 >> 172.21.15.40:0/4014802256 conn(0x7f76a0069720 msgr2=0x7f76a01410d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:23.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.811+0000 7f76a69e2640 1 -- 172.21.15.40:0/4014802256 shutdown_connections 2023-10-18T09:02:23.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:23.811+0000 7f76a69e2640 1 -- 172.21.15.40:0/4014802256 wait complete. 2023-10-18T09:02:23.834 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:02:24.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.061+0000 7f0db7d25640 1 -- 172.21.15.40:0/4135245785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db006b710 msgr2=0x7f0db006bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:24.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.061+0000 7f0db7d25640 1 --2- 172.21.15.40:0/4135245785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db006b710 0x7f0db006bb80 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009be0 tx=0x7f0da802c0c0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.061+0000 7f0db7d25640 1 -- 172.21.15.40:0/4135245785 shutdown_connections 2023-10-18T09:02:24.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.061+0000 7f0db7d25640 1 --2- 172.21.15.40:0/4135245785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db006b710 0x7f0db006bb80 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.061+0000 7f0db7d25640 1 --2- 172.21.15.40:0/4135245785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0db015a470 0x7f0db00c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.061+0000 7f0db7d25640 1 -- 172.21.15.40:0/4135245785 >> 172.21.15.40:0/4135245785 conn(0x7f0db006cee0 msgr2=0x7f0db0069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:24.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.061+0000 7f0db7d25640 1 -- 172.21.15.40:0/4135245785 shutdown_connections 2023-10-18T09:02:24.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.061+0000 7f0db7d25640 1 -- 172.21.15.40:0/4135245785 wait complete. 2023-10-18T09:02:24.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.063+0000 7f0db7d25640 1 Processor -- start 2023-10-18T09:02:24.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.063+0000 7f0db7d25640 1 -- start start 2023-10-18T09:02:24.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.064+0000 7f0db7d25640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0db006b710 0x7f0db015d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:24.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.064+0000 7f0db7d25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00cdbb0 0x7f0db015a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:24.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.064+0000 7f0db7d25640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db015a470 0x7f0db01e3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:24.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.064+0000 7f0db7d25640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db01e55c0 con 0x7f0db00cdbb0 2023-10-18T09:02:24.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.064+0000 7f0db7d25640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db015d740 con 0x7f0db015a470 2023-10-18T09:02:24.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.064+0000 7f0db7d25640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db00ab8c0 con 0x7f0db006b710 2023-10-18T09:02:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.064+0000 7f0db52c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00cdbb0 0x7f0db015a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.064+0000 7f0db52c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00cdbb0 0x7f0db015a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34458/0 (socket says 172.21.15.40:34458) 2023-10-18T09:02:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.065+0000 7f0db52c0640 1 -- 172.21.15.40:0/3047106555 learned_addr learned my addr 172.21.15.40:0/3047106555 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.065+0000 7f0db5ac1640 1 --2- 172.21.15.40:0/3047106555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0db006b710 0x7f0db015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.065+0000 7f0db62c2640 1 --2- 172.21.15.40:0/3047106555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db015a470 0x7f0db01e3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.065+0000 7f0db52c0640 1 -- 172.21.15.40:0/3047106555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0db006b710 msgr2=0x7f0db015d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.065+0000 7f0db52c0640 1 --2- 172.21.15.40:0/3047106555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0db006b710 0x7f0db015d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.065+0000 7f0db52c0640 1 -- 172.21.15.40:0/3047106555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db015a470 msgr2=0x7f0db01e3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.065+0000 7f0db52c0640 1 --2- 172.21.15.40:0/3047106555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db015a470 0x7f0db01e3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.065+0000 7f0db52c0640 1 -- 172.21.15.40:0/3047106555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da8009840 con 0x7f0db00cdbb0 2023-10-18T09:02:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.065+0000 7f0db52c0640 1 --2- 172.21.15.40:0/3047106555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00cdbb0 0x7f0db015a9c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f0da000c970 tx=0x7f0da000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.066+0000 7f0da67fc640 1 -- 172.21.15.40:0/3047106555 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da0007b50 con 0x7f0db00cdbb0 2023-10-18T09:02:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.066+0000 7f0da67fc640 1 -- 172.21.15.40:0/3047106555 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0da0007da0 con 0x7f0db00cdbb0 2023-10-18T09:02:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.066+0000 7f0da67fc640 1 -- 172.21.15.40:0/3047106555 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da000f6b0 con 0x7f0db00cdbb0 2023-10-18T09:02:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.066+0000 7f0db7d25640 1 -- 172.21.15.40:0/3047106555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db00abba0 con 0x7f0db00cdbb0 2023-10-18T09:02:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.066+0000 7f0db7d25640 1 -- 172.21.15.40:0/3047106555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db01e51e0 con 0x7f0db00cdbb0 2023-10-18T09:02:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.067+0000 7f0db7d25640 1 -- 172.21.15.40:0/3047106555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db0066fd0 con 0x7f0db00cdbb0 2023-10-18T09:02:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.068+0000 7f0da67fc640 1 -- 172.21.15.40:0/3047106555 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0da000f810 con 0x7f0db00cdbb0 2023-10-18T09:02:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.069+0000 7f0da67fc640 1 --2- 172.21.15.40:0/3047106555 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0d9406ae40 0x7f0d9406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.069+0000 7f0da67fc640 1 -- 172.21.15.40:0/3047106555 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f0da0089fb0 con 0x7f0db00cdbb0 2023-10-18T09:02:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.070+0000 7f0db5ac1640 1 --2- 172.21.15.40:0/3047106555 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0d9406ae40 0x7f0d9406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:24.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.071+0000 7f0db5ac1640 1 --2- 172.21.15.40:0/3047106555 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0d9406ae40 0x7f0d9406d300 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0dac007fd0 tx=0x7f0dac009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.074+0000 7f0da67fc640 1 -- 172.21.15.40:0/3047106555 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0da0054e80 con 0x7f0db00cdbb0 2023-10-18T09:02:24.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.272+0000 7f0db7d25640 1 -- 172.21.15.40:0/3047106555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0db006bd00 con 0x7f0db00cdbb0 2023-10-18T09:02:24.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.275+0000 7f0da67fc640 1 -- 172.21.15.40:0/3047106555 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16873 (secure 0 0 0) 0x7f0da001d020 con 0x7f0db00cdbb0 2023-10-18T09:02:24.279 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:24.279 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":118,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:02:09.285724+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:01:35.453236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-18T09:01:36.012460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1","mappings":[{"from":2,"to":7}]},{"pgid":"13.14","mappings":[{"from":6,"to":5}]},{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.20","mappings":[{"from":1,"to":5}]},{"pgid":"13.34","mappings":[{"from":6,"to":5}]},{"pgid":"13.38","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:02:24.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.278+0000 7f0db7d25640 1 -- 172.21.15.40:0/3047106555 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0d9406ae40 msgr2=0x7f0d9406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:24.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.278+0000 7f0db7d25640 1 --2- 172.21.15.40:0/3047106555 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0d9406ae40 0x7f0d9406d300 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0dac007fd0 tx=0x7f0dac009040 comp rx=0 tx=0).stop 2023-10-18T09:02:24.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.278+0000 7f0db7d25640 1 -- 172.21.15.40:0/3047106555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00cdbb0 msgr2=0x7f0db015a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:24.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.278+0000 7f0db7d25640 1 --2- 172.21.15.40:0/3047106555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00cdbb0 0x7f0db015a9c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f0da000c970 tx=0x7f0da000ce30 comp rx=0 tx=0).stop 2023-10-18T09:02:24.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.278+0000 7f0db7d25640 1 -- 172.21.15.40:0/3047106555 shutdown_connections 2023-10-18T09:02:24.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.278+0000 7f0db7d25640 1 --2- 172.21.15.40:0/3047106555 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0d9406ae40 0x7f0d9406d300 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.278+0000 7f0db7d25640 1 --2- 172.21.15.40:0/3047106555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0db015a470 0x7f0db01e3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.278+0000 7f0db7d25640 1 --2- 172.21.15.40:0/3047106555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0db00cdbb0 0x7f0db015a9c0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.278+0000 7f0db7d25640 1 --2- 172.21.15.40:0/3047106555 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0db006b710 0x7f0db015d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.278+0000 7f0db7d25640 1 -- 172.21.15.40:0/3047106555 >> 172.21.15.40:0/3047106555 conn(0x7f0db006cee0 msgr2=0x7f0db0069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:24.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.278+0000 7f0db7d25640 1 -- 172.21.15.40:0/3047106555 shutdown_connections 2023-10-18T09:02:24.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.279+0000 7f0db7d25640 1 -- 172.21.15.40:0/3047106555 wait complete. 2023-10-18T09:02:24.301 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T09:02:24.302 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:02:24.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.534+0000 7feb14e58640 1 -- 172.21.15.40:0/3897816010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb100c9b00 msgr2=0x7feb100c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:24.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.534+0000 7feb14e58640 1 --2- 172.21.15.40:0/3897816010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb100c9b00 0x7feb100c9f90 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7feb1006c860 tx=0x7feb0802bd90 comp rx=0 tx=0).stop 2023-10-18T09:02:24.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.535+0000 7feb14e58640 1 -- 172.21.15.40:0/3897816010 shutdown_connections 2023-10-18T09:02:24.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.535+0000 7feb14e58640 1 --2- 172.21.15.40:0/3897816010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb100c9b00 0x7feb100c9f90 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.535+0000 7feb14e58640 1 --2- 172.21.15.40:0/3897816010 >> v2:172.21.15.144:3300/0 conn(0x7feb1015cf30 0x7feb1015d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.535+0000 7feb14e58640 1 --2- 172.21.15.40:0/3897816010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb1013c470 0x7feb1013f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.535+0000 7feb14e58640 1 -- 172.21.15.40:0/3897816010 >> 172.21.15.40:0/3897816010 conn(0x7feb1006cee0 msgr2=0x7feb10069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:24.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.535+0000 7feb14e58640 1 -- 172.21.15.40:0/3897816010 shutdown_connections 2023-10-18T09:02:24.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.535+0000 7feb14e58640 1 -- 172.21.15.40:0/3897816010 wait complete. 2023-10-18T09:02:24.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.536+0000 7feb14e58640 1 Processor -- start 2023-10-18T09:02:24.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.537+0000 7feb14e58640 1 -- start start 2023-10-18T09:02:24.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.538+0000 7feb14e58640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb100c9b00 0x7feb10158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:24.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.538+0000 7feb14e58640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb1013c470 0x7feb1013b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:24.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.538+0000 7feb14e58640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb1015cf30 0x7feb1013a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:24.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.538+0000 7feb14e58640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb10140330 con 0x7feb1015cf30 2023-10-18T09:02:24.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.538+0000 7feb14e58640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb10140050 con 0x7feb1013c470 2023-10-18T09:02:24.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.538+0000 7feb14e58640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb100acad0 con 0x7feb100c9b00 2023-10-18T09:02:24.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.538+0000 7feb0ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb1015cf30 0x7feb1013a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:24.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.538+0000 7feb0ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb1015cf30 0x7feb1013a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34482/0 (socket says 172.21.15.40:34482) 2023-10-18T09:02:24.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.538+0000 7feb0ed9d640 1 -- 172.21.15.40:0/2567139126 learned_addr learned my addr 172.21.15.40:0/2567139126 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:24.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.538+0000 7feb0e59c640 1 --2- 172.21.15.40:0/2567139126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb100c9b00 0x7feb10158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:24.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.538+0000 7feb0dd9b640 1 --2- 172.21.15.40:0/2567139126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb1013c470 0x7feb1013b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:24.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.539+0000 7feb0ed9d640 1 -- 172.21.15.40:0/2567139126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb100c9b00 msgr2=0x7feb10158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:24.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.539+0000 7feb0ed9d640 1 --2- 172.21.15.40:0/2567139126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb100c9b00 0x7feb10158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.539+0000 7feb0ed9d640 1 -- 172.21.15.40:0/2567139126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb1013c470 msgr2=0x7feb1013b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:24.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.539+0000 7feb0ed9d640 1 --2- 172.21.15.40:0/2567139126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb1013c470 0x7feb1013b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.539+0000 7feb0ed9d640 1 -- 172.21.15.40:0/2567139126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb08009560 con 0x7feb1015cf30 2023-10-18T09:02:24.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.539+0000 7feb0ed9d640 1 --2- 172.21.15.40:0/2567139126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb1015cf30 0x7feb1013a1f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7feb08036040 tx=0x7feb0802dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:24.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.540+0000 7feb06ffd640 1 -- 172.21.15.40:0/2567139126 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb08005d50 con 0x7feb1015cf30 2023-10-18T09:02:24.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.540+0000 7feb06ffd640 1 -- 172.21.15.40:0/2567139126 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feb080038f0 con 0x7feb1015cf30 2023-10-18T09:02:24.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.540+0000 7feb06ffd640 1 -- 172.21.15.40:0/2567139126 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb08003d00 con 0x7feb1015cf30 2023-10-18T09:02:24.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.540+0000 7feb14e58640 1 -- 172.21.15.40:0/2567139126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb100acdb0 con 0x7feb1015cf30 2023-10-18T09:02:24.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.540+0000 7feb14e58640 1 -- 172.21.15.40:0/2567139126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb10158c40 con 0x7feb1015cf30 2023-10-18T09:02:24.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.541+0000 7feb14e58640 1 -- 172.21.15.40:0/2567139126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb100ac930 con 0x7feb1015cf30 2023-10-18T09:02:24.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.542+0000 7feb06ffd640 1 -- 172.21.15.40:0/2567139126 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feb0803e450 con 0x7feb1015cf30 2023-10-18T09:02:24.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.543+0000 7feb06ffd640 1 --2- 172.21.15.40:0/2567139126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feaf006ae40 0x7feaf006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:24.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.543+0000 7feb06ffd640 1 -- 172.21.15.40:0/2567139126 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7feb080ac120 con 0x7feb1015cf30 2023-10-18T09:02:24.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.544+0000 7feb0e59c640 1 --2- 172.21.15.40:0/2567139126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feaf006ae40 0x7feaf006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:24.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.545+0000 7feb0e59c640 1 --2- 172.21.15.40:0/2567139126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feaf006ae40 0x7feaf006d300 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7feb00005fd0 tx=0x7feb000095e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:24.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.547+0000 7feb06ffd640 1 -- 172.21.15.40:0/2567139126 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb100ac930 con 0x7feb1015cf30 2023-10-18T09:02:24.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.779+0000 7feb14e58640 1 -- 172.21.15.40:0/2567139126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb100ac930 con 0x7feb1015cf30 2023-10-18T09:02:24.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.780+0000 7feb06ffd640 1 -- 172.21.15.40:0/2567139126 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4918 (secure 0 0 0) 0x7feb100ac930 con 0x7feb1015cf30 2023-10-18T09:02:24.783 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:24.784 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2565609773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2565609773},{"type":"v1","addr":"172.21.15.144:6836","nonce":2565609773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":5872,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3371548374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3371548374},{"type":"v1","addr":"172.21.15.144:6837","nonce":3371548374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3328648628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3328648628},{"type":"v1","addr":"172.21.15.40:6833","nonce":3328648628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:01:36.995669+0000","modified":"2023-10-18T09:01:40.068484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/891188043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":891188043},{"type":"v1","addr":"172.21.15.40:6835","nonce":891188043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-18T09:02:24.784 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 49 2023-10-18T09:02:24.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.783+0000 7feb14e58640 1 -- 172.21.15.40:0/2567139126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feaf006ae40 msgr2=0x7feaf006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.783+0000 7feb14e58640 1 --2- 172.21.15.40:0/2567139126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feaf006ae40 0x7feaf006d300 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7feb00005fd0 tx=0x7feb000095e0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.783+0000 7feb14e58640 1 -- 172.21.15.40:0/2567139126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb1015cf30 msgr2=0x7feb1013a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.783+0000 7feb14e58640 1 --2- 172.21.15.40:0/2567139126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb1015cf30 0x7feb1013a1f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7feb08036040 tx=0x7feb0802dce0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.783+0000 7feb14e58640 1 -- 172.21.15.40:0/2567139126 shutdown_connections 2023-10-18T09:02:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.783+0000 7feb14e58640 1 --2- 172.21.15.40:0/2567139126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feaf006ae40 0x7feaf006d300 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.783+0000 7feb14e58640 1 --2- 172.21.15.40:0/2567139126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb1015cf30 0x7feb1013a1f0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.783+0000 7feb14e58640 1 --2- 172.21.15.40:0/2567139126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb1013c470 0x7feb1013b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.783+0000 7feb14e58640 1 --2- 172.21.15.40:0/2567139126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb100c9b00 0x7feb10158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:24.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.783+0000 7feb14e58640 1 -- 172.21.15.40:0/2567139126 >> 172.21.15.40:0/2567139126 conn(0x7feb1006cee0 msgr2=0x7feb10069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:24.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.783+0000 7feb14e58640 1 -- 172.21.15.40:0/2567139126 shutdown_connections 2023-10-18T09:02:24.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:24.784+0000 7feb14e58640 1 -- 172.21.15.40:0/2567139126 wait complete. 2023-10-18T09:02:24.807 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:02:25.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.033+0000 7fdf309e2640 1 -- 172.21.15.40:0/987713303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c0c9f90 msgr2=0x7fdf2c06b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:25.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.033+0000 7fdf309e2640 1 --2- 172.21.15.40:0/987713303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c0c9f90 0x7fdf2c06b4d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c06af60 tx=0x7fdf2402c240 comp rx=0 tx=0).stop 2023-10-18T09:02:25.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.033+0000 7fdf309e2640 1 -- 172.21.15.40:0/987713303 shutdown_connections 2023-10-18T09:02:25.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.033+0000 7fdf309e2640 1 --2- 172.21.15.40:0/987713303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c0c9f90 0x7fdf2c06b4d0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.033+0000 7fdf309e2640 1 --2- 172.21.15.40:0/987713303 >> v2:172.21.15.144:3300/0 conn(0x7fdf2c1425a0 0x7fdf2c0c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.033+0000 7fdf309e2640 1 --2- 172.21.15.40:0/987713303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf2c1420f0 0x7fdf2c0b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.033+0000 7fdf309e2640 1 -- 172.21.15.40:0/987713303 >> 172.21.15.40:0/987713303 conn(0x7fdf2c069720 msgr2=0x7fdf2c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:25.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.033+0000 7fdf309e2640 1 -- 172.21.15.40:0/987713303 shutdown_connections 2023-10-18T09:02:25.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.033+0000 7fdf309e2640 1 -- 172.21.15.40:0/987713303 wait complete. 2023-10-18T09:02:25.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.035+0000 7fdf309e2640 1 Processor -- start 2023-10-18T09:02:25.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.036+0000 7fdf309e2640 1 -- start start 2023-10-18T09:02:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.037+0000 7fdf309e2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf2c0c9f90 0x7fdf2c1413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.037+0000 7fdf309e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c1420f0 0x7fdf2c158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.037+0000 7fdf309e2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf2c1425a0 0x7fdf2c070650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.037+0000 7fdf309e2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c1406b0 con 0x7fdf2c1420f0 2023-10-18T09:02:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.037+0000 7fdf309e2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c141900 con 0x7fdf2c1425a0 2023-10-18T09:02:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.037+0000 7fdf309e2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c13faa0 con 0x7fdf2c0c9f90 2023-10-18T09:02:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.037+0000 7fdf23fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c1420f0 0x7fdf2c158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.037+0000 7fdf23fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c1420f0 0x7fdf2c158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50096/0 (socket says 172.21.15.40:50096) 2023-10-18T09:02:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.037+0000 7fdf23fff640 1 -- 172.21.15.40:0/3756351734 learned_addr learned my addr 172.21.15.40:0/3756351734 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.038+0000 7fdf2b7fe640 1 --2- 172.21.15.40:0/3756351734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf2c0c9f90 0x7fdf2c1413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.038+0000 7fdf2bfff640 1 --2- 172.21.15.40:0/3756351734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf2c1425a0 0x7fdf2c070650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.038+0000 7fdf23fff640 1 -- 172.21.15.40:0/3756351734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf2c0c9f90 msgr2=0x7fdf2c1413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.038+0000 7fdf23fff640 1 --2- 172.21.15.40:0/3756351734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf2c0c9f90 0x7fdf2c1413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.038+0000 7fdf23fff640 1 -- 172.21.15.40:0/3756351734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf2c1425a0 msgr2=0x7fdf2c070650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:25.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.038+0000 7fdf23fff640 1 --2- 172.21.15.40:0/3756351734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf2c1425a0 0x7fdf2c070650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.038+0000 7fdf23fff640 1 -- 172.21.15.40:0/3756351734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf24009900 con 0x7fdf2c1420f0 2023-10-18T09:02:25.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.038+0000 7fdf2bfff640 1 --2- 172.21.15.40:0/3756351734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf2c1425a0 0x7fdf2c070650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:25.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.038+0000 7fdf23fff640 1 --2- 172.21.15.40:0/3756351734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c1420f0 0x7fdf2c158c90 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400d740 tx=0x7fdf1400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:25.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.038+0000 7fdf28ff9640 1 -- 172.21.15.40:0/3756351734 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14009c30 con 0x7fdf2c1420f0 2023-10-18T09:02:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.039+0000 7fdf28ff9640 1 -- 172.21.15.40:0/3756351734 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdf14009e80 con 0x7fdf2c1420f0 2023-10-18T09:02:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.039+0000 7fdf28ff9640 1 -- 172.21.15.40:0/3756351734 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf140106c0 con 0x7fdf2c1420f0 2023-10-18T09:02:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.039+0000 7fdf309e2640 1 -- 172.21.15.40:0/3756351734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c13fd80 con 0x7fdf2c1420f0 2023-10-18T09:02:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.039+0000 7fdf309e2640 1 -- 172.21.15.40:0/3756351734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c140340 con 0x7fdf2c1420f0 2023-10-18T09:02:25.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.040+0000 7fdf309e2640 1 -- 172.21.15.40:0/3756351734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2c140500 con 0x7fdf2c1420f0 2023-10-18T09:02:25.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.041+0000 7fdf28ff9640 1 -- 172.21.15.40:0/3756351734 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdf14010820 con 0x7fdf2c1420f0 2023-10-18T09:02:25.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.042+0000 7fdf28ff9640 1 --2- 172.21.15.40:0/3756351734 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf0c06ae70 0x7fdf0c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:25.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.042+0000 7fdf28ff9640 1 -- 172.21.15.40:0/3756351734 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7fdf14089b20 con 0x7fdf2c1420f0 2023-10-18T09:02:25.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.043+0000 7fdf2b7fe640 1 --2- 172.21.15.40:0/3756351734 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf0c06ae70 0x7fdf0c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:25.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.044+0000 7fdf2b7fe640 1 --2- 172.21.15.40:0/3756351734 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf0c06ae70 0x7fdf0c06d330 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c1409c0 tx=0x7fdf1c005e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:25.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.047+0000 7fdf28ff9640 1 -- 172.21.15.40:0/3756351734 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf14054bb0 con 0x7fdf2c1420f0 2023-10-18T09:02:25.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.280+0000 7fdf309e2640 1 -- 172.21.15.40:0/3756351734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf2c159950 con 0x7fdf2c1420f0 2023-10-18T09:02:25.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.281+0000 7fdf28ff9640 1 -- 172.21.15.40:0/3756351734 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fdf2c159950 con 0x7fdf2c1420f0 2023-10-18T09:02:25.284 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:25.284 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2565609773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2565609773},{"type":"v1","addr":"172.21.15.144:6836","nonce":2565609773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":5872,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3371548374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3371548374},{"type":"v1","addr":"172.21.15.144:6837","nonce":3371548374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3328648628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3328648628},{"type":"v1","addr":"172.21.15.40:6833","nonce":3328648628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:01:36.995669+0000","modified":"2023-10-18T09:01:40.068484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/891188043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":891188043},{"type":"v1","addr":"172.21.15.40:6835","nonce":891188043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-18T09:02:25.285 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 49 2023-10-18T09:02:25.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.283+0000 7fdf309e2640 1 -- 172.21.15.40:0/3756351734 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf0c06ae70 msgr2=0x7fdf0c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:25.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.283+0000 7fdf309e2640 1 --2- 172.21.15.40:0/3756351734 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf0c06ae70 0x7fdf0c06d330 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c1409c0 tx=0x7fdf1c005e50 comp rx=0 tx=0).stop 2023-10-18T09:02:25.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.283+0000 7fdf309e2640 1 -- 172.21.15.40:0/3756351734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c1420f0 msgr2=0x7fdf2c158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:25.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.283+0000 7fdf309e2640 1 --2- 172.21.15.40:0/3756351734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c1420f0 0x7fdf2c158c90 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400d740 tx=0x7fdf1400dc00 comp rx=0 tx=0).stop 2023-10-18T09:02:25.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.284+0000 7fdf309e2640 1 -- 172.21.15.40:0/3756351734 shutdown_connections 2023-10-18T09:02:25.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.284+0000 7fdf309e2640 1 --2- 172.21.15.40:0/3756351734 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdf0c06ae70 0x7fdf0c06d330 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.284+0000 7fdf309e2640 1 --2- 172.21.15.40:0/3756351734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf2c1425a0 0x7fdf2c070650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.284+0000 7fdf309e2640 1 --2- 172.21.15.40:0/3756351734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf2c1420f0 0x7fdf2c158c90 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.284+0000 7fdf309e2640 1 --2- 172.21.15.40:0/3756351734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf2c0c9f90 0x7fdf2c1413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.284+0000 7fdf309e2640 1 -- 172.21.15.40:0/3756351734 >> 172.21.15.40:0/3756351734 conn(0x7fdf2c069720 msgr2=0x7fdf2c0caf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:25.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.284+0000 7fdf309e2640 1 -- 172.21.15.40:0/3756351734 shutdown_connections 2023-10-18T09:02:25.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.284+0000 7fdf309e2640 1 -- 172.21.15.40:0/3756351734 wait complete. 2023-10-18T09:02:25.304 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:02:25.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.498+0000 7f686b59e640 1 -- 172.21.15.40:0/884176312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c070500 msgr2=0x7f686c15b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.498+0000 7f686b59e640 1 --2- 172.21.15.40:0/884176312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c070500 0x7f686c15b9e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f6854005fd0 tx=0x7f685402bd80 comp rx=0 tx=0).stop 2023-10-18T09:02:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.498+0000 7f686b59e640 1 -- 172.21.15.40:0/884176312 shutdown_connections 2023-10-18T09:02:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.498+0000 7f686b59e640 1 --2- 172.21.15.40:0/884176312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c070500 0x7f686c15b9e0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.498+0000 7f686b59e640 1 --2- 172.21.15.40:0/884176312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f686c144d30 0x7f686c0b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.498+0000 7f686b59e640 1 -- 172.21.15.40:0/884176312 >> 172.21.15.40:0/884176312 conn(0x7f686c0afd80 msgr2=0x7f686c0b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:25.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.498+0000 7f686b59e640 1 -- 172.21.15.40:0/884176312 shutdown_connections 2023-10-18T09:02:25.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.498+0000 7f686b59e640 1 -- 172.21.15.40:0/884176312 wait complete. 2023-10-18T09:02:25.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.500+0000 7f686b59e640 1 Processor -- start 2023-10-18T09:02:25.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.500+0000 7f686b59e640 1 -- start start 2023-10-18T09:02:25.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.501+0000 7f686b59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f686c069720 0x7f686c1f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:25.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.502+0000 7f686b59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f686c070500 0x7f686c13db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:25.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.502+0000 7f686b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c144d30 0x7f686c0c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:25.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.502+0000 7f686b59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1f1dc0 con 0x7f686c144d30 2023-10-18T09:02:25.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.502+0000 7f686b59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c13e050 con 0x7f686c069720 2023-10-18T09:02:25.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.502+0000 7f686b59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686c0c3530 con 0x7f686c070500 2023-10-18T09:02:25.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.502+0000 7f686ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c144d30 0x7f686c0c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:25.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.502+0000 7f686ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c144d30 0x7f686c0c2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50124/0 (socket says 172.21.15.40:50124) 2023-10-18T09:02:25.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.502+0000 7f686ad9d640 1 -- 172.21.15.40:0/465864649 learned_addr learned my addr 172.21.15.40:0/465864649 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:25.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.502+0000 7f6861d9b640 1 --2- 172.21.15.40:0/465864649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f686c070500 0x7f686c13db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:25.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.502+0000 7f686a59c640 1 --2- 172.21.15.40:0/465864649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f686c069720 0x7f686c1f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:25.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.502+0000 7f686ad9d640 1 -- 172.21.15.40:0/465864649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f686c070500 msgr2=0x7f686c13db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:25.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.502+0000 7f686ad9d640 1 --2- 172.21.15.40:0/465864649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f686c070500 0x7f686c13db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.502+0000 7f686ad9d640 1 -- 172.21.15.40:0/465864649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f686c069720 msgr2=0x7f686c1f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:25.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.502+0000 7f686ad9d640 1 --2- 172.21.15.40:0/465864649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f686c069720 0x7f686c1f5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.503+0000 7f686ad9d640 1 -- 172.21.15.40:0/465864649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6854009560 con 0x7f686c144d30 2023-10-18T09:02:25.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.503+0000 7f686ad9d640 1 --2- 172.21.15.40:0/465864649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c144d30 0x7f686c0c2ff0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f686400c7b0 tx=0x7f686400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:25.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.503+0000 7f68637fe640 1 -- 172.21.15.40:0/465864649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864004b60 con 0x7f686c144d30 2023-10-18T09:02:25.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.503+0000 7f68637fe640 1 -- 172.21.15.40:0/465864649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6864004db0 con 0x7f686c144d30 2023-10-18T09:02:25.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.503+0000 7f68637fe640 1 -- 172.21.15.40:0/465864649 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686400aab0 con 0x7f686c144d30 2023-10-18T09:02:25.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.503+0000 7f686b59e640 1 -- 172.21.15.40:0/465864649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686c0c3970 con 0x7f686c144d30 2023-10-18T09:02:25.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.503+0000 7f686b59e640 1 -- 172.21.15.40:0/465864649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686c0c3f30 con 0x7f686c144d30 2023-10-18T09:02:25.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.504+0000 7f686b59e640 1 -- 172.21.15.40:0/465864649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686c1f1f70 con 0x7f686c144d30 2023-10-18T09:02:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.505+0000 7f68637fe640 1 -- 172.21.15.40:0/465864649 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f686400ac10 con 0x7f686c144d30 2023-10-18T09:02:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.507+0000 7f68637fe640 1 --2- 172.21.15.40:0/465864649 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f684c06ae40 0x7f684c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.507+0000 7f68637fe640 1 -- 172.21.15.40:0/465864649 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7f6864089c00 con 0x7f686c144d30 2023-10-18T09:02:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.507+0000 7f686a59c640 1 --2- 172.21.15.40:0/465864649 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f684c06ae40 0x7f684c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.508+0000 7f686a59c640 1 --2- 172.21.15.40:0/465864649 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f684c06ae40 0x7f684c06d300 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f685c005d70 tx=0x7f685c005d00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:25.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.511+0000 7f68637fe640 1 -- 172.21.15.40:0/465864649 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f686c1f1f70 con 0x7f686c144d30 2023-10-18T09:02:25.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.705+0000 7f686b59e640 1 -- 172.21.15.40:0/465864649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f686c1f1f70 con 0x7f686c144d30 2023-10-18T09:02:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.708+0000 7f68637fe640 1 -- 172.21.15.40:0/465864649 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16873 (secure 0 0 0) 0x7f686c1f1f70 con 0x7f686c144d30 2023-10-18T09:02:25.712 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:25.712 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":118,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:02:09.285724+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:01:35.453236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-18T09:01:36.012460+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1","mappings":[{"from":2,"to":7}]},{"pgid":"13.14","mappings":[{"from":6,"to":5}]},{"pgid":"13.1c","mappings":[{"from":1,"to":5}]},{"pgid":"13.20","mappings":[{"from":1,"to":5}]},{"pgid":"13.34","mappings":[{"from":6,"to":5}]},{"pgid":"13.38","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:02:25.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.711+0000 7f686b59e640 1 -- 172.21.15.40:0/465864649 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f684c06ae40 msgr2=0x7f684c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:25.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.711+0000 7f686b59e640 1 --2- 172.21.15.40:0/465864649 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f684c06ae40 0x7f684c06d300 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f685c005d70 tx=0x7f685c005d00 comp rx=0 tx=0).stop 2023-10-18T09:02:25.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.711+0000 7f686b59e640 1 -- 172.21.15.40:0/465864649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c144d30 msgr2=0x7f686c0c2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:25.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.711+0000 7f686b59e640 1 --2- 172.21.15.40:0/465864649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c144d30 0x7f686c0c2ff0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f686400c7b0 tx=0x7f686400cc70 comp rx=0 tx=0).stop 2023-10-18T09:02:25.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.711+0000 7f686b59e640 1 -- 172.21.15.40:0/465864649 shutdown_connections 2023-10-18T09:02:25.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.711+0000 7f686b59e640 1 --2- 172.21.15.40:0/465864649 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f684c06ae40 0x7f684c06d300 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.711+0000 7f686b59e640 1 --2- 172.21.15.40:0/465864649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c144d30 0x7f686c0c2ff0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.711+0000 7f686b59e640 1 --2- 172.21.15.40:0/465864649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f686c070500 0x7f686c13db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.711+0000 7f686b59e640 1 --2- 172.21.15.40:0/465864649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f686c069720 0x7f686c1f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.711+0000 7f686b59e640 1 -- 172.21.15.40:0/465864649 >> 172.21.15.40:0/465864649 conn(0x7f686c0afd80 msgr2=0x7f686c0b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:25.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.711+0000 7f686b59e640 1 -- 172.21.15.40:0/465864649 shutdown_connections 2023-10-18T09:02:25.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.711+0000 7f686b59e640 1 -- 172.21.15.40:0/465864649 wait complete. 2023-10-18T09:02:25.731 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T09:02:25.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.951+0000 7ff2c1d8e640 1 -- 172.21.15.40:0/3131017815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2bc0b1b90 msgr2=0x7ff2bc0b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:25.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.951+0000 7ff2c1d8e640 1 --2- 172.21.15.40:0/3131017815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2bc0b1b90 0x7ff2bc0b07d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4005fd0 tx=0x7ff2b402c2e0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.951+0000 7ff2c1d8e640 1 -- 172.21.15.40:0/3131017815 shutdown_connections 2023-10-18T09:02:25.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.951+0000 7ff2c1d8e640 1 --2- 172.21.15.40:0/3131017815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2bc0b1b90 0x7ff2bc0b07d0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.951+0000 7ff2c1d8e640 1 --2- 172.21.15.40:0/3131017815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2bc0cd4c0 0x7ff2bc0b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.951+0000 7ff2c1d8e640 1 -- 172.21.15.40:0/3131017815 >> 172.21.15.40:0/3131017815 conn(0x7ff2bc06cee0 msgr2=0x7ff2bc069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:25.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.951+0000 7ff2c1d8e640 1 -- 172.21.15.40:0/3131017815 shutdown_connections 2023-10-18T09:02:25.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.951+0000 7ff2c1d8e640 1 -- 172.21.15.40:0/3131017815 wait complete. 2023-10-18T09:02:25.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.953+0000 7ff2c1d8e640 1 Processor -- start 2023-10-18T09:02:25.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.953+0000 7ff2c1d8e640 1 -- start start 2023-10-18T09:02:25.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.954+0000 7ff2c1d8e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2bc0b1b90 0x7ff2bc1f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:25.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.954+0000 7ff2c1d8e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2bc0cd4c0 0x7ff2bc0abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:25.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.954+0000 7ff2c1d8e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2bc15eaa0 0x7ff2bc1454b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:25.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.954+0000 7ff2c1d8e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc1f58f0 con 0x7ff2bc0cd4c0 2023-10-18T09:02:25.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.954+0000 7ff2c1d8e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc145f80 con 0x7ff2bc0b1b90 2023-10-18T09:02:25.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.954+0000 7ff2c1d8e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc146260 con 0x7ff2bc15eaa0 2023-10-18T09:02:25.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.954+0000 7ff2baffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2bc0cd4c0 0x7ff2bc0abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:25.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.954+0000 7ff2baffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2bc0cd4c0 0x7ff2bc0abb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50130/0 (socket says 172.21.15.40:50130) 2023-10-18T09:02:25.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.954+0000 7ff2baffd640 1 -- 172.21.15.40:0/132200234 learned_addr learned my addr 172.21.15.40:0/132200234 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:25.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.955+0000 7ff2baffd640 1 -- 172.21.15.40:0/132200234 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2bc15eaa0 msgr2=0x7ff2bc1454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.955+0000 7ff2baffd640 1 --2- 172.21.15.40:0/132200234 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2bc15eaa0 0x7ff2bc1454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.955+0000 7ff2baffd640 1 -- 172.21.15.40:0/132200234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2bc0b1b90 msgr2=0x7ff2bc1f5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.955+0000 7ff2baffd640 1 --2- 172.21.15.40:0/132200234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2bc0b1b90 0x7ff2bc1f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.955+0000 7ff2baffd640 1 -- 172.21.15.40:0/132200234 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b4009900 con 0x7ff2bc0cd4c0 2023-10-18T09:02:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.955+0000 7ff2baffd640 1 --2- 172.21.15.40:0/132200234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2bc0cd4c0 0x7ff2bc0abb80 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000d770 tx=0x7ff2b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.955+0000 7ff2a3fff640 1 -- 172.21.15.40:0/132200234 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0009c30 con 0x7ff2bc0cd4c0 2023-10-18T09:02:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.955+0000 7ff2a3fff640 1 -- 172.21.15.40:0/132200234 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff2b000de80 con 0x7ff2bc0cd4c0 2023-10-18T09:02:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.955+0000 7ff2a3fff640 1 -- 172.21.15.40:0/132200234 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0005280 con 0x7ff2bc0cd4c0 2023-10-18T09:02:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.955+0000 7ff2c1d8e640 1 -- 172.21.15.40:0/132200234 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2bc070370 con 0x7ff2bc0cd4c0 2023-10-18T09:02:25.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.956+0000 7ff2c1d8e640 1 -- 172.21.15.40:0/132200234 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2bc1f62c0 con 0x7ff2bc0cd4c0 2023-10-18T09:02:25.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.957+0000 7ff2c1d8e640 1 -- 172.21.15.40:0/132200234 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2bc146480 con 0x7ff2bc0cd4c0 2023-10-18T09:02:25.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.958+0000 7ff2a3fff640 1 -- 172.21.15.40:0/132200234 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff2b00053e0 con 0x7ff2bc0cd4c0 2023-10-18T09:02:25.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.958+0000 7ff2a3fff640 1 --2- 172.21.15.40:0/132200234 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2a806ae40 0x7ff2a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:25.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.959+0000 7ff2a3fff640 1 -- 172.21.15.40:0/132200234 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5883+0+0 (secure 0 0 0) 0x7ff2b0089970 con 0x7ff2bc0cd4c0 2023-10-18T09:02:25.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.959+0000 7ff2bb7fe640 1 --2- 172.21.15.40:0/132200234 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2a806ae40 0x7ff2a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:25.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.960+0000 7ff2bb7fe640 1 --2- 172.21.15.40:0/132200234 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2a806ae40 0x7ff2a806d300 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac006fd0 tx=0x7ff2ac006bc0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:25.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:25.963+0000 7ff2a3fff640 1 -- 172.21.15.40:0/132200234 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff2b00548d0 con 0x7ff2bc0cd4c0 2023-10-18T09:02:26.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.191+0000 7ff2c1d8e640 1 -- 172.21.15.40:0/132200234 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff2bc0b2610 con 0x7ff2bc0cd4c0 2023-10-18T09:02:26.195 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:02:26.192+0000 7f6a0f710640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T09:02:26.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.196+0000 7ff2a3fff640 1 -- 172.21.15.40:0/132200234 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff2b0017020 con 0x7ff2bc0cd4c0 2023-10-18T09:02:26.199 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T09:02:26.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.200+0000 7ff2c1d8e640 1 -- 172.21.15.40:0/132200234 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2a806ae40 msgr2=0x7ff2a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:26.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.200+0000 7ff2c1d8e640 1 --2- 172.21.15.40:0/132200234 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2a806ae40 0x7ff2a806d300 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac006fd0 tx=0x7ff2ac006bc0 comp rx=0 tx=0).stop 2023-10-18T09:02:26.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.200+0000 7ff2c1d8e640 1 -- 172.21.15.40:0/132200234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2bc0cd4c0 msgr2=0x7ff2bc0abb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:26.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.200+0000 7ff2c1d8e640 1 --2- 172.21.15.40:0/132200234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2bc0cd4c0 0x7ff2bc0abb80 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000d770 tx=0x7ff2b000dc30 comp rx=0 tx=0).stop 2023-10-18T09:02:26.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.200+0000 7ff2c1d8e640 1 -- 172.21.15.40:0/132200234 shutdown_connections 2023-10-18T09:02:26.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.200+0000 7ff2c1d8e640 1 --2- 172.21.15.40:0/132200234 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff2a806ae40 0x7ff2a806d300 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:26.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.200+0000 7ff2c1d8e640 1 --2- 172.21.15.40:0/132200234 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2bc15eaa0 0x7ff2bc1454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:26.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.200+0000 7ff2c1d8e640 1 --2- 172.21.15.40:0/132200234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2bc0cd4c0 0x7ff2bc0abb80 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.200+0000 7ff2c1d8e640 1 --2- 172.21.15.40:0/132200234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2bc0b1b90 0x7ff2bc1f5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.200+0000 7ff2c1d8e640 1 -- 172.21.15.40:0/132200234 >> 172.21.15.40:0/132200234 conn(0x7ff2bc06cee0 msgr2=0x7ff2bc0ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.200+0000 7ff2c1d8e640 1 -- 172.21.15.40:0/132200234 shutdown_connections 2023-10-18T09:02:26.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.200+0000 7ff2c1d8e640 1 -- 172.21.15.40:0/132200234 wait complete. 2023-10-18T09:02:26.221 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T09:02:26.292 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T09:02:26.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.426+0000 7ff022139640 1 -- 172.21.15.40:0/1944939019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff008005180 msgr2=0x7ff00800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:26.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.426+0000 7ff022139640 1 --2- 172.21.15.40:0/1944939019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff008005180 0x7ff00800b980 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff014006f00 tx=0x7ff0080057f0 comp rx=0 tx=0).stop 2023-10-18T09:02:26.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.426+0000 7ff022139640 1 -- 172.21.15.40:0/1944939019 shutdown_connections 2023-10-18T09:02:26.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.426+0000 7ff022139640 1 --2- 172.21.15.40:0/1944939019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff008005180 0x7ff00800b980 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:26.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.426+0000 7ff022139640 1 -- 172.21.15.40:0/1944939019 >> 172.21.15.40:0/1944939019 conn(0x7ff01c069720 msgr2=0x7ff01c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:26.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.426+0000 7ff022139640 1 -- 172.21.15.40:0/1944939019 shutdown_connections 2023-10-18T09:02:26.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.426+0000 7ff022139640 1 -- 172.21.15.40:0/1944939019 wait complete. 2023-10-18T09:02:26.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.428+0000 7ff022139640 1 Processor -- start 2023-10-18T09:02:26.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.428+0000 7ff022139640 1 -- start start 2023-10-18T09:02:26.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.429+0000 7ff022139640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff008005180 0x7ff01c0b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:26.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.429+0000 7ff022139640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff01c1399e0 0x7ff01c0ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:26.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.429+0000 7ff022139640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff01c139e40 0x7ff01c0abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:26.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.429+0000 7ff022139640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c0b0a80 con 0x7ff01c139e40 2023-10-18T09:02:26.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.429+0000 7ff022139640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c0b1ef0 con 0x7ff01c1399e0 2023-10-18T09:02:26.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.429+0000 7ff022139640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff01c0b0cd0 con 0x7ff008005180 2023-10-18T09:02:26.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.430+0000 7ff01bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff01c139e40 0x7ff01c0abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:26.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.430+0000 7ff01bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff01c139e40 0x7ff01c0abb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50170/0 (socket says 172.21.15.40:50170) 2023-10-18T09:02:26.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.430+0000 7ff01bfff640 1 -- 172.21.15.40:0/1791797833 learned_addr learned my addr 172.21.15.40:0/1791797833 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:26.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.430+0000 7ff01b7fe640 1 --2- 172.21.15.40:0/1791797833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff008005180 0x7ff01c0b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:26.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.430+0000 7ff01affd640 1 --2- 172.21.15.40:0/1791797833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff01c1399e0 0x7ff01c0ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:26.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.430+0000 7ff01bfff640 1 -- 172.21.15.40:0/1791797833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff008005180 msgr2=0x7ff01c0b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:26.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.430+0000 7ff01bfff640 1 --2- 172.21.15.40:0/1791797833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff008005180 0x7ff01c0b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:26.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.430+0000 7ff01bfff640 1 -- 172.21.15.40:0/1791797833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff01c1399e0 msgr2=0x7ff01c0ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:26.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.430+0000 7ff01bfff640 1 --2- 172.21.15.40:0/1791797833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff01c1399e0 0x7ff01c0ab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:26.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.430+0000 7ff01bfff640 1 -- 172.21.15.40:0/1791797833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01c06bfe0 con 0x7ff01c139e40 2023-10-18T09:02:26.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.431+0000 7ff01bfff640 1 --2- 172.21.15.40:0/1791797833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff01c139e40 0x7ff01c0abb60 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff01400c290 tx=0x7ff0140016a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:26.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.431+0000 7ff013fff640 1 -- 172.21.15.40:0/1791797833 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0140037f0 con 0x7ff01c139e40 2023-10-18T09:02:26.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.431+0000 7ff013fff640 1 -- 172.21.15.40:0/1791797833 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff014003a40 con 0x7ff01c139e40 2023-10-18T09:02:26.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.431+0000 7ff013fff640 1 -- 172.21.15.40:0/1791797833 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff014003e30 con 0x7ff01c139e40 2023-10-18T09:02:26.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.431+0000 7ff022139640 1 -- 172.21.15.40:0/1791797833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff01c0b0fb0 con 0x7ff01c139e40 2023-10-18T09:02:26.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.431+0000 7ff022139640 1 -- 172.21.15.40:0/1791797833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01c0b1290 con 0x7ff01c139e40 2023-10-18T09:02:26.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.432+0000 7ff022139640 1 -- 172.21.15.40:0/1791797833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff01c066fd0 con 0x7ff01c139e40 2023-10-18T09:02:26.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.433+0000 7ff013fff640 1 -- 172.21.15.40:0/1791797833 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff014032070 con 0x7ff01c139e40 2023-10-18T09:02:26.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.434+0000 7ff013fff640 1 --2- 172.21.15.40:0/1791797833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feffc06ae40 0x7feffc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:26.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.434+0000 7ff013fff640 1 -- 172.21.15.40:0/1791797833 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5969+0+0 (secure 0 0 0) 0x7ff01403c070 con 0x7ff01c139e40 2023-10-18T09:02:26.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.435+0000 7ff01b7fe640 1 --2- 172.21.15.40:0/1791797833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feffc06ae40 0x7feffc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:26.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.436+0000 7ff01b7fe640 1 --2- 172.21.15.40:0/1791797833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feffc06ae40 0x7feffc06d300 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff00c005fd0 tx=0x7ff00c009500 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:26.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.438+0000 7ff013fff640 1 -- 172.21.15.40:0/1791797833 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff0140757e0 con 0x7ff01c139e40 2023-10-18T09:02:26.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:26.669+0000 7ff022139640 1 -- 172.21.15.40:0/1791797833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff01c0cdbb0 con 0x7ff01c139e40 2023-10-18T09:02:27.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.199+0000 7ff013fff640 1 -- 172.21.15.40:0/1791797833 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff014078e00 con 0x7ff01c139e40 2023-10-18T09:02:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.203+0000 7ff022139640 1 -- 172.21.15.40:0/1791797833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feffc06ae40 msgr2=0x7feffc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.203+0000 7ff022139640 1 --2- 172.21.15.40:0/1791797833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feffc06ae40 0x7feffc06d300 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff00c005fd0 tx=0x7ff00c009500 comp rx=0 tx=0).stop 2023-10-18T09:02:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.203+0000 7ff022139640 1 -- 172.21.15.40:0/1791797833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff01c139e40 msgr2=0x7ff01c0abb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.203+0000 7ff022139640 1 --2- 172.21.15.40:0/1791797833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff01c139e40 0x7ff01c0abb60 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff01400c290 tx=0x7ff0140016a0 comp rx=0 tx=0).stop 2023-10-18T09:02:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.203+0000 7ff022139640 1 -- 172.21.15.40:0/1791797833 shutdown_connections 2023-10-18T09:02:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.203+0000 7ff022139640 1 --2- 172.21.15.40:0/1791797833 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feffc06ae40 0x7feffc06d300 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.203+0000 7ff022139640 1 --2- 172.21.15.40:0/1791797833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff01c139e40 0x7ff01c0abb60 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.203+0000 7ff022139640 1 --2- 172.21.15.40:0/1791797833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff01c1399e0 0x7ff01c0ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.203+0000 7ff022139640 1 --2- 172.21.15.40:0/1791797833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff008005180 0x7ff01c0b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.203+0000 7ff022139640 1 -- 172.21.15.40:0/1791797833 >> 172.21.15.40:0/1791797833 conn(0x7ff01c069720 msgr2=0x7ff01c13bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.203+0000 7ff022139640 1 -- 172.21.15.40:0/1791797833 shutdown_connections 2023-10-18T09:02:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.203+0000 7ff022139640 1 -- 172.21.15.40:0/1791797833 wait complete. 2023-10-18T09:02:27.229 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T09:02:27.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.458+0000 7f5f2af0c640 1 -- 172.21.15.40:0/1561334685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f10004d00 msgr2=0x7f5f10005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:27.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.458+0000 7f5f2af0c640 1 --2- 172.21.15.40:0/1561334685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f10004d00 0x7f5f10005180 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c005fd0 tx=0x7f5f100057f0 comp rx=0 tx=0).stop 2023-10-18T09:02:27.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.458+0000 7f5f2a70b640 1 -- 172.21.15.40:0/1561334685 reap_dead start 2023-10-18T09:02:27.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.458+0000 7f5f2af0c640 1 -- 172.21.15.40:0/1561334685 shutdown_connections 2023-10-18T09:02:27.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.458+0000 7f5f2af0c640 1 -- 172.21.15.40:0/1561334685 >> 172.21.15.40:0/1561334685 conn(0x7f5f2408aca0 msgr2=0x7f5f2409a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:27.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.458+0000 7f5f2af0c640 1 -- 172.21.15.40:0/1561334685 shutdown_connections 2023-10-18T09:02:27.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.459+0000 7f5f2af0c640 1 -- 172.21.15.40:0/1561334685 wait complete. 2023-10-18T09:02:27.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.461+0000 7f5f2af0c640 1 Processor -- start 2023-10-18T09:02:27.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.461+0000 7f5f2af0c640 1 -- start start 2023-10-18T09:02:27.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.462+0000 7f5f2af0c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f10004d00 0x7f5f241510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:27.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.462+0000 7f5f2af0c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f240c57d0 0x7f5f240b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:27.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.462+0000 7f5f2af0c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f240af3b0 0x7f5f240af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:27.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.462+0000 7f5f2af0c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f24153fe0 con 0x7f5f10004d00 2023-10-18T09:02:27.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.462+0000 7f5f2af0c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f240afd70 con 0x7f5f240af3b0 2023-10-18T09:02:27.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.462+0000 7f5f2af0c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f240b61c0 con 0x7f5f240c57d0 2023-10-18T09:02:27.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.463+0000 7f5f29f0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f10004d00 0x7f5f241510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:27.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.463+0000 7f5f29f0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f10004d00 0x7f5f241510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50184/0 (socket says 172.21.15.40:50184) 2023-10-18T09:02:27.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.463+0000 7f5f29f0a640 1 -- 172.21.15.40:0/1913056873 learned_addr learned my addr 172.21.15.40:0/1913056873 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:27.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.463+0000 7f5f29709640 1 --2- 172.21.15.40:0/1913056873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f240c57d0 0x7f5f240b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:27.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.463+0000 7f5f2a70b640 1 --2- 172.21.15.40:0/1913056873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f240af3b0 0x7f5f240af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:27.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.463+0000 7f5f29f0a640 1 -- 172.21.15.40:0/1913056873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f240c57d0 msgr2=0x7f5f240b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:27.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.463+0000 7f5f29f0a640 1 --2- 172.21.15.40:0/1913056873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f240c57d0 0x7f5f240b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.463+0000 7f5f29f0a640 1 -- 172.21.15.40:0/1913056873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f240af3b0 msgr2=0x7f5f240af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.463+0000 7f5f29f0a640 1 --2- 172.21.15.40:0/1913056873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f240af3b0 0x7f5f240af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.463+0000 7f5f29f0a640 1 -- 172.21.15.40:0/1913056873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f240c5d90 con 0x7f5f10004d00 2023-10-18T09:02:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.463+0000 7f5f29709640 1 --2- 172.21.15.40:0/1913056873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f240c57d0 0x7f5f240b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:27.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.463+0000 7f5f2a70b640 1 --2- 172.21.15.40:0/1913056873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f240af3b0 0x7f5f240af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.464+0000 7f5f29f0a640 1 --2- 172.21.15.40:0/1913056873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f10004d00 0x7f5f241510b0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f5f20001eb0 tx=0x7f5f2000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.464+0000 7f5f1a7fc640 1 -- 172.21.15.40:0/1913056873 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20004460 con 0x7f5f10004d00 2023-10-18T09:02:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.464+0000 7f5f1a7fc640 1 -- 172.21.15.40:0/1913056873 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5f2000ee90 con 0x7f5f10004d00 2023-10-18T09:02:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.464+0000 7f5f1a7fc640 1 -- 172.21.15.40:0/1913056873 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2000caa0 con 0x7f5f10004d00 2023-10-18T09:02:27.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.464+0000 7f5f2af0c640 1 -- 172.21.15.40:0/1913056873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f240c6070 con 0x7f5f10004d00 2023-10-18T09:02:27.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.464+0000 7f5f2af0c640 1 -- 172.21.15.40:0/1913056873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f240c6630 con 0x7f5f10004d00 2023-10-18T09:02:27.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.465+0000 7f5f2af0c640 1 -- 172.21.15.40:0/1913056873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f24071da0 con 0x7f5f10004d00 2023-10-18T09:02:27.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.466+0000 7f5f1a7fc640 1 -- 172.21.15.40:0/1913056873 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5f2000cc00 con 0x7f5f10004d00 2023-10-18T09:02:27.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.467+0000 7f5f1a7fc640 1 --2- 172.21.15.40:0/1913056873 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f0806ae40 0x7f5f0806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:27.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.467+0000 7f5f1a7fc640 1 -- 172.21.15.40:0/1913056873 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5969+0+0 (secure 0 0 0) 0x7f5f20089fb0 con 0x7f5f10004d00 2023-10-18T09:02:27.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.468+0000 7f5f29709640 1 --2- 172.21.15.40:0/1913056873 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f0806ae40 0x7f5f0806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:27.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.469+0000 7f5f29709640 1 --2- 172.21.15.40:0/1913056873 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f0806ae40 0x7f5f0806d300 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5f14006fd0 tx=0x7f5f14006b90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:27.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.471+0000 7f5f1a7fc640 1 -- 172.21.15.40:0/1913056873 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f20054eb0 con 0x7f5f10004d00 2023-10-18T09:02:27.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:27.661+0000 7f5f2af0c640 1 -- 172.21.15.40:0/1913056873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5f24066e00 con 0x7f5f10004d00 2023-10-18T09:02:28.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.213+0000 7f5f1a7fc640 1 -- 172.21.15.40:0/1913056873 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v120) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5f20015020 con 0x7f5f10004d00 2023-10-18T09:02:28.217 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-10-18T09:02:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.217+0000 7f5f2af0c640 1 -- 172.21.15.40:0/1913056873 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f0806ae40 msgr2=0x7f5f0806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.217+0000 7f5f2af0c640 1 --2- 172.21.15.40:0/1913056873 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f0806ae40 0x7f5f0806d300 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5f14006fd0 tx=0x7f5f14006b90 comp rx=0 tx=0).stop 2023-10-18T09:02:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.217+0000 7f5f2af0c640 1 -- 172.21.15.40:0/1913056873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f10004d00 msgr2=0x7f5f241510b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.217+0000 7f5f2af0c640 1 --2- 172.21.15.40:0/1913056873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f10004d00 0x7f5f241510b0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f5f20001eb0 tx=0x7f5f2000eb80 comp rx=0 tx=0).stop 2023-10-18T09:02:28.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.217+0000 7f5f2af0c640 1 -- 172.21.15.40:0/1913056873 shutdown_connections 2023-10-18T09:02:28.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.217+0000 7f5f2af0c640 1 --2- 172.21.15.40:0/1913056873 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f0806ae40 0x7f5f0806d300 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:28.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.217+0000 7f5f2af0c640 1 --2- 172.21.15.40:0/1913056873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f240af3b0 0x7f5f240af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:28.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.217+0000 7f5f2af0c640 1 --2- 172.21.15.40:0/1913056873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f240c57d0 0x7f5f240b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:28.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.217+0000 7f5f2af0c640 1 --2- 172.21.15.40:0/1913056873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f10004d00 0x7f5f241510b0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:28.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.217+0000 7f5f2af0c640 1 -- 172.21.15.40:0/1913056873 >> 172.21.15.40:0/1913056873 conn(0x7f5f2408aca0 msgr2=0x7f5f240b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.218+0000 7f5f2af0c640 1 -- 172.21.15.40:0/1913056873 shutdown_connections 2023-10-18T09:02:28.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.218+0000 7f5f2af0c640 1 -- 172.21.15.40:0/1913056873 wait complete. 2023-10-18T09:02:28.242 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T09:02:28.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.469+0000 7fd22e5f0640 1 -- 172.21.15.40:0/3881982687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd214003f20 msgr2=0x7fd2140047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:28.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.469+0000 7fd22e5f0640 1 --2- 172.21.15.40:0/3881982687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd214003f20 0x7fd2140047c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fd210006fd0 tx=0x7fd2140057f0 comp rx=0 tx=0).stop 2023-10-18T09:02:28.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.469+0000 7fd22cb8d640 1 -- 172.21.15.40:0/3881982687 reap_dead start 2023-10-18T09:02:28.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.469+0000 7fd22e5f0640 1 -- 172.21.15.40:0/3881982687 shutdown_connections 2023-10-18T09:02:28.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.469+0000 7fd22e5f0640 1 -- 172.21.15.40:0/3881982687 >> 172.21.15.40:0/3881982687 conn(0x7fd2280aec00 msgr2=0x7fd2280aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:28.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.469+0000 7fd22e5f0640 1 -- 172.21.15.40:0/3881982687 shutdown_connections 2023-10-18T09:02:28.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.469+0000 7fd22e5f0640 1 -- 172.21.15.40:0/3881982687 wait complete. 2023-10-18T09:02:28.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.471+0000 7fd22e5f0640 1 Processor -- start 2023-10-18T09:02:28.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.472+0000 7fd22e5f0640 1 -- start start 2023-10-18T09:02:28.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.473+0000 7fd22e5f0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd214003f20 0x7fd228156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:28.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.473+0000 7fd22e5f0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2280c2b80 0x7fd2280ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:28.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.473+0000 7fd22e5f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280c3000 0x7fd2280ae350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:28.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.473+0000 7fd22e5f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2280c0770 con 0x7fd2280c3000 2023-10-18T09:02:28.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.473+0000 7fd22e5f0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2280c0a50 con 0x7fd2280c2b80 2023-10-18T09:02:28.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.473+0000 7fd22e5f0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2280ae890 con 0x7fd214003f20 2023-10-18T09:02:28.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.473+0000 7fd22cb8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280c3000 0x7fd2280ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:28.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.473+0000 7fd22cb8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280c3000 0x7fd2280ae350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50200/0 (socket says 172.21.15.40:50200) 2023-10-18T09:02:28.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.474+0000 7fd22cb8d640 1 -- 172.21.15.40:0/1978548085 learned_addr learned my addr 172.21.15.40:0/1978548085 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:28.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.474+0000 7fd227fff640 1 --2- 172.21.15.40:0/1978548085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd214003f20 0x7fd228156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:28.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.474+0000 7fd21ffff640 1 --2- 172.21.15.40:0/1978548085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2280c2b80 0x7fd2280ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:28.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.474+0000 7fd22cb8d640 1 -- 172.21.15.40:0/1978548085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd214003f20 msgr2=0x7fd228156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:28.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.474+0000 7fd22cb8d640 1 --2- 172.21.15.40:0/1978548085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd214003f20 0x7fd228156640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:28.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.474+0000 7fd22cb8d640 1 -- 172.21.15.40:0/1978548085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2280c2b80 msgr2=0x7fd2280ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:28.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.474+0000 7fd22cb8d640 1 --2- 172.21.15.40:0/1978548085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2280c2b80 0x7fd2280ade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:28.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.474+0000 7fd22cb8d640 1 -- 172.21.15.40:0/1978548085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2280c35d0 con 0x7fd2280c3000 2023-10-18T09:02:28.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.474+0000 7fd22cb8d640 1 --2- 172.21.15.40:0/1978548085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280c3000 0x7fd2280ae350 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fd2280af780 tx=0x7fd22000db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:28.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.475+0000 7fd2257fa640 1 -- 172.21.15.40:0/1978548085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220015030 con 0x7fd2280c3000 2023-10-18T09:02:28.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.475+0000 7fd2257fa640 1 -- 172.21.15.40:0/1978548085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd220003c80 con 0x7fd2280c3000 2023-10-18T09:02:28.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.475+0000 7fd2257fa640 1 -- 172.21.15.40:0/1978548085 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22000ba90 con 0x7fd2280c3000 2023-10-18T09:02:28.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.475+0000 7fd22e5f0640 1 -- 172.21.15.40:0/1978548085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2280c38b0 con 0x7fd2280c3000 2023-10-18T09:02:28.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.475+0000 7fd22e5f0640 1 -- 172.21.15.40:0/1978548085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd228156b80 con 0x7fd2280c3000 2023-10-18T09:02:28.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.476+0000 7fd22e5f0640 1 -- 172.21.15.40:0/1978548085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd228066fd0 con 0x7fd2280c3000 2023-10-18T09:02:28.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.477+0000 7fd2257fa640 1 -- 172.21.15.40:0/1978548085 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd220018070 con 0x7fd2280c3000 2023-10-18T09:02:28.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.478+0000 7fd2257fa640 1 --2- 172.21.15.40:0/1978548085 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd20806ae70 0x7fd20806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:28.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.478+0000 7fd2257fa640 1 -- 172.21.15.40:0/1978548085 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5531+0+0 (secure 0 0 0) 0x7fd220016070 con 0x7fd2280c3000 2023-10-18T09:02:28.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.479+0000 7fd227fff640 1 --2- 172.21.15.40:0/1978548085 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd20806ae70 0x7fd20806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:28.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.480+0000 7fd227fff640 1 --2- 172.21.15.40:0/1978548085 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd20806ae70 0x7fd20806d330 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd218009aa0 tx=0x7fd218005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:28.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.483+0000 7fd2257fa640 1 -- 172.21.15.40:0/1978548085 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd220058240 con 0x7fd2280c3000 2023-10-18T09:02:28.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:28.684+0000 7fd22e5f0640 1 -- 172.21.15.40:0/1978548085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd2280af7f0 con 0x7fd2280c3000 2023-10-18T09:02:29.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.225+0000 7fd2257fa640 1 -- 172.21.15.40:0/1978548085 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v121) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd220057dd0 con 0x7fd2280c3000 2023-10-18T09:02:29.229 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-10-18T09:02:29.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.231+0000 7fd22e5f0640 1 -- 172.21.15.40:0/1978548085 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd20806ae70 msgr2=0x7fd20806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:29.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.231+0000 7fd22e5f0640 1 --2- 172.21.15.40:0/1978548085 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd20806ae70 0x7fd20806d330 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd218009aa0 tx=0x7fd218005f90 comp rx=0 tx=0).stop 2023-10-18T09:02:29.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.231+0000 7fd22e5f0640 1 -- 172.21.15.40:0/1978548085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280c3000 msgr2=0x7fd2280ae350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:29.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.231+0000 7fd22e5f0640 1 --2- 172.21.15.40:0/1978548085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280c3000 0x7fd2280ae350 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fd2280af780 tx=0x7fd22000db50 comp rx=0 tx=0).stop 2023-10-18T09:02:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.232+0000 7fd22e5f0640 1 -- 172.21.15.40:0/1978548085 shutdown_connections 2023-10-18T09:02:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.232+0000 7fd22e5f0640 1 --2- 172.21.15.40:0/1978548085 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd20806ae70 0x7fd20806d330 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.232+0000 7fd22e5f0640 1 --2- 172.21.15.40:0/1978548085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280c3000 0x7fd2280ae350 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.232+0000 7fd22e5f0640 1 --2- 172.21.15.40:0/1978548085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2280c2b80 0x7fd2280ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:29.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.232+0000 7fd22e5f0640 1 --2- 172.21.15.40:0/1978548085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd214003f20 0x7fd228156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:29.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.232+0000 7fd22e5f0640 1 -- 172.21.15.40:0/1978548085 >> 172.21.15.40:0/1978548085 conn(0x7fd2280aec00 msgr2=0x7fd2281586a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:29.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.232+0000 7fd22e5f0640 1 -- 172.21.15.40:0/1978548085 shutdown_connections 2023-10-18T09:02:29.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.233+0000 7fd22e5f0640 1 -- 172.21.15.40:0/1978548085 wait complete. 2023-10-18T09:02:29.260 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 22 2023-10-18T09:02:29.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.472+0000 7fcb9b81a640 1 -- 172.21.15.40:0/675093373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb940cd9a0 msgr2=0x7fcb9406be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:29.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.472+0000 7fcb9b81a640 1 --2- 172.21.15.40:0/675093373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb940cd9a0 0x7fcb9406be20 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c005fd0 tx=0x7fcb8c02c240 comp rx=0 tx=0).stop 2023-10-18T09:02:29.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.472+0000 7fcb9b81a640 1 -- 172.21.15.40:0/675093373 shutdown_connections 2023-10-18T09:02:29.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.473+0000 7fcb9b81a640 1 --2- 172.21.15.40:0/675093373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb940cd9a0 0x7fcb9406be20 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:29.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.473+0000 7fcb9b81a640 1 --2- 172.21.15.40:0/675093373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb9406d710 0x7fcb94151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:29.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.473+0000 7fcb9b81a640 1 -- 172.21.15.40:0/675093373 >> 172.21.15.40:0/675093373 conn(0x7fcb9406cee0 msgr2=0x7fcb94069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:29.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.473+0000 7fcb9b81a640 1 -- 172.21.15.40:0/675093373 shutdown_connections 2023-10-18T09:02:29.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.473+0000 7fcb9b81a640 1 -- 172.21.15.40:0/675093373 wait complete. 2023-10-18T09:02:29.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.474+0000 7fcb9b81a640 1 Processor -- start 2023-10-18T09:02:29.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.475+0000 7fcb9b81a640 1 -- start start 2023-10-18T09:02:29.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb9b81a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb9406d710 0x7fcb94147a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:29.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb9b81a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb940cd9a0 0x7fcb94143cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:29.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb9b81a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb941586a0 0x7fcb940a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:29.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb9b81a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb940a8d60 con 0x7fcb941586a0 2023-10-18T09:02:29.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb9b81a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb940a8150 con 0x7fcb9406d710 2023-10-18T09:02:29.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb9b81a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb940a8430 con 0x7fcb940cd9a0 2023-10-18T09:02:29.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb99db7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb941586a0 0x7fcb940a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:29.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb99db7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb941586a0 0x7fcb940a77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50218/0 (socket says 172.21.15.40:50218) 2023-10-18T09:02:29.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb99db7640 1 -- 172.21.15.40:0/131734215 learned_addr learned my addr 172.21.15.40:0/131734215 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:29.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb995b6640 1 --2- 172.21.15.40:0/131734215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb9406d710 0x7fcb94147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:29.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb98db5640 1 --2- 172.21.15.40:0/131734215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb940cd9a0 0x7fcb94143cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:29.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb99db7640 1 -- 172.21.15.40:0/131734215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb940cd9a0 msgr2=0x7fcb94143cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:29.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb99db7640 1 --2- 172.21.15.40:0/131734215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb940cd9a0 0x7fcb94143cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:29.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb99db7640 1 -- 172.21.15.40:0/131734215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb9406d710 msgr2=0x7fcb94147a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:29.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb99db7640 1 --2- 172.21.15.40:0/131734215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb9406d710 0x7fcb94147a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:29.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb99db7640 1 -- 172.21.15.40:0/131734215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c009900 con 0x7fcb941586a0 2023-10-18T09:02:29.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb98db5640 1 --2- 172.21.15.40:0/131734215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb940cd9a0 0x7fcb94143cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:29.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.476+0000 7fcb995b6640 1 --2- 172.21.15.40:0/131734215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb9406d710 0x7fcb94147a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:29.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.477+0000 7fcb99db7640 1 --2- 172.21.15.40:0/131734215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb941586a0 0x7fcb940a77e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c037040 tx=0x7fcb8c035a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:29.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.477+0000 7fcb89ffb640 1 -- 172.21.15.40:0/131734215 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c034440 con 0x7fcb941586a0 2023-10-18T09:02:29.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.477+0000 7fcb89ffb640 1 -- 172.21.15.40:0/131734215 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb8c004330 con 0x7fcb941586a0 2023-10-18T09:02:29.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.477+0000 7fcb89ffb640 1 -- 172.21.15.40:0/131734215 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c0046e0 con 0x7fcb941586a0 2023-10-18T09:02:29.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.477+0000 7fcb9b81a640 1 -- 172.21.15.40:0/131734215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb940a8710 con 0x7fcb941586a0 2023-10-18T09:02:29.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.477+0000 7fcb9b81a640 1 -- 172.21.15.40:0/131734215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb94144230 con 0x7fcb941586a0 2023-10-18T09:02:29.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.478+0000 7fcb9b81a640 1 -- 172.21.15.40:0/131734215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb940a8f10 con 0x7fcb941586a0 2023-10-18T09:02:29.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.479+0000 7fcb89ffb640 1 -- 172.21.15.40:0/131734215 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcb8c03b040 con 0x7fcb941586a0 2023-10-18T09:02:29.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.480+0000 7fcb89ffb640 1 --2- 172.21.15.40:0/131734215 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcb7806ae40 0x7fcb7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:29.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.480+0000 7fcb89ffb640 1 -- 172.21.15.40:0/131734215 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcb8c0ac0b0 con 0x7fcb941586a0 2023-10-18T09:02:29.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.481+0000 7fcb995b6640 1 --2- 172.21.15.40:0/131734215 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcb7806ae40 0x7fcb7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:29.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.482+0000 7fcb995b6640 1 --2- 172.21.15.40:0/131734215 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcb7806ae40 0x7fcb7806d300 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000a870 tx=0x7fcb90008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:29.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.484+0000 7fcb89ffb640 1 -- 172.21.15.40:0/131734215 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb940a8f10 con 0x7fcb941586a0 2023-10-18T09:02:29.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.689+0000 7fcb9b81a640 1 -- 172.21.15.40:0/131734215 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config reset", "num": 22} v 0) v1 -- 0x7fcb94066fd0 con 0x7fcb941586a0 2023-10-18T09:02:29.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.697+0000 7fcb89ffb640 1 -- 172.21.15.40:0/131734215 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 22}]=0 v32) v1 ==== 71+0+0 (secure 0 0 0) 0x7fcb8c039080 con 0x7fcb941586a0 2023-10-18T09:02:29.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.700+0000 7fcb9b81a640 1 -- 172.21.15.40:0/131734215 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcb7806ae40 msgr2=0x7fcb7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:29.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.700+0000 7fcb9b81a640 1 --2- 172.21.15.40:0/131734215 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcb7806ae40 0x7fcb7806d300 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000a870 tx=0x7fcb90008040 comp rx=0 tx=0).stop 2023-10-18T09:02:29.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.700+0000 7fcb9b81a640 1 -- 172.21.15.40:0/131734215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb941586a0 msgr2=0x7fcb940a77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:29.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.700+0000 7fcb9b81a640 1 --2- 172.21.15.40:0/131734215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb941586a0 0x7fcb940a77e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c037040 tx=0x7fcb8c035a40 comp rx=0 tx=0).stop 2023-10-18T09:02:29.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.700+0000 7fcb9b81a640 1 -- 172.21.15.40:0/131734215 shutdown_connections 2023-10-18T09:02:29.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.700+0000 7fcb9b81a640 1 --2- 172.21.15.40:0/131734215 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcb7806ae40 0x7fcb7806d300 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:29.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.700+0000 7fcb9b81a640 1 --2- 172.21.15.40:0/131734215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb941586a0 0x7fcb940a77e0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:29.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.700+0000 7fcb9b81a640 1 --2- 172.21.15.40:0/131734215 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb940cd9a0 0x7fcb94143cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:29.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.700+0000 7fcb9b81a640 1 --2- 172.21.15.40:0/131734215 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb9406d710 0x7fcb94147a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:29.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.700+0000 7fcb9b81a640 1 -- 172.21.15.40:0/131734215 >> 172.21.15.40:0/131734215 conn(0x7fcb9406cee0 msgr2=0x7fcb94069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:29.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.700+0000 7fcb9b81a640 1 -- 172.21.15.40:0/131734215 shutdown_connections 2023-10-18T09:02:29.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.701+0000 7fcb9b81a640 1 -- 172.21.15.40:0/131734215 wait complete. 2023-10-18T09:02:29.722 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2023-10-18T09:02:29.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.949+0000 7f59f3121640 1 -- 172.21.15.40:0/1458607179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59ec0ca160 msgr2=0x7f59ec0cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:29.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.949+0000 7f59f3121640 1 --2- 172.21.15.40:0/1458607179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59ec0ca160 0x7f59ec0cafc0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f59e4005fd0 tx=0x7f59e402c240 comp rx=0 tx=0).stop 2023-10-18T09:02:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.950+0000 7f59f3121640 1 -- 172.21.15.40:0/1458607179 shutdown_connections 2023-10-18T09:02:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.950+0000 7f59f3121640 1 --2- 172.21.15.40:0/1458607179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59ec0ca160 0x7f59ec0cafc0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.950+0000 7f59f3121640 1 --2- 172.21.15.40:0/1458607179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec1420f0 0x7f59ec0b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.950+0000 7f59f3121640 1 -- 172.21.15.40:0/1458607179 >> 172.21.15.40:0/1458607179 conn(0x7f59ec069720 msgr2=0x7f59ec069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:29.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.950+0000 7f59f3121640 1 -- 172.21.15.40:0/1458607179 shutdown_connections 2023-10-18T09:02:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.950+0000 7f59f3121640 1 -- 172.21.15.40:0/1458607179 wait complete. 2023-10-18T09:02:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.952+0000 7f59f3121640 1 Processor -- start 2023-10-18T09:02:29.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.952+0000 7f59f3121640 1 -- start start 2023-10-18T09:02:29.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.953+0000 7f59f3121640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59ec0ca160 0x7f59ec0a7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:29.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.953+0000 7f59f3121640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ec1420f0 0x7f59ec158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:29.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.953+0000 7f59f3121640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec1425a0 0x7f59ec15b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:29.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.953+0000 7f59f3121640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec15cd30 con 0x7f59ec0ca160 2023-10-18T09:02:29.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.953+0000 7f59f3121640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec159030 con 0x7f59ec1420f0 2023-10-18T09:02:29.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.953+0000 7f59f3121640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec15c1b0 con 0x7f59ec1425a0 2023-10-18T09:02:29.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.953+0000 7f59f211f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59ec0ca160 0x7f59ec0a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:29.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.953+0000 7f59f211f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59ec0ca160 0x7f59ec0a7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50234/0 (socket says 172.21.15.40:50234) 2023-10-18T09:02:29.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.953+0000 7f59f211f640 1 -- 172.21.15.40:0/1955609370 learned_addr learned my addr 172.21.15.40:0/1955609370 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:29.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.953+0000 7f59f2920640 1 --2- 172.21.15.40:0/1955609370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec1425a0 0x7f59ec15b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:29.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.953+0000 7f59f191e640 1 --2- 172.21.15.40:0/1955609370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ec1420f0 0x7f59ec158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:29.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.953+0000 7f59f211f640 1 -- 172.21.15.40:0/1955609370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec1425a0 msgr2=0x7f59ec15b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.953+0000 7f59f211f640 1 --2- 172.21.15.40:0/1955609370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec1425a0 0x7f59ec15b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.953+0000 7f59f211f640 1 -- 172.21.15.40:0/1955609370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ec1420f0 msgr2=0x7f59ec158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.953+0000 7f59f211f640 1 --2- 172.21.15.40:0/1955609370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ec1420f0 0x7f59ec158af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.953+0000 7f59f211f640 1 -- 172.21.15.40:0/1955609370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e4009900 con 0x7f59ec0ca160 2023-10-18T09:02:29.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.954+0000 7f59f211f640 1 --2- 172.21.15.40:0/1955609370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59ec0ca160 0x7f59ec0a7780 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f59e800b4a0 tx=0x7f59e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:29.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.954+0000 7f59e2ffd640 1 -- 172.21.15.40:0/1955609370 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8010610 con 0x7f59ec0ca160 2023-10-18T09:02:29.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.954+0000 7f59e2ffd640 1 -- 172.21.15.40:0/1955609370 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f59e8010860 con 0x7f59ec0ca160 2023-10-18T09:02:29.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.954+0000 7f59e2ffd640 1 -- 172.21.15.40:0/1955609370 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8010c70 con 0x7f59ec0ca160 2023-10-18T09:02:29.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.954+0000 7f59f3121640 1 -- 172.21.15.40:0/1955609370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ec15c490 con 0x7f59ec0ca160 2023-10-18T09:02:29.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.954+0000 7f59f3121640 1 -- 172.21.15.40:0/1955609370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ec15ca50 con 0x7f59ec0ca160 2023-10-18T09:02:29.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.955+0000 7f59f3121640 1 -- 172.21.15.40:0/1955609370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ec066e00 con 0x7f59ec0ca160 2023-10-18T09:02:29.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.956+0000 7f59e2ffd640 1 -- 172.21.15.40:0/1955609370 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f59e8017360 con 0x7f59ec0ca160 2023-10-18T09:02:29.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.957+0000 7f59e2ffd640 1 --2- 172.21.15.40:0/1955609370 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f59d006ae40 0x7f59d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:29.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.957+0000 7f59e2ffd640 1 -- 172.21.15.40:0/1955609370 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f59e8089320 con 0x7f59ec0ca160 2023-10-18T09:02:29.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.958+0000 7f59f191e640 1 --2- 172.21.15.40:0/1955609370 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f59d006ae40 0x7f59d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:29.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.959+0000 7f59f191e640 1 --2- 172.21.15.40:0/1955609370 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f59d006ae40 0x7f59d006d300 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f59ec15aee0 tx=0x7f59dc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:29.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:29.964+0000 7f59e2ffd640 1 -- 172.21.15.40:0/1955609370 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f59e80545a0 con 0x7f59ec0ca160 2023-10-18T09:02:30.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:30.235+0000 7f59f3121640 1 -- 172.21.15.40:0/1955609370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7f59ec06d920 con 0x7f59ec0ca160 2023-10-18T09:02:31.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.222+0000 7f59e2ffd640 1 -- 172.21.15.40:0/1955609370 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v376) v1 ==== 137+0+0 (secure 0 0 0) 0x7f59e8057bc0 con 0x7f59ec0ca160 2023-10-18T09:02:31.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.226+0000 7f59f3121640 1 -- 172.21.15.40:0/1955609370 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f59d006ae40 msgr2=0x7f59d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.226+0000 7f59f3121640 1 --2- 172.21.15.40:0/1955609370 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f59d006ae40 0x7f59d006d300 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f59ec15aee0 tx=0x7f59dc008040 comp rx=0 tx=0).stop 2023-10-18T09:02:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.226+0000 7f59f3121640 1 -- 172.21.15.40:0/1955609370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59ec0ca160 msgr2=0x7f59ec0a7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.226+0000 7f59f3121640 1 --2- 172.21.15.40:0/1955609370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59ec0ca160 0x7f59ec0a7780 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f59e800b4a0 tx=0x7f59e800b960 comp rx=0 tx=0).stop 2023-10-18T09:02:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.226+0000 7f59f3121640 1 -- 172.21.15.40:0/1955609370 shutdown_connections 2023-10-18T09:02:31.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.226+0000 7f59f3121640 1 --2- 172.21.15.40:0/1955609370 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f59d006ae40 0x7f59d006d300 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.226+0000 7f59f3121640 1 --2- 172.21.15.40:0/1955609370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59ec1425a0 0x7f59ec15b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.226+0000 7f59f3121640 1 --2- 172.21.15.40:0/1955609370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59ec1420f0 0x7f59ec158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.226+0000 7f59f3121640 1 --2- 172.21.15.40:0/1955609370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59ec0ca160 0x7f59ec0a7780 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.226+0000 7f59f3121640 1 -- 172.21.15.40:0/1955609370 >> 172.21.15.40:0/1955609370 conn(0x7f59ec069720 msgr2=0x7f59ec072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:31.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.226+0000 7f59f3121640 1 -- 172.21.15.40:0/1955609370 shutdown_connections 2023-10-18T09:02:31.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.226+0000 7f59f3121640 1 -- 172.21.15.40:0/1955609370 wait complete. 2023-10-18T09:02:31.251 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-10-18T09:02:31.252 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestQuotaFull) That when an OSD epoch barrier is set on an MDS, subsequently 2023-10-18T09:02:31.252 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestQuotaFull) 2023-10-18T09:02:31.253 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:02:31.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.484+0000 7f83ad6ea640 1 -- 172.21.15.40:0/4228192540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a80c9f90 msgr2=0x7f83a806b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:31.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.484+0000 7f83ad6ea640 1 --2- 172.21.15.40:0/4228192540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a80c9f90 0x7f83a806b4d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f83a80ccea0 tx=0x7f83a002c240 comp rx=0 tx=0).stop 2023-10-18T09:02:31.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.484+0000 7f83ad6ea640 1 -- 172.21.15.40:0/4228192540 shutdown_connections 2023-10-18T09:02:31.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.484+0000 7f83ad6ea640 1 --2- 172.21.15.40:0/4228192540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a80c9f90 0x7f83a806b4d0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.484+0000 7f83ad6ea640 1 --2- 172.21.15.40:0/4228192540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83a81425a0 0x7f83a80c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.484+0000 7f83ad6ea640 1 --2- 172.21.15.40:0/4228192540 >> v2:172.21.15.144:3300/0 conn(0x7f83a81420f0 0x7f83a80b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.484+0000 7f83ad6ea640 1 -- 172.21.15.40:0/4228192540 >> 172.21.15.40:0/4228192540 conn(0x7f83a8069720 msgr2=0x7f83a8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:31.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.484+0000 7f83ad6ea640 1 -- 172.21.15.40:0/4228192540 shutdown_connections 2023-10-18T09:02:31.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.484+0000 7f83ad6ea640 1 -- 172.21.15.40:0/4228192540 wait complete. 2023-10-18T09:02:31.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.486+0000 7f83ad6ea640 1 Processor -- start 2023-10-18T09:02:31.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.486+0000 7f83ad6ea640 1 -- start start 2023-10-18T09:02:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.487+0000 7f83ad6ea640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83a80c9f90 0x7f83a8158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.487+0000 7f83ad6ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a81420f0 0x7f83a815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.487+0000 7f83ad6ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83a81425a0 0x7f83a813d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:31.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.487+0000 7f83ad6ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a813cde0 con 0x7f83a81420f0 2023-10-18T09:02:31.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.487+0000 7f83ad6ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a812bfb0 con 0x7f83a81425a0 2023-10-18T09:02:31.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.487+0000 7f83ad6ea640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a813c160 con 0x7f83a80c9f90 2023-10-18T09:02:31.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.488+0000 7f839ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a81420f0 0x7f83a815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:31.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.488+0000 7f839ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a81420f0 0x7f83a815c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50260/0 (socket says 172.21.15.40:50260) 2023-10-18T09:02:31.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.488+0000 7f839ffff640 1 -- 172.21.15.40:0/1002691533 learned_addr learned my addr 172.21.15.40:0/1002691533 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:31.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.488+0000 7f83a7fff640 1 --2- 172.21.15.40:0/1002691533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83a80c9f90 0x7f83a8158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:31.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.488+0000 7f83acee9640 1 --2- 172.21.15.40:0/1002691533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83a81425a0 0x7f83a813d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.488+0000 7f839ffff640 1 -- 172.21.15.40:0/1002691533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83a80c9f90 msgr2=0x7f83a8158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.488+0000 7f839ffff640 1 --2- 172.21.15.40:0/1002691533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83a80c9f90 0x7f83a8158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.488+0000 7f839ffff640 1 -- 172.21.15.40:0/1002691533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83a81425a0 msgr2=0x7f83a813d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:31.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.488+0000 7f839ffff640 1 --2- 172.21.15.40:0/1002691533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83a81425a0 0x7f83a813d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.488+0000 7f839ffff640 1 -- 172.21.15.40:0/1002691533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a0009900 con 0x7f83a81420f0 2023-10-18T09:02:31.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.488+0000 7f83acee9640 1 --2- 172.21.15.40:0/1002691533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83a81425a0 0x7f83a813d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:31.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.489+0000 7f839ffff640 1 --2- 172.21.15.40:0/1002691533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a81420f0 0x7f83a815c040 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f839000d6d0 tx=0x7f839000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:31.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.489+0000 7f83a57fa640 1 -- 172.21.15.40:0/1002691533 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390005220 con 0x7f83a81420f0 2023-10-18T09:02:31.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.489+0000 7f83a57fa640 1 -- 172.21.15.40:0/1002691533 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8390005490 con 0x7f83a81420f0 2023-10-18T09:02:31.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.489+0000 7f83a57fa640 1 -- 172.21.15.40:0/1002691533 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390009a80 con 0x7f83a81420f0 2023-10-18T09:02:31.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.489+0000 7f83ad6ea640 1 -- 172.21.15.40:0/1002691533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a813c440 con 0x7f83a81420f0 2023-10-18T09:02:31.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.489+0000 7f83ad6ea640 1 -- 172.21.15.40:0/1002691533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a813ca00 con 0x7f83a81420f0 2023-10-18T09:02:31.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.490+0000 7f83ad6ea640 1 -- 172.21.15.40:0/1002691533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a8066e00 con 0x7f83a81420f0 2023-10-18T09:02:31.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.491+0000 7f83a57fa640 1 -- 172.21.15.40:0/1002691533 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f83900047a0 con 0x7f83a81420f0 2023-10-18T09:02:31.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.493+0000 7f83a57fa640 1 --2- 172.21.15.40:0/1002691533 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f838806ae40 0x7f838806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:31.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.493+0000 7f83a57fa640 1 -- 172.21.15.40:0/1002691533 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f83900897d0 con 0x7f83a81420f0 2023-10-18T09:02:31.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.494+0000 7f83a7fff640 1 --2- 172.21.15.40:0/1002691533 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f838806ae40 0x7f838806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:31.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.495+0000 7f83a7fff640 1 --2- 172.21.15.40:0/1002691533 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f838806ae40 0x7f838806d300 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8398005fd0 tx=0x7f839800a4e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:31.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.498+0000 7f83a57fa640 1 -- 172.21.15.40:0/1002691533 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8390054a50 con 0x7f83a81420f0 2023-10-18T09:02:31.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.692+0000 7f83ad6ea640 1 -- 172.21.15.40:0/1002691533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83a806d920 con 0x7f83a81420f0 2023-10-18T09:02:31.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.694+0000 7f83a57fa640 1 -- 172.21.15.40:0/1002691533 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+13007 (secure 0 0 0) 0x7f8390017020 con 0x7f83a81420f0 2023-10-18T09:02:31.698 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:31.698 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":121,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:02:29.215252+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/891188043":"2023-10-19T09:02:26.193541+0000","172.21.15.40:6834/891188043":"2023-10-19T09:02:26.193541+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:02:31.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.697+0000 7f83ad6ea640 1 -- 172.21.15.40:0/1002691533 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f838806ae40 msgr2=0x7f838806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:31.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.697+0000 7f83ad6ea640 1 --2- 172.21.15.40:0/1002691533 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f838806ae40 0x7f838806d300 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8398005fd0 tx=0x7f839800a4e0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.697+0000 7f83ad6ea640 1 -- 172.21.15.40:0/1002691533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a81420f0 msgr2=0x7f83a815c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:31.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.697+0000 7f83ad6ea640 1 --2- 172.21.15.40:0/1002691533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a81420f0 0x7f83a815c040 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f839000d6d0 tx=0x7f839000db90 comp rx=0 tx=0).stop 2023-10-18T09:02:31.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.697+0000 7f83ad6ea640 1 -- 172.21.15.40:0/1002691533 shutdown_connections 2023-10-18T09:02:31.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.697+0000 7f83ad6ea640 1 --2- 172.21.15.40:0/1002691533 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f838806ae40 0x7f838806d300 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.697+0000 7f83ad6ea640 1 --2- 172.21.15.40:0/1002691533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83a81425a0 0x7f83a813d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.697+0000 7f83ad6ea640 1 --2- 172.21.15.40:0/1002691533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83a81420f0 0x7f83a815c040 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.697+0000 7f83ad6ea640 1 --2- 172.21.15.40:0/1002691533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83a80c9f90 0x7f83a8158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.697+0000 7f83ad6ea640 1 -- 172.21.15.40:0/1002691533 >> 172.21.15.40:0/1002691533 conn(0x7f83a8069720 msgr2=0x7f83a8072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:31.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.698+0000 7f83ad6ea640 1 -- 172.21.15.40:0/1002691533 shutdown_connections 2023-10-18T09:02:31.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.698+0000 7f83ad6ea640 1 -- 172.21.15.40:0/1002691533 wait complete. 2023-10-18T09:02:31.721 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:02:31.722 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:02:31.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.953+0000 7f3606247640 1 -- 172.21.15.40:0/672634267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f360015b4d0 msgr2=0x7f36000addf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:31.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.953+0000 7f3606247640 1 --2- 172.21.15.40:0/672634267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f360015b4d0 0x7f36000addf0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f35fc005fd0 tx=0x7f35fc02c170 comp rx=0 tx=0).stop 2023-10-18T09:02:31.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.954+0000 7f3606247640 1 -- 172.21.15.40:0/672634267 shutdown_connections 2023-10-18T09:02:31.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.954+0000 7f3606247640 1 --2- 172.21.15.40:0/672634267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36000d1960 0x7f36000cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.954+0000 7f3606247640 1 --2- 172.21.15.40:0/672634267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f360015b4d0 0x7f36000addf0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.954+0000 7f3606247640 1 -- 172.21.15.40:0/672634267 >> 172.21.15.40:0/672634267 conn(0x7f360008ad30 msgr2=0x7f360009a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:31.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.954+0000 7f3606247640 1 -- 172.21.15.40:0/672634267 shutdown_connections 2023-10-18T09:02:31.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.954+0000 7f3606247640 1 -- 172.21.15.40:0/672634267 wait complete. 2023-10-18T09:02:31.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.955+0000 7f3606247640 1 Processor -- start 2023-10-18T09:02:31.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.956+0000 7f3606247640 1 -- start start 2023-10-18T09:02:31.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.956+0000 7f3606247640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36000d1960 0x7f36000b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:31.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.956+0000 7f3606247640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f360015b4d0 0x7f360014a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:31.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.957+0000 7f3606247640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f360015b8b0 0x7f360015df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:31.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.957+0000 7f3606247640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360009b3f0 con 0x7f36000d1960 2023-10-18T09:02:31.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.957+0000 7f3606247640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360014a5c0 con 0x7f360015b8b0 2023-10-18T09:02:31.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.957+0000 7f3606247640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36000b4000 con 0x7f360015b4d0 2023-10-18T09:02:31.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.957+0000 7f3605245640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36000d1960 0x7f36000b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:31.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.957+0000 7f3605245640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36000d1960 0x7f36000b68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50270/0 (socket says 172.21.15.40:50270) 2023-10-18T09:02:31.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.957+0000 7f3605245640 1 -- 172.21.15.40:0/3178064883 learned_addr learned my addr 172.21.15.40:0/3178064883 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:31.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.957+0000 7f3605245640 1 -- 172.21.15.40:0/3178064883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f360015b4d0 msgr2=0x7f360014a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:31.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.957+0000 7f3605245640 1 --2- 172.21.15.40:0/3178064883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f360015b4d0 0x7f360014a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.957+0000 7f3605245640 1 -- 172.21.15.40:0/3178064883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f360015b8b0 msgr2=0x7f360015df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:31.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.957+0000 7f3605245640 1 --2- 172.21.15.40:0/3178064883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f360015b8b0 0x7f360015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:31.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.957+0000 7f3605245640 1 -- 172.21.15.40:0/3178064883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc009840 con 0x7f36000d1960 2023-10-18T09:02:31.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.958+0000 7f3605245640 1 --2- 172.21.15.40:0/3178064883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36000d1960 0x7f36000b68f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f35fc006010 tx=0x7f35fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:31.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.958+0000 7f35f5ffb640 1 -- 172.21.15.40:0/3178064883 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc0046f0 con 0x7f36000d1960 2023-10-18T09:02:31.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.958+0000 7f35f5ffb640 1 -- 172.21.15.40:0/3178064883 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f35fc0345a0 con 0x7f36000d1960 2023-10-18T09:02:31.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.958+0000 7f35f5ffb640 1 -- 172.21.15.40:0/3178064883 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc034990 con 0x7f36000d1960 2023-10-18T09:02:31.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.958+0000 7f3606247640 1 -- 172.21.15.40:0/3178064883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36000b6f20 con 0x7f36000d1960 2023-10-18T09:02:31.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.958+0000 7f3606247640 1 -- 172.21.15.40:0/3178064883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36000b4580 con 0x7f36000d1960 2023-10-18T09:02:31.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.959+0000 7f3606247640 1 -- 172.21.15.40:0/3178064883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3600071da0 con 0x7f36000d1960 2023-10-18T09:02:31.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.960+0000 7f35f5ffb640 1 -- 172.21.15.40:0/3178064883 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f35fc034af0 con 0x7f36000d1960 2023-10-18T09:02:31.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.961+0000 7f35f5ffb640 1 --2- 172.21.15.40:0/3178064883 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35e406ae40 0x7f35e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:31.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.961+0000 7f35f5ffb640 1 -- 172.21.15.40:0/3178064883 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f35fc0ac550 con 0x7f36000d1960 2023-10-18T09:02:31.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.962+0000 7f3604a44640 1 --2- 172.21.15.40:0/3178064883 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35e406ae40 0x7f35e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:31.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.963+0000 7f3604a44640 1 --2- 172.21.15.40:0/3178064883 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35e406ae40 0x7f35e406d300 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f35f0007fd0 tx=0x7f35f0007920 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:31.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:31.965+0000 7f35f5ffb640 1 -- 172.21.15.40:0/3178064883 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35fc077a10 con 0x7f36000d1960 2023-10-18T09:02:32.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.160+0000 7f3606247640 1 -- 172.21.15.40:0/3178064883 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3600066e00 con 0x7f36000d1960 2023-10-18T09:02:32.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.161+0000 7f35f5ffb640 1 -- 172.21.15.40:0/3178064883 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35fc043020 con 0x7f36000d1960 2023-10-18T09:02:32.164 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:02:32.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.163+0000 7f3606247640 1 -- 172.21.15.40:0/3178064883 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35e406ae40 msgr2=0x7f35e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:32.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.163+0000 7f3606247640 1 --2- 172.21.15.40:0/3178064883 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35e406ae40 0x7f35e406d300 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f35f0007fd0 tx=0x7f35f0007920 comp rx=0 tx=0).stop 2023-10-18T09:02:32.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.163+0000 7f3606247640 1 -- 172.21.15.40:0/3178064883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36000d1960 msgr2=0x7f36000b68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:32.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.163+0000 7f3606247640 1 --2- 172.21.15.40:0/3178064883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36000d1960 0x7f36000b68f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f35fc006010 tx=0x7f35fc0042f0 comp rx=0 tx=0).stop 2023-10-18T09:02:32.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.163+0000 7f3606247640 1 -- 172.21.15.40:0/3178064883 shutdown_connections 2023-10-18T09:02:32.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.163+0000 7f3606247640 1 --2- 172.21.15.40:0/3178064883 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f35e406ae40 0x7f35e406d300 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:32.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.163+0000 7f3606247640 1 --2- 172.21.15.40:0/3178064883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f360015b8b0 0x7f360015df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:32.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.163+0000 7f3606247640 1 --2- 172.21.15.40:0/3178064883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f360015b4d0 0x7f360014a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:32.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.163+0000 7f3606247640 1 --2- 172.21.15.40:0/3178064883 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36000d1960 0x7f36000b68f0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:32.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.163+0000 7f3606247640 1 -- 172.21.15.40:0/3178064883 >> 172.21.15.40:0/3178064883 conn(0x7f360008ad30 msgr2=0x7f360006bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:32.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.164+0000 7f3606247640 1 -- 172.21.15.40:0/3178064883 shutdown_connections 2023-10-18T09:02:32.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.164+0000 7f3606247640 1 -- 172.21.15.40:0/3178064883 wait complete. 2023-10-18T09:02:32.189 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2023-10-18T09:02:32.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.415+0000 7f3ab6168640 1 -- 172.21.15.40:0/4042845277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab00654f0 msgr2=0x7f3ab0070240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:32.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.415+0000 7f3ab6168640 1 --2- 172.21.15.40:0/4042845277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab00654f0 0x7f3ab0070240 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0071ed0 tx=0x7f3a9802bd90 comp rx=0 tx=0).stop 2023-10-18T09:02:32.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.415+0000 7f3ab6168640 1 -- 172.21.15.40:0/4042845277 shutdown_connections 2023-10-18T09:02:32.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.415+0000 7f3ab6168640 1 --2- 172.21.15.40:0/4042845277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ab006bec0 0x7f3ab00cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:32.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.415+0000 7f3ab6168640 1 --2- 172.21.15.40:0/4042845277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab00654f0 0x7f3ab0070240 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:32.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.415+0000 7f3ab6168640 1 --2- 172.21.15.40:0/4042845277 >> v2:172.21.15.144:3300/0 conn(0x7f3ab006fe60 0x7f3ab00cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:32.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.415+0000 7f3ab6168640 1 -- 172.21.15.40:0/4042845277 >> 172.21.15.40:0/4042845277 conn(0x7f3ab00af530 msgr2=0x7f3ab00af910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:32.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.415+0000 7f3ab6168640 1 -- 172.21.15.40:0/4042845277 shutdown_connections 2023-10-18T09:02:32.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.415+0000 7f3ab6168640 1 -- 172.21.15.40:0/4042845277 wait complete. 2023-10-18T09:02:32.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.417+0000 7f3ab6168640 1 Processor -- start 2023-10-18T09:02:32.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.417+0000 7f3ab6168640 1 -- start start 2023-10-18T09:02:32.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.419+0000 7f3ab6168640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ab00654f0 0x7f3ab01483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:32.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.419+0000 7f3ab6168640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab006bec0 0x7f3ab006dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:32.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.419+0000 7f3ab6168640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ab006fe60 0x7f3ab014af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:32.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.419+0000 7f3ab6168640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab00c31e0 con 0x7f3ab006bec0 2023-10-18T09:02:32.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.419+0000 7f3ab6168640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab015eab0 con 0x7f3ab006fe60 2023-10-18T09:02:32.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.419+0000 7f3ab6168640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab0145ae0 con 0x7f3ab00654f0 2023-10-18T09:02:32.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.419+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab006bec0 0x7f3ab006dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:32.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.419+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab006bec0 0x7f3ab006dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50290/0 (socket says 172.21.15.40:50290) 2023-10-18T09:02:32.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.419+0000 7f3aa7fff640 1 -- 172.21.15.40:0/2340396547 learned_addr learned my addr 172.21.15.40:0/2340396547 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:32.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.420+0000 7f3aaf7fe640 1 --2- 172.21.15.40:0/2340396547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ab00654f0 0x7f3ab01483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:32.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.420+0000 7f3aaffff640 1 --2- 172.21.15.40:0/2340396547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ab006fe60 0x7f3ab014af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:32.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.420+0000 7f3aa7fff640 1 -- 172.21.15.40:0/2340396547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ab00654f0 msgr2=0x7f3ab01483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:32.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.420+0000 7f3aa7fff640 1 --2- 172.21.15.40:0/2340396547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ab00654f0 0x7f3ab01483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:32.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.420+0000 7f3aa7fff640 1 -- 172.21.15.40:0/2340396547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ab006fe60 msgr2=0x7f3ab014af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:32.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.420+0000 7f3aa7fff640 1 --2- 172.21.15.40:0/2340396547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ab006fe60 0x7f3ab014af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:32.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.420+0000 7f3aa7fff640 1 -- 172.21.15.40:0/2340396547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a98009560 con 0x7f3ab006bec0 2023-10-18T09:02:32.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.420+0000 7f3aaffff640 1 --2- 172.21.15.40:0/2340396547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ab006fe60 0x7f3ab014af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:32.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.420+0000 7f3aaf7fe640 1 --2- 172.21.15.40:0/2340396547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ab00654f0 0x7f3ab01483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:32.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.420+0000 7f3aa7fff640 1 --2- 172.21.15.40:0/2340396547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab006bec0 0x7f3ab006dc30 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009850 tx=0x7f3a9802dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:32.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.420+0000 7f3aacff9640 1 -- 172.21.15.40:0/2340396547 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98003800 con 0x7f3ab006bec0 2023-10-18T09:02:32.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.421+0000 7f3aacff9640 1 -- 172.21.15.40:0/2340396547 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3a98003a50 con 0x7f3ab006bec0 2023-10-18T09:02:32.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.421+0000 7f3aacff9640 1 -- 172.21.15.40:0/2340396547 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98003e20 con 0x7f3ab006bec0 2023-10-18T09:02:32.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.421+0000 7f3ab6168640 1 -- 172.21.15.40:0/2340396547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab0145dc0 con 0x7f3ab006bec0 2023-10-18T09:02:32.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.421+0000 7f3ab6168640 1 -- 172.21.15.40:0/2340396547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab00c2a10 con 0x7f3ab006bec0 2023-10-18T09:02:32.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.422+0000 7f3ab6168640 1 -- 172.21.15.40:0/2340396547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab0146280 con 0x7f3ab006bec0 2023-10-18T09:02:32.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.423+0000 7f3aacff9640 1 -- 172.21.15.40:0/2340396547 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3a9803e450 con 0x7f3ab006bec0 2023-10-18T09:02:32.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.424+0000 7f3aacff9640 1 --2- 172.21.15.40:0/2340396547 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3a9006ae40 0x7f3a9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:32.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.425+0000 7f3aacff9640 1 -- 172.21.15.40:0/2340396547 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3a980abf10 con 0x7f3ab006bec0 2023-10-18T09:02:32.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.425+0000 7f3aaf7fe640 1 --2- 172.21.15.40:0/2340396547 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3a9006ae40 0x7f3a9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:32.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.426+0000 7f3aaf7fe640 1 --2- 172.21.15.40:0/2340396547 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3a9006ae40 0x7f3a9006d300 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000ba00 tx=0x7f3aa0005c80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:32.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.432+0000 7f3aacff9640 1 -- 172.21.15.40:0/2340396547 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3a98077220 con 0x7f3ab006bec0 2023-10-18T09:02:32.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:32.695+0000 7f3ab6168640 1 -- 172.21.15.40:0/2340396547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f3ab0066fd0 con 0x7f3ab006bec0 2023-10-18T09:02:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.227+0000 7f3aacff9640 1 -- 172.21.15.40:0/2340396547 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v378) v1 ==== 131+0+0 (secure 0 0 0) 0x7f3a980420c0 con 0x7f3ab006bec0 2023-10-18T09:02:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.231+0000 7f3ab6168640 1 -- 172.21.15.40:0/2340396547 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3a9006ae40 msgr2=0x7f3a9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.231+0000 7f3ab6168640 1 --2- 172.21.15.40:0/2340396547 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3a9006ae40 0x7f3a9006d300 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000ba00 tx=0x7f3aa0005c80 comp rx=0 tx=0).stop 2023-10-18T09:02:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.231+0000 7f3ab6168640 1 -- 172.21.15.40:0/2340396547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab006bec0 msgr2=0x7f3ab006dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:33.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.231+0000 7f3ab6168640 1 --2- 172.21.15.40:0/2340396547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab006bec0 0x7f3ab006dc30 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009850 tx=0x7f3a9802dac0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.231+0000 7f3ab6168640 1 -- 172.21.15.40:0/2340396547 shutdown_connections 2023-10-18T09:02:33.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.231+0000 7f3ab6168640 1 --2- 172.21.15.40:0/2340396547 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3a9006ae40 0x7f3a9006d300 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.231+0000 7f3ab6168640 1 --2- 172.21.15.40:0/2340396547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ab006fe60 0x7f3ab014af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.231+0000 7f3ab6168640 1 --2- 172.21.15.40:0/2340396547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab006bec0 0x7f3ab006dc30 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.231+0000 7f3ab6168640 1 --2- 172.21.15.40:0/2340396547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ab00654f0 0x7f3ab01483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.231+0000 7f3ab6168640 1 -- 172.21.15.40:0/2340396547 >> 172.21.15.40:0/2340396547 conn(0x7f3ab00af530 msgr2=0x7f3ab00ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:33.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.231+0000 7f3ab6168640 1 -- 172.21.15.40:0/2340396547 shutdown_connections 2023-10-18T09:02:33.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.231+0000 7f3ab6168640 1 -- 172.21.15.40:0/2340396547 wait complete. 2023-10-18T09:02:33.258 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T09:02:33.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.487+0000 7f3e254a3640 1 -- 172.21.15.40:0/313066272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e20157cd0 msgr2=0x7f3e20158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:33.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.487+0000 7f3e254a3640 1 --2- 172.21.15.40:0/313066272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e20157cd0 0x7f3e20158150 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3e14005fd0 tx=0x7f3e1402d400 comp rx=0 tx=0).stop 2023-10-18T09:02:33.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.488+0000 7f3e254a3640 1 -- 172.21.15.40:0/313066272 shutdown_connections 2023-10-18T09:02:33.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.488+0000 7f3e254a3640 1 --2- 172.21.15.40:0/313066272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e20157cd0 0x7f3e20158150 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.488+0000 7f3e254a3640 1 --2- 172.21.15.40:0/313066272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e2006d6f0 0x7f3e20071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.488+0000 7f3e254a3640 1 -- 172.21.15.40:0/313066272 >> 172.21.15.40:0/313066272 conn(0x7f3e200aed70 msgr2=0x7f3e200af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:33.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.488+0000 7f3e254a3640 1 -- 172.21.15.40:0/313066272 shutdown_connections 2023-10-18T09:02:33.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.488+0000 7f3e254a3640 1 -- 172.21.15.40:0/313066272 wait complete. 2023-10-18T09:02:33.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.489+0000 7f3e254a3640 1 Processor -- start 2023-10-18T09:02:33.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.490+0000 7f3e254a3640 1 -- start start 2023-10-18T09:02:33.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.491+0000 7f3e254a3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e2006d6f0 0x7f3e201349b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:33.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.491+0000 7f3e254a3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e20157cd0 0x7f3e20154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:33.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.491+0000 7f3e254a3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e2015af90 0x7f3e20159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:33.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.491+0000 7f3e254a3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e20134ef0 con 0x7f3e20157cd0 2023-10-18T09:02:33.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.491+0000 7f3e254a3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e20135390 con 0x7f3e2015af90 2023-10-18T09:02:33.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.491+0000 7f3e254a3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e20154c70 con 0x7f3e2006d6f0 2023-10-18T09:02:33.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.491+0000 7f3e1f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e20157cd0 0x7f3e20154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:33.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.491+0000 7f3e1f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e20157cd0 0x7f3e20154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50292/0 (socket says 172.21.15.40:50292) 2023-10-18T09:02:33.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.491+0000 7f3e1f7fe640 1 -- 172.21.15.40:0/1170231062 learned_addr learned my addr 172.21.15.40:0/1170231062 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:33.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.491+0000 7f3e1ffff640 1 --2- 172.21.15.40:0/1170231062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e2006d6f0 0x7f3e201349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:33.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.491+0000 7f3e24ca2640 1 --2- 172.21.15.40:0/1170231062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e2015af90 0x7f3e20159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:33.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.492+0000 7f3e1f7fe640 1 -- 172.21.15.40:0/1170231062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e2006d6f0 msgr2=0x7f3e201349b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:33.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.492+0000 7f3e1f7fe640 1 --2- 172.21.15.40:0/1170231062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e2006d6f0 0x7f3e201349b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.492+0000 7f3e1f7fe640 1 -- 172.21.15.40:0/1170231062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e2015af90 msgr2=0x7f3e20159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:33.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.492+0000 7f3e1f7fe640 1 --2- 172.21.15.40:0/1170231062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e2015af90 0x7f3e20159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.492+0000 7f3e1f7fe640 1 -- 172.21.15.40:0/1170231062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e10009490 con 0x7f3e20157cd0 2023-10-18T09:02:33.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.492+0000 7f3e1f7fe640 1 --2- 172.21.15.40:0/1170231062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e20157cd0 0x7f3e20154240 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3e14006010 tx=0x7f3e14032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:33.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.492+0000 7f3e1cff9640 1 -- 172.21.15.40:0/1170231062 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14004240 con 0x7f3e20157cd0 2023-10-18T09:02:33.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.492+0000 7f3e1cff9640 1 -- 172.21.15.40:0/1170231062 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3e14004490 con 0x7f3e20157cd0 2023-10-18T09:02:33.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.492+0000 7f3e254a3640 1 -- 172.21.15.40:0/1170231062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1400a950 con 0x7f3e20157cd0 2023-10-18T09:02:33.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.492+0000 7f3e1cff9640 1 -- 172.21.15.40:0/1170231062 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14004860 con 0x7f3e20157cd0 2023-10-18T09:02:33.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.493+0000 7f3e254a3640 1 -- 172.21.15.40:0/1170231062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e20152340 con 0x7f3e20157cd0 2023-10-18T09:02:33.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.494+0000 7f3e254a3640 1 -- 172.21.15.40:0/1170231062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e20066e00 con 0x7f3e20157cd0 2023-10-18T09:02:33.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.495+0000 7f3e1cff9640 1 -- 172.21.15.40:0/1170231062 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3e14034440 con 0x7f3e20157cd0 2023-10-18T09:02:33.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.496+0000 7f3e1cff9640 1 --2- 172.21.15.40:0/1170231062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e0c06ae70 0x7f3e0c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:33.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.496+0000 7f3e1cff9640 1 -- 172.21.15.40:0/1170231062 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3e140aca20 con 0x7f3e20157cd0 2023-10-18T09:02:33.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.496+0000 7f3e1ffff640 1 --2- 172.21.15.40:0/1170231062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e0c06ae70 0x7f3e0c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:33.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.497+0000 7f3e1ffff640 1 --2- 172.21.15.40:0/1170231062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e0c06ae70 0x7f3e0c06d330 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009d60 tx=0x7f3e10005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:33.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.500+0000 7f3e1cff9640 1 -- 172.21.15.40:0/1170231062 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e14077e70 con 0x7f3e20157cd0 2023-10-18T09:02:33.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.693+0000 7f3e254a3640 1 -- 172.21.15.40:0/1170231062 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f3e200706b0 con 0x7f3e20157cd0 2023-10-18T09:02:33.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.695+0000 7f3e1cff9640 1 -- 172.21.15.40:0/1170231062 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v32) v1 ==== 59+0+1452 (secure 0 0 0) 0x7f3e1404c5c0 con 0x7f3e20157cd0 2023-10-18T09:02:33.698 INFO:teuthology.orchestra.run.smithi040.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T09:02:33.698 INFO:teuthology.orchestra.run.smithi040.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T09:02:33.698 INFO:teuthology.orchestra.run.smithi040.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T09:02:33.698 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds 20/20 2023-10-18T09:02:33.699 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T09:02:33.699 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_ms 1/1 2023-10-18T09:02:33.699 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T09:02:33.699 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T09:02:33.699 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T09:02:33.700 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_size 100 2023-10-18T09:02:33.700 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_frag true 2023-10-18T09:02:33.700 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_scatterstat true 2023-10-18T09:02:33.700 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T09:02:33.701 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_verify_scatter true 2023-10-18T09:02:33.701 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T09:02:33.701 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T09:02:33.701 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T09:02:33.701 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced client_mount_timeout 600 2023-10-18T09:02:33.702 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_client 20/20 2023-10-18T09:02:33.702 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_ms 1/1 2023-10-18T09:02:33.702 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T09:02:33.702 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T09:02:33.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.697+0000 7f3e254a3640 1 -- 172.21.15.40:0/1170231062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e0c06ae70 msgr2=0x7f3e0c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:33.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.697+0000 7f3e254a3640 1 --2- 172.21.15.40:0/1170231062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e0c06ae70 0x7f3e0c06d330 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009d60 tx=0x7f3e10005f90 comp rx=0 tx=0).stop 2023-10-18T09:02:33.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.697+0000 7f3e254a3640 1 -- 172.21.15.40:0/1170231062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e20157cd0 msgr2=0x7f3e20154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:33.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.697+0000 7f3e254a3640 1 --2- 172.21.15.40:0/1170231062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e20157cd0 0x7f3e20154240 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f3e14006010 tx=0x7f3e14032c10 comp rx=0 tx=0).stop 2023-10-18T09:02:33.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.697+0000 7f3e254a3640 1 -- 172.21.15.40:0/1170231062 shutdown_connections 2023-10-18T09:02:33.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.697+0000 7f3e254a3640 1 --2- 172.21.15.40:0/1170231062 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e0c06ae70 0x7f3e0c06d330 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.697+0000 7f3e254a3640 1 --2- 172.21.15.40:0/1170231062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e2015af90 0x7f3e20159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.697+0000 7f3e254a3640 1 --2- 172.21.15.40:0/1170231062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e20157cd0 0x7f3e20154240 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.697+0000 7f3e254a3640 1 --2- 172.21.15.40:0/1170231062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e2006d6f0 0x7f3e201349b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.697+0000 7f3e254a3640 1 -- 172.21.15.40:0/1170231062 >> 172.21.15.40:0/1170231062 conn(0x7f3e200aed70 msgr2=0x7f3e201592d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:33.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.697+0000 7f3e254a3640 1 -- 172.21.15.40:0/1170231062 shutdown_connections 2023-10-18T09:02:33.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.697+0000 7f3e254a3640 1 -- 172.21.15.40:0/1170231062 wait complete. 2023-10-18T09:02:33.719 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T09:02:33.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.947+0000 7f47460b4640 1 -- 172.21.15.40:0/3091949489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f474006d5d0 msgr2=0x7f47400c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:33.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.947+0000 7f47460b4640 1 --2- 172.21.15.40:0/3091949489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f474006d5d0 0x7f47400c9620 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f47400b2620 tx=0x7f473c02bcf0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.948+0000 7f47460b4640 1 -- 172.21.15.40:0/3091949489 shutdown_connections 2023-10-18T09:02:33.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.948+0000 7f47460b4640 1 --2- 172.21.15.40:0/3091949489 >> v2:172.21.15.144:3300/0 conn(0x7f474006c2d0 0x7f4740070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.948+0000 7f47460b4640 1 --2- 172.21.15.40:0/3091949489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f474006d5d0 0x7f47400c9620 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.948+0000 7f47460b4640 1 -- 172.21.15.40:0/3091949489 >> 172.21.15.40:0/3091949489 conn(0x7f474006ca10 msgr2=0x7f474006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:33.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.948+0000 7f47460b4640 1 -- 172.21.15.40:0/3091949489 shutdown_connections 2023-10-18T09:02:33.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.948+0000 7f47460b4640 1 -- 172.21.15.40:0/3091949489 wait complete. 2023-10-18T09:02:33.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.949+0000 7f47460b4640 1 Processor -- start 2023-10-18T09:02:33.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.950+0000 7f47460b4640 1 -- start start 2023-10-18T09:02:33.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.951+0000 7f47460b4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f474006c2d0 0x7f4740154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:33.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.951+0000 7f47460b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f474006d5d0 0x7f47401f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:33.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.951+0000 7f47460b4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f474015af70 0x7f4740147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:33.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.951+0000 7f47460b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4740148820 con 0x7f474006d5d0 2023-10-18T09:02:33.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.951+0000 7f47460b4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47401520e0 con 0x7f474015af70 2023-10-18T09:02:33.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.951+0000 7f47460b4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47401523c0 con 0x7f474006c2d0 2023-10-18T09:02:33.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.951+0000 7f47448b1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f474006d5d0 0x7f47401f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:33.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.951+0000 7f47448b1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f474006d5d0 0x7f47401f17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50316/0 (socket says 172.21.15.40:50316) 2023-10-18T09:02:33.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.951+0000 7f47448b1640 1 -- 172.21.15.40:0/1721789623 learned_addr learned my addr 172.21.15.40:0/1721789623 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:33.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.951+0000 7f47450b2640 1 --2- 172.21.15.40:0/1721789623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f474006c2d0 0x7f4740154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:33.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.951+0000 7f47448b1640 1 -- 172.21.15.40:0/1721789623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f474006c2d0 msgr2=0x7f4740154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:33.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.951+0000 7f47448b1640 1 --2- 172.21.15.40:0/1721789623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f474006c2d0 0x7f4740154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.951+0000 7f47448b1640 1 -- 172.21.15.40:0/1721789623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f474015af70 msgr2=0x7f4740147e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:33.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.951+0000 7f47458b3640 1 --2- 172.21.15.40:0/1721789623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f474015af70 0x7f4740147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:33.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.952+0000 7f47448b1640 1 --2- 172.21.15.40:0/1721789623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f474015af70 0x7f4740147e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:33.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.952+0000 7f47448b1640 1 -- 172.21.15.40:0/1721789623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c009490 con 0x7f474006d5d0 2023-10-18T09:02:33.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.952+0000 7f47448b1640 1 --2- 172.21.15.40:0/1721789623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f474006d5d0 0x7f47401f17f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f473000c3e0 tx=0x7f473000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:33.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.952+0000 7f4735ffb640 1 -- 172.21.15.40:0/1721789623 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730011790 con 0x7f474006d5d0 2023-10-18T09:02:33.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.952+0000 7f4735ffb640 1 -- 172.21.15.40:0/1721789623 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f47300119e0 con 0x7f474006d5d0 2023-10-18T09:02:33.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.952+0000 7f47460b4640 1 -- 172.21.15.40:0/1721789623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47401517e0 con 0x7f474006d5d0 2023-10-18T09:02:33.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.952+0000 7f4735ffb640 1 -- 172.21.15.40:0/1721789623 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730011df0 con 0x7f474006d5d0 2023-10-18T09:02:33.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.952+0000 7f47460b4640 1 -- 172.21.15.40:0/1721789623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47401526a0 con 0x7f474006d5d0 2023-10-18T09:02:33.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.953+0000 7f47460b4640 1 -- 172.21.15.40:0/1721789623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4740151d00 con 0x7f474006d5d0 2023-10-18T09:02:33.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.954+0000 7f4735ffb640 1 -- 172.21.15.40:0/1721789623 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f473001c460 con 0x7f474006d5d0 2023-10-18T09:02:33.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.955+0000 7f4735ffb640 1 --2- 172.21.15.40:0/1721789623 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f472406ae40 0x7f472406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:33.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.955+0000 7f4735ffb640 1 -- 172.21.15.40:0/1721789623 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4730089260 con 0x7f474006d5d0 2023-10-18T09:02:33.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.956+0000 7f47450b2640 1 --2- 172.21.15.40:0/1721789623 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f472406ae40 0x7f472406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:33.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.957+0000 7f47450b2640 1 --2- 172.21.15.40:0/1721789623 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f472406ae40 0x7f472406d300 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f473c036000 tx=0x7f473c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:33.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:33.959+0000 7f4735ffb640 1 -- 172.21.15.40:0/1721789623 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4740151d00 con 0x7f474006d5d0 2023-10-18T09:02:34.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.154+0000 7f47460b4640 1 -- 172.21.15.40:0/1721789623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4740151d00 con 0x7f474006d5d0 2023-10-18T09:02:34.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.163+0000 7f4735ffb640 1 -- 172.21.15.40:0/1721789623 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v33)=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4740151d00 con 0x7f474006d5d0 2023-10-18T09:02:34.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.165+0000 7f47460b4640 1 -- 172.21.15.40:0/1721789623 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f472406ae40 msgr2=0x7f472406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:34.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.165+0000 7f47460b4640 1 --2- 172.21.15.40:0/1721789623 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f472406ae40 0x7f472406d300 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f473c036000 tx=0x7f473c004730 comp rx=0 tx=0).stop 2023-10-18T09:02:34.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.165+0000 7f47460b4640 1 -- 172.21.15.40:0/1721789623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f474006d5d0 msgr2=0x7f47401f17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:34.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.165+0000 7f47460b4640 1 --2- 172.21.15.40:0/1721789623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f474006d5d0 0x7f47401f17f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f473000c3e0 tx=0x7f473000c8a0 comp rx=0 tx=0).stop 2023-10-18T09:02:34.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.166+0000 7f47460b4640 1 -- 172.21.15.40:0/1721789623 shutdown_connections 2023-10-18T09:02:34.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.166+0000 7f47460b4640 1 --2- 172.21.15.40:0/1721789623 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f472406ae40 0x7f472406d300 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:34.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.166+0000 7f47460b4640 1 --2- 172.21.15.40:0/1721789623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f474015af70 0x7f4740147e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:34.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.166+0000 7f47460b4640 1 --2- 172.21.15.40:0/1721789623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f474006d5d0 0x7f47401f17f0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:34.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.166+0000 7f47460b4640 1 --2- 172.21.15.40:0/1721789623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f474006c2d0 0x7f4740154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.166+0000 7f47460b4640 1 -- 172.21.15.40:0/1721789623 >> 172.21.15.40:0/1721789623 conn(0x7f474006ca10 msgr2=0x7f47400b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.166+0000 7f47460b4640 1 -- 172.21.15.40:0/1721789623 shutdown_connections 2023-10-18T09:02:34.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.166+0000 7f47460b4640 1 -- 172.21.15.40:0/1721789623 wait complete. 2023-10-18T09:02:34.188 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T09:02:34.189 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:02:34.312 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:02:34.434 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T09:02:34.435 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:02:34.435 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:02:34.500 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:02:34.500 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:02:34.501 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:02:34.561 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:02:34.562 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:02:34.562 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:02:34.627 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:02:34.627 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:02:34.628 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:02:34.687 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:02:34.688 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T09:02:34.688 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:02:34.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.928+0000 7fc3b610c640 1 -- 172.21.15.40:0/2772029808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b00b15b0 msgr2=0x7fc3b00c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:34.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.928+0000 7fc3b610c640 1 --2- 172.21.15.40:0/2772029808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b00b15b0 0x7fc3b00c99c0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fc3b006b050 tx=0x7fc3a402bd60 comp rx=0 tx=0).stop 2023-10-18T09:02:34.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.928+0000 7fc3b610c640 1 -- 172.21.15.40:0/2772029808 shutdown_connections 2023-10-18T09:02:34.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.928+0000 7fc3b610c640 1 --2- 172.21.15.40:0/2772029808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3b00c9f00 0x7fc3b0157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:34.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.928+0000 7fc3b610c640 1 --2- 172.21.15.40:0/2772029808 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b00b15b0 0x7fc3b00c99c0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:34.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.928+0000 7fc3b610c640 1 -- 172.21.15.40:0/2772029808 >> 172.21.15.40:0/2772029808 conn(0x7fc3b0069720 msgr2=0x7fc3b0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:34.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.928+0000 7fc3b610c640 1 -- 172.21.15.40:0/2772029808 shutdown_connections 2023-10-18T09:02:34.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.928+0000 7fc3b610c640 1 -- 172.21.15.40:0/2772029808 wait complete. 2023-10-18T09:02:34.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.930+0000 7fc3b610c640 1 Processor -- start 2023-10-18T09:02:34.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.930+0000 7fc3b610c640 1 -- start start 2023-10-18T09:02:34.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.931+0000 7fc3b610c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3b006d6f0 0x7fc3b0149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:34.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.931+0000 7fc3b610c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3b00b15b0 0x7fc3b0149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.931+0000 7fc3b610c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b00c9f00 0x7fc3b014b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.931+0000 7fc3b610c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b0159900 con 0x7fc3b00c9f00 2023-10-18T09:02:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.931+0000 7fc3b610c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b0159be0 con 0x7fc3b006d6f0 2023-10-18T09:02:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.931+0000 7fc3b610c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b0159ec0 con 0x7fc3b00b15b0 2023-10-18T09:02:34.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.931+0000 7fc3b590b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b00c9f00 0x7fc3b014b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:34.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.931+0000 7fc3b590b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b00c9f00 0x7fc3b014b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48114/0 (socket says 172.21.15.40:48114) 2023-10-18T09:02:34.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.931+0000 7fc3b590b640 1 -- 172.21.15.40:0/2401726261 learned_addr learned my addr 172.21.15.40:0/2401726261 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:34.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.931+0000 7fc3b510a640 1 --2- 172.21.15.40:0/2401726261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3b006d6f0 0x7fc3b0149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:34.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.931+0000 7fc3b4909640 1 --2- 172.21.15.40:0/2401726261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3b00b15b0 0x7fc3b0149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.931+0000 7fc3b590b640 1 -- 172.21.15.40:0/2401726261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3b00b15b0 msgr2=0x7fc3b0149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.931+0000 7fc3b590b640 1 --2- 172.21.15.40:0/2401726261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3b00b15b0 0x7fc3b0149940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.931+0000 7fc3b590b640 1 -- 172.21.15.40:0/2401726261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3b006d6f0 msgr2=0x7fc3b0149400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.931+0000 7fc3b590b640 1 --2- 172.21.15.40:0/2401726261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3b006d6f0 0x7fc3b0149400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.932+0000 7fc3b590b640 1 -- 172.21.15.40:0/2401726261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a4009560 con 0x7fc3b00c9f00 2023-10-18T09:02:34.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.932+0000 7fc3b590b640 1 --2- 172.21.15.40:0/2401726261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b00c9f00 0x7fc3b014b370 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800b970 tx=0x7fc3a800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:34.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.932+0000 7fc3a1ffb640 1 -- 172.21.15.40:0/2401726261 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a800cb40 con 0x7fc3b00c9f00 2023-10-18T09:02:34.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.932+0000 7fc3a1ffb640 1 -- 172.21.15.40:0/2401726261 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc3a800cdb0 con 0x7fc3b00c9f00 2023-10-18T09:02:34.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.932+0000 7fc3a1ffb640 1 -- 172.21.15.40:0/2401726261 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a80146a0 con 0x7fc3b00c9f00 2023-10-18T09:02:34.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.932+0000 7fc3b610c640 1 -- 172.21.15.40:0/2401726261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b015bc10 con 0x7fc3b00c9f00 2023-10-18T09:02:34.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.932+0000 7fc3b610c640 1 -- 172.21.15.40:0/2401726261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b015a480 con 0x7fc3b00c9f00 2023-10-18T09:02:34.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.933+0000 7fc3b610c640 1 -- 172.21.15.40:0/2401726261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b0066e00 con 0x7fc3b00c9f00 2023-10-18T09:02:34.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.934+0000 7fc3a1ffb640 1 -- 172.21.15.40:0/2401726261 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc3a801d3b0 con 0x7fc3b00c9f00 2023-10-18T09:02:34.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.936+0000 7fc3a1ffb640 1 --2- 172.21.15.40:0/2401726261 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc39406ae40 0x7fc39406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:34.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.936+0000 7fc3a1ffb640 1 -- 172.21.15.40:0/2401726261 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc3a8089530 con 0x7fc3b00c9f00 2023-10-18T09:02:34.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.937+0000 7fc3b510a640 1 --2- 172.21.15.40:0/2401726261 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc39406ae40 0x7fc39406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:34.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.938+0000 7fc3b510a640 1 --2- 172.21.15.40:0/2401726261 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc39406ae40 0x7fc39406d300 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00c97c0 tx=0x7fc3ac009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:34.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:34.941+0000 7fc3a1ffb640 1 -- 172.21.15.40:0/2401726261 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc3a80548d0 con 0x7fc3b00c9f00 2023-10-18T09:02:35.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.176+0000 7fc3b610c640 1 -- 172.21.15.40:0/2401726261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3b0071500 con 0x7fc3b00c9f00 2023-10-18T09:02:35.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.177+0000 7fc3a1ffb640 1 -- 172.21.15.40:0/2401726261 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fc3a8019090 con 0x7fc3b00c9f00 2023-10-18T09:02:35.180 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:35.180 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2565609773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2565609773},{"type":"v1","addr":"172.21.15.144:6836","nonce":2565609773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":5872,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3371548374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3371548374},{"type":"v1","addr":"172.21.15.144:6837","nonce":3371548374}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/3328648628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3328648628},{"type":"v1","addr":"172.21.15.40:6833","nonce":3328648628}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1679163992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1679163992},{"type":"v1","addr":"172.21.15.40:6835","nonce":1679163992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[]} 2023-10-18T09:02:35.180 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 51 2023-10-18T09:02:35.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.180+0000 7fc3b610c640 1 -- 172.21.15.40:0/2401726261 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc39406ae40 msgr2=0x7fc39406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.180+0000 7fc3b610c640 1 --2- 172.21.15.40:0/2401726261 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc39406ae40 0x7fc39406d300 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00c97c0 tx=0x7fc3ac009040 comp rx=0 tx=0).stop 2023-10-18T09:02:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.180+0000 7fc3b610c640 1 -- 172.21.15.40:0/2401726261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b00c9f00 msgr2=0x7fc3b014b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.180+0000 7fc3b610c640 1 --2- 172.21.15.40:0/2401726261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b00c9f00 0x7fc3b014b370 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800b970 tx=0x7fc3a800be30 comp rx=0 tx=0).stop 2023-10-18T09:02:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.180+0000 7fc3b610c640 1 -- 172.21.15.40:0/2401726261 shutdown_connections 2023-10-18T09:02:35.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.180+0000 7fc3b610c640 1 --2- 172.21.15.40:0/2401726261 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc39406ae40 0x7fc39406d300 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.180+0000 7fc3b610c640 1 --2- 172.21.15.40:0/2401726261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b00c9f00 0x7fc3b014b370 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.180+0000 7fc3b610c640 1 --2- 172.21.15.40:0/2401726261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3b00b15b0 0x7fc3b0149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.180+0000 7fc3b610c640 1 --2- 172.21.15.40:0/2401726261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3b006d6f0 0x7fc3b0149400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.180+0000 7fc3b610c640 1 -- 172.21.15.40:0/2401726261 >> 172.21.15.40:0/2401726261 conn(0x7fc3b0069720 msgr2=0x7fc3b0156b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:35.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.180+0000 7fc3b610c640 1 -- 172.21.15.40:0/2401726261 shutdown_connections 2023-10-18T09:02:35.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.181+0000 7fc3b610c640 1 -- 172.21.15.40:0/2401726261 wait complete. 2023-10-18T09:02:35.204 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T09:02:35.204 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T09:02:35.204 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T09:02:35.205 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:02:35.205 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T09:02:35.205 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T09:02:35.206 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T09:02:35.206 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:02:35.206 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T09:02:35.206 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T09:02:35.207 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T09:02:35.207 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:02:35.207 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T09:02:35.208 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T09:02:35.208 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T09:02:35.208 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:02:35.222 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T09:02:35.223 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T09:02:35.224 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T09:02:35.224 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T09:02:35.229 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T09:02:35.229 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T09:02:35.230 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T09:02:35.230 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T09:02:35.272 INFO:tasks.ceph.mds.c:Started 2023-10-18T09:02:35.272 DEBUG:teuthology.parallel:result is None 2023-10-18T09:02:35.277 INFO:tasks.ceph.mds.b:Started 2023-10-18T09:02:35.278 DEBUG:teuthology.parallel:result is None 2023-10-18T09:02:35.279 INFO:tasks.ceph.mds.a:Started 2023-10-18T09:02:35.279 DEBUG:teuthology.parallel:result is None 2023-10-18T09:02:35.284 INFO:tasks.ceph.mds.d:Started 2023-10-18T09:02:35.285 DEBUG:teuthology.parallel:result is None 2023-10-18T09:02:35.285 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T09:02:35.434 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T09:02:35.438 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-10-18T09:02:35.445 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-18T09:02:35.449 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-18T09:02:35.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.483+0000 7f7d84656640 1 -- 172.21.15.40:0/4006126472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d7c15eaa0 msgr2=0x7f7d7c0ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:35.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.483+0000 7f7d84656640 1 --2- 172.21.15.40:0/4006126472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d7c15eaa0 0x7f7d7c0ccdb0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009900 tx=0x7f7d7802be00 comp rx=0 tx=0).stop 2023-10-18T09:02:35.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.483+0000 7f7d84656640 1 -- 172.21.15.40:0/4006126472 shutdown_connections 2023-10-18T09:02:35.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.483+0000 7f7d84656640 1 --2- 172.21.15.40:0/4006126472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d7c0b1b90 0x7f7d7c0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:35.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.483+0000 7f7d84656640 1 --2- 172.21.15.40:0/4006126472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d7c15eaa0 0x7f7d7c0ccdb0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:35.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.483+0000 7f7d84656640 1 -- 172.21.15.40:0/4006126472 >> 172.21.15.40:0/4006126472 conn(0x7f7d7c06cee0 msgr2=0x7f7d7c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:35.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.483+0000 7f7d84656640 1 -- 172.21.15.40:0/4006126472 shutdown_connections 2023-10-18T09:02:35.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.483+0000 7f7d84656640 1 -- 172.21.15.40:0/4006126472 wait complete. 2023-10-18T09:02:35.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.485+0000 7f7d84656640 1 Processor -- start 2023-10-18T09:02:35.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.485+0000 7f7d84656640 1 -- start start 2023-10-18T09:02:35.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.486+0000 7f7d84656640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d7c06ffe0 0x7f7d7c155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:35.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.486+0000 7f7d84656640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d7c0b1b90 0x7f7d7c153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:35.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.486+0000 7f7d84656640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d7c15eaa0 0x7f7d7c0cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:35.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.486+0000 7f7d84656640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0c9b00 con 0x7f7d7c0b1b90 2023-10-18T09:02:35.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.486+0000 7f7d84656640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c1535b0 con 0x7f7d7c15eaa0 2023-10-18T09:02:35.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.486+0000 7f7d84656640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0ccad0 con 0x7f7d7c06ffe0 2023-10-18T09:02:35.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.486+0000 7f7d81bf1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d7c0b1b90 0x7f7d7c153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:35.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.486+0000 7f7d81bf1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d7c0b1b90 0x7f7d7c153070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48148/0 (socket says 172.21.15.40:48148) 2023-10-18T09:02:35.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.486+0000 7f7d81bf1640 1 -- 172.21.15.40:0/730948686 learned_addr learned my addr 172.21.15.40:0/730948686 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:35.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.486+0000 7f7d823f2640 1 --2- 172.21.15.40:0/730948686 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d7c06ffe0 0x7f7d7c155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:35.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.487+0000 7f7d81bf1640 1 -- 172.21.15.40:0/730948686 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d7c06ffe0 msgr2=0x7f7d7c155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:35.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.487+0000 7f7d81bf1640 1 --2- 172.21.15.40:0/730948686 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d7c06ffe0 0x7f7d7c155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:35.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.487+0000 7f7d81bf1640 1 -- 172.21.15.40:0/730948686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d7c15eaa0 msgr2=0x7f7d7c0cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:35.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.487+0000 7f7d81bf1640 1 --2- 172.21.15.40:0/730948686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d7c15eaa0 0x7f7d7c0cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:35.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.487+0000 7f7d81bf1640 1 -- 172.21.15.40:0/730948686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d78009560 con 0x7f7d7c0b1b90 2023-10-18T09:02:35.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.487+0000 7f7d823f2640 1 --2- 172.21.15.40:0/730948686 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d7c06ffe0 0x7f7d7c155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:02:35.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.487+0000 7f7d81bf1640 1 --2- 172.21.15.40:0/730948686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d7c0b1b90 0x7f7d7c153070 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00e9f0 tx=0x7f7d6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:35.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.487+0000 7f7d72ffd640 1 -- 172.21.15.40:0/730948686 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c009700 con 0x7f7d7c0b1b90 2023-10-18T09:02:35.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.487+0000 7f7d72ffd640 1 -- 172.21.15.40:0/730948686 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7d6c00ce70 con 0x7f7d7c0b1b90 2023-10-18T09:02:35.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.487+0000 7f7d84656640 1 -- 172.21.15.40:0/730948686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7c0c9d50 con 0x7f7d7c0b1b90 2023-10-18T09:02:35.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.487+0000 7f7d72ffd640 1 -- 172.21.15.40:0/730948686 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c010670 con 0x7f7d7c0b1b90 2023-10-18T09:02:35.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.487+0000 7f7d84656640 1 -- 172.21.15.40:0/730948686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7c0ca310 con 0x7f7d7c0b1b90 2023-10-18T09:02:35.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.488+0000 7f7d84656640 1 -- 172.21.15.40:0/730948686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7c0ca4d0 con 0x7f7d7c0b1b90 2023-10-18T09:02:35.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.490+0000 7f7d72ffd640 1 -- 172.21.15.40:0/730948686 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7d6c0107d0 con 0x7f7d7c0b1b90 2023-10-18T09:02:35.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.491+0000 7f7d72ffd640 1 --2- 172.21.15.40:0/730948686 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7d6006ae40 0x7f7d6006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:35.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.491+0000 7f7d72ffd640 1 -- 172.21.15.40:0/730948686 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7d6c014070 con 0x7f7d7c0b1b90 2023-10-18T09:02:35.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.491+0000 7f7d823f2640 1 --2- 172.21.15.40:0/730948686 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7d6006ae40 0x7f7d6006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:35.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.492+0000 7f7d823f2640 1 --2- 172.21.15.40:0/730948686 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7d6006ae40 0x7f7d6006d300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7d78035000 tx=0x7f7d78004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:35.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.496+0000 7f7d72ffd640 1 -- 172.21.15.40:0/730948686 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d6c051ca0 con 0x7f7d7c0b1b90 2023-10-18T09:02:35.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:35.706+0000 7f7d84656640 1 -- 172.21.15.40:0/730948686 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7d7c066fd0 con 0x7f7d7c0b1b90 2023-10-18T09:02:36.277 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:02:36.287 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:02:37.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.071+0000 7f7d72ffd640 1 -- 172.21.15.40:0/730948686 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v126) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7d6c058e40 con 0x7f7d7c0b1b90 2023-10-18T09:02:37.074 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2023-10-18T09:02:37.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.075+0000 7f7d84656640 1 -- 172.21.15.40:0/730948686 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7d6006ae40 msgr2=0x7f7d6006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:37.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.075+0000 7f7d84656640 1 --2- 172.21.15.40:0/730948686 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7d6006ae40 0x7f7d6006d300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7d78035000 tx=0x7f7d78004730 comp rx=0 tx=0).stop 2023-10-18T09:02:37.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.075+0000 7f7d84656640 1 -- 172.21.15.40:0/730948686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d7c0b1b90 msgr2=0x7f7d7c153070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:37.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.075+0000 7f7d84656640 1 --2- 172.21.15.40:0/730948686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d7c0b1b90 0x7f7d7c153070 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00e9f0 tx=0x7f7d6c00eeb0 comp rx=0 tx=0).stop 2023-10-18T09:02:37.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.075+0000 7f7d84656640 1 -- 172.21.15.40:0/730948686 shutdown_connections 2023-10-18T09:02:37.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.075+0000 7f7d84656640 1 --2- 172.21.15.40:0/730948686 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7d6006ae40 0x7f7d6006d300 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:37.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.075+0000 7f7d84656640 1 --2- 172.21.15.40:0/730948686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d7c15eaa0 0x7f7d7c0cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:37.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.075+0000 7f7d84656640 1 --2- 172.21.15.40:0/730948686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7d7c0b1b90 0x7f7d7c153070 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:37.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.075+0000 7f7d84656640 1 --2- 172.21.15.40:0/730948686 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d7c06ffe0 0x7f7d7c155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:37.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.075+0000 7f7d84656640 1 -- 172.21.15.40:0/730948686 >> 172.21.15.40:0/730948686 conn(0x7f7d7c06cee0 msgr2=0x7f7d7c06f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:37.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.075+0000 7f7d84656640 1 -- 172.21.15.40:0/730948686 shutdown_connections 2023-10-18T09:02:37.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.075+0000 7f7d84656640 1 -- 172.21.15.40:0/730948686 wait complete. 2023-10-18T09:02:37.097 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T09:02:37.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.327+0000 7fda0b0c2640 1 -- 172.21.15.40:0/578775780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f00039a0 msgr2=0x7fd9f0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:37.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.327+0000 7fda0b0c2640 1 --2- 172.21.15.40:0/578775780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f00039a0 0x7fd9f0003dc0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c640 tx=0x7fd9f00057f0 comp rx=0 tx=0).stop 2023-10-18T09:02:37.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.327+0000 7fda0965f640 1 -- 172.21.15.40:0/578775780 reap_dead start 2023-10-18T09:02:37.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.327+0000 7fda0b0c2640 1 -- 172.21.15.40:0/578775780 shutdown_connections 2023-10-18T09:02:37.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.327+0000 7fda0b0c2640 1 -- 172.21.15.40:0/578775780 >> 172.21.15.40:0/578775780 conn(0x7fda0406cee0 msgr2=0x7fda04069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:37.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.327+0000 7fda0b0c2640 1 -- 172.21.15.40:0/578775780 shutdown_connections 2023-10-18T09:02:37.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.328+0000 7fda0b0c2640 1 -- 172.21.15.40:0/578775780 wait complete. 2023-10-18T09:02:37.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.329+0000 7fda0b0c2640 1 Processor -- start 2023-10-18T09:02:37.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.330+0000 7fda0b0c2640 1 -- start start 2023-10-18T09:02:37.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.330+0000 7fda0b0c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9f00039a0 0x7fda04150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:37.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.331+0000 7fda0b0c2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda040c8c60 0x7fda04150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:37.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.331+0000 7fda0b0c2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda04155300 0x7fda04155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:37.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.331+0000 7fda0b0c2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda04152150 con 0x7fda040c8c60 2023-10-18T09:02:37.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.331+0000 7fda0b0c2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda04144690 con 0x7fd9f00039a0 2023-10-18T09:02:37.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.331+0000 7fda0b0c2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda04155cd0 con 0x7fda04155300 2023-10-18T09:02:37.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.331+0000 7fda03fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda040c8c60 0x7fda04150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:37.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.331+0000 7fda03fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda040c8c60 0x7fda04150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48176/0 (socket says 172.21.15.40:48176) 2023-10-18T09:02:37.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.331+0000 7fda03fff640 1 -- 172.21.15.40:0/412208931 learned_addr learned my addr 172.21.15.40:0/412208931 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:37.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.331+0000 7fda03fff640 1 -- 172.21.15.40:0/412208931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda04155300 msgr2=0x7fda04155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:37.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.331+0000 7fda03fff640 1 --2- 172.21.15.40:0/412208931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda04155300 0x7fda04155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:37.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.331+0000 7fda03fff640 1 -- 172.21.15.40:0/412208931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9f00039a0 msgr2=0x7fda04150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:37.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.331+0000 7fda03fff640 1 --2- 172.21.15.40:0/412208931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9f00039a0 0x7fda04150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:37.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.331+0000 7fda03fff640 1 -- 172.21.15.40:0/412208931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda04152550 con 0x7fda040c8c60 2023-10-18T09:02:37.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.332+0000 7fda03fff640 1 --2- 172.21.15.40:0/412208931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda040c8c60 0x7fda04150ea0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec005fd0 tx=0x7fd9ec00db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:37.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.332+0000 7fda017fa640 1 -- 172.21.15.40:0/412208931 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec004220 con 0x7fda040c8c60 2023-10-18T09:02:37.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.332+0000 7fda017fa640 1 -- 172.21.15.40:0/412208931 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd9ec0044b0 con 0x7fda040c8c60 2023-10-18T09:02:37.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.332+0000 7fda017fa640 1 -- 172.21.15.40:0/412208931 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec009a60 con 0x7fda040c8c60 2023-10-18T09:02:37.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.332+0000 7fda0b0c2640 1 -- 172.21.15.40:0/412208931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda041514d0 con 0x7fda040c8c60 2023-10-18T09:02:37.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.332+0000 7fda0b0c2640 1 -- 172.21.15.40:0/412208931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda04151a90 con 0x7fda040c8c60 2023-10-18T09:02:37.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.333+0000 7fda0b0c2640 1 -- 172.21.15.40:0/412208931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda04066fd0 con 0x7fda040c8c60 2023-10-18T09:02:37.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.334+0000 7fda017fa640 1 -- 172.21.15.40:0/412208931 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd9ec009bc0 con 0x7fda040c8c60 2023-10-18T09:02:37.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.335+0000 7fda017fa640 1 --2- 172.21.15.40:0/412208931 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd9e406ae40 0x7fd9e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:37.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.335+0000 7fda017fa640 1 -- 172.21.15.40:0/412208931 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd9ec0893d0 con 0x7fda040c8c60 2023-10-18T09:02:37.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.336+0000 7fda08e5e640 1 --2- 172.21.15.40:0/412208931 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd9e406ae40 0x7fd9e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:37.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.337+0000 7fda08e5e640 1 --2- 172.21.15.40:0/412208931 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd9e406ae40 0x7fd9e406d300 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4034000 tx=0x7fd9f4039040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:37.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.340+0000 7fda017fa640 1 -- 172.21.15.40:0/412208931 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9ec0547c0 con 0x7fda040c8c60 2023-10-18T09:02:37.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.610+0000 7fda0b0c2640 1 -- 172.21.15.40:0/412208931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fda040697c0 con 0x7fda040c8c60 2023-10-18T09:02:37.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.611+0000 7fda017fa640 1 -- 172.21.15.40:0/412208931 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd9ec057de0 con 0x7fda040c8c60 2023-10-18T09:02:37.613 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:37.613 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:02:37.614 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2023-10-18T09:02:37.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.614 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2023-10-18T09:02:37.614 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lYsb5CxAAGButZ6+Usc62kJ0ksxZrYw==", 2023-10-18T09:02:37.615 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.615 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:02:37.615 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:02:37.615 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:02:37.616 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:02:37.616 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.616 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2023-10-18T09:02:37.616 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lAqlhMBAApAhxCYB9d9bz1GibADDRSQ==", 2023-10-18T09:02:37.617 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.617 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:02:37.617 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:02:37.617 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:02:37.617 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:02:37.618 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.618 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2023-10-18T09:02:37.618 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lPVh+HhAAzucSx+ksPDBkqJGa8+pyJw==", 2023-10-18T09:02:37.619 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.619 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:02:37.619 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:02:37.619 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:02:37.619 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:02:37.620 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.620 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2023-10-18T09:02:37.620 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lP7QCBhAAog7qDCgGVv0T7WKsRkATkA==", 2023-10-18T09:02:37.621 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.621 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:02:37.621 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:02:37.621 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:02:37.621 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:02:37.622 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.622 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2023-10-18T09:02:37.622 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lVx+hAhAA6ZpIyZKhbqy8tV84piGILA==", 2023-10-18T09:02:37.622 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.623 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:02:37.623 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:02:37.623 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:02:37.623 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.624 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2023-10-18T09:02:37.624 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lk9LwExAA7zFzQOn2hcvnck2cOZZ7AA==", 2023-10-18T09:02:37.624 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.624 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:02:37.625 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:02:37.625 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:02:37.625 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.626 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2023-10-18T09:02:37.626 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lK930JRAA1GyKRkIpXPHHGqY8owYWRQ==", 2023-10-18T09:02:37.626 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.626 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:02:37.626 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:02:37.626 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:02:37.627 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.627 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.627 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2023-10-18T09:02:37.627 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9liBWyNRAAMHBBdG5nbrGQupqvUtuKMA==", 2023-10-18T09:02:37.628 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.628 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:02:37.628 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:02:37.628 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:02:37.628 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.629 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.629 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2023-10-18T09:02:37.629 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9loDoOAhAAjHZub4wnB7dCr86rxAtI5g==", 2023-10-18T09:02:37.629 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.629 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:02:37.630 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:02:37.630 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:02:37.630 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.630 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.631 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2023-10-18T09:02:37.631 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lhqNfERAAw410g3DwSXrK23YkxVBFVw==", 2023-10-18T09:02:37.631 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.631 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:02:37.631 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:02:37.632 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:02:37.632 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.632 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.632 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2023-10-18T09:02:37.633 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lRRScIRAAg0pmfxCQBCPYodj1Lko+Lw==", 2023-10-18T09:02:37.633 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.633 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:02:37.633 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:02:37.633 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:02:37.633 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.634 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2023-10-18T09:02:37.634 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9ld1dXMhAAyovlrKOhVuynEiuPTgFcjQ==", 2023-10-18T09:02:37.634 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.635 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:02:37.635 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:02:37.635 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:02:37.635 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.636 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.636 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2023-10-18T09:02:37.636 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w==", 2023-10-18T09:02:37.636 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.636 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:02:37.637 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2023-10-18T09:02:37.637 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T09:02:37.637 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.638 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2023-10-18T09:02:37.638 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lAB3oKhAAD43WFdC23WGF6WUi4JKsvg==", 2023-10-18T09:02:37.638 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.638 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:02:37.638 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:02:37.638 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:02:37.639 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:02:37.639 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.639 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2023-10-18T09:02:37.640 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lu57fHxAAdzsVU2KKsWRkaARRS8TGjw==", 2023-10-18T09:02:37.640 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.640 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:02:37.640 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:02:37.640 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:02:37.641 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:02:37.641 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.641 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2023-10-18T09:02:37.642 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lygWsNRAA1OziXxk7hBtTw2Aq8OlSKQ==", 2023-10-18T09:02:37.642 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.642 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:02:37.642 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:02:37.642 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:02:37.643 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:02:37.643 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.643 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.643 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2023-10-18T09:02:37.643 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBMnS9lnrUjNhAAHP2eezRHF4/Jgox7iPwy7g==", 2023-10-18T09:02:37.644 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.644 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:02:37.644 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:02:37.644 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:02:37.644 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:02:37.645 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.645 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2023-10-18T09:02:37.645 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9lCf/xOhAAJHVoKIswWUcVGKQpiaJ12A==", 2023-10-18T09:02:37.646 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.646 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:02:37.646 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T09:02:37.646 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:02:37.646 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:02:37.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:37.647 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2023-10-18T09:02:37.647 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9loxRXLhAAQK1n1/4PB35umHxOKQzz7Q==", 2023-10-18T09:02:37.647 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:02:37.648 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:02:37.648 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T09:02:37.648 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:02:37.648 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.648 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:37.649 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:02:37.649 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:02:37.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.614+0000 7fda0b0c2640 1 -- 172.21.15.40:0/412208931 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd9e406ae40 msgr2=0x7fd9e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:37.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.614+0000 7fda0b0c2640 1 --2- 172.21.15.40:0/412208931 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd9e406ae40 0x7fd9e406d300 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4034000 tx=0x7fd9f4039040 comp rx=0 tx=0).stop 2023-10-18T09:02:37.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.614+0000 7fda0b0c2640 1 -- 172.21.15.40:0/412208931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda040c8c60 msgr2=0x7fda04150ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:37.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.614+0000 7fda0b0c2640 1 --2- 172.21.15.40:0/412208931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda040c8c60 0x7fda04150ea0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec005fd0 tx=0x7fd9ec00db60 comp rx=0 tx=0).stop 2023-10-18T09:02:37.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.614+0000 7fda0b0c2640 1 -- 172.21.15.40:0/412208931 shutdown_connections 2023-10-18T09:02:37.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.614+0000 7fda0b0c2640 1 --2- 172.21.15.40:0/412208931 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd9e406ae40 0x7fd9e406d300 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.614+0000 7fda0b0c2640 1 --2- 172.21.15.40:0/412208931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda04155300 0x7fda04155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.614+0000 7fda0b0c2640 1 --2- 172.21.15.40:0/412208931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda040c8c60 0x7fda04150ea0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.614+0000 7fda0b0c2640 1 --2- 172.21.15.40:0/412208931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9f00039a0 0x7fda04150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.614+0000 7fda0b0c2640 1 -- 172.21.15.40:0/412208931 >> 172.21.15.40:0/412208931 conn(0x7fda0406cee0 msgr2=0x7fda04157e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:37.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.614+0000 7fda0b0c2640 1 -- 172.21.15.40:0/412208931 shutdown_connections 2023-10-18T09:02:37.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.614+0000 7fda0b0c2640 1 -- 172.21.15.40:0/412208931 wait complete. 2023-10-18T09:02:37.653 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:02:37.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.883+0000 7f8e05ac9640 1 -- 172.21.15.40:0/3273915736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e001586a0 msgr2=0x7f8e000b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:37.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.883+0000 7f8e05ac9640 1 --2- 172.21.15.40:0/3273915736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e001586a0 0x7f8e000b0bc0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8e000acc20 tx=0x7f8df402be40 comp rx=0 tx=0).stop 2023-10-18T09:02:37.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.884+0000 7f8e05ac9640 1 -- 172.21.15.40:0/3273915736 shutdown_connections 2023-10-18T09:02:37.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.884+0000 7f8e05ac9640 1 --2- 172.21.15.40:0/3273915736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e001586a0 0x7f8e000b0bc0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:37.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.884+0000 7f8e05ac9640 1 --2- 172.21.15.40:0/3273915736 >> v2:172.21.15.144:3300/0 conn(0x7f8e0006d710 0x7f8e00151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:37.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.884+0000 7f8e05ac9640 1 -- 172.21.15.40:0/3273915736 >> 172.21.15.40:0/3273915736 conn(0x7f8e0006cee0 msgr2=0x7f8e00069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:37.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.884+0000 7f8e05ac9640 1 -- 172.21.15.40:0/3273915736 shutdown_connections 2023-10-18T09:02:37.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.884+0000 7f8e05ac9640 1 -- 172.21.15.40:0/3273915736 wait complete. 2023-10-18T09:02:37.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.886+0000 7f8e05ac9640 1 Processor -- start 2023-10-18T09:02:37.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.886+0000 7f8e05ac9640 1 -- start start 2023-10-18T09:02:37.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.887+0000 7f8e05ac9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e0006d710 0x7f8e0014de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:37.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.887+0000 7f8e05ac9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e000b1100 0x7f8e00144e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:37.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.887+0000 7f8e05ac9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e001586a0 0x7f8e001499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:37.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.887+0000 7f8e05ac9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e00148a00 con 0x7f8e000b1100 2023-10-18T09:02:37.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.887+0000 7f8e05ac9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e000cbd30 con 0x7f8e0006d710 2023-10-18T09:02:37.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.887+0000 7f8e05ac9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e00149540 con 0x7f8e001586a0 2023-10-18T09:02:37.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.887+0000 7f8dff7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e0006d710 0x7f8e0014de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:37.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.888+0000 7f8dfeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e000b1100 0x7f8e00144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:37.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.888+0000 7f8dff7fe640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e0006d710 0x7f8e0014de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.40:36978/0 (socket says 172.21.15.40:36978) 2023-10-18T09:02:37.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.888+0000 7f8dff7fe640 1 -- 172.21.15.40:0/3258265388 learned_addr learned my addr 172.21.15.40:0/3258265388 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:37.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.888+0000 7f8dfffff640 1 --2- 172.21.15.40:0/3258265388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e001586a0 0x7f8e001499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:37.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.888+0000 7f8dfeffd640 1 -- 172.21.15.40:0/3258265388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e001586a0 msgr2=0x7f8e001499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:37.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.888+0000 7f8dfeffd640 1 --2- 172.21.15.40:0/3258265388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e001586a0 0x7f8e001499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:37.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.888+0000 7f8dfeffd640 1 -- 172.21.15.40:0/3258265388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e0006d710 msgr2=0x7f8e0014de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:37.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.888+0000 7f8dfeffd640 1 --2- 172.21.15.40:0/3258265388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e0006d710 0x7f8e0014de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:37.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.888+0000 7f8dfeffd640 1 -- 172.21.15.40:0/3258265388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df4009560 con 0x7f8e000b1100 2023-10-18T09:02:37.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.889+0000 7f8dfeffd640 1 --2- 172.21.15.40:0/3258265388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e000b1100 0x7f8e00144e30 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8df4033000 tx=0x7f8df402db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:37.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.889+0000 7f8de7fff640 1 -- 172.21.15.40:0/3258265388 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4003800 con 0x7f8e000b1100 2023-10-18T09:02:37.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.889+0000 7f8de7fff640 1 -- 172.21.15.40:0/3258265388 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8df4003a50 con 0x7f8e000b1100 2023-10-18T09:02:37.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.889+0000 7f8de7fff640 1 -- 172.21.15.40:0/3258265388 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df4003e20 con 0x7f8e000b1100 2023-10-18T09:02:37.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.889+0000 7f8e05ac9640 1 -- 172.21.15.40:0/3258265388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e00158c30 con 0x7f8e000b1100 2023-10-18T09:02:37.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.889+0000 7f8e05ac9640 1 -- 172.21.15.40:0/3258265388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e00148ef0 con 0x7f8e000b1100 2023-10-18T09:02:37.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.890+0000 7f8e05ac9640 1 -- 172.21.15.40:0/3258265388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e00066fd0 con 0x7f8e000b1100 2023-10-18T09:02:37.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.891+0000 7f8de7fff640 1 -- 172.21.15.40:0/3258265388 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8df403e450 con 0x7f8e000b1100 2023-10-18T09:02:37.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.892+0000 7f8de7fff640 1 --2- 172.21.15.40:0/3258265388 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8dec06ae40 0x7f8dec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:37.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.892+0000 7f8de7fff640 1 -- 172.21.15.40:0/3258265388 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8df40abc80 con 0x7f8e000b1100 2023-10-18T09:02:37.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.893+0000 7f8dff7fe640 1 --2- 172.21.15.40:0/3258265388 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8dec06ae40 0x7f8dec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:37.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.894+0000 7f8dff7fe640 1 --2- 172.21.15.40:0/3258265388 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8dec06ae40 0x7f8dec06d300 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8e0014e770 tx=0x7f8df0009500 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:37.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:37.897+0000 7f8de7fff640 1 -- 172.21.15.40:0/3258265388 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8df40771b0 con 0x7f8e000b1100 2023-10-18T09:02:38.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.092+0000 7f8e05ac9640 1 -- 172.21.15.40:0/3258265388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e00149680 con 0x7f8e000b1100 2023-10-18T09:02:38.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.095+0000 7f8de7fff640 1 -- 172.21.15.40:0/3258265388 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+12880 (secure 0 0 0) 0x7f8df4042090 con 0x7f8e000b1100 2023-10-18T09:02:38.097 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:38.098 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":126,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:02:37.067537+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:02:38.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.098+0000 7f8e05ac9640 1 -- 172.21.15.40:0/3258265388 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8dec06ae40 msgr2=0x7f8dec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:38.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.098+0000 7f8e05ac9640 1 --2- 172.21.15.40:0/3258265388 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8dec06ae40 0x7f8dec06d300 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8e0014e770 tx=0x7f8df0009500 comp rx=0 tx=0).stop 2023-10-18T09:02:38.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.098+0000 7f8e05ac9640 1 -- 172.21.15.40:0/3258265388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e000b1100 msgr2=0x7f8e00144e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:38.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.098+0000 7f8e05ac9640 1 --2- 172.21.15.40:0/3258265388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e000b1100 0x7f8e00144e30 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8df4033000 tx=0x7f8df402db10 comp rx=0 tx=0).stop 2023-10-18T09:02:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.098+0000 7f8e05ac9640 1 -- 172.21.15.40:0/3258265388 shutdown_connections 2023-10-18T09:02:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.098+0000 7f8e05ac9640 1 --2- 172.21.15.40:0/3258265388 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8dec06ae40 0x7f8dec06d300 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.098+0000 7f8e05ac9640 1 --2- 172.21.15.40:0/3258265388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e001586a0 0x7f8e001499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.098+0000 7f8e05ac9640 1 --2- 172.21.15.40:0/3258265388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8e000b1100 0x7f8e00144e30 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.098+0000 7f8e05ac9640 1 --2- 172.21.15.40:0/3258265388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e0006d710 0x7f8e0014de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.098+0000 7f8e05ac9640 1 -- 172.21.15.40:0/3258265388 >> 172.21.15.40:0/3258265388 conn(0x7f8e0006cee0 msgr2=0x7f8e0006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.098+0000 7f8e05ac9640 1 -- 172.21.15.40:0/3258265388 shutdown_connections 2023-10-18T09:02:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.099+0000 7f8e05ac9640 1 -- 172.21.15.40:0/3258265388 wait complete. 2023-10-18T09:02:38.119 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:02:38.120 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:02:38.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.342+0000 7fe5eafe2640 1 -- 172.21.15.40:0/1628186533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5e406d830 msgr2=0x7fe5e406dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:38.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.342+0000 7fe5eafe2640 1 --2- 172.21.15.40:0/1628186533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5e406d830 0x7fe5e406dc10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4005fd0 tx=0x7fe5d402c240 comp rx=0 tx=0).stop 2023-10-18T09:02:38.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.343+0000 7fe5eafe2640 1 -- 172.21.15.40:0/1628186533 shutdown_connections 2023-10-18T09:02:38.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.343+0000 7fe5eafe2640 1 --2- 172.21.15.40:0/1628186533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5e40b12d0 0x7fe5e40b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.343+0000 7fe5eafe2640 1 --2- 172.21.15.40:0/1628186533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5e406d830 0x7fe5e406dc10 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.343+0000 7fe5eafe2640 1 -- 172.21.15.40:0/1628186533 >> 172.21.15.40:0/1628186533 conn(0x7fe5e40af3c0 msgr2=0x7fe5e40af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:38.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.343+0000 7fe5eafe2640 1 -- 172.21.15.40:0/1628186533 shutdown_connections 2023-10-18T09:02:38.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.343+0000 7fe5eafe2640 1 -- 172.21.15.40:0/1628186533 wait complete. 2023-10-18T09:02:38.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.345+0000 7fe5eafe2640 1 Processor -- start 2023-10-18T09:02:38.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.345+0000 7fe5eafe2640 1 -- start start 2023-10-18T09:02:38.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.346+0000 7fe5eafe2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5e406d830 0x7fe5e4144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:38.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.346+0000 7fe5eafe2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5e40b12d0 0x7fe5e41e6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:38.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.346+0000 7fe5eafe2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5e40cb2d0 0x7fe5e4147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:38.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.346+0000 7fe5eafe2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e41591e0 con 0x7fe5e40cb2d0 2023-10-18T09:02:38.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.346+0000 7fe5eafe2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e41585d0 con 0x7fe5e406d830 2023-10-18T09:02:38.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.346+0000 7fe5eafe2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e41588b0 con 0x7fe5e40b12d0 2023-10-18T09:02:38.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.346+0000 7fe5e957f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5e40cb2d0 0x7fe5e4147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:38.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.346+0000 7fe5e957f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5e40cb2d0 0x7fe5e4147f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48210/0 (socket says 172.21.15.40:48210) 2023-10-18T09:02:38.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.346+0000 7fe5e957f640 1 -- 172.21.15.40:0/606888739 learned_addr learned my addr 172.21.15.40:0/606888739 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:38.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.346+0000 7fe5e8d7e640 1 --2- 172.21.15.40:0/606888739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5e406d830 0x7fe5e4144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:38.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.346+0000 7fe5e3fff640 1 --2- 172.21.15.40:0/606888739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5e40b12d0 0x7fe5e41e6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:38.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.347+0000 7fe5e957f640 1 -- 172.21.15.40:0/606888739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5e40b12d0 msgr2=0x7fe5e41e6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:38.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.347+0000 7fe5e957f640 1 --2- 172.21.15.40:0/606888739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5e40b12d0 0x7fe5e41e6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.347+0000 7fe5e957f640 1 -- 172.21.15.40:0/606888739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5e406d830 msgr2=0x7fe5e4144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:38.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.347+0000 7fe5e957f640 1 --2- 172.21.15.40:0/606888739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5e406d830 0x7fe5e4144250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.347+0000 7fe5e957f640 1 -- 172.21.15.40:0/606888739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d4009900 con 0x7fe5e40cb2d0 2023-10-18T09:02:38.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.347+0000 7fe5e3fff640 1 --2- 172.21.15.40:0/606888739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5e40b12d0 0x7fe5e41e6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:38.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.347+0000 7fe5e957f640 1 --2- 172.21.15.40:0/606888739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5e40cb2d0 0x7fe5e4147f80 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00c970 tx=0x7fe5dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:38.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.347+0000 7fe5e17fa640 1 -- 172.21.15.40:0/606888739 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc007bb0 con 0x7fe5e40cb2d0 2023-10-18T09:02:38.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.347+0000 7fe5e17fa640 1 -- 172.21.15.40:0/606888739 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe5dc007e00 con 0x7fe5e40cb2d0 2023-10-18T09:02:38.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.347+0000 7fe5eafe2640 1 -- 172.21.15.40:0/606888739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e40b0b80 con 0x7fe5e40cb2d0 2023-10-18T09:02:38.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.347+0000 7fe5e17fa640 1 -- 172.21.15.40:0/606888739 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc005280 con 0x7fe5e40cb2d0 2023-10-18T09:02:38.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.347+0000 7fe5eafe2640 1 -- 172.21.15.40:0/606888739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e4158e70 con 0x7fe5e40cb2d0 2023-10-18T09:02:38.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.349+0000 7fe5eafe2640 1 -- 172.21.15.40:0/606888739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e4066a90 con 0x7fe5e40cb2d0 2023-10-18T09:02:38.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.350+0000 7fe5e17fa640 1 -- 172.21.15.40:0/606888739 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe5dc0053e0 con 0x7fe5e40cb2d0 2023-10-18T09:02:38.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.351+0000 7fe5e17fa640 1 --2- 172.21.15.40:0/606888739 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe5d006ae40 0x7fe5d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:38.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.351+0000 7fe5e17fa640 1 -- 172.21.15.40:0/606888739 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe5dc089860 con 0x7fe5e40cb2d0 2023-10-18T09:02:38.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.351+0000 7fe5e8d7e640 1 --2- 172.21.15.40:0/606888739 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe5d006ae40 0x7fe5d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:38.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.352+0000 7fe5e8d7e640 1 --2- 172.21.15.40:0/606888739 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe5d006ae40 0x7fe5d006d300 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4036000 tx=0x7fe5d4004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:38.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.355+0000 7fe5e17fa640 1 -- 172.21.15.40:0/606888739 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5dc054bc0 con 0x7fe5e40cb2d0 2023-10-18T09:02:38.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.549+0000 7fe5eafe2640 1 -- 172.21.15.40:0/606888739 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5e415b700 con 0x7fe5e40cb2d0 2023-10-18T09:02:38.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.550+0000 7fe5e17fa640 1 -- 172.21.15.40:0/606888739 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5dc01d020 con 0x7fe5e40cb2d0 2023-10-18T09:02:38.552 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:02:38.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.552+0000 7fe5eafe2640 1 -- 172.21.15.40:0/606888739 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe5d006ae40 msgr2=0x7fe5d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:38.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.552+0000 7fe5eafe2640 1 --2- 172.21.15.40:0/606888739 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe5d006ae40 0x7fe5d006d300 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4036000 tx=0x7fe5d4004ab0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.552+0000 7fe5eafe2640 1 -- 172.21.15.40:0/606888739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5e40cb2d0 msgr2=0x7fe5e4147f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:38.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.552+0000 7fe5eafe2640 1 --2- 172.21.15.40:0/606888739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5e40cb2d0 0x7fe5e4147f80 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00c970 tx=0x7fe5dc00ce30 comp rx=0 tx=0).stop 2023-10-18T09:02:38.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.552+0000 7fe5eafe2640 1 -- 172.21.15.40:0/606888739 shutdown_connections 2023-10-18T09:02:38.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.552+0000 7fe5eafe2640 1 --2- 172.21.15.40:0/606888739 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe5d006ae40 0x7fe5d006d300 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.552+0000 7fe5eafe2640 1 --2- 172.21.15.40:0/606888739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe5e40cb2d0 0x7fe5e4147f80 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.552+0000 7fe5eafe2640 1 --2- 172.21.15.40:0/606888739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5e40b12d0 0x7fe5e41e6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.552+0000 7fe5eafe2640 1 --2- 172.21.15.40:0/606888739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5e406d830 0x7fe5e4144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.552+0000 7fe5eafe2640 1 -- 172.21.15.40:0/606888739 >> 172.21.15.40:0/606888739 conn(0x7fe5e40af3c0 msgr2=0x7fe5e40cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:38.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.553+0000 7fe5eafe2640 1 -- 172.21.15.40:0/606888739 shutdown_connections 2023-10-18T09:02:38.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.553+0000 7fe5eafe2640 1 -- 172.21.15.40:0/606888739 wait complete. 2023-10-18T09:02:38.572 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T09:02:38.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.802+0000 7fd845bbf640 1 -- 172.21.15.40:0/750294890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd840158100 msgr2=0x7fd8400b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:38.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.802+0000 7fd845bbf640 1 --2- 172.21.15.40:0/750294890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd840158100 0x7fd8400b02a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd834009830 tx=0x7fd83402bcd0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.802+0000 7fd845bbf640 1 -- 172.21.15.40:0/750294890 shutdown_connections 2023-10-18T09:02:38.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.802+0000 7fd845bbf640 1 --2- 172.21.15.40:0/750294890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8401584e0 0x7fd84015bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.802+0000 7fd845bbf640 1 --2- 172.21.15.40:0/750294890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd840158100 0x7fd8400b02a0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.802+0000 7fd845bbf640 1 -- 172.21.15.40:0/750294890 >> 172.21.15.40:0/750294890 conn(0x7fd8400b29e0 msgr2=0x7fd8400aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:38.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.803+0000 7fd845bbf640 1 -- 172.21.15.40:0/750294890 shutdown_connections 2023-10-18T09:02:38.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.803+0000 7fd845bbf640 1 -- 172.21.15.40:0/750294890 wait complete. 2023-10-18T09:02:38.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.805+0000 7fd845bbf640 1 Processor -- start 2023-10-18T09:02:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.805+0000 7fd845bbf640 1 -- start start 2023-10-18T09:02:38.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.806+0000 7fd845bbf640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd840158100 0x7fd8401f1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:38.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.806+0000 7fd845bbf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8401584e0 0x7fd840070da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:38.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.806+0000 7fd845bbf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd84015c9c0 0x7fd8400712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:38.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.806+0000 7fd845bbf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8401f1cb0 con 0x7fd8401584e0 2023-10-18T09:02:38.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.806+0000 7fd845bbf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8401e28f0 con 0x7fd84015c9c0 2023-10-18T09:02:38.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.806+0000 7fd845bbf640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8401e2bd0 con 0x7fd840158100 2023-10-18T09:02:38.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.806+0000 7fd83ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8401584e0 0x7fd840070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:38.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.807+0000 7fd83ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8401584e0 0x7fd840070da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48222/0 (socket says 172.21.15.40:48222) 2023-10-18T09:02:38.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.807+0000 7fd83ffff640 1 -- 172.21.15.40:0/3411334413 learned_addr learned my addr 172.21.15.40:0/3411334413 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:38.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.807+0000 7fd844bbd640 1 --2- 172.21.15.40:0/3411334413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd840158100 0x7fd8401f1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:38.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.807+0000 7fd83ffff640 1 -- 172.21.15.40:0/3411334413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd840158100 msgr2=0x7fd8401f1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:38.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.807+0000 7fd83ffff640 1 --2- 172.21.15.40:0/3411334413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd840158100 0x7fd8401f1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.807+0000 7fd8453be640 1 --2- 172.21.15.40:0/3411334413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd84015c9c0 0x7fd8400712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:38.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.807+0000 7fd83ffff640 1 -- 172.21.15.40:0/3411334413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd84015c9c0 msgr2=0x7fd8400712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:38.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.807+0000 7fd83ffff640 1 --2- 172.21.15.40:0/3411334413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd84015c9c0 0x7fd8400712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:38.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.807+0000 7fd83ffff640 1 -- 172.21.15.40:0/3411334413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd834009490 con 0x7fd8401584e0 2023-10-18T09:02:38.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.807+0000 7fd8453be640 1 --2- 172.21.15.40:0/3411334413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd84015c9c0 0x7fd8400712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:38.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.807+0000 7fd83ffff640 1 --2- 172.21.15.40:0/3411334413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8401584e0 0x7fd840070da0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00c970 tx=0x7fd82c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:38.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.808+0000 7fd83d7fa640 1 -- 172.21.15.40:0/3411334413 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c004440 con 0x7fd8401584e0 2023-10-18T09:02:38.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.808+0000 7fd83d7fa640 1 -- 172.21.15.40:0/3411334413 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd82c007950 con 0x7fd8401584e0 2023-10-18T09:02:38.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.808+0000 7fd83d7fa640 1 -- 172.21.15.40:0/3411334413 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c007d60 con 0x7fd8401584e0 2023-10-18T09:02:38.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.808+0000 7fd845bbf640 1 -- 172.21.15.40:0/3411334413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8401e2eb0 con 0x7fd8401584e0 2023-10-18T09:02:38.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.808+0000 7fd845bbf640 1 -- 172.21.15.40:0/3411334413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8401e3470 con 0x7fd8401584e0 2023-10-18T09:02:38.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.809+0000 7fd845bbf640 1 -- 172.21.15.40:0/3411334413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd840066a90 con 0x7fd8401584e0 2023-10-18T09:02:38.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.810+0000 7fd83d7fa640 1 -- 172.21.15.40:0/3411334413 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd82c0035f0 con 0x7fd8401584e0 2023-10-18T09:02:38.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.811+0000 7fd83d7fa640 1 --2- 172.21.15.40:0/3411334413 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd82406ae40 0x7fd82406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:38.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.811+0000 7fd83d7fa640 1 -- 172.21.15.40:0/3411334413 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd82c0899d0 con 0x7fd8401584e0 2023-10-18T09:02:38.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.812+0000 7fd844bbd640 1 --2- 172.21.15.40:0/3411334413 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd82406ae40 0x7fd82406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:38.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.813+0000 7fd844bbd640 1 --2- 172.21.15.40:0/3411334413 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd82406ae40 0x7fd82406d300 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd834036000 tx=0x7fd834004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:38.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:38.815+0000 7fd83d7fa640 1 -- 172.21.15.40:0/3411334413 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd82c054d30 con 0x7fd8401584e0 2023-10-18T09:02:39.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.015+0000 7fd845bbf640 1 -- 172.21.15.40:0/3411334413 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd840066e00 con 0x7fd8401584e0 2023-10-18T09:02:39.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.016+0000 7fd83d7fa640 1 -- 172.21.15.40:0/3411334413 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v126) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd82c01e090 con 0x7fd8401584e0 2023-10-18T09:02:39.019 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:39.019 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-10-18T09:02:39.019 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:02:39.020 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-10-18T09:02:39.020 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-10-18T09:02:39.020 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:02:39.021 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-10-18T09:02:39.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.019+0000 7fd845bbf640 1 -- 172.21.15.40:0/3411334413 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd82406ae40 msgr2=0x7fd82406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:39.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.019+0000 7fd845bbf640 1 --2- 172.21.15.40:0/3411334413 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd82406ae40 0x7fd82406d300 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd834036000 tx=0x7fd834004730 comp rx=0 tx=0).stop 2023-10-18T09:02:39.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.019+0000 7fd845bbf640 1 -- 172.21.15.40:0/3411334413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8401584e0 msgr2=0x7fd840070da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:39.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.019+0000 7fd845bbf640 1 --2- 172.21.15.40:0/3411334413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8401584e0 0x7fd840070da0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00c970 tx=0x7fd82c00ce30 comp rx=0 tx=0).stop 2023-10-18T09:02:39.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.019+0000 7fd845bbf640 1 -- 172.21.15.40:0/3411334413 shutdown_connections 2023-10-18T09:02:39.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.019+0000 7fd845bbf640 1 --2- 172.21.15.40:0/3411334413 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd82406ae40 0x7fd82406d300 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.019+0000 7fd845bbf640 1 --2- 172.21.15.40:0/3411334413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd84015c9c0 0x7fd8400712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.019+0000 7fd845bbf640 1 --2- 172.21.15.40:0/3411334413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8401584e0 0x7fd840070da0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.019+0000 7fd845bbf640 1 --2- 172.21.15.40:0/3411334413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd840158100 0x7fd8401f1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.019+0000 7fd845bbf640 1 -- 172.21.15.40:0/3411334413 >> 172.21.15.40:0/3411334413 conn(0x7fd8400b29e0 msgr2=0x7fd840156d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:39.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.020+0000 7fd845bbf640 1 -- 172.21.15.40:0/3411334413 shutdown_connections 2023-10-18T09:02:39.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.020+0000 7fd845bbf640 1 -- 172.21.15.40:0/3411334413 wait complete. 2023-10-18T09:02:39.041 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T09:02:39.041 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T09:02:39.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.274+0000 7eff0ada0640 1 -- 172.21.15.40:0/305232322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff04157320 msgr2=0x7eff040b2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:39.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.274+0000 7eff0ada0640 1 --2- 172.21.15.40:0/305232322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff04157320 0x7eff040b2a20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efefc00acf0 tx=0x7efefc02d400 comp rx=0 tx=0).stop 2023-10-18T09:02:39.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.274+0000 7eff0ada0640 1 -- 172.21.15.40:0/305232322 shutdown_connections 2023-10-18T09:02:39.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.274+0000 7eff0ada0640 1 --2- 172.21.15.40:0/305232322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff04157320 0x7eff040b2a20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:39.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.274+0000 7eff0ada0640 1 -- 172.21.15.40:0/305232322 >> 172.21.15.40:0/305232322 conn(0x7eff0406b7d0 msgr2=0x7eff0406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:39.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.274+0000 7eff0ada0640 1 -- 172.21.15.40:0/305232322 shutdown_connections 2023-10-18T09:02:39.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.274+0000 7eff0ada0640 1 -- 172.21.15.40:0/305232322 wait complete. 2023-10-18T09:02:39.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.277+0000 7eff0ada0640 1 Processor -- start 2023-10-18T09:02:39.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.277+0000 7eff0ada0640 1 -- start start 2023-10-18T09:02:39.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.278+0000 7eff0ada0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff0406d570 0x7eff041e2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:39.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.278+0000 7eff0ada0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff0406fb90 0x7eff0415a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:39.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.278+0000 7eff0ada0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff04157320 0x7eff04145700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:39.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.278+0000 7eff0ada0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff041e4550 con 0x7eff04157320 2023-10-18T09:02:39.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.278+0000 7eff0ada0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff041e34b0 con 0x7eff0406d570 2023-10-18T09:02:39.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.278+0000 7eff0ada0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff041e3790 con 0x7eff0406fb90 2023-10-18T09:02:39.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.279+0000 7eff0a59f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff04157320 0x7eff04145700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:39.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.279+0000 7eff0a59f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff04157320 0x7eff04145700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48228/0 (socket says 172.21.15.40:48228) 2023-10-18T09:02:39.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.279+0000 7eff0a59f640 1 -- 172.21.15.40:0/544479041 learned_addr learned my addr 172.21.15.40:0/544479041 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:39.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.279+0000 7eff09d9e640 1 --2- 172.21.15.40:0/544479041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff0406d570 0x7eff041e2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:39.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.279+0000 7eff0959d640 1 --2- 172.21.15.40:0/544479041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff0406fb90 0x7eff0415a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:39.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.279+0000 7eff0a59f640 1 -- 172.21.15.40:0/544479041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff0406fb90 msgr2=0x7eff0415a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:39.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.279+0000 7eff0a59f640 1 --2- 172.21.15.40:0/544479041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff0406fb90 0x7eff0415a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:39.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.279+0000 7eff0a59f640 1 -- 172.21.15.40:0/544479041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff0406d570 msgr2=0x7eff041e2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:39.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.279+0000 7eff0a59f640 1 --2- 172.21.15.40:0/544479041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff0406d570 0x7eff041e2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:39.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.279+0000 7eff0a59f640 1 -- 172.21.15.40:0/544479041 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc00a950 con 0x7eff04157320 2023-10-18T09:02:39.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.279+0000 7eff09d9e640 1 --2- 172.21.15.40:0/544479041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff0406d570 0x7eff041e2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:02:39.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.280+0000 7eff0a59f640 1 --2- 172.21.15.40:0/544479041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff04157320 0x7eff04145700 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7efefc006010 tx=0x7efefc032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:39.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.280+0000 7efefa7fc640 1 -- 172.21.15.40:0/544479041 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc004240 con 0x7eff04157320 2023-10-18T09:02:39.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.280+0000 7efefa7fc640 1 -- 172.21.15.40:0/544479041 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efefc004490 con 0x7eff04157320 2023-10-18T09:02:39.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.280+0000 7efefa7fc640 1 -- 172.21.15.40:0/544479041 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc004860 con 0x7eff04157320 2023-10-18T09:02:39.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.280+0000 7eff0ada0640 1 -- 172.21.15.40:0/544479041 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff041e3a70 con 0x7eff04157320 2023-10-18T09:02:39.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.280+0000 7eff0ada0640 1 -- 172.21.15.40:0/544479041 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff041e4030 con 0x7eff04157320 2023-10-18T09:02:39.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.281+0000 7eff0ada0640 1 -- 172.21.15.40:0/544479041 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff04066ea0 con 0x7eff04157320 2023-10-18T09:02:39.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.282+0000 7efefa7fc640 1 -- 172.21.15.40:0/544479041 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efefc034440 con 0x7eff04157320 2023-10-18T09:02:39.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.283+0000 7efefa7fc640 1 --2- 172.21.15.40:0/544479041 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efee806ae40 0x7efee806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:39.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.283+0000 7efefa7fc640 1 -- 172.21.15.40:0/544479041 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efefc0aca60 con 0x7eff04157320 2023-10-18T09:02:39.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.284+0000 7eff09d9e640 1 --2- 172.21.15.40:0/544479041 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efee806ae40 0x7efee806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:39.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.285+0000 7eff09d9e640 1 --2- 172.21.15.40:0/544479041 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efee806ae40 0x7efee806d300 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7eff000011e0 tx=0x7eff00009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:39.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.287+0000 7efefa7fc640 1 -- 172.21.15.40:0/544479041 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efefc077e50 con 0x7eff04157320 2023-10-18T09:02:39.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:39.477+0000 7eff0ada0640 1 -- 172.21.15.40:0/544479041 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7eff0415a7d0 con 0x7eff04157320 2023-10-18T09:02:40.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.291+0000 7efefa7fc640 1 -- 172.21.15.40:0/544479041 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v127) v1 ==== 138+0+0 (secure 0 0 0) 0x7efefc0420b0 con 0x7eff04157320 2023-10-18T09:02:40.294 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-10-18T09:02:40.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.295+0000 7eff0ada0640 1 -- 172.21.15.40:0/544479041 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efee806ae40 msgr2=0x7efee806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:40.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.295+0000 7eff0ada0640 1 --2- 172.21.15.40:0/544479041 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efee806ae40 0x7efee806d300 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7eff000011e0 tx=0x7eff00009040 comp rx=0 tx=0).stop 2023-10-18T09:02:40.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.295+0000 7eff0ada0640 1 -- 172.21.15.40:0/544479041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff04157320 msgr2=0x7eff04145700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:40.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.295+0000 7eff0ada0640 1 --2- 172.21.15.40:0/544479041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff04157320 0x7eff04145700 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7efefc006010 tx=0x7efefc032b90 comp rx=0 tx=0).stop 2023-10-18T09:02:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.295+0000 7eff0ada0640 1 -- 172.21.15.40:0/544479041 shutdown_connections 2023-10-18T09:02:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.295+0000 7eff0ada0640 1 --2- 172.21.15.40:0/544479041 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efee806ae40 0x7efee806d300 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:40.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.295+0000 7eff0ada0640 1 --2- 172.21.15.40:0/544479041 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff04157320 0x7eff04145700 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:40.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.295+0000 7eff0ada0640 1 --2- 172.21.15.40:0/544479041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff0406fb90 0x7eff0415a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:40.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.295+0000 7eff0ada0640 1 --2- 172.21.15.40:0/544479041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff0406d570 0x7eff041e2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:40.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.295+0000 7eff0ada0640 1 -- 172.21.15.40:0/544479041 >> 172.21.15.40:0/544479041 conn(0x7eff0406b7d0 msgr2=0x7eff04154100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:40.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.295+0000 7eff0ada0640 1 -- 172.21.15.40:0/544479041 shutdown_connections 2023-10-18T09:02:40.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.295+0000 7eff0ada0640 1 -- 172.21.15.40:0/544479041 wait complete. 2023-10-18T09:02:40.323 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T09:02:40.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.551+0000 7f5aaf59e640 1 -- 172.21.15.40:0/2984516781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab00710a0 msgr2=0x7f5ab0157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:40.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.551+0000 7f5aaf59e640 1 --2- 172.21.15.40:0/2984516781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab00710a0 0x7f5ab0157ad0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5ab00aef60 tx=0x7f5aa002bd10 comp rx=0 tx=0).stop 2023-10-18T09:02:40.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.551+0000 7f5aaf59e640 1 -- 172.21.15.40:0/2984516781 shutdown_connections 2023-10-18T09:02:40.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.551+0000 7f5aaf59e640 1 --2- 172.21.15.40:0/2984516781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab00710a0 0x7f5ab0157ad0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:40.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.551+0000 7f5aaf59e640 1 --2- 172.21.15.40:0/2984516781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab015af70 0x7f5ab0070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:40.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.551+0000 7f5aaf59e640 1 --2- 172.21.15.40:0/2984516781 >> v2:172.21.15.144:3300/0 conn(0x7f5ab006d5d0 0x7f5ab00c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:40.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.551+0000 7f5aaf59e640 1 -- 172.21.15.40:0/2984516781 >> 172.21.15.40:0/2984516781 conn(0x7f5ab006ca10 msgr2=0x7f5ab006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:40.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.551+0000 7f5aaf59e640 1 -- 172.21.15.40:0/2984516781 shutdown_connections 2023-10-18T09:02:40.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.552+0000 7f5aaf59e640 1 -- 172.21.15.40:0/2984516781 wait complete. 2023-10-18T09:02:40.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.553+0000 7f5aaf59e640 1 Processor -- start 2023-10-18T09:02:40.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.554+0000 7f5aaf59e640 1 -- start start 2023-10-18T09:02:40.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.554+0000 7f5aaf59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab006d5d0 0x7f5ab0155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:40.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.554+0000 7f5aaf59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ab00710a0 0x7f5ab01f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:40.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.555+0000 7f5aaf59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab015af70 0x7f5ab01f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:40.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.555+0000 7f5aaf59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab014f190 con 0x7f5ab015af70 2023-10-18T09:02:40.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.555+0000 7f5aaf59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab014f750 con 0x7f5ab00710a0 2023-10-18T09:02:40.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.555+0000 7f5aaf59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab01f3030 con 0x7f5ab006d5d0 2023-10-18T09:02:40.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.555+0000 7f5aaed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab015af70 0x7f5ab01f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:40.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.555+0000 7f5aaed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab015af70 0x7f5ab01f1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48234/0 (socket says 172.21.15.40:48234) 2023-10-18T09:02:40.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.555+0000 7f5aaed9d640 1 -- 172.21.15.40:0/927973851 learned_addr learned my addr 172.21.15.40:0/927973851 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:40.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.555+0000 7f5aae59c640 1 --2- 172.21.15.40:0/927973851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab006d5d0 0x7f5ab0155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:40.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.555+0000 7f5aadd9b640 1 --2- 172.21.15.40:0/927973851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ab00710a0 0x7f5ab01f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:40.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.555+0000 7f5aaed9d640 1 -- 172.21.15.40:0/927973851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab006d5d0 msgr2=0x7f5ab0155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:40.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.555+0000 7f5aaed9d640 1 --2- 172.21.15.40:0/927973851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab006d5d0 0x7f5ab0155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:40.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.555+0000 7f5aaed9d640 1 -- 172.21.15.40:0/927973851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ab00710a0 msgr2=0x7f5ab01f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:40.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.555+0000 7f5aaed9d640 1 --2- 172.21.15.40:0/927973851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ab00710a0 0x7f5ab01f2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:40.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.555+0000 7f5aaed9d640 1 -- 172.21.15.40:0/927973851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa0009560 con 0x7f5ab015af70 2023-10-18T09:02:40.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.556+0000 7f5aaed9d640 1 --2- 172.21.15.40:0/927973851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab015af70 0x7f5ab01f1950 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0036040 tx=0x7f5aa002dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.556+0000 7f5a9effd640 1 -- 172.21.15.40:0/927973851 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0003800 con 0x7f5ab015af70 2023-10-18T09:02:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.556+0000 7f5a9effd640 1 -- 172.21.15.40:0/927973851 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5aa0003a50 con 0x7f5ab015af70 2023-10-18T09:02:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.556+0000 7f5a9effd640 1 -- 172.21.15.40:0/927973851 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0003ea0 con 0x7f5ab015af70 2023-10-18T09:02:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.556+0000 7f5aaf59e640 1 -- 172.21.15.40:0/927973851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab01f2420 con 0x7f5ab015af70 2023-10-18T09:02:40.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.556+0000 7f5aaf59e640 1 -- 172.21.15.40:0/927973851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab014f3e0 con 0x7f5ab015af70 2023-10-18T09:02:40.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.557+0000 7f5aaf59e640 1 -- 172.21.15.40:0/927973851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab01f2940 con 0x7f5ab015af70 2023-10-18T09:02:40.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.559+0000 7f5a9effd640 1 -- 172.21.15.40:0/927973851 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5aa003e450 con 0x7f5ab015af70 2023-10-18T09:02:40.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.560+0000 7f5a9effd640 1 --2- 172.21.15.40:0/927973851 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a9006ae40 0x7f5a9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:40.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.560+0000 7f5a9effd640 1 -- 172.21.15.40:0/927973851 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f5aa00abfc0 con 0x7f5ab015af70 2023-10-18T09:02:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.561+0000 7f5aae59c640 1 --2- 172.21.15.40:0/927973851 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a9006ae40 0x7f5a9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:40.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.562+0000 7f5aae59c640 1 --2- 172.21.15.40:0/927973851 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a9006ae40 0x7f5a9006d300 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8005c40 tx=0x7f5aa800a780 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:40.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.565+0000 7f5a9effd640 1 -- 172.21.15.40:0/927973851 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5aa0077360 con 0x7f5ab015af70 2023-10-18T09:02:40.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:40.757+0000 7f5aaf59e640 1 -- 172.21.15.40:0/927973851 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5ab014f8c0 con 0x7f5ab015af70 2023-10-18T09:02:41.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.297+0000 7f5a9effd640 1 -- 172.21.15.40:0/927973851 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v128) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5aa007a980 con 0x7f5ab015af70 2023-10-18T09:02:41.299 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-10-18T09:02:41.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.301+0000 7f5aaf59e640 1 -- 172.21.15.40:0/927973851 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a9006ae40 msgr2=0x7f5a9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:41.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.301+0000 7f5aaf59e640 1 --2- 172.21.15.40:0/927973851 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a9006ae40 0x7f5a9006d300 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8005c40 tx=0x7f5aa800a780 comp rx=0 tx=0).stop 2023-10-18T09:02:41.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.301+0000 7f5aaf59e640 1 -- 172.21.15.40:0/927973851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab015af70 msgr2=0x7f5ab01f1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:41.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.301+0000 7f5aaf59e640 1 --2- 172.21.15.40:0/927973851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab015af70 0x7f5ab01f1950 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0036040 tx=0x7f5aa002dad0 comp rx=0 tx=0).stop 2023-10-18T09:02:41.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.302+0000 7f5aaf59e640 1 -- 172.21.15.40:0/927973851 shutdown_connections 2023-10-18T09:02:41.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.302+0000 7f5aaf59e640 1 --2- 172.21.15.40:0/927973851 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5a9006ae40 0x7f5a9006d300 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:41.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.302+0000 7f5aaf59e640 1 --2- 172.21.15.40:0/927973851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ab015af70 0x7f5ab01f1950 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:41.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.302+0000 7f5aaf59e640 1 --2- 172.21.15.40:0/927973851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ab00710a0 0x7f5ab01f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:41.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.302+0000 7f5aaf59e640 1 --2- 172.21.15.40:0/927973851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ab006d5d0 0x7f5ab0155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:41.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.302+0000 7f5aaf59e640 1 -- 172.21.15.40:0/927973851 >> 172.21.15.40:0/927973851 conn(0x7f5ab006ca10 msgr2=0x7f5ab006cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:41.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.303+0000 7f5aaf59e640 1 -- 172.21.15.40:0/927973851 shutdown_connections 2023-10-18T09:02:41.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.303+0000 7f5aaf59e640 1 -- 172.21.15.40:0/927973851 wait complete. 2023-10-18T09:02:41.333 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T09:02:41.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.537+0000 7fe6d2667640 1 -- 172.21.15.40:0/1416868226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6cc15af70 msgr2=0x7fe6cc0710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:41.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.537+0000 7fe6d2667640 1 --2- 172.21.15.40:0/1416868226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6cc15af70 0x7fe6cc0710a0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4005fd0 tx=0x7fe6c402c1b0 comp rx=0 tx=0).stop 2023-10-18T09:02:41.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.537+0000 7fe6d2667640 1 -- 172.21.15.40:0/1416868226 shutdown_connections 2023-10-18T09:02:41.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.537+0000 7fe6d2667640 1 --2- 172.21.15.40:0/1416868226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6cc15af70 0x7fe6cc0710a0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:41.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.537+0000 7fe6d2667640 1 --2- 172.21.15.40:0/1416868226 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6cc06c2d0 0x7fe6cc070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:41.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.537+0000 7fe6d2667640 1 -- 172.21.15.40:0/1416868226 >> 172.21.15.40:0/1416868226 conn(0x7fe6cc06ca10 msgr2=0x7fe6cc06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:41.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.537+0000 7fe6d2667640 1 -- 172.21.15.40:0/1416868226 shutdown_connections 2023-10-18T09:02:41.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.537+0000 7fe6d2667640 1 -- 172.21.15.40:0/1416868226 wait complete. 2023-10-18T09:02:41.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.539+0000 7fe6d2667640 1 Processor -- start 2023-10-18T09:02:41.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.539+0000 7fe6d2667640 1 -- start start 2023-10-18T09:02:41.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.541+0000 7fe6d2667640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6cc06c2d0 0x7fe6cc161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:41.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.541+0000 7fe6d2667640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6cc0c9620 0x7fe6cc161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:41.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.541+0000 7fe6d2667640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6cc15af70 0x7fe6cc153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:41.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.541+0000 7fe6d2667640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc160bd0 con 0x7fe6cc15af70 2023-10-18T09:02:41.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.541+0000 7fe6d2667640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc14dfe0 con 0x7fe6cc06c2d0 2023-10-18T09:02:41.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.541+0000 7fe6d2667640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc1605c0 con 0x7fe6cc0c9620 2023-10-18T09:02:41.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.541+0000 7fe6d1e66640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6cc15af70 0x7fe6cc153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:41.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.541+0000 7fe6d1e66640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6cc15af70 0x7fe6cc153580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48246/0 (socket says 172.21.15.40:48246) 2023-10-18T09:02:41.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.541+0000 7fe6d1e66640 1 -- 172.21.15.40:0/1843790663 learned_addr learned my addr 172.21.15.40:0/1843790663 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:41.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.541+0000 7fe6d1665640 1 --2- 172.21.15.40:0/1843790663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6cc06c2d0 0x7fe6cc161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:41.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.541+0000 7fe6d0e64640 1 --2- 172.21.15.40:0/1843790663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6cc0c9620 0x7fe6cc161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:41.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.541+0000 7fe6d1e66640 1 -- 172.21.15.40:0/1843790663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6cc0c9620 msgr2=0x7fe6cc161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:41.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.541+0000 7fe6d1e66640 1 --2- 172.21.15.40:0/1843790663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6cc0c9620 0x7fe6cc161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:41.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.542+0000 7fe6d1e66640 1 -- 172.21.15.40:0/1843790663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6cc06c2d0 msgr2=0x7fe6cc161cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:41.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.542+0000 7fe6d1e66640 1 --2- 172.21.15.40:0/1843790663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6cc06c2d0 0x7fe6cc161cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:41.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.542+0000 7fe6d1e66640 1 -- 172.21.15.40:0/1843790663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c4009900 con 0x7fe6cc15af70 2023-10-18T09:02:41.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.542+0000 7fe6d0e64640 1 --2- 172.21.15.40:0/1843790663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6cc0c9620 0x7fe6cc161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:41.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.542+0000 7fe6d1e66640 1 --2- 172.21.15.40:0/1843790663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6cc15af70 0x7fe6cc153580 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4037040 tx=0x7fe6c4035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:41.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.542+0000 7fe6c1ffb640 1 -- 172.21.15.40:0/1843790663 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4034440 con 0x7fe6cc15af70 2023-10-18T09:02:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.542+0000 7fe6c1ffb640 1 -- 172.21.15.40:0/1843790663 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe6c4004540 con 0x7fe6cc15af70 2023-10-18T09:02:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.542+0000 7fe6c1ffb640 1 -- 172.21.15.40:0/1843790663 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c403f440 con 0x7fe6cc15af70 2023-10-18T09:02:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.542+0000 7fe6d2667640 1 -- 172.21.15.40:0/1843790663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6cc14dd00 con 0x7fe6cc15af70 2023-10-18T09:02:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.543+0000 7fe6d2667640 1 -- 172.21.15.40:0/1843790663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6cc162200 con 0x7fe6cc15af70 2023-10-18T09:02:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.544+0000 7fe6d2667640 1 -- 172.21.15.40:0/1843790663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6cc162390 con 0x7fe6cc15af70 2023-10-18T09:02:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.545+0000 7fe6c1ffb640 1 -- 172.21.15.40:0/1843790663 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe6c403b040 con 0x7fe6cc15af70 2023-10-18T09:02:41.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.546+0000 7fe6c1ffb640 1 --2- 172.21.15.40:0/1843790663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe6b006ae40 0x7fe6b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:41.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.546+0000 7fe6c1ffb640 1 -- 172.21.15.40:0/1843790663 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe6c40ac520 con 0x7fe6cc15af70 2023-10-18T09:02:41.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.546+0000 7fe6d1665640 1 --2- 172.21.15.40:0/1843790663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe6b006ae40 0x7fe6b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:41.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.548+0000 7fe6d1665640 1 --2- 172.21.15.40:0/1843790663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe6b006ae40 0x7fe6b006d300 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8004ea0 tx=0x7fe6c8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:41.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.551+0000 7fe6c1ffb640 1 -- 172.21.15.40:0/1843790663 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6cc162390 con 0x7fe6cc15af70 2023-10-18T09:02:41.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:41.784+0000 7fe6d2667640 1 -- 172.21.15.40:0/1843790663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe6cc162390 con 0x7fe6cc15af70 2023-10-18T09:02:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.326+0000 7fe6c1ffb640 1 -- 172.21.15.40:0/1843790663 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v53) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe6cc162390 con 0x7fe6cc15af70 2023-10-18T09:02:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 14 and data pool 15 2023-10-18T09:02:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.330+0000 7fe6d2667640 1 -- 172.21.15.40:0/1843790663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe6b006ae40 msgr2=0x7fe6b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:42.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.330+0000 7fe6d2667640 1 --2- 172.21.15.40:0/1843790663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe6b006ae40 0x7fe6b006d300 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8004ea0 tx=0x7fe6c8009040 comp rx=0 tx=0).stop 2023-10-18T09:02:42.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.330+0000 7fe6d2667640 1 -- 172.21.15.40:0/1843790663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6cc15af70 msgr2=0x7fe6cc153580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:42.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.330+0000 7fe6d2667640 1 --2- 172.21.15.40:0/1843790663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6cc15af70 0x7fe6cc153580 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4037040 tx=0x7fe6c4035db0 comp rx=0 tx=0).stop 2023-10-18T09:02:42.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.331+0000 7fe6d2667640 1 -- 172.21.15.40:0/1843790663 shutdown_connections 2023-10-18T09:02:42.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.331+0000 7fe6d2667640 1 --2- 172.21.15.40:0/1843790663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe6b006ae40 0x7fe6b006d300 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:42.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.331+0000 7fe6d2667640 1 --2- 172.21.15.40:0/1843790663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6cc15af70 0x7fe6cc153580 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:42.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.331+0000 7fe6d2667640 1 --2- 172.21.15.40:0/1843790663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6cc0c9620 0x7fe6cc161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:42.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.331+0000 7fe6d2667640 1 --2- 172.21.15.40:0/1843790663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6cc06c2d0 0x7fe6cc161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:42.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.331+0000 7fe6d2667640 1 -- 172.21.15.40:0/1843790663 >> 172.21.15.40:0/1843790663 conn(0x7fe6cc06ca10 msgr2=0x7fe6cc1625d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:42.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.331+0000 7fe6d2667640 1 -- 172.21.15.40:0/1843790663 shutdown_connections 2023-10-18T09:02:42.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.331+0000 7fe6d2667640 1 -- 172.21.15.40:0/1843790663 wait complete. 2023-10-18T09:02:42.358 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:02:42.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.559+0000 7fcaff59e640 1 -- 172.21.15.40:0/2370730633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb0006d6f0 msgr2=0x7fcb000b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:42.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.559+0000 7fcaff59e640 1 --2- 172.21.15.40:0/2370730633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb0006d6f0 0x7fcb000b15b0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0005fd0 tx=0x7fcaf002be20 comp rx=0 tx=0).stop 2023-10-18T09:02:42.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.559+0000 7fcaff59e640 1 -- 172.21.15.40:0/2370730633 shutdown_connections 2023-10-18T09:02:42.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.559+0000 7fcaff59e640 1 --2- 172.21.15.40:0/2370730633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb0006d6f0 0x7fcb000b15b0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:42.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.559+0000 7fcaff59e640 1 -- 172.21.15.40:0/2370730633 >> 172.21.15.40:0/2370730633 conn(0x7fcb00069720 msgr2=0x7fcb00069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:42.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.559+0000 7fcaff59e640 1 -- 172.21.15.40:0/2370730633 shutdown_connections 2023-10-18T09:02:42.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.559+0000 7fcaff59e640 1 -- 172.21.15.40:0/2370730633 wait complete. 2023-10-18T09:02:42.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.560+0000 7fcaff59e640 1 Processor -- start 2023-10-18T09:02:42.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.561+0000 7fcaff59e640 1 -- start start 2023-10-18T09:02:42.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.561+0000 7fcaff59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb0006d6f0 0x7fcb000ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:42.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.561+0000 7fcaff59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb000b0630 0x7fcb000ab960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:42.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.562+0000 7fcaff59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb000ca0d0 0x7fcb000abea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:42.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.562+0000 7fcaff59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb001f37c0 con 0x7fcb0006d6f0 2023-10-18T09:02:42.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.562+0000 7fcaff59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb000aea90 con 0x7fcb000b0630 2023-10-18T09:02:42.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.562+0000 7fcaff59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb001f34e0 con 0x7fcb000ca0d0 2023-10-18T09:02:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.562+0000 7fcafe59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb0006d6f0 0x7fcb000ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.562+0000 7fcafe59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb0006d6f0 0x7fcb000ae550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48260/0 (socket says 172.21.15.40:48260) 2023-10-18T09:02:42.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.562+0000 7fcafe59c640 1 -- 172.21.15.40:0/2402574925 learned_addr learned my addr 172.21.15.40:0/2402574925 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.562+0000 7fcaf7fff640 1 --2- 172.21.15.40:0/2402574925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb000b0630 0x7fcb000ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.562+0000 7fcafe59c640 1 -- 172.21.15.40:0/2402574925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb000ca0d0 msgr2=0x7fcb000abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:42.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.562+0000 7fcafed9d640 1 --2- 172.21.15.40:0/2402574925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb000ca0d0 0x7fcb000abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:42.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.562+0000 7fcafe59c640 1 --2- 172.21.15.40:0/2402574925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb000ca0d0 0x7fcb000abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:42.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.562+0000 7fcafe59c640 1 -- 172.21.15.40:0/2402574925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb000b0630 msgr2=0x7fcb000ab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:42.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.562+0000 7fcafe59c640 1 --2- 172.21.15.40:0/2402574925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb000b0630 0x7fcb000ab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:42.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.562+0000 7fcafe59c640 1 -- 172.21.15.40:0/2402574925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf0009560 con 0x7fcb0006d6f0 2023-10-18T09:02:42.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.562+0000 7fcaf7fff640 1 --2- 172.21.15.40:0/2402574925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb000b0630 0x7fcb000ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:42.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.562+0000 7fcafed9d640 1 --2- 172.21.15.40:0/2402574925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb000ca0d0 0x7fcb000abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:42.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.563+0000 7fcafe59c640 1 --2- 172.21.15.40:0/2402574925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb0006d6f0 0x7fcb000ae550 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009850 tx=0x7fcaf002dba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:42.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.563+0000 7fcaf6ffd640 1 -- 172.21.15.40:0/2402574925 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf00035f0 con 0x7fcb0006d6f0 2023-10-18T09:02:42.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.563+0000 7fcaf6ffd640 1 -- 172.21.15.40:0/2402574925 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcaf0003840 con 0x7fcb0006d6f0 2023-10-18T09:02:42.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.563+0000 7fcaf6ffd640 1 -- 172.21.15.40:0/2402574925 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0003bf0 con 0x7fcb0006d6f0 2023-10-18T09:02:42.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.563+0000 7fcaff59e640 1 -- 172.21.15.40:0/2402574925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0015bef0 con 0x7fcb0006d6f0 2023-10-18T09:02:42.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.563+0000 7fcaff59e640 1 -- 172.21.15.40:0/2402574925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb001f3e80 con 0x7fcb0006d6f0 2023-10-18T09:02:42.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.564+0000 7fcaff59e640 1 -- 172.21.15.40:0/2402574925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb00066e00 con 0x7fcb0006d6f0 2023-10-18T09:02:42.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.565+0000 7fcaf6ffd640 1 -- 172.21.15.40:0/2402574925 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcaf0003d50 con 0x7fcb0006d6f0 2023-10-18T09:02:42.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.566+0000 7fcaf6ffd640 1 --2- 172.21.15.40:0/2402574925 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcae006ae40 0x7fcae006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:42.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.566+0000 7fcaf6ffd640 1 -- 172.21.15.40:0/2402574925 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcaf00ac340 con 0x7fcb0006d6f0 2023-10-18T09:02:42.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.567+0000 7fcaf7fff640 1 --2- 172.21.15.40:0/2402574925 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcae006ae40 0x7fcae006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.568+0000 7fcaf7fff640 1 --2- 172.21.15.40:0/2402574925 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcae006ae40 0x7fcae006d300 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcb00156260 tx=0x7fcae8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:42.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.570+0000 7fcaf6ffd640 1 -- 172.21.15.40:0/2402574925 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcaf0077500 con 0x7fcb0006d6f0 2023-10-18T09:02:42.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.764+0000 7fcaff59e640 1 -- 172.21.15.40:0/2402574925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb000caf90 con 0x7fcb0006d6f0 2023-10-18T09:02:42.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.768+0000 7fcaf6ffd640 1 -- 172.21.15.40:0/2402574925 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16545 (secure 0 0 0) 0x7fcaf007ab20 con 0x7fcb0006d6f0 2023-10-18T09:02:42.770 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:42.771 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":129,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:02:41.785771+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:02:39.531482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-18T09:02:40.812071+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:02:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.770+0000 7fcaff59e640 1 -- 172.21.15.40:0/2402574925 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcae006ae40 msgr2=0x7fcae006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.770+0000 7fcaff59e640 1 --2- 172.21.15.40:0/2402574925 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcae006ae40 0x7fcae006d300 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcb00156260 tx=0x7fcae8009040 comp rx=0 tx=0).stop 2023-10-18T09:02:42.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.770+0000 7fcaff59e640 1 -- 172.21.15.40:0/2402574925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb0006d6f0 msgr2=0x7fcb000ae550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.770+0000 7fcaff59e640 1 --2- 172.21.15.40:0/2402574925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb0006d6f0 0x7fcb000ae550 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009850 tx=0x7fcaf002dba0 comp rx=0 tx=0).stop 2023-10-18T09:02:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.771+0000 7fcaff59e640 1 -- 172.21.15.40:0/2402574925 shutdown_connections 2023-10-18T09:02:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.771+0000 7fcaff59e640 1 --2- 172.21.15.40:0/2402574925 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcae006ae40 0x7fcae006d300 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.771+0000 7fcaff59e640 1 --2- 172.21.15.40:0/2402574925 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb000ca0d0 0x7fcb000abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:42.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.771+0000 7fcaff59e640 1 --2- 172.21.15.40:0/2402574925 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb000b0630 0x7fcb000ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:42.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.771+0000 7fcaff59e640 1 --2- 172.21.15.40:0/2402574925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb0006d6f0 0x7fcb000ae550 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:42.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.771+0000 7fcaff59e640 1 -- 172.21.15.40:0/2402574925 >> 172.21.15.40:0/2402574925 conn(0x7fcb00069720 msgr2=0x7fcb000c9950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:42.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.771+0000 7fcaff59e640 1 -- 172.21.15.40:0/2402574925 shutdown_connections 2023-10-18T09:02:42.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:42.771+0000 7fcaff59e640 1 -- 172.21.15.40:0/2402574925 wait complete. 2023-10-18T09:02:42.793 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:02:43.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.027+0000 7fe0eb59e640 1 -- 172.21.15.40:0/4104278848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0b15b0 msgr2=0x7fe0ec0b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.027+0000 7fe0eb59e640 1 --2- 172.21.15.40:0/4104278848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0b15b0 0x7fe0ec0b0630 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec06b050 tx=0x7fe0dc02bda0 comp rx=0 tx=0).stop 2023-10-18T09:02:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.027+0000 7fe0eb59e640 1 -- 172.21.15.40:0/4104278848 shutdown_connections 2023-10-18T09:02:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.027+0000 7fe0eb59e640 1 --2- 172.21.15.40:0/4104278848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0ec0b2270 0x7fe0ec0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.027+0000 7fe0eb59e640 1 --2- 172.21.15.40:0/4104278848 >> v2:172.21.15.144:3300/0 conn(0x7fe0ec0b1e10 0x7fe0ec0c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:43.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.027+0000 7fe0eb59e640 1 --2- 172.21.15.40:0/4104278848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0b15b0 0x7fe0ec0b0630 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:43.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.027+0000 7fe0eb59e640 1 -- 172.21.15.40:0/4104278848 >> 172.21.15.40:0/4104278848 conn(0x7fe0ec069720 msgr2=0x7fe0ec069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:43.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.027+0000 7fe0eb59e640 1 -- 172.21.15.40:0/4104278848 shutdown_connections 2023-10-18T09:02:43.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.027+0000 7fe0eb59e640 1 -- 172.21.15.40:0/4104278848 wait complete. 2023-10-18T09:02:43.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.029+0000 7fe0eb59e640 1 Processor -- start 2023-10-18T09:02:43.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.029+0000 7fe0eb59e640 1 -- start start 2023-10-18T09:02:43.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.030+0000 7fe0eb59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0ec0b15b0 0x7fe0ec151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:43.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.030+0000 7fe0eb59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0b1e10 0x7fe0ec152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:43.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.030+0000 7fe0eb59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0ec0b2270 0x7fe0ec13d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:43.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.030+0000 7fe0eb59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec150d00 con 0x7fe0ec0b1e10 2023-10-18T09:02:43.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.030+0000 7fe0eb59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec150fe0 con 0x7fe0ec0b15b0 2023-10-18T09:02:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.030+0000 7fe0eb59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec1512c0 con 0x7fe0ec0b2270 2023-10-18T09:02:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.030+0000 7fe0e9d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0b1e10 0x7fe0ec152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.030+0000 7fe0e9d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0b1e10 0x7fe0ec152ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48282/0 (socket says 172.21.15.40:48282) 2023-10-18T09:02:43.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.030+0000 7fe0e9d9b640 1 -- 172.21.15.40:0/641836740 learned_addr learned my addr 172.21.15.40:0/641836740 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:43.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.031+0000 7fe0ea59c640 1 --2- 172.21.15.40:0/641836740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0ec0b15b0 0x7fe0ec151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:43.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.031+0000 7fe0e9d9b640 1 -- 172.21.15.40:0/641836740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0ec0b2270 msgr2=0x7fe0ec13d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:43.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.031+0000 7fe0e9d9b640 1 --2- 172.21.15.40:0/641836740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0ec0b2270 0x7fe0ec13d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:43.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.031+0000 7fe0e9d9b640 1 -- 172.21.15.40:0/641836740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0ec0b15b0 msgr2=0x7fe0ec151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:43.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.031+0000 7fe0e9d9b640 1 --2- 172.21.15.40:0/641836740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0ec0b15b0 0x7fe0ec151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:43.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.031+0000 7fe0e9d9b640 1 -- 172.21.15.40:0/641836740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc009560 con 0x7fe0ec0b1e10 2023-10-18T09:02:43.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.031+0000 7fe0ea59c640 1 --2- 172.21.15.40:0/641836740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0ec0b15b0 0x7fe0ec151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:43.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.031+0000 7fe0e9d9b640 1 --2- 172.21.15.40:0/641836740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0b1e10 0x7fe0ec152ad0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400db00 tx=0x7fe0d400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:43.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.032+0000 7fe0e2ffd640 1 -- 172.21.15.40:0/641836740 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400bea0 con 0x7fe0ec0b1e10 2023-10-18T09:02:43.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.032+0000 7fe0e2ffd640 1 -- 172.21.15.40:0/641836740 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe0d4003a60 con 0x7fe0ec0b1e10 2023-10-18T09:02:43.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.032+0000 7fe0eb59e640 1 -- 172.21.15.40:0/641836740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ec1515a0 con 0x7fe0ec0b1e10 2023-10-18T09:02:43.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.032+0000 7fe0e2ffd640 1 -- 172.21.15.40:0/641836740 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4003e70 con 0x7fe0ec0b1e10 2023-10-18T09:02:43.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.032+0000 7fe0eb59e640 1 -- 172.21.15.40:0/641836740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ec153010 con 0x7fe0ec0b1e10 2023-10-18T09:02:43.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.033+0000 7fe0eb59e640 1 -- 172.21.15.40:0/641836740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ec1531d0 con 0x7fe0ec0b1e10 2023-10-18T09:02:43.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.034+0000 7fe0e2ffd640 1 -- 172.21.15.40:0/641836740 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe0d4004020 con 0x7fe0ec0b1e10 2023-10-18T09:02:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.035+0000 7fe0e2ffd640 1 --2- 172.21.15.40:0/641836740 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe0cc06ae70 0x7fe0cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.035+0000 7fe0e2ffd640 1 -- 172.21.15.40:0/641836740 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0d4089e90 con 0x7fe0ec0b1e10 2023-10-18T09:02:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.036+0000 7fe0ea59c640 1 --2- 172.21.15.40:0/641836740 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe0cc06ae70 0x7fe0cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:43.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.037+0000 7fe0ea59c640 1 --2- 172.21.15.40:0/641836740 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe0cc06ae70 0x7fe0cc06d330 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc035000 tx=0x7fe0dc004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:43.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.040+0000 7fe0e2ffd640 1 -- 172.21.15.40:0/641836740 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe0d4054ea0 con 0x7fe0ec0b1e10 2023-10-18T09:02:43.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.239+0000 7fe0eb59e640 1 -- 172.21.15.40:0/641836740 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0ec066e00 con 0x7fe0ec0b1e10 2023-10-18T09:02:43.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.243+0000 7fe0e2ffd640 1 -- 172.21.15.40:0/641836740 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+16532 (secure 0 0 0) 0x7fe0d408b090 con 0x7fe0ec0b1e10 2023-10-18T09:02:43.246 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:43.246 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":130,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:02:43.051244+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:02:39.531482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-18T09:02:40.812071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:02:43.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.245+0000 7fe0eb59e640 1 -- 172.21.15.40:0/641836740 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe0cc06ae70 msgr2=0x7fe0cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:43.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.245+0000 7fe0eb59e640 1 --2- 172.21.15.40:0/641836740 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe0cc06ae70 0x7fe0cc06d330 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc035000 tx=0x7fe0dc004730 comp rx=0 tx=0).stop 2023-10-18T09:02:43.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.245+0000 7fe0eb59e640 1 -- 172.21.15.40:0/641836740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0b1e10 msgr2=0x7fe0ec152ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:43.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.245+0000 7fe0eb59e640 1 --2- 172.21.15.40:0/641836740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0b1e10 0x7fe0ec152ad0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400db00 tx=0x7fe0d400b260 comp rx=0 tx=0).stop 2023-10-18T09:02:43.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.246+0000 7fe0eb59e640 1 -- 172.21.15.40:0/641836740 shutdown_connections 2023-10-18T09:02:43.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.246+0000 7fe0eb59e640 1 --2- 172.21.15.40:0/641836740 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe0cc06ae70 0x7fe0cc06d330 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:43.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.246+0000 7fe0eb59e640 1 --2- 172.21.15.40:0/641836740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0ec0b2270 0x7fe0ec13d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:43.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.246+0000 7fe0eb59e640 1 --2- 172.21.15.40:0/641836740 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0ec0b1e10 0x7fe0ec152ad0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:43.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.246+0000 7fe0eb59e640 1 --2- 172.21.15.40:0/641836740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0ec0b15b0 0x7fe0ec151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:43.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.246+0000 7fe0eb59e640 1 -- 172.21.15.40:0/641836740 >> 172.21.15.40:0/641836740 conn(0x7fe0ec069720 msgr2=0x7fe0ec069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:43.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.246+0000 7fe0eb59e640 1 -- 172.21.15.40:0/641836740 shutdown_connections 2023-10-18T09:02:43.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.246+0000 7fe0eb59e640 1 -- 172.21.15.40:0/641836740 wait complete. 2023-10-18T09:02:43.269 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T09:02:43.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.502+0000 7f7612903640 1 -- 172.21.15.40:0/237482281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f760c15a490 msgr2=0x7f760c0cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:43.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.502+0000 7f7612903640 1 --2- 172.21.15.40:0/237482281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f760c15a490 0x7f760c0cc470 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f760c06d930 tx=0x7f75fc02bd90 comp rx=0 tx=0).stop 2023-10-18T09:02:43.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.502+0000 7f7612903640 1 -- 172.21.15.40:0/237482281 shutdown_connections 2023-10-18T09:02:43.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.502+0000 7f7612903640 1 --2- 172.21.15.40:0/237482281 >> v2:172.21.15.144:3300/0 conn(0x7f760c06baf0 0x7f760c06bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:43.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.502+0000 7f7612903640 1 --2- 172.21.15.40:0/237482281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f760c15a490 0x7f760c0cc470 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:43.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.502+0000 7f7612903640 1 --2- 172.21.15.40:0/237482281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f760c1593c0 0x7f760c0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:43.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.502+0000 7f7612903640 1 -- 172.21.15.40:0/237482281 >> 172.21.15.40:0/237482281 conn(0x7f760c06ca10 msgr2=0x7f760c06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:43.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.502+0000 7f7612903640 1 -- 172.21.15.40:0/237482281 shutdown_connections 2023-10-18T09:02:43.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.502+0000 7f7612903640 1 -- 172.21.15.40:0/237482281 wait complete. 2023-10-18T09:02:43.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.504+0000 7f7612903640 1 Processor -- start 2023-10-18T09:02:43.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.504+0000 7f7612903640 1 -- start start 2023-10-18T09:02:43.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.505+0000 7f7612903640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f760c06baf0 0x7f760c143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:43.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.505+0000 7f7612903640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f760c1593c0 0x7f760c143210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:43.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.505+0000 7f7612903640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f760c15a490 0x7f760c0aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:43.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.505+0000 7f7612903640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760c145f40 con 0x7f760c15a490 2023-10-18T09:02:43.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.505+0000 7f7612903640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760c146220 con 0x7f760c1593c0 2023-10-18T09:02:43.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.505+0000 7f7612903640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760c146500 con 0x7f760c06baf0 2023-10-18T09:02:43.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.505+0000 7f7612102640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f760c15a490 0x7f760c0aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:43.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.506+0000 7f7612102640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f760c15a490 0x7f760c0aa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48302/0 (socket says 172.21.15.40:48302) 2023-10-18T09:02:43.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.506+0000 7f7612102640 1 -- 172.21.15.40:0/1444375625 learned_addr learned my addr 172.21.15.40:0/1444375625 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:43.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.506+0000 7f7611100640 1 --2- 172.21.15.40:0/1444375625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f760c1593c0 0x7f760c143210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:43.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.506+0000 7f7611901640 1 --2- 172.21.15.40:0/1444375625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f760c06baf0 0x7f760c143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:43.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.506+0000 7f7612102640 1 -- 172.21.15.40:0/1444375625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f760c06baf0 msgr2=0x7f760c143b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:43.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.506+0000 7f7612102640 1 --2- 172.21.15.40:0/1444375625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f760c06baf0 0x7f760c143b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:43.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.506+0000 7f7612102640 1 -- 172.21.15.40:0/1444375625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f760c1593c0 msgr2=0x7f760c143210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:43.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.506+0000 7f7612102640 1 --2- 172.21.15.40:0/1444375625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f760c1593c0 0x7f760c143210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:43.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.506+0000 7f7612102640 1 -- 172.21.15.40:0/1444375625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75fc009560 con 0x7f760c15a490 2023-10-18T09:02:43.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.506+0000 7f7612102640 1 --2- 172.21.15.40:0/1444375625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f760c15a490 0x7f760c0aa880 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f760400eaa0 tx=0x7f760400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:43.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.507+0000 7f76027fc640 1 -- 172.21.15.40:0/1444375625 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760400cdb0 con 0x7f760c15a490 2023-10-18T09:02:43.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.507+0000 7f76027fc640 1 -- 172.21.15.40:0/1444375625 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7604004b20 con 0x7f760c15a490 2023-10-18T09:02:43.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.507+0000 7f76027fc640 1 -- 172.21.15.40:0/1444375625 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604005020 con 0x7f760c15a490 2023-10-18T09:02:43.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.507+0000 7f7612903640 1 -- 172.21.15.40:0/1444375625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760c1467e0 con 0x7f760c15a490 2023-10-18T09:02:43.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.507+0000 7f7612903640 1 -- 172.21.15.40:0/1444375625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f760c143750 con 0x7f760c15a490 2023-10-18T09:02:43.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.508+0000 7f7612903640 1 -- 172.21.15.40:0/1444375625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f760c1e1930 con 0x7f760c15a490 2023-10-18T09:02:43.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.509+0000 7f76027fc640 1 -- 172.21.15.40:0/1444375625 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7604005180 con 0x7f760c15a490 2023-10-18T09:02:43.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.510+0000 7f76027fc640 1 --2- 172.21.15.40:0/1444375625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f75f006ae40 0x7f75f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:43.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.510+0000 7f76027fc640 1 -- 172.21.15.40:0/1444375625 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f760408a070 con 0x7f760c15a490 2023-10-18T09:02:43.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.511+0000 7f7611901640 1 --2- 172.21.15.40:0/1444375625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f75f006ae40 0x7f75f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:43.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.512+0000 7f7611901640 1 --2- 172.21.15.40:0/1444375625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f75f006ae40 0x7f75f006d300 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f760800cac0 tx=0x7f760800c4a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:43.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.514+0000 7f76027fc640 1 -- 172.21.15.40:0/1444375625 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f76040bcca0 con 0x7f760c15a490 2023-10-18T09:02:43.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:43.745+0000 7f7612903640 1 -- 172.21.15.40:0/1444375625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f760c066e00 con 0x7f760c15a490 2023-10-18T09:02:44.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.348+0000 7f76027fc640 1 -- 172.21.15.40:0/1444375625 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v56) v1 ==== 118+0+0 (secure 0 0 0) 0x7f760401f020 con 0x7f760c15a490 2023-10-18T09:02:44.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.352+0000 7f7612903640 1 -- 172.21.15.40:0/1444375625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f75f006ae40 msgr2=0x7f75f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:44.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.352+0000 7f7612903640 1 --2- 172.21.15.40:0/1444375625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f75f006ae40 0x7f75f006d300 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f760800cac0 tx=0x7f760800c4a0 comp rx=0 tx=0).stop 2023-10-18T09:02:44.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.352+0000 7f7612903640 1 -- 172.21.15.40:0/1444375625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f760c15a490 msgr2=0x7f760c0aa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:44.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.352+0000 7f7612903640 1 --2- 172.21.15.40:0/1444375625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f760c15a490 0x7f760c0aa880 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f760400eaa0 tx=0x7f760400ef60 comp rx=0 tx=0).stop 2023-10-18T09:02:44.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.352+0000 7f7612903640 1 -- 172.21.15.40:0/1444375625 shutdown_connections 2023-10-18T09:02:44.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.352+0000 7f7612903640 1 --2- 172.21.15.40:0/1444375625 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f75f006ae40 0x7f75f006d300 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:44.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.352+0000 7f7612903640 1 --2- 172.21.15.40:0/1444375625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f760c15a490 0x7f760c0aa880 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:44.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.352+0000 7f7612903640 1 --2- 172.21.15.40:0/1444375625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f760c1593c0 0x7f760c143210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:44.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.352+0000 7f7612903640 1 --2- 172.21.15.40:0/1444375625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f760c06baf0 0x7f760c143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:44.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.352+0000 7f7612903640 1 -- 172.21.15.40:0/1444375625 >> 172.21.15.40:0/1444375625 conn(0x7f760c06ca10 msgr2=0x7f760c070fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:44.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.353+0000 7f7612903640 1 -- 172.21.15.40:0/1444375625 shutdown_connections 2023-10-18T09:02:44.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.353+0000 7f7612903640 1 -- 172.21.15.40:0/1444375625 wait complete. 2023-10-18T09:02:44.378 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T09:02:44.379 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T09:02:44.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.616+0000 7f1b338f2640 1 -- 172.21.15.40:0/2465143040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b18004d00 msgr2=0x7f1b18005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:44.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.616+0000 7f1b338f2640 1 --2- 172.21.15.40:0/2465143040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b18004d00 0x7f1b18005180 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400c610 tx=0x7f1b180057f0 comp rx=0 tx=0).stop 2023-10-18T09:02:44.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.617+0000 7f1b31e8f640 1 -- 172.21.15.40:0/2465143040 reap_dead start 2023-10-18T09:02:44.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.617+0000 7f1b338f2640 1 -- 172.21.15.40:0/2465143040 shutdown_connections 2023-10-18T09:02:44.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.617+0000 7f1b338f2640 1 -- 172.21.15.40:0/2465143040 >> 172.21.15.40:0/2465143040 conn(0x7f1b2c069720 msgr2=0x7f1b2c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:44.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.617+0000 7f1b338f2640 1 -- 172.21.15.40:0/2465143040 shutdown_connections 2023-10-18T09:02:44.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.617+0000 7f1b338f2640 1 -- 172.21.15.40:0/2465143040 wait complete. 2023-10-18T09:02:44.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.619+0000 7f1b338f2640 1 Processor -- start 2023-10-18T09:02:44.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.619+0000 7f1b338f2640 1 -- start start 2023-10-18T09:02:44.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.620+0000 7f1b338f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b18004d00 0x7f1b2c0af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:44.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.620+0000 7f1b338f2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b2c1359c0 0x7f1b2c0ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:44.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.620+0000 7f1b338f2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b2c153710 0x7f1b2c13e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:44.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.620+0000 7f1b338f2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c153b90 con 0x7f1b18004d00 2023-10-18T09:02:44.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.620+0000 7f1b338f2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c13be80 con 0x7f1b2c153710 2023-10-18T09:02:44.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.620+0000 7f1b338f2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c0ae510 con 0x7f1b2c1359c0 2023-10-18T09:02:44.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.620+0000 7f1b3168e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b18004d00 0x7f1b2c0af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:44.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.620+0000 7f1b3168e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b18004d00 0x7f1b2c0af5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48324/0 (socket says 172.21.15.40:48324) 2023-10-18T09:02:44.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.620+0000 7f1b3168e640 1 -- 172.21.15.40:0/3188654675 learned_addr learned my addr 172.21.15.40:0/3188654675 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:44.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.621+0000 7f1b31e8f640 1 --2- 172.21.15.40:0/3188654675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b2c153710 0x7f1b2c13e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:44.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.621+0000 7f1b30e8d640 1 --2- 172.21.15.40:0/3188654675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b2c1359c0 0x7f1b2c0ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:44.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.621+0000 7f1b3168e640 1 -- 172.21.15.40:0/3188654675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b2c1359c0 msgr2=0x7f1b2c0ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:44.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.621+0000 7f1b3168e640 1 --2- 172.21.15.40:0/3188654675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b2c1359c0 0x7f1b2c0ade80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:44.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.621+0000 7f1b3168e640 1 -- 172.21.15.40:0/3188654675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b2c153710 msgr2=0x7f1b2c13e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:44.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.621+0000 7f1b3168e640 1 --2- 172.21.15.40:0/3188654675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b2c153710 0x7f1b2c13e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:44.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.621+0000 7f1b3168e640 1 -- 172.21.15.40:0/3188654675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2c0ae7f0 con 0x7f1b18004d00 2023-10-18T09:02:44.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.621+0000 7f1b3168e640 1 --2- 172.21.15.40:0/3188654675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b18004d00 0x7f1b2c0af5b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f1b28010a10 tx=0x7f1b28010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:44.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.621+0000 7f1b21ffb640 1 -- 172.21.15.40:0/3188654675 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b280096a0 con 0x7f1b18004d00 2023-10-18T09:02:44.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.621+0000 7f1b21ffb640 1 -- 172.21.15.40:0/3188654675 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1b2800ece0 con 0x7f1b18004d00 2023-10-18T09:02:44.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.621+0000 7f1b21ffb640 1 -- 172.21.15.40:0/3188654675 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28005280 con 0x7f1b18004d00 2023-10-18T09:02:44.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.621+0000 7f1b338f2640 1 -- 172.21.15.40:0/3188654675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2c0aead0 con 0x7f1b18004d00 2023-10-18T09:02:44.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.622+0000 7f1b338f2640 1 -- 172.21.15.40:0/3188654675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2c0af090 con 0x7f1b18004d00 2023-10-18T09:02:44.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.623+0000 7f1b338f2640 1 -- 172.21.15.40:0/3188654675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2c066fd0 con 0x7f1b18004d00 2023-10-18T09:02:44.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.624+0000 7f1b21ffb640 1 -- 172.21.15.40:0/3188654675 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1b280053e0 con 0x7f1b18004d00 2023-10-18T09:02:44.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.625+0000 7f1b21ffb640 1 --2- 172.21.15.40:0/3188654675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b1006ae40 0x7f1b1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:44.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.625+0000 7f1b21ffb640 1 -- 172.21.15.40:0/3188654675 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b28016070 con 0x7f1b18004d00 2023-10-18T09:02:44.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.625+0000 7f1b30e8d640 1 --2- 172.21.15.40:0/3188654675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b1006ae40 0x7f1b1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:44.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.627+0000 7f1b30e8d640 1 --2- 172.21.15.40:0/3188654675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b1006ae40 0x7f1b1006d300 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0caac0 tx=0x7f1b1c005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:44.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.629+0000 7f1b21ffb640 1 -- 172.21.15.40:0/3188654675 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b28058fc0 con 0x7f1b18004d00 2023-10-18T09:02:44.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:44.861+0000 7f1b338f2640 1 -- 172.21.15.40:0/3188654675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1b2c0cbc70 con 0x7f1b18004d00 2023-10-18T09:02:45.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.359+0000 7f1b21ffb640 1 -- 172.21.15.40:0/3188654675 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v57) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1b28058b50 con 0x7f1b18004d00 2023-10-18T09:02:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.362+0000 7f1b338f2640 1 -- 172.21.15.40:0/3188654675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b1006ae40 msgr2=0x7f1b1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.362+0000 7f1b338f2640 1 --2- 172.21.15.40:0/3188654675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b1006ae40 0x7f1b1006d300 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0caac0 tx=0x7f1b1c005f90 comp rx=0 tx=0).stop 2023-10-18T09:02:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.362+0000 7f1b338f2640 1 -- 172.21.15.40:0/3188654675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b18004d00 msgr2=0x7f1b2c0af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.362+0000 7f1b338f2640 1 --2- 172.21.15.40:0/3188654675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b18004d00 0x7f1b2c0af5b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f1b28010a10 tx=0x7f1b28010ed0 comp rx=0 tx=0).stop 2023-10-18T09:02:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.363+0000 7f1b338f2640 1 -- 172.21.15.40:0/3188654675 shutdown_connections 2023-10-18T09:02:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.363+0000 7f1b338f2640 1 --2- 172.21.15.40:0/3188654675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b1006ae40 0x7f1b1006d300 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.363+0000 7f1b338f2640 1 --2- 172.21.15.40:0/3188654675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b2c153710 0x7f1b2c13e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.363+0000 7f1b338f2640 1 --2- 172.21.15.40:0/3188654675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b2c1359c0 0x7f1b2c0ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.363+0000 7f1b338f2640 1 --2- 172.21.15.40:0/3188654675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b18004d00 0x7f1b2c0af5b0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.363+0000 7f1b338f2640 1 -- 172.21.15.40:0/3188654675 >> 172.21.15.40:0/3188654675 conn(0x7f1b2c069720 msgr2=0x7f1b2c0cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.363+0000 7f1b338f2640 1 -- 172.21.15.40:0/3188654675 shutdown_connections 2023-10-18T09:02:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.363+0000 7f1b338f2640 1 -- 172.21.15.40:0/3188654675 wait complete. 2023-10-18T09:02:45.387 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T09:02:45.388 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:02:45.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.623+0000 7fa40f59e640 1 -- 172.21.15.40:0/2320580237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4101420f0 msgr2=0x7fa4100b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:45.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.623+0000 7fa40f59e640 1 --2- 172.21.15.40:0/2320580237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4101420f0 0x7fa4100b0a80 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fa400009830 tx=0x7fa40002bd60 comp rx=0 tx=0).stop 2023-10-18T09:02:45.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.623+0000 7fa40f59e640 1 -- 172.21.15.40:0/2320580237 shutdown_connections 2023-10-18T09:02:45.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.623+0000 7fa40f59e640 1 --2- 172.21.15.40:0/2320580237 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4101420f0 0x7fa4100b0a80 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:45.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.623+0000 7fa40f59e640 1 -- 172.21.15.40:0/2320580237 >> 172.21.15.40:0/2320580237 conn(0x7fa410069720 msgr2=0x7fa410069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:45.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.623+0000 7fa40f59e640 1 -- 172.21.15.40:0/2320580237 shutdown_connections 2023-10-18T09:02:45.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.623+0000 7fa40f59e640 1 -- 172.21.15.40:0/2320580237 wait complete. 2023-10-18T09:02:45.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.625+0000 7fa40f59e640 1 Processor -- start 2023-10-18T09:02:45.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.625+0000 7fa40f59e640 1 -- start start 2023-10-18T09:02:45.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.626+0000 7fa40f59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa4100ca160 0x7fa410159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:45.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.626+0000 7fa40f59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4101420f0 0x7fa41012f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:45.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.626+0000 7fa40f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4101425a0 0x7fa410140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:45.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.626+0000 7fa40f59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa410140b10 con 0x7fa4101425a0 2023-10-18T09:02:45.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.626+0000 7fa40f59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa410159f30 con 0x7fa4101420f0 2023-10-18T09:02:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.626+0000 7fa40f59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4101401e0 con 0x7fa4100ca160 2023-10-18T09:02:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.626+0000 7fa40ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4101425a0 0x7fa410140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.626+0000 7fa40ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4101425a0 0x7fa410140fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54478/0 (socket says 172.21.15.40:54478) 2023-10-18T09:02:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.626+0000 7fa40ed9d640 1 -- 172.21.15.40:0/2462306745 learned_addr learned my addr 172.21.15.40:0/2462306745 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:45.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.626+0000 7fa407fff640 1 --2- 172.21.15.40:0/2462306745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4101420f0 0x7fa41012f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:45.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.626+0000 7fa40ed9d640 1 -- 172.21.15.40:0/2462306745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa4100ca160 msgr2=0x7fa410159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:45.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.626+0000 7fa40ed9d640 1 --2- 172.21.15.40:0/2462306745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa4100ca160 0x7fa410159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:45.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.626+0000 7fa40ed9d640 1 -- 172.21.15.40:0/2462306745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4101420f0 msgr2=0x7fa41012f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:45.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.626+0000 7fa40ed9d640 1 --2- 172.21.15.40:0/2462306745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4101420f0 0x7fa41012f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:45.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.627+0000 7fa40ed9d640 1 -- 172.21.15.40:0/2462306745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa400009490 con 0x7fa4101425a0 2023-10-18T09:02:45.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.627+0000 7fa40ed9d640 1 --2- 172.21.15.40:0/2462306745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4101425a0 0x7fa410140fa0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fa408004a30 tx=0x7fa40800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:45.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.627+0000 7fa406ffd640 1 -- 172.21.15.40:0/2462306745 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40800ac30 con 0x7fa4101425a0 2023-10-18T09:02:45.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.627+0000 7fa406ffd640 1 -- 172.21.15.40:0/2462306745 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa408003970 con 0x7fa4101425a0 2023-10-18T09:02:45.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.627+0000 7fa406ffd640 1 -- 172.21.15.40:0/2462306745 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40800aea0 con 0x7fa4101425a0 2023-10-18T09:02:45.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.627+0000 7fa40f59e640 1 -- 172.21.15.40:0/2462306745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4101404c0 con 0x7fa4101425a0 2023-10-18T09:02:45.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.627+0000 7fa40f59e640 1 -- 172.21.15.40:0/2462306745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41013d070 con 0x7fa4101425a0 2023-10-18T09:02:45.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.629+0000 7fa40f59e640 1 -- 172.21.15.40:0/2462306745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa410066e00 con 0x7fa4101425a0 2023-10-18T09:02:45.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.630+0000 7fa406ffd640 1 -- 172.21.15.40:0/2462306745 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa408004030 con 0x7fa4101425a0 2023-10-18T09:02:45.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.630+0000 7fa406ffd640 1 --2- 172.21.15.40:0/2462306745 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa3f006ae70 0x7fa3f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:45.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.631+0000 7fa406ffd640 1 -- 172.21.15.40:0/2462306745 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa40808a0e0 con 0x7fa4101425a0 2023-10-18T09:02:45.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.631+0000 7fa40e59c640 1 --2- 172.21.15.40:0/2462306745 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa3f006ae70 0x7fa3f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:45.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.632+0000 7fa40e59c640 1 --2- 172.21.15.40:0/2462306745 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa3f006ae70 0x7fa3f006d330 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa400036000 tx=0x7fa4000046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:45.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.635+0000 7fa406ffd640 1 -- 172.21.15.40:0/2462306745 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa408055100 con 0x7fa4101425a0 2023-10-18T09:02:45.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.863+0000 7fa40f59e640 1 -- 172.21.15.40:0/2462306745 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa410140c50 con 0x7fa4101425a0 2023-10-18T09:02:45.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.864+0000 7fa406ffd640 1 -- 172.21.15.40:0/2462306745 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fa408058720 con 0x7fa4101425a0 2023-10-18T09:02:45.866 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:45.866 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/700924196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":700924196},{"type":"v1","addr":"172.21.15.40:6833","nonce":700924196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/877414874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":877414874},{"type":"v1","addr":"172.21.15.144:6835","nonce":877414874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2596943574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2596943574},{"type":"v1","addr":"172.21.15.144:6837","nonce":2596943574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:02:41.786627+0000","modified":"2023-10-18T09:02:44.862740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1801003551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1801003551},{"type":"v1","addr":"172.21.15.40:6835","nonce":1801003551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-18T09:02:45.867 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2023-10-18T09:02:45.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.866+0000 7fa40f59e640 1 -- 172.21.15.40:0/2462306745 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa3f006ae70 msgr2=0x7fa3f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:45.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.866+0000 7fa40f59e640 1 --2- 172.21.15.40:0/2462306745 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa3f006ae70 0x7fa3f006d330 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa400036000 tx=0x7fa4000046c0 comp rx=0 tx=0).stop 2023-10-18T09:02:45.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.866+0000 7fa40f59e640 1 -- 172.21.15.40:0/2462306745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4101425a0 msgr2=0x7fa410140fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:45.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.866+0000 7fa40f59e640 1 --2- 172.21.15.40:0/2462306745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4101425a0 0x7fa410140fa0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fa408004a30 tx=0x7fa40800cf90 comp rx=0 tx=0).stop 2023-10-18T09:02:45.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.866+0000 7fa40f59e640 1 -- 172.21.15.40:0/2462306745 shutdown_connections 2023-10-18T09:02:45.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.866+0000 7fa40f59e640 1 --2- 172.21.15.40:0/2462306745 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa3f006ae70 0x7fa3f006d330 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:45.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.866+0000 7fa40f59e640 1 --2- 172.21.15.40:0/2462306745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4101425a0 0x7fa410140fa0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:45.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.866+0000 7fa40f59e640 1 --2- 172.21.15.40:0/2462306745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4101420f0 0x7fa41012f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:45.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.866+0000 7fa40f59e640 1 --2- 172.21.15.40:0/2462306745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa4100ca160 0x7fa410159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:45.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.866+0000 7fa40f59e640 1 -- 172.21.15.40:0/2462306745 >> 172.21.15.40:0/2462306745 conn(0x7fa410069720 msgr2=0x7fa4100aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:45.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.867+0000 7fa40f59e640 1 -- 172.21.15.40:0/2462306745 shutdown_connections 2023-10-18T09:02:45.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:45.867+0000 7fa40f59e640 1 -- 172.21.15.40:0/2462306745 wait complete. 2023-10-18T09:02:45.888 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:02:46.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.114+0000 7f5532a44640 1 -- 172.21.15.40:0/2639070801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f552c151bd0 msgr2=0x7f552c1586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.114+0000 7f5532a44640 1 --2- 172.21.15.40:0/2639070801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f552c151bd0 0x7f552c1586a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f552c06c840 tx=0x7f551c02be30 comp rx=0 tx=0).stop 2023-10-18T09:02:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.114+0000 7f5532a44640 1 -- 172.21.15.40:0/2639070801 shutdown_connections 2023-10-18T09:02:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.114+0000 7f5532a44640 1 --2- 172.21.15.40:0/2639070801 >> v2:172.21.15.144:3300/0 conn(0x7f552c0b0bc0 0x7f552c0b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.114+0000 7f5532a44640 1 --2- 172.21.15.40:0/2639070801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f552c151bd0 0x7f552c1586a0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.114+0000 7f5532a44640 1 -- 172.21.15.40:0/2639070801 >> 172.21.15.40:0/2639070801 conn(0x7f552c06cee0 msgr2=0x7f552c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:46.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.114+0000 7f5532a44640 1 -- 172.21.15.40:0/2639070801 shutdown_connections 2023-10-18T09:02:46.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.114+0000 7f5532a44640 1 -- 172.21.15.40:0/2639070801 wait complete. 2023-10-18T09:02:46.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.116+0000 7f5532a44640 1 Processor -- start 2023-10-18T09:02:46.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.116+0000 7f5532a44640 1 -- start start 2023-10-18T09:02:46.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.117+0000 7f5532a44640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f552c06b710 0x7f552c1567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:46.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.117+0000 7f5532a44640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f552c0b0bc0 0x7f552c14f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:46.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.117+0000 7f5532a44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f552c151bd0 0x7f552c1481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:46.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.117+0000 7f5532a44640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c150900 con 0x7f552c06b710 2023-10-18T09:02:46.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.117+0000 7f5532a44640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c14fcf0 con 0x7f552c151bd0 2023-10-18T09:02:46.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.117+0000 7f5532a44640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552c14ffd0 con 0x7f552c0b0bc0 2023-10-18T09:02:46.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.117+0000 7f552bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f552c06b710 0x7f552c1567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:46.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.118+0000 7f552bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f552c06b710 0x7f552c1567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54492/0 (socket says 172.21.15.40:54492) 2023-10-18T09:02:46.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.118+0000 7f552bfff640 1 -- 172.21.15.40:0/2902275798 learned_addr learned my addr 172.21.15.40:0/2902275798 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:46.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.118+0000 7f5530fe1640 1 --2- 172.21.15.40:0/2902275798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f552c151bd0 0x7f552c1481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:46.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.118+0000 7f552b7fe640 1 --2- 172.21.15.40:0/2902275798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f552c0b0bc0 0x7f552c14f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:46.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.118+0000 7f552bfff640 1 -- 172.21.15.40:0/2902275798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f552c0b0bc0 msgr2=0x7f552c14f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:46.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.118+0000 7f552bfff640 1 --2- 172.21.15.40:0/2902275798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f552c0b0bc0 0x7f552c14f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.118+0000 7f552bfff640 1 -- 172.21.15.40:0/2902275798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f552c151bd0 msgr2=0x7f552c1481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:46.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.118+0000 7f552bfff640 1 --2- 172.21.15.40:0/2902275798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f552c151bd0 0x7f552c1481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.118+0000 7f552bfff640 1 -- 172.21.15.40:0/2902275798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c009560 con 0x7f552c06b710 2023-10-18T09:02:46.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.118+0000 7f5530fe1640 1 --2- 172.21.15.40:0/2902275798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f552c151bd0 0x7f552c1481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:46.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.118+0000 7f552b7fe640 1 --2- 172.21.15.40:0/2902275798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f552c0b0bc0 0x7f552c14f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:46.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.118+0000 7f552bfff640 1 --2- 172.21.15.40:0/2902275798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f552c06b710 0x7f552c1567b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f551c009850 tx=0x7f551c02de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:46.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.119+0000 7f5528ff9640 1 -- 172.21.15.40:0/2902275798 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c003800 con 0x7f552c06b710 2023-10-18T09:02:46.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.119+0000 7f5528ff9640 1 -- 172.21.15.40:0/2902275798 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f551c003a50 con 0x7f552c06b710 2023-10-18T09:02:46.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.119+0000 7f5528ff9640 1 -- 172.21.15.40:0/2902275798 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c003e40 con 0x7f552c06b710 2023-10-18T09:02:46.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.119+0000 7f5532a44640 1 -- 172.21.15.40:0/2902275798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552c1502b0 con 0x7f552c06b710 2023-10-18T09:02:46.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.119+0000 7f5532a44640 1 -- 172.21.15.40:0/2902275798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552c148ee0 con 0x7f552c06b710 2023-10-18T09:02:46.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.120+0000 7f5532a44640 1 -- 172.21.15.40:0/2902275798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552c066fd0 con 0x7f552c06b710 2023-10-18T09:02:46.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.121+0000 7f5528ff9640 1 -- 172.21.15.40:0/2902275798 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f551c03e450 con 0x7f552c06b710 2023-10-18T09:02:46.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.122+0000 7f5528ff9640 1 --2- 172.21.15.40:0/2902275798 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f551806ae70 0x7f551806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:46.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.122+0000 7f5528ff9640 1 -- 172.21.15.40:0/2902275798 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f551c0ac160 con 0x7f552c06b710 2023-10-18T09:02:46.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.123+0000 7f552b7fe640 1 --2- 172.21.15.40:0/2902275798 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f551806ae70 0x7f551806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:46.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.124+0000 7f552b7fe640 1 --2- 172.21.15.40:0/2902275798 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f551806ae70 0x7f551806d330 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5520006fd0 tx=0x7f5520008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:46.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.127+0000 7f5528ff9640 1 -- 172.21.15.40:0/2902275798 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f551c077200 con 0x7f552c06b710 2023-10-18T09:02:46.267 INFO:tasks.ceph.mds.a.smithi040.stderr:2023-10-18T09:02:46.265+0000 7f9d169b8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T09:02:46.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.321+0000 7f5532a44640 1 -- 172.21.15.40:0/2902275798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f552c06fb50 con 0x7f552c06b710 2023-10-18T09:02:46.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.324+0000 7f5528ff9640 1 -- 172.21.15.40:0/2902275798 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+16532 (secure 0 0 0) 0x7f551c07a820 con 0x7f552c06b710 2023-10-18T09:02:46.327 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:46.327 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":130,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:02:43.051244+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:02:39.531482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-18T09:02:40.812071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:02:46.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.326+0000 7f5532a44640 1 -- 172.21.15.40:0/2902275798 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f551806ae70 msgr2=0x7f551806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:46.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.326+0000 7f5532a44640 1 --2- 172.21.15.40:0/2902275798 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f551806ae70 0x7f551806d330 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5520006fd0 tx=0x7f5520008040 comp rx=0 tx=0).stop 2023-10-18T09:02:46.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.326+0000 7f5532a44640 1 -- 172.21.15.40:0/2902275798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f552c06b710 msgr2=0x7f552c1567b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:46.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.326+0000 7f5532a44640 1 --2- 172.21.15.40:0/2902275798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f552c06b710 0x7f552c1567b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f551c009850 tx=0x7f551c02de00 comp rx=0 tx=0).stop 2023-10-18T09:02:46.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.327+0000 7f5532a44640 1 -- 172.21.15.40:0/2902275798 shutdown_connections 2023-10-18T09:02:46.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.327+0000 7f5532a44640 1 --2- 172.21.15.40:0/2902275798 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f551806ae70 0x7f551806d330 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.327+0000 7f5532a44640 1 --2- 172.21.15.40:0/2902275798 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f552c151bd0 0x7f552c1481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.327+0000 7f5532a44640 1 --2- 172.21.15.40:0/2902275798 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f552c0b0bc0 0x7f552c14f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.327+0000 7f5532a44640 1 --2- 172.21.15.40:0/2902275798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f552c06b710 0x7f552c1567b0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.327+0000 7f5532a44640 1 -- 172.21.15.40:0/2902275798 >> 172.21.15.40:0/2902275798 conn(0x7f552c06cee0 msgr2=0x7f552c0cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:46.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.327+0000 7f5532a44640 1 -- 172.21.15.40:0/2902275798 shutdown_connections 2023-10-18T09:02:46.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.327+0000 7f5532a44640 1 -- 172.21.15.40:0/2902275798 wait complete. 2023-10-18T09:02:46.349 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T09:02:46.350 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:02:46.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.578+0000 7f0ca1d43640 1 -- 172.21.15.40:0/2104017877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c9c0b0630 msgr2=0x7f0c9c0c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:46.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.578+0000 7f0ca1d43640 1 --2- 172.21.15.40:0/2104017877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c9c0b0630 0x7f0c9c0c99c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c005fd0 tx=0x7f0c8c02be20 comp rx=0 tx=0).stop 2023-10-18T09:02:46.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.579+0000 7f0ca1d43640 1 -- 172.21.15.40:0/2104017877 shutdown_connections 2023-10-18T09:02:46.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.579+0000 7f0ca1d43640 1 --2- 172.21.15.40:0/2104017877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c9c0b0630 0x7f0c9c0c99c0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.579+0000 7f0ca1d43640 1 --2- 172.21.15.40:0/2104017877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c06d6f0 0x7f0c9c0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.579+0000 7f0ca1d43640 1 -- 172.21.15.40:0/2104017877 >> 172.21.15.40:0/2104017877 conn(0x7f0c9c069720 msgr2=0x7f0c9c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:46.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.579+0000 7f0ca1d43640 1 -- 172.21.15.40:0/2104017877 shutdown_connections 2023-10-18T09:02:46.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.579+0000 7f0ca1d43640 1 -- 172.21.15.40:0/2104017877 wait complete. 2023-10-18T09:02:46.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.581+0000 7f0ca1d43640 1 Processor -- start 2023-10-18T09:02:46.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.581+0000 7f0ca1d43640 1 -- start start 2023-10-18T09:02:46.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.582+0000 7f0ca1d43640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c06d6f0 0x7f0c9c146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.582+0000 7f0ca1d43640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c9c0b0630 0x7f0c9c1e5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.582+0000 7f0ca1d43640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c0c9f00 0x7f0c9c1539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.582+0000 7f0ca1d43640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c152f70 con 0x7f0c9c0b0630 2023-10-18T09:02:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.582+0000 7f0ca1d43640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c1e4db0 con 0x7f0c9c0c9f00 2023-10-18T09:02:46.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.582+0000 7f0ca1d43640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c1e41a0 con 0x7f0c9c06d6f0 2023-10-18T09:02:46.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.582+0000 7f0c93fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c9c0b0630 0x7f0c9c1e5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:46.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.582+0000 7f0c93fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c9c0b0630 0x7f0c9c1e5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54508/0 (socket says 172.21.15.40:54508) 2023-10-18T09:02:46.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.582+0000 7f0c93fff640 1 -- 172.21.15.40:0/1214310821 learned_addr learned my addr 172.21.15.40:0/1214310821 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:46.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.582+0000 7f0ca0d41640 1 --2- 172.21.15.40:0/1214310821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c06d6f0 0x7f0c9c146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:46.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.582+0000 7f0ca1542640 1 --2- 172.21.15.40:0/1214310821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c0c9f00 0x7f0c9c1539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:46.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.583+0000 7f0c93fff640 1 -- 172.21.15.40:0/1214310821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c06d6f0 msgr2=0x7f0c9c146c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:46.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.583+0000 7f0c93fff640 1 --2- 172.21.15.40:0/1214310821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c06d6f0 0x7f0c9c146c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.583+0000 7f0c93fff640 1 -- 172.21.15.40:0/1214310821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c0c9f00 msgr2=0x7f0c9c1539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:46.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.583+0000 7f0c93fff640 1 --2- 172.21.15.40:0/1214310821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c0c9f00 0x7f0c9c1539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.583+0000 7f0c93fff640 1 -- 172.21.15.40:0/1214310821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c009560 con 0x7f0c9c0b0630 2023-10-18T09:02:46.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.583+0000 7f0ca0d41640 1 --2- 172.21.15.40:0/1214310821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c06d6f0 0x7f0c9c146c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:46.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.583+0000 7f0ca1542640 1 --2- 172.21.15.40:0/1214310821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c0c9f00 0x7f0c9c1539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:46.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.583+0000 7f0c93fff640 1 --2- 172.21.15.40:0/1214310821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c9c0b0630 0x7f0c9c1e5bc0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c036040 tx=0x7f0c8c02daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:46.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.583+0000 7f0c917fa640 1 -- 172.21.15.40:0/1214310821 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c0035f0 con 0x7f0c9c0b0630 2023-10-18T09:02:46.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.583+0000 7f0c917fa640 1 -- 172.21.15.40:0/1214310821 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0c8c003840 con 0x7f0c9c0b0630 2023-10-18T09:02:46.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.583+0000 7f0c917fa640 1 -- 172.21.15.40:0/1214310821 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c003c30 con 0x7f0c9c0b0630 2023-10-18T09:02:46.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.583+0000 7f0ca1d43640 1 -- 172.21.15.40:0/1214310821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9c1e4480 con 0x7f0c9c0b0630 2023-10-18T09:02:46.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.584+0000 7f0ca1d43640 1 -- 172.21.15.40:0/1214310821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9c1e4a40 con 0x7f0c9c0b0630 2023-10-18T09:02:46.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.585+0000 7f0ca1d43640 1 -- 172.21.15.40:0/1214310821 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9c1e4f60 con 0x7f0c9c0b0630 2023-10-18T09:02:46.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.585+0000 7f0c917fa640 1 -- 172.21.15.40:0/1214310821 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0c8c03e450 con 0x7f0c9c0b0630 2023-10-18T09:02:46.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.586+0000 7f0c917fa640 1 --2- 172.21.15.40:0/1214310821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0c8806ae70 0x7f0c8806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:46.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.586+0000 7f0c917fa640 1 -- 172.21.15.40:0/1214310821 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c8c0ac160 con 0x7f0c9c0b0630 2023-10-18T09:02:46.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.587+0000 7f0ca0d41640 1 --2- 172.21.15.40:0/1214310821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0c8806ae70 0x7f0c8806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:46.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.588+0000 7f0ca0d41640 1 --2- 172.21.15.40:0/1214310821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0c8806ae70 0x7f0c8806d330 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0c98005fd0 tx=0x7f0c9800a4d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:46.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.591+0000 7f0c917fa640 1 -- 172.21.15.40:0/1214310821 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0c9c1e4f60 con 0x7f0c9c0b0630 2023-10-18T09:02:46.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.780+0000 7f0ca1d43640 1 -- 172.21.15.40:0/1214310821 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c9c0ad430 con 0x7f0c8806ae70 2023-10-18T09:02:46.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.802+0000 7f0c917fa640 1 -- 172.21.15.40:0/1214310821 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279691 (secure 0 0 0) 0x7f0c9c0ad430 con 0x7f0c8806ae70 2023-10-18T09:02:46.806 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:46.810 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:02:46.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.808+0000 7f0ca1d43640 1 -- 172.21.15.40:0/1214310821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0c8806ae70 msgr2=0x7f0c8806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:46.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.808+0000 7f0ca1d43640 1 --2- 172.21.15.40:0/1214310821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0c8806ae70 0x7f0c8806d330 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0c98005fd0 tx=0x7f0c9800a4d0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.808+0000 7f0ca1d43640 1 -- 172.21.15.40:0/1214310821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c9c0b0630 msgr2=0x7f0c9c1e5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:46.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.808+0000 7f0ca1d43640 1 --2- 172.21.15.40:0/1214310821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c9c0b0630 0x7f0c9c1e5bc0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c036040 tx=0x7f0c8c02daf0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.808+0000 7f0ca1d43640 1 -- 172.21.15.40:0/1214310821 shutdown_connections 2023-10-18T09:02:46.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.808+0000 7f0ca1d43640 1 --2- 172.21.15.40:0/1214310821 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0c8806ae70 0x7f0c8806d330 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.808+0000 7f0ca1d43640 1 --2- 172.21.15.40:0/1214310821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c0c9f00 0x7f0c9c1539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.808+0000 7f0ca1d43640 1 --2- 172.21.15.40:0/1214310821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c9c0b0630 0x7f0c9c1e5bc0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.808+0000 7f0ca1d43640 1 --2- 172.21.15.40:0/1214310821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c06d6f0 0x7f0c9c146c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.808+0000 7f0ca1d43640 1 -- 172.21.15.40:0/1214310821 >> 172.21.15.40:0/1214310821 conn(0x7f0c9c069720 msgr2=0x7f0c9c0abd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.809+0000 7f0ca1d43640 1 -- 172.21.15.40:0/1214310821 shutdown_connections 2023-10-18T09:02:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:46.809+0000 7f0ca1d43640 1 -- 172.21.15.40:0/1214310821 wait complete. 2023-10-18T09:02:46.831 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2023-10-18T09:02:45.059881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592666,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":198,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":150,"acting":150,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6789,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555710,"statfs":{"total":1600000000,"available":1593047973,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2298,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.765889"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365400+0000","last_change":"2023-10-18T09:02:42.080221+0000","last_active":"2023-10-18T09:02:43.365400+0000","last_peered":"2023-10-18T09:02:43.365400+0000","last_clean":"2023-10-18T09:02:43.365400+0000","last_became_active":"2023-10-18T09:02:42.079702+0000","last_became_peered":"2023-10-18T09:02:42.079702+0000","last_unstale":"2023-10-18T09:02:43.365400+0000","last_undegraded":"2023-10-18T09:02:43.365400+0000","last_fullsized":"2023-10-18T09:02:43.365400+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:35:08.016956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364295+0000","last_change":"2023-10-18T09:02:42.090937+0000","last_active":"2023-10-18T09:02:43.364295+0000","last_peered":"2023-10-18T09:02:43.364295+0000","last_clean":"2023-10-18T09:02:43.364295+0000","last_became_active":"2023-10-18T09:02:42.089232+0000","last_became_peered":"2023-10-18T09:02:42.089232+0000","last_unstale":"2023-10-18T09:02:43.364295+0000","last_undegraded":"2023-10-18T09:02:43.364295+0000","last_fullsized":"2023-10-18T09:02:43.364295+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:29:13.401772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363272+0000","last_change":"2023-10-18T09:02:42.080834+0000","last_active":"2023-10-18T09:02:43.363272+0000","last_peered":"2023-10-18T09:02:43.363272+0000","last_clean":"2023-10-18T09:02:43.363272+0000","last_became_active":"2023-10-18T09:02:42.080312+0000","last_became_peered":"2023-10-18T09:02:42.080312+0000","last_unstale":"2023-10-18T09:02:43.363272+0000","last_undegraded":"2023-10-18T09:02:43.363272+0000","last_fullsized":"2023-10-18T09:02:43.363272+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:55:45.693647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.060211+0000","last_change":"2023-10-18T09:02:42.068042+0000","last_active":"2023-10-18T09:02:43.060211+0000","last_peered":"2023-10-18T09:02:43.060211+0000","last_clean":"2023-10-18T09:02:43.060211+0000","last_became_active":"2023-10-18T09:02:42.067375+0000","last_became_peered":"2023-10-18T09:02:42.067375+0000","last_unstale":"2023-10-18T09:02:43.060211+0000","last_undegraded":"2023-10-18T09:02:43.060211+0000","last_fullsized":"2023-10-18T09:02:43.060211+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:23.596366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364061+0000","last_change":"2023-10-18T09:02:42.088051+0000","last_active":"2023-10-18T09:02:43.364061+0000","last_peered":"2023-10-18T09:02:43.364061+0000","last_clean":"2023-10-18T09:02:43.364061+0000","last_became_active":"2023-10-18T09:02:42.087079+0000","last_became_peered":"2023-10-18T09:02:42.087079+0000","last_unstale":"2023-10-18T09:02:43.364061+0000","last_undegraded":"2023-10-18T09:02:43.364061+0000","last_fullsized":"2023-10-18T09:02:43.364061+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:20:48.771067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364349+0000","last_change":"2023-10-18T09:02:42.087238+0000","last_active":"2023-10-18T09:02:43.364349+0000","last_peered":"2023-10-18T09:02:43.364349+0000","last_clean":"2023-10-18T09:02:43.364349+0000","last_became_active":"2023-10-18T09:02:42.086757+0000","last_became_peered":"2023-10-18T09:02:42.086757+0000","last_unstale":"2023-10-18T09:02:43.364349+0000","last_undegraded":"2023-10-18T09:02:43.364349+0000","last_fullsized":"2023-10-18T09:02:43.364349+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:04:00.233351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.060512+0000","last_change":"2023-10-18T09:02:42.091570+0000","last_active":"2023-10-18T09:02:43.060512+0000","last_peered":"2023-10-18T09:02:43.060512+0000","last_clean":"2023-10-18T09:02:43.060512+0000","last_became_active":"2023-10-18T09:02:42.091234+0000","last_became_peered":"2023-10-18T09:02:42.091234+0000","last_unstale":"2023-10-18T09:02:43.060512+0000","last_undegraded":"2023-10-18T09:02:43.060512+0000","last_fullsized":"2023-10-18T09:02:43.060512+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:08.005031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364708+0000","last_change":"2023-10-18T09:02:42.083599+0000","last_active":"2023-10-18T09:02:43.364708+0000","last_peered":"2023-10-18T09:02:43.364708+0000","last_clean":"2023-10-18T09:02:43.364708+0000","last_became_active":"2023-10-18T09:02:42.083003+0000","last_became_peered":"2023-10-18T09:02:42.083003+0000","last_unstale":"2023-10-18T09:02:43.364708+0000","last_undegraded":"2023-10-18T09:02:43.364708+0000","last_fullsized":"2023-10-18T09:02:43.364708+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:56:44.616338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364656+0000","last_change":"2023-10-18T09:02:42.088619+0000","last_active":"2023-10-18T09:02:43.364656+0000","last_peered":"2023-10-18T09:02:43.364656+0000","last_clean":"2023-10-18T09:02:43.364656+0000","last_became_active":"2023-10-18T09:02:42.085289+0000","last_became_peered":"2023-10-18T09:02:42.085289+0000","last_unstale":"2023-10-18T09:02:43.364656+0000","last_undegraded":"2023-10-18T09:02:43.364656+0000","last_fullsized":"2023-10-18T09:02:43.364656+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:39.491657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364490+0000","last_change":"2023-10-18T09:02:42.090410+0000","last_active":"2023-10-18T09:02:43.364490+0000","last_peered":"2023-10-18T09:02:43.364490+0000","last_clean":"2023-10-18T09:02:43.364490+0000","last_became_active":"2023-10-18T09:02:42.088799+0000","last_became_peered":"2023-10-18T09:02:42.088799+0000","last_unstale":"2023-10-18T09:02:43.364490+0000","last_undegraded":"2023-10-18T09:02:43.364490+0000","last_fullsized":"2023-10-18T09:02:43.364490+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:23.596366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364840+0000","last_change":"2023-10-18T09:02:42.089210+0000","last_active":"2023-10-18T09:02:43.364840+0000","last_peered":"2023-10-18T09:02:43.364840+0000","last_clean":"2023-10-18T09:02:43.364840+0000","last_became_active":"2023-10-18T09:02:42.085622+0000","last_became_peered":"2023-10-18T09:02:42.085622+0000","last_unstale":"2023-10-18T09:02:43.364840+0000","last_undegraded":"2023-10-18T09:02:43.364840+0000","last_fullsized":"2023-10-18T09:02:43.364840+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:28.892720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365089+0000","last_change":"2023-10-18T09:02:42.084409+0000","last_active":"2023-10-18T09:02:43.365089+0000","last_peered":"2023-10-18T09:02:43.365089+0000","last_clean":"2023-10-18T09:02:43.365089+0000","last_became_active":"2023-10-18T09:02:42.084040+0000","last_became_peered":"2023-10-18T09:02:42.084040+0000","last_unstale":"2023-10-18T09:02:43.365089+0000","last_undegraded":"2023-10-18T09:02:43.365089+0000","last_fullsized":"2023-10-18T09:02:43.365089+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:07:12.903518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363409+0000","last_change":"2023-10-18T09:02:42.083371+0000","last_active":"2023-10-18T09:02:43.363409+0000","last_peered":"2023-10-18T09:02:43.363409+0000","last_clean":"2023-10-18T09:02:43.363409+0000","last_became_active":"2023-10-18T09:02:42.079991+0000","last_became_peered":"2023-10-18T09:02:42.079991+0000","last_unstale":"2023-10-18T09:02:43.363409+0000","last_undegraded":"2023-10-18T09:02:43.363409+0000","last_fullsized":"2023-10-18T09:02:43.363409+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:02.686085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364895+0000","last_change":"2023-10-18T09:02:42.089481+0000","last_active":"2023-10-18T09:02:43.364895+0000","last_peered":"2023-10-18T09:02:43.364895+0000","last_clean":"2023-10-18T09:02:43.364895+0000","last_became_active":"2023-10-18T09:02:42.085532+0000","last_became_peered":"2023-10-18T09:02:42.085532+0000","last_unstale":"2023-10-18T09:02:43.364895+0000","last_undegraded":"2023-10-18T09:02:43.364895+0000","last_fullsized":"2023-10-18T09:02:43.364895+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:29.393784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364221+0000","last_change":"2023-10-18T09:02:42.087324+0000","last_active":"2023-10-18T09:02:43.364221+0000","last_peered":"2023-10-18T09:02:43.364221+0000","last_clean":"2023-10-18T09:02:43.364221+0000","last_became_active":"2023-10-18T09:02:42.086227+0000","last_became_peered":"2023-10-18T09:02:42.086227+0000","last_unstale":"2023-10-18T09:02:43.364221+0000","last_undegraded":"2023-10-18T09:02:43.364221+0000","last_fullsized":"2023-10-18T09:02:43.364221+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:44.776977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.060813+0000","last_change":"2023-10-18T09:02:42.075089+0000","last_active":"2023-10-18T09:02:43.060813+0000","last_peered":"2023-10-18T09:02:43.060813+0000","last_clean":"2023-10-18T09:02:43.060813+0000","last_became_active":"2023-10-18T09:02:42.074631+0000","last_became_peered":"2023-10-18T09:02:42.074631+0000","last_unstale":"2023-10-18T09:02:43.060813+0000","last_undegraded":"2023-10-18T09:02:43.060813+0000","last_fullsized":"2023-10-18T09:02:43.060813+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:29.393784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365359+0000","last_change":"2023-10-18T09:02:42.083732+0000","last_active":"2023-10-18T09:02:43.365359+0000","last_peered":"2023-10-18T09:02:43.365359+0000","last_clean":"2023-10-18T09:02:43.365359+0000","last_became_active":"2023-10-18T09:02:42.083320+0000","last_became_peered":"2023-10-18T09:02:42.083320+0000","last_unstale":"2023-10-18T09:02:43.365359+0000","last_undegraded":"2023-10-18T09:02:43.365359+0000","last_fullsized":"2023-10-18T09:02:43.365359+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:51.546758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.059314+0000","last_change":"2023-10-18T09:02:42.065219+0000","last_active":"2023-10-18T09:02:43.059314+0000","last_peered":"2023-10-18T09:02:43.059314+0000","last_clean":"2023-10-18T09:02:43.059314+0000","last_became_active":"2023-10-18T09:02:42.063676+0000","last_became_peered":"2023-10-18T09:02:42.063676+0000","last_unstale":"2023-10-18T09:02:43.059314+0000","last_undegraded":"2023-10-18T09:02:43.059314+0000","last_fullsized":"2023-10-18T09:02:43.059314+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:42:09.257183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363448+0000","last_change":"2023-10-18T09:02:42.086237+0000","last_active":"2023-10-18T09:02:43.363448+0000","last_peered":"2023-10-18T09:02:43.363448+0000","last_clean":"2023-10-18T09:02:43.363448+0000","last_became_active":"2023-10-18T09:02:42.085928+0000","last_became_peered":"2023-10-18T09:02:42.085928+0000","last_unstale":"2023-10-18T09:02:43.363448+0000","last_undegraded":"2023-10-18T09:02:43.363448+0000","last_fullsized":"2023-10-18T09:02:43.363448+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:01.446785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364671+0000","last_change":"2023-10-18T09:02:42.092619+0000","last_active":"2023-10-18T09:02:43.364671+0000","last_peered":"2023-10-18T09:02:43.364671+0000","last_clean":"2023-10-18T09:02:43.364671+0000","last_became_active":"2023-10-18T09:02:42.092298+0000","last_became_peered":"2023-10-18T09:02:42.092298+0000","last_unstale":"2023-10-18T09:02:43.364671+0000","last_undegraded":"2023-10-18T09:02:43.364671+0000","last_fullsized":"2023-10-18T09:02:43.364671+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:25:22.025388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363511+0000","last_change":"2023-10-18T09:02:42.091546+0000","last_active":"2023-10-18T09:02:43.363511+0000","last_peered":"2023-10-18T09:02:43.363511+0000","last_clean":"2023-10-18T09:02:43.363511+0000","last_became_active":"2023-10-18T09:02:42.091240+0000","last_became_peered":"2023-10-18T09:02:42.091240+0000","last_unstale":"2023-10-18T09:02:43.363511+0000","last_undegraded":"2023-10-18T09:02:43.363511+0000","last_fullsized":"2023-10-18T09:02:43.363511+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:04:37.072672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365003+0000","last_change":"2023-10-18T09:02:42.084270+0000","last_active":"2023-10-18T09:02:43.365003+0000","last_peered":"2023-10-18T09:02:43.365003+0000","last_clean":"2023-10-18T09:02:43.365003+0000","last_became_active":"2023-10-18T09:02:42.079899+0000","last_became_peered":"2023-10-18T09:02:42.079899+0000","last_unstale":"2023-10-18T09:02:43.365003+0000","last_undegraded":"2023-10-18T09:02:43.365003+0000","last_fullsized":"2023-10-18T09:02:43.365003+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:57:02.012639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"129'4","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364600+0000","last_change":"2023-10-18T09:02:41.324352+0000","last_active":"2023-10-18T09:02:43.364600+0000","last_peered":"2023-10-18T09:02:43.364600+0000","last_clean":"2023-10-18T09:02:43.364600+0000","last_became_active":"2023-10-18T09:02:41.324090+0000","last_became_peered":"2023-10-18T09:02:41.324090+0000","last_unstale":"2023-10-18T09:02:43.364600+0000","last_undegraded":"2023-10-18T09:02:43.364600+0000","last_fullsized":"2023-10-18T09:02:43.364600+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:22.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:40.275820+0000","last_change":"2023-10-18T09:02:40.275820+0000","last_active":"2023-10-18T09:02:40.275820+0000","last_peered":"2023-10-18T09:02:40.275820+0000","last_clean":"2023-10-18T09:02:40.275820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:40.275820+0000","last_undegraded":"2023-10-18T09:02:40.275820+0000","last_fullsized":"2023-10-18T09:02:40.275820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365702+0000","last_change":"2023-10-18T09:02:42.088381+0000","last_active":"2023-10-18T09:02:43.365702+0000","last_peered":"2023-10-18T09:02:43.365702+0000","last_clean":"2023-10-18T09:02:43.365702+0000","last_became_active":"2023-10-18T09:02:42.088134+0000","last_became_peered":"2023-10-18T09:02:42.088134+0000","last_unstale":"2023-10-18T09:02:43.365702+0000","last_undegraded":"2023-10-18T09:02:43.365702+0000","last_fullsized":"2023-10-18T09:02:43.365702+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:32:19.050992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"129'2","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365216+0000","last_change":"2023-10-18T09:02:41.325861+0000","last_active":"2023-10-18T09:02:43.365216+0000","last_peered":"2023-10-18T09:02:43.365216+0000","last_clean":"2023-10-18T09:02:43.365216+0000","last_became_active":"2023-10-18T09:02:41.325588+0000","last_became_peered":"2023-10-18T09:02:41.325588+0000","last_unstale":"2023-10-18T09:02:43.365216+0000","last_undegraded":"2023-10-18T09:02:43.365216+0000","last_fullsized":"2023-10-18T09:02:43.365216+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:15.138604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365379+0000","last_change":"2023-10-18T09:02:42.080067+0000","last_active":"2023-10-18T09:02:43.365379+0000","last_peered":"2023-10-18T09:02:43.365379+0000","last_clean":"2023-10-18T09:02:43.365379+0000","last_became_active":"2023-10-18T09:02:42.079804+0000","last_became_peered":"2023-10-18T09:02:42.079804+0000","last_unstale":"2023-10-18T09:02:43.365379+0000","last_undegraded":"2023-10-18T09:02:43.365379+0000","last_fullsized":"2023-10-18T09:02:43.365379+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:33.367613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"129'2","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365497+0000","last_change":"2023-10-18T09:02:41.323282+0000","last_active":"2023-10-18T09:02:43.365497+0000","last_peered":"2023-10-18T09:02:43.365497+0000","last_clean":"2023-10-18T09:02:43.365497+0000","last_became_active":"2023-10-18T09:02:41.323046+0000","last_became_peered":"2023-10-18T09:02:41.323046+0000","last_unstale":"2023-10-18T09:02:43.365497+0000","last_undegraded":"2023-10-18T09:02:43.365497+0000","last_fullsized":"2023-10-18T09:02:43.365497+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:08.235986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364441+0000","last_change":"2023-10-18T09:02:42.080475+0000","last_active":"2023-10-18T09:02:43.364441+0000","last_peered":"2023-10-18T09:02:43.364441+0000","last_clean":"2023-10-18T09:02:43.364441+0000","last_became_active":"2023-10-18T09:02:42.080086+0000","last_became_peered":"2023-10-18T09:02:42.080086+0000","last_unstale":"2023-10-18T09:02:43.364441+0000","last_undegraded":"2023-10-18T09:02:43.364441+0000","last_fullsized":"2023-10-18T09:02:43.364441+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:04:30.422371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"129'1","reported_seq":13,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363865+0000","last_change":"2023-10-18T09:02:41.322086+0000","last_active":"2023-10-18T09:02:43.363865+0000","last_peered":"2023-10-18T09:02:43.363865+0000","last_clean":"2023-10-18T09:02:43.363865+0000","last_became_active":"2023-10-18T09:02:41.321844+0000","last_became_peered":"2023-10-18T09:02:41.321844+0000","last_unstale":"2023-10-18T09:02:43.363865+0000","last_undegraded":"2023-10-18T09:02:43.363865+0000","last_fullsized":"2023-10-18T09:02:43.363865+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:01.502838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"129'1","reported_seq":13,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364723+0000","last_change":"2023-10-18T09:02:41.320152+0000","last_active":"2023-10-18T09:02:43.364723+0000","last_peered":"2023-10-18T09:02:43.364723+0000","last_clean":"2023-10-18T09:02:43.364723+0000","last_became_active":"2023-10-18T09:02:41.315903+0000","last_became_peered":"2023-10-18T09:02:41.315903+0000","last_unstale":"2023-10-18T09:02:43.364723+0000","last_undegraded":"2023-10-18T09:02:43.364723+0000","last_fullsized":"2023-10-18T09:02:43.364723+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:44:24.756584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.059053+0000","last_change":"2023-10-18T09:02:41.325209+0000","last_active":"2023-10-18T09:02:43.059053+0000","last_peered":"2023-10-18T09:02:43.059053+0000","last_clean":"2023-10-18T09:02:43.059053+0000","last_became_active":"2023-10-18T09:02:41.324895+0000","last_became_peered":"2023-10-18T09:02:41.324895+0000","last_unstale":"2023-10-18T09:02:43.059053+0000","last_undegraded":"2023-10-18T09:02:43.059053+0000","last_fullsized":"2023-10-18T09:02:43.059053+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:32.924193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364696+0000","last_change":"2023-10-18T09:02:42.078871+0000","last_active":"2023-10-18T09:02:43.364696+0000","last_peered":"2023-10-18T09:02:43.364696+0000","last_clean":"2023-10-18T09:02:43.364696+0000","last_became_active":"2023-10-18T09:02:42.078441+0000","last_became_peered":"2023-10-18T09:02:42.078441+0000","last_unstale":"2023-10-18T09:02:43.364696+0000","last_undegraded":"2023-10-18T09:02:43.364696+0000","last_fullsized":"2023-10-18T09:02:43.364696+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:16.154132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363877+0000","last_change":"2023-10-18T09:02:41.306939+0000","last_active":"2023-10-18T09:02:43.363877+0000","last_peered":"2023-10-18T09:02:43.363877+0000","last_clean":"2023-10-18T09:02:43.363877+0000","last_became_active":"2023-10-18T09:02:41.306708+0000","last_became_peered":"2023-10-18T09:02:41.306708+0000","last_unstale":"2023-10-18T09:02:43.363877+0000","last_undegraded":"2023-10-18T09:02:43.363877+0000","last_fullsized":"2023-10-18T09:02:43.363877+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:54.131748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364702+0000","last_change":"2023-10-18T09:02:42.090046+0000","last_active":"2023-10-18T09:02:43.364702+0000","last_peered":"2023-10-18T09:02:43.364702+0000","last_clean":"2023-10-18T09:02:43.364702+0000","last_became_active":"2023-10-18T09:02:42.087516+0000","last_became_peered":"2023-10-18T09:02:42.087516+0000","last_unstale":"2023-10-18T09:02:43.364702+0000","last_undegraded":"2023-10-18T09:02:43.364702+0000","last_fullsized":"2023-10-18T09:02:43.364702+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:08.005031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365822+0000","last_change":"2023-10-18T09:02:41.316687+0000","last_active":"2023-10-18T09:02:43.365822+0000","last_peered":"2023-10-18T09:02:43.365822+0000","last_clean":"2023-10-18T09:02:43.365822+0000","last_became_active":"2023-10-18T09:02:41.315644+0000","last_became_peered":"2023-10-18T09:02:41.315644+0000","last_unstale":"2023-10-18T09:02:43.365822+0000","last_undegraded":"2023-10-18T09:02:43.365822+0000","last_fullsized":"2023-10-18T09:02:43.365822+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:46.569258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364981+0000","last_change":"2023-10-18T09:02:42.085493+0000","last_active":"2023-10-18T09:02:43.364981+0000","last_peered":"2023-10-18T09:02:43.364981+0000","last_clean":"2023-10-18T09:02:43.364981+0000","last_became_active":"2023-10-18T09:02:42.084741+0000","last_became_peered":"2023-10-18T09:02:42.084741+0000","last_unstale":"2023-10-18T09:02:43.364981+0000","last_undegraded":"2023-10-18T09:02:43.364981+0000","last_fullsized":"2023-10-18T09:02:43.364981+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:47.432868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:40.275820+0000","last_change":"2023-10-18T09:02:40.275820+0000","last_active":"2023-10-18T09:02:40.275820+0000","last_peered":"2023-10-18T09:02:40.275820+0000","last_clean":"2023-10-18T09:02:40.275820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:40.275820+0000","last_undegraded":"2023-10-18T09:02:40.275820+0000","last_fullsized":"2023-10-18T09:02:40.275820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.058194+0000","last_change":"2023-10-18T09:02:42.080269+0000","last_active":"2023-10-18T09:02:43.058194+0000","last_peered":"2023-10-18T09:02:43.058194+0000","last_clean":"2023-10-18T09:02:43.058194+0000","last_became_active":"2023-10-18T09:02:42.079964+0000","last_became_peered":"2023-10-18T09:02:42.079964+0000","last_unstale":"2023-10-18T09:02:43.058194+0000","last_undegraded":"2023-10-18T09:02:43.058194+0000","last_fullsized":"2023-10-18T09:02:43.058194+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:01.446785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:40.275820+0000","last_change":"2023-10-18T09:02:40.275820+0000","last_active":"2023-10-18T09:02:40.275820+0000","last_peered":"2023-10-18T09:02:40.275820+0000","last_clean":"2023-10-18T09:02:40.275820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:40.275820+0000","last_undegraded":"2023-10-18T09:02:40.275820+0000","last_fullsized":"2023-10-18T09:02:40.275820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"129'6","reported_seq":18,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365159+0000","last_change":"2023-10-18T09:02:41.323868+0000","last_active":"2023-10-18T09:02:43.365159+0000","last_peered":"2023-10-18T09:02:43.365159+0000","last_clean":"2023-10-18T09:02:43.365159+0000","last_became_active":"2023-10-18T09:02:41.323344+0000","last_became_peered":"2023-10-18T09:02:41.323344+0000","last_unstale":"2023-10-18T09:02:43.365159+0000","last_undegraded":"2023-10-18T09:02:43.365159+0000","last_fullsized":"2023-10-18T09:02:43.365159+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:17.988078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"129'2","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364821+0000","last_change":"2023-10-18T09:02:41.325268+0000","last_active":"2023-10-18T09:02:43.364821+0000","last_peered":"2023-10-18T09:02:43.364821+0000","last_clean":"2023-10-18T09:02:43.364821+0000","last_became_active":"2023-10-18T09:02:41.325032+0000","last_became_peered":"2023-10-18T09:02:41.325032+0000","last_unstale":"2023-10-18T09:02:43.364821+0000","last_undegraded":"2023-10-18T09:02:43.364821+0000","last_fullsized":"2023-10-18T09:02:43.364821+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:01:27.410695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363553+0000","last_change":"2023-10-18T09:02:42.089943+0000","last_active":"2023-10-18T09:02:43.363553+0000","last_peered":"2023-10-18T09:02:43.363553+0000","last_clean":"2023-10-18T09:02:43.363553+0000","last_became_active":"2023-10-18T09:02:42.089539+0000","last_became_peered":"2023-10-18T09:02:42.089539+0000","last_unstale":"2023-10-18T09:02:43.363553+0000","last_undegraded":"2023-10-18T09:02:43.363553+0000","last_fullsized":"2023-10-18T09:02:43.363553+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:10:42.282131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363733+0000","last_change":"2023-10-18T09:02:41.320186+0000","last_active":"2023-10-18T09:02:43.363733+0000","last_peered":"2023-10-18T09:02:43.363733+0000","last_clean":"2023-10-18T09:02:43.363733+0000","last_became_active":"2023-10-18T09:02:41.318713+0000","last_became_peered":"2023-10-18T09:02:41.318713+0000","last_unstale":"2023-10-18T09:02:43.363733+0000","last_undegraded":"2023-10-18T09:02:43.363733+0000","last_fullsized":"2023-10-18T09:02:43.363733+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:55.396520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365550+0000","last_change":"2023-10-18T09:02:42.079283+0000","last_active":"2023-10-18T09:02:43.365550+0000","last_peered":"2023-10-18T09:02:43.365550+0000","last_clean":"2023-10-18T09:02:43.365550+0000","last_became_active":"2023-10-18T09:02:42.078582+0000","last_became_peered":"2023-10-18T09:02:42.078582+0000","last_unstale":"2023-10-18T09:02:43.365550+0000","last_undegraded":"2023-10-18T09:02:43.365550+0000","last_fullsized":"2023-10-18T09:02:43.365550+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:56:15.946301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365110+0000","last_change":"2023-10-18T09:02:41.325004+0000","last_active":"2023-10-18T09:02:43.365110+0000","last_peered":"2023-10-18T09:02:43.365110+0000","last_clean":"2023-10-18T09:02:43.365110+0000","last_became_active":"2023-10-18T09:02:41.324569+0000","last_became_peered":"2023-10-18T09:02:41.324569+0000","last_unstale":"2023-10-18T09:02:43.365110+0000","last_undegraded":"2023-10-18T09:02:43.365110+0000","last_fullsized":"2023-10-18T09:02:43.365110+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:08.235986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364615+0000","last_change":"2023-10-18T09:02:42.082257+0000","last_active":"2023-10-18T09:02:43.364615+0000","last_peered":"2023-10-18T09:02:43.364615+0000","last_clean":"2023-10-18T09:02:43.364615+0000","last_became_active":"2023-10-18T09:02:42.081448+0000","last_became_peered":"2023-10-18T09:02:42.081448+0000","last_unstale":"2023-10-18T09:02:43.364615+0000","last_undegraded":"2023-10-18T09:02:43.364615+0000","last_fullsized":"2023-10-18T09:02:43.364615+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:48:22.448169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"129'8","reported_seq":20,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363710+0000","last_change":"2023-10-18T09:02:41.314679+0000","last_active":"2023-10-18T09:02:43.363710+0000","last_peered":"2023-10-18T09:02:43.363710+0000","last_clean":"2023-10-18T09:02:43.363710+0000","last_became_active":"2023-10-18T09:02:41.314090+0000","last_became_peered":"2023-10-18T09:02:41.314090+0000","last_unstale":"2023-10-18T09:02:43.363710+0000","last_undegraded":"2023-10-18T09:02:43.363710+0000","last_fullsized":"2023-10-18T09:02:43.363710+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:48.824395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.058507+0000","last_change":"2023-10-18T09:02:42.079433+0000","last_active":"2023-10-18T09:02:43.058507+0000","last_peered":"2023-10-18T09:02:43.058507+0000","last_clean":"2023-10-18T09:02:43.058507+0000","last_became_active":"2023-10-18T09:02:42.079132+0000","last_became_peered":"2023-10-18T09:02:42.079132+0000","last_unstale":"2023-10-18T09:02:43.058507+0000","last_undegraded":"2023-10-18T09:02:43.058507+0000","last_fullsized":"2023-10-18T09:02:43.058507+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:20:48.771067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"129'2","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365515+0000","last_change":"2023-10-18T09:02:41.326713+0000","last_active":"2023-10-18T09:02:43.365515+0000","last_peered":"2023-10-18T09:02:43.365515+0000","last_clean":"2023-10-18T09:02:43.365515+0000","last_became_active":"2023-10-18T09:02:41.326367+0000","last_became_peered":"2023-10-18T09:02:41.326367+0000","last_unstale":"2023-10-18T09:02:43.365515+0000","last_undegraded":"2023-10-18T09:02:43.365515+0000","last_fullsized":"2023-10-18T09:02:43.365515+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:36:50.180955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364226+0000","last_change":"2023-10-18T09:02:42.070109+0000","last_active":"2023-10-18T09:02:43.364226+0000","last_peered":"2023-10-18T09:02:43.364226+0000","last_clean":"2023-10-18T09:02:43.364226+0000","last_became_active":"2023-10-18T09:02:42.069438+0000","last_became_peered":"2023-10-18T09:02:42.069438+0000","last_unstale":"2023-10-18T09:02:43.364226+0000","last_undegraded":"2023-10-18T09:02:43.364226+0000","last_fullsized":"2023-10-18T09:02:43.364226+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:40:40.941852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"129'1","reported_seq":13,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.059690+0000","last_change":"2023-10-18T09:02:41.321101+0000","last_active":"2023-10-18T09:02:43.059690+0000","last_peered":"2023-10-18T09:02:43.059690+0000","last_clean":"2023-10-18T09:02:43.059690+0000","last_became_active":"2023-10-18T09:02:41.320839+0000","last_became_peered":"2023-10-18T09:02:41.320839+0000","last_unstale":"2023-10-18T09:02:43.059690+0000","last_undegraded":"2023-10-18T09:02:43.059690+0000","last_fullsized":"2023-10-18T09:02:43.059690+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:44:24.756584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:41.291348+0000","last_change":"2023-10-18T09:02:41.291348+0000","last_active":"2023-10-18T09:02:41.291348+0000","last_peered":"2023-10-18T09:02:41.291348+0000","last_clean":"2023-10-18T09:02:41.291348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:41.291348+0000","last_undegraded":"2023-10-18T09:02:41.291348+0000","last_fullsized":"2023-10-18T09:02:41.291348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365106+0000","last_change":"2023-10-18T09:02:41.323827+0000","last_active":"2023-10-18T09:02:43.365106+0000","last_peered":"2023-10-18T09:02:43.365106+0000","last_clean":"2023-10-18T09:02:43.365106+0000","last_became_active":"2023-10-18T09:02:41.323562+0000","last_became_peered":"2023-10-18T09:02:41.323562+0000","last_unstale":"2023-10-18T09:02:43.365106+0000","last_undegraded":"2023-10-18T09:02:43.365106+0000","last_fullsized":"2023-10-18T09:02:43.365106+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:06.471711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365164+0000","last_change":"2023-10-18T09:02:42.085887+0000","last_active":"2023-10-18T09:02:43.365164+0000","last_peered":"2023-10-18T09:02:43.365164+0000","last_clean":"2023-10-18T09:02:43.365164+0000","last_became_active":"2023-10-18T09:02:42.083398+0000","last_became_peered":"2023-10-18T09:02:42.083398+0000","last_unstale":"2023-10-18T09:02:43.365164+0000","last_undegraded":"2023-10-18T09:02:43.365164+0000","last_fullsized":"2023-10-18T09:02:43.365164+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:59.346182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"129'2","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364839+0000","last_change":"2023-10-18T09:02:41.326500+0000","last_active":"2023-10-18T09:02:43.364839+0000","last_peered":"2023-10-18T09:02:43.364839+0000","last_clean":"2023-10-18T09:02:43.364839+0000","last_became_active":"2023-10-18T09:02:41.326167+0000","last_became_peered":"2023-10-18T09:02:41.326167+0000","last_unstale":"2023-10-18T09:02:43.364839+0000","last_undegraded":"2023-10-18T09:02:43.364839+0000","last_fullsized":"2023-10-18T09:02:43.364839+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:32.924193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364619+0000","last_change":"2023-10-18T09:02:42.086551+0000","last_active":"2023-10-18T09:02:43.364619+0000","last_peered":"2023-10-18T09:02:43.364619+0000","last_clean":"2023-10-18T09:02:43.364619+0000","last_became_active":"2023-10-18T09:02:42.086260+0000","last_became_peered":"2023-10-18T09:02:42.086260+0000","last_unstale":"2023-10-18T09:02:43.364619+0000","last_undegraded":"2023-10-18T09:02:43.364619+0000","last_fullsized":"2023-10-18T09:02:43.364619+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:47.246408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363679+0000","last_change":"2023-10-18T09:02:41.314198+0000","last_active":"2023-10-18T09:02:43.363679+0000","last_peered":"2023-10-18T09:02:43.363679+0000","last_clean":"2023-10-18T09:02:43.363679+0000","last_became_active":"2023-10-18T09:02:41.313600+0000","last_became_peered":"2023-10-18T09:02:41.313600+0000","last_unstale":"2023-10-18T09:02:43.363679+0000","last_undegraded":"2023-10-18T09:02:43.363679+0000","last_fullsized":"2023-10-18T09:02:43.363679+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:03:35.850431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363605+0000","last_change":"2023-10-18T09:02:42.080339+0000","last_active":"2023-10-18T09:02:43.363605+0000","last_peered":"2023-10-18T09:02:43.363605+0000","last_clean":"2023-10-18T09:02:43.363605+0000","last_became_active":"2023-10-18T09:02:42.079566+0000","last_became_peered":"2023-10-18T09:02:42.079566+0000","last_unstale":"2023-10-18T09:02:43.363605+0000","last_undegraded":"2023-10-18T09:02:43.363605+0000","last_fullsized":"2023-10-18T09:02:43.363605+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:41.023294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364126+0000","last_change":"2023-10-18T09:02:41.322158+0000","last_active":"2023-10-18T09:02:43.364126+0000","last_peered":"2023-10-18T09:02:43.364126+0000","last_clean":"2023-10-18T09:02:43.364126+0000","last_became_active":"2023-10-18T09:02:41.321849+0000","last_became_peered":"2023-10-18T09:02:41.321849+0000","last_unstale":"2023-10-18T09:02:43.364126+0000","last_undegraded":"2023-10-18T09:02:43.364126+0000","last_fullsized":"2023-10-18T09:02:43.364126+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:19:22.924072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.061109+0000","last_change":"2023-10-18T09:02:42.090016+0000","last_active":"2023-10-18T09:02:43.061109+0000","last_peered":"2023-10-18T09:02:43.061109+0000","last_clean":"2023-10-18T09:02:43.061109+0000","last_became_active":"2023-10-18T09:02:42.089666+0000","last_became_peered":"2023-10-18T09:02:42.089666+0000","last_unstale":"2023-10-18T09:02:43.061109+0000","last_undegraded":"2023-10-18T09:02:43.061109+0000","last_fullsized":"2023-10-18T09:02:43.061109+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:51.546758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365650+0000","last_change":"2023-10-18T09:02:41.321706+0000","last_active":"2023-10-18T09:02:43.365650+0000","last_peered":"2023-10-18T09:02:43.365650+0000","last_clean":"2023-10-18T09:02:43.365650+0000","last_became_active":"2023-10-18T09:02:41.315780+0000","last_became_peered":"2023-10-18T09:02:41.315780+0000","last_unstale":"2023-10-18T09:02:43.365650+0000","last_undegraded":"2023-10-18T09:02:43.365650+0000","last_fullsized":"2023-10-18T09:02:43.365650+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:39:26.937813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364379+0000","last_change":"2023-10-18T09:02:42.086641+0000","last_active":"2023-10-18T09:02:43.364379+0000","last_peered":"2023-10-18T09:02:43.364379+0000","last_clean":"2023-10-18T09:02:43.364379+0000","last_became_active":"2023-10-18T09:02:42.082674+0000","last_became_peered":"2023-10-18T09:02:42.082674+0000","last_unstale":"2023-10-18T09:02:43.364379+0000","last_undegraded":"2023-10-18T09:02:43.364379+0000","last_fullsized":"2023-10-18T09:02:43.364379+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:42:09.257183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"129'2","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364473+0000","last_change":"2023-10-18T09:02:41.323424+0000","last_active":"2023-10-18T09:02:43.364473+0000","last_peered":"2023-10-18T09:02:43.364473+0000","last_clean":"2023-10-18T09:02:43.364473+0000","last_became_active":"2023-10-18T09:02:41.323104+0000","last_became_peered":"2023-10-18T09:02:41.323104+0000","last_unstale":"2023-10-18T09:02:43.364473+0000","last_undegraded":"2023-10-18T09:02:43.364473+0000","last_fullsized":"2023-10-18T09:02:43.364473+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:41.805471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363723+0000","last_change":"2023-10-18T09:02:42.085022+0000","last_active":"2023-10-18T09:02:43.363723+0000","last_peered":"2023-10-18T09:02:43.363723+0000","last_clean":"2023-10-18T09:02:43.363723+0000","last_became_active":"2023-10-18T09:02:42.084618+0000","last_became_peered":"2023-10-18T09:02:42.084618+0000","last_unstale":"2023-10-18T09:02:43.363723+0000","last_undegraded":"2023-10-18T09:02:43.363723+0000","last_fullsized":"2023-10-18T09:02:43.363723+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:56:57.656254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364950+0000","last_change":"2023-10-18T09:02:41.326389+0000","last_active":"2023-10-18T09:02:43.364950+0000","last_peered":"2023-10-18T09:02:43.364950+0000","last_clean":"2023-10-18T09:02:43.364950+0000","last_became_active":"2023-10-18T09:02:41.325910+0000","last_became_peered":"2023-10-18T09:02:41.325910+0000","last_unstale":"2023-10-18T09:02:43.364950+0000","last_undegraded":"2023-10-18T09:02:43.364950+0000","last_fullsized":"2023-10-18T09:02:43.364950+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:14:46.185906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363755+0000","last_change":"2023-10-18T09:02:42.089791+0000","last_active":"2023-10-18T09:02:43.363755+0000","last_peered":"2023-10-18T09:02:43.363755+0000","last_clean":"2023-10-18T09:02:43.363755+0000","last_became_active":"2023-10-18T09:02:42.089011+0000","last_became_peered":"2023-10-18T09:02:42.089011+0000","last_unstale":"2023-10-18T09:02:43.363755+0000","last_undegraded":"2023-10-18T09:02:43.363755+0000","last_fullsized":"2023-10-18T09:02:43.363755+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:14:37.134014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363549+0000","last_change":"2023-10-18T09:02:41.324279+0000","last_active":"2023-10-18T09:02:43.363549+0000","last_peered":"2023-10-18T09:02:43.363549+0000","last_clean":"2023-10-18T09:02:43.363549+0000","last_became_active":"2023-10-18T09:02:41.324023+0000","last_became_peered":"2023-10-18T09:02:41.324023+0000","last_unstale":"2023-10-18T09:02:43.363549+0000","last_undegraded":"2023-10-18T09:02:43.363549+0000","last_fullsized":"2023-10-18T09:02:43.363549+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:46.417340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.058846+0000","last_change":"2023-10-18T09:02:42.063326+0000","last_active":"2023-10-18T09:02:43.058846+0000","last_peered":"2023-10-18T09:02:43.058846+0000","last_clean":"2023-10-18T09:02:43.058846+0000","last_became_active":"2023-10-18T09:02:42.062523+0000","last_became_peered":"2023-10-18T09:02:42.062523+0000","last_unstale":"2023-10-18T09:02:43.058846+0000","last_undegraded":"2023-10-18T09:02:43.058846+0000","last_fullsized":"2023-10-18T09:02:43.058846+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:29:13.401772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.061308+0000","last_change":"2023-10-18T09:02:41.323581+0000","last_active":"2023-10-18T09:02:43.061308+0000","last_peered":"2023-10-18T09:02:43.061308+0000","last_clean":"2023-10-18T09:02:43.061308+0000","last_became_active":"2023-10-18T09:02:41.322732+0000","last_became_peered":"2023-10-18T09:02:41.322732+0000","last_unstale":"2023-10-18T09:02:43.061308+0000","last_undegraded":"2023-10-18T09:02:43.061308+0000","last_fullsized":"2023-10-18T09:02:43.061308+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:06.471711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364977+0000","last_change":"2023-10-18T09:02:42.082002+0000","last_active":"2023-10-18T09:02:43.364977+0000","last_peered":"2023-10-18T09:02:43.364977+0000","last_clean":"2023-10-18T09:02:43.364977+0000","last_became_active":"2023-10-18T09:02:42.080759+0000","last_became_peered":"2023-10-18T09:02:42.080759+0000","last_unstale":"2023-10-18T09:02:43.364977+0000","last_undegraded":"2023-10-18T09:02:43.364977+0000","last_fullsized":"2023-10-18T09:02:43.364977+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:19.783453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365760+0000","last_change":"2023-10-18T09:02:41.324636+0000","last_active":"2023-10-18T09:02:43.365760+0000","last_peered":"2023-10-18T09:02:43.365760+0000","last_clean":"2023-10-18T09:02:43.365760+0000","last_became_active":"2023-10-18T09:02:41.324274+0000","last_became_peered":"2023-10-18T09:02:41.324274+0000","last_unstale":"2023-10-18T09:02:43.365760+0000","last_undegraded":"2023-10-18T09:02:43.365760+0000","last_fullsized":"2023-10-18T09:02:43.365760+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:59.217823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365321+0000","last_change":"2023-10-18T09:02:42.085172+0000","last_active":"2023-10-18T09:02:43.365321+0000","last_peered":"2023-10-18T09:02:43.365321+0000","last_clean":"2023-10-18T09:02:43.365321+0000","last_became_active":"2023-10-18T09:02:42.083392+0000","last_became_peered":"2023-10-18T09:02:42.083392+0000","last_unstale":"2023-10-18T09:02:43.365321+0000","last_undegraded":"2023-10-18T09:02:43.365321+0000","last_fullsized":"2023-10-18T09:02:43.365321+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:49.808515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.059983+0000","last_change":"2023-10-18T09:02:41.323775+0000","last_active":"2023-10-18T09:02:43.059983+0000","last_peered":"2023-10-18T09:02:43.059983+0000","last_clean":"2023-10-18T09:02:43.059983+0000","last_became_active":"2023-10-18T09:02:41.319998+0000","last_became_peered":"2023-10-18T09:02:41.319998+0000","last_unstale":"2023-10-18T09:02:43.059983+0000","last_undegraded":"2023-10-18T09:02:43.059983+0000","last_fullsized":"2023-10-18T09:02:43.059983+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:22.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365325+0000","last_change":"2023-10-18T09:02:42.086084+0000","last_active":"2023-10-18T09:02:43.365325+0000","last_peered":"2023-10-18T09:02:43.365325+0000","last_clean":"2023-10-18T09:02:43.365325+0000","last_became_active":"2023-10-18T09:02:42.083915+0000","last_became_peered":"2023-10-18T09:02:42.083915+0000","last_unstale":"2023-10-18T09:02:43.365325+0000","last_undegraded":"2023-10-18T09:02:43.365325+0000","last_fullsized":"2023-10-18T09:02:43.365325+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:42.119673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:40.275820+0000","last_change":"2023-10-18T09:02:40.275820+0000","last_active":"2023-10-18T09:02:40.275820+0000","last_peered":"2023-10-18T09:02:40.275820+0000","last_clean":"2023-10-18T09:02:40.275820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:40.275820+0000","last_undegraded":"2023-10-18T09:02:40.275820+0000","last_fullsized":"2023-10-18T09:02:40.275820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.058820+0000","last_change":"2023-10-18T09:02:42.064831+0000","last_active":"2023-10-18T09:02:43.058820+0000","last_peered":"2023-10-18T09:02:43.058820+0000","last_clean":"2023-10-18T09:02:43.058820+0000","last_became_active":"2023-10-18T09:02:42.062648+0000","last_became_peered":"2023-10-18T09:02:42.062648+0000","last_unstale":"2023-10-18T09:02:43.058820+0000","last_undegraded":"2023-10-18T09:02:43.058820+0000","last_fullsized":"2023-10-18T09:02:43.058820+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:44.776977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365434+0000","last_change":"2023-10-18T09:02:41.325536+0000","last_active":"2023-10-18T09:02:43.365434+0000","last_peered":"2023-10-18T09:02:43.365434+0000","last_clean":"2023-10-18T09:02:43.365434+0000","last_became_active":"2023-10-18T09:02:41.325256+0000","last_became_peered":"2023-10-18T09:02:41.325256+0000","last_unstale":"2023-10-18T09:02:43.365434+0000","last_undegraded":"2023-10-18T09:02:43.365434+0000","last_fullsized":"2023-10-18T09:02:43.365434+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:27:06.010665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364344+0000","last_change":"2023-10-18T09:02:42.084253+0000","last_active":"2023-10-18T09:02:43.364344+0000","last_peered":"2023-10-18T09:02:43.364344+0000","last_clean":"2023-10-18T09:02:43.364344+0000","last_became_active":"2023-10-18T09:02:42.083866+0000","last_became_peered":"2023-10-18T09:02:42.083866+0000","last_unstale":"2023-10-18T09:02:43.364344+0000","last_undegraded":"2023-10-18T09:02:43.364344+0000","last_fullsized":"2023-10-18T09:02:43.364344+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:47:59.013663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-18T09:02:40.275820+0000","last_change":"2023-10-18T09:02:40.275820+0000","last_active":"2023-10-18T09:02:40.275820+0000","last_peered":"2023-10-18T09:02:40.275820+0000","last_clean":"2023-10-18T09:02:40.275820+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:02:40.275820+0000","last_undegraded":"2023-10-18T09:02:40.275820+0000","last_fullsized":"2023-10-18T09:02:40.275820+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":284,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-18T09:02:36.378804+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:02:36.378804+0000","last_peered":"2023-10-18T09:02:36.378804+0000","last_clean":"2023-10-18T09:02:36.378804+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:02:36.378804+0000","last_undegraded":"2023-10-18T09:02:36.378804+0000","last_fullsized":"2023-10-18T09:02:36.378804+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":94,"acting":94,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2298,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":54,"acting":54,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1262,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194049,"statfs":{"total":200000000,"available":198707098,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:02:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1240000000000001,"15min":1.036},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.60999999999999999},"max":{"1min":1.6040000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.129,"15min":1.046},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4450000000000001,"5min":1.97,"15min":1.97},"last":0.70999999999999996}]},{"osd":1,"last update":"Wed Oct 18 09:02:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.159,"15min":1.153},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.61799999999999999},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":1.357},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.151,"15min":1.129},"min":{"1min":0.75,"5min":0.61099999999999999,"15min":0.58799999999999997},"max":{"1min":1.504,"5min":1.7450000000000001,"15min":1.7989999999999999},"last":0.97599999999999998}]},{"osd":2,"last update":"Wed Oct 18 09:02:18 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3300000000000001,"15min":1.258},"min":{"1min":0.85199999999999998,"5min":0.78600000000000003,"15min":0.65400000000000003},"max":{"1min":1.7490000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.385},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.284,"15min":1.218},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":1.218}]},{"osd":3,"last update":"Wed Oct 18 09:02:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.296,"15min":1.2709999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.3740000000000001,"15min":1.333},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6990000000000001,"5min":1.873,"15min":1.9930000000000001},"last":1.2689999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:02:18 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.256,"15min":1.1950000000000001},"min":{"1min":0.66000000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.923,"5min":1.923,"15min":2.1070000000000002},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.345,"5min":1.3220000000000001,"15min":1.3799999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.5169999999999999},"last":1.488}]},{"osd":5,"last update":"Wed Oct 18 09:02:18 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.353,"15min":1.353},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.77,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.4319999999999999,"15min":1.4239999999999999},"min":{"1min":0.748,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.956,"5min":2.0259999999999998,"15min":2.0270000000000001},"last":1.3280000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:02:18 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.498,"15min":1.472},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.3959999999999999},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.4890000000000001,"15min":1.508},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.607}]}]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":711,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194601,"statfs":{"total":200000000,"available":199271647,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:02:10 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.242,"15min":1.1299999999999999},"min":{"1min":0.86199999999999999,"5min":0.623,"15min":0.60299999999999998},"max":{"1min":1.508,"5min":1.651,"15min":1.6850000000000001},"last":1.52},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.351,"15min":1.268},"min":{"1min":0.98399999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.6950000000000001,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.5960000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:02:10 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.121,"15min":1.0940000000000001},"min":{"1min":0.79200000000000004,"5min":0.65600000000000003,"15min":0.63800000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.246,"15min":1.2010000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.776,"5min":1.962,"15min":2.9609999999999999},"last":1.0529999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:02:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.1950000000000001,"15min":1.1779999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.5129999999999999,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.23,"5min":1.2609999999999999,"15min":1.2130000000000001},"min":{"1min":0.873,"5min":0.873,"15min":0.66300000000000003},"max":{"1min":1.5980000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.80000000000000004}]},{"osd":3,"last update":"Wed Oct 18 09:02:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2849999999999999,"15min":1.2649999999999999},"min":{"1min":0.81000000000000005,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.645,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.425},{"interface":"front","average":{"1min":1.345,"5min":1.292,"15min":1.2410000000000001},"min":{"1min":1.107,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.706,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.0940000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:02:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.2490000000000001,"15min":1.2},"min":{"1min":0.877,"5min":0.625,"15min":0.625},"max":{"1min":2.0230000000000001,"5min":2.444,"15min":2.444},"last":1.377},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.294,"15min":1.2789999999999999},"min":{"1min":0.97399999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.6859999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.268}]},{"osd":5,"last update":"Wed Oct 18 09:02:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4319999999999999,"15min":1.3779999999999999},"min":{"1min":1.1399999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.9670000000000001,"5min":2.032,"15min":2.032},"last":1.49},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.3,"15min":1.2470000000000001},"min":{"1min":0.98999999999999999,"5min":0.629,"15min":0.49199999999999999},"max":{"1min":1.6699999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.1990000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:02:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.417,"15min":1.4039999999999999},"min":{"1min":1.034,"5min":0.73999999999999999,"15min":0.61899999999999999},"max":{"1min":1.8029999999999999,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.466},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5249999999999999,"15min":1.528},"min":{"1min":1.2310000000000001,"5min":0.83099999999999996,"15min":0.68500000000000005},"max":{"1min":2.0800000000000001,"5min":2.3140000000000001,"15min":2.794},"last":1.6299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574943,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194601,"statfs":{"total":200000000,"available":199272229,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:02:17 2023","interfaces":[{"interface":"back","average":{"1min":1.6779999999999999,"5min":1.448,"15min":1.276},"min":{"1min":0.83099999999999996,"5min":0.68300000000000005,"15min":0.63400000000000001},"max":{"1min":9.609,"5min":9.609,"15min":9.609},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.333,"15min":1.3049999999999999},"min":{"1min":0.80000000000000004,"5min":0.71999999999999997,"15min":0.60799999999999998},"max":{"1min":1.623,"5min":5.3029999999999999,"15min":5.3029999999999999},"last":1.3560000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:02:17 2023","interfaces":[{"interface":"back","average":{"1min":1.857,"5min":1.345,"15min":1.2589999999999999},"min":{"1min":0.747,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":11.68,"5min":11.68,"15min":11.68},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2310000000000001,"15min":1.2390000000000001},"min":{"1min":0.97599999999999998,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.885,"5min":2.367,"15min":3.7050000000000001},"last":0.93200000000000005}]},{"osd":2,"last update":"Wed Oct 18 09:02:17 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2470000000000001,"15min":1.2609999999999999},"min":{"1min":0.79300000000000004,"5min":0.78100000000000003,"15min":0.755},"max":{"1min":1.732,"5min":2.0350000000000001,"15min":2.2090000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.262,"15min":1.282},"min":{"1min":0.95599999999999996,"5min":0.75900000000000001,"15min":0.72399999999999998},"max":{"1min":1.5549999999999999,"5min":2.3069999999999999,"15min":3.8180000000000001},"last":1.1339999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:02:17 2023","interfaces":[{"interface":"back","average":{"1min":1.7629999999999999,"5min":1.3500000000000001,"15min":1.2749999999999999},"min":{"1min":0.754,"5min":0.68799999999999994,"15min":0.65400000000000003},"max":{"1min":11.797000000000001,"5min":11.797000000000001,"15min":11.797000000000001},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.365,"15min":1.339},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.6699999999999999,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.3819999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:02:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3740000000000001,"15min":1.4039999999999999},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.75900000000000001},"max":{"1min":2.0680000000000001,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.298,"15min":1.452},"min":{"1min":0.83699999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6779999999999999,"5min":4.4429999999999996,"15min":5.4580000000000002},"last":1.0409999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:02:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.2849999999999999,"15min":1.3660000000000001},"min":{"1min":0.873,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.96,"5min":4.5469999999999997,"15min":5.141},"last":1.331},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.478,"15min":1.5509999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.71099999999999997},"max":{"1min":1.75,"5min":2.657,"15min":2.657},"last":1.54}]},{"osd":7,"last update":"Wed Oct 18 09:02:17 2023","interfaces":[{"interface":"back","average":{"1min":2.052,"5min":1.5860000000000001,"15min":1.6439999999999999},"min":{"1min":1.0580000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":11.829000000000001,"5min":11.829000000000001,"15min":11.829000000000001},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.423,"15min":1.4870000000000001},"min":{"1min":1.0189999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.613,"5min":4.4690000000000003,"15min":4.8419999999999996},"last":1.4179999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":711,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194601,"statfs":{"total":200000000,"available":199271582,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.1539999999999999,"15min":1.1240000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":3.1819999999999999,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.147,"15min":1.1679999999999999},"min":{"1min":0.63300000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":3.234,"5min":3.234,"15min":3.234},"last":0.94999999999999996}]},{"osd":1,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.153,"15min":1.115},"min":{"1min":0.76100000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":3.718,"5min":3.718,"15min":3.718},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.2010000000000001,"15min":1.226},"min":{"1min":0.71899999999999997,"5min":0.627,"15min":0.627},"max":{"1min":3.4350000000000001,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.359}]},{"osd":2,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.2370000000000001,"15min":1.196},"min":{"1min":0.90900000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.1749999999999998,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":1.296},{"interface":"front","average":{"1min":1.508,"5min":1.335,"15min":1.3149999999999999},"min":{"1min":1.0429999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":3.532,"5min":3.532,"15min":3.532},"last":1.1339999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.296,"15min":1.298},"min":{"1min":0.81100000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":3.363,"5min":3.363,"15min":3.363},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.4239999999999999,"15min":1.417},"min":{"1min":0.93500000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":4.3179999999999996,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":1.087}]},{"osd":5,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.2969999999999999,"15min":1.3009999999999999},"min":{"1min":0.86599999999999999,"5min":0.498,"15min":0.498},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.3200000000000001,"15min":1.3540000000000001},"min":{"1min":0.97999999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":4.0780000000000003,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":1.2050000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.4350000000000001,"15min":1.4019999999999999},"min":{"1min":1.04,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":4.4429999999999996,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.417},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.486,"15min":1.546},"min":{"1min":0.98899999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.1239999999999997,"5min":4.407,"15min":4.407},"last":1.52}]},{"osd":7,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.4510000000000001,"15min":1.4990000000000001},"min":{"1min":0.98399999999999999,"5min":0.438,"15min":0.438},"max":{"1min":4.5170000000000003,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":1.052},{"interface":"front","average":{"1min":1.726,"5min":1.548,"15min":1.554},"min":{"1min":1.1879999999999999,"5min":0.747,"15min":0.747},"max":{"1min":4.5460000000000003,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":1.4830000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":686,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194626,"statfs":{"total":200000000,"available":199297466,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.88700000000000001,"15min":0.81599999999999995},"min":{"1min":0.40600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":1.415,"5min":1.9590000000000001,"15min":5.5730000000000004},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.93999999999999995,"15min":0.90500000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.446,"5min":2.4569999999999999,"15min":6.5599999999999996},"last":0.81699999999999995}]},{"osd":1,"last update":"Wed Oct 18 09:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.999,"15min":0.96899999999999997},"min":{"1min":0.39100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.234,"5min":1.839,"15min":6.8099999999999996},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.2529999999999999,"15min":1.159},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.8080000000000001,"5min":2.0720000000000001,"15min":3.1920000000000002},"last":1.5449999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.052,"15min":1.036},"min":{"1min":0.46800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.788,"5min":2.0600000000000001,"15min":7.4580000000000002},"last":1.905},{"interface":"front","average":{"1min":0.996,"5min":1.149,"15min":1.0880000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.51400000000000001},"max":{"1min":1.7450000000000001,"5min":3.169,"15min":3.5110000000000001},"last":1.212}]},{"osd":4,"last update":"Wed Oct 18 09:02:15 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.292,"15min":1.325},"min":{"1min":0.71899999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.8640000000000001,"5min":1.9259999999999999,"15min":18.664999999999999},"last":2.0329999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.292,"15min":1.2010000000000001},"min":{"1min":0.878,"5min":0.69099999999999995,"15min":0.64000000000000001},"max":{"1min":1.837,"5min":2.093,"15min":7.0700000000000003},"last":1.375}]},{"osd":5,"last update":"Wed Oct 18 09:02:15 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.304,"15min":1.2529999999999999},"min":{"1min":0.77600000000000002,"5min":0.72399999999999998,"15min":0.67700000000000005},"max":{"1min":1.925,"5min":2.1680000000000001,"15min":6.9139999999999997},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.4119999999999999,"15min":1.3799999999999999},"min":{"1min":0.85199999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.843,"5min":2.0179999999999998,"15min":16.390999999999998},"last":1.8140000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:02:15 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.4490000000000001,"15min":1.4390000000000001},"min":{"1min":0.77200000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.7390000000000001,"5min":2.129,"15min":7.7270000000000003},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.462,"15min":1.3799999999999999},"min":{"1min":0.872,"5min":0.77200000000000002,"15min":0.60199999999999998},"max":{"1min":1.8919999999999999,"5min":2.3759999999999999,"15min":7.242},"last":1.4710000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:02:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.4970000000000001,"15min":1.387},"min":{"1min":0.84099999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.2770000000000001,"5min":2.2770000000000001,"15min":4.7969999999999997},"last":1.651},{"interface":"front","average":{"1min":1.365,"5min":1.4910000000000001,"15min":1.4750000000000001},"min":{"1min":0.82099999999999995,"5min":0.622,"15min":0.622},"max":{"1min":1.9810000000000001,"5min":2.319,"15min":12.946},"last":1.752}]}]},{"osd":2,"up_from":13,"seq":55834574943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194601,"statfs":{"total":200000000,"available":199272265,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:02:14 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89000000000000001,"15min":0.92000000000000004},"min":{"1min":0.65600000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.5660000000000001,"5min":1.706,"15min":4.2210000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.173,"5min":1.0329999999999999,"15min":1.044},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.3959999999999999,"5min":4.3959999999999999,"15min":4.3959999999999999},"last":0.92800000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:02:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.1200000000000001,"15min":1.0760000000000001},"min":{"1min":0.55900000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":7.3390000000000004,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1819999999999999,"15min":1.115},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":2.448,"5min":2.448,"15min":2.5350000000000001},"last":1.2849999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:02:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.1799999999999999,"15min":1.1439999999999999},"min":{"1min":0.61099999999999999,"5min":0.54600000000000004,"15min":0.48199999999999998},"max":{"1min":7.6369999999999996,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.3680000000000001,"15min":1.4450000000000001},"min":{"1min":0.80400000000000005,"5min":0.57899999999999996,"15min":0.54100000000000004},"max":{"1min":1.7649999999999999,"5min":2.2829999999999999,"15min":7.5570000000000004},"last":1.1870000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:02:14 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.4510000000000001,"15min":1.48},"min":{"1min":1.04,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.3809999999999998,"5min":2.4119999999999999,"15min":7.6189999999999998},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5009999999999999,"15min":1.464},"min":{"1min":1.0589999999999999,"5min":0.93799999999999994,"15min":0.86599999999999999},"max":{"1min":3.423,"5min":3.423,"15min":3.423},"last":1.601}]},{"osd":5,"last update":"Wed Oct 18 09:02:14 2023","interfaces":[{"interface":"back","average":{"1min":1.748,"5min":1.401,"15min":1.3109999999999999},"min":{"1min":0.93400000000000005,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":10.507,"5min":10.507,"15min":10.507},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.482,"15min":1.546},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.81399999999999995},"max":{"1min":1.772,"5min":2.536,"15min":7.657},"last":1.482}]},{"osd":6,"last update":"Wed Oct 18 09:02:14 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.52,"15min":1.5249999999999999},"min":{"1min":1.0489999999999999,"5min":0.82699999999999996,"15min":0.74299999999999999},"max":{"1min":2.8410000000000002,"5min":2.8780000000000001,"15min":3.5299999999999998},"last":1.52},{"interface":"front","average":{"1min":1.835,"5min":1.9910000000000001,"15min":1.6140000000000001},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.77600000000000002},"max":{"1min":10.561,"5min":37.771999999999998,"15min":37.771999999999998},"last":1.643}]},{"osd":7,"last update":"Wed Oct 18 09:02:14 2023","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.5820000000000001,"15min":1.5740000000000001},"min":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.83199999999999996},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":3.3620000000000001},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.9299999999999999,"5min":1.629,"15min":1.522},"min":{"1min":1.0149999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":10.956,"5min":10.956,"15min":10.956},"last":1.768}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":709,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194602,"statfs":{"total":200000000,"available":199273318,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.90800000000000003,"15min":0.83199999999999996},"min":{"1min":0.58699999999999997,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":1.266,"5min":1.5589999999999999,"15min":1.5960000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.069,"5min":1.079,"15min":0.94699999999999995},"min":{"1min":0.64100000000000001,"5min":0.501,"15min":0.443},"max":{"1min":1.597,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.645}]},{"osd":2,"last update":"Wed Oct 18 09:02:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.044,"15min":0.95699999999999996},"min":{"1min":0.64700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.5389999999999999,"5min":1.6899999999999999,"15min":1.859},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.95899999999999996,"15min":0.874},"min":{"1min":0.55600000000000005,"5min":0.44800000000000001,"15min":0.36699999999999999},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.90000000000000002}]},{"osd":3,"last update":"Wed Oct 18 09:02:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1060000000000001,"15min":1.0369999999999999},"min":{"1min":0.73499999999999999,"5min":0.49199999999999999,"15min":0.436},"max":{"1min":1.413,"5min":1.617,"15min":2.1059999999999999},"last":1.204},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.089,"15min":1.0580000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.472,"5min":1.9370000000000001,"15min":3.7010000000000001},"last":1.1140000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:02:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.333,"15min":1.266},"min":{"1min":0.93000000000000005,"5min":0.82099999999999995,"15min":0.59999999999999998},"max":{"1min":1.655,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.343,"15min":1.2390000000000001},"min":{"1min":0.997,"5min":0.754,"15min":0.754},"max":{"1min":1.893,"5min":1.903,"15min":2.0470000000000002},"last":1.512}]},{"osd":5,"last update":"Wed Oct 18 09:02:21 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.3959999999999999,"15min":1.3149999999999999},"min":{"1min":0.99299999999999999,"5min":0.86599999999999999,"15min":0.65400000000000003},"max":{"1min":1.7809999999999999,"5min":2.161,"15min":3.5750000000000002},"last":1.577},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.4650000000000001,"15min":1.381},"min":{"1min":1.0549999999999999,"5min":0.90500000000000003,"15min":0.83899999999999997},"max":{"1min":1.8140000000000001,"5min":1.964,"15min":2.3769999999999998},"last":1.55}]},{"osd":6,"last update":"Wed Oct 18 09:02:21 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.4179999999999999,"15min":1.3140000000000001},"min":{"1min":0.95599999999999996,"5min":0.752,"15min":0.752},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.2050000000000001},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4299999999999999,"15min":1.3029999999999999},"min":{"1min":1.1679999999999999,"5min":0.78900000000000003,"15min":0.63100000000000001},"max":{"1min":1.8540000000000001,"5min":2.121,"15min":2.121},"last":1.46}]},{"osd":7,"last update":"Wed Oct 18 09:02:21 2023","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.51,"15min":1.389},"min":{"1min":1.1839999999999999,"5min":0.84399999999999997,"15min":0.70499999999999996},"max":{"1min":1.913,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.71},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.5049999999999999,"15min":1.393},"min":{"1min":0.98199999999999998,"5min":0.82299999999999995,"15min":0.746},"max":{"1min":1.873,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.4219999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1286,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194025,"statfs":{"total":200000000,"available":198682368,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.83399999999999996,"15min":0.90600000000000003},"min":{"1min":0.442,"5min":0.371,"15min":0.37},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":3.4420000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.046,"5min":1.0920000000000001,"15min":1.085},"min":{"1min":0.56200000000000006,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.806,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.115}]},{"osd":2,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.876,"15min":0.85099999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.1459999999999999,"15min":1.1319999999999999},"min":{"1min":0.503,"5min":0.437,"15min":0.437},"max":{"1min":1.718,"5min":1.929,"15min":1.929},"last":1.2589999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.161,"15min":1.2649999999999999},"min":{"1min":0.54000000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.98,"5min":1.98,"15min":2.7639999999999998},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.4159999999999999,"15min":1.4950000000000001},"min":{"1min":0.61299999999999999,"5min":0.58499999999999996,"15min":0.58399999999999996},"max":{"1min":1.792,"5min":2.3639999999999999,"15min":3.5720000000000001},"last":1.3799999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.236,"15min":1.29},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.671,"5min":1.8819999999999999,"15min":4.492},"last":1.345},{"interface":"front","average":{"1min":1.29,"5min":1.2929999999999999,"15min":1.2629999999999999},"min":{"1min":0.879,"5min":0.879,"15min":0.72599999999999998},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.423}]},{"osd":5,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2809999999999999,"15min":1.3939999999999999},"min":{"1min":0.79100000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.7989999999999999,"5min":2.4510000000000001,"15min":4.2779999999999996},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3200000000000001,"15min":1.369},"min":{"1min":0.74399999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.847,"5min":1.847,"15min":1.8740000000000001},"last":1.294}]},{"osd":6,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4470000000000001,"15min":1.5589999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.798,"5min":2.0419999999999998,"15min":3.5299999999999998},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.385,"15min":1.4790000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":3.1440000000000001,"5min":3.1440000000000001,"15min":4.4349999999999996},"last":1.1739999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.5469999999999999,"15min":1.6259999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.956,"5min":2.2000000000000002,"15min":3.492},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.4650000000000001,"15min":1.6359999999999999},"min":{"1min":0.86499999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":4.5419999999999998},"last":1.226}]}]}],"pool_statfs":[]}} 2023-10-18T09:02:46.838 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:02:47.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.110+0000 7fb4bbfff640 1 -- 172.21.15.40:0/1742217692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc0b15b0 msgr2=0x7fb4bc0b1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:47.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.110+0000 7fb4bbfff640 1 --2- 172.21.15.40:0/1742217692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc0b15b0 0x7fb4bc0b1e10 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4005fd0 tx=0x7fb4a402be10 comp rx=0 tx=0).stop 2023-10-18T09:02:47.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.110+0000 7fb4bbfff640 1 -- 172.21.15.40:0/1742217692 shutdown_connections 2023-10-18T09:02:47.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.110+0000 7fb4bbfff640 1 --2- 172.21.15.40:0/1742217692 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4bc0c99c0 0x7fb4bc0c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.110+0000 7fb4bbfff640 1 --2- 172.21.15.40:0/1742217692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc0b15b0 0x7fb4bc0b1e10 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.110+0000 7fb4bbfff640 1 -- 172.21.15.40:0/1742217692 >> 172.21.15.40:0/1742217692 conn(0x7fb4bc069720 msgr2=0x7fb4bc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:47.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.110+0000 7fb4bbfff640 1 -- 172.21.15.40:0/1742217692 shutdown_connections 2023-10-18T09:02:47.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.110+0000 7fb4bbfff640 1 -- 172.21.15.40:0/1742217692 wait complete. 2023-10-18T09:02:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.112+0000 7fb4bbfff640 1 Processor -- start 2023-10-18T09:02:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.112+0000 7fb4bbfff640 1 -- start start 2023-10-18T09:02:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.113+0000 7fb4bbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4bc06d6f0 0x7fb4bc152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.113+0000 7fb4bbfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4bc0b15b0 0x7fb4bc13ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.113+0000 7fb4bbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc0c99c0 0x7fb4bc0acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.113+0000 7fb4bbfff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc153890 con 0x7fb4bc0c99c0 2023-10-18T09:02:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.113+0000 7fb4bbfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc152970 con 0x7fb4bc06d6f0 2023-10-18T09:02:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.113+0000 7fb4bbfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc152c10 con 0x7fb4bc0b15b0 2023-10-18T09:02:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.113+0000 7fb4bb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc0c99c0 0x7fb4bc0acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.113+0000 7fb4bb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc0c99c0 0x7fb4bc0acae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54528/0 (socket says 172.21.15.40:54528) 2023-10-18T09:02:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.113+0000 7fb4bb7fe640 1 -- 172.21.15.40:0/2642975126 learned_addr learned my addr 172.21.15.40:0/2642975126 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:47.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.113+0000 7fb4baffd640 1 --2- 172.21.15.40:0/2642975126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4bc06d6f0 0x7fb4bc152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:47.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.114+0000 7fb4b27fc640 1 --2- 172.21.15.40:0/2642975126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4bc0b15b0 0x7fb4bc13ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:47.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.114+0000 7fb4bb7fe640 1 -- 172.21.15.40:0/2642975126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4bc0b15b0 msgr2=0x7fb4bc13ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:47.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.114+0000 7fb4bb7fe640 1 --2- 172.21.15.40:0/2642975126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4bc0b15b0 0x7fb4bc13ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.114+0000 7fb4bb7fe640 1 -- 172.21.15.40:0/2642975126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4bc06d6f0 msgr2=0x7fb4bc152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:47.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.114+0000 7fb4bb7fe640 1 --2- 172.21.15.40:0/2642975126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4bc06d6f0 0x7fb4bc152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.114+0000 7fb4bb7fe640 1 -- 172.21.15.40:0/2642975126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a4009560 con 0x7fb4bc0c99c0 2023-10-18T09:02:47.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.114+0000 7fb4bb7fe640 1 --2- 172.21.15.40:0/2642975126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc0c99c0 0x7fb4bc0acae0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400cab0 tx=0x7fb4b400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:47.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.114+0000 7fb4b8ff9640 1 -- 172.21.15.40:0/2642975126 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4007d90 con 0x7fb4bc0c99c0 2023-10-18T09:02:47.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.114+0000 7fb4b8ff9640 1 -- 172.21.15.40:0/2642975126 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb4b400f450 con 0x7fb4bc0c99c0 2023-10-18T09:02:47.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.114+0000 7fb4b8ff9640 1 -- 172.21.15.40:0/2642975126 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400f840 con 0x7fb4bc0c99c0 2023-10-18T09:02:47.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.114+0000 7fb4bbfff640 1 -- 172.21.15.40:0/2642975126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4bc152ef0 con 0x7fb4bc0c99c0 2023-10-18T09:02:47.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.114+0000 7fb4bbfff640 1 -- 172.21.15.40:0/2642975126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4bc1534b0 con 0x7fb4bc0c99c0 2023-10-18T09:02:47.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.115+0000 7fb4bbfff640 1 -- 172.21.15.40:0/2642975126 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4bc1539d0 con 0x7fb4bc0c99c0 2023-10-18T09:02:47.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.117+0000 7fb4b8ff9640 1 -- 172.21.15.40:0/2642975126 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb4b400f9a0 con 0x7fb4bc0c99c0 2023-10-18T09:02:47.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.117+0000 7fb4b8ff9640 1 --2- 172.21.15.40:0/2642975126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb49c06ae40 0x7fb49c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:47.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.118+0000 7fb4b8ff9640 1 -- 172.21.15.40:0/2642975126 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4b408a230 con 0x7fb4bc0c99c0 2023-10-18T09:02:47.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.118+0000 7fb4baffd640 1 --2- 172.21.15.40:0/2642975126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb49c06ae40 0x7fb49c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:47.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.119+0000 7fb4baffd640 1 --2- 172.21.15.40:0/2642975126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb49c06ae40 0x7fb49c06d300 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac000f60 tx=0x7fb4ac009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:47.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.122+0000 7fb4b8ff9640 1 -- 172.21.15.40:0/2642975126 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb4bc1539d0 con 0x7fb4bc0c99c0 2023-10-18T09:02:47.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.310+0000 7fb4bbfff640 1 -- 172.21.15.40:0/2642975126 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4bc157d40 con 0x7fb49c06ae40 2023-10-18T09:02:47.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.333+0000 7fb4b8ff9640 1 -- 172.21.15.40:0/2642975126 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282332 (secure 0 0 0) 0x7fb4ac001c70 con 0x7fb49c06ae40 2023-10-18T09:02:47.339 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:02:47.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.338+0000 7fb4bbfff640 1 -- 172.21.15.40:0/2642975126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb49c06ae40 msgr2=0x7fb49c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:47.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.338+0000 7fb4bbfff640 1 --2- 172.21.15.40:0/2642975126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb49c06ae40 0x7fb49c06d300 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac000f60 tx=0x7fb4ac009040 comp rx=0 tx=0).stop 2023-10-18T09:02:47.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.338+0000 7fb4bbfff640 1 -- 172.21.15.40:0/2642975126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc0c99c0 msgr2=0x7fb4bc0acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:47.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.338+0000 7fb4bbfff640 1 --2- 172.21.15.40:0/2642975126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc0c99c0 0x7fb4bc0acae0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400cab0 tx=0x7fb4b400cf70 comp rx=0 tx=0).stop 2023-10-18T09:02:47.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.338+0000 7fb4bbfff640 1 -- 172.21.15.40:0/2642975126 shutdown_connections 2023-10-18T09:02:47.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.338+0000 7fb4bbfff640 1 --2- 172.21.15.40:0/2642975126 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb49c06ae40 0x7fb49c06d300 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.338+0000 7fb4bbfff640 1 --2- 172.21.15.40:0/2642975126 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc0c99c0 0x7fb4bc0acae0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.338+0000 7fb4bbfff640 1 --2- 172.21.15.40:0/2642975126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4bc0b15b0 0x7fb4bc13ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.338+0000 7fb4bbfff640 1 --2- 172.21.15.40:0/2642975126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4bc06d6f0 0x7fb4bc152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.338+0000 7fb4bbfff640 1 -- 172.21.15.40:0/2642975126 >> 172.21.15.40:0/2642975126 conn(0x7fb4bc069720 msgr2=0x7fb4bc156290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:47.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.338+0000 7fb4bbfff640 1 -- 172.21.15.40:0/2642975126 shutdown_connections 2023-10-18T09:02:47.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.339+0000 7fb4bbfff640 1 -- 172.21.15.40:0/2642975126 wait complete. 2023-10-18T09:02:47.362 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2023-10-18T09:02:47.060698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6839,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555660,"statfs":{"total":1600000000,"available":1592995893,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002232"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365400+0000","last_change":"2023-10-18T09:02:42.080221+0000","last_active":"2023-10-18T09:02:43.365400+0000","last_peered":"2023-10-18T09:02:43.365400+0000","last_clean":"2023-10-18T09:02:43.365400+0000","last_became_active":"2023-10-18T09:02:42.079702+0000","last_became_peered":"2023-10-18T09:02:42.079702+0000","last_unstale":"2023-10-18T09:02:43.365400+0000","last_undegraded":"2023-10-18T09:02:43.365400+0000","last_fullsized":"2023-10-18T09:02:43.365400+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:35:08.016956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364295+0000","last_change":"2023-10-18T09:02:42.090937+0000","last_active":"2023-10-18T09:02:43.364295+0000","last_peered":"2023-10-18T09:02:43.364295+0000","last_clean":"2023-10-18T09:02:43.364295+0000","last_became_active":"2023-10-18T09:02:42.089232+0000","last_became_peered":"2023-10-18T09:02:42.089232+0000","last_unstale":"2023-10-18T09:02:43.364295+0000","last_undegraded":"2023-10-18T09:02:43.364295+0000","last_fullsized":"2023-10-18T09:02:43.364295+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:29:13.401772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363272+0000","last_change":"2023-10-18T09:02:42.080834+0000","last_active":"2023-10-18T09:02:43.363272+0000","last_peered":"2023-10-18T09:02:43.363272+0000","last_clean":"2023-10-18T09:02:43.363272+0000","last_became_active":"2023-10-18T09:02:42.080312+0000","last_became_peered":"2023-10-18T09:02:42.080312+0000","last_unstale":"2023-10-18T09:02:43.363272+0000","last_undegraded":"2023-10-18T09:02:43.363272+0000","last_fullsized":"2023-10-18T09:02:43.363272+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:55:45.693647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.057532+0000","last_change":"2023-10-18T09:02:42.085407+0000","last_active":"2023-10-18T09:02:43.057532+0000","last_peered":"2023-10-18T09:02:43.057532+0000","last_clean":"2023-10-18T09:02:43.057532+0000","last_became_active":"2023-10-18T09:02:42.085020+0000","last_became_peered":"2023-10-18T09:02:42.085020+0000","last_unstale":"2023-10-18T09:02:43.057532+0000","last_undegraded":"2023-10-18T09:02:43.057532+0000","last_fullsized":"2023-10-18T09:02:43.057532+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:34:21.799724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.060211+0000","last_change":"2023-10-18T09:02:42.068042+0000","last_active":"2023-10-18T09:02:43.060211+0000","last_peered":"2023-10-18T09:02:43.060211+0000","last_clean":"2023-10-18T09:02:43.060211+0000","last_became_active":"2023-10-18T09:02:42.067375+0000","last_became_peered":"2023-10-18T09:02:42.067375+0000","last_unstale":"2023-10-18T09:02:43.060211+0000","last_undegraded":"2023-10-18T09:02:43.060211+0000","last_fullsized":"2023-10-18T09:02:43.060211+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:23.596366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.060309+0000","last_change":"2023-10-18T09:02:42.076730+0000","last_active":"2023-10-18T09:02:43.060309+0000","last_peered":"2023-10-18T09:02:43.060309+0000","last_clean":"2023-10-18T09:02:43.060309+0000","last_became_active":"2023-10-18T09:02:42.074592+0000","last_became_peered":"2023-10-18T09:02:42.074592+0000","last_unstale":"2023-10-18T09:02:43.060309+0000","last_undegraded":"2023-10-18T09:02:43.060309+0000","last_fullsized":"2023-10-18T09:02:43.060309+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:07.487239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.058265+0000","last_change":"2023-10-18T09:02:42.071777+0000","last_active":"2023-10-18T09:02:43.058265+0000","last_peered":"2023-10-18T09:02:43.058265+0000","last_clean":"2023-10-18T09:02:43.058265+0000","last_became_active":"2023-10-18T09:02:42.068437+0000","last_became_peered":"2023-10-18T09:02:42.068437+0000","last_unstale":"2023-10-18T09:02:43.058265+0000","last_undegraded":"2023-10-18T09:02:43.058265+0000","last_fullsized":"2023-10-18T09:02:43.058265+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:23.391425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364061+0000","last_change":"2023-10-18T09:02:42.088051+0000","last_active":"2023-10-18T09:02:43.364061+0000","last_peered":"2023-10-18T09:02:43.364061+0000","last_clean":"2023-10-18T09:02:43.364061+0000","last_became_active":"2023-10-18T09:02:42.087079+0000","last_became_peered":"2023-10-18T09:02:42.087079+0000","last_unstale":"2023-10-18T09:02:43.364061+0000","last_undegraded":"2023-10-18T09:02:43.364061+0000","last_fullsized":"2023-10-18T09:02:43.364061+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:20:48.771067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364349+0000","last_change":"2023-10-18T09:02:42.087238+0000","last_active":"2023-10-18T09:02:43.364349+0000","last_peered":"2023-10-18T09:02:43.364349+0000","last_clean":"2023-10-18T09:02:43.364349+0000","last_became_active":"2023-10-18T09:02:42.086757+0000","last_became_peered":"2023-10-18T09:02:42.086757+0000","last_unstale":"2023-10-18T09:02:43.364349+0000","last_undegraded":"2023-10-18T09:02:43.364349+0000","last_fullsized":"2023-10-18T09:02:43.364349+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:04:00.233351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.060512+0000","last_change":"2023-10-18T09:02:42.091570+0000","last_active":"2023-10-18T09:02:43.060512+0000","last_peered":"2023-10-18T09:02:43.060512+0000","last_clean":"2023-10-18T09:02:43.060512+0000","last_became_active":"2023-10-18T09:02:42.091234+0000","last_became_peered":"2023-10-18T09:02:42.091234+0000","last_unstale":"2023-10-18T09:02:43.060512+0000","last_undegraded":"2023-10-18T09:02:43.060512+0000","last_fullsized":"2023-10-18T09:02:43.060512+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:08.005031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.060591+0000","last_change":"2023-10-18T09:02:42.086366+0000","last_active":"2023-10-18T09:02:43.060591+0000","last_peered":"2023-10-18T09:02:43.060591+0000","last_clean":"2023-10-18T09:02:43.060591+0000","last_became_active":"2023-10-18T09:02:42.085520+0000","last_became_peered":"2023-10-18T09:02:42.085520+0000","last_unstale":"2023-10-18T09:02:43.060591+0000","last_undegraded":"2023-10-18T09:02:43.060591+0000","last_fullsized":"2023-10-18T09:02:43.060591+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:01.446785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364708+0000","last_change":"2023-10-18T09:02:42.083599+0000","last_active":"2023-10-18T09:02:43.364708+0000","last_peered":"2023-10-18T09:02:43.364708+0000","last_clean":"2023-10-18T09:02:43.364708+0000","last_became_active":"2023-10-18T09:02:42.083003+0000","last_became_peered":"2023-10-18T09:02:42.083003+0000","last_unstale":"2023-10-18T09:02:43.364708+0000","last_undegraded":"2023-10-18T09:02:43.364708+0000","last_fullsized":"2023-10-18T09:02:43.364708+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:56:44.616338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.056827+0000","last_change":"2023-10-18T09:02:42.084269+0000","last_active":"2023-10-18T09:02:43.056827+0000","last_peered":"2023-10-18T09:02:43.056827+0000","last_clean":"2023-10-18T09:02:43.056827+0000","last_became_active":"2023-10-18T09:02:42.083988+0000","last_became_peered":"2023-10-18T09:02:42.083988+0000","last_unstale":"2023-10-18T09:02:43.056827+0000","last_undegraded":"2023-10-18T09:02:43.056827+0000","last_fullsized":"2023-10-18T09:02:43.056827+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:31:53.246355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364656+0000","last_change":"2023-10-18T09:02:42.088619+0000","last_active":"2023-10-18T09:02:43.364656+0000","last_peered":"2023-10-18T09:02:43.364656+0000","last_clean":"2023-10-18T09:02:43.364656+0000","last_became_active":"2023-10-18T09:02:42.085289+0000","last_became_peered":"2023-10-18T09:02:42.085289+0000","last_unstale":"2023-10-18T09:02:43.364656+0000","last_undegraded":"2023-10-18T09:02:43.364656+0000","last_fullsized":"2023-10-18T09:02:43.364656+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:39.491657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.057314+0000","last_change":"2023-10-18T09:02:42.091120+0000","last_active":"2023-10-18T09:02:43.057314+0000","last_peered":"2023-10-18T09:02:43.057314+0000","last_clean":"2023-10-18T09:02:43.057314+0000","last_became_active":"2023-10-18T09:02:42.090815+0000","last_became_peered":"2023-10-18T09:02:42.090815+0000","last_unstale":"2023-10-18T09:02:43.057314+0000","last_undegraded":"2023-10-18T09:02:43.057314+0000","last_fullsized":"2023-10-18T09:02:43.057314+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:42:09.241740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.060940+0000","last_change":"2023-10-18T09:02:42.079896+0000","last_active":"2023-10-18T09:02:43.060940+0000","last_peered":"2023-10-18T09:02:43.060940+0000","last_clean":"2023-10-18T09:02:43.060940+0000","last_became_active":"2023-10-18T09:02:42.079533+0000","last_became_peered":"2023-10-18T09:02:42.079533+0000","last_unstale":"2023-10-18T09:02:43.060940+0000","last_undegraded":"2023-10-18T09:02:43.060940+0000","last_fullsized":"2023-10-18T09:02:43.060940+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:20:48.771067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364490+0000","last_change":"2023-10-18T09:02:42.090410+0000","last_active":"2023-10-18T09:02:43.364490+0000","last_peered":"2023-10-18T09:02:43.364490+0000","last_clean":"2023-10-18T09:02:43.364490+0000","last_became_active":"2023-10-18T09:02:42.088799+0000","last_became_peered":"2023-10-18T09:02:42.088799+0000","last_unstale":"2023-10-18T09:02:43.364490+0000","last_undegraded":"2023-10-18T09:02:43.364490+0000","last_fullsized":"2023-10-18T09:02:43.364490+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:23.596366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.057010+0000","last_change":"2023-10-18T09:02:42.074524+0000","last_active":"2023-10-18T09:02:43.057010+0000","last_peered":"2023-10-18T09:02:43.057010+0000","last_clean":"2023-10-18T09:02:43.057010+0000","last_became_active":"2023-10-18T09:02:42.073874+0000","last_became_peered":"2023-10-18T09:02:42.073874+0000","last_unstale":"2023-10-18T09:02:43.057010+0000","last_undegraded":"2023-10-18T09:02:43.057010+0000","last_fullsized":"2023-10-18T09:02:43.057010+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:40:20.110737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364840+0000","last_change":"2023-10-18T09:02:42.089210+0000","last_active":"2023-10-18T09:02:43.364840+0000","last_peered":"2023-10-18T09:02:43.364840+0000","last_clean":"2023-10-18T09:02:43.364840+0000","last_became_active":"2023-10-18T09:02:42.085622+0000","last_became_peered":"2023-10-18T09:02:42.085622+0000","last_unstale":"2023-10-18T09:02:43.364840+0000","last_undegraded":"2023-10-18T09:02:43.364840+0000","last_fullsized":"2023-10-18T09:02:43.364840+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:08:28.892720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365089+0000","last_change":"2023-10-18T09:02:42.084409+0000","last_active":"2023-10-18T09:02:43.365089+0000","last_peered":"2023-10-18T09:02:43.365089+0000","last_clean":"2023-10-18T09:02:43.365089+0000","last_became_active":"2023-10-18T09:02:42.084040+0000","last_became_peered":"2023-10-18T09:02:42.084040+0000","last_unstale":"2023-10-18T09:02:43.365089+0000","last_undegraded":"2023-10-18T09:02:43.365089+0000","last_fullsized":"2023-10-18T09:02:43.365089+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:07:12.903518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363409+0000","last_change":"2023-10-18T09:02:42.083371+0000","last_active":"2023-10-18T09:02:43.363409+0000","last_peered":"2023-10-18T09:02:43.363409+0000","last_clean":"2023-10-18T09:02:43.363409+0000","last_became_active":"2023-10-18T09:02:42.079991+0000","last_became_peered":"2023-10-18T09:02:42.079991+0000","last_unstale":"2023-10-18T09:02:43.363409+0000","last_undegraded":"2023-10-18T09:02:43.363409+0000","last_fullsized":"2023-10-18T09:02:43.363409+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:02.686085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364895+0000","last_change":"2023-10-18T09:02:42.089481+0000","last_active":"2023-10-18T09:02:43.364895+0000","last_peered":"2023-10-18T09:02:43.364895+0000","last_clean":"2023-10-18T09:02:43.364895+0000","last_became_active":"2023-10-18T09:02:42.085532+0000","last_became_peered":"2023-10-18T09:02:42.085532+0000","last_unstale":"2023-10-18T09:02:43.364895+0000","last_undegraded":"2023-10-18T09:02:43.364895+0000","last_fullsized":"2023-10-18T09:02:43.364895+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:29.393784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364221+0000","last_change":"2023-10-18T09:02:42.087324+0000","last_active":"2023-10-18T09:02:43.364221+0000","last_peered":"2023-10-18T09:02:43.364221+0000","last_clean":"2023-10-18T09:02:43.364221+0000","last_became_active":"2023-10-18T09:02:42.086227+0000","last_became_peered":"2023-10-18T09:02:42.086227+0000","last_unstale":"2023-10-18T09:02:43.364221+0000","last_undegraded":"2023-10-18T09:02:43.364221+0000","last_fullsized":"2023-10-18T09:02:43.364221+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:44.776977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.058198+0000","last_change":"2023-10-18T09:02:42.071752+0000","last_active":"2023-10-18T09:02:43.058198+0000","last_peered":"2023-10-18T09:02:43.058198+0000","last_clean":"2023-10-18T09:02:43.058198+0000","last_became_active":"2023-10-18T09:02:42.071327+0000","last_became_peered":"2023-10-18T09:02:42.071327+0000","last_unstale":"2023-10-18T09:02:43.058198+0000","last_undegraded":"2023-10-18T09:02:43.058198+0000","last_fullsized":"2023-10-18T09:02:43.058198+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:09:33.158265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.060813+0000","last_change":"2023-10-18T09:02:42.075089+0000","last_active":"2023-10-18T09:02:43.060813+0000","last_peered":"2023-10-18T09:02:43.060813+0000","last_clean":"2023-10-18T09:02:43.060813+0000","last_became_active":"2023-10-18T09:02:42.074631+0000","last_became_peered":"2023-10-18T09:02:42.074631+0000","last_unstale":"2023-10-18T09:02:43.060813+0000","last_undegraded":"2023-10-18T09:02:43.060813+0000","last_fullsized":"2023-10-18T09:02:43.060813+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:41:29.393784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365359+0000","last_change":"2023-10-18T09:02:42.083732+0000","last_active":"2023-10-18T09:02:43.365359+0000","last_peered":"2023-10-18T09:02:43.365359+0000","last_clean":"2023-10-18T09:02:43.365359+0000","last_became_active":"2023-10-18T09:02:42.083320+0000","last_became_peered":"2023-10-18T09:02:42.083320+0000","last_unstale":"2023-10-18T09:02:43.365359+0000","last_undegraded":"2023-10-18T09:02:43.365359+0000","last_fullsized":"2023-10-18T09:02:43.365359+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:51.546758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.058580+0000","last_change":"2023-10-18T09:02:42.077360+0000","last_active":"2023-10-18T09:02:43.058580+0000","last_peered":"2023-10-18T09:02:43.058580+0000","last_clean":"2023-10-18T09:02:43.058580+0000","last_became_active":"2023-10-18T09:02:42.075817+0000","last_became_peered":"2023-10-18T09:02:42.075817+0000","last_unstale":"2023-10-18T09:02:43.058580+0000","last_undegraded":"2023-10-18T09:02:43.058580+0000","last_fullsized":"2023-10-18T09:02:43.058580+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:44:25.772112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.059314+0000","last_change":"2023-10-18T09:02:42.065219+0000","last_active":"2023-10-18T09:02:43.059314+0000","last_peered":"2023-10-18T09:02:43.059314+0000","last_clean":"2023-10-18T09:02:43.059314+0000","last_became_active":"2023-10-18T09:02:42.063676+0000","last_became_peered":"2023-10-18T09:02:42.063676+0000","last_unstale":"2023-10-18T09:02:43.059314+0000","last_undegraded":"2023-10-18T09:02:43.059314+0000","last_fullsized":"2023-10-18T09:02:43.059314+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:42:09.257183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363448+0000","last_change":"2023-10-18T09:02:42.086237+0000","last_active":"2023-10-18T09:02:43.363448+0000","last_peered":"2023-10-18T09:02:43.363448+0000","last_clean":"2023-10-18T09:02:43.363448+0000","last_became_active":"2023-10-18T09:02:42.085928+0000","last_became_peered":"2023-10-18T09:02:42.085928+0000","last_unstale":"2023-10-18T09:02:43.363448+0000","last_undegraded":"2023-10-18T09:02:43.363448+0000","last_fullsized":"2023-10-18T09:02:43.363448+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:01.446785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364671+0000","last_change":"2023-10-18T09:02:42.092619+0000","last_active":"2023-10-18T09:02:43.364671+0000","last_peered":"2023-10-18T09:02:43.364671+0000","last_clean":"2023-10-18T09:02:43.364671+0000","last_became_active":"2023-10-18T09:02:42.092298+0000","last_became_peered":"2023-10-18T09:02:42.092298+0000","last_unstale":"2023-10-18T09:02:43.364671+0000","last_undegraded":"2023-10-18T09:02:43.364671+0000","last_fullsized":"2023-10-18T09:02:43.364671+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:25:22.025388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.056931+0000","last_change":"2023-10-18T09:02:42.063983+0000","last_active":"2023-10-18T09:02:43.056931+0000","last_peered":"2023-10-18T09:02:43.056931+0000","last_clean":"2023-10-18T09:02:43.056931+0000","last_became_active":"2023-10-18T09:02:42.063270+0000","last_became_peered":"2023-10-18T09:02:42.063270+0000","last_unstale":"2023-10-18T09:02:43.056931+0000","last_undegraded":"2023-10-18T09:02:43.056931+0000","last_fullsized":"2023-10-18T09:02:43.056931+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:24:01.777445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363511+0000","last_change":"2023-10-18T09:02:42.091546+0000","last_active":"2023-10-18T09:02:43.363511+0000","last_peered":"2023-10-18T09:02:43.363511+0000","last_clean":"2023-10-18T09:02:43.363511+0000","last_became_active":"2023-10-18T09:02:42.091240+0000","last_became_peered":"2023-10-18T09:02:42.091240+0000","last_unstale":"2023-10-18T09:02:43.363511+0000","last_undegraded":"2023-10-18T09:02:43.363511+0000","last_fullsized":"2023-10-18T09:02:43.363511+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:04:37.072672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365003+0000","last_change":"2023-10-18T09:02:42.084270+0000","last_active":"2023-10-18T09:02:43.365003+0000","last_peered":"2023-10-18T09:02:43.365003+0000","last_clean":"2023-10-18T09:02:43.365003+0000","last_became_active":"2023-10-18T09:02:42.079899+0000","last_became_peered":"2023-10-18T09:02:42.079899+0000","last_unstale":"2023-10-18T09:02:43.365003+0000","last_undegraded":"2023-10-18T09:02:43.365003+0000","last_fullsized":"2023-10-18T09:02:43.365003+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:57:02.012639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"129'4","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364600+0000","last_change":"2023-10-18T09:02:41.324352+0000","last_active":"2023-10-18T09:02:43.364600+0000","last_peered":"2023-10-18T09:02:43.364600+0000","last_clean":"2023-10-18T09:02:43.364600+0000","last_became_active":"2023-10-18T09:02:41.324090+0000","last_became_peered":"2023-10-18T09:02:41.324090+0000","last_unstale":"2023-10-18T09:02:43.364600+0000","last_undegraded":"2023-10-18T09:02:43.364600+0000","last_fullsized":"2023-10-18T09:02:43.364600+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:22.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.057184+0000","last_change":"2023-10-18T09:02:42.066269+0000","last_active":"2023-10-18T09:02:43.057184+0000","last_peered":"2023-10-18T09:02:43.057184+0000","last_clean":"2023-10-18T09:02:43.057184+0000","last_became_active":"2023-10-18T09:02:42.065189+0000","last_became_peered":"2023-10-18T09:02:42.065189+0000","last_unstale":"2023-10-18T09:02:43.057184+0000","last_undegraded":"2023-10-18T09:02:43.057184+0000","last_fullsized":"2023-10-18T09:02:43.057184+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:01.427949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"129'1","reported_seq":13,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.058445+0000","last_change":"2023-10-18T09:02:41.311634+0000","last_active":"2023-10-18T09:02:43.058445+0000","last_peered":"2023-10-18T09:02:43.058445+0000","last_clean":"2023-10-18T09:02:43.058445+0000","last_became_active":"2023-10-18T09:02:41.310999+0000","last_became_peered":"2023-10-18T09:02:41.310999+0000","last_unstale":"2023-10-18T09:02:43.058445+0000","last_undegraded":"2023-10-18T09:02:43.058445+0000","last_fullsized":"2023-10-18T09:02:43.058445+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:56:43.600810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365702+0000","last_change":"2023-10-18T09:02:42.088381+0000","last_active":"2023-10-18T09:02:43.365702+0000","last_peered":"2023-10-18T09:02:43.365702+0000","last_clean":"2023-10-18T09:02:43.365702+0000","last_became_active":"2023-10-18T09:02:42.088134+0000","last_became_peered":"2023-10-18T09:02:42.088134+0000","last_unstale":"2023-10-18T09:02:43.365702+0000","last_undegraded":"2023-10-18T09:02:43.365702+0000","last_fullsized":"2023-10-18T09:02:43.365702+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:32:19.050992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"129'2","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365216+0000","last_change":"2023-10-18T09:02:41.325861+0000","last_active":"2023-10-18T09:02:43.365216+0000","last_peered":"2023-10-18T09:02:43.365216+0000","last_clean":"2023-10-18T09:02:43.365216+0000","last_became_active":"2023-10-18T09:02:41.325588+0000","last_became_peered":"2023-10-18T09:02:41.325588+0000","last_unstale":"2023-10-18T09:02:43.365216+0000","last_undegraded":"2023-10-18T09:02:43.365216+0000","last_fullsized":"2023-10-18T09:02:43.365216+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:15.138604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365379+0000","last_change":"2023-10-18T09:02:42.080067+0000","last_active":"2023-10-18T09:02:43.365379+0000","last_peered":"2023-10-18T09:02:43.365379+0000","last_clean":"2023-10-18T09:02:43.365379+0000","last_became_active":"2023-10-18T09:02:42.079804+0000","last_became_peered":"2023-10-18T09:02:42.079804+0000","last_unstale":"2023-10-18T09:02:43.365379+0000","last_undegraded":"2023-10-18T09:02:43.365379+0000","last_fullsized":"2023-10-18T09:02:43.365379+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:33.367613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"129'2","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365497+0000","last_change":"2023-10-18T09:02:41.323282+0000","last_active":"2023-10-18T09:02:43.365497+0000","last_peered":"2023-10-18T09:02:43.365497+0000","last_clean":"2023-10-18T09:02:43.365497+0000","last_became_active":"2023-10-18T09:02:41.323046+0000","last_became_peered":"2023-10-18T09:02:41.323046+0000","last_unstale":"2023-10-18T09:02:43.365497+0000","last_undegraded":"2023-10-18T09:02:43.365497+0000","last_fullsized":"2023-10-18T09:02:43.365497+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:08.235986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364441+0000","last_change":"2023-10-18T09:02:42.080475+0000","last_active":"2023-10-18T09:02:43.364441+0000","last_peered":"2023-10-18T09:02:43.364441+0000","last_clean":"2023-10-18T09:02:43.364441+0000","last_became_active":"2023-10-18T09:02:42.080086+0000","last_became_peered":"2023-10-18T09:02:42.080086+0000","last_unstale":"2023-10-18T09:02:43.364441+0000","last_undegraded":"2023-10-18T09:02:43.364441+0000","last_fullsized":"2023-10-18T09:02:43.364441+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:04:30.422371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"129'1","reported_seq":13,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363865+0000","last_change":"2023-10-18T09:02:41.322086+0000","last_active":"2023-10-18T09:02:43.363865+0000","last_peered":"2023-10-18T09:02:43.363865+0000","last_clean":"2023-10-18T09:02:43.363865+0000","last_became_active":"2023-10-18T09:02:41.321844+0000","last_became_peered":"2023-10-18T09:02:41.321844+0000","last_unstale":"2023-10-18T09:02:43.363865+0000","last_undegraded":"2023-10-18T09:02:43.363865+0000","last_fullsized":"2023-10-18T09:02:43.363865+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:01.502838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.058920+0000","last_change":"2023-10-18T09:02:42.076616+0000","last_active":"2023-10-18T09:02:43.058920+0000","last_peered":"2023-10-18T09:02:43.058920+0000","last_clean":"2023-10-18T09:02:43.058920+0000","last_became_active":"2023-10-18T09:02:42.075349+0000","last_became_peered":"2023-10-18T09:02:42.075349+0000","last_unstale":"2023-10-18T09:02:43.058920+0000","last_undegraded":"2023-10-18T09:02:43.058920+0000","last_fullsized":"2023-10-18T09:02:43.058920+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:14:47.201434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"129'1","reported_seq":13,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364723+0000","last_change":"2023-10-18T09:02:41.320152+0000","last_active":"2023-10-18T09:02:43.364723+0000","last_peered":"2023-10-18T09:02:43.364723+0000","last_clean":"2023-10-18T09:02:43.364723+0000","last_became_active":"2023-10-18T09:02:41.315903+0000","last_became_peered":"2023-10-18T09:02:41.315903+0000","last_unstale":"2023-10-18T09:02:43.364723+0000","last_undegraded":"2023-10-18T09:02:43.364723+0000","last_fullsized":"2023-10-18T09:02:43.364723+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:44:24.756584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.057425+0000","last_change":"2023-10-18T09:02:42.090298+0000","last_active":"2023-10-18T09:02:43.057425+0000","last_peered":"2023-10-18T09:02:43.057425+0000","last_clean":"2023-10-18T09:02:43.057425+0000","last_became_active":"2023-10-18T09:02:42.089865+0000","last_became_peered":"2023-10-18T09:02:42.089865+0000","last_unstale":"2023-10-18T09:02:43.057425+0000","last_undegraded":"2023-10-18T09:02:43.057425+0000","last_fullsized":"2023-10-18T09:02:43.057425+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:38:57.034944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.059053+0000","last_change":"2023-10-18T09:02:41.325209+0000","last_active":"2023-10-18T09:02:43.059053+0000","last_peered":"2023-10-18T09:02:43.059053+0000","last_clean":"2023-10-18T09:02:43.059053+0000","last_became_active":"2023-10-18T09:02:41.324895+0000","last_became_peered":"2023-10-18T09:02:41.324895+0000","last_unstale":"2023-10-18T09:02:43.059053+0000","last_undegraded":"2023-10-18T09:02:43.059053+0000","last_fullsized":"2023-10-18T09:02:43.059053+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:32.924193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364696+0000","last_change":"2023-10-18T09:02:42.078871+0000","last_active":"2023-10-18T09:02:43.364696+0000","last_peered":"2023-10-18T09:02:43.364696+0000","last_clean":"2023-10-18T09:02:43.364696+0000","last_became_active":"2023-10-18T09:02:42.078441+0000","last_became_peered":"2023-10-18T09:02:42.078441+0000","last_unstale":"2023-10-18T09:02:43.364696+0000","last_undegraded":"2023-10-18T09:02:43.364696+0000","last_fullsized":"2023-10-18T09:02:43.364696+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:16.154132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363877+0000","last_change":"2023-10-18T09:02:41.306939+0000","last_active":"2023-10-18T09:02:43.363877+0000","last_peered":"2023-10-18T09:02:43.363877+0000","last_clean":"2023-10-18T09:02:43.363877+0000","last_became_active":"2023-10-18T09:02:41.306708+0000","last_became_peered":"2023-10-18T09:02:41.306708+0000","last_unstale":"2023-10-18T09:02:43.363877+0000","last_undegraded":"2023-10-18T09:02:43.363877+0000","last_fullsized":"2023-10-18T09:02:43.363877+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:11:54.131748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364702+0000","last_change":"2023-10-18T09:02:42.090046+0000","last_active":"2023-10-18T09:02:43.364702+0000","last_peered":"2023-10-18T09:02:43.364702+0000","last_clean":"2023-10-18T09:02:43.364702+0000","last_became_active":"2023-10-18T09:02:42.087516+0000","last_became_peered":"2023-10-18T09:02:42.087516+0000","last_unstale":"2023-10-18T09:02:43.364702+0000","last_undegraded":"2023-10-18T09:02:43.364702+0000","last_fullsized":"2023-10-18T09:02:43.364702+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:08.005031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365822+0000","last_change":"2023-10-18T09:02:41.316687+0000","last_active":"2023-10-18T09:02:43.365822+0000","last_peered":"2023-10-18T09:02:43.365822+0000","last_clean":"2023-10-18T09:02:43.365822+0000","last_became_active":"2023-10-18T09:02:41.315644+0000","last_became_peered":"2023-10-18T09:02:41.315644+0000","last_unstale":"2023-10-18T09:02:43.365822+0000","last_undegraded":"2023-10-18T09:02:43.365822+0000","last_fullsized":"2023-10-18T09:02:43.365822+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:46.569258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364981+0000","last_change":"2023-10-18T09:02:42.085493+0000","last_active":"2023-10-18T09:02:43.364981+0000","last_peered":"2023-10-18T09:02:43.364981+0000","last_clean":"2023-10-18T09:02:43.364981+0000","last_became_active":"2023-10-18T09:02:42.084741+0000","last_became_peered":"2023-10-18T09:02:42.084741+0000","last_unstale":"2023-10-18T09:02:43.364981+0000","last_undegraded":"2023-10-18T09:02:43.364981+0000","last_fullsized":"2023-10-18T09:02:43.364981+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:47.432868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.056669+0000","last_change":"2023-10-18T09:02:41.322963+0000","last_active":"2023-10-18T09:02:43.056669+0000","last_peered":"2023-10-18T09:02:43.056669+0000","last_clean":"2023-10-18T09:02:43.056669+0000","last_became_active":"2023-10-18T09:02:41.322711+0000","last_became_peered":"2023-10-18T09:02:41.322711+0000","last_unstale":"2023-10-18T09:02:43.056669+0000","last_undegraded":"2023-10-18T09:02:43.056669+0000","last_fullsized":"2023-10-18T09:02:43.056669+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:30.366977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.058194+0000","last_change":"2023-10-18T09:02:42.080269+0000","last_active":"2023-10-18T09:02:43.058194+0000","last_peered":"2023-10-18T09:02:43.058194+0000","last_clean":"2023-10-18T09:02:43.058194+0000","last_became_active":"2023-10-18T09:02:42.079964+0000","last_became_peered":"2023-10-18T09:02:42.079964+0000","last_unstale":"2023-10-18T09:02:43.058194+0000","last_undegraded":"2023-10-18T09:02:43.058194+0000","last_fullsized":"2023-10-18T09:02:43.058194+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:01.446785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"129'3","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.058952+0000","last_change":"2023-10-18T09:02:41.322507+0000","last_active":"2023-10-18T09:02:43.058952+0000","last_peered":"2023-10-18T09:02:43.058952+0000","last_clean":"2023-10-18T09:02:43.058952+0000","last_became_active":"2023-10-18T09:02:41.322267+0000","last_became_peered":"2023-10-18T09:02:41.322267+0000","last_unstale":"2023-10-18T09:02:43.058952+0000","last_undegraded":"2023-10-18T09:02:43.058952+0000","last_fullsized":"2023-10-18T09:02:43.058952+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:36:50.180955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.058700+0000","last_change":"2023-10-18T09:02:42.087462+0000","last_active":"2023-10-18T09:02:43.058700+0000","last_peered":"2023-10-18T09:02:43.058700+0000","last_clean":"2023-10-18T09:02:43.058700+0000","last_became_active":"2023-10-18T09:02:42.087195+0000","last_became_peered":"2023-10-18T09:02:42.087195+0000","last_unstale":"2023-10-18T09:02:43.058700+0000","last_undegraded":"2023-10-18T09:02:43.058700+0000","last_fullsized":"2023-10-18T09:02:43.058700+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:33.939721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"129'6","reported_seq":18,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365159+0000","last_change":"2023-10-18T09:02:41.323868+0000","last_active":"2023-10-18T09:02:43.365159+0000","last_peered":"2023-10-18T09:02:43.365159+0000","last_clean":"2023-10-18T09:02:43.365159+0000","last_became_active":"2023-10-18T09:02:41.323344+0000","last_became_peered":"2023-10-18T09:02:41.323344+0000","last_unstale":"2023-10-18T09:02:43.365159+0000","last_undegraded":"2023-10-18T09:02:43.365159+0000","last_fullsized":"2023-10-18T09:02:43.365159+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:17.988078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.057081+0000","last_change":"2023-10-18T09:02:42.077590+0000","last_active":"2023-10-18T09:02:43.057081+0000","last_peered":"2023-10-18T09:02:43.057081+0000","last_clean":"2023-10-18T09:02:43.057081+0000","last_became_active":"2023-10-18T09:02:42.077040+0000","last_became_peered":"2023-10-18T09:02:42.077040+0000","last_unstale":"2023-10-18T09:02:43.057081+0000","last_undegraded":"2023-10-18T09:02:43.057081+0000","last_fullsized":"2023-10-18T09:02:43.057081+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:48:35.894607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"129'2","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364821+0000","last_change":"2023-10-18T09:02:41.325268+0000","last_active":"2023-10-18T09:02:43.364821+0000","last_peered":"2023-10-18T09:02:43.364821+0000","last_clean":"2023-10-18T09:02:43.364821+0000","last_became_active":"2023-10-18T09:02:41.325032+0000","last_became_peered":"2023-10-18T09:02:41.325032+0000","last_unstale":"2023-10-18T09:02:43.364821+0000","last_undegraded":"2023-10-18T09:02:43.364821+0000","last_fullsized":"2023-10-18T09:02:43.364821+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:01:27.410695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363553+0000","last_change":"2023-10-18T09:02:42.089943+0000","last_active":"2023-10-18T09:02:43.363553+0000","last_peered":"2023-10-18T09:02:43.363553+0000","last_clean":"2023-10-18T09:02:43.363553+0000","last_became_active":"2023-10-18T09:02:42.089539+0000","last_became_peered":"2023-10-18T09:02:42.089539+0000","last_unstale":"2023-10-18T09:02:43.363553+0000","last_undegraded":"2023-10-18T09:02:43.363553+0000","last_fullsized":"2023-10-18T09:02:43.363553+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:10:42.282131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363733+0000","last_change":"2023-10-18T09:02:41.320186+0000","last_active":"2023-10-18T09:02:43.363733+0000","last_peered":"2023-10-18T09:02:43.363733+0000","last_clean":"2023-10-18T09:02:43.363733+0000","last_became_active":"2023-10-18T09:02:41.318713+0000","last_became_peered":"2023-10-18T09:02:41.318713+0000","last_unstale":"2023-10-18T09:02:43.363733+0000","last_undegraded":"2023-10-18T09:02:43.363733+0000","last_fullsized":"2023-10-18T09:02:43.363733+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:02:55.396520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365550+0000","last_change":"2023-10-18T09:02:42.079283+0000","last_active":"2023-10-18T09:02:43.365550+0000","last_peered":"2023-10-18T09:02:43.365550+0000","last_clean":"2023-10-18T09:02:43.365550+0000","last_became_active":"2023-10-18T09:02:42.078582+0000","last_became_peered":"2023-10-18T09:02:42.078582+0000","last_unstale":"2023-10-18T09:02:43.365550+0000","last_undegraded":"2023-10-18T09:02:43.365550+0000","last_fullsized":"2023-10-18T09:02:43.365550+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:56:15.946301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365110+0000","last_change":"2023-10-18T09:02:41.325004+0000","last_active":"2023-10-18T09:02:43.365110+0000","last_peered":"2023-10-18T09:02:43.365110+0000","last_clean":"2023-10-18T09:02:43.365110+0000","last_became_active":"2023-10-18T09:02:41.324569+0000","last_became_peered":"2023-10-18T09:02:41.324569+0000","last_unstale":"2023-10-18T09:02:43.365110+0000","last_undegraded":"2023-10-18T09:02:43.365110+0000","last_fullsized":"2023-10-18T09:02:43.365110+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:08.235986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364615+0000","last_change":"2023-10-18T09:02:42.082257+0000","last_active":"2023-10-18T09:02:43.364615+0000","last_peered":"2023-10-18T09:02:43.364615+0000","last_clean":"2023-10-18T09:02:43.364615+0000","last_became_active":"2023-10-18T09:02:42.081448+0000","last_became_peered":"2023-10-18T09:02:42.081448+0000","last_unstale":"2023-10-18T09:02:43.364615+0000","last_undegraded":"2023-10-18T09:02:43.364615+0000","last_fullsized":"2023-10-18T09:02:43.364615+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:48:22.448169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"129'8","reported_seq":20,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363710+0000","last_change":"2023-10-18T09:02:41.314679+0000","last_active":"2023-10-18T09:02:43.363710+0000","last_peered":"2023-10-18T09:02:43.363710+0000","last_clean":"2023-10-18T09:02:43.363710+0000","last_became_active":"2023-10-18T09:02:41.314090+0000","last_became_peered":"2023-10-18T09:02:41.314090+0000","last_unstale":"2023-10-18T09:02:43.363710+0000","last_undegraded":"2023-10-18T09:02:43.363710+0000","last_fullsized":"2023-10-18T09:02:43.363710+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:50:48.824395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.058507+0000","last_change":"2023-10-18T09:02:42.079433+0000","last_active":"2023-10-18T09:02:43.058507+0000","last_peered":"2023-10-18T09:02:43.058507+0000","last_clean":"2023-10-18T09:02:43.058507+0000","last_became_active":"2023-10-18T09:02:42.079132+0000","last_became_peered":"2023-10-18T09:02:42.079132+0000","last_unstale":"2023-10-18T09:02:43.058507+0000","last_undegraded":"2023-10-18T09:02:43.058507+0000","last_fullsized":"2023-10-18T09:02:43.058507+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:20:48.771067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"129'2","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365515+0000","last_change":"2023-10-18T09:02:41.326713+0000","last_active":"2023-10-18T09:02:43.365515+0000","last_peered":"2023-10-18T09:02:43.365515+0000","last_clean":"2023-10-18T09:02:43.365515+0000","last_became_active":"2023-10-18T09:02:41.326367+0000","last_became_peered":"2023-10-18T09:02:41.326367+0000","last_unstale":"2023-10-18T09:02:43.365515+0000","last_undegraded":"2023-10-18T09:02:43.365515+0000","last_fullsized":"2023-10-18T09:02:43.365515+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:36:50.180955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364226+0000","last_change":"2023-10-18T09:02:42.070109+0000","last_active":"2023-10-18T09:02:43.364226+0000","last_peered":"2023-10-18T09:02:43.364226+0000","last_clean":"2023-10-18T09:02:43.364226+0000","last_became_active":"2023-10-18T09:02:42.069438+0000","last_became_peered":"2023-10-18T09:02:42.069438+0000","last_unstale":"2023-10-18T09:02:43.364226+0000","last_undegraded":"2023-10-18T09:02:43.364226+0000","last_fullsized":"2023-10-18T09:02:43.364226+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:40:40.941852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"129'1","reported_seq":13,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.059690+0000","last_change":"2023-10-18T09:02:41.321101+0000","last_active":"2023-10-18T09:02:43.059690+0000","last_peered":"2023-10-18T09:02:43.059690+0000","last_clean":"2023-10-18T09:02:43.059690+0000","last_became_active":"2023-10-18T09:02:41.320839+0000","last_became_peered":"2023-10-18T09:02:41.320839+0000","last_unstale":"2023-10-18T09:02:43.059690+0000","last_undegraded":"2023-10-18T09:02:43.059690+0000","last_fullsized":"2023-10-18T09:02:43.059690+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:44:24.756584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.061240+0000","last_change":"2023-10-18T09:02:42.086884+0000","last_active":"2023-10-18T09:02:43.061240+0000","last_peered":"2023-10-18T09:02:43.061240+0000","last_clean":"2023-10-18T09:02:43.061240+0000","last_became_active":"2023-10-18T09:02:42.085989+0000","last_became_peered":"2023-10-18T09:02:42.085989+0000","last_unstale":"2023-10-18T09:02:43.061240+0000","last_undegraded":"2023-10-18T09:02:43.061240+0000","last_fullsized":"2023-10-18T09:02:43.061240+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:44.776977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365106+0000","last_change":"2023-10-18T09:02:41.323827+0000","last_active":"2023-10-18T09:02:43.365106+0000","last_peered":"2023-10-18T09:02:43.365106+0000","last_clean":"2023-10-18T09:02:43.365106+0000","last_became_active":"2023-10-18T09:02:41.323562+0000","last_became_peered":"2023-10-18T09:02:41.323562+0000","last_unstale":"2023-10-18T09:02:43.365106+0000","last_undegraded":"2023-10-18T09:02:43.365106+0000","last_fullsized":"2023-10-18T09:02:43.365106+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:06.471711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365164+0000","last_change":"2023-10-18T09:02:42.085887+0000","last_active":"2023-10-18T09:02:43.365164+0000","last_peered":"2023-10-18T09:02:43.365164+0000","last_clean":"2023-10-18T09:02:43.365164+0000","last_became_active":"2023-10-18T09:02:42.083398+0000","last_became_peered":"2023-10-18T09:02:42.083398+0000","last_unstale":"2023-10-18T09:02:43.365164+0000","last_undegraded":"2023-10-18T09:02:43.365164+0000","last_fullsized":"2023-10-18T09:02:43.365164+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:59.346182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"129'2","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364839+0000","last_change":"2023-10-18T09:02:41.326500+0000","last_active":"2023-10-18T09:02:43.364839+0000","last_peered":"2023-10-18T09:02:43.364839+0000","last_clean":"2023-10-18T09:02:43.364839+0000","last_became_active":"2023-10-18T09:02:41.326167+0000","last_became_peered":"2023-10-18T09:02:41.326167+0000","last_unstale":"2023-10-18T09:02:43.364839+0000","last_undegraded":"2023-10-18T09:02:43.364839+0000","last_fullsized":"2023-10-18T09:02:43.364839+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:32.924193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364619+0000","last_change":"2023-10-18T09:02:42.086551+0000","last_active":"2023-10-18T09:02:43.364619+0000","last_peered":"2023-10-18T09:02:43.364619+0000","last_clean":"2023-10-18T09:02:43.364619+0000","last_became_active":"2023-10-18T09:02:42.086260+0000","last_became_peered":"2023-10-18T09:02:42.086260+0000","last_unstale":"2023-10-18T09:02:43.364619+0000","last_undegraded":"2023-10-18T09:02:43.364619+0000","last_fullsized":"2023-10-18T09:02:43.364619+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:57:47.246408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363679+0000","last_change":"2023-10-18T09:02:41.314198+0000","last_active":"2023-10-18T09:02:43.363679+0000","last_peered":"2023-10-18T09:02:43.363679+0000","last_clean":"2023-10-18T09:02:43.363679+0000","last_became_active":"2023-10-18T09:02:41.313600+0000","last_became_peered":"2023-10-18T09:02:41.313600+0000","last_unstale":"2023-10-18T09:02:43.363679+0000","last_undegraded":"2023-10-18T09:02:43.363679+0000","last_fullsized":"2023-10-18T09:02:43.363679+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:03:35.850431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363605+0000","last_change":"2023-10-18T09:02:42.080339+0000","last_active":"2023-10-18T09:02:43.363605+0000","last_peered":"2023-10-18T09:02:43.363605+0000","last_clean":"2023-10-18T09:02:43.363605+0000","last_became_active":"2023-10-18T09:02:42.079566+0000","last_became_peered":"2023-10-18T09:02:42.079566+0000","last_unstale":"2023-10-18T09:02:43.363605+0000","last_undegraded":"2023-10-18T09:02:43.363605+0000","last_fullsized":"2023-10-18T09:02:43.363605+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:41.023294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364126+0000","last_change":"2023-10-18T09:02:41.322158+0000","last_active":"2023-10-18T09:02:43.364126+0000","last_peered":"2023-10-18T09:02:43.364126+0000","last_clean":"2023-10-18T09:02:43.364126+0000","last_became_active":"2023-10-18T09:02:41.321849+0000","last_became_peered":"2023-10-18T09:02:41.321849+0000","last_unstale":"2023-10-18T09:02:43.364126+0000","last_undegraded":"2023-10-18T09:02:43.364126+0000","last_fullsized":"2023-10-18T09:02:43.364126+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:19:22.924072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.061109+0000","last_change":"2023-10-18T09:02:42.090016+0000","last_active":"2023-10-18T09:02:43.061109+0000","last_peered":"2023-10-18T09:02:43.061109+0000","last_clean":"2023-10-18T09:02:43.061109+0000","last_became_active":"2023-10-18T09:02:42.089666+0000","last_became_peered":"2023-10-18T09:02:42.089666+0000","last_unstale":"2023-10-18T09:02:43.061109+0000","last_undegraded":"2023-10-18T09:02:43.061109+0000","last_fullsized":"2023-10-18T09:02:43.061109+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:51.546758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365650+0000","last_change":"2023-10-18T09:02:41.321706+0000","last_active":"2023-10-18T09:02:43.365650+0000","last_peered":"2023-10-18T09:02:43.365650+0000","last_clean":"2023-10-18T09:02:43.365650+0000","last_became_active":"2023-10-18T09:02:41.315780+0000","last_became_peered":"2023-10-18T09:02:41.315780+0000","last_unstale":"2023-10-18T09:02:43.365650+0000","last_undegraded":"2023-10-18T09:02:43.365650+0000","last_fullsized":"2023-10-18T09:02:43.365650+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:39:26.937813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364379+0000","last_change":"2023-10-18T09:02:42.086641+0000","last_active":"2023-10-18T09:02:43.364379+0000","last_peered":"2023-10-18T09:02:43.364379+0000","last_clean":"2023-10-18T09:02:43.364379+0000","last_became_active":"2023-10-18T09:02:42.082674+0000","last_became_peered":"2023-10-18T09:02:42.082674+0000","last_unstale":"2023-10-18T09:02:43.364379+0000","last_undegraded":"2023-10-18T09:02:43.364379+0000","last_fullsized":"2023-10-18T09:02:43.364379+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:42:09.257183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"129'2","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364473+0000","last_change":"2023-10-18T09:02:41.323424+0000","last_active":"2023-10-18T09:02:43.364473+0000","last_peered":"2023-10-18T09:02:43.364473+0000","last_clean":"2023-10-18T09:02:43.364473+0000","last_became_active":"2023-10-18T09:02:41.323104+0000","last_became_peered":"2023-10-18T09:02:41.323104+0000","last_unstale":"2023-10-18T09:02:43.364473+0000","last_undegraded":"2023-10-18T09:02:43.364473+0000","last_fullsized":"2023-10-18T09:02:43.364473+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:41.805471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363723+0000","last_change":"2023-10-18T09:02:42.085022+0000","last_active":"2023-10-18T09:02:43.363723+0000","last_peered":"2023-10-18T09:02:43.363723+0000","last_clean":"2023-10-18T09:02:43.363723+0000","last_became_active":"2023-10-18T09:02:42.084618+0000","last_became_peered":"2023-10-18T09:02:42.084618+0000","last_unstale":"2023-10-18T09:02:43.363723+0000","last_undegraded":"2023-10-18T09:02:43.363723+0000","last_fullsized":"2023-10-18T09:02:43.363723+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:56:57.656254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364950+0000","last_change":"2023-10-18T09:02:41.326389+0000","last_active":"2023-10-18T09:02:43.364950+0000","last_peered":"2023-10-18T09:02:43.364950+0000","last_clean":"2023-10-18T09:02:43.364950+0000","last_became_active":"2023-10-18T09:02:41.325910+0000","last_became_peered":"2023-10-18T09:02:41.325910+0000","last_unstale":"2023-10-18T09:02:43.364950+0000","last_undegraded":"2023-10-18T09:02:43.364950+0000","last_fullsized":"2023-10-18T09:02:43.364950+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:14:46.185906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363755+0000","last_change":"2023-10-18T09:02:42.089791+0000","last_active":"2023-10-18T09:02:43.363755+0000","last_peered":"2023-10-18T09:02:43.363755+0000","last_clean":"2023-10-18T09:02:43.363755+0000","last_became_active":"2023-10-18T09:02:42.089011+0000","last_became_peered":"2023-10-18T09:02:42.089011+0000","last_unstale":"2023-10-18T09:02:43.363755+0000","last_undegraded":"2023-10-18T09:02:43.363755+0000","last_fullsized":"2023-10-18T09:02:43.363755+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:14:37.134014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.363549+0000","last_change":"2023-10-18T09:02:41.324279+0000","last_active":"2023-10-18T09:02:43.363549+0000","last_peered":"2023-10-18T09:02:43.363549+0000","last_clean":"2023-10-18T09:02:43.363549+0000","last_became_active":"2023-10-18T09:02:41.324023+0000","last_became_peered":"2023-10-18T09:02:41.324023+0000","last_unstale":"2023-10-18T09:02:43.363549+0000","last_undegraded":"2023-10-18T09:02:43.363549+0000","last_fullsized":"2023-10-18T09:02:43.363549+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:46.417340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.058846+0000","last_change":"2023-10-18T09:02:42.063326+0000","last_active":"2023-10-18T09:02:43.058846+0000","last_peered":"2023-10-18T09:02:43.058846+0000","last_clean":"2023-10-18T09:02:43.058846+0000","last_became_active":"2023-10-18T09:02:42.062523+0000","last_became_peered":"2023-10-18T09:02:42.062523+0000","last_unstale":"2023-10-18T09:02:43.058846+0000","last_undegraded":"2023-10-18T09:02:43.058846+0000","last_fullsized":"2023-10-18T09:02:43.058846+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:29:13.401772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.061308+0000","last_change":"2023-10-18T09:02:41.323581+0000","last_active":"2023-10-18T09:02:43.061308+0000","last_peered":"2023-10-18T09:02:43.061308+0000","last_clean":"2023-10-18T09:02:43.061308+0000","last_became_active":"2023-10-18T09:02:41.322732+0000","last_became_peered":"2023-10-18T09:02:41.322732+0000","last_unstale":"2023-10-18T09:02:43.061308+0000","last_undegraded":"2023-10-18T09:02:43.061308+0000","last_fullsized":"2023-10-18T09:02:43.061308+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:06.471711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364977+0000","last_change":"2023-10-18T09:02:42.082002+0000","last_active":"2023-10-18T09:02:43.364977+0000","last_peered":"2023-10-18T09:02:43.364977+0000","last_clean":"2023-10-18T09:02:43.364977+0000","last_became_active":"2023-10-18T09:02:42.080759+0000","last_became_peered":"2023-10-18T09:02:42.080759+0000","last_unstale":"2023-10-18T09:02:43.364977+0000","last_undegraded":"2023-10-18T09:02:43.364977+0000","last_fullsized":"2023-10-18T09:02:43.364977+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:36:19.783453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365760+0000","last_change":"2023-10-18T09:02:41.324636+0000","last_active":"2023-10-18T09:02:43.365760+0000","last_peered":"2023-10-18T09:02:43.365760+0000","last_clean":"2023-10-18T09:02:43.365760+0000","last_became_active":"2023-10-18T09:02:41.324274+0000","last_became_peered":"2023-10-18T09:02:41.324274+0000","last_unstale":"2023-10-18T09:02:43.365760+0000","last_undegraded":"2023-10-18T09:02:43.365760+0000","last_fullsized":"2023-10-18T09:02:43.365760+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:03:59.217823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365321+0000","last_change":"2023-10-18T09:02:42.085172+0000","last_active":"2023-10-18T09:02:43.365321+0000","last_peered":"2023-10-18T09:02:43.365321+0000","last_clean":"2023-10-18T09:02:43.365321+0000","last_became_active":"2023-10-18T09:02:42.083392+0000","last_became_peered":"2023-10-18T09:02:42.083392+0000","last_unstale":"2023-10-18T09:02:43.365321+0000","last_undegraded":"2023-10-18T09:02:43.365321+0000","last_fullsized":"2023-10-18T09:02:43.365321+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:49.808515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.059983+0000","last_change":"2023-10-18T09:02:41.323775+0000","last_active":"2023-10-18T09:02:43.059983+0000","last_peered":"2023-10-18T09:02:43.059983+0000","last_clean":"2023-10-18T09:02:43.059983+0000","last_became_active":"2023-10-18T09:02:41.319998+0000","last_became_peered":"2023-10-18T09:02:41.319998+0000","last_unstale":"2023-10-18T09:02:43.059983+0000","last_undegraded":"2023-10-18T09:02:43.059983+0000","last_fullsized":"2023-10-18T09:02:43.059983+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:22.375897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365325+0000","last_change":"2023-10-18T09:02:42.086084+0000","last_active":"2023-10-18T09:02:43.365325+0000","last_peered":"2023-10-18T09:02:43.365325+0000","last_clean":"2023-10-18T09:02:43.365325+0000","last_became_active":"2023-10-18T09:02:42.083915+0000","last_became_peered":"2023-10-18T09:02:42.083915+0000","last_unstale":"2023-10-18T09:02:43.365325+0000","last_undegraded":"2023-10-18T09:02:43.365325+0000","last_fullsized":"2023-10-18T09:02:43.365325+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:42.119673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.057291+0000","last_change":"2023-10-18T09:02:41.302865+0000","last_active":"2023-10-18T09:02:43.057291+0000","last_peered":"2023-10-18T09:02:43.057291+0000","last_clean":"2023-10-18T09:02:43.057291+0000","last_became_active":"2023-10-18T09:02:41.302547+0000","last_became_peered":"2023-10-18T09:02:41.302547+0000","last_unstale":"2023-10-18T09:02:43.057291+0000","last_undegraded":"2023-10-18T09:02:43.057291+0000","last_fullsized":"2023-10-18T09:02:43.057291+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:44:17.565558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.058820+0000","last_change":"2023-10-18T09:02:42.064831+0000","last_active":"2023-10-18T09:02:43.058820+0000","last_peered":"2023-10-18T09:02:43.058820+0000","last_clean":"2023-10-18T09:02:43.058820+0000","last_became_active":"2023-10-18T09:02:42.062648+0000","last_became_peered":"2023-10-18T09:02:42.062648+0000","last_unstale":"2023-10-18T09:02:43.058820+0000","last_undegraded":"2023-10-18T09:02:43.058820+0000","last_fullsized":"2023-10-18T09:02:43.058820+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:44.776977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.365434+0000","last_change":"2023-10-18T09:02:41.325536+0000","last_active":"2023-10-18T09:02:43.365434+0000","last_peered":"2023-10-18T09:02:43.365434+0000","last_clean":"2023-10-18T09:02:43.365434+0000","last_became_active":"2023-10-18T09:02:41.325256+0000","last_became_peered":"2023-10-18T09:02:41.325256+0000","last_unstale":"2023-10-18T09:02:43.365434+0000","last_undegraded":"2023-10-18T09:02:43.365434+0000","last_fullsized":"2023-10-18T09:02:43.365434+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:27:06.010665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.364344+0000","last_change":"2023-10-18T09:02:42.084253+0000","last_active":"2023-10-18T09:02:43.364344+0000","last_peered":"2023-10-18T09:02:43.364344+0000","last_clean":"2023-10-18T09:02:43.364344+0000","last_became_active":"2023-10-18T09:02:42.083866+0000","last_became_peered":"2023-10-18T09:02:42.083866+0000","last_unstale":"2023-10-18T09:02:43.364344+0000","last_undegraded":"2023-10-18T09:02:43.364344+0000","last_fullsized":"2023-10-18T09:02:43.364344+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:41.291348+0000","last_clean_scrub_stamp":"2023-10-18T09:02:41.291348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:47:59.013663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"129'1","reported_seq":13,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-18T09:02:43.058690+0000","last_change":"2023-10-18T09:02:41.316833+0000","last_active":"2023-10-18T09:02:43.058690+0000","last_peered":"2023-10-18T09:02:43.058690+0000","last_clean":"2023-10-18T09:02:43.058690+0000","last_became_active":"2023-10-18T09:02:41.316375+0000","last_became_peered":"2023-10-18T09:02:41.316375+0000","last_unstale":"2023-10-18T09:02:43.058690+0000","last_undegraded":"2023-10-18T09:02:43.058690+0000","last_fullsized":"2023-10-18T09:02:43.058690+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:02:40.275820+0000","last_clean_scrub_stamp":"2023-10-18T09:02:40.275820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:46.417340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":288,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-18T09:02:42.065370+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:02:42.065370+0000","last_peered":"2023-10-18T09:02:42.065370+0000","last_clean":"2023-10-18T09:02:42.065370+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:02:42.065370+0000","last_undegraded":"2023-10-18T09:02:42.065370+0000","last_fullsized":"2023-10-18T09:02:42.065370+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574942,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1287,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194025,"statfs":{"total":200000000,"available":198681819,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:02:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1240000000000001,"15min":1.036},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.60999999999999999},"max":{"1min":1.6040000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.129,"15min":1.046},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4450000000000001,"5min":1.97,"15min":1.97},"last":0.64500000000000002}]},{"osd":1,"last update":"Wed Oct 18 09:02:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.159,"15min":1.153},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.61799999999999999},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":1.004},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.151,"15min":1.129},"min":{"1min":0.75,"5min":0.61099999999999999,"15min":0.58799999999999997},"max":{"1min":1.504,"5min":1.7450000000000001,"15min":1.7989999999999999},"last":1.0640000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:02:18 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3300000000000001,"15min":1.258},"min":{"1min":0.85199999999999998,"5min":0.78600000000000003,"15min":0.65400000000000003},"max":{"1min":1.7490000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.157},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.284,"15min":1.218},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":1.345}]},{"osd":3,"last update":"Wed Oct 18 09:02:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.296,"15min":1.2709999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":1.097},{"interface":"front","average":{"1min":1.363,"5min":1.3740000000000001,"15min":1.333},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6990000000000001,"5min":1.873,"15min":1.9930000000000001},"last":0.97299999999999998}]},{"osd":4,"last update":"Wed Oct 18 09:02:18 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.256,"15min":1.1950000000000001},"min":{"1min":0.66000000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.923,"5min":1.923,"15min":2.1070000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.345,"5min":1.3220000000000001,"15min":1.3799999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.5169999999999999},"last":1.1839999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:02:18 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.353,"15min":1.353},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.77,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.4319999999999999,"15min":1.4239999999999999},"min":{"1min":0.748,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.956,"5min":2.0259999999999998,"15min":2.0270000000000001},"last":1.4430000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:02:18 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.498,"15min":1.472},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.3959999999999999},"last":1.399},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.4890000000000001,"15min":1.508},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.268}]}]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":711,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194601,"statfs":{"total":200000000,"available":199271647,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:02:10 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.242,"15min":1.1299999999999999},"min":{"1min":0.86199999999999999,"5min":0.623,"15min":0.60299999999999998},"max":{"1min":1.508,"5min":1.651,"15min":1.6850000000000001},"last":1.52},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.351,"15min":1.268},"min":{"1min":0.98399999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.6950000000000001,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.5960000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:02:10 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.121,"15min":1.0940000000000001},"min":{"1min":0.79200000000000004,"5min":0.65600000000000003,"15min":0.63800000000000001},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.246,"15min":1.2010000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.776,"5min":1.962,"15min":2.9609999999999999},"last":1.0529999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:02:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.1950000000000001,"15min":1.1779999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.5129999999999999,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.23,"5min":1.2609999999999999,"15min":1.2130000000000001},"min":{"1min":0.873,"5min":0.873,"15min":0.66300000000000003},"max":{"1min":1.5980000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.80000000000000004}]},{"osd":3,"last update":"Wed Oct 18 09:02:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2849999999999999,"15min":1.2649999999999999},"min":{"1min":0.81000000000000005,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.645,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.425},{"interface":"front","average":{"1min":1.345,"5min":1.292,"15min":1.2410000000000001},"min":{"1min":1.107,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.706,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.0940000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:02:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.2490000000000001,"15min":1.2},"min":{"1min":0.877,"5min":0.625,"15min":0.625},"max":{"1min":2.0230000000000001,"5min":2.444,"15min":2.444},"last":1.377},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.294,"15min":1.2789999999999999},"min":{"1min":0.97399999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.6859999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.268}]},{"osd":5,"last update":"Wed Oct 18 09:02:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4319999999999999,"15min":1.3779999999999999},"min":{"1min":1.1399999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.9670000000000001,"5min":2.032,"15min":2.032},"last":1.49},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.3,"15min":1.2470000000000001},"min":{"1min":0.98999999999999999,"5min":0.629,"15min":0.49199999999999999},"max":{"1min":1.6699999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.1990000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:02:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.417,"15min":1.4039999999999999},"min":{"1min":1.034,"5min":0.73999999999999999,"15min":0.61899999999999999},"max":{"1min":1.8029999999999999,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.466},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5249999999999999,"15min":1.528},"min":{"1min":1.2310000000000001,"5min":0.83099999999999996,"15min":0.68500000000000005},"max":{"1min":2.0800000000000001,"5min":2.3140000000000001,"15min":2.794},"last":1.6299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574943,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194601,"statfs":{"total":200000000,"available":199272229,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:02:17 2023","interfaces":[{"interface":"back","average":{"1min":1.6779999999999999,"5min":1.448,"15min":1.276},"min":{"1min":0.83099999999999996,"5min":0.68300000000000005,"15min":0.63400000000000001},"max":{"1min":9.609,"5min":9.609,"15min":9.609},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.333,"15min":1.3049999999999999},"min":{"1min":0.80000000000000004,"5min":0.71999999999999997,"15min":0.60799999999999998},"max":{"1min":1.623,"5min":5.3029999999999999,"15min":5.3029999999999999},"last":1.3560000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:02:17 2023","interfaces":[{"interface":"back","average":{"1min":1.857,"5min":1.345,"15min":1.2589999999999999},"min":{"1min":0.747,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":11.68,"5min":11.68,"15min":11.68},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2310000000000001,"15min":1.2390000000000001},"min":{"1min":0.97599999999999998,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.885,"5min":2.367,"15min":3.7050000000000001},"last":0.93200000000000005}]},{"osd":2,"last update":"Wed Oct 18 09:02:17 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2470000000000001,"15min":1.2609999999999999},"min":{"1min":0.79300000000000004,"5min":0.78100000000000003,"15min":0.755},"max":{"1min":1.732,"5min":2.0350000000000001,"15min":2.2090000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.262,"15min":1.282},"min":{"1min":0.95599999999999996,"5min":0.75900000000000001,"15min":0.72399999999999998},"max":{"1min":1.5549999999999999,"5min":2.3069999999999999,"15min":3.8180000000000001},"last":1.1339999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:02:17 2023","interfaces":[{"interface":"back","average":{"1min":1.7629999999999999,"5min":1.3500000000000001,"15min":1.2749999999999999},"min":{"1min":0.754,"5min":0.68799999999999994,"15min":0.65400000000000003},"max":{"1min":11.797000000000001,"5min":11.797000000000001,"15min":11.797000000000001},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.365,"15min":1.339},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.6699999999999999,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.3819999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:02:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3740000000000001,"15min":1.4039999999999999},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.75900000000000001},"max":{"1min":2.0680000000000001,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.298,"15min":1.452},"min":{"1min":0.83699999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6779999999999999,"5min":4.4429999999999996,"15min":5.4580000000000002},"last":1.0409999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:02:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.2849999999999999,"15min":1.3660000000000001},"min":{"1min":0.873,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.96,"5min":4.5469999999999997,"15min":5.141},"last":1.331},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.478,"15min":1.5509999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.71099999999999997},"max":{"1min":1.75,"5min":2.657,"15min":2.657},"last":1.54}]},{"osd":7,"last update":"Wed Oct 18 09:02:17 2023","interfaces":[{"interface":"back","average":{"1min":2.052,"5min":1.5860000000000001,"15min":1.6439999999999999},"min":{"1min":1.0580000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":11.829000000000001,"5min":11.829000000000001,"15min":11.829000000000001},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.423,"15min":1.4870000000000001},"min":{"1min":1.0189999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.613,"5min":4.4690000000000003,"15min":4.8419999999999996},"last":1.4179999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":711,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194601,"statfs":{"total":200000000,"available":199271582,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.1539999999999999,"15min":1.1240000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":3.1819999999999999,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.147,"15min":1.1679999999999999},"min":{"1min":0.63300000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":3.234,"5min":3.234,"15min":3.234},"last":0.94999999999999996}]},{"osd":1,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.153,"15min":1.115},"min":{"1min":0.76100000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":3.718,"5min":3.718,"15min":3.718},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.2010000000000001,"15min":1.226},"min":{"1min":0.71899999999999997,"5min":0.627,"15min":0.627},"max":{"1min":3.4350000000000001,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.359}]},{"osd":2,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.2370000000000001,"15min":1.196},"min":{"1min":0.90900000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.1749999999999998,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":1.296},{"interface":"front","average":{"1min":1.508,"5min":1.335,"15min":1.3149999999999999},"min":{"1min":1.0429999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":3.532,"5min":3.532,"15min":3.532},"last":1.1339999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.296,"15min":1.298},"min":{"1min":0.81100000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":3.363,"5min":3.363,"15min":3.363},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.4239999999999999,"15min":1.417},"min":{"1min":0.93500000000000005,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":4.3179999999999996,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":1.087}]},{"osd":5,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.2969999999999999,"15min":1.3009999999999999},"min":{"1min":0.86599999999999999,"5min":0.498,"15min":0.498},"max":{"1min":3.6160000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.3200000000000001,"15min":1.3540000000000001},"min":{"1min":0.97999999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":4.0780000000000003,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":1.2050000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.4350000000000001,"15min":1.4019999999999999},"min":{"1min":1.04,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":4.4429999999999996,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.417},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.486,"15min":1.546},"min":{"1min":0.98899999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":4.1239999999999997,"5min":4.407,"15min":4.407},"last":1.52}]},{"osd":7,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.4510000000000001,"15min":1.4990000000000001},"min":{"1min":0.98399999999999999,"5min":0.438,"15min":0.438},"max":{"1min":4.5170000000000003,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":1.052},{"interface":"front","average":{"1min":1.726,"5min":1.548,"15min":1.554},"min":{"1min":1.1879999999999999,"5min":0.747,"15min":0.747},"max":{"1min":4.5460000000000003,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":1.4830000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":712,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194600,"statfs":{"total":200000000,"available":199270665,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.88700000000000001,"15min":0.81599999999999995},"min":{"1min":0.40600000000000003,"5min":0.376,"15min":0.376},"max":{"1min":1.415,"5min":1.9590000000000001,"15min":5.5730000000000004},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.93999999999999995,"15min":0.90500000000000003},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.446,"5min":2.4569999999999999,"15min":6.5599999999999996},"last":0.83199999999999996}]},{"osd":1,"last update":"Wed Oct 18 09:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.999,"15min":0.96899999999999997},"min":{"1min":0.39100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.234,"5min":1.839,"15min":6.8099999999999996},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.2529999999999999,"15min":1.159},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.8080000000000001,"5min":2.0720000000000001,"15min":3.1920000000000002},"last":1.1220000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:02:15 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.052,"15min":1.036},"min":{"1min":0.46800000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.788,"5min":2.0600000000000001,"15min":7.4580000000000002},"last":1.163},{"interface":"front","average":{"1min":0.996,"5min":1.149,"15min":1.0880000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.51400000000000001},"max":{"1min":1.7450000000000001,"5min":3.169,"15min":3.5110000000000001},"last":1.706}]},{"osd":4,"last update":"Wed Oct 18 09:02:15 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.292,"15min":1.325},"min":{"1min":0.71899999999999997,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.8640000000000001,"5min":1.9259999999999999,"15min":18.664999999999999},"last":1.224},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.292,"15min":1.2010000000000001},"min":{"1min":0.878,"5min":0.69099999999999995,"15min":0.64000000000000001},"max":{"1min":1.837,"5min":2.093,"15min":7.0700000000000003},"last":1.0109999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:02:15 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.304,"15min":1.2529999999999999},"min":{"1min":0.77600000000000002,"5min":0.72399999999999998,"15min":0.67700000000000005},"max":{"1min":1.925,"5min":2.1680000000000001,"15min":6.9139999999999997},"last":1.081},{"interface":"front","average":{"1min":1.244,"5min":1.4119999999999999,"15min":1.3799999999999999},"min":{"1min":0.85199999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.843,"5min":2.0179999999999998,"15min":16.390999999999998},"last":1.4219999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:02:15 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.4490000000000001,"15min":1.4390000000000001},"min":{"1min":0.77200000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.7390000000000001,"5min":2.129,"15min":7.7270000000000003},"last":1.363},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.462,"15min":1.3799999999999999},"min":{"1min":0.872,"5min":0.77200000000000002,"15min":0.60199999999999998},"max":{"1min":1.8919999999999999,"5min":2.3759999999999999,"15min":7.242},"last":1.5169999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:02:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.4970000000000001,"15min":1.387},"min":{"1min":0.84099999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.2770000000000001,"5min":2.2770000000000001,"15min":4.7969999999999997},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.365,"5min":1.4910000000000001,"15min":1.4750000000000001},"min":{"1min":0.82099999999999995,"5min":0.622,"15min":0.622},"max":{"1min":1.9810000000000001,"5min":2.319,"15min":12.946},"last":1.6359999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574943,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194601,"statfs":{"total":200000000,"available":199272265,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:02:14 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89000000000000001,"15min":0.92000000000000004},"min":{"1min":0.65600000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.5660000000000001,"5min":1.706,"15min":4.2210000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.173,"5min":1.0329999999999999,"15min":1.044},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":4.3959999999999999,"5min":4.3959999999999999,"15min":4.3959999999999999},"last":0.92800000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:02:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.1200000000000001,"15min":1.0760000000000001},"min":{"1min":0.55900000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":7.3390000000000004,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1819999999999999,"15min":1.115},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":2.448,"5min":2.448,"15min":2.5350000000000001},"last":1.2849999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:02:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.1799999999999999,"15min":1.1439999999999999},"min":{"1min":0.61099999999999999,"5min":0.54600000000000004,"15min":0.48199999999999998},"max":{"1min":7.6369999999999996,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.3680000000000001,"15min":1.4450000000000001},"min":{"1min":0.80400000000000005,"5min":0.57899999999999996,"15min":0.54100000000000004},"max":{"1min":1.7649999999999999,"5min":2.2829999999999999,"15min":7.5570000000000004},"last":1.1870000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:02:14 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.4510000000000001,"15min":1.48},"min":{"1min":1.04,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.3809999999999998,"5min":2.4119999999999999,"15min":7.6189999999999998},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5009999999999999,"15min":1.464},"min":{"1min":1.0589999999999999,"5min":0.93799999999999994,"15min":0.86599999999999999},"max":{"1min":3.423,"5min":3.423,"15min":3.423},"last":1.601}]},{"osd":5,"last update":"Wed Oct 18 09:02:14 2023","interfaces":[{"interface":"back","average":{"1min":1.748,"5min":1.401,"15min":1.3109999999999999},"min":{"1min":0.93400000000000005,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":10.507,"5min":10.507,"15min":10.507},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.482,"15min":1.546},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.81399999999999995},"max":{"1min":1.772,"5min":2.536,"15min":7.657},"last":1.482}]},{"osd":6,"last update":"Wed Oct 18 09:02:14 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.52,"15min":1.5249999999999999},"min":{"1min":1.0489999999999999,"5min":0.82699999999999996,"15min":0.74299999999999999},"max":{"1min":2.8410000000000002,"5min":2.8780000000000001,"15min":3.5299999999999998},"last":1.52},{"interface":"front","average":{"1min":1.835,"5min":1.9910000000000001,"15min":1.6140000000000001},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.77600000000000002},"max":{"1min":10.561,"5min":37.771999999999998,"15min":37.771999999999998},"last":1.643}]},{"osd":7,"last update":"Wed Oct 18 09:02:14 2023","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.5820000000000001,"15min":1.5740000000000001},"min":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.83199999999999996},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":3.3620000000000001},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.9299999999999999,"5min":1.629,"15min":1.522},"min":{"1min":1.0149999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":10.956,"5min":10.956,"15min":10.956},"last":1.768}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":709,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194602,"statfs":{"total":200000000,"available":199273318,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:02:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.90800000000000003,"15min":0.83199999999999996},"min":{"1min":0.58699999999999997,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":1.266,"5min":1.5589999999999999,"15min":1.5960000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.069,"5min":1.079,"15min":0.94699999999999995},"min":{"1min":0.64100000000000001,"5min":0.501,"15min":0.443},"max":{"1min":1.597,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.645}]},{"osd":2,"last update":"Wed Oct 18 09:02:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.044,"15min":0.95699999999999996},"min":{"1min":0.64700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.5389999999999999,"5min":1.6899999999999999,"15min":1.859},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.95899999999999996,"15min":0.874},"min":{"1min":0.55600000000000005,"5min":0.44800000000000001,"15min":0.36699999999999999},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.90000000000000002}]},{"osd":3,"last update":"Wed Oct 18 09:02:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1060000000000001,"15min":1.0369999999999999},"min":{"1min":0.73499999999999999,"5min":0.49199999999999999,"15min":0.436},"max":{"1min":1.413,"5min":1.617,"15min":2.1059999999999999},"last":1.204},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.089,"15min":1.0580000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.472,"5min":1.9370000000000001,"15min":3.7010000000000001},"last":1.1140000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:02:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.333,"15min":1.266},"min":{"1min":0.93000000000000005,"5min":0.82099999999999995,"15min":0.59999999999999998},"max":{"1min":1.655,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.343,"15min":1.2390000000000001},"min":{"1min":0.997,"5min":0.754,"15min":0.754},"max":{"1min":1.893,"5min":1.903,"15min":2.0470000000000002},"last":1.512}]},{"osd":5,"last update":"Wed Oct 18 09:02:21 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.3959999999999999,"15min":1.3149999999999999},"min":{"1min":0.99299999999999999,"5min":0.86599999999999999,"15min":0.65400000000000003},"max":{"1min":1.7809999999999999,"5min":2.161,"15min":3.5750000000000002},"last":1.577},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.4650000000000001,"15min":1.381},"min":{"1min":1.0549999999999999,"5min":0.90500000000000003,"15min":0.83899999999999997},"max":{"1min":1.8140000000000001,"5min":1.964,"15min":2.3769999999999998},"last":1.55}]},{"osd":6,"last update":"Wed Oct 18 09:02:21 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.4179999999999999,"15min":1.3140000000000001},"min":{"1min":0.95599999999999996,"5min":0.752,"15min":0.752},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.2050000000000001},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4299999999999999,"15min":1.3029999999999999},"min":{"1min":1.1679999999999999,"5min":0.78900000000000003,"15min":0.63100000000000001},"max":{"1min":1.8540000000000001,"5min":2.121,"15min":2.121},"last":1.46}]},{"osd":7,"last update":"Wed Oct 18 09:02:21 2023","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.51,"15min":1.389},"min":{"1min":1.1839999999999999,"5min":0.84399999999999997,"15min":0.70499999999999996},"max":{"1min":1.913,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.71},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.5049999999999999,"15min":1.393},"min":{"1min":0.98199999999999998,"5min":0.82299999999999995,"15min":0.746},"max":{"1min":1.873,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.4219999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1286,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194025,"statfs":{"total":200000000,"available":198682368,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.83399999999999996,"15min":0.90600000000000003},"min":{"1min":0.442,"5min":0.371,"15min":0.37},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":3.4420000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.046,"5min":1.0920000000000001,"15min":1.085},"min":{"1min":0.56200000000000006,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.806,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.115}]},{"osd":2,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.876,"15min":0.85099999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.1459999999999999,"15min":1.1319999999999999},"min":{"1min":0.503,"5min":0.437,"15min":0.437},"max":{"1min":1.718,"5min":1.929,"15min":1.929},"last":1.2589999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.161,"15min":1.2649999999999999},"min":{"1min":0.54000000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.98,"5min":1.98,"15min":2.7639999999999998},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.4159999999999999,"15min":1.4950000000000001},"min":{"1min":0.61299999999999999,"5min":0.58499999999999996,"15min":0.58399999999999996},"max":{"1min":1.792,"5min":2.3639999999999999,"15min":3.5720000000000001},"last":1.3799999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.236,"15min":1.29},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.671,"5min":1.8819999999999999,"15min":4.492},"last":1.345},{"interface":"front","average":{"1min":1.29,"5min":1.2929999999999999,"15min":1.2629999999999999},"min":{"1min":0.879,"5min":0.879,"15min":0.72599999999999998},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.423}]},{"osd":5,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2809999999999999,"15min":1.3939999999999999},"min":{"1min":0.79100000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.7989999999999999,"5min":2.4510000000000001,"15min":4.2779999999999996},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3200000000000001,"15min":1.369},"min":{"1min":0.74399999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.847,"5min":1.847,"15min":1.8740000000000001},"last":1.294}]},{"osd":6,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.4470000000000001,"15min":1.5589999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.798,"5min":2.0419999999999998,"15min":3.5299999999999998},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.385,"15min":1.4790000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":3.1440000000000001,"5min":3.1440000000000001,"15min":4.4349999999999996},"last":1.1739999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:02:19 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.5469999999999999,"15min":1.6259999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.956,"5min":2.2000000000000002,"15min":3.492},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.4650000000000001,"15min":1.6359999999999999},"min":{"1min":0.86499999999999999,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":3.2570000000000001,"5min":3.2570000000000001,"15min":4.5419999999999998},"last":1.226}]}]}],"pool_statfs":[]}} 2023-10-18T09:02:47.368 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T09:02:47.369 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T09:02:47.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.648+0000 7ff500953640 1 -- 172.21.15.40:0/560411949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e8005180 msgr2=0x7ff4e800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:47.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.648+0000 7ff500953640 1 --2- 172.21.15.40:0/560411949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e8005180 0x7ff4e800b980 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc157500 tx=0x7ff4e80057f0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.648+0000 7ff500953640 1 -- 172.21.15.40:0/560411949 shutdown_connections 2023-10-18T09:02:47.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.648+0000 7ff500953640 1 --2- 172.21.15.40:0/560411949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e8005180 0x7ff4e800b980 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.648+0000 7ff500953640 1 -- 172.21.15.40:0/560411949 >> 172.21.15.40:0/560411949 conn(0x7ff4fc069720 msgr2=0x7ff4fc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:47.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.649+0000 7ff500953640 1 -- 172.21.15.40:0/560411949 shutdown_connections 2023-10-18T09:02:47.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.649+0000 7ff500953640 1 -- 172.21.15.40:0/560411949 wait complete. 2023-10-18T09:02:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.650+0000 7ff500953640 1 Processor -- start 2023-10-18T09:02:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.650+0000 7ff500953640 1 -- start start 2023-10-18T09:02:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.651+0000 7ff500953640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e8005180 0x7ff4fc1ef4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.651+0000 7ff500953640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4fc154400 0x7ff4fc156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.651+0000 7ff500953640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4fc156cb0 0x7ff4fc13eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:47.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.651+0000 7ff500953640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc1efa00 con 0x7ff4e8005180 2023-10-18T09:02:47.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.651+0000 7ff500953640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc1f3a20 con 0x7ff4fc156cb0 2023-10-18T09:02:47.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.651+0000 7ff500953640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc157170 con 0x7ff4fc154400 2023-10-18T09:02:47.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.652+0000 7ff4fa59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e8005180 0x7ff4fc1ef4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:47.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.652+0000 7ff4fa59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e8005180 0x7ff4fc1ef4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54554/0 (socket says 172.21.15.40:54554) 2023-10-18T09:02:47.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.652+0000 7ff4fa59c640 1 -- 172.21.15.40:0/1664044038 learned_addr learned my addr 172.21.15.40:0/1664044038 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:47.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.652+0000 7ff4fad9d640 1 --2- 172.21.15.40:0/1664044038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4fc156cb0 0x7ff4fc13eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:47.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.652+0000 7ff4f3fff640 1 --2- 172.21.15.40:0/1664044038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4fc154400 0x7ff4fc156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:47.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.652+0000 7ff4fa59c640 1 -- 172.21.15.40:0/1664044038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4fc154400 msgr2=0x7ff4fc156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:47.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.652+0000 7ff4fa59c640 1 --2- 172.21.15.40:0/1664044038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4fc154400 0x7ff4fc156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.652+0000 7ff4fa59c640 1 -- 172.21.15.40:0/1664044038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4fc156cb0 msgr2=0x7ff4fc13eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:47.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.652+0000 7ff4fa59c640 1 --2- 172.21.15.40:0/1664044038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4fc156cb0 0x7ff4fc13eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.652+0000 7ff4fa59c640 1 -- 172.21.15.40:0/1664044038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4fc1f30f0 con 0x7ff4e8005180 2023-10-18T09:02:47.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.652+0000 7ff4fa59c640 1 --2- 172.21.15.40:0/1664044038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e8005180 0x7ff4fc1ef4c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0117d0 tx=0x7ff4ec011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:47.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.653+0000 7ff4f2ffd640 1 -- 172.21.15.40:0/1664044038 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec011ee0 con 0x7ff4e8005180 2023-10-18T09:02:47.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.653+0000 7ff4f2ffd640 1 -- 172.21.15.40:0/1664044038 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff4ec00f940 con 0x7ff4e8005180 2023-10-18T09:02:47.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.653+0000 7ff4f2ffd640 1 -- 172.21.15.40:0/1664044038 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec00fcd0 con 0x7ff4e8005180 2023-10-18T09:02:47.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.653+0000 7ff500953640 1 -- 172.21.15.40:0/1664044038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4fc1f33d0 con 0x7ff4e8005180 2023-10-18T09:02:47.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.653+0000 7ff500953640 1 -- 172.21.15.40:0/1664044038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4fc1f3eb0 con 0x7ff4e8005180 2023-10-18T09:02:47.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.654+0000 7ff500953640 1 -- 172.21.15.40:0/1664044038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4fc066fd0 con 0x7ff4e8005180 2023-10-18T09:02:47.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.655+0000 7ff4f2ffd640 1 -- 172.21.15.40:0/1664044038 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff4ec0035f0 con 0x7ff4e8005180 2023-10-18T09:02:47.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.657+0000 7ff4f2ffd640 1 --2- 172.21.15.40:0/1664044038 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff4dc06ae40 0x7ff4dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:47.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.657+0000 7ff4f2ffd640 1 -- 172.21.15.40:0/1664044038 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4ec08a4a0 con 0x7ff4e8005180 2023-10-18T09:02:47.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.657+0000 7ff4f3fff640 1 --2- 172.21.15.40:0/1664044038 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff4dc06ae40 0x7ff4dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:47.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.658+0000 7ff4f3fff640 1 --2- 172.21.15.40:0/1664044038 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff4dc06ae40 0x7ff4dc06d300 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc15c930 tx=0x7ff4e40068e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:47.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.661+0000 7ff4f2ffd640 1 -- 172.21.15.40:0/1664044038 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff4ec0554b0 con 0x7ff4e8005180 2023-10-18T09:02:47.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.930+0000 7ff500953640 1 -- 172.21.15.40:0/1664044038 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff4fc0cbc70 con 0x7ff4e8005180 2023-10-18T09:02:47.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.935+0000 7ff4f2ffd640 1 -- 172.21.15.40:0/1664044038 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff4ec058ad0 con 0x7ff4e8005180 2023-10-18T09:02:47.937 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2023-10-18T09:02:47.937 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w== 2023-10-18T09:02:47.938 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2023-10-18T09:02:47.938 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2023-10-18T09:02:47.938 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T09:02:47.938 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2023-10-18T09:02:47.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.937+0000 7ff500953640 1 -- 172.21.15.40:0/1664044038 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff4dc06ae40 msgr2=0x7ff4dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:47.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.937+0000 7ff500953640 1 --2- 172.21.15.40:0/1664044038 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff4dc06ae40 0x7ff4dc06d300 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc15c930 tx=0x7ff4e40068e0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.937+0000 7ff500953640 1 -- 172.21.15.40:0/1664044038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e8005180 msgr2=0x7ff4fc1ef4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:47.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.937+0000 7ff500953640 1 --2- 172.21.15.40:0/1664044038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e8005180 0x7ff4fc1ef4c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0117d0 tx=0x7ff4ec011c90 comp rx=0 tx=0).stop 2023-10-18T09:02:47.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.938+0000 7ff500953640 1 -- 172.21.15.40:0/1664044038 shutdown_connections 2023-10-18T09:02:47.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.938+0000 7ff500953640 1 --2- 172.21.15.40:0/1664044038 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff4dc06ae40 0x7ff4dc06d300 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.938+0000 7ff500953640 1 --2- 172.21.15.40:0/1664044038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4fc156cb0 0x7ff4fc13eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.938+0000 7ff500953640 1 --2- 172.21.15.40:0/1664044038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4fc154400 0x7ff4fc156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.938+0000 7ff500953640 1 --2- 172.21.15.40:0/1664044038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4e8005180 0x7ff4fc1ef4c0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:47.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.938+0000 7ff500953640 1 -- 172.21.15.40:0/1664044038 >> 172.21.15.40:0/1664044038 conn(0x7ff4fc069720 msgr2=0x7ff4fc13b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:47.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.938+0000 7ff500953640 1 -- 172.21.15.40:0/1664044038 shutdown_connections 2023-10-18T09:02:47.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:47.938+0000 7ff500953640 1 -- 172.21.15.40:0/1664044038 wait complete. 2023-10-18T09:02:47.960 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:02:48.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.188+0000 7f969567c640 1 -- 172.21.15.40:0/3151905747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9678004d00 msgr2=0x7f9678005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:48.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.188+0000 7f969567c640 1 --2- 172.21.15.40:0/3151905747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9678004d00 0x7f9678005180 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f9688001d50 tx=0x7f96780057f0 comp rx=0 tx=0).stop 2023-10-18T09:02:48.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.188+0000 7f968f7fe640 1 -- 172.21.15.40:0/3151905747 reap_dead start 2023-10-18T09:02:48.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.189+0000 7f969567c640 1 -- 172.21.15.40:0/3151905747 shutdown_connections 2023-10-18T09:02:48.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.189+0000 7f969567c640 1 -- 172.21.15.40:0/3151905747 >> 172.21.15.40:0/3151905747 conn(0x7f969006cee0 msgr2=0x7f9690069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:48.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.189+0000 7f969567c640 1 -- 172.21.15.40:0/3151905747 shutdown_connections 2023-10-18T09:02:48.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.189+0000 7f969567c640 1 -- 172.21.15.40:0/3151905747 wait complete. 2023-10-18T09:02:48.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.190+0000 7f969567c640 1 Processor -- start 2023-10-18T09:02:48.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.191+0000 7f969567c640 1 -- start start 2023-10-18T09:02:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.191+0000 7f969567c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9678004d00 0x7f969006b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.191+0000 7f969567c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96900af0e0 0x7f9690140790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.192+0000 7f969567c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96900af560 0x7f9690140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:48.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.192+0000 7f969567c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96900aecc0 con 0x7f9678004d00 2023-10-18T09:02:48.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.192+0000 7f969567c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96900ae0d0 con 0x7f96900af560 2023-10-18T09:02:48.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.192+0000 7f969567c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96900ae320 con 0x7f96900af0e0 2023-10-18T09:02:48.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.192+0000 7f968effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9678004d00 0x7f969006b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:48.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.192+0000 7f968effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9678004d00 0x7f969006b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54582/0 (socket says 172.21.15.40:54582) 2023-10-18T09:02:48.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.192+0000 7f968effd640 1 -- 172.21.15.40:0/2189408387 learned_addr learned my addr 172.21.15.40:0/2189408387 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:48.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.192+0000 7f968e7fc640 1 --2- 172.21.15.40:0/2189408387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96900af0e0 0x7f9690140790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:48.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.192+0000 7f968f7fe640 1 --2- 172.21.15.40:0/2189408387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96900af560 0x7f9690140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:48.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.192+0000 7f968effd640 1 -- 172.21.15.40:0/2189408387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96900af0e0 msgr2=0x7f9690140790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:48.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.192+0000 7f968effd640 1 --2- 172.21.15.40:0/2189408387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96900af0e0 0x7f9690140790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:48.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.192+0000 7f968effd640 1 -- 172.21.15.40:0/2189408387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96900af560 msgr2=0x7f9690140cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:48.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.192+0000 7f968effd640 1 --2- 172.21.15.40:0/2189408387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96900af560 0x7f9690140cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:48.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.192+0000 7f968effd640 1 -- 172.21.15.40:0/2189408387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96900ae600 con 0x7f9678004d00 2023-10-18T09:02:48.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.193+0000 7f968effd640 1 --2- 172.21.15.40:0/2189408387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9678004d00 0x7f969006b980 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f968000ead0 tx=0x7f968000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:48.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.193+0000 7f96777fe640 1 -- 172.21.15.40:0/2189408387 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680013070 con 0x7f9678004d00 2023-10-18T09:02:48.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.193+0000 7f96777fe640 1 -- 172.21.15.40:0/2189408387 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f968000ddb0 con 0x7f9678004d00 2023-10-18T09:02:48.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.193+0000 7f96777fe640 1 -- 172.21.15.40:0/2189408387 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680017650 con 0x7f9678004d00 2023-10-18T09:02:48.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.193+0000 7f969567c640 1 -- 172.21.15.40:0/2189408387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96900ae8e0 con 0x7f9678004d00 2023-10-18T09:02:48.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.193+0000 7f969567c640 1 -- 172.21.15.40:0/2189408387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969006c0a0 con 0x7f9678004d00 2023-10-18T09:02:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.195+0000 7f969567c640 1 -- 172.21.15.40:0/2189408387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9690066fd0 con 0x7f9678004d00 2023-10-18T09:02:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.195+0000 7f96777fe640 1 -- 172.21.15.40:0/2189408387 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f96800177b0 con 0x7f9678004d00 2023-10-18T09:02:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.196+0000 7f96777fe640 1 --2- 172.21.15.40:0/2189408387 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f967c06ae70 0x7f967c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.197+0000 7f96777fe640 1 -- 172.21.15.40:0/2189408387 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9680089ad0 con 0x7f9678004d00 2023-10-18T09:02:48.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.197+0000 7f968e7fc640 1 --2- 172.21.15.40:0/2189408387 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f967c06ae70 0x7f967c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:48.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.198+0000 7f968e7fc640 1 --2- 172.21.15.40:0/2189408387 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f967c06ae70 0x7f967c06d330 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9684001ce0 tx=0x7f9684000ef0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:48.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.201+0000 7f96777fe640 1 -- 172.21.15.40:0/2189408387 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9680054c20 con 0x7f9678004d00 2023-10-18T09:02:48.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.437+0000 7f969567c640 1 -- 172.21.15.40:0/2189408387 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f969006c0a0 con 0x7f9678004d00 2023-10-18T09:02:48.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.438+0000 7f96777fe640 1 -- 172.21.15.40:0/2189408387 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f969006c0a0 con 0x7f9678004d00 2023-10-18T09:02:48.440 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:48.441 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/700924196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":700924196},{"type":"v1","addr":"172.21.15.40:6833","nonce":700924196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/877414874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":877414874},{"type":"v1","addr":"172.21.15.144:6835","nonce":877414874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2596943574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2596943574},{"type":"v1","addr":"172.21.15.144:6837","nonce":2596943574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:02:41.786627+0000","modified":"2023-10-18T09:02:44.862740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1801003551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1801003551},{"type":"v1","addr":"172.21.15.40:6835","nonce":1801003551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-18T09:02:48.441 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2023-10-18T09:02:48.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.441+0000 7f969567c640 1 -- 172.21.15.40:0/2189408387 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f967c06ae70 msgr2=0x7f967c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:48.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.441+0000 7f969567c640 1 --2- 172.21.15.40:0/2189408387 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f967c06ae70 0x7f967c06d330 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9684001ce0 tx=0x7f9684000ef0 comp rx=0 tx=0).stop 2023-10-18T09:02:48.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.441+0000 7f969567c640 1 -- 172.21.15.40:0/2189408387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9678004d00 msgr2=0x7f969006b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:48.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.441+0000 7f969567c640 1 --2- 172.21.15.40:0/2189408387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9678004d00 0x7f969006b980 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f968000ead0 tx=0x7f968000ef90 comp rx=0 tx=0).stop 2023-10-18T09:02:48.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.441+0000 7f969567c640 1 -- 172.21.15.40:0/2189408387 shutdown_connections 2023-10-18T09:02:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.441+0000 7f969567c640 1 --2- 172.21.15.40:0/2189408387 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f967c06ae70 0x7f967c06d330 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.441+0000 7f969567c640 1 --2- 172.21.15.40:0/2189408387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96900af560 0x7f9690140cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.441+0000 7f969567c640 1 --2- 172.21.15.40:0/2189408387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96900af0e0 0x7f9690140790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.441+0000 7f969567c640 1 --2- 172.21.15.40:0/2189408387 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9678004d00 0x7f969006b980 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.441+0000 7f969567c640 1 -- 172.21.15.40:0/2189408387 >> 172.21.15.40:0/2189408387 conn(0x7f969006cee0 msgr2=0x7f969015a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:48.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.441+0000 7f969567c640 1 -- 172.21.15.40:0/2189408387 shutdown_connections 2023-10-18T09:02:48.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:48.442+0000 7f969567c640 1 -- 172.21.15.40:0/2189408387 wait complete. 2023-10-18T09:02:48.866 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T09:02:41.786627+0000', 'modified': '2023-10-18T09:02:44.862740+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7449': {'gid': 7449, 'name': 'a', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/1801003551', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 1801003551}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 1801003551}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T09:02:48.867 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T09:02:48.867 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-18T09:02:49.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.112+0000 7f8d47fff640 1 -- 172.21.15.40:0/2002418141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d480d0020 msgr2=0x7f8d480ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:49.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.112+0000 7f8d47fff640 1 --2- 172.21.15.40:0/2002418141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d480d0020 0x7f8d480ae960 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8d40005fd0 tx=0x7f8d4002c1b0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.112+0000 7f8d47fff640 1 -- 172.21.15.40:0/2002418141 shutdown_connections 2023-10-18T09:02:49.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.112+0000 7f8d47fff640 1 --2- 172.21.15.40:0/2002418141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d480d0020 0x7f8d480ae960 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.112+0000 7f8d47fff640 1 --2- 172.21.15.40:0/2002418141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d4815b640 0x7f8d480d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.112+0000 7f8d47fff640 1 -- 172.21.15.40:0/2002418141 >> 172.21.15.40:0/2002418141 conn(0x7f8d4808ad00 msgr2=0x7f8d4809a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:49.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.112+0000 7f8d47fff640 1 -- 172.21.15.40:0/2002418141 shutdown_connections 2023-10-18T09:02:49.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.112+0000 7f8d47fff640 1 -- 172.21.15.40:0/2002418141 wait complete. 2023-10-18T09:02:49.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.114+0000 7f8d47fff640 1 Processor -- start 2023-10-18T09:02:49.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.114+0000 7f8d47fff640 1 -- start start 2023-10-18T09:02:49.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.115+0000 7f8d47fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d480b01e0 0x7f8d480b6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.115+0000 7f8d47fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d480d0020 0x7f8d480b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.115+0000 7f8d47fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d4815b640 0x7f8d480b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.115+0000 7f8d47fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4815cde0 con 0x7f8d4815b640 2023-10-18T09:02:49.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.115+0000 7f8d47fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480b6cd0 con 0x7f8d480d0020 2023-10-18T09:02:49.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.115+0000 7f8d47fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4815c1d0 con 0x7f8d480b01e0 2023-10-18T09:02:49.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.115+0000 7f8d477fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d4815b640 0x7f8d480b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.115+0000 7f8d477fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d4815b640 0x7f8d480b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54594/0 (socket says 172.21.15.40:54594) 2023-10-18T09:02:49.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.115+0000 7f8d477fe640 1 -- 172.21.15.40:0/666250838 learned_addr learned my addr 172.21.15.40:0/666250838 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:49.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.115+0000 7f8d46ffd640 1 --2- 172.21.15.40:0/666250838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d480b01e0 0x7f8d480b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.116+0000 7f8d477fe640 1 -- 172.21.15.40:0/666250838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d480b01e0 msgr2=0x7f8d480b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:49.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.116+0000 7f8d477fe640 1 --2- 172.21.15.40:0/666250838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d480b01e0 0x7f8d480b6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.116+0000 7f8d477fe640 1 -- 172.21.15.40:0/666250838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d480d0020 msgr2=0x7f8d480b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:49.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.116+0000 7f8d3ffff640 1 --2- 172.21.15.40:0/666250838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d480d0020 0x7f8d480b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.116+0000 7f8d477fe640 1 --2- 172.21.15.40:0/666250838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d480d0020 0x7f8d480b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.116+0000 7f8d477fe640 1 -- 172.21.15.40:0/666250838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d40009900 con 0x7f8d4815b640 2023-10-18T09:02:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.116+0000 7f8d46ffd640 1 --2- 172.21.15.40:0/666250838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d480b01e0 0x7f8d480b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:02:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.116+0000 7f8d3ffff640 1 --2- 172.21.15.40:0/666250838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d480d0020 0x7f8d480b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.116+0000 7f8d477fe640 1 --2- 172.21.15.40:0/666250838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d4815b640 0x7f8d480b4f40 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8d40037040 tx=0x7f8d40035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.117+0000 7f8d44ff9640 1 -- 172.21.15.40:0/666250838 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d40034440 con 0x7f8d4815b640 2023-10-18T09:02:49.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.117+0000 7f8d44ff9640 1 -- 172.21.15.40:0/666250838 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d40004250 con 0x7f8d4815b640 2023-10-18T09:02:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.117+0000 7f8d44ff9640 1 -- 172.21.15.40:0/666250838 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d40004600 con 0x7f8d4815b640 2023-10-18T09:02:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.117+0000 7f8d47fff640 1 -- 172.21.15.40:0/666250838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4815c4b0 con 0x7f8d4815b640 2023-10-18T09:02:49.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.117+0000 7f8d47fff640 1 -- 172.21.15.40:0/666250838 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4815ca70 con 0x7f8d4815b640 2023-10-18T09:02:49.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.119+0000 7f8d44ff9640 1 -- 172.21.15.40:0/666250838 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8d4003b040 con 0x7f8d4815b640 2023-10-18T09:02:49.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.121+0000 7f8d44ff9640 1 --2- 172.21.15.40:0/666250838 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d2806ae40 0x7f8d2806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.121+0000 7f8d44ff9640 1 -- 172.21.15.40:0/666250838 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d400ac3c0 con 0x7f8d4815b640 2023-10-18T09:02:49.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.121+0000 7f8d503ef740 1 Processor -- start 2023-10-18T09:02:49.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.121+0000 7f8d503ef740 1 -- start start 2023-10-18T09:02:49.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.121+0000 7f8d46ffd640 1 --2- 172.21.15.40:0/666250838 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d2806ae40 0x7f8d2806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.122+0000 7f8d503ef740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d365970 0x55d93d338ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.122+0000 7f8d503ef740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365d50 0x55d93d368700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.122+0000 7f8d503ef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d3390e0 0x55d93d368c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.122+0000 7f8d503ef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d93d2ab740 con 0x55d93d3390e0 2023-10-18T09:02:49.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.122+0000 7f8d503ef740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d93d2a8e30 con 0x55d93d365d50 2023-10-18T09:02:49.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.122+0000 7f8d503ef740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d93d2a4720 con 0x55d93d365970 2023-10-18T09:02:49.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.122+0000 7f8d477fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d3390e0 0x55d93d368c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.123+0000 7f8d477fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d3390e0 0x55d93d368c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54596/0 (socket says 172.21.15.40:54596) 2023-10-18T09:02:49.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.123+0000 7f8d477fe640 1 -- 172.21.15.40:0/1664453341 learned_addr learned my addr 172.21.15.40:0/1664453341 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:49.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.123+0000 7f8d46ffd640 1 --2- 172.21.15.40:0/666250838 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d2806ae40 0x7f8d2806d300 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8d38005fd0 tx=0x7f8d3800f360 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:49.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.123+0000 7f8d46ffd640 1 --2- 172.21.15.40:0/1664453341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d365970 0x55d93d338ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.123+0000 7f8d3ffff640 1 --2- 172.21.15.40:0/1664453341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365d50 0x55d93d368700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.123+0000 7f8d477fe640 1 -- 172.21.15.40:0/1664453341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d365970 msgr2=0x55d93d338ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:02:49.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.123+0000 7f8d477fe640 1 --2- 172.21.15.40:0/1664453341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d365970 0x55d93d338ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.123+0000 7f8d477fe640 1 -- 172.21.15.40:0/1664453341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365d50 msgr2=0x55d93d368700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:02:49.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.123+0000 7f8d477fe640 1 --2- 172.21.15.40:0/1664453341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365d50 0x55d93d368700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.123+0000 7f8d477fe640 1 -- 172.21.15.40:0/1664453341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d93d268910 con 0x55d93d3390e0 2023-10-18T09:02:49.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.123+0000 7f8d3ffff640 1 --2- 172.21.15.40:0/1664453341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365d50 0x55d93d368700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:49.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.123+0000 7f8d477fe640 1 --2- 172.21.15.40:0/1664453341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d3390e0 0x55d93d368c40 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8d40077590 tx=0x7f8d4007ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c296185d0de2146 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.123+0000 7f8d3d7fa640 1 -- 172.21.15.40:0/1664453341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4007b090 con 0x55d93d3390e0 2023-10-18T09:02:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.124+0000 7f8d3d7fa640 1 -- 172.21.15.40:0/1664453341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d4007f1e0 con 0x55d93d3390e0 2023-10-18T09:02:49.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.124+0000 7f8d3d7fa640 1 -- 172.21.15.40:0/1664453341 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d40083390 con 0x55d93d3390e0 2023-10-18T09:02:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.124+0000 7f8d503ef740 1 -- 172.21.15.40:0/1664453341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d3390e0 msgr2=0x55d93d368c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.124+0000 7f8d503ef740 1 --2- 172.21.15.40:0/1664453341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d3390e0 0x55d93d368c40 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8d40077590 tx=0x7f8d4007ac90 comp rx=0 tx=0).stop 2023-10-18T09:02:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.124+0000 7f8d503ef740 1 -- 172.21.15.40:0/1664453341 shutdown_connections 2023-10-18T09:02:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.124+0000 7f8d503ef740 1 --2- 172.21.15.40:0/1664453341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d3390e0 0x55d93d368c40 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.124+0000 7f8d503ef740 1 --2- 172.21.15.40:0/1664453341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365d50 0x55d93d368700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.124+0000 7f8d503ef740 1 --2- 172.21.15.40:0/1664453341 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d365970 0x55d93d338ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.124+0000 7f8d503ef740 1 -- 172.21.15.40:0/1664453341 >> 172.21.15.40:0/1664453341 conn(0x55d93d33ae10 msgr2=0x55d93d383700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:49.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.124+0000 7f8d503ef740 1 -- 172.21.15.40:0/1664453341 shutdown_connections 2023-10-18T09:02:49.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.124+0000 7f8d503ef740 1 -- 172.21.15.40:0/1664453341 wait complete. 2023-10-18T09:02:49.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.125+0000 7f8d503ef740 1 -- start start 2023-10-18T09:02:49.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.125+0000 7f8d503ef740 1 Processor -- start 2023-10-18T09:02:49.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d503ef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365970 0x55d93d3696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d503ef740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d365d50 0x55d93d37be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d503ef740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d3632c0 0x55d93d357680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d503ef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d93d242970 con 0x55d93d365970 2023-10-18T09:02:49.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d503ef740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d93d242610 con 0x55d93d3632c0 2023-10-18T09:02:49.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d503ef740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d93d008f20 con 0x55d93d365d50 2023-10-18T09:02:49.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d503ef740 10 client.0 _ll_register_callbacks cb 0x55d93d2660c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:02:49.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d46ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365970 0x55d93d3696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d46ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365970 0x55d93d3696f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54600/0 (socket says 172.21.15.40:54600) 2023-10-18T09:02:49.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d46ffd640 1 -- 172.21.15.40:0/3063235109 learned_addr learned my addr 172.21.15.40:0/3063235109 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:49.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d3ffff640 1 --2- 172.21.15.40:0/3063235109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d365d50 0x55d93d37be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d477fe640 1 --2- 172.21.15.40:0/3063235109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d3632c0 0x55d93d357680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d46ffd640 1 -- 172.21.15.40:0/3063235109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d365d50 msgr2=0x55d93d37be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:02:49.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d46ffd640 1 --2- 172.21.15.40:0/3063235109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d365d50 0x55d93d37be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d46ffd640 1 -- 172.21.15.40:0/3063235109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d3632c0 msgr2=0x55d93d357680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:02:49.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d46ffd640 1 --2- 172.21.15.40:0/3063235109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d3632c0 0x55d93d357680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d46ffd640 1 -- 172.21.15.40:0/3063235109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4815ca70 con 0x55d93d365970 2023-10-18T09:02:49.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.127+0000 7f8d477fe640 1 --2- 172.21.15.40:0/3063235109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d3632c0 0x55d93d357680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:49.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.128+0000 7f8d46ffd640 1 --2- 172.21.15.40:0/3063235109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365970 0x55d93d3696f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f8d38011470 tx=0x7f8d380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba37609faa769a74 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:49.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.128+0000 7f8d267fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T09:02:49.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.128+0000 7f8d267fc640 1 -- 172.21.15.40:0/3063235109 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3801f840 con 0x55d93d365970 2023-10-18T09:02:49.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.128+0000 7f8d267fc640 1 -- 172.21.15.40:0/3063235109 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d3801f9a0 con 0x55d93d365970 2023-10-18T09:02:49.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.128+0000 7f8d503ef740 1 -- 172.21.15.40:0/3063235109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d93d29d700 con 0x55d93d365970 2023-10-18T09:02:49.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.128+0000 7f8d267fc640 1 -- 172.21.15.40:0/3063235109 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d3801fb60 con 0x55d93d365970 2023-10-18T09:02:49.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.128+0000 7f8d503ef740 10 client.7596 fetch_fsmap learned FSMap version 57 2023-10-18T09:02:49.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.128+0000 7f8d503ef740 1 -- 172.21.15.40:0/3063235109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55d93d2a2da0 con 0x55d93d365970 2023-10-18T09:02:49.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.129+0000 7f8d267fc640 1 -- 172.21.15.40:0/3063235109 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55d93d29d700 con 0x55d93d365970 2023-10-18T09:02:49.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.129+0000 7f8d267fc640 1 -- 172.21.15.40:0/3063235109 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8d38025020 con 0x55d93d365970 2023-10-18T09:02:49.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.129+0000 7f8d503ef740 10 client.7596 fetch_fsmap finished waiting for FSMap version 57 2023-10-18T09:02:49.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.129+0000 7f8d503ef740 10 client.7596 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T09:02:49.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.129+0000 7f8d503ef740 20 client.7596 populate_metadata read hostname 'smithi040' 2023-10-18T09:02:49.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.129+0000 7f8d503ef740 1 --2- 172.21.15.40:0/3063235109 >> [v2:172.21.15.40:6834/1801003551,v1:172.21.15.40:6835/1801003551] conn(0x55d93d3937c0 0x55d93d37a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.129+0000 7f8d503ef740 4 client.7596 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:02:49.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.129+0000 7f8d503ef740 1 -- 172.21.15.40:0/3063235109 --> [v2:172.21.15.40:6834/1801003551,v1:172.21.15.40:6835/1801003551] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d93d29d420 con 0x55d93d3937c0 2023-10-18T09:02:49.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.129+0000 7f8d3ffff640 1 --2- 172.21.15.40:0/3063235109 >> [v2:172.21.15.40:6834/1801003551,v1:172.21.15.40:6835/1801003551] conn(0x55d93d3937c0 0x55d93d37a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.130+0000 7f8d3ffff640 1 --2- 172.21.15.40:0/3063235109 >> [v2:172.21.15.40:6834/1801003551,v1:172.21.15.40:6835/1801003551] conn(0x55d93d3937c0 0x55d93d37a320 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6b811ea6262a3d7 server_cookie=81e2136b3b4baeb3 in_seq=0 out_seq=0 2023-10-18T09:02:49.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.130+0000 7f8d267fc640 10 client.7596 ms_handle_connect on v2:172.21.15.40:6834/1801003551 2023-10-18T09:02:49.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.131+0000 7f8d267fc640 1 -- 172.21.15.40:0/3063235109 <== mds.0 v2:172.21.15.40:6834/1801003551 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d93d29d420 con 0x55d93d3937c0 2023-10-18T09:02:49.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.131+0000 7f8d267fc640 10 client.7596 handle_command_reply: tid=0 2023-10-18T09:02:49.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.131+0000 7f8d503ef740 1 client.7596 shutdown 2023-10-18T09:02:49.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.131+0000 7f8d503ef740 1 -- 172.21.15.40:0/3063235109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365970 msgr2=0x55d93d3696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:49.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.131+0000 7f8d503ef740 1 --2- 172.21.15.40:0/3063235109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365970 0x55d93d3696f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f8d38011470 tx=0x7f8d380046c0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.131+0000 7f8d503ef740 1 -- 172.21.15.40:0/3063235109 shutdown_connections 2023-10-18T09:02:49.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.131+0000 7f8d503ef740 1 --2- 172.21.15.40:0/3063235109 >> [v2:172.21.15.40:6834/1801003551,v1:172.21.15.40:6835/1801003551] conn(0x55d93d3937c0 0x55d93d37a320 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.131+0000 7f8d503ef740 1 --2- 172.21.15.40:0/3063235109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d3632c0 0x55d93d357680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.131+0000 7f8d503ef740 1 --2- 172.21.15.40:0/3063235109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d365d50 0x55d93d37be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.131+0000 7f8d503ef740 1 --2- 172.21.15.40:0/3063235109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365970 0x55d93d3696f0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.131+0000 7f8d267fc640 0 client.7596 ms_handle_reset on v2:172.21.15.40:6834/1801003551 2023-10-18T09:02:49.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.131+0000 7f8d503ef740 1 -- 172.21.15.40:0/3063235109 >> 172.21.15.40:0/3063235109 conn(0x55d93d3390e0 msgr2=0x55d93d33c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:49.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.131+0000 7f8d503ef740 1 -- 172.21.15.40:0/3063235109 shutdown_connections 2023-10-18T09:02:49.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.131+0000 7f8d503ef740 1 -- 172.21.15.40:0/3063235109 wait complete. 2023-10-18T09:02:49.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.131+0000 7f8d503ef740 20 client.7596 trim_cache size 0 max 16384 2023-10-18T09:02:49.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.148+0000 7f8d503ef740 1 Processor -- start 2023-10-18T09:02:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.149+0000 7f8d503ef740 1 -- start start 2023-10-18T09:02:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.149+0000 7f8d503ef740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d3632c0 0x55d93d33cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.149+0000 7f8d503ef740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365970 0x55d93d383c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.149+0000 7f8d503ef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365d50 0x55d93d386dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.149+0000 7f8d503ef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d93d384410 con 0x55d93d365d50 2023-10-18T09:02:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.149+0000 7f8d503ef740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d93d3399a0 con 0x55d93d365970 2023-10-18T09:02:49.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.149+0000 7f8d503ef740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d93d37bcc0 con 0x55d93d3632c0 2023-10-18T09:02:49.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.150+0000 7f8d477fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365d50 0x55d93d386dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.150+0000 7f8d477fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365d50 0x55d93d386dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54604/0 (socket says 172.21.15.40:54604) 2023-10-18T09:02:49.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.150+0000 7f8d477fe640 1 -- 172.21.15.40:0/2493234736 learned_addr learned my addr 172.21.15.40:0/2493234736 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:49.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.150+0000 7f8d3ffff640 1 --2- 172.21.15.40:0/2493234736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365970 0x55d93d383c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.150+0000 7f8d477fe640 1 -- 172.21.15.40:0/2493234736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d3632c0 msgr2=0x55d93d33cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:02:49.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.150+0000 7f8d477fe640 1 --2- 172.21.15.40:0/2493234736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d3632c0 0x55d93d33cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.150+0000 7f8d477fe640 1 -- 172.21.15.40:0/2493234736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365970 msgr2=0x55d93d383c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:02:49.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.150+0000 7f8d477fe640 1 --2- 172.21.15.40:0/2493234736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365970 0x55d93d383c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.150+0000 7f8d477fe640 1 -- 172.21.15.40:0/2493234736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d93d241b80 con 0x55d93d365d50 2023-10-18T09:02:49.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.150+0000 7f8d3ffff640 1 --2- 172.21.15.40:0/2493234736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365970 0x55d93d383c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:49.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.150+0000 7f8d477fe640 1 --2- 172.21.15.40:0/2493234736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365d50 0x55d93d386dc0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f8d4007af30 tx=0x7f8d4002c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b2497a0323f647 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:49.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.151+0000 7f8d3dffb640 1 -- 172.21.15.40:0/2493234736 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4007b090 con 0x55d93d365d50 2023-10-18T09:02:49.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.151+0000 7f8d3dffb640 1 -- 172.21.15.40:0/2493234736 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d40083390 con 0x55d93d365d50 2023-10-18T09:02:49.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.151+0000 7f8d3dffb640 1 -- 172.21.15.40:0/2493234736 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d40004760 con 0x55d93d365d50 2023-10-18T09:02:49.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.151+0000 7f8d503ef740 1 -- 172.21.15.40:0/2493234736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365d50 msgr2=0x55d93d386dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:49.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.151+0000 7f8d503ef740 1 --2- 172.21.15.40:0/2493234736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365d50 0x55d93d386dc0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f8d4007af30 tx=0x7f8d4002c1b0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.151+0000 7f8d503ef740 1 -- 172.21.15.40:0/2493234736 shutdown_connections 2023-10-18T09:02:49.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.151+0000 7f8d503ef740 1 --2- 172.21.15.40:0/2493234736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365d50 0x55d93d386dc0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.151+0000 7f8d503ef740 1 --2- 172.21.15.40:0/2493234736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365970 0x55d93d383c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.151+0000 7f8d503ef740 1 --2- 172.21.15.40:0/2493234736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d3632c0 0x55d93d33cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.151+0000 7f8d503ef740 1 -- 172.21.15.40:0/2493234736 >> 172.21.15.40:0/2493234736 conn(0x55d93d3390e0 msgr2=0x55d93d36be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:49.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.151+0000 7f8d503ef740 1 -- 172.21.15.40:0/2493234736 shutdown_connections 2023-10-18T09:02:49.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.151+0000 7f8d503ef740 1 -- 172.21.15.40:0/2493234736 wait complete. 2023-10-18T09:02:49.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.152+0000 7f8d503ef740 1 -- start start 2023-10-18T09:02:49.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.152+0000 7f8d503ef740 1 Processor -- start 2023-10-18T09:02:49.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.153+0000 7f8d503ef740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d3632c0 0x55d93d364b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.153+0000 7f8d503ef740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365970 0x55d93d365070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.153+0000 7f8d503ef740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365d50 0x55d93d379090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.153+0000 7f8d503ef740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d93d3857e0 con 0x55d93d365970 2023-10-18T09:02:49.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.153+0000 7f8d503ef740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d93d33db70 con 0x55d93d365d50 2023-10-18T09:02:49.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.153+0000 7f8d503ef740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d93d384b70 con 0x55d93d3632c0 2023-10-18T09:02:49.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.153+0000 7f8d503ef740 10 client.0 _ll_register_callbacks cb 0x7f8d40005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:02:49.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.154+0000 7f8d3ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365970 0x55d93d365070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.154+0000 7f8d3ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365970 0x55d93d365070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54616/0 (socket says 172.21.15.40:54616) 2023-10-18T09:02:49.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.154+0000 7f8d3ffff640 1 -- 172.21.15.40:0/3680553665 learned_addr learned my addr 172.21.15.40:0/3680553665 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:49.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.154+0000 7f8d46ffd640 1 --2- 172.21.15.40:0/3680553665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d3632c0 0x55d93d364b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.154+0000 7f8d477fe640 1 --2- 172.21.15.40:0/3680553665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365d50 0x55d93d379090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.154+0000 7f8d3ffff640 1 -- 172.21.15.40:0/3680553665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d3632c0 msgr2=0x55d93d364b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:02:49.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.154+0000 7f8d3ffff640 1 --2- 172.21.15.40:0/3680553665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d3632c0 0x55d93d364b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.154+0000 7f8d3ffff640 1 -- 172.21.15.40:0/3680553665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365d50 msgr2=0x55d93d379090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:02:49.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.154+0000 7f8d3ffff640 1 --2- 172.21.15.40:0/3680553665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365d50 0x55d93d379090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.154+0000 7f8d3ffff640 1 -- 172.21.15.40:0/3680553665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d93d268910 con 0x55d93d365970 2023-10-18T09:02:49.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.154+0000 7f8d3ffff640 1 --2- 172.21.15.40:0/3680553665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365970 0x55d93d365070 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f8d30001700 tx=0x7f8d30005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff1deafc0e03a5c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:49.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.154+0000 7f8d267fc640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T09:02:49.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.155+0000 7f8d267fc640 1 -- 172.21.15.40:0/3680553665 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30007440 con 0x55d93d365970 2023-10-18T09:02:49.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.155+0000 7f8d267fc640 1 -- 172.21.15.40:0/3680553665 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d3000c040 con 0x55d93d365970 2023-10-18T09:02:49.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.155+0000 7f8d503ef740 1 -- 172.21.15.40:0/3680553665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d4007f1e0 con 0x55d93d365970 2023-10-18T09:02:49.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.155+0000 7f8d267fc640 1 -- 172.21.15.40:0/3680553665 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d300075a0 con 0x55d93d365970 2023-10-18T09:02:49.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.155+0000 7f8d503ef740 10 client.7602 fetch_fsmap learned FSMap version 57 2023-10-18T09:02:49.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.155+0000 7f8d503ef740 1 -- 172.21.15.40:0/3680553665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55d93d267170 con 0x55d93d365970 2023-10-18T09:02:49.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.155+0000 7f8d267fc640 1 -- 172.21.15.40:0/3680553665 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d4007f1e0 con 0x55d93d365970 2023-10-18T09:02:49.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.155+0000 7f8d267fc640 1 -- 172.21.15.40:0/3680553665 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8d30012040 con 0x55d93d365970 2023-10-18T09:02:49.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.155+0000 7f8d503ef740 10 client.7602 fetch_fsmap finished waiting for FSMap version 57 2023-10-18T09:02:49.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.155+0000 7f8d503ef740 10 client.7602 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-18T09:02:49.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.155+0000 7f8d503ef740 20 client.7602 populate_metadata read hostname 'smithi040' 2023-10-18T09:02:49.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.155+0000 7f8d503ef740 1 --2- 172.21.15.40:0/3680553665 >> [v2:172.21.15.40:6834/1801003551,v1:172.21.15.40:6835/1801003551] conn(0x55d93d379720 0x55d93d379b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.155+0000 7f8d503ef740 4 client.7602 mds_command: new command op to 7449 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:02:49.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.155+0000 7f8d503ef740 1 -- 172.21.15.40:0/3680553665 --> [v2:172.21.15.40:6834/1801003551,v1:172.21.15.40:6835/1801003551] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d93cfa66c0 con 0x55d93d379720 2023-10-18T09:02:49.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.156+0000 7f8d46ffd640 1 --2- 172.21.15.40:0/3680553665 >> [v2:172.21.15.40:6834/1801003551,v1:172.21.15.40:6835/1801003551] conn(0x55d93d379720 0x55d93d379b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.156+0000 7f8d267fc640 10 client.7602 ms_handle_connect on v2:172.21.15.40:6834/1801003551 2023-10-18T09:02:49.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.156+0000 7f8d46ffd640 1 --2- 172.21.15.40:0/3680553665 >> [v2:172.21.15.40:6834/1801003551,v1:172.21.15.40:6835/1801003551] conn(0x55d93d379720 0x55d93d379b00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a4b30e093bf5f04 server_cookie=e80ae8a9a839bda6 in_seq=0 out_seq=0 2023-10-18T09:02:49.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.157+0000 7f8d267fc640 1 -- 172.21.15.40:0/3680553665 <== mds.0 v2:172.21.15.40:6834/1801003551 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+320 (crc 0 0 0) 0x55d93cfa66c0 con 0x55d93d379720 2023-10-18T09:02:49.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.157+0000 7f8d267fc640 10 client.7602 handle_command_reply: tid=0 2023-10-18T09:02:49.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.157+0000 7f8d503ef740 1 client.7602 shutdown 2023-10-18T09:02:49.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.157+0000 7f8d503ef740 1 -- 172.21.15.40:0/3680553665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365970 msgr2=0x55d93d365070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:49.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.157+0000 7f8d503ef740 1 --2- 172.21.15.40:0/3680553665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365970 0x55d93d365070 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f8d30001700 tx=0x7f8d30005480 comp rx=0 tx=0).stop 2023-10-18T09:02:49.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.157+0000 7f8d503ef740 1 -- 172.21.15.40:0/3680553665 shutdown_connections 2023-10-18T09:02:49.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.157+0000 7f8d503ef740 1 --2- 172.21.15.40:0/3680553665 >> [v2:172.21.15.40:6834/1801003551,v1:172.21.15.40:6835/1801003551] conn(0x55d93d379720 0x55d93d379b00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.157+0000 7f8d503ef740 1 --2- 172.21.15.40:0/3680553665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d93d365d50 0x55d93d379090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.157+0000 7f8d503ef740 1 --2- 172.21.15.40:0/3680553665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d93d365970 0x55d93d365070 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.157+0000 7f8d503ef740 1 --2- 172.21.15.40:0/3680553665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d93d3632c0 0x55d93d364b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.157+0000 7f8d503ef740 1 -- 172.21.15.40:0/3680553665 >> 172.21.15.40:0/3680553665 conn(0x55d93d3390e0 msgr2=0x55d93d337c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:49.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.157+0000 7f8d267fc640 0 client.7602 ms_handle_reset on v2:172.21.15.40:6834/1801003551 2023-10-18T09:02:49.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.157+0000 7f8d503ef740 1 -- 172.21.15.40:0/3680553665 shutdown_connections 2023-10-18T09:02:49.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.157+0000 7f8d503ef740 1 -- 172.21.15.40:0/3680553665 wait complete. 2023-10-18T09:02:49.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.158+0000 7f8d503ef740 20 client.7602 trim_cache size 0 max 16384 2023-10-18T09:02:49.180 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:02:49.180 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:02:49.180 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-10-18T09:02:49.180 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 7449, 2023-10-18T09:02:49.181 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-10-18T09:02:49.181 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-10-18T09:02:49.181 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-10-18T09:02:49.181 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.828602193, 2023-10-18T09:02:49.181 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 57, 2023-10-18T09:02:49.182 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 130, 2023-10-18T09:02:49.182 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 130, 2023-10-18T09:02:49.182 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 13.714908824 2023-10-18T09:02:49.182 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:02:49.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.158+0000 7f8d47fff640 1 -- 172.21.15.40:0/666250838 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d2806ae40 msgr2=0x7f8d2806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:49.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.158+0000 7f8d47fff640 1 --2- 172.21.15.40:0/666250838 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d2806ae40 0x7f8d2806d300 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8d38005fd0 tx=0x7f8d3800f360 comp rx=0 tx=0).stop 2023-10-18T09:02:49.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.158+0000 7f8d47fff640 1 -- 172.21.15.40:0/666250838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d4815b640 msgr2=0x7f8d480b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:49.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.158+0000 7f8d47fff640 1 --2- 172.21.15.40:0/666250838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d4815b640 0x7f8d480b4f40 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8d40037040 tx=0x7f8d40035db0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.158+0000 7f8d47fff640 1 -- 172.21.15.40:0/666250838 shutdown_connections 2023-10-18T09:02:49.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.158+0000 7f8d47fff640 1 --2- 172.21.15.40:0/666250838 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d2806ae40 0x7f8d2806d300 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.158+0000 7f8d47fff640 1 --2- 172.21.15.40:0/666250838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d4815b640 0x7f8d480b4f40 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.158+0000 7f8d47fff640 1 --2- 172.21.15.40:0/666250838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d480d0020 0x7f8d480b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.158+0000 7f8d47fff640 1 --2- 172.21.15.40:0/666250838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d480b01e0 0x7f8d480b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.158+0000 7f8d47fff640 1 -- 172.21.15.40:0/666250838 >> 172.21.15.40:0/666250838 conn(0x7f8d4808ad00 msgr2=0x7f8d4806bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:49.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.158+0000 7f8d47fff640 1 -- 172.21.15.40:0/666250838 shutdown_connections 2023-10-18T09:02:49.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.158+0000 7f8d47fff640 1 -- 172.21.15.40:0/666250838 wait complete. 2023-10-18T09:02:49.188 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T09:02:49.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.415+0000 7f51ce935640 1 -- 172.21.15.40:0/3044100276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c80ce590 msgr2=0x7f51c80d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:49.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.415+0000 7f51ce935640 1 --2- 172.21.15.40:0/3044100276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c80ce590 0x7f51c80d0f40 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f51b8005fd0 tx=0x7f51b802c240 comp rx=0 tx=0).stop 2023-10-18T09:02:49.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.415+0000 7f51ce935640 1 -- 172.21.15.40:0/3044100276 shutdown_connections 2023-10-18T09:02:49.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.416+0000 7f51ce935640 1 --2- 172.21.15.40:0/3044100276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c80ce590 0x7f51c80d0f40 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.416+0000 7f51ce935640 1 --2- 172.21.15.40:0/3044100276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51c815b360 0x7f51c80cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.416+0000 7f51ce935640 1 -- 172.21.15.40:0/3044100276 >> 172.21.15.40:0/3044100276 conn(0x7f51c808acc0 msgr2=0x7f51c809a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:49.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.416+0000 7f51ce935640 1 -- 172.21.15.40:0/3044100276 shutdown_connections 2023-10-18T09:02:49.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.416+0000 7f51ce935640 1 -- 172.21.15.40:0/3044100276 wait complete. 2023-10-18T09:02:49.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.418+0000 7f51ce935640 1 Processor -- start 2023-10-18T09:02:49.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.418+0000 7f51ce935640 1 -- start start 2023-10-18T09:02:49.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.419+0000 7f51ce935640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51c80ce590 0x7f51c8155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.419+0000 7f51ce935640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c80d1480 0x7f51c80c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.419+0000 7f51ce935640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c815b360 0x7f51c80cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.419+0000 7f51ce935640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c8155d70 con 0x7f51c815b360 2023-10-18T09:02:49.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.419+0000 7f51ce935640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c80c8ff0 con 0x7f51c80d1480 2023-10-18T09:02:49.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.419+0000 7f51ce935640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c80c83e0 con 0x7f51c80ce590 2023-10-18T09:02:49.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.420+0000 7f51ce134640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c815b360 0x7f51c80cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.420+0000 7f51ce134640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c815b360 0x7f51c80cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54628/0 (socket says 172.21.15.40:54628) 2023-10-18T09:02:49.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.420+0000 7f51ce134640 1 -- 172.21.15.40:0/3332319792 learned_addr learned my addr 172.21.15.40:0/3332319792 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:49.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.420+0000 7f51cd933640 1 --2- 172.21.15.40:0/3332319792 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51c80ce590 0x7f51c8155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.420+0000 7f51cd132640 1 --2- 172.21.15.40:0/3332319792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c80d1480 0x7f51c80c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.420+0000 7f51ce134640 1 -- 172.21.15.40:0/3332319792 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51c80ce590 msgr2=0x7f51c8155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:49.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.420+0000 7f51ce134640 1 --2- 172.21.15.40:0/3332319792 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51c80ce590 0x7f51c8155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.420+0000 7f51ce134640 1 -- 172.21.15.40:0/3332319792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c80d1480 msgr2=0x7f51c80c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:49.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.420+0000 7f51ce134640 1 --2- 172.21.15.40:0/3332319792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c80d1480 0x7f51c80c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.420+0000 7f51ce134640 1 -- 172.21.15.40:0/3332319792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b8009900 con 0x7f51c815b360 2023-10-18T09:02:49.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.421+0000 7f51ce134640 1 --2- 172.21.15.40:0/3332319792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c815b360 0x7f51c80cc3c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f51c000c410 tx=0x7f51c000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:49.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.421+0000 7f51be7fc640 1 -- 172.21.15.40:0/3332319792 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c0011790 con 0x7f51c815b360 2023-10-18T09:02:49.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.421+0000 7f51be7fc640 1 -- 172.21.15.40:0/3332319792 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f51c00119e0 con 0x7f51c815b360 2023-10-18T09:02:49.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.421+0000 7f51be7fc640 1 -- 172.21.15.40:0/3332319792 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c0011df0 con 0x7f51c815b360 2023-10-18T09:02:49.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.421+0000 7f51ce935640 1 -- 172.21.15.40:0/3332319792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c80c86c0 con 0x7f51c815b360 2023-10-18T09:02:49.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.421+0000 7f51ce935640 1 -- 172.21.15.40:0/3332319792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c80c8c80 con 0x7f51c815b360 2023-10-18T09:02:49.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.422+0000 7f51ce935640 1 -- 172.21.15.40:0/3332319792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c8071da0 con 0x7f51c815b360 2023-10-18T09:02:49.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.423+0000 7f51be7fc640 1 -- 172.21.15.40:0/3332319792 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f51c001c460 con 0x7f51c815b360 2023-10-18T09:02:49.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.424+0000 7f51be7fc640 1 --2- 172.21.15.40:0/3332319792 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f51ac06ae40 0x7f51ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:49.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.424+0000 7f51be7fc640 1 -- 172.21.15.40:0/3332319792 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51c0089600 con 0x7f51c815b360 2023-10-18T09:02:49.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.425+0000 7f51cd933640 1 --2- 172.21.15.40:0/3332319792 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f51ac06ae40 0x7f51ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:49.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.426+0000 7f51cd933640 1 --2- 172.21.15.40:0/3332319792 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f51ac06ae40 0x7f51ac06d300 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f51c4005fd0 tx=0x7f51c4005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:49.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.429+0000 7f51be7fc640 1 -- 172.21.15.40:0/3332319792 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51c0054610 con 0x7f51c815b360 2023-10-18T09:02:49.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.663+0000 7f51ce935640 1 -- 172.21.15.40:0/3332319792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f51c8066e00 con 0x7f51c815b360 2023-10-18T09:02:49.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.664+0000 7f51be7fc640 1 -- 172.21.15.40:0/3332319792 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7f51c0057c30 con 0x7f51c815b360 2023-10-18T09:02:49.667 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T09:02:49.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.667+0000 7f51ce935640 1 -- 172.21.15.40:0/3332319792 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f51ac06ae40 msgr2=0x7f51ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:49.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.667+0000 7f51ce935640 1 --2- 172.21.15.40:0/3332319792 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f51ac06ae40 0x7f51ac06d300 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f51c4005fd0 tx=0x7f51c4005850 comp rx=0 tx=0).stop 2023-10-18T09:02:49.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.667+0000 7f51ce935640 1 -- 172.21.15.40:0/3332319792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c815b360 msgr2=0x7f51c80cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:49.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.667+0000 7f51ce935640 1 --2- 172.21.15.40:0/3332319792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c815b360 0x7f51c80cc3c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f51c000c410 tx=0x7f51c000c8d0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.667+0000 7f51ce935640 1 -- 172.21.15.40:0/3332319792 shutdown_connections 2023-10-18T09:02:49.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.667+0000 7f51ce935640 1 --2- 172.21.15.40:0/3332319792 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f51ac06ae40 0x7f51ac06d300 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.667+0000 7f51ce935640 1 --2- 172.21.15.40:0/3332319792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c815b360 0x7f51c80cc3c0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.667+0000 7f51ce935640 1 --2- 172.21.15.40:0/3332319792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51c80d1480 0x7f51c80c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.667+0000 7f51ce935640 1 --2- 172.21.15.40:0/3332319792 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51c80ce590 0x7f51c8155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.667+0000 7f51ce935640 1 -- 172.21.15.40:0/3332319792 >> 172.21.15.40:0/3332319792 conn(0x7f51c808acc0 msgr2=0x7f51c815a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.667+0000 7f51ce935640 1 -- 172.21.15.40:0/3332319792 shutdown_connections 2023-10-18T09:02:49.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:49.667+0000 7f51ce935640 1 -- 172.21.15.40:0/3332319792 wait complete. 2023-10-18T09:02:49.687 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T09:02:49.688 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T09:02:49.688 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-10-18T09:02:49.689 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T09:02:49.689 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T09:02:49.689 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T09:02:49.690 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T09:02:49.690 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T09:02:49.691 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:02:49.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:49.692 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-10-18T09:02:49.745 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T09:02:49.745 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T09:02:49.746 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T09:02:49.746 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:02:49.746 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-18T09:02:49.746 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:02:49.746 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:02:49.747 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:02:49.747 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-10-18T09:02:49.747 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:02:49.747 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:02:49.747 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-10-18T09:02:49.748 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T09:02:49.748 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-10-18T09:02:49.748 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-10-18T09:02:49.748 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-18T09:02:49.748 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 41955sec preferred_lft 41955sec 2023-10-18T09:02:49.749 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2023-10-18T09:02:49.749 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:02:49.749 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:02:49.749 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-10-18T09:02:49.749 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-10-18T09:02:49.750 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T09:02:49.750 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:02:49.750 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T09:02:49.750 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:02:49.751 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2026:95ff:fea9:a463/64 scope link 2023-10-18T09:02:49.751 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:02:49.751 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:02:49.751 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T09:02:49.752 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::80bd:e5ff:fe0f:6f68/64 scope link 2023-10-18T09:02:49.752 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:02:49.752 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:02:49.752 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether ea:2b:21:f5:06:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T09:02:49.752 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::e82b:21ff:fef5:6af/64 scope link 2023-10-18T09:02:49.752 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:02:49.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:49.754 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T09:02:49.856 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-10-18T09:02:49.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:49.861 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-10-18T09:02:49.922 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-10-18T09:02:49.922 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T09:02:49.922 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-18T09:02:49.922 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-18T09:02:49.923 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T09:02:49.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:49.924 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T09:02:49.924 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T09:02:49.924 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-18T09:02:49.925 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-18T09:02:49.925 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-18T09:02:49.925 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T09:02:50.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:50.130 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-18T09:02:50.182 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T09:02:50.182 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T09:02:50.183 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:02:50.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:50.184 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-10-18T09:02:50.279 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:02:50.280 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T09:02:50.338 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:02:50.339 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:02:50.339 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T09:02:50.395 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-18T09:02:50.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:50.396 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:02:50.547 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:02:50.547 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:02:50.582 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:02:50.583 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:02:50.583 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T09:02:50.583 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T09:02:50.583 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T09:02:50.585 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-18T09:02:50.585 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2023-10-18T09:02:50.585 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T09:02:50.585 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T09:02:50.586 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T09:02:50.589 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T09:02:50.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:50.590 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T09:02:50.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:50.685 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:02:50.759 INFO:teuthology.orchestra.run.smithi040.stdout:client6193 2023-10-18T09:02:50.760 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:02:50.760 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:02:50.818 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T09:02:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.051+0000 7f7ef5a59640 1 -- 172.21.15.40:0/2070620893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef00b0070 msgr2=0x7f7ef006b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.051+0000 7f7ef5a59640 1 --2- 172.21.15.40:0/2070620893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef00b0070 0x7f7ef006b4d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0066610 tx=0x7f7ee802bcf0 comp rx=0 tx=0).stop 2023-10-18T09:02:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.051+0000 7f7ef5a59640 1 -- 172.21.15.40:0/2070620893 shutdown_connections 2023-10-18T09:02:51.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.051+0000 7f7ef5a59640 1 --2- 172.21.15.40:0/2070620893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef00b0070 0x7f7ef006b4d0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:51.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.051+0000 7f7ef5a59640 1 --2- 172.21.15.40:0/2070620893 >> v2:172.21.15.144:3300/0 conn(0x7f7ef00b0dd0 0x7f7ef015a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:51.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.051+0000 7f7ef5a59640 1 -- 172.21.15.40:0/2070620893 >> 172.21.15.40:0/2070620893 conn(0x7f7ef00aed70 msgr2=0x7f7ef00af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:51.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.051+0000 7f7ef5a59640 1 -- 172.21.15.40:0/2070620893 shutdown_connections 2023-10-18T09:02:51.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.051+0000 7f7ef5a59640 1 -- 172.21.15.40:0/2070620893 wait complete. 2023-10-18T09:02:51.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.053+0000 7f7ef5a59640 1 Processor -- start 2023-10-18T09:02:51.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.053+0000 7f7ef5a59640 1 -- start start 2023-10-18T09:02:51.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.054+0000 7f7ef5a59640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef00b0070 0x7f7ef0158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:51.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.054+0000 7f7ef5a59640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ef00b0dd0 0x7f7ef00c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:51.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.054+0000 7f7ef5a59640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ef015a990 0x7f7ef00ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:51.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.054+0000 7f7ef5a59640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef00c6d00 con 0x7f7ef00b0070 2023-10-18T09:02:51.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.055+0000 7f7ef5a59640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef01591a0 con 0x7f7ef00b0dd0 2023-10-18T09:02:51.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.055+0000 7f7ef5a59640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef0135980 con 0x7f7ef015a990 2023-10-18T09:02:51.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.055+0000 7f7ef4a57640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef00b0070 0x7f7ef0158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:51.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.055+0000 7f7ef4a57640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef00b0070 0x7f7ef0158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54644/0 (socket says 172.21.15.40:54644) 2023-10-18T09:02:51.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.055+0000 7f7ef4a57640 1 -- 172.21.15.40:0/3623607814 learned_addr learned my addr 172.21.15.40:0/3623607814 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:51.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.055+0000 7f7ef4a57640 1 -- 172.21.15.40:0/3623607814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ef015a990 msgr2=0x7f7ef00ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:51.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.055+0000 7f7ef4a57640 1 --2- 172.21.15.40:0/3623607814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ef015a990 0x7f7ef00ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:51.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.055+0000 7f7ef4a57640 1 -- 172.21.15.40:0/3623607814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ef00b0dd0 msgr2=0x7f7ef00c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:51.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.055+0000 7f7ef4a57640 1 --2- 172.21.15.40:0/3623607814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ef00b0dd0 0x7f7ef00c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:51.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.055+0000 7f7ef4a57640 1 -- 172.21.15.40:0/3623607814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee8009490 con 0x7f7ef00b0070 2023-10-18T09:02:51.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.055+0000 7f7ef4a57640 1 --2- 172.21.15.40:0/3623607814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef00b0070 0x7f7ef0158c60 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000eab0 tx=0x7f7ee000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:51.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.056+0000 7f7eed7fa640 1 -- 172.21.15.40:0/3623607814 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee000ce40 con 0x7f7ef00b0070 2023-10-18T09:02:51.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.056+0000 7f7eed7fa640 1 -- 172.21.15.40:0/3623607814 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7ee0011560 con 0x7f7ef00b0070 2023-10-18T09:02:51.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.056+0000 7f7eed7fa640 1 -- 172.21.15.40:0/3623607814 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0011910 con 0x7f7ef00b0070 2023-10-18T09:02:51.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.056+0000 7f7ef5a59640 1 -- 172.21.15.40:0/3623607814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef0135c60 con 0x7f7ef00b0070 2023-10-18T09:02:51.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.056+0000 7f7ef5a59640 1 -- 172.21.15.40:0/3623607814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef0136220 con 0x7f7ef00b0070 2023-10-18T09:02:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.057+0000 7f7ef5a59640 1 -- 172.21.15.40:0/3623607814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef0066e00 con 0x7f7ef00b0070 2023-10-18T09:02:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.058+0000 7f7eed7fa640 1 -- 172.21.15.40:0/3623607814 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7ee0011a70 con 0x7f7ef00b0070 2023-10-18T09:02:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.060+0000 7f7eed7fa640 1 --2- 172.21.15.40:0/3623607814 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ed006ae40 0x7f7ed006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.060+0000 7f7eed7fa640 1 -- 172.21.15.40:0/3623607814 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ee008a420 con 0x7f7ef00b0070 2023-10-18T09:02:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.060+0000 7f7eeffff640 1 --2- 172.21.15.40:0/3623607814 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ed006ae40 0x7f7ed006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:51.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.062+0000 7f7eeffff640 1 --2- 172.21.15.40:0/3623607814 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ed006ae40 0x7f7ed006d300 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00abfa0 tx=0x7f7ed8007930 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:51.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.065+0000 7f7eed7fa640 1 -- 172.21.15.40:0/3623607814 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ee0058ec0 con 0x7f7ef00b0070 2023-10-18T09:02:51.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.296+0000 7f7ef5a59640 1 -- 172.21.15.40:0/3623607814 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7ef0159d60 con 0x7f7ef00b0070 2023-10-18T09:02:51.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.296+0000 7f7eed7fa640 1 -- 172.21.15.40:0/3623607814 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7ee0058a50 con 0x7f7ef00b0070 2023-10-18T09:02:51.299 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T09:02:51.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.299+0000 7f7ef5a59640 1 -- 172.21.15.40:0/3623607814 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ed006ae40 msgr2=0x7f7ed006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:51.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.299+0000 7f7ef5a59640 1 --2- 172.21.15.40:0/3623607814 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ed006ae40 0x7f7ed006d300 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00abfa0 tx=0x7f7ed8007930 comp rx=0 tx=0).stop 2023-10-18T09:02:51.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.299+0000 7f7ef5a59640 1 -- 172.21.15.40:0/3623607814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef00b0070 msgr2=0x7f7ef0158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:51.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.299+0000 7f7ef5a59640 1 --2- 172.21.15.40:0/3623607814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef00b0070 0x7f7ef0158c60 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000eab0 tx=0x7f7ee000ef70 comp rx=0 tx=0).stop 2023-10-18T09:02:51.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.299+0000 7f7ef5a59640 1 -- 172.21.15.40:0/3623607814 shutdown_connections 2023-10-18T09:02:51.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.299+0000 7f7ef5a59640 1 --2- 172.21.15.40:0/3623607814 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ed006ae40 0x7f7ed006d300 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:51.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.299+0000 7f7ef5a59640 1 --2- 172.21.15.40:0/3623607814 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ef015a990 0x7f7ef00ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:51.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.299+0000 7f7ef5a59640 1 --2- 172.21.15.40:0/3623607814 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ef00b0dd0 0x7f7ef00c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:51.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.299+0000 7f7ef5a59640 1 --2- 172.21.15.40:0/3623607814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ef00b0070 0x7f7ef0158c60 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:51.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.299+0000 7f7ef5a59640 1 -- 172.21.15.40:0/3623607814 >> 172.21.15.40:0/3623607814 conn(0x7f7ef00aed70 msgr2=0x7f7ef006f810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:51.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.299+0000 7f7ef5a59640 1 -- 172.21.15.40:0/3623607814 shutdown_connections 2023-10-18T09:02:51.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:51.299+0000 7f7ef5a59640 1 -- 172.21.15.40:0/3623607814 wait complete. 2023-10-18T09:02:51.320 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T09:02:51.321 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T09:02:51.321 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-10-18T09:02:51.321 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-18T09:02:51.322 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T09:02:51.322 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T09:02:51.322 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T09:02:51.323 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T09:02:51.323 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:02:51.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:51.324 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-10-18T09:02:51.384 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T09:02:51.385 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T09:02:51.385 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T09:02:51.385 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:02:51.385 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-18T09:02:51.385 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:02:51.386 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:02:51.386 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:02:51.386 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2023-10-18T09:02:51.386 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:02:51.386 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:02:51.387 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2023-10-18T09:02:51.387 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:02:51.387 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-10-18T09:02:51.387 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T09:02:51.387 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-10-18T09:02:51.388 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T09:02:51.388 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 41946sec preferred_lft 41946sec 2023-10-18T09:02:51.388 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2023-10-18T09:02:51.388 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:02:51.388 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T09:02:51.389 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff 2023-10-18T09:02:51.389 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T09:02:51.389 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:02:51.389 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1493:a0ff:fed2:2c1/64 scope link 2023-10-18T09:02:51.390 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:02:51.390 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:02:51.390 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 4a:6e:a9:f0:92:62 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T09:02:51.390 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::486e:a9ff:fef0:9262/64 scope link 2023-10-18T09:02:51.390 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:02:51.391 INFO:teuthology.orchestra.run.smithi144.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:02:51.391 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T09:02:51.391 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::345a:c5ff:fe54:de6e/64 scope link 2023-10-18T09:02:51.391 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:02:51.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:51.393 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T09:02:51.485 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-10-18T09:02:51.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:51.490 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-10-18T09:02:51.541 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-10-18T09:02:51.541 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T09:02:51.541 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T09:02:51.542 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T09:02:51.542 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T09:02:51.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:51.543 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T09:02:51.543 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T09:02:51.543 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T09:02:51.544 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T09:02:51.544 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T09:02:51.544 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T09:02:51.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:51.722 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-18T09:02:51.774 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T09:02:51.775 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-18T09:02:51.776 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:02:51.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:51.777 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2023-10-18T09:02:51.864 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:02:51.864 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-18T09:02:51.918 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:02:51.919 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:02:51.919 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-18T09:02:51.973 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-18T09:02:51.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:51.974 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:02:52.118 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:02:52.118 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:02:52.151 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:02:52.151 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:02:52.151 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T09:02:52.151 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with new device syntax: 1@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T09:02:52.152 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T09:02:52.154 INFO:teuthology.orchestra.run.smithi144.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-18T09:02:52.154 INFO:teuthology.orchestra.run.smithi144.stdout: You just mounted a file system that supports labels which does not 2023-10-18T09:02:52.154 INFO:teuthology.orchestra.run.smithi144.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T09:02:52.155 INFO:teuthology.orchestra.run.smithi144.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T09:02:52.155 INFO:teuthology.orchestra.run.smithi144.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T09:02:52.158 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T09:02:52.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:52.158 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-18T09:02:52.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:52.243 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:02:52.299 INFO:teuthology.orchestra.run.smithi144.stdout:client6199 2023-10-18T09:02:52.300 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:02:52.300 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:02:52.353 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:02:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.593+0000 7ff19279e640 1 -- 172.21.15.40:0/371841891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18c15b880 msgr2=0x7ff18c0cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.593+0000 7ff19279e640 1 --2- 172.21.15.40:0/371841891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18c15b880 0x7ff18c0cfe60 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7ff17c005fd0 tx=0x7ff17c02bd90 comp rx=0 tx=0).stop 2023-10-18T09:02:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.593+0000 7ff19279e640 1 -- 172.21.15.40:0/371841891 shutdown_connections 2023-10-18T09:02:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.593+0000 7ff19279e640 1 --2- 172.21.15.40:0/371841891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff18c0cdca0 0x7ff18c0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.593+0000 7ff19279e640 1 --2- 172.21.15.40:0/371841891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18c15b880 0x7ff18c0cfe60 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.593+0000 7ff19279e640 1 -- 172.21.15.40:0/371841891 >> 172.21.15.40:0/371841891 conn(0x7ff18c08aca0 msgr2=0x7ff18c09a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.593+0000 7ff19279e640 1 -- 172.21.15.40:0/371841891 shutdown_connections 2023-10-18T09:02:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.593+0000 7ff19279e640 1 -- 172.21.15.40:0/371841891 wait complete. 2023-10-18T09:02:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.595+0000 7ff19279e640 1 Processor -- start 2023-10-18T09:02:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.595+0000 7ff19279e640 1 -- start start 2023-10-18T09:02:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.597+0000 7ff19279e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff18c0cdca0 0x7ff18c151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.597+0000 7ff19279e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18c15b4a0 0x7ff18c1f2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.597+0000 7ff19279e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff18c0ca320 0x7ff18c0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.597+0000 7ff19279e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c1f3000 con 0x7ff18c15b4a0 2023-10-18T09:02:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.597+0000 7ff19279e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c151d70 con 0x7ff18c0cdca0 2023-10-18T09:02:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.597+0000 7ff19279e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c1f2550 con 0x7ff18c0ca320 2023-10-18T09:02:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.597+0000 7ff190f9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18c15b4a0 0x7ff18c1f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.597+0000 7ff190f9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18c15b4a0 0x7ff18c1f2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54672/0 (socket says 172.21.15.40:54672) 2023-10-18T09:02:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.597+0000 7ff190f9b640 1 -- 172.21.15.40:0/164580616 learned_addr learned my addr 172.21.15.40:0/164580616 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.597+0000 7ff19179c640 1 --2- 172.21.15.40:0/164580616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff18c0cdca0 0x7ff18c151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.597+0000 7ff191f9d640 1 --2- 172.21.15.40:0/164580616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff18c0ca320 0x7ff18c0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:52.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.598+0000 7ff190f9b640 1 -- 172.21.15.40:0/164580616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff18c0ca320 msgr2=0x7ff18c0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:52.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.598+0000 7ff190f9b640 1 --2- 172.21.15.40:0/164580616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff18c0ca320 0x7ff18c0c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:52.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.598+0000 7ff190f9b640 1 -- 172.21.15.40:0/164580616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff18c0cdca0 msgr2=0x7ff18c151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:52.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.598+0000 7ff190f9b640 1 --2- 172.21.15.40:0/164580616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff18c0cdca0 0x7ff18c151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:52.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.598+0000 7ff190f9b640 1 -- 172.21.15.40:0/164580616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c009560 con 0x7ff18c15b4a0 2023-10-18T09:02:52.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.598+0000 7ff190f9b640 1 --2- 172.21.15.40:0/164580616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18c15b4a0 0x7ff18c1f2010 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff17c036040 tx=0x7ff17c02db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:52.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.598+0000 7ff181ffb640 1 -- 172.21.15.40:0/164580616 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c003800 con 0x7ff18c15b4a0 2023-10-18T09:02:52.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.598+0000 7ff181ffb640 1 -- 172.21.15.40:0/164580616 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff17c003a50 con 0x7ff18c15b4a0 2023-10-18T09:02:52.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.598+0000 7ff181ffb640 1 -- 172.21.15.40:0/164580616 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c003e00 con 0x7ff18c15b4a0 2023-10-18T09:02:52.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.598+0000 7ff19279e640 1 -- 172.21.15.40:0/164580616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18c1f2760 con 0x7ff18c15b4a0 2023-10-18T09:02:52.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.599+0000 7ff19279e640 1 -- 172.21.15.40:0/164580616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18c1f2d20 con 0x7ff18c15b4a0 2023-10-18T09:02:52.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.600+0000 7ff19279e640 1 -- 172.21.15.40:0/164580616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18c071da0 con 0x7ff18c15b4a0 2023-10-18T09:02:52.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.601+0000 7ff181ffb640 1 -- 172.21.15.40:0/164580616 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff17c03e450 con 0x7ff18c15b4a0 2023-10-18T09:02:52.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.602+0000 7ff181ffb640 1 --2- 172.21.15.40:0/164580616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff17006ae70 0x7ff17006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:52.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.602+0000 7ff181ffb640 1 -- 172.21.15.40:0/164580616 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff17c0ac210 con 0x7ff18c15b4a0 2023-10-18T09:02:52.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.602+0000 7ff19179c640 1 --2- 172.21.15.40:0/164580616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff17006ae70 0x7ff17006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:52.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.604+0000 7ff19179c640 1 --2- 172.21.15.40:0/164580616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff17006ae70 0x7ff17006d330 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff188007fd0 tx=0x7ff188009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:52.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.606+0000 7ff181ffb640 1 -- 172.21.15.40:0/164580616 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff17c077340 con 0x7ff18c15b4a0 2023-10-18T09:02:52.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.841+0000 7ff19279e640 1 -- 172.21.15.40:0/164580616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff18c0667d0 con 0x7ff18c15b4a0 2023-10-18T09:02:52.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.843+0000 7ff181ffb640 1 -- 172.21.15.40:0/164580616 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4915 (secure 0 0 0) 0x7ff17c07a960 con 0x7ff18c15b4a0 2023-10-18T09:02:52.845 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:52.845 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/700924196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":700924196},{"type":"v1","addr":"172.21.15.40:6833","nonce":700924196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/877414874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":877414874},{"type":"v1","addr":"172.21.15.144:6835","nonce":877414874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2596943574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2596943574},{"type":"v1","addr":"172.21.15.144:6837","nonce":2596943574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:02:41.786627+0000","modified":"2023-10-18T09:02:44.862740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1801003551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1801003551},{"type":"v1","addr":"172.21.15.40:6835","nonce":1801003551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-18T09:02:52.846 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2023-10-18T09:02:52.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.845+0000 7ff19279e640 1 -- 172.21.15.40:0/164580616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff17006ae70 msgr2=0x7ff17006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:52.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.846+0000 7ff19279e640 1 --2- 172.21.15.40:0/164580616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff17006ae70 0x7ff17006d330 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff188007fd0 tx=0x7ff188009040 comp rx=0 tx=0).stop 2023-10-18T09:02:52.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.846+0000 7ff19279e640 1 -- 172.21.15.40:0/164580616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18c15b4a0 msgr2=0x7ff18c1f2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:52.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.846+0000 7ff19279e640 1 --2- 172.21.15.40:0/164580616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18c15b4a0 0x7ff18c1f2010 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff17c036040 tx=0x7ff17c02db10 comp rx=0 tx=0).stop 2023-10-18T09:02:52.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.846+0000 7ff19279e640 1 -- 172.21.15.40:0/164580616 shutdown_connections 2023-10-18T09:02:52.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.846+0000 7ff19279e640 1 --2- 172.21.15.40:0/164580616 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff17006ae70 0x7ff17006d330 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:52.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.846+0000 7ff19279e640 1 --2- 172.21.15.40:0/164580616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff18c0ca320 0x7ff18c0c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:52.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.846+0000 7ff19279e640 1 --2- 172.21.15.40:0/164580616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff18c15b4a0 0x7ff18c1f2010 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:52.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.846+0000 7ff19279e640 1 --2- 172.21.15.40:0/164580616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff18c0cdca0 0x7ff18c151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:52.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.846+0000 7ff19279e640 1 -- 172.21.15.40:0/164580616 >> 172.21.15.40:0/164580616 conn(0x7ff18c08aca0 msgr2=0x7ff18c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:52.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.846+0000 7ff19279e640 1 -- 172.21.15.40:0/164580616 shutdown_connections 2023-10-18T09:02:52.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:52.846+0000 7ff19279e640 1 -- 172.21.15.40:0/164580616 wait complete. 2023-10-18T09:02:52.868 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-18T09:02:53.118 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":7449,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.770946464,"mdsmap_epoch":57,"osdmap_epoch":130,"osdmap_epoch_barrier":130,"uptime":17.657262457000002} 2023-10-18T09:02:53.118 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 7449, "mdsmap_epoch": 57, "osdmap_epoch": 130, "osdmap_epoch_barrier": 130, "rank_uptime": 10.770946464, "state": "up:active", "uptime": 17.657262457, "want_state": "up:active", "whoami": 0 } 2023-10-18T09:02:53.119 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-18T09:02:53.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.353+0000 7f3fdcc0e640 1 -- 172.21.15.40:0/1302254945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd806f590 msgr2=0x7f3fd80c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:53.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.353+0000 7f3fdcc0e640 1 --2- 172.21.15.40:0/1302254945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd806f590 0x7f3fd80c84a0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80c75f0 tx=0x7f3fc002bd50 comp rx=0 tx=0).stop 2023-10-18T09:02:53.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.353+0000 7f3fdcc0e640 1 -- 172.21.15.40:0/1302254945 shutdown_connections 2023-10-18T09:02:53.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.353+0000 7f3fdcc0e640 1 --2- 172.21.15.40:0/1302254945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fd80c99c0 0x7f3fd80c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:53.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.353+0000 7f3fdcc0e640 1 --2- 172.21.15.40:0/1302254945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd806f590 0x7f3fd80c84a0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:53.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.353+0000 7f3fdcc0e640 1 -- 172.21.15.40:0/1302254945 >> 172.21.15.40:0/1302254945 conn(0x7f3fd80aec00 msgr2=0x7f3fd80aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:53.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.353+0000 7f3fdcc0e640 1 -- 172.21.15.40:0/1302254945 shutdown_connections 2023-10-18T09:02:53.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.353+0000 7f3fdcc0e640 1 -- 172.21.15.40:0/1302254945 wait complete. 2023-10-18T09:02:53.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.355+0000 7f3fdcc0e640 1 Processor -- start 2023-10-18T09:02:53.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.356+0000 7f3fdcc0e640 1 -- start start 2023-10-18T09:02:53.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.357+0000 7f3fdcc0e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fd806f590 0x7f3fd80abd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:53.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.357+0000 7f3fdcc0e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd80c99c0 0x7f3fd8157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:53.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.357+0000 7f3fdcc0e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fd8159340 0x7f3fd80c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:53.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.357+0000 7f3fdcc0e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd80ac2b0 con 0x7f3fd80c99c0 2023-10-18T09:02:53.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.357+0000 7f3fdcc0e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd8157b70 con 0x7f3fd8159340 2023-10-18T09:02:53.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.357+0000 7f3fdcc0e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd80ab820 con 0x7f3fd806f590 2023-10-18T09:02:53.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.357+0000 7f3fcffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd80c99c0 0x7f3fd8157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:53.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.357+0000 7f3fcffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd80c99c0 0x7f3fd8157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54688/0 (socket says 172.21.15.40:54688) 2023-10-18T09:02:53.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.357+0000 7f3fcffff640 1 -- 172.21.15.40:0/1687040890 learned_addr learned my addr 172.21.15.40:0/1687040890 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:53.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.357+0000 7f3fcffff640 1 -- 172.21.15.40:0/1687040890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fd806f590 msgr2=0x7f3fd80abd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:53.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.357+0000 7f3fd6d9d640 1 --2- 172.21.15.40:0/1687040890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fd8159340 0x7f3fd80c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:53.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.357+0000 7f3fcffff640 1 --2- 172.21.15.40:0/1687040890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fd806f590 0x7f3fd80abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:53.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.357+0000 7f3fcffff640 1 -- 172.21.15.40:0/1687040890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fd8159340 msgr2=0x7f3fd80c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:53.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.357+0000 7f3fcffff640 1 --2- 172.21.15.40:0/1687040890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fd8159340 0x7f3fd80c62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:53.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.357+0000 7f3fcffff640 1 -- 172.21.15.40:0/1687040890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc0009560 con 0x7f3fd80c99c0 2023-10-18T09:02:53.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.358+0000 7f3fcffff640 1 --2- 172.21.15.40:0/1687040890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd80c99c0 0x7f3fd8157630 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009850 tx=0x7f3fc002dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:53.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.358+0000 7f3fceffd640 1 -- 172.21.15.40:0/1687040890 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0005650 con 0x7f3fd80c99c0 2023-10-18T09:02:53.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.358+0000 7f3fceffd640 1 -- 172.21.15.40:0/1687040890 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3fc0003800 con 0x7f3fd80c99c0 2023-10-18T09:02:53.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.358+0000 7f3fceffd640 1 -- 172.21.15.40:0/1687040890 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0003bf0 con 0x7f3fd80c99c0 2023-10-18T09:02:53.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.358+0000 7f3fdcc0e640 1 -- 172.21.15.40:0/1687040890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd80aac10 con 0x7f3fd80c99c0 2023-10-18T09:02:53.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.358+0000 7f3fdcc0e640 1 -- 172.21.15.40:0/1687040890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd80ab1d0 con 0x7f3fd80c99c0 2023-10-18T09:02:53.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.359+0000 7f3fdcc0e640 1 -- 172.21.15.40:0/1687040890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd8066fd0 con 0x7f3fd80c99c0 2023-10-18T09:02:53.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.360+0000 7f3fceffd640 1 -- 172.21.15.40:0/1687040890 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3fc0003d50 con 0x7f3fd80c99c0 2023-10-18T09:02:53.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.362+0000 7f3fceffd640 1 --2- 172.21.15.40:0/1687040890 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3fb806ae40 0x7f3fb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:53.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.362+0000 7f3fceffd640 1 -- 172.21.15.40:0/1687040890 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fc00abe70 con 0x7f3fd80c99c0 2023-10-18T09:02:53.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.363+0000 7f3fd659c640 1 --2- 172.21.15.40:0/1687040890 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3fb806ae40 0x7f3fb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:53.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.364+0000 7f3fd659c640 1 --2- 172.21.15.40:0/1687040890 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3fb806ae40 0x7f3fb806d300 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8006fd0 tx=0x7f3fc8006bc0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:53.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.366+0000 7f3fceffd640 1 -- 172.21.15.40:0/1687040890 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3fc0077030 con 0x7f3fd80c99c0 2023-10-18T09:02:53.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:53.560+0000 7f3fdcc0e640 1 -- 172.21.15.40:0/1687040890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f3fd80c89e0 con 0x7f3fd80c99c0 2023-10-18T09:02:54.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.073+0000 7f3fceffd640 1 -- 172.21.15.40:0/1687040890 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v131) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3fd80c89e0 con 0x7f3fd80c99c0 2023-10-18T09:02:54.075 INFO:teuthology.orchestra.run.smithi040.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-18T09:02:54.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.077+0000 7f3fdcc0e640 1 -- 172.21.15.40:0/1687040890 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3fb806ae40 msgr2=0x7f3fb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:54.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.077+0000 7f3fdcc0e640 1 --2- 172.21.15.40:0/1687040890 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3fb806ae40 0x7f3fb806d300 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8006fd0 tx=0x7f3fc8006bc0 comp rx=0 tx=0).stop 2023-10-18T09:02:54.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.077+0000 7f3fdcc0e640 1 -- 172.21.15.40:0/1687040890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd80c99c0 msgr2=0x7f3fd8157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:54.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.077+0000 7f3fdcc0e640 1 --2- 172.21.15.40:0/1687040890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd80c99c0 0x7f3fd8157630 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009850 tx=0x7f3fc002dd20 comp rx=0 tx=0).stop 2023-10-18T09:02:54.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.077+0000 7f3fdcc0e640 1 -- 172.21.15.40:0/1687040890 shutdown_connections 2023-10-18T09:02:54.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.077+0000 7f3fdcc0e640 1 --2- 172.21.15.40:0/1687040890 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3fb806ae40 0x7f3fb806d300 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:54.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.077+0000 7f3fdcc0e640 1 --2- 172.21.15.40:0/1687040890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fd8159340 0x7f3fd80c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:54.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.077+0000 7f3fdcc0e640 1 --2- 172.21.15.40:0/1687040890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd80c99c0 0x7f3fd8157630 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:54.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.077+0000 7f3fdcc0e640 1 --2- 172.21.15.40:0/1687040890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3fd806f590 0x7f3fd80abd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:54.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.077+0000 7f3fdcc0e640 1 -- 172.21.15.40:0/1687040890 >> 172.21.15.40:0/1687040890 conn(0x7f3fd80aec00 msgr2=0x7f3fd815b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:54.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.077+0000 7f3fdcc0e640 1 -- 172.21.15.40:0/1687040890 shutdown_connections 2023-10-18T09:02:54.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.077+0000 7f3fdcc0e640 1 -- 172.21.15.40:0/1687040890 wait complete. 2023-10-18T09:02:54.102 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2023-10-18T09:02:54.102 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-10-18T09:02:54.103 DEBUG:teuthology.orchestra.run.smithi040:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-18T09:02:54.103 DEBUG:teuthology.orchestra.run.smithi040:> os.fchmod(fd, 0O666) 2023-10-18T09:02:54.103 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(fd) 2023-10-18T09:02:54.104 DEBUG:teuthology.orchestra.run.smithi040:> os.close(fd) 2023-10-18T09:02:54.104 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-10-18T09:02:54.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T09:02:54.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:54.283 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:02:54.392 INFO:teuthology.orchestra.run.smithi040.stdout:client6193 2023-10-18T09:02:54.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:54.393 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/029855e3-538a-4478-b323-f54ab0f3a56d.client6193/osdmap') 2023-10-18T09:02:54.492 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 130 barrier 130 flags 0x588000 2023-10-18T09:02:54.493 INFO:teuthology.orchestra.run.smithi040.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T09:02:54.493 INFO:teuthology.orchestra.run.smithi040.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T09:02:54.493 INFO:teuthology.orchestra.run.smithi040.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T09:02:54.493 INFO:teuthology.orchestra.run.smithi040.stdout:osd0 (2)172.21.15.40:6800 100% (exists, up) 100% -1 2023-10-18T09:02:54.493 INFO:teuthology.orchestra.run.smithi040.stdout:osd1 (2)172.21.15.40:6808 100% (exists, up) 100% -1 2023-10-18T09:02:54.494 INFO:teuthology.orchestra.run.smithi040.stdout:osd2 (2)172.21.15.40:6823 100% (exists, up) 100% -1 2023-10-18T09:02:54.494 INFO:teuthology.orchestra.run.smithi040.stdout:osd3 (2)172.21.15.40:6816 100% (exists, up) 100% -1 2023-10-18T09:02:54.494 INFO:teuthology.orchestra.run.smithi040.stdout:osd4 (2)172.21.15.144:6810 100% (exists, up) 100% -1 2023-10-18T09:02:54.494 INFO:teuthology.orchestra.run.smithi040.stdout:osd5 (2)172.21.15.144:6802 100% (exists, up) 100% -1 2023-10-18T09:02:54.494 INFO:teuthology.orchestra.run.smithi040.stdout:osd6 (2)172.21.15.144:6818 100% (exists, up) 100% -1 2023-10-18T09:02:54.495 INFO:teuthology.orchestra.run.smithi040.stdout:osd7 (2)172.21.15.144:6824 100% (exists, up) 100% -1 2023-10-18T09:02:54.495 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records in 2023-10-18T09:02:54.495 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records out 2023-10-18T09:02:54.495 INFO:teuthology.orchestra.run.smithi040.stderr:799 bytes copied, 0.00012506 s, 6.4 MB/s 2023-10-18T09:02:54.497 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2023-10-18T09:02:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.727+0000 7f15fce6f640 1 -- 172.21.15.40:0/3205298139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15f80cd9a0 msgr2=0x7f15f806be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.727+0000 7f15fce6f640 1 --2- 172.21.15.40:0/3205298139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15f80cd9a0 0x7f15f806be20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f15f0005fd0 tx=0x7f15f002c2e0 comp rx=0 tx=0).stop 2023-10-18T09:02:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.727+0000 7f15fce6f640 1 -- 172.21.15.40:0/3205298139 shutdown_connections 2023-10-18T09:02:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.727+0000 7f15fce6f640 1 --2- 172.21.15.40:0/3205298139 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15f80cd9a0 0x7f15f806be20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.727+0000 7f15fce6f640 1 --2- 172.21.15.40:0/3205298139 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15f806d710 0x7f15f8151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.727+0000 7f15fce6f640 1 -- 172.21.15.40:0/3205298139 >> 172.21.15.40:0/3205298139 conn(0x7f15f806cee0 msgr2=0x7f15f8069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.728+0000 7f15fce6f640 1 -- 172.21.15.40:0/3205298139 shutdown_connections 2023-10-18T09:02:54.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.728+0000 7f15fce6f640 1 -- 172.21.15.40:0/3205298139 wait complete. 2023-10-18T09:02:54.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.730+0000 7f15fce6f640 1 Processor -- start 2023-10-18T09:02:54.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.730+0000 7f15fce6f640 1 -- start start 2023-10-18T09:02:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.732+0000 7f15fce6f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15f806d710 0x7f15f8149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.732+0000 7f15fce6f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15f80cd9a0 0x7f15f8156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.732+0000 7f15fce6f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15f81586a0 0x7f15f81e2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:54.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.732+0000 7f15fce6f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f81e19c0 con 0x7f15f80cd9a0 2023-10-18T09:02:54.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.732+0000 7f15fce6f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f8149740 con 0x7f15f806d710 2023-10-18T09:02:54.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.732+0000 7f15fce6f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f81e1bb0 con 0x7f15f81586a0 2023-10-18T09:02:54.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.732+0000 7f15f5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15f80cd9a0 0x7f15f8156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:54.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.732+0000 7f15f5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15f80cd9a0 0x7f15f8156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54690/0 (socket says 172.21.15.40:54690) 2023-10-18T09:02:54.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.732+0000 7f15f5d9b640 1 -- 172.21.15.40:0/1109397496 learned_addr learned my addr 172.21.15.40:0/1109397496 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:54.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.732+0000 7f15f6d9d640 1 --2- 172.21.15.40:0/1109397496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15f81586a0 0x7f15f81e2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:54.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.732+0000 7f15f659c640 1 --2- 172.21.15.40:0/1109397496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15f806d710 0x7f15f8149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:54.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.733+0000 7f15f5d9b640 1 -- 172.21.15.40:0/1109397496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15f81586a0 msgr2=0x7f15f81e2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:54.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.733+0000 7f15f5d9b640 1 --2- 172.21.15.40:0/1109397496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15f81586a0 0x7f15f81e2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:54.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.733+0000 7f15f5d9b640 1 -- 172.21.15.40:0/1109397496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15f806d710 msgr2=0x7f15f8149200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:54.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.733+0000 7f15f5d9b640 1 --2- 172.21.15.40:0/1109397496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15f806d710 0x7f15f8149200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:54.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.733+0000 7f15f5d9b640 1 -- 172.21.15.40:0/1109397496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f0009900 con 0x7f15f80cd9a0 2023-10-18T09:02:54.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.733+0000 7f15f6d9d640 1 --2- 172.21.15.40:0/1109397496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15f81586a0 0x7f15f81e2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:02:54.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.733+0000 7f15f5d9b640 1 --2- 172.21.15.40:0/1109397496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15f80cd9a0 0x7f15f8156bf0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00c480 tx=0x7f15ec00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:54.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.733+0000 7f15e6ffd640 1 -- 172.21.15.40:0/1109397496 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0052d0 con 0x7f15f80cd9a0 2023-10-18T09:02:54.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.733+0000 7f15e6ffd640 1 -- 172.21.15.40:0/1109397496 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f15ec005430 con 0x7f15f80cd9a0 2023-10-18T09:02:54.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.734+0000 7f15fce6f640 1 -- 172.21.15.40:0/1109397496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f81e1e50 con 0x7f15f80cd9a0 2023-10-18T09:02:54.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.734+0000 7f15e6ffd640 1 -- 172.21.15.40:0/1109397496 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec012890 con 0x7f15f80cd9a0 2023-10-18T09:02:54.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.734+0000 7f15fce6f640 1 -- 172.21.15.40:0/1109397496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f8154650 con 0x7f15f80cd9a0 2023-10-18T09:02:54.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.735+0000 7f15fce6f640 1 -- 172.21.15.40:0/1109397496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f81544b0 con 0x7f15f80cd9a0 2023-10-18T09:02:54.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.736+0000 7f15e6ffd640 1 -- 172.21.15.40:0/1109397496 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f15ec0035f0 con 0x7f15f80cd9a0 2023-10-18T09:02:54.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.737+0000 7f15e6ffd640 1 --2- 172.21.15.40:0/1109397496 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f15d806ae70 0x7f15d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:54.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.737+0000 7f15e6ffd640 1 -- 172.21.15.40:0/1109397496 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15ec089360 con 0x7f15f80cd9a0 2023-10-18T09:02:54.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.737+0000 7f15f659c640 1 --2- 172.21.15.40:0/1109397496 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f15d806ae70 0x7f15d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:54.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.739+0000 7f15f659c640 1 --2- 172.21.15.40:0/1109397496 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f15d806ae70 0x7f15d806d330 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f15f80ae4c0 tx=0x7f15e8005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:54.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.741+0000 7f15e6ffd640 1 -- 172.21.15.40:0/1109397496 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15f81544b0 con 0x7f15f80cd9a0 2023-10-18T09:02:54.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:54.948+0000 7f15fce6f640 1 -- 172.21.15.40:0/1109397496 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7f15f81544b0 con 0x7f15f80cd9a0 2023-10-18T09:02:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.085+0000 7f15e6ffd640 1 -- 172.21.15.40:0/1109397496 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v132) v1 ==== 93+0+0 (secure 0 0 0) 0x7f15f81544b0 con 0x7f15f80cd9a0 2023-10-18T09:02:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:pauserd,pausewr is set 2023-10-18T09:02:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.089+0000 7f15fce6f640 1 -- 172.21.15.40:0/1109397496 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f15d806ae70 msgr2=0x7f15d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.089+0000 7f15fce6f640 1 --2- 172.21.15.40:0/1109397496 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f15d806ae70 0x7f15d806d330 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f15f80ae4c0 tx=0x7f15e8005f90 comp rx=0 tx=0).stop 2023-10-18T09:02:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.089+0000 7f15fce6f640 1 -- 172.21.15.40:0/1109397496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15f80cd9a0 msgr2=0x7f15f8156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.089+0000 7f15fce6f640 1 --2- 172.21.15.40:0/1109397496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15f80cd9a0 0x7f15f8156bf0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00c480 tx=0x7f15ec00c940 comp rx=0 tx=0).stop 2023-10-18T09:02:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.089+0000 7f15fce6f640 1 -- 172.21.15.40:0/1109397496 shutdown_connections 2023-10-18T09:02:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.089+0000 7f15fce6f640 1 --2- 172.21.15.40:0/1109397496 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f15d806ae70 0x7f15d806d330 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.089+0000 7f15fce6f640 1 --2- 172.21.15.40:0/1109397496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15f81586a0 0x7f15f81e2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:55.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.089+0000 7f15fce6f640 1 --2- 172.21.15.40:0/1109397496 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15f80cd9a0 0x7f15f8156bf0 secure :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00c480 tx=0x7f15ec00c940 comp rx=0 tx=0).stop 2023-10-18T09:02:55.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.089+0000 7f15fce6f640 1 --2- 172.21.15.40:0/1109397496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15f806d710 0x7f15f8149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:55.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.089+0000 7f15fce6f640 1 -- 172.21.15.40:0/1109397496 >> 172.21.15.40:0/1109397496 conn(0x7f15f806cee0 msgr2=0x7f15f806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:55.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.090+0000 7f15fce6f640 1 -- 172.21.15.40:0/1109397496 shutdown_connections 2023-10-18T09:02:55.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.090+0000 7f15fce6f640 1 -- 172.21.15.40:0/1109397496 wait complete. 2023-10-18T09:02:55.115 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2023-10-18T09:02:55.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.348+0000 7f2a4635e640 1 -- 172.21.15.40:0/1135925313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a28003970 msgr2=0x7f2a28003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:55.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.348+0000 7f2a4635e640 1 --2- 172.21.15.40:0/1135925313 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a28003970 0x7f2a28003d90 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009b50 tx=0x7f2a280057f0 comp rx=0 tx=0).stop 2023-10-18T09:02:55.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.348+0000 7f2a448fb640 1 -- 172.21.15.40:0/1135925313 reap_dead start 2023-10-18T09:02:55.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.349+0000 7f2a4635e640 1 -- 172.21.15.40:0/1135925313 shutdown_connections 2023-10-18T09:02:55.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.349+0000 7f2a4635e640 1 -- 172.21.15.40:0/1135925313 >> 172.21.15.40:0/1135925313 conn(0x7f2a4006cee0 msgr2=0x7f2a40069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:55.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.349+0000 7f2a4635e640 1 -- 172.21.15.40:0/1135925313 shutdown_connections 2023-10-18T09:02:55.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.349+0000 7f2a4635e640 1 -- 172.21.15.40:0/1135925313 wait complete. 2023-10-18T09:02:55.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.350+0000 7f2a4635e640 1 Processor -- start 2023-10-18T09:02:55.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.351+0000 7f2a4635e640 1 -- start start 2023-10-18T09:02:55.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.352+0000 7f2a4635e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a28003970 0x7f2a401480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:55.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.352+0000 7f2a4635e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a4014ecf0 0x7f2a401e35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:55.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.352+0000 7f2a4635e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a401e3b20 0x7f2a401566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:55.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.352+0000 7f2a4635e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401e4310 con 0x7f2a28003970 2023-10-18T09:02:55.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.352+0000 7f2a4635e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a400ca0f0 con 0x7f2a4014ecf0 2023-10-18T09:02:55.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.352+0000 7f2a4635e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401e4030 con 0x7f2a401e3b20 2023-10-18T09:02:55.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.352+0000 7f2a3ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a28003970 0x7f2a401480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:55.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.352+0000 7f2a3ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a28003970 0x7f2a401480f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45618/0 (socket says 172.21.15.40:45618) 2023-10-18T09:02:55.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.352+0000 7f2a3ffff640 1 -- 172.21.15.40:0/1733449711 learned_addr learned my addr 172.21.15.40:0/1733449711 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:55.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.352+0000 7f2a3f7fe640 1 --2- 172.21.15.40:0/1733449711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a4014ecf0 0x7f2a401e35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:55.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.352+0000 7f2a448fb640 1 --2- 172.21.15.40:0/1733449711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a401e3b20 0x7f2a401566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:55.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.352+0000 7f2a3ffff640 1 -- 172.21.15.40:0/1733449711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a401e3b20 msgr2=0x7f2a401566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:55.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.352+0000 7f2a3ffff640 1 --2- 172.21.15.40:0/1733449711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a401e3b20 0x7f2a401566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:55.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.352+0000 7f2a3ffff640 1 -- 172.21.15.40:0/1733449711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a4014ecf0 msgr2=0x7f2a401e35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:55.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.352+0000 7f2a3ffff640 1 --2- 172.21.15.40:0/1733449711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a4014ecf0 0x7f2a401e35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.352+0000 7f2a3ffff640 1 -- 172.21.15.40:0/1733449711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4015c1d0 con 0x7f2a28003970 2023-10-18T09:02:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.353+0000 7f2a3ffff640 1 --2- 172.21.15.40:0/1733449711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a28003970 0x7f2a401480f0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f2a3002cbc0 tx=0x7f2a30033c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:55.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.353+0000 7f2a3cff9640 1 -- 172.21.15.40:0/1733449711 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30004220 con 0x7f2a28003970 2023-10-18T09:02:55.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.353+0000 7f2a3cff9640 1 -- 172.21.15.40:0/1733449711 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2a30004380 con 0x7f2a28003970 2023-10-18T09:02:55.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.353+0000 7f2a3cff9640 1 -- 172.21.15.40:0/1733449711 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3003d880 con 0x7f2a28003970 2023-10-18T09:02:55.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.353+0000 7f2a4635e640 1 -- 172.21.15.40:0/1733449711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a400ca390 con 0x7f2a28003970 2023-10-18T09:02:55.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.353+0000 7f2a4635e640 1 -- 172.21.15.40:0/1733449711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a40156c30 con 0x7f2a28003970 2023-10-18T09:02:55.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.354+0000 7f2a4635e640 1 -- 172.21.15.40:0/1733449711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a40066fd0 con 0x7f2a28003970 2023-10-18T09:02:55.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.357+0000 7f2a3cff9640 1 -- 172.21.15.40:0/1733449711 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2a3003d9e0 con 0x7f2a28003970 2023-10-18T09:02:55.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.357+0000 7f2a3cff9640 1 --2- 172.21.15.40:0/1733449711 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a2c06ae70 0x7f2a2c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:55.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.357+0000 7f2a3cff9640 1 -- 172.21.15.40:0/1733449711 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a300ac630 con 0x7f2a28003970 2023-10-18T09:02:55.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.358+0000 7f2a3cff9640 1 -- 172.21.15.40:0/1733449711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=133+}) v3 -- 0x7f2a2c0729a0 con 0x7f2a28003970 2023-10-18T09:02:55.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.358+0000 7f2a3f7fe640 1 --2- 172.21.15.40:0/1733449711 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a2c06ae70 0x7f2a2c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:55.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.359+0000 7f2a3f7fe640 1 --2- 172.21.15.40:0/1733449711 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a2c06ae70 0x7f2a2c06d330 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2a4014e320 tx=0x7f2a34005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:55.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.361+0000 7f2a3cff9640 1 -- 172.21.15.40:0/1733449711 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a3004a020 con 0x7f2a28003970 2023-10-18T09:02:55.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:55.564+0000 7f2a4635e640 1 -- 172.21.15.40:0/1733449711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7f2a401e4310 con 0x7f2a28003970 2023-10-18T09:02:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.089+0000 7f2a3cff9640 1 -- 172.21.15.40:0/1733449711 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v133) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2a30077970 con 0x7f2a28003970 2023-10-18T09:02:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:pauserd,pausewr is unset 2023-10-18T09:02:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.090+0000 7f2a3cff9640 1 -- 172.21.15.40:0/1733449711 <== mon.0 v2:172.21.15.40:3300/0 8 ==== osd_map(133..133 src has 1..133) v4 ==== 288+0+0 (secure 0 0 0) 0x7f2a3007bd30 con 0x7f2a28003970 2023-10-18T09:02:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.090+0000 7f2a3cff9640 1 -- 172.21.15.40:0/1733449711 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=134}) v3 -- 0x7f2a2c0733d0 con 0x7f2a28003970 2023-10-18T09:02:56.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.093+0000 7f2a4635e640 1 -- 172.21.15.40:0/1733449711 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a2c06ae70 msgr2=0x7f2a2c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.093+0000 7f2a4635e640 1 --2- 172.21.15.40:0/1733449711 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a2c06ae70 0x7f2a2c06d330 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2a4014e320 tx=0x7f2a34005f90 comp rx=0 tx=0).stop 2023-10-18T09:02:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.093+0000 7f2a4635e640 1 -- 172.21.15.40:0/1733449711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a28003970 msgr2=0x7f2a401480f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.093+0000 7f2a4635e640 1 --2- 172.21.15.40:0/1733449711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a28003970 0x7f2a401480f0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f2a3002cbc0 tx=0x7f2a30033c30 comp rx=0 tx=0).stop 2023-10-18T09:02:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.094+0000 7f2a4635e640 1 -- 172.21.15.40:0/1733449711 shutdown_connections 2023-10-18T09:02:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.094+0000 7f2a4635e640 1 --2- 172.21.15.40:0/1733449711 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2a2c06ae70 0x7f2a2c06d330 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.094+0000 7f2a4635e640 1 --2- 172.21.15.40:0/1733449711 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a401e3b20 0x7f2a401566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:56.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.094+0000 7f2a4635e640 1 --2- 172.21.15.40:0/1733449711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a4014ecf0 0x7f2a401e35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:56.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.094+0000 7f2a4635e640 1 --2- 172.21.15.40:0/1733449711 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a28003970 0x7f2a401480f0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:56.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.094+0000 7f2a4635e640 1 -- 172.21.15.40:0/1733449711 >> 172.21.15.40:0/1733449711 conn(0x7f2a4006cee0 msgr2=0x7f2a400cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:56.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.095+0000 7f2a4635e640 1 -- 172.21.15.40:0/1733449711 shutdown_connections 2023-10-18T09:02:56.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.095+0000 7f2a4635e640 1 -- 172.21.15.40:0/1733449711 wait complete. 2023-10-18T09:02:56.119 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:02:56.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.344+0000 7fdd7df83640 1 -- 172.21.15.40:0/1209648269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd780cd4c0 msgr2=0x7fdd780b0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:56.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.344+0000 7fdd7df83640 1 --2- 172.21.15.40:0/1209648269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd780cd4c0 0x7fdd780b0290 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009830 tx=0x7fdd6002be90 comp rx=0 tx=0).stop 2023-10-18T09:02:56.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.344+0000 7fdd7df83640 1 -- 172.21.15.40:0/1209648269 shutdown_connections 2023-10-18T09:02:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.344+0000 7fdd7df83640 1 --2- 172.21.15.40:0/1209648269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd780cd4c0 0x7fdd780b0290 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.344+0000 7fdd7df83640 1 -- 172.21.15.40:0/1209648269 >> 172.21.15.40:0/1209648269 conn(0x7fdd7806cee0 msgr2=0x7fdd78069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.344+0000 7fdd7df83640 1 -- 172.21.15.40:0/1209648269 shutdown_connections 2023-10-18T09:02:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.344+0000 7fdd7df83640 1 -- 172.21.15.40:0/1209648269 wait complete. 2023-10-18T09:02:56.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.346+0000 7fdd7df83640 1 Processor -- start 2023-10-18T09:02:56.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.346+0000 7fdd7df83640 1 -- start start 2023-10-18T09:02:56.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.347+0000 7fdd7df83640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd780b1b90 0x7fdd7814ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:56.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.347+0000 7fdd7df83640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd780cd4c0 0x7fdd780cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:56.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.347+0000 7fdd7df83640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7815eaa0 0x7fdd780c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:56.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.347+0000 7fdd7df83640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780cbcb0 con 0x7fdd7815eaa0 2023-10-18T09:02:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.347+0000 7fdd7df83640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78156800 con 0x7fdd780b1b90 2023-10-18T09:02:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.347+0000 7fdd7df83640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780cbee0 con 0x7fdd780cd4c0 2023-10-18T09:02:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.348+0000 7fdd77fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7815eaa0 0x7fdd780c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.348+0000 7fdd77fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7815eaa0 0x7fdd780c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45632/0 (socket says 172.21.15.40:45632) 2023-10-18T09:02:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.348+0000 7fdd77fff640 1 -- 172.21.15.40:0/2662796681 learned_addr learned my addr 172.21.15.40:0/2662796681 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.348+0000 7fdd777fe640 1 --2- 172.21.15.40:0/2662796681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd780b1b90 0x7fdd7814ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.348+0000 7fdd6ffff640 1 --2- 172.21.15.40:0/2662796681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd780cd4c0 0x7fdd780cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.348+0000 7fdd77fff640 1 -- 172.21.15.40:0/2662796681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd780cd4c0 msgr2=0x7fdd780cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.348+0000 7fdd77fff640 1 --2- 172.21.15.40:0/2662796681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd780cd4c0 0x7fdd780cb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.348+0000 7fdd77fff640 1 -- 172.21.15.40:0/2662796681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd780b1b90 msgr2=0x7fdd7814ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:56.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.348+0000 7fdd77fff640 1 --2- 172.21.15.40:0/2662796681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd780b1b90 0x7fdd7814ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:56.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.348+0000 7fdd77fff640 1 -- 172.21.15.40:0/2662796681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd60009490 con 0x7fdd7815eaa0 2023-10-18T09:02:56.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.348+0000 7fdd77fff640 1 --2- 172.21.15.40:0/2662796681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7815eaa0 0x7fdd780c9b00 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000cb40 tx=0x7fdd7000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:56.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.349+0000 7fdd74ff9640 1 -- 172.21.15.40:0/2662796681 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7000aca0 con 0x7fdd7815eaa0 2023-10-18T09:02:56.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.349+0000 7fdd74ff9640 1 -- 172.21.15.40:0/2662796681 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdd70012420 con 0x7fdd7815eaa0 2023-10-18T09:02:56.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.349+0000 7fdd74ff9640 1 -- 172.21.15.40:0/2662796681 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70012850 con 0x7fdd7815eaa0 2023-10-18T09:02:56.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.349+0000 7fdd7df83640 1 -- 172.21.15.40:0/2662796681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd780cc1c0 con 0x7fdd7815eaa0 2023-10-18T09:02:56.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.349+0000 7fdd7df83640 1 -- 172.21.15.40:0/2662796681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd780cc4a0 con 0x7fdd7815eaa0 2023-10-18T09:02:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.350+0000 7fdd7df83640 1 -- 172.21.15.40:0/2662796681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd781562f0 con 0x7fdd7815eaa0 2023-10-18T09:02:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.351+0000 7fdd74ff9640 1 -- 172.21.15.40:0/2662796681 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdd7001a3b0 con 0x7fdd7815eaa0 2023-10-18T09:02:56.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.352+0000 7fdd74ff9640 1 --2- 172.21.15.40:0/2662796681 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdd5806ae70 0x7fdd5806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:56.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.352+0000 7fdd74ff9640 1 -- 172.21.15.40:0/2662796681 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd70089f30 con 0x7fdd7815eaa0 2023-10-18T09:02:56.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.353+0000 7fdd777fe640 1 --2- 172.21.15.40:0/2662796681 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdd5806ae70 0x7fdd5806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:56.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.354+0000 7fdd777fe640 1 --2- 172.21.15.40:0/2662796681 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdd5806ae70 0x7fdd5806d330 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdd68007fd0 tx=0x7fdd68009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:56.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.357+0000 7fdd74ff9640 1 -- 172.21.15.40:0/2662796681 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd781562f0 con 0x7fdd7815eaa0 2023-10-18T09:02:56.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.554+0000 7fdd7df83640 1 -- 172.21.15.40:0/2662796681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd781562f0 con 0x7fdd7815eaa0 2023-10-18T09:02:56.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.557+0000 7fdd74ff9640 1 -- 172.21.15.40:0/2662796681 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+16539 (secure 0 0 0) 0x7fdd781562f0 con 0x7fdd7815eaa0 2023-10-18T09:02:56.560 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:56.560 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":133,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:02:56.085956+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:02:39.531482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-18T09:02:40.812071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:02:56.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.560+0000 7fdd7df83640 1 -- 172.21.15.40:0/2662796681 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdd5806ae70 msgr2=0x7fdd5806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.560+0000 7fdd7df83640 1 --2- 172.21.15.40:0/2662796681 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdd5806ae70 0x7fdd5806d330 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdd68007fd0 tx=0x7fdd68009040 comp rx=0 tx=0).stop 2023-10-18T09:02:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.560+0000 7fdd7df83640 1 -- 172.21.15.40:0/2662796681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7815eaa0 msgr2=0x7fdd780c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.560+0000 7fdd7df83640 1 --2- 172.21.15.40:0/2662796681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7815eaa0 0x7fdd780c9b00 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000cb40 tx=0x7fdd7000a060 comp rx=0 tx=0).stop 2023-10-18T09:02:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.560+0000 7fdd7df83640 1 -- 172.21.15.40:0/2662796681 shutdown_connections 2023-10-18T09:02:56.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.560+0000 7fdd7df83640 1 --2- 172.21.15.40:0/2662796681 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdd5806ae70 0x7fdd5806d330 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.560+0000 7fdd7df83640 1 --2- 172.21.15.40:0/2662796681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd7815eaa0 0x7fdd780c9b00 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.560+0000 7fdd7df83640 1 --2- 172.21.15.40:0/2662796681 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd780cd4c0 0x7fdd780cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.560+0000 7fdd7df83640 1 --2- 172.21.15.40:0/2662796681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd780b1b90 0x7fdd7814ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.560+0000 7fdd7df83640 1 -- 172.21.15.40:0/2662796681 >> 172.21.15.40:0/2662796681 conn(0x7fdd7806cee0 msgr2=0x7fdd780b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:56.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.561+0000 7fdd7df83640 1 -- 172.21.15.40:0/2662796681 shutdown_connections 2023-10-18T09:02:56.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56.561+0000 7fdd7df83640 1 -- 172.21.15.40:0/2662796681 wait complete. 2023-10-18T09:02:56.583 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2023-10-18T09:02:56.584 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-10-18T09:02:56.584 DEBUG:teuthology.orchestra.run.smithi040:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-18T09:02:56.585 DEBUG:teuthology.orchestra.run.smithi040:> os.fchmod(fd, 0O666) 2023-10-18T09:02:56.585 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(fd) 2023-10-18T09:02:56.586 DEBUG:teuthology.orchestra.run.smithi040:> os.close(fd) 2023-10-18T09:02:56.586 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-10-18T09:02:56.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T09:02:56.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:56.738 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:02:56.847 INFO:teuthology.orchestra.run.smithi040.stdout:client6193 2023-10-18T09:02:56.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:56.848 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/029855e3-538a-4478-b323-f54ab0f3a56d.client6193/osdmap') 2023-10-18T09:02:56.951 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 130 barrier 130 flags 0x588000 2023-10-18T09:02:56.951 INFO:teuthology.orchestra.run.smithi040.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T09:02:56.951 INFO:teuthology.orchestra.run.smithi040.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T09:02:56.952 INFO:teuthology.orchestra.run.smithi040.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T09:02:56.952 INFO:teuthology.orchestra.run.smithi040.stdout:osd0 (2)172.21.15.40:6800 100% (exists, up) 100% -1 2023-10-18T09:02:56.952 INFO:teuthology.orchestra.run.smithi040.stdout:osd1 (2)172.21.15.40:6808 100% (exists, up) 100% -1 2023-10-18T09:02:56.953 INFO:teuthology.orchestra.run.smithi040.stdout:osd2 (2)172.21.15.40:6823 100% (exists, up) 100% -1 2023-10-18T09:02:56.953 INFO:teuthology.orchestra.run.smithi040.stdout:osd3 (2)172.21.15.40:6816 100% (exists, up) 100% -1 2023-10-18T09:02:56.953 INFO:teuthology.orchestra.run.smithi040.stdout:osd4 (2)172.21.15.144:6810 100% (exists, up) 100% -1 2023-10-18T09:02:56.953 INFO:teuthology.orchestra.run.smithi040.stdout:osd5 (2)172.21.15.144:6802 100% (exists, up) 100% -1 2023-10-18T09:02:56.954 INFO:teuthology.orchestra.run.smithi040.stdout:osd6 (2)172.21.15.144:6818 100% (exists, up) 100% -1 2023-10-18T09:02:56.954 INFO:teuthology.orchestra.run.smithi040.stdout:osd7 (2)172.21.15.144:6824 100% (exists, up) 100% -1 2023-10-18T09:02:56.956 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records in 2023-10-18T09:02:56.957 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records out 2023-10-18T09:02:56.957 INFO:teuthology.orchestra.run.smithi040.stderr:799 bytes copied, 9.3618e-05 s, 8.5 MB/s 2023-10-18T09:02:56.957 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:02:57.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.190+0000 7fd22e9ed640 1 -- 172.21.15.40:0/3564975591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280b0630 msgr2=0x7fd2280c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:57.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.190+0000 7fd22e9ed640 1 --2- 172.21.15.40:0/3564975591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280b0630 0x7fd2280c99c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd22806aeb0 tx=0x7fd21802bd90 comp rx=0 tx=0).stop 2023-10-18T09:02:57.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.190+0000 7fd22e9ed640 1 -- 172.21.15.40:0/3564975591 shutdown_connections 2023-10-18T09:02:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.190+0000 7fd22e9ed640 1 --2- 172.21.15.40:0/3564975591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2280c9f00 0x7fd228157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.190+0000 7fd22e9ed640 1 --2- 172.21.15.40:0/3564975591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280b0630 0x7fd2280c99c0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.190+0000 7fd22e9ed640 1 --2- 172.21.15.40:0/3564975591 >> v2:172.21.15.144:3300/0 conn(0x7fd22806d6f0 0x7fd2280b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.191+0000 7fd22e9ed640 1 -- 172.21.15.40:0/3564975591 >> 172.21.15.40:0/3564975591 conn(0x7fd228069720 msgr2=0x7fd228069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:57.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.191+0000 7fd22e9ed640 1 -- 172.21.15.40:0/3564975591 shutdown_connections 2023-10-18T09:02:57.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.191+0000 7fd22e9ed640 1 -- 172.21.15.40:0/3564975591 wait complete. 2023-10-18T09:02:57.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.192+0000 7fd22e9ed640 1 Processor -- start 2023-10-18T09:02:57.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.193+0000 7fd22e9ed640 1 -- start start 2023-10-18T09:02:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.193+0000 7fd22e9ed640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd22806d6f0 0x7fd2281f2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.193+0000 7fd22e9ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280b0630 0x7fd2281545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.193+0000 7fd22e9ed640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2280c9f00 0x7fd22813a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.193+0000 7fd22e9ed640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2281f2c90 con 0x7fd2280b0630 2023-10-18T09:02:57.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.193+0000 7fd22e9ed640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd228148da0 con 0x7fd22806d6f0 2023-10-18T09:02:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.193+0000 7fd22e9ed640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2281f5780 con 0x7fd2280c9f00 2023-10-18T09:02:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.194+0000 7fd22d1ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280b0630 0x7fd2281545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.194+0000 7fd22d1ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280b0630 0x7fd2281545c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45654/0 (socket says 172.21.15.40:45654) 2023-10-18T09:02:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.194+0000 7fd22d1ea640 1 -- 172.21.15.40:0/3315228035 learned_addr learned my addr 172.21.15.40:0/3315228035 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.194+0000 7fd22d9eb640 1 --2- 172.21.15.40:0/3315228035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd22806d6f0 0x7fd2281f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:57.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.194+0000 7fd22d1ea640 1 -- 172.21.15.40:0/3315228035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2280c9f00 msgr2=0x7fd22813a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:57.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.194+0000 7fd22d1ea640 1 --2- 172.21.15.40:0/3315228035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2280c9f00 0x7fd22813a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:57.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.194+0000 7fd22d1ea640 1 -- 172.21.15.40:0/3315228035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd22806d6f0 msgr2=0x7fd2281f2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:57.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.194+0000 7fd22d1ea640 1 --2- 172.21.15.40:0/3315228035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd22806d6f0 0x7fd2281f2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:57.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.194+0000 7fd22d1ea640 1 -- 172.21.15.40:0/3315228035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd218009560 con 0x7fd2280b0630 2023-10-18T09:02:57.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.194+0000 7fd22d9eb640 1 --2- 172.21.15.40:0/3315228035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd22806d6f0 0x7fd2281f2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:57.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.195+0000 7fd22d1ea640 1 --2- 172.21.15.40:0/3315228035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280b0630 0x7fd2281545c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fd218009850 tx=0x7fd21802da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.195+0000 7fd21e7fc640 1 -- 172.21.15.40:0/3315228035 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd218003800 con 0x7fd2280b0630 2023-10-18T09:02:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.195+0000 7fd21e7fc640 1 -- 172.21.15.40:0/3315228035 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd218003a50 con 0x7fd2280b0630 2023-10-18T09:02:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.195+0000 7fd22e9ed640 1 -- 172.21.15.40:0/3315228035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2281f5a60 con 0x7fd2280b0630 2023-10-18T09:02:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.195+0000 7fd21e7fc640 1 -- 172.21.15.40:0/3315228035 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd218003e20 con 0x7fd2280b0630 2023-10-18T09:02:57.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.195+0000 7fd22e9ed640 1 -- 172.21.15.40:0/3315228035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2281f6020 con 0x7fd2280b0630 2023-10-18T09:02:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.196+0000 7fd22e9ed640 1 -- 172.21.15.40:0/3315228035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd228066e00 con 0x7fd2280b0630 2023-10-18T09:02:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.198+0000 7fd21e7fc640 1 -- 172.21.15.40:0/3315228035 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd21803e450 con 0x7fd2280b0630 2023-10-18T09:02:57.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.199+0000 7fd21e7fc640 1 --2- 172.21.15.40:0/3315228035 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd20c06ae40 0x7fd20c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:57.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.199+0000 7fd21e7fc640 1 -- 172.21.15.40:0/3315228035 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2180ac1c0 con 0x7fd2280b0630 2023-10-18T09:02:57.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.200+0000 7fd22d9eb640 1 --2- 172.21.15.40:0/3315228035 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd20c06ae40 0x7fd20c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:57.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.201+0000 7fd22d9eb640 1 --2- 172.21.15.40:0/3315228035 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd20c06ae40 0x7fd20c06d300 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd224005fd0 tx=0x7fd2240094d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:57.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.203+0000 7fd21e7fc640 1 -- 172.21.15.40:0/3315228035 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd2180771d0 con 0x7fd2280b0630 2023-10-18T09:02:57.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.437+0000 7fd22e9ed640 1 -- 172.21.15.40:0/3315228035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd22806a480 con 0x7fd2280b0630 2023-10-18T09:02:57.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.438+0000 7fd21e7fc640 1 -- 172.21.15.40:0/3315228035 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fd21807a7f0 con 0x7fd2280b0630 2023-10-18T09:02:57.441 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:57.441 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/700924196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":700924196},{"type":"v1","addr":"172.21.15.40:6833","nonce":700924196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/877414874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":877414874},{"type":"v1","addr":"172.21.15.144:6835","nonce":877414874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2596943574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2596943574},{"type":"v1","addr":"172.21.15.144:6837","nonce":2596943574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:02:41.786627+0000","modified":"2023-10-18T09:02:44.862740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1801003551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1801003551},{"type":"v1","addr":"172.21.15.40:6835","nonce":1801003551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-18T09:02:57.442 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2023-10-18T09:02:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.441+0000 7fd22e9ed640 1 -- 172.21.15.40:0/3315228035 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd20c06ae40 msgr2=0x7fd20c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.441+0000 7fd22e9ed640 1 --2- 172.21.15.40:0/3315228035 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd20c06ae40 0x7fd20c06d300 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd224005fd0 tx=0x7fd2240094d0 comp rx=0 tx=0).stop 2023-10-18T09:02:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.441+0000 7fd22e9ed640 1 -- 172.21.15.40:0/3315228035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280b0630 msgr2=0x7fd2281545c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.441+0000 7fd22e9ed640 1 --2- 172.21.15.40:0/3315228035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280b0630 0x7fd2281545c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fd218009850 tx=0x7fd21802da60 comp rx=0 tx=0).stop 2023-10-18T09:02:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.442+0000 7fd22e9ed640 1 -- 172.21.15.40:0/3315228035 shutdown_connections 2023-10-18T09:02:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.442+0000 7fd22e9ed640 1 --2- 172.21.15.40:0/3315228035 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd20c06ae40 0x7fd20c06d300 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.442+0000 7fd22e9ed640 1 --2- 172.21.15.40:0/3315228035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2280c9f00 0x7fd22813a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.442+0000 7fd22e9ed640 1 --2- 172.21.15.40:0/3315228035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2280b0630 0x7fd2281545c0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.442+0000 7fd22e9ed640 1 --2- 172.21.15.40:0/3315228035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd22806d6f0 0x7fd2281f2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.442+0000 7fd22e9ed640 1 -- 172.21.15.40:0/3315228035 >> 172.21.15.40:0/3315228035 conn(0x7fd228069720 msgr2=0x7fd228156260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.442+0000 7fd22e9ed640 1 -- 172.21.15.40:0/3315228035 shutdown_connections 2023-10-18T09:02:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57.442+0000 7fd22e9ed640 1 -- 172.21.15.40:0/3315228035 wait complete. 2023-10-18T09:02:57.463 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json osdmap barrier 133 2023-10-18T09:02:57.706 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-18T09:02:57.706 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2023-10-18T09:02:57.707 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-10-18T09:02:57.707 DEBUG:teuthology.orchestra.run.smithi040:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-18T09:02:57.708 DEBUG:teuthology.orchestra.run.smithi040:> os.fchmod(fd, 0O666) 2023-10-18T09:02:57.708 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(fd) 2023-10-18T09:02:57.708 DEBUG:teuthology.orchestra.run.smithi040:> os.close(fd) 2023-10-18T09:02:57.709 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-10-18T09:02:57.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T09:02:57.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:57.884 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:02:57.993 INFO:teuthology.orchestra.run.smithi040.stdout:client6193 2023-10-18T09:02:57.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:57.994 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/029855e3-538a-4478-b323-f54ab0f3a56d.client6193/osdmap') 2023-10-18T09:02:58.098 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 133 barrier 133 flags 0x588000 2023-10-18T09:02:58.099 INFO:teuthology.orchestra.run.smithi040.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T09:02:58.099 INFO:teuthology.orchestra.run.smithi040.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T09:02:58.099 INFO:teuthology.orchestra.run.smithi040.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T09:02:58.099 INFO:teuthology.orchestra.run.smithi040.stdout:osd0 (2)172.21.15.40:6800 100% (exists, up) 100% -1 2023-10-18T09:02:58.100 INFO:teuthology.orchestra.run.smithi040.stdout:osd1 (2)172.21.15.40:6808 100% (exists, up) 100% -1 2023-10-18T09:02:58.100 INFO:teuthology.orchestra.run.smithi040.stdout:osd2 (2)172.21.15.40:6823 100% (exists, up) 100% -1 2023-10-18T09:02:58.100 INFO:teuthology.orchestra.run.smithi040.stdout:osd3 (2)172.21.15.40:6816 100% (exists, up) 100% -1 2023-10-18T09:02:58.100 INFO:teuthology.orchestra.run.smithi040.stdout:osd4 (2)172.21.15.144:6810 100% (exists, up) 100% -1 2023-10-18T09:02:58.100 INFO:teuthology.orchestra.run.smithi040.stdout:osd5 (2)172.21.15.144:6802 100% (exists, up) 100% -1 2023-10-18T09:02:58.100 INFO:teuthology.orchestra.run.smithi040.stdout:osd6 (2)172.21.15.144:6818 100% (exists, up) 100% -1 2023-10-18T09:02:58.101 INFO:teuthology.orchestra.run.smithi040.stdout:osd7 (2)172.21.15.144:6824 100% (exists, up) 100% -1 2023-10-18T09:02:58.101 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records in 2023-10-18T09:02:58.101 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records out 2023-10-18T09:02:58.101 INFO:teuthology.orchestra.run.smithi040.stderr:799 bytes copied, 0.000127734 s, 6.3 MB/s 2023-10-18T09:02:58.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:58.103 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:02:58.161 INFO:teuthology.orchestra.run.smithi040.stdout:client6193 2023-10-18T09:02:58.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:02:58.163 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/029855e3-538a-4478-b323-f54ab0f3a56d.client6193/osdmap') 2023-10-18T09:02:58.260 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 133 barrier 133 flags 0x588000 2023-10-18T09:02:58.260 INFO:teuthology.orchestra.run.smithi040.stdout:pool 1 '.mgr' type 1 size 2 min_size 1 pg_num 1 pg_num_mask 0 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T09:02:58.260 INFO:teuthology.orchestra.run.smithi040.stdout:pool 14 'cephfs_metadata' type 1 size 2 min_size 1 pg_num 32 pg_num_mask 31 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T09:02:58.260 INFO:teuthology.orchestra.run.smithi040.stdout:pool 15 'cephfs_data' type 1 size 2 min_size 1 pg_num 64 pg_num_mask 63 flags 0x1 lfor 0 read_tier -1 write_tier -1 2023-10-18T09:02:58.261 INFO:teuthology.orchestra.run.smithi040.stdout:osd0 (2)172.21.15.40:6800 100% (exists, up) 100% -1 2023-10-18T09:02:58.261 INFO:teuthology.orchestra.run.smithi040.stdout:osd1 (2)172.21.15.40:6808 100% (exists, up) 100% -1 2023-10-18T09:02:58.261 INFO:teuthology.orchestra.run.smithi040.stdout:osd2 (2)172.21.15.40:6823 100% (exists, up) 100% -1 2023-10-18T09:02:58.261 INFO:teuthology.orchestra.run.smithi040.stdout:osd3 (2)172.21.15.40:6816 100% (exists, up) 100% -1 2023-10-18T09:02:58.261 INFO:teuthology.orchestra.run.smithi040.stdout:osd4 (2)172.21.15.144:6810 100% (exists, up) 100% -1 2023-10-18T09:02:58.262 INFO:teuthology.orchestra.run.smithi040.stdout:osd5 (2)172.21.15.144:6802 100% (exists, up) 100% -1 2023-10-18T09:02:58.262 INFO:teuthology.orchestra.run.smithi040.stdout:osd6 (2)172.21.15.144:6818 100% (exists, up) 100% -1 2023-10-18T09:02:58.262 INFO:teuthology.orchestra.run.smithi040.stdout:osd7 (2)172.21.15.144:6824 100% (exists, up) 100% -1 2023-10-18T09:02:58.262 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records in 2023-10-18T09:02:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:1+1 records out 2023-10-18T09:02:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:799 bytes copied, 0.00012275 s, 6.5 MB/s 2023-10-18T09:02:58.264 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-18T09:02:58.265 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T09:02:58.265 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:02:58.401 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:02:58.528 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T09:02:58.529 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:02:58.529 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:02:58.593 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:02:58.594 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:02:58.654 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:02:58.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.888+0000 7ff590eb4640 1 -- 172.21.15.40:0/1959899431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff578003970 msgr2=0x7ff578003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:58.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.888+0000 7ff590eb4640 1 --2- 172.21.15.40:0/1959899431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff578003970 0x7ff578003d90 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00c290 tx=0x7ff5780057f0 comp rx=0 tx=0).stop 2023-10-18T09:02:58.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.888+0000 7ff58bfff640 1 -- 172.21.15.40:0/1959899431 reap_dead start 2023-10-18T09:02:58.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.888+0000 7ff590eb4640 1 -- 172.21.15.40:0/1959899431 shutdown_connections 2023-10-18T09:02:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.889+0000 7ff590eb4640 1 -- 172.21.15.40:0/1959899431 >> 172.21.15.40:0/1959899431 conn(0x7ff58c069720 msgr2=0x7ff58c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.889+0000 7ff590eb4640 1 -- 172.21.15.40:0/1959899431 shutdown_connections 2023-10-18T09:02:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.889+0000 7ff590eb4640 1 -- 172.21.15.40:0/1959899431 wait complete. 2023-10-18T09:02:58.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.891+0000 7ff590eb4640 1 Processor -- start 2023-10-18T09:02:58.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.891+0000 7ff590eb4640 1 -- start start 2023-10-18T09:02:58.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.892+0000 7ff590eb4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff578003970 0x7ff58c15c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:58.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.892+0000 7ff590eb4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff58c13f180 0x7ff58c13f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:58.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.892+0000 7ff590eb4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff58c1f2080 0x7ff58c1f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:58.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.892+0000 7ff590eb4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c15c810 con 0x7ff58c13f180 2023-10-18T09:02:58.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.892+0000 7ff590eb4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c13fb40 con 0x7ff578003970 2023-10-18T09:02:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.892+0000 7ff590eb4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c12e030 con 0x7ff58c1f2080 2023-10-18T09:02:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.892+0000 7ff582ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff58c13f180 0x7ff58c13f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.892+0000 7ff582ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff58c13f180 0x7ff58c13f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45668/0 (socket says 172.21.15.40:45668) 2023-10-18T09:02:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.892+0000 7ff582ffd640 1 -- 172.21.15.40:0/1019118591 learned_addr learned my addr 172.21.15.40:0/1019118591 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.892+0000 7ff58b7fe640 1 --2- 172.21.15.40:0/1019118591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff578003970 0x7ff58c15c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:58.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.892+0000 7ff58bfff640 1 --2- 172.21.15.40:0/1019118591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff58c1f2080 0x7ff58c1f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:58.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.892+0000 7ff582ffd640 1 -- 172.21.15.40:0/1019118591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff58c1f2080 msgr2=0x7ff58c1f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:58.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.893+0000 7ff582ffd640 1 --2- 172.21.15.40:0/1019118591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff58c1f2080 0x7ff58c1f2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:58.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.893+0000 7ff582ffd640 1 -- 172.21.15.40:0/1019118591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff578003970 msgr2=0x7ff58c15c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:58.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.893+0000 7ff582ffd640 1 --2- 172.21.15.40:0/1019118591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff578003970 0x7ff58c15c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.893+0000 7ff582ffd640 1 -- 172.21.15.40:0/1019118591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c12d700 con 0x7ff58c13f180 2023-10-18T09:02:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.893+0000 7ff58bfff640 1 --2- 172.21.15.40:0/1019118591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff58c1f2080 0x7ff58c1f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.893+0000 7ff582ffd640 1 --2- 172.21.15.40:0/1019118591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff58c13f180 0x7ff58c13f600 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ff57400e970 tx=0x7ff57400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:58.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.893+0000 7ff588ff9640 1 -- 172.21.15.40:0/1019118591 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57400ccc0 con 0x7ff58c13f180 2023-10-18T09:02:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.893+0000 7ff588ff9640 1 -- 172.21.15.40:0/1019118591 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff57400ce20 con 0x7ff58c13f180 2023-10-18T09:02:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.893+0000 7ff588ff9640 1 -- 172.21.15.40:0/1019118591 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff574005280 con 0x7ff58c13f180 2023-10-18T09:02:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.893+0000 7ff590eb4640 1 -- 172.21.15.40:0/1019118591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58c12d9e0 con 0x7ff58c13f180 2023-10-18T09:02:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.893+0000 7ff590eb4640 1 -- 172.21.15.40:0/1019118591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58c12e4c0 con 0x7ff58c13f180 2023-10-18T09:02:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.894+0000 7ff590eb4640 1 -- 172.21.15.40:0/1019118591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58c0c98c0 con 0x7ff58c13f180 2023-10-18T09:02:58.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.895+0000 7ff588ff9640 1 -- 172.21.15.40:0/1019118591 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff5740053e0 con 0x7ff58c13f180 2023-10-18T09:02:58.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.896+0000 7ff588ff9640 1 --2- 172.21.15.40:0/1019118591 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff56c06ae70 0x7ff56c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:58.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.896+0000 7ff588ff9640 1 -- 172.21.15.40:0/1019118591 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff574014070 con 0x7ff58c13f180 2023-10-18T09:02:58.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.897+0000 7ff58b7fe640 1 --2- 172.21.15.40:0/1019118591 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff56c06ae70 0x7ff56c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:58.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.898+0000 7ff58b7fe640 1 --2- 172.21.15.40:0/1019118591 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff56c06ae70 0x7ff56c06d330 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff57c034000 tx=0x7ff57c001a70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:58.901+0000 7ff588ff9640 1 -- 172.21.15.40:0/1019118591 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff574054fc0 con 0x7ff58c13f180 2023-10-18T09:02:59.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.095+0000 7ff590eb4640 1 -- 172.21.15.40:0/1019118591 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff58c0af340 con 0x7ff58c13f180 2023-10-18T09:02:59.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.097+0000 7ff588ff9640 1 -- 172.21.15.40:0/1019118591 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+16539 (secure 0 0 0) 0x7ff5740585e0 con 0x7ff58c13f180 2023-10-18T09:02:59.099 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:02:59.100 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":133,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:02:56.085956+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:02:39.531482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-18T09:02:40.812071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:02:59.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.099+0000 7ff590eb4640 1 -- 172.21.15.40:0/1019118591 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff56c06ae70 msgr2=0x7ff56c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:59.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.099+0000 7ff590eb4640 1 --2- 172.21.15.40:0/1019118591 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff56c06ae70 0x7ff56c06d330 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff57c034000 tx=0x7ff57c001a70 comp rx=0 tx=0).stop 2023-10-18T09:02:59.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.099+0000 7ff590eb4640 1 -- 172.21.15.40:0/1019118591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff58c13f180 msgr2=0x7ff58c13f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:59.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.099+0000 7ff590eb4640 1 --2- 172.21.15.40:0/1019118591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff58c13f180 0x7ff58c13f600 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ff57400e970 tx=0x7ff57400ee30 comp rx=0 tx=0).stop 2023-10-18T09:02:59.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.100+0000 7ff590eb4640 1 -- 172.21.15.40:0/1019118591 shutdown_connections 2023-10-18T09:02:59.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.100+0000 7ff590eb4640 1 --2- 172.21.15.40:0/1019118591 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff56c06ae70 0x7ff56c06d330 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.100+0000 7ff590eb4640 1 --2- 172.21.15.40:0/1019118591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff58c1f2080 0x7ff58c1f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.100+0000 7ff590eb4640 1 --2- 172.21.15.40:0/1019118591 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff58c13f180 0x7ff58c13f600 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.100+0000 7ff590eb4640 1 --2- 172.21.15.40:0/1019118591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff578003970 0x7ff58c15c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.100+0000 7ff590eb4640 1 -- 172.21.15.40:0/1019118591 >> 172.21.15.40:0/1019118591 conn(0x7ff58c069720 msgr2=0x7ff58c13b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:59.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.100+0000 7ff590eb4640 1 -- 172.21.15.40:0/1019118591 shutdown_connections 2023-10-18T09:02:59.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.100+0000 7ff590eb4640 1 -- 172.21.15.40:0/1019118591 wait complete. 2023-10-18T09:02:59.122 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:02:39.531482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:02:40.812071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:02:59.123 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:02:59.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.351+0000 7f28abfff640 1 -- 172.21.15.40:0/2492736349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28ac15e9f0 msgr2=0x7f28ac0b1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:59.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.351+0000 7f28abfff640 1 --2- 172.21.15.40:0/2492736349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28ac15e9f0 0x7f28ac0b1a50 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f289c005fd0 tx=0x7f289c02c0f0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.351+0000 7f28abfff640 1 -- 172.21.15.40:0/2492736349 shutdown_connections 2023-10-18T09:02:59.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.351+0000 7f28abfff640 1 --2- 172.21.15.40:0/2492736349 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28ac0b1f90 0x7f28ac0d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.351+0000 7f28abfff640 1 --2- 172.21.15.40:0/2492736349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28ac15e9f0 0x7f28ac0b1a50 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.351+0000 7f28abfff640 1 -- 172.21.15.40:0/2492736349 >> 172.21.15.40:0/2492736349 conn(0x7f28ac08ad10 msgr2=0x7f28ac09a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:59.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.351+0000 7f28abfff640 1 -- 172.21.15.40:0/2492736349 shutdown_connections 2023-10-18T09:02:59.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.352+0000 7f28abfff640 1 -- 172.21.15.40:0/2492736349 wait complete. 2023-10-18T09:02:59.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.353+0000 7f28abfff640 1 Processor -- start 2023-10-18T09:02:59.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.353+0000 7f28abfff640 1 -- start start 2023-10-18T09:02:59.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.354+0000 7f28abfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28ac0b1f90 0x7f28ac151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:59.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.354+0000 7f28abfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28ac0d1470 0x7f28ac0ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:59.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.354+0000 7f28abfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28ac15e9f0 0x7f28ac0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:59.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.354+0000 7f28abfff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac1e5460 con 0x7f28ac0d1470 2023-10-18T09:02:59.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.354+0000 7f28abfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac0c83c0 con 0x7f28ac15e9f0 2023-10-18T09:02:59.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.354+0000 7f28abfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac1525b0 con 0x7f28ac0b1f90 2023-10-18T09:02:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.355+0000 7f28aa7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28ac0d1470 0x7f28ac0ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.355+0000 7f28aa7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28ac0d1470 0x7f28ac0ca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45686/0 (socket says 172.21.15.40:45686) 2023-10-18T09:02:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.355+0000 7f28aa7fc640 1 -- 172.21.15.40:0/562894875 learned_addr learned my addr 172.21.15.40:0/562894875 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.355+0000 7f28ab7fe640 1 --2- 172.21.15.40:0/562894875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28ac15e9f0 0x7f28ac0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.355+0000 7f28aaffd640 1 --2- 172.21.15.40:0/562894875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28ac0b1f90 0x7f28ac151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:59.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.355+0000 7f28aa7fc640 1 -- 172.21.15.40:0/562894875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28ac0b1f90 msgr2=0x7f28ac151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:59.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.355+0000 7f28aa7fc640 1 --2- 172.21.15.40:0/562894875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28ac0b1f90 0x7f28ac151110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.355+0000 7f28aa7fc640 1 -- 172.21.15.40:0/562894875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28ac15e9f0 msgr2=0x7f28ac0caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:59.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.355+0000 7f28aa7fc640 1 --2- 172.21.15.40:0/562894875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28ac15e9f0 0x7f28ac0caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.355+0000 7f28aa7fc640 1 -- 172.21.15.40:0/562894875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c009840 con 0x7f28ac0d1470 2023-10-18T09:02:59.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.355+0000 7f28ab7fe640 1 --2- 172.21.15.40:0/562894875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28ac15e9f0 0x7f28ac0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:02:59.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.356+0000 7f28aa7fc640 1 --2- 172.21.15.40:0/562894875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28ac0d1470 0x7f28ac0ca560 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f289400c890 tx=0x7f289400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:59.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.356+0000 7f28a37fe640 1 -- 172.21.15.40:0/562894875 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2894004220 con 0x7f28ac0d1470 2023-10-18T09:02:59.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.356+0000 7f28a37fe640 1 -- 172.21.15.40:0/562894875 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2894004470 con 0x7f28ac0d1470 2023-10-18T09:02:59.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.356+0000 7f28a37fe640 1 -- 172.21.15.40:0/562894875 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2894007c10 con 0x7f28ac0d1470 2023-10-18T09:02:59.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.356+0000 7f28abfff640 1 -- 172.21.15.40:0/562894875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ac152890 con 0x7f28ac0d1470 2023-10-18T09:02:59.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.356+0000 7f28abfff640 1 -- 172.21.15.40:0/562894875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ac0c8610 con 0x7f28ac0d1470 2023-10-18T09:02:59.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.357+0000 7f28abfff640 1 -- 172.21.15.40:0/562894875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ac071da0 con 0x7f28ac0d1470 2023-10-18T09:02:59.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.358+0000 7f28a37fe640 1 -- 172.21.15.40:0/562894875 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2894005020 con 0x7f28ac0d1470 2023-10-18T09:02:59.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.359+0000 7f28a37fe640 1 --2- 172.21.15.40:0/562894875 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f288c06ae70 0x7f288c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:59.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.359+0000 7f28a37fe640 1 -- 172.21.15.40:0/562894875 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2894089b80 con 0x7f28ac0d1470 2023-10-18T09:02:59.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.360+0000 7f28aaffd640 1 --2- 172.21.15.40:0/562894875 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f288c06ae70 0x7f288c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:59.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.361+0000 7f28aaffd640 1 --2- 172.21.15.40:0/562894875 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f288c06ae70 0x7f288c06d330 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f289c035000 tx=0x7f289c004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:59.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.363+0000 7f28a37fe640 1 -- 172.21.15.40:0/562894875 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2894054c30 con 0x7f28ac0d1470 2023-10-18T09:02:59.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.555+0000 7f28abfff640 1 -- 172.21.15.40:0/562894875 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28ac0c8830 con 0x7f28ac0d1470 2023-10-18T09:02:59.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.556+0000 7f28a37fe640 1 -- 172.21.15.40:0/562894875 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7f289401d070 con 0x7f28ac0d1470 2023-10-18T09:02:59.558 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:02:59.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.559+0000 7f28abfff640 1 -- 172.21.15.40:0/562894875 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f288c06ae70 msgr2=0x7f288c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:59.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.559+0000 7f28abfff640 1 --2- 172.21.15.40:0/562894875 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f288c06ae70 0x7f288c06d330 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f289c035000 tx=0x7f289c004ab0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.559+0000 7f28abfff640 1 -- 172.21.15.40:0/562894875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28ac0d1470 msgr2=0x7f28ac0ca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:59.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.559+0000 7f28abfff640 1 --2- 172.21.15.40:0/562894875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28ac0d1470 0x7f28ac0ca560 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f289400c890 tx=0x7f289400cd50 comp rx=0 tx=0).stop 2023-10-18T09:02:59.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.559+0000 7f28abfff640 1 -- 172.21.15.40:0/562894875 shutdown_connections 2023-10-18T09:02:59.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.559+0000 7f28abfff640 1 --2- 172.21.15.40:0/562894875 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f288c06ae70 0x7f288c06d330 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.559+0000 7f28abfff640 1 --2- 172.21.15.40:0/562894875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28ac15e9f0 0x7f28ac0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.559+0000 7f28abfff640 1 --2- 172.21.15.40:0/562894875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28ac0d1470 0x7f28ac0ca560 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.559+0000 7f28abfff640 1 --2- 172.21.15.40:0/562894875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28ac0b1f90 0x7f28ac151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.559+0000 7f28abfff640 1 -- 172.21.15.40:0/562894875 >> 172.21.15.40:0/562894875 conn(0x7f28ac08ad10 msgr2=0x7f28ac06be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:59.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.559+0000 7f28abfff640 1 -- 172.21.15.40:0/562894875 shutdown_connections 2023-10-18T09:02:59.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.560+0000 7f28abfff640 1 -- 172.21.15.40:0/562894875 wait complete. 2023-10-18T09:02:59.580 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:02:59.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.814+0000 7f669d935640 1 -- 172.21.15.40:0/2875788301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669815af70 msgr2=0x7f6698157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:59.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.814+0000 7f669d935640 1 --2- 172.21.15.40:0/2875788301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669815af70 0x7f6698157ad0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f6690005fd0 tx=0x7f669002c240 comp rx=0 tx=0).stop 2023-10-18T09:02:59.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.814+0000 7f669d935640 1 -- 172.21.15.40:0/2875788301 shutdown_connections 2023-10-18T09:02:59.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.814+0000 7f669d935640 1 --2- 172.21.15.40:0/2875788301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f669815af70 0x7f6698157ad0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.814+0000 7f669d935640 1 --2- 172.21.15.40:0/2875788301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6698071270 0x7f66980c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.814+0000 7f669d935640 1 -- 172.21.15.40:0/2875788301 >> 172.21.15.40:0/2875788301 conn(0x7f669806ca10 msgr2=0x7f669806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:02:59.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.815+0000 7f669d935640 1 -- 172.21.15.40:0/2875788301 shutdown_connections 2023-10-18T09:02:59.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.815+0000 7f669d935640 1 -- 172.21.15.40:0/2875788301 wait complete. 2023-10-18T09:02:59.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.816+0000 7f669d935640 1 Processor -- start 2023-10-18T09:02:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.816+0000 7f669d935640 1 -- start start 2023-10-18T09:02:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.817+0000 7f669d935640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f669806d5d0 0x7f6698150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.817+0000 7f669d935640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6698071270 0x7f66981623c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.817+0000 7f669d935640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f669815af70 0x7f66981614e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:59.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.817+0000 7f669d935640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6698155060 con 0x7f6698071270 2023-10-18T09:02:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.817+0000 7f669d935640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6698151990 con 0x7f669806d5d0 2023-10-18T09:02:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.817+0000 7f669d935640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6698151c70 con 0x7f669815af70 2023-10-18T09:02:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.818+0000 7f6697fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6698071270 0x7f66981623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.818+0000 7f6697fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6698071270 0x7f66981623c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45706/0 (socket says 172.21.15.40:45706) 2023-10-18T09:02:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.818+0000 7f6697fff640 1 -- 172.21.15.40:0/3167530282 learned_addr learned my addr 172.21.15.40:0/3167530282 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:02:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.818+0000 7f669d134640 1 --2- 172.21.15.40:0/3167530282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f669815af70 0x7f66981614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.818+0000 7f669c933640 1 --2- 172.21.15.40:0/3167530282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f669806d5d0 0x7f6698150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:59.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.819+0000 7f6697fff640 1 -- 172.21.15.40:0/3167530282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f669815af70 msgr2=0x7f66981614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.819+0000 7f6697fff640 1 --2- 172.21.15.40:0/3167530282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f669815af70 0x7f66981614e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.819+0000 7f6697fff640 1 -- 172.21.15.40:0/3167530282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f669806d5d0 msgr2=0x7f6698150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:02:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.819+0000 7f6697fff640 1 --2- 172.21.15.40:0/3167530282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f669806d5d0 0x7f6698150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:02:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.819+0000 7f6697fff640 1 -- 172.21.15.40:0/3167530282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6690009900 con 0x7f6698071270 2023-10-18T09:02:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.819+0000 7f6697fff640 1 --2- 172.21.15.40:0/3167530282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6698071270 0x7f66981623c0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f668400c920 tx=0x7f668400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.819+0000 7f66957fa640 1 -- 172.21.15.40:0/3167530282 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684007c70 con 0x7f6698071270 2023-10-18T09:02:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.819+0000 7f66957fa640 1 -- 172.21.15.40:0/3167530282 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f668400f450 con 0x7f6698071270 2023-10-18T09:02:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.819+0000 7f669d935640 1 -- 172.21.15.40:0/3167530282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6698151f50 con 0x7f6698071270 2023-10-18T09:02:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.819+0000 7f66957fa640 1 -- 172.21.15.40:0/3167530282 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668400f860 con 0x7f6698071270 2023-10-18T09:02:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.820+0000 7f669d935640 1 -- 172.21.15.40:0/3167530282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6698154880 con 0x7f6698071270 2023-10-18T09:02:59.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.821+0000 7f669d935640 1 -- 172.21.15.40:0/3167530282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6698066e00 con 0x7f6698071270 2023-10-18T09:02:59.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.822+0000 7f66957fa640 1 -- 172.21.15.40:0/3167530282 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f668400f9c0 con 0x7f6698071270 2023-10-18T09:02:59.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.823+0000 7f66957fa640 1 --2- 172.21.15.40:0/3167530282 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f667c06ae40 0x7f667c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:02:59.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.823+0000 7f66957fa640 1 -- 172.21.15.40:0/3167530282 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6684089e70 con 0x7f6698071270 2023-10-18T09:02:59.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.824+0000 7f669c933640 1 --2- 172.21.15.40:0/3167530282 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f667c06ae40 0x7f667c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:02:59.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.825+0000 7f669c933640 1 --2- 172.21.15.40:0/3167530282 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f667c06ae40 0x7f667c06d300 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f668c000f60 tx=0x7f668c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:02:59.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:02:59.828+0000 7f66957fa640 1 -- 172.21.15.40:0/3167530282 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6684054f30 con 0x7f6698071270 2023-10-18T09:03:00.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.019+0000 7f669d935640 1 -- 172.21.15.40:0/3167530282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6698157da0 con 0x7f6698071270 2023-10-18T09:03:00.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.019+0000 7f66957fa640 1 -- 172.21.15.40:0/3167530282 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v133) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6684058550 con 0x7f6698071270 2023-10-18T09:03:00.022 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:03:00.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.022+0000 7f669d935640 1 -- 172.21.15.40:0/3167530282 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f667c06ae40 msgr2=0x7f667c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:00.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.022+0000 7f669d935640 1 --2- 172.21.15.40:0/3167530282 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f667c06ae40 0x7f667c06d300 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f668c000f60 tx=0x7f668c009040 comp rx=0 tx=0).stop 2023-10-18T09:03:00.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.022+0000 7f669d935640 1 -- 172.21.15.40:0/3167530282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6698071270 msgr2=0x7f66981623c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:00.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.022+0000 7f669d935640 1 --2- 172.21.15.40:0/3167530282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6698071270 0x7f66981623c0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f668400c920 tx=0x7f668400cde0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.022+0000 7f669d935640 1 -- 172.21.15.40:0/3167530282 shutdown_connections 2023-10-18T09:03:00.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.022+0000 7f669d935640 1 --2- 172.21.15.40:0/3167530282 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f667c06ae40 0x7f667c06d300 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.022+0000 7f669d935640 1 --2- 172.21.15.40:0/3167530282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f669815af70 0x7f66981614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.022+0000 7f669d935640 1 --2- 172.21.15.40:0/3167530282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6698071270 0x7f66981623c0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.022+0000 7f669d935640 1 --2- 172.21.15.40:0/3167530282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f669806d5d0 0x7f6698150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.022+0000 7f669d935640 1 -- 172.21.15.40:0/3167530282 >> 172.21.15.40:0/3167530282 conn(0x7f669806ca10 msgr2=0x7f6698150570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:00.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.022+0000 7f669d935640 1 -- 172.21.15.40:0/3167530282 shutdown_connections 2023-10-18T09:03:00.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.022+0000 7f669d935640 1 -- 172.21.15.40:0/3167530282 wait complete. 2023-10-18T09:03:00.044 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:03:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.284+0000 7f0f11487640 1 -- 172.21.15.40:0/2320158877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0c06bc40 msgr2=0x7f0f0c0c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.284+0000 7f0f11487640 1 --2- 172.21.15.40:0/2320158877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0c06bc40 0x7f0f0c0c9f60 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c06b500 tx=0x7f0f0002bd90 comp rx=0 tx=0).stop 2023-10-18T09:03:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.284+0000 7f0f11487640 1 -- 172.21.15.40:0/2320158877 shutdown_connections 2023-10-18T09:03:00.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.284+0000 7f0f11487640 1 --2- 172.21.15.40:0/2320158877 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f0c0717a0 0x7f0f0c0b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.284+0000 7f0f11487640 1 --2- 172.21.15.40:0/2320158877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0c06bc40 0x7f0f0c0c9f60 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.284+0000 7f0f11487640 1 --2- 172.21.15.40:0/2320158877 >> v2:172.21.15.144:3300/0 conn(0x7f0f0c0c9640 0x7f0f0c0c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.284+0000 7f0f11487640 1 -- 172.21.15.40:0/2320158877 >> 172.21.15.40:0/2320158877 conn(0x7f0f0c069720 msgr2=0x7f0f0c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:00.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.284+0000 7f0f11487640 1 -- 172.21.15.40:0/2320158877 shutdown_connections 2023-10-18T09:03:00.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.284+0000 7f0f11487640 1 -- 172.21.15.40:0/2320158877 wait complete. 2023-10-18T09:03:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.286+0000 7f0f11487640 1 Processor -- start 2023-10-18T09:03:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.286+0000 7f0f11487640 1 -- start start 2023-10-18T09:03:00.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.287+0000 7f0f11487640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f0c06bc40 0x7f0f0c14ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:00.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.287+0000 7f0f11487640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f0c0717a0 0x7f0f0c0ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:00.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.287+0000 7f0f11487640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0c0c9640 0x7f0f0c070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:00.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.287+0000 7f0f11487640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0ad200 con 0x7f0f0c0c9640 2023-10-18T09:03:00.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.287+0000 7f0f11487640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0ad450 con 0x7f0f0c06bc40 2023-10-18T09:03:00.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.287+0000 7f0f11487640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c0ad730 con 0x7f0f0c0717a0 2023-10-18T09:03:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.287+0000 7f0f0b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0c0c9640 0x7f0f0c070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.287+0000 7f0f0b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0c0c9640 0x7f0f0c070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45728/0 (socket says 172.21.15.40:45728) 2023-10-18T09:03:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.287+0000 7f0f0b7fe640 1 -- 172.21.15.40:0/1132239846 learned_addr learned my addr 172.21.15.40:0/1132239846 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.288+0000 7f0f0affd640 1 --2- 172.21.15.40:0/1132239846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f0c06bc40 0x7f0f0c14ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.288+0000 7f0f0a7fc640 1 --2- 172.21.15.40:0/1132239846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f0c0717a0 0x7f0f0c0ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.288+0000 7f0f0b7fe640 1 -- 172.21.15.40:0/1132239846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f0c0717a0 msgr2=0x7f0f0c0ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:00.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.288+0000 7f0f0b7fe640 1 --2- 172.21.15.40:0/1132239846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f0c0717a0 0x7f0f0c0ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.288+0000 7f0f0b7fe640 1 -- 172.21.15.40:0/1132239846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f0c06bc40 msgr2=0x7f0f0c14ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:00.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.288+0000 7f0f0b7fe640 1 --2- 172.21.15.40:0/1132239846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f0c06bc40 0x7f0f0c14ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.288+0000 7f0f0b7fe640 1 -- 172.21.15.40:0/1132239846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f00009560 con 0x7f0f0c0c9640 2023-10-18T09:03:00.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.288+0000 7f0f0affd640 1 --2- 172.21.15.40:0/1132239846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f0c06bc40 0x7f0f0c14ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:00.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.288+0000 7f0f0b7fe640 1 --2- 172.21.15.40:0/1132239846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0c0c9640 0x7f0f0c070700 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400cad0 tx=0x7f0f0400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.288+0000 7f0ef37fe640 1 -- 172.21.15.40:0/1132239846 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0400ae20 con 0x7f0f0c0c9640 2023-10-18T09:03:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.288+0000 7f0ef37fe640 1 -- 172.21.15.40:0/1132239846 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0f04003aa0 con 0x7f0f0c0c9640 2023-10-18T09:03:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.288+0000 7f0ef37fe640 1 -- 172.21.15.40:0/1132239846 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04004030 con 0x7f0f0c0c9640 2023-10-18T09:03:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.288+0000 7f0f11487640 1 -- 172.21.15.40:0/1132239846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0c0ada10 con 0x7f0f0c0c9640 2023-10-18T09:03:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.289+0000 7f0f11487640 1 -- 172.21.15.40:0/1132239846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0c0adfd0 con 0x7f0f0c0c9640 2023-10-18T09:03:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.290+0000 7f0f11487640 1 -- 172.21.15.40:0/1132239846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0c0ae1c0 con 0x7f0f0c0c9640 2023-10-18T09:03:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.291+0000 7f0ef37fe640 1 -- 172.21.15.40:0/1132239846 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0f04004190 con 0x7f0f0c0c9640 2023-10-18T09:03:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.292+0000 7f0ef37fe640 1 --2- 172.21.15.40:0/1132239846 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ef806ae40 0x7f0ef806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:00.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.292+0000 7f0ef37fe640 1 -- 172.21.15.40:0/1132239846 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f04089f00 con 0x7f0f0c0c9640 2023-10-18T09:03:00.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.292+0000 7f0f0affd640 1 --2- 172.21.15.40:0/1132239846 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ef806ae40 0x7f0ef806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:00.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.294+0000 7f0f0affd640 1 --2- 172.21.15.40:0/1132239846 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ef806ae40 0x7f0ef806d300 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0058c0 tx=0x7f0efc005c10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:00.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.296+0000 7f0ef37fe640 1 -- 172.21.15.40:0/1132239846 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0f04054f10 con 0x7f0f0c0c9640 2023-10-18T09:03:00.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.490+0000 7f0f11487640 1 -- 172.21.15.40:0/1132239846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f0c066fd0 con 0x7f0f0c0c9640 2023-10-18T09:03:00.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.491+0000 7f0ef37fe640 1 -- 172.21.15.40:0/1132239846 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v133) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f0401d020 con 0x7f0f0c0c9640 2023-10-18T09:03:00.494 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:03:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.494+0000 7f0f11487640 1 -- 172.21.15.40:0/1132239846 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ef806ae40 msgr2=0x7f0ef806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.494+0000 7f0f11487640 1 --2- 172.21.15.40:0/1132239846 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ef806ae40 0x7f0ef806d300 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0058c0 tx=0x7f0efc005c10 comp rx=0 tx=0).stop 2023-10-18T09:03:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.494+0000 7f0f11487640 1 -- 172.21.15.40:0/1132239846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0c0c9640 msgr2=0x7f0f0c070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:00.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.494+0000 7f0f11487640 1 --2- 172.21.15.40:0/1132239846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0c0c9640 0x7f0f0c070700 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400cad0 tx=0x7f0f0400cf90 comp rx=0 tx=0).stop 2023-10-18T09:03:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.494+0000 7f0f11487640 1 -- 172.21.15.40:0/1132239846 shutdown_connections 2023-10-18T09:03:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.494+0000 7f0f11487640 1 --2- 172.21.15.40:0/1132239846 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ef806ae40 0x7f0ef806d300 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.494+0000 7f0f11487640 1 --2- 172.21.15.40:0/1132239846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0c0c9640 0x7f0f0c070700 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.494+0000 7f0f11487640 1 --2- 172.21.15.40:0/1132239846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f0c0717a0 0x7f0f0c0ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.494+0000 7f0f11487640 1 --2- 172.21.15.40:0/1132239846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f0c06bc40 0x7f0f0c14ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.494+0000 7f0f11487640 1 -- 172.21.15.40:0/1132239846 >> 172.21.15.40:0/1132239846 conn(0x7f0f0c069720 msgr2=0x7f0f0c15a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.494+0000 7f0f11487640 1 -- 172.21.15.40:0/1132239846 shutdown_connections 2023-10-18T09:03:00.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.494+0000 7f0f11487640 1 -- 172.21.15.40:0/1132239846 wait complete. 2023-10-18T09:03:00.516 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:03:00.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.739+0000 7feb04a44640 1 -- 172.21.15.40:0/3943018461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb0013f5a0 msgr2=0x7feb000c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:00.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.739+0000 7feb04a44640 1 --2- 172.21.15.40:0/3943018461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb0013f5a0 0x7feb000c9b00 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7feae8009860 tx=0x7feae802bd40 comp rx=0 tx=0).stop 2023-10-18T09:03:00.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.739+0000 7feb04a44640 1 -- 172.21.15.40:0/3943018461 shutdown_connections 2023-10-18T09:03:00.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.739+0000 7feb04a44640 1 --2- 172.21.15.40:0/3943018461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb000ca040 0x7feb000cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.739+0000 7feb04a44640 1 --2- 172.21.15.40:0/3943018461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb0013f5a0 0x7feb000c9b00 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.739+0000 7feb04a44640 1 -- 172.21.15.40:0/3943018461 >> 172.21.15.40:0/3943018461 conn(0x7feb0006cee0 msgr2=0x7feb00069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:00.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.739+0000 7feb04a44640 1 -- 172.21.15.40:0/3943018461 shutdown_connections 2023-10-18T09:03:00.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.739+0000 7feb04a44640 1 -- 172.21.15.40:0/3943018461 wait complete. 2023-10-18T09:03:00.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.741+0000 7feb04a44640 1 Processor -- start 2023-10-18T09:03:00.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.741+0000 7feb04a44640 1 -- start start 2023-10-18T09:03:00.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.742+0000 7feb04a44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb000ca040 0x7feb00155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.742+0000 7feb04a44640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb0013c470 0x7feb0013b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.742+0000 7feb04a44640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb0013f5a0 0x7feb00156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.742+0000 7feb04a44640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb00155720 con 0x7feb0013c470 2023-10-18T09:03:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.742+0000 7feb04a44640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb00156cd0 con 0x7feb000ca040 2023-10-18T09:03:00.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.742+0000 7feb04a44640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb00156fb0 con 0x7feb0013f5a0 2023-10-18T09:03:00.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.742+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb0013c470 0x7feb0013b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:00.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.742+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb0013c470 0x7feb0013b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45734/0 (socket says 172.21.15.40:45734) 2023-10-18T09:03:00.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.742+0000 7feaf7fff640 1 -- 172.21.15.40:0/715564138 learned_addr learned my addr 172.21.15.40:0/715564138 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.743+0000 7feafed9d640 1 --2- 172.21.15.40:0/715564138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb0013f5a0 0x7feb00156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.743+0000 7feaf7fff640 1 -- 172.21.15.40:0/715564138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb0013f5a0 msgr2=0x7feb00156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.743+0000 7feafe59c640 1 --2- 172.21.15.40:0/715564138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb000ca040 0x7feb00155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.743+0000 7feaf7fff640 1 --2- 172.21.15.40:0/715564138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb0013f5a0 0x7feb00156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.743+0000 7feaf7fff640 1 -- 172.21.15.40:0/715564138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb000ca040 msgr2=0x7feb00155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:00.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.743+0000 7feaf7fff640 1 --2- 172.21.15.40:0/715564138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb000ca040 0x7feb00155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.743+0000 7feaf7fff640 1 -- 172.21.15.40:0/715564138 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae8009490 con 0x7feb0013c470 2023-10-18T09:03:00.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.743+0000 7feafe59c640 1 --2- 172.21.15.40:0/715564138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb000ca040 0x7feb00155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:00.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.743+0000 7feaf7fff640 1 --2- 172.21.15.40:0/715564138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb0013c470 0x7feb0013b340 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7feae8009980 tx=0x7feae8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:00.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.744+0000 7feaf6ffd640 1 -- 172.21.15.40:0/715564138 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8003e30 con 0x7feb0013c470 2023-10-18T09:03:00.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.744+0000 7feaf6ffd640 1 -- 172.21.15.40:0/715564138 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feae8005020 con 0x7feb0013c470 2023-10-18T09:03:00.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.744+0000 7feaf6ffd640 1 -- 172.21.15.40:0/715564138 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae8005430 con 0x7feb0013c470 2023-10-18T09:03:00.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.744+0000 7feb04a44640 1 -- 172.21.15.40:0/715564138 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb00155060 con 0x7feb0013c470 2023-10-18T09:03:00.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.744+0000 7feb04a44640 1 -- 172.21.15.40:0/715564138 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0013bdf0 con 0x7feb0013c470 2023-10-18T09:03:00.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.745+0000 7feb04a44640 1 -- 172.21.15.40:0/715564138 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb00155860 con 0x7feb0013c470 2023-10-18T09:03:00.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.746+0000 7feaf6ffd640 1 -- 172.21.15.40:0/715564138 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feae80047a0 con 0x7feb0013c470 2023-10-18T09:03:00.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.747+0000 7feaf6ffd640 1 --2- 172.21.15.40:0/715564138 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feae006ae70 0x7feae006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:00.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.747+0000 7feaf6ffd640 1 -- 172.21.15.40:0/715564138 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feae80aba20 con 0x7feb0013c470 2023-10-18T09:03:00.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.748+0000 7feafe59c640 1 --2- 172.21.15.40:0/715564138 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feae006ae70 0x7feae006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:00.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.749+0000 7feafe59c640 1 --2- 172.21.15.40:0/715564138 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feae006ae70 0x7feae006d330 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7feb000af2b0 tx=0x7feaf0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:00.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.755+0000 7feaf6ffd640 1 -- 172.21.15.40:0/715564138 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb00155860 con 0x7feb0013c470 2023-10-18T09:03:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.963+0000 7feb04a44640 1 -- 172.21.15.40:0/715564138 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feb00155860 con 0x7feb0013c470 2023-10-18T09:03:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.963+0000 7feaf6ffd640 1 -- 172.21.15.40:0/715564138 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v133) v1 ==== 80+0+0 (secure 0 0 0) 0x7feb00155860 con 0x7feb0013c470 2023-10-18T09:03:00.966 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T09:03:00.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.966+0000 7feb04a44640 1 -- 172.21.15.40:0/715564138 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feae006ae70 msgr2=0x7feae006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:00.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.966+0000 7feb04a44640 1 --2- 172.21.15.40:0/715564138 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feae006ae70 0x7feae006d330 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7feb000af2b0 tx=0x7feaf0009040 comp rx=0 tx=0).stop 2023-10-18T09:03:00.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.966+0000 7feb04a44640 1 -- 172.21.15.40:0/715564138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb0013c470 msgr2=0x7feb0013b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:00.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.966+0000 7feb04a44640 1 --2- 172.21.15.40:0/715564138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb0013c470 0x7feb0013b340 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7feae8009980 tx=0x7feae8005dc0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.966+0000 7feb04a44640 1 -- 172.21.15.40:0/715564138 shutdown_connections 2023-10-18T09:03:00.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.966+0000 7feb04a44640 1 --2- 172.21.15.40:0/715564138 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feae006ae70 0x7feae006d330 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.966+0000 7feb04a44640 1 --2- 172.21.15.40:0/715564138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb0013f5a0 0x7feb00156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.966+0000 7feb04a44640 1 --2- 172.21.15.40:0/715564138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb0013c470 0x7feb0013b340 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.966+0000 7feb04a44640 1 --2- 172.21.15.40:0/715564138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb000ca040 0x7feb00155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.966+0000 7feb04a44640 1 -- 172.21.15.40:0/715564138 >> 172.21.15.40:0/715564138 conn(0x7feb0006cee0 msgr2=0x7feb00069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.966+0000 7feb04a44640 1 -- 172.21.15.40:0/715564138 shutdown_connections 2023-10-18T09:03:00.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:00.966+0000 7feb04a44640 1 -- 172.21.15.40:0/715564138 wait complete. 2023-10-18T09:03:00.987 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-18T09:03:00.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:00.987 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-18T09:03:01.125 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T09:03:01.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:01.126 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T09:03:01.175 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:03:01.175 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:03:01.220 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:03:01.221 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:03:01.317 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:03:01.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.548+0000 7f4317039640 1 -- 172.21.15.40:0/660214368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42fc004d00 msgr2=0x7f42fc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:01.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.548+0000 7f4317039640 1 --2- 172.21.15.40:0/660214368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42fc004d00 0x7f42fc005180 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f431006a2c0 tx=0x7f42fc0057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:01.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.548+0000 7f4316838640 1 -- 172.21.15.40:0/660214368 reap_dead start 2023-10-18T09:03:01.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.548+0000 7f4317039640 1 -- 172.21.15.40:0/660214368 shutdown_connections 2023-10-18T09:03:01.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.548+0000 7f4317039640 1 -- 172.21.15.40:0/660214368 >> 172.21.15.40:0/660214368 conn(0x7f4310069720 msgr2=0x7f4310069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:01.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.548+0000 7f4317039640 1 -- 172.21.15.40:0/660214368 shutdown_connections 2023-10-18T09:03:01.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.548+0000 7f4317039640 1 -- 172.21.15.40:0/660214368 wait complete. 2023-10-18T09:03:01.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.550+0000 7f4317039640 1 Processor -- start 2023-10-18T09:03:01.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.550+0000 7f4317039640 1 -- start start 2023-10-18T09:03:01.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.551+0000 7f4317039640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42fc004d00 0x7f431012a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:01.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.551+0000 7f4317039640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431013b230 0x7f4310141220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:01.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.551+0000 7f4317039640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4310157a30 0x7f431015a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:01.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.551+0000 7f4317039640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4310141760 con 0x7f42fc004d00 2023-10-18T09:03:01.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.551+0000 7f4317039640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431013bb10 con 0x7f431013b230 2023-10-18T09:03:01.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.551+0000 7f4317039640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43100ad970 con 0x7f4310157a30 2023-10-18T09:03:01.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.551+0000 7f4316037640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42fc004d00 0x7f431012a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:01.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.552+0000 7f4316037640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42fc004d00 0x7f431012a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45756/0 (socket says 172.21.15.40:45756) 2023-10-18T09:03:01.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.552+0000 7f4316037640 1 -- 172.21.15.40:0/1014479063 learned_addr learned my addr 172.21.15.40:0/1014479063 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:01.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.552+0000 7f4315836640 1 --2- 172.21.15.40:0/1014479063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431013b230 0x7f4310141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:01.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.552+0000 7f4316838640 1 --2- 172.21.15.40:0/1014479063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4310157a30 0x7f431015a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:01.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.552+0000 7f4316037640 1 -- 172.21.15.40:0/1014479063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4310157a30 msgr2=0x7f431015a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:01.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.552+0000 7f4316037640 1 --2- 172.21.15.40:0/1014479063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4310157a30 0x7f431015a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:01.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.552+0000 7f4316037640 1 -- 172.21.15.40:0/1014479063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431013b230 msgr2=0x7f4310141220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:01.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.552+0000 7f4316037640 1 --2- 172.21.15.40:0/1014479063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431013b230 0x7f4310141220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:01.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.552+0000 7f4316037640 1 -- 172.21.15.40:0/1014479063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43100acd60 con 0x7f42fc004d00 2023-10-18T09:03:01.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.552+0000 7f4316037640 1 --2- 172.21.15.40:0/1014479063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42fc004d00 0x7f431012a1c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f430c010a10 tx=0x7f430c010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:01.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.553+0000 7f4306ffd640 1 -- 172.21.15.40:0/1014479063 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c00ec80 con 0x7f42fc004d00 2023-10-18T09:03:01.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.553+0000 7f4306ffd640 1 -- 172.21.15.40:0/1014479063 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f430c00ede0 con 0x7f42fc004d00 2023-10-18T09:03:01.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.553+0000 7f4306ffd640 1 -- 172.21.15.40:0/1014479063 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c004290 con 0x7f42fc004d00 2023-10-18T09:03:01.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.553+0000 7f4317039640 1 -- 172.21.15.40:0/1014479063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43100ad040 con 0x7f42fc004d00 2023-10-18T09:03:01.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.553+0000 7f4317039640 1 -- 172.21.15.40:0/1014479063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43100ad600 con 0x7f42fc004d00 2023-10-18T09:03:01.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.554+0000 7f4317039640 1 -- 172.21.15.40:0/1014479063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4310066e00 con 0x7f42fc004d00 2023-10-18T09:03:01.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.555+0000 7f4306ffd640 1 -- 172.21.15.40:0/1014479063 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f430c0035f0 con 0x7f42fc004d00 2023-10-18T09:03:01.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.556+0000 7f4306ffd640 1 --2- 172.21.15.40:0/1014479063 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f42f406ae40 0x7f42f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:01.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.556+0000 7f4306ffd640 1 -- 172.21.15.40:0/1014479063 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f430c016070 con 0x7f42fc004d00 2023-10-18T09:03:01.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.557+0000 7f4315836640 1 --2- 172.21.15.40:0/1014479063 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f42f406ae40 0x7f42f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:01.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.558+0000 7f4315836640 1 --2- 172.21.15.40:0/1014479063 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f42f406ae40 0x7f42f406d300 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f43101f4ea0 tx=0x7f4300005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:01.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.560+0000 7f4306ffd640 1 -- 172.21.15.40:0/1014479063 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f430c055540 con 0x7f42fc004d00 2023-10-18T09:03:01.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.756+0000 7f4317039640 1 -- 172.21.15.40:0/1014479063 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f431006f730 con 0x7f42fc004d00 2023-10-18T09:03:01.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.759+0000 7f4306ffd640 1 -- 172.21.15.40:0/1014479063 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+16539 (secure 0 0 0) 0x7f430c058b60 con 0x7f42fc004d00 2023-10-18T09:03:01.761 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:01.761 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":133,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:02:56.085956+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:02:39.531482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-18T09:02:40.812071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:03:01.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.761+0000 7f4317039640 1 -- 172.21.15.40:0/1014479063 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f42f406ae40 msgr2=0x7f42f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:01.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.761+0000 7f4317039640 1 --2- 172.21.15.40:0/1014479063 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f42f406ae40 0x7f42f406d300 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f43101f4ea0 tx=0x7f4300005f90 comp rx=0 tx=0).stop 2023-10-18T09:03:01.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.762+0000 7f4317039640 1 -- 172.21.15.40:0/1014479063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42fc004d00 msgr2=0x7f431012a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:01.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.762+0000 7f4317039640 1 --2- 172.21.15.40:0/1014479063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42fc004d00 0x7f431012a1c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f430c010a10 tx=0x7f430c010ed0 comp rx=0 tx=0).stop 2023-10-18T09:03:01.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.762+0000 7f4317039640 1 -- 172.21.15.40:0/1014479063 shutdown_connections 2023-10-18T09:03:01.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.762+0000 7f4317039640 1 --2- 172.21.15.40:0/1014479063 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f42f406ae40 0x7f42f406d300 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:01.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.762+0000 7f4317039640 1 --2- 172.21.15.40:0/1014479063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4310157a30 0x7f431015a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:01.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.762+0000 7f4317039640 1 --2- 172.21.15.40:0/1014479063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431013b230 0x7f4310141220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:01.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.762+0000 7f4317039640 1 --2- 172.21.15.40:0/1014479063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42fc004d00 0x7f431012a1c0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:01.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.762+0000 7f4317039640 1 -- 172.21.15.40:0/1014479063 >> 172.21.15.40:0/1014479063 conn(0x7f4310069720 msgr2=0x7f431013d730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:01.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.762+0000 7f4317039640 1 -- 172.21.15.40:0/1014479063 shutdown_connections 2023-10-18T09:03:01.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:01.762+0000 7f4317039640 1 -- 172.21.15.40:0/1014479063 wait complete. 2023-10-18T09:03:01.783 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:02:39.531482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:02:40.812071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:03:01.784 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:03:02.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.014+0000 7f90a0b0a640 1 -- 172.21.15.40:0/1881186557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f909c0b1900 msgr2=0x7f909c0af310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:02.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.014+0000 7f90a0b0a640 1 --2- 172.21.15.40:0/1881186557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f909c0b1900 0x7f909c0af310 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9094005fd0 tx=0x7f909402c0f0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.015+0000 7f90a0b0a640 1 -- 172.21.15.40:0/1881186557 shutdown_connections 2023-10-18T09:03:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.015+0000 7f90a0b0a640 1 --2- 172.21.15.40:0/1881186557 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f909c0b1900 0x7f909c0af310 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.015+0000 7f90a0b0a640 1 --2- 172.21.15.40:0/1881186557 >> v2:172.21.15.144:3300/0 conn(0x7f909c06d600 0x7f909c0c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.015+0000 7f90a0b0a640 1 -- 172.21.15.40:0/1881186557 >> 172.21.15.40:0/1881186557 conn(0x7f909c069720 msgr2=0x7f909c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.015+0000 7f90a0b0a640 1 -- 172.21.15.40:0/1881186557 shutdown_connections 2023-10-18T09:03:02.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.015+0000 7f90a0b0a640 1 -- 172.21.15.40:0/1881186557 wait complete. 2023-10-18T09:03:02.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.016+0000 7f90a0b0a640 1 Processor -- start 2023-10-18T09:03:02.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.017+0000 7f90a0b0a640 1 -- start start 2023-10-18T09:03:02.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.017+0000 7f90a0b0a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f909c06d600 0x7f909c13aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:02.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.017+0000 7f90a0b0a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f909c0b1900 0x7f909c1f3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:02.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.017+0000 7f90a0b0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f909c1405a0 0x7f909c13ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:02.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.017+0000 7f90a0b0a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c13b650 con 0x7f909c1405a0 2023-10-18T09:03:02.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.017+0000 7f90a0b0a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c13af90 con 0x7f909c06d600 2023-10-18T09:03:02.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.017+0000 7f90a0b0a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c13b8a0 con 0x7f909c0b1900 2023-10-18T09:03:02.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.018+0000 7f909bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f909c1405a0 0x7f909c13ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:02.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.018+0000 7f909bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f909c1405a0 0x7f909c13ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45762/0 (socket says 172.21.15.40:45762) 2023-10-18T09:03:02.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.018+0000 7f909bfff640 1 -- 172.21.15.40:0/4246258734 learned_addr learned my addr 172.21.15.40:0/4246258734 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:02.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.018+0000 7f909b7fe640 1 --2- 172.21.15.40:0/4246258734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f909c06d600 0x7f909c13aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:02.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.018+0000 7f9093fff640 1 --2- 172.21.15.40:0/4246258734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f909c0b1900 0x7f909c1f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:02.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.018+0000 7f909bfff640 1 -- 172.21.15.40:0/4246258734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f909c0b1900 msgr2=0x7f909c1f3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:02.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.018+0000 7f909bfff640 1 --2- 172.21.15.40:0/4246258734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f909c0b1900 0x7f909c1f3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.018+0000 7f909bfff640 1 -- 172.21.15.40:0/4246258734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f909c06d600 msgr2=0x7f909c13aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:02.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.018+0000 7f909bfff640 1 --2- 172.21.15.40:0/4246258734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f909c06d600 0x7f909c13aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.019+0000 7f909bfff640 1 -- 172.21.15.40:0/4246258734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9094009840 con 0x7f909c1405a0 2023-10-18T09:03:02.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.019+0000 7f909bfff640 1 --2- 172.21.15.40:0/4246258734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f909c1405a0 0x7f909c13ebd0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f9094036040 tx=0x7f90940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:02.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.019+0000 7f9098ff9640 1 -- 172.21.15.40:0/4246258734 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094003e60 con 0x7f909c1405a0 2023-10-18T09:03:02.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.019+0000 7f9098ff9640 1 -- 172.21.15.40:0/4246258734 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f90940046f0 con 0x7f909c1405a0 2023-10-18T09:03:02.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.019+0000 7f90a0b0a640 1 -- 172.21.15.40:0/4246258734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f909c13bb80 con 0x7f909c1405a0 2023-10-18T09:03:02.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.019+0000 7f9098ff9640 1 -- 172.21.15.40:0/4246258734 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094034800 con 0x7f909c1405a0 2023-10-18T09:03:02.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.019+0000 7f90a0b0a640 1 -- 172.21.15.40:0/4246258734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909c13c140 con 0x7f909c1405a0 2023-10-18T09:03:02.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.020+0000 7f90a0b0a640 1 -- 172.21.15.40:0/4246258734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f909c066e00 con 0x7f909c1405a0 2023-10-18T09:03:02.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.021+0000 7f9098ff9640 1 -- 172.21.15.40:0/4246258734 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f909402cd50 con 0x7f909c1405a0 2023-10-18T09:03:02.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.022+0000 7f9098ff9640 1 --2- 172.21.15.40:0/4246258734 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f907c06ae40 0x7f907c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:02.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.022+0000 7f9098ff9640 1 -- 172.21.15.40:0/4246258734 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90940ac490 con 0x7f909c1405a0 2023-10-18T09:03:02.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.023+0000 7f909b7fe640 1 --2- 172.21.15.40:0/4246258734 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f907c06ae40 0x7f907c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:02.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.024+0000 7f909b7fe640 1 --2- 172.21.15.40:0/4246258734 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f907c06ae40 0x7f907c06d300 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f909c0c90b0 tx=0x7f908c005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:02.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.026+0000 7f9098ff9640 1 -- 172.21.15.40:0/4246258734 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f90940774a0 con 0x7f909c1405a0 2023-10-18T09:03:02.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.223+0000 7f90a0b0a640 1 -- 172.21.15.40:0/4246258734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f909c0aca00 con 0x7f909c1405a0 2023-10-18T09:03:02.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.224+0000 7f9098ff9640 1 -- 172.21.15.40:0/4246258734 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9094043020 con 0x7f909c1405a0 2023-10-18T09:03:02.226 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:03:02.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.227+0000 7f90a0b0a640 1 -- 172.21.15.40:0/4246258734 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f907c06ae40 msgr2=0x7f907c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:02.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.227+0000 7f90a0b0a640 1 --2- 172.21.15.40:0/4246258734 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f907c06ae40 0x7f907c06d300 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f909c0c90b0 tx=0x7f908c005f90 comp rx=0 tx=0).stop 2023-10-18T09:03:02.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.227+0000 7f90a0b0a640 1 -- 172.21.15.40:0/4246258734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f909c1405a0 msgr2=0x7f909c13ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:02.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.227+0000 7f90a0b0a640 1 --2- 172.21.15.40:0/4246258734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f909c1405a0 0x7f909c13ebd0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f9094036040 tx=0x7f90940042f0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.227+0000 7f90a0b0a640 1 -- 172.21.15.40:0/4246258734 shutdown_connections 2023-10-18T09:03:02.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.227+0000 7f90a0b0a640 1 --2- 172.21.15.40:0/4246258734 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f907c06ae40 0x7f907c06d300 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.227+0000 7f90a0b0a640 1 --2- 172.21.15.40:0/4246258734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f909c1405a0 0x7f909c13ebd0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.227+0000 7f90a0b0a640 1 --2- 172.21.15.40:0/4246258734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f909c0b1900 0x7f909c1f3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.227+0000 7f90a0b0a640 1 --2- 172.21.15.40:0/4246258734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f909c06d600 0x7f909c13aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.227+0000 7f90a0b0a640 1 -- 172.21.15.40:0/4246258734 >> 172.21.15.40:0/4246258734 conn(0x7f909c069720 msgr2=0x7f909c0abbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:02.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.227+0000 7f90a0b0a640 1 -- 172.21.15.40:0/4246258734 shutdown_connections 2023-10-18T09:03:02.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.227+0000 7f90a0b0a640 1 -- 172.21.15.40:0/4246258734 wait complete. 2023-10-18T09:03:02.249 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:03:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.464+0000 7f8d4910c640 1 -- 172.21.15.40:0/1312573098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d44070d60 msgr2=0x7f8d440cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.464+0000 7f8d4910c640 1 --2- 172.21.15.40:0/1312573098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d44070d60 0x7f8d440cb420 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c005fd0 tx=0x7f8d3c02c1b0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.464+0000 7f8d4910c640 1 -- 172.21.15.40:0/1312573098 shutdown_connections 2023-10-18T09:03:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.464+0000 7f8d4910c640 1 --2- 172.21.15.40:0/1312573098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d44070d60 0x7f8d440cb420 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.464+0000 7f8d4910c640 1 --2- 172.21.15.40:0/1312573098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d4415bbf0 0x7f8d4415bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.464+0000 7f8d4910c640 1 -- 172.21.15.40:0/1312573098 >> 172.21.15.40:0/1312573098 conn(0x7f8d44069d50 msgr2=0x7f8d440aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.464+0000 7f8d4910c640 1 -- 172.21.15.40:0/1312573098 shutdown_connections 2023-10-18T09:03:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.465+0000 7f8d4910c640 1 -- 172.21.15.40:0/1312573098 wait complete. 2023-10-18T09:03:02.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.466+0000 7f8d4910c640 1 Processor -- start 2023-10-18T09:03:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.466+0000 7f8d4910c640 1 -- start start 2023-10-18T09:03:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.467+0000 7f8d4910c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d44070d60 0x7f8d441584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.467+0000 7f8d4910c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d440afd60 0x7f8d44153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.467+0000 7f8d4910c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d4415bbf0 0x7f8d440c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.467+0000 7f8d4910c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d440c90a0 con 0x7f8d440afd60 2023-10-18T09:03:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.467+0000 7f8d4910c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d440a9ee0 con 0x7f8d4415bbf0 2023-10-18T09:03:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.467+0000 7f8d4910c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d440aa1c0 con 0x7f8d44070d60 2023-10-18T09:03:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.468+0000 7f8d3bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d440afd60 0x7f8d44153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.468+0000 7f8d3bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d440afd60 0x7f8d44153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45780/0 (socket says 172.21.15.40:45780) 2023-10-18T09:03:02.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.468+0000 7f8d3bfff640 1 -- 172.21.15.40:0/231095619 learned_addr learned my addr 172.21.15.40:0/231095619 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:02.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.468+0000 7f8d4890b640 1 --2- 172.21.15.40:0/231095619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d4415bbf0 0x7f8d440c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:02.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.468+0000 7f8d43fff640 1 --2- 172.21.15.40:0/231095619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d44070d60 0x7f8d441584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:02.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.468+0000 7f8d3bfff640 1 -- 172.21.15.40:0/231095619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d44070d60 msgr2=0x7f8d441584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:02.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.468+0000 7f8d3bfff640 1 --2- 172.21.15.40:0/231095619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d44070d60 0x7f8d441584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.468+0000 7f8d3bfff640 1 -- 172.21.15.40:0/231095619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d4415bbf0 msgr2=0x7f8d440c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:02.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.468+0000 7f8d3bfff640 1 --2- 172.21.15.40:0/231095619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d4415bbf0 0x7f8d440c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.468+0000 7f8d3bfff640 1 -- 172.21.15.40:0/231095619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c009900 con 0x7f8d440afd60 2023-10-18T09:03:02.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.468+0000 7f8d4890b640 1 --2- 172.21.15.40:0/231095619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d4415bbf0 0x7f8d440c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:03:02.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.469+0000 7f8d3bfff640 1 --2- 172.21.15.40:0/231095619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d440afd60 0x7f8d44153860 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00c450 tx=0x7f8d2c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:02.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.469+0000 7f8d417fa640 1 -- 172.21.15.40:0/231095619 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c0052d0 con 0x7f8d440afd60 2023-10-18T09:03:02.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.469+0000 7f8d417fa640 1 -- 172.21.15.40:0/231095619 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d2c005430 con 0x7f8d440afd60 2023-10-18T09:03:02.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.469+0000 7f8d4910c640 1 -- 172.21.15.40:0/231095619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d440aa4a0 con 0x7f8d440afd60 2023-10-18T09:03:02.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.469+0000 7f8d417fa640 1 -- 172.21.15.40:0/231095619 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c012890 con 0x7f8d440afd60 2023-10-18T09:03:02.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.469+0000 7f8d4910c640 1 -- 172.21.15.40:0/231095619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d44158a20 con 0x7f8d440afd60 2023-10-18T09:03:02.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.470+0000 7f8d4910c640 1 -- 172.21.15.40:0/231095619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d440aa9c0 con 0x7f8d440afd60 2023-10-18T09:03:02.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.471+0000 7f8d417fa640 1 -- 172.21.15.40:0/231095619 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8d2c0035f0 con 0x7f8d440afd60 2023-10-18T09:03:02.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.472+0000 7f8d417fa640 1 --2- 172.21.15.40:0/231095619 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d2406ae40 0x7f8d2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:02.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.472+0000 7f8d417fa640 1 -- 172.21.15.40:0/231095619 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d2c0893f0 con 0x7f8d440afd60 2023-10-18T09:03:02.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.473+0000 7f8d43fff640 1 --2- 172.21.15.40:0/231095619 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d2406ae40 0x7f8d2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:02.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.474+0000 7f8d43fff640 1 --2- 172.21.15.40:0/231095619 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d2406ae40 0x7f8d2406d300 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8d34005fd0 tx=0x7f8d34005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:02.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.476+0000 7f8d417fa640 1 -- 172.21.15.40:0/231095619 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8d440aa9c0 con 0x7f8d440afd60 2023-10-18T09:03:02.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.669+0000 7f8d4910c640 1 -- 172.21.15.40:0/231095619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d440aa9c0 con 0x7f8d440afd60 2023-10-18T09:03:02.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.670+0000 7f8d417fa640 1 -- 172.21.15.40:0/231095619 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v133) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d440aa9c0 con 0x7f8d440afd60 2023-10-18T09:03:02.672 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:03:02.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.672+0000 7f8d4910c640 1 -- 172.21.15.40:0/231095619 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d2406ae40 msgr2=0x7f8d2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:02.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.672+0000 7f8d4910c640 1 --2- 172.21.15.40:0/231095619 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d2406ae40 0x7f8d2406d300 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8d34005fd0 tx=0x7f8d34005850 comp rx=0 tx=0).stop 2023-10-18T09:03:02.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.673+0000 7f8d4910c640 1 -- 172.21.15.40:0/231095619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d440afd60 msgr2=0x7f8d44153860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.673+0000 7f8d4910c640 1 --2- 172.21.15.40:0/231095619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d440afd60 0x7f8d44153860 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00c450 tx=0x7f8d2c00c910 comp rx=0 tx=0).stop 2023-10-18T09:03:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.673+0000 7f8d4910c640 1 -- 172.21.15.40:0/231095619 shutdown_connections 2023-10-18T09:03:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.673+0000 7f8d4910c640 1 --2- 172.21.15.40:0/231095619 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d2406ae40 0x7f8d2406d300 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.673+0000 7f8d4910c640 1 --2- 172.21.15.40:0/231095619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d4415bbf0 0x7f8d440c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.673+0000 7f8d4910c640 1 --2- 172.21.15.40:0/231095619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d440afd60 0x7f8d44153860 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.673+0000 7f8d4910c640 1 --2- 172.21.15.40:0/231095619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d44070d60 0x7f8d441584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.673+0000 7f8d4910c640 1 -- 172.21.15.40:0/231095619 >> 172.21.15.40:0/231095619 conn(0x7f8d44069d50 msgr2=0x7f8d441e6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:02.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.673+0000 7f8d4910c640 1 -- 172.21.15.40:0/231095619 shutdown_connections 2023-10-18T09:03:02.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.673+0000 7f8d4910c640 1 -- 172.21.15.40:0/231095619 wait complete. 2023-10-18T09:03:02.694 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:03:02.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.926+0000 7f91e3fff640 1 -- 172.21.15.40:0/4277299760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91e415b760 msgr2=0x7f91e40b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:02.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.926+0000 7f91e3fff640 1 --2- 172.21.15.40:0/4277299760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91e415b760 0x7f91e40b1120 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f91d8005fd0 tx=0x7f91d802be20 comp rx=0 tx=0).stop 2023-10-18T09:03:02.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.927+0000 7f91e3fff640 1 -- 172.21.15.40:0/4277299760 shutdown_connections 2023-10-18T09:03:02.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.927+0000 7f91e3fff640 1 --2- 172.21.15.40:0/4277299760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91e415b760 0x7f91e40b1120 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.927+0000 7f91e3fff640 1 -- 172.21.15.40:0/4277299760 >> 172.21.15.40:0/4277299760 conn(0x7f91e408acb0 msgr2=0x7f91e409a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:02.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.927+0000 7f91e3fff640 1 -- 172.21.15.40:0/4277299760 shutdown_connections 2023-10-18T09:03:02.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.927+0000 7f91e3fff640 1 -- 172.21.15.40:0/4277299760 wait complete. 2023-10-18T09:03:02.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.929+0000 7f91e3fff640 1 Processor -- start 2023-10-18T09:03:02.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.929+0000 7f91e3fff640 1 -- start start 2023-10-18T09:03:02.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.930+0000 7f91e3fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91e40d1580 0x7f91e41e4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:02.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.930+0000 7f91e3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91e415b380 0x7f91e40c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:02.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.930+0000 7f91e3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91e415b760 0x7f91e40ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:02.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.930+0000 7f91e3fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e41e6200 con 0x7f91e415b760 2023-10-18T09:03:02.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.930+0000 7f91e3fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e40c6f30 con 0x7f91e415b380 2023-10-18T09:03:02.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.930+0000 7f91e3fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e41e5160 con 0x7f91e40d1580 2023-10-18T09:03:02.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.930+0000 7f91e37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91e415b760 0x7f91e40ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:02.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.930+0000 7f91e37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91e415b760 0x7f91e40ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45796/0 (socket says 172.21.15.40:45796) 2023-10-18T09:03:02.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.930+0000 7f91e37fe640 1 -- 172.21.15.40:0/3709618805 learned_addr learned my addr 172.21.15.40:0/3709618805 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:02.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.930+0000 7f91e27fc640 1 --2- 172.21.15.40:0/3709618805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91e415b380 0x7f91e40c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.930+0000 7f91e2ffd640 1 --2- 172.21.15.40:0/3709618805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91e40d1580 0x7f91e41e4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.930+0000 7f91e37fe640 1 -- 172.21.15.40:0/3709618805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91e40d1580 msgr2=0x7f91e41e4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.930+0000 7f91e37fe640 1 --2- 172.21.15.40:0/3709618805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91e40d1580 0x7f91e41e4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.931+0000 7f91e37fe640 1 -- 172.21.15.40:0/3709618805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91e415b380 msgr2=0x7f91e40c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:02.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.931+0000 7f91e37fe640 1 --2- 172.21.15.40:0/3709618805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91e415b380 0x7f91e40c7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.931+0000 7f91e37fe640 1 -- 172.21.15.40:0/3709618805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d8009560 con 0x7f91e415b760 2023-10-18T09:03:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.931+0000 7f91e37fe640 1 --2- 172.21.15.40:0/3709618805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91e415b760 0x7f91e40ca2b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f91d400c430 tx=0x7f91d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.931+0000 7f91cbfff640 1 -- 172.21.15.40:0/3709618805 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d40035f0 con 0x7f91e415b760 2023-10-18T09:03:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.931+0000 7f91cbfff640 1 -- 172.21.15.40:0/3709618805 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f91d4003770 con 0x7f91e415b760 2023-10-18T09:03:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.931+0000 7f91e3fff640 1 -- 172.21.15.40:0/3709618805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e41e5440 con 0x7f91e415b760 2023-10-18T09:03:02.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.931+0000 7f91cbfff640 1 -- 172.21.15.40:0/3709618805 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4003b80 con 0x7f91e415b760 2023-10-18T09:03:02.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.932+0000 7f91e3fff640 1 -- 172.21.15.40:0/3709618805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e41e5a00 con 0x7f91e415b760 2023-10-18T09:03:02.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.933+0000 7f91e3fff640 1 -- 172.21.15.40:0/3709618805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e4066e00 con 0x7f91e415b760 2023-10-18T09:03:02.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.934+0000 7f91cbfff640 1 -- 172.21.15.40:0/3709618805 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f91d4003ce0 con 0x7f91e415b760 2023-10-18T09:03:02.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.935+0000 7f91cbfff640 1 --2- 172.21.15.40:0/3709618805 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f91d006ae90 0x7f91d006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:02.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.935+0000 7f91cbfff640 1 -- 172.21.15.40:0/3709618805 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91d4088e60 con 0x7f91e415b760 2023-10-18T09:03:02.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.935+0000 7f91e2ffd640 1 --2- 172.21.15.40:0/3709618805 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f91d006ae90 0x7f91d006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:02.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.936+0000 7f91e2ffd640 1 --2- 172.21.15.40:0/3709618805 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f91d006ae90 0x7f91d006d350 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f91dc005520 tx=0x7f91dc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:02.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:02.939+0000 7f91cbfff640 1 -- 172.21.15.40:0/3709618805 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f91d4053ef0 con 0x7f91e415b760 2023-10-18T09:03:03.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.134+0000 7f91e3fff640 1 -- 172.21.15.40:0/3709618805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91e40ca090 con 0x7f91e415b760 2023-10-18T09:03:03.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.136+0000 7f91cbfff640 1 -- 172.21.15.40:0/3709618805 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v133) v1 ==== 100+0+11 (secure 0 0 0) 0x7f91d4017020 con 0x7f91e415b760 2023-10-18T09:03:03.138 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:03:03.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.138+0000 7f91e3fff640 1 -- 172.21.15.40:0/3709618805 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f91d006ae90 msgr2=0x7f91d006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:03.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.138+0000 7f91e3fff640 1 --2- 172.21.15.40:0/3709618805 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f91d006ae90 0x7f91d006d350 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f91dc005520 tx=0x7f91dc009040 comp rx=0 tx=0).stop 2023-10-18T09:03:03.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.138+0000 7f91e3fff640 1 -- 172.21.15.40:0/3709618805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91e415b760 msgr2=0x7f91e40ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:03.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.138+0000 7f91e3fff640 1 --2- 172.21.15.40:0/3709618805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91e415b760 0x7f91e40ca2b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f91d400c430 tx=0x7f91d400c8f0 comp rx=0 tx=0).stop 2023-10-18T09:03:03.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.138+0000 7f91e3fff640 1 -- 172.21.15.40:0/3709618805 shutdown_connections 2023-10-18T09:03:03.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.138+0000 7f91e3fff640 1 --2- 172.21.15.40:0/3709618805 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f91d006ae90 0x7f91d006d350 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:03.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.138+0000 7f91e3fff640 1 --2- 172.21.15.40:0/3709618805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91e415b760 0x7f91e40ca2b0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:03.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.138+0000 7f91e3fff640 1 --2- 172.21.15.40:0/3709618805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91e415b380 0x7f91e40c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:03.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.138+0000 7f91e3fff640 1 --2- 172.21.15.40:0/3709618805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91e40d1580 0x7f91e41e4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:03.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.138+0000 7f91e3fff640 1 -- 172.21.15.40:0/3709618805 >> 172.21.15.40:0/3709618805 conn(0x7f91e408acb0 msgr2=0x7f91e415db80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:03.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.138+0000 7f91e3fff640 1 -- 172.21.15.40:0/3709618805 shutdown_connections 2023-10-18T09:03:03.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.139+0000 7f91e3fff640 1 -- 172.21.15.40:0/3709618805 wait complete. 2023-10-18T09:03:03.161 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:03:03.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.382+0000 7f3522dbf640 1 -- 172.21.15.40:0/1616037489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3508003f40 msgr2=0x7f35080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:03.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.382+0000 7f3522dbf640 1 --2- 172.21.15.40:0/1616037489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3508003f40 0x7f35080047c0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f350c00bb20 tx=0x7f35080057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:03.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.382+0000 7f35225be640 1 -- 172.21.15.40:0/1616037489 reap_dead start 2023-10-18T09:03:03.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.382+0000 7f3522dbf640 1 -- 172.21.15.40:0/1616037489 shutdown_connections 2023-10-18T09:03:03.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.382+0000 7f3522dbf640 1 -- 172.21.15.40:0/1616037489 >> 172.21.15.40:0/1616037489 conn(0x7f351c069720 msgr2=0x7f351c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:03.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.382+0000 7f3522dbf640 1 -- 172.21.15.40:0/1616037489 shutdown_connections 2023-10-18T09:03:03.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.382+0000 7f3522dbf640 1 -- 172.21.15.40:0/1616037489 wait complete. 2023-10-18T09:03:03.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.384+0000 7f3522dbf640 1 Processor -- start 2023-10-18T09:03:03.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.384+0000 7f3522dbf640 1 -- start start 2023-10-18T09:03:03.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.385+0000 7f3522dbf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3508003f40 0x7f351c0ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:03.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.385+0000 7f3522dbf640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f351c158740 0x7f351c1360f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:03.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.385+0000 7f3522dbf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f351c0ae0a0 0x7f351c140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:03.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.385+0000 7f3522dbf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c136630 con 0x7f3508003f40 2023-10-18T09:03:03.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.385+0000 7f3522dbf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c158bc0 con 0x7f351c0ae0a0 2023-10-18T09:03:03.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.385+0000 7f3522dbf640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351c0ad000 con 0x7f351c158740 2023-10-18T09:03:03.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.385+0000 7f3521dbd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3508003f40 0x7f351c0ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:03.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.385+0000 7f3521dbd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3508003f40 0x7f351c0ac970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45822/0 (socket says 172.21.15.40:45822) 2023-10-18T09:03:03.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.385+0000 7f3521dbd640 1 -- 172.21.15.40:0/2097281825 learned_addr learned my addr 172.21.15.40:0/2097281825 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:03.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.385+0000 7f35225be640 1 --2- 172.21.15.40:0/2097281825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f351c0ae0a0 0x7f351c140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:03.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.385+0000 7f3521dbd640 1 -- 172.21.15.40:0/2097281825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f351c158740 msgr2=0x7f351c1360f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:03.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.385+0000 7f35215bc640 1 --2- 172.21.15.40:0/2097281825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f351c158740 0x7f351c1360f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:03.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.386+0000 7f3521dbd640 1 --2- 172.21.15.40:0/2097281825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f351c158740 0x7f351c1360f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:03.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.386+0000 7f3521dbd640 1 -- 172.21.15.40:0/2097281825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f351c0ae0a0 msgr2=0x7f351c140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:03.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.386+0000 7f3521dbd640 1 --2- 172.21.15.40:0/2097281825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f351c0ae0a0 0x7f351c140e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:03.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.386+0000 7f3521dbd640 1 -- 172.21.15.40:0/2097281825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c0ad2e0 con 0x7f3508003f40 2023-10-18T09:03:03.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.386+0000 7f35225be640 1 --2- 172.21.15.40:0/2097281825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f351c0ae0a0 0x7f351c140e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:03.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.386+0000 7f3521dbd640 1 --2- 172.21.15.40:0/2097281825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3508003f40 0x7f351c0ac970 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f351800e5c0 tx=0x7f3518001820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:03.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.386+0000 7f35127fc640 1 -- 172.21.15.40:0/2097281825 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351800c8f0 con 0x7f3508003f40 2023-10-18T09:03:03.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.386+0000 7f35127fc640 1 -- 172.21.15.40:0/2097281825 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f351800cb60 con 0x7f3508003f40 2023-10-18T09:03:03.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.386+0000 7f35127fc640 1 -- 172.21.15.40:0/2097281825 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518011460 con 0x7f3508003f40 2023-10-18T09:03:03.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.386+0000 7f3522dbf640 1 -- 172.21.15.40:0/2097281825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351c0ad5c0 con 0x7f3508003f40 2023-10-18T09:03:03.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.386+0000 7f3522dbf640 1 -- 172.21.15.40:0/2097281825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351c0adb80 con 0x7f3508003f40 2023-10-18T09:03:03.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.388+0000 7f3522dbf640 1 -- 172.21.15.40:0/2097281825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f351c066e00 con 0x7f3508003f40 2023-10-18T09:03:03.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.388+0000 7f35127fc640 1 -- 172.21.15.40:0/2097281825 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f35180115c0 con 0x7f3508003f40 2023-10-18T09:03:03.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.389+0000 7f35127fc640 1 --2- 172.21.15.40:0/2097281825 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f350006ae70 0x7f350006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:03.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.390+0000 7f35127fc640 1 -- 172.21.15.40:0/2097281825 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3518089ed0 con 0x7f3508003f40 2023-10-18T09:03:03.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.390+0000 7f35215bc640 1 --2- 172.21.15.40:0/2097281825 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f350006ae70 0x7f350006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:03.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.391+0000 7f35215bc640 1 --2- 172.21.15.40:0/2097281825 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f350006ae70 0x7f350006d330 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f350c037000 tx=0x7f350c02e040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:03.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.394+0000 7f35127fc640 1 -- 172.21.15.40:0/2097281825 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3518054ee0 con 0x7f3508003f40 2023-10-18T09:03:03.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.603+0000 7f3522dbf640 1 -- 172.21.15.40:0/2097281825 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f351c06d920 con 0x7f3508003f40 2023-10-18T09:03:03.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.604+0000 7f35127fc640 1 -- 172.21.15.40:0/2097281825 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v133) v1 ==== 80+0+0 (secure 0 0 0) 0x7f351801f020 con 0x7f3508003f40 2023-10-18T09:03:03.606 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T09:03:03.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.606+0000 7f3522dbf640 1 -- 172.21.15.40:0/2097281825 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f350006ae70 msgr2=0x7f350006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:03.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.606+0000 7f3522dbf640 1 --2- 172.21.15.40:0/2097281825 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f350006ae70 0x7f350006d330 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f350c037000 tx=0x7f350c02e040 comp rx=0 tx=0).stop 2023-10-18T09:03:03.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.606+0000 7f3522dbf640 1 -- 172.21.15.40:0/2097281825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3508003f40 msgr2=0x7f351c0ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:03.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.606+0000 7f3522dbf640 1 --2- 172.21.15.40:0/2097281825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3508003f40 0x7f351c0ac970 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f351800e5c0 tx=0x7f3518001820 comp rx=0 tx=0).stop 2023-10-18T09:03:03.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.606+0000 7f3522dbf640 1 -- 172.21.15.40:0/2097281825 shutdown_connections 2023-10-18T09:03:03.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.606+0000 7f3522dbf640 1 --2- 172.21.15.40:0/2097281825 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f350006ae70 0x7f350006d330 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:03.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.606+0000 7f3522dbf640 1 --2- 172.21.15.40:0/2097281825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f351c0ae0a0 0x7f351c140e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:03.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.606+0000 7f3522dbf640 1 --2- 172.21.15.40:0/2097281825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f351c158740 0x7f351c1360f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:03.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.606+0000 7f3522dbf640 1 --2- 172.21.15.40:0/2097281825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3508003f40 0x7f351c0ac970 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:03.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.606+0000 7f3522dbf640 1 -- 172.21.15.40:0/2097281825 >> 172.21.15.40:0/2097281825 conn(0x7f351c069720 msgr2=0x7f351c159950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:03.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.607+0000 7f3522dbf640 1 -- 172.21.15.40:0/2097281825 shutdown_connections 2023-10-18T09:03:03.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:03.607+0000 7f3522dbf640 1 -- 172.21.15.40:0/2097281825 wait complete. 2023-10-18T09:03:03.628 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-18T09:03:03.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:03.629 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-18T09:03:03.744 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T09:03:03.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:03.745 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T09:03:03.799 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:03:03.800 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:03:03.864 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:03:03.865 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:03:03.865 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:03:03.925 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:03:03.926 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:03:04.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.165+0000 7f38d8ca2640 1 -- 172.21.15.40:0/2000929407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38d41586a0 msgr2=0x7f38d406b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:04.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.165+0000 7f38d8ca2640 1 --2- 172.21.15.40:0/2000929407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38d41586a0 0x7f38d406b710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f38c8005fd0 tx=0x7f38c802be10 comp rx=0 tx=0).stop 2023-10-18T09:03:04.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.165+0000 7f38d8ca2640 1 -- 172.21.15.40:0/2000929407 shutdown_connections 2023-10-18T09:03:04.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.165+0000 7f38d8ca2640 1 --2- 172.21.15.40:0/2000929407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38d41586a0 0x7f38d406b710 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.165+0000 7f38d8ca2640 1 --2- 172.21.15.40:0/2000929407 >> v2:172.21.15.144:3300/0 conn(0x7f38d406d710 0x7f38d4151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.165+0000 7f38d8ca2640 1 -- 172.21.15.40:0/2000929407 >> 172.21.15.40:0/2000929407 conn(0x7f38d406cee0 msgr2=0x7f38d4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:04.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.165+0000 7f38d8ca2640 1 -- 172.21.15.40:0/2000929407 shutdown_connections 2023-10-18T09:03:04.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.165+0000 7f38d8ca2640 1 -- 172.21.15.40:0/2000929407 wait complete. 2023-10-18T09:03:04.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.167+0000 7f38d8ca2640 1 Processor -- start 2023-10-18T09:03:04.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.167+0000 7f38d8ca2640 1 -- start start 2023-10-18T09:03:04.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.168+0000 7f38d8ca2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38d406bc50 0x7f38d414b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:04.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.168+0000 7f38d8ca2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38d406d710 0x7f38d41efe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:04.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.168+0000 7f38d8ca2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38d41586a0 0x7f38d41f0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:04.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.168+0000 7f38d8ca2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d4156e50 con 0x7f38d406bc50 2023-10-18T09:03:04.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.168+0000 7f38d8ca2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d414a170 con 0x7f38d41586a0 2023-10-18T09:03:04.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.168+0000 7f38d8ca2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d414a450 con 0x7f38d406d710 2023-10-18T09:03:04.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.168+0000 7f38d259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38d406bc50 0x7f38d414b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:04.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.168+0000 7f38d259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38d406bc50 0x7f38d414b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45826/0 (socket says 172.21.15.40:45826) 2023-10-18T09:03:04.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.168+0000 7f38d259c640 1 -- 172.21.15.40:0/3719699039 learned_addr learned my addr 172.21.15.40:0/3719699039 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:04.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.169+0000 7f38d1d9b640 1 --2- 172.21.15.40:0/3719699039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38d406d710 0x7f38d41efe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:04.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.169+0000 7f38d259c640 1 -- 172.21.15.40:0/3719699039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38d406d710 msgr2=0x7f38d41efe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:04.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.169+0000 7f38d259c640 1 --2- 172.21.15.40:0/3719699039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38d406d710 0x7f38d41efe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.169+0000 7f38d2d9d640 1 --2- 172.21.15.40:0/3719699039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38d41586a0 0x7f38d41f0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:04.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.169+0000 7f38d259c640 1 -- 172.21.15.40:0/3719699039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38d41586a0 msgr2=0x7f38d41f0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:04.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.169+0000 7f38d259c640 1 --2- 172.21.15.40:0/3719699039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38d41586a0 0x7f38d41f0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.169+0000 7f38d259c640 1 -- 172.21.15.40:0/3719699039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c8009560 con 0x7f38d406bc50 2023-10-18T09:03:04.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.169+0000 7f38d2d9d640 1 --2- 172.21.15.40:0/3719699039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38d41586a0 0x7f38d41f0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:04.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.169+0000 7f38d259c640 1 --2- 172.21.15.40:0/3719699039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38d406bc50 0x7f38d414b160 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f38c400ebc0 tx=0x7f38c400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:04.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.170+0000 7f38c2ffd640 1 -- 172.21.15.40:0/3719699039 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c40099b0 con 0x7f38d406bc50 2023-10-18T09:03:04.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.170+0000 7f38c2ffd640 1 -- 172.21.15.40:0/3719699039 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f38c4010720 con 0x7f38d406bc50 2023-10-18T09:03:04.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.170+0000 7f38c2ffd640 1 -- 172.21.15.40:0/3719699039 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4010b30 con 0x7f38d406bc50 2023-10-18T09:03:04.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.170+0000 7f38d8ca2640 1 -- 172.21.15.40:0/3719699039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d415c1d0 con 0x7f38d406bc50 2023-10-18T09:03:04.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.170+0000 7f38d8ca2640 1 -- 172.21.15.40:0/3719699039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d414aa10 con 0x7f38d406bc50 2023-10-18T09:03:04.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.171+0000 7f38d8ca2640 1 -- 172.21.15.40:0/3719699039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d40b2170 con 0x7f38d406bc50 2023-10-18T09:03:04.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.172+0000 7f38c2ffd640 1 -- 172.21.15.40:0/3719699039 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f38c4010c90 con 0x7f38d406bc50 2023-10-18T09:03:04.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.173+0000 7f38c2ffd640 1 --2- 172.21.15.40:0/3719699039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f38b406ae40 0x7f38b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:04.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.173+0000 7f38c2ffd640 1 -- 172.21.15.40:0/3719699039 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38c4014070 con 0x7f38d406bc50 2023-10-18T09:03:04.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.174+0000 7f38d1d9b640 1 --2- 172.21.15.40:0/3719699039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f38b406ae40 0x7f38b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:04.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.175+0000 7f38d1d9b640 1 --2- 172.21.15.40:0/3719699039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f38b406ae40 0x7f38b406d300 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f38c8035000 tx=0x7f38c8004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:04.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.177+0000 7f38c2ffd640 1 -- 172.21.15.40:0/3719699039 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38c4051ca0 con 0x7f38d406bc50 2023-10-18T09:03:04.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.407+0000 7f38d8ca2640 1 -- 172.21.15.40:0/3719699039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38d4066fd0 con 0x7f38d406bc50 2023-10-18T09:03:04.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.408+0000 7f38c2ffd640 1 -- 172.21.15.40:0/3719699039 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f38c4088e60 con 0x7f38d406bc50 2023-10-18T09:03:04.410 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:04.411 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/700924196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":700924196},{"type":"v1","addr":"172.21.15.40:6833","nonce":700924196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/877414874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":877414874},{"type":"v1","addr":"172.21.15.144:6835","nonce":877414874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2596943574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2596943574},{"type":"v1","addr":"172.21.15.144:6837","nonce":2596943574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:02:41.786627+0000","modified":"2023-10-18T09:02:44.862740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1801003551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1801003551},{"type":"v1","addr":"172.21.15.40:6835","nonce":1801003551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-18T09:03:04.411 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2023-10-18T09:03:04.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.410+0000 7f38d8ca2640 1 -- 172.21.15.40:0/3719699039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f38b406ae40 msgr2=0x7f38b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:04.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.410+0000 7f38d8ca2640 1 --2- 172.21.15.40:0/3719699039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f38b406ae40 0x7f38b406d300 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f38c8035000 tx=0x7f38c8004730 comp rx=0 tx=0).stop 2023-10-18T09:03:04.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.410+0000 7f38d8ca2640 1 -- 172.21.15.40:0/3719699039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38d406bc50 msgr2=0x7f38d414b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:04.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.410+0000 7f38d8ca2640 1 --2- 172.21.15.40:0/3719699039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38d406bc50 0x7f38d414b160 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f38c400ebc0 tx=0x7f38c400c6a0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.410+0000 7f38d8ca2640 1 -- 172.21.15.40:0/3719699039 shutdown_connections 2023-10-18T09:03:04.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.411+0000 7f38d8ca2640 1 --2- 172.21.15.40:0/3719699039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f38b406ae40 0x7f38b406d300 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.411+0000 7f38d8ca2640 1 --2- 172.21.15.40:0/3719699039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38d41586a0 0x7f38d41f0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.411+0000 7f38d8ca2640 1 --2- 172.21.15.40:0/3719699039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38d406d710 0x7f38d41efe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.411+0000 7f38d8ca2640 1 --2- 172.21.15.40:0/3719699039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38d406bc50 0x7f38d414b160 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.411+0000 7f38d8ca2640 1 -- 172.21.15.40:0/3719699039 >> 172.21.15.40:0/3719699039 conn(0x7f38d406cee0 msgr2=0x7f38d4148aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:04.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.411+0000 7f38d8ca2640 1 -- 172.21.15.40:0/3719699039 shutdown_connections 2023-10-18T09:03:04.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.411+0000 7f38d8ca2640 1 -- 172.21.15.40:0/3719699039 wait complete. 2023-10-18T09:03:04.433 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:03:04.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.668+0000 7f4d9dfd7640 1 -- 172.21.15.40:0/912824133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d88003970 msgr2=0x7f4d88003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:04.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.668+0000 7f4d9dfd7640 1 --2- 172.21.15.40:0/912824133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d88003970 0x7f4d88003d90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c006fc0 tx=0x7f4d880057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.668+0000 7f4d9dfd7640 1 -- 172.21.15.40:0/912824133 shutdown_connections 2023-10-18T09:03:04.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.668+0000 7f4d9dfd7640 1 --2- 172.21.15.40:0/912824133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d88003970 0x7f4d88003d90 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.668+0000 7f4d9dfd7640 1 -- 172.21.15.40:0/912824133 >> 172.21.15.40:0/912824133 conn(0x7f4d98069720 msgr2=0x7f4d98069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:04.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.668+0000 7f4d9dfd7640 1 -- 172.21.15.40:0/912824133 shutdown_connections 2023-10-18T09:03:04.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.668+0000 7f4d9dfd7640 1 -- 172.21.15.40:0/912824133 wait complete. 2023-10-18T09:03:04.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.670+0000 7f4d9dfd7640 1 Processor -- start 2023-10-18T09:03:04.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.670+0000 7f4d9dfd7640 1 -- start start 2023-10-18T09:03:04.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.671+0000 7f4d9dfd7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d88003970 0x7f4d9814de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:04.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.671+0000 7f4d9dfd7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d98151240 0x7f4d981f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:04.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.671+0000 7f4d9dfd7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d98149840 0x7f4d981529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:04.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.671+0000 7f4d9dfd7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9814bf70 con 0x7f4d88003970 2023-10-18T09:03:04.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.671+0000 7f4d9dfd7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9814b4d0 con 0x7f4d98151240 2023-10-18T09:03:04.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.671+0000 7f4d9dfd7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9814b1f0 con 0x7f4d98149840 2023-10-18T09:03:04.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.672+0000 7f4d9cfd5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d88003970 0x7f4d9814de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:04.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.672+0000 7f4d9cfd5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d88003970 0x7f4d9814de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45844/0 (socket says 172.21.15.40:45844) 2023-10-18T09:03:04.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.672+0000 7f4d9cfd5640 1 -- 172.21.15.40:0/740538802 learned_addr learned my addr 172.21.15.40:0/740538802 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:04.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.672+0000 7f4d97fff640 1 --2- 172.21.15.40:0/740538802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d98151240 0x7f4d981f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:04.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.672+0000 7f4d9d7d6640 1 --2- 172.21.15.40:0/740538802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d98149840 0x7f4d981529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:04.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.672+0000 7f4d9cfd5640 1 -- 172.21.15.40:0/740538802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d98149840 msgr2=0x7f4d981529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:04.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.672+0000 7f4d9cfd5640 1 --2- 172.21.15.40:0/740538802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d98149840 0x7f4d981529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.672+0000 7f4d9cfd5640 1 -- 172.21.15.40:0/740538802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d98151240 msgr2=0x7f4d981f0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:04.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.672+0000 7f4d9cfd5640 1 --2- 172.21.15.40:0/740538802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d98151240 0x7f4d981f0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.672+0000 7f4d9cfd5640 1 -- 172.21.15.40:0/740538802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9814b920 con 0x7f4d88003970 2023-10-18T09:03:04.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.672+0000 7f4d9d7d6640 1 --2- 172.21.15.40:0/740538802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d98149840 0x7f4d981529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:04.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.673+0000 7f4d9cfd5640 1 --2- 172.21.15.40:0/740538802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d88003970 0x7f4d9814de60 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c000f60 tx=0x7f4d8c001580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:04.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.673+0000 7f4d957fa640 1 -- 172.21.15.40:0/740538802 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c006030 con 0x7f4d88003970 2023-10-18T09:03:04.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.673+0000 7f4d957fa640 1 -- 172.21.15.40:0/740538802 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4d8c006190 con 0x7f4d88003970 2023-10-18T09:03:04.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.673+0000 7f4d957fa640 1 -- 172.21.15.40:0/740538802 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c006540 con 0x7f4d88003970 2023-10-18T09:03:04.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.673+0000 7f4d9dfd7640 1 -- 172.21.15.40:0/740538802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9814bc00 con 0x7f4d88003970 2023-10-18T09:03:04.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.673+0000 7f4d9dfd7640 1 -- 172.21.15.40:0/740538802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d98149cf0 con 0x7f4d88003970 2023-10-18T09:03:04.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.674+0000 7f4d9dfd7640 1 -- 172.21.15.40:0/740538802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d98066e00 con 0x7f4d88003970 2023-10-18T09:03:04.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.675+0000 7f4d957fa640 1 -- 172.21.15.40:0/740538802 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4d8c031030 con 0x7f4d88003970 2023-10-18T09:03:04.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.676+0000 7f4d957fa640 1 --2- 172.21.15.40:0/740538802 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4d7c06ae40 0x7f4d7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:04.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.676+0000 7f4d957fa640 1 -- 172.21.15.40:0/740538802 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d8c0aa760 con 0x7f4d88003970 2023-10-18T09:03:04.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.677+0000 7f4d97fff640 1 --2- 172.21.15.40:0/740538802 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4d7c06ae40 0x7f4d7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:04.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.678+0000 7f4d97fff640 1 --2- 172.21.15.40:0/740538802 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4d7c06ae40 0x7f4d7c06d300 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4d84005fd0 tx=0x7f4d8400c420 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:04.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.680+0000 7f4d957fa640 1 -- 172.21.15.40:0/740538802 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d8c075800 con 0x7f4d88003970 2023-10-18T09:03:04.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.876+0000 7f4d9dfd7640 1 -- 172.21.15.40:0/740538802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d981f3280 con 0x7f4d88003970 2023-10-18T09:03:04.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.879+0000 7f4d957fa640 1 -- 172.21.15.40:0/740538802 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+16539 (secure 0 0 0) 0x7f4d8c044090 con 0x7f4d88003970 2023-10-18T09:03:04.882 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:04.882 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":133,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:02:56.085956+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:02:39.531482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-18T09:02:40.812071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:03:04.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.882+0000 7f4d9dfd7640 1 -- 172.21.15.40:0/740538802 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4d7c06ae40 msgr2=0x7f4d7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:04.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.882+0000 7f4d9dfd7640 1 --2- 172.21.15.40:0/740538802 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4d7c06ae40 0x7f4d7c06d300 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4d84005fd0 tx=0x7f4d8400c420 comp rx=0 tx=0).stop 2023-10-18T09:03:04.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.882+0000 7f4d9dfd7640 1 -- 172.21.15.40:0/740538802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d88003970 msgr2=0x7f4d9814de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:04.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.882+0000 7f4d9dfd7640 1 --2- 172.21.15.40:0/740538802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d88003970 0x7f4d9814de60 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c000f60 tx=0x7f4d8c001580 comp rx=0 tx=0).stop 2023-10-18T09:03:04.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.882+0000 7f4d9dfd7640 1 -- 172.21.15.40:0/740538802 shutdown_connections 2023-10-18T09:03:04.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.882+0000 7f4d9dfd7640 1 --2- 172.21.15.40:0/740538802 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4d7c06ae40 0x7f4d7c06d300 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.882+0000 7f4d9dfd7640 1 --2- 172.21.15.40:0/740538802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d98149840 0x7f4d981529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.882+0000 7f4d9dfd7640 1 --2- 172.21.15.40:0/740538802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d98151240 0x7f4d981f0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.882+0000 7f4d9dfd7640 1 --2- 172.21.15.40:0/740538802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d88003970 0x7f4d9814de60 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:04.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.882+0000 7f4d9dfd7640 1 -- 172.21.15.40:0/740538802 >> 172.21.15.40:0/740538802 conn(0x7f4d98069720 msgr2=0x7f4d98069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:04.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.882+0000 7f4d9dfd7640 1 -- 172.21.15.40:0/740538802 shutdown_connections 2023-10-18T09:03:04.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:04.882+0000 7f4d9dfd7640 1 -- 172.21.15.40:0/740538802 wait complete. 2023-10-18T09:03:04.905 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:02:39.531482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:02:40.812071+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:03:04.906 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:03:05.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.132+0000 7fa30732c640 1 -- 172.21.15.40:0/2649563424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa300097a10 msgr2=0x7fa3000ae920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:05.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.132+0000 7fa30732c640 1 --2- 172.21.15.40:0/2649563424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa300097a10 0x7fa3000ae920 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fa3000b5090 tx=0x7fa2fc02be40 comp rx=0 tx=0).stop 2023-10-18T09:03:05.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.132+0000 7fa30732c640 1 -- 172.21.15.40:0/2649563424 shutdown_connections 2023-10-18T09:03:05.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.132+0000 7fa30732c640 1 --2- 172.21.15.40:0/2649563424 >> v2:172.21.15.144:3300/0 conn(0x7fa3000b16f0 0x7fa300097df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:05.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.132+0000 7fa30732c640 1 --2- 172.21.15.40:0/2649563424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa300097a10 0x7fa3000ae920 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:05.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.132+0000 7fa30732c640 1 -- 172.21.15.40:0/2649563424 >> 172.21.15.40:0/2649563424 conn(0x7fa3000b0ad0 msgr2=0x7fa3000b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:05.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.133+0000 7fa30732c640 1 -- 172.21.15.40:0/2649563424 shutdown_connections 2023-10-18T09:03:05.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.133+0000 7fa30732c640 1 -- 172.21.15.40:0/2649563424 wait complete. 2023-10-18T09:03:05.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.134+0000 7fa30732c640 1 Processor -- start 2023-10-18T09:03:05.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.135+0000 7fa30732c640 1 -- start start 2023-10-18T09:03:05.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.135+0000 7fa30732c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa30006aa40 0x7fa3000b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:05.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.136+0000 7fa30732c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa300097a10 0x7fa300158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:05.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.136+0000 7fa30732c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3000b16f0 0x7fa300159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:05.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.136+0000 7fa30732c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa300153020 con 0x7fa3000b16f0 2023-10-18T09:03:05.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.136+0000 7fa30732c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa300158a90 con 0x7fa30006aa40 2023-10-18T09:03:05.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.136+0000 7fa30732c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3001e5750 con 0x7fa300097a10 2023-10-18T09:03:05.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.136+0000 7fa306b2b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3000b16f0 0x7fa300159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:05.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.136+0000 7fa306b2b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3000b16f0 0x7fa300159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60504/0 (socket says 172.21.15.40:60504) 2023-10-18T09:03:05.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.136+0000 7fa306b2b640 1 -- 172.21.15.40:0/2435382824 learned_addr learned my addr 172.21.15.40:0/2435382824 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:05.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.136+0000 7fa305b29640 1 --2- 172.21.15.40:0/2435382824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa300097a10 0x7fa300158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:05.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.136+0000 7fa30632a640 1 --2- 172.21.15.40:0/2435382824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa30006aa40 0x7fa3000b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:05.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.136+0000 7fa306b2b640 1 -- 172.21.15.40:0/2435382824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa300097a10 msgr2=0x7fa300158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:05.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.136+0000 7fa306b2b640 1 --2- 172.21.15.40:0/2435382824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa300097a10 0x7fa300158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:05.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.136+0000 7fa306b2b640 1 -- 172.21.15.40:0/2435382824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa30006aa40 msgr2=0x7fa3000b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:05.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.136+0000 7fa306b2b640 1 --2- 172.21.15.40:0/2435382824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa30006aa40 0x7fa3000b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:05.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.136+0000 7fa306b2b640 1 -- 172.21.15.40:0/2435382824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc009560 con 0x7fa3000b16f0 2023-10-18T09:03:05.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.136+0000 7fa30632a640 1 --2- 172.21.15.40:0/2435382824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa30006aa40 0x7fa3000b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:05.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.137+0000 7fa306b2b640 1 --2- 172.21.15.40:0/2435382824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3000b16f0 0x7fa300159e40 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800c7a0 tx=0x7fa2f800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:05.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.137+0000 7fa2f6ffd640 1 -- 172.21.15.40:0/2435382824 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8003b70 con 0x7fa3000b16f0 2023-10-18T09:03:05.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.137+0000 7fa2f6ffd640 1 -- 172.21.15.40:0/2435382824 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa2f8003cf0 con 0x7fa3000b16f0 2023-10-18T09:03:05.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.137+0000 7fa2f6ffd640 1 -- 172.21.15.40:0/2435382824 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f800aab0 con 0x7fa3000b16f0 2023-10-18T09:03:05.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.137+0000 7fa30732c640 1 -- 172.21.15.40:0/2435382824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3001e5a30 con 0x7fa3000b16f0 2023-10-18T09:03:05.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.137+0000 7fa30732c640 1 -- 172.21.15.40:0/2435382824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3001e5ff0 con 0x7fa3000b16f0 2023-10-18T09:03:05.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.139+0000 7fa30732c640 1 -- 172.21.15.40:0/2435382824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa300066e00 con 0x7fa3000b16f0 2023-10-18T09:03:05.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.139+0000 7fa2f6ffd640 1 -- 172.21.15.40:0/2435382824 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa2f800ac10 con 0x7fa3000b16f0 2023-10-18T09:03:05.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.140+0000 7fa2f6ffd640 1 --2- 172.21.15.40:0/2435382824 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa2e406ae40 0x7fa2e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:05.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.140+0000 7fa2f6ffd640 1 -- 172.21.15.40:0/2435382824 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2f80896d0 con 0x7fa3000b16f0 2023-10-18T09:03:05.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.141+0000 7fa30632a640 1 --2- 172.21.15.40:0/2435382824 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa2e406ae40 0x7fa2e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:05.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.142+0000 7fa30632a640 1 --2- 172.21.15.40:0/2435382824 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa2e406ae40 0x7fa2e406d300 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc035000 tx=0x7fa2fc004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:05.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.145+0000 7fa2f6ffd640 1 -- 172.21.15.40:0/2435382824 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2f80546e0 con 0x7fa3000b16f0 2023-10-18T09:03:05.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.339+0000 7fa30732c640 1 -- 172.21.15.40:0/2435382824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3000c9ce0 con 0x7fa3000b16f0 2023-10-18T09:03:05.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.340+0000 7fa2f6ffd640 1 -- 172.21.15.40:0/2435382824 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2f8057d00 con 0x7fa3000b16f0 2023-10-18T09:03:05.342 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:03:05.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.342+0000 7fa30732c640 1 -- 172.21.15.40:0/2435382824 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa2e406ae40 msgr2=0x7fa2e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:05.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.342+0000 7fa30732c640 1 --2- 172.21.15.40:0/2435382824 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa2e406ae40 0x7fa2e406d300 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc035000 tx=0x7fa2fc004730 comp rx=0 tx=0).stop 2023-10-18T09:03:05.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.342+0000 7fa30732c640 1 -- 172.21.15.40:0/2435382824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3000b16f0 msgr2=0x7fa300159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.342+0000 7fa30732c640 1 --2- 172.21.15.40:0/2435382824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3000b16f0 0x7fa300159e40 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800c7a0 tx=0x7fa2f800cc60 comp rx=0 tx=0).stop 2023-10-18T09:03:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.343+0000 7fa30732c640 1 -- 172.21.15.40:0/2435382824 shutdown_connections 2023-10-18T09:03:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.343+0000 7fa30732c640 1 --2- 172.21.15.40:0/2435382824 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa2e406ae40 0x7fa2e406d300 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.343+0000 7fa30732c640 1 --2- 172.21.15.40:0/2435382824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3000b16f0 0x7fa300159e40 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:05.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.343+0000 7fa30732c640 1 --2- 172.21.15.40:0/2435382824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa300097a10 0x7fa300158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.343+0000 7fa30732c640 1 --2- 172.21.15.40:0/2435382824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa30006aa40 0x7fa3000b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.343+0000 7fa30732c640 1 -- 172.21.15.40:0/2435382824 >> 172.21.15.40:0/2435382824 conn(0x7fa3000b0ad0 msgr2=0x7fa3000af870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.343+0000 7fa30732c640 1 -- 172.21.15.40:0/2435382824 shutdown_connections 2023-10-18T09:03:05.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.343+0000 7fa30732c640 1 -- 172.21.15.40:0/2435382824 wait complete. 2023-10-18T09:03:05.364 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:03:05.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.602+0000 7f87d2ba4640 1 -- 172.21.15.40:0/3739572615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b8005e10 msgr2=0x7f87b8006290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:05.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.602+0000 7f87d2ba4640 1 --2- 172.21.15.40:0/3739572615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b8005e10 0x7f87b8006290 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f87c4006ed0 tx=0x7f87b8006900 comp rx=0 tx=0).stop 2023-10-18T09:03:05.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.602+0000 7f87d1141640 1 -- 172.21.15.40:0/3739572615 reap_dead start 2023-10-18T09:03:05.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.603+0000 7f87d2ba4640 1 -- 172.21.15.40:0/3739572615 shutdown_connections 2023-10-18T09:03:05.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.603+0000 7f87d2ba4640 1 -- 172.21.15.40:0/3739572615 >> 172.21.15.40:0/3739572615 conn(0x7f87cc069720 msgr2=0x7f87cc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:05.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.603+0000 7f87d2ba4640 1 -- 172.21.15.40:0/3739572615 shutdown_connections 2023-10-18T09:03:05.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.603+0000 7f87d2ba4640 1 -- 172.21.15.40:0/3739572615 wait complete. 2023-10-18T09:03:05.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.605+0000 7f87d2ba4640 1 Processor -- start 2023-10-18T09:03:05.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.605+0000 7f87d2ba4640 1 -- start start 2023-10-18T09:03:05.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.606+0000 7f87d2ba4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b8005e10 0x7f87cc0c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:05.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.606+0000 7f87d2ba4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87cc158e10 0x7f87cc15a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:05.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.606+0000 7f87d2ba4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87cc14b8d0 0x7f87cc153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:05.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.606+0000 7f87d2ba4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc15a180 con 0x7f87b8005e10 2023-10-18T09:03:05.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.606+0000 7f87d2ba4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc159290 con 0x7f87cc14b8d0 2023-10-18T09:03:05.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.606+0000 7f87d2ba4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc151300 con 0x7f87cc158e10 2023-10-18T09:03:05.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.606+0000 7f87d0940640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b8005e10 0x7f87cc0c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:05.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.606+0000 7f87d0940640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b8005e10 0x7f87cc0c30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60522/0 (socket says 172.21.15.40:60522) 2023-10-18T09:03:05.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.606+0000 7f87d0940640 1 -- 172.21.15.40:0/2255552403 learned_addr learned my addr 172.21.15.40:0/2255552403 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:05.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.606+0000 7f87cbfff640 1 --2- 172.21.15.40:0/2255552403 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87cc158e10 0x7f87cc15a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:05.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.606+0000 7f87d0940640 1 -- 172.21.15.40:0/2255552403 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87cc158e10 msgr2=0x7f87cc15a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:05.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.606+0000 7f87d0940640 1 --2- 172.21.15.40:0/2255552403 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87cc158e10 0x7f87cc15a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:05.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.606+0000 7f87d0940640 1 -- 172.21.15.40:0/2255552403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87cc14b8d0 msgr2=0x7f87cc153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:05.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.606+0000 7f87d0940640 1 --2- 172.21.15.40:0/2255552403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87cc14b8d0 0x7f87cc153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:05.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.606+0000 7f87d0940640 1 -- 172.21.15.40:0/2255552403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc159500 con 0x7f87b8005e10 2023-10-18T09:03:05.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.606+0000 7f87cbfff640 1 --2- 172.21.15.40:0/2255552403 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87cc158e10 0x7f87cc15a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.607+0000 7f87d0940640 1 --2- 172.21.15.40:0/2255552403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b8005e10 0x7f87cc0c30e0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00d6f0 tx=0x7f87bc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.607+0000 7f87c97fa640 1 -- 172.21.15.40:0/2255552403 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc00de00 con 0x7f87b8005e10 2023-10-18T09:03:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.607+0000 7f87c97fa640 1 -- 172.21.15.40:0/2255552403 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f87bc0063d0 con 0x7f87b8005e10 2023-10-18T09:03:05.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.607+0000 7f87c97fa640 1 -- 172.21.15.40:0/2255552403 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc00b9b0 con 0x7f87b8005e10 2023-10-18T09:03:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.607+0000 7f87d2ba4640 1 -- 172.21.15.40:0/2255552403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc1597e0 con 0x7f87b8005e10 2023-10-18T09:03:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.607+0000 7f87d2ba4640 1 -- 172.21.15.40:0/2255552403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc159da0 con 0x7f87b8005e10 2023-10-18T09:03:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.609+0000 7f87d2ba4640 1 -- 172.21.15.40:0/2255552403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87cc066fd0 con 0x7f87b8005e10 2023-10-18T09:03:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.610+0000 7f87c97fa640 1 -- 172.21.15.40:0/2255552403 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f87bc00bb10 con 0x7f87b8005e10 2023-10-18T09:03:05.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.610+0000 7f87c97fa640 1 --2- 172.21.15.40:0/2255552403 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87ac06ae40 0x7f87ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:05.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.611+0000 7f87c97fa640 1 -- 172.21.15.40:0/2255552403 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87bc089f30 con 0x7f87b8005e10 2023-10-18T09:03:05.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.611+0000 7f87cbfff640 1 --2- 172.21.15.40:0/2255552403 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87ac06ae40 0x7f87ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:05.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.612+0000 7f87cbfff640 1 --2- 172.21.15.40:0/2255552403 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87ac06ae40 0x7f87ac06d300 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f87cc150a30 tx=0x7f87b4009500 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:05.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.615+0000 7f87c97fa640 1 -- 172.21.15.40:0/2255552403 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87bc054f40 con 0x7f87b8005e10 2023-10-18T09:03:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.808+0000 7f87d2ba4640 1 -- 172.21.15.40:0/2255552403 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87cc0b2250 con 0x7f87b8005e10 2023-10-18T09:03:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.809+0000 7f87c97fa640 1 -- 172.21.15.40:0/2255552403 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v133) v1 ==== 104+0+11 (secure 0 0 0) 0x7f87bc0140a0 con 0x7f87b8005e10 2023-10-18T09:03:05.811 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:03:05.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.812+0000 7f87d2ba4640 1 -- 172.21.15.40:0/2255552403 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87ac06ae40 msgr2=0x7f87ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:05.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.812+0000 7f87d2ba4640 1 --2- 172.21.15.40:0/2255552403 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87ac06ae40 0x7f87ac06d300 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f87cc150a30 tx=0x7f87b4009500 comp rx=0 tx=0).stop 2023-10-18T09:03:05.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.812+0000 7f87d2ba4640 1 -- 172.21.15.40:0/2255552403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b8005e10 msgr2=0x7f87cc0c30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:05.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.812+0000 7f87d2ba4640 1 --2- 172.21.15.40:0/2255552403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b8005e10 0x7f87cc0c30e0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f87bc00d6f0 tx=0x7f87bc00dbb0 comp rx=0 tx=0).stop 2023-10-18T09:03:05.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.812+0000 7f87d2ba4640 1 -- 172.21.15.40:0/2255552403 shutdown_connections 2023-10-18T09:03:05.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.812+0000 7f87d2ba4640 1 --2- 172.21.15.40:0/2255552403 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f87ac06ae40 0x7f87ac06d300 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:05.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.812+0000 7f87d2ba4640 1 --2- 172.21.15.40:0/2255552403 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87cc14b8d0 0x7f87cc153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:05.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.812+0000 7f87d2ba4640 1 --2- 172.21.15.40:0/2255552403 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87cc158e10 0x7f87cc15a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:05.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.812+0000 7f87d2ba4640 1 --2- 172.21.15.40:0/2255552403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87b8005e10 0x7f87cc0c30e0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:05.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.812+0000 7f87d2ba4640 1 -- 172.21.15.40:0/2255552403 >> 172.21.15.40:0/2255552403 conn(0x7f87cc069720 msgr2=0x7f87cc0ab550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:05.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.812+0000 7f87d2ba4640 1 -- 172.21.15.40:0/2255552403 shutdown_connections 2023-10-18T09:03:05.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:05.812+0000 7f87d2ba4640 1 -- 172.21.15.40:0/2255552403 wait complete. 2023-10-18T09:03:05.834 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:03:06.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.066+0000 7fc0e88d1640 1 -- 172.21.15.40:0/2951440836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0d00047c0 msgr2=0x7fc0d0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:06.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.066+0000 7fc0e88d1640 1 --2- 172.21.15.40:0/2951440836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0d00047c0 0x7fc0d0004c40 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00c6f0 tx=0x7fc0d00057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:06.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.066+0000 7fc0e3fff640 1 -- 172.21.15.40:0/2951440836 reap_dead start 2023-10-18T09:03:06.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.066+0000 7fc0e88d1640 1 -- 172.21.15.40:0/2951440836 shutdown_connections 2023-10-18T09:03:06.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.066+0000 7fc0e88d1640 1 -- 172.21.15.40:0/2951440836 >> 172.21.15.40:0/2951440836 conn(0x7fc0e409a610 msgr2=0x7fc0e40d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:06.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.066+0000 7fc0e88d1640 1 -- 172.21.15.40:0/2951440836 shutdown_connections 2023-10-18T09:03:06.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.066+0000 7fc0e88d1640 1 -- 172.21.15.40:0/2951440836 wait complete. 2023-10-18T09:03:06.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.068+0000 7fc0e88d1640 1 Processor -- start 2023-10-18T09:03:06.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.068+0000 7fc0e88d1640 1 -- start start 2023-10-18T09:03:06.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.070+0000 7fc0e88d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0d00047c0 0x7fc0e409ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.070+0000 7fc0e88d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0e409ba90 0x7fc0e409cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.070+0000 7fc0e88d1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e414d2d0 0x7fc0e414d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.070+0000 7fc0e88d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40c72e0 con 0x7fc0e409ba90 2023-10-18T09:03:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.070+0000 7fc0e88d1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40c4db0 con 0x7fc0d00047c0 2023-10-18T09:03:06.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.070+0000 7fc0e88d1640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e414dc70 con 0x7fc0e414d2d0 2023-10-18T09:03:06.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.071+0000 7fc0e2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0e409ba90 0x7fc0e409cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:06.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.071+0000 7fc0e2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0e409ba90 0x7fc0e409cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60536/0 (socket says 172.21.15.40:60536) 2023-10-18T09:03:06.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.071+0000 7fc0e2ffd640 1 -- 172.21.15.40:0/3250645317 learned_addr learned my addr 172.21.15.40:0/3250645317 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:06.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.071+0000 7fc0e37fe640 1 --2- 172.21.15.40:0/3250645317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0d00047c0 0x7fc0e409ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:06.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.071+0000 7fc0e3fff640 1 --2- 172.21.15.40:0/3250645317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e414d2d0 0x7fc0e414d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:06.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.071+0000 7fc0e2ffd640 1 -- 172.21.15.40:0/3250645317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e414d2d0 msgr2=0x7fc0e414d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:06.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.071+0000 7fc0e2ffd640 1 --2- 172.21.15.40:0/3250645317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e414d2d0 0x7fc0e414d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:06.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.071+0000 7fc0e2ffd640 1 -- 172.21.15.40:0/3250645317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0d00047c0 msgr2=0x7fc0e409ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:06.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.071+0000 7fc0e2ffd640 1 --2- 172.21.15.40:0/3250645317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0d00047c0 0x7fc0e409ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:06.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.071+0000 7fc0e2ffd640 1 -- 172.21.15.40:0/3250645317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e40c7530 con 0x7fc0e409ba90 2023-10-18T09:03:06.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.071+0000 7fc0e3fff640 1 --2- 172.21.15.40:0/3250645317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e414d2d0 0x7fc0e414d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:03:06.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.072+0000 7fc0e2ffd640 1 --2- 172.21.15.40:0/3250645317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0e409ba90 0x7fc0e409cb80 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc035040 tx=0x7fc0cc003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:06.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.072+0000 7fc0dbfff640 1 -- 172.21.15.40:0/3250645317 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc038640 con 0x7fc0e409ba90 2023-10-18T09:03:06.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.072+0000 7fc0dbfff640 1 -- 172.21.15.40:0/3250645317 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc0cc038890 con 0x7fc0e409ba90 2023-10-18T09:03:06.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.072+0000 7fc0e88d1640 1 -- 172.21.15.40:0/3250645317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e40c7810 con 0x7fc0e409ba90 2023-10-18T09:03:06.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.072+0000 7fc0dbfff640 1 -- 172.21.15.40:0/3250645317 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc038c60 con 0x7fc0e409ba90 2023-10-18T09:03:06.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.072+0000 7fc0e88d1640 1 -- 172.21.15.40:0/3250645317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e409bf10 con 0x7fc0e409ba90 2023-10-18T09:03:06.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.074+0000 7fc0e88d1640 1 -- 172.21.15.40:0/3250645317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e4071da0 con 0x7fc0e409ba90 2023-10-18T09:03:06.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.074+0000 7fc0dbfff640 1 -- 172.21.15.40:0/3250645317 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc0cc02e430 con 0x7fc0e409ba90 2023-10-18T09:03:06.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.076+0000 7fc0dbfff640 1 --2- 172.21.15.40:0/3250645317 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc0c406ae70 0x7fc0c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:06.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.076+0000 7fc0dbfff640 1 -- 172.21.15.40:0/3250645317 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0cc0aafd0 con 0x7fc0e409ba90 2023-10-18T09:03:06.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.076+0000 7fc0e37fe640 1 --2- 172.21.15.40:0/3250645317 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc0c406ae70 0x7fc0c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:06.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.078+0000 7fc0e37fe640 1 --2- 172.21.15.40:0/3250645317 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc0c406ae70 0x7fc0c406d330 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4001b20 tx=0x7fc0d4001970 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.081+0000 7fc0dbfff640 1 -- 172.21.15.40:0/3250645317 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc0cc076070 con 0x7fc0e409ba90 2023-10-18T09:03:06.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.278+0000 7fc0e88d1640 1 -- 172.21.15.40:0/3250645317 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc0e4066e00 con 0x7fc0e409ba90 2023-10-18T09:03:06.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.280+0000 7fc0dbfff640 1 -- 172.21.15.40:0/3250645317 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v133) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc0cc047020 con 0x7fc0e409ba90 2023-10-18T09:03:06.282 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:03:06.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.282+0000 7fc0e88d1640 1 -- 172.21.15.40:0/3250645317 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc0c406ae70 msgr2=0x7fc0c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:06.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.282+0000 7fc0e88d1640 1 --2- 172.21.15.40:0/3250645317 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc0c406ae70 0x7fc0c406d330 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc0d4001b20 tx=0x7fc0d4001970 comp rx=0 tx=0).stop 2023-10-18T09:03:06.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.283+0000 7fc0e88d1640 1 -- 172.21.15.40:0/3250645317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0e409ba90 msgr2=0x7fc0e409cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:06.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.283+0000 7fc0e88d1640 1 --2- 172.21.15.40:0/3250645317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0e409ba90 0x7fc0e409cb80 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc035040 tx=0x7fc0cc003e80 comp rx=0 tx=0).stop 2023-10-18T09:03:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.283+0000 7fc0e88d1640 1 -- 172.21.15.40:0/3250645317 shutdown_connections 2023-10-18T09:03:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.283+0000 7fc0e88d1640 1 --2- 172.21.15.40:0/3250645317 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc0c406ae70 0x7fc0c406d330 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.283+0000 7fc0e88d1640 1 --2- 172.21.15.40:0/3250645317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0e414d2d0 0x7fc0e414d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.283+0000 7fc0e88d1640 1 --2- 172.21.15.40:0/3250645317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0e409ba90 0x7fc0e409cb80 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.283+0000 7fc0e88d1640 1 --2- 172.21.15.40:0/3250645317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0d00047c0 0x7fc0e409ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:06.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.283+0000 7fc0e88d1640 1 -- 172.21.15.40:0/3250645317 >> 172.21.15.40:0/3250645317 conn(0x7fc0e409a610 msgr2=0x7fc0e406a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:06.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.283+0000 7fc0e88d1640 1 -- 172.21.15.40:0/3250645317 shutdown_connections 2023-10-18T09:03:06.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.283+0000 7fc0e88d1640 1 -- 172.21.15.40:0/3250645317 wait complete. 2023-10-18T09:03:06.304 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:03:06.431 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:03:06.441 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:03:06.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.536+0000 7f32266ec640 1 -- 172.21.15.40:0/2860975894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32080039a0 msgr2=0x7f3208003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:06.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.536+0000 7f32266ec640 1 --2- 172.21.15.40:0/2860975894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32080039a0 0x7f3208003dc0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3210009b80 tx=0x7f32080057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:06.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.536+0000 7f3224c89640 1 -- 172.21.15.40:0/2860975894 reap_dead start 2023-10-18T09:03:06.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.536+0000 7f32266ec640 1 -- 172.21.15.40:0/2860975894 shutdown_connections 2023-10-18T09:03:06.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.536+0000 7f32266ec640 1 -- 172.21.15.40:0/2860975894 >> 172.21.15.40:0/2860975894 conn(0x7f32200b0d30 msgr2=0x7f32200b1140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:06.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.536+0000 7f32266ec640 1 -- 172.21.15.40:0/2860975894 shutdown_connections 2023-10-18T09:03:06.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.536+0000 7f32266ec640 1 -- 172.21.15.40:0/2860975894 wait complete. 2023-10-18T09:03:06.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.538+0000 7f32266ec640 1 Processor -- start 2023-10-18T09:03:06.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.538+0000 7f32266ec640 1 -- start start 2023-10-18T09:03:06.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.539+0000 7f32266ec640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32080039a0 0x7f3220155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:06.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.539+0000 7f32266ec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f322014d940 0x7f32200c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:06.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.539+0000 7f32266ec640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3220154850 0x7f3220154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:06.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.539+0000 7f32266ec640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322014d220 con 0x7f322014d940 2023-10-18T09:03:06.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.539+0000 7f32266ec640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3220156420 con 0x7f32080039a0 2023-10-18T09:03:06.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.539+0000 7f32266ec640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32201537b0 con 0x7f3220154850 2023-10-18T09:03:06.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.540+0000 7f321f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f322014d940 0x7f32200c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:06.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.540+0000 7f321f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f322014d940 0x7f32200c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60552/0 (socket says 172.21.15.40:60552) 2023-10-18T09:03:06.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.540+0000 7f321f7fe640 1 -- 172.21.15.40:0/4008137969 learned_addr learned my addr 172.21.15.40:0/4008137969 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:06.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.540+0000 7f321ffff640 1 --2- 172.21.15.40:0/4008137969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32080039a0 0x7f3220155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:06.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.540+0000 7f3224c89640 1 --2- 172.21.15.40:0/4008137969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3220154850 0x7f3220154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:06.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.540+0000 7f321f7fe640 1 -- 172.21.15.40:0/4008137969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3220154850 msgr2=0x7f3220154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:06.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.540+0000 7f321f7fe640 1 --2- 172.21.15.40:0/4008137969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3220154850 0x7f3220154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:06.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.540+0000 7f321f7fe640 1 -- 172.21.15.40:0/4008137969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32080039a0 msgr2=0x7f3220155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:06.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.540+0000 7f321f7fe640 1 --2- 172.21.15.40:0/4008137969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32080039a0 0x7f3220155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:06.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.540+0000 7f321f7fe640 1 -- 172.21.15.40:0/4008137969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3220153a90 con 0x7f322014d940 2023-10-18T09:03:06.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.541+0000 7f321f7fe640 1 --2- 172.21.15.40:0/4008137969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f322014d940 0x7f32200c9000 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3214010a10 tx=0x7f3214010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:06.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.541+0000 7f321cff9640 1 -- 172.21.15.40:0/4008137969 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214009720 con 0x7f322014d940 2023-10-18T09:03:06.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.541+0000 7f321cff9640 1 -- 172.21.15.40:0/4008137969 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f321400eda0 con 0x7f322014d940 2023-10-18T09:03:06.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.541+0000 7f321cff9640 1 -- 172.21.15.40:0/4008137969 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214005280 con 0x7f322014d940 2023-10-18T09:03:06.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.541+0000 7f32266ec640 1 -- 172.21.15.40:0/4008137969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3220153d70 con 0x7f322014d940 2023-10-18T09:03:06.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.541+0000 7f32266ec640 1 -- 172.21.15.40:0/4008137969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3220154330 con 0x7f322014d940 2023-10-18T09:03:06.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.542+0000 7f32266ec640 1 -- 172.21.15.40:0/4008137969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3220066a90 con 0x7f322014d940 2023-10-18T09:03:06.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.543+0000 7f321cff9640 1 -- 172.21.15.40:0/4008137969 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f32140053e0 con 0x7f322014d940 2023-10-18T09:03:06.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.545+0000 7f321cff9640 1 --2- 172.21.15.40:0/4008137969 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f320c06ae70 0x7f320c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:06.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.545+0000 7f321cff9640 1 -- 172.21.15.40:0/4008137969 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3214016070 con 0x7f322014d940 2023-10-18T09:03:06.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.545+0000 7f321ffff640 1 --2- 172.21.15.40:0/4008137969 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f320c06ae70 0x7f320c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:06.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.546+0000 7f321ffff640 1 --2- 172.21.15.40:0/4008137969 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f320c06ae70 0x7f320c06d330 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3210036000 tx=0x7f32100046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:06.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.549+0000 7f321cff9640 1 -- 172.21.15.40:0/4008137969 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32140553c0 con 0x7f322014d940 2023-10-18T09:03:06.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.779+0000 7f32266ec640 1 -- 172.21.15.40:0/4008137969 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32200739c0 con 0x7f322014d940 2023-10-18T09:03:06.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.781+0000 7f321cff9640 1 -- 172.21.15.40:0/4008137969 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f321405de60 con 0x7f322014d940 2023-10-18T09:03:06.783 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:06.784 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/700924196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":700924196},{"type":"v1","addr":"172.21.15.40:6833","nonce":700924196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/877414874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":877414874},{"type":"v1","addr":"172.21.15.144:6835","nonce":877414874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2596943574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2596943574},{"type":"v1","addr":"172.21.15.144:6837","nonce":2596943574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:02:41.786627+0000","modified":"2023-10-18T09:02:44.862740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1801003551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1801003551},{"type":"v1","addr":"172.21.15.40:6835","nonce":1801003551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-18T09:03:06.784 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2023-10-18T09:03:06.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.784+0000 7f32266ec640 1 -- 172.21.15.40:0/4008137969 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f320c06ae70 msgr2=0x7f320c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:06.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.784+0000 7f32266ec640 1 --2- 172.21.15.40:0/4008137969 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f320c06ae70 0x7f320c06d330 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3210036000 tx=0x7f32100046c0 comp rx=0 tx=0).stop 2023-10-18T09:03:06.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.784+0000 7f32266ec640 1 -- 172.21.15.40:0/4008137969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f322014d940 msgr2=0x7f32200c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:06.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.784+0000 7f32266ec640 1 --2- 172.21.15.40:0/4008137969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f322014d940 0x7f32200c9000 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3214010a10 tx=0x7f3214010e10 comp rx=0 tx=0).stop 2023-10-18T09:03:06.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.784+0000 7f32266ec640 1 -- 172.21.15.40:0/4008137969 shutdown_connections 2023-10-18T09:03:06.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.784+0000 7f32266ec640 1 --2- 172.21.15.40:0/4008137969 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f320c06ae70 0x7f320c06d330 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:06.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.784+0000 7f32266ec640 1 --2- 172.21.15.40:0/4008137969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3220154850 0x7f3220154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:06.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.784+0000 7f32266ec640 1 --2- 172.21.15.40:0/4008137969 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f322014d940 0x7f32200c9000 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.784+0000 7f32266ec640 1 --2- 172.21.15.40:0/4008137969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32080039a0 0x7f3220155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.784+0000 7f32266ec640 1 -- 172.21.15.40:0/4008137969 >> 172.21.15.40:0/4008137969 conn(0x7f32200b0d30 msgr2=0x7f3220156c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.784+0000 7f32266ec640 1 -- 172.21.15.40:0/4008137969 shutdown_connections 2023-10-18T09:03:06.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:06.784+0000 7f32266ec640 1 -- 172.21.15.40:0/4008137969 wait complete. 2023-10-18T09:03:06.806 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:03:07.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.036+0000 7f002ffff640 1 -- 172.21.15.40:0/4016181085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300b4a00 msgr2=0x7f00300cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:07.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.036+0000 7f002ffff640 1 --2- 172.21.15.40:0/4016181085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300b4a00 0x7f00300cdd50 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f00300708a0 tx=0x7f002002c190 comp rx=0 tx=0).stop 2023-10-18T09:03:07.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.037+0000 7f002ffff640 1 -- 172.21.15.40:0/4016181085 shutdown_connections 2023-10-18T09:03:07.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.037+0000 7f002ffff640 1 --2- 172.21.15.40:0/4016181085 >> v2:172.21.15.144:3300/0 conn(0x7f003006b8c0 0x7f003015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.037+0000 7f002ffff640 1 --2- 172.21.15.40:0/4016181085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300b4a00 0x7f00300cdd50 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.037+0000 7f002ffff640 1 -- 172.21.15.40:0/4016181085 >> 172.21.15.40:0/4016181085 conn(0x7f00300b0af0 msgr2=0x7f00300b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:07.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.037+0000 7f002ffff640 1 -- 172.21.15.40:0/4016181085 shutdown_connections 2023-10-18T09:03:07.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.037+0000 7f002ffff640 1 -- 172.21.15.40:0/4016181085 wait complete. 2023-10-18T09:03:07.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.038+0000 7f002ffff640 1 Processor -- start 2023-10-18T09:03:07.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.039+0000 7f002ffff640 1 -- start start 2023-10-18T09:03:07.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f003006ac60 0x7f00300c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:07.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002ffff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f003006b8c0 0x7f003015dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:07.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300b4a00 0x7f00300b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:07.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002ffff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00300c5800 con 0x7f00300b4a00 2023-10-18T09:03:07.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002ffff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003015e020 con 0x7f003006ac60 2023-10-18T09:03:07.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002ffff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003015b370 con 0x7f003006b8c0 2023-10-18T09:03:07.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300b4a00 0x7f00300b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:07.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300b4a00 0x7f00300b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60576/0 (socket says 172.21.15.40:60576) 2023-10-18T09:03:07.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002f7fe640 1 -- 172.21.15.40:0/887693698 learned_addr learned my addr 172.21.15.40:0/887693698 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:07.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002effd640 1 --2- 172.21.15.40:0/887693698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f003006ac60 0x7f00300c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:07.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002e7fc640 1 --2- 172.21.15.40:0/887693698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f003006b8c0 0x7f003015dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:07.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002f7fe640 1 -- 172.21.15.40:0/887693698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f003006b8c0 msgr2=0x7f003015dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:07.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002f7fe640 1 --2- 172.21.15.40:0/887693698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f003006b8c0 0x7f003015dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002f7fe640 1 -- 172.21.15.40:0/887693698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f003006ac60 msgr2=0x7f00300c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:07.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002f7fe640 1 --2- 172.21.15.40:0/887693698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f003006ac60 0x7f00300c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002f7fe640 1 -- 172.21.15.40:0/887693698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0020009840 con 0x7f00300b4a00 2023-10-18T09:03:07.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002e7fc640 1 --2- 172.21.15.40:0/887693698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f003006b8c0 0x7f003015dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:07.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.040+0000 7f002effd640 1 --2- 172.21.15.40:0/887693698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f003006ac60 0x7f00300c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:03:07.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.041+0000 7f002f7fe640 1 --2- 172.21.15.40:0/887693698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300b4a00 0x7f00300b20f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f002800d780 tx=0x7f002800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.041+0000 7f00177fe640 1 -- 172.21.15.40:0/887693698 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0028009bd0 con 0x7f00300b4a00 2023-10-18T09:03:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.041+0000 7f00177fe640 1 -- 172.21.15.40:0/887693698 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0028009e20 con 0x7f00300b4a00 2023-10-18T09:03:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.041+0000 7f00177fe640 1 -- 172.21.15.40:0/887693698 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0028005280 con 0x7f00300b4a00 2023-10-18T09:03:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.041+0000 7f002ffff640 1 -- 172.21.15.40:0/887693698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00301554f0 con 0x7f00300b4a00 2023-10-18T09:03:07.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.041+0000 7f002ffff640 1 -- 172.21.15.40:0/887693698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003015b5c0 con 0x7f00300b4a00 2023-10-18T09:03:07.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.043+0000 7f002ffff640 1 -- 172.21.15.40:0/887693698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00300b1dd0 con 0x7f00300b4a00 2023-10-18T09:03:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.043+0000 7f00177fe640 1 -- 172.21.15.40:0/887693698 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f00280053e0 con 0x7f00300b4a00 2023-10-18T09:03:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.045+0000 7f00177fe640 1 --2- 172.21.15.40:0/887693698 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f001c06ae40 0x7f001c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.045+0000 7f00177fe640 1 -- 172.21.15.40:0/887693698 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00280896e0 con 0x7f00300b4a00 2023-10-18T09:03:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.046+0000 7f002effd640 1 --2- 172.21.15.40:0/887693698 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f001c06ae40 0x7f001c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.047+0000 7f002effd640 1 --2- 172.21.15.40:0/887693698 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f001c06ae40 0x7f001c06d300 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0020035000 tx=0x7f002002df90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.051+0000 7f00177fe640 1 -- 172.21.15.40:0/887693698 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0028054820 con 0x7f00300b4a00 2023-10-18T09:03:07.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.251+0000 7f002ffff640 1 -- 172.21.15.40:0/887693698 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0030066e00 con 0x7f00300b4a00 2023-10-18T09:03:07.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.253+0000 7f00177fe640 1 -- 172.21.15.40:0/887693698 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+16539 (secure 0 0 0) 0x7f0028017090 con 0x7f00300b4a00 2023-10-18T09:03:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:07.256 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":133,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:02:56.085956+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:02:39.531482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-18T09:02:40.812071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:03:07.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.255+0000 7f002ffff640 1 -- 172.21.15.40:0/887693698 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f001c06ae40 msgr2=0x7f001c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:07.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.255+0000 7f002ffff640 1 --2- 172.21.15.40:0/887693698 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f001c06ae40 0x7f001c06d300 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0020035000 tx=0x7f002002df90 comp rx=0 tx=0).stop 2023-10-18T09:03:07.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.255+0000 7f002ffff640 1 -- 172.21.15.40:0/887693698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300b4a00 msgr2=0x7f00300b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:07.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.255+0000 7f002ffff640 1 --2- 172.21.15.40:0/887693698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300b4a00 0x7f00300b20f0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f002800d780 tx=0x7f002800dc40 comp rx=0 tx=0).stop 2023-10-18T09:03:07.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.256+0000 7f002ffff640 1 -- 172.21.15.40:0/887693698 shutdown_connections 2023-10-18T09:03:07.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.256+0000 7f002ffff640 1 --2- 172.21.15.40:0/887693698 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f001c06ae40 0x7f001c06d300 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.256+0000 7f002ffff640 1 --2- 172.21.15.40:0/887693698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00300b4a00 0x7f00300b20f0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.256+0000 7f002ffff640 1 --2- 172.21.15.40:0/887693698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f003006b8c0 0x7f003015dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.256+0000 7f002ffff640 1 --2- 172.21.15.40:0/887693698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f003006ac60 0x7f00300c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.256+0000 7f002ffff640 1 -- 172.21.15.40:0/887693698 >> 172.21.15.40:0/887693698 conn(0x7f00300b0af0 msgr2=0x7f00300b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.256+0000 7f002ffff640 1 -- 172.21.15.40:0/887693698 shutdown_connections 2023-10-18T09:03:07.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.256+0000 7f002ffff640 1 -- 172.21.15.40:0/887693698 wait complete. 2023-10-18T09:03:07.276 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T09:03:07.277 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:03:07.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.507+0000 7f0c0207e640 1 -- 172.21.15.40:0/2213578519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bfc1425a0 msgr2=0x7f0bfc0cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:07.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.507+0000 7f0c0207e640 1 --2- 172.21.15.40:0/2213578519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bfc1425a0 0x7f0bfc0cafc0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc071ab0 tx=0x7f0bf402c2f0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.507+0000 7f0c0207e640 1 -- 172.21.15.40:0/2213578519 shutdown_connections 2023-10-18T09:03:07.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.507+0000 7f0c0207e640 1 --2- 172.21.15.40:0/2213578519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bfc1425a0 0x7f0bfc0cafc0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.507+0000 7f0c0207e640 1 --2- 172.21.15.40:0/2213578519 >> v2:172.21.15.144:3300/0 conn(0x7f0bfc0ca160 0x7f0bfc0b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.507+0000 7f0c0207e640 1 -- 172.21.15.40:0/2213578519 >> 172.21.15.40:0/2213578519 conn(0x7f0bfc069720 msgr2=0x7f0bfc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:07.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.508+0000 7f0c0207e640 1 -- 172.21.15.40:0/2213578519 shutdown_connections 2023-10-18T09:03:07.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.508+0000 7f0c0207e640 1 -- 172.21.15.40:0/2213578519 wait complete. 2023-10-18T09:03:07.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.509+0000 7f0c0207e640 1 Processor -- start 2023-10-18T09:03:07.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.510+0000 7f0c0207e640 1 -- start start 2023-10-18T09:03:07.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.510+0000 7f0c0207e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bfc0ca160 0x7f0bfc0adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:07.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.510+0000 7f0c0207e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bfc1420f0 0x7f0bfc132e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:07.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.510+0000 7f0c0207e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bfc1425a0 0x7f0bfc158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:07.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.511+0000 7f0c0207e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc0ad690 con 0x7f0bfc0ca160 2023-10-18T09:03:07.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.511+0000 7f0c0207e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc0c2180 con 0x7f0bfc1425a0 2023-10-18T09:03:07.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.511+0000 7f0c0207e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc0adff0 con 0x7f0bfc1420f0 2023-10-18T09:03:07.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.511+0000 7f0c0107c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bfc0ca160 0x7f0bfc0adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:07.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.511+0000 7f0c0107c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bfc0ca160 0x7f0bfc0adab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60604/0 (socket says 172.21.15.40:60604) 2023-10-18T09:03:07.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.511+0000 7f0c0107c640 1 -- 172.21.15.40:0/1592017548 learned_addr learned my addr 172.21.15.40:0/1592017548 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:07.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.511+0000 7f0c0187d640 1 --2- 172.21.15.40:0/1592017548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bfc1425a0 0x7f0bfc158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:07.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.511+0000 7f0c0107c640 1 -- 172.21.15.40:0/1592017548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bfc1420f0 msgr2=0x7f0bfc132e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:07.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.511+0000 7f0c0087b640 1 --2- 172.21.15.40:0/1592017548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bfc1420f0 0x7f0bfc132e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:07.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.511+0000 7f0c0107c640 1 --2- 172.21.15.40:0/1592017548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bfc1420f0 0x7f0bfc132e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.511+0000 7f0c0107c640 1 -- 172.21.15.40:0/1592017548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bfc1425a0 msgr2=0x7f0bfc158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:07.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.511+0000 7f0c0107c640 1 --2- 172.21.15.40:0/1592017548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bfc1425a0 0x7f0bfc158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.511+0000 7f0c0107c640 1 -- 172.21.15.40:0/1592017548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf4009900 con 0x7f0bfc0ca160 2023-10-18T09:03:07.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.511+0000 7f0c0187d640 1 --2- 172.21.15.40:0/1592017548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bfc1425a0 0x7f0bfc158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:07.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.512+0000 7f0c0107c640 1 --2- 172.21.15.40:0/1592017548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bfc0ca160 0x7f0bfc0adab0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800c4b0 tx=0x7f0bf800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:07.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.512+0000 7f0bf1ffb640 1 -- 172.21.15.40:0/1592017548 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8005480 con 0x7f0bfc0ca160 2023-10-18T09:03:07.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.512+0000 7f0bf1ffb640 1 -- 172.21.15.40:0/1592017548 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0bf80036e0 con 0x7f0bfc0ca160 2023-10-18T09:03:07.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.512+0000 7f0bf1ffb640 1 -- 172.21.15.40:0/1592017548 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8003af0 con 0x7f0bfc0ca160 2023-10-18T09:03:07.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.512+0000 7f0c0207e640 1 -- 172.21.15.40:0/1592017548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bfc0aca10 con 0x7f0bfc0ca160 2023-10-18T09:03:07.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.512+0000 7f0c0207e640 1 -- 172.21.15.40:0/1592017548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bfc0ad2b0 con 0x7f0bfc0ca160 2023-10-18T09:03:07.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.513+0000 7f0c0207e640 1 -- 172.21.15.40:0/1592017548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bfc0c98c0 con 0x7f0bfc0ca160 2023-10-18T09:03:07.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.514+0000 7f0bf1ffb640 1 -- 172.21.15.40:0/1592017548 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0bf8003c50 con 0x7f0bfc0ca160 2023-10-18T09:03:07.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.515+0000 7f0bf1ffb640 1 --2- 172.21.15.40:0/1592017548 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0be006ae40 0x7f0be006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:07.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.515+0000 7f0bf1ffb640 1 -- 172.21.15.40:0/1592017548 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bf8088e20 con 0x7f0bfc0ca160 2023-10-18T09:03:07.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.516+0000 7f0c0087b640 1 --2- 172.21.15.40:0/1592017548 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0be006ae40 0x7f0be006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:07.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.517+0000 7f0c0087b640 1 --2- 172.21.15.40:0/1592017548 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0be006ae40 0x7f0be006d300 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005fd0 tx=0x7f0bec005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:07.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.519+0000 7f0bf1ffb640 1 -- 172.21.15.40:0/1592017548 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0bf8053ed0 con 0x7f0bfc0ca160 2023-10-18T09:03:07.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.755+0000 7f0c0207e640 1 -- 172.21.15.40:0/1592017548 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bfc0ace00 con 0x7f0bfc0ca160 2023-10-18T09:03:07.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.756+0000 7f0bf1ffb640 1 -- 172.21.15.40:0/1592017548 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f0bf8017070 con 0x7f0bfc0ca160 2023-10-18T09:03:07.759 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:07.759 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/700924196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":700924196},{"type":"v1","addr":"172.21.15.40:6833","nonce":700924196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/877414874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":877414874},{"type":"v1","addr":"172.21.15.144:6835","nonce":877414874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2596943574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2596943574},{"type":"v1","addr":"172.21.15.144:6837","nonce":2596943574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:02:41.786627+0000","modified":"2023-10-18T09:02:44.862740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1801003551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1801003551},{"type":"v1","addr":"172.21.15.40:6835","nonce":1801003551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-18T09:03:07.759 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2023-10-18T09:03:07.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.759+0000 7f0c0207e640 1 -- 172.21.15.40:0/1592017548 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0be006ae40 msgr2=0x7f0be006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:07.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.759+0000 7f0c0207e640 1 --2- 172.21.15.40:0/1592017548 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0be006ae40 0x7f0be006d300 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005fd0 tx=0x7f0bec005850 comp rx=0 tx=0).stop 2023-10-18T09:03:07.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.759+0000 7f0c0207e640 1 -- 172.21.15.40:0/1592017548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bfc0ca160 msgr2=0x7f0bfc0adab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:07.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.759+0000 7f0c0207e640 1 --2- 172.21.15.40:0/1592017548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bfc0ca160 0x7f0bfc0adab0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800c4b0 tx=0x7f0bf800c970 comp rx=0 tx=0).stop 2023-10-18T09:03:07.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.759+0000 7f0c0207e640 1 -- 172.21.15.40:0/1592017548 shutdown_connections 2023-10-18T09:03:07.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.759+0000 7f0c0207e640 1 --2- 172.21.15.40:0/1592017548 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0be006ae40 0x7f0be006d300 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.759+0000 7f0c0207e640 1 --2- 172.21.15.40:0/1592017548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bfc1425a0 0x7f0bfc158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.759+0000 7f0c0207e640 1 --2- 172.21.15.40:0/1592017548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bfc1420f0 0x7f0bfc132e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.759+0000 7f0c0207e640 1 --2- 172.21.15.40:0/1592017548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bfc0ca160 0x7f0bfc0adab0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:07.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.759+0000 7f0c0207e640 1 -- 172.21.15.40:0/1592017548 >> 172.21.15.40:0/1592017548 conn(0x7f0bfc069720 msgr2=0x7f0bfc072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:07.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.759+0000 7f0c0207e640 1 -- 172.21.15.40:0/1592017548 shutdown_connections 2023-10-18T09:03:07.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:07.759+0000 7f0c0207e640 1 -- 172.21.15.40:0/1592017548 wait complete. 2023-10-18T09:03:07.781 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:03:08.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.006+0000 7f507ca75640 1 -- 172.21.15.40:0/3078823407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50781425a0 msgr2=0x7f50780c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:08.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.006+0000 7f507ca75640 1 --2- 172.21.15.40:0/3078823407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50781425a0 0x7f50780c9a50 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f50780ccea0 tx=0x7f506c02bd70 comp rx=0 tx=0).stop 2023-10-18T09:03:08.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.006+0000 7f507ca75640 1 -- 172.21.15.40:0/3078823407 shutdown_connections 2023-10-18T09:03:08.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.006+0000 7f507ca75640 1 --2- 172.21.15.40:0/3078823407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50781425a0 0x7f50780c9a50 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.006+0000 7f507ca75640 1 --2- 172.21.15.40:0/3078823407 >> v2:172.21.15.144:3300/0 conn(0x7f50781420f0 0x7f50780b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.006+0000 7f507ca75640 1 -- 172.21.15.40:0/3078823407 >> 172.21.15.40:0/3078823407 conn(0x7f5078069720 msgr2=0x7f5078069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:08.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.006+0000 7f507ca75640 1 -- 172.21.15.40:0/3078823407 shutdown_connections 2023-10-18T09:03:08.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.006+0000 7f507ca75640 1 -- 172.21.15.40:0/3078823407 wait complete. 2023-10-18T09:03:08.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.009+0000 7f507ca75640 1 Processor -- start 2023-10-18T09:03:08.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.009+0000 7f507ca75640 1 -- start start 2023-10-18T09:03:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.010+0000 7f507ca75640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50780c9f90 0x7f5078141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.010+0000 7f507ca75640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50781420f0 0x7f5078158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.010+0000 7f507ca75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50781425a0 0x7f507813f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.010+0000 7f507ca75640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5078140920 con 0x7f50781425a0 2023-10-18T09:03:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.010+0000 7f507ca75640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5078140050 con 0x7f50781420f0 2023-10-18T09:03:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.010+0000 7f507ca75640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5078140330 con 0x7f50780c9f90 2023-10-18T09:03:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.010+0000 7f5077fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50781425a0 0x7f507813f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.011+0000 7f5077fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50781425a0 0x7f507813f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60624/0 (socket says 172.21.15.40:60624) 2023-10-18T09:03:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.011+0000 7f5077fff640 1 -- 172.21.15.40:0/3971492720 learned_addr learned my addr 172.21.15.40:0/3971492720 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.011+0000 7f50777fe640 1 --2- 172.21.15.40:0/3971492720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50780c9f90 0x7f5078141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.011+0000 7f5076ffd640 1 --2- 172.21.15.40:0/3971492720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50781420f0 0x7f5078158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.011+0000 7f5077fff640 1 -- 172.21.15.40:0/3971492720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50780c9f90 msgr2=0x7f5078141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.011+0000 7f5077fff640 1 --2- 172.21.15.40:0/3971492720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50780c9f90 0x7f5078141370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.011+0000 7f5077fff640 1 -- 172.21.15.40:0/3971492720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50781420f0 msgr2=0x7f5078158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:08.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.011+0000 7f5077fff640 1 --2- 172.21.15.40:0/3971492720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50781420f0 0x7f5078158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.011+0000 7f5077fff640 1 -- 172.21.15.40:0/3971492720 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c009490 con 0x7f50781425a0 2023-10-18T09:03:08.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.011+0000 7f5076ffd640 1 --2- 172.21.15.40:0/3971492720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50781420f0 0x7f5078158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:08.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.011+0000 7f5077fff640 1 --2- 172.21.15.40:0/3971492720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50781425a0 0x7f507813f6e0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f507000cbc0 tx=0x7f507000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.012+0000 7f505ffff640 1 -- 172.21.15.40:0/3971492720 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507000ad40 con 0x7f50781425a0 2023-10-18T09:03:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.012+0000 7f505ffff640 1 -- 172.21.15.40:0/3971492720 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5070012420 con 0x7f50781425a0 2023-10-18T09:03:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.012+0000 7f505ffff640 1 -- 172.21.15.40:0/3971492720 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070012850 con 0x7f50781425a0 2023-10-18T09:03:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.012+0000 7f507ca75640 1 -- 172.21.15.40:0/3971492720 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5078140b70 con 0x7f50781425a0 2023-10-18T09:03:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.012+0000 7f507ca75640 1 -- 172.21.15.40:0/3971492720 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5078159180 con 0x7f50781425a0 2023-10-18T09:03:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.013+0000 7f507ca75640 1 -- 172.21.15.40:0/3971492720 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5078066e00 con 0x7f50781425a0 2023-10-18T09:03:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.014+0000 7f505ffff640 1 -- 172.21.15.40:0/3971492720 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f507001a3b0 con 0x7f50781425a0 2023-10-18T09:03:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.015+0000 7f505ffff640 1 --2- 172.21.15.40:0/3971492720 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f506406ae40 0x7f506406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.015+0000 7f505ffff640 1 -- 172.21.15.40:0/3971492720 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5070089ed0 con 0x7f50781425a0 2023-10-18T09:03:08.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.016+0000 7f50777fe640 1 --2- 172.21.15.40:0/3971492720 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f506406ae40 0x7f506406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:08.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.017+0000 7f50777fe640 1 --2- 172.21.15.40:0/3971492720 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f506406ae40 0x7f506406d300 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5068005e10 tx=0x7f5068005da0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:08.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.019+0000 7f505ffff640 1 -- 172.21.15.40:0/3971492720 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5070054f70 con 0x7f50781425a0 2023-10-18T09:03:08.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.256+0000 7f507ca75640 1 -- 172.21.15.40:0/3971492720 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f507815d980 con 0x7f50781425a0 2023-10-18T09:03:08.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.257+0000 7f505ffff640 1 -- 172.21.15.40:0/3971492720 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f5070016090 con 0x7f50781425a0 2023-10-18T09:03:08.259 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:08.259 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/700924196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":700924196},{"type":"v1","addr":"172.21.15.40:6833","nonce":700924196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/877414874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":877414874},{"type":"v1","addr":"172.21.15.144:6835","nonce":877414874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2596943574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2596943574},{"type":"v1","addr":"172.21.15.144:6837","nonce":2596943574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:02:41.786627+0000","modified":"2023-10-18T09:02:44.862740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7449":{"gid":7449,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/1801003551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1801003551},{"type":"v1","addr":"172.21.15.40:6835","nonce":1801003551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-18T09:03:08.260 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 57 2023-10-18T09:03:08.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.260+0000 7f507ca75640 1 -- 172.21.15.40:0/3971492720 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f506406ae40 msgr2=0x7f506406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:08.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.260+0000 7f507ca75640 1 --2- 172.21.15.40:0/3971492720 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f506406ae40 0x7f506406d300 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5068005e10 tx=0x7f5068005da0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.260+0000 7f507ca75640 1 -- 172.21.15.40:0/3971492720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50781425a0 msgr2=0x7f507813f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:08.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.260+0000 7f507ca75640 1 --2- 172.21.15.40:0/3971492720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50781425a0 0x7f507813f6e0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f507000cbc0 tx=0x7f507000a430 comp rx=0 tx=0).stop 2023-10-18T09:03:08.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.260+0000 7f507ca75640 1 -- 172.21.15.40:0/3971492720 shutdown_connections 2023-10-18T09:03:08.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.260+0000 7f507ca75640 1 --2- 172.21.15.40:0/3971492720 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f506406ae40 0x7f506406d300 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.260+0000 7f507ca75640 1 --2- 172.21.15.40:0/3971492720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50781425a0 0x7f507813f6e0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.260+0000 7f507ca75640 1 --2- 172.21.15.40:0/3971492720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50781420f0 0x7f5078158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.260+0000 7f507ca75640 1 --2- 172.21.15.40:0/3971492720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50780c9f90 0x7f5078141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.260+0000 7f507ca75640 1 -- 172.21.15.40:0/3971492720 >> 172.21.15.40:0/3971492720 conn(0x7f5078069720 msgr2=0x7f507813ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:08.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.260+0000 7f507ca75640 1 -- 172.21.15.40:0/3971492720 shutdown_connections 2023-10-18T09:03:08.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.260+0000 7f507ca75640 1 -- 172.21.15.40:0/3971492720 wait complete. 2023-10-18T09:03:08.282 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:03:08.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.508+0000 7f4a5c882640 1 -- 172.21.15.40:0/4250196951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a580b15b0 msgr2=0x7f4a580c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:08.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.508+0000 7f4a5c882640 1 --2- 172.21.15.40:0/4250196951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a580b15b0 0x7f4a580c99c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c005fd0 tx=0x7f4a4c02be20 comp rx=0 tx=0).stop 2023-10-18T09:03:08.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.508+0000 7f4a5c882640 1 -- 172.21.15.40:0/4250196951 shutdown_connections 2023-10-18T09:03:08.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.508+0000 7f4a5c882640 1 --2- 172.21.15.40:0/4250196951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a580c9f00 0x7f4a58157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.508+0000 7f4a5c882640 1 --2- 172.21.15.40:0/4250196951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a580b15b0 0x7f4a580c99c0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.508+0000 7f4a5c882640 1 -- 172.21.15.40:0/4250196951 >> 172.21.15.40:0/4250196951 conn(0x7f4a58069720 msgr2=0x7f4a58069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:08.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.508+0000 7f4a5c882640 1 -- 172.21.15.40:0/4250196951 shutdown_connections 2023-10-18T09:03:08.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.508+0000 7f4a5c882640 1 -- 172.21.15.40:0/4250196951 wait complete. 2023-10-18T09:03:08.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.510+0000 7f4a5c882640 1 Processor -- start 2023-10-18T09:03:08.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.510+0000 7f4a5c882640 1 -- start start 2023-10-18T09:03:08.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.511+0000 7f4a5c882640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a5806d6f0 0x7f4a5813a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:08.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.511+0000 7f4a5c882640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a580b15b0 0x7f4a581566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:08.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.511+0000 7f4a5c882640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a580c9f00 0x7f4a58156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:08.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.511+0000 7f4a5c882640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5813b360 con 0x7f4a5806d6f0 2023-10-18T09:03:08.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.511+0000 7f4a5c882640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5813b570 con 0x7f4a580c9f00 2023-10-18T09:03:08.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.511+0000 7f4a5c882640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5813b850 con 0x7f4a580b15b0 2023-10-18T09:03:08.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.511+0000 7f4a56d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a5806d6f0 0x7f4a5813a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:08.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.511+0000 7f4a56d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a5806d6f0 0x7f4a5813a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60640/0 (socket says 172.21.15.40:60640) 2023-10-18T09:03:08.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.511+0000 7f4a56d9d640 1 -- 172.21.15.40:0/3189955928 learned_addr learned my addr 172.21.15.40:0/3189955928 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.512+0000 7f4a56d9d640 1 -- 172.21.15.40:0/3189955928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a580b15b0 msgr2=0x7f4a581566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.512+0000 7f4a5659c640 1 --2- 172.21.15.40:0/3189955928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a580b15b0 0x7f4a581566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.512+0000 7f4a5759e640 1 --2- 172.21.15.40:0/3189955928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a580c9f00 0x7f4a58156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.512+0000 7f4a56d9d640 1 --2- 172.21.15.40:0/3189955928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a580b15b0 0x7f4a581566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.512+0000 7f4a56d9d640 1 -- 172.21.15.40:0/3189955928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a580c9f00 msgr2=0x7f4a58156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:08.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.512+0000 7f4a56d9d640 1 --2- 172.21.15.40:0/3189955928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a580c9f00 0x7f4a58156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.512+0000 7f4a56d9d640 1 -- 172.21.15.40:0/3189955928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c009560 con 0x7f4a5806d6f0 2023-10-18T09:03:08.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.512+0000 7f4a5759e640 1 --2- 172.21.15.40:0/3189955928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a580c9f00 0x7f4a58156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:08.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.512+0000 7f4a56d9d640 1 --2- 172.21.15.40:0/3189955928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a5806d6f0 0x7f4a5813a860 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800c4d0 tx=0x7f4a4800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:08.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.512+0000 7f4a3f7fe640 1 -- 172.21.15.40:0/3189955928 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a480052d0 con 0x7f4a5806d6f0 2023-10-18T09:03:08.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.512+0000 7f4a3f7fe640 1 -- 172.21.15.40:0/3189955928 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4a48005430 con 0x7f4a5806d6f0 2023-10-18T09:03:08.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.512+0000 7f4a5c882640 1 -- 172.21.15.40:0/3189955928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5813bb30 con 0x7f4a5806d6f0 2023-10-18T09:03:08.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.512+0000 7f4a3f7fe640 1 -- 172.21.15.40:0/3189955928 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a48012890 con 0x7f4a5806d6f0 2023-10-18T09:03:08.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.513+0000 7f4a5c882640 1 -- 172.21.15.40:0/3189955928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5813c0f0 con 0x7f4a5806d6f0 2023-10-18T09:03:08.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.514+0000 7f4a5c882640 1 -- 172.21.15.40:0/3189955928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a58066e00 con 0x7f4a5806d6f0 2023-10-18T09:03:08.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.515+0000 7f4a3f7fe640 1 -- 172.21.15.40:0/3189955928 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4a480035f0 con 0x7f4a5806d6f0 2023-10-18T09:03:08.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.516+0000 7f4a3f7fe640 1 --2- 172.21.15.40:0/3189955928 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a4406ae40 0x7f4a4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:08.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.516+0000 7f4a3f7fe640 1 -- 172.21.15.40:0/3189955928 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a480890f0 con 0x7f4a5806d6f0 2023-10-18T09:03:08.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.516+0000 7f4a5659c640 1 --2- 172.21.15.40:0/3189955928 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a4406ae40 0x7f4a4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:08.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.517+0000 7f4a5659c640 1 --2- 172.21.15.40:0/3189955928 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a4406ae40 0x7f4a4406d300 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c035000 tx=0x7f4a4c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:08.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.520+0000 7f4a3f7fe640 1 -- 172.21.15.40:0/3189955928 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a48054190 con 0x7f4a5806d6f0 2023-10-18T09:03:08.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.715+0000 7f4a5c882640 1 -- 172.21.15.40:0/3189955928 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a580ca350 con 0x7f4a5806d6f0 2023-10-18T09:03:08.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.718+0000 7f4a3f7fe640 1 -- 172.21.15.40:0/3189955928 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+16539 (secure 0 0 0) 0x7f4a580ca350 con 0x7f4a5806d6f0 2023-10-18T09:03:08.720 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:08.721 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":133,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:02:56.085956+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:02:39.531482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-18T09:02:40.812071+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:03:08.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.720+0000 7f4a5c882640 1 -- 172.21.15.40:0/3189955928 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a4406ae40 msgr2=0x7f4a4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:08.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.720+0000 7f4a5c882640 1 --2- 172.21.15.40:0/3189955928 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a4406ae40 0x7f4a4406d300 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c035000 tx=0x7f4a4c004730 comp rx=0 tx=0).stop 2023-10-18T09:03:08.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.720+0000 7f4a5c882640 1 -- 172.21.15.40:0/3189955928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a5806d6f0 msgr2=0x7f4a5813a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:08.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.720+0000 7f4a5c882640 1 --2- 172.21.15.40:0/3189955928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a5806d6f0 0x7f4a5813a860 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800c4d0 tx=0x7f4a4800c990 comp rx=0 tx=0).stop 2023-10-18T09:03:08.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.720+0000 7f4a5c882640 1 -- 172.21.15.40:0/3189955928 shutdown_connections 2023-10-18T09:03:08.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.721+0000 7f4a5c882640 1 --2- 172.21.15.40:0/3189955928 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4a4406ae40 0x7f4a4406d300 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.721+0000 7f4a5c882640 1 --2- 172.21.15.40:0/3189955928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a580c9f00 0x7f4a58156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.721+0000 7f4a5c882640 1 --2- 172.21.15.40:0/3189955928 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4a580b15b0 0x7f4a581566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.721+0000 7f4a5c882640 1 --2- 172.21.15.40:0/3189955928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a5806d6f0 0x7f4a5813a860 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.721+0000 7f4a5c882640 1 -- 172.21.15.40:0/3189955928 >> 172.21.15.40:0/3189955928 conn(0x7f4a58069720 msgr2=0x7f4a581573b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:08.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.721+0000 7f4a5c882640 1 -- 172.21.15.40:0/3189955928 shutdown_connections 2023-10-18T09:03:08.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.721+0000 7f4a5c882640 1 -- 172.21.15.40:0/3189955928 wait complete. 2023-10-18T09:03:08.743 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T09:03:08.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.970+0000 7f67b55d0640 1 -- 172.21.15.40:0/624079401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67b00ca160 msgr2=0x7f67b00cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:08.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.970+0000 7f67b55d0640 1 --2- 172.21.15.40:0/624079401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67b00ca160 0x7f67b00cafc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f67a8005fd0 tx=0x7f67a802be10 comp rx=0 tx=0).stop 2023-10-18T09:03:08.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.970+0000 7f67b55d0640 1 -- 172.21.15.40:0/624079401 shutdown_connections 2023-10-18T09:03:08.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.970+0000 7f67b55d0640 1 --2- 172.21.15.40:0/624079401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67b00ca160 0x7f67b00cafc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.970+0000 7f67b55d0640 1 --2- 172.21.15.40:0/624079401 >> v2:172.21.15.144:3300/0 conn(0x7f67b01420f0 0x7f67b00b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.970+0000 7f67b55d0640 1 -- 172.21.15.40:0/624079401 >> 172.21.15.40:0/624079401 conn(0x7f67b0069720 msgr2=0x7f67b0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:08.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.970+0000 7f67b55d0640 1 -- 172.21.15.40:0/624079401 shutdown_connections 2023-10-18T09:03:08.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.970+0000 7f67b55d0640 1 -- 172.21.15.40:0/624079401 wait complete. 2023-10-18T09:03:08.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.972+0000 7f67b55d0640 1 Processor -- start 2023-10-18T09:03:08.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.972+0000 7f67b55d0640 1 -- start start 2023-10-18T09:03:08.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.973+0000 7f67b55d0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b00ca160 0x7f67b00710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:08.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.973+0000 7f67b55d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67b01420f0 0x7f67b015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:08.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.973+0000 7f67b55d0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67b01425a0 0x7f67b006f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:08.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.973+0000 7f67b55d0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b0071630 con 0x7f67b00ca160 2023-10-18T09:03:08.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.973+0000 7f67b55d0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b0070050 con 0x7f67b01420f0 2023-10-18T09:03:08.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.973+0000 7f67b55d0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b0070330 con 0x7f67b01425a0 2023-10-18T09:03:08.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.973+0000 7f67affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b00ca160 0x7f67b00710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:08.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.973+0000 7f67affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b00ca160 0x7f67b00710f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60654/0 (socket says 172.21.15.40:60654) 2023-10-18T09:03:08.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.973+0000 7f67affff640 1 -- 172.21.15.40:0/144827247 learned_addr learned my addr 172.21.15.40:0/144827247 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:08.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.974+0000 7f67af7fe640 1 --2- 172.21.15.40:0/144827247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67b01420f0 0x7f67b015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:08.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.974+0000 7f67affff640 1 -- 172.21.15.40:0/144827247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67b01425a0 msgr2=0x7f67b006f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:08.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.974+0000 7f67affff640 1 --2- 172.21.15.40:0/144827247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67b01425a0 0x7f67b006f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.974+0000 7f67affff640 1 -- 172.21.15.40:0/144827247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67b01420f0 msgr2=0x7f67b015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:08.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.974+0000 7f67affff640 1 --2- 172.21.15.40:0/144827247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67b01420f0 0x7f67b015b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:08.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.974+0000 7f67affff640 1 -- 172.21.15.40:0/144827247 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a8009560 con 0x7f67b00ca160 2023-10-18T09:03:08.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.974+0000 7f67affff640 1 --2- 172.21.15.40:0/144827247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b00ca160 0x7f67b00710f0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f67a000ebc0 tx=0x7f67a000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:08.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.974+0000 7f67acff9640 1 -- 172.21.15.40:0/144827247 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a00099b0 con 0x7f67b00ca160 2023-10-18T09:03:08.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.974+0000 7f67acff9640 1 -- 172.21.15.40:0/144827247 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f67a0010720 con 0x7f67b00ca160 2023-10-18T09:03:08.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.974+0000 7f67acff9640 1 -- 172.21.15.40:0/144827247 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0010b30 con 0x7f67b00ca160 2023-10-18T09:03:08.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.974+0000 7f67b55d0640 1 -- 172.21.15.40:0/144827247 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b0070610 con 0x7f67b00ca160 2023-10-18T09:03:08.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.975+0000 7f67b55d0640 1 -- 172.21.15.40:0/144827247 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b0070bd0 con 0x7f67b00ca160 2023-10-18T09:03:08.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.976+0000 7f67b55d0640 1 -- 172.21.15.40:0/144827247 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b0070d90 con 0x7f67b00ca160 2023-10-18T09:03:08.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.977+0000 7f67acff9640 1 -- 172.21.15.40:0/144827247 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f67a0010c90 con 0x7f67b00ca160 2023-10-18T09:03:08.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.977+0000 7f67acff9640 1 --2- 172.21.15.40:0/144827247 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f679006ae40 0x7f679006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:08.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.978+0000 7f67acff9640 1 -- 172.21.15.40:0/144827247 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67a0014070 con 0x7f67b00ca160 2023-10-18T09:03:08.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.978+0000 7f67af7fe640 1 --2- 172.21.15.40:0/144827247 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f679006ae40 0x7f679006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:08.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.979+0000 7f67af7fe640 1 --2- 172.21.15.40:0/144827247 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f679006ae40 0x7f679006d300 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f67b0137820 tx=0x7f6798009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:08.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:08.981+0000 7f67acff9640 1 -- 172.21.15.40:0/144827247 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f67a0051ca0 con 0x7f67b00ca160 2023-10-18T09:03:09.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.215+0000 7f67b55d0640 1 -- 172.21.15.40:0/144827247 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f67b0066e00 con 0x7f67b00ca160 2023-10-18T09:03:09.219 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:03:09.217+0000 7f6a0f710640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T09:03:09.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.221+0000 7f67acff9640 1 -- 172.21.15.40:0/144827247 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7f67a0088e60 con 0x7f67b00ca160 2023-10-18T09:03:09.223 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T09:03:09.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.226+0000 7f67b55d0640 1 -- 172.21.15.40:0/144827247 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f679006ae40 msgr2=0x7f679006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:09.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.226+0000 7f67b55d0640 1 --2- 172.21.15.40:0/144827247 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f679006ae40 0x7f679006d300 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f67b0137820 tx=0x7f6798009040 comp rx=0 tx=0).stop 2023-10-18T09:03:09.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.226+0000 7f67b55d0640 1 -- 172.21.15.40:0/144827247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b00ca160 msgr2=0x7f67b00710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:09.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.226+0000 7f67b55d0640 1 --2- 172.21.15.40:0/144827247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b00ca160 0x7f67b00710f0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f67a000ebc0 tx=0x7f67a000c6a0 comp rx=0 tx=0).stop 2023-10-18T09:03:09.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.227+0000 7f67b55d0640 1 -- 172.21.15.40:0/144827247 shutdown_connections 2023-10-18T09:03:09.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.227+0000 7f67b55d0640 1 --2- 172.21.15.40:0/144827247 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f679006ae40 0x7f679006d300 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:09.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.227+0000 7f67b55d0640 1 --2- 172.21.15.40:0/144827247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67b01425a0 0x7f67b006f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:09.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.227+0000 7f67b55d0640 1 --2- 172.21.15.40:0/144827247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67b01420f0 0x7f67b015b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:09.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.227+0000 7f67b55d0640 1 --2- 172.21.15.40:0/144827247 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b00ca160 0x7f67b00710f0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:09.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.227+0000 7f67b55d0640 1 -- 172.21.15.40:0/144827247 >> 172.21.15.40:0/144827247 conn(0x7f67b0069720 msgr2=0x7f67b0159a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:09.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.227+0000 7f67b55d0640 1 -- 172.21.15.40:0/144827247 shutdown_connections 2023-10-18T09:03:09.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.227+0000 7f67b55d0640 1 -- 172.21.15.40:0/144827247 wait complete. 2023-10-18T09:03:09.252 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T09:03:09.258 INFO:tasks.ceph.mds.a.smithi040.stderr: -4877> 2023-10-18T09:02:46.265+0000 7f9d169b8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T09:03:09.320 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-10-18T09:03:09.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.455+0000 7fabfb116640 1 -- 172.21.15.40:0/2004454875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabf413c550 msgr2=0x7fabf413ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.455+0000 7fabfb116640 1 --2- 172.21.15.40:0/2004454875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabf413c550 0x7fabf413ec50 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fabe8005fd0 tx=0x7fabe802be10 comp rx=0 tx=0).stop 2023-10-18T09:03:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.455+0000 7fabfb116640 1 -- 172.21.15.40:0/2004454875 shutdown_connections 2023-10-18T09:03:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.455+0000 7fabfb116640 1 --2- 172.21.15.40:0/2004454875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabf413f190 0x7fabf4070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:09.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.455+0000 7fabfb116640 1 --2- 172.21.15.40:0/2004454875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabf413c550 0x7fabf413ec50 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:09.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.455+0000 7fabfb116640 1 -- 172.21.15.40:0/2004454875 >> 172.21.15.40:0/2004454875 conn(0x7fabf4069720 msgr2=0x7fabf4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:09.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.455+0000 7fabfb116640 1 -- 172.21.15.40:0/2004454875 shutdown_connections 2023-10-18T09:03:09.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.455+0000 7fabfb116640 1 -- 172.21.15.40:0/2004454875 wait complete. 2023-10-18T09:03:09.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.457+0000 7fabfb116640 1 Processor -- start 2023-10-18T09:03:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.457+0000 7fabfb116640 1 -- start start 2023-10-18T09:03:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.458+0000 7fabfb116640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabf4071240 0x7fabf41e4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.458+0000 7fabfb116640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabf413c550 0x7fabf4158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.458+0000 7fabfb116640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf413f190 0x7fabf413a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.458+0000 7fabfb116640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf41e3bb0 con 0x7fabf4071240 2023-10-18T09:03:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.458+0000 7fabfb116640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf41e4b40 con 0x7fabf413f190 2023-10-18T09:03:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.458+0000 7fabfb116640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf41e2f30 con 0x7fabf413c550 2023-10-18T09:03:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.458+0000 7fabf8eb2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabf4071240 0x7fabf41e4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.458+0000 7fabf8eb2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabf4071240 0x7fabf41e4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60676/0 (socket says 172.21.15.40:60676) 2023-10-18T09:03:09.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.458+0000 7fabf8eb2640 1 -- 172.21.15.40:0/201276846 learned_addr learned my addr 172.21.15.40:0/201276846 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:09.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.459+0000 7fabf8eb2640 1 -- 172.21.15.40:0/201276846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabf413c550 msgr2=0x7fabf4158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:09.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.459+0000 7fabf8eb2640 1 --2- 172.21.15.40:0/201276846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabf413c550 0x7fabf4158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.459+0000 7fabf8eb2640 1 -- 172.21.15.40:0/201276846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf413f190 msgr2=0x7fabf413a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.459+0000 7fabf8eb2640 1 --2- 172.21.15.40:0/201276846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf413f190 0x7fabf413a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.459+0000 7fabf8eb2640 1 -- 172.21.15.40:0/201276846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe8009560 con 0x7fabf4071240 2023-10-18T09:03:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.459+0000 7fabf8eb2640 1 --2- 172.21.15.40:0/201276846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabf4071240 0x7fabf41e4600 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009850 tx=0x7fabe802db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.460+0000 7fabf17fa640 1 -- 172.21.15.40:0/201276846 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe80035f0 con 0x7fabf4071240 2023-10-18T09:03:09.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.460+0000 7fabf17fa640 1 -- 172.21.15.40:0/201276846 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fabe8003840 con 0x7fabf4071240 2023-10-18T09:03:09.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.460+0000 7fabfb116640 1 -- 172.21.15.40:0/201276846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf41e3210 con 0x7fabf4071240 2023-10-18T09:03:09.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.460+0000 7fabf17fa640 1 -- 172.21.15.40:0/201276846 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8003bf0 con 0x7fabf4071240 2023-10-18T09:03:09.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.460+0000 7fabfb116640 1 -- 172.21.15.40:0/201276846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf41e37d0 con 0x7fabf4071240 2023-10-18T09:03:09.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.461+0000 7fabfb116640 1 -- 172.21.15.40:0/201276846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf413bf60 con 0x7fabf4071240 2023-10-18T09:03:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.462+0000 7fabf17fa640 1 -- 172.21.15.40:0/201276846 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fabe8003d50 con 0x7fabf4071240 2023-10-18T09:03:09.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.464+0000 7fabf17fa640 1 --2- 172.21.15.40:0/201276846 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fabd806ae40 0x7fabd806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:09.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.464+0000 7fabf17fa640 1 -- 172.21.15.40:0/201276846 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fabe80ac340 con 0x7fabf4071240 2023-10-18T09:03:09.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.464+0000 7fabf3fff640 1 --2- 172.21.15.40:0/201276846 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fabd806ae40 0x7fabd806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:09.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.466+0000 7fabf3fff640 1 --2- 172.21.15.40:0/201276846 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fabd806ae40 0x7fabd806d300 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fabf40aebd0 tx=0x7fabe0005b50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:09.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.468+0000 7fabf17fa640 1 -- 172.21.15.40:0/201276846 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fabe80774a0 con 0x7fabf4071240 2023-10-18T09:03:09.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:09.698+0000 7fabfb116640 1 -- 172.21.15.40:0/201276846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fabf4066fd0 con 0x7fabf4071240 2023-10-18T09:03:10.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.224+0000 7fabf17fa640 1 -- 172.21.15.40:0/201276846 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7fabe807aac0 con 0x7fabf4071240 2023-10-18T09:03:10.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.228+0000 7fabfb116640 1 -- 172.21.15.40:0/201276846 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fabd806ae40 msgr2=0x7fabd806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:10.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.228+0000 7fabfb116640 1 --2- 172.21.15.40:0/201276846 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fabd806ae40 0x7fabd806d300 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fabf40aebd0 tx=0x7fabe0005b50 comp rx=0 tx=0).stop 2023-10-18T09:03:10.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.228+0000 7fabfb116640 1 -- 172.21.15.40:0/201276846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabf4071240 msgr2=0x7fabf41e4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:10.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.228+0000 7fabfb116640 1 --2- 172.21.15.40:0/201276846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabf4071240 0x7fabf41e4600 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009850 tx=0x7fabe802db90 comp rx=0 tx=0).stop 2023-10-18T09:03:10.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.228+0000 7fabfb116640 1 -- 172.21.15.40:0/201276846 shutdown_connections 2023-10-18T09:03:10.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.228+0000 7fabfb116640 1 --2- 172.21.15.40:0/201276846 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fabd806ae40 0x7fabd806d300 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:10.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.228+0000 7fabfb116640 1 --2- 172.21.15.40:0/201276846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf413f190 0x7fabf413a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:10.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.228+0000 7fabfb116640 1 --2- 172.21.15.40:0/201276846 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabf413c550 0x7fabf4158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:10.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.228+0000 7fabfb116640 1 --2- 172.21.15.40:0/201276846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabf4071240 0x7fabf41e4600 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:10.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.228+0000 7fabfb116640 1 -- 172.21.15.40:0/201276846 >> 172.21.15.40:0/201276846 conn(0x7fabf4069720 msgr2=0x7fabf4157d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:10.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.228+0000 7fabfb116640 1 -- 172.21.15.40:0/201276846 shutdown_connections 2023-10-18T09:03:10.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.228+0000 7fabfb116640 1 -- 172.21.15.40:0/201276846 wait complete. 2023-10-18T09:03:10.252 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T09:03:10.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.475+0000 7fc82e765640 1 -- 172.21.15.40:0/445227285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc82813eaa0 msgr2=0x7fc82813eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:10.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.475+0000 7fc82e765640 1 --2- 172.21.15.40:0/445227285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc82813eaa0 0x7fc82813eef0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fc82815a870 tx=0x7fc82002c260 comp rx=0 tx=0).stop 2023-10-18T09:03:10.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.475+0000 7fc82e765640 1 -- 172.21.15.40:0/445227285 shutdown_connections 2023-10-18T09:03:10.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.475+0000 7fc82e765640 1 --2- 172.21.15.40:0/445227285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc82813eaa0 0x7fc82813eef0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:10.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.475+0000 7fc82e765640 1 --2- 172.21.15.40:0/445227285 >> v2:172.21.15.144:3300/0 conn(0x7fc8280c9f40 0x7fc8280cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:10.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.475+0000 7fc82e765640 1 -- 172.21.15.40:0/445227285 >> 172.21.15.40:0/445227285 conn(0x7fc8280aeb90 msgr2=0x7fc8280aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:10.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.475+0000 7fc82e765640 1 -- 172.21.15.40:0/445227285 shutdown_connections 2023-10-18T09:03:10.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.475+0000 7fc82e765640 1 -- 172.21.15.40:0/445227285 wait complete. 2023-10-18T09:03:10.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.477+0000 7fc82e765640 1 Processor -- start 2023-10-18T09:03:10.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.477+0000 7fc82e765640 1 -- start start 2023-10-18T09:03:10.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.478+0000 7fc82e765640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8280c9f40 0x7fc8281f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:10.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.478+0000 7fc82e765640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc82813c220 0x7fc8280afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.478+0000 7fc82e765640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc82813eaa0 0x7fc828153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.478+0000 7fc82e765640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc828153aa0 con 0x7fc8280c9f40 2023-10-18T09:03:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.478+0000 7fc82e765640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280b0c30 con 0x7fc82813eaa0 2023-10-18T09:03:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.478+0000 7fc82e765640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280b0390 con 0x7fc82813c220 2023-10-18T09:03:10.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.479+0000 7fc82d763640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8280c9f40 0x7fc8281f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.479+0000 7fc82d763640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8280c9f40 0x7fc8281f2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60692/0 (socket says 172.21.15.40:60692) 2023-10-18T09:03:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.479+0000 7fc82d763640 1 -- 172.21.15.40:0/761107452 learned_addr learned my addr 172.21.15.40:0/761107452 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.479+0000 7fc82cf62640 1 --2- 172.21.15.40:0/761107452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc82813c220 0x7fc8280afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.479+0000 7fc82df64640 1 --2- 172.21.15.40:0/761107452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc82813eaa0 0x7fc828153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:10.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.479+0000 7fc82d763640 1 -- 172.21.15.40:0/761107452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc82813c220 msgr2=0x7fc8280afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.479+0000 7fc82d763640 1 --2- 172.21.15.40:0/761107452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc82813c220 0x7fc8280afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.479+0000 7fc82d763640 1 -- 172.21.15.40:0/761107452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc82813eaa0 msgr2=0x7fc828153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.479+0000 7fc82d763640 1 --2- 172.21.15.40:0/761107452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc82813eaa0 0x7fc828153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:10.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.479+0000 7fc82d763640 1 -- 172.21.15.40:0/761107452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc820009900 con 0x7fc8280c9f40 2023-10-18T09:03:10.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.480+0000 7fc82d763640 1 --2- 172.21.15.40:0/761107452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8280c9f40 0x7fc8281f2950 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fc82400c410 tx=0x7fc82400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:10.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.480+0000 7fc81dffb640 1 -- 172.21.15.40:0/761107452 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8240035f0 con 0x7fc8280c9f40 2023-10-18T09:03:10.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.480+0000 7fc81dffb640 1 -- 172.21.15.40:0/761107452 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc824003750 con 0x7fc8280c9f40 2023-10-18T09:03:10.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.480+0000 7fc81dffb640 1 -- 172.21.15.40:0/761107452 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824003b60 con 0x7fc8280c9f40 2023-10-18T09:03:10.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.480+0000 7fc82e765640 1 -- 172.21.15.40:0/761107452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8280b0670 con 0x7fc8280c9f40 2023-10-18T09:03:10.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.480+0000 7fc82e765640 1 -- 172.21.15.40:0/761107452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8281f3050 con 0x7fc8280c9f40 2023-10-18T09:03:10.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.481+0000 7fc82e765640 1 -- 172.21.15.40:0/761107452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc828073570 con 0x7fc8280c9f40 2023-10-18T09:03:10.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.482+0000 7fc81dffb640 1 -- 172.21.15.40:0/761107452 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc824003cc0 con 0x7fc8280c9f40 2023-10-18T09:03:10.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.483+0000 7fc81dffb640 1 --2- 172.21.15.40:0/761107452 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc80c06ae40 0x7fc80c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:10.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.483+0000 7fc81dffb640 1 -- 172.21.15.40:0/761107452 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc824088a40 con 0x7fc8280c9f40 2023-10-18T09:03:10.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.484+0000 7fc82cf62640 1 --2- 172.21.15.40:0/761107452 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc80c06ae40 0x7fc80c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:10.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.485+0000 7fc82cf62640 1 --2- 172.21.15.40:0/761107452 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc80c06ae40 0x7fc80c06d300 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc828073270 tx=0x7fc818008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:10.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.488+0000 7fc81dffb640 1 -- 172.21.15.40:0/761107452 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc824053a80 con 0x7fc8280c9f40 2023-10-18T09:03:10.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:10.681+0000 7fc82e765640 1 -- 172.21.15.40:0/761107452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc828066e00 con 0x7fc8280c9f40 2023-10-18T09:03:11.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.227+0000 7fc81dffb640 1 -- 172.21.15.40:0/761107452 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v135) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc8240170d0 con 0x7fc8280c9f40 2023-10-18T09:03:11.230 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-10-18T09:03:11.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.232+0000 7fc82e765640 1 -- 172.21.15.40:0/761107452 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc80c06ae40 msgr2=0x7fc80c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:11.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.232+0000 7fc82e765640 1 --2- 172.21.15.40:0/761107452 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc80c06ae40 0x7fc80c06d300 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc828073270 tx=0x7fc818008040 comp rx=0 tx=0).stop 2023-10-18T09:03:11.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.232+0000 7fc82e765640 1 -- 172.21.15.40:0/761107452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8280c9f40 msgr2=0x7fc8281f2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:11.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.232+0000 7fc82e765640 1 --2- 172.21.15.40:0/761107452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8280c9f40 0x7fc8281f2950 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fc82400c410 tx=0x7fc82400c8d0 comp rx=0 tx=0).stop 2023-10-18T09:03:11.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.232+0000 7fc82e765640 1 -- 172.21.15.40:0/761107452 shutdown_connections 2023-10-18T09:03:11.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.232+0000 7fc82e765640 1 --2- 172.21.15.40:0/761107452 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc80c06ae40 0x7fc80c06d300 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:11.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.232+0000 7fc82e765640 1 --2- 172.21.15.40:0/761107452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc82813eaa0 0x7fc828153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:11.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.232+0000 7fc82e765640 1 --2- 172.21.15.40:0/761107452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc82813c220 0x7fc8280afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:11.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.232+0000 7fc82e765640 1 --2- 172.21.15.40:0/761107452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8280c9f40 0x7fc8281f2950 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:11.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.232+0000 7fc82e765640 1 -- 172.21.15.40:0/761107452 >> 172.21.15.40:0/761107452 conn(0x7fc8280aeb90 msgr2=0x7fc82813c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:11.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.233+0000 7fc82e765640 1 -- 172.21.15.40:0/761107452 shutdown_connections 2023-10-18T09:03:11.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.233+0000 7fc82e765640 1 -- 172.21.15.40:0/761107452 wait complete. 2023-10-18T09:03:11.256 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T09:03:11.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.486+0000 7f80a7fff640 1 -- 172.21.15.40:0/1034838059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80a80c8a50 msgr2=0x7f80a80cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:11.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.486+0000 7f80a7fff640 1 --2- 172.21.15.40:0/1034838059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80a80c8a50 0x7f80a80cdb20 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009860 tx=0x7f80a002bd40 comp rx=0 tx=0).stop 2023-10-18T09:03:11.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.486+0000 7f80a7fff640 1 -- 172.21.15.40:0/1034838059 shutdown_connections 2023-10-18T09:03:11.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.486+0000 7f80a7fff640 1 --2- 172.21.15.40:0/1034838059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80a80c8a50 0x7f80a80cdb20 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:11.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.486+0000 7f80a7fff640 1 --2- 172.21.15.40:0/1034838059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a80b20a0 0x7f80a80ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:11.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.486+0000 7f80a7fff640 1 -- 172.21.15.40:0/1034838059 >> 172.21.15.40:0/1034838059 conn(0x7f80a809a5b0 msgr2=0x7f80a80d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:11.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.486+0000 7f80a7fff640 1 -- 172.21.15.40:0/1034838059 shutdown_connections 2023-10-18T09:03:11.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.486+0000 7f80a7fff640 1 -- 172.21.15.40:0/1034838059 wait complete. 2023-10-18T09:03:11.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.488+0000 7f80a7fff640 1 Processor -- start 2023-10-18T09:03:11.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.488+0000 7f80a7fff640 1 -- start start 2023-10-18T09:03:11.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.489+0000 7f80a7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80a80b20a0 0x7f80a81f1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:11.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.489+0000 7f80a7fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a80c8a50 0x7f80a80c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:11.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.489+0000 7f80a7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a815ebe0 0x7f80a815d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:11.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.489+0000 7f80a7fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a81f1c90 con 0x7f80a80b20a0 2023-10-18T09:03:11.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.489+0000 7f80a7fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a80c7190 con 0x7f80a815ebe0 2023-10-18T09:03:11.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.489+0000 7f80a7fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a80c7470 con 0x7f80a80c8a50 2023-10-18T09:03:11.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.489+0000 7f80a6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80a80b20a0 0x7f80a81f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:11.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.489+0000 7f80a6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80a80b20a0 0x7f80a81f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60712/0 (socket says 172.21.15.40:60712) 2023-10-18T09:03:11.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.489+0000 7f80a6ffd640 1 -- 172.21.15.40:0/1505687865 learned_addr learned my addr 172.21.15.40:0/1505687865 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:11.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.489+0000 7f80a67fc640 1 --2- 172.21.15.40:0/1505687865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a80c8a50 0x7f80a80c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:11.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.489+0000 7f80a77fe640 1 --2- 172.21.15.40:0/1505687865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a815ebe0 0x7f80a815d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:11.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.489+0000 7f80a6ffd640 1 -- 172.21.15.40:0/1505687865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a80c8a50 msgr2=0x7f80a80c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:11.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.489+0000 7f80a6ffd640 1 --2- 172.21.15.40:0/1505687865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a80c8a50 0x7f80a80c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:11.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.489+0000 7f80a6ffd640 1 -- 172.21.15.40:0/1505687865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a815ebe0 msgr2=0x7f80a815d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:11.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.489+0000 7f80a6ffd640 1 --2- 172.21.15.40:0/1505687865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a815ebe0 0x7f80a815d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:11.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.490+0000 7f80a6ffd640 1 -- 172.21.15.40:0/1505687865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a0009490 con 0x7f80a80b20a0 2023-10-18T09:03:11.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.490+0000 7f80a77fe640 1 --2- 172.21.15.40:0/1505687865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a815ebe0 0x7f80a815d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:11.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.490+0000 7f80a6ffd640 1 --2- 172.21.15.40:0/1505687865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80a80b20a0 0x7f80a81f1750 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f809800cb50 tx=0x7f809800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:11.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.490+0000 7f809ffff640 1 -- 172.21.15.40:0/1505687865 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809800ad40 con 0x7f80a80b20a0 2023-10-18T09:03:11.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.490+0000 7f809ffff640 1 -- 172.21.15.40:0/1505687865 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8098012420 con 0x7f80a80b20a0 2023-10-18T09:03:11.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.490+0000 7f80a7fff640 1 -- 172.21.15.40:0/1505687865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a80c7750 con 0x7f80a80b20a0 2023-10-18T09:03:11.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.490+0000 7f809ffff640 1 -- 172.21.15.40:0/1505687865 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098012870 con 0x7f80a80b20a0 2023-10-18T09:03:11.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.490+0000 7f80a7fff640 1 -- 172.21.15.40:0/1505687865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a80b4c80 con 0x7f80a80b20a0 2023-10-18T09:03:11.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.491+0000 7f80a7fff640 1 -- 172.21.15.40:0/1505687865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a8071da0 con 0x7f80a80b20a0 2023-10-18T09:03:11.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.493+0000 7f809ffff640 1 -- 172.21.15.40:0/1505687865 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f809801a3b0 con 0x7f80a80b20a0 2023-10-18T09:03:11.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.493+0000 7f809ffff640 1 --2- 172.21.15.40:0/1505687865 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f808806ae40 0x7f808806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:11.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.494+0000 7f809ffff640 1 -- 172.21.15.40:0/1505687865 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f8098089d00 con 0x7f80a80b20a0 2023-10-18T09:03:11.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.494+0000 7f80a67fc640 1 --2- 172.21.15.40:0/1505687865 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f808806ae40 0x7f808806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:11.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.495+0000 7f80a67fc640 1 --2- 172.21.15.40:0/1505687865 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f808806ae40 0x7f808806d300 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f80a81f52a0 tx=0x7f809000a4c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:11.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.498+0000 7f809ffff640 1 -- 172.21.15.40:0/1505687865 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8098054f10 con 0x7f80a80b20a0 2023-10-18T09:03:11.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:11.691+0000 7f80a7fff640 1 -- 172.21.15.40:0/1505687865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f80a80c8090 con 0x7f80a80b20a0 2023-10-18T09:03:12.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.238+0000 7f809ffff640 1 -- 172.21.15.40:0/1505687865 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v136) v1 ==== 169+0+0 (secure 0 0 0) 0x7f80980160d0 con 0x7f80a80b20a0 2023-10-18T09:03:12.241 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-10-18T09:03:12.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.244+0000 7f80a7fff640 1 -- 172.21.15.40:0/1505687865 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f808806ae40 msgr2=0x7f808806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:12.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.244+0000 7f80a7fff640 1 --2- 172.21.15.40:0/1505687865 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f808806ae40 0x7f808806d300 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f80a81f52a0 tx=0x7f809000a4c0 comp rx=0 tx=0).stop 2023-10-18T09:03:12.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.244+0000 7f80a7fff640 1 -- 172.21.15.40:0/1505687865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80a80b20a0 msgr2=0x7f80a81f1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:12.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.244+0000 7f80a7fff640 1 --2- 172.21.15.40:0/1505687865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80a80b20a0 0x7f80a81f1750 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f809800cb50 tx=0x7f809800a430 comp rx=0 tx=0).stop 2023-10-18T09:03:12.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.244+0000 7f80a7fff640 1 -- 172.21.15.40:0/1505687865 shutdown_connections 2023-10-18T09:03:12.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.244+0000 7f80a7fff640 1 --2- 172.21.15.40:0/1505687865 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f808806ae40 0x7f808806d300 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:12.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.244+0000 7f80a7fff640 1 --2- 172.21.15.40:0/1505687865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80a815ebe0 0x7f80a815d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:12.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.244+0000 7f80a7fff640 1 --2- 172.21.15.40:0/1505687865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f80a80c8a50 0x7f80a80c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:12.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.244+0000 7f80a7fff640 1 --2- 172.21.15.40:0/1505687865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80a80b20a0 0x7f80a81f1750 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:12.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.244+0000 7f80a7fff640 1 -- 172.21.15.40:0/1505687865 >> 172.21.15.40:0/1505687865 conn(0x7f80a809a5b0 msgr2=0x7f80a8155770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:12.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.245+0000 7f80a7fff640 1 -- 172.21.15.40:0/1505687865 shutdown_connections 2023-10-18T09:03:12.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.245+0000 7f80a7fff640 1 -- 172.21.15.40:0/1505687865 wait complete. 2023-10-18T09:03:12.269 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 22 2023-10-18T09:03:12.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.502+0000 7f510bc44640 1 -- 172.21.15.40:0/2723932312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5104144840 msgr2=0x7f510415f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.502+0000 7f510bc44640 1 --2- 172.21.15.40:0/2723932312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5104144840 0x7f510415f0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5100005fd0 tx=0x7f510002bd90 comp rx=0 tx=0).stop 2023-10-18T09:03:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.502+0000 7f510bc44640 1 -- 172.21.15.40:0/2723932312 shutdown_connections 2023-10-18T09:03:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.502+0000 7f510bc44640 1 --2- 172.21.15.40:0/2723932312 >> v2:172.21.15.144:3300/0 conn(0x7f510406d1c0 0x7f5104069720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.502+0000 7f510bc44640 1 --2- 172.21.15.40:0/2723932312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5104144840 0x7f510415f0d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:12.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.502+0000 7f510bc44640 1 -- 172.21.15.40:0/2723932312 >> 172.21.15.40:0/2723932312 conn(0x7f51040aff00 msgr2=0x7f51040b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:12.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.502+0000 7f510bc44640 1 -- 172.21.15.40:0/2723932312 shutdown_connections 2023-10-18T09:03:12.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.502+0000 7f510bc44640 1 -- 172.21.15.40:0/2723932312 wait complete. 2023-10-18T09:03:12.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.504+0000 7f510bc44640 1 Processor -- start 2023-10-18T09:03:12.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.504+0000 7f510bc44640 1 -- start start 2023-10-18T09:03:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.505+0000 7f510bc44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5104069c60 0x7f51041f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.505+0000 7f510bc44640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f510406d1c0 0x7f51040cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.505+0000 7f510bc44640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5104144840 0x7f51040cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.505+0000 7f510bc44640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51040c4190 con 0x7f5104144840 2023-10-18T09:03:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.505+0000 7f510bc44640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5104141470 con 0x7f5104069c60 2023-10-18T09:03:12.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.505+0000 7f510bc44640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51040c3b10 con 0x7f510406d1c0 2023-10-18T09:03:12.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.505+0000 7f510a1e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5104144840 0x7f51040cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:12.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.505+0000 7f510a1e1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5104144840 0x7f51040cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60716/0 (socket says 172.21.15.40:60716) 2023-10-18T09:03:12.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.505+0000 7f510a1e1640 1 -- 172.21.15.40:0/4106293556 learned_addr learned my addr 172.21.15.40:0/4106293556 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.506+0000 7f51099e0640 1 --2- 172.21.15.40:0/4106293556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5104069c60 0x7f51041f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.506+0000 7f51091df640 1 --2- 172.21.15.40:0/4106293556 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f510406d1c0 0x7f51040cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.506+0000 7f510a1e1640 1 -- 172.21.15.40:0/4106293556 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f510406d1c0 msgr2=0x7f51040cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.506+0000 7f510a1e1640 1 --2- 172.21.15.40:0/4106293556 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f510406d1c0 0x7f51040cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.506+0000 7f510a1e1640 1 -- 172.21.15.40:0/4106293556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5104069c60 msgr2=0x7f51041f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.506+0000 7f510a1e1640 1 --2- 172.21.15.40:0/4106293556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5104069c60 0x7f51041f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.506+0000 7f510a1e1640 1 -- 172.21.15.40:0/4106293556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5100009560 con 0x7f5104144840 2023-10-18T09:03:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.506+0000 7f51099e0640 1 --2- 172.21.15.40:0/4106293556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5104069c60 0x7f51041f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:03:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.506+0000 7f51091df640 1 --2- 172.21.15.40:0/4106293556 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f510406d1c0 0x7f51040cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.506+0000 7f510a1e1640 1 --2- 172.21.15.40:0/4106293556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5104144840 0x7f51040cb950 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00c7a0 tx=0x7f50fc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.507+0000 7f50fa7fc640 1 -- 172.21.15.40:0/4106293556 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc003b70 con 0x7f5104144840 2023-10-18T09:03:12.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.507+0000 7f50fa7fc640 1 -- 172.21.15.40:0/4106293556 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f50fc003cf0 con 0x7f5104144840 2023-10-18T09:03:12.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.507+0000 7f50fa7fc640 1 -- 172.21.15.40:0/4106293556 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc003ea0 con 0x7f5104144840 2023-10-18T09:03:12.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.507+0000 7f510bc44640 1 -- 172.21.15.40:0/4106293556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51041f61f0 con 0x7f5104144840 2023-10-18T09:03:12.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.507+0000 7f510bc44640 1 -- 172.21.15.40:0/4106293556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51040c3780 con 0x7f5104144840 2023-10-18T09:03:12.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.508+0000 7f510bc44640 1 -- 172.21.15.40:0/4106293556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5104066fd0 con 0x7f5104144840 2023-10-18T09:03:12.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.509+0000 7f50fa7fc640 1 -- 172.21.15.40:0/4106293556 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f50fc00aab0 con 0x7f5104144840 2023-10-18T09:03:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.510+0000 7f50fa7fc640 1 --2- 172.21.15.40:0/4106293556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f50e806ae40 0x7f50e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.510+0000 7f50fa7fc640 1 -- 172.21.15.40:0/4106293556 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f50fc089390 con 0x7f5104144840 2023-10-18T09:03:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.511+0000 7f51099e0640 1 --2- 172.21.15.40:0/4106293556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f50e806ae40 0x7f50e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.512+0000 7f51099e0640 1 --2- 172.21.15.40:0/4106293556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f50e806ae40 0x7f50e806d300 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5100035000 tx=0x7f5100004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:12.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.515+0000 7f50fa7fc640 1 -- 172.21.15.40:0/4106293556 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f50fc0546a0 con 0x7f5104144840 2023-10-18T09:03:12.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.709+0000 7f510bc44640 1 -- 172.21.15.40:0/4106293556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config reset", "num": 22} v 0) v1 -- 0x7f510406a560 con 0x7f5104144840 2023-10-18T09:03:12.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.716+0000 7f50fa7fc640 1 -- 172.21.15.40:0/4106293556 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 22}]=0 v34) v1 ==== 71+0+0 (secure 0 0 0) 0x7f50fc057cc0 con 0x7f5104144840 2023-10-18T09:03:12.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.719+0000 7f510bc44640 1 -- 172.21.15.40:0/4106293556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f50e806ae40 msgr2=0x7f50e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:12.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.719+0000 7f510bc44640 1 --2- 172.21.15.40:0/4106293556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f50e806ae40 0x7f50e806d300 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5100035000 tx=0x7f5100004730 comp rx=0 tx=0).stop 2023-10-18T09:03:12.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.719+0000 7f510bc44640 1 -- 172.21.15.40:0/4106293556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5104144840 msgr2=0x7f51040cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:12.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.719+0000 7f510bc44640 1 --2- 172.21.15.40:0/4106293556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5104144840 0x7f51040cb950 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00c7a0 tx=0x7f50fc00cc60 comp rx=0 tx=0).stop 2023-10-18T09:03:12.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.719+0000 7f510bc44640 1 -- 172.21.15.40:0/4106293556 shutdown_connections 2023-10-18T09:03:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.719+0000 7f510bc44640 1 --2- 172.21.15.40:0/4106293556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f50e806ae40 0x7f50e806d300 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.719+0000 7f510bc44640 1 --2- 172.21.15.40:0/4106293556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5104144840 0x7f51040cb950 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.719+0000 7f510bc44640 1 --2- 172.21.15.40:0/4106293556 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f510406d1c0 0x7f51040cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.719+0000 7f510bc44640 1 --2- 172.21.15.40:0/4106293556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5104069c60 0x7f51041f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.719+0000 7f510bc44640 1 -- 172.21.15.40:0/4106293556 >> 172.21.15.40:0/4106293556 conn(0x7f51040aff00 msgr2=0x7f5104143090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.719+0000 7f510bc44640 1 -- 172.21.15.40:0/4106293556 shutdown_connections 2023-10-18T09:03:12.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.719+0000 7f510bc44640 1 -- 172.21.15.40:0/4106293556 wait complete. 2023-10-18T09:03:12.742 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2023-10-18T09:03:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.972+0000 7f3e6e4fc640 1 -- 172.21.15.40:0/2161233189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e54003ee0 msgr2=0x7f3e540047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.972+0000 7f3e6e4fc640 1 --2- 172.21.15.40:0/2161233189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e54003ee0 0x7f3e540047c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009b50 tx=0x7f3e540057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.972+0000 7f3e6dcfb640 1 -- 172.21.15.40:0/2161233189 reap_dead start 2023-10-18T09:03:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.972+0000 7f3e6e4fc640 1 -- 172.21.15.40:0/2161233189 shutdown_connections 2023-10-18T09:03:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.972+0000 7f3e6e4fc640 1 -- 172.21.15.40:0/2161233189 >> 172.21.15.40:0/2161233189 conn(0x7f3e68069720 msgr2=0x7f3e68069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.972+0000 7f3e6e4fc640 1 -- 172.21.15.40:0/2161233189 shutdown_connections 2023-10-18T09:03:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.972+0000 7f3e6e4fc640 1 -- 172.21.15.40:0/2161233189 wait complete. 2023-10-18T09:03:12.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.974+0000 7f3e6e4fc640 1 Processor -- start 2023-10-18T09:03:12.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.974+0000 7f3e6e4fc640 1 -- start start 2023-10-18T09:03:12.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.975+0000 7f3e6e4fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e54003ee0 0x7f3e6813d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:12.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.975+0000 7f3e6e4fc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e681e5100 0x7f3e6813cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:12.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.975+0000 7f3e6e4fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e681e6c30 0x7f3e681e70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:12.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.975+0000 7f3e6e4fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e681e7620 con 0x7f3e681e6c30 2023-10-18T09:03:12.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.975+0000 7f3e6e4fc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6813dea0 con 0x7f3e54003ee0 2023-10-18T09:03:12.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.975+0000 7f3e6e4fc640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e681e5580 con 0x7f3e681e5100 2023-10-18T09:03:12.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.975+0000 7f3e6dcfb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e681e6c30 0x7f3e681e70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:12.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.975+0000 7f3e6dcfb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e681e6c30 0x7f3e681e70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60730/0 (socket says 172.21.15.40:60730) 2023-10-18T09:03:12.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.975+0000 7f3e6dcfb640 1 -- 172.21.15.40:0/1720747719 learned_addr learned my addr 172.21.15.40:0/1720747719 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:12.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.976+0000 7f3e6d4fa640 1 --2- 172.21.15.40:0/1720747719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e54003ee0 0x7f3e6813d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:12.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.976+0000 7f3e6ccf9640 1 --2- 172.21.15.40:0/1720747719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e681e5100 0x7f3e6813cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:12.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.976+0000 7f3e6dcfb640 1 -- 172.21.15.40:0/1720747719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e681e5100 msgr2=0x7f3e6813cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:12.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.976+0000 7f3e6dcfb640 1 --2- 172.21.15.40:0/1720747719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e681e5100 0x7f3e6813cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:12.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.976+0000 7f3e6dcfb640 1 -- 172.21.15.40:0/1720747719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e54003ee0 msgr2=0x7f3e6813d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:12.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.976+0000 7f3e6dcfb640 1 --2- 172.21.15.40:0/1720747719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e54003ee0 0x7f3e6813d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:12.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.976+0000 7f3e6dcfb640 1 -- 172.21.15.40:0/1720747719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e681e5850 con 0x7f3e681e6c30 2023-10-18T09:03:12.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.976+0000 7f3e6dcfb640 1 --2- 172.21.15.40:0/1720747719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e681e6c30 0x7f3e681e70e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f3e680b20e0 tx=0x7f3e6000eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:12.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.976+0000 7f3e5dffb640 1 -- 172.21.15.40:0/1720747719 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6000c970 con 0x7f3e681e6c30 2023-10-18T09:03:12.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.977+0000 7f3e5dffb640 1 -- 172.21.15.40:0/1720747719 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3e6000cbc0 con 0x7f3e681e6c30 2023-10-18T09:03:12.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.977+0000 7f3e5dffb640 1 -- 172.21.15.40:0/1720747719 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e60011460 con 0x7f3e681e6c30 2023-10-18T09:03:12.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.977+0000 7f3e6e4fc640 1 -- 172.21.15.40:0/1720747719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e681e5b30 con 0x7f3e681e6c30 2023-10-18T09:03:12.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.977+0000 7f3e6e4fc640 1 -- 172.21.15.40:0/1720747719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e681e60f0 con 0x7f3e681e6c30 2023-10-18T09:03:12.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.978+0000 7f3e6e4fc640 1 -- 172.21.15.40:0/1720747719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e681e6310 con 0x7f3e681e6c30 2023-10-18T09:03:12.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.979+0000 7f3e5dffb640 1 -- 172.21.15.40:0/1720747719 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3e600115c0 con 0x7f3e681e6c30 2023-10-18T09:03:12.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.980+0000 7f3e5dffb640 1 --2- 172.21.15.40:0/1720747719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e4c06ae40 0x7f3e4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:12.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.980+0000 7f3e5dffb640 1 -- 172.21.15.40:0/1720747719 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3e600899c0 con 0x7f3e681e6c30 2023-10-18T09:03:12.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.981+0000 7f3e6d4fa640 1 --2- 172.21.15.40:0/1720747719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e4c06ae40 0x7f3e4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:12.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.982+0000 7f3e6d4fa640 1 --2- 172.21.15.40:0/1720747719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e4c06ae40 0x7f3e4c06d300 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3e64000c00 tx=0x7f3e64006b60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:12.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:12.985+0000 7f3e5dffb640 1 -- 172.21.15.40:0/1720747719 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e60054cd0 con 0x7f3e681e6c30 2023-10-18T09:03:13.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:13.256+0000 7f3e6e4fc640 1 -- 172.21.15.40:0/1720747719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f3e68066e00 con 0x7f3e681e6c30 2023-10-18T09:03:14.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.245+0000 7f3e5dffb640 1 -- 172.21.15.40:0/1720747719 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v418) v1 ==== 128+0+0 (secure 0 0 0) 0x7f3e600582f0 con 0x7f3e681e6c30 2023-10-18T09:03:14.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.248+0000 7f3e6e4fc640 1 -- 172.21.15.40:0/1720747719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e4c06ae40 msgr2=0x7f3e4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:14.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.248+0000 7f3e6e4fc640 1 --2- 172.21.15.40:0/1720747719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e4c06ae40 0x7f3e4c06d300 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3e64000c00 tx=0x7f3e64006b60 comp rx=0 tx=0).stop 2023-10-18T09:03:14.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.248+0000 7f3e6e4fc640 1 -- 172.21.15.40:0/1720747719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e681e6c30 msgr2=0x7f3e681e70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:14.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.248+0000 7f3e6e4fc640 1 --2- 172.21.15.40:0/1720747719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e681e6c30 0x7f3e681e70e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f3e680b20e0 tx=0x7f3e6000eae0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.248+0000 7f3e6e4fc640 1 -- 172.21.15.40:0/1720747719 shutdown_connections 2023-10-18T09:03:14.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.248+0000 7f3e6e4fc640 1 --2- 172.21.15.40:0/1720747719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3e4c06ae40 0x7f3e4c06d300 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.248+0000 7f3e6e4fc640 1 --2- 172.21.15.40:0/1720747719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e681e6c30 0x7f3e681e70e0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.248+0000 7f3e6e4fc640 1 --2- 172.21.15.40:0/1720747719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e681e5100 0x7f3e6813cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.248+0000 7f3e6e4fc640 1 --2- 172.21.15.40:0/1720747719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e54003ee0 0x7f3e6813d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.248+0000 7f3e6e4fc640 1 -- 172.21.15.40:0/1720747719 >> 172.21.15.40:0/1720747719 conn(0x7f3e68069720 msgr2=0x7f3e68141380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:14.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.249+0000 7f3e6e4fc640 1 -- 172.21.15.40:0/1720747719 shutdown_connections 2023-10-18T09:03:14.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.249+0000 7f3e6e4fc640 1 -- 172.21.15.40:0/1720747719 wait complete. 2023-10-18T09:03:14.273 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2023-10-18T09:03:14.274 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) 2023-10-18T09:03:14.275 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:03:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.510+0000 7f7b0d8ea640 1 -- 172.21.15.40:0/823554195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b0806d6f0 msgr2=0x7f7b080b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.510+0000 7f7b0d8ea640 1 --2- 172.21.15.40:0/823554195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b0806d6f0 0x7f7b080b15b0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f7b0806b050 tx=0x7f7afc02bda0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.511+0000 7f7b0d8ea640 1 -- 172.21.15.40:0/823554195 shutdown_connections 2023-10-18T09:03:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.511+0000 7f7b0d8ea640 1 --2- 172.21.15.40:0/823554195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b080c9f00 0x7f7b08157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.511+0000 7f7b0d8ea640 1 --2- 172.21.15.40:0/823554195 >> v2:172.21.15.144:3300/0 conn(0x7f7b080b0630 0x7f7b080c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.511+0000 7f7b0d8ea640 1 --2- 172.21.15.40:0/823554195 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b0806d6f0 0x7f7b080b15b0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.511+0000 7f7b0d8ea640 1 -- 172.21.15.40:0/823554195 >> 172.21.15.40:0/823554195 conn(0x7f7b08069720 msgr2=0x7f7b08069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:14.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.511+0000 7f7b0d8ea640 1 -- 172.21.15.40:0/823554195 shutdown_connections 2023-10-18T09:03:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.511+0000 7f7b0d8ea640 1 -- 172.21.15.40:0/823554195 wait complete. 2023-10-18T09:03:14.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.512+0000 7f7b0d8ea640 1 Processor -- start 2023-10-18T09:03:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.513+0000 7f7b0d8ea640 1 -- start start 2023-10-18T09:03:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.513+0000 7f7b0d8ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b0806d6f0 0x7f7b08141880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.513+0000 7f7b0d8ea640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b080b0630 0x7f7b081499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.514+0000 7f7b0d8ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b080c9f00 0x7f7b08149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:14.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.514+0000 7f7b0d8ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b08148c20 con 0x7f7b080c9f00 2023-10-18T09:03:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.514+0000 7f7b0d8ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b08148ec0 con 0x7f7b0806d6f0 2023-10-18T09:03:14.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.514+0000 7f7b0d8ea640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b081491a0 con 0x7f7b080b0630 2023-10-18T09:03:14.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.514+0000 7f7b0d0e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b080c9f00 0x7f7b08149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:14.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.514+0000 7f7b0d0e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b080c9f00 0x7f7b08149f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60758/0 (socket says 172.21.15.40:60758) 2023-10-18T09:03:14.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.514+0000 7f7b0d0e9640 1 -- 172.21.15.40:0/3013733080 learned_addr learned my addr 172.21.15.40:0/3013733080 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:14.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.514+0000 7f7b0c8e8640 1 --2- 172.21.15.40:0/3013733080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b0806d6f0 0x7f7b08141880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:14.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.514+0000 7f7b07fff640 1 --2- 172.21.15.40:0/3013733080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b080b0630 0x7f7b081499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:14.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.514+0000 7f7b0d0e9640 1 -- 172.21.15.40:0/3013733080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b080b0630 msgr2=0x7f7b081499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:14.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.514+0000 7f7b0d0e9640 1 --2- 172.21.15.40:0/3013733080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b080b0630 0x7f7b081499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.514+0000 7f7b0d0e9640 1 -- 172.21.15.40:0/3013733080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b0806d6f0 msgr2=0x7f7b08141880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:14.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.514+0000 7f7b0d0e9640 1 --2- 172.21.15.40:0/3013733080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b0806d6f0 0x7f7b08141880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.514+0000 7f7b0d0e9640 1 -- 172.21.15.40:0/3013733080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc009560 con 0x7f7b080c9f00 2023-10-18T09:03:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.515+0000 7f7b07fff640 1 --2- 172.21.15.40:0/3013733080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b080b0630 0x7f7b081499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.515+0000 7f7b0d0e9640 1 --2- 172.21.15.40:0/3013733080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b080c9f00 0x7f7b08149f30 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000cab0 tx=0x7f7b0000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.515+0000 7f7b057fa640 1 -- 172.21.15.40:0/3013733080 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00007d90 con 0x7f7b080c9f00 2023-10-18T09:03:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.515+0000 7f7b057fa640 1 -- 172.21.15.40:0/3013733080 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7b0000f450 con 0x7f7b080c9f00 2023-10-18T09:03:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.515+0000 7f7b057fa640 1 -- 172.21.15.40:0/3013733080 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0000f840 con 0x7f7b080c9f00 2023-10-18T09:03:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.515+0000 7f7b0d8ea640 1 -- 172.21.15.40:0/3013733080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b081494d0 con 0x7f7b080c9f00 2023-10-18T09:03:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.515+0000 7f7b0d8ea640 1 -- 172.21.15.40:0/3013733080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b08144040 con 0x7f7b080c9f00 2023-10-18T09:03:14.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.516+0000 7f7b0d8ea640 1 -- 172.21.15.40:0/3013733080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b081496c0 con 0x7f7b080c9f00 2023-10-18T09:03:14.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.517+0000 7f7b057fa640 1 -- 172.21.15.40:0/3013733080 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7b0000f9a0 con 0x7f7b080c9f00 2023-10-18T09:03:14.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.518+0000 7f7b057fa640 1 --2- 172.21.15.40:0/3013733080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7aec06ae70 0x7f7aec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:14.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.519+0000 7f7b057fa640 1 -- 172.21.15.40:0/3013733080 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7b00089df0 con 0x7f7b080c9f00 2023-10-18T09:03:14.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.519+0000 7f7b0c8e8640 1 --2- 172.21.15.40:0/3013733080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7aec06ae70 0x7f7aec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:14.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.520+0000 7f7b0c8e8640 1 --2- 172.21.15.40:0/3013733080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7aec06ae70 0x7f7aec06d330 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7afc035000 tx=0x7f7afc004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:14.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.523+0000 7f7b057fa640 1 -- 172.21.15.40:0/3013733080 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b00055100 con 0x7f7b080c9f00 2023-10-18T09:03:14.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.718+0000 7f7b0d8ea640 1 -- 172.21.15.40:0/3013733080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b08066e00 con 0x7f7b080c9f00 2023-10-18T09:03:14.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.720+0000 7f7b057fa640 1 -- 172.21.15.40:0/3013733080 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+13009 (secure 0 0 0) 0x7f7b0001d090 con 0x7f7b080c9f00 2023-10-18T09:03:14.722 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:14.722 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":136,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:03:12.228447+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/1801003551":"2023-10-19T09:03:09.217217+0000","172.21.15.40:6834/1801003551":"2023-10-19T09:03:09.217217+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:03:14.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.722+0000 7f7b0d8ea640 1 -- 172.21.15.40:0/3013733080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7aec06ae70 msgr2=0x7f7aec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:14.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.722+0000 7f7b0d8ea640 1 --2- 172.21.15.40:0/3013733080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7aec06ae70 0x7f7aec06d330 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f7afc035000 tx=0x7f7afc004730 comp rx=0 tx=0).stop 2023-10-18T09:03:14.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.722+0000 7f7b0d8ea640 1 -- 172.21.15.40:0/3013733080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b080c9f00 msgr2=0x7f7b08149f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:14.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.722+0000 7f7b0d8ea640 1 --2- 172.21.15.40:0/3013733080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b080c9f00 0x7f7b08149f30 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000cab0 tx=0x7f7b0000cf70 comp rx=0 tx=0).stop 2023-10-18T09:03:14.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.723+0000 7f7b0d8ea640 1 -- 172.21.15.40:0/3013733080 shutdown_connections 2023-10-18T09:03:14.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.723+0000 7f7b0d8ea640 1 --2- 172.21.15.40:0/3013733080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7aec06ae70 0x7f7aec06d330 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.723+0000 7f7b0d8ea640 1 --2- 172.21.15.40:0/3013733080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b080c9f00 0x7f7b08149f30 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.723+0000 7f7b0d8ea640 1 --2- 172.21.15.40:0/3013733080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b080b0630 0x7f7b081499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.723+0000 7f7b0d8ea640 1 --2- 172.21.15.40:0/3013733080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b0806d6f0 0x7f7b08141880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.723+0000 7f7b0d8ea640 1 -- 172.21.15.40:0/3013733080 >> 172.21.15.40:0/3013733080 conn(0x7f7b08069720 msgr2=0x7f7b08157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:14.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.723+0000 7f7b0d8ea640 1 -- 172.21.15.40:0/3013733080 shutdown_connections 2023-10-18T09:03:14.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.723+0000 7f7b0d8ea640 1 -- 172.21.15.40:0/3013733080 wait complete. 2023-10-18T09:03:14.745 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:03:14.746 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:03:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.981+0000 7f27b7475640 1 -- 172.21.15.40:0/3640898454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b00ac7b0 msgr2=0x7f27b00cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.981+0000 7f27b7475640 1 --2- 172.21.15.40:0/3640898454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b00ac7b0 0x7f27b00cc1f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009830 tx=0x7f27a002bd60 comp rx=0 tx=0).stop 2023-10-18T09:03:14.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.982+0000 7f27b7475640 1 -- 172.21.15.40:0/3640898454 shutdown_connections 2023-10-18T09:03:14.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.982+0000 7f27b7475640 1 --2- 172.21.15.40:0/3640898454 >> v2:172.21.15.144:3300/0 conn(0x7f27b00aed00 0x7f27b00af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.982+0000 7f27b7475640 1 --2- 172.21.15.40:0/3640898454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b00ac7b0 0x7f27b00cc1f0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.982+0000 7f27b7475640 1 -- 172.21.15.40:0/3640898454 >> 172.21.15.40:0/3640898454 conn(0x7f27b0069720 msgr2=0x7f27b0069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:14.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.982+0000 7f27b7475640 1 -- 172.21.15.40:0/3640898454 shutdown_connections 2023-10-18T09:03:14.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.982+0000 7f27b7475640 1 -- 172.21.15.40:0/3640898454 wait complete. 2023-10-18T09:03:14.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.983+0000 7f27b7475640 1 Processor -- start 2023-10-18T09:03:14.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.984+0000 7f27b7475640 1 -- start start 2023-10-18T09:03:14.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.985+0000 7f27b7475640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27b00ac7b0 0x7f27b0141b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:14.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.985+0000 7f27b7475640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27b00aed00 0x7f27b015b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:14.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.985+0000 7f27b7475640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b0155cd0 0x7f27b00cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:14.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.985+0000 7f27b7475640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b0148e70 con 0x7f27b0155cd0 2023-10-18T09:03:14.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.985+0000 7f27b7475640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b01520f0 con 0x7f27b00ac7b0 2023-10-18T09:03:14.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.985+0000 7f27b7475640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b01514e0 con 0x7f27b00aed00 2023-10-18T09:03:14.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.985+0000 7f27b5a12640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b0155cd0 0x7f27b00cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:14.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.985+0000 7f27b5a12640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b0155cd0 0x7f27b00cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54514/0 (socket says 172.21.15.40:54514) 2023-10-18T09:03:14.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.985+0000 7f27b5a12640 1 -- 172.21.15.40:0/2242637182 learned_addr learned my addr 172.21.15.40:0/2242637182 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:14.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.985+0000 7f27b5211640 1 --2- 172.21.15.40:0/2242637182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27b00ac7b0 0x7f27b0141b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:14.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.985+0000 7f27b4a10640 1 --2- 172.21.15.40:0/2242637182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27b00aed00 0x7f27b015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:14.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.985+0000 7f27b5a12640 1 -- 172.21.15.40:0/2242637182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27b00aed00 msgr2=0x7f27b015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:14.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.985+0000 7f27b5a12640 1 --2- 172.21.15.40:0/2242637182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27b00aed00 0x7f27b015b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.985+0000 7f27b5a12640 1 -- 172.21.15.40:0/2242637182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27b00ac7b0 msgr2=0x7f27b0141b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:14.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.985+0000 7f27b5a12640 1 --2- 172.21.15.40:0/2242637182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27b00ac7b0 0x7f27b0141b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:14.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.985+0000 7f27b5a12640 1 -- 172.21.15.40:0/2242637182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a0009490 con 0x7f27b0155cd0 2023-10-18T09:03:14.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.986+0000 7f27b5a12640 1 --2- 172.21.15.40:0/2242637182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b0155cd0 0x7f27b00cc770 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f27a800eaa0 tx=0x7f27a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:14.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.986+0000 7f27a5ffb640 1 -- 172.21.15.40:0/2242637182 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a800cdb0 con 0x7f27b0155cd0 2023-10-18T09:03:14.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.986+0000 7f27a5ffb640 1 -- 172.21.15.40:0/2242637182 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f27a8004b20 con 0x7f27b0155cd0 2023-10-18T09:03:14.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.986+0000 7f27a5ffb640 1 -- 172.21.15.40:0/2242637182 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a8005020 con 0x7f27b0155cd0 2023-10-18T09:03:14.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.986+0000 7f27b7475640 1 -- 172.21.15.40:0/2242637182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b01517c0 con 0x7f27b0155cd0 2023-10-18T09:03:14.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.986+0000 7f27b7475640 1 -- 172.21.15.40:0/2242637182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b0151d80 con 0x7f27b0155cd0 2023-10-18T09:03:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.988+0000 7f27b7475640 1 -- 172.21.15.40:0/2242637182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b0152260 con 0x7f27b0155cd0 2023-10-18T09:03:14.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.988+0000 7f27a5ffb640 1 -- 172.21.15.40:0/2242637182 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f27a8005180 con 0x7f27b0155cd0 2023-10-18T09:03:14.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.989+0000 7f27a5ffb640 1 --2- 172.21.15.40:0/2242637182 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f279406ae40 0x7f279406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:14.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.989+0000 7f27a5ffb640 1 -- 172.21.15.40:0/2242637182 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f27a8089dc0 con 0x7f27b0155cd0 2023-10-18T09:03:14.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.990+0000 7f27b5211640 1 --2- 172.21.15.40:0/2242637182 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f279406ae40 0x7f279406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:14.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.991+0000 7f27b5211640 1 --2- 172.21.15.40:0/2242637182 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f279406ae40 0x7f279406d300 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f27ac007fd0 tx=0x7f27ac009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:14.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:14.993+0000 7f27a5ffb640 1 -- 172.21.15.40:0/2242637182 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27a8055160 con 0x7f27b0155cd0 2023-10-18T09:03:15.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.191+0000 7f27b7475640 1 -- 172.21.15.40:0/2242637182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27b0066fd0 con 0x7f27b0155cd0 2023-10-18T09:03:15.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.191+0000 7f27a5ffb640 1 -- 172.21.15.40:0/2242637182 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27a801f020 con 0x7f27b0155cd0 2023-10-18T09:03:15.194 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:03:15.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.194+0000 7f27b7475640 1 -- 172.21.15.40:0/2242637182 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f279406ae40 msgr2=0x7f279406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:15.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.194+0000 7f27b7475640 1 --2- 172.21.15.40:0/2242637182 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f279406ae40 0x7f279406d300 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f27ac007fd0 tx=0x7f27ac009040 comp rx=0 tx=0).stop 2023-10-18T09:03:15.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.194+0000 7f27b7475640 1 -- 172.21.15.40:0/2242637182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b0155cd0 msgr2=0x7f27b00cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:15.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.194+0000 7f27b7475640 1 --2- 172.21.15.40:0/2242637182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b0155cd0 0x7f27b00cc770 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f27a800eaa0 tx=0x7f27a800ef60 comp rx=0 tx=0).stop 2023-10-18T09:03:15.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.195+0000 7f27b7475640 1 -- 172.21.15.40:0/2242637182 shutdown_connections 2023-10-18T09:03:15.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.195+0000 7f27b7475640 1 --2- 172.21.15.40:0/2242637182 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f279406ae40 0x7f279406d300 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:15.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.195+0000 7f27b7475640 1 --2- 172.21.15.40:0/2242637182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b0155cd0 0x7f27b00cc770 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:15.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.195+0000 7f27b7475640 1 --2- 172.21.15.40:0/2242637182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27b00aed00 0x7f27b015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:15.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.195+0000 7f27b7475640 1 --2- 172.21.15.40:0/2242637182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27b00ac7b0 0x7f27b0141b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:15.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.195+0000 7f27b7475640 1 -- 172.21.15.40:0/2242637182 >> 172.21.15.40:0/2242637182 conn(0x7f27b0069720 msgr2=0x7f27b00ad330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:15.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.195+0000 7f27b7475640 1 -- 172.21.15.40:0/2242637182 shutdown_connections 2023-10-18T09:03:15.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.195+0000 7f27b7475640 1 -- 172.21.15.40:0/2242637182 wait complete. 2023-10-18T09:03:15.217 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2023-10-18T09:03:15.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.446+0000 7fd45617d640 1 -- 172.21.15.40:0/3718583050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd43c0047c0 msgr2=0x7fd43c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:15.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.446+0000 7fd45617d640 1 --2- 172.21.15.40:0/3718583050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd43c0047c0 0x7fd43c004c40 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd438001650 tx=0x7fd43c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:15.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.446+0000 7fd45617d640 1 -- 172.21.15.40:0/3718583050 shutdown_connections 2023-10-18T09:03:15.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.446+0000 7fd45617d640 1 --2- 172.21.15.40:0/3718583050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd43c0047c0 0x7fd43c004c40 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:15.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.446+0000 7fd45617d640 1 -- 172.21.15.40:0/3718583050 >> 172.21.15.40:0/3718583050 conn(0x7fd45006cee0 msgr2=0x7fd450069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:15.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.446+0000 7fd45617d640 1 -- 172.21.15.40:0/3718583050 shutdown_connections 2023-10-18T09:03:15.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.446+0000 7fd45617d640 1 -- 172.21.15.40:0/3718583050 wait complete. 2023-10-18T09:03:15.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.448+0000 7fd45617d640 1 Processor -- start 2023-10-18T09:03:15.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.448+0000 7fd45617d640 1 -- start start 2023-10-18T09:03:15.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.449+0000 7fd45617d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd43c0047c0 0x7fd4501e1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:15.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.449+0000 7fd45617d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd45014d7e0 0x7fd450148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:15.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.449+0000 7fd45617d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd45014e0d0 0x7fd45014e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:15.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.449+0000 7fd45617d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4501f1b60 con 0x7fd43c0047c0 2023-10-18T09:03:15.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.449+0000 7fd45617d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45014ea90 con 0x7fd45014e0d0 2023-10-18T09:03:15.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.449+0000 7fd45617d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45014beb0 con 0x7fd45014d7e0 2023-10-18T09:03:15.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.450+0000 7fd44f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd43c0047c0 0x7fd4501e1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:15.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.450+0000 7fd44f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd43c0047c0 0x7fd4501e1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54534/0 (socket says 172.21.15.40:54534) 2023-10-18T09:03:15.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.450+0000 7fd44f7fe640 1 -- 172.21.15.40:0/580163156 learned_addr learned my addr 172.21.15.40:0/580163156 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:15.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.450+0000 7fd44ffff640 1 --2- 172.21.15.40:0/580163156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd45014e0d0 0x7fd45014e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:15.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.450+0000 7fd447fff640 1 --2- 172.21.15.40:0/580163156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd45014d7e0 0x7fd450148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:15.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.450+0000 7fd44f7fe640 1 -- 172.21.15.40:0/580163156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd45014d7e0 msgr2=0x7fd450148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:15.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.450+0000 7fd44f7fe640 1 --2- 172.21.15.40:0/580163156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd45014d7e0 0x7fd450148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:15.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.450+0000 7fd44f7fe640 1 -- 172.21.15.40:0/580163156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd45014e0d0 msgr2=0x7fd45014e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:15.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.450+0000 7fd44f7fe640 1 --2- 172.21.15.40:0/580163156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd45014e0d0 0x7fd45014e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.450+0000 7fd44f7fe640 1 -- 172.21.15.40:0/580163156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45014c190 con 0x7fd43c0047c0 2023-10-18T09:03:15.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.450+0000 7fd44f7fe640 1 --2- 172.21.15.40:0/580163156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd43c0047c0 0x7fd4501e1700 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fd44000c650 tx=0x7fd44000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:15.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.451+0000 7fd44cff9640 1 -- 172.21.15.40:0/580163156 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd440005270 con 0x7fd43c0047c0 2023-10-18T09:03:15.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.451+0000 7fd44cff9640 1 -- 172.21.15.40:0/580163156 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd44000a640 con 0x7fd43c0047c0 2023-10-18T09:03:15.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.451+0000 7fd45617d640 1 -- 172.21.15.40:0/580163156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45014c470 con 0x7fd43c0047c0 2023-10-18T09:03:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.451+0000 7fd44cff9640 1 -- 172.21.15.40:0/580163156 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44000aa50 con 0x7fd43c0047c0 2023-10-18T09:03:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.451+0000 7fd45617d640 1 -- 172.21.15.40:0/580163156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45014dc60 con 0x7fd43c0047c0 2023-10-18T09:03:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.452+0000 7fd45617d640 1 -- 172.21.15.40:0/580163156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd450066fd0 con 0x7fd43c0047c0 2023-10-18T09:03:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.453+0000 7fd44cff9640 1 -- 172.21.15.40:0/580163156 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd44000abb0 con 0x7fd43c0047c0 2023-10-18T09:03:15.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.454+0000 7fd44cff9640 1 --2- 172.21.15.40:0/580163156 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd43006ae40 0x7fd43006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:15.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.454+0000 7fd44cff9640 1 -- 172.21.15.40:0/580163156 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd440088d70 con 0x7fd43c0047c0 2023-10-18T09:03:15.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.455+0000 7fd447fff640 1 --2- 172.21.15.40:0/580163156 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd43006ae40 0x7fd43006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:15.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.456+0000 7fd447fff640 1 --2- 172.21.15.40:0/580163156 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd43006ae40 0x7fd43006d300 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd438034000 tx=0x7fd438001190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:15.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.459+0000 7fd44cff9640 1 -- 172.21.15.40:0/580163156 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd440054080 con 0x7fd43c0047c0 2023-10-18T09:03:15.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:15.725+0000 7fd45617d640 1 -- 172.21.15.40:0/580163156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7fd4500cdba0 con 0x7fd43c0047c0 2023-10-18T09:03:16.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.256+0000 7fd44cff9640 1 -- 172.21.15.40:0/580163156 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v420) v1 ==== 143+0+0 (secure 0 0 0) 0x7fd440017020 con 0x7fd43c0047c0 2023-10-18T09:03:16.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.259+0000 7fd45617d640 1 -- 172.21.15.40:0/580163156 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd43006ae40 msgr2=0x7fd43006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:16.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.259+0000 7fd45617d640 1 --2- 172.21.15.40:0/580163156 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd43006ae40 0x7fd43006d300 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd438034000 tx=0x7fd438001190 comp rx=0 tx=0).stop 2023-10-18T09:03:16.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.259+0000 7fd45617d640 1 -- 172.21.15.40:0/580163156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd43c0047c0 msgr2=0x7fd4501e1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:16.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.259+0000 7fd45617d640 1 --2- 172.21.15.40:0/580163156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd43c0047c0 0x7fd4501e1700 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fd44000c650 tx=0x7fd44000cb10 comp rx=0 tx=0).stop 2023-10-18T09:03:16.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.260+0000 7fd45617d640 1 -- 172.21.15.40:0/580163156 shutdown_connections 2023-10-18T09:03:16.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.260+0000 7fd45617d640 1 --2- 172.21.15.40:0/580163156 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd43006ae40 0x7fd43006d300 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:16.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.260+0000 7fd45617d640 1 --2- 172.21.15.40:0/580163156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd45014e0d0 0x7fd45014e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:16.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.260+0000 7fd45617d640 1 --2- 172.21.15.40:0/580163156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd45014d7e0 0x7fd450148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:16.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.260+0000 7fd45617d640 1 --2- 172.21.15.40:0/580163156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd43c0047c0 0x7fd4501e1700 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:16.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.260+0000 7fd45617d640 1 -- 172.21.15.40:0/580163156 >> 172.21.15.40:0/580163156 conn(0x7fd45006cee0 msgr2=0x7fd450069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:16.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.260+0000 7fd45617d640 1 -- 172.21.15.40:0/580163156 shutdown_connections 2023-10-18T09:03:16.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.260+0000 7fd45617d640 1 -- 172.21.15.40:0/580163156 wait complete. 2023-10-18T09:03:16.283 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T09:03:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.504+0000 7f4627fff640 1 -- 172.21.15.40:0/3089471698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46100058d0 msgr2=0x7f4610005d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.504+0000 7f4627fff640 1 --2- 172.21.15.40:0/3089471698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46100058d0 0x7f4610005d50 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f461c001680 tx=0x7f4610006900 comp rx=0 tx=0).stop 2023-10-18T09:03:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.504+0000 7f4627fff640 1 -- 172.21.15.40:0/3089471698 shutdown_connections 2023-10-18T09:03:16.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.504+0000 7f4627fff640 1 --2- 172.21.15.40:0/3089471698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46100058d0 0x7f4610005d50 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:16.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.504+0000 7f4627fff640 1 -- 172.21.15.40:0/3089471698 >> 172.21.15.40:0/3089471698 conn(0x7f46280aed70 msgr2=0x7f46280af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:16.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.504+0000 7f4627fff640 1 -- 172.21.15.40:0/3089471698 shutdown_connections 2023-10-18T09:03:16.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.504+0000 7f4627fff640 1 -- 172.21.15.40:0/3089471698 wait complete. 2023-10-18T09:03:16.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.506+0000 7f4627fff640 1 Processor -- start 2023-10-18T09:03:16.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.506+0000 7f4627fff640 1 -- start start 2023-10-18T09:03:16.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.507+0000 7f4627fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46100058d0 0x7f462813e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:16.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.507+0000 7f4627fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46281e06b0 0x7f462806b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:16.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.507+0000 7f4627fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46281ef9f0 0x7f462806bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:16.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.507+0000 7f4627fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46281eff40 con 0x7f46281e06b0 2023-10-18T09:03:16.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.507+0000 7f4627fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46281f0220 con 0x7f46100058d0 2023-10-18T09:03:16.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.507+0000 7f4627fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46281f0500 con 0x7f46281ef9f0 2023-10-18T09:03:16.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.508+0000 7f46267fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46281e06b0 0x7f462806b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:16.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.508+0000 7f46267fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46281e06b0 0x7f462806b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54560/0 (socket says 172.21.15.40:54560) 2023-10-18T09:03:16.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.508+0000 7f46267fc640 1 -- 172.21.15.40:0/3629335921 learned_addr learned my addr 172.21.15.40:0/3629335921 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:16.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.508+0000 7f4626ffd640 1 --2- 172.21.15.40:0/3629335921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46100058d0 0x7f462813e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:16.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.508+0000 7f46267fc640 1 -- 172.21.15.40:0/3629335921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46281ef9f0 msgr2=0x7f462806bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:16.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.508+0000 7f46267fc640 1 --2- 172.21.15.40:0/3629335921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46281ef9f0 0x7f462806bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:16.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.508+0000 7f46267fc640 1 -- 172.21.15.40:0/3629335921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46100058d0 msgr2=0x7f462813e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:16.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.508+0000 7f46267fc640 1 --2- 172.21.15.40:0/3629335921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46100058d0 0x7f462813e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:16.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.508+0000 7f46267fc640 1 -- 172.21.15.40:0/3629335921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46281f07e0 con 0x7f46281e06b0 2023-10-18T09:03:16.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.508+0000 7f46267fc640 1 --2- 172.21.15.40:0/3629335921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46281e06b0 0x7f462806b640 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f461c02c0d0 tx=0x7f461c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:16.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.509+0000 7f460ffff640 1 -- 172.21.15.40:0/3629335921 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c003b70 con 0x7f46281e06b0 2023-10-18T09:03:16.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.509+0000 7f460ffff640 1 -- 172.21.15.40:0/3629335921 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f461c003dc0 con 0x7f46281e06b0 2023-10-18T09:03:16.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.509+0000 7f460ffff640 1 -- 172.21.15.40:0/3629335921 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c02c790 con 0x7f46281e06b0 2023-10-18T09:03:16.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.509+0000 7f4627fff640 1 -- 172.21.15.40:0/3629335921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462813ec40 con 0x7f46281e06b0 2023-10-18T09:03:16.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.509+0000 7f4627fff640 1 -- 172.21.15.40:0/3629335921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4628154860 con 0x7f46281e06b0 2023-10-18T09:03:16.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.510+0000 7f4627fff640 1 -- 172.21.15.40:0/3629335921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46281f0a20 con 0x7f46281e06b0 2023-10-18T09:03:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.511+0000 7f460ffff640 1 -- 172.21.15.40:0/3629335921 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f461c02c8f0 con 0x7f46281e06b0 2023-10-18T09:03:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.512+0000 7f460ffff640 1 --2- 172.21.15.40:0/3629335921 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f461406ae70 0x7f461406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:16.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.512+0000 7f460ffff640 1 -- 172.21.15.40:0/3629335921 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f461c0aa560 con 0x7f46281e06b0 2023-10-18T09:03:16.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.513+0000 7f4626ffd640 1 --2- 172.21.15.40:0/3629335921 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f461406ae70 0x7f461406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:16.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.514+0000 7f4626ffd640 1 --2- 172.21.15.40:0/3629335921 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f461406ae70 0x7f461406d330 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f462804f250 tx=0x7f4618009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:16.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.517+0000 7f460ffff640 1 -- 172.21.15.40:0/3629335921 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f461c075870 con 0x7f46281e06b0 2023-10-18T09:03:16.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.712+0000 7f4627fff640 1 -- 172.21.15.40:0/3629335921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f4628066e00 con 0x7f46281e06b0 2023-10-18T09:03:16.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.713+0000 7f460ffff640 1 -- 172.21.15.40:0/3629335921 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v34) v1 ==== 59+0+1452 (secure 0 0 0) 0x7f461c078e90 con 0x7f46281e06b0 2023-10-18T09:03:16.716 INFO:teuthology.orchestra.run.smithi040.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T09:03:16.716 INFO:teuthology.orchestra.run.smithi040.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T09:03:16.716 INFO:teuthology.orchestra.run.smithi040.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T09:03:16.716 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds 20/20 2023-10-18T09:03:16.716 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T09:03:16.717 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_ms 1/1 2023-10-18T09:03:16.717 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T09:03:16.717 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T09:03:16.717 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T09:03:16.717 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_size 100 2023-10-18T09:03:16.717 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_frag true 2023-10-18T09:03:16.718 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_scatterstat true 2023-10-18T09:03:16.718 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T09:03:16.718 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_verify_scatter true 2023-10-18T09:03:16.718 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T09:03:16.718 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T09:03:16.719 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T09:03:16.719 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced client_mount_timeout 600 2023-10-18T09:03:16.719 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_client 20/20 2023-10-18T09:03:16.719 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_ms 1/1 2023-10-18T09:03:16.719 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T09:03:16.720 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T09:03:16.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.716+0000 7f4627fff640 1 -- 172.21.15.40:0/3629335921 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f461406ae70 msgr2=0x7f461406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:16.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.716+0000 7f4627fff640 1 --2- 172.21.15.40:0/3629335921 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f461406ae70 0x7f461406d330 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f462804f250 tx=0x7f4618009040 comp rx=0 tx=0).stop 2023-10-18T09:03:16.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.716+0000 7f4627fff640 1 -- 172.21.15.40:0/3629335921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46281e06b0 msgr2=0x7f462806b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:16.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.716+0000 7f4627fff640 1 --2- 172.21.15.40:0/3629335921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46281e06b0 0x7f462806b640 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f461c02c0d0 tx=0x7f461c0035f0 comp rx=0 tx=0).stop 2023-10-18T09:03:16.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.716+0000 7f4627fff640 1 -- 172.21.15.40:0/3629335921 shutdown_connections 2023-10-18T09:03:16.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.716+0000 7f4627fff640 1 --2- 172.21.15.40:0/3629335921 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f461406ae70 0x7f461406d330 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:16.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.716+0000 7f4627fff640 1 --2- 172.21.15.40:0/3629335921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46281ef9f0 0x7f462806bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:16.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.716+0000 7f4627fff640 1 --2- 172.21.15.40:0/3629335921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46281e06b0 0x7f462806b640 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:16.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.716+0000 7f4627fff640 1 --2- 172.21.15.40:0/3629335921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46100058d0 0x7f462813e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:16.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.716+0000 7f4627fff640 1 -- 172.21.15.40:0/3629335921 >> 172.21.15.40:0/3629335921 conn(0x7f46280aed70 msgr2=0x7f46281407d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:16.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.716+0000 7f4627fff640 1 -- 172.21.15.40:0/3629335921 shutdown_connections 2023-10-18T09:03:16.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.716+0000 7f4627fff640 1 -- 172.21.15.40:0/3629335921 wait complete. 2023-10-18T09:03:16.737 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T09:03:16.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.971+0000 7f0c2dc60640 1 -- 172.21.15.40:0/835888497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c28071270 msgr2=0x7f0c2806c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:16.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.971+0000 7f0c2dc60640 1 --2- 172.21.15.40:0/835888497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c28071270 0x7f0c2806c100 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f0c20005fd0 tx=0x7f0c2002c240 comp rx=0 tx=0).stop 2023-10-18T09:03:16.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.972+0000 7f0c2dc60640 1 -- 172.21.15.40:0/835888497 shutdown_connections 2023-10-18T09:03:16.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.972+0000 7f0c2dc60640 1 --2- 172.21.15.40:0/835888497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c28071270 0x7f0c2806c100 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:16.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.972+0000 7f0c2dc60640 1 --2- 172.21.15.40:0/835888497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c280c9620 0x7f0c2806bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:16.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.972+0000 7f0c2dc60640 1 -- 172.21.15.40:0/835888497 >> 172.21.15.40:0/835888497 conn(0x7f0c2806ca10 msgr2=0x7f0c2806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:16.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.972+0000 7f0c2dc60640 1 -- 172.21.15.40:0/835888497 shutdown_connections 2023-10-18T09:03:16.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.972+0000 7f0c2dc60640 1 -- 172.21.15.40:0/835888497 wait complete. 2023-10-18T09:03:16.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.973+0000 7f0c2dc60640 1 Processor -- start 2023-10-18T09:03:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.974+0000 7f0c2dc60640 1 -- start start 2023-10-18T09:03:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.974+0000 7f0c2dc60640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c28071270 0x7f0c28152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c2dc60640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c280c9620 0x7f0c280cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c2dc60640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c2815af70 0x7f0c28158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:16.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c2dc60640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c28158b70 con 0x7f0c2815af70 2023-10-18T09:03:16.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c2dc60640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c28158e50 con 0x7f0c28071270 2023-10-18T09:03:16.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c2dc60640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c28159130 con 0x7f0c280c9620 2023-10-18T09:03:16.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c2d45f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c2815af70 0x7f0c28158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:16.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c2d45f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c2815af70 0x7f0c28158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54572/0 (socket says 172.21.15.40:54572) 2023-10-18T09:03:16.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c2d45f640 1 -- 172.21.15.40:0/1433368276 learned_addr learned my addr 172.21.15.40:0/1433368276 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c2cc5e640 1 --2- 172.21.15.40:0/1433368276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c28071270 0x7f0c28152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c2d45f640 1 -- 172.21.15.40:0/1433368276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c280c9620 msgr2=0x7f0c280cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c1ffff640 1 --2- 172.21.15.40:0/1433368276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c280c9620 0x7f0c280cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c2d45f640 1 --2- 172.21.15.40:0/1433368276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c280c9620 0x7f0c280cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:16.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c2d45f640 1 -- 172.21.15.40:0/1433368276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c28071270 msgr2=0x7f0c28152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c2d45f640 1 --2- 172.21.15.40:0/1433368276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c28071270 0x7f0c28152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c2d45f640 1 -- 172.21.15.40:0/1433368276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c20009900 con 0x7f0c2815af70 2023-10-18T09:03:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c2cc5e640 1 --2- 172.21.15.40:0/1433368276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c28071270 0x7f0c28152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:16.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.975+0000 7f0c1ffff640 1 --2- 172.21.15.40:0/1433368276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c280c9620 0x7f0c280cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:16.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.976+0000 7f0c2d45f640 1 --2- 172.21.15.40:0/1433368276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c2815af70 0x7f0c28158120 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f0c20037040 tx=0x7f0c20035a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:16.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.976+0000 7f0c1d7fa640 1 -- 172.21.15.40:0/1433368276 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c20034440 con 0x7f0c2815af70 2023-10-18T09:03:16.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.976+0000 7f0c1d7fa640 1 -- 172.21.15.40:0/1433368276 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0c20004330 con 0x7f0c2815af70 2023-10-18T09:03:16.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.976+0000 7f0c1d7fa640 1 -- 172.21.15.40:0/1433368276 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c200046c0 con 0x7f0c2815af70 2023-10-18T09:03:16.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.976+0000 7f0c2dc60640 1 -- 172.21.15.40:0/1433368276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c28159410 con 0x7f0c2815af70 2023-10-18T09:03:16.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.976+0000 7f0c2dc60640 1 -- 172.21.15.40:0/1433368276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c28148b20 con 0x7f0c2815af70 2023-10-18T09:03:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.978+0000 7f0c2dc60640 1 -- 172.21.15.40:0/1433368276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c28148cb0 con 0x7f0c2815af70 2023-10-18T09:03:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.979+0000 7f0c1d7fa640 1 -- 172.21.15.40:0/1433368276 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0c2003b040 con 0x7f0c2815af70 2023-10-18T09:03:16.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.980+0000 7f0c1d7fa640 1 --2- 172.21.15.40:0/1433368276 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0c1406ae40 0x7f0c1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:16.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.981+0000 7f0c1d7fa640 1 -- 172.21.15.40:0/1433368276 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0c200abf30 con 0x7f0c2815af70 2023-10-18T09:03:16.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.981+0000 7f0c2cc5e640 1 --2- 172.21.15.40:0/1433368276 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0c1406ae40 0x7f0c1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:16.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.982+0000 7f0c2cc5e640 1 --2- 172.21.15.40:0/1433368276 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0c1406ae40 0x7f0c1406d300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009b50 tx=0x7f0c24005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:16.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:16.985+0000 7f0c1d7fa640 1 -- 172.21.15.40:0/1433368276 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0c28148cb0 con 0x7f0c2815af70 2023-10-18T09:03:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.181+0000 7f0c2dc60640 1 -- 172.21.15.40:0/1433368276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0c28159930 con 0x7f0c2815af70 2023-10-18T09:03:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.186+0000 7f0c1d7fa640 1 -- 172.21.15.40:0/1433368276 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v35)=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0c28159930 con 0x7f0c2815af70 2023-10-18T09:03:17.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.189+0000 7f0c2dc60640 1 -- 172.21.15.40:0/1433368276 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0c1406ae40 msgr2=0x7f0c1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:17.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.189+0000 7f0c2dc60640 1 --2- 172.21.15.40:0/1433368276 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0c1406ae40 0x7f0c1406d300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009b50 tx=0x7f0c24005f90 comp rx=0 tx=0).stop 2023-10-18T09:03:17.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.189+0000 7f0c2dc60640 1 -- 172.21.15.40:0/1433368276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c2815af70 msgr2=0x7f0c28158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.189+0000 7f0c2dc60640 1 --2- 172.21.15.40:0/1433368276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c2815af70 0x7f0c28158120 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f0c20037040 tx=0x7f0c20035a20 comp rx=0 tx=0).stop 2023-10-18T09:03:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.190+0000 7f0c2dc60640 1 -- 172.21.15.40:0/1433368276 shutdown_connections 2023-10-18T09:03:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.190+0000 7f0c2dc60640 1 --2- 172.21.15.40:0/1433368276 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0c1406ae40 0x7f0c1406d300 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.190+0000 7f0c2dc60640 1 --2- 172.21.15.40:0/1433368276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c2815af70 0x7f0c28158120 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.190+0000 7f0c2dc60640 1 --2- 172.21.15.40:0/1433368276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c280c9620 0x7f0c280cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:17.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.190+0000 7f0c2dc60640 1 --2- 172.21.15.40:0/1433368276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c28071270 0x7f0c28152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:17.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.190+0000 7f0c2dc60640 1 -- 172.21.15.40:0/1433368276 >> 172.21.15.40:0/1433368276 conn(0x7f0c2806ca10 msgr2=0x7f0c280b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:17.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.190+0000 7f0c2dc60640 1 -- 172.21.15.40:0/1433368276 shutdown_connections 2023-10-18T09:03:17.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.190+0000 7f0c2dc60640 1 -- 172.21.15.40:0/1433368276 wait complete. 2023-10-18T09:03:17.210 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T09:03:17.210 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:03:17.334 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:03:17.456 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T09:03:17.457 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:03:17.457 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:03:17.523 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:03:17.524 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:03:17.524 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:03:17.581 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:03:17.582 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:03:17.582 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:03:17.646 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:03:17.647 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:03:17.647 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:03:17.707 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:03:17.707 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T09:03:17.708 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:03:17.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.948+0000 7f8287fff640 1 -- 172.21.15.40:0/3744115826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82880ab0d0 msgr2=0x7f828806b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:17.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.948+0000 7f8287fff640 1 --2- 172.21.15.40:0/3744115826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82880ab0d0 0x7f828806b4d0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f828806af60 tx=0x7f828002be20 comp rx=0 tx=0).stop 2023-10-18T09:03:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.949+0000 7f8287fff640 1 -- 172.21.15.40:0/3744115826 shutdown_connections 2023-10-18T09:03:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.949+0000 7f8287fff640 1 --2- 172.21.15.40:0/3744115826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82880ab0d0 0x7f828806b4d0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.949+0000 7f8287fff640 1 --2- 172.21.15.40:0/3744115826 >> v2:172.21.15.144:3300/0 conn(0x7f8288071440 0x7f82880aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.949+0000 7f8287fff640 1 --2- 172.21.15.40:0/3744115826 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8288152560 0x7f8288070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.949+0000 7f8287fff640 1 -- 172.21.15.40:0/3744115826 >> 172.21.15.40:0/3744115826 conn(0x7f8288069720 msgr2=0x7f8288069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:17.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.949+0000 7f8287fff640 1 -- 172.21.15.40:0/3744115826 shutdown_connections 2023-10-18T09:03:17.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.949+0000 7f8287fff640 1 -- 172.21.15.40:0/3744115826 wait complete. 2023-10-18T09:03:17.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.951+0000 7f8287fff640 1 Processor -- start 2023-10-18T09:03:17.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.951+0000 7f8287fff640 1 -- start start 2023-10-18T09:03:17.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.952+0000 7f8287fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8288071440 0x7f8288150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:17.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.952+0000 7f8287fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82880ab0d0 0x7f828813c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:17.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.952+0000 7f8287fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8288152560 0x7f8288139a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:17.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.952+0000 7f8287fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828813aed0 con 0x7f82880ab0d0 2023-10-18T09:03:17.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.952+0000 7f8287fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828813c9b0 con 0x7f8288071440 2023-10-18T09:03:17.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.952+0000 7f8287fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828813a2c0 con 0x7f8288152560 2023-10-18T09:03:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.952+0000 7f827e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82880ab0d0 0x7f828813c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.952+0000 7f827e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82880ab0d0 0x7f828813c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54588/0 (socket says 172.21.15.40:54588) 2023-10-18T09:03:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.952+0000 7f827e7fc640 1 -- 172.21.15.40:0/3712128265 learned_addr learned my addr 172.21.15.40:0/3712128265 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:17.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.952+0000 7f8286ffd640 1 --2- 172.21.15.40:0/3712128265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8288071440 0x7f8288150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:17.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.952+0000 7f827e7fc640 1 -- 172.21.15.40:0/3712128265 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8288152560 msgr2=0x7f8288139a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:17.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.952+0000 7f827e7fc640 1 --2- 172.21.15.40:0/3712128265 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8288152560 0x7f8288139a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:17.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.952+0000 7f827e7fc640 1 -- 172.21.15.40:0/3712128265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8288071440 msgr2=0x7f8288150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:17.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.952+0000 7f827e7fc640 1 --2- 172.21.15.40:0/3712128265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8288071440 0x7f8288150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:17.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.953+0000 7f827e7fc640 1 -- 172.21.15.40:0/3712128265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8280009560 con 0x7f82880ab0d0 2023-10-18T09:03:17.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.953+0000 7f827e7fc640 1 --2- 172.21.15.40:0/3712128265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82880ab0d0 0x7f828813c470 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f827000c920 tx=0x7f827000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:17.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.953+0000 7f827ffff640 1 -- 172.21.15.40:0/3712128265 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270007c70 con 0x7f82880ab0d0 2023-10-18T09:03:17.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.953+0000 7f827ffff640 1 -- 172.21.15.40:0/3712128265 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f827000f450 con 0x7f82880ab0d0 2023-10-18T09:03:17.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.953+0000 7f827ffff640 1 -- 172.21.15.40:0/3712128265 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827000f860 con 0x7f82880ab0d0 2023-10-18T09:03:17.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.953+0000 7f8287fff640 1 -- 172.21.15.40:0/3712128265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828813a5a0 con 0x7f82880ab0d0 2023-10-18T09:03:17.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.953+0000 7f8287fff640 1 -- 172.21.15.40:0/3712128265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828813ab60 con 0x7f82880ab0d0 2023-10-18T09:03:17.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.954+0000 7f8287fff640 1 -- 172.21.15.40:0/3712128265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828813b040 con 0x7f82880ab0d0 2023-10-18T09:03:17.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.955+0000 7f827ffff640 1 -- 172.21.15.40:0/3712128265 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f827000f9c0 con 0x7f82880ab0d0 2023-10-18T09:03:17.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.956+0000 7f827ffff640 1 --2- 172.21.15.40:0/3712128265 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f826806ae40 0x7f826806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:17.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.956+0000 7f827ffff640 1 -- 172.21.15.40:0/3712128265 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8270089c40 con 0x7f82880ab0d0 2023-10-18T09:03:17.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.957+0000 7f8286ffd640 1 --2- 172.21.15.40:0/3712128265 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f826806ae40 0x7f826806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:17.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.958+0000 7f8286ffd640 1 --2- 172.21.15.40:0/3712128265 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f826806ae40 0x7f826806d300 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8278009f20 tx=0x7f82780094f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:17.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:17.960+0000 7f827ffff640 1 -- 172.21.15.40:0/3712128265 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8270054f50 con 0x7f82880ab0d0 2023-10-18T09:03:18.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.191+0000 7f8287fff640 1 -- 172.21.15.40:0/3712128265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8288066e00 con 0x7f82880ab0d0 2023-10-18T09:03:18.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.192+0000 7f827ffff640 1 -- 172.21.15.40:0/3712128265 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f827001d020 con 0x7f82880ab0d0 2023-10-18T09:03:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:18.195 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":59,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5648,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/700924196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":700924196},{"type":"v1","addr":"172.21.15.40:6833","nonce":700924196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5657,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/877414874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":877414874},{"type":"v1","addr":"172.21.15.144:6835","nonce":877414874}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6121,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2596943574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2596943574},{"type":"v1","addr":"172.21.15.144:6837","nonce":2596943574}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":7803,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/4289424556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":4289424556},{"type":"v1","addr":"172.21.15.40:6835","nonce":4289424556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59}],"filesystems":[]} 2023-10-18T09:03:18.195 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 59 2023-10-18T09:03:18.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.195+0000 7f8287fff640 1 -- 172.21.15.40:0/3712128265 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f826806ae40 msgr2=0x7f826806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:18.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.195+0000 7f8287fff640 1 --2- 172.21.15.40:0/3712128265 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f826806ae40 0x7f826806d300 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8278009f20 tx=0x7f82780094f0 comp rx=0 tx=0).stop 2023-10-18T09:03:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.195+0000 7f8287fff640 1 -- 172.21.15.40:0/3712128265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82880ab0d0 msgr2=0x7f828813c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.195+0000 7f8287fff640 1 --2- 172.21.15.40:0/3712128265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82880ab0d0 0x7f828813c470 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f827000c920 tx=0x7f827000cde0 comp rx=0 tx=0).stop 2023-10-18T09:03:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.195+0000 7f8287fff640 1 -- 172.21.15.40:0/3712128265 shutdown_connections 2023-10-18T09:03:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.195+0000 7f8287fff640 1 --2- 172.21.15.40:0/3712128265 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f826806ae40 0x7f826806d300 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.195+0000 7f8287fff640 1 --2- 172.21.15.40:0/3712128265 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8288152560 0x7f8288139a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.195+0000 7f8287fff640 1 --2- 172.21.15.40:0/3712128265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82880ab0d0 0x7f828813c470 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.195+0000 7f8287fff640 1 --2- 172.21.15.40:0/3712128265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8288071440 0x7f8288150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.195+0000 7f8287fff640 1 -- 172.21.15.40:0/3712128265 >> 172.21.15.40:0/3712128265 conn(0x7f8288069720 msgr2=0x7f828814e260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.195+0000 7f8287fff640 1 -- 172.21.15.40:0/3712128265 shutdown_connections 2023-10-18T09:03:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.195+0000 7f8287fff640 1 -- 172.21.15.40:0/3712128265 wait complete. 2023-10-18T09:03:18.218 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T09:03:18.218 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T09:03:18.218 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T09:03:18.219 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:03:18.219 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T09:03:18.219 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T09:03:18.220 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T09:03:18.220 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:03:18.220 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T09:03:18.220 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T09:03:18.221 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T09:03:18.221 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:03:18.221 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T09:03:18.221 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T09:03:18.222 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T09:03:18.222 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:03:18.236 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T09:03:18.237 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T09:03:18.238 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T09:03:18.238 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T09:03:18.245 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T09:03:18.245 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T09:03:18.248 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T09:03:18.248 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T09:03:18.286 INFO:tasks.ceph.mds.a:Started 2023-10-18T09:03:18.287 DEBUG:teuthology.parallel:result is None 2023-10-18T09:03:18.293 INFO:tasks.ceph.mds.c:Started 2023-10-18T09:03:18.294 DEBUG:teuthology.parallel:result is None 2023-10-18T09:03:18.296 INFO:tasks.ceph.mds.b:Started 2023-10-18T09:03:18.296 DEBUG:teuthology.parallel:result is None 2023-10-18T09:03:18.304 INFO:tasks.ceph.mds.d:Started 2023-10-18T09:03:18.304 DEBUG:teuthology.parallel:result is None 2023-10-18T09:03:18.305 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T09:03:18.449 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-10-18T09:03:18.454 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T09:03:18.462 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-18T09:03:18.462 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-18T09:03:18.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.500+0000 7ffb08e4b640 1 -- 172.21.15.40:0/3798041115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb040cd3e0 msgr2=0x7ffb0415c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:18.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.500+0000 7ffb08e4b640 1 --2- 172.21.15.40:0/3798041115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb040cd3e0 0x7ffb0415c860 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ffb040ca970 tx=0x7ffaf402bda0 comp rx=0 tx=0).stop 2023-10-18T09:03:18.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.500+0000 7ffb08e4b640 1 -- 172.21.15.40:0/3798041115 shutdown_connections 2023-10-18T09:03:18.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.500+0000 7ffb08e4b640 1 --2- 172.21.15.40:0/3798041115 >> v2:172.21.15.144:3300/0 conn(0x7ffb040b0620 0x7ffb040b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:18.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.500+0000 7ffb08e4b640 1 --2- 172.21.15.40:0/3798041115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb040cd7c0 0x7ffb040b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:18.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.500+0000 7ffb08e4b640 1 --2- 172.21.15.40:0/3798041115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb040cd3e0 0x7ffb0415c860 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:18.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.500+0000 7ffb08e4b640 1 -- 172.21.15.40:0/3798041115 >> 172.21.15.40:0/3798041115 conn(0x7ffb0406cee0 msgr2=0x7ffb04069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:18.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.500+0000 7ffb08e4b640 1 -- 172.21.15.40:0/3798041115 shutdown_connections 2023-10-18T09:03:18.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.500+0000 7ffb08e4b640 1 -- 172.21.15.40:0/3798041115 wait complete. 2023-10-18T09:03:18.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.501+0000 7ffb08e4b640 1 Processor -- start 2023-10-18T09:03:18.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.502+0000 7ffb08e4b640 1 -- start start 2023-10-18T09:03:18.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.502+0000 7ffb08e4b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb040b0620 0x7ffb04147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:18.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.503+0000 7ffb08e4b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb040cd3e0 0x7ffb04135bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:18.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.503+0000 7ffb08e4b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb040cd7c0 0x7ffb040caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:18.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.503+0000 7ffb08e4b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0414a7f0 con 0x7ffb040cd7c0 2023-10-18T09:03:18.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.503+0000 7ffb08e4b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb04149900 con 0x7ffb040cd3e0 2023-10-18T09:03:18.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.503+0000 7ffb08e4b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb04149be0 con 0x7ffb040b0620 2023-10-18T09:03:18.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.503+0000 7ffb02d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb040cd7c0 0x7ffb040caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:18.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.503+0000 7ffb02d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb040cd7c0 0x7ffb040caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54622/0 (socket says 172.21.15.40:54622) 2023-10-18T09:03:18.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.503+0000 7ffb02d9d640 1 -- 172.21.15.40:0/2357721670 learned_addr learned my addr 172.21.15.40:0/2357721670 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:18.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.503+0000 7ffafbfff640 1 --2- 172.21.15.40:0/2357721670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb040cd3e0 0x7ffb04135bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:18.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.503+0000 7ffb0259c640 1 --2- 172.21.15.40:0/2357721670 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb040b0620 0x7ffb04147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.503+0000 7ffb02d9d640 1 -- 172.21.15.40:0/2357721670 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb040b0620 msgr2=0x7ffb04147a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.503+0000 7ffb02d9d640 1 --2- 172.21.15.40:0/2357721670 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb040b0620 0x7ffb04147a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.503+0000 7ffb02d9d640 1 -- 172.21.15.40:0/2357721670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb040cd3e0 msgr2=0x7ffb04135bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.503+0000 7ffb02d9d640 1 --2- 172.21.15.40:0/2357721670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb040cd3e0 0x7ffb04135bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.503+0000 7ffb02d9d640 1 -- 172.21.15.40:0/2357721670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf4009560 con 0x7ffb040cd7c0 2023-10-18T09:03:18.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.503+0000 7ffafbfff640 1 --2- 172.21.15.40:0/2357721670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb040cd3e0 0x7ffb04135bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:03:18.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.504+0000 7ffb02d9d640 1 --2- 172.21.15.40:0/2357721670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb040cd7c0 0x7ffb040caeb0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00cce0 tx=0x7ffafc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:18.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.504+0000 7ffafaffd640 1 -- 172.21.15.40:0/2357721670 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00f450 con 0x7ffb040cd7c0 2023-10-18T09:03:18.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.504+0000 7ffafaffd640 1 -- 172.21.15.40:0/2357721670 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffafc00f6a0 con 0x7ffb040cd7c0 2023-10-18T09:03:18.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.504+0000 7ffb08e4b640 1 -- 172.21.15.40:0/2357721670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb04149ec0 con 0x7ffb040cd7c0 2023-10-18T09:03:18.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.504+0000 7ffafaffd640 1 -- 172.21.15.40:0/2357721670 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00fab0 con 0x7ffb040cd7c0 2023-10-18T09:03:18.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.504+0000 7ffb08e4b640 1 -- 172.21.15.40:0/2357721670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0414a480 con 0x7ffb040cd7c0 2023-10-18T09:03:18.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.505+0000 7ffb08e4b640 1 -- 172.21.15.40:0/2357721670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb041f3290 con 0x7ffb040cd7c0 2023-10-18T09:03:18.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.506+0000 7ffafaffd640 1 -- 172.21.15.40:0/2357721670 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffafc00fc10 con 0x7ffb040cd7c0 2023-10-18T09:03:18.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.507+0000 7ffafaffd640 1 --2- 172.21.15.40:0/2357721670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffae406ae40 0x7ffae406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:18.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.507+0000 7ffafaffd640 1 -- 172.21.15.40:0/2357721670 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffafc089f10 con 0x7ffb040cd7c0 2023-10-18T09:03:18.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.508+0000 7ffb0259c640 1 --2- 172.21.15.40:0/2357721670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffae406ae40 0x7ffae406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:18.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.509+0000 7ffb0259c640 1 --2- 172.21.15.40:0/2357721670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffae406ae40 0x7ffae406d300 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4035000 tx=0x7ffaf4004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:18.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.512+0000 7ffafaffd640 1 -- 172.21.15.40:0/2357721670 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffafc0552c0 con 0x7ffb040cd7c0 2023-10-18T09:03:18.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:18.717+0000 7ffb08e4b640 1 -- 172.21.15.40:0/2357721670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ffb04066fd0 con 0x7ffb040cd7c0 2023-10-18T09:03:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.291+0000 7ffafaffd640 1 -- 172.21.15.40:0/2357721670 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v138) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffafc0588e0 con 0x7ffb040cd7c0 2023-10-18T09:03:19.294 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2023-10-18T09:03:19.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.295+0000 7ffb08e4b640 1 -- 172.21.15.40:0/2357721670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffae406ae40 msgr2=0x7ffae406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:19.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.295+0000 7ffb08e4b640 1 --2- 172.21.15.40:0/2357721670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffae406ae40 0x7ffae406d300 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4035000 tx=0x7ffaf4004730 comp rx=0 tx=0).stop 2023-10-18T09:03:19.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.295+0000 7ffb08e4b640 1 -- 172.21.15.40:0/2357721670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb040cd7c0 msgr2=0x7ffb040caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:19.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.295+0000 7ffb08e4b640 1 --2- 172.21.15.40:0/2357721670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb040cd7c0 0x7ffb040caeb0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00cce0 tx=0x7ffafc00a620 comp rx=0 tx=0).stop 2023-10-18T09:03:19.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.295+0000 7ffb08e4b640 1 -- 172.21.15.40:0/2357721670 shutdown_connections 2023-10-18T09:03:19.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.295+0000 7ffb08e4b640 1 --2- 172.21.15.40:0/2357721670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ffae406ae40 0x7ffae406d300 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:19.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.295+0000 7ffb08e4b640 1 --2- 172.21.15.40:0/2357721670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb040cd7c0 0x7ffb040caeb0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:19.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.295+0000 7ffb08e4b640 1 --2- 172.21.15.40:0/2357721670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb040cd3e0 0x7ffb04135bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.295+0000 7ffb08e4b640 1 --2- 172.21.15.40:0/2357721670 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb040b0620 0x7ffb04147a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.295+0000 7ffb08e4b640 1 -- 172.21.15.40:0/2357721670 >> 172.21.15.40:0/2357721670 conn(0x7ffb0406cee0 msgr2=0x7ffb04069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.295+0000 7ffb08e4b640 1 -- 172.21.15.40:0/2357721670 shutdown_connections 2023-10-18T09:03:19.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.295+0000 7ffb08e4b640 1 -- 172.21.15.40:0/2357721670 wait complete. 2023-10-18T09:03:19.320 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T09:03:19.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.550+0000 7fb3cdabb640 1 -- 172.21.15.40:0/4148777676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b0003970 msgr2=0x7fb3b0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:19.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.550+0000 7fb3cdabb640 1 --2- 172.21.15.40:0/4148777676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b0003970 0x7fb3b0003d90 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4001680 tx=0x7fb3b00057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:19.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.551+0000 7fb3cdabb640 1 -- 172.21.15.40:0/4148777676 shutdown_connections 2023-10-18T09:03:19.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.551+0000 7fb3cdabb640 1 --2- 172.21.15.40:0/4148777676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b0003970 0x7fb3b0003d90 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:19.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.551+0000 7fb3cdabb640 1 -- 172.21.15.40:0/4148777676 >> 172.21.15.40:0/4148777676 conn(0x7fb3c8069720 msgr2=0x7fb3c8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:19.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.551+0000 7fb3cdabb640 1 -- 172.21.15.40:0/4148777676 shutdown_connections 2023-10-18T09:03:19.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.551+0000 7fb3cdabb640 1 -- 172.21.15.40:0/4148777676 wait complete. 2023-10-18T09:03:19.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.552+0000 7fb3cdabb640 1 Processor -- start 2023-10-18T09:03:19.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.553+0000 7fb3cdabb640 1 -- start start 2023-10-18T09:03:19.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.553+0000 7fb3cdabb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b0003970 0x7fb3c8159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:19.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.553+0000 7fb3cdabb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3c8159e90 0x7fb3c813f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:19.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.553+0000 7fb3cdabb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c813f670 0x7fb3c813faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:19.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.553+0000 7fb3cdabb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c813aed0 con 0x7fb3b0003970 2023-10-18T09:03:19.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.553+0000 7fb3cdabb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c815a310 con 0x7fb3c813f670 2023-10-18T09:03:19.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.553+0000 7fb3cdabb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c81ef5e0 con 0x7fb3c8159e90 2023-10-18T09:03:19.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.554+0000 7fb3ccab9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b0003970 0x7fb3c8159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:19.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.554+0000 7fb3ccab9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b0003970 0x7fb3c8159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54640/0 (socket says 172.21.15.40:54640) 2023-10-18T09:03:19.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.554+0000 7fb3ccab9640 1 -- 172.21.15.40:0/3903398942 learned_addr learned my addr 172.21.15.40:0/3903398942 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:19.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.554+0000 7fb3cd2ba640 1 --2- 172.21.15.40:0/3903398942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c813f670 0x7fb3c813faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:19.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.554+0000 7fb3ccab9640 1 -- 172.21.15.40:0/3903398942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3c8159e90 msgr2=0x7fb3c813f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:19.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.554+0000 7fb3bffff640 1 --2- 172.21.15.40:0/3903398942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3c8159e90 0x7fb3c813f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:19.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.554+0000 7fb3ccab9640 1 --2- 172.21.15.40:0/3903398942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3c8159e90 0x7fb3c813f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:19.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.554+0000 7fb3ccab9640 1 -- 172.21.15.40:0/3903398942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c813f670 msgr2=0x7fb3c813faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:19.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.554+0000 7fb3ccab9640 1 --2- 172.21.15.40:0/3903398942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c813f670 0x7fb3c813faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:19.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.554+0000 7fb3ccab9640 1 -- 172.21.15.40:0/3903398942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c815a580 con 0x7fb3b0003970 2023-10-18T09:03:19.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.554+0000 7fb3cd2ba640 1 --2- 172.21.15.40:0/3903398942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c813f670 0x7fb3c813faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:19.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.555+0000 7fb3ccab9640 1 --2- 172.21.15.40:0/3903398942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b0003970 0x7fb3c8159950 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40017a0 tx=0x7fb3c4006360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.555+0000 7fb3bd7fa640 1 -- 172.21.15.40:0/3903398942 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4006540 con 0x7fb3b0003970 2023-10-18T09:03:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.555+0000 7fb3bd7fa640 1 -- 172.21.15.40:0/3903398942 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb3c4003c70 con 0x7fb3b0003970 2023-10-18T09:03:19.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.555+0000 7fb3bd7fa640 1 -- 172.21.15.40:0/3903398942 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c402c790 con 0x7fb3b0003970 2023-10-18T09:03:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.555+0000 7fb3cdabb640 1 -- 172.21.15.40:0/3903398942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c815a860 con 0x7fb3b0003970 2023-10-18T09:03:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.555+0000 7fb3cdabb640 1 -- 172.21.15.40:0/3903398942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c815ae20 con 0x7fb3b0003970 2023-10-18T09:03:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.556+0000 7fb3cdabb640 1 -- 172.21.15.40:0/3903398942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c8066e00 con 0x7fb3b0003970 2023-10-18T09:03:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.557+0000 7fb3bd7fa640 1 -- 172.21.15.40:0/3903398942 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb3c402c8f0 con 0x7fb3b0003970 2023-10-18T09:03:19.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.558+0000 7fb3bd7fa640 1 --2- 172.21.15.40:0/3903398942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3b406ae70 0x7fb3b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:19.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.558+0000 7fb3bd7fa640 1 -- 172.21.15.40:0/3903398942 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb3c40aa650 con 0x7fb3b0003970 2023-10-18T09:03:19.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.559+0000 7fb3bffff640 1 --2- 172.21.15.40:0/3903398942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3b406ae70 0x7fb3b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:19.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.560+0000 7fb3bffff640 1 --2- 172.21.15.40:0/3903398942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3b406ae70 0x7fb3b406d330 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80058c0 tx=0x7fb3b8005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:19.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.563+0000 7fb3bd7fa640 1 -- 172.21.15.40:0/3903398942 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3c40759b0 con 0x7fb3b0003970 2023-10-18T09:03:19.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.825+0000 7fb3cdabb640 1 -- 172.21.15.40:0/3903398942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb3c806d920 con 0x7fb3b0003970 2023-10-18T09:03:19.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.826+0000 7fb3bd7fa640 1 -- 172.21.15.40:0/3903398942 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb3c4044070 con 0x7fb3b0003970 2023-10-18T09:03:19.829 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:19.830 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:03:19.830 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2023-10-18T09:03:19.830 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.830 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2023-10-18T09:03:19.830 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lYsb5CxAAGButZ6+Usc62kJ0ksxZrYw==", 2023-10-18T09:03:19.831 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.831 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:03:19.831 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:03:19.831 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:03:19.832 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:03:19.832 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.832 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.832 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.832 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2023-10-18T09:03:19.833 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lAqlhMBAApAhxCYB9d9bz1GibADDRSQ==", 2023-10-18T09:03:19.833 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.833 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:03:19.833 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:03:19.833 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:03:19.834 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:03:19.834 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.834 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.834 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.835 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2023-10-18T09:03:19.835 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lPVh+HhAAzucSx+ksPDBkqJGa8+pyJw==", 2023-10-18T09:03:19.835 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.835 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:03:19.835 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:03:19.836 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:03:19.836 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:03:19.836 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.836 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.836 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.837 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2023-10-18T09:03:19.837 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lP7QCBhAAog7qDCgGVv0T7WKsRkATkA==", 2023-10-18T09:03:19.837 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.837 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:03:19.837 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:03:19.838 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:03:19.838 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:03:19.838 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.838 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.839 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.839 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2023-10-18T09:03:19.839 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lVx+hAhAA6ZpIyZKhbqy8tV84piGILA==", 2023-10-18T09:03:19.839 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.839 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:03:19.840 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:03:19.840 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:03:19.840 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.840 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.840 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.841 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2023-10-18T09:03:19.841 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lk9LwExAA7zFzQOn2hcvnck2cOZZ7AA==", 2023-10-18T09:03:19.841 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.841 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:03:19.841 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:03:19.842 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:03:19.842 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.842 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.843 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2023-10-18T09:03:19.843 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lK930JRAA1GyKRkIpXPHHGqY8owYWRQ==", 2023-10-18T09:03:19.843 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.843 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:03:19.843 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:03:19.844 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:03:19.844 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.844 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.844 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2023-10-18T09:03:19.845 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9liBWyNRAAMHBBdG5nbrGQupqvUtuKMA==", 2023-10-18T09:03:19.845 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.845 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:03:19.845 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:03:19.845 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:03:19.846 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.846 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2023-10-18T09:03:19.846 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9loDoOAhAAjHZub4wnB7dCr86rxAtI5g==", 2023-10-18T09:03:19.847 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.847 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:03:19.847 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:03:19.847 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:03:19.848 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.848 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2023-10-18T09:03:19.848 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lhqNfERAAw410g3DwSXrK23YkxVBFVw==", 2023-10-18T09:03:19.849 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.849 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:03:19.849 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:03:19.849 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:03:19.849 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.850 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2023-10-18T09:03:19.850 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lRRScIRAAg0pmfxCQBCPYodj1Lko+Lw==", 2023-10-18T09:03:19.850 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:03:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:03:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:03:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2023-10-18T09:03:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9ld1dXMhAAyovlrKOhVuynEiuPTgFcjQ==", 2023-10-18T09:03:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:03:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:03:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:03:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2023-10-18T09:03:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w==", 2023-10-18T09:03:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:03:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2023-10-18T09:03:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T09:03:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2023-10-18T09:03:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lAB3oKhAAD43WFdC23WGF6WUi4JKsvg==", 2023-10-18T09:03:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:03:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:03:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:03:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:03:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2023-10-18T09:03:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lu57fHxAAdzsVU2KKsWRkaARRS8TGjw==", 2023-10-18T09:03:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:03:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:03:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:03:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:03:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2023-10-18T09:03:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lygWsNRAA1OziXxk7hBtTw2Aq8OlSKQ==", 2023-10-18T09:03:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:03:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:03:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:03:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:03:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2023-10-18T09:03:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBMnS9lnrUjNhAAHP2eezRHF4/Jgox7iPwy7g==", 2023-10-18T09:03:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:03:19.863 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:03:19.863 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:03:19.863 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:03:19.863 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.864 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2023-10-18T09:03:19.864 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9lCf/xOhAAJHVoKIswWUcVGKQpiaJ12A==", 2023-10-18T09:03:19.864 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.865 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:03:19.865 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T09:03:19.865 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:03:19.865 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:19.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:19.866 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2023-10-18T09:03:19.866 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9loxRXLhAAQK1n1/4PB35umHxOKQzz7Q==", 2023-10-18T09:03:19.866 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:03:19.867 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:03:19.867 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T09:03:19.867 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:03:19.867 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.867 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:19.868 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:19.868 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:03:19.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.828+0000 7fb3cdabb640 1 -- 172.21.15.40:0/3903398942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3b406ae70 msgr2=0x7fb3b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:19.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.828+0000 7fb3cdabb640 1 --2- 172.21.15.40:0/3903398942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3b406ae70 0x7fb3b406d330 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80058c0 tx=0x7fb3b8005850 comp rx=0 tx=0).stop 2023-10-18T09:03:19.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.828+0000 7fb3cdabb640 1 -- 172.21.15.40:0/3903398942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b0003970 msgr2=0x7fb3c8159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:19.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.828+0000 7fb3cdabb640 1 --2- 172.21.15.40:0/3903398942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b0003970 0x7fb3c8159950 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40017a0 tx=0x7fb3c4006360 comp rx=0 tx=0).stop 2023-10-18T09:03:19.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.828+0000 7fb3cdabb640 1 -- 172.21.15.40:0/3903398942 shutdown_connections 2023-10-18T09:03:19.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.828+0000 7fb3cdabb640 1 --2- 172.21.15.40:0/3903398942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3b406ae70 0x7fb3b406d330 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:19.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.828+0000 7fb3cdabb640 1 --2- 172.21.15.40:0/3903398942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c813f670 0x7fb3c813faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:19.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.828+0000 7fb3cdabb640 1 --2- 172.21.15.40:0/3903398942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3c8159e90 0x7fb3c813f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:19.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.828+0000 7fb3cdabb640 1 --2- 172.21.15.40:0/3903398942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3b0003970 0x7fb3c8159950 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:19.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.828+0000 7fb3cdabb640 1 -- 172.21.15.40:0/3903398942 >> 172.21.15.40:0/3903398942 conn(0x7fb3c8069720 msgr2=0x7fb3c80c9260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:19.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.829+0000 7fb3cdabb640 1 -- 172.21.15.40:0/3903398942 shutdown_connections 2023-10-18T09:03:19.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:19.829+0000 7fb3cdabb640 1 -- 172.21.15.40:0/3903398942 wait complete. 2023-10-18T09:03:19.872 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:03:20.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.103+0000 7fdd62f93640 1 -- 172.21.15.40:0/1621009804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5c0ca630 msgr2=0x7fdd5c0cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:20.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.103+0000 7fdd62f93640 1 --2- 172.21.15.40:0/1621009804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5c0ca630 0x7fdd5c0cd0c0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c06fd70 tx=0x7fdd5802be40 comp rx=0 tx=0).stop 2023-10-18T09:03:20.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.104+0000 7fdd62f93640 1 -- 172.21.15.40:0/1621009804 shutdown_connections 2023-10-18T09:03:20.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.104+0000 7fdd62f93640 1 --2- 172.21.15.40:0/1621009804 >> v2:172.21.15.144:3300/0 conn(0x7fdd5c0caa10 0x7fdd5c06be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:20.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.104+0000 7fdd62f93640 1 --2- 172.21.15.40:0/1621009804 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5c0ca630 0x7fdd5c0cd0c0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:20.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.104+0000 7fdd62f93640 1 -- 172.21.15.40:0/1621009804 >> 172.21.15.40:0/1621009804 conn(0x7fdd5c069b40 msgr2=0x7fdd5c069f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:20.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.104+0000 7fdd62f93640 1 -- 172.21.15.40:0/1621009804 shutdown_connections 2023-10-18T09:03:20.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.104+0000 7fdd62f93640 1 -- 172.21.15.40:0/1621009804 wait complete. 2023-10-18T09:03:20.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.106+0000 7fdd62f93640 1 Processor -- start 2023-10-18T09:03:20.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.106+0000 7fdd62f93640 1 -- start start 2023-10-18T09:03:20.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.107+0000 7fdd62f93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5c0ca630 0x7fdd5c0aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:20.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.107+0000 7fdd62f93640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd5c0caa10 0x7fdd5c13fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:20.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.107+0000 7fdd62f93640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd5c15cd20 0x7fdd5c1f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:20.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.107+0000 7fdd62f93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c140320 con 0x7fdd5c0ca630 2023-10-18T09:03:20.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.107+0000 7fdd62f93640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c1f17f0 con 0x7fdd5c15cd20 2023-10-18T09:03:20.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.107+0000 7fdd62f93640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c1407f0 con 0x7fdd5c0caa10 2023-10-18T09:03:20.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.107+0000 7fdd61f91640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5c0ca630 0x7fdd5c0aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:20.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.107+0000 7fdd61f91640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5c0ca630 0x7fdd5c0aa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54664/0 (socket says 172.21.15.40:54664) 2023-10-18T09:03:20.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.107+0000 7fdd61f91640 1 -- 172.21.15.40:0/4173613876 learned_addr learned my addr 172.21.15.40:0/4173613876 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:20.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.108+0000 7fdd61f91640 1 -- 172.21.15.40:0/4173613876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd5c0caa10 msgr2=0x7fdd5c13fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:20.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.108+0000 7fdd61f91640 1 --2- 172.21.15.40:0/4173613876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd5c0caa10 0x7fdd5c13fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:20.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.108+0000 7fdd61f91640 1 -- 172.21.15.40:0/4173613876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd5c15cd20 msgr2=0x7fdd5c1f58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:20.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.108+0000 7fdd62792640 1 --2- 172.21.15.40:0/4173613876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd5c15cd20 0x7fdd5c1f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:20.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.108+0000 7fdd61f91640 1 --2- 172.21.15.40:0/4173613876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd5c15cd20 0x7fdd5c1f58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:20.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.108+0000 7fdd61f91640 1 -- 172.21.15.40:0/4173613876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd58009560 con 0x7fdd5c0ca630 2023-10-18T09:03:20.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.108+0000 7fdd61f91640 1 --2- 172.21.15.40:0/4173613876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5c0ca630 0x7fdd5c0aa880 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fdd58036040 tx=0x7fdd5802db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:20.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.108+0000 7fdd527fc640 1 -- 172.21.15.40:0/4173613876 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58003800 con 0x7fdd5c0ca630 2023-10-18T09:03:20.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.108+0000 7fdd527fc640 1 -- 172.21.15.40:0/4173613876 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdd58003a50 con 0x7fdd5c0ca630 2023-10-18T09:03:20.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.108+0000 7fdd527fc640 1 -- 172.21.15.40:0/4173613876 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58003e20 con 0x7fdd5c0ca630 2023-10-18T09:03:20.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.108+0000 7fdd62f93640 1 -- 172.21.15.40:0/4173613876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c070f70 con 0x7fdd5c0ca630 2023-10-18T09:03:20.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.108+0000 7fdd62f93640 1 -- 172.21.15.40:0/4173613876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c159280 con 0x7fdd5c0ca630 2023-10-18T09:03:20.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.110+0000 7fdd62f93640 1 -- 172.21.15.40:0/4173613876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c073250 con 0x7fdd5c0ca630 2023-10-18T09:03:20.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.111+0000 7fdd527fc640 1 -- 172.21.15.40:0/4173613876 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdd5803e450 con 0x7fdd5c0ca630 2023-10-18T09:03:20.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.111+0000 7fdd527fc640 1 --2- 172.21.15.40:0/4173613876 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdd4006ae70 0x7fdd4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:20.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.112+0000 7fdd527fc640 1 -- 172.21.15.40:0/4173613876 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdd580abf10 con 0x7fdd5c0ca630 2023-10-18T09:03:20.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.112+0000 7fdd61790640 1 --2- 172.21.15.40:0/4173613876 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdd4006ae70 0x7fdd4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:20.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.113+0000 7fdd61790640 1 --2- 172.21.15.40:0/4173613876 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdd4006ae70 0x7fdd4006d330 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c005fd0 tx=0x7fdd4c005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:20.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.116+0000 7fdd527fc640 1 -- 172.21.15.40:0/4173613876 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd58077390 con 0x7fdd5c0ca630 2023-10-18T09:03:20.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.313+0000 7fdd62f93640 1 -- 172.21.15.40:0/4173613876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd5c066e00 con 0x7fdd5c0ca630 2023-10-18T09:03:20.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.315+0000 7fdd527fc640 1 -- 172.21.15.40:0/4173613876 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+12880 (secure 0 0 0) 0x7fdd58042020 con 0x7fdd5c0ca630 2023-10-18T09:03:20.318 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:20.319 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":140,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:03:19.309230+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:03:20.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.318+0000 7fdd62f93640 1 -- 172.21.15.40:0/4173613876 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdd4006ae70 msgr2=0x7fdd4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:20.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.318+0000 7fdd62f93640 1 --2- 172.21.15.40:0/4173613876 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdd4006ae70 0x7fdd4006d330 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c005fd0 tx=0x7fdd4c005850 comp rx=0 tx=0).stop 2023-10-18T09:03:20.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.318+0000 7fdd62f93640 1 -- 172.21.15.40:0/4173613876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5c0ca630 msgr2=0x7fdd5c0aa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:20.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.318+0000 7fdd62f93640 1 --2- 172.21.15.40:0/4173613876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5c0ca630 0x7fdd5c0aa880 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fdd58036040 tx=0x7fdd5802db70 comp rx=0 tx=0).stop 2023-10-18T09:03:20.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.318+0000 7fdd62f93640 1 -- 172.21.15.40:0/4173613876 shutdown_connections 2023-10-18T09:03:20.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.318+0000 7fdd62f93640 1 --2- 172.21.15.40:0/4173613876 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdd4006ae70 0x7fdd4006d330 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:20.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.318+0000 7fdd62f93640 1 --2- 172.21.15.40:0/4173613876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd5c15cd20 0x7fdd5c1f58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:20.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.318+0000 7fdd62f93640 1 --2- 172.21.15.40:0/4173613876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdd5c0caa10 0x7fdd5c13fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:20.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.318+0000 7fdd62f93640 1 --2- 172.21.15.40:0/4173613876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5c0ca630 0x7fdd5c0aa880 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:20.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.318+0000 7fdd62f93640 1 -- 172.21.15.40:0/4173613876 >> 172.21.15.40:0/4173613876 conn(0x7fdd5c069b40 msgr2=0x7fdd5c0cbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:20.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.318+0000 7fdd62f93640 1 -- 172.21.15.40:0/4173613876 shutdown_connections 2023-10-18T09:03:20.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.318+0000 7fdd62f93640 1 -- 172.21.15.40:0/4173613876 wait complete. 2023-10-18T09:03:20.342 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:03:20.342 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:03:20.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.562+0000 7fecba207640 1 -- 172.21.15.40:0/1956442382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feca0005180 msgr2=0x7feca000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:20.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.562+0000 7fecba207640 1 --2- 172.21.15.40:0/1956442382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feca0005180 0x7feca000b980 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fecac001d80 tx=0x7feca00057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:20.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.563+0000 7fecba207640 1 -- 172.21.15.40:0/1956442382 shutdown_connections 2023-10-18T09:03:20.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.563+0000 7fecba207640 1 --2- 172.21.15.40:0/1956442382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feca0005180 0x7feca000b980 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:20.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.563+0000 7fecba207640 1 -- 172.21.15.40:0/1956442382 >> 172.21.15.40:0/1956442382 conn(0x7fecb4069720 msgr2=0x7fecb4069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:20.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.563+0000 7fecba207640 1 -- 172.21.15.40:0/1956442382 shutdown_connections 2023-10-18T09:03:20.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.563+0000 7fecba207640 1 -- 172.21.15.40:0/1956442382 wait complete. 2023-10-18T09:03:20.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.564+0000 7fecba207640 1 Processor -- start 2023-10-18T09:03:20.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.565+0000 7fecba207640 1 -- start start 2023-10-18T09:03:20.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.565+0000 7fecba207640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feca0005180 0x7fecb413bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:20.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.566+0000 7fecba207640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecb41e3830 0x7fecb41f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:20.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.566+0000 7fecba207640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecb41f2460 0x7fecb41e4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:20.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.566+0000 7fecba207640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb415ab40 con 0x7feca0005180 2023-10-18T09:03:20.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.566+0000 7fecba207640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb415ae20 con 0x7fecb41e3830 2023-10-18T09:03:20.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.566+0000 7fecba207640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb415b100 con 0x7fecb41f2460 2023-10-18T09:03:20.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.566+0000 7fecb37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feca0005180 0x7fecb413bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:20.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.566+0000 7fecb37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feca0005180 0x7fecb413bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54680/0 (socket says 172.21.15.40:54680) 2023-10-18T09:03:20.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.566+0000 7fecb37fe640 1 -- 172.21.15.40:0/2352503337 learned_addr learned my addr 172.21.15.40:0/2352503337 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:20.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.566+0000 7fecb37fe640 1 -- 172.21.15.40:0/2352503337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecb41f2460 msgr2=0x7fecb41e4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:20.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.566+0000 7fecb37fe640 1 --2- 172.21.15.40:0/2352503337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecb41f2460 0x7fecb41e4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:20.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.566+0000 7fecb37fe640 1 -- 172.21.15.40:0/2352503337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecb41e3830 msgr2=0x7fecb41f1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:20.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.566+0000 7fecb37fe640 1 --2- 172.21.15.40:0/2352503337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecb41e3830 0x7fecb41f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:20.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.566+0000 7fecb37fe640 1 -- 172.21.15.40:0/2352503337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb415b3e0 con 0x7feca0005180 2023-10-18T09:03:20.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.566+0000 7fecb37fe640 1 --2- 172.21.15.40:0/2352503337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feca0005180 0x7fecb413bb50 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7feca4001200 tx=0x7feca400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:20.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.567+0000 7fecb0ff9640 1 -- 172.21.15.40:0/2352503337 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca400de60 con 0x7feca0005180 2023-10-18T09:03:20.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.567+0000 7fecb0ff9640 1 -- 172.21.15.40:0/2352503337 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feca40052e0 con 0x7feca0005180 2023-10-18T09:03:20.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.567+0000 7fecb0ff9640 1 -- 172.21.15.40:0/2352503337 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca400b8f0 con 0x7feca0005180 2023-10-18T09:03:20.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.567+0000 7fecba207640 1 -- 172.21.15.40:0/2352503337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb415b6c0 con 0x7feca0005180 2023-10-18T09:03:20.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.567+0000 7fecba207640 1 -- 172.21.15.40:0/2352503337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb4159190 con 0x7feca0005180 2023-10-18T09:03:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.568+0000 7fecba207640 1 -- 172.21.15.40:0/2352503337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb415bbe0 con 0x7feca0005180 2023-10-18T09:03:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.569+0000 7fecb0ff9640 1 -- 172.21.15.40:0/2352503337 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feca400ba50 con 0x7feca0005180 2023-10-18T09:03:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.570+0000 7fecb0ff9640 1 --2- 172.21.15.40:0/2352503337 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fec9406ae40 0x7fec9406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.570+0000 7fecb0ff9640 1 -- 172.21.15.40:0/2352503337 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feca4088cc0 con 0x7feca0005180 2023-10-18T09:03:20.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.571+0000 7fecabfff640 1 --2- 172.21.15.40:0/2352503337 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fec9406ae40 0x7fec9406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:20.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.572+0000 7fecabfff640 1 --2- 172.21.15.40:0/2352503337 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fec9406ae40 0x7fec9406d300 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fecb415a9f0 tx=0x7fec9c009530 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:20.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.574+0000 7fecb0ff9640 1 -- 172.21.15.40:0/2352503337 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fecb415bbe0 con 0x7feca0005180 2023-10-18T09:03:20.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.768+0000 7fecba207640 1 -- 172.21.15.40:0/2352503337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fecb415bbe0 con 0x7feca0005180 2023-10-18T09:03:20.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.769+0000 7fecb0ff9640 1 -- 172.21.15.40:0/2352503337 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) v1 ==== 93+0+10 (secure 0 0 0) 0x7fecb415bbe0 con 0x7feca0005180 2023-10-18T09:03:20.772 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:03:20.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.771+0000 7fecba207640 1 -- 172.21.15.40:0/2352503337 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fec9406ae40 msgr2=0x7fec9406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:20.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.771+0000 7fecba207640 1 --2- 172.21.15.40:0/2352503337 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fec9406ae40 0x7fec9406d300 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fecb415a9f0 tx=0x7fec9c009530 comp rx=0 tx=0).stop 2023-10-18T09:03:20.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.771+0000 7fecba207640 1 -- 172.21.15.40:0/2352503337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feca0005180 msgr2=0x7fecb413bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:20.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.771+0000 7fecba207640 1 --2- 172.21.15.40:0/2352503337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feca0005180 0x7fecb413bb50 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7feca4001200 tx=0x7feca400dc10 comp rx=0 tx=0).stop 2023-10-18T09:03:20.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.771+0000 7fecba207640 1 -- 172.21.15.40:0/2352503337 shutdown_connections 2023-10-18T09:03:20.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.771+0000 7fecba207640 1 --2- 172.21.15.40:0/2352503337 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fec9406ae40 0x7fec9406d300 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:20.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.771+0000 7fecba207640 1 --2- 172.21.15.40:0/2352503337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecb41f2460 0x7fecb41e4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:20.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.771+0000 7fecba207640 1 --2- 172.21.15.40:0/2352503337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecb41e3830 0x7fecb41f1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:20.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.771+0000 7fecba207640 1 --2- 172.21.15.40:0/2352503337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feca0005180 0x7fecb413bb50 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:20.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.771+0000 7fecba207640 1 -- 172.21.15.40:0/2352503337 >> 172.21.15.40:0/2352503337 conn(0x7fecb4069720 msgr2=0x7fecb40b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:20.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.772+0000 7fecba207640 1 -- 172.21.15.40:0/2352503337 shutdown_connections 2023-10-18T09:03:20.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:20.772+0000 7fecba207640 1 -- 172.21.15.40:0/2352503337 wait complete. 2023-10-18T09:03:20.793 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T09:03:21.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.021+0000 7f5890a2a640 1 -- 172.21.15.40:0/2856995934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c0ca0d0 msgr2=0x7f588c157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:21.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.021+0000 7f5890a2a640 1 --2- 172.21.15.40:0/2856995934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c0ca0d0 0x7f588c157d40 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f5884005fd0 tx=0x7f588402c0e0 comp rx=0 tx=0).stop 2023-10-18T09:03:21.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.021+0000 7f5890a2a640 1 -- 172.21.15.40:0/2856995934 shutdown_connections 2023-10-18T09:03:21.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.021+0000 7f5890a2a640 1 --2- 172.21.15.40:0/2856995934 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c0ca0d0 0x7f588c157d40 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:21.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.021+0000 7f5890a2a640 1 --2- 172.21.15.40:0/2856995934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f588c06d6f0 0x7f588c0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:21.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.021+0000 7f5890a2a640 1 -- 172.21.15.40:0/2856995934 >> 172.21.15.40:0/2856995934 conn(0x7f588c069720 msgr2=0x7f588c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:21.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.021+0000 7f5890a2a640 1 -- 172.21.15.40:0/2856995934 shutdown_connections 2023-10-18T09:03:21.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.021+0000 7f5890a2a640 1 -- 172.21.15.40:0/2856995934 wait complete. 2023-10-18T09:03:21.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.023+0000 7f5890a2a640 1 Processor -- start 2023-10-18T09:03:21.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.023+0000 7f5890a2a640 1 -- start start 2023-10-18T09:03:21.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.024+0000 7f5890a2a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f588c06d6f0 0x7f588c1f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:21.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.024+0000 7f5890a2a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f588c0b0630 0x7f588c1447f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:21.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.024+0000 7f5890a2a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c0ca0d0 0x7f588c159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:21.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.024+0000 7f5890a2a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c1f5bf0 con 0x7f588c0ca0d0 2023-10-18T09:03:21.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.024+0000 7f5890a2a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c150b10 con 0x7f588c06d6f0 2023-10-18T09:03:21.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.024+0000 7f5890a2a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c150df0 con 0x7f588c0b0630 2023-10-18T09:03:21.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.024+0000 7f588b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c0ca0d0 0x7f588c159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:21.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.024+0000 7f588b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c0ca0d0 0x7f588c159840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54694/0 (socket says 172.21.15.40:54694) 2023-10-18T09:03:21.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.024+0000 7f588b59e640 1 -- 172.21.15.40:0/1425750179 learned_addr learned my addr 172.21.15.40:0/1425750179 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:21.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.025+0000 7f588ad9d640 1 --2- 172.21.15.40:0/1425750179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f588c06d6f0 0x7f588c1f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:21.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.025+0000 7f5883fff640 1 --2- 172.21.15.40:0/1425750179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f588c0b0630 0x7f588c1447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:21.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.025+0000 7f588b59e640 1 -- 172.21.15.40:0/1425750179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f588c0b0630 msgr2=0x7f588c1447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:21.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.025+0000 7f588b59e640 1 --2- 172.21.15.40:0/1425750179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f588c0b0630 0x7f588c1447f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:21.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.025+0000 7f588b59e640 1 -- 172.21.15.40:0/1425750179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f588c06d6f0 msgr2=0x7f588c1f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:21.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.025+0000 7f588b59e640 1 --2- 172.21.15.40:0/1425750179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f588c06d6f0 0x7f588c1f56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:21.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.025+0000 7f588b59e640 1 -- 172.21.15.40:0/1425750179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5884009840 con 0x7f588c0ca0d0 2023-10-18T09:03:21.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.025+0000 7f588b59e640 1 --2- 172.21.15.40:0/1425750179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c0ca0d0 0x7f588c159840 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f5884036040 tx=0x7f5884005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:21.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.026+0000 7f58837fe640 1 -- 172.21.15.40:0/1425750179 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5884004680 con 0x7f588c0ca0d0 2023-10-18T09:03:21.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.026+0000 7f58837fe640 1 -- 172.21.15.40:0/1425750179 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f58840345a0 con 0x7f588c0ca0d0 2023-10-18T09:03:21.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.026+0000 7f58837fe640 1 -- 172.21.15.40:0/1425750179 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58840349b0 con 0x7f588c0ca0d0 2023-10-18T09:03:21.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.026+0000 7f5890a2a640 1 -- 172.21.15.40:0/1425750179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588c1510d0 con 0x7f588c0ca0d0 2023-10-18T09:03:21.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.026+0000 7f5890a2a640 1 -- 172.21.15.40:0/1425750179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c151690 con 0x7f588c0ca0d0 2023-10-18T09:03:21.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.027+0000 7f5890a2a640 1 -- 172.21.15.40:0/1425750179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588c066e00 con 0x7f588c0ca0d0 2023-10-18T09:03:21.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.028+0000 7f58837fe640 1 -- 172.21.15.40:0/1425750179 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5884034b10 con 0x7f588c0ca0d0 2023-10-18T09:03:21.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.029+0000 7f58837fe640 1 --2- 172.21.15.40:0/1425750179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f586c06ae70 0x7f586c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:21.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.029+0000 7f58837fe640 1 -- 172.21.15.40:0/1425750179 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f58840ac480 con 0x7f588c0ca0d0 2023-10-18T09:03:21.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.029+0000 7f588ad9d640 1 --2- 172.21.15.40:0/1425750179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f586c06ae70 0x7f586c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:21.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.030+0000 7f588ad9d640 1 --2- 172.21.15.40:0/1425750179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f586c06ae70 0x7f586c06d330 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f587c005fd0 tx=0x7f587c00c520 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:21.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.033+0000 7f58837fe640 1 -- 172.21.15.40:0/1425750179 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f58840777e0 con 0x7f588c0ca0d0 2023-10-18T09:03:21.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.235+0000 7f5890a2a640 1 -- 172.21.15.40:0/1425750179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f588c153810 con 0x7f588c0ca0d0 2023-10-18T09:03:21.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.235+0000 7f58837fe640 1 -- 172.21.15.40:0/1425750179 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v140) v1 ==== 84+0+65 (secure 0 0 0) 0x7f588407ae00 con 0x7f588c0ca0d0 2023-10-18T09:03:21.239 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:21.239 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-10-18T09:03:21.239 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:21.240 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-10-18T09:03:21.240 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-10-18T09:03:21.240 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:21.240 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-10-18T09:03:21.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.238+0000 7f5890a2a640 1 -- 172.21.15.40:0/1425750179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f586c06ae70 msgr2=0x7f586c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:21.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.238+0000 7f5890a2a640 1 --2- 172.21.15.40:0/1425750179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f586c06ae70 0x7f586c06d330 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f587c005fd0 tx=0x7f587c00c520 comp rx=0 tx=0).stop 2023-10-18T09:03:21.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.238+0000 7f5890a2a640 1 -- 172.21.15.40:0/1425750179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c0ca0d0 msgr2=0x7f588c159840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:21.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.238+0000 7f5890a2a640 1 --2- 172.21.15.40:0/1425750179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c0ca0d0 0x7f588c159840 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f5884036040 tx=0x7f5884005c80 comp rx=0 tx=0).stop 2023-10-18T09:03:21.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.238+0000 7f5890a2a640 1 -- 172.21.15.40:0/1425750179 shutdown_connections 2023-10-18T09:03:21.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.238+0000 7f5890a2a640 1 --2- 172.21.15.40:0/1425750179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f586c06ae70 0x7f586c06d330 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:21.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.238+0000 7f5890a2a640 1 --2- 172.21.15.40:0/1425750179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f588c0ca0d0 0x7f588c159840 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:21.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.238+0000 7f5890a2a640 1 --2- 172.21.15.40:0/1425750179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f588c0b0630 0x7f588c1447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:21.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.238+0000 7f5890a2a640 1 --2- 172.21.15.40:0/1425750179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f588c06d6f0 0x7f588c1f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:21.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.238+0000 7f5890a2a640 1 -- 172.21.15.40:0/1425750179 >> 172.21.15.40:0/1425750179 conn(0x7f588c069720 msgr2=0x7f588c14f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:21.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.238+0000 7f5890a2a640 1 -- 172.21.15.40:0/1425750179 shutdown_connections 2023-10-18T09:03:21.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.238+0000 7f5890a2a640 1 -- 172.21.15.40:0/1425750179 wait complete. 2023-10-18T09:03:21.261 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T09:03:21.262 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T09:03:21.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.495+0000 7fed33fff640 1 -- 172.21.15.40:0/1555259096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed34070c40 msgr2=0x7fed340710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:21.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.495+0000 7fed33fff640 1 --2- 172.21.15.40:0/1555259096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed34070c40 0x7fed340710a0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009830 tx=0x7fed1c02bc40 comp rx=0 tx=0).stop 2023-10-18T09:03:21.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.495+0000 7fed33fff640 1 -- 172.21.15.40:0/1555259096 shutdown_connections 2023-10-18T09:03:21.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.495+0000 7fed33fff640 1 --2- 172.21.15.40:0/1555259096 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed34070c40 0x7fed340710a0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:21.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.495+0000 7fed33fff640 1 --2- 172.21.15.40:0/1555259096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed340b22f0 0x7fed340b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:21.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.495+0000 7fed33fff640 1 -- 172.21.15.40:0/1555259096 >> 172.21.15.40:0/1555259096 conn(0x7fed3406b7d0 msgr2=0x7fed3406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:21.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.496+0000 7fed33fff640 1 -- 172.21.15.40:0/1555259096 shutdown_connections 2023-10-18T09:03:21.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.496+0000 7fed33fff640 1 -- 172.21.15.40:0/1555259096 wait complete. 2023-10-18T09:03:21.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.497+0000 7fed33fff640 1 Processor -- start 2023-10-18T09:03:21.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.497+0000 7fed33fff640 1 -- start start 2023-10-18T09:03:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.498+0000 7fed33fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed34070c40 0x7fed341f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.498+0000 7fed33fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed340b22f0 0x7fed3414bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.498+0000 7fed33fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed340ca690 0x7fed3415b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.498+0000 7fed33fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed341f61f0 con 0x7fed340b22f0 2023-10-18T09:03:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.498+0000 7fed33fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3414c0c0 con 0x7fed34070c40 2023-10-18T09:03:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.498+0000 7fed33fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed341589d0 con 0x7fed340ca690 2023-10-18T09:03:21.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.499+0000 7fed327fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed340b22f0 0x7fed3414bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.499+0000 7fed327fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed340b22f0 0x7fed3414bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54712/0 (socket says 172.21.15.40:54712) 2023-10-18T09:03:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.499+0000 7fed327fc640 1 -- 172.21.15.40:0/1709835469 learned_addr learned my addr 172.21.15.40:0/1709835469 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.499+0000 7fed32ffd640 1 --2- 172.21.15.40:0/1709835469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed34070c40 0x7fed341f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.499+0000 7fed337fe640 1 --2- 172.21.15.40:0/1709835469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed340ca690 0x7fed3415b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:21.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.499+0000 7fed327fc640 1 -- 172.21.15.40:0/1709835469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed340ca690 msgr2=0x7fed3415b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.499+0000 7fed327fc640 1 --2- 172.21.15.40:0/1709835469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed340ca690 0x7fed3415b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.499+0000 7fed327fc640 1 -- 172.21.15.40:0/1709835469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed34070c40 msgr2=0x7fed341f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.499+0000 7fed327fc640 1 --2- 172.21.15.40:0/1709835469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed34070c40 0x7fed341f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:21.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.499+0000 7fed327fc640 1 -- 172.21.15.40:0/1709835469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c009490 con 0x7fed340b22f0 2023-10-18T09:03:21.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.500+0000 7fed327fc640 1 --2- 172.21.15.40:0/1709835469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed340b22f0 0x7fed3414bb80 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009950 tx=0x7fed1c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.500+0000 7fed2b7fe640 1 -- 172.21.15.40:0/1709835469 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c003a00 con 0x7fed340b22f0 2023-10-18T09:03:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.500+0000 7fed2b7fe640 1 -- 172.21.15.40:0/1709835469 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fed1c003c70 con 0x7fed340b22f0 2023-10-18T09:03:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.500+0000 7fed2b7fe640 1 -- 172.21.15.40:0/1709835469 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c034440 con 0x7fed340b22f0 2023-10-18T09:03:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.500+0000 7fed33fff640 1 -- 172.21.15.40:0/1709835469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed34157dc0 con 0x7fed340b22f0 2023-10-18T09:03:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.500+0000 7fed33fff640 1 -- 172.21.15.40:0/1709835469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed34158380 con 0x7fed340b22f0 2023-10-18T09:03:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.501+0000 7fed33fff640 1 -- 172.21.15.40:0/1709835469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed34072050 con 0x7fed340b22f0 2023-10-18T09:03:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.502+0000 7fed2b7fe640 1 -- 172.21.15.40:0/1709835469 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fed1c03b040 con 0x7fed340b22f0 2023-10-18T09:03:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.503+0000 7fed2b7fe640 1 --2- 172.21.15.40:0/1709835469 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fed1406ae40 0x7fed1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.503+0000 7fed2b7fe640 1 -- 172.21.15.40:0/1709835469 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fed1c0ab4d0 con 0x7fed340b22f0 2023-10-18T09:03:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.504+0000 7fed32ffd640 1 --2- 172.21.15.40:0/1709835469 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fed1406ae40 0x7fed1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:21.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.505+0000 7fed32ffd640 1 --2- 172.21.15.40:0/1709835469 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fed1406ae40 0x7fed1406d300 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fed24005fd0 tx=0x7fed24009870 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:21.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.508+0000 7fed2b7fe640 1 -- 172.21.15.40:0/1709835469 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed1c076830 con 0x7fed340b22f0 2023-10-18T09:03:21.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:21.701+0000 7fed33fff640 1 -- 172.21.15.40:0/1709835469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fed34066ea0 con 0x7fed340b22f0 2023-10-18T09:03:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.294+0000 7fed2b7fe640 1 -- 172.21.15.40:0/1709835469 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v141) v1 ==== 138+0+0 (secure 0 0 0) 0x7fed1c0390b0 con 0x7fed340b22f0 2023-10-18T09:03:22.298 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-10-18T09:03:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.298+0000 7fed33fff640 1 -- 172.21.15.40:0/1709835469 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fed1406ae40 msgr2=0x7fed1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.298+0000 7fed33fff640 1 --2- 172.21.15.40:0/1709835469 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fed1406ae40 0x7fed1406d300 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fed24005fd0 tx=0x7fed24009870 comp rx=0 tx=0).stop 2023-10-18T09:03:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.298+0000 7fed33fff640 1 -- 172.21.15.40:0/1709835469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed340b22f0 msgr2=0x7fed3414bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:22.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.298+0000 7fed33fff640 1 --2- 172.21.15.40:0/1709835469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed340b22f0 0x7fed3414bb80 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009950 tx=0x7fed1c004430 comp rx=0 tx=0).stop 2023-10-18T09:03:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.299+0000 7fed33fff640 1 -- 172.21.15.40:0/1709835469 shutdown_connections 2023-10-18T09:03:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.299+0000 7fed33fff640 1 --2- 172.21.15.40:0/1709835469 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fed1406ae40 0x7fed1406d300 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.299+0000 7fed33fff640 1 --2- 172.21.15.40:0/1709835469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fed340ca690 0x7fed3415b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.299+0000 7fed33fff640 1 --2- 172.21.15.40:0/1709835469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed340b22f0 0x7fed3414bb80 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:22.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.299+0000 7fed33fff640 1 --2- 172.21.15.40:0/1709835469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed34070c40 0x7fed341f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.299+0000 7fed33fff640 1 -- 172.21.15.40:0/1709835469 >> 172.21.15.40:0/1709835469 conn(0x7fed3406b7d0 msgr2=0x7fed3406cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.299+0000 7fed33fff640 1 -- 172.21.15.40:0/1709835469 shutdown_connections 2023-10-18T09:03:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.299+0000 7fed33fff640 1 -- 172.21.15.40:0/1709835469 wait complete. 2023-10-18T09:03:22.330 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T09:03:22.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.554+0000 7f348359e640 1 -- 172.21.15.40:0/894053388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484071350 msgr2=0x7f348406bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:22.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.554+0000 7f348359e640 1 --2- 172.21.15.40:0/894053388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484071350 0x7f348406bb10 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f347c009830 tx=0x7f347c02bd60 comp rx=0 tx=0).stop 2023-10-18T09:03:22.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.554+0000 7f348359e640 1 -- 172.21.15.40:0/894053388 shutdown_connections 2023-10-18T09:03:22.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.554+0000 7f348359e640 1 --2- 172.21.15.40:0/894053388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484071350 0x7f348406bb10 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:22.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.554+0000 7f348359e640 1 -- 172.21.15.40:0/894053388 >> 172.21.15.40:0/894053388 conn(0x7f348406ca10 msgr2=0x7f348406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:22.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.554+0000 7f348359e640 1 -- 172.21.15.40:0/894053388 shutdown_connections 2023-10-18T09:03:22.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.554+0000 7f348359e640 1 -- 172.21.15.40:0/894053388 wait complete. 2023-10-18T09:03:22.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.556+0000 7f348359e640 1 Processor -- start 2023-10-18T09:03:22.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.556+0000 7f348359e640 1 -- start start 2023-10-18T09:03:22.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.557+0000 7f348359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484071350 0x7f3484151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:22.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.557+0000 7f348359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34840cce20 0x7f34840add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.557+0000 7f348359e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3484155ad0 0x7f34840ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.557+0000 7f348359e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34841f5f50 con 0x7f3484071350 2023-10-18T09:03:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.557+0000 7f348359e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34841f5380 con 0x7f34840cce20 2023-10-18T09:03:22.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.557+0000 7f348359e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34841f5620 con 0x7f3484155ad0 2023-10-18T09:03:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.558+0000 7f348259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484071350 0x7f3484151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.558+0000 7f348259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484071350 0x7f3484151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54736/0 (socket says 172.21.15.40:54736) 2023-10-18T09:03:22.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.558+0000 7f348259c640 1 -- 172.21.15.40:0/1167577701 learned_addr learned my addr 172.21.15.40:0/1167577701 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:22.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.558+0000 7f3482d9d640 1 --2- 172.21.15.40:0/1167577701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3484155ad0 0x7f34840ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:22.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.558+0000 7f3479d9b640 1 --2- 172.21.15.40:0/1167577701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34840cce20 0x7f34840add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:22.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.558+0000 7f348259c640 1 -- 172.21.15.40:0/1167577701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3484155ad0 msgr2=0x7f34840ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:22.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.558+0000 7f348259c640 1 --2- 172.21.15.40:0/1167577701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3484155ad0 0x7f34840ae2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:22.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.558+0000 7f348259c640 1 -- 172.21.15.40:0/1167577701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34840cce20 msgr2=0x7f34840add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:22.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.558+0000 7f348259c640 1 --2- 172.21.15.40:0/1167577701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34840cce20 0x7f34840add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:22.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.558+0000 7f348259c640 1 -- 172.21.15.40:0/1167577701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c009490 con 0x7f3484071350 2023-10-18T09:03:22.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.558+0000 7f3479d9b640 1 --2- 172.21.15.40:0/1167577701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34840cce20 0x7f34840add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:22.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.559+0000 7f348259c640 1 --2- 172.21.15.40:0/1167577701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484071350 0x7f3484151ce0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f347400c730 tx=0x7f347400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:22.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.559+0000 7f347b7fe640 1 -- 172.21.15.40:0/1167577701 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474003b70 con 0x7f3484071350 2023-10-18T09:03:22.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.559+0000 7f347b7fe640 1 -- 172.21.15.40:0/1167577701 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3474003cf0 con 0x7f3484071350 2023-10-18T09:03:22.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.559+0000 7f347b7fe640 1 -- 172.21.15.40:0/1167577701 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347400aab0 con 0x7f3484071350 2023-10-18T09:03:22.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.559+0000 7f348359e640 1 -- 172.21.15.40:0/1167577701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34841f5900 con 0x7f3484071350 2023-10-18T09:03:22.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.559+0000 7f348359e640 1 -- 172.21.15.40:0/1167577701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3484152220 con 0x7f3484071350 2023-10-18T09:03:22.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.560+0000 7f348359e640 1 -- 172.21.15.40:0/1167577701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34841523e0 con 0x7f3484071350 2023-10-18T09:03:22.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.561+0000 7f347b7fe640 1 -- 172.21.15.40:0/1167577701 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f347400ac10 con 0x7f3484071350 2023-10-18T09:03:22.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.562+0000 7f347b7fe640 1 --2- 172.21.15.40:0/1167577701 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f346406ae40 0x7f346406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:22.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.562+0000 7f347b7fe640 1 -- 172.21.15.40:0/1167577701 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f34740894b0 con 0x7f3484071350 2023-10-18T09:03:22.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.563+0000 7f3479d9b640 1 --2- 172.21.15.40:0/1167577701 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f346406ae40 0x7f346406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:22.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.564+0000 7f3479d9b640 1 --2- 172.21.15.40:0/1167577701 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f346406ae40 0x7f346406d300 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f34841f6370 tx=0x7f346c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:22.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.568+0000 7f347b7fe640 1 -- 172.21.15.40:0/1167577701 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f34740546a0 con 0x7f3484071350 2023-10-18T09:03:22.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:22.761+0000 7f348359e640 1 -- 172.21.15.40:0/1167577701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3484066e00 con 0x7f3484071350 2023-10-18T09:03:23.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.301+0000 7f347b7fe640 1 -- 172.21.15.40:0/1167577701 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v142) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3474057cc0 con 0x7f3484071350 2023-10-18T09:03:23.305 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-10-18T09:03:23.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.304+0000 7f348359e640 1 -- 172.21.15.40:0/1167577701 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f346406ae40 msgr2=0x7f346406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:23.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.304+0000 7f348359e640 1 --2- 172.21.15.40:0/1167577701 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f346406ae40 0x7f346406d300 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f34841f6370 tx=0x7f346c009040 comp rx=0 tx=0).stop 2023-10-18T09:03:23.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.304+0000 7f348359e640 1 -- 172.21.15.40:0/1167577701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484071350 msgr2=0x7f3484151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:23.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.304+0000 7f348359e640 1 --2- 172.21.15.40:0/1167577701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484071350 0x7f3484151ce0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f347400c730 tx=0x7f347400cbf0 comp rx=0 tx=0).stop 2023-10-18T09:03:23.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.305+0000 7f348359e640 1 -- 172.21.15.40:0/1167577701 shutdown_connections 2023-10-18T09:03:23.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.305+0000 7f348359e640 1 --2- 172.21.15.40:0/1167577701 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f346406ae40 0x7f346406d300 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:23.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.305+0000 7f348359e640 1 --2- 172.21.15.40:0/1167577701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3484155ad0 0x7f34840ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:23.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.305+0000 7f348359e640 1 --2- 172.21.15.40:0/1167577701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34840cce20 0x7f34840add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:23.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.305+0000 7f348359e640 1 --2- 172.21.15.40:0/1167577701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3484071350 0x7f3484151ce0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:23.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.305+0000 7f348359e640 1 -- 172.21.15.40:0/1167577701 >> 172.21.15.40:0/1167577701 conn(0x7f348406ca10 msgr2=0x7f3484072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:23.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.306+0000 7f348359e640 1 -- 172.21.15.40:0/1167577701 shutdown_connections 2023-10-18T09:03:23.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.306+0000 7f348359e640 1 -- 172.21.15.40:0/1167577701 wait complete. 2023-10-18T09:03:23.340 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T09:03:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.532+0000 7f97b92f3640 1 -- 172.21.15.40:0/3014152558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97b415af70 msgr2=0x7f97b40710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:23.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.532+0000 7f97b92f3640 1 --2- 172.21.15.40:0/3014152558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97b415af70 0x7f97b40710a0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f97ac005fd0 tx=0x7f97ac02d130 comp rx=0 tx=0).stop 2023-10-18T09:03:23.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.532+0000 7f97b92f3640 1 -- 172.21.15.40:0/3014152558 shutdown_connections 2023-10-18T09:03:23.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.532+0000 7f97b92f3640 1 --2- 172.21.15.40:0/3014152558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97b415af70 0x7f97b40710a0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:23.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.532+0000 7f97b92f3640 1 --2- 172.21.15.40:0/3014152558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97b406c2d0 0x7f97b4070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:23.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.532+0000 7f97b92f3640 1 -- 172.21.15.40:0/3014152558 >> 172.21.15.40:0/3014152558 conn(0x7f97b406ca10 msgr2=0x7f97b406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:23.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.532+0000 7f97b92f3640 1 -- 172.21.15.40:0/3014152558 shutdown_connections 2023-10-18T09:03:23.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.532+0000 7f97b92f3640 1 -- 172.21.15.40:0/3014152558 wait complete. 2023-10-18T09:03:23.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.534+0000 7f97b92f3640 1 Processor -- start 2023-10-18T09:03:23.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.534+0000 7f97b92f3640 1 -- start start 2023-10-18T09:03:23.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.535+0000 7f97b92f3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97b406c2d0 0x7f97b40a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:23.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.535+0000 7f97b92f3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97b40c9620 0x7f97b40c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:23.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.535+0000 7f97b92f3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97b415af70 0x7f97b40a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:23.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.535+0000 7f97b92f3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b4148dd0 con 0x7f97b415af70 2023-10-18T09:03:23.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.535+0000 7f97b92f3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b41490b0 con 0x7f97b40c9620 2023-10-18T09:03:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.535+0000 7f97b92f3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b4149390 con 0x7f97b406c2d0 2023-10-18T09:03:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.535+0000 7f97b8af2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97b415af70 0x7f97b40a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.535+0000 7f97b8af2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97b415af70 0x7f97b40a69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54754/0 (socket says 172.21.15.40:54754) 2023-10-18T09:03:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.535+0000 7f97b8af2640 1 -- 172.21.15.40:0/3444657285 learned_addr learned my addr 172.21.15.40:0/3444657285 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:23.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.536+0000 7f97b37fe640 1 --2- 172.21.15.40:0/3444657285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97b40c9620 0x7f97b40c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:23.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.536+0000 7f97b8af2640 1 -- 172.21.15.40:0/3444657285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97b406c2d0 msgr2=0x7f97b40a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:23.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.536+0000 7f97b3fff640 1 --2- 172.21.15.40:0/3444657285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97b406c2d0 0x7f97b40a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:23.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.536+0000 7f97b8af2640 1 --2- 172.21.15.40:0/3444657285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97b406c2d0 0x7f97b40a9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:23.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.536+0000 7f97b8af2640 1 -- 172.21.15.40:0/3444657285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97b40c9620 msgr2=0x7f97b40c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:23.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.536+0000 7f97b8af2640 1 --2- 172.21.15.40:0/3444657285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97b40c9620 0x7f97b40c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:23.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.536+0000 7f97b8af2640 1 -- 172.21.15.40:0/3444657285 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac00aa10 con 0x7f97b415af70 2023-10-18T09:03:23.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.536+0000 7f97b37fe640 1 --2- 172.21.15.40:0/3444657285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97b40c9620 0x7f97b40c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:23.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.536+0000 7f97b8af2640 1 --2- 172.21.15.40:0/3444657285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97b415af70 0x7f97b40a69d0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f97ac038040 tx=0x7f97ac036db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.536+0000 7f97b0ff9640 1 -- 172.21.15.40:0/3444657285 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac035440 con 0x7f97b415af70 2023-10-18T09:03:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.536+0000 7f97b0ff9640 1 -- 172.21.15.40:0/3444657285 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f97ac004540 con 0x7f97b415af70 2023-10-18T09:03:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.536+0000 7f97b92f3640 1 -- 172.21.15.40:0/3444657285 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b4149670 con 0x7f97b415af70 2023-10-18T09:03:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.537+0000 7f97b0ff9640 1 -- 172.21.15.40:0/3444657285 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac040450 con 0x7f97b415af70 2023-10-18T09:03:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.537+0000 7f97b92f3640 1 -- 172.21.15.40:0/3444657285 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b40a9ba0 con 0x7f97b415af70 2023-10-18T09:03:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.538+0000 7f97b92f3640 1 -- 172.21.15.40:0/3444657285 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b4066e00 con 0x7f97b415af70 2023-10-18T09:03:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.539+0000 7f97b0ff9640 1 -- 172.21.15.40:0/3444657285 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f97ac03c070 con 0x7f97b415af70 2023-10-18T09:03:23.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.540+0000 7f97b0ff9640 1 --2- 172.21.15.40:0/3444657285 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f97a006ae40 0x7f97a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:23.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.540+0000 7f97b0ff9640 1 -- 172.21.15.40:0/3444657285 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f97ac0ace90 con 0x7f97b415af70 2023-10-18T09:03:23.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.540+0000 7f97b3fff640 1 --2- 172.21.15.40:0/3444657285 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f97a006ae40 0x7f97a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:23.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.541+0000 7f97b3fff640 1 --2- 172.21.15.40:0/3444657285 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f97a006ae40 0x7f97a006d300 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f97a4000f60 tx=0x7f97a4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:23.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.543+0000 7f97b0ff9640 1 -- 172.21.15.40:0/3444657285 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f97ac0781a0 con 0x7f97b415af70 2023-10-18T09:03:23.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:23.779+0000 7f97b92f3640 1 -- 172.21.15.40:0/3444657285 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f97b406bfd0 con 0x7f97b415af70 2023-10-18T09:03:24.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.372+0000 7f97b0ff9640 1 -- 172.21.15.40:0/3444657285 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v62) v1 ==== 174+0+0 (secure 0 0 0) 0x7f97ac0440c0 con 0x7f97b415af70 2023-10-18T09:03:24.376 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 16 and data pool 17 2023-10-18T09:03:24.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.379+0000 7f97b92f3640 1 -- 172.21.15.40:0/3444657285 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f97a006ae40 msgr2=0x7f97a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:24.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.379+0000 7f97b92f3640 1 --2- 172.21.15.40:0/3444657285 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f97a006ae40 0x7f97a006d300 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f97a4000f60 tx=0x7f97a4009040 comp rx=0 tx=0).stop 2023-10-18T09:03:24.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.379+0000 7f97b92f3640 1 -- 172.21.15.40:0/3444657285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97b415af70 msgr2=0x7f97b40a69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:24.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.379+0000 7f97b92f3640 1 --2- 172.21.15.40:0/3444657285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97b415af70 0x7f97b40a69d0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f97ac038040 tx=0x7f97ac036db0 comp rx=0 tx=0).stop 2023-10-18T09:03:24.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.380+0000 7f97b92f3640 1 -- 172.21.15.40:0/3444657285 shutdown_connections 2023-10-18T09:03:24.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.380+0000 7f97b92f3640 1 --2- 172.21.15.40:0/3444657285 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f97a006ae40 0x7f97a006d300 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:24.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.380+0000 7f97b92f3640 1 --2- 172.21.15.40:0/3444657285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97b415af70 0x7f97b40a69d0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:24.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.380+0000 7f97b92f3640 1 --2- 172.21.15.40:0/3444657285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97b40c9620 0x7f97b40c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:24.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.380+0000 7f97b92f3640 1 --2- 172.21.15.40:0/3444657285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97b406c2d0 0x7f97b40a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:24.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.380+0000 7f97b92f3640 1 -- 172.21.15.40:0/3444657285 >> 172.21.15.40:0/3444657285 conn(0x7f97b406ca10 msgr2=0x7f97b40ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:24.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.381+0000 7f97b92f3640 1 -- 172.21.15.40:0/3444657285 shutdown_connections 2023-10-18T09:03:24.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.381+0000 7f97b92f3640 1 -- 172.21.15.40:0/3444657285 wait complete. 2023-10-18T09:03:24.409 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:03:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.593+0000 7f5b02106640 1 -- 172.21.15.40:0/2878424382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ae80039a0 msgr2=0x7f5ae8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.593+0000 7f5b02106640 1 --2- 172.21.15.40:0/2878424382 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ae80039a0 0x7f5ae8003dc0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f5af8001650 tx=0x7f5ae80057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.593+0000 7f5b01905640 1 -- 172.21.15.40:0/2878424382 reap_dead start 2023-10-18T09:03:24.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.593+0000 7f5b02106640 1 -- 172.21.15.40:0/2878424382 shutdown_connections 2023-10-18T09:03:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.593+0000 7f5b02106640 1 -- 172.21.15.40:0/2878424382 >> 172.21.15.40:0/2878424382 conn(0x7f5afc069720 msgr2=0x7f5afc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.593+0000 7f5b02106640 1 -- 172.21.15.40:0/2878424382 shutdown_connections 2023-10-18T09:03:24.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.593+0000 7f5b02106640 1 -- 172.21.15.40:0/2878424382 wait complete. 2023-10-18T09:03:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.595+0000 7f5b02106640 1 Processor -- start 2023-10-18T09:03:24.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.595+0000 7f5b02106640 1 -- start start 2023-10-18T09:03:24.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.596+0000 7f5b02106640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ae80039a0 0x7f5afc0abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:24.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.596+0000 7f5b02106640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5afc0ac230 0x7f5afc139c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:24.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.596+0000 7f5b02106640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5afc13a150 0x7f5afc0ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:24.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.596+0000 7f5b02106640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc13a5d0 con 0x7f5afc13a150 2023-10-18T09:03:24.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.596+0000 7f5b02106640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc0aeba0 con 0x7f5afc0ac230 2023-10-18T09:03:24.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.596+0000 7f5b02106640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc0aee80 con 0x7f5ae80039a0 2023-10-18T09:03:24.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.596+0000 7f5b01905640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5afc13a150 0x7f5afc0ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:24.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.596+0000 7f5b01905640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5afc13a150 0x7f5afc0ac970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54778/0 (socket says 172.21.15.40:54778) 2023-10-18T09:03:24.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.596+0000 7f5b01905640 1 -- 172.21.15.40:0/1090675893 learned_addr learned my addr 172.21.15.40:0/1090675893 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:24.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.597+0000 7f5b01104640 1 --2- 172.21.15.40:0/1090675893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ae80039a0 0x7f5afc0abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:24.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.597+0000 7f5b00903640 1 --2- 172.21.15.40:0/1090675893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5afc0ac230 0x7f5afc139c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:24.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.597+0000 7f5b01905640 1 -- 172.21.15.40:0/1090675893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ae80039a0 msgr2=0x7f5afc0abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:24.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.597+0000 7f5b01905640 1 --2- 172.21.15.40:0/1090675893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ae80039a0 0x7f5afc0abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:24.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.597+0000 7f5b01905640 1 -- 172.21.15.40:0/1090675893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5afc0ac230 msgr2=0x7f5afc139c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:24.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.597+0000 7f5b01905640 1 --2- 172.21.15.40:0/1090675893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5afc0ac230 0x7f5afc139c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:24.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.597+0000 7f5b01905640 1 -- 172.21.15.40:0/1090675893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc0af160 con 0x7f5afc13a150 2023-10-18T09:03:24.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.597+0000 7f5b01104640 1 --2- 172.21.15.40:0/1090675893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ae80039a0 0x7f5afc0abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:24.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.597+0000 7f5b01905640 1 --2- 172.21.15.40:0/1090675893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5afc13a150 0x7f5afc0ac970 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f5afc06a550 tx=0x7f5af400fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:24.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.597+0000 7f5af1ffb640 1 -- 172.21.15.40:0/1090675893 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af400dda0 con 0x7f5afc13a150 2023-10-18T09:03:24.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.597+0000 7f5af1ffb640 1 -- 172.21.15.40:0/1090675893 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5af4011440 con 0x7f5afc13a150 2023-10-18T09:03:24.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.597+0000 7f5af1ffb640 1 -- 172.21.15.40:0/1090675893 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4011850 con 0x7f5afc13a150 2023-10-18T09:03:24.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.597+0000 7f5b02106640 1 -- 172.21.15.40:0/1090675893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5afc0af440 con 0x7f5afc13a150 2023-10-18T09:03:24.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.598+0000 7f5b02106640 1 -- 172.21.15.40:0/1090675893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5afc14edb0 con 0x7f5afc13a150 2023-10-18T09:03:24.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.599+0000 7f5b02106640 1 -- 172.21.15.40:0/1090675893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5afc066e00 con 0x7f5afc13a150 2023-10-18T09:03:24.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.599+0000 7f5af1ffb640 1 -- 172.21.15.40:0/1090675893 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5af40119b0 con 0x7f5afc13a150 2023-10-18T09:03:24.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.600+0000 7f5af1ffb640 1 --2- 172.21.15.40:0/1090675893 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ae006ae40 0x7f5ae006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:24.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.600+0000 7f5af1ffb640 1 -- 172.21.15.40:0/1090675893 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5af4089930 con 0x7f5afc13a150 2023-10-18T09:03:24.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.601+0000 7f5b01104640 1 --2- 172.21.15.40:0/1090675893 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ae006ae40 0x7f5ae006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:24.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.602+0000 7f5b01104640 1 --2- 172.21.15.40:0/1090675893 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ae006ae40 0x7f5ae006d300 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5af8034000 tx=0x7f5af802f040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:24.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.605+0000 7f5af1ffb640 1 -- 172.21.15.40:0/1090675893 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5af40549f0 con 0x7f5afc13a150 2023-10-18T09:03:24.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.788+0000 7f5b02106640 1 -- 172.21.15.40:0/1090675893 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5afc06d920 con 0x7f5afc13a150 2023-10-18T09:03:24.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.790+0000 7f5af1ffb640 1 -- 172.21.15.40:0/1090675893 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f5af4021090 con 0x7f5afc13a150 2023-10-18T09:03:24.794 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:24.794 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":143,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:03:23.781764+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:03:21.754758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-18T09:03:22.819486+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:03:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.793+0000 7f5b02106640 1 -- 172.21.15.40:0/1090675893 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ae006ae40 msgr2=0x7f5ae006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.793+0000 7f5b02106640 1 --2- 172.21.15.40:0/1090675893 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ae006ae40 0x7f5ae006d300 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5af8034000 tx=0x7f5af802f040 comp rx=0 tx=0).stop 2023-10-18T09:03:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.793+0000 7f5b02106640 1 -- 172.21.15.40:0/1090675893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5afc13a150 msgr2=0x7f5afc0ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.793+0000 7f5b02106640 1 --2- 172.21.15.40:0/1090675893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5afc13a150 0x7f5afc0ac970 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f5afc06a550 tx=0x7f5af400fe60 comp rx=0 tx=0).stop 2023-10-18T09:03:24.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.793+0000 7f5b02106640 1 -- 172.21.15.40:0/1090675893 shutdown_connections 2023-10-18T09:03:24.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.793+0000 7f5b02106640 1 --2- 172.21.15.40:0/1090675893 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ae006ae40 0x7f5ae006d300 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.793+0000 7f5b02106640 1 --2- 172.21.15.40:0/1090675893 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5afc13a150 0x7f5afc0ac970 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.793+0000 7f5b02106640 1 --2- 172.21.15.40:0/1090675893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5afc0ac230 0x7f5afc139c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.793+0000 7f5b02106640 1 --2- 172.21.15.40:0/1090675893 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ae80039a0 0x7f5afc0abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.793+0000 7f5b02106640 1 -- 172.21.15.40:0/1090675893 >> 172.21.15.40:0/1090675893 conn(0x7f5afc069720 msgr2=0x7f5afc13f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:24.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.793+0000 7f5b02106640 1 -- 172.21.15.40:0/1090675893 shutdown_connections 2023-10-18T09:03:24.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:24.793+0000 7f5b02106640 1 -- 172.21.15.40:0/1090675893 wait complete. 2023-10-18T09:03:24.815 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:03:25.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.042+0000 7f49aaba4640 1 -- 172.21.15.40:0/1504367741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4990004d00 msgr2=0x7f4990005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:25.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.042+0000 7f49aaba4640 1 --2- 172.21.15.40:0/1504367741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4990004d00 0x7f4990005180 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f499c009b20 tx=0x7f49900057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.042+0000 7f49a9141640 1 -- 172.21.15.40:0/1504367741 reap_dead start 2023-10-18T09:03:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.042+0000 7f49aaba4640 1 -- 172.21.15.40:0/1504367741 shutdown_connections 2023-10-18T09:03:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.042+0000 7f49aaba4640 1 -- 172.21.15.40:0/1504367741 >> 172.21.15.40:0/1504367741 conn(0x7f49a406cee0 msgr2=0x7f49a4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.042+0000 7f49aaba4640 1 -- 172.21.15.40:0/1504367741 shutdown_connections 2023-10-18T09:03:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.042+0000 7f49aaba4640 1 -- 172.21.15.40:0/1504367741 wait complete. 2023-10-18T09:03:25.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.044+0000 7f49aaba4640 1 Processor -- start 2023-10-18T09:03:25.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.044+0000 7f49aaba4640 1 -- start start 2023-10-18T09:03:25.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.045+0000 7f49aaba4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4990004d00 0x7f49a40cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:25.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.045+0000 7f49aaba4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49a40ccef0 0x7f49a413c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:25.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.045+0000 7f49aaba4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49a413c740 0x7f49a413cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:25.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.045+0000 7f49aaba4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a414ffe0 con 0x7f4990004d00 2023-10-18T09:03:25.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.045+0000 7f49aaba4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a41502c0 con 0x7f49a413c740 2023-10-18T09:03:25.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.045+0000 7f49aaba4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a41505a0 con 0x7f49a40ccef0 2023-10-18T09:03:25.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.045+0000 7f49a8940640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4990004d00 0x7f49a40cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:25.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.045+0000 7f49a8940640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4990004d00 0x7f49a40cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49610/0 (socket says 172.21.15.40:49610) 2023-10-18T09:03:25.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.045+0000 7f49a8940640 1 -- 172.21.15.40:0/797367832 learned_addr learned my addr 172.21.15.40:0/797367832 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:25.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.045+0000 7f49a3fff640 1 --2- 172.21.15.40:0/797367832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49a40ccef0 0x7f49a413c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:25.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.045+0000 7f49a8940640 1 -- 172.21.15.40:0/797367832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49a40ccef0 msgr2=0x7f49a413c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.045+0000 7f49a8940640 1 --2- 172.21.15.40:0/797367832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49a40ccef0 0x7f49a413c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.045+0000 7f49a8940640 1 -- 172.21.15.40:0/797367832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49a413c740 msgr2=0x7f49a413cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.045+0000 7f49a8940640 1 --2- 172.21.15.40:0/797367832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49a413c740 0x7f49a413cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.045+0000 7f49a8940640 1 -- 172.21.15.40:0/797367832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a4150880 con 0x7f4990004d00 2023-10-18T09:03:25.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.046+0000 7f49a8940640 1 --2- 172.21.15.40:0/797367832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4990004d00 0x7f49a40cc9b0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f49940013d0 tx=0x7f4994004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:25.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.046+0000 7f49a17fa640 1 -- 172.21.15.40:0/797367832 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499400ec50 con 0x7f4990004d00 2023-10-18T09:03:25.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.046+0000 7f49a17fa640 1 -- 172.21.15.40:0/797367832 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f499400eec0 con 0x7f4990004d00 2023-10-18T09:03:25.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.046+0000 7f49a17fa640 1 -- 172.21.15.40:0/797367832 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49940116c0 con 0x7f4990004d00 2023-10-18T09:03:25.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.046+0000 7f49aaba4640 1 -- 172.21.15.40:0/797367832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a4150b60 con 0x7f4990004d00 2023-10-18T09:03:25.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.046+0000 7f49aaba4640 1 -- 172.21.15.40:0/797367832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a41f4d50 con 0x7f4990004d00 2023-10-18T09:03:25.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.047+0000 7f49aaba4640 1 -- 172.21.15.40:0/797367832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a4066fd0 con 0x7f4990004d00 2023-10-18T09:03:25.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.048+0000 7f49a17fa640 1 -- 172.21.15.40:0/797367832 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4994011820 con 0x7f4990004d00 2023-10-18T09:03:25.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.049+0000 7f49a17fa640 1 --2- 172.21.15.40:0/797367832 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f498406ae70 0x7f498406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:25.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.049+0000 7f49a17fa640 1 -- 172.21.15.40:0/797367832 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f499408a510 con 0x7f4990004d00 2023-10-18T09:03:25.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.050+0000 7f49a3fff640 1 --2- 172.21.15.40:0/797367832 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f498406ae70 0x7f498406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:25.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.051+0000 7f49a3fff640 1 --2- 172.21.15.40:0/797367832 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f498406ae70 0x7f498406d330 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f498c006fd0 tx=0x7f498c000ed0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:25.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.054+0000 7f49a17fa640 1 -- 172.21.15.40:0/797367832 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f49940555c0 con 0x7f4990004d00 2023-10-18T09:03:25.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.247+0000 7f49aaba4640 1 -- 172.21.15.40:0/797367832 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49a4151080 con 0x7f4990004d00 2023-10-18T09:03:25.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.250+0000 7f49a17fa640 1 -- 172.21.15.40:0/797367832 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f49a4151080 con 0x7f4990004d00 2023-10-18T09:03:25.254 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:25.254 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":143,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:03:23.781764+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:03:21.754758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-18T09:03:22.819486+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:03:25.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.252+0000 7f49aaba4640 1 -- 172.21.15.40:0/797367832 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f498406ae70 msgr2=0x7f498406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:25.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.252+0000 7f49aaba4640 1 --2- 172.21.15.40:0/797367832 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f498406ae70 0x7f498406d330 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f498c006fd0 tx=0x7f498c000ed0 comp rx=0 tx=0).stop 2023-10-18T09:03:25.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.252+0000 7f49aaba4640 1 -- 172.21.15.40:0/797367832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4990004d00 msgr2=0x7f49a40cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:25.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.252+0000 7f49aaba4640 1 --2- 172.21.15.40:0/797367832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4990004d00 0x7f49a40cc9b0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f49940013d0 tx=0x7f4994004f90 comp rx=0 tx=0).stop 2023-10-18T09:03:25.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.252+0000 7f49aaba4640 1 -- 172.21.15.40:0/797367832 shutdown_connections 2023-10-18T09:03:25.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.252+0000 7f49aaba4640 1 --2- 172.21.15.40:0/797367832 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f498406ae70 0x7f498406d330 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:25.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.252+0000 7f49aaba4640 1 --2- 172.21.15.40:0/797367832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49a413c740 0x7f49a413cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:25.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.252+0000 7f49aaba4640 1 --2- 172.21.15.40:0/797367832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49a40ccef0 0x7f49a413c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:25.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.252+0000 7f49aaba4640 1 --2- 172.21.15.40:0/797367832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4990004d00 0x7f49a40cc9b0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:25.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.252+0000 7f49aaba4640 1 -- 172.21.15.40:0/797367832 >> 172.21.15.40:0/797367832 conn(0x7f49a406cee0 msgr2=0x7f49a40b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:25.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.253+0000 7f49aaba4640 1 -- 172.21.15.40:0/797367832 shutdown_connections 2023-10-18T09:03:25.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.253+0000 7f49aaba4640 1 -- 172.21.15.40:0/797367832 wait complete. 2023-10-18T09:03:25.274 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T09:03:25.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.507+0000 7f137ce90640 1 -- 172.21.15.40:0/2668580378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f137815b400 msgr2=0x7f13780b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:25.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.507+0000 7f137ce90640 1 --2- 172.21.15.40:0/2668580378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f137815b400 0x7f13780b11a0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f1368005fd0 tx=0x7f136802bd90 comp rx=0 tx=0).stop 2023-10-18T09:03:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.507+0000 7f137ce90640 1 -- 172.21.15.40:0/2668580378 shutdown_connections 2023-10-18T09:03:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.507+0000 7f137ce90640 1 --2- 172.21.15.40:0/2668580378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f137815b7e0 0x7f13780cf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.507+0000 7f137ce90640 1 --2- 172.21.15.40:0/2668580378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f137815b400 0x7f13780b11a0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.507+0000 7f137ce90640 1 -- 172.21.15.40:0/2668580378 >> 172.21.15.40:0/2668580378 conn(0x7f137808ad30 msgr2=0x7f137809a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:25.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.507+0000 7f137ce90640 1 -- 172.21.15.40:0/2668580378 shutdown_connections 2023-10-18T09:03:25.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.507+0000 7f137ce90640 1 -- 172.21.15.40:0/2668580378 wait complete. 2023-10-18T09:03:25.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.509+0000 7f137ce90640 1 Processor -- start 2023-10-18T09:03:25.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.509+0000 7f137ce90640 1 -- start start 2023-10-18T09:03:25.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.510+0000 7f137ce90640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13780cdbe0 0x7f1378157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:25.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.510+0000 7f137ce90640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f137815b400 0x7f13780c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:25.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.510+0000 7f137ce90640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f137815b7e0 0x7f13781596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:25.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.510+0000 7f137ce90640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1378159090 con 0x7f137815b7e0 2023-10-18T09:03:25.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.510+0000 7f137ce90640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13780b3f30 con 0x7f137815b400 2023-10-18T09:03:25.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.510+0000 7f137ce90640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13780c7a60 con 0x7f13780cdbe0 2023-10-18T09:03:25.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.510+0000 7f1377fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f137815b7e0 0x7f13781596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:25.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.510+0000 7f1377fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f137815b7e0 0x7f13781596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49626/0 (socket says 172.21.15.40:49626) 2023-10-18T09:03:25.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.510+0000 7f1377fff640 1 -- 172.21.15.40:0/3744264551 learned_addr learned my addr 172.21.15.40:0/3744264551 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:25.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.510+0000 7f1376ffd640 1 --2- 172.21.15.40:0/3744264551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f137815b400 0x7f13780c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:25.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.510+0000 7f13777fe640 1 --2- 172.21.15.40:0/3744264551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13780cdbe0 0x7f1378157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:25.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.511+0000 7f1377fff640 1 -- 172.21.15.40:0/3744264551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13780cdbe0 msgr2=0x7f1378157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:25.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.511+0000 7f1377fff640 1 --2- 172.21.15.40:0/3744264551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13780cdbe0 0x7f1378157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:25.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.511+0000 7f1377fff640 1 -- 172.21.15.40:0/3744264551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f137815b400 msgr2=0x7f13780c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:25.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.511+0000 7f1377fff640 1 --2- 172.21.15.40:0/3744264551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f137815b400 0x7f13780c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:25.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.511+0000 7f1377fff640 1 -- 172.21.15.40:0/3744264551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1368009560 con 0x7f137815b7e0 2023-10-18T09:03:25.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.511+0000 7f1377fff640 1 --2- 172.21.15.40:0/3744264551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f137815b7e0 0x7f13781596f0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f137000c780 tx=0x7f137000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:25.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.511+0000 7f136ffff640 1 -- 172.21.15.40:0/3744264551 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1370004b60 con 0x7f137815b7e0 2023-10-18T09:03:25.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.511+0000 7f136ffff640 1 -- 172.21.15.40:0/3744264551 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f137000ce90 con 0x7f137815b7e0 2023-10-18T09:03:25.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.512+0000 7f136ffff640 1 -- 172.21.15.40:0/3744264551 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137000aab0 con 0x7f137815b7e0 2023-10-18T09:03:25.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.512+0000 7f137ce90640 1 -- 172.21.15.40:0/3744264551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1378158340 con 0x7f137815b7e0 2023-10-18T09:03:25.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.512+0000 7f137ce90640 1 -- 172.21.15.40:0/3744264551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1378158900 con 0x7f137815b7e0 2023-10-18T09:03:25.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.513+0000 7f137ce90640 1 -- 172.21.15.40:0/3744264551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1378071da0 con 0x7f137815b7e0 2023-10-18T09:03:25.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.514+0000 7f136ffff640 1 -- 172.21.15.40:0/3744264551 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f137000ac10 con 0x7f137815b7e0 2023-10-18T09:03:25.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.515+0000 7f136ffff640 1 --2- 172.21.15.40:0/3744264551 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f135806ae70 0x7f135806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:25.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.515+0000 7f136ffff640 1 -- 172.21.15.40:0/3744264551 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1370089900 con 0x7f137815b7e0 2023-10-18T09:03:25.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.516+0000 7f13777fe640 1 --2- 172.21.15.40:0/3744264551 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f135806ae70 0x7f135806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:25.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.517+0000 7f13777fe640 1 --2- 172.21.15.40:0/3744264551 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f135806ae70 0x7f135806d330 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1368035000 tx=0x7f1368004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:25.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.519+0000 7f136ffff640 1 -- 172.21.15.40:0/3744264551 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1370054a40 con 0x7f137815b7e0 2023-10-18T09:03:25.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:25.748+0000 7f137ce90640 1 -- 172.21.15.40:0/3744264551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f13780b40a0 con 0x7f137815b7e0 2023-10-18T09:03:26.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.390+0000 7f136ffff640 1 -- 172.21.15.40:0/3744264551 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v65) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1370058060 con 0x7f137815b7e0 2023-10-18T09:03:26.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.394+0000 7f137ce90640 1 -- 172.21.15.40:0/3744264551 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f135806ae70 msgr2=0x7f135806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:26.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.394+0000 7f137ce90640 1 --2- 172.21.15.40:0/3744264551 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f135806ae70 0x7f135806d330 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1368035000 tx=0x7f1368004730 comp rx=0 tx=0).stop 2023-10-18T09:03:26.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.394+0000 7f137ce90640 1 -- 172.21.15.40:0/3744264551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f137815b7e0 msgr2=0x7f13781596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:26.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.394+0000 7f137ce90640 1 --2- 172.21.15.40:0/3744264551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f137815b7e0 0x7f13781596f0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f137000c780 tx=0x7f137000cc40 comp rx=0 tx=0).stop 2023-10-18T09:03:26.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.394+0000 7f137ce90640 1 -- 172.21.15.40:0/3744264551 shutdown_connections 2023-10-18T09:03:26.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.394+0000 7f137ce90640 1 --2- 172.21.15.40:0/3744264551 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f135806ae70 0x7f135806d330 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:26.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.394+0000 7f137ce90640 1 --2- 172.21.15.40:0/3744264551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f137815b7e0 0x7f13781596f0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:26.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.394+0000 7f137ce90640 1 --2- 172.21.15.40:0/3744264551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f137815b400 0x7f13780c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:26.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.394+0000 7f137ce90640 1 --2- 172.21.15.40:0/3744264551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13780cdbe0 0x7f1378157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:26.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.394+0000 7f137ce90640 1 -- 172.21.15.40:0/3744264551 >> 172.21.15.40:0/3744264551 conn(0x7f137808ad30 msgr2=0x7f137815a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:26.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.394+0000 7f137ce90640 1 -- 172.21.15.40:0/3744264551 shutdown_connections 2023-10-18T09:03:26.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.394+0000 7f137ce90640 1 -- 172.21.15.40:0/3744264551 wait complete. 2023-10-18T09:03:26.419 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T09:03:26.420 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T09:03:26.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.647+0000 7f4e5759e640 1 -- 172.21.15.40:0/274286661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e40003f40 msgr2=0x7f4e400047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:26.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.647+0000 7f4e5759e640 1 --2- 172.21.15.40:0/274286661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e40003f40 0x7f4e400047c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009b50 tx=0x7f4e400057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:26.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.647+0000 7f4e56d9d640 1 -- 172.21.15.40:0/274286661 reap_dead start 2023-10-18T09:03:26.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.648+0000 7f4e5759e640 1 -- 172.21.15.40:0/274286661 shutdown_connections 2023-10-18T09:03:26.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.648+0000 7f4e5759e640 1 -- 172.21.15.40:0/274286661 >> 172.21.15.40:0/274286661 conn(0x7f4e5806ca10 msgr2=0x7f4e5806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:26.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.648+0000 7f4e5759e640 1 -- 172.21.15.40:0/274286661 shutdown_connections 2023-10-18T09:03:26.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.648+0000 7f4e5759e640 1 -- 172.21.15.40:0/274286661 wait complete. 2023-10-18T09:03:26.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.649+0000 7f4e5759e640 1 Processor -- start 2023-10-18T09:03:26.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.650+0000 7f4e5759e640 1 -- start start 2023-10-18T09:03:26.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.650+0000 7f4e5759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e40003f40 0x7f4e58137660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:26.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.650+0000 7f4e5759e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e58148130 0x7f4e5814e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:26.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.651+0000 7f4e5759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e5814ebb0 0x7f4e580ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:26.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.651+0000 7f4e5759e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e581504a0 con 0x7f4e5814ebb0 2023-10-18T09:03:26.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.651+0000 7f4e5759e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e58150710 con 0x7f4e40003f40 2023-10-18T09:03:26.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.651+0000 7f4e5759e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e581509f0 con 0x7f4e58148130 2023-10-18T09:03:26.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.651+0000 7f4e56d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e5814ebb0 0x7f4e580ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:26.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.651+0000 7f4e56d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e5814ebb0 0x7f4e580ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49634/0 (socket says 172.21.15.40:49634) 2023-10-18T09:03:26.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.651+0000 7f4e56d9d640 1 -- 172.21.15.40:0/649939434 learned_addr learned my addr 172.21.15.40:0/649939434 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:26.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.651+0000 7f4e55d9b640 1 --2- 172.21.15.40:0/649939434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e58148130 0x7f4e5814e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:26.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.651+0000 7f4e5659c640 1 --2- 172.21.15.40:0/649939434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e40003f40 0x7f4e58137660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:26.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.651+0000 7f4e56d9d640 1 -- 172.21.15.40:0/649939434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e58148130 msgr2=0x7f4e5814e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:26.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.651+0000 7f4e56d9d640 1 --2- 172.21.15.40:0/649939434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e58148130 0x7f4e5814e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.651+0000 7f4e56d9d640 1 -- 172.21.15.40:0/649939434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e40003f40 msgr2=0x7f4e58137660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.651+0000 7f4e56d9d640 1 --2- 172.21.15.40:0/649939434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e40003f40 0x7f4e58137660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:26.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.651+0000 7f4e56d9d640 1 -- 172.21.15.40:0/649939434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e58150cd0 con 0x7f4e5814ebb0 2023-10-18T09:03:26.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.652+0000 7f4e56d9d640 1 --2- 172.21.15.40:0/649939434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e5814ebb0 0x7f4e580ccdc0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4e580afcb0 tx=0x7f4e50010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:26.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.652+0000 7f4e46ffd640 1 -- 172.21.15.40:0/649939434 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5000eeb0 con 0x7f4e5814ebb0 2023-10-18T09:03:26.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.652+0000 7f4e46ffd640 1 -- 172.21.15.40:0/649939434 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4e50003a60 con 0x7f4e5814ebb0 2023-10-18T09:03:26.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.652+0000 7f4e5759e640 1 -- 172.21.15.40:0/649939434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e58150fb0 con 0x7f4e5814ebb0 2023-10-18T09:03:26.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.652+0000 7f4e46ffd640 1 -- 172.21.15.40:0/649939434 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50003e70 con 0x7f4e5814ebb0 2023-10-18T09:03:26.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.652+0000 7f4e5759e640 1 -- 172.21.15.40:0/649939434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e581514d0 con 0x7f4e5814ebb0 2023-10-18T09:03:26.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.653+0000 7f4e5759e640 1 -- 172.21.15.40:0/649939434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e58070b40 con 0x7f4e5814ebb0 2023-10-18T09:03:26.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.654+0000 7f4e46ffd640 1 -- 172.21.15.40:0/649939434 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4e50019030 con 0x7f4e5814ebb0 2023-10-18T09:03:26.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.656+0000 7f4e46ffd640 1 --2- 172.21.15.40:0/649939434 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4e3806ae70 0x7f4e3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:26.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.656+0000 7f4e46ffd640 1 -- 172.21.15.40:0/649939434 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e50017070 con 0x7f4e5814ebb0 2023-10-18T09:03:26.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.657+0000 7f4e5659c640 1 --2- 172.21.15.40:0/649939434 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4e3806ae70 0x7f4e3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:26.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.658+0000 7f4e5659c640 1 --2- 172.21.15.40:0/649939434 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4e3806ae70 0x7f4e3806d330 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4e48000c00 tx=0x7f4e48001cf0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:26.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.661+0000 7f4e46ffd640 1 -- 172.21.15.40:0/649939434 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4e58070b40 con 0x7f4e5814ebb0 2023-10-18T09:03:26.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:26.895+0000 7f4e5759e640 1 -- 172.21.15.40:0/649939434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4e58070b40 con 0x7f4e5814ebb0 2023-10-18T09:03:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.400+0000 7f4e46ffd640 1 -- 172.21.15.40:0/649939434 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v66) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4e58070b40 con 0x7f4e5814ebb0 2023-10-18T09:03:27.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.404+0000 7f4e5759e640 1 -- 172.21.15.40:0/649939434 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4e3806ae70 msgr2=0x7f4e3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:27.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.404+0000 7f4e5759e640 1 --2- 172.21.15.40:0/649939434 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4e3806ae70 0x7f4e3806d330 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4e48000c00 tx=0x7f4e48001cf0 comp rx=0 tx=0).stop 2023-10-18T09:03:27.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.404+0000 7f4e5759e640 1 -- 172.21.15.40:0/649939434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e5814ebb0 msgr2=0x7f4e580ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:27.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.404+0000 7f4e5759e640 1 --2- 172.21.15.40:0/649939434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e5814ebb0 0x7f4e580ccdc0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4e580afcb0 tx=0x7f4e50010f40 comp rx=0 tx=0).stop 2023-10-18T09:03:27.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.405+0000 7f4e5759e640 1 -- 172.21.15.40:0/649939434 shutdown_connections 2023-10-18T09:03:27.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.405+0000 7f4e5759e640 1 --2- 172.21.15.40:0/649939434 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4e3806ae70 0x7f4e3806d330 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:27.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.405+0000 7f4e5759e640 1 --2- 172.21.15.40:0/649939434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e5814ebb0 0x7f4e580ccdc0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:27.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.405+0000 7f4e5759e640 1 --2- 172.21.15.40:0/649939434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e58148130 0x7f4e5814e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:27.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.405+0000 7f4e5759e640 1 --2- 172.21.15.40:0/649939434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e40003f40 0x7f4e58137660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:27.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.405+0000 7f4e5759e640 1 -- 172.21.15.40:0/649939434 >> 172.21.15.40:0/649939434 conn(0x7f4e5806ca10 msgr2=0x7f4e58155510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:27.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.405+0000 7f4e5759e640 1 -- 172.21.15.40:0/649939434 shutdown_connections 2023-10-18T09:03:27.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.405+0000 7f4e5759e640 1 -- 172.21.15.40:0/649939434 wait complete. 2023-10-18T09:03:27.434 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T09:03:27.434 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:03:27.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.667+0000 7f0580d73640 1 -- 172.21.15.40:0/3110685240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05680058d0 msgr2=0x7f0568005d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:27.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.667+0000 7f0580d73640 1 --2- 172.21.15.40:0/3110685240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05680058d0 0x7f0568005d50 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f0564001680 tx=0x7f0568006900 comp rx=0 tx=0).stop 2023-10-18T09:03:27.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.667+0000 7f0580d73640 1 -- 172.21.15.40:0/3110685240 shutdown_connections 2023-10-18T09:03:27.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.667+0000 7f0580d73640 1 --2- 172.21.15.40:0/3110685240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05680058d0 0x7f0568005d50 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:27.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.667+0000 7f0580d73640 1 -- 172.21.15.40:0/3110685240 >> 172.21.15.40:0/3110685240 conn(0x7f057c06cee0 msgr2=0x7f057c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:27.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.667+0000 7f0580d73640 1 -- 172.21.15.40:0/3110685240 shutdown_connections 2023-10-18T09:03:27.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.667+0000 7f0580d73640 1 -- 172.21.15.40:0/3110685240 wait complete. 2023-10-18T09:03:27.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.669+0000 7f0580d73640 1 Processor -- start 2023-10-18T09:03:27.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.670+0000 7f0580d73640 1 -- start start 2023-10-18T09:03:27.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.671+0000 7f0580d73640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05680058d0 0x7f057c156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:27.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.671+0000 7f0580d73640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f057c1eda20 0x7f057c1edea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:27.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.671+0000 7f0580d73640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f057c152cc0 0x7f057c153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:27.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.671+0000 7f0580d73640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c1ee3e0 con 0x7f05680058d0 2023-10-18T09:03:27.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.671+0000 7f0580d73640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c1539f0 con 0x7f057c1eda20 2023-10-18T09:03:27.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.671+0000 7f0580d73640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057c153cd0 con 0x7f057c152cc0 2023-10-18T09:03:27.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.671+0000 7f057a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05680058d0 0x7f057c156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.671+0000 7f057a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05680058d0 0x7f057c156cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49638/0 (socket says 172.21.15.40:49638) 2023-10-18T09:03:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.671+0000 7f057a59c640 1 -- 172.21.15.40:0/2088839491 learned_addr learned my addr 172.21.15.40:0/2088839491 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.671+0000 7f0579d9b640 1 --2- 172.21.15.40:0/2088839491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f057c1eda20 0x7f057c1edea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.671+0000 7f057a59c640 1 -- 172.21.15.40:0/2088839491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f057c152cc0 msgr2=0x7f057c153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:27.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.671+0000 7f057a59c640 1 --2- 172.21.15.40:0/2088839491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f057c152cc0 0x7f057c153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:27.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.671+0000 7f057a59c640 1 -- 172.21.15.40:0/2088839491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f057c1eda20 msgr2=0x7f057c1edea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:27.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.671+0000 7f057a59c640 1 --2- 172.21.15.40:0/2088839491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f057c1eda20 0x7f057c1edea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:27.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.671+0000 7f057a59c640 1 -- 172.21.15.40:0/2088839491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c153fb0 con 0x7f05680058d0 2023-10-18T09:03:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.672+0000 7f057a59c640 1 --2- 172.21.15.40:0/2088839491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05680058d0 0x7f057c156cb0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f056c00d8f0 tx=0x7f056c00ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.672+0000 7f0572ffd640 1 -- 172.21.15.40:0/2088839491 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c001d60 con 0x7f05680058d0 2023-10-18T09:03:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.672+0000 7f0572ffd640 1 -- 172.21.15.40:0/2088839491 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f056c0063d0 con 0x7f05680058d0 2023-10-18T09:03:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.672+0000 7f0580d73640 1 -- 172.21.15.40:0/2088839491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057c1eec80 con 0x7f05680058d0 2023-10-18T09:03:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.672+0000 7f0572ffd640 1 -- 172.21.15.40:0/2088839491 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c00b9f0 con 0x7f05680058d0 2023-10-18T09:03:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.672+0000 7f0580d73640 1 -- 172.21.15.40:0/2088839491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057c153680 con 0x7f05680058d0 2023-10-18T09:03:27.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.673+0000 7f0580d73640 1 -- 172.21.15.40:0/2088839491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057c066fd0 con 0x7f05680058d0 2023-10-18T09:03:27.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.674+0000 7f0572ffd640 1 -- 172.21.15.40:0/2088839491 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f056c00bb50 con 0x7f05680058d0 2023-10-18T09:03:27.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.675+0000 7f0572ffd640 1 --2- 172.21.15.40:0/2088839491 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f055c06ae70 0x7f055c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:27.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.675+0000 7f0572ffd640 1 -- 172.21.15.40:0/2088839491 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f056c089fd0 con 0x7f05680058d0 2023-10-18T09:03:27.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.676+0000 7f0579d9b640 1 --2- 172.21.15.40:0/2088839491 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f055c06ae70 0x7f055c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:27.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.677+0000 7f0579d9b640 1 --2- 172.21.15.40:0/2088839491 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f055c06ae70 0x7f055c06d330 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0564034000 tx=0x7f0564039040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:27.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.679+0000 7f0572ffd640 1 -- 172.21.15.40:0/2088839491 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f056c054fe0 con 0x7f05680058d0 2023-10-18T09:03:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.911+0000 7f0580d73640 1 -- 172.21.15.40:0/2088839491 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f057c0cc490 con 0x7f05680058d0 2023-10-18T09:03:27.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.912+0000 7f0572ffd640 1 -- 172.21.15.40:0/2088839491 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f056c014020 con 0x7f05680058d0 2023-10-18T09:03:27.916 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:27.916 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6382,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/4165164542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":4165164542},{"type":"v1","addr":"172.21.15.144:6836","nonce":4165164542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3045960359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3045960359},{"type":"v1","addr":"172.21.15.144:6837","nonce":3045960359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2869040025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2869040025},{"type":"v1","addr":"172.21.15.40:6833","nonce":2869040025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:03:23.782442+0000","modified":"2023-10-18T09:03:26.897709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7893},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7893":{"gid":7893,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2600609919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2600609919},{"type":"v1","addr":"172.21.15.40:6835","nonce":2600609919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-18T09:03:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 66 2023-10-18T09:03:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.915+0000 7f0580d73640 1 -- 172.21.15.40:0/2088839491 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f055c06ae70 msgr2=0x7f055c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:27.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.915+0000 7f0580d73640 1 --2- 172.21.15.40:0/2088839491 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f055c06ae70 0x7f055c06d330 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0564034000 tx=0x7f0564039040 comp rx=0 tx=0).stop 2023-10-18T09:03:27.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.915+0000 7f0580d73640 1 -- 172.21.15.40:0/2088839491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05680058d0 msgr2=0x7f057c156cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:27.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.915+0000 7f0580d73640 1 --2- 172.21.15.40:0/2088839491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05680058d0 0x7f057c156cb0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f056c00d8f0 tx=0x7f056c00ddb0 comp rx=0 tx=0).stop 2023-10-18T09:03:27.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.915+0000 7f0580d73640 1 -- 172.21.15.40:0/2088839491 shutdown_connections 2023-10-18T09:03:27.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.915+0000 7f0580d73640 1 --2- 172.21.15.40:0/2088839491 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f055c06ae70 0x7f055c06d330 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:27.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.915+0000 7f0580d73640 1 --2- 172.21.15.40:0/2088839491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f057c152cc0 0x7f057c153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:27.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.915+0000 7f0580d73640 1 --2- 172.21.15.40:0/2088839491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f057c1eda20 0x7f057c1edea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:27.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.915+0000 7f0580d73640 1 --2- 172.21.15.40:0/2088839491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05680058d0 0x7f057c156cb0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:27.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.915+0000 7f0580d73640 1 -- 172.21.15.40:0/2088839491 >> 172.21.15.40:0/2088839491 conn(0x7f057c06cee0 msgr2=0x7f057c150bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:27.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.915+0000 7f0580d73640 1 -- 172.21.15.40:0/2088839491 shutdown_connections 2023-10-18T09:03:27.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:27.916+0000 7f0580d73640 1 -- 172.21.15.40:0/2088839491 wait complete. 2023-10-18T09:03:27.938 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:03:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.171+0000 7f06c6d0a640 1 -- 172.21.15.40:0/715629407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06c01420f0 msgr2=0x7f06c00b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.171+0000 7f06c6d0a640 1 --2- 172.21.15.40:0/715629407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06c01420f0 0x7f06c00b0a80 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f06bc005fd0 tx=0x7f06bc02be10 comp rx=0 tx=0).stop 2023-10-18T09:03:28.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.172+0000 7f06c6d0a640 1 -- 172.21.15.40:0/715629407 shutdown_connections 2023-10-18T09:03:28.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.172+0000 7f06c6d0a640 1 --2- 172.21.15.40:0/715629407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c00ca160 0x7f06c00cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.172+0000 7f06c6d0a640 1 --2- 172.21.15.40:0/715629407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06c01420f0 0x7f06c00b0a80 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.172+0000 7f06c6d0a640 1 -- 172.21.15.40:0/715629407 >> 172.21.15.40:0/715629407 conn(0x7f06c0069720 msgr2=0x7f06c0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:28.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.172+0000 7f06c6d0a640 1 -- 172.21.15.40:0/715629407 shutdown_connections 2023-10-18T09:03:28.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.172+0000 7f06c6d0a640 1 -- 172.21.15.40:0/715629407 wait complete. 2023-10-18T09:03:28.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.174+0000 7f06c6d0a640 1 Processor -- start 2023-10-18T09:03:28.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.174+0000 7f06c6d0a640 1 -- start start 2023-10-18T09:03:28.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.175+0000 7f06c6d0a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c00ca160 0x7f06c01e6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:28.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.175+0000 7f06c6d0a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c01420f0 0x7f06c013f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:28.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.175+0000 7f06c6d0a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06c01425a0 0x7f06c013c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:28.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.175+0000 7f06c6d0a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c01f5d20 con 0x7f06c01425a0 2023-10-18T09:03:28.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.175+0000 7f06c6d0a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c013a980 con 0x7f06c00ca160 2023-10-18T09:03:28.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.175+0000 7f06c6d0a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c01f5110 con 0x7f06c01420f0 2023-10-18T09:03:28.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.175+0000 7f06c6509640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06c01425a0 0x7f06c013c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:28.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.175+0000 7f06c6509640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06c01425a0 0x7f06c013c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49656/0 (socket says 172.21.15.40:49656) 2023-10-18T09:03:28.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.175+0000 7f06c6509640 1 -- 172.21.15.40:0/294774182 learned_addr learned my addr 172.21.15.40:0/294774182 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:28.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.176+0000 7f06c5d08640 1 --2- 172.21.15.40:0/294774182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c00ca160 0x7f06c01e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:28.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.176+0000 7f06c5507640 1 --2- 172.21.15.40:0/294774182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c01420f0 0x7f06c013f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:28.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.176+0000 7f06c6509640 1 -- 172.21.15.40:0/294774182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c01420f0 msgr2=0x7f06c013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:28.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.176+0000 7f06c6509640 1 --2- 172.21.15.40:0/294774182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c01420f0 0x7f06c013f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.176+0000 7f06c6509640 1 -- 172.21.15.40:0/294774182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c00ca160 msgr2=0x7f06c01e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:28.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.176+0000 7f06c6509640 1 --2- 172.21.15.40:0/294774182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c00ca160 0x7f06c01e6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.176+0000 7f06c6509640 1 -- 172.21.15.40:0/294774182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc009560 con 0x7f06c01425a0 2023-10-18T09:03:28.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.176+0000 7f06c5507640 1 --2- 172.21.15.40:0/294774182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c01420f0 0x7f06c013f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:28.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.176+0000 7f06c6509640 1 --2- 172.21.15.40:0/294774182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06c01425a0 0x7f06c013c4e0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f06b800eb10 tx=0x7f06b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:28.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.176+0000 7f06b67fc640 1 -- 172.21.15.40:0/294774182 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b800ce80 con 0x7f06c01425a0 2023-10-18T09:03:28.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.176+0000 7f06b67fc640 1 -- 172.21.15.40:0/294774182 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f06b8004c10 con 0x7f06c01425a0 2023-10-18T09:03:28.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.176+0000 7f06b67fc640 1 -- 172.21.15.40:0/294774182 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8005020 con 0x7f06c01425a0 2023-10-18T09:03:28.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.176+0000 7f06c6d0a640 1 -- 172.21.15.40:0/294774182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c01f53f0 con 0x7f06c01425a0 2023-10-18T09:03:28.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.177+0000 7f06c6d0a640 1 -- 172.21.15.40:0/294774182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c01f59b0 con 0x7f06c01425a0 2023-10-18T09:03:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.178+0000 7f06c6d0a640 1 -- 172.21.15.40:0/294774182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c0066e00 con 0x7f06c01425a0 2023-10-18T09:03:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.179+0000 7f06b67fc640 1 -- 172.21.15.40:0/294774182 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f06b8005180 con 0x7f06c01425a0 2023-10-18T09:03:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.179+0000 7f06b67fc640 1 --2- 172.21.15.40:0/294774182 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f06a406ae40 0x7f06a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.180+0000 7f06b67fc640 1 -- 172.21.15.40:0/294774182 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06b8014070 con 0x7f06c01425a0 2023-10-18T09:03:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.180+0000 7f06c5d08640 1 --2- 172.21.15.40:0/294774182 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f06a406ae40 0x7f06a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:28.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.181+0000 7f06c5d08640 1 --2- 172.21.15.40:0/294774182 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f06a406ae40 0x7f06a406d300 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f06bc035000 tx=0x7f06bc004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:28.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.183+0000 7f06b67fc640 1 -- 172.21.15.40:0/294774182 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f06b8055590 con 0x7f06c01425a0 2023-10-18T09:03:28.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.381+0000 7f06c6d0a640 1 -- 172.21.15.40:0/294774182 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06c006d920 con 0x7f06c01425a0 2023-10-18T09:03:28.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.383+0000 7f06b67fc640 1 -- 172.21.15.40:0/294774182 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+16540 (secure 0 0 0) 0x7f06b801d090 con 0x7f06c01425a0 2023-10-18T09:03:28.387 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:28.387 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":144,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:03:25.341390+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:03:21.754758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-18T09:03:22.819486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:03:28.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.386+0000 7f06c6d0a640 1 -- 172.21.15.40:0/294774182 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f06a406ae40 msgr2=0x7f06a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:28.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.386+0000 7f06c6d0a640 1 --2- 172.21.15.40:0/294774182 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f06a406ae40 0x7f06a406d300 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f06bc035000 tx=0x7f06bc004730 comp rx=0 tx=0).stop 2023-10-18T09:03:28.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.386+0000 7f06c6d0a640 1 -- 172.21.15.40:0/294774182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06c01425a0 msgr2=0x7f06c013c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:28.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.386+0000 7f06c6d0a640 1 --2- 172.21.15.40:0/294774182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06c01425a0 0x7f06c013c4e0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f06b800eb10 tx=0x7f06b800c2d0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.386+0000 7f06c6d0a640 1 -- 172.21.15.40:0/294774182 shutdown_connections 2023-10-18T09:03:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.386+0000 7f06c6d0a640 1 --2- 172.21.15.40:0/294774182 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f06a406ae40 0x7f06a406d300 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.386+0000 7f06c6d0a640 1 --2- 172.21.15.40:0/294774182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06c01425a0 0x7f06c013c4e0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.386+0000 7f06c6d0a640 1 --2- 172.21.15.40:0/294774182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06c01420f0 0x7f06c013f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.386+0000 7f06c6d0a640 1 --2- 172.21.15.40:0/294774182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06c00ca160 0x7f06c01e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.386+0000 7f06c6d0a640 1 -- 172.21.15.40:0/294774182 >> 172.21.15.40:0/294774182 conn(0x7f06c0069720 msgr2=0x7f06c00acce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:28.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.386+0000 7f06c6d0a640 1 -- 172.21.15.40:0/294774182 shutdown_connections 2023-10-18T09:03:28.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.386+0000 7f06c6d0a640 1 -- 172.21.15.40:0/294774182 wait complete. 2023-10-18T09:03:28.410 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T09:03:28.411 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:03:28.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.640+0000 7f6f791d7640 1 -- 172.21.15.40:0/600828450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f740b1e10 msgr2=0x7f6f740c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:28.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.640+0000 7f6f791d7640 1 --2- 172.21.15.40:0/600828450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f740b1e10 0x7f6f740c99c0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f6f7406f590 tx=0x7f6f5c02bd90 comp rx=0 tx=0).stop 2023-10-18T09:03:28.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.640+0000 7f6f791d7640 1 -- 172.21.15.40:0/600828450 shutdown_connections 2023-10-18T09:03:28.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.640+0000 7f6f791d7640 1 --2- 172.21.15.40:0/600828450 >> v2:172.21.15.144:3300/0 conn(0x7f6f740b2270 0x7f6f740c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.640+0000 7f6f791d7640 1 --2- 172.21.15.40:0/600828450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f740b1e10 0x7f6f740c99c0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.640+0000 7f6f791d7640 1 --2- 172.21.15.40:0/600828450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f740b15b0 0x7f6f740b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.640+0000 7f6f791d7640 1 -- 172.21.15.40:0/600828450 >> 172.21.15.40:0/600828450 conn(0x7f6f74069720 msgr2=0x7f6f74069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:28.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.640+0000 7f6f791d7640 1 -- 172.21.15.40:0/600828450 shutdown_connections 2023-10-18T09:03:28.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.640+0000 7f6f791d7640 1 -- 172.21.15.40:0/600828450 wait complete. 2023-10-18T09:03:28.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.642+0000 7f6f791d7640 1 Processor -- start 2023-10-18T09:03:28.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.642+0000 7f6f791d7640 1 -- start start 2023-10-18T09:03:28.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.643+0000 7f6f791d7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f740b15b0 0x7f6f740ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:28.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.643+0000 7f6f791d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f740b1e10 0x7f6f740c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:28.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.643+0000 7f6f791d7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f740b2270 0x7f6f740c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:28.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.643+0000 7f6f791d7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7414edd0 con 0x7f6f740b2270 2023-10-18T09:03:28.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.643+0000 7f6f791d7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740afac0 con 0x7f6f740b1e10 2023-10-18T09:03:28.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.643+0000 7f6f791d7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7414ea30 con 0x7f6f740b15b0 2023-10-18T09:03:28.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.644+0000 7f6f789d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f740b2270 0x7f6f740c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:28.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.644+0000 7f6f789d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f740b2270 0x7f6f740c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49684/0 (socket says 172.21.15.40:49684) 2023-10-18T09:03:28.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.644+0000 7f6f789d6640 1 -- 172.21.15.40:0/75122774 learned_addr learned my addr 172.21.15.40:0/75122774 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:28.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.644+0000 7f6f73fff640 1 --2- 172.21.15.40:0/75122774 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f740b15b0 0x7f6f740ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:28.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.644+0000 7f6f6bfff640 1 --2- 172.21.15.40:0/75122774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f740b1e10 0x7f6f740c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:28.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.644+0000 7f6f789d6640 1 -- 172.21.15.40:0/75122774 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f740b15b0 msgr2=0x7f6f740ae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:28.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.644+0000 7f6f789d6640 1 --2- 172.21.15.40:0/75122774 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f740b15b0 0x7f6f740ae9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.644+0000 7f6f789d6640 1 -- 172.21.15.40:0/75122774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f740b1e10 msgr2=0x7f6f740c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:28.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.644+0000 7f6f789d6640 1 --2- 172.21.15.40:0/75122774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f740b1e10 0x7f6f740c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.644+0000 7f6f789d6640 1 -- 172.21.15.40:0/75122774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c009560 con 0x7f6f740b2270 2023-10-18T09:03:28.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.644+0000 7f6f6bfff640 1 --2- 172.21.15.40:0/75122774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f740b1e10 0x7f6f740c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:28.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.645+0000 7f6f789d6640 1 --2- 172.21.15.40:0/75122774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f740b2270 0x7f6f740c9d90 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009d30 tx=0x7f6f6c00c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:28.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.645+0000 7f6f717fa640 1 -- 172.21.15.40:0/75122774 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c018070 con 0x7f6f740b2270 2023-10-18T09:03:28.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.645+0000 7f6f717fa640 1 -- 172.21.15.40:0/75122774 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6f6c010040 con 0x7f6f740b2270 2023-10-18T09:03:28.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.645+0000 7f6f717fa640 1 -- 172.21.15.40:0/75122774 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c009e20 con 0x7f6f740b2270 2023-10-18T09:03:28.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.645+0000 7f6f791d7640 1 -- 172.21.15.40:0/75122774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f740af220 con 0x7f6f740b2270 2023-10-18T09:03:28.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.645+0000 7f6f791d7640 1 -- 172.21.15.40:0/75122774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f740af7e0 con 0x7f6f740b2270 2023-10-18T09:03:28.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.646+0000 7f6f791d7640 1 -- 172.21.15.40:0/75122774 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7414ef10 con 0x7f6f740b2270 2023-10-18T09:03:28.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.647+0000 7f6f717fa640 1 -- 172.21.15.40:0/75122774 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6f6c014430 con 0x7f6f740b2270 2023-10-18T09:03:28.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.648+0000 7f6f717fa640 1 --2- 172.21.15.40:0/75122774 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6f5406ae40 0x7f6f5406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:28.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.648+0000 7f6f717fa640 1 -- 172.21.15.40:0/75122774 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f6c089970 con 0x7f6f740b2270 2023-10-18T09:03:28.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.649+0000 7f6f73fff640 1 --2- 172.21.15.40:0/75122774 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6f5406ae40 0x7f6f5406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:28.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.650+0000 7f6f73fff640 1 --2- 172.21.15.40:0/75122774 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6f5406ae40 0x7f6f5406d300 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6f64005fd0 tx=0x7f6f64005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:28.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.652+0000 7f6f717fa640 1 -- 172.21.15.40:0/75122774 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6f7414ef10 con 0x7f6f740b2270 2023-10-18T09:03:28.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.845+0000 7f6f791d7640 1 -- 172.21.15.40:0/75122774 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f740b1990 con 0x7f6f5406ae40 2023-10-18T09:03:28.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.868+0000 7f6f717fa640 1 -- 172.21.15.40:0/75122774 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279054 (secure 0 0 0) 0x7f6f6400b360 con 0x7f6f5406ae40 2023-10-18T09:03:28.875 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:28.877 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:03:28.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.875+0000 7f6f791d7640 1 -- 172.21.15.40:0/75122774 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6f5406ae40 msgr2=0x7f6f5406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:28.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.875+0000 7f6f791d7640 1 --2- 172.21.15.40:0/75122774 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6f5406ae40 0x7f6f5406d300 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6f64005fd0 tx=0x7f6f64005850 comp rx=0 tx=0).stop 2023-10-18T09:03:28.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.875+0000 7f6f791d7640 1 -- 172.21.15.40:0/75122774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f740b2270 msgr2=0x7f6f740c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:28.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.875+0000 7f6f791d7640 1 --2- 172.21.15.40:0/75122774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f740b2270 0x7f6f740c9d90 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009d30 tx=0x7f6f6c00c560 comp rx=0 tx=0).stop 2023-10-18T09:03:28.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.875+0000 7f6f791d7640 1 -- 172.21.15.40:0/75122774 shutdown_connections 2023-10-18T09:03:28.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.875+0000 7f6f791d7640 1 --2- 172.21.15.40:0/75122774 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6f5406ae40 0x7f6f5406d300 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.875+0000 7f6f791d7640 1 --2- 172.21.15.40:0/75122774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f740b2270 0x7f6f740c9d90 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.875+0000 7f6f791d7640 1 --2- 172.21.15.40:0/75122774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f740b1e10 0x7f6f740c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.875+0000 7f6f791d7640 1 --2- 172.21.15.40:0/75122774 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f740b15b0 0x7f6f740ae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:28.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.875+0000 7f6f791d7640 1 -- 172.21.15.40:0/75122774 >> 172.21.15.40:0/75122774 conn(0x7f6f74069720 msgr2=0x7f6f74155830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:28.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.875+0000 7f6f791d7640 1 -- 172.21.15.40:0/75122774 shutdown_connections 2023-10-18T09:03:28.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:28.875+0000 7f6f791d7640 1 -- 172.21.15.40:0/75122774 wait complete. 2023-10-18T09:03:28.898 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2023-10-18T09:03:27.075260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7371,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555128,"statfs":{"total":1600000000,"available":1592451471,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.776138"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313192+0000","last_change":"2023-10-18T09:03:23.313192+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313192+0000","last_undegraded":"2023-10-18T09:03:23.313192+0000","last_fullsized":"2023-10-18T09:03:23.313192+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.322631+0000","last_change":"2023-10-18T09:03:23.322631+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.322631+0000","last_undegraded":"2023-10-18T09:03:23.322631+0000","last_fullsized":"2023-10-18T09:03:23.322631+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.372965+0000","last_change":"2023-10-18T09:03:24.371528+0000","last_active":"2023-10-18T09:03:24.372965+0000","last_peered":"2023-10-18T09:03:24.372965+0000","last_clean":"2023-10-18T09:03:24.372965+0000","last_became_active":"2023-10-18T09:03:24.370994+0000","last_became_peered":"2023-10-18T09:03:24.370994+0000","last_unstale":"2023-10-18T09:03:24.372965+0000","last_undegraded":"2023-10-18T09:03:24.372965+0000","last_fullsized":"2023-10-18T09:03:24.372965+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:31.969146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.325742+0000","last_change":"2023-10-18T09:03:23.325742+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.325742+0000","last_undegraded":"2023-10-18T09:03:23.325742+0000","last_fullsized":"2023-10-18T09:03:23.325742+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313411+0000","last_change":"2023-10-18T09:03:23.313411+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313411+0000","last_undegraded":"2023-10-18T09:03:23.313411+0000","last_fullsized":"2023-10-18T09:03:23.313411+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.319674+0000","last_change":"2023-10-18T09:03:23.319674+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.319674+0000","last_undegraded":"2023-10-18T09:03:23.319674+0000","last_fullsized":"2023-10-18T09:03:23.319674+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.327137+0000","last_change":"2023-10-18T09:03:23.327137+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.327137+0000","last_undegraded":"2023-10-18T09:03:23.327137+0000","last_fullsized":"2023-10-18T09:03:23.327137+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.373894+0000","last_change":"2023-10-18T09:03:24.373509+0000","last_active":"2023-10-18T09:03:24.373894+0000","last_peered":"2023-10-18T09:03:24.373894+0000","last_clean":"2023-10-18T09:03:24.373894+0000","last_became_active":"2023-10-18T09:03:24.372325+0000","last_became_peered":"2023-10-18T09:03:24.372325+0000","last_unstale":"2023-10-18T09:03:24.373894+0000","last_undegraded":"2023-10-18T09:03:24.373894+0000","last_fullsized":"2023-10-18T09:03:24.373894+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:00.288118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.325288+0000","last_change":"2023-10-18T09:03:23.325288+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.325288+0000","last_undegraded":"2023-10-18T09:03:23.325288+0000","last_fullsized":"2023-10-18T09:03:23.325288+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.317783+0000","last_change":"2023-10-18T09:03:23.317783+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.317783+0000","last_undegraded":"2023-10-18T09:03:23.317783+0000","last_fullsized":"2023-10-18T09:03:23.317783+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.316307+0000","last_change":"2023-10-18T09:03:23.316307+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.316307+0000","last_undegraded":"2023-10-18T09:03:23.316307+0000","last_fullsized":"2023-10-18T09:03:23.316307+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321900+0000","last_change":"2023-10-18T09:03:23.321900+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321900+0000","last_undegraded":"2023-10-18T09:03:23.321900+0000","last_fullsized":"2023-10-18T09:03:23.321900+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.378701+0000","last_change":"2023-10-18T09:03:24.378634+0000","last_active":"2023-10-18T09:03:24.378701+0000","last_peered":"2023-10-18T09:03:24.378701+0000","last_clean":"2023-10-18T09:03:24.378701+0000","last_became_active":"2023-10-18T09:03:24.378364+0000","last_became_peered":"2023-10-18T09:03:24.378364+0000","last_unstale":"2023-10-18T09:03:24.378701+0000","last_undegraded":"2023-10-18T09:03:24.378701+0000","last_fullsized":"2023-10-18T09:03:24.378701+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:26:40.521285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.372987+0000","last_change":"2023-10-18T09:03:24.371034+0000","last_active":"2023-10-18T09:03:24.372987+0000","last_peered":"2023-10-18T09:03:24.372987+0000","last_clean":"2023-10-18T09:03:24.372987+0000","last_became_active":"2023-10-18T09:03:24.370294+0000","last_became_peered":"2023-10-18T09:03:24.370294+0000","last_unstale":"2023-10-18T09:03:24.372987+0000","last_undegraded":"2023-10-18T09:03:24.372987+0000","last_fullsized":"2023-10-18T09:03:24.372987+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:13:18.579341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321363+0000","last_change":"2023-10-18T09:03:23.321363+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321363+0000","last_undegraded":"2023-10-18T09:03:23.321363+0000","last_fullsized":"2023-10-18T09:03:23.321363+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312843+0000","last_change":"2023-10-18T09:03:23.312843+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312843+0000","last_undegraded":"2023-10-18T09:03:23.312843+0000","last_fullsized":"2023-10-18T09:03:23.312843+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.322169+0000","last_change":"2023-10-18T09:03:23.322169+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.322169+0000","last_undegraded":"2023-10-18T09:03:23.322169+0000","last_fullsized":"2023-10-18T09:03:23.322169+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321038+0000","last_change":"2023-10-18T09:03:23.321038+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321038+0000","last_undegraded":"2023-10-18T09:03:23.321038+0000","last_fullsized":"2023-10-18T09:03:23.321038+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.373932+0000","last_change":"2023-10-18T09:03:24.373744+0000","last_active":"2023-10-18T09:03:24.373932+0000","last_peered":"2023-10-18T09:03:24.373932+0000","last_clean":"2023-10-18T09:03:24.373932+0000","last_became_active":"2023-10-18T09:03:24.373488+0000","last_became_peered":"2023-10-18T09:03:24.373488+0000","last_unstale":"2023-10-18T09:03:24.373932+0000","last_undegraded":"2023-10-18T09:03:24.373932+0000","last_fullsized":"2023-10-18T09:03:24.373932+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:21.541741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313740+0000","last_change":"2023-10-18T09:03:23.313740+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313740+0000","last_undegraded":"2023-10-18T09:03:23.313740+0000","last_fullsized":"2023-10-18T09:03:23.313740+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.315443+0000","last_change":"2023-10-18T09:03:23.315443+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.315443+0000","last_undegraded":"2023-10-18T09:03:23.315443+0000","last_fullsized":"2023-10-18T09:03:23.315443+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.378342+0000","last_change":"2023-10-18T09:03:24.376004+0000","last_active":"2023-10-18T09:03:24.378342+0000","last_peered":"2023-10-18T09:03:24.378342+0000","last_clean":"2023-10-18T09:03:24.378342+0000","last_became_active":"2023-10-18T09:03:24.375694+0000","last_became_peered":"2023-10-18T09:03:24.375694+0000","last_unstale":"2023-10-18T09:03:24.378342+0000","last_undegraded":"2023-10-18T09:03:24.378342+0000","last_fullsized":"2023-10-18T09:03:24.378342+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:45:05.600764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312725+0000","last_change":"2023-10-18T09:03:23.312725+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312725+0000","last_undegraded":"2023-10-18T09:03:23.312725+0000","last_fullsized":"2023-10-18T09:03:23.312725+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.365671+0000","last_change":"2023-10-18T09:03:24.364680+0000","last_active":"2023-10-18T09:03:24.365671+0000","last_peered":"2023-10-18T09:03:24.365671+0000","last_clean":"2023-10-18T09:03:24.365671+0000","last_became_active":"2023-10-18T09:03:24.364143+0000","last_became_peered":"2023-10-18T09:03:24.364143+0000","last_unstale":"2023-10-18T09:03:24.365671+0000","last_undegraded":"2023-10-18T09:03:24.365671+0000","last_fullsized":"2023-10-18T09:03:24.365671+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:42:51.261581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312889+0000","last_change":"2023-10-18T09:03:23.312889+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312889+0000","last_undegraded":"2023-10-18T09:03:23.312889+0000","last_fullsized":"2023-10-18T09:03:23.312889+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312482+0000","last_change":"2023-10-18T09:03:23.312482+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312482+0000","last_undegraded":"2023-10-18T09:03:23.312482+0000","last_fullsized":"2023-10-18T09:03:23.312482+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.318515+0000","last_change":"2023-10-18T09:03:23.318515+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.318515+0000","last_undegraded":"2023-10-18T09:03:23.318515+0000","last_fullsized":"2023-10-18T09:03:23.318515+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.374017+0000","last_change":"2023-10-18T09:03:24.371719+0000","last_active":"2023-10-18T09:03:24.374017+0000","last_peered":"2023-10-18T09:03:24.374017+0000","last_clean":"2023-10-18T09:03:24.374017+0000","last_became_active":"2023-10-18T09:03:24.371095+0000","last_became_peered":"2023-10-18T09:03:24.371095+0000","last_unstale":"2023-10-18T09:03:24.374017+0000","last_undegraded":"2023-10-18T09:03:24.374017+0000","last_fullsized":"2023-10-18T09:03:24.374017+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:50.009429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.317757+0000","last_change":"2023-10-18T09:03:23.317757+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.317757+0000","last_undegraded":"2023-10-18T09:03:23.317757+0000","last_fullsized":"2023-10-18T09:03:23.317757+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.326266+0000","last_change":"2023-10-18T09:03:23.326266+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.326266+0000","last_undegraded":"2023-10-18T09:03:23.326266+0000","last_fullsized":"2023-10-18T09:03:23.326266+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312103+0000","last_change":"2023-10-18T09:03:23.312103+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312103+0000","last_undegraded":"2023-10-18T09:03:23.312103+0000","last_fullsized":"2023-10-18T09:03:23.312103+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.318051+0000","last_change":"2023-10-18T09:03:23.318051+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.318051+0000","last_undegraded":"2023-10-18T09:03:23.318051+0000","last_fullsized":"2023-10-18T09:03:23.318051+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.320206+0000","last_change":"2023-10-18T09:03:23.320206+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.320206+0000","last_undegraded":"2023-10-18T09:03:23.320206+0000","last_fullsized":"2023-10-18T09:03:23.320206+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.329894+0000","last_change":"2023-10-18T09:03:23.329530+0000","last_active":"2023-10-18T09:03:23.329894+0000","last_peered":"2023-10-18T09:03:23.329894+0000","last_clean":"2023-10-18T09:03:23.329894+0000","last_became_active":"2023-10-18T09:03:23.329181+0000","last_became_peered":"2023-10-18T09:03:23.329181+0000","last_unstale":"2023-10-18T09:03:23.329894+0000","last_undegraded":"2023-10-18T09:03:23.329894+0000","last_fullsized":"2023-10-18T09:03:23.329894+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:29.736042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.318339+0000","last_change":"2023-10-18T09:03:23.318339+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.318339+0000","last_undegraded":"2023-10-18T09:03:23.318339+0000","last_fullsized":"2023-10-18T09:03:23.318339+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"143'1","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.391133+0000","last_change":"2023-10-18T09:03:23.331319+0000","last_active":"2023-10-18T09:03:24.391133+0000","last_peered":"2023-10-18T09:03:24.391133+0000","last_clean":"2023-10-18T09:03:24.391133+0000","last_became_active":"2023-10-18T09:03:23.331015+0000","last_became_peered":"2023-10-18T09:03:23.331015+0000","last_unstale":"2023-10-18T09:03:24.391133+0000","last_undegraded":"2023-10-18T09:03:24.391133+0000","last_fullsized":"2023-10-18T09:03:24.391133+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:40.692371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.319346+0000","last_change":"2023-10-18T09:03:23.319346+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.319346+0000","last_undegraded":"2023-10-18T09:03:23.319346+0000","last_fullsized":"2023-10-18T09:03:23.319346+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331754+0000","last_change":"2023-10-18T09:03:23.330634+0000","last_active":"2023-10-18T09:03:23.331754+0000","last_peered":"2023-10-18T09:03:23.331754+0000","last_clean":"2023-10-18T09:03:23.331754+0000","last_became_active":"2023-10-18T09:03:23.329893+0000","last_became_peered":"2023-10-18T09:03:23.329893+0000","last_unstale":"2023-10-18T09:03:23.331754+0000","last_undegraded":"2023-10-18T09:03:23.331754+0000","last_fullsized":"2023-10-18T09:03:23.331754+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:20:04.939191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.373913+0000","last_change":"2023-10-18T09:03:24.373601+0000","last_active":"2023-10-18T09:03:24.373913+0000","last_peered":"2023-10-18T09:03:24.373913+0000","last_clean":"2023-10-18T09:03:24.373913+0000","last_became_active":"2023-10-18T09:03:24.373369+0000","last_became_peered":"2023-10-18T09:03:24.373369+0000","last_unstale":"2023-10-18T09:03:24.373913+0000","last_undegraded":"2023-10-18T09:03:24.373913+0000","last_fullsized":"2023-10-18T09:03:24.373913+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:48.301601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.328765+0000","last_change":"2023-10-18T09:03:23.328512+0000","last_active":"2023-10-18T09:03:23.328765+0000","last_peered":"2023-10-18T09:03:23.328765+0000","last_clean":"2023-10-18T09:03:23.328765+0000","last_became_active":"2023-10-18T09:03:23.328090+0000","last_became_peered":"2023-10-18T09:03:23.328090+0000","last_unstale":"2023-10-18T09:03:23.328765+0000","last_undegraded":"2023-10-18T09:03:23.328765+0000","last_fullsized":"2023-10-18T09:03:23.328765+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:23.820590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.320552+0000","last_change":"2023-10-18T09:03:23.320552+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.320552+0000","last_undegraded":"2023-10-18T09:03:23.320552+0000","last_fullsized":"2023-10-18T09:03:23.320552+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.329917+0000","last_change":"2023-10-18T09:03:23.329851+0000","last_active":"2023-10-18T09:03:23.329917+0000","last_peered":"2023-10-18T09:03:23.329917+0000","last_clean":"2023-10-18T09:03:23.329917+0000","last_became_active":"2023-10-18T09:03:23.329208+0000","last_became_peered":"2023-10-18T09:03:23.329208+0000","last_unstale":"2023-10-18T09:03:23.329917+0000","last_undegraded":"2023-10-18T09:03:23.329917+0000","last_fullsized":"2023-10-18T09:03:23.329917+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:08.163420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321572+0000","last_change":"2023-10-18T09:03:23.321572+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321572+0000","last_undegraded":"2023-10-18T09:03:23.321572+0000","last_fullsized":"2023-10-18T09:03:23.321572+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"143'1","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.392871+0000","last_change":"2023-10-18T09:03:23.329790+0000","last_active":"2023-10-18T09:03:24.392871+0000","last_peered":"2023-10-18T09:03:24.392871+0000","last_clean":"2023-10-18T09:03:24.392871+0000","last_became_active":"2023-10-18T09:03:23.325710+0000","last_became_peered":"2023-10-18T09:03:23.325710+0000","last_unstale":"2023-10-18T09:03:24.392871+0000","last_undegraded":"2023-10-18T09:03:24.392871+0000","last_fullsized":"2023-10-18T09:03:24.392871+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:06:18.861877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.316958+0000","last_change":"2023-10-18T09:03:23.316958+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.316958+0000","last_undegraded":"2023-10-18T09:03:23.316958+0000","last_fullsized":"2023-10-18T09:03:23.316958+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.318565+0000","last_change":"2023-10-18T09:03:23.318325+0000","last_active":"2023-10-18T09:03:23.318565+0000","last_peered":"2023-10-18T09:03:23.318565+0000","last_clean":"2023-10-18T09:03:23.318565+0000","last_became_active":"2023-10-18T09:03:23.316556+0000","last_became_peered":"2023-10-18T09:03:23.316556+0000","last_unstale":"2023-10-18T09:03:23.318565+0000","last_undegraded":"2023-10-18T09:03:23.318565+0000","last_fullsized":"2023-10-18T09:03:23.318565+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:25:52.365591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.322477+0000","last_change":"2023-10-18T09:03:23.322477+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.322477+0000","last_undegraded":"2023-10-18T09:03:23.322477+0000","last_fullsized":"2023-10-18T09:03:23.322477+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.334091+0000","last_change":"2023-10-18T09:03:23.334041+0000","last_active":"2023-10-18T09:03:23.334091+0000","last_peered":"2023-10-18T09:03:23.334091+0000","last_clean":"2023-10-18T09:03:23.334091+0000","last_became_active":"2023-10-18T09:03:23.333830+0000","last_became_peered":"2023-10-18T09:03:23.333830+0000","last_unstale":"2023-10-18T09:03:23.334091+0000","last_undegraded":"2023-10-18T09:03:23.334091+0000","last_fullsized":"2023-10-18T09:03:23.334091+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:23.484620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.324949+0000","last_change":"2023-10-18T09:03:23.324949+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.324949+0000","last_undegraded":"2023-10-18T09:03:23.324949+0000","last_fullsized":"2023-10-18T09:03:23.324949+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.328748+0000","last_change":"2023-10-18T09:03:23.328337+0000","last_active":"2023-10-18T09:03:23.328748+0000","last_peered":"2023-10-18T09:03:23.328748+0000","last_clean":"2023-10-18T09:03:23.328748+0000","last_became_active":"2023-10-18T09:03:23.325594+0000","last_became_peered":"2023-10-18T09:03:23.325594+0000","last_unstale":"2023-10-18T09:03:23.328748+0000","last_undegraded":"2023-10-18T09:03:23.328748+0000","last_fullsized":"2023-10-18T09:03:23.328748+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:42.662714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321318+0000","last_change":"2023-10-18T09:03:23.321318+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321318+0000","last_undegraded":"2023-10-18T09:03:23.321318+0000","last_fullsized":"2023-10-18T09:03:23.321318+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.346035+0000","last_change":"2023-10-18T09:03:23.325401+0000","last_active":"2023-10-18T09:03:24.346035+0000","last_peered":"2023-10-18T09:03:24.346035+0000","last_clean":"2023-10-18T09:03:24.346035+0000","last_became_active":"2023-10-18T09:03:23.324277+0000","last_became_peered":"2023-10-18T09:03:23.324277+0000","last_unstale":"2023-10-18T09:03:24.346035+0000","last_undegraded":"2023-10-18T09:03:24.346035+0000","last_fullsized":"2023-10-18T09:03:24.346035+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:06:07.682599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.356612+0000","last_change":"2023-10-18T09:03:24.350913+0000","last_active":"2023-10-18T09:03:24.356612+0000","last_peered":"2023-10-18T09:03:24.356612+0000","last_clean":"2023-10-18T09:03:24.356612+0000","last_became_active":"2023-10-18T09:03:24.350214+0000","last_became_peered":"2023-10-18T09:03:24.350214+0000","last_unstale":"2023-10-18T09:03:24.356612+0000","last_undegraded":"2023-10-18T09:03:24.356612+0000","last_fullsized":"2023-10-18T09:03:24.356612+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:24:33.551156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331509+0000","last_change":"2023-10-18T09:03:23.331283+0000","last_active":"2023-10-18T09:03:23.331509+0000","last_peered":"2023-10-18T09:03:23.331509+0000","last_clean":"2023-10-18T09:03:23.331509+0000","last_became_active":"2023-10-18T09:03:23.329500+0000","last_became_peered":"2023-10-18T09:03:23.329500+0000","last_unstale":"2023-10-18T09:03:23.331509+0000","last_undegraded":"2023-10-18T09:03:23.331509+0000","last_fullsized":"2023-10-18T09:03:23.331509+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:13.492685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.326069+0000","last_change":"2023-10-18T09:03:23.326069+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.326069+0000","last_undegraded":"2023-10-18T09:03:23.326069+0000","last_fullsized":"2023-10-18T09:03:23.326069+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.334640+0000","last_change":"2023-10-18T09:03:23.334514+0000","last_active":"2023-10-18T09:03:23.334640+0000","last_peered":"2023-10-18T09:03:23.334640+0000","last_clean":"2023-10-18T09:03:23.334640+0000","last_became_active":"2023-10-18T09:03:23.334268+0000","last_became_peered":"2023-10-18T09:03:23.334268+0000","last_unstale":"2023-10-18T09:03:23.334640+0000","last_undegraded":"2023-10-18T09:03:23.334640+0000","last_fullsized":"2023-10-18T09:03:23.334640+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:16.978313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.367932+0000","last_change":"2023-10-18T09:03:24.367289+0000","last_active":"2023-10-18T09:03:24.367932+0000","last_peered":"2023-10-18T09:03:24.367932+0000","last_clean":"2023-10-18T09:03:24.367932+0000","last_became_active":"2023-10-18T09:03:24.366724+0000","last_became_peered":"2023-10-18T09:03:24.366724+0000","last_unstale":"2023-10-18T09:03:24.367932+0000","last_undegraded":"2023-10-18T09:03:24.367932+0000","last_fullsized":"2023-10-18T09:03:24.367932+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:29:55.406170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.325184+0000","last_change":"2023-10-18T09:03:23.323723+0000","last_active":"2023-10-18T09:03:23.325184+0000","last_peered":"2023-10-18T09:03:23.325184+0000","last_clean":"2023-10-18T09:03:23.325184+0000","last_became_active":"2023-10-18T09:03:23.322838+0000","last_became_peered":"2023-10-18T09:03:23.322838+0000","last_unstale":"2023-10-18T09:03:23.325184+0000","last_undegraded":"2023-10-18T09:03:23.325184+0000","last_fullsized":"2023-10-18T09:03:23.325184+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:12.382096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.375498+0000","last_change":"2023-10-18T09:03:24.374652+0000","last_active":"2023-10-18T09:03:24.375498+0000","last_peered":"2023-10-18T09:03:24.375498+0000","last_clean":"2023-10-18T09:03:24.375498+0000","last_became_active":"2023-10-18T09:03:24.366671+0000","last_became_peered":"2023-10-18T09:03:24.366671+0000","last_unstale":"2023-10-18T09:03:24.375498+0000","last_undegraded":"2023-10-18T09:03:24.375498+0000","last_fullsized":"2023-10-18T09:03:24.375498+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:07:26.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.334932+0000","last_change":"2023-10-18T09:03:23.334809+0000","last_active":"2023-10-18T09:03:23.334932+0000","last_peered":"2023-10-18T09:03:23.334932+0000","last_clean":"2023-10-18T09:03:23.334932+0000","last_became_active":"2023-10-18T09:03:23.334509+0000","last_became_peered":"2023-10-18T09:03:23.334509+0000","last_unstale":"2023-10-18T09:03:23.334932+0000","last_undegraded":"2023-10-18T09:03:23.334932+0000","last_fullsized":"2023-10-18T09:03:23.334932+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:13.492685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.356667+0000","last_change":"2023-10-18T09:03:24.353278+0000","last_active":"2023-10-18T09:03:24.356667+0000","last_peered":"2023-10-18T09:03:24.356667+0000","last_clean":"2023-10-18T09:03:24.356667+0000","last_became_active":"2023-10-18T09:03:24.352158+0000","last_became_peered":"2023-10-18T09:03:24.352158+0000","last_unstale":"2023-10-18T09:03:24.356667+0000","last_undegraded":"2023-10-18T09:03:24.356667+0000","last_fullsized":"2023-10-18T09:03:24.356667+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:05.753025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.332360+0000","last_change":"2023-10-18T09:03:23.332284+0000","last_active":"2023-10-18T09:03:23.332360+0000","last_peered":"2023-10-18T09:03:23.332360+0000","last_clean":"2023-10-18T09:03:23.332360+0000","last_became_active":"2023-10-18T09:03:23.332040+0000","last_became_peered":"2023-10-18T09:03:23.332040+0000","last_unstale":"2023-10-18T09:03:23.332360+0000","last_undegraded":"2023-10-18T09:03:23.332360+0000","last_fullsized":"2023-10-18T09:03:23.332360+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:41:21.941443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313059+0000","last_change":"2023-10-18T09:03:23.313059+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313059+0000","last_undegraded":"2023-10-18T09:03:23.313059+0000","last_fullsized":"2023-10-18T09:03:23.313059+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.333121+0000","last_change":"2023-10-18T09:03:23.332929+0000","last_active":"2023-10-18T09:03:23.333121+0000","last_peered":"2023-10-18T09:03:23.333121+0000","last_clean":"2023-10-18T09:03:23.333121+0000","last_became_active":"2023-10-18T09:03:23.332508+0000","last_became_peered":"2023-10-18T09:03:23.332508+0000","last_unstale":"2023-10-18T09:03:23.333121+0000","last_undegraded":"2023-10-18T09:03:23.333121+0000","last_fullsized":"2023-10-18T09:03:23.333121+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:30.964339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.319843+0000","last_change":"2023-10-18T09:03:23.319843+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.319843+0000","last_undegraded":"2023-10-18T09:03:23.319843+0000","last_fullsized":"2023-10-18T09:03:23.319843+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331736+0000","last_change":"2023-10-18T09:03:23.330153+0000","last_active":"2023-10-18T09:03:23.331736+0000","last_peered":"2023-10-18T09:03:23.331736+0000","last_clean":"2023-10-18T09:03:23.331736+0000","last_became_active":"2023-10-18T09:03:23.325376+0000","last_became_peered":"2023-10-18T09:03:23.325376+0000","last_unstale":"2023-10-18T09:03:23.331736+0000","last_undegraded":"2023-10-18T09:03:23.331736+0000","last_fullsized":"2023-10-18T09:03:23.331736+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:43.431276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.374379+0000","last_change":"2023-10-18T09:03:24.374330+0000","last_active":"2023-10-18T09:03:24.374379+0000","last_peered":"2023-10-18T09:03:24.374379+0000","last_clean":"2023-10-18T09:03:24.374379+0000","last_became_active":"2023-10-18T09:03:24.374144+0000","last_became_peered":"2023-10-18T09:03:24.374144+0000","last_unstale":"2023-10-18T09:03:24.374379+0000","last_undegraded":"2023-10-18T09:03:24.374379+0000","last_fullsized":"2023-10-18T09:03:24.374379+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:59.934981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.325277+0000","last_change":"2023-10-18T09:03:23.324222+0000","last_active":"2023-10-18T09:03:23.325277+0000","last_peered":"2023-10-18T09:03:23.325277+0000","last_clean":"2023-10-18T09:03:23.325277+0000","last_became_active":"2023-10-18T09:03:23.323810+0000","last_became_peered":"2023-10-18T09:03:23.323810+0000","last_unstale":"2023-10-18T09:03:23.325277+0000","last_undegraded":"2023-10-18T09:03:23.325277+0000","last_fullsized":"2023-10-18T09:03:23.325277+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:20.536935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.371800+0000","last_change":"2023-10-18T09:03:24.370815+0000","last_active":"2023-10-18T09:03:24.371800+0000","last_peered":"2023-10-18T09:03:24.371800+0000","last_clean":"2023-10-18T09:03:24.371800+0000","last_became_active":"2023-10-18T09:03:24.370553+0000","last_became_peered":"2023-10-18T09:03:24.370553+0000","last_unstale":"2023-10-18T09:03:24.371800+0000","last_undegraded":"2023-10-18T09:03:24.371800+0000","last_fullsized":"2023-10-18T09:03:24.371800+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:17.983120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.332741+0000","last_change":"2023-10-18T09:03:23.332652+0000","last_active":"2023-10-18T09:03:23.332741+0000","last_peered":"2023-10-18T09:03:23.332741+0000","last_clean":"2023-10-18T09:03:23.332741+0000","last_became_active":"2023-10-18T09:03:23.331276+0000","last_became_peered":"2023-10-18T09:03:23.331276+0000","last_unstale":"2023-10-18T09:03:23.332741+0000","last_undegraded":"2023-10-18T09:03:23.332741+0000","last_fullsized":"2023-10-18T09:03:23.332741+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:12.382096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.367994+0000","last_change":"2023-10-18T09:03:24.367856+0000","last_active":"2023-10-18T09:03:24.367994+0000","last_peered":"2023-10-18T09:03:24.367994+0000","last_clean":"2023-10-18T09:03:24.367994+0000","last_became_active":"2023-10-18T09:03:24.367568+0000","last_became_peered":"2023-10-18T09:03:24.367568+0000","last_unstale":"2023-10-18T09:03:24.367994+0000","last_undegraded":"2023-10-18T09:03:24.367994+0000","last_fullsized":"2023-10-18T09:03:24.367994+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:14.497491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.325153+0000","last_change":"2023-10-18T09:03:23.322408+0000","last_active":"2023-10-18T09:03:23.325153+0000","last_peered":"2023-10-18T09:03:23.325153+0000","last_clean":"2023-10-18T09:03:23.325153+0000","last_became_active":"2023-10-18T09:03:23.321746+0000","last_became_peered":"2023-10-18T09:03:23.321746+0000","last_unstale":"2023-10-18T09:03:23.325153+0000","last_undegraded":"2023-10-18T09:03:23.325153+0000","last_fullsized":"2023-10-18T09:03:23.325153+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:08.225684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313248+0000","last_change":"2023-10-18T09:03:23.313248+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313248+0000","last_undegraded":"2023-10-18T09:03:23.313248+0000","last_fullsized":"2023-10-18T09:03:23.313248+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.325186+0000","last_change":"2023-10-18T09:03:23.324369+0000","last_active":"2023-10-18T09:03:23.325186+0000","last_peered":"2023-10-18T09:03:23.325186+0000","last_clean":"2023-10-18T09:03:23.325186+0000","last_became_active":"2023-10-18T09:03:23.323637+0000","last_became_peered":"2023-10-18T09:03:23.323637+0000","last_unstale":"2023-10-18T09:03:23.325186+0000","last_undegraded":"2023-10-18T09:03:23.325186+0000","last_fullsized":"2023-10-18T09:03:23.325186+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:18:41.207403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.379077+0000","last_change":"2023-10-18T09:03:24.378954+0000","last_active":"2023-10-18T09:03:24.379077+0000","last_peered":"2023-10-18T09:03:24.379077+0000","last_clean":"2023-10-18T09:03:24.379077+0000","last_became_active":"2023-10-18T09:03:24.378534+0000","last_became_peered":"2023-10-18T09:03:24.378534+0000","last_unstale":"2023-10-18T09:03:24.379077+0000","last_undegraded":"2023-10-18T09:03:24.379077+0000","last_fullsized":"2023-10-18T09:03:24.379077+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:14:13.807602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.332533+0000","last_change":"2023-10-18T09:03:23.332462+0000","last_active":"2023-10-18T09:03:23.332533+0000","last_peered":"2023-10-18T09:03:23.332533+0000","last_clean":"2023-10-18T09:03:23.332533+0000","last_became_active":"2023-10-18T09:03:23.332239+0000","last_became_peered":"2023-10-18T09:03:23.332239+0000","last_unstale":"2023-10-18T09:03:23.332533+0000","last_undegraded":"2023-10-18T09:03:23.332533+0000","last_fullsized":"2023-10-18T09:03:23.332533+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:25.333743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.361716+0000","last_change":"2023-10-18T09:03:24.361276+0000","last_active":"2023-10-18T09:03:24.361716+0000","last_peered":"2023-10-18T09:03:24.361716+0000","last_clean":"2023-10-18T09:03:24.361716+0000","last_became_active":"2023-10-18T09:03:24.360521+0000","last_became_peered":"2023-10-18T09:03:24.360521+0000","last_unstale":"2023-10-18T09:03:24.361716+0000","last_undegraded":"2023-10-18T09:03:24.361716+0000","last_fullsized":"2023-10-18T09:03:24.361716+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:11.398182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331491+0000","last_change":"2023-10-18T09:03:23.330931+0000","last_active":"2023-10-18T09:03:23.331491+0000","last_peered":"2023-10-18T09:03:23.331491+0000","last_clean":"2023-10-18T09:03:23.331491+0000","last_became_active":"2023-10-18T09:03:23.330686+0000","last_became_peered":"2023-10-18T09:03:23.330686+0000","last_unstale":"2023-10-18T09:03:23.331491+0000","last_undegraded":"2023-10-18T09:03:23.331491+0000","last_fullsized":"2023-10-18T09:03:23.331491+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:47.296794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.317564+0000","last_change":"2023-10-18T09:03:23.317564+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.317564+0000","last_undegraded":"2023-10-18T09:03:23.317564+0000","last_fullsized":"2023-10-18T09:03:23.317564+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"143'2","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.393968+0000","last_change":"2023-10-18T09:03:23.330533+0000","last_active":"2023-10-18T09:03:24.393968+0000","last_peered":"2023-10-18T09:03:24.393968+0000","last_clean":"2023-10-18T09:03:24.393968+0000","last_became_active":"2023-10-18T09:03:23.330093+0000","last_became_peered":"2023-10-18T09:03:23.330093+0000","last_unstale":"2023-10-18T09:03:24.393968+0000","last_undegraded":"2023-10-18T09:03:24.393968+0000","last_fullsized":"2023-10-18T09:03:24.393968+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:54.344592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.315057+0000","last_change":"2023-10-18T09:03:23.315057+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.315057+0000","last_undegraded":"2023-10-18T09:03:23.315057+0000","last_fullsized":"2023-10-18T09:03:23.315057+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.330035+0000","last_change":"2023-10-18T09:03:23.329947+0000","last_active":"2023-10-18T09:03:23.330035+0000","last_peered":"2023-10-18T09:03:23.330035+0000","last_clean":"2023-10-18T09:03:23.330035+0000","last_became_active":"2023-10-18T09:03:23.329667+0000","last_became_peered":"2023-10-18T09:03:23.329667+0000","last_unstale":"2023-10-18T09:03:23.330035+0000","last_undegraded":"2023-10-18T09:03:23.330035+0000","last_fullsized":"2023-10-18T09:03:23.330035+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:23.484620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313161+0000","last_change":"2023-10-18T09:03:23.313161+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313161+0000","last_undegraded":"2023-10-18T09:03:23.313161+0000","last_fullsized":"2023-10-18T09:03:23.313161+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.333102+0000","last_change":"2023-10-18T09:03:23.332779+0000","last_active":"2023-10-18T09:03:23.333102+0000","last_peered":"2023-10-18T09:03:23.333102+0000","last_clean":"2023-10-18T09:03:23.333102+0000","last_became_active":"2023-10-18T09:03:23.332152+0000","last_became_peered":"2023-10-18T09:03:23.332152+0000","last_unstale":"2023-10-18T09:03:23.333102+0000","last_undegraded":"2023-10-18T09:03:23.333102+0000","last_fullsized":"2023-10-18T09:03:23.333102+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:40.692371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313904+0000","last_change":"2023-10-18T09:03:23.313904+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313904+0000","last_undegraded":"2023-10-18T09:03:23.313904+0000","last_fullsized":"2023-10-18T09:03:23.313904+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.346056+0000","last_change":"2023-10-18T09:03:23.325136+0000","last_active":"2023-10-18T09:03:24.346056+0000","last_peered":"2023-10-18T09:03:24.346056+0000","last_clean":"2023-10-18T09:03:24.346056+0000","last_became_active":"2023-10-18T09:03:23.323782+0000","last_became_peered":"2023-10-18T09:03:23.323782+0000","last_unstale":"2023-10-18T09:03:24.346056+0000","last_undegraded":"2023-10-18T09:03:24.346056+0000","last_fullsized":"2023-10-18T09:03:24.346056+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:42.460476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.318651+0000","last_change":"2023-10-18T09:03:23.318651+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.318651+0000","last_undegraded":"2023-10-18T09:03:23.318651+0000","last_fullsized":"2023-10-18T09:03:23.318651+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.329867+0000","last_change":"2023-10-18T09:03:23.329448+0000","last_active":"2023-10-18T09:03:23.329867+0000","last_peered":"2023-10-18T09:03:23.329867+0000","last_clean":"2023-10-18T09:03:23.329867+0000","last_became_active":"2023-10-18T09:03:23.329094+0000","last_became_peered":"2023-10-18T09:03:23.329094+0000","last_unstale":"2023-10-18T09:03:23.329867+0000","last_undegraded":"2023-10-18T09:03:23.329867+0000","last_fullsized":"2023-10-18T09:03:23.329867+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:42:23.072395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.314648+0000","last_change":"2023-10-18T09:03:23.314648+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.314648+0000","last_undegraded":"2023-10-18T09:03:23.314648+0000","last_fullsized":"2023-10-18T09:03:23.314648+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.324173+0000","last_change":"2023-10-18T09:03:23.323348+0000","last_active":"2023-10-18T09:03:23.324173+0000","last_peered":"2023-10-18T09:03:23.324173+0000","last_clean":"2023-10-18T09:03:23.324173+0000","last_became_active":"2023-10-18T09:03:23.323009+0000","last_became_peered":"2023-10-18T09:03:23.323009+0000","last_unstale":"2023-10-18T09:03:23.324173+0000","last_undegraded":"2023-10-18T09:03:23.324173+0000","last_fullsized":"2023-10-18T09:03:23.324173+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:07:37.803900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.373379+0000","last_change":"2023-10-18T09:03:24.373169+0000","last_active":"2023-10-18T09:03:24.373379+0000","last_peered":"2023-10-18T09:03:24.373379+0000","last_clean":"2023-10-18T09:03:24.373379+0000","last_became_active":"2023-10-18T09:03:24.370785+0000","last_became_peered":"2023-10-18T09:03:24.370785+0000","last_unstale":"2023-10-18T09:03:24.373379+0000","last_undegraded":"2023-10-18T09:03:24.373379+0000","last_fullsized":"2023-10-18T09:03:24.373379+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:24.489427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331718+0000","last_change":"2023-10-18T09:03:23.329465+0000","last_active":"2023-10-18T09:03:23.331718+0000","last_peered":"2023-10-18T09:03:23.331718+0000","last_clean":"2023-10-18T09:03:23.331718+0000","last_became_active":"2023-10-18T09:03:23.329200+0000","last_became_peered":"2023-10-18T09:03:23.329200+0000","last_unstale":"2023-10-18T09:03:23.331718+0000","last_undegraded":"2023-10-18T09:03:23.331718+0000","last_fullsized":"2023-10-18T09:03:23.331718+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:11.421962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.319950+0000","last_change":"2023-10-18T09:03:23.319950+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.319950+0000","last_undegraded":"2023-10-18T09:03:23.319950+0000","last_fullsized":"2023-10-18T09:03:23.319950+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331450+0000","last_change":"2023-10-18T09:03:23.329760+0000","last_active":"2023-10-18T09:03:23.331450+0000","last_peered":"2023-10-18T09:03:23.331450+0000","last_clean":"2023-10-18T09:03:23.331450+0000","last_became_active":"2023-10-18T09:03:23.329479+0000","last_became_peered":"2023-10-18T09:03:23.329479+0000","last_unstale":"2023-10-18T09:03:23.331450+0000","last_undegraded":"2023-10-18T09:03:23.331450+0000","last_fullsized":"2023-10-18T09:03:23.331450+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:20.536935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.378559+0000","last_change":"2023-10-18T09:03:24.378443+0000","last_active":"2023-10-18T09:03:24.378559+0000","last_peered":"2023-10-18T09:03:24.378559+0000","last_clean":"2023-10-18T09:03:24.378559+0000","last_became_active":"2023-10-18T09:03:24.378196+0000","last_became_peered":"2023-10-18T09:03:24.378196+0000","last_unstale":"2023-10-18T09:03:24.378559+0000","last_undegraded":"2023-10-18T09:03:24.378559+0000","last_fullsized":"2023-10-18T09:03:24.378559+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:13:51.906537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.333073+0000","last_change":"2023-10-18T09:03:23.332893+0000","last_active":"2023-10-18T09:03:23.333073+0000","last_peered":"2023-10-18T09:03:23.333073+0000","last_clean":"2023-10-18T09:03:23.333073+0000","last_became_active":"2023-10-18T09:03:23.332302+0000","last_became_peered":"2023-10-18T09:03:23.332302+0000","last_unstale":"2023-10-18T09:03:23.333073+0000","last_undegraded":"2023-10-18T09:03:23.333073+0000","last_fullsized":"2023-10-18T09:03:23.333073+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:34:22.081612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":301,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.308489+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:03:23.308489+0000","last_peered":"2023-10-18T09:03:23.308489+0000","last_clean":"2023-10-18T09:03:23.308489+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:03:23.308489+0000","last_undegraded":"2023-10-18T09:03:23.308489+0000","last_fullsized":"2023-10-18T09:03:23.308489+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1361,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193951,"statfs":{"total":200000000,"available":198606297,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.135,"15min":1.038},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.579,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.109,"15min":1.046},"min":{"1min":0.64500000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.421,"5min":1.97,"15min":1.97},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1819999999999999,"15min":1.155},"min":{"1min":0.83299999999999996,"5min":0.70099999999999996,"15min":0.61799999999999999},"max":{"1min":1.595,"5min":1.829,"15min":1.829},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.1240000000000001,"15min":1.1259999999999999},"min":{"1min":0.72799999999999998,"5min":0.61099999999999999,"15min":0.58799999999999997},"max":{"1min":1.4239999999999999,"5min":1.657,"15min":1.7989999999999999},"last":0.73399999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3220000000000001,"15min":1.264},"min":{"1min":0.94699999999999995,"5min":0.78600000000000003,"15min":0.65400000000000003},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.9730000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.206,"5min":1.27,"15min":1.2190000000000001},"min":{"1min":0.79000000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.601,"5min":1.863,"15min":1.863},"last":1.121}]},{"osd":3,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2829999999999999,"15min":1.27},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.5509999999999999,"5min":1.98,"15min":1.98},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3740000000000001,"15min":1.3340000000000001},"min":{"1min":0.97299999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7210000000000001,"5min":1.756,"15min":1.9930000000000001},"last":0.83199999999999996}]},{"osd":4,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2629999999999999,"15min":1.1990000000000001},"min":{"1min":0.69899999999999995,"5min":0.66000000000000003,"15min":0.60499999999999998},"max":{"1min":1.802,"5min":1.923,"15min":2.1070000000000002},"last":1.012},{"interface":"front","average":{"1min":1.363,"5min":1.341,"15min":1.377},"min":{"1min":0.94699999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.768,"5min":2.0369999999999999,"15min":2.5169999999999999},"last":1.5209999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.349,"15min":1.3520000000000001},"min":{"1min":1.093,"5min":0.622,"15min":0.622},"max":{"1min":1.599,"5min":1.9199999999999999,"15min":2.0720000000000001},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.444,"15min":1.423},"min":{"1min":1.119,"5min":0.748,"15min":0.67900000000000005},"max":{"1min":1.77,"5min":2.0259999999999998,"15min":2.0270000000000001},"last":1.5609999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.506,"15min":1.4750000000000001},"min":{"1min":1.274,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.8440000000000001,"5min":2.0779999999999998,"15min":2.3959999999999999},"last":1.472},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"min":{"1min":1.1899999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.873,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.595}]}]},{"osd":6,"up_from":13,"seq":55834574950,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":777,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194535,"statfs":{"total":200000000,"available":199204056,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.252,"15min":1.141},"min":{"1min":0.83499999999999996,"5min":0.64700000000000002,"15min":0.60299999999999998},"max":{"1min":1.5940000000000001,"5min":1.627,"15min":1.6850000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.357,"15min":1.274},"min":{"1min":0.81799999999999995,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8149999999999999,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.5640000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.101,"15min":1.091},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63800000000000001},"max":{"1min":1.5269999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.119},{"interface":"front","average":{"1min":1.145,"5min":1.2230000000000001,"15min":1.198},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5940000000000001,"5min":1.8560000000000001,"15min":2.9609999999999999},"last":1.04}]},{"osd":2,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.169,"15min":1.1699999999999999},"min":{"1min":0.84899999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4850000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.236,"15min":1.2090000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.66300000000000003},"max":{"1min":1.627,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.1919999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3009999999999999,"15min":1.2709999999999999},"min":{"1min":0.98299999999999998,"5min":0.79400000000000004,"15min":0.79200000000000004},"max":{"1min":1.627,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.286,"15min":1.2430000000000001},"min":{"1min":0.84999999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7889999999999999},"last":1.27}]},{"osd":4,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.254,"15min":1.208},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.706,"5min":2.444,"15min":2.444},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.282},"min":{"1min":0.91700000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7749999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.0720000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.452,"15min":1.3839999999999999},"min":{"1min":1.0309999999999999,"5min":0.625,"15min":0.58899999999999997},"max":{"1min":1.722,"5min":2.032,"15min":2.032},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3220000000000001,"15min":1.256},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.49199999999999999},"max":{"1min":1.679,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.387}]},{"osd":7,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.4099999999999999,"15min":1.4019999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.768,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.347},{"interface":"front","average":{"1min":1.548,"5min":1.5429999999999999,"15min":1.5289999999999999},"min":{"1min":1.087,"5min":0.83099999999999996,"15min":0.68500000000000005},"max":{"1min":1.982,"5min":2.3140000000000001,"15min":2.794},"last":1.226}]}]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":771,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194541,"statfs":{"total":200000000,"available":199210331,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.3400000000000001,"15min":1.278},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.63400000000000001},"max":{"1min":1.8819999999999999,"5min":9.609,"15min":9.609},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.286,"15min":1.3},"min":{"1min":0.754,"5min":0.751,"15min":0.60799999999999998},"max":{"1min":1.7090000000000001,"5min":1.8260000000000001,"15min":5.3029999999999999},"last":1.5820000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.3169999999999999,"15min":1.252},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6439999999999999,"5min":11.68,"15min":11.68},"last":1.5329999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.1819999999999999,"15min":1.2210000000000001},"min":{"1min":0.70099999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3360000000000001,"5min":2.2410000000000001,"15min":3.7050000000000001},"last":1.175}]},{"osd":2,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2589999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.755},"max":{"1min":1.7030000000000001,"5min":2.0350000000000001,"15min":2.2090000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2450000000000001,"15min":1.276},"min":{"1min":0.94399999999999995,"5min":0.75900000000000001,"15min":0.72399999999999998},"max":{"1min":1.6910000000000001,"5min":2.153,"15min":3.8180000000000001},"last":1.694}]},{"osd":3,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.3200000000000001,"15min":1.272},"min":{"1min":0.72299999999999998,"5min":0.68799999999999994,"15min":0.65400000000000003},"max":{"1min":1.4179999999999999,"5min":11.797000000000001,"15min":11.797000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.454,"5min":1.3740000000000001,"15min":1.347},"min":{"1min":1.135,"5min":0.747,"15min":0.747},"max":{"1min":1.798,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.2769999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.389,"15min":1.4079999999999999},"min":{"1min":1.157,"5min":0.89900000000000002,"15min":0.75900000000000001},"max":{"1min":1.8100000000000001,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.252,"15min":1.4330000000000001},"min":{"1min":0.88700000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.629,"5min":2.4049999999999998,"15min":5.4580000000000002},"last":1.4079999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2649999999999999,"15min":1.357},"min":{"1min":0.84499999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.5780000000000001,"5min":2.5190000000000001,"15min":5.141},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.629,"5min":1.498,"15min":1.552},"min":{"1min":1.127,"5min":0.82499999999999996,"15min":0.71099999999999997},"max":{"1min":2.0350000000000001,"5min":2.657,"15min":2.657},"last":1.6479999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.5469999999999999,"15min":1.6240000000000001},"min":{"1min":1.2130000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6970000000000001,"5min":11.829000000000001,"15min":11.829000000000001},"last":1.75},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.405,"15min":1.4830000000000001},"min":{"1min":1.0920000000000001,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.837,"5min":2.9420000000000002,"15min":4.8419999999999996},"last":1.492}]}]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":770,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194542,"statfs":{"total":200000000,"available":199211350,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.1479999999999999,"15min":1.1220000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5680000000000001,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.071,"5min":1.1619999999999999,"15min":1.1599999999999999},"min":{"1min":0.71099999999999997,"5min":0.63300000000000001,"15min":0.60099999999999998},"max":{"1min":1.6719999999999999,"5min":3.234,"15min":3.234},"last":1.05}]},{"osd":1,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.1419999999999999,"15min":1.1100000000000001},"min":{"1min":0.64700000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.484,"5min":3.718,"15min":3.718},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.2,"15min":1.224},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.7529999999999999,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.419}]},{"osd":2,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.218,"15min":1.1930000000000001},"min":{"1min":0.71799999999999997,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4099999999999999,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":1.331},{"interface":"front","average":{"1min":1.24,"5min":1.3380000000000001,"15min":1.3089999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.58899999999999997},"max":{"1min":1.6870000000000001,"5min":3.532,"15min":3.532},"last":1.6739999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2709999999999999,"15min":1.2889999999999999},"min":{"1min":0.78200000000000003,"5min":0.71899999999999997,"15min":0.67000000000000004},"max":{"1min":1.5700000000000001,"5min":3.363,"15min":3.363},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.399,"15min":1.409},"min":{"1min":0.84999999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.603,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":1.2190000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.274,"15min":1.2969999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.498},"max":{"1min":1.7909999999999999,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.296,"15min":1.341},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.38400000000000001},"max":{"1min":1.571,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":1.1759999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.3720000000000001,"15min":1.3939999999999999},"min":{"1min":0.78300000000000003,"5min":0.64300000000000002,"15min":0.51600000000000001},"max":{"1min":1.478,"5min":4.4429999999999996,"15min":5.0049999999999999},"last":1.726},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4510000000000001,"15min":1.532},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.45400000000000001},"max":{"1min":1.706,"5min":4.1239999999999997,"15min":4.407},"last":1.756}]},{"osd":7,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.4330000000000001,"15min":1.484},"min":{"1min":0.85599999999999998,"5min":0.82499999999999996,"15min":0.438},"max":{"1min":1.7410000000000001,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.536,"15min":1.546},"min":{"1min":0.92000000000000004,"5min":0.78700000000000003,"15min":0.747},"max":{"1min":1.7410000000000001,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":1.966}]}]},{"osd":3,"up_from":13,"seq":55834574952,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":785,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194526,"statfs":{"total":200000000,"available":199195599,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":0.94999999999999996,"15min":0.84599999999999997},"min":{"1min":0.63800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":4.774,"5min":4.774,"15min":5.5730000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.96299999999999997,"15min":0.91600000000000004},"min":{"1min":0.53600000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.2370000000000001,"5min":3.2370000000000001,"15min":6.5599999999999996},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.0049999999999999,"15min":0.96899999999999997},"min":{"1min":0.55200000000000005,"5min":0.39100000000000001,"15min":0.36799999999999999},"max":{"1min":1.4790000000000001,"5min":1.839,"15min":6.8099999999999996},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2629999999999999,"15min":1.1679999999999999},"min":{"1min":0.85799999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6659999999999999,"5min":2.0720000000000001,"15min":3.1920000000000002},"last":1.0800000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.1060000000000001,"15min":1.052},"min":{"1min":0.62,"5min":0.46800000000000003,"15min":0.45300000000000001},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":7.4580000000000002},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.456,"5min":1.2150000000000001,"15min":1.117},"min":{"1min":0.78700000000000003,"5min":0.53200000000000003,"15min":0.51400000000000001},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":0.77800000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.327,"15min":1.339},"min":{"1min":0.82099999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":18.664999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3009999999999999,"15min":1.2110000000000001},"min":{"1min":0.92700000000000005,"5min":0.877,"15min":0.64000000000000001},"max":{"1min":2.0230000000000001,"5min":2.093,"15min":7.0700000000000003},"last":0.96799999999999997}]},{"osd":5,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.333,"15min":1.266},"min":{"1min":0.90500000000000003,"5min":0.77600000000000002,"15min":0.67700000000000005},"max":{"1min":1.9099999999999999,"5min":2.1680000000000001,"15min":6.9139999999999997},"last":1.252},{"interface":"front","average":{"1min":1.6930000000000001,"5min":1.4650000000000001,"15min":1.409},"min":{"1min":1.1140000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":4.5540000000000003,"5min":4.5540000000000003,"15min":16.390999999999998},"last":1.1870000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.4850000000000001,"15min":1.4510000000000001},"min":{"1min":0.92200000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.375,"5min":3.375,"15min":7.7270000000000003},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.494,"15min":1.3959999999999999},"min":{"1min":1.0189999999999999,"5min":0.872,"15min":0.60199999999999998},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":7.242},"last":1.3799999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.821,"5min":1.5660000000000001,"15min":1.4219999999999999},"min":{"1min":1.226,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":4.8630000000000004,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.8089999999999999,"5min":1.5589999999999999,"15min":1.502},"min":{"1min":0.98399999999999999,"5min":0.622,"15min":0.622},"max":{"1min":5.4900000000000002,"5min":5.4900000000000002,"15min":12.946},"last":1.286}]}]},{"osd":2,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":777,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194535,"statfs":{"total":200000000,"available":199204056,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.88200000000000001,"15min":0.91300000000000003},"min":{"1min":0.56000000000000005,"5min":0.48599999999999999,"15min":0.36899999999999999},"max":{"1min":1.3300000000000001,"5min":1.706,"15min":4.2210000000000001},"last":1.046},{"interface":"front","average":{"1min":1.012,"5min":1.042,"15min":1.04},"min":{"1min":0.59299999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.617,"5min":4.3959999999999999,"15min":4.3959999999999999},"last":0.88700000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.1359999999999999,"15min":1.077},"min":{"1min":0.65300000000000002,"5min":0.55900000000000005,"15min":0.49099999999999999},"max":{"1min":1.522,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.181,"15min":1.121},"min":{"1min":0.77000000000000002,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":1.552,"5min":2.448,"15min":2.5350000000000001},"last":0.94899999999999995}]},{"osd":3,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.194,"15min":1.141},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.48199999999999998},"max":{"1min":1.7569999999999999,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.365,"15min":1.427},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.54100000000000004},"max":{"1min":1.728,"5min":2.2829999999999999,"15min":7.5570000000000004},"last":1.502}]},{"osd":4,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4530000000000001,"15min":1.4730000000000001},"min":{"1min":1.0289999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.879,"5min":2.4119999999999999,"15min":7.6189999999999998},"last":1.167},{"interface":"front","average":{"1min":1.371,"5min":1.4910000000000001,"15min":1.458},"min":{"1min":0.98599999999999999,"5min":0.98099999999999998,"15min":0.86599999999999999},"max":{"1min":1.7,"5min":3.423,"15min":3.423},"last":1.4199999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.407,"15min":1.3089999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.61599999999999999},"max":{"1min":1.6000000000000001,"5min":10.507,"15min":10.507},"last":1.7869999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.4910000000000001,"15min":1.5349999999999999},"min":{"1min":1.1020000000000001,"5min":0.88100000000000001,"15min":0.81399999999999995},"max":{"1min":1.9550000000000001,"5min":2.536,"15min":7.657},"last":1.3260000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.52,"15min":1.518},"min":{"1min":1.2210000000000001,"5min":1.0289999999999999,"15min":0.74299999999999999},"max":{"1min":1.9099999999999999,"5min":2.8780000000000001,"15min":3.5299999999999998},"last":1.466},{"interface":"front","average":{"1min":1.399,"5min":1.5640000000000001,"15min":1.611},"min":{"1min":0.96499999999999997,"5min":0.85899999999999999,"15min":0.77600000000000002},"max":{"1min":2.0190000000000001,"5min":10.561,"15min":37.771999999999998},"last":1.6619999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5740000000000001},"min":{"1min":1.26,"5min":0.97599999999999998,"15min":0.83199999999999996},"max":{"1min":2.2519999999999998,"5min":2.7890000000000001,"15min":3.3620000000000001},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.6339999999999999,"15min":1.5209999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.77600000000000002},"max":{"1min":2.1059999999999999,"5min":10.956,"15min":10.956},"last":1.825}]}]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":776,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194536,"statfs":{"total":200000000,"available":199205075,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.93899999999999995,"15min":0.84399999999999997},"min":{"1min":0.42799999999999999,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.331},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.091,"15min":0.95799999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.645,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.1220000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.0489999999999999,"15min":0.96299999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.589,"5min":1.6899999999999999,"15min":1.859},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.99299999999999999,"15min":0.88700000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.36699999999999999},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.702}]},{"osd":3,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.139,"15min":1.0469999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.436},"max":{"1min":1.915,"5min":1.915,"15min":2.1059999999999999},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0960000000000001,"15min":1.0589999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.7110000000000001,"5min":1.9370000000000001,"15min":3.7010000000000001},"last":1.399}]},{"osd":4,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.3460000000000001,"15min":1.27},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7430000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.3799999999999999,"15min":1.256},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.829,"5min":1.903,"15min":2.0470000000000002},"last":1.7989999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4199999999999999,"15min":1.325},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.984,"5min":2.161,"15min":3.5750000000000002},"last":1.9490000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.4890000000000001,"15min":1.3919999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8320000000000001,"5min":1.964,"15min":2.3769999999999998},"last":1.6559999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.454,"15min":1.3260000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.905,"5min":2.0419999999999998,"15min":2.2050000000000001},"last":1.905},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.452,"15min":1.3149999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.63100000000000001},"max":{"1min":2.1040000000000001,"5min":2.121,"15min":2.121},"last":1.601}]},{"osd":7,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.524,"15min":1.399},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.2000000000000002,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.7549999999999999},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5329999999999999,"15min":1.4039999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.2469999999999999,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.839}]}]},{"osd":0,"up_from":13,"seq":55834574950,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1352,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193959,"statfs":{"total":200000000,"available":198614707,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81899999999999995,"15min":0.89000000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":1.3320000000000001,"15min":3.4420000000000002},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.083,"15min":1.0780000000000001},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.46500000000000002},"max":{"1min":1.3200000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.2130000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.85699999999999998,"15min":0.84199999999999997},"min":{"1min":0.44400000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.244,"5min":1.679,"15min":1.679},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.071,"5min":1.1359999999999999,"15min":1.1279999999999999},"min":{"1min":0.83999999999999997,"5min":0.503,"15min":0.437},"max":{"1min":1.446,"5min":1.929,"15min":1.929},"last":0.97499999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.135,"15min":1.234},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4419999999999999,"5min":1.98,"15min":2.7639999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3799999999999999,"15min":1.4710000000000001},"min":{"1min":0.71999999999999997,"5min":0.61299999999999999,"15min":0.58399999999999996},"max":{"1min":1.677,"5min":2.3639999999999999,"15min":3.5720000000000001},"last":1.3100000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.228,"15min":1.278},"min":{"1min":0.85999999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7410000000000001,"5min":1.8819999999999999,"15min":4.492},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.282,"15min":1.2589999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.518,"5min":1.927,"15min":1.927},"last":1.1719999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.256,"15min":1.373},"min":{"1min":0.83399999999999996,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.839,"5min":2.4510000000000001,"15min":4.2779999999999996},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.3049999999999999,"15min":1.3540000000000001},"min":{"1min":0.751,"5min":0.74399999999999999,"15min":0.73099999999999998},"max":{"1min":1.573,"5min":1.847,"15min":1.8740000000000001},"last":1.573}]},{"osd":6,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.4179999999999999,"15min":1.538},"min":{"1min":0.79900000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.613,"5min":2.0419999999999998,"15min":3.5299999999999998},"last":1.369},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.363,"15min":1.458},"min":{"1min":0.78000000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.665,"5min":3.1440000000000001,"15min":4.4349999999999996},"last":1.5549999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.524,"15min":1.6080000000000001},"min":{"1min":0.88,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.8120000000000001,"5min":2.2000000000000002,"15min":3.492},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.446,"15min":1.6080000000000001},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1869999999999998,"5min":3.2570000000000001,"15min":4.5419999999999998},"last":1.385}]}]}],"pool_statfs":[]}} 2023-10-18T09:03:28.904 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:03:29.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.176+0000 7f6abca85640 1 -- 172.21.15.40:0/331102552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa40051d0 msgr2=0x7f6aa400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:29.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.176+0000 7f6abca85640 1 --2- 172.21.15.40:0/331102552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa40051d0 0x7f6aa400b980 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000a2c0 tx=0x7f6aa40057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:29.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.176+0000 7f6abca85640 1 -- 172.21.15.40:0/331102552 shutdown_connections 2023-10-18T09:03:29.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.176+0000 7f6abca85640 1 --2- 172.21.15.40:0/331102552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aa40051d0 0x7f6aa400b980 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:29.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.176+0000 7f6abca85640 1 -- 172.21.15.40:0/331102552 >> 172.21.15.40:0/331102552 conn(0x7f6ab8069720 msgr2=0x7f6ab8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:29.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.176+0000 7f6abca85640 1 -- 172.21.15.40:0/331102552 shutdown_connections 2023-10-18T09:03:29.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.176+0000 7f6abca85640 1 -- 172.21.15.40:0/331102552 wait complete. 2023-10-18T09:03:29.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.178+0000 7f6abca85640 1 Processor -- start 2023-10-18T09:03:29.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.178+0000 7f6abca85640 1 -- start start 2023-10-18T09:03:29.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.179+0000 7f6abca85640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa40051d0 0x7f6ab80acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:29.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.179+0000 7f6abca85640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab815a320 0x7f6ab815a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:29.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.179+0000 7f6abca85640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab814ef30 0x7f6ab814f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:29.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.179+0000 7f6abca85640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab814df20 con 0x7f6ab815a320 2023-10-18T09:03:29.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.179+0000 7f6abca85640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab814e170 con 0x7f6ab814ef30 2023-10-18T09:03:29.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.179+0000 7f6abca85640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab814e450 con 0x7f6aa40051d0 2023-10-18T09:03:29.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.180+0000 7f6aaffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab815a320 0x7f6ab815a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:29.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.180+0000 7f6aaffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab815a320 0x7f6ab815a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49694/0 (socket says 172.21.15.40:49694) 2023-10-18T09:03:29.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.180+0000 7f6aaffff640 1 -- 172.21.15.40:0/1563265935 learned_addr learned my addr 172.21.15.40:0/1563265935 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:29.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.180+0000 7f6ab77fe640 1 --2- 172.21.15.40:0/1563265935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa40051d0 0x7f6ab80acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:29.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.180+0000 7f6ab7fff640 1 --2- 172.21.15.40:0/1563265935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab814ef30 0x7f6ab814f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:29.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.180+0000 7f6aaffff640 1 -- 172.21.15.40:0/1563265935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa40051d0 msgr2=0x7f6ab80acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:29.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.180+0000 7f6aaffff640 1 --2- 172.21.15.40:0/1563265935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa40051d0 0x7f6ab80acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:29.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.180+0000 7f6aaffff640 1 -- 172.21.15.40:0/1563265935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab814ef30 msgr2=0x7f6ab814f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:29.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.180+0000 7f6aaffff640 1 --2- 172.21.15.40:0/1563265935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab814ef30 0x7f6ab814f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:29.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.180+0000 7f6aaffff640 1 -- 172.21.15.40:0/1563265935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab814e730 con 0x7f6ab815a320 2023-10-18T09:03:29.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.180+0000 7f6aaffff640 1 --2- 172.21.15.40:0/1563265935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab815a320 0x7f6ab815a7a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000d600 tx=0x7f6aa000da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:29.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.181+0000 7f6ab4ff9640 1 -- 172.21.15.40:0/1563265935 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa000f880 con 0x7f6ab815a320 2023-10-18T09:03:29.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.181+0000 7f6ab4ff9640 1 -- 172.21.15.40:0/1563265935 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6aa000fad0 con 0x7f6ab815a320 2023-10-18T09:03:29.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.181+0000 7f6abca85640 1 -- 172.21.15.40:0/1563265935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab814ea10 con 0x7f6ab815a320 2023-10-18T09:03:29.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.181+0000 7f6ab4ff9640 1 -- 172.21.15.40:0/1563265935 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0010450 con 0x7f6ab815a320 2023-10-18T09:03:29.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.181+0000 7f6abca85640 1 -- 172.21.15.40:0/1563265935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab814f8f0 con 0x7f6ab815a320 2023-10-18T09:03:29.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.182+0000 7f6abca85640 1 -- 172.21.15.40:0/1563265935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab80caa40 con 0x7f6ab815a320 2023-10-18T09:03:29.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.183+0000 7f6ab4ff9640 1 -- 172.21.15.40:0/1563265935 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6aa00105b0 con 0x7f6ab815a320 2023-10-18T09:03:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.184+0000 7f6ab4ff9640 1 --2- 172.21.15.40:0/1563265935 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6a9806ae40 0x7f6a9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.184+0000 7f6ab4ff9640 1 -- 172.21.15.40:0/1563265935 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6aa00898d0 con 0x7f6ab815a320 2023-10-18T09:03:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.184+0000 7f6ab77fe640 1 --2- 172.21.15.40:0/1563265935 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6a9806ae40 0x7f6a9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.186+0000 7f6ab77fe640 1 --2- 172.21.15.40:0/1563265935 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6a9806ae40 0x7f6a9806d300 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8000c00 tx=0x7f6aa8001b00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.188+0000 7f6ab4ff9640 1 -- 172.21.15.40:0/1563265935 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6aa0054970 con 0x7f6ab815a320 2023-10-18T09:03:29.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.378+0000 7f6abca85640 1 -- 172.21.15.40:0/1563265935 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ab814ec00 con 0x7f6a9806ae40 2023-10-18T09:03:29.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.401+0000 7f6ab4ff9640 1 -- 172.21.15.40:0/1563265935 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279054 (secure 0 0 0) 0x7f6ab814ec00 con 0x7f6a9806ae40 2023-10-18T09:03:29.408 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:29.410 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:03:29.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.406+0000 7f6abca85640 1 -- 172.21.15.40:0/1563265935 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6a9806ae40 msgr2=0x7f6a9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:29.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.406+0000 7f6abca85640 1 --2- 172.21.15.40:0/1563265935 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6a9806ae40 0x7f6a9806d300 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8000c00 tx=0x7f6aa8001b00 comp rx=0 tx=0).stop 2023-10-18T09:03:29.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.406+0000 7f6abca85640 1 -- 172.21.15.40:0/1563265935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab815a320 msgr2=0x7f6ab815a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:29.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.406+0000 7f6abca85640 1 --2- 172.21.15.40:0/1563265935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab815a320 0x7f6ab815a7a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000d600 tx=0x7f6aa000da00 comp rx=0 tx=0).stop 2023-10-18T09:03:29.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.406+0000 7f6abca85640 1 -- 172.21.15.40:0/1563265935 shutdown_connections 2023-10-18T09:03:29.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.406+0000 7f6abca85640 1 --2- 172.21.15.40:0/1563265935 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6a9806ae40 0x7f6a9806d300 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:29.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.406+0000 7f6abca85640 1 --2- 172.21.15.40:0/1563265935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab814ef30 0x7f6ab814f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:29.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.406+0000 7f6abca85640 1 --2- 172.21.15.40:0/1563265935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ab815a320 0x7f6ab815a7a0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:29.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.406+0000 7f6abca85640 1 --2- 172.21.15.40:0/1563265935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6aa40051d0 0x7f6ab80acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:29.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.406+0000 7f6abca85640 1 -- 172.21.15.40:0/1563265935 >> 172.21.15.40:0/1563265935 conn(0x7f6ab8069720 msgr2=0x7f6ab8155670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:29.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.407+0000 7f6abca85640 1 -- 172.21.15.40:0/1563265935 shutdown_connections 2023-10-18T09:03:29.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.407+0000 7f6abca85640 1 -- 172.21.15.40:0/1563265935 wait complete. 2023-10-18T09:03:29.430 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2023-10-18T09:03:29.075882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7371,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555128,"statfs":{"total":1600000000,"available":1592451471,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001852"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313192+0000","last_change":"2023-10-18T09:03:23.313192+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313192+0000","last_undegraded":"2023-10-18T09:03:23.313192+0000","last_fullsized":"2023-10-18T09:03:23.313192+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.322631+0000","last_change":"2023-10-18T09:03:23.322631+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.322631+0000","last_undegraded":"2023-10-18T09:03:23.322631+0000","last_fullsized":"2023-10-18T09:03:23.322631+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.372965+0000","last_change":"2023-10-18T09:03:24.371528+0000","last_active":"2023-10-18T09:03:24.372965+0000","last_peered":"2023-10-18T09:03:24.372965+0000","last_clean":"2023-10-18T09:03:24.372965+0000","last_became_active":"2023-10-18T09:03:24.370994+0000","last_became_peered":"2023-10-18T09:03:24.370994+0000","last_unstale":"2023-10-18T09:03:24.372965+0000","last_undegraded":"2023-10-18T09:03:24.372965+0000","last_fullsized":"2023-10-18T09:03:24.372965+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:31.969146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.325742+0000","last_change":"2023-10-18T09:03:23.325742+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.325742+0000","last_undegraded":"2023-10-18T09:03:23.325742+0000","last_fullsized":"2023-10-18T09:03:23.325742+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313411+0000","last_change":"2023-10-18T09:03:23.313411+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313411+0000","last_undegraded":"2023-10-18T09:03:23.313411+0000","last_fullsized":"2023-10-18T09:03:23.313411+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.319674+0000","last_change":"2023-10-18T09:03:23.319674+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.319674+0000","last_undegraded":"2023-10-18T09:03:23.319674+0000","last_fullsized":"2023-10-18T09:03:23.319674+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.327137+0000","last_change":"2023-10-18T09:03:23.327137+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.327137+0000","last_undegraded":"2023-10-18T09:03:23.327137+0000","last_fullsized":"2023-10-18T09:03:23.327137+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.373894+0000","last_change":"2023-10-18T09:03:24.373509+0000","last_active":"2023-10-18T09:03:24.373894+0000","last_peered":"2023-10-18T09:03:24.373894+0000","last_clean":"2023-10-18T09:03:24.373894+0000","last_became_active":"2023-10-18T09:03:24.372325+0000","last_became_peered":"2023-10-18T09:03:24.372325+0000","last_unstale":"2023-10-18T09:03:24.373894+0000","last_undegraded":"2023-10-18T09:03:24.373894+0000","last_fullsized":"2023-10-18T09:03:24.373894+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:00.288118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.325288+0000","last_change":"2023-10-18T09:03:23.325288+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.325288+0000","last_undegraded":"2023-10-18T09:03:23.325288+0000","last_fullsized":"2023-10-18T09:03:23.325288+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.317783+0000","last_change":"2023-10-18T09:03:23.317783+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.317783+0000","last_undegraded":"2023-10-18T09:03:23.317783+0000","last_fullsized":"2023-10-18T09:03:23.317783+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.316307+0000","last_change":"2023-10-18T09:03:23.316307+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.316307+0000","last_undegraded":"2023-10-18T09:03:23.316307+0000","last_fullsized":"2023-10-18T09:03:23.316307+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321900+0000","last_change":"2023-10-18T09:03:23.321900+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321900+0000","last_undegraded":"2023-10-18T09:03:23.321900+0000","last_fullsized":"2023-10-18T09:03:23.321900+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.378701+0000","last_change":"2023-10-18T09:03:24.378634+0000","last_active":"2023-10-18T09:03:24.378701+0000","last_peered":"2023-10-18T09:03:24.378701+0000","last_clean":"2023-10-18T09:03:24.378701+0000","last_became_active":"2023-10-18T09:03:24.378364+0000","last_became_peered":"2023-10-18T09:03:24.378364+0000","last_unstale":"2023-10-18T09:03:24.378701+0000","last_undegraded":"2023-10-18T09:03:24.378701+0000","last_fullsized":"2023-10-18T09:03:24.378701+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:26:40.521285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.372987+0000","last_change":"2023-10-18T09:03:24.371034+0000","last_active":"2023-10-18T09:03:24.372987+0000","last_peered":"2023-10-18T09:03:24.372987+0000","last_clean":"2023-10-18T09:03:24.372987+0000","last_became_active":"2023-10-18T09:03:24.370294+0000","last_became_peered":"2023-10-18T09:03:24.370294+0000","last_unstale":"2023-10-18T09:03:24.372987+0000","last_undegraded":"2023-10-18T09:03:24.372987+0000","last_fullsized":"2023-10-18T09:03:24.372987+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:13:18.579341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321363+0000","last_change":"2023-10-18T09:03:23.321363+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321363+0000","last_undegraded":"2023-10-18T09:03:23.321363+0000","last_fullsized":"2023-10-18T09:03:23.321363+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312843+0000","last_change":"2023-10-18T09:03:23.312843+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312843+0000","last_undegraded":"2023-10-18T09:03:23.312843+0000","last_fullsized":"2023-10-18T09:03:23.312843+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.322169+0000","last_change":"2023-10-18T09:03:23.322169+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.322169+0000","last_undegraded":"2023-10-18T09:03:23.322169+0000","last_fullsized":"2023-10-18T09:03:23.322169+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321038+0000","last_change":"2023-10-18T09:03:23.321038+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321038+0000","last_undegraded":"2023-10-18T09:03:23.321038+0000","last_fullsized":"2023-10-18T09:03:23.321038+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.373932+0000","last_change":"2023-10-18T09:03:24.373744+0000","last_active":"2023-10-18T09:03:24.373932+0000","last_peered":"2023-10-18T09:03:24.373932+0000","last_clean":"2023-10-18T09:03:24.373932+0000","last_became_active":"2023-10-18T09:03:24.373488+0000","last_became_peered":"2023-10-18T09:03:24.373488+0000","last_unstale":"2023-10-18T09:03:24.373932+0000","last_undegraded":"2023-10-18T09:03:24.373932+0000","last_fullsized":"2023-10-18T09:03:24.373932+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:21.541741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313740+0000","last_change":"2023-10-18T09:03:23.313740+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313740+0000","last_undegraded":"2023-10-18T09:03:23.313740+0000","last_fullsized":"2023-10-18T09:03:23.313740+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.315443+0000","last_change":"2023-10-18T09:03:23.315443+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.315443+0000","last_undegraded":"2023-10-18T09:03:23.315443+0000","last_fullsized":"2023-10-18T09:03:23.315443+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.378342+0000","last_change":"2023-10-18T09:03:24.376004+0000","last_active":"2023-10-18T09:03:24.378342+0000","last_peered":"2023-10-18T09:03:24.378342+0000","last_clean":"2023-10-18T09:03:24.378342+0000","last_became_active":"2023-10-18T09:03:24.375694+0000","last_became_peered":"2023-10-18T09:03:24.375694+0000","last_unstale":"2023-10-18T09:03:24.378342+0000","last_undegraded":"2023-10-18T09:03:24.378342+0000","last_fullsized":"2023-10-18T09:03:24.378342+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:45:05.600764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312725+0000","last_change":"2023-10-18T09:03:23.312725+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312725+0000","last_undegraded":"2023-10-18T09:03:23.312725+0000","last_fullsized":"2023-10-18T09:03:23.312725+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.365671+0000","last_change":"2023-10-18T09:03:24.364680+0000","last_active":"2023-10-18T09:03:24.365671+0000","last_peered":"2023-10-18T09:03:24.365671+0000","last_clean":"2023-10-18T09:03:24.365671+0000","last_became_active":"2023-10-18T09:03:24.364143+0000","last_became_peered":"2023-10-18T09:03:24.364143+0000","last_unstale":"2023-10-18T09:03:24.365671+0000","last_undegraded":"2023-10-18T09:03:24.365671+0000","last_fullsized":"2023-10-18T09:03:24.365671+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:42:51.261581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312889+0000","last_change":"2023-10-18T09:03:23.312889+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312889+0000","last_undegraded":"2023-10-18T09:03:23.312889+0000","last_fullsized":"2023-10-18T09:03:23.312889+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312482+0000","last_change":"2023-10-18T09:03:23.312482+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312482+0000","last_undegraded":"2023-10-18T09:03:23.312482+0000","last_fullsized":"2023-10-18T09:03:23.312482+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.318515+0000","last_change":"2023-10-18T09:03:23.318515+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.318515+0000","last_undegraded":"2023-10-18T09:03:23.318515+0000","last_fullsized":"2023-10-18T09:03:23.318515+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.374017+0000","last_change":"2023-10-18T09:03:24.371719+0000","last_active":"2023-10-18T09:03:24.374017+0000","last_peered":"2023-10-18T09:03:24.374017+0000","last_clean":"2023-10-18T09:03:24.374017+0000","last_became_active":"2023-10-18T09:03:24.371095+0000","last_became_peered":"2023-10-18T09:03:24.371095+0000","last_unstale":"2023-10-18T09:03:24.374017+0000","last_undegraded":"2023-10-18T09:03:24.374017+0000","last_fullsized":"2023-10-18T09:03:24.374017+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:50.009429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.317757+0000","last_change":"2023-10-18T09:03:23.317757+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.317757+0000","last_undegraded":"2023-10-18T09:03:23.317757+0000","last_fullsized":"2023-10-18T09:03:23.317757+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.326266+0000","last_change":"2023-10-18T09:03:23.326266+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.326266+0000","last_undegraded":"2023-10-18T09:03:23.326266+0000","last_fullsized":"2023-10-18T09:03:23.326266+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312103+0000","last_change":"2023-10-18T09:03:23.312103+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312103+0000","last_undegraded":"2023-10-18T09:03:23.312103+0000","last_fullsized":"2023-10-18T09:03:23.312103+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.318051+0000","last_change":"2023-10-18T09:03:23.318051+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.318051+0000","last_undegraded":"2023-10-18T09:03:23.318051+0000","last_fullsized":"2023-10-18T09:03:23.318051+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.320206+0000","last_change":"2023-10-18T09:03:23.320206+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.320206+0000","last_undegraded":"2023-10-18T09:03:23.320206+0000","last_fullsized":"2023-10-18T09:03:23.320206+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.329894+0000","last_change":"2023-10-18T09:03:23.329530+0000","last_active":"2023-10-18T09:03:23.329894+0000","last_peered":"2023-10-18T09:03:23.329894+0000","last_clean":"2023-10-18T09:03:23.329894+0000","last_became_active":"2023-10-18T09:03:23.329181+0000","last_became_peered":"2023-10-18T09:03:23.329181+0000","last_unstale":"2023-10-18T09:03:23.329894+0000","last_undegraded":"2023-10-18T09:03:23.329894+0000","last_fullsized":"2023-10-18T09:03:23.329894+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:29.736042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.318339+0000","last_change":"2023-10-18T09:03:23.318339+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.318339+0000","last_undegraded":"2023-10-18T09:03:23.318339+0000","last_fullsized":"2023-10-18T09:03:23.318339+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"143'1","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.391133+0000","last_change":"2023-10-18T09:03:23.331319+0000","last_active":"2023-10-18T09:03:24.391133+0000","last_peered":"2023-10-18T09:03:24.391133+0000","last_clean":"2023-10-18T09:03:24.391133+0000","last_became_active":"2023-10-18T09:03:23.331015+0000","last_became_peered":"2023-10-18T09:03:23.331015+0000","last_unstale":"2023-10-18T09:03:24.391133+0000","last_undegraded":"2023-10-18T09:03:24.391133+0000","last_fullsized":"2023-10-18T09:03:24.391133+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:40.692371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.319346+0000","last_change":"2023-10-18T09:03:23.319346+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.319346+0000","last_undegraded":"2023-10-18T09:03:23.319346+0000","last_fullsized":"2023-10-18T09:03:23.319346+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331754+0000","last_change":"2023-10-18T09:03:23.330634+0000","last_active":"2023-10-18T09:03:23.331754+0000","last_peered":"2023-10-18T09:03:23.331754+0000","last_clean":"2023-10-18T09:03:23.331754+0000","last_became_active":"2023-10-18T09:03:23.329893+0000","last_became_peered":"2023-10-18T09:03:23.329893+0000","last_unstale":"2023-10-18T09:03:23.331754+0000","last_undegraded":"2023-10-18T09:03:23.331754+0000","last_fullsized":"2023-10-18T09:03:23.331754+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:20:04.939191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.373913+0000","last_change":"2023-10-18T09:03:24.373601+0000","last_active":"2023-10-18T09:03:24.373913+0000","last_peered":"2023-10-18T09:03:24.373913+0000","last_clean":"2023-10-18T09:03:24.373913+0000","last_became_active":"2023-10-18T09:03:24.373369+0000","last_became_peered":"2023-10-18T09:03:24.373369+0000","last_unstale":"2023-10-18T09:03:24.373913+0000","last_undegraded":"2023-10-18T09:03:24.373913+0000","last_fullsized":"2023-10-18T09:03:24.373913+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:48.301601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.328765+0000","last_change":"2023-10-18T09:03:23.328512+0000","last_active":"2023-10-18T09:03:23.328765+0000","last_peered":"2023-10-18T09:03:23.328765+0000","last_clean":"2023-10-18T09:03:23.328765+0000","last_became_active":"2023-10-18T09:03:23.328090+0000","last_became_peered":"2023-10-18T09:03:23.328090+0000","last_unstale":"2023-10-18T09:03:23.328765+0000","last_undegraded":"2023-10-18T09:03:23.328765+0000","last_fullsized":"2023-10-18T09:03:23.328765+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:23.820590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.320552+0000","last_change":"2023-10-18T09:03:23.320552+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.320552+0000","last_undegraded":"2023-10-18T09:03:23.320552+0000","last_fullsized":"2023-10-18T09:03:23.320552+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.329917+0000","last_change":"2023-10-18T09:03:23.329851+0000","last_active":"2023-10-18T09:03:23.329917+0000","last_peered":"2023-10-18T09:03:23.329917+0000","last_clean":"2023-10-18T09:03:23.329917+0000","last_became_active":"2023-10-18T09:03:23.329208+0000","last_became_peered":"2023-10-18T09:03:23.329208+0000","last_unstale":"2023-10-18T09:03:23.329917+0000","last_undegraded":"2023-10-18T09:03:23.329917+0000","last_fullsized":"2023-10-18T09:03:23.329917+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:08.163420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321572+0000","last_change":"2023-10-18T09:03:23.321572+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321572+0000","last_undegraded":"2023-10-18T09:03:23.321572+0000","last_fullsized":"2023-10-18T09:03:23.321572+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"143'1","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.392871+0000","last_change":"2023-10-18T09:03:23.329790+0000","last_active":"2023-10-18T09:03:24.392871+0000","last_peered":"2023-10-18T09:03:24.392871+0000","last_clean":"2023-10-18T09:03:24.392871+0000","last_became_active":"2023-10-18T09:03:23.325710+0000","last_became_peered":"2023-10-18T09:03:23.325710+0000","last_unstale":"2023-10-18T09:03:24.392871+0000","last_undegraded":"2023-10-18T09:03:24.392871+0000","last_fullsized":"2023-10-18T09:03:24.392871+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:06:18.861877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.316958+0000","last_change":"2023-10-18T09:03:23.316958+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.316958+0000","last_undegraded":"2023-10-18T09:03:23.316958+0000","last_fullsized":"2023-10-18T09:03:23.316958+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.318565+0000","last_change":"2023-10-18T09:03:23.318325+0000","last_active":"2023-10-18T09:03:23.318565+0000","last_peered":"2023-10-18T09:03:23.318565+0000","last_clean":"2023-10-18T09:03:23.318565+0000","last_became_active":"2023-10-18T09:03:23.316556+0000","last_became_peered":"2023-10-18T09:03:23.316556+0000","last_unstale":"2023-10-18T09:03:23.318565+0000","last_undegraded":"2023-10-18T09:03:23.318565+0000","last_fullsized":"2023-10-18T09:03:23.318565+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:25:52.365591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.322477+0000","last_change":"2023-10-18T09:03:23.322477+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.322477+0000","last_undegraded":"2023-10-18T09:03:23.322477+0000","last_fullsized":"2023-10-18T09:03:23.322477+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.334091+0000","last_change":"2023-10-18T09:03:23.334041+0000","last_active":"2023-10-18T09:03:23.334091+0000","last_peered":"2023-10-18T09:03:23.334091+0000","last_clean":"2023-10-18T09:03:23.334091+0000","last_became_active":"2023-10-18T09:03:23.333830+0000","last_became_peered":"2023-10-18T09:03:23.333830+0000","last_unstale":"2023-10-18T09:03:23.334091+0000","last_undegraded":"2023-10-18T09:03:23.334091+0000","last_fullsized":"2023-10-18T09:03:23.334091+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:23.484620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.324949+0000","last_change":"2023-10-18T09:03:23.324949+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.324949+0000","last_undegraded":"2023-10-18T09:03:23.324949+0000","last_fullsized":"2023-10-18T09:03:23.324949+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.328748+0000","last_change":"2023-10-18T09:03:23.328337+0000","last_active":"2023-10-18T09:03:23.328748+0000","last_peered":"2023-10-18T09:03:23.328748+0000","last_clean":"2023-10-18T09:03:23.328748+0000","last_became_active":"2023-10-18T09:03:23.325594+0000","last_became_peered":"2023-10-18T09:03:23.325594+0000","last_unstale":"2023-10-18T09:03:23.328748+0000","last_undegraded":"2023-10-18T09:03:23.328748+0000","last_fullsized":"2023-10-18T09:03:23.328748+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:42.662714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321318+0000","last_change":"2023-10-18T09:03:23.321318+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321318+0000","last_undegraded":"2023-10-18T09:03:23.321318+0000","last_fullsized":"2023-10-18T09:03:23.321318+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.346035+0000","last_change":"2023-10-18T09:03:23.325401+0000","last_active":"2023-10-18T09:03:24.346035+0000","last_peered":"2023-10-18T09:03:24.346035+0000","last_clean":"2023-10-18T09:03:24.346035+0000","last_became_active":"2023-10-18T09:03:23.324277+0000","last_became_peered":"2023-10-18T09:03:23.324277+0000","last_unstale":"2023-10-18T09:03:24.346035+0000","last_undegraded":"2023-10-18T09:03:24.346035+0000","last_fullsized":"2023-10-18T09:03:24.346035+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:06:07.682599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.356612+0000","last_change":"2023-10-18T09:03:24.350913+0000","last_active":"2023-10-18T09:03:24.356612+0000","last_peered":"2023-10-18T09:03:24.356612+0000","last_clean":"2023-10-18T09:03:24.356612+0000","last_became_active":"2023-10-18T09:03:24.350214+0000","last_became_peered":"2023-10-18T09:03:24.350214+0000","last_unstale":"2023-10-18T09:03:24.356612+0000","last_undegraded":"2023-10-18T09:03:24.356612+0000","last_fullsized":"2023-10-18T09:03:24.356612+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:24:33.551156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331509+0000","last_change":"2023-10-18T09:03:23.331283+0000","last_active":"2023-10-18T09:03:23.331509+0000","last_peered":"2023-10-18T09:03:23.331509+0000","last_clean":"2023-10-18T09:03:23.331509+0000","last_became_active":"2023-10-18T09:03:23.329500+0000","last_became_peered":"2023-10-18T09:03:23.329500+0000","last_unstale":"2023-10-18T09:03:23.331509+0000","last_undegraded":"2023-10-18T09:03:23.331509+0000","last_fullsized":"2023-10-18T09:03:23.331509+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:13.492685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.326069+0000","last_change":"2023-10-18T09:03:23.326069+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.326069+0000","last_undegraded":"2023-10-18T09:03:23.326069+0000","last_fullsized":"2023-10-18T09:03:23.326069+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.334640+0000","last_change":"2023-10-18T09:03:23.334514+0000","last_active":"2023-10-18T09:03:23.334640+0000","last_peered":"2023-10-18T09:03:23.334640+0000","last_clean":"2023-10-18T09:03:23.334640+0000","last_became_active":"2023-10-18T09:03:23.334268+0000","last_became_peered":"2023-10-18T09:03:23.334268+0000","last_unstale":"2023-10-18T09:03:23.334640+0000","last_undegraded":"2023-10-18T09:03:23.334640+0000","last_fullsized":"2023-10-18T09:03:23.334640+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:16.978313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.367932+0000","last_change":"2023-10-18T09:03:24.367289+0000","last_active":"2023-10-18T09:03:24.367932+0000","last_peered":"2023-10-18T09:03:24.367932+0000","last_clean":"2023-10-18T09:03:24.367932+0000","last_became_active":"2023-10-18T09:03:24.366724+0000","last_became_peered":"2023-10-18T09:03:24.366724+0000","last_unstale":"2023-10-18T09:03:24.367932+0000","last_undegraded":"2023-10-18T09:03:24.367932+0000","last_fullsized":"2023-10-18T09:03:24.367932+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:29:55.406170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.325184+0000","last_change":"2023-10-18T09:03:23.323723+0000","last_active":"2023-10-18T09:03:23.325184+0000","last_peered":"2023-10-18T09:03:23.325184+0000","last_clean":"2023-10-18T09:03:23.325184+0000","last_became_active":"2023-10-18T09:03:23.322838+0000","last_became_peered":"2023-10-18T09:03:23.322838+0000","last_unstale":"2023-10-18T09:03:23.325184+0000","last_undegraded":"2023-10-18T09:03:23.325184+0000","last_fullsized":"2023-10-18T09:03:23.325184+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:12.382096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.375498+0000","last_change":"2023-10-18T09:03:24.374652+0000","last_active":"2023-10-18T09:03:24.375498+0000","last_peered":"2023-10-18T09:03:24.375498+0000","last_clean":"2023-10-18T09:03:24.375498+0000","last_became_active":"2023-10-18T09:03:24.366671+0000","last_became_peered":"2023-10-18T09:03:24.366671+0000","last_unstale":"2023-10-18T09:03:24.375498+0000","last_undegraded":"2023-10-18T09:03:24.375498+0000","last_fullsized":"2023-10-18T09:03:24.375498+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:07:26.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.334932+0000","last_change":"2023-10-18T09:03:23.334809+0000","last_active":"2023-10-18T09:03:23.334932+0000","last_peered":"2023-10-18T09:03:23.334932+0000","last_clean":"2023-10-18T09:03:23.334932+0000","last_became_active":"2023-10-18T09:03:23.334509+0000","last_became_peered":"2023-10-18T09:03:23.334509+0000","last_unstale":"2023-10-18T09:03:23.334932+0000","last_undegraded":"2023-10-18T09:03:23.334932+0000","last_fullsized":"2023-10-18T09:03:23.334932+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:13.492685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.356667+0000","last_change":"2023-10-18T09:03:24.353278+0000","last_active":"2023-10-18T09:03:24.356667+0000","last_peered":"2023-10-18T09:03:24.356667+0000","last_clean":"2023-10-18T09:03:24.356667+0000","last_became_active":"2023-10-18T09:03:24.352158+0000","last_became_peered":"2023-10-18T09:03:24.352158+0000","last_unstale":"2023-10-18T09:03:24.356667+0000","last_undegraded":"2023-10-18T09:03:24.356667+0000","last_fullsized":"2023-10-18T09:03:24.356667+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:05.753025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.332360+0000","last_change":"2023-10-18T09:03:23.332284+0000","last_active":"2023-10-18T09:03:23.332360+0000","last_peered":"2023-10-18T09:03:23.332360+0000","last_clean":"2023-10-18T09:03:23.332360+0000","last_became_active":"2023-10-18T09:03:23.332040+0000","last_became_peered":"2023-10-18T09:03:23.332040+0000","last_unstale":"2023-10-18T09:03:23.332360+0000","last_undegraded":"2023-10-18T09:03:23.332360+0000","last_fullsized":"2023-10-18T09:03:23.332360+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:41:21.941443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313059+0000","last_change":"2023-10-18T09:03:23.313059+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313059+0000","last_undegraded":"2023-10-18T09:03:23.313059+0000","last_fullsized":"2023-10-18T09:03:23.313059+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.333121+0000","last_change":"2023-10-18T09:03:23.332929+0000","last_active":"2023-10-18T09:03:23.333121+0000","last_peered":"2023-10-18T09:03:23.333121+0000","last_clean":"2023-10-18T09:03:23.333121+0000","last_became_active":"2023-10-18T09:03:23.332508+0000","last_became_peered":"2023-10-18T09:03:23.332508+0000","last_unstale":"2023-10-18T09:03:23.333121+0000","last_undegraded":"2023-10-18T09:03:23.333121+0000","last_fullsized":"2023-10-18T09:03:23.333121+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:30.964339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.319843+0000","last_change":"2023-10-18T09:03:23.319843+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.319843+0000","last_undegraded":"2023-10-18T09:03:23.319843+0000","last_fullsized":"2023-10-18T09:03:23.319843+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331736+0000","last_change":"2023-10-18T09:03:23.330153+0000","last_active":"2023-10-18T09:03:23.331736+0000","last_peered":"2023-10-18T09:03:23.331736+0000","last_clean":"2023-10-18T09:03:23.331736+0000","last_became_active":"2023-10-18T09:03:23.325376+0000","last_became_peered":"2023-10-18T09:03:23.325376+0000","last_unstale":"2023-10-18T09:03:23.331736+0000","last_undegraded":"2023-10-18T09:03:23.331736+0000","last_fullsized":"2023-10-18T09:03:23.331736+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:43.431276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.374379+0000","last_change":"2023-10-18T09:03:24.374330+0000","last_active":"2023-10-18T09:03:24.374379+0000","last_peered":"2023-10-18T09:03:24.374379+0000","last_clean":"2023-10-18T09:03:24.374379+0000","last_became_active":"2023-10-18T09:03:24.374144+0000","last_became_peered":"2023-10-18T09:03:24.374144+0000","last_unstale":"2023-10-18T09:03:24.374379+0000","last_undegraded":"2023-10-18T09:03:24.374379+0000","last_fullsized":"2023-10-18T09:03:24.374379+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:59.934981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.325277+0000","last_change":"2023-10-18T09:03:23.324222+0000","last_active":"2023-10-18T09:03:23.325277+0000","last_peered":"2023-10-18T09:03:23.325277+0000","last_clean":"2023-10-18T09:03:23.325277+0000","last_became_active":"2023-10-18T09:03:23.323810+0000","last_became_peered":"2023-10-18T09:03:23.323810+0000","last_unstale":"2023-10-18T09:03:23.325277+0000","last_undegraded":"2023-10-18T09:03:23.325277+0000","last_fullsized":"2023-10-18T09:03:23.325277+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:20.536935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.371800+0000","last_change":"2023-10-18T09:03:24.370815+0000","last_active":"2023-10-18T09:03:24.371800+0000","last_peered":"2023-10-18T09:03:24.371800+0000","last_clean":"2023-10-18T09:03:24.371800+0000","last_became_active":"2023-10-18T09:03:24.370553+0000","last_became_peered":"2023-10-18T09:03:24.370553+0000","last_unstale":"2023-10-18T09:03:24.371800+0000","last_undegraded":"2023-10-18T09:03:24.371800+0000","last_fullsized":"2023-10-18T09:03:24.371800+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:17.983120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.332741+0000","last_change":"2023-10-18T09:03:23.332652+0000","last_active":"2023-10-18T09:03:23.332741+0000","last_peered":"2023-10-18T09:03:23.332741+0000","last_clean":"2023-10-18T09:03:23.332741+0000","last_became_active":"2023-10-18T09:03:23.331276+0000","last_became_peered":"2023-10-18T09:03:23.331276+0000","last_unstale":"2023-10-18T09:03:23.332741+0000","last_undegraded":"2023-10-18T09:03:23.332741+0000","last_fullsized":"2023-10-18T09:03:23.332741+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:12.382096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.367994+0000","last_change":"2023-10-18T09:03:24.367856+0000","last_active":"2023-10-18T09:03:24.367994+0000","last_peered":"2023-10-18T09:03:24.367994+0000","last_clean":"2023-10-18T09:03:24.367994+0000","last_became_active":"2023-10-18T09:03:24.367568+0000","last_became_peered":"2023-10-18T09:03:24.367568+0000","last_unstale":"2023-10-18T09:03:24.367994+0000","last_undegraded":"2023-10-18T09:03:24.367994+0000","last_fullsized":"2023-10-18T09:03:24.367994+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:14.497491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.325153+0000","last_change":"2023-10-18T09:03:23.322408+0000","last_active":"2023-10-18T09:03:23.325153+0000","last_peered":"2023-10-18T09:03:23.325153+0000","last_clean":"2023-10-18T09:03:23.325153+0000","last_became_active":"2023-10-18T09:03:23.321746+0000","last_became_peered":"2023-10-18T09:03:23.321746+0000","last_unstale":"2023-10-18T09:03:23.325153+0000","last_undegraded":"2023-10-18T09:03:23.325153+0000","last_fullsized":"2023-10-18T09:03:23.325153+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:08.225684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313248+0000","last_change":"2023-10-18T09:03:23.313248+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313248+0000","last_undegraded":"2023-10-18T09:03:23.313248+0000","last_fullsized":"2023-10-18T09:03:23.313248+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.325186+0000","last_change":"2023-10-18T09:03:23.324369+0000","last_active":"2023-10-18T09:03:23.325186+0000","last_peered":"2023-10-18T09:03:23.325186+0000","last_clean":"2023-10-18T09:03:23.325186+0000","last_became_active":"2023-10-18T09:03:23.323637+0000","last_became_peered":"2023-10-18T09:03:23.323637+0000","last_unstale":"2023-10-18T09:03:23.325186+0000","last_undegraded":"2023-10-18T09:03:23.325186+0000","last_fullsized":"2023-10-18T09:03:23.325186+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:18:41.207403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.379077+0000","last_change":"2023-10-18T09:03:24.378954+0000","last_active":"2023-10-18T09:03:24.379077+0000","last_peered":"2023-10-18T09:03:24.379077+0000","last_clean":"2023-10-18T09:03:24.379077+0000","last_became_active":"2023-10-18T09:03:24.378534+0000","last_became_peered":"2023-10-18T09:03:24.378534+0000","last_unstale":"2023-10-18T09:03:24.379077+0000","last_undegraded":"2023-10-18T09:03:24.379077+0000","last_fullsized":"2023-10-18T09:03:24.379077+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:14:13.807602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.332533+0000","last_change":"2023-10-18T09:03:23.332462+0000","last_active":"2023-10-18T09:03:23.332533+0000","last_peered":"2023-10-18T09:03:23.332533+0000","last_clean":"2023-10-18T09:03:23.332533+0000","last_became_active":"2023-10-18T09:03:23.332239+0000","last_became_peered":"2023-10-18T09:03:23.332239+0000","last_unstale":"2023-10-18T09:03:23.332533+0000","last_undegraded":"2023-10-18T09:03:23.332533+0000","last_fullsized":"2023-10-18T09:03:23.332533+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:25.333743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.361716+0000","last_change":"2023-10-18T09:03:24.361276+0000","last_active":"2023-10-18T09:03:24.361716+0000","last_peered":"2023-10-18T09:03:24.361716+0000","last_clean":"2023-10-18T09:03:24.361716+0000","last_became_active":"2023-10-18T09:03:24.360521+0000","last_became_peered":"2023-10-18T09:03:24.360521+0000","last_unstale":"2023-10-18T09:03:24.361716+0000","last_undegraded":"2023-10-18T09:03:24.361716+0000","last_fullsized":"2023-10-18T09:03:24.361716+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:11.398182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331491+0000","last_change":"2023-10-18T09:03:23.330931+0000","last_active":"2023-10-18T09:03:23.331491+0000","last_peered":"2023-10-18T09:03:23.331491+0000","last_clean":"2023-10-18T09:03:23.331491+0000","last_became_active":"2023-10-18T09:03:23.330686+0000","last_became_peered":"2023-10-18T09:03:23.330686+0000","last_unstale":"2023-10-18T09:03:23.331491+0000","last_undegraded":"2023-10-18T09:03:23.331491+0000","last_fullsized":"2023-10-18T09:03:23.331491+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:47.296794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.317564+0000","last_change":"2023-10-18T09:03:23.317564+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.317564+0000","last_undegraded":"2023-10-18T09:03:23.317564+0000","last_fullsized":"2023-10-18T09:03:23.317564+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"143'2","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.393968+0000","last_change":"2023-10-18T09:03:23.330533+0000","last_active":"2023-10-18T09:03:24.393968+0000","last_peered":"2023-10-18T09:03:24.393968+0000","last_clean":"2023-10-18T09:03:24.393968+0000","last_became_active":"2023-10-18T09:03:23.330093+0000","last_became_peered":"2023-10-18T09:03:23.330093+0000","last_unstale":"2023-10-18T09:03:24.393968+0000","last_undegraded":"2023-10-18T09:03:24.393968+0000","last_fullsized":"2023-10-18T09:03:24.393968+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:54.344592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.315057+0000","last_change":"2023-10-18T09:03:23.315057+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.315057+0000","last_undegraded":"2023-10-18T09:03:23.315057+0000","last_fullsized":"2023-10-18T09:03:23.315057+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.330035+0000","last_change":"2023-10-18T09:03:23.329947+0000","last_active":"2023-10-18T09:03:23.330035+0000","last_peered":"2023-10-18T09:03:23.330035+0000","last_clean":"2023-10-18T09:03:23.330035+0000","last_became_active":"2023-10-18T09:03:23.329667+0000","last_became_peered":"2023-10-18T09:03:23.329667+0000","last_unstale":"2023-10-18T09:03:23.330035+0000","last_undegraded":"2023-10-18T09:03:23.330035+0000","last_fullsized":"2023-10-18T09:03:23.330035+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:23.484620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313161+0000","last_change":"2023-10-18T09:03:23.313161+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313161+0000","last_undegraded":"2023-10-18T09:03:23.313161+0000","last_fullsized":"2023-10-18T09:03:23.313161+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.333102+0000","last_change":"2023-10-18T09:03:23.332779+0000","last_active":"2023-10-18T09:03:23.333102+0000","last_peered":"2023-10-18T09:03:23.333102+0000","last_clean":"2023-10-18T09:03:23.333102+0000","last_became_active":"2023-10-18T09:03:23.332152+0000","last_became_peered":"2023-10-18T09:03:23.332152+0000","last_unstale":"2023-10-18T09:03:23.333102+0000","last_undegraded":"2023-10-18T09:03:23.333102+0000","last_fullsized":"2023-10-18T09:03:23.333102+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:40.692371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313904+0000","last_change":"2023-10-18T09:03:23.313904+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313904+0000","last_undegraded":"2023-10-18T09:03:23.313904+0000","last_fullsized":"2023-10-18T09:03:23.313904+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.346056+0000","last_change":"2023-10-18T09:03:23.325136+0000","last_active":"2023-10-18T09:03:24.346056+0000","last_peered":"2023-10-18T09:03:24.346056+0000","last_clean":"2023-10-18T09:03:24.346056+0000","last_became_active":"2023-10-18T09:03:23.323782+0000","last_became_peered":"2023-10-18T09:03:23.323782+0000","last_unstale":"2023-10-18T09:03:24.346056+0000","last_undegraded":"2023-10-18T09:03:24.346056+0000","last_fullsized":"2023-10-18T09:03:24.346056+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:42.460476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.318651+0000","last_change":"2023-10-18T09:03:23.318651+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.318651+0000","last_undegraded":"2023-10-18T09:03:23.318651+0000","last_fullsized":"2023-10-18T09:03:23.318651+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.329867+0000","last_change":"2023-10-18T09:03:23.329448+0000","last_active":"2023-10-18T09:03:23.329867+0000","last_peered":"2023-10-18T09:03:23.329867+0000","last_clean":"2023-10-18T09:03:23.329867+0000","last_became_active":"2023-10-18T09:03:23.329094+0000","last_became_peered":"2023-10-18T09:03:23.329094+0000","last_unstale":"2023-10-18T09:03:23.329867+0000","last_undegraded":"2023-10-18T09:03:23.329867+0000","last_fullsized":"2023-10-18T09:03:23.329867+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:42:23.072395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.314648+0000","last_change":"2023-10-18T09:03:23.314648+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.314648+0000","last_undegraded":"2023-10-18T09:03:23.314648+0000","last_fullsized":"2023-10-18T09:03:23.314648+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.324173+0000","last_change":"2023-10-18T09:03:23.323348+0000","last_active":"2023-10-18T09:03:23.324173+0000","last_peered":"2023-10-18T09:03:23.324173+0000","last_clean":"2023-10-18T09:03:23.324173+0000","last_became_active":"2023-10-18T09:03:23.323009+0000","last_became_peered":"2023-10-18T09:03:23.323009+0000","last_unstale":"2023-10-18T09:03:23.324173+0000","last_undegraded":"2023-10-18T09:03:23.324173+0000","last_fullsized":"2023-10-18T09:03:23.324173+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:07:37.803900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.373379+0000","last_change":"2023-10-18T09:03:24.373169+0000","last_active":"2023-10-18T09:03:24.373379+0000","last_peered":"2023-10-18T09:03:24.373379+0000","last_clean":"2023-10-18T09:03:24.373379+0000","last_became_active":"2023-10-18T09:03:24.370785+0000","last_became_peered":"2023-10-18T09:03:24.370785+0000","last_unstale":"2023-10-18T09:03:24.373379+0000","last_undegraded":"2023-10-18T09:03:24.373379+0000","last_fullsized":"2023-10-18T09:03:24.373379+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:24.489427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331718+0000","last_change":"2023-10-18T09:03:23.329465+0000","last_active":"2023-10-18T09:03:23.331718+0000","last_peered":"2023-10-18T09:03:23.331718+0000","last_clean":"2023-10-18T09:03:23.331718+0000","last_became_active":"2023-10-18T09:03:23.329200+0000","last_became_peered":"2023-10-18T09:03:23.329200+0000","last_unstale":"2023-10-18T09:03:23.331718+0000","last_undegraded":"2023-10-18T09:03:23.331718+0000","last_fullsized":"2023-10-18T09:03:23.331718+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:11.421962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.319950+0000","last_change":"2023-10-18T09:03:23.319950+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.319950+0000","last_undegraded":"2023-10-18T09:03:23.319950+0000","last_fullsized":"2023-10-18T09:03:23.319950+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331450+0000","last_change":"2023-10-18T09:03:23.329760+0000","last_active":"2023-10-18T09:03:23.331450+0000","last_peered":"2023-10-18T09:03:23.331450+0000","last_clean":"2023-10-18T09:03:23.331450+0000","last_became_active":"2023-10-18T09:03:23.329479+0000","last_became_peered":"2023-10-18T09:03:23.329479+0000","last_unstale":"2023-10-18T09:03:23.331450+0000","last_undegraded":"2023-10-18T09:03:23.331450+0000","last_fullsized":"2023-10-18T09:03:23.331450+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:20.536935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.378559+0000","last_change":"2023-10-18T09:03:24.378443+0000","last_active":"2023-10-18T09:03:24.378559+0000","last_peered":"2023-10-18T09:03:24.378559+0000","last_clean":"2023-10-18T09:03:24.378559+0000","last_became_active":"2023-10-18T09:03:24.378196+0000","last_became_peered":"2023-10-18T09:03:24.378196+0000","last_unstale":"2023-10-18T09:03:24.378559+0000","last_undegraded":"2023-10-18T09:03:24.378559+0000","last_fullsized":"2023-10-18T09:03:24.378559+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:13:51.906537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.333073+0000","last_change":"2023-10-18T09:03:23.332893+0000","last_active":"2023-10-18T09:03:23.333073+0000","last_peered":"2023-10-18T09:03:23.333073+0000","last_clean":"2023-10-18T09:03:23.333073+0000","last_became_active":"2023-10-18T09:03:23.332302+0000","last_became_peered":"2023-10-18T09:03:23.332302+0000","last_unstale":"2023-10-18T09:03:23.333073+0000","last_undegraded":"2023-10-18T09:03:23.333073+0000","last_fullsized":"2023-10-18T09:03:23.333073+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:34:22.081612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":301,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.308489+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:03:23.308489+0000","last_peered":"2023-10-18T09:03:23.308489+0000","last_clean":"2023-10-18T09:03:23.308489+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:03:23.308489+0000","last_undegraded":"2023-10-18T09:03:23.308489+0000","last_fullsized":"2023-10-18T09:03:23.308489+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1361,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193951,"statfs":{"total":200000000,"available":198606297,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.135,"15min":1.038},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.579,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.109,"15min":1.046},"min":{"1min":0.64500000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.421,"5min":1.97,"15min":1.97},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1819999999999999,"15min":1.155},"min":{"1min":0.83299999999999996,"5min":0.70099999999999996,"15min":0.61799999999999999},"max":{"1min":1.595,"5min":1.829,"15min":1.829},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.1240000000000001,"15min":1.1259999999999999},"min":{"1min":0.72799999999999998,"5min":0.61099999999999999,"15min":0.58799999999999997},"max":{"1min":1.4239999999999999,"5min":1.657,"15min":1.7989999999999999},"last":0.73399999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3220000000000001,"15min":1.264},"min":{"1min":0.94699999999999995,"5min":0.78600000000000003,"15min":0.65400000000000003},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.9730000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.206,"5min":1.27,"15min":1.2190000000000001},"min":{"1min":0.79000000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.601,"5min":1.863,"15min":1.863},"last":1.121}]},{"osd":3,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2829999999999999,"15min":1.27},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.5509999999999999,"5min":1.98,"15min":1.98},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3740000000000001,"15min":1.3340000000000001},"min":{"1min":0.97299999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7210000000000001,"5min":1.756,"15min":1.9930000000000001},"last":0.83199999999999996}]},{"osd":4,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2629999999999999,"15min":1.1990000000000001},"min":{"1min":0.69899999999999995,"5min":0.66000000000000003,"15min":0.60499999999999998},"max":{"1min":1.802,"5min":1.923,"15min":2.1070000000000002},"last":1.012},{"interface":"front","average":{"1min":1.363,"5min":1.341,"15min":1.377},"min":{"1min":0.94699999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.768,"5min":2.0369999999999999,"15min":2.5169999999999999},"last":1.5209999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.349,"15min":1.3520000000000001},"min":{"1min":1.093,"5min":0.622,"15min":0.622},"max":{"1min":1.599,"5min":1.9199999999999999,"15min":2.0720000000000001},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.444,"15min":1.423},"min":{"1min":1.119,"5min":0.748,"15min":0.67900000000000005},"max":{"1min":1.77,"5min":2.0259999999999998,"15min":2.0270000000000001},"last":1.5609999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.506,"15min":1.4750000000000001},"min":{"1min":1.274,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.8440000000000001,"5min":2.0779999999999998,"15min":2.3959999999999999},"last":1.472},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"min":{"1min":1.1899999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.873,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.595}]}]},{"osd":6,"up_from":13,"seq":55834574950,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":777,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194535,"statfs":{"total":200000000,"available":199204056,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.252,"15min":1.141},"min":{"1min":0.83499999999999996,"5min":0.64700000000000002,"15min":0.60299999999999998},"max":{"1min":1.5940000000000001,"5min":1.627,"15min":1.6850000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.357,"15min":1.274},"min":{"1min":0.81799999999999995,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8149999999999999,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.5640000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.101,"15min":1.091},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63800000000000001},"max":{"1min":1.5269999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.119},{"interface":"front","average":{"1min":1.145,"5min":1.2230000000000001,"15min":1.198},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5940000000000001,"5min":1.8560000000000001,"15min":2.9609999999999999},"last":1.04}]},{"osd":2,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.169,"15min":1.1699999999999999},"min":{"1min":0.84899999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4850000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.236,"15min":1.2090000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.66300000000000003},"max":{"1min":1.627,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.1919999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3009999999999999,"15min":1.2709999999999999},"min":{"1min":0.98299999999999998,"5min":0.79400000000000004,"15min":0.79200000000000004},"max":{"1min":1.627,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.286,"15min":1.2430000000000001},"min":{"1min":0.84999999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7889999999999999},"last":1.27}]},{"osd":4,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.254,"15min":1.208},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.706,"5min":2.444,"15min":2.444},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.282},"min":{"1min":0.91700000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7749999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.0720000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.452,"15min":1.3839999999999999},"min":{"1min":1.0309999999999999,"5min":0.625,"15min":0.58899999999999997},"max":{"1min":1.722,"5min":2.032,"15min":2.032},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3220000000000001,"15min":1.256},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.49199999999999999},"max":{"1min":1.679,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.387}]},{"osd":7,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.4099999999999999,"15min":1.4019999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.768,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.347},{"interface":"front","average":{"1min":1.548,"5min":1.5429999999999999,"15min":1.5289999999999999},"min":{"1min":1.087,"5min":0.83099999999999996,"15min":0.68500000000000005},"max":{"1min":1.982,"5min":2.3140000000000001,"15min":2.794},"last":1.226}]}]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":771,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194541,"statfs":{"total":200000000,"available":199210331,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.3400000000000001,"15min":1.278},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.63400000000000001},"max":{"1min":1.8819999999999999,"5min":9.609,"15min":9.609},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.286,"15min":1.3},"min":{"1min":0.754,"5min":0.751,"15min":0.60799999999999998},"max":{"1min":1.7090000000000001,"5min":1.8260000000000001,"15min":5.3029999999999999},"last":1.5820000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.3169999999999999,"15min":1.252},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6439999999999999,"5min":11.68,"15min":11.68},"last":1.5329999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.1819999999999999,"15min":1.2210000000000001},"min":{"1min":0.70099999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3360000000000001,"5min":2.2410000000000001,"15min":3.7050000000000001},"last":1.175}]},{"osd":2,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2589999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.755},"max":{"1min":1.7030000000000001,"5min":2.0350000000000001,"15min":2.2090000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2450000000000001,"15min":1.276},"min":{"1min":0.94399999999999995,"5min":0.75900000000000001,"15min":0.72399999999999998},"max":{"1min":1.6910000000000001,"5min":2.153,"15min":3.8180000000000001},"last":1.694}]},{"osd":3,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.3200000000000001,"15min":1.272},"min":{"1min":0.72299999999999998,"5min":0.68799999999999994,"15min":0.65400000000000003},"max":{"1min":1.4179999999999999,"5min":11.797000000000001,"15min":11.797000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.454,"5min":1.3740000000000001,"15min":1.347},"min":{"1min":1.135,"5min":0.747,"15min":0.747},"max":{"1min":1.798,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.2769999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.389,"15min":1.4079999999999999},"min":{"1min":1.157,"5min":0.89900000000000002,"15min":0.75900000000000001},"max":{"1min":1.8100000000000001,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.252,"15min":1.4330000000000001},"min":{"1min":0.88700000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.629,"5min":2.4049999999999998,"15min":5.4580000000000002},"last":1.4079999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2649999999999999,"15min":1.357},"min":{"1min":0.84499999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.5780000000000001,"5min":2.5190000000000001,"15min":5.141},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.629,"5min":1.498,"15min":1.552},"min":{"1min":1.127,"5min":0.82499999999999996,"15min":0.71099999999999997},"max":{"1min":2.0350000000000001,"5min":2.657,"15min":2.657},"last":1.6479999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.5469999999999999,"15min":1.6240000000000001},"min":{"1min":1.2130000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6970000000000001,"5min":11.829000000000001,"15min":11.829000000000001},"last":1.75},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.405,"15min":1.4830000000000001},"min":{"1min":1.0920000000000001,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.837,"5min":2.9420000000000002,"15min":4.8419999999999996},"last":1.492}]}]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":770,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194542,"statfs":{"total":200000000,"available":199211350,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.1479999999999999,"15min":1.1220000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5680000000000001,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.071,"5min":1.1619999999999999,"15min":1.1599999999999999},"min":{"1min":0.71099999999999997,"5min":0.63300000000000001,"15min":0.60099999999999998},"max":{"1min":1.6719999999999999,"5min":3.234,"15min":3.234},"last":1.05}]},{"osd":1,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.1419999999999999,"15min":1.1100000000000001},"min":{"1min":0.64700000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.484,"5min":3.718,"15min":3.718},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.2,"15min":1.224},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.7529999999999999,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.419}]},{"osd":2,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.218,"15min":1.1930000000000001},"min":{"1min":0.71799999999999997,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4099999999999999,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":1.331},{"interface":"front","average":{"1min":1.24,"5min":1.3380000000000001,"15min":1.3089999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.58899999999999997},"max":{"1min":1.6870000000000001,"5min":3.532,"15min":3.532},"last":1.6739999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2709999999999999,"15min":1.2889999999999999},"min":{"1min":0.78200000000000003,"5min":0.71899999999999997,"15min":0.67000000000000004},"max":{"1min":1.5700000000000001,"5min":3.363,"15min":3.363},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.399,"15min":1.409},"min":{"1min":0.84999999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.603,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":1.2190000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.274,"15min":1.2969999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.498},"max":{"1min":1.7909999999999999,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.296,"15min":1.341},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.38400000000000001},"max":{"1min":1.571,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":1.1759999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.3720000000000001,"15min":1.3939999999999999},"min":{"1min":0.78300000000000003,"5min":0.64300000000000002,"15min":0.51600000000000001},"max":{"1min":1.478,"5min":4.4429999999999996,"15min":5.0049999999999999},"last":1.726},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4510000000000001,"15min":1.532},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.45400000000000001},"max":{"1min":1.706,"5min":4.1239999999999997,"15min":4.407},"last":1.756}]},{"osd":7,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.4330000000000001,"15min":1.484},"min":{"1min":0.85599999999999998,"5min":0.82499999999999996,"15min":0.438},"max":{"1min":1.7410000000000001,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.536,"15min":1.546},"min":{"1min":0.92000000000000004,"5min":0.78700000000000003,"15min":0.747},"max":{"1min":1.7410000000000001,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":1.966}]}]},{"osd":3,"up_from":13,"seq":55834574952,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":785,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194526,"statfs":{"total":200000000,"available":199195599,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":0.94999999999999996,"15min":0.84599999999999997},"min":{"1min":0.63800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":4.774,"5min":4.774,"15min":5.5730000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.96299999999999997,"15min":0.91600000000000004},"min":{"1min":0.53600000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.2370000000000001,"5min":3.2370000000000001,"15min":6.5599999999999996},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.0049999999999999,"15min":0.96899999999999997},"min":{"1min":0.55200000000000005,"5min":0.39100000000000001,"15min":0.36799999999999999},"max":{"1min":1.4790000000000001,"5min":1.839,"15min":6.8099999999999996},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2629999999999999,"15min":1.1679999999999999},"min":{"1min":0.85799999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6659999999999999,"5min":2.0720000000000001,"15min":3.1920000000000002},"last":1.0800000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.1060000000000001,"15min":1.052},"min":{"1min":0.62,"5min":0.46800000000000003,"15min":0.45300000000000001},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":7.4580000000000002},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.456,"5min":1.2150000000000001,"15min":1.117},"min":{"1min":0.78700000000000003,"5min":0.53200000000000003,"15min":0.51400000000000001},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":0.77800000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.327,"15min":1.339},"min":{"1min":0.82099999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":18.664999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3009999999999999,"15min":1.2110000000000001},"min":{"1min":0.92700000000000005,"5min":0.877,"15min":0.64000000000000001},"max":{"1min":2.0230000000000001,"5min":2.093,"15min":7.0700000000000003},"last":0.96799999999999997}]},{"osd":5,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.333,"15min":1.266},"min":{"1min":0.90500000000000003,"5min":0.77600000000000002,"15min":0.67700000000000005},"max":{"1min":1.9099999999999999,"5min":2.1680000000000001,"15min":6.9139999999999997},"last":1.252},{"interface":"front","average":{"1min":1.6930000000000001,"5min":1.4650000000000001,"15min":1.409},"min":{"1min":1.1140000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":4.5540000000000003,"5min":4.5540000000000003,"15min":16.390999999999998},"last":1.1870000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.4850000000000001,"15min":1.4510000000000001},"min":{"1min":0.92200000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.375,"5min":3.375,"15min":7.7270000000000003},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.494,"15min":1.3959999999999999},"min":{"1min":1.0189999999999999,"5min":0.872,"15min":0.60199999999999998},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":7.242},"last":1.3799999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.821,"5min":1.5660000000000001,"15min":1.4219999999999999},"min":{"1min":1.226,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":4.8630000000000004,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.8089999999999999,"5min":1.5589999999999999,"15min":1.502},"min":{"1min":0.98399999999999999,"5min":0.622,"15min":0.622},"max":{"1min":5.4900000000000002,"5min":5.4900000000000002,"15min":12.946},"last":1.286}]}]},{"osd":2,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":777,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194535,"statfs":{"total":200000000,"available":199204056,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.88200000000000001,"15min":0.91300000000000003},"min":{"1min":0.56000000000000005,"5min":0.48599999999999999,"15min":0.36899999999999999},"max":{"1min":1.3300000000000001,"5min":1.706,"15min":4.2210000000000001},"last":1.046},{"interface":"front","average":{"1min":1.012,"5min":1.042,"15min":1.04},"min":{"1min":0.59299999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.617,"5min":4.3959999999999999,"15min":4.3959999999999999},"last":0.88700000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.1359999999999999,"15min":1.077},"min":{"1min":0.65300000000000002,"5min":0.55900000000000005,"15min":0.49099999999999999},"max":{"1min":1.522,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.181,"15min":1.121},"min":{"1min":0.77000000000000002,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":1.552,"5min":2.448,"15min":2.5350000000000001},"last":0.94899999999999995}]},{"osd":3,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.194,"15min":1.141},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.48199999999999998},"max":{"1min":1.7569999999999999,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.365,"15min":1.427},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.54100000000000004},"max":{"1min":1.728,"5min":2.2829999999999999,"15min":7.5570000000000004},"last":1.502}]},{"osd":4,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4530000000000001,"15min":1.4730000000000001},"min":{"1min":1.0289999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.879,"5min":2.4119999999999999,"15min":7.6189999999999998},"last":1.167},{"interface":"front","average":{"1min":1.371,"5min":1.4910000000000001,"15min":1.458},"min":{"1min":0.98599999999999999,"5min":0.98099999999999998,"15min":0.86599999999999999},"max":{"1min":1.7,"5min":3.423,"15min":3.423},"last":1.4199999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.407,"15min":1.3089999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.61599999999999999},"max":{"1min":1.6000000000000001,"5min":10.507,"15min":10.507},"last":1.7869999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.4910000000000001,"15min":1.5349999999999999},"min":{"1min":1.1020000000000001,"5min":0.88100000000000001,"15min":0.81399999999999995},"max":{"1min":1.9550000000000001,"5min":2.536,"15min":7.657},"last":1.3260000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.52,"15min":1.518},"min":{"1min":1.2210000000000001,"5min":1.0289999999999999,"15min":0.74299999999999999},"max":{"1min":1.9099999999999999,"5min":2.8780000000000001,"15min":3.5299999999999998},"last":1.466},{"interface":"front","average":{"1min":1.399,"5min":1.5640000000000001,"15min":1.611},"min":{"1min":0.96499999999999997,"5min":0.85899999999999999,"15min":0.77600000000000002},"max":{"1min":2.0190000000000001,"5min":10.561,"15min":37.771999999999998},"last":1.6619999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5740000000000001},"min":{"1min":1.26,"5min":0.97599999999999998,"15min":0.83199999999999996},"max":{"1min":2.2519999999999998,"5min":2.7890000000000001,"15min":3.3620000000000001},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.6339999999999999,"15min":1.5209999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.77600000000000002},"max":{"1min":2.1059999999999999,"5min":10.956,"15min":10.956},"last":1.825}]}]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":776,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194536,"statfs":{"total":200000000,"available":199205075,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.93899999999999995,"15min":0.84399999999999997},"min":{"1min":0.42799999999999999,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.331},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.091,"15min":0.95799999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.645,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.1220000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.0489999999999999,"15min":0.96299999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.589,"5min":1.6899999999999999,"15min":1.859},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.99299999999999999,"15min":0.88700000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.36699999999999999},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.702}]},{"osd":3,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.139,"15min":1.0469999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.436},"max":{"1min":1.915,"5min":1.915,"15min":2.1059999999999999},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0960000000000001,"15min":1.0589999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.7110000000000001,"5min":1.9370000000000001,"15min":3.7010000000000001},"last":1.399}]},{"osd":4,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.3460000000000001,"15min":1.27},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7430000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.3799999999999999,"15min":1.256},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.829,"5min":1.903,"15min":2.0470000000000002},"last":1.7989999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4199999999999999,"15min":1.325},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.984,"5min":2.161,"15min":3.5750000000000002},"last":1.9490000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.4890000000000001,"15min":1.3919999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8320000000000001,"5min":1.964,"15min":2.3769999999999998},"last":1.6559999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.454,"15min":1.3260000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.905,"5min":2.0419999999999998,"15min":2.2050000000000001},"last":1.905},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.452,"15min":1.3149999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.63100000000000001},"max":{"1min":2.1040000000000001,"5min":2.121,"15min":2.121},"last":1.601}]},{"osd":7,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.524,"15min":1.399},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.2000000000000002,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.7549999999999999},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5329999999999999,"15min":1.4039999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.2469999999999999,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.839}]}]},{"osd":0,"up_from":13,"seq":55834574950,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1352,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193959,"statfs":{"total":200000000,"available":198614707,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81899999999999995,"15min":0.89000000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":1.3320000000000001,"15min":3.4420000000000002},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.083,"15min":1.0780000000000001},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.46500000000000002},"max":{"1min":1.3200000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.2130000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.85699999999999998,"15min":0.84199999999999997},"min":{"1min":0.44400000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.244,"5min":1.679,"15min":1.679},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.071,"5min":1.1359999999999999,"15min":1.1279999999999999},"min":{"1min":0.83999999999999997,"5min":0.503,"15min":0.437},"max":{"1min":1.446,"5min":1.929,"15min":1.929},"last":0.97499999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.135,"15min":1.234},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4419999999999999,"5min":1.98,"15min":2.7639999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3799999999999999,"15min":1.4710000000000001},"min":{"1min":0.71999999999999997,"5min":0.61299999999999999,"15min":0.58399999999999996},"max":{"1min":1.677,"5min":2.3639999999999999,"15min":3.5720000000000001},"last":1.3100000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.228,"15min":1.278},"min":{"1min":0.85999999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7410000000000001,"5min":1.8819999999999999,"15min":4.492},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.282,"15min":1.2589999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.518,"5min":1.927,"15min":1.927},"last":1.1719999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.256,"15min":1.373},"min":{"1min":0.83399999999999996,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.839,"5min":2.4510000000000001,"15min":4.2779999999999996},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.3049999999999999,"15min":1.3540000000000001},"min":{"1min":0.751,"5min":0.74399999999999999,"15min":0.73099999999999998},"max":{"1min":1.573,"5min":1.847,"15min":1.8740000000000001},"last":1.573}]},{"osd":6,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.4179999999999999,"15min":1.538},"min":{"1min":0.79900000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.613,"5min":2.0419999999999998,"15min":3.5299999999999998},"last":1.369},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.363,"15min":1.458},"min":{"1min":0.78000000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.665,"5min":3.1440000000000001,"15min":4.4349999999999996},"last":1.5549999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.524,"15min":1.6080000000000001},"min":{"1min":0.88,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.8120000000000001,"5min":2.2000000000000002,"15min":3.492},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.446,"15min":1.6080000000000001},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1869999999999998,"5min":3.2570000000000001,"15min":4.5419999999999998},"last":1.385}]}]}],"pool_statfs":[]}} 2023-10-18T09:03:29.436 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:03:29.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.704+0000 7f97955ed640 1 -- 172.21.15.40:0/2491589979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f979006d6f0 msgr2=0x7f97900b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:29.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.704+0000 7f97955ed640 1 --2- 172.21.15.40:0/2491589979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f979006d6f0 0x7f97900b15b0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f9780009830 tx=0x7f978002bcd0 comp rx=0 tx=0).stop 2023-10-18T09:03:29.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.704+0000 7f97955ed640 1 -- 172.21.15.40:0/2491589979 shutdown_connections 2023-10-18T09:03:29.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.704+0000 7f97955ed640 1 --2- 172.21.15.40:0/2491589979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f979006d6f0 0x7f97900b15b0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:29.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.704+0000 7f97955ed640 1 -- 172.21.15.40:0/2491589979 >> 172.21.15.40:0/2491589979 conn(0x7f9790069720 msgr2=0x7f9790069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:29.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.704+0000 7f97955ed640 1 -- 172.21.15.40:0/2491589979 shutdown_connections 2023-10-18T09:03:29.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.704+0000 7f97955ed640 1 -- 172.21.15.40:0/2491589979 wait complete. 2023-10-18T09:03:29.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.706+0000 7f97955ed640 1 Processor -- start 2023-10-18T09:03:29.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.706+0000 7f97955ed640 1 -- start start 2023-10-18T09:03:29.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.707+0000 7f97955ed640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f979006d6f0 0x7f97901f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:29.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.707+0000 7f97955ed640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97900b0630 0x7f9790159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:29.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.707+0000 7f97955ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97900ca0d0 0x7f979014a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:29.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.707+0000 7f97955ed640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9790148c20 con 0x7f97900ca0d0 2023-10-18T09:03:29.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.707+0000 7f97955ed640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9790148f00 con 0x7f979006d6f0 2023-10-18T09:03:29.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.707+0000 7f97955ed640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97901491e0 con 0x7f97900b0630 2023-10-18T09:03:29.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.707+0000 7f9794dec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97900ca0d0 0x7f979014a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:29.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.708+0000 7f9794dec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97900ca0d0 0x7f979014a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49698/0 (socket says 172.21.15.40:49698) 2023-10-18T09:03:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.708+0000 7f9794dec640 1 -- 172.21.15.40:0/2752508308 learned_addr learned my addr 172.21.15.40:0/2752508308 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.708+0000 7f978ffff640 1 --2- 172.21.15.40:0/2752508308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f979006d6f0 0x7f97901f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.708+0000 7f97877fe640 1 --2- 172.21.15.40:0/2752508308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97900b0630 0x7f9790159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.708+0000 7f9794dec640 1 -- 172.21.15.40:0/2752508308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97900b0630 msgr2=0x7f9790159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.708+0000 7f9794dec640 1 --2- 172.21.15.40:0/2752508308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97900b0630 0x7f9790159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.708+0000 7f9794dec640 1 -- 172.21.15.40:0/2752508308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f979006d6f0 msgr2=0x7f97901f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.708+0000 7f9794dec640 1 --2- 172.21.15.40:0/2752508308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f979006d6f0 0x7f97901f2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.708+0000 7f9794dec640 1 -- 172.21.15.40:0/2752508308 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9780009490 con 0x7f97900ca0d0 2023-10-18T09:03:29.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.708+0000 7f97877fe640 1 --2- 172.21.15.40:0/2752508308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97900b0630 0x7f9790159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:29.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.708+0000 7f9794dec640 1 --2- 172.21.15.40:0/2752508308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97900ca0d0 0x7f979014a190 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f978800c4a0 tx=0x7f978800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:29.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.709+0000 7f978d7fa640 1 -- 172.21.15.40:0/2752508308 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788005480 con 0x7f97900ca0d0 2023-10-18T09:03:29.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.709+0000 7f978d7fa640 1 -- 172.21.15.40:0/2752508308 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f97880036e0 con 0x7f97900ca0d0 2023-10-18T09:03:29.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.709+0000 7f97955ed640 1 -- 172.21.15.40:0/2752508308 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97901494c0 con 0x7f97900ca0d0 2023-10-18T09:03:29.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.709+0000 7f978d7fa640 1 -- 172.21.15.40:0/2752508308 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788003af0 con 0x7f97900ca0d0 2023-10-18T09:03:29.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.709+0000 7f97955ed640 1 -- 172.21.15.40:0/2752508308 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97901f2b60 con 0x7f97900ca0d0 2023-10-18T09:03:29.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.710+0000 7f97955ed640 1 -- 172.21.15.40:0/2752508308 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97901499e0 con 0x7f97900ca0d0 2023-10-18T09:03:29.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.711+0000 7f978d7fa640 1 -- 172.21.15.40:0/2752508308 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9788003c50 con 0x7f97900ca0d0 2023-10-18T09:03:29.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.712+0000 7f978d7fa640 1 --2- 172.21.15.40:0/2752508308 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f977006ae40 0x7f977006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:29.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.712+0000 7f978d7fa640 1 -- 172.21.15.40:0/2752508308 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9788088ec0 con 0x7f97900ca0d0 2023-10-18T09:03:29.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.713+0000 7f978ffff640 1 --2- 172.21.15.40:0/2752508308 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f977006ae40 0x7f977006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:29.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.714+0000 7f978ffff640 1 --2- 172.21.15.40:0/2752508308 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f977006ae40 0x7f977006d300 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9780036000 tx=0x7f9780004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:29.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.717+0000 7f978d7fa640 1 -- 172.21.15.40:0/2752508308 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f97901499e0 con 0x7f97900ca0d0 2023-10-18T09:03:29.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.908+0000 7f97955ed640 1 -- 172.21.15.40:0/2752508308 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97900b0b10 con 0x7f977006ae40 2023-10-18T09:03:29.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.932+0000 7f978d7fa640 1 -- 172.21.15.40:0/2752508308 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279054 (secure 0 0 0) 0x7f9780004400 con 0x7f977006ae40 2023-10-18T09:03:29.939 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:29.940 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:03:29.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.937+0000 7f97955ed640 1 -- 172.21.15.40:0/2752508308 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f977006ae40 msgr2=0x7f977006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:29.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.937+0000 7f97955ed640 1 --2- 172.21.15.40:0/2752508308 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f977006ae40 0x7f977006d300 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9780036000 tx=0x7f9780004730 comp rx=0 tx=0).stop 2023-10-18T09:03:29.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.937+0000 7f97955ed640 1 -- 172.21.15.40:0/2752508308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97900ca0d0 msgr2=0x7f979014a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:29.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.937+0000 7f97955ed640 1 --2- 172.21.15.40:0/2752508308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97900ca0d0 0x7f979014a190 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f978800c4a0 tx=0x7f978800c960 comp rx=0 tx=0).stop 2023-10-18T09:03:29.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.937+0000 7f97955ed640 1 -- 172.21.15.40:0/2752508308 shutdown_connections 2023-10-18T09:03:29.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.937+0000 7f97955ed640 1 --2- 172.21.15.40:0/2752508308 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f977006ae40 0x7f977006d300 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:29.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.937+0000 7f97955ed640 1 --2- 172.21.15.40:0/2752508308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97900ca0d0 0x7f979014a190 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:29.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.937+0000 7f97955ed640 1 --2- 172.21.15.40:0/2752508308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97900b0630 0x7f9790159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:29.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.937+0000 7f97955ed640 1 --2- 172.21.15.40:0/2752508308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f979006d6f0 0x7f97901f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:29.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.937+0000 7f97955ed640 1 -- 172.21.15.40:0/2752508308 >> 172.21.15.40:0/2752508308 conn(0x7f9790069720 msgr2=0x7f979014fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:29.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.938+0000 7f97955ed640 1 -- 172.21.15.40:0/2752508308 shutdown_connections 2023-10-18T09:03:29.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:29.938+0000 7f97955ed640 1 -- 172.21.15.40:0/2752508308 wait complete. 2023-10-18T09:03:29.961 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2023-10-18T09:03:29.075882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7371,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555128,"statfs":{"total":1600000000,"available":1592451471,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001852"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313192+0000","last_change":"2023-10-18T09:03:23.313192+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313192+0000","last_undegraded":"2023-10-18T09:03:23.313192+0000","last_fullsized":"2023-10-18T09:03:23.313192+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.322631+0000","last_change":"2023-10-18T09:03:23.322631+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.322631+0000","last_undegraded":"2023-10-18T09:03:23.322631+0000","last_fullsized":"2023-10-18T09:03:23.322631+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.372965+0000","last_change":"2023-10-18T09:03:24.371528+0000","last_active":"2023-10-18T09:03:24.372965+0000","last_peered":"2023-10-18T09:03:24.372965+0000","last_clean":"2023-10-18T09:03:24.372965+0000","last_became_active":"2023-10-18T09:03:24.370994+0000","last_became_peered":"2023-10-18T09:03:24.370994+0000","last_unstale":"2023-10-18T09:03:24.372965+0000","last_undegraded":"2023-10-18T09:03:24.372965+0000","last_fullsized":"2023-10-18T09:03:24.372965+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:31.969146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.325742+0000","last_change":"2023-10-18T09:03:23.325742+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.325742+0000","last_undegraded":"2023-10-18T09:03:23.325742+0000","last_fullsized":"2023-10-18T09:03:23.325742+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313411+0000","last_change":"2023-10-18T09:03:23.313411+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313411+0000","last_undegraded":"2023-10-18T09:03:23.313411+0000","last_fullsized":"2023-10-18T09:03:23.313411+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.319674+0000","last_change":"2023-10-18T09:03:23.319674+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.319674+0000","last_undegraded":"2023-10-18T09:03:23.319674+0000","last_fullsized":"2023-10-18T09:03:23.319674+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.327137+0000","last_change":"2023-10-18T09:03:23.327137+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.327137+0000","last_undegraded":"2023-10-18T09:03:23.327137+0000","last_fullsized":"2023-10-18T09:03:23.327137+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.373894+0000","last_change":"2023-10-18T09:03:24.373509+0000","last_active":"2023-10-18T09:03:24.373894+0000","last_peered":"2023-10-18T09:03:24.373894+0000","last_clean":"2023-10-18T09:03:24.373894+0000","last_became_active":"2023-10-18T09:03:24.372325+0000","last_became_peered":"2023-10-18T09:03:24.372325+0000","last_unstale":"2023-10-18T09:03:24.373894+0000","last_undegraded":"2023-10-18T09:03:24.373894+0000","last_fullsized":"2023-10-18T09:03:24.373894+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:00.288118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.325288+0000","last_change":"2023-10-18T09:03:23.325288+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.325288+0000","last_undegraded":"2023-10-18T09:03:23.325288+0000","last_fullsized":"2023-10-18T09:03:23.325288+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.317783+0000","last_change":"2023-10-18T09:03:23.317783+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.317783+0000","last_undegraded":"2023-10-18T09:03:23.317783+0000","last_fullsized":"2023-10-18T09:03:23.317783+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.316307+0000","last_change":"2023-10-18T09:03:23.316307+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.316307+0000","last_undegraded":"2023-10-18T09:03:23.316307+0000","last_fullsized":"2023-10-18T09:03:23.316307+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321900+0000","last_change":"2023-10-18T09:03:23.321900+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321900+0000","last_undegraded":"2023-10-18T09:03:23.321900+0000","last_fullsized":"2023-10-18T09:03:23.321900+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.378701+0000","last_change":"2023-10-18T09:03:24.378634+0000","last_active":"2023-10-18T09:03:24.378701+0000","last_peered":"2023-10-18T09:03:24.378701+0000","last_clean":"2023-10-18T09:03:24.378701+0000","last_became_active":"2023-10-18T09:03:24.378364+0000","last_became_peered":"2023-10-18T09:03:24.378364+0000","last_unstale":"2023-10-18T09:03:24.378701+0000","last_undegraded":"2023-10-18T09:03:24.378701+0000","last_fullsized":"2023-10-18T09:03:24.378701+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:26:40.521285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.372987+0000","last_change":"2023-10-18T09:03:24.371034+0000","last_active":"2023-10-18T09:03:24.372987+0000","last_peered":"2023-10-18T09:03:24.372987+0000","last_clean":"2023-10-18T09:03:24.372987+0000","last_became_active":"2023-10-18T09:03:24.370294+0000","last_became_peered":"2023-10-18T09:03:24.370294+0000","last_unstale":"2023-10-18T09:03:24.372987+0000","last_undegraded":"2023-10-18T09:03:24.372987+0000","last_fullsized":"2023-10-18T09:03:24.372987+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:13:18.579341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321363+0000","last_change":"2023-10-18T09:03:23.321363+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321363+0000","last_undegraded":"2023-10-18T09:03:23.321363+0000","last_fullsized":"2023-10-18T09:03:23.321363+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312843+0000","last_change":"2023-10-18T09:03:23.312843+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312843+0000","last_undegraded":"2023-10-18T09:03:23.312843+0000","last_fullsized":"2023-10-18T09:03:23.312843+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.322169+0000","last_change":"2023-10-18T09:03:23.322169+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.322169+0000","last_undegraded":"2023-10-18T09:03:23.322169+0000","last_fullsized":"2023-10-18T09:03:23.322169+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321038+0000","last_change":"2023-10-18T09:03:23.321038+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321038+0000","last_undegraded":"2023-10-18T09:03:23.321038+0000","last_fullsized":"2023-10-18T09:03:23.321038+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.373932+0000","last_change":"2023-10-18T09:03:24.373744+0000","last_active":"2023-10-18T09:03:24.373932+0000","last_peered":"2023-10-18T09:03:24.373932+0000","last_clean":"2023-10-18T09:03:24.373932+0000","last_became_active":"2023-10-18T09:03:24.373488+0000","last_became_peered":"2023-10-18T09:03:24.373488+0000","last_unstale":"2023-10-18T09:03:24.373932+0000","last_undegraded":"2023-10-18T09:03:24.373932+0000","last_fullsized":"2023-10-18T09:03:24.373932+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:21.541741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313740+0000","last_change":"2023-10-18T09:03:23.313740+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313740+0000","last_undegraded":"2023-10-18T09:03:23.313740+0000","last_fullsized":"2023-10-18T09:03:23.313740+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.315443+0000","last_change":"2023-10-18T09:03:23.315443+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.315443+0000","last_undegraded":"2023-10-18T09:03:23.315443+0000","last_fullsized":"2023-10-18T09:03:23.315443+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.378342+0000","last_change":"2023-10-18T09:03:24.376004+0000","last_active":"2023-10-18T09:03:24.378342+0000","last_peered":"2023-10-18T09:03:24.378342+0000","last_clean":"2023-10-18T09:03:24.378342+0000","last_became_active":"2023-10-18T09:03:24.375694+0000","last_became_peered":"2023-10-18T09:03:24.375694+0000","last_unstale":"2023-10-18T09:03:24.378342+0000","last_undegraded":"2023-10-18T09:03:24.378342+0000","last_fullsized":"2023-10-18T09:03:24.378342+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:45:05.600764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312725+0000","last_change":"2023-10-18T09:03:23.312725+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312725+0000","last_undegraded":"2023-10-18T09:03:23.312725+0000","last_fullsized":"2023-10-18T09:03:23.312725+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.365671+0000","last_change":"2023-10-18T09:03:24.364680+0000","last_active":"2023-10-18T09:03:24.365671+0000","last_peered":"2023-10-18T09:03:24.365671+0000","last_clean":"2023-10-18T09:03:24.365671+0000","last_became_active":"2023-10-18T09:03:24.364143+0000","last_became_peered":"2023-10-18T09:03:24.364143+0000","last_unstale":"2023-10-18T09:03:24.365671+0000","last_undegraded":"2023-10-18T09:03:24.365671+0000","last_fullsized":"2023-10-18T09:03:24.365671+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:42:51.261581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312889+0000","last_change":"2023-10-18T09:03:23.312889+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312889+0000","last_undegraded":"2023-10-18T09:03:23.312889+0000","last_fullsized":"2023-10-18T09:03:23.312889+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312482+0000","last_change":"2023-10-18T09:03:23.312482+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312482+0000","last_undegraded":"2023-10-18T09:03:23.312482+0000","last_fullsized":"2023-10-18T09:03:23.312482+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.318515+0000","last_change":"2023-10-18T09:03:23.318515+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.318515+0000","last_undegraded":"2023-10-18T09:03:23.318515+0000","last_fullsized":"2023-10-18T09:03:23.318515+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.374017+0000","last_change":"2023-10-18T09:03:24.371719+0000","last_active":"2023-10-18T09:03:24.374017+0000","last_peered":"2023-10-18T09:03:24.374017+0000","last_clean":"2023-10-18T09:03:24.374017+0000","last_became_active":"2023-10-18T09:03:24.371095+0000","last_became_peered":"2023-10-18T09:03:24.371095+0000","last_unstale":"2023-10-18T09:03:24.374017+0000","last_undegraded":"2023-10-18T09:03:24.374017+0000","last_fullsized":"2023-10-18T09:03:24.374017+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:50.009429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.317757+0000","last_change":"2023-10-18T09:03:23.317757+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.317757+0000","last_undegraded":"2023-10-18T09:03:23.317757+0000","last_fullsized":"2023-10-18T09:03:23.317757+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.326266+0000","last_change":"2023-10-18T09:03:23.326266+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.326266+0000","last_undegraded":"2023-10-18T09:03:23.326266+0000","last_fullsized":"2023-10-18T09:03:23.326266+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312103+0000","last_change":"2023-10-18T09:03:23.312103+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312103+0000","last_undegraded":"2023-10-18T09:03:23.312103+0000","last_fullsized":"2023-10-18T09:03:23.312103+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.318051+0000","last_change":"2023-10-18T09:03:23.318051+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.318051+0000","last_undegraded":"2023-10-18T09:03:23.318051+0000","last_fullsized":"2023-10-18T09:03:23.318051+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.320206+0000","last_change":"2023-10-18T09:03:23.320206+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.320206+0000","last_undegraded":"2023-10-18T09:03:23.320206+0000","last_fullsized":"2023-10-18T09:03:23.320206+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.329894+0000","last_change":"2023-10-18T09:03:23.329530+0000","last_active":"2023-10-18T09:03:23.329894+0000","last_peered":"2023-10-18T09:03:23.329894+0000","last_clean":"2023-10-18T09:03:23.329894+0000","last_became_active":"2023-10-18T09:03:23.329181+0000","last_became_peered":"2023-10-18T09:03:23.329181+0000","last_unstale":"2023-10-18T09:03:23.329894+0000","last_undegraded":"2023-10-18T09:03:23.329894+0000","last_fullsized":"2023-10-18T09:03:23.329894+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:29.736042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.318339+0000","last_change":"2023-10-18T09:03:23.318339+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.318339+0000","last_undegraded":"2023-10-18T09:03:23.318339+0000","last_fullsized":"2023-10-18T09:03:23.318339+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"143'1","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.391133+0000","last_change":"2023-10-18T09:03:23.331319+0000","last_active":"2023-10-18T09:03:24.391133+0000","last_peered":"2023-10-18T09:03:24.391133+0000","last_clean":"2023-10-18T09:03:24.391133+0000","last_became_active":"2023-10-18T09:03:23.331015+0000","last_became_peered":"2023-10-18T09:03:23.331015+0000","last_unstale":"2023-10-18T09:03:24.391133+0000","last_undegraded":"2023-10-18T09:03:24.391133+0000","last_fullsized":"2023-10-18T09:03:24.391133+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:40.692371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.319346+0000","last_change":"2023-10-18T09:03:23.319346+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.319346+0000","last_undegraded":"2023-10-18T09:03:23.319346+0000","last_fullsized":"2023-10-18T09:03:23.319346+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331754+0000","last_change":"2023-10-18T09:03:23.330634+0000","last_active":"2023-10-18T09:03:23.331754+0000","last_peered":"2023-10-18T09:03:23.331754+0000","last_clean":"2023-10-18T09:03:23.331754+0000","last_became_active":"2023-10-18T09:03:23.329893+0000","last_became_peered":"2023-10-18T09:03:23.329893+0000","last_unstale":"2023-10-18T09:03:23.331754+0000","last_undegraded":"2023-10-18T09:03:23.331754+0000","last_fullsized":"2023-10-18T09:03:23.331754+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:20:04.939191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.373913+0000","last_change":"2023-10-18T09:03:24.373601+0000","last_active":"2023-10-18T09:03:24.373913+0000","last_peered":"2023-10-18T09:03:24.373913+0000","last_clean":"2023-10-18T09:03:24.373913+0000","last_became_active":"2023-10-18T09:03:24.373369+0000","last_became_peered":"2023-10-18T09:03:24.373369+0000","last_unstale":"2023-10-18T09:03:24.373913+0000","last_undegraded":"2023-10-18T09:03:24.373913+0000","last_fullsized":"2023-10-18T09:03:24.373913+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:48.301601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.328765+0000","last_change":"2023-10-18T09:03:23.328512+0000","last_active":"2023-10-18T09:03:23.328765+0000","last_peered":"2023-10-18T09:03:23.328765+0000","last_clean":"2023-10-18T09:03:23.328765+0000","last_became_active":"2023-10-18T09:03:23.328090+0000","last_became_peered":"2023-10-18T09:03:23.328090+0000","last_unstale":"2023-10-18T09:03:23.328765+0000","last_undegraded":"2023-10-18T09:03:23.328765+0000","last_fullsized":"2023-10-18T09:03:23.328765+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:23.820590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.320552+0000","last_change":"2023-10-18T09:03:23.320552+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.320552+0000","last_undegraded":"2023-10-18T09:03:23.320552+0000","last_fullsized":"2023-10-18T09:03:23.320552+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.329917+0000","last_change":"2023-10-18T09:03:23.329851+0000","last_active":"2023-10-18T09:03:23.329917+0000","last_peered":"2023-10-18T09:03:23.329917+0000","last_clean":"2023-10-18T09:03:23.329917+0000","last_became_active":"2023-10-18T09:03:23.329208+0000","last_became_peered":"2023-10-18T09:03:23.329208+0000","last_unstale":"2023-10-18T09:03:23.329917+0000","last_undegraded":"2023-10-18T09:03:23.329917+0000","last_fullsized":"2023-10-18T09:03:23.329917+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:08.163420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321572+0000","last_change":"2023-10-18T09:03:23.321572+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321572+0000","last_undegraded":"2023-10-18T09:03:23.321572+0000","last_fullsized":"2023-10-18T09:03:23.321572+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"143'1","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.392871+0000","last_change":"2023-10-18T09:03:23.329790+0000","last_active":"2023-10-18T09:03:24.392871+0000","last_peered":"2023-10-18T09:03:24.392871+0000","last_clean":"2023-10-18T09:03:24.392871+0000","last_became_active":"2023-10-18T09:03:23.325710+0000","last_became_peered":"2023-10-18T09:03:23.325710+0000","last_unstale":"2023-10-18T09:03:24.392871+0000","last_undegraded":"2023-10-18T09:03:24.392871+0000","last_fullsized":"2023-10-18T09:03:24.392871+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:06:18.861877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.316958+0000","last_change":"2023-10-18T09:03:23.316958+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.316958+0000","last_undegraded":"2023-10-18T09:03:23.316958+0000","last_fullsized":"2023-10-18T09:03:23.316958+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.318565+0000","last_change":"2023-10-18T09:03:23.318325+0000","last_active":"2023-10-18T09:03:23.318565+0000","last_peered":"2023-10-18T09:03:23.318565+0000","last_clean":"2023-10-18T09:03:23.318565+0000","last_became_active":"2023-10-18T09:03:23.316556+0000","last_became_peered":"2023-10-18T09:03:23.316556+0000","last_unstale":"2023-10-18T09:03:23.318565+0000","last_undegraded":"2023-10-18T09:03:23.318565+0000","last_fullsized":"2023-10-18T09:03:23.318565+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:25:52.365591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.322477+0000","last_change":"2023-10-18T09:03:23.322477+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.322477+0000","last_undegraded":"2023-10-18T09:03:23.322477+0000","last_fullsized":"2023-10-18T09:03:23.322477+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.334091+0000","last_change":"2023-10-18T09:03:23.334041+0000","last_active":"2023-10-18T09:03:23.334091+0000","last_peered":"2023-10-18T09:03:23.334091+0000","last_clean":"2023-10-18T09:03:23.334091+0000","last_became_active":"2023-10-18T09:03:23.333830+0000","last_became_peered":"2023-10-18T09:03:23.333830+0000","last_unstale":"2023-10-18T09:03:23.334091+0000","last_undegraded":"2023-10-18T09:03:23.334091+0000","last_fullsized":"2023-10-18T09:03:23.334091+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:23.484620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.324949+0000","last_change":"2023-10-18T09:03:23.324949+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.324949+0000","last_undegraded":"2023-10-18T09:03:23.324949+0000","last_fullsized":"2023-10-18T09:03:23.324949+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.328748+0000","last_change":"2023-10-18T09:03:23.328337+0000","last_active":"2023-10-18T09:03:23.328748+0000","last_peered":"2023-10-18T09:03:23.328748+0000","last_clean":"2023-10-18T09:03:23.328748+0000","last_became_active":"2023-10-18T09:03:23.325594+0000","last_became_peered":"2023-10-18T09:03:23.325594+0000","last_unstale":"2023-10-18T09:03:23.328748+0000","last_undegraded":"2023-10-18T09:03:23.328748+0000","last_fullsized":"2023-10-18T09:03:23.328748+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:42.662714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321318+0000","last_change":"2023-10-18T09:03:23.321318+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321318+0000","last_undegraded":"2023-10-18T09:03:23.321318+0000","last_fullsized":"2023-10-18T09:03:23.321318+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.346035+0000","last_change":"2023-10-18T09:03:23.325401+0000","last_active":"2023-10-18T09:03:24.346035+0000","last_peered":"2023-10-18T09:03:24.346035+0000","last_clean":"2023-10-18T09:03:24.346035+0000","last_became_active":"2023-10-18T09:03:23.324277+0000","last_became_peered":"2023-10-18T09:03:23.324277+0000","last_unstale":"2023-10-18T09:03:24.346035+0000","last_undegraded":"2023-10-18T09:03:24.346035+0000","last_fullsized":"2023-10-18T09:03:24.346035+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:06:07.682599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.356612+0000","last_change":"2023-10-18T09:03:24.350913+0000","last_active":"2023-10-18T09:03:24.356612+0000","last_peered":"2023-10-18T09:03:24.356612+0000","last_clean":"2023-10-18T09:03:24.356612+0000","last_became_active":"2023-10-18T09:03:24.350214+0000","last_became_peered":"2023-10-18T09:03:24.350214+0000","last_unstale":"2023-10-18T09:03:24.356612+0000","last_undegraded":"2023-10-18T09:03:24.356612+0000","last_fullsized":"2023-10-18T09:03:24.356612+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:24:33.551156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331509+0000","last_change":"2023-10-18T09:03:23.331283+0000","last_active":"2023-10-18T09:03:23.331509+0000","last_peered":"2023-10-18T09:03:23.331509+0000","last_clean":"2023-10-18T09:03:23.331509+0000","last_became_active":"2023-10-18T09:03:23.329500+0000","last_became_peered":"2023-10-18T09:03:23.329500+0000","last_unstale":"2023-10-18T09:03:23.331509+0000","last_undegraded":"2023-10-18T09:03:23.331509+0000","last_fullsized":"2023-10-18T09:03:23.331509+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:13.492685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.326069+0000","last_change":"2023-10-18T09:03:23.326069+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.326069+0000","last_undegraded":"2023-10-18T09:03:23.326069+0000","last_fullsized":"2023-10-18T09:03:23.326069+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.334640+0000","last_change":"2023-10-18T09:03:23.334514+0000","last_active":"2023-10-18T09:03:23.334640+0000","last_peered":"2023-10-18T09:03:23.334640+0000","last_clean":"2023-10-18T09:03:23.334640+0000","last_became_active":"2023-10-18T09:03:23.334268+0000","last_became_peered":"2023-10-18T09:03:23.334268+0000","last_unstale":"2023-10-18T09:03:23.334640+0000","last_undegraded":"2023-10-18T09:03:23.334640+0000","last_fullsized":"2023-10-18T09:03:23.334640+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:16.978313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.367932+0000","last_change":"2023-10-18T09:03:24.367289+0000","last_active":"2023-10-18T09:03:24.367932+0000","last_peered":"2023-10-18T09:03:24.367932+0000","last_clean":"2023-10-18T09:03:24.367932+0000","last_became_active":"2023-10-18T09:03:24.366724+0000","last_became_peered":"2023-10-18T09:03:24.366724+0000","last_unstale":"2023-10-18T09:03:24.367932+0000","last_undegraded":"2023-10-18T09:03:24.367932+0000","last_fullsized":"2023-10-18T09:03:24.367932+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:29:55.406170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.325184+0000","last_change":"2023-10-18T09:03:23.323723+0000","last_active":"2023-10-18T09:03:23.325184+0000","last_peered":"2023-10-18T09:03:23.325184+0000","last_clean":"2023-10-18T09:03:23.325184+0000","last_became_active":"2023-10-18T09:03:23.322838+0000","last_became_peered":"2023-10-18T09:03:23.322838+0000","last_unstale":"2023-10-18T09:03:23.325184+0000","last_undegraded":"2023-10-18T09:03:23.325184+0000","last_fullsized":"2023-10-18T09:03:23.325184+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:12.382096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.375498+0000","last_change":"2023-10-18T09:03:24.374652+0000","last_active":"2023-10-18T09:03:24.375498+0000","last_peered":"2023-10-18T09:03:24.375498+0000","last_clean":"2023-10-18T09:03:24.375498+0000","last_became_active":"2023-10-18T09:03:24.366671+0000","last_became_peered":"2023-10-18T09:03:24.366671+0000","last_unstale":"2023-10-18T09:03:24.375498+0000","last_undegraded":"2023-10-18T09:03:24.375498+0000","last_fullsized":"2023-10-18T09:03:24.375498+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:07:26.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.334932+0000","last_change":"2023-10-18T09:03:23.334809+0000","last_active":"2023-10-18T09:03:23.334932+0000","last_peered":"2023-10-18T09:03:23.334932+0000","last_clean":"2023-10-18T09:03:23.334932+0000","last_became_active":"2023-10-18T09:03:23.334509+0000","last_became_peered":"2023-10-18T09:03:23.334509+0000","last_unstale":"2023-10-18T09:03:23.334932+0000","last_undegraded":"2023-10-18T09:03:23.334932+0000","last_fullsized":"2023-10-18T09:03:23.334932+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:13.492685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.356667+0000","last_change":"2023-10-18T09:03:24.353278+0000","last_active":"2023-10-18T09:03:24.356667+0000","last_peered":"2023-10-18T09:03:24.356667+0000","last_clean":"2023-10-18T09:03:24.356667+0000","last_became_active":"2023-10-18T09:03:24.352158+0000","last_became_peered":"2023-10-18T09:03:24.352158+0000","last_unstale":"2023-10-18T09:03:24.356667+0000","last_undegraded":"2023-10-18T09:03:24.356667+0000","last_fullsized":"2023-10-18T09:03:24.356667+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:05.753025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.332360+0000","last_change":"2023-10-18T09:03:23.332284+0000","last_active":"2023-10-18T09:03:23.332360+0000","last_peered":"2023-10-18T09:03:23.332360+0000","last_clean":"2023-10-18T09:03:23.332360+0000","last_became_active":"2023-10-18T09:03:23.332040+0000","last_became_peered":"2023-10-18T09:03:23.332040+0000","last_unstale":"2023-10-18T09:03:23.332360+0000","last_undegraded":"2023-10-18T09:03:23.332360+0000","last_fullsized":"2023-10-18T09:03:23.332360+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:41:21.941443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313059+0000","last_change":"2023-10-18T09:03:23.313059+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313059+0000","last_undegraded":"2023-10-18T09:03:23.313059+0000","last_fullsized":"2023-10-18T09:03:23.313059+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.333121+0000","last_change":"2023-10-18T09:03:23.332929+0000","last_active":"2023-10-18T09:03:23.333121+0000","last_peered":"2023-10-18T09:03:23.333121+0000","last_clean":"2023-10-18T09:03:23.333121+0000","last_became_active":"2023-10-18T09:03:23.332508+0000","last_became_peered":"2023-10-18T09:03:23.332508+0000","last_unstale":"2023-10-18T09:03:23.333121+0000","last_undegraded":"2023-10-18T09:03:23.333121+0000","last_fullsized":"2023-10-18T09:03:23.333121+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:30.964339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.319843+0000","last_change":"2023-10-18T09:03:23.319843+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.319843+0000","last_undegraded":"2023-10-18T09:03:23.319843+0000","last_fullsized":"2023-10-18T09:03:23.319843+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331736+0000","last_change":"2023-10-18T09:03:23.330153+0000","last_active":"2023-10-18T09:03:23.331736+0000","last_peered":"2023-10-18T09:03:23.331736+0000","last_clean":"2023-10-18T09:03:23.331736+0000","last_became_active":"2023-10-18T09:03:23.325376+0000","last_became_peered":"2023-10-18T09:03:23.325376+0000","last_unstale":"2023-10-18T09:03:23.331736+0000","last_undegraded":"2023-10-18T09:03:23.331736+0000","last_fullsized":"2023-10-18T09:03:23.331736+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:43.431276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.374379+0000","last_change":"2023-10-18T09:03:24.374330+0000","last_active":"2023-10-18T09:03:24.374379+0000","last_peered":"2023-10-18T09:03:24.374379+0000","last_clean":"2023-10-18T09:03:24.374379+0000","last_became_active":"2023-10-18T09:03:24.374144+0000","last_became_peered":"2023-10-18T09:03:24.374144+0000","last_unstale":"2023-10-18T09:03:24.374379+0000","last_undegraded":"2023-10-18T09:03:24.374379+0000","last_fullsized":"2023-10-18T09:03:24.374379+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:59.934981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.325277+0000","last_change":"2023-10-18T09:03:23.324222+0000","last_active":"2023-10-18T09:03:23.325277+0000","last_peered":"2023-10-18T09:03:23.325277+0000","last_clean":"2023-10-18T09:03:23.325277+0000","last_became_active":"2023-10-18T09:03:23.323810+0000","last_became_peered":"2023-10-18T09:03:23.323810+0000","last_unstale":"2023-10-18T09:03:23.325277+0000","last_undegraded":"2023-10-18T09:03:23.325277+0000","last_fullsized":"2023-10-18T09:03:23.325277+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:20.536935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.371800+0000","last_change":"2023-10-18T09:03:24.370815+0000","last_active":"2023-10-18T09:03:24.371800+0000","last_peered":"2023-10-18T09:03:24.371800+0000","last_clean":"2023-10-18T09:03:24.371800+0000","last_became_active":"2023-10-18T09:03:24.370553+0000","last_became_peered":"2023-10-18T09:03:24.370553+0000","last_unstale":"2023-10-18T09:03:24.371800+0000","last_undegraded":"2023-10-18T09:03:24.371800+0000","last_fullsized":"2023-10-18T09:03:24.371800+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:17.983120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.332741+0000","last_change":"2023-10-18T09:03:23.332652+0000","last_active":"2023-10-18T09:03:23.332741+0000","last_peered":"2023-10-18T09:03:23.332741+0000","last_clean":"2023-10-18T09:03:23.332741+0000","last_became_active":"2023-10-18T09:03:23.331276+0000","last_became_peered":"2023-10-18T09:03:23.331276+0000","last_unstale":"2023-10-18T09:03:23.332741+0000","last_undegraded":"2023-10-18T09:03:23.332741+0000","last_fullsized":"2023-10-18T09:03:23.332741+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:12.382096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.367994+0000","last_change":"2023-10-18T09:03:24.367856+0000","last_active":"2023-10-18T09:03:24.367994+0000","last_peered":"2023-10-18T09:03:24.367994+0000","last_clean":"2023-10-18T09:03:24.367994+0000","last_became_active":"2023-10-18T09:03:24.367568+0000","last_became_peered":"2023-10-18T09:03:24.367568+0000","last_unstale":"2023-10-18T09:03:24.367994+0000","last_undegraded":"2023-10-18T09:03:24.367994+0000","last_fullsized":"2023-10-18T09:03:24.367994+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:14.497491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.325153+0000","last_change":"2023-10-18T09:03:23.322408+0000","last_active":"2023-10-18T09:03:23.325153+0000","last_peered":"2023-10-18T09:03:23.325153+0000","last_clean":"2023-10-18T09:03:23.325153+0000","last_became_active":"2023-10-18T09:03:23.321746+0000","last_became_peered":"2023-10-18T09:03:23.321746+0000","last_unstale":"2023-10-18T09:03:23.325153+0000","last_undegraded":"2023-10-18T09:03:23.325153+0000","last_fullsized":"2023-10-18T09:03:23.325153+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:08.225684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313248+0000","last_change":"2023-10-18T09:03:23.313248+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313248+0000","last_undegraded":"2023-10-18T09:03:23.313248+0000","last_fullsized":"2023-10-18T09:03:23.313248+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.325186+0000","last_change":"2023-10-18T09:03:23.324369+0000","last_active":"2023-10-18T09:03:23.325186+0000","last_peered":"2023-10-18T09:03:23.325186+0000","last_clean":"2023-10-18T09:03:23.325186+0000","last_became_active":"2023-10-18T09:03:23.323637+0000","last_became_peered":"2023-10-18T09:03:23.323637+0000","last_unstale":"2023-10-18T09:03:23.325186+0000","last_undegraded":"2023-10-18T09:03:23.325186+0000","last_fullsized":"2023-10-18T09:03:23.325186+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:18:41.207403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.379077+0000","last_change":"2023-10-18T09:03:24.378954+0000","last_active":"2023-10-18T09:03:24.379077+0000","last_peered":"2023-10-18T09:03:24.379077+0000","last_clean":"2023-10-18T09:03:24.379077+0000","last_became_active":"2023-10-18T09:03:24.378534+0000","last_became_peered":"2023-10-18T09:03:24.378534+0000","last_unstale":"2023-10-18T09:03:24.379077+0000","last_undegraded":"2023-10-18T09:03:24.379077+0000","last_fullsized":"2023-10-18T09:03:24.379077+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:14:13.807602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.332533+0000","last_change":"2023-10-18T09:03:23.332462+0000","last_active":"2023-10-18T09:03:23.332533+0000","last_peered":"2023-10-18T09:03:23.332533+0000","last_clean":"2023-10-18T09:03:23.332533+0000","last_became_active":"2023-10-18T09:03:23.332239+0000","last_became_peered":"2023-10-18T09:03:23.332239+0000","last_unstale":"2023-10-18T09:03:23.332533+0000","last_undegraded":"2023-10-18T09:03:23.332533+0000","last_fullsized":"2023-10-18T09:03:23.332533+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:25.333743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.361716+0000","last_change":"2023-10-18T09:03:24.361276+0000","last_active":"2023-10-18T09:03:24.361716+0000","last_peered":"2023-10-18T09:03:24.361716+0000","last_clean":"2023-10-18T09:03:24.361716+0000","last_became_active":"2023-10-18T09:03:24.360521+0000","last_became_peered":"2023-10-18T09:03:24.360521+0000","last_unstale":"2023-10-18T09:03:24.361716+0000","last_undegraded":"2023-10-18T09:03:24.361716+0000","last_fullsized":"2023-10-18T09:03:24.361716+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:11.398182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331491+0000","last_change":"2023-10-18T09:03:23.330931+0000","last_active":"2023-10-18T09:03:23.331491+0000","last_peered":"2023-10-18T09:03:23.331491+0000","last_clean":"2023-10-18T09:03:23.331491+0000","last_became_active":"2023-10-18T09:03:23.330686+0000","last_became_peered":"2023-10-18T09:03:23.330686+0000","last_unstale":"2023-10-18T09:03:23.331491+0000","last_undegraded":"2023-10-18T09:03:23.331491+0000","last_fullsized":"2023-10-18T09:03:23.331491+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:47.296794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.317564+0000","last_change":"2023-10-18T09:03:23.317564+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.317564+0000","last_undegraded":"2023-10-18T09:03:23.317564+0000","last_fullsized":"2023-10-18T09:03:23.317564+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"143'2","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.393968+0000","last_change":"2023-10-18T09:03:23.330533+0000","last_active":"2023-10-18T09:03:24.393968+0000","last_peered":"2023-10-18T09:03:24.393968+0000","last_clean":"2023-10-18T09:03:24.393968+0000","last_became_active":"2023-10-18T09:03:23.330093+0000","last_became_peered":"2023-10-18T09:03:23.330093+0000","last_unstale":"2023-10-18T09:03:24.393968+0000","last_undegraded":"2023-10-18T09:03:24.393968+0000","last_fullsized":"2023-10-18T09:03:24.393968+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:54.344592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.315057+0000","last_change":"2023-10-18T09:03:23.315057+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.315057+0000","last_undegraded":"2023-10-18T09:03:23.315057+0000","last_fullsized":"2023-10-18T09:03:23.315057+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.330035+0000","last_change":"2023-10-18T09:03:23.329947+0000","last_active":"2023-10-18T09:03:23.330035+0000","last_peered":"2023-10-18T09:03:23.330035+0000","last_clean":"2023-10-18T09:03:23.330035+0000","last_became_active":"2023-10-18T09:03:23.329667+0000","last_became_peered":"2023-10-18T09:03:23.329667+0000","last_unstale":"2023-10-18T09:03:23.330035+0000","last_undegraded":"2023-10-18T09:03:23.330035+0000","last_fullsized":"2023-10-18T09:03:23.330035+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:23.484620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313161+0000","last_change":"2023-10-18T09:03:23.313161+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313161+0000","last_undegraded":"2023-10-18T09:03:23.313161+0000","last_fullsized":"2023-10-18T09:03:23.313161+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.333102+0000","last_change":"2023-10-18T09:03:23.332779+0000","last_active":"2023-10-18T09:03:23.333102+0000","last_peered":"2023-10-18T09:03:23.333102+0000","last_clean":"2023-10-18T09:03:23.333102+0000","last_became_active":"2023-10-18T09:03:23.332152+0000","last_became_peered":"2023-10-18T09:03:23.332152+0000","last_unstale":"2023-10-18T09:03:23.333102+0000","last_undegraded":"2023-10-18T09:03:23.333102+0000","last_fullsized":"2023-10-18T09:03:23.333102+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:40.692371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313904+0000","last_change":"2023-10-18T09:03:23.313904+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313904+0000","last_undegraded":"2023-10-18T09:03:23.313904+0000","last_fullsized":"2023-10-18T09:03:23.313904+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.346056+0000","last_change":"2023-10-18T09:03:23.325136+0000","last_active":"2023-10-18T09:03:24.346056+0000","last_peered":"2023-10-18T09:03:24.346056+0000","last_clean":"2023-10-18T09:03:24.346056+0000","last_became_active":"2023-10-18T09:03:23.323782+0000","last_became_peered":"2023-10-18T09:03:23.323782+0000","last_unstale":"2023-10-18T09:03:24.346056+0000","last_undegraded":"2023-10-18T09:03:24.346056+0000","last_fullsized":"2023-10-18T09:03:24.346056+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:42.460476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.318651+0000","last_change":"2023-10-18T09:03:23.318651+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.318651+0000","last_undegraded":"2023-10-18T09:03:23.318651+0000","last_fullsized":"2023-10-18T09:03:23.318651+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.329867+0000","last_change":"2023-10-18T09:03:23.329448+0000","last_active":"2023-10-18T09:03:23.329867+0000","last_peered":"2023-10-18T09:03:23.329867+0000","last_clean":"2023-10-18T09:03:23.329867+0000","last_became_active":"2023-10-18T09:03:23.329094+0000","last_became_peered":"2023-10-18T09:03:23.329094+0000","last_unstale":"2023-10-18T09:03:23.329867+0000","last_undegraded":"2023-10-18T09:03:23.329867+0000","last_fullsized":"2023-10-18T09:03:23.329867+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:42:23.072395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.314648+0000","last_change":"2023-10-18T09:03:23.314648+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.314648+0000","last_undegraded":"2023-10-18T09:03:23.314648+0000","last_fullsized":"2023-10-18T09:03:23.314648+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.324173+0000","last_change":"2023-10-18T09:03:23.323348+0000","last_active":"2023-10-18T09:03:23.324173+0000","last_peered":"2023-10-18T09:03:23.324173+0000","last_clean":"2023-10-18T09:03:23.324173+0000","last_became_active":"2023-10-18T09:03:23.323009+0000","last_became_peered":"2023-10-18T09:03:23.323009+0000","last_unstale":"2023-10-18T09:03:23.324173+0000","last_undegraded":"2023-10-18T09:03:23.324173+0000","last_fullsized":"2023-10-18T09:03:23.324173+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:07:37.803900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.373379+0000","last_change":"2023-10-18T09:03:24.373169+0000","last_active":"2023-10-18T09:03:24.373379+0000","last_peered":"2023-10-18T09:03:24.373379+0000","last_clean":"2023-10-18T09:03:24.373379+0000","last_became_active":"2023-10-18T09:03:24.370785+0000","last_became_peered":"2023-10-18T09:03:24.370785+0000","last_unstale":"2023-10-18T09:03:24.373379+0000","last_undegraded":"2023-10-18T09:03:24.373379+0000","last_fullsized":"2023-10-18T09:03:24.373379+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:24.489427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331718+0000","last_change":"2023-10-18T09:03:23.329465+0000","last_active":"2023-10-18T09:03:23.331718+0000","last_peered":"2023-10-18T09:03:23.331718+0000","last_clean":"2023-10-18T09:03:23.331718+0000","last_became_active":"2023-10-18T09:03:23.329200+0000","last_became_peered":"2023-10-18T09:03:23.329200+0000","last_unstale":"2023-10-18T09:03:23.331718+0000","last_undegraded":"2023-10-18T09:03:23.331718+0000","last_fullsized":"2023-10-18T09:03:23.331718+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:11.421962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.319950+0000","last_change":"2023-10-18T09:03:23.319950+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.319950+0000","last_undegraded":"2023-10-18T09:03:23.319950+0000","last_fullsized":"2023-10-18T09:03:23.319950+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331450+0000","last_change":"2023-10-18T09:03:23.329760+0000","last_active":"2023-10-18T09:03:23.331450+0000","last_peered":"2023-10-18T09:03:23.331450+0000","last_clean":"2023-10-18T09:03:23.331450+0000","last_became_active":"2023-10-18T09:03:23.329479+0000","last_became_peered":"2023-10-18T09:03:23.329479+0000","last_unstale":"2023-10-18T09:03:23.331450+0000","last_undegraded":"2023-10-18T09:03:23.331450+0000","last_fullsized":"2023-10-18T09:03:23.331450+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:20.536935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.378559+0000","last_change":"2023-10-18T09:03:24.378443+0000","last_active":"2023-10-18T09:03:24.378559+0000","last_peered":"2023-10-18T09:03:24.378559+0000","last_clean":"2023-10-18T09:03:24.378559+0000","last_became_active":"2023-10-18T09:03:24.378196+0000","last_became_peered":"2023-10-18T09:03:24.378196+0000","last_unstale":"2023-10-18T09:03:24.378559+0000","last_undegraded":"2023-10-18T09:03:24.378559+0000","last_fullsized":"2023-10-18T09:03:24.378559+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:13:51.906537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.333073+0000","last_change":"2023-10-18T09:03:23.332893+0000","last_active":"2023-10-18T09:03:23.333073+0000","last_peered":"2023-10-18T09:03:23.333073+0000","last_clean":"2023-10-18T09:03:23.333073+0000","last_became_active":"2023-10-18T09:03:23.332302+0000","last_became_peered":"2023-10-18T09:03:23.332302+0000","last_unstale":"2023-10-18T09:03:23.333073+0000","last_undegraded":"2023-10-18T09:03:23.333073+0000","last_fullsized":"2023-10-18T09:03:23.333073+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:34:22.081612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":301,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.308489+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:03:23.308489+0000","last_peered":"2023-10-18T09:03:23.308489+0000","last_clean":"2023-10-18T09:03:23.308489+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:03:23.308489+0000","last_undegraded":"2023-10-18T09:03:23.308489+0000","last_fullsized":"2023-10-18T09:03:23.308489+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1361,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193951,"statfs":{"total":200000000,"available":198606297,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.135,"15min":1.038},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.579,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.109,"15min":1.046},"min":{"1min":0.64500000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.421,"5min":1.97,"15min":1.97},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1819999999999999,"15min":1.155},"min":{"1min":0.83299999999999996,"5min":0.70099999999999996,"15min":0.61799999999999999},"max":{"1min":1.595,"5min":1.829,"15min":1.829},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.1240000000000001,"15min":1.1259999999999999},"min":{"1min":0.72799999999999998,"5min":0.61099999999999999,"15min":0.58799999999999997},"max":{"1min":1.4239999999999999,"5min":1.657,"15min":1.7989999999999999},"last":0.73399999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3220000000000001,"15min":1.264},"min":{"1min":0.94699999999999995,"5min":0.78600000000000003,"15min":0.65400000000000003},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.9730000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.206,"5min":1.27,"15min":1.2190000000000001},"min":{"1min":0.79000000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.601,"5min":1.863,"15min":1.863},"last":1.121}]},{"osd":3,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2829999999999999,"15min":1.27},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.5509999999999999,"5min":1.98,"15min":1.98},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3740000000000001,"15min":1.3340000000000001},"min":{"1min":0.97299999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7210000000000001,"5min":1.756,"15min":1.9930000000000001},"last":0.83199999999999996}]},{"osd":4,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2629999999999999,"15min":1.1990000000000001},"min":{"1min":0.69899999999999995,"5min":0.66000000000000003,"15min":0.60499999999999998},"max":{"1min":1.802,"5min":1.923,"15min":2.1070000000000002},"last":1.012},{"interface":"front","average":{"1min":1.363,"5min":1.341,"15min":1.377},"min":{"1min":0.94699999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.768,"5min":2.0369999999999999,"15min":2.5169999999999999},"last":1.5209999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.349,"15min":1.3520000000000001},"min":{"1min":1.093,"5min":0.622,"15min":0.622},"max":{"1min":1.599,"5min":1.9199999999999999,"15min":2.0720000000000001},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.444,"15min":1.423},"min":{"1min":1.119,"5min":0.748,"15min":0.67900000000000005},"max":{"1min":1.77,"5min":2.0259999999999998,"15min":2.0270000000000001},"last":1.5609999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.506,"15min":1.4750000000000001},"min":{"1min":1.274,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.8440000000000001,"5min":2.0779999999999998,"15min":2.3959999999999999},"last":1.472},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"min":{"1min":1.1899999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.873,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.595}]}]},{"osd":6,"up_from":13,"seq":55834574950,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":777,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194535,"statfs":{"total":200000000,"available":199204056,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.252,"15min":1.141},"min":{"1min":0.83499999999999996,"5min":0.64700000000000002,"15min":0.60299999999999998},"max":{"1min":1.5940000000000001,"5min":1.627,"15min":1.6850000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.357,"15min":1.274},"min":{"1min":0.81799999999999995,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8149999999999999,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.5640000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.101,"15min":1.091},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63800000000000001},"max":{"1min":1.5269999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.119},{"interface":"front","average":{"1min":1.145,"5min":1.2230000000000001,"15min":1.198},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5940000000000001,"5min":1.8560000000000001,"15min":2.9609999999999999},"last":1.04}]},{"osd":2,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.169,"15min":1.1699999999999999},"min":{"1min":0.84899999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4850000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.236,"15min":1.2090000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.66300000000000003},"max":{"1min":1.627,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.1919999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3009999999999999,"15min":1.2709999999999999},"min":{"1min":0.98299999999999998,"5min":0.79400000000000004,"15min":0.79200000000000004},"max":{"1min":1.627,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.286,"15min":1.2430000000000001},"min":{"1min":0.84999999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7889999999999999},"last":1.27}]},{"osd":4,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.254,"15min":1.208},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.706,"5min":2.444,"15min":2.444},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.282},"min":{"1min":0.91700000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7749999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.0720000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.452,"15min":1.3839999999999999},"min":{"1min":1.0309999999999999,"5min":0.625,"15min":0.58899999999999997},"max":{"1min":1.722,"5min":2.032,"15min":2.032},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3220000000000001,"15min":1.256},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.49199999999999999},"max":{"1min":1.679,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.387}]},{"osd":7,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.4099999999999999,"15min":1.4019999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.768,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.347},{"interface":"front","average":{"1min":1.548,"5min":1.5429999999999999,"15min":1.5289999999999999},"min":{"1min":1.087,"5min":0.83099999999999996,"15min":0.68500000000000005},"max":{"1min":1.982,"5min":2.3140000000000001,"15min":2.794},"last":1.226}]}]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":771,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194541,"statfs":{"total":200000000,"available":199210331,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.3400000000000001,"15min":1.278},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.63400000000000001},"max":{"1min":1.8819999999999999,"5min":9.609,"15min":9.609},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.286,"15min":1.3},"min":{"1min":0.754,"5min":0.751,"15min":0.60799999999999998},"max":{"1min":1.7090000000000001,"5min":1.8260000000000001,"15min":5.3029999999999999},"last":1.5820000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.3169999999999999,"15min":1.252},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6439999999999999,"5min":11.68,"15min":11.68},"last":1.5329999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.1819999999999999,"15min":1.2210000000000001},"min":{"1min":0.70099999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3360000000000001,"5min":2.2410000000000001,"15min":3.7050000000000001},"last":1.175}]},{"osd":2,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2589999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.755},"max":{"1min":1.7030000000000001,"5min":2.0350000000000001,"15min":2.2090000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2450000000000001,"15min":1.276},"min":{"1min":0.94399999999999995,"5min":0.75900000000000001,"15min":0.72399999999999998},"max":{"1min":1.6910000000000001,"5min":2.153,"15min":3.8180000000000001},"last":1.694}]},{"osd":3,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.3200000000000001,"15min":1.272},"min":{"1min":0.72299999999999998,"5min":0.68799999999999994,"15min":0.65400000000000003},"max":{"1min":1.4179999999999999,"5min":11.797000000000001,"15min":11.797000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.454,"5min":1.3740000000000001,"15min":1.347},"min":{"1min":1.135,"5min":0.747,"15min":0.747},"max":{"1min":1.798,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.2769999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.389,"15min":1.4079999999999999},"min":{"1min":1.157,"5min":0.89900000000000002,"15min":0.75900000000000001},"max":{"1min":1.8100000000000001,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.252,"15min":1.4330000000000001},"min":{"1min":0.88700000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.629,"5min":2.4049999999999998,"15min":5.4580000000000002},"last":1.4079999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2649999999999999,"15min":1.357},"min":{"1min":0.84499999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.5780000000000001,"5min":2.5190000000000001,"15min":5.141},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.629,"5min":1.498,"15min":1.552},"min":{"1min":1.127,"5min":0.82499999999999996,"15min":0.71099999999999997},"max":{"1min":2.0350000000000001,"5min":2.657,"15min":2.657},"last":1.6479999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.5469999999999999,"15min":1.6240000000000001},"min":{"1min":1.2130000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6970000000000001,"5min":11.829000000000001,"15min":11.829000000000001},"last":1.75},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.405,"15min":1.4830000000000001},"min":{"1min":1.0920000000000001,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.837,"5min":2.9420000000000002,"15min":4.8419999999999996},"last":1.492}]}]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":770,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194542,"statfs":{"total":200000000,"available":199211350,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.1479999999999999,"15min":1.1220000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5680000000000001,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.071,"5min":1.1619999999999999,"15min":1.1599999999999999},"min":{"1min":0.71099999999999997,"5min":0.63300000000000001,"15min":0.60099999999999998},"max":{"1min":1.6719999999999999,"5min":3.234,"15min":3.234},"last":1.05}]},{"osd":1,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.1419999999999999,"15min":1.1100000000000001},"min":{"1min":0.64700000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.484,"5min":3.718,"15min":3.718},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.2,"15min":1.224},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.7529999999999999,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.419}]},{"osd":2,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.218,"15min":1.1930000000000001},"min":{"1min":0.71799999999999997,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4099999999999999,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":1.331},{"interface":"front","average":{"1min":1.24,"5min":1.3380000000000001,"15min":1.3089999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.58899999999999997},"max":{"1min":1.6870000000000001,"5min":3.532,"15min":3.532},"last":1.6739999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2709999999999999,"15min":1.2889999999999999},"min":{"1min":0.78200000000000003,"5min":0.71899999999999997,"15min":0.67000000000000004},"max":{"1min":1.5700000000000001,"5min":3.363,"15min":3.363},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.399,"15min":1.409},"min":{"1min":0.84999999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.603,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":1.2190000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.274,"15min":1.2969999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.498},"max":{"1min":1.7909999999999999,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.296,"15min":1.341},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.38400000000000001},"max":{"1min":1.571,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":1.1759999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.3720000000000001,"15min":1.3939999999999999},"min":{"1min":0.78300000000000003,"5min":0.64300000000000002,"15min":0.51600000000000001},"max":{"1min":1.478,"5min":4.4429999999999996,"15min":5.0049999999999999},"last":1.726},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4510000000000001,"15min":1.532},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.45400000000000001},"max":{"1min":1.706,"5min":4.1239999999999997,"15min":4.407},"last":1.756}]},{"osd":7,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.4330000000000001,"15min":1.484},"min":{"1min":0.85599999999999998,"5min":0.82499999999999996,"15min":0.438},"max":{"1min":1.7410000000000001,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.536,"15min":1.546},"min":{"1min":0.92000000000000004,"5min":0.78700000000000003,"15min":0.747},"max":{"1min":1.7410000000000001,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":1.966}]}]},{"osd":3,"up_from":13,"seq":55834574952,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":785,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194526,"statfs":{"total":200000000,"available":199195599,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":0.94999999999999996,"15min":0.84599999999999997},"min":{"1min":0.63800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":4.774,"5min":4.774,"15min":5.5730000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.96299999999999997,"15min":0.91600000000000004},"min":{"1min":0.53600000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.2370000000000001,"5min":3.2370000000000001,"15min":6.5599999999999996},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.0049999999999999,"15min":0.96899999999999997},"min":{"1min":0.55200000000000005,"5min":0.39100000000000001,"15min":0.36799999999999999},"max":{"1min":1.4790000000000001,"5min":1.839,"15min":6.8099999999999996},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2629999999999999,"15min":1.1679999999999999},"min":{"1min":0.85799999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6659999999999999,"5min":2.0720000000000001,"15min":3.1920000000000002},"last":1.0800000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.1060000000000001,"15min":1.052},"min":{"1min":0.62,"5min":0.46800000000000003,"15min":0.45300000000000001},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":7.4580000000000002},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.456,"5min":1.2150000000000001,"15min":1.117},"min":{"1min":0.78700000000000003,"5min":0.53200000000000003,"15min":0.51400000000000001},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":0.77800000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.327,"15min":1.339},"min":{"1min":0.82099999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":18.664999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3009999999999999,"15min":1.2110000000000001},"min":{"1min":0.92700000000000005,"5min":0.877,"15min":0.64000000000000001},"max":{"1min":2.0230000000000001,"5min":2.093,"15min":7.0700000000000003},"last":0.96799999999999997}]},{"osd":5,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.333,"15min":1.266},"min":{"1min":0.90500000000000003,"5min":0.77600000000000002,"15min":0.67700000000000005},"max":{"1min":1.9099999999999999,"5min":2.1680000000000001,"15min":6.9139999999999997},"last":1.252},{"interface":"front","average":{"1min":1.6930000000000001,"5min":1.4650000000000001,"15min":1.409},"min":{"1min":1.1140000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":4.5540000000000003,"5min":4.5540000000000003,"15min":16.390999999999998},"last":1.1870000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.4850000000000001,"15min":1.4510000000000001},"min":{"1min":0.92200000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.375,"5min":3.375,"15min":7.7270000000000003},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.494,"15min":1.3959999999999999},"min":{"1min":1.0189999999999999,"5min":0.872,"15min":0.60199999999999998},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":7.242},"last":1.3799999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.821,"5min":1.5660000000000001,"15min":1.4219999999999999},"min":{"1min":1.226,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":4.8630000000000004,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.8089999999999999,"5min":1.5589999999999999,"15min":1.502},"min":{"1min":0.98399999999999999,"5min":0.622,"15min":0.622},"max":{"1min":5.4900000000000002,"5min":5.4900000000000002,"15min":12.946},"last":1.286}]}]},{"osd":2,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":777,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194535,"statfs":{"total":200000000,"available":199204056,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.88200000000000001,"15min":0.91300000000000003},"min":{"1min":0.56000000000000005,"5min":0.48599999999999999,"15min":0.36899999999999999},"max":{"1min":1.3300000000000001,"5min":1.706,"15min":4.2210000000000001},"last":1.046},{"interface":"front","average":{"1min":1.012,"5min":1.042,"15min":1.04},"min":{"1min":0.59299999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.617,"5min":4.3959999999999999,"15min":4.3959999999999999},"last":0.88700000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.1359999999999999,"15min":1.077},"min":{"1min":0.65300000000000002,"5min":0.55900000000000005,"15min":0.49099999999999999},"max":{"1min":1.522,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.181,"15min":1.121},"min":{"1min":0.77000000000000002,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":1.552,"5min":2.448,"15min":2.5350000000000001},"last":0.94899999999999995}]},{"osd":3,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.194,"15min":1.141},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.48199999999999998},"max":{"1min":1.7569999999999999,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.365,"15min":1.427},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.54100000000000004},"max":{"1min":1.728,"5min":2.2829999999999999,"15min":7.5570000000000004},"last":1.502}]},{"osd":4,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4530000000000001,"15min":1.4730000000000001},"min":{"1min":1.0289999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.879,"5min":2.4119999999999999,"15min":7.6189999999999998},"last":1.167},{"interface":"front","average":{"1min":1.371,"5min":1.4910000000000001,"15min":1.458},"min":{"1min":0.98599999999999999,"5min":0.98099999999999998,"15min":0.86599999999999999},"max":{"1min":1.7,"5min":3.423,"15min":3.423},"last":1.4199999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.407,"15min":1.3089999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.61599999999999999},"max":{"1min":1.6000000000000001,"5min":10.507,"15min":10.507},"last":1.7869999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.4910000000000001,"15min":1.5349999999999999},"min":{"1min":1.1020000000000001,"5min":0.88100000000000001,"15min":0.81399999999999995},"max":{"1min":1.9550000000000001,"5min":2.536,"15min":7.657},"last":1.3260000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.52,"15min":1.518},"min":{"1min":1.2210000000000001,"5min":1.0289999999999999,"15min":0.74299999999999999},"max":{"1min":1.9099999999999999,"5min":2.8780000000000001,"15min":3.5299999999999998},"last":1.466},{"interface":"front","average":{"1min":1.399,"5min":1.5640000000000001,"15min":1.611},"min":{"1min":0.96499999999999997,"5min":0.85899999999999999,"15min":0.77600000000000002},"max":{"1min":2.0190000000000001,"5min":10.561,"15min":37.771999999999998},"last":1.6619999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5740000000000001},"min":{"1min":1.26,"5min":0.97599999999999998,"15min":0.83199999999999996},"max":{"1min":2.2519999999999998,"5min":2.7890000000000001,"15min":3.3620000000000001},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.6339999999999999,"15min":1.5209999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.77600000000000002},"max":{"1min":2.1059999999999999,"5min":10.956,"15min":10.956},"last":1.825}]}]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":776,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194536,"statfs":{"total":200000000,"available":199205075,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.93899999999999995,"15min":0.84399999999999997},"min":{"1min":0.42799999999999999,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.331},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.091,"15min":0.95799999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.645,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.1220000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.0489999999999999,"15min":0.96299999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.589,"5min":1.6899999999999999,"15min":1.859},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.99299999999999999,"15min":0.88700000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.36699999999999999},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.702}]},{"osd":3,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.139,"15min":1.0469999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.436},"max":{"1min":1.915,"5min":1.915,"15min":2.1059999999999999},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0960000000000001,"15min":1.0589999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.7110000000000001,"5min":1.9370000000000001,"15min":3.7010000000000001},"last":1.399}]},{"osd":4,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.3460000000000001,"15min":1.27},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7430000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.3799999999999999,"15min":1.256},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.829,"5min":1.903,"15min":2.0470000000000002},"last":1.7989999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4199999999999999,"15min":1.325},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.984,"5min":2.161,"15min":3.5750000000000002},"last":1.9490000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.4890000000000001,"15min":1.3919999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8320000000000001,"5min":1.964,"15min":2.3769999999999998},"last":1.6559999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.454,"15min":1.3260000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.905,"5min":2.0419999999999998,"15min":2.2050000000000001},"last":1.905},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.452,"15min":1.3149999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.63100000000000001},"max":{"1min":2.1040000000000001,"5min":2.121,"15min":2.121},"last":1.601}]},{"osd":7,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.524,"15min":1.399},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.2000000000000002,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.7549999999999999},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5329999999999999,"15min":1.4039999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.2469999999999999,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.839}]}]},{"osd":0,"up_from":13,"seq":55834574950,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1352,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193959,"statfs":{"total":200000000,"available":198614707,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81899999999999995,"15min":0.89000000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":1.3320000000000001,"15min":3.4420000000000002},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.083,"15min":1.0780000000000001},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.46500000000000002},"max":{"1min":1.3200000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.2130000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.85699999999999998,"15min":0.84199999999999997},"min":{"1min":0.44400000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.244,"5min":1.679,"15min":1.679},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.071,"5min":1.1359999999999999,"15min":1.1279999999999999},"min":{"1min":0.83999999999999997,"5min":0.503,"15min":0.437},"max":{"1min":1.446,"5min":1.929,"15min":1.929},"last":0.97499999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.135,"15min":1.234},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4419999999999999,"5min":1.98,"15min":2.7639999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3799999999999999,"15min":1.4710000000000001},"min":{"1min":0.71999999999999997,"5min":0.61299999999999999,"15min":0.58399999999999996},"max":{"1min":1.677,"5min":2.3639999999999999,"15min":3.5720000000000001},"last":1.3100000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.228,"15min":1.278},"min":{"1min":0.85999999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7410000000000001,"5min":1.8819999999999999,"15min":4.492},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.282,"15min":1.2589999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.518,"5min":1.927,"15min":1.927},"last":1.1719999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.256,"15min":1.373},"min":{"1min":0.83399999999999996,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.839,"5min":2.4510000000000001,"15min":4.2779999999999996},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.3049999999999999,"15min":1.3540000000000001},"min":{"1min":0.751,"5min":0.74399999999999999,"15min":0.73099999999999998},"max":{"1min":1.573,"5min":1.847,"15min":1.8740000000000001},"last":1.573}]},{"osd":6,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.4179999999999999,"15min":1.538},"min":{"1min":0.79900000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.613,"5min":2.0419999999999998,"15min":3.5299999999999998},"last":1.369},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.363,"15min":1.458},"min":{"1min":0.78000000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.665,"5min":3.1440000000000001,"15min":4.4349999999999996},"last":1.5549999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.524,"15min":1.6080000000000001},"min":{"1min":0.88,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.8120000000000001,"5min":2.2000000000000002,"15min":3.492},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.446,"15min":1.6080000000000001},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1869999999999998,"5min":3.2570000000000001,"15min":4.5419999999999998},"last":1.385}]}]}],"pool_statfs":[]}} 2023-10-18T09:03:29.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3f is not active+clean 2023-10-18T09:03:29.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.313192+0000', 'last_change': '2023-10-18T09:03:23.313192+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.313192+0000', 'last_undegraded': '2023-10-18T09:03:23.313192+0000', 'last_fullsized': '2023-10-18T09:03:23.313192+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T09:03:29.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3e is not active+clean 2023-10-18T09:03:29.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.322631+0000', 'last_change': '2023-10-18T09:03:23.322631+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.322631+0000', 'last_undegraded': '2023-10-18T09:03:23.322631+0000', 'last_fullsized': '2023-10-18T09:03:23.322631+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T09:03:29.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3c is not active+clean 2023-10-18T09:03:29.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.325742+0000', 'last_change': '2023-10-18T09:03:23.325742+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.325742+0000', 'last_undegraded': '2023-10-18T09:03:23.325742+0000', 'last_fullsized': '2023-10-18T09:03:23.325742+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T09:03:29.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3b is not active+clean 2023-10-18T09:03:29.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.313411+0000', 'last_change': '2023-10-18T09:03:23.313411+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.313411+0000', 'last_undegraded': '2023-10-18T09:03:23.313411+0000', 'last_fullsized': '2023-10-18T09:03:23.313411+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T09:03:29.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3a is not active+clean 2023-10-18T09:03:29.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.319674+0000', 'last_change': '2023-10-18T09:03:23.319674+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.319674+0000', 'last_undegraded': '2023-10-18T09:03:23.319674+0000', 'last_fullsized': '2023-10-18T09:03:23.319674+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T09:03:29.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.39 is not active+clean 2023-10-18T09:03:29.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.327137+0000', 'last_change': '2023-10-18T09:03:23.327137+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.327137+0000', 'last_undegraded': '2023-10-18T09:03:23.327137+0000', 'last_fullsized': '2023-10-18T09:03:23.327137+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T09:03:29.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.37 is not active+clean 2023-10-18T09:03:29.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.325288+0000', 'last_change': '2023-10-18T09:03:23.325288+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.325288+0000', 'last_undegraded': '2023-10-18T09:03:23.325288+0000', 'last_fullsized': '2023-10-18T09:03:23.325288+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T09:03:29.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.36 is not active+clean 2023-10-18T09:03:29.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.317783+0000', 'last_change': '2023-10-18T09:03:23.317783+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.317783+0000', 'last_undegraded': '2023-10-18T09:03:23.317783+0000', 'last_fullsized': '2023-10-18T09:03:23.317783+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T09:03:29.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.35 is not active+clean 2023-10-18T09:03:29.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.316307+0000', 'last_change': '2023-10-18T09:03:23.316307+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.316307+0000', 'last_undegraded': '2023-10-18T09:03:23.316307+0000', 'last_fullsized': '2023-10-18T09:03:23.316307+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T09:03:29.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.34 is not active+clean 2023-10-18T09:03:29.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.321900+0000', 'last_change': '2023-10-18T09:03:23.321900+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.321900+0000', 'last_undegraded': '2023-10-18T09:03:23.321900+0000', 'last_fullsized': '2023-10-18T09:03:23.321900+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T09:03:29.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.31 is not active+clean 2023-10-18T09:03:29.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.321363+0000', 'last_change': '2023-10-18T09:03:23.321363+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.321363+0000', 'last_undegraded': '2023-10-18T09:03:23.321363+0000', 'last_fullsized': '2023-10-18T09:03:23.321363+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T09:03:29.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.30 is not active+clean 2023-10-18T09:03:29.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.312843+0000', 'last_change': '2023-10-18T09:03:23.312843+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.312843+0000', 'last_undegraded': '2023-10-18T09:03:23.312843+0000', 'last_fullsized': '2023-10-18T09:03:23.312843+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T09:03:29.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2f is not active+clean 2023-10-18T09:03:29.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.322169+0000', 'last_change': '2023-10-18T09:03:23.322169+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.322169+0000', 'last_undegraded': '2023-10-18T09:03:23.322169+0000', 'last_fullsized': '2023-10-18T09:03:23.322169+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T09:03:29.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2e is not active+clean 2023-10-18T09:03:29.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.321038+0000', 'last_change': '2023-10-18T09:03:23.321038+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.321038+0000', 'last_undegraded': '2023-10-18T09:03:23.321038+0000', 'last_fullsized': '2023-10-18T09:03:23.321038+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T09:03:29.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2c is not active+clean 2023-10-18T09:03:29.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.313740+0000', 'last_change': '2023-10-18T09:03:23.313740+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.313740+0000', 'last_undegraded': '2023-10-18T09:03:23.313740+0000', 'last_fullsized': '2023-10-18T09:03:23.313740+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T09:03:29.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2b is not active+clean 2023-10-18T09:03:29.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.315443+0000', 'last_change': '2023-10-18T09:03:23.315443+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.315443+0000', 'last_undegraded': '2023-10-18T09:03:23.315443+0000', 'last_fullsized': '2023-10-18T09:03:23.315443+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T09:03:29.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.29 is not active+clean 2023-10-18T09:03:29.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.312725+0000', 'last_change': '2023-10-18T09:03:23.312725+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.312725+0000', 'last_undegraded': '2023-10-18T09:03:23.312725+0000', 'last_fullsized': '2023-10-18T09:03:23.312725+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T09:03:29.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.27 is not active+clean 2023-10-18T09:03:29.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.312889+0000', 'last_change': '2023-10-18T09:03:23.312889+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.312889+0000', 'last_undegraded': '2023-10-18T09:03:23.312889+0000', 'last_fullsized': '2023-10-18T09:03:23.312889+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T09:03:29.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.26 is not active+clean 2023-10-18T09:03:29.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.312482+0000', 'last_change': '2023-10-18T09:03:23.312482+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.312482+0000', 'last_undegraded': '2023-10-18T09:03:23.312482+0000', 'last_fullsized': '2023-10-18T09:03:23.312482+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T09:03:29.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.25 is not active+clean 2023-10-18T09:03:29.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.318515+0000', 'last_change': '2023-10-18T09:03:23.318515+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.318515+0000', 'last_undegraded': '2023-10-18T09:03:23.318515+0000', 'last_fullsized': '2023-10-18T09:03:23.318515+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T09:03:29.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.23 is not active+clean 2023-10-18T09:03:29.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.317757+0000', 'last_change': '2023-10-18T09:03:23.317757+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.317757+0000', 'last_undegraded': '2023-10-18T09:03:23.317757+0000', 'last_fullsized': '2023-10-18T09:03:23.317757+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T09:03:29.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.22 is not active+clean 2023-10-18T09:03:29.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.326266+0000', 'last_change': '2023-10-18T09:03:23.326266+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.326266+0000', 'last_undegraded': '2023-10-18T09:03:23.326266+0000', 'last_fullsized': '2023-10-18T09:03:23.326266+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T09:03:29.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.21 is not active+clean 2023-10-18T09:03:29.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.312103+0000', 'last_change': '2023-10-18T09:03:23.312103+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.312103+0000', 'last_undegraded': '2023-10-18T09:03:23.312103+0000', 'last_fullsized': '2023-10-18T09:03:23.312103+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T09:03:29.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.20 is not active+clean 2023-10-18T09:03:29.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.318051+0000', 'last_change': '2023-10-18T09:03:23.318051+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.318051+0000', 'last_undegraded': '2023-10-18T09:03:23.318051+0000', 'last_fullsized': '2023-10-18T09:03:23.318051+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T09:03:29.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1e is not active+clean 2023-10-18T09:03:29.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.320206+0000', 'last_change': '2023-10-18T09:03:23.320206+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.320206+0000', 'last_undegraded': '2023-10-18T09:03:23.320206+0000', 'last_fullsized': '2023-10-18T09:03:23.320206+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T09:03:29.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1f is not active+clean 2023-10-18T09:03:29.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.318339+0000', 'last_change': '2023-10-18T09:03:23.318339+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.318339+0000', 'last_undegraded': '2023-10-18T09:03:23.318339+0000', 'last_fullsized': '2023-10-18T09:03:23.318339+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T09:03:29.992 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1c is not active+clean 2023-10-18T09:03:29.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.319346+0000', 'last_change': '2023-10-18T09:03:23.319346+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.319346+0000', 'last_undegraded': '2023-10-18T09:03:23.319346+0000', 'last_fullsized': '2023-10-18T09:03:23.319346+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T09:03:29.993 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1a is not active+clean 2023-10-18T09:03:29.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.320552+0000', 'last_change': '2023-10-18T09:03:23.320552+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.320552+0000', 'last_undegraded': '2023-10-18T09:03:23.320552+0000', 'last_fullsized': '2023-10-18T09:03:23.320552+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T09:03:29.993 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1b is not active+clean 2023-10-18T09:03:29.994 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.321572+0000', 'last_change': '2023-10-18T09:03:23.321572+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.321572+0000', 'last_undegraded': '2023-10-18T09:03:23.321572+0000', 'last_fullsized': '2023-10-18T09:03:23.321572+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T09:03:29.994 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.18 is not active+clean 2023-10-18T09:03:29.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.316958+0000', 'last_change': '2023-10-18T09:03:23.316958+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.316958+0000', 'last_undegraded': '2023-10-18T09:03:23.316958+0000', 'last_fullsized': '2023-10-18T09:03:23.316958+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T09:03:29.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.19 is not active+clean 2023-10-18T09:03:29.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.322477+0000', 'last_change': '2023-10-18T09:03:23.322477+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.322477+0000', 'last_undegraded': '2023-10-18T09:03:23.322477+0000', 'last_fullsized': '2023-10-18T09:03:23.322477+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T09:03:29.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.16 is not active+clean 2023-10-18T09:03:29.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.324949+0000', 'last_change': '2023-10-18T09:03:23.324949+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.324949+0000', 'last_undegraded': '2023-10-18T09:03:23.324949+0000', 'last_fullsized': '2023-10-18T09:03:23.324949+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T09:03:29.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.17 is not active+clean 2023-10-18T09:03:29.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.321318+0000', 'last_change': '2023-10-18T09:03:23.321318+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.321318+0000', 'last_undegraded': '2023-10-18T09:03:23.321318+0000', 'last_fullsized': '2023-10-18T09:03:23.321318+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T09:03:29.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.15 is not active+clean 2023-10-18T09:03:29.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.326069+0000', 'last_change': '2023-10-18T09:03:23.326069+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.326069+0000', 'last_undegraded': '2023-10-18T09:03:23.326069+0000', 'last_fullsized': '2023-10-18T09:03:23.326069+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-18T09:03:29.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.e is not active+clean 2023-10-18T09:03:29.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.313059+0000', 'last_change': '2023-10-18T09:03:23.313059+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.313059+0000', 'last_undegraded': '2023-10-18T09:03:23.313059+0000', 'last_fullsized': '2023-10-18T09:03:23.313059+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-18T09:03:29.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.f is not active+clean 2023-10-18T09:03:29.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.319843+0000', 'last_change': '2023-10-18T09:03:23.319843+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.319843+0000', 'last_undegraded': '2023-10-18T09:03:23.319843+0000', 'last_fullsized': '2023-10-18T09:03:23.319843+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-18T09:03:29.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.b is not active+clean 2023-10-18T09:03:30.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.313248+0000', 'last_change': '2023-10-18T09:03:23.313248+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.313248+0000', 'last_undegraded': '2023-10-18T09:03:23.313248+0000', 'last_fullsized': '2023-10-18T09:03:23.313248+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T09:03:30.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.6 is not active+clean 2023-10-18T09:03:30.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.317564+0000', 'last_change': '2023-10-18T09:03:23.317564+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.317564+0000', 'last_undegraded': '2023-10-18T09:03:23.317564+0000', 'last_fullsized': '2023-10-18T09:03:23.317564+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T09:03:30.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.7 is not active+clean 2023-10-18T09:03:30.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.315057+0000', 'last_change': '2023-10-18T09:03:23.315057+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.315057+0000', 'last_undegraded': '2023-10-18T09:03:23.315057+0000', 'last_fullsized': '2023-10-18T09:03:23.315057+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T09:03:30.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.4 is not active+clean 2023-10-18T09:03:30.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.313161+0000', 'last_change': '2023-10-18T09:03:23.313161+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.313161+0000', 'last_undegraded': '2023-10-18T09:03:23.313161+0000', 'last_fullsized': '2023-10-18T09:03:23.313161+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T09:03:30.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.5 is not active+clean 2023-10-18T09:03:30.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.313904+0000', 'last_change': '2023-10-18T09:03:23.313904+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.313904+0000', 'last_undegraded': '2023-10-18T09:03:23.313904+0000', 'last_fullsized': '2023-10-18T09:03:23.313904+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-18T09:03:30.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2 is not active+clean 2023-10-18T09:03:30.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.318651+0000', 'last_change': '2023-10-18T09:03:23.318651+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.318651+0000', 'last_undegraded': '2023-10-18T09:03:23.318651+0000', 'last_fullsized': '2023-10-18T09:03:23.318651+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T09:03:30.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3 is not active+clean 2023-10-18T09:03:30.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.314648+0000', 'last_change': '2023-10-18T09:03:23.314648+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.314648+0000', 'last_undegraded': '2023-10-18T09:03:23.314648+0000', 'last_fullsized': '2023-10-18T09:03:23.314648+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-18T09:03:30.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1 is not active+clean 2023-10-18T09:03:30.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 142, 'state': 'creating+peering', 'last_fresh': '2023-10-18T09:03:23.319950+0000', 'last_change': '2023-10-18T09:03:23.319950+0000', 'last_active': '2023-10-18T09:03:23.295746+0000', 'last_peered': '2023-10-18T09:03:23.295746+0000', 'last_clean': '2023-10-18T09:03:23.295746+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-18T09:03:23.319950+0000', 'last_undegraded': '2023-10-18T09:03:23.319950+0000', 'last_fullsized': '2023-10-18T09:03:23.319950+0000', 'mapping_epoch': 142, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 142, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'last_clean_scrub_stamp': '2023-10-18T09:03:23.295746+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-18T09:03:30.006 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-18T09:03:30.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.243+0000 7fd49e475640 1 -- 172.21.15.40:0/226219905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd49806d400 msgr2=0x7fd4980708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:30.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.243+0000 7fd49e475640 1 --2- 172.21.15.40:0/226219905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd49806d400 0x7fd4980708e0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd4980700f0 tx=0x7fd49002bd30 comp rx=0 tx=0).stop 2023-10-18T09:03:30.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.244+0000 7fd49e475640 1 -- 172.21.15.40:0/226219905 shutdown_connections 2023-10-18T09:03:30.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.244+0000 7fd49e475640 1 --2- 172.21.15.40:0/226219905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd49806d400 0x7fd4980708e0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:30.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.244+0000 7fd49e475640 1 --2- 172.21.15.40:0/226219905 >> v2:172.21.15.144:3300/0 conn(0x7fd498069e30 0x7fd49806cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:30.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.244+0000 7fd49e475640 1 -- 172.21.15.40:0/226219905 >> 172.21.15.40:0/226219905 conn(0x7fd49806a830 msgr2=0x7fd49806ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:30.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.244+0000 7fd49e475640 1 -- 172.21.15.40:0/226219905 shutdown_connections 2023-10-18T09:03:30.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.244+0000 7fd49e475640 1 -- 172.21.15.40:0/226219905 wait complete. 2023-10-18T09:03:30.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.246+0000 7fd49e475640 1 Processor -- start 2023-10-18T09:03:30.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.246+0000 7fd49e475640 1 -- start start 2023-10-18T09:03:30.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.247+0000 7fd49e475640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd498069e30 0x7fd4981f1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:30.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.247+0000 7fd49e475640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd49806d400 0x7fd49815dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:30.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.247+0000 7fd49e475640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd498155bc0 0x7fd49814fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:30.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.247+0000 7fd49e475640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49814ebf0 con 0x7fd498069e30 2023-10-18T09:03:30.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.247+0000 7fd49e475640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4980aeea0 con 0x7fd498155bc0 2023-10-18T09:03:30.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.247+0000 7fd49e475640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49814ee40 con 0x7fd49806d400 2023-10-18T09:03:30.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.248+0000 7fd497fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd498069e30 0x7fd4981f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:30.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.248+0000 7fd497fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd498069e30 0x7fd4981f1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49720/0 (socket says 172.21.15.40:49720) 2023-10-18T09:03:30.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.248+0000 7fd497fff640 1 -- 172.21.15.40:0/1392431809 learned_addr learned my addr 172.21.15.40:0/1392431809 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:30.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.248+0000 7fd49ca12640 1 --2- 172.21.15.40:0/1392431809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd498155bc0 0x7fd49814fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:30.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.248+0000 7fd497fff640 1 -- 172.21.15.40:0/1392431809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd49806d400 msgr2=0x7fd49815dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:30.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.248+0000 7fd497fff640 1 --2- 172.21.15.40:0/1392431809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd49806d400 0x7fd49815dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:30.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.248+0000 7fd497fff640 1 -- 172.21.15.40:0/1392431809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd498155bc0 msgr2=0x7fd49814fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:30.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.248+0000 7fd497fff640 1 --2- 172.21.15.40:0/1392431809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd498155bc0 0x7fd49814fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.248+0000 7fd497fff640 1 -- 172.21.15.40:0/1392431809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd490009560 con 0x7fd498069e30 2023-10-18T09:03:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.249+0000 7fd497fff640 1 --2- 172.21.15.40:0/1392431809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd498069e30 0x7fd4981f1630 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd48800ba50 tx=0x7fd48800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.249+0000 7fd494ff9640 1 -- 172.21.15.40:0/1392431809 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48800cc10 con 0x7fd498069e30 2023-10-18T09:03:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.249+0000 7fd494ff9640 1 -- 172.21.15.40:0/1392431809 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd48800ce60 con 0x7fd498069e30 2023-10-18T09:03:30.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.249+0000 7fd494ff9640 1 -- 172.21.15.40:0/1392431809 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488014680 con 0x7fd498069e30 2023-10-18T09:03:30.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.249+0000 7fd49e475640 1 -- 172.21.15.40:0/1392431809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49814f120 con 0x7fd498069e30 2023-10-18T09:03:30.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.249+0000 7fd49e475640 1 -- 172.21.15.40:0/1392431809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49815e2e0 con 0x7fd498069e30 2023-10-18T09:03:30.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.250+0000 7fd49e475640 1 -- 172.21.15.40:0/1392431809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49806b1b0 con 0x7fd498069e30 2023-10-18T09:03:30.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.251+0000 7fd494ff9640 1 -- 172.21.15.40:0/1392431809 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd48801d3b0 con 0x7fd498069e30 2023-10-18T09:03:30.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.252+0000 7fd494ff9640 1 --2- 172.21.15.40:0/1392431809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd47806ae40 0x7fd47806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:30.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.252+0000 7fd494ff9640 1 -- 172.21.15.40:0/1392431809 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4880898a0 con 0x7fd498069e30 2023-10-18T09:03:30.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.253+0000 7fd4977fe640 1 --2- 172.21.15.40:0/1392431809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd47806ae40 0x7fd47806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:30.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.254+0000 7fd4977fe640 1 --2- 172.21.15.40:0/1392431809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd47806ae40 0x7fd47806d300 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd4981f4fd0 tx=0x7fd480005bf0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:30.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.256+0000 7fd494ff9640 1 -- 172.21.15.40:0/1392431809 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4880549e0 con 0x7fd498069e30 2023-10-18T09:03:30.302 INFO:tasks.ceph.mds.c.smithi040.stderr:2023-10-18T09:03:30.299+0000 7f1ad5eb1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T09:03:30.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.520+0000 7fd49e475640 1 -- 172.21.15.40:0/1392431809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd498066fd0 con 0x7fd498069e30 2023-10-18T09:03:30.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.521+0000 7fd494ff9640 1 -- 172.21.15.40:0/1392431809 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1251 (secure 0 0 0) 0x7fd488019080 con 0x7fd498069e30 2023-10-18T09:03:30.524 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:30.525 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":514,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":144,"num_osds":8,"num_up_osds":8,"osd_up_since":1697619304,"num_in_osds":8,"osd_in_since":1697619300,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":53},{"state_name":"creating+peering","count":44}],"num_pgs":97,"num_pools":3,"num_objects":3,"data_bytes":590368,"bytes_used":7548529,"bytes_avail":1592451471,"bytes_total":1600000000,"inactive_pgs_ratio":0.45360824465751648,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":66,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":7893}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2023-10-18T09:02:33.056080+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-18T09:03:30.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.523+0000 7fd49e475640 1 -- 172.21.15.40:0/1392431809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd47806ae40 msgr2=0x7fd47806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:30.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.523+0000 7fd49e475640 1 --2- 172.21.15.40:0/1392431809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd47806ae40 0x7fd47806d300 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd4981f4fd0 tx=0x7fd480005bf0 comp rx=0 tx=0).stop 2023-10-18T09:03:30.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.524+0000 7fd49e475640 1 -- 172.21.15.40:0/1392431809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd498069e30 msgr2=0x7fd4981f1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:30.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.524+0000 7fd49e475640 1 --2- 172.21.15.40:0/1392431809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd498069e30 0x7fd4981f1630 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd48800ba50 tx=0x7fd48800bf10 comp rx=0 tx=0).stop 2023-10-18T09:03:30.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.524+0000 7fd49e475640 1 -- 172.21.15.40:0/1392431809 shutdown_connections 2023-10-18T09:03:30.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.524+0000 7fd49e475640 1 --2- 172.21.15.40:0/1392431809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd47806ae40 0x7fd47806d300 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:30.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.524+0000 7fd49e475640 1 --2- 172.21.15.40:0/1392431809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd498155bc0 0x7fd49814fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:30.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.524+0000 7fd49e475640 1 --2- 172.21.15.40:0/1392431809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd49806d400 0x7fd49815dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:30.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.524+0000 7fd49e475640 1 --2- 172.21.15.40:0/1392431809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd498069e30 0x7fd4981f1630 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:30.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.524+0000 7fd49e475640 1 -- 172.21.15.40:0/1392431809 >> 172.21.15.40:0/1392431809 conn(0x7fd49806a830 msgr2=0x7fd498069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:30.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.524+0000 7fd49e475640 1 -- 172.21.15.40:0/1392431809 shutdown_connections 2023-10-18T09:03:30.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.524+0000 7fd49e475640 1 -- 172.21.15.40:0/1392431809 wait complete. 2023-10-18T09:03:30.548 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-18T09:03:30.549 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:03:30.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.766+0000 7f0bd25d0640 1 -- 172.21.15.40:0/1281052954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bb80039a0 msgr2=0x7f0bb8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:30.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.766+0000 7f0bd25d0640 1 --2- 172.21.15.40:0/1281052954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bb80039a0 0x7f0bb8003dc0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc00c610 tx=0x7f0bb80057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:30.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.767+0000 7f0bd0b6d640 1 -- 172.21.15.40:0/1281052954 reap_dead start 2023-10-18T09:03:30.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.767+0000 7f0bd25d0640 1 -- 172.21.15.40:0/1281052954 shutdown_connections 2023-10-18T09:03:30.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.767+0000 7f0bd25d0640 1 -- 172.21.15.40:0/1281052954 >> 172.21.15.40:0/1281052954 conn(0x7f0bcc06cee0 msgr2=0x7f0bcc069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:30.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.767+0000 7f0bd25d0640 1 -- 172.21.15.40:0/1281052954 shutdown_connections 2023-10-18T09:03:30.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.767+0000 7f0bd25d0640 1 -- 172.21.15.40:0/1281052954 wait complete. 2023-10-18T09:03:30.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.769+0000 7f0bd25d0640 1 Processor -- start 2023-10-18T09:03:30.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.769+0000 7f0bd25d0640 1 -- start start 2023-10-18T09:03:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.770+0000 7f0bd25d0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bb80039a0 0x7f0bcc13abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.770+0000 7f0bd25d0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc1f5b00 0x7f0bcc1574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.770+0000 7f0bd25d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bcc0cd270 0x7f0bcc0cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.770+0000 7f0bd25d0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0acd10 con 0x7f0bcc1f5b00 2023-10-18T09:03:30.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.770+0000 7f0bd25d0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0cdc30 con 0x7f0bcc0cd270 2023-10-18T09:03:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.770+0000 7f0bd25d0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc157f10 con 0x7f0bb80039a0 2023-10-18T09:03:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.770+0000 7f0bc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc1f5b00 0x7f0bcc1574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.770+0000 7f0bc37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc1f5b00 0x7f0bcc1574d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49744/0 (socket says 172.21.15.40:49744) 2023-10-18T09:03:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.770+0000 7f0bc37fe640 1 -- 172.21.15.40:0/3269408892 learned_addr learned my addr 172.21.15.40:0/3269408892 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.770+0000 7f0bcbfff640 1 --2- 172.21.15.40:0/3269408892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bb80039a0 0x7f0bcc13abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:30.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.770+0000 7f0bc37fe640 1 -- 172.21.15.40:0/3269408892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bb80039a0 msgr2=0x7f0bcc13abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.770+0000 7f0bc37fe640 1 --2- 172.21.15.40:0/3269408892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bb80039a0 0x7f0bcc13abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.770+0000 7f0bc37fe640 1 -- 172.21.15.40:0/3269408892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bcc0cd270 msgr2=0x7f0bcc0cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.770+0000 7f0bd0b6d640 1 --2- 172.21.15.40:0/3269408892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bcc0cd270 0x7f0bcc0cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.771+0000 7f0bc37fe640 1 --2- 172.21.15.40:0/3269408892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bcc0cd270 0x7f0bcc0cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.771+0000 7f0bc37fe640 1 -- 172.21.15.40:0/3269408892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc1581b0 con 0x7f0bcc1f5b00 2023-10-18T09:03:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.771+0000 7f0bd0b6d640 1 --2- 172.21.15.40:0/3269408892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bcc0cd270 0x7f0bcc0cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:30.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.771+0000 7f0bc37fe640 1 --2- 172.21.15.40:0/3269408892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc1f5b00 0x7f0bcc1574d0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40109a0 tx=0x7f0bb4010e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:30.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.771+0000 7f0bc97fa640 1 -- 172.21.15.40:0/3269408892 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb400eda0 con 0x7f0bcc1f5b00 2023-10-18T09:03:30.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.771+0000 7f0bc97fa640 1 -- 172.21.15.40:0/3269408892 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0bb4004030 con 0x7f0bcc1f5b00 2023-10-18T09:03:30.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.771+0000 7f0bc97fa640 1 -- 172.21.15.40:0/3269408892 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4004420 con 0x7f0bcc1f5b00 2023-10-18T09:03:30.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.771+0000 7f0bd25d0640 1 -- 172.21.15.40:0/3269408892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bcc158490 con 0x7f0bcc1f5b00 2023-10-18T09:03:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.771+0000 7f0bd25d0640 1 -- 172.21.15.40:0/3269408892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bcc157a10 con 0x7f0bcc1f5b00 2023-10-18T09:03:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.773+0000 7f0bd25d0640 1 -- 172.21.15.40:0/3269408892 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bcc0ace50 con 0x7f0bcc1f5b00 2023-10-18T09:03:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.773+0000 7f0bc97fa640 1 -- 172.21.15.40:0/3269408892 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0bb40035f0 con 0x7f0bcc1f5b00 2023-10-18T09:03:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.774+0000 7f0bc97fa640 1 --2- 172.21.15.40:0/3269408892 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0bac06ae70 0x7f0bac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.774+0000 7f0bc97fa640 1 -- 172.21.15.40:0/3269408892 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bb4016070 con 0x7f0bcc1f5b00 2023-10-18T09:03:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.775+0000 7f0bcbfff640 1 --2- 172.21.15.40:0/3269408892 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0bac06ae70 0x7f0bac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.776+0000 7f0bcbfff640 1 --2- 172.21.15.40:0/3269408892 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0bac06ae70 0x7f0bac06d330 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc034000 tx=0x7f0bbc02e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.778+0000 7f0bc97fa640 1 -- 172.21.15.40:0/3269408892 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0bcc0ace50 con 0x7f0bcc1f5b00 2023-10-18T09:03:30.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.968+0000 7f0bd25d0640 1 -- 172.21.15.40:0/3269408892 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0bcc0cbce0 con 0x7f0bac06ae70 2023-10-18T09:03:30.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.991+0000 7f0bc97fa640 1 -- 172.21.15.40:0/3269408892 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279054 (secure 0 0 0) 0x7f0bbc001930 con 0x7f0bac06ae70 2023-10-18T09:03:30.998 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:31.000 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:03:31.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.998+0000 7f0bd25d0640 1 -- 172.21.15.40:0/3269408892 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0bac06ae70 msgr2=0x7f0bac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:31.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.998+0000 7f0bd25d0640 1 --2- 172.21.15.40:0/3269408892 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0bac06ae70 0x7f0bac06d330 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc034000 tx=0x7f0bbc02e000 comp rx=0 tx=0).stop 2023-10-18T09:03:31.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.998+0000 7f0bd25d0640 1 -- 172.21.15.40:0/3269408892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc1f5b00 msgr2=0x7f0bcc1574d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:31.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.998+0000 7f0bd25d0640 1 --2- 172.21.15.40:0/3269408892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc1f5b00 0x7f0bcc1574d0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40109a0 tx=0x7f0bb4010e60 comp rx=0 tx=0).stop 2023-10-18T09:03:31.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.998+0000 7f0bd25d0640 1 -- 172.21.15.40:0/3269408892 shutdown_connections 2023-10-18T09:03:31.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.998+0000 7f0bd25d0640 1 --2- 172.21.15.40:0/3269408892 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0bac06ae70 0x7f0bac06d330 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:31.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.998+0000 7f0bd25d0640 1 --2- 172.21.15.40:0/3269408892 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bcc0cd270 0x7f0bcc0cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:31.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.998+0000 7f0bd25d0640 1 --2- 172.21.15.40:0/3269408892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bcc1f5b00 0x7f0bcc1574d0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:31.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.998+0000 7f0bd25d0640 1 --2- 172.21.15.40:0/3269408892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0bb80039a0 0x7f0bcc13abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:31.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.998+0000 7f0bd25d0640 1 -- 172.21.15.40:0/3269408892 >> 172.21.15.40:0/3269408892 conn(0x7f0bcc06cee0 msgr2=0x7f0bcc13ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:31.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.998+0000 7f0bd25d0640 1 -- 172.21.15.40:0/3269408892 shutdown_connections 2023-10-18T09:03:31.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:30.998+0000 7f0bd25d0640 1 -- 172.21.15.40:0/3269408892 wait complete. 2023-10-18T09:03:31.021 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2023-10-18T09:03:29.075882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":162,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7371,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555128,"statfs":{"total":1600000000,"available":1592451471,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001852"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313192+0000","last_change":"2023-10-18T09:03:23.313192+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313192+0000","last_undegraded":"2023-10-18T09:03:23.313192+0000","last_fullsized":"2023-10-18T09:03:23.313192+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.322631+0000","last_change":"2023-10-18T09:03:23.322631+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.322631+0000","last_undegraded":"2023-10-18T09:03:23.322631+0000","last_fullsized":"2023-10-18T09:03:23.322631+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.372965+0000","last_change":"2023-10-18T09:03:24.371528+0000","last_active":"2023-10-18T09:03:24.372965+0000","last_peered":"2023-10-18T09:03:24.372965+0000","last_clean":"2023-10-18T09:03:24.372965+0000","last_became_active":"2023-10-18T09:03:24.370994+0000","last_became_peered":"2023-10-18T09:03:24.370994+0000","last_unstale":"2023-10-18T09:03:24.372965+0000","last_undegraded":"2023-10-18T09:03:24.372965+0000","last_fullsized":"2023-10-18T09:03:24.372965+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:31.969146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.325742+0000","last_change":"2023-10-18T09:03:23.325742+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.325742+0000","last_undegraded":"2023-10-18T09:03:23.325742+0000","last_fullsized":"2023-10-18T09:03:23.325742+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313411+0000","last_change":"2023-10-18T09:03:23.313411+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313411+0000","last_undegraded":"2023-10-18T09:03:23.313411+0000","last_fullsized":"2023-10-18T09:03:23.313411+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.319674+0000","last_change":"2023-10-18T09:03:23.319674+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.319674+0000","last_undegraded":"2023-10-18T09:03:23.319674+0000","last_fullsized":"2023-10-18T09:03:23.319674+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.327137+0000","last_change":"2023-10-18T09:03:23.327137+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.327137+0000","last_undegraded":"2023-10-18T09:03:23.327137+0000","last_fullsized":"2023-10-18T09:03:23.327137+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.373894+0000","last_change":"2023-10-18T09:03:24.373509+0000","last_active":"2023-10-18T09:03:24.373894+0000","last_peered":"2023-10-18T09:03:24.373894+0000","last_clean":"2023-10-18T09:03:24.373894+0000","last_became_active":"2023-10-18T09:03:24.372325+0000","last_became_peered":"2023-10-18T09:03:24.372325+0000","last_unstale":"2023-10-18T09:03:24.373894+0000","last_undegraded":"2023-10-18T09:03:24.373894+0000","last_fullsized":"2023-10-18T09:03:24.373894+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:00.288118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.325288+0000","last_change":"2023-10-18T09:03:23.325288+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.325288+0000","last_undegraded":"2023-10-18T09:03:23.325288+0000","last_fullsized":"2023-10-18T09:03:23.325288+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.317783+0000","last_change":"2023-10-18T09:03:23.317783+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.317783+0000","last_undegraded":"2023-10-18T09:03:23.317783+0000","last_fullsized":"2023-10-18T09:03:23.317783+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.316307+0000","last_change":"2023-10-18T09:03:23.316307+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.316307+0000","last_undegraded":"2023-10-18T09:03:23.316307+0000","last_fullsized":"2023-10-18T09:03:23.316307+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321900+0000","last_change":"2023-10-18T09:03:23.321900+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321900+0000","last_undegraded":"2023-10-18T09:03:23.321900+0000","last_fullsized":"2023-10-18T09:03:23.321900+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.378701+0000","last_change":"2023-10-18T09:03:24.378634+0000","last_active":"2023-10-18T09:03:24.378701+0000","last_peered":"2023-10-18T09:03:24.378701+0000","last_clean":"2023-10-18T09:03:24.378701+0000","last_became_active":"2023-10-18T09:03:24.378364+0000","last_became_peered":"2023-10-18T09:03:24.378364+0000","last_unstale":"2023-10-18T09:03:24.378701+0000","last_undegraded":"2023-10-18T09:03:24.378701+0000","last_fullsized":"2023-10-18T09:03:24.378701+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:26:40.521285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.372987+0000","last_change":"2023-10-18T09:03:24.371034+0000","last_active":"2023-10-18T09:03:24.372987+0000","last_peered":"2023-10-18T09:03:24.372987+0000","last_clean":"2023-10-18T09:03:24.372987+0000","last_became_active":"2023-10-18T09:03:24.370294+0000","last_became_peered":"2023-10-18T09:03:24.370294+0000","last_unstale":"2023-10-18T09:03:24.372987+0000","last_undegraded":"2023-10-18T09:03:24.372987+0000","last_fullsized":"2023-10-18T09:03:24.372987+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:13:18.579341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321363+0000","last_change":"2023-10-18T09:03:23.321363+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321363+0000","last_undegraded":"2023-10-18T09:03:23.321363+0000","last_fullsized":"2023-10-18T09:03:23.321363+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312843+0000","last_change":"2023-10-18T09:03:23.312843+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312843+0000","last_undegraded":"2023-10-18T09:03:23.312843+0000","last_fullsized":"2023-10-18T09:03:23.312843+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.322169+0000","last_change":"2023-10-18T09:03:23.322169+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.322169+0000","last_undegraded":"2023-10-18T09:03:23.322169+0000","last_fullsized":"2023-10-18T09:03:23.322169+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321038+0000","last_change":"2023-10-18T09:03:23.321038+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321038+0000","last_undegraded":"2023-10-18T09:03:23.321038+0000","last_fullsized":"2023-10-18T09:03:23.321038+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.373932+0000","last_change":"2023-10-18T09:03:24.373744+0000","last_active":"2023-10-18T09:03:24.373932+0000","last_peered":"2023-10-18T09:03:24.373932+0000","last_clean":"2023-10-18T09:03:24.373932+0000","last_became_active":"2023-10-18T09:03:24.373488+0000","last_became_peered":"2023-10-18T09:03:24.373488+0000","last_unstale":"2023-10-18T09:03:24.373932+0000","last_undegraded":"2023-10-18T09:03:24.373932+0000","last_fullsized":"2023-10-18T09:03:24.373932+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:21.541741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313740+0000","last_change":"2023-10-18T09:03:23.313740+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313740+0000","last_undegraded":"2023-10-18T09:03:23.313740+0000","last_fullsized":"2023-10-18T09:03:23.313740+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.315443+0000","last_change":"2023-10-18T09:03:23.315443+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.315443+0000","last_undegraded":"2023-10-18T09:03:23.315443+0000","last_fullsized":"2023-10-18T09:03:23.315443+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.378342+0000","last_change":"2023-10-18T09:03:24.376004+0000","last_active":"2023-10-18T09:03:24.378342+0000","last_peered":"2023-10-18T09:03:24.378342+0000","last_clean":"2023-10-18T09:03:24.378342+0000","last_became_active":"2023-10-18T09:03:24.375694+0000","last_became_peered":"2023-10-18T09:03:24.375694+0000","last_unstale":"2023-10-18T09:03:24.378342+0000","last_undegraded":"2023-10-18T09:03:24.378342+0000","last_fullsized":"2023-10-18T09:03:24.378342+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:45:05.600764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312725+0000","last_change":"2023-10-18T09:03:23.312725+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312725+0000","last_undegraded":"2023-10-18T09:03:23.312725+0000","last_fullsized":"2023-10-18T09:03:23.312725+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.365671+0000","last_change":"2023-10-18T09:03:24.364680+0000","last_active":"2023-10-18T09:03:24.365671+0000","last_peered":"2023-10-18T09:03:24.365671+0000","last_clean":"2023-10-18T09:03:24.365671+0000","last_became_active":"2023-10-18T09:03:24.364143+0000","last_became_peered":"2023-10-18T09:03:24.364143+0000","last_unstale":"2023-10-18T09:03:24.365671+0000","last_undegraded":"2023-10-18T09:03:24.365671+0000","last_fullsized":"2023-10-18T09:03:24.365671+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:42:51.261581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312889+0000","last_change":"2023-10-18T09:03:23.312889+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312889+0000","last_undegraded":"2023-10-18T09:03:23.312889+0000","last_fullsized":"2023-10-18T09:03:23.312889+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312482+0000","last_change":"2023-10-18T09:03:23.312482+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312482+0000","last_undegraded":"2023-10-18T09:03:23.312482+0000","last_fullsized":"2023-10-18T09:03:23.312482+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.318515+0000","last_change":"2023-10-18T09:03:23.318515+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.318515+0000","last_undegraded":"2023-10-18T09:03:23.318515+0000","last_fullsized":"2023-10-18T09:03:23.318515+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.374017+0000","last_change":"2023-10-18T09:03:24.371719+0000","last_active":"2023-10-18T09:03:24.374017+0000","last_peered":"2023-10-18T09:03:24.374017+0000","last_clean":"2023-10-18T09:03:24.374017+0000","last_became_active":"2023-10-18T09:03:24.371095+0000","last_became_peered":"2023-10-18T09:03:24.371095+0000","last_unstale":"2023-10-18T09:03:24.374017+0000","last_undegraded":"2023-10-18T09:03:24.374017+0000","last_fullsized":"2023-10-18T09:03:24.374017+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:50.009429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.317757+0000","last_change":"2023-10-18T09:03:23.317757+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.317757+0000","last_undegraded":"2023-10-18T09:03:23.317757+0000","last_fullsized":"2023-10-18T09:03:23.317757+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.326266+0000","last_change":"2023-10-18T09:03:23.326266+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.326266+0000","last_undegraded":"2023-10-18T09:03:23.326266+0000","last_fullsized":"2023-10-18T09:03:23.326266+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.312103+0000","last_change":"2023-10-18T09:03:23.312103+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.312103+0000","last_undegraded":"2023-10-18T09:03:23.312103+0000","last_fullsized":"2023-10-18T09:03:23.312103+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.318051+0000","last_change":"2023-10-18T09:03:23.318051+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.318051+0000","last_undegraded":"2023-10-18T09:03:23.318051+0000","last_fullsized":"2023-10-18T09:03:23.318051+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.320206+0000","last_change":"2023-10-18T09:03:23.320206+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.320206+0000","last_undegraded":"2023-10-18T09:03:23.320206+0000","last_fullsized":"2023-10-18T09:03:23.320206+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.329894+0000","last_change":"2023-10-18T09:03:23.329530+0000","last_active":"2023-10-18T09:03:23.329894+0000","last_peered":"2023-10-18T09:03:23.329894+0000","last_clean":"2023-10-18T09:03:23.329894+0000","last_became_active":"2023-10-18T09:03:23.329181+0000","last_became_peered":"2023-10-18T09:03:23.329181+0000","last_unstale":"2023-10-18T09:03:23.329894+0000","last_undegraded":"2023-10-18T09:03:23.329894+0000","last_fullsized":"2023-10-18T09:03:23.329894+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:45:29.736042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.318339+0000","last_change":"2023-10-18T09:03:23.318339+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.318339+0000","last_undegraded":"2023-10-18T09:03:23.318339+0000","last_fullsized":"2023-10-18T09:03:23.318339+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"143'1","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.391133+0000","last_change":"2023-10-18T09:03:23.331319+0000","last_active":"2023-10-18T09:03:24.391133+0000","last_peered":"2023-10-18T09:03:24.391133+0000","last_clean":"2023-10-18T09:03:24.391133+0000","last_became_active":"2023-10-18T09:03:23.331015+0000","last_became_peered":"2023-10-18T09:03:23.331015+0000","last_unstale":"2023-10-18T09:03:24.391133+0000","last_undegraded":"2023-10-18T09:03:24.391133+0000","last_fullsized":"2023-10-18T09:03:24.391133+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:40.692371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.319346+0000","last_change":"2023-10-18T09:03:23.319346+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.319346+0000","last_undegraded":"2023-10-18T09:03:23.319346+0000","last_fullsized":"2023-10-18T09:03:23.319346+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331754+0000","last_change":"2023-10-18T09:03:23.330634+0000","last_active":"2023-10-18T09:03:23.331754+0000","last_peered":"2023-10-18T09:03:23.331754+0000","last_clean":"2023-10-18T09:03:23.331754+0000","last_became_active":"2023-10-18T09:03:23.329893+0000","last_became_peered":"2023-10-18T09:03:23.329893+0000","last_unstale":"2023-10-18T09:03:23.331754+0000","last_undegraded":"2023-10-18T09:03:23.331754+0000","last_fullsized":"2023-10-18T09:03:23.331754+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:20:04.939191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.373913+0000","last_change":"2023-10-18T09:03:24.373601+0000","last_active":"2023-10-18T09:03:24.373913+0000","last_peered":"2023-10-18T09:03:24.373913+0000","last_clean":"2023-10-18T09:03:24.373913+0000","last_became_active":"2023-10-18T09:03:24.373369+0000","last_became_peered":"2023-10-18T09:03:24.373369+0000","last_unstale":"2023-10-18T09:03:24.373913+0000","last_undegraded":"2023-10-18T09:03:24.373913+0000","last_fullsized":"2023-10-18T09:03:24.373913+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:48.301601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.328765+0000","last_change":"2023-10-18T09:03:23.328512+0000","last_active":"2023-10-18T09:03:23.328765+0000","last_peered":"2023-10-18T09:03:23.328765+0000","last_clean":"2023-10-18T09:03:23.328765+0000","last_became_active":"2023-10-18T09:03:23.328090+0000","last_became_peered":"2023-10-18T09:03:23.328090+0000","last_unstale":"2023-10-18T09:03:23.328765+0000","last_undegraded":"2023-10-18T09:03:23.328765+0000","last_fullsized":"2023-10-18T09:03:23.328765+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:59:23.820590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.320552+0000","last_change":"2023-10-18T09:03:23.320552+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.320552+0000","last_undegraded":"2023-10-18T09:03:23.320552+0000","last_fullsized":"2023-10-18T09:03:23.320552+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.329917+0000","last_change":"2023-10-18T09:03:23.329851+0000","last_active":"2023-10-18T09:03:23.329917+0000","last_peered":"2023-10-18T09:03:23.329917+0000","last_clean":"2023-10-18T09:03:23.329917+0000","last_became_active":"2023-10-18T09:03:23.329208+0000","last_became_peered":"2023-10-18T09:03:23.329208+0000","last_unstale":"2023-10-18T09:03:23.329917+0000","last_undegraded":"2023-10-18T09:03:23.329917+0000","last_fullsized":"2023-10-18T09:03:23.329917+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:32:08.163420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321572+0000","last_change":"2023-10-18T09:03:23.321572+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321572+0000","last_undegraded":"2023-10-18T09:03:23.321572+0000","last_fullsized":"2023-10-18T09:03:23.321572+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"143'1","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.392871+0000","last_change":"2023-10-18T09:03:23.329790+0000","last_active":"2023-10-18T09:03:24.392871+0000","last_peered":"2023-10-18T09:03:24.392871+0000","last_clean":"2023-10-18T09:03:24.392871+0000","last_became_active":"2023-10-18T09:03:23.325710+0000","last_became_peered":"2023-10-18T09:03:23.325710+0000","last_unstale":"2023-10-18T09:03:24.392871+0000","last_undegraded":"2023-10-18T09:03:24.392871+0000","last_fullsized":"2023-10-18T09:03:24.392871+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:06:18.861877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.316958+0000","last_change":"2023-10-18T09:03:23.316958+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.316958+0000","last_undegraded":"2023-10-18T09:03:23.316958+0000","last_fullsized":"2023-10-18T09:03:23.316958+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.318565+0000","last_change":"2023-10-18T09:03:23.318325+0000","last_active":"2023-10-18T09:03:23.318565+0000","last_peered":"2023-10-18T09:03:23.318565+0000","last_clean":"2023-10-18T09:03:23.318565+0000","last_became_active":"2023-10-18T09:03:23.316556+0000","last_became_peered":"2023-10-18T09:03:23.316556+0000","last_unstale":"2023-10-18T09:03:23.318565+0000","last_undegraded":"2023-10-18T09:03:23.318565+0000","last_fullsized":"2023-10-18T09:03:23.318565+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:25:52.365591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.322477+0000","last_change":"2023-10-18T09:03:23.322477+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.322477+0000","last_undegraded":"2023-10-18T09:03:23.322477+0000","last_fullsized":"2023-10-18T09:03:23.322477+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.334091+0000","last_change":"2023-10-18T09:03:23.334041+0000","last_active":"2023-10-18T09:03:23.334091+0000","last_peered":"2023-10-18T09:03:23.334091+0000","last_clean":"2023-10-18T09:03:23.334091+0000","last_became_active":"2023-10-18T09:03:23.333830+0000","last_became_peered":"2023-10-18T09:03:23.333830+0000","last_unstale":"2023-10-18T09:03:23.334091+0000","last_undegraded":"2023-10-18T09:03:23.334091+0000","last_fullsized":"2023-10-18T09:03:23.334091+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:23.484620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.324949+0000","last_change":"2023-10-18T09:03:23.324949+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.324949+0000","last_undegraded":"2023-10-18T09:03:23.324949+0000","last_fullsized":"2023-10-18T09:03:23.324949+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.328748+0000","last_change":"2023-10-18T09:03:23.328337+0000","last_active":"2023-10-18T09:03:23.328748+0000","last_peered":"2023-10-18T09:03:23.328748+0000","last_clean":"2023-10-18T09:03:23.328748+0000","last_became_active":"2023-10-18T09:03:23.325594+0000","last_became_peered":"2023-10-18T09:03:23.325594+0000","last_unstale":"2023-10-18T09:03:23.328748+0000","last_undegraded":"2023-10-18T09:03:23.328748+0000","last_fullsized":"2023-10-18T09:03:23.328748+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:35:42.662714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.321318+0000","last_change":"2023-10-18T09:03:23.321318+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.321318+0000","last_undegraded":"2023-10-18T09:03:23.321318+0000","last_fullsized":"2023-10-18T09:03:23.321318+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.346035+0000","last_change":"2023-10-18T09:03:23.325401+0000","last_active":"2023-10-18T09:03:24.346035+0000","last_peered":"2023-10-18T09:03:24.346035+0000","last_clean":"2023-10-18T09:03:24.346035+0000","last_became_active":"2023-10-18T09:03:23.324277+0000","last_became_peered":"2023-10-18T09:03:23.324277+0000","last_unstale":"2023-10-18T09:03:24.346035+0000","last_undegraded":"2023-10-18T09:03:24.346035+0000","last_fullsized":"2023-10-18T09:03:24.346035+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:06:07.682599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.356612+0000","last_change":"2023-10-18T09:03:24.350913+0000","last_active":"2023-10-18T09:03:24.356612+0000","last_peered":"2023-10-18T09:03:24.356612+0000","last_clean":"2023-10-18T09:03:24.356612+0000","last_became_active":"2023-10-18T09:03:24.350214+0000","last_became_peered":"2023-10-18T09:03:24.350214+0000","last_unstale":"2023-10-18T09:03:24.356612+0000","last_undegraded":"2023-10-18T09:03:24.356612+0000","last_fullsized":"2023-10-18T09:03:24.356612+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:24:33.551156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331509+0000","last_change":"2023-10-18T09:03:23.331283+0000","last_active":"2023-10-18T09:03:23.331509+0000","last_peered":"2023-10-18T09:03:23.331509+0000","last_clean":"2023-10-18T09:03:23.331509+0000","last_became_active":"2023-10-18T09:03:23.329500+0000","last_became_peered":"2023-10-18T09:03:23.329500+0000","last_unstale":"2023-10-18T09:03:23.331509+0000","last_undegraded":"2023-10-18T09:03:23.331509+0000","last_fullsized":"2023-10-18T09:03:23.331509+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:13.492685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.326069+0000","last_change":"2023-10-18T09:03:23.326069+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.326069+0000","last_undegraded":"2023-10-18T09:03:23.326069+0000","last_fullsized":"2023-10-18T09:03:23.326069+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.334640+0000","last_change":"2023-10-18T09:03:23.334514+0000","last_active":"2023-10-18T09:03:23.334640+0000","last_peered":"2023-10-18T09:03:23.334640+0000","last_clean":"2023-10-18T09:03:23.334640+0000","last_became_active":"2023-10-18T09:03:23.334268+0000","last_became_peered":"2023-10-18T09:03:23.334268+0000","last_unstale":"2023-10-18T09:03:23.334640+0000","last_undegraded":"2023-10-18T09:03:23.334640+0000","last_fullsized":"2023-10-18T09:03:23.334640+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:16.978313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.367932+0000","last_change":"2023-10-18T09:03:24.367289+0000","last_active":"2023-10-18T09:03:24.367932+0000","last_peered":"2023-10-18T09:03:24.367932+0000","last_clean":"2023-10-18T09:03:24.367932+0000","last_became_active":"2023-10-18T09:03:24.366724+0000","last_became_peered":"2023-10-18T09:03:24.366724+0000","last_unstale":"2023-10-18T09:03:24.367932+0000","last_undegraded":"2023-10-18T09:03:24.367932+0000","last_fullsized":"2023-10-18T09:03:24.367932+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:29:55.406170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.325184+0000","last_change":"2023-10-18T09:03:23.323723+0000","last_active":"2023-10-18T09:03:23.325184+0000","last_peered":"2023-10-18T09:03:23.325184+0000","last_clean":"2023-10-18T09:03:23.325184+0000","last_became_active":"2023-10-18T09:03:23.322838+0000","last_became_peered":"2023-10-18T09:03:23.322838+0000","last_unstale":"2023-10-18T09:03:23.325184+0000","last_undegraded":"2023-10-18T09:03:23.325184+0000","last_fullsized":"2023-10-18T09:03:23.325184+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:12.382096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.375498+0000","last_change":"2023-10-18T09:03:24.374652+0000","last_active":"2023-10-18T09:03:24.375498+0000","last_peered":"2023-10-18T09:03:24.375498+0000","last_clean":"2023-10-18T09:03:24.375498+0000","last_became_active":"2023-10-18T09:03:24.366671+0000","last_became_peered":"2023-10-18T09:03:24.366671+0000","last_unstale":"2023-10-18T09:03:24.375498+0000","last_undegraded":"2023-10-18T09:03:24.375498+0000","last_fullsized":"2023-10-18T09:03:24.375498+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:07:26.781375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.334932+0000","last_change":"2023-10-18T09:03:23.334809+0000","last_active":"2023-10-18T09:03:23.334932+0000","last_peered":"2023-10-18T09:03:23.334932+0000","last_clean":"2023-10-18T09:03:23.334932+0000","last_became_active":"2023-10-18T09:03:23.334509+0000","last_became_peered":"2023-10-18T09:03:23.334509+0000","last_unstale":"2023-10-18T09:03:23.334932+0000","last_undegraded":"2023-10-18T09:03:23.334932+0000","last_fullsized":"2023-10-18T09:03:23.334932+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:13.492685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.356667+0000","last_change":"2023-10-18T09:03:24.353278+0000","last_active":"2023-10-18T09:03:24.356667+0000","last_peered":"2023-10-18T09:03:24.356667+0000","last_clean":"2023-10-18T09:03:24.356667+0000","last_became_active":"2023-10-18T09:03:24.352158+0000","last_became_peered":"2023-10-18T09:03:24.352158+0000","last_unstale":"2023-10-18T09:03:24.356667+0000","last_undegraded":"2023-10-18T09:03:24.356667+0000","last_fullsized":"2023-10-18T09:03:24.356667+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:05.753025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.332360+0000","last_change":"2023-10-18T09:03:23.332284+0000","last_active":"2023-10-18T09:03:23.332360+0000","last_peered":"2023-10-18T09:03:23.332360+0000","last_clean":"2023-10-18T09:03:23.332360+0000","last_became_active":"2023-10-18T09:03:23.332040+0000","last_became_peered":"2023-10-18T09:03:23.332040+0000","last_unstale":"2023-10-18T09:03:23.332360+0000","last_undegraded":"2023-10-18T09:03:23.332360+0000","last_fullsized":"2023-10-18T09:03:23.332360+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:41:21.941443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313059+0000","last_change":"2023-10-18T09:03:23.313059+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313059+0000","last_undegraded":"2023-10-18T09:03:23.313059+0000","last_fullsized":"2023-10-18T09:03:23.313059+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.333121+0000","last_change":"2023-10-18T09:03:23.332929+0000","last_active":"2023-10-18T09:03:23.333121+0000","last_peered":"2023-10-18T09:03:23.333121+0000","last_clean":"2023-10-18T09:03:23.333121+0000","last_became_active":"2023-10-18T09:03:23.332508+0000","last_became_peered":"2023-10-18T09:03:23.332508+0000","last_unstale":"2023-10-18T09:03:23.333121+0000","last_undegraded":"2023-10-18T09:03:23.333121+0000","last_fullsized":"2023-10-18T09:03:23.333121+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:30.964339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.319843+0000","last_change":"2023-10-18T09:03:23.319843+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.319843+0000","last_undegraded":"2023-10-18T09:03:23.319843+0000","last_fullsized":"2023-10-18T09:03:23.319843+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331736+0000","last_change":"2023-10-18T09:03:23.330153+0000","last_active":"2023-10-18T09:03:23.331736+0000","last_peered":"2023-10-18T09:03:23.331736+0000","last_clean":"2023-10-18T09:03:23.331736+0000","last_became_active":"2023-10-18T09:03:23.325376+0000","last_became_peered":"2023-10-18T09:03:23.325376+0000","last_unstale":"2023-10-18T09:03:23.331736+0000","last_undegraded":"2023-10-18T09:03:23.331736+0000","last_fullsized":"2023-10-18T09:03:23.331736+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:53:43.431276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.374379+0000","last_change":"2023-10-18T09:03:24.374330+0000","last_active":"2023-10-18T09:03:24.374379+0000","last_peered":"2023-10-18T09:03:24.374379+0000","last_clean":"2023-10-18T09:03:24.374379+0000","last_became_active":"2023-10-18T09:03:24.374144+0000","last_became_peered":"2023-10-18T09:03:24.374144+0000","last_unstale":"2023-10-18T09:03:24.374379+0000","last_undegraded":"2023-10-18T09:03:24.374379+0000","last_fullsized":"2023-10-18T09:03:24.374379+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:59.934981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.325277+0000","last_change":"2023-10-18T09:03:23.324222+0000","last_active":"2023-10-18T09:03:23.325277+0000","last_peered":"2023-10-18T09:03:23.325277+0000","last_clean":"2023-10-18T09:03:23.325277+0000","last_became_active":"2023-10-18T09:03:23.323810+0000","last_became_peered":"2023-10-18T09:03:23.323810+0000","last_unstale":"2023-10-18T09:03:23.325277+0000","last_undegraded":"2023-10-18T09:03:23.325277+0000","last_fullsized":"2023-10-18T09:03:23.325277+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:20.536935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.371800+0000","last_change":"2023-10-18T09:03:24.370815+0000","last_active":"2023-10-18T09:03:24.371800+0000","last_peered":"2023-10-18T09:03:24.371800+0000","last_clean":"2023-10-18T09:03:24.371800+0000","last_became_active":"2023-10-18T09:03:24.370553+0000","last_became_peered":"2023-10-18T09:03:24.370553+0000","last_unstale":"2023-10-18T09:03:24.371800+0000","last_undegraded":"2023-10-18T09:03:24.371800+0000","last_fullsized":"2023-10-18T09:03:24.371800+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:17.983120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.332741+0000","last_change":"2023-10-18T09:03:23.332652+0000","last_active":"2023-10-18T09:03:23.332741+0000","last_peered":"2023-10-18T09:03:23.332741+0000","last_clean":"2023-10-18T09:03:23.332741+0000","last_became_active":"2023-10-18T09:03:23.331276+0000","last_became_peered":"2023-10-18T09:03:23.331276+0000","last_unstale":"2023-10-18T09:03:23.332741+0000","last_undegraded":"2023-10-18T09:03:23.332741+0000","last_fullsized":"2023-10-18T09:03:23.332741+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:30:12.382096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.367994+0000","last_change":"2023-10-18T09:03:24.367856+0000","last_active":"2023-10-18T09:03:24.367994+0000","last_peered":"2023-10-18T09:03:24.367994+0000","last_clean":"2023-10-18T09:03:24.367994+0000","last_became_active":"2023-10-18T09:03:24.367568+0000","last_became_peered":"2023-10-18T09:03:24.367568+0000","last_unstale":"2023-10-18T09:03:24.367994+0000","last_undegraded":"2023-10-18T09:03:24.367994+0000","last_fullsized":"2023-10-18T09:03:24.367994+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:14.497491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.325153+0000","last_change":"2023-10-18T09:03:23.322408+0000","last_active":"2023-10-18T09:03:23.325153+0000","last_peered":"2023-10-18T09:03:23.325153+0000","last_clean":"2023-10-18T09:03:23.325153+0000","last_became_active":"2023-10-18T09:03:23.321746+0000","last_became_peered":"2023-10-18T09:03:23.321746+0000","last_unstale":"2023-10-18T09:03:23.325153+0000","last_undegraded":"2023-10-18T09:03:23.325153+0000","last_fullsized":"2023-10-18T09:03:23.325153+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:08.225684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313248+0000","last_change":"2023-10-18T09:03:23.313248+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313248+0000","last_undegraded":"2023-10-18T09:03:23.313248+0000","last_fullsized":"2023-10-18T09:03:23.313248+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.325186+0000","last_change":"2023-10-18T09:03:23.324369+0000","last_active":"2023-10-18T09:03:23.325186+0000","last_peered":"2023-10-18T09:03:23.325186+0000","last_clean":"2023-10-18T09:03:23.325186+0000","last_became_active":"2023-10-18T09:03:23.323637+0000","last_became_peered":"2023-10-18T09:03:23.323637+0000","last_unstale":"2023-10-18T09:03:23.325186+0000","last_undegraded":"2023-10-18T09:03:23.325186+0000","last_fullsized":"2023-10-18T09:03:23.325186+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:18:41.207403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.379077+0000","last_change":"2023-10-18T09:03:24.378954+0000","last_active":"2023-10-18T09:03:24.379077+0000","last_peered":"2023-10-18T09:03:24.379077+0000","last_clean":"2023-10-18T09:03:24.379077+0000","last_became_active":"2023-10-18T09:03:24.378534+0000","last_became_peered":"2023-10-18T09:03:24.378534+0000","last_unstale":"2023-10-18T09:03:24.379077+0000","last_undegraded":"2023-10-18T09:03:24.379077+0000","last_fullsized":"2023-10-18T09:03:24.379077+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:14:13.807602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.332533+0000","last_change":"2023-10-18T09:03:23.332462+0000","last_active":"2023-10-18T09:03:23.332533+0000","last_peered":"2023-10-18T09:03:23.332533+0000","last_clean":"2023-10-18T09:03:23.332533+0000","last_became_active":"2023-10-18T09:03:23.332239+0000","last_became_peered":"2023-10-18T09:03:23.332239+0000","last_unstale":"2023-10-18T09:03:23.332533+0000","last_undegraded":"2023-10-18T09:03:23.332533+0000","last_fullsized":"2023-10-18T09:03:23.332533+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:52:25.333743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.361716+0000","last_change":"2023-10-18T09:03:24.361276+0000","last_active":"2023-10-18T09:03:24.361716+0000","last_peered":"2023-10-18T09:03:24.361716+0000","last_clean":"2023-10-18T09:03:24.361716+0000","last_became_active":"2023-10-18T09:03:24.360521+0000","last_became_peered":"2023-10-18T09:03:24.360521+0000","last_unstale":"2023-10-18T09:03:24.361716+0000","last_undegraded":"2023-10-18T09:03:24.361716+0000","last_fullsized":"2023-10-18T09:03:24.361716+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:11.398182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331491+0000","last_change":"2023-10-18T09:03:23.330931+0000","last_active":"2023-10-18T09:03:23.331491+0000","last_peered":"2023-10-18T09:03:23.331491+0000","last_clean":"2023-10-18T09:03:23.331491+0000","last_became_active":"2023-10-18T09:03:23.330686+0000","last_became_peered":"2023-10-18T09:03:23.330686+0000","last_unstale":"2023-10-18T09:03:23.331491+0000","last_undegraded":"2023-10-18T09:03:23.331491+0000","last_fullsized":"2023-10-18T09:03:23.331491+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:00:47.296794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.317564+0000","last_change":"2023-10-18T09:03:23.317564+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.317564+0000","last_undegraded":"2023-10-18T09:03:23.317564+0000","last_fullsized":"2023-10-18T09:03:23.317564+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"143'2","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.393968+0000","last_change":"2023-10-18T09:03:23.330533+0000","last_active":"2023-10-18T09:03:24.393968+0000","last_peered":"2023-10-18T09:03:24.393968+0000","last_clean":"2023-10-18T09:03:24.393968+0000","last_became_active":"2023-10-18T09:03:23.330093+0000","last_became_peered":"2023-10-18T09:03:23.330093+0000","last_unstale":"2023-10-18T09:03:24.393968+0000","last_undegraded":"2023-10-18T09:03:24.393968+0000","last_fullsized":"2023-10-18T09:03:24.393968+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:54.344592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.315057+0000","last_change":"2023-10-18T09:03:23.315057+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.315057+0000","last_undegraded":"2023-10-18T09:03:23.315057+0000","last_fullsized":"2023-10-18T09:03:23.315057+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.330035+0000","last_change":"2023-10-18T09:03:23.329947+0000","last_active":"2023-10-18T09:03:23.330035+0000","last_peered":"2023-10-18T09:03:23.330035+0000","last_clean":"2023-10-18T09:03:23.330035+0000","last_became_active":"2023-10-18T09:03:23.329667+0000","last_became_peered":"2023-10-18T09:03:23.329667+0000","last_unstale":"2023-10-18T09:03:23.330035+0000","last_undegraded":"2023-10-18T09:03:23.330035+0000","last_fullsized":"2023-10-18T09:03:23.330035+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:23.484620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313161+0000","last_change":"2023-10-18T09:03:23.313161+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313161+0000","last_undegraded":"2023-10-18T09:03:23.313161+0000","last_fullsized":"2023-10-18T09:03:23.313161+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.333102+0000","last_change":"2023-10-18T09:03:23.332779+0000","last_active":"2023-10-18T09:03:23.333102+0000","last_peered":"2023-10-18T09:03:23.333102+0000","last_clean":"2023-10-18T09:03:23.333102+0000","last_became_active":"2023-10-18T09:03:23.332152+0000","last_became_peered":"2023-10-18T09:03:23.332152+0000","last_unstale":"2023-10-18T09:03:23.333102+0000","last_undegraded":"2023-10-18T09:03:23.333102+0000","last_fullsized":"2023-10-18T09:03:23.333102+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:40.692371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.313904+0000","last_change":"2023-10-18T09:03:23.313904+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.313904+0000","last_undegraded":"2023-10-18T09:03:23.313904+0000","last_fullsized":"2023-10-18T09:03:23.313904+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.346056+0000","last_change":"2023-10-18T09:03:23.325136+0000","last_active":"2023-10-18T09:03:24.346056+0000","last_peered":"2023-10-18T09:03:24.346056+0000","last_clean":"2023-10-18T09:03:24.346056+0000","last_became_active":"2023-10-18T09:03:23.323782+0000","last_became_peered":"2023-10-18T09:03:23.323782+0000","last_unstale":"2023-10-18T09:03:24.346056+0000","last_undegraded":"2023-10-18T09:03:24.346056+0000","last_fullsized":"2023-10-18T09:03:24.346056+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:42.460476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.318651+0000","last_change":"2023-10-18T09:03:23.318651+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.318651+0000","last_undegraded":"2023-10-18T09:03:23.318651+0000","last_fullsized":"2023-10-18T09:03:23.318651+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.329867+0000","last_change":"2023-10-18T09:03:23.329448+0000","last_active":"2023-10-18T09:03:23.329867+0000","last_peered":"2023-10-18T09:03:23.329867+0000","last_clean":"2023-10-18T09:03:23.329867+0000","last_became_active":"2023-10-18T09:03:23.329094+0000","last_became_peered":"2023-10-18T09:03:23.329094+0000","last_unstale":"2023-10-18T09:03:23.329867+0000","last_undegraded":"2023-10-18T09:03:23.329867+0000","last_fullsized":"2023-10-18T09:03:23.329867+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:42:23.072395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.314648+0000","last_change":"2023-10-18T09:03:23.314648+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.314648+0000","last_undegraded":"2023-10-18T09:03:23.314648+0000","last_fullsized":"2023-10-18T09:03:23.314648+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.324173+0000","last_change":"2023-10-18T09:03:23.323348+0000","last_active":"2023-10-18T09:03:23.324173+0000","last_peered":"2023-10-18T09:03:23.324173+0000","last_clean":"2023-10-18T09:03:23.324173+0000","last_became_active":"2023-10-18T09:03:23.323009+0000","last_became_peered":"2023-10-18T09:03:23.323009+0000","last_unstale":"2023-10-18T09:03:23.324173+0000","last_undegraded":"2023-10-18T09:03:23.324173+0000","last_fullsized":"2023-10-18T09:03:23.324173+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:07:37.803900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.373379+0000","last_change":"2023-10-18T09:03:24.373169+0000","last_active":"2023-10-18T09:03:24.373379+0000","last_peered":"2023-10-18T09:03:24.373379+0000","last_clean":"2023-10-18T09:03:24.373379+0000","last_became_active":"2023-10-18T09:03:24.370785+0000","last_became_peered":"2023-10-18T09:03:24.370785+0000","last_unstale":"2023-10-18T09:03:24.373379+0000","last_undegraded":"2023-10-18T09:03:24.373379+0000","last_fullsized":"2023-10-18T09:03:24.373379+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:25:24.489427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331718+0000","last_change":"2023-10-18T09:03:23.329465+0000","last_active":"2023-10-18T09:03:23.331718+0000","last_peered":"2023-10-18T09:03:23.331718+0000","last_clean":"2023-10-18T09:03:23.331718+0000","last_became_active":"2023-10-18T09:03:23.329200+0000","last_became_peered":"2023-10-18T09:03:23.329200+0000","last_unstale":"2023-10-18T09:03:23.331718+0000","last_undegraded":"2023-10-18T09:03:23.331718+0000","last_fullsized":"2023-10-18T09:03:23.331718+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:11.421962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":142,"state":"creating+peering","last_fresh":"2023-10-18T09:03:23.319950+0000","last_change":"2023-10-18T09:03:23.319950+0000","last_active":"2023-10-18T09:03:23.295746+0000","last_peered":"2023-10-18T09:03:23.295746+0000","last_clean":"2023-10-18T09:03:23.295746+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-18T09:03:23.319950+0000","last_undegraded":"2023-10-18T09:03:23.319950+0000","last_fullsized":"2023-10-18T09:03:23.319950+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.331450+0000","last_change":"2023-10-18T09:03:23.329760+0000","last_active":"2023-10-18T09:03:23.331450+0000","last_peered":"2023-10-18T09:03:23.331450+0000","last_clean":"2023-10-18T09:03:23.331450+0000","last_became_active":"2023-10-18T09:03:23.329479+0000","last_became_peered":"2023-10-18T09:03:23.329479+0000","last_unstale":"2023-10-18T09:03:23.331450+0000","last_undegraded":"2023-10-18T09:03:23.331450+0000","last_fullsized":"2023-10-18T09:03:23.331450+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:50:20.536935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.378559+0000","last_change":"2023-10-18T09:03:24.378443+0000","last_active":"2023-10-18T09:03:24.378559+0000","last_peered":"2023-10-18T09:03:24.378559+0000","last_clean":"2023-10-18T09:03:24.378559+0000","last_became_active":"2023-10-18T09:03:24.378196+0000","last_became_peered":"2023-10-18T09:03:24.378196+0000","last_unstale":"2023-10-18T09:03:24.378559+0000","last_undegraded":"2023-10-18T09:03:24.378559+0000","last_fullsized":"2023-10-18T09:03:24.378559+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:13:51.906537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.333073+0000","last_change":"2023-10-18T09:03:23.332893+0000","last_active":"2023-10-18T09:03:23.333073+0000","last_peered":"2023-10-18T09:03:23.333073+0000","last_clean":"2023-10-18T09:03:23.333073+0000","last_became_active":"2023-10-18T09:03:23.332302+0000","last_became_peered":"2023-10-18T09:03:23.332302+0000","last_unstale":"2023-10-18T09:03:23.333073+0000","last_undegraded":"2023-10-18T09:03:23.333073+0000","last_fullsized":"2023-10-18T09:03:23.333073+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:34:22.081612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":301,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-18T09:03:23.308489+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:03:23.308489+0000","last_peered":"2023-10-18T09:03:23.308489+0000","last_clean":"2023-10-18T09:03:23.308489+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:03:23.308489+0000","last_undegraded":"2023-10-18T09:03:23.308489+0000","last_fullsized":"2023-10-18T09:03:23.308489+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1361,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193951,"statfs":{"total":200000000,"available":198606297,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.135,"15min":1.038},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.579,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.109,"15min":1.046},"min":{"1min":0.64500000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.421,"5min":1.97,"15min":1.97},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1819999999999999,"15min":1.155},"min":{"1min":0.83299999999999996,"5min":0.70099999999999996,"15min":0.61799999999999999},"max":{"1min":1.595,"5min":1.829,"15min":1.829},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.1240000000000001,"15min":1.1259999999999999},"min":{"1min":0.72799999999999998,"5min":0.61099999999999999,"15min":0.58799999999999997},"max":{"1min":1.4239999999999999,"5min":1.657,"15min":1.7989999999999999},"last":0.73399999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3220000000000001,"15min":1.264},"min":{"1min":0.94699999999999995,"5min":0.78600000000000003,"15min":0.65400000000000003},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.9730000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.206,"5min":1.27,"15min":1.2190000000000001},"min":{"1min":0.79000000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.601,"5min":1.863,"15min":1.863},"last":1.121}]},{"osd":3,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2829999999999999,"15min":1.27},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.5509999999999999,"5min":1.98,"15min":1.98},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3740000000000001,"15min":1.3340000000000001},"min":{"1min":0.97299999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7210000000000001,"5min":1.756,"15min":1.9930000000000001},"last":0.83199999999999996}]},{"osd":4,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2629999999999999,"15min":1.1990000000000001},"min":{"1min":0.69899999999999995,"5min":0.66000000000000003,"15min":0.60499999999999998},"max":{"1min":1.802,"5min":1.923,"15min":2.1070000000000002},"last":1.012},{"interface":"front","average":{"1min":1.363,"5min":1.341,"15min":1.377},"min":{"1min":0.94699999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.768,"5min":2.0369999999999999,"15min":2.5169999999999999},"last":1.5209999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.349,"15min":1.3520000000000001},"min":{"1min":1.093,"5min":0.622,"15min":0.622},"max":{"1min":1.599,"5min":1.9199999999999999,"15min":2.0720000000000001},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.444,"15min":1.423},"min":{"1min":1.119,"5min":0.748,"15min":0.67900000000000005},"max":{"1min":1.77,"5min":2.0259999999999998,"15min":2.0270000000000001},"last":1.5609999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.506,"15min":1.4750000000000001},"min":{"1min":1.274,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.8440000000000001,"5min":2.0779999999999998,"15min":2.3959999999999999},"last":1.472},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"min":{"1min":1.1899999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.873,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.595}]}]},{"osd":6,"up_from":13,"seq":55834574950,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":777,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194535,"statfs":{"total":200000000,"available":199204056,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.252,"15min":1.141},"min":{"1min":0.83499999999999996,"5min":0.64700000000000002,"15min":0.60299999999999998},"max":{"1min":1.5940000000000001,"5min":1.627,"15min":1.6850000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.357,"15min":1.274},"min":{"1min":0.81799999999999995,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8149999999999999,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.5640000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.101,"15min":1.091},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63800000000000001},"max":{"1min":1.5269999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.119},{"interface":"front","average":{"1min":1.145,"5min":1.2230000000000001,"15min":1.198},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5940000000000001,"5min":1.8560000000000001,"15min":2.9609999999999999},"last":1.04}]},{"osd":2,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.169,"15min":1.1699999999999999},"min":{"1min":0.84899999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4850000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.236,"15min":1.2090000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.66300000000000003},"max":{"1min":1.627,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.1919999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3009999999999999,"15min":1.2709999999999999},"min":{"1min":0.98299999999999998,"5min":0.79400000000000004,"15min":0.79200000000000004},"max":{"1min":1.627,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.286,"15min":1.2430000000000001},"min":{"1min":0.84999999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7889999999999999},"last":1.27}]},{"osd":4,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.254,"15min":1.208},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.706,"5min":2.444,"15min":2.444},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.282},"min":{"1min":0.91700000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7749999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.0720000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.452,"15min":1.3839999999999999},"min":{"1min":1.0309999999999999,"5min":0.625,"15min":0.58899999999999997},"max":{"1min":1.722,"5min":2.032,"15min":2.032},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3220000000000001,"15min":1.256},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.49199999999999999},"max":{"1min":1.679,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.387}]},{"osd":7,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.4099999999999999,"15min":1.4019999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.768,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.347},{"interface":"front","average":{"1min":1.548,"5min":1.5429999999999999,"15min":1.5289999999999999},"min":{"1min":1.087,"5min":0.83099999999999996,"15min":0.68500000000000005},"max":{"1min":1.982,"5min":2.3140000000000001,"15min":2.794},"last":1.226}]}]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":771,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194541,"statfs":{"total":200000000,"available":199210331,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.3400000000000001,"15min":1.278},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.63400000000000001},"max":{"1min":1.8819999999999999,"5min":9.609,"15min":9.609},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.286,"15min":1.3},"min":{"1min":0.754,"5min":0.751,"15min":0.60799999999999998},"max":{"1min":1.7090000000000001,"5min":1.8260000000000001,"15min":5.3029999999999999},"last":1.5820000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.3169999999999999,"15min":1.252},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6439999999999999,"5min":11.68,"15min":11.68},"last":1.5329999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.1819999999999999,"15min":1.2210000000000001},"min":{"1min":0.70099999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3360000000000001,"5min":2.2410000000000001,"15min":3.7050000000000001},"last":1.175}]},{"osd":2,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2589999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.755},"max":{"1min":1.7030000000000001,"5min":2.0350000000000001,"15min":2.2090000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2450000000000001,"15min":1.276},"min":{"1min":0.94399999999999995,"5min":0.75900000000000001,"15min":0.72399999999999998},"max":{"1min":1.6910000000000001,"5min":2.153,"15min":3.8180000000000001},"last":1.694}]},{"osd":3,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.3200000000000001,"15min":1.272},"min":{"1min":0.72299999999999998,"5min":0.68799999999999994,"15min":0.65400000000000003},"max":{"1min":1.4179999999999999,"5min":11.797000000000001,"15min":11.797000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.454,"5min":1.3740000000000001,"15min":1.347},"min":{"1min":1.135,"5min":0.747,"15min":0.747},"max":{"1min":1.798,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.2769999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.389,"15min":1.4079999999999999},"min":{"1min":1.157,"5min":0.89900000000000002,"15min":0.75900000000000001},"max":{"1min":1.8100000000000001,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.252,"15min":1.4330000000000001},"min":{"1min":0.88700000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.629,"5min":2.4049999999999998,"15min":5.4580000000000002},"last":1.4079999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2649999999999999,"15min":1.357},"min":{"1min":0.84499999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.5780000000000001,"5min":2.5190000000000001,"15min":5.141},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.629,"5min":1.498,"15min":1.552},"min":{"1min":1.127,"5min":0.82499999999999996,"15min":0.71099999999999997},"max":{"1min":2.0350000000000001,"5min":2.657,"15min":2.657},"last":1.6479999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.5469999999999999,"15min":1.6240000000000001},"min":{"1min":1.2130000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6970000000000001,"5min":11.829000000000001,"15min":11.829000000000001},"last":1.75},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.405,"15min":1.4830000000000001},"min":{"1min":1.0920000000000001,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.837,"5min":2.9420000000000002,"15min":4.8419999999999996},"last":1.492}]}]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":770,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194542,"statfs":{"total":200000000,"available":199211350,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.1479999999999999,"15min":1.1220000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5680000000000001,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.071,"5min":1.1619999999999999,"15min":1.1599999999999999},"min":{"1min":0.71099999999999997,"5min":0.63300000000000001,"15min":0.60099999999999998},"max":{"1min":1.6719999999999999,"5min":3.234,"15min":3.234},"last":1.05}]},{"osd":1,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.1419999999999999,"15min":1.1100000000000001},"min":{"1min":0.64700000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.484,"5min":3.718,"15min":3.718},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.2,"15min":1.224},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.7529999999999999,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.419}]},{"osd":2,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.218,"15min":1.1930000000000001},"min":{"1min":0.71799999999999997,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4099999999999999,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":1.331},{"interface":"front","average":{"1min":1.24,"5min":1.3380000000000001,"15min":1.3089999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.58899999999999997},"max":{"1min":1.6870000000000001,"5min":3.532,"15min":3.532},"last":1.6739999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2709999999999999,"15min":1.2889999999999999},"min":{"1min":0.78200000000000003,"5min":0.71899999999999997,"15min":0.67000000000000004},"max":{"1min":1.5700000000000001,"5min":3.363,"15min":3.363},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.399,"15min":1.409},"min":{"1min":0.84999999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.603,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":1.2190000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.274,"15min":1.2969999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.498},"max":{"1min":1.7909999999999999,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.296,"15min":1.341},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.38400000000000001},"max":{"1min":1.571,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":1.1759999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.3720000000000001,"15min":1.3939999999999999},"min":{"1min":0.78300000000000003,"5min":0.64300000000000002,"15min":0.51600000000000001},"max":{"1min":1.478,"5min":4.4429999999999996,"15min":5.0049999999999999},"last":1.726},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4510000000000001,"15min":1.532},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.45400000000000001},"max":{"1min":1.706,"5min":4.1239999999999997,"15min":4.407},"last":1.756}]},{"osd":7,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.4330000000000001,"15min":1.484},"min":{"1min":0.85599999999999998,"5min":0.82499999999999996,"15min":0.438},"max":{"1min":1.7410000000000001,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.536,"15min":1.546},"min":{"1min":0.92000000000000004,"5min":0.78700000000000003,"15min":0.747},"max":{"1min":1.7410000000000001,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":1.966}]}]},{"osd":3,"up_from":13,"seq":55834574952,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":785,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194526,"statfs":{"total":200000000,"available":199195599,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":0.94999999999999996,"15min":0.84599999999999997},"min":{"1min":0.63800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":4.774,"5min":4.774,"15min":5.5730000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.96299999999999997,"15min":0.91600000000000004},"min":{"1min":0.53600000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.2370000000000001,"5min":3.2370000000000001,"15min":6.5599999999999996},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.0049999999999999,"15min":0.96899999999999997},"min":{"1min":0.55200000000000005,"5min":0.39100000000000001,"15min":0.36799999999999999},"max":{"1min":1.4790000000000001,"5min":1.839,"15min":6.8099999999999996},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2629999999999999,"15min":1.1679999999999999},"min":{"1min":0.85799999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6659999999999999,"5min":2.0720000000000001,"15min":3.1920000000000002},"last":1.0800000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.1060000000000001,"15min":1.052},"min":{"1min":0.62,"5min":0.46800000000000003,"15min":0.45300000000000001},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":7.4580000000000002},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.456,"5min":1.2150000000000001,"15min":1.117},"min":{"1min":0.78700000000000003,"5min":0.53200000000000003,"15min":0.51400000000000001},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":0.77800000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.327,"15min":1.339},"min":{"1min":0.82099999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":18.664999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3009999999999999,"15min":1.2110000000000001},"min":{"1min":0.92700000000000005,"5min":0.877,"15min":0.64000000000000001},"max":{"1min":2.0230000000000001,"5min":2.093,"15min":7.0700000000000003},"last":0.96799999999999997}]},{"osd":5,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.333,"15min":1.266},"min":{"1min":0.90500000000000003,"5min":0.77600000000000002,"15min":0.67700000000000005},"max":{"1min":1.9099999999999999,"5min":2.1680000000000001,"15min":6.9139999999999997},"last":1.252},{"interface":"front","average":{"1min":1.6930000000000001,"5min":1.4650000000000001,"15min":1.409},"min":{"1min":1.1140000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":4.5540000000000003,"5min":4.5540000000000003,"15min":16.390999999999998},"last":1.1870000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.4850000000000001,"15min":1.4510000000000001},"min":{"1min":0.92200000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.375,"5min":3.375,"15min":7.7270000000000003},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.494,"15min":1.3959999999999999},"min":{"1min":1.0189999999999999,"5min":0.872,"15min":0.60199999999999998},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":7.242},"last":1.3799999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.821,"5min":1.5660000000000001,"15min":1.4219999999999999},"min":{"1min":1.226,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":4.8630000000000004,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.8089999999999999,"5min":1.5589999999999999,"15min":1.502},"min":{"1min":0.98399999999999999,"5min":0.622,"15min":0.622},"max":{"1min":5.4900000000000002,"5min":5.4900000000000002,"15min":12.946},"last":1.286}]}]},{"osd":2,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":777,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194535,"statfs":{"total":200000000,"available":199204056,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.88200000000000001,"15min":0.91300000000000003},"min":{"1min":0.56000000000000005,"5min":0.48599999999999999,"15min":0.36899999999999999},"max":{"1min":1.3300000000000001,"5min":1.706,"15min":4.2210000000000001},"last":1.046},{"interface":"front","average":{"1min":1.012,"5min":1.042,"15min":1.04},"min":{"1min":0.59299999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.617,"5min":4.3959999999999999,"15min":4.3959999999999999},"last":0.88700000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.1359999999999999,"15min":1.077},"min":{"1min":0.65300000000000002,"5min":0.55900000000000005,"15min":0.49099999999999999},"max":{"1min":1.522,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.181,"15min":1.121},"min":{"1min":0.77000000000000002,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":1.552,"5min":2.448,"15min":2.5350000000000001},"last":0.94899999999999995}]},{"osd":3,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.194,"15min":1.141},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.48199999999999998},"max":{"1min":1.7569999999999999,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.365,"15min":1.427},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.54100000000000004},"max":{"1min":1.728,"5min":2.2829999999999999,"15min":7.5570000000000004},"last":1.502}]},{"osd":4,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4530000000000001,"15min":1.4730000000000001},"min":{"1min":1.0289999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.879,"5min":2.4119999999999999,"15min":7.6189999999999998},"last":1.167},{"interface":"front","average":{"1min":1.371,"5min":1.4910000000000001,"15min":1.458},"min":{"1min":0.98599999999999999,"5min":0.98099999999999998,"15min":0.86599999999999999},"max":{"1min":1.7,"5min":3.423,"15min":3.423},"last":1.4199999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.407,"15min":1.3089999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.61599999999999999},"max":{"1min":1.6000000000000001,"5min":10.507,"15min":10.507},"last":1.7869999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.4910000000000001,"15min":1.5349999999999999},"min":{"1min":1.1020000000000001,"5min":0.88100000000000001,"15min":0.81399999999999995},"max":{"1min":1.9550000000000001,"5min":2.536,"15min":7.657},"last":1.3260000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.52,"15min":1.518},"min":{"1min":1.2210000000000001,"5min":1.0289999999999999,"15min":0.74299999999999999},"max":{"1min":1.9099999999999999,"5min":2.8780000000000001,"15min":3.5299999999999998},"last":1.466},{"interface":"front","average":{"1min":1.399,"5min":1.5640000000000001,"15min":1.611},"min":{"1min":0.96499999999999997,"5min":0.85899999999999999,"15min":0.77600000000000002},"max":{"1min":2.0190000000000001,"5min":10.561,"15min":37.771999999999998},"last":1.6619999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5740000000000001},"min":{"1min":1.26,"5min":0.97599999999999998,"15min":0.83199999999999996},"max":{"1min":2.2519999999999998,"5min":2.7890000000000001,"15min":3.3620000000000001},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.6339999999999999,"15min":1.5209999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.77600000000000002},"max":{"1min":2.1059999999999999,"5min":10.956,"15min":10.956},"last":1.825}]}]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":776,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194536,"statfs":{"total":200000000,"available":199205075,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.93899999999999995,"15min":0.84399999999999997},"min":{"1min":0.42799999999999999,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.331},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.091,"15min":0.95799999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.645,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.1220000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.0489999999999999,"15min":0.96299999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.589,"5min":1.6899999999999999,"15min":1.859},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.99299999999999999,"15min":0.88700000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.36699999999999999},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.702}]},{"osd":3,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.139,"15min":1.0469999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.436},"max":{"1min":1.915,"5min":1.915,"15min":2.1059999999999999},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0960000000000001,"15min":1.0589999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.7110000000000001,"5min":1.9370000000000001,"15min":3.7010000000000001},"last":1.399}]},{"osd":4,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.3460000000000001,"15min":1.27},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7430000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.3799999999999999,"15min":1.256},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.829,"5min":1.903,"15min":2.0470000000000002},"last":1.7989999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4199999999999999,"15min":1.325},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.984,"5min":2.161,"15min":3.5750000000000002},"last":1.9490000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.4890000000000001,"15min":1.3919999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8320000000000001,"5min":1.964,"15min":2.3769999999999998},"last":1.6559999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.454,"15min":1.3260000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.905,"5min":2.0419999999999998,"15min":2.2050000000000001},"last":1.905},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.452,"15min":1.3149999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.63100000000000001},"max":{"1min":2.1040000000000001,"5min":2.121,"15min":2.121},"last":1.601}]},{"osd":7,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.524,"15min":1.399},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.2000000000000002,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.7549999999999999},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5329999999999999,"15min":1.4039999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.2469999999999999,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.839}]}]},{"osd":0,"up_from":13,"seq":55834574950,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1352,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193959,"statfs":{"total":200000000,"available":198614707,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81899999999999995,"15min":0.89000000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":1.3320000000000001,"15min":3.4420000000000002},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.083,"15min":1.0780000000000001},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.46500000000000002},"max":{"1min":1.3200000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.2130000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.85699999999999998,"15min":0.84199999999999997},"min":{"1min":0.44400000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.244,"5min":1.679,"15min":1.679},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.071,"5min":1.1359999999999999,"15min":1.1279999999999999},"min":{"1min":0.83999999999999997,"5min":0.503,"15min":0.437},"max":{"1min":1.446,"5min":1.929,"15min":1.929},"last":0.97499999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.135,"15min":1.234},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4419999999999999,"5min":1.98,"15min":2.7639999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3799999999999999,"15min":1.4710000000000001},"min":{"1min":0.71999999999999997,"5min":0.61299999999999999,"15min":0.58399999999999996},"max":{"1min":1.677,"5min":2.3639999999999999,"15min":3.5720000000000001},"last":1.3100000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.228,"15min":1.278},"min":{"1min":0.85999999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7410000000000001,"5min":1.8819999999999999,"15min":4.492},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.282,"15min":1.2589999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.518,"5min":1.927,"15min":1.927},"last":1.1719999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.256,"15min":1.373},"min":{"1min":0.83399999999999996,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.839,"5min":2.4510000000000001,"15min":4.2779999999999996},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.3049999999999999,"15min":1.3540000000000001},"min":{"1min":0.751,"5min":0.74399999999999999,"15min":0.73099999999999998},"max":{"1min":1.573,"5min":1.847,"15min":1.8740000000000001},"last":1.573}]},{"osd":6,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.4179999999999999,"15min":1.538},"min":{"1min":0.79900000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.613,"5min":2.0419999999999998,"15min":3.5299999999999998},"last":1.369},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.363,"15min":1.458},"min":{"1min":0.78000000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.665,"5min":3.1440000000000001,"15min":4.4349999999999996},"last":1.5549999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.524,"15min":1.6080000000000001},"min":{"1min":0.88,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.8120000000000001,"5min":2.2000000000000002,"15min":3.492},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.446,"15min":1.6080000000000001},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1869999999999998,"5min":3.2570000000000001,"15min":4.5419999999999998},"last":1.385}]}]}],"pool_statfs":[]}} 2023-10-18T09:03:34.061 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:03:34.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.303+0000 7f6316ec2640 1 -- 172.21.15.40:0/1351597588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6310151bd0 msgr2=0x7f63101586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:34.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.303+0000 7f6316ec2640 1 --2- 172.21.15.40:0/1351597588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6310151bd0 0x7f63101586a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6304005fd0 tx=0x7f630402be10 comp rx=0 tx=0).stop 2023-10-18T09:03:34.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.304+0000 7f6316ec2640 1 -- 172.21.15.40:0/1351597588 shutdown_connections 2023-10-18T09:03:34.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.304+0000 7f6316ec2640 1 --2- 172.21.15.40:0/1351597588 >> v2:172.21.15.144:3300/0 conn(0x7f63100b0bc0 0x7f63100b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:34.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.304+0000 7f6316ec2640 1 --2- 172.21.15.40:0/1351597588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6310151bd0 0x7f63101586a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:34.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.304+0000 7f6316ec2640 1 -- 172.21.15.40:0/1351597588 >> 172.21.15.40:0/1351597588 conn(0x7f631006cee0 msgr2=0x7f6310069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:34.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.304+0000 7f6316ec2640 1 -- 172.21.15.40:0/1351597588 shutdown_connections 2023-10-18T09:03:34.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.304+0000 7f6316ec2640 1 -- 172.21.15.40:0/1351597588 wait complete. 2023-10-18T09:03:34.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.306+0000 7f6316ec2640 1 Processor -- start 2023-10-18T09:03:34.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.306+0000 7f6316ec2640 1 -- start start 2023-10-18T09:03:34.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.308+0000 7f6316ec2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f631006b710 0x7f6310159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:34.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.308+0000 7f6316ec2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63100b0bc0 0x7f63100cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:34.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.308+0000 7f6316ec2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6310151bd0 0x7f63100cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:34.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.308+0000 7f6316ec2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631014fc30 con 0x7f631006b710 2023-10-18T09:03:34.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.308+0000 7f6316ec2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63100cc200 con 0x7f63100b0bc0 2023-10-18T09:03:34.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.308+0000 7f6316ec2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63100cc3f0 con 0x7f6310151bd0 2023-10-18T09:03:34.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.308+0000 7f6314c5e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f631006b710 0x7f6310159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:34.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.308+0000 7f6314c5e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f631006b710 0x7f6310159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49746/0 (socket says 172.21.15.40:49746) 2023-10-18T09:03:34.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.308+0000 7f6314c5e640 1 -- 172.21.15.40:0/1437810152 learned_addr learned my addr 172.21.15.40:0/1437810152 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:34.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.309+0000 7f630ffff640 1 --2- 172.21.15.40:0/1437810152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63100b0bc0 0x7f63100cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:34.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.309+0000 7f631545f640 1 --2- 172.21.15.40:0/1437810152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6310151bd0 0x7f63100cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:34.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.309+0000 7f6314c5e640 1 -- 172.21.15.40:0/1437810152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6310151bd0 msgr2=0x7f63100cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:34.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.309+0000 7f6314c5e640 1 --2- 172.21.15.40:0/1437810152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6310151bd0 0x7f63100cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:34.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.309+0000 7f6314c5e640 1 -- 172.21.15.40:0/1437810152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63100b0bc0 msgr2=0x7f63100cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:34.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.309+0000 7f6314c5e640 1 --2- 172.21.15.40:0/1437810152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63100b0bc0 0x7f63100cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:34.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.309+0000 7f6314c5e640 1 -- 172.21.15.40:0/1437810152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6304009560 con 0x7f631006b710 2023-10-18T09:03:34.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.309+0000 7f6314c5e640 1 --2- 172.21.15.40:0/1437810152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f631006b710 0x7f6310159000 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f6304009850 tx=0x7f630402da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.310+0000 7f630d7fa640 1 -- 172.21.15.40:0/1437810152 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630402de70 con 0x7f631006b710 2023-10-18T09:03:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.310+0000 7f6316ec2640 1 -- 172.21.15.40:0/1437810152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631014f290 con 0x7f631006b710 2023-10-18T09:03:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.310+0000 7f630d7fa640 1 -- 172.21.15.40:0/1437810152 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f63040035f0 con 0x7f631006b710 2023-10-18T09:03:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.310+0000 7f630d7fa640 1 -- 172.21.15.40:0/1437810152 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63040039a0 con 0x7f631006b710 2023-10-18T09:03:34.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.310+0000 7f6316ec2640 1 -- 172.21.15.40:0/1437810152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631014a1b0 con 0x7f631006b710 2023-10-18T09:03:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.311+0000 7f6316ec2640 1 -- 172.21.15.40:0/1437810152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631014fd70 con 0x7f631006b710 2023-10-18T09:03:34.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.313+0000 7f630d7fa640 1 -- 172.21.15.40:0/1437810152 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6304003b00 con 0x7f631006b710 2023-10-18T09:03:34.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.314+0000 7f630d7fa640 1 --2- 172.21.15.40:0/1437810152 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f62f406ae40 0x7f62f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:34.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.314+0000 7f630d7fa640 1 -- 172.21.15.40:0/1437810152 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63040ac2a0 con 0x7f631006b710 2023-10-18T09:03:34.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.314+0000 7f630ffff640 1 --2- 172.21.15.40:0/1437810152 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f62f406ae40 0x7f62f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:34.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.316+0000 7f630ffff640 1 --2- 172.21.15.40:0/1437810152 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f62f406ae40 0x7f62f406d300 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6310149ec0 tx=0x7f62fc005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:34.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.319+0000 7f630d7fa640 1 -- 172.21.15.40:0/1437810152 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f631014fd70 con 0x7f631006b710 2023-10-18T09:03:34.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.509+0000 7f6316ec2640 1 -- 172.21.15.40:0/1437810152 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6310151fb0 con 0x7f62f406ae40 2023-10-18T09:03:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.532+0000 7f630d7fa640 1 -- 172.21.15.40:0/1437810152 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282690 (secure 0 0 0) 0x7f62fc016070 con 0x7f62f406ae40 2023-10-18T09:03:34.540 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:34.541 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:03:34.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.538+0000 7f6316ec2640 1 -- 172.21.15.40:0/1437810152 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f62f406ae40 msgr2=0x7f62f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:34.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.538+0000 7f6316ec2640 1 --2- 172.21.15.40:0/1437810152 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f62f406ae40 0x7f62f406d300 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6310149ec0 tx=0x7f62fc005f90 comp rx=0 tx=0).stop 2023-10-18T09:03:34.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.538+0000 7f6316ec2640 1 -- 172.21.15.40:0/1437810152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f631006b710 msgr2=0x7f6310159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:34.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.538+0000 7f6316ec2640 1 --2- 172.21.15.40:0/1437810152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f631006b710 0x7f6310159000 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f6304009850 tx=0x7f630402da70 comp rx=0 tx=0).stop 2023-10-18T09:03:34.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.538+0000 7f6316ec2640 1 -- 172.21.15.40:0/1437810152 shutdown_connections 2023-10-18T09:03:34.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.538+0000 7f6316ec2640 1 --2- 172.21.15.40:0/1437810152 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f62f406ae40 0x7f62f406d300 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:34.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.538+0000 7f6316ec2640 1 --2- 172.21.15.40:0/1437810152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6310151bd0 0x7f63100cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:34.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.538+0000 7f6316ec2640 1 --2- 172.21.15.40:0/1437810152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63100b0bc0 0x7f63100cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:34.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.538+0000 7f6316ec2640 1 --2- 172.21.15.40:0/1437810152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f631006b710 0x7f6310159000 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:34.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.538+0000 7f6316ec2640 1 -- 172.21.15.40:0/1437810152 >> 172.21.15.40:0/1437810152 conn(0x7f631006cee0 msgr2=0x7f6310070150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.539+0000 7f6316ec2640 1 -- 172.21.15.40:0/1437810152 shutdown_connections 2023-10-18T09:03:34.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.539+0000 7f6316ec2640 1 -- 172.21.15.40:0/1437810152 wait complete. 2023-10-18T09:03:34.563 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2023-10-18T09:03:33.077810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7475,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555024,"statfs":{"total":1600000000,"available":1592345349,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.733523"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.358535+0000","last_change":"2023-10-18T09:03:24.371852+0000","last_active":"2023-10-18T09:03:25.358535+0000","last_peered":"2023-10-18T09:03:25.358535+0000","last_clean":"2023-10-18T09:03:25.358535+0000","last_became_active":"2023-10-18T09:03:24.371509+0000","last_became_peered":"2023-10-18T09:03:24.371509+0000","last_unstale":"2023-10-18T09:03:25.358535+0000","last_undegraded":"2023-10-18T09:03:25.358535+0000","last_fullsized":"2023-10-18T09:03:25.358535+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:35:23.094616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.356107+0000","last_change":"2023-10-18T09:03:24.370405+0000","last_active":"2023-10-18T09:03:25.356107+0000","last_peered":"2023-10-18T09:03:25.356107+0000","last_clean":"2023-10-18T09:03:25.356107+0000","last_became_active":"2023-10-18T09:03:24.369952+0000","last_became_peered":"2023-10-18T09:03:24.369952+0000","last_unstale":"2023-10-18T09:03:25.356107+0000","last_undegraded":"2023-10-18T09:03:25.356107+0000","last_fullsized":"2023-10-18T09:03:25.356107+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:09:00.288118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.350987+0000","last_change":"2023-10-18T09:03:24.371528+0000","last_active":"2023-10-18T09:03:25.350987+0000","last_peered":"2023-10-18T09:03:25.350987+0000","last_clean":"2023-10-18T09:03:25.350987+0000","last_became_active":"2023-10-18T09:03:24.370994+0000","last_became_peered":"2023-10-18T09:03:24.370994+0000","last_unstale":"2023-10-18T09:03:25.350987+0000","last_undegraded":"2023-10-18T09:03:25.350987+0000","last_fullsized":"2023-10-18T09:03:25.350987+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:13.219125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.355636+0000","last_change":"2023-10-18T09:03:24.368500+0000","last_active":"2023-10-18T09:03:25.355636+0000","last_peered":"2023-10-18T09:03:25.355636+0000","last_clean":"2023-10-18T09:03:25.355636+0000","last_became_active":"2023-10-18T09:03:24.363436+0000","last_became_peered":"2023-10-18T09:03:24.363436+0000","last_unstale":"2023-10-18T09:03:25.355636+0000","last_undegraded":"2023-10-18T09:03:25.355636+0000","last_fullsized":"2023-10-18T09:03:25.355636+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:33:09.714854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.356769+0000","last_change":"2023-10-18T09:03:24.377875+0000","last_active":"2023-10-18T09:03:25.356769+0000","last_peered":"2023-10-18T09:03:25.356769+0000","last_clean":"2023-10-18T09:03:25.356769+0000","last_became_active":"2023-10-18T09:03:24.377405+0000","last_became_peered":"2023-10-18T09:03:24.377405+0000","last_unstale":"2023-10-18T09:03:25.356769+0000","last_undegraded":"2023-10-18T09:03:25.356769+0000","last_fullsized":"2023-10-18T09:03:25.356769+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:55:28.487902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.356876+0000","last_change":"2023-10-18T09:03:24.381908+0000","last_active":"2023-10-18T09:03:25.356876+0000","last_peered":"2023-10-18T09:03:25.356876+0000","last_clean":"2023-10-18T09:03:25.356876+0000","last_became_active":"2023-10-18T09:03:24.381431+0000","last_became_peered":"2023-10-18T09:03:24.381431+0000","last_unstale":"2023-10-18T09:03:25.356876+0000","last_undegraded":"2023-10-18T09:03:25.356876+0000","last_fullsized":"2023-10-18T09:03:25.356876+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:43.432347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.350069+0000","last_change":"2023-10-18T09:03:24.368235+0000","last_active":"2023-10-18T09:03:25.350069+0000","last_peered":"2023-10-18T09:03:25.350069+0000","last_clean":"2023-10-18T09:03:25.350069+0000","last_became_active":"2023-10-18T09:03:24.367739+0000","last_became_peered":"2023-10-18T09:03:24.367739+0000","last_unstale":"2023-10-18T09:03:25.350069+0000","last_undegraded":"2023-10-18T09:03:25.350069+0000","last_fullsized":"2023-10-18T09:03:25.350069+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:44:58.431996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.351462+0000","last_change":"2023-10-18T09:03:24.373509+0000","last_active":"2023-10-18T09:03:25.351462+0000","last_peered":"2023-10-18T09:03:25.351462+0000","last_clean":"2023-10-18T09:03:25.351462+0000","last_became_active":"2023-10-18T09:03:24.372325+0000","last_became_peered":"2023-10-18T09:03:24.372325+0000","last_unstale":"2023-10-18T09:03:25.351462+0000","last_undegraded":"2023-10-18T09:03:25.351462+0000","last_fullsized":"2023-10-18T09:03:25.351462+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:26:13.222732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.355981+0000","last_change":"2023-10-18T09:03:24.368758+0000","last_active":"2023-10-18T09:03:25.355981+0000","last_peered":"2023-10-18T09:03:25.355981+0000","last_clean":"2023-10-18T09:03:25.355981+0000","last_became_active":"2023-10-18T09:03:24.368193+0000","last_became_peered":"2023-10-18T09:03:24.368193+0000","last_unstale":"2023-10-18T09:03:25.355981+0000","last_undegraded":"2023-10-18T09:03:25.355981+0000","last_fullsized":"2023-10-18T09:03:25.355981+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:13:21.900955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.359295+0000","last_change":"2023-10-18T09:03:24.371194+0000","last_active":"2023-10-18T09:03:25.359295+0000","last_peered":"2023-10-18T09:03:25.359295+0000","last_clean":"2023-10-18T09:03:25.359295+0000","last_became_active":"2023-10-18T09:03:24.370564+0000","last_became_peered":"2023-10-18T09:03:24.370564+0000","last_unstale":"2023-10-18T09:03:25.359295+0000","last_undegraded":"2023-10-18T09:03:25.359295+0000","last_fullsized":"2023-10-18T09:03:25.359295+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:15:16.874098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.355785+0000","last_change":"2023-10-18T09:03:24.367861+0000","last_active":"2023-10-18T09:03:25.355785+0000","last_peered":"2023-10-18T09:03:25.355785+0000","last_clean":"2023-10-18T09:03:25.355785+0000","last_became_active":"2023-10-18T09:03:24.367589+0000","last_became_peered":"2023-10-18T09:03:24.367589+0000","last_unstale":"2023-10-18T09:03:25.355785+0000","last_undegraded":"2023-10-18T09:03:25.355785+0000","last_fullsized":"2023-10-18T09:03:25.355785+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:17.983120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.360247+0000","last_change":"2023-10-18T09:03:24.380444+0000","last_active":"2023-10-18T09:03:25.360247+0000","last_peered":"2023-10-18T09:03:25.360247+0000","last_clean":"2023-10-18T09:03:25.360247+0000","last_became_active":"2023-10-18T09:03:24.380210+0000","last_became_peered":"2023-10-18T09:03:24.380210+0000","last_unstale":"2023-10-18T09:03:25.360247+0000","last_undegraded":"2023-10-18T09:03:25.360247+0000","last_fullsized":"2023-10-18T09:03:25.360247+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:38:34.099637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.351768+0000","last_change":"2023-10-18T09:03:24.378634+0000","last_active":"2023-10-18T09:03:25.351768+0000","last_peered":"2023-10-18T09:03:25.351768+0000","last_clean":"2023-10-18T09:03:25.351768+0000","last_became_active":"2023-10-18T09:03:24.378364+0000","last_became_peered":"2023-10-18T09:03:24.378364+0000","last_unstale":"2023-10-18T09:03:25.351768+0000","last_undegraded":"2023-10-18T09:03:25.351768+0000","last_fullsized":"2023-10-18T09:03:25.351768+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:48:34.905257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.349596+0000","last_change":"2023-10-18T09:03:24.371034+0000","last_active":"2023-10-18T09:03:25.349596+0000","last_peered":"2023-10-18T09:03:25.349596+0000","last_clean":"2023-10-18T09:03:25.349596+0000","last_became_active":"2023-10-18T09:03:24.370294+0000","last_became_peered":"2023-10-18T09:03:24.370294+0000","last_unstale":"2023-10-18T09:03:25.349596+0000","last_undegraded":"2023-10-18T09:03:25.349596+0000","last_fullsized":"2023-10-18T09:03:25.349596+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:10.543548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.355741+0000","last_change":"2023-10-18T09:03:24.380910+0000","last_active":"2023-10-18T09:03:25.355741+0000","last_peered":"2023-10-18T09:03:25.355741+0000","last_clean":"2023-10-18T09:03:25.355741+0000","last_became_active":"2023-10-18T09:03:24.380390+0000","last_became_peered":"2023-10-18T09:03:24.380390+0000","last_unstale":"2023-10-18T09:03:25.355741+0000","last_undegraded":"2023-10-18T09:03:25.355741+0000","last_fullsized":"2023-10-18T09:03:25.355741+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:24:43.781843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.358689+0000","last_change":"2023-10-18T09:03:24.371024+0000","last_active":"2023-10-18T09:03:25.358689+0000","last_peered":"2023-10-18T09:03:25.358689+0000","last_clean":"2023-10-18T09:03:25.358689+0000","last_became_active":"2023-10-18T09:03:24.370196+0000","last_became_peered":"2023-10-18T09:03:24.370196+0000","last_unstale":"2023-10-18T09:03:25.358689+0000","last_undegraded":"2023-10-18T09:03:25.358689+0000","last_fullsized":"2023-10-18T09:03:25.358689+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:18:25.271832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.356436+0000","last_change":"2023-10-18T09:03:24.367257+0000","last_active":"2023-10-18T09:03:25.356436+0000","last_peered":"2023-10-18T09:03:25.356436+0000","last_clean":"2023-10-18T09:03:25.356436+0000","last_became_active":"2023-10-18T09:03:24.367021+0000","last_became_peered":"2023-10-18T09:03:24.367021+0000","last_unstale":"2023-10-18T09:03:25.356436+0000","last_undegraded":"2023-10-18T09:03:25.356436+0000","last_fullsized":"2023-10-18T09:03:25.356436+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:10.543548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.356366+0000","last_change":"2023-10-18T09:03:24.370322+0000","last_active":"2023-10-18T09:03:25.356366+0000","last_peered":"2023-10-18T09:03:25.356366+0000","last_clean":"2023-10-18T09:03:25.356366+0000","last_became_active":"2023-10-18T09:03:24.369927+0000","last_became_peered":"2023-10-18T09:03:24.369927+0000","last_unstale":"2023-10-18T09:03:25.356366+0000","last_undegraded":"2023-10-18T09:03:25.356366+0000","last_fullsized":"2023-10-18T09:03:25.356366+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:15:22.978564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.349984+0000","last_change":"2023-10-18T09:03:24.373744+0000","last_active":"2023-10-18T09:03:25.349984+0000","last_peered":"2023-10-18T09:03:25.349984+0000","last_clean":"2023-10-18T09:03:25.349984+0000","last_became_active":"2023-10-18T09:03:24.373488+0000","last_became_peered":"2023-10-18T09:03:24.373488+0000","last_unstale":"2023-10-18T09:03:25.349984+0000","last_undegraded":"2023-10-18T09:03:25.349984+0000","last_fullsized":"2023-10-18T09:03:25.349984+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:15:22.978564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.356097+0000","last_change":"2023-10-18T09:03:24.363072+0000","last_active":"2023-10-18T09:03:25.356097+0000","last_peered":"2023-10-18T09:03:25.356097+0000","last_clean":"2023-10-18T09:03:25.356097+0000","last_became_active":"2023-10-18T09:03:24.362780+0000","last_became_peered":"2023-10-18T09:03:24.362780+0000","last_unstale":"2023-10-18T09:03:25.356097+0000","last_undegraded":"2023-10-18T09:03:25.356097+0000","last_fullsized":"2023-10-18T09:03:25.356097+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:22:59.934981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.356345+0000","last_change":"2023-10-18T09:03:24.363762+0000","last_active":"2023-10-18T09:03:25.356345+0000","last_peered":"2023-10-18T09:03:25.356345+0000","last_clean":"2023-10-18T09:03:25.356345+0000","last_became_active":"2023-10-18T09:03:24.362308+0000","last_became_peered":"2023-10-18T09:03:24.362308+0000","last_unstale":"2023-10-18T09:03:25.356345+0000","last_undegraded":"2023-10-18T09:03:25.356345+0000","last_fullsized":"2023-10-18T09:03:25.356345+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:26:40.521285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.346429+0000","last_change":"2023-10-18T09:03:24.376004+0000","last_active":"2023-10-18T09:03:25.346429+0000","last_peered":"2023-10-18T09:03:25.346429+0000","last_clean":"2023-10-18T09:03:25.346429+0000","last_became_active":"2023-10-18T09:03:24.375694+0000","last_became_peered":"2023-10-18T09:03:24.375694+0000","last_unstale":"2023-10-18T09:03:25.346429+0000","last_undegraded":"2023-10-18T09:03:25.346429+0000","last_fullsized":"2023-10-18T09:03:25.346429+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:07:38.808707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.355781+0000","last_change":"2023-10-18T09:03:24.370959+0000","last_active":"2023-10-18T09:03:25.355781+0000","last_peered":"2023-10-18T09:03:25.355781+0000","last_clean":"2023-10-18T09:03:25.355781+0000","last_became_active":"2023-10-18T09:03:24.370661+0000","last_became_peered":"2023-10-18T09:03:24.370661+0000","last_unstale":"2023-10-18T09:03:25.355781+0000","last_undegraded":"2023-10-18T09:03:25.355781+0000","last_fullsized":"2023-10-18T09:03:25.355781+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:31.969146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.346702+0000","last_change":"2023-10-18T09:03:24.364680+0000","last_active":"2023-10-18T09:03:25.346702+0000","last_peered":"2023-10-18T09:03:25.346702+0000","last_clean":"2023-10-18T09:03:25.346702+0000","last_became_active":"2023-10-18T09:03:24.364143+0000","last_became_peered":"2023-10-18T09:03:24.364143+0000","last_unstale":"2023-10-18T09:03:25.346702+0000","last_undegraded":"2023-10-18T09:03:25.346702+0000","last_fullsized":"2023-10-18T09:03:25.346702+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:21:44.950207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.356435+0000","last_change":"2023-10-18T09:03:24.366494+0000","last_active":"2023-10-18T09:03:25.356435+0000","last_peered":"2023-10-18T09:03:25.356435+0000","last_clean":"2023-10-18T09:03:25.356435+0000","last_became_active":"2023-10-18T09:03:24.366248+0000","last_became_peered":"2023-10-18T09:03:24.366248+0000","last_unstale":"2023-10-18T09:03:25.356435+0000","last_undegraded":"2023-10-18T09:03:25.356435+0000","last_fullsized":"2023-10-18T09:03:25.356435+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:48.037396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.358838+0000","last_change":"2023-10-18T09:03:24.372556+0000","last_active":"2023-10-18T09:03:25.358838+0000","last_peered":"2023-10-18T09:03:25.358838+0000","last_clean":"2023-10-18T09:03:25.358838+0000","last_became_active":"2023-10-18T09:03:24.372285+0000","last_became_peered":"2023-10-18T09:03:24.372285+0000","last_unstale":"2023-10-18T09:03:25.358838+0000","last_undegraded":"2023-10-18T09:03:25.358838+0000","last_fullsized":"2023-10-18T09:03:25.358838+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:26.362262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.356096+0000","last_change":"2023-10-18T09:03:24.374132+0000","last_active":"2023-10-18T09:03:25.356096+0000","last_peered":"2023-10-18T09:03:25.356096+0000","last_clean":"2023-10-18T09:03:25.356096+0000","last_became_active":"2023-10-18T09:03:24.373594+0000","last_became_peered":"2023-10-18T09:03:24.373594+0000","last_unstale":"2023-10-18T09:03:25.356096+0000","last_undegraded":"2023-10-18T09:03:25.356096+0000","last_fullsized":"2023-10-18T09:03:25.356096+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:02.115135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.346564+0000","last_change":"2023-10-18T09:03:24.371719+0000","last_active":"2023-10-18T09:03:25.346564+0000","last_peered":"2023-10-18T09:03:25.346564+0000","last_clean":"2023-10-18T09:03:25.346564+0000","last_became_active":"2023-10-18T09:03:24.371095+0000","last_became_peered":"2023-10-18T09:03:24.371095+0000","last_unstale":"2023-10-18T09:03:25.346564+0000","last_undegraded":"2023-10-18T09:03:25.346564+0000","last_fullsized":"2023-10-18T09:03:25.346564+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:30.212162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.355840+0000","last_change":"2023-10-18T09:03:24.367460+0000","last_active":"2023-10-18T09:03:25.355840+0000","last_peered":"2023-10-18T09:03:25.355840+0000","last_clean":"2023-10-18T09:03:25.355840+0000","last_became_active":"2023-10-18T09:03:24.364592+0000","last_became_peered":"2023-10-18T09:03:24.364592+0000","last_unstale":"2023-10-18T09:03:25.355840+0000","last_undegraded":"2023-10-18T09:03:25.355840+0000","last_fullsized":"2023-10-18T09:03:25.355840+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:14:13.807602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.350984+0000","last_change":"2023-10-18T09:03:24.379415+0000","last_active":"2023-10-18T09:03:25.350984+0000","last_peered":"2023-10-18T09:03:25.350984+0000","last_clean":"2023-10-18T09:03:25.350984+0000","last_became_active":"2023-10-18T09:03:24.379135+0000","last_became_peered":"2023-10-18T09:03:24.379135+0000","last_unstale":"2023-10-18T09:03:25.350984+0000","last_undegraded":"2023-10-18T09:03:25.350984+0000","last_fullsized":"2023-10-18T09:03:25.350984+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:03:01.008004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.358969+0000","last_change":"2023-10-18T09:03:24.364700+0000","last_active":"2023-10-18T09:03:25.358969+0000","last_peered":"2023-10-18T09:03:25.358969+0000","last_clean":"2023-10-18T09:03:25.358969+0000","last_became_active":"2023-10-18T09:03:24.364300+0000","last_became_peered":"2023-10-18T09:03:24.364300+0000","last_unstale":"2023-10-18T09:03:25.358969+0000","last_undegraded":"2023-10-18T09:03:25.358969+0000","last_fullsized":"2023-10-18T09:03:25.358969+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:15:31.768036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.356642+0000","last_change":"2023-10-18T09:03:24.380893+0000","last_active":"2023-10-18T09:03:25.356642+0000","last_peered":"2023-10-18T09:03:25.356642+0000","last_clean":"2023-10-18T09:03:25.356642+0000","last_became_active":"2023-10-18T09:03:24.380633+0000","last_became_peered":"2023-10-18T09:03:24.380633+0000","last_unstale":"2023-10-18T09:03:25.356642+0000","last_undegraded":"2023-10-18T09:03:25.356642+0000","last_fullsized":"2023-10-18T09:03:25.356642+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:17.899005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.357059+0000","last_change":"2023-10-18T09:03:24.374257+0000","last_active":"2023-10-18T09:03:25.357059+0000","last_peered":"2023-10-18T09:03:25.357059+0000","last_clean":"2023-10-18T09:03:25.357059+0000","last_became_active":"2023-10-18T09:03:24.372800+0000","last_became_peered":"2023-10-18T09:03:24.372800+0000","last_unstale":"2023-10-18T09:03:25.357059+0000","last_undegraded":"2023-10-18T09:03:25.357059+0000","last_fullsized":"2023-10-18T09:03:25.357059+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:39.039342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"143'4","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.395307+0000","last_change":"2023-10-18T09:03:23.329530+0000","last_active":"2023-10-18T09:03:24.395307+0000","last_peered":"2023-10-18T09:03:24.395307+0000","last_clean":"2023-10-18T09:03:24.395307+0000","last_became_active":"2023-10-18T09:03:23.329181+0000","last_became_peered":"2023-10-18T09:03:23.329181+0000","last_unstale":"2023-10-18T09:03:24.395307+0000","last_undegraded":"2023-10-18T09:03:24.395307+0000","last_fullsized":"2023-10-18T09:03:24.395307+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:23.860592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.356728+0000","last_change":"2023-10-18T09:03:24.372759+0000","last_active":"2023-10-18T09:03:25.356728+0000","last_peered":"2023-10-18T09:03:25.356728+0000","last_clean":"2023-10-18T09:03:25.356728+0000","last_became_active":"2023-10-18T09:03:24.372497+0000","last_became_peered":"2023-10-18T09:03:24.372497+0000","last_unstale":"2023-10-18T09:03:25.356728+0000","last_undegraded":"2023-10-18T09:03:25.356728+0000","last_fullsized":"2023-10-18T09:03:25.356728+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:12:16.648188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"143'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.093036+0000","last_change":"2023-10-18T09:03:23.331319+0000","last_active":"2023-10-18T09:03:25.093036+0000","last_peered":"2023-10-18T09:03:25.093036+0000","last_clean":"2023-10-18T09:03:25.093036+0000","last_became_active":"2023-10-18T09:03:23.331015+0000","last_became_peered":"2023-10-18T09:03:23.331015+0000","last_unstale":"2023-10-18T09:03:25.093036+0000","last_undegraded":"2023-10-18T09:03:25.093036+0000","last_fullsized":"2023-10-18T09:03:25.093036+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:40.692371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.357003+0000","last_change":"2023-10-18T09:03:24.381724+0000","last_active":"2023-10-18T09:03:25.357003+0000","last_peered":"2023-10-18T09:03:25.357003+0000","last_clean":"2023-10-18T09:03:25.357003+0000","last_became_active":"2023-10-18T09:03:24.380696+0000","last_became_peered":"2023-10-18T09:03:24.380696+0000","last_unstale":"2023-10-18T09:03:25.357003+0000","last_undegraded":"2023-10-18T09:03:25.357003+0000","last_fullsized":"2023-10-18T09:03:25.357003+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:42:51.246138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.395811+0000","last_change":"2023-10-18T09:03:23.330634+0000","last_active":"2023-10-18T09:03:24.395811+0000","last_peered":"2023-10-18T09:03:24.395811+0000","last_clean":"2023-10-18T09:03:24.395811+0000","last_became_active":"2023-10-18T09:03:23.329893+0000","last_became_peered":"2023-10-18T09:03:23.329893+0000","last_unstale":"2023-10-18T09:03:24.395811+0000","last_undegraded":"2023-10-18T09:03:24.395811+0000","last_fullsized":"2023-10-18T09:03:24.395811+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:23:15.710814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.351069+0000","last_change":"2023-10-18T09:03:24.373601+0000","last_active":"2023-10-18T09:03:25.351069+0000","last_peered":"2023-10-18T09:03:25.351069+0000","last_clean":"2023-10-18T09:03:25.351069+0000","last_became_active":"2023-10-18T09:03:24.373369+0000","last_became_peered":"2023-10-18T09:03:24.373369+0000","last_unstale":"2023-10-18T09:03:25.351069+0000","last_undegraded":"2023-10-18T09:03:25.351069+0000","last_fullsized":"2023-10-18T09:03:25.351069+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:12:27.988841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.395890+0000","last_change":"2023-10-18T09:03:23.328512+0000","last_active":"2023-10-18T09:03:24.395890+0000","last_peered":"2023-10-18T09:03:24.395890+0000","last_clean":"2023-10-18T09:03:24.395890+0000","last_became_active":"2023-10-18T09:03:23.328090+0000","last_became_peered":"2023-10-18T09:03:23.328090+0000","last_unstale":"2023-10-18T09:03:24.395890+0000","last_undegraded":"2023-10-18T09:03:24.395890+0000","last_fullsized":"2023-10-18T09:03:24.395890+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:58:33.816682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.358058+0000","last_change":"2023-10-18T09:03:24.365173+0000","last_active":"2023-10-18T09:03:25.358058+0000","last_peered":"2023-10-18T09:03:25.358058+0000","last_clean":"2023-10-18T09:03:25.358058+0000","last_became_active":"2023-10-18T09:03:24.362687+0000","last_became_peered":"2023-10-18T09:03:24.362687+0000","last_unstale":"2023-10-18T09:03:25.358058+0000","last_undegraded":"2023-10-18T09:03:25.358058+0000","last_fullsized":"2023-10-18T09:03:25.358058+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:58:34.070107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"143'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.394291+0000","last_change":"2023-10-18T09:03:23.329851+0000","last_active":"2023-10-18T09:03:24.394291+0000","last_peered":"2023-10-18T09:03:24.394291+0000","last_clean":"2023-10-18T09:03:24.394291+0000","last_became_active":"2023-10-18T09:03:23.329208+0000","last_became_peered":"2023-10-18T09:03:23.329208+0000","last_unstale":"2023-10-18T09:03:24.394291+0000","last_undegraded":"2023-10-18T09:03:24.394291+0000","last_fullsized":"2023-10-18T09:03:24.394291+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:01.110328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.356778+0000","last_change":"2023-10-18T09:03:24.370233+0000","last_active":"2023-10-18T09:03:25.356778+0000","last_peered":"2023-10-18T09:03:25.356778+0000","last_clean":"2023-10-18T09:03:25.356778+0000","last_became_active":"2023-10-18T09:03:24.369656+0000","last_became_peered":"2023-10-18T09:03:24.369656+0000","last_unstale":"2023-10-18T09:03:25.356778+0000","last_undegraded":"2023-10-18T09:03:25.356778+0000","last_fullsized":"2023-10-18T09:03:25.356778+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:12:16.648188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"143'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.082979+0000","last_change":"2023-10-18T09:03:23.329790+0000","last_active":"2023-10-18T09:03:25.082979+0000","last_peered":"2023-10-18T09:03:25.082979+0000","last_clean":"2023-10-18T09:03:25.082979+0000","last_became_active":"2023-10-18T09:03:23.325710+0000","last_became_peered":"2023-10-18T09:03:23.325710+0000","last_unstale":"2023-10-18T09:03:25.082979+0000","last_undegraded":"2023-10-18T09:03:25.082979+0000","last_fullsized":"2023-10-18T09:03:25.082979+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:06:18.861877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.359665+0000","last_change":"2023-10-18T09:03:24.377431+0000","last_active":"2023-10-18T09:03:25.359665+0000","last_peered":"2023-10-18T09:03:25.359665+0000","last_clean":"2023-10-18T09:03:25.359665+0000","last_became_active":"2023-10-18T09:03:24.376534+0000","last_became_peered":"2023-10-18T09:03:24.376534+0000","last_unstale":"2023-10-18T09:03:25.359665+0000","last_undegraded":"2023-10-18T09:03:25.359665+0000","last_fullsized":"2023-10-18T09:03:25.359665+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:35:08.407250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.347323+0000","last_change":"2023-10-18T09:03:23.318325+0000","last_active":"2023-10-18T09:03:24.347323+0000","last_peered":"2023-10-18T09:03:24.347323+0000","last_clean":"2023-10-18T09:03:24.347323+0000","last_became_active":"2023-10-18T09:03:23.316556+0000","last_became_peered":"2023-10-18T09:03:23.316556+0000","last_unstale":"2023-10-18T09:03:24.347323+0000","last_undegraded":"2023-10-18T09:03:24.347323+0000","last_fullsized":"2023-10-18T09:03:24.347323+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:27.888562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.349376+0000","last_change":"2023-10-18T09:03:24.369791+0000","last_active":"2023-10-18T09:03:25.349376+0000","last_peered":"2023-10-18T09:03:25.349376+0000","last_clean":"2023-10-18T09:03:25.349376+0000","last_became_active":"2023-10-18T09:03:24.369399+0000","last_became_peered":"2023-10-18T09:03:24.369399+0000","last_unstale":"2023-10-18T09:03:25.349376+0000","last_undegraded":"2023-10-18T09:03:25.349376+0000","last_fullsized":"2023-10-18T09:03:25.349376+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:03.804122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.358965+0000","last_change":"2023-10-18T09:03:23.334041+0000","last_active":"2023-10-18T09:03:24.358965+0000","last_peered":"2023-10-18T09:03:24.358965+0000","last_clean":"2023-10-18T09:03:24.358965+0000","last_became_active":"2023-10-18T09:03:23.333830+0000","last_became_peered":"2023-10-18T09:03:23.333830+0000","last_unstale":"2023-10-18T09:03:24.358965+0000","last_undegraded":"2023-10-18T09:03:24.358965+0000","last_fullsized":"2023-10-18T09:03:24.358965+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:37:49.381700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.349210+0000","last_change":"2023-10-18T09:03:24.376790+0000","last_active":"2023-10-18T09:03:25.349210+0000","last_peered":"2023-10-18T09:03:25.349210+0000","last_clean":"2023-10-18T09:03:25.349210+0000","last_became_active":"2023-10-18T09:03:24.376590+0000","last_became_peered":"2023-10-18T09:03:24.376590+0000","last_unstale":"2023-10-18T09:03:25.349210+0000","last_undegraded":"2023-10-18T09:03:25.349210+0000","last_fullsized":"2023-10-18T09:03:25.349210+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:42:51.246138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.360012+0000","last_change":"2023-10-18T09:03:23.328337+0000","last_active":"2023-10-18T09:03:24.360012+0000","last_peered":"2023-10-18T09:03:24.360012+0000","last_clean":"2023-10-18T09:03:24.360012+0000","last_became_active":"2023-10-18T09:03:23.325594+0000","last_became_peered":"2023-10-18T09:03:23.325594+0000","last_unstale":"2023-10-18T09:03:24.360012+0000","last_undegraded":"2023-10-18T09:03:24.360012+0000","last_fullsized":"2023-10-18T09:03:24.360012+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:54.483816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.357628+0000","last_change":"2023-10-18T09:03:24.365816+0000","last_active":"2023-10-18T09:03:25.357628+0000","last_peered":"2023-10-18T09:03:25.357628+0000","last_clean":"2023-10-18T09:03:25.357628+0000","last_became_active":"2023-10-18T09:03:24.365511+0000","last_became_peered":"2023-10-18T09:03:24.365511+0000","last_unstale":"2023-10-18T09:03:25.357628+0000","last_undegraded":"2023-10-18T09:03:25.357628+0000","last_fullsized":"2023-10-18T09:03:25.357628+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:13.219125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.083057+0000","last_change":"2023-10-18T09:03:23.325401+0000","last_active":"2023-10-18T09:03:25.083057+0000","last_peered":"2023-10-18T09:03:25.083057+0000","last_clean":"2023-10-18T09:03:25.083057+0000","last_became_active":"2023-10-18T09:03:23.324277+0000","last_became_peered":"2023-10-18T09:03:23.324277+0000","last_unstale":"2023-10-18T09:03:25.083057+0000","last_undegraded":"2023-10-18T09:03:25.083057+0000","last_fullsized":"2023-10-18T09:03:25.083057+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:06:07.682599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.346895+0000","last_change":"2023-10-18T09:03:24.350913+0000","last_active":"2023-10-18T09:03:25.346895+0000","last_peered":"2023-10-18T09:03:25.346895+0000","last_clean":"2023-10-18T09:03:25.346895+0000","last_became_active":"2023-10-18T09:03:24.350214+0000","last_became_peered":"2023-10-18T09:03:24.350214+0000","last_unstale":"2023-10-18T09:03:25.346895+0000","last_undegraded":"2023-10-18T09:03:25.346895+0000","last_fullsized":"2023-10-18T09:03:25.346895+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:06:05.212554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"143'3","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.394507+0000","last_change":"2023-10-18T09:03:23.331283+0000","last_active":"2023-10-18T09:03:24.394507+0000","last_peered":"2023-10-18T09:03:24.394507+0000","last_clean":"2023-10-18T09:03:24.394507+0000","last_became_active":"2023-10-18T09:03:23.329500+0000","last_became_peered":"2023-10-18T09:03:23.329500+0000","last_unstale":"2023-10-18T09:03:24.394507+0000","last_undegraded":"2023-10-18T09:03:24.394507+0000","last_fullsized":"2023-10-18T09:03:24.394507+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:45:27.534355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.349314+0000","last_change":"2023-10-18T09:03:24.373081+0000","last_active":"2023-10-18T09:03:25.349314+0000","last_peered":"2023-10-18T09:03:25.349314+0000","last_clean":"2023-10-18T09:03:25.349314+0000","last_became_active":"2023-10-18T09:03:24.372855+0000","last_became_peered":"2023-10-18T09:03:24.372855+0000","last_unstale":"2023-10-18T09:03:25.349314+0000","last_undegraded":"2023-10-18T09:03:25.349314+0000","last_fullsized":"2023-10-18T09:03:25.349314+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:39.039342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"143'6","reported_seq":18,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.398830+0000","last_change":"2023-10-18T09:03:23.334514+0000","last_active":"2023-10-18T09:03:24.398830+0000","last_peered":"2023-10-18T09:03:24.398830+0000","last_clean":"2023-10-18T09:03:24.398830+0000","last_became_active":"2023-10-18T09:03:23.334268+0000","last_became_peered":"2023-10-18T09:03:23.334268+0000","last_unstale":"2023-10-18T09:03:24.398830+0000","last_undegraded":"2023-10-18T09:03:24.398830+0000","last_fullsized":"2023-10-18T09:03:24.398830+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:32:54.930401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.347319+0000","last_change":"2023-10-18T09:03:24.367289+0000","last_active":"2023-10-18T09:03:25.347319+0000","last_peered":"2023-10-18T09:03:25.347319+0000","last_clean":"2023-10-18T09:03:25.347319+0000","last_became_active":"2023-10-18T09:03:24.366724+0000","last_became_peered":"2023-10-18T09:03:24.366724+0000","last_unstale":"2023-10-18T09:03:25.347319+0000","last_undegraded":"2023-10-18T09:03:25.347319+0000","last_fullsized":"2023-10-18T09:03:25.347319+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:19:03.820837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.393533+0000","last_change":"2023-10-18T09:03:23.323723+0000","last_active":"2023-10-18T09:03:24.393533+0000","last_peered":"2023-10-18T09:03:24.393533+0000","last_clean":"2023-10-18T09:03:24.393533+0000","last_became_active":"2023-10-18T09:03:23.322838+0000","last_became_peered":"2023-10-18T09:03:23.322838+0000","last_unstale":"2023-10-18T09:03:24.393533+0000","last_undegraded":"2023-10-18T09:03:24.393533+0000","last_fullsized":"2023-10-18T09:03:24.393533+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:44:59.580677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.347054+0000","last_change":"2023-10-18T09:03:24.374652+0000","last_active":"2023-10-18T09:03:25.347054+0000","last_peered":"2023-10-18T09:03:25.347054+0000","last_clean":"2023-10-18T09:03:25.347054+0000","last_became_active":"2023-10-18T09:03:24.366671+0000","last_became_peered":"2023-10-18T09:03:24.366671+0000","last_unstale":"2023-10-18T09:03:25.347054+0000","last_undegraded":"2023-10-18T09:03:25.347054+0000","last_fullsized":"2023-10-18T09:03:25.347054+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:37.026471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.357672+0000","last_change":"2023-10-18T09:03:23.334809+0000","last_active":"2023-10-18T09:03:24.357672+0000","last_peered":"2023-10-18T09:03:24.357672+0000","last_clean":"2023-10-18T09:03:24.357672+0000","last_became_active":"2023-10-18T09:03:23.334509+0000","last_became_peered":"2023-10-18T09:03:23.334509+0000","last_unstale":"2023-10-18T09:03:24.357672+0000","last_undegraded":"2023-10-18T09:03:24.357672+0000","last_fullsized":"2023-10-18T09:03:24.357672+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:14:12.802795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.346664+0000","last_change":"2023-10-18T09:03:24.353278+0000","last_active":"2023-10-18T09:03:25.346664+0000","last_peered":"2023-10-18T09:03:25.346664+0000","last_clean":"2023-10-18T09:03:25.346664+0000","last_became_active":"2023-10-18T09:03:24.352158+0000","last_became_peered":"2023-10-18T09:03:24.352158+0000","last_unstale":"2023-10-18T09:03:25.346664+0000","last_undegraded":"2023-10-18T09:03:25.346664+0000","last_fullsized":"2023-10-18T09:03:25.346664+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:19:56.695387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"143'8","reported_seq":20,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.396925+0000","last_change":"2023-10-18T09:03:23.332284+0000","last_active":"2023-10-18T09:03:24.396925+0000","last_peered":"2023-10-18T09:03:24.396925+0000","last_clean":"2023-10-18T09:03:24.396925+0000","last_became_active":"2023-10-18T09:03:23.332040+0000","last_became_peered":"2023-10-18T09:03:23.332040+0000","last_unstale":"2023-10-18T09:03:24.396925+0000","last_undegraded":"2023-10-18T09:03:24.396925+0000","last_fullsized":"2023-10-18T09:03:24.396925+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:05:29.476796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.357108+0000","last_change":"2023-10-18T09:03:24.373025+0000","last_active":"2023-10-18T09:03:25.357108+0000","last_peered":"2023-10-18T09:03:25.357108+0000","last_clean":"2023-10-18T09:03:25.357108+0000","last_became_active":"2023-10-18T09:03:24.372660+0000","last_became_peered":"2023-10-18T09:03:24.372660+0000","last_unstale":"2023-10-18T09:03:25.357108+0000","last_undegraded":"2023-10-18T09:03:25.357108+0000","last_fullsized":"2023-10-18T09:03:25.357108+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:09.230491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"143'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.396079+0000","last_change":"2023-10-18T09:03:23.332929+0000","last_active":"2023-10-18T09:03:24.396079+0000","last_peered":"2023-10-18T09:03:24.396079+0000","last_clean":"2023-10-18T09:03:24.396079+0000","last_became_active":"2023-10-18T09:03:23.332508+0000","last_became_peered":"2023-10-18T09:03:23.332508+0000","last_unstale":"2023-10-18T09:03:24.396079+0000","last_undegraded":"2023-10-18T09:03:24.396079+0000","last_fullsized":"2023-10-18T09:03:24.396079+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:12.214318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.357263+0000","last_change":"2023-10-18T09:03:24.378670+0000","last_active":"2023-10-18T09:03:25.357263+0000","last_peered":"2023-10-18T09:03:25.357263+0000","last_clean":"2023-10-18T09:03:25.357263+0000","last_became_active":"2023-10-18T09:03:24.378424+0000","last_became_peered":"2023-10-18T09:03:24.378424+0000","last_unstale":"2023-10-18T09:03:25.357263+0000","last_undegraded":"2023-10-18T09:03:25.357263+0000","last_fullsized":"2023-10-18T09:03:25.357263+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:35:03.804122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.396068+0000","last_change":"2023-10-18T09:03:23.330153+0000","last_active":"2023-10-18T09:03:24.396068+0000","last_peered":"2023-10-18T09:03:24.396068+0000","last_clean":"2023-10-18T09:03:24.396068+0000","last_became_active":"2023-10-18T09:03:23.325376+0000","last_became_peered":"2023-10-18T09:03:23.325376+0000","last_unstale":"2023-10-18T09:03:24.396068+0000","last_undegraded":"2023-10-18T09:03:24.396068+0000","last_fullsized":"2023-10-18T09:03:24.396068+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:16:09.490436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.350086+0000","last_change":"2023-10-18T09:03:24.374330+0000","last_active":"2023-10-18T09:03:25.350086+0000","last_peered":"2023-10-18T09:03:25.350086+0000","last_clean":"2023-10-18T09:03:25.350086+0000","last_became_active":"2023-10-18T09:03:24.374144+0000","last_became_peered":"2023-10-18T09:03:24.374144+0000","last_unstale":"2023-10-18T09:03:25.350086+0000","last_undegraded":"2023-10-18T09:03:25.350086+0000","last_fullsized":"2023-10-18T09:03:25.350086+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:12:16.648188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"143'1","reported_seq":13,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.391380+0000","last_change":"2023-10-18T09:03:23.324222+0000","last_active":"2023-10-18T09:03:24.391380+0000","last_peered":"2023-10-18T09:03:24.391380+0000","last_clean":"2023-10-18T09:03:24.391380+0000","last_became_active":"2023-10-18T09:03:23.323810+0000","last_became_peered":"2023-10-18T09:03:23.323810+0000","last_unstale":"2023-10-18T09:03:24.391380+0000","last_undegraded":"2023-10-18T09:03:24.391380+0000","last_fullsized":"2023-10-18T09:03:24.391380+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:08:59.283311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.351695+0000","last_change":"2023-10-18T09:03:24.370815+0000","last_active":"2023-10-18T09:03:25.351695+0000","last_peered":"2023-10-18T09:03:25.351695+0000","last_clean":"2023-10-18T09:03:25.351695+0000","last_became_active":"2023-10-18T09:03:24.370553+0000","last_became_peered":"2023-10-18T09:03:24.370553+0000","last_unstale":"2023-10-18T09:03:25.351695+0000","last_undegraded":"2023-10-18T09:03:25.351695+0000","last_fullsized":"2023-10-18T09:03:25.351695+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:28.732066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.365956+0000","last_change":"2023-10-18T09:03:23.332652+0000","last_active":"2023-10-18T09:03:24.365956+0000","last_peered":"2023-10-18T09:03:24.365956+0000","last_clean":"2023-10-18T09:03:24.365956+0000","last_became_active":"2023-10-18T09:03:23.331276+0000","last_became_peered":"2023-10-18T09:03:23.331276+0000","last_unstale":"2023-10-18T09:03:24.365956+0000","last_undegraded":"2023-10-18T09:03:24.365956+0000","last_fullsized":"2023-10-18T09:03:24.365956+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:42.427540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.350705+0000","last_change":"2023-10-18T09:03:24.367856+0000","last_active":"2023-10-18T09:03:25.350705+0000","last_peered":"2023-10-18T09:03:25.350705+0000","last_clean":"2023-10-18T09:03:25.350705+0000","last_became_active":"2023-10-18T09:03:24.367568+0000","last_became_peered":"2023-10-18T09:03:24.367568+0000","last_unstale":"2023-10-18T09:03:25.350705+0000","last_undegraded":"2023-10-18T09:03:25.350705+0000","last_fullsized":"2023-10-18T09:03:25.350705+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:03:43.148081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.397330+0000","last_change":"2023-10-18T09:03:23.322408+0000","last_active":"2023-10-18T09:03:24.397330+0000","last_peered":"2023-10-18T09:03:24.397330+0000","last_clean":"2023-10-18T09:03:24.397330+0000","last_became_active":"2023-10-18T09:03:23.321746+0000","last_became_peered":"2023-10-18T09:03:23.321746+0000","last_unstale":"2023-10-18T09:03:24.397330+0000","last_undegraded":"2023-10-18T09:03:24.397330+0000","last_fullsized":"2023-10-18T09:03:24.397330+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:09:23.860592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.358954+0000","last_change":"2023-10-18T09:03:24.370721+0000","last_active":"2023-10-18T09:03:25.358954+0000","last_peered":"2023-10-18T09:03:25.358954+0000","last_clean":"2023-10-18T09:03:25.358954+0000","last_became_active":"2023-10-18T09:03:24.370447+0000","last_became_peered":"2023-10-18T09:03:24.370447+0000","last_unstale":"2023-10-18T09:03:25.358954+0000","last_undegraded":"2023-10-18T09:03:25.358954+0000","last_fullsized":"2023-10-18T09:03:25.358954+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:19.959226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.356530+0000","last_change":"2023-10-18T09:03:23.324369+0000","last_active":"2023-10-18T09:03:24.356530+0000","last_peered":"2023-10-18T09:03:24.356530+0000","last_clean":"2023-10-18T09:03:24.356530+0000","last_became_active":"2023-10-18T09:03:23.323637+0000","last_became_peered":"2023-10-18T09:03:23.323637+0000","last_unstale":"2023-10-18T09:03:24.356530+0000","last_undegraded":"2023-10-18T09:03:24.356530+0000","last_fullsized":"2023-10-18T09:03:24.356530+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:26:44.259621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.352009+0000","last_change":"2023-10-18T09:03:24.378954+0000","last_active":"2023-10-18T09:03:25.352009+0000","last_peered":"2023-10-18T09:03:25.352009+0000","last_clean":"2023-10-18T09:03:25.352009+0000","last_became_active":"2023-10-18T09:03:24.378534+0000","last_became_peered":"2023-10-18T09:03:24.378534+0000","last_unstale":"2023-10-18T09:03:25.352009+0000","last_undegraded":"2023-10-18T09:03:25.352009+0000","last_fullsized":"2023-10-18T09:03:25.352009+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:42:46.622374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.369038+0000","last_change":"2023-10-18T09:03:23.332462+0000","last_active":"2023-10-18T09:03:24.369038+0000","last_peered":"2023-10-18T09:03:24.369038+0000","last_clean":"2023-10-18T09:03:24.369038+0000","last_became_active":"2023-10-18T09:03:23.332239+0000","last_became_peered":"2023-10-18T09:03:23.332239+0000","last_unstale":"2023-10-18T09:03:24.369038+0000","last_undegraded":"2023-10-18T09:03:24.369038+0000","last_fullsized":"2023-10-18T09:03:24.369038+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:32:34.245946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.347273+0000","last_change":"2023-10-18T09:03:24.361276+0000","last_active":"2023-10-18T09:03:25.347273+0000","last_peered":"2023-10-18T09:03:25.347273+0000","last_clean":"2023-10-18T09:03:25.347273+0000","last_became_active":"2023-10-18T09:03:24.360521+0000","last_became_peered":"2023-10-18T09:03:24.360521+0000","last_unstale":"2023-10-18T09:03:25.347273+0000","last_undegraded":"2023-10-18T09:03:25.347273+0000","last_fullsized":"2023-10-18T09:03:25.347273+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:37:36.479285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.355875+0000","last_change":"2023-10-18T09:03:23.330931+0000","last_active":"2023-10-18T09:03:24.355875+0000","last_peered":"2023-10-18T09:03:24.355875+0000","last_clean":"2023-10-18T09:03:24.355875+0000","last_became_active":"2023-10-18T09:03:23.330686+0000","last_became_peered":"2023-10-18T09:03:23.330686+0000","last_unstale":"2023-10-18T09:03:24.355875+0000","last_undegraded":"2023-10-18T09:03:24.355875+0000","last_fullsized":"2023-10-18T09:03:24.355875+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:27.727259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.357037+0000","last_change":"2023-10-18T09:03:24.371672+0000","last_active":"2023-10-18T09:03:25.357037+0000","last_peered":"2023-10-18T09:03:25.357037+0000","last_clean":"2023-10-18T09:03:25.357037+0000","last_became_active":"2023-10-18T09:03:24.371431+0000","last_became_peered":"2023-10-18T09:03:24.371431+0000","last_unstale":"2023-10-18T09:03:25.357037+0000","last_undegraded":"2023-10-18T09:03:25.357037+0000","last_fullsized":"2023-10-18T09:03:25.357037+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:03:43.148081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"143'2","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.083084+0000","last_change":"2023-10-18T09:03:23.330533+0000","last_active":"2023-10-18T09:03:25.083084+0000","last_peered":"2023-10-18T09:03:25.083084+0000","last_clean":"2023-10-18T09:03:25.083084+0000","last_became_active":"2023-10-18T09:03:23.330093+0000","last_became_peered":"2023-10-18T09:03:23.330093+0000","last_unstale":"2023-10-18T09:03:25.083084+0000","last_undegraded":"2023-10-18T09:03:25.083084+0000","last_fullsized":"2023-10-18T09:03:25.083084+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:42:54.344592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.360614+0000","last_change":"2023-10-18T09:03:24.372679+0000","last_active":"2023-10-18T09:03:25.360614+0000","last_peered":"2023-10-18T09:03:25.360614+0000","last_clean":"2023-10-18T09:03:25.360614+0000","last_became_active":"2023-10-18T09:03:24.371834+0000","last_became_peered":"2023-10-18T09:03:24.371834+0000","last_unstale":"2023-10-18T09:03:25.360614+0000","last_undegraded":"2023-10-18T09:03:25.360614+0000","last_fullsized":"2023-10-18T09:03:25.360614+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:33:33.148900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"144'1","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.357327+0000","last_change":"2023-10-18T09:03:23.329947+0000","last_active":"2023-10-18T09:03:25.357327+0000","last_peered":"2023-10-18T09:03:25.357327+0000","last_clean":"2023-10-18T09:03:25.357327+0000","last_became_active":"2023-10-18T09:03:23.329667+0000","last_became_peered":"2023-10-18T09:03:23.329667+0000","last_unstale":"2023-10-18T09:03:25.357327+0000","last_undegraded":"2023-10-18T09:03:25.357327+0000","last_fullsized":"2023-10-18T09:03:25.357327+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:09.538742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.360038+0000","last_change":"2023-10-18T09:03:24.377192+0000","last_active":"2023-10-18T09:03:25.360038+0000","last_peered":"2023-10-18T09:03:25.360038+0000","last_clean":"2023-10-18T09:03:25.360038+0000","last_became_active":"2023-10-18T09:03:24.375992+0000","last_became_peered":"2023-10-18T09:03:24.375992+0000","last_unstale":"2023-10-18T09:03:25.360038+0000","last_undegraded":"2023-10-18T09:03:25.360038+0000","last_fullsized":"2023-10-18T09:03:25.360038+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:47:56.951578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.362626+0000","last_change":"2023-10-18T09:03:23.332779+0000","last_active":"2023-10-18T09:03:24.362626+0000","last_peered":"2023-10-18T09:03:24.362626+0000","last_clean":"2023-10-18T09:03:24.362626+0000","last_became_active":"2023-10-18T09:03:23.332152+0000","last_became_peered":"2023-10-18T09:03:23.332152+0000","last_unstale":"2023-10-18T09:03:24.362626+0000","last_undegraded":"2023-10-18T09:03:24.362626+0000","last_fullsized":"2023-10-18T09:03:24.362626+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:03:25.612553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.357004+0000","last_change":"2023-10-18T09:03:24.371808+0000","last_active":"2023-10-18T09:03:25.357004+0000","last_peered":"2023-10-18T09:03:25.357004+0000","last_clean":"2023-10-18T09:03:25.357004+0000","last_became_active":"2023-10-18T09:03:24.371571+0000","last_became_peered":"2023-10-18T09:03:24.371571+0000","last_unstale":"2023-10-18T09:03:25.357004+0000","last_undegraded":"2023-10-18T09:03:25.357004+0000","last_fullsized":"2023-10-18T09:03:25.357004+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:32:55.935208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.083210+0000","last_change":"2023-10-18T09:03:23.325136+0000","last_active":"2023-10-18T09:03:25.083210+0000","last_peered":"2023-10-18T09:03:25.083210+0000","last_clean":"2023-10-18T09:03:25.083210+0000","last_became_active":"2023-10-18T09:03:23.323782+0000","last_became_peered":"2023-10-18T09:03:23.323782+0000","last_unstale":"2023-10-18T09:03:25.083210+0000","last_undegraded":"2023-10-18T09:03:25.083210+0000","last_fullsized":"2023-10-18T09:03:25.083210+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:51:42.460476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.356951+0000","last_change":"2023-10-18T09:03:24.369012+0000","last_active":"2023-10-18T09:03:25.356951+0000","last_peered":"2023-10-18T09:03:25.356951+0000","last_clean":"2023-10-18T09:03:25.356951+0000","last_became_active":"2023-10-18T09:03:24.368776+0000","last_became_peered":"2023-10-18T09:03:24.368776+0000","last_unstale":"2023-10-18T09:03:25.356951+0000","last_undegraded":"2023-10-18T09:03:25.356951+0000","last_fullsized":"2023-10-18T09:03:25.356951+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:32:55.935208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.361782+0000","last_change":"2023-10-18T09:03:23.329448+0000","last_active":"2023-10-18T09:03:24.361782+0000","last_peered":"2023-10-18T09:03:24.361782+0000","last_clean":"2023-10-18T09:03:24.361782+0000","last_became_active":"2023-10-18T09:03:23.329094+0000","last_became_peered":"2023-10-18T09:03:23.329094+0000","last_unstale":"2023-10-18T09:03:24.361782+0000","last_undegraded":"2023-10-18T09:03:24.361782+0000","last_fullsized":"2023-10-18T09:03:24.361782+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:32:34.245946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.356686+0000","last_change":"2023-10-18T09:03:24.364068+0000","last_active":"2023-10-18T09:03:25.356686+0000","last_peered":"2023-10-18T09:03:25.356686+0000","last_clean":"2023-10-18T09:03:25.356686+0000","last_became_active":"2023-10-18T09:03:24.363167+0000","last_became_peered":"2023-10-18T09:03:24.363167+0000","last_unstale":"2023-10-18T09:03:25.356686+0000","last_undegraded":"2023-10-18T09:03:25.356686+0000","last_fullsized":"2023-10-18T09:03:25.356686+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:15:22.978564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.349166+0000","last_change":"2023-10-18T09:03:23.323348+0000","last_active":"2023-10-18T09:03:24.349166+0000","last_peered":"2023-10-18T09:03:24.349166+0000","last_clean":"2023-10-18T09:03:24.349166+0000","last_became_active":"2023-10-18T09:03:23.323009+0000","last_became_peered":"2023-10-18T09:03:23.323009+0000","last_unstale":"2023-10-18T09:03:24.349166+0000","last_undegraded":"2023-10-18T09:03:24.349166+0000","last_fullsized":"2023-10-18T09:03:24.349166+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:13.492685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.350323+0000","last_change":"2023-10-18T09:03:24.373169+0000","last_active":"2023-10-18T09:03:25.350323+0000","last_peered":"2023-10-18T09:03:25.350323+0000","last_clean":"2023-10-18T09:03:25.350323+0000","last_became_active":"2023-10-18T09:03:24.370785+0000","last_became_peered":"2023-10-18T09:03:24.370785+0000","last_unstale":"2023-10-18T09:03:25.350323+0000","last_undegraded":"2023-10-18T09:03:25.350323+0000","last_fullsized":"2023-10-18T09:03:25.350323+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:32:55.935208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.356931+0000","last_change":"2023-10-18T09:03:23.329465+0000","last_active":"2023-10-18T09:03:24.356931+0000","last_peered":"2023-10-18T09:03:24.356931+0000","last_clean":"2023-10-18T09:03:24.356931+0000","last_became_active":"2023-10-18T09:03:23.329200+0000","last_became_peered":"2023-10-18T09:03:23.329200+0000","last_unstale":"2023-10-18T09:03:24.356931+0000","last_undegraded":"2023-10-18T09:03:24.356931+0000","last_fullsized":"2023-10-18T09:03:24.356931+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:24:09.742710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.358386+0000","last_change":"2023-10-18T09:03:24.374759+0000","last_active":"2023-10-18T09:03:25.358386+0000","last_peered":"2023-10-18T09:03:25.358386+0000","last_clean":"2023-10-18T09:03:25.358386+0000","last_became_active":"2023-10-18T09:03:24.374073+0000","last_became_peered":"2023-10-18T09:03:24.374073+0000","last_unstale":"2023-10-18T09:03:25.358386+0000","last_undegraded":"2023-10-18T09:03:25.358386+0000","last_fullsized":"2023-10-18T09:03:25.358386+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:07.960850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.362852+0000","last_change":"2023-10-18T09:03:23.329760+0000","last_active":"2023-10-18T09:03:24.362852+0000","last_peered":"2023-10-18T09:03:24.362852+0000","last_clean":"2023-10-18T09:03:24.362852+0000","last_became_active":"2023-10-18T09:03:23.329479+0000","last_became_peered":"2023-10-18T09:03:23.329479+0000","last_unstale":"2023-10-18T09:03:24.362852+0000","last_undegraded":"2023-10-18T09:03:24.362852+0000","last_fullsized":"2023-10-18T09:03:24.362852+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:08.059445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":11,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-18T09:03:25.346799+0000","last_change":"2023-10-18T09:03:24.378443+0000","last_active":"2023-10-18T09:03:25.346799+0000","last_peered":"2023-10-18T09:03:25.346799+0000","last_clean":"2023-10-18T09:03:25.346799+0000","last_became_active":"2023-10-18T09:03:24.378196+0000","last_became_peered":"2023-10-18T09:03:24.378196+0000","last_unstale":"2023-10-18T09:03:25.346799+0000","last_undegraded":"2023-10-18T09:03:25.346799+0000","last_fullsized":"2023-10-18T09:03:25.346799+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:23.295746+0000","last_clean_scrub_stamp":"2023-10-18T09:03:23.295746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:44:01.824311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.365748+0000","last_change":"2023-10-18T09:03:23.332893+0000","last_active":"2023-10-18T09:03:24.365748+0000","last_peered":"2023-10-18T09:03:24.365748+0000","last_clean":"2023-10-18T09:03:24.365748+0000","last_became_active":"2023-10-18T09:03:23.332302+0000","last_became_peered":"2023-10-18T09:03:23.332302+0000","last_unstale":"2023-10-18T09:03:24.365748+0000","last_undegraded":"2023-10-18T09:03:24.365748+0000","last_fullsized":"2023-10-18T09:03:24.365748+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:03:22.290939+0000","last_clean_scrub_stamp":"2023-10-18T09:03:22.290939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:16.894198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":302,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-18T09:03:24.362014+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:03:24.362014+0000","last_peered":"2023-10-18T09:03:24.362014+0000","last_clean":"2023-10-18T09:03:24.362014+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:03:24.362014+0000","last_undegraded":"2023-10-18T09:03:24.362014+0000","last_fullsized":"2023-10-18T09:03:24.362014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1367,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193945,"statfs":{"total":200000000,"available":198599997,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.135,"15min":1.038},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.579,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.288},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.109,"15min":1.046},"min":{"1min":0.64500000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.421,"5min":1.97,"15min":1.97},"last":0.95699999999999996}]},{"osd":1,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1819999999999999,"15min":1.155},"min":{"1min":0.83299999999999996,"5min":0.70099999999999996,"15min":0.61799999999999999},"max":{"1min":1.595,"5min":1.829,"15min":1.829},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.1240000000000001,"15min":1.1259999999999999},"min":{"1min":0.72799999999999998,"5min":0.61099999999999999,"15min":0.58799999999999997},"max":{"1min":1.4239999999999999,"5min":1.657,"15min":1.7989999999999999},"last":1.0149999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3220000000000001,"15min":1.264},"min":{"1min":0.94699999999999995,"5min":0.78600000000000003,"15min":0.65400000000000003},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.9730000000000001},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.27,"15min":1.2190000000000001},"min":{"1min":0.79000000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.601,"5min":1.863,"15min":1.863},"last":1.0669999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.2829999999999999,"15min":1.27},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.5509999999999999,"5min":1.98,"15min":1.98},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3740000000000001,"15min":1.3340000000000001},"min":{"1min":0.97299999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7210000000000001,"5min":1.756,"15min":1.9930000000000001},"last":1.2010000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2629999999999999,"15min":1.1990000000000001},"min":{"1min":0.69899999999999995,"5min":0.66000000000000003,"15min":0.60499999999999998},"max":{"1min":1.802,"5min":1.923,"15min":2.1070000000000002},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.363,"5min":1.341,"15min":1.377},"min":{"1min":0.94699999999999995,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.768,"5min":2.0369999999999999,"15min":2.5169999999999999},"last":1.093}]},{"osd":5,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.349,"15min":1.3520000000000001},"min":{"1min":1.093,"5min":0.622,"15min":0.622},"max":{"1min":1.599,"5min":1.9199999999999999,"15min":2.0720000000000001},"last":1.446},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.444,"15min":1.423},"min":{"1min":1.119,"5min":0.748,"15min":0.67900000000000005},"max":{"1min":1.77,"5min":2.0259999999999998,"15min":2.0270000000000001},"last":1.1559999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.506,"15min":1.4750000000000001},"min":{"1min":1.274,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.8440000000000001,"5min":2.0779999999999998,"15min":2.3959999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"min":{"1min":1.1899999999999999,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.873,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.4910000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":790,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194522,"statfs":{"total":200000000,"available":199190911,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.252,"15min":1.141},"min":{"1min":0.83499999999999996,"5min":0.64700000000000002,"15min":0.60299999999999998},"max":{"1min":1.5940000000000001,"5min":1.627,"15min":1.6850000000000001},"last":1.052},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.357,"15min":1.274},"min":{"1min":0.81799999999999995,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8149999999999999,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.548}]},{"osd":1,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.101,"15min":1.091},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63800000000000001},"max":{"1min":1.5269999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.145,"5min":1.2230000000000001,"15min":1.198},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5940000000000001,"5min":1.8560000000000001,"15min":2.9609999999999999},"last":1.323}]},{"osd":2,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.169,"15min":1.1699999999999999},"min":{"1min":0.84899999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4850000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.8620000000000001},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.236,"15min":1.2090000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.66300000000000003},"max":{"1min":1.627,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.4490000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3009999999999999,"15min":1.2709999999999999},"min":{"1min":0.98299999999999998,"5min":0.79400000000000004,"15min":0.79200000000000004},"max":{"1min":1.627,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.153},{"interface":"front","average":{"1min":1.242,"5min":1.286,"15min":1.2430000000000001},"min":{"1min":0.84999999999999998,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7889999999999999},"last":1.2330000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.254,"15min":1.208},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.706,"5min":2.444,"15min":2.444},"last":1.802},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.282},"min":{"1min":0.91700000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7749999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.756}]},{"osd":5,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.452,"15min":1.3839999999999999},"min":{"1min":1.0309999999999999,"5min":0.625,"15min":0.58899999999999997},"max":{"1min":1.722,"5min":2.032,"15min":2.032},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3220000000000001,"15min":1.256},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.49199999999999999},"max":{"1min":1.679,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":1.5289999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.4099999999999999,"15min":1.4019999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.768,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.548,"5min":1.5429999999999999,"15min":1.5289999999999999},"min":{"1min":1.087,"5min":0.83099999999999996,"15min":0.68500000000000005},"max":{"1min":1.982,"5min":2.3140000000000001,"15min":2.794},"last":1.9199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574952,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":790,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194521,"statfs":{"total":200000000,"available":199190377,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.3400000000000001,"15min":1.278},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.63400000000000001},"max":{"1min":1.8819999999999999,"5min":9.609,"15min":9.609},"last":1.571},{"interface":"front","average":{"1min":1.228,"5min":1.286,"15min":1.3},"min":{"1min":0.754,"5min":0.751,"15min":0.60799999999999998},"max":{"1min":1.7090000000000001,"5min":1.8260000000000001,"15min":5.3029999999999999},"last":1.046}]},{"osd":1,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.3169999999999999,"15min":1.252},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6439999999999999,"5min":11.68,"15min":11.68},"last":1.2470000000000001},{"interface":"front","average":{"1min":0.999,"5min":1.1819999999999999,"15min":1.2210000000000001},"min":{"1min":0.70099999999999996,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.3360000000000001,"5min":2.2410000000000001,"15min":3.7050000000000001},"last":0.94699999999999995}]},{"osd":2,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2589999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.755},"max":{"1min":1.7030000000000001,"5min":2.0350000000000001,"15min":2.2090000000000001},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2450000000000001,"15min":1.276},"min":{"1min":0.94399999999999995,"5min":0.75900000000000001,"15min":0.72399999999999998},"max":{"1min":1.6910000000000001,"5min":2.153,"15min":3.8180000000000001},"last":1.139}]},{"osd":3,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.3200000000000001,"15min":1.272},"min":{"1min":0.72299999999999998,"5min":0.68799999999999994,"15min":0.65400000000000003},"max":{"1min":1.4179999999999999,"5min":11.797000000000001,"15min":11.797000000000001},"last":1.097},{"interface":"front","average":{"1min":1.454,"5min":1.3740000000000001,"15min":1.347},"min":{"1min":1.135,"5min":0.747,"15min":0.747},"max":{"1min":1.798,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.653}]},{"osd":4,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.389,"15min":1.4079999999999999},"min":{"1min":1.157,"5min":0.89900000000000002,"15min":0.75900000000000001},"max":{"1min":1.8100000000000001,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.252,"15min":1.4330000000000001},"min":{"1min":0.88700000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.629,"5min":2.4049999999999998,"15min":5.4580000000000002},"last":1.2230000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2649999999999999,"15min":1.357},"min":{"1min":0.84499999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.5780000000000001,"5min":2.5190000000000001,"15min":5.141},"last":0.998},{"interface":"front","average":{"1min":1.629,"5min":1.498,"15min":1.552},"min":{"1min":1.127,"5min":0.82499999999999996,"15min":0.71099999999999997},"max":{"1min":2.0350000000000001,"5min":2.657,"15min":2.657},"last":1.7330000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:03:17 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.5469999999999999,"15min":1.6240000000000001},"min":{"1min":1.2130000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":1.6970000000000001,"5min":11.829000000000001,"15min":11.829000000000001},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.405,"15min":1.4830000000000001},"min":{"1min":1.0920000000000001,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.837,"5min":2.9420000000000002,"15min":4.8419999999999996},"last":1.198}]}]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":789,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194523,"statfs":{"total":200000000,"available":199192000,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.1479999999999999,"15min":1.1220000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5680000000000001,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.071,"5min":1.1619999999999999,"15min":1.1599999999999999},"min":{"1min":0.71099999999999997,"5min":0.63300000000000001,"15min":0.60099999999999998},"max":{"1min":1.6719999999999999,"5min":3.234,"15min":3.234},"last":0.878}]},{"osd":1,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.1419999999999999,"15min":1.1100000000000001},"min":{"1min":0.64700000000000002,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.484,"5min":3.718,"15min":3.718},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.2,"15min":1.224},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.7529999999999999,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.127}]},{"osd":2,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.218,"15min":1.1930000000000001},"min":{"1min":0.71799999999999997,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4099999999999999,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":1.101},{"interface":"front","average":{"1min":1.24,"5min":1.3380000000000001,"15min":1.3089999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.58899999999999997},"max":{"1min":1.6870000000000001,"5min":3.532,"15min":3.532},"last":1.24}]},{"osd":3,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2709999999999999,"15min":1.2889999999999999},"min":{"1min":0.78200000000000003,"5min":0.71899999999999997,"15min":0.67000000000000004},"max":{"1min":1.5700000000000001,"5min":3.363,"15min":3.363},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.399,"15min":1.409},"min":{"1min":0.84999999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.603,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":1.393}]},{"osd":5,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.274,"15min":1.2969999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.498},"max":{"1min":1.7909999999999999,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":1.343},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.296,"15min":1.341},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.38400000000000001},"max":{"1min":1.571,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":1.2929999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.3720000000000001,"15min":1.3939999999999999},"min":{"1min":0.78300000000000003,"5min":0.64300000000000002,"15min":0.51600000000000001},"max":{"1min":1.478,"5min":4.4429999999999996,"15min":5.0049999999999999},"last":1.873},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.4510000000000001,"15min":1.532},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.45400000000000001},"max":{"1min":1.706,"5min":4.1239999999999997,"15min":4.407},"last":1.4390000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:03:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.4330000000000001,"15min":1.484},"min":{"1min":0.85599999999999998,"5min":0.82499999999999996,"15min":0.438},"max":{"1min":1.7410000000000001,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.536,"15min":1.546},"min":{"1min":0.92000000000000004,"5min":0.78700000000000003,"15min":0.747},"max":{"1min":1.7410000000000001,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":1.806}]}]},{"osd":3,"up_from":13,"seq":55834574953,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":791,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194520,"statfs":{"total":200000000,"available":199189299,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":0.94999999999999996,"15min":0.84599999999999997},"min":{"1min":0.63800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":4.774,"5min":4.774,"15min":5.5730000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.0069999999999999,"5min":0.96299999999999997,"15min":0.91600000000000004},"min":{"1min":0.53600000000000003,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.2370000000000001,"5min":3.2370000000000001,"15min":6.5599999999999996},"last":0.66100000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.0049999999999999,"15min":0.96899999999999997},"min":{"1min":0.55200000000000005,"5min":0.39100000000000001,"15min":0.36799999999999999},"max":{"1min":1.4790000000000001,"5min":1.839,"15min":6.8099999999999996},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2629999999999999,"15min":1.1679999999999999},"min":{"1min":0.85799999999999998,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6659999999999999,"5min":2.0720000000000001,"15min":3.1920000000000002},"last":1.0800000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.1060000000000001,"15min":1.052},"min":{"1min":0.62,"5min":0.46800000000000003,"15min":0.45300000000000001},"max":{"1min":2.6269999999999998,"5min":2.6269999999999998,"15min":7.4580000000000002},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.456,"5min":1.2150000000000001,"15min":1.117},"min":{"1min":0.78700000000000003,"5min":0.53200000000000003,"15min":0.51400000000000001},"max":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":0.77800000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.327,"15min":1.339},"min":{"1min":0.82099999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":3.1139999999999999,"5min":3.1139999999999999,"15min":18.664999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3009999999999999,"15min":1.2110000000000001},"min":{"1min":0.92700000000000005,"5min":0.877,"15min":0.64000000000000001},"max":{"1min":2.0230000000000001,"5min":2.093,"15min":7.0700000000000003},"last":0.96799999999999997}]},{"osd":5,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.333,"15min":1.266},"min":{"1min":0.90500000000000003,"5min":0.77600000000000002,"15min":0.67700000000000005},"max":{"1min":1.9099999999999999,"5min":2.1680000000000001,"15min":6.9139999999999997},"last":1.252},{"interface":"front","average":{"1min":1.6930000000000001,"5min":1.4650000000000001,"15min":1.409},"min":{"1min":1.1140000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":4.5540000000000003,"5min":4.5540000000000003,"15min":16.390999999999998},"last":1.1870000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.4850000000000001,"15min":1.4510000000000001},"min":{"1min":0.92200000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.375,"5min":3.375,"15min":7.7270000000000003},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.494,"15min":1.3959999999999999},"min":{"1min":1.0189999999999999,"5min":0.872,"15min":0.60199999999999998},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":7.242},"last":1.3799999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:03:16 2023","interfaces":[{"interface":"back","average":{"1min":1.821,"5min":1.5660000000000001,"15min":1.4219999999999999},"min":{"1min":1.226,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":4.8630000000000004,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.8089999999999999,"5min":1.5589999999999999,"15min":1.502},"min":{"1min":0.98399999999999999,"5min":0.622,"15min":0.622},"max":{"1min":5.4900000000000002,"5min":5.4900000000000002,"15min":12.946},"last":1.286}]}]},{"osd":2,"up_from":13,"seq":55834574952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":790,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194522,"statfs":{"total":200000000,"available":199190981,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.88200000000000001,"15min":0.91300000000000003},"min":{"1min":0.56000000000000005,"5min":0.48599999999999999,"15min":0.36899999999999999},"max":{"1min":1.3300000000000001,"5min":1.706,"15min":4.2210000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.012,"5min":1.042,"15min":1.04},"min":{"1min":0.59299999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.617,"5min":4.3959999999999999,"15min":4.3959999999999999},"last":0.60599999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.1359999999999999,"15min":1.077},"min":{"1min":0.65300000000000002,"5min":0.55900000000000005,"15min":0.49099999999999999},"max":{"1min":1.522,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.181,"15min":1.121},"min":{"1min":0.77000000000000002,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":1.552,"5min":2.448,"15min":2.5350000000000001},"last":0.70399999999999996}]},{"osd":3,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.194,"15min":1.141},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.48199999999999998},"max":{"1min":1.7569999999999999,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.365,"15min":1.427},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.54100000000000004},"max":{"1min":1.728,"5min":2.2829999999999999,"15min":7.5570000000000004},"last":0.748}]},{"osd":4,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.4530000000000001,"15min":1.4730000000000001},"min":{"1min":1.0289999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.879,"5min":2.4119999999999999,"15min":7.6189999999999998},"last":1.165},{"interface":"front","average":{"1min":1.371,"5min":1.4910000000000001,"15min":1.458},"min":{"1min":0.98599999999999999,"5min":0.98099999999999998,"15min":0.86599999999999999},"max":{"1min":1.7,"5min":3.423,"15min":3.423},"last":1.002}]},{"osd":5,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.407,"15min":1.3089999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.61599999999999999},"max":{"1min":1.6000000000000001,"5min":10.507,"15min":10.507},"last":1.04},{"interface":"front","average":{"1min":1.448,"5min":1.4910000000000001,"15min":1.5349999999999999},"min":{"1min":1.1020000000000001,"5min":0.88100000000000001,"15min":0.81399999999999995},"max":{"1min":1.9550000000000001,"5min":2.536,"15min":7.657},"last":1.1080000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.52,"15min":1.518},"min":{"1min":1.2210000000000001,"5min":1.0289999999999999,"15min":0.74299999999999999},"max":{"1min":1.9099999999999999,"5min":2.8780000000000001,"15min":3.5299999999999998},"last":1.077},{"interface":"front","average":{"1min":1.399,"5min":1.5640000000000001,"15min":1.611},"min":{"1min":0.96499999999999997,"5min":0.85899999999999999,"15min":0.77600000000000002},"max":{"1min":2.0190000000000001,"5min":10.561,"15min":37.771999999999998},"last":1.3580000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:03:18 2023","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5740000000000001},"min":{"1min":1.26,"5min":0.97599999999999998,"15min":0.83199999999999996},"max":{"1min":2.2519999999999998,"5min":2.7890000000000001,"15min":3.3620000000000001},"last":1.123},{"interface":"front","average":{"1min":1.474,"5min":1.6339999999999999,"15min":1.5209999999999999},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.77600000000000002},"max":{"1min":2.1059999999999999,"5min":10.956,"15min":10.956},"last":1.2669999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":789,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194523,"statfs":{"total":200000000,"available":199191876,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.93899999999999995,"15min":0.84399999999999997},"min":{"1min":0.42799999999999999,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":1.079},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.091,"15min":0.95799999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.645,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.1639999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.0489999999999999,"15min":0.96299999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.589,"5min":1.6899999999999999,"15min":1.859},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.99299999999999999,"15min":0.88700000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.36699999999999999},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.66100000000000003}]},{"osd":3,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.139,"15min":1.0469999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.436},"max":{"1min":1.915,"5min":1.915,"15min":2.1059999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0960000000000001,"15min":1.0589999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.7110000000000001,"5min":1.9370000000000001,"15min":3.7010000000000001},"last":1.3779999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.3460000000000001,"15min":1.27},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7430000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.3799999999999999,"15min":1.256},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.829,"5min":1.903,"15min":2.0470000000000002},"last":1.3400000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4199999999999999,"15min":1.325},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.984,"5min":2.161,"15min":3.5750000000000002},"last":1.77},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.4890000000000001,"15min":1.3919999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8320000000000001,"5min":1.964,"15min":2.3769999999999998},"last":1.597}]},{"osd":6,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.454,"15min":1.3260000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.905,"5min":2.0419999999999998,"15min":2.2050000000000001},"last":1.544},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.452,"15min":1.3149999999999999},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.63100000000000001},"max":{"1min":2.1040000000000001,"5min":2.121,"15min":2.121},"last":1.4890000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:03:22 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.524,"15min":1.399},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.2000000000000002,"5min":2.2229999999999999,"15min":2.2229999999999999},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5329999999999999,"15min":1.4039999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.2469999999999999,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.4359999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":36,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1367,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193945,"statfs":{"total":200000000,"available":198599908,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.81899999999999995,"15min":0.89000000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":1.3320000000000001,"15min":3.4420000000000002},"last":1.014},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.083,"15min":1.0780000000000001},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.46500000000000002},"max":{"1min":1.3200000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.6559999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.85699999999999998,"15min":0.84199999999999997},"min":{"1min":0.44400000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.244,"5min":1.679,"15min":1.679},"last":1.482},{"interface":"front","average":{"1min":1.071,"5min":1.1359999999999999,"15min":1.1279999999999999},"min":{"1min":0.83999999999999997,"5min":0.503,"15min":0.437},"max":{"1min":1.446,"5min":1.929,"15min":1.929},"last":1.2929999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.135,"15min":1.234},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4419999999999999,"5min":1.98,"15min":2.7639999999999998},"last":1.857},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3799999999999999,"15min":1.4710000000000001},"min":{"1min":0.71999999999999997,"5min":0.61299999999999999,"15min":0.58399999999999996},"max":{"1min":1.677,"5min":2.3639999999999999,"15min":3.5720000000000001},"last":1.3600000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.228,"15min":1.278},"min":{"1min":0.85999999999999999,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.7410000000000001,"5min":1.8819999999999999,"15min":4.492},"last":1.7150000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.282,"15min":1.2589999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.518,"5min":1.927,"15min":1.927},"last":1.694}]},{"osd":5,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.256,"15min":1.373},"min":{"1min":0.83399999999999996,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.839,"5min":2.4510000000000001,"15min":4.2779999999999996},"last":1.931},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.3049999999999999,"15min":1.3540000000000001},"min":{"1min":0.751,"5min":0.74399999999999999,"15min":0.73099999999999998},"max":{"1min":1.573,"5min":1.847,"15min":1.8740000000000001},"last":1.6220000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.4179999999999999,"15min":1.538},"min":{"1min":0.79900000000000004,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.613,"5min":2.0419999999999998,"15min":3.5299999999999998},"last":1.8100000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.363,"15min":1.458},"min":{"1min":0.78000000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.665,"5min":3.1440000000000001,"15min":4.4349999999999996},"last":1.554}]},{"osd":7,"last update":"Wed Oct 18 09:03:23 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.524,"15min":1.6080000000000001},"min":{"1min":0.88,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.8120000000000001,"5min":2.2000000000000002,"15min":3.492},"last":1.8919999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.446,"15min":1.6080000000000001},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.1869999999999998,"5min":3.2570000000000001,"15min":4.5419999999999998},"last":2.0499999999999998}]}]}],"pool_statfs":[]}} 2023-10-18T09:03:34.569 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T09:03:34.569 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T09:03:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.842+0000 7f1fdd467640 1 -- 172.21.15.40:0/1539006463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fc40047c0 msgr2=0x7f1fc4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:34.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.842+0000 7f1fdd467640 1 --2- 172.21.15.40:0/1539006463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fc40047c0 0x7f1fc4004c40 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0001eb0 tx=0x7f1fc40057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:34.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.842+0000 7f1fdcc66640 1 -- 172.21.15.40:0/1539006463 reap_dead start 2023-10-18T09:03:34.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.842+0000 7f1fdd467640 1 -- 172.21.15.40:0/1539006463 shutdown_connections 2023-10-18T09:03:34.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.842+0000 7f1fdd467640 1 -- 172.21.15.40:0/1539006463 >> 172.21.15.40:0/1539006463 conn(0x7f1fd806ca10 msgr2=0x7f1fd806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.842+0000 7f1fdd467640 1 -- 172.21.15.40:0/1539006463 shutdown_connections 2023-10-18T09:03:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.842+0000 7f1fdd467640 1 -- 172.21.15.40:0/1539006463 wait complete. 2023-10-18T09:03:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.844+0000 7f1fdd467640 1 Processor -- start 2023-10-18T09:03:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.844+0000 7f1fdd467640 1 -- start start 2023-10-18T09:03:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.845+0000 7f1fdd467640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc40047c0 0x7f1fd81f1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.845+0000 7f1fdd467640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fd81e2890 0x7f1fd8156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.845+0000 7f1fdd467640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fd81e2d10 0x7f1fd8156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:34.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.845+0000 7f1fdd467640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd81e3550 con 0x7f1fd81e2890 2023-10-18T09:03:34.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.845+0000 7f1fdd467640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd81f1c70 con 0x7f1fc40047c0 2023-10-18T09:03:34.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.845+0000 7f1fdd467640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd814a4a0 con 0x7f1fd81e2d10 2023-10-18T09:03:34.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.845+0000 7f1fcffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fd81e2890 0x7f1fd8156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.845+0000 7f1fcffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fd81e2890 0x7f1fd8156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49774/0 (socket says 172.21.15.40:49774) 2023-10-18T09:03:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.845+0000 7f1fcffff640 1 -- 172.21.15.40:0/1769833280 learned_addr learned my addr 172.21.15.40:0/1769833280 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.846+0000 7f1fcffff640 1 -- 172.21.15.40:0/1769833280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fd81e2d10 msgr2=0x7f1fd8156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.846+0000 7f1fd7fff640 1 --2- 172.21.15.40:0/1769833280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc40047c0 0x7f1fd81f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.846+0000 7f1fdcc66640 1 --2- 172.21.15.40:0/1769833280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fd81e2d10 0x7f1fd8156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.846+0000 7f1fcffff640 1 --2- 172.21.15.40:0/1769833280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fd81e2d10 0x7f1fd8156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.846+0000 7f1fcffff640 1 -- 172.21.15.40:0/1769833280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc40047c0 msgr2=0x7f1fd81f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.846+0000 7f1fcffff640 1 --2- 172.21.15.40:0/1769833280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc40047c0 0x7f1fd81f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.846+0000 7f1fcffff640 1 -- 172.21.15.40:0/1769833280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd8158250 con 0x7f1fd81e2890 2023-10-18T09:03:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.846+0000 7f1fcffff640 1 --2- 172.21.15.40:0/1769833280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fd81e2890 0x7f1fd8156560 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000c260 tx=0x7f1fc00018a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:34.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.847+0000 7f1fd57fa640 1 -- 172.21.15.40:0/1769833280 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00035f0 con 0x7f1fd81e2890 2023-10-18T09:03:34.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.847+0000 7f1fd57fa640 1 -- 172.21.15.40:0/1769833280 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1fc0003840 con 0x7f1fd81e2890 2023-10-18T09:03:34.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.847+0000 7f1fd57fa640 1 -- 172.21.15.40:0/1769833280 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0003c30 con 0x7f1fd81e2890 2023-10-18T09:03:34.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.847+0000 7f1fdd467640 1 -- 172.21.15.40:0/1769833280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd81e3270 con 0x7f1fd81e2890 2023-10-18T09:03:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.847+0000 7f1fdd467640 1 -- 172.21.15.40:0/1769833280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd8157ee0 con 0x7f1fd81e2890 2023-10-18T09:03:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.848+0000 7f1fdd467640 1 -- 172.21.15.40:0/1769833280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd8157a60 con 0x7f1fd81e2890 2023-10-18T09:03:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.849+0000 7f1fd57fa640 1 -- 172.21.15.40:0/1769833280 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1fc0032070 con 0x7f1fd81e2890 2023-10-18T09:03:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.850+0000 7f1fd57fa640 1 --2- 172.21.15.40:0/1769833280 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1fb806ae70 0x7f1fb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.850+0000 7f1fd57fa640 1 -- 172.21.15.40:0/1769833280 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fc00aa910 con 0x7f1fd81e2890 2023-10-18T09:03:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.850+0000 7f1fd7fff640 1 --2- 172.21.15.40:0/1769833280 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1fb806ae70 0x7f1fb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:34.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.852+0000 7f1fd7fff640 1 --2- 172.21.15.40:0/1769833280 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1fb806ae70 0x7f1fb806d330 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800daa0 tx=0x7f1fc8005ed0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:34.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:34.854+0000 7f1fd57fa640 1 -- 172.21.15.40:0/1769833280 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1fd8157a60 con 0x7f1fd81e2890 2023-10-18T09:03:35.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.122+0000 7f1fdd467640 1 -- 172.21.15.40:0/1769833280 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1fd80c9b60 con 0x7f1fd81e2890 2023-10-18T09:03:35.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.127+0000 7f1fd57fa640 1 -- 172.21.15.40:0/1769833280 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1fd80c9b60 con 0x7f1fd81e2890 2023-10-18T09:03:35.131 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2023-10-18T09:03:35.132 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w== 2023-10-18T09:03:35.132 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2023-10-18T09:03:35.132 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2023-10-18T09:03:35.132 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T09:03:35.132 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2023-10-18T09:03:35.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.130+0000 7f1fdd467640 1 -- 172.21.15.40:0/1769833280 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1fb806ae70 msgr2=0x7f1fb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:35.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.130+0000 7f1fdd467640 1 --2- 172.21.15.40:0/1769833280 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1fb806ae70 0x7f1fb806d330 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800daa0 tx=0x7f1fc8005ed0 comp rx=0 tx=0).stop 2023-10-18T09:03:35.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.130+0000 7f1fdd467640 1 -- 172.21.15.40:0/1769833280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fd81e2890 msgr2=0x7f1fd8156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:35.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.130+0000 7f1fdd467640 1 --2- 172.21.15.40:0/1769833280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fd81e2890 0x7f1fd8156560 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000c260 tx=0x7f1fc00018a0 comp rx=0 tx=0).stop 2023-10-18T09:03:35.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.131+0000 7f1fdd467640 1 -- 172.21.15.40:0/1769833280 shutdown_connections 2023-10-18T09:03:35.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.131+0000 7f1fdd467640 1 --2- 172.21.15.40:0/1769833280 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1fb806ae70 0x7f1fb806d330 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:35.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.131+0000 7f1fdd467640 1 --2- 172.21.15.40:0/1769833280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1fd81e2d10 0x7f1fd8156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:35.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.131+0000 7f1fdd467640 1 --2- 172.21.15.40:0/1769833280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fd81e2890 0x7f1fd8156560 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:35.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.131+0000 7f1fdd467640 1 --2- 172.21.15.40:0/1769833280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fc40047c0 0x7f1fd81f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:35.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.131+0000 7f1fdd467640 1 -- 172.21.15.40:0/1769833280 >> 172.21.15.40:0/1769833280 conn(0x7f1fd806ca10 msgr2=0x7f1fd80b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:35.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.131+0000 7f1fdd467640 1 -- 172.21.15.40:0/1769833280 shutdown_connections 2023-10-18T09:03:35.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.131+0000 7f1fdd467640 1 -- 172.21.15.40:0/1769833280 wait complete. 2023-10-18T09:03:35.157 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:03:35.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.387+0000 7f1b52728640 1 -- 172.21.15.40:0/1802460953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b380039a0 msgr2=0x7f1b38003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:35.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.387+0000 7f1b52728640 1 --2- 172.21.15.40:0/1802460953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b380039a0 0x7f1b38003dc0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009b20 tx=0x7f1b380057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:35.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.387+0000 7f1b51f27640 1 -- 172.21.15.40:0/1802460953 reap_dead start 2023-10-18T09:03:35.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.387+0000 7f1b52728640 1 -- 172.21.15.40:0/1802460953 shutdown_connections 2023-10-18T09:03:35.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.387+0000 7f1b52728640 1 -- 172.21.15.40:0/1802460953 >> 172.21.15.40:0/1802460953 conn(0x7f1b4c069720 msgr2=0x7f1b4c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:35.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.387+0000 7f1b52728640 1 -- 172.21.15.40:0/1802460953 shutdown_connections 2023-10-18T09:03:35.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.387+0000 7f1b52728640 1 -- 172.21.15.40:0/1802460953 wait complete. 2023-10-18T09:03:35.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.389+0000 7f1b52728640 1 Processor -- start 2023-10-18T09:03:35.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.389+0000 7f1b52728640 1 -- start start 2023-10-18T09:03:35.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.390+0000 7f1b52728640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b380039a0 0x7f1b4c1f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:35.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.390+0000 7f1b52728640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b4c0c9260 0x7f1b4c159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:35.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.390+0000 7f1b52728640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4c13a9a0 0x7f1b4c1f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:35.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.390+0000 7f1b52728640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c1f5f40 con 0x7f1b4c0c9260 2023-10-18T09:03:35.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.390+0000 7f1b52728640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c1f1ca0 con 0x7f1b4c13a9a0 2023-10-18T09:03:35.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.390+0000 7f1b52728640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c15cce0 con 0x7f1b380039a0 2023-10-18T09:03:35.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.390+0000 7f1b50f25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b4c0c9260 0x7f1b4c159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:35.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.391+0000 7f1b50f25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b4c0c9260 0x7f1b4c159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59780/0 (socket says 172.21.15.40:59780) 2023-10-18T09:03:35.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.391+0000 7f1b50f25640 1 -- 172.21.15.40:0/1292569793 learned_addr learned my addr 172.21.15.40:0/1292569793 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.391+0000 7f1b50f25640 1 -- 172.21.15.40:0/1292569793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b380039a0 msgr2=0x7f1b4c1f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.391+0000 7f1b50f25640 1 --2- 172.21.15.40:0/1292569793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b380039a0 0x7f1b4c1f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.391+0000 7f1b50f25640 1 -- 172.21.15.40:0/1292569793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4c13a9a0 msgr2=0x7f1b4c1f20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.391+0000 7f1b51f27640 1 --2- 172.21.15.40:0/1292569793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4c13a9a0 0x7f1b4c1f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:35.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.391+0000 7f1b50f25640 1 --2- 172.21.15.40:0/1292569793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4c13a9a0 0x7f1b4c1f20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:35.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.391+0000 7f1b50f25640 1 -- 172.21.15.40:0/1292569793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c1f19b0 con 0x7f1b4c0c9260 2023-10-18T09:03:35.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.391+0000 7f1b50f25640 1 --2- 172.21.15.40:0/1292569793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b4c0c9260 0x7f1b4c159950 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c006b00 tx=0x7f1b3c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:35.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.392+0000 7f1b41ffb640 1 -- 172.21.15.40:0/1292569793 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c001d10 con 0x7f1b4c0c9260 2023-10-18T09:03:35.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.392+0000 7f1b41ffb640 1 -- 172.21.15.40:0/1292569793 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1b3c011640 con 0x7f1b4c0c9260 2023-10-18T09:03:35.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.392+0000 7f1b41ffb640 1 -- 172.21.15.40:0/1292569793 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c011a50 con 0x7f1b4c0c9260 2023-10-18T09:03:35.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.392+0000 7f1b52728640 1 -- 172.21.15.40:0/1292569793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4c0c9740 con 0x7f1b4c0c9260 2023-10-18T09:03:35.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.392+0000 7f1b52728640 1 -- 172.21.15.40:0/1292569793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4c15ca00 con 0x7f1b4c0c9260 2023-10-18T09:03:35.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.393+0000 7f1b52728640 1 -- 172.21.15.40:0/1292569793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4c066e00 con 0x7f1b4c0c9260 2023-10-18T09:03:35.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.394+0000 7f1b41ffb640 1 -- 172.21.15.40:0/1292569793 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1b3c011bb0 con 0x7f1b4c0c9260 2023-10-18T09:03:35.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.395+0000 7f1b41ffb640 1 --2- 172.21.15.40:0/1292569793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b3006ae70 0x7f1b3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:35.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.395+0000 7f1b41ffb640 1 -- 172.21.15.40:0/1292569793 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b3c08a3d0 con 0x7f1b4c0c9260 2023-10-18T09:03:35.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.396+0000 7f1b51726640 1 --2- 172.21.15.40:0/1292569793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b3006ae70 0x7f1b3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:35.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.397+0000 7f1b51726640 1 --2- 172.21.15.40:0/1292569793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b3006ae70 0x7f1b3006d330 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1b48036000 tx=0x7f1b48035040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:35.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.399+0000 7f1b41ffb640 1 -- 172.21.15.40:0/1292569793 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b3c0553e0 con 0x7f1b4c0c9260 2023-10-18T09:03:35.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.631+0000 7f1b52728640 1 -- 172.21.15.40:0/1292569793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b4c1f1e10 con 0x7f1b4c0c9260 2023-10-18T09:03:35.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.632+0000 7f1b41ffb640 1 -- 172.21.15.40:0/1292569793 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f1b4c1f1e10 con 0x7f1b4c0c9260 2023-10-18T09:03:35.635 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:35.636 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6382,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/4165164542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":4165164542},{"type":"v1","addr":"172.21.15.144:6836","nonce":4165164542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3045960359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3045960359},{"type":"v1","addr":"172.21.15.144:6837","nonce":3045960359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2869040025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2869040025},{"type":"v1","addr":"172.21.15.40:6833","nonce":2869040025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:03:23.782442+0000","modified":"2023-10-18T09:03:26.897709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7893},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7893":{"gid":7893,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2600609919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2600609919},{"type":"v1","addr":"172.21.15.40:6835","nonce":2600609919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-18T09:03:35.636 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 66 2023-10-18T09:03:35.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.635+0000 7f1b52728640 1 -- 172.21.15.40:0/1292569793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b3006ae70 msgr2=0x7f1b3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:35.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.635+0000 7f1b52728640 1 --2- 172.21.15.40:0/1292569793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b3006ae70 0x7f1b3006d330 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1b48036000 tx=0x7f1b48035040 comp rx=0 tx=0).stop 2023-10-18T09:03:35.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.635+0000 7f1b52728640 1 -- 172.21.15.40:0/1292569793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b4c0c9260 msgr2=0x7f1b4c159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:35.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.635+0000 7f1b52728640 1 --2- 172.21.15.40:0/1292569793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b4c0c9260 0x7f1b4c159950 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c006b00 tx=0x7f1b3c001630 comp rx=0 tx=0).stop 2023-10-18T09:03:35.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.635+0000 7f1b52728640 1 -- 172.21.15.40:0/1292569793 shutdown_connections 2023-10-18T09:03:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.635+0000 7f1b52728640 1 --2- 172.21.15.40:0/1292569793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1b3006ae70 0x7f1b3006d330 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.635+0000 7f1b52728640 1 --2- 172.21.15.40:0/1292569793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b4c13a9a0 0x7f1b4c1f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.635+0000 7f1b52728640 1 --2- 172.21.15.40:0/1292569793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b4c0c9260 0x7f1b4c159950 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.635+0000 7f1b52728640 1 --2- 172.21.15.40:0/1292569793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1b380039a0 0x7f1b4c1f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.635+0000 7f1b52728640 1 -- 172.21.15.40:0/1292569793 >> 172.21.15.40:0/1292569793 conn(0x7f1b4c069720 msgr2=0x7f1b4c13c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:35.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.635+0000 7f1b52728640 1 -- 172.21.15.40:0/1292569793 shutdown_connections 2023-10-18T09:03:35.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:35.635+0000 7f1b52728640 1 -- 172.21.15.40:0/1292569793 wait complete. 2023-10-18T09:03:36.061 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T09:03:23.782442+0000', 'modified': '2023-10-18T09:03:26.897709+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7893}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7893': {'gid': 7893, 'name': 'c', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/2600609919', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 2600609919}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 2600609919}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T09:03:36.061 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T09:03:36.061 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-18T09:03:36.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.294+0000 7fda611c5640 1 -- 172.21.15.40:0/2354081243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5c06ffe0 msgr2=0x7fda5c06b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:36.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.294+0000 7fda611c5640 1 --2- 172.21.15.40:0/2354081243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5c06ffe0 0x7fda5c06b710 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0ce190 tx=0x7fda5402bd80 comp rx=0 tx=0).stop 2023-10-18T09:03:36.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.295+0000 7fda611c5640 1 -- 172.21.15.40:0/2354081243 shutdown_connections 2023-10-18T09:03:36.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.295+0000 7fda611c5640 1 --2- 172.21.15.40:0/2354081243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5c06ffe0 0x7fda5c06b710 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.295+0000 7fda611c5640 1 --2- 172.21.15.40:0/2354081243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda5c0b1b90 0x7fda5c0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.295+0000 7fda611c5640 1 --2- 172.21.15.40:0/2354081243 >> v2:172.21.15.144:3300/0 conn(0x7fda5c15eaa0 0x7fda5c0ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.295+0000 7fda611c5640 1 -- 172.21.15.40:0/2354081243 >> 172.21.15.40:0/2354081243 conn(0x7fda5c06cee0 msgr2=0x7fda5c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:36.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.295+0000 7fda611c5640 1 -- 172.21.15.40:0/2354081243 shutdown_connections 2023-10-18T09:03:36.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.295+0000 7fda611c5640 1 -- 172.21.15.40:0/2354081243 wait complete. 2023-10-18T09:03:36.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.296+0000 7fda611c5640 1 Processor -- start 2023-10-18T09:03:36.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.297+0000 7fda611c5640 1 -- start start 2023-10-18T09:03:36.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.297+0000 7fda611c5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda5c06ffe0 0x7fda5c15b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.298+0000 7fda611c5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda5c0b1b90 0x7fda5c0cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.298+0000 7fda611c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5c15eaa0 0x7fda5c149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.298+0000 7fda611c5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0cc6b0 con 0x7fda5c15eaa0 2023-10-18T09:03:36.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.298+0000 7fda611c5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0cc990 con 0x7fda5c06ffe0 2023-10-18T09:03:36.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.298+0000 7fda611c5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c155630 con 0x7fda5c0b1b90 2023-10-18T09:03:36.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.298+0000 7fda5b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5c15eaa0 0x7fda5c149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.298+0000 7fda5b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5c15eaa0 0x7fda5c149690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59786/0 (socket says 172.21.15.40:59786) 2023-10-18T09:03:36.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.298+0000 7fda5b59e640 1 -- 172.21.15.40:0/3915900672 learned_addr learned my addr 172.21.15.40:0/3915900672 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:36.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.298+0000 7fda5ad9d640 1 --2- 172.21.15.40:0/3915900672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda5c06ffe0 0x7fda5c15b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.298+0000 7fda5b59e640 1 -- 172.21.15.40:0/3915900672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda5c0b1b90 msgr2=0x7fda5c0cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:36.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.298+0000 7fda5b59e640 1 --2- 172.21.15.40:0/3915900672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda5c0b1b90 0x7fda5c0cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.298+0000 7fda5b59e640 1 -- 172.21.15.40:0/3915900672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda5c06ffe0 msgr2=0x7fda5c15b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:36.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.298+0000 7fda5b59e640 1 --2- 172.21.15.40:0/3915900672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda5c06ffe0 0x7fda5c15b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.298+0000 7fda5b59e640 1 -- 172.21.15.40:0/3915900672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda54009560 con 0x7fda5c15eaa0 2023-10-18T09:03:36.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.299+0000 7fda5b59e640 1 --2- 172.21.15.40:0/3915900672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5c15eaa0 0x7fda5c149690 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fda54006010 tx=0x7fda5402db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:36.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.299+0000 7fda437fe640 1 -- 172.21.15.40:0/3915900672 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda540035f0 con 0x7fda5c15eaa0 2023-10-18T09:03:36.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.299+0000 7fda437fe640 1 -- 172.21.15.40:0/3915900672 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fda54003860 con 0x7fda5c15eaa0 2023-10-18T09:03:36.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.299+0000 7fda437fe640 1 -- 172.21.15.40:0/3915900672 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54003c70 con 0x7fda5c15eaa0 2023-10-18T09:03:36.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.299+0000 7fda611c5640 1 -- 172.21.15.40:0/3915900672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c155910 con 0x7fda5c15eaa0 2023-10-18T09:03:36.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.299+0000 7fda611c5640 1 -- 172.21.15.40:0/3915900672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c15b710 con 0x7fda5c15eaa0 2023-10-18T09:03:36.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.301+0000 7fda437fe640 1 -- 172.21.15.40:0/3915900672 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fda5403e450 con 0x7fda5c15eaa0 2023-10-18T09:03:36.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.302+0000 7fda437fe640 1 --2- 172.21.15.40:0/3915900672 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fda4806ae40 0x7fda4806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.302+0000 7fda437fe640 1 -- 172.21.15.40:0/3915900672 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda540ac290 con 0x7fda5c15eaa0 2023-10-18T09:03:36.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.303+0000 7fda5ad9d640 1 --2- 172.21.15.40:0/3915900672 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fda4806ae40 0x7fda4806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.303+0000 7fda63e31740 1 Processor -- start 2023-10-18T09:03:36.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.303+0000 7fda63e31740 1 -- start start 2023-10-18T09:03:36.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.304+0000 7fda63e31740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc6f7d0 0x55995dc42a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.304+0000 7fda63e31740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc6fbb0 0x55995dc72560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.304+0000 7fda5ad9d640 1 --2- 172.21.15.40:0/3915900672 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fda4806ae40 0x7fda4806d300 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0abc90 tx=0x7fda4c007310 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:36.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.304+0000 7fda63e31740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc42f40 0x55995dc72aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.304+0000 7fda63e31740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995dbb5740 con 0x55995dc42f40 2023-10-18T09:03:36.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.304+0000 7fda63e31740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995dbb2e30 con 0x55995dc6f7d0 2023-10-18T09:03:36.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.304+0000 7fda63e31740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55995dbae720 con 0x55995dc6fbb0 2023-10-18T09:03:36.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.305+0000 7fda5b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc42f40 0x55995dc72aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.305+0000 7fda5b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc42f40 0x55995dc72aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59798/0 (socket says 172.21.15.40:59798) 2023-10-18T09:03:36.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.305+0000 7fda5b59e640 1 -- 172.21.15.40:0/2468756981 learned_addr learned my addr 172.21.15.40:0/2468756981 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:36.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.305+0000 7fda5ad9d640 1 --2- 172.21.15.40:0/2468756981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc6f7d0 0x55995dc42a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.305+0000 7fda5a59c640 1 --2- 172.21.15.40:0/2468756981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc6fbb0 0x55995dc72560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.305+0000 7fda5b59e640 1 -- 172.21.15.40:0/2468756981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc6fbb0 msgr2=0x55995dc72560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:03:36.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.305+0000 7fda5b59e640 1 --2- 172.21.15.40:0/2468756981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc6fbb0 0x55995dc72560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.305+0000 7fda5b59e640 1 -- 172.21.15.40:0/2468756981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc6f7d0 msgr2=0x55995dc42a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:03:36.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.305+0000 7fda5b59e640 1 --2- 172.21.15.40:0/2468756981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc6f7d0 0x55995dc42a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.305+0000 7fda5b59e640 1 -- 172.21.15.40:0/2468756981 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55995db72910 con 0x55995dc42f40 2023-10-18T09:03:36.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.305+0000 7fda5a59c640 1 --2- 172.21.15.40:0/2468756981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc6fbb0 0x55995dc72560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:36.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.305+0000 7fda5b59e640 1 --2- 172.21.15.40:0/2468756981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc42f40 0x55995dc72aa0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fda54082ee0 tx=0x7fda54032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556b1354fbd973c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:36.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.306+0000 7fda40ff9640 1 -- 172.21.15.40:0/2468756981 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5407b110 con 0x55995dc42f40 2023-10-18T09:03:36.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.306+0000 7fda40ff9640 1 -- 172.21.15.40:0/2468756981 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fda5407f260 con 0x55995dc42f40 2023-10-18T09:03:36.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.306+0000 7fda40ff9640 1 -- 172.21.15.40:0/2468756981 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54083410 con 0x55995dc42f40 2023-10-18T09:03:36.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.306+0000 7fda63e31740 1 -- 172.21.15.40:0/2468756981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc42f40 msgr2=0x55995dc72aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:36.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.306+0000 7fda63e31740 1 --2- 172.21.15.40:0/2468756981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc42f40 0x55995dc72aa0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fda54082ee0 tx=0x7fda54032340 comp rx=0 tx=0).stop 2023-10-18T09:03:36.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.306+0000 7fda63e31740 1 -- 172.21.15.40:0/2468756981 shutdown_connections 2023-10-18T09:03:36.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.306+0000 7fda63e31740 1 --2- 172.21.15.40:0/2468756981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc42f40 0x55995dc72aa0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.306+0000 7fda63e31740 1 --2- 172.21.15.40:0/2468756981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc6fbb0 0x55995dc72560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.306+0000 7fda63e31740 1 --2- 172.21.15.40:0/2468756981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc6f7d0 0x55995dc42a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.306+0000 7fda63e31740 1 -- 172.21.15.40:0/2468756981 >> 172.21.15.40:0/2468756981 conn(0x55995dc44c70 msgr2=0x55995dc8d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:36.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.307+0000 7fda63e31740 1 -- 172.21.15.40:0/2468756981 shutdown_connections 2023-10-18T09:03:36.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.307+0000 7fda63e31740 1 -- 172.21.15.40:0/2468756981 wait complete. 2023-10-18T09:03:36.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.307+0000 7fda63e31740 1 -- start start 2023-10-18T09:03:36.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.308+0000 7fda63e31740 1 Processor -- start 2023-10-18T09:03:36.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda63e31740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6f7d0 0x55995dc72220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda63e31740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc6fbb0 0x55995dc77880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda63e31740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc9e950 0x55995dc9edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda63e31740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995db4c970 con 0x55995dc6f7d0 2023-10-18T09:03:36.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda63e31740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995db4c610 con 0x55995dc6fbb0 2023-10-18T09:03:36.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda63e31740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55995d912f20 con 0x55995dc9e950 2023-10-18T09:03:36.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda63e31740 10 client.0 _ll_register_callbacks cb 0x55995db700c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:03:36.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda5ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6f7d0 0x55995dc72220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda5ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6f7d0 0x55995dc72220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59800/0 (socket says 172.21.15.40:59800) 2023-10-18T09:03:36.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda5ad9d640 1 -- 172.21.15.40:0/2231995281 learned_addr learned my addr 172.21.15.40:0/2231995281 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:36.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda5a59c640 1 --2- 172.21.15.40:0/2231995281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc6fbb0 0x55995dc77880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda5ad9d640 1 -- 172.21.15.40:0/2231995281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc9e950 msgr2=0x55995dc9edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:03:36.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda5b59e640 1 --2- 172.21.15.40:0/2231995281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc9e950 0x55995dc9edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda5ad9d640 1 --2- 172.21.15.40:0/2231995281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc9e950 0x55995dc9edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda5ad9d640 1 -- 172.21.15.40:0/2231995281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc6fbb0 msgr2=0x55995dc77880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:03:36.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda5ad9d640 1 --2- 172.21.15.40:0/2231995281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc6fbb0 0x55995dc77880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda5ad9d640 1 -- 172.21.15.40:0/2231995281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c15b710 con 0x55995dc6f7d0 2023-10-18T09:03:36.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.309+0000 7fda5a59c640 1 --2- 172.21.15.40:0/2231995281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc6fbb0 0x55995dc77880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:36.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.310+0000 7fda5ad9d640 1 --2- 172.21.15.40:0/2231995281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6f7d0 0x55995dc72220 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fda4c013e20 tx=0x7fda4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e13b976299d5b63 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:36.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.310+0000 7fda39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T09:03:36.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.310+0000 7fda39ffb640 1 -- 172.21.15.40:0/2231995281 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00b040 con 0x55995dc6f7d0 2023-10-18T09:03:36.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.310+0000 7fda63e31740 1 -- 172.21.15.40:0/2231995281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55995dba7700 con 0x55995dc6f7d0 2023-10-18T09:03:36.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.310+0000 7fda39ffb640 1 -- 172.21.15.40:0/2231995281 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda4c011e40 con 0x55995dc6f7d0 2023-10-18T09:03:36.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.310+0000 7fda39ffb640 1 -- 172.21.15.40:0/2231995281 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda4c00ced0 con 0x55995dc6f7d0 2023-10-18T09:03:36.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.310+0000 7fda63e31740 10 client.8055 fetch_fsmap learned FSMap version 66 2023-10-18T09:03:36.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.311+0000 7fda63e31740 1 -- 172.21.15.40:0/2231995281 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x55995dbacda0 con 0x55995dc6f7d0 2023-10-18T09:03:36.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.311+0000 7fda39ffb640 1 -- 172.21.15.40:0/2231995281 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55995dba7700 con 0x55995dc6f7d0 2023-10-18T09:03:36.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.311+0000 7fda39ffb640 1 -- 172.21.15.40:0/2231995281 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fda4c01f420 con 0x55995dc6f7d0 2023-10-18T09:03:36.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.311+0000 7fda63e31740 10 client.8055 fetch_fsmap finished waiting for FSMap version 66 2023-10-18T09:03:36.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.311+0000 7fda63e31740 10 client.8055 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T09:03:36.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.311+0000 7fda63e31740 20 client.8055 populate_metadata read hostname 'smithi040' 2023-10-18T09:03:36.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.311+0000 7fda63e31740 1 --2- 172.21.15.40:0/2231995281 >> [v2:172.21.15.40:6834/2600609919,v1:172.21.15.40:6835/2600609919] conn(0x55995dca1480 0x55995dc9d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.311+0000 7fda63e31740 4 client.8055 mds_command: new command op to 7893 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:03:36.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.311+0000 7fda63e31740 1 -- 172.21.15.40:0/2231995281 --> [v2:172.21.15.40:6834/2600609919,v1:172.21.15.40:6835/2600609919] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55995dba7420 con 0x55995dca1480 2023-10-18T09:03:36.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.311+0000 7fda5a59c640 1 --2- 172.21.15.40:0/2231995281 >> [v2:172.21.15.40:6834/2600609919,v1:172.21.15.40:6835/2600609919] conn(0x55995dca1480 0x55995dc9d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.312+0000 7fda5a59c640 1 --2- 172.21.15.40:0/2231995281 >> [v2:172.21.15.40:6834/2600609919,v1:172.21.15.40:6835/2600609919] conn(0x55995dca1480 0x55995dc9d7d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8b775b62646e2dc server_cookie=b460a347177f4dbc in_seq=0 out_seq=0 2023-10-18T09:03:36.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.312+0000 7fda39ffb640 10 client.8055 ms_handle_connect on v2:172.21.15.40:6834/2600609919 2023-10-18T09:03:36.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.313+0000 7fda39ffb640 1 -- 172.21.15.40:0/2231995281 <== mds.0 v2:172.21.15.40:6834/2600609919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55995dba7420 con 0x55995dca1480 2023-10-18T09:03:36.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.313+0000 7fda39ffb640 10 client.8055 handle_command_reply: tid=0 2023-10-18T09:03:36.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.313+0000 7fda63e31740 1 client.8055 shutdown 2023-10-18T09:03:36.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.313+0000 7fda63e31740 1 -- 172.21.15.40:0/2231995281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6f7d0 msgr2=0x55995dc72220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:36.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.313+0000 7fda63e31740 1 --2- 172.21.15.40:0/2231995281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6f7d0 0x55995dc72220 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fda4c013e20 tx=0x7fda4c004a40 comp rx=0 tx=0).stop 2023-10-18T09:03:36.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.313+0000 7fda63e31740 1 -- 172.21.15.40:0/2231995281 shutdown_connections 2023-10-18T09:03:36.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.313+0000 7fda63e31740 1 --2- 172.21.15.40:0/2231995281 >> [v2:172.21.15.40:6834/2600609919,v1:172.21.15.40:6835/2600609919] conn(0x55995dca1480 0x55995dc9d7d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.313+0000 7fda63e31740 1 --2- 172.21.15.40:0/2231995281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc9e950 0x55995dc9edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.313+0000 7fda63e31740 1 --2- 172.21.15.40:0/2231995281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc6fbb0 0x55995dc77880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.313+0000 7fda63e31740 1 --2- 172.21.15.40:0/2231995281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6f7d0 0x55995dc72220 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.313+0000 7fda39ffb640 0 client.8055 ms_handle_reset on v2:172.21.15.40:6834/2600609919 2023-10-18T09:03:36.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.313+0000 7fda63e31740 1 -- 172.21.15.40:0/2231995281 >> 172.21.15.40:0/2231995281 conn(0x55995dc42f40 msgr2=0x55995dc90a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:36.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.313+0000 7fda63e31740 1 -- 172.21.15.40:0/2231995281 shutdown_connections 2023-10-18T09:03:36.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.313+0000 7fda63e31740 1 -- 172.21.15.40:0/2231995281 wait complete. 2023-10-18T09:03:36.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.313+0000 7fda63e31740 20 client.8055 trim_cache size 0 max 16384 2023-10-18T09:03:36.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.330+0000 7fda63e31740 1 Processor -- start 2023-10-18T09:03:36.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.330+0000 7fda63e31740 1 -- start start 2023-10-18T09:03:36.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.331+0000 7fda63e31740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6f7d0 0x55995dc90a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.331+0000 7fda63e31740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc6fbb0 0x55995dc90fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.331+0000 7fda63e31740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc9e950 0x55995dc8d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.331+0000 7fda63e31740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995dc48980 con 0x55995dc6f7d0 2023-10-18T09:03:36.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.331+0000 7fda63e31740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995dc77b70 con 0x55995dc9e950 2023-10-18T09:03:36.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.331+0000 7fda63e31740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55995dc6b3b0 con 0x55995dc6fbb0 2023-10-18T09:03:36.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.331+0000 7fda5ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6f7d0 0x55995dc90a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.331+0000 7fda5ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6f7d0 0x55995dc90a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59810/0 (socket says 172.21.15.40:59810) 2023-10-18T09:03:36.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.331+0000 7fda5ad9d640 1 -- 172.21.15.40:0/3992868360 learned_addr learned my addr 172.21.15.40:0/3992868360 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:36.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.332+0000 7fda5b59e640 1 --2- 172.21.15.40:0/3992868360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc9e950 0x55995dc8d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.332+0000 7fda5ad9d640 1 -- 172.21.15.40:0/3992868360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc6fbb0 msgr2=0x55995dc90fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:03:36.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.332+0000 7fda5ad9d640 1 --2- 172.21.15.40:0/3992868360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc6fbb0 0x55995dc90fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.332+0000 7fda5ad9d640 1 -- 172.21.15.40:0/3992868360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc9e950 msgr2=0x55995dc8d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:03:36.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.332+0000 7fda5ad9d640 1 --2- 172.21.15.40:0/3992868360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc9e950 0x55995dc8d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.332+0000 7fda5ad9d640 1 -- 172.21.15.40:0/3992868360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55995db4bb80 con 0x55995dc6f7d0 2023-10-18T09:03:36.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.332+0000 7fda5b59e640 1 --2- 172.21.15.40:0/3992868360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc9e950 0x55995dc8d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:36.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.332+0000 7fda5ad9d640 1 --2- 172.21.15.40:0/3992868360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6f7d0 0x55995dc90a70 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fda4c013810 tx=0x7fda4c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4f4a406bebce0b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:36.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.333+0000 7fda417fa640 1 -- 172.21.15.40:0/3992868360 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c007500 con 0x55995dc6f7d0 2023-10-18T09:03:36.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.333+0000 7fda417fa640 1 -- 172.21.15.40:0/3992868360 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fda4c01f420 con 0x55995dc6f7d0 2023-10-18T09:03:36.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.333+0000 7fda417fa640 1 -- 172.21.15.40:0/3992868360 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c01f590 con 0x55995dc6f7d0 2023-10-18T09:03:36.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.333+0000 7fda63e31740 1 -- 172.21.15.40:0/3992868360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6f7d0 msgr2=0x55995dc90a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:36.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.333+0000 7fda63e31740 1 --2- 172.21.15.40:0/3992868360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6f7d0 0x55995dc90a70 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fda4c013810 tx=0x7fda4c004ab0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.333+0000 7fda63e31740 1 -- 172.21.15.40:0/3992868360 shutdown_connections 2023-10-18T09:03:36.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.333+0000 7fda63e31740 1 --2- 172.21.15.40:0/3992868360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc9e950 0x55995dc8d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.333+0000 7fda63e31740 1 --2- 172.21.15.40:0/3992868360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc6fbb0 0x55995dc90fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.333+0000 7fda63e31740 1 --2- 172.21.15.40:0/3992868360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6f7d0 0x55995dc90a70 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.333+0000 7fda63e31740 1 -- 172.21.15.40:0/3992868360 >> 172.21.15.40:0/3992868360 conn(0x55995dc42f40 msgr2=0x55995dc6a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:36.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.333+0000 7fda63e31740 1 -- 172.21.15.40:0/3992868360 shutdown_connections 2023-10-18T09:03:36.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.333+0000 7fda63e31740 1 -- 172.21.15.40:0/3992868360 wait complete. 2023-10-18T09:03:36.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.334+0000 7fda63e31740 1 -- start start 2023-10-18T09:03:36.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.334+0000 7fda63e31740 1 Processor -- start 2023-10-18T09:03:36.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.335+0000 7fda63e31740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc6f7d0 0x55995dca7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.335+0000 7fda63e31740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6fbb0 0x55995dc49940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.335+0000 7fda63e31740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc9e950 0x55995dcf50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.335+0000 7fda63e31740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995dcf9b10 con 0x55995dc6fbb0 2023-10-18T09:03:36.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.335+0000 7fda63e31740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55995dca41e0 con 0x55995dc9e950 2023-10-18T09:03:36.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.335+0000 7fda63e31740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55995dca4600 con 0x55995dc6f7d0 2023-10-18T09:03:36.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.335+0000 7fda63e31740 10 client.0 _ll_register_callbacks cb 0x7fda54077450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:03:36.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.336+0000 7fda5a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6fbb0 0x55995dc49940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.336+0000 7fda5a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6fbb0 0x55995dc49940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59814/0 (socket says 172.21.15.40:59814) 2023-10-18T09:03:36.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.336+0000 7fda5a59c640 1 -- 172.21.15.40:0/3329606330 learned_addr learned my addr 172.21.15.40:0/3329606330 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:36.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.336+0000 7fda5a59c640 1 -- 172.21.15.40:0/3329606330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc6f7d0 msgr2=0x55995dca7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:03:36.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.336+0000 7fda5a59c640 1 --2- 172.21.15.40:0/3329606330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc6f7d0 0x55995dca7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.336+0000 7fda5a59c640 1 -- 172.21.15.40:0/3329606330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc9e950 msgr2=0x55995dcf50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:03:36.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.336+0000 7fda5a59c640 1 --2- 172.21.15.40:0/3329606330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc9e950 0x55995dcf50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.336+0000 7fda5a59c640 1 -- 172.21.15.40:0/3329606330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55995dbacda0 con 0x55995dc6fbb0 2023-10-18T09:03:36.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.336+0000 7fda5a59c640 1 --2- 172.21.15.40:0/3329606330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6fbb0 0x55995dc49940 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fda50006fd0 tx=0x7fda5000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e18805f9cebbaf server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:36.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.336+0000 7fda39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T09:03:36.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.337+0000 7fda39ffb640 1 -- 172.21.15.40:0/3329606330 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5000b040 con 0x55995dc6fbb0 2023-10-18T09:03:36.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.337+0000 7fda39ffb640 1 -- 172.21.15.40:0/3329606330 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda50014860 con 0x55995dc6fbb0 2023-10-18T09:03:36.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.337+0000 7fda63e31740 1 -- 172.21.15.40:0/3329606330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda5407f260 con 0x55995dc6fbb0 2023-10-18T09:03:36.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.337+0000 7fda39ffb640 1 -- 172.21.15.40:0/3329606330 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda50016580 con 0x55995dc6fbb0 2023-10-18T09:03:36.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.337+0000 7fda63e31740 10 client.8061 fetch_fsmap learned FSMap version 66 2023-10-18T09:03:36.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.337+0000 7fda63e31740 1 -- 172.21.15.40:0/3329606330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x55995db71170 con 0x55995dc6fbb0 2023-10-18T09:03:36.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.337+0000 7fda39ffb640 1 -- 172.21.15.40:0/3329606330 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fda5407f260 con 0x55995dc6fbb0 2023-10-18T09:03:36.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.337+0000 7fda39ffb640 1 -- 172.21.15.40:0/3329606330 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fda50014b10 con 0x55995dc6fbb0 2023-10-18T09:03:36.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.337+0000 7fda63e31740 10 client.8061 fetch_fsmap finished waiting for FSMap version 66 2023-10-18T09:03:36.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.337+0000 7fda63e31740 10 client.8061 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T09:03:36.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.337+0000 7fda63e31740 20 client.8061 populate_metadata read hostname 'smithi040' 2023-10-18T09:03:36.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.337+0000 7fda63e31740 1 --2- 172.21.15.40:0/3329606330 >> [v2:172.21.15.40:6834/2600609919,v1:172.21.15.40:6835/2600609919] conn(0x55995dc90bb0 0x55995dc90f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.337+0000 7fda63e31740 4 client.8061 mds_command: new command op to 7893 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:03:36.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.337+0000 7fda63e31740 1 -- 172.21.15.40:0/3329606330 --> [v2:172.21.15.40:6834/2600609919,v1:172.21.15.40:6835/2600609919] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55995d8b06c0 con 0x55995dc90bb0 2023-10-18T09:03:36.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.338+0000 7fda5ad9d640 1 --2- 172.21.15.40:0/3329606330 >> [v2:172.21.15.40:6834/2600609919,v1:172.21.15.40:6835/2600609919] conn(0x55995dc90bb0 0x55995dc90f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.338+0000 7fda5ad9d640 1 --2- 172.21.15.40:0/3329606330 >> [v2:172.21.15.40:6834/2600609919,v1:172.21.15.40:6835/2600609919] conn(0x55995dc90bb0 0x55995dc90f90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0e87d1ffb408c6 server_cookie=bb41ef68de94bc9f in_seq=0 out_seq=0 2023-10-18T09:03:36.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.338+0000 7fda39ffb640 10 client.8061 ms_handle_connect on v2:172.21.15.40:6834/2600609919 2023-10-18T09:03:36.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.338+0000 7fda39ffb640 1 -- 172.21.15.40:0/3329606330 <== mds.0 v2:172.21.15.40:6834/2600609919 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55995d8b06c0 con 0x55995dc90bb0 2023-10-18T09:03:36.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.338+0000 7fda39ffb640 10 client.8061 handle_command_reply: tid=0 2023-10-18T09:03:36.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.339+0000 7fda63e31740 1 client.8061 shutdown 2023-10-18T09:03:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.339+0000 7fda63e31740 1 -- 172.21.15.40:0/3329606330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6fbb0 msgr2=0x55995dc49940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.339+0000 7fda63e31740 1 --2- 172.21.15.40:0/3329606330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6fbb0 0x55995dc49940 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fda50006fd0 tx=0x7fda5000be20 comp rx=0 tx=0).stop 2023-10-18T09:03:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.339+0000 7fda63e31740 1 -- 172.21.15.40:0/3329606330 shutdown_connections 2023-10-18T09:03:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.339+0000 7fda63e31740 1 --2- 172.21.15.40:0/3329606330 >> [v2:172.21.15.40:6834/2600609919,v1:172.21.15.40:6835/2600609919] conn(0x55995dc90bb0 0x55995dc90f90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.339+0000 7fda63e31740 1 --2- 172.21.15.40:0/3329606330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55995dc9e950 0x55995dcf50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.339+0000 7fda63e31740 1 --2- 172.21.15.40:0/3329606330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55995dc6fbb0 0x55995dc49940 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.339+0000 7fda63e31740 1 --2- 172.21.15.40:0/3329606330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55995dc6f7d0 0x55995dca7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.339+0000 7fda39ffb640 0 client.8061 ms_handle_reset on v2:172.21.15.40:6834/2600609919 2023-10-18T09:03:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.339+0000 7fda63e31740 1 -- 172.21.15.40:0/3329606330 >> 172.21.15.40:0/3329606330 conn(0x55995dc42f40 msgr2=0x55995dc41a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.339+0000 7fda63e31740 1 -- 172.21.15.40:0/3329606330 shutdown_connections 2023-10-18T09:03:36.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.339+0000 7fda63e31740 1 -- 172.21.15.40:0/3329606330 wait complete. 2023-10-18T09:03:36.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.339+0000 7fda63e31740 20 client.8061 trim_cache size 0 max 16384 2023-10-18T09:03:36.354 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:03:36.354 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:03:36.355 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-10-18T09:03:36.355 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 7893, 2023-10-18T09:03:36.355 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-10-18T09:03:36.355 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-10-18T09:03:36.355 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-10-18T09:03:36.355 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 11.96034182, 2023-10-18T09:03:36.356 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 66, 2023-10-18T09:03:36.356 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 144, 2023-10-18T09:03:36.356 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 144, 2023-10-18T09:03:36.356 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 17.881929832000001 2023-10-18T09:03:36.356 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:03:36.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.340+0000 7fda611c5640 1 -- 172.21.15.40:0/3915900672 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fda4806ae40 msgr2=0x7fda4806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:36.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.340+0000 7fda611c5640 1 --2- 172.21.15.40:0/3915900672 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fda4806ae40 0x7fda4806d300 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0abc90 tx=0x7fda4c007310 comp rx=0 tx=0).stop 2023-10-18T09:03:36.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.340+0000 7fda611c5640 1 -- 172.21.15.40:0/3915900672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5c15eaa0 msgr2=0x7fda5c149690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:36.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.340+0000 7fda611c5640 1 --2- 172.21.15.40:0/3915900672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5c15eaa0 0x7fda5c149690 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fda54006010 tx=0x7fda5402db00 comp rx=0 tx=0).stop 2023-10-18T09:03:36.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.340+0000 7fda611c5640 1 -- 172.21.15.40:0/3915900672 shutdown_connections 2023-10-18T09:03:36.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.340+0000 7fda611c5640 1 --2- 172.21.15.40:0/3915900672 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fda4806ae40 0x7fda4806d300 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.340+0000 7fda611c5640 1 --2- 172.21.15.40:0/3915900672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda5c15eaa0 0x7fda5c149690 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.340+0000 7fda611c5640 1 --2- 172.21.15.40:0/3915900672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda5c0b1b90 0x7fda5c0cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.340+0000 7fda611c5640 1 --2- 172.21.15.40:0/3915900672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda5c06ffe0 0x7fda5c15b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.340+0000 7fda611c5640 1 -- 172.21.15.40:0/3915900672 >> 172.21.15.40:0/3915900672 conn(0x7fda5c06cee0 msgr2=0x7fda5c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:36.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.340+0000 7fda611c5640 1 -- 172.21.15.40:0/3915900672 shutdown_connections 2023-10-18T09:03:36.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.340+0000 7fda611c5640 1 -- 172.21.15.40:0/3915900672 wait complete. 2023-10-18T09:03:36.361 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T09:03:36.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.583+0000 7f9198494640 1 -- 172.21.15.40:0/3967886713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f917c006290 msgr2=0x7f917c00ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.583+0000 7f9198494640 1 --2- 172.21.15.40:0/3967886713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f917c006290 0x7f917c00ca90 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f9188001d80 tx=0x7f917c006900 comp rx=0 tx=0).stop 2023-10-18T09:03:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.583+0000 7f9198494640 1 -- 172.21.15.40:0/3967886713 shutdown_connections 2023-10-18T09:03:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.583+0000 7f9198494640 1 --2- 172.21.15.40:0/3967886713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f917c006290 0x7f917c00ca90 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.583+0000 7f9198494640 1 -- 172.21.15.40:0/3967886713 >> 172.21.15.40:0/3967886713 conn(0x7f9190069720 msgr2=0x7f9190069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.583+0000 7f9198494640 1 -- 172.21.15.40:0/3967886713 shutdown_connections 2023-10-18T09:03:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.583+0000 7f9198494640 1 -- 172.21.15.40:0/3967886713 wait complete. 2023-10-18T09:03:36.588 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:03:36.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.585+0000 7f9198494640 1 Processor -- start 2023-10-18T09:03:36.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.585+0000 7f9198494640 1 -- start start 2023-10-18T09:03:36.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.586+0000 7f9198494640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f917c006290 0x7f9190149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.586+0000 7f9198494640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91900cd6c0 0x7f91901505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.586+0000 7f9198494640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91900cdb40 0x7f9190150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.586+0000 7f9198494640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9190149db0 con 0x7f91900cd6c0 2023-10-18T09:03:36.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.586+0000 7f9198494640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91901e0fd0 con 0x7f91900cdb40 2023-10-18T09:03:36.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.586+0000 7f9198494640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91901e12b0 con 0x7f917c006290 2023-10-18T09:03:36.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.586+0000 7f9195a2f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91900cd6c0 0x7f91901505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.587+0000 7f9195a2f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91900cd6c0 0x7f91901505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59836/0 (socket says 172.21.15.40:59836) 2023-10-18T09:03:36.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.587+0000 7f9195a2f640 1 -- 172.21.15.40:0/2501633254 learned_addr learned my addr 172.21.15.40:0/2501633254 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:36.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.587+0000 7f9196230640 1 --2- 172.21.15.40:0/2501633254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f917c006290 0x7f9190149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.587+0000 7f9196a31640 1 --2- 172.21.15.40:0/2501633254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91900cdb40 0x7f9190150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.587+0000 7f9195a2f640 1 -- 172.21.15.40:0/2501633254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f917c006290 msgr2=0x7f9190149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.587+0000 7f9195a2f640 1 --2- 172.21.15.40:0/2501633254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f917c006290 0x7f9190149870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.587+0000 7f9195a2f640 1 -- 172.21.15.40:0/2501633254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91900cdb40 msgr2=0x7f9190150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.587+0000 7f9195a2f640 1 --2- 172.21.15.40:0/2501633254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91900cdb40 0x7f9190150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.587+0000 7f9195a2f640 1 -- 172.21.15.40:0/2501633254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91901e1590 con 0x7f91900cd6c0 2023-10-18T09:03:36.594 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:03:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.587+0000 7f9195a2f640 1 --2- 172.21.15.40:0/2501633254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91900cd6c0 0x7f91901505f0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f918000d970 tx=0x7f918000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.588+0000 7f9186ffd640 1 -- 172.21.15.40:0/2501633254 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180006e80 con 0x7f91900cd6c0 2023-10-18T09:03:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.588+0000 7f9186ffd640 1 -- 172.21.15.40:0/2501633254 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f91800063d0 con 0x7f91900cd6c0 2023-10-18T09:03:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.588+0000 7f9186ffd640 1 -- 172.21.15.40:0/2501633254 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918000ba30 con 0x7f91900cd6c0 2023-10-18T09:03:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.588+0000 7f9198494640 1 -- 172.21.15.40:0/2501633254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91901e1870 con 0x7f91900cd6c0 2023-10-18T09:03:36.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.588+0000 7f9198494640 1 -- 172.21.15.40:0/2501633254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9190154eb0 con 0x7f91900cd6c0 2023-10-18T09:03:36.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.589+0000 7f9198494640 1 -- 172.21.15.40:0/2501633254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9190066fd0 con 0x7f91900cd6c0 2023-10-18T09:03:36.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.590+0000 7f9186ffd640 1 -- 172.21.15.40:0/2501633254 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f918000bb90 con 0x7f91900cd6c0 2023-10-18T09:03:36.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.591+0000 7f9186ffd640 1 --2- 172.21.15.40:0/2501633254 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f917406ae70 0x7f917406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:36.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.591+0000 7f9186ffd640 1 -- 172.21.15.40:0/2501633254 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9180089e70 con 0x7f91900cd6c0 2023-10-18T09:03:36.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.592+0000 7f9196230640 1 --2- 172.21.15.40:0/2501633254 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f917406ae70 0x7f917406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:36.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.593+0000 7f9196230640 1 --2- 172.21.15.40:0/2501633254 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f917406ae70 0x7f917406d330 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f918c0057c0 tx=0x7f918c0097e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:36.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.595+0000 7f9186ffd640 1 -- 172.21.15.40:0/2501633254 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9180054e80 con 0x7f91900cd6c0 2023-10-18T09:03:36.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.833+0000 7f9198494640 1 -- 172.21.15.40:0/2501633254 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9190158760 con 0x7f91900cd6c0 2023-10-18T09:03:36.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.834+0000 7f9186ffd640 1 -- 172.21.15.40:0/2501633254 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9190158760 con 0x7f91900cd6c0 2023-10-18T09:03:36.837 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T09:03:36.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.837+0000 7f9198494640 1 -- 172.21.15.40:0/2501633254 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f917406ae70 msgr2=0x7f917406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:36.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.837+0000 7f9198494640 1 --2- 172.21.15.40:0/2501633254 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f917406ae70 0x7f917406d330 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f918c0057c0 tx=0x7f918c0097e0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.837+0000 7f9198494640 1 -- 172.21.15.40:0/2501633254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91900cd6c0 msgr2=0x7f91901505f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:36.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.837+0000 7f9198494640 1 --2- 172.21.15.40:0/2501633254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91900cd6c0 0x7f91901505f0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f918000d970 tx=0x7f918000dd70 comp rx=0 tx=0).stop 2023-10-18T09:03:36.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.837+0000 7f9198494640 1 -- 172.21.15.40:0/2501633254 shutdown_connections 2023-10-18T09:03:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.837+0000 7f9198494640 1 --2- 172.21.15.40:0/2501633254 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f917406ae70 0x7f917406d330 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.837+0000 7f9198494640 1 --2- 172.21.15.40:0/2501633254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91900cdb40 0x7f9190150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.837+0000 7f9198494640 1 --2- 172.21.15.40:0/2501633254 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91900cd6c0 0x7f91901505f0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.837+0000 7f9198494640 1 --2- 172.21.15.40:0/2501633254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f917c006290 0x7f9190149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:36.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.837+0000 7f9198494640 1 -- 172.21.15.40:0/2501633254 >> 172.21.15.40:0/2501633254 conn(0x7f9190069720 msgr2=0x7f9190155700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:36.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.837+0000 7f9198494640 1 -- 172.21.15.40:0/2501633254 shutdown_connections 2023-10-18T09:03:36.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:36.837+0000 7f9198494640 1 -- 172.21.15.40:0/2501633254 wait complete. 2023-10-18T09:03:36.858 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T09:03:36.859 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T09:03:36.859 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-10-18T09:03:36.860 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T09:03:36.860 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T09:03:36.860 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T09:03:36.860 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T09:03:36.861 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T09:03:36.861 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:03:36.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:36.862 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-10-18T09:03:36.915 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T09:03:36.915 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T09:03:36.915 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T09:03:36.916 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:03:36.916 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-18T09:03:36.916 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:03:36.916 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:03:36.916 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:03:36.917 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-10-18T09:03:36.917 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:03:36.917 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:03:36.918 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-10-18T09:03:36.918 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T09:03:36.918 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-10-18T09:03:36.918 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-10-18T09:03:36.918 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-18T09:03:36.918 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 41907sec preferred_lft 41907sec 2023-10-18T09:03:36.919 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2023-10-18T09:03:36.919 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:03:36.919 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:03:36.919 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-10-18T09:03:36.919 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-10-18T09:03:36.919 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T09:03:36.920 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:03:36.920 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T09:03:36.920 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:03:36.920 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2026:95ff:fea9:a463/64 scope link 2023-10-18T09:03:36.920 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:03:36.921 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:03:36.921 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T09:03:36.921 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::80bd:e5ff:fe0f:6f68/64 scope link 2023-10-18T09:03:36.921 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:03:36.921 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:03:36.922 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether ea:2b:21:f5:06:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T09:03:36.922 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::e82b:21ff:fef5:6af/64 scope link 2023-10-18T09:03:36.922 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:03:36.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:36.923 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T09:03:37.019 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-10-18T09:03:37.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:37.024 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-10-18T09:03:37.084 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-10-18T09:03:37.084 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T09:03:37.084 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-18T09:03:37.085 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-18T09:03:37.085 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T09:03:37.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:37.086 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T09:03:37.086 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T09:03:37.086 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-18T09:03:37.087 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-18T09:03:37.087 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-18T09:03:37.087 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T09:03:37.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:37.292 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-18T09:03:37.346 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T09:03:37.347 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T09:03:37.348 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:03:37.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:37.348 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-10-18T09:03:37.452 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:03:37.453 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T09:03:37.512 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:03:37.513 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:03:37.513 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T09:03:37.569 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-18T09:03:37.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:37.570 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:03:37.724 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:03:37.725 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:03:37.757 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:03:37.757 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:03:37.758 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T09:03:37.758 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T09:03:37.758 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T09:03:37.762 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-18T09:03:37.762 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2023-10-18T09:03:37.762 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T09:03:37.762 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T09:03:37.763 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T09:03:37.766 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T09:03:37.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:37.767 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T09:03:37.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:37.859 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:03:37.923 INFO:teuthology.orchestra.run.smithi040.stdout:client6472 2023-10-18T09:03:37.923 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:03:37.924 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:03:37.979 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T09:03:38.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.214+0000 7f6b07483640 1 -- 172.21.15.40:0/1448654281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec006290 msgr2=0x7f6aec00ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:38.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.214+0000 7f6b07483640 1 --2- 172.21.15.40:0/1448654281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec006290 0x7f6aec00ca90 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f6af800c640 tx=0x7f6aec006900 comp rx=0 tx=0).stop 2023-10-18T09:03:38.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.214+0000 7f6b07483640 1 -- 172.21.15.40:0/1448654281 shutdown_connections 2023-10-18T09:03:38.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.214+0000 7f6b07483640 1 --2- 172.21.15.40:0/1448654281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec006290 0x7f6aec00ca90 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:38.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.214+0000 7f6b07483640 1 -- 172.21.15.40:0/1448654281 >> 172.21.15.40:0/1448654281 conn(0x7f6b000aed70 msgr2=0x7f6b000af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:38.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.214+0000 7f6b07483640 1 -- 172.21.15.40:0/1448654281 shutdown_connections 2023-10-18T09:03:38.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.214+0000 7f6b07483640 1 -- 172.21.15.40:0/1448654281 wait complete. 2023-10-18T09:03:38.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.216+0000 7f6b07483640 1 Processor -- start 2023-10-18T09:03:38.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.216+0000 7f6b07483640 1 -- start start 2023-10-18T09:03:38.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.217+0000 7f6b07483640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec006290 0x7f6b00138280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:38.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.217+0000 7f6b07483640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b00138b00 0x7f6b00140240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:38.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.217+0000 7f6b07483640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b000cbf10 0x7f6b000cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:38.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.217+0000 7f6b07483640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0013aef0 con 0x7f6aec006290 2023-10-18T09:03:38.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.217+0000 7f6b07483640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b000cc8d0 con 0x7f6b000cbf10 2023-10-18T09:03:38.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.217+0000 7f6b07483640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0013a020 con 0x7f6b00138b00 2023-10-18T09:03:38.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.217+0000 7f6b06481640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec006290 0x7f6b00138280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:38.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.217+0000 7f6b06481640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec006290 0x7f6b00138280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59850/0 (socket says 172.21.15.40:59850) 2023-10-18T09:03:38.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.217+0000 7f6b06481640 1 -- 172.21.15.40:0/2710497485 learned_addr learned my addr 172.21.15.40:0/2710497485 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:38.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.218+0000 7f6b05c80640 1 --2- 172.21.15.40:0/2710497485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b00138b00 0x7f6b00140240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:38.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.218+0000 7f6b06481640 1 -- 172.21.15.40:0/2710497485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b00138b00 msgr2=0x7f6b00140240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:38.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.218+0000 7f6b06c82640 1 --2- 172.21.15.40:0/2710497485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b000cbf10 0x7f6b000cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:38.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.218+0000 7f6b06481640 1 --2- 172.21.15.40:0/2710497485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b00138b00 0x7f6b00140240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:38.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.218+0000 7f6b06481640 1 -- 172.21.15.40:0/2710497485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b000cbf10 msgr2=0x7f6b000cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:38.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.218+0000 7f6b06481640 1 --2- 172.21.15.40:0/2710497485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b000cbf10 0x7f6b000cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:38.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.218+0000 7f6b06481640 1 -- 172.21.15.40:0/2710497485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b001387c0 con 0x7f6aec006290 2023-10-18T09:03:38.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.218+0000 7f6b05c80640 1 --2- 172.21.15.40:0/2710497485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b00138b00 0x7f6b00140240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:38.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.218+0000 7f6b06481640 1 --2- 172.21.15.40:0/2710497485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec006290 0x7f6b00138280 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f6afc0117d0 tx=0x7f6afc011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:38.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.219+0000 7f6af6ffd640 1 -- 172.21.15.40:0/2710497485 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc011ee0 con 0x7f6aec006290 2023-10-18T09:03:38.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.219+0000 7f6af6ffd640 1 -- 172.21.15.40:0/2710497485 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6afc00f940 con 0x7f6aec006290 2023-10-18T09:03:38.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.219+0000 7f6af6ffd640 1 -- 172.21.15.40:0/2710497485 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc00fcd0 con 0x7f6aec006290 2023-10-18T09:03:38.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.219+0000 7f6b07483640 1 -- 172.21.15.40:0/2710497485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0013a270 con 0x7f6aec006290 2023-10-18T09:03:38.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.219+0000 7f6b07483640 1 -- 172.21.15.40:0/2710497485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0013a830 con 0x7f6aec006290 2023-10-18T09:03:38.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.220+0000 7f6b07483640 1 -- 172.21.15.40:0/2710497485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0013b160 con 0x7f6aec006290 2023-10-18T09:03:38.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.221+0000 7f6af6ffd640 1 -- 172.21.15.40:0/2710497485 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6afc0035f0 con 0x7f6aec006290 2023-10-18T09:03:38.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.222+0000 7f6af6ffd640 1 --2- 172.21.15.40:0/2710497485 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ae406ae40 0x7f6ae406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:38.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.222+0000 7f6af6ffd640 1 -- 172.21.15.40:0/2710497485 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6afc08a390 con 0x7f6aec006290 2023-10-18T09:03:38.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.222+0000 7f6b05c80640 1 --2- 172.21.15.40:0/2710497485 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ae406ae40 0x7f6ae406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:38.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.224+0000 7f6b05c80640 1 --2- 172.21.15.40:0/2710497485 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ae406ae40 0x7f6ae406d300 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6b00158910 tx=0x7f6af0006a50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:38.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.226+0000 7f6af6ffd640 1 -- 172.21.15.40:0/2710497485 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6afc0554d0 con 0x7f6aec006290 2023-10-18T09:03:38.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.460+0000 7f6b07483640 1 -- 172.21.15.40:0/2710497485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6b00073820 con 0x7f6aec006290 2023-10-18T09:03:38.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.461+0000 7f6af6ffd640 1 -- 172.21.15.40:0/2710497485 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6afc058af0 con 0x7f6aec006290 2023-10-18T09:03:38.465 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T09:03:38.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.464+0000 7f6b07483640 1 -- 172.21.15.40:0/2710497485 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ae406ae40 msgr2=0x7f6ae406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:38.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.464+0000 7f6b07483640 1 --2- 172.21.15.40:0/2710497485 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ae406ae40 0x7f6ae406d300 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6b00158910 tx=0x7f6af0006a50 comp rx=0 tx=0).stop 2023-10-18T09:03:38.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.464+0000 7f6b07483640 1 -- 172.21.15.40:0/2710497485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec006290 msgr2=0x7f6b00138280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.464+0000 7f6b07483640 1 --2- 172.21.15.40:0/2710497485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec006290 0x7f6b00138280 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f6afc0117d0 tx=0x7f6afc011c90 comp rx=0 tx=0).stop 2023-10-18T09:03:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.464+0000 7f6b07483640 1 -- 172.21.15.40:0/2710497485 shutdown_connections 2023-10-18T09:03:38.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.464+0000 7f6b07483640 1 --2- 172.21.15.40:0/2710497485 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6ae406ae40 0x7f6ae406d300 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.464+0000 7f6b07483640 1 --2- 172.21.15.40:0/2710497485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b000cbf10 0x7f6b000cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.464+0000 7f6b07483640 1 --2- 172.21.15.40:0/2710497485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b00138b00 0x7f6b00140240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.464+0000 7f6b07483640 1 --2- 172.21.15.40:0/2710497485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6aec006290 0x7f6b00138280 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.464+0000 7f6b07483640 1 -- 172.21.15.40:0/2710497485 >> 172.21.15.40:0/2710497485 conn(0x7f6b000aed70 msgr2=0x7f6b00139460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:38.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.464+0000 7f6b07483640 1 -- 172.21.15.40:0/2710497485 shutdown_connections 2023-10-18T09:03:38.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:38.464+0000 7f6b07483640 1 -- 172.21.15.40:0/2710497485 wait complete. 2023-10-18T09:03:38.488 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T09:03:38.489 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T09:03:38.489 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-10-18T09:03:38.490 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-18T09:03:38.491 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T09:03:38.491 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T09:03:38.492 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T09:03:38.492 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T09:03:38.493 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:03:38.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:38.494 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-10-18T09:03:38.555 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T09:03:38.555 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T09:03:38.556 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T09:03:38.556 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:03:38.556 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-18T09:03:38.556 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:03:38.556 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:03:38.557 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:03:38.557 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2023-10-18T09:03:38.557 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:03:38.557 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:03:38.557 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2023-10-18T09:03:38.558 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:03:38.558 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-10-18T09:03:38.559 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T09:03:38.559 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-10-18T09:03:38.559 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T09:03:38.559 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 41899sec preferred_lft 41899sec 2023-10-18T09:03:38.559 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2023-10-18T09:03:38.560 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:03:38.560 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T09:03:38.560 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff 2023-10-18T09:03:38.560 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T09:03:38.560 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:03:38.561 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1493:a0ff:fed2:2c1/64 scope link 2023-10-18T09:03:38.561 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:03:38.561 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:03:38.561 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 4a:6e:a9:f0:92:62 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T09:03:38.561 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::486e:a9ff:fef0:9262/64 scope link 2023-10-18T09:03:38.562 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:03:38.562 INFO:teuthology.orchestra.run.smithi144.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:03:38.562 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T09:03:38.562 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::345a:c5ff:fe54:de6e/64 scope link 2023-10-18T09:03:38.562 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:03:38.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:38.564 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T09:03:38.660 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-10-18T09:03:38.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:38.665 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-10-18T09:03:38.717 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-10-18T09:03:38.717 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T09:03:38.718 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T09:03:38.718 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T09:03:38.718 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T09:03:38.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:38.720 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T09:03:38.720 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T09:03:38.720 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T09:03:38.721 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T09:03:38.721 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T09:03:38.721 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T09:03:38.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:38.904 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-18T09:03:38.955 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T09:03:38.955 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-18T09:03:38.956 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:03:38.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:38.957 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2023-10-18T09:03:39.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:03:39.043 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-18T09:03:39.095 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:03:39.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:03:39.096 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-18T09:03:39.146 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-18T09:03:39.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:39.147 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:03:39.284 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:03:39.284 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:03:39.315 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:03:39.316 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:03:39.316 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T09:03:39.316 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with new device syntax: 1@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T09:03:39.316 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T09:03:39.317 INFO:teuthology.orchestra.run.smithi144.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-18T09:03:39.317 INFO:teuthology.orchestra.run.smithi144.stdout: You just mounted a file system that supports labels which does not 2023-10-18T09:03:39.318 INFO:teuthology.orchestra.run.smithi144.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T09:03:39.318 INFO:teuthology.orchestra.run.smithi144.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T09:03:39.318 INFO:teuthology.orchestra.run.smithi144.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T09:03:39.321 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T09:03:39.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:39.322 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-18T09:03:39.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:39.399 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:03:39.455 INFO:teuthology.orchestra.run.smithi144.stdout:client5906 2023-10-18T09:03:39.456 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:03:39.456 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:03:39.512 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:03:39.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.753+0000 7f4796b5e640 1 -- 172.21.15.40:0/1542121993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f477c003970 msgr2=0x7f477c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:39.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.753+0000 7f4796b5e640 1 --2- 172.21.15.40:0/1542121993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f477c003970 0x7f477c003d90 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f478c009b50 tx=0x7f477c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:39.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.753+0000 7f479635d640 1 -- 172.21.15.40:0/1542121993 reap_dead start 2023-10-18T09:03:39.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.753+0000 7f4796b5e640 1 -- 172.21.15.40:0/1542121993 shutdown_connections 2023-10-18T09:03:39.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.753+0000 7f4796b5e640 1 -- 172.21.15.40:0/1542121993 >> 172.21.15.40:0/1542121993 conn(0x7f4790069720 msgr2=0x7f4790069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:39.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.753+0000 7f4796b5e640 1 -- 172.21.15.40:0/1542121993 shutdown_connections 2023-10-18T09:03:39.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.753+0000 7f4796b5e640 1 -- 172.21.15.40:0/1542121993 wait complete. 2023-10-18T09:03:39.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.755+0000 7f4796b5e640 1 Processor -- start 2023-10-18T09:03:39.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.755+0000 7f4796b5e640 1 -- start start 2023-10-18T09:03:39.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.756+0000 7f4796b5e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f477c003970 0x7f47901f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:39.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.756+0000 7f4796b5e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f479014d410 0x7f4790137b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:39.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.756+0000 7f4796b5e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f479015b8d0 0x7f479015bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:39.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.756+0000 7f4796b5e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479014e470 con 0x7f479014d410 2023-10-18T09:03:39.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.756+0000 7f4796b5e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479015c290 con 0x7f479015b8d0 2023-10-18T09:03:39.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.756+0000 7f4796b5e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479014f120 con 0x7f477c003970 2023-10-18T09:03:39.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.756+0000 7f479535b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f479014d410 0x7f4790137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:39.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.756+0000 7f479535b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f479014d410 0x7f4790137b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59870/0 (socket says 172.21.15.40:59870) 2023-10-18T09:03:39.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.756+0000 7f479535b640 1 -- 172.21.15.40:0/642789913 learned_addr learned my addr 172.21.15.40:0/642789913 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:39.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.757+0000 7f4795b5c640 1 --2- 172.21.15.40:0/642789913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f477c003970 0x7f47901f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:39.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.757+0000 7f479535b640 1 -- 172.21.15.40:0/642789913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f477c003970 msgr2=0x7f47901f5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:39.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.757+0000 7f479535b640 1 --2- 172.21.15.40:0/642789913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f477c003970 0x7f47901f5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:39.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.757+0000 7f479535b640 1 -- 172.21.15.40:0/642789913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f479015b8d0 msgr2=0x7f479015bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:39.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.757+0000 7f479535b640 1 --2- 172.21.15.40:0/642789913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f479015b8d0 0x7f479015bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:39.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.757+0000 7f479535b640 1 -- 172.21.15.40:0/642789913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47901f62c0 con 0x7f479014d410 2023-10-18T09:03:39.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.757+0000 7f479535b640 1 --2- 172.21.15.40:0/642789913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f479014d410 0x7f4790137b00 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f4780011a80 tx=0x7f4780011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:39.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.758+0000 7f47867fc640 1 -- 172.21.15.40:0/642789913 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478000fcf0 con 0x7f479014d410 2023-10-18T09:03:39.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.758+0000 7f47867fc640 1 -- 172.21.15.40:0/642789913 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4780012450 con 0x7f479014d410 2023-10-18T09:03:39.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.758+0000 7f47867fc640 1 -- 172.21.15.40:0/642789913 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780012800 con 0x7f479014d410 2023-10-18T09:03:39.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.758+0000 7f4796b5e640 1 -- 172.21.15.40:0/642789913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479014edf0 con 0x7f479014d410 2023-10-18T09:03:39.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.758+0000 7f4796b5e640 1 -- 172.21.15.40:0/642789913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47900712d0 con 0x7f479014d410 2023-10-18T09:03:39.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.759+0000 7f4796b5e640 1 -- 172.21.15.40:0/642789913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479014ec50 con 0x7f479014d410 2023-10-18T09:03:39.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.760+0000 7f47867fc640 1 -- 172.21.15.40:0/642789913 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4780012960 con 0x7f479014d410 2023-10-18T09:03:39.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.761+0000 7f47867fc640 1 --2- 172.21.15.40:0/642789913 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f477406ae70 0x7f477406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:39.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.761+0000 7f47867fc640 1 -- 172.21.15.40:0/642789913 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f478008a330 con 0x7f479014d410 2023-10-18T09:03:39.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.762+0000 7f4795b5c640 1 --2- 172.21.15.40:0/642789913 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f477406ae70 0x7f477406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:39.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.763+0000 7f4795b5c640 1 --2- 172.21.15.40:0/642789913 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f477406ae70 0x7f477406d330 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f478c036000 tx=0x7f478c035040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:39.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.766+0000 7f47867fc640 1 -- 172.21.15.40:0/642789913 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f479014ec50 con 0x7f479014d410 2023-10-18T09:03:39.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.995+0000 7f4796b5e640 1 -- 172.21.15.40:0/642789913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f479014ec50 con 0x7f479014d410 2023-10-18T09:03:39.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.996+0000 7f47867fc640 1 -- 172.21.15.40:0/642789913 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f479014ec50 con 0x7f479014d410 2023-10-18T09:03:39.999 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:40.000 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6382,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/4165164542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":4165164542},{"type":"v1","addr":"172.21.15.144:6836","nonce":4165164542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3045960359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3045960359},{"type":"v1","addr":"172.21.15.144:6837","nonce":3045960359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2869040025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2869040025},{"type":"v1","addr":"172.21.15.40:6833","nonce":2869040025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:03:23.782442+0000","modified":"2023-10-18T09:03:26.897709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7893},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7893":{"gid":7893,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2600609919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2600609919},{"type":"v1","addr":"172.21.15.40:6835","nonce":2600609919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-18T09:03:40.000 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 66 2023-10-18T09:03:40.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.998+0000 7f4796b5e640 1 -- 172.21.15.40:0/642789913 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f477406ae70 msgr2=0x7f477406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:40.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.998+0000 7f4796b5e640 1 --2- 172.21.15.40:0/642789913 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f477406ae70 0x7f477406d330 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f478c036000 tx=0x7f478c035040 comp rx=0 tx=0).stop 2023-10-18T09:03:40.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.998+0000 7f4796b5e640 1 -- 172.21.15.40:0/642789913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f479014d410 msgr2=0x7f4790137b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:40.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.998+0000 7f4796b5e640 1 --2- 172.21.15.40:0/642789913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f479014d410 0x7f4790137b00 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f4780011a80 tx=0x7f4780011f40 comp rx=0 tx=0).stop 2023-10-18T09:03:40.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.999+0000 7f4796b5e640 1 -- 172.21.15.40:0/642789913 shutdown_connections 2023-10-18T09:03:40.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.999+0000 7f4796b5e640 1 --2- 172.21.15.40:0/642789913 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f477406ae70 0x7f477406d330 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:40.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.999+0000 7f4796b5e640 1 --2- 172.21.15.40:0/642789913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f479015b8d0 0x7f479015bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:40.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.999+0000 7f4796b5e640 1 --2- 172.21.15.40:0/642789913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f479014d410 0x7f4790137b00 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:40.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.999+0000 7f4796b5e640 1 --2- 172.21.15.40:0/642789913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f477c003970 0x7f47901f5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:40.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.999+0000 7f4796b5e640 1 -- 172.21.15.40:0/642789913 >> 172.21.15.40:0/642789913 conn(0x7f4790069720 msgr2=0x7f4790156260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:40.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.999+0000 7f4796b5e640 1 -- 172.21.15.40:0/642789913 shutdown_connections 2023-10-18T09:03:40.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:39.999+0000 7f4796b5e640 1 -- 172.21.15.40:0/642789913 wait complete. 2023-10-18T09:03:40.022 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-18T09:03:40.271 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":7893,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.872505718999999,"mdsmap_epoch":66,"osdmap_epoch":144,"osdmap_epoch_barrier":144,"uptime":21.794107316000002} 2023-10-18T09:03:40.272 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 7893, "mdsmap_epoch": 66, "osdmap_epoch": 144, "osdmap_epoch_barrier": 144, "rank_uptime": 15.872505719, "state": "up:active", "uptime": 21.794107316, "want_state": "up:active", "whoami": 0 } 2023-10-18T09:03:40.272 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-18T09:03:40.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.510+0000 7fde7aac3640 1 -- 172.21.15.40:0/1198851020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde60003970 msgr2=0x7fde60003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:40.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.510+0000 7fde7aac3640 1 --2- 172.21.15.40:0/1198851020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde60003970 0x7fde60003d90 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fde70001680 tx=0x7fde600057f0 comp rx=0 tx=0).stop 2023-10-18T09:03:40.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.510+0000 7fde7aac3640 1 -- 172.21.15.40:0/1198851020 shutdown_connections 2023-10-18T09:03:40.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.510+0000 7fde7aac3640 1 --2- 172.21.15.40:0/1198851020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde60003970 0x7fde60003d90 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:40.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.510+0000 7fde7aac3640 1 -- 172.21.15.40:0/1198851020 >> 172.21.15.40:0/1198851020 conn(0x7fde740aeb90 msgr2=0x7fde740aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:40.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.510+0000 7fde7aac3640 1 -- 172.21.15.40:0/1198851020 shutdown_connections 2023-10-18T09:03:40.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.510+0000 7fde7aac3640 1 -- 172.21.15.40:0/1198851020 wait complete. 2023-10-18T09:03:40.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.512+0000 7fde7aac3640 1 Processor -- start 2023-10-18T09:03:40.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.512+0000 7fde7aac3640 1 -- start start 2023-10-18T09:03:40.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.513+0000 7fde7aac3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde60003970 0x7fde741e60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:40.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.513+0000 7fde7aac3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde74156250 0x7fde7413e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:40.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.513+0000 7fde7aac3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde741566d0 0x7fde7413eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:40.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.513+0000 7fde7aac3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde74154d70 con 0x7fde741566d0 2023-10-18T09:03:40.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.513+0000 7fde7aac3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde74155050 con 0x7fde74156250 2023-10-18T09:03:40.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.513+0000 7fde7aac3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde74155330 con 0x7fde60003970 2023-10-18T09:03:40.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.514+0000 7fde7a2c2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde741566d0 0x7fde7413eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:40.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.514+0000 7fde7a2c2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde741566d0 0x7fde7413eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59892/0 (socket says 172.21.15.40:59892) 2023-10-18T09:03:40.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.514+0000 7fde7a2c2640 1 -- 172.21.15.40:0/3071967465 learned_addr learned my addr 172.21.15.40:0/3071967465 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:40.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.514+0000 7fde79ac1640 1 --2- 172.21.15.40:0/3071967465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde60003970 0x7fde741e60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:40.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.514+0000 7fde792c0640 1 --2- 172.21.15.40:0/3071967465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde74156250 0x7fde7413e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:40.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.514+0000 7fde7a2c2640 1 -- 172.21.15.40:0/3071967465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde60003970 msgr2=0x7fde741e60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.514+0000 7fde7a2c2640 1 --2- 172.21.15.40:0/3071967465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde60003970 0x7fde741e60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.514+0000 7fde7a2c2640 1 -- 172.21.15.40:0/3071967465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde74156250 msgr2=0x7fde7413e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.514+0000 7fde7a2c2640 1 --2- 172.21.15.40:0/3071967465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde74156250 0x7fde7413e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.514+0000 7fde7a2c2640 1 -- 172.21.15.40:0/3071967465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde74155610 con 0x7fde741566d0 2023-10-18T09:03:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.515+0000 7fde7a2c2640 1 --2- 172.21.15.40:0/3071967465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde741566d0 0x7fde7413eac0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0012e0 tx=0x7fde6c00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.515+0000 7fde6a7fc640 1 -- 172.21.15.40:0/3071967465 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c003b80 con 0x7fde741566d0 2023-10-18T09:03:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.515+0000 7fde6a7fc640 1 -- 172.21.15.40:0/3071967465 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fde6c003d00 con 0x7fde741566d0 2023-10-18T09:03:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.515+0000 7fde6a7fc640 1 -- 172.21.15.40:0/3071967465 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c005330 con 0x7fde741566d0 2023-10-18T09:03:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.515+0000 7fde7aac3640 1 -- 172.21.15.40:0/3071967465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde741558f0 con 0x7fde741566d0 2023-10-18T09:03:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.515+0000 7fde7aac3640 1 -- 172.21.15.40:0/3071967465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde741e65f0 con 0x7fde741566d0 2023-10-18T09:03:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.516+0000 7fde7aac3640 1 -- 172.21.15.40:0/3071967465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde7414de20 con 0x7fde741566d0 2023-10-18T09:03:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.517+0000 7fde6a7fc640 1 -- 172.21.15.40:0/3071967465 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fde6c005490 con 0x7fde741566d0 2023-10-18T09:03:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.518+0000 7fde6a7fc640 1 --2- 172.21.15.40:0/3071967465 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde5806ae40 0x7fde5806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.518+0000 7fde6a7fc640 1 -- 172.21.15.40:0/3071967465 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde6c088f90 con 0x7fde741566d0 2023-10-18T09:03:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.519+0000 7fde79ac1640 1 --2- 172.21.15.40:0/3071967465 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde5806ae40 0x7fde5806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.520+0000 7fde79ac1640 1 --2- 172.21.15.40:0/3071967465 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde5806ae40 0x7fde5806d300 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fde70034000 tx=0x7fde7002e040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:40.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.523+0000 7fde6a7fc640 1 -- 172.21.15.40:0/3071967465 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde7414de20 con 0x7fde741566d0 2023-10-18T09:03:40.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:40.717+0000 7fde7aac3640 1 -- 172.21.15.40:0/3071967465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fde74155470 con 0x7fde741566d0 2023-10-18T09:03:41.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:41.531+0000 7fde6a7fc640 1 -- 172.21.15.40:0/3071967465 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v145) v1 ==== 181+0+0 (secure 0 0 0) 0x7fde74155470 con 0x7fde741566d0 2023-10-18T09:03:41.536 INFO:teuthology.orchestra.run.smithi040.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-18T09:03:41.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:41.536+0000 7fde7aac3640 1 -- 172.21.15.40:0/3071967465 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde5806ae40 msgr2=0x7fde5806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:41.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:41.536+0000 7fde7aac3640 1 --2- 172.21.15.40:0/3071967465 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde5806ae40 0x7fde5806d300 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fde70034000 tx=0x7fde7002e040 comp rx=0 tx=0).stop 2023-10-18T09:03:41.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:41.536+0000 7fde7aac3640 1 -- 172.21.15.40:0/3071967465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde741566d0 msgr2=0x7fde7413eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:41.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:41.536+0000 7fde7aac3640 1 --2- 172.21.15.40:0/3071967465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde741566d0 0x7fde7413eac0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0012e0 tx=0x7fde6c00dcd0 comp rx=0 tx=0).stop 2023-10-18T09:03:41.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:41.536+0000 7fde7aac3640 1 -- 172.21.15.40:0/3071967465 shutdown_connections 2023-10-18T09:03:41.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:41.536+0000 7fde7aac3640 1 --2- 172.21.15.40:0/3071967465 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde5806ae40 0x7fde5806d300 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:41.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:41.536+0000 7fde7aac3640 1 --2- 172.21.15.40:0/3071967465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde741566d0 0x7fde7413eac0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:41.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:41.536+0000 7fde7aac3640 1 --2- 172.21.15.40:0/3071967465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde74156250 0x7fde7413e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:41.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:41.536+0000 7fde7aac3640 1 --2- 172.21.15.40:0/3071967465 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde60003970 0x7fde741e60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:41.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:41.536+0000 7fde7aac3640 1 -- 172.21.15.40:0/3071967465 >> 172.21.15.40:0/3071967465 conn(0x7fde740aeb90 msgr2=0x7fde74070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:41.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:41.537+0000 7fde7aac3640 1 -- 172.21.15.40:0/3071967465 shutdown_connections 2023-10-18T09:03:41.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:41.537+0000 7fde7aac3640 1 -- 172.21.15.40:0/3071967465 wait complete. 2023-10-18T09:03:41.565 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-18T09:03:41.807 INFO:teuthology.orchestra.run.smithi040.stdout:{"osd_mon_report_interval":"5"} 2023-10-18T09:03:41.808 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-18T09:03:41.808 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2023-10-18T09:03:41.809 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:03:41.809 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:03:41.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:41.868 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2023-10-18T09:03:42.104 INFO:teuthology.orchestra.run.smithi040.stderr:16+0 records in 2023-10-18T09:03:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:16+0 records out 2023-10-18T09:03:42.105 INFO:teuthology.orchestra.run.smithi040.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.17441 s, 96.2 MB/s 2023-10-18T09:03:42.107 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:03:42.107 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:03:42.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:42.158 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2023-10-18T09:03:42.456 INFO:teuthology.orchestra.run.smithi040.stderr:17+0 records in 2023-10-18T09:03:42.457 INFO:teuthology.orchestra.run.smithi040.stderr:17+0 records out 2023-10-18T09:03:42.457 INFO:teuthology.orchestra.run.smithi040.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.195351 s, 91.3 MB/s 2023-10-18T09:03:42.458 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-18T09:03:42.459 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:03:42.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.675+0000 7fde5a2ea640 1 -- 172.21.15.40:0/3400426961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5406bae0 msgr2=0x7fde5415b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.675+0000 7fde5a2ea640 1 --2- 172.21.15.40:0/3400426961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5406bae0 0x7fde5415b950 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fde4c005fd0 tx=0x7fde4c02c170 comp rx=0 tx=0).stop 2023-10-18T09:03:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.675+0000 7fde5a2ea640 1 -- 172.21.15.40:0/3400426961 shutdown_connections 2023-10-18T09:03:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.675+0000 7fde5a2ea640 1 --2- 172.21.15.40:0/3400426961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5406bae0 0x7fde5415b950 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.675+0000 7fde5a2ea640 1 -- 172.21.15.40:0/3400426961 >> 172.21.15.40:0/3400426961 conn(0x7fde540b3330 msgr2=0x7fde54069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.675+0000 7fde5a2ea640 1 -- 172.21.15.40:0/3400426961 shutdown_connections 2023-10-18T09:03:42.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.675+0000 7fde5a2ea640 1 -- 172.21.15.40:0/3400426961 wait complete. 2023-10-18T09:03:42.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.677+0000 7fde5a2ea640 1 Processor -- start 2023-10-18T09:03:42.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.677+0000 7fde5a2ea640 1 -- start start 2023-10-18T09:03:42.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.678+0000 7fde5a2ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5406bae0 0x7fde54154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:42.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.678+0000 7fde5a2ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde5406d380 0x7fde5414c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:42.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.678+0000 7fde5a2ea640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde541559a0 0x7fde540ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:42.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.678+0000 7fde5a2ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde54153a10 con 0x7fde5406bae0 2023-10-18T09:03:42.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.678+0000 7fde5a2ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde54155030 con 0x7fde5406d380 2023-10-18T09:03:42.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.678+0000 7fde5a2ea640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5414cac0 con 0x7fde541559a0 2023-10-18T09:03:42.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.678+0000 7fde592e8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5406bae0 0x7fde54154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.678+0000 7fde592e8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5406bae0 0x7fde54154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59906/0 (socket says 172.21.15.40:59906) 2023-10-18T09:03:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.678+0000 7fde592e8640 1 -- 172.21.15.40:0/2957938675 learned_addr learned my addr 172.21.15.40:0/2957938675 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.678+0000 7fde592e8640 1 -- 172.21.15.40:0/2957938675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde541559a0 msgr2=0x7fde540ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.678+0000 7fde592e8640 1 --2- 172.21.15.40:0/2957938675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde541559a0 0x7fde540ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.678+0000 7fde592e8640 1 -- 172.21.15.40:0/2957938675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde5406d380 msgr2=0x7fde5414c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:42.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.678+0000 7fde592e8640 1 --2- 172.21.15.40:0/2957938675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde5406d380 0x7fde5414c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.678+0000 7fde592e8640 1 -- 172.21.15.40:0/2957938675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c009840 con 0x7fde5406bae0 2023-10-18T09:03:42.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.679+0000 7fde592e8640 1 --2- 172.21.15.40:0/2957938675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5406bae0 0x7fde54154af0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fde5000c480 tx=0x7fde5000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:42.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.679+0000 7fde45ffb640 1 -- 172.21.15.40:0/2957938675 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde500052d0 con 0x7fde5406bae0 2023-10-18T09:03:42.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.679+0000 7fde45ffb640 1 -- 172.21.15.40:0/2957938675 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fde50005430 con 0x7fde5406bae0 2023-10-18T09:03:42.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.679+0000 7fde45ffb640 1 -- 172.21.15.40:0/2957938675 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50012890 con 0x7fde5406bae0 2023-10-18T09:03:42.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.679+0000 7fde5a2ea640 1 -- 172.21.15.40:0/2957938675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde540ac1d0 con 0x7fde5406bae0 2023-10-18T09:03:42.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.679+0000 7fde5a2ea640 1 -- 172.21.15.40:0/2957938675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde540ac790 con 0x7fde5406bae0 2023-10-18T09:03:42.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.680+0000 7fde5a2ea640 1 -- 172.21.15.40:0/2957938675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde54073570 con 0x7fde5406bae0 2023-10-18T09:03:42.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.681+0000 7fde45ffb640 1 -- 172.21.15.40:0/2957938675 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fde500035f0 con 0x7fde5406bae0 2023-10-18T09:03:42.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.683+0000 7fde45ffb640 1 --2- 172.21.15.40:0/2957938675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde3806ae40 0x7fde3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:42.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.683+0000 7fde45ffb640 1 -- 172.21.15.40:0/2957938675 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde500893a0 con 0x7fde5406bae0 2023-10-18T09:03:42.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.683+0000 7fde58ae7640 1 --2- 172.21.15.40:0/2957938675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde3806ae40 0x7fde3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:42.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.684+0000 7fde58ae7640 1 --2- 172.21.15.40:0/2957938675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde3806ae40 0x7fde3806d300 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fde48007fd0 tx=0x7fde48007920 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:42.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.687+0000 7fde45ffb640 1 -- 172.21.15.40:0/2957938675 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde50057f80 con 0x7fde5406bae0 2023-10-18T09:03:42.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.885+0000 7fde5a2ea640 1 -- 172.21.15.40:0/2957938675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fde54066a00 con 0x7fde5406bae0 2023-10-18T09:03:42.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.888+0000 7fde45ffb640 1 -- 172.21.15.40:0/2957938675 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v145) v1 ==== 81+0+33982 (secure 0 0 0) 0x7fde50057b10 con 0x7fde5406bae0 2023-10-18T09:03:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:42.892 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:03:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 145, 2023-10-18T09:03:42.892 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:03:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:03:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:03:41.527027+0000", 2023-10-18T09:03:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:03:42.893 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:03:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:03:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:03:42.894 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:03:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:03:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:03:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:03:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:03:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:42.895 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:03:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:03:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:03:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:03:42.896 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:03:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 17, 2023-10-18T09:03:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:03:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:03:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:03:42.897 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:03:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:03:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:03:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:03:42.898 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:03:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:03:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:03:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:03:42.899 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:03:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:03:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:03:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:03:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:03:42.900 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:03:42.901 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:03:42.901 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:03:42.901 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:03:42.901 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:03:42.901 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:03:42.901 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:03:42.902 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:03:42.902 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:03:42.902 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:03:42.902 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:03:42.902 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:03:42.903 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:03:42.903 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:03:42.903 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:03:42.903 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:03:42.904 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:03:42.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.904 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:03:42.904 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:03:42.904 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:03:42.904 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:03:42.905 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:03:42.905 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:03:42.905 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:03:42.905 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:03:42.906 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:03:42.906 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:03:42.906 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:03:42.906 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:03:42.906 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:03:42.907 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:03:42.907 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:03:42.907 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:03:42.907 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:03:42.908 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:03:42.908 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:03:42.908 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:03:42.908 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:03:42.908 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:03:42.909 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:03:42.909 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:03:42.909 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:03:42.909 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:03:42.910 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:03:42.910 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.910 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:03:42.910 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:03:42.911 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:03:42.911 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:03:42.911 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:03:42.911 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:03:42.911 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:03:42.912 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:03:42.912 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:03:42.912 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:03:42.912 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:03:42.912 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:03:42.913 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:03:42.913 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:03:42.913 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.913 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:03:42.914 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:03:42.914 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.914 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:03:42.914 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:03:42.914 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:03:42.915 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:03:42.915 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:03:42.915 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:03:42.915 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:03:42.916 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:03:42.916 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:03:42.916 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:42.916 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.916 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.917 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 16, 2023-10-18T09:03:42.917 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:03:42.917 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:03:21.754758+0000", 2023-10-18T09:03:42.917 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:03:42.917 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:03:42.918 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:03:42.918 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:03:42.918 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:03:42.918 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:03:42.919 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:03:42.919 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:03:42.919 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:03:42.919 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:03:42.920 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:03:42.920 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:03:42.920 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:03:42.920 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:03:42.920 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:03:42.921 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:03:42.921 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:03:42.921 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:03:42.921 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:03:42.921 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:03:42.921 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:03:42.922 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:03:42.922 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:03:42.922 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:03:42.922 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.922 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "143", 2023-10-18T09:03:42.923 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:03:42.923 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:03:42.923 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:03:42.923 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:03:42.923 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:03:42.923 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:03:42.924 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:03:42.924 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:03:42.924 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:03:42.924 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:03:42.924 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:03:42.925 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:03:42.925 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:03:42.925 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:03:42.925 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:03:42.926 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:03:42.926 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:03:42.926 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:03:42.927 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:03:42.927 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:03:42.927 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:03:42.927 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:03:42.927 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:03:42.928 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:03:42.928 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:03:42.928 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:03:42.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.929 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:03:42.929 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:03:42.929 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:03:42.929 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:03:42.929 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:03:42.930 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:03:42.930 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:03:42.930 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:03:42.931 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:03:42.931 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:03:42.931 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:03:42.931 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:03:42.931 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:03:42.932 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:03:42.932 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:03:42.932 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.932 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:03:42.932 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:03:42.932 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:03:42.933 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:42.933 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.933 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:03:42.933 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T09:03:42.933 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T09:03:42.934 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:03:42.934 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T09:03:42.934 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T09:03:42.934 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:03:42.934 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:03:42.935 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:03:42.935 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:42.935 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.935 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.936 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 17, 2023-10-18T09:03:42.936 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:03:42.936 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:03:22.819486+0000", 2023-10-18T09:03:42.937 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:03:42.937 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:03:42.937 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:03:42.938 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:03:42.938 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:03:42.938 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:03:42.939 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:03:42.939 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:03:42.939 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:03:42.939 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:03:42.940 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:03:42.940 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:03:42.940 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:03:42.941 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:03:42.941 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:03:42.941 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:03:42.942 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:03:42.942 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:03:42.942 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:03:42.942 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:03:42.945 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:03:42.945 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:03:42.946 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:03:42.946 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:03:42.946 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.947 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "145", 2023-10-18T09:03:42.947 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:03:42.947 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:03:42.948 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:03:42.948 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:03:42.948 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:03:42.949 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:03:42.949 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:03:42.949 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:03:42.950 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:03:42.950 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 33554432, 2023-10-18T09:03:42.950 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:03:42.950 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:03:42.951 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:03:42.951 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:03:42.951 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:03:42.951 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:03:42.952 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:03:42.952 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:03:42.952 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:03:42.952 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:03:42.952 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:03:42.953 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:03:42.953 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:03:42.953 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:03:42.953 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:03:42.953 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:03:42.954 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.954 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:03:42.954 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:03:42.954 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:03:42.955 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:03:42.955 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:03:42.955 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:03:42.955 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:03:42.955 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:03:42.956 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:03:42.956 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:03:42.956 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:03:42.956 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:03:42.957 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:03:42.957 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:03:42.957 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.957 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:03:42.957 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:03:42.957 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:03:42.958 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:42.958 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.958 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:03:42.958 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T09:03:42.959 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T09:03:42.959 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:03:42.959 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T09:03:42.959 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T09:03:42.959 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:03:42.960 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:03:42.960 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:03:42.960 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:42.961 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:42.961 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:42.961 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:03:42.961 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.961 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:03:42.962 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:42.962 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:42.962 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:42.962 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:42.962 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:42.963 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:42.963 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:42.963 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:42.963 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:42.963 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:42.964 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:42.964 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:42.964 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:42.964 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.964 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:42.965 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:03:42.965 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:42.965 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.965 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.965 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:42.965 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:03:42.966 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:42.966 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:42.966 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:42.966 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.966 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:42.967 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:42.967 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.967 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:42.967 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:03:42.967 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:42.967 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.968 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.968 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:42.968 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:03:42.968 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:42.968 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:42.969 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:42.969 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.969 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:42.969 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:42.969 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.969 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:42.970 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:03:42.970 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:42.970 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.971 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:42.971 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:03:42.972 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:42.972 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:42.972 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:42.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.973 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:42.973 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:42.973 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.973 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:42.974 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:03:42.974 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:42.974 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.974 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.975 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:42.975 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:03:42.975 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:42.975 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:42.976 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:42.976 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.976 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:03:42.976 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:03:42.976 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:03:42.977 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:03:42.977 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:42.977 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:42.977 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:42.977 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:42.977 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.978 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.978 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:03:42.978 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:42.978 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:42.978 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:42.979 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:42.979 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:42.979 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:42.979 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:42.979 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:42.980 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:42.980 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:42.980 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:42.980 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:42.981 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:42.981 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.981 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:42.981 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:03:42.982 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:42.982 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.982 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.982 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:42.983 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:03:42.983 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:42.983 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:42.983 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:42.983 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.984 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:42.984 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:42.984 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.984 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:42.984 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:03:42.984 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:42.985 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.985 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.985 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:42.985 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:03:42.986 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:42.986 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:42.986 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:42.986 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.986 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:42.986 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:42.987 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.987 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:42.987 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:03:42.987 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:42.987 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.988 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.988 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:42.988 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:03:42.988 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:42.988 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:42.988 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:42.989 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.989 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:42.989 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:42.989 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.989 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:42.989 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:03:42.990 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:42.990 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.990 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.990 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:42.990 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:03:42.991 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:42.991 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:42.991 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:42.991 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.991 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:03:42.992 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:03:42.992 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:03:42.992 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:03:42.993 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:42.993 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:42.993 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:42.993 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:42.994 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.994 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.994 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:03:42.994 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:42.994 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:42.995 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:42.995 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:42.995 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:42.995 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:42.995 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:42.996 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:42.996 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:42.996 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:42.996 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:42.996 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:42.996 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:42.997 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.997 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:42.997 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:03:42.997 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:42.997 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.997 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.998 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:42.998 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:03:42.998 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:42.998 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:42.998 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:42.999 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:42.999 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:42.999 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:42.999 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:42.999 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:42.999 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:03:43.000 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:43.000 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.000 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.000 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.001 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:03:43.001 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:43.001 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.001 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.001 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.002 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:43.002 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.002 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.002 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.002 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:03:43.003 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:43.003 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.003 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.003 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.004 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:03:43.004 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:43.004 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.004 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.004 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.005 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:43.005 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.005 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.005 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.005 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:03:43.006 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:43.006 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.006 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.006 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.006 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:03:43.007 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:43.007 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.007 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.007 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.007 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:03:43.007 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:03:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:03:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:03:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:43.008 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:03:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:43.009 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:43.010 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:43.011 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:03:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.012 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:03:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.013 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:43.014 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:03:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:43.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:03:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.016 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.017 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:03:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.018 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:03:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.020 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:03:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.021 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:03:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.022 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.023 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:03:43.023 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:03:43.023 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:03:43.023 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:03:43.023 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:43.024 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:43.024 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:43.024 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.024 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.025 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:03:43.025 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:43.025 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:43.025 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:43.025 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:43.026 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:43.026 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:43.026 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:43.026 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:43.026 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.027 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:03:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.028 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.029 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.029 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:03:43.029 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:43.029 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.029 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.029 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.030 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:43.030 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.030 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.030 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.030 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:03:43.031 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:43.031 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.031 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.031 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.031 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:03:43.032 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:43.032 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.032 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.032 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.032 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:43.033 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.033 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.033 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.033 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:03:43.033 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:43.034 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.034 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.034 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.034 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:03:43.034 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:43.035 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.035 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.035 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.035 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:43.035 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.036 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.036 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.036 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:03:43.036 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:43.036 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.037 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.037 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.037 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:03:43.037 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:43.037 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.037 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.038 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.038 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:03:43.038 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:03:43.038 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:03:43.038 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:03:43.039 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:43.039 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:43.039 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:43.039 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.039 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.040 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.040 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:03:43.040 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:43.040 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:43.040 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:43.041 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:43.041 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:43.041 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:43.041 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:43.041 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:43.042 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:43.042 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:43.042 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:43.042 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:43.042 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.043 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.043 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.043 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:03:43.043 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:43.044 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.044 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.044 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.044 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:03:43.044 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.045 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:03:43.046 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.047 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:03:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:43.048 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:03:43.049 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.052 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:03:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.053 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.054 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:03:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:43.055 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:03:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:43.056 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:03:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:03:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:03:43.057 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:03:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.058 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:03:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:43.059 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:43.060 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.061 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:03:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.062 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:03:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.063 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.064 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:03:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:03:43.065 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.066 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:03:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:43.067 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:03:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.068 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.069 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:03:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.070 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:03:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.071 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:03:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:03:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:03:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:03:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:43.072 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.073 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:03:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:43.074 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:43.075 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.076 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:03:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.077 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:03:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.078 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:03:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:43.079 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:03:43.080 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:43.081 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:03:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:43.082 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:03:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:43.083 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.084 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:03:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.085 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:03:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:03:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:03:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:03:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:03:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.088 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:03:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:03:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:43.089 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:43.090 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:43.090 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:43.090 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:03:43.090 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:43.091 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.091 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.091 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:03:43.091 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:43.091 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:43.092 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:43.092 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:43.092 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:43.092 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:03:43.092 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:43.092 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.093 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.093 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:03:43.093 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:43.093 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:43.093 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:43.094 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:43.094 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:43.094 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:03:43.094 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:43.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.095 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.095 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:03:43.095 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:43.095 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:43.095 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:43.096 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:43.096 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:43.096 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:03:43.096 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:43.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.097 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.097 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:03:43.097 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:43.097 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:43.097 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:43.098 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:43.098 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:43.098 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:03:43.098 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:43.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.099 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.099 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:03:43.099 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:43.099 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:43.099 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:43.100 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:43.100 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:43.100 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:03:43.100 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:43.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.101 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:03:43.101 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:43.101 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:43.101 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:43.102 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:43.102 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:43.102 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:03:43.103 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:43.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.104 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:43.104 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:03:43.104 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:43.104 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:43.104 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:43.105 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:43.105 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:43.105 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:03:43.106 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:43.106 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.106 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:43.106 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:03:43.107 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:03:43.107 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:03:43.107 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:03:43.107 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:03:43.107 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:03:43.107 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:03:43.108 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:03:43.108 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:03:43.108 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:03:43.108 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:03:43.108 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:03:43.109 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:03:43.109 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:03:43.109 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:43.109 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:03:43.109 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:03:43.109 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:03:43.110 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:03:43.110 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:03:43.110 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:03:43.110 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:03:43.110 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:03:43.111 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:03:43.111 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:03:43.111 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:03:43.111 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:43.111 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:03:43.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.891+0000 7fde5a2ea640 1 -- 172.21.15.40:0/2957938675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde3806ae40 msgr2=0x7fde3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.891+0000 7fde5a2ea640 1 --2- 172.21.15.40:0/2957938675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde3806ae40 0x7fde3806d300 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fde48007fd0 tx=0x7fde48007920 comp rx=0 tx=0).stop 2023-10-18T09:03:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.891+0000 7fde5a2ea640 1 -- 172.21.15.40:0/2957938675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5406bae0 msgr2=0x7fde54154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:43.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.891+0000 7fde5a2ea640 1 --2- 172.21.15.40:0/2957938675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5406bae0 0x7fde54154af0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fde5000c480 tx=0x7fde5000c940 comp rx=0 tx=0).stop 2023-10-18T09:03:43.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.891+0000 7fde5a2ea640 1 -- 172.21.15.40:0/2957938675 shutdown_connections 2023-10-18T09:03:43.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.891+0000 7fde5a2ea640 1 --2- 172.21.15.40:0/2957938675 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde3806ae40 0x7fde3806d300 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:43.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.891+0000 7fde5a2ea640 1 --2- 172.21.15.40:0/2957938675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde541559a0 0x7fde540ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:43.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.891+0000 7fde5a2ea640 1 --2- 172.21.15.40:0/2957938675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde5406d380 0x7fde5414c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:43.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.891+0000 7fde5a2ea640 1 --2- 172.21.15.40:0/2957938675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde5406bae0 0x7fde54154af0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:43.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.891+0000 7fde5a2ea640 1 -- 172.21.15.40:0/2957938675 >> 172.21.15.40:0/2957938675 conn(0x7fde540b3330 msgr2=0x7fde54069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:43.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.891+0000 7fde5a2ea640 1 -- 172.21.15.40:0/2957938675 shutdown_connections 2023-10-18T09:03:43.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:42.891+0000 7fde5a2ea640 1 -- 172.21.15.40:0/2957938675 wait complete. 2023-10-18T09:03:43.118 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-18T09:03:48.119 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:03:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.357+0000 7fac8b7fe640 1 -- 172.21.15.40:0/1407985752 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c034ae0 con 0x7fac8c0b15b0 2023-10-18T09:03:48.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.358+0000 7fac91be2640 1 -- 172.21.15.40:0/1407985752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac8c0b15b0 msgr2=0x7fac8c0b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:48.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.358+0000 7fac91be2640 1 --2- 172.21.15.40:0/1407985752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac8c0b15b0 0x7fac8c0b0630 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fac8c06aeb0 tx=0x7fac7c02c230 comp rx=0 tx=0).stop 2023-10-18T09:03:48.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.358+0000 7fac91be2640 1 -- 172.21.15.40:0/1407985752 shutdown_connections 2023-10-18T09:03:48.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.358+0000 7fac91be2640 1 --2- 172.21.15.40:0/1407985752 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac8c0b1e10 0x7fac8c0c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:48.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.358+0000 7fac91be2640 1 --2- 172.21.15.40:0/1407985752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac8c0b15b0 0x7fac8c0b0630 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:48.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.358+0000 7fac91be2640 1 -- 172.21.15.40:0/1407985752 >> 172.21.15.40:0/1407985752 conn(0x7fac8c069720 msgr2=0x7fac8c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:48.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.358+0000 7fac91be2640 1 -- 172.21.15.40:0/1407985752 shutdown_connections 2023-10-18T09:03:48.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.358+0000 7fac91be2640 1 -- 172.21.15.40:0/1407985752 wait complete. 2023-10-18T09:03:48.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.360+0000 7fac91be2640 1 Processor -- start 2023-10-18T09:03:48.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.360+0000 7fac91be2640 1 -- start start 2023-10-18T09:03:48.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.361+0000 7fac91be2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac8c0b15b0 0x7fac8c0cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:48.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.361+0000 7fac91be2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac8c0b1e10 0x7fac8c13c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:48.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.361+0000 7fac91be2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac8c0b2270 0x7fac8c13c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:48.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.361+0000 7fac91be2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c13ce60 con 0x7fac8c0b1e10 2023-10-18T09:03:48.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.361+0000 7fac91be2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c13d1d0 con 0x7fac8c0b2270 2023-10-18T09:03:48.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.361+0000 7fac91be2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c14fe90 con 0x7fac8c0b15b0 2023-10-18T09:03:48.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.361+0000 7fac8bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac8c0b1e10 0x7fac8c13c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:48.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.362+0000 7fac8bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac8c0b1e10 0x7fac8c13c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48092/0 (socket says 172.21.15.40:48092) 2023-10-18T09:03:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.362+0000 7fac8bfff640 1 -- 172.21.15.40:0/2689388728 learned_addr learned my addr 172.21.15.40:0/2689388728 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.362+0000 7fac90be0640 1 --2- 172.21.15.40:0/2689388728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac8c0b15b0 0x7fac8c0cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.362+0000 7fac913e1640 1 --2- 172.21.15.40:0/2689388728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac8c0b2270 0x7fac8c13c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.362+0000 7fac8bfff640 1 -- 172.21.15.40:0/2689388728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac8c0b15b0 msgr2=0x7fac8c0cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.362+0000 7fac8bfff640 1 --2- 172.21.15.40:0/2689388728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac8c0b15b0 0x7fac8c0cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.362+0000 7fac8bfff640 1 -- 172.21.15.40:0/2689388728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac8c0b2270 msgr2=0x7fac8c13c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.362+0000 7fac8bfff640 1 --2- 172.21.15.40:0/2689388728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac8c0b2270 0x7fac8c13c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.362+0000 7fac8bfff640 1 -- 172.21.15.40:0/2689388728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c009900 con 0x7fac8c0b1e10 2023-10-18T09:03:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.363+0000 7fac8bfff640 1 --2- 172.21.15.40:0/2689388728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac8c0b1e10 0x7fac8c13c3e0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fac7400b4d0 tx=0x7fac7400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.363+0000 7fac897fa640 1 -- 172.21.15.40:0/2689388728 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac74010870 con 0x7fac8c0b1e10 2023-10-18T09:03:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.363+0000 7fac897fa640 1 -- 172.21.15.40:0/2689388728 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fac74010ac0 con 0x7fac8c0b1e10 2023-10-18T09:03:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.363+0000 7fac91be2640 1 -- 172.21.15.40:0/2689388728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac8c14efa0 con 0x7fac8c0b1e10 2023-10-18T09:03:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.363+0000 7fac897fa640 1 -- 172.21.15.40:0/2689388728 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7400f3c0 con 0x7fac8c0b1e10 2023-10-18T09:03:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.363+0000 7fac91be2640 1 -- 172.21.15.40:0/2689388728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac8c14f560 con 0x7fac8c0b1e10 2023-10-18T09:03:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.365+0000 7fac91be2640 1 -- 172.21.15.40:0/2689388728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac8c066e00 con 0x7fac8c0b1e10 2023-10-18T09:03:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.366+0000 7fac897fa640 1 -- 172.21.15.40:0/2689388728 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fac7400f520 con 0x7fac8c0b1e10 2023-10-18T09:03:48.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.366+0000 7fac897fa640 1 --2- 172.21.15.40:0/2689388728 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac6c06ae70 0x7fac6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:48.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.367+0000 7fac897fa640 1 -- 172.21.15.40:0/2689388728 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac74089830 con 0x7fac8c0b1e10 2023-10-18T09:03:48.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.367+0000 7fac90be0640 1 --2- 172.21.15.40:0/2689388728 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac6c06ae70 0x7fac6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:48.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.369+0000 7fac90be0640 1 --2- 172.21.15.40:0/2689388728 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac6c06ae70 0x7fac6c06d330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fac7c036000 tx=0x7fac7c004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.372+0000 7fac897fa640 1 -- 172.21.15.40:0/2689388728 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac740548e0 con 0x7fac8c0b1e10 2023-10-18T09:03:48.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.569+0000 7fac91be2640 1 -- 172.21.15.40:0/2689388728 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fac8c15bed0 con 0x7fac8c0b1e10 2023-10-18T09:03:48.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.571+0000 7fac897fa640 1 -- 172.21.15.40:0/2689388728 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v145) v1 ==== 81+0+33982 (secure 0 0 0) 0x7fac7401e090 con 0x7fac8c0b1e10 2023-10-18T09:03:48.576 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:48.577 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:03:48.577 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 145, 2023-10-18T09:03:48.577 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:03:48.578 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:03:48.578 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:03:41.527027+0000", 2023-10-18T09:03:48.579 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:03:48.579 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:03:48.580 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:03:48.580 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:03:48.581 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:03:48.581 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:03:48.582 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:03:48.582 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:03:48.582 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:03:48.583 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:48.583 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:03:48.583 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:03:48.584 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:03:48.584 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:03:48.584 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:03:48.585 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 17, 2023-10-18T09:03:48.585 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:03:48.585 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:03:48.586 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:03:48.586 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:03:48.586 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:03:48.587 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:03:48.587 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.587 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:03:48.588 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:03:48.588 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:03:48.588 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:03:48.589 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:03:48.589 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:03:48.589 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:03:48.590 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:03:48.590 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:03:48.590 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:03:48.591 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:03:48.591 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:03:48.592 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:03:48.592 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:03:48.592 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:03:48.592 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:03:48.593 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:03:48.593 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:03:48.593 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:03:48.594 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:03:48.594 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:03:48.595 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:03:48.595 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:03:48.595 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:03:48.596 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:03:48.596 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:03:48.596 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:03:48.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.597 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:03:48.598 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:03:48.598 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:03:48.598 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:03:48.599 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:03:48.599 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:03:48.600 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:03:48.600 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:03:48.600 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:03:48.601 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:03:48.601 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:03:48.601 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:03:48.601 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:03:48.602 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:03:48.602 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:03:48.602 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:03:48.602 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:03:48.603 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:03:48.603 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:03:48.603 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:03:48.603 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:03:48.603 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:03:48.604 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:03:48.604 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:03:48.604 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:03:48.604 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:03:48.604 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:03:48.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.605 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:03:48.605 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:03:48.605 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:03:48.605 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:03:48.605 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:03:48.606 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:03:48.606 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:03:48.606 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:03:48.606 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:03:48.606 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:03:48.607 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:03:48.607 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:03:48.607 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:03:48.607 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:03:48.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.608 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:03:48.608 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:03:48.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.608 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:03:48.608 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:03:48.609 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:03:48.609 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:03:48.609 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:03:48.609 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:03:48.609 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:03:48.610 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:03:48.610 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:03:48.610 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.611 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 16, 2023-10-18T09:03:48.611 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:03:48.611 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:03:21.754758+0000", 2023-10-18T09:03:48.612 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:03:48.612 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:03:48.612 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:03:48.613 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:03:48.613 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:03:48.613 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:03:48.614 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:03:48.614 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:03:48.615 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:03:48.615 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:03:48.615 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:03:48.616 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:03:48.616 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:03:48.616 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:03:48.617 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:03:48.617 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:03:48.618 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:03:48.618 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:03:48.618 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:03:48.619 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:03:48.619 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:03:48.620 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:03:48.620 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:03:48.620 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:03:48.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.621 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "143", 2023-10-18T09:03:48.621 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:03:48.622 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:03:48.622 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:03:48.622 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:03:48.623 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:03:48.623 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:03:48.623 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:03:48.624 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:03:48.624 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:03:48.624 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:03:48.625 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:03:48.625 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:03:48.625 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:03:48.626 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:03:48.626 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:03:48.627 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:03:48.627 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:03:48.627 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:03:48.628 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:03:48.628 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:03:48.628 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:03:48.628 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:03:48.629 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:03:48.629 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:03:48.629 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:03:48.630 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:03:48.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.630 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:03:48.631 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:03:48.631 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:03:48.631 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:03:48.632 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:03:48.632 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:03:48.632 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:03:48.633 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:03:48.633 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:03:48.633 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:03:48.634 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:03:48.634 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:03:48.634 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:03:48.635 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:03:48.635 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:03:48.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.636 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:03:48.636 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:03:48.636 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:03:48.636 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.637 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:03:48.637 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T09:03:48.637 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T09:03:48.637 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:03:48.638 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T09:03:48.638 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T09:03:48.638 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:03:48.638 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:03:48.638 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:03:48.638 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.639 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 17, 2023-10-18T09:03:48.639 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:03:48.639 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:03:22.819486+0000", 2023-10-18T09:03:48.640 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:03:48.640 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:03:48.640 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:03:48.640 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:03:48.640 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:03:48.641 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:03:48.641 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:03:48.641 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:03:48.641 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:03:48.641 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:03:48.642 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:03:48.642 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:03:48.642 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:03:48.642 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:03:48.642 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:03:48.642 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:03:48.643 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:03:48.643 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:03:48.643 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:03:48.643 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:03:48.643 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:03:48.644 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:03:48.644 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:03:48.644 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:03:48.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.644 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "145", 2023-10-18T09:03:48.644 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:03:48.645 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:03:48.645 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:03:48.645 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:03:48.645 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:03:48.645 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:03:48.646 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:03:48.646 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:03:48.646 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:03:48.646 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 33554432, 2023-10-18T09:03:48.646 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:03:48.647 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:03:48.647 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:03:48.647 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:03:48.647 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:03:48.647 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:03:48.648 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:03:48.648 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:03:48.648 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:03:48.648 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:03:48.648 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:03:48.648 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:03:48.649 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:03:48.649 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:03:48.649 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:03:48.649 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:03:48.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.650 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:03:48.650 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:03:48.650 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:03:48.650 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:03:48.650 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:03:48.651 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:03:48.651 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:03:48.651 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:03:48.651 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:03:48.651 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:03:48.652 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:03:48.652 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:03:48.652 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:03:48.652 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:03:48.653 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.653 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:03:48.653 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:03:48.653 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:03:48.653 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.654 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.654 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:03:48.654 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T09:03:48.654 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T09:03:48.654 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:03:48.655 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T09:03:48.655 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T09:03:48.655 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:03:48.655 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:03:48.655 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:03:48.656 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.656 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.656 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:48.656 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:03:48.656 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.657 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:03:48.657 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:48.657 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:48.657 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:48.657 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:48.657 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:48.658 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:48.658 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:48.658 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:48.658 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:48.658 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:48.659 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:48.659 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:48.659 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.659 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.659 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.660 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:03:48.660 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:48.660 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.660 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.660 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.661 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:03:48.661 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:48.661 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.661 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.661 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.661 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:48.662 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.662 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.662 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.662 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:03:48.662 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:48.663 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.663 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.663 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.663 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:03:48.663 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:48.663 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.664 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.664 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.664 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:48.664 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.664 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.665 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.665 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:03:48.665 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:48.665 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.665 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.666 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.666 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:03:48.666 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:48.666 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.666 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.667 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.667 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:48.667 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.667 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.667 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.667 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:03:48.668 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:48.668 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.668 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.668 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.668 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:03:48.668 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:48.669 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.669 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.669 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:03:48.669 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:03:48.669 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:03:48.670 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:03:48.670 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:48.670 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:48.670 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:48.670 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.671 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.671 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.671 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:03:48.671 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:48.671 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:48.672 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:48.672 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:48.672 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:48.672 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:48.672 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:48.672 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:48.673 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:48.673 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:48.673 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:48.673 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:48.673 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.673 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.674 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.674 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:03:48.674 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:48.674 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.674 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.675 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.675 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:03:48.675 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:48.675 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.675 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.675 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.676 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:48.676 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.676 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.676 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.676 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:03:48.677 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:48.677 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.677 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.677 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.677 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:03:48.678 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:48.678 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.678 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.678 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.678 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:48.679 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.679 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.679 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.679 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:03:48.679 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:48.679 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.680 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.680 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.680 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:03:48.680 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:48.680 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.681 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.681 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.681 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:48.681 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.681 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.682 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.682 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:03:48.682 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:48.682 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.682 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.683 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.683 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:03:48.683 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:48.683 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.683 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.684 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.684 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:03:48.684 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:03:48.684 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:03:48.684 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:03:48.684 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.685 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.686 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:03:48.686 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:48.686 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:48.686 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:48.686 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:48.687 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:48.687 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:48.687 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:48.687 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:48.687 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:48.688 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:48.688 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:48.688 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:48.688 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.688 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.689 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.689 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:03:48.689 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:48.689 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.689 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.689 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.690 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:03:48.690 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:48.690 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.690 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.690 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.691 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:48.691 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.691 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.691 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.691 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:03:48.692 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:48.692 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.692 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.692 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.692 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:03:48.693 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:48.693 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.693 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.693 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.693 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:48.694 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.694 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.694 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.694 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:03:48.694 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:48.695 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.695 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.695 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.695 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:03:48.695 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:48.696 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.696 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.696 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.696 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:48.696 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.696 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.697 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.697 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:03:48.697 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:48.697 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.697 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.697 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.698 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:03:48.698 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:48.698 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.698 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.698 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.699 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:03:48.699 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:03:48.699 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:03:48.699 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:03:48.699 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:48.700 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:48.700 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:48.700 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.700 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.700 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.700 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:03:48.701 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:48.701 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:48.701 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:48.701 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:48.701 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:48.702 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:48.702 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:48.702 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:48.702 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:48.702 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:48.702 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:48.703 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:48.703 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.703 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.703 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:03:48.704 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:48.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.704 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.704 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.704 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:03:48.704 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:48.705 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.705 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.705 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:48.705 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.706 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.706 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:03:48.706 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:48.706 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.707 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.707 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:03:48.707 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:48.707 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.707 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.708 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.708 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:48.708 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.708 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.708 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.708 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:03:48.709 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:48.709 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.709 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.709 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.709 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:03:48.710 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:48.710 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.710 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.710 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.710 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:48.711 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.711 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.711 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.711 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:03:48.711 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:48.712 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.712 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.712 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.712 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:03:48.712 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:48.712 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.713 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.713 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.713 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:03:48.713 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:03:48.713 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:03:48.714 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:03:48.714 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:48.714 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:48.714 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:48.714 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.714 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.715 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.715 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:03:48.715 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:48.715 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:48.715 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:48.716 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:48.716 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:48.716 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:48.716 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:48.716 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:48.717 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:48.717 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:48.717 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:48.717 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:48.717 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.717 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.718 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.718 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:03:48.718 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:48.718 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.718 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.719 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.719 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:03:48.719 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:48.719 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.719 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.720 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.720 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:48.720 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.720 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.720 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.721 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:03:48.721 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:48.721 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.721 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.721 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.722 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:03:48.722 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:48.722 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.722 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.723 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:48.723 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.723 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.723 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.723 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:03:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:03:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.726 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.726 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.726 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:03:48.726 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:48.726 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.727 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.727 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.727 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:03:48.727 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:48.727 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.728 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.728 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.728 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:03:48.728 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:03:48.728 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:03:48.729 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:03:48.729 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:48.729 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:48.729 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:48.729 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.729 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.730 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.730 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:03:48.730 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:48.730 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:48.730 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:48.731 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:48.731 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:48.731 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:48.731 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:48.731 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:48.732 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:48.732 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:48.732 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:48.732 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:48.732 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.732 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.733 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.733 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:03:48.733 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:48.733 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.733 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.734 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.734 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:03:48.734 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:48.734 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.734 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.734 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.735 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:48.735 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.735 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.735 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.735 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:03:48.736 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:48.736 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.736 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.736 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.736 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:03:48.737 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:48.737 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.737 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.737 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.737 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:48.737 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.738 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.738 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.738 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:03:48.738 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:48.738 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.739 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.739 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.739 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:03:48.739 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:48.739 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.740 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.740 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.740 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:48.740 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.740 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.741 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.741 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:03:48.741 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:48.741 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.741 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.742 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.742 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:03:48.742 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:48.742 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.742 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.743 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.743 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:03:48.743 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:03:48.743 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:03:48.743 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:03:48.744 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:48.744 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:48.744 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:48.744 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.744 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.744 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.745 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:03:48.745 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:48.745 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:48.745 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:48.745 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:48.746 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:48.746 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:48.746 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:48.746 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:48.746 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:48.746 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:48.747 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:48.747 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:48.747 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.747 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.747 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.748 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:03:48.748 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:48.748 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.748 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.748 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.749 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:03:48.749 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:48.749 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.749 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.750 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:48.750 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.750 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.750 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:03:48.751 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:48.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.751 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.751 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.751 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:03:48.751 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:48.752 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.752 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.752 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:48.752 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.752 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.753 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.753 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:03:48.753 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:48.753 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.753 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.754 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.754 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:03:48.754 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:48.754 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.754 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.755 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:48.755 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.755 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.755 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.756 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:03:48.756 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:48.756 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.756 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.756 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.756 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:03:48.757 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:48.757 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.757 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.757 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.757 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:03:48.757 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:03:48.758 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:03:48.758 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:03:48.758 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:48.758 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:48.758 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:48.759 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.759 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.759 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.759 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:03:48.759 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:48.760 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:48.760 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:48.760 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:48.760 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:48.760 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:48.761 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:48.761 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:48.761 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:48.761 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:48.761 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:48.761 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:48.762 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.762 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.762 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.762 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:03:48.762 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:48.763 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.763 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.763 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.763 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:03:48.763 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:48.763 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.764 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.764 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.764 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:48.764 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.764 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.765 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.765 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:03:48.765 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:48.765 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.765 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.765 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.766 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:03:48.766 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:48.766 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.766 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.766 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.767 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:48.767 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.767 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.767 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.767 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:03:48.767 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:48.768 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.768 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.768 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.768 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:03:48.768 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:48.768 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.769 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.769 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.769 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:48.769 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:48.769 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.769 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:48.770 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:03:48.770 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:48.770 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.770 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.770 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:48.771 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:03:48.771 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:48.771 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.771 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.771 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.772 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:03:48.772 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:03:48.772 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:03:48.772 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:03:48.772 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:48.772 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:48.773 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:48.773 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:48.773 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.773 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:48.773 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:03:48.774 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.774 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:03:48.774 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:48.774 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:48.774 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:48.775 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:48.775 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:48.775 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:03:48.775 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:48.775 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.776 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.776 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:03:48.776 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:48.776 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:48.776 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:48.776 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:48.777 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:48.777 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:03:48.777 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:48.777 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.777 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.778 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:03:48.778 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:48.778 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:48.778 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:48.778 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:48.779 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:48.779 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:03:48.779 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:48.779 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.779 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.780 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:03:48.780 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:48.780 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:48.780 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:48.780 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:48.781 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:48.781 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:03:48.781 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:48.781 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.781 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.782 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:03:48.782 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:48.782 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:48.782 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:48.782 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:48.783 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:48.783 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:03:48.783 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:48.783 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.783 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.783 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:03:48.784 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:48.784 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:48.784 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:48.784 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:48.784 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:48.785 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:03:48.785 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:48.785 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.785 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.785 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:03:48.785 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:48.786 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:48.786 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:48.786 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:48.786 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:48.786 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:03:48.786 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:48.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:48.787 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:03:48.787 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:48.787 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:48.788 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:48.788 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:48.788 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:48.788 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:03:48.788 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:48.789 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.789 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:48.789 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:03:48.789 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:03:48.789 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:03:48.789 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:03:48.790 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:03:48.790 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:03:48.790 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:03:48.790 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:03:48.791 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:03:48.791 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:03:48.791 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:03:48.791 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:03:48.791 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:03:48.791 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:03:48.792 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:48.792 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:03:48.792 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:03:48.792 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:03:48.793 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:03:48.793 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:03:48.793 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:03:48.793 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:03:48.793 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:03:48.793 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:03:48.794 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:03:48.794 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:03:48.794 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:48.794 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:03:48.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.574+0000 7fac91be2640 1 -- 172.21.15.40:0/2689388728 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac6c06ae70 msgr2=0x7fac6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:48.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.574+0000 7fac91be2640 1 --2- 172.21.15.40:0/2689388728 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac6c06ae70 0x7fac6c06d330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fac7c036000 tx=0x7fac7c004ab0 comp rx=0 tx=0).stop 2023-10-18T09:03:48.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.574+0000 7fac91be2640 1 -- 172.21.15.40:0/2689388728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac8c0b1e10 msgr2=0x7fac8c13c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:48.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.574+0000 7fac91be2640 1 --2- 172.21.15.40:0/2689388728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac8c0b1e10 0x7fac8c13c3e0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fac7400b4d0 tx=0x7fac7400b990 comp rx=0 tx=0).stop 2023-10-18T09:03:48.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.574+0000 7fac91be2640 1 -- 172.21.15.40:0/2689388728 shutdown_connections 2023-10-18T09:03:48.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.574+0000 7fac91be2640 1 --2- 172.21.15.40:0/2689388728 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fac6c06ae70 0x7fac6c06d330 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:48.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.574+0000 7fac91be2640 1 --2- 172.21.15.40:0/2689388728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac8c0b2270 0x7fac8c13c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:48.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.574+0000 7fac91be2640 1 --2- 172.21.15.40:0/2689388728 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac8c0b1e10 0x7fac8c13c3e0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:48.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.574+0000 7fac91be2640 1 --2- 172.21.15.40:0/2689388728 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac8c0b15b0 0x7fac8c0cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:48.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.574+0000 7fac91be2640 1 -- 172.21.15.40:0/2689388728 >> 172.21.15.40:0/2689388728 conn(0x7fac8c069720 msgr2=0x7fac8c0abce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:48.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.574+0000 7fac91be2640 1 -- 172.21.15.40:0/2689388728 shutdown_connections 2023-10-18T09:03:48.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:48.574+0000 7fac91be2640 1 -- 172.21.15.40:0/2689388728 wait complete. 2023-10-18T09:03:48.799 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-18T09:03:53.800 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:03:54.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.045+0000 7fd1ebfff640 1 -- 172.21.15.40:0/3618713564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec152560 msgr2=0x7fd1ec070f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:54.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.045+0000 7fd1ebfff640 1 --2- 172.21.15.40:0/3618713564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec152560 0x7fd1ec070f00 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009860 tx=0x7fd1dc02bdd0 comp rx=0 tx=0).stop 2023-10-18T09:03:54.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.045+0000 7fd1ebfff640 1 -- 172.21.15.40:0/3618713564 shutdown_connections 2023-10-18T09:03:54.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.045+0000 7fd1ebfff640 1 --2- 172.21.15.40:0/3618713564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1ec0ab2a0 0x7fd1ec0cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:54.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.045+0000 7fd1ebfff640 1 --2- 172.21.15.40:0/3618713564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec152560 0x7fd1ec070f00 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:54.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.045+0000 7fd1ebfff640 1 -- 172.21.15.40:0/3618713564 >> 172.21.15.40:0/3618713564 conn(0x7fd1ec069720 msgr2=0x7fd1ec069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:54.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.045+0000 7fd1ebfff640 1 -- 172.21.15.40:0/3618713564 shutdown_connections 2023-10-18T09:03:54.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.045+0000 7fd1ebfff640 1 -- 172.21.15.40:0/3618713564 wait complete. 2023-10-18T09:03:54.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.047+0000 7fd1ebfff640 1 Processor -- start 2023-10-18T09:03:54.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.047+0000 7fd1ebfff640 1 -- start start 2023-10-18T09:03:54.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.048+0000 7fd1ebfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1ec071440 0x7fd1ec1518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:54.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.048+0000 7fd1ebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0ab2a0 0x7fd1ec149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:54.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.048+0000 7fd1ebfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1ec152560 0x7fd1ec14fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:54.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.048+0000 7fd1ebfff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec151e00 con 0x7fd1ec0ab2a0 2023-10-18T09:03:54.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.048+0000 7fd1ebfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec150820 con 0x7fd1ec071440 2023-10-18T09:03:54.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.048+0000 7fd1ebfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec150b00 con 0x7fd1ec152560 2023-10-18T09:03:54.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.049+0000 7fd1ea7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0ab2a0 0x7fd1ec149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:54.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.049+0000 7fd1ea7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0ab2a0 0x7fd1ec149ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48114/0 (socket says 172.21.15.40:48114) 2023-10-18T09:03:54.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.049+0000 7fd1ea7fc640 1 -- 172.21.15.40:0/2752261419 learned_addr learned my addr 172.21.15.40:0/2752261419 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:54.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.049+0000 7fd1eaffd640 1 --2- 172.21.15.40:0/2752261419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1ec071440 0x7fd1ec1518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:54.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.049+0000 7fd1eb7fe640 1 --2- 172.21.15.40:0/2752261419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1ec152560 0x7fd1ec14fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:54.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.049+0000 7fd1ea7fc640 1 -- 172.21.15.40:0/2752261419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1ec152560 msgr2=0x7fd1ec14fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:54.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.049+0000 7fd1ea7fc640 1 --2- 172.21.15.40:0/2752261419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1ec152560 0x7fd1ec14fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:54.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.049+0000 7fd1ea7fc640 1 -- 172.21.15.40:0/2752261419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1ec071440 msgr2=0x7fd1ec1518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:54.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.049+0000 7fd1ea7fc640 1 --2- 172.21.15.40:0/2752261419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1ec071440 0x7fd1ec1518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:54.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.049+0000 7fd1ea7fc640 1 -- 172.21.15.40:0/2752261419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc009490 con 0x7fd1ec0ab2a0 2023-10-18T09:03:54.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.049+0000 7fd1eb7fe640 1 --2- 172.21.15.40:0/2752261419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1ec152560 0x7fd1ec14fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:03:54.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.050+0000 7fd1ea7fc640 1 --2- 172.21.15.40:0/2752261419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0ab2a0 0x7fd1ec149ea0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000c700 tx=0x7fd1e000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:54.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.050+0000 7fd1d3fff640 1 -- 172.21.15.40:0/2752261419 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e000a840 con 0x7fd1ec0ab2a0 2023-10-18T09:03:54.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.050+0000 7fd1d3fff640 1 -- 172.21.15.40:0/2752261419 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd1e000aa90 con 0x7fd1ec0ab2a0 2023-10-18T09:03:54.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.050+0000 7fd1d3fff640 1 -- 172.21.15.40:0/2752261419 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0012420 con 0x7fd1ec0ab2a0 2023-10-18T09:03:54.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.050+0000 7fd1ebfff640 1 -- 172.21.15.40:0/2752261419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec150de0 con 0x7fd1ec0ab2a0 2023-10-18T09:03:54.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.050+0000 7fd1ebfff640 1 -- 172.21.15.40:0/2752261419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec1513a0 con 0x7fd1ec0ab2a0 2023-10-18T09:03:54.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.051+0000 7fd1ebfff640 1 -- 172.21.15.40:0/2752261419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec066e00 con 0x7fd1ec0ab2a0 2023-10-18T09:03:54.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.052+0000 7fd1d3fff640 1 -- 172.21.15.40:0/2752261419 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd1e001b400 con 0x7fd1ec0ab2a0 2023-10-18T09:03:54.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.053+0000 7fd1d3fff640 1 --2- 172.21.15.40:0/2752261419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd1d806ae40 0x7fd1d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:54.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.053+0000 7fd1d3fff640 1 -- 172.21.15.40:0/2752261419 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1e0089920 con 0x7fd1ec0ab2a0 2023-10-18T09:03:54.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.054+0000 7fd1d3fff640 1 -- 172.21.15.40:0/2752261419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=147}) v3 -- 0x7fd1d8072970 con 0x7fd1ec0ab2a0 2023-10-18T09:03:54.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.054+0000 7fd1eaffd640 1 --2- 172.21.15.40:0/2752261419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd1d806ae40 0x7fd1d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:54.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.055+0000 7fd1eaffd640 1 --2- 172.21.15.40:0/2752261419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd1d806ae40 0x7fd1d806d300 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc036000 tx=0x7fd1dc004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:54.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.061+0000 7fd1d3fff640 1 -- 172.21.15.40:0/2752261419 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1e0016060 con 0x7fd1ec0ab2a0 2023-10-18T09:03:54.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.259+0000 7fd1ebfff640 1 -- 172.21.15.40:0/2752261419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd1ec071890 con 0x7fd1ec0ab2a0 2023-10-18T09:03:54.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.261+0000 7fd1d3fff640 1 -- 172.21.15.40:0/2752261419 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v146) v1 ==== 81+0+34001 (secure 0 0 0) 0x7fd1ec071890 con 0x7fd1ec0ab2a0 2023-10-18T09:03:54.264 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:54.265 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:03:54.265 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 146, 2023-10-18T09:03:54.265 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:03:54.265 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:03:54.265 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:03:51.075532+0000", 2023-10-18T09:03:54.266 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:03:54.266 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:03:54.266 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:03:54.266 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:03:54.267 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:03:54.267 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:03:54.267 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:03:54.267 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:03:54.267 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:03:54.267 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:54.268 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:03:54.268 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:03:54.268 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:03:54.268 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:03:54.268 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:03:54.269 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 17, 2023-10-18T09:03:54.269 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:03:54.269 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:03:54.269 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:03:54.269 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:03:54.269 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:03:54.270 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:03:54.270 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.270 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:03:54.270 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:03:54.270 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:03:54.271 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:03:54.271 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:03:54.271 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:03:54.271 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:03:54.271 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:03:54.272 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:03:54.272 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:03:54.272 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:03:54.272 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:03:54.272 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:03:54.272 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:03:54.273 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:03:54.273 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:03:54.273 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:03:54.273 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:03:54.273 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:03:54.274 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:03:54.274 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:03:54.274 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:03:54.274 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:03:54.274 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:03:54.275 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:03:54.275 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:03:54.275 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:03:54.275 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.275 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:03:54.275 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:03:54.276 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:03:54.276 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:03:54.276 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:03:54.276 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:03:54.276 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:03:54.276 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:03:54.277 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:03:54.277 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:03:54.277 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:03:54.277 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:03:54.278 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:03:54.278 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:03:54.278 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:03:54.279 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:03:54.279 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:03:54.280 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:03:54.280 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:03:54.280 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:03:54.281 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:03:54.281 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:03:54.281 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:03:54.282 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:03:54.282 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:03:54.282 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:03:54.283 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:03:54.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.283 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:03:54.284 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:03:54.284 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:03:54.285 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:03:54.285 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:03:54.285 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:03:54.286 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:03:54.286 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:03:54.286 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:03:54.287 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:03:54.287 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:03:54.287 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:03:54.287 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:03:54.287 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:03:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:03:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:03:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:03:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:03:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:03:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:03:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:03:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:03:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:03:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:03:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:03:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 16, 2023-10-18T09:03:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:03:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:03:21.754758+0000", 2023-10-18T09:03:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:03:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:03:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:03:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:03:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:03:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:03:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:03:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:03:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:03:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:03:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:03:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:03:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:03:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:03:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:03:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:03:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:03:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:03:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:03:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:03:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:03:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:03:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:03:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:03:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "143", 2023-10-18T09:03:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:03:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:03:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:03:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:03:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:03:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:03:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:03:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:03:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:03:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:03:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:03:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:03:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:03:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:03:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:03:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:03:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:03:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:03:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:03:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:03:54.302 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:03:54.302 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:03:54.302 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:03:54.302 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:03:54.302 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:03:54.303 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:03:54.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.303 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:03:54.303 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:03:54.303 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:03:54.304 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:03:54.304 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:03:54.304 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:03:54.304 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:03:54.304 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:03:54.305 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:03:54.305 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:03:54.305 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:03:54.305 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:03:54.305 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:03:54.306 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:03:54.306 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:03:54.306 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.306 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:03:54.306 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:03:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:03:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:03:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T09:03:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T09:03:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:03:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T09:03:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T09:03:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:03:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:03:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:03:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 17, 2023-10-18T09:03:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:03:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:03:22.819486+0000", 2023-10-18T09:03:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1027, 2023-10-18T09:03:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-18T09:03:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:03:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:03:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:03:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:03:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:03:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:03:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:03:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:03:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:03:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:03:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:03:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:03:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:03:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:03:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:03:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:03:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:03:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:03:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:03:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:03:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:03:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:03:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "146", 2023-10-18T09:03:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:03:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:03:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:03:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:03:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:03:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:03:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:03:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:03:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:03:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 33554432, 2023-10-18T09:03:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:03:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:03:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:03:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:03:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:03:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:03:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:03:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:03:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:03:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:03:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:03:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:03:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:03:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:03:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:03:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:03:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:03:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:03:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:03:54.321 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:03:54.321 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:03:54.321 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:03:54.321 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:03:54.321 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:03:54.322 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:03:54.322 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:03:54.322 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:03:54.322 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:03:54.322 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:03:54.323 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:03:54.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.323 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:03:54.323 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:03:54.323 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:03:54.324 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.324 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.324 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:03:54.324 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T09:03:54.324 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T09:03:54.325 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:03:54.325 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T09:03:54.325 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T09:03:54.325 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:03:54.325 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:03:54.326 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:03:54.326 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.326 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.326 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:54.326 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:03:54.327 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.327 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:03:54.327 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:54.327 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:54.327 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:54.328 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:54.328 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:54.328 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:54.328 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:54.328 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:54.328 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:54.329 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:54.329 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:54.329 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:54.329 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.329 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.330 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.330 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:03:54.330 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:54.330 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.331 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.331 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.331 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:03:54.331 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:54.332 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.332 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.332 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:54.332 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.332 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.333 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.333 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:03:54.333 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:54.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.334 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.334 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:03:54.334 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:54.334 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.335 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.335 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:54.335 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.336 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.336 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:03:54.336 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:54.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.337 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.337 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:03:54.337 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:54.337 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.337 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.338 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:54.338 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.339 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.339 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:03:54.339 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:54.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.340 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.340 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:03:54.340 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:54.340 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.340 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.341 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:03:54.341 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:03:54.341 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:03:54.341 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:03:54.342 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:54.342 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:54.342 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:54.342 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.343 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.343 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:03:54.343 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:54.343 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:54.344 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:54.344 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:54.344 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:54.344 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:54.345 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:54.345 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:54.345 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:54.345 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:54.346 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:54.346 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:54.346 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.347 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.347 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:03:54.347 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:54.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.348 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.348 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:03:54.348 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:54.348 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.349 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.349 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:54.349 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.350 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.350 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:03:54.350 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:54.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.351 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.351 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:03:54.351 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:54.351 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.351 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.352 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:54.352 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.352 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:03:54.353 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:54.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.354 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:03:54.354 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:54.354 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.354 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.355 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:54.355 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.356 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.356 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:03:54.356 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:54.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.356 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.357 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:03:54.357 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:54.357 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.357 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.358 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:03:54.358 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:03:54.358 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:03:54.358 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:03:54.359 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:54.359 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:54.359 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:54.360 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.360 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.360 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:03:54.361 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:54.361 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:54.361 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:54.361 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:54.362 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:54.362 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:54.362 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:54.363 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:54.363 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:54.363 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:54.363 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:54.363 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:54.364 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.364 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.364 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.364 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:03:54.364 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:54.364 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.365 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.365 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.365 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:03:54.366 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:54.366 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.366 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.366 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.366 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:54.367 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.367 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.367 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.367 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:03:54.367 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:54.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.368 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.368 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.368 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:03:54.368 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:54.369 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.369 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.369 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.369 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:54.369 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.369 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.370 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.370 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:03:54.370 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:54.370 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.371 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.371 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.371 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:03:54.371 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:54.371 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.371 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.372 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.372 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:54.372 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.372 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.372 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.373 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:03:54.373 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:54.373 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.373 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.373 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.374 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:03:54.374 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:54.374 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.374 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.374 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.374 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:03:54.375 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:03:54.375 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:03:54.375 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:03:54.375 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:54.375 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:54.376 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:54.376 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.376 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.376 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.377 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:03:54.377 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:54.377 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:54.377 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:54.377 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:54.378 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:54.378 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:54.378 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:54.378 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:54.378 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:54.379 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:54.379 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:54.379 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:54.379 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.379 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.379 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.380 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:03:54.380 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:54.380 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.380 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.380 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.381 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:03:54.381 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:54.381 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.381 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.381 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.381 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:54.382 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.382 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.382 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.382 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:03:54.382 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:54.383 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.383 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.383 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.383 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:03:54.383 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:54.384 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.384 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.384 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.384 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:54.384 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.385 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.385 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.385 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:03:54.385 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:54.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.385 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.386 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.386 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:03:54.386 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:54.386 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.387 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.387 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.387 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:54.387 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.387 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.388 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.388 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:03:54.388 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:54.388 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.389 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.389 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:03:54.389 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:54.389 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.390 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.390 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:03:54.390 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:03:54.390 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:03:54.390 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:03:54.391 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:54.391 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:54.391 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:54.391 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.392 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.392 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:03:54.392 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:54.392 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:54.392 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:54.393 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:54.393 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:54.393 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:54.393 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:54.393 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:54.393 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:54.394 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:54.394 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:54.394 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:54.394 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.394 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.395 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.395 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:03:54.395 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:54.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.396 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.396 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:03:54.396 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:54.396 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.396 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.397 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:54.397 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.397 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.397 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.398 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:03:54.398 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:54.398 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.398 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.399 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.399 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:03:54.399 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:54.399 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.400 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.400 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:54.400 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.400 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.400 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.401 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:03:54.401 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:54.401 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.401 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.401 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.402 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:03:54.402 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:54.402 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.402 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.403 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:54.403 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.403 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.403 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.403 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:03:54.404 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:54.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.404 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.404 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.404 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:03:54.405 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:54.405 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.405 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.405 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:03:54.406 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:03:54.406 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:03:54.406 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:03:54.406 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:54.406 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:54.407 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:54.407 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.407 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.408 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:03:54.408 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:54.408 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:54.408 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:54.408 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:54.409 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:54.409 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:54.409 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:54.409 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:54.409 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:54.410 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:54.410 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:54.410 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:54.410 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.410 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.411 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.411 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:03:54.411 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:54.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.411 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.411 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.412 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:03:54.412 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:54.412 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.412 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.413 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:54.413 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.413 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.413 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.413 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:03:54.414 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:54.414 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.414 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.414 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.414 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:03:54.415 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:54.415 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.415 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.415 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:54.415 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.416 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.416 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.416 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:03:54.416 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:54.416 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.417 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.417 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.417 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:03:54.417 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:54.418 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.418 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.418 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.418 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:54.419 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.419 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.419 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.419 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:03:54.419 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:54.420 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.420 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.420 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.420 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:03:54.420 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:54.420 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.421 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.421 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:03:54.421 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:03:54.421 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:03:54.422 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:03:54.422 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:54.422 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:54.422 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:54.422 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.423 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.423 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.423 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:03:54.423 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:54.423 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:54.423 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:54.424 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:54.424 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:54.424 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:54.424 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:54.425 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:54.425 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:54.425 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:54.425 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:54.425 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:54.425 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.426 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.426 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.426 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:03:54.426 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:54.426 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.427 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.427 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.427 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:03:54.427 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:54.428 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.428 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.428 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.428 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:54.429 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.429 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.429 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.429 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:03:54.430 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:54.430 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.430 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.430 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.430 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:03:54.431 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:54.431 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.431 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.432 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.432 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:54.432 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.432 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.433 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.433 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:03:54.433 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:54.433 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.433 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.434 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.434 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:03:54.434 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:54.434 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.434 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.435 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.435 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:54.435 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.435 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.435 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.436 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:03:54.436 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:54.436 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.436 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.436 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.437 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:03:54.437 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:54.437 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.437 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.438 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:03:54.438 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:03:54.438 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:03:54.438 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:03:54.438 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:54.439 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:54.439 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:54.439 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.439 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.439 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.440 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:03:54.440 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:54.440 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:54.440 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:54.440 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:54.441 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:54.441 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:54.441 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:54.441 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:54.441 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:54.441 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:54.442 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:54.442 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:54.442 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.442 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.442 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.443 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:03:54.443 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:54.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.443 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.444 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.444 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:03:54.444 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:54.444 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.444 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.445 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:54.445 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.445 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.445 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.445 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:03:54.446 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:54.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.446 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.446 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.446 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:03:54.447 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:54.447 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.447 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.447 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:54.447 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.448 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.448 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:03:54.448 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:54.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.449 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.449 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:03:54.449 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:54.449 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.449 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.450 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:54.450 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:54.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.450 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:54.450 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:03:54.450 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:54.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.451 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:54.451 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:03:54.451 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:54.452 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.452 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.452 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:03:54.452 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:03:54.452 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:03:54.453 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:03:54.453 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:54.453 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:54.453 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:54.453 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:54.454 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.454 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:54.454 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:03:54.454 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.454 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:03:54.454 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:54.455 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:54.455 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:54.455 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:54.455 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:54.455 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:03:54.456 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:54.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.456 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:03:54.456 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:54.456 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:54.457 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:54.457 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:54.457 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:54.457 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:03:54.457 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:54.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.458 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:03:54.458 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:54.458 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:54.458 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:54.459 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:54.459 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:54.459 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:03:54.459 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:54.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.460 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.460 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:03:54.460 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:54.460 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:54.460 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:54.461 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:54.461 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:54.461 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:03:54.461 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:54.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.462 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:03:54.462 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:54.462 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:54.462 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:54.462 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:54.463 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:54.463 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:03:54.463 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:54.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.464 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:03:54.464 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:54.464 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:54.464 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:54.464 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:54.465 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:54.465 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:03:54.465 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:54.465 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.465 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.466 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:03:54.466 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:54.466 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:54.466 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:54.466 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:54.466 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:54.467 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:03:54.467 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:54.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.467 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:54.467 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:03:54.467 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:54.468 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:54.468 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:54.468 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:54.468 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:54.468 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:03:54.468 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:54.469 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.469 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:54.469 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:03:54.469 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:03:54.469 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:03:54.469 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:03:54.470 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:03:54.470 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:03:54.470 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:03:54.470 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:03:54.470 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:03:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:03:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:03:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:03:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:03:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:03:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:03:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:03:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:03:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:03:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:03:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:03:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:03:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:03:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:03:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:03:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:03:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:54.474 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:03:54.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.264+0000 7fd1ebfff640 1 -- 172.21.15.40:0/2752261419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd1d806ae40 msgr2=0x7fd1d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:54.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.264+0000 7fd1ebfff640 1 --2- 172.21.15.40:0/2752261419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd1d806ae40 0x7fd1d806d300 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc036000 tx=0x7fd1dc004730 comp rx=0 tx=0).stop 2023-10-18T09:03:54.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.264+0000 7fd1ebfff640 1 -- 172.21.15.40:0/2752261419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0ab2a0 msgr2=0x7fd1ec149ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:54.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.264+0000 7fd1ebfff640 1 --2- 172.21.15.40:0/2752261419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0ab2a0 0x7fd1ec149ea0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000c700 tx=0x7fd1e000cbc0 comp rx=0 tx=0).stop 2023-10-18T09:03:54.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.264+0000 7fd1ebfff640 1 -- 172.21.15.40:0/2752261419 shutdown_connections 2023-10-18T09:03:54.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.264+0000 7fd1ebfff640 1 --2- 172.21.15.40:0/2752261419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd1d806ae40 0x7fd1d806d300 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:54.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.264+0000 7fd1ebfff640 1 --2- 172.21.15.40:0/2752261419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1ec152560 0x7fd1ec14fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:54.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.264+0000 7fd1ebfff640 1 --2- 172.21.15.40:0/2752261419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ec0ab2a0 0x7fd1ec149ea0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:54.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.264+0000 7fd1ebfff640 1 --2- 172.21.15.40:0/2752261419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1ec071440 0x7fd1ec1518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:54.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.264+0000 7fd1ebfff640 1 -- 172.21.15.40:0/2752261419 >> 172.21.15.40:0/2752261419 conn(0x7fd1ec069720 msgr2=0x7fd1ec069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:54.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.264+0000 7fd1ebfff640 1 -- 172.21.15.40:0/2752261419 shutdown_connections 2023-10-18T09:03:54.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:54.264+0000 7fd1ebfff640 1 -- 172.21.15.40:0/2752261419 wait complete. 2023-10-18T09:03:54.479 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-18T09:03:54.479 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:03:54.479 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:03:54.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:54.541 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2023-10-18T09:03:55.004 INFO:teuthology.orchestra.run.smithi040.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2023-10-18T09:03:55.004 INFO:teuthology.orchestra.run.smithi040.stderr:50+0 records in 2023-10-18T09:03:55.004 INFO:teuthology.orchestra.run.smithi040.stderr:50+0 records out 2023-10-18T09:03:55.005 INFO:teuthology.orchestra.run.smithi040.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.40554 s, 129 MB/s 2023-10-18T09:03:55.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:03:55.006 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:03:55.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.227+0000 7fd0100f2640 1 -- 172.21.15.40:0/2114053850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0080b12d0 msgr2=0x7fd0081625a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.227+0000 7fd0100f2640 1 --2- 172.21.15.40:0/2114053850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0080b12d0 0x7fd0081625a0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fd000005fd0 tx=0x7fd00002c0f0 comp rx=0 tx=0).stop 2023-10-18T09:03:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.227+0000 7fd0100f2640 1 -- 172.21.15.40:0/2114053850 shutdown_connections 2023-10-18T09:03:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.227+0000 7fd0100f2640 1 --2- 172.21.15.40:0/2114053850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0080b12d0 0x7fd0081625a0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.227+0000 7fd0100f2640 1 --2- 172.21.15.40:0/2114053850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd00806d710 0x7fd008151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.227+0000 7fd0100f2640 1 -- 172.21.15.40:0/2114053850 >> 172.21.15.40:0/2114053850 conn(0x7fd00806cee0 msgr2=0x7fd008069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.227+0000 7fd0100f2640 1 -- 172.21.15.40:0/2114053850 shutdown_connections 2023-10-18T09:03:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.227+0000 7fd0100f2640 1 -- 172.21.15.40:0/2114053850 wait complete. 2023-10-18T09:03:55.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.229+0000 7fd0100f2640 1 Processor -- start 2023-10-18T09:03:55.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.229+0000 7fd0100f2640 1 -- start start 2023-10-18T09:03:55.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.230+0000 7fd0100f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00806d710 0x7fd0081490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.230+0000 7fd0100f2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0080b12d0 0x7fd008144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.230+0000 7fd0100f2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0081586a0 0x7fd00814bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.230+0000 7fd0100f2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd008149600 con 0x7fd00806d710 2023-10-18T09:03:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.230+0000 7fd0100f2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd008149e70 con 0x7fd0081586a0 2023-10-18T09:03:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.230+0000 7fd0100f2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0081498b0 con 0x7fd0080b12d0 2023-10-18T09:03:55.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.230+0000 7fd00de8e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00806d710 0x7fd0081490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:55.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.230+0000 7fd00de8e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00806d710 0x7fd0081490c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57828/0 (socket says 172.21.15.40:57828) 2023-10-18T09:03:55.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.230+0000 7fd00de8e640 1 -- 172.21.15.40:0/2004178878 learned_addr learned my addr 172.21.15.40:0/2004178878 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:55.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.231+0000 7fd00e68f640 1 --2- 172.21.15.40:0/2004178878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0081586a0 0x7fd00814bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.231+0000 7fd00de8e640 1 -- 172.21.15.40:0/2004178878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0080b12d0 msgr2=0x7fd008144f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.231+0000 7fd00de8e640 1 --2- 172.21.15.40:0/2004178878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0080b12d0 0x7fd008144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.231+0000 7fd00de8e640 1 -- 172.21.15.40:0/2004178878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0081586a0 msgr2=0x7fd00814bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.231+0000 7fd00de8e640 1 --2- 172.21.15.40:0/2004178878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0081586a0 0x7fd00814bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.231+0000 7fd00de8e640 1 -- 172.21.15.40:0/2004178878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd000009840 con 0x7fd00806d710 2023-10-18T09:03:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.231+0000 7fd00de8e640 1 --2- 172.21.15.40:0/2004178878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00806d710 0x7fd0081490c0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fd00400c920 tx=0x7fd00400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.231+0000 7fcffe7fc640 1 -- 172.21.15.40:0/2004178878 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd004007c70 con 0x7fd00806d710 2023-10-18T09:03:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.231+0000 7fcffe7fc640 1 -- 172.21.15.40:0/2004178878 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd00400f450 con 0x7fd00806d710 2023-10-18T09:03:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.232+0000 7fd0100f2640 1 -- 172.21.15.40:0/2004178878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd008149b90 con 0x7fd00806d710 2023-10-18T09:03:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.232+0000 7fcffe7fc640 1 -- 172.21.15.40:0/2004178878 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00400f860 con 0x7fd00806d710 2023-10-18T09:03:55.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.232+0000 7fd0100f2640 1 -- 172.21.15.40:0/2004178878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00814a5b0 con 0x7fd00806d710 2023-10-18T09:03:55.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.233+0000 7fd0100f2640 1 -- 172.21.15.40:0/2004178878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00814a7a0 con 0x7fd00806d710 2023-10-18T09:03:55.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.234+0000 7fcffe7fc640 1 -- 172.21.15.40:0/2004178878 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd00400f9c0 con 0x7fd00806d710 2023-10-18T09:03:55.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.235+0000 7fcffe7fc640 1 --2- 172.21.15.40:0/2004178878 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfec06ae40 0x7fcfec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:55.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.235+0000 7fcffe7fc640 1 -- 172.21.15.40:0/2004178878 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd004089fe0 con 0x7fd00806d710 2023-10-18T09:03:55.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.235+0000 7fcffe7fc640 1 -- 172.21.15.40:0/2004178878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=147}) v3 -- 0x7fcfec072970 con 0x7fd00806d710 2023-10-18T09:03:55.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.235+0000 7fd00d68d640 1 --2- 172.21.15.40:0/2004178878 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfec06ae40 0x7fcfec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:55.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.237+0000 7fd00d68d640 1 --2- 172.21.15.40:0/2004178878 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfec06ae40 0x7fcfec06d300 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcff8000f60 tx=0x7fcff8007a30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:55.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.239+0000 7fcffe7fc640 1 -- 172.21.15.40:0/2004178878 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd00401d060 con 0x7fd00806d710 2023-10-18T09:03:55.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.438+0000 7fd0100f2640 1 -- 172.21.15.40:0/2004178878 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd00815c170 con 0x7fd00806d710 2023-10-18T09:03:55.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.441+0000 7fcffe7fc640 1 -- 172.21.15.40:0/2004178878 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v146) v1 ==== 81+0+34001 (secure 0 0 0) 0x7fd004054ff0 con 0x7fd00806d710 2023-10-18T09:03:55.444 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:55.445 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:03:55.445 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 146, 2023-10-18T09:03:55.445 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:03:55.445 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:03:55.446 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:03:51.075532+0000", 2023-10-18T09:03:55.446 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:03:55.446 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:03:55.446 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:03:55.446 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:03:55.447 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:03:55.447 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:03:55.447 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:03:55.447 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:03:55.447 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:03:55.447 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:55.448 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:03:55.448 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:03:55.448 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:03:55.448 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:03:55.448 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:03:55.448 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 17, 2023-10-18T09:03:55.449 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:03:55.449 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:03:55.449 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:03:55.449 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:03:55.449 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:03:55.450 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:03:55.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.450 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:03:55.450 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:03:55.450 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:03:55.451 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:03:55.451 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:03:55.451 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:03:55.451 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:03:55.451 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:03:55.452 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:03:55.452 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:03:55.452 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:03:55.452 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:03:55.452 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:03:55.453 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:03:55.453 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:03:55.453 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:03:55.453 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:03:55.453 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:03:55.454 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:03:55.454 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:03:55.454 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:03:55.454 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:03:55.455 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:03:55.455 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:03:55.455 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:03:55.455 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:03:55.455 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:03:55.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.456 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:03:55.456 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:03:55.456 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:03:55.456 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:03:55.457 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:03:55.457 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:03:55.457 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:03:55.457 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:03:55.458 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:03:55.458 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:03:55.458 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:03:55.458 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:03:55.459 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:03:55.459 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:03:55.459 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:03:55.459 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:03:55.459 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:03:55.460 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:03:55.460 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:03:55.460 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:03:55.460 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:03:55.460 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:03:55.461 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:03:55.461 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:03:55.461 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:03:55.461 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:03:55.462 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:03:55.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.462 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:03:55.462 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:03:55.463 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:03:55.463 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:03:55.463 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:03:55.463 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:03:55.463 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:03:55.464 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:03:55.464 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:03:55.464 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:03:55.464 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:03:55.465 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:03:55.465 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:03:55.465 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:03:55.465 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.465 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:03:55.466 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:03:55.466 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.466 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:03:55.466 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:03:55.467 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:03:55.467 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:03:55.467 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:03:55.467 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:03:55.467 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:03:55.468 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:03:55.468 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:03:55.468 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.468 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.469 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 16, 2023-10-18T09:03:55.469 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:03:55.469 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:03:21.754758+0000", 2023-10-18T09:03:55.469 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:03:55.469 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:03:55.470 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:03:55.470 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:03:55.470 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:03:55.470 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:03:55.470 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:03:55.471 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:03:55.471 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:03:55.471 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:03:55.471 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:03:55.471 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:03:55.472 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:03:55.472 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:03:55.472 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:03:55.472 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:03:55.472 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:03:55.473 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:03:55.473 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:03:55.473 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:03:55.473 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:03:55.473 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:03:55.473 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:03:55.474 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:03:55.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.474 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "143", 2023-10-18T09:03:55.474 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:03:55.474 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:03:55.475 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:03:55.475 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:03:55.475 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:03:55.475 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:03:55.475 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:03:55.476 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:03:55.476 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:03:55.476 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:03:55.476 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:03:55.476 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:03:55.476 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:03:55.477 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:03:55.477 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:03:55.477 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:03:55.477 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:03:55.477 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:03:55.478 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:03:55.478 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:03:55.478 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:03:55.478 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:03:55.478 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:03:55.479 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:03:55.479 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:03:55.479 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:03:55.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.479 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:03:55.480 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:03:55.480 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:03:55.480 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:03:55.480 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:03:55.480 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:03:55.481 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:03:55.481 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:03:55.482 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:03:55.482 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:03:55.482 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:03:55.482 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:03:55.483 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:03:55.483 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:03:55.483 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:03:55.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.483 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:03:55.484 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:03:55.484 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:03:55.484 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.484 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.484 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:03:55.485 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T09:03:55.485 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T09:03:55.485 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:03:55.485 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T09:03:55.486 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T09:03:55.486 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:03:55.486 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:03:55.486 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:03:55.486 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.487 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.487 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 17, 2023-10-18T09:03:55.487 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:03:55.488 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:03:22.819486+0000", 2023-10-18T09:03:55.488 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1027, 2023-10-18T09:03:55.488 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-18T09:03:55.488 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:03:55.488 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:03:55.489 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:03:55.489 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:03:55.489 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:03:55.489 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:03:55.490 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:03:55.490 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:03:55.490 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:03:55.490 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:03:55.490 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:03:55.491 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:03:55.491 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:03:55.491 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:03:55.491 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:03:55.491 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:03:55.492 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:03:55.492 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:03:55.492 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:03:55.492 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:03:55.492 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:03:55.493 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:03:55.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.493 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "146", 2023-10-18T09:03:55.493 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:03:55.493 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:03:55.494 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:03:55.494 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:03:55.494 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:03:55.494 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:03:55.495 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:03:55.495 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:03:55.495 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:03:55.495 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 33554432, 2023-10-18T09:03:55.495 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:03:55.496 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:03:55.496 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:03:55.496 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:03:55.496 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:03:55.497 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:03:55.497 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:03:55.497 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:03:55.497 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:03:55.497 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:03:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:03:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:03:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:03:55.498 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:03:55.499 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:03:55.499 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:03:55.499 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.499 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:03:55.499 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:03:55.500 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:03:55.500 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:03:55.500 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:03:55.500 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:03:55.500 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:03:55.501 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:03:55.501 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:03:55.501 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:03:55.501 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:03:55.501 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:03:55.502 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:03:55.502 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:03:55.502 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.502 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:03:55.503 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:03:55.503 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:03:55.503 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.503 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.504 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:03:55.504 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T09:03:55.504 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T09:03:55.504 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:03:55.504 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T09:03:55.505 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T09:03:55.505 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:03:55.505 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:03:55.505 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:03:55.505 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.506 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.506 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:55.506 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:03:55.506 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.507 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:03:55.507 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:55.507 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:55.507 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:55.508 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:55.508 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:55.508 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:55.508 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:55.508 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:55.509 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:55.509 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:55.509 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:55.509 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:55.509 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.510 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.510 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.510 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:03:55.510 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:55.511 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.511 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.511 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.511 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:03:55.511 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:55.511 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.512 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.512 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.512 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:55.512 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.513 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.513 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.513 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:03:55.513 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:55.513 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.514 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.514 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.514 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:03:55.514 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:55.514 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.515 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.515 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.515 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:55.515 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.516 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.516 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.516 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:03:55.516 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:55.516 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.517 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.517 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.517 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:03:55.517 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:55.518 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.518 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.518 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.518 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:55.519 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.519 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.519 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.519 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:03:55.519 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:55.520 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.520 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.520 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.520 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:03:55.521 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:55.521 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.521 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.521 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.521 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:03:55.522 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:03:55.522 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:03:55.522 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:03:55.522 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:55.522 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:55.523 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:55.523 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.523 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.523 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.523 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:03:55.524 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:55.524 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:55.524 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:55.524 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:55.524 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:55.525 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:55.525 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:55.525 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:55.525 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:55.525 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:55.526 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:55.526 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:55.526 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.526 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.527 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.527 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:03:55.527 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:55.527 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.527 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.528 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.528 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:03:55.528 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:55.528 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.528 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.528 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.529 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:55.529 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.530 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.530 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.530 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:03:55.531 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:55.531 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.531 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.531 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.531 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:03:55.532 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:55.532 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.532 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.532 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.533 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:55.533 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.533 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.533 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.533 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:03:55.533 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:55.534 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.534 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.534 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.534 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:03:55.535 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:55.535 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.535 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.535 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.535 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:55.536 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.536 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.536 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.536 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:03:55.536 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:55.537 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.537 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.537 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.537 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:03:55.538 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:55.538 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.538 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.538 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.538 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:03:55.539 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:03:55.539 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:03:55.539 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:03:55.539 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:55.539 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:55.540 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:55.540 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.540 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.540 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.541 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:03:55.541 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:55.541 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:55.541 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:55.542 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:55.542 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:55.542 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:55.542 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:55.542 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:55.543 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:55.543 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:55.543 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:55.543 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:55.544 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.544 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.544 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.544 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:03:55.544 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:55.545 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.545 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.545 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.545 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:03:55.545 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:55.546 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.546 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.546 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.546 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:55.546 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.547 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.547 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.547 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:03:55.547 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:55.548 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.548 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.548 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.548 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:03:55.549 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:55.549 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.549 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.549 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.549 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:55.550 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.550 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.550 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.550 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:03:55.550 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:55.551 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.551 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.551 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.551 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:03:55.552 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:55.552 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.552 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.552 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.552 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:55.553 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.553 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.553 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.553 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:03:55.554 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:55.554 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.554 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.554 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.555 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:03:55.555 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:55.555 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.555 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.556 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:03:55.556 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:03:55.556 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:03:55.556 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:03:55.557 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:55.557 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:55.557 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:55.557 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.558 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:03:55.558 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:55.558 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:55.558 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:55.559 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:55.559 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:55.559 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:55.559 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:55.559 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:55.560 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:55.560 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:55.560 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:55.560 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:55.561 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.561 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.561 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.561 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:03:55.561 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:55.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.562 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.562 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.562 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:03:55.562 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:55.563 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.563 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.563 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:55.564 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.564 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.564 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.564 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:03:55.565 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:55.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.565 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.565 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.565 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:03:55.566 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:55.566 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.566 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.566 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.566 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:55.567 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.567 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.567 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.567 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:03:55.568 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:55.568 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.568 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.568 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.568 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:03:55.569 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:55.569 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.569 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.570 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:55.570 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.570 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.570 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.570 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:03:55.571 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:55.571 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.571 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.571 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.572 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:03:55.572 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:55.572 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.572 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.573 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:03:55.573 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:03:55.573 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:03:55.573 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:03:55.574 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:55.574 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:55.574 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:55.574 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.575 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:03:55.576 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:55.576 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:55.576 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:55.576 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:55.577 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:55.577 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:55.577 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:55.577 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:55.578 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:55.578 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:55.578 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:55.578 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:55.578 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.579 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.579 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:03:55.579 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:55.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.580 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.580 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.580 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:03:55.580 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:55.580 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.581 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.581 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.581 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:55.581 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.581 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.582 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.582 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:03:55.582 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:55.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.582 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.583 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.583 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:03:55.583 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:55.583 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.584 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.584 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.584 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:55.584 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.585 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.585 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.585 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:03:55.585 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:55.585 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.586 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.586 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.586 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:03:55.586 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:55.586 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.587 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.587 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.587 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:55.587 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.588 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.588 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.588 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:03:55.588 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:55.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.589 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.589 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.589 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:03:55.589 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:55.590 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.590 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.590 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:03:55.590 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:03:55.591 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:03:55.591 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:03:55.591 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:55.591 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:55.592 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:55.592 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.592 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.592 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:03:55.593 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:55.593 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:55.593 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:55.593 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:55.593 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:55.594 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:55.594 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:55.594 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:55.594 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:55.594 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:55.595 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:55.595 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:55.595 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.595 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.596 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:03:55.596 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:55.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.597 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.597 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:03:55.597 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:55.597 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.598 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.598 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:55.598 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.599 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.599 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:03:55.599 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:55.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.600 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.600 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:03:55.600 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:55.600 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.601 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.601 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:55.601 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.602 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.602 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:03:55.602 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:55.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.603 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.603 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:03:55.603 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:55.603 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.604 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.604 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:55.604 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.605 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.605 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:03:55.605 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:55.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.606 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.606 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:03:55.606 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:55.607 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.607 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.607 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:03:55.607 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:03:55.608 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:03:55.608 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:03:55.608 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:55.608 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:55.609 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:55.609 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.609 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.609 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:03:55.610 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:55.610 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:55.610 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:55.610 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:55.610 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:55.611 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:55.611 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:55.611 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:55.612 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:55.612 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:55.612 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:55.612 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:55.612 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.613 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.613 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:03:55.613 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:55.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.614 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.614 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:03:55.614 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:55.615 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.615 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.615 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:55.615 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.616 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.616 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:03:55.616 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:55.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.617 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:03:55.617 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:55.618 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.618 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.618 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:55.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:03:55.619 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:55.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.620 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.620 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:03:55.620 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:55.621 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.621 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.621 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:55.621 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.622 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.622 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:03:55.622 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:55.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.623 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.623 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:03:55.623 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:55.624 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.624 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.624 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.624 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:03:55.625 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:03:55.625 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:03:55.625 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:03:55.625 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:55.625 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:55.626 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:55.626 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.626 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.626 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:03:55.627 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:55.627 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:55.627 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:55.627 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:55.627 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:55.628 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:55.628 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:55.628 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:55.628 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:55.629 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:55.629 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:55.629 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:55.629 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.629 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.630 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.630 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:03:55.630 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:55.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.630 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.631 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.631 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:03:55.631 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:55.631 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.632 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.632 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:55.632 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.632 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.633 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.633 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:03:55.633 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:55.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.633 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.633 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.634 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:03:55.634 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:55.634 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.634 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.635 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:55.635 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.635 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.635 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.635 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:03:55.636 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:55.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.636 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.636 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.636 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:03:55.636 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:55.637 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.637 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.637 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:55.637 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:55.638 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.638 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:55.638 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:03:55.638 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:55.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.639 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:55.639 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:03:55.639 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:55.639 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.640 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.640 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:03:55.640 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:03:55.641 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:03:55.641 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:03:55.641 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:55.641 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:55.642 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:55.642 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:55.642 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.642 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:55.643 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:03:55.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.643 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:03:55.643 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:55.643 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:55.644 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:55.644 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:55.644 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:55.644 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:03:55.644 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:55.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:03:55.645 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:55.646 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:55.646 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:55.646 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:55.646 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:55.647 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:03:55.647 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:55.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.647 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:03:55.648 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:55.648 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:55.648 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:55.648 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:55.648 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:55.649 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:03:55.649 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:55.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.649 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.649 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:03:55.650 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:55.650 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:55.650 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:55.650 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:55.650 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:55.651 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:03:55.651 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:55.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.651 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.651 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:03:55.652 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:55.652 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:55.652 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:55.652 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:55.652 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:55.653 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:03:55.653 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:55.653 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.653 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.653 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:03:55.654 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:55.654 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:55.654 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:55.654 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:55.654 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:55.655 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:03:55.655 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:55.655 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.655 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.655 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:03:55.656 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:55.656 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:55.656 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:55.656 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:55.656 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:55.657 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:03:55.657 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:55.657 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.657 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:55.657 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:03:55.658 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:55.658 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:55.658 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:55.658 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:55.658 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:55.659 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:03:55.659 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:55.659 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.659 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:55.659 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:03:55.659 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:03:55.660 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:03:55.660 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:03:55.660 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:03:55.660 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:03:55.661 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:03:55.661 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:03:55.661 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:03:55.661 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:03:55.661 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:03:55.662 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:03:55.662 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:03:55.662 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:03:55.662 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.663 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:55.663 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:03:55.663 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:03:55.663 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:03:55.663 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:03:55.664 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:03:55.664 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:03:55.664 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:03:55.664 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:03:55.664 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:03:55.665 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:03:55.665 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:03:55.665 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:55.665 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:03:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.444+0000 7fd0100f2640 1 -- 172.21.15.40:0/2004178878 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfec06ae40 msgr2=0x7fcfec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.444+0000 7fd0100f2640 1 --2- 172.21.15.40:0/2004178878 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfec06ae40 0x7fcfec06d300 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcff8000f60 tx=0x7fcff8007a30 comp rx=0 tx=0).stop 2023-10-18T09:03:55.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.444+0000 7fd0100f2640 1 -- 172.21.15.40:0/2004178878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00806d710 msgr2=0x7fd0081490c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:55.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.444+0000 7fd0100f2640 1 --2- 172.21.15.40:0/2004178878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00806d710 0x7fd0081490c0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fd00400c920 tx=0x7fd00400cde0 comp rx=0 tx=0).stop 2023-10-18T09:03:55.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.445+0000 7fd0100f2640 1 -- 172.21.15.40:0/2004178878 shutdown_connections 2023-10-18T09:03:55.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.445+0000 7fd0100f2640 1 --2- 172.21.15.40:0/2004178878 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcfec06ae40 0x7fcfec06d300 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:55.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.445+0000 7fd0100f2640 1 --2- 172.21.15.40:0/2004178878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0081586a0 0x7fd00814bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:55.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.445+0000 7fd0100f2640 1 --2- 172.21.15.40:0/2004178878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0080b12d0 0x7fd008144f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:55.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.445+0000 7fd0100f2640 1 --2- 172.21.15.40:0/2004178878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd00806d710 0x7fd0081490c0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:55.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.445+0000 7fd0100f2640 1 -- 172.21.15.40:0/2004178878 >> 172.21.15.40:0/2004178878 conn(0x7fd00806cee0 msgr2=0x7fd0081566f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:55.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.445+0000 7fd0100f2640 1 -- 172.21.15.40:0/2004178878 shutdown_connections 2023-10-18T09:03:55.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.445+0000 7fd0100f2640 1 -- 172.21.15.40:0/2004178878 wait complete. 2023-10-18T09:03:55.670 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:03:55.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.892+0000 7fd4a25ee640 1 -- 172.21.15.40:0/1761286652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd49c151bd0 msgr2=0x7fd49c1586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:55.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.892+0000 7fd4a25ee640 1 --2- 172.21.15.40:0/1761286652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd49c151bd0 0x7fd49c1586a0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00a940 tx=0x7fd48c02d130 comp rx=0 tx=0).stop 2023-10-18T09:03:55.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.892+0000 7fd4a25ee640 1 -- 172.21.15.40:0/1761286652 shutdown_connections 2023-10-18T09:03:55.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.892+0000 7fd4a25ee640 1 --2- 172.21.15.40:0/1761286652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd49c151bd0 0x7fd49c1586a0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:55.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.892+0000 7fd4a25ee640 1 -- 172.21.15.40:0/1761286652 >> 172.21.15.40:0/1761286652 conn(0x7fd49c06cee0 msgr2=0x7fd49c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:55.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.892+0000 7fd4a25ee640 1 -- 172.21.15.40:0/1761286652 shutdown_connections 2023-10-18T09:03:55.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.892+0000 7fd4a25ee640 1 -- 172.21.15.40:0/1761286652 wait complete. 2023-10-18T09:03:55.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.894+0000 7fd4a25ee640 1 Processor -- start 2023-10-18T09:03:55.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.894+0000 7fd4a25ee640 1 -- start start 2023-10-18T09:03:55.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.895+0000 7fd4a25ee640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd49c0b0bc0 0x7fd49c137970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:55.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.895+0000 7fd4a25ee640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd49c0cd9a0 0x7fd49c147370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:55.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.895+0000 7fd4a25ee640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd49c151bd0 0x7fd49c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:55.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.895+0000 7fd4a25ee640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c148bf0 con 0x7fd49c0cd9a0 2023-10-18T09:03:55.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.895+0000 7fd4a25ee640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c147dc0 con 0x7fd49c151bd0 2023-10-18T09:03:55.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.895+0000 7fd4a25ee640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c1480a0 con 0x7fd49c0b0bc0 2023-10-18T09:03:55.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.895+0000 7fd493fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd49c0cd9a0 0x7fd49c147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:55.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.895+0000 7fd493fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd49c0cd9a0 0x7fd49c147370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57858/0 (socket says 172.21.15.40:57858) 2023-10-18T09:03:55.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.895+0000 7fd493fff640 1 -- 172.21.15.40:0/598432742 learned_addr learned my addr 172.21.15.40:0/598432742 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:55.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.896+0000 7fd49bfff640 1 --2- 172.21.15.40:0/598432742 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd49c0b0bc0 0x7fd49c137970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:55.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.896+0000 7fd4a0b8b640 1 --2- 172.21.15.40:0/598432742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd49c151bd0 0x7fd49c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:55.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.896+0000 7fd493fff640 1 -- 172.21.15.40:0/598432742 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd49c0b0bc0 msgr2=0x7fd49c137970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.896+0000 7fd493fff640 1 --2- 172.21.15.40:0/598432742 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd49c0b0bc0 0x7fd49c137970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.896+0000 7fd493fff640 1 -- 172.21.15.40:0/598432742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd49c151bd0 msgr2=0x7fd49c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.896+0000 7fd493fff640 1 --2- 172.21.15.40:0/598432742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd49c151bd0 0x7fd49c159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:55.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.896+0000 7fd493fff640 1 -- 172.21.15.40:0/598432742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c00a5a0 con 0x7fd49c0cd9a0 2023-10-18T09:03:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.896+0000 7fd493fff640 1 --2- 172.21.15.40:0/598432742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd49c0cd9a0 0x7fd49c147370 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fd48400c450 tx=0x7fd48400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.896+0000 7fd4997fa640 1 -- 172.21.15.40:0/598432742 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4840052d0 con 0x7fd49c0cd9a0 2023-10-18T09:03:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.896+0000 7fd4997fa640 1 -- 172.21.15.40:0/598432742 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd484005430 con 0x7fd49c0cd9a0 2023-10-18T09:03:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.896+0000 7fd4a25ee640 1 -- 172.21.15.40:0/598432742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49c148e40 con 0x7fd49c0cd9a0 2023-10-18T09:03:55.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.896+0000 7fd4997fa640 1 -- 172.21.15.40:0/598432742 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd484012890 con 0x7fd49c0cd9a0 2023-10-18T09:03:55.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.897+0000 7fd4a25ee640 1 -- 172.21.15.40:0/598432742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49c137eb0 con 0x7fd49c0cd9a0 2023-10-18T09:03:55.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.898+0000 7fd4a25ee640 1 -- 172.21.15.40:0/598432742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49c151730 con 0x7fd49c0cd9a0 2023-10-18T09:03:55.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.899+0000 7fd4997fa640 1 -- 172.21.15.40:0/598432742 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd4840035f0 con 0x7fd49c0cd9a0 2023-10-18T09:03:55.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.900+0000 7fd4997fa640 1 --2- 172.21.15.40:0/598432742 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd47c06ae40 0x7fd47c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:55.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.901+0000 7fd4997fa640 1 -- 172.21.15.40:0/598432742 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd484089230 con 0x7fd49c0cd9a0 2023-10-18T09:03:55.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.901+0000 7fd4997fa640 1 -- 172.21.15.40:0/598432742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=147}) v3 -- 0x7fd47c072970 con 0x7fd49c0cd9a0 2023-10-18T09:03:55.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.901+0000 7fd49bfff640 1 --2- 172.21.15.40:0/598432742 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd47c06ae40 0x7fd47c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:55.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.902+0000 7fd49bfff640 1 --2- 172.21.15.40:0/598432742 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd47c06ae40 0x7fd47c06d300 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd48c037000 tx=0x7fd48c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:55.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:55.905+0000 7fd4997fa640 1 -- 172.21.15.40:0/598432742 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd484054370 con 0x7fd49c0cd9a0 2023-10-18T09:03:56.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.141+0000 7fd4a25ee640 1 -- 172.21.15.40:0/598432742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd49c066fd0 con 0x7fd49c0cd9a0 2023-10-18T09:03:56.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.142+0000 7fd4997fa640 1 -- 172.21.15.40:0/598432742 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fd484057990 con 0x7fd49c0cd9a0 2023-10-18T09:03:56.145 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:56.145 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6382,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/4165164542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":4165164542},{"type":"v1","addr":"172.21.15.144:6836","nonce":4165164542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3045960359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3045960359},{"type":"v1","addr":"172.21.15.144:6837","nonce":3045960359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2869040025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2869040025},{"type":"v1","addr":"172.21.15.40:6833","nonce":2869040025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:03:23.782442+0000","modified":"2023-10-18T09:03:26.897709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7893},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7893":{"gid":7893,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2600609919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2600609919},{"type":"v1","addr":"172.21.15.40:6835","nonce":2600609919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-18T09:03:56.146 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 66 2023-10-18T09:03:56.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.145+0000 7fd4a25ee640 1 -- 172.21.15.40:0/598432742 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd47c06ae40 msgr2=0x7fd47c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:56.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.145+0000 7fd4a25ee640 1 --2- 172.21.15.40:0/598432742 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd47c06ae40 0x7fd47c06d300 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd48c037000 tx=0x7fd48c004730 comp rx=0 tx=0).stop 2023-10-18T09:03:56.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.145+0000 7fd4a25ee640 1 -- 172.21.15.40:0/598432742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd49c0cd9a0 msgr2=0x7fd49c147370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:56.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.145+0000 7fd4a25ee640 1 --2- 172.21.15.40:0/598432742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd49c0cd9a0 0x7fd49c147370 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fd48400c450 tx=0x7fd48400c910 comp rx=0 tx=0).stop 2023-10-18T09:03:56.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.145+0000 7fd4a25ee640 1 -- 172.21.15.40:0/598432742 shutdown_connections 2023-10-18T09:03:56.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.145+0000 7fd4a25ee640 1 --2- 172.21.15.40:0/598432742 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd47c06ae40 0x7fd47c06d300 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:56.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.145+0000 7fd4a25ee640 1 --2- 172.21.15.40:0/598432742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd49c151bd0 0x7fd49c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:56.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.145+0000 7fd4a25ee640 1 --2- 172.21.15.40:0/598432742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd49c0cd9a0 0x7fd49c147370 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:56.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.145+0000 7fd4a25ee640 1 --2- 172.21.15.40:0/598432742 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd49c0b0bc0 0x7fd49c137970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:56.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.145+0000 7fd4a25ee640 1 -- 172.21.15.40:0/598432742 >> 172.21.15.40:0/598432742 conn(0x7fd49c06cee0 msgr2=0x7fd49c06b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:56.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.146+0000 7fd4a25ee640 1 -- 172.21.15.40:0/598432742 shutdown_connections 2023-10-18T09:03:56.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.146+0000 7fd4a25ee640 1 -- 172.21.15.40:0/598432742 wait complete. 2023-10-18T09:03:56.166 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-18T09:03:56.420 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":7893,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":32.021573744000001,"mdsmap_epoch":66,"osdmap_epoch":146,"osdmap_epoch_barrier":146,"uptime":37.943172769999997} 2023-10-18T09:03:56.420 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 7893, "mdsmap_epoch": 66, "osdmap_epoch": 146, "osdmap_epoch_barrier": 146, "rank_uptime": 32.021573744, "state": "up:active", "uptime": 37.94317277, "want_state": "up:active", "whoami": 0 } 2023-10-18T09:03:56.421 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-18T09:03:56.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:56.421 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-18T09:03:56.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:03:56.480 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-18T09:03:56.547 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:03:56.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.781+0000 7f92d715e640 1 -- 172.21.15.40:0/2367202225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92d00b01e0 msgr2=0x7f92d00ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:56.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.781+0000 7f92d715e640 1 --2- 172.21.15.40:0/2367202225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92d00b01e0 0x7f92d00ae960 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f92d008aba0 tx=0x7f92c802c140 comp rx=0 tx=0).stop 2023-10-18T09:03:56.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.782+0000 7f92d715e640 1 -- 172.21.15.40:0/2367202225 shutdown_connections 2023-10-18T09:03:56.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.782+0000 7f92d715e640 1 --2- 172.21.15.40:0/2367202225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92d00b01e0 0x7f92d00ae960 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:56.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.782+0000 7f92d715e640 1 --2- 172.21.15.40:0/2367202225 >> v2:172.21.15.144:3300/0 conn(0x7f92d00d0020 0x7f92d00d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:56.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.782+0000 7f92d715e640 1 -- 172.21.15.40:0/2367202225 >> 172.21.15.40:0/2367202225 conn(0x7f92d008ad00 msgr2=0x7f92d009a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:56.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.782+0000 7f92d715e640 1 -- 172.21.15.40:0/2367202225 shutdown_connections 2023-10-18T09:03:56.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.782+0000 7f92d715e640 1 -- 172.21.15.40:0/2367202225 wait complete. 2023-10-18T09:03:56.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.783+0000 7f92d715e640 1 Processor -- start 2023-10-18T09:03:56.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.784+0000 7f92d715e640 1 -- start start 2023-10-18T09:03:56.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.784+0000 7f92d715e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d00b01e0 0x7f92d0158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:56.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.785+0000 7f92d715e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d00d0020 0x7f92d00b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:56.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.785+0000 7f92d715e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92d015b640 0x7f92d00b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:56.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.785+0000 7f92d715e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d00b3ef0 con 0x7f92d015b640 2023-10-18T09:03:56.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.785+0000 7f92d715e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d0159130 con 0x7f92d00b01e0 2023-10-18T09:03:56.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.785+0000 7f92d715e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d00b4140 con 0x7f92d00d0020 2023-10-18T09:03:56.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.785+0000 7f92d695d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92d015b640 0x7f92d00b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:56.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.785+0000 7f92d695d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92d015b640 0x7f92d00b6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57878/0 (socket says 172.21.15.40:57878) 2023-10-18T09:03:56.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.785+0000 7f92d695d640 1 -- 172.21.15.40:0/752583291 learned_addr learned my addr 172.21.15.40:0/752583291 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:03:56.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.785+0000 7f92d615c640 1 --2- 172.21.15.40:0/752583291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d00b01e0 0x7f92d0158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:56.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.785+0000 7f92d595b640 1 --2- 172.21.15.40:0/752583291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d00d0020 0x7f92d00b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:56.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.785+0000 7f92d695d640 1 -- 172.21.15.40:0/752583291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d00d0020 msgr2=0x7f92d00b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:56.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.785+0000 7f92d695d640 1 --2- 172.21.15.40:0/752583291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d00d0020 0x7f92d00b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:56.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.785+0000 7f92d695d640 1 -- 172.21.15.40:0/752583291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d00b01e0 msgr2=0x7f92d0158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:56.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.785+0000 7f92d695d640 1 --2- 172.21.15.40:0/752583291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d00b01e0 0x7f92d0158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:56.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.785+0000 7f92d695d640 1 -- 172.21.15.40:0/752583291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c8009900 con 0x7f92d015b640 2023-10-18T09:03:56.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.786+0000 7f92d695d640 1 --2- 172.21.15.40:0/752583291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92d015b640 0x7f92d00b6080 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009d60 tx=0x7f92c8003bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:56.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.786+0000 7f92c2ffd640 1 -- 172.21.15.40:0/752583291 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8004240 con 0x7f92d015b640 2023-10-18T09:03:56.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.786+0000 7f92c2ffd640 1 -- 172.21.15.40:0/752583291 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f92c8004490 con 0x7f92d015b640 2023-10-18T09:03:56.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.786+0000 7f92c2ffd640 1 -- 172.21.15.40:0/752583291 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8004880 con 0x7f92d015b640 2023-10-18T09:03:56.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.786+0000 7f92d715e640 1 -- 172.21.15.40:0/752583291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d00b4420 con 0x7f92d015b640 2023-10-18T09:03:56.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.786+0000 7f92d715e640 1 -- 172.21.15.40:0/752583291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d00b49e0 con 0x7f92d015b640 2023-10-18T09:03:56.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.787+0000 7f92d715e640 1 -- 172.21.15.40:0/752583291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d0071da0 con 0x7f92d015b640 2023-10-18T09:03:56.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.788+0000 7f92c2ffd640 1 -- 172.21.15.40:0/752583291 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f92c803b040 con 0x7f92d015b640 2023-10-18T09:03:56.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.789+0000 7f92c2ffd640 1 --2- 172.21.15.40:0/752583291 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f92b406ae40 0x7f92b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:03:56.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.789+0000 7f92c2ffd640 1 -- 172.21.15.40:0/752583291 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92c80ac3e0 con 0x7f92d015b640 2023-10-18T09:03:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.790+0000 7f92c2ffd640 1 -- 172.21.15.40:0/752583291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=147}) v3 -- 0x7f92b4072970 con 0x7f92d015b640 2023-10-18T09:03:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.790+0000 7f92d615c640 1 --2- 172.21.15.40:0/752583291 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f92b406ae40 0x7f92b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:03:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.791+0000 7f92d615c640 1 --2- 172.21.15.40:0/752583291 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f92b406ae40 0x7f92b406d300 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f92cc006fd0 tx=0x7f92cc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:03:56.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.794+0000 7f92c2ffd640 1 -- 172.21.15.40:0/752583291 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92c80773f0 con 0x7f92d015b640 2023-10-18T09:03:56.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.987+0000 7f92d715e640 1 -- 172.21.15.40:0/752583291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f92d0066e00 con 0x7f92d015b640 2023-10-18T09:03:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.989+0000 7f92c2ffd640 1 -- 172.21.15.40:0/752583291 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v146) v1 ==== 81+0+34001 (secure 0 0 0) 0x7f92c8039090 con 0x7f92d015b640 2023-10-18T09:03:56.992 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:03:56.992 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:03:56.992 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 146, 2023-10-18T09:03:56.992 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:03:56.993 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:03:56.993 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:03:51.075532+0000", 2023-10-18T09:03:56.993 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:03:56.993 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:03:56.993 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:03:56.994 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:03:56.994 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:03:56.994 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:03:56.994 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:03:56.994 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:03:56.995 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:03:56.995 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:56.995 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:03:56.995 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:03:56.995 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:03:56.995 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:03:56.996 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:03:56.996 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 17, 2023-10-18T09:03:56.996 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:03:56.996 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:03:56.996 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:03:56.997 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:03:56.997 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:03:56.997 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:03:56.997 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:56.997 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:03:56.998 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:03:56.998 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:03:56.998 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:03:56.998 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:03:56.998 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:03:56.999 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:03:56.999 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:03:56.999 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:03:56.999 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:03:56.999 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:03:56.999 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:03:57.000 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:03:57.000 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:03:57.000 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:03:57.000 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:03:57.000 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:03:57.001 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:03:57.001 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:03:57.001 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:03:57.001 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:03:57.001 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:03:57.002 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:03:57.002 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:03:57.002 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:03:57.002 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:03:57.002 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:03:57.003 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.003 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:03:57.003 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:03:57.003 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:03:57.004 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:03:57.004 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:03:57.004 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:03:57.004 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:03:57.004 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:03:57.004 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:03:57.005 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:03:57.005 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:03:57.005 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:03:57.005 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:03:57.005 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:03:57.006 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:03:57.006 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:03:57.006 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:03:57.006 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:03:57.006 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:03:57.007 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:03:57.007 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:03:57.007 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:03:57.007 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:03:57.007 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:03:57.007 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:03:57.008 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:03:57.008 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:03:57.008 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.008 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:03:57.008 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:03:57.009 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:03:57.009 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:03:57.009 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:03:57.009 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:03:57.009 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:03:57.010 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:03:57.010 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:03:57.010 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:03:57.010 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:03:57.010 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:03:57.011 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:03:57.011 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:03:57.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.011 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:03:57.011 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:03:57.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.012 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:03:57.012 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:03:57.012 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:03:57.012 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:03:57.012 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:03:57.013 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:03:57.013 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:03:57.013 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:03:57.013 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:03:57.013 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.014 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.014 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 16, 2023-10-18T09:03:57.014 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:03:57.014 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:03:21.754758+0000", 2023-10-18T09:03:57.014 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:03:57.015 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:03:57.015 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:03:57.015 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:03:57.015 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:03:57.015 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:03:57.016 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:03:57.016 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:03:57.016 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:03:57.016 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:03:57.016 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:03:57.017 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:03:57.017 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:03:57.017 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:03:57.017 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:03:57.017 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:03:57.017 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:03:57.018 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:03:57.018 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:03:57.018 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:03:57.018 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:03:57.018 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:03:57.019 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:03:57.019 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:03:57.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.019 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "143", 2023-10-18T09:03:57.019 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:03:57.020 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:03:57.020 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:03:57.020 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:03:57.020 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:03:57.020 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:03:57.020 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:03:57.021 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:03:57.021 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:03:57.021 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:03:57.021 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:03:57.021 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:03:57.021 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:03:57.022 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:03:57.022 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:03:57.022 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:03:57.022 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:03:57.022 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:03:57.023 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:03:57.023 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:03:57.023 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:03:57.023 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:03:57.023 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:03:57.023 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:03:57.024 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:03:57.024 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:03:57.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.024 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:03:57.024 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:03:57.025 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:03:57.025 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:03:57.025 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:03:57.025 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:03:57.025 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:03:57.025 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:03:57.026 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:03:57.026 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:03:57.026 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:03:57.026 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:03:57.026 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:03:57.026 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:03:57.027 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:03:57.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.027 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:03:57.027 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:03:57.027 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:03:57.027 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.028 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:03:57.028 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T09:03:57.028 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T09:03:57.028 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:03:57.028 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T09:03:57.029 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T09:03:57.029 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:03:57.029 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:03:57.029 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:03:57.029 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.030 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.030 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 17, 2023-10-18T09:03:57.030 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:03:57.030 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:03:22.819486+0000", 2023-10-18T09:03:57.031 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1027, 2023-10-18T09:03:57.031 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-18T09:03:57.031 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:03:57.031 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:03:57.031 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:03:57.031 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:03:57.032 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:03:57.032 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:03:57.032 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:03:57.032 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:03:57.032 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:03:57.032 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:03:57.033 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:03:57.033 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:03:57.033 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:03:57.033 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:03:57.033 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:03:57.033 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:03:57.034 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:03:57.034 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:03:57.034 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:03:57.034 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:03:57.034 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:03:57.035 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:03:57.035 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.035 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "146", 2023-10-18T09:03:57.035 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:03:57.035 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:03:57.036 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:03:57.036 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:03:57.036 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:03:57.036 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:03:57.036 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:03:57.037 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:03:57.037 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:03:57.037 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 33554432, 2023-10-18T09:03:57.037 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:03:57.037 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:03:57.038 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:03:57.038 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:03:57.038 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:03:57.038 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:03:57.038 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:03:57.038 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:03:57.039 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:03:57.039 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:03:57.039 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:03:57.039 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:03:57.039 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:03:57.040 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:03:57.040 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:03:57.040 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:03:57.040 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.040 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:03:57.040 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:03:57.041 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:03:57.041 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:03:57.041 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:03:57.041 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:03:57.041 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:03:57.041 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:03:57.042 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:03:57.042 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:03:57.042 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:03:57.042 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:03:57.042 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:03:57.043 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:03:57.043 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.043 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:03:57.043 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:03:57.043 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:03:57.044 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.044 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.044 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:03:57.044 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T09:03:57.044 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T09:03:57.044 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:03:57.045 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T09:03:57.045 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T09:03:57.045 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:03:57.045 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:03:57.045 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:03:57.046 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.046 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.046 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:57.046 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:03:57.046 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.047 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:03:57.047 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:57.047 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:57.047 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:57.047 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:57.047 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:57.048 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:57.048 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:57.048 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:57.048 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:57.048 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:57.048 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:57.049 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:57.049 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.049 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.049 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.049 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:03:57.050 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:57.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.050 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.050 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.050 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:03:57.051 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:57.051 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.051 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.051 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.051 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:57.051 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.052 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.052 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.052 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:03:57.052 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:57.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.052 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.053 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.053 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:03:57.053 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:57.053 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.053 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.054 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.054 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:57.054 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.054 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.054 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.055 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:03:57.055 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:57.055 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.055 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.055 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.056 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:03:57.056 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:57.056 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.056 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.056 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.057 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:57.057 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.057 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.057 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.057 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:03:57.058 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:57.058 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.058 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.058 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.058 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:03:57.059 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:03:57.059 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.059 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.059 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.059 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:03:57.060 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:03:57.060 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:03:57.060 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:03:57.060 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:57.060 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:57.060 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:57.061 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.061 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.061 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.061 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:03:57.061 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:57.061 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:57.062 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:57.062 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:57.062 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:57.062 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:57.062 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:57.062 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:57.063 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:57.063 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:57.063 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:57.063 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:57.063 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.063 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.064 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.064 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:03:57.064 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:57.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.064 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.064 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.065 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:03:57.065 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:57.065 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.065 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.065 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.065 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:57.066 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.066 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.066 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.066 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:03:57.066 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:57.067 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.067 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.067 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.067 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:03:57.067 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:57.067 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.068 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.068 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:57.068 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.068 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.068 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.069 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:03:57.069 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:57.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.069 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.069 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.070 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:03:57.070 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:57.070 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.070 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.070 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.070 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:57.071 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.071 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.071 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.071 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:03:57.071 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:57.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.072 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.072 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.072 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:03:57.072 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:03:57.072 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.073 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.073 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:03:57.073 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:03:57.073 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:03:57.073 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:03:57.074 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:57.074 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:57.074 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:57.074 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.074 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.075 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.075 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:03:57.075 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:57.075 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:57.075 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:57.075 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:57.076 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:57.076 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:57.076 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:57.076 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:57.076 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:57.076 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:57.077 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:57.077 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:57.077 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.077 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.077 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.078 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:03:57.078 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:57.078 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.078 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.078 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.079 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:03:57.079 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:57.079 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.079 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.079 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.079 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:57.080 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.080 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.080 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.080 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:03:57.080 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:57.081 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.081 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.081 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.081 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:03:57.081 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:57.082 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.082 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.082 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.082 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:57.082 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.083 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.083 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.083 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:03:57.083 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:57.083 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.084 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.084 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.084 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:03:57.084 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:57.084 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.084 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.085 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.085 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:57.085 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.085 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.085 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.086 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:03:57.086 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:57.086 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.086 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.086 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.087 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:03:57.087 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:03:57.087 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.087 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.087 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.087 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:03:57.087 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:03:57.088 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:03:57.088 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:03:57.088 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:57.088 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:57.088 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:57.089 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.089 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.089 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.089 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:03:57.089 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:57.090 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:57.090 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:57.090 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:57.090 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:57.090 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:57.090 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:57.091 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:57.091 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:57.091 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:57.091 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:57.091 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:57.092 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.092 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.092 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.092 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:03:57.092 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:57.093 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.093 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.093 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.093 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:03:57.093 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:57.093 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.094 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.094 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:57.094 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.095 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.095 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:03:57.095 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:57.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.095 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.096 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.096 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:03:57.096 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:57.096 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.096 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.097 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:57.097 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.097 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.097 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.097 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:03:57.098 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:57.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.098 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.098 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.098 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:03:57.099 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:57.099 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.099 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.099 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.099 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:57.100 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.100 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.100 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.100 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:03:57.100 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:57.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.101 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.101 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:03:57.101 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:03:57.101 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.101 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.102 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:03:57.102 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:03:57.102 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:03:57.102 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:03:57.102 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:57.103 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:57.103 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:57.103 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.103 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.103 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:03:57.104 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:57.104 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:57.104 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:57.104 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:57.104 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:57.104 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:57.105 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:57.105 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:57.105 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:57.105 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:57.105 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:57.105 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:57.106 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.106 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.106 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.106 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:03:57.106 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:57.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.107 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.107 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:03:57.107 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:57.108 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.108 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.108 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:57.108 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.109 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.109 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:03:57.109 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:57.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.109 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.110 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.110 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:03:57.110 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:57.110 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.110 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.111 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.111 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:57.111 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.111 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.112 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:03:57.112 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:57.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.112 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.112 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.112 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:03:57.113 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:57.113 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.113 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.113 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:57.114 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.114 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.114 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:03:57.114 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:57.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.115 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.115 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:03:57.115 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:03:57.115 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.116 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.116 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:03:57.116 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:03:57.116 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:03:57.117 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:03:57.117 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:57.117 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:57.117 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:57.117 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.118 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:03:57.118 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:57.118 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:57.118 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:57.119 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:57.119 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:57.119 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:57.119 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:57.119 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:57.120 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:57.120 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:57.120 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:57.120 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:57.120 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.121 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.121 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:03:57.121 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:57.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.122 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.122 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:03:57.122 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:57.122 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.122 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.123 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:57.123 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.123 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.123 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:03:57.124 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:57.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.124 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.124 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:03:57.124 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:57.125 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.125 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.125 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:57.125 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.126 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.126 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:03:57.126 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:57.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.127 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.127 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:03:57.127 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:57.127 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.127 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.128 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.128 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:57.128 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.128 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.128 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:03:57.129 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:57.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.129 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.129 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:03:57.130 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:03:57.130 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.130 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.130 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:03:57.130 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:03:57.131 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:03:57.131 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:03:57.131 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:57.131 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:57.131 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:57.131 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.132 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:03:57.132 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:57.132 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:57.132 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:57.133 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:57.133 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:57.133 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:57.133 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:57.133 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:57.134 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:57.134 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:57.134 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:57.134 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:57.134 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.135 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.135 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:03:57.135 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:57.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.136 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.136 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:03:57.136 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:57.136 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.136 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.137 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:57.137 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.137 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.137 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:03:57.138 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:57.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.138 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.138 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:03:57.139 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:57.139 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.139 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.139 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:57.140 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.140 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.140 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:03:57.140 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:57.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.141 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:03:57.141 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:57.141 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.141 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.142 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:57.142 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.142 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.143 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:03:57.143 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:57.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.143 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.143 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:03:57.144 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:03:57.144 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.144 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.144 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:03:57.145 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:03:57.145 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:03:57.145 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:03:57.145 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:57.145 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:57.145 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:57.146 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.146 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.146 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:03:57.146 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:03:57.146 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:03:57.147 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:03:57.147 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:03:57.147 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:03:57.147 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:03:57.147 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:03:57.148 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:03:57.148 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:03:57.148 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:03:57.148 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:03:57.148 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:03:57.148 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.149 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.149 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:03:57.149 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:57.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.150 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.150 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:03:57.150 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:57.150 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.151 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.151 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:03:57.151 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.151 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.152 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:03:57.152 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:57.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.152 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.153 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:03:57.153 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:57.153 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.153 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.154 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:03:57.154 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.154 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.154 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:03:57.155 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:57.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.155 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.155 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:03:57.155 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:57.156 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.156 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.156 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.156 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:03:57.156 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:03:57.157 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.157 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:03:57.157 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:03:57.157 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:57.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.158 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:03:57.158 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:03:57.158 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:03:57.158 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.158 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.159 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:03:57.159 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:03:57.159 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:03:57.159 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:03:57.160 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:03:57.160 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:03:57.160 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:03:57.160 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:03:57.160 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.160 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:57.161 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:03:57.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.161 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:03:57.161 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:57.161 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:57.162 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:57.162 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:57.162 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:57.162 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:03:57.162 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:57.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.163 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:03:57.163 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:57.163 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:57.163 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:57.164 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:57.164 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:57.164 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:03:57.164 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:57.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.164 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:03:57.165 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:57.165 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:57.165 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:57.165 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:57.165 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:57.165 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:03:57.166 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:57.166 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.166 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.166 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:03:57.166 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:57.167 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:57.167 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:57.167 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:57.167 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:57.167 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:03:57.167 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:57.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.168 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.168 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:03:57.168 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:57.168 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:57.168 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:57.169 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:57.169 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:57.169 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:03:57.169 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:57.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.169 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.170 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:03:57.170 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:57.170 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:57.170 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:57.170 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:57.171 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:57.171 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:03:57.171 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:57.171 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.171 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.172 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:03:57.172 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:57.172 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:57.172 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:57.172 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:57.173 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:57.173 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:03:57.173 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:57.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:03:57.173 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:03:57.174 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:03:57.174 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:03:57.174 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:03:57.174 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:03:57.174 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:03:57.175 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:03:57.175 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:03:57.175 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.175 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:03:57.175 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:03:57.176 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:03:57.176 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:03:57.176 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:03:57.176 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:03:57.176 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:03:57.177 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:03:57.177 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:03:57.177 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:03:57.177 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:03:57.177 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:03:57.177 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:03:57.178 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:03:57.178 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:03:57.178 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.178 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:03:57.178 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:03:57.179 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:03:57.179 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:03:57.179 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:03:57.179 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:03:57.179 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:03:57.180 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:03:57.180 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:03:57.180 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:03:57.180 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:03:57.180 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:03:57.180 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:03:57.181 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:03:57.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.992+0000 7f92d715e640 1 -- 172.21.15.40:0/752583291 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f92b406ae40 msgr2=0x7f92b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:57.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.992+0000 7f92d715e640 1 --2- 172.21.15.40:0/752583291 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f92b406ae40 0x7f92b406d300 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f92cc006fd0 tx=0x7f92cc008040 comp rx=0 tx=0).stop 2023-10-18T09:03:57.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.992+0000 7f92d715e640 1 -- 172.21.15.40:0/752583291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92d015b640 msgr2=0x7f92d00b6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:03:57.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.992+0000 7f92d715e640 1 --2- 172.21.15.40:0/752583291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92d015b640 0x7f92d00b6080 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009d60 tx=0x7f92c8003bf0 comp rx=0 tx=0).stop 2023-10-18T09:03:57.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.992+0000 7f92d715e640 1 -- 172.21.15.40:0/752583291 shutdown_connections 2023-10-18T09:03:57.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.992+0000 7f92d715e640 1 --2- 172.21.15.40:0/752583291 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f92b406ae40 0x7f92b406d300 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:57.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.992+0000 7f92d715e640 1 --2- 172.21.15.40:0/752583291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92d015b640 0x7f92d00b6080 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:57.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.992+0000 7f92d715e640 1 --2- 172.21.15.40:0/752583291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d00d0020 0x7f92d00b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:57.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.992+0000 7f92d715e640 1 --2- 172.21.15.40:0/752583291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d00b01e0 0x7f92d0158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:03:57.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.992+0000 7f92d715e640 1 -- 172.21.15.40:0/752583291 >> 172.21.15.40:0/752583291 conn(0x7f92d008ad00 msgr2=0x7f92d00b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:03:57.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.992+0000 7f92d715e640 1 -- 172.21.15.40:0/752583291 shutdown_connections 2023-10-18T09:03:57.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:03:56.992+0000 7f92d715e640 1 -- 172.21.15.40:0/752583291 wait complete. 2023-10-18T09:03:57.185 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-18T09:04:02.186 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:04:02.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.423+0000 7f09cd847640 1 -- 172.21.15.40:0/965561250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09b80039a0 msgr2=0x7f09b8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:02.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.423+0000 7f09cd847640 1 --2- 172.21.15.40:0/965561250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09b80039a0 0x7f09b8003dc0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f09bc001680 tx=0x7f09b80057f0 comp rx=0 tx=0).stop 2023-10-18T09:04:02.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.423+0000 7f09cd046640 1 -- 172.21.15.40:0/965561250 reap_dead start 2023-10-18T09:04:02.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.423+0000 7f09cd847640 1 -- 172.21.15.40:0/965561250 shutdown_connections 2023-10-18T09:04:02.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.423+0000 7f09cd847640 1 -- 172.21.15.40:0/965561250 >> 172.21.15.40:0/965561250 conn(0x7f09c8069720 msgr2=0x7f09c8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:02.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.423+0000 7f09cd847640 1 -- 172.21.15.40:0/965561250 shutdown_connections 2023-10-18T09:04:02.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.423+0000 7f09cd847640 1 -- 172.21.15.40:0/965561250 wait complete. 2023-10-18T09:04:02.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.425+0000 7f09cd847640 1 Processor -- start 2023-10-18T09:04:02.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.425+0000 7f09cd847640 1 -- start start 2023-10-18T09:04:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.426+0000 7f09cd847640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09b80039a0 0x7f09c8158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.426+0000 7f09cd847640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09c81599e0 0x7f09c8159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:02.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.426+0000 7f09cd847640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09c8139d40 0x7f09c813a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.426+0000 7f09cd847640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c815a3a0 con 0x7f09b80039a0 2023-10-18T09:04:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.426+0000 7f09cd847640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c81f0a30 con 0x7f09c8139d40 2023-10-18T09:04:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.426+0000 7f09cd847640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c81f0c80 con 0x7f09c81599e0 2023-10-18T09:04:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.427+0000 7f09cc845640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09b80039a0 0x7f09c8158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:02.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.427+0000 7f09cc845640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09b80039a0 0x7f09c8158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57900/0 (socket says 172.21.15.40:57900) 2023-10-18T09:04:02.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.427+0000 7f09cc845640 1 -- 172.21.15.40:0/2885660236 learned_addr learned my addr 172.21.15.40:0/2885660236 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:02.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.427+0000 7f09c7fff640 1 --2- 172.21.15.40:0/2885660236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09c81599e0 0x7f09c8159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:02.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.427+0000 7f09cd046640 1 --2- 172.21.15.40:0/2885660236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09c8139d40 0x7f09c813a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:02.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.427+0000 7f09cc845640 1 -- 172.21.15.40:0/2885660236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09c81599e0 msgr2=0x7f09c8159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:02.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.427+0000 7f09cc845640 1 --2- 172.21.15.40:0/2885660236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09c81599e0 0x7f09c8159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:02.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.427+0000 7f09cc845640 1 -- 172.21.15.40:0/2885660236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09c8139d40 msgr2=0x7f09c813a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:02.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.427+0000 7f09cc845640 1 --2- 172.21.15.40:0/2885660236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09c8139d40 0x7f09c813a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:02.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.427+0000 7f09cc845640 1 -- 172.21.15.40:0/2885660236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c81f0f60 con 0x7f09b80039a0 2023-10-18T09:04:02.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.427+0000 7f09cc845640 1 --2- 172.21.15.40:0/2885660236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09b80039a0 0x7f09c8158cb0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0017a0 tx=0x7f09bc006360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:02.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.428+0000 7f09c57fa640 1 -- 172.21.15.40:0/2885660236 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc003b80 con 0x7f09b80039a0 2023-10-18T09:04:02.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.428+0000 7f09c57fa640 1 -- 172.21.15.40:0/2885660236 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f09bc003dd0 con 0x7f09b80039a0 2023-10-18T09:04:02.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.428+0000 7f09cd847640 1 -- 172.21.15.40:0/2885660236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c81f1240 con 0x7f09b80039a0 2023-10-18T09:04:02.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.428+0000 7f09c57fa640 1 -- 172.21.15.40:0/2885660236 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc02c7b0 con 0x7f09b80039a0 2023-10-18T09:04:02.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.428+0000 7f09cd847640 1 -- 172.21.15.40:0/2885660236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c81591f0 con 0x7f09b80039a0 2023-10-18T09:04:02.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.429+0000 7f09cd847640 1 -- 172.21.15.40:0/2885660236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c80c98c0 con 0x7f09b80039a0 2023-10-18T09:04:02.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.430+0000 7f09c57fa640 1 -- 172.21.15.40:0/2885660236 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f09bc02c910 con 0x7f09b80039a0 2023-10-18T09:04:02.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.431+0000 7f09c57fa640 1 --2- 172.21.15.40:0/2885660236 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09ac06ae70 0x7f09ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:02.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.431+0000 7f09c57fa640 1 -- 172.21.15.40:0/2885660236 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09bc0aaa20 con 0x7f09b80039a0 2023-10-18T09:04:02.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.431+0000 7f09c57fa640 1 -- 172.21.15.40:0/2885660236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=147}) v3 -- 0x7f09ac0729a0 con 0x7f09b80039a0 2023-10-18T09:04:02.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.432+0000 7f09c7fff640 1 --2- 172.21.15.40:0/2885660236 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09ac06ae70 0x7f09ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:02.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.433+0000 7f09c7fff640 1 --2- 172.21.15.40:0/2885660236 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09ac06ae70 0x7f09ac06d330 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005bc0 tx=0x7f09b4005ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:02.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.436+0000 7f09c57fa640 1 -- 172.21.15.40:0/2885660236 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f09bc075a30 con 0x7f09b80039a0 2023-10-18T09:04:02.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.635+0000 7f09cd847640 1 -- 172.21.15.40:0/2885660236 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f09c8066e00 con 0x7f09b80039a0 2023-10-18T09:04:02.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.637+0000 7f09c57fa640 1 -- 172.21.15.40:0/2885660236 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v146) v1 ==== 81+0+34001 (secure 0 0 0) 0x7f09bc079050 con 0x7f09b80039a0 2023-10-18T09:04:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:02.640 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:04:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 146, 2023-10-18T09:04:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:04:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:04:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:03:51.075532+0000", 2023-10-18T09:04:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:04:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:04:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:04:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:04:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:04:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:04:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:04:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:04:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:04:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:04:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:04:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:04:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:04:02.644 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:04:02.644 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:04:02.644 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 17, 2023-10-18T09:04:02.644 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:04:02.644 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:04:02.645 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:04:02.645 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:04:02.645 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:04:02.645 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:04:02.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.646 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:04:02.646 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:04:02.646 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:04:02.646 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:04:02.647 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:04:02.647 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:04:02.647 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:04:02.647 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:04:02.647 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:04:02.648 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:04:02.648 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:04:02.648 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:04:02.648 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:04:02.648 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:04:02.649 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:04:02.649 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:04:02.649 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:04:02.649 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:04:02.649 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:04:02.650 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:04:02.650 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:04:02.650 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:04:02.650 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:04:02.651 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:04:02.651 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:04:02.651 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:04:02.651 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:04:02.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.652 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:04:02.652 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:04:02.652 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:04:02.652 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:04:02.652 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:04:02.652 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:04:02.653 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:04:02.653 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:04:02.653 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:04:02.653 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:04:02.653 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:04:02.654 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:04:02.654 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:04:02.654 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:04:02.654 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:04:02.654 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:04:02.655 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:04:02.655 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:04:02.655 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:04:02.655 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:04:02.655 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:04:02.655 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:04:02.656 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:04:02.656 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:04:02.656 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:04:02.656 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:04:02.656 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:04:02.657 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.657 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:04:02.657 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:04:02.657 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:04:02.657 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:04:02.657 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:04:02.658 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:04:02.658 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:04:02.658 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:04:02.658 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:04:02.658 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:04:02.659 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:04:02.659 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:04:02.659 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:04:02.659 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:04:02.659 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.660 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:04:02.660 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:04:02.660 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.660 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:04:02.660 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:04:02.661 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:04:02.661 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:04:02.661 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:04:02.661 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:04:02.661 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:04:02.661 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:04:02.662 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:04:02.662 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.662 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.662 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.663 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 16, 2023-10-18T09:04:02.663 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:04:02.663 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:03:21.754758+0000", 2023-10-18T09:04:02.663 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:04:02.663 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:04:02.663 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:04:02.664 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:04:02.664 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:04:02.664 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:04:02.664 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:04:02.664 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:04:02.665 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:04:02.665 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:04:02.665 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:04:02.665 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:04:02.665 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:04:02.666 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:04:02.666 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:04:02.666 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:04:02.666 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:04:02.666 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:04:02.667 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:04:02.667 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:04:02.667 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:04:02.667 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:04:02.667 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:04:02.668 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:04:02.668 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.668 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "143", 2023-10-18T09:04:02.668 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:04:02.668 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:04:02.669 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:04:02.669 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:04:02.669 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:04:02.669 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:04:02.669 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:04:02.670 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:04:02.670 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:04:02.670 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:04:02.670 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:04:02.670 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:04:02.671 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:04:02.671 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:04:02.671 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:04:02.671 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:04:02.671 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:04:02.672 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:04:02.672 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:04:02.672 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:04:02.672 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:04:02.672 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:04:02.673 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:04:02.673 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:04:02.673 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:04:02.673 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:04:02.674 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.674 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:04:02.674 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:04:02.674 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:04:02.674 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:04:02.675 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:04:02.675 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:04:02.675 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:04:02.675 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:04:02.676 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:04:02.676 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:04:02.676 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:04:02.676 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:04:02.677 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:04:02.677 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:04:02.677 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:04:02.677 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.678 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:04:02.678 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:04:02.678 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:04:02.678 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.679 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.679 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:04:02.679 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T09:04:02.679 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T09:04:02.679 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:04:02.680 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T09:04:02.680 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T09:04:02.680 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:04:02.680 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:04:02.680 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:04:02.681 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.681 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.681 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.681 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 17, 2023-10-18T09:04:02.681 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:04:02.682 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:03:22.819486+0000", 2023-10-18T09:04:02.682 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1027, 2023-10-18T09:04:02.682 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-18T09:04:02.682 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:04:02.683 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:04:02.683 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:04:02.683 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:04:02.683 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:04:02.683 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:04:02.683 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:04:02.684 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:04:02.684 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:04:02.684 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:04:02.684 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:04:02.684 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:04:02.685 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:04:02.685 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:04:02.685 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:04:02.685 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:04:02.685 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:04:02.686 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:04:02.686 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:04:02.686 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:04:02.686 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:04:02.687 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:04:02.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.687 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "146", 2023-10-18T09:04:02.687 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:04:02.687 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:04:02.688 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:04:02.688 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:04:02.688 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:04:02.688 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:04:02.689 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:04:02.689 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:04:02.689 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:04:02.689 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 33554432, 2023-10-18T09:04:02.690 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:04:02.690 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:04:02.690 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:04:02.690 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:04:02.690 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:04:02.691 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:04:02.691 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:04:02.691 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:04:02.691 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:04:02.692 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:04:02.692 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:04:02.692 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:04:02.692 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:04:02.692 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:04:02.692 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:04:02.693 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:04:02.693 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.693 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:04:02.693 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:04:02.693 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:04:02.694 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:04:02.694 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:04:02.694 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:04:02.694 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:04:02.694 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:04:02.695 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:04:02.695 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:04:02.695 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:04:02.695 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:04:02.695 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:04:02.695 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:04:02.696 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.696 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:04:02.696 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:04:02.696 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:04:02.697 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.697 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.697 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:04:02.697 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T09:04:02.697 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T09:04:02.697 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:04:02.698 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T09:04:02.698 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T09:04:02.698 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:04:02.698 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:04:02.699 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:04:02.699 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.699 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.699 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:04:02.700 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:04:02.700 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.701 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:04:02.701 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:02.701 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:02.702 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:02.702 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:02.703 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:02.703 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:02.703 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:02.704 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:02.704 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:02.704 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:02.705 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:02.705 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:02.706 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.706 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.707 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:04:02.707 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:02.707 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.708 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.708 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.709 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:04:02.709 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:02.709 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.710 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.710 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.710 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:02.711 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.711 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.711 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.712 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:04:02.712 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:02.713 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.713 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.713 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.714 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:04:02.714 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:02.714 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.715 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.715 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.715 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:02.716 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.716 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.717 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.717 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:04:02.717 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:02.718 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.718 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.718 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.719 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:04:02.719 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:02.720 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.720 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.720 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.721 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:02.721 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.721 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.722 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.722 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:04:02.722 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:02.723 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.723 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:04:02.724 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.725 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.726 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:04:02.726 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:04:02.726 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:04:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:04:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:02.727 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.728 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:04:02.729 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:02.730 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:02.730 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:02.730 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:02.731 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:02.731 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:02.731 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:02.732 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:02.733 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:02.733 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.733 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.734 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.734 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:04:02.734 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:02.734 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.734 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.735 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.735 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:04:02.735 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:02.735 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.735 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.735 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.736 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:02.736 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.736 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.736 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.737 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:04:02.737 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:02.737 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.737 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.737 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.738 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:04:02.738 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:02.738 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.738 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.738 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.739 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:02.739 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.739 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.739 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.739 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:04:02.740 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:02.740 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.740 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.740 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.740 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:04:02.741 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:02.741 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.741 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.741 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.741 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:02.742 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.742 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.742 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.742 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:04:02.742 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:02.743 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.743 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.743 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.743 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:04:02.743 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:02.744 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.744 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.744 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.744 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:04:02.744 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:04:02.744 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:04:02.745 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:04:02.745 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:02.745 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:02.745 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:02.745 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.746 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.746 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.746 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:04:02.746 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:02.747 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:02.747 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:02.747 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:02.747 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:02.747 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:02.747 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:02.748 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:02.748 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:02.748 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:02.748 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:02.748 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:02.749 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.749 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.749 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:04:02.749 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:02.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.750 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.750 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:04:02.750 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:02.751 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.751 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.751 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:02.752 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.752 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.752 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.752 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:04:02.752 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:02.753 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.753 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.753 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.753 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:04:02.753 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:02.754 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.754 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.754 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.754 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:02.754 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.755 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.755 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.755 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:04:02.755 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:02.756 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.756 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.756 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.756 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:04:02.757 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:02.757 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.757 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.757 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.757 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:02.758 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.758 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.758 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.758 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:04:02.758 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:02.758 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.759 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.759 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.759 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:04:02.759 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:02.759 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.760 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.760 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.760 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:04:02.760 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:04:02.760 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:04:02.761 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:04:02.761 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:02.761 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:02.761 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:02.761 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.762 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.762 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.762 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:04:02.762 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:02.762 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:02.763 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:02.763 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:02.763 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:02.763 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:02.763 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:02.763 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:02.764 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:02.764 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:02.764 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:02.764 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:02.764 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.765 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.765 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.765 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:04:02.765 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:02.765 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.766 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.766 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.766 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:04:02.766 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:02.766 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.767 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.767 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.767 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:02.767 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.767 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.767 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.768 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:04:02.768 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:02.768 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.768 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.768 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.769 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:04:02.769 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:02.769 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.769 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.769 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.770 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:02.770 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.770 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.770 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.770 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:04:02.771 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:02.771 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.771 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.771 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.771 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:04:02.771 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:02.772 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.772 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.772 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.772 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:02.772 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.773 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.773 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.773 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:04:02.773 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:02.773 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.774 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.774 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.774 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:04:02.774 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:02.774 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.775 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.775 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.775 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:04:02.775 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:04:02.775 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:04:02.776 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:04:02.776 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:02.776 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:02.776 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:02.776 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.777 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.777 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.777 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:04:02.777 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:02.777 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:02.777 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:02.778 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:02.778 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:02.778 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:02.778 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:02.778 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:02.779 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:02.779 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:02.779 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:02.779 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:02.779 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.779 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.780 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.780 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:04:02.780 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:02.780 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.780 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.781 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.781 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:04:02.781 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:02.781 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.781 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.782 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:02.782 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.782 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.782 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.783 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:04:02.783 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:02.783 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.783 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.783 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.784 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:04:02.784 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:02.784 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.784 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.784 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:02.785 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.785 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.785 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.785 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:04:02.785 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:02.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.786 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.786 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.786 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:04:02.786 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:02.787 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.787 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.787 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:02.787 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.788 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.788 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:04:02.788 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:02.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.789 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.789 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:04:02.789 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:02.789 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.789 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.790 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:04:02.790 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:04:02.790 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:04:02.790 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:04:02.791 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:02.791 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:02.791 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:02.791 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.792 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:04:02.792 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:02.792 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:02.792 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:02.792 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:02.793 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:02.793 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:02.793 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:02.793 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:02.793 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:02.794 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:02.794 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:02.794 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:02.794 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.795 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.795 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:04:02.795 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:02.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.795 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.796 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.796 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:04:02.796 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:02.796 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.796 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.796 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.797 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:02.797 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.797 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.797 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.797 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:04:02.798 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:02.798 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.798 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.798 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.798 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:04:02.799 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:02.799 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.799 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.799 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.799 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:02.800 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.800 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.800 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.800 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:04:02.800 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:02.800 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.801 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.801 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.801 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:04:02.801 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:02.801 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.802 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.802 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.802 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:02.802 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.802 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.803 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.803 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:04:02.803 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:02.803 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.803 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.804 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.804 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:04:02.804 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:02.804 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.804 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.805 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.805 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:04:02.805 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:04:02.805 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:04:02.805 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:04:02.805 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:02.806 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:02.806 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:02.806 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.806 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.806 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:04:02.807 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:02.807 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:02.807 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:02.807 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:02.808 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:02.808 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:02.808 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:02.808 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:02.808 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:02.808 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:02.809 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:02.809 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:02.809 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.809 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.809 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.810 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:04:02.810 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:02.810 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.810 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.810 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:04:02.811 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:02.811 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.811 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.811 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.811 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:02.812 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.812 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.812 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.812 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:04:02.812 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:02.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.813 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.813 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.813 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:04:02.813 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:02.814 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.814 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.814 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:02.814 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.814 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.815 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.815 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:04:02.815 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:02.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.815 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.816 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.817 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:04:02.817 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:02.817 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.818 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.818 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:02.818 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.818 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.819 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.819 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:04:02.819 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:02.819 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.819 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.820 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.820 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:04:02.820 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:02.820 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.820 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.821 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.821 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:04:02.821 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:04:02.821 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:04:02.821 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:04:02.821 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:02.822 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:02.822 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:02.822 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.822 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.823 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.823 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:04:02.823 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:02.823 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:02.823 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:02.824 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:02.824 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:02.824 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:02.824 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:02.824 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:02.825 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:02.825 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:02.825 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:02.825 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:02.825 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.825 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.826 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.826 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:04:02.826 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:02.826 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.826 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.827 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.827 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:04:02.827 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:02.827 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.827 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.827 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.828 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:02.828 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.828 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.828 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.828 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:04:02.829 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:02.829 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.829 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.829 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.829 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:04:02.830 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:02.830 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.830 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.830 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.830 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:02.831 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.831 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.831 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.831 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:04:02.831 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:02.832 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.832 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.832 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.832 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:04:02.832 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:02.832 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.833 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.833 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.833 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:02.833 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:02.833 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.834 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:02.834 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:04:02.834 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:02.834 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.834 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.835 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:02.835 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:04:02.835 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:02.835 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.835 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.836 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.836 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:04:02.836 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:04:02.836 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:04:02.836 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:04:02.836 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:02.837 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:02.837 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:02.837 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:02.837 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.837 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:04:02.838 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:04:02.838 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.838 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:04:02.838 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:02.838 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:02.839 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:02.839 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:02.839 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:02.839 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:04:02.839 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:02.839 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.840 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.840 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:04:02.840 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:02.840 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:02.840 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:02.841 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:02.841 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:02.841 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:04:02.841 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:02.841 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.842 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:04:02.842 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:02.842 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:02.842 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:02.842 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:02.843 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:02.843 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:04:02.843 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:02.843 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.844 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:04:02.844 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:02.844 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:02.844 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:02.844 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:02.845 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:02.845 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:04:02.845 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:02.845 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.845 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.846 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:04:02.846 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:02.846 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:02.846 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:02.846 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:02.847 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:02.847 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:04:02.847 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:02.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.847 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.847 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:04:02.848 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:02.848 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:02.848 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:02.848 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:02.848 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:02.849 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:04:02.849 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:02.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.849 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.849 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:04:02.850 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:02.850 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:02.850 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:02.850 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:02.850 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:02.851 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:04:02.851 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:02.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:02.851 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:04:02.852 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:02.852 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:02.852 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:02.852 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:02.852 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:02.852 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:04:02.853 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:02.853 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.853 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:04:02.853 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:04:02.853 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:04:02.853 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:04:02.854 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:04:02.854 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:04:02.854 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:04:02.854 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:04:02.854 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:04:02.855 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:04:02.855 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:04:02.855 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:04:02.855 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:04:02.855 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:04:02.855 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:04:02.856 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:02.856 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:04:02.856 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:04:02.856 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:04:02.857 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:04:02.857 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:04:02.857 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:04:02.857 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:04:02.857 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:04:02.857 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:04:02.858 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:04:02.858 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:04:02.858 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:02.858 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:04:02.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.639+0000 7f09cd847640 1 -- 172.21.15.40:0/2885660236 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09ac06ae70 msgr2=0x7f09ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:02.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.639+0000 7f09cd847640 1 --2- 172.21.15.40:0/2885660236 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09ac06ae70 0x7f09ac06d330 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005bc0 tx=0x7f09b4005ab0 comp rx=0 tx=0).stop 2023-10-18T09:04:02.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.639+0000 7f09cd847640 1 -- 172.21.15.40:0/2885660236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09b80039a0 msgr2=0x7f09c8158cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:02.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.639+0000 7f09cd847640 1 --2- 172.21.15.40:0/2885660236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09b80039a0 0x7f09c8158cb0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0017a0 tx=0x7f09bc006360 comp rx=0 tx=0).stop 2023-10-18T09:04:02.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.640+0000 7f09cd847640 1 -- 172.21.15.40:0/2885660236 shutdown_connections 2023-10-18T09:04:02.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.640+0000 7f09cd847640 1 --2- 172.21.15.40:0/2885660236 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09ac06ae70 0x7f09ac06d330 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:02.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.640+0000 7f09cd847640 1 --2- 172.21.15.40:0/2885660236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09c8139d40 0x7f09c813a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:02.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.640+0000 7f09cd847640 1 --2- 172.21.15.40:0/2885660236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09c81599e0 0x7f09c8159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:02.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.640+0000 7f09cd847640 1 --2- 172.21.15.40:0/2885660236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09b80039a0 0x7f09c8158cb0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:02.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.640+0000 7f09cd847640 1 -- 172.21.15.40:0/2885660236 >> 172.21.15.40:0/2885660236 conn(0x7f09c8069720 msgr2=0x7f09c80c9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:02.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.640+0000 7f09cd847640 1 -- 172.21.15.40:0/2885660236 shutdown_connections 2023-10-18T09:04:02.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:02.640+0000 7f09cd847640 1 -- 172.21.15.40:0/2885660236 wait complete. 2023-10-18T09:04:02.863 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-18T09:04:06.734 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:04:06.744 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:04:07.864 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:04:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.105+0000 7f635cf25640 1 -- 172.21.15.40:0/1425161686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63580cc990 msgr2=0x7f63580ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:08.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.105+0000 7f635cf25640 1 --2- 172.21.15.40:0/1425161686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63580cc990 0x7f63580ccd90 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f635806c840 tx=0x7f635002bd70 comp rx=0 tx=0).stop 2023-10-18T09:04:08.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.106+0000 7f635cf25640 1 -- 172.21.15.40:0/1425161686 shutdown_connections 2023-10-18T09:04:08.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.106+0000 7f635cf25640 1 --2- 172.21.15.40:0/1425161686 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63580cc990 0x7f63580ccd90 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:08.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.106+0000 7f635cf25640 1 --2- 172.21.15.40:0/1425161686 >> v2:172.21.15.144:3300/0 conn(0x7f63580b0bc0 0x7f63580b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:08.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.106+0000 7f635cf25640 1 -- 172.21.15.40:0/1425161686 >> 172.21.15.40:0/1425161686 conn(0x7f635806cee0 msgr2=0x7f6358069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:08.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.106+0000 7f635cf25640 1 -- 172.21.15.40:0/1425161686 shutdown_connections 2023-10-18T09:04:08.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.106+0000 7f635cf25640 1 -- 172.21.15.40:0/1425161686 wait complete. 2023-10-18T09:04:08.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.107+0000 7f635cf25640 1 Processor -- start 2023-10-18T09:04:08.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.108+0000 7f635cf25640 1 -- start start 2023-10-18T09:04:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.109+0000 7f635cf25640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63580b0bc0 0x7f6358137ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:08.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.109+0000 7f635cf25640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63580cc990 0x7f63580706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:08.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.109+0000 7f635cf25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6358151bd0 0x7f6358070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:08.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.109+0000 7f635cf25640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635815a280 con 0x7f6358151bd0 2023-10-18T09:04:08.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.109+0000 7f635cf25640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6358159670 con 0x7f63580cc990 2023-10-18T09:04:08.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.109+0000 7f635cf25640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6358159950 con 0x7f63580b0bc0 2023-10-18T09:04:08.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.109+0000 7f6356d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6358151bd0 0x7f6358070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:08.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.109+0000 7f6356d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6358151bd0 0x7f6358070c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57896/0 (socket says 172.21.15.40:57896) 2023-10-18T09:04:08.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.109+0000 7f6356d9d640 1 -- 172.21.15.40:0/3985280278 learned_addr learned my addr 172.21.15.40:0/3985280278 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:08.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.109+0000 7f6355d9b640 1 --2- 172.21.15.40:0/3985280278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63580cc990 0x7f63580706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:08.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.109+0000 7f635659c640 1 --2- 172.21.15.40:0/3985280278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63580b0bc0 0x7f6358137ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:08.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.109+0000 7f6356d9d640 1 -- 172.21.15.40:0/3985280278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63580b0bc0 msgr2=0x7f6358137ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:08.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.109+0000 7f6356d9d640 1 --2- 172.21.15.40:0/3985280278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63580b0bc0 0x7f6358137ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:08.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.109+0000 7f6356d9d640 1 -- 172.21.15.40:0/3985280278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63580cc990 msgr2=0x7f63580706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:08.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.109+0000 7f6356d9d640 1 --2- 172.21.15.40:0/3985280278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63580cc990 0x7f63580706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:08.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.109+0000 7f6356d9d640 1 -- 172.21.15.40:0/3985280278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6350009490 con 0x7f6358151bd0 2023-10-18T09:04:08.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.110+0000 7f6356d9d640 1 --2- 172.21.15.40:0/3985280278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6358151bd0 0x7f6358070c20 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f6350009920 tx=0x7f6350003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:08.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.110+0000 7f634effd640 1 -- 172.21.15.40:0/3985280278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350003e20 con 0x7f6358151bd0 2023-10-18T09:04:08.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.110+0000 7f634effd640 1 -- 172.21.15.40:0/3985280278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6350034440 con 0x7f6358151bd0 2023-10-18T09:04:08.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.110+0000 7f634effd640 1 -- 172.21.15.40:0/3985280278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6350034830 con 0x7f6358151bd0 2023-10-18T09:04:08.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.110+0000 7f635cf25640 1 -- 172.21.15.40:0/3985280278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6358159c30 con 0x7f6358151bd0 2023-10-18T09:04:08.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.110+0000 7f635cf25640 1 -- 172.21.15.40:0/3985280278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63581588c0 con 0x7f6358151bd0 2023-10-18T09:04:08.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.111+0000 7f635cf25640 1 -- 172.21.15.40:0/3985280278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635815a430 con 0x7f6358151bd0 2023-10-18T09:04:08.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.113+0000 7f634effd640 1 -- 172.21.15.40:0/3985280278 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f635003b040 con 0x7f6358151bd0 2023-10-18T09:04:08.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.114+0000 7f634effd640 1 --2- 172.21.15.40:0/3985280278 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f633806af10 0x7f633806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:08.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.114+0000 7f634effd640 1 -- 172.21.15.40:0/3985280278 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f63500abb70 con 0x7f6358151bd0 2023-10-18T09:04:08.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.114+0000 7f635659c640 1 --2- 172.21.15.40:0/3985280278 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f633806af10 0x7f633806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:08.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.115+0000 7f635659c640 1 --2- 172.21.15.40:0/3985280278 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f633806af10 0x7f633806d3d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6348007fd0 tx=0x7f6348009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:08.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.120+0000 7f634effd640 1 -- 172.21.15.40:0/3985280278 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f635815a430 con 0x7f6358151bd0 2023-10-18T09:04:08.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.315+0000 7f635cf25640 1 -- 172.21.15.40:0/3985280278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6358066fd0 con 0x7f6358151bd0 2023-10-18T09:04:08.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.318+0000 7f634effd640 1 -- 172.21.15.40:0/3985280278 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v149) v1 ==== 81+0+34357 (secure 0 0 0) 0x7f6350039090 con 0x7f6358151bd0 2023-10-18T09:04:08.321 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:08.322 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:04:08.322 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 149, 2023-10-18T09:04:08.322 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:04:08.322 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:04:08.322 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:04:06.082636+0000", 2023-10-18T09:04:08.323 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:04:08.323 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:04:08.323 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:04:08.323 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:04:08.324 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:04:08.324 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:04:08.324 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:04:08.324 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:04:08.324 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:04:08.324 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:04:08.325 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:04:08.325 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:04:08.325 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:04:08.325 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:04:08.325 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:04:08.325 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 17, 2023-10-18T09:04:08.326 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:04:08.326 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:04:08.326 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "luminous", 2023-10-18T09:04:08.326 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:04:08.326 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:04:08.327 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:04:08.327 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.327 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:04:08.327 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:04:08.327 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:04:08.327 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:04:08.328 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:04:08.328 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:04:08.328 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:04:08.328 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:04:08.328 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:04:08.328 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:04:08.329 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:04:08.329 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:04:08.329 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:04:08.329 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:04:08.329 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:04:08.329 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:04:08.330 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:04:08.330 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:04:08.330 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:04:08.330 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:04:08.330 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:04:08.331 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:04:08.331 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:04:08.331 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:04:08.331 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:04:08.331 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:04:08.332 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:04:08.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.332 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:04:08.332 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:04:08.332 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:04:08.332 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:04:08.333 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:04:08.333 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:04:08.333 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:04:08.333 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:04:08.333 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:04:08.333 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:04:08.334 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:04:08.334 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:04:08.334 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:04:08.334 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:04:08.334 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:04:08.338 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:04:08.338 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:04:08.338 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:04:08.339 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:04:08.339 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:04:08.339 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:04:08.339 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:04:08.339 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:04:08.340 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:04:08.340 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:04:08.340 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:04:08.340 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:04:08.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.340 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:04:08.341 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:04:08.341 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:04:08.341 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:04:08.341 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:04:08.341 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:04:08.342 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:04:08.342 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:04:08.342 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:04:08.342 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:04:08.342 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:04:08.343 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:04:08.343 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:04:08.343 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:04:08.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.343 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:04:08.343 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:04:08.344 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.344 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:04:08.344 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:04:08.344 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:04:08.344 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:04:08.345 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:04:08.345 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:04:08.345 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:04:08.345 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:04:08.345 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:04:08.346 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.346 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 16, 2023-10-18T09:04:08.346 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:04:08.347 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:03:21.754758+0000", 2023-10-18T09:04:08.347 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:04:08.347 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:04:08.347 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:04:08.347 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:04:08.347 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:04:08.348 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:04:08.348 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:04:08.348 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:04:08.348 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:04:08.348 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:04:08.349 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:04:08.349 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:04:08.349 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:04:08.349 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:04:08.349 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:04:08.349 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:04:08.350 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:04:08.350 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:04:08.350 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:04:08.350 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:04:08.350 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:04:08.351 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:04:08.351 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:04:08.351 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:04:08.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.351 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "143", 2023-10-18T09:04:08.351 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:04:08.352 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:04:08.352 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:04:08.352 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:04:08.352 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:04:08.352 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:04:08.353 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:04:08.353 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:04:08.353 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:04:08.353 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:04:08.353 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:04:08.354 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:04:08.354 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:04:08.354 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:04:08.354 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:04:08.354 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:04:08.355 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:04:08.355 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:04:08.355 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:04:08.355 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:04:08.355 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:04:08.355 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:04:08.356 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:04:08.356 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:04:08.356 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:04:08.356 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:04:08.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.356 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:04:08.357 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:04:08.357 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:04:08.357 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:04:08.357 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:04:08.357 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:04:08.357 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:04:08.358 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:04:08.358 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:04:08.358 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:04:08.358 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:04:08.358 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:04:08.359 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:04:08.359 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:04:08.359 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:04:08.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.359 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:04:08.360 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:04:08.360 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:04:08.360 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.360 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:04:08.360 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T09:04:08.361 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T09:04:08.361 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:04:08.361 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T09:04:08.361 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T09:04:08.361 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:04:08.362 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:04:08.362 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:04:08.362 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.362 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.363 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.363 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 17, 2023-10-18T09:04:08.363 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:04:08.363 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:03:22.819486+0000", 2023-10-18T09:04:08.363 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:04:08.364 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:04:08.364 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:04:08.364 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:04:08.364 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:04:08.364 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:04:08.364 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:04:08.365 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:04:08.365 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:04:08.365 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:04:08.365 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:04:08.365 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:04:08.366 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:04:08.366 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:04:08.366 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:04:08.366 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:04:08.366 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:04:08.367 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:04:08.367 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:04:08.367 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:04:08.367 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:04:08.367 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:04:08.368 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:04:08.368 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:04:08.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.368 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "149", 2023-10-18T09:04:08.368 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:04:08.369 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:04:08.369 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:04:08.369 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:04:08.369 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:04:08.369 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:04:08.370 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:04:08.370 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:04:08.370 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:04:08.370 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 33554432, 2023-10-18T09:04:08.370 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:04:08.371 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:04:08.371 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:04:08.371 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:04:08.371 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:04:08.371 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:04:08.372 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:04:08.372 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:04:08.372 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:04:08.372 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:04:08.372 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:04:08.373 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:04:08.373 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:04:08.373 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:04:08.373 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:04:08.373 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:04:08.373 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.374 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:04:08.374 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:04:08.374 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:04:08.374 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:04:08.374 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:04:08.375 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:04:08.375 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:04:08.375 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:04:08.375 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:04:08.375 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:04:08.376 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:04:08.376 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:04:08.376 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:04:08.376 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:04:08.376 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.376 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:04:08.377 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:04:08.377 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:04:08.377 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.377 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.377 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:04:08.377 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T09:04:08.378 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T09:04:08.378 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:04:08.378 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T09:04:08.378 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T09:04:08.379 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:04:08.379 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:04:08.379 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:04:08.379 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.379 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.379 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:04:08.380 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:04:08.380 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.380 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:04:08.380 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:08.380 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:08.381 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:08.381 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:08.381 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:08.381 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:08.381 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:08.382 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:08.382 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:08.382 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:08.382 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:08.382 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:08.383 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.383 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.383 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.383 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:04:08.383 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:08.384 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.384 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.384 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.384 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:04:08.384 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:08.385 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.385 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.385 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:08.385 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.385 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.386 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.386 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:04:08.386 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:08.386 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.387 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.387 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.387 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:04:08.387 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:08.387 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.387 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.388 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.388 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:08.388 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.388 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.389 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.389 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:04:08.389 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:08.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.389 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.390 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:04:08.390 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:08.390 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.390 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.391 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:08.391 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.391 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.391 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.391 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:04:08.392 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:08.392 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.392 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.392 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.392 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:04:08.392 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:08.393 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.393 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.393 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:04:08.393 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:04:08.393 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:04:08.394 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:04:08.394 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:08.394 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:08.394 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:08.394 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.395 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:04:08.395 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:08.395 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:08.395 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:08.396 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:08.396 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:08.396 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:08.396 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:08.396 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:08.397 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:08.397 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:08.397 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:08.397 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:08.397 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.397 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.398 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.398 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:04:08.398 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:08.398 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.398 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.399 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.399 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:04:08.399 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:08.399 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.399 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.400 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:08.400 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.400 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.400 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.401 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:04:08.401 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:08.401 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.401 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.401 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.401 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:04:08.402 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:08.402 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.402 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.402 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:08.403 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.403 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.403 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.403 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:04:08.403 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:08.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.404 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.404 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.404 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:04:08.404 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:08.404 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.405 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.405 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:08.405 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.405 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.406 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.406 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:04:08.406 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:08.406 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.406 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.406 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.407 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:04:08.407 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:08.407 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.407 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.408 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:04:08.408 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:04:08.408 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:04:08.408 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:04:08.408 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:08.408 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:08.409 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:08.409 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.409 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.409 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.409 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:04:08.410 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:08.410 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:08.410 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:08.410 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:08.410 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:08.410 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:08.411 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:08.411 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:08.411 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:08.411 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:08.411 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:08.412 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:08.412 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.412 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.412 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.412 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:04:08.412 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:08.413 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.413 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.413 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.413 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:04:08.413 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:08.414 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.414 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.414 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.414 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:08.414 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.415 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.415 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.415 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:04:08.415 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:08.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.415 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.416 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.416 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:04:08.416 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:08.416 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.416 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.417 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.417 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:08.417 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.417 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.417 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.418 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:04:08.418 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:08.418 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.418 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.418 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.419 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:04:08.419 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:08.419 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.419 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.419 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.419 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:08.420 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.420 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.420 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.420 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:04:08.420 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:08.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.421 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.421 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.421 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:04:08.421 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:08.421 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.422 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.422 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.422 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:04:08.422 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:04:08.422 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:04:08.423 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:04:08.423 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:08.423 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:08.423 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:08.423 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.424 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.424 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.424 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:04:08.424 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:08.424 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:08.425 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:08.425 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:08.425 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:08.425 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:08.425 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:08.426 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:08.426 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 147, 2023-10-18T09:04:08.426 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:08.426 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:08.426 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:08.427 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.427 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.427 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.427 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:04:08.427 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:08.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.428 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.428 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.428 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:04:08.428 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:08.428 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.429 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.429 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.429 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:08.429 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.429 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.429 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.430 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:04:08.430 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:08.430 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.430 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.430 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.431 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:04:08.431 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:08.431 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.431 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.431 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.432 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:08.432 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.432 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.432 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.432 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:04:08.432 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:08.433 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.433 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.433 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.433 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:04:08.433 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:08.434 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.434 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.434 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.434 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:08.434 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.435 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.435 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.435 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:04:08.435 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:08.435 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.436 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.436 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.436 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:04:08.436 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:08.436 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.436 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.437 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:04:08.437 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:04:08.437 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:04:08.437 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:04:08.438 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:08.438 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:08.438 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:08.438 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.438 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.439 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.439 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:04:08.439 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:08.439 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:08.439 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:08.440 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:08.440 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:08.440 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:08.440 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:08.440 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:08.441 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:08.441 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:08.441 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:08.441 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:08.441 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.441 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.442 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.442 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:04:08.442 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:08.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.442 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.443 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.443 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:04:08.443 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:08.443 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.443 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.444 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:08.444 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.444 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.444 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.444 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:04:08.445 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:08.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.445 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.445 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.445 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:04:08.446 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:08.446 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.446 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.446 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:08.447 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.447 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.447 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:04:08.448 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:08.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.448 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.448 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:04:08.449 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:08.449 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.449 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.449 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:08.450 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.450 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.450 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:04:08.450 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:08.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.451 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.451 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:04:08.451 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:08.452 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.452 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.452 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:04:08.452 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:04:08.452 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:04:08.453 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:04:08.453 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:08.453 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:08.453 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:08.453 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.454 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.454 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:04:08.454 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:08.454 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:08.455 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:08.455 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:08.455 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:08.455 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:08.455 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:08.456 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:08.456 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:08.456 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:08.456 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:08.456 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:08.456 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.457 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.457 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:04:08.457 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:08.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.458 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.458 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:04:08.458 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:08.458 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.459 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.459 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:08.459 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.460 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.460 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:04:08.460 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:08.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.460 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.461 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.461 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:04:08.461 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:08.461 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.461 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.462 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:08.462 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.462 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.463 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:04:08.463 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:08.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.463 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.463 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:04:08.464 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:08.464 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.464 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.464 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:08.465 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.465 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.465 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.465 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:04:08.465 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:08.466 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.466 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.466 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:04:08.466 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:08.467 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.467 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.467 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:04:08.467 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:04:08.468 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:04:08.468 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:04:08.468 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:08.468 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:08.468 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:08.469 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.469 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.469 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.469 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:04:08.469 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:08.469 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:08.470 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:08.470 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:08.470 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:08.470 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:08.470 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:08.471 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:08.471 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:08.471 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:08.471 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:08.471 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:08.472 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.472 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.472 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.472 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:04:08.472 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:08.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.473 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.473 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.473 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:04:08.473 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:08.473 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.474 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.474 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:08.474 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.474 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.475 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.475 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:04:08.475 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:08.475 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.475 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.476 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.476 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:04:08.476 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:08.476 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.476 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.477 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:08.477 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.477 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.477 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.477 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:04:08.478 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:08.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.478 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.478 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:04:08.479 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:08.479 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.479 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.479 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:08.479 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.480 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.480 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.480 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:04:08.480 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:08.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.480 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.481 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.481 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:04:08.481 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:08.481 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.481 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.481 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.482 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:04:08.482 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:04:08.482 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:04:08.482 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:04:08.483 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:08.483 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:08.483 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:08.483 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.484 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.484 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:04:08.484 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:08.484 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:08.484 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:08.485 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:08.485 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:08.485 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:08.485 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:08.485 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:08.485 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 147, 2023-10-18T09:04:08.486 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:08.486 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:08.486 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:08.486 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.486 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.486 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.487 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:04:08.487 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:08.487 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.487 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.488 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:04:08.488 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:08.488 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.488 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.488 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.488 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:08.489 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.489 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.489 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.489 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:04:08.489 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:08.489 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.490 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.490 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.490 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:04:08.490 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:08.490 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.491 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.491 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.491 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:08.491 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.491 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.492 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.492 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:04:08.492 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:08.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.492 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.492 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.493 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:04:08.493 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:08.493 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.493 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.494 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:08.494 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:08.494 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.494 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:08.494 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:04:08.494 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:08.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.495 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.495 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:08.495 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:04:08.495 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:08.496 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.496 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.496 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.496 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:04:08.496 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:04:08.496 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:04:08.497 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:04:08.497 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:08.497 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:08.497 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:08.497 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.497 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.498 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:04:08.498 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:04:08.498 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.498 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:04:08.499 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:08.499 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:08.499 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:08.499 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:08.499 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:08.499 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:04:08.500 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:08.500 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.500 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.500 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:04:08.500 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:08.500 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:08.501 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:08.501 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:08.501 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:08.501 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:04:08.501 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:08.501 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.502 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.502 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:04:08.502 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:08.502 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:08.502 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:08.503 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:08.503 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:08.503 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:04:08.503 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:08.503 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.504 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.504 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:04:08.504 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:08.504 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:08.504 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:08.504 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:08.505 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:08.505 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:04:08.505 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:08.505 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.505 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.506 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:04:08.506 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:08.506 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:08.506 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:08.506 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:08.507 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:08.507 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:04:08.507 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:08.507 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.507 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.507 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:04:08.508 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:08.508 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:08.508 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:08.508 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:08.508 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:08.508 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:04:08.509 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:08.509 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.509 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.509 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:04:08.509 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:08.509 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:08.510 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:08.510 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:08.510 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:08.510 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:04:08.510 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:08.511 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.511 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.511 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:04:08.511 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:08.511 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:08.511 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:08.512 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:08.512 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:08.512 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:04:08.512 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:08.512 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.512 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:04:08.513 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:04:08.513 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [ 2023-10-18T09:04:08.513 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.513 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "17.9", 2023-10-18T09:04:08.514 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:04:08.514 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.514 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 0, 2023-10-18T09:04:08.514 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 2 2023-10-18T09:04:08.514 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.515 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.515 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.515 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.515 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "17.c", 2023-10-18T09:04:08.515 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:04:08.515 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.516 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 0, 2023-10-18T09:04:08.516 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 5 2023-10-18T09:04:08.516 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.516 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:08.516 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.517 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:04:08.517 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:04:08.517 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:04:08.517 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:04:08.517 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:04:08.517 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:04:08.518 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:04:08.518 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:04:08.518 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:04:08.518 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:04:08.518 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:04:08.519 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:04:08.519 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:04:08.519 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.519 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.519 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:04:08.520 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:04:08.520 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:04:08.520 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:04:08.520 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:04:08.520 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:04:08.520 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:04:08.521 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:04:08.521 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:04:08.521 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:04:08.521 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:04:08.521 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:08.521 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:04:08.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.321+0000 7f635cf25640 1 -- 172.21.15.40:0/3985280278 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f633806af10 msgr2=0x7f633806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:08.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.321+0000 7f635cf25640 1 --2- 172.21.15.40:0/3985280278 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f633806af10 0x7f633806d3d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6348007fd0 tx=0x7f6348009040 comp rx=0 tx=0).stop 2023-10-18T09:04:08.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.321+0000 7f635cf25640 1 -- 172.21.15.40:0/3985280278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6358151bd0 msgr2=0x7f6358070c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:08.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.321+0000 7f635cf25640 1 --2- 172.21.15.40:0/3985280278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6358151bd0 0x7f6358070c20 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f6350009920 tx=0x7f6350003820 comp rx=0 tx=0).stop 2023-10-18T09:04:08.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.321+0000 7f635cf25640 1 -- 172.21.15.40:0/3985280278 shutdown_connections 2023-10-18T09:04:08.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.321+0000 7f635cf25640 1 --2- 172.21.15.40:0/3985280278 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f633806af10 0x7f633806d3d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:08.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.321+0000 7f635cf25640 1 --2- 172.21.15.40:0/3985280278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6358151bd0 0x7f6358070c20 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.321+0000 7f635cf25640 1 --2- 172.21.15.40:0/3985280278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63580cc990 0x7f63580706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.321+0000 7f635cf25640 1 --2- 172.21.15.40:0/3985280278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63580b0bc0 0x7f6358137ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.321+0000 7f635cf25640 1 -- 172.21.15.40:0/3985280278 >> 172.21.15.40:0/3985280278 conn(0x7f635806cee0 msgr2=0x7f6358069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.321+0000 7f635cf25640 1 -- 172.21.15.40:0/3985280278 shutdown_connections 2023-10-18T09:04:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.321+0000 7f635cf25640 1 -- 172.21.15.40:0/3985280278 wait complete. 2023-10-18T09:04:08.526 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-18T09:04:08.526 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:04:08.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.762+0000 7ff8e4d35640 1 -- 172.21.15.40:0/274879024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e0157870 msgr2=0x7ff8e00ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:08.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.762+0000 7ff8e4d35640 1 --2- 172.21.15.40:0/274879024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e0157870 0x7ff8e00ca350 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00ccf90 tx=0x7ff8d002bd10 comp rx=0 tx=0).stop 2023-10-18T09:04:08.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.762+0000 7ff8e4d35640 1 -- 172.21.15.40:0/274879024 shutdown_connections 2023-10-18T09:04:08.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.762+0000 7ff8e4d35640 1 --2- 172.21.15.40:0/274879024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e00b2250 0x7ff8e00caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:08.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.762+0000 7ff8e4d35640 1 --2- 172.21.15.40:0/274879024 >> v2:172.21.15.144:3300/0 conn(0x7ff8e00ca890 0x7ff8e00b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:08.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.762+0000 7ff8e4d35640 1 --2- 172.21.15.40:0/274879024 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e0157870 0x7ff8e00ca350 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:08.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.762+0000 7ff8e4d35640 1 -- 172.21.15.40:0/274879024 >> 172.21.15.40:0/274879024 conn(0x7ff8e0069720 msgr2=0x7ff8e0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:08.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.762+0000 7ff8e4d35640 1 -- 172.21.15.40:0/274879024 shutdown_connections 2023-10-18T09:04:08.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.762+0000 7ff8e4d35640 1 -- 172.21.15.40:0/274879024 wait complete. 2023-10-18T09:04:08.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.764+0000 7ff8e4d35640 1 Processor -- start 2023-10-18T09:04:08.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.764+0000 7ff8e4d35640 1 -- start start 2023-10-18T09:04:08.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.765+0000 7ff8e4d35640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e00b2250 0x7ff8e014c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:08.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.765+0000 7ff8e4d35640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8e00ca890 0x7ff8e014d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:08.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.765+0000 7ff8e4d35640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e0157870 0x7ff8e0159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:08.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.765+0000 7ff8e4d35640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e014c660 con 0x7ff8e00b2250 2023-10-18T09:04:08.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.765+0000 7ff8e4d35640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e014ef20 con 0x7ff8e00ca890 2023-10-18T09:04:08.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.765+0000 7ff8e4d35640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e014de80 con 0x7ff8e0157870 2023-10-18T09:04:08.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.766+0000 7ff8df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e00b2250 0x7ff8e014c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:08.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.766+0000 7ff8df7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e00b2250 0x7ff8e014c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57914/0 (socket says 172.21.15.40:57914) 2023-10-18T09:04:08.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.766+0000 7ff8df7fe640 1 -- 172.21.15.40:0/2014603910 learned_addr learned my addr 172.21.15.40:0/2014603910 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:08.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.766+0000 7ff8d6ffd640 1 --2- 172.21.15.40:0/2014603910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8e00ca890 0x7ff8e014d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:08.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.766+0000 7ff8df7fe640 1 -- 172.21.15.40:0/2014603910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e0157870 msgr2=0x7ff8e0159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:08.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.766+0000 7ff8dffff640 1 --2- 172.21.15.40:0/2014603910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e0157870 0x7ff8e0159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:08.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.766+0000 7ff8df7fe640 1 --2- 172.21.15.40:0/2014603910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e0157870 0x7ff8e0159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:08.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.766+0000 7ff8df7fe640 1 -- 172.21.15.40:0/2014603910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8e00ca890 msgr2=0x7ff8e014d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:08.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.766+0000 7ff8df7fe640 1 --2- 172.21.15.40:0/2014603910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8e00ca890 0x7ff8e014d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:08.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.766+0000 7ff8df7fe640 1 -- 172.21.15.40:0/2014603910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d0009560 con 0x7ff8e00b2250 2023-10-18T09:04:08.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.766+0000 7ff8d6ffd640 1 --2- 172.21.15.40:0/2014603910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8e00ca890 0x7ff8e014d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:08.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.766+0000 7ff8df7fe640 1 --2- 172.21.15.40:0/2014603910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e00b2250 0x7ff8e014c120 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009850 tx=0x7ff8d002dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:08.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.767+0000 7ff8dcff9640 1 -- 172.21.15.40:0/2014603910 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0005650 con 0x7ff8e00b2250 2023-10-18T09:04:08.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.767+0000 7ff8dcff9640 1 -- 172.21.15.40:0/2014603910 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff8d00038f0 con 0x7ff8e00b2250 2023-10-18T09:04:08.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.767+0000 7ff8dcff9640 1 -- 172.21.15.40:0/2014603910 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0003ce0 con 0x7ff8e00b2250 2023-10-18T09:04:08.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.767+0000 7ff8e4d35640 1 -- 172.21.15.40:0/2014603910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e014e160 con 0x7ff8e00b2250 2023-10-18T09:04:08.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.767+0000 7ff8e4d35640 1 -- 172.21.15.40:0/2014603910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e014e720 con 0x7ff8e00b2250 2023-10-18T09:04:08.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.768+0000 7ff8e4d35640 1 -- 172.21.15.40:0/2014603910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e0066e00 con 0x7ff8e00b2250 2023-10-18T09:04:08.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.769+0000 7ff8dcff9640 1 -- 172.21.15.40:0/2014603910 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff8d003e450 con 0x7ff8e00b2250 2023-10-18T09:04:08.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.770+0000 7ff8dcff9640 1 --2- 172.21.15.40:0/2014603910 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff8c006ae40 0x7ff8c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:08.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.771+0000 7ff8dcff9640 1 -- 172.21.15.40:0/2014603910 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff8d00abfd0 con 0x7ff8e00b2250 2023-10-18T09:04:08.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.771+0000 7ff8d6ffd640 1 --2- 172.21.15.40:0/2014603910 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff8c006ae40 0x7ff8c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:08.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.772+0000 7ff8d6ffd640 1 --2- 172.21.15.40:0/2014603910 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff8c006ae40 0x7ff8c006d300 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0157280 tx=0x7ff8c80073c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:08.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.775+0000 7ff8dcff9640 1 -- 172.21.15.40:0/2014603910 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff8d0077070 con 0x7ff8e00b2250 2023-10-18T09:04:08.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.972+0000 7ff8e4d35640 1 -- 172.21.15.40:0/2014603910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff8e0153aa0 con 0x7ff8e00b2250 2023-10-18T09:04:08.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.976+0000 7ff8dcff9640 1 -- 172.21.15.40:0/2014603910 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v149) v1 ==== 81+0+34357 (secure 0 0 0) 0x7ff8d0042090 con 0x7ff8e00b2250 2023-10-18T09:04:08.979 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:08.979 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:04:08.979 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 149, 2023-10-18T09:04:08.979 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:04:08.980 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:04:08.980 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:04:06.082636+0000", 2023-10-18T09:04:08.980 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:04:08.980 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:04:08.980 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:04:08.981 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:04:08.981 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:04:08.981 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:04:08.981 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:04:08.982 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:04:08.982 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:04:08.982 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:04:08.982 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:04:08.982 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:04:08.983 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:04:08.983 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:04:08.983 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:04:08.983 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 17, 2023-10-18T09:04:08.984 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:04:08.984 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:04:08.984 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "luminous", 2023-10-18T09:04:08.984 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:04:08.984 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:04:08.985 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:04:08.985 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:08.985 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:04:08.985 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:04:08.985 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:04:08.986 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:04:08.986 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:04:08.986 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:04:08.986 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:04:08.987 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:04:08.987 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:04:08.987 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:04:08.987 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:04:08.987 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:04:08.988 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:04:08.988 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:04:08.988 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:04:08.988 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:04:08.988 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:04:08.989 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:04:08.989 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:04:08.989 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:04:08.989 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:04:08.989 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:04:08.990 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:04:08.990 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:04:08.990 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:04:08.990 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:04:08.991 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:04:08.991 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.991 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:04:08.991 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:04:08.991 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:04:08.992 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:04:08.992 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:04:08.992 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:04:08.992 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:04:08.992 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:04:08.993 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:04:08.993 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:04:08.993 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:04:08.993 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:04:08.994 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:04:08.994 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:04:08.994 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:04:08.994 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:04:08.994 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:04:08.995 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:04:08.995 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:04:08.995 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:04:08.995 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:04:08.995 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:04:08.996 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:04:08.996 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:04:08.996 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:04:08.996 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:04:08.996 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:04:08.997 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:08.997 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:04:08.997 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:04:08.997 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:04:08.998 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:04:08.998 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:04:08.998 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:04:08.998 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:04:08.998 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:04:08.999 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:04:08.999 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:04:08.999 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:04:08.999 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:04:09.000 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:04:09.000 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:04:09.000 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.000 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:04:09.000 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:04:09.001 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.001 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:04:09.001 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:04:09.001 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:04:09.001 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:04:09.002 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:04:09.002 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:04:09.002 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:04:09.002 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:04:09.002 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:04:09.003 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.003 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.003 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.003 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 16, 2023-10-18T09:04:09.003 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:04:09.004 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:03:21.754758+0000", 2023-10-18T09:04:09.004 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:04:09.004 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:04:09.004 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:04:09.005 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:04:09.005 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:04:09.005 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:04:09.005 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:04:09.005 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:04:09.006 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:04:09.006 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:04:09.006 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:04:09.006 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:04:09.006 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:04:09.007 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:04:09.007 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:04:09.007 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:04:09.007 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:04:09.007 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:04:09.008 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:04:09.008 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:04:09.008 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:04:09.008 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:04:09.008 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:04:09.009 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:04:09.009 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.009 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "143", 2023-10-18T09:04:09.009 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:04:09.010 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:04:09.010 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:04:09.010 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:04:09.010 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:04:09.010 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:04:09.011 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:04:09.011 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:04:09.011 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:04:09.011 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:04:09.011 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:04:09.012 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:04:09.012 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:04:09.012 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:04:09.012 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:04:09.012 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:04:09.013 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:04:09.013 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:04:09.013 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:04:09.013 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:04:09.013 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:04:09.014 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:04:09.014 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:04:09.014 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:04:09.014 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:04:09.015 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:04:09.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.015 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:04:09.015 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:04:09.015 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:04:09.016 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:04:09.016 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:04:09.016 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:04:09.016 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:04:09.016 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:04:09.017 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:04:09.017 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:04:09.017 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:04:09.017 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:04:09.018 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:04:09.018 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:04:09.018 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:04:09.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.018 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:04:09.019 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:04:09.019 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:04:09.019 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.019 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:04:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T09:04:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T09:04:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:04:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T09:04:09.020 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T09:04:09.021 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:04:09.021 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:04:09.021 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:04:09.021 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.022 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.022 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.022 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 17, 2023-10-18T09:04:09.022 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:04:09.022 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:03:22.819486+0000", 2023-10-18T09:04:09.023 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:04:09.023 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:04:09.023 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:04:09.023 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:04:09.023 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:04:09.024 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:04:09.024 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:04:09.024 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:04:09.024 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:04:09.024 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:04:09.025 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:04:09.025 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:04:09.025 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:04:09.025 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:04:09.025 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:04:09.026 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:04:09.026 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:04:09.026 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:04:09.026 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:04:09.027 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:04:09.027 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:04:09.027 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:04:09.027 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:04:09.027 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:04:09.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.028 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "149", 2023-10-18T09:04:09.028 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:04:09.028 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:04:09.029 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:04:09.029 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:04:09.029 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:04:09.029 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:04:09.029 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:04:09.030 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:04:09.030 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:04:09.030 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 33554432, 2023-10-18T09:04:09.030 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:04:09.030 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:04:09.031 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:04:09.031 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:04:09.031 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:04:09.031 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:04:09.031 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:04:09.032 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:04:09.032 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:04:09.032 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:04:09.032 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:04:09.032 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:04:09.033 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:04:09.033 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:04:09.033 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:04:09.033 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:04:09.033 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.033 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:04:09.034 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:04:09.034 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:04:09.034 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:04:09.034 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:04:09.035 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:04:09.035 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:04:09.035 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:04:09.035 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:04:09.035 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:04:09.036 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:04:09.036 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:04:09.036 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:04:09.036 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:04:09.036 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.037 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:04:09.037 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:04:09.037 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:04:09.037 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.038 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.038 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:04:09.038 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.75, 2023-10-18T09:04:09.038 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.75, 2023-10-18T09:04:09.038 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:04:09.039 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.75, 2023-10-18T09:04:09.039 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.75, 2023-10-18T09:04:09.039 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:04:09.039 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:04:09.040 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:04:09.040 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.040 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.040 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:04:09.040 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:04:09.041 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.041 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:04:09.041 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:09.041 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:09.041 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:09.042 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:09.042 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:09.042 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:09.042 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:09.042 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:09.043 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:09.043 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:09.043 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:09.043 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:09.044 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.044 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.044 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.044 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:04:09.044 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:09.045 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.045 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.045 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.045 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:04:09.045 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:09.046 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.046 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.046 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.046 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:09.046 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.047 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.047 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.047 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:04:09.047 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:09.047 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.048 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.048 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.048 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:04:09.048 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:09.048 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.049 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.049 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.049 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:09.049 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.049 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.050 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.050 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:04:09.050 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:09.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.051 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.051 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.051 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:04:09.051 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:09.051 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.052 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.052 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:09.052 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.052 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.053 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.053 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:04:09.053 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:09.053 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.054 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.054 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.054 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:04:09.054 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:04:09.055 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.055 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.055 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.055 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:04:09.055 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:04:09.056 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:04:09.056 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:04:09.056 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:09.056 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:09.057 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:09.057 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.057 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.057 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.057 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:04:09.058 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:09.058 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:09.058 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:09.058 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:09.059 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:09.059 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:09.059 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:09.059 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:09.059 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:09.060 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:09.060 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:09.060 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:09.060 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.060 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.061 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.061 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:04:09.061 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:09.061 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.061 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.062 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.062 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:04:09.062 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:09.062 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.062 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.063 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.063 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:09.063 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.063 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.063 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.064 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:04:09.064 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:09.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.064 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.064 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.065 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:04:09.065 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:09.065 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.065 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.065 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.066 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:09.066 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.066 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.066 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.066 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:04:09.067 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:09.067 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.067 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.067 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.068 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:04:09.068 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:09.068 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.068 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.069 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:09.069 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.069 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.069 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.070 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:04:09.070 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:09.070 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.070 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.070 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.071 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:04:09.071 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:04:09.071 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.071 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.072 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:04:09.072 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:04:09.072 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:04:09.072 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:04:09.073 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:09.073 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:09.073 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:09.073 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.074 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.074 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:04:09.074 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:09.074 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:09.074 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:09.075 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:09.075 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:09.075 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:09.075 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:09.075 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:09.076 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:09.076 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:09.076 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:09.076 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:09.076 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.077 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.077 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.077 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:04:09.077 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:09.077 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.078 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.078 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.078 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:04:09.078 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:09.079 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.079 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.079 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.079 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:09.079 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.080 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.080 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.080 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:04:09.080 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:09.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.081 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.081 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.081 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:04:09.081 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:09.082 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.082 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.082 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.082 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:09.082 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.083 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.083 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.083 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:04:09.083 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:09.084 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.084 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.084 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.084 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:04:09.084 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:09.085 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.085 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.085 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.085 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:09.085 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.086 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.086 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.086 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:04:09.086 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:09.086 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.087 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.087 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.087 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:04:09.087 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:04:09.088 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.088 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.088 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.088 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:04:09.088 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:04:09.089 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:04:09.089 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:04:09.089 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:09.089 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:09.089 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:09.090 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.090 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.090 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.090 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:04:09.090 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:09.091 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:09.091 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:09.091 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:09.091 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:09.091 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:09.092 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:09.092 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:09.092 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 147, 2023-10-18T09:04:09.092 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:09.092 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:09.093 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:09.093 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.093 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.093 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.093 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:04:09.094 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:09.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.094 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.095 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:04:09.095 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:09.095 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.095 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.096 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:09.096 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.096 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.096 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:04:09.097 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:09.097 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.097 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.097 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.097 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:04:09.098 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:09.098 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.098 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.099 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:09.099 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.099 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.099 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.099 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:04:09.100 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:09.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.100 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.100 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.100 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:04:09.101 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:09.101 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.101 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.101 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.101 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:09.102 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.102 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.102 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.102 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:04:09.102 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:09.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.103 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.103 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.103 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:04:09.104 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:04:09.104 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.104 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.104 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.104 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:04:09.105 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:04:09.105 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:04:09.105 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:04:09.105 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:09.105 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:09.106 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:09.106 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.106 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.106 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:04:09.107 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:09.107 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:09.107 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:09.107 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:09.108 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:09.108 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:09.108 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:09.108 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:09.108 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:09.109 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:09.109 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:09.109 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:09.109 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.109 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.110 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.110 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:04:09.110 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:09.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.111 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.111 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:04:09.111 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:09.111 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.111 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.112 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:09.112 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.112 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.113 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.113 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:04:09.113 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:09.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.113 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.114 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.114 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:04:09.114 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:09.114 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.114 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.115 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:09.115 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.115 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.116 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:04:09.116 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:09.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.116 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.117 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:04:09.117 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:09.117 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.117 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.118 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:09.118 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.119 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.119 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:04:09.119 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:09.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.120 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.120 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:04:09.120 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:04:09.120 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.120 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.121 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:04:09.121 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:04:09.121 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:04:09.122 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:04:09.122 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:09.122 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:09.122 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:09.122 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.123 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:04:09.123 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:09.123 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:09.123 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:09.124 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:09.124 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:09.124 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:09.124 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:09.124 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:09.125 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:09.125 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:09.125 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:09.125 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:09.125 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.126 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.126 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:04:09.126 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:09.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.127 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.127 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:04:09.127 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:09.127 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.127 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.128 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.128 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:09.128 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.128 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.128 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:04:09.129 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:09.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.129 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.129 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:04:09.129 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:09.130 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.130 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.130 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:09.130 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.131 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.131 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:04:09.131 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:09.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.132 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.132 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:04:09.132 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:09.132 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.133 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.133 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:09.133 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.133 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.133 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.134 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:04:09.134 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:09.134 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.134 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.135 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:04:09.135 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:04:09.135 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.135 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.135 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:04:09.136 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:04:09.136 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:04:09.136 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:04:09.136 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:09.136 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:09.136 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:09.137 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.137 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:04:09.137 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:09.138 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:09.138 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:09.138 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:09.138 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:09.138 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:09.139 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:09.139 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:09.139 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 142, 2023-10-18T09:04:09.139 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:09.139 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:09.140 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:09.140 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.140 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.140 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:04:09.141 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:09.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.141 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:04:09.141 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:09.142 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.142 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.142 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:09.142 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.143 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.143 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:04:09.143 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:09.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.144 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.144 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:04:09.144 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:09.144 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.145 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.145 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:09.145 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.145 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.146 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:04:09.146 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:09.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.146 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.146 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.147 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:04:09.147 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:09.147 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.147 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.148 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:09.148 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.148 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.148 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:04:09.149 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:09.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.149 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.149 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:04:09.150 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:04:09.150 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.150 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.150 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:04:09.151 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:04:09.151 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:04:09.151 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:04:09.151 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:09.151 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:09.151 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:09.152 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.152 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:04:09.152 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:04:09.153 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:04:09.153 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:04:09.153 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:04:09.153 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:04:09.153 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:04:09.153 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:04:09.154 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:04:09.154 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 147, 2023-10-18T09:04:09.154 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:04:09.154 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:04:09.155 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:04:09.155 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.155 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.155 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:04:09.156 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:09.156 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.156 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.156 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.156 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:04:09.156 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:09.157 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.157 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.157 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:04:09.157 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.158 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.158 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:04:09.158 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:09.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.159 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.159 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.159 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:04:09.159 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:09.159 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.160 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.160 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:04:09.160 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.160 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.161 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.161 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:04:09.161 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:09.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.161 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.162 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:04:09.162 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:09.162 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.162 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.163 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:04:09.163 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:04:09.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.163 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:04:09.164 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:04:09.164 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:09.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.164 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:04:09.165 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:04:09.165 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:04:09.165 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.165 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.166 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:04:09.166 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:04:09.166 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:04:09.166 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:04:09.166 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:04:09.167 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:04:09.167 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:04:09.167 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.167 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.167 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:04:09.167 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:04:09.168 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.168 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:04:09.168 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:09.168 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:09.168 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:09.169 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:09.169 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:09.169 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:04:09.169 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:09.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.170 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:04:09.170 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:09.170 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:09.170 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:09.171 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:09.171 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:09.171 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:04:09.171 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:09.171 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.172 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:04:09.172 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:09.172 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:09.172 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:09.172 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:09.173 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:09.173 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:04:09.173 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:09.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.174 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:04:09.174 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:09.174 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:09.174 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:09.174 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:09.175 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:09.175 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:04:09.175 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:09.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.175 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.176 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:04:09.176 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:09.176 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:09.176 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:09.176 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:09.176 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:09.177 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:04:09.177 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:09.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.177 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.177 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:04:09.178 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:09.178 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:09.178 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:09.178 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:09.178 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:09.179 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:04:09.179 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:09.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.179 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.179 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:04:09.179 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:09.180 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:09.180 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:09.180 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:09.180 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:09.180 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:04:09.181 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:09.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.181 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:04:09.181 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:04:09.182 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:04:09.182 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:04:09.182 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:04:09.182 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:04:09.182 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:04:09.183 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:04:09.183 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.183 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:04:09.183 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:04:09.183 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [ 2023-10-18T09:04:09.183 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.184 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "17.9", 2023-10-18T09:04:09.184 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:04:09.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.184 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 0, 2023-10-18T09:04:09.184 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 2 2023-10-18T09:04:09.185 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.185 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.185 INFO:teuthology.orchestra.run.smithi040.stdout: "pgid": "17.c", 2023-10-18T09:04:09.185 INFO:teuthology.orchestra.run.smithi040.stdout: "mappings": [ 2023-10-18T09:04:09.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:09.186 INFO:teuthology.orchestra.run.smithi040.stdout: "from": 0, 2023-10-18T09:04:09.186 INFO:teuthology.orchestra.run.smithi040.stdout: "to": 5 2023-10-18T09:04:09.186 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.186 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:09.187 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.187 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:04:09.187 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:04:09.187 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:04:09.187 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:04:09.188 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:04:09.188 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:04:09.188 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:04:09.188 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:04:09.188 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:04:09.189 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:04:09.189 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:04:09.189 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:04:09.189 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:04:09.189 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:09.190 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:04:09.190 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:04:09.190 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:04:09.190 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:04:09.191 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:04:09.191 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:04:09.191 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:04:09.191 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:04:09.191 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:04:09.192 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:04:09.192 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:04:09.192 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:09.192 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:04:09.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.978+0000 7ff8e4d35640 1 -- 172.21.15.40:0/2014603910 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff8c006ae40 msgr2=0x7ff8c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:09.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.978+0000 7ff8e4d35640 1 --2- 172.21.15.40:0/2014603910 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff8c006ae40 0x7ff8c006d300 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0157280 tx=0x7ff8c80073c0 comp rx=0 tx=0).stop 2023-10-18T09:04:09.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.978+0000 7ff8e4d35640 1 -- 172.21.15.40:0/2014603910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e00b2250 msgr2=0x7ff8e014c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:09.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.978+0000 7ff8e4d35640 1 --2- 172.21.15.40:0/2014603910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e00b2250 0x7ff8e014c120 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009850 tx=0x7ff8d002dce0 comp rx=0 tx=0).stop 2023-10-18T09:04:09.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.979+0000 7ff8e4d35640 1 -- 172.21.15.40:0/2014603910 shutdown_connections 2023-10-18T09:04:09.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.979+0000 7ff8e4d35640 1 --2- 172.21.15.40:0/2014603910 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff8c006ae40 0x7ff8c006d300 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:09.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.979+0000 7ff8e4d35640 1 --2- 172.21.15.40:0/2014603910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e0157870 0x7ff8e0159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:09.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.979+0000 7ff8e4d35640 1 --2- 172.21.15.40:0/2014603910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8e00ca890 0x7ff8e014d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:09.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.979+0000 7ff8e4d35640 1 --2- 172.21.15.40:0/2014603910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e00b2250 0x7ff8e014c120 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:09.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.979+0000 7ff8e4d35640 1 -- 172.21.15.40:0/2014603910 >> 172.21.15.40:0/2014603910 conn(0x7ff8e0069720 msgr2=0x7ff8e00c9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:09.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.979+0000 7ff8e4d35640 1 -- 172.21.15.40:0/2014603910 shutdown_connections 2023-10-18T09:04:09.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:08.979+0000 7ff8e4d35640 1 -- 172.21.15.40:0/2014603910 wait complete. 2023-10-18T09:04:09.197 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:04:09.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.434+0000 7fc85251d640 1 -- 172.21.15.40:0/347648848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84c0b12d0 msgr2=0x7fc84c1635b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:09.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.434+0000 7fc85251d640 1 --2- 172.21.15.40:0/347648848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84c0b12d0 0x7fc84c1635b0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fc844005fd0 tx=0x7fc84402c0f0 comp rx=0 tx=0).stop 2023-10-18T09:04:09.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.434+0000 7fc85251d640 1 -- 172.21.15.40:0/347648848 shutdown_connections 2023-10-18T09:04:09.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.434+0000 7fc85251d640 1 --2- 172.21.15.40:0/347648848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84c0b12d0 0x7fc84c1635b0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:09.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.434+0000 7fc85251d640 1 -- 172.21.15.40:0/347648848 >> 172.21.15.40:0/347648848 conn(0x7fc84c06cee0 msgr2=0x7fc84c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:09.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.434+0000 7fc85251d640 1 -- 172.21.15.40:0/347648848 shutdown_connections 2023-10-18T09:04:09.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.434+0000 7fc85251d640 1 -- 172.21.15.40:0/347648848 wait complete. 2023-10-18T09:04:09.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.436+0000 7fc85251d640 1 Processor -- start 2023-10-18T09:04:09.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.436+0000 7fc85251d640 1 -- start start 2023-10-18T09:04:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.437+0000 7fc85251d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc84c06d710 0x7fc84c1366f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:09.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.437+0000 7fc85251d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84c0b12d0 0x7fc84c1570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:09.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.437+0000 7fc85251d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc84c1586a0 0x7fc84c1495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:09.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.437+0000 7fc85251d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c148070 con 0x7fc84c0b12d0 2023-10-18T09:04:09.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.437+0000 7fc85251d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c1575e0 con 0x7fc84c06d710 2023-10-18T09:04:09.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.437+0000 7fc85251d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c1343a0 con 0x7fc84c1586a0 2023-10-18T09:04:09.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.437+0000 7fc843fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84c0b12d0 0x7fc84c1570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:09.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.438+0000 7fc843fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84c0b12d0 0x7fc84c1570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57932/0 (socket says 172.21.15.40:57932) 2023-10-18T09:04:09.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.438+0000 7fc843fff640 1 -- 172.21.15.40:0/103437452 learned_addr learned my addr 172.21.15.40:0/103437452 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:09.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.438+0000 7fc84bfff640 1 --2- 172.21.15.40:0/103437452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc84c06d710 0x7fc84c1366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:09.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.438+0000 7fc850aba640 1 --2- 172.21.15.40:0/103437452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc84c1586a0 0x7fc84c1495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:09.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.438+0000 7fc843fff640 1 -- 172.21.15.40:0/103437452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc84c1586a0 msgr2=0x7fc84c1495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:09.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.438+0000 7fc843fff640 1 --2- 172.21.15.40:0/103437452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc84c1586a0 0x7fc84c1495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:09.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.438+0000 7fc843fff640 1 -- 172.21.15.40:0/103437452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc84c06d710 msgr2=0x7fc84c1366f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:09.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.438+0000 7fc843fff640 1 --2- 172.21.15.40:0/103437452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc84c06d710 0x7fc84c1366f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:09.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.438+0000 7fc843fff640 1 -- 172.21.15.40:0/103437452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc844009840 con 0x7fc84c0b12d0 2023-10-18T09:04:09.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.438+0000 7fc84bfff640 1 --2- 172.21.15.40:0/103437452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc84c06d710 0x7fc84c1366f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:04:09.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.439+0000 7fc843fff640 1 --2- 172.21.15.40:0/103437452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84c0b12d0 0x7fc84c1570a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc83400c3c0 tx=0x7fc83400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:09.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.439+0000 7fc8497fa640 1 -- 172.21.15.40:0/103437452 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834004230 con 0x7fc84c0b12d0 2023-10-18T09:04:09.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.439+0000 7fc8497fa640 1 -- 172.21.15.40:0/103437452 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc8340044a0 con 0x7fc84c0b12d0 2023-10-18T09:04:09.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.439+0000 7fc8497fa640 1 -- 172.21.15.40:0/103437452 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc834003850 con 0x7fc84c0b12d0 2023-10-18T09:04:09.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.439+0000 7fc85251d640 1 -- 172.21.15.40:0/103437452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84c134700 con 0x7fc84c0b12d0 2023-10-18T09:04:09.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.439+0000 7fc85251d640 1 -- 172.21.15.40:0/103437452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84c136fb0 con 0x7fc84c0b12d0 2023-10-18T09:04:09.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.440+0000 7fc85251d640 1 -- 172.21.15.40:0/103437452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84c066fd0 con 0x7fc84c0b12d0 2023-10-18T09:04:09.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.441+0000 7fc8497fa640 1 -- 172.21.15.40:0/103437452 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc8340039b0 con 0x7fc84c0b12d0 2023-10-18T09:04:09.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.443+0000 7fc8497fa640 1 --2- 172.21.15.40:0/103437452 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc82c06ae40 0x7fc82c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:09.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.443+0000 7fc8497fa640 1 -- 172.21.15.40:0/103437452 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc834088e30 con 0x7fc84c0b12d0 2023-10-18T09:04:09.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.443+0000 7fc84bfff640 1 --2- 172.21.15.40:0/103437452 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc82c06ae40 0x7fc82c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:09.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.445+0000 7fc84bfff640 1 --2- 172.21.15.40:0/103437452 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc82c06ae40 0x7fc82c06d300 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc84c150d50 tx=0x7fc83c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:09.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.447+0000 7fc8497fa640 1 -- 172.21.15.40:0/103437452 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc834053e00 con 0x7fc84c0b12d0 2023-10-18T09:04:09.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.681+0000 7fc85251d640 1 -- 172.21.15.40:0/103437452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc84c06fb50 con 0x7fc84c0b12d0 2023-10-18T09:04:09.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.682+0000 7fc8497fa640 1 -- 172.21.15.40:0/103437452 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fc834017070 con 0x7fc84c0b12d0 2023-10-18T09:04:09.685 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:09.685 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6382,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/4165164542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":4165164542},{"type":"v1","addr":"172.21.15.144:6836","nonce":4165164542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3045960359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3045960359},{"type":"v1","addr":"172.21.15.144:6837","nonce":3045960359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2869040025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2869040025},{"type":"v1","addr":"172.21.15.40:6833","nonce":2869040025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:03:23.782442+0000","modified":"2023-10-18T09:03:26.897709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7893},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7893":{"gid":7893,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2600609919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2600609919},{"type":"v1","addr":"172.21.15.40:6835","nonce":2600609919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-18T09:04:09.685 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 66 2023-10-18T09:04:09.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.685+0000 7fc85251d640 1 -- 172.21.15.40:0/103437452 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc82c06ae40 msgr2=0x7fc82c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:09.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.685+0000 7fc85251d640 1 --2- 172.21.15.40:0/103437452 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc82c06ae40 0x7fc82c06d300 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc84c150d50 tx=0x7fc83c009040 comp rx=0 tx=0).stop 2023-10-18T09:04:09.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.685+0000 7fc85251d640 1 -- 172.21.15.40:0/103437452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84c0b12d0 msgr2=0x7fc84c1570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:09.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.685+0000 7fc85251d640 1 --2- 172.21.15.40:0/103437452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84c0b12d0 0x7fc84c1570a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fc83400c3c0 tx=0x7fc83400c880 comp rx=0 tx=0).stop 2023-10-18T09:04:09.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.685+0000 7fc85251d640 1 -- 172.21.15.40:0/103437452 shutdown_connections 2023-10-18T09:04:09.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.685+0000 7fc85251d640 1 --2- 172.21.15.40:0/103437452 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc82c06ae40 0x7fc82c06d300 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:09.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.685+0000 7fc85251d640 1 --2- 172.21.15.40:0/103437452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc84c1586a0 0x7fc84c1495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:09.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.685+0000 7fc85251d640 1 --2- 172.21.15.40:0/103437452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc84c0b12d0 0x7fc84c1570a0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:09.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.685+0000 7fc85251d640 1 --2- 172.21.15.40:0/103437452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc84c06d710 0x7fc84c1366f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:09.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.686+0000 7fc85251d640 1 -- 172.21.15.40:0/103437452 >> 172.21.15.40:0/103437452 conn(0x7fc84c06cee0 msgr2=0x7fc84c159580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.686+0000 7fc85251d640 1 -- 172.21.15.40:0/103437452 shutdown_connections 2023-10-18T09:04:09.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:09.686+0000 7fc85251d640 1 -- 172.21.15.40:0/103437452 wait complete. 2023-10-18T09:04:09.708 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-18T09:04:09.947 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":7893,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":45.549828812000001,"mdsmap_epoch":66,"osdmap_epoch":149,"osdmap_epoch_barrier":146,"uptime":51.471429704000002} 2023-10-18T09:04:09.948 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 7893, "mdsmap_epoch": 66, "osdmap_epoch": 149, "osdmap_epoch_barrier": 146, "rank_uptime": 45.549828812, "state": "up:active", "uptime": 51.471429704, "want_state": "up:active", "whoami": 0 } 2023-10-18T09:04:09.948 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-18T09:04:09.949 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:04:09.949 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:04:10.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:10.004 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-18T09:04:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:50+0 records in 2023-10-18T09:04:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:50+0 records out 2023-10-18T09:04:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.383383 s, 137 MB/s 2023-10-18T09:04:10.449 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T09:04:10.449 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:04:10.549 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:04:10.675 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T09:04:10.676 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:04:10.676 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:04:10.743 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:04:10.744 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:04:10.802 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:04:11.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.035+0000 7f37c63bf640 1 -- 172.21.15.40:0/1691292086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37c00adea0 msgr2=0x7f37c0097a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:11.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.035+0000 7f37c63bf640 1 --2- 172.21.15.40:0/1691292086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37c00adea0 0x7f37c0097a10 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f37c006d2c0 tx=0x7f37b802c260 comp rx=0 tx=0).stop 2023-10-18T09:04:11.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.035+0000 7f37c63bf640 1 -- 172.21.15.40:0/1691292086 shutdown_connections 2023-10-18T09:04:11.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.035+0000 7f37c63bf640 1 --2- 172.21.15.40:0/1691292086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37c00adea0 0x7f37c0097a10 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.035+0000 7f37c63bf640 1 --2- 172.21.15.40:0/1691292086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c0159570 0x7f37c00cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.035+0000 7f37c63bf640 1 --2- 172.21.15.40:0/1691292086 >> v2:172.21.15.144:3300/0 conn(0x7f37c00cd1c0 0x7f37c00ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.035+0000 7f37c63bf640 1 -- 172.21.15.40:0/1691292086 >> 172.21.15.40:0/1691292086 conn(0x7f37c00b0ab0 msgr2=0x7f37c00b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:11.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.035+0000 7f37c63bf640 1 -- 172.21.15.40:0/1691292086 shutdown_connections 2023-10-18T09:04:11.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.036+0000 7f37c63bf640 1 -- 172.21.15.40:0/1691292086 wait complete. 2023-10-18T09:04:11.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.037+0000 7f37c63bf640 1 Processor -- start 2023-10-18T09:04:11.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.037+0000 7f37c63bf640 1 -- start start 2023-10-18T09:04:11.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.038+0000 7f37c63bf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c00adea0 0x7f37c015c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:11.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.038+0000 7f37c63bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37c00cd1c0 0x7f37c00c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:11.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.038+0000 7f37c63bf640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c0159570 0x7f37c00c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:11.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.038+0000 7f37c63bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c015b330 con 0x7f37c00cd1c0 2023-10-18T09:04:11.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.038+0000 7f37c63bf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c015c760 con 0x7f37c00adea0 2023-10-18T09:04:11.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.038+0000 7f37c63bf640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c015a720 con 0x7f37c0159570 2023-10-18T09:04:11.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.039+0000 7f37c4bbc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37c00cd1c0 0x7f37c00c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:11.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.039+0000 7f37c4bbc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37c00cd1c0 0x7f37c00c2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57946/0 (socket says 172.21.15.40:57946) 2023-10-18T09:04:11.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.039+0000 7f37c4bbc640 1 -- 172.21.15.40:0/2173009991 learned_addr learned my addr 172.21.15.40:0/2173009991 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:11.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.039+0000 7f37c5bbe640 1 --2- 172.21.15.40:0/2173009991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c0159570 0x7f37c00c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:11.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.039+0000 7f37c53bd640 1 --2- 172.21.15.40:0/2173009991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c00adea0 0x7f37c015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:11.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.039+0000 7f37c4bbc640 1 -- 172.21.15.40:0/2173009991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c0159570 msgr2=0x7f37c00c34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:11.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.039+0000 7f37c4bbc640 1 --2- 172.21.15.40:0/2173009991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c0159570 0x7f37c00c34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.039+0000 7f37c4bbc640 1 -- 172.21.15.40:0/2173009991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c00adea0 msgr2=0x7f37c015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:11.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.039+0000 7f37c4bbc640 1 --2- 172.21.15.40:0/2173009991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c00adea0 0x7f37c015c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.039+0000 7f37c4bbc640 1 -- 172.21.15.40:0/2173009991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b8009900 con 0x7f37c00cd1c0 2023-10-18T09:04:11.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.040+0000 7f37c4bbc640 1 --2- 172.21.15.40:0/2173009991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37c00cd1c0 0x7f37c00c2f80 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f37b000c970 tx=0x7f37b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:11.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.040+0000 7f37b5ffb640 1 -- 172.21.15.40:0/2173009991 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b0007bb0 con 0x7f37c00cd1c0 2023-10-18T09:04:11.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.040+0000 7f37b5ffb640 1 -- 172.21.15.40:0/2173009991 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f37b0007e00 con 0x7f37c00cd1c0 2023-10-18T09:04:11.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.040+0000 7f37c63bf640 1 -- 172.21.15.40:0/2173009991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c015aa00 con 0x7f37c00cd1c0 2023-10-18T09:04:11.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.040+0000 7f37b5ffb640 1 -- 172.21.15.40:0/2173009991 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b000f6b0 con 0x7f37c00cd1c0 2023-10-18T09:04:11.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.040+0000 7f37c63bf640 1 -- 172.21.15.40:0/2173009991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c015afc0 con 0x7f37c00cd1c0 2023-10-18T09:04:11.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.041+0000 7f37c63bf640 1 -- 172.21.15.40:0/2173009991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c0067030 con 0x7f37c00cd1c0 2023-10-18T09:04:11.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.042+0000 7f37b5ffb640 1 -- 172.21.15.40:0/2173009991 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f37b000f810 con 0x7f37c00cd1c0 2023-10-18T09:04:11.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.043+0000 7f37b5ffb640 1 --2- 172.21.15.40:0/2173009991 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f37a406ae40 0x7f37a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:11.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.043+0000 7f37b5ffb640 1 -- 172.21.15.40:0/2173009991 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f37b0089ea0 con 0x7f37c00cd1c0 2023-10-18T09:04:11.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.044+0000 7f37c53bd640 1 --2- 172.21.15.40:0/2173009991 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f37a406ae40 0x7f37a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:11.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.045+0000 7f37c53bd640 1 --2- 172.21.15.40:0/2173009991 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f37a406ae40 0x7f37a406d300 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00bb20 tx=0x7f37bc005e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:11.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.047+0000 7f37b5ffb640 1 -- 172.21.15.40:0/2173009991 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f37b0054f00 con 0x7f37c00cd1c0 2023-10-18T09:04:11.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.246+0000 7f37c63bf640 1 -- 172.21.15.40:0/2173009991 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37c0073570 con 0x7f37c00cd1c0 2023-10-18T09:04:11.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.249+0000 7f37b5ffb640 1 -- 172.21.15.40:0/2173009991 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16643 (secure 0 0 0) 0x7f37b0058520 con 0x7f37c00cd1c0 2023-10-18T09:04:11.252 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:11.252 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":149,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:04:06.082636+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:03:21.754758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-18T09:03:22.819486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.9","mappings":[{"from":0,"to":2}]},{"pgid":"17.c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:04:11.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.252+0000 7f37c63bf640 1 -- 172.21.15.40:0/2173009991 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f37a406ae40 msgr2=0x7f37a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:11.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.252+0000 7f37c63bf640 1 --2- 172.21.15.40:0/2173009991 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f37a406ae40 0x7f37a406d300 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00bb20 tx=0x7f37bc005e50 comp rx=0 tx=0).stop 2023-10-18T09:04:11.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.252+0000 7f37c63bf640 1 -- 172.21.15.40:0/2173009991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37c00cd1c0 msgr2=0x7f37c00c2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:11.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.252+0000 7f37c63bf640 1 --2- 172.21.15.40:0/2173009991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37c00cd1c0 0x7f37c00c2f80 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f37b000c970 tx=0x7f37b000ce30 comp rx=0 tx=0).stop 2023-10-18T09:04:11.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.252+0000 7f37c63bf640 1 -- 172.21.15.40:0/2173009991 shutdown_connections 2023-10-18T09:04:11.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.252+0000 7f37c63bf640 1 --2- 172.21.15.40:0/2173009991 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f37a406ae40 0x7f37a406d300 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.252+0000 7f37c63bf640 1 --2- 172.21.15.40:0/2173009991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37c0159570 0x7f37c00c34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.252+0000 7f37c63bf640 1 --2- 172.21.15.40:0/2173009991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37c00cd1c0 0x7f37c00c2f80 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.252+0000 7f37c63bf640 1 --2- 172.21.15.40:0/2173009991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37c00adea0 0x7f37c015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.252+0000 7f37c63bf640 1 -- 172.21.15.40:0/2173009991 >> 172.21.15.40:0/2173009991 conn(0x7f37c00b0ab0 msgr2=0x7f37c0158680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:11.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.252+0000 7f37c63bf640 1 -- 172.21.15.40:0/2173009991 shutdown_connections 2023-10-18T09:04:11.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.252+0000 7f37c63bf640 1 -- 172.21.15.40:0/2173009991 wait complete. 2023-10-18T09:04:11.274 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:03:21.754758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:03:22.819486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:04:11.275 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:04:11.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.511+0000 7f5f1c2f7640 1 -- 172.21.15.40:0/695727902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1413cc50 msgr2=0x7f5f1413d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:11.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.512+0000 7f5f1c2f7640 1 --2- 172.21.15.40:0/695727902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1413cc50 0x7f5f1413d0e0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c005fd0 tx=0x7f5f0c02c0e0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.512+0000 7f5f1c2f7640 1 -- 172.21.15.40:0/695727902 shutdown_connections 2023-10-18T09:04:11.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.512+0000 7f5f1c2f7640 1 --2- 172.21.15.40:0/695727902 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1413cc50 0x7f5f1413d0e0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.512+0000 7f5f1c2f7640 1 --2- 172.21.15.40:0/695727902 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f140aed00 0x7f5f140af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.512+0000 7f5f1c2f7640 1 -- 172.21.15.40:0/695727902 >> 172.21.15.40:0/695727902 conn(0x7f5f14069720 msgr2=0x7f5f14069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:11.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.512+0000 7f5f1c2f7640 1 -- 172.21.15.40:0/695727902 shutdown_connections 2023-10-18T09:04:11.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.512+0000 7f5f1c2f7640 1 -- 172.21.15.40:0/695727902 wait complete. 2023-10-18T09:04:11.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.513+0000 7f5f1c2f7640 1 Processor -- start 2023-10-18T09:04:11.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.514+0000 7f5f1c2f7640 1 -- start start 2023-10-18T09:04:11.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f1c2f7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f140aed00 0x7f5f1413a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:11.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f1c2f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1413c580 0x7f5f140a7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:11.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f1c2f7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1413cc50 0x7f5f140aa750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:11.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f1c2f7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f140ab3c0 con 0x7f5f1413c580 2023-10-18T09:04:11.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f1c2f7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f140a95c0 con 0x7f5f1413cc50 2023-10-18T09:04:11.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f1c2f7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1413b1a0 con 0x7f5f140aed00 2023-10-18T09:04:11.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f19892640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1413c580 0x7f5f140a7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:11.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f19892640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1413c580 0x7f5f140a7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57954/0 (socket says 172.21.15.40:57954) 2023-10-18T09:04:11.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f19892640 1 -- 172.21.15.40:0/2136169159 learned_addr learned my addr 172.21.15.40:0/2136169159 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:11.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f1a093640 1 --2- 172.21.15.40:0/2136169159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f140aed00 0x7f5f1413a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:11.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f19892640 1 -- 172.21.15.40:0/2136169159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f140aed00 msgr2=0x7f5f1413a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:11.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f1a894640 1 --2- 172.21.15.40:0/2136169159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1413cc50 0x7f5f140aa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:11.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f19892640 1 --2- 172.21.15.40:0/2136169159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f140aed00 0x7f5f1413a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f19892640 1 -- 172.21.15.40:0/2136169159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1413cc50 msgr2=0x7f5f140aa750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f19892640 1 --2- 172.21.15.40:0/2136169159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1413cc50 0x7f5f140aa750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f19892640 1 -- 172.21.15.40:0/2136169159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c009840 con 0x7f5f1413c580 2023-10-18T09:04:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.515+0000 7f5f1a093640 1 --2- 172.21.15.40:0/2136169159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f140aed00 0x7f5f1413a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.516+0000 7f5f1a894640 1 --2- 172.21.15.40:0/2136169159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1413cc50 0x7f5f140aa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:11.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.516+0000 7f5f19892640 1 --2- 172.21.15.40:0/2136169159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1413c580 0x7f5f140a7e90 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400d950 tx=0x7f5f0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:11.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.516+0000 7f5f0affd640 1 -- 172.21.15.40:0/2136169159 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04009c30 con 0x7f5f1413c580 2023-10-18T09:04:11.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.516+0000 7f5f0affd640 1 -- 172.21.15.40:0/2136169159 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5f04009e80 con 0x7f5f1413c580 2023-10-18T09:04:11.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.516+0000 7f5f0affd640 1 -- 172.21.15.40:0/2136169159 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f040106c0 con 0x7f5f1413c580 2023-10-18T09:04:11.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.516+0000 7f5f1c2f7640 1 -- 172.21.15.40:0/2136169159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f140a8520 con 0x7f5f1413c580 2023-10-18T09:04:11.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.516+0000 7f5f1c2f7640 1 -- 172.21.15.40:0/2136169159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f140a8ae0 con 0x7f5f1413c580 2023-10-18T09:04:11.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.517+0000 7f5f1c2f7640 1 -- 172.21.15.40:0/2136169159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f14066fd0 con 0x7f5f1413c580 2023-10-18T09:04:11.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.518+0000 7f5f0affd640 1 -- 172.21.15.40:0/2136169159 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5f04010820 con 0x7f5f1413c580 2023-10-18T09:04:11.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.519+0000 7f5f0affd640 1 --2- 172.21.15.40:0/2136169159 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ef806ae40 0x7f5ef806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:11.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.520+0000 7f5f0affd640 1 -- 172.21.15.40:0/2136169159 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5f04089b50 con 0x7f5f1413c580 2023-10-18T09:04:11.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.520+0000 7f5f1a093640 1 --2- 172.21.15.40:0/2136169159 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ef806ae40 0x7f5ef806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:11.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.521+0000 7f5f1a093640 1 --2- 172.21.15.40:0/2136169159 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ef806ae40 0x7f5ef806d300 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5f10000f60 tx=0x7f5f10009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:11.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.524+0000 7f5f0affd640 1 -- 172.21.15.40:0/2136169159 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f04054bb0 con 0x7f5f1413c580 2023-10-18T09:04:11.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.716+0000 7f5f1c2f7640 1 -- 172.21.15.40:0/2136169159 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f140afbb0 con 0x7f5f1413c580 2023-10-18T09:04:11.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.717+0000 7f5f0affd640 1 -- 172.21.15.40:0/2136169159 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v149) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f040170a0 con 0x7f5f1413c580 2023-10-18T09:04:11.719 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:04:11.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.719+0000 7f5f1c2f7640 1 -- 172.21.15.40:0/2136169159 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ef806ae40 msgr2=0x7f5ef806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:11.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.719+0000 7f5f1c2f7640 1 --2- 172.21.15.40:0/2136169159 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ef806ae40 0x7f5ef806d300 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5f10000f60 tx=0x7f5f10009040 comp rx=0 tx=0).stop 2023-10-18T09:04:11.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.719+0000 7f5f1c2f7640 1 -- 172.21.15.40:0/2136169159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1413c580 msgr2=0x7f5f140a7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:11.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.719+0000 7f5f1c2f7640 1 --2- 172.21.15.40:0/2136169159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1413c580 0x7f5f140a7e90 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400d950 tx=0x7f5f0400de10 comp rx=0 tx=0).stop 2023-10-18T09:04:11.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.719+0000 7f5f1c2f7640 1 -- 172.21.15.40:0/2136169159 shutdown_connections 2023-10-18T09:04:11.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.719+0000 7f5f1c2f7640 1 --2- 172.21.15.40:0/2136169159 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ef806ae40 0x7f5ef806d300 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.719+0000 7f5f1c2f7640 1 --2- 172.21.15.40:0/2136169159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1413cc50 0x7f5f140aa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.719+0000 7f5f1c2f7640 1 --2- 172.21.15.40:0/2136169159 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1413c580 0x7f5f140a7e90 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.719+0000 7f5f1c2f7640 1 --2- 172.21.15.40:0/2136169159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f140aed00 0x7f5f1413a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.719+0000 7f5f1c2f7640 1 -- 172.21.15.40:0/2136169159 >> 172.21.15.40:0/2136169159 conn(0x7f5f14069720 msgr2=0x7f5f14069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:11.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.719+0000 7f5f1c2f7640 1 -- 172.21.15.40:0/2136169159 shutdown_connections 2023-10-18T09:04:11.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.719+0000 7f5f1c2f7640 1 -- 172.21.15.40:0/2136169159 wait complete. 2023-10-18T09:04:11.741 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:04:11.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.975+0000 7f6a86fd7640 1 -- 172.21.15.40:0/164847264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a8015eac0 msgr2=0x7f6a800cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:11.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.975+0000 7f6a86fd7640 1 --2- 172.21.15.40:0/164847264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a8015eac0 0x7f6a800cff70 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f6a800b72b0 tx=0x7f6a7c02c110 comp rx=0 tx=0).stop 2023-10-18T09:04:11.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.975+0000 7f6a86fd7640 1 -- 172.21.15.40:0/164847264 shutdown_connections 2023-10-18T09:04:11.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.975+0000 7f6a86fd7640 1 --2- 172.21.15.40:0/164847264 >> v2:172.21.15.144:3300/0 conn(0x7f6a800d19b0 0x7f6a800ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.975+0000 7f6a86fd7640 1 --2- 172.21.15.40:0/164847264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a8015eac0 0x7f6a800cff70 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.975+0000 7f6a86fd7640 1 -- 172.21.15.40:0/164847264 >> 172.21.15.40:0/164847264 conn(0x7f6a8009a610 msgr2=0x7f6a800d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:11.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.976+0000 7f6a86fd7640 1 -- 172.21.15.40:0/164847264 shutdown_connections 2023-10-18T09:04:11.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.976+0000 7f6a86fd7640 1 -- 172.21.15.40:0/164847264 wait complete. 2023-10-18T09:04:11.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.977+0000 7f6a86fd7640 1 Processor -- start 2023-10-18T09:04:11.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.977+0000 7f6a86fd7640 1 -- start start 2023-10-18T09:04:11.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.978+0000 7f6a86fd7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a800d19b0 0x7f6a800c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:11.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.978+0000 7f6a86fd7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a8015eac0 0x7f6a800ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:11.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.978+0000 7f6a86fd7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a8015eea0 0x7f6a800c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:11.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.978+0000 7f6a86fd7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a80158c90 con 0x7f6a800d19b0 2023-10-18T09:04:11.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.978+0000 7f6a86fd7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800cae40 con 0x7f6a8015eac0 2023-10-18T09:04:11.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.978+0000 7f6a86fd7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800cb120 con 0x7f6a8015eea0 2023-10-18T09:04:11.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.979+0000 7f6a85fd5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a800d19b0 0x7f6a800c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:11.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.979+0000 7f6a85fd5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a800d19b0 0x7f6a800c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57964/0 (socket says 172.21.15.40:57964) 2023-10-18T09:04:11.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.979+0000 7f6a85fd5640 1 -- 172.21.15.40:0/3636999504 learned_addr learned my addr 172.21.15.40:0/3636999504 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:11.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.979+0000 7f6a857d4640 1 --2- 172.21.15.40:0/3636999504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a8015eac0 0x7f6a800ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:11.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.979+0000 7f6a867d6640 1 --2- 172.21.15.40:0/3636999504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a8015eea0 0x7f6a800c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:11.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.979+0000 7f6a85fd5640 1 -- 172.21.15.40:0/3636999504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a8015eea0 msgr2=0x7f6a800c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:11.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.979+0000 7f6a85fd5640 1 --2- 172.21.15.40:0/3636999504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a8015eea0 0x7f6a800c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.979+0000 7f6a85fd5640 1 -- 172.21.15.40:0/3636999504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a8015eac0 msgr2=0x7f6a800ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:11.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.979+0000 7f6a85fd5640 1 --2- 172.21.15.40:0/3636999504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a8015eac0 0x7f6a800ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:11.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.979+0000 7f6a85fd5640 1 -- 172.21.15.40:0/3636999504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c009840 con 0x7f6a800d19b0 2023-10-18T09:04:11.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.980+0000 7f6a85fd5640 1 --2- 172.21.15.40:0/3636999504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a800d19b0 0x7f6a800c7030 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c000f80 tx=0x7f6a7c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:11.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.980+0000 7f6a767fc640 1 -- 172.21.15.40:0/3636999504 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c004760 con 0x7f6a800d19b0 2023-10-18T09:04:11.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.980+0000 7f6a767fc640 1 -- 172.21.15.40:0/3636999504 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6a7c0345a0 con 0x7f6a800d19b0 2023-10-18T09:04:11.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.980+0000 7f6a767fc640 1 -- 172.21.15.40:0/3636999504 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c034970 con 0x7f6a800d19b0 2023-10-18T09:04:11.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.980+0000 7f6a86fd7640 1 -- 172.21.15.40:0/3636999504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a800cb400 con 0x7f6a800d19b0 2023-10-18T09:04:11.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.980+0000 7f6a86fd7640 1 -- 172.21.15.40:0/3636999504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a800c7570 con 0x7f6a800d19b0 2023-10-18T09:04:11.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.981+0000 7f6a86fd7640 1 -- 172.21.15.40:0/3636999504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a80071da0 con 0x7f6a800d19b0 2023-10-18T09:04:11.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.982+0000 7f6a767fc640 1 -- 172.21.15.40:0/3636999504 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6a7c034ad0 con 0x7f6a800d19b0 2023-10-18T09:04:11.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.984+0000 7f6a767fc640 1 --2- 172.21.15.40:0/3636999504 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6a6406ae40 0x7f6a6406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:11.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.984+0000 7f6a767fc640 1 -- 172.21.15.40:0/3636999504 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6a7c0ac8e0 con 0x7f6a800d19b0 2023-10-18T09:04:11.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.984+0000 7f6a857d4640 1 --2- 172.21.15.40:0/3636999504 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6a6406ae40 0x7f6a6406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:11.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.986+0000 7f6a857d4640 1 --2- 172.21.15.40:0/3636999504 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6a6406ae40 0x7f6a6406d300 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6a70007fd0 tx=0x7f6a70009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:11.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:11.988+0000 7f6a767fc640 1 -- 172.21.15.40:0/3636999504 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6a7c077950 con 0x7f6a800d19b0 2023-10-18T09:04:12.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.186+0000 7f6a86fd7640 1 -- 172.21.15.40:0/3636999504 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6a800c7730 con 0x7f6a800d19b0 2023-10-18T09:04:12.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.187+0000 7f6a767fc640 1 -- 172.21.15.40:0/3636999504 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v149) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6a7c043020 con 0x7f6a800d19b0 2023-10-18T09:04:12.189 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:04:12.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.189+0000 7f6a86fd7640 1 -- 172.21.15.40:0/3636999504 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6a6406ae40 msgr2=0x7f6a6406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:12.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.189+0000 7f6a86fd7640 1 --2- 172.21.15.40:0/3636999504 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6a6406ae40 0x7f6a6406d300 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6a70007fd0 tx=0x7f6a70009040 comp rx=0 tx=0).stop 2023-10-18T09:04:12.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.189+0000 7f6a86fd7640 1 -- 172.21.15.40:0/3636999504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a800d19b0 msgr2=0x7f6a800c7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:12.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.189+0000 7f6a86fd7640 1 --2- 172.21.15.40:0/3636999504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a800d19b0 0x7f6a800c7030 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c000f80 tx=0x7f6a7c004360 comp rx=0 tx=0).stop 2023-10-18T09:04:12.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.189+0000 7f6a86fd7640 1 -- 172.21.15.40:0/3636999504 shutdown_connections 2023-10-18T09:04:12.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.189+0000 7f6a86fd7640 1 --2- 172.21.15.40:0/3636999504 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6a6406ae40 0x7f6a6406d300 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:12.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.189+0000 7f6a86fd7640 1 --2- 172.21.15.40:0/3636999504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a8015eea0 0x7f6a800c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:12.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.189+0000 7f6a86fd7640 1 --2- 172.21.15.40:0/3636999504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a8015eac0 0x7f6a800ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:12.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.189+0000 7f6a86fd7640 1 --2- 172.21.15.40:0/3636999504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a800d19b0 0x7f6a800c7030 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:12.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.189+0000 7f6a86fd7640 1 -- 172.21.15.40:0/3636999504 >> 172.21.15.40:0/3636999504 conn(0x7f6a8009a610 msgr2=0x7f6a800ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:12.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.190+0000 7f6a86fd7640 1 -- 172.21.15.40:0/3636999504 shutdown_connections 2023-10-18T09:04:12.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.190+0000 7f6a86fd7640 1 -- 172.21.15.40:0/3636999504 wait complete. 2023-10-18T09:04:12.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:04:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.449+0000 7f199d7d0640 1 -- 172.21.15.40:0/3904714541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1998157800 msgr2=0x7f19980cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.449+0000 7f199d7d0640 1 --2- 172.21.15.40:0/3904714541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1998157800 0x7f19980cca30 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f1980005fd0 tx=0x7f198002be20 comp rx=0 tx=0).stop 2023-10-18T09:04:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.449+0000 7f199d7d0640 1 -- 172.21.15.40:0/3904714541 shutdown_connections 2023-10-18T09:04:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.449+0000 7f199d7d0640 1 --2- 172.21.15.40:0/3904714541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1998157800 0x7f19980cca30 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:12.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.449+0000 7f199d7d0640 1 --2- 172.21.15.40:0/3904714541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f199806d660 0x7f19980b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.449+0000 7f199d7d0640 1 -- 172.21.15.40:0/3904714541 >> 172.21.15.40:0/3904714541 conn(0x7f1998069720 msgr2=0x7f1998069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:12.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.449+0000 7f199d7d0640 1 -- 172.21.15.40:0/3904714541 shutdown_connections 2023-10-18T09:04:12.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.449+0000 7f199d7d0640 1 -- 172.21.15.40:0/3904714541 wait complete. 2023-10-18T09:04:12.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.451+0000 7f199d7d0640 1 Processor -- start 2023-10-18T09:04:12.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.451+0000 7f199d7d0640 1 -- start start 2023-10-18T09:04:12.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.452+0000 7f199d7d0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f199806d660 0x7f19981f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:12.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.452+0000 7f199d7d0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19980ccf70 0x7f199814cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:12.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.452+0000 7f199d7d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1998157800 0x7f19980c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:12.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.452+0000 7f199d7d0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1998157d70 con 0x7f19980ccf70 2023-10-18T09:04:12.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.452+0000 7f199d7d0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19981f2360 con 0x7f1998157800 2023-10-18T09:04:12.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.452+0000 7f199d7d0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19981f2640 con 0x7f199806d660 2023-10-18T09:04:12.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.452+0000 7f198ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19980ccf70 0x7f199814cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:12.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.452+0000 7f198ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19980ccf70 0x7f199814cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57976/0 (socket says 172.21.15.40:57976) 2023-10-18T09:04:12.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.452+0000 7f198ffff640 1 -- 172.21.15.40:0/3312920931 learned_addr learned my addr 172.21.15.40:0/3312920931 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:12.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.453+0000 7f1996ffd640 1 --2- 172.21.15.40:0/3312920931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f199806d660 0x7f19981f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:12.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.453+0000 7f198ffff640 1 -- 172.21.15.40:0/3312920931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f199806d660 msgr2=0x7f19981f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:12.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.453+0000 7f198ffff640 1 --2- 172.21.15.40:0/3312920931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f199806d660 0x7f19981f1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:12.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.453+0000 7f19977fe640 1 --2- 172.21.15.40:0/3312920931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1998157800 0x7f19980c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.453+0000 7f198ffff640 1 -- 172.21.15.40:0/3312920931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1998157800 msgr2=0x7f19980c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.453+0000 7f198ffff640 1 --2- 172.21.15.40:0/3312920931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1998157800 0x7f19980c6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.453+0000 7f198ffff640 1 -- 172.21.15.40:0/3312920931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980009560 con 0x7f19980ccf70 2023-10-18T09:04:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.453+0000 7f19977fe640 1 --2- 172.21.15.40:0/3312920931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1998157800 0x7f19980c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:12.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.453+0000 7f198ffff640 1 --2- 172.21.15.40:0/3312920931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19980ccf70 0x7f199814cdc0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f1980009850 tx=0x7f198002da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.454+0000 7f198f7fe640 1 -- 172.21.15.40:0/3312920931 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198002de60 con 0x7f19980ccf70 2023-10-18T09:04:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.454+0000 7f198f7fe640 1 -- 172.21.15.40:0/3312920931 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f19800036e0 con 0x7f19980ccf70 2023-10-18T09:04:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.454+0000 7f198f7fe640 1 -- 172.21.15.40:0/3312920931 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980003a90 con 0x7f19980ccf70 2023-10-18T09:04:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.454+0000 7f199d7d0640 1 -- 172.21.15.40:0/3312920931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19981f2920 con 0x7f19980ccf70 2023-10-18T09:04:12.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.454+0000 7f199d7d0640 1 -- 172.21.15.40:0/3312920931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19980c7470 con 0x7f19980ccf70 2023-10-18T09:04:12.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.455+0000 7f199d7d0640 1 -- 172.21.15.40:0/3312920931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1998066fd0 con 0x7f19980ccf70 2023-10-18T09:04:12.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.456+0000 7f198f7fe640 1 -- 172.21.15.40:0/3312920931 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1980003bf0 con 0x7f19980ccf70 2023-10-18T09:04:12.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.457+0000 7f198f7fe640 1 --2- 172.21.15.40:0/3312920931 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f197806ae40 0x7f197806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:12.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.457+0000 7f198f7fe640 1 -- 172.21.15.40:0/3312920931 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f19800ac270 con 0x7f19980ccf70 2023-10-18T09:04:12.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.458+0000 7f1996ffd640 1 --2- 172.21.15.40:0/3312920931 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f197806ae40 0x7f197806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:12.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.459+0000 7f1996ffd640 1 --2- 172.21.15.40:0/3312920931 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f197806ae40 0x7f197806d300 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1988009b60 tx=0x7f1988005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:12.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.461+0000 7f198f7fe640 1 -- 172.21.15.40:0/3312920931 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1980077360 con 0x7f19980ccf70 2023-10-18T09:04:12.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.652+0000 7f199d7d0640 1 -- 172.21.15.40:0/3312920931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f19980cc810 con 0x7f19980ccf70 2023-10-18T09:04:12.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.653+0000 7f198f7fe640 1 -- 172.21.15.40:0/3312920931 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v149) v1 ==== 100+0+11 (secure 0 0 0) 0x7f198007a980 con 0x7f19980ccf70 2023-10-18T09:04:12.655 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:04:12.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.656+0000 7f199d7d0640 1 -- 172.21.15.40:0/3312920931 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f197806ae40 msgr2=0x7f197806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:12.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.656+0000 7f199d7d0640 1 --2- 172.21.15.40:0/3312920931 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f197806ae40 0x7f197806d300 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1988009b60 tx=0x7f1988005f90 comp rx=0 tx=0).stop 2023-10-18T09:04:12.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.656+0000 7f199d7d0640 1 -- 172.21.15.40:0/3312920931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19980ccf70 msgr2=0x7f199814cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:12.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.656+0000 7f199d7d0640 1 --2- 172.21.15.40:0/3312920931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19980ccf70 0x7f199814cdc0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f1980009850 tx=0x7f198002da60 comp rx=0 tx=0).stop 2023-10-18T09:04:12.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.656+0000 7f199d7d0640 1 -- 172.21.15.40:0/3312920931 shutdown_connections 2023-10-18T09:04:12.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.656+0000 7f199d7d0640 1 --2- 172.21.15.40:0/3312920931 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f197806ae40 0x7f197806d300 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:12.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.656+0000 7f199d7d0640 1 --2- 172.21.15.40:0/3312920931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1998157800 0x7f19980c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:12.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.656+0000 7f199d7d0640 1 --2- 172.21.15.40:0/3312920931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19980ccf70 0x7f199814cdc0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:12.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.656+0000 7f199d7d0640 1 --2- 172.21.15.40:0/3312920931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f199806d660 0x7f19981f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:12.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.656+0000 7f199d7d0640 1 -- 172.21.15.40:0/3312920931 >> 172.21.15.40:0/3312920931 conn(0x7f1998069720 msgr2=0x7f19980c1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:12.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.656+0000 7f199d7d0640 1 -- 172.21.15.40:0/3312920931 shutdown_connections 2023-10-18T09:04:12.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.656+0000 7f199d7d0640 1 -- 172.21.15.40:0/3312920931 wait complete. 2023-10-18T09:04:12.677 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:04:12.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.913+0000 7f533a647640 1 -- 172.21.15.40:0/3606505839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5320004d00 msgr2=0x7f5320005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:12.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.913+0000 7f533a647640 1 --2- 172.21.15.40:0/3606505839 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5320004d00 0x7f5320005180 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f532c00c260 tx=0x7f53200057f0 comp rx=0 tx=0).stop 2023-10-18T09:04:12.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.913+0000 7f5339e46640 1 -- 172.21.15.40:0/3606505839 reap_dead start 2023-10-18T09:04:12.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.913+0000 7f533a647640 1 -- 172.21.15.40:0/3606505839 shutdown_connections 2023-10-18T09:04:12.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.913+0000 7f533a647640 1 -- 172.21.15.40:0/3606505839 >> 172.21.15.40:0/3606505839 conn(0x7f533408ad70 msgr2=0x7f533409a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:12.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.913+0000 7f533a647640 1 -- 172.21.15.40:0/3606505839 shutdown_connections 2023-10-18T09:04:12.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.913+0000 7f533a647640 1 -- 172.21.15.40:0/3606505839 wait complete. 2023-10-18T09:04:12.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.915+0000 7f533a647640 1 Processor -- start 2023-10-18T09:04:12.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.915+0000 7f533a647640 1 -- start start 2023-10-18T09:04:12.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.916+0000 7f533a647640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5320004d00 0x7f53340ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:12.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.916+0000 7f533a647640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53340c58a0 0x7f53341524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:12.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.916+0000 7f533a647640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5334154d50 0x7f533415d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:12.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.916+0000 7f533a647640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5334153250 con 0x7f5334154d50 2023-10-18T09:04:12.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.916+0000 7f533a647640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5334154040 con 0x7f53340c58a0 2023-10-18T09:04:12.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.916+0000 7f533a647640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5334153030 con 0x7f5320004d00 2023-10-18T09:04:12.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.916+0000 7f5339e46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5334154d50 0x7f533415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:12.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.916+0000 7f5339e46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5334154d50 0x7f533415d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57992/0 (socket says 172.21.15.40:57992) 2023-10-18T09:04:12.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.916+0000 7f5339e46640 1 -- 172.21.15.40:0/400777095 learned_addr learned my addr 172.21.15.40:0/400777095 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:12.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.917+0000 7f5339645640 1 --2- 172.21.15.40:0/400777095 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5320004d00 0x7f53340ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:12.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.917+0000 7f5338e44640 1 --2- 172.21.15.40:0/400777095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53340c58a0 0x7f53341524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:12.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.917+0000 7f5339e46640 1 -- 172.21.15.40:0/400777095 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5320004d00 msgr2=0x7f53340ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:12.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.917+0000 7f5339e46640 1 --2- 172.21.15.40:0/400777095 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5320004d00 0x7f53340ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:12.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.917+0000 7f5339e46640 1 -- 172.21.15.40:0/400777095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53340c58a0 msgr2=0x7f53341524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:12.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.917+0000 7f5339e46640 1 --2- 172.21.15.40:0/400777095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53340c58a0 0x7f53341524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:12.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.917+0000 7f5339e46640 1 -- 172.21.15.40:0/400777095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5334153430 con 0x7f5334154d50 2023-10-18T09:04:12.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.917+0000 7f5339e46640 1 --2- 172.21.15.40:0/400777095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5334154d50 0x7f533415d280 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f532c02ca00 tx=0x7f532c001430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:12.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.918+0000 7f5329ffb640 1 -- 172.21.15.40:0/400777095 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c0062d0 con 0x7f5334154d50 2023-10-18T09:04:12.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.918+0000 7f5329ffb640 1 -- 172.21.15.40:0/400777095 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f532c001660 con 0x7f5334154d50 2023-10-18T09:04:12.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.918+0000 7f5329ffb640 1 -- 172.21.15.40:0/400777095 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c006650 con 0x7f5334154d50 2023-10-18T09:04:12.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.918+0000 7f533a647640 1 -- 172.21.15.40:0/400777095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5334153710 con 0x7f5334154d50 2023-10-18T09:04:12.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.918+0000 7f533a647640 1 -- 172.21.15.40:0/400777095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5334153cd0 con 0x7f5334154d50 2023-10-18T09:04:12.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.919+0000 7f533a647640 1 -- 172.21.15.40:0/400777095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5334071da0 con 0x7f5334154d50 2023-10-18T09:04:12.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.920+0000 7f5329ffb640 1 -- 172.21.15.40:0/400777095 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f532c031030 con 0x7f5334154d50 2023-10-18T09:04:12.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.921+0000 7f5329ffb640 1 --2- 172.21.15.40:0/400777095 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f531806ae70 0x7f531806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:12.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.921+0000 7f5329ffb640 1 -- 172.21.15.40:0/400777095 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f532c0aa840 con 0x7f5334154d50 2023-10-18T09:04:12.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.922+0000 7f5339645640 1 --2- 172.21.15.40:0/400777095 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f531806ae70 0x7f531806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:12.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.923+0000 7f5339645640 1 --2- 172.21.15.40:0/400777095 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f531806ae70 0x7f531806d330 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5330003fd0 tx=0x7f5330003970 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:12.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:12.926+0000 7f5329ffb640 1 -- 172.21.15.40:0/400777095 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f532c079330 con 0x7f5334154d50 2023-10-18T09:04:13.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.136+0000 7f533a647640 1 -- 172.21.15.40:0/400777095 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5334066e00 con 0x7f5334154d50 2023-10-18T09:04:13.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.137+0000 7f5329ffb640 1 -- 172.21.15.40:0/400777095 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v149) v1 ==== 80+0+0 (secure 0 0 0) 0x7f532c078ec0 con 0x7f5334154d50 2023-10-18T09:04:13.139 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T09:04:13.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.139+0000 7f533a647640 1 -- 172.21.15.40:0/400777095 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f531806ae70 msgr2=0x7f531806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:13.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.139+0000 7f533a647640 1 --2- 172.21.15.40:0/400777095 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f531806ae70 0x7f531806d330 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5330003fd0 tx=0x7f5330003970 comp rx=0 tx=0).stop 2023-10-18T09:04:13.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.139+0000 7f533a647640 1 -- 172.21.15.40:0/400777095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5334154d50 msgr2=0x7f533415d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:13.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.139+0000 7f533a647640 1 --2- 172.21.15.40:0/400777095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5334154d50 0x7f533415d280 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f532c02ca00 tx=0x7f532c001430 comp rx=0 tx=0).stop 2023-10-18T09:04:13.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.140+0000 7f533a647640 1 -- 172.21.15.40:0/400777095 shutdown_connections 2023-10-18T09:04:13.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.140+0000 7f533a647640 1 --2- 172.21.15.40:0/400777095 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f531806ae70 0x7f531806d330 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:13.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.140+0000 7f533a647640 1 --2- 172.21.15.40:0/400777095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5334154d50 0x7f533415d280 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:13.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.140+0000 7f533a647640 1 --2- 172.21.15.40:0/400777095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53340c58a0 0x7f53341524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:13.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.140+0000 7f533a647640 1 --2- 172.21.15.40:0/400777095 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5320004d00 0x7f53340ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:13.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.140+0000 7f533a647640 1 -- 172.21.15.40:0/400777095 >> 172.21.15.40:0/400777095 conn(0x7f533408ad70 msgr2=0x7f533415eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:13.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.140+0000 7f533a647640 1 -- 172.21.15.40:0/400777095 shutdown_connections 2023-10-18T09:04:13.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.140+0000 7f533a647640 1 -- 172.21.15.40:0/400777095 wait complete. 2023-10-18T09:04:13.162 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-18T09:04:13.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:13.163 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-18T09:04:13.313 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T09:04:13.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:13.314 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T09:04:13.365 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:04:13.365 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:04:13.409 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:04:13.410 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:04:13.505 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:04:13.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.738+0000 7f505dfff640 1 -- 172.21.15.40:0/1104405615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f505806d6f0 msgr2=0x7f50580b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:13.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.738+0000 7f505dfff640 1 --2- 172.21.15.40:0/1104405615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f505806d6f0 0x7f50580b15b0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f50580caf60 tx=0x7f504c02bdb0 comp rx=0 tx=0).stop 2023-10-18T09:04:13.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.738+0000 7f505dfff640 1 -- 172.21.15.40:0/1104405615 shutdown_connections 2023-10-18T09:04:13.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.738+0000 7f505dfff640 1 --2- 172.21.15.40:0/1104405615 >> v2:172.21.15.144:3300/0 conn(0x7f50580c99c0 0x7f50580c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:13.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.738+0000 7f505dfff640 1 --2- 172.21.15.40:0/1104405615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f505806d6f0 0x7f50580b15b0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:13.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.738+0000 7f505dfff640 1 -- 172.21.15.40:0/1104405615 >> 172.21.15.40:0/1104405615 conn(0x7f5058069720 msgr2=0x7f5058069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:13.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.738+0000 7f505dfff640 1 -- 172.21.15.40:0/1104405615 shutdown_connections 2023-10-18T09:04:13.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.738+0000 7f505dfff640 1 -- 172.21.15.40:0/1104405615 wait complete. 2023-10-18T09:04:13.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.740+0000 7f505dfff640 1 Processor -- start 2023-10-18T09:04:13.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.740+0000 7f505dfff640 1 -- start start 2023-10-18T09:04:13.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.741+0000 7f505dfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f505806d6f0 0x7f505814bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:13.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.741+0000 7f505dfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50580b0630 0x7f50580ab120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:13.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.741+0000 7f505dfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50580c99c0 0x7f5058155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:13.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.741+0000 7f505dfff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5058156510 con 0x7f50580c99c0 2023-10-18T09:04:13.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.741+0000 7f505dfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505814f3c0 con 0x7f50580b0630 2023-10-18T09:04:13.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.741+0000 7f505dfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5058156750 con 0x7f505806d6f0 2023-10-18T09:04:13.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.741+0000 7f505d7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50580c99c0 0x7f5058155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:13.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.741+0000 7f505d7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50580c99c0 0x7f5058155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58004/0 (socket says 172.21.15.40:58004) 2023-10-18T09:04:13.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.741+0000 7f505d7fe640 1 -- 172.21.15.40:0/524888554 learned_addr learned my addr 172.21.15.40:0/524888554 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:13.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.742+0000 7f505d7fe640 1 -- 172.21.15.40:0/524888554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f505806d6f0 msgr2=0x7f505814bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:13.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.742+0000 7f505d7fe640 1 --2- 172.21.15.40:0/524888554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f505806d6f0 0x7f505814bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:13.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.742+0000 7f505d7fe640 1 -- 172.21.15.40:0/524888554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50580b0630 msgr2=0x7f50580ab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:13.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.742+0000 7f5057fff640 1 --2- 172.21.15.40:0/524888554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50580b0630 0x7f50580ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:13.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.742+0000 7f505d7fe640 1 --2- 172.21.15.40:0/524888554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50580b0630 0x7f50580ab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:13.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.742+0000 7f505d7fe640 1 -- 172.21.15.40:0/524888554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c009560 con 0x7f50580c99c0 2023-10-18T09:04:13.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.742+0000 7f5057fff640 1 --2- 172.21.15.40:0/524888554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50580b0630 0x7f50580ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:13.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.742+0000 7f505d7fe640 1 --2- 172.21.15.40:0/524888554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50580c99c0 0x7f5058155fd0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f505000b970 tx=0x7f505000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:13.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.742+0000 7f50557fa640 1 -- 172.21.15.40:0/524888554 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505000cb40 con 0x7f50580c99c0 2023-10-18T09:04:13.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.742+0000 7f50557fa640 1 -- 172.21.15.40:0/524888554 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f505000cdb0 con 0x7f50580c99c0 2023-10-18T09:04:13.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.742+0000 7f50557fa640 1 -- 172.21.15.40:0/524888554 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50500146a0 con 0x7f50580c99c0 2023-10-18T09:04:13.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.742+0000 7f505dfff640 1 -- 172.21.15.40:0/524888554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5058156a30 con 0x7f50580c99c0 2023-10-18T09:04:13.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.743+0000 7f505dfff640 1 -- 172.21.15.40:0/524888554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5058156ff0 con 0x7f50580c99c0 2023-10-18T09:04:13.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.744+0000 7f505dfff640 1 -- 172.21.15.40:0/524888554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5058157210 con 0x7f50580c99c0 2023-10-18T09:04:13.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.745+0000 7f50557fa640 1 -- 172.21.15.40:0/524888554 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f505001d3b0 con 0x7f50580c99c0 2023-10-18T09:04:13.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.745+0000 7f50557fa640 1 --2- 172.21.15.40:0/524888554 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f503c06ae40 0x7f503c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:13.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.746+0000 7f50557fa640 1 -- 172.21.15.40:0/524888554 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5050089880 con 0x7f50580c99c0 2023-10-18T09:04:13.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.746+0000 7f505cffd640 1 --2- 172.21.15.40:0/524888554 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f503c06ae40 0x7f503c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:13.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.747+0000 7f505cffd640 1 --2- 172.21.15.40:0/524888554 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f503c06ae40 0x7f503c06d300 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f504c035000 tx=0x7f504c0046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:13.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.750+0000 7f50557fa640 1 -- 172.21.15.40:0/524888554 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5050054850 con 0x7f50580c99c0 2023-10-18T09:04:13.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.944+0000 7f505dfff640 1 -- 172.21.15.40:0/524888554 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5058066e00 con 0x7f50580c99c0 2023-10-18T09:04:13.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.946+0000 7f50557fa640 1 -- 172.21.15.40:0/524888554 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16643 (secure 0 0 0) 0x7f5050019090 con 0x7f50580c99c0 2023-10-18T09:04:13.948 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:13.949 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":149,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:04:06.082636+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:03:21.754758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-18T09:03:22.819486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.9","mappings":[{"from":0,"to":2}]},{"pgid":"17.c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:04:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.948+0000 7f505dfff640 1 -- 172.21.15.40:0/524888554 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f503c06ae40 msgr2=0x7f503c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.948+0000 7f505dfff640 1 --2- 172.21.15.40:0/524888554 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f503c06ae40 0x7f503c06d300 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f504c035000 tx=0x7f504c0046c0 comp rx=0 tx=0).stop 2023-10-18T09:04:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.948+0000 7f505dfff640 1 -- 172.21.15.40:0/524888554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50580c99c0 msgr2=0x7f5058155fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.948+0000 7f505dfff640 1 --2- 172.21.15.40:0/524888554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50580c99c0 0x7f5058155fd0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f505000b970 tx=0x7f505000be30 comp rx=0 tx=0).stop 2023-10-18T09:04:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.948+0000 7f505dfff640 1 -- 172.21.15.40:0/524888554 shutdown_connections 2023-10-18T09:04:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.948+0000 7f505dfff640 1 --2- 172.21.15.40:0/524888554 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f503c06ae40 0x7f503c06d300 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.948+0000 7f505dfff640 1 --2- 172.21.15.40:0/524888554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50580c99c0 0x7f5058155fd0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.948+0000 7f505dfff640 1 --2- 172.21.15.40:0/524888554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50580b0630 0x7f50580ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.948+0000 7f505dfff640 1 --2- 172.21.15.40:0/524888554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f505806d6f0 0x7f505814bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.948+0000 7f505dfff640 1 -- 172.21.15.40:0/524888554 >> 172.21.15.40:0/524888554 conn(0x7f5058069720 msgr2=0x7f505814da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.949+0000 7f505dfff640 1 -- 172.21.15.40:0/524888554 shutdown_connections 2023-10-18T09:04:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:13.949+0000 7f505dfff640 1 -- 172.21.15.40:0/524888554 wait complete. 2023-10-18T09:04:13.969 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:03:21.754758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:03:22.819486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:04:13.970 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:04:14.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.196+0000 7f01faa4e640 1 -- 172.21.15.40:0/3563179719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01f40ccda0 msgr2=0x7f01f406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:14.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.196+0000 7f01faa4e640 1 --2- 172.21.15.40:0/3563179719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01f40ccda0 0x7f01f406d4b0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f01f409c110 tx=0x7f01e402bcf0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.196+0000 7f01faa4e640 1 -- 172.21.15.40:0/3563179719 shutdown_connections 2023-10-18T09:04:14.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.196+0000 7f01faa4e640 1 --2- 172.21.15.40:0/3563179719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f406d9f0 0x7f01f4070510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.196+0000 7f01faa4e640 1 --2- 172.21.15.40:0/3563179719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01f40ccda0 0x7f01f406d4b0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.196+0000 7f01faa4e640 1 --2- 172.21.15.40:0/3563179719 >> v2:172.21.15.144:3300/0 conn(0x7f01f415b660 0x7f01f40cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.196+0000 7f01faa4e640 1 -- 172.21.15.40:0/3563179719 >> 172.21.15.40:0/3563179719 conn(0x7f01f40cdf70 msgr2=0x7f01f40ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:14.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.196+0000 7f01faa4e640 1 -- 172.21.15.40:0/3563179719 shutdown_connections 2023-10-18T09:04:14.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.196+0000 7f01faa4e640 1 -- 172.21.15.40:0/3563179719 wait complete. 2023-10-18T09:04:14.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.198+0000 7f01faa4e640 1 Processor -- start 2023-10-18T09:04:14.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.198+0000 7f01faa4e640 1 -- start start 2023-10-18T09:04:14.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.200+0000 7f01faa4e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f406d9f0 0x7f01f414dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:14.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.200+0000 7f01faa4e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01f40ccda0 0x7f01f40ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:14.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.200+0000 7f01faa4e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01f40ab7d0 0x7f01f40ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:14.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.200+0000 7f01faa4e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40ae730 con 0x7f01f40ab7d0 2023-10-18T09:04:14.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.200+0000 7f01faa4e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40adaf0 con 0x7f01f40ccda0 2023-10-18T09:04:14.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.200+0000 7f01faa4e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40add90 con 0x7f01f406d9f0 2023-10-18T09:04:14.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.200+0000 7f01fa24d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01f40ab7d0 0x7f01f40ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:14.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.200+0000 7f01fa24d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01f40ab7d0 0x7f01f40ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58030/0 (socket says 172.21.15.40:58030) 2023-10-18T09:04:14.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.200+0000 7f01fa24d640 1 -- 172.21.15.40:0/3273018072 learned_addr learned my addr 172.21.15.40:0/3273018072 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:14.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.200+0000 7f01f924b640 1 --2- 172.21.15.40:0/3273018072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01f40ccda0 0x7f01f40ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:14.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.200+0000 7f01f9a4c640 1 --2- 172.21.15.40:0/3273018072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f406d9f0 0x7f01f414dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:14.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.201+0000 7f01fa24d640 1 -- 172.21.15.40:0/3273018072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f406d9f0 msgr2=0x7f01f414dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:14.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.201+0000 7f01fa24d640 1 --2- 172.21.15.40:0/3273018072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f406d9f0 0x7f01f414dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.201+0000 7f01fa24d640 1 -- 172.21.15.40:0/3273018072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01f40ccda0 msgr2=0x7f01f40ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:14.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.201+0000 7f01fa24d640 1 --2- 172.21.15.40:0/3273018072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01f40ccda0 0x7f01f40ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.201+0000 7f01fa24d640 1 -- 172.21.15.40:0/3273018072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e4009560 con 0x7f01f40ab7d0 2023-10-18T09:04:14.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.201+0000 7f01fa24d640 1 --2- 172.21.15.40:0/3273018072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01f40ab7d0 0x7f01f40ca650 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00d650 tx=0x7f01ec00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:14.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.201+0000 7f01ea7fc640 1 -- 172.21.15.40:0/3273018072 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec005220 con 0x7f01f40ab7d0 2023-10-18T09:04:14.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.201+0000 7f01ea7fc640 1 -- 172.21.15.40:0/3273018072 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f01ec005490 con 0x7f01f40ab7d0 2023-10-18T09:04:14.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.201+0000 7f01ea7fc640 1 -- 172.21.15.40:0/3273018072 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec009a80 con 0x7f01f40ab7d0 2023-10-18T09:04:14.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.201+0000 7f01faa4e640 1 -- 172.21.15.40:0/3273018072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f40ae070 con 0x7f01f40ab7d0 2023-10-18T09:04:14.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.201+0000 7f01faa4e640 1 -- 172.21.15.40:0/3273018072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f414e240 con 0x7f01f40ab7d0 2023-10-18T09:04:14.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.203+0000 7f01faa4e640 1 -- 172.21.15.40:0/3273018072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f40ae870 con 0x7f01f40ab7d0 2023-10-18T09:04:14.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.204+0000 7f01ea7fc640 1 -- 172.21.15.40:0/3273018072 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f01ec009be0 con 0x7f01f40ab7d0 2023-10-18T09:04:14.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.204+0000 7f01ea7fc640 1 --2- 172.21.15.40:0/3273018072 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f01d806ae70 0x7f01d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:14.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.205+0000 7f01ea7fc640 1 -- 172.21.15.40:0/3273018072 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f01ec0897f0 con 0x7f01f40ab7d0 2023-10-18T09:04:14.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.205+0000 7f01f9a4c640 1 --2- 172.21.15.40:0/3273018072 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f01d806ae70 0x7f01d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:14.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.206+0000 7f01f9a4c640 1 --2- 172.21.15.40:0/3273018072 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f01d806ae70 0x7f01d806d330 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f01f0005fd0 tx=0x7f01f00097d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:14.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.209+0000 7f01ea7fc640 1 -- 172.21.15.40:0/3273018072 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f01f40ae870 con 0x7f01f40ab7d0 2023-10-18T09:04:14.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.399+0000 7f01faa4e640 1 -- 172.21.15.40:0/3273018072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01f40ae870 con 0x7f01f40ab7d0 2023-10-18T09:04:14.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.400+0000 7f01ea7fc640 1 -- 172.21.15.40:0/3273018072 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v149) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01f40ae870 con 0x7f01f40ab7d0 2023-10-18T09:04:14.403 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:04:14.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.402+0000 7f01faa4e640 1 -- 172.21.15.40:0/3273018072 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f01d806ae70 msgr2=0x7f01d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:14.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.402+0000 7f01faa4e640 1 --2- 172.21.15.40:0/3273018072 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f01d806ae70 0x7f01d806d330 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f01f0005fd0 tx=0x7f01f00097d0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.402+0000 7f01faa4e640 1 -- 172.21.15.40:0/3273018072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01f40ab7d0 msgr2=0x7f01f40ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.402+0000 7f01faa4e640 1 --2- 172.21.15.40:0/3273018072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01f40ab7d0 0x7f01f40ca650 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00d650 tx=0x7f01ec00db10 comp rx=0 tx=0).stop 2023-10-18T09:04:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.403+0000 7f01faa4e640 1 -- 172.21.15.40:0/3273018072 shutdown_connections 2023-10-18T09:04:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.403+0000 7f01faa4e640 1 --2- 172.21.15.40:0/3273018072 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f01d806ae70 0x7f01d806d330 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.403+0000 7f01faa4e640 1 --2- 172.21.15.40:0/3273018072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01f40ab7d0 0x7f01f40ca650 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.403+0000 7f01faa4e640 1 --2- 172.21.15.40:0/3273018072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01f40ccda0 0x7f01f40ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.403+0000 7f01faa4e640 1 --2- 172.21.15.40:0/3273018072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01f406d9f0 0x7f01f414dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.403+0000 7f01faa4e640 1 -- 172.21.15.40:0/3273018072 >> 172.21.15.40:0/3273018072 conn(0x7f01f40cdf70 msgr2=0x7f01f40b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.403+0000 7f01faa4e640 1 -- 172.21.15.40:0/3273018072 shutdown_connections 2023-10-18T09:04:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.403+0000 7f01faa4e640 1 -- 172.21.15.40:0/3273018072 wait complete. 2023-10-18T09:04:14.424 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:04:14.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.657+0000 7f4cdb154640 1 -- 172.21.15.40:0/1556585017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd406d5d0 msgr2=0x7f4cd40c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:14.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.657+0000 7f4cdb154640 1 --2- 172.21.15.40:0/1556585017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd406d5d0 0x7f4cd40c9620 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000a940 tx=0x7f4cd002d000 comp rx=0 tx=0).stop 2023-10-18T09:04:14.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.657+0000 7f4cdb154640 1 -- 172.21.15.40:0/1556585017 shutdown_connections 2023-10-18T09:04:14.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.657+0000 7f4cdb154640 1 --2- 172.21.15.40:0/1556585017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd406d5d0 0x7f4cd40c9620 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.657+0000 7f4cdb154640 1 -- 172.21.15.40:0/1556585017 >> 172.21.15.40:0/1556585017 conn(0x7f4cd406ca10 msgr2=0x7f4cd406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:14.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.657+0000 7f4cdb154640 1 -- 172.21.15.40:0/1556585017 shutdown_connections 2023-10-18T09:04:14.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.657+0000 7f4cdb154640 1 -- 172.21.15.40:0/1556585017 wait complete. 2023-10-18T09:04:14.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.659+0000 7f4cdb154640 1 Processor -- start 2023-10-18T09:04:14.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.659+0000 7f4cdb154640 1 -- start start 2023-10-18T09:04:14.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.660+0000 7f4cdb154640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cd406d5d0 0x7f4cd41e18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:14.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.660+0000 7f4cdb154640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd4071270 0x7f4cd413f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:14.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.660+0000 7f4cdb154640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd415af70 0x7f4cd41ef160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:14.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.660+0000 7f4cdb154640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd413d790 con 0x7f4cd4071270 2023-10-18T09:04:14.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.660+0000 7f4cdb154640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd41402e0 con 0x7f4cd406d5d0 2023-10-18T09:04:14.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.660+0000 7f4cdb154640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd41405c0 con 0x7f4cd415af70 2023-10-18T09:04:14.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.660+0000 7f4cd9951640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd4071270 0x7f4cd413f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:14.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.660+0000 7f4cd9951640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd4071270 0x7f4cd413f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58054/0 (socket says 172.21.15.40:58054) 2023-10-18T09:04:14.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.660+0000 7f4cd9951640 1 -- 172.21.15.40:0/512668746 learned_addr learned my addr 172.21.15.40:0/512668746 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:14.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.661+0000 7f4cda152640 1 --2- 172.21.15.40:0/512668746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cd406d5d0 0x7f4cd41e18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:14.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.661+0000 7f4cda953640 1 --2- 172.21.15.40:0/512668746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd415af70 0x7f4cd41ef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:14.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.661+0000 7f4cd9951640 1 -- 172.21.15.40:0/512668746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd415af70 msgr2=0x7f4cd41ef160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:14.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.661+0000 7f4cd9951640 1 --2- 172.21.15.40:0/512668746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd415af70 0x7f4cd41ef160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.661+0000 7f4cd9951640 1 -- 172.21.15.40:0/512668746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cd406d5d0 msgr2=0x7f4cd41e18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:14.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.661+0000 7f4cd9951640 1 --2- 172.21.15.40:0/512668746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cd406d5d0 0x7f4cd41e18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.661+0000 7f4cd9951640 1 -- 172.21.15.40:0/512668746 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd000a5a0 con 0x7f4cd4071270 2023-10-18T09:04:14.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.661+0000 7f4cda953640 1 --2- 172.21.15.40:0/512668746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd415af70 0x7f4cd41ef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:14.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.661+0000 7f4cd9951640 1 --2- 172.21.15.40:0/512668746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd4071270 0x7f4cd413f820 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400c3c0 tx=0x7f4cc400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:14.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.661+0000 7f4ccaffd640 1 -- 172.21.15.40:0/512668746 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4011790 con 0x7f4cd4071270 2023-10-18T09:04:14.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.662+0000 7f4ccaffd640 1 -- 172.21.15.40:0/512668746 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4cc40119e0 con 0x7f4cd4071270 2023-10-18T09:04:14.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.662+0000 7f4ccaffd640 1 -- 172.21.15.40:0/512668746 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4011e10 con 0x7f4cd4071270 2023-10-18T09:04:14.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.662+0000 7f4cdb154640 1 -- 172.21.15.40:0/512668746 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd413d9a0 con 0x7f4cd4071270 2023-10-18T09:04:14.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.662+0000 7f4cdb154640 1 -- 172.21.15.40:0/512668746 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd4143530 con 0x7f4cd4071270 2023-10-18T09:04:14.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.663+0000 7f4cdb154640 1 -- 172.21.15.40:0/512668746 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd413dec0 con 0x7f4cd4071270 2023-10-18T09:04:14.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.664+0000 7f4ccaffd640 1 -- 172.21.15.40:0/512668746 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4cc401c460 con 0x7f4cd4071270 2023-10-18T09:04:14.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.665+0000 7f4ccaffd640 1 --2- 172.21.15.40:0/512668746 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4cb806ae70 0x7f4cb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:14.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.665+0000 7f4ccaffd640 1 -- 172.21.15.40:0/512668746 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4cc4089630 con 0x7f4cd4071270 2023-10-18T09:04:14.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.665+0000 7f4cda152640 1 --2- 172.21.15.40:0/512668746 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4cb806ae70 0x7f4cb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:14.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.666+0000 7f4cda152640 1 --2- 172.21.15.40:0/512668746 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4cb806ae70 0x7f4cb806d330 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0037000 tx=0x7f4cd0004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:14.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.668+0000 7f4ccaffd640 1 -- 172.21.15.40:0/512668746 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4cd413dec0 con 0x7f4cd4071270 2023-10-18T09:04:14.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.861+0000 7f4cdb154640 1 -- 172.21.15.40:0/512668746 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4cd40c9b60 con 0x7f4cd4071270 2023-10-18T09:04:14.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.862+0000 7f4ccaffd640 1 -- 172.21.15.40:0/512668746 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v149) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4cd40c9b60 con 0x7f4cd4071270 2023-10-18T09:04:14.864 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:04:14.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.865+0000 7f4cdb154640 1 -- 172.21.15.40:0/512668746 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4cb806ae70 msgr2=0x7f4cb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:14.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.865+0000 7f4cdb154640 1 --2- 172.21.15.40:0/512668746 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4cb806ae70 0x7f4cb806d330 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0037000 tx=0x7f4cd0004730 comp rx=0 tx=0).stop 2023-10-18T09:04:14.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.865+0000 7f4cdb154640 1 -- 172.21.15.40:0/512668746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd4071270 msgr2=0x7f4cd413f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:14.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.865+0000 7f4cdb154640 1 --2- 172.21.15.40:0/512668746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd4071270 0x7f4cd413f820 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400c3c0 tx=0x7f4cc400c880 comp rx=0 tx=0).stop 2023-10-18T09:04:14.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.865+0000 7f4cdb154640 1 -- 172.21.15.40:0/512668746 shutdown_connections 2023-10-18T09:04:14.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.865+0000 7f4cdb154640 1 --2- 172.21.15.40:0/512668746 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4cb806ae70 0x7f4cb806d330 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.865+0000 7f4cdb154640 1 --2- 172.21.15.40:0/512668746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4cd415af70 0x7f4cd41ef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.865+0000 7f4cdb154640 1 --2- 172.21.15.40:0/512668746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4cd4071270 0x7f4cd413f820 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.865+0000 7f4cdb154640 1 --2- 172.21.15.40:0/512668746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cd406d5d0 0x7f4cd41e18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:14.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.865+0000 7f4cdb154640 1 -- 172.21.15.40:0/512668746 >> 172.21.15.40:0/512668746 conn(0x7f4cd406ca10 msgr2=0x7f4cd4070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:14.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.865+0000 7f4cdb154640 1 -- 172.21.15.40:0/512668746 shutdown_connections 2023-10-18T09:04:14.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:14.865+0000 7f4cdb154640 1 -- 172.21.15.40:0/512668746 wait complete. 2023-10-18T09:04:14.885 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:04:15.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.112+0000 7f52d8ac3640 1 -- 172.21.15.40:0/2134637268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52d415eac0 msgr2=0x7f52d40cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:15.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.112+0000 7f52d8ac3640 1 --2- 172.21.15.40:0/2134637268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52d415eac0 0x7f52d40cff70 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f52d4069c40 tx=0x7f52c402bd70 comp rx=0 tx=0).stop 2023-10-18T09:04:15.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.112+0000 7f52d8ac3640 1 -- 172.21.15.40:0/2134637268 shutdown_connections 2023-10-18T09:04:15.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.112+0000 7f52d8ac3640 1 --2- 172.21.15.40:0/2134637268 >> v2:172.21.15.144:3300/0 conn(0x7f52d415eea0 0x7f52d40b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:15.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.112+0000 7f52d8ac3640 1 --2- 172.21.15.40:0/2134637268 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52d415eac0 0x7f52d40cff70 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:15.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.112+0000 7f52d8ac3640 1 -- 172.21.15.40:0/2134637268 >> 172.21.15.40:0/2134637268 conn(0x7f52d409a610 msgr2=0x7f52d40d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:15.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.113+0000 7f52d8ac3640 1 -- 172.21.15.40:0/2134637268 shutdown_connections 2023-10-18T09:04:15.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.113+0000 7f52d8ac3640 1 -- 172.21.15.40:0/2134637268 wait complete. 2023-10-18T09:04:15.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.114+0000 7f52d8ac3640 1 Processor -- start 2023-10-18T09:04:15.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.114+0000 7f52d8ac3640 1 -- start start 2023-10-18T09:04:15.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.115+0000 7f52d8ac3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52d40d19b0 0x7f52d4158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:15.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.115+0000 7f52d8ac3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52d415eac0 0x7f52d40cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:15.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.115+0000 7f52d8ac3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52d415eea0 0x7f52d40b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:15.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.115+0000 7f52d8ac3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d40cb170 con 0x7f52d415eea0 2023-10-18T09:04:15.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.115+0000 7f52d8ac3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d415bf30 con 0x7f52d415eac0 2023-10-18T09:04:15.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.115+0000 7f52d8ac3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d415c210 con 0x7f52d40d19b0 2023-10-18T09:04:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.116+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52d415eea0 0x7f52d40b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.116+0000 7f52d3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52d415eea0 0x7f52d40b6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55820/0 (socket says 172.21.15.40:55820) 2023-10-18T09:04:15.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.116+0000 7f52d3fff640 1 -- 172.21.15.40:0/2363546860 learned_addr learned my addr 172.21.15.40:0/2363546860 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.116+0000 7f52cbfff640 1 --2- 172.21.15.40:0/2363546860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52d415eac0 0x7f52d40cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.116+0000 7f52d3fff640 1 -- 172.21.15.40:0/2363546860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52d40d19b0 msgr2=0x7f52d4158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.116+0000 7f52d37fe640 1 --2- 172.21.15.40:0/2363546860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52d40d19b0 0x7f52d4158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.116+0000 7f52d3fff640 1 --2- 172.21.15.40:0/2363546860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52d40d19b0 0x7f52d4158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:15.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.116+0000 7f52d3fff640 1 -- 172.21.15.40:0/2363546860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52d415eac0 msgr2=0x7f52d40cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.116+0000 7f52d3fff640 1 --2- 172.21.15.40:0/2363546860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52d415eac0 0x7f52d40cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.116+0000 7f52d3fff640 1 -- 172.21.15.40:0/2363546860 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c4009490 con 0x7f52d415eea0 2023-10-18T09:04:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.116+0000 7f52d37fe640 1 --2- 172.21.15.40:0/2363546860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52d40d19b0 0x7f52d4158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.116+0000 7f52cbfff640 1 --2- 172.21.15.40:0/2363546860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52d415eac0 0x7f52d40cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:04:15.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.117+0000 7f52d3fff640 1 --2- 172.21.15.40:0/2363546860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52d415eea0 0x7f52d40b6000 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00c760 tx=0x7f52cc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:15.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.119+0000 7f52d0ff9640 1 -- 172.21.15.40:0/2363546860 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc003da0 con 0x7f52d415eea0 2023-10-18T09:04:15.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.119+0000 7f52d0ff9640 1 -- 172.21.15.40:0/2363546860 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f52cc00ce70 con 0x7f52d415eea0 2023-10-18T09:04:15.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.119+0000 7f52d0ff9640 1 -- 172.21.15.40:0/2363546860 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc0052b0 con 0x7f52d415eea0 2023-10-18T09:04:15.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.119+0000 7f52d8ac3640 1 -- 172.21.15.40:0/2363546860 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d415c4f0 con 0x7f52d415eea0 2023-10-18T09:04:15.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.119+0000 7f52d8ac3640 1 -- 172.21.15.40:0/2363546860 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d415b970 con 0x7f52d415eea0 2023-10-18T09:04:15.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.120+0000 7f52d8ac3640 1 -- 172.21.15.40:0/2363546860 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d415ced0 con 0x7f52d415eea0 2023-10-18T09:04:15.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.121+0000 7f52d0ff9640 1 -- 172.21.15.40:0/2363546860 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f52cc005410 con 0x7f52d415eea0 2023-10-18T09:04:15.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.122+0000 7f52d0ff9640 1 --2- 172.21.15.40:0/2363546860 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f52b406ae40 0x7f52b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:15.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.122+0000 7f52d0ff9640 1 -- 172.21.15.40:0/2363546860 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f52cc089680 con 0x7f52d415eea0 2023-10-18T09:04:15.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.122+0000 7f52d37fe640 1 --2- 172.21.15.40:0/2363546860 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f52b406ae40 0x7f52b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:15.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.124+0000 7f52d37fe640 1 --2- 172.21.15.40:0/2363546860 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f52b406ae40 0x7f52b406d300 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f52c4036000 tx=0x7f52c4004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:15.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.127+0000 7f52d0ff9640 1 -- 172.21.15.40:0/2363546860 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f52cc0546e0 con 0x7f52d415eea0 2023-10-18T09:04:15.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.320+0000 7f52d8ac3640 1 -- 172.21.15.40:0/2363546860 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f52d4071da0 con 0x7f52d415eea0 2023-10-18T09:04:15.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.321+0000 7f52d0ff9640 1 -- 172.21.15.40:0/2363546860 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v149) v1 ==== 100+0+11 (secure 0 0 0) 0x7f52cc057d00 con 0x7f52d415eea0 2023-10-18T09:04:15.324 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:04:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.324+0000 7f52d8ac3640 1 -- 172.21.15.40:0/2363546860 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f52b406ae40 msgr2=0x7f52b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:15.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.324+0000 7f52d8ac3640 1 --2- 172.21.15.40:0/2363546860 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f52b406ae40 0x7f52b406d300 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f52c4036000 tx=0x7f52c4004730 comp rx=0 tx=0).stop 2023-10-18T09:04:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.324+0000 7f52d8ac3640 1 -- 172.21.15.40:0/2363546860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52d415eea0 msgr2=0x7f52d40b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.324+0000 7f52d8ac3640 1 --2- 172.21.15.40:0/2363546860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52d415eea0 0x7f52d40b6000 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00c760 tx=0x7f52cc00cc20 comp rx=0 tx=0).stop 2023-10-18T09:04:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.324+0000 7f52d8ac3640 1 -- 172.21.15.40:0/2363546860 shutdown_connections 2023-10-18T09:04:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.324+0000 7f52d8ac3640 1 --2- 172.21.15.40:0/2363546860 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f52b406ae40 0x7f52b406d300 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:15.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.324+0000 7f52d8ac3640 1 --2- 172.21.15.40:0/2363546860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52d415eea0 0x7f52d40b6000 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.324+0000 7f52d8ac3640 1 --2- 172.21.15.40:0/2363546860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52d415eac0 0x7f52d40cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.324+0000 7f52d8ac3640 1 --2- 172.21.15.40:0/2363546860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52d40d19b0 0x7f52d4158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.324+0000 7f52d8ac3640 1 -- 172.21.15.40:0/2363546860 >> 172.21.15.40:0/2363546860 conn(0x7f52d409a610 msgr2=0x7f52d415a750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.324+0000 7f52d8ac3640 1 -- 172.21.15.40:0/2363546860 shutdown_connections 2023-10-18T09:04:15.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.324+0000 7f52d8ac3640 1 -- 172.21.15.40:0/2363546860 wait complete. 2023-10-18T09:04:15.346 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:04:15.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.576+0000 7f19debb8640 1 -- 172.21.15.40:0/1514182575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d806d6f0 msgr2=0x7f19d80b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:15.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.576+0000 7f19debb8640 1 --2- 172.21.15.40:0/1514182575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d806d6f0 0x7f19d80b15b0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f19d806b050 tx=0x7f19d402bd70 comp rx=0 tx=0).stop 2023-10-18T09:04:15.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.576+0000 7f19debb8640 1 -- 172.21.15.40:0/1514182575 shutdown_connections 2023-10-18T09:04:15.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.576+0000 7f19debb8640 1 --2- 172.21.15.40:0/1514182575 >> v2:172.21.15.144:3300/0 conn(0x7f19d80b0630 0x7f19d80c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:15.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.576+0000 7f19debb8640 1 --2- 172.21.15.40:0/1514182575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d806d6f0 0x7f19d80b15b0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:15.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.576+0000 7f19debb8640 1 -- 172.21.15.40:0/1514182575 >> 172.21.15.40:0/1514182575 conn(0x7f19d8069720 msgr2=0x7f19d8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:15.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.576+0000 7f19debb8640 1 -- 172.21.15.40:0/1514182575 shutdown_connections 2023-10-18T09:04:15.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.576+0000 7f19debb8640 1 -- 172.21.15.40:0/1514182575 wait complete. 2023-10-18T09:04:15.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.578+0000 7f19debb8640 1 Processor -- start 2023-10-18T09:04:15.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.578+0000 7f19debb8640 1 -- start start 2023-10-18T09:04:15.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.579+0000 7f19debb8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19d806d6f0 0x7f19d80af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:15.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.579+0000 7f19debb8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19d80b0630 0x7f19d814fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:15.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.579+0000 7f19debb8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d80c9f00 0x7f19d8150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:15.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.579+0000 7f19debb8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d8159dd0 con 0x7f19d80c9f00 2023-10-18T09:04:15.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.579+0000 7f19debb8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d8158ae0 con 0x7f19d80b0630 2023-10-18T09:04:15.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.579+0000 7f19debb8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80afb20 con 0x7f19d806d6f0 2023-10-18T09:04:15.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.579+0000 7f19de3b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d80c9f00 0x7f19d8150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:15.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.579+0000 7f19de3b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d80c9f00 0x7f19d8150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55844/0 (socket says 172.21.15.40:55844) 2023-10-18T09:04:15.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.579+0000 7f19de3b7640 1 -- 172.21.15.40:0/4008290161 learned_addr learned my addr 172.21.15.40:0/4008290161 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:15.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.580+0000 7f19dd3b5640 1 --2- 172.21.15.40:0/4008290161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19d80b0630 0x7f19d814fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:15.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.580+0000 7f19ddbb6640 1 --2- 172.21.15.40:0/4008290161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19d806d6f0 0x7f19d80af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:15.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.580+0000 7f19de3b7640 1 -- 172.21.15.40:0/4008290161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19d806d6f0 msgr2=0x7f19d80af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:15.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.580+0000 7f19de3b7640 1 --2- 172.21.15.40:0/4008290161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19d806d6f0 0x7f19d80af5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:15.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.580+0000 7f19de3b7640 1 -- 172.21.15.40:0/4008290161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19d80b0630 msgr2=0x7f19d814fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:15.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.580+0000 7f19de3b7640 1 --2- 172.21.15.40:0/4008290161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19d80b0630 0x7f19d814fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:15.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.580+0000 7f19de3b7640 1 -- 172.21.15.40:0/4008290161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d4009490 con 0x7f19d80c9f00 2023-10-18T09:04:15.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.580+0000 7f19de3b7640 1 --2- 172.21.15.40:0/4008290161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d80c9f00 0x7f19d8150010 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f19d000ca80 tx=0x7f19d000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:15.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.581+0000 7f19ce7fc640 1 -- 172.21.15.40:0/4008290161 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d000ac30 con 0x7f19d80c9f00 2023-10-18T09:04:15.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.581+0000 7f19ce7fc640 1 -- 172.21.15.40:0/4008290161 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f19d000ae80 con 0x7f19d80c9f00 2023-10-18T09:04:15.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.581+0000 7f19debb8640 1 -- 172.21.15.40:0/4008290161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d814bd50 con 0x7f19d80c9f00 2023-10-18T09:04:15.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.581+0000 7f19ce7fc640 1 -- 172.21.15.40:0/4008290161 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0012680 con 0x7f19d80c9f00 2023-10-18T09:04:15.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.581+0000 7f19debb8640 1 -- 172.21.15.40:0/4008290161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d8159010 con 0x7f19d80c9f00 2023-10-18T09:04:15.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.582+0000 7f19debb8640 1 -- 172.21.15.40:0/4008290161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d8066e00 con 0x7f19d80c9f00 2023-10-18T09:04:15.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.583+0000 7f19ce7fc640 1 -- 172.21.15.40:0/4008290161 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f19d001b400 con 0x7f19d80c9f00 2023-10-18T09:04:15.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.584+0000 7f19ce7fc640 1 --2- 172.21.15.40:0/4008290161 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f19bc06ae70 0x7f19bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:15.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.585+0000 7f19ce7fc640 1 -- 172.21.15.40:0/4008290161 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f19d008a150 con 0x7f19d80c9f00 2023-10-18T09:04:15.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.585+0000 7f19ddbb6640 1 --2- 172.21.15.40:0/4008290161 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f19bc06ae70 0x7f19bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:15.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.586+0000 7f19ddbb6640 1 --2- 172.21.15.40:0/4008290161 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f19bc06ae70 0x7f19bc06d330 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f19d4036000 tx=0x7f19d4004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:15.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.589+0000 7f19ce7fc640 1 -- 172.21.15.40:0/4008290161 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f19d0055090 con 0x7f19d80c9f00 2023-10-18T09:04:15.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.798+0000 7f19debb8640 1 -- 172.21.15.40:0/4008290161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f19d8156020 con 0x7f19d80c9f00 2023-10-18T09:04:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.799+0000 7f19ce7fc640 1 -- 172.21.15.40:0/4008290161 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v149) v1 ==== 80+0+0 (secure 0 0 0) 0x7f19d00586b0 con 0x7f19d80c9f00 2023-10-18T09:04:15.801 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T09:04:15.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.802+0000 7f19debb8640 1 -- 172.21.15.40:0/4008290161 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f19bc06ae70 msgr2=0x7f19bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:15.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.802+0000 7f19debb8640 1 --2- 172.21.15.40:0/4008290161 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f19bc06ae70 0x7f19bc06d330 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f19d4036000 tx=0x7f19d4004730 comp rx=0 tx=0).stop 2023-10-18T09:04:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.802+0000 7f19debb8640 1 -- 172.21.15.40:0/4008290161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d80c9f00 msgr2=0x7f19d8150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.802+0000 7f19debb8640 1 --2- 172.21.15.40:0/4008290161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d80c9f00 0x7f19d8150010 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f19d000ca80 tx=0x7f19d000cf40 comp rx=0 tx=0).stop 2023-10-18T09:04:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.802+0000 7f19debb8640 1 -- 172.21.15.40:0/4008290161 shutdown_connections 2023-10-18T09:04:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.802+0000 7f19debb8640 1 --2- 172.21.15.40:0/4008290161 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f19bc06ae70 0x7f19bc06d330 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:15.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.802+0000 7f19debb8640 1 --2- 172.21.15.40:0/4008290161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d80c9f00 0x7f19d8150010 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.802+0000 7f19debb8640 1 --2- 172.21.15.40:0/4008290161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19d80b0630 0x7f19d814fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.802+0000 7f19debb8640 1 --2- 172.21.15.40:0/4008290161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19d806d6f0 0x7f19d80af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.802+0000 7f19debb8640 1 -- 172.21.15.40:0/4008290161 >> 172.21.15.40:0/4008290161 conn(0x7f19d8069720 msgr2=0x7f19d8072d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.802+0000 7f19debb8640 1 -- 172.21.15.40:0/4008290161 shutdown_connections 2023-10-18T09:04:15.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:15.802+0000 7f19debb8640 1 -- 172.21.15.40:0/4008290161 wait complete. 2023-10-18T09:04:15.824 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-18T09:04:15.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:15.825 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-18T09:04:15.938 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T09:04:15.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:15.939 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T09:04:15.992 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:04:15.993 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:04:16.056 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:04:16.057 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:04:16.057 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:04:16.117 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:04:16.118 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:04:16.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.345+0000 7f1ab5593640 1 -- 172.21.15.40:0/955841214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab00b15b0 msgr2=0x7f1ab00b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:16.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.345+0000 7f1ab5593640 1 --2- 172.21.15.40:0/955841214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab00b15b0 0x7f1ab00b0630 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f1ab006b050 tx=0x7f1aa002c990 comp rx=0 tx=0).stop 2023-10-18T09:04:16.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.345+0000 7f1ab5593640 1 -- 172.21.15.40:0/955841214 shutdown_connections 2023-10-18T09:04:16.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.345+0000 7f1ab5593640 1 --2- 172.21.15.40:0/955841214 >> v2:172.21.15.144:3300/0 conn(0x7f1ab00b1e10 0x7f1ab00c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:16.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.345+0000 7f1ab5593640 1 --2- 172.21.15.40:0/955841214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab00b15b0 0x7f1ab00b0630 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:16.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.345+0000 7f1ab5593640 1 -- 172.21.15.40:0/955841214 >> 172.21.15.40:0/955841214 conn(0x7f1ab0069720 msgr2=0x7f1ab0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:16.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.345+0000 7f1ab5593640 1 -- 172.21.15.40:0/955841214 shutdown_connections 2023-10-18T09:04:16.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.345+0000 7f1ab5593640 1 -- 172.21.15.40:0/955841214 wait complete. 2023-10-18T09:04:16.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.347+0000 7f1ab5593640 1 Processor -- start 2023-10-18T09:04:16.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.347+0000 7f1ab5593640 1 -- start start 2023-10-18T09:04:16.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.348+0000 7f1ab5593640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ab00b15b0 0x7f1ab0147b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:16.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.348+0000 7f1ab5593640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab00b1e10 0x7f1ab0154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:16.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.348+0000 7f1ab5593640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ab00b2270 0x7f1ab01391b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:16.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.348+0000 7f1ab5593640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab0154fa0 con 0x7f1ab00b1e10 2023-10-18T09:04:16.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.348+0000 7f1ab5593640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab0139db0 con 0x7f1ab00b2270 2023-10-18T09:04:16.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.348+0000 7f1ab5593640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab01549e0 con 0x7f1ab00b15b0 2023-10-18T09:04:16.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.348+0000 7f1aaf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab00b1e10 0x7f1ab0154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:16.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.348+0000 7f1aaf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab00b1e10 0x7f1ab0154300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55864/0 (socket says 172.21.15.40:55864) 2023-10-18T09:04:16.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.348+0000 7f1aaf7fe640 1 -- 172.21.15.40:0/2062767708 learned_addr learned my addr 172.21.15.40:0/2062767708 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:16.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.349+0000 7f1aaf7fe640 1 -- 172.21.15.40:0/2062767708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ab00b15b0 msgr2=0x7f1ab0147b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:16.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.349+0000 7f1aaf7fe640 1 --2- 172.21.15.40:0/2062767708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ab00b15b0 0x7f1ab0147b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:16.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.349+0000 7f1aaf7fe640 1 -- 172.21.15.40:0/2062767708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ab00b2270 msgr2=0x7f1ab01391b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:16.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.349+0000 7f1ab4d92640 1 --2- 172.21.15.40:0/2062767708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ab00b2270 0x7f1ab01391b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:16.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.349+0000 7f1aaf7fe640 1 --2- 172.21.15.40:0/2062767708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ab00b2270 0x7f1ab01391b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:16.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.349+0000 7f1aaf7fe640 1 -- 172.21.15.40:0/2062767708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa000b340 con 0x7f1ab00b1e10 2023-10-18T09:04:16.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.349+0000 7f1ab4d92640 1 --2- 172.21.15.40:0/2062767708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ab00b2270 0x7f1ab01391b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.349+0000 7f1aaf7fe640 1 --2- 172.21.15.40:0/2062767708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab00b1e10 0x7f1ab0154300 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400c920 tx=0x7f1aa400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.350+0000 7f1aacff9640 1 -- 172.21.15.40:0/2062767708 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4007c70 con 0x7f1ab00b1e10 2023-10-18T09:04:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.350+0000 7f1aacff9640 1 -- 172.21.15.40:0/2062767708 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1aa400f450 con 0x7f1ab00b1e10 2023-10-18T09:04:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.350+0000 7f1ab5593640 1 -- 172.21.15.40:0/2062767708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab0154cc0 con 0x7f1ab00b1e10 2023-10-18T09:04:16.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.350+0000 7f1aacff9640 1 -- 172.21.15.40:0/2062767708 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa400f860 con 0x7f1ab00b1e10 2023-10-18T09:04:16.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.350+0000 7f1ab5593640 1 -- 172.21.15.40:0/2062767708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab01f1ed0 con 0x7f1ab00b1e10 2023-10-18T09:04:16.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.351+0000 7f1ab5593640 1 -- 172.21.15.40:0/2062767708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab01f1a30 con 0x7f1ab00b1e10 2023-10-18T09:04:16.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.352+0000 7f1aacff9640 1 -- 172.21.15.40:0/2062767708 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1aa400f9c0 con 0x7f1ab00b1e10 2023-10-18T09:04:16.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.353+0000 7f1aacff9640 1 --2- 172.21.15.40:0/2062767708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1a9c06ae40 0x7f1a9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:16.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.353+0000 7f1aacff9640 1 -- 172.21.15.40:0/2062767708 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1aa4089f90 con 0x7f1ab00b1e10 2023-10-18T09:04:16.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.353+0000 7f1aacff9640 1 -- 172.21.15.40:0/2062767708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f1a9c072a30 con 0x7f1ab00b1e10 2023-10-18T09:04:16.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.354+0000 7f1aaffff640 1 --2- 172.21.15.40:0/2062767708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1a9c06ae40 0x7f1a9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:16.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.355+0000 7f1aaffff640 1 --2- 172.21.15.40:0/2062767708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1a9c06ae40 0x7f1a9c06d300 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0035000 tx=0x7f1aa0004a40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:16.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.358+0000 7f1aacff9640 1 -- 172.21.15.40:0/2062767708 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1aa401d060 con 0x7f1ab00b1e10 2023-10-18T09:04:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.589+0000 7f1ab5593640 1 -- 172.21.15.40:0/2062767708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ab0066e00 con 0x7f1ab00b1e10 2023-10-18T09:04:16.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.590+0000 7f1aacff9640 1 -- 172.21.15.40:0/2062767708 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f1aa4054f60 con 0x7f1ab00b1e10 2023-10-18T09:04:16.593 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:16.593 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6382,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/4165164542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":4165164542},{"type":"v1","addr":"172.21.15.144:6836","nonce":4165164542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3045960359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3045960359},{"type":"v1","addr":"172.21.15.144:6837","nonce":3045960359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2869040025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2869040025},{"type":"v1","addr":"172.21.15.40:6833","nonce":2869040025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:03:23.782442+0000","modified":"2023-10-18T09:03:26.897709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7893},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7893":{"gid":7893,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2600609919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2600609919},{"type":"v1","addr":"172.21.15.40:6835","nonce":2600609919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-18T09:04:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 66 2023-10-18T09:04:16.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.593+0000 7f1ab5593640 1 -- 172.21.15.40:0/2062767708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1a9c06ae40 msgr2=0x7f1a9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:16.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.593+0000 7f1ab5593640 1 --2- 172.21.15.40:0/2062767708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1a9c06ae40 0x7f1a9c06d300 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0035000 tx=0x7f1aa0004a40 comp rx=0 tx=0).stop 2023-10-18T09:04:16.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.593+0000 7f1ab5593640 1 -- 172.21.15.40:0/2062767708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab00b1e10 msgr2=0x7f1ab0154300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:16.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.593+0000 7f1ab5593640 1 --2- 172.21.15.40:0/2062767708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab00b1e10 0x7f1ab0154300 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400c920 tx=0x7f1aa400cde0 comp rx=0 tx=0).stop 2023-10-18T09:04:16.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.593+0000 7f1ab5593640 1 -- 172.21.15.40:0/2062767708 shutdown_connections 2023-10-18T09:04:16.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.593+0000 7f1ab5593640 1 --2- 172.21.15.40:0/2062767708 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1a9c06ae40 0x7f1a9c06d300 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:16.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.593+0000 7f1ab5593640 1 --2- 172.21.15.40:0/2062767708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ab00b2270 0x7f1ab01391b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:16.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.593+0000 7f1ab5593640 1 --2- 172.21.15.40:0/2062767708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab00b1e10 0x7f1ab0154300 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:16.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.593+0000 7f1ab5593640 1 --2- 172.21.15.40:0/2062767708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ab00b15b0 0x7f1ab0147b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:16.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.593+0000 7f1ab5593640 1 -- 172.21.15.40:0/2062767708 >> 172.21.15.40:0/2062767708 conn(0x7f1ab0069720 msgr2=0x7f1ab00abb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:16.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.593+0000 7f1ab5593640 1 -- 172.21.15.40:0/2062767708 shutdown_connections 2023-10-18T09:04:16.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.594+0000 7f1ab5593640 1 -- 172.21.15.40:0/2062767708 wait complete. 2023-10-18T09:04:16.616 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:04:16.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.842+0000 7f8bb3a64640 1 -- 172.21.15.40:0/629501509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b98004a80 msgr2=0x7f8b98004ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:16.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.842+0000 7f8bb3a64640 1 --2- 172.21.15.40:0/629501509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b98004a80 0x7f8b98004ea0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009b50 tx=0x7f8b98006900 comp rx=0 tx=0).stop 2023-10-18T09:04:16.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.842+0000 7f8bb3a64640 1 -- 172.21.15.40:0/629501509 shutdown_connections 2023-10-18T09:04:16.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.842+0000 7f8bb3a64640 1 --2- 172.21.15.40:0/629501509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b98004a80 0x7f8b98004ea0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:16.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.842+0000 7f8bb3a64640 1 -- 172.21.15.40:0/629501509 >> 172.21.15.40:0/629501509 conn(0x7f8bac06cee0 msgr2=0x7f8bac069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:16.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.842+0000 7f8bb3a64640 1 -- 172.21.15.40:0/629501509 shutdown_connections 2023-10-18T09:04:16.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.842+0000 7f8bb3a64640 1 -- 172.21.15.40:0/629501509 wait complete. 2023-10-18T09:04:16.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.844+0000 7f8bb3a64640 1 Processor -- start 2023-10-18T09:04:16.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.844+0000 7f8bb3a64640 1 -- start start 2023-10-18T09:04:16.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.845+0000 7f8bb3a64640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b98004a80 0x7f8bac149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:16.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.845+0000 7f8bb3a64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0ac100 0x7f8bac0c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:16.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.845+0000 7f8bb3a64640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bac0ca500 0x7f8bac0afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:16.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.845+0000 7f8bb3a64640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac0cb220 con 0x7f8bac0ac100 2023-10-18T09:04:16.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.845+0000 7f8bb3a64640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac14b640 con 0x7f8bac0ca500 2023-10-18T09:04:16.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.845+0000 7f8bb3a64640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac0ca980 con 0x7f8b98004a80 2023-10-18T09:04:16.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.845+0000 7f8bb0fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0ac100 0x7f8bac0c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:16.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.846+0000 7f8bb0fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0ac100 0x7f8bac0c9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55890/0 (socket says 172.21.15.40:55890) 2023-10-18T09:04:16.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.846+0000 7f8bb0fff640 1 -- 172.21.15.40:0/3468351854 learned_addr learned my addr 172.21.15.40:0/3468351854 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:16.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.846+0000 7f8bb1800640 1 --2- 172.21.15.40:0/3468351854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b98004a80 0x7f8bac149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:16.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.846+0000 7f8bb2001640 1 --2- 172.21.15.40:0/3468351854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bac0ca500 0x7f8bac0afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:16.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.846+0000 7f8bb0fff640 1 -- 172.21.15.40:0/3468351854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b98004a80 msgr2=0x7f8bac149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:16.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.846+0000 7f8bb0fff640 1 --2- 172.21.15.40:0/3468351854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b98004a80 0x7f8bac149bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:16.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.846+0000 7f8bb0fff640 1 -- 172.21.15.40:0/3468351854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bac0ca500 msgr2=0x7f8bac0afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:16.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.846+0000 7f8bb0fff640 1 --2- 172.21.15.40:0/3468351854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bac0ca500 0x7f8bac0afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:16.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.846+0000 7f8bb0fff640 1 -- 172.21.15.40:0/3468351854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac0cac60 con 0x7f8bac0ac100 2023-10-18T09:04:16.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.846+0000 7f8bb0fff640 1 --2- 172.21.15.40:0/3468351854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0ac100 0x7f8bac0c9fc0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000d780 tx=0x7f8ba000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:16.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.847+0000 7f8b9dffb640 1 -- 172.21.15.40:0/3468351854 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000de30 con 0x7f8bac0ac100 2023-10-18T09:04:16.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.847+0000 7f8b9dffb640 1 -- 172.21.15.40:0/3468351854 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8ba00052e0 con 0x7f8bac0ac100 2023-10-18T09:04:16.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.847+0000 7f8b9dffb640 1 -- 172.21.15.40:0/3468351854 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000b920 con 0x7f8bac0ac100 2023-10-18T09:04:16.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.847+0000 7f8bb3a64640 1 -- 172.21.15.40:0/3468351854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bac0caf40 con 0x7f8bac0ac100 2023-10-18T09:04:16.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.847+0000 7f8bb3a64640 1 -- 172.21.15.40:0/3468351854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bac0b0350 con 0x7f8bac0ac100 2023-10-18T09:04:16.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.848+0000 7f8bb3a64640 1 -- 172.21.15.40:0/3468351854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bac15c0a0 con 0x7f8bac0ac100 2023-10-18T09:04:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.849+0000 7f8b9dffb640 1 -- 172.21.15.40:0/3468351854 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8ba000ba80 con 0x7f8bac0ac100 2023-10-18T09:04:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.850+0000 7f8b9dffb640 1 --2- 172.21.15.40:0/3468351854 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b9006ae70 0x7f8b9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.850+0000 7f8b9dffb640 1 -- 172.21.15.40:0/3468351854 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8ba0088ef0 con 0x7f8bac0ac100 2023-10-18T09:04:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.850+0000 7f8b9dffb640 1 -- 172.21.15.40:0/3468351854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f8b90072a60 con 0x7f8bac0ac100 2023-10-18T09:04:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.851+0000 7f8bb1800640 1 --2- 172.21.15.40:0/3468351854 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b9006ae70 0x7f8b9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:16.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.852+0000 7f8bb1800640 1 --2- 172.21.15.40:0/3468351854 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b9006ae70 0x7f8b9006d330 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8036000 tx=0x7f8ba80046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:16.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:16.854+0000 7f8b9dffb640 1 -- 172.21.15.40:0/3468351854 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ba0019020 con 0x7f8bac0ac100 2023-10-18T09:04:17.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.054+0000 7f8bb3a64640 1 -- 172.21.15.40:0/3468351854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bac14b7b0 con 0x7f8bac0ac100 2023-10-18T09:04:17.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.056+0000 7f8b9dffb640 1 -- 172.21.15.40:0/3468351854 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16662 (secure 0 0 0) 0x7f8bac15c0a0 con 0x7f8bac0ac100 2023-10-18T09:04:17.059 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:17.059 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":150,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:04:16.086898+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:03:21.754758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-18T09:03:22.819486+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.9","mappings":[{"from":0,"to":2}]},{"pgid":"17.c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:04:17.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.059+0000 7f8bb3a64640 1 -- 172.21.15.40:0/3468351854 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b9006ae70 msgr2=0x7f8b9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:17.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.059+0000 7f8bb3a64640 1 --2- 172.21.15.40:0/3468351854 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b9006ae70 0x7f8b9006d330 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8036000 tx=0x7f8ba80046c0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.059+0000 7f8bb3a64640 1 -- 172.21.15.40:0/3468351854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0ac100 msgr2=0x7f8bac0c9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:17.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.059+0000 7f8bb3a64640 1 --2- 172.21.15.40:0/3468351854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0ac100 0x7f8bac0c9fc0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000d780 tx=0x7f8ba000dc40 comp rx=0 tx=0).stop 2023-10-18T09:04:17.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.059+0000 7f8bb3a64640 1 -- 172.21.15.40:0/3468351854 shutdown_connections 2023-10-18T09:04:17.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.059+0000 7f8bb3a64640 1 --2- 172.21.15.40:0/3468351854 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8b9006ae70 0x7f8b9006d330 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.060+0000 7f8bb3a64640 1 --2- 172.21.15.40:0/3468351854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bac0ca500 0x7f8bac0afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.060+0000 7f8bb3a64640 1 --2- 172.21.15.40:0/3468351854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0ac100 0x7f8bac0c9fc0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.060+0000 7f8bb3a64640 1 --2- 172.21.15.40:0/3468351854 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8b98004a80 0x7f8bac149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.060+0000 7f8bb3a64640 1 -- 172.21.15.40:0/3468351854 >> 172.21.15.40:0/3468351854 conn(0x7f8bac06cee0 msgr2=0x7f8bac0706f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:17.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.060+0000 7f8bb3a64640 1 -- 172.21.15.40:0/3468351854 shutdown_connections 2023-10-18T09:04:17.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.060+0000 7f8bb3a64640 1 -- 172.21.15.40:0/3468351854 wait complete. 2023-10-18T09:04:17.082 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:03:21.754758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:03:22.819486+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '150', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:04:17.083 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:04:17.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.318+0000 7feab80cf640 1 -- 172.21.15.40:0/4003266713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00aed00 msgr2=0x7feab00af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:17.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.318+0000 7feab80cf640 1 --2- 172.21.15.40:0/4003266713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00aed00 0x7feab00af1b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009ca0 tx=0x7feaa802c2d0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.318+0000 7feab80cf640 1 -- 172.21.15.40:0/4003266713 shutdown_connections 2023-10-18T09:04:17.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.318+0000 7feab80cf640 1 --2- 172.21.15.40:0/4003266713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00aed00 0x7feab00af1b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.318+0000 7feab80cf640 1 --2- 172.21.15.40:0/4003266713 >> v2:172.21.15.144:3300/0 conn(0x7feab00ac7b0 0x7feab00cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.318+0000 7feab80cf640 1 -- 172.21.15.40:0/4003266713 >> 172.21.15.40:0/4003266713 conn(0x7feab0069720 msgr2=0x7feab0069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:17.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.318+0000 7feab80cf640 1 -- 172.21.15.40:0/4003266713 shutdown_connections 2023-10-18T09:04:17.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.318+0000 7feab80cf640 1 -- 172.21.15.40:0/4003266713 wait complete. 2023-10-18T09:04:17.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.320+0000 7feab80cf640 1 Processor -- start 2023-10-18T09:04:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.320+0000 7feab80cf640 1 -- start start 2023-10-18T09:04:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.321+0000 7feab80cf640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00ac7b0 0x7feab0152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.321+0000 7feab80cf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feab00aed00 0x7feab006f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.321+0000 7feab80cf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feab0155cd0 0x7feab006fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.321+0000 7feab80cf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab01e5990 con 0x7feab00aed00 2023-10-18T09:04:17.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.321+0000 7feab80cf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab0153320 con 0x7feab0155cd0 2023-10-18T09:04:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.321+0000 7feab80cf640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab01e4a70 con 0x7feab00ac7b0 2023-10-18T09:04:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.321+0000 7feab566a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feab00aed00 0x7feab006f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.321+0000 7feab566a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feab00aed00 0x7feab006f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55898/0 (socket says 172.21.15.40:55898) 2023-10-18T09:04:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.321+0000 7feab566a640 1 -- 172.21.15.40:0/900507453 learned_addr learned my addr 172.21.15.40:0/900507453 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:17.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.322+0000 7feab5e6b640 1 --2- 172.21.15.40:0/900507453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00ac7b0 0x7feab0152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.322+0000 7feab666c640 1 --2- 172.21.15.40:0/900507453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feab0155cd0 0x7feab006fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.322+0000 7feab566a640 1 -- 172.21.15.40:0/900507453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00ac7b0 msgr2=0x7feab0152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.322+0000 7feab566a640 1 --2- 172.21.15.40:0/900507453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00ac7b0 0x7feab0152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.322+0000 7feab566a640 1 -- 172.21.15.40:0/900507453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feab0155cd0 msgr2=0x7feab006fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:17.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.322+0000 7feab566a640 1 --2- 172.21.15.40:0/900507453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feab0155cd0 0x7feab006fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.322+0000 7feab566a640 1 -- 172.21.15.40:0/900507453 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa8009900 con 0x7feab00aed00 2023-10-18T09:04:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.322+0000 7feab566a640 1 --2- 172.21.15.40:0/900507453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feab00aed00 0x7feab006f600 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7feaa000e9f0 tx=0x7feaa000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.322+0000 7feaa67fc640 1 -- 172.21.15.40:0/900507453 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0009700 con 0x7feab00aed00 2023-10-18T09:04:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.323+0000 7feaa67fc640 1 -- 172.21.15.40:0/900507453 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feaa000cd80 con 0x7feab00aed00 2023-10-18T09:04:17.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.323+0000 7feaa67fc640 1 -- 172.21.15.40:0/900507453 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0004280 con 0x7feab00aed00 2023-10-18T09:04:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.323+0000 7feab80cf640 1 -- 172.21.15.40:0/900507453 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab01e4d10 con 0x7feab00aed00 2023-10-18T09:04:17.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.323+0000 7feab80cf640 1 -- 172.21.15.40:0/900507453 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab01e52d0 con 0x7feab00aed00 2023-10-18T09:04:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.324+0000 7feab80cf640 1 -- 172.21.15.40:0/900507453 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab0066fd0 con 0x7feab00aed00 2023-10-18T09:04:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.325+0000 7feaa67fc640 1 -- 172.21.15.40:0/900507453 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feaa00035f0 con 0x7feab00aed00 2023-10-18T09:04:17.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.326+0000 7feaa67fc640 1 --2- 172.21.15.40:0/900507453 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea9406ae40 0x7fea9406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.326+0000 7feaa67fc640 1 -- 172.21.15.40:0/900507453 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feaa0014070 con 0x7feab00aed00 2023-10-18T09:04:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.326+0000 7feaa67fc640 1 -- 172.21.15.40:0/900507453 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7fea94072a30 con 0x7feab00aed00 2023-10-18T09:04:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.327+0000 7feab5e6b640 1 --2- 172.21.15.40:0/900507453 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea9406ae40 0x7fea9406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:17.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.328+0000 7feab5e6b640 1 --2- 172.21.15.40:0/900507453 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea9406ae40 0x7fea9406d300 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7feaac007fd0 tx=0x7feaac009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:17.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.330+0000 7feaa67fc640 1 -- 172.21.15.40:0/900507453 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feaa005de60 con 0x7feab00aed00 2023-10-18T09:04:17.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.521+0000 7feab80cf640 1 -- 172.21.15.40:0/900507453 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feab00b03a0 con 0x7feab00aed00 2023-10-18T09:04:17.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.522+0000 7feaa67fc640 1 -- 172.21.15.40:0/900507453 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v150) v1 ==== 93+0+10 (secure 0 0 0) 0x7feaa005de60 con 0x7feab00aed00 2023-10-18T09:04:17.524 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:04:17.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.525+0000 7feab80cf640 1 -- 172.21.15.40:0/900507453 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea9406ae40 msgr2=0x7fea9406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:17.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.525+0000 7feab80cf640 1 --2- 172.21.15.40:0/900507453 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea9406ae40 0x7fea9406d300 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7feaac007fd0 tx=0x7feaac009040 comp rx=0 tx=0).stop 2023-10-18T09:04:17.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.525+0000 7feab80cf640 1 -- 172.21.15.40:0/900507453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feab00aed00 msgr2=0x7feab006f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:17.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.525+0000 7feab80cf640 1 --2- 172.21.15.40:0/900507453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feab00aed00 0x7feab006f600 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7feaa000e9f0 tx=0x7feaa000eeb0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.525+0000 7feab80cf640 1 -- 172.21.15.40:0/900507453 shutdown_connections 2023-10-18T09:04:17.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.525+0000 7feab80cf640 1 --2- 172.21.15.40:0/900507453 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fea9406ae40 0x7fea9406d300 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.525+0000 7feab80cf640 1 --2- 172.21.15.40:0/900507453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feab0155cd0 0x7feab006fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.525+0000 7feab80cf640 1 --2- 172.21.15.40:0/900507453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feab00aed00 0x7feab006f600 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.525+0000 7feab80cf640 1 --2- 172.21.15.40:0/900507453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feab00ac7b0 0x7feab0152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.525+0000 7feab80cf640 1 -- 172.21.15.40:0/900507453 >> 172.21.15.40:0/900507453 conn(0x7feab0069720 msgr2=0x7feab00ae2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:17.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.525+0000 7feab80cf640 1 -- 172.21.15.40:0/900507453 shutdown_connections 2023-10-18T09:04:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.525+0000 7feab80cf640 1 -- 172.21.15.40:0/900507453 wait complete. 2023-10-18T09:04:17.547 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:04:17.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.784+0000 7f1007600640 1 -- 172.21.15.40:0/311471760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fec004d00 msgr2=0x7f0fec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:17.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.784+0000 7f1007600640 1 --2- 172.21.15.40:0/311471760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fec004d00 0x7f0fec005180 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009b50 tx=0x7f0fec0057f0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.785+0000 7f1005b9d640 1 -- 172.21.15.40:0/311471760 reap_dead start 2023-10-18T09:04:17.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.785+0000 7f1007600640 1 -- 172.21.15.40:0/311471760 shutdown_connections 2023-10-18T09:04:17.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.785+0000 7f1007600640 1 -- 172.21.15.40:0/311471760 >> 172.21.15.40:0/311471760 conn(0x7f1000069720 msgr2=0x7f1000069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:17.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.785+0000 7f1007600640 1 -- 172.21.15.40:0/311471760 shutdown_connections 2023-10-18T09:04:17.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.785+0000 7f1007600640 1 -- 172.21.15.40:0/311471760 wait complete. 2023-10-18T09:04:17.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.787+0000 7f1007600640 1 Processor -- start 2023-10-18T09:04:17.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.787+0000 7f1007600640 1 -- start start 2023-10-18T09:04:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.788+0000 7f1007600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fec004d00 0x7f1000156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:17.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.788+0000 7f1007600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1000154170 0x7f10001545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:17.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.788+0000 7f1007600640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10001387e0 0x7f1000138c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:17.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.788+0000 7f1007600640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100013a390 con 0x7f10001387e0 2023-10-18T09:04:17.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.788+0000 7f1007600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1000154b30 con 0x7f1000154170 2023-10-18T09:04:17.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.788+0000 7f1007600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1000154d80 con 0x7f0fec004d00 2023-10-18T09:04:17.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.788+0000 7f1005b9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10001387e0 0x7f1000138c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:17.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.788+0000 7f1005b9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10001387e0 0x7f1000138c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55926/0 (socket says 172.21.15.40:55926) 2023-10-18T09:04:17.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.788+0000 7f1005b9d640 1 -- 172.21.15.40:0/1699577524 learned_addr learned my addr 172.21.15.40:0/1699577524 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:17.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.788+0000 7f100539c640 1 --2- 172.21.15.40:0/1699577524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fec004d00 0x7f1000156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:17.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.788+0000 7f1005b9d640 1 -- 172.21.15.40:0/1699577524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fec004d00 msgr2=0x7f1000156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:17.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.788+0000 7f1005b9d640 1 --2- 172.21.15.40:0/1699577524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fec004d00 0x7f1000156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.788+0000 7f1005b9d640 1 -- 172.21.15.40:0/1699577524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1000154170 msgr2=0x7f10001545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:17.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.788+0000 7f1005b9d640 1 --2- 172.21.15.40:0/1699577524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1000154170 0x7f10001545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:17.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.788+0000 7f1005b9d640 1 -- 172.21.15.40:0/1699577524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1000155060 con 0x7f10001387e0 2023-10-18T09:04:17.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.789+0000 7f1005b9d640 1 --2- 172.21.15.40:0/1699577524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10001387e0 0x7f1000138c40 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009b50 tx=0x7f0ff8033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:17.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.789+0000 7f0ff5ffb640 1 -- 172.21.15.40:0/1699577524 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8033ea0 con 0x7f10001387e0 2023-10-18T09:04:17.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.789+0000 7f0ff5ffb640 1 -- 172.21.15.40:0/1699577524 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0ff8004230 con 0x7f10001387e0 2023-10-18T09:04:17.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.789+0000 7f0ff5ffb640 1 -- 172.21.15.40:0/1699577524 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80035f0 con 0x7f10001387e0 2023-10-18T09:04:17.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.789+0000 7f1007600640 1 -- 172.21.15.40:0/1699577524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1000155340 con 0x7f10001387e0 2023-10-18T09:04:17.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.789+0000 7f1007600640 1 -- 172.21.15.40:0/1699577524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1000155900 con 0x7f10001387e0 2023-10-18T09:04:17.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.790+0000 7f1007600640 1 -- 172.21.15.40:0/1699577524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1000066fd0 con 0x7f10001387e0 2023-10-18T09:04:17.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.791+0000 7f0ff5ffb640 1 -- 172.21.15.40:0/1699577524 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0ff8003750 con 0x7f10001387e0 2023-10-18T09:04:17.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.792+0000 7f0ff5ffb640 1 --2- 172.21.15.40:0/1699577524 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0fe406ae40 0x7f0fe406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:17.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.793+0000 7f0ff5ffb640 1 -- 172.21.15.40:0/1699577524 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0ff80ac230 con 0x7f10001387e0 2023-10-18T09:04:17.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.793+0000 7f0ff5ffb640 1 -- 172.21.15.40:0/1699577524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f0fe4072a30 con 0x7f10001387e0 2023-10-18T09:04:17.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.793+0000 7f100539c640 1 --2- 172.21.15.40:0/1699577524 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0fe406ae40 0x7f0fe406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:17.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.794+0000 7f100539c640 1 --2- 172.21.15.40:0/1699577524 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0fe406ae40 0x7f0fe406d300 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc005fd0 tx=0x7f0ffc009500 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:17.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.797+0000 7f0ff5ffb640 1 -- 172.21.15.40:0/1699577524 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ff8042060 con 0x7f10001387e0 2023-10-18T09:04:17.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.992+0000 7f1007600640 1 -- 172.21.15.40:0/1699577524 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1000157800 con 0x7f10001387e0 2023-10-18T09:04:17.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.993+0000 7f0ff5ffb640 1 -- 172.21.15.40:0/1699577524 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v150) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0ff8042070 con 0x7f10001387e0 2023-10-18T09:04:17.995 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:04:17.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.996+0000 7f1007600640 1 -- 172.21.15.40:0/1699577524 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0fe406ae40 msgr2=0x7f0fe406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:17.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.996+0000 7f1007600640 1 --2- 172.21.15.40:0/1699577524 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0fe406ae40 0x7f0fe406d300 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc005fd0 tx=0x7f0ffc009500 comp rx=0 tx=0).stop 2023-10-18T09:04:17.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.996+0000 7f1007600640 1 -- 172.21.15.40:0/1699577524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10001387e0 msgr2=0x7f1000138c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:17.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.996+0000 7f1007600640 1 --2- 172.21.15.40:0/1699577524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10001387e0 0x7f1000138c40 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009b50 tx=0x7f0ff8033b60 comp rx=0 tx=0).stop 2023-10-18T09:04:17.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.996+0000 7f1007600640 1 -- 172.21.15.40:0/1699577524 shutdown_connections 2023-10-18T09:04:18.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.996+0000 7f1007600640 1 --2- 172.21.15.40:0/1699577524 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0fe406ae40 0x7f0fe406d300 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.996+0000 7f1007600640 1 --2- 172.21.15.40:0/1699577524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10001387e0 0x7f1000138c40 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.996+0000 7f1007600640 1 --2- 172.21.15.40:0/1699577524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1000154170 0x7f10001545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.996+0000 7f1007600640 1 --2- 172.21.15.40:0/1699577524 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fec004d00 0x7f1000156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.996+0000 7f1007600640 1 -- 172.21.15.40:0/1699577524 >> 172.21.15.40:0/1699577524 conn(0x7f1000069720 msgr2=0x7f100013ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:18.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.996+0000 7f1007600640 1 -- 172.21.15.40:0/1699577524 shutdown_connections 2023-10-18T09:04:18.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:17.996+0000 7f1007600640 1 -- 172.21.15.40:0/1699577524 wait complete. 2023-10-18T09:04:18.018 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:04:18.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.257+0000 7f1943ade640 1 -- 172.21.15.40:0/1497829011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19240039a0 msgr2=0x7f1924003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:18.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.257+0000 7f1943ade640 1 --2- 172.21.15.40:0/1497829011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19240039a0 0x7f1924003dc0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f192c009ed0 tx=0x7f19240057f0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.257+0000 7f194207b640 1 -- 172.21.15.40:0/1497829011 reap_dead start 2023-10-18T09:04:18.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.257+0000 7f1943ade640 1 -- 172.21.15.40:0/1497829011 shutdown_connections 2023-10-18T09:04:18.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.257+0000 7f1943ade640 1 -- 172.21.15.40:0/1497829011 >> 172.21.15.40:0/1497829011 conn(0x7f193c069720 msgr2=0x7f193c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:18.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.257+0000 7f1943ade640 1 -- 172.21.15.40:0/1497829011 shutdown_connections 2023-10-18T09:04:18.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.257+0000 7f1943ade640 1 -- 172.21.15.40:0/1497829011 wait complete. 2023-10-18T09:04:18.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.259+0000 7f1943ade640 1 Processor -- start 2023-10-18T09:04:18.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.259+0000 7f1943ade640 1 -- start start 2023-10-18T09:04:18.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.260+0000 7f1943ade640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19240039a0 0x7f193c0a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:18.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.260+0000 7f1943ade640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f193c13afc0 0x7f193c14d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:18.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.260+0000 7f1943ade640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f193c14ddb0 0x7f193c13fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:18.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.260+0000 7f1943ade640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c14e810 con 0x7f19240039a0 2023-10-18T09:04:18.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.260+0000 7f1943ade640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c13ab30 con 0x7f193c13afc0 2023-10-18T09:04:18.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.260+0000 7f1943ade640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c14e250 con 0x7f193c14ddb0 2023-10-18T09:04:18.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.260+0000 7f194187a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19240039a0 0x7f193c0a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:18.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.260+0000 7f194187a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19240039a0 0x7f193c0a9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55946/0 (socket says 172.21.15.40:55946) 2023-10-18T09:04:18.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.260+0000 7f194187a640 1 -- 172.21.15.40:0/4162923529 learned_addr learned my addr 172.21.15.40:0/4162923529 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:18.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.260+0000 7f194187a640 1 -- 172.21.15.40:0/4162923529 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f193c14ddb0 msgr2=0x7f193c13fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:18.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.260+0000 7f194187a640 1 --2- 172.21.15.40:0/4162923529 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f193c14ddb0 0x7f193c13fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.260+0000 7f194187a640 1 -- 172.21.15.40:0/4162923529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f193c13afc0 msgr2=0x7f193c14d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:18.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.260+0000 7f194187a640 1 --2- 172.21.15.40:0/4162923529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f193c13afc0 0x7f193c14d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.260+0000 7f194187a640 1 -- 172.21.15.40:0/4162923529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c14e530 con 0x7f19240039a0 2023-10-18T09:04:18.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.261+0000 7f194187a640 1 --2- 172.21.15.40:0/4162923529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19240039a0 0x7f193c0a9080 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f192c009e20 tx=0x7f192c031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.261+0000 7f193a7fc640 1 -- 172.21.15.40:0/4162923529 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c03f070 con 0x7f19240039a0 2023-10-18T09:04:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.261+0000 7f193a7fc640 1 -- 172.21.15.40:0/4162923529 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f192c03a7a0 con 0x7f19240039a0 2023-10-18T09:04:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.261+0000 7f193a7fc640 1 -- 172.21.15.40:0/4162923529 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c03abb0 con 0x7f19240039a0 2023-10-18T09:04:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.261+0000 7f1943ade640 1 -- 172.21.15.40:0/4162923529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193c13a7c0 con 0x7f19240039a0 2023-10-18T09:04:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.261+0000 7f1943ade640 1 -- 172.21.15.40:0/4162923529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193c13b440 con 0x7f19240039a0 2023-10-18T09:04:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.262+0000 7f1943ade640 1 -- 172.21.15.40:0/4162923529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193c13a620 con 0x7f19240039a0 2023-10-18T09:04:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.263+0000 7f193a7fc640 1 -- 172.21.15.40:0/4162923529 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f192c03ad10 con 0x7f19240039a0 2023-10-18T09:04:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.265+0000 7f193a7fc640 1 --2- 172.21.15.40:0/4162923529 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f192806ae90 0x7f192806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:18.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.265+0000 7f193a7fc640 1 -- 172.21.15.40:0/4162923529 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f192c0ac260 con 0x7f19240039a0 2023-10-18T09:04:18.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.265+0000 7f193a7fc640 1 -- 172.21.15.40:0/4162923529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f1928072a80 con 0x7f19240039a0 2023-10-18T09:04:18.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.266+0000 7f1941079640 1 --2- 172.21.15.40:0/4162923529 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f192806ae90 0x7f192806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:18.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.267+0000 7f1941079640 1 --2- 172.21.15.40:0/4162923529 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f192806ae90 0x7f192806d350 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f19300017a0 tx=0x7f1930001b90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:18.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.270+0000 7f193a7fc640 1 -- 172.21.15.40:0/4162923529 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f192c035060 con 0x7f19240039a0 2023-10-18T09:04:18.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.464+0000 7f1943ade640 1 -- 172.21.15.40:0/4162923529 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f193c14e770 con 0x7f19240039a0 2023-10-18T09:04:18.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.465+0000 7f193a7fc640 1 -- 172.21.15.40:0/4162923529 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v150) v1 ==== 100+0+11 (secure 0 0 0) 0x7f192c048070 con 0x7f19240039a0 2023-10-18T09:04:18.467 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:04:18.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.467+0000 7f1943ade640 1 -- 172.21.15.40:0/4162923529 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f192806ae90 msgr2=0x7f192806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:18.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.467+0000 7f1943ade640 1 --2- 172.21.15.40:0/4162923529 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f192806ae90 0x7f192806d350 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f19300017a0 tx=0x7f1930001b90 comp rx=0 tx=0).stop 2023-10-18T09:04:18.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.467+0000 7f1943ade640 1 -- 172.21.15.40:0/4162923529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19240039a0 msgr2=0x7f193c0a9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.467+0000 7f1943ade640 1 --2- 172.21.15.40:0/4162923529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19240039a0 0x7f193c0a9080 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f192c009e20 tx=0x7f192c031750 comp rx=0 tx=0).stop 2023-10-18T09:04:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.468+0000 7f1943ade640 1 -- 172.21.15.40:0/4162923529 shutdown_connections 2023-10-18T09:04:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.468+0000 7f1943ade640 1 --2- 172.21.15.40:0/4162923529 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f192806ae90 0x7f192806d350 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.468+0000 7f1943ade640 1 --2- 172.21.15.40:0/4162923529 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f193c14ddb0 0x7f193c13fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.468+0000 7f1943ade640 1 --2- 172.21.15.40:0/4162923529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f193c13afc0 0x7f193c14d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.468+0000 7f1943ade640 1 --2- 172.21.15.40:0/4162923529 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19240039a0 0x7f193c0a9080 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.468+0000 7f1943ade640 1 -- 172.21.15.40:0/4162923529 >> 172.21.15.40:0/4162923529 conn(0x7f193c069720 msgr2=0x7f193c0ae500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.468+0000 7f1943ade640 1 -- 172.21.15.40:0/4162923529 shutdown_connections 2023-10-18T09:04:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.468+0000 7f1943ade640 1 -- 172.21.15.40:0/4162923529 wait complete. 2023-10-18T09:04:18.490 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:04:18.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.720+0000 7f4e505ce640 1 -- 172.21.15.40:0/3176682307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e34003970 msgr2=0x7f4e34003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:18.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.720+0000 7f4e505ce640 1 --2- 172.21.15.40:0/3176682307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e34003970 0x7f4e34003d90 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f4e44006f10 tx=0x7f4e340057f0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.720+0000 7f4e4eb6b640 1 -- 172.21.15.40:0/3176682307 reap_dead start 2023-10-18T09:04:18.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.720+0000 7f4e505ce640 1 -- 172.21.15.40:0/3176682307 shutdown_connections 2023-10-18T09:04:18.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.720+0000 7f4e505ce640 1 -- 172.21.15.40:0/3176682307 >> 172.21.15.40:0/3176682307 conn(0x7f4e4806cee0 msgr2=0x7f4e48069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:18.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.720+0000 7f4e505ce640 1 -- 172.21.15.40:0/3176682307 shutdown_connections 2023-10-18T09:04:18.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.720+0000 7f4e505ce640 1 -- 172.21.15.40:0/3176682307 wait complete. 2023-10-18T09:04:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.722+0000 7f4e505ce640 1 Processor -- start 2023-10-18T09:04:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.722+0000 7f4e505ce640 1 -- start start 2023-10-18T09:04:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.723+0000 7f4e505ce640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e34003970 0x7f4e481ef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.723+0000 7f4e505ce640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e481451d0 0x7f4e48141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:18.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.723+0000 7f4e505ce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e481f35e0 0x7f4e480af370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:18.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.723+0000 7f4e505ce640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e481f4510 con 0x7f4e481f35e0 2023-10-18T09:04:18.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.723+0000 7f4e505ce640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e48141b20 con 0x7f4e34003970 2023-10-18T09:04:18.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.723+0000 7f4e505ce640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e481efef0 con 0x7f4e481451d0 2023-10-18T09:04:18.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.723+0000 7f4e4eb6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e481f35e0 0x7f4e480af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:18.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.723+0000 7f4e4eb6b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e481f35e0 0x7f4e480af370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55968/0 (socket says 172.21.15.40:55968) 2023-10-18T09:04:18.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.723+0000 7f4e4eb6b640 1 -- 172.21.15.40:0/1675762919 learned_addr learned my addr 172.21.15.40:0/1675762919 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:18.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.724+0000 7f4e4e36a640 1 --2- 172.21.15.40:0/1675762919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e34003970 0x7f4e481ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:18.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.724+0000 7f4e4db69640 1 --2- 172.21.15.40:0/1675762919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e481451d0 0x7f4e48141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:18.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.724+0000 7f4e4eb6b640 1 -- 172.21.15.40:0/1675762919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e481451d0 msgr2=0x7f4e48141fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:18.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.724+0000 7f4e4eb6b640 1 --2- 172.21.15.40:0/1675762919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e481451d0 0x7f4e48141fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.724+0000 7f4e4eb6b640 1 -- 172.21.15.40:0/1675762919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e34003970 msgr2=0x7f4e481ef9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:18.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.724+0000 7f4e4eb6b640 1 --2- 172.21.15.40:0/1675762919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e34003970 0x7f4e481ef9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.724+0000 7f4e4eb6b640 1 -- 172.21.15.40:0/1675762919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4815c1d0 con 0x7f4e481f35e0 2023-10-18T09:04:18.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.724+0000 7f4e4eb6b640 1 --2- 172.21.15.40:0/1675762919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e481f35e0 0x7f4e480af370 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f4e4806fb20 tx=0x7f4e40009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:18.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.725+0000 7f4e3effd640 1 -- 172.21.15.40:0/1675762919 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40004d20 con 0x7f4e481f35e0 2023-10-18T09:04:18.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.725+0000 7f4e3effd640 1 -- 172.21.15.40:0/1675762919 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4e40011710 con 0x7f4e481f35e0 2023-10-18T09:04:18.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.725+0000 7f4e3effd640 1 -- 172.21.15.40:0/1675762919 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40011b20 con 0x7f4e481f35e0 2023-10-18T09:04:18.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.725+0000 7f4e505ce640 1 -- 172.21.15.40:0/1675762919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e481f3f50 con 0x7f4e481f35e0 2023-10-18T09:04:18.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.725+0000 7f4e505ce640 1 -- 172.21.15.40:0/1675762919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e481417b0 con 0x7f4e481f35e0 2023-10-18T09:04:18.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.726+0000 7f4e505ce640 1 -- 172.21.15.40:0/1675762919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e48141c90 con 0x7f4e481f35e0 2023-10-18T09:04:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.727+0000 7f4e3effd640 1 -- 172.21.15.40:0/1675762919 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4e40011c80 con 0x7f4e481f35e0 2023-10-18T09:04:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.729+0000 7f4e3effd640 1 --2- 172.21.15.40:0/1675762919 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4e2c06ae40 0x7f4e2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.729+0000 7f4e3effd640 1 -- 172.21.15.40:0/1675762919 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4e40015070 con 0x7f4e481f35e0 2023-10-18T09:04:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.729+0000 7f4e3effd640 1 -- 172.21.15.40:0/1675762919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f4e2c072a30 con 0x7f4e481f35e0 2023-10-18T09:04:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.729+0000 7f4e4e36a640 1 --2- 172.21.15.40:0/1675762919 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4e2c06ae40 0x7f4e2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:18.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.730+0000 7f4e4e36a640 1 --2- 172.21.15.40:0/1675762919 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4e2c06ae40 0x7f4e2c06d300 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4e44034000 tx=0x7f4e4402e040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:18.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.732+0000 7f4e3effd640 1 -- 172.21.15.40:0/1675762919 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4e40055010 con 0x7f4e481f35e0 2023-10-18T09:04:18.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.964+0000 7f4e505ce640 1 -- 172.21.15.40:0/1675762919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e48066fd0 con 0x7f4e481f35e0 2023-10-18T09:04:18.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.966+0000 7f4e3effd640 1 -- 172.21.15.40:0/1675762919 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f4e40058630 con 0x7f4e481f35e0 2023-10-18T09:04:18.968 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:18.968 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6382,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/4165164542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":4165164542},{"type":"v1","addr":"172.21.15.144:6836","nonce":4165164542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3045960359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3045960359},{"type":"v1","addr":"172.21.15.144:6837","nonce":3045960359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2869040025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2869040025},{"type":"v1","addr":"172.21.15.40:6833","nonce":2869040025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:03:23.782442+0000","modified":"2023-10-18T09:03:26.897709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7893},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7893":{"gid":7893,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2600609919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2600609919},{"type":"v1","addr":"172.21.15.40:6835","nonce":2600609919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-18T09:04:18.969 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 66 2023-10-18T09:04:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.968+0000 7f4e505ce640 1 -- 172.21.15.40:0/1675762919 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4e2c06ae40 msgr2=0x7f4e2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.968+0000 7f4e505ce640 1 --2- 172.21.15.40:0/1675762919 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4e2c06ae40 0x7f4e2c06d300 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4e44034000 tx=0x7f4e4402e040 comp rx=0 tx=0).stop 2023-10-18T09:04:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.968+0000 7f4e505ce640 1 -- 172.21.15.40:0/1675762919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e481f35e0 msgr2=0x7f4e480af370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.968+0000 7f4e505ce640 1 --2- 172.21.15.40:0/1675762919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e481f35e0 0x7f4e480af370 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f4e4806fb20 tx=0x7f4e40009a80 comp rx=0 tx=0).stop 2023-10-18T09:04:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.969+0000 7f4e505ce640 1 -- 172.21.15.40:0/1675762919 shutdown_connections 2023-10-18T09:04:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.969+0000 7f4e505ce640 1 --2- 172.21.15.40:0/1675762919 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4e2c06ae40 0x7f4e2c06d300 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.969+0000 7f4e505ce640 1 --2- 172.21.15.40:0/1675762919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e481f35e0 0x7f4e480af370 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.969+0000 7f4e505ce640 1 --2- 172.21.15.40:0/1675762919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e481451d0 0x7f4e48141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.969+0000 7f4e505ce640 1 --2- 172.21.15.40:0/1675762919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e34003970 0x7f4e481ef9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.969+0000 7f4e505ce640 1 -- 172.21.15.40:0/1675762919 >> 172.21.15.40:0/1675762919 conn(0x7f4e4806cee0 msgr2=0x7f4e480ad4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.969+0000 7f4e505ce640 1 -- 172.21.15.40:0/1675762919 shutdown_connections 2023-10-18T09:04:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:18.969+0000 7f4e505ce640 1 -- 172.21.15.40:0/1675762919 wait complete. 2023-10-18T09:04:18.991 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:04:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.212+0000 7f1ceaa90640 1 -- 172.21.15.40:0/677105712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce40b15b0 msgr2=0x7f1ce40c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.212+0000 7f1ceaa90640 1 --2- 172.21.15.40:0/677105712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce40b15b0 0x7f1ce40c99c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4005fd0 tx=0x7f1cd402be10 comp rx=0 tx=0).stop 2023-10-18T09:04:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.212+0000 7f1ceaa90640 1 -- 172.21.15.40:0/677105712 shutdown_connections 2023-10-18T09:04:19.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.212+0000 7f1ceaa90640 1 --2- 172.21.15.40:0/677105712 >> v2:172.21.15.144:3300/0 conn(0x7f1ce40c9f00 0x7f1ce4157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.212+0000 7f1ceaa90640 1 --2- 172.21.15.40:0/677105712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce40b15b0 0x7f1ce40c99c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.212+0000 7f1ceaa90640 1 -- 172.21.15.40:0/677105712 >> 172.21.15.40:0/677105712 conn(0x7f1ce4069720 msgr2=0x7f1ce4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:19.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.212+0000 7f1ceaa90640 1 -- 172.21.15.40:0/677105712 shutdown_connections 2023-10-18T09:04:19.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.212+0000 7f1ceaa90640 1 -- 172.21.15.40:0/677105712 wait complete. 2023-10-18T09:04:19.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.214+0000 7f1ceaa90640 1 Processor -- start 2023-10-18T09:04:19.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.214+0000 7f1ceaa90640 1 -- start start 2023-10-18T09:04:19.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.215+0000 7f1ceaa90640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce406d6f0 0x7f1ce4159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:19.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.215+0000 7f1ceaa90640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce40b15b0 0x7f1ce40af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:19.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.215+0000 7f1ceaa90640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ce40c9f00 0x7f1ce40af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:19.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.215+0000 7f1ceaa90640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce41591a0 con 0x7f1ce40c9f00 2023-10-18T09:04:19.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.215+0000 7f1ceaa90640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce4159480 con 0x7f1ce40b15b0 2023-10-18T09:04:19.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.215+0000 7f1ceaa90640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce4146310 con 0x7f1ce406d6f0 2023-10-18T09:04:19.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.215+0000 7f1cea28f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ce40c9f00 0x7f1ce40af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:19.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.215+0000 7f1cea28f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ce40c9f00 0x7f1ce40af5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55982/0 (socket says 172.21.15.40:55982) 2023-10-18T09:04:19.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.215+0000 7f1cea28f640 1 -- 172.21.15.40:0/262966444 learned_addr learned my addr 172.21.15.40:0/262966444 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:19.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.216+0000 7f1ce9a8e640 1 --2- 172.21.15.40:0/262966444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce406d6f0 0x7f1ce4159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:19.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.216+0000 7f1ce928d640 1 --2- 172.21.15.40:0/262966444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce40b15b0 0x7f1ce40af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:19.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.216+0000 7f1cea28f640 1 -- 172.21.15.40:0/262966444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce406d6f0 msgr2=0x7f1ce4159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:19.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.216+0000 7f1cea28f640 1 --2- 172.21.15.40:0/262966444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce406d6f0 0x7f1ce4159910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.216+0000 7f1cea28f640 1 -- 172.21.15.40:0/262966444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce40b15b0 msgr2=0x7f1ce40af070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:19.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.216+0000 7f1cea28f640 1 --2- 172.21.15.40:0/262966444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce40b15b0 0x7f1ce40af070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.216+0000 7f1cea28f640 1 -- 172.21.15.40:0/262966444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd4009560 con 0x7f1ce40c9f00 2023-10-18T09:04:19.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.216+0000 7f1cea28f640 1 --2- 172.21.15.40:0/262966444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ce40c9f00 0x7f1ce40af5b0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00ea80 tx=0x7f1cdc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:19.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.216+0000 7f1cda7fc640 1 -- 172.21.15.40:0/262966444 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc00cea0 con 0x7f1ce40c9f00 2023-10-18T09:04:19.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.216+0000 7f1cda7fc640 1 -- 172.21.15.40:0/262966444 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1cdc004b20 con 0x7f1ce40c9f00 2023-10-18T09:04:19.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.216+0000 7f1cda7fc640 1 -- 172.21.15.40:0/262966444 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc005020 con 0x7f1ce40c9f00 2023-10-18T09:04:19.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.217+0000 7f1ceaa90640 1 -- 172.21.15.40:0/262966444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce415bc10 con 0x7f1ce40c9f00 2023-10-18T09:04:19.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.217+0000 7f1ceaa90640 1 -- 172.21.15.40:0/262966444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce41468d0 con 0x7f1ce40c9f00 2023-10-18T09:04:19.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.218+0000 7f1ceaa90640 1 -- 172.21.15.40:0/262966444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce41595f0 con 0x7f1ce40c9f00 2023-10-18T09:04:19.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.219+0000 7f1cda7fc640 1 -- 172.21.15.40:0/262966444 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1cdc005180 con 0x7f1ce40c9f00 2023-10-18T09:04:19.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.220+0000 7f1cda7fc640 1 --2- 172.21.15.40:0/262966444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cc806ae40 0x7f1cc806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:19.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.220+0000 7f1cda7fc640 1 -- 172.21.15.40:0/262966444 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1cdc014070 con 0x7f1ce40c9f00 2023-10-18T09:04:19.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.220+0000 7f1cda7fc640 1 -- 172.21.15.40:0/262966444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f1cc8072a30 con 0x7f1ce40c9f00 2023-10-18T09:04:19.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.220+0000 7f1ce9a8e640 1 --2- 172.21.15.40:0/262966444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cc806ae40 0x7f1cc806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:19.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.221+0000 7f1ce9a8e640 1 --2- 172.21.15.40:0/262966444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cc806ae40 0x7f1cc806d300 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00010e0 tx=0x7f1ce0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:19.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.223+0000 7f1cda7fc640 1 -- 172.21.15.40:0/262966444 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1cdc0553b0 con 0x7f1ce40c9f00 2023-10-18T09:04:19.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.417+0000 7f1ceaa90640 1 -- 172.21.15.40:0/262966444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ce4066e00 con 0x7f1ce40c9f00 2023-10-18T09:04:19.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.418+0000 7f1cda7fc640 1 -- 172.21.15.40:0/262966444 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16662 (secure 0 0 0) 0x7f1cdc0589d0 con 0x7f1ce40c9f00 2023-10-18T09:04:19.421 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:19.421 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":150,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:04:16.086898+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:03:21.754758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-18T09:03:22.819486+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.9","mappings":[{"from":0,"to":2}]},{"pgid":"17.c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:04:19.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.421+0000 7f1ceaa90640 1 -- 172.21.15.40:0/262966444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cc806ae40 msgr2=0x7f1cc806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.421+0000 7f1ceaa90640 1 --2- 172.21.15.40:0/262966444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cc806ae40 0x7f1cc806d300 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00010e0 tx=0x7f1ce0009040 comp rx=0 tx=0).stop 2023-10-18T09:04:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.421+0000 7f1ceaa90640 1 -- 172.21.15.40:0/262966444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ce40c9f00 msgr2=0x7f1ce40af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.421+0000 7f1ceaa90640 1 --2- 172.21.15.40:0/262966444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ce40c9f00 0x7f1ce40af5b0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00ea80 tx=0x7f1cdc00ef40 comp rx=0 tx=0).stop 2023-10-18T09:04:19.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.421+0000 7f1ceaa90640 1 -- 172.21.15.40:0/262966444 shutdown_connections 2023-10-18T09:04:19.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.421+0000 7f1ceaa90640 1 --2- 172.21.15.40:0/262966444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1cc806ae40 0x7f1cc806d300 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.421+0000 7f1ceaa90640 1 --2- 172.21.15.40:0/262966444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ce40c9f00 0x7f1ce40af5b0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.421+0000 7f1ceaa90640 1 --2- 172.21.15.40:0/262966444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce40b15b0 0x7f1ce40af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.421+0000 7f1ceaa90640 1 --2- 172.21.15.40:0/262966444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce406d6f0 0x7f1ce4159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.421+0000 7f1ceaa90640 1 -- 172.21.15.40:0/262966444 >> 172.21.15.40:0/262966444 conn(0x7f1ce4069720 msgr2=0x7f1ce4157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:19.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.421+0000 7f1ceaa90640 1 -- 172.21.15.40:0/262966444 shutdown_connections 2023-10-18T09:04:19.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.421+0000 7f1ceaa90640 1 -- 172.21.15.40:0/262966444 wait complete. 2023-10-18T09:04:19.442 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T09:04:19.442 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:04:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.669+0000 7fe782821640 1 -- 172.21.15.40:0/386569760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c1586a0 msgr2=0x7fe77c0cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.669+0000 7fe782821640 1 --2- 172.21.15.40:0/386569760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c1586a0 0x7fe77c0cc990 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe764009830 tx=0x7fe76402bd60 comp rx=0 tx=0).stop 2023-10-18T09:04:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.669+0000 7fe782821640 1 -- 172.21.15.40:0/386569760 shutdown_connections 2023-10-18T09:04:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.669+0000 7fe782821640 1 --2- 172.21.15.40:0/386569760 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe77c0cced0 0x7fe77c0cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.669+0000 7fe782821640 1 --2- 172.21.15.40:0/386569760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c1586a0 0x7fe77c0cc990 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.669+0000 7fe782821640 1 -- 172.21.15.40:0/386569760 >> 172.21.15.40:0/386569760 conn(0x7fe77c06cee0 msgr2=0x7fe77c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.669+0000 7fe782821640 1 -- 172.21.15.40:0/386569760 shutdown_connections 2023-10-18T09:04:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.669+0000 7fe782821640 1 -- 172.21.15.40:0/386569760 wait complete. 2023-10-18T09:04:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.671+0000 7fe782821640 1 Processor -- start 2023-10-18T09:04:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.671+0000 7fe782821640 1 -- start start 2023-10-18T09:04:19.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.672+0000 7fe782821640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe77c06d710 0x7fe77c1f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.672+0000 7fe782821640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0cced0 0x7fe77c0cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.672+0000 7fe782821640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe77c1586a0 0x7fe77c156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.672+0000 7fe782821640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c1f5ea0 con 0x7fe77c0cced0 2023-10-18T09:04:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.672+0000 7fe782821640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c1f4f10 con 0x7fe77c06d710 2023-10-18T09:04:19.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.672+0000 7fe782821640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c0cba60 con 0x7fe77c1586a0 2023-10-18T09:04:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.673+0000 7fe773fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0cced0 0x7fe77c0cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.673+0000 7fe773fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0cced0 0x7fe77c0cb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56002/0 (socket says 172.21.15.40:56002) 2023-10-18T09:04:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.673+0000 7fe773fff640 1 -- 172.21.15.40:0/548056657 learned_addr learned my addr 172.21.15.40:0/548056657 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.673+0000 7fe77bfff640 1 --2- 172.21.15.40:0/548056657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe77c06d710 0x7fe77c1f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:19.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.673+0000 7fe780dbe640 1 --2- 172.21.15.40:0/548056657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe77c1586a0 0x7fe77c156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:19.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.673+0000 7fe773fff640 1 -- 172.21.15.40:0/548056657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe77c1586a0 msgr2=0x7fe77c156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:19.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.673+0000 7fe773fff640 1 --2- 172.21.15.40:0/548056657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe77c1586a0 0x7fe77c156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.673+0000 7fe773fff640 1 -- 172.21.15.40:0/548056657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe77c06d710 msgr2=0x7fe77c1f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:19.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.673+0000 7fe773fff640 1 --2- 172.21.15.40:0/548056657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe77c06d710 0x7fe77c1f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.673+0000 7fe773fff640 1 -- 172.21.15.40:0/548056657 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe764009490 con 0x7fe77c0cced0 2023-10-18T09:04:19.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.674+0000 7fe773fff640 1 --2- 172.21.15.40:0/548056657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0cced0 0x7fe77c0cb180 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe764009780 tx=0x7fe7640038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.674+0000 7fe7797fa640 1 -- 172.21.15.40:0/548056657 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764003ea0 con 0x7fe77c0cced0 2023-10-18T09:04:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.674+0000 7fe7797fa640 1 -- 172.21.15.40:0/548056657 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe764034440 con 0x7fe77c0cced0 2023-10-18T09:04:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.674+0000 7fe7797fa640 1 -- 172.21.15.40:0/548056657 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764034850 con 0x7fe77c0cced0 2023-10-18T09:04:19.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.674+0000 7fe782821640 1 -- 172.21.15.40:0/548056657 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77c0cbd40 con 0x7fe77c0cced0 2023-10-18T09:04:19.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.674+0000 7fe782821640 1 -- 172.21.15.40:0/548056657 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77c1f5440 con 0x7fe77c0cced0 2023-10-18T09:04:19.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.675+0000 7fe782821640 1 -- 172.21.15.40:0/548056657 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe77c1f5660 con 0x7fe77c0cced0 2023-10-18T09:04:19.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.676+0000 7fe7797fa640 1 -- 172.21.15.40:0/548056657 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe76403b040 con 0x7fe77c0cced0 2023-10-18T09:04:19.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.677+0000 7fe7797fa640 1 --2- 172.21.15.40:0/548056657 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe75c06ae70 0x7fe75c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:19.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.677+0000 7fe7797fa640 1 -- 172.21.15.40:0/548056657 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe7640aba50 con 0x7fe77c0cced0 2023-10-18T09:04:19.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.677+0000 7fe7797fa640 1 -- 172.21.15.40:0/548056657 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7fe75c072a60 con 0x7fe77c0cced0 2023-10-18T09:04:19.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.678+0000 7fe77bfff640 1 --2- 172.21.15.40:0/548056657 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe75c06ae70 0x7fe75c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:19.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.679+0000 7fe77bfff640 1 --2- 172.21.15.40:0/548056657 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe75c06ae70 0x7fe75c06d330 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0046b0 tx=0x7fe76c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:19.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.681+0000 7fe7797fa640 1 -- 172.21.15.40:0/548056657 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe764041060 con 0x7fe77c0cced0 2023-10-18T09:04:19.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.903+0000 7fe782821640 1 -- 172.21.15.40:0/548056657 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe77c15c140 con 0x7fe77c0cced0 2023-10-18T09:04:19.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.904+0000 7fe7797fa640 1 -- 172.21.15.40:0/548056657 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fe764041070 con 0x7fe77c0cced0 2023-10-18T09:04:19.907 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:19.907 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6382,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/4165164542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":4165164542},{"type":"v1","addr":"172.21.15.144:6836","nonce":4165164542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3045960359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3045960359},{"type":"v1","addr":"172.21.15.144:6837","nonce":3045960359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2869040025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2869040025},{"type":"v1","addr":"172.21.15.40:6833","nonce":2869040025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:03:23.782442+0000","modified":"2023-10-18T09:03:26.897709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7893},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7893":{"gid":7893,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2600609919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2600609919},{"type":"v1","addr":"172.21.15.40:6835","nonce":2600609919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-18T09:04:19.907 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 66 2023-10-18T09:04:19.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.907+0000 7fe782821640 1 -- 172.21.15.40:0/548056657 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe75c06ae70 msgr2=0x7fe75c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:19.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.907+0000 7fe782821640 1 --2- 172.21.15.40:0/548056657 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe75c06ae70 0x7fe75c06d330 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0046b0 tx=0x7fe76c009040 comp rx=0 tx=0).stop 2023-10-18T09:04:19.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.907+0000 7fe782821640 1 -- 172.21.15.40:0/548056657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0cced0 msgr2=0x7fe77c0cb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:19.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.907+0000 7fe782821640 1 --2- 172.21.15.40:0/548056657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0cced0 0x7fe77c0cb180 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe764009780 tx=0x7fe7640038a0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.907+0000 7fe782821640 1 -- 172.21.15.40:0/548056657 shutdown_connections 2023-10-18T09:04:19.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.907+0000 7fe782821640 1 --2- 172.21.15.40:0/548056657 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe75c06ae70 0x7fe75c06d330 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.907+0000 7fe782821640 1 --2- 172.21.15.40:0/548056657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe77c1586a0 0x7fe77c156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.907+0000 7fe782821640 1 --2- 172.21.15.40:0/548056657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe77c0cced0 0x7fe77c0cb180 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.907+0000 7fe782821640 1 --2- 172.21.15.40:0/548056657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe77c06d710 0x7fe77c1f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.907+0000 7fe782821640 1 -- 172.21.15.40:0/548056657 >> 172.21.15.40:0/548056657 conn(0x7fe77c06cee0 msgr2=0x7fe77c14df40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.907+0000 7fe782821640 1 -- 172.21.15.40:0/548056657 shutdown_connections 2023-10-18T09:04:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:19.907+0000 7fe782821640 1 -- 172.21.15.40:0/548056657 wait complete. 2023-10-18T09:04:19.927 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:04:20.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.148+0000 7ff43599e640 1 -- 172.21.15.40:0/3142456271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff430152560 msgr2=0x7ff430070f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:20.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.148+0000 7ff43599e640 1 --2- 172.21.15.40:0/3142456271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff430152560 0x7ff430070f00 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7ff4300cb300 tx=0x7ff42402c2e0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.148+0000 7ff43599e640 1 -- 172.21.15.40:0/3142456271 shutdown_connections 2023-10-18T09:04:20.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.148+0000 7ff43599e640 1 --2- 172.21.15.40:0/3142456271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4300ab0d0 0x7ff43006b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.148+0000 7ff43599e640 1 --2- 172.21.15.40:0/3142456271 >> v2:172.21.15.144:3300/0 conn(0x7ff430071440 0x7ff4300aab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.148+0000 7ff43599e640 1 --2- 172.21.15.40:0/3142456271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff430152560 0x7ff430070f00 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.148+0000 7ff43599e640 1 -- 172.21.15.40:0/3142456271 >> 172.21.15.40:0/3142456271 conn(0x7ff430069720 msgr2=0x7ff430069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:20.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.148+0000 7ff43599e640 1 -- 172.21.15.40:0/3142456271 shutdown_connections 2023-10-18T09:04:20.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.148+0000 7ff43599e640 1 -- 172.21.15.40:0/3142456271 wait complete. 2023-10-18T09:04:20.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.150+0000 7ff43599e640 1 Processor -- start 2023-10-18T09:04:20.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.150+0000 7ff43599e640 1 -- start start 2023-10-18T09:04:20.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.151+0000 7ff43599e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff430071440 0x7ff43014ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:20.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.151+0000 7ff43599e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4300ab0d0 0x7ff4301544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:20.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.151+0000 7ff43599e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff430152560 0x7ff4301494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:20.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.151+0000 7ff43599e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43014efa0 con 0x7ff430071440 2023-10-18T09:04:20.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.151+0000 7ff43599e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff430154050 con 0x7ff430152560 2023-10-18T09:04:20.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.151+0000 7ff43599e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff430153440 con 0x7ff4300ab0d0 2023-10-18T09:04:20.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.151+0000 7ff43499c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff430071440 0x7ff43014ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:20.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.151+0000 7ff43499c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff430071440 0x7ff43014ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56006/0 (socket says 172.21.15.40:56006) 2023-10-18T09:04:20.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.151+0000 7ff43499c640 1 -- 172.21.15.40:0/3910780564 learned_addr learned my addr 172.21.15.40:0/3910780564 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:20.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.152+0000 7ff42ffff640 1 --2- 172.21.15.40:0/3910780564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4300ab0d0 0x7ff4301544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:20.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.152+0000 7ff43519d640 1 --2- 172.21.15.40:0/3910780564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff430152560 0x7ff4301494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:20.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.152+0000 7ff43499c640 1 -- 172.21.15.40:0/3910780564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4300ab0d0 msgr2=0x7ff4301544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:20.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.152+0000 7ff43499c640 1 --2- 172.21.15.40:0/3910780564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4300ab0d0 0x7ff4301544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.152+0000 7ff43499c640 1 -- 172.21.15.40:0/3910780564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff430152560 msgr2=0x7ff4301494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:20.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.152+0000 7ff43499c640 1 --2- 172.21.15.40:0/3910780564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff430152560 0x7ff4301494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.152+0000 7ff43499c640 1 -- 172.21.15.40:0/3910780564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff424009900 con 0x7ff430071440 2023-10-18T09:04:20.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.152+0000 7ff43519d640 1 --2- 172.21.15.40:0/3910780564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff430152560 0x7ff4301494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:20.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.152+0000 7ff43499c640 1 --2- 172.21.15.40:0/3910780564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff430071440 0x7ff43014ea60 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7ff424000f80 tx=0x7ff424035b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:20.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.152+0000 7ff42d7fa640 1 -- 172.21.15.40:0/3910780564 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff424004240 con 0x7ff430071440 2023-10-18T09:04:20.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.152+0000 7ff42d7fa640 1 -- 172.21.15.40:0/3910780564 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff4240043a0 con 0x7ff430071440 2023-10-18T09:04:20.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.153+0000 7ff42d7fa640 1 -- 172.21.15.40:0/3910780564 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff424004770 con 0x7ff430071440 2023-10-18T09:04:20.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.153+0000 7ff43599e640 1 -- 172.21.15.40:0/3910780564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff430153720 con 0x7ff430071440 2023-10-18T09:04:20.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.153+0000 7ff43599e640 1 -- 172.21.15.40:0/3910780564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff430153ce0 con 0x7ff430071440 2023-10-18T09:04:20.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.154+0000 7ff43599e640 1 -- 172.21.15.40:0/3910780564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff430066e00 con 0x7ff430071440 2023-10-18T09:04:20.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.155+0000 7ff42d7fa640 1 -- 172.21.15.40:0/3910780564 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff42403b040 con 0x7ff430071440 2023-10-18T09:04:20.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.156+0000 7ff42d7fa640 1 --2- 172.21.15.40:0/3910780564 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff41406ae40 0x7ff41406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:20.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.156+0000 7ff42d7fa640 1 -- 172.21.15.40:0/3910780564 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff4240ac440 con 0x7ff430071440 2023-10-18T09:04:20.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.156+0000 7ff42d7fa640 1 -- 172.21.15.40:0/3910780564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7ff414072a30 con 0x7ff430071440 2023-10-18T09:04:20.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.156+0000 7ff42ffff640 1 --2- 172.21.15.40:0/3910780564 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff41406ae40 0x7ff41406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:20.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.158+0000 7ff42ffff640 1 --2- 172.21.15.40:0/3910780564 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff41406ae40 0x7ff41406d300 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff43014a700 tx=0x7ff41c00c460 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.161+0000 7ff42d7fa640 1 -- 172.21.15.40:0/3910780564 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff424043060 con 0x7ff430071440 2023-10-18T09:04:20.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.400+0000 7ff43599e640 1 -- 172.21.15.40:0/3910780564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff43015b4c0 con 0x7ff430071440 2023-10-18T09:04:20.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.400+0000 7ff42d7fa640 1 -- 172.21.15.40:0/3910780564 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4921 (secure 0 0 0) 0x7ff424039090 con 0x7ff430071440 2023-10-18T09:04:20.403 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:20.403 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6382,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/4165164542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":4165164542},{"type":"v1","addr":"172.21.15.144:6836","nonce":4165164542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3045960359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3045960359},{"type":"v1","addr":"172.21.15.144:6837","nonce":3045960359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2869040025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2869040025},{"type":"v1","addr":"172.21.15.40:6833","nonce":2869040025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:03:23.782442+0000","modified":"2023-10-18T09:03:26.897709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7893},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7893":{"gid":7893,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/2600609919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":2600609919},{"type":"v1","addr":"172.21.15.40:6835","nonce":2600609919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-18T09:04:20.404 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 66 2023-10-18T09:04:20.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.403+0000 7ff43599e640 1 -- 172.21.15.40:0/3910780564 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff41406ae40 msgr2=0x7ff41406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:20.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.403+0000 7ff43599e640 1 --2- 172.21.15.40:0/3910780564 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff41406ae40 0x7ff41406d300 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff43014a700 tx=0x7ff41c00c460 comp rx=0 tx=0).stop 2023-10-18T09:04:20.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.403+0000 7ff43599e640 1 -- 172.21.15.40:0/3910780564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff430071440 msgr2=0x7ff43014ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:20.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.403+0000 7ff43599e640 1 --2- 172.21.15.40:0/3910780564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff430071440 0x7ff43014ea60 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7ff424000f80 tx=0x7ff424035b50 comp rx=0 tx=0).stop 2023-10-18T09:04:20.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.403+0000 7ff43599e640 1 -- 172.21.15.40:0/3910780564 shutdown_connections 2023-10-18T09:04:20.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.403+0000 7ff43599e640 1 --2- 172.21.15.40:0/3910780564 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff41406ae40 0x7ff41406d300 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.403+0000 7ff43599e640 1 --2- 172.21.15.40:0/3910780564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff430152560 0x7ff4301494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.403+0000 7ff43599e640 1 --2- 172.21.15.40:0/3910780564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4300ab0d0 0x7ff4301544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.403+0000 7ff43599e640 1 --2- 172.21.15.40:0/3910780564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff430071440 0x7ff43014ea60 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.403+0000 7ff43599e640 1 -- 172.21.15.40:0/3910780564 >> 172.21.15.40:0/3910780564 conn(0x7ff430069720 msgr2=0x7ff43014a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:20.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.403+0000 7ff43599e640 1 -- 172.21.15.40:0/3910780564 shutdown_connections 2023-10-18T09:04:20.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.403+0000 7ff43599e640 1 -- 172.21.15.40:0/3910780564 wait complete. 2023-10-18T09:04:20.426 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:04:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.649+0000 7f4423fff640 1 -- 172.21.15.40:0/1365707160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f440c003f20 msgr2=0x7f440c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.649+0000 7f4423fff640 1 --2- 172.21.15.40:0/1365707160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f440c003f20 0x7f440c0047c0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f4418009b50 tx=0x7f440c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.649+0000 7f44237fe640 1 -- 172.21.15.40:0/1365707160 reap_dead start 2023-10-18T09:04:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.649+0000 7f4423fff640 1 -- 172.21.15.40:0/1365707160 shutdown_connections 2023-10-18T09:04:20.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.649+0000 7f4423fff640 1 -- 172.21.15.40:0/1365707160 >> 172.21.15.40:0/1365707160 conn(0x7f4424069720 msgr2=0x7f4424069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:20.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.649+0000 7f4423fff640 1 -- 172.21.15.40:0/1365707160 shutdown_connections 2023-10-18T09:04:20.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.649+0000 7f4423fff640 1 -- 172.21.15.40:0/1365707160 wait complete. 2023-10-18T09:04:20.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.651+0000 7f4423fff640 1 Processor -- start 2023-10-18T09:04:20.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.651+0000 7f4423fff640 1 -- start start 2023-10-18T09:04:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.652+0000 7f4423fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f440c003f20 0x7f4424158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.652+0000 7f4423fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4424159950 0x7f442413ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.652+0000 7f4423fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44240ab8d0 0x7f44240abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.652+0000 7f4423fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442415af60 con 0x7f44240ab8d0 2023-10-18T09:04:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.652+0000 7f4423fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44240ac290 con 0x7f440c003f20 2023-10-18T09:04:20.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.652+0000 7f4423fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442415a990 con 0x7f4424159950 2023-10-18T09:04:20.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.652+0000 7f44237fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44240ab8d0 0x7f44240abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:20.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.652+0000 7f44237fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44240ab8d0 0x7f44240abd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56016/0 (socket says 172.21.15.40:56016) 2023-10-18T09:04:20.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.652+0000 7f44237fe640 1 -- 172.21.15.40:0/835473862 learned_addr learned my addr 172.21.15.40:0/835473862 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:20.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.653+0000 7f4422ffd640 1 --2- 172.21.15.40:0/835473862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f440c003f20 0x7f4424158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:20.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.653+0000 7f44227fc640 1 --2- 172.21.15.40:0/835473862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4424159950 0x7f442413ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:20.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.653+0000 7f44237fe640 1 -- 172.21.15.40:0/835473862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4424159950 msgr2=0x7f442413ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:20.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.653+0000 7f44237fe640 1 --2- 172.21.15.40:0/835473862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4424159950 0x7f442413ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.653+0000 7f44237fe640 1 -- 172.21.15.40:0/835473862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f440c003f20 msgr2=0x7f4424158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:20.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.653+0000 7f44237fe640 1 --2- 172.21.15.40:0/835473862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f440c003f20 0x7f4424158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.653+0000 7f44237fe640 1 -- 172.21.15.40:0/835473862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4424159e10 con 0x7f44240ab8d0 2023-10-18T09:04:20.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.653+0000 7f44227fc640 1 --2- 172.21.15.40:0/835473862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4424159950 0x7f442413ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:20.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.653+0000 7f44237fe640 1 --2- 172.21.15.40:0/835473862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44240ab8d0 0x7f44240abd50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f441c004a50 tx=0x7f441c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:20.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.653+0000 7f440b7fe640 1 -- 172.21.15.40:0/835473862 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c00ad10 con 0x7f44240ab8d0 2023-10-18T09:04:20.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.653+0000 7f440b7fe640 1 -- 172.21.15.40:0/835473862 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f441c007070 con 0x7f44240ab8d0 2023-10-18T09:04:20.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.653+0000 7f4423fff640 1 -- 172.21.15.40:0/835473862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442415a0f0 con 0x7f44240ab8d0 2023-10-18T09:04:20.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.653+0000 7f440b7fe640 1 -- 172.21.15.40:0/835473862 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c0073e0 con 0x7f44240ab8d0 2023-10-18T09:04:20.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.653+0000 7f4423fff640 1 -- 172.21.15.40:0/835473862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442415a6b0 con 0x7f44240ab8d0 2023-10-18T09:04:20.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.654+0000 7f4423fff640 1 -- 172.21.15.40:0/835473862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4424066e00 con 0x7f44240ab8d0 2023-10-18T09:04:20.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.655+0000 7f440b7fe640 1 -- 172.21.15.40:0/835473862 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f441c007750 con 0x7f44240ab8d0 2023-10-18T09:04:20.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.657+0000 7f440b7fe640 1 --2- 172.21.15.40:0/835473862 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f441006ae40 0x7f441006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:20.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.657+0000 7f440b7fe640 1 -- 172.21.15.40:0/835473862 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f441c089af0 con 0x7f44240ab8d0 2023-10-18T09:04:20.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.657+0000 7f440b7fe640 1 -- 172.21.15.40:0/835473862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7f4410072a30 con 0x7f44240ab8d0 2023-10-18T09:04:20.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.658+0000 7f4422ffd640 1 --2- 172.21.15.40:0/835473862 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f441006ae40 0x7f441006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:20.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.659+0000 7f4422ffd640 1 --2- 172.21.15.40:0/835473862 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f441006ae40 0x7f441006d300 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f442413ba80 tx=0x7f4414005740 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:20.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.661+0000 7f440b7fe640 1 -- 172.21.15.40:0/835473862 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f441c017070 con 0x7f44240ab8d0 2023-10-18T09:04:20.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.847+0000 7f4423fff640 1 -- 172.21.15.40:0/835473862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4424140190 con 0x7f44240ab8d0 2023-10-18T09:04:20.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.850+0000 7f440b7fe640 1 -- 172.21.15.40:0/835473862 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+16662 (secure 0 0 0) 0x7f441c054b50 con 0x7f44240ab8d0 2023-10-18T09:04:20.854 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:20.854 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":150,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:04:16.086898+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:03:21.754758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-18T09:03:22.819486+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.9","mappings":[{"from":0,"to":2}]},{"pgid":"17.c","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:04:20.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.853+0000 7f4423fff640 1 -- 172.21.15.40:0/835473862 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f441006ae40 msgr2=0x7f441006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:20.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.853+0000 7f4423fff640 1 --2- 172.21.15.40:0/835473862 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f441006ae40 0x7f441006d300 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f442413ba80 tx=0x7f4414005740 comp rx=0 tx=0).stop 2023-10-18T09:04:20.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.853+0000 7f4423fff640 1 -- 172.21.15.40:0/835473862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44240ab8d0 msgr2=0x7f44240abd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:20.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.853+0000 7f4423fff640 1 --2- 172.21.15.40:0/835473862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44240ab8d0 0x7f44240abd50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f441c004a50 tx=0x7f441c00a430 comp rx=0 tx=0).stop 2023-10-18T09:04:20.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.853+0000 7f4423fff640 1 -- 172.21.15.40:0/835473862 shutdown_connections 2023-10-18T09:04:20.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.853+0000 7f4423fff640 1 --2- 172.21.15.40:0/835473862 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f441006ae40 0x7f441006d300 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.853+0000 7f4423fff640 1 --2- 172.21.15.40:0/835473862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44240ab8d0 0x7f44240abd50 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.853+0000 7f4423fff640 1 --2- 172.21.15.40:0/835473862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4424159950 0x7f442413ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.853+0000 7f4423fff640 1 --2- 172.21.15.40:0/835473862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f440c003f20 0x7f4424158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:20.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.853+0000 7f4423fff640 1 -- 172.21.15.40:0/835473862 >> 172.21.15.40:0/835473862 conn(0x7f4424069720 msgr2=0x7f442413c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:20.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.854+0000 7f4423fff640 1 -- 172.21.15.40:0/835473862 shutdown_connections 2023-10-18T09:04:20.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:20.854+0000 7f4423fff640 1 -- 172.21.15.40:0/835473862 wait complete. 2023-10-18T09:04:20.875 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T09:04:21.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.102+0000 7fe8bd393640 1 -- 172.21.15.40:0/4255234764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8a4003970 msgr2=0x7fe8a4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:21.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.102+0000 7fe8bd393640 1 --2- 172.21.15.40:0/4255234764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8a4003970 0x7fe8a4003d90 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8001680 tx=0x7fe8a40057f0 comp rx=0 tx=0).stop 2023-10-18T09:04:21.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.102+0000 7fe8bd393640 1 -- 172.21.15.40:0/4255234764 shutdown_connections 2023-10-18T09:04:21.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.102+0000 7fe8bd393640 1 --2- 172.21.15.40:0/4255234764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8a4003970 0x7fe8a4003d90 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:21.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.102+0000 7fe8bd393640 1 -- 172.21.15.40:0/4255234764 >> 172.21.15.40:0/4255234764 conn(0x7fe8b808ad00 msgr2=0x7fe8b809a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:21.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.102+0000 7fe8bd393640 1 -- 172.21.15.40:0/4255234764 shutdown_connections 2023-10-18T09:04:21.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.102+0000 7fe8bd393640 1 -- 172.21.15.40:0/4255234764 wait complete. 2023-10-18T09:04:21.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.104+0000 7fe8bd393640 1 Processor -- start 2023-10-18T09:04:21.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.104+0000 7fe8bd393640 1 -- start start 2023-10-18T09:04:21.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.105+0000 7fe8bd393640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8a4003970 0x7fe8b815a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:21.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.105+0000 7fe8bd393640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8b8159c20 0x7fe8b815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:21.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.105+0000 7fe8bd393640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b8159370 0x7fe8b80b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:21.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.105+0000 7fe8bd393640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b815df20 con 0x7fe8b8159c20 2023-10-18T09:04:21.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.105+0000 7fe8bd393640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b815d910 con 0x7fe8b8159370 2023-10-18T09:04:21.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.105+0000 7fe8bd393640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b80b6970 con 0x7fe8a4003970 2023-10-18T09:04:21.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.106+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8b8159c20 0x7fe8b815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:21.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.106+0000 7fe8b77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8b8159c20 0x7fe8b815e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56034/0 (socket says 172.21.15.40:56034) 2023-10-18T09:04:21.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.106+0000 7fe8b77fe640 1 -- 172.21.15.40:0/3962762077 learned_addr learned my addr 172.21.15.40:0/3962762077 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:21.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.106+0000 7fe8b7fff640 1 --2- 172.21.15.40:0/3962762077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8a4003970 0x7fe8b815a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:21.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.106+0000 7fe8bcb92640 1 --2- 172.21.15.40:0/3962762077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b8159370 0x7fe8b80b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:21.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.106+0000 7fe8b77fe640 1 -- 172.21.15.40:0/3962762077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8a4003970 msgr2=0x7fe8b815a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:21.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.106+0000 7fe8b77fe640 1 --2- 172.21.15.40:0/3962762077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8a4003970 0x7fe8b815a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:21.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.106+0000 7fe8b77fe640 1 -- 172.21.15.40:0/3962762077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b8159370 msgr2=0x7fe8b80b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:21.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.106+0000 7fe8b77fe640 1 --2- 172.21.15.40:0/3962762077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b8159370 0x7fe8b80b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:21.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.106+0000 7fe8b77fe640 1 -- 172.21.15.40:0/3962762077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b80b6690 con 0x7fe8b8159c20 2023-10-18T09:04:21.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.106+0000 7fe8bcb92640 1 --2- 172.21.15.40:0/3962762077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b8159370 0x7fe8b80b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:21.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.106+0000 7fe8b77fe640 1 --2- 172.21.15.40:0/3962762077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8b8159c20 0x7fe8b815e3b0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000fa00 tx=0x7fe8a000fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:21.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.107+0000 7fe8b4ff9640 1 -- 172.21.15.40:0/3962762077 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000bcb0 con 0x7fe8b8159c20 2023-10-18T09:04:21.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.107+0000 7fe8b4ff9640 1 -- 172.21.15.40:0/3962762077 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe8a0004030 con 0x7fe8b8159c20 2023-10-18T09:04:21.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.107+0000 7fe8b4ff9640 1 -- 172.21.15.40:0/3962762077 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0004420 con 0x7fe8b8159c20 2023-10-18T09:04:21.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.107+0000 7fe8bd393640 1 -- 172.21.15.40:0/3962762077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b815dbb0 con 0x7fe8b8159c20 2023-10-18T09:04:21.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.107+0000 7fe8bd393640 1 -- 172.21.15.40:0/3962762077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b81599d0 con 0x7fe8b8159c20 2023-10-18T09:04:21.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.108+0000 7fe8bd393640 1 -- 172.21.15.40:0/3962762077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b8071da0 con 0x7fe8b8159c20 2023-10-18T09:04:21.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.109+0000 7fe8b4ff9640 1 -- 172.21.15.40:0/3962762077 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe8a00035f0 con 0x7fe8b8159c20 2023-10-18T09:04:21.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.110+0000 7fe8b4ff9640 1 --2- 172.21.15.40:0/3962762077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe89806ae70 0x7fe89806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:21.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.110+0000 7fe8b4ff9640 1 -- 172.21.15.40:0/3962762077 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe8a008a060 con 0x7fe8b8159c20 2023-10-18T09:04:21.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.110+0000 7fe8b4ff9640 1 -- 172.21.15.40:0/3962762077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=151}) v3 -- 0x7fe898072a60 con 0x7fe8b8159c20 2023-10-18T09:04:21.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.110+0000 7fe8b7fff640 1 --2- 172.21.15.40:0/3962762077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe89806ae70 0x7fe89806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:21.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.112+0000 7fe8b7fff640 1 --2- 172.21.15.40:0/3962762077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe89806ae70 0x7fe89806d330 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8034000 tx=0x7fe8a8001190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:21.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.117+0000 7fe8b4ff9640 1 -- 172.21.15.40:0/3962762077 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8a00bcca0 con 0x7fe8b8159c20 2023-10-18T09:04:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.352+0000 7fe8bd393640 1 -- 172.21.15.40:0/3962762077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe8b815d770 con 0x7fe8b8159c20 2023-10-18T09:04:21.355 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:04:21.353+0000 7f6a0f710640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T09:04:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.356+0000 7fe8b4ff9640 1 -- 172.21.15.40:0/3962762077 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe8b815d770 con 0x7fe8b8159c20 2023-10-18T09:04:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T09:04:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.359+0000 7fe8bd393640 1 -- 172.21.15.40:0/3962762077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe89806ae70 msgr2=0x7fe89806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.359+0000 7fe8bd393640 1 --2- 172.21.15.40:0/3962762077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe89806ae70 0x7fe89806d330 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8034000 tx=0x7fe8a8001190 comp rx=0 tx=0).stop 2023-10-18T09:04:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.359+0000 7fe8bd393640 1 -- 172.21.15.40:0/3962762077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8b8159c20 msgr2=0x7fe8b815e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:21.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.359+0000 7fe8bd393640 1 --2- 172.21.15.40:0/3962762077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8b8159c20 0x7fe8b815e3b0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000fa00 tx=0x7fe8a000fec0 comp rx=0 tx=0).stop 2023-10-18T09:04:21.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.362+0000 7fe8bd393640 1 -- 172.21.15.40:0/3962762077 shutdown_connections 2023-10-18T09:04:21.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.362+0000 7fe8bd393640 1 --2- 172.21.15.40:0/3962762077 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe89806ae70 0x7fe89806d330 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:21.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.362+0000 7fe8bd393640 1 --2- 172.21.15.40:0/3962762077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8b8159370 0x7fe8b80b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:21.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.362+0000 7fe8bd393640 1 --2- 172.21.15.40:0/3962762077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8b8159c20 0x7fe8b815e3b0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:21.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.362+0000 7fe8bd393640 1 --2- 172.21.15.40:0/3962762077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8a4003970 0x7fe8b815a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:21.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.362+0000 7fe8bd393640 1 -- 172.21.15.40:0/3962762077 >> 172.21.15.40:0/3962762077 conn(0x7fe8b808ad00 msgr2=0x7fe8b806afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:21.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.364+0000 7fe8bd393640 1 -- 172.21.15.40:0/3962762077 shutdown_connections 2023-10-18T09:04:21.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.364+0000 7fe8bd393640 1 -- 172.21.15.40:0/3962762077 wait complete. 2023-10-18T09:04:21.386 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T09:04:21.411 INFO:tasks.ceph.mds.c.smithi040.stderr: -9578> 2023-10-18T09:03:30.299+0000 7f1ad5eb1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T09:04:21.466 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T09:04:21.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.608+0000 7fa907fff640 1 -- 172.21.15.40:0/2064778527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9080ca6c0 msgr2=0x7fa90815ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:21.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.608+0000 7fa907fff640 1 --2- 172.21.15.40:0/2064778527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9080ca6c0 0x7fa90815ce50 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa9080cece0 tx=0x7fa8f002bd90 comp rx=0 tx=0).stop 2023-10-18T09:04:21.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.608+0000 7fa907fff640 1 -- 172.21.15.40:0/2064778527 shutdown_connections 2023-10-18T09:04:21.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.608+0000 7fa907fff640 1 --2- 172.21.15.40:0/2064778527 >> v2:172.21.15.144:3300/0 conn(0x7fa9080cab40 0x7fa90815d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:21.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.608+0000 7fa907fff640 1 --2- 172.21.15.40:0/2064778527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9080ca6c0 0x7fa90815ce50 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:21.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.608+0000 7fa907fff640 1 --2- 172.21.15.40:0/2064778527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa908159360 0x7fa9080cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:21.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.608+0000 7fa907fff640 1 -- 172.21.15.40:0/2064778527 >> 172.21.15.40:0/2064778527 conn(0x7fa9080b0ad0 msgr2=0x7fa9080b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:21.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.608+0000 7fa907fff640 1 -- 172.21.15.40:0/2064778527 shutdown_connections 2023-10-18T09:04:21.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.608+0000 7fa907fff640 1 -- 172.21.15.40:0/2064778527 wait complete. 2023-10-18T09:04:21.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.610+0000 7fa907fff640 1 Processor -- start 2023-10-18T09:04:21.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.610+0000 7fa907fff640 1 -- start start 2023-10-18T09:04:21.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.611+0000 7fa907fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9080ca6c0 0x7fa9080ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:21.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.611+0000 7fa907fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9080cab40 0x7fa9080b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:21.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.611+0000 7fa907fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa908159360 0x7fa908152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:21.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.611+0000 7fa907fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080af6c0 con 0x7fa9080cab40 2023-10-18T09:04:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.611+0000 7fa907fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90806a9f0 con 0x7fa9080ca6c0 2023-10-18T09:04:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.611+0000 7fa907fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080aee20 con 0x7fa908159360 2023-10-18T09:04:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.611+0000 7fa9067fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9080cab40 0x7fa9080b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.611+0000 7fa9067fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9080cab40 0x7fa9080b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56084/0 (socket says 172.21.15.40:56084) 2023-10-18T09:04:21.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.611+0000 7fa9067fc640 1 -- 172.21.15.40:0/3505173454 learned_addr learned my addr 172.21.15.40:0/3505173454 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.612+0000 7fa9067fc640 1 -- 172.21.15.40:0/3505173454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa908159360 msgr2=0x7fa908152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.612+0000 7fa9067fc640 1 --2- 172.21.15.40:0/3505173454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa908159360 0x7fa908152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.612+0000 7fa906ffd640 1 --2- 172.21.15.40:0/3505173454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9080ca6c0 0x7fa9080ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.612+0000 7fa9067fc640 1 -- 172.21.15.40:0/3505173454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9080ca6c0 msgr2=0x7fa9080ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:21.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.612+0000 7fa9067fc640 1 --2- 172.21.15.40:0/3505173454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9080ca6c0 0x7fa9080ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.612+0000 7fa9067fc640 1 -- 172.21.15.40:0/3505173454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f0009560 con 0x7fa9080cab40 2023-10-18T09:04:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.612+0000 7fa9067fc640 1 --2- 172.21.15.40:0/3505173454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9080cab40 0x7fa9080b0040 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0036040 tx=0x7fa8f002dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.613+0000 7fa8ff7fe640 1 -- 172.21.15.40:0/3505173454 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0005650 con 0x7fa9080cab40 2023-10-18T09:04:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.613+0000 7fa8ff7fe640 1 -- 172.21.15.40:0/3505173454 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa8f00038f0 con 0x7fa9080cab40 2023-10-18T09:04:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.613+0000 7fa8ff7fe640 1 -- 172.21.15.40:0/3505173454 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0003ce0 con 0x7fa9080cab40 2023-10-18T09:04:21.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.613+0000 7fa907fff640 1 -- 172.21.15.40:0/3505173454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9080af100 con 0x7fa9080cab40 2023-10-18T09:04:21.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.613+0000 7fa907fff640 1 -- 172.21.15.40:0/3505173454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa908152a80 con 0x7fa9080cab40 2023-10-18T09:04:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.614+0000 7fa907fff640 1 -- 172.21.15.40:0/3505173454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa908073570 con 0x7fa9080cab40 2023-10-18T09:04:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.615+0000 7fa8ff7fe640 1 -- 172.21.15.40:0/3505173454 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa8f003e450 con 0x7fa9080cab40 2023-10-18T09:04:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.616+0000 7fa8ff7fe640 1 --2- 172.21.15.40:0/3505173454 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa8e806ae40 0x7fa8e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:21.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.616+0000 7fa8ff7fe640 1 -- 172.21.15.40:0/3505173454 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa8f00ac0a0 con 0x7fa9080cab40 2023-10-18T09:04:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.616+0000 7fa8ff7fe640 1 -- 172.21.15.40:0/3505173454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=152}) v3 -- 0x7fa8e8072b40 con 0x7fa9080cab40 2023-10-18T09:04:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.617+0000 7fa906ffd640 1 --2- 172.21.15.40:0/3505173454 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa8e806ae40 0x7fa8e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.618+0000 7fa906ffd640 1 --2- 172.21.15.40:0/3505173454 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa8e806ae40 0x7fa8e806d300 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa9080b1660 tx=0x7fa8f8007310 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.620+0000 7fa8ff7fe640 1 -- 172.21.15.40:0/3505173454 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa8f0042060 con 0x7fa9080cab40 2023-10-18T09:04:21.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:21.852+0000 7fa907fff640 1 -- 172.21.15.40:0/3505173454 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa9080c97b0 con 0x7fa9080cab40 2023-10-18T09:04:22.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.360+0000 7fa8ff7fe640 1 -- 172.21.15.40:0/3505173454 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v68) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa8f00771d0 con 0x7fa9080cab40 2023-10-18T09:04:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.364+0000 7fa907fff640 1 -- 172.21.15.40:0/3505173454 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa8e806ae40 msgr2=0x7fa8e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.364+0000 7fa907fff640 1 --2- 172.21.15.40:0/3505173454 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa8e806ae40 0x7fa8e806d300 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa9080b1660 tx=0x7fa8f8007310 comp rx=0 tx=0).stop 2023-10-18T09:04:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.364+0000 7fa907fff640 1 -- 172.21.15.40:0/3505173454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9080cab40 msgr2=0x7fa9080b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.364+0000 7fa907fff640 1 --2- 172.21.15.40:0/3505173454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9080cab40 0x7fa9080b0040 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0036040 tx=0x7fa8f002dd60 comp rx=0 tx=0).stop 2023-10-18T09:04:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.364+0000 7fa907fff640 1 -- 172.21.15.40:0/3505173454 shutdown_connections 2023-10-18T09:04:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.364+0000 7fa907fff640 1 --2- 172.21.15.40:0/3505173454 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa8e806ae40 0x7fa8e806d300 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.364+0000 7fa907fff640 1 --2- 172.21.15.40:0/3505173454 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa908159360 0x7fa908152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.364+0000 7fa907fff640 1 --2- 172.21.15.40:0/3505173454 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9080cab40 0x7fa9080b0040 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.364+0000 7fa907fff640 1 --2- 172.21.15.40:0/3505173454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9080ca6c0 0x7fa9080ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.364+0000 7fa907fff640 1 -- 172.21.15.40:0/3505173454 >> 172.21.15.40:0/3505173454 conn(0x7fa9080b0ad0 msgr2=0x7fa908158610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.364+0000 7fa907fff640 1 -- 172.21.15.40:0/3505173454 shutdown_connections 2023-10-18T09:04:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.365+0000 7fa907fff640 1 -- 172.21.15.40:0/3505173454 wait complete. 2023-10-18T09:04:22.389 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T09:04:22.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.626+0000 7fd77359e640 1 -- 172.21.15.40:0/971642656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd774069720 msgr2=0x7fd774069bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:22.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.626+0000 7fd77359e640 1 --2- 172.21.15.40:0/971642656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd774069720 0x7fd774069bb0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd77406b960 tx=0x7fd76c02bd90 comp rx=0 tx=0).stop 2023-10-18T09:04:22.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.626+0000 7fd77359e640 1 -- 172.21.15.40:0/971642656 shutdown_connections 2023-10-18T09:04:22.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.626+0000 7fd77359e640 1 --2- 172.21.15.40:0/971642656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd774069720 0x7fd774069bb0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:22.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.626+0000 7fd77359e640 1 --2- 172.21.15.40:0/971642656 >> v2:172.21.15.144:3300/0 conn(0x7fd77415fc20 0x7fd7740c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:22.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.626+0000 7fd77359e640 1 --2- 172.21.15.40:0/971642656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd77415f840 0x7fd77415c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:22.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.626+0000 7fd77359e640 1 -- 172.21.15.40:0/971642656 >> 172.21.15.40:0/971642656 conn(0x7fd7740ca950 msgr2=0x7fd7740cad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:22.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.626+0000 7fd77359e640 1 -- 172.21.15.40:0/971642656 shutdown_connections 2023-10-18T09:04:22.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.626+0000 7fd77359e640 1 -- 172.21.15.40:0/971642656 wait complete. 2023-10-18T09:04:22.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.628+0000 7fd77359e640 1 Processor -- start 2023-10-18T09:04:22.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.628+0000 7fd77359e640 1 -- start start 2023-10-18T09:04:22.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.629+0000 7fd77359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd774069720 0x7fd7740c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:22.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.629+0000 7fd77359e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd77415f840 0x7fd7740c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:22.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.629+0000 7fd77359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd77415fc20 0x7fd7740ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:22.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.629+0000 7fd77359e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7741f18d0 con 0x7fd77415fc20 2023-10-18T09:04:22.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.629+0000 7fd77359e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740aaea0 con 0x7fd774069720 2023-10-18T09:04:22.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.629+0000 7fd77359e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740ab180 con 0x7fd77415f840 2023-10-18T09:04:22.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.629+0000 7fd772d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd77415fc20 0x7fd7740ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:22.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.629+0000 7fd772d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd77415fc20 0x7fd7740ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56108/0 (socket says 172.21.15.40:56108) 2023-10-18T09:04:22.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.629+0000 7fd772d9d640 1 -- 172.21.15.40:0/1313585485 learned_addr learned my addr 172.21.15.40:0/1313585485 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:22.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.629+0000 7fd77259c640 1 --2- 172.21.15.40:0/1313585485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd774069720 0x7fd7740c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:22.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.629+0000 7fd769d9b640 1 --2- 172.21.15.40:0/1313585485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd77415f840 0x7fd7740c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:22.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.630+0000 7fd772d9d640 1 -- 172.21.15.40:0/1313585485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd77415f840 msgr2=0x7fd7740c56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:22.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.630+0000 7fd772d9d640 1 --2- 172.21.15.40:0/1313585485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd77415f840 0x7fd7740c56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:22.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.630+0000 7fd772d9d640 1 -- 172.21.15.40:0/1313585485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd774069720 msgr2=0x7fd7740c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:22.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.630+0000 7fd772d9d640 1 --2- 172.21.15.40:0/1313585485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd774069720 0x7fd7740c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:22.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.630+0000 7fd772d9d640 1 -- 172.21.15.40:0/1313585485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76c009560 con 0x7fd77415fc20 2023-10-18T09:04:22.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.630+0000 7fd772d9d640 1 --2- 172.21.15.40:0/1313585485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd77415fc20 0x7fd7740ccec0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd76c036040 tx=0x7fd76c02dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:22.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.630+0000 7fd76b7fe640 1 -- 172.21.15.40:0/1313585485 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c003800 con 0x7fd77415fc20 2023-10-18T09:04:22.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.630+0000 7fd76b7fe640 1 -- 172.21.15.40:0/1313585485 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd76c003a50 con 0x7fd77415fc20 2023-10-18T09:04:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.630+0000 7fd76b7fe640 1 -- 172.21.15.40:0/1313585485 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c003e60 con 0x7fd77415fc20 2023-10-18T09:04:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.630+0000 7fd77359e640 1 -- 172.21.15.40:0/1313585485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7740ab460 con 0x7fd77415fc20 2023-10-18T09:04:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.631+0000 7fd77359e640 1 -- 172.21.15.40:0/1313585485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7740aba20 con 0x7fd77415fc20 2023-10-18T09:04:22.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.632+0000 7fd77359e640 1 -- 172.21.15.40:0/1313585485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7740abc10 con 0x7fd77415fc20 2023-10-18T09:04:22.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.633+0000 7fd76b7fe640 1 -- 172.21.15.40:0/1313585485 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd76c03e450 con 0x7fd77415fc20 2023-10-18T09:04:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.634+0000 7fd76b7fe640 1 --2- 172.21.15.40:0/1313585485 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd75406ae40 0x7fd75406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.634+0000 7fd76b7fe640 1 -- 172.21.15.40:0/1313585485 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd76c0ac290 con 0x7fd77415fc20 2023-10-18T09:04:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.634+0000 7fd76b7fe640 1 -- 172.21.15.40:0/1313585485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=152}) v3 -- 0x7fd754072b40 con 0x7fd77415fc20 2023-10-18T09:04:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.634+0000 7fd77259c640 1 --2- 172.21.15.40:0/1313585485 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd75406ae40 0x7fd75406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.636+0000 7fd77259c640 1 --2- 172.21.15.40:0/1313585485 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd75406ae40 0x7fd75406d300 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd764005fd0 tx=0x7fd7640093c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.638+0000 7fd76b7fe640 1 -- 172.21.15.40:0/1313585485 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd76c042060 con 0x7fd77415fc20 2023-10-18T09:04:22.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:22.840+0000 7fd77359e640 1 -- 172.21.15.40:0/1313585485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd774073570 con 0x7fd77415fc20 2023-10-18T09:04:23.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.364+0000 7fd76b7fe640 1 -- 172.21.15.40:0/1313585485 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v152) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd76c077340 con 0x7fd77415fc20 2023-10-18T09:04:23.366 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-10-18T09:04:23.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.366+0000 7fd76b7fe640 1 -- 172.21.15.40:0/1313585485 <== mon.0 v2:172.21.15.40:3300/0 8 ==== osd_map(152..152 src has 1..152) v4 ==== 354+0+0 (secure 0 0 0) 0x7fd76c07b700 con 0x7fd77415fc20 2023-10-18T09:04:23.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.366+0000 7fd76b7fe640 1 -- 172.21.15.40:0/1313585485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=153}) v3 -- 0x7fd754070f70 con 0x7fd77415fc20 2023-10-18T09:04:23.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.368+0000 7fd77359e640 1 -- 172.21.15.40:0/1313585485 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd75406ae40 msgr2=0x7fd75406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:23.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.368+0000 7fd77359e640 1 --2- 172.21.15.40:0/1313585485 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd75406ae40 0x7fd75406d300 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd764005fd0 tx=0x7fd7640093c0 comp rx=0 tx=0).stop 2023-10-18T09:04:23.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.368+0000 7fd77359e640 1 -- 172.21.15.40:0/1313585485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd77415fc20 msgr2=0x7fd7740ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:23.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.368+0000 7fd77359e640 1 --2- 172.21.15.40:0/1313585485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd77415fc20 0x7fd7740ccec0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd76c036040 tx=0x7fd76c02dab0 comp rx=0 tx=0).stop 2023-10-18T09:04:23.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.369+0000 7fd77359e640 1 -- 172.21.15.40:0/1313585485 shutdown_connections 2023-10-18T09:04:23.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.369+0000 7fd77359e640 1 --2- 172.21.15.40:0/1313585485 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd75406ae40 0x7fd75406d300 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:23.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.369+0000 7fd77359e640 1 --2- 172.21.15.40:0/1313585485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd77415fc20 0x7fd7740ccec0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:23.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.369+0000 7fd77359e640 1 --2- 172.21.15.40:0/1313585485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd77415f840 0x7fd7740c56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:23.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.369+0000 7fd77359e640 1 --2- 172.21.15.40:0/1313585485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd774069720 0x7fd7740c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:23.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.369+0000 7fd77359e640 1 -- 172.21.15.40:0/1313585485 >> 172.21.15.40:0/1313585485 conn(0x7fd7740ca950 msgr2=0x7fd77415cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:23.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.375+0000 7fd77359e640 1 -- 172.21.15.40:0/1313585485 shutdown_connections 2023-10-18T09:04:23.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.375+0000 7fd77359e640 1 -- 172.21.15.40:0/1313585485 wait complete. 2023-10-18T09:04:23.401 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T09:04:23.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.617+0000 7feed9247640 1 -- 172.21.15.40:0/1466851408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feed414e350 msgr2=0x7feed40cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:23.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.617+0000 7feed9247640 1 --2- 172.21.15.40:0/1466851408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feed414e350 0x7feed40cc240 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7feebc005fd0 tx=0x7feebc02be20 comp rx=0 tx=0).stop 2023-10-18T09:04:23.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.618+0000 7feed9247640 1 -- 172.21.15.40:0/1466851408 shutdown_connections 2023-10-18T09:04:23.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.618+0000 7feed9247640 1 --2- 172.21.15.40:0/1466851408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feed414e350 0x7feed40cc240 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:23.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.618+0000 7feed9247640 1 --2- 172.21.15.40:0/1466851408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feed406d550 0x7feed406b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:23.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.618+0000 7feed9247640 1 -- 172.21.15.40:0/1466851408 >> 172.21.15.40:0/1466851408 conn(0x7feed40aeb90 msgr2=0x7feed40aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:23.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.618+0000 7feed9247640 1 -- 172.21.15.40:0/1466851408 shutdown_connections 2023-10-18T09:04:23.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.618+0000 7feed9247640 1 -- 172.21.15.40:0/1466851408 wait complete. 2023-10-18T09:04:23.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.620+0000 7feed9247640 1 Processor -- start 2023-10-18T09:04:23.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.620+0000 7feed9247640 1 -- start start 2023-10-18T09:04:23.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.621+0000 7feed9247640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feed406d550 0x7feed4151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:23.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.621+0000 7feed9247640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed40cc780 0x7feed4144cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:23.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.621+0000 7feed9247640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feed414e350 0x7feed4145200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:23.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.621+0000 7feed9247640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed414d010 con 0x7feed40cc780 2023-10-18T09:04:23.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.621+0000 7feed9247640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed414c390 con 0x7feed406d550 2023-10-18T09:04:23.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.621+0000 7feed9247640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed414c670 con 0x7feed414e350 2023-10-18T09:04:23.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.621+0000 7feecbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed40cc780 0x7feed4144cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:23.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.621+0000 7feecbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed40cc780 0x7feed4144cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56118/0 (socket says 172.21.15.40:56118) 2023-10-18T09:04:23.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.621+0000 7feecbfff640 1 -- 172.21.15.40:0/577192544 learned_addr learned my addr 172.21.15.40:0/577192544 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:23.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.622+0000 7feed3fff640 1 --2- 172.21.15.40:0/577192544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feed406d550 0x7feed4151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:23.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.622+0000 7feed8a46640 1 --2- 172.21.15.40:0/577192544 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feed414e350 0x7feed4145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:23.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.622+0000 7feecbfff640 1 -- 172.21.15.40:0/577192544 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feed414e350 msgr2=0x7feed4145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:23.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.622+0000 7feecbfff640 1 --2- 172.21.15.40:0/577192544 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feed414e350 0x7feed4145200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:23.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.622+0000 7feecbfff640 1 -- 172.21.15.40:0/577192544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feed406d550 msgr2=0x7feed4151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:23.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.622+0000 7feecbfff640 1 --2- 172.21.15.40:0/577192544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feed406d550 0x7feed4151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:23.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.622+0000 7feecbfff640 1 -- 172.21.15.40:0/577192544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc009560 con 0x7feed40cc780 2023-10-18T09:04:23.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.622+0000 7feed3fff640 1 --2- 172.21.15.40:0/577192544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feed406d550 0x7feed4151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:23.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.622+0000 7feed8a46640 1 --2- 172.21.15.40:0/577192544 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feed414e350 0x7feed4145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:23.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.622+0000 7feecbfff640 1 --2- 172.21.15.40:0/577192544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed40cc780 0x7feed4144cc0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7feebc009850 tx=0x7feebc02da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:23.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.622+0000 7feed17fa640 1 -- 172.21.15.40:0/577192544 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc02de60 con 0x7feed40cc780 2023-10-18T09:04:23.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.623+0000 7feed17fa640 1 -- 172.21.15.40:0/577192544 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feebc0036e0 con 0x7feed40cc780 2023-10-18T09:04:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.623+0000 7feed17fa640 1 -- 172.21.15.40:0/577192544 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc003a90 con 0x7feed40cc780 2023-10-18T09:04:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.623+0000 7feed9247640 1 -- 172.21.15.40:0/577192544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed414c950 con 0x7feed40cc780 2023-10-18T09:04:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.623+0000 7feed9247640 1 -- 172.21.15.40:0/577192544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed4145740 con 0x7feed40cc780 2023-10-18T09:04:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.624+0000 7feed9247640 1 -- 172.21.15.40:0/577192544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed414d150 con 0x7feed40cc780 2023-10-18T09:04:23.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.625+0000 7feed17fa640 1 -- 172.21.15.40:0/577192544 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feebc003bf0 con 0x7feed40cc780 2023-10-18T09:04:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.626+0000 7feed17fa640 1 --2- 172.21.15.40:0/577192544 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feeb406ae40 0x7feeb406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.626+0000 7feed17fa640 1 -- 172.21.15.40:0/577192544 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5415+0+0 (secure 0 0 0) 0x7feebc0ac130 con 0x7feed40cc780 2023-10-18T09:04:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.626+0000 7feed17fa640 1 -- 172.21.15.40:0/577192544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=153}) v3 -- 0x7feeb4071cd0 con 0x7feed40cc780 2023-10-18T09:04:23.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.627+0000 7feed3fff640 1 --2- 172.21.15.40:0/577192544 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feeb406ae40 0x7feeb406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:23.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.628+0000 7feed3fff640 1 --2- 172.21.15.40:0/577192544 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feeb406ae40 0x7feeb406d300 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7feec400abc0 tx=0x7feec4005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:23.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.630+0000 7feed17fa640 1 -- 172.21.15.40:0/577192544 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feed414d150 con 0x7feed40cc780 2023-10-18T09:04:23.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:23.826+0000 7feed9247640 1 -- 172.21.15.40:0/577192544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feed414d150 con 0x7feed40cc780 2023-10-18T09:04:24.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.387+0000 7feed17fa640 1 -- 172.21.15.40:0/577192544 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v153) v1 ==== 169+0+0 (secure 0 0 0) 0x7feed414d150 con 0x7feed40cc780 2023-10-18T09:04:24.390 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-10-18T09:04:24.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.388+0000 7feed17fa640 1 -- 172.21.15.40:0/577192544 <== mon.0 v2:172.21.15.40:3300/0 8 ==== osd_map(153..153 src has 1..153) v4 ==== 330+0+0 (secure 0 0 0) 0x7feebc07b740 con 0x7feed40cc780 2023-10-18T09:04:24.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.388+0000 7feed17fa640 1 -- 172.21.15.40:0/577192544 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=154}) v3 -- 0x7feeb4070a60 con 0x7feed40cc780 2023-10-18T09:04:24.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.394+0000 7feed9247640 1 -- 172.21.15.40:0/577192544 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feeb406ae40 msgr2=0x7feeb406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:24.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.394+0000 7feed9247640 1 --2- 172.21.15.40:0/577192544 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feeb406ae40 0x7feeb406d300 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7feec400abc0 tx=0x7feec4005f90 comp rx=0 tx=0).stop 2023-10-18T09:04:24.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.394+0000 7feed9247640 1 -- 172.21.15.40:0/577192544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed40cc780 msgr2=0x7feed4144cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:24.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.394+0000 7feed9247640 1 --2- 172.21.15.40:0/577192544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed40cc780 0x7feed4144cc0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7feebc009850 tx=0x7feebc02da60 comp rx=0 tx=0).stop 2023-10-18T09:04:24.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.394+0000 7feed9247640 1 -- 172.21.15.40:0/577192544 shutdown_connections 2023-10-18T09:04:24.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.394+0000 7feed9247640 1 --2- 172.21.15.40:0/577192544 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feeb406ae40 0x7feeb406d300 secure :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7feec400abc0 tx=0x7feec4005f90 comp rx=0 tx=0).stop 2023-10-18T09:04:24.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.394+0000 7feed9247640 1 --2- 172.21.15.40:0/577192544 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feed414e350 0x7feed4145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:24.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.394+0000 7feed9247640 1 --2- 172.21.15.40:0/577192544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feed40cc780 0x7feed4144cc0 secure :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7feebc009850 tx=0x7feebc02da60 comp rx=0 tx=0).stop 2023-10-18T09:04:24.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.394+0000 7feed9247640 1 --2- 172.21.15.40:0/577192544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feed406d550 0x7feed4151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:24.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.394+0000 7feed9247640 1 -- 172.21.15.40:0/577192544 >> 172.21.15.40:0/577192544 conn(0x7feed40aeb90 msgr2=0x7feed4159f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:24.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.399+0000 7feed9247640 1 -- 172.21.15.40:0/577192544 shutdown_connections 2023-10-18T09:04:24.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.399+0000 7feed9247640 1 -- 172.21.15.40:0/577192544 wait complete. 2023-10-18T09:04:24.429 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 22 2023-10-18T09:04:24.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.630+0000 7f7461dc4640 1 -- 172.21.15.40:0/2393713858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c0b15b0 msgr2=0x7f745c0c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:24.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.630+0000 7f7461dc4640 1 --2- 172.21.15.40:0/2393713858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c0b15b0 0x7f745c0c99c0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f745c0caf30 tx=0x7f744c02bd70 comp rx=0 tx=0).stop 2023-10-18T09:04:24.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.631+0000 7f7461dc4640 1 -- 172.21.15.40:0/2393713858 shutdown_connections 2023-10-18T09:04:24.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.631+0000 7f7461dc4640 1 --2- 172.21.15.40:0/2393713858 >> v2:172.21.15.144:3300/0 conn(0x7f745c0c9f00 0x7f745c157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:24.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.631+0000 7f7461dc4640 1 --2- 172.21.15.40:0/2393713858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c0b15b0 0x7f745c0c99c0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:24.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.631+0000 7f7461dc4640 1 -- 172.21.15.40:0/2393713858 >> 172.21.15.40:0/2393713858 conn(0x7f745c069720 msgr2=0x7f745c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:24.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.631+0000 7f7461dc4640 1 -- 172.21.15.40:0/2393713858 shutdown_connections 2023-10-18T09:04:24.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.631+0000 7f7461dc4640 1 -- 172.21.15.40:0/2393713858 wait complete. 2023-10-18T09:04:24.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.633+0000 7f7461dc4640 1 Processor -- start 2023-10-18T09:04:24.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.633+0000 7f7461dc4640 1 -- start start 2023-10-18T09:04:24.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.634+0000 7f7461dc4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c06d6f0 0x7f745c13b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:24.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.634+0000 7f7461dc4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f745c0b15b0 0x7f745c0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:24.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.634+0000 7f7461dc4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f745c0c9f00 0x7f745c0af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:24.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.634+0000 7f7461dc4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c13be60 con 0x7f745c06d6f0 2023-10-18T09:04:24.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.634+0000 7f7461dc4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c152bf0 con 0x7f745c0b15b0 2023-10-18T09:04:24.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.634+0000 7f7461dc4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745c152630 con 0x7f745c0c9f00 2023-10-18T09:04:24.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.634+0000 7f7460dc2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c06d6f0 0x7f745c13b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:24.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.634+0000 7f7460dc2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c06d6f0 0x7f745c13b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56150/0 (socket says 172.21.15.40:56150) 2023-10-18T09:04:24.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.634+0000 7f7460dc2640 1 -- 172.21.15.40:0/1347693663 learned_addr learned my addr 172.21.15.40:0/1347693663 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:24.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.634+0000 7f7453fff640 1 --2- 172.21.15.40:0/1347693663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f745c0b15b0 0x7f745c0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:24.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.635+0000 7f7460dc2640 1 -- 172.21.15.40:0/1347693663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f745c0c9f00 msgr2=0x7f745c0af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:24.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.635+0000 7f7460dc2640 1 --2- 172.21.15.40:0/1347693663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f745c0c9f00 0x7f745c0af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:24.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.635+0000 7f7460dc2640 1 -- 172.21.15.40:0/1347693663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f745c0b15b0 msgr2=0x7f745c0aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:24.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.635+0000 7f7460dc2640 1 --2- 172.21.15.40:0/1347693663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f745c0b15b0 0x7f745c0aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:24.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.635+0000 7f7460dc2640 1 -- 172.21.15.40:0/1347693663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c009490 con 0x7f745c06d6f0 2023-10-18T09:04:24.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.635+0000 7f7460dc2640 1 --2- 172.21.15.40:0/1347693663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c06d6f0 0x7f745c13b920 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f745800c490 tx=0x7f745800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:24.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.635+0000 7f74517fa640 1 -- 172.21.15.40:0/1347693663 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7458012830 con 0x7f745c06d6f0 2023-10-18T09:04:24.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.635+0000 7f74517fa640 1 -- 172.21.15.40:0/1347693663 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7458012a80 con 0x7f745c06d6f0 2023-10-18T09:04:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.635+0000 7f74517fa640 1 -- 172.21.15.40:0/1347693663 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7458011380 con 0x7f745c06d6f0 2023-10-18T09:04:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.635+0000 7f7461dc4640 1 -- 172.21.15.40:0/1347693663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745c152910 con 0x7f745c06d6f0 2023-10-18T09:04:24.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.635+0000 7f7461dc4640 1 -- 172.21.15.40:0/1347693663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745c0af810 con 0x7f745c06d6f0 2023-10-18T09:04:24.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.636+0000 7f7461dc4640 1 -- 172.21.15.40:0/1347693663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745c150010 con 0x7f745c06d6f0 2023-10-18T09:04:24.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.638+0000 7f74517fa640 1 -- 172.21.15.40:0/1347693663 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f74580114e0 con 0x7f745c06d6f0 2023-10-18T09:04:24.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.638+0000 7f74517fa640 1 --2- 172.21.15.40:0/1347693663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f744806ae40 0x7f744806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:24.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.639+0000 7f74517fa640 1 -- 172.21.15.40:0/1347693663 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7458089590 con 0x7f745c06d6f0 2023-10-18T09:04:24.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.639+0000 7f7453fff640 1 --2- 172.21.15.40:0/1347693663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f744806ae40 0x7f744806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:24.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.640+0000 7f7453fff640 1 --2- 172.21.15.40:0/1347693663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f744806ae40 0x7f744806d300 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f744c036000 tx=0x7f744c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:24.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.643+0000 7f74517fa640 1 -- 172.21.15.40:0/1347693663 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f745c150010 con 0x7f745c06d6f0 2023-10-18T09:04:24.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.835+0000 7f7461dc4640 1 -- 172.21.15.40:0/1347693663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config reset", "num": 22} v 0) v1 -- 0x7f745c150010 con 0x7f745c06d6f0 2023-10-18T09:04:24.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.841+0000 7f74517fa640 1 -- 172.21.15.40:0/1347693663 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 22}]=0 v36) v1 ==== 71+0+0 (secure 0 0 0) 0x7f745c150010 con 0x7f745c06d6f0 2023-10-18T09:04:24.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.844+0000 7f7461dc4640 1 -- 172.21.15.40:0/1347693663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f744806ae40 msgr2=0x7f744806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:24.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.844+0000 7f7461dc4640 1 --2- 172.21.15.40:0/1347693663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f744806ae40 0x7f744806d300 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f744c036000 tx=0x7f744c004730 comp rx=0 tx=0).stop 2023-10-18T09:04:24.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.844+0000 7f7461dc4640 1 -- 172.21.15.40:0/1347693663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c06d6f0 msgr2=0x7f745c13b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:24.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.844+0000 7f7461dc4640 1 --2- 172.21.15.40:0/1347693663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c06d6f0 0x7f745c13b920 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f745800c490 tx=0x7f745800c950 comp rx=0 tx=0).stop 2023-10-18T09:04:24.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.844+0000 7f7461dc4640 1 -- 172.21.15.40:0/1347693663 shutdown_connections 2023-10-18T09:04:24.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.844+0000 7f7461dc4640 1 --2- 172.21.15.40:0/1347693663 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f744806ae40 0x7f744806d300 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:24.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.844+0000 7f7461dc4640 1 --2- 172.21.15.40:0/1347693663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f745c0c9f00 0x7f745c0af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:24.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.844+0000 7f7461dc4640 1 --2- 172.21.15.40:0/1347693663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f745c0b15b0 0x7f745c0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:24.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.844+0000 7f7461dc4640 1 --2- 172.21.15.40:0/1347693663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c06d6f0 0x7f745c13b920 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:24.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.844+0000 7f7461dc4640 1 -- 172.21.15.40:0/1347693663 >> 172.21.15.40:0/1347693663 conn(0x7f745c069720 msgr2=0x7f745c1557f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:24.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.844+0000 7f7461dc4640 1 -- 172.21.15.40:0/1347693663 shutdown_connections 2023-10-18T09:04:24.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:24.844+0000 7f7461dc4640 1 -- 172.21.15.40:0/1347693663 wait complete. 2023-10-18T09:04:24.866 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2023-10-18T09:04:25.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.094+0000 7f1462fbf640 1 -- 172.21.15.40:0/1425181868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14480052a0 msgr2=0x7f144800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:25.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.094+0000 7f1462fbf640 1 --2- 172.21.15.40:0/1425181868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14480052a0 0x7f144800b980 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f1454001d50 tx=0x7f14480057f0 comp rx=0 tx=0).stop 2023-10-18T09:04:25.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.094+0000 7f1462fbf640 1 -- 172.21.15.40:0/1425181868 shutdown_connections 2023-10-18T09:04:25.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.094+0000 7f1462fbf640 1 --2- 172.21.15.40:0/1425181868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14480052a0 0x7f144800b980 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:25.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.094+0000 7f1462fbf640 1 -- 172.21.15.40:0/1425181868 >> 172.21.15.40:0/1425181868 conn(0x7f145c0afdf0 msgr2=0x7f145c0b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:25.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.094+0000 7f1462fbf640 1 -- 172.21.15.40:0/1425181868 shutdown_connections 2023-10-18T09:04:25.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.094+0000 7f1462fbf640 1 -- 172.21.15.40:0/1425181868 wait complete. 2023-10-18T09:04:25.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.096+0000 7f1462fbf640 1 Processor -- start 2023-10-18T09:04:25.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.096+0000 7f1462fbf640 1 -- start start 2023-10-18T09:04:25.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.097+0000 7f1462fbf640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14480052a0 0x7f145c157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:25.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.097+0000 7f1462fbf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145c0b1cd0 0x7f145c0b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:25.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.097+0000 7f1462fbf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f145c13c6d0 0x7f145c13cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:25.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.097+0000 7f1462fbf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c0b2b30 con 0x7f145c13c6d0 2023-10-18T09:04:25.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.097+0000 7f1462fbf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c1581e0 con 0x7f145c0b1cd0 2023-10-18T09:04:25.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.097+0000 7f1462fbf640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145c0ab4d0 con 0x7f14480052a0 2023-10-18T09:04:25.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.097+0000 7f14627be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f145c13c6d0 0x7f145c13cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:25.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.097+0000 7f14627be640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f145c13c6d0 0x7f145c13cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40150/0 (socket says 172.21.15.40:40150) 2023-10-18T09:04:25.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.097+0000 7f14627be640 1 -- 172.21.15.40:0/49450939 learned_addr learned my addr 172.21.15.40:0/49450939 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:25.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.098+0000 7f14617bc640 1 --2- 172.21.15.40:0/49450939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145c0b1cd0 0x7f145c0b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:25.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.098+0000 7f14627be640 1 -- 172.21.15.40:0/49450939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14480052a0 msgr2=0x7f145c157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:25.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.098+0000 7f14627be640 1 --2- 172.21.15.40:0/49450939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14480052a0 0x7f145c157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:25.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.098+0000 7f14627be640 1 -- 172.21.15.40:0/49450939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145c0b1cd0 msgr2=0x7f145c0b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:25.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.098+0000 7f14627be640 1 --2- 172.21.15.40:0/49450939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145c0b1cd0 0x7f145c0b2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:25.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.098+0000 7f14627be640 1 -- 172.21.15.40:0/49450939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c0ab7b0 con 0x7f145c13c6d0 2023-10-18T09:04:25.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.098+0000 7f14627be640 1 --2- 172.21.15.40:0/49450939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f145c13c6d0 0x7f145c13cb50 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f1454001390 tx=0x7f1454003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:25.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.098+0000 7f14527fc640 1 -- 172.21.15.40:0/49450939 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14540060d0 con 0x7f145c13c6d0 2023-10-18T09:04:25.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.098+0000 7f14527fc640 1 -- 172.21.15.40:0/49450939 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1454006340 con 0x7f145c13c6d0 2023-10-18T09:04:25.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.098+0000 7f14527fc640 1 -- 172.21.15.40:0/49450939 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454006750 con 0x7f145c13c6d0 2023-10-18T09:04:25.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.098+0000 7f1462fbf640 1 -- 172.21.15.40:0/49450939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145c0b11f0 con 0x7f145c13c6d0 2023-10-18T09:04:25.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.099+0000 7f1462fbf640 1 -- 172.21.15.40:0/49450939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145c0b17b0 con 0x7f145c13c6d0 2023-10-18T09:04:25.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.100+0000 7f1462fbf640 1 -- 172.21.15.40:0/49450939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145c0b19a0 con 0x7f145c13c6d0 2023-10-18T09:04:25.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.101+0000 7f14527fc640 1 -- 172.21.15.40:0/49450939 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1454034620 con 0x7f145c13c6d0 2023-10-18T09:04:25.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.102+0000 7f14527fc640 1 --2- 172.21.15.40:0/49450939 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f144006ae90 0x7f144006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:25.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.102+0000 7f14527fc640 1 -- 172.21.15.40:0/49450939 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f14540aa760 con 0x7f145c13c6d0 2023-10-18T09:04:25.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.103+0000 7f1461fbd640 1 --2- 172.21.15.40:0/49450939 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f144006ae90 0x7f144006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:25.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.104+0000 7f1461fbd640 1 --2- 172.21.15.40:0/49450939 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f144006ae90 0x7f144006d350 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f145c13efc0 tx=0x7f1458000f50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:25.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.106+0000 7f14527fc640 1 -- 172.21.15.40:0/49450939 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1454075b80 con 0x7f145c13c6d0 2023-10-18T09:04:25.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:25.373+0000 7f1462fbf640 1 -- 172.21.15.40:0/49450939 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f145c066e00 con 0x7f145c13c6d0 2023-10-18T09:04:26.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.373+0000 7f14527fc640 1 -- 172.21.15.40:0/49450939 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v480) v1 ==== 140+0+0 (secure 0 0 0) 0x7f1454043020 con 0x7f145c13c6d0 2023-10-18T09:04:26.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.376+0000 7f1462fbf640 1 -- 172.21.15.40:0/49450939 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f144006ae90 msgr2=0x7f144006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:26.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.376+0000 7f1462fbf640 1 --2- 172.21.15.40:0/49450939 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f144006ae90 0x7f144006d350 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f145c13efc0 tx=0x7f1458000f50 comp rx=0 tx=0).stop 2023-10-18T09:04:26.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.376+0000 7f1462fbf640 1 -- 172.21.15.40:0/49450939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f145c13c6d0 msgr2=0x7f145c13cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:26.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.376+0000 7f1462fbf640 1 --2- 172.21.15.40:0/49450939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f145c13c6d0 0x7f145c13cb50 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f1454001390 tx=0x7f1454003f50 comp rx=0 tx=0).stop 2023-10-18T09:04:26.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.377+0000 7f1462fbf640 1 -- 172.21.15.40:0/49450939 shutdown_connections 2023-10-18T09:04:26.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.377+0000 7f1462fbf640 1 --2- 172.21.15.40:0/49450939 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f144006ae90 0x7f144006d350 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:26.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.377+0000 7f1462fbf640 1 --2- 172.21.15.40:0/49450939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f145c13c6d0 0x7f145c13cb50 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:26.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.377+0000 7f1462fbf640 1 --2- 172.21.15.40:0/49450939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f145c0b1cd0 0x7f145c0b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:26.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.377+0000 7f1462fbf640 1 --2- 172.21.15.40:0/49450939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14480052a0 0x7f145c157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:26.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.377+0000 7f1462fbf640 1 -- 172.21.15.40:0/49450939 >> 172.21.15.40:0/49450939 conn(0x7f145c0afdf0 msgr2=0x7f145c15cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:26.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.377+0000 7f1462fbf640 1 -- 172.21.15.40:0/49450939 shutdown_connections 2023-10-18T09:04:26.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.377+0000 7f1462fbf640 1 -- 172.21.15.40:0/49450939 wait complete. 2023-10-18T09:04:26.399 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-10-18T09:04:26.400 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) 2023-10-18T09:04:26.401 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:04:26.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.631+0000 7fb3c4cbb640 1 -- 172.21.15.40:0/857456588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c015b3e0 msgr2=0x7fb3c0070f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:26.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.631+0000 7fb3c4cbb640 1 --2- 172.21.15.40:0/857456588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c015b3e0 0x7fb3c0070f90 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fb3c006a5e0 tx=0x7fb3b002c110 comp rx=0 tx=0).stop 2023-10-18T09:04:26.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.631+0000 7fb3c4cbb640 1 -- 172.21.15.40:0/857456588 shutdown_connections 2023-10-18T09:04:26.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.631+0000 7fb3c4cbb640 1 --2- 172.21.15.40:0/857456588 >> v2:172.21.15.144:3300/0 conn(0x7fb3c015eec0 0x7fb3c00c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:26.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.631+0000 7fb3c4cbb640 1 --2- 172.21.15.40:0/857456588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c015b3e0 0x7fb3c0070f90 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:26.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.631+0000 7fb3c4cbb640 1 -- 172.21.15.40:0/857456588 >> 172.21.15.40:0/857456588 conn(0x7fb3c00ca960 msgr2=0x7fb3c00cad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:26.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.631+0000 7fb3c4cbb640 1 -- 172.21.15.40:0/857456588 shutdown_connections 2023-10-18T09:04:26.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.631+0000 7fb3c4cbb640 1 -- 172.21.15.40:0/857456588 wait complete. 2023-10-18T09:04:26.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.633+0000 7fb3c4cbb640 1 Processor -- start 2023-10-18T09:04:26.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.633+0000 7fb3c4cbb640 1 -- start start 2023-10-18T09:04:26.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.634+0000 7fb3c4cbb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3c015b3e0 0x7fb3c0156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:26.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.634+0000 7fb3c4cbb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c015eec0 0x7fb3c00c1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:26.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.634+0000 7fb3c4cbb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c015f350 0x7fb3c00c2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:26.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.634+0000 7fb3c4cbb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c0157b20 con 0x7fb3c015f350 2023-10-18T09:04:26.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.634+0000 7fb3c4cbb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c006de70 con 0x7fb3c015eec0 2023-10-18T09:04:26.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.634+0000 7fb3c4cbb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c00c8910 con 0x7fb3c015b3e0 2023-10-18T09:04:26.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.635+0000 7fb3bffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c015f350 0x7fb3c00c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:26.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.635+0000 7fb3bffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c015f350 0x7fb3c00c2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40168/0 (socket says 172.21.15.40:40168) 2023-10-18T09:04:26.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.635+0000 7fb3bffff640 1 -- 172.21.15.40:0/1333855080 learned_addr learned my addr 172.21.15.40:0/1333855080 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:26.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.635+0000 7fb3bf7fe640 1 --2- 172.21.15.40:0/1333855080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3c015b3e0 0x7fb3c0156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:26.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.635+0000 7fb3beffd640 1 --2- 172.21.15.40:0/1333855080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c015eec0 0x7fb3c00c1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:26.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.635+0000 7fb3bffff640 1 -- 172.21.15.40:0/1333855080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3c015b3e0 msgr2=0x7fb3c0156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:26.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.635+0000 7fb3bffff640 1 --2- 172.21.15.40:0/1333855080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3c015b3e0 0x7fb3c0156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:26.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.635+0000 7fb3bffff640 1 -- 172.21.15.40:0/1333855080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c015eec0 msgr2=0x7fb3c00c1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:26.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.635+0000 7fb3bffff640 1 --2- 172.21.15.40:0/1333855080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c015eec0 0x7fb3c00c1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:26.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.635+0000 7fb3bffff640 1 -- 172.21.15.40:0/1333855080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b0009840 con 0x7fb3c015f350 2023-10-18T09:04:26.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.635+0000 7fb3bf7fe640 1 --2- 172.21.15.40:0/1333855080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3c015b3e0 0x7fb3c0156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:04:26.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.635+0000 7fb3beffd640 1 --2- 172.21.15.40:0/1333855080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c015eec0 0x7fb3c00c1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:04:26.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.635+0000 7fb3bffff640 1 --2- 172.21.15.40:0/1333855080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c015f350 0x7fb3c00c2080 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800c390 tx=0x7fb3b800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:26.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.636+0000 7fb3b7fff640 1 -- 172.21.15.40:0/1333855080 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8011790 con 0x7fb3c015f350 2023-10-18T09:04:26.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.636+0000 7fb3b7fff640 1 -- 172.21.15.40:0/1333855080 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb3b80119e0 con 0x7fb3c015f350 2023-10-18T09:04:26.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.636+0000 7fb3b7fff640 1 -- 172.21.15.40:0/1333855080 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8011df0 con 0x7fb3c015f350 2023-10-18T09:04:26.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.636+0000 7fb3c4cbb640 1 -- 172.21.15.40:0/1333855080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c006d260 con 0x7fb3c015f350 2023-10-18T09:04:26.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.636+0000 7fb3c4cbb640 1 -- 172.21.15.40:0/1333855080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c006d820 con 0x7fb3c015f350 2023-10-18T09:04:26.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.637+0000 7fb3c4cbb640 1 -- 172.21.15.40:0/1333855080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c00ca6e0 con 0x7fb3c015f350 2023-10-18T09:04:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.638+0000 7fb3b7fff640 1 -- 172.21.15.40:0/1333855080 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb3b801c460 con 0x7fb3c015f350 2023-10-18T09:04:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.639+0000 7fb3b7fff640 1 --2- 172.21.15.40:0/1333855080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3a006ae40 0x7fb3a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.639+0000 7fb3b7fff640 1 -- 172.21.15.40:0/1333855080 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb3b8089340 con 0x7fb3c015f350 2023-10-18T09:04:26.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.640+0000 7fb3bf7fe640 1 --2- 172.21.15.40:0/1333855080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3a006ae40 0x7fb3a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:26.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.641+0000 7fb3bf7fe640 1 --2- 172.21.15.40:0/1333855080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3a006ae40 0x7fb3a006d300 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0035000 tx=0x7fb3b0004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:26.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.644+0000 7fb3b7fff640 1 -- 172.21.15.40:0/1333855080 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3b8054650 con 0x7fb3c015f350 2023-10-18T09:04:26.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.838+0000 7fb3c4cbb640 1 -- 172.21.15.40:0/1333855080 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3c015c7d0 con 0x7fb3c015f350 2023-10-18T09:04:26.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.840+0000 7fb3b7fff640 1 -- 172.21.15.40:0/1333855080 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+13009 (secure 0 0 0) 0x7fb3c015c7d0 con 0x7fb3c015f350 2023-10-18T09:04:26.843 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:26.843 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":153,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:04:24.383871+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/2600609919":"2023-10-19T09:04:21.353783+0000","172.21.15.40:6834/2600609919":"2023-10-19T09:04:21.353783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:04:26.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.843+0000 7fb3c4cbb640 1 -- 172.21.15.40:0/1333855080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3a006ae40 msgr2=0x7fb3a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:26.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.843+0000 7fb3c4cbb640 1 --2- 172.21.15.40:0/1333855080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3a006ae40 0x7fb3a006d300 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0035000 tx=0x7fb3b0004ab0 comp rx=0 tx=0).stop 2023-10-18T09:04:26.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.843+0000 7fb3c4cbb640 1 -- 172.21.15.40:0/1333855080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c015f350 msgr2=0x7fb3c00c2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:26.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.843+0000 7fb3c4cbb640 1 --2- 172.21.15.40:0/1333855080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c015f350 0x7fb3c00c2080 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800c390 tx=0x7fb3b800c850 comp rx=0 tx=0).stop 2023-10-18T09:04:26.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.843+0000 7fb3c4cbb640 1 -- 172.21.15.40:0/1333855080 shutdown_connections 2023-10-18T09:04:26.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.843+0000 7fb3c4cbb640 1 --2- 172.21.15.40:0/1333855080 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb3a006ae40 0x7fb3a006d300 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:26.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.843+0000 7fb3c4cbb640 1 --2- 172.21.15.40:0/1333855080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3c015f350 0x7fb3c00c2080 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:26.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.843+0000 7fb3c4cbb640 1 --2- 172.21.15.40:0/1333855080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3c015eec0 0x7fb3c00c1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:26.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.843+0000 7fb3c4cbb640 1 --2- 172.21.15.40:0/1333855080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb3c015b3e0 0x7fb3c0156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:26.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.843+0000 7fb3c4cbb640 1 -- 172.21.15.40:0/1333855080 >> 172.21.15.40:0/1333855080 conn(0x7fb3c00ca960 msgr2=0x7fb3c00ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:26.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.844+0000 7fb3c4cbb640 1 -- 172.21.15.40:0/1333855080 shutdown_connections 2023-10-18T09:04:26.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:26.844+0000 7fb3c4cbb640 1 -- 172.21.15.40:0/1333855080 wait complete. 2023-10-18T09:04:26.867 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:04:26.868 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:04:27.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.106+0000 7f63b48d1640 1 -- 172.21.15.40:0/4164736687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b00b1900 msgr2=0x7f63b00af310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:27.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.106+0000 7f63b48d1640 1 --2- 172.21.15.40:0/4164736687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b00b1900 0x7f63b00af310 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f63a8005fd0 tx=0x7f63a802bd90 comp rx=0 tx=0).stop 2023-10-18T09:04:27.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.106+0000 7f63b48d1640 1 -- 172.21.15.40:0/4164736687 shutdown_connections 2023-10-18T09:04:27.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.106+0000 7f63b48d1640 1 --2- 172.21.15.40:0/4164736687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b00b1900 0x7f63b00af310 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:27.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.106+0000 7f63b48d1640 1 --2- 172.21.15.40:0/4164736687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63b01405a0 0x7f63b00c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:27.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.106+0000 7f63b48d1640 1 -- 172.21.15.40:0/4164736687 >> 172.21.15.40:0/4164736687 conn(0x7f63b0069720 msgr2=0x7f63b0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:27.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.106+0000 7f63b48d1640 1 -- 172.21.15.40:0/4164736687 shutdown_connections 2023-10-18T09:04:27.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.106+0000 7f63b48d1640 1 -- 172.21.15.40:0/4164736687 wait complete. 2023-10-18T09:04:27.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.108+0000 7f63b48d1640 1 Processor -- start 2023-10-18T09:04:27.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.108+0000 7f63b48d1640 1 -- start start 2023-10-18T09:04:27.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.109+0000 7f63b48d1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63b006d600 0x7f63b00c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:27.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.109+0000 7f63b48d1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b00b1900 0x7f63b013b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:27.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.109+0000 7f63b48d1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63b01405a0 0x7f63b013bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:27.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.109+0000 7f63b48d1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b013c2a0 con 0x7f63b00b1900 2023-10-18T09:04:27.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.109+0000 7f63b48d1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b013c580 con 0x7f63b006d600 2023-10-18T09:04:27.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.109+0000 7f63b48d1640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b013c860 con 0x7f63b01405a0 2023-10-18T09:04:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.109+0000 7f63a6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b00b1900 0x7f63b013b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.109+0000 7f63a6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b00b1900 0x7f63b013b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40182/0 (socket says 172.21.15.40:40182) 2023-10-18T09:04:27.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.109+0000 7f63a6ffd640 1 -- 172.21.15.40:0/4088874200 learned_addr learned my addr 172.21.15.40:0/4088874200 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:27.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.109+0000 7f63affff640 1 --2- 172.21.15.40:0/4088874200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63b01405a0 0x7f63b013bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:27.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.109+0000 7f63af7fe640 1 --2- 172.21.15.40:0/4088874200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63b006d600 0x7f63b00c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:27.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.110+0000 7f63a6ffd640 1 -- 172.21.15.40:0/4088874200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63b01405a0 msgr2=0x7f63b013bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:27.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.110+0000 7f63a6ffd640 1 --2- 172.21.15.40:0/4088874200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63b01405a0 0x7f63b013bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:27.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.110+0000 7f63a6ffd640 1 -- 172.21.15.40:0/4088874200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63b006d600 msgr2=0x7f63b00c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:27.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.110+0000 7f63a6ffd640 1 --2- 172.21.15.40:0/4088874200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63b006d600 0x7f63b00c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:27.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.110+0000 7f63a6ffd640 1 -- 172.21.15.40:0/4088874200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a8009560 con 0x7f63b00b1900 2023-10-18T09:04:27.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.110+0000 7f63affff640 1 --2- 172.21.15.40:0/4088874200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63b01405a0 0x7f63b013bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:27.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.110+0000 7f63af7fe640 1 --2- 172.21.15.40:0/4088874200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63b006d600 0x7f63b00c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.110+0000 7f63a6ffd640 1 --2- 172.21.15.40:0/4088874200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b00b1900 0x7f63b013b720 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f639800db90 tx=0x7f639800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.110+0000 7f63acff9640 1 -- 172.21.15.40:0/4088874200 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639800e650 con 0x7f63b00b1900 2023-10-18T09:04:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.110+0000 7f63acff9640 1 -- 172.21.15.40:0/4088874200 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f639800e8a0 con 0x7f63b00b1900 2023-10-18T09:04:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.110+0000 7f63acff9640 1 -- 172.21.15.40:0/4088874200 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639800ecb0 con 0x7f63b00b1900 2023-10-18T09:04:27.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.110+0000 7f63b48d1640 1 -- 172.21.15.40:0/4088874200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b00acc50 con 0x7f63b00b1900 2023-10-18T09:04:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.110+0000 7f63b48d1640 1 -- 172.21.15.40:0/4088874200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b01f5160 con 0x7f63b00b1900 2023-10-18T09:04:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.111+0000 7f63b48d1640 1 -- 172.21.15.40:0/4088874200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b0066e00 con 0x7f63b00b1900 2023-10-18T09:04:27.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.112+0000 7f63acff9640 1 -- 172.21.15.40:0/4088874200 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6398016440 con 0x7f63b00b1900 2023-10-18T09:04:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.113+0000 7f63acff9640 1 --2- 172.21.15.40:0/4088874200 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f639006ae40 0x7f639006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.113+0000 7f63acff9640 1 -- 172.21.15.40:0/4088874200 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6398089b30 con 0x7f63b00b1900 2023-10-18T09:04:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.114+0000 7f63af7fe640 1 --2- 172.21.15.40:0/4088874200 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f639006ae40 0x7f639006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:27.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.115+0000 7f63af7fe640 1 --2- 172.21.15.40:0/4088874200 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f639006ae40 0x7f639006d300 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f63a0004520 tx=0x7f63a0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:27.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.118+0000 7f63acff9640 1 -- 172.21.15.40:0/4088874200 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6398054ef0 con 0x7f63b00b1900 2023-10-18T09:04:27.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.310+0000 7f63b48d1640 1 -- 172.21.15.40:0/4088874200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f63b015acd0 con 0x7f63b00b1900 2023-10-18T09:04:27.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.311+0000 7f63acff9640 1 -- 172.21.15.40:0/4088874200 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v153) v1 ==== 93+0+10 (secure 0 0 0) 0x7f639801b020 con 0x7f63b00b1900 2023-10-18T09:04:27.314 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:04:27.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.314+0000 7f63b48d1640 1 -- 172.21.15.40:0/4088874200 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f639006ae40 msgr2=0x7f639006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:27.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.314+0000 7f63b48d1640 1 --2- 172.21.15.40:0/4088874200 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f639006ae40 0x7f639006d300 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f63a0004520 tx=0x7f63a0009040 comp rx=0 tx=0).stop 2023-10-18T09:04:27.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.314+0000 7f63b48d1640 1 -- 172.21.15.40:0/4088874200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b00b1900 msgr2=0x7f63b013b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:27.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.314+0000 7f63b48d1640 1 --2- 172.21.15.40:0/4088874200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b00b1900 0x7f63b013b720 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f639800db90 tx=0x7f639800a620 comp rx=0 tx=0).stop 2023-10-18T09:04:27.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.314+0000 7f63b48d1640 1 -- 172.21.15.40:0/4088874200 shutdown_connections 2023-10-18T09:04:27.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.314+0000 7f63b48d1640 1 --2- 172.21.15.40:0/4088874200 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f639006ae40 0x7f639006d300 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:27.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.314+0000 7f63b48d1640 1 --2- 172.21.15.40:0/4088874200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f63b01405a0 0x7f63b013bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:27.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.314+0000 7f63b48d1640 1 --2- 172.21.15.40:0/4088874200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b00b1900 0x7f63b013b720 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:27.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.314+0000 7f63b48d1640 1 --2- 172.21.15.40:0/4088874200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63b006d600 0x7f63b00c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:27.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.314+0000 7f63b48d1640 1 -- 172.21.15.40:0/4088874200 >> 172.21.15.40:0/4088874200 conn(0x7f63b0069720 msgr2=0x7f63b00b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:27.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.314+0000 7f63b48d1640 1 -- 172.21.15.40:0/4088874200 shutdown_connections 2023-10-18T09:04:27.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.314+0000 7f63b48d1640 1 -- 172.21.15.40:0/4088874200 wait complete. 2023-10-18T09:04:27.336 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2023-10-18T09:04:27.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.556+0000 7fd74ee40640 1 -- 172.21.15.40:0/1244603949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd734003970 msgr2=0x7fd734003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:27.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.556+0000 7fd74ee40640 1 --2- 172.21.15.40:0/1244603949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd734003970 0x7fd734003d90 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fd744009b50 tx=0x7fd7340057f0 comp rx=0 tx=0).stop 2023-10-18T09:04:27.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.556+0000 7fd74e63f640 1 -- 172.21.15.40:0/1244603949 reap_dead start 2023-10-18T09:04:27.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.557+0000 7fd74ee40640 1 -- 172.21.15.40:0/1244603949 shutdown_connections 2023-10-18T09:04:27.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.557+0000 7fd74ee40640 1 -- 172.21.15.40:0/1244603949 >> 172.21.15.40:0/1244603949 conn(0x7fd748069720 msgr2=0x7fd748069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:27.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.557+0000 7fd74ee40640 1 -- 172.21.15.40:0/1244603949 shutdown_connections 2023-10-18T09:04:27.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.557+0000 7fd74ee40640 1 -- 172.21.15.40:0/1244603949 wait complete. 2023-10-18T09:04:27.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.558+0000 7fd74ee40640 1 Processor -- start 2023-10-18T09:04:27.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.559+0000 7fd74ee40640 1 -- start start 2023-10-18T09:04:27.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74ee40640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd734003970 0x7fd7481e37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:27.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74ee40640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd748150cb0 0x7fd7481e3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:27.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74ee40640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd748151130 0x7fd7481e4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:27.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74ee40640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd748151700 con 0x7fd734003970 2023-10-18T09:04:27.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74ee40640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7481519e0 con 0x7fd748150cb0 2023-10-18T09:04:27.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74ee40640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd748151cc0 con 0x7fd748151130 2023-10-18T09:04:27.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74de3e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd734003970 0x7fd7481e37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:27.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74de3e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd734003970 0x7fd7481e37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40200/0 (socket says 172.21.15.40:40200) 2023-10-18T09:04:27.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74de3e640 1 -- 172.21.15.40:0/2338411706 learned_addr learned my addr 172.21.15.40:0/2338411706 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74e63f640 1 --2- 172.21.15.40:0/2338411706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd748151130 0x7fd7481e4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:27.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74d63d640 1 --2- 172.21.15.40:0/2338411706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd748150cb0 0x7fd7481e3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74de3e640 1 -- 172.21.15.40:0/2338411706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd748151130 msgr2=0x7fd7481e4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74de3e640 1 --2- 172.21.15.40:0/2338411706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd748151130 0x7fd7481e4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74de3e640 1 -- 172.21.15.40:0/2338411706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd748150cb0 msgr2=0x7fd7481e3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:27.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74de3e640 1 --2- 172.21.15.40:0/2338411706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd748150cb0 0x7fd7481e3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74de3e640 1 -- 172.21.15.40:0/2338411706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd748151fa0 con 0x7fd734003970 2023-10-18T09:04:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.560+0000 7fd74d63d640 1 --2- 172.21.15.40:0/2338411706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd748150cb0 0x7fd7481e3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.561+0000 7fd74de3e640 1 --2- 172.21.15.40:0/2338411706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd734003970 0x7fd7481e37a0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd744009b50 tx=0x7fd744033bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.561+0000 7fd73e7fc640 1 -- 172.21.15.40:0/2338411706 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744004230 con 0x7fd734003970 2023-10-18T09:04:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.561+0000 7fd73e7fc640 1 -- 172.21.15.40:0/2338411706 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd7440043b0 con 0x7fd734003970 2023-10-18T09:04:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.561+0000 7fd74ee40640 1 -- 172.21.15.40:0/2338411706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd748152280 con 0x7fd734003970 2023-10-18T09:04:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.561+0000 7fd73e7fc640 1 -- 172.21.15.40:0/2338411706 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744003850 con 0x7fd734003970 2023-10-18T09:04:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.561+0000 7fd74ee40640 1 -- 172.21.15.40:0/2338411706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7481e5d40 con 0x7fd734003970 2023-10-18T09:04:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.562+0000 7fd74ee40640 1 -- 172.21.15.40:0/2338411706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd748066e00 con 0x7fd734003970 2023-10-18T09:04:27.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.563+0000 7fd73e7fc640 1 -- 172.21.15.40:0/2338411706 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd7440039b0 con 0x7fd734003970 2023-10-18T09:04:27.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.565+0000 7fd73e7fc640 1 --2- 172.21.15.40:0/2338411706 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd72c06ae70 0x7fd72c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:27.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.565+0000 7fd73e7fc640 1 -- 172.21.15.40:0/2338411706 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd7440abf00 con 0x7fd734003970 2023-10-18T09:04:27.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.565+0000 7fd74d63d640 1 --2- 172.21.15.40:0/2338411706 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd72c06ae70 0x7fd72c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:27.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.566+0000 7fd74d63d640 1 --2- 172.21.15.40:0/2338411706 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd72c06ae70 0x7fd72c06d330 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd74806c120 tx=0x7fd738005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.569+0000 7fd73e7fc640 1 -- 172.21.15.40:0/2338411706 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd744077250 con 0x7fd734003970 2023-10-18T09:04:27.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:27.832+0000 7fd74ee40640 1 -- 172.21.15.40:0/2338411706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7fd748069fd0 con 0x7fd734003970 2023-10-18T09:04:28.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.385+0000 7fd73e7fc640 1 -- 172.21.15.40:0/2338411706 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v482) v1 ==== 135+0+0 (secure 0 0 0) 0x7fd74404b0c0 con 0x7fd734003970 2023-10-18T09:04:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.389+0000 7fd74ee40640 1 -- 172.21.15.40:0/2338411706 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd72c06ae70 msgr2=0x7fd72c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:28.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.389+0000 7fd74ee40640 1 --2- 172.21.15.40:0/2338411706 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd72c06ae70 0x7fd72c06d330 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd74806c120 tx=0x7fd738005f90 comp rx=0 tx=0).stop 2023-10-18T09:04:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.389+0000 7fd74ee40640 1 -- 172.21.15.40:0/2338411706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd734003970 msgr2=0x7fd7481e37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.389+0000 7fd74ee40640 1 --2- 172.21.15.40:0/2338411706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd734003970 0x7fd7481e37a0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd744009b50 tx=0x7fd744033bd0 comp rx=0 tx=0).stop 2023-10-18T09:04:28.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.390+0000 7fd74ee40640 1 -- 172.21.15.40:0/2338411706 shutdown_connections 2023-10-18T09:04:28.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.390+0000 7fd74ee40640 1 --2- 172.21.15.40:0/2338411706 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd72c06ae70 0x7fd72c06d330 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:28.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.390+0000 7fd74ee40640 1 --2- 172.21.15.40:0/2338411706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd748151130 0x7fd7481e4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:28.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.390+0000 7fd74ee40640 1 --2- 172.21.15.40:0/2338411706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd748150cb0 0x7fd7481e3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:28.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.390+0000 7fd74ee40640 1 --2- 172.21.15.40:0/2338411706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd734003970 0x7fd7481e37a0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:28.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.390+0000 7fd74ee40640 1 -- 172.21.15.40:0/2338411706 >> 172.21.15.40:0/2338411706 conn(0x7fd748069720 msgr2=0x7fd748154780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:28.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.390+0000 7fd74ee40640 1 -- 172.21.15.40:0/2338411706 shutdown_connections 2023-10-18T09:04:28.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.390+0000 7fd74ee40640 1 -- 172.21.15.40:0/2338411706 wait complete. 2023-10-18T09:04:28.413 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T09:04:28.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.645+0000 7fa4925f6640 1 -- 172.21.15.40:0/1150975067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa478004d00 msgr2=0x7fa478005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:28.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.645+0000 7fa4925f6640 1 --2- 172.21.15.40:0/1150975067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa478004d00 0x7fa478005180 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fa484001680 tx=0x7fa4780057f0 comp rx=0 tx=0).stop 2023-10-18T09:04:28.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.645+0000 7fa491df5640 1 -- 172.21.15.40:0/1150975067 reap_dead start 2023-10-18T09:04:28.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.645+0000 7fa4925f6640 1 -- 172.21.15.40:0/1150975067 shutdown_connections 2023-10-18T09:04:28.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.645+0000 7fa4925f6640 1 -- 172.21.15.40:0/1150975067 >> 172.21.15.40:0/1150975067 conn(0x7fa48c0aed70 msgr2=0x7fa48c0af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:28.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.645+0000 7fa4925f6640 1 -- 172.21.15.40:0/1150975067 shutdown_connections 2023-10-18T09:04:28.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.645+0000 7fa4925f6640 1 -- 172.21.15.40:0/1150975067 wait complete. 2023-10-18T09:04:28.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.647+0000 7fa4925f6640 1 Processor -- start 2023-10-18T09:04:28.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.647+0000 7fa4925f6640 1 -- start start 2023-10-18T09:04:28.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa4925f6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa478004d00 0x7fa48c140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:28.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa4925f6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c0a9f20 0x7fa48c0aa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:28.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa4925f6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa48c0c1fe0 0x7fa48c0c2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:28.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa4925f6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c0ab6b0 con 0x7fa478004d00 2023-10-18T09:04:28.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa4925f6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c0c29a0 con 0x7fa48c0a9f20 2023-10-18T09:04:28.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa4925f6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c0aaa30 con 0x7fa48c0c1fe0 2023-10-18T09:04:28.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa4915f4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa478004d00 0x7fa48c140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:28.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa4915f4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa478004d00 0x7fa48c140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40226/0 (socket says 172.21.15.40:40226) 2023-10-18T09:04:28.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa4915f4640 1 -- 172.21.15.40:0/1156499528 learned_addr learned my addr 172.21.15.40:0/1156499528 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:28.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa490df3640 1 --2- 172.21.15.40:0/1156499528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c0a9f20 0x7fa48c0aa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:28.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa491df5640 1 --2- 172.21.15.40:0/1156499528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa48c0c1fe0 0x7fa48c0c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:28.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa4915f4640 1 -- 172.21.15.40:0/1156499528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa48c0c1fe0 msgr2=0x7fa48c0c2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:28.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa4915f4640 1 --2- 172.21.15.40:0/1156499528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa48c0c1fe0 0x7fa48c0c2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:28.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa4915f4640 1 -- 172.21.15.40:0/1156499528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c0a9f20 msgr2=0x7fa48c0aa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:28.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa4915f4640 1 --2- 172.21.15.40:0/1156499528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c0a9f20 0x7fa48c0aa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:28.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa4915f4640 1 -- 172.21.15.40:0/1156499528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c0aad10 con 0x7fa478004d00 2023-10-18T09:04:28.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa490df3640 1 --2- 172.21.15.40:0/1156499528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c0a9f20 0x7fa48c0aa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:28.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.648+0000 7fa491df5640 1 --2- 172.21.15.40:0/1156499528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa48c0c1fe0 0x7fa48c0c2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:28.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.649+0000 7fa4915f4640 1 --2- 172.21.15.40:0/1156499528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa478004d00 0x7fa48c140da0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fa4880012e0 tx=0x7fa48800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:28.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.649+0000 7fa481ffb640 1 -- 172.21.15.40:0/1156499528 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa488003970 con 0x7fa478004d00 2023-10-18T09:04:28.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.649+0000 7fa481ffb640 1 -- 172.21.15.40:0/1156499528 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa488003be0 con 0x7fa478004d00 2023-10-18T09:04:28.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.649+0000 7fa4925f6640 1 -- 172.21.15.40:0/1156499528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48c0aaff0 con 0x7fa478004d00 2023-10-18T09:04:28.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.649+0000 7fa481ffb640 1 -- 172.21.15.40:0/1156499528 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48800a890 con 0x7fa478004d00 2023-10-18T09:04:28.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.649+0000 7fa4925f6640 1 -- 172.21.15.40:0/1156499528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c1412e0 con 0x7fa478004d00 2023-10-18T09:04:28.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.650+0000 7fa4925f6640 1 -- 172.21.15.40:0/1156499528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa48c0ab9d0 con 0x7fa478004d00 2023-10-18T09:04:28.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.651+0000 7fa481ffb640 1 -- 172.21.15.40:0/1156499528 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa48800aa10 con 0x7fa478004d00 2023-10-18T09:04:28.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.652+0000 7fa481ffb640 1 --2- 172.21.15.40:0/1156499528 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa47006ae70 0x7fa47006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:28.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.652+0000 7fa481ffb640 1 -- 172.21.15.40:0/1156499528 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa488089160 con 0x7fa478004d00 2023-10-18T09:04:28.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.653+0000 7fa490df3640 1 --2- 172.21.15.40:0/1156499528 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa47006ae70 0x7fa47006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:28.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.654+0000 7fa490df3640 1 --2- 172.21.15.40:0/1156499528 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa47006ae70 0x7fa47006d330 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005fd0 tx=0x7fa47c0094d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:28.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.656+0000 7fa481ffb640 1 -- 172.21.15.40:0/1156499528 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa48c0ab9d0 con 0x7fa478004d00 2023-10-18T09:04:28.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.843+0000 7fa4925f6640 1 -- 172.21.15.40:0/1156499528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7fa48c0ab990 con 0x7fa478004d00 2023-10-18T09:04:28.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.844+0000 7fa481ffb640 1 -- 172.21.15.40:0/1156499528 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v36) v1 ==== 59+0+1452 (secure 0 0 0) 0x7fa488018080 con 0x7fa478004d00 2023-10-18T09:04:28.847 INFO:teuthology.orchestra.run.smithi040.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T09:04:28.847 INFO:teuthology.orchestra.run.smithi040.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T09:04:28.847 INFO:teuthology.orchestra.run.smithi040.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T09:04:28.847 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds 20/20 2023-10-18T09:04:28.848 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T09:04:28.848 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_ms 1/1 2023-10-18T09:04:28.848 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T09:04:28.848 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T09:04:28.849 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T09:04:28.849 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_size 100 2023-10-18T09:04:28.849 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_frag true 2023-10-18T09:04:28.849 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_scatterstat true 2023-10-18T09:04:28.849 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T09:04:28.850 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_verify_scatter true 2023-10-18T09:04:28.850 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T09:04:28.850 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T09:04:28.850 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T09:04:28.850 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced client_mount_timeout 600 2023-10-18T09:04:28.851 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_client 20/20 2023-10-18T09:04:28.851 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_ms 1/1 2023-10-18T09:04:28.851 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T09:04:28.851 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T09:04:28.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.847+0000 7fa4925f6640 1 -- 172.21.15.40:0/1156499528 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa47006ae70 msgr2=0x7fa47006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:28.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.847+0000 7fa4925f6640 1 --2- 172.21.15.40:0/1156499528 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa47006ae70 0x7fa47006d330 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005fd0 tx=0x7fa47c0094d0 comp rx=0 tx=0).stop 2023-10-18T09:04:28.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.847+0000 7fa4925f6640 1 -- 172.21.15.40:0/1156499528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa478004d00 msgr2=0x7fa48c140da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:28.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.847+0000 7fa4925f6640 1 --2- 172.21.15.40:0/1156499528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa478004d00 0x7fa48c140da0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fa4880012e0 tx=0x7fa48800a060 comp rx=0 tx=0).stop 2023-10-18T09:04:28.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.847+0000 7fa4925f6640 1 -- 172.21.15.40:0/1156499528 shutdown_connections 2023-10-18T09:04:28.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.847+0000 7fa4925f6640 1 --2- 172.21.15.40:0/1156499528 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa47006ae70 0x7fa47006d330 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:28.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.847+0000 7fa4925f6640 1 --2- 172.21.15.40:0/1156499528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa48c0c1fe0 0x7fa48c0c2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:28.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.847+0000 7fa4925f6640 1 --2- 172.21.15.40:0/1156499528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c0a9f20 0x7fa48c0aa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:28.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.847+0000 7fa4925f6640 1 --2- 172.21.15.40:0/1156499528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa478004d00 0x7fa48c140da0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:28.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.847+0000 7fa4925f6640 1 -- 172.21.15.40:0/1156499528 >> 172.21.15.40:0/1156499528 conn(0x7fa48c0aed70 msgr2=0x7fa48c1e5500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:28.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.847+0000 7fa4925f6640 1 -- 172.21.15.40:0/1156499528 shutdown_connections 2023-10-18T09:04:28.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:28.847+0000 7fa4925f6640 1 -- 172.21.15.40:0/1156499528 wait complete. 2023-10-18T09:04:28.867 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T09:04:29.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.091+0000 7f8d29705640 1 -- 172.21.15.40:0/3613292200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d24071410 msgr2=0x7f8d240cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:29.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.091+0000 7f8d29705640 1 --2- 172.21.15.40:0/3613292200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d24071410 0x7f8d240cc1c0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c009830 tx=0x7f8d1c02bd60 comp rx=0 tx=0).stop 2023-10-18T09:04:29.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.091+0000 7f8d29705640 1 -- 172.21.15.40:0/3613292200 shutdown_connections 2023-10-18T09:04:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.091+0000 7f8d29705640 1 --2- 172.21.15.40:0/3613292200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d24071410 0x7f8d240cc1c0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.091+0000 7f8d29705640 1 --2- 172.21.15.40:0/3613292200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d2413c550 0x7f8d2413ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.091+0000 7f8d29705640 1 -- 172.21.15.40:0/3613292200 >> 172.21.15.40:0/3613292200 conn(0x7f8d24069720 msgr2=0x7f8d24069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.091+0000 7f8d29705640 1 -- 172.21.15.40:0/3613292200 shutdown_connections 2023-10-18T09:04:29.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.091+0000 7f8d29705640 1 -- 172.21.15.40:0/3613292200 wait complete. 2023-10-18T09:04:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.093+0000 7f8d29705640 1 Processor -- start 2023-10-18T09:04:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.093+0000 7f8d29705640 1 -- start start 2023-10-18T09:04:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.094+0000 7f8d29705640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d24071410 0x7f8d240b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:29.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.094+0000 7f8d29705640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d2413c550 0x7f8d240b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:29.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.094+0000 7f8d29705640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2413f190 0x7f8d240cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:29.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.094+0000 7f8d29705640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d24155d10 con 0x7f8d2413f190 2023-10-18T09:04:29.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.094+0000 7f8d29705640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d240b0e60 con 0x7f8d2413c550 2023-10-18T09:04:29.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.094+0000 7f8d29705640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d240b1cc0 con 0x7f8d24071410 2023-10-18T09:04:29.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.095+0000 7f8d237fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2413f190 0x7f8d240cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:29.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.095+0000 7f8d237fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2413f190 0x7f8d240cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40242/0 (socket says 172.21.15.40:40242) 2023-10-18T09:04:29.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.095+0000 7f8d237fe640 1 -- 172.21.15.40:0/928074805 learned_addr learned my addr 172.21.15.40:0/928074805 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:29.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.095+0000 7f8d22ffd640 1 --2- 172.21.15.40:0/928074805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d24071410 0x7f8d240b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:29.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.095+0000 7f8d237fe640 1 -- 172.21.15.40:0/928074805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d24071410 msgr2=0x7f8d240b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:29.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.095+0000 7f8d227fc640 1 --2- 172.21.15.40:0/928074805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d2413c550 0x7f8d240b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:29.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.095+0000 7f8d237fe640 1 --2- 172.21.15.40:0/928074805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d24071410 0x7f8d240b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:29.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.095+0000 7f8d237fe640 1 -- 172.21.15.40:0/928074805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d2413c550 msgr2=0x7f8d240b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:29.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.095+0000 7f8d237fe640 1 --2- 172.21.15.40:0/928074805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d2413c550 0x7f8d240b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:29.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.095+0000 7f8d237fe640 1 -- 172.21.15.40:0/928074805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c009490 con 0x7f8d2413f190 2023-10-18T09:04:29.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.095+0000 7f8d22ffd640 1 --2- 172.21.15.40:0/928074805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d24071410 0x7f8d240b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:29.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.095+0000 7f8d227fc640 1 --2- 172.21.15.40:0/928074805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d2413c550 0x7f8d240b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:29.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.096+0000 7f8d237fe640 1 --2- 172.21.15.40:0/928074805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2413f190 0x7f8d240cd080 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c02c370 tx=0x7f8d1c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:29.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.096+0000 7f8d1b7fe640 1 -- 172.21.15.40:0/928074805 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c003ea0 con 0x7f8d2413f190 2023-10-18T09:04:29.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.096+0000 7f8d1b7fe640 1 -- 172.21.15.40:0/928074805 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d1c005020 con 0x7f8d2413f190 2023-10-18T09:04:29.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.096+0000 7f8d29705640 1 -- 172.21.15.40:0/928074805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d240b1fa0 con 0x7f8d2413f190 2023-10-18T09:04:29.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.096+0000 7f8d1b7fe640 1 -- 172.21.15.40:0/928074805 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0053b0 con 0x7f8d2413f190 2023-10-18T09:04:29.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.096+0000 7f8d29705640 1 -- 172.21.15.40:0/928074805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d24155590 con 0x7f8d2413f190 2023-10-18T09:04:29.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.097+0000 7f8d29705640 1 -- 172.21.15.40:0/928074805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d24155e50 con 0x7f8d2413f190 2023-10-18T09:04:29.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.098+0000 7f8d1b7fe640 1 -- 172.21.15.40:0/928074805 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8d1c005510 con 0x7f8d2413f190 2023-10-18T09:04:29.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.099+0000 7f8d1b7fe640 1 --2- 172.21.15.40:0/928074805 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d0406ae40 0x7f8d0406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:29.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.099+0000 7f8d1b7fe640 1 -- 172.21.15.40:0/928074805 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8d1c0abc40 con 0x7f8d2413f190 2023-10-18T09:04:29.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.100+0000 7f8d22ffd640 1 --2- 172.21.15.40:0/928074805 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d0406ae40 0x7f8d0406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:29.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.101+0000 7f8d22ffd640 1 --2- 172.21.15.40:0/928074805 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d0406ae40 0x7f8d0406d300 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009b40 tx=0x7f8d14005e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:29.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.103+0000 7f8d1b7fe640 1 -- 172.21.15.40:0/928074805 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8d24155e50 con 0x7f8d2413f190 2023-10-18T09:04:29.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.295+0000 7f8d29705640 1 -- 172.21.15.40:0/928074805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8d24155e50 con 0x7f8d2413f190 2023-10-18T09:04:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.300+0000 7f8d1b7fe640 1 -- 172.21.15.40:0/928074805 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v37)=0 v37) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8d24155e50 con 0x7f8d2413f190 2023-10-18T09:04:29.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.303+0000 7f8d29705640 1 -- 172.21.15.40:0/928074805 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d0406ae40 msgr2=0x7f8d0406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:29.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.303+0000 7f8d29705640 1 --2- 172.21.15.40:0/928074805 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d0406ae40 0x7f8d0406d300 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009b40 tx=0x7f8d14005e50 comp rx=0 tx=0).stop 2023-10-18T09:04:29.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.304+0000 7f8d29705640 1 -- 172.21.15.40:0/928074805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2413f190 msgr2=0x7f8d240cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:29.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.304+0000 7f8d29705640 1 --2- 172.21.15.40:0/928074805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2413f190 0x7f8d240cd080 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c02c370 tx=0x7f8d1c0038a0 comp rx=0 tx=0).stop 2023-10-18T09:04:29.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.304+0000 7f8d29705640 1 -- 172.21.15.40:0/928074805 shutdown_connections 2023-10-18T09:04:29.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.304+0000 7f8d29705640 1 --2- 172.21.15.40:0/928074805 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8d0406ae40 0x7f8d0406d300 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:29.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.304+0000 7f8d29705640 1 --2- 172.21.15.40:0/928074805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d2413f190 0x7f8d240cd080 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:29.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.304+0000 7f8d29705640 1 --2- 172.21.15.40:0/928074805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d2413c550 0x7f8d240b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:29.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.304+0000 7f8d29705640 1 --2- 172.21.15.40:0/928074805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d24071410 0x7f8d240b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:29.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.304+0000 7f8d29705640 1 -- 172.21.15.40:0/928074805 >> 172.21.15.40:0/928074805 conn(0x7f8d24069720 msgr2=0x7f8d240ad9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:29.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.304+0000 7f8d29705640 1 -- 172.21.15.40:0/928074805 shutdown_connections 2023-10-18T09:04:29.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:29.304+0000 7f8d29705640 1 -- 172.21.15.40:0/928074805 wait complete. 2023-10-18T09:04:29.324 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T09:04:29.325 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:04:29.455 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:04:29.585 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T09:04:29.586 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:04:29.586 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:04:29.649 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:04:29.650 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:04:29.650 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:04:29.711 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:04:29.712 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:04:29.712 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:04:29.776 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:04:29.777 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:04:29.777 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:04:29.838 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:04:29.839 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T09:04:29.839 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:04:30.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.075+0000 7f9f42fb0640 1 -- 172.21.15.40:0/1309696371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f3c0cc990 msgr2=0x7f9f3c0ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:30.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.075+0000 7f9f42fb0640 1 --2- 172.21.15.40:0/1309696371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f3c0cc990 0x7f9f3c0ccde0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9f34005fd0 tx=0x7f9f3402c2e0 comp rx=0 tx=0).stop 2023-10-18T09:04:30.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.075+0000 7f9f42fb0640 1 -- 172.21.15.40:0/1309696371 shutdown_connections 2023-10-18T09:04:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.075+0000 7f9f42fb0640 1 --2- 172.21.15.40:0/1309696371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f3c0cc990 0x7f9f3c0ccde0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.075+0000 7f9f42fb0640 1 --2- 172.21.15.40:0/1309696371 >> v2:172.21.15.144:3300/0 conn(0x7f9f3c0b0bc0 0x7f9f3c0b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.075+0000 7f9f42fb0640 1 -- 172.21.15.40:0/1309696371 >> 172.21.15.40:0/1309696371 conn(0x7f9f3c06cee0 msgr2=0x7f9f3c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.075+0000 7f9f42fb0640 1 -- 172.21.15.40:0/1309696371 shutdown_connections 2023-10-18T09:04:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.076+0000 7f9f42fb0640 1 -- 172.21.15.40:0/1309696371 wait complete. 2023-10-18T09:04:30.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.078+0000 7f9f42fb0640 1 Processor -- start 2023-10-18T09:04:30.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.078+0000 7f9f42fb0640 1 -- start start 2023-10-18T09:04:30.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.079+0000 7f9f42fb0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f3c0b0bc0 0x7f9f3c1f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.079+0000 7f9f42fb0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f3c0cc990 0x7f9f3c14adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.079+0000 7f9f42fb0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f3c151bd0 0x7f9f3c159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.079+0000 7f9f42fb0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c1f52f0 con 0x7f9f3c0cc990 2023-10-18T09:04:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.079+0000 7f9f42fb0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c1f6210 con 0x7f9f3c151bd0 2023-10-18T09:04:30.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.079+0000 7f9f42fb0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c15a340 con 0x7f9f3c0b0bc0 2023-10-18T09:04:30.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.079+0000 7f9f33fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f3c0cc990 0x7f9f3c14adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:30.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.079+0000 7f9f33fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f3c0cc990 0x7f9f3c14adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40248/0 (socket says 172.21.15.40:40248) 2023-10-18T09:04:30.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.079+0000 7f9f33fff640 1 -- 172.21.15.40:0/2922465376 learned_addr learned my addr 172.21.15.40:0/2922465376 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:30.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.080+0000 7f9f4154d640 1 --2- 172.21.15.40:0/2922465376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f3c151bd0 0x7f9f3c159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:30.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.080+0000 7f9f40d4c640 1 --2- 172.21.15.40:0/2922465376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f3c0b0bc0 0x7f9f3c1f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:30.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.080+0000 7f9f33fff640 1 -- 172.21.15.40:0/2922465376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f3c0b0bc0 msgr2=0x7f9f3c1f5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:30.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.080+0000 7f9f33fff640 1 --2- 172.21.15.40:0/2922465376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f3c0b0bc0 0x7f9f3c1f5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:30.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.080+0000 7f9f33fff640 1 -- 172.21.15.40:0/2922465376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f3c151bd0 msgr2=0x7f9f3c159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:30.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.080+0000 7f9f33fff640 1 --2- 172.21.15.40:0/2922465376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f3c151bd0 0x7f9f3c159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:30.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.080+0000 7f9f33fff640 1 -- 172.21.15.40:0/2922465376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f34009900 con 0x7f9f3c0cc990 2023-10-18T09:04:30.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.080+0000 7f9f4154d640 1 --2- 172.21.15.40:0/2922465376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f3c151bd0 0x7f9f3c159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:04:30.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.080+0000 7f9f33fff640 1 --2- 172.21.15.40:0/2922465376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f3c0cc990 0x7f9f3c14adc0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00e9f0 tx=0x7f9f2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:30.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.081+0000 7f9f317fa640 1 -- 172.21.15.40:0/2922465376 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c009700 con 0x7f9f3c0cc990 2023-10-18T09:04:30.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.081+0000 7f9f317fa640 1 -- 172.21.15.40:0/2922465376 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9f2c00ce70 con 0x7f9f3c0cc990 2023-10-18T09:04:30.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.081+0000 7f9f317fa640 1 -- 172.21.15.40:0/2922465376 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c010670 con 0x7f9f3c0cc990 2023-10-18T09:04:30.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.081+0000 7f9f42fb0640 1 -- 172.21.15.40:0/2922465376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f3c15a620 con 0x7f9f3c0cc990 2023-10-18T09:04:30.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.081+0000 7f9f42fb0640 1 -- 172.21.15.40:0/2922465376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f3c1f57b0 con 0x7f9f3c0cc990 2023-10-18T09:04:30.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.082+0000 7f9f42fb0640 1 -- 172.21.15.40:0/2922465376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f3c1f5970 con 0x7f9f3c0cc990 2023-10-18T09:04:30.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.083+0000 7f9f317fa640 1 -- 172.21.15.40:0/2922465376 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9f2c0107d0 con 0x7f9f3c0cc990 2023-10-18T09:04:30.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.084+0000 7f9f317fa640 1 --2- 172.21.15.40:0/2922465376 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9f2806ae70 0x7f9f2806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:30.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.084+0000 7f9f317fa640 1 -- 172.21.15.40:0/2922465376 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9f2c014070 con 0x7f9f3c0cc990 2023-10-18T09:04:30.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.085+0000 7f9f40d4c640 1 --2- 172.21.15.40:0/2922465376 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9f2806ae70 0x7f9f2806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:30.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.086+0000 7f9f40d4c640 1 --2- 172.21.15.40:0/2922465376 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9f2806ae70 0x7f9f2806d330 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9f38000f60 tx=0x7f9f38009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:30.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.088+0000 7f9f317fa640 1 -- 172.21.15.40:0/2922465376 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9f2c051ca0 con 0x7f9f3c0cc990 2023-10-18T09:04:30.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.320+0000 7f9f42fb0640 1 -- 172.21.15.40:0/2922465376 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f3c15a860 con 0x7f9f3c0cc990 2023-10-18T09:04:30.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.321+0000 7f9f317fa640 1 -- 172.21.15.40:0/2922465376 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f9f2c058dd0 con 0x7f9f3c0cc990 2023-10-18T09:04:30.324 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:30.324 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":68,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6382,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/4165164542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":4165164542},{"type":"v1","addr":"172.21.15.144:6836","nonce":4165164542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6385,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3045960359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3045960359},{"type":"v1","addr":"172.21.15.144:6837","nonce":3045960359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2869040025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2869040025},{"type":"v1","addr":"172.21.15.40:6833","nonce":2869040025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8310,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/184921015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":184921015},{"type":"v1","addr":"172.21.15.40:6835","nonce":184921015}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[]} 2023-10-18T09:04:30.324 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 68 2023-10-18T09:04:30.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.324+0000 7f9f42fb0640 1 -- 172.21.15.40:0/2922465376 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9f2806ae70 msgr2=0x7f9f2806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:30.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.324+0000 7f9f42fb0640 1 --2- 172.21.15.40:0/2922465376 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9f2806ae70 0x7f9f2806d330 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9f38000f60 tx=0x7f9f38009040 comp rx=0 tx=0).stop 2023-10-18T09:04:30.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.324+0000 7f9f42fb0640 1 -- 172.21.15.40:0/2922465376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f3c0cc990 msgr2=0x7f9f3c14adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:30.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.324+0000 7f9f42fb0640 1 --2- 172.21.15.40:0/2922465376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f3c0cc990 0x7f9f3c14adc0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00e9f0 tx=0x7f9f2c00eeb0 comp rx=0 tx=0).stop 2023-10-18T09:04:30.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.324+0000 7f9f42fb0640 1 -- 172.21.15.40:0/2922465376 shutdown_connections 2023-10-18T09:04:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.324+0000 7f9f42fb0640 1 --2- 172.21.15.40:0/2922465376 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9f2806ae70 0x7f9f2806d330 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.324+0000 7f9f42fb0640 1 --2- 172.21.15.40:0/2922465376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f3c151bd0 0x7f9f3c159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.324+0000 7f9f42fb0640 1 --2- 172.21.15.40:0/2922465376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9f3c0cc990 0x7f9f3c14adc0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.324+0000 7f9f42fb0640 1 --2- 172.21.15.40:0/2922465376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9f3c0b0bc0 0x7f9f3c1f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.324+0000 7f9f42fb0640 1 -- 172.21.15.40:0/2922465376 >> 172.21.15.40:0/2922465376 conn(0x7f9f3c06cee0 msgr2=0x7f9f3c14fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:30.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.324+0000 7f9f42fb0640 1 -- 172.21.15.40:0/2922465376 shutdown_connections 2023-10-18T09:04:30.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.324+0000 7f9f42fb0640 1 -- 172.21.15.40:0/2922465376 wait complete. 2023-10-18T09:04:30.346 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T09:04:30.347 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T09:04:30.347 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T09:04:30.348 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:04:30.348 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T09:04:30.348 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T09:04:30.348 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T09:04:30.349 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:04:30.349 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T09:04:30.349 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T09:04:30.350 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T09:04:30.350 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:04:30.350 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T09:04:30.350 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T09:04:30.351 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T09:04:30.351 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:04:30.364 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T09:04:30.364 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T09:04:30.365 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T09:04:30.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T09:04:30.369 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T09:04:30.370 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T09:04:30.375 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T09:04:30.375 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T09:04:30.413 INFO:tasks.ceph.mds.a:Started 2023-10-18T09:04:30.413 DEBUG:teuthology.parallel:result is None 2023-10-18T09:04:30.420 INFO:tasks.ceph.mds.c:Started 2023-10-18T09:04:30.420 DEBUG:teuthology.parallel:result is None 2023-10-18T09:04:30.423 INFO:tasks.ceph.mds.d:Started 2023-10-18T09:04:30.424 DEBUG:teuthology.parallel:result is None 2023-10-18T09:04:30.431 INFO:tasks.ceph.mds.b:Started 2023-10-18T09:04:30.431 DEBUG:teuthology.parallel:result is None 2023-10-18T09:04:30.432 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T09:04:30.579 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T09:04:30.582 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-10-18T09:04:30.584 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-18T09:04:30.591 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-18T09:04:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.633+0000 7ff79f368640 1 -- 172.21.15.40:0/86385722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7980b1b90 msgr2=0x7ff7980b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.633+0000 7ff79f368640 1 --2- 172.21.15.40:0/86385722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7980b1b90 0x7ff7980b07d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff790005fd0 tx=0x7ff79002c170 comp rx=0 tx=0).stop 2023-10-18T09:04:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.633+0000 7ff79f368640 1 -- 172.21.15.40:0/86385722 shutdown_connections 2023-10-18T09:04:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.633+0000 7ff79f368640 1 --2- 172.21.15.40:0/86385722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7980b1b90 0x7ff7980b07d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:30.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.633+0000 7ff79f368640 1 --2- 172.21.15.40:0/86385722 >> v2:172.21.15.144:3300/0 conn(0x7ff7980cd4c0 0x7ff7980b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:30.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.633+0000 7ff79f368640 1 -- 172.21.15.40:0/86385722 >> 172.21.15.40:0/86385722 conn(0x7ff79806cee0 msgr2=0x7ff798069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:30.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.633+0000 7ff79f368640 1 -- 172.21.15.40:0/86385722 shutdown_connections 2023-10-18T09:04:30.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.633+0000 7ff79f368640 1 -- 172.21.15.40:0/86385722 wait complete. 2023-10-18T09:04:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.635+0000 7ff79f368640 1 Processor -- start 2023-10-18T09:04:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.635+0000 7ff79f368640 1 -- start start 2023-10-18T09:04:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.636+0000 7ff79f368640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7980b1b90 0x7ff79814bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.636+0000 7ff79f368640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7980cd4c0 0x7ff7980cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.636+0000 7ff79f368640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff79815eaa0 0x7ff7980cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.636+0000 7ff79f368640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980abf70 con 0x7ff79815eaa0 2023-10-18T09:04:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.636+0000 7ff79f368640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79814c200 con 0x7ff7980cd4c0 2023-10-18T09:04:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.636+0000 7ff79f368640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980ab080 con 0x7ff7980b1b90 2023-10-18T09:04:30.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.636+0000 7ff79d905640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff79815eaa0 0x7ff7980cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:30.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.636+0000 7ff79d905640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff79815eaa0 0x7ff7980cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40282/0 (socket says 172.21.15.40:40282) 2023-10-18T09:04:30.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.636+0000 7ff79d905640 1 -- 172.21.15.40:0/3278850962 learned_addr learned my addr 172.21.15.40:0/3278850962 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:30.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.637+0000 7ff79c903640 1 --2- 172.21.15.40:0/3278850962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7980cd4c0 0x7ff7980cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:30.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.637+0000 7ff79d104640 1 --2- 172.21.15.40:0/3278850962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7980b1b90 0x7ff79814bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:30.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.637+0000 7ff79d905640 1 -- 172.21.15.40:0/3278850962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7980b1b90 msgr2=0x7ff79814bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:30.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.637+0000 7ff79d905640 1 --2- 172.21.15.40:0/3278850962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7980b1b90 0x7ff79814bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:30.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.637+0000 7ff79d905640 1 -- 172.21.15.40:0/3278850962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7980cd4c0 msgr2=0x7ff7980cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:30.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.637+0000 7ff79d905640 1 --2- 172.21.15.40:0/3278850962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7980cd4c0 0x7ff7980cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:30.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.637+0000 7ff79d905640 1 -- 172.21.15.40:0/3278850962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff790009840 con 0x7ff79815eaa0 2023-10-18T09:04:30.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.637+0000 7ff79d104640 1 --2- 172.21.15.40:0/3278850962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7980b1b90 0x7ff79814bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:30.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.637+0000 7ff79d905640 1 --2- 172.21.15.40:0/3278850962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff79815eaa0 0x7ff7980cb650 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7ff790036040 tx=0x7ff790004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:30.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.637+0000 7ff78dffb640 1 -- 172.21.15.40:0/3278850962 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790004440 con 0x7ff79815eaa0 2023-10-18T09:04:30.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.638+0000 7ff78dffb640 1 -- 172.21.15.40:0/3278850962 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff790004690 con 0x7ff79815eaa0 2023-10-18T09:04:30.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.638+0000 7ff78dffb640 1 -- 172.21.15.40:0/3278850962 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7900345a0 con 0x7ff79815eaa0 2023-10-18T09:04:30.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.638+0000 7ff79f368640 1 -- 172.21.15.40:0/3278850962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7980ab360 con 0x7ff79815eaa0 2023-10-18T09:04:30.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.638+0000 7ff79f368640 1 -- 172.21.15.40:0/3278850962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7980ab920 con 0x7ff79815eaa0 2023-10-18T09:04:30.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.639+0000 7ff79f368640 1 -- 172.21.15.40:0/3278850962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff798066fd0 con 0x7ff79815eaa0 2023-10-18T09:04:30.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.640+0000 7ff78dffb640 1 -- 172.21.15.40:0/3278850962 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff790034700 con 0x7ff79815eaa0 2023-10-18T09:04:30.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.641+0000 7ff78dffb640 1 --2- 172.21.15.40:0/3278850962 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff77c06ae40 0x7ff77c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:30.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.641+0000 7ff78dffb640 1 -- 172.21.15.40:0/3278850962 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff7900ac370 con 0x7ff79815eaa0 2023-10-18T09:04:30.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.641+0000 7ff79d104640 1 --2- 172.21.15.40:0/3278850962 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff77c06ae40 0x7ff77c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:30.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.642+0000 7ff79d104640 1 --2- 172.21.15.40:0/3278850962 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff77c06ae40 0x7ff77c06d300 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff7940011e0 tx=0x7ff794009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:30.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.645+0000 7ff78dffb640 1 -- 172.21.15.40:0/3278850962 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff790077680 con 0x7ff79815eaa0 2023-10-18T09:04:30.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:30.845+0000 7ff79f368640 1 -- 172.21.15.40:0/3278850962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff7980b15b0 con 0x7ff79815eaa0 2023-10-18T09:04:32.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.124+0000 7ff78dffb640 1 -- 172.21.15.40:0/3278850962 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v158) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff79007aca0 con 0x7ff79815eaa0 2023-10-18T09:04:32.127 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2023-10-18T09:04:32.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.128+0000 7ff79f368640 1 -- 172.21.15.40:0/3278850962 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff77c06ae40 msgr2=0x7ff77c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:32.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.128+0000 7ff79f368640 1 --2- 172.21.15.40:0/3278850962 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff77c06ae40 0x7ff77c06d300 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff7940011e0 tx=0x7ff794009040 comp rx=0 tx=0).stop 2023-10-18T09:04:32.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.128+0000 7ff79f368640 1 -- 172.21.15.40:0/3278850962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff79815eaa0 msgr2=0x7ff7980cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:32.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.128+0000 7ff79f368640 1 --2- 172.21.15.40:0/3278850962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff79815eaa0 0x7ff7980cb650 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7ff790036040 tx=0x7ff790004040 comp rx=0 tx=0).stop 2023-10-18T09:04:32.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.128+0000 7ff79f368640 1 -- 172.21.15.40:0/3278850962 shutdown_connections 2023-10-18T09:04:32.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.128+0000 7ff79f368640 1 --2- 172.21.15.40:0/3278850962 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff77c06ae40 0x7ff77c06d300 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:32.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.128+0000 7ff79f368640 1 --2- 172.21.15.40:0/3278850962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff79815eaa0 0x7ff7980cb650 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:32.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.128+0000 7ff79f368640 1 --2- 172.21.15.40:0/3278850962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7980cd4c0 0x7ff7980cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:32.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.128+0000 7ff79f368640 1 --2- 172.21.15.40:0/3278850962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7980b1b90 0x7ff79814bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:32.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.128+0000 7ff79f368640 1 -- 172.21.15.40:0/3278850962 >> 172.21.15.40:0/3278850962 conn(0x7ff79806cee0 msgr2=0x7ff7980b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:32.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.129+0000 7ff79f368640 1 -- 172.21.15.40:0/3278850962 shutdown_connections 2023-10-18T09:04:32.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.129+0000 7ff79f368640 1 -- 172.21.15.40:0/3278850962 wait complete. 2023-10-18T09:04:32.153 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T09:04:32.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.386+0000 7f7462c8a640 1 -- 172.21.15.40:0/1948388260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c06d6f0 msgr2=0x7f745c0b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:32.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.386+0000 7f7462c8a640 1 --2- 172.21.15.40:0/1948388260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c06d6f0 0x7f745c0b15b0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f7458005fd0 tx=0x7f745802be10 comp rx=0 tx=0).stop 2023-10-18T09:04:32.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.386+0000 7f7462c8a640 1 -- 172.21.15.40:0/1948388260 shutdown_connections 2023-10-18T09:04:32.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.386+0000 7f7462c8a640 1 --2- 172.21.15.40:0/1948388260 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f745c0ca0d0 0x7f745c157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:32.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.386+0000 7f7462c8a640 1 --2- 172.21.15.40:0/1948388260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c06d6f0 0x7f745c0b15b0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:32.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.386+0000 7f7462c8a640 1 -- 172.21.15.40:0/1948388260 >> 172.21.15.40:0/1948388260 conn(0x7f745c069720 msgr2=0x7f745c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:32.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.386+0000 7f7462c8a640 1 -- 172.21.15.40:0/1948388260 shutdown_connections 2023-10-18T09:04:32.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.386+0000 7f7462c8a640 1 -- 172.21.15.40:0/1948388260 wait complete. 2023-10-18T09:04:32.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.388+0000 7f7462c8a640 1 Processor -- start 2023-10-18T09:04:32.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.388+0000 7f7462c8a640 1 -- start start 2023-10-18T09:04:32.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.389+0000 7f7462c8a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f745c06d6f0 0x7f745c13b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:32.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.389+0000 7f7462c8a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f745c0b0630 0x7f745c14ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:32.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.389+0000 7f7462c8a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c0ca0d0 0x7f745c13be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:32.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.389+0000 7f7462c8a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c1567c0 con 0x7f745c0ca0d0 2023-10-18T09:04:32.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.389+0000 7f7462c8a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c155bd0 con 0x7f745c06d6f0 2023-10-18T09:04:32.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.389+0000 7f7462c8a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745c155e90 con 0x7f745c0b0630 2023-10-18T09:04:32.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.389+0000 7f7462489640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c0ca0d0 0x7f745c13be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:32.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.389+0000 7f7462489640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c0ca0d0 0x7f745c13be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40312/0 (socket says 172.21.15.40:40312) 2023-10-18T09:04:32.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.389+0000 7f7462489640 1 -- 172.21.15.40:0/2582810482 learned_addr learned my addr 172.21.15.40:0/2582810482 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:32.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.390+0000 7f7461487640 1 --2- 172.21.15.40:0/2582810482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f745c0b0630 0x7f745c14ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:32.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.390+0000 7f7462489640 1 -- 172.21.15.40:0/2582810482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f745c0b0630 msgr2=0x7f745c14ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:32.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.390+0000 7f7461c88640 1 --2- 172.21.15.40:0/2582810482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f745c06d6f0 0x7f745c13b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:32.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.390+0000 7f7462489640 1 --2- 172.21.15.40:0/2582810482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f745c0b0630 0x7f745c14ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:32.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.390+0000 7f7462489640 1 -- 172.21.15.40:0/2582810482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f745c06d6f0 msgr2=0x7f745c13b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:32.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.390+0000 7f7462489640 1 --2- 172.21.15.40:0/2582810482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f745c06d6f0 0x7f745c13b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:32.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.390+0000 7f7462489640 1 -- 172.21.15.40:0/2582810482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7458009560 con 0x7f745c0ca0d0 2023-10-18T09:04:32.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.390+0000 7f7461487640 1 --2- 172.21.15.40:0/2582810482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f745c0b0630 0x7f745c14ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:32.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.390+0000 7f7462489640 1 --2- 172.21.15.40:0/2582810482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c0ca0d0 0x7f745c13be90 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f745400ddc0 tx=0x7f745400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:32.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.390+0000 7f74527fc640 1 -- 172.21.15.40:0/2582810482 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7454004d20 con 0x7f745c0ca0d0 2023-10-18T09:04:32.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.391+0000 7f74527fc640 1 -- 172.21.15.40:0/2582810482 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f745400be40 con 0x7f745c0ca0d0 2023-10-18T09:04:32.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.391+0000 7f74527fc640 1 -- 172.21.15.40:0/2582810482 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745400e8c0 con 0x7f745c0ca0d0 2023-10-18T09:04:32.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.391+0000 7f7462c8a640 1 -- 172.21.15.40:0/2582810482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745c156170 con 0x7f745c0ca0d0 2023-10-18T09:04:32.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.391+0000 7f7462c8a640 1 -- 172.21.15.40:0/2582810482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745c13bb30 con 0x7f745c0ca0d0 2023-10-18T09:04:32.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.392+0000 7f7462c8a640 1 -- 172.21.15.40:0/2582810482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745c156970 con 0x7f745c0ca0d0 2023-10-18T09:04:32.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.393+0000 7f74527fc640 1 -- 172.21.15.40:0/2582810482 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f745400ea20 con 0x7f745c0ca0d0 2023-10-18T09:04:32.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.394+0000 7f74527fc640 1 --2- 172.21.15.40:0/2582810482 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f744006ae70 0x7f744006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:32.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.394+0000 7f74527fc640 1 -- 172.21.15.40:0/2582810482 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7454089e40 con 0x7f745c0ca0d0 2023-10-18T09:04:32.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.394+0000 7f7461c88640 1 --2- 172.21.15.40:0/2582810482 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f744006ae70 0x7f744006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:32.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.395+0000 7f7461c88640 1 --2- 172.21.15.40:0/2582810482 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f744006ae70 0x7f744006d330 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7458035000 tx=0x7f7458004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:32.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.398+0000 7f74527fc640 1 -- 172.21.15.40:0/2582810482 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f745c156970 con 0x7f745c0ca0d0 2023-10-18T09:04:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.668+0000 7f7462c8a640 1 -- 172.21.15.40:0/2582810482 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f745c066e00 con 0x7f745c0ca0d0 2023-10-18T09:04:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.669+0000 7f74527fc640 1 -- 172.21.15.40:0/2582810482 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7454055110 con 0x7f745c0ca0d0 2023-10-18T09:04:32.672 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:32.672 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:04:32.672 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2023-10-18T09:04:32.672 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.672 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2023-10-18T09:04:32.673 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lYsb5CxAAGButZ6+Usc62kJ0ksxZrYw==", 2023-10-18T09:04:32.673 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.673 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:04:32.673 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:04:32.673 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:04:32.674 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:04:32.674 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.674 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.674 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.674 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2023-10-18T09:04:32.675 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lAqlhMBAApAhxCYB9d9bz1GibADDRSQ==", 2023-10-18T09:04:32.675 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.675 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:04:32.675 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:04:32.675 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:04:32.675 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:04:32.676 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.676 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.676 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.676 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2023-10-18T09:04:32.676 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lPVh+HhAAzucSx+ksPDBkqJGa8+pyJw==", 2023-10-18T09:04:32.677 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.677 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:04:32.677 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:04:32.677 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:04:32.677 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:04:32.678 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.678 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.678 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.678 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2023-10-18T09:04:32.678 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lP7QCBhAAog7qDCgGVv0T7WKsRkATkA==", 2023-10-18T09:04:32.679 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.679 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:04:32.679 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:04:32.679 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:04:32.679 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:04:32.679 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.680 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.680 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.680 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2023-10-18T09:04:32.680 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lVx+hAhAA6ZpIyZKhbqy8tV84piGILA==", 2023-10-18T09:04:32.680 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.681 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:04:32.681 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:04:32.681 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:04:32.681 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.681 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.682 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.682 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2023-10-18T09:04:32.682 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lk9LwExAA7zFzQOn2hcvnck2cOZZ7AA==", 2023-10-18T09:04:32.682 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.682 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:04:32.683 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:04:32.683 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:04:32.683 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.683 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.683 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.684 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2023-10-18T09:04:32.684 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lK930JRAA1GyKRkIpXPHHGqY8owYWRQ==", 2023-10-18T09:04:32.684 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.684 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:04:32.684 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:04:32.684 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:04:32.685 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.685 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.685 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.685 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2023-10-18T09:04:32.685 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9liBWyNRAAMHBBdG5nbrGQupqvUtuKMA==", 2023-10-18T09:04:32.686 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.686 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:04:32.686 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:04:32.686 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:04:32.686 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.687 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.687 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2023-10-18T09:04:32.687 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9loDoOAhAAjHZub4wnB7dCr86rxAtI5g==", 2023-10-18T09:04:32.687 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.688 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:04:32.688 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:04:32.688 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:04:32.688 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.688 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.688 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.689 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2023-10-18T09:04:32.689 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lhqNfERAAw410g3DwSXrK23YkxVBFVw==", 2023-10-18T09:04:32.689 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.689 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:04:32.689 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:04:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:04:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2023-10-18T09:04:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lRRScIRAAg0pmfxCQBCPYodj1Lko+Lw==", 2023-10-18T09:04:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:04:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:04:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:04:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2023-10-18T09:04:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9ld1dXMhAAyovlrKOhVuynEiuPTgFcjQ==", 2023-10-18T09:04:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:04:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:04:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:04:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2023-10-18T09:04:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w==", 2023-10-18T09:04:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:04:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2023-10-18T09:04:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T09:04:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2023-10-18T09:04:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lAB3oKhAAD43WFdC23WGF6WUi4JKsvg==", 2023-10-18T09:04:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:04:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:04:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:04:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:04:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2023-10-18T09:04:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lu57fHxAAdzsVU2KKsWRkaARRS8TGjw==", 2023-10-18T09:04:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:04:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:04:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:04:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:04:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2023-10-18T09:04:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lygWsNRAA1OziXxk7hBtTw2Aq8OlSKQ==", 2023-10-18T09:04:32.700 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.700 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:04:32.700 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:04:32.700 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:04:32.700 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:04:32.701 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.701 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.701 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.701 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2023-10-18T09:04:32.701 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBMnS9lnrUjNhAAHP2eezRHF4/Jgox7iPwy7g==", 2023-10-18T09:04:32.702 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.702 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:04:32.702 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:04:32.702 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:04:32.702 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:04:32.703 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.703 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.703 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2023-10-18T09:04:32.703 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9lCf/xOhAAJHVoKIswWUcVGKQpiaJ12A==", 2023-10-18T09:04:32.703 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.704 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:04:32.704 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T09:04:32.704 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:04:32.704 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:04:32.704 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:32.705 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2023-10-18T09:04:32.705 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9loxRXLhAAQK1n1/4PB35umHxOKQzz7Q==", 2023-10-18T09:04:32.705 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:04:32.705 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:04:32.705 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T09:04:32.706 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:04:32.706 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.706 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:32.706 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:04:32.706 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:04:32.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.672+0000 7f7462c8a640 1 -- 172.21.15.40:0/2582810482 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f744006ae70 msgr2=0x7f744006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:32.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.672+0000 7f7462c8a640 1 --2- 172.21.15.40:0/2582810482 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f744006ae70 0x7f744006d330 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7458035000 tx=0x7f7458004730 comp rx=0 tx=0).stop 2023-10-18T09:04:32.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.672+0000 7f7462c8a640 1 -- 172.21.15.40:0/2582810482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c0ca0d0 msgr2=0x7f745c13be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:32.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.672+0000 7f7462c8a640 1 --2- 172.21.15.40:0/2582810482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c0ca0d0 0x7f745c13be90 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f745400ddc0 tx=0x7f745400b730 comp rx=0 tx=0).stop 2023-10-18T09:04:32.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.672+0000 7f7462c8a640 1 -- 172.21.15.40:0/2582810482 shutdown_connections 2023-10-18T09:04:32.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.672+0000 7f7462c8a640 1 --2- 172.21.15.40:0/2582810482 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f744006ae70 0x7f744006d330 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:32.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.672+0000 7f7462c8a640 1 --2- 172.21.15.40:0/2582810482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f745c0ca0d0 0x7f745c13be90 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:32.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.672+0000 7f7462c8a640 1 --2- 172.21.15.40:0/2582810482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f745c0b0630 0x7f745c14ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:32.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.672+0000 7f7462c8a640 1 --2- 172.21.15.40:0/2582810482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f745c06d6f0 0x7f745c13b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:32.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.672+0000 7f7462c8a640 1 -- 172.21.15.40:0/2582810482 >> 172.21.15.40:0/2582810482 conn(0x7f745c069720 msgr2=0x7f745c159d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:32.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.672+0000 7f7462c8a640 1 -- 172.21.15.40:0/2582810482 shutdown_connections 2023-10-18T09:04:32.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.672+0000 7f7462c8a640 1 -- 172.21.15.40:0/2582810482 wait complete. 2023-10-18T09:04:32.711 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:04:32.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.933+0000 7f66da610640 1 -- 172.21.15.40:0/3486708433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d406e3b0 msgr2=0x7f66d415b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:32.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.933+0000 7f66da610640 1 --2- 172.21.15.40:0/3486708433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d406e3b0 0x7f66d415b9e0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f66c8005fd0 tx=0x7f66c802bd10 comp rx=0 tx=0).stop 2023-10-18T09:04:32.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.934+0000 7f66da610640 1 -- 172.21.15.40:0/3486708433 shutdown_connections 2023-10-18T09:04:32.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.934+0000 7f66da610640 1 --2- 172.21.15.40:0/3486708433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d406e3b0 0x7f66d415b9e0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:32.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.934+0000 7f66da610640 1 -- 172.21.15.40:0/3486708433 >> 172.21.15.40:0/3486708433 conn(0x7f66d40655f0 msgr2=0x7f66d40659d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:32.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.934+0000 7f66da610640 1 -- 172.21.15.40:0/3486708433 shutdown_connections 2023-10-18T09:04:32.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.934+0000 7f66da610640 1 -- 172.21.15.40:0/3486708433 wait complete. 2023-10-18T09:04:32.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.935+0000 7f66da610640 1 Processor -- start 2023-10-18T09:04:32.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.936+0000 7f66da610640 1 -- start start 2023-10-18T09:04:32.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.936+0000 7f66da610640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66d406e3b0 0x7f66d40c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:32.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.936+0000 7f66da610640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d40cfd70 0x7f66d40cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:32.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.936+0000 7f66da610640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66d41590f0 0x7f66d415dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:32.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.936+0000 7f66da610640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d4151d70 con 0x7f66d40cfd70 2023-10-18T09:04:32.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.936+0000 7f66da610640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d40c8d30 con 0x7f66d406e3b0 2023-10-18T09:04:32.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.937+0000 7f66da610640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d4152010 con 0x7f66d41590f0 2023-10-18T09:04:32.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.937+0000 7f66d37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d40cfd70 0x7f66d40cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:32.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.937+0000 7f66d37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d40cfd70 0x7f66d40cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40336/0 (socket says 172.21.15.40:40336) 2023-10-18T09:04:32.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.937+0000 7f66d37fe640 1 -- 172.21.15.40:0/1873168065 learned_addr learned my addr 172.21.15.40:0/1873168065 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:32.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.937+0000 7f66d3fff640 1 --2- 172.21.15.40:0/1873168065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66d406e3b0 0x7f66d40c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:32.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.937+0000 7f66d8bad640 1 --2- 172.21.15.40:0/1873168065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66d41590f0 0x7f66d415dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:32.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.937+0000 7f66d37fe640 1 -- 172.21.15.40:0/1873168065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66d41590f0 msgr2=0x7f66d415dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:32.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.937+0000 7f66d37fe640 1 --2- 172.21.15.40:0/1873168065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66d41590f0 0x7f66d415dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:32.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.937+0000 7f66d37fe640 1 -- 172.21.15.40:0/1873168065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66d406e3b0 msgr2=0x7f66d40c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.937+0000 7f66d37fe640 1 --2- 172.21.15.40:0/1873168065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66d406e3b0 0x7f66d40c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.937+0000 7f66d37fe640 1 -- 172.21.15.40:0/1873168065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c8009560 con 0x7f66d40cfd70 2023-10-18T09:04:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.937+0000 7f66d3fff640 1 --2- 172.21.15.40:0/1873168065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66d406e3b0 0x7f66d40c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.938+0000 7f66d37fe640 1 --2- 172.21.15.40:0/1873168065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d40cfd70 0x7f66d40cac90 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f66c8036040 tx=0x7f66c802dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.938+0000 7f66d0ff9640 1 -- 172.21.15.40:0/1873168065 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c8003800 con 0x7f66d40cfd70 2023-10-18T09:04:32.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.938+0000 7f66d0ff9640 1 -- 172.21.15.40:0/1873168065 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f66c8003a50 con 0x7f66d40cfd70 2023-10-18T09:04:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.938+0000 7f66d0ff9640 1 -- 172.21.15.40:0/1873168065 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c8003e20 con 0x7f66d40cfd70 2023-10-18T09:04:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.938+0000 7f66da610640 1 -- 172.21.15.40:0/1873168065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d41522f0 con 0x7f66d40cfd70 2023-10-18T09:04:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.938+0000 7f66da610640 1 -- 172.21.15.40:0/1873168065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d40c8970 con 0x7f66d40cfd70 2023-10-18T09:04:32.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.939+0000 7f66da610640 1 -- 172.21.15.40:0/1873168065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d40663a0 con 0x7f66d40cfd70 2023-10-18T09:04:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.940+0000 7f66d0ff9640 1 -- 172.21.15.40:0/1873168065 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f66c803e450 con 0x7f66d40cfd70 2023-10-18T09:04:32.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.941+0000 7f66d0ff9640 1 --2- 172.21.15.40:0/1873168065 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f66c006ae40 0x7f66c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:32.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.941+0000 7f66d0ff9640 1 -- 172.21.15.40:0/1873168065 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f66c80abf20 con 0x7f66d40cfd70 2023-10-18T09:04:32.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.942+0000 7f66d3fff640 1 --2- 172.21.15.40:0/1873168065 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f66c006ae40 0x7f66c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:32.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.943+0000 7f66d3fff640 1 --2- 172.21.15.40:0/1873168065 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f66c006ae40 0x7f66c006d300 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f66c400a830 tx=0x7f66c4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:32.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:32.945+0000 7f66d0ff9640 1 -- 172.21.15.40:0/1873168065 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66c80774d0 con 0x7f66d40cfd70 2023-10-18T09:04:33.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.134+0000 7f66da610640 1 -- 172.21.15.40:0/1873168065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66d415ff00 con 0x7f66d40cfd70 2023-10-18T09:04:33.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.136+0000 7f66d0ff9640 1 -- 172.21.15.40:0/1873168065 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+12880 (secure 0 0 0) 0x7f66c8043020 con 0x7f66d40cfd70 2023-10-18T09:04:33.139 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:33.139 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":158,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:04:32.121157+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:04:33.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.138+0000 7f66da610640 1 -- 172.21.15.40:0/1873168065 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f66c006ae40 msgr2=0x7f66c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:33.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.138+0000 7f66da610640 1 --2- 172.21.15.40:0/1873168065 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f66c006ae40 0x7f66c006d300 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f66c400a830 tx=0x7f66c4008040 comp rx=0 tx=0).stop 2023-10-18T09:04:33.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.138+0000 7f66da610640 1 -- 172.21.15.40:0/1873168065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d40cfd70 msgr2=0x7f66d40cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:33.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.138+0000 7f66da610640 1 --2- 172.21.15.40:0/1873168065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d40cfd70 0x7f66d40cac90 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f66c8036040 tx=0x7f66c802dce0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.138+0000 7f66da610640 1 -- 172.21.15.40:0/1873168065 shutdown_connections 2023-10-18T09:04:33.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.138+0000 7f66da610640 1 --2- 172.21.15.40:0/1873168065 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f66c006ae40 0x7f66c006d300 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.138+0000 7f66da610640 1 --2- 172.21.15.40:0/1873168065 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66d41590f0 0x7f66d415dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.138+0000 7f66da610640 1 --2- 172.21.15.40:0/1873168065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d40cfd70 0x7f66d40cac90 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.138+0000 7f66da610640 1 --2- 172.21.15.40:0/1873168065 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66d406e3b0 0x7f66d40c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.138+0000 7f66da610640 1 -- 172.21.15.40:0/1873168065 >> 172.21.15.40:0/1873168065 conn(0x7f66d40655f0 msgr2=0x7f66d406b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:33.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.139+0000 7f66da610640 1 -- 172.21.15.40:0/1873168065 shutdown_connections 2023-10-18T09:04:33.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.139+0000 7f66da610640 1 -- 172.21.15.40:0/1873168065 wait complete. 2023-10-18T09:04:33.159 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:04:33.160 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:04:33.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.390+0000 7f07ea721640 1 -- 172.21.15.40:0/1613308073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e40af310 msgr2=0x7f07e40ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:33.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.390+0000 7f07ea721640 1 --2- 172.21.15.40:0/1613308073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e40af310 0x7f07e40ca5f0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f07dc005fd0 tx=0x7f07dc02c0e0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.391+0000 7f07ea721640 1 -- 172.21.15.40:0/1613308073 shutdown_connections 2023-10-18T09:04:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.391+0000 7f07ea721640 1 --2- 172.21.15.40:0/1613308073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e40af310 0x7f07e40ca5f0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.391+0000 7f07ea721640 1 -- 172.21.15.40:0/1613308073 >> 172.21.15.40:0/1613308073 conn(0x7f07e4069720 msgr2=0x7f07e4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.391+0000 7f07ea721640 1 -- 172.21.15.40:0/1613308073 shutdown_connections 2023-10-18T09:04:33.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.391+0000 7f07ea721640 1 -- 172.21.15.40:0/1613308073 wait complete. 2023-10-18T09:04:33.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.393+0000 7f07ea721640 1 Processor -- start 2023-10-18T09:04:33.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.393+0000 7f07ea721640 1 -- start start 2023-10-18T09:04:33.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.394+0000 7f07ea721640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e40af310 0x7f07e41530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:33.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.394+0000 7f07ea721640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07e40b2c20 0x7f07e41508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:33.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.394+0000 7f07ea721640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07e415a200 0x7f07e40adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:33.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.394+0000 7f07ea721640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e4070c40 con 0x7f07e40af310 2023-10-18T09:04:33.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.394+0000 7f07ea721640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e414f460 con 0x7f07e415a200 2023-10-18T09:04:33.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.394+0000 7f07ea721640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e406ffc0 con 0x7f07e40b2c20 2023-10-18T09:04:33.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.394+0000 7f07e971f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e40af310 0x7f07e41530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:33.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.394+0000 7f07e971f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e40af310 0x7f07e41530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40358/0 (socket says 172.21.15.40:40358) 2023-10-18T09:04:33.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.394+0000 7f07e971f640 1 -- 172.21.15.40:0/3791127373 learned_addr learned my addr 172.21.15.40:0/3791127373 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:33.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.394+0000 7f07e9f20640 1 --2- 172.21.15.40:0/3791127373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07e415a200 0x7f07e40adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:33.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.394+0000 7f07e971f640 1 -- 172.21.15.40:0/3791127373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07e40b2c20 msgr2=0x7f07e41508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:33.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.394+0000 7f07e8f1e640 1 --2- 172.21.15.40:0/3791127373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07e40b2c20 0x7f07e41508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:33.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.395+0000 7f07e971f640 1 --2- 172.21.15.40:0/3791127373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07e40b2c20 0x7f07e41508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.395+0000 7f07e971f640 1 -- 172.21.15.40:0/3791127373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07e415a200 msgr2=0x7f07e40adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:33.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.395+0000 7f07e971f640 1 --2- 172.21.15.40:0/3791127373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07e415a200 0x7f07e40adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.395+0000 7f07e971f640 1 -- 172.21.15.40:0/3791127373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc009840 con 0x7f07e40af310 2023-10-18T09:04:33.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.395+0000 7f07e971f640 1 --2- 172.21.15.40:0/3791127373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e40af310 0x7f07e41530b0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f07e000c490 tx=0x7f07e000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:33.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.395+0000 7f07d9ffb640 1 -- 172.21.15.40:0/3791127373 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0012830 con 0x7f07e40af310 2023-10-18T09:04:33.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.395+0000 7f07d9ffb640 1 -- 172.21.15.40:0/3791127373 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f07e0012a80 con 0x7f07e40af310 2023-10-18T09:04:33.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.395+0000 7f07ea721640 1 -- 172.21.15.40:0/3791127373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e40702a0 con 0x7f07e40af310 2023-10-18T09:04:33.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.395+0000 7f07d9ffb640 1 -- 172.21.15.40:0/3791127373 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0011380 con 0x7f07e40af310 2023-10-18T09:04:33.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.395+0000 7f07ea721640 1 -- 172.21.15.40:0/3791127373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e4070860 con 0x7f07e40af310 2023-10-18T09:04:33.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.397+0000 7f07ea721640 1 -- 172.21.15.40:0/3791127373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e4070d80 con 0x7f07e40af310 2023-10-18T09:04:33.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.398+0000 7f07d9ffb640 1 -- 172.21.15.40:0/3791127373 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f07e00114e0 con 0x7f07e40af310 2023-10-18T09:04:33.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.399+0000 7f07d9ffb640 1 --2- 172.21.15.40:0/3791127373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f07c806ae40 0x7f07c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:33.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.399+0000 7f07d9ffb640 1 -- 172.21.15.40:0/3791127373 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f07e0089200 con 0x7f07e40af310 2023-10-18T09:04:33.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.399+0000 7f07e8f1e640 1 --2- 172.21.15.40:0/3791127373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f07c806ae40 0x7f07c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:33.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.401+0000 7f07e8f1e640 1 --2- 172.21.15.40:0/3791127373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f07c806ae40 0x7f07c806d300 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f07e41591c0 tx=0x7f07d4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:33.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.403+0000 7f07d9ffb640 1 -- 172.21.15.40:0/3791127373 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f07e00545f0 con 0x7f07e40af310 2023-10-18T09:04:33.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.602+0000 7f07ea721640 1 -- 172.21.15.40:0/3791127373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07e4066e00 con 0x7f07e40af310 2023-10-18T09:04:33.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.603+0000 7f07d9ffb640 1 -- 172.21.15.40:0/3791127373 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) v1 ==== 93+0+10 (secure 0 0 0) 0x7f07e0057c10 con 0x7f07e40af310 2023-10-18T09:04:33.606 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:04:33.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.606+0000 7f07ea721640 1 -- 172.21.15.40:0/3791127373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f07c806ae40 msgr2=0x7f07c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:33.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.606+0000 7f07ea721640 1 --2- 172.21.15.40:0/3791127373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f07c806ae40 0x7f07c806d300 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f07e41591c0 tx=0x7f07d4009040 comp rx=0 tx=0).stop 2023-10-18T09:04:33.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.606+0000 7f07ea721640 1 -- 172.21.15.40:0/3791127373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e40af310 msgr2=0x7f07e41530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:33.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.606+0000 7f07ea721640 1 --2- 172.21.15.40:0/3791127373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e40af310 0x7f07e41530b0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f07e000c490 tx=0x7f07e000c950 comp rx=0 tx=0).stop 2023-10-18T09:04:33.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.606+0000 7f07ea721640 1 -- 172.21.15.40:0/3791127373 shutdown_connections 2023-10-18T09:04:33.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.606+0000 7f07ea721640 1 --2- 172.21.15.40:0/3791127373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f07c806ae40 0x7f07c806d300 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.606+0000 7f07ea721640 1 --2- 172.21.15.40:0/3791127373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07e415a200 0x7f07e40adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.606+0000 7f07ea721640 1 --2- 172.21.15.40:0/3791127373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07e40b2c20 0x7f07e41508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.606+0000 7f07ea721640 1 --2- 172.21.15.40:0/3791127373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e40af310 0x7f07e41530b0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.606+0000 7f07ea721640 1 -- 172.21.15.40:0/3791127373 >> 172.21.15.40:0/3791127373 conn(0x7f07e4069720 msgr2=0x7f07e40ab5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:33.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.606+0000 7f07ea721640 1 -- 172.21.15.40:0/3791127373 shutdown_connections 2023-10-18T09:04:33.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.606+0000 7f07ea721640 1 -- 172.21.15.40:0/3791127373 wait complete. 2023-10-18T09:04:33.629 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T09:04:33.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.859+0000 7f4eeaba5640 1 -- 172.21.15.40:0/2837373922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ee40b1e10 msgr2=0x7f4ee40c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:33.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.859+0000 7f4eeaba5640 1 --2- 172.21.15.40:0/2837373922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ee40b1e10 0x7f4ee40c99c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4005fd0 tx=0x7f4ed402d150 comp rx=0 tx=0).stop 2023-10-18T09:04:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.860+0000 7f4eeaba5640 1 -- 172.21.15.40:0/2837373922 shutdown_connections 2023-10-18T09:04:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.860+0000 7f4eeaba5640 1 --2- 172.21.15.40:0/2837373922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ee40b1e10 0x7f4ee40c99c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.860+0000 7f4eeaba5640 1 --2- 172.21.15.40:0/2837373922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ee40b15b0 0x7f4ee40b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.860+0000 7f4eeaba5640 1 -- 172.21.15.40:0/2837373922 >> 172.21.15.40:0/2837373922 conn(0x7f4ee4069720 msgr2=0x7f4ee4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:33.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.860+0000 7f4eeaba5640 1 -- 172.21.15.40:0/2837373922 shutdown_connections 2023-10-18T09:04:33.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.860+0000 7f4eeaba5640 1 -- 172.21.15.40:0/2837373922 wait complete. 2023-10-18T09:04:33.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.862+0000 7f4eeaba5640 1 Processor -- start 2023-10-18T09:04:33.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.862+0000 7f4eeaba5640 1 -- start start 2023-10-18T09:04:33.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.863+0000 7f4eeaba5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ee40b15b0 0x7f4ee414d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:33.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.864+0000 7f4eeaba5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee40b1e10 0x7f4ee40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:33.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.864+0000 7f4eeaba5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ee40b2270 0x7f4ee4159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:33.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.864+0000 7f4eeaba5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee40a7380 con 0x7f4ee40b1e10 2023-10-18T09:04:33.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.864+0000 7f4eeaba5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee40a7660 con 0x7f4ee40b15b0 2023-10-18T09:04:33.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.864+0000 7f4eeaba5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee40a7940 con 0x7f4ee40b2270 2023-10-18T09:04:33.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.864+0000 7f4ee93a2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee40b1e10 0x7f4ee40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:33.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.864+0000 7f4ee93a2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee40b1e10 0x7f4ee40aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40374/0 (socket says 172.21.15.40:40374) 2023-10-18T09:04:33.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.864+0000 7f4ee93a2640 1 -- 172.21.15.40:0/2858236320 learned_addr learned my addr 172.21.15.40:0/2858236320 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:33.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.864+0000 7f4ee9ba3640 1 --2- 172.21.15.40:0/2858236320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ee40b15b0 0x7f4ee414d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:33.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.864+0000 7f4eea3a4640 1 --2- 172.21.15.40:0/2858236320 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ee40b2270 0x7f4ee4159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:33.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.864+0000 7f4ee93a2640 1 -- 172.21.15.40:0/2858236320 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ee40b2270 msgr2=0x7f4ee4159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:33.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.864+0000 7f4ee93a2640 1 --2- 172.21.15.40:0/2858236320 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ee40b2270 0x7f4ee4159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.864+0000 7f4ee93a2640 1 -- 172.21.15.40:0/2858236320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ee40b15b0 msgr2=0x7f4ee414d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:33.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.864+0000 7f4ee93a2640 1 --2- 172.21.15.40:0/2858236320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ee40b15b0 0x7f4ee414d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:33.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.865+0000 7f4ee93a2640 1 -- 172.21.15.40:0/2858236320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed400a670 con 0x7f4ee40b1e10 2023-10-18T09:04:33.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.865+0000 7f4ee93a2640 1 --2- 172.21.15.40:0/2858236320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee40b1e10 0x7f4ee40aed60 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4006010 tx=0x7f4ed4004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:33.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.865+0000 7f4eda7fc640 1 -- 172.21.15.40:0/2858236320 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4004e50 con 0x7f4ee40b1e10 2023-10-18T09:04:33.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.865+0000 7f4eda7fc640 1 -- 172.21.15.40:0/2858236320 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4ed40035f0 con 0x7f4ee40b1e10 2023-10-18T09:04:33.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.865+0000 7f4eda7fc640 1 -- 172.21.15.40:0/2858236320 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4003a00 con 0x7f4ee40b1e10 2023-10-18T09:04:33.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.865+0000 7f4eeaba5640 1 -- 172.21.15.40:0/2858236320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee40a7c20 con 0x7f4ee40b1e10 2023-10-18T09:04:33.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.866+0000 7f4eeaba5640 1 -- 172.21.15.40:0/2858236320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee40a8420 con 0x7f4ee40b1e10 2023-10-18T09:04:33.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.867+0000 7f4eeaba5640 1 -- 172.21.15.40:0/2858236320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee4066e00 con 0x7f4ee40b1e10 2023-10-18T09:04:33.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.868+0000 7f4eda7fc640 1 -- 172.21.15.40:0/2858236320 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4ed4003b60 con 0x7f4ee40b1e10 2023-10-18T09:04:33.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.869+0000 7f4eda7fc640 1 --2- 172.21.15.40:0/2858236320 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4ec806ae40 0x7f4ec806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:33.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.869+0000 7f4eda7fc640 1 -- 172.21.15.40:0/2858236320 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4ed40ac570 con 0x7f4ee40b1e10 2023-10-18T09:04:33.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.870+0000 7f4ee9ba3640 1 --2- 172.21.15.40:0/2858236320 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4ec806ae40 0x7f4ec806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:33.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.871+0000 7f4ee9ba3640 1 --2- 172.21.15.40:0/2858236320 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4ec806ae40 0x7f4ec806d300 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009b60 tx=0x7f4ee0005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:33.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:33.874+0000 7f4eda7fc640 1 -- 172.21.15.40:0/2858236320 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4ed40778d0 con 0x7f4ee40b1e10 2023-10-18T09:04:34.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.070+0000 7f4eeaba5640 1 -- 172.21.15.40:0/2858236320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4ee40caf90 con 0x7f4ee40b1e10 2023-10-18T09:04:34.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.070+0000 7f4eda7fc640 1 -- 172.21.15.40:0/2858236320 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v158) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4ed4042060 con 0x7f4ee40b1e10 2023-10-18T09:04:34.073 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:34.073 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-10-18T09:04:34.073 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:04:34.074 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-10-18T09:04:34.074 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-10-18T09:04:34.074 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:04:34.074 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-10-18T09:04:34.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.073+0000 7f4eeaba5640 1 -- 172.21.15.40:0/2858236320 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4ec806ae40 msgr2=0x7f4ec806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:34.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.073+0000 7f4eeaba5640 1 --2- 172.21.15.40:0/2858236320 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4ec806ae40 0x7f4ec806d300 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009b60 tx=0x7f4ee0005f90 comp rx=0 tx=0).stop 2023-10-18T09:04:34.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.073+0000 7f4eeaba5640 1 -- 172.21.15.40:0/2858236320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee40b1e10 msgr2=0x7f4ee40aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:34.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.073+0000 7f4eeaba5640 1 --2- 172.21.15.40:0/2858236320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee40b1e10 0x7f4ee40aed60 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4006010 tx=0x7f4ed4004a50 comp rx=0 tx=0).stop 2023-10-18T09:04:34.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.073+0000 7f4eeaba5640 1 -- 172.21.15.40:0/2858236320 shutdown_connections 2023-10-18T09:04:34.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.073+0000 7f4eeaba5640 1 --2- 172.21.15.40:0/2858236320 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4ec806ae40 0x7f4ec806d300 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:34.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.073+0000 7f4eeaba5640 1 --2- 172.21.15.40:0/2858236320 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ee40b2270 0x7f4ee4159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:34.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.073+0000 7f4eeaba5640 1 --2- 172.21.15.40:0/2858236320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ee40b1e10 0x7f4ee40aed60 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:34.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.073+0000 7f4eeaba5640 1 --2- 172.21.15.40:0/2858236320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ee40b15b0 0x7f4ee414d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:34.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.073+0000 7f4eeaba5640 1 -- 172.21.15.40:0/2858236320 >> 172.21.15.40:0/2858236320 conn(0x7f4ee4069720 msgr2=0x7f4ee40cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:34.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.073+0000 7f4eeaba5640 1 -- 172.21.15.40:0/2858236320 shutdown_connections 2023-10-18T09:04:34.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.073+0000 7f4eeaba5640 1 -- 172.21.15.40:0/2858236320 wait complete. 2023-10-18T09:04:34.094 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T09:04:34.094 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T09:04:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.332+0000 7f6c07f25640 1 -- 172.21.15.40:0/3569087752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c0015b460 msgr2=0x7f6c000cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.332+0000 7f6c07f25640 1 --2- 172.21.15.40:0/3569087752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c0015b460 0x7f6c000cae80 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc005fd0 tx=0x7f6bfc02be20 comp rx=0 tx=0).stop 2023-10-18T09:04:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.332+0000 7f6c07f25640 1 -- 172.21.15.40:0/3569087752 shutdown_connections 2023-10-18T09:04:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.332+0000 7f6c07f25640 1 --2- 172.21.15.40:0/3569087752 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c0015fd90 0x7f6c000c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.332+0000 7f6c07f25640 1 --2- 172.21.15.40:0/3569087752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c0015b460 0x7f6c000cae80 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.332+0000 7f6c07f25640 1 -- 172.21.15.40:0/3569087752 >> 172.21.15.40:0/3569087752 conn(0x7f6c000aec00 msgr2=0x7f6c000aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:34.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.332+0000 7f6c07f25640 1 -- 172.21.15.40:0/3569087752 shutdown_connections 2023-10-18T09:04:34.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.332+0000 7f6c07f25640 1 -- 172.21.15.40:0/3569087752 wait complete. 2023-10-18T09:04:34.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.334+0000 7f6c07f25640 1 Processor -- start 2023-10-18T09:04:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.335+0000 7f6c07f25640 1 -- start start 2023-10-18T09:04:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.336+0000 7f6c07f25640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c000c88d0 0x7f6c001f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.336+0000 7f6c07f25640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c0015b460 0x7f6c00148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:34.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.336+0000 7f6c07f25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c0015fd90 0x7f6c001f1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:34.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.336+0000 7f6c07f25640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c00149230 con 0x7f6c0015fd90 2023-10-18T09:04:34.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.336+0000 7f6c07f25640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c00149420 con 0x7f6c0015b460 2023-10-18T09:04:34.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.336+0000 7f6c07f25640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c00149700 con 0x7f6c000c88d0 2023-10-18T09:04:34.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.336+0000 7f6c064c2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c0015fd90 0x7f6c001f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:34.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.336+0000 7f6c064c2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c0015fd90 0x7f6c001f1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40388/0 (socket says 172.21.15.40:40388) 2023-10-18T09:04:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.336+0000 7f6c064c2640 1 -- 172.21.15.40:0/3752650947 learned_addr learned my addr 172.21.15.40:0/3752650947 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.337+0000 7f6c05cc1640 1 --2- 172.21.15.40:0/3752650947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c000c88d0 0x7f6c001f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.337+0000 7f6c054c0640 1 --2- 172.21.15.40:0/3752650947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c0015b460 0x7f6c00148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.337+0000 7f6c064c2640 1 -- 172.21.15.40:0/3752650947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c000c88d0 msgr2=0x7f6c001f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.337+0000 7f6c064c2640 1 --2- 172.21.15.40:0/3752650947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c000c88d0 0x7f6c001f5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.337+0000 7f6c064c2640 1 -- 172.21.15.40:0/3752650947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c0015b460 msgr2=0x7f6c00148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.337+0000 7f6c064c2640 1 --2- 172.21.15.40:0/3752650947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c0015b460 0x7f6c00148cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.337+0000 7f6c064c2640 1 -- 172.21.15.40:0/3752650947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc009560 con 0x7f6c0015fd90 2023-10-18T09:04:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.337+0000 7f6c064c2640 1 --2- 172.21.15.40:0/3752650947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c0015fd90 0x7f6c001f1520 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800c700 tx=0x7f6bf800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.337+0000 7f6bf67fc640 1 -- 172.21.15.40:0/3752650947 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf800aa50 con 0x7f6c0015fd90 2023-10-18T09:04:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.338+0000 7f6bf67fc640 1 -- 172.21.15.40:0/3752650947 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6bf800aca0 con 0x7f6c0015fd90 2023-10-18T09:04:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.338+0000 7f6bf67fc640 1 -- 172.21.15.40:0/3752650947 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8012680 con 0x7f6c0015fd90 2023-10-18T09:04:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.338+0000 7f6c07f25640 1 -- 172.21.15.40:0/3752650947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c001499e0 con 0x7f6c0015fd90 2023-10-18T09:04:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.338+0000 7f6c07f25640 1 -- 172.21.15.40:0/3752650947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c000c40e0 con 0x7f6c0015fd90 2023-10-18T09:04:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.339+0000 7f6c07f25640 1 -- 172.21.15.40:0/3752650947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0015f840 con 0x7f6c0015fd90 2023-10-18T09:04:34.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.340+0000 7f6bf67fc640 1 -- 172.21.15.40:0/3752650947 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6bf801b400 con 0x7f6c0015fd90 2023-10-18T09:04:34.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.341+0000 7f6bf67fc640 1 --2- 172.21.15.40:0/3752650947 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6be406ae40 0x7f6be406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:34.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.342+0000 7f6bf67fc640 1 -- 172.21.15.40:0/3752650947 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6bf80899e0 con 0x7f6c0015fd90 2023-10-18T09:04:34.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.342+0000 7f6c05cc1640 1 --2- 172.21.15.40:0/3752650947 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6be406ae40 0x7f6be406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:34.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.343+0000 7f6c05cc1640 1 --2- 172.21.15.40:0/3752650947 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6be406ae40 0x7f6be406d300 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc035000 tx=0x7f6bfc004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:34.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.346+0000 7f6bf67fc640 1 -- 172.21.15.40:0/3752650947 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6bf8054dd0 con 0x7f6c0015fd90 2023-10-18T09:04:34.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:34.541+0000 7f6c07f25640 1 -- 172.21.15.40:0/3752650947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6c000c2900 con 0x7f6c0015fd90 2023-10-18T09:04:35.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.440+0000 7f6bf67fc640 1 -- 172.21.15.40:0/3752650947 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v159) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6c000c2900 con 0x7f6c0015fd90 2023-10-18T09:04:35.443 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-10-18T09:04:35.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.443+0000 7f6c07f25640 1 -- 172.21.15.40:0/3752650947 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6be406ae40 msgr2=0x7f6be406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:35.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.443+0000 7f6c07f25640 1 --2- 172.21.15.40:0/3752650947 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6be406ae40 0x7f6be406d300 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc035000 tx=0x7f6bfc004730 comp rx=0 tx=0).stop 2023-10-18T09:04:35.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.443+0000 7f6c07f25640 1 -- 172.21.15.40:0/3752650947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c0015fd90 msgr2=0x7f6c001f1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:35.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.444+0000 7f6c07f25640 1 --2- 172.21.15.40:0/3752650947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c0015fd90 0x7f6c001f1520 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800c700 tx=0x7f6bf800cbc0 comp rx=0 tx=0).stop 2023-10-18T09:04:35.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.444+0000 7f6c07f25640 1 -- 172.21.15.40:0/3752650947 shutdown_connections 2023-10-18T09:04:35.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.444+0000 7f6c07f25640 1 --2- 172.21.15.40:0/3752650947 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6be406ae40 0x7f6be406d300 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:35.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.444+0000 7f6c07f25640 1 --2- 172.21.15.40:0/3752650947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c0015fd90 0x7f6c001f1520 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:35.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.444+0000 7f6c07f25640 1 --2- 172.21.15.40:0/3752650947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c0015b460 0x7f6c00148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:35.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.444+0000 7f6c07f25640 1 --2- 172.21.15.40:0/3752650947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c000c88d0 0x7f6c001f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:35.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.444+0000 7f6c07f25640 1 -- 172.21.15.40:0/3752650947 >> 172.21.15.40:0/3752650947 conn(0x7f6c000aec00 msgr2=0x7f6c000acb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:35.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.445+0000 7f6c07f25640 1 -- 172.21.15.40:0/3752650947 shutdown_connections 2023-10-18T09:04:35.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.445+0000 7f6c07f25640 1 -- 172.21.15.40:0/3752650947 wait complete. 2023-10-18T09:04:35.474 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T09:04:35.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.708+0000 7feb94f85640 1 -- 172.21.15.40:0/3767799225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb9015b760 msgr2=0x7feb900b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:35.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.708+0000 7feb94f85640 1 --2- 172.21.15.40:0/3767799225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb9015b760 0x7feb900b1120 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7feb84009830 tx=0x7feb8402bd50 comp rx=0 tx=0).stop 2023-10-18T09:04:35.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.708+0000 7feb94f85640 1 -- 172.21.15.40:0/3767799225 shutdown_connections 2023-10-18T09:04:35.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.708+0000 7feb94f85640 1 --2- 172.21.15.40:0/3767799225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb900d1580 0x7feb900cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:35.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.708+0000 7feb94f85640 1 --2- 172.21.15.40:0/3767799225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb9015b760 0x7feb900b1120 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:35.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.708+0000 7feb94f85640 1 -- 172.21.15.40:0/3767799225 >> 172.21.15.40:0/3767799225 conn(0x7feb9008acb0 msgr2=0x7feb9009a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:35.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.708+0000 7feb94f85640 1 -- 172.21.15.40:0/3767799225 shutdown_connections 2023-10-18T09:04:35.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.709+0000 7feb94f85640 1 -- 172.21.15.40:0/3767799225 wait complete. 2023-10-18T09:04:35.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.710+0000 7feb94f85640 1 Processor -- start 2023-10-18T09:04:35.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.710+0000 7feb94f85640 1 -- start start 2023-10-18T09:04:35.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.711+0000 7feb94f85640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb900d1580 0x7feb901557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:35.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.711+0000 7feb94f85640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb9015b380 0x7feb900c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:35.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.711+0000 7feb94f85640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb900c52c0 0x7feb900b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:35.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.711+0000 7feb94f85640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb900c5750 con 0x7feb900c52c0 2023-10-18T09:04:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.711+0000 7feb94f85640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb900b5740 con 0x7feb9015b380 2023-10-18T09:04:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.711+0000 7feb94f85640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb900b4b30 con 0x7feb900d1580 2023-10-18T09:04:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.712+0000 7feb8ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb900c52c0 0x7feb900b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.712+0000 7feb8ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb900c52c0 0x7feb900b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43392/0 (socket says 172.21.15.40:43392) 2023-10-18T09:04:35.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.712+0000 7feb8ffff640 1 -- 172.21.15.40:0/1704773414 learned_addr learned my addr 172.21.15.40:0/1704773414 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:35.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.712+0000 7feb8f7fe640 1 --2- 172.21.15.40:0/1704773414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb900d1580 0x7feb901557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:35.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.712+0000 7feb8effd640 1 --2- 172.21.15.40:0/1704773414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb9015b380 0x7feb900c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:35.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.712+0000 7feb8ffff640 1 -- 172.21.15.40:0/1704773414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb900d1580 msgr2=0x7feb901557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:35.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.712+0000 7feb8ffff640 1 --2- 172.21.15.40:0/1704773414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb900d1580 0x7feb901557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:35.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.712+0000 7feb8ffff640 1 -- 172.21.15.40:0/1704773414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb9015b380 msgr2=0x7feb900c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:35.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.712+0000 7feb8ffff640 1 --2- 172.21.15.40:0/1704773414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb9015b380 0x7feb900c4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:35.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.712+0000 7feb8ffff640 1 -- 172.21.15.40:0/1704773414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb84009490 con 0x7feb900c52c0 2023-10-18T09:04:35.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.712+0000 7feb8ffff640 1 --2- 172.21.15.40:0/1704773414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb900c52c0 0x7feb900b3e70 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7feb8800cd10 tx=0x7feb8800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:35.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.713+0000 7feb77fff640 1 -- 172.21.15.40:0/1704773414 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8800f450 con 0x7feb900c52c0 2023-10-18T09:04:35.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.713+0000 7feb77fff640 1 -- 172.21.15.40:0/1704773414 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feb8800f6a0 con 0x7feb900c52c0 2023-10-18T09:04:35.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.713+0000 7feb77fff640 1 -- 172.21.15.40:0/1704773414 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8800fab0 con 0x7feb900c52c0 2023-10-18T09:04:35.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.713+0000 7feb94f85640 1 -- 172.21.15.40:0/1704773414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb900b4e10 con 0x7feb900c52c0 2023-10-18T09:04:35.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.713+0000 7feb94f85640 1 -- 172.21.15.40:0/1704773414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb900b53d0 con 0x7feb900c52c0 2023-10-18T09:04:35.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.714+0000 7feb94f85640 1 -- 172.21.15.40:0/1704773414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb90071da0 con 0x7feb900c52c0 2023-10-18T09:04:35.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.715+0000 7feb77fff640 1 -- 172.21.15.40:0/1704773414 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feb8800fc10 con 0x7feb900c52c0 2023-10-18T09:04:35.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.716+0000 7feb77fff640 1 --2- 172.21.15.40:0/1704773414 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb7c06ae40 0x7feb7c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:35.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.716+0000 7feb77fff640 1 -- 172.21.15.40:0/1704773414 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5227+0+0 (secure 0 0 0) 0x7feb8808a150 con 0x7feb900c52c0 2023-10-18T09:04:35.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.717+0000 7feb8f7fe640 1 --2- 172.21.15.40:0/1704773414 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb7c06ae40 0x7feb7c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:35.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.718+0000 7feb8f7fe640 1 --2- 172.21.15.40:0/1704773414 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb7c06ae40 0x7feb7c06d300 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7feb80007fd0 tx=0x7feb80009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:35.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.721+0000 7feb77fff640 1 -- 172.21.15.40:0/1704773414 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb88055340 con 0x7feb900c52c0 2023-10-18T09:04:35.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:35.917+0000 7feb94f85640 1 -- 172.21.15.40:0/1704773414 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7feb90066e00 con 0x7feb900c52c0 2023-10-18T09:04:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.454+0000 7feb77fff640 1 -- 172.21.15.40:0/1704773414 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v160) v1 ==== 170+0+0 (secure 0 0 0) 0x7feb88058960 con 0x7feb900c52c0 2023-10-18T09:04:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-10-18T09:04:36.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.457+0000 7feb94f85640 1 -- 172.21.15.40:0/1704773414 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb7c06ae40 msgr2=0x7feb7c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:36.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.457+0000 7feb94f85640 1 --2- 172.21.15.40:0/1704773414 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb7c06ae40 0x7feb7c06d300 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7feb80007fd0 tx=0x7feb80009040 comp rx=0 tx=0).stop 2023-10-18T09:04:36.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.457+0000 7feb94f85640 1 -- 172.21.15.40:0/1704773414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb900c52c0 msgr2=0x7feb900b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:36.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.457+0000 7feb94f85640 1 --2- 172.21.15.40:0/1704773414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb900c52c0 0x7feb900b3e70 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7feb8800cd10 tx=0x7feb8800a620 comp rx=0 tx=0).stop 2023-10-18T09:04:36.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.458+0000 7feb94f85640 1 -- 172.21.15.40:0/1704773414 shutdown_connections 2023-10-18T09:04:36.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.458+0000 7feb94f85640 1 --2- 172.21.15.40:0/1704773414 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb7c06ae40 0x7feb7c06d300 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:36.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.458+0000 7feb94f85640 1 --2- 172.21.15.40:0/1704773414 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb900c52c0 0x7feb900b3e70 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:36.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.458+0000 7feb94f85640 1 --2- 172.21.15.40:0/1704773414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb9015b380 0x7feb900c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:36.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.458+0000 7feb94f85640 1 --2- 172.21.15.40:0/1704773414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb900d1580 0x7feb901557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:36.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.458+0000 7feb94f85640 1 -- 172.21.15.40:0/1704773414 >> 172.21.15.40:0/1704773414 conn(0x7feb9008acb0 msgr2=0x7feb900b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:36.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.458+0000 7feb94f85640 1 -- 172.21.15.40:0/1704773414 shutdown_connections 2023-10-18T09:04:36.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.459+0000 7feb94f85640 1 -- 172.21.15.40:0/1704773414 wait complete. 2023-10-18T09:04:36.495 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T09:04:36.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.701+0000 7f21124da640 1 -- 172.21.15.40:0/4161622975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f210c0cae80 msgr2=0x7f210c0cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:36.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.701+0000 7f21124da640 1 --2- 172.21.15.40:0/4161622975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f210c0cae80 0x7f210c0cb2f0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f20f4005fd0 tx=0x7f20f402d160 comp rx=0 tx=0).stop 2023-10-18T09:04:36.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.701+0000 7f21124da640 1 -- 172.21.15.40:0/4161622975 shutdown_connections 2023-10-18T09:04:36.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.701+0000 7f21124da640 1 --2- 172.21.15.40:0/4161622975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f210c0cae80 0x7f210c0cb2f0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:36.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.701+0000 7f21124da640 1 -- 172.21.15.40:0/4161622975 >> 172.21.15.40:0/4161622975 conn(0x7f210c069720 msgr2=0x7f210c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:36.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.701+0000 7f21124da640 1 -- 172.21.15.40:0/4161622975 shutdown_connections 2023-10-18T09:04:36.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.701+0000 7f21124da640 1 -- 172.21.15.40:0/4161622975 wait complete. 2023-10-18T09:04:36.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.703+0000 7f21124da640 1 Processor -- start 2023-10-18T09:04:36.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.703+0000 7f21124da640 1 -- start start 2023-10-18T09:04:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.704+0000 7f21124da640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f210c0cae80 0x7f210c153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.704+0000 7f21124da640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f210c0cb830 0x7f210c0afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.704+0000 7f21124da640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f210c157800 0x7f210c0b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.704+0000 7f21124da640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c1538a0 con 0x7f210c0cae80 2023-10-18T09:04:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.704+0000 7f21124da640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c13ec30 con 0x7f210c0cb830 2023-10-18T09:04:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.704+0000 7f21124da640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c13ef10 con 0x7f210c157800 2023-10-18T09:04:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.704+0000 7f210bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f210c0cae80 0x7f210c153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.704+0000 7f210bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f210c0cae80 0x7f210c153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43406/0 (socket says 172.21.15.40:43406) 2023-10-18T09:04:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.704+0000 7f210bfff640 1 -- 172.21.15.40:0/3329146001 learned_addr learned my addr 172.21.15.40:0/3329146001 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.705+0000 7f210bfff640 1 -- 172.21.15.40:0/3329146001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f210c157800 msgr2=0x7f210c0b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.705+0000 7f2103fff640 1 --2- 172.21.15.40:0/3329146001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f210c0cb830 0x7f210c0afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.705+0000 7f210bfff640 1 --2- 172.21.15.40:0/3329146001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f210c157800 0x7f210c0b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.705+0000 7f210bfff640 1 -- 172.21.15.40:0/3329146001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f210c0cb830 msgr2=0x7f210c0afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:36.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.705+0000 7f210bfff640 1 --2- 172.21.15.40:0/3329146001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f210c0cb830 0x7f210c0afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:36.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.705+0000 7f210bfff640 1 -- 172.21.15.40:0/3329146001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f400a670 con 0x7f210c0cae80 2023-10-18T09:04:36.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.705+0000 7f210bfff640 1 --2- 172.21.15.40:0/3329146001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f210c0cae80 0x7f210c153360 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00b3f0 tx=0x7f20fc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:36.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.705+0000 7f21097fa640 1 -- 172.21.15.40:0/3329146001 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc011830 con 0x7f210c0cae80 2023-10-18T09:04:36.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.705+0000 7f21097fa640 1 -- 172.21.15.40:0/3329146001 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f20fc011a80 con 0x7f210c0cae80 2023-10-18T09:04:36.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.705+0000 7f21097fa640 1 -- 172.21.15.40:0/3329146001 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc011e90 con 0x7f210c0cae80 2023-10-18T09:04:36.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.705+0000 7f21124da640 1 -- 172.21.15.40:0/3329146001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210c13f1f0 con 0x7f210c0cae80 2023-10-18T09:04:36.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.706+0000 7f21124da640 1 -- 172.21.15.40:0/3329146001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210c13f7b0 con 0x7f210c0cae80 2023-10-18T09:04:36.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.707+0000 7f21124da640 1 -- 172.21.15.40:0/3329146001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210c13c8c0 con 0x7f210c0cae80 2023-10-18T09:04:36.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.708+0000 7f21097fa640 1 -- 172.21.15.40:0/3329146001 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f20fc01a400 con 0x7f210c0cae80 2023-10-18T09:04:36.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.709+0000 7f21097fa640 1 --2- 172.21.15.40:0/3329146001 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f20ec06ae40 0x7f20ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:36.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.709+0000 7f21097fa640 1 -- 172.21.15.40:0/3329146001 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f20fc089710 con 0x7f210c0cae80 2023-10-18T09:04:36.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.710+0000 7f2103fff640 1 --2- 172.21.15.40:0/3329146001 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f20ec06ae40 0x7f20ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:36.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.711+0000 7f2103fff640 1 --2- 172.21.15.40:0/3329146001 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f20ec06ae40 0x7f20ec06d300 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f20f4036000 tx=0x7f20f4004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:36.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.714+0000 7f21097fa640 1 -- 172.21.15.40:0/3329146001 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f210c13c8c0 con 0x7f210c0cae80 2023-10-18T09:04:36.877 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:04:36.886 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:04:36.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:36.948+0000 7f21124da640 1 -- 172.21.15.40:0/3329146001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f210c13c8c0 con 0x7f210c0cae80 2023-10-18T09:04:37.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.458+0000 7f21097fa640 1 -- 172.21.15.40:0/3329146001 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v70) v1 ==== 174+0+0 (secure 0 0 0) 0x7f210c13c8c0 con 0x7f210c0cae80 2023-10-18T09:04:37.461 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 18 and data pool 19 2023-10-18T09:04:37.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.463+0000 7f21124da640 1 -- 172.21.15.40:0/3329146001 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f20ec06ae40 msgr2=0x7f20ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:37.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.463+0000 7f21124da640 1 --2- 172.21.15.40:0/3329146001 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f20ec06ae40 0x7f20ec06d300 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f20f4036000 tx=0x7f20f4004730 comp rx=0 tx=0).stop 2023-10-18T09:04:37.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.463+0000 7f21124da640 1 -- 172.21.15.40:0/3329146001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f210c0cae80 msgr2=0x7f210c153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:37.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.463+0000 7f21124da640 1 --2- 172.21.15.40:0/3329146001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f210c0cae80 0x7f210c153360 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00b3f0 tx=0x7f20fc00b8b0 comp rx=0 tx=0).stop 2023-10-18T09:04:37.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.463+0000 7f21124da640 1 -- 172.21.15.40:0/3329146001 shutdown_connections 2023-10-18T09:04:37.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.463+0000 7f21124da640 1 --2- 172.21.15.40:0/3329146001 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f20ec06ae40 0x7f20ec06d300 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:37.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.463+0000 7f21124da640 1 --2- 172.21.15.40:0/3329146001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f210c157800 0x7f210c0b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:37.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.463+0000 7f21124da640 1 --2- 172.21.15.40:0/3329146001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f210c0cb830 0x7f210c0afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:37.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.463+0000 7f21124da640 1 --2- 172.21.15.40:0/3329146001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f210c0cae80 0x7f210c153360 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:37.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.463+0000 7f21124da640 1 -- 172.21.15.40:0/3329146001 >> 172.21.15.40:0/3329146001 conn(0x7f210c069720 msgr2=0x7f210c06f570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:37.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.465+0000 7f21124da640 1 -- 172.21.15.40:0/3329146001 shutdown_connections 2023-10-18T09:04:37.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.465+0000 7f21124da640 1 -- 172.21.15.40:0/3329146001 wait complete. 2023-10-18T09:04:37.500 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:04:37.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.693+0000 7fcde3297640 1 -- 172.21.15.40:0/3570908632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc80047c0 msgr2=0x7fcdc8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:37.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.693+0000 7fcde3297640 1 --2- 172.21.15.40:0/3570908632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc80047c0 0x7fcdc8004c40 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00baf0 tx=0x7fcdc80057f0 comp rx=0 tx=0).stop 2023-10-18T09:04:37.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.693+0000 7fcde1834640 1 -- 172.21.15.40:0/3570908632 reap_dead start 2023-10-18T09:04:37.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.694+0000 7fcde3297640 1 -- 172.21.15.40:0/3570908632 shutdown_connections 2023-10-18T09:04:37.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.694+0000 7fcde3297640 1 -- 172.21.15.40:0/3570908632 >> 172.21.15.40:0/3570908632 conn(0x7fcddc0cf3e0 msgr2=0x7fcddc0cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:37.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.694+0000 7fcde3297640 1 -- 172.21.15.40:0/3570908632 shutdown_connections 2023-10-18T09:04:37.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.694+0000 7fcde3297640 1 -- 172.21.15.40:0/3570908632 wait complete. 2023-10-18T09:04:37.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.695+0000 7fcde3297640 1 Processor -- start 2023-10-18T09:04:37.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.696+0000 7fcde3297640 1 -- start start 2023-10-18T09:04:37.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.697+0000 7fcde3297640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc80047c0 0x7fcddc155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:37.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.697+0000 7fcde3297640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcddc1f6030 0x7fcddc0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:37.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.697+0000 7fcde3297640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcddc0b0e90 0x7fcddc1f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:37.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.697+0000 7fcde3297640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0b1320 con 0x7fcdc80047c0 2023-10-18T09:04:37.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.697+0000 7fcde3297640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc1f5300 con 0x7fcddc0b0e90 2023-10-18T09:04:37.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.697+0000 7fcde3297640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc1f2af0 con 0x7fcddc1f6030 2023-10-18T09:04:37.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.697+0000 7fcde1033640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc80047c0 0x7fcddc155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:37.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.697+0000 7fcde1033640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc80047c0 0x7fcddc155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43428/0 (socket says 172.21.15.40:43428) 2023-10-18T09:04:37.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.697+0000 7fcde1033640 1 -- 172.21.15.40:0/166012315 learned_addr learned my addr 172.21.15.40:0/166012315 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:37.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.697+0000 7fcde1033640 1 -- 172.21.15.40:0/166012315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcddc1f6030 msgr2=0x7fcddc0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:37.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.697+0000 7fcde1033640 1 --2- 172.21.15.40:0/166012315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcddc1f6030 0x7fcddc0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:37.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.697+0000 7fcde1033640 1 -- 172.21.15.40:0/166012315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcddc0b0e90 msgr2=0x7fcddc1f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:37.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.697+0000 7fcde1033640 1 --2- 172.21.15.40:0/166012315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcddc0b0e90 0x7fcddc1f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:37.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.697+0000 7fcde1033640 1 -- 172.21.15.40:0/166012315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc1f5550 con 0x7fcdc80047c0 2023-10-18T09:04:37.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.698+0000 7fcde1033640 1 --2- 172.21.15.40:0/166012315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc80047c0 0x7fcddc155260 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80109a0 tx=0x7fcdd8010e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:37.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.698+0000 7fcdd1ffb640 1 -- 172.21.15.40:0/166012315 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd800eda0 con 0x7fcdc80047c0 2023-10-18T09:04:37.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.698+0000 7fcdd1ffb640 1 -- 172.21.15.40:0/166012315 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcdd8004030 con 0x7fcdc80047c0 2023-10-18T09:04:37.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.698+0000 7fcde3297640 1 -- 172.21.15.40:0/166012315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcddc1f5830 con 0x7fcdc80047c0 2023-10-18T09:04:37.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.698+0000 7fcdd1ffb640 1 -- 172.21.15.40:0/166012315 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8004420 con 0x7fcdc80047c0 2023-10-18T09:04:37.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.698+0000 7fcde3297640 1 -- 172.21.15.40:0/166012315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc1f2230 con 0x7fcdc80047c0 2023-10-18T09:04:37.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.699+0000 7fcde3297640 1 -- 172.21.15.40:0/166012315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcddc066d40 con 0x7fcdc80047c0 2023-10-18T09:04:37.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.700+0000 7fcdd1ffb640 1 -- 172.21.15.40:0/166012315 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcdd80035f0 con 0x7fcdc80047c0 2023-10-18T09:04:37.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.701+0000 7fcdd1ffb640 1 --2- 172.21.15.40:0/166012315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcdc006ae40 0x7fcdc006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:37.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.701+0000 7fcdd1ffb640 1 -- 172.21.15.40:0/166012315 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdd8016070 con 0x7fcdc80047c0 2023-10-18T09:04:37.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.702+0000 7fcde0832640 1 --2- 172.21.15.40:0/166012315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcdc006ae40 0x7fcdc006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:37.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.703+0000 7fcde0832640 1 --2- 172.21.15.40:0/166012315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcdc006ae40 0x7fcdc006d300 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc037000 tx=0x7fcdcc02e040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:37.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.706+0000 7fcdd1ffb640 1 -- 172.21.15.40:0/166012315 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcdd8055600 con 0x7fcdc80047c0 2023-10-18T09:04:37.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.900+0000 7fcde3297640 1 -- 172.21.15.40:0/166012315 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcddc0739c0 con 0x7fcdc80047c0 2023-10-18T09:04:37.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.902+0000 7fcdd1ffb640 1 -- 172.21.15.40:0/166012315 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+16605 (secure 0 0 0) 0x7fcdd8058c20 con 0x7fcdc80047c0 2023-10-18T09:04:37.905 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:37.905 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":161,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:04:36.949711+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:04:34.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-18T09:04:35.972659+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:04:37.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.906+0000 7fcde3297640 1 -- 172.21.15.40:0/166012315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcdc006ae40 msgr2=0x7fcdc006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:37.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.906+0000 7fcde3297640 1 --2- 172.21.15.40:0/166012315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcdc006ae40 0x7fcdc006d300 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc037000 tx=0x7fcdcc02e040 comp rx=0 tx=0).stop 2023-10-18T09:04:37.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.906+0000 7fcde3297640 1 -- 172.21.15.40:0/166012315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc80047c0 msgr2=0x7fcddc155260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:37.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.906+0000 7fcde3297640 1 --2- 172.21.15.40:0/166012315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc80047c0 0x7fcddc155260 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80109a0 tx=0x7fcdd8010e60 comp rx=0 tx=0).stop 2023-10-18T09:04:37.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.906+0000 7fcde3297640 1 -- 172.21.15.40:0/166012315 shutdown_connections 2023-10-18T09:04:37.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.906+0000 7fcde3297640 1 --2- 172.21.15.40:0/166012315 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcdc006ae40 0x7fcdc006d300 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:37.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.906+0000 7fcde3297640 1 --2- 172.21.15.40:0/166012315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcddc0b0e90 0x7fcddc1f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:37.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.906+0000 7fcde3297640 1 --2- 172.21.15.40:0/166012315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcddc1f6030 0x7fcddc0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:37.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.906+0000 7fcde3297640 1 --2- 172.21.15.40:0/166012315 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc80047c0 0x7fcddc155260 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:37.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.906+0000 7fcde3297640 1 -- 172.21.15.40:0/166012315 >> 172.21.15.40:0/166012315 conn(0x7fcddc0cf3e0 msgr2=0x7fcddc0b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:37.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.906+0000 7fcde3297640 1 -- 172.21.15.40:0/166012315 shutdown_connections 2023-10-18T09:04:37.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:37.906+0000 7fcde3297640 1 -- 172.21.15.40:0/166012315 wait complete. 2023-10-18T09:04:37.928 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:04:38.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.157+0000 7f57d764b640 1 -- 172.21.15.40:0/3739238205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d006d710 msgr2=0x7f57d0151bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:38.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.157+0000 7f57d764b640 1 --2- 172.21.15.40:0/3739238205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d006d710 0x7f57d0151bd0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f57d006c840 tx=0x7f57cc02bd90 comp rx=0 tx=0).stop 2023-10-18T09:04:38.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.157+0000 7f57d764b640 1 -- 172.21.15.40:0/3739238205 shutdown_connections 2023-10-18T09:04:38.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.157+0000 7f57d764b640 1 --2- 172.21.15.40:0/3739238205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d006bc50 0x7f57d006c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:38.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.157+0000 7f57d764b640 1 --2- 172.21.15.40:0/3739238205 >> v2:172.21.15.144:3300/0 conn(0x7f57d01586a0 0x7f57d006b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:38.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.157+0000 7f57d764b640 1 --2- 172.21.15.40:0/3739238205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d006d710 0x7f57d0151bd0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:38.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.157+0000 7f57d764b640 1 -- 172.21.15.40:0/3739238205 >> 172.21.15.40:0/3739238205 conn(0x7f57d006cee0 msgr2=0x7f57d0069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:38.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.157+0000 7f57d764b640 1 -- 172.21.15.40:0/3739238205 shutdown_connections 2023-10-18T09:04:38.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.157+0000 7f57d764b640 1 -- 172.21.15.40:0/3739238205 wait complete. 2023-10-18T09:04:38.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.159+0000 7f57d764b640 1 Processor -- start 2023-10-18T09:04:38.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.159+0000 7f57d764b640 1 -- start start 2023-10-18T09:04:38.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.160+0000 7f57d764b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d006bc50 0x7f57d014b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:38.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.160+0000 7f57d764b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d006d710 0x7f57d015a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:38.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.160+0000 7f57d764b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d01586a0 0x7f57d015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:38.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.160+0000 7f57d764b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d014ea70 con 0x7f57d006d710 2023-10-18T09:04:38.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.160+0000 7f57d764b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d0157190 con 0x7f57d01586a0 2023-10-18T09:04:38.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.160+0000 7f57d764b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d014d260 con 0x7f57d006bc50 2023-10-18T09:04:38.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.161+0000 7f57d4be6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d006d710 0x7f57d015a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:38.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.161+0000 7f57d4be6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d006d710 0x7f57d015a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43446/0 (socket says 172.21.15.40:43446) 2023-10-18T09:04:38.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.161+0000 7f57d4be6640 1 -- 172.21.15.40:0/2832365309 learned_addr learned my addr 172.21.15.40:0/2832365309 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:38.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.161+0000 7f57d53e7640 1 --2- 172.21.15.40:0/2832365309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d006bc50 0x7f57d014b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:38.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.161+0000 7f57d5be8640 1 --2- 172.21.15.40:0/2832365309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d01586a0 0x7f57d015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:38.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.161+0000 7f57d4be6640 1 -- 172.21.15.40:0/2832365309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d006bc50 msgr2=0x7f57d014b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:38.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.161+0000 7f57d4be6640 1 --2- 172.21.15.40:0/2832365309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d006bc50 0x7f57d014b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:38.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.161+0000 7f57d4be6640 1 -- 172.21.15.40:0/2832365309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d01586a0 msgr2=0x7f57d015a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:38.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.161+0000 7f57d4be6640 1 --2- 172.21.15.40:0/2832365309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d01586a0 0x7f57d015a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:38.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.161+0000 7f57d4be6640 1 -- 172.21.15.40:0/2832365309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc009560 con 0x7f57d006d710 2023-10-18T09:04:38.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.161+0000 7f57d5be8640 1 --2- 172.21.15.40:0/2832365309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d01586a0 0x7f57d015a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:38.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.162+0000 7f57d4be6640 1 --2- 172.21.15.40:0/2832365309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d006d710 0x7f57d015a190 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f57c000c970 tx=0x7f57c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:38.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.162+0000 7f57c5ffb640 1 -- 172.21.15.40:0/2832365309 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0007bb0 con 0x7f57d006d710 2023-10-18T09:04:38.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.162+0000 7f57c5ffb640 1 -- 172.21.15.40:0/2832365309 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f57c0007e00 con 0x7f57d006d710 2023-10-18T09:04:38.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.162+0000 7f57c5ffb640 1 -- 172.21.15.40:0/2832365309 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0005280 con 0x7f57d006d710 2023-10-18T09:04:38.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.162+0000 7f57d764b640 1 -- 172.21.15.40:0/2832365309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d014d540 con 0x7f57d006d710 2023-10-18T09:04:38.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.162+0000 7f57d764b640 1 -- 172.21.15.40:0/2832365309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d014db00 con 0x7f57d006d710 2023-10-18T09:04:38.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.163+0000 7f57d764b640 1 -- 172.21.15.40:0/2832365309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d0066fd0 con 0x7f57d006d710 2023-10-18T09:04:38.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.164+0000 7f57c5ffb640 1 -- 172.21.15.40:0/2832365309 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f57c00053e0 con 0x7f57d006d710 2023-10-18T09:04:38.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.165+0000 7f57c5ffb640 1 --2- 172.21.15.40:0/2832365309 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57b406ae40 0x7f57b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:38.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.165+0000 7f57c5ffb640 1 -- 172.21.15.40:0/2832365309 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57c0089aa0 con 0x7f57d006d710 2023-10-18T09:04:38.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.166+0000 7f57d53e7640 1 --2- 172.21.15.40:0/2832365309 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57b406ae40 0x7f57b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:38.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.167+0000 7f57d53e7640 1 --2- 172.21.15.40:0/2832365309 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57b406ae40 0x7f57b406d300 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f57cc035000 tx=0x7f57cc004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:38.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.170+0000 7f57c5ffb640 1 -- 172.21.15.40:0/2832365309 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57c0054c00 con 0x7f57d006d710 2023-10-18T09:04:38.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.362+0000 7f57d764b640 1 -- 172.21.15.40:0/2832365309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57d0164590 con 0x7f57d006d710 2023-10-18T09:04:38.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.365+0000 7f57c5ffb640 1 -- 172.21.15.40:0/2832365309 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f57c001d070 con 0x7f57d006d710 2023-10-18T09:04:38.368 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:38.368 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":161,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:04:36.949711+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:04:34.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-18T09:04:35.972659+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:04:38.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.368+0000 7f57d764b640 1 -- 172.21.15.40:0/2832365309 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57b406ae40 msgr2=0x7f57b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:38.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.368+0000 7f57d764b640 1 --2- 172.21.15.40:0/2832365309 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57b406ae40 0x7f57b406d300 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f57cc035000 tx=0x7f57cc004730 comp rx=0 tx=0).stop 2023-10-18T09:04:38.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.368+0000 7f57d764b640 1 -- 172.21.15.40:0/2832365309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d006d710 msgr2=0x7f57d015a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:38.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.368+0000 7f57d764b640 1 --2- 172.21.15.40:0/2832365309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d006d710 0x7f57d015a190 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f57c000c970 tx=0x7f57c000ce30 comp rx=0 tx=0).stop 2023-10-18T09:04:38.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.368+0000 7f57d764b640 1 -- 172.21.15.40:0/2832365309 shutdown_connections 2023-10-18T09:04:38.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.368+0000 7f57d764b640 1 --2- 172.21.15.40:0/2832365309 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57b406ae40 0x7f57b406d300 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:38.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.368+0000 7f57d764b640 1 --2- 172.21.15.40:0/2832365309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57d01586a0 0x7f57d015a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:38.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.368+0000 7f57d764b640 1 --2- 172.21.15.40:0/2832365309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d006d710 0x7f57d015a190 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:38.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.368+0000 7f57d764b640 1 --2- 172.21.15.40:0/2832365309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d006bc50 0x7f57d014b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:38.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.368+0000 7f57d764b640 1 -- 172.21.15.40:0/2832365309 >> 172.21.15.40:0/2832365309 conn(0x7f57d006cee0 msgr2=0x7f57d0069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:38.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.368+0000 7f57d764b640 1 -- 172.21.15.40:0/2832365309 shutdown_connections 2023-10-18T09:04:38.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.368+0000 7f57d764b640 1 -- 172.21.15.40:0/2832365309 wait complete. 2023-10-18T09:04:38.390 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T09:04:38.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.631+0000 7fdcf5bd0640 1 -- 172.21.15.40:0/1832542776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcf015af70 msgr2=0x7fdcf0070b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:38.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.631+0000 7fdcf5bd0640 1 --2- 172.21.15.40:0/1832542776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcf015af70 0x7fdcf0070b60 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc005fd0 tx=0x7fdcdc02bcf0 comp rx=0 tx=0).stop 2023-10-18T09:04:38.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.631+0000 7fdcf5bd0640 1 -- 172.21.15.40:0/1832542776 shutdown_connections 2023-10-18T09:04:38.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.631+0000 7fdcf5bd0640 1 --2- 172.21.15.40:0/1832542776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcf015af70 0x7fdcf0070b60 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:38.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.631+0000 7fdcf5bd0640 1 --2- 172.21.15.40:0/1832542776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf006d5d0 0x7fdcf00c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:38.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.631+0000 7fdcf5bd0640 1 -- 172.21.15.40:0/1832542776 >> 172.21.15.40:0/1832542776 conn(0x7fdcf006ca10 msgr2=0x7fdcf006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:38.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.631+0000 7fdcf5bd0640 1 -- 172.21.15.40:0/1832542776 shutdown_connections 2023-10-18T09:04:38.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.631+0000 7fdcf5bd0640 1 -- 172.21.15.40:0/1832542776 wait complete. 2023-10-18T09:04:38.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.633+0000 7fdcf5bd0640 1 Processor -- start 2023-10-18T09:04:38.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.633+0000 7fdcf5bd0640 1 -- start start 2023-10-18T09:04:38.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.634+0000 7fdcf5bd0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf006d5d0 0x7fdcf0159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:38.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.634+0000 7fdcf5bd0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcf00710a0 0x7fdcf0149d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:38.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.635+0000 7fdcf5bd0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcf015af70 0x7fdcf01e1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:38.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.635+0000 7fdcf5bd0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf01e2930 con 0x7fdcf00710a0 2023-10-18T09:04:38.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.635+0000 7fdcf5bd0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf0159fc0 con 0x7fdcf015af70 2023-10-18T09:04:38.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.635+0000 7fdcf5bd0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf01f1ad0 con 0x7fdcf006d5d0 2023-10-18T09:04:38.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.635+0000 7fdceffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcf00710a0 0x7fdcf0149d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:38.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.635+0000 7fdcf4bce640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf006d5d0 0x7fdcf0159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:38.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.635+0000 7fdceffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcf00710a0 0x7fdcf0149d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43474/0 (socket says 172.21.15.40:43474) 2023-10-18T09:04:38.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.635+0000 7fdceffff640 1 -- 172.21.15.40:0/872787508 learned_addr learned my addr 172.21.15.40:0/872787508 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:38.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.635+0000 7fdcf53cf640 1 --2- 172.21.15.40:0/872787508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcf015af70 0x7fdcf01e1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:38.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.636+0000 7fdceffff640 1 -- 172.21.15.40:0/872787508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf006d5d0 msgr2=0x7fdcf0159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.636+0000 7fdceffff640 1 --2- 172.21.15.40:0/872787508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf006d5d0 0x7fdcf0159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.636+0000 7fdceffff640 1 -- 172.21.15.40:0/872787508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcf015af70 msgr2=0x7fdcf01e1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.636+0000 7fdceffff640 1 --2- 172.21.15.40:0/872787508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcf015af70 0x7fdcf01e1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.636+0000 7fdceffff640 1 -- 172.21.15.40:0/872787508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc009560 con 0x7fdcf00710a0 2023-10-18T09:04:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.636+0000 7fdceffff640 1 --2- 172.21.15.40:0/872787508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcf00710a0 0x7fdcf0149d80 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009850 tx=0x7fdcdc02da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.636+0000 7fdced7fa640 1 -- 172.21.15.40:0/872787508 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc02de70 con 0x7fdcf00710a0 2023-10-18T09:04:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.636+0000 7fdced7fa640 1 -- 172.21.15.40:0/872787508 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdcdc0036e0 con 0x7fdcf00710a0 2023-10-18T09:04:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.636+0000 7fdced7fa640 1 -- 172.21.15.40:0/872787508 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc003a90 con 0x7fdcf00710a0 2023-10-18T09:04:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.636+0000 7fdcf5bd0640 1 -- 172.21.15.40:0/872787508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf015a290 con 0x7fdcf00710a0 2023-10-18T09:04:38.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.637+0000 7fdcf5bd0640 1 -- 172.21.15.40:0/872787508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf015a850 con 0x7fdcf00710a0 2023-10-18T09:04:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.638+0000 7fdcf5bd0640 1 -- 172.21.15.40:0/872787508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf0066e00 con 0x7fdcf00710a0 2023-10-18T09:04:38.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.639+0000 7fdced7fa640 1 -- 172.21.15.40:0/872787508 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdcdc003bf0 con 0x7fdcf00710a0 2023-10-18T09:04:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.640+0000 7fdced7fa640 1 --2- 172.21.15.40:0/872787508 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdcd406ae40 0x7fdcd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.640+0000 7fdced7fa640 1 -- 172.21.15.40:0/872787508 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdcdc0abfc0 con 0x7fdcf00710a0 2023-10-18T09:04:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.641+0000 7fdcf4bce640 1 --2- 172.21.15.40:0/872787508 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdcd406ae40 0x7fdcd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:38.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.642+0000 7fdcf4bce640 1 --2- 172.21.15.40:0/872787508 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdcd406ae40 0x7fdcd406d300 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdce40058c0 tx=0x7fdce4005f60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:38.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.644+0000 7fdced7fa640 1 -- 172.21.15.40:0/872787508 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdcdc077080 con 0x7fdcf00710a0 2023-10-18T09:04:38.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:38.876+0000 7fdcf5bd0640 1 -- 172.21.15.40:0/872787508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdcf00c9b60 con 0x7fdcf00710a0 2023-10-18T09:04:39.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.505+0000 7fdced7fa640 1 -- 172.21.15.40:0/872787508 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdcf00c9b60 con 0x7fdcf00710a0 2023-10-18T09:04:39.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.509+0000 7fdcf5bd0640 1 -- 172.21.15.40:0/872787508 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdcd406ae40 msgr2=0x7fdcd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:39.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.509+0000 7fdcf5bd0640 1 --2- 172.21.15.40:0/872787508 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdcd406ae40 0x7fdcd406d300 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdce40058c0 tx=0x7fdce4005f60 comp rx=0 tx=0).stop 2023-10-18T09:04:39.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.509+0000 7fdcf5bd0640 1 -- 172.21.15.40:0/872787508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcf00710a0 msgr2=0x7fdcf0149d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:39.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.509+0000 7fdcf5bd0640 1 --2- 172.21.15.40:0/872787508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcf00710a0 0x7fdcf0149d80 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009850 tx=0x7fdcdc02da70 comp rx=0 tx=0).stop 2023-10-18T09:04:39.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.509+0000 7fdcf5bd0640 1 -- 172.21.15.40:0/872787508 shutdown_connections 2023-10-18T09:04:39.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.509+0000 7fdcf5bd0640 1 --2- 172.21.15.40:0/872787508 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fdcd406ae40 0x7fdcd406d300 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:39.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.509+0000 7fdcf5bd0640 1 --2- 172.21.15.40:0/872787508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcf015af70 0x7fdcf01e1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:39.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.509+0000 7fdcf5bd0640 1 --2- 172.21.15.40:0/872787508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdcf00710a0 0x7fdcf0149d80 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:39.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.509+0000 7fdcf5bd0640 1 --2- 172.21.15.40:0/872787508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdcf006d5d0 0x7fdcf0159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:39.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.509+0000 7fdcf5bd0640 1 -- 172.21.15.40:0/872787508 >> 172.21.15.40:0/872787508 conn(0x7fdcf006ca10 msgr2=0x7fdcf00afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:39.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.510+0000 7fdcf5bd0640 1 -- 172.21.15.40:0/872787508 shutdown_connections 2023-10-18T09:04:39.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.510+0000 7fdcf5bd0640 1 -- 172.21.15.40:0/872787508 wait complete. 2023-10-18T09:04:39.535 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T09:04:39.536 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T09:04:39.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.779+0000 7f70a3fff640 1 -- 172.21.15.40:0/2598486575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70a406d5d0 msgr2=0x7f70a40c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:39.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.779+0000 7f70a3fff640 1 --2- 172.21.15.40:0/2598486575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70a406d5d0 0x7f70a40c9620 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7094005fd0 tx=0x7f709402bd80 comp rx=0 tx=0).stop 2023-10-18T09:04:39.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.780+0000 7f70a3fff640 1 -- 172.21.15.40:0/2598486575 shutdown_connections 2023-10-18T09:04:39.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.780+0000 7f70a3fff640 1 --2- 172.21.15.40:0/2598486575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70a415af70 0x7f70a4070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:39.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.780+0000 7f70a3fff640 1 --2- 172.21.15.40:0/2598486575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70a406d5d0 0x7f70a40c9620 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:39.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.780+0000 7f70a3fff640 1 -- 172.21.15.40:0/2598486575 >> 172.21.15.40:0/2598486575 conn(0x7f70a406ca10 msgr2=0x7f70a406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:39.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.780+0000 7f70a3fff640 1 -- 172.21.15.40:0/2598486575 shutdown_connections 2023-10-18T09:04:39.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.780+0000 7f70a3fff640 1 -- 172.21.15.40:0/2598486575 wait complete. 2023-10-18T09:04:39.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.782+0000 7f70a3fff640 1 Processor -- start 2023-10-18T09:04:39.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.783+0000 7f70a3fff640 1 -- start start 2023-10-18T09:04:39.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.784+0000 7f70a3fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70a406d5d0 0x7f70a41513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:39.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.784+0000 7f70a3fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70a40710a0 0x7f70a41f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:39.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.784+0000 7f70a3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a415af70 0x7f70a4159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:39.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.784+0000 7f70a3fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a4150f50 con 0x7f70a415af70 2023-10-18T09:04:39.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.784+0000 7f70a3fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a415a6d0 con 0x7f70a40710a0 2023-10-18T09:04:39.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.784+0000 7f70a3fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a415a9b0 con 0x7f70a406d5d0 2023-10-18T09:04:39.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.784+0000 7f70a37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a415af70 0x7f70a4159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:39.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.784+0000 7f70a37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a415af70 0x7f70a4159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43482/0 (socket says 172.21.15.40:43482) 2023-10-18T09:04:39.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.784+0000 7f70a37fe640 1 -- 172.21.15.40:0/3029422607 learned_addr learned my addr 172.21.15.40:0/3029422607 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:39.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.784+0000 7f70a2ffd640 1 --2- 172.21.15.40:0/3029422607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70a406d5d0 0x7f70a41513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:39.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.785+0000 7f709bfff640 1 --2- 172.21.15.40:0/3029422607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70a40710a0 0x7f70a41f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:39.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.785+0000 7f70a37fe640 1 -- 172.21.15.40:0/3029422607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70a406d5d0 msgr2=0x7f70a41513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:39.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.785+0000 7f70a37fe640 1 --2- 172.21.15.40:0/3029422607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70a406d5d0 0x7f70a41513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:39.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.785+0000 7f70a37fe640 1 -- 172.21.15.40:0/3029422607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70a40710a0 msgr2=0x7f70a41f12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:39.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.785+0000 7f70a37fe640 1 --2- 172.21.15.40:0/3029422607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70a40710a0 0x7f70a41f12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:39.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.785+0000 7f70a37fe640 1 -- 172.21.15.40:0/3029422607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7094009560 con 0x7f70a415af70 2023-10-18T09:04:39.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.785+0000 7f70a37fe640 1 --2- 172.21.15.40:0/3029422607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a415af70 0x7f70a4159c10 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f709c00c7a0 tx=0x7f709c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:39.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.786+0000 7f709b7fe640 1 -- 172.21.15.40:0/3029422607 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c003b70 con 0x7f70a415af70 2023-10-18T09:04:39.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.786+0000 7f709b7fe640 1 -- 172.21.15.40:0/3029422607 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f709c003cf0 con 0x7f70a415af70 2023-10-18T09:04:39.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.786+0000 7f709b7fe640 1 -- 172.21.15.40:0/3029422607 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c00aab0 con 0x7f70a415af70 2023-10-18T09:04:39.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.786+0000 7f70a3fff640 1 -- 172.21.15.40:0/3029422607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a4150900 con 0x7f70a415af70 2023-10-18T09:04:39.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.786+0000 7f70a3fff640 1 -- 172.21.15.40:0/3029422607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a414e300 con 0x7f70a415af70 2023-10-18T09:04:39.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.788+0000 7f70a3fff640 1 -- 172.21.15.40:0/3029422607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a4150760 con 0x7f70a415af70 2023-10-18T09:04:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.789+0000 7f709b7fe640 1 -- 172.21.15.40:0/3029422607 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f709c00ac10 con 0x7f70a415af70 2023-10-18T09:04:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.789+0000 7f709b7fe640 1 --2- 172.21.15.40:0/3029422607 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f708406ae40 0x7f708406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.790+0000 7f709b7fe640 1 -- 172.21.15.40:0/3029422607 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f709c0892f0 con 0x7f70a415af70 2023-10-18T09:04:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.790+0000 7f70a2ffd640 1 --2- 172.21.15.40:0/3029422607 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f708406ae40 0x7f708406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.792+0000 7f70a2ffd640 1 --2- 172.21.15.40:0/3029422607 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f708406ae40 0x7f708406d300 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7094035000 tx=0x7f7094004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:39.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:39.794+0000 7f709b7fe640 1 -- 172.21.15.40:0/3029422607 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f70a4150760 con 0x7f70a415af70 2023-10-18T09:04:40.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.028+0000 7f70a3fff640 1 -- 172.21.15.40:0/3029422607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f70a40c9b60 con 0x7f70a415af70 2023-10-18T09:04:40.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.507+0000 7f709b7fe640 1 -- 172.21.15.40:0/3029422607 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) v1 ==== 122+0+0 (secure 0 0 0) 0x7f70a40c9b60 con 0x7f70a415af70 2023-10-18T09:04:40.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.511+0000 7f70a3fff640 1 -- 172.21.15.40:0/3029422607 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f708406ae40 msgr2=0x7f708406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:40.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.511+0000 7f70a3fff640 1 --2- 172.21.15.40:0/3029422607 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f708406ae40 0x7f708406d300 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7094035000 tx=0x7f7094004730 comp rx=0 tx=0).stop 2023-10-18T09:04:40.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.511+0000 7f70a3fff640 1 -- 172.21.15.40:0/3029422607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a415af70 msgr2=0x7f70a4159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:40.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.511+0000 7f70a3fff640 1 --2- 172.21.15.40:0/3029422607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a415af70 0x7f70a4159c10 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f709c00c7a0 tx=0x7f709c00cc60 comp rx=0 tx=0).stop 2023-10-18T09:04:40.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.512+0000 7f70a3fff640 1 -- 172.21.15.40:0/3029422607 shutdown_connections 2023-10-18T09:04:40.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.512+0000 7f70a3fff640 1 --2- 172.21.15.40:0/3029422607 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f708406ae40 0x7f708406d300 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:40.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.512+0000 7f70a3fff640 1 --2- 172.21.15.40:0/3029422607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70a415af70 0x7f70a4159c10 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:40.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.512+0000 7f70a3fff640 1 --2- 172.21.15.40:0/3029422607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70a40710a0 0x7f70a41f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:40.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.512+0000 7f70a3fff640 1 --2- 172.21.15.40:0/3029422607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70a406d5d0 0x7f70a41513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:40.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.512+0000 7f70a3fff640 1 -- 172.21.15.40:0/3029422607 >> 172.21.15.40:0/3029422607 conn(0x7f70a406ca10 msgr2=0x7f70a40cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:40.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.512+0000 7f70a3fff640 1 -- 172.21.15.40:0/3029422607 shutdown_connections 2023-10-18T09:04:40.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.512+0000 7f70a3fff640 1 -- 172.21.15.40:0/3029422607 wait complete. 2023-10-18T09:04:40.537 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T09:04:40.538 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:04:40.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.762+0000 7f57fdd93640 1 -- 172.21.15.40:0/2479108960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f806d6f0 msgr2=0x7f57f80b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:40.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.762+0000 7f57fdd93640 1 --2- 172.21.15.40:0/2479108960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f806d6f0 0x7f57f80b15b0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f57ec005fd0 tx=0x7f57ec02be20 comp rx=0 tx=0).stop 2023-10-18T09:04:40.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.763+0000 7f57fdd93640 1 -- 172.21.15.40:0/2479108960 shutdown_connections 2023-10-18T09:04:40.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.763+0000 7f57fdd93640 1 --2- 172.21.15.40:0/2479108960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f806d6f0 0x7f57f80b15b0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:40.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.763+0000 7f57fdd93640 1 -- 172.21.15.40:0/2479108960 >> 172.21.15.40:0/2479108960 conn(0x7f57f8069720 msgr2=0x7f57f8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:40.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.763+0000 7f57fdd93640 1 -- 172.21.15.40:0/2479108960 shutdown_connections 2023-10-18T09:04:40.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.763+0000 7f57fdd93640 1 -- 172.21.15.40:0/2479108960 wait complete. 2023-10-18T09:04:40.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.764+0000 7f57fdd93640 1 Processor -- start 2023-10-18T09:04:40.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.765+0000 7f57fdd93640 1 -- start start 2023-10-18T09:04:40.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.766+0000 7f57fdd93640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57f806d6f0 0x7f57f81383c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:40.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.766+0000 7f57fdd93640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57f80b0630 0x7f57f81378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:40.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.766+0000 7f57fdd93640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f80ca0d0 0x7f57f8148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:40.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.766+0000 7f57fdd93640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f813a7c0 con 0x7f57f80ca0d0 2023-10-18T09:04:40.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.766+0000 7f57fdd93640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f813aaa0 con 0x7f57f806d6f0 2023-10-18T09:04:40.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.766+0000 7f57fdd93640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f813ad80 con 0x7f57f80b0630 2023-10-18T09:04:40.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.766+0000 7f57fd592640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f80ca0d0 0x7f57f8148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:40.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.766+0000 7f57fd592640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f80ca0d0 0x7f57f8148fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43498/0 (socket says 172.21.15.40:43498) 2023-10-18T09:04:40.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.766+0000 7f57fd592640 1 -- 172.21.15.40:0/404062990 learned_addr learned my addr 172.21.15.40:0/404062990 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:40.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.766+0000 7f57fcd91640 1 --2- 172.21.15.40:0/404062990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57f806d6f0 0x7f57f81383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:40.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.766+0000 7f57f7fff640 1 --2- 172.21.15.40:0/404062990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57f80b0630 0x7f57f81378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:40.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.766+0000 7f57fd592640 1 -- 172.21.15.40:0/404062990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57f80b0630 msgr2=0x7f57f81378c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:40.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.766+0000 7f57fd592640 1 --2- 172.21.15.40:0/404062990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57f80b0630 0x7f57f81378c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:40.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.767+0000 7f57fd592640 1 -- 172.21.15.40:0/404062990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57f806d6f0 msgr2=0x7f57f81383c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:40.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.767+0000 7f57fd592640 1 --2- 172.21.15.40:0/404062990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57f806d6f0 0x7f57f81383c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:40.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.767+0000 7f57fd592640 1 -- 172.21.15.40:0/404062990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec009560 con 0x7f57f80ca0d0 2023-10-18T09:04:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.767+0000 7f57fd592640 1 --2- 172.21.15.40:0/404062990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f80ca0d0 0x7f57f8148fe0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f57f000b990 tx=0x7f57f000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.767+0000 7f57f57fa640 1 -- 172.21.15.40:0/404062990 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f000cba0 con 0x7f57f80ca0d0 2023-10-18T09:04:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.767+0000 7f57f57fa640 1 -- 172.21.15.40:0/404062990 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f57f000cdf0 con 0x7f57f80ca0d0 2023-10-18T09:04:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.767+0000 7f57f57fa640 1 -- 172.21.15.40:0/404062990 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f00146a0 con 0x7f57f80ca0d0 2023-10-18T09:04:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.767+0000 7f57fdd93640 1 -- 172.21.15.40:0/404062990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f815bef0 con 0x7f57f80ca0d0 2023-10-18T09:04:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.767+0000 7f57fdd93640 1 -- 172.21.15.40:0/404062990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f8137e00 con 0x7f57f80ca0d0 2023-10-18T09:04:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.769+0000 7f57fdd93640 1 -- 172.21.15.40:0/404062990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f8149fd0 con 0x7f57f80ca0d0 2023-10-18T09:04:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.770+0000 7f57f57fa640 1 -- 172.21.15.40:0/404062990 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f57f00148c0 con 0x7f57f80ca0d0 2023-10-18T09:04:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.771+0000 7f57f57fa640 1 --2- 172.21.15.40:0/404062990 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57dc06ae40 0x7f57dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:40.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.771+0000 7f57f57fa640 1 -- 172.21.15.40:0/404062990 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57f00898a0 con 0x7f57f80ca0d0 2023-10-18T09:04:40.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.771+0000 7f57fcd91640 1 --2- 172.21.15.40:0/404062990 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57dc06ae40 0x7f57dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:40.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.772+0000 7f57fcd91640 1 --2- 172.21.15.40:0/404062990 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57dc06ae40 0x7f57dc06d300 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f57ec035000 tx=0x7f57ec004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:40.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:40.775+0000 7f57f57fa640 1 -- 172.21.15.40:0/404062990 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57f8149fd0 con 0x7f57f80ca0d0 2023-10-18T09:04:41.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.013+0000 7f57fdd93640 1 -- 172.21.15.40:0/404062990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57f8149fd0 con 0x7f57f80ca0d0 2023-10-18T09:04:41.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.014+0000 7f57f57fa640 1 -- 172.21.15.40:0/404062990 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f57f8149fd0 con 0x7f57f80ca0d0 2023-10-18T09:04:41.018 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:41.018 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3243865919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3243865919},{"type":"v1","addr":"172.21.15.40:6835","nonce":3243865919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6047,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/504515257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":504515257},{"type":"v1","addr":"172.21.15.144:6835","nonce":504515257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2513603652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2513603652},{"type":"v1","addr":"172.21.15.144:6837","nonce":2513603652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:04:36.950432+0000","modified":"2023-10-18T09:04:40.030449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8388":{"gid":8388,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/3348437493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3348437493},{"type":"v1","addr":"172.21.15.40:6833","nonce":3348437493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-18T09:04:41.019 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 74 2023-10-18T09:04:41.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.017+0000 7f57fdd93640 1 -- 172.21.15.40:0/404062990 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57dc06ae40 msgr2=0x7f57dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:41.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.017+0000 7f57fdd93640 1 --2- 172.21.15.40:0/404062990 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57dc06ae40 0x7f57dc06d300 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f57ec035000 tx=0x7f57ec004730 comp rx=0 tx=0).stop 2023-10-18T09:04:41.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.017+0000 7f57fdd93640 1 -- 172.21.15.40:0/404062990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f80ca0d0 msgr2=0x7f57f8148fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:41.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.017+0000 7f57fdd93640 1 --2- 172.21.15.40:0/404062990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f80ca0d0 0x7f57f8148fe0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f57f000b990 tx=0x7f57f000be50 comp rx=0 tx=0).stop 2023-10-18T09:04:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.017+0000 7f57fdd93640 1 -- 172.21.15.40:0/404062990 shutdown_connections 2023-10-18T09:04:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.017+0000 7f57fdd93640 1 --2- 172.21.15.40:0/404062990 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57dc06ae40 0x7f57dc06d300 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.017+0000 7f57fdd93640 1 --2- 172.21.15.40:0/404062990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f80ca0d0 0x7f57f8148fe0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.017+0000 7f57fdd93640 1 --2- 172.21.15.40:0/404062990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57f80b0630 0x7f57f81378c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.017+0000 7f57fdd93640 1 --2- 172.21.15.40:0/404062990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57f806d6f0 0x7f57f81383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.017+0000 7f57fdd93640 1 -- 172.21.15.40:0/404062990 >> 172.21.15.40:0/404062990 conn(0x7f57f8069720 msgr2=0x7f57f814e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:41.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.018+0000 7f57fdd93640 1 -- 172.21.15.40:0/404062990 shutdown_connections 2023-10-18T09:04:41.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.018+0000 7f57fdd93640 1 -- 172.21.15.40:0/404062990 wait complete. 2023-10-18T09:04:41.040 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:04:41.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.266+0000 7f2533fff640 1 -- 172.21.15.40:0/2060937519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25341420f0 msgr2=0x7f25340b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:41.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.266+0000 7f2533fff640 1 --2- 172.21.15.40:0/2060937519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25341420f0 0x7f25340b0a80 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f253415dfb0 tx=0x7f252402bd90 comp rx=0 tx=0).stop 2023-10-18T09:04:41.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.266+0000 7f2533fff640 1 -- 172.21.15.40:0/2060937519 shutdown_connections 2023-10-18T09:04:41.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.266+0000 7f2533fff640 1 --2- 172.21.15.40:0/2060937519 >> v2:172.21.15.144:3300/0 conn(0x7f25340c9f90 0x7f253406b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.266+0000 7f2533fff640 1 --2- 172.21.15.40:0/2060937519 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25341425a0 0x7f25340c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.266+0000 7f2533fff640 1 --2- 172.21.15.40:0/2060937519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25341420f0 0x7f25340b0a80 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.266+0000 7f2533fff640 1 -- 172.21.15.40:0/2060937519 >> 172.21.15.40:0/2060937519 conn(0x7f2534069720 msgr2=0x7f2534069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:41.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.266+0000 7f2533fff640 1 -- 172.21.15.40:0/2060937519 shutdown_connections 2023-10-18T09:04:41.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.266+0000 7f2533fff640 1 -- 172.21.15.40:0/2060937519 wait complete. 2023-10-18T09:04:41.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.268+0000 7f2533fff640 1 Processor -- start 2023-10-18T09:04:41.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.268+0000 7f2533fff640 1 -- start start 2023-10-18T09:04:41.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.269+0000 7f2533fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25340c9f90 0x7f2534159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:41.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.269+0000 7f2533fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25341420f0 0x7f2534158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:41.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.269+0000 7f2533fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25341425a0 0x7f2534158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:41.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.269+0000 7f2533fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2534131710 con 0x7f25340c9f90 2023-10-18T09:04:41.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.269+0000 7f2533fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2534131430 con 0x7f25341425a0 2023-10-18T09:04:41.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.269+0000 7f2533fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25341f31a0 con 0x7f25341420f0 2023-10-18T09:04:41.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.270+0000 7f2532ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25340c9f90 0x7f2534159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:41.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.270+0000 7f2532ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25340c9f90 0x7f2534159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43520/0 (socket says 172.21.15.40:43520) 2023-10-18T09:04:41.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.270+0000 7f2532ffd640 1 -- 172.21.15.40:0/38608948 learned_addr learned my addr 172.21.15.40:0/38608948 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:41.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.270+0000 7f25337fe640 1 --2- 172.21.15.40:0/38608948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25341425a0 0x7f2534158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:41.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.270+0000 7f25327fc640 1 --2- 172.21.15.40:0/38608948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25341420f0 0x7f2534158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:41.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.270+0000 7f2532ffd640 1 -- 172.21.15.40:0/38608948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25341420f0 msgr2=0x7f2534158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:41.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.270+0000 7f2532ffd640 1 --2- 172.21.15.40:0/38608948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25341420f0 0x7f2534158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.270+0000 7f2532ffd640 1 -- 172.21.15.40:0/38608948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25341425a0 msgr2=0x7f2534158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:41.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.270+0000 7f2532ffd640 1 --2- 172.21.15.40:0/38608948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25341425a0 0x7f2534158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.270+0000 7f2532ffd640 1 -- 172.21.15.40:0/38608948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2524009560 con 0x7f25340c9f90 2023-10-18T09:04:41.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.270+0000 7f25337fe640 1 --2- 172.21.15.40:0/38608948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25341425a0 0x7f2534158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:41.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.271+0000 7f2532ffd640 1 --2- 172.21.15.40:0/38608948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25340c9f90 0x7f2534159c30 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f2524009850 tx=0x7f252402da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:41.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.271+0000 7f251b7fe640 1 -- 172.21.15.40:0/38608948 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252402de60 con 0x7f25340c9f90 2023-10-18T09:04:41.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.271+0000 7f251b7fe640 1 -- 172.21.15.40:0/38608948 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f25240036e0 con 0x7f25340c9f90 2023-10-18T09:04:41.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.271+0000 7f251b7fe640 1 -- 172.21.15.40:0/38608948 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2524003a90 con 0x7f25340c9f90 2023-10-18T09:04:41.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.271+0000 7f2533fff640 1 -- 172.21.15.40:0/38608948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25341f3480 con 0x7f25340c9f90 2023-10-18T09:04:41.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.271+0000 7f2533fff640 1 -- 172.21.15.40:0/38608948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253415a170 con 0x7f25340c9f90 2023-10-18T09:04:41.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.273+0000 7f2533fff640 1 -- 172.21.15.40:0/38608948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25341f3000 con 0x7f25340c9f90 2023-10-18T09:04:41.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.274+0000 7f251b7fe640 1 -- 172.21.15.40:0/38608948 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2524003bf0 con 0x7f25340c9f90 2023-10-18T09:04:41.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.275+0000 7f251b7fe640 1 --2- 172.21.15.40:0/38608948 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f252006ae40 0x7f252006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:41.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.275+0000 7f251b7fe640 1 -- 172.21.15.40:0/38608948 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25240ac280 con 0x7f25340c9f90 2023-10-18T09:04:41.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.276+0000 7f25327fc640 1 --2- 172.21.15.40:0/38608948 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f252006ae40 0x7f252006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:41.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.277+0000 7f25327fc640 1 --2- 172.21.15.40:0/38608948 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f252006ae40 0x7f252006d300 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f25280058c0 tx=0x7f2528005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:41.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.279+0000 7f251b7fe640 1 -- 172.21.15.40:0/38608948 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f25341f3000 con 0x7f25340c9f90 2023-10-18T09:04:41.427 INFO:tasks.ceph.mds.c.smithi040.stderr:2023-10-18T09:04:41.423+0000 7f97bacbf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T09:04:41.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.482+0000 7f2533fff640 1 -- 172.21.15.40:0/38608948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25341f3000 con 0x7f25340c9f90 2023-10-18T09:04:41.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.483+0000 7f251b7fe640 1 -- 172.21.15.40:0/38608948 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16592 (secure 0 0 0) 0x7f25341f3000 con 0x7f25340c9f90 2023-10-18T09:04:41.488 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:41.488 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":162,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:04:38.458686+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:04:34.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-18T09:04:35.972659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:04:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.486+0000 7f2533fff640 1 -- 172.21.15.40:0/38608948 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f252006ae40 msgr2=0x7f252006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.486+0000 7f2533fff640 1 --2- 172.21.15.40:0/38608948 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f252006ae40 0x7f252006d300 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f25280058c0 tx=0x7f2528005850 comp rx=0 tx=0).stop 2023-10-18T09:04:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.486+0000 7f2533fff640 1 -- 172.21.15.40:0/38608948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25340c9f90 msgr2=0x7f2534159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.486+0000 7f2533fff640 1 --2- 172.21.15.40:0/38608948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25340c9f90 0x7f2534159c30 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f2524009850 tx=0x7f252402da60 comp rx=0 tx=0).stop 2023-10-18T09:04:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.486+0000 7f2533fff640 1 -- 172.21.15.40:0/38608948 shutdown_connections 2023-10-18T09:04:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.486+0000 7f2533fff640 1 --2- 172.21.15.40:0/38608948 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f252006ae40 0x7f252006d300 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.486+0000 7f2533fff640 1 --2- 172.21.15.40:0/38608948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25341425a0 0x7f2534158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.486+0000 7f2533fff640 1 --2- 172.21.15.40:0/38608948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25341420f0 0x7f2534158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.486+0000 7f2533fff640 1 --2- 172.21.15.40:0/38608948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25340c9f90 0x7f2534159c30 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.487+0000 7f2533fff640 1 -- 172.21.15.40:0/38608948 >> 172.21.15.40:0/38608948 conn(0x7f2534069720 msgr2=0x7f253413fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:41.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.487+0000 7f2533fff640 1 -- 172.21.15.40:0/38608948 shutdown_connections 2023-10-18T09:04:41.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.487+0000 7f2533fff640 1 -- 172.21.15.40:0/38608948 wait complete. 2023-10-18T09:04:41.510 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T09:04:41.511 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:04:41.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.735+0000 7f54150f2640 1 -- 172.21.15.40:0/2328909299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54100cd0a0 msgr2=0x7f54100cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:41.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.735+0000 7f54150f2640 1 --2- 172.21.15.40:0/2328909299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54100cd0a0 0x7f54100cd4a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f5408005fd0 tx=0x7f540802c0f0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.735+0000 7f54150f2640 1 -- 172.21.15.40:0/2328909299 shutdown_connections 2023-10-18T09:04:41.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.735+0000 7f54150f2640 1 --2- 172.21.15.40:0/2328909299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54100cd0a0 0x7f54100cd4a0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.735+0000 7f54150f2640 1 -- 172.21.15.40:0/2328909299 >> 172.21.15.40:0/2328909299 conn(0x7f541006cee0 msgr2=0x7f5410069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:41.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.735+0000 7f54150f2640 1 -- 172.21.15.40:0/2328909299 shutdown_connections 2023-10-18T09:04:41.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.735+0000 7f54150f2640 1 -- 172.21.15.40:0/2328909299 wait complete. 2023-10-18T09:04:41.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.737+0000 7f54150f2640 1 Processor -- start 2023-10-18T09:04:41.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.737+0000 7f54150f2640 1 -- start start 2023-10-18T09:04:41.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.738+0000 7f54150f2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541006d710 0x7f5410142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:41.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.738+0000 7f54150f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54100cd0a0 0x7f5410070150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:41.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.738+0000 7f54150f2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54101586a0 0x7f5410070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:41.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.738+0000 7f54150f2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5410142cd0 con 0x7f54100cd0a0 2023-10-18T09:04:41.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.738+0000 7f54150f2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5410135150 con 0x7f54101586a0 2023-10-18T09:04:41.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.738+0000 7f54150f2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54101f21c0 con 0x7f541006d710 2023-10-18T09:04:41.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.738+0000 7f540e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54100cd0a0 0x7f5410070150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:41.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.739+0000 7f540e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54100cd0a0 0x7f5410070150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43550/0 (socket says 172.21.15.40:43550) 2023-10-18T09:04:41.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.739+0000 7f540e59c640 1 -- 172.21.15.40:0/1551972119 learned_addr learned my addr 172.21.15.40:0/1551972119 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:41.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.739+0000 7f540f59e640 1 --2- 172.21.15.40:0/1551972119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54101586a0 0x7f5410070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:41.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.739+0000 7f540ed9d640 1 --2- 172.21.15.40:0/1551972119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541006d710 0x7f5410142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:41.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.739+0000 7f540e59c640 1 -- 172.21.15.40:0/1551972119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541006d710 msgr2=0x7f5410142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:41.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.739+0000 7f540e59c640 1 --2- 172.21.15.40:0/1551972119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541006d710 0x7f5410142790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.739+0000 7f540e59c640 1 -- 172.21.15.40:0/1551972119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54101586a0 msgr2=0x7f5410070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:41.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.739+0000 7f540e59c640 1 --2- 172.21.15.40:0/1551972119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54101586a0 0x7f5410070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.739+0000 7f540e59c640 1 -- 172.21.15.40:0/1551972119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5408009840 con 0x7f54100cd0a0 2023-10-18T09:04:41.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.739+0000 7f540ed9d640 1 --2- 172.21.15.40:0/1551972119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541006d710 0x7f5410142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:41.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.739+0000 7f540e59c640 1 --2- 172.21.15.40:0/1551972119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54100cd0a0 0x7f5410070150 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f540400c390 tx=0x7f540400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:41.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.740+0000 7f53f77fe640 1 -- 172.21.15.40:0/1551972119 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5404004230 con 0x7f54100cd0a0 2023-10-18T09:04:41.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.740+0000 7f53f77fe640 1 -- 172.21.15.40:0/1551972119 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f54040044a0 con 0x7f54100cd0a0 2023-10-18T09:04:41.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.740+0000 7f53f77fe640 1 -- 172.21.15.40:0/1551972119 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5404003850 con 0x7f54100cd0a0 2023-10-18T09:04:41.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.740+0000 7f54150f2640 1 -- 172.21.15.40:0/1551972119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5410132e40 con 0x7f54100cd0a0 2023-10-18T09:04:41.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.740+0000 7f54150f2640 1 -- 172.21.15.40:0/1551972119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5410133400 con 0x7f54100cd0a0 2023-10-18T09:04:41.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.741+0000 7f54150f2640 1 -- 172.21.15.40:0/1551972119 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5410073780 con 0x7f54100cd0a0 2023-10-18T09:04:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.742+0000 7f53f77fe640 1 -- 172.21.15.40:0/1551972119 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f54040039b0 con 0x7f54100cd0a0 2023-10-18T09:04:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.743+0000 7f53f77fe640 1 --2- 172.21.15.40:0/1551972119 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f53fc06ae40 0x7f53fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.743+0000 7f53f77fe640 1 -- 172.21.15.40:0/1551972119 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5404088f70 con 0x7f54100cd0a0 2023-10-18T09:04:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.744+0000 7f540ed9d640 1 --2- 172.21.15.40:0/1551972119 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f53fc06ae40 0x7f53fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:41.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.745+0000 7f540ed9d640 1 --2- 172.21.15.40:0/1551972119 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f53fc06ae40 0x7f53fc06d300 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5400003eb0 tx=0x7f5400009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:41.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.747+0000 7f53f77fe640 1 -- 172.21.15.40:0/1551972119 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5404053fb0 con 0x7f54100cd0a0 2023-10-18T09:04:41.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.936+0000 7f54150f2640 1 -- 172.21.15.40:0/1551972119 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54100cd9e0 con 0x7f53fc06ae40 2023-10-18T09:04:41.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.959+0000 7f53f77fe640 1 -- 172.21.15.40:0/1551972119 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282632 (secure 0 0 0) 0x7f5400005030 con 0x7f53fc06ae40 2023-10-18T09:04:41.967 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:41.969 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:04:41.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.966+0000 7f54150f2640 1 -- 172.21.15.40:0/1551972119 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f53fc06ae40 msgr2=0x7f53fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:41.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.966+0000 7f54150f2640 1 --2- 172.21.15.40:0/1551972119 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f53fc06ae40 0x7f53fc06d300 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5400003eb0 tx=0x7f5400009040 comp rx=0 tx=0).stop 2023-10-18T09:04:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.966+0000 7f54150f2640 1 -- 172.21.15.40:0/1551972119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54100cd0a0 msgr2=0x7f5410070150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.966+0000 7f54150f2640 1 --2- 172.21.15.40:0/1551972119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54100cd0a0 0x7f5410070150 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f540400c390 tx=0x7f540400c850 comp rx=0 tx=0).stop 2023-10-18T09:04:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.966+0000 7f54150f2640 1 -- 172.21.15.40:0/1551972119 shutdown_connections 2023-10-18T09:04:41.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.966+0000 7f54150f2640 1 --2- 172.21.15.40:0/1551972119 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f53fc06ae40 0x7f53fc06d300 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.966+0000 7f54150f2640 1 --2- 172.21.15.40:0/1551972119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54101586a0 0x7f5410070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.966+0000 7f54150f2640 1 --2- 172.21.15.40:0/1551972119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54100cd0a0 0x7f5410070150 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.966+0000 7f54150f2640 1 --2- 172.21.15.40:0/1551972119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f541006d710 0x7f5410142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.966+0000 7f54150f2640 1 -- 172.21.15.40:0/1551972119 >> 172.21.15.40:0/1551972119 conn(0x7f541006cee0 msgr2=0x7f5410159060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.966+0000 7f54150f2640 1 -- 172.21.15.40:0/1551972119 shutdown_connections 2023-10-18T09:04:41.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:41.966+0000 7f54150f2640 1 -- 172.21.15.40:0/1551972119 wait complete. 2023-10-18T09:04:41.990 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2023-10-18T09:04:41.103750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8274,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554225,"statfs":{"total":1600000000,"available":1591526617,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.659809"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.490895+0000","last_change":"2023-10-18T09:04:37.490831+0000","last_active":"2023-10-18T09:04:37.490895+0000","last_peered":"2023-10-18T09:04:37.490895+0000","last_clean":"2023-10-18T09:04:37.490895+0000","last_became_active":"2023-10-18T09:04:37.490510+0000","last_became_peered":"2023-10-18T09:04:37.490510+0000","last_unstale":"2023-10-18T09:04:37.490895+0000","last_undegraded":"2023-10-18T09:04:37.490895+0000","last_fullsized":"2023-10-18T09:04:37.490895+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:18.541619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.491808+0000","last_change":"2023-10-18T09:04:37.491742+0000","last_active":"2023-10-18T09:04:37.491808+0000","last_peered":"2023-10-18T09:04:37.491808+0000","last_clean":"2023-10-18T09:04:37.491808+0000","last_became_active":"2023-10-18T09:04:37.491527+0000","last_became_peered":"2023-10-18T09:04:37.491527+0000","last_unstale":"2023-10-18T09:04:37.491808+0000","last_undegraded":"2023-10-18T09:04:37.491808+0000","last_fullsized":"2023-10-18T09:04:37.491808+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:09.081005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.500346+0000","last_change":"2023-10-18T09:04:37.500271+0000","last_active":"2023-10-18T09:04:37.500346+0000","last_peered":"2023-10-18T09:04:37.500346+0000","last_clean":"2023-10-18T09:04:37.500346+0000","last_became_active":"2023-10-18T09:04:37.499853+0000","last_became_peered":"2023-10-18T09:04:37.499853+0000","last_unstale":"2023-10-18T09:04:37.500346+0000","last_undegraded":"2023-10-18T09:04:37.500346+0000","last_fullsized":"2023-10-18T09:04:37.500346+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:17.069751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.482658+0000","last_change":"2023-10-18T09:04:37.482251+0000","last_active":"2023-10-18T09:04:37.482658+0000","last_peered":"2023-10-18T09:04:37.482658+0000","last_clean":"2023-10-18T09:04:37.482658+0000","last_became_active":"2023-10-18T09:04:37.482031+0000","last_became_peered":"2023-10-18T09:04:37.482031+0000","last_unstale":"2023-10-18T09:04:37.482658+0000","last_undegraded":"2023-10-18T09:04:37.482658+0000","last_fullsized":"2023-10-18T09:04:37.482658+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:31:08.551966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.468253+0000","last_change":"2023-10-18T09:04:37.491538+0000","last_active":"2023-10-18T09:04:38.468253+0000","last_peered":"2023-10-18T09:04:38.468253+0000","last_clean":"2023-10-18T09:04:38.468253+0000","last_became_active":"2023-10-18T09:04:37.491305+0000","last_became_peered":"2023-10-18T09:04:37.491305+0000","last_unstale":"2023-10-18T09:04:38.468253+0000","last_undegraded":"2023-10-18T09:04:38.468253+0000","last_fullsized":"2023-10-18T09:04:38.468253+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:16.055093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.469666+0000","last_change":"2023-10-18T09:04:37.489837+0000","last_active":"2023-10-18T09:04:38.469666+0000","last_peered":"2023-10-18T09:04:38.469666+0000","last_clean":"2023-10-18T09:04:38.469666+0000","last_became_active":"2023-10-18T09:04:37.489056+0000","last_became_peered":"2023-10-18T09:04:37.489056+0000","last_unstale":"2023-10-18T09:04:38.469666+0000","last_undegraded":"2023-10-18T09:04:38.469666+0000","last_fullsized":"2023-10-18T09:04:38.469666+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:04:19.707753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.467967+0000","last_change":"2023-10-18T09:04:37.497488+0000","last_active":"2023-10-18T09:04:38.467967+0000","last_peered":"2023-10-18T09:04:38.467967+0000","last_clean":"2023-10-18T09:04:38.467967+0000","last_became_active":"2023-10-18T09:04:37.497031+0000","last_became_peered":"2023-10-18T09:04:37.497031+0000","last_unstale":"2023-10-18T09:04:38.467967+0000","last_undegraded":"2023-10-18T09:04:38.467967+0000","last_fullsized":"2023-10-18T09:04:38.467967+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:50:14.955866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.498157+0000","last_change":"2023-10-18T09:04:37.498089+0000","last_active":"2023-10-18T09:04:37.498157+0000","last_peered":"2023-10-18T09:04:37.498157+0000","last_clean":"2023-10-18T09:04:37.498157+0000","last_became_active":"2023-10-18T09:04:37.496058+0000","last_became_peered":"2023-10-18T09:04:37.496058+0000","last_unstale":"2023-10-18T09:04:37.498157+0000","last_undegraded":"2023-10-18T09:04:37.498157+0000","last_fullsized":"2023-10-18T09:04:37.498157+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:47:38.150973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.495636+0000","last_change":"2023-10-18T09:04:37.495358+0000","last_active":"2023-10-18T09:04:37.495636+0000","last_peered":"2023-10-18T09:04:37.495636+0000","last_clean":"2023-10-18T09:04:37.495636+0000","last_became_active":"2023-10-18T09:04:37.495122+0000","last_became_peered":"2023-10-18T09:04:37.495122+0000","last_unstale":"2023-10-18T09:04:37.495636+0000","last_undegraded":"2023-10-18T09:04:37.495636+0000","last_fullsized":"2023-10-18T09:04:37.495636+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:14:42.919064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.487957+0000","last_change":"2023-10-18T09:04:37.487883+0000","last_active":"2023-10-18T09:04:37.487957+0000","last_peered":"2023-10-18T09:04:37.487957+0000","last_clean":"2023-10-18T09:04:37.487957+0000","last_became_active":"2023-10-18T09:04:37.487614+0000","last_became_peered":"2023-10-18T09:04:37.487614+0000","last_unstale":"2023-10-18T09:04:37.487957+0000","last_undegraded":"2023-10-18T09:04:37.487957+0000","last_fullsized":"2023-10-18T09:04:37.487957+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:26.364922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.489021+0000","last_change":"2023-10-18T09:04:37.488428+0000","last_active":"2023-10-18T09:04:37.489021+0000","last_peered":"2023-10-18T09:04:37.489021+0000","last_clean":"2023-10-18T09:04:37.489021+0000","last_became_active":"2023-10-18T09:04:37.488192+0000","last_became_peered":"2023-10-18T09:04:37.488192+0000","last_unstale":"2023-10-18T09:04:37.489021+0000","last_undegraded":"2023-10-18T09:04:37.489021+0000","last_fullsized":"2023-10-18T09:04:37.489021+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:09.554842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.499822+0000","last_change":"2023-10-18T09:04:37.499716+0000","last_active":"2023-10-18T09:04:37.499822+0000","last_peered":"2023-10-18T09:04:37.499822+0000","last_clean":"2023-10-18T09:04:37.499822+0000","last_became_active":"2023-10-18T09:04:37.498099+0000","last_became_peered":"2023-10-18T09:04:37.498099+0000","last_unstale":"2023-10-18T09:04:37.499822+0000","last_undegraded":"2023-10-18T09:04:37.499822+0000","last_fullsized":"2023-10-18T09:04:37.499822+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:29.338172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.481050+0000","last_change":"2023-10-18T09:04:37.479677+0000","last_active":"2023-10-18T09:04:37.481050+0000","last_peered":"2023-10-18T09:04:37.481050+0000","last_clean":"2023-10-18T09:04:37.481050+0000","last_became_active":"2023-10-18T09:04:37.479264+0000","last_became_peered":"2023-10-18T09:04:37.479264+0000","last_unstale":"2023-10-18T09:04:37.481050+0000","last_undegraded":"2023-10-18T09:04:37.481050+0000","last_fullsized":"2023-10-18T09:04:37.481050+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:50:29.089915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.482468+0000","last_change":"2023-10-18T09:04:37.482046+0000","last_active":"2023-10-18T09:04:37.482468+0000","last_peered":"2023-10-18T09:04:37.482468+0000","last_clean":"2023-10-18T09:04:37.482468+0000","last_became_active":"2023-10-18T09:04:37.481814+0000","last_became_peered":"2023-10-18T09:04:37.481814+0000","last_unstale":"2023-10-18T09:04:37.482468+0000","last_undegraded":"2023-10-18T09:04:37.482468+0000","last_fullsized":"2023-10-18T09:04:37.482468+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:10:13.433915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.499838+0000","last_change":"2023-10-18T09:04:37.499786+0000","last_active":"2023-10-18T09:04:37.499838+0000","last_peered":"2023-10-18T09:04:37.499838+0000","last_clean":"2023-10-18T09:04:37.499838+0000","last_became_active":"2023-10-18T09:04:37.499337+0000","last_became_peered":"2023-10-18T09:04:37.499337+0000","last_unstale":"2023-10-18T09:04:37.499838+0000","last_undegraded":"2023-10-18T09:04:37.499838+0000","last_fullsized":"2023-10-18T09:04:37.499838+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:24.543978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464804+0000","last_change":"2023-10-18T09:04:37.482627+0000","last_active":"2023-10-18T09:04:38.464804+0000","last_peered":"2023-10-18T09:04:38.464804+0000","last_clean":"2023-10-18T09:04:38.464804+0000","last_became_active":"2023-10-18T09:04:37.481814+0000","last_became_peered":"2023-10-18T09:04:37.481814+0000","last_unstale":"2023-10-18T09:04:38.464804+0000","last_undegraded":"2023-10-18T09:04:38.464804+0000","last_fullsized":"2023-10-18T09:04:38.464804+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:31.938682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.468626+0000","last_change":"2023-10-18T09:04:37.492664+0000","last_active":"2023-10-18T09:04:38.468626+0000","last_peered":"2023-10-18T09:04:38.468626+0000","last_clean":"2023-10-18T09:04:38.468626+0000","last_became_active":"2023-10-18T09:04:37.491870+0000","last_became_peered":"2023-10-18T09:04:37.491870+0000","last_unstale":"2023-10-18T09:04:38.468626+0000","last_undegraded":"2023-10-18T09:04:38.468626+0000","last_fullsized":"2023-10-18T09:04:38.468626+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:21:23.553384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464230+0000","last_change":"2023-10-18T09:04:37.475818+0000","last_active":"2023-10-18T09:04:38.464230+0000","last_peered":"2023-10-18T09:04:38.464230+0000","last_clean":"2023-10-18T09:04:38.464230+0000","last_became_active":"2023-10-18T09:04:37.475037+0000","last_became_peered":"2023-10-18T09:04:37.475037+0000","last_unstale":"2023-10-18T09:04:38.464230+0000","last_undegraded":"2023-10-18T09:04:38.464230+0000","last_fullsized":"2023-10-18T09:04:38.464230+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:01:41.587901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.497143+0000","last_change":"2023-10-18T09:04:37.496952+0000","last_active":"2023-10-18T09:04:37.497143+0000","last_peered":"2023-10-18T09:04:37.497143+0000","last_clean":"2023-10-18T09:04:37.497143+0000","last_became_active":"2023-10-18T09:04:37.496762+0000","last_became_peered":"2023-10-18T09:04:37.496762+0000","last_unstale":"2023-10-18T09:04:37.497143+0000","last_undegraded":"2023-10-18T09:04:37.497143+0000","last_fullsized":"2023-10-18T09:04:37.497143+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:05:52.916098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464214+0000","last_change":"2023-10-18T09:04:37.476148+0000","last_active":"2023-10-18T09:04:38.464214+0000","last_peered":"2023-10-18T09:04:38.464214+0000","last_clean":"2023-10-18T09:04:38.464214+0000","last_became_active":"2023-10-18T09:04:37.475900+0000","last_became_peered":"2023-10-18T09:04:37.475900+0000","last_unstale":"2023-10-18T09:04:38.464214+0000","last_undegraded":"2023-10-18T09:04:38.464214+0000","last_fullsized":"2023-10-18T09:04:38.464214+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:49:21.451994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.469080+0000","last_change":"2023-10-18T09:04:37.494887+0000","last_active":"2023-10-18T09:04:38.469080+0000","last_peered":"2023-10-18T09:04:38.469080+0000","last_clean":"2023-10-18T09:04:38.469080+0000","last_became_active":"2023-10-18T09:04:37.491960+0000","last_became_peered":"2023-10-18T09:04:37.491960+0000","last_unstale":"2023-10-18T09:04:38.469080+0000","last_undegraded":"2023-10-18T09:04:38.469080+0000","last_fullsized":"2023-10-18T09:04:38.469080+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:34.585957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.486268+0000","last_change":"2023-10-18T09:04:37.485697+0000","last_active":"2023-10-18T09:04:37.486268+0000","last_peered":"2023-10-18T09:04:37.486268+0000","last_clean":"2023-10-18T09:04:37.486268+0000","last_became_active":"2023-10-18T09:04:37.485367+0000","last_became_peered":"2023-10-18T09:04:37.485367+0000","last_unstale":"2023-10-18T09:04:37.486268+0000","last_undegraded":"2023-10-18T09:04:37.486268+0000","last_fullsized":"2023-10-18T09:04:37.486268+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:55:18.748505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.499361+0000","last_change":"2023-10-18T09:04:37.499141+0000","last_active":"2023-10-18T09:04:37.499361+0000","last_peered":"2023-10-18T09:04:37.499361+0000","last_clean":"2023-10-18T09:04:37.499361+0000","last_became_active":"2023-10-18T09:04:37.496770+0000","last_became_peered":"2023-10-18T09:04:37.496770+0000","last_unstale":"2023-10-18T09:04:37.499361+0000","last_undegraded":"2023-10-18T09:04:37.499361+0000","last_fullsized":"2023-10-18T09:04:37.499361+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:05:39.746894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.486286+0000","last_change":"2023-10-18T09:04:37.485905+0000","last_active":"2023-10-18T09:04:37.486286+0000","last_peered":"2023-10-18T09:04:37.486286+0000","last_clean":"2023-10-18T09:04:37.486286+0000","last_became_active":"2023-10-18T09:04:37.485669+0000","last_became_peered":"2023-10-18T09:04:37.485669+0000","last_unstale":"2023-10-18T09:04:37.486286+0000","last_undegraded":"2023-10-18T09:04:37.486286+0000","last_fullsized":"2023-10-18T09:04:37.486286+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:02:32.105013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.482433+0000","last_change":"2023-10-18T09:04:37.479545+0000","last_active":"2023-10-18T09:04:37.482433+0000","last_peered":"2023-10-18T09:04:37.482433+0000","last_clean":"2023-10-18T09:04:37.482433+0000","last_became_active":"2023-10-18T09:04:37.479139+0000","last_became_peered":"2023-10-18T09:04:37.479139+0000","last_unstale":"2023-10-18T09:04:37.482433+0000","last_undegraded":"2023-10-18T09:04:37.482433+0000","last_fullsized":"2023-10-18T09:04:37.482433+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:24:13.080777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464960+0000","last_change":"2023-10-18T09:04:37.482431+0000","last_active":"2023-10-18T09:04:38.464960+0000","last_peered":"2023-10-18T09:04:38.464960+0000","last_clean":"2023-10-18T09:04:38.464960+0000","last_became_active":"2023-10-18T09:04:37.481153+0000","last_became_peered":"2023-10-18T09:04:37.481153+0000","last_unstale":"2023-10-18T09:04:38.464960+0000","last_undegraded":"2023-10-18T09:04:38.464960+0000","last_fullsized":"2023-10-18T09:04:38.464960+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:44.793477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.496430+0000","last_change":"2023-10-18T09:04:37.495936+0000","last_active":"2023-10-18T09:04:37.496430+0000","last_peered":"2023-10-18T09:04:37.496430+0000","last_clean":"2023-10-18T09:04:37.496430+0000","last_became_active":"2023-10-18T09:04:37.495701+0000","last_became_peered":"2023-10-18T09:04:37.495701+0000","last_unstale":"2023-10-18T09:04:37.496430+0000","last_undegraded":"2023-10-18T09:04:37.496430+0000","last_fullsized":"2023-10-18T09:04:37.496430+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:08:39.927171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.484429+0000","last_change":"2023-10-18T09:04:37.483427+0000","last_active":"2023-10-18T09:04:37.484429+0000","last_peered":"2023-10-18T09:04:37.484429+0000","last_clean":"2023-10-18T09:04:37.484429+0000","last_became_active":"2023-10-18T09:04:37.483010+0000","last_became_peered":"2023-10-18T09:04:37.483010+0000","last_unstale":"2023-10-18T09:04:37.484429+0000","last_undegraded":"2023-10-18T09:04:37.484429+0000","last_fullsized":"2023-10-18T09:04:37.484429+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:24:27.643288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.496449+0000","last_change":"2023-10-18T09:04:37.496057+0000","last_active":"2023-10-18T09:04:37.496449+0000","last_peered":"2023-10-18T09:04:37.496449+0000","last_clean":"2023-10-18T09:04:37.496449+0000","last_became_active":"2023-10-18T09:04:37.495731+0000","last_became_peered":"2023-10-18T09:04:37.495731+0000","last_unstale":"2023-10-18T09:04:37.496449+0000","last_undegraded":"2023-10-18T09:04:37.496449+0000","last_fullsized":"2023-10-18T09:04:37.496449+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:56.596979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.499678+0000","last_change":"2023-10-18T09:04:37.499609+0000","last_active":"2023-10-18T09:04:37.499678+0000","last_peered":"2023-10-18T09:04:37.499678+0000","last_clean":"2023-10-18T09:04:37.499678+0000","last_became_active":"2023-10-18T09:04:37.498990+0000","last_became_peered":"2023-10-18T09:04:37.498990+0000","last_unstale":"2023-10-18T09:04:37.499678+0000","last_undegraded":"2023-10-18T09:04:37.499678+0000","last_fullsized":"2023-10-18T09:04:37.499678+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:03.155226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.467867+0000","last_change":"2023-10-18T09:04:37.485718+0000","last_active":"2023-10-18T09:04:38.467867+0000","last_peered":"2023-10-18T09:04:38.467867+0000","last_clean":"2023-10-18T09:04:38.467867+0000","last_became_active":"2023-10-18T09:04:37.484473+0000","last_became_peered":"2023-10-18T09:04:37.484473+0000","last_unstale":"2023-10-18T09:04:38.467867+0000","last_undegraded":"2023-10-18T09:04:38.467867+0000","last_fullsized":"2023-10-18T09:04:38.467867+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:19:46.360261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.468289+0000","last_change":"2023-10-18T09:04:37.492800+0000","last_active":"2023-10-18T09:04:38.468289+0000","last_peered":"2023-10-18T09:04:38.468289+0000","last_clean":"2023-10-18T09:04:38.468289+0000","last_became_active":"2023-10-18T09:04:37.492171+0000","last_became_peered":"2023-10-18T09:04:37.492171+0000","last_unstale":"2023-10-18T09:04:38.468289+0000","last_undegraded":"2023-10-18T09:04:38.468289+0000","last_fullsized":"2023-10-18T09:04:38.468289+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:06:39.079565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.463961+0000","last_change":"2023-10-18T09:04:37.480923+0000","last_active":"2023-10-18T09:04:38.463961+0000","last_peered":"2023-10-18T09:04:38.463961+0000","last_clean":"2023-10-18T09:04:38.463961+0000","last_became_active":"2023-10-18T09:04:37.480684+0000","last_became_peered":"2023-10-18T09:04:37.480684+0000","last_unstale":"2023-10-18T09:04:38.463961+0000","last_undegraded":"2023-10-18T09:04:38.463961+0000","last_fullsized":"2023-10-18T09:04:38.463961+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:14:10.648103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"161'4","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.466523+0000","last_change":"2023-10-18T09:04:36.471445+0000","last_active":"2023-10-18T09:04:38.466523+0000","last_peered":"2023-10-18T09:04:38.466523+0000","last_clean":"2023-10-18T09:04:38.466523+0000","last_became_active":"2023-10-18T09:04:36.470511+0000","last_became_peered":"2023-10-18T09:04:36.470511+0000","last_unstale":"2023-10-18T09:04:38.466523+0000","last_undegraded":"2023-10-18T09:04:38.466523+0000","last_fullsized":"2023-10-18T09:04:38.466523+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:43:28.013241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.499016+0000","last_change":"2023-10-18T09:04:37.498948+0000","last_active":"2023-10-18T09:04:37.499016+0000","last_peered":"2023-10-18T09:04:37.499016+0000","last_clean":"2023-10-18T09:04:37.499016+0000","last_became_active":"2023-10-18T09:04:37.496366+0000","last_became_peered":"2023-10-18T09:04:37.496366+0000","last_unstale":"2023-10-18T09:04:37.499016+0000","last_undegraded":"2023-10-18T09:04:37.499016+0000","last_fullsized":"2023-10-18T09:04:37.499016+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:01:14.003255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"161'1","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.491453+0000","last_change":"2023-10-18T09:04:36.478506+0000","last_active":"2023-10-18T09:04:37.491453+0000","last_peered":"2023-10-18T09:04:37.491453+0000","last_clean":"2023-10-18T09:04:37.491453+0000","last_became_active":"2023-10-18T09:04:36.477741+0000","last_became_peered":"2023-10-18T09:04:36.477741+0000","last_unstale":"2023-10-18T09:04:37.491453+0000","last_undegraded":"2023-10-18T09:04:37.491453+0000","last_fullsized":"2023-10-18T09:04:37.491453+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:00.168165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.468195+0000","last_change":"2023-10-18T09:04:37.496186+0000","last_active":"2023-10-18T09:04:38.468195+0000","last_peered":"2023-10-18T09:04:38.468195+0000","last_clean":"2023-10-18T09:04:38.468195+0000","last_became_active":"2023-10-18T09:04:37.495917+0000","last_became_peered":"2023-10-18T09:04:37.495917+0000","last_unstale":"2023-10-18T09:04:38.468195+0000","last_undegraded":"2023-10-18T09:04:38.468195+0000","last_fullsized":"2023-10-18T09:04:38.468195+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:08:25.262401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"161'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.488786+0000","last_change":"2023-10-18T09:04:36.476781+0000","last_active":"2023-10-18T09:04:37.488786+0000","last_peered":"2023-10-18T09:04:37.488786+0000","last_clean":"2023-10-18T09:04:37.488786+0000","last_became_active":"2023-10-18T09:04:36.476457+0000","last_became_peered":"2023-10-18T09:04:36.476457+0000","last_unstale":"2023-10-18T09:04:37.488786+0000","last_undegraded":"2023-10-18T09:04:37.488786+0000","last_fullsized":"2023-10-18T09:04:37.488786+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:40.977761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.468593+0000","last_change":"2023-10-18T09:04:37.489105+0000","last_active":"2023-10-18T09:04:38.468593+0000","last_peered":"2023-10-18T09:04:38.468593+0000","last_clean":"2023-10-18T09:04:38.468593+0000","last_became_active":"2023-10-18T09:04:37.488451+0000","last_became_peered":"2023-10-18T09:04:37.488451+0000","last_unstale":"2023-10-18T09:04:38.468593+0000","last_undegraded":"2023-10-18T09:04:38.468593+0000","last_fullsized":"2023-10-18T09:04:38.468593+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:30:05.448881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"161'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.485161+0000","last_change":"2023-10-18T09:04:36.472754+0000","last_active":"2023-10-18T09:04:37.485161+0000","last_peered":"2023-10-18T09:04:37.485161+0000","last_clean":"2023-10-18T09:04:37.485161+0000","last_became_active":"2023-10-18T09:04:36.472506+0000","last_became_peered":"2023-10-18T09:04:36.472506+0000","last_unstale":"2023-10-18T09:04:37.485161+0000","last_undegraded":"2023-10-18T09:04:37.485161+0000","last_fullsized":"2023-10-18T09:04:37.485161+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:16:35.109333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464444+0000","last_change":"2023-10-18T09:04:37.475267+0000","last_active":"2023-10-18T09:04:38.464444+0000","last_peered":"2023-10-18T09:04:38.464444+0000","last_clean":"2023-10-18T09:04:38.464444+0000","last_became_active":"2023-10-18T09:04:37.473167+0000","last_became_peered":"2023-10-18T09:04:37.473167+0000","last_unstale":"2023-10-18T09:04:38.464444+0000","last_undegraded":"2023-10-18T09:04:38.464444+0000","last_fullsized":"2023-10-18T09:04:38.464444+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:19:15.019455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"161'1","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.482597+0000","last_change":"2023-10-18T09:04:36.469557+0000","last_active":"2023-10-18T09:04:37.482597+0000","last_peered":"2023-10-18T09:04:37.482597+0000","last_clean":"2023-10-18T09:04:37.482597+0000","last_became_active":"2023-10-18T09:04:36.469311+0000","last_became_peered":"2023-10-18T09:04:36.469311+0000","last_unstale":"2023-10-18T09:04:37.482597+0000","last_undegraded":"2023-10-18T09:04:37.482597+0000","last_fullsized":"2023-10-18T09:04:37.482597+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:04:55.278850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.468954+0000","last_change":"2023-10-18T09:04:37.492521+0000","last_active":"2023-10-18T09:04:38.468954+0000","last_peered":"2023-10-18T09:04:38.468954+0000","last_clean":"2023-10-18T09:04:38.468954+0000","last_became_active":"2023-10-18T09:04:37.491449+0000","last_became_peered":"2023-10-18T09:04:37.491449+0000","last_unstale":"2023-10-18T09:04:38.468954+0000","last_undegraded":"2023-10-18T09:04:38.468954+0000","last_fullsized":"2023-10-18T09:04:38.468954+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:30:02.310872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"161'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464572+0000","last_change":"2023-10-18T09:04:36.466293+0000","last_active":"2023-10-18T09:04:38.464572+0000","last_peered":"2023-10-18T09:04:38.464572+0000","last_clean":"2023-10-18T09:04:38.464572+0000","last_became_active":"2023-10-18T09:04:36.466055+0000","last_became_peered":"2023-10-18T09:04:36.466055+0000","last_unstale":"2023-10-18T09:04:38.464572+0000","last_undegraded":"2023-10-18T09:04:38.464572+0000","last_fullsized":"2023-10-18T09:04:38.464572+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:45.608235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.488389+0000","last_change":"2023-10-18T09:04:37.487476+0000","last_active":"2023-10-18T09:04:37.488389+0000","last_peered":"2023-10-18T09:04:37.488389+0000","last_clean":"2023-10-18T09:04:37.488389+0000","last_became_active":"2023-10-18T09:04:37.487188+0000","last_became_peered":"2023-10-18T09:04:37.487188+0000","last_unstale":"2023-10-18T09:04:37.488389+0000","last_undegraded":"2023-10-18T09:04:37.488389+0000","last_fullsized":"2023-10-18T09:04:37.488389+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:29:21.820463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.466288+0000","last_change":"2023-10-18T09:04:36.484738+0000","last_active":"2023-10-18T09:04:38.466288+0000","last_peered":"2023-10-18T09:04:38.466288+0000","last_clean":"2023-10-18T09:04:38.466288+0000","last_became_active":"2023-10-18T09:04:36.484459+0000","last_became_peered":"2023-10-18T09:04:36.484459+0000","last_unstale":"2023-10-18T09:04:38.466288+0000","last_undegraded":"2023-10-18T09:04:38.466288+0000","last_fullsized":"2023-10-18T09:04:38.466288+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:26:15.613015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.475051+0000","last_change":"2023-10-18T09:04:37.474563+0000","last_active":"2023-10-18T09:04:37.475051+0000","last_peered":"2023-10-18T09:04:37.475051+0000","last_clean":"2023-10-18T09:04:37.475051+0000","last_became_active":"2023-10-18T09:04:37.474327+0000","last_became_peered":"2023-10-18T09:04:37.474327+0000","last_unstale":"2023-10-18T09:04:37.475051+0000","last_undegraded":"2023-10-18T09:04:37.475051+0000","last_fullsized":"2023-10-18T09:04:37.475051+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:45.114942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.474300+0000","last_change":"2023-10-18T09:04:36.481384+0000","last_active":"2023-10-18T09:04:37.474300+0000","last_peered":"2023-10-18T09:04:37.474300+0000","last_clean":"2023-10-18T09:04:37.474300+0000","last_became_active":"2023-10-18T09:04:36.481043+0000","last_became_peered":"2023-10-18T09:04:36.481043+0000","last_unstale":"2023-10-18T09:04:37.474300+0000","last_undegraded":"2023-10-18T09:04:37.474300+0000","last_fullsized":"2023-10-18T09:04:37.474300+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:20:40.862835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.484435+0000","last_change":"2023-10-18T09:04:37.484242+0000","last_active":"2023-10-18T09:04:37.484435+0000","last_peered":"2023-10-18T09:04:37.484435+0000","last_clean":"2023-10-18T09:04:37.484435+0000","last_became_active":"2023-10-18T09:04:37.483807+0000","last_became_peered":"2023-10-18T09:04:37.483807+0000","last_unstale":"2023-10-18T09:04:37.484435+0000","last_undegraded":"2023-10-18T09:04:37.484435+0000","last_fullsized":"2023-10-18T09:04:37.484435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:02.637433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.477985+0000","last_change":"2023-10-18T09:04:36.479268+0000","last_active":"2023-10-18T09:04:37.477985+0000","last_peered":"2023-10-18T09:04:37.477985+0000","last_clean":"2023-10-18T09:04:37.477985+0000","last_became_active":"2023-10-18T09:04:36.478796+0000","last_became_peered":"2023-10-18T09:04:36.478796+0000","last_unstale":"2023-10-18T09:04:37.477985+0000","last_undegraded":"2023-10-18T09:04:37.477985+0000","last_fullsized":"2023-10-18T09:04:37.477985+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:52.652054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.494013+0000","last_change":"2023-10-18T09:04:37.492588+0000","last_active":"2023-10-18T09:04:37.494013+0000","last_peered":"2023-10-18T09:04:37.494013+0000","last_clean":"2023-10-18T09:04:37.494013+0000","last_became_active":"2023-10-18T09:04:37.492318+0000","last_became_peered":"2023-10-18T09:04:37.492318+0000","last_unstale":"2023-10-18T09:04:37.494013+0000","last_undegraded":"2023-10-18T09:04:37.494013+0000","last_fullsized":"2023-10-18T09:04:37.494013+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:24:13.080777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464214+0000","last_change":"2023-10-18T09:04:36.475728+0000","last_active":"2023-10-18T09:04:38.464214+0000","last_peered":"2023-10-18T09:04:38.464214+0000","last_clean":"2023-10-18T09:04:38.464214+0000","last_became_active":"2023-10-18T09:04:36.475221+0000","last_became_peered":"2023-10-18T09:04:36.475221+0000","last_unstale":"2023-10-18T09:04:38.464214+0000","last_undegraded":"2023-10-18T09:04:38.464214+0000","last_fullsized":"2023-10-18T09:04:38.464214+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:01:12.988228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.466342+0000","last_change":"2023-10-18T09:04:37.475822+0000","last_active":"2023-10-18T09:04:38.466342+0000","last_peered":"2023-10-18T09:04:38.466342+0000","last_clean":"2023-10-18T09:04:38.466342+0000","last_became_active":"2023-10-18T09:04:37.475163+0000","last_became_peered":"2023-10-18T09:04:37.475163+0000","last_unstale":"2023-10-18T09:04:38.466342+0000","last_undegraded":"2023-10-18T09:04:38.466342+0000","last_fullsized":"2023-10-18T09:04:38.466342+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:43:44.007470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"161'3","reported_seq":14,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.495304+0000","last_change":"2023-10-18T09:04:36.480880+0000","last_active":"2023-10-18T09:04:37.495304+0000","last_peered":"2023-10-18T09:04:37.495304+0000","last_clean":"2023-10-18T09:04:37.495304+0000","last_became_active":"2023-10-18T09:04:36.480577+0000","last_became_peered":"2023-10-18T09:04:36.480577+0000","last_unstale":"2023-10-18T09:04:37.495304+0000","last_undegraded":"2023-10-18T09:04:37.495304+0000","last_fullsized":"2023-10-18T09:04:37.495304+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:25.938371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.466275+0000","last_change":"2023-10-18T09:04:37.497277+0000","last_active":"2023-10-18T09:04:38.466275+0000","last_peered":"2023-10-18T09:04:38.466275+0000","last_clean":"2023-10-18T09:04:38.466275+0000","last_became_active":"2023-10-18T09:04:37.497040+0000","last_became_peered":"2023-10-18T09:04:37.497040+0000","last_unstale":"2023-10-18T09:04:38.466275+0000","last_undegraded":"2023-10-18T09:04:38.466275+0000","last_fullsized":"2023-10-18T09:04:38.466275+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:50:00.258088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"161'6","reported_seq":17,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.500880+0000","last_change":"2023-10-18T09:04:36.484500+0000","last_active":"2023-10-18T09:04:37.500880+0000","last_peered":"2023-10-18T09:04:37.500880+0000","last_clean":"2023-10-18T09:04:37.500880+0000","last_became_active":"2023-10-18T09:04:36.484205+0000","last_became_peered":"2023-10-18T09:04:36.484205+0000","last_unstale":"2023-10-18T09:04:37.500880+0000","last_undegraded":"2023-10-18T09:04:37.500880+0000","last_fullsized":"2023-10-18T09:04:37.500880+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:44:22.781501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.486999+0000","last_change":"2023-10-18T09:04:37.486852+0000","last_active":"2023-10-18T09:04:37.486999+0000","last_peered":"2023-10-18T09:04:37.486999+0000","last_clean":"2023-10-18T09:04:37.486999+0000","last_became_active":"2023-10-18T09:04:37.486601+0000","last_became_peered":"2023-10-18T09:04:37.486601+0000","last_unstale":"2023-10-18T09:04:37.486999+0000","last_undegraded":"2023-10-18T09:04:37.486999+0000","last_fullsized":"2023-10-18T09:04:37.486999+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:46:18.746561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.472166+0000","last_change":"2023-10-18T09:04:36.478034+0000","last_active":"2023-10-18T09:04:37.472166+0000","last_peered":"2023-10-18T09:04:37.472166+0000","last_clean":"2023-10-18T09:04:37.472166+0000","last_became_active":"2023-10-18T09:04:36.477811+0000","last_became_peered":"2023-10-18T09:04:36.477811+0000","last_unstale":"2023-10-18T09:04:37.472166+0000","last_undegraded":"2023-10-18T09:04:37.472166+0000","last_fullsized":"2023-10-18T09:04:37.472166+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:22:42.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.497913+0000","last_change":"2023-10-18T09:04:37.497813+0000","last_active":"2023-10-18T09:04:37.497913+0000","last_peered":"2023-10-18T09:04:37.497913+0000","last_clean":"2023-10-18T09:04:37.497913+0000","last_became_active":"2023-10-18T09:04:37.497528+0000","last_became_peered":"2023-10-18T09:04:37.497528+0000","last_unstale":"2023-10-18T09:04:37.497913+0000","last_undegraded":"2023-10-18T09:04:37.497913+0000","last_fullsized":"2023-10-18T09:04:37.497913+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:27:26.368529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.471920+0000","last_change":"2023-10-18T09:04:36.482256+0000","last_active":"2023-10-18T09:04:37.471920+0000","last_peered":"2023-10-18T09:04:37.471920+0000","last_clean":"2023-10-18T09:04:37.471920+0000","last_became_active":"2023-10-18T09:04:36.481678+0000","last_became_peered":"2023-10-18T09:04:36.481678+0000","last_unstale":"2023-10-18T09:04:37.471920+0000","last_undegraded":"2023-10-18T09:04:37.471920+0000","last_fullsized":"2023-10-18T09:04:37.471920+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:13:40.119610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.499241+0000","last_change":"2023-10-18T09:04:37.498864+0000","last_active":"2023-10-18T09:04:37.499241+0000","last_peered":"2023-10-18T09:04:37.499241+0000","last_clean":"2023-10-18T09:04:37.499241+0000","last_became_active":"2023-10-18T09:04:37.496458+0000","last_became_peered":"2023-10-18T09:04:37.496458+0000","last_unstale":"2023-10-18T09:04:37.499241+0000","last_undegraded":"2023-10-18T09:04:37.499241+0000","last_fullsized":"2023-10-18T09:04:37.499241+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:34:11.046102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"161'8","reported_seq":19,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.504419+0000","last_change":"2023-10-18T09:04:36.474474+0000","last_active":"2023-10-18T09:04:37.504419+0000","last_peered":"2023-10-18T09:04:37.504419+0000","last_clean":"2023-10-18T09:04:37.504419+0000","last_became_active":"2023-10-18T09:04:36.473849+0000","last_became_peered":"2023-10-18T09:04:36.473849+0000","last_unstale":"2023-10-18T09:04:37.504419+0000","last_undegraded":"2023-10-18T09:04:37.504419+0000","last_fullsized":"2023-10-18T09:04:37.504419+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:45.608235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.495604+0000","last_change":"2023-10-18T09:04:37.495199+0000","last_active":"2023-10-18T09:04:37.495604+0000","last_peered":"2023-10-18T09:04:37.495604+0000","last_clean":"2023-10-18T09:04:37.495604+0000","last_became_active":"2023-10-18T09:04:37.494917+0000","last_became_peered":"2023-10-18T09:04:37.494917+0000","last_unstale":"2023-10-18T09:04:37.495604+0000","last_undegraded":"2023-10-18T09:04:37.495604+0000","last_fullsized":"2023-10-18T09:04:37.495604+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:15:38.352083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"161'1","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.497691+0000","last_change":"2023-10-18T09:04:36.484780+0000","last_active":"2023-10-18T09:04:37.497691+0000","last_peered":"2023-10-18T09:04:37.497691+0000","last_clean":"2023-10-18T09:04:37.497691+0000","last_became_active":"2023-10-18T09:04:36.484512+0000","last_became_peered":"2023-10-18T09:04:36.484512+0000","last_unstale":"2023-10-18T09:04:37.497691+0000","last_undegraded":"2023-10-18T09:04:37.497691+0000","last_fullsized":"2023-10-18T09:04:37.497691+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:22.674318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.489276+0000","last_change":"2023-10-18T09:04:37.487955+0000","last_active":"2023-10-18T09:04:37.489276+0000","last_peered":"2023-10-18T09:04:37.489276+0000","last_clean":"2023-10-18T09:04:37.489276+0000","last_became_active":"2023-10-18T09:04:37.487720+0000","last_became_peered":"2023-10-18T09:04:37.487720+0000","last_unstale":"2023-10-18T09:04:37.489276+0000","last_undegraded":"2023-10-18T09:04:37.489276+0000","last_fullsized":"2023-10-18T09:04:37.489276+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:28:31.128917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"161'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.496383+0000","last_change":"2023-10-18T09:04:36.483032+0000","last_active":"2023-10-18T09:04:37.496383+0000","last_peered":"2023-10-18T09:04:37.496383+0000","last_clean":"2023-10-18T09:04:37.496383+0000","last_became_active":"2023-10-18T09:04:36.482811+0000","last_became_peered":"2023-10-18T09:04:36.482811+0000","last_unstale":"2023-10-18T09:04:37.496383+0000","last_undegraded":"2023-10-18T09:04:37.496383+0000","last_fullsized":"2023-10-18T09:04:37.496383+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:09.373841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464079+0000","last_change":"2023-10-18T09:04:37.480507+0000","last_active":"2023-10-18T09:04:38.464079+0000","last_peered":"2023-10-18T09:04:38.464079+0000","last_clean":"2023-10-18T09:04:38.464079+0000","last_became_active":"2023-10-18T09:04:37.480272+0000","last_became_peered":"2023-10-18T09:04:37.480272+0000","last_unstale":"2023-10-18T09:04:38.464079+0000","last_undegraded":"2023-10-18T09:04:38.464079+0000","last_fullsized":"2023-10-18T09:04:38.464079+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:31:31.734747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"161'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.470193+0000","last_change":"2023-10-18T09:04:36.473416+0000","last_active":"2023-10-18T09:04:38.470193+0000","last_peered":"2023-10-18T09:04:38.470193+0000","last_clean":"2023-10-18T09:04:38.470193+0000","last_became_active":"2023-10-18T09:04:36.472278+0000","last_became_peered":"2023-10-18T09:04:36.472278+0000","last_unstale":"2023-10-18T09:04:38.470193+0000","last_undegraded":"2023-10-18T09:04:38.470193+0000","last_fullsized":"2023-10-18T09:04:38.470193+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:15.629297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.482451+0000","last_change":"2023-10-18T09:04:37.479718+0000","last_active":"2023-10-18T09:04:37.482451+0000","last_peered":"2023-10-18T09:04:37.482451+0000","last_clean":"2023-10-18T09:04:37.482451+0000","last_became_active":"2023-10-18T09:04:37.479434+0000","last_became_peered":"2023-10-18T09:04:37.479434+0000","last_unstale":"2023-10-18T09:04:37.482451+0000","last_undegraded":"2023-10-18T09:04:37.482451+0000","last_fullsized":"2023-10-18T09:04:37.482451+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:26.953399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.475095+0000","last_change":"2023-10-18T09:04:36.476351+0000","last_active":"2023-10-18T09:04:37.475095+0000","last_peered":"2023-10-18T09:04:37.475095+0000","last_clean":"2023-10-18T09:04:37.475095+0000","last_became_active":"2023-10-18T09:04:36.475442+0000","last_became_peered":"2023-10-18T09:04:36.475442+0000","last_unstale":"2023-10-18T09:04:37.475095+0000","last_undegraded":"2023-10-18T09:04:37.475095+0000","last_fullsized":"2023-10-18T09:04:37.475095+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:53.827947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.485430+0000","last_change":"2023-10-18T09:04:37.485334+0000","last_active":"2023-10-18T09:04:37.485430+0000","last_peered":"2023-10-18T09:04:37.485430+0000","last_clean":"2023-10-18T09:04:37.485430+0000","last_became_active":"2023-10-18T09:04:37.485122+0000","last_became_peered":"2023-10-18T09:04:37.485122+0000","last_unstale":"2023-10-18T09:04:37.485430+0000","last_undegraded":"2023-10-18T09:04:37.485430+0000","last_fullsized":"2023-10-18T09:04:37.485430+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:46:20.922307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"161'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.497372+0000","last_change":"2023-10-18T09:04:36.478652+0000","last_active":"2023-10-18T09:04:37.497372+0000","last_peered":"2023-10-18T09:04:37.497372+0000","last_clean":"2023-10-18T09:04:37.497372+0000","last_became_active":"2023-10-18T09:04:36.478372+0000","last_became_peered":"2023-10-18T09:04:36.478372+0000","last_unstale":"2023-10-18T09:04:37.497372+0000","last_undegraded":"2023-10-18T09:04:37.497372+0000","last_fullsized":"2023-10-18T09:04:37.497372+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:10:12.418887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.466328+0000","last_change":"2023-10-18T09:04:37.498983+0000","last_active":"2023-10-18T09:04:38.466328+0000","last_peered":"2023-10-18T09:04:38.466328+0000","last_clean":"2023-10-18T09:04:38.466328+0000","last_became_active":"2023-10-18T09:04:37.498731+0000","last_became_peered":"2023-10-18T09:04:37.498731+0000","last_unstale":"2023-10-18T09:04:38.466328+0000","last_undegraded":"2023-10-18T09:04:38.466328+0000","last_fullsized":"2023-10-18T09:04:38.466328+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:10:01.517852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464328+0000","last_change":"2023-10-18T09:04:36.482989+0000","last_active":"2023-10-18T09:04:38.464328+0000","last_peered":"2023-10-18T09:04:38.464328+0000","last_clean":"2023-10-18T09:04:38.464328+0000","last_became_active":"2023-10-18T09:04:36.482640+0000","last_became_peered":"2023-10-18T09:04:36.482640+0000","last_unstale":"2023-10-18T09:04:38.464328+0000","last_undegraded":"2023-10-18T09:04:38.464328+0000","last_fullsized":"2023-10-18T09:04:38.464328+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:29:20.805435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.486878+0000","last_change":"2023-10-18T09:04:37.486734+0000","last_active":"2023-10-18T09:04:37.486878+0000","last_peered":"2023-10-18T09:04:37.486878+0000","last_clean":"2023-10-18T09:04:37.486878+0000","last_became_active":"2023-10-18T09:04:37.486483+0000","last_became_peered":"2023-10-18T09:04:37.486483+0000","last_unstale":"2023-10-18T09:04:37.486878+0000","last_undegraded":"2023-10-18T09:04:37.486878+0000","last_fullsized":"2023-10-18T09:04:37.486878+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:53.667082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464329+0000","last_change":"2023-10-18T09:04:36.477401+0000","last_active":"2023-10-18T09:04:38.464329+0000","last_peered":"2023-10-18T09:04:38.464329+0000","last_clean":"2023-10-18T09:04:38.464329+0000","last_became_active":"2023-10-18T09:04:36.477056+0000","last_became_peered":"2023-10-18T09:04:36.477056+0000","last_unstale":"2023-10-18T09:04:38.464329+0000","last_undegraded":"2023-10-18T09:04:38.464329+0000","last_fullsized":"2023-10-18T09:04:38.464329+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:15:37.337056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.467927+0000","last_change":"2023-10-18T09:04:37.488951+0000","last_active":"2023-10-18T09:04:38.467927+0000","last_peered":"2023-10-18T09:04:38.467927+0000","last_clean":"2023-10-18T09:04:38.467927+0000","last_became_active":"2023-10-18T09:04:37.487915+0000","last_became_peered":"2023-10-18T09:04:37.487915+0000","last_unstale":"2023-10-18T09:04:38.467927+0000","last_undegraded":"2023-10-18T09:04:38.467927+0000","last_fullsized":"2023-10-18T09:04:38.467927+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:17:44.053614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.483956+0000","last_change":"2023-10-18T09:04:36.481368+0000","last_active":"2023-10-18T09:04:37.483956+0000","last_peered":"2023-10-18T09:04:37.483956+0000","last_clean":"2023-10-18T09:04:37.483956+0000","last_became_active":"2023-10-18T09:04:36.481009+0000","last_became_peered":"2023-10-18T09:04:36.481009+0000","last_unstale":"2023-10-18T09:04:37.483956+0000","last_undegraded":"2023-10-18T09:04:37.483956+0000","last_fullsized":"2023-10-18T09:04:37.483956+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:10:08.053287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.466675+0000","last_change":"2023-10-18T09:04:37.487140+0000","last_active":"2023-10-18T09:04:38.466675+0000","last_peered":"2023-10-18T09:04:38.466675+0000","last_clean":"2023-10-18T09:04:38.466675+0000","last_became_active":"2023-10-18T09:04:37.486128+0000","last_became_peered":"2023-10-18T09:04:37.486128+0000","last_unstale":"2023-10-18T09:04:38.466675+0000","last_undegraded":"2023-10-18T09:04:38.466675+0000","last_fullsized":"2023-10-18T09:04:38.466675+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:38:21.843954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"161'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.481425+0000","last_change":"2023-10-18T09:04:36.480113+0000","last_active":"2023-10-18T09:04:37.481425+0000","last_peered":"2023-10-18T09:04:37.481425+0000","last_clean":"2023-10-18T09:04:37.481425+0000","last_became_active":"2023-10-18T09:04:36.479802+0000","last_became_peered":"2023-10-18T09:04:36.479802+0000","last_unstale":"2023-10-18T09:04:37.481425+0000","last_undegraded":"2023-10-18T09:04:37.481425+0000","last_fullsized":"2023-10-18T09:04:37.481425+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:03.392350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.497657+0000","last_change":"2023-10-18T09:04:37.497545+0000","last_active":"2023-10-18T09:04:37.497657+0000","last_peered":"2023-10-18T09:04:37.497657+0000","last_clean":"2023-10-18T09:04:37.497657+0000","last_became_active":"2023-10-18T09:04:37.497301+0000","last_became_peered":"2023-10-18T09:04:37.497301+0000","last_unstale":"2023-10-18T09:04:37.497657+0000","last_undegraded":"2023-10-18T09:04:37.497657+0000","last_fullsized":"2023-10-18T09:04:37.497657+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:42.351629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.466784+0000","last_change":"2023-10-18T09:04:36.482262+0000","last_active":"2023-10-18T09:04:38.466784+0000","last_peered":"2023-10-18T09:04:38.466784+0000","last_clean":"2023-10-18T09:04:38.466784+0000","last_became_active":"2023-10-18T09:04:36.481928+0000","last_became_peered":"2023-10-18T09:04:36.481928+0000","last_unstale":"2023-10-18T09:04:38.466784+0000","last_undegraded":"2023-10-18T09:04:38.466784+0000","last_fullsized":"2023-10-18T09:04:38.466784+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:33.159619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.482412+0000","last_change":"2023-10-18T09:04:37.479395+0000","last_active":"2023-10-18T09:04:37.482412+0000","last_peered":"2023-10-18T09:04:37.482412+0000","last_clean":"2023-10-18T09:04:37.482412+0000","last_became_active":"2023-10-18T09:04:37.478828+0000","last_became_peered":"2023-10-18T09:04:37.478828+0000","last_unstale":"2023-10-18T09:04:37.482412+0000","last_undegraded":"2023-10-18T09:04:37.482412+0000","last_fullsized":"2023-10-18T09:04:37.482412+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:28:31.128917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.471000+0000","last_change":"2023-10-18T09:04:36.482319+0000","last_active":"2023-10-18T09:04:37.471000+0000","last_peered":"2023-10-18T09:04:37.471000+0000","last_clean":"2023-10-18T09:04:37.471000+0000","last_became_active":"2023-10-18T09:04:36.482100+0000","last_became_peered":"2023-10-18T09:04:36.482100+0000","last_unstale":"2023-10-18T09:04:37.471000+0000","last_undegraded":"2023-10-18T09:04:37.471000+0000","last_fullsized":"2023-10-18T09:04:37.471000+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:00.168165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.466071+0000","last_change":"2023-10-18T09:04:37.493295+0000","last_active":"2023-10-18T09:04:38.466071+0000","last_peered":"2023-10-18T09:04:38.466071+0000","last_clean":"2023-10-18T09:04:38.466071+0000","last_became_active":"2023-10-18T09:04:37.491590+0000","last_became_peered":"2023-10-18T09:04:37.491590+0000","last_unstale":"2023-10-18T09:04:38.466071+0000","last_undegraded":"2023-10-18T09:04:38.466071+0000","last_fullsized":"2023-10-18T09:04:38.466071+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:25:29.576966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.480409+0000","last_change":"2023-10-18T09:04:36.474249+0000","last_active":"2023-10-18T09:04:37.480409+0000","last_peered":"2023-10-18T09:04:37.480409+0000","last_clean":"2023-10-18T09:04:37.480409+0000","last_became_active":"2023-10-18T09:04:36.473336+0000","last_became_peered":"2023-10-18T09:04:36.473336+0000","last_unstale":"2023-10-18T09:04:37.480409+0000","last_undegraded":"2023-10-18T09:04:37.480409+0000","last_fullsized":"2023-10-18T09:04:37.480409+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:34:10.320203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.467138+0000","last_change":"2023-10-18T09:04:37.488680+0000","last_active":"2023-10-18T09:04:38.467138+0000","last_peered":"2023-10-18T09:04:38.467138+0000","last_clean":"2023-10-18T09:04:38.467138+0000","last_became_active":"2023-10-18T09:04:37.488362+0000","last_became_peered":"2023-10-18T09:04:37.488362+0000","last_unstale":"2023-10-18T09:04:38.467138+0000","last_undegraded":"2023-10-18T09:04:38.467138+0000","last_fullsized":"2023-10-18T09:04:38.467138+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:45.059619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.471332+0000","last_change":"2023-10-18T09:04:36.481955+0000","last_active":"2023-10-18T09:04:37.471332+0000","last_peered":"2023-10-18T09:04:37.471332+0000","last_clean":"2023-10-18T09:04:37.471332+0000","last_became_active":"2023-10-18T09:04:36.481665+0000","last_became_peered":"2023-10-18T09:04:36.481665+0000","last_unstale":"2023-10-18T09:04:37.471332+0000","last_undegraded":"2023-10-18T09:04:37.471332+0000","last_fullsized":"2023-10-18T09:04:37.471332+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:28.778957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.489296+0000","last_change":"2023-10-18T09:04:37.485920+0000","last_active":"2023-10-18T09:04:37.489296+0000","last_peered":"2023-10-18T09:04:37.489296+0000","last_clean":"2023-10-18T09:04:37.489296+0000","last_became_active":"2023-10-18T09:04:37.485646+0000","last_became_peered":"2023-10-18T09:04:37.485646+0000","last_unstale":"2023-10-18T09:04:37.489296+0000","last_undegraded":"2023-10-18T09:04:37.489296+0000","last_fullsized":"2023-10-18T09:04:37.489296+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:45.114942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.467500+0000","last_change":"2023-10-18T09:04:36.474466+0000","last_active":"2023-10-18T09:04:38.467500+0000","last_peered":"2023-10-18T09:04:38.467500+0000","last_clean":"2023-10-18T09:04:38.467500+0000","last_became_active":"2023-10-18T09:04:36.473572+0000","last_became_peered":"2023-10-18T09:04:36.473572+0000","last_unstale":"2023-10-18T09:04:38.467500+0000","last_undegraded":"2023-10-18T09:04:38.467500+0000","last_fullsized":"2023-10-18T09:04:38.467500+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:45.314291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464182+0000","last_change":"2023-10-18T09:04:37.489671+0000","last_active":"2023-10-18T09:04:38.464182+0000","last_peered":"2023-10-18T09:04:38.464182+0000","last_clean":"2023-10-18T09:04:38.464182+0000","last_became_active":"2023-10-18T09:04:37.489434+0000","last_became_peered":"2023-10-18T09:04:37.489434+0000","last_unstale":"2023-10-18T09:04:38.464182+0000","last_undegraded":"2023-10-18T09:04:38.464182+0000","last_fullsized":"2023-10-18T09:04:38.464182+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:07:54.115841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464707+0000","last_change":"2023-10-18T09:04:36.475206+0000","last_active":"2023-10-18T09:04:38.464707+0000","last_peered":"2023-10-18T09:04:38.464707+0000","last_clean":"2023-10-18T09:04:38.464707+0000","last_became_active":"2023-10-18T09:04:36.474771+0000","last_became_peered":"2023-10-18T09:04:36.474771+0000","last_unstale":"2023-10-18T09:04:38.464707+0000","last_undegraded":"2023-10-18T09:04:38.464707+0000","last_fullsized":"2023-10-18T09:04:38.464707+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:23:19.444961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.484487+0000","last_change":"2023-10-18T09:04:37.483172+0000","last_active":"2023-10-18T09:04:37.484487+0000","last_peered":"2023-10-18T09:04:37.484487+0000","last_clean":"2023-10-18T09:04:37.484487+0000","last_became_active":"2023-10-18T09:04:37.482822+0000","last_became_peered":"2023-10-18T09:04:37.482822+0000","last_unstale":"2023-10-18T09:04:37.484487+0000","last_undegraded":"2023-10-18T09:04:37.484487+0000","last_fullsized":"2023-10-18T09:04:37.484487+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:23.689345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.483431+0000","last_change":"2023-10-18T09:04:36.478477+0000","last_active":"2023-10-18T09:04:37.483431+0000","last_peered":"2023-10-18T09:04:37.483431+0000","last_clean":"2023-10-18T09:04:37.483431+0000","last_became_active":"2023-10-18T09:04:36.478223+0000","last_became_peered":"2023-10-18T09:04:36.478223+0000","last_unstale":"2023-10-18T09:04:37.483431+0000","last_undegraded":"2023-10-18T09:04:37.483431+0000","last_fullsized":"2023-10-18T09:04:37.483431+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:11.857064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.468561+0000","last_change":"2023-10-18T09:04:37.486003+0000","last_active":"2023-10-18T09:04:38.468561+0000","last_peered":"2023-10-18T09:04:38.468561+0000","last_clean":"2023-10-18T09:04:38.468561+0000","last_became_active":"2023-10-18T09:04:37.485197+0000","last_became_peered":"2023-10-18T09:04:37.485197+0000","last_unstale":"2023-10-18T09:04:38.468561+0000","last_undegraded":"2023-10-18T09:04:38.468561+0000","last_fullsized":"2023-10-18T09:04:38.468561+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:33:09.755834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"161'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.468861+0000","last_change":"2023-10-18T09:04:36.477786+0000","last_active":"2023-10-18T09:04:38.468861+0000","last_peered":"2023-10-18T09:04:38.468861+0000","last_clean":"2023-10-18T09:04:38.468861+0000","last_became_active":"2023-10-18T09:04:36.477028+0000","last_became_peered":"2023-10-18T09:04:36.477028+0000","last_unstale":"2023-10-18T09:04:38.468861+0000","last_undegraded":"2023-10-18T09:04:38.468861+0000","last_fullsized":"2023-10-18T09:04:38.468861+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:02:13.687471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":320,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.467642+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:04:37.467642+0000","last_peered":"2023-10-18T09:04:37.467642+0000","last_clean":"2023-10-18T09:04:37.467642+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:04:37.467642+0000","last_undegraded":"2023-10-18T09:04:37.467642+0000","last_fullsized":"2023-10-18T09:04:37.467642+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574965,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1472,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193839,"statfs":{"total":200000000,"available":198492084,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.1080000000000001,"15min":1.042},"min":{"1min":0.70099999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5620000000000001,"5min":1.6040000000000001,"15min":1.6990000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0880000000000001,"15min":1.05},"min":{"1min":0.63300000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.552,"5min":1.6439999999999999,"15min":1.97},"last":1.2509999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.177,"15min":1.1539999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.61799999999999999},"max":{"1min":1.6839999999999999,"5min":1.829,"15min":1.829},"last":1.179},{"interface":"front","average":{"1min":1.038,"5min":1.089,"15min":1.121},"min":{"1min":0.73399999999999999,"5min":0.61099999999999999,"15min":0.58799999999999997},"max":{"1min":1.409,"5min":1.5469999999999999,"15min":1.7989999999999999},"last":1.468}]},{"osd":2,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3089999999999999,"15min":1.27},"min":{"1min":0.94099999999999995,"5min":0.78600000000000003,"15min":0.65400000000000003},"max":{"1min":1.7250000000000001,"5min":1.8169999999999999,"15min":1.9730000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.266,"15min":1.224},"min":{"1min":0.79400000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.671,"5min":1.863,"15min":1.863},"last":1.369}]},{"osd":3,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2809999999999999,"15min":1.266},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.677,"5min":1.98,"15min":1.98},"last":1.319},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3480000000000001,"15min":1.3320000000000001},"min":{"1min":0.83199999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7210000000000001,"5min":1.756,"15min":1.9930000000000001},"last":1.7170000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.2370000000000001,"15min":1.196},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60499999999999998},"max":{"1min":1.6459999999999999,"5min":1.923,"15min":2.1070000000000002},"last":1.133},{"interface":"front","average":{"1min":1.27,"5min":1.335,"15min":1.3680000000000001},"min":{"1min":0.77800000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.8169999999999999,"5min":2.0369999999999999,"15min":2.5169999999999999},"last":1.4139999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.323,"15min":1.347},"min":{"1min":0.89300000000000002,"5min":0.622,"15min":0.622},"max":{"1min":1.6339999999999999,"5min":1.831,"15min":2.0720000000000001},"last":1.7989999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.425,"15min":1.4179999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9490000000000001,"5min":2.0259999999999998,"15min":2.0270000000000001},"last":1.544}]},{"osd":6,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.502,"15min":1.472},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.0139999999999998,"5min":2.0779999999999998,"15min":2.3959999999999999},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.496,"15min":1.5},"min":{"1min":0.80800000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8779999999999999,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.6499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574965,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":889,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194423,"statfs":{"total":200000000,"available":199089390,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:04:11 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.256,"15min":1.1539999999999999},"min":{"1min":0.85299999999999998,"5min":0.64700000000000002,"15min":0.60299999999999998},"max":{"1min":1.4950000000000001,"5min":1.627,"15min":1.6850000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.353,"15min":1.2769999999999999},"min":{"1min":0.88800000000000001,"5min":0.81799999999999995,"15min":0.64700000000000002},"max":{"1min":1.5640000000000001,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.0900000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:04:11 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.107,"15min":1.0940000000000001},"min":{"1min":0.68799999999999994,"5min":0.63900000000000001,"15min":0.63800000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.2050000000000001,"15min":1.1970000000000001},"min":{"1min":0.73099999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5720000000000001,"5min":1.8560000000000001,"15min":2.9609999999999999},"last":1.2669999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:04:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.1839999999999999,"15min":1.1759999999999999},"min":{"1min":0.75900000000000001,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.232,"15min":1.212},"min":{"1min":0.91200000000000003,"5min":0.73399999999999999,"15min":0.66300000000000003},"max":{"1min":1.5620000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.4139999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:04:11 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.296,"15min":1.268},"min":{"1min":0.89100000000000001,"5min":0.81000000000000005,"15min":0.79200000000000004},"max":{"1min":1.5629999999999999,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.333},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3120000000000001,"15min":1.2549999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.3029999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:04:11 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.2889999999999999,"15min":1.2230000000000001},"min":{"1min":0.82699999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.8089999999999999,"5min":2.444,"15min":2.444},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.3049999999999999,"15min":1.2809999999999999},"min":{"1min":0.86099999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.756,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.1659999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:04:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4399999999999999,"15min":1.3879999999999999},"min":{"1min":0.99299999999999999,"5min":0.625,"15min":0.58899999999999997},"max":{"1min":1.675,"5min":2.032,"15min":2.032},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.3580000000000001,"15min":1.2729999999999999},"min":{"1min":0.91300000000000003,"5min":0.55100000000000005,"15min":0.49199999999999999},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.218}]},{"osd":7,"last update":"Wed Oct 18 09:04:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4119999999999999,"15min":1.4039999999999999},"min":{"1min":0.89400000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6830000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.48},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.552,"15min":1.528},"min":{"1min":1.226,"5min":0.83099999999999996,"15min":0.68500000000000005},"max":{"1min":1.9199999999999999,"5min":2.3140000000000001,"15min":2.794},"last":1.3999999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":887,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194424,"statfs":{"total":200000000,"available":199090890,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:04:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.325,"15min":1.278},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.63400000000000001},"max":{"1min":1.595,"5min":9.609,"15min":9.609},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2909999999999999,"15min":1.302},"min":{"1min":0.94499999999999995,"5min":0.751,"15min":0.60799999999999998},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":5.3029999999999999},"last":1.4450000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:04:18 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3380000000000001,"15min":1.254},"min":{"1min":0.94299999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5329999999999999,"5min":11.68,"15min":11.68},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.21,"5min":1.2050000000000001,"15min":1.218},"min":{"1min":0.83799999999999997,"5min":0.70099999999999996,"15min":0.67000000000000004},"max":{"1min":1.841,"5min":2.2410000000000001,"15min":3.7050000000000001},"last":1.0109999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:04:18 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2390000000000001,"15min":1.252},"min":{"1min":0.79700000000000004,"5min":0.76800000000000002,"15min":0.755},"max":{"1min":1.556,"5min":2.0350000000000001,"15min":2.2090000000000001},"last":1.276},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.276,"15min":1.2769999999999999},"min":{"1min":0.79000000000000004,"5min":0.75900000000000001,"15min":0.72399999999999998},"max":{"1min":1.7270000000000001,"5min":2.153,"15min":3.8180000000000001},"last":1.3280000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:04:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.327,"15min":1.2669999999999999},"min":{"1min":0.77500000000000002,"5min":0.68799999999999994,"15min":0.65400000000000003},"max":{"1min":1.9810000000000001,"5min":11.797000000000001,"15min":11.797000000000001},"last":1.212},{"interface":"front","average":{"1min":1.377,"5min":1.379,"15min":1.3500000000000001},"min":{"1min":1.0089999999999999,"5min":0.747,"15min":0.747},"max":{"1min":1.7410000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.476}]},{"osd":4,"last update":"Wed Oct 18 09:04:18 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.361,"15min":1.3999999999999999},"min":{"1min":0.92600000000000005,"5min":0.89900000000000002,"15min":0.75900000000000001},"max":{"1min":1.6459999999999999,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.415},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.2609999999999999,"15min":1.417},"min":{"1min":0.96599999999999997,"5min":0.83699999999999997,"15min":0.64100000000000001},"max":{"1min":2.1360000000000001,"5min":2.4049999999999998,"15min":5.4580000000000002},"last":1.55}]},{"osd":6,"last update":"Wed Oct 18 09:04:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.266,"15min":1.343},"min":{"1min":0.94399999999999995,"5min":0.80300000000000005,"15min":0.67400000000000004},"max":{"1min":2.1030000000000002,"5min":2.5190000000000001,"15min":5.141},"last":1.371},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5209999999999999,"15min":1.5509999999999999},"min":{"1min":1.3,"5min":0.82499999999999996,"15min":0.71099999999999997},"max":{"1min":1.9059999999999999,"5min":2.657,"15min":2.657},"last":1.597}]},{"osd":7,"last update":"Wed Oct 18 09:04:18 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.5680000000000001,"15min":1.6120000000000001},"min":{"1min":1.0529999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.444,"5min":11.829000000000001,"15min":11.829000000000001},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.405,"15min":1.4750000000000001},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.73399999999999999},"max":{"1min":2.6419999999999999,"5min":2.9420000000000002,"15min":4.8419999999999996},"last":1.5}]}]},{"osd":4,"up_from":13,"seq":55834574965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":892,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194419,"statfs":{"total":200000000,"available":199085651,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.157,"15min":1.121},"min":{"1min":0.76800000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.8100000000000001,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.1579999999999999,"15min":1.153},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6339999999999999,"5min":3.234,"15min":3.234},"last":1.0369999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.169,"15min":1.117},"min":{"1min":0.73699999999999999,"5min":0.64700000000000002,"15min":0.60799999999999998},"max":{"1min":1.5840000000000001,"5min":3.718,"15min":3.718},"last":1.202},{"interface":"front","average":{"1min":1.181,"5min":1.216,"15min":1.2190000000000001},"min":{"1min":0.76300000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5529999999999999,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":0.90400000000000003}]},{"osd":2,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.216,"15min":1.1950000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.56699999999999995},"max":{"1min":1.589,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.347,"15min":1.3080000000000001},"min":{"1min":0.90700000000000003,"5min":0.79500000000000004,"15min":0.58899999999999997},"max":{"1min":1.704,"5min":3.532,"15min":3.532},"last":1.1080000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.2749999999999999,"15min":1.2829999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.67000000000000004},"max":{"1min":1.641,"5min":3.363,"15min":3.363},"last":1.462},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3979999999999999,"15min":1.407},"min":{"1min":1.0249999999999999,"5min":0.84999999999999998,"15min":0.68899999999999995},"max":{"1min":1.675,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":1.375}]},{"osd":5,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.304,"15min":1.3029999999999999},"min":{"1min":1.046,"5min":0.64100000000000001,"15min":0.498},"max":{"1min":1.712,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":1.246},{"interface":"front","average":{"1min":1.361,"5min":1.323,"15min":1.3400000000000001},"min":{"1min":1.097,"5min":0.70699999999999996,"15min":0.38400000000000001},"max":{"1min":1.5800000000000001,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":1.3160000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.4019999999999999,"15min":1.403},"min":{"1min":1.1779999999999999,"5min":0.78300000000000003,"15min":0.51600000000000001},"max":{"1min":1.873,"5min":4.4429999999999996,"15min":5.0049999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.476,"15min":1.5309999999999999},"min":{"1min":1.375,"5min":0.80100000000000005,"15min":0.45400000000000001},"max":{"1min":1.871,"5min":4.1239999999999997,"15min":4.407},"last":1.411}]},{"osd":7,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.452,"15min":1.4790000000000001},"min":{"1min":1.149,"5min":0.85599999999999998,"15min":0.438},"max":{"1min":1.804,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.595,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":1.1160000000000001,"5min":0.92000000000000004,"15min":0.747},"max":{"1min":1.966,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":1.5189999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":894,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194418,"statfs":{"total":200000000,"available":199084542,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:04:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.93300000000000005,"15min":0.85399999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.827,"5min":4.774,"15min":5.5730000000000004},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.94299999999999995,"15min":0.92000000000000004},"min":{"1min":0.40899999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.5880000000000001,"5min":3.2370000000000001,"15min":6.5599999999999996},"last":0.95199999999999996}]},{"osd":1,"last update":"Wed Oct 18 09:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.004,"15min":0.97799999999999998},"min":{"1min":0.73299999999999998,"5min":0.39100000000000001,"15min":0.36799999999999999},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":6.8099999999999996},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.2509999999999999,"15min":1.1799999999999999},"min":{"1min":0.80500000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.702,"5min":1.8080000000000001,"15min":3.1920000000000002},"last":1.7949999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.121,"15min":1.0600000000000001},"min":{"1min":0.58999999999999997,"5min":0.46800000000000003,"15min":0.45300000000000001},"max":{"1min":1.647,"5min":2.6269999999999998,"15min":7.4580000000000002},"last":1.9550000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.206,"15min":1.125},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.51400000000000001},"max":{"1min":1.52,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.3109999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.3200000000000001,"15min":1.347},"min":{"1min":0.65400000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.635,"5min":3.1139999999999999,"15min":18.664999999999999},"last":1.633},{"interface":"front","average":{"1min":1.335,"5min":1.306,"15min":1.2250000000000001},"min":{"1min":0.91300000000000003,"5min":0.877,"15min":0.64000000000000001},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":7.0700000000000003},"last":2.0289999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.327,"15min":1.2749999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.8540000000000001,"5min":1.998,"15min":6.9139999999999997},"last":1.8560000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4750000000000001,"15min":1.421},"min":{"1min":0.72999999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.855,"5min":4.5540000000000003,"15min":16.390999999999998},"last":2.6499999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.478,"15min":1.4570000000000001},"min":{"1min":0.98899999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.1040000000000001,"5min":3.375,"15min":7.7270000000000003},"last":2.0870000000000002},{"interface":"front","average":{"1min":1.498,"5min":1.4950000000000001,"15min":1.409},"min":{"1min":1.131,"5min":0.872,"15min":0.60199999999999998},"max":{"1min":2.351,"5min":2.4860000000000002,"15min":7.242},"last":1.538}]},{"osd":7,"last update":"Wed Oct 18 09:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.5780000000000001,"15min":1.4399999999999999},"min":{"1min":0.90200000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.96,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":2.8170000000000002},{"interface":"front","average":{"1min":1.522,"5min":1.556,"15min":1.5109999999999999},"min":{"1min":0.82899999999999996,"5min":0.622,"15min":0.622},"max":{"1min":1.915,"5min":5.4900000000000002,"15min":12.946},"last":2.4590000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":888,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194424,"statfs":{"total":200000000,"available":199090386,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.84299999999999997,"15min":0.90100000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.095,"5min":1.5660000000000001,"15min":4.2210000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.90000000000000002,"5min":1.0349999999999999,"15min":1.028},"min":{"1min":0.40200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.514,"5min":4.3959999999999999,"15min":4.3959999999999999},"last":0.59699999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1359999999999999,"15min":1.077},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.49099999999999999},"max":{"1min":1.6200000000000001,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":1.238},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.1419999999999999,"15min":1.1080000000000001},"min":{"1min":0.57299999999999995,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":1.9510000000000001,"5min":2.448,"15min":2.5350000000000001},"last":0.68100000000000005}]},{"osd":3,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.173,"15min":1.131},"min":{"1min":0.57899999999999996,"5min":0.51200000000000001,"15min":0.48199999999999998},"max":{"1min":1.6379999999999999,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.232,"5min":1.3180000000000001,"15min":1.407},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.54100000000000004},"max":{"1min":1.6100000000000001,"5min":2.2829999999999999,"15min":7.5570000000000004},"last":1.339}]},{"osd":4,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.417,"15min":1.4590000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.8680000000000001,"5min":2.4119999999999999,"15min":7.6189999999999998},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.4650000000000001,"15min":1.446},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.86599999999999999},"max":{"1min":1.8959999999999999,"5min":3.423,"15min":3.423},"last":1.0289999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.4019999999999999,"15min":1.306},"min":{"1min":0.874,"5min":0.754,"15min":0.61599999999999999},"max":{"1min":1.8049999999999999,"5min":10.507,"15min":10.507},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.46,"15min":1.5169999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81399999999999995},"max":{"1min":1.9319999999999999,"5min":2.536,"15min":7.657},"last":1.4570000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.482,"15min":1.4970000000000001},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.74299999999999999},"max":{"1min":1.891,"5min":2.8780000000000001,"15min":3.5299999999999998},"last":1.085},{"interface":"front","average":{"1min":1.379,"5min":1.5449999999999999,"15min":1.607},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.77600000000000002},"max":{"1min":1.895,"5min":10.561,"15min":37.771999999999998},"last":0.997}]},{"osd":7,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.5740000000000001,"15min":1.5589999999999999},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.83199999999999996},"max":{"1min":1.732,"5min":2.7890000000000001,"15min":3.3620000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.607,"15min":1.5129999999999999},"min":{"1min":0.98299999999999998,"5min":0.90200000000000002,"15min":0.77600000000000002},"max":{"1min":1.831,"5min":10.956,"15min":10.956},"last":1.0589999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":886,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194425,"statfs":{"total":200000000,"available":199092021,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.94999999999999996,"15min":0.85499999999999998},"min":{"1min":0.37,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":1.6519999999999999,"5min":1.659,"15min":1.659},"last":0.378},{"interface":"front","average":{"1min":1.147,"5min":1.1060000000000001,"15min":0.97499999999999998},"min":{"1min":0.72799999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.681,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.63}]},{"osd":2,"last update":"Wed Oct 18 09:04:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.046,"15min":0.96999999999999997},"min":{"1min":0.443,"5min":0.317,"15min":0.317},"max":{"1min":1.4430000000000001,"5min":1.6899999999999999,"15min":1.859},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.997,"5min":0.98899999999999999,"15min":0.89800000000000002},"min":{"1min":0.58099999999999996,"5min":0.38600000000000001,"15min":0.36699999999999999},"max":{"1min":1.3660000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.60299999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:04:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.123,"15min":1.0509999999999999},"min":{"1min":0.63300000000000001,"5min":0.46899999999999997,"15min":0.436},"max":{"1min":1.49,"5min":1.915,"15min":2.1059999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.0920000000000001,"15min":1.0649999999999999},"min":{"1min":0.65700000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.736,"5min":1.9370000000000001,"15min":3.7010000000000001},"last":0.56399999999999995}]},{"osd":4,"last update":"Wed Oct 18 09:04:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.331,"15min":1.2749999999999999},"min":{"1min":0.88200000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7070000000000001,"5min":1.909,"15min":2.0489999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3680000000000001,"15min":1.266},"min":{"1min":1.048,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.609,"5min":1.893,"15min":2.0470000000000002},"last":0.83499999999999996}]},{"osd":5,"last update":"Wed Oct 18 09:04:25 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.4299999999999999,"15min":1.3380000000000001},"min":{"1min":1.0009999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.857,"5min":2.161,"15min":3.5750000000000002},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.492,"15min":1.403},"min":{"1min":0.76500000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.3769999999999998},"last":0.86399999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:04:25 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.4490000000000001,"15min":1.3380000000000001},"min":{"1min":0.80000000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.9890000000000001,"5min":2.0419999999999998,"15min":2.2050000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.407,"5min":1.4279999999999999,"15min":1.3260000000000001},"min":{"1min":1.0009999999999999,"5min":0.73399999999999999,"15min":0.63100000000000001},"max":{"1min":1.7709999999999999,"5min":2.1040000000000001,"15min":2.121},"last":1.0589999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:04:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5,"15min":1.4079999999999999},"min":{"1min":0.875,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.893,"5min":2.2000000000000002,"15min":2.2229999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.474,"5min":1.5149999999999999,"15min":1.4119999999999999},"min":{"1min":0.93799999999999994,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.1269999999999998,"5min":2.2469999999999999,"15min":2.4180000000000001},"last":0.81100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1463,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193849,"statfs":{"total":200000000,"available":198501653,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.83499999999999996,"15min":0.88600000000000001},"min":{"1min":0.45600000000000002,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":1.2430000000000001,"5min":1.3320000000000001,"15min":3.4420000000000002},"last":0.30599999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.1180000000000001,"15min":1.087},"min":{"1min":0.61699999999999999,"5min":0.56200000000000006,"15min":0.46500000000000002},"max":{"1min":1.6950000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.46300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.482,"5min":1.679,"15min":1.679},"last":0.61599999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.141,"15min":1.127},"min":{"1min":0.67800000000000005,"5min":0.503,"15min":0.437},"max":{"1min":1.5629999999999999,"5min":1.929,"15min":1.929},"last":0.754}]},{"osd":3,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.129,"15min":1.22},"min":{"1min":0.622,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.857,"5min":1.98,"15min":2.7639999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.391,"15min":1.46},"min":{"1min":0.79500000000000004,"5min":0.61299999999999999,"15min":0.58399999999999996},"max":{"1min":2.1200000000000001,"5min":2.3639999999999999,"15min":3.5720000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2350000000000001,"15min":1.2689999999999999},"min":{"1min":0.876,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":4.492},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.292,"5min":1.2709999999999999,"15min":1.262},"min":{"1min":0.873,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.716,"5min":1.927,"15min":1.927},"last":0.82999999999999996}]},{"osd":5,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.3029999999999999,"15min":1.367},"min":{"1min":0.875,"5min":0.79100000000000004,"15min":0.64300000000000002},"max":{"1min":2.1779999999999999,"5min":2.4510000000000001,"15min":4.2779999999999996},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.323,"15min":1.3520000000000001},"min":{"1min":0.84699999999999998,"5min":0.74399999999999999,"15min":0.73099999999999998},"max":{"1min":1.8089999999999999,"5min":1.847,"15min":1.8740000000000001},"last":0.875}]},{"osd":6,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4219999999999999,"15min":1.524},"min":{"1min":0.93700000000000006,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":3.5299999999999998},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.327,"5min":1.3600000000000001,"15min":1.444},"min":{"1min":1.0289999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.7909999999999999,"5min":3.1440000000000001,"15min":4.4349999999999996},"last":1.032}]},{"osd":7,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.542,"15min":1.601},"min":{"1min":1.0289999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.2629999999999999,"5min":2.2629999999999999,"15min":3.492},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.4570000000000001,"15min":1.591},"min":{"1min":0.99299999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.3090000000000002,"5min":3.2570000000000001,"15min":4.5419999999999998},"last":0.98699999999999999}]}]}],"pool_statfs":[]}} 2023-10-18T09:04:41.996 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:04:42.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.263+0000 7fd9151e9640 1 -- 172.21.15.40:0/3418452812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9100b15b0 msgr2=0x7fd9100b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:42.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.263+0000 7fd9151e9640 1 --2- 172.21.15.40:0/3418452812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9100b15b0 0x7fd9100b0630 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd900005fd0 tx=0x7fd90002be20 comp rx=0 tx=0).stop 2023-10-18T09:04:42.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.263+0000 7fd9151e9640 1 -- 172.21.15.40:0/3418452812 shutdown_connections 2023-10-18T09:04:42.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.263+0000 7fd9151e9640 1 --2- 172.21.15.40:0/3418452812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9100ca0d0 0x7fd910157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:42.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.263+0000 7fd9151e9640 1 --2- 172.21.15.40:0/3418452812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9100b15b0 0x7fd9100b0630 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:42.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.263+0000 7fd9151e9640 1 -- 172.21.15.40:0/3418452812 >> 172.21.15.40:0/3418452812 conn(0x7fd910069720 msgr2=0x7fd910069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:42.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.263+0000 7fd9151e9640 1 -- 172.21.15.40:0/3418452812 shutdown_connections 2023-10-18T09:04:42.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.263+0000 7fd9151e9640 1 -- 172.21.15.40:0/3418452812 wait complete. 2023-10-18T09:04:42.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.265+0000 7fd9151e9640 1 Processor -- start 2023-10-18T09:04:42.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.265+0000 7fd9151e9640 1 -- start start 2023-10-18T09:04:42.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.266+0000 7fd9151e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9100b15b0 0x7fd91014a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:42.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.266+0000 7fd9151e9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9100b1e10 0x7fd9101ef680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:42.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.266+0000 7fd9151e9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9100ca0d0 0x7fd9101efbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:42.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.266+0000 7fd9151e9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91014ad20 con 0x7fd9100b15b0 2023-10-18T09:04:42.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.266+0000 7fd9151e9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9101f0210 con 0x7fd9100b1e10 2023-10-18T09:04:42.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.266+0000 7fd9151e9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9101f04f0 con 0x7fd9100ca0d0 2023-10-18T09:04:42.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.266+0000 7fd90ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9100b15b0 0x7fd91014a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:42.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.267+0000 7fd90ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9100b15b0 0x7fd91014a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43566/0 (socket says 172.21.15.40:43566) 2023-10-18T09:04:42.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.267+0000 7fd90ffff640 1 -- 172.21.15.40:0/2427816367 learned_addr learned my addr 172.21.15.40:0/2427816367 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:42.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.267+0000 7fd90f7fe640 1 --2- 172.21.15.40:0/2427816367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9100b1e10 0x7fd9101ef680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:42.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.267+0000 7fd9149e8640 1 --2- 172.21.15.40:0/2427816367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9100ca0d0 0x7fd9101efbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:42.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.267+0000 7fd90ffff640 1 -- 172.21.15.40:0/2427816367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9100ca0d0 msgr2=0x7fd9101efbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:42.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.267+0000 7fd90ffff640 1 --2- 172.21.15.40:0/2427816367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9100ca0d0 0x7fd9101efbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:42.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.267+0000 7fd90ffff640 1 -- 172.21.15.40:0/2427816367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9100b1e10 msgr2=0x7fd9101ef680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.267+0000 7fd90ffff640 1 --2- 172.21.15.40:0/2427816367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9100b1e10 0x7fd9101ef680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.267+0000 7fd90ffff640 1 -- 172.21.15.40:0/2427816367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd900009560 con 0x7fd9100b15b0 2023-10-18T09:04:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.267+0000 7fd90f7fe640 1 --2- 172.21.15.40:0/2427816367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9100b1e10 0x7fd9101ef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.267+0000 7fd90ffff640 1 --2- 172.21.15.40:0/2427816367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9100b15b0 0x7fd91014a7e0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fd900009850 tx=0x7fd90002daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:42.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.268+0000 7fd90cff9640 1 -- 172.21.15.40:0/2427816367 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9000035f0 con 0x7fd9100b15b0 2023-10-18T09:04:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.268+0000 7fd90cff9640 1 -- 172.21.15.40:0/2427816367 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd900003840 con 0x7fd9100b15b0 2023-10-18T09:04:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.268+0000 7fd90cff9640 1 -- 172.21.15.40:0/2427816367 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900003bf0 con 0x7fd9100b15b0 2023-10-18T09:04:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.268+0000 7fd9151e9640 1 -- 172.21.15.40:0/2427816367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9101f07d0 con 0x7fd9100b15b0 2023-10-18T09:04:42.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.268+0000 7fd9151e9640 1 -- 172.21.15.40:0/2427816367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9101f0d90 con 0x7fd9100b15b0 2023-10-18T09:04:42.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.269+0000 7fd9151e9640 1 -- 172.21.15.40:0/2427816367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd910066e00 con 0x7fd9100b15b0 2023-10-18T09:04:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.270+0000 7fd90cff9640 1 -- 172.21.15.40:0/2427816367 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd900003d50 con 0x7fd9100b15b0 2023-10-18T09:04:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.271+0000 7fd90cff9640 1 --2- 172.21.15.40:0/2427816367 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd8fc06ae70 0x7fd8fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.271+0000 7fd90cff9640 1 -- 172.21.15.40:0/2427816367 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9000ac1e0 con 0x7fd9100b15b0 2023-10-18T09:04:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.272+0000 7fd90f7fe640 1 --2- 172.21.15.40:0/2427816367 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd8fc06ae70 0x7fd8fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.273+0000 7fd90f7fe640 1 --2- 172.21.15.40:0/2427816367 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd8fc06ae70 0x7fd8fc06d330 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd9100b00d0 tx=0x7fd904009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.276+0000 7fd90cff9640 1 -- 172.21.15.40:0/2427816367 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9000773b0 con 0x7fd9100b15b0 2023-10-18T09:04:42.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.464+0000 7fd9151e9640 1 -- 172.21.15.40:0/2427816367 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd91015bed0 con 0x7fd8fc06ae70 2023-10-18T09:04:42.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.487+0000 7fd90cff9640 1 -- 172.21.15.40:0/2427816367 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282632 (secure 0 0 0) 0x7fd91015bed0 con 0x7fd8fc06ae70 2023-10-18T09:04:42.495 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:42.497 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:04:42.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.494+0000 7fd9151e9640 1 -- 172.21.15.40:0/2427816367 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd8fc06ae70 msgr2=0x7fd8fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:42.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.494+0000 7fd9151e9640 1 --2- 172.21.15.40:0/2427816367 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd8fc06ae70 0x7fd8fc06d330 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd9100b00d0 tx=0x7fd904009040 comp rx=0 tx=0).stop 2023-10-18T09:04:42.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.494+0000 7fd9151e9640 1 -- 172.21.15.40:0/2427816367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9100b15b0 msgr2=0x7fd91014a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:42.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.494+0000 7fd9151e9640 1 --2- 172.21.15.40:0/2427816367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9100b15b0 0x7fd91014a7e0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fd900009850 tx=0x7fd90002daf0 comp rx=0 tx=0).stop 2023-10-18T09:04:42.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.494+0000 7fd9151e9640 1 -- 172.21.15.40:0/2427816367 shutdown_connections 2023-10-18T09:04:42.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.494+0000 7fd9151e9640 1 --2- 172.21.15.40:0/2427816367 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd8fc06ae70 0x7fd8fc06d330 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:42.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.494+0000 7fd9151e9640 1 --2- 172.21.15.40:0/2427816367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9100ca0d0 0x7fd9101efbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:42.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.494+0000 7fd9151e9640 1 --2- 172.21.15.40:0/2427816367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9100b1e10 0x7fd9101ef680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:42.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.494+0000 7fd9151e9640 1 --2- 172.21.15.40:0/2427816367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9100b15b0 0x7fd91014a7e0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:42.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.494+0000 7fd9151e9640 1 -- 172.21.15.40:0/2427816367 >> 172.21.15.40:0/2427816367 conn(0x7fd910069720 msgr2=0x7fd9101594d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:42.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.495+0000 7fd9151e9640 1 -- 172.21.15.40:0/2427816367 shutdown_connections 2023-10-18T09:04:42.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.495+0000 7fd9151e9640 1 -- 172.21.15.40:0/2427816367 wait complete. 2023-10-18T09:04:42.517 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2023-10-18T09:04:41.103750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8274,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554225,"statfs":{"total":1600000000,"available":1591526617,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.659809"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.490895+0000","last_change":"2023-10-18T09:04:37.490831+0000","last_active":"2023-10-18T09:04:37.490895+0000","last_peered":"2023-10-18T09:04:37.490895+0000","last_clean":"2023-10-18T09:04:37.490895+0000","last_became_active":"2023-10-18T09:04:37.490510+0000","last_became_peered":"2023-10-18T09:04:37.490510+0000","last_unstale":"2023-10-18T09:04:37.490895+0000","last_undegraded":"2023-10-18T09:04:37.490895+0000","last_fullsized":"2023-10-18T09:04:37.490895+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:18.541619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.491808+0000","last_change":"2023-10-18T09:04:37.491742+0000","last_active":"2023-10-18T09:04:37.491808+0000","last_peered":"2023-10-18T09:04:37.491808+0000","last_clean":"2023-10-18T09:04:37.491808+0000","last_became_active":"2023-10-18T09:04:37.491527+0000","last_became_peered":"2023-10-18T09:04:37.491527+0000","last_unstale":"2023-10-18T09:04:37.491808+0000","last_undegraded":"2023-10-18T09:04:37.491808+0000","last_fullsized":"2023-10-18T09:04:37.491808+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:09.081005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.500346+0000","last_change":"2023-10-18T09:04:37.500271+0000","last_active":"2023-10-18T09:04:37.500346+0000","last_peered":"2023-10-18T09:04:37.500346+0000","last_clean":"2023-10-18T09:04:37.500346+0000","last_became_active":"2023-10-18T09:04:37.499853+0000","last_became_peered":"2023-10-18T09:04:37.499853+0000","last_unstale":"2023-10-18T09:04:37.500346+0000","last_undegraded":"2023-10-18T09:04:37.500346+0000","last_fullsized":"2023-10-18T09:04:37.500346+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:02:17.069751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.482658+0000","last_change":"2023-10-18T09:04:37.482251+0000","last_active":"2023-10-18T09:04:37.482658+0000","last_peered":"2023-10-18T09:04:37.482658+0000","last_clean":"2023-10-18T09:04:37.482658+0000","last_became_active":"2023-10-18T09:04:37.482031+0000","last_became_peered":"2023-10-18T09:04:37.482031+0000","last_unstale":"2023-10-18T09:04:37.482658+0000","last_undegraded":"2023-10-18T09:04:37.482658+0000","last_fullsized":"2023-10-18T09:04:37.482658+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:31:08.551966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.468253+0000","last_change":"2023-10-18T09:04:37.491538+0000","last_active":"2023-10-18T09:04:38.468253+0000","last_peered":"2023-10-18T09:04:38.468253+0000","last_clean":"2023-10-18T09:04:38.468253+0000","last_became_active":"2023-10-18T09:04:37.491305+0000","last_became_peered":"2023-10-18T09:04:37.491305+0000","last_unstale":"2023-10-18T09:04:38.468253+0000","last_undegraded":"2023-10-18T09:04:38.468253+0000","last_fullsized":"2023-10-18T09:04:38.468253+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:16.055093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.469666+0000","last_change":"2023-10-18T09:04:37.489837+0000","last_active":"2023-10-18T09:04:38.469666+0000","last_peered":"2023-10-18T09:04:38.469666+0000","last_clean":"2023-10-18T09:04:38.469666+0000","last_became_active":"2023-10-18T09:04:37.489056+0000","last_became_peered":"2023-10-18T09:04:37.489056+0000","last_unstale":"2023-10-18T09:04:38.469666+0000","last_undegraded":"2023-10-18T09:04:38.469666+0000","last_fullsized":"2023-10-18T09:04:38.469666+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:04:19.707753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.467967+0000","last_change":"2023-10-18T09:04:37.497488+0000","last_active":"2023-10-18T09:04:38.467967+0000","last_peered":"2023-10-18T09:04:38.467967+0000","last_clean":"2023-10-18T09:04:38.467967+0000","last_became_active":"2023-10-18T09:04:37.497031+0000","last_became_peered":"2023-10-18T09:04:37.497031+0000","last_unstale":"2023-10-18T09:04:38.467967+0000","last_undegraded":"2023-10-18T09:04:38.467967+0000","last_fullsized":"2023-10-18T09:04:38.467967+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:50:14.955866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.498157+0000","last_change":"2023-10-18T09:04:37.498089+0000","last_active":"2023-10-18T09:04:37.498157+0000","last_peered":"2023-10-18T09:04:37.498157+0000","last_clean":"2023-10-18T09:04:37.498157+0000","last_became_active":"2023-10-18T09:04:37.496058+0000","last_became_peered":"2023-10-18T09:04:37.496058+0000","last_unstale":"2023-10-18T09:04:37.498157+0000","last_undegraded":"2023-10-18T09:04:37.498157+0000","last_fullsized":"2023-10-18T09:04:37.498157+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:47:38.150973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.495636+0000","last_change":"2023-10-18T09:04:37.495358+0000","last_active":"2023-10-18T09:04:37.495636+0000","last_peered":"2023-10-18T09:04:37.495636+0000","last_clean":"2023-10-18T09:04:37.495636+0000","last_became_active":"2023-10-18T09:04:37.495122+0000","last_became_peered":"2023-10-18T09:04:37.495122+0000","last_unstale":"2023-10-18T09:04:37.495636+0000","last_undegraded":"2023-10-18T09:04:37.495636+0000","last_fullsized":"2023-10-18T09:04:37.495636+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:14:42.919064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.487957+0000","last_change":"2023-10-18T09:04:37.487883+0000","last_active":"2023-10-18T09:04:37.487957+0000","last_peered":"2023-10-18T09:04:37.487957+0000","last_clean":"2023-10-18T09:04:37.487957+0000","last_became_active":"2023-10-18T09:04:37.487614+0000","last_became_peered":"2023-10-18T09:04:37.487614+0000","last_unstale":"2023-10-18T09:04:37.487957+0000","last_undegraded":"2023-10-18T09:04:37.487957+0000","last_fullsized":"2023-10-18T09:04:37.487957+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:26.364922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.489021+0000","last_change":"2023-10-18T09:04:37.488428+0000","last_active":"2023-10-18T09:04:37.489021+0000","last_peered":"2023-10-18T09:04:37.489021+0000","last_clean":"2023-10-18T09:04:37.489021+0000","last_became_active":"2023-10-18T09:04:37.488192+0000","last_became_peered":"2023-10-18T09:04:37.488192+0000","last_unstale":"2023-10-18T09:04:37.489021+0000","last_undegraded":"2023-10-18T09:04:37.489021+0000","last_fullsized":"2023-10-18T09:04:37.489021+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:09.554842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.499822+0000","last_change":"2023-10-18T09:04:37.499716+0000","last_active":"2023-10-18T09:04:37.499822+0000","last_peered":"2023-10-18T09:04:37.499822+0000","last_clean":"2023-10-18T09:04:37.499822+0000","last_became_active":"2023-10-18T09:04:37.498099+0000","last_became_peered":"2023-10-18T09:04:37.498099+0000","last_unstale":"2023-10-18T09:04:37.499822+0000","last_undegraded":"2023-10-18T09:04:37.499822+0000","last_fullsized":"2023-10-18T09:04:37.499822+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:44:29.338172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.481050+0000","last_change":"2023-10-18T09:04:37.479677+0000","last_active":"2023-10-18T09:04:37.481050+0000","last_peered":"2023-10-18T09:04:37.481050+0000","last_clean":"2023-10-18T09:04:37.481050+0000","last_became_active":"2023-10-18T09:04:37.479264+0000","last_became_peered":"2023-10-18T09:04:37.479264+0000","last_unstale":"2023-10-18T09:04:37.481050+0000","last_undegraded":"2023-10-18T09:04:37.481050+0000","last_fullsized":"2023-10-18T09:04:37.481050+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:50:29.089915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.482468+0000","last_change":"2023-10-18T09:04:37.482046+0000","last_active":"2023-10-18T09:04:37.482468+0000","last_peered":"2023-10-18T09:04:37.482468+0000","last_clean":"2023-10-18T09:04:37.482468+0000","last_became_active":"2023-10-18T09:04:37.481814+0000","last_became_peered":"2023-10-18T09:04:37.481814+0000","last_unstale":"2023-10-18T09:04:37.482468+0000","last_undegraded":"2023-10-18T09:04:37.482468+0000","last_fullsized":"2023-10-18T09:04:37.482468+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:10:13.433915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.499838+0000","last_change":"2023-10-18T09:04:37.499786+0000","last_active":"2023-10-18T09:04:37.499838+0000","last_peered":"2023-10-18T09:04:37.499838+0000","last_clean":"2023-10-18T09:04:37.499838+0000","last_became_active":"2023-10-18T09:04:37.499337+0000","last_became_peered":"2023-10-18T09:04:37.499337+0000","last_unstale":"2023-10-18T09:04:37.499838+0000","last_undegraded":"2023-10-18T09:04:37.499838+0000","last_fullsized":"2023-10-18T09:04:37.499838+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:43:24.543978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464804+0000","last_change":"2023-10-18T09:04:37.482627+0000","last_active":"2023-10-18T09:04:38.464804+0000","last_peered":"2023-10-18T09:04:38.464804+0000","last_clean":"2023-10-18T09:04:38.464804+0000","last_became_active":"2023-10-18T09:04:37.481814+0000","last_became_peered":"2023-10-18T09:04:37.481814+0000","last_unstale":"2023-10-18T09:04:38.464804+0000","last_undegraded":"2023-10-18T09:04:38.464804+0000","last_fullsized":"2023-10-18T09:04:38.464804+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:36:31.938682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.468626+0000","last_change":"2023-10-18T09:04:37.492664+0000","last_active":"2023-10-18T09:04:38.468626+0000","last_peered":"2023-10-18T09:04:38.468626+0000","last_clean":"2023-10-18T09:04:38.468626+0000","last_became_active":"2023-10-18T09:04:37.491870+0000","last_became_peered":"2023-10-18T09:04:37.491870+0000","last_unstale":"2023-10-18T09:04:38.468626+0000","last_undegraded":"2023-10-18T09:04:38.468626+0000","last_fullsized":"2023-10-18T09:04:38.468626+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:21:23.553384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464230+0000","last_change":"2023-10-18T09:04:37.475818+0000","last_active":"2023-10-18T09:04:38.464230+0000","last_peered":"2023-10-18T09:04:38.464230+0000","last_clean":"2023-10-18T09:04:38.464230+0000","last_became_active":"2023-10-18T09:04:37.475037+0000","last_became_peered":"2023-10-18T09:04:37.475037+0000","last_unstale":"2023-10-18T09:04:38.464230+0000","last_undegraded":"2023-10-18T09:04:38.464230+0000","last_fullsized":"2023-10-18T09:04:38.464230+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:01:41.587901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.497143+0000","last_change":"2023-10-18T09:04:37.496952+0000","last_active":"2023-10-18T09:04:37.497143+0000","last_peered":"2023-10-18T09:04:37.497143+0000","last_clean":"2023-10-18T09:04:37.497143+0000","last_became_active":"2023-10-18T09:04:37.496762+0000","last_became_peered":"2023-10-18T09:04:37.496762+0000","last_unstale":"2023-10-18T09:04:37.497143+0000","last_undegraded":"2023-10-18T09:04:37.497143+0000","last_fullsized":"2023-10-18T09:04:37.497143+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:05:52.916098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464214+0000","last_change":"2023-10-18T09:04:37.476148+0000","last_active":"2023-10-18T09:04:38.464214+0000","last_peered":"2023-10-18T09:04:38.464214+0000","last_clean":"2023-10-18T09:04:38.464214+0000","last_became_active":"2023-10-18T09:04:37.475900+0000","last_became_peered":"2023-10-18T09:04:37.475900+0000","last_unstale":"2023-10-18T09:04:38.464214+0000","last_undegraded":"2023-10-18T09:04:38.464214+0000","last_fullsized":"2023-10-18T09:04:38.464214+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:49:21.451994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.469080+0000","last_change":"2023-10-18T09:04:37.494887+0000","last_active":"2023-10-18T09:04:38.469080+0000","last_peered":"2023-10-18T09:04:38.469080+0000","last_clean":"2023-10-18T09:04:38.469080+0000","last_became_active":"2023-10-18T09:04:37.491960+0000","last_became_peered":"2023-10-18T09:04:37.491960+0000","last_unstale":"2023-10-18T09:04:38.469080+0000","last_undegraded":"2023-10-18T09:04:38.469080+0000","last_fullsized":"2023-10-18T09:04:38.469080+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:34.585957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.486268+0000","last_change":"2023-10-18T09:04:37.485697+0000","last_active":"2023-10-18T09:04:37.486268+0000","last_peered":"2023-10-18T09:04:37.486268+0000","last_clean":"2023-10-18T09:04:37.486268+0000","last_became_active":"2023-10-18T09:04:37.485367+0000","last_became_peered":"2023-10-18T09:04:37.485367+0000","last_unstale":"2023-10-18T09:04:37.486268+0000","last_undegraded":"2023-10-18T09:04:37.486268+0000","last_fullsized":"2023-10-18T09:04:37.486268+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:55:18.748505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.499361+0000","last_change":"2023-10-18T09:04:37.499141+0000","last_active":"2023-10-18T09:04:37.499361+0000","last_peered":"2023-10-18T09:04:37.499361+0000","last_clean":"2023-10-18T09:04:37.499361+0000","last_became_active":"2023-10-18T09:04:37.496770+0000","last_became_peered":"2023-10-18T09:04:37.496770+0000","last_unstale":"2023-10-18T09:04:37.499361+0000","last_undegraded":"2023-10-18T09:04:37.499361+0000","last_fullsized":"2023-10-18T09:04:37.499361+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:05:39.746894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.486286+0000","last_change":"2023-10-18T09:04:37.485905+0000","last_active":"2023-10-18T09:04:37.486286+0000","last_peered":"2023-10-18T09:04:37.486286+0000","last_clean":"2023-10-18T09:04:37.486286+0000","last_became_active":"2023-10-18T09:04:37.485669+0000","last_became_peered":"2023-10-18T09:04:37.485669+0000","last_unstale":"2023-10-18T09:04:37.486286+0000","last_undegraded":"2023-10-18T09:04:37.486286+0000","last_fullsized":"2023-10-18T09:04:37.486286+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:02:32.105013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.482433+0000","last_change":"2023-10-18T09:04:37.479545+0000","last_active":"2023-10-18T09:04:37.482433+0000","last_peered":"2023-10-18T09:04:37.482433+0000","last_clean":"2023-10-18T09:04:37.482433+0000","last_became_active":"2023-10-18T09:04:37.479139+0000","last_became_peered":"2023-10-18T09:04:37.479139+0000","last_unstale":"2023-10-18T09:04:37.482433+0000","last_undegraded":"2023-10-18T09:04:37.482433+0000","last_fullsized":"2023-10-18T09:04:37.482433+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:24:13.080777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464960+0000","last_change":"2023-10-18T09:04:37.482431+0000","last_active":"2023-10-18T09:04:38.464960+0000","last_peered":"2023-10-18T09:04:38.464960+0000","last_clean":"2023-10-18T09:04:38.464960+0000","last_became_active":"2023-10-18T09:04:37.481153+0000","last_became_peered":"2023-10-18T09:04:37.481153+0000","last_unstale":"2023-10-18T09:04:38.464960+0000","last_undegraded":"2023-10-18T09:04:38.464960+0000","last_fullsized":"2023-10-18T09:04:38.464960+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:40:44.793477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.496430+0000","last_change":"2023-10-18T09:04:37.495936+0000","last_active":"2023-10-18T09:04:37.496430+0000","last_peered":"2023-10-18T09:04:37.496430+0000","last_clean":"2023-10-18T09:04:37.496430+0000","last_became_active":"2023-10-18T09:04:37.495701+0000","last_became_peered":"2023-10-18T09:04:37.495701+0000","last_unstale":"2023-10-18T09:04:37.496430+0000","last_undegraded":"2023-10-18T09:04:37.496430+0000","last_fullsized":"2023-10-18T09:04:37.496430+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:08:39.927171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.484429+0000","last_change":"2023-10-18T09:04:37.483427+0000","last_active":"2023-10-18T09:04:37.484429+0000","last_peered":"2023-10-18T09:04:37.484429+0000","last_clean":"2023-10-18T09:04:37.484429+0000","last_became_active":"2023-10-18T09:04:37.483010+0000","last_became_peered":"2023-10-18T09:04:37.483010+0000","last_unstale":"2023-10-18T09:04:37.484429+0000","last_undegraded":"2023-10-18T09:04:37.484429+0000","last_fullsized":"2023-10-18T09:04:37.484429+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:24:27.643288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.496449+0000","last_change":"2023-10-18T09:04:37.496057+0000","last_active":"2023-10-18T09:04:37.496449+0000","last_peered":"2023-10-18T09:04:37.496449+0000","last_clean":"2023-10-18T09:04:37.496449+0000","last_became_active":"2023-10-18T09:04:37.495731+0000","last_became_peered":"2023-10-18T09:04:37.495731+0000","last_unstale":"2023-10-18T09:04:37.496449+0000","last_undegraded":"2023-10-18T09:04:37.496449+0000","last_fullsized":"2023-10-18T09:04:37.496449+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:01:56.596979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.499678+0000","last_change":"2023-10-18T09:04:37.499609+0000","last_active":"2023-10-18T09:04:37.499678+0000","last_peered":"2023-10-18T09:04:37.499678+0000","last_clean":"2023-10-18T09:04:37.499678+0000","last_became_active":"2023-10-18T09:04:37.498990+0000","last_became_peered":"2023-10-18T09:04:37.498990+0000","last_unstale":"2023-10-18T09:04:37.499678+0000","last_undegraded":"2023-10-18T09:04:37.499678+0000","last_fullsized":"2023-10-18T09:04:37.499678+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:17:03.155226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.467867+0000","last_change":"2023-10-18T09:04:37.485718+0000","last_active":"2023-10-18T09:04:38.467867+0000","last_peered":"2023-10-18T09:04:38.467867+0000","last_clean":"2023-10-18T09:04:38.467867+0000","last_became_active":"2023-10-18T09:04:37.484473+0000","last_became_peered":"2023-10-18T09:04:37.484473+0000","last_unstale":"2023-10-18T09:04:38.467867+0000","last_undegraded":"2023-10-18T09:04:38.467867+0000","last_fullsized":"2023-10-18T09:04:38.467867+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:19:46.360261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.468289+0000","last_change":"2023-10-18T09:04:37.492800+0000","last_active":"2023-10-18T09:04:38.468289+0000","last_peered":"2023-10-18T09:04:38.468289+0000","last_clean":"2023-10-18T09:04:38.468289+0000","last_became_active":"2023-10-18T09:04:37.492171+0000","last_became_peered":"2023-10-18T09:04:37.492171+0000","last_unstale":"2023-10-18T09:04:38.468289+0000","last_undegraded":"2023-10-18T09:04:38.468289+0000","last_fullsized":"2023-10-18T09:04:38.468289+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:06:39.079565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.463961+0000","last_change":"2023-10-18T09:04:37.480923+0000","last_active":"2023-10-18T09:04:38.463961+0000","last_peered":"2023-10-18T09:04:38.463961+0000","last_clean":"2023-10-18T09:04:38.463961+0000","last_became_active":"2023-10-18T09:04:37.480684+0000","last_became_peered":"2023-10-18T09:04:37.480684+0000","last_unstale":"2023-10-18T09:04:38.463961+0000","last_undegraded":"2023-10-18T09:04:38.463961+0000","last_fullsized":"2023-10-18T09:04:38.463961+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:14:10.648103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"161'4","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.466523+0000","last_change":"2023-10-18T09:04:36.471445+0000","last_active":"2023-10-18T09:04:38.466523+0000","last_peered":"2023-10-18T09:04:38.466523+0000","last_clean":"2023-10-18T09:04:38.466523+0000","last_became_active":"2023-10-18T09:04:36.470511+0000","last_became_peered":"2023-10-18T09:04:36.470511+0000","last_unstale":"2023-10-18T09:04:38.466523+0000","last_undegraded":"2023-10-18T09:04:38.466523+0000","last_fullsized":"2023-10-18T09:04:38.466523+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:43:28.013241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.499016+0000","last_change":"2023-10-18T09:04:37.498948+0000","last_active":"2023-10-18T09:04:37.499016+0000","last_peered":"2023-10-18T09:04:37.499016+0000","last_clean":"2023-10-18T09:04:37.499016+0000","last_became_active":"2023-10-18T09:04:37.496366+0000","last_became_peered":"2023-10-18T09:04:37.496366+0000","last_unstale":"2023-10-18T09:04:37.499016+0000","last_undegraded":"2023-10-18T09:04:37.499016+0000","last_fullsized":"2023-10-18T09:04:37.499016+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:01:14.003255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"161'1","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.491453+0000","last_change":"2023-10-18T09:04:36.478506+0000","last_active":"2023-10-18T09:04:37.491453+0000","last_peered":"2023-10-18T09:04:37.491453+0000","last_clean":"2023-10-18T09:04:37.491453+0000","last_became_active":"2023-10-18T09:04:36.477741+0000","last_became_peered":"2023-10-18T09:04:36.477741+0000","last_unstale":"2023-10-18T09:04:37.491453+0000","last_undegraded":"2023-10-18T09:04:37.491453+0000","last_fullsized":"2023-10-18T09:04:37.491453+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:00.168165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.468195+0000","last_change":"2023-10-18T09:04:37.496186+0000","last_active":"2023-10-18T09:04:38.468195+0000","last_peered":"2023-10-18T09:04:38.468195+0000","last_clean":"2023-10-18T09:04:38.468195+0000","last_became_active":"2023-10-18T09:04:37.495917+0000","last_became_peered":"2023-10-18T09:04:37.495917+0000","last_unstale":"2023-10-18T09:04:38.468195+0000","last_undegraded":"2023-10-18T09:04:38.468195+0000","last_fullsized":"2023-10-18T09:04:38.468195+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:08:25.262401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"161'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.488786+0000","last_change":"2023-10-18T09:04:36.476781+0000","last_active":"2023-10-18T09:04:37.488786+0000","last_peered":"2023-10-18T09:04:37.488786+0000","last_clean":"2023-10-18T09:04:37.488786+0000","last_became_active":"2023-10-18T09:04:36.476457+0000","last_became_peered":"2023-10-18T09:04:36.476457+0000","last_unstale":"2023-10-18T09:04:37.488786+0000","last_undegraded":"2023-10-18T09:04:37.488786+0000","last_fullsized":"2023-10-18T09:04:37.488786+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:46:40.977761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.468593+0000","last_change":"2023-10-18T09:04:37.489105+0000","last_active":"2023-10-18T09:04:38.468593+0000","last_peered":"2023-10-18T09:04:38.468593+0000","last_clean":"2023-10-18T09:04:38.468593+0000","last_became_active":"2023-10-18T09:04:37.488451+0000","last_became_peered":"2023-10-18T09:04:37.488451+0000","last_unstale":"2023-10-18T09:04:38.468593+0000","last_undegraded":"2023-10-18T09:04:38.468593+0000","last_fullsized":"2023-10-18T09:04:38.468593+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:30:05.448881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"161'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.485161+0000","last_change":"2023-10-18T09:04:36.472754+0000","last_active":"2023-10-18T09:04:37.485161+0000","last_peered":"2023-10-18T09:04:37.485161+0000","last_clean":"2023-10-18T09:04:37.485161+0000","last_became_active":"2023-10-18T09:04:36.472506+0000","last_became_peered":"2023-10-18T09:04:36.472506+0000","last_unstale":"2023-10-18T09:04:37.485161+0000","last_undegraded":"2023-10-18T09:04:37.485161+0000","last_fullsized":"2023-10-18T09:04:37.485161+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:16:35.109333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464444+0000","last_change":"2023-10-18T09:04:37.475267+0000","last_active":"2023-10-18T09:04:38.464444+0000","last_peered":"2023-10-18T09:04:38.464444+0000","last_clean":"2023-10-18T09:04:38.464444+0000","last_became_active":"2023-10-18T09:04:37.473167+0000","last_became_peered":"2023-10-18T09:04:37.473167+0000","last_unstale":"2023-10-18T09:04:38.464444+0000","last_undegraded":"2023-10-18T09:04:38.464444+0000","last_fullsized":"2023-10-18T09:04:38.464444+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:19:15.019455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"161'1","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.482597+0000","last_change":"2023-10-18T09:04:36.469557+0000","last_active":"2023-10-18T09:04:37.482597+0000","last_peered":"2023-10-18T09:04:37.482597+0000","last_clean":"2023-10-18T09:04:37.482597+0000","last_became_active":"2023-10-18T09:04:36.469311+0000","last_became_peered":"2023-10-18T09:04:36.469311+0000","last_unstale":"2023-10-18T09:04:37.482597+0000","last_undegraded":"2023-10-18T09:04:37.482597+0000","last_fullsized":"2023-10-18T09:04:37.482597+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:04:55.278850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.468954+0000","last_change":"2023-10-18T09:04:37.492521+0000","last_active":"2023-10-18T09:04:38.468954+0000","last_peered":"2023-10-18T09:04:38.468954+0000","last_clean":"2023-10-18T09:04:38.468954+0000","last_became_active":"2023-10-18T09:04:37.491449+0000","last_became_peered":"2023-10-18T09:04:37.491449+0000","last_unstale":"2023-10-18T09:04:38.468954+0000","last_undegraded":"2023-10-18T09:04:38.468954+0000","last_fullsized":"2023-10-18T09:04:38.468954+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:30:02.310872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"161'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464572+0000","last_change":"2023-10-18T09:04:36.466293+0000","last_active":"2023-10-18T09:04:38.464572+0000","last_peered":"2023-10-18T09:04:38.464572+0000","last_clean":"2023-10-18T09:04:38.464572+0000","last_became_active":"2023-10-18T09:04:36.466055+0000","last_became_peered":"2023-10-18T09:04:36.466055+0000","last_unstale":"2023-10-18T09:04:38.464572+0000","last_undegraded":"2023-10-18T09:04:38.464572+0000","last_fullsized":"2023-10-18T09:04:38.464572+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:45.608235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.488389+0000","last_change":"2023-10-18T09:04:37.487476+0000","last_active":"2023-10-18T09:04:37.488389+0000","last_peered":"2023-10-18T09:04:37.488389+0000","last_clean":"2023-10-18T09:04:37.488389+0000","last_became_active":"2023-10-18T09:04:37.487188+0000","last_became_peered":"2023-10-18T09:04:37.487188+0000","last_unstale":"2023-10-18T09:04:37.488389+0000","last_undegraded":"2023-10-18T09:04:37.488389+0000","last_fullsized":"2023-10-18T09:04:37.488389+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:29:21.820463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.466288+0000","last_change":"2023-10-18T09:04:36.484738+0000","last_active":"2023-10-18T09:04:38.466288+0000","last_peered":"2023-10-18T09:04:38.466288+0000","last_clean":"2023-10-18T09:04:38.466288+0000","last_became_active":"2023-10-18T09:04:36.484459+0000","last_became_peered":"2023-10-18T09:04:36.484459+0000","last_unstale":"2023-10-18T09:04:38.466288+0000","last_undegraded":"2023-10-18T09:04:38.466288+0000","last_fullsized":"2023-10-18T09:04:38.466288+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:26:15.613015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.475051+0000","last_change":"2023-10-18T09:04:37.474563+0000","last_active":"2023-10-18T09:04:37.475051+0000","last_peered":"2023-10-18T09:04:37.475051+0000","last_clean":"2023-10-18T09:04:37.475051+0000","last_became_active":"2023-10-18T09:04:37.474327+0000","last_became_peered":"2023-10-18T09:04:37.474327+0000","last_unstale":"2023-10-18T09:04:37.475051+0000","last_undegraded":"2023-10-18T09:04:37.475051+0000","last_fullsized":"2023-10-18T09:04:37.475051+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:45.114942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.474300+0000","last_change":"2023-10-18T09:04:36.481384+0000","last_active":"2023-10-18T09:04:37.474300+0000","last_peered":"2023-10-18T09:04:37.474300+0000","last_clean":"2023-10-18T09:04:37.474300+0000","last_became_active":"2023-10-18T09:04:36.481043+0000","last_became_peered":"2023-10-18T09:04:36.481043+0000","last_unstale":"2023-10-18T09:04:37.474300+0000","last_undegraded":"2023-10-18T09:04:37.474300+0000","last_fullsized":"2023-10-18T09:04:37.474300+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:20:40.862835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.484435+0000","last_change":"2023-10-18T09:04:37.484242+0000","last_active":"2023-10-18T09:04:37.484435+0000","last_peered":"2023-10-18T09:04:37.484435+0000","last_clean":"2023-10-18T09:04:37.484435+0000","last_became_active":"2023-10-18T09:04:37.483807+0000","last_became_peered":"2023-10-18T09:04:37.483807+0000","last_unstale":"2023-10-18T09:04:37.484435+0000","last_undegraded":"2023-10-18T09:04:37.484435+0000","last_fullsized":"2023-10-18T09:04:37.484435+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:02.637433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.477985+0000","last_change":"2023-10-18T09:04:36.479268+0000","last_active":"2023-10-18T09:04:37.477985+0000","last_peered":"2023-10-18T09:04:37.477985+0000","last_clean":"2023-10-18T09:04:37.477985+0000","last_became_active":"2023-10-18T09:04:36.478796+0000","last_became_peered":"2023-10-18T09:04:36.478796+0000","last_unstale":"2023-10-18T09:04:37.477985+0000","last_undegraded":"2023-10-18T09:04:37.477985+0000","last_fullsized":"2023-10-18T09:04:37.477985+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:52.652054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.494013+0000","last_change":"2023-10-18T09:04:37.492588+0000","last_active":"2023-10-18T09:04:37.494013+0000","last_peered":"2023-10-18T09:04:37.494013+0000","last_clean":"2023-10-18T09:04:37.494013+0000","last_became_active":"2023-10-18T09:04:37.492318+0000","last_became_peered":"2023-10-18T09:04:37.492318+0000","last_unstale":"2023-10-18T09:04:37.494013+0000","last_undegraded":"2023-10-18T09:04:37.494013+0000","last_fullsized":"2023-10-18T09:04:37.494013+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:24:13.080777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464214+0000","last_change":"2023-10-18T09:04:36.475728+0000","last_active":"2023-10-18T09:04:38.464214+0000","last_peered":"2023-10-18T09:04:38.464214+0000","last_clean":"2023-10-18T09:04:38.464214+0000","last_became_active":"2023-10-18T09:04:36.475221+0000","last_became_peered":"2023-10-18T09:04:36.475221+0000","last_unstale":"2023-10-18T09:04:38.464214+0000","last_undegraded":"2023-10-18T09:04:38.464214+0000","last_fullsized":"2023-10-18T09:04:38.464214+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:01:12.988228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.466342+0000","last_change":"2023-10-18T09:04:37.475822+0000","last_active":"2023-10-18T09:04:38.466342+0000","last_peered":"2023-10-18T09:04:38.466342+0000","last_clean":"2023-10-18T09:04:38.466342+0000","last_became_active":"2023-10-18T09:04:37.475163+0000","last_became_peered":"2023-10-18T09:04:37.475163+0000","last_unstale":"2023-10-18T09:04:38.466342+0000","last_undegraded":"2023-10-18T09:04:38.466342+0000","last_fullsized":"2023-10-18T09:04:38.466342+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:43:44.007470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"161'3","reported_seq":14,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.495304+0000","last_change":"2023-10-18T09:04:36.480880+0000","last_active":"2023-10-18T09:04:37.495304+0000","last_peered":"2023-10-18T09:04:37.495304+0000","last_clean":"2023-10-18T09:04:37.495304+0000","last_became_active":"2023-10-18T09:04:36.480577+0000","last_became_peered":"2023-10-18T09:04:36.480577+0000","last_unstale":"2023-10-18T09:04:37.495304+0000","last_undegraded":"2023-10-18T09:04:37.495304+0000","last_fullsized":"2023-10-18T09:04:37.495304+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:25.938371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.466275+0000","last_change":"2023-10-18T09:04:37.497277+0000","last_active":"2023-10-18T09:04:38.466275+0000","last_peered":"2023-10-18T09:04:38.466275+0000","last_clean":"2023-10-18T09:04:38.466275+0000","last_became_active":"2023-10-18T09:04:37.497040+0000","last_became_peered":"2023-10-18T09:04:37.497040+0000","last_unstale":"2023-10-18T09:04:38.466275+0000","last_undegraded":"2023-10-18T09:04:38.466275+0000","last_fullsized":"2023-10-18T09:04:38.466275+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:50:00.258088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"161'6","reported_seq":17,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.500880+0000","last_change":"2023-10-18T09:04:36.484500+0000","last_active":"2023-10-18T09:04:37.500880+0000","last_peered":"2023-10-18T09:04:37.500880+0000","last_clean":"2023-10-18T09:04:37.500880+0000","last_became_active":"2023-10-18T09:04:36.484205+0000","last_became_peered":"2023-10-18T09:04:36.484205+0000","last_unstale":"2023-10-18T09:04:37.500880+0000","last_undegraded":"2023-10-18T09:04:37.500880+0000","last_fullsized":"2023-10-18T09:04:37.500880+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:44:22.781501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.486999+0000","last_change":"2023-10-18T09:04:37.486852+0000","last_active":"2023-10-18T09:04:37.486999+0000","last_peered":"2023-10-18T09:04:37.486999+0000","last_clean":"2023-10-18T09:04:37.486999+0000","last_became_active":"2023-10-18T09:04:37.486601+0000","last_became_peered":"2023-10-18T09:04:37.486601+0000","last_unstale":"2023-10-18T09:04:37.486999+0000","last_undegraded":"2023-10-18T09:04:37.486999+0000","last_fullsized":"2023-10-18T09:04:37.486999+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:46:18.746561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.472166+0000","last_change":"2023-10-18T09:04:36.478034+0000","last_active":"2023-10-18T09:04:37.472166+0000","last_peered":"2023-10-18T09:04:37.472166+0000","last_clean":"2023-10-18T09:04:37.472166+0000","last_became_active":"2023-10-18T09:04:36.477811+0000","last_became_peered":"2023-10-18T09:04:36.477811+0000","last_unstale":"2023-10-18T09:04:37.472166+0000","last_undegraded":"2023-10-18T09:04:37.472166+0000","last_fullsized":"2023-10-18T09:04:37.472166+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:22:42.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.497913+0000","last_change":"2023-10-18T09:04:37.497813+0000","last_active":"2023-10-18T09:04:37.497913+0000","last_peered":"2023-10-18T09:04:37.497913+0000","last_clean":"2023-10-18T09:04:37.497913+0000","last_became_active":"2023-10-18T09:04:37.497528+0000","last_became_peered":"2023-10-18T09:04:37.497528+0000","last_unstale":"2023-10-18T09:04:37.497913+0000","last_undegraded":"2023-10-18T09:04:37.497913+0000","last_fullsized":"2023-10-18T09:04:37.497913+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:27:26.368529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.471920+0000","last_change":"2023-10-18T09:04:36.482256+0000","last_active":"2023-10-18T09:04:37.471920+0000","last_peered":"2023-10-18T09:04:37.471920+0000","last_clean":"2023-10-18T09:04:37.471920+0000","last_became_active":"2023-10-18T09:04:36.481678+0000","last_became_peered":"2023-10-18T09:04:36.481678+0000","last_unstale":"2023-10-18T09:04:37.471920+0000","last_undegraded":"2023-10-18T09:04:37.471920+0000","last_fullsized":"2023-10-18T09:04:37.471920+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:13:40.119610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.499241+0000","last_change":"2023-10-18T09:04:37.498864+0000","last_active":"2023-10-18T09:04:37.499241+0000","last_peered":"2023-10-18T09:04:37.499241+0000","last_clean":"2023-10-18T09:04:37.499241+0000","last_became_active":"2023-10-18T09:04:37.496458+0000","last_became_peered":"2023-10-18T09:04:37.496458+0000","last_unstale":"2023-10-18T09:04:37.499241+0000","last_undegraded":"2023-10-18T09:04:37.499241+0000","last_fullsized":"2023-10-18T09:04:37.499241+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:34:11.046102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"161'8","reported_seq":19,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.504419+0000","last_change":"2023-10-18T09:04:36.474474+0000","last_active":"2023-10-18T09:04:37.504419+0000","last_peered":"2023-10-18T09:04:37.504419+0000","last_clean":"2023-10-18T09:04:37.504419+0000","last_became_active":"2023-10-18T09:04:36.473849+0000","last_became_peered":"2023-10-18T09:04:36.473849+0000","last_unstale":"2023-10-18T09:04:37.504419+0000","last_undegraded":"2023-10-18T09:04:37.504419+0000","last_fullsized":"2023-10-18T09:04:37.504419+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:45.608235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.495604+0000","last_change":"2023-10-18T09:04:37.495199+0000","last_active":"2023-10-18T09:04:37.495604+0000","last_peered":"2023-10-18T09:04:37.495604+0000","last_clean":"2023-10-18T09:04:37.495604+0000","last_became_active":"2023-10-18T09:04:37.494917+0000","last_became_peered":"2023-10-18T09:04:37.494917+0000","last_unstale":"2023-10-18T09:04:37.495604+0000","last_undegraded":"2023-10-18T09:04:37.495604+0000","last_fullsized":"2023-10-18T09:04:37.495604+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:15:38.352083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"161'1","reported_seq":12,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.497691+0000","last_change":"2023-10-18T09:04:36.484780+0000","last_active":"2023-10-18T09:04:37.497691+0000","last_peered":"2023-10-18T09:04:37.497691+0000","last_clean":"2023-10-18T09:04:37.497691+0000","last_became_active":"2023-10-18T09:04:36.484512+0000","last_became_peered":"2023-10-18T09:04:36.484512+0000","last_unstale":"2023-10-18T09:04:37.497691+0000","last_undegraded":"2023-10-18T09:04:37.497691+0000","last_fullsized":"2023-10-18T09:04:37.497691+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:22.674318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.489276+0000","last_change":"2023-10-18T09:04:37.487955+0000","last_active":"2023-10-18T09:04:37.489276+0000","last_peered":"2023-10-18T09:04:37.489276+0000","last_clean":"2023-10-18T09:04:37.489276+0000","last_became_active":"2023-10-18T09:04:37.487720+0000","last_became_peered":"2023-10-18T09:04:37.487720+0000","last_unstale":"2023-10-18T09:04:37.489276+0000","last_undegraded":"2023-10-18T09:04:37.489276+0000","last_fullsized":"2023-10-18T09:04:37.489276+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:28:31.128917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"161'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.496383+0000","last_change":"2023-10-18T09:04:36.483032+0000","last_active":"2023-10-18T09:04:37.496383+0000","last_peered":"2023-10-18T09:04:37.496383+0000","last_clean":"2023-10-18T09:04:37.496383+0000","last_became_active":"2023-10-18T09:04:36.482811+0000","last_became_peered":"2023-10-18T09:04:36.482811+0000","last_unstale":"2023-10-18T09:04:37.496383+0000","last_undegraded":"2023-10-18T09:04:37.496383+0000","last_fullsized":"2023-10-18T09:04:37.496383+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:09.373841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464079+0000","last_change":"2023-10-18T09:04:37.480507+0000","last_active":"2023-10-18T09:04:38.464079+0000","last_peered":"2023-10-18T09:04:38.464079+0000","last_clean":"2023-10-18T09:04:38.464079+0000","last_became_active":"2023-10-18T09:04:37.480272+0000","last_became_peered":"2023-10-18T09:04:37.480272+0000","last_unstale":"2023-10-18T09:04:38.464079+0000","last_undegraded":"2023-10-18T09:04:38.464079+0000","last_fullsized":"2023-10-18T09:04:38.464079+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:31:31.734747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"161'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.470193+0000","last_change":"2023-10-18T09:04:36.473416+0000","last_active":"2023-10-18T09:04:38.470193+0000","last_peered":"2023-10-18T09:04:38.470193+0000","last_clean":"2023-10-18T09:04:38.470193+0000","last_became_active":"2023-10-18T09:04:36.472278+0000","last_became_peered":"2023-10-18T09:04:36.472278+0000","last_unstale":"2023-10-18T09:04:38.470193+0000","last_undegraded":"2023-10-18T09:04:38.470193+0000","last_fullsized":"2023-10-18T09:04:38.470193+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:15.629297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.482451+0000","last_change":"2023-10-18T09:04:37.479718+0000","last_active":"2023-10-18T09:04:37.482451+0000","last_peered":"2023-10-18T09:04:37.482451+0000","last_clean":"2023-10-18T09:04:37.482451+0000","last_became_active":"2023-10-18T09:04:37.479434+0000","last_became_peered":"2023-10-18T09:04:37.479434+0000","last_unstale":"2023-10-18T09:04:37.482451+0000","last_undegraded":"2023-10-18T09:04:37.482451+0000","last_fullsized":"2023-10-18T09:04:37.482451+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:15:26.953399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.475095+0000","last_change":"2023-10-18T09:04:36.476351+0000","last_active":"2023-10-18T09:04:37.475095+0000","last_peered":"2023-10-18T09:04:37.475095+0000","last_clean":"2023-10-18T09:04:37.475095+0000","last_became_active":"2023-10-18T09:04:36.475442+0000","last_became_peered":"2023-10-18T09:04:36.475442+0000","last_unstale":"2023-10-18T09:04:37.475095+0000","last_undegraded":"2023-10-18T09:04:37.475095+0000","last_fullsized":"2023-10-18T09:04:37.475095+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:53.827947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.485430+0000","last_change":"2023-10-18T09:04:37.485334+0000","last_active":"2023-10-18T09:04:37.485430+0000","last_peered":"2023-10-18T09:04:37.485430+0000","last_clean":"2023-10-18T09:04:37.485430+0000","last_became_active":"2023-10-18T09:04:37.485122+0000","last_became_peered":"2023-10-18T09:04:37.485122+0000","last_unstale":"2023-10-18T09:04:37.485430+0000","last_undegraded":"2023-10-18T09:04:37.485430+0000","last_fullsized":"2023-10-18T09:04:37.485430+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:46:20.922307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"161'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.497372+0000","last_change":"2023-10-18T09:04:36.478652+0000","last_active":"2023-10-18T09:04:37.497372+0000","last_peered":"2023-10-18T09:04:37.497372+0000","last_clean":"2023-10-18T09:04:37.497372+0000","last_became_active":"2023-10-18T09:04:36.478372+0000","last_became_peered":"2023-10-18T09:04:36.478372+0000","last_unstale":"2023-10-18T09:04:37.497372+0000","last_undegraded":"2023-10-18T09:04:37.497372+0000","last_fullsized":"2023-10-18T09:04:37.497372+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:10:12.418887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.466328+0000","last_change":"2023-10-18T09:04:37.498983+0000","last_active":"2023-10-18T09:04:38.466328+0000","last_peered":"2023-10-18T09:04:38.466328+0000","last_clean":"2023-10-18T09:04:38.466328+0000","last_became_active":"2023-10-18T09:04:37.498731+0000","last_became_peered":"2023-10-18T09:04:37.498731+0000","last_unstale":"2023-10-18T09:04:38.466328+0000","last_undegraded":"2023-10-18T09:04:38.466328+0000","last_fullsized":"2023-10-18T09:04:38.466328+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:10:01.517852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464328+0000","last_change":"2023-10-18T09:04:36.482989+0000","last_active":"2023-10-18T09:04:38.464328+0000","last_peered":"2023-10-18T09:04:38.464328+0000","last_clean":"2023-10-18T09:04:38.464328+0000","last_became_active":"2023-10-18T09:04:36.482640+0000","last_became_peered":"2023-10-18T09:04:36.482640+0000","last_unstale":"2023-10-18T09:04:38.464328+0000","last_undegraded":"2023-10-18T09:04:38.464328+0000","last_fullsized":"2023-10-18T09:04:38.464328+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:29:20.805435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.486878+0000","last_change":"2023-10-18T09:04:37.486734+0000","last_active":"2023-10-18T09:04:37.486878+0000","last_peered":"2023-10-18T09:04:37.486878+0000","last_clean":"2023-10-18T09:04:37.486878+0000","last_became_active":"2023-10-18T09:04:37.486483+0000","last_became_peered":"2023-10-18T09:04:37.486483+0000","last_unstale":"2023-10-18T09:04:37.486878+0000","last_undegraded":"2023-10-18T09:04:37.486878+0000","last_fullsized":"2023-10-18T09:04:37.486878+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:27:53.667082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464329+0000","last_change":"2023-10-18T09:04:36.477401+0000","last_active":"2023-10-18T09:04:38.464329+0000","last_peered":"2023-10-18T09:04:38.464329+0000","last_clean":"2023-10-18T09:04:38.464329+0000","last_became_active":"2023-10-18T09:04:36.477056+0000","last_became_peered":"2023-10-18T09:04:36.477056+0000","last_unstale":"2023-10-18T09:04:38.464329+0000","last_undegraded":"2023-10-18T09:04:38.464329+0000","last_fullsized":"2023-10-18T09:04:38.464329+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:15:37.337056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.467927+0000","last_change":"2023-10-18T09:04:37.488951+0000","last_active":"2023-10-18T09:04:38.467927+0000","last_peered":"2023-10-18T09:04:38.467927+0000","last_clean":"2023-10-18T09:04:38.467927+0000","last_became_active":"2023-10-18T09:04:37.487915+0000","last_became_peered":"2023-10-18T09:04:37.487915+0000","last_unstale":"2023-10-18T09:04:38.467927+0000","last_undegraded":"2023-10-18T09:04:38.467927+0000","last_fullsized":"2023-10-18T09:04:38.467927+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:17:44.053614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.483956+0000","last_change":"2023-10-18T09:04:36.481368+0000","last_active":"2023-10-18T09:04:37.483956+0000","last_peered":"2023-10-18T09:04:37.483956+0000","last_clean":"2023-10-18T09:04:37.483956+0000","last_became_active":"2023-10-18T09:04:36.481009+0000","last_became_peered":"2023-10-18T09:04:36.481009+0000","last_unstale":"2023-10-18T09:04:37.483956+0000","last_undegraded":"2023-10-18T09:04:37.483956+0000","last_fullsized":"2023-10-18T09:04:37.483956+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:10:08.053287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.466675+0000","last_change":"2023-10-18T09:04:37.487140+0000","last_active":"2023-10-18T09:04:38.466675+0000","last_peered":"2023-10-18T09:04:38.466675+0000","last_clean":"2023-10-18T09:04:38.466675+0000","last_became_active":"2023-10-18T09:04:37.486128+0000","last_became_peered":"2023-10-18T09:04:37.486128+0000","last_unstale":"2023-10-18T09:04:38.466675+0000","last_undegraded":"2023-10-18T09:04:38.466675+0000","last_fullsized":"2023-10-18T09:04:38.466675+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:38:21.843954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"161'2","reported_seq":13,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.481425+0000","last_change":"2023-10-18T09:04:36.480113+0000","last_active":"2023-10-18T09:04:37.481425+0000","last_peered":"2023-10-18T09:04:37.481425+0000","last_clean":"2023-10-18T09:04:37.481425+0000","last_became_active":"2023-10-18T09:04:36.479802+0000","last_became_peered":"2023-10-18T09:04:36.479802+0000","last_unstale":"2023-10-18T09:04:37.481425+0000","last_undegraded":"2023-10-18T09:04:37.481425+0000","last_fullsized":"2023-10-18T09:04:37.481425+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:03.392350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.497657+0000","last_change":"2023-10-18T09:04:37.497545+0000","last_active":"2023-10-18T09:04:37.497657+0000","last_peered":"2023-10-18T09:04:37.497657+0000","last_clean":"2023-10-18T09:04:37.497657+0000","last_became_active":"2023-10-18T09:04:37.497301+0000","last_became_peered":"2023-10-18T09:04:37.497301+0000","last_unstale":"2023-10-18T09:04:37.497657+0000","last_undegraded":"2023-10-18T09:04:37.497657+0000","last_fullsized":"2023-10-18T09:04:37.497657+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:42.351629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.466784+0000","last_change":"2023-10-18T09:04:36.482262+0000","last_active":"2023-10-18T09:04:38.466784+0000","last_peered":"2023-10-18T09:04:38.466784+0000","last_clean":"2023-10-18T09:04:38.466784+0000","last_became_active":"2023-10-18T09:04:36.481928+0000","last_became_peered":"2023-10-18T09:04:36.481928+0000","last_unstale":"2023-10-18T09:04:38.466784+0000","last_undegraded":"2023-10-18T09:04:38.466784+0000","last_fullsized":"2023-10-18T09:04:38.466784+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:33.159619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.482412+0000","last_change":"2023-10-18T09:04:37.479395+0000","last_active":"2023-10-18T09:04:37.482412+0000","last_peered":"2023-10-18T09:04:37.482412+0000","last_clean":"2023-10-18T09:04:37.482412+0000","last_became_active":"2023-10-18T09:04:37.478828+0000","last_became_peered":"2023-10-18T09:04:37.478828+0000","last_unstale":"2023-10-18T09:04:37.482412+0000","last_undegraded":"2023-10-18T09:04:37.482412+0000","last_fullsized":"2023-10-18T09:04:37.482412+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:28:31.128917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.471000+0000","last_change":"2023-10-18T09:04:36.482319+0000","last_active":"2023-10-18T09:04:37.471000+0000","last_peered":"2023-10-18T09:04:37.471000+0000","last_clean":"2023-10-18T09:04:37.471000+0000","last_became_active":"2023-10-18T09:04:36.482100+0000","last_became_peered":"2023-10-18T09:04:36.482100+0000","last_unstale":"2023-10-18T09:04:37.471000+0000","last_undegraded":"2023-10-18T09:04:37.471000+0000","last_fullsized":"2023-10-18T09:04:37.471000+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:03:00.168165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.466071+0000","last_change":"2023-10-18T09:04:37.493295+0000","last_active":"2023-10-18T09:04:38.466071+0000","last_peered":"2023-10-18T09:04:38.466071+0000","last_clean":"2023-10-18T09:04:38.466071+0000","last_became_active":"2023-10-18T09:04:37.491590+0000","last_became_peered":"2023-10-18T09:04:37.491590+0000","last_unstale":"2023-10-18T09:04:38.466071+0000","last_undegraded":"2023-10-18T09:04:38.466071+0000","last_fullsized":"2023-10-18T09:04:38.466071+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:25:29.576966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.480409+0000","last_change":"2023-10-18T09:04:36.474249+0000","last_active":"2023-10-18T09:04:37.480409+0000","last_peered":"2023-10-18T09:04:37.480409+0000","last_clean":"2023-10-18T09:04:37.480409+0000","last_became_active":"2023-10-18T09:04:36.473336+0000","last_became_peered":"2023-10-18T09:04:36.473336+0000","last_unstale":"2023-10-18T09:04:37.480409+0000","last_undegraded":"2023-10-18T09:04:37.480409+0000","last_fullsized":"2023-10-18T09:04:37.480409+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:34:10.320203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.467138+0000","last_change":"2023-10-18T09:04:37.488680+0000","last_active":"2023-10-18T09:04:38.467138+0000","last_peered":"2023-10-18T09:04:38.467138+0000","last_clean":"2023-10-18T09:04:38.467138+0000","last_became_active":"2023-10-18T09:04:37.488362+0000","last_became_peered":"2023-10-18T09:04:37.488362+0000","last_unstale":"2023-10-18T09:04:38.467138+0000","last_undegraded":"2023-10-18T09:04:38.467138+0000","last_fullsized":"2023-10-18T09:04:38.467138+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:45.059619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.471332+0000","last_change":"2023-10-18T09:04:36.481955+0000","last_active":"2023-10-18T09:04:37.471332+0000","last_peered":"2023-10-18T09:04:37.471332+0000","last_clean":"2023-10-18T09:04:37.471332+0000","last_became_active":"2023-10-18T09:04:36.481665+0000","last_became_peered":"2023-10-18T09:04:36.481665+0000","last_unstale":"2023-10-18T09:04:37.471332+0000","last_undegraded":"2023-10-18T09:04:37.471332+0000","last_fullsized":"2023-10-18T09:04:37.471332+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:28.778957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.489296+0000","last_change":"2023-10-18T09:04:37.485920+0000","last_active":"2023-10-18T09:04:37.489296+0000","last_peered":"2023-10-18T09:04:37.489296+0000","last_clean":"2023-10-18T09:04:37.489296+0000","last_became_active":"2023-10-18T09:04:37.485646+0000","last_became_peered":"2023-10-18T09:04:37.485646+0000","last_unstale":"2023-10-18T09:04:37.489296+0000","last_undegraded":"2023-10-18T09:04:37.489296+0000","last_fullsized":"2023-10-18T09:04:37.489296+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:44:45.114942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.467500+0000","last_change":"2023-10-18T09:04:36.474466+0000","last_active":"2023-10-18T09:04:38.467500+0000","last_peered":"2023-10-18T09:04:38.467500+0000","last_clean":"2023-10-18T09:04:38.467500+0000","last_became_active":"2023-10-18T09:04:36.473572+0000","last_became_peered":"2023-10-18T09:04:36.473572+0000","last_unstale":"2023-10-18T09:04:38.467500+0000","last_undegraded":"2023-10-18T09:04:38.467500+0000","last_fullsized":"2023-10-18T09:04:38.467500+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:48:45.314291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464182+0000","last_change":"2023-10-18T09:04:37.489671+0000","last_active":"2023-10-18T09:04:38.464182+0000","last_peered":"2023-10-18T09:04:38.464182+0000","last_clean":"2023-10-18T09:04:38.464182+0000","last_became_active":"2023-10-18T09:04:37.489434+0000","last_became_peered":"2023-10-18T09:04:37.489434+0000","last_unstale":"2023-10-18T09:04:38.464182+0000","last_undegraded":"2023-10-18T09:04:38.464182+0000","last_fullsized":"2023-10-18T09:04:38.464182+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:07:54.115841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.464707+0000","last_change":"2023-10-18T09:04:36.475206+0000","last_active":"2023-10-18T09:04:38.464707+0000","last_peered":"2023-10-18T09:04:38.464707+0000","last_clean":"2023-10-18T09:04:38.464707+0000","last_became_active":"2023-10-18T09:04:36.474771+0000","last_became_peered":"2023-10-18T09:04:36.474771+0000","last_unstale":"2023-10-18T09:04:38.464707+0000","last_undegraded":"2023-10-18T09:04:38.464707+0000","last_fullsized":"2023-10-18T09:04:38.464707+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:23:19.444961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.484487+0000","last_change":"2023-10-18T09:04:37.483172+0000","last_active":"2023-10-18T09:04:37.484487+0000","last_peered":"2023-10-18T09:04:37.484487+0000","last_clean":"2023-10-18T09:04:37.484487+0000","last_became_active":"2023-10-18T09:04:37.482822+0000","last_became_peered":"2023-10-18T09:04:37.482822+0000","last_unstale":"2023-10-18T09:04:37.484487+0000","last_undegraded":"2023-10-18T09:04:37.484487+0000","last_fullsized":"2023-10-18T09:04:37.484487+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:31:23.689345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.483431+0000","last_change":"2023-10-18T09:04:36.478477+0000","last_active":"2023-10-18T09:04:37.483431+0000","last_peered":"2023-10-18T09:04:37.483431+0000","last_clean":"2023-10-18T09:04:37.483431+0000","last_became_active":"2023-10-18T09:04:36.478223+0000","last_became_peered":"2023-10-18T09:04:36.478223+0000","last_unstale":"2023-10-18T09:04:37.483431+0000","last_undegraded":"2023-10-18T09:04:37.483431+0000","last_fullsized":"2023-10-18T09:04:37.483431+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:11.857064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.468561+0000","last_change":"2023-10-18T09:04:37.486003+0000","last_active":"2023-10-18T09:04:38.468561+0000","last_peered":"2023-10-18T09:04:38.468561+0000","last_clean":"2023-10-18T09:04:38.468561+0000","last_became_active":"2023-10-18T09:04:37.485197+0000","last_became_peered":"2023-10-18T09:04:37.485197+0000","last_unstale":"2023-10-18T09:04:38.468561+0000","last_undegraded":"2023-10-18T09:04:38.468561+0000","last_fullsized":"2023-10-18T09:04:38.468561+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:36.441543+0000","last_clean_scrub_stamp":"2023-10-18T09:04:36.441543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:33:09.755834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"161'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-18T09:04:38.468861+0000","last_change":"2023-10-18T09:04:36.477786+0000","last_active":"2023-10-18T09:04:38.468861+0000","last_peered":"2023-10-18T09:04:38.468861+0000","last_clean":"2023-10-18T09:04:38.468861+0000","last_became_active":"2023-10-18T09:04:36.477028+0000","last_became_peered":"2023-10-18T09:04:36.477028+0000","last_unstale":"2023-10-18T09:04:38.468861+0000","last_undegraded":"2023-10-18T09:04:38.468861+0000","last_fullsized":"2023-10-18T09:04:38.468861+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:04:35.426515+0000","last_clean_scrub_stamp":"2023-10-18T09:04:35.426515+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:02:13.687471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":320,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-18T09:04:37.467642+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:04:37.467642+0000","last_peered":"2023-10-18T09:04:37.467642+0000","last_clean":"2023-10-18T09:04:37.467642+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:04:37.467642+0000","last_undegraded":"2023-10-18T09:04:37.467642+0000","last_fullsized":"2023-10-18T09:04:37.467642+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574965,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1472,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193839,"statfs":{"total":200000000,"available":198492084,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.1080000000000001,"15min":1.042},"min":{"1min":0.70099999999999996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.5620000000000001,"5min":1.6040000000000001,"15min":1.6990000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0880000000000001,"15min":1.05},"min":{"1min":0.63300000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.552,"5min":1.6439999999999999,"15min":1.97},"last":1.2509999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.177,"15min":1.1539999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.61799999999999999},"max":{"1min":1.6839999999999999,"5min":1.829,"15min":1.829},"last":1.179},{"interface":"front","average":{"1min":1.038,"5min":1.089,"15min":1.121},"min":{"1min":0.73399999999999999,"5min":0.61099999999999999,"15min":0.58799999999999997},"max":{"1min":1.409,"5min":1.5469999999999999,"15min":1.7989999999999999},"last":1.468}]},{"osd":2,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.3089999999999999,"15min":1.27},"min":{"1min":0.94099999999999995,"5min":0.78600000000000003,"15min":0.65400000000000003},"max":{"1min":1.7250000000000001,"5min":1.8169999999999999,"15min":1.9730000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.266,"15min":1.224},"min":{"1min":0.79400000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.671,"5min":1.863,"15min":1.863},"last":1.369}]},{"osd":3,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.2809999999999999,"15min":1.266},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.677,"5min":1.98,"15min":1.98},"last":1.319},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3480000000000001,"15min":1.3320000000000001},"min":{"1min":0.83199999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7210000000000001,"5min":1.756,"15min":1.9930000000000001},"last":1.7170000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.2370000000000001,"15min":1.196},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60499999999999998},"max":{"1min":1.6459999999999999,"5min":1.923,"15min":2.1070000000000002},"last":1.133},{"interface":"front","average":{"1min":1.27,"5min":1.335,"15min":1.3680000000000001},"min":{"1min":0.77800000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.8169999999999999,"5min":2.0369999999999999,"15min":2.5169999999999999},"last":1.4139999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.323,"15min":1.347},"min":{"1min":0.89300000000000002,"5min":0.622,"15min":0.622},"max":{"1min":1.6339999999999999,"5min":1.831,"15min":2.0720000000000001},"last":1.7989999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.425,"15min":1.4179999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9490000000000001,"5min":2.0259999999999998,"15min":2.0270000000000001},"last":1.544}]},{"osd":6,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.502,"15min":1.472},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.0139999999999998,"5min":2.0779999999999998,"15min":2.3959999999999999},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.496,"15min":1.5},"min":{"1min":0.80800000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8779999999999999,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.6499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574965,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":889,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194423,"statfs":{"total":200000000,"available":199089390,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:04:11 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.256,"15min":1.1539999999999999},"min":{"1min":0.85299999999999998,"5min":0.64700000000000002,"15min":0.60299999999999998},"max":{"1min":1.4950000000000001,"5min":1.627,"15min":1.6850000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.353,"15min":1.2769999999999999},"min":{"1min":0.88800000000000001,"5min":0.81799999999999995,"15min":0.64700000000000002},"max":{"1min":1.5640000000000001,"5min":2.0990000000000002,"15min":2.0990000000000002},"last":1.0900000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:04:11 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.107,"15min":1.0940000000000001},"min":{"1min":0.68799999999999994,"5min":0.63900000000000001,"15min":0.63800000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.2050000000000001,"15min":1.1970000000000001},"min":{"1min":0.73099999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5720000000000001,"5min":1.8560000000000001,"15min":2.9609999999999999},"last":1.2669999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:04:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.1839999999999999,"15min":1.1759999999999999},"min":{"1min":0.75900000000000001,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.232,"15min":1.212},"min":{"1min":0.91200000000000003,"5min":0.73399999999999999,"15min":0.66300000000000003},"max":{"1min":1.5620000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.4139999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:04:11 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.296,"15min":1.268},"min":{"1min":0.89100000000000001,"5min":0.81000000000000005,"15min":0.79200000000000004},"max":{"1min":1.5629999999999999,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.333},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3120000000000001,"15min":1.2549999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.3029999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:04:11 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.2889999999999999,"15min":1.2230000000000001},"min":{"1min":0.82699999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.8089999999999999,"5min":2.444,"15min":2.444},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.3049999999999999,"15min":1.2809999999999999},"min":{"1min":0.86099999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.756,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.1659999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:04:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.4399999999999999,"15min":1.3879999999999999},"min":{"1min":0.99299999999999999,"5min":0.625,"15min":0.58899999999999997},"max":{"1min":1.675,"5min":2.032,"15min":2.032},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.3580000000000001,"15min":1.2729999999999999},"min":{"1min":0.91300000000000003,"5min":0.55100000000000005,"15min":0.49199999999999999},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.218}]},{"osd":7,"last update":"Wed Oct 18 09:04:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4119999999999999,"15min":1.4039999999999999},"min":{"1min":0.89400000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.6830000000000001,"5min":2.3540000000000001,"15min":2.3540000000000001},"last":1.48},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.552,"15min":1.528},"min":{"1min":1.226,"5min":0.83099999999999996,"15min":0.68500000000000005},"max":{"1min":1.9199999999999999,"5min":2.3140000000000001,"15min":2.794},"last":1.3999999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":887,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194424,"statfs":{"total":200000000,"available":199090890,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:04:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.325,"15min":1.278},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.63400000000000001},"max":{"1min":1.595,"5min":9.609,"15min":9.609},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.2909999999999999,"15min":1.302},"min":{"1min":0.94499999999999995,"5min":0.751,"15min":0.60799999999999998},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":5.3029999999999999},"last":1.4450000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:04:18 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3380000000000001,"15min":1.254},"min":{"1min":0.94299999999999995,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5329999999999999,"5min":11.68,"15min":11.68},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.21,"5min":1.2050000000000001,"15min":1.218},"min":{"1min":0.83799999999999997,"5min":0.70099999999999996,"15min":0.67000000000000004},"max":{"1min":1.841,"5min":2.2410000000000001,"15min":3.7050000000000001},"last":1.0109999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:04:18 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2390000000000001,"15min":1.252},"min":{"1min":0.79700000000000004,"5min":0.76800000000000002,"15min":0.755},"max":{"1min":1.556,"5min":2.0350000000000001,"15min":2.2090000000000001},"last":1.276},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.276,"15min":1.2769999999999999},"min":{"1min":0.79000000000000004,"5min":0.75900000000000001,"15min":0.72399999999999998},"max":{"1min":1.7270000000000001,"5min":2.153,"15min":3.8180000000000001},"last":1.3280000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:04:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.327,"15min":1.2669999999999999},"min":{"1min":0.77500000000000002,"5min":0.68799999999999994,"15min":0.65400000000000003},"max":{"1min":1.9810000000000001,"5min":11.797000000000001,"15min":11.797000000000001},"last":1.212},{"interface":"front","average":{"1min":1.377,"5min":1.379,"15min":1.3500000000000001},"min":{"1min":1.0089999999999999,"5min":0.747,"15min":0.747},"max":{"1min":1.7410000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":1.476}]},{"osd":4,"last update":"Wed Oct 18 09:04:18 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.361,"15min":1.3999999999999999},"min":{"1min":0.92600000000000005,"5min":0.89900000000000002,"15min":0.75900000000000001},"max":{"1min":1.6459999999999999,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":1.415},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.2609999999999999,"15min":1.417},"min":{"1min":0.96599999999999997,"5min":0.83699999999999997,"15min":0.64100000000000001},"max":{"1min":2.1360000000000001,"5min":2.4049999999999998,"15min":5.4580000000000002},"last":1.55}]},{"osd":6,"last update":"Wed Oct 18 09:04:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.266,"15min":1.343},"min":{"1min":0.94399999999999995,"5min":0.80300000000000005,"15min":0.67400000000000004},"max":{"1min":2.1030000000000002,"5min":2.5190000000000001,"15min":5.141},"last":1.371},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.5209999999999999,"15min":1.5509999999999999},"min":{"1min":1.3,"5min":0.82499999999999996,"15min":0.71099999999999997},"max":{"1min":1.9059999999999999,"5min":2.657,"15min":2.657},"last":1.597}]},{"osd":7,"last update":"Wed Oct 18 09:04:18 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.5680000000000001,"15min":1.6120000000000001},"min":{"1min":1.0529999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.444,"5min":11.829000000000001,"15min":11.829000000000001},"last":1.6279999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.405,"15min":1.4750000000000001},"min":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.73399999999999999},"max":{"1min":2.6419999999999999,"5min":2.9420000000000002,"15min":4.8419999999999996},"last":1.5}]}]},{"osd":4,"up_from":13,"seq":55834574965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":892,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194419,"statfs":{"total":200000000,"available":199085651,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.157,"15min":1.121},"min":{"1min":0.76800000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.8100000000000001,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.1579999999999999,"15min":1.153},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6339999999999999,"5min":3.234,"15min":3.234},"last":1.0369999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.169,"15min":1.117},"min":{"1min":0.73699999999999999,"5min":0.64700000000000002,"15min":0.60799999999999998},"max":{"1min":1.5840000000000001,"5min":3.718,"15min":3.718},"last":1.202},{"interface":"front","average":{"1min":1.181,"5min":1.216,"15min":1.2190000000000001},"min":{"1min":0.76300000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5529999999999999,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":0.90400000000000003}]},{"osd":2,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.216,"15min":1.1950000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.56699999999999995},"max":{"1min":1.589,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.347,"15min":1.3080000000000001},"min":{"1min":0.90700000000000003,"5min":0.79500000000000004,"15min":0.58899999999999997},"max":{"1min":1.704,"5min":3.532,"15min":3.532},"last":1.1080000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.2749999999999999,"15min":1.2829999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.67000000000000004},"max":{"1min":1.641,"5min":3.363,"15min":3.363},"last":1.462},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3979999999999999,"15min":1.407},"min":{"1min":1.0249999999999999,"5min":0.84999999999999998,"15min":0.68899999999999995},"max":{"1min":1.675,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":1.375}]},{"osd":5,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.304,"15min":1.3029999999999999},"min":{"1min":1.046,"5min":0.64100000000000001,"15min":0.498},"max":{"1min":1.712,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":1.246},{"interface":"front","average":{"1min":1.361,"5min":1.323,"15min":1.3400000000000001},"min":{"1min":1.097,"5min":0.70699999999999996,"15min":0.38400000000000001},"max":{"1min":1.5800000000000001,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":1.3160000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.4019999999999999,"15min":1.403},"min":{"1min":1.1779999999999999,"5min":0.78300000000000003,"15min":0.51600000000000001},"max":{"1min":1.873,"5min":4.4429999999999996,"15min":5.0049999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.476,"15min":1.5309999999999999},"min":{"1min":1.375,"5min":0.80100000000000005,"15min":0.45400000000000001},"max":{"1min":1.871,"5min":4.1239999999999997,"15min":4.407},"last":1.411}]},{"osd":7,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.452,"15min":1.4790000000000001},"min":{"1min":1.149,"5min":0.85599999999999998,"15min":0.438},"max":{"1min":1.804,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.595,"5min":1.5489999999999999,"15min":1.5489999999999999},"min":{"1min":1.1160000000000001,"5min":0.92000000000000004,"15min":0.747},"max":{"1min":1.966,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":1.5189999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":894,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194418,"statfs":{"total":200000000,"available":199084542,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:04:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.93300000000000005,"15min":0.85399999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.827,"5min":4.774,"15min":5.5730000000000004},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.94299999999999995,"15min":0.92000000000000004},"min":{"1min":0.40899999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.5880000000000001,"5min":3.2370000000000001,"15min":6.5599999999999996},"last":0.95199999999999996}]},{"osd":1,"last update":"Wed Oct 18 09:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.004,"15min":0.97799999999999998},"min":{"1min":0.73299999999999998,"5min":0.39100000000000001,"15min":0.36799999999999999},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":6.8099999999999996},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.2509999999999999,"15min":1.1799999999999999},"min":{"1min":0.80500000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.702,"5min":1.8080000000000001,"15min":3.1920000000000002},"last":1.7949999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.121,"15min":1.0600000000000001},"min":{"1min":0.58999999999999997,"5min":0.46800000000000003,"15min":0.45300000000000001},"max":{"1min":1.647,"5min":2.6269999999999998,"15min":7.4580000000000002},"last":1.9550000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.206,"15min":1.125},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.51400000000000001},"max":{"1min":1.52,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.3109999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.3200000000000001,"15min":1.347},"min":{"1min":0.65400000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.635,"5min":3.1139999999999999,"15min":18.664999999999999},"last":1.633},{"interface":"front","average":{"1min":1.335,"5min":1.306,"15min":1.2250000000000001},"min":{"1min":0.91300000000000003,"5min":0.877,"15min":0.64000000000000001},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":7.0700000000000003},"last":2.0289999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.327,"15min":1.2749999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.8540000000000001,"5min":1.998,"15min":6.9139999999999997},"last":1.8560000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4750000000000001,"15min":1.421},"min":{"1min":0.72999999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.855,"5min":4.5540000000000003,"15min":16.390999999999998},"last":2.6499999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.478,"15min":1.4570000000000001},"min":{"1min":0.98899999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.1040000000000001,"5min":3.375,"15min":7.7270000000000003},"last":2.0870000000000002},{"interface":"front","average":{"1min":1.498,"5min":1.4950000000000001,"15min":1.409},"min":{"1min":1.131,"5min":0.872,"15min":0.60199999999999998},"max":{"1min":2.351,"5min":2.4860000000000002,"15min":7.242},"last":1.538}]},{"osd":7,"last update":"Wed Oct 18 09:04:17 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.5780000000000001,"15min":1.4399999999999999},"min":{"1min":0.90200000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.96,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":2.8170000000000002},{"interface":"front","average":{"1min":1.522,"5min":1.556,"15min":1.5109999999999999},"min":{"1min":0.82899999999999996,"5min":0.622,"15min":0.622},"max":{"1min":1.915,"5min":5.4900000000000002,"15min":12.946},"last":2.4590000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":888,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194424,"statfs":{"total":200000000,"available":199090386,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.84299999999999997,"15min":0.90100000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.095,"5min":1.5660000000000001,"15min":4.2210000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.90000000000000002,"5min":1.0349999999999999,"15min":1.028},"min":{"1min":0.40200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.514,"5min":4.3959999999999999,"15min":4.3959999999999999},"last":0.59699999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1359999999999999,"15min":1.077},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.49099999999999999},"max":{"1min":1.6200000000000001,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":1.238},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.1419999999999999,"15min":1.1080000000000001},"min":{"1min":0.57299999999999995,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":1.9510000000000001,"5min":2.448,"15min":2.5350000000000001},"last":0.68100000000000005}]},{"osd":3,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.173,"15min":1.131},"min":{"1min":0.57899999999999996,"5min":0.51200000000000001,"15min":0.48199999999999998},"max":{"1min":1.6379999999999999,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.232,"5min":1.3180000000000001,"15min":1.407},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.54100000000000004},"max":{"1min":1.6100000000000001,"5min":2.2829999999999999,"15min":7.5570000000000004},"last":1.339}]},{"osd":4,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.417,"15min":1.4590000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.8680000000000001,"5min":2.4119999999999999,"15min":7.6189999999999998},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.4650000000000001,"15min":1.446},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.86599999999999999},"max":{"1min":1.8959999999999999,"5min":3.423,"15min":3.423},"last":1.0289999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.4019999999999999,"15min":1.306},"min":{"1min":0.874,"5min":0.754,"15min":0.61599999999999999},"max":{"1min":1.8049999999999999,"5min":10.507,"15min":10.507},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.46,"15min":1.5169999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81399999999999995},"max":{"1min":1.9319999999999999,"5min":2.536,"15min":7.657},"last":1.4570000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.482,"15min":1.4970000000000001},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.74299999999999999},"max":{"1min":1.891,"5min":2.8780000000000001,"15min":3.5299999999999998},"last":1.085},{"interface":"front","average":{"1min":1.379,"5min":1.5449999999999999,"15min":1.607},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.77600000000000002},"max":{"1min":1.895,"5min":10.561,"15min":37.771999999999998},"last":0.997}]},{"osd":7,"last update":"Wed Oct 18 09:04:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.5740000000000001,"15min":1.5589999999999999},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.83199999999999996},"max":{"1min":1.732,"5min":2.7890000000000001,"15min":3.3620000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.607,"15min":1.5129999999999999},"min":{"1min":0.98299999999999998,"5min":0.90200000000000002,"15min":0.77600000000000002},"max":{"1min":1.831,"5min":10.956,"15min":10.956},"last":1.0589999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":886,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194425,"statfs":{"total":200000000,"available":199092021,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.94999999999999996,"15min":0.85499999999999998},"min":{"1min":0.37,"5min":0.34499999999999997,"15min":0.315},"max":{"1min":1.6519999999999999,"5min":1.659,"15min":1.659},"last":0.378},{"interface":"front","average":{"1min":1.147,"5min":1.1060000000000001,"15min":0.97499999999999998},"min":{"1min":0.72799999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.681,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.63}]},{"osd":2,"last update":"Wed Oct 18 09:04:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.046,"15min":0.96999999999999997},"min":{"1min":0.443,"5min":0.317,"15min":0.317},"max":{"1min":1.4430000000000001,"5min":1.6899999999999999,"15min":1.859},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.997,"5min":0.98899999999999999,"15min":0.89800000000000002},"min":{"1min":0.58099999999999996,"5min":0.38600000000000001,"15min":0.36699999999999999},"max":{"1min":1.3660000000000001,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.60299999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:04:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.123,"15min":1.0509999999999999},"min":{"1min":0.63300000000000001,"5min":0.46899999999999997,"15min":0.436},"max":{"1min":1.49,"5min":1.915,"15min":2.1059999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.0920000000000001,"15min":1.0649999999999999},"min":{"1min":0.65700000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.736,"5min":1.9370000000000001,"15min":3.7010000000000001},"last":0.56399999999999995}]},{"osd":4,"last update":"Wed Oct 18 09:04:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.331,"15min":1.2749999999999999},"min":{"1min":0.88200000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7070000000000001,"5min":1.909,"15min":2.0489999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3680000000000001,"15min":1.266},"min":{"1min":1.048,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.609,"5min":1.893,"15min":2.0470000000000002},"last":0.83499999999999996}]},{"osd":5,"last update":"Wed Oct 18 09:04:25 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.4299999999999999,"15min":1.3380000000000001},"min":{"1min":1.0009999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.857,"5min":2.161,"15min":3.5750000000000002},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.492,"15min":1.403},"min":{"1min":0.76500000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.3769999999999998},"last":0.86399999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:04:25 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.4490000000000001,"15min":1.3380000000000001},"min":{"1min":0.80000000000000004,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.9890000000000001,"5min":2.0419999999999998,"15min":2.2050000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.407,"5min":1.4279999999999999,"15min":1.3260000000000001},"min":{"1min":1.0009999999999999,"5min":0.73399999999999999,"15min":0.63100000000000001},"max":{"1min":1.7709999999999999,"5min":2.1040000000000001,"15min":2.121},"last":1.0589999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:04:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5,"15min":1.4079999999999999},"min":{"1min":0.875,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.893,"5min":2.2000000000000002,"15min":2.2229999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.474,"5min":1.5149999999999999,"15min":1.4119999999999999},"min":{"1min":0.93799999999999994,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.1269999999999998,"5min":2.2469999999999999,"15min":2.4180000000000001},"last":0.81100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1463,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193849,"statfs":{"total":200000000,"available":198501653,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.83499999999999996,"15min":0.88600000000000001},"min":{"1min":0.45600000000000002,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":1.2430000000000001,"5min":1.3320000000000001,"15min":3.4420000000000002},"last":0.30599999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.1180000000000001,"15min":1.087},"min":{"1min":0.61699999999999999,"5min":0.56200000000000006,"15min":0.46500000000000002},"max":{"1min":1.6950000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.46300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.482,"5min":1.679,"15min":1.679},"last":0.61599999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.141,"15min":1.127},"min":{"1min":0.67800000000000005,"5min":0.503,"15min":0.437},"max":{"1min":1.5629999999999999,"5min":1.929,"15min":1.929},"last":0.754}]},{"osd":3,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.129,"15min":1.22},"min":{"1min":0.622,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.857,"5min":1.98,"15min":2.7639999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.391,"15min":1.46},"min":{"1min":0.79500000000000004,"5min":0.61299999999999999,"15min":0.58399999999999996},"max":{"1min":2.1200000000000001,"5min":2.3639999999999999,"15min":3.5720000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2350000000000001,"15min":1.2689999999999999},"min":{"1min":0.876,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":4.492},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.292,"5min":1.2709999999999999,"15min":1.262},"min":{"1min":0.873,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.716,"5min":1.927,"15min":1.927},"last":0.82999999999999996}]},{"osd":5,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.3029999999999999,"15min":1.367},"min":{"1min":0.875,"5min":0.79100000000000004,"15min":0.64300000000000002},"max":{"1min":2.1779999999999999,"5min":2.4510000000000001,"15min":4.2779999999999996},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.323,"15min":1.3520000000000001},"min":{"1min":0.84699999999999998,"5min":0.74399999999999999,"15min":0.73099999999999998},"max":{"1min":1.8089999999999999,"5min":1.847,"15min":1.8740000000000001},"last":0.875}]},{"osd":6,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4219999999999999,"15min":1.524},"min":{"1min":0.93700000000000006,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":3.5299999999999998},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.327,"5min":1.3600000000000001,"15min":1.444},"min":{"1min":1.0289999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.7909999999999999,"5min":3.1440000000000001,"15min":4.4349999999999996},"last":1.032}]},{"osd":7,"last update":"Wed Oct 18 09:04:24 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.542,"15min":1.601},"min":{"1min":1.0289999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.2629999999999999,"5min":2.2629999999999999,"15min":3.492},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.4570000000000001,"15min":1.591},"min":{"1min":0.99299999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.3090000000000002,"5min":3.2570000000000001,"15min":4.5419999999999998},"last":0.98699999999999999}]}]}],"pool_statfs":[]}} 2023-10-18T09:04:42.523 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T09:04:42.524 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T09:04:42.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.803+0000 7fc70ca52640 1 -- 172.21.15.40:0/3456518564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70806d5d0 msgr2=0x7fc7080c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:42.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.803+0000 7fc70ca52640 1 --2- 172.21.15.40:0/3456518564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70806d5d0 0x7fc7080c9620 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fc7080b2620 tx=0x7fc6f802bcf0 comp rx=0 tx=0).stop 2023-10-18T09:04:42.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.803+0000 7fc70ca52640 1 -- 172.21.15.40:0/3456518564 shutdown_connections 2023-10-18T09:04:42.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.803+0000 7fc70ca52640 1 --2- 172.21.15.40:0/3456518564 >> v2:172.21.15.144:3300/0 conn(0x7fc708071270 0x7fc708157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:42.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.803+0000 7fc70ca52640 1 --2- 172.21.15.40:0/3456518564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70806d5d0 0x7fc7080c9620 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:42.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.803+0000 7fc70ca52640 1 -- 172.21.15.40:0/3456518564 >> 172.21.15.40:0/3456518564 conn(0x7fc70806ca10 msgr2=0x7fc70806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:42.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.803+0000 7fc70ca52640 1 -- 172.21.15.40:0/3456518564 shutdown_connections 2023-10-18T09:04:42.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.803+0000 7fc70ca52640 1 -- 172.21.15.40:0/3456518564 wait complete. 2023-10-18T09:04:42.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.805+0000 7fc70ca52640 1 Processor -- start 2023-10-18T09:04:42.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.805+0000 7fc70ca52640 1 -- start start 2023-10-18T09:04:42.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.806+0000 7fc70ca52640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70806d5d0 0x7fc708150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:42.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.806+0000 7fc70ca52640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc708071270 0x7fc708155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:42.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.806+0000 7fc70ca52640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc70815af70 0x7fc7080b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:42.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.806+0000 7fc70ca52640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081f1d80 con 0x7fc70806d5d0 2023-10-18T09:04:42.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.806+0000 7fc70ca52640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081f5830 con 0x7fc708071270 2023-10-18T09:04:42.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.806+0000 7fc70ca52640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7081f5b10 con 0x7fc70815af70 2023-10-18T09:04:42.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.806+0000 7fc7077fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70806d5d0 0x7fc708150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:42.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.806+0000 7fc7077fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70806d5d0 0x7fc708150d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43570/0 (socket says 172.21.15.40:43570) 2023-10-18T09:04:42.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.806+0000 7fc7077fe640 1 -- 172.21.15.40:0/1911805363 learned_addr learned my addr 172.21.15.40:0/1911805363 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:42.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.807+0000 7fc6fffff640 1 --2- 172.21.15.40:0/1911805363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc708071270 0x7fc708155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:42.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.807+0000 7fc7077fe640 1 -- 172.21.15.40:0/1911805363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc70815af70 msgr2=0x7fc7080b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:42.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.807+0000 7fc7077fe640 1 --2- 172.21.15.40:0/1911805363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc70815af70 0x7fc7080b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.807+0000 7fc7077fe640 1 -- 172.21.15.40:0/1911805363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc708071270 msgr2=0x7fc708155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.807+0000 7fc7077fe640 1 --2- 172.21.15.40:0/1911805363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc708071270 0x7fc708155510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.807+0000 7fc7077fe640 1 -- 172.21.15.40:0/1911805363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f8009490 con 0x7fc70806d5d0 2023-10-18T09:04:42.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.807+0000 7fc7077fe640 1 --2- 172.21.15.40:0/1911805363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70806d5d0 0x7fc708150d60 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009780 tx=0x7fc6f80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:42.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.807+0000 7fc704ff9640 1 -- 172.21.15.40:0/1911805363 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8003ea0 con 0x7fc70806d5d0 2023-10-18T09:04:42.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.807+0000 7fc704ff9640 1 -- 172.21.15.40:0/1911805363 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc6f8005020 con 0x7fc70806d5d0 2023-10-18T09:04:42.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.807+0000 7fc704ff9640 1 -- 172.21.15.40:0/1911805363 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f80053f0 con 0x7fc70806d5d0 2023-10-18T09:04:42.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.807+0000 7fc70ca52640 1 -- 172.21.15.40:0/1911805363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7081f5df0 con 0x7fc70806d5d0 2023-10-18T09:04:42.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.808+0000 7fc70ca52640 1 -- 172.21.15.40:0/1911805363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc708155a50 con 0x7fc70806d5d0 2023-10-18T09:04:42.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.809+0000 7fc70ca52640 1 -- 172.21.15.40:0/1911805363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc708066e00 con 0x7fc70806d5d0 2023-10-18T09:04:42.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.810+0000 7fc704ff9640 1 -- 172.21.15.40:0/1911805363 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc6f80047a0 con 0x7fc70806d5d0 2023-10-18T09:04:42.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.811+0000 7fc704ff9640 1 --2- 172.21.15.40:0/1911805363 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc6e806ae40 0x7fc6e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:42.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.811+0000 7fc704ff9640 1 -- 172.21.15.40:0/1911805363 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6f80abae0 con 0x7fc70806d5d0 2023-10-18T09:04:42.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.812+0000 7fc6fffff640 1 --2- 172.21.15.40:0/1911805363 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc6e806ae40 0x7fc6e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:42.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.813+0000 7fc6fffff640 1 --2- 172.21.15.40:0/1911805363 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc6e806ae40 0x7fc6e806d300 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0001270 tx=0x7fc6f0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:42.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:42.815+0000 7fc704ff9640 1 -- 172.21.15.40:0/1911805363 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc6f8076b80 con 0x7fc70806d5d0 2023-10-18T09:04:43.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.078+0000 7fc70ca52640 1 -- 172.21.15.40:0/1911805363 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc7080c9190 con 0x7fc70806d5d0 2023-10-18T09:04:43.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.082+0000 7fc704ff9640 1 -- 172.21.15.40:0/1911805363 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc6f80420b0 con 0x7fc70806d5d0 2023-10-18T09:04:43.086 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2023-10-18T09:04:43.086 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w== 2023-10-18T09:04:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2023-10-18T09:04:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2023-10-18T09:04:43.087 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T09:04:43.087 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2023-10-18T09:04:43.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.086+0000 7fc70ca52640 1 -- 172.21.15.40:0/1911805363 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc6e806ae40 msgr2=0x7fc6e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:43.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.086+0000 7fc70ca52640 1 --2- 172.21.15.40:0/1911805363 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc6e806ae40 0x7fc6e806d300 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0001270 tx=0x7fc6f0009040 comp rx=0 tx=0).stop 2023-10-18T09:04:43.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.086+0000 7fc70ca52640 1 -- 172.21.15.40:0/1911805363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70806d5d0 msgr2=0x7fc708150d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:43.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.086+0000 7fc70ca52640 1 --2- 172.21.15.40:0/1911805363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70806d5d0 0x7fc708150d60 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009780 tx=0x7fc6f80038a0 comp rx=0 tx=0).stop 2023-10-18T09:04:43.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.086+0000 7fc70ca52640 1 -- 172.21.15.40:0/1911805363 shutdown_connections 2023-10-18T09:04:43.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.086+0000 7fc70ca52640 1 --2- 172.21.15.40:0/1911805363 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc6e806ae40 0x7fc6e806d300 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:43.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.086+0000 7fc70ca52640 1 --2- 172.21.15.40:0/1911805363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc70815af70 0x7fc7080b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:43.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.086+0000 7fc70ca52640 1 --2- 172.21.15.40:0/1911805363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc708071270 0x7fc708155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:43.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.086+0000 7fc70ca52640 1 --2- 172.21.15.40:0/1911805363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc70806d5d0 0x7fc708150d60 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:43.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.086+0000 7fc70ca52640 1 -- 172.21.15.40:0/1911805363 >> 172.21.15.40:0/1911805363 conn(0x7fc70806ca10 msgr2=0x7fc7080c8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:43.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.086+0000 7fc70ca52640 1 -- 172.21.15.40:0/1911805363 shutdown_connections 2023-10-18T09:04:43.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.086+0000 7fc70ca52640 1 -- 172.21.15.40:0/1911805363 wait complete. 2023-10-18T09:04:43.107 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:04:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.332+0000 7faa4ba75640 1 -- 172.21.15.40:0/3194177507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa4406dd90 msgr2=0x7faa4415be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.332+0000 7faa4ba75640 1 --2- 172.21.15.40:0/3194177507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa4406dd90 0x7faa4415be70 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7faa440b2c90 tx=0x7faa3c02c250 comp rx=0 tx=0).stop 2023-10-18T09:04:43.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.333+0000 7faa4ba75640 1 -- 172.21.15.40:0/3194177507 shutdown_connections 2023-10-18T09:04:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.333+0000 7faa4ba75640 1 --2- 172.21.15.40:0/3194177507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa4406dd90 0x7faa4415be70 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.333+0000 7faa4ba75640 1 --2- 172.21.15.40:0/3194177507 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa44158580 0x7faa4406d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.333+0000 7faa4ba75640 1 --2- 172.21.15.40:0/3194177507 >> v2:172.21.15.144:3300/0 conn(0x7faa441581a0 0x7faa440acc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.333+0000 7faa4ba75640 1 -- 172.21.15.40:0/3194177507 >> 172.21.15.40:0/3194177507 conn(0x7faa440af4c0 msgr2=0x7faa440af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.333+0000 7faa4ba75640 1 -- 172.21.15.40:0/3194177507 shutdown_connections 2023-10-18T09:04:43.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.333+0000 7faa4ba75640 1 -- 172.21.15.40:0/3194177507 wait complete. 2023-10-18T09:04:43.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.334+0000 7faa4ba75640 1 Processor -- start 2023-10-18T09:04:43.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.335+0000 7faa4ba75640 1 -- start start 2023-10-18T09:04:43.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.335+0000 7faa4ba75640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4406dd90 0x7faa4414fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:43.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.336+0000 7faa4ba75640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa441581a0 0x7faa440aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:43.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.336+0000 7faa4ba75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa44156940 0x7faa440addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:43.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.336+0000 7faa4ba75640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4414ff60 con 0x7faa44156940 2023-10-18T09:04:43.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.336+0000 7faa4ba75640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa44158970 con 0x7faa4406dd90 2023-10-18T09:04:43.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.336+0000 7faa4ba75640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa44156e70 con 0x7faa441581a0 2023-10-18T09:04:43.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.336+0000 7faa4a012640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa44156940 0x7faa440addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:43.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.336+0000 7faa4a012640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa44156940 0x7faa440addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43598/0 (socket says 172.21.15.40:43598) 2023-10-18T09:04:43.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.336+0000 7faa4a012640 1 -- 172.21.15.40:0/3894788785 learned_addr learned my addr 172.21.15.40:0/3894788785 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:43.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.336+0000 7faa49811640 1 --2- 172.21.15.40:0/3894788785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4406dd90 0x7faa4414fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:43.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.336+0000 7faa49010640 1 --2- 172.21.15.40:0/3894788785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa441581a0 0x7faa440aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:43.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.336+0000 7faa4a012640 1 -- 172.21.15.40:0/3894788785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa441581a0 msgr2=0x7faa440aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:43.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.336+0000 7faa4a012640 1 --2- 172.21.15.40:0/3894788785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa441581a0 0x7faa440aec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:43.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.336+0000 7faa4a012640 1 -- 172.21.15.40:0/3894788785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4406dd90 msgr2=0x7faa4414fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:43.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.336+0000 7faa4a012640 1 --2- 172.21.15.40:0/3894788785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4406dd90 0x7faa4414fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:43.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.336+0000 7faa4a012640 1 -- 172.21.15.40:0/3894788785 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c009900 con 0x7faa44156940 2023-10-18T09:04:43.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.337+0000 7faa4a012640 1 --2- 172.21.15.40:0/3894788785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa44156940 0x7faa440addd0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7faa3c037040 tx=0x7faa3c035db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:43.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.337+0000 7faa3a7fc640 1 -- 172.21.15.40:0/3894788785 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c034440 con 0x7faa44156940 2023-10-18T09:04:43.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.337+0000 7faa3a7fc640 1 -- 172.21.15.40:0/3894788785 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faa3c004250 con 0x7faa44156940 2023-10-18T09:04:43.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.337+0000 7faa3a7fc640 1 -- 172.21.15.40:0/3894788785 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c004640 con 0x7faa44156940 2023-10-18T09:04:43.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.337+0000 7faa4ba75640 1 -- 172.21.15.40:0/3894788785 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa44157150 con 0x7faa44156940 2023-10-18T09:04:43.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.337+0000 7faa4ba75640 1 -- 172.21.15.40:0/3894788785 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa44157710 con 0x7faa44156940 2023-10-18T09:04:43.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.338+0000 7faa4ba75640 1 -- 172.21.15.40:0/3894788785 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa440ce800 con 0x7faa44156940 2023-10-18T09:04:43.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.339+0000 7faa3a7fc640 1 -- 172.21.15.40:0/3894788785 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faa3c03b040 con 0x7faa44156940 2023-10-18T09:04:43.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.340+0000 7faa3a7fc640 1 --2- 172.21.15.40:0/3894788785 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa2806ae70 0x7faa2806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:43.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.340+0000 7faa3a7fc640 1 -- 172.21.15.40:0/3894788785 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa3c0ac210 con 0x7faa44156940 2023-10-18T09:04:43.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.341+0000 7faa49811640 1 --2- 172.21.15.40:0/3894788785 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa2806ae70 0x7faa2806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:43.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.342+0000 7faa49811640 1 --2- 172.21.15.40:0/3894788785 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa2806ae70 0x7faa2806d330 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7faa40005fd0 tx=0x7faa40005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:43.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.345+0000 7faa3a7fc640 1 -- 172.21.15.40:0/3894788785 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa3c077220 con 0x7faa44156940 2023-10-18T09:04:43.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.576+0000 7faa4ba75640 1 -- 172.21.15.40:0/3894788785 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa44158b30 con 0x7faa44156940 2023-10-18T09:04:43.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.577+0000 7faa3a7fc640 1 -- 172.21.15.40:0/3894788785 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4918 (secure 0 0 0) 0x7faa3c07a840 con 0x7faa44156940 2023-10-18T09:04:43.580 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:43.581 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3243865919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3243865919},{"type":"v1","addr":"172.21.15.40:6835","nonce":3243865919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6047,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/504515257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":504515257},{"type":"v1","addr":"172.21.15.144:6835","nonce":504515257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2513603652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2513603652},{"type":"v1","addr":"172.21.15.144:6837","nonce":2513603652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:04:36.950432+0000","modified":"2023-10-18T09:04:40.030449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8388":{"gid":8388,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/3348437493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3348437493},{"type":"v1","addr":"172.21.15.40:6833","nonce":3348437493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-18T09:04:43.581 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 74 2023-10-18T09:04:43.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.579+0000 7faa4ba75640 1 -- 172.21.15.40:0/3894788785 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa2806ae70 msgr2=0x7faa2806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:43.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.580+0000 7faa4ba75640 1 --2- 172.21.15.40:0/3894788785 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa2806ae70 0x7faa2806d330 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7faa40005fd0 tx=0x7faa40005850 comp rx=0 tx=0).stop 2023-10-18T09:04:43.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.580+0000 7faa4ba75640 1 -- 172.21.15.40:0/3894788785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa44156940 msgr2=0x7faa440addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:43.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.580+0000 7faa4ba75640 1 --2- 172.21.15.40:0/3894788785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa44156940 0x7faa440addd0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7faa3c037040 tx=0x7faa3c035db0 comp rx=0 tx=0).stop 2023-10-18T09:04:43.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.580+0000 7faa4ba75640 1 -- 172.21.15.40:0/3894788785 shutdown_connections 2023-10-18T09:04:43.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.580+0000 7faa4ba75640 1 --2- 172.21.15.40:0/3894788785 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faa2806ae70 0x7faa2806d330 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:43.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.580+0000 7faa4ba75640 1 --2- 172.21.15.40:0/3894788785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa44156940 0x7faa440addd0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:43.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.580+0000 7faa4ba75640 1 --2- 172.21.15.40:0/3894788785 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa441581a0 0x7faa440aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:43.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.580+0000 7faa4ba75640 1 --2- 172.21.15.40:0/3894788785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4406dd90 0x7faa4414fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:43.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.580+0000 7faa4ba75640 1 -- 172.21.15.40:0/3894788785 >> 172.21.15.40:0/3894788785 conn(0x7faa440af4c0 msgr2=0x7faa4406fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:43.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.580+0000 7faa4ba75640 1 -- 172.21.15.40:0/3894788785 shutdown_connections 2023-10-18T09:04:43.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:43.580+0000 7faa4ba75640 1 -- 172.21.15.40:0/3894788785 wait complete. 2023-10-18T09:04:44.006 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T09:04:36.950432+0000', 'modified': '2023-10-18T09:04:40.030449+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8388}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8388': {'gid': 8388, 'name': 'c', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6833/3348437493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 3348437493}, {'type': 'v1', 'addr': '172.21.15.40:6833', 'nonce': 3348437493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T09:04:44.007 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T09:04:44.007 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-18T09:04:44.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.258+0000 7f45e72f2640 1 -- 172.21.15.40:0/2913563774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45cc004d00 msgr2=0x7f45cc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:44.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.258+0000 7f45e72f2640 1 --2- 172.21.15.40:0/2913563774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45cc004d00 0x7f45cc005180 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f45d800c640 tx=0x7f45cc0057f0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.258+0000 7f45e588f640 1 -- 172.21.15.40:0/2913563774 reap_dead start 2023-10-18T09:04:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.258+0000 7f45e72f2640 1 -- 172.21.15.40:0/2913563774 shutdown_connections 2023-10-18T09:04:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.258+0000 7f45e72f2640 1 -- 172.21.15.40:0/2913563774 >> 172.21.15.40:0/2913563774 conn(0x7f45e00af530 msgr2=0x7f45e00af910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.258+0000 7f45e72f2640 1 -- 172.21.15.40:0/2913563774 shutdown_connections 2023-10-18T09:04:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.258+0000 7f45e72f2640 1 -- 172.21.15.40:0/2913563774 wait complete. 2023-10-18T09:04:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.260+0000 7f45e72f2640 1 Processor -- start 2023-10-18T09:04:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.260+0000 7f45e72f2640 1 -- start start 2023-10-18T09:04:44.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.261+0000 7f45e72f2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45cc004d00 0x7f45e015bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.261+0000 7f45e72f2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45e01566c0 0x7f45e00c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.261+0000 7f45e72f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45e00ac890 0x7f45e00acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.261+0000 7f45e72f2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e00ad250 con 0x7f45e00ac890 2023-10-18T09:04:44.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.261+0000 7f45e72f2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e00c2f10 con 0x7f45cc004d00 2023-10-18T09:04:44.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.261+0000 7f45e72f2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e015c360 con 0x7f45e01566c0 2023-10-18T09:04:44.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.262+0000 7f45e588f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45e00ac890 0x7f45e00acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.262+0000 7f45e588f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45e00ac890 0x7f45e00acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43618/0 (socket says 172.21.15.40:43618) 2023-10-18T09:04:44.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.262+0000 7f45e588f640 1 -- 172.21.15.40:0/2457160232 learned_addr learned my addr 172.21.15.40:0/2457160232 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:44.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.262+0000 7f45e488d640 1 --2- 172.21.15.40:0/2457160232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45e01566c0 0x7f45e00c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.262+0000 7f45e508e640 1 --2- 172.21.15.40:0/2457160232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45cc004d00 0x7f45e015bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.262+0000 7f45e588f640 1 -- 172.21.15.40:0/2457160232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45e01566c0 msgr2=0x7f45e00c29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:44.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.262+0000 7f45e588f640 1 --2- 172.21.15.40:0/2457160232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45e01566c0 0x7f45e00c29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.262+0000 7f45e588f640 1 -- 172.21.15.40:0/2457160232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45cc004d00 msgr2=0x7f45e015bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:44.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.262+0000 7f45e588f640 1 --2- 172.21.15.40:0/2457160232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45cc004d00 0x7f45e015bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.262+0000 7f45e588f640 1 -- 172.21.15.40:0/2457160232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e015c640 con 0x7f45e00ac890 2023-10-18T09:04:44.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.262+0000 7f45e488d640 1 --2- 172.21.15.40:0/2457160232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45e01566c0 0x7f45e00c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:44.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.262+0000 7f45e508e640 1 --2- 172.21.15.40:0/2457160232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45cc004d00 0x7f45e015bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.262+0000 7f45e588f640 1 --2- 172.21.15.40:0/2457160232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45e00ac890 0x7f45e00acd10 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f45d800c760 tx=0x7f45d8001730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.263+0000 7f45d5ffb640 1 -- 172.21.15.40:0/2457160232 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d80018b0 con 0x7f45e00ac890 2023-10-18T09:04:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.263+0000 7f45d5ffb640 1 -- 172.21.15.40:0/2457160232 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f45d8003970 con 0x7f45e00ac890 2023-10-18T09:04:44.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.263+0000 7f45d5ffb640 1 -- 172.21.15.40:0/2457160232 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8003d00 con 0x7f45e00ac890 2023-10-18T09:04:44.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.263+0000 7f45e72f2640 1 -- 172.21.15.40:0/2457160232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e015c920 con 0x7f45e00ac890 2023-10-18T09:04:44.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.263+0000 7f45e72f2640 1 -- 172.21.15.40:0/2457160232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e015d160 con 0x7f45e00ac890 2023-10-18T09:04:44.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.265+0000 7f45d5ffb640 1 -- 172.21.15.40:0/2457160232 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f45d8031030 con 0x7f45e00ac890 2023-10-18T09:04:44.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.266+0000 7f45d5ffb640 1 --2- 172.21.15.40:0/2457160232 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f45c406ae40 0x7f45c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.266+0000 7f45d5ffb640 1 -- 172.21.15.40:0/2457160232 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45d80aa2b0 con 0x7f45e00ac890 2023-10-18T09:04:44.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.267+0000 7f45e9b22740 1 Processor -- start 2023-10-18T09:04:44.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.267+0000 7f45e508e640 1 --2- 172.21.15.40:0/2457160232 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f45c406ae40 0x7f45c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.267+0000 7f45e9b22740 1 -- start start 2023-10-18T09:04:44.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.268+0000 7f45e9b22740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7a7d0 0x55d309d4da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.268+0000 7f45e9b22740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7abb0 0x55d309d7d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.268+0000 7f45e9b22740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d4df40 0x55d309d7daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.268+0000 7f45e9b22740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d309cc0740 con 0x55d309d4df40 2023-10-18T09:04:44.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.268+0000 7f45e9b22740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d309cbde30 con 0x55d309d7abb0 2023-10-18T09:04:44.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.268+0000 7f45e9b22740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d309cb9720 con 0x55d309d7a7d0 2023-10-18T09:04:44.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.268+0000 7f45e588f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d4df40 0x55d309d7daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.268+0000 7f45e588f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d4df40 0x55d309d7daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43632/0 (socket says 172.21.15.40:43632) 2023-10-18T09:04:44.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.268+0000 7f45e588f640 1 -- 172.21.15.40:0/280583710 learned_addr learned my addr 172.21.15.40:0/280583710 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:44.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.268+0000 7f45e508e640 1 --2- 172.21.15.40:0/2457160232 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f45c406ae40 0x7f45c406d300 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009b30 tx=0x7f45dc005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:44.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.269+0000 7f45e588f640 1 -- 172.21.15.40:0/280583710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7a7d0 msgr2=0x55d309d4da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:04:44.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.269+0000 7f45e588f640 1 --2- 172.21.15.40:0/280583710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7a7d0 0x55d309d4da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.269+0000 7f45e588f640 1 -- 172.21.15.40:0/280583710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7abb0 msgr2=0x55d309d7d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:04:44.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.269+0000 7f45e488d640 1 --2- 172.21.15.40:0/280583710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7abb0 0x55d309d7d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.269+0000 7f45e588f640 1 --2- 172.21.15.40:0/280583710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7abb0 0x55d309d7d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.269+0000 7f45e588f640 1 -- 172.21.15.40:0/280583710 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d309c7d910 con 0x55d309d4df40 2023-10-18T09:04:44.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.269+0000 7f45e588f640 1 --2- 172.21.15.40:0/280583710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d4df40 0x55d309d7daa0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f45d8078fa0 tx=0x7f45d8003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fbc8008623e226 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:44.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.269+0000 7f45c2ffd640 1 -- 172.21.15.40:0/280583710 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d807dea0 con 0x55d309d4df40 2023-10-18T09:04:44.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.269+0000 7f45c2ffd640 1 -- 172.21.15.40:0/280583710 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f45d80aa0c0 con 0x55d309d4df40 2023-10-18T09:04:44.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.269+0000 7f45c2ffd640 1 -- 172.21.15.40:0/280583710 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d807d160 con 0x55d309d4df40 2023-10-18T09:04:44.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.270+0000 7f45e9b22740 1 -- 172.21.15.40:0/280583710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d4df40 msgr2=0x55d309d7daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:44.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.270+0000 7f45e9b22740 1 --2- 172.21.15.40:0/280583710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d4df40 0x55d309d7daa0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f45d8078fa0 tx=0x7f45d8003e60 comp rx=0 tx=0).stop 2023-10-18T09:04:44.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.270+0000 7f45e9b22740 1 -- 172.21.15.40:0/280583710 shutdown_connections 2023-10-18T09:04:44.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.270+0000 7f45e9b22740 1 --2- 172.21.15.40:0/280583710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d4df40 0x55d309d7daa0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.270+0000 7f45e9b22740 1 --2- 172.21.15.40:0/280583710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7abb0 0x55d309d7d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.270+0000 7f45e9b22740 1 --2- 172.21.15.40:0/280583710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7a7d0 0x55d309d4da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.270+0000 7f45e9b22740 1 -- 172.21.15.40:0/280583710 >> 172.21.15.40:0/280583710 conn(0x55d309d4fc70 msgr2=0x55d309d98560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:44.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.270+0000 7f45e9b22740 1 -- 172.21.15.40:0/280583710 shutdown_connections 2023-10-18T09:04:44.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.270+0000 7f45e9b22740 1 -- 172.21.15.40:0/280583710 wait complete. 2023-10-18T09:04:44.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.271+0000 7f45e9b22740 1 -- start start 2023-10-18T09:04:44.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.271+0000 7f45e9b22740 1 Processor -- start 2023-10-18T09:04:44.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.272+0000 7f45e9b22740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7a7d0 0x55d309d6ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.272+0000 7f45e9b22740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7abb0 0x55d309d9b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.272+0000 7f45e9b22740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7f5a0 0x55d309d9bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.272+0000 7f45e9b22740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d309c57970 con 0x55d309d7a7d0 2023-10-18T09:04:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.272+0000 7f45e9b22740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d309c57610 con 0x55d309d7abb0 2023-10-18T09:04:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.272+0000 7f45e9b22740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d309a1df20 con 0x55d309d7f5a0 2023-10-18T09:04:44.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.272+0000 7f45e9b22740 10 client.0 _ll_register_callbacks cb 0x55d309c7b0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:04:44.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.273+0000 7f45e508e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7a7d0 0x55d309d6ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.273+0000 7f45e508e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7a7d0 0x55d309d6ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43636/0 (socket says 172.21.15.40:43636) 2023-10-18T09:04:44.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.273+0000 7f45e508e640 1 -- 172.21.15.40:0/136571021 learned_addr learned my addr 172.21.15.40:0/136571021 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:44.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.273+0000 7f45e588f640 1 --2- 172.21.15.40:0/136571021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7f5a0 0x55d309d9bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.273+0000 7f45e508e640 1 -- 172.21.15.40:0/136571021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7f5a0 msgr2=0x55d309d9bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:04:44.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.273+0000 7f45e508e640 1 --2- 172.21.15.40:0/136571021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7f5a0 0x55d309d9bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.273+0000 7f45e488d640 1 --2- 172.21.15.40:0/136571021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7abb0 0x55d309d9b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.273+0000 7f45e508e640 1 -- 172.21.15.40:0/136571021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7abb0 msgr2=0x55d309d9b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:04:44.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.273+0000 7f45e508e640 1 --2- 172.21.15.40:0/136571021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7abb0 0x55d309d9b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.273+0000 7f45e508e640 1 -- 172.21.15.40:0/136571021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e015d160 con 0x55d309d7a7d0 2023-10-18T09:04:44.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.273+0000 7f45e508e640 1 --2- 172.21.15.40:0/136571021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7a7d0 0x55d309d6ca70 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f45dc012930 tx=0x7f45dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb661463482a6d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:44.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.273+0000 7f45bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T09:04:44.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.274+0000 7f45bbfff640 1 -- 172.21.15.40:0/136571021 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc00fad0 con 0x55d309d7a7d0 2023-10-18T09:04:44.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.274+0000 7f45bbfff640 1 -- 172.21.15.40:0/136571021 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45dc018070 con 0x55d309d7a7d0 2023-10-18T09:04:44.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.274+0000 7f45e9b22740 1 -- 172.21.15.40:0/136571021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d309cb2700 con 0x55d309d7a7d0 2023-10-18T09:04:44.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.274+0000 7f45bbfff640 1 -- 172.21.15.40:0/136571021 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45dc00fc30 con 0x55d309d7a7d0 2023-10-18T09:04:44.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.274+0000 7f45e9b22740 10 client.8538 fetch_fsmap learned FSMap version 74 2023-10-18T09:04:44.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.274+0000 7f45e9b22740 1 -- 172.21.15.40:0/136571021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55d309cb7da0 con 0x55d309d7a7d0 2023-10-18T09:04:44.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.274+0000 7f45bbfff640 1 -- 172.21.15.40:0/136571021 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55d309cb2700 con 0x55d309d7a7d0 2023-10-18T09:04:44.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.274+0000 7f45bbfff640 1 -- 172.21.15.40:0/136571021 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f45dc025020 con 0x55d309d7a7d0 2023-10-18T09:04:44.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.275+0000 7f45e9b22740 10 client.8538 fetch_fsmap finished waiting for FSMap version 74 2023-10-18T09:04:44.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.275+0000 7f45e9b22740 10 client.8538 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T09:04:44.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.275+0000 7f45e9b22740 20 client.8538 populate_metadata read hostname 'smithi040' 2023-10-18T09:04:44.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.275+0000 7f45e9b22740 1 --2- 172.21.15.40:0/136571021 >> [v2:172.21.15.40:6832/3348437493,v1:172.21.15.40:6833/3348437493] conn(0x55d309d8f610 0x55d309db57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.275+0000 7f45e9b22740 4 client.8538 mds_command: new command op to 8388 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:04:44.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.275+0000 7f45e9b22740 1 -- 172.21.15.40:0/136571021 --> [v2:172.21.15.40:6832/3348437493,v1:172.21.15.40:6833/3348437493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d309cb2420 con 0x55d309d8f610 2023-10-18T09:04:44.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.275+0000 7f45e488d640 1 --2- 172.21.15.40:0/136571021 >> [v2:172.21.15.40:6832/3348437493,v1:172.21.15.40:6833/3348437493] conn(0x55d309d8f610 0x55d309db57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.276+0000 7f45e488d640 1 --2- 172.21.15.40:0/136571021 >> [v2:172.21.15.40:6832/3348437493,v1:172.21.15.40:6833/3348437493] conn(0x55d309d8f610 0x55d309db57a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f94260448476b31d server_cookie=de8444338125861f in_seq=0 out_seq=0 2023-10-18T09:04:44.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.276+0000 7f45bbfff640 10 client.8538 ms_handle_connect on v2:172.21.15.40:6832/3348437493 2023-10-18T09:04:44.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.276+0000 7f45bbfff640 1 -- 172.21.15.40:0/136571021 <== mds.0 v2:172.21.15.40:6832/3348437493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d309cb2420 con 0x55d309d8f610 2023-10-18T09:04:44.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.276+0000 7f45bbfff640 10 client.8538 handle_command_reply: tid=0 2023-10-18T09:04:44.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.277+0000 7f45e9b22740 1 client.8538 shutdown 2023-10-18T09:04:44.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.277+0000 7f45e9b22740 1 -- 172.21.15.40:0/136571021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7a7d0 msgr2=0x55d309d6ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:44.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.277+0000 7f45e9b22740 1 --2- 172.21.15.40:0/136571021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7a7d0 0x55d309d6ca70 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f45dc012930 tx=0x7f45dc004730 comp rx=0 tx=0).stop 2023-10-18T09:04:44.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.277+0000 7f45e9b22740 1 -- 172.21.15.40:0/136571021 shutdown_connections 2023-10-18T09:04:44.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.277+0000 7f45e9b22740 1 --2- 172.21.15.40:0/136571021 >> [v2:172.21.15.40:6832/3348437493,v1:172.21.15.40:6833/3348437493] conn(0x55d309d8f610 0x55d309db57a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.277+0000 7f45e9b22740 1 --2- 172.21.15.40:0/136571021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7f5a0 0x55d309d9bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.277+0000 7f45e9b22740 1 --2- 172.21.15.40:0/136571021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7abb0 0x55d309d9b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.277+0000 7f45e9b22740 1 --2- 172.21.15.40:0/136571021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7a7d0 0x55d309d6ca70 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.277+0000 7f45bbfff640 0 client.8538 ms_handle_reset on v2:172.21.15.40:6832/3348437493 2023-10-18T09:04:44.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.277+0000 7f45e9b22740 1 -- 172.21.15.40:0/136571021 >> 172.21.15.40:0/136571021 conn(0x55d309d4df40 msgr2=0x55d309d51650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:44.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.277+0000 7f45e9b22740 1 -- 172.21.15.40:0/136571021 shutdown_connections 2023-10-18T09:04:44.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.277+0000 7f45e9b22740 1 -- 172.21.15.40:0/136571021 wait complete. 2023-10-18T09:04:44.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.277+0000 7f45e9b22740 20 client.8538 trim_cache size 0 max 16384 2023-10-18T09:04:44.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.294+0000 7f45e9b22740 1 Processor -- start 2023-10-18T09:04:44.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.294+0000 7f45e9b22740 1 -- start start 2023-10-18T09:04:44.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.295+0000 7f45e9b22740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7a7d0 0x55d309db99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.295+0000 7f45e9b22740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7abb0 0x55d309d9a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.295+0000 7f45e9b22740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7f5a0 0x55d309d9aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.295+0000 7f45e9b22740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d309d79b30 con 0x55d309d7a7d0 2023-10-18T09:04:44.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.295+0000 7f45e9b22740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d309d8fbf0 con 0x55d309d7f5a0 2023-10-18T09:04:44.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.295+0000 7f45e9b22740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d309d7a420 con 0x55d309d7abb0 2023-10-18T09:04:44.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.295+0000 7f45e508e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7a7d0 0x55d309db99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.295+0000 7f45e508e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7a7d0 0x55d309db99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43652/0 (socket says 172.21.15.40:43652) 2023-10-18T09:04:44.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.295+0000 7f45e508e640 1 -- 172.21.15.40:0/3274400926 learned_addr learned my addr 172.21.15.40:0/3274400926 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:44.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.296+0000 7f45e508e640 1 -- 172.21.15.40:0/3274400926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7abb0 msgr2=0x55d309d9a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:04:44.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.296+0000 7f45e508e640 1 --2- 172.21.15.40:0/3274400926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7abb0 0x55d309d9a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.296+0000 7f45e508e640 1 -- 172.21.15.40:0/3274400926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7f5a0 msgr2=0x55d309d9aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:04:44.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.296+0000 7f45e508e640 1 --2- 172.21.15.40:0/3274400926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7f5a0 0x55d309d9aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.296+0000 7f45e508e640 1 -- 172.21.15.40:0/3274400926 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d309c56b80 con 0x55d309d7a7d0 2023-10-18T09:04:44.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.296+0000 7f45e508e640 1 --2- 172.21.15.40:0/3274400926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7a7d0 0x55d309db99b0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0098c0 tx=0x7f45dc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87df2077741edff5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.296+0000 7f45c37fe640 1 -- 172.21.15.40:0/3274400926 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc00f9a0 con 0x55d309d7a7d0 2023-10-18T09:04:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.296+0000 7f45c37fe640 1 -- 172.21.15.40:0/3274400926 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f45dc016070 con 0x55d309d7a7d0 2023-10-18T09:04:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.296+0000 7f45c37fe640 1 -- 172.21.15.40:0/3274400926 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc01f530 con 0x55d309d7a7d0 2023-10-18T09:04:44.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.297+0000 7f45e9b22740 1 -- 172.21.15.40:0/3274400926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7a7d0 msgr2=0x55d309db99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:44.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.297+0000 7f45e9b22740 1 --2- 172.21.15.40:0/3274400926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7a7d0 0x55d309db99b0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0098c0 tx=0x7f45dc016000 comp rx=0 tx=0).stop 2023-10-18T09:04:44.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.297+0000 7f45e9b22740 1 -- 172.21.15.40:0/3274400926 shutdown_connections 2023-10-18T09:04:44.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.297+0000 7f45e9b22740 1 --2- 172.21.15.40:0/3274400926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7f5a0 0x55d309d9aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.297+0000 7f45e9b22740 1 --2- 172.21.15.40:0/3274400926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7abb0 0x55d309d9a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.297+0000 7f45e9b22740 1 --2- 172.21.15.40:0/3274400926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7a7d0 0x55d309db99b0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.297+0000 7f45e9b22740 1 -- 172.21.15.40:0/3274400926 >> 172.21.15.40:0/3274400926 conn(0x55d309d4df40 msgr2=0x55d309d9b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:44.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.297+0000 7f45e9b22740 1 -- 172.21.15.40:0/3274400926 shutdown_connections 2023-10-18T09:04:44.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.297+0000 7f45e9b22740 1 -- 172.21.15.40:0/3274400926 wait complete. 2023-10-18T09:04:44.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.298+0000 7f45e9b22740 1 -- start start 2023-10-18T09:04:44.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.298+0000 7f45e9b22740 1 Processor -- start 2023-10-18T09:04:44.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.299+0000 7f45e9b22740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7a7d0 0x55d309d9af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.299+0000 7f45e9b22740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7abb0 0x55d309e00b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.299+0000 7f45e9b22740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7f5a0 0x55d309d8def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.299+0000 7f45e9b22740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d309d6b910 con 0x55d309d7f5a0 2023-10-18T09:04:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.299+0000 7f45e9b22740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d309d8ecf0 con 0x55d309d7a7d0 2023-10-18T09:04:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.299+0000 7f45e9b22740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d309d6b6c0 con 0x55d309d7abb0 2023-10-18T09:04:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.299+0000 7f45e9b22740 10 client.0 _ll_register_callbacks cb 0x55d309d80a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:04:44.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.299+0000 7f45e588f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7f5a0 0x55d309d8def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.299+0000 7f45e588f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7f5a0 0x55d309d8def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43658/0 (socket says 172.21.15.40:43658) 2023-10-18T09:04:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.299+0000 7f45e588f640 1 -- 172.21.15.40:0/3572390299 learned_addr learned my addr 172.21.15.40:0/3572390299 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.300+0000 7f45e488d640 1 --2- 172.21.15.40:0/3572390299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7abb0 0x55d309e00b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.300+0000 7f45e508e640 1 --2- 172.21.15.40:0/3572390299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7a7d0 0x55d309d9af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.300+0000 7f45e588f640 1 -- 172.21.15.40:0/3572390299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7abb0 msgr2=0x55d309e00b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:04:44.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.300+0000 7f45e588f640 1 --2- 172.21.15.40:0/3572390299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7abb0 0x55d309e00b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.300+0000 7f45e588f640 1 -- 172.21.15.40:0/3572390299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7a7d0 msgr2=0x55d309d9af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:04:44.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.300+0000 7f45e588f640 1 --2- 172.21.15.40:0/3572390299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7a7d0 0x55d309d9af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.300+0000 7f45e588f640 1 -- 172.21.15.40:0/3572390299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d309cb7da0 con 0x55d309d7f5a0 2023-10-18T09:04:44.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.300+0000 7f45e588f640 1 --2- 172.21.15.40:0/3572390299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7f5a0 0x55d309d8def0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f45d8078fa0 tx=0x7f45d8002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137e7bfadc019d34 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:44.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.300+0000 7f45bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T09:04:44.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.300+0000 7f45bbfff640 1 -- 172.21.15.40:0/3572390299 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d807dea0 con 0x55d309d7f5a0 2023-10-18T09:04:44.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.300+0000 7f45bbfff640 1 -- 172.21.15.40:0/3572390299 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45d800c2f0 con 0x55d309d7f5a0 2023-10-18T09:04:44.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.300+0000 7f45e9b22740 1 -- 172.21.15.40:0/3572390299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45d80aa0c0 con 0x55d309d7f5a0 2023-10-18T09:04:44.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.301+0000 7f45bbfff640 1 -- 172.21.15.40:0/3572390299 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45d807d160 con 0x55d309d7f5a0 2023-10-18T09:04:44.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.301+0000 7f45e9b22740 10 client.8544 fetch_fsmap learned FSMap version 74 2023-10-18T09:04:44.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.301+0000 7f45e9b22740 1 -- 172.21.15.40:0/3572390299 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55d309c7c170 con 0x55d309d7f5a0 2023-10-18T09:04:44.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.301+0000 7f45bbfff640 1 -- 172.21.15.40:0/3572390299 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f45d809b080 con 0x55d309d7f5a0 2023-10-18T09:04:44.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.301+0000 7f45bbfff640 1 -- 172.21.15.40:0/3572390299 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f45d807f0d0 con 0x55d309d7f5a0 2023-10-18T09:04:44.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.301+0000 7f45e9b22740 10 client.8544 fetch_fsmap finished waiting for FSMap version 74 2023-10-18T09:04:44.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.301+0000 7f45e9b22740 10 client.8544 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T09:04:44.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.301+0000 7f45e9b22740 20 client.8544 populate_metadata read hostname 'smithi040' 2023-10-18T09:04:44.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.301+0000 7f45e9b22740 1 --2- 172.21.15.40:0/3572390299 >> [v2:172.21.15.40:6832/3348437493,v1:172.21.15.40:6833/3348437493] conn(0x55d309d53090 0x55d309d53470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.301+0000 7f45e9b22740 4 client.8544 mds_command: new command op to 8388 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:04:44.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.301+0000 7f45e9b22740 1 -- 172.21.15.40:0/3572390299 --> [v2:172.21.15.40:6832/3348437493,v1:172.21.15.40:6833/3348437493] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d3099bb6c0 con 0x55d309d53090 2023-10-18T09:04:44.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.301+0000 7f45e508e640 1 --2- 172.21.15.40:0/3572390299 >> [v2:172.21.15.40:6832/3348437493,v1:172.21.15.40:6833/3348437493] conn(0x55d309d53090 0x55d309d53470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.302+0000 7f45e508e640 1 --2- 172.21.15.40:0/3572390299 >> [v2:172.21.15.40:6832/3348437493,v1:172.21.15.40:6833/3348437493] conn(0x55d309d53090 0x55d309d53470 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=948c7d15d4a0ce47 server_cookie=9879ca9fe5e6da32 in_seq=0 out_seq=0 2023-10-18T09:04:44.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.302+0000 7f45bbfff640 10 client.8544 ms_handle_connect on v2:172.21.15.40:6832/3348437493 2023-10-18T09:04:44.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.302+0000 7f45bbfff640 1 -- 172.21.15.40:0/3572390299 <== mds.0 v2:172.21.15.40:6832/3348437493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55d3099bb6c0 con 0x55d309d53090 2023-10-18T09:04:44.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.302+0000 7f45bbfff640 10 client.8544 handle_command_reply: tid=0 2023-10-18T09:04:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.302+0000 7f45e9b22740 1 client.8544 shutdown 2023-10-18T09:04:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.302+0000 7f45e9b22740 1 -- 172.21.15.40:0/3572390299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7f5a0 msgr2=0x55d309d8def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.302+0000 7f45e9b22740 1 --2- 172.21.15.40:0/3572390299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7f5a0 0x55d309d8def0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f45d8078fa0 tx=0x7f45d8002170 comp rx=0 tx=0).stop 2023-10-18T09:04:44.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.302+0000 7f45e9b22740 1 -- 172.21.15.40:0/3572390299 shutdown_connections 2023-10-18T09:04:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.302+0000 7f45e9b22740 1 --2- 172.21.15.40:0/3572390299 >> [v2:172.21.15.40:6832/3348437493,v1:172.21.15.40:6833/3348437493] conn(0x55d309d53090 0x55d309d53470 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.302+0000 7f45e9b22740 1 --2- 172.21.15.40:0/3572390299 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55d309d7f5a0 0x55d309d8def0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.302+0000 7f45e9b22740 1 --2- 172.21.15.40:0/3572390299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d309d7abb0 0x55d309e00b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.302+0000 7f45e9b22740 1 --2- 172.21.15.40:0/3572390299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d309d7a7d0 0x55d309d9af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.302+0000 7f45bbfff640 0 client.8544 ms_handle_reset on v2:172.21.15.40:6832/3348437493 2023-10-18T09:04:44.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.302+0000 7f45e9b22740 1 -- 172.21.15.40:0/3572390299 >> 172.21.15.40:0/3572390299 conn(0x55d309d4df40 msgr2=0x55d309d9b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.302+0000 7f45e9b22740 1 -- 172.21.15.40:0/3572390299 shutdown_connections 2023-10-18T09:04:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.302+0000 7f45e9b22740 1 -- 172.21.15.40:0/3572390299 wait complete. 2023-10-18T09:04:44.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.303+0000 7f45e9b22740 20 client.8544 trim_cache size 0 max 16384 2023-10-18T09:04:44.318 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:04:44.319 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:04:44.319 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-10-18T09:04:44.319 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 8388, 2023-10-18T09:04:44.319 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-10-18T09:04:44.320 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-10-18T09:04:44.320 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-10-18T09:04:44.320 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.8434679879999996, 2023-10-18T09:04:44.320 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 74, 2023-10-18T09:04:44.321 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 162, 2023-10-18T09:04:44.321 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 162, 2023-10-18T09:04:44.321 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 13.7215547 2023-10-18T09:04:44.321 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:04:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.303+0000 7f45e72f2640 1 -- 172.21.15.40:0/2457160232 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f45c406ae40 msgr2=0x7f45c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.303+0000 7f45e72f2640 1 --2- 172.21.15.40:0/2457160232 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f45c406ae40 0x7f45c406d300 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009b30 tx=0x7f45dc005f90 comp rx=0 tx=0).stop 2023-10-18T09:04:44.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.303+0000 7f45e72f2640 1 -- 172.21.15.40:0/2457160232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45e00ac890 msgr2=0x7f45e00acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.303+0000 7f45e72f2640 1 --2- 172.21.15.40:0/2457160232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45e00ac890 0x7f45e00acd10 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f45d800c760 tx=0x7f45d8001730 comp rx=0 tx=0).stop 2023-10-18T09:04:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.303+0000 7f45e72f2640 1 -- 172.21.15.40:0/2457160232 shutdown_connections 2023-10-18T09:04:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.303+0000 7f45e72f2640 1 --2- 172.21.15.40:0/2457160232 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f45c406ae40 0x7f45c406d300 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.303+0000 7f45e72f2640 1 --2- 172.21.15.40:0/2457160232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45e00ac890 0x7f45e00acd10 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.303+0000 7f45e72f2640 1 --2- 172.21.15.40:0/2457160232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45e01566c0 0x7f45e00c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.303+0000 7f45e72f2640 1 --2- 172.21.15.40:0/2457160232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45cc004d00 0x7f45e015bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.303+0000 7f45e72f2640 1 -- 172.21.15.40:0/2457160232 >> 172.21.15.40:0/2457160232 conn(0x7f45e00af530 msgr2=0x7f45e0158e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.304+0000 7f45e72f2640 1 -- 172.21.15.40:0/2457160232 shutdown_connections 2023-10-18T09:04:44.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.304+0000 7f45e72f2640 1 -- 172.21.15.40:0/2457160232 wait complete. 2023-10-18T09:04:44.325 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T09:04:44.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.546+0000 7ff332184640 1 -- 172.21.15.40:0/3707644017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff32c071320 msgr2=0x7ff32c0aed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.546+0000 7ff332184640 1 --2- 172.21.15.40:0/3707644017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff32c071320 0x7ff32c0aed70 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7ff32c06da50 tx=0x7ff31402bc80 comp rx=0 tx=0).stop 2023-10-18T09:04:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.546+0000 7ff332184640 1 -- 172.21.15.40:0/3707644017 shutdown_connections 2023-10-18T09:04:44.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.546+0000 7ff332184640 1 --2- 172.21.15.40:0/3707644017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff32c071320 0x7ff32c0aed70 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.546+0000 7ff332184640 1 --2- 172.21.15.40:0/3707644017 >> v2:172.21.15.144:3300/0 conn(0x7ff32c155c60 0x7ff32c070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.546+0000 7ff332184640 1 -- 172.21.15.40:0/3707644017 >> 172.21.15.40:0/3707644017 conn(0x7ff32c069720 msgr2=0x7ff32c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:44.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.546+0000 7ff332184640 1 -- 172.21.15.40:0/3707644017 shutdown_connections 2023-10-18T09:04:44.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.546+0000 7ff332184640 1 -- 172.21.15.40:0/3707644017 wait complete. 2023-10-18T09:04:44.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.548+0000 7ff332184640 1 Processor -- start 2023-10-18T09:04:44.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.548+0000 7ff332184640 1 -- start start 2023-10-18T09:04:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.549+0000 7ff332184640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff32c071320 0x7ff32c14eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.549+0000 7ff332184640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff32c0af2b0 0x7ff32c14c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.549+0000 7ff332184640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff32c155c60 0x7ff32c069b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.549+0000 7ff332184640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c14de30 con 0x7ff32c071320 2023-10-18T09:04:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.549+0000 7ff332184640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c14f0b0 con 0x7ff32c0af2b0 2023-10-18T09:04:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.549+0000 7ff332184640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c14d1b0 con 0x7ff32c155c60 2023-10-18T09:04:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.550+0000 7ff32b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff32c071320 0x7ff32c14eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.550+0000 7ff32b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff32c071320 0x7ff32c14eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43678/0 (socket says 172.21.15.40:43678) 2023-10-18T09:04:44.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.550+0000 7ff32b7fe640 1 -- 172.21.15.40:0/1012104194 learned_addr learned my addr 172.21.15.40:0/1012104194 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.550+0000 7ff323fff640 1 --2- 172.21.15.40:0/1012104194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff32c0af2b0 0x7ff32c14c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.550+0000 7ff32bfff640 1 --2- 172.21.15.40:0/1012104194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff32c155c60 0x7ff32c069b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.550+0000 7ff32b7fe640 1 -- 172.21.15.40:0/1012104194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff32c155c60 msgr2=0x7ff32c069b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.550+0000 7ff32b7fe640 1 --2- 172.21.15.40:0/1012104194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff32c155c60 0x7ff32c069b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.550+0000 7ff32b7fe640 1 -- 172.21.15.40:0/1012104194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff32c0af2b0 msgr2=0x7ff32c14c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.550+0000 7ff32b7fe640 1 --2- 172.21.15.40:0/1012104194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff32c0af2b0 0x7ff32c14c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.550+0000 7ff32b7fe640 1 -- 172.21.15.40:0/1012104194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff314009560 con 0x7ff32c071320 2023-10-18T09:04:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.550+0000 7ff323fff640 1 --2- 172.21.15.40:0/1012104194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff32c0af2b0 0x7ff32c14c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.550+0000 7ff32b7fe640 1 --2- 172.21.15.40:0/1012104194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff32c071320 0x7ff32c14eb70 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00d740 tx=0x7ff31c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:44.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.551+0000 7ff328ff9640 1 -- 172.21.15.40:0/1012104194 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c009c30 con 0x7ff32c071320 2023-10-18T09:04:44.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.551+0000 7ff328ff9640 1 -- 172.21.15.40:0/1012104194 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff31c009e80 con 0x7ff32c071320 2023-10-18T09:04:44.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.551+0000 7ff328ff9640 1 -- 172.21.15.40:0/1012104194 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c0106c0 con 0x7ff32c071320 2023-10-18T09:04:44.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.551+0000 7ff332184640 1 -- 172.21.15.40:0/1012104194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32c14d490 con 0x7ff32c071320 2023-10-18T09:04:44.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.551+0000 7ff332184640 1 -- 172.21.15.40:0/1012104194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32c14da50 con 0x7ff32c071320 2023-10-18T09:04:44.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.552+0000 7ff332184640 1 -- 172.21.15.40:0/1012104194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32c066fd0 con 0x7ff32c071320 2023-10-18T09:04:44.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.553+0000 7ff328ff9640 1 -- 172.21.15.40:0/1012104194 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff31c010820 con 0x7ff32c071320 2023-10-18T09:04:44.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.555+0000 7ff328ff9640 1 --2- 172.21.15.40:0/1012104194 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff30c06ae40 0x7ff30c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:44.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.555+0000 7ff328ff9640 1 -- 172.21.15.40:0/1012104194 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff31c089bb0 con 0x7ff32c071320 2023-10-18T09:04:44.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.555+0000 7ff323fff640 1 --2- 172.21.15.40:0/1012104194 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff30c06ae40 0x7ff30c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:44.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.556+0000 7ff323fff640 1 --2- 172.21.15.40:0/1012104194 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff30c06ae40 0x7ff30c06d300 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff314035000 tx=0x7ff314004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:44.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.559+0000 7ff328ff9640 1 -- 172.21.15.40:0/1012104194 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff31c054bc0 con 0x7ff32c071320 2023-10-18T09:04:44.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.792+0000 7ff332184640 1 -- 172.21.15.40:0/1012104194 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff32c0b1bd0 con 0x7ff32c071320 2023-10-18T09:04:44.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.793+0000 7ff328ff9640 1 -- 172.21.15.40:0/1012104194 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff31c021070 con 0x7ff32c071320 2023-10-18T09:04:44.797 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T09:04:44.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.795+0000 7ff332184640 1 -- 172.21.15.40:0/1012104194 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff30c06ae40 msgr2=0x7ff30c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:44.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.795+0000 7ff332184640 1 --2- 172.21.15.40:0/1012104194 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff30c06ae40 0x7ff30c06d300 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff314035000 tx=0x7ff314004730 comp rx=0 tx=0).stop 2023-10-18T09:04:44.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.795+0000 7ff332184640 1 -- 172.21.15.40:0/1012104194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff32c071320 msgr2=0x7ff32c14eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:44.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.795+0000 7ff332184640 1 --2- 172.21.15.40:0/1012104194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff32c071320 0x7ff32c14eb70 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00d740 tx=0x7ff31c00dc00 comp rx=0 tx=0).stop 2023-10-18T09:04:44.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.796+0000 7ff332184640 1 -- 172.21.15.40:0/1012104194 shutdown_connections 2023-10-18T09:04:44.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.796+0000 7ff332184640 1 --2- 172.21.15.40:0/1012104194 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff30c06ae40 0x7ff30c06d300 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.796+0000 7ff332184640 1 --2- 172.21.15.40:0/1012104194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff32c155c60 0x7ff32c069b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.796+0000 7ff332184640 1 --2- 172.21.15.40:0/1012104194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff32c0af2b0 0x7ff32c14c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.796+0000 7ff332184640 1 --2- 172.21.15.40:0/1012104194 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff32c071320 0x7ff32c14eb70 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:44.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.796+0000 7ff332184640 1 -- 172.21.15.40:0/1012104194 >> 172.21.15.40:0/1012104194 conn(0x7ff32c069720 msgr2=0x7ff32c15e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:44.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.796+0000 7ff332184640 1 -- 172.21.15.40:0/1012104194 shutdown_connections 2023-10-18T09:04:44.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:44.796+0000 7ff332184640 1 -- 172.21.15.40:0/1012104194 wait complete. 2023-10-18T09:04:44.819 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T09:04:44.819 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T09:04:44.819 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-10-18T09:04:44.820 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T09:04:44.820 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T09:04:44.820 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T09:04:44.821 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T09:04:44.821 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T09:04:44.821 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:04:44.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:44.822 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-10-18T09:04:44.884 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T09:04:44.884 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T09:04:44.884 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T09:04:44.884 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:04:44.884 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-18T09:04:44.885 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:04:44.885 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:04:44.885 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:04:44.885 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-10-18T09:04:44.885 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:04:44.886 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:04:44.886 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-10-18T09:04:44.886 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T09:04:44.886 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-10-18T09:04:44.886 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-10-18T09:04:44.887 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-18T09:04:44.887 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 41839sec preferred_lft 41839sec 2023-10-18T09:04:44.887 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2023-10-18T09:04:44.887 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:04:44.887 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:04:44.888 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-10-18T09:04:44.888 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-10-18T09:04:44.888 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T09:04:44.888 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:04:44.888 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T09:04:44.889 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:04:44.889 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2026:95ff:fea9:a463/64 scope link 2023-10-18T09:04:44.889 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:04:44.889 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:04:44.889 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T09:04:44.890 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::80bd:e5ff:fe0f:6f68/64 scope link 2023-10-18T09:04:44.890 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:04:44.890 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:04:44.890 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether ea:2b:21:f5:06:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T09:04:44.890 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::e82b:21ff:fef5:6af/64 scope link 2023-10-18T09:04:44.891 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:04:44.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:44.891 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T09:04:44.992 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-10-18T09:04:44.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:44.997 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-10-18T09:04:45.052 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-10-18T09:04:45.052 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T09:04:45.053 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-18T09:04:45.053 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-18T09:04:45.053 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T09:04:45.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:45.054 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T09:04:45.055 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T09:04:45.055 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-18T09:04:45.055 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-18T09:04:45.056 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-18T09:04:45.056 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T09:04:45.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:45.254 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-18T09:04:45.309 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T09:04:45.310 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T09:04:45.311 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:04:45.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:45.312 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-10-18T09:04:45.416 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:04:45.416 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T09:04:45.474 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:04:45.475 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:04:45.476 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T09:04:45.530 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-18T09:04:45.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:45.531 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:04:45.677 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:04:45.677 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:04:45.710 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:04:45.710 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:04:45.711 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T09:04:45.711 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T09:04:45.711 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T09:04:45.713 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-18T09:04:45.714 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2023-10-18T09:04:45.714 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T09:04:45.714 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T09:04:45.714 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T09:04:45.717 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T09:04:45.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:45.718 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T09:04:45.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:45.807 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:04:45.885 INFO:teuthology.orchestra.run.smithi040.stdout:client8553 2023-10-18T09:04:45.886 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:04:45.886 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:04:45.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T09:04:46.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.177+0000 7fb48bdce640 1 -- 172.21.15.40:0/3977136119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb484151cb0 msgr2=0x7fb484158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:46.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.177+0000 7fb48bdce640 1 --2- 172.21.15.40:0/3977136119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb484151cb0 0x7fb484158ca0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fb4840ce070 tx=0x7fb48002bd70 comp rx=0 tx=0).stop 2023-10-18T09:04:46.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.178+0000 7fb48bdce640 1 -- 172.21.15.40:0/3977136119 shutdown_connections 2023-10-18T09:04:46.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.178+0000 7fb48bdce640 1 --2- 172.21.15.40:0/3977136119 >> v2:172.21.15.144:3300/0 conn(0x7fb484158310 0x7fb484070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:46.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.178+0000 7fb48bdce640 1 --2- 172.21.15.40:0/3977136119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb484151cb0 0x7fb484158ca0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:46.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.178+0000 7fb48bdce640 1 -- 172.21.15.40:0/3977136119 >> 172.21.15.40:0/3977136119 conn(0x7fb484069720 msgr2=0x7fb484069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:46.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.178+0000 7fb48bdce640 1 -- 172.21.15.40:0/3977136119 shutdown_connections 2023-10-18T09:04:46.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.178+0000 7fb48bdce640 1 -- 172.21.15.40:0/3977136119 wait complete. 2023-10-18T09:04:46.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.179+0000 7fb48bdce640 1 Processor -- start 2023-10-18T09:04:46.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.180+0000 7fb48bdce640 1 -- start start 2023-10-18T09:04:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.181+0000 7fb48bdce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb48406fea0 0x7fb4841f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.181+0000 7fb48bdce640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb484151cb0 0x7fb484147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.181+0000 7fb48bdce640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb484158310 0x7fb48414b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.181+0000 7fb48bdce640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4841493d0 con 0x7fb48406fea0 2023-10-18T09:04:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.181+0000 7fb48bdce640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48414c010 con 0x7fb484158310 2023-10-18T09:04:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.181+0000 7fb48bdce640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48414c2f0 con 0x7fb484151cb0 2023-10-18T09:04:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.181+0000 7fb489b6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb48406fea0 0x7fb4841f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.181+0000 7fb489b6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb48406fea0 0x7fb4841f06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33688/0 (socket says 172.21.15.40:33688) 2023-10-18T09:04:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.181+0000 7fb489b6a640 1 -- 172.21.15.40:0/1262887304 learned_addr learned my addr 172.21.15.40:0/1262887304 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:46.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.181+0000 7fb489b6a640 1 -- 172.21.15.40:0/1262887304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb484151cb0 msgr2=0x7fb484147c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:46.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.181+0000 7fb489b6a640 1 --2- 172.21.15.40:0/1262887304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb484151cb0 0x7fb484147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:46.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.181+0000 7fb489b6a640 1 -- 172.21.15.40:0/1262887304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb484158310 msgr2=0x7fb48414b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:46.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.181+0000 7fb489b6a640 1 --2- 172.21.15.40:0/1262887304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb484158310 0x7fb48414b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:46.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.181+0000 7fb489b6a640 1 -- 172.21.15.40:0/1262887304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb480009490 con 0x7fb48406fea0 2023-10-18T09:04:46.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.182+0000 7fb489b6a640 1 --2- 172.21.15.40:0/1262887304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb48406fea0 0x7fb4841f06c0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fb480000c00 tx=0x7fb4800039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:46.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.182+0000 7fb47a7fc640 1 -- 172.21.15.40:0/1262887304 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480034440 con 0x7fb48406fea0 2023-10-18T09:04:46.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.182+0000 7fb47a7fc640 1 -- 172.21.15.40:0/1262887304 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb4800346b0 con 0x7fb48406fea0 2023-10-18T09:04:46.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.182+0000 7fb47a7fc640 1 -- 172.21.15.40:0/1262887304 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480034ac0 con 0x7fb48406fea0 2023-10-18T09:04:46.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.182+0000 7fb48bdce640 1 -- 172.21.15.40:0/1262887304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48414c5d0 con 0x7fb48406fea0 2023-10-18T09:04:46.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.182+0000 7fb48bdce640 1 -- 172.21.15.40:0/1262887304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb484148150 con 0x7fb48406fea0 2023-10-18T09:04:46.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.183+0000 7fb48bdce640 1 -- 172.21.15.40:0/1262887304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb484149860 con 0x7fb48406fea0 2023-10-18T09:04:46.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.184+0000 7fb47a7fc640 1 -- 172.21.15.40:0/1262887304 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb48003b040 con 0x7fb48406fea0 2023-10-18T09:04:46.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.185+0000 7fb47a7fc640 1 --2- 172.21.15.40:0/1262887304 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb46806ae40 0x7fb46806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:46.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.185+0000 7fb47a7fc640 1 -- 172.21.15.40:0/1262887304 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4800abd90 con 0x7fb48406fea0 2023-10-18T09:04:46.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.186+0000 7fb489369640 1 --2- 172.21.15.40:0/1262887304 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb46806ae40 0x7fb46806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:46.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.187+0000 7fb489369640 1 --2- 172.21.15.40:0/1262887304 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb46806ae40 0x7fb46806d300 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb474000f60 tx=0x7fb474007a30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.190+0000 7fb47a7fc640 1 -- 172.21.15.40:0/1262887304 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb484149860 con 0x7fb48406fea0 2023-10-18T09:04:46.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.420+0000 7fb48bdce640 1 -- 172.21.15.40:0/1262887304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb484149860 con 0x7fb48406fea0 2023-10-18T09:04:46.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.421+0000 7fb47a7fc640 1 -- 172.21.15.40:0/1262887304 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb484149860 con 0x7fb48406fea0 2023-10-18T09:04:46.425 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T09:04:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.424+0000 7fb48bdce640 1 -- 172.21.15.40:0/1262887304 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb46806ae40 msgr2=0x7fb46806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.424+0000 7fb48bdce640 1 --2- 172.21.15.40:0/1262887304 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb46806ae40 0x7fb46806d300 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb474000f60 tx=0x7fb474007a30 comp rx=0 tx=0).stop 2023-10-18T09:04:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.424+0000 7fb48bdce640 1 -- 172.21.15.40:0/1262887304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb48406fea0 msgr2=0x7fb4841f06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.424+0000 7fb48bdce640 1 --2- 172.21.15.40:0/1262887304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb48406fea0 0x7fb4841f06c0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fb480000c00 tx=0x7fb4800039e0 comp rx=0 tx=0).stop 2023-10-18T09:04:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.424+0000 7fb48bdce640 1 -- 172.21.15.40:0/1262887304 shutdown_connections 2023-10-18T09:04:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.424+0000 7fb48bdce640 1 --2- 172.21.15.40:0/1262887304 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb46806ae40 0x7fb46806d300 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.424+0000 7fb48bdce640 1 --2- 172.21.15.40:0/1262887304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb484158310 0x7fb48414b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.424+0000 7fb48bdce640 1 --2- 172.21.15.40:0/1262887304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb484151cb0 0x7fb484147c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.424+0000 7fb48bdce640 1 --2- 172.21.15.40:0/1262887304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb48406fea0 0x7fb4841f06c0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.424+0000 7fb48bdce640 1 -- 172.21.15.40:0/1262887304 >> 172.21.15.40:0/1262887304 conn(0x7fb484069720 msgr2=0x7fb484148a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:46.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.424+0000 7fb48bdce640 1 -- 172.21.15.40:0/1262887304 shutdown_connections 2023-10-18T09:04:46.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:46.425+0000 7fb48bdce640 1 -- 172.21.15.40:0/1262887304 wait complete. 2023-10-18T09:04:46.448 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T09:04:46.448 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T09:04:46.449 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-10-18T09:04:46.449 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-18T09:04:46.449 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T09:04:46.450 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T09:04:46.450 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T09:04:46.450 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T09:04:46.451 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:04:46.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:46.451 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-10-18T09:04:46.513 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T09:04:46.514 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T09:04:46.514 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T09:04:46.514 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:04:46.514 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-18T09:04:46.515 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:04:46.515 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:04:46.515 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:04:46.515 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2023-10-18T09:04:46.515 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:04:46.515 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:04:46.516 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2023-10-18T09:04:46.516 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:04:46.516 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-10-18T09:04:46.517 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T09:04:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-10-18T09:04:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T09:04:46.517 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 41831sec preferred_lft 41831sec 2023-10-18T09:04:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2023-10-18T09:04:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:04:46.518 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T09:04:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff 2023-10-18T09:04:46.518 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T09:04:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:04:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1493:a0ff:fed2:2c1/64 scope link 2023-10-18T09:04:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:04:46.519 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:04:46.519 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 4a:6e:a9:f0:92:62 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T09:04:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::486e:a9ff:fef0:9262/64 scope link 2023-10-18T09:04:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:04:46.520 INFO:teuthology.orchestra.run.smithi144.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:04:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T09:04:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::345a:c5ff:fe54:de6e/64 scope link 2023-10-18T09:04:46.520 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:04:46.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:46.522 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T09:04:46.607 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-10-18T09:04:46.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:46.611 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-10-18T09:04:46.664 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-10-18T09:04:46.664 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T09:04:46.664 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T09:04:46.664 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T09:04:46.665 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T09:04:46.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:46.665 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T09:04:46.666 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T09:04:46.666 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T09:04:46.667 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T09:04:46.667 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T09:04:46.667 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T09:04:46.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:46.844 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-18T09:04:46.898 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T09:04:46.898 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-18T09:04:46.898 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:04:46.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:46.899 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2023-10-18T09:04:46.994 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:04:46.994 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-18T09:04:47.050 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:04:47.051 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:04:47.051 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-18T09:04:47.104 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-18T09:04:47.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:47.105 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:04:47.248 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:04:47.248 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:04:47.279 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:04:47.279 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:04:47.280 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T09:04:47.280 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with new device syntax: 1@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T09:04:47.280 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T09:04:47.281 INFO:teuthology.orchestra.run.smithi144.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-18T09:04:47.282 INFO:teuthology.orchestra.run.smithi144.stdout: You just mounted a file system that supports labels which does not 2023-10-18T09:04:47.282 INFO:teuthology.orchestra.run.smithi144.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T09:04:47.282 INFO:teuthology.orchestra.run.smithi144.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T09:04:47.282 INFO:teuthology.orchestra.run.smithi144.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T09:04:47.285 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T09:04:47.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:47.286 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-18T09:04:47.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:04:47.364 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:04:47.424 INFO:teuthology.orchestra.run.smithi144.stdout:client6095 2023-10-18T09:04:47.425 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:04:47.425 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:04:47.480 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:04:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.723+0000 7fc035dc4640 1 -- 172.21.15.40:0/635423038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc03006d6f0 msgr2=0x7fc0300b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.723+0000 7fc035dc4640 1 --2- 172.21.15.40:0/635423038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc03006d6f0 0x7fc0300b15b0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fc0300caf30 tx=0x7fc02402d0b0 comp rx=0 tx=0).stop 2023-10-18T09:04:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.723+0000 7fc035dc4640 1 -- 172.21.15.40:0/635423038 shutdown_connections 2023-10-18T09:04:47.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.723+0000 7fc035dc4640 1 --2- 172.21.15.40:0/635423038 >> v2:172.21.15.144:3300/0 conn(0x7fc0300ca0d0 0x7fc030157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:47.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.723+0000 7fc035dc4640 1 --2- 172.21.15.40:0/635423038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc03006d6f0 0x7fc0300b15b0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:47.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.723+0000 7fc035dc4640 1 -- 172.21.15.40:0/635423038 >> 172.21.15.40:0/635423038 conn(0x7fc030069720 msgr2=0x7fc030069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:47.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.723+0000 7fc035dc4640 1 -- 172.21.15.40:0/635423038 shutdown_connections 2023-10-18T09:04:47.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.723+0000 7fc035dc4640 1 -- 172.21.15.40:0/635423038 wait complete. 2023-10-18T09:04:47.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.725+0000 7fc035dc4640 1 Processor -- start 2023-10-18T09:04:47.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.725+0000 7fc035dc4640 1 -- start start 2023-10-18T09:04:47.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.726+0000 7fc035dc4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc03006d6f0 0x7fc0300af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:47.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.726+0000 7fc035dc4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0300b0630 0x7fc0301492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:47.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.726+0000 7fc035dc4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0300ca0d0 0x7fc0301f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:47.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.726+0000 7fc035dc4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0300af940 con 0x7fc0300ca0d0 2023-10-18T09:04:47.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.726+0000 7fc035dc4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc030149e30 con 0x7fc03006d6f0 2023-10-18T09:04:47.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.726+0000 7fc035dc4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03014fdb0 con 0x7fc0300b0630 2023-10-18T09:04:47.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.726+0000 7fc0355c3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0300ca0d0 0x7fc0301f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:47.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.727+0000 7fc0355c3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0300ca0d0 0x7fc0301f1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33692/0 (socket says 172.21.15.40:33692) 2023-10-18T09:04:47.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.727+0000 7fc0355c3640 1 -- 172.21.15.40:0/51662039 learned_addr learned my addr 172.21.15.40:0/51662039 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:47.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.727+0000 7fc034dc2640 1 --2- 172.21.15.40:0/51662039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc03006d6f0 0x7fc0300af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:47.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.727+0000 7fc02ffff640 1 --2- 172.21.15.40:0/51662039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0300b0630 0x7fc0301492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:47.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.727+0000 7fc0355c3640 1 -- 172.21.15.40:0/51662039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0300b0630 msgr2=0x7fc0301492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:47.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.727+0000 7fc0355c3640 1 --2- 172.21.15.40:0/51662039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0300b0630 0x7fc0301492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:47.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.727+0000 7fc0355c3640 1 -- 172.21.15.40:0/51662039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc03006d6f0 msgr2=0x7fc0300af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:47.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.727+0000 7fc0355c3640 1 --2- 172.21.15.40:0/51662039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc03006d6f0 0x7fc0300af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:47.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.727+0000 7fc0355c3640 1 -- 172.21.15.40:0/51662039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02400a5a0 con 0x7fc0300ca0d0 2023-10-18T09:04:47.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.727+0000 7fc0355c3640 1 --2- 172.21.15.40:0/51662039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0300ca0d0 0x7fc0301f1f80 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fc02800b970 tx=0x7fc02800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.728+0000 7fc02d7fa640 1 -- 172.21.15.40:0/51662039 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02800cb40 con 0x7fc0300ca0d0 2023-10-18T09:04:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.728+0000 7fc02d7fa640 1 -- 172.21.15.40:0/51662039 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc02800cdb0 con 0x7fc0300ca0d0 2023-10-18T09:04:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.728+0000 7fc02d7fa640 1 -- 172.21.15.40:0/51662039 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0280146a0 con 0x7fc0300ca0d0 2023-10-18T09:04:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.728+0000 7fc035dc4640 1 -- 172.21.15.40:0/51662039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc030150090 con 0x7fc0300ca0d0 2023-10-18T09:04:47.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.728+0000 7fc035dc4640 1 -- 172.21.15.40:0/51662039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc030150650 con 0x7fc0300ca0d0 2023-10-18T09:04:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.729+0000 7fc035dc4640 1 -- 172.21.15.40:0/51662039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc030150870 con 0x7fc0300ca0d0 2023-10-18T09:04:47.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.730+0000 7fc02d7fa640 1 -- 172.21.15.40:0/51662039 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc02801d3b0 con 0x7fc0300ca0d0 2023-10-18T09:04:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.732+0000 7fc02d7fa640 1 --2- 172.21.15.40:0/51662039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc01406ae40 0x7fc01406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.732+0000 7fc02d7fa640 1 -- 172.21.15.40:0/51662039 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc028089830 con 0x7fc0300ca0d0 2023-10-18T09:04:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.732+0000 7fc034dc2640 1 --2- 172.21.15.40:0/51662039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc01406ae40 0x7fc01406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:47.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.734+0000 7fc034dc2640 1 --2- 172.21.15.40:0/51662039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc01406ae40 0x7fc01406d300 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc024037000 tx=0x7fc024004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:47.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.736+0000 7fc02d7fa640 1 -- 172.21.15.40:0/51662039 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc0280548d0 con 0x7fc0300ca0d0 2023-10-18T09:04:47.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.968+0000 7fc035dc4640 1 -- 172.21.15.40:0/51662039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc030066e00 con 0x7fc0300ca0d0 2023-10-18T09:04:47.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.969+0000 7fc02d7fa640 1 -- 172.21.15.40:0/51662039 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fc028019090 con 0x7fc0300ca0d0 2023-10-18T09:04:47.973 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:04:47.973 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3243865919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3243865919},{"type":"v1","addr":"172.21.15.40:6835","nonce":3243865919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6047,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/504515257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":504515257},{"type":"v1","addr":"172.21.15.144:6835","nonce":504515257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2513603652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2513603652},{"type":"v1","addr":"172.21.15.144:6837","nonce":2513603652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:04:36.950432+0000","modified":"2023-10-18T09:04:40.030449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8388":{"gid":8388,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/3348437493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3348437493},{"type":"v1","addr":"172.21.15.40:6833","nonce":3348437493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-18T09:04:47.973 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 74 2023-10-18T09:04:47.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.972+0000 7fc035dc4640 1 -- 172.21.15.40:0/51662039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc01406ae40 msgr2=0x7fc01406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:47.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.972+0000 7fc035dc4640 1 --2- 172.21.15.40:0/51662039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc01406ae40 0x7fc01406d300 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc024037000 tx=0x7fc024004730 comp rx=0 tx=0).stop 2023-10-18T09:04:47.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.972+0000 7fc035dc4640 1 -- 172.21.15.40:0/51662039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0300ca0d0 msgr2=0x7fc0301f1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:47.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.972+0000 7fc035dc4640 1 --2- 172.21.15.40:0/51662039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0300ca0d0 0x7fc0301f1f80 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fc02800b970 tx=0x7fc02800be30 comp rx=0 tx=0).stop 2023-10-18T09:04:47.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.972+0000 7fc035dc4640 1 -- 172.21.15.40:0/51662039 shutdown_connections 2023-10-18T09:04:47.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.972+0000 7fc035dc4640 1 --2- 172.21.15.40:0/51662039 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc01406ae40 0x7fc01406d300 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:47.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.972+0000 7fc035dc4640 1 --2- 172.21.15.40:0/51662039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0300ca0d0 0x7fc0301f1f80 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:47.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.972+0000 7fc035dc4640 1 --2- 172.21.15.40:0/51662039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0300b0630 0x7fc0301492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:47.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.972+0000 7fc035dc4640 1 --2- 172.21.15.40:0/51662039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc03006d6f0 0x7fc0300af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:47.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.972+0000 7fc035dc4640 1 -- 172.21.15.40:0/51662039 >> 172.21.15.40:0/51662039 conn(0x7fc030069720 msgr2=0x7fc030154780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:47.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.973+0000 7fc035dc4640 1 -- 172.21.15.40:0/51662039 shutdown_connections 2023-10-18T09:04:47.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:47.973+0000 7fc035dc4640 1 -- 172.21.15.40:0/51662039 wait complete. 2023-10-18T09:04:47.996 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-18T09:04:48.237 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":8388,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.759498376,"mdsmap_epoch":74,"osdmap_epoch":162,"osdmap_epoch_barrier":162,"uptime":17.637595329} 2023-10-18T09:04:48.238 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 8388, "mdsmap_epoch": 74, "osdmap_epoch": 162, "osdmap_epoch_barrier": 162, "rank_uptime": 10.759498376, "state": "up:active", "uptime": 17.637595329, "want_state": "up:active", "whoami": 0 } 2023-10-18T09:04:48.238 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-18T09:04:48.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.470+0000 7f262a613640 1 -- 172.21.15.40:0/4193777797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2624158ec0 msgr2=0x7f26240d1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:48.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.470+0000 7f262a613640 1 --2- 172.21.15.40:0/4193777797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2624158ec0 0x7f26240d1030 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f262406c8e0 tx=0x7f262002bda0 comp rx=0 tx=0).stop 2023-10-18T09:04:48.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.471+0000 7f262a613640 1 -- 172.21.15.40:0/4193777797 shutdown_connections 2023-10-18T09:04:48.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.471+0000 7f262a613640 1 --2- 172.21.15.40:0/4193777797 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26240cdd80 0x7f26240ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:48.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.471+0000 7f262a613640 1 --2- 172.21.15.40:0/4193777797 >> v2:172.21.15.144:3300/0 conn(0x7f26240cfff0 0x7f26240d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:48.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.471+0000 7f262a613640 1 --2- 172.21.15.40:0/4193777797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2624158ec0 0x7f26240d1030 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:48.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.471+0000 7f262a613640 1 -- 172.21.15.40:0/4193777797 >> 172.21.15.40:0/4193777797 conn(0x7f262408aca0 msgr2=0x7f262409a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:48.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.471+0000 7f262a613640 1 -- 172.21.15.40:0/4193777797 shutdown_connections 2023-10-18T09:04:48.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.471+0000 7f262a613640 1 -- 172.21.15.40:0/4193777797 wait complete. 2023-10-18T09:04:48.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.473+0000 7f262a613640 1 Processor -- start 2023-10-18T09:04:48.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.473+0000 7f262a613640 1 -- start start 2023-10-18T09:04:48.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.474+0000 7f262a613640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26240cdd80 0x7f262414dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:48.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.474+0000 7f262a613640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26240cfff0 0x7f26240c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:48.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.474+0000 7f262a613640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2624158ec0 0x7f26240c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:48.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.474+0000 7f262a613640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26240c8070 con 0x7f26240cfff0 2023-10-18T09:04:48.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.474+0000 7f262a613640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262414be00 con 0x7f26240cdd80 2023-10-18T09:04:48.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.474+0000 7f262a613640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26240c7df0 con 0x7f2624158ec0 2023-10-18T09:04:48.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.474+0000 7f2628e10640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26240cfff0 0x7f26240c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:48.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.474+0000 7f2628e10640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26240cfff0 0x7f26240c7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33706/0 (socket says 172.21.15.40:33706) 2023-10-18T09:04:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.474+0000 7f2628e10640 1 -- 172.21.15.40:0/3208642008 learned_addr learned my addr 172.21.15.40:0/3208642008 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:04:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.474+0000 7f2629e12640 1 --2- 172.21.15.40:0/3208642008 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2624158ec0 0x7f26240c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.475+0000 7f2629611640 1 --2- 172.21.15.40:0/3208642008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26240cdd80 0x7f262414dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.475+0000 7f2628e10640 1 -- 172.21.15.40:0/3208642008 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2624158ec0 msgr2=0x7f26240c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.475+0000 7f2628e10640 1 --2- 172.21.15.40:0/3208642008 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2624158ec0 0x7f26240c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:48.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.475+0000 7f2628e10640 1 -- 172.21.15.40:0/3208642008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26240cdd80 msgr2=0x7f262414dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:48.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.475+0000 7f2628e10640 1 --2- 172.21.15.40:0/3208642008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26240cdd80 0x7f262414dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:48.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.475+0000 7f2628e10640 1 -- 172.21.15.40:0/3208642008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2620009560 con 0x7f26240cfff0 2023-10-18T09:04:48.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.475+0000 7f2629611640 1 --2- 172.21.15.40:0/3208642008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26240cdd80 0x7f262414dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:04:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.475+0000 7f2628e10640 1 --2- 172.21.15.40:0/3208642008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26240cfff0 0x7f26240c7370 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f261400d7a0 tx=0x7f261400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.475+0000 7f2619ffb640 1 -- 172.21.15.40:0/3208642008 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614009c30 con 0x7f26240cfff0 2023-10-18T09:04:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.475+0000 7f2619ffb640 1 -- 172.21.15.40:0/3208642008 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2614009e80 con 0x7f26240cfff0 2023-10-18T09:04:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.475+0000 7f2619ffb640 1 -- 172.21.15.40:0/3208642008 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614004260 con 0x7f26240cfff0 2023-10-18T09:04:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.475+0000 7f262a613640 1 -- 172.21.15.40:0/3208642008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262414b4d0 con 0x7f26240cfff0 2023-10-18T09:04:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.476+0000 7f262a613640 1 -- 172.21.15.40:0/3208642008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262414ba90 con 0x7f26240cfff0 2023-10-18T09:04:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.477+0000 7f262a613640 1 -- 172.21.15.40:0/3208642008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2624071da0 con 0x7f26240cfff0 2023-10-18T09:04:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.478+0000 7f2619ffb640 1 -- 172.21.15.40:0/3208642008 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f26140035f0 con 0x7f26240cfff0 2023-10-18T09:04:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.479+0000 7f2619ffb640 1 --2- 172.21.15.40:0/3208642008 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f260806ae40 0x7f260806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:04:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.479+0000 7f2619ffb640 1 -- 172.21.15.40:0/3208642008 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2614089770 con 0x7f26240cfff0 2023-10-18T09:04:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.480+0000 7f2629611640 1 --2- 172.21.15.40:0/3208642008 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f260806ae40 0x7f260806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:04:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.481+0000 7f2629611640 1 --2- 172.21.15.40:0/3208642008 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f260806ae40 0x7f260806d300 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2620035000 tx=0x7f2620004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:04:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.483+0000 7f2619ffb640 1 -- 172.21.15.40:0/3208642008 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2614054820 con 0x7f26240cfff0 2023-10-18T09:04:48.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:48.678+0000 7f262a613640 1 -- 172.21.15.40:0/3208642008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f2624066e00 con 0x7f26240cfff0 2023-10-18T09:04:49.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:49.160+0000 7f2619ffb640 1 -- 172.21.15.40:0/3208642008 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v163) v1 ==== 181+0+0 (secure 0 0 0) 0x7f26140200c0 con 0x7f26240cfff0 2023-10-18T09:04:49.164 INFO:teuthology.orchestra.run.smithi040.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-18T09:04:49.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:49.164+0000 7f262a613640 1 -- 172.21.15.40:0/3208642008 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f260806ae40 msgr2=0x7f260806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:49.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:49.165+0000 7f262a613640 1 --2- 172.21.15.40:0/3208642008 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f260806ae40 0x7f260806d300 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2620035000 tx=0x7f2620004730 comp rx=0 tx=0).stop 2023-10-18T09:04:49.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:49.165+0000 7f262a613640 1 -- 172.21.15.40:0/3208642008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26240cfff0 msgr2=0x7f26240c7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:04:49.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:49.165+0000 7f262a613640 1 --2- 172.21.15.40:0/3208642008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26240cfff0 0x7f26240c7370 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f261400d7a0 tx=0x7f261400dc60 comp rx=0 tx=0).stop 2023-10-18T09:04:49.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:49.165+0000 7f262a613640 1 -- 172.21.15.40:0/3208642008 shutdown_connections 2023-10-18T09:04:49.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:49.165+0000 7f262a613640 1 --2- 172.21.15.40:0/3208642008 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f260806ae40 0x7f260806d300 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:49.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:49.165+0000 7f262a613640 1 --2- 172.21.15.40:0/3208642008 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2624158ec0 0x7f26240c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:49.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:49.165+0000 7f262a613640 1 --2- 172.21.15.40:0/3208642008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26240cfff0 0x7f26240c7370 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:49.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:49.165+0000 7f262a613640 1 --2- 172.21.15.40:0/3208642008 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26240cdd80 0x7f262414dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:04:49.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:49.165+0000 7f262a613640 1 -- 172.21.15.40:0/3208642008 >> 172.21.15.40:0/3208642008 conn(0x7f262408aca0 msgr2=0x7f2624158520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:04:49.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:49.165+0000 7f262a613640 1 -- 172.21.15.40:0/3208642008 shutdown_connections 2023-10-18T09:04:49.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:04:49.165+0000 7f262a613640 1 -- 172.21.15.40:0/3208642008 wait complete. 2023-10-18T09:04:49.191 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-18T09:04:49.430 INFO:teuthology.orchestra.run.smithi040.stdout:{"osd_mon_report_interval":"5"} 2023-10-18T09:04:49.431 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-18T09:04:49.431 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-18T09:04:49.667 INFO:teuthology.orchestra.run.smithi040.stdout:{"mon_tick_interval":"5"} 2023-10-18T09:04:49.668 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-18T09:04:49.669 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2023-10-18T09:04:49.669 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2023-10-18T09:04:49.670 DEBUG:teuthology.orchestra.run.smithi040:> import time 2023-10-18T09:04:49.670 DEBUG:teuthology.orchestra.run.smithi040:> import datetime 2023-10-18T09:04:49.671 DEBUG:teuthology.orchestra.run.smithi040:> import subprocess 2023-10-18T09:04:49.671 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-10-18T09:04:49.671 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:04:49.672 DEBUG:teuthology.orchestra.run.smithi040:> # Write some buffered data through before going full, all should be well 2023-10-18T09:04:49.672 DEBUG:teuthology.orchestra.run.smithi040:> print("writing some data through which we expect to succeed") 2023-10-18T09:04:49.672 DEBUG:teuthology.orchestra.run.smithi040:> bytes = 0 2023-10-18T09:04:49.673 DEBUG:teuthology.orchestra.run.smithi040:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-18T09:04:49.673 DEBUG:teuthology.orchestra.run.smithi040:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-18T09:04:49.673 DEBUG:teuthology.orchestra.run.smithi040:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2023-10-18T09:04:49.674 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(f) 2023-10-18T09:04:49.674 DEBUG:teuthology.orchestra.run.smithi040:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-18T09:04:49.675 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:04:49.675 DEBUG:teuthology.orchestra.run.smithi040:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-18T09:04:49.675 DEBUG:teuthology.orchestra.run.smithi040:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-18T09:04:49.676 DEBUG:teuthology.orchestra.run.smithi040:> # buffered IO, the error should always only appear from fsync and not 2023-10-18T09:04:49.676 DEBUG:teuthology.orchestra.run.smithi040:> # from write 2023-10-18T09:04:49.676 DEBUG:teuthology.orchestra.run.smithi040:> full = False 2023-10-18T09:04:49.676 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:04:49.677 DEBUG:teuthology.orchestra.run.smithi040:> for n in range(0, int(32 * 0.9)): 2023-10-18T09:04:49.677 DEBUG:teuthology.orchestra.run.smithi040:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-18T09:04:49.677 DEBUG:teuthology.orchestra.run.smithi040:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2023-10-18T09:04:49.678 DEBUG:teuthology.orchestra.run.smithi040:> print("done writing {0} bytes".format(bytes)) 2023-10-18T09:04:49.678 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:04:49.679 DEBUG:teuthology.orchestra.run.smithi040:> # OK, now we should sneak in under the full condition 2023-10-18T09:04:49.679 DEBUG:teuthology.orchestra.run.smithi040:> # due to the time it takes the OSDs to report to the 2023-10-18T09:04:49.679 DEBUG:teuthology.orchestra.run.smithi040:> # mons, and get a successful fsync on our full-making data 2023-10-18T09:04:49.679 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(f) 2023-10-18T09:04:49.680 DEBUG:teuthology.orchestra.run.smithi040:> print("successfully fsync'"'"'ed prior to getting full state reported") 2023-10-18T09:04:49.680 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:04:49.681 DEBUG:teuthology.orchestra.run.smithi040:> # buffered write, add more dirty data to the buffer 2023-10-18T09:04:49.681 DEBUG:teuthology.orchestra.run.smithi040:> print("starting buffered write") 2023-10-18T09:04:49.681 DEBUG:teuthology.orchestra.run.smithi040:> try: 2023-10-18T09:04:49.682 DEBUG:teuthology.orchestra.run.smithi040:> for n in range(0, int(32 * 0.2)): 2023-10-18T09:04:49.682 DEBUG:teuthology.orchestra.run.smithi040:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-18T09:04:49.683 DEBUG:teuthology.orchestra.run.smithi040:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-18T09:04:49.683 DEBUG:teuthology.orchestra.run.smithi040:> time.sleep(15.0) 2023-10-18T09:04:49.683 DEBUG:teuthology.orchestra.run.smithi040:> except OSError: 2023-10-18T09:04:49.684 DEBUG:teuthology.orchestra.run.smithi040:> pass; 2023-10-18T09:04:49.684 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:04:49.684 DEBUG:teuthology.orchestra.run.smithi040:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2023-10-18T09:04:49.685 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:04:49.685 DEBUG:teuthology.orchestra.run.smithi040:> # Wait long enough for a background flush that should fail 2023-10-18T09:04:49.685 DEBUG:teuthology.orchestra.run.smithi040:> time.sleep(30) 2023-10-18T09:04:49.686 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:04:49.686 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2023-10-18T09:04:49.686 DEBUG:teuthology.orchestra.run.smithi040:> # ...and check that the failed background flush is reflected in fclose 2023-10-18T09:04:49.687 DEBUG:teuthology.orchestra.run.smithi040:> try: 2023-10-18T09:04:49.687 DEBUG:teuthology.orchestra.run.smithi040:> os.close(f) 2023-10-18T09:04:49.687 DEBUG:teuthology.orchestra.run.smithi040:> except OSError: 2023-10-18T09:04:49.688 DEBUG:teuthology.orchestra.run.smithi040:> print("close() returned an error as expected") 2023-10-18T09:04:49.688 DEBUG:teuthology.orchestra.run.smithi040:> else: 2023-10-18T09:04:49.689 DEBUG:teuthology.orchestra.run.smithi040:> raise RuntimeError("close() failed to raise error") 2023-10-18T09:04:49.689 DEBUG:teuthology.orchestra.run.smithi040:> else: 2023-10-18T09:04:49.689 DEBUG:teuthology.orchestra.run.smithi040:> # The kernel cephfs client does not raise errors on fclose 2023-10-18T09:04:49.690 DEBUG:teuthology.orchestra.run.smithi040:> os.close(f) 2023-10-18T09:04:49.690 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:04:49.691 DEBUG:teuthology.orchestra.run.smithi040:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-18T09:04:49.691 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-10-18T09:05:07.038 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:05:07.048 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:05:37.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:05:37.223 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:06:07.356 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:06:07.366 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:06:37.523 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:06:37.533 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:06:50.915 INFO:teuthology.orchestra.run.smithi040.stdout:writing some data through which we expect to succeed 2023-10-18T09:06:50.916 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-18T09:06:50.916 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 1572864 bytes via buffered write, may repeat 2023-10-18T09:06:50.916 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 2621440 bytes via buffered write, may repeat 2023-10-18T09:06:50.917 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 3670016 bytes via buffered write, may repeat 2023-10-18T09:06:50.917 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 4718592 bytes via buffered write, may repeat 2023-10-18T09:06:50.917 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 5767168 bytes via buffered write, may repeat 2023-10-18T09:06:50.917 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 6815744 bytes via buffered write, may repeat 2023-10-18T09:06:50.917 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 7864320 bytes via buffered write, may repeat 2023-10-18T09:06:50.918 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 8912896 bytes via buffered write, may repeat 2023-10-18T09:06:50.918 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 9961472 bytes via buffered write, may repeat 2023-10-18T09:06:50.918 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 11010048 bytes via buffered write, may repeat 2023-10-18T09:06:50.918 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 12058624 bytes via buffered write, may repeat 2023-10-18T09:06:50.918 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 13107200 bytes via buffered write, may repeat 2023-10-18T09:06:50.919 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 14155776 bytes via buffered write, may repeat 2023-10-18T09:06:50.919 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 15204352 bytes via buffered write, may repeat 2023-10-18T09:06:50.919 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 16252928 bytes via buffered write, may repeat 2023-10-18T09:06:50.919 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 17301504 bytes via buffered write, may repeat 2023-10-18T09:06:50.919 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 18350080 bytes via buffered write, may repeat 2023-10-18T09:06:50.920 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 19398656 bytes via buffered write, may repeat 2023-10-18T09:06:50.920 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 20447232 bytes via buffered write, may repeat 2023-10-18T09:06:50.920 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 21495808 bytes via buffered write, may repeat 2023-10-18T09:06:50.920 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 22544384 bytes via buffered write, may repeat 2023-10-18T09:06:50.920 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 23592960 bytes via buffered write, may repeat 2023-10-18T09:06:50.921 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 24641536 bytes via buffered write, may repeat 2023-10-18T09:06:50.921 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 25690112 bytes via buffered write, may repeat 2023-10-18T09:06:50.921 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 26738688 bytes via buffered write, may repeat 2023-10-18T09:06:50.921 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 27787264 bytes via buffered write, may repeat 2023-10-18T09:06:50.921 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 28835840 bytes via buffered write, may repeat 2023-10-18T09:06:50.921 INFO:teuthology.orchestra.run.smithi040.stdout:wrote 29884416 bytes via buffered write, may repeat 2023-10-18T09:06:50.922 INFO:teuthology.orchestra.run.smithi040.stdout:done writing 29884416 bytes 2023-10-18T09:06:50.922 INFO:teuthology.orchestra.run.smithi040.stdout:successfully fsync'ed prior to getting full state reported 2023-10-18T09:06:50.922 INFO:teuthology.orchestra.run.smithi040.stdout:starting buffered write 2023-10-18T09:06:50.922 INFO:teuthology.orchestra.run.smithi040.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-18T09:06:50.922 INFO:teuthology.orchestra.run.smithi040.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-18T09:06:50.923 INFO:teuthology.orchestra.run.smithi040.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-18T09:06:50.923 INFO:teuthology.orchestra.run.smithi040.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-18T09:06:50.923 INFO:teuthology.orchestra.run.smithi040.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-18T09:06:50.923 INFO:teuthology.orchestra.run.smithi040.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-18T09:06:50.923 INFO:teuthology.orchestra.run.smithi040.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2023-10-18T09:06:50.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T09:06:50.949 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T09:06:50.950 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:06:51.082 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:06:51.218 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T09:06:51.218 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:06:51.219 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:06:51.283 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:06:51.283 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:06:51.341 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:06:51.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.574+0000 7f2a07b9e640 1 -- 172.21.15.40:0/2313357838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00144840 msgr2=0x7f2a0015f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:51.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.575+0000 7f2a07b9e640 1 --2- 172.21.15.40:0/2313357838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00144840 0x7f2a0015f0d0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f29fc005fd0 tx=0x7f29fc02c1b0 comp rx=0 tx=0).stop 2023-10-18T09:06:51.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.575+0000 7f2a07b9e640 1 -- 172.21.15.40:0/2313357838 shutdown_connections 2023-10-18T09:06:51.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.575+0000 7f2a07b9e640 1 --2- 172.21.15.40:0/2313357838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a00069e30 0x7f2a000b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:51.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.575+0000 7f2a07b9e640 1 --2- 172.21.15.40:0/2313357838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00144840 0x7f2a0015f0d0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:51.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.575+0000 7f2a07b9e640 1 -- 172.21.15.40:0/2313357838 >> 172.21.15.40:0/2313357838 conn(0x7f2a000aff00 msgr2=0x7f2a000b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:51.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.575+0000 7f2a07b9e640 1 -- 172.21.15.40:0/2313357838 shutdown_connections 2023-10-18T09:06:51.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.575+0000 7f2a07b9e640 1 -- 172.21.15.40:0/2313357838 wait complete. 2023-10-18T09:06:51.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.577+0000 7f2a07b9e640 1 Processor -- start 2023-10-18T09:06:51.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.577+0000 7f2a07b9e640 1 -- start start 2023-10-18T09:06:51.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.578+0000 7f2a07b9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00069e30 0x7f2a0015b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:51.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.578+0000 7f2a07b9e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a0006d1c0 0x7f2a000cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:51.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.578+0000 7f2a07b9e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a00144840 0x7f2a000cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:51.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.578+0000 7f2a07b9e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0015b550 con 0x7f2a00069e30 2023-10-18T09:06:51.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.578+0000 7f2a07b9e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0013df60 con 0x7f2a0006d1c0 2023-10-18T09:06:51.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.578+0000 7f2a07b9e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0013d330 con 0x7f2a00144840 2023-10-18T09:06:51.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.578+0000 7f2a0593a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00069e30 0x7f2a0015b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:51.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.578+0000 7f2a0593a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00069e30 0x7f2a0015b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41986/0 (socket says 172.21.15.40:41986) 2023-10-18T09:06:51.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.578+0000 7f2a0593a640 1 -- 172.21.15.40:0/3198790509 learned_addr learned my addr 172.21.15.40:0/3198790509 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:51.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.578+0000 7f2a05139640 1 --2- 172.21.15.40:0/3198790509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a0006d1c0 0x7f2a000cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:51.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.578+0000 7f2a0613b640 1 --2- 172.21.15.40:0/3198790509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a00144840 0x7f2a000cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:51.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.579+0000 7f2a0593a640 1 -- 172.21.15.40:0/3198790509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a00144840 msgr2=0x7f2a000cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:51.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.579+0000 7f2a0593a640 1 --2- 172.21.15.40:0/3198790509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a00144840 0x7f2a000cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:51.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.579+0000 7f2a0593a640 1 -- 172.21.15.40:0/3198790509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a0006d1c0 msgr2=0x7f2a000cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:51.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.579+0000 7f2a0593a640 1 --2- 172.21.15.40:0/3198790509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a0006d1c0 0x7f2a000cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:51.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.579+0000 7f2a0593a640 1 -- 172.21.15.40:0/3198790509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc009900 con 0x7f2a00069e30 2023-10-18T09:06:51.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.579+0000 7f2a0593a640 1 --2- 172.21.15.40:0/3198790509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00069e30 0x7f2a0015b010 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f29fc006010 tx=0x7f29fc035e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:51.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.579+0000 7f29f67fc640 1 -- 172.21.15.40:0/3198790509 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc034440 con 0x7f2a00069e30 2023-10-18T09:06:51.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.579+0000 7f29f67fc640 1 -- 172.21.15.40:0/3198790509 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29fc004540 con 0x7f2a00069e30 2023-10-18T09:06:51.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.579+0000 7f29f67fc640 1 -- 172.21.15.40:0/3198790509 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc040450 con 0x7f2a00069e30 2023-10-18T09:06:51.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.580+0000 7f2a07b9e640 1 -- 172.21.15.40:0/3198790509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0013d610 con 0x7f2a00069e30 2023-10-18T09:06:51.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.580+0000 7f2a07b9e640 1 -- 172.21.15.40:0/3198790509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0013dbd0 con 0x7f2a00069e30 2023-10-18T09:06:51.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.581+0000 7f2a07b9e640 1 -- 172.21.15.40:0/3198790509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a00066fd0 con 0x7f2a00069e30 2023-10-18T09:06:51.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.582+0000 7f29f67fc640 1 -- 172.21.15.40:0/3198790509 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f29fc00a040 con 0x7f2a00069e30 2023-10-18T09:06:51.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.583+0000 7f29f67fc640 1 --2- 172.21.15.40:0/3198790509 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29e406ae40 0x7f29e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:51.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.583+0000 7f29f67fc640 1 -- 172.21.15.40:0/3198790509 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f29fc0ac7f0 con 0x7f2a00069e30 2023-10-18T09:06:51.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.583+0000 7f29f67fc640 1 -- 172.21.15.40:0/3198790509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f29e4072a30 con 0x7f2a00069e30 2023-10-18T09:06:51.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.583+0000 7f2a05139640 1 --2- 172.21.15.40:0/3198790509 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29e406ae40 0x7f29e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:51.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.585+0000 7f2a05139640 1 --2- 172.21.15.40:0/3198790509 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29e406ae40 0x7f29e406d300 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2a000ce070 tx=0x7f29f0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:51.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.588+0000 7f29f67fc640 1 -- 172.21.15.40:0/3198790509 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29fc039070 con 0x7f2a00069e30 2023-10-18T09:06:51.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.788+0000 7f2a07b9e640 1 -- 172.21.15.40:0/3198790509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a0006d630 con 0x7f2a00069e30 2023-10-18T09:06:51.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.791+0000 7f29f67fc640 1 -- 172.21.15.40:0/3198790509 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+16660 (secure 0 0 0) 0x7f29fc044030 con 0x7f2a00069e30 2023-10-18T09:06:51.794 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:06:51.794 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":166,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:06:11.132972+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:04:34.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-18T09:04:35.972659+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.26","mappings":[{"from":2,"to":0}]},{"pgid":"19.33","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:06:51.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.794+0000 7f2a07b9e640 1 -- 172.21.15.40:0/3198790509 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29e406ae40 msgr2=0x7f29e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:51.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.794+0000 7f2a07b9e640 1 --2- 172.21.15.40:0/3198790509 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29e406ae40 0x7f29e406d300 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2a000ce070 tx=0x7f29f0009040 comp rx=0 tx=0).stop 2023-10-18T09:06:51.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.794+0000 7f2a07b9e640 1 -- 172.21.15.40:0/3198790509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00069e30 msgr2=0x7f2a0015b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:51.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.794+0000 7f2a07b9e640 1 --2- 172.21.15.40:0/3198790509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00069e30 0x7f2a0015b010 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f29fc006010 tx=0x7f29fc035e30 comp rx=0 tx=0).stop 2023-10-18T09:06:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.794+0000 7f2a07b9e640 1 -- 172.21.15.40:0/3198790509 shutdown_connections 2023-10-18T09:06:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.794+0000 7f2a07b9e640 1 --2- 172.21.15.40:0/3198790509 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f29e406ae40 0x7f29e406d300 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.794+0000 7f2a07b9e640 1 --2- 172.21.15.40:0/3198790509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2a00144840 0x7f2a000cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.794+0000 7f2a07b9e640 1 --2- 172.21.15.40:0/3198790509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a0006d1c0 0x7f2a000cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.794+0000 7f2a07b9e640 1 --2- 172.21.15.40:0/3198790509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a00069e30 0x7f2a0015b010 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:51.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.794+0000 7f2a07b9e640 1 -- 172.21.15.40:0/3198790509 >> 172.21.15.40:0/3198790509 conn(0x7f2a000aff00 msgr2=0x7f2a0006c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:51.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.795+0000 7f2a07b9e640 1 -- 172.21.15.40:0/3198790509 shutdown_connections 2023-10-18T09:06:51.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:51.795+0000 7f2a07b9e640 1 -- 172.21.15.40:0/3198790509 wait complete. 2023-10-18T09:06:51.817 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:04:34.597892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '161', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:04:35.972659+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '166', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:06:51.818 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:06:52.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.052+0000 7f3c59b55640 1 -- 172.21.15.40:0/144365055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c3c004d00 msgr2=0x7f3c3c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:52.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.052+0000 7f3c59b55640 1 --2- 172.21.15.40:0/144365055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c3c004d00 0x7f3c3c005180 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c005fd0 tx=0x7f3c3c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.052+0000 7f3c59354640 1 -- 172.21.15.40:0/144365055 reap_dead start 2023-10-18T09:06:52.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.052+0000 7f3c59b55640 1 -- 172.21.15.40:0/144365055 shutdown_connections 2023-10-18T09:06:52.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.052+0000 7f3c59b55640 1 -- 172.21.15.40:0/144365055 >> 172.21.15.40:0/144365055 conn(0x7f3c54069720 msgr2=0x7f3c54069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:52.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.052+0000 7f3c59b55640 1 -- 172.21.15.40:0/144365055 shutdown_connections 2023-10-18T09:06:52.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.052+0000 7f3c59b55640 1 -- 172.21.15.40:0/144365055 wait complete. 2023-10-18T09:06:52.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.054+0000 7f3c59b55640 1 Processor -- start 2023-10-18T09:06:52.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.054+0000 7f3c59b55640 1 -- start start 2023-10-18T09:06:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.055+0000 7f3c59b55640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c3c004d00 0x7f3c54152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.055+0000 7f3c59b55640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c540701d0 0x7f3c541e2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.055+0000 7f3c59b55640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c5415ba90 0x7f3c5415bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.055+0000 7f3c59b55640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c540ad6b0 con 0x7f3c3c004d00 2023-10-18T09:06:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.055+0000 7f3c59b55640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5415c480 con 0x7f3c5415ba90 2023-10-18T09:06:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.055+0000 7f3c59b55640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c541e3550 con 0x7f3c540701d0 2023-10-18T09:06:52.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.056+0000 7f3c58b53640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c3c004d00 0x7f3c54152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:52.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.056+0000 7f3c58b53640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c3c004d00 0x7f3c54152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42004/0 (socket says 172.21.15.40:42004) 2023-10-18T09:06:52.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.056+0000 7f3c58b53640 1 -- 172.21.15.40:0/732131886 learned_addr learned my addr 172.21.15.40:0/732131886 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:52.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.056+0000 7f3c59354640 1 --2- 172.21.15.40:0/732131886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c5415ba90 0x7f3c5415bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:52.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.056+0000 7f3c4bfff640 1 --2- 172.21.15.40:0/732131886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c540701d0 0x7f3c541e2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:52.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.056+0000 7f3c58b53640 1 -- 172.21.15.40:0/732131886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c540701d0 msgr2=0x7f3c541e2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:52.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.056+0000 7f3c58b53640 1 --2- 172.21.15.40:0/732131886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c540701d0 0x7f3c541e2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.056+0000 7f3c58b53640 1 -- 172.21.15.40:0/732131886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c5415ba90 msgr2=0x7f3c5415bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:52.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.056+0000 7f3c58b53640 1 --2- 172.21.15.40:0/732131886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c5415ba90 0x7f3c5415bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.056+0000 7f3c58b53640 1 -- 172.21.15.40:0/732131886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c540acd10 con 0x7f3c3c004d00 2023-10-18T09:06:52.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.057+0000 7f3c58b53640 1 --2- 172.21.15.40:0/732131886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c3c004d00 0x7f3c54152f90 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000c530 tx=0x7f3c5000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:52.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.057+0000 7f3c497fa640 1 -- 172.21.15.40:0/732131886 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c500052d0 con 0x7f3c3c004d00 2023-10-18T09:06:52.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.057+0000 7f3c497fa640 1 -- 172.21.15.40:0/732131886 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3c50005430 con 0x7f3c3c004d00 2023-10-18T09:06:52.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.057+0000 7f3c497fa640 1 -- 172.21.15.40:0/732131886 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50012890 con 0x7f3c3c004d00 2023-10-18T09:06:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.057+0000 7f3c59b55640 1 -- 172.21.15.40:0/732131886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c540acff0 con 0x7f3c3c004d00 2023-10-18T09:06:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.057+0000 7f3c59b55640 1 -- 172.21.15.40:0/732131886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c541534d0 con 0x7f3c3c004d00 2023-10-18T09:06:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.058+0000 7f3c59b55640 1 -- 172.21.15.40:0/732131886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c54153690 con 0x7f3c3c004d00 2023-10-18T09:06:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.059+0000 7f3c497fa640 1 -- 172.21.15.40:0/732131886 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3c500129f0 con 0x7f3c3c004d00 2023-10-18T09:06:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.060+0000 7f3c497fa640 1 --2- 172.21.15.40:0/732131886 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3c4006ae40 0x7f3c4006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:52.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.060+0000 7f3c497fa640 1 -- 172.21.15.40:0/732131886 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3c50089380 con 0x7f3c3c004d00 2023-10-18T09:06:52.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.060+0000 7f3c497fa640 1 -- 172.21.15.40:0/732131886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f3c40072a30 con 0x7f3c3c004d00 2023-10-18T09:06:52.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.061+0000 7f3c4bfff640 1 --2- 172.21.15.40:0/732131886 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3c4006ae40 0x7f3c4006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:52.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.062+0000 7f3c4bfff640 1 --2- 172.21.15.40:0/732131886 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3c4006ae40 0x7f3c4006d300 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3c44001eb0 tx=0x7f3c44001470 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:52.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.064+0000 7f3c497fa640 1 -- 172.21.15.40:0/732131886 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c54153690 con 0x7f3c3c004d00 2023-10-18T09:06:52.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.265+0000 7f3c59b55640 1 -- 172.21.15.40:0/732131886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c54153690 con 0x7f3c3c004d00 2023-10-18T09:06:52.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.266+0000 7f3c497fa640 1 -- 172.21.15.40:0/732131886 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v166) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c54153690 con 0x7f3c3c004d00 2023-10-18T09:06:52.269 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:06:52.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.269+0000 7f3c59b55640 1 -- 172.21.15.40:0/732131886 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3c4006ae40 msgr2=0x7f3c4006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:52.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.269+0000 7f3c59b55640 1 --2- 172.21.15.40:0/732131886 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3c4006ae40 0x7f3c4006d300 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3c44001eb0 tx=0x7f3c44001470 comp rx=0 tx=0).stop 2023-10-18T09:06:52.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.269+0000 7f3c59b55640 1 -- 172.21.15.40:0/732131886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c3c004d00 msgr2=0x7f3c54152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:52.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.269+0000 7f3c59b55640 1 --2- 172.21.15.40:0/732131886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c3c004d00 0x7f3c54152f90 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000c530 tx=0x7f3c5000c9f0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.269+0000 7f3c59b55640 1 -- 172.21.15.40:0/732131886 shutdown_connections 2023-10-18T09:06:52.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.269+0000 7f3c59b55640 1 --2- 172.21.15.40:0/732131886 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3c4006ae40 0x7f3c4006d300 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.269+0000 7f3c59b55640 1 --2- 172.21.15.40:0/732131886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c5415ba90 0x7f3c5415bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.269+0000 7f3c59b55640 1 --2- 172.21.15.40:0/732131886 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c540701d0 0x7f3c541e2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.269+0000 7f3c59b55640 1 --2- 172.21.15.40:0/732131886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c3c004d00 0x7f3c54152f90 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.269+0000 7f3c59b55640 1 -- 172.21.15.40:0/732131886 >> 172.21.15.40:0/732131886 conn(0x7f3c54069720 msgr2=0x7f3c540aa910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:52.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.270+0000 7f3c59b55640 1 -- 172.21.15.40:0/732131886 shutdown_connections 2023-10-18T09:06:52.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.270+0000 7f3c59b55640 1 -- 172.21.15.40:0/732131886 wait complete. 2023-10-18T09:06:52.292 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:06:52.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.528+0000 7fbb04ba9640 1 -- 172.21.15.40:0/1634221694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb000cbf20 msgr2=0x7fbb000cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:52.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.528+0000 7fbb04ba9640 1 --2- 172.21.15.40:0/1634221694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb000cbf20 0x7fbb000cae80 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8005fd0 tx=0x7fbaf802bfc0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.528+0000 7fbb04ba9640 1 -- 172.21.15.40:0/1634221694 shutdown_connections 2023-10-18T09:06:52.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.528+0000 7fbb04ba9640 1 --2- 172.21.15.40:0/1634221694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb000cbf20 0x7fbb000cae80 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.528+0000 7fbb04ba9640 1 --2- 172.21.15.40:0/1634221694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb0015b4d0 0x7fbb000ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.528+0000 7fbb04ba9640 1 -- 172.21.15.40:0/1634221694 >> 172.21.15.40:0/1634221694 conn(0x7fbb00069720 msgr2=0x7fbb00069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:52.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.528+0000 7fbb04ba9640 1 -- 172.21.15.40:0/1634221694 shutdown_connections 2023-10-18T09:06:52.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.528+0000 7fbb04ba9640 1 -- 172.21.15.40:0/1634221694 wait complete. 2023-10-18T09:06:52.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.530+0000 7fbb04ba9640 1 Processor -- start 2023-10-18T09:06:52.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.530+0000 7fbb04ba9640 1 -- start start 2023-10-18T09:06:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.531+0000 7fbb04ba9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb000ca580 0x7fbb00144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.531+0000 7fbb04ba9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb000cbf20 0x7fbb001e50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.531+0000 7fbb04ba9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb0015b4d0 0x7fbb001e5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.531+0000 7fbb04ba9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb000acb50 con 0x7fbb000ca580 2023-10-18T09:06:52.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.531+0000 7fbb04ba9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb000acd80 con 0x7fbb0015b4d0 2023-10-18T09:06:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.531+0000 7fbb04ba9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb001e6d60 con 0x7fbb000cbf20 2023-10-18T09:06:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.531+0000 7fbafe59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb000ca580 0x7fbb00144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.531+0000 7fbafe59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb000ca580 0x7fbb00144900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42018/0 (socket says 172.21.15.40:42018) 2023-10-18T09:06:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.531+0000 7fbafe59c640 1 -- 172.21.15.40:0/641889670 learned_addr learned my addr 172.21.15.40:0/641889670 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.532+0000 7fbafe59c640 1 -- 172.21.15.40:0/641889670 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb000cbf20 msgr2=0x7fbb001e50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.532+0000 7fbafe59c640 1 --2- 172.21.15.40:0/641889670 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb000cbf20 0x7fbb001e50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.532+0000 7fbafe59c640 1 -- 172.21.15.40:0/641889670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb0015b4d0 msgr2=0x7fbb001e5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.532+0000 7fbafed9d640 1 --2- 172.21.15.40:0/641889670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb0015b4d0 0x7fbb001e5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.532+0000 7fbafe59c640 1 --2- 172.21.15.40:0/641889670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb0015b4d0 0x7fbb001e5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.532+0000 7fbafe59c640 1 -- 172.21.15.40:0/641889670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf8009840 con 0x7fbb000ca580 2023-10-18T09:06:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.532+0000 7fbafe59c640 1 --2- 172.21.15.40:0/641889670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb000ca580 0x7fbb00144900 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000e790 tx=0x7fbaf000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:52.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.532+0000 7fbaeeffd640 1 -- 172.21.15.40:0/641889670 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0009580 con 0x7fbb000ca580 2023-10-18T09:06:52.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.532+0000 7fbaeeffd640 1 -- 172.21.15.40:0/641889670 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbaf000cdb0 con 0x7fbb000ca580 2023-10-18T09:06:52.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.532+0000 7fbaeeffd640 1 -- 172.21.15.40:0/641889670 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0010690 con 0x7fbb000ca580 2023-10-18T09:06:52.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.533+0000 7fbb04ba9640 1 -- 172.21.15.40:0/641889670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb001e7040 con 0x7fbb000ca580 2023-10-18T09:06:52.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.533+0000 7fbb04ba9640 1 -- 172.21.15.40:0/641889670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb001e7320 con 0x7fbb000ca580 2023-10-18T09:06:52.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.534+0000 7fbb04ba9640 1 -- 172.21.15.40:0/641889670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb00066fd0 con 0x7fbb000ca580 2023-10-18T09:06:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.535+0000 7fbaeeffd640 1 -- 172.21.15.40:0/641889670 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbaf00107f0 con 0x7fbb000ca580 2023-10-18T09:06:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.536+0000 7fbaeeffd640 1 --2- 172.21.15.40:0/641889670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbae006ae40 0x7fbae006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.536+0000 7fbaeeffd640 1 -- 172.21.15.40:0/641889670 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbaf0014070 con 0x7fbb000ca580 2023-10-18T09:06:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.536+0000 7fbaeeffd640 1 -- 172.21.15.40:0/641889670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7fbae0072a30 con 0x7fbb000ca580 2023-10-18T09:06:52.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.536+0000 7fbafdd9b640 1 --2- 172.21.15.40:0/641889670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbae006ae40 0x7fbae006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:52.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.537+0000 7fbafdd9b640 1 --2- 172.21.15.40:0/641889670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbae006ae40 0x7fbae006d300 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4007fd0 tx=0x7fbaf40078e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:52.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.540+0000 7fbaeeffd640 1 -- 172.21.15.40:0/641889670 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbaf0051ca0 con 0x7fbb000ca580 2023-10-18T09:06:52.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.733+0000 7fbb04ba9640 1 -- 172.21.15.40:0/641889670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbb0015ba20 con 0x7fbb000ca580 2023-10-18T09:06:52.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.734+0000 7fbaeeffd640 1 -- 172.21.15.40:0/641889670 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v166) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbaf0055140 con 0x7fbb000ca580 2023-10-18T09:06:52.737 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:06:52.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.738+0000 7fbb04ba9640 1 -- 172.21.15.40:0/641889670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbae006ae40 msgr2=0x7fbae006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:52.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.738+0000 7fbb04ba9640 1 --2- 172.21.15.40:0/641889670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbae006ae40 0x7fbae006d300 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4007fd0 tx=0x7fbaf40078e0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.738+0000 7fbb04ba9640 1 -- 172.21.15.40:0/641889670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb000ca580 msgr2=0x7fbb00144900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:52.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.738+0000 7fbb04ba9640 1 --2- 172.21.15.40:0/641889670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb000ca580 0x7fbb00144900 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000e790 tx=0x7fbaf000ec50 comp rx=0 tx=0).stop 2023-10-18T09:06:52.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.738+0000 7fbb04ba9640 1 -- 172.21.15.40:0/641889670 shutdown_connections 2023-10-18T09:06:52.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.738+0000 7fbb04ba9640 1 --2- 172.21.15.40:0/641889670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbae006ae40 0x7fbae006d300 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.738+0000 7fbb04ba9640 1 --2- 172.21.15.40:0/641889670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb0015b4d0 0x7fbb001e5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.738+0000 7fbb04ba9640 1 --2- 172.21.15.40:0/641889670 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb000cbf20 0x7fbb001e50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.738+0000 7fbb04ba9640 1 --2- 172.21.15.40:0/641889670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb000ca580 0x7fbb00144900 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.738+0000 7fbb04ba9640 1 -- 172.21.15.40:0/641889670 >> 172.21.15.40:0/641889670 conn(0x7fbb00069720 msgr2=0x7fbb000cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:52.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.738+0000 7fbb04ba9640 1 -- 172.21.15.40:0/641889670 shutdown_connections 2023-10-18T09:06:52.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.738+0000 7fbb04ba9640 1 -- 172.21.15.40:0/641889670 wait complete. 2023-10-18T09:06:52.760 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:06:52.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.996+0000 7f47c7060640 1 -- 172.21.15.40:0/547720429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47ac0051d0 msgr2=0x7f47ac00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:52.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.996+0000 7f47c7060640 1 --2- 172.21.15.40:0/547720429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47ac0051d0 0x7f47ac00b980 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f47b8001d50 tx=0x7f47ac0057f0 comp rx=0 tx=0).stop 2023-10-18T09:06:52.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.996+0000 7f47c7060640 1 -- 172.21.15.40:0/547720429 shutdown_connections 2023-10-18T09:06:53.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.996+0000 7f47c7060640 1 --2- 172.21.15.40:0/547720429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47ac0051d0 0x7f47ac00b980 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:53.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.996+0000 7f47c7060640 1 -- 172.21.15.40:0/547720429 >> 172.21.15.40:0/547720429 conn(0x7f47c006ca10 msgr2=0x7f47c006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:53.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.996+0000 7f47c7060640 1 -- 172.21.15.40:0/547720429 shutdown_connections 2023-10-18T09:06:53.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.996+0000 7f47c7060640 1 -- 172.21.15.40:0/547720429 wait complete. 2023-10-18T09:06:53.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.998+0000 7f47c7060640 1 Processor -- start 2023-10-18T09:06:53.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.998+0000 7f47c7060640 1 -- start start 2023-10-18T09:06:53.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.999+0000 7f47c7060640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47ac0051d0 0x7f47c01554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.999+0000 7f47c7060640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47c0156bf0 0x7f47c01524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.999+0000 7f47c7060640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c014cc20 0x7f47c01f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.999+0000 7f47c7060640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c01529f0 con 0x7f47c014cc20 2023-10-18T09:06:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.999+0000 7f47c7060640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c0155b50 con 0x7f47c0156bf0 2023-10-18T09:06:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:52.999+0000 7f47c7060640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c0155e30 con 0x7f47ac0051d0 2023-10-18T09:06:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.000+0000 7f47c685f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c014cc20 0x7f47c01f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.000+0000 7f47c685f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c014cc20 0x7f47c01f53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42022/0 (socket says 172.21.15.40:42022) 2023-10-18T09:06:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.000+0000 7f47c685f640 1 -- 172.21.15.40:0/3112391457 learned_addr learned my addr 172.21.15.40:0/3112391457 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.000+0000 7f47c585d640 1 --2- 172.21.15.40:0/3112391457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47c0156bf0 0x7f47c01524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:53.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.000+0000 7f47c605e640 1 --2- 172.21.15.40:0/3112391457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47ac0051d0 0x7f47c01554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.000+0000 7f47c685f640 1 -- 172.21.15.40:0/3112391457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47ac0051d0 msgr2=0x7f47c01554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.000+0000 7f47c685f640 1 --2- 172.21.15.40:0/3112391457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47ac0051d0 0x7f47c01554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.000+0000 7f47c685f640 1 -- 172.21.15.40:0/3112391457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47c0156bf0 msgr2=0x7f47c01524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.000+0000 7f47c685f640 1 --2- 172.21.15.40:0/3112391457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47c0156bf0 0x7f47c01524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:53.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.000+0000 7f47c685f640 1 -- 172.21.15.40:0/3112391457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c0156110 con 0x7f47c014cc20 2023-10-18T09:06:53.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.000+0000 7f47c605e640 1 --2- 172.21.15.40:0/3112391457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47ac0051d0 0x7f47c01554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:06:53.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.001+0000 7f47c685f640 1 --2- 172.21.15.40:0/3112391457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c014cc20 0x7f47c01f53e0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f47b8001d50 tx=0x7f47b8003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:53.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.001+0000 7f47b6ffd640 1 -- 172.21.15.40:0/3112391457 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b80060d0 con 0x7f47c014cc20 2023-10-18T09:06:53.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.001+0000 7f47b6ffd640 1 -- 172.21.15.40:0/3112391457 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f47b8006320 con 0x7f47c014cc20 2023-10-18T09:06:53.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.001+0000 7f47b6ffd640 1 -- 172.21.15.40:0/3112391457 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b80066f0 con 0x7f47c014cc20 2023-10-18T09:06:53.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.001+0000 7f47c7060640 1 -- 172.21.15.40:0/3112391457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c01563f0 con 0x7f47c014cc20 2023-10-18T09:06:53.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.001+0000 7f47c7060640 1 -- 172.21.15.40:0/3112391457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c014d0d0 con 0x7f47c014cc20 2023-10-18T09:06:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.002+0000 7f47c7060640 1 -- 172.21.15.40:0/3112391457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c0156910 con 0x7f47c014cc20 2023-10-18T09:06:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.003+0000 7f47b6ffd640 1 -- 172.21.15.40:0/3112391457 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f47b8034620 con 0x7f47c014cc20 2023-10-18T09:06:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.004+0000 7f47b6ffd640 1 --2- 172.21.15.40:0/3112391457 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f47a406ae90 0x7f47a406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:53.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.004+0000 7f47b6ffd640 1 -- 172.21.15.40:0/3112391457 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f47b80aaa10 con 0x7f47c014cc20 2023-10-18T09:06:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.004+0000 7f47b6ffd640 1 -- 172.21.15.40:0/3112391457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f47a4072a80 con 0x7f47c014cc20 2023-10-18T09:06:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.005+0000 7f47c605e640 1 --2- 172.21.15.40:0/3112391457 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f47a406ae90 0x7f47a406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:53.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.006+0000 7f47c605e640 1 --2- 172.21.15.40:0/3112391457 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f47a406ae90 0x7f47a406d350 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00ab40 tx=0x7f47bc005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:53.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.009+0000 7f47b6ffd640 1 -- 172.21.15.40:0/3112391457 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f47b8043070 con 0x7f47c014cc20 2023-10-18T09:06:53.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.204+0000 7f47c7060640 1 -- 172.21.15.40:0/3112391457 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47c014d260 con 0x7f47c014cc20 2023-10-18T09:06:53.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.204+0000 7f47b6ffd640 1 -- 172.21.15.40:0/3112391457 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v166) v1 ==== 100+0+11 (secure 0 0 0) 0x7f47c0156910 con 0x7f47c014cc20 2023-10-18T09:06:53.207 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:06:53.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.207+0000 7f47c7060640 1 -- 172.21.15.40:0/3112391457 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f47a406ae90 msgr2=0x7f47a406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:53.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.207+0000 7f47c7060640 1 --2- 172.21.15.40:0/3112391457 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f47a406ae90 0x7f47a406d350 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00ab40 tx=0x7f47bc005f90 comp rx=0 tx=0).stop 2023-10-18T09:06:53.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.207+0000 7f47c7060640 1 -- 172.21.15.40:0/3112391457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c014cc20 msgr2=0x7f47c01f53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:53.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.207+0000 7f47c7060640 1 --2- 172.21.15.40:0/3112391457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c014cc20 0x7f47c01f53e0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f47b8001d50 tx=0x7f47b8003f30 comp rx=0 tx=0).stop 2023-10-18T09:06:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.207+0000 7f47c7060640 1 -- 172.21.15.40:0/3112391457 shutdown_connections 2023-10-18T09:06:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.207+0000 7f47c7060640 1 --2- 172.21.15.40:0/3112391457 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f47a406ae90 0x7f47a406d350 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.207+0000 7f47c7060640 1 --2- 172.21.15.40:0/3112391457 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47c014cc20 0x7f47c01f53e0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.207+0000 7f47c7060640 1 --2- 172.21.15.40:0/3112391457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47c0156bf0 0x7f47c01524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:53.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.207+0000 7f47c7060640 1 --2- 172.21.15.40:0/3112391457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47ac0051d0 0x7f47c01554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.207+0000 7f47c7060640 1 -- 172.21.15.40:0/3112391457 >> 172.21.15.40:0/3112391457 conn(0x7f47c006ca10 msgr2=0x7f47c01544b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.207+0000 7f47c7060640 1 -- 172.21.15.40:0/3112391457 shutdown_connections 2023-10-18T09:06:53.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.208+0000 7f47c7060640 1 -- 172.21.15.40:0/3112391457 wait complete. 2023-10-18T09:06:53.230 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:06:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.464+0000 7f8a96e02640 1 -- 172.21.15.40:0/732575482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a900b2ed0 msgr2=0x7f8a9006d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.464+0000 7f8a96e02640 1 --2- 172.21.15.40:0/732575482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a900b2ed0 0x7f8a9006d2c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f8a9006d1f0 tx=0x7f8a8002c110 comp rx=0 tx=0).stop 2023-10-18T09:06:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.464+0000 7f8a96e02640 1 -- 172.21.15.40:0/732575482 shutdown_connections 2023-10-18T09:06:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.464+0000 7f8a96e02640 1 --2- 172.21.15.40:0/732575482 >> v2:172.21.15.144:3300/0 conn(0x7f8a9006d800 0x7f8a90069720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:53.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.464+0000 7f8a96e02640 1 --2- 172.21.15.40:0/732575482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a900b2ed0 0x7f8a9006d2c0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:53.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.464+0000 7f8a96e02640 1 -- 172.21.15.40:0/732575482 >> 172.21.15.40:0/732575482 conn(0x7f8a900cdfd0 msgr2=0x7f8a900ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:53.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.464+0000 7f8a96e02640 1 -- 172.21.15.40:0/732575482 shutdown_connections 2023-10-18T09:06:53.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.464+0000 7f8a96e02640 1 -- 172.21.15.40:0/732575482 wait complete. 2023-10-18T09:06:53.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.466+0000 7f8a96e02640 1 Processor -- start 2023-10-18T09:06:53.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.466+0000 7f8a96e02640 1 -- start start 2023-10-18T09:06:53.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.467+0000 7f8a96e02640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a9006d800 0x7f8a90149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:53.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.467+0000 7f8a96e02640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a900b2ed0 0x7f8a90152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:53.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.467+0000 7f8a96e02640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a9015b530 0x7f8a9014b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:53.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.467+0000 7f8a96e02640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90156660 con 0x7f8a900b2ed0 2023-10-18T09:06:53.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.467+0000 7f8a96e02640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90156050 con 0x7f8a9015b530 2023-10-18T09:06:53.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.467+0000 7f8a96e02640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90152aa0 con 0x7f8a9006d800 2023-10-18T09:06:53.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.468+0000 7f8a955ff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a900b2ed0 0x7f8a90152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:53.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.468+0000 7f8a955ff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a900b2ed0 0x7f8a90152560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42038/0 (socket says 172.21.15.40:42038) 2023-10-18T09:06:53.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.468+0000 7f8a955ff640 1 -- 172.21.15.40:0/2905945759 learned_addr learned my addr 172.21.15.40:0/2905945759 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:53.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.468+0000 7f8a96601640 1 --2- 172.21.15.40:0/2905945759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a9015b530 0x7f8a9014b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:53.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.468+0000 7f8a95e00640 1 --2- 172.21.15.40:0/2905945759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a9006d800 0x7f8a90149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:53.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.468+0000 7f8a955ff640 1 -- 172.21.15.40:0/2905945759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a9006d800 msgr2=0x7f8a90149610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:53.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.468+0000 7f8a955ff640 1 --2- 172.21.15.40:0/2905945759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a9006d800 0x7f8a90149610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:53.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.468+0000 7f8a955ff640 1 -- 172.21.15.40:0/2905945759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a9015b530 msgr2=0x7f8a9014b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:53.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.468+0000 7f8a955ff640 1 --2- 172.21.15.40:0/2905945759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a9015b530 0x7f8a9014b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:53.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.468+0000 7f8a955ff640 1 -- 172.21.15.40:0/2905945759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a80009840 con 0x7f8a900b2ed0 2023-10-18T09:06:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.468+0000 7f8a95e00640 1 --2- 172.21.15.40:0/2905945759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a9006d800 0x7f8a90149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:06:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.469+0000 7f8a955ff640 1 --2- 172.21.15.40:0/2905945759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a900b2ed0 0x7f8a90152560 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f8a80036040 tx=0x7f8a800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.469+0000 7f8a867fc640 1 -- 172.21.15.40:0/2905945759 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a800046f0 con 0x7f8a900b2ed0 2023-10-18T09:06:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.469+0000 7f8a867fc640 1 -- 172.21.15.40:0/2905945759 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8a80004870 con 0x7f8a900b2ed0 2023-10-18T09:06:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.469+0000 7f8a96e02640 1 -- 172.21.15.40:0/2905945759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a900cd080 con 0x7f8a900b2ed0 2023-10-18T09:06:53.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.469+0000 7f8a867fc640 1 -- 172.21.15.40:0/2905945759 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a800347e0 con 0x7f8a900b2ed0 2023-10-18T09:06:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.469+0000 7f8a96e02640 1 -- 172.21.15.40:0/2905945759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9014be20 con 0x7f8a900b2ed0 2023-10-18T09:06:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.470+0000 7f8a96e02640 1 -- 172.21.15.40:0/2905945759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a90066a90 con 0x7f8a900b2ed0 2023-10-18T09:06:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.472+0000 7f8a867fc640 1 -- 172.21.15.40:0/2905945759 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8a80034940 con 0x7f8a900b2ed0 2023-10-18T09:06:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.474+0000 7f8a867fc640 1 --2- 172.21.15.40:0/2905945759 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8a7406ae40 0x7f8a7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.474+0000 7f8a867fc640 1 -- 172.21.15.40:0/2905945759 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8a800ac860 con 0x7f8a900b2ed0 2023-10-18T09:06:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.474+0000 7f8a867fc640 1 -- 172.21.15.40:0/2905945759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f8a74072a30 con 0x7f8a900b2ed0 2023-10-18T09:06:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.475+0000 7f8a95e00640 1 --2- 172.21.15.40:0/2905945759 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8a7406ae40 0x7f8a7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.476+0000 7f8a95e00640 1 --2- 172.21.15.40:0/2905945759 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8a7406ae40 0x7f8a7406d300 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0011e0 tx=0x7f8a8c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.479+0000 7f8a867fc640 1 -- 172.21.15.40:0/2905945759 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a80043070 con 0x7f8a900b2ed0 2023-10-18T09:06:53.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.692+0000 7f8a96e02640 1 -- 172.21.15.40:0/2905945759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8a900b1ab0 con 0x7f8a900b2ed0 2023-10-18T09:06:53.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.693+0000 7f8a867fc640 1 -- 172.21.15.40:0/2905945759 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v166) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8a80077830 con 0x7f8a900b2ed0 2023-10-18T09:06:53.695 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T09:06:53.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.695+0000 7f8a96e02640 1 -- 172.21.15.40:0/2905945759 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8a7406ae40 msgr2=0x7f8a7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:53.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.695+0000 7f8a96e02640 1 --2- 172.21.15.40:0/2905945759 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8a7406ae40 0x7f8a7406d300 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0011e0 tx=0x7f8a8c009040 comp rx=0 tx=0).stop 2023-10-18T09:06:53.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.695+0000 7f8a96e02640 1 -- 172.21.15.40:0/2905945759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a900b2ed0 msgr2=0x7f8a90152560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:53.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.695+0000 7f8a96e02640 1 --2- 172.21.15.40:0/2905945759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a900b2ed0 0x7f8a90152560 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f8a80036040 tx=0x7f8a800042f0 comp rx=0 tx=0).stop 2023-10-18T09:06:53.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.695+0000 7f8a96e02640 1 -- 172.21.15.40:0/2905945759 shutdown_connections 2023-10-18T09:06:53.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.695+0000 7f8a96e02640 1 --2- 172.21.15.40:0/2905945759 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8a7406ae40 0x7f8a7406d300 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:53.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.695+0000 7f8a96e02640 1 --2- 172.21.15.40:0/2905945759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a9015b530 0x7f8a9014b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:53.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.695+0000 7f8a96e02640 1 --2- 172.21.15.40:0/2905945759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a900b2ed0 0x7f8a90152560 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:53.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.695+0000 7f8a96e02640 1 --2- 172.21.15.40:0/2905945759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a9006d800 0x7f8a90149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:53.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.695+0000 7f8a96e02640 1 -- 172.21.15.40:0/2905945759 >> 172.21.15.40:0/2905945759 conn(0x7f8a900cdfd0 msgr2=0x7f8a900b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:53.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.696+0000 7f8a96e02640 1 -- 172.21.15.40:0/2905945759 shutdown_connections 2023-10-18T09:06:53.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:53.696+0000 7f8a96e02640 1 -- 172.21.15.40:0/2905945759 wait complete. 2023-10-18T09:06:53.718 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-18T09:06:53.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:06:53.719 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-18T09:06:53.830 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T09:06:53.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:06:53.833 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T09:06:53.885 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:06:53.885 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:06:53.945 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:06:53.946 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:06:54.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:06:54.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.242+0000 7efccb59e640 1 -- 172.21.15.40:0/172789980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efccc141c80 msgr2=0x7efccc15bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:54.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.242+0000 7efccb59e640 1 --2- 172.21.15.40:0/172789980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efccc141c80 0x7efccc15bce0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7efccc06a2a0 tx=0x7efcbc02bdb0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.243+0000 7efccb59e640 1 -- 172.21.15.40:0/172789980 shutdown_connections 2023-10-18T09:06:54.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.243+0000 7efccb59e640 1 --2- 172.21.15.40:0/172789980 >> v2:172.21.15.144:3300/0 conn(0x7efccc06f600 0x7efccc06fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.243+0000 7efccb59e640 1 --2- 172.21.15.40:0/172789980 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efccc141c80 0x7efccc15bce0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.243+0000 7efccb59e640 1 -- 172.21.15.40:0/172789980 >> 172.21.15.40:0/172789980 conn(0x7efccc069720 msgr2=0x7efccc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:54.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.243+0000 7efccb59e640 1 -- 172.21.15.40:0/172789980 shutdown_connections 2023-10-18T09:06:54.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.243+0000 7efccb59e640 1 -- 172.21.15.40:0/172789980 wait complete. 2023-10-18T09:06:54.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.245+0000 7efccb59e640 1 Processor -- start 2023-10-18T09:06:54.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.245+0000 7efccb59e640 1 -- start start 2023-10-18T09:06:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.246+0000 7efccb59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efccc06f600 0x7efccc13bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:54.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.246+0000 7efccb59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efccc0cc190 0x7efccc06b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:54.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.246+0000 7efccb59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc141c80 0x7efccc06ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:54.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.246+0000 7efccb59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efccc1f24f0 con 0x7efccc0cc190 2023-10-18T09:06:54.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.246+0000 7efccb59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efccc1f27d0 con 0x7efccc141c80 2023-10-18T09:06:54.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.246+0000 7efccb59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efccc1f2ad0 con 0x7efccc06f600 2023-10-18T09:06:54.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.247+0000 7efcc3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efccc0cc190 0x7efccc06b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:54.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.247+0000 7efcc3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efccc0cc190 0x7efccc06b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42070/0 (socket says 172.21.15.40:42070) 2023-10-18T09:06:54.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.247+0000 7efcc3fff640 1 -- 172.21.15.40:0/3507415861 learned_addr learned my addr 172.21.15.40:0/3507415861 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:54.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.247+0000 7efccad9d640 1 --2- 172.21.15.40:0/3507415861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc141c80 0x7efccc06ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:54.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.247+0000 7efcca59c640 1 --2- 172.21.15.40:0/3507415861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efccc06f600 0x7efccc13bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:54.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.247+0000 7efcc3fff640 1 -- 172.21.15.40:0/3507415861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efccc06f600 msgr2=0x7efccc13bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:54.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.247+0000 7efcc3fff640 1 --2- 172.21.15.40:0/3507415861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efccc06f600 0x7efccc13bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.247+0000 7efcc3fff640 1 -- 172.21.15.40:0/3507415861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc141c80 msgr2=0x7efccc06ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:54.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.247+0000 7efcc3fff640 1 --2- 172.21.15.40:0/3507415861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc141c80 0x7efccc06ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.247+0000 7efcc3fff640 1 -- 172.21.15.40:0/3507415861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc009560 con 0x7efccc0cc190 2023-10-18T09:06:54.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.247+0000 7efcca59c640 1 --2- 172.21.15.40:0/3507415861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efccc06f600 0x7efccc13bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:06:54.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.248+0000 7efcc3fff640 1 --2- 172.21.15.40:0/3507415861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efccc0cc190 0x7efccc06b4d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7efcb400c400 tx=0x7efcb400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:54.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.248+0000 7efcc2ffd640 1 -- 172.21.15.40:0/3507415861 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4011790 con 0x7efccc0cc190 2023-10-18T09:06:54.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.248+0000 7efcc2ffd640 1 -- 172.21.15.40:0/3507415861 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efcb4011a20 con 0x7efccc0cc190 2023-10-18T09:06:54.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.248+0000 7efcc2ffd640 1 -- 172.21.15.40:0/3507415861 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4011e30 con 0x7efccc0cc190 2023-10-18T09:06:54.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.248+0000 7efccb59e640 1 -- 172.21.15.40:0/3507415861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efccc1f2db0 con 0x7efccc0cc190 2023-10-18T09:06:54.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.248+0000 7efccb59e640 1 -- 172.21.15.40:0/3507415861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efccc0c2270 con 0x7efccc0cc190 2023-10-18T09:06:54.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.250+0000 7efccb59e640 1 -- 172.21.15.40:0/3507415861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efccc0c98c0 con 0x7efccc0cc190 2023-10-18T09:06:54.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.250+0000 7efcc2ffd640 1 -- 172.21.15.40:0/3507415861 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efcb401c460 con 0x7efccc0cc190 2023-10-18T09:06:54.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.251+0000 7efcc2ffd640 1 --2- 172.21.15.40:0/3507415861 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efcac06ae40 0x7efcac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:54.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.251+0000 7efcc2ffd640 1 -- 172.21.15.40:0/3507415861 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efcb4089930 con 0x7efccc0cc190 2023-10-18T09:06:54.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.252+0000 7efcc2ffd640 1 -- 172.21.15.40:0/3507415861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7efcac072a30 con 0x7efccc0cc190 2023-10-18T09:06:54.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.252+0000 7efcca59c640 1 --2- 172.21.15.40:0/3507415861 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efcac06ae40 0x7efcac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:54.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.253+0000 7efcca59c640 1 --2- 172.21.15.40:0/3507415861 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efcac06ae40 0x7efcac06d300 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efcbc035000 tx=0x7efcbc004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:54.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.256+0000 7efcc2ffd640 1 -- 172.21.15.40:0/3507415861 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efcb4054900 con 0x7efccc0cc190 2023-10-18T09:06:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.451+0000 7efccb59e640 1 -- 172.21.15.40:0/3507415861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efccc1f32d0 con 0x7efccc0cc190 2023-10-18T09:06:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.454+0000 7efcc2ffd640 1 -- 172.21.15.40:0/3507415861 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+16660 (secure 0 0 0) 0x7efccc1f32d0 con 0x7efccc0cc190 2023-10-18T09:06:54.457 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:06:54.457 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":166,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:06:11.132972+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:04:34.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-18T09:04:35.972659+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.26","mappings":[{"from":2,"to":0}]},{"pgid":"19.33","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:06:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.457+0000 7efccb59e640 1 -- 172.21.15.40:0/3507415861 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efcac06ae40 msgr2=0x7efcac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.457+0000 7efccb59e640 1 --2- 172.21.15.40:0/3507415861 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efcac06ae40 0x7efcac06d300 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efcbc035000 tx=0x7efcbc004730 comp rx=0 tx=0).stop 2023-10-18T09:06:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.457+0000 7efccb59e640 1 -- 172.21.15.40:0/3507415861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efccc0cc190 msgr2=0x7efccc06b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.457+0000 7efccb59e640 1 --2- 172.21.15.40:0/3507415861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efccc0cc190 0x7efccc06b4d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7efcb400c400 tx=0x7efcb400c8c0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.457+0000 7efccb59e640 1 -- 172.21.15.40:0/3507415861 shutdown_connections 2023-10-18T09:06:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.457+0000 7efccb59e640 1 --2- 172.21.15.40:0/3507415861 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efcac06ae40 0x7efcac06d300 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.457+0000 7efccb59e640 1 --2- 172.21.15.40:0/3507415861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc141c80 0x7efccc06ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.457+0000 7efccb59e640 1 --2- 172.21.15.40:0/3507415861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efccc0cc190 0x7efccc06b4d0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.457+0000 7efccb59e640 1 --2- 172.21.15.40:0/3507415861 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efccc06f600 0x7efccc13bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.457+0000 7efccb59e640 1 -- 172.21.15.40:0/3507415861 >> 172.21.15.40:0/3507415861 conn(0x7efccc069720 msgr2=0x7efccc15aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:54.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.457+0000 7efccb59e640 1 -- 172.21.15.40:0/3507415861 shutdown_connections 2023-10-18T09:06:54.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.457+0000 7efccb59e640 1 -- 172.21.15.40:0/3507415861 wait complete. 2023-10-18T09:06:54.480 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:04:34.597892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '161', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:04:35.972659+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '166', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:06:54.480 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:06:54.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.711+0000 7fe49cb54640 1 -- 172.21.15.40:0/1380845811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe498144c10 msgr2=0x7fe49815b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:54.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.711+0000 7fe49cb54640 1 --2- 172.21.15.40:0/1380845811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe498144c10 0x7fe49815b8d0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fe488009830 tx=0x7fe48802bd50 comp rx=0 tx=0).stop 2023-10-18T09:06:54.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.712+0000 7fe49cb54640 1 -- 172.21.15.40:0/1380845811 shutdown_connections 2023-10-18T09:06:54.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.712+0000 7fe49cb54640 1 --2- 172.21.15.40:0/1380845811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4980cadc0 0x7fe4980cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.712+0000 7fe49cb54640 1 --2- 172.21.15.40:0/1380845811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe498144c10 0x7fe49815b8d0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.712+0000 7fe49cb54640 1 -- 172.21.15.40:0/1380845811 >> 172.21.15.40:0/1380845811 conn(0x7fe498069720 msgr2=0x7fe498069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:54.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.712+0000 7fe49cb54640 1 -- 172.21.15.40:0/1380845811 shutdown_connections 2023-10-18T09:06:54.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.712+0000 7fe49cb54640 1 -- 172.21.15.40:0/1380845811 wait complete. 2023-10-18T09:06:54.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.713+0000 7fe49cb54640 1 Processor -- start 2023-10-18T09:06:54.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.714+0000 7fe49cb54640 1 -- start start 2023-10-18T09:06:54.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.714+0000 7fe49cb54640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4980ca380 0x7fe498135b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:54.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe49cb54640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4980cadc0 0x7fe4980b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:54.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe49cb54640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe498144c10 0x7fe4980afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:54.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe49cb54640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4980b1e30 con 0x7fe4980cadc0 2023-10-18T09:06:54.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe49cb54640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4980b0850 con 0x7fe4980ca380 2023-10-18T09:06:54.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe49cb54640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4980b0b30 con 0x7fe498144c10 2023-10-18T09:06:54.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe496ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4980cadc0 0x7fe4980b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:54.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe496ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4980cadc0 0x7fe4980b18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42094/0 (socket says 172.21.15.40:42094) 2023-10-18T09:06:54.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe496ffd640 1 -- 172.21.15.40:0/2934029034 learned_addr learned my addr 172.21.15.40:0/2934029034 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:54.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe4977fe640 1 --2- 172.21.15.40:0/2934029034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4980ca380 0x7fe498135b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe497fff640 1 --2- 172.21.15.40:0/2934029034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe498144c10 0x7fe4980afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:54.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe496ffd640 1 -- 172.21.15.40:0/2934029034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe498144c10 msgr2=0x7fe4980afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe496ffd640 1 --2- 172.21.15.40:0/2934029034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe498144c10 0x7fe4980afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe496ffd640 1 -- 172.21.15.40:0/2934029034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4980ca380 msgr2=0x7fe498135b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:54.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe496ffd640 1 --2- 172.21.15.40:0/2934029034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4980ca380 0x7fe498135b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe496ffd640 1 -- 172.21.15.40:0/2934029034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe488009490 con 0x7fe4980cadc0 2023-10-18T09:06:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe4977fe640 1 --2- 172.21.15.40:0/2934029034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4980ca380 0x7fe498135b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:06:54.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.715+0000 7fe497fff640 1 --2- 172.21.15.40:0/2934029034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe498144c10 0x7fe4980afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:06:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.716+0000 7fe496ffd640 1 --2- 172.21.15.40:0/2934029034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4980cadc0 0x7fe4980b18f0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00c500 tx=0x7fe48c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.716+0000 7fe47ffff640 1 -- 172.21.15.40:0/2934029034 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c012830 con 0x7fe4980cadc0 2023-10-18T09:06:54.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.716+0000 7fe47ffff640 1 -- 172.21.15.40:0/2934029034 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe48c012a80 con 0x7fe4980cadc0 2023-10-18T09:06:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.716+0000 7fe47ffff640 1 -- 172.21.15.40:0/2934029034 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c012e90 con 0x7fe4980cadc0 2023-10-18T09:06:54.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.716+0000 7fe49cb54640 1 -- 172.21.15.40:0/2934029034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4980b0e10 con 0x7fe4980cadc0 2023-10-18T09:06:54.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.716+0000 7fe49cb54640 1 -- 172.21.15.40:0/2934029034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4980b13d0 con 0x7fe4980cadc0 2023-10-18T09:06:54.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.717+0000 7fe49cb54640 1 -- 172.21.15.40:0/2934029034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4980b15c0 con 0x7fe4980cadc0 2023-10-18T09:06:54.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.718+0000 7fe47ffff640 1 -- 172.21.15.40:0/2934029034 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe48c01a3b0 con 0x7fe4980cadc0 2023-10-18T09:06:54.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.719+0000 7fe47ffff640 1 --2- 172.21.15.40:0/2934029034 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe48406ae40 0x7fe48406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:54.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.719+0000 7fe47ffff640 1 -- 172.21.15.40:0/2934029034 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe48c089590 con 0x7fe4980cadc0 2023-10-18T09:06:54.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.719+0000 7fe47ffff640 1 -- 172.21.15.40:0/2934029034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7fe484072a30 con 0x7fe4980cadc0 2023-10-18T09:06:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.720+0000 7fe4977fe640 1 --2- 172.21.15.40:0/2934029034 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe48406ae40 0x7fe48406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:54.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.721+0000 7fe4977fe640 1 --2- 172.21.15.40:0/2934029034 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe48406ae40 0x7fe48406d300 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe488036000 tx=0x7fe488004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:54.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.723+0000 7fe47ffff640 1 -- 172.21.15.40:0/2934029034 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe48c016060 con 0x7fe4980cadc0 2023-10-18T09:06:54.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.916+0000 7fe49cb54640 1 -- 172.21.15.40:0/2934029034 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe498066e00 con 0x7fe4980cadc0 2023-10-18T09:06:54.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.916+0000 7fe47ffff640 1 -- 172.21.15.40:0/2934029034 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v166) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe48c058000 con 0x7fe4980cadc0 2023-10-18T09:06:54.919 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:06:54.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.919+0000 7fe49cb54640 1 -- 172.21.15.40:0/2934029034 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe48406ae40 msgr2=0x7fe48406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:54.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.919+0000 7fe49cb54640 1 --2- 172.21.15.40:0/2934029034 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe48406ae40 0x7fe48406d300 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe488036000 tx=0x7fe488004730 comp rx=0 tx=0).stop 2023-10-18T09:06:54.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.919+0000 7fe49cb54640 1 -- 172.21.15.40:0/2934029034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4980cadc0 msgr2=0x7fe4980b18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:54.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.919+0000 7fe49cb54640 1 --2- 172.21.15.40:0/2934029034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4980cadc0 0x7fe4980b18f0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00c500 tx=0x7fe48c00c9c0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.919+0000 7fe49cb54640 1 -- 172.21.15.40:0/2934029034 shutdown_connections 2023-10-18T09:06:54.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.919+0000 7fe49cb54640 1 --2- 172.21.15.40:0/2934029034 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe48406ae40 0x7fe48406d300 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.919+0000 7fe49cb54640 1 --2- 172.21.15.40:0/2934029034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe498144c10 0x7fe4980afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.919+0000 7fe49cb54640 1 --2- 172.21.15.40:0/2934029034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4980cadc0 0x7fe4980b18f0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.919+0000 7fe49cb54640 1 --2- 172.21.15.40:0/2934029034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4980ca380 0x7fe498135b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:54.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.919+0000 7fe49cb54640 1 -- 172.21.15.40:0/2934029034 >> 172.21.15.40:0/2934029034 conn(0x7fe498069720 msgr2=0x7fe498070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:54.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.919+0000 7fe49cb54640 1 -- 172.21.15.40:0/2934029034 shutdown_connections 2023-10-18T09:06:54.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:54.920+0000 7fe49cb54640 1 -- 172.21.15.40:0/2934029034 wait complete. 2023-10-18T09:06:54.940 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:06:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.176+0000 7f9fdad3c640 1 -- 172.21.15.40:0/1593820444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd40ae520 msgr2=0x7f9fd40ae9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.176+0000 7f9fdad3c640 1 --2- 172.21.15.40:0/1593820444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd40ae520 0x7f9fd40ae9d0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f9fd415f030 tx=0x7f9fcc02c260 comp rx=0 tx=0).stop 2023-10-18T09:06:55.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.176+0000 7f9fdad3c640 1 -- 172.21.15.40:0/1593820444 shutdown_connections 2023-10-18T09:06:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.176+0000 7f9fdad3c640 1 --2- 172.21.15.40:0/1593820444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd40ae520 0x7f9fd40ae9d0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.176+0000 7f9fdad3c640 1 --2- 172.21.15.40:0/1593820444 >> v2:172.21.15.144:3300/0 conn(0x7f9fd406a830 0x7f9fd40ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.176+0000 7f9fdad3c640 1 -- 172.21.15.40:0/1593820444 >> 172.21.15.40:0/1593820444 conn(0x7f9fd40b0ab0 msgr2=0x7f9fd40b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.176+0000 7f9fdad3c640 1 -- 172.21.15.40:0/1593820444 shutdown_connections 2023-10-18T09:06:55.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.176+0000 7f9fdad3c640 1 -- 172.21.15.40:0/1593820444 wait complete. 2023-10-18T09:06:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.178+0000 7f9fdad3c640 1 Processor -- start 2023-10-18T09:06:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.178+0000 7f9fdad3c640 1 -- start start 2023-10-18T09:06:55.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.179+0000 7f9fdad3c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fd406a830 0x7f9fd414a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:55.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.179+0000 7f9fdad3c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd4097a10 0x7f9fd415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:55.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.179+0000 7f9fdad3c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9fd40ae520 0x7f9fd415cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:55.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.179+0000 7f9fdad3c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd4157de0 con 0x7f9fd4097a10 2023-10-18T09:06:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.179+0000 7f9fdad3c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd4159e30 con 0x7f9fd406a830 2023-10-18T09:06:55.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.179+0000 7f9fdad3c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd414a9e0 con 0x7f9fd40ae520 2023-10-18T09:06:55.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.179+0000 7f9fd9539640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd4097a10 0x7f9fd415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:55.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.180+0000 7f9fd9539640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd4097a10 0x7f9fd415c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43252/0 (socket says 172.21.15.40:43252) 2023-10-18T09:06:55.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.180+0000 7f9fd9539640 1 -- 172.21.15.40:0/3957457915 learned_addr learned my addr 172.21.15.40:0/3957457915 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:55.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.180+0000 7f9fd9d3a640 1 --2- 172.21.15.40:0/3957457915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fd406a830 0x7f9fd414a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:55.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.180+0000 7f9fd9539640 1 -- 172.21.15.40:0/3957457915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9fd40ae520 msgr2=0x7f9fd415cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:55.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.180+0000 7f9fd9539640 1 --2- 172.21.15.40:0/3957457915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9fd40ae520 0x7f9fd415cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:55.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.180+0000 7f9fd9539640 1 -- 172.21.15.40:0/3957457915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fd406a830 msgr2=0x7f9fd414a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:55.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.180+0000 7f9fd9539640 1 --2- 172.21.15.40:0/3957457915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fd406a830 0x7f9fd414a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:55.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.180+0000 7f9fd9539640 1 -- 172.21.15.40:0/3957457915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fcc009900 con 0x7f9fd4097a10 2023-10-18T09:06:55.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.180+0000 7f9fd9539640 1 --2- 172.21.15.40:0/3957457915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd4097a10 0x7f9fd415c5a0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400d770 tx=0x7f9fc400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:55.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.181+0000 7f9fca7fc640 1 -- 172.21.15.40:0/3957457915 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4009c30 con 0x7f9fd4097a10 2023-10-18T09:06:55.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.181+0000 7f9fca7fc640 1 -- 172.21.15.40:0/3957457915 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9fc400de80 con 0x7f9fd4097a10 2023-10-18T09:06:55.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.181+0000 7f9fca7fc640 1 -- 172.21.15.40:0/3957457915 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4005280 con 0x7f9fd4097a10 2023-10-18T09:06:55.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.181+0000 7f9fdad3c640 1 -- 172.21.15.40:0/3957457915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd4157540 con 0x7f9fd4097a10 2023-10-18T09:06:55.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.181+0000 7f9fdad3c640 1 -- 172.21.15.40:0/3957457915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd4157b00 con 0x7f9fd4097a10 2023-10-18T09:06:55.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.182+0000 7f9fdad3c640 1 -- 172.21.15.40:0/3957457915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd4159b90 con 0x7f9fd4097a10 2023-10-18T09:06:55.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.183+0000 7f9fca7fc640 1 -- 172.21.15.40:0/3957457915 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9fc40053e0 con 0x7f9fd4097a10 2023-10-18T09:06:55.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.184+0000 7f9fca7fc640 1 --2- 172.21.15.40:0/3957457915 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9fb806ae40 0x7f9fb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:55.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.184+0000 7f9fca7fc640 1 -- 172.21.15.40:0/3957457915 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9fc4089900 con 0x7f9fd4097a10 2023-10-18T09:06:55.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.184+0000 7f9fca7fc640 1 -- 172.21.15.40:0/3957457915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f9fb8072a30 con 0x7f9fd4097a10 2023-10-18T09:06:55.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.185+0000 7f9fd9d3a640 1 --2- 172.21.15.40:0/3957457915 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9fb806ae40 0x7f9fb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:55.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.186+0000 7f9fd9d3a640 1 --2- 172.21.15.40:0/3957457915 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9fb806ae40 0x7f9fb806d300 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0004ea0 tx=0x7f9fd0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:55.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.188+0000 7f9fca7fc640 1 -- 172.21.15.40:0/3957457915 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9fc4017020 con 0x7f9fd4097a10 2023-10-18T09:06:55.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.379+0000 7f9fdad3c640 1 -- 172.21.15.40:0/3957457915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9fd4159f70 con 0x7f9fd4097a10 2023-10-18T09:06:55.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.380+0000 7f9fca7fc640 1 -- 172.21.15.40:0/3957457915 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v166) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9fd4159f70 con 0x7f9fd4097a10 2023-10-18T09:06:55.382 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:06:55.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.382+0000 7f9fdad3c640 1 -- 172.21.15.40:0/3957457915 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9fb806ae40 msgr2=0x7f9fb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:55.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.382+0000 7f9fdad3c640 1 --2- 172.21.15.40:0/3957457915 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9fb806ae40 0x7f9fb806d300 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0004ea0 tx=0x7f9fd0009040 comp rx=0 tx=0).stop 2023-10-18T09:06:55.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.382+0000 7f9fdad3c640 1 -- 172.21.15.40:0/3957457915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd4097a10 msgr2=0x7f9fd415c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.382+0000 7f9fdad3c640 1 --2- 172.21.15.40:0/3957457915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd4097a10 0x7f9fd415c5a0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400d770 tx=0x7f9fc400dc30 comp rx=0 tx=0).stop 2023-10-18T09:06:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.382+0000 7f9fdad3c640 1 -- 172.21.15.40:0/3957457915 shutdown_connections 2023-10-18T09:06:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.383+0000 7f9fdad3c640 1 --2- 172.21.15.40:0/3957457915 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9fb806ae40 0x7f9fb806d300 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.383+0000 7f9fdad3c640 1 --2- 172.21.15.40:0/3957457915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9fd40ae520 0x7f9fd415cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.383+0000 7f9fdad3c640 1 --2- 172.21.15.40:0/3957457915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fd4097a10 0x7f9fd415c5a0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:55.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.383+0000 7f9fdad3c640 1 --2- 172.21.15.40:0/3957457915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9fd406a830 0x7f9fd414a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:55.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.383+0000 7f9fdad3c640 1 -- 172.21.15.40:0/3957457915 >> 172.21.15.40:0/3957457915 conn(0x7f9fd40b0ab0 msgr2=0x7f9fd4097df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:55.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.383+0000 7f9fdad3c640 1 -- 172.21.15.40:0/3957457915 shutdown_connections 2023-10-18T09:06:55.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.383+0000 7f9fdad3c640 1 -- 172.21.15.40:0/3957457915 wait complete. 2023-10-18T09:06:55.404 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:06:55.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.631+0000 7f970b524640 1 -- 172.21.15.40:0/2804302424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9704157800 msgr2=0x7f97040b1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.631+0000 7f970b524640 1 --2- 172.21.15.40:0/2804302424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9704157800 0x7f97040b1cc0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f96f8009900 tx=0x7f96f802bd90 comp rx=0 tx=0).stop 2023-10-18T09:06:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.631+0000 7f970b524640 1 -- 172.21.15.40:0/2804302424 shutdown_connections 2023-10-18T09:06:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.631+0000 7f970b524640 1 --2- 172.21.15.40:0/2804302424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9704157800 0x7f97040b1cc0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:55.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.631+0000 7f970b524640 1 -- 172.21.15.40:0/2804302424 >> 172.21.15.40:0/2804302424 conn(0x7f9704069720 msgr2=0x7f9704069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.631+0000 7f970b524640 1 -- 172.21.15.40:0/2804302424 shutdown_connections 2023-10-18T09:06:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.631+0000 7f970b524640 1 -- 172.21.15.40:0/2804302424 wait complete. 2023-10-18T09:06:55.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.633+0000 7f970b524640 1 Processor -- start 2023-10-18T09:06:55.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.633+0000 7f970b524640 1 -- start start 2023-10-18T09:06:55.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.634+0000 7f970b524640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970406d660 0x7f970414f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:55.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.634+0000 7f970b524640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f970406f570 0x7f97041588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:55.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.634+0000 7f970b524640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9704157800 0x7f97041f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:55.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.634+0000 7f970b524640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970414fc50 con 0x7f970406d660 2023-10-18T09:06:55.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.634+0000 7f970b524640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9704150e80 con 0x7f9704157800 2023-10-18T09:06:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.634+0000 7f970b524640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97040ab800 con 0x7f970406f570 2023-10-18T09:06:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.634+0000 7f97092c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970406d660 0x7f970414f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.634+0000 7f97092c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970406d660 0x7f970414f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43268/0 (socket says 172.21.15.40:43268) 2023-10-18T09:06:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.634+0000 7f97092c0640 1 -- 172.21.15.40:0/3378762676 learned_addr learned my addr 172.21.15.40:0/3378762676 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:55.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.635+0000 7f97092c0640 1 -- 172.21.15.40:0/3378762676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f970406f570 msgr2=0x7f97041588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:55.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.635+0000 7f97092c0640 1 --2- 172.21.15.40:0/3378762676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f970406f570 0x7f97041588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:55.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.635+0000 7f97092c0640 1 -- 172.21.15.40:0/3378762676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9704157800 msgr2=0x7f97041f5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:55.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.635+0000 7f97092c0640 1 --2- 172.21.15.40:0/3378762676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9704157800 0x7f97041f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:55.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.635+0000 7f97092c0640 1 -- 172.21.15.40:0/3378762676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f8009560 con 0x7f970406d660 2023-10-18T09:06:55.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.635+0000 7f97092c0640 1 --2- 172.21.15.40:0/3378762676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970406d660 0x7f970414f710 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f96f400c870 tx=0x7f96f400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:55.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.635+0000 7f9701ffb640 1 -- 172.21.15.40:0/3378762676 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f4003b70 con 0x7f970406d660 2023-10-18T09:06:55.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.635+0000 7f9701ffb640 1 -- 172.21.15.40:0/3378762676 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f96f4003cf0 con 0x7f970406d660 2023-10-18T09:06:55.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.635+0000 7f9701ffb640 1 -- 172.21.15.40:0/3378762676 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f4003ea0 con 0x7f970406d660 2023-10-18T09:06:55.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.635+0000 7f970b524640 1 -- 172.21.15.40:0/3378762676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97040abae0 con 0x7f970406d660 2023-10-18T09:06:55.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.635+0000 7f970b524640 1 -- 172.21.15.40:0/3378762676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97040ac0a0 con 0x7f970406d660 2023-10-18T09:06:55.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.637+0000 7f970b524640 1 -- 172.21.15.40:0/3378762676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97040ac230 con 0x7f970406d660 2023-10-18T09:06:55.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.637+0000 7f9701ffb640 1 -- 172.21.15.40:0/3378762676 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f96f400aab0 con 0x7f970406d660 2023-10-18T09:06:55.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.638+0000 7f9701ffb640 1 --2- 172.21.15.40:0/3378762676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f96f006ae40 0x7f96f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:55.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.639+0000 7f9701ffb640 1 -- 172.21.15.40:0/3378762676 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f96f4089500 con 0x7f970406d660 2023-10-18T09:06:55.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.639+0000 7f9701ffb640 1 -- 172.21.15.40:0/3378762676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f96f0072a30 con 0x7f970406d660 2023-10-18T09:06:55.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.639+0000 7f9708abf640 1 --2- 172.21.15.40:0/3378762676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f96f006ae40 0x7f96f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:55.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.640+0000 7f9708abf640 1 --2- 172.21.15.40:0/3378762676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f96f006ae40 0x7f96f006d300 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f96f8035000 tx=0x7f96f80046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:55.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.643+0000 7f9701ffb640 1 -- 172.21.15.40:0/3378762676 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f96f4054570 con 0x7f970406d660 2023-10-18T09:06:55.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.834+0000 7f970b524640 1 -- 172.21.15.40:0/3378762676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97040cc450 con 0x7f970406d660 2023-10-18T09:06:55.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.835+0000 7f9701ffb640 1 -- 172.21.15.40:0/3378762676 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v166) v1 ==== 100+0+11 (secure 0 0 0) 0x7f97040cc450 con 0x7f970406d660 2023-10-18T09:06:55.838 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:06:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.838+0000 7f970b524640 1 -- 172.21.15.40:0/3378762676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f96f006ae40 msgr2=0x7f96f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.838+0000 7f970b524640 1 --2- 172.21.15.40:0/3378762676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f96f006ae40 0x7f96f006d300 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f96f8035000 tx=0x7f96f80046c0 comp rx=0 tx=0).stop 2023-10-18T09:06:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.838+0000 7f970b524640 1 -- 172.21.15.40:0/3378762676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970406d660 msgr2=0x7f970414f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.838+0000 7f970b524640 1 --2- 172.21.15.40:0/3378762676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970406d660 0x7f970414f710 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f96f400c870 tx=0x7f96f400cd30 comp rx=0 tx=0).stop 2023-10-18T09:06:55.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.838+0000 7f970b524640 1 -- 172.21.15.40:0/3378762676 shutdown_connections 2023-10-18T09:06:55.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.838+0000 7f970b524640 1 --2- 172.21.15.40:0/3378762676 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f96f006ae40 0x7f96f006d300 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:55.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.838+0000 7f970b524640 1 --2- 172.21.15.40:0/3378762676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9704157800 0x7f97041f5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:55.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.838+0000 7f970b524640 1 --2- 172.21.15.40:0/3378762676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f970406f570 0x7f97041588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:55.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.838+0000 7f970b524640 1 --2- 172.21.15.40:0/3378762676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f970406d660 0x7f970414f710 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:55.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.838+0000 7f970b524640 1 -- 172.21.15.40:0/3378762676 >> 172.21.15.40:0/3378762676 conn(0x7f9704069720 msgr2=0x7f97040acbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:55.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.838+0000 7f970b524640 1 -- 172.21.15.40:0/3378762676 shutdown_connections 2023-10-18T09:06:55.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:55.838+0000 7f970b524640 1 -- 172.21.15.40:0/3378762676 wait complete. 2023-10-18T09:06:55.861 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:06:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.091+0000 7f67b99a9640 1 -- 172.21.15.40:0/2314687689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b40b1b90 msgr2=0x7f67b40cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:56.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.091+0000 7f67b99a9640 1 --2- 172.21.15.40:0/2314687689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b40b1b90 0x7f67b40cd2f0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f67b40ce190 tx=0x7f67a802bd90 comp rx=0 tx=0).stop 2023-10-18T09:06:56.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.091+0000 7f67b99a9640 1 -- 172.21.15.40:0/2314687689 shutdown_connections 2023-10-18T09:06:56.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.091+0000 7f67b99a9640 1 --2- 172.21.15.40:0/2314687689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67b406ffe0 0x7f67b406b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:56.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.091+0000 7f67b99a9640 1 --2- 172.21.15.40:0/2314687689 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b40b1b90 0x7f67b40cd2f0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:56.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.091+0000 7f67b99a9640 1 --2- 172.21.15.40:0/2314687689 >> v2:172.21.15.144:3300/0 conn(0x7f67b415eaa0 0x7f67b40ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:56.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.091+0000 7f67b99a9640 1 -- 172.21.15.40:0/2314687689 >> 172.21.15.40:0/2314687689 conn(0x7f67b406cee0 msgr2=0x7f67b4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:56.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.091+0000 7f67b99a9640 1 -- 172.21.15.40:0/2314687689 shutdown_connections 2023-10-18T09:06:56.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.091+0000 7f67b99a9640 1 -- 172.21.15.40:0/2314687689 wait complete. 2023-10-18T09:06:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.093+0000 7f67b99a9640 1 Processor -- start 2023-10-18T09:06:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.093+0000 7f67b99a9640 1 -- start start 2023-10-18T09:06:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.094+0000 7f67b99a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67b406ffe0 0x7f67b40c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:56.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.094+0000 7f67b99a9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67b40b1b90 0x7f67b4155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.094+0000 7f67b99a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b415eaa0 0x7f67b4157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.094+0000 7f67b99a9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b41598d0 con 0x7f67b415eaa0 2023-10-18T09:06:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.094+0000 7f67b99a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b4159bb0 con 0x7f67b406ffe0 2023-10-18T09:06:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.094+0000 7f67b99a9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b4159e90 con 0x7f67b40b1b90 2023-10-18T09:06:56.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.094+0000 7f67b37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b415eaa0 0x7f67b4157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:56.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.095+0000 7f67b37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b415eaa0 0x7f67b4157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43280/0 (socket says 172.21.15.40:43280) 2023-10-18T09:06:56.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.095+0000 7f67b37fe640 1 -- 172.21.15.40:0/3160384358 learned_addr learned my addr 172.21.15.40:0/3160384358 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:56.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.095+0000 7f67b2ffd640 1 --2- 172.21.15.40:0/3160384358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67b406ffe0 0x7f67b40c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:56.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.095+0000 7f67b27fc640 1 --2- 172.21.15.40:0/3160384358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67b40b1b90 0x7f67b4155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:56.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.095+0000 7f67b37fe640 1 -- 172.21.15.40:0/3160384358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67b40b1b90 msgr2=0x7f67b4155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:56.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.095+0000 7f67b37fe640 1 --2- 172.21.15.40:0/3160384358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67b40b1b90 0x7f67b4155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:56.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.095+0000 7f67b37fe640 1 -- 172.21.15.40:0/3160384358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67b406ffe0 msgr2=0x7f67b40c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:56.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.095+0000 7f67b37fe640 1 --2- 172.21.15.40:0/3160384358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67b406ffe0 0x7f67b40c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:56.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.095+0000 7f67b37fe640 1 -- 172.21.15.40:0/3160384358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a8009560 con 0x7f67b415eaa0 2023-10-18T09:06:56.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.095+0000 7f67b37fe640 1 --2- 172.21.15.40:0/3160384358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b415eaa0 0x7f67b4157500 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00ca40 tx=0x7f67ac00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:56.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.096+0000 7f679bfff640 1 -- 172.21.15.40:0/3160384358 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00adc0 con 0x7f67b415eaa0 2023-10-18T09:06:56.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.096+0000 7f679bfff640 1 -- 172.21.15.40:0/3160384358 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f67ac004c50 con 0x7f67b415eaa0 2023-10-18T09:06:56.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.096+0000 7f679bfff640 1 -- 172.21.15.40:0/3160384358 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac005280 con 0x7f67b415eaa0 2023-10-18T09:06:56.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.096+0000 7f67b99a9640 1 -- 172.21.15.40:0/3160384358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b415a170 con 0x7f67b415eaa0 2023-10-18T09:06:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.096+0000 7f67b99a9640 1 -- 172.21.15.40:0/3160384358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b40ac420 con 0x7f67b415eaa0 2023-10-18T09:06:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.097+0000 7f67b99a9640 1 -- 172.21.15.40:0/3160384358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b415a690 con 0x7f67b415eaa0 2023-10-18T09:06:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.098+0000 7f679bfff640 1 -- 172.21.15.40:0/3160384358 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f67ac0053e0 con 0x7f67b415eaa0 2023-10-18T09:06:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.099+0000 7f679bfff640 1 --2- 172.21.15.40:0/3160384358 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f67a006ae40 0x7f67a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.099+0000 7f679bfff640 1 -- 172.21.15.40:0/3160384358 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f67ac089f10 con 0x7f67b415eaa0 2023-10-18T09:06:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.099+0000 7f679bfff640 1 -- 172.21.15.40:0/3160384358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=167}) v3 -- 0x7f67a0072a30 con 0x7f67b415eaa0 2023-10-18T09:06:56.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.100+0000 7f67b2ffd640 1 --2- 172.21.15.40:0/3160384358 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f67a006ae40 0x7f67a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:56.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.101+0000 7f67b2ffd640 1 --2- 172.21.15.40:0/3160384358 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f67a006ae40 0x7f67a006d300 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f67a4005d70 tx=0x7f67a4005d00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:56.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.104+0000 7f679bfff640 1 -- 172.21.15.40:0/3160384358 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f67ac01d070 con 0x7f67b415eaa0 2023-10-18T09:06:56.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.155+0000 7f679bfff640 1 -- 172.21.15.40:0/3160384358 <== mon.0 v2:172.21.15.40:3300/0 7 ==== osd_map(167..167 src has 1..167) v4 ==== 737+0+0 (secure 0 0 0) 0x7f67ac004e40 con 0x7f67b415eaa0 2023-10-18T09:06:56.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.155+0000 7f679bfff640 1 -- 172.21.15.40:0/3160384358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=168}) v3 -- 0x7f67a0073560 con 0x7f67b415eaa0 2023-10-18T09:06:56.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.326+0000 7f67b99a9640 1 -- 172.21.15.40:0/3160384358 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f67b4070c90 con 0x7f67b415eaa0 2023-10-18T09:06:56.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.327+0000 7f679bfff640 1 -- 172.21.15.40:0/3160384358 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v167) v1 ==== 80+0+0 (secure 0 0 0) 0x7f67b415a690 con 0x7f67b415eaa0 2023-10-18T09:06:56.329 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T09:06:56.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.330+0000 7f67b99a9640 1 -- 172.21.15.40:0/3160384358 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f67a006ae40 msgr2=0x7f67a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:56.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.330+0000 7f67b99a9640 1 --2- 172.21.15.40:0/3160384358 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f67a006ae40 0x7f67a006d300 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f67a4005d70 tx=0x7f67a4005d00 comp rx=0 tx=0).stop 2023-10-18T09:06:56.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.330+0000 7f67b99a9640 1 -- 172.21.15.40:0/3160384358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b415eaa0 msgr2=0x7f67b4157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:56.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.330+0000 7f67b99a9640 1 --2- 172.21.15.40:0/3160384358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b415eaa0 0x7f67b4157500 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00ca40 tx=0x7f67ac00cf00 comp rx=0 tx=0).stop 2023-10-18T09:06:56.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.330+0000 7f67b99a9640 1 -- 172.21.15.40:0/3160384358 shutdown_connections 2023-10-18T09:06:56.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.330+0000 7f67b99a9640 1 --2- 172.21.15.40:0/3160384358 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f67a006ae40 0x7f67a006d300 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:56.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.330+0000 7f67b99a9640 1 --2- 172.21.15.40:0/3160384358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67b415eaa0 0x7f67b4157500 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:56.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.330+0000 7f67b99a9640 1 --2- 172.21.15.40:0/3160384358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67b40b1b90 0x7f67b4155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:56.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.330+0000 7f67b99a9640 1 --2- 172.21.15.40:0/3160384358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67b406ffe0 0x7f67b40c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:56.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.330+0000 7f67b99a9640 1 -- 172.21.15.40:0/3160384358 >> 172.21.15.40:0/3160384358 conn(0x7f67b406cee0 msgr2=0x7f67b40ae270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:56.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.331+0000 7f67b99a9640 1 -- 172.21.15.40:0/3160384358 shutdown_connections 2023-10-18T09:06:56.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.331+0000 7f67b99a9640 1 -- 172.21.15.40:0/3160384358 wait complete. 2023-10-18T09:06:56.353 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-18T09:06:56.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:06:56.354 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-18T09:06:56.464 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T09:06:56.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:06:56.465 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T09:06:56.522 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:06:56.523 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:06:56.586 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:06:56.586 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:06:56.587 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:06:56.646 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:06:56.647 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:06:56.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.884+0000 7fc8d9a1a640 1 -- 172.21.15.40:0/1915057006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8bc005180 msgr2=0x7fc8bc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:56.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.884+0000 7fc8d9a1a640 1 --2- 172.21.15.40:0/1915057006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8bc005180 0x7fc8bc00b980 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00c710 tx=0x7fc8bc0057f0 comp rx=0 tx=0).stop 2023-10-18T09:06:56.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.885+0000 7fc8d9a1a640 1 -- 172.21.15.40:0/1915057006 shutdown_connections 2023-10-18T09:06:56.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.885+0000 7fc8d9a1a640 1 --2- 172.21.15.40:0/1915057006 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8bc005180 0x7fc8bc00b980 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:56.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.885+0000 7fc8d9a1a640 1 -- 172.21.15.40:0/1915057006 >> 172.21.15.40:0/1915057006 conn(0x7fc8d406cee0 msgr2=0x7fc8d4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:56.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.885+0000 7fc8d9a1a640 1 -- 172.21.15.40:0/1915057006 shutdown_connections 2023-10-18T09:06:56.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.885+0000 7fc8d9a1a640 1 -- 172.21.15.40:0/1915057006 wait complete. 2023-10-18T09:06:56.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.887+0000 7fc8d9a1a640 1 Processor -- start 2023-10-18T09:06:56.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.887+0000 7fc8d9a1a640 1 -- start start 2023-10-18T09:06:56.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.889+0000 7fc8d9a1a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8bc005180 0x7fc8d4159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:56.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.889+0000 7fc8d9a1a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8d4155670 0x7fc8d4155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:56.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.889+0000 7fc8d9a1a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d40cc450 0x7fc8d415af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:56.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.889+0000 7fc8d9a1a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d40cbfc0 con 0x7fc8bc005180 2023-10-18T09:06:56.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.889+0000 7fc8d9a1a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d40cb3b0 con 0x7fc8d40cc450 2023-10-18T09:06:56.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.889+0000 7fc8d9a1a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d40cb690 con 0x7fc8d4155670 2023-10-18T09:06:56.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.889+0000 7fc8d2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8bc005180 0x7fc8d4159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:56.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.889+0000 7fc8d2ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8bc005180 0x7fc8d4159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43296/0 (socket says 172.21.15.40:43296) 2023-10-18T09:06:56.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.889+0000 7fc8d2ffd640 1 -- 172.21.15.40:0/1040310792 learned_addr learned my addr 172.21.15.40:0/1040310792 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:56.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.889+0000 7fc8d27fc640 1 --2- 172.21.15.40:0/1040310792 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8d4155670 0x7fc8d4155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:56.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.889+0000 7fc8d37fe640 1 --2- 172.21.15.40:0/1040310792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d40cc450 0x7fc8d415af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:56.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.889+0000 7fc8d2ffd640 1 -- 172.21.15.40:0/1040310792 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8d4155670 msgr2=0x7fc8d4155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:56.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.889+0000 7fc8d2ffd640 1 --2- 172.21.15.40:0/1040310792 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8d4155670 0x7fc8d4155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:56.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.889+0000 7fc8d2ffd640 1 -- 172.21.15.40:0/1040310792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d40cc450 msgr2=0x7fc8d415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:56.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.889+0000 7fc8d2ffd640 1 --2- 172.21.15.40:0/1040310792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d40cc450 0x7fc8d415af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:56.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.889+0000 7fc8d2ffd640 1 -- 172.21.15.40:0/1040310792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d40cb970 con 0x7fc8bc005180 2023-10-18T09:06:56.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.890+0000 7fc8d2ffd640 1 --2- 172.21.15.40:0/1040310792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8bc005180 0x7fc8d4159e00 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400c730 tx=0x7fc8c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:56.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.890+0000 7fc8bbfff640 1 -- 172.21.15.40:0/1040310792 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4005270 con 0x7fc8bc005180 2023-10-18T09:06:56.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.890+0000 7fc8bbfff640 1 -- 172.21.15.40:0/1040310792 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc8c400a640 con 0x7fc8bc005180 2023-10-18T09:06:56.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.890+0000 7fc8d9a1a640 1 -- 172.21.15.40:0/1040310792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d40cbc50 con 0x7fc8bc005180 2023-10-18T09:06:56.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.890+0000 7fc8bbfff640 1 -- 172.21.15.40:0/1040310792 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c400aa30 con 0x7fc8bc005180 2023-10-18T09:06:56.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.890+0000 7fc8d9a1a640 1 -- 172.21.15.40:0/1040310792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d41564c0 con 0x7fc8bc005180 2023-10-18T09:06:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.892+0000 7fc8d9a1a640 1 -- 172.21.15.40:0/1040310792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d40b1730 con 0x7fc8bc005180 2023-10-18T09:06:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.892+0000 7fc8bbfff640 1 -- 172.21.15.40:0/1040310792 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc8c400ab90 con 0x7fc8bc005180 2023-10-18T09:06:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.894+0000 7fc8bbfff640 1 --2- 172.21.15.40:0/1040310792 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8c006ae40 0x7fc8c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.894+0000 7fc8bbfff640 1 -- 172.21.15.40:0/1040310792 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc8c4088f30 con 0x7fc8bc005180 2023-10-18T09:06:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.894+0000 7fc8d27fc640 1 --2- 172.21.15.40:0/1040310792 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8c006ae40 0x7fc8c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.896+0000 7fc8d27fc640 1 --2- 172.21.15.40:0/1040310792 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8c006ae40 0x7fc8c006d300 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8005b40 tx=0x7fc8c8009710 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:56.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:56.898+0000 7fc8bbfff640 1 -- 172.21.15.40:0/1040310792 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc8c4054040 con 0x7fc8bc005180 2023-10-18T09:06:57.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.132+0000 7fc8d9a1a640 1 -- 172.21.15.40:0/1040310792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8d40cbde0 con 0x7fc8bc005180 2023-10-18T09:06:57.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.133+0000 7fc8bbfff640 1 -- 172.21.15.40:0/1040310792 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fc8c4017070 con 0x7fc8bc005180 2023-10-18T09:06:57.136 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:06:57.136 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3243865919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3243865919},{"type":"v1","addr":"172.21.15.40:6835","nonce":3243865919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6047,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/504515257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":504515257},{"type":"v1","addr":"172.21.15.144:6835","nonce":504515257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2513603652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2513603652},{"type":"v1","addr":"172.21.15.144:6837","nonce":2513603652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:04:36.950432+0000","modified":"2023-10-18T09:04:40.030449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8388":{"gid":8388,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/3348437493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3348437493},{"type":"v1","addr":"172.21.15.40:6833","nonce":3348437493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-18T09:06:57.137 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 74 2023-10-18T09:06:57.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.136+0000 7fc8d9a1a640 1 -- 172.21.15.40:0/1040310792 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8c006ae40 msgr2=0x7fc8c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:57.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.136+0000 7fc8d9a1a640 1 --2- 172.21.15.40:0/1040310792 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8c006ae40 0x7fc8c006d300 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8005b40 tx=0x7fc8c8009710 comp rx=0 tx=0).stop 2023-10-18T09:06:57.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.136+0000 7fc8d9a1a640 1 -- 172.21.15.40:0/1040310792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8bc005180 msgr2=0x7fc8d4159e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.136+0000 7fc8d9a1a640 1 --2- 172.21.15.40:0/1040310792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8bc005180 0x7fc8d4159e00 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400c730 tx=0x7fc8c400cbf0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.136+0000 7fc8d9a1a640 1 -- 172.21.15.40:0/1040310792 shutdown_connections 2023-10-18T09:06:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.136+0000 7fc8d9a1a640 1 --2- 172.21.15.40:0/1040310792 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc8c006ae40 0x7fc8c006d300 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.136+0000 7fc8d9a1a640 1 --2- 172.21.15.40:0/1040310792 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8d40cc450 0x7fc8d415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.136+0000 7fc8d9a1a640 1 --2- 172.21.15.40:0/1040310792 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8d4155670 0x7fc8d4155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.136+0000 7fc8d9a1a640 1 --2- 172.21.15.40:0/1040310792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8bc005180 0x7fc8d4159e00 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.137+0000 7fc8d9a1a640 1 -- 172.21.15.40:0/1040310792 >> 172.21.15.40:0/1040310792 conn(0x7fc8d406cee0 msgr2=0x7fc8d40ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:57.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.137+0000 7fc8d9a1a640 1 -- 172.21.15.40:0/1040310792 shutdown_connections 2023-10-18T09:06:57.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.137+0000 7fc8d9a1a640 1 -- 172.21.15.40:0/1040310792 wait complete. 2023-10-18T09:06:57.159 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:06:57.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.394+0000 7f0a13fff640 1 -- 172.21.15.40:0/4034926424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a1406d6f0 msgr2=0x7f0a140b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:57.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.394+0000 7f0a13fff640 1 --2- 172.21.15.40:0/4034926424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a1406d6f0 0x7f0a140b15b0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009830 tx=0x7f0a0402bd60 comp rx=0 tx=0).stop 2023-10-18T09:06:57.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.394+0000 7f0a13fff640 1 -- 172.21.15.40:0/4034926424 shutdown_connections 2023-10-18T09:06:57.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.394+0000 7f0a13fff640 1 --2- 172.21.15.40:0/4034926424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a1406d6f0 0x7f0a140b15b0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.394+0000 7f0a13fff640 1 -- 172.21.15.40:0/4034926424 >> 172.21.15.40:0/4034926424 conn(0x7f0a14069720 msgr2=0x7f0a14069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:57.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.394+0000 7f0a13fff640 1 -- 172.21.15.40:0/4034926424 shutdown_connections 2023-10-18T09:06:57.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.394+0000 7f0a13fff640 1 -- 172.21.15.40:0/4034926424 wait complete. 2023-10-18T09:06:57.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.396+0000 7f0a13fff640 1 Processor -- start 2023-10-18T09:06:57.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.396+0000 7f0a13fff640 1 -- start start 2023-10-18T09:06:57.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.397+0000 7f0a13fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a1406d6f0 0x7f0a1414b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:57.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.397+0000 7f0a13fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a140b0630 0x7f0a14155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:57.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.397+0000 7f0a13fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a140c99c0 0x7f0a14155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:57.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.397+0000 7f0a13fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a14151330 con 0x7f0a1406d6f0 2023-10-18T09:06:57.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.397+0000 7f0a13fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a14151610 con 0x7f0a140b0630 2023-10-18T09:06:57.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.397+0000 7f0a13fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a141518f0 con 0x7f0a140c99c0 2023-10-18T09:06:57.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.398+0000 7f0a12ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a1406d6f0 0x7f0a1414b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:57.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.398+0000 7f0a12ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a1406d6f0 0x7f0a1414b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43306/0 (socket says 172.21.15.40:43306) 2023-10-18T09:06:57.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.398+0000 7f0a12ffd640 1 -- 172.21.15.40:0/2803955276 learned_addr learned my addr 172.21.15.40:0/2803955276 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:57.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.398+0000 7f0a12ffd640 1 -- 172.21.15.40:0/2803955276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a140c99c0 msgr2=0x7f0a14155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:57.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.398+0000 7f0a12ffd640 1 --2- 172.21.15.40:0/2803955276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a140c99c0 0x7f0a14155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.398+0000 7f0a0a7fc640 1 --2- 172.21.15.40:0/2803955276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a140b0630 0x7f0a14155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:57.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.398+0000 7f0a12ffd640 1 -- 172.21.15.40:0/2803955276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a140b0630 msgr2=0x7f0a14155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:57.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.398+0000 7f0a12ffd640 1 --2- 172.21.15.40:0/2803955276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a140b0630 0x7f0a14155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.398+0000 7f0a12ffd640 1 -- 172.21.15.40:0/2803955276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a04009490 con 0x7f0a1406d6f0 2023-10-18T09:06:57.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.398+0000 7f0a0a7fc640 1 --2- 172.21.15.40:0/2803955276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a140b0630 0x7f0a14155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:06:57.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.399+0000 7f0a12ffd640 1 --2- 172.21.15.40:0/2803955276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a1406d6f0 0x7f0a1414b840 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009780 tx=0x7f0a040038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:57.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.399+0000 7f0a0bfff640 1 -- 172.21.15.40:0/2803955276 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04003ea0 con 0x7f0a1406d6f0 2023-10-18T09:06:57.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.399+0000 7f0a0bfff640 1 -- 172.21.15.40:0/2803955276 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0a04034550 con 0x7f0a1406d6f0 2023-10-18T09:06:57.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.399+0000 7f0a0bfff640 1 -- 172.21.15.40:0/2803955276 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04034960 con 0x7f0a1406d6f0 2023-10-18T09:06:57.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.399+0000 7f0a13fff640 1 -- 172.21.15.40:0/2803955276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a14151bd0 con 0x7f0a1406d6f0 2023-10-18T09:06:57.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.399+0000 7f0a13fff640 1 -- 172.21.15.40:0/2803955276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a14150f50 con 0x7f0a1406d6f0 2023-10-18T09:06:57.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.401+0000 7f0a13fff640 1 -- 172.21.15.40:0/2803955276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a141520f0 con 0x7f0a1406d6f0 2023-10-18T09:06:57.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.402+0000 7f0a0bfff640 1 -- 172.21.15.40:0/2803955276 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0a0403b040 con 0x7f0a1406d6f0 2023-10-18T09:06:57.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.403+0000 7f0a0bfff640 1 --2- 172.21.15.40:0/2803955276 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09f406ae40 0x7f09f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:57.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.403+0000 7f0a0bfff640 1 -- 172.21.15.40:0/2803955276 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0a040abce0 con 0x7f0a1406d6f0 2023-10-18T09:06:57.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.404+0000 7f0a0a7fc640 1 --2- 172.21.15.40:0/2803955276 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09f406ae40 0x7f09f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:57.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.405+0000 7f0a0a7fc640 1 --2- 172.21.15.40:0/2803955276 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09f406ae40 0x7f09f406d300 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0a14158e20 tx=0x7f09fc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:57.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.408+0000 7f0a0bfff640 1 -- 172.21.15.40:0/2803955276 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0a141520f0 con 0x7f0a1406d6f0 2023-10-18T09:06:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.602+0000 7f0a13fff640 1 -- 172.21.15.40:0/2803955276 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a141520f0 con 0x7f0a1406d6f0 2023-10-18T09:06:57.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.605+0000 7f0a0bfff640 1 -- 172.21.15.40:0/2803955276 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f0a141520f0 con 0x7f0a1406d6f0 2023-10-18T09:06:57.608 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:06:57.608 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":167,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:06:56.150783+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:04:34.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-18T09:04:35.972659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.26","mappings":[{"from":2,"to":0}]},{"pgid":"19.33","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:06:57.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.607+0000 7f0a13fff640 1 -- 172.21.15.40:0/2803955276 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09f406ae40 msgr2=0x7f09f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:57.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.607+0000 7f0a13fff640 1 --2- 172.21.15.40:0/2803955276 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09f406ae40 0x7f09f406d300 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0a14158e20 tx=0x7f09fc009040 comp rx=0 tx=0).stop 2023-10-18T09:06:57.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.608+0000 7f0a13fff640 1 -- 172.21.15.40:0/2803955276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a1406d6f0 msgr2=0x7f0a1414b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:57.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.608+0000 7f0a13fff640 1 --2- 172.21.15.40:0/2803955276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a1406d6f0 0x7f0a1414b840 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009780 tx=0x7f0a040038a0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.608+0000 7f0a13fff640 1 -- 172.21.15.40:0/2803955276 shutdown_connections 2023-10-18T09:06:57.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.608+0000 7f0a13fff640 1 --2- 172.21.15.40:0/2803955276 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f09f406ae40 0x7f09f406d300 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.608+0000 7f0a13fff640 1 --2- 172.21.15.40:0/2803955276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a140c99c0 0x7f0a14155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.608+0000 7f0a13fff640 1 --2- 172.21.15.40:0/2803955276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a140b0630 0x7f0a14155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.608+0000 7f0a13fff640 1 --2- 172.21.15.40:0/2803955276 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a1406d6f0 0x7f0a1414b840 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.608+0000 7f0a13fff640 1 -- 172.21.15.40:0/2803955276 >> 172.21.15.40:0/2803955276 conn(0x7f0a14069720 msgr2=0x7f0a14152b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:57.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.608+0000 7f0a13fff640 1 -- 172.21.15.40:0/2803955276 shutdown_connections 2023-10-18T09:06:57.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.608+0000 7f0a13fff640 1 -- 172.21.15.40:0/2803955276 wait complete. 2023-10-18T09:06:57.628 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:04:34.597892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '161', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:04:35.972659+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '167', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:06:57.629 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:06:57.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.853+0000 7fc983fff640 1 -- 172.21.15.40:0/3291475541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc98415b8b0 msgr2=0x7fc9840d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:57.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.853+0000 7fc983fff640 1 --2- 172.21.15.40:0/3291475541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc98415b8b0 0x7fc9840d1250 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc978005fd0 tx=0x7fc97802beb0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.853+0000 7fc983fff640 1 -- 172.21.15.40:0/3291475541 shutdown_connections 2023-10-18T09:06:57.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.853+0000 7fc983fff640 1 --2- 172.21.15.40:0/3291475541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc9840d1790 0x7fc9840cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.853+0000 7fc983fff640 1 --2- 172.21.15.40:0/3291475541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc98415b8b0 0x7fc9840d1250 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.853+0000 7fc983fff640 1 -- 172.21.15.40:0/3291475541 >> 172.21.15.40:0/3291475541 conn(0x7fc98408ad30 msgr2=0x7fc98409a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:57.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.853+0000 7fc983fff640 1 -- 172.21.15.40:0/3291475541 shutdown_connections 2023-10-18T09:06:57.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.853+0000 7fc983fff640 1 -- 172.21.15.40:0/3291475541 wait complete. 2023-10-18T09:06:57.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.855+0000 7fc983fff640 1 Processor -- start 2023-10-18T09:06:57.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.855+0000 7fc983fff640 1 -- start start 2023-10-18T09:06:57.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.856+0000 7fc983fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9840d1790 0x7fc98415e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:57.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.856+0000 7fc983fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc98415b4d0 0x7fc9840b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:57.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.856+0000 7fc983fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc98415b8b0 0x7fc984158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:57.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.856+0000 7fc983fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98415cd10 con 0x7fc9840d1790 2023-10-18T09:06:57.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.856+0000 7fc983fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98415cf90 con 0x7fc98415b4d0 2023-10-18T09:06:57.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.856+0000 7fc983fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98415d270 con 0x7fc98415b8b0 2023-10-18T09:06:57.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.856+0000 7fc982ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9840d1790 0x7fc98415e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:57.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.856+0000 7fc982ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9840d1790 0x7fc98415e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43320/0 (socket says 172.21.15.40:43320) 2023-10-18T09:06:57.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.856+0000 7fc982ffd640 1 -- 172.21.15.40:0/2406780212 learned_addr learned my addr 172.21.15.40:0/2406780212 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:57.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.857+0000 7fc9827fc640 1 --2- 172.21.15.40:0/2406780212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc98415b4d0 0x7fc9840b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:57.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.857+0000 7fc9837fe640 1 --2- 172.21.15.40:0/2406780212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc98415b8b0 0x7fc984158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:57.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.857+0000 7fc982ffd640 1 -- 172.21.15.40:0/2406780212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc98415b8b0 msgr2=0x7fc984158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:57.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.857+0000 7fc982ffd640 1 --2- 172.21.15.40:0/2406780212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc98415b8b0 0x7fc984158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.857+0000 7fc982ffd640 1 -- 172.21.15.40:0/2406780212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc98415b4d0 msgr2=0x7fc9840b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:57.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.857+0000 7fc982ffd640 1 --2- 172.21.15.40:0/2406780212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc98415b4d0 0x7fc9840b4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:57.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.857+0000 7fc982ffd640 1 -- 172.21.15.40:0/2406780212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c009840 con 0x7fc9840d1790 2023-10-18T09:06:57.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.857+0000 7fc9837fe640 1 --2- 172.21.15.40:0/2406780212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc98415b8b0 0x7fc984158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:06:57.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.857+0000 7fc982ffd640 1 --2- 172.21.15.40:0/2406780212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9840d1790 0x7fc98415e030 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fc97400c420 tx=0x7fc97400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:57.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.857+0000 7fc96b7fe640 1 -- 172.21.15.40:0/2406780212 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9740052d0 con 0x7fc9840d1790 2023-10-18T09:06:57.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.858+0000 7fc96b7fe640 1 -- 172.21.15.40:0/2406780212 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc9740035f0 con 0x7fc9840d1790 2023-10-18T09:06:57.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.858+0000 7fc96b7fe640 1 -- 172.21.15.40:0/2406780212 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974003a20 con 0x7fc9840d1790 2023-10-18T09:06:57.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.858+0000 7fc983fff640 1 -- 172.21.15.40:0/2406780212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc978009560 con 0x7fc9840d1790 2023-10-18T09:06:57.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.858+0000 7fc983fff640 1 -- 172.21.15.40:0/2406780212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98415d830 con 0x7fc9840d1790 2023-10-18T09:06:57.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.859+0000 7fc983fff640 1 -- 172.21.15.40:0/2406780212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc984071da0 con 0x7fc9840d1790 2023-10-18T09:06:57.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.860+0000 7fc96b7fe640 1 -- 172.21.15.40:0/2406780212 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc974003b80 con 0x7fc9840d1790 2023-10-18T09:06:57.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.861+0000 7fc96b7fe640 1 --2- 172.21.15.40:0/2406780212 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc97006ae40 0x7fc97006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:57.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.861+0000 7fc96b7fe640 1 -- 172.21.15.40:0/2406780212 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc974088f60 con 0x7fc9840d1790 2023-10-18T09:06:57.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.862+0000 7fc9827fc640 1 --2- 172.21.15.40:0/2406780212 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc97006ae40 0x7fc97006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:57.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.863+0000 7fc9827fc640 1 --2- 172.21.15.40:0/2406780212 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc97006ae40 0x7fc97006d300 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc978035000 tx=0x7fc97802e040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:57.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:57.865+0000 7fc96b7fe640 1 -- 172.21.15.40:0/2406780212 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc974053f30 con 0x7fc9840d1790 2023-10-18T09:06:58.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.058+0000 7fc983fff640 1 -- 172.21.15.40:0/2406780212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc984066e00 con 0x7fc9840d1790 2023-10-18T09:06:58.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.059+0000 7fc96b7fe640 1 -- 172.21.15.40:0/2406780212 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v167) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9740170a0 con 0x7fc9840d1790 2023-10-18T09:06:58.062 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:06:58.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.062+0000 7fc983fff640 1 -- 172.21.15.40:0/2406780212 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc97006ae40 msgr2=0x7fc97006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:58.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.062+0000 7fc983fff640 1 --2- 172.21.15.40:0/2406780212 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc97006ae40 0x7fc97006d300 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc978035000 tx=0x7fc97802e040 comp rx=0 tx=0).stop 2023-10-18T09:06:58.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.062+0000 7fc983fff640 1 -- 172.21.15.40:0/2406780212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9840d1790 msgr2=0x7fc98415e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:58.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.062+0000 7fc983fff640 1 --2- 172.21.15.40:0/2406780212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9840d1790 0x7fc98415e030 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fc97400c420 tx=0x7fc97400c8e0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.062+0000 7fc983fff640 1 -- 172.21.15.40:0/2406780212 shutdown_connections 2023-10-18T09:06:58.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.062+0000 7fc983fff640 1 --2- 172.21.15.40:0/2406780212 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc97006ae40 0x7fc97006d300 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.062+0000 7fc983fff640 1 --2- 172.21.15.40:0/2406780212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc98415b8b0 0x7fc984158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.062+0000 7fc983fff640 1 --2- 172.21.15.40:0/2406780212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc98415b4d0 0x7fc9840b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.062+0000 7fc983fff640 1 --2- 172.21.15.40:0/2406780212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9840d1790 0x7fc98415e030 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.062+0000 7fc983fff640 1 -- 172.21.15.40:0/2406780212 >> 172.21.15.40:0/2406780212 conn(0x7fc98408ad30 msgr2=0x7fc98415a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:58.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.062+0000 7fc983fff640 1 -- 172.21.15.40:0/2406780212 shutdown_connections 2023-10-18T09:06:58.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.062+0000 7fc983fff640 1 -- 172.21.15.40:0/2406780212 wait complete. 2023-10-18T09:06:58.084 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:06:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.316+0000 7f5f1759e640 1 -- 172.21.15.40:0/3559195146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1806c2d0 msgr2=0x7f5f18070b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.316+0000 7f5f1759e640 1 --2- 172.21.15.40:0/3559195146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1806c2d0 0x7f5f18070b60 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f5f00005fd0 tx=0x7f5f0002bd90 comp rx=0 tx=0).stop 2023-10-18T09:06:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.316+0000 7f5f1759e640 1 -- 172.21.15.40:0/3559195146 shutdown_connections 2023-10-18T09:06:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.316+0000 7f5f1759e640 1 --2- 172.21.15.40:0/3559195146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1806c2d0 0x7f5f18070b60 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.316+0000 7f5f1759e640 1 -- 172.21.15.40:0/3559195146 >> 172.21.15.40:0/3559195146 conn(0x7f5f1806ca10 msgr2=0x7f5f1806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:58.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.316+0000 7f5f1759e640 1 -- 172.21.15.40:0/3559195146 shutdown_connections 2023-10-18T09:06:58.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.316+0000 7f5f1759e640 1 -- 172.21.15.40:0/3559195146 wait complete. 2023-10-18T09:06:58.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.318+0000 7f5f1759e640 1 Processor -- start 2023-10-18T09:06:58.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.318+0000 7f5f1759e640 1 -- start start 2023-10-18T09:06:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.319+0000 7f5f1759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1806c2d0 0x7f5f180b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.319+0000 7f5f1759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f180c9620 0x7f5f18141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.319+0000 7f5f1759e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f1815af70 0x7f5f1814c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.319+0000 7f5f1759e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f181464d0 con 0x7f5f1806c2d0 2023-10-18T09:06:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.319+0000 7f5f1759e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f18142090 con 0x7f5f180c9620 2023-10-18T09:06:58.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.319+0000 7f5f1759e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180b0fa0 con 0x7f5f1815af70 2023-10-18T09:06:58.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.319+0000 7f5f1659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1806c2d0 0x7f5f180b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:58.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.320+0000 7f5f1659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1806c2d0 0x7f5f180b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43348/0 (socket says 172.21.15.40:43348) 2023-10-18T09:06:58.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.320+0000 7f5f1659c640 1 -- 172.21.15.40:0/1581986327 learned_addr learned my addr 172.21.15.40:0/1581986327 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:58.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.320+0000 7f5f1659c640 1 -- 172.21.15.40:0/1581986327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f1815af70 msgr2=0x7f5f1814c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:58.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.320+0000 7f5f1659c640 1 --2- 172.21.15.40:0/1581986327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f1815af70 0x7f5f1814c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.320+0000 7f5f0ffff640 1 --2- 172.21.15.40:0/1581986327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f180c9620 0x7f5f18141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:58.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.320+0000 7f5f1659c640 1 -- 172.21.15.40:0/1581986327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f180c9620 msgr2=0x7f5f18141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:58.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.320+0000 7f5f1659c640 1 --2- 172.21.15.40:0/1581986327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f180c9620 0x7f5f18141b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.320+0000 7f5f1659c640 1 -- 172.21.15.40:0/1581986327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f00009560 con 0x7f5f1806c2d0 2023-10-18T09:06:58.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.320+0000 7f5f1659c640 1 --2- 172.21.15.40:0/1581986327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1806c2d0 0x7f5f180b08a0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800c410 tx=0x7f5f0800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:58.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.321+0000 7f5f0effd640 1 -- 172.21.15.40:0/1581986327 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f080035f0 con 0x7f5f1806c2d0 2023-10-18T09:06:58.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.321+0000 7f5f0effd640 1 -- 172.21.15.40:0/1581986327 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5f08003750 con 0x7f5f1806c2d0 2023-10-18T09:06:58.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.321+0000 7f5f0effd640 1 -- 172.21.15.40:0/1581986327 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08003ba0 con 0x7f5f1806c2d0 2023-10-18T09:06:58.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.321+0000 7f5f1759e640 1 -- 172.21.15.40:0/1581986327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1815a310 con 0x7f5f1806c2d0 2023-10-18T09:06:58.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.321+0000 7f5f1759e640 1 -- 172.21.15.40:0/1581986327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1815a8d0 con 0x7f5f1806c2d0 2023-10-18T09:06:58.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.322+0000 7f5f1759e640 1 -- 172.21.15.40:0/1581986327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f18066e00 con 0x7f5f1806c2d0 2023-10-18T09:06:58.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.323+0000 7f5f0effd640 1 -- 172.21.15.40:0/1581986327 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5f08003d00 con 0x7f5f1806c2d0 2023-10-18T09:06:58.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.324+0000 7f5f0effd640 1 --2- 172.21.15.40:0/1581986327 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ef806ae40 0x7f5ef806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:58.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.324+0000 7f5f0effd640 1 -- 172.21.15.40:0/1581986327 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5f08088a40 con 0x7f5f1806c2d0 2023-10-18T09:06:58.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.324+0000 7f5f0ffff640 1 --2- 172.21.15.40:0/1581986327 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ef806ae40 0x7f5ef806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:58.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.325+0000 7f5f0ffff640 1 --2- 172.21.15.40:0/1581986327 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ef806ae40 0x7f5ef806d300 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5f00035000 tx=0x7f5f00004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:58.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.328+0000 7f5f0effd640 1 -- 172.21.15.40:0/1581986327 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f08053aa0 con 0x7f5f1806c2d0 2023-10-18T09:06:58.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.513+0000 7f5f1759e640 1 -- 172.21.15.40:0/1581986327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f1814fa20 con 0x7f5f1806c2d0 2023-10-18T09:06:58.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.514+0000 7f5f0effd640 1 -- 172.21.15.40:0/1581986327 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v167) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f08017020 con 0x7f5f1806c2d0 2023-10-18T09:06:58.517 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:06:58.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.517+0000 7f5f1759e640 1 -- 172.21.15.40:0/1581986327 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ef806ae40 msgr2=0x7f5ef806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:58.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.517+0000 7f5f1759e640 1 --2- 172.21.15.40:0/1581986327 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ef806ae40 0x7f5ef806d300 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5f00035000 tx=0x7f5f00004730 comp rx=0 tx=0).stop 2023-10-18T09:06:58.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.517+0000 7f5f1759e640 1 -- 172.21.15.40:0/1581986327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1806c2d0 msgr2=0x7f5f180b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:58.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.517+0000 7f5f1759e640 1 --2- 172.21.15.40:0/1581986327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1806c2d0 0x7f5f180b08a0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800c410 tx=0x7f5f0800c8d0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.517+0000 7f5f1759e640 1 -- 172.21.15.40:0/1581986327 shutdown_connections 2023-10-18T09:06:58.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.517+0000 7f5f1759e640 1 --2- 172.21.15.40:0/1581986327 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5ef806ae40 0x7f5ef806d300 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.517+0000 7f5f1759e640 1 --2- 172.21.15.40:0/1581986327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f1815af70 0x7f5f1814c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.517+0000 7f5f1759e640 1 --2- 172.21.15.40:0/1581986327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f180c9620 0x7f5f18141b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.517+0000 7f5f1759e640 1 --2- 172.21.15.40:0/1581986327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1806c2d0 0x7f5f180b08a0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.517+0000 7f5f1759e640 1 -- 172.21.15.40:0/1581986327 >> 172.21.15.40:0/1581986327 conn(0x7f5f1806ca10 msgr2=0x7f5f18155510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:58.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.517+0000 7f5f1759e640 1 -- 172.21.15.40:0/1581986327 shutdown_connections 2023-10-18T09:06:58.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.517+0000 7f5f1759e640 1 -- 172.21.15.40:0/1581986327 wait complete. 2023-10-18T09:06:58.537 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:06:58.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.758+0000 7f15f3143640 1 -- 172.21.15.40:0/1858369746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15ec0b1de0 msgr2=0x7f15ec0affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:58.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.758+0000 7f15f3143640 1 --2- 172.21.15.40:0/1858369746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15ec0b1de0 0x7f15ec0affb0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f15e4005fd0 tx=0x7f15e402c260 comp rx=0 tx=0).stop 2023-10-18T09:06:58.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.758+0000 7f15f3143640 1 -- 172.21.15.40:0/1858369746 shutdown_connections 2023-10-18T09:06:58.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.758+0000 7f15f3143640 1 --2- 172.21.15.40:0/1858369746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15ec0b1de0 0x7f15ec0affb0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.758+0000 7f15f3143640 1 -- 172.21.15.40:0/1858369746 >> 172.21.15.40:0/1858369746 conn(0x7f15ec09a580 msgr2=0x7f15ec0d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:58.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.758+0000 7f15f3143640 1 -- 172.21.15.40:0/1858369746 shutdown_connections 2023-10-18T09:06:58.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.758+0000 7f15f3143640 1 -- 172.21.15.40:0/1858369746 wait complete. 2023-10-18T09:06:58.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.760+0000 7f15f3143640 1 Processor -- start 2023-10-18T09:06:58.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.761+0000 7f15f3143640 1 -- start start 2023-10-18T09:06:58.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.762+0000 7f15f3143640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15ec0b1de0 0x7f15ec1571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:58.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.762+0000 7f15f3143640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0ce2d0 0x7f15ec15e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:58.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.762+0000 7f15f3143640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15ec15ed60 0x7f15ec0cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:58.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.762+0000 7f15f3143640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec15e5d0 con 0x7f15ec15ed60 2023-10-18T09:06:58.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.762+0000 7f15f3143640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec0c89b0 con 0x7f15ec0b1de0 2023-10-18T09:06:58.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.762+0000 7f15f3143640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec157880 con 0x7f15ec0ce2d0 2023-10-18T09:06:58.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.762+0000 7f15f2942640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15ec15ed60 0x7f15ec0cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:58.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.762+0000 7f15f2942640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15ec15ed60 0x7f15ec0cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43370/0 (socket says 172.21.15.40:43370) 2023-10-18T09:06:58.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.762+0000 7f15f2942640 1 -- 172.21.15.40:0/2898570844 learned_addr learned my addr 172.21.15.40:0/2898570844 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:58.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.763+0000 7f15f2141640 1 --2- 172.21.15.40:0/2898570844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15ec0b1de0 0x7f15ec1571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:58.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.763+0000 7f15f1940640 1 --2- 172.21.15.40:0/2898570844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0ce2d0 0x7f15ec15e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:58.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.763+0000 7f15f2942640 1 -- 172.21.15.40:0/2898570844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0ce2d0 msgr2=0x7f15ec15e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:58.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.763+0000 7f15f2942640 1 --2- 172.21.15.40:0/2898570844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0ce2d0 0x7f15ec15e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.763+0000 7f15f2942640 1 -- 172.21.15.40:0/2898570844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15ec0b1de0 msgr2=0x7f15ec1571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:58.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.763+0000 7f15f2942640 1 --2- 172.21.15.40:0/2898570844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15ec0b1de0 0x7f15ec1571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.763+0000 7f15f2942640 1 -- 172.21.15.40:0/2898570844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e4009900 con 0x7f15ec15ed60 2023-10-18T09:06:58.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.763+0000 7f15f2141640 1 --2- 172.21.15.40:0/2898570844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15ec0b1de0 0x7f15ec1571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:06:58.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.763+0000 7f15f2942640 1 --2- 172.21.15.40:0/2898570844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15ec15ed60 0x7f15ec0cc570 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f15e4006010 tx=0x7f15e4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:58.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.763+0000 7f15e2ffd640 1 -- 172.21.15.40:0/2898570844 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4004240 con 0x7f15ec15ed60 2023-10-18T09:06:58.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.763+0000 7f15e2ffd640 1 -- 172.21.15.40:0/2898570844 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f15e4004490 con 0x7f15ec15ed60 2023-10-18T09:06:58.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.763+0000 7f15f3143640 1 -- 172.21.15.40:0/2898570844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ec0c83f0 con 0x7f15ec15ed60 2023-10-18T09:06:58.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.764+0000 7f15e2ffd640 1 -- 172.21.15.40:0/2898570844 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4004840 con 0x7f15ec15ed60 2023-10-18T09:06:58.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.764+0000 7f15f3143640 1 -- 172.21.15.40:0/2898570844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ec1f26c0 con 0x7f15ec15ed60 2023-10-18T09:06:58.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.765+0000 7f15f3143640 1 -- 172.21.15.40:0/2898570844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ec1f2ca0 con 0x7f15ec15ed60 2023-10-18T09:06:58.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.766+0000 7f15e2ffd640 1 -- 172.21.15.40:0/2898570844 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f15e403b040 con 0x7f15ec15ed60 2023-10-18T09:06:58.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.767+0000 7f15e2ffd640 1 --2- 172.21.15.40:0/2898570844 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f15d006ae40 0x7f15d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:58.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.768+0000 7f15e2ffd640 1 -- 172.21.15.40:0/2898570844 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f15e40ac4f0 con 0x7f15ec15ed60 2023-10-18T09:06:58.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.768+0000 7f15f2141640 1 --2- 172.21.15.40:0/2898570844 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f15d006ae40 0x7f15d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:58.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.769+0000 7f15f2141640 1 --2- 172.21.15.40:0/2898570844 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f15d006ae40 0x7f15d006d300 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f15e800a830 tx=0x7f15e8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:58.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.772+0000 7f15e2ffd640 1 -- 172.21.15.40:0/2898570844 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15e4077550 con 0x7f15ec15ed60 2023-10-18T09:06:58.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.961+0000 7f15f3143640 1 -- 172.21.15.40:0/2898570844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f15ec1f28e0 con 0x7f15ec15ed60 2023-10-18T09:06:58.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.962+0000 7f15e2ffd640 1 -- 172.21.15.40:0/2898570844 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v167) v1 ==== 100+0+11 (secure 0 0 0) 0x7f15e407ab70 con 0x7f15ec15ed60 2023-10-18T09:06:58.965 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:06:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.964+0000 7f15f3143640 1 -- 172.21.15.40:0/2898570844 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f15d006ae40 msgr2=0x7f15d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.964+0000 7f15f3143640 1 --2- 172.21.15.40:0/2898570844 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f15d006ae40 0x7f15d006d300 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f15e800a830 tx=0x7f15e8008040 comp rx=0 tx=0).stop 2023-10-18T09:06:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.964+0000 7f15f3143640 1 -- 172.21.15.40:0/2898570844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15ec15ed60 msgr2=0x7f15ec0cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.964+0000 7f15f3143640 1 --2- 172.21.15.40:0/2898570844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15ec15ed60 0x7f15ec0cc570 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f15e4006010 tx=0x7f15e4003ba0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.965+0000 7f15f3143640 1 -- 172.21.15.40:0/2898570844 shutdown_connections 2023-10-18T09:06:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.965+0000 7f15f3143640 1 --2- 172.21.15.40:0/2898570844 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f15d006ae40 0x7f15d006d300 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.965+0000 7f15f3143640 1 --2- 172.21.15.40:0/2898570844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15ec15ed60 0x7f15ec0cc570 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.965+0000 7f15f3143640 1 --2- 172.21.15.40:0/2898570844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0ce2d0 0x7f15ec15e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.965+0000 7f15f3143640 1 --2- 172.21.15.40:0/2898570844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15ec0b1de0 0x7f15ec1571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:58.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.965+0000 7f15f3143640 1 -- 172.21.15.40:0/2898570844 >> 172.21.15.40:0/2898570844 conn(0x7f15ec09a580 msgr2=0x7f15ec06b160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:58.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.965+0000 7f15f3143640 1 -- 172.21.15.40:0/2898570844 shutdown_connections 2023-10-18T09:06:58.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:58.965+0000 7f15f3143640 1 -- 172.21.15.40:0/2898570844 wait complete. 2023-10-18T09:06:58.986 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:06:59.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.211+0000 7fe6256b4640 1 -- 172.21.15.40:0/937565871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe608003f70 msgr2=0x7fe6080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:59.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.211+0000 7fe6256b4640 1 --2- 172.21.15.40:0/937565871 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe608003f70 0x7fe6080047c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fe614009b50 tx=0x7fe6080057f0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.211+0000 7fe624eb3640 1 -- 172.21.15.40:0/937565871 reap_dead start 2023-10-18T09:06:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.211+0000 7fe6256b4640 1 -- 172.21.15.40:0/937565871 shutdown_connections 2023-10-18T09:06:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.211+0000 7fe6256b4640 1 -- 172.21.15.40:0/937565871 >> 172.21.15.40:0/937565871 conn(0x7fe620069720 msgr2=0x7fe620069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.212+0000 7fe6256b4640 1 -- 172.21.15.40:0/937565871 shutdown_connections 2023-10-18T09:06:59.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.212+0000 7fe6256b4640 1 -- 172.21.15.40:0/937565871 wait complete. 2023-10-18T09:06:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.213+0000 7fe6256b4640 1 Processor -- start 2023-10-18T09:06:59.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.213+0000 7fe6256b4640 1 -- start start 2023-10-18T09:06:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.214+0000 7fe6256b4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe608003f70 0x7fe620158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:59.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.214+0000 7fe6256b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620137ff0 0x7fe620138470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.214+0000 7fe6256b4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe62013b800 0x7fe62013bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.214+0000 7fe6256b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62013c1f0 con 0x7fe620137ff0 2023-10-18T09:06:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.214+0000 7fe6256b4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6201389b0 con 0x7fe608003f70 2023-10-18T09:06:59.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.214+0000 7fe6256b4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6201f34a0 con 0x7fe62013b800 2023-10-18T09:06:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.215+0000 7fe61f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620137ff0 0x7fe620138470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.215+0000 7fe61f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620137ff0 0x7fe620138470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43394/0 (socket says 172.21.15.40:43394) 2023-10-18T09:06:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.215+0000 7fe61f7fe640 1 -- 172.21.15.40:0/1078874380 learned_addr learned my addr 172.21.15.40:0/1078874380 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.215+0000 7fe61ffff640 1 --2- 172.21.15.40:0/1078874380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe608003f70 0x7fe620158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.215+0000 7fe624eb3640 1 --2- 172.21.15.40:0/1078874380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe62013b800 0x7fe62013bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:59.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.215+0000 7fe61f7fe640 1 -- 172.21.15.40:0/1078874380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe62013b800 msgr2=0x7fe62013bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.215+0000 7fe61f7fe640 1 --2- 172.21.15.40:0/1078874380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe62013b800 0x7fe62013bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.215+0000 7fe61f7fe640 1 -- 172.21.15.40:0/1078874380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe608003f70 msgr2=0x7fe620158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.215+0000 7fe61f7fe640 1 --2- 172.21.15.40:0/1078874380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe608003f70 0x7fe620158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.215+0000 7fe61f7fe640 1 -- 172.21.15.40:0/1078874380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62013ffb0 con 0x7fe620137ff0 2023-10-18T09:06:59.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.215+0000 7fe61ffff640 1 --2- 172.21.15.40:0/1078874380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe608003f70 0x7fe620158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:06:59.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.216+0000 7fe61f7fe640 1 --2- 172.21.15.40:0/1078874380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620137ff0 0x7fe620138470 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fe61402cba0 tx=0x7fe614033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.217+0000 7fe61cff9640 1 -- 172.21.15.40:0/1078874380 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe614004460 con 0x7fe620137ff0 2023-10-18T09:06:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.217+0000 7fe61cff9640 1 -- 172.21.15.40:0/1078874380 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe614033e40 con 0x7fe620137ff0 2023-10-18T09:06:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.217+0000 7fe61cff9640 1 -- 172.21.15.40:0/1078874380 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe614003850 con 0x7fe620137ff0 2023-10-18T09:06:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.217+0000 7fe6256b4640 1 -- 172.21.15.40:0/1078874380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62013c3e0 con 0x7fe620137ff0 2023-10-18T09:06:59.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.218+0000 7fe6256b4640 1 -- 172.21.15.40:0/1078874380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62013f960 con 0x7fe620137ff0 2023-10-18T09:06:59.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.219+0000 7fe6256b4640 1 -- 172.21.15.40:0/1078874380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6200ccdc0 con 0x7fe620137ff0 2023-10-18T09:06:59.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.220+0000 7fe61cff9640 1 -- 172.21.15.40:0/1078874380 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe6140039b0 con 0x7fe620137ff0 2023-10-18T09:06:59.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.222+0000 7fe61cff9640 1 --2- 172.21.15.40:0/1078874380 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe60c06ae40 0x7fe60c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:59.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.222+0000 7fe61cff9640 1 -- 172.21.15.40:0/1078874380 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe6140ac500 con 0x7fe620137ff0 2023-10-18T09:06:59.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.222+0000 7fe61ffff640 1 --2- 172.21.15.40:0/1078874380 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe60c06ae40 0x7fe60c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:59.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.224+0000 7fe61ffff640 1 --2- 172.21.15.40:0/1078874380 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe60c06ae40 0x7fe60c06d300 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe610001710 tx=0x7fe610009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:59.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.226+0000 7fe61cff9640 1 -- 172.21.15.40:0/1078874380 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6140774d0 con 0x7fe620137ff0 2023-10-18T09:06:59.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.464+0000 7fe6256b4640 1 -- 172.21.15.40:0/1078874380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6200c98c0 con 0x7fe620137ff0 2023-10-18T09:06:59.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.465+0000 7fe61cff9640 1 -- 172.21.15.40:0/1078874380 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fe61404a090 con 0x7fe620137ff0 2023-10-18T09:06:59.467 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:06:59.468 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3243865919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3243865919},{"type":"v1","addr":"172.21.15.40:6835","nonce":3243865919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6047,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/504515257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":504515257},{"type":"v1","addr":"172.21.15.144:6835","nonce":504515257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2513603652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2513603652},{"type":"v1","addr":"172.21.15.144:6837","nonce":2513603652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:04:36.950432+0000","modified":"2023-10-18T09:04:40.030449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8388":{"gid":8388,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/3348437493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3348437493},{"type":"v1","addr":"172.21.15.40:6833","nonce":3348437493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-18T09:06:59.468 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 74 2023-10-18T09:06:59.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.467+0000 7fe6256b4640 1 -- 172.21.15.40:0/1078874380 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe60c06ae40 msgr2=0x7fe60c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:59.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.467+0000 7fe6256b4640 1 --2- 172.21.15.40:0/1078874380 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe60c06ae40 0x7fe60c06d300 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe610001710 tx=0x7fe610009040 comp rx=0 tx=0).stop 2023-10-18T09:06:59.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.467+0000 7fe6256b4640 1 -- 172.21.15.40:0/1078874380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620137ff0 msgr2=0x7fe620138470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:59.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.467+0000 7fe6256b4640 1 --2- 172.21.15.40:0/1078874380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620137ff0 0x7fe620138470 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fe61402cba0 tx=0x7fe614033c10 comp rx=0 tx=0).stop 2023-10-18T09:06:59.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.467+0000 7fe6256b4640 1 -- 172.21.15.40:0/1078874380 shutdown_connections 2023-10-18T09:06:59.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.467+0000 7fe6256b4640 1 --2- 172.21.15.40:0/1078874380 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe60c06ae40 0x7fe60c06d300 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.467+0000 7fe6256b4640 1 --2- 172.21.15.40:0/1078874380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe62013b800 0x7fe62013bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.467+0000 7fe6256b4640 1 --2- 172.21.15.40:0/1078874380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe620137ff0 0x7fe620138470 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.467+0000 7fe6256b4640 1 --2- 172.21.15.40:0/1078874380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe608003f70 0x7fe620158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.467+0000 7fe6256b4640 1 -- 172.21.15.40:0/1078874380 >> 172.21.15.40:0/1078874380 conn(0x7fe620069720 msgr2=0x7fe620072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:59.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.467+0000 7fe6256b4640 1 -- 172.21.15.40:0/1078874380 shutdown_connections 2023-10-18T09:06:59.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.467+0000 7fe6256b4640 1 -- 172.21.15.40:0/1078874380 wait complete. 2023-10-18T09:06:59.490 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:06:59.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.707+0000 7f0ff083f640 1 -- 172.21.15.40:0/3600691206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fec1559a0 msgr2=0x7f0fec15b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:59.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.707+0000 7f0ff083f640 1 --2- 172.21.15.40:0/3600691206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fec1559a0 0x7f0fec15b950 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f0fec06a270 tx=0x7f0fdc02c260 comp rx=0 tx=0).stop 2023-10-18T09:06:59.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.707+0000 7f0ff083f640 1 -- 172.21.15.40:0/3600691206 shutdown_connections 2023-10-18T09:06:59.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.707+0000 7f0ff083f640 1 --2- 172.21.15.40:0/3600691206 >> v2:172.21.15.144:3300/0 conn(0x7f0fec06bae0 0x7f0fec06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.707+0000 7f0ff083f640 1 --2- 172.21.15.40:0/3600691206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fec1559a0 0x7f0fec15b950 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.707+0000 7f0ff083f640 1 -- 172.21.15.40:0/3600691206 >> 172.21.15.40:0/3600691206 conn(0x7f0fec0b3330 msgr2=0x7f0fec069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:59.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.707+0000 7f0ff083f640 1 -- 172.21.15.40:0/3600691206 shutdown_connections 2023-10-18T09:06:59.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.707+0000 7f0ff083f640 1 -- 172.21.15.40:0/3600691206 wait complete. 2023-10-18T09:06:59.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.709+0000 7f0ff083f640 1 Processor -- start 2023-10-18T09:06:59.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.709+0000 7f0ff083f640 1 -- start start 2023-10-18T09:06:59.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.710+0000 7f0ff083f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fec06bae0 0x7f0fec153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:59.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.710+0000 7f0ff083f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fec06d1b0 0x7f0fec1e3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:59.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.710+0000 7f0ff083f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fec1559a0 0x7f0fec0abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:59.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.710+0000 7f0ff083f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec1f1b30 con 0x7f0fec06d1b0 2023-10-18T09:06:59.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.710+0000 7f0ff083f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec1e4090 con 0x7f0fec06bae0 2023-10-18T09:06:59.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.710+0000 7f0ff083f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec1e4310 con 0x7f0fec1559a0 2023-10-18T09:06:59.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.711+0000 7f0fe259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fec06d1b0 0x7f0fec1e3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:59.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.711+0000 7f0fe259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fec06d1b0 0x7f0fec1e3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43414/0 (socket says 172.21.15.40:43414) 2023-10-18T09:06:59.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.711+0000 7f0fe259c640 1 -- 172.21.15.40:0/1122692036 learned_addr learned my addr 172.21.15.40:0/1122692036 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:06:59.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.711+0000 7f0fead9d640 1 --2- 172.21.15.40:0/1122692036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fec06bae0 0x7f0fec153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:59.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.711+0000 7f0fe259c640 1 -- 172.21.15.40:0/1122692036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fec1559a0 msgr2=0x7f0fec0abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:59.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.711+0000 7f0fe259c640 1 --2- 172.21.15.40:0/1122692036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fec1559a0 0x7f0fec0abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.711+0000 7f0fe259c640 1 -- 172.21.15.40:0/1122692036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fec06bae0 msgr2=0x7f0fec153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:59.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.711+0000 7f0fe259c640 1 --2- 172.21.15.40:0/1122692036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fec06bae0 0x7f0fec153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.711+0000 7f0fe259c640 1 -- 172.21.15.40:0/1122692036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc009900 con 0x7f0fec06d1b0 2023-10-18T09:06:59.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.711+0000 7f0fe259c640 1 --2- 172.21.15.40:0/1122692036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fec06d1b0 0x7f0fec1e3b50 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400d700 tx=0x7f0fd400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:59.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.712+0000 7f0fe3fff640 1 -- 172.21.15.40:0/1122692036 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4005220 con 0x7f0fec06d1b0 2023-10-18T09:06:59.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.712+0000 7f0fe3fff640 1 -- 172.21.15.40:0/1122692036 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0fd4005470 con 0x7f0fec06d1b0 2023-10-18T09:06:59.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.712+0000 7f0fe3fff640 1 -- 172.21.15.40:0/1122692036 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4009a80 con 0x7f0fec06d1b0 2023-10-18T09:06:59.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.712+0000 7f0ff083f640 1 -- 172.21.15.40:0/1122692036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fec1e45f0 con 0x7f0fec06d1b0 2023-10-18T09:06:59.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.712+0000 7f0ff083f640 1 -- 172.21.15.40:0/1122692036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fec1e4bb0 con 0x7f0fec06d1b0 2023-10-18T09:06:59.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.713+0000 7f0ff083f640 1 -- 172.21.15.40:0/1122692036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fec1f1c70 con 0x7f0fec06d1b0 2023-10-18T09:06:59.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.714+0000 7f0fe3fff640 1 -- 172.21.15.40:0/1122692036 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0fd4009be0 con 0x7f0fec06d1b0 2023-10-18T09:06:59.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.716+0000 7f0fe3fff640 1 --2- 172.21.15.40:0/1122692036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0fcc06ae40 0x7f0fcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:06:59.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.716+0000 7f0fe3fff640 1 -- 172.21.15.40:0/1122692036 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0fd4089880 con 0x7f0fec06d1b0 2023-10-18T09:06:59.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.716+0000 7f0fead9d640 1 --2- 172.21.15.40:0/1122692036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0fcc06ae40 0x7f0fcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:06:59.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.717+0000 7f0fead9d640 1 --2- 172.21.15.40:0/1122692036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0fcc06ae40 0x7f0fcc06d300 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc036000 tx=0x7f0fdc004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:06:59.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.721+0000 7f0fe3fff640 1 -- 172.21.15.40:0/1122692036 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0fec1f1c70 con 0x7f0fec06d1b0 2023-10-18T09:06:59.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.916+0000 7f0ff083f640 1 -- 172.21.15.40:0/1122692036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fec1f1c70 con 0x7f0fec06d1b0 2023-10-18T09:06:59.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.918+0000 7f0fe3fff640 1 -- 172.21.15.40:0/1122692036 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f0fec1f1c70 con 0x7f0fec06d1b0 2023-10-18T09:06:59.921 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:06:59.921 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":167,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:06:56.150783+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:04:34.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-18T09:04:35.972659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.26","mappings":[{"from":2,"to":0}]},{"pgid":"19.33","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:06:59.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.921+0000 7f0ff083f640 1 -- 172.21.15.40:0/1122692036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0fcc06ae40 msgr2=0x7f0fcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:59.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.921+0000 7f0ff083f640 1 --2- 172.21.15.40:0/1122692036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0fcc06ae40 0x7f0fcc06d300 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc036000 tx=0x7f0fdc004ab0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.921+0000 7f0ff083f640 1 -- 172.21.15.40:0/1122692036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fec06d1b0 msgr2=0x7f0fec1e3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:06:59.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.921+0000 7f0ff083f640 1 --2- 172.21.15.40:0/1122692036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fec06d1b0 0x7f0fec1e3b50 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400d700 tx=0x7f0fd400dbc0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.921+0000 7f0ff083f640 1 -- 172.21.15.40:0/1122692036 shutdown_connections 2023-10-18T09:06:59.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.921+0000 7f0ff083f640 1 --2- 172.21.15.40:0/1122692036 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0fcc06ae40 0x7f0fcc06d300 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.921+0000 7f0ff083f640 1 --2- 172.21.15.40:0/1122692036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fec1559a0 0x7f0fec0abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.921+0000 7f0ff083f640 1 --2- 172.21.15.40:0/1122692036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fec06d1b0 0x7f0fec1e3b50 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.921+0000 7f0ff083f640 1 --2- 172.21.15.40:0/1122692036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fec06bae0 0x7f0fec153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:06:59.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.921+0000 7f0ff083f640 1 -- 172.21.15.40:0/1122692036 >> 172.21.15.40:0/1122692036 conn(0x7f0fec0b3330 msgr2=0x7f0fec0ae4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:06:59.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.921+0000 7f0ff083f640 1 -- 172.21.15.40:0/1122692036 shutdown_connections 2023-10-18T09:06:59.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:06:59.921+0000 7f0ff083f640 1 -- 172.21.15.40:0/1122692036 wait complete. 2023-10-18T09:06:59.943 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T09:06:59.944 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:07:00.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.165+0000 7fd29d380640 1 -- 172.21.15.40:0/3965575694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2800039a0 msgr2=0x7fd280003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:00.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.165+0000 7fd29d380640 1 --2- 172.21.15.40:0/3965575694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2800039a0 0x7fd280003dc0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fd28800c640 tx=0x7fd2800057f0 comp rx=0 tx=0).stop 2023-10-18T09:07:00.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.165+0000 7fd29cb7f640 1 -- 172.21.15.40:0/3965575694 reap_dead start 2023-10-18T09:07:00.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.166+0000 7fd29d380640 1 -- 172.21.15.40:0/3965575694 shutdown_connections 2023-10-18T09:07:00.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.166+0000 7fd29d380640 1 -- 172.21.15.40:0/3965575694 >> 172.21.15.40:0/3965575694 conn(0x7fd298069720 msgr2=0x7fd298069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:00.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.166+0000 7fd29d380640 1 -- 172.21.15.40:0/3965575694 shutdown_connections 2023-10-18T09:07:00.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.166+0000 7fd29d380640 1 -- 172.21.15.40:0/3965575694 wait complete. 2023-10-18T09:07:00.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.168+0000 7fd29d380640 1 Processor -- start 2023-10-18T09:07:00.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.168+0000 7fd29d380640 1 -- start start 2023-10-18T09:07:00.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.169+0000 7fd29d380640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2800039a0 0x7fd29813a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:00.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.169+0000 7fd29d380640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd298152440 0x7fd29813bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:00.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.169+0000 7fd29d380640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd29813da60 0x7fd29813df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:00.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.169+0000 7fd29d380640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29813a2b0 con 0x7fd298152440 2023-10-18T09:07:00.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.169+0000 7fd29d380640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29813e450 con 0x7fd2800039a0 2023-10-18T09:07:00.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.169+0000 7fd29d380640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29813ac00 con 0x7fd29813da60 2023-10-18T09:07:00.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.169+0000 7fd2977fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd298152440 0x7fd29813bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:00.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.169+0000 7fd2977fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd298152440 0x7fd29813bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43424/0 (socket says 172.21.15.40:43424) 2023-10-18T09:07:00.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.169+0000 7fd2977fe640 1 -- 172.21.15.40:0/3622475342 learned_addr learned my addr 172.21.15.40:0/3622475342 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:00.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.169+0000 7fd2977fe640 1 -- 172.21.15.40:0/3622475342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd29813da60 msgr2=0x7fd29813df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:00.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.169+0000 7fd2977fe640 1 --2- 172.21.15.40:0/3622475342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd29813da60 0x7fd29813df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:00.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.169+0000 7fd2977fe640 1 -- 172.21.15.40:0/3622475342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2800039a0 msgr2=0x7fd29813a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:00.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.169+0000 7fd2977fe640 1 --2- 172.21.15.40:0/3622475342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2800039a0 0x7fd29813a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:00.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.169+0000 7fd2977fe640 1 -- 172.21.15.40:0/3622475342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29813aee0 con 0x7fd298152440 2023-10-18T09:07:00.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.170+0000 7fd2977fe640 1 --2- 172.21.15.40:0/3622475342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd298152440 0x7fd29813bca0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00fb70 tx=0x7fd28c00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:00.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.170+0000 7fd294ff9640 1 -- 172.21.15.40:0/3622475342 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c004230 con 0x7fd298152440 2023-10-18T09:07:00.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.170+0000 7fd294ff9640 1 -- 172.21.15.40:0/3622475342 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd28c0044a0 con 0x7fd298152440 2023-10-18T09:07:00.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.170+0000 7fd294ff9640 1 -- 172.21.15.40:0/3622475342 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c003850 con 0x7fd298152440 2023-10-18T09:07:00.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.170+0000 7fd29d380640 1 -- 172.21.15.40:0/3622475342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29813b1c0 con 0x7fd298152440 2023-10-18T09:07:00.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.170+0000 7fd29d380640 1 -- 172.21.15.40:0/3622475342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29813b780 con 0x7fd298152440 2023-10-18T09:07:00.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.171+0000 7fd29d380640 1 -- 172.21.15.40:0/3622475342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd298070850 con 0x7fd298152440 2023-10-18T09:07:00.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.173+0000 7fd294ff9640 1 -- 172.21.15.40:0/3622475342 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd28c0039b0 con 0x7fd298152440 2023-10-18T09:07:00.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.173+0000 7fd294ff9640 1 --2- 172.21.15.40:0/3622475342 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd28406ae70 0x7fd28406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:00.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.174+0000 7fd294ff9640 1 -- 172.21.15.40:0/3622475342 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd28c08a9f0 con 0x7fd298152440 2023-10-18T09:07:00.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.174+0000 7fd297fff640 1 --2- 172.21.15.40:0/3622475342 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd28406ae70 0x7fd28406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:00.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.175+0000 7fd297fff640 1 --2- 172.21.15.40:0/3622475342 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd28406ae70 0x7fd28406d330 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd28800c640 tx=0x7fd288039040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:00.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.178+0000 7fd294ff9640 1 -- 172.21.15.40:0/3622475342 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd28c0559c0 con 0x7fd298152440 2023-10-18T09:07:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.413+0000 7fd29d380640 1 -- 172.21.15.40:0/3622475342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2980caf90 con 0x7fd298152440 2023-10-18T09:07:00.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.414+0000 7fd294ff9640 1 -- 172.21.15.40:0/3622475342 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fd28c01d020 con 0x7fd298152440 2023-10-18T09:07:00.416 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:07:00.417 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3243865919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3243865919},{"type":"v1","addr":"172.21.15.40:6835","nonce":3243865919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6047,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/504515257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":504515257},{"type":"v1","addr":"172.21.15.144:6835","nonce":504515257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2513603652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2513603652},{"type":"v1","addr":"172.21.15.144:6837","nonce":2513603652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:04:36.950432+0000","modified":"2023-10-18T09:04:40.030449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8388":{"gid":8388,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/3348437493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3348437493},{"type":"v1","addr":"172.21.15.40:6833","nonce":3348437493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-18T09:07:00.417 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 74 2023-10-18T09:07:00.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.416+0000 7fd29d380640 1 -- 172.21.15.40:0/3622475342 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd28406ae70 msgr2=0x7fd28406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:00.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.416+0000 7fd29d380640 1 --2- 172.21.15.40:0/3622475342 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd28406ae70 0x7fd28406d330 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd28800c640 tx=0x7fd288039040 comp rx=0 tx=0).stop 2023-10-18T09:07:00.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.416+0000 7fd29d380640 1 -- 172.21.15.40:0/3622475342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd298152440 msgr2=0x7fd29813bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:00.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.416+0000 7fd29d380640 1 --2- 172.21.15.40:0/3622475342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd298152440 0x7fd29813bca0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00fb70 tx=0x7fd28c00c7a0 comp rx=0 tx=0).stop 2023-10-18T09:07:00.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.416+0000 7fd29d380640 1 -- 172.21.15.40:0/3622475342 shutdown_connections 2023-10-18T09:07:00.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.416+0000 7fd29d380640 1 --2- 172.21.15.40:0/3622475342 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd28406ae70 0x7fd28406d330 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:00.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.416+0000 7fd29d380640 1 --2- 172.21.15.40:0/3622475342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd29813da60 0x7fd29813df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:00.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.416+0000 7fd29d380640 1 --2- 172.21.15.40:0/3622475342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd298152440 0x7fd29813bca0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:00.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.416+0000 7fd29d380640 1 --2- 172.21.15.40:0/3622475342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2800039a0 0x7fd29813a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:00.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.416+0000 7fd29d380640 1 -- 172.21.15.40:0/3622475342 >> 172.21.15.40:0/3622475342 conn(0x7fd298069720 msgr2=0x7fd2981598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:00.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.417+0000 7fd29d380640 1 -- 172.21.15.40:0/3622475342 shutdown_connections 2023-10-18T09:07:00.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.417+0000 7fd29d380640 1 -- 172.21.15.40:0/3622475342 wait complete. 2023-10-18T09:07:00.438 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:07:00.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.666+0000 7f55374b4640 1 -- 172.21.15.40:0/443416456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c003970 msgr2=0x7f551c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:00.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.666+0000 7f55374b4640 1 --2- 172.21.15.40:0/443416456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f551c003970 0x7f551c003d90 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f553015af40 tx=0x7f551c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:07:00.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.666+0000 7f5536cb3640 1 -- 172.21.15.40:0/443416456 reap_dead start 2023-10-18T09:07:00.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.666+0000 7f55374b4640 1 -- 172.21.15.40:0/443416456 shutdown_connections 2023-10-18T09:07:00.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.666+0000 7f55374b4640 1 -- 172.21.15.40:0/443416456 >> 172.21.15.40:0/443416456 conn(0x7f5530069720 msgr2=0x7f5530069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:00.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.666+0000 7f55374b4640 1 -- 172.21.15.40:0/443416456 shutdown_connections 2023-10-18T09:07:00.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.666+0000 7f55374b4640 1 -- 172.21.15.40:0/443416456 wait complete. 2023-10-18T09:07:00.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.668+0000 7f55374b4640 1 Processor -- start 2023-10-18T09:07:00.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.668+0000 7f55374b4640 1 -- start start 2023-10-18T09:07:00.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.669+0000 7f55374b4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f551c003970 0x7f55301e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:00.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.669+0000 7f55374b4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5530152840 0x7f5530151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:00.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.669+0000 7f55374b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55300aee80 0x7f55300af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:00.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.669+0000 7f55374b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5530157ee0 con 0x7f55300aee80 2023-10-18T09:07:00.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.669+0000 7f55374b4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5530159880 con 0x7f551c003970 2023-10-18T09:07:00.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.669+0000 7f55374b4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55300af840 con 0x7f5530152840 2023-10-18T09:07:00.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.669+0000 7f5536cb3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55300aee80 0x7f55300af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:00.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.669+0000 7f5536cb3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55300aee80 0x7f55300af300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43436/0 (socket says 172.21.15.40:43436) 2023-10-18T09:07:00.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.669+0000 7f5536cb3640 1 -- 172.21.15.40:0/600416225 learned_addr learned my addr 172.21.15.40:0/600416225 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:00.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.670+0000 7f55364b2640 1 --2- 172.21.15.40:0/600416225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f551c003970 0x7f55301e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:00.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.670+0000 7f5536cb3640 1 -- 172.21.15.40:0/600416225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5530152840 msgr2=0x7f5530151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:00.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.670+0000 7f5536cb3640 1 --2- 172.21.15.40:0/600416225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5530152840 0x7f5530151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:00.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.670+0000 7f5536cb3640 1 -- 172.21.15.40:0/600416225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f551c003970 msgr2=0x7f55301e6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:00.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.670+0000 7f5536cb3640 1 --2- 172.21.15.40:0/600416225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f551c003970 0x7f55301e6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:00.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.670+0000 7f5536cb3640 1 -- 172.21.15.40:0/600416225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5530159ad0 con 0x7f55300aee80 2023-10-18T09:07:00.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.670+0000 7f5536cb3640 1 --2- 172.21.15.40:0/600416225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55300aee80 0x7f55300af300 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f5528005fd0 tx=0x7f5528009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:00.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.670+0000 7f5526ffd640 1 -- 172.21.15.40:0/600416225 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528004d20 con 0x7f55300aee80 2023-10-18T09:07:00.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.670+0000 7f5526ffd640 1 -- 172.21.15.40:0/600416225 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5528011710 con 0x7f55300aee80 2023-10-18T09:07:00.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.670+0000 7f5526ffd640 1 -- 172.21.15.40:0/600416225 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528011b20 con 0x7f55300aee80 2023-10-18T09:07:00.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.670+0000 7f55374b4640 1 -- 172.21.15.40:0/600416225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5530159db0 con 0x7f55300aee80 2023-10-18T09:07:00.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.670+0000 7f55374b4640 1 -- 172.21.15.40:0/600416225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5530151fa0 con 0x7f55300aee80 2023-10-18T09:07:00.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.671+0000 7f55374b4640 1 -- 172.21.15.40:0/600416225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5530069b00 con 0x7f55300aee80 2023-10-18T09:07:00.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.672+0000 7f5526ffd640 1 -- 172.21.15.40:0/600416225 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5528011c80 con 0x7f55300aee80 2023-10-18T09:07:00.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.673+0000 7f5526ffd640 1 --2- 172.21.15.40:0/600416225 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f551406ae70 0x7f551406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:00.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.673+0000 7f5526ffd640 1 -- 172.21.15.40:0/600416225 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f552808a170 con 0x7f55300aee80 2023-10-18T09:07:00.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.674+0000 7f55364b2640 1 --2- 172.21.15.40:0/600416225 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f551406ae70 0x7f551406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:00.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.675+0000 7f55364b2640 1 --2- 172.21.15.40:0/600416225 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f551406ae70 0x7f551406d330 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f552c034000 tx=0x7f552c02e040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:00.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.678+0000 7f5526ffd640 1 -- 172.21.15.40:0/600416225 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55280551d0 con 0x7f55300aee80 2023-10-18T09:07:00.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.898+0000 7f55374b4640 1 -- 172.21.15.40:0/600416225 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f553015bbb0 con 0x7f55300aee80 2023-10-18T09:07:00.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.899+0000 7f5526ffd640 1 -- 172.21.15.40:0/600416225 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f55280587f0 con 0x7f55300aee80 2023-10-18T09:07:00.902 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:07:00.902 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3243865919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3243865919},{"type":"v1","addr":"172.21.15.40:6835","nonce":3243865919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6047,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/504515257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":504515257},{"type":"v1","addr":"172.21.15.144:6835","nonce":504515257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2513603652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2513603652},{"type":"v1","addr":"172.21.15.144:6837","nonce":2513603652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:04:36.950432+0000","modified":"2023-10-18T09:04:40.030449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8388":{"gid":8388,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6833/3348437493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3348437493},{"type":"v1","addr":"172.21.15.40:6833","nonce":3348437493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-18T09:07:00.903 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 74 2023-10-18T09:07:00.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.902+0000 7f55374b4640 1 -- 172.21.15.40:0/600416225 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f551406ae70 msgr2=0x7f551406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:00.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.902+0000 7f55374b4640 1 --2- 172.21.15.40:0/600416225 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f551406ae70 0x7f551406d330 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f552c034000 tx=0x7f552c02e040 comp rx=0 tx=0).stop 2023-10-18T09:07:00.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.902+0000 7f55374b4640 1 -- 172.21.15.40:0/600416225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55300aee80 msgr2=0x7f55300af300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:00.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.902+0000 7f55374b4640 1 --2- 172.21.15.40:0/600416225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55300aee80 0x7f55300af300 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f5528005fd0 tx=0x7f5528009a80 comp rx=0 tx=0).stop 2023-10-18T09:07:00.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.902+0000 7f55374b4640 1 -- 172.21.15.40:0/600416225 shutdown_connections 2023-10-18T09:07:00.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.902+0000 7f55374b4640 1 --2- 172.21.15.40:0/600416225 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f551406ae70 0x7f551406d330 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:00.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.902+0000 7f55374b4640 1 --2- 172.21.15.40:0/600416225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55300aee80 0x7f55300af300 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:00.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.902+0000 7f55374b4640 1 --2- 172.21.15.40:0/600416225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5530152840 0x7f5530151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:00.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.902+0000 7f55374b4640 1 --2- 172.21.15.40:0/600416225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f551c003970 0x7f55301e6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:00.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.902+0000 7f55374b4640 1 -- 172.21.15.40:0/600416225 >> 172.21.15.40:0/600416225 conn(0x7f5530069720 msgr2=0x7f55300cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:00.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.902+0000 7f55374b4640 1 -- 172.21.15.40:0/600416225 shutdown_connections 2023-10-18T09:07:00.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:00.902+0000 7f55374b4640 1 -- 172.21.15.40:0/600416225 wait complete. 2023-10-18T09:07:00.923 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:07:01.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.141+0000 7fe45df5e640 1 -- 172.21.15.40:0/3869406426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe45815b640 msgr2=0x7fe4580d1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:01.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.141+0000 7fe45df5e640 1 --2- 172.21.15.40:0/3869406426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe45815b640 0x7fe4580d1a90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe454005fd0 tx=0x7fe45402bcf0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.142+0000 7fe45df5e640 1 -- 172.21.15.40:0/3869406426 shutdown_connections 2023-10-18T09:07:01.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.142+0000 7fe45df5e640 1 --2- 172.21.15.40:0/3869406426 >> v2:172.21.15.144:3300/0 conn(0x7fe4580d0020 0x7fe4580ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.142+0000 7fe45df5e640 1 --2- 172.21.15.40:0/3869406426 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe45815b640 0x7fe4580d1a90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.142+0000 7fe45df5e640 1 -- 172.21.15.40:0/3869406426 >> 172.21.15.40:0/3869406426 conn(0x7fe45808ad00 msgr2=0x7fe45809a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:01.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.142+0000 7fe45df5e640 1 -- 172.21.15.40:0/3869406426 shutdown_connections 2023-10-18T09:07:01.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.142+0000 7fe45df5e640 1 -- 172.21.15.40:0/3869406426 wait complete. 2023-10-18T09:07:01.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.144+0000 7fe45df5e640 1 Processor -- start 2023-10-18T09:07:01.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.144+0000 7fe45df5e640 1 -- start start 2023-10-18T09:07:01.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.145+0000 7fe45df5e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4580b01e0 0x7fe4581583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:01.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.145+0000 7fe45df5e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4580d0020 0x7fe4580c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:01.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.146+0000 7fe45df5e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe45815b640 0x7fe4580cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:01.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.146+0000 7fe45df5e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45815d140 con 0x7fe45815b640 2023-10-18T09:07:01.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.146+0000 7fe45df5e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45815c4c0 con 0x7fe4580d0020 2023-10-18T09:07:01.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.146+0000 7fe45df5e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45815c7a0 con 0x7fe4580b01e0 2023-10-18T09:07:01.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.146+0000 7fe45d75d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe45815b640 0x7fe4580cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:01.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.146+0000 7fe45d75d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe45815b640 0x7fe4580cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43450/0 (socket says 172.21.15.40:43450) 2023-10-18T09:07:01.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.146+0000 7fe45d75d640 1 -- 172.21.15.40:0/282559113 learned_addr learned my addr 172.21.15.40:0/282559113 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:01.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.146+0000 7fe45cf5c640 1 --2- 172.21.15.40:0/282559113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4580b01e0 0x7fe4581583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:01.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.146+0000 7fe44ffff640 1 --2- 172.21.15.40:0/282559113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4580d0020 0x7fe4580c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:01.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.146+0000 7fe45d75d640 1 -- 172.21.15.40:0/282559113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4580b01e0 msgr2=0x7fe4581583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:01.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.146+0000 7fe45d75d640 1 --2- 172.21.15.40:0/282559113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4580b01e0 0x7fe4581583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.146+0000 7fe45d75d640 1 -- 172.21.15.40:0/282559113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4580d0020 msgr2=0x7fe4580c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:01.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.146+0000 7fe45d75d640 1 --2- 172.21.15.40:0/282559113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4580d0020 0x7fe4580c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.146+0000 7fe45d75d640 1 -- 172.21.15.40:0/282559113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe454009560 con 0x7fe45815b640 2023-10-18T09:07:01.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.147+0000 7fe45d75d640 1 --2- 172.21.15.40:0/282559113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe45815b640 0x7fe4580cd6d0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fe45000eb10 tx=0x7fe45000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:01.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.147+0000 7fe44d7fa640 1 -- 172.21.15.40:0/282559113 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45000ce80 con 0x7fe45815b640 2023-10-18T09:07:01.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.147+0000 7fe44d7fa640 1 -- 172.21.15.40:0/282559113 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe450004c10 con 0x7fe45815b640 2023-10-18T09:07:01.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.147+0000 7fe44d7fa640 1 -- 172.21.15.40:0/282559113 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450005020 con 0x7fe45815b640 2023-10-18T09:07:01.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.147+0000 7fe45df5e640 1 -- 172.21.15.40:0/282559113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45815ca80 con 0x7fe45815b640 2023-10-18T09:07:01.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.147+0000 7fe45df5e640 1 -- 172.21.15.40:0/282559113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4580c9e40 con 0x7fe45815b640 2023-10-18T09:07:01.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.148+0000 7fe45df5e640 1 -- 172.21.15.40:0/282559113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe458071da0 con 0x7fe45815b640 2023-10-18T09:07:01.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.149+0000 7fe44d7fa640 1 -- 172.21.15.40:0/282559113 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe450005180 con 0x7fe45815b640 2023-10-18T09:07:01.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.150+0000 7fe44d7fa640 1 --2- 172.21.15.40:0/282559113 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe44406ae40 0x7fe44406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:01.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.150+0000 7fe44d7fa640 1 -- 172.21.15.40:0/282559113 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe450014070 con 0x7fe45815b640 2023-10-18T09:07:01.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.151+0000 7fe45cf5c640 1 --2- 172.21.15.40:0/282559113 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe44406ae40 0x7fe44406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:01.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.152+0000 7fe45cf5c640 1 --2- 172.21.15.40:0/282559113 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe44406ae40 0x7fe44406d300 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe454035000 tx=0x7fe454004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:01.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.156+0000 7fe44d7fa640 1 -- 172.21.15.40:0/282559113 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe4500555d0 con 0x7fe45815b640 2023-10-18T09:07:01.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.357+0000 7fe45df5e640 1 -- 172.21.15.40:0/282559113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe45815d280 con 0x7fe45815b640 2023-10-18T09:07:01.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.359+0000 7fe44d7fa640 1 -- 172.21.15.40:0/282559113 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+16641 (secure 0 0 0) 0x7fe45815d280 con 0x7fe45815b640 2023-10-18T09:07:01.362 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:07:01.362 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":167,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:06:56.150783+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:04:34.597892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-18T09:04:35.972659+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.26","mappings":[{"from":2,"to":0}]},{"pgid":"19.33","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:07:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.362+0000 7fe45df5e640 1 -- 172.21.15.40:0/282559113 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe44406ae40 msgr2=0x7fe44406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.362+0000 7fe45df5e640 1 --2- 172.21.15.40:0/282559113 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe44406ae40 0x7fe44406d300 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe454035000 tx=0x7fe454004730 comp rx=0 tx=0).stop 2023-10-18T09:07:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.362+0000 7fe45df5e640 1 -- 172.21.15.40:0/282559113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe45815b640 msgr2=0x7fe4580cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.362+0000 7fe45df5e640 1 --2- 172.21.15.40:0/282559113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe45815b640 0x7fe4580cd6d0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fe45000eb10 tx=0x7fe45000c2d0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.362+0000 7fe45df5e640 1 -- 172.21.15.40:0/282559113 shutdown_connections 2023-10-18T09:07:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.362+0000 7fe45df5e640 1 --2- 172.21.15.40:0/282559113 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe44406ae40 0x7fe44406d300 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.362+0000 7fe45df5e640 1 --2- 172.21.15.40:0/282559113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe45815b640 0x7fe4580cd6d0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.362+0000 7fe45df5e640 1 --2- 172.21.15.40:0/282559113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4580d0020 0x7fe4580c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.362+0000 7fe45df5e640 1 --2- 172.21.15.40:0/282559113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4580b01e0 0x7fe4581583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.362+0000 7fe45df5e640 1 -- 172.21.15.40:0/282559113 >> 172.21.15.40:0/282559113 conn(0x7fe45808ad00 msgr2=0x7fe45815a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:01.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.362+0000 7fe45df5e640 1 -- 172.21.15.40:0/282559113 shutdown_connections 2023-10-18T09:07:01.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.362+0000 7fe45df5e640 1 -- 172.21.15.40:0/282559113 wait complete. 2023-10-18T09:07:01.384 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T09:07:01.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.621+0000 7f1488839640 1 -- 172.21.15.40:0/3972498785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14840ca0d0 msgr2=0x7f1484157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:01.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.621+0000 7f1488839640 1 --2- 172.21.15.40:0/3972498785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14840ca0d0 0x7f1484157d40 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f147c005fd0 tx=0x7f147c02d410 comp rx=0 tx=0).stop 2023-10-18T09:07:01.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.621+0000 7f1488839640 1 -- 172.21.15.40:0/3972498785 shutdown_connections 2023-10-18T09:07:01.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.621+0000 7f1488839640 1 --2- 172.21.15.40:0/3972498785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14840ca0d0 0x7f1484157d40 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.621+0000 7f1488839640 1 -- 172.21.15.40:0/3972498785 >> 172.21.15.40:0/3972498785 conn(0x7f1484069720 msgr2=0x7f1484069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:01.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.621+0000 7f1488839640 1 -- 172.21.15.40:0/3972498785 shutdown_connections 2023-10-18T09:07:01.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.621+0000 7f1488839640 1 -- 172.21.15.40:0/3972498785 wait complete. 2023-10-18T09:07:01.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.623+0000 7f1488839640 1 Processor -- start 2023-10-18T09:07:01.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.623+0000 7f1488839640 1 -- start start 2023-10-18T09:07:01.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.624+0000 7f1488839640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14840b15b0 0x7f1484144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:01.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.624+0000 7f1488839640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14840b1e10 0x7f1484151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:01.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.624+0000 7f1488839640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14840ca0d0 0x7f1484156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:01.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.624+0000 7f1488839640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1484144ed0 con 0x7f14840b1e10 2023-10-18T09:07:01.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.624+0000 7f1488839640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1484153360 con 0x7f14840ca0d0 2023-10-18T09:07:01.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.624+0000 7f1488839640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1484152750 con 0x7f14840b15b0 2023-10-18T09:07:01.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.624+0000 7f1482ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14840b1e10 0x7f1484151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:01.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.624+0000 7f1482ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14840b1e10 0x7f1484151c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43470/0 (socket says 172.21.15.40:43470) 2023-10-18T09:07:01.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.625+0000 7f1482ffd640 1 -- 172.21.15.40:0/2249799399 learned_addr learned my addr 172.21.15.40:0/2249799399 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:01.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.625+0000 7f1483fff640 1 --2- 172.21.15.40:0/2249799399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14840ca0d0 0x7f1484156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:01.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.625+0000 7f1482ffd640 1 -- 172.21.15.40:0/2249799399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14840b15b0 msgr2=0x7f1484144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:01.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.625+0000 7f14837fe640 1 --2- 172.21.15.40:0/2249799399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14840b15b0 0x7f1484144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:01.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.625+0000 7f1482ffd640 1 --2- 172.21.15.40:0/2249799399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14840b15b0 0x7f1484144990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.625+0000 7f1482ffd640 1 -- 172.21.15.40:0/2249799399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14840ca0d0 msgr2=0x7f1484156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:01.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.625+0000 7f1482ffd640 1 --2- 172.21.15.40:0/2249799399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14840ca0d0 0x7f1484156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.625+0000 7f1482ffd640 1 -- 172.21.15.40:0/2249799399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147c00a950 con 0x7f14840b1e10 2023-10-18T09:07:01.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.625+0000 7f1483fff640 1 --2- 172.21.15.40:0/2249799399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14840ca0d0 0x7f1484156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:01.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.625+0000 7f14837fe640 1 --2- 172.21.15.40:0/2249799399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14840b15b0 0x7f1484144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:01.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.625+0000 7f1482ffd640 1 --2- 172.21.15.40:0/2249799399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14840b1e10 0x7f1484151c70 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f147800c490 tx=0x7f147800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:01.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.626+0000 7f146bfff640 1 -- 172.21.15.40:0/2249799399 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478012830 con 0x7f14840b1e10 2023-10-18T09:07:01.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.626+0000 7f146bfff640 1 -- 172.21.15.40:0/2249799399 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1478012a80 con 0x7f14840b1e10 2023-10-18T09:07:01.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.626+0000 7f146bfff640 1 -- 172.21.15.40:0/2249799399 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478011380 con 0x7f14840b1e10 2023-10-18T09:07:01.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.626+0000 7f1488839640 1 -- 172.21.15.40:0/2249799399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1484152a30 con 0x7f14840b1e10 2023-10-18T09:07:01.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.626+0000 7f1488839640 1 -- 172.21.15.40:0/2249799399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1484152ff0 con 0x7f14840b1e10 2023-10-18T09:07:01.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.627+0000 7f1488839640 1 -- 172.21.15.40:0/2249799399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14840ca890 con 0x7f14840b1e10 2023-10-18T09:07:01.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.628+0000 7f146bfff640 1 -- 172.21.15.40:0/2249799399 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f14780114e0 con 0x7f14840b1e10 2023-10-18T09:07:01.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.629+0000 7f146bfff640 1 --2- 172.21.15.40:0/2249799399 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f147006ae40 0x7f147006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:01.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.629+0000 7f146bfff640 1 -- 172.21.15.40:0/2249799399 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1478089590 con 0x7f14840b1e10 2023-10-18T09:07:01.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.630+0000 7f14837fe640 1 --2- 172.21.15.40:0/2249799399 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f147006ae40 0x7f147006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:01.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.631+0000 7f14837fe640 1 --2- 172.21.15.40:0/2249799399 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f147006ae40 0x7f147006d300 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1474007fd0 tx=0x7f1474009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:01.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.634+0000 7f146bfff640 1 -- 172.21.15.40:0/2249799399 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14780545f0 con 0x7f14840b1e10 2023-10-18T09:07:01.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.874+0000 7f1488839640 1 -- 172.21.15.40:0/2249799399 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f14841e30a0 con 0x7f14840b1e10 2023-10-18T09:07:01.877 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:07:01.875+0000 7f6a0f710640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T09:07:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.879+0000 7f146bfff640 1 -- 172.21.15.40:0/2249799399 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1478057c10 con 0x7f14840b1e10 2023-10-18T09:07:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T09:07:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.886+0000 7f1488839640 1 -- 172.21.15.40:0/2249799399 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f147006ae40 msgr2=0x7f147006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.886+0000 7f1488839640 1 --2- 172.21.15.40:0/2249799399 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f147006ae40 0x7f147006d300 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1474007fd0 tx=0x7f1474009040 comp rx=0 tx=0).stop 2023-10-18T09:07:01.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.886+0000 7f1488839640 1 -- 172.21.15.40:0/2249799399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14840b1e10 msgr2=0x7f1484151c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.886+0000 7f1488839640 1 --2- 172.21.15.40:0/2249799399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14840b1e10 0x7f1484151c70 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f147800c490 tx=0x7f147800c950 comp rx=0 tx=0).stop 2023-10-18T09:07:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.886+0000 7f1488839640 1 -- 172.21.15.40:0/2249799399 shutdown_connections 2023-10-18T09:07:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.886+0000 7f1488839640 1 --2- 172.21.15.40:0/2249799399 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f147006ae40 0x7f147006d300 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.886+0000 7f1488839640 1 --2- 172.21.15.40:0/2249799399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14840ca0d0 0x7f1484156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.886+0000 7f1488839640 1 --2- 172.21.15.40:0/2249799399 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14840b1e10 0x7f1484151c70 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.886+0000 7f1488839640 1 --2- 172.21.15.40:0/2249799399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f14840b15b0 0x7f1484144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.886+0000 7f1488839640 1 -- 172.21.15.40:0/2249799399 >> 172.21.15.40:0/2249799399 conn(0x7f1484069720 msgr2=0x7f148414fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.887+0000 7f1488839640 1 -- 172.21.15.40:0/2249799399 shutdown_connections 2023-10-18T09:07:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:01.887+0000 7f1488839640 1 -- 172.21.15.40:0/2249799399 wait complete. 2023-10-18T09:07:01.910 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T09:07:01.976 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T09:07:02.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.113+0000 7fcac2432640 1 -- 172.21.15.40:0/3455300939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc06f570 msgr2=0x7fcabc06f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:02.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.113+0000 7fcac2432640 1 --2- 172.21.15.40:0/3455300939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc06f570 0x7fcabc06f9e0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fcab4005fd0 tx=0x7fcab402c2d0 comp rx=0 tx=0).stop 2023-10-18T09:07:02.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.113+0000 7fcac2432640 1 -- 172.21.15.40:0/3455300939 shutdown_connections 2023-10-18T09:07:02.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.113+0000 7fcac2432640 1 --2- 172.21.15.40:0/3455300939 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc06f570 0x7fcabc06f9e0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:02.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.113+0000 7fcac2432640 1 --2- 172.21.15.40:0/3455300939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcabc06d660 0x7fcabc0cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:02.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.113+0000 7fcac2432640 1 -- 172.21.15.40:0/3455300939 >> 172.21.15.40:0/3455300939 conn(0x7fcabc069720 msgr2=0x7fcabc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:02.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.113+0000 7fcac2432640 1 -- 172.21.15.40:0/3455300939 shutdown_connections 2023-10-18T09:07:02.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.113+0000 7fcac2432640 1 -- 172.21.15.40:0/3455300939 wait complete. 2023-10-18T09:07:02.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.115+0000 7fcac2432640 1 Processor -- start 2023-10-18T09:07:02.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.115+0000 7fcac2432640 1 -- start start 2023-10-18T09:07:02.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.116+0000 7fcac2432640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc06d660 0x7fcabc0ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:02.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.116+0000 7fcac2432640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcabc06f570 0x7fcabc0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:02.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.116+0000 7fcac2432640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcabc157800 0x7fcabc14f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:02.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.116+0000 7fcac2432640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc1f20f0 con 0x7fcabc06d660 2023-10-18T09:07:02.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.116+0000 7fcac2432640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc1f23d0 con 0x7fcabc06f570 2023-10-18T09:07:02.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.116+0000 7fcac2432640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc1f26b0 con 0x7fcabc157800 2023-10-18T09:07:02.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.116+0000 7fcabbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc06d660 0x7fcabc0ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:02.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.117+0000 7fcabbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc06d660 0x7fcabc0ace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43512/0 (socket says 172.21.15.40:43512) 2023-10-18T09:07:02.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.117+0000 7fcabbfff640 1 -- 172.21.15.40:0/438539133 learned_addr learned my addr 172.21.15.40:0/438539133 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:02.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.117+0000 7fcab37fe640 1 --2- 172.21.15.40:0/438539133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcabc06f570 0x7fcabc0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:02.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.117+0000 7fcac09cf640 1 --2- 172.21.15.40:0/438539133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcabc157800 0x7fcabc14f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:02.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.117+0000 7fcabbfff640 1 -- 172.21.15.40:0/438539133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcabc157800 msgr2=0x7fcabc14f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:02.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.117+0000 7fcabbfff640 1 --2- 172.21.15.40:0/438539133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcabc157800 0x7fcabc14f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:02.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.117+0000 7fcabbfff640 1 -- 172.21.15.40:0/438539133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcabc06f570 msgr2=0x7fcabc0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:02.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.117+0000 7fcabbfff640 1 --2- 172.21.15.40:0/438539133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcabc06f570 0x7fcabc0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:02.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.117+0000 7fcabbfff640 1 -- 172.21.15.40:0/438539133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab4009900 con 0x7fcabc06d660 2023-10-18T09:07:02.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.117+0000 7fcab37fe640 1 --2- 172.21.15.40:0/438539133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcabc06f570 0x7fcabc0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:02.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.117+0000 7fcabbfff640 1 --2- 172.21.15.40:0/438539133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc06d660 0x7fcabc0ace20 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00c890 tx=0x7fcaac00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:02.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.118+0000 7fcab97fa640 1 -- 172.21.15.40:0/438539133 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac004220 con 0x7fcabc06d660 2023-10-18T09:07:02.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.118+0000 7fcab97fa640 1 -- 172.21.15.40:0/438539133 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcaac004470 con 0x7fcabc06d660 2023-10-18T09:07:02.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.118+0000 7fcac2432640 1 -- 172.21.15.40:0/438539133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcabc1f2990 con 0x7fcabc06d660 2023-10-18T09:07:02.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.118+0000 7fcab97fa640 1 -- 172.21.15.40:0/438539133 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac007c10 con 0x7fcabc06d660 2023-10-18T09:07:02.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.118+0000 7fcac2432640 1 -- 172.21.15.40:0/438539133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcabc0ad360 con 0x7fcabc06d660 2023-10-18T09:07:02.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.119+0000 7fcac2432640 1 -- 172.21.15.40:0/438539133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcabc1f2eb0 con 0x7fcabc06d660 2023-10-18T09:07:02.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.120+0000 7fcab97fa640 1 -- 172.21.15.40:0/438539133 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcaac005020 con 0x7fcabc06d660 2023-10-18T09:07:02.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.121+0000 7fcab97fa640 1 --2- 172.21.15.40:0/438539133 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fca9c06ae40 0x7fca9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:02.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.121+0000 7fcab97fa640 1 -- 172.21.15.40:0/438539133 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fcaac089cd0 con 0x7fcabc06d660 2023-10-18T09:07:02.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.122+0000 7fcab37fe640 1 --2- 172.21.15.40:0/438539133 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fca9c06ae40 0x7fca9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:02.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.123+0000 7fcab37fe640 1 --2- 172.21.15.40:0/438539133 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fca9c06ae40 0x7fca9c06d300 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fcabc1efb40 tx=0x7fcaa4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:02.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.125+0000 7fcab97fa640 1 -- 172.21.15.40:0/438539133 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcabc1f2eb0 con 0x7fcabc06d660 2023-10-18T09:07:02.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.357+0000 7fcac2432640 1 -- 172.21.15.40:0/438539133 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcabc1f2eb0 con 0x7fcabc06d660 2023-10-18T09:07:02.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.882+0000 7fcab97fa640 1 -- 172.21.15.40:0/438539133 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcaac01d020 con 0x7fcabc06d660 2023-10-18T09:07:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.886+0000 7fcac2432640 1 -- 172.21.15.40:0/438539133 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fca9c06ae40 msgr2=0x7fca9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:02.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.886+0000 7fcac2432640 1 --2- 172.21.15.40:0/438539133 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fca9c06ae40 0x7fca9c06d300 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fcabc1efb40 tx=0x7fcaa4009040 comp rx=0 tx=0).stop 2023-10-18T09:07:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.886+0000 7fcac2432640 1 -- 172.21.15.40:0/438539133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc06d660 msgr2=0x7fcabc0ace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.886+0000 7fcac2432640 1 --2- 172.21.15.40:0/438539133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc06d660 0x7fcabc0ace20 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00c890 tx=0x7fcaac00cd50 comp rx=0 tx=0).stop 2023-10-18T09:07:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.886+0000 7fcac2432640 1 -- 172.21.15.40:0/438539133 shutdown_connections 2023-10-18T09:07:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.886+0000 7fcac2432640 1 --2- 172.21.15.40:0/438539133 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fca9c06ae40 0x7fca9c06d300 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:02.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.886+0000 7fcac2432640 1 --2- 172.21.15.40:0/438539133 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcabc157800 0x7fcabc14f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.886+0000 7fcac2432640 1 --2- 172.21.15.40:0/438539133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcabc06f570 0x7fcabc0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.886+0000 7fcac2432640 1 --2- 172.21.15.40:0/438539133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcabc06d660 0x7fcabc0ace20 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.886+0000 7fcac2432640 1 -- 172.21.15.40:0/438539133 >> 172.21.15.40:0/438539133 conn(0x7fcabc069720 msgr2=0x7fcabc156bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.886+0000 7fcac2432640 1 -- 172.21.15.40:0/438539133 shutdown_connections 2023-10-18T09:07:02.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:02.886+0000 7fcac2432640 1 -- 172.21.15.40:0/438539133 wait complete. 2023-10-18T09:07:02.909 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T09:07:03.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.127+0000 7f04f28a3640 1 -- 172.21.15.40:0/1601673184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ec15b580 msgr2=0x7f04ec0cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:03.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.127+0000 7f04f28a3640 1 --2- 172.21.15.40:0/1601673184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ec15b580 0x7f04ec0cf250 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0b3e00 tx=0x7f04e802bdb0 comp rx=0 tx=0).stop 2023-10-18T09:07:03.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.127+0000 7f04f28a3640 1 -- 172.21.15.40:0/1601673184 shutdown_connections 2023-10-18T09:07:03.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.127+0000 7f04f28a3640 1 --2- 172.21.15.40:0/1601673184 >> v2:172.21.15.144:3300/0 conn(0x7f04ec0b0090 0x7f04ec0cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:03.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.127+0000 7f04f28a3640 1 --2- 172.21.15.40:0/1601673184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ec15b580 0x7f04ec0cf250 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:03.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.127+0000 7f04f28a3640 1 -- 172.21.15.40:0/1601673184 >> 172.21.15.40:0/1601673184 conn(0x7f04ec09a5d0 msgr2=0x7f04ec0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:03.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.127+0000 7f04f28a3640 1 -- 172.21.15.40:0/1601673184 shutdown_connections 2023-10-18T09:07:03.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.127+0000 7f04f28a3640 1 -- 172.21.15.40:0/1601673184 wait complete. 2023-10-18T09:07:03.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.129+0000 7f04f28a3640 1 Processor -- start 2023-10-18T09:07:03.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.129+0000 7f04f28a3640 1 -- start start 2023-10-18T09:07:03.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.130+0000 7f04f28a3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ec0b0090 0x7f04ec14fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:03.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.130+0000 7f04f28a3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04ec0cfd90 0x7f04ec0c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:03.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.130+0000 7f04f28a3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04ec15b580 0x7f04ec0c52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:03.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.130+0000 7f04f28a3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec0c6520 con 0x7f04ec0b0090 2023-10-18T09:07:03.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.130+0000 7f04f28a3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec0c42c0 con 0x7f04ec0cfd90 2023-10-18T09:07:03.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.130+0000 7f04f28a3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec0c5910 con 0x7f04ec15b580 2023-10-18T09:07:03.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.131+0000 7f04f18a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ec0b0090 0x7f04ec14fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:03.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.131+0000 7f04f18a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ec0b0090 0x7f04ec14fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43522/0 (socket says 172.21.15.40:43522) 2023-10-18T09:07:03.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.131+0000 7f04f18a1640 1 -- 172.21.15.40:0/1961479158 learned_addr learned my addr 172.21.15.40:0/1961479158 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:03.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.131+0000 7f04f20a2640 1 --2- 172.21.15.40:0/1961479158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04ec15b580 0x7f04ec0c52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:03.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.131+0000 7f04f18a1640 1 -- 172.21.15.40:0/1961479158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04ec15b580 msgr2=0x7f04ec0c52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:03.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.131+0000 7f04f18a1640 1 --2- 172.21.15.40:0/1961479158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04ec15b580 0x7f04ec0c52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:03.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.131+0000 7f04f18a1640 1 -- 172.21.15.40:0/1961479158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04ec0cfd90 msgr2=0x7f04ec0c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:03.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.131+0000 7f04f10a0640 1 --2- 172.21.15.40:0/1961479158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04ec0cfd90 0x7f04ec0c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:03.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.131+0000 7f04f18a1640 1 --2- 172.21.15.40:0/1961479158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04ec0cfd90 0x7f04ec0c4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:03.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.131+0000 7f04f18a1640 1 -- 172.21.15.40:0/1961479158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e8009560 con 0x7f04ec0b0090 2023-10-18T09:07:03.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.131+0000 7f04f20a2640 1 --2- 172.21.15.40:0/1961479158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04ec15b580 0x7f04ec0c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:03.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.132+0000 7f04f18a1640 1 --2- 172.21.15.40:0/1961479158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ec0b0090 0x7f04ec14fa80 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009850 tx=0x7f04e802dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:03.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.132+0000 7f04e27fc640 1 -- 172.21.15.40:0/1961479158 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e8005650 con 0x7f04ec0b0090 2023-10-18T09:07:03.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.132+0000 7f04e27fc640 1 -- 172.21.15.40:0/1961479158 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f04e80038f0 con 0x7f04ec0b0090 2023-10-18T09:07:03.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.132+0000 7f04e27fc640 1 -- 172.21.15.40:0/1961479158 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e8003ce0 con 0x7f04ec0b0090 2023-10-18T09:07:03.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.132+0000 7f04f28a3640 1 -- 172.21.15.40:0/1961479158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ec0c5bf0 con 0x7f04ec0b0090 2023-10-18T09:07:03.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.132+0000 7f04f28a3640 1 -- 172.21.15.40:0/1961479158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ec0c61b0 con 0x7f04ec0b0090 2023-10-18T09:07:03.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.133+0000 7f04f28a3640 1 -- 172.21.15.40:0/1961479158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04ec071da0 con 0x7f04ec0b0090 2023-10-18T09:07:03.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.134+0000 7f04e27fc640 1 -- 172.21.15.40:0/1961479158 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f04e803e450 con 0x7f04ec0b0090 2023-10-18T09:07:03.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.135+0000 7f04e27fc640 1 --2- 172.21.15.40:0/1961479158 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f04d006ae40 0x7f04d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:03.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.135+0000 7f04e27fc640 1 -- 172.21.15.40:0/1961479158 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f04e80ac060 con 0x7f04ec0b0090 2023-10-18T09:07:03.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.136+0000 7f04f10a0640 1 --2- 172.21.15.40:0/1961479158 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f04d006ae40 0x7f04d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:03.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.137+0000 7f04f10a0640 1 --2- 172.21.15.40:0/1961479158 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f04d006ae40 0x7f04d006d300 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f04ec153410 tx=0x7f04dc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:03.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.139+0000 7f04e27fc640 1 -- 172.21.15.40:0/1961479158 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f04e80deca0 con 0x7f04ec0b0090 2023-10-18T09:07:03.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:03.337+0000 7f04f28a3640 1 -- 172.21.15.40:0/1961479158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f04ec066e00 con 0x7f04ec0b0090 2023-10-18T09:07:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.249+0000 7f04e27fc640 1 -- 172.21.15.40:0/1961479158 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v169) v1 ==== 181+0+0 (secure 0 0 0) 0x7f04e807ab00 con 0x7f04ec0b0090 2023-10-18T09:07:04.251 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-10-18T09:07:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.253+0000 7f04f28a3640 1 -- 172.21.15.40:0/1961479158 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f04d006ae40 msgr2=0x7f04d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.253+0000 7f04f28a3640 1 --2- 172.21.15.40:0/1961479158 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f04d006ae40 0x7f04d006d300 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f04ec153410 tx=0x7f04dc008040 comp rx=0 tx=0).stop 2023-10-18T09:07:04.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.253+0000 7f04f28a3640 1 -- 172.21.15.40:0/1961479158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ec0b0090 msgr2=0x7f04ec14fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:04.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.253+0000 7f04f28a3640 1 --2- 172.21.15.40:0/1961479158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ec0b0090 0x7f04ec14fa80 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009850 tx=0x7f04e802dd80 comp rx=0 tx=0).stop 2023-10-18T09:07:04.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.253+0000 7f04f28a3640 1 -- 172.21.15.40:0/1961479158 shutdown_connections 2023-10-18T09:07:04.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.253+0000 7f04f28a3640 1 --2- 172.21.15.40:0/1961479158 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f04d006ae40 0x7f04d006d300 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:04.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.253+0000 7f04f28a3640 1 --2- 172.21.15.40:0/1961479158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04ec15b580 0x7f04ec0c52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:04.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.253+0000 7f04f28a3640 1 --2- 172.21.15.40:0/1961479158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04ec0cfd90 0x7f04ec0c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:04.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.253+0000 7f04f28a3640 1 --2- 172.21.15.40:0/1961479158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04ec0b0090 0x7f04ec14fa80 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:04.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.253+0000 7f04f28a3640 1 -- 172.21.15.40:0/1961479158 >> 172.21.15.40:0/1961479158 conn(0x7f04ec09a5d0 msgr2=0x7f04ec15bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:04.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.254+0000 7f04f28a3640 1 -- 172.21.15.40:0/1961479158 shutdown_connections 2023-10-18T09:07:04.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.254+0000 7f04f28a3640 1 -- 172.21.15.40:0/1961479158 wait complete. 2023-10-18T09:07:04.277 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T09:07:04.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.511+0000 7f047ef35640 1 -- 172.21.15.40:0/1560211628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f047813eaa0 msgr2=0x7f047813eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:04.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.511+0000 7f047ef35640 1 --2- 172.21.15.40:0/1560211628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f047813eaa0 0x7f047813eef0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f047806a6c0 tx=0x7f047002bda0 comp rx=0 tx=0).stop 2023-10-18T09:07:04.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.511+0000 7f047ef35640 1 -- 172.21.15.40:0/1560211628 shutdown_connections 2023-10-18T09:07:04.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.511+0000 7f047ef35640 1 --2- 172.21.15.40:0/1560211628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f047813eaa0 0x7f047813eef0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:04.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.511+0000 7f047ef35640 1 --2- 172.21.15.40:0/1560211628 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04780c9830 0x7f04780c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:04.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.511+0000 7f047ef35640 1 --2- 172.21.15.40:0/1560211628 >> v2:172.21.15.144:3300/0 conn(0x7f047813c220 0x7f04780cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:04.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.511+0000 7f047ef35640 1 -- 172.21.15.40:0/1560211628 >> 172.21.15.40:0/1560211628 conn(0x7f04780aeb90 msgr2=0x7f04780aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:04.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.511+0000 7f047ef35640 1 -- 172.21.15.40:0/1560211628 shutdown_connections 2023-10-18T09:07:04.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.511+0000 7f047ef35640 1 -- 172.21.15.40:0/1560211628 wait complete. 2023-10-18T09:07:04.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.513+0000 7f047ef35640 1 Processor -- start 2023-10-18T09:07:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.513+0000 7f047ef35640 1 -- start start 2023-10-18T09:07:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.514+0000 7f047ef35640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04780c9830 0x7f047806fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.514+0000 7f047ef35640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f047813c220 0x7f0478159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:04.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.514+0000 7f047ef35640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f047813eaa0 0x7f0478153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:04.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.514+0000 7f047ef35640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047815a1f0 con 0x7f04780c9830 2023-10-18T09:07:04.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.514+0000 7f047ef35640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0478070550 con 0x7f047813eaa0 2023-10-18T09:07:04.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.514+0000 7f047ef35640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0478070830 con 0x7f047813c220 2023-10-18T09:07:04.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.514+0000 7f047df33640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04780c9830 0x7f047806fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:04.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.514+0000 7f047df33640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04780c9830 0x7f047806fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43542/0 (socket says 172.21.15.40:43542) 2023-10-18T09:07:04.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.514+0000 7f047df33640 1 -- 172.21.15.40:0/2864196420 learned_addr learned my addr 172.21.15.40:0/2864196420 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:04.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.515+0000 7f047e734640 1 --2- 172.21.15.40:0/2864196420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f047813eaa0 0x7f0478153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:04.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.515+0000 7f047d732640 1 --2- 172.21.15.40:0/2864196420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f047813c220 0x7f0478159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:04.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.515+0000 7f047df33640 1 -- 172.21.15.40:0/2864196420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f047813c220 msgr2=0x7f0478159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:04.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.515+0000 7f047df33640 1 --2- 172.21.15.40:0/2864196420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f047813c220 0x7f0478159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:04.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.515+0000 7f047df33640 1 -- 172.21.15.40:0/2864196420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f047813eaa0 msgr2=0x7f0478153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:04.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.515+0000 7f047df33640 1 --2- 172.21.15.40:0/2864196420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f047813eaa0 0x7f0478153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:04.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.515+0000 7f047df33640 1 -- 172.21.15.40:0/2864196420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0470009560 con 0x7f04780c9830 2023-10-18T09:07:04.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.515+0000 7f047d732640 1 --2- 172.21.15.40:0/2864196420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f047813c220 0x7f0478159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:04.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.515+0000 7f047e734640 1 --2- 172.21.15.40:0/2864196420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f047813eaa0 0x7f0478153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:04.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.515+0000 7f047df33640 1 --2- 172.21.15.40:0/2864196420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04780c9830 0x7f047806fe70 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f047400eab0 tx=0x7f047400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:04.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.516+0000 7f046e7fc640 1 -- 172.21.15.40:0/2864196420 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047400ce40 con 0x7f04780c9830 2023-10-18T09:07:04.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.516+0000 7f046e7fc640 1 -- 172.21.15.40:0/2864196420 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0474011560 con 0x7f04780c9830 2023-10-18T09:07:04.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.516+0000 7f046e7fc640 1 -- 172.21.15.40:0/2864196420 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0474011910 con 0x7f04780c9830 2023-10-18T09:07:04.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.516+0000 7f047ef35640 1 -- 172.21.15.40:0/2864196420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0478070b10 con 0x7f04780c9830 2023-10-18T09:07:04.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.516+0000 7f047ef35640 1 -- 172.21.15.40:0/2864196420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0478153870 con 0x7f04780c9830 2023-10-18T09:07:04.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.517+0000 7f047ef35640 1 -- 172.21.15.40:0/2864196420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0478066e00 con 0x7f04780c9830 2023-10-18T09:07:04.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.518+0000 7f046e7fc640 1 -- 172.21.15.40:0/2864196420 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0474011a70 con 0x7f04780c9830 2023-10-18T09:07:04.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.519+0000 7f046e7fc640 1 --2- 172.21.15.40:0/2864196420 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f045c06ae70 0x7f045c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:04.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.519+0000 7f046e7fc640 1 -- 172.21.15.40:0/2864196420 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f047408a2e0 con 0x7f04780c9830 2023-10-18T09:07:04.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.519+0000 7f047d732640 1 --2- 172.21.15.40:0/2864196420 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f045c06ae70 0x7f045c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:04.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.521+0000 7f047d732640 1 --2- 172.21.15.40:0/2864196420 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f045c06ae70 0x7f045c06d330 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f047813b2d0 tx=0x7f0468005e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:04.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.523+0000 7f046e7fc640 1 -- 172.21.15.40:0/2864196420 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0474055380 con 0x7f04780c9830 2023-10-18T09:07:04.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:04.720+0000 7f047ef35640 1 -- 172.21.15.40:0/2864196420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f04780cba10 con 0x7f04780c9830 2023-10-18T09:07:05.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.259+0000 7f046e7fc640 1 -- 172.21.15.40:0/2864196420 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v170) v1 ==== 169+0+0 (secure 0 0 0) 0x7f047401f020 con 0x7f04780c9830 2023-10-18T09:07:05.262 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-10-18T09:07:05.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.264+0000 7f047ef35640 1 -- 172.21.15.40:0/2864196420 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f045c06ae70 msgr2=0x7f045c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:05.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.264+0000 7f047ef35640 1 --2- 172.21.15.40:0/2864196420 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f045c06ae70 0x7f045c06d330 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f047813b2d0 tx=0x7f0468005e50 comp rx=0 tx=0).stop 2023-10-18T09:07:05.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.264+0000 7f047ef35640 1 -- 172.21.15.40:0/2864196420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04780c9830 msgr2=0x7f047806fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:05.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.264+0000 7f047ef35640 1 --2- 172.21.15.40:0/2864196420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04780c9830 0x7f047806fe70 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f047400eab0 tx=0x7f047400ef70 comp rx=0 tx=0).stop 2023-10-18T09:07:05.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.264+0000 7f047ef35640 1 -- 172.21.15.40:0/2864196420 shutdown_connections 2023-10-18T09:07:05.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.264+0000 7f047ef35640 1 --2- 172.21.15.40:0/2864196420 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f045c06ae70 0x7f045c06d330 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.264+0000 7f047ef35640 1 --2- 172.21.15.40:0/2864196420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f047813eaa0 0x7f0478153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.264+0000 7f047ef35640 1 --2- 172.21.15.40:0/2864196420 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f047813c220 0x7f0478159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.264+0000 7f047ef35640 1 --2- 172.21.15.40:0/2864196420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f04780c9830 0x7f047806fe70 secure :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f047400eab0 tx=0x7f047400ef70 comp rx=0 tx=0).stop 2023-10-18T09:07:05.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.264+0000 7f047ef35640 1 -- 172.21.15.40:0/2864196420 >> 172.21.15.40:0/2864196420 conn(0x7f04780aeb90 msgr2=0x7f04780c9080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.265+0000 7f047ef35640 1 -- 172.21.15.40:0/2864196420 shutdown_connections 2023-10-18T09:07:05.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.265+0000 7f047ef35640 1 -- 172.21.15.40:0/2864196420 wait complete. 2023-10-18T09:07:05.288 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 22 2023-10-18T09:07:05.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.506+0000 7f364a0f2640 1 -- 172.21.15.40:0/2573781225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440cfd70 msgr2=0x7f364415bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:05.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.506+0000 7f364a0f2640 1 --2- 172.21.15.40:0/2573781225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440cfd70 0x7f364415bf20 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f364406ff10 tx=0x7f363c02bff0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.506+0000 7f364a0f2640 1 -- 172.21.15.40:0/2573781225 shutdown_connections 2023-10-18T09:07:05.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.506+0000 7f364a0f2640 1 --2- 172.21.15.40:0/2573781225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440cfd70 0x7f364415bf20 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.506+0000 7f364a0f2640 1 --2- 172.21.15.40:0/2573781225 >> v2:172.21.15.144:3300/0 conn(0x7f364406e3b0 0x7f364415b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.506+0000 7f364a0f2640 1 -- 172.21.15.40:0/2573781225 >> 172.21.15.40:0/2573781225 conn(0x7f36440655f0 msgr2=0x7f36440659d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:05.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.506+0000 7f364a0f2640 1 -- 172.21.15.40:0/2573781225 shutdown_connections 2023-10-18T09:07:05.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.506+0000 7f364a0f2640 1 -- 172.21.15.40:0/2573781225 wait complete. 2023-10-18T09:07:05.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.508+0000 7f364a0f2640 1 Processor -- start 2023-10-18T09:07:05.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.508+0000 7f364a0f2640 1 -- start start 2023-10-18T09:07:05.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.509+0000 7f364a0f2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f364406e3b0 0x7f36440ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:05.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.509+0000 7f364a0f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440cfd70 0x7f364414dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:05.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.509+0000 7f364a0f2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36441590f0 0x7f364415a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:05.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.509+0000 7f364a0f2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36440cb470 con 0x7f36440cfd70 2023-10-18T09:07:05.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.509+0000 7f364a0f2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36440cbe90 con 0x7f36441590f0 2023-10-18T09:07:05.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.509+0000 7f364a0f2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36440cc730 con 0x7f364406e3b0 2023-10-18T09:07:05.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.510+0000 7f3642ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440cfd70 0x7f364414dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:05.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.510+0000 7f3642ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440cfd70 0x7f364414dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53552/0 (socket says 172.21.15.40:53552) 2023-10-18T09:07:05.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.510+0000 7f3642ffd640 1 -- 172.21.15.40:0/2570919754 learned_addr learned my addr 172.21.15.40:0/2570919754 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:05.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.510+0000 7f3643fff640 1 --2- 172.21.15.40:0/2570919754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36441590f0 0x7f364415a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:05.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.510+0000 7f36437fe640 1 --2- 172.21.15.40:0/2570919754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f364406e3b0 0x7f36440ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:05.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.510+0000 7f3642ffd640 1 -- 172.21.15.40:0/2570919754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f364406e3b0 msgr2=0x7f36440ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:05.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.510+0000 7f3642ffd640 1 --2- 172.21.15.40:0/2570919754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f364406e3b0 0x7f36440ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.510+0000 7f3642ffd640 1 -- 172.21.15.40:0/2570919754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36441590f0 msgr2=0x7f364415a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:05.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.510+0000 7f3642ffd640 1 --2- 172.21.15.40:0/2570919754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36441590f0 0x7f364415a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.510+0000 7f3642ffd640 1 -- 172.21.15.40:0/2570919754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c009840 con 0x7f36440cfd70 2023-10-18T09:07:05.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.510+0000 7f36437fe640 1 --2- 172.21.15.40:0/2570919754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f364406e3b0 0x7f36440ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:05.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.510+0000 7f3642ffd640 1 --2- 172.21.15.40:0/2570919754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440cfd70 0x7f364414dd80 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f363800c890 tx=0x7f363800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:05.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.511+0000 7f362bfff640 1 -- 172.21.15.40:0/2570919754 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638004220 con 0x7f36440cfd70 2023-10-18T09:07:05.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.511+0000 7f362bfff640 1 -- 172.21.15.40:0/2570919754 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3638004470 con 0x7f36440cfd70 2023-10-18T09:07:05.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.511+0000 7f362bfff640 1 -- 172.21.15.40:0/2570919754 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638007c10 con 0x7f36440cfd70 2023-10-18T09:07:05.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.511+0000 7f364a0f2640 1 -- 172.21.15.40:0/2570919754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36440ca620 con 0x7f36440cfd70 2023-10-18T09:07:05.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.511+0000 7f364a0f2640 1 -- 172.21.15.40:0/2570919754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36440ca880 con 0x7f36440cfd70 2023-10-18T09:07:05.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.512+0000 7f364a0f2640 1 -- 172.21.15.40:0/2570919754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36440663a0 con 0x7f36440cfd70 2023-10-18T09:07:05.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.513+0000 7f362bfff640 1 -- 172.21.15.40:0/2570919754 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3638005020 con 0x7f36440cfd70 2023-10-18T09:07:05.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.514+0000 7f362bfff640 1 --2- 172.21.15.40:0/2570919754 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f363006ae70 0x7f363006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:05.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.514+0000 7f362bfff640 1 -- 172.21.15.40:0/2570919754 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3638089880 con 0x7f36440cfd70 2023-10-18T09:07:05.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.515+0000 7f36437fe640 1 --2- 172.21.15.40:0/2570919754 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f363006ae70 0x7f363006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:05.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.516+0000 7f36437fe640 1 --2- 172.21.15.40:0/2570919754 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f363006ae70 0x7f363006d330 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f36340011e0 tx=0x7f3634009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:05.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.518+0000 7f362bfff640 1 -- 172.21.15.40:0/2570919754 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3638054c20 con 0x7f36440cfd70 2023-10-18T09:07:05.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.713+0000 7f364a0f2640 1 -- 172.21.15.40:0/2570919754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config reset", "num": 22} v 0) v1 -- 0x7f3644066fd0 con 0x7f36440cfd70 2023-10-18T09:07:05.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.721+0000 7f362bfff640 1 -- 172.21.15.40:0/2570919754 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 22}]=0 v38) v1 ==== 71+0+0 (secure 0 0 0) 0x7f363801d020 con 0x7f36440cfd70 2023-10-18T09:07:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.724+0000 7f364a0f2640 1 -- 172.21.15.40:0/2570919754 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f363006ae70 msgr2=0x7f363006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.724+0000 7f364a0f2640 1 --2- 172.21.15.40:0/2570919754 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f363006ae70 0x7f363006d330 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f36340011e0 tx=0x7f3634009040 comp rx=0 tx=0).stop 2023-10-18T09:07:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.724+0000 7f364a0f2640 1 -- 172.21.15.40:0/2570919754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440cfd70 msgr2=0x7f364414dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:05.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.724+0000 7f364a0f2640 1 --2- 172.21.15.40:0/2570919754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440cfd70 0x7f364414dd80 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f363800c890 tx=0x7f363800cd50 comp rx=0 tx=0).stop 2023-10-18T09:07:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.724+0000 7f364a0f2640 1 -- 172.21.15.40:0/2570919754 shutdown_connections 2023-10-18T09:07:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.724+0000 7f364a0f2640 1 --2- 172.21.15.40:0/2570919754 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f363006ae70 0x7f363006d330 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.724+0000 7f364a0f2640 1 --2- 172.21.15.40:0/2570919754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36441590f0 0x7f364415a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.724+0000 7f364a0f2640 1 --2- 172.21.15.40:0/2570919754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36440cfd70 0x7f364414dd80 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.724+0000 7f364a0f2640 1 --2- 172.21.15.40:0/2570919754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f364406e3b0 0x7f36440ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.724+0000 7f364a0f2640 1 -- 172.21.15.40:0/2570919754 >> 172.21.15.40:0/2570919754 conn(0x7f36440655f0 msgr2=0x7f364415ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:05.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.724+0000 7f364a0f2640 1 -- 172.21.15.40:0/2570919754 shutdown_connections 2023-10-18T09:07:05.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.725+0000 7f364a0f2640 1 -- 172.21.15.40:0/2570919754 wait complete. 2023-10-18T09:07:05.746 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2023-10-18T09:07:05.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.974+0000 7ff71154e640 1 -- 172.21.15.40:0/2842399104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff70c0ca160 msgr2=0x7ff70c0cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.974+0000 7ff71154e640 1 --2- 172.21.15.40:0/2842399104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff70c0ca160 0x7ff70c0cafc0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0ccea0 tx=0x7ff70402c2f0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.975+0000 7ff71154e640 1 -- 172.21.15.40:0/2842399104 shutdown_connections 2023-10-18T09:07:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.975+0000 7ff71154e640 1 --2- 172.21.15.40:0/2842399104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff70c0ca160 0x7ff70c0cafc0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.975+0000 7ff71154e640 1 --2- 172.21.15.40:0/2842399104 >> v2:172.21.15.144:3300/0 conn(0x7ff70c1420f0 0x7ff70c0b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.975+0000 7ff71154e640 1 -- 172.21.15.40:0/2842399104 >> 172.21.15.40:0/2842399104 conn(0x7ff70c069720 msgr2=0x7ff70c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:05.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.975+0000 7ff71154e640 1 -- 172.21.15.40:0/2842399104 shutdown_connections 2023-10-18T09:07:05.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.975+0000 7ff71154e640 1 -- 172.21.15.40:0/2842399104 wait complete. 2023-10-18T09:07:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.976+0000 7ff71154e640 1 Processor -- start 2023-10-18T09:07:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.977+0000 7ff71154e640 1 -- start start 2023-10-18T09:07:05.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.977+0000 7ff71154e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c0ca160 0x7ff70c139520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.977+0000 7ff71154e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff70c1420f0 0x7ff70c13c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.977+0000 7ff71154e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff70c1425a0 0x7ff70c13e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.978+0000 7ff71154e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c13cb40 con 0x7ff70c1420f0 2023-10-18T09:07:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.978+0000 7ff71154e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c13ce20 con 0x7ff70c0ca160 2023-10-18T09:07:05.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.978+0000 7ff71154e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c13d100 con 0x7ff70c1425a0 2023-10-18T09:07:05.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.978+0000 7ff70b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff70c1420f0 0x7ff70c13c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:05.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.978+0000 7ff70b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff70c1420f0 0x7ff70c13c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53560/0 (socket says 172.21.15.40:53560) 2023-10-18T09:07:05.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.978+0000 7ff70b7fe640 1 -- 172.21.15.40:0/3536405053 learned_addr learned my addr 172.21.15.40:0/3536405053 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:05.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.978+0000 7ff710d4d640 1 --2- 172.21.15.40:0/3536405053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff70c1425a0 0x7ff70c13e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:05.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.978+0000 7ff70bfff640 1 --2- 172.21.15.40:0/3536405053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c0ca160 0x7ff70c139520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:05.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.978+0000 7ff70b7fe640 1 -- 172.21.15.40:0/3536405053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff70c1425a0 msgr2=0x7ff70c13e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:05.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.978+0000 7ff70b7fe640 1 --2- 172.21.15.40:0/3536405053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff70c1425a0 0x7ff70c13e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.978+0000 7ff70b7fe640 1 -- 172.21.15.40:0/3536405053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c0ca160 msgr2=0x7ff70c139520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:05.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.978+0000 7ff70b7fe640 1 --2- 172.21.15.40:0/3536405053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c0ca160 0x7ff70c139520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:05.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.978+0000 7ff70b7fe640 1 -- 172.21.15.40:0/3536405053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff704009900 con 0x7ff70c1420f0 2023-10-18T09:07:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.978+0000 7ff70bfff640 1 --2- 172.21.15.40:0/3536405053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c0ca160 0x7ff70c139520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.979+0000 7ff70b7fe640 1 --2- 172.21.15.40:0/3536405053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff70c1420f0 0x7ff70c13c160 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7ff70000c410 tx=0x7ff70000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.979+0000 7ff708ff9640 1 -- 172.21.15.40:0/3536405053 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7000035f0 con 0x7ff70c1420f0 2023-10-18T09:07:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.979+0000 7ff708ff9640 1 -- 172.21.15.40:0/3536405053 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff700003770 con 0x7ff70c1420f0 2023-10-18T09:07:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.979+0000 7ff708ff9640 1 -- 172.21.15.40:0/3536405053 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff700003b80 con 0x7ff70c1420f0 2023-10-18T09:07:05.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.979+0000 7ff71154e640 1 -- 172.21.15.40:0/3536405053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70c13d3e0 con 0x7ff70c1420f0 2023-10-18T09:07:05.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.979+0000 7ff71154e640 1 -- 172.21.15.40:0/3536405053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70c139a60 con 0x7ff70c1420f0 2023-10-18T09:07:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.980+0000 7ff71154e640 1 -- 172.21.15.40:0/3536405053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff70c13d900 con 0x7ff70c1420f0 2023-10-18T09:07:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.982+0000 7ff708ff9640 1 -- 172.21.15.40:0/3536405053 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff700003ce0 con 0x7ff70c1420f0 2023-10-18T09:07:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.982+0000 7ff708ff9640 1 --2- 172.21.15.40:0/3536405053 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff6f806ae40 0x7ff6f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:05.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.983+0000 7ff708ff9640 1 -- 172.21.15.40:0/3536405053 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff700088ac0 con 0x7ff70c1420f0 2023-10-18T09:07:05.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.983+0000 7ff70bfff640 1 --2- 172.21.15.40:0/3536405053 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff6f806ae40 0x7ff6f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:05.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.984+0000 7ff70bfff640 1 --2- 172.21.15.40:0/3536405053 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff6f806ae40 0x7ff6f806d300 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009b40 tx=0x7ff6fc005e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:05.987+0000 7ff708ff9640 1 -- 172.21.15.40:0/3536405053 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff70c13d900 con 0x7ff70c1420f0 2023-10-18T09:07:06.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.247+0000 7ff71154e640 1 -- 172.21.15.40:0/3536405053 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7ff70c13d900 con 0x7ff70c1420f0 2023-10-18T09:07:06.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.262+0000 7ff708ff9640 1 -- 172.21.15.40:0/3536405053 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v580) v1 ==== 132+0+0 (secure 0 0 0) 0x7ff70c13d900 con 0x7ff70c1420f0 2023-10-18T09:07:06.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.264+0000 7ff71154e640 1 -- 172.21.15.40:0/3536405053 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff6f806ae40 msgr2=0x7ff6f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:06.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.264+0000 7ff71154e640 1 --2- 172.21.15.40:0/3536405053 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff6f806ae40 0x7ff6f806d300 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009b40 tx=0x7ff6fc005e50 comp rx=0 tx=0).stop 2023-10-18T09:07:06.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.264+0000 7ff71154e640 1 -- 172.21.15.40:0/3536405053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff70c1420f0 msgr2=0x7ff70c13c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:06.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.264+0000 7ff71154e640 1 --2- 172.21.15.40:0/3536405053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff70c1420f0 0x7ff70c13c160 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7ff70000c410 tx=0x7ff70000c8d0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.265+0000 7ff71154e640 1 -- 172.21.15.40:0/3536405053 shutdown_connections 2023-10-18T09:07:06.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.265+0000 7ff71154e640 1 --2- 172.21.15.40:0/3536405053 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff6f806ae40 0x7ff6f806d300 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.265+0000 7ff71154e640 1 --2- 172.21.15.40:0/3536405053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff70c1425a0 0x7ff70c13e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.265+0000 7ff71154e640 1 --2- 172.21.15.40:0/3536405053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff70c1420f0 0x7ff70c13c160 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.265+0000 7ff71154e640 1 --2- 172.21.15.40:0/3536405053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c0ca160 0x7ff70c139520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.265+0000 7ff71154e640 1 -- 172.21.15.40:0/3536405053 >> 172.21.15.40:0/3536405053 conn(0x7ff70c069720 msgr2=0x7ff70c0abdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:06.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.265+0000 7ff71154e640 1 -- 172.21.15.40:0/3536405053 shutdown_connections 2023-10-18T09:07:06.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.265+0000 7ff71154e640 1 -- 172.21.15.40:0/3536405053 wait complete. 2023-10-18T09:07:06.286 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-10-18T09:07:06.286 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) That when the full flag is encountered during asynchronous 2023-10-18T09:07:06.287 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) 2023-10-18T09:07:06.287 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:07:06.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.509+0000 7f715ea7f640 1 -- 172.21.15.40:0/1463037670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71440047c0 msgr2=0x7f7144004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:06.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.509+0000 7f715ea7f640 1 --2- 172.21.15.40:0/1463037670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71440047c0 0x7f7144004c40 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f7148009b50 tx=0x7f71440057f0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.509+0000 7f715e27e640 1 -- 172.21.15.40:0/1463037670 reap_dead start 2023-10-18T09:07:06.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.509+0000 7f715ea7f640 1 -- 172.21.15.40:0/1463037670 shutdown_connections 2023-10-18T09:07:06.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.509+0000 7f715ea7f640 1 -- 172.21.15.40:0/1463037670 >> 172.21.15.40:0/1463037670 conn(0x7f715808ad00 msgr2=0x7f715809a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:06.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.510+0000 7f715ea7f640 1 -- 172.21.15.40:0/1463037670 shutdown_connections 2023-10-18T09:07:06.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.510+0000 7f715ea7f640 1 -- 172.21.15.40:0/1463037670 wait complete. 2023-10-18T09:07:06.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.511+0000 7f715ea7f640 1 Processor -- start 2023-10-18T09:07:06.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.511+0000 7f715ea7f640 1 -- start start 2023-10-18T09:07:06.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.512+0000 7f715ea7f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71440047c0 0x7f71580c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:06.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.512+0000 7f715ea7f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7158157590 0x7f7158156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:06.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.512+0000 7f715ea7f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71580b59a0 0x7f71580b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:06.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.512+0000 7f715ea7f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71580b6390 con 0x7f7158157590 2023-10-18T09:07:06.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.512+0000 7f715ea7f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7158157a70 con 0x7f71580b59a0 2023-10-18T09:07:06.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.512+0000 7f715ea7f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7158157d50 con 0x7f71440047c0 2023-10-18T09:07:06.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.513+0000 7f715d27c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7158157590 0x7f7158156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:06.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.513+0000 7f715d27c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7158157590 0x7f7158156d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53576/0 (socket says 172.21.15.40:53576) 2023-10-18T09:07:06.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.513+0000 7f715d27c640 1 -- 172.21.15.40:0/1004523012 learned_addr learned my addr 172.21.15.40:0/1004523012 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:06.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.513+0000 7f715da7d640 1 --2- 172.21.15.40:0/1004523012 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71440047c0 0x7f71580c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:06.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.513+0000 7f715e27e640 1 --2- 172.21.15.40:0/1004523012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71580b59a0 0x7f71580b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:06.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.513+0000 7f715d27c640 1 -- 172.21.15.40:0/1004523012 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71440047c0 msgr2=0x7f71580c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:06.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.513+0000 7f715d27c640 1 --2- 172.21.15.40:0/1004523012 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71440047c0 0x7f71580c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.513+0000 7f715d27c640 1 -- 172.21.15.40:0/1004523012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71580b59a0 msgr2=0x7f71580b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:06.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.513+0000 7f715d27c640 1 --2- 172.21.15.40:0/1004523012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71580b59a0 0x7f71580b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.513+0000 7f715d27c640 1 -- 172.21.15.40:0/1004523012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7158158030 con 0x7f7158157590 2023-10-18T09:07:06.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.513+0000 7f715e27e640 1 --2- 172.21.15.40:0/1004523012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71580b59a0 0x7f71580b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:06.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.513+0000 7f715da7d640 1 --2- 172.21.15.40:0/1004523012 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71440047c0 0x7f71580c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:06.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.513+0000 7f715d27c640 1 --2- 172.21.15.40:0/1004523012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7158157590 0x7f7158156d00 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f714802cbc0 tx=0x7f7148033cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:06.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.514+0000 7f714e7fc640 1 -- 172.21.15.40:0/1004523012 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148004020 con 0x7f7158157590 2023-10-18T09:07:06.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.514+0000 7f714e7fc640 1 -- 172.21.15.40:0/1004523012 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f71480041a0 con 0x7f7158157590 2023-10-18T09:07:06.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.514+0000 7f715ea7f640 1 -- 172.21.15.40:0/1004523012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7158158310 con 0x7f7158157590 2023-10-18T09:07:06.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.514+0000 7f714e7fc640 1 -- 172.21.15.40:0/1004523012 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714803d6e0 con 0x7f7158157590 2023-10-18T09:07:06.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.514+0000 7f715ea7f640 1 -- 172.21.15.40:0/1004523012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7158158d80 con 0x7f7158157590 2023-10-18T09:07:06.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.515+0000 7f715ea7f640 1 -- 172.21.15.40:0/1004523012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7158071da0 con 0x7f7158157590 2023-10-18T09:07:06.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.516+0000 7f714e7fc640 1 -- 172.21.15.40:0/1004523012 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f714803d840 con 0x7f7158157590 2023-10-18T09:07:06.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.517+0000 7f714e7fc640 1 --2- 172.21.15.40:0/1004523012 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f713c06ae40 0x7f713c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:06.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.518+0000 7f714e7fc640 1 -- 172.21.15.40:0/1004523012 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f71480abff0 con 0x7f7158157590 2023-10-18T09:07:06.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.518+0000 7f715da7d640 1 --2- 172.21.15.40:0/1004523012 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f713c06ae40 0x7f713c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:06.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.519+0000 7f715da7d640 1 --2- 172.21.15.40:0/1004523012 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f713c06ae40 0x7f713c06d300 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f71540016d0 tx=0x7f7154009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:06.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.522+0000 7f714e7fc640 1 -- 172.21.15.40:0/1004523012 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7148077390 con 0x7f7158157590 2023-10-18T09:07:06.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.716+0000 7f715ea7f640 1 -- 172.21.15.40:0/1004523012 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7158066e00 con 0x7f7158157590 2023-10-18T09:07:06.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.718+0000 7f714e7fc640 1 -- 172.21.15.40:0/1004523012 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+13009 (secure 0 0 0) 0x7f7148043090 con 0x7f7158157590 2023-10-18T09:07:06.720 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:07:06.721 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":170,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:07:05.250155+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6833/3348437493":"2023-10-19T09:07:01.875726+0000","172.21.15.40:6832/3348437493":"2023-10-19T09:07:01.875726+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:07:06.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.721+0000 7f715ea7f640 1 -- 172.21.15.40:0/1004523012 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f713c06ae40 msgr2=0x7f713c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:06.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.721+0000 7f715ea7f640 1 --2- 172.21.15.40:0/1004523012 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f713c06ae40 0x7f713c06d300 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f71540016d0 tx=0x7f7154009040 comp rx=0 tx=0).stop 2023-10-18T09:07:06.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.721+0000 7f715ea7f640 1 -- 172.21.15.40:0/1004523012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7158157590 msgr2=0x7f7158156d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:06.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.721+0000 7f715ea7f640 1 --2- 172.21.15.40:0/1004523012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7158157590 0x7f7158156d00 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f714802cbc0 tx=0x7f7148033cd0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.721+0000 7f715ea7f640 1 -- 172.21.15.40:0/1004523012 shutdown_connections 2023-10-18T09:07:06.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.721+0000 7f715ea7f640 1 --2- 172.21.15.40:0/1004523012 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f713c06ae40 0x7f713c06d300 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.721+0000 7f715ea7f640 1 --2- 172.21.15.40:0/1004523012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71580b59a0 0x7f71580b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.721+0000 7f715ea7f640 1 --2- 172.21.15.40:0/1004523012 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7158157590 0x7f7158156d00 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.721+0000 7f715ea7f640 1 --2- 172.21.15.40:0/1004523012 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71440047c0 0x7f71580c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.721+0000 7f715ea7f640 1 -- 172.21.15.40:0/1004523012 >> 172.21.15.40:0/1004523012 conn(0x7f715808ad00 msgr2=0x7f71580cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:06.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.721+0000 7f715ea7f640 1 -- 172.21.15.40:0/1004523012 shutdown_connections 2023-10-18T09:07:06.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.721+0000 7f715ea7f640 1 -- 172.21.15.40:0/1004523012 wait complete. 2023-10-18T09:07:06.745 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:07:06.745 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:07:06.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.986+0000 7fa3f7460640 1 -- 172.21.15.40:0/35687734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f015fba0 msgr2=0x7fa3f00cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:06.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.986+0000 7fa3f7460640 1 --2- 172.21.15.40:0/35687734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f015fba0 0x7fa3f00cadc0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00ca810 tx=0x7fa3e802c080 comp rx=0 tx=0).stop 2023-10-18T09:07:06.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.986+0000 7fa3f7460640 1 -- 172.21.15.40:0/35687734 shutdown_connections 2023-10-18T09:07:06.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.986+0000 7fa3f7460640 1 --2- 172.21.15.40:0/35687734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f015fba0 0x7fa3f00cadc0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.986+0000 7fa3f7460640 1 --2- 172.21.15.40:0/35687734 >> v2:172.21.15.144:3300/0 conn(0x7fa3f00ca380 0x7fa3f015f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.986+0000 7fa3f7460640 1 -- 172.21.15.40:0/35687734 >> 172.21.15.40:0/35687734 conn(0x7fa3f0069720 msgr2=0x7fa3f0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:06.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.986+0000 7fa3f7460640 1 -- 172.21.15.40:0/35687734 shutdown_connections 2023-10-18T09:07:06.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.986+0000 7fa3f7460640 1 -- 172.21.15.40:0/35687734 wait complete. 2023-10-18T09:07:06.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.988+0000 7fa3f7460640 1 Processor -- start 2023-10-18T09:07:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.988+0000 7fa3f7460640 1 -- start start 2023-10-18T09:07:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.989+0000 7fa3f7460640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3f00ca380 0x7fa3f0143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.989+0000 7fa3f7460640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f0144c10 0x7fa3f0149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.989+0000 7fa3f7460640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f013d820 0x7fa3f0149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.989+0000 7fa3f7460640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f013e030 con 0x7fa3f0144c10 2023-10-18T09:07:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.989+0000 7fa3f7460640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f0144150 con 0x7fa3f013d820 2023-10-18T09:07:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.989+0000 7fa3f7460640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f013dd50 con 0x7fa3f00ca380 2023-10-18T09:07:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.989+0000 7fa3f5c5d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f0144c10 0x7fa3f0149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.990+0000 7fa3f5c5d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f0144c10 0x7fa3f0149780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53592/0 (socket says 172.21.15.40:53592) 2023-10-18T09:07:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.990+0000 7fa3f5c5d640 1 -- 172.21.15.40:0/3656198401 learned_addr learned my addr 172.21.15.40:0/3656198401 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.990+0000 7fa3f645e640 1 --2- 172.21.15.40:0/3656198401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3f00ca380 0x7fa3f0143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.990+0000 7fa3f6c5f640 1 --2- 172.21.15.40:0/3656198401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f013d820 0x7fa3f0149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.990+0000 7fa3f5c5d640 1 -- 172.21.15.40:0/3656198401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3f00ca380 msgr2=0x7fa3f0143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.990+0000 7fa3f5c5d640 1 --2- 172.21.15.40:0/3656198401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3f00ca380 0x7fa3f0143c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.990+0000 7fa3f5c5d640 1 -- 172.21.15.40:0/3656198401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f013d820 msgr2=0x7fa3f0149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.990+0000 7fa3f5c5d640 1 --2- 172.21.15.40:0/3656198401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f013d820 0x7fa3f0149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.990+0000 7fa3f5c5d640 1 -- 172.21.15.40:0/3656198401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e8009840 con 0x7fa3f0144c10 2023-10-18T09:07:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.990+0000 7fa3f645e640 1 --2- 172.21.15.40:0/3656198401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3f00ca380 0x7fa3f0143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.990+0000 7fa3f6c5f640 1 --2- 172.21.15.40:0/3656198401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f013d820 0x7fa3f0149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.990+0000 7fa3f5c5d640 1 --2- 172.21.15.40:0/3656198401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f0144c10 0x7fa3f0149780 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000d980 tx=0x7fa3e000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.991+0000 7fa3e6ffd640 1 -- 172.21.15.40:0/3656198401 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0009c30 con 0x7fa3f0144c10 2023-10-18T09:07:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.991+0000 7fa3e6ffd640 1 -- 172.21.15.40:0/3656198401 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa3e0009e80 con 0x7fa3f0144c10 2023-10-18T09:07:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.991+0000 7fa3f7460640 1 -- 172.21.15.40:0/3656198401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f00cb7d0 con 0x7fa3f0144c10 2023-10-18T09:07:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.991+0000 7fa3e6ffd640 1 -- 172.21.15.40:0/3656198401 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00106a0 con 0x7fa3f0144c10 2023-10-18T09:07:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.991+0000 7fa3f7460640 1 -- 172.21.15.40:0/3656198401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f00702a0 con 0x7fa3f0144c10 2023-10-18T09:07:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.992+0000 7fa3f7460640 1 -- 172.21.15.40:0/3656198401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f006fdf0 con 0x7fa3f0144c10 2023-10-18T09:07:07.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.993+0000 7fa3e6ffd640 1 -- 172.21.15.40:0/3656198401 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa3e0010800 con 0x7fa3f0144c10 2023-10-18T09:07:07.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.994+0000 7fa3e6ffd640 1 --2- 172.21.15.40:0/3656198401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa3d406ae40 0x7fa3d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.994+0000 7fa3e6ffd640 1 -- 172.21.15.40:0/3656198401 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa3e00897d0 con 0x7fa3f0144c10 2023-10-18T09:07:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.995+0000 7fa3f645e640 1 --2- 172.21.15.40:0/3656198401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa3d406ae40 0x7fa3d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.996+0000 7fa3f645e640 1 --2- 172.21.15.40:0/3656198401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa3d406ae40 0x7fa3d406d300 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0011e0 tx=0x7fa3ec009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:06.998+0000 7fa3e6ffd640 1 -- 172.21.15.40:0/3656198401 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa3e0054b70 con 0x7fa3f0144c10 2023-10-18T09:07:07.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.192+0000 7fa3f7460640 1 -- 172.21.15.40:0/3656198401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3f0066e00 con 0x7fa3f0144c10 2023-10-18T09:07:07.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.193+0000 7fa3e6ffd640 1 -- 172.21.15.40:0/3656198401 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v170) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa3e00200a0 con 0x7fa3f0144c10 2023-10-18T09:07:07.198 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:07:07.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.196+0000 7fa3f7460640 1 -- 172.21.15.40:0/3656198401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa3d406ae40 msgr2=0x7fa3d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:07.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.196+0000 7fa3f7460640 1 --2- 172.21.15.40:0/3656198401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa3d406ae40 0x7fa3d406d300 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0011e0 tx=0x7fa3ec009040 comp rx=0 tx=0).stop 2023-10-18T09:07:07.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.196+0000 7fa3f7460640 1 -- 172.21.15.40:0/3656198401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f0144c10 msgr2=0x7fa3f0149780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:07.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.196+0000 7fa3f7460640 1 --2- 172.21.15.40:0/3656198401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f0144c10 0x7fa3f0149780 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000d980 tx=0x7fa3e000de40 comp rx=0 tx=0).stop 2023-10-18T09:07:07.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.196+0000 7fa3f7460640 1 -- 172.21.15.40:0/3656198401 shutdown_connections 2023-10-18T09:07:07.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.196+0000 7fa3f7460640 1 --2- 172.21.15.40:0/3656198401 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa3d406ae40 0x7fa3d406d300 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:07.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.196+0000 7fa3f7460640 1 --2- 172.21.15.40:0/3656198401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f013d820 0x7fa3f0149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:07.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.196+0000 7fa3f7460640 1 --2- 172.21.15.40:0/3656198401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3f0144c10 0x7fa3f0149780 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:07.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.196+0000 7fa3f7460640 1 --2- 172.21.15.40:0/3656198401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3f00ca380 0x7fa3f0143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:07.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.196+0000 7fa3f7460640 1 -- 172.21.15.40:0/3656198401 >> 172.21.15.40:0/3656198401 conn(0x7fa3f0069720 msgr2=0x7fa3f015abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:07.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.196+0000 7fa3f7460640 1 -- 172.21.15.40:0/3656198401 shutdown_connections 2023-10-18T09:07:07.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.196+0000 7fa3f7460640 1 -- 172.21.15.40:0/3656198401 wait complete. 2023-10-18T09:07:07.217 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2023-10-18T09:07:07.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.439+0000 7fc29bb25640 1 -- 172.21.15.40:0/462786437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940ce430 msgr2=0x7fc2940b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:07.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.439+0000 7fc29bb25640 1 --2- 172.21.15.40:0/462786437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940ce430 0x7fc2940b4e00 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fc29415f3d0 tx=0x7fc28c02bd70 comp rx=0 tx=0).stop 2023-10-18T09:07:07.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.440+0000 7fc29bb25640 1 -- 172.21.15.40:0/462786437 shutdown_connections 2023-10-18T09:07:07.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.440+0000 7fc29bb25640 1 --2- 172.21.15.40:0/462786437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940ce430 0x7fc2940b4e00 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:07.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.440+0000 7fc29bb25640 1 --2- 172.21.15.40:0/462786437 >> v2:172.21.15.144:3300/0 conn(0x7fc2940b7000 0x7fc2940cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:07.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.440+0000 7fc29bb25640 1 -- 172.21.15.40:0/462786437 >> 172.21.15.40:0/462786437 conn(0x7fc29406a780 msgr2=0x7fc29406ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:07.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.440+0000 7fc29bb25640 1 -- 172.21.15.40:0/462786437 shutdown_connections 2023-10-18T09:07:07.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.440+0000 7fc29bb25640 1 -- 172.21.15.40:0/462786437 wait complete. 2023-10-18T09:07:07.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.442+0000 7fc29bb25640 1 Processor -- start 2023-10-18T09:07:07.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.442+0000 7fc29bb25640 1 -- start start 2023-10-18T09:07:07.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.443+0000 7fc29bb25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940b7000 0x7fc2940ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:07.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.443+0000 7fc29bb25640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2940ce430 0x7fc2940b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:07.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.443+0000 7fc29bb25640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc294158a00 0x7fc2941544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:07.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.443+0000 7fc29bb25640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc294070aa0 con 0x7fc2940b7000 2023-10-18T09:07:07.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.443+0000 7fc29bb25640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29406fce0 con 0x7fc294158a00 2023-10-18T09:07:07.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.443+0000 7fc29bb25640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29406ffc0 con 0x7fc2940ce430 2023-10-18T09:07:07.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.444+0000 7fc2998c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940b7000 0x7fc2940ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:07.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.444+0000 7fc2998c1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940b7000 0x7fc2940ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53608/0 (socket says 172.21.15.40:53608) 2023-10-18T09:07:07.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.444+0000 7fc2998c1640 1 -- 172.21.15.40:0/30080362 learned_addr learned my addr 172.21.15.40:0/30080362 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:07.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.444+0000 7fc2990c0640 1 --2- 172.21.15.40:0/30080362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2940ce430 0x7fc2940b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:07.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.444+0000 7fc29a0c2640 1 --2- 172.21.15.40:0/30080362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc294158a00 0x7fc2941544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:07.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.444+0000 7fc2998c1640 1 -- 172.21.15.40:0/30080362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2940ce430 msgr2=0x7fc2940b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:07.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.444+0000 7fc2998c1640 1 --2- 172.21.15.40:0/30080362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2940ce430 0x7fc2940b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:07.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.444+0000 7fc2998c1640 1 -- 172.21.15.40:0/30080362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc294158a00 msgr2=0x7fc2941544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:07.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.444+0000 7fc2998c1640 1 --2- 172.21.15.40:0/30080362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc294158a00 0x7fc2941544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:07.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.444+0000 7fc2998c1640 1 -- 172.21.15.40:0/30080362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c009490 con 0x7fc2940b7000 2023-10-18T09:07:07.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.444+0000 7fc2990c0640 1 --2- 172.21.15.40:0/30080362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2940ce430 0x7fc2940b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.445+0000 7fc2998c1640 1 --2- 172.21.15.40:0/30080362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940b7000 0x7fc2940ca780 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fc29000cbc0 tx=0x7fc29000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.445+0000 7fc28a7fc640 1 -- 172.21.15.40:0/30080362 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29000ad40 con 0x7fc2940b7000 2023-10-18T09:07:07.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.445+0000 7fc28a7fc640 1 -- 172.21.15.40:0/30080362 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc290012420 con 0x7fc2940b7000 2023-10-18T09:07:07.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.445+0000 7fc28a7fc640 1 -- 172.21.15.40:0/30080362 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc290012830 con 0x7fc2940b7000 2023-10-18T09:07:07.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.445+0000 7fc29bb25640 1 -- 172.21.15.40:0/30080362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2940702a0 con 0x7fc2940b7000 2023-10-18T09:07:07.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.445+0000 7fc29bb25640 1 -- 172.21.15.40:0/30080362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2940b0c10 con 0x7fc2940b7000 2023-10-18T09:07:07.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.446+0000 7fc29bb25640 1 -- 172.21.15.40:0/30080362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2940707c0 con 0x7fc2940b7000 2023-10-18T09:07:07.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.447+0000 7fc28a7fc640 1 -- 172.21.15.40:0/30080362 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc29001a3b0 con 0x7fc2940b7000 2023-10-18T09:07:07.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.448+0000 7fc28a7fc640 1 --2- 172.21.15.40:0/30080362 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc27806ae40 0x7fc27806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:07.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.448+0000 7fc28a7fc640 1 -- 172.21.15.40:0/30080362 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc290089ba0 con 0x7fc2940b7000 2023-10-18T09:07:07.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.449+0000 7fc2990c0640 1 --2- 172.21.15.40:0/30080362 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc27806ae40 0x7fc27806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:07.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.450+0000 7fc2990c0640 1 --2- 172.21.15.40:0/30080362 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc27806ae40 0x7fc27806d300 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc284006fd0 tx=0x7fc284008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:07.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.452+0000 7fc28a7fc640 1 -- 172.21.15.40:0/30080362 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2940707c0 con 0x7fc2940b7000 2023-10-18T09:07:07.671 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:07:07.680 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:07:07.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:07.721+0000 7fc29bb25640 1 -- 172.21.15.40:0/30080362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7fc2940707c0 con 0x7fc2940b7000 2023-10-18T09:07:08.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.278+0000 7fc28a7fc640 1 -- 172.21.15.40:0/30080362 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v582) v1 ==== 134+0+0 (secure 0 0 0) 0x7fc2940707c0 con 0x7fc2940b7000 2023-10-18T09:07:08.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.282+0000 7fc29bb25640 1 -- 172.21.15.40:0/30080362 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc27806ae40 msgr2=0x7fc27806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:08.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.282+0000 7fc29bb25640 1 --2- 172.21.15.40:0/30080362 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc27806ae40 0x7fc27806d300 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc284006fd0 tx=0x7fc284008040 comp rx=0 tx=0).stop 2023-10-18T09:07:08.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.282+0000 7fc29bb25640 1 -- 172.21.15.40:0/30080362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940b7000 msgr2=0x7fc2940ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:08.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.282+0000 7fc29bb25640 1 --2- 172.21.15.40:0/30080362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940b7000 0x7fc2940ca780 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fc29000cbc0 tx=0x7fc29000a430 comp rx=0 tx=0).stop 2023-10-18T09:07:08.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.282+0000 7fc29bb25640 1 -- 172.21.15.40:0/30080362 shutdown_connections 2023-10-18T09:07:08.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.282+0000 7fc29bb25640 1 --2- 172.21.15.40:0/30080362 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc27806ae40 0x7fc27806d300 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:08.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.282+0000 7fc29bb25640 1 --2- 172.21.15.40:0/30080362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc294158a00 0x7fc2941544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:08.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.282+0000 7fc29bb25640 1 --2- 172.21.15.40:0/30080362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2940ce430 0x7fc2940b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:08.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.282+0000 7fc29bb25640 1 --2- 172.21.15.40:0/30080362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2940b7000 0x7fc2940ca780 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:08.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.282+0000 7fc29bb25640 1 -- 172.21.15.40:0/30080362 >> 172.21.15.40:0/30080362 conn(0x7fc29406a780 msgr2=0x7fc29406ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:08.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.282+0000 7fc29bb25640 1 -- 172.21.15.40:0/30080362 shutdown_connections 2023-10-18T09:07:08.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.282+0000 7fc29bb25640 1 -- 172.21.15.40:0/30080362 wait complete. 2023-10-18T09:07:08.306 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T09:07:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.532+0000 7f2637e44640 1 -- 172.21.15.40:0/2192130548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2630142cc0 msgr2=0x7f26300cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.532+0000 7f2637e44640 1 --2- 172.21.15.40:0/2192130548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2630142cc0 0x7f26300cc710 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f26300b2510 tx=0x7f262c02bdb0 comp rx=0 tx=0).stop 2023-10-18T09:07:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.532+0000 7f2637e44640 1 -- 172.21.15.40:0/2192130548 shutdown_connections 2023-10-18T09:07:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.532+0000 7f2637e44640 1 --2- 172.21.15.40:0/2192130548 >> v2:172.21.15.144:3300/0 conn(0x7f263015cf80 0x7f26300aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:08.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.532+0000 7f2637e44640 1 --2- 172.21.15.40:0/2192130548 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2630142cc0 0x7f26300cc710 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:08.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.532+0000 7f2637e44640 1 -- 172.21.15.40:0/2192130548 >> 172.21.15.40:0/2192130548 conn(0x7f2630069720 msgr2=0x7f2630069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:08.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.533+0000 7f2637e44640 1 -- 172.21.15.40:0/2192130548 shutdown_connections 2023-10-18T09:07:08.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.533+0000 7f2637e44640 1 -- 172.21.15.40:0/2192130548 wait complete. 2023-10-18T09:07:08.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.534+0000 7f2637e44640 1 Processor -- start 2023-10-18T09:07:08.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.534+0000 7f2637e44640 1 -- start start 2023-10-18T09:07:08.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.535+0000 7f2637e44640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2630142cc0 0x7f26300c2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:08.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.535+0000 7f2637e44640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f263015cf80 0x7f26300ab550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:08.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.535+0000 7f2637e44640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f263015d410 0x7f26300af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:08.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.535+0000 7f2637e44640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26300aba90 con 0x7f2630142cc0 2023-10-18T09:07:08.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.535+0000 7f2637e44640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26300c26e0 con 0x7f263015d410 2023-10-18T09:07:08.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.535+0000 7f2637e44640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26300c29c0 con 0x7f263015cf80 2023-10-18T09:07:08.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.536+0000 7f2635be0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2630142cc0 0x7f26300c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:08.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.536+0000 7f2635be0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2630142cc0 0x7f26300c2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53628/0 (socket says 172.21.15.40:53628) 2023-10-18T09:07:08.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.536+0000 7f2635be0640 1 -- 172.21.15.40:0/3085161435 learned_addr learned my addr 172.21.15.40:0/3085161435 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:08.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.536+0000 7f26363e1640 1 --2- 172.21.15.40:0/3085161435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f263015d410 0x7f26300af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:08.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.536+0000 7f26353df640 1 --2- 172.21.15.40:0/3085161435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f263015cf80 0x7f26300ab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:08.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.536+0000 7f2635be0640 1 -- 172.21.15.40:0/3085161435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f263015cf80 msgr2=0x7f26300ab550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:08.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.536+0000 7f2635be0640 1 --2- 172.21.15.40:0/3085161435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f263015cf80 0x7f26300ab550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:08.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.536+0000 7f2635be0640 1 -- 172.21.15.40:0/3085161435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f263015d410 msgr2=0x7f26300af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:08.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.536+0000 7f2635be0640 1 --2- 172.21.15.40:0/3085161435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f263015d410 0x7f26300af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:08.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.536+0000 7f2635be0640 1 -- 172.21.15.40:0/3085161435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c009560 con 0x7f2630142cc0 2023-10-18T09:07:08.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.536+0000 7f26353df640 1 --2- 172.21.15.40:0/3085161435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f263015cf80 0x7f26300ab550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:08.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.537+0000 7f2635be0640 1 --2- 172.21.15.40:0/3085161435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2630142cc0 0x7f26300c2050 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f262c009850 tx=0x7f262c02dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.537+0000 7f26267fc640 1 -- 172.21.15.40:0/3085161435 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c005650 con 0x7f2630142cc0 2023-10-18T09:07:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.537+0000 7f26267fc640 1 -- 172.21.15.40:0/3085161435 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f262c0038f0 con 0x7f2630142cc0 2023-10-18T09:07:08.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.537+0000 7f26267fc640 1 -- 172.21.15.40:0/3085161435 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c003ce0 con 0x7f2630142cc0 2023-10-18T09:07:08.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.537+0000 7f2637e44640 1 -- 172.21.15.40:0/3085161435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26300c2ca0 con 0x7f2630142cc0 2023-10-18T09:07:08.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.537+0000 7f2637e44640 1 -- 172.21.15.40:0/3085161435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26300c3260 con 0x7f2630142cc0 2023-10-18T09:07:08.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.538+0000 7f2637e44640 1 -- 172.21.15.40:0/3085161435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2630066fd0 con 0x7f2630142cc0 2023-10-18T09:07:08.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.539+0000 7f26267fc640 1 -- 172.21.15.40:0/3085161435 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f262c03e450 con 0x7f2630142cc0 2023-10-18T09:07:08.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.540+0000 7f26267fc640 1 --2- 172.21.15.40:0/3085161435 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f261406ae40 0x7f261406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:08.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.540+0000 7f26267fc640 1 -- 172.21.15.40:0/3085161435 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f262c0abca0 con 0x7f2630142cc0 2023-10-18T09:07:08.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.541+0000 7f26353df640 1 --2- 172.21.15.40:0/3085161435 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f261406ae40 0x7f261406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:08.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.542+0000 7f26353df640 1 --2- 172.21.15.40:0/3085161435 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f261406ae40 0x7f261406d300 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f26300abc10 tx=0x7f2620005d30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:08.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.545+0000 7f26267fc640 1 -- 172.21.15.40:0/3085161435 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f262c077040 con 0x7f2630142cc0 2023-10-18T09:07:08.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.737+0000 7f2637e44640 1 -- 172.21.15.40:0/3085161435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f26300c3440 con 0x7f2630142cc0 2023-10-18T09:07:08.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.738+0000 7f26267fc640 1 -- 172.21.15.40:0/3085161435 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v38) v1 ==== 59+0+1452 (secure 0 0 0) 0x7f262c0425c0 con 0x7f2630142cc0 2023-10-18T09:07:08.741 INFO:teuthology.orchestra.run.smithi040.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T09:07:08.741 INFO:teuthology.orchestra.run.smithi040.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T09:07:08.741 INFO:teuthology.orchestra.run.smithi040.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T09:07:08.741 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds 20/20 2023-10-18T09:07:08.741 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T09:07:08.742 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_ms 1/1 2023-10-18T09:07:08.742 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T09:07:08.742 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T09:07:08.742 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T09:07:08.742 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_size 100 2023-10-18T09:07:08.743 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_frag true 2023-10-18T09:07:08.743 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_scatterstat true 2023-10-18T09:07:08.743 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T09:07:08.743 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_verify_scatter true 2023-10-18T09:07:08.743 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T09:07:08.744 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T09:07:08.744 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T09:07:08.744 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced client_mount_timeout 600 2023-10-18T09:07:08.744 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_client 20/20 2023-10-18T09:07:08.744 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_ms 1/1 2023-10-18T09:07:08.744 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T09:07:08.745 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T09:07:08.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.741+0000 7f2637e44640 1 -- 172.21.15.40:0/3085161435 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f261406ae40 msgr2=0x7f261406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:08.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.741+0000 7f2637e44640 1 --2- 172.21.15.40:0/3085161435 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f261406ae40 0x7f261406d300 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f26300abc10 tx=0x7f2620005d30 comp rx=0 tx=0).stop 2023-10-18T09:07:08.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.741+0000 7f2637e44640 1 -- 172.21.15.40:0/3085161435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2630142cc0 msgr2=0x7f26300c2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:08.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.741+0000 7f2637e44640 1 --2- 172.21.15.40:0/3085161435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2630142cc0 0x7f26300c2050 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f262c009850 tx=0x7f262c02dd80 comp rx=0 tx=0).stop 2023-10-18T09:07:08.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.741+0000 7f2637e44640 1 -- 172.21.15.40:0/3085161435 shutdown_connections 2023-10-18T09:07:08.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.741+0000 7f2637e44640 1 --2- 172.21.15.40:0/3085161435 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f261406ae40 0x7f261406d300 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:08.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.741+0000 7f2637e44640 1 --2- 172.21.15.40:0/3085161435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f263015d410 0x7f26300af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:08.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.741+0000 7f2637e44640 1 --2- 172.21.15.40:0/3085161435 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f263015cf80 0x7f26300ab550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:08.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.741+0000 7f2637e44640 1 --2- 172.21.15.40:0/3085161435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2630142cc0 0x7f26300c2050 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:08.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.741+0000 7f2637e44640 1 -- 172.21.15.40:0/3085161435 >> 172.21.15.40:0/3085161435 conn(0x7f2630069720 msgr2=0x7f2630141330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:08.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.741+0000 7f2637e44640 1 -- 172.21.15.40:0/3085161435 shutdown_connections 2023-10-18T09:07:08.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.741+0000 7f2637e44640 1 -- 172.21.15.40:0/3085161435 wait complete. 2023-10-18T09:07:08.761 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T09:07:08.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.994+0000 7fd918c26640 1 -- 172.21.15.40:0/3078154842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9140b0a80 msgr2=0x7fd914159f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:08.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.994+0000 7fd918c26640 1 --2- 172.21.15.40:0/3078154842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9140b0a80 0x7fd914159f00 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd904005fd0 tx=0x7fd90402be20 comp rx=0 tx=0).stop 2023-10-18T09:07:08.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.994+0000 7fd918c26640 1 -- 172.21.15.40:0/3078154842 shutdown_connections 2023-10-18T09:07:08.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.994+0000 7fd918c26640 1 --2- 172.21.15.40:0/3078154842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9140b0190 0x7fd91406ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:08.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.994+0000 7fd918c26640 1 --2- 172.21.15.40:0/3078154842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9140b0a80 0x7fd914159f00 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:08.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.994+0000 7fd918c26640 1 -- 172.21.15.40:0/3078154842 >> 172.21.15.40:0/3078154842 conn(0x7fd914069720 msgr2=0x7fd914069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:08.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.994+0000 7fd918c26640 1 -- 172.21.15.40:0/3078154842 shutdown_connections 2023-10-18T09:07:08.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.995+0000 7fd918c26640 1 -- 172.21.15.40:0/3078154842 wait complete. 2023-10-18T09:07:08.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.996+0000 7fd918c26640 1 Processor -- start 2023-10-18T09:07:08.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.996+0000 7fd918c26640 1 -- start start 2023-10-18T09:07:09.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.997+0000 7fd918c26640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9140b0190 0x7fd91406b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:09.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.997+0000 7fd918c26640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9140b0a80 0x7fd9141f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:09.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.997+0000 7fd918c26640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9141589c0 0x7fd9141e0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:09.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.997+0000 7fd918c26640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141e6c30 con 0x7fd9141589c0 2023-10-18T09:07:09.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.997+0000 7fd918c26640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141e4620 con 0x7fd9140b0a80 2023-10-18T09:07:09.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.997+0000 7fd918c26640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141e48c0 con 0x7fd9140b0190 2023-10-18T09:07:09.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.998+0000 7fd912d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9141589c0 0x7fd9141e0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:09.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.998+0000 7fd912d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9141589c0 0x7fd9141e0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53658/0 (socket says 172.21.15.40:53658) 2023-10-18T09:07:09.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.998+0000 7fd912d9d640 1 -- 172.21.15.40:0/286024009 learned_addr learned my addr 172.21.15.40:0/286024009 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:09.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.998+0000 7fd912d9d640 1 -- 172.21.15.40:0/286024009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9140b0190 msgr2=0x7fd91406b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:09.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.998+0000 7fd91259c640 1 --2- 172.21.15.40:0/286024009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9140b0190 0x7fd91406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:09.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.998+0000 7fd912d9d640 1 --2- 172.21.15.40:0/286024009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9140b0190 0x7fd91406b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:09.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.998+0000 7fd90bfff640 1 --2- 172.21.15.40:0/286024009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9140b0a80 0x7fd9141f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:09.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.998+0000 7fd912d9d640 1 -- 172.21.15.40:0/286024009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9140b0a80 msgr2=0x7fd9141f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:09.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.998+0000 7fd912d9d640 1 --2- 172.21.15.40:0/286024009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9140b0a80 0x7fd9141f1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:09.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.998+0000 7fd912d9d640 1 -- 172.21.15.40:0/286024009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd904009560 con 0x7fd9141589c0 2023-10-18T09:07:09.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.998+0000 7fd91259c640 1 --2- 172.21.15.40:0/286024009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9140b0190 0x7fd91406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:09.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.999+0000 7fd912d9d640 1 --2- 172.21.15.40:0/286024009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9141589c0 0x7fd9141e0ca0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00cce0 tx=0x7fd90c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:09.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.999+0000 7fd90affd640 1 -- 172.21.15.40:0/286024009 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00f450 con 0x7fd9141589c0 2023-10-18T09:07:09.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.999+0000 7fd90affd640 1 -- 172.21.15.40:0/286024009 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd90c00f6c0 con 0x7fd9141589c0 2023-10-18T09:07:09.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.999+0000 7fd90affd640 1 -- 172.21.15.40:0/286024009 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00fad0 con 0x7fd9141589c0 2023-10-18T09:07:09.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.999+0000 7fd918c26640 1 -- 172.21.15.40:0/286024009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9141e4ba0 con 0x7fd9141589c0 2023-10-18T09:07:09.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:08.999+0000 7fd918c26640 1 -- 172.21.15.40:0/286024009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9141e1860 con 0x7fd9141589c0 2023-10-18T09:07:09.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.000+0000 7fd918c26640 1 -- 172.21.15.40:0/286024009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9141e50c0 con 0x7fd9141589c0 2023-10-18T09:07:09.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.001+0000 7fd90affd640 1 -- 172.21.15.40:0/286024009 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd90c00fc30 con 0x7fd9141589c0 2023-10-18T09:07:09.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.002+0000 7fd90affd640 1 --2- 172.21.15.40:0/286024009 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd8f406ae70 0x7fd8f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:09.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.003+0000 7fd90affd640 1 -- 172.21.15.40:0/286024009 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd90c089e80 con 0x7fd9141589c0 2023-10-18T09:07:09.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.003+0000 7fd91259c640 1 --2- 172.21.15.40:0/286024009 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd8f406ae70 0x7fd8f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:09.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.004+0000 7fd91259c640 1 --2- 172.21.15.40:0/286024009 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd8f406ae70 0x7fd8f406d330 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd904035000 tx=0x7fd904004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:09.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.007+0000 7fd90affd640 1 -- 172.21.15.40:0/286024009 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9141e50c0 con 0x7fd9141589c0 2023-10-18T09:07:09.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.201+0000 7fd918c26640 1 -- 172.21.15.40:0/286024009 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd9141e50c0 con 0x7fd9141589c0 2023-10-18T09:07:09.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.207+0000 7fd90affd640 1 -- 172.21.15.40:0/286024009 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v39)=0 v39) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd9141e50c0 con 0x7fd9141589c0 2023-10-18T09:07:09.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.210+0000 7fd918c26640 1 -- 172.21.15.40:0/286024009 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd8f406ae70 msgr2=0x7fd8f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:09.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.210+0000 7fd918c26640 1 --2- 172.21.15.40:0/286024009 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd8f406ae70 0x7fd8f406d330 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd904035000 tx=0x7fd904004730 comp rx=0 tx=0).stop 2023-10-18T09:07:09.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.210+0000 7fd918c26640 1 -- 172.21.15.40:0/286024009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9141589c0 msgr2=0x7fd9141e0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:09.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.210+0000 7fd918c26640 1 --2- 172.21.15.40:0/286024009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9141589c0 0x7fd9141e0ca0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00cce0 tx=0x7fd90c00a620 comp rx=0 tx=0).stop 2023-10-18T09:07:09.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.210+0000 7fd918c26640 1 -- 172.21.15.40:0/286024009 shutdown_connections 2023-10-18T09:07:09.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.210+0000 7fd918c26640 1 --2- 172.21.15.40:0/286024009 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd8f406ae70 0x7fd8f406d330 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:09.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.210+0000 7fd918c26640 1 --2- 172.21.15.40:0/286024009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9141589c0 0x7fd9141e0ca0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:09.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.210+0000 7fd918c26640 1 --2- 172.21.15.40:0/286024009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9140b0a80 0x7fd9141f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:09.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.210+0000 7fd918c26640 1 --2- 172.21.15.40:0/286024009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9140b0190 0x7fd91406b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:09.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.210+0000 7fd918c26640 1 -- 172.21.15.40:0/286024009 >> 172.21.15.40:0/286024009 conn(0x7fd914069720 msgr2=0x7fd91415aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:09.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.210+0000 7fd918c26640 1 -- 172.21.15.40:0/286024009 shutdown_connections 2023-10-18T09:07:09.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.210+0000 7fd918c26640 1 -- 172.21.15.40:0/286024009 wait complete. 2023-10-18T09:07:09.233 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T09:07:09.234 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:07:09.357 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:07:09.479 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T09:07:09.480 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:07:09.480 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:07:09.544 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:07:09.545 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:07:09.545 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:07:09.605 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:07:09.605 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:07:09.605 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:07:09.666 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:07:09.667 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:07:09.667 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:07:09.726 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:07:09.726 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T09:07:09.727 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:07:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.965+0000 7f6b65fe7640 1 -- 172.21.15.40:0/1439270516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b4c003970 msgr2=0x7f6b4c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.965+0000 7f6b65fe7640 1 --2- 172.21.15.40:0/1439270516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b4c003970 0x7f6b4c003d90 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009b50 tx=0x7f6b4c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:07:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.965+0000 7f6b5ffff640 1 -- 172.21.15.40:0/1439270516 reap_dead start 2023-10-18T09:07:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.965+0000 7f6b65fe7640 1 -- 172.21.15.40:0/1439270516 shutdown_connections 2023-10-18T09:07:09.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.965+0000 7f6b65fe7640 1 -- 172.21.15.40:0/1439270516 >> 172.21.15.40:0/1439270516 conn(0x7f6b6006cee0 msgr2=0x7f6b60069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:09.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.965+0000 7f6b65fe7640 1 -- 172.21.15.40:0/1439270516 shutdown_connections 2023-10-18T09:07:09.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.965+0000 7f6b65fe7640 1 -- 172.21.15.40:0/1439270516 wait complete. 2023-10-18T09:07:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.967+0000 7f6b65fe7640 1 Processor -- start 2023-10-18T09:07:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.967+0000 7f6b65fe7640 1 -- start start 2023-10-18T09:07:09.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.968+0000 7f6b65fe7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c003970 0x7f6b601485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:09.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.968+0000 7f6b65fe7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6014bc90 0x7f6b60150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:09.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.968+0000 7f6b65fe7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b6014c0f0 0x7f6b6014ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:09.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.968+0000 7f6b65fe7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6014b800 con 0x7f6b6014c0f0 2023-10-18T09:07:09.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.968+0000 7f6b65fe7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6014abf0 con 0x7f6b4c003970 2023-10-18T09:07:09.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.968+0000 7f6b65fe7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6014aed0 con 0x7f6b6014bc90 2023-10-18T09:07:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.969+0000 7f6b5ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b6014c0f0 0x7f6b6014ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.969+0000 7f6b5ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b6014c0f0 0x7f6b6014ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53686/0 (socket says 172.21.15.40:53686) 2023-10-18T09:07:09.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.969+0000 7f6b5ffff640 1 -- 172.21.15.40:0/932780558 learned_addr learned my addr 172.21.15.40:0/932780558 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:09.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.969+0000 7f6b56ffd640 1 --2- 172.21.15.40:0/932780558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6014bc90 0x7f6b60150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:09.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.969+0000 7f6b5f7fe640 1 --2- 172.21.15.40:0/932780558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c003970 0x7f6b601485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:09.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.969+0000 7f6b5ffff640 1 -- 172.21.15.40:0/932780558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6014bc90 msgr2=0x7f6b60150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:09.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.969+0000 7f6b5ffff640 1 --2- 172.21.15.40:0/932780558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6014bc90 0x7f6b60150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:09.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.969+0000 7f6b5ffff640 1 -- 172.21.15.40:0/932780558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c003970 msgr2=0x7f6b601485b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:09.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.969+0000 7f6b5ffff640 1 --2- 172.21.15.40:0/932780558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c003970 0x7f6b601485b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:09.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.969+0000 7f6b5ffff640 1 -- 172.21.15.40:0/932780558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6014b1b0 con 0x7f6b6014c0f0 2023-10-18T09:07:09.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.970+0000 7f6b5ffff640 1 --2- 172.21.15.40:0/932780558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b6014c0f0 0x7f6b6014ddb0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f6b6006a330 tx=0x7f6b5800da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:09.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.970+0000 7f6b5cff9640 1 -- 172.21.15.40:0/932780558 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58006440 con 0x7f6b6014c0f0 2023-10-18T09:07:09.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.970+0000 7f6b5cff9640 1 -- 172.21.15.40:0/932780558 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6b580036e0 con 0x7f6b6014c0f0 2023-10-18T09:07:09.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.970+0000 7f6b65fe7640 1 -- 172.21.15.40:0/932780558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6014b490 con 0x7f6b6014c0f0 2023-10-18T09:07:09.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.970+0000 7f6b5cff9640 1 -- 172.21.15.40:0/932780558 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58003ad0 con 0x7f6b6014c0f0 2023-10-18T09:07:09.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.970+0000 7f6b65fe7640 1 -- 172.21.15.40:0/932780558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6014a830 con 0x7f6b6014c0f0 2023-10-18T09:07:09.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.972+0000 7f6b65fe7640 1 -- 172.21.15.40:0/932780558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6014b620 con 0x7f6b6014c0f0 2023-10-18T09:07:09.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.973+0000 7f6b5cff9640 1 -- 172.21.15.40:0/932780558 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6b58003c30 con 0x7f6b6014c0f0 2023-10-18T09:07:09.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.974+0000 7f6b5cff9640 1 --2- 172.21.15.40:0/932780558 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b4006ae40 0x7f6b4006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:09.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.974+0000 7f6b5cff9640 1 -- 172.21.15.40:0/932780558 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6b58089500 con 0x7f6b6014c0f0 2023-10-18T09:07:09.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.974+0000 7f6b5f7fe640 1 --2- 172.21.15.40:0/932780558 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b4006ae40 0x7f6b4006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:09.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.975+0000 7f6b5f7fe640 1 --2- 172.21.15.40:0/932780558 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b4006ae40 0x7f6b4006d300 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009b50 tx=0x7f6b50035040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:09.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:09.978+0000 7f6b5cff9640 1 -- 172.21.15.40:0/932780558 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b580548b0 con 0x7f6b6014c0f0 2023-10-18T09:07:10.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.213+0000 7f6b65fe7640 1 -- 172.21.15.40:0/932780558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b60066fd0 con 0x7f6b6014c0f0 2023-10-18T09:07:10.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.214+0000 7f6b5cff9640 1 -- 172.21.15.40:0/932780558 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f6b58057ed0 con 0x7f6b6014c0f0 2023-10-18T09:07:10.217 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:07:10.217 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3243865919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":3243865919},{"type":"v1","addr":"172.21.15.40:6835","nonce":3243865919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6047,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/504515257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":504515257},{"type":"v1","addr":"172.21.15.144:6835","nonce":504515257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2513603652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6836","nonce":2513603652},{"type":"v1","addr":"172.21.15.144:6837","nonce":2513603652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/2286294608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":2286294608},{"type":"v1","addr":"172.21.15.40:6833","nonce":2286294608}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[]} 2023-10-18T09:07:10.217 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 76 2023-10-18T09:07:10.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.217+0000 7f6b65fe7640 1 -- 172.21.15.40:0/932780558 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b4006ae40 msgr2=0x7f6b4006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:10.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.217+0000 7f6b65fe7640 1 --2- 172.21.15.40:0/932780558 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b4006ae40 0x7f6b4006d300 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009b50 tx=0x7f6b50035040 comp rx=0 tx=0).stop 2023-10-18T09:07:10.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.217+0000 7f6b65fe7640 1 -- 172.21.15.40:0/932780558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b6014c0f0 msgr2=0x7f6b6014ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:10.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.217+0000 7f6b65fe7640 1 --2- 172.21.15.40:0/932780558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b6014c0f0 0x7f6b6014ddb0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f6b6006a330 tx=0x7f6b5800da10 comp rx=0 tx=0).stop 2023-10-18T09:07:10.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.217+0000 7f6b65fe7640 1 -- 172.21.15.40:0/932780558 shutdown_connections 2023-10-18T09:07:10.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.217+0000 7f6b65fe7640 1 --2- 172.21.15.40:0/932780558 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6b4006ae40 0x7f6b4006d300 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:10.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.217+0000 7f6b65fe7640 1 --2- 172.21.15.40:0/932780558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b6014c0f0 0x7f6b6014ddb0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:10.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.217+0000 7f6b65fe7640 1 --2- 172.21.15.40:0/932780558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b6014bc90 0x7f6b60150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:10.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.217+0000 7f6b65fe7640 1 --2- 172.21.15.40:0/932780558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c003970 0x7f6b601485b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:10.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.217+0000 7f6b65fe7640 1 -- 172.21.15.40:0/932780558 >> 172.21.15.40:0/932780558 conn(0x7f6b6006cee0 msgr2=0x7f6b600713f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:10.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.217+0000 7f6b65fe7640 1 -- 172.21.15.40:0/932780558 shutdown_connections 2023-10-18T09:07:10.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.217+0000 7f6b65fe7640 1 -- 172.21.15.40:0/932780558 wait complete. 2023-10-18T09:07:10.240 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T09:07:10.240 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T09:07:10.240 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T09:07:10.241 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:07:10.241 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T09:07:10.241 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T09:07:10.242 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T09:07:10.242 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:07:10.242 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T09:07:10.242 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T09:07:10.243 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T09:07:10.243 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:07:10.243 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T09:07:10.243 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T09:07:10.244 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T09:07:10.244 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:07:10.258 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T09:07:10.258 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T09:07:10.259 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T09:07:10.260 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T09:07:10.263 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T09:07:10.264 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T09:07:10.270 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T09:07:10.270 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T09:07:10.308 INFO:tasks.ceph.mds.a:Started 2023-10-18T09:07:10.309 DEBUG:teuthology.parallel:result is None 2023-10-18T09:07:10.316 INFO:tasks.ceph.mds.c:Started 2023-10-18T09:07:10.316 DEBUG:teuthology.parallel:result is None 2023-10-18T09:07:10.317 INFO:tasks.ceph.mds.b:Started 2023-10-18T09:07:10.318 DEBUG:teuthology.parallel:result is None 2023-10-18T09:07:10.323 INFO:tasks.ceph.mds.d:Started 2023-10-18T09:07:10.323 DEBUG:teuthology.parallel:result is None 2023-10-18T09:07:10.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T09:07:10.473 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T09:07:10.474 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-10-18T09:07:10.490 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-18T09:07:10.490 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-18T09:07:10.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.520+0000 7fd88e27c640 1 -- 172.21.15.40:0/3802864256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8880ca210 msgr2=0x7fd88815d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:10.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.520+0000 7fd88e27c640 1 --2- 172.21.15.40:0/3802864256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8880ca210 0x7fd88815d390 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fd880005fd0 tx=0x7fd88002c2e0 comp rx=0 tx=0).stop 2023-10-18T09:07:10.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.520+0000 7fd88e27c640 1 -- 172.21.15.40:0/3802864256 shutdown_connections 2023-10-18T09:07:10.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.520+0000 7fd88e27c640 1 --2- 172.21.15.40:0/3802864256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8880ca210 0x7fd88815d390 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:10.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.520+0000 7fd88e27c640 1 --2- 172.21.15.40:0/3802864256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd88813c470 0x7fd88813f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:10.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.520+0000 7fd88e27c640 1 -- 172.21.15.40:0/3802864256 >> 172.21.15.40:0/3802864256 conn(0x7fd88806cee0 msgr2=0x7fd888069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:10.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.520+0000 7fd88e27c640 1 -- 172.21.15.40:0/3802864256 shutdown_connections 2023-10-18T09:07:10.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.520+0000 7fd88e27c640 1 -- 172.21.15.40:0/3802864256 wait complete. 2023-10-18T09:07:10.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.522+0000 7fd88e27c640 1 Processor -- start 2023-10-18T09:07:10.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.522+0000 7fd88e27c640 1 -- start start 2023-10-18T09:07:10.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.523+0000 7fd88e27c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8880ca210 0x7fd88813a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:10.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.523+0000 7fd88e27c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88813c470 0x7fd8880ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:10.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.523+0000 7fd88e27c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd88815cf30 0x7fd888154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:10.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.523+0000 7fd88e27c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88813ae70 con 0x7fd88815cf30 2023-10-18T09:07:10.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.523+0000 7fd88e27c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8880ae520 con 0x7fd88813c470 2023-10-18T09:07:10.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.523+0000 7fd88e27c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8881540a0 con 0x7fd8880ca210 2023-10-18T09:07:10.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.523+0000 7fd88c819640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd88815cf30 0x7fd888154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:10.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.523+0000 7fd88c819640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd88815cf30 0x7fd888154b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53726/0 (socket says 172.21.15.40:53726) 2023-10-18T09:07:10.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.523+0000 7fd88c819640 1 -- 172.21.15.40:0/182053983 learned_addr learned my addr 172.21.15.40:0/182053983 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:10.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.524+0000 7fd887fff640 1 --2- 172.21.15.40:0/182053983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8880ca210 0x7fd88813a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:10.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.524+0000 7fd8877fe640 1 --2- 172.21.15.40:0/182053983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88813c470 0x7fd8880ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:10.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.524+0000 7fd88c819640 1 -- 172.21.15.40:0/182053983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8880ca210 msgr2=0x7fd88813a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:10.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.524+0000 7fd88c819640 1 --2- 172.21.15.40:0/182053983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8880ca210 0x7fd88813a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:10.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.524+0000 7fd88c819640 1 -- 172.21.15.40:0/182053983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88813c470 msgr2=0x7fd8880ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:10.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.524+0000 7fd88c819640 1 --2- 172.21.15.40:0/182053983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88813c470 0x7fd8880ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:10.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.524+0000 7fd88c819640 1 -- 172.21.15.40:0/182053983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd880009900 con 0x7fd88815cf30 2023-10-18T09:07:10.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.524+0000 7fd88c819640 1 --2- 172.21.15.40:0/182053983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd88815cf30 0x7fd888154b40 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fd880037040 tx=0x7fd880035ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:10.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.524+0000 7fd884ff9640 1 -- 172.21.15.40:0/182053983 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880004240 con 0x7fd88815cf30 2023-10-18T09:07:10.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.525+0000 7fd884ff9640 1 -- 172.21.15.40:0/182053983 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd880004490 con 0x7fd88815cf30 2023-10-18T09:07:10.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.525+0000 7fd884ff9640 1 -- 172.21.15.40:0/182053983 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880004820 con 0x7fd88815cf30 2023-10-18T09:07:10.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.525+0000 7fd88e27c640 1 -- 172.21.15.40:0/182053983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8880ae240 con 0x7fd88815cf30 2023-10-18T09:07:10.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.525+0000 7fd88e27c640 1 -- 172.21.15.40:0/182053983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd888154620 con 0x7fd88815cf30 2023-10-18T09:07:10.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.526+0000 7fd88e27c640 1 -- 172.21.15.40:0/182053983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd888154810 con 0x7fd88815cf30 2023-10-18T09:07:10.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.527+0000 7fd884ff9640 1 -- 172.21.15.40:0/182053983 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd88003b040 con 0x7fd88815cf30 2023-10-18T09:07:10.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.528+0000 7fd884ff9640 1 --2- 172.21.15.40:0/182053983 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd87406ae40 0x7fd87406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:10.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.528+0000 7fd884ff9640 1 -- 172.21.15.40:0/182053983 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd8800aba60 con 0x7fd88815cf30 2023-10-18T09:07:10.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.528+0000 7fd887fff640 1 --2- 172.21.15.40:0/182053983 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd87406ae40 0x7fd87406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:10.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.529+0000 7fd887fff640 1 --2- 172.21.15.40:0/182053983 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd87406ae40 0x7fd87406d300 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd8880ae8c0 tx=0x7fd878005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:10.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.532+0000 7fd884ff9640 1 -- 172.21.15.40:0/182053983 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd880076eb0 con 0x7fd88815cf30 2023-10-18T09:07:10.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:10.739+0000 7fd88e27c640 1 -- 172.21.15.40:0/182053983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd888066fb0 con 0x7fd88815cf30 2023-10-18T09:07:12.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.200+0000 7fd884ff9640 1 -- 172.21.15.40:0/182053983 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v175) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd88007a4d0 con 0x7fd88815cf30 2023-10-18T09:07:12.203 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2023-10-18T09:07:12.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.204+0000 7fd88e27c640 1 -- 172.21.15.40:0/182053983 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd87406ae40 msgr2=0x7fd87406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.204+0000 7fd88e27c640 1 --2- 172.21.15.40:0/182053983 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd87406ae40 0x7fd87406d300 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd8880ae8c0 tx=0x7fd878005f90 comp rx=0 tx=0).stop 2023-10-18T09:07:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.204+0000 7fd88e27c640 1 -- 172.21.15.40:0/182053983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd88815cf30 msgr2=0x7fd888154b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.204+0000 7fd88e27c640 1 --2- 172.21.15.40:0/182053983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd88815cf30 0x7fd888154b40 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fd880037040 tx=0x7fd880035ad0 comp rx=0 tx=0).stop 2023-10-18T09:07:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.204+0000 7fd88e27c640 1 -- 172.21.15.40:0/182053983 shutdown_connections 2023-10-18T09:07:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.204+0000 7fd88e27c640 1 --2- 172.21.15.40:0/182053983 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd87406ae40 0x7fd87406d300 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:12.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.204+0000 7fd88e27c640 1 --2- 172.21.15.40:0/182053983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd88815cf30 0x7fd888154b40 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:12.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.204+0000 7fd88e27c640 1 --2- 172.21.15.40:0/182053983 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd88813c470 0x7fd8880ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:12.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.204+0000 7fd88e27c640 1 --2- 172.21.15.40:0/182053983 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8880ca210 0x7fd88813a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:12.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.204+0000 7fd88e27c640 1 -- 172.21.15.40:0/182053983 >> 172.21.15.40:0/182053983 conn(0x7fd88806cee0 msgr2=0x7fd888153360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:12.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.204+0000 7fd88e27c640 1 -- 172.21.15.40:0/182053983 shutdown_connections 2023-10-18T09:07:12.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.204+0000 7fd88e27c640 1 -- 172.21.15.40:0/182053983 wait complete. 2023-10-18T09:07:12.227 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T09:07:12.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.462+0000 7f4419573640 1 -- 172.21.15.40:0/1793099062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44140d0100 msgr2=0x7f44140d1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:12.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.462+0000 7f4419573640 1 --2- 172.21.15.40:0/1793099062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44140d0100 0x7f44140d1480 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009900 tx=0x7f43fc02bea0 comp rx=0 tx=0).stop 2023-10-18T09:07:12.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.462+0000 7f4419573640 1 -- 172.21.15.40:0/1793099062 shutdown_connections 2023-10-18T09:07:12.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.462+0000 7f4419573640 1 --2- 172.21.15.40:0/1793099062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44140d0100 0x7f44140d1480 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:12.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.462+0000 7f4419573640 1 --2- 172.21.15.40:0/1793099062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f441415b760 0x7f44140cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:12.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.462+0000 7f4419573640 1 -- 172.21.15.40:0/1793099062 >> 172.21.15.40:0/1793099062 conn(0x7f441409a9a0 msgr2=0x7f4414088fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:12.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.462+0000 7f4419573640 1 -- 172.21.15.40:0/1793099062 shutdown_connections 2023-10-18T09:07:12.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.462+0000 7f4419573640 1 -- 172.21.15.40:0/1793099062 wait complete. 2023-10-18T09:07:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.464+0000 7f4419573640 1 Processor -- start 2023-10-18T09:07:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.464+0000 7f4419573640 1 -- start start 2023-10-18T09:07:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.465+0000 7f4419573640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44140d0100 0x7f44140c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.465+0000 7f4419573640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44140d19c0 0x7f4414148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.465+0000 7f4419573640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f441415b760 0x7f44140c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.465+0000 7f4419573640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44140c8000 con 0x7f44140d0100 2023-10-18T09:07:12.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.465+0000 7f4419573640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44141e35b0 con 0x7f441415b760 2023-10-18T09:07:12.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.465+0000 7f4419573640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44141e3890 con 0x7f44140d19c0 2023-10-18T09:07:12.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.466+0000 7f4412ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44140d0100 0x7f44140c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:12.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.466+0000 7f4412ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44140d0100 0x7f44140c7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53728/0 (socket says 172.21.15.40:53728) 2023-10-18T09:07:12.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.466+0000 7f4412ffd640 1 -- 172.21.15.40:0/292148043 learned_addr learned my addr 172.21.15.40:0/292148043 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:12.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.466+0000 7f44127fc640 1 --2- 172.21.15.40:0/292148043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44140d19c0 0x7f4414148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:12.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.466+0000 7f44137fe640 1 --2- 172.21.15.40:0/292148043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f441415b760 0x7f44140c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:12.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.466+0000 7f4412ffd640 1 -- 172.21.15.40:0/292148043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44140d19c0 msgr2=0x7f4414148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:12.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.466+0000 7f4412ffd640 1 --2- 172.21.15.40:0/292148043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44140d19c0 0x7f4414148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:12.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.466+0000 7f4412ffd640 1 -- 172.21.15.40:0/292148043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f441415b760 msgr2=0x7f44140c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:12.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.466+0000 7f4412ffd640 1 --2- 172.21.15.40:0/292148043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f441415b760 0x7f44140c9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:12.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.466+0000 7f4412ffd640 1 -- 172.21.15.40:0/292148043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc009560 con 0x7f44140d0100 2023-10-18T09:07:12.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.467+0000 7f4412ffd640 1 --2- 172.21.15.40:0/292148043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44140d0100 0x7f44140c7ac0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f440400eb00 tx=0x7f440400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:12.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.467+0000 7f440b7fe640 1 -- 172.21.15.40:0/292148043 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4404004b20 con 0x7f44140d0100 2023-10-18T09:07:12.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.467+0000 7f440b7fe640 1 -- 172.21.15.40:0/292148043 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4404004ca0 con 0x7f44140d0100 2023-10-18T09:07:12.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.467+0000 7f440b7fe640 1 -- 172.21.15.40:0/292148043 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4404005280 con 0x7f44140d0100 2023-10-18T09:07:12.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.467+0000 7f4419573640 1 -- 172.21.15.40:0/292148043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44141e3b70 con 0x7f44140d0100 2023-10-18T09:07:12.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.467+0000 7f4419573640 1 -- 172.21.15.40:0/292148043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44141e0ec0 con 0x7f44140d0100 2023-10-18T09:07:12.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.469+0000 7f4419573640 1 -- 172.21.15.40:0/292148043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44140739c0 con 0x7f44140d0100 2023-10-18T09:07:12.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.470+0000 7f440b7fe640 1 -- 172.21.15.40:0/292148043 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f44040053e0 con 0x7f44140d0100 2023-10-18T09:07:12.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.470+0000 7f440b7fe640 1 --2- 172.21.15.40:0/292148043 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f43f406ae40 0x7f43f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:12.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.471+0000 7f440b7fe640 1 -- 172.21.15.40:0/292148043 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4404014070 con 0x7f44140d0100 2023-10-18T09:07:12.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.471+0000 7f44127fc640 1 --2- 172.21.15.40:0/292148043 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f43f406ae40 0x7f43f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:12.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.472+0000 7f44127fc640 1 --2- 172.21.15.40:0/292148043 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f43f406ae40 0x7f43f406d300 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f43fc035000 tx=0x7f43fc02e040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:12.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.475+0000 7f440b7fe640 1 -- 172.21.15.40:0/292148043 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4404055340 con 0x7f44140d0100 2023-10-18T09:07:12.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.744+0000 7f4419573640 1 -- 172.21.15.40:0/292148043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f44140cddd0 con 0x7f44140d0100 2023-10-18T09:07:12.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.745+0000 7f440b7fe640 1 -- 172.21.15.40:0/292148043 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4404058960 con 0x7f44140d0100 2023-10-18T09:07:12.748 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:07:12.748 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:07:12.748 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2023-10-18T09:07:12.748 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.748 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2023-10-18T09:07:12.749 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lYsb5CxAAGButZ6+Usc62kJ0ksxZrYw==", 2023-10-18T09:07:12.749 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.749 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:07:12.749 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:07:12.749 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:07:12.750 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:07:12.750 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.751 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2023-10-18T09:07:12.751 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lAqlhMBAApAhxCYB9d9bz1GibADDRSQ==", 2023-10-18T09:07:12.751 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.751 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:07:12.751 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:07:12.752 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:07:12.752 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:07:12.752 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.752 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.753 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2023-10-18T09:07:12.753 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lPVh+HhAAzucSx+ksPDBkqJGa8+pyJw==", 2023-10-18T09:07:12.753 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.753 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:07:12.753 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:07:12.754 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:07:12.754 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:07:12.754 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.754 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.755 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2023-10-18T09:07:12.755 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lP7QCBhAAog7qDCgGVv0T7WKsRkATkA==", 2023-10-18T09:07:12.755 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.755 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:07:12.756 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:07:12.756 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:07:12.756 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:07:12.756 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.756 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.756 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.757 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2023-10-18T09:07:12.757 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lVx+hAhAA6ZpIyZKhbqy8tV84piGILA==", 2023-10-18T09:07:12.757 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.757 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:07:12.758 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:07:12.758 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:07:12.758 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.758 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.758 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.759 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2023-10-18T09:07:12.759 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lk9LwExAA7zFzQOn2hcvnck2cOZZ7AA==", 2023-10-18T09:07:12.759 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.759 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:07:12.760 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:07:12.760 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:07:12.760 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.760 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.760 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.761 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2023-10-18T09:07:12.761 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lK930JRAA1GyKRkIpXPHHGqY8owYWRQ==", 2023-10-18T09:07:12.761 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.761 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:07:12.761 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:07:12.762 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:07:12.762 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.762 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.762 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.762 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2023-10-18T09:07:12.763 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9liBWyNRAAMHBBdG5nbrGQupqvUtuKMA==", 2023-10-18T09:07:12.763 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.763 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:07:12.763 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:07:12.763 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:07:12.764 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.764 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.764 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.764 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2023-10-18T09:07:12.764 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9loDoOAhAAjHZub4wnB7dCr86rxAtI5g==", 2023-10-18T09:07:12.765 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:07:12.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:07:12.765 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:07:12.766 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.766 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.766 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.766 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2023-10-18T09:07:12.766 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lhqNfERAAw410g3DwSXrK23YkxVBFVw==", 2023-10-18T09:07:12.767 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:07:12.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:07:12.767 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:07:12.768 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.768 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.768 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.768 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2023-10-18T09:07:12.769 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lRRScIRAAg0pmfxCQBCPYodj1Lko+Lw==", 2023-10-18T09:07:12.769 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:07:12.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:07:12.770 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:07:12.770 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.770 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.770 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.770 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2023-10-18T09:07:12.771 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9ld1dXMhAAyovlrKOhVuynEiuPTgFcjQ==", 2023-10-18T09:07:12.771 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:07:12.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:07:12.771 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:07:12.771 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.772 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.772 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.772 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2023-10-18T09:07:12.772 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w==", 2023-10-18T09:07:12.773 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:07:12.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2023-10-18T09:07:12.773 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T09:07:12.773 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.774 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.774 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.774 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2023-10-18T09:07:12.774 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lAB3oKhAAD43WFdC23WGF6WUi4JKsvg==", 2023-10-18T09:07:12.775 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:07:12.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:07:12.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:07:12.776 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:07:12.776 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.776 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.776 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.776 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2023-10-18T09:07:12.777 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lu57fHxAAdzsVU2KKsWRkaARRS8TGjw==", 2023-10-18T09:07:12.777 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:07:12.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:07:12.778 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:07:12.778 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:07:12.778 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.778 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.778 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.778 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2023-10-18T09:07:12.779 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lygWsNRAA1OziXxk7hBtTw2Aq8OlSKQ==", 2023-10-18T09:07:12.779 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:07:12.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:07:12.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:07:12.780 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:07:12.780 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.780 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.780 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.780 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2023-10-18T09:07:12.781 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBMnS9lnrUjNhAAHP2eezRHF4/Jgox7iPwy7g==", 2023-10-18T09:07:12.781 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:07:12.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:07:12.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:07:12.782 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:07:12.782 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.782 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.782 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2023-10-18T09:07:12.783 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9lCf/xOhAAJHVoKIswWUcVGKQpiaJ12A==", 2023-10-18T09:07:12.783 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:07:12.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T09:07:12.783 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:07:12.784 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:07:12.784 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:12.784 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2023-10-18T09:07:12.784 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9loxRXLhAAQK1n1/4PB35umHxOKQzz7Q==", 2023-10-18T09:07:12.785 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:07:12.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:07:12.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T09:07:12.785 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:07:12.785 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.786 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:12.786 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:07:12.786 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:07:12.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.747+0000 7f4419573640 1 -- 172.21.15.40:0/292148043 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f43f406ae40 msgr2=0x7f43f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:12.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.747+0000 7f4419573640 1 --2- 172.21.15.40:0/292148043 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f43f406ae40 0x7f43f406d300 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f43fc035000 tx=0x7f43fc02e040 comp rx=0 tx=0).stop 2023-10-18T09:07:12.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.748+0000 7f4419573640 1 -- 172.21.15.40:0/292148043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44140d0100 msgr2=0x7f44140c7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:12.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.748+0000 7f4419573640 1 --2- 172.21.15.40:0/292148043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44140d0100 0x7f44140c7ac0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f440400eb00 tx=0x7f440400c2d0 comp rx=0 tx=0).stop 2023-10-18T09:07:12.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.748+0000 7f4419573640 1 -- 172.21.15.40:0/292148043 shutdown_connections 2023-10-18T09:07:12.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.748+0000 7f4419573640 1 --2- 172.21.15.40:0/292148043 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f43f406ae40 0x7f43f406d300 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:12.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.748+0000 7f4419573640 1 --2- 172.21.15.40:0/292148043 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f441415b760 0x7f44140c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:12.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.748+0000 7f4419573640 1 --2- 172.21.15.40:0/292148043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44140d19c0 0x7f4414148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:12.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.748+0000 7f4419573640 1 --2- 172.21.15.40:0/292148043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44140d0100 0x7f44140c7ac0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:12.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.748+0000 7f4419573640 1 -- 172.21.15.40:0/292148043 >> 172.21.15.40:0/292148043 conn(0x7f441409a9a0 msgr2=0x7f4414072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:12.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.748+0000 7f4419573640 1 -- 172.21.15.40:0/292148043 shutdown_connections 2023-10-18T09:07:12.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:12.748+0000 7f4419573640 1 -- 172.21.15.40:0/292148043 wait complete. 2023-10-18T09:07:12.791 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:07:13.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.024+0000 7efe4eb35640 1 -- 172.21.15.40:0/2202555769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe340052a0 msgr2=0x7efe3400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:13.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.024+0000 7efe4eb35640 1 --2- 172.21.15.40:0/2202555769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe340052a0 0x7efe3400b980 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7efe4000c640 tx=0x7efe340057f0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.024+0000 7efe4eb35640 1 -- 172.21.15.40:0/2202555769 shutdown_connections 2023-10-18T09:07:13.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.024+0000 7efe4eb35640 1 --2- 172.21.15.40:0/2202555769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe340052a0 0x7efe3400b980 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.024+0000 7efe4eb35640 1 -- 172.21.15.40:0/2202555769 >> 172.21.15.40:0/2202555769 conn(0x7efe4808ad00 msgr2=0x7efe4809a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:13.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.024+0000 7efe4eb35640 1 -- 172.21.15.40:0/2202555769 shutdown_connections 2023-10-18T09:07:13.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.024+0000 7efe4eb35640 1 -- 172.21.15.40:0/2202555769 wait complete. 2023-10-18T09:07:13.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.026+0000 7efe4eb35640 1 Processor -- start 2023-10-18T09:07:13.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.026+0000 7efe4eb35640 1 -- start start 2023-10-18T09:07:13.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.027+0000 7efe4eb35640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe340052a0 0x7efe480b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:13.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.027+0000 7efe4eb35640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe4809cc90 0x7efe481f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:13.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.027+0000 7efe4eb35640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe481593e0 0x7efe480cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:13.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.027+0000 7efe4eb35640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe481f2450 con 0x7efe340052a0 2023-10-18T09:07:13.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.027+0000 7efe4eb35640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe480b54c0 con 0x7efe481593e0 2023-10-18T09:07:13.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.027+0000 7efe4eb35640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe480b57a0 con 0x7efe4809cc90 2023-10-18T09:07:13.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.027+0000 7efe4db33640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe340052a0 0x7efe480b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:13.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.027+0000 7efe4db33640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe340052a0 0x7efe480b4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53748/0 (socket says 172.21.15.40:53748) 2023-10-18T09:07:13.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.027+0000 7efe4db33640 1 -- 172.21.15.40:0/2049004882 learned_addr learned my addr 172.21.15.40:0/2049004882 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:13.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.028+0000 7efe4d332640 1 --2- 172.21.15.40:0/2049004882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe4809cc90 0x7efe481f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:13.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.028+0000 7efe4e334640 1 --2- 172.21.15.40:0/2049004882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe481593e0 0x7efe480cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:13.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.028+0000 7efe4db33640 1 -- 172.21.15.40:0/2049004882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe4809cc90 msgr2=0x7efe481f1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.028+0000 7efe4db33640 1 --2- 172.21.15.40:0/2049004882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe4809cc90 0x7efe481f1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.028+0000 7efe4db33640 1 -- 172.21.15.40:0/2049004882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe481593e0 msgr2=0x7efe480cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.028+0000 7efe4db33640 1 --2- 172.21.15.40:0/2049004882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe481593e0 0x7efe480cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.028+0000 7efe4db33640 1 -- 172.21.15.40:0/2049004882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe480b5a80 con 0x7efe340052a0 2023-10-18T09:07:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.028+0000 7efe4e334640 1 --2- 172.21.15.40:0/2049004882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe481593e0 0x7efe480cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:13.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.028+0000 7efe4db33640 1 --2- 172.21.15.40:0/2049004882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe340052a0 0x7efe480b4e00 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7efe4400d580 tx=0x7efe4400da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.028+0000 7efe3e7fc640 1 -- 172.21.15.40:0/2049004882 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44012630 con 0x7efe340052a0 2023-10-18T09:07:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.029+0000 7efe3e7fc640 1 -- 172.21.15.40:0/2049004882 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efe44012790 con 0x7efe340052a0 2023-10-18T09:07:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.029+0000 7efe3e7fc640 1 -- 172.21.15.40:0/2049004882 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44012ba0 con 0x7efe340052a0 2023-10-18T09:07:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.029+0000 7efe4eb35640 1 -- 172.21.15.40:0/2049004882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe480b5d60 con 0x7efe340052a0 2023-10-18T09:07:13.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.029+0000 7efe4eb35640 1 -- 172.21.15.40:0/2049004882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe48159890 con 0x7efe340052a0 2023-10-18T09:07:13.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.030+0000 7efe4eb35640 1 -- 172.21.15.40:0/2049004882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe48071da0 con 0x7efe340052a0 2023-10-18T09:07:13.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.031+0000 7efe3e7fc640 1 -- 172.21.15.40:0/2049004882 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efe4401b400 con 0x7efe340052a0 2023-10-18T09:07:13.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.032+0000 7efe3e7fc640 1 --2- 172.21.15.40:0/2049004882 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe2c06ae70 0x7efe2c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:13.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.032+0000 7efe3e7fc640 1 -- 172.21.15.40:0/2049004882 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efe44089540 con 0x7efe340052a0 2023-10-18T09:07:13.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.033+0000 7efe4d332640 1 --2- 172.21.15.40:0/2049004882 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe2c06ae70 0x7efe2c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:13.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.034+0000 7efe4d332640 1 --2- 172.21.15.40:0/2049004882 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe2c06ae70 0x7efe2c06d330 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efe4806bdf0 tx=0x7efe38005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:13.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.036+0000 7efe3e7fc640 1 -- 172.21.15.40:0/2049004882 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe440548b0 con 0x7efe340052a0 2023-10-18T09:07:13.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.234+0000 7efe4eb35640 1 -- 172.21.15.40:0/2049004882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe48066e00 con 0x7efe340052a0 2023-10-18T09:07:13.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.236+0000 7efe3e7fc640 1 -- 172.21.15.40:0/2049004882 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+12880 (secure 0 0 0) 0x7efe44057ed0 con 0x7efe340052a0 2023-10-18T09:07:13.238 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:07:13.239 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":175,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:07:12.197280+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:07:13.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.238+0000 7efe4eb35640 1 -- 172.21.15.40:0/2049004882 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe2c06ae70 msgr2=0x7efe2c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:13.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.238+0000 7efe4eb35640 1 --2- 172.21.15.40:0/2049004882 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe2c06ae70 0x7efe2c06d330 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efe4806bdf0 tx=0x7efe38005f90 comp rx=0 tx=0).stop 2023-10-18T09:07:13.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.238+0000 7efe4eb35640 1 -- 172.21.15.40:0/2049004882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe340052a0 msgr2=0x7efe480b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:13.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.238+0000 7efe4eb35640 1 --2- 172.21.15.40:0/2049004882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe340052a0 0x7efe480b4e00 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7efe4400d580 tx=0x7efe4400da40 comp rx=0 tx=0).stop 2023-10-18T09:07:13.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.238+0000 7efe4eb35640 1 -- 172.21.15.40:0/2049004882 shutdown_connections 2023-10-18T09:07:13.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.238+0000 7efe4eb35640 1 --2- 172.21.15.40:0/2049004882 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7efe2c06ae70 0x7efe2c06d330 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.238+0000 7efe4eb35640 1 --2- 172.21.15.40:0/2049004882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe481593e0 0x7efe480cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.238+0000 7efe4eb35640 1 --2- 172.21.15.40:0/2049004882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe4809cc90 0x7efe481f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.238+0000 7efe4eb35640 1 --2- 172.21.15.40:0/2049004882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe340052a0 0x7efe480b4e00 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.238+0000 7efe4eb35640 1 -- 172.21.15.40:0/2049004882 >> 172.21.15.40:0/2049004882 conn(0x7efe4808ad00 msgr2=0x7efe480af610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:13.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.239+0000 7efe4eb35640 1 -- 172.21.15.40:0/2049004882 shutdown_connections 2023-10-18T09:07:13.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.239+0000 7efe4eb35640 1 -- 172.21.15.40:0/2049004882 wait complete. 2023-10-18T09:07:13.260 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:07:13.261 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:07:13.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.493+0000 7f8233580640 1 -- 172.21.15.40:0/3253481847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b2c20 msgr2=0x7f822c0ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:13.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.493+0000 7f8233580640 1 --2- 172.21.15.40:0/3253481847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b2c20 0x7f822c0ca5f0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f8228009830 tx=0x7f822802bcd0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.494+0000 7f8233580640 1 -- 172.21.15.40:0/3253481847 shutdown_connections 2023-10-18T09:07:13.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.494+0000 7f8233580640 1 --2- 172.21.15.40:0/3253481847 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822c15a200 0x7f822c0b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.494+0000 7f8233580640 1 --2- 172.21.15.40:0/3253481847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b2c20 0x7f822c0ca5f0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.494+0000 7f8233580640 1 -- 172.21.15.40:0/3253481847 >> 172.21.15.40:0/3253481847 conn(0x7f822c069720 msgr2=0x7f822c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:13.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.494+0000 7f8233580640 1 -- 172.21.15.40:0/3253481847 shutdown_connections 2023-10-18T09:07:13.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.494+0000 7f8233580640 1 -- 172.21.15.40:0/3253481847 wait complete. 2023-10-18T09:07:13.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.495+0000 7f8233580640 1 Processor -- start 2023-10-18T09:07:13.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.496+0000 7f8233580640 1 -- start start 2023-10-18T09:07:13.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f8233580640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822c0aec00 0x7f822c14ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:13.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f8233580640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b2c20 0x7f822c151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:13.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f8233580640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822c15a200 0x7f822c0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:13.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f8233580640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c06f860 con 0x7f822c0b2c20 2023-10-18T09:07:13.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f8233580640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c06f510 con 0x7f822c15a200 2023-10-18T09:07:13.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f8233580640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822c1523a0 con 0x7f822c0aec00 2023-10-18T09:07:13.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f8231d7d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b2c20 0x7f822c151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:13.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f8231d7d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b2c20 0x7f822c151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53774/0 (socket says 172.21.15.40:53774) 2023-10-18T09:07:13.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f8231d7d640 1 -- 172.21.15.40:0/2128823803 learned_addr learned my addr 172.21.15.40:0/2128823803 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:13.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f823257e640 1 --2- 172.21.15.40:0/2128823803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822c0aec00 0x7f822c14ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:13.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f8231d7d640 1 -- 172.21.15.40:0/2128823803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822c0aec00 msgr2=0x7f822c14ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:13.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f8231d7d640 1 --2- 172.21.15.40:0/2128823803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822c0aec00 0x7f822c14ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f8231d7d640 1 -- 172.21.15.40:0/2128823803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822c15a200 msgr2=0x7f822c0cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:13.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f8232d7f640 1 --2- 172.21.15.40:0/2128823803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822c15a200 0x7f822c0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:13.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f8231d7d640 1 --2- 172.21.15.40:0/2128823803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822c15a200 0x7f822c0cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f8231d7d640 1 -- 172.21.15.40:0/2128823803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8228009490 con 0x7f822c0b2c20 2023-10-18T09:07:13.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f823257e640 1 --2- 172.21.15.40:0/2128823803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822c0aec00 0x7f822c14ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:13.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.497+0000 7f8232d7f640 1 --2- 172.21.15.40:0/2128823803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822c15a200 0x7f822c0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:13.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.498+0000 7f8231d7d640 1 --2- 172.21.15.40:0/2128823803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b2c20 0x7f822c151950 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f821c00d950 tx=0x7f821c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:13.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.498+0000 7f8222ffd640 1 -- 172.21.15.40:0/2128823803 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c009c30 con 0x7f822c0b2c20 2023-10-18T09:07:13.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.498+0000 7f8222ffd640 1 -- 172.21.15.40:0/2128823803 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f821c009e80 con 0x7f822c0b2c20 2023-10-18T09:07:13.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.498+0000 7f8222ffd640 1 -- 172.21.15.40:0/2128823803 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c0106c0 con 0x7f822c0b2c20 2023-10-18T09:07:13.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.498+0000 7f8233580640 1 -- 172.21.15.40:0/2128823803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822c152680 con 0x7f822c0b2c20 2023-10-18T09:07:13.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.498+0000 7f8233580640 1 -- 172.21.15.40:0/2128823803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822c14f1d0 con 0x7f822c0b2c20 2023-10-18T09:07:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.500+0000 7f8233580640 1 -- 172.21.15.40:0/2128823803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822c06f9d0 con 0x7f822c0b2c20 2023-10-18T09:07:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.501+0000 7f8222ffd640 1 -- 172.21.15.40:0/2128823803 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f821c010820 con 0x7f822c0b2c20 2023-10-18T09:07:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.502+0000 7f8222ffd640 1 --2- 172.21.15.40:0/2128823803 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f821006ae90 0x7f821006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.502+0000 7f8222ffd640 1 -- 172.21.15.40:0/2128823803 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f821c0897b0 con 0x7f822c0b2c20 2023-10-18T09:07:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.502+0000 7f823257e640 1 --2- 172.21.15.40:0/2128823803 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f821006ae90 0x7f821006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:13.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.503+0000 7f823257e640 1 --2- 172.21.15.40:0/2128823803 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f821006ae90 0x7f821006d350 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8228036000 tx=0x7f8228004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:13.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.506+0000 7f8222ffd640 1 -- 172.21.15.40:0/2128823803 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f821c054b90 con 0x7f822c0b2c20 2023-10-18T09:07:13.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.699+0000 7f8233580640 1 -- 172.21.15.40:0/2128823803 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f822c066e00 con 0x7f822c0b2c20 2023-10-18T09:07:13.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.700+0000 7f8222ffd640 1 -- 172.21.15.40:0/2128823803 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v175) v1 ==== 93+0+10 (secure 0 0 0) 0x7f821c0200a0 con 0x7f822c0b2c20 2023-10-18T09:07:13.702 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:07:13.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.702+0000 7f8233580640 1 -- 172.21.15.40:0/2128823803 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f821006ae90 msgr2=0x7f821006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:13.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.702+0000 7f8233580640 1 --2- 172.21.15.40:0/2128823803 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f821006ae90 0x7f821006d350 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8228036000 tx=0x7f8228004730 comp rx=0 tx=0).stop 2023-10-18T09:07:13.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.702+0000 7f8233580640 1 -- 172.21.15.40:0/2128823803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b2c20 msgr2=0x7f822c151950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:13.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.702+0000 7f8233580640 1 --2- 172.21.15.40:0/2128823803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b2c20 0x7f822c151950 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f821c00d950 tx=0x7f821c00de10 comp rx=0 tx=0).stop 2023-10-18T09:07:13.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.703+0000 7f8233580640 1 -- 172.21.15.40:0/2128823803 shutdown_connections 2023-10-18T09:07:13.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.703+0000 7f8233580640 1 --2- 172.21.15.40:0/2128823803 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f821006ae90 0x7f821006d350 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.703+0000 7f8233580640 1 --2- 172.21.15.40:0/2128823803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822c15a200 0x7f822c0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.703+0000 7f8233580640 1 --2- 172.21.15.40:0/2128823803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f822c0b2c20 0x7f822c151950 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.703+0000 7f8233580640 1 --2- 172.21.15.40:0/2128823803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f822c0aec00 0x7f822c14ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.703+0000 7f8233580640 1 -- 172.21.15.40:0/2128823803 >> 172.21.15.40:0/2128823803 conn(0x7f822c069720 msgr2=0x7f822c0ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:13.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.703+0000 7f8233580640 1 -- 172.21.15.40:0/2128823803 shutdown_connections 2023-10-18T09:07:13.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.703+0000 7f8233580640 1 -- 172.21.15.40:0/2128823803 wait complete. 2023-10-18T09:07:13.725 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T09:07:13.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.941+0000 7fcc8a343640 1 -- 172.21.15.40:0/1729119327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc8415b4a0 msgr2=0x7fcc840cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:13.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.941+0000 7fcc8a343640 1 --2- 172.21.15.40:0/1729119327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc8415b4a0 0x7fcc840cfe60 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fcc80005fd0 tx=0x7fcc8002bd80 comp rx=0 tx=0).stop 2023-10-18T09:07:13.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.941+0000 7fcc8a343640 1 -- 172.21.15.40:0/1729119327 shutdown_connections 2023-10-18T09:07:13.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.941+0000 7fcc8a343640 1 --2- 172.21.15.40:0/1729119327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc840cdca0 0x7fcc840ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.941+0000 7fcc8a343640 1 --2- 172.21.15.40:0/1729119327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc8415b4a0 0x7fcc840cfe60 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.941+0000 7fcc8a343640 1 -- 172.21.15.40:0/1729119327 >> 172.21.15.40:0/1729119327 conn(0x7fcc8408aca0 msgr2=0x7fcc8409a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:13.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.941+0000 7fcc8a343640 1 -- 172.21.15.40:0/1729119327 shutdown_connections 2023-10-18T09:07:13.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.941+0000 7fcc8a343640 1 -- 172.21.15.40:0/1729119327 wait complete. 2023-10-18T09:07:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.943+0000 7fcc8a343640 1 Processor -- start 2023-10-18T09:07:13.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.943+0000 7fcc8a343640 1 -- start start 2023-10-18T09:07:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.944+0000 7fcc8a343640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc840cdca0 0x7fcc84151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:13.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.944+0000 7fcc8a343640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc8415b4a0 0x7fcc841f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.944+0000 7fcc8a343640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc8415c240 0x7fcc840c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.944+0000 7fcc8a343640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8415d3d0 con 0x7fcc8415b4a0 2023-10-18T09:07:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.944+0000 7fcc8a343640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc841f6120 con 0x7fcc8415c240 2023-10-18T09:07:13.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.944+0000 7fcc8a343640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8415c7c0 con 0x7fcc840cdca0 2023-10-18T09:07:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.945+0000 7fcc88b40640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc8415b4a0 0x7fcc841f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.945+0000 7fcc88b40640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc8415b4a0 0x7fcc841f5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53798/0 (socket says 172.21.15.40:53798) 2023-10-18T09:07:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.945+0000 7fcc88b40640 1 -- 172.21.15.40:0/1841282941 learned_addr learned my addr 172.21.15.40:0/1841282941 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.945+0000 7fcc89341640 1 --2- 172.21.15.40:0/1841282941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc840cdca0 0x7fcc84151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:13.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.945+0000 7fcc89b42640 1 --2- 172.21.15.40:0/1841282941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc8415c240 0x7fcc840c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.945+0000 7fcc88b40640 1 -- 172.21.15.40:0/1841282941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc840cdca0 msgr2=0x7fcc84151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.945+0000 7fcc88b40640 1 --2- 172.21.15.40:0/1841282941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc840cdca0 0x7fcc84151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.945+0000 7fcc88b40640 1 -- 172.21.15.40:0/1841282941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc8415c240 msgr2=0x7fcc840c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.945+0000 7fcc88b40640 1 --2- 172.21.15.40:0/1841282941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc8415c240 0x7fcc840c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:13.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.945+0000 7fcc88b40640 1 -- 172.21.15.40:0/1841282941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc80009560 con 0x7fcc8415b4a0 2023-10-18T09:07:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.945+0000 7fcc88b40640 1 --2- 172.21.15.40:0/1841282941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc8415b4a0 0x7fcc841f5be0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400b3e0 tx=0x7fcc7400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.946+0000 7fcc79ffb640 1 -- 172.21.15.40:0/1841282941 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc74011830 con 0x7fcc8415b4a0 2023-10-18T09:07:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.946+0000 7fcc79ffb640 1 -- 172.21.15.40:0/1841282941 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcc74011aa0 con 0x7fcc8415b4a0 2023-10-18T09:07:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.946+0000 7fcc79ffb640 1 -- 172.21.15.40:0/1841282941 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc74010380 con 0x7fcc8415b4a0 2023-10-18T09:07:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.946+0000 7fcc8a343640 1 -- 172.21.15.40:0/1841282941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8415caa0 con 0x7fcc8415b4a0 2023-10-18T09:07:13.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.946+0000 7fcc8a343640 1 -- 172.21.15.40:0/1841282941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8415d060 con 0x7fcc8415b4a0 2023-10-18T09:07:13.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.947+0000 7fcc8a343640 1 -- 172.21.15.40:0/1841282941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc84071da0 con 0x7fcc8415b4a0 2023-10-18T09:07:13.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.948+0000 7fcc79ffb640 1 -- 172.21.15.40:0/1841282941 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcc740104e0 con 0x7fcc8415b4a0 2023-10-18T09:07:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.949+0000 7fcc79ffb640 1 --2- 172.21.15.40:0/1841282941 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcc6806ae40 0x7fcc6806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.950+0000 7fcc79ffb640 1 -- 172.21.15.40:0/1841282941 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcc74089540 con 0x7fcc8415b4a0 2023-10-18T09:07:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.950+0000 7fcc89341640 1 --2- 172.21.15.40:0/1841282941 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcc6806ae40 0x7fcc6806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:13.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.951+0000 7fcc89341640 1 --2- 172.21.15.40:0/1841282941 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcc6806ae40 0x7fcc6806d300 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fcc80035000 tx=0x7fcc80004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:13.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:13.954+0000 7fcc79ffb640 1 -- 172.21.15.40:0/1841282941 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc740549d0 con 0x7fcc8415b4a0 2023-10-18T09:07:14.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.150+0000 7fcc8a343640 1 -- 172.21.15.40:0/1841282941 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcc840667d0 con 0x7fcc8415b4a0 2023-10-18T09:07:14.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.150+0000 7fcc79ffb640 1 -- 172.21.15.40:0/1841282941 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v175) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcc74015060 con 0x7fcc8415b4a0 2023-10-18T09:07:14.154 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:07:14.154 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-10-18T09:07:14.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:07:14.155 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-10-18T09:07:14.155 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-10-18T09:07:14.155 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:07:14.155 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-10-18T09:07:14.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.152+0000 7fcc8a343640 1 -- 172.21.15.40:0/1841282941 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcc6806ae40 msgr2=0x7fcc6806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:14.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.153+0000 7fcc8a343640 1 --2- 172.21.15.40:0/1841282941 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcc6806ae40 0x7fcc6806d300 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fcc80035000 tx=0x7fcc80004730 comp rx=0 tx=0).stop 2023-10-18T09:07:14.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.153+0000 7fcc8a343640 1 -- 172.21.15.40:0/1841282941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc8415b4a0 msgr2=0x7fcc841f5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:14.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.153+0000 7fcc8a343640 1 --2- 172.21.15.40:0/1841282941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc8415b4a0 0x7fcc841f5be0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400b3e0 tx=0x7fcc7400b8a0 comp rx=0 tx=0).stop 2023-10-18T09:07:14.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.153+0000 7fcc8a343640 1 -- 172.21.15.40:0/1841282941 shutdown_connections 2023-10-18T09:07:14.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.153+0000 7fcc8a343640 1 --2- 172.21.15.40:0/1841282941 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcc6806ae40 0x7fcc6806d300 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:14.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.153+0000 7fcc8a343640 1 --2- 172.21.15.40:0/1841282941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc8415c240 0x7fcc840c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:14.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.153+0000 7fcc8a343640 1 --2- 172.21.15.40:0/1841282941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc8415b4a0 0x7fcc841f5be0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:14.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.153+0000 7fcc8a343640 1 --2- 172.21.15.40:0/1841282941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc840cdca0 0x7fcc84151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:14.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.153+0000 7fcc8a343640 1 -- 172.21.15.40:0/1841282941 >> 172.21.15.40:0/1841282941 conn(0x7fcc8408aca0 msgr2=0x7fcc8415a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:14.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.153+0000 7fcc8a343640 1 -- 172.21.15.40:0/1841282941 shutdown_connections 2023-10-18T09:07:14.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.153+0000 7fcc8a343640 1 -- 172.21.15.40:0/1841282941 wait complete. 2023-10-18T09:07:14.172 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T09:07:14.172 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T09:07:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.404+0000 7febb759e640 1 -- 172.21.15.40:0/3324774594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feba00039a0 msgr2=0x7feba0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:14.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.404+0000 7febb759e640 1 --2- 172.21.15.40:0/3324774594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feba00039a0 0x7feba0003dc0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7feba8006fd0 tx=0x7feba00057f0 comp rx=0 tx=0).stop 2023-10-18T09:07:14.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.404+0000 7febb6d9d640 1 -- 172.21.15.40:0/3324774594 reap_dead start 2023-10-18T09:07:14.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.405+0000 7febb759e640 1 -- 172.21.15.40:0/3324774594 shutdown_connections 2023-10-18T09:07:14.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.405+0000 7febb759e640 1 -- 172.21.15.40:0/3324774594 >> 172.21.15.40:0/3324774594 conn(0x7febb806b7d0 msgr2=0x7febb806bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:14.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.405+0000 7febb759e640 1 -- 172.21.15.40:0/3324774594 shutdown_connections 2023-10-18T09:07:14.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.405+0000 7febb759e640 1 -- 172.21.15.40:0/3324774594 wait complete. 2023-10-18T09:07:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.406+0000 7febb759e640 1 Processor -- start 2023-10-18T09:07:14.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.407+0000 7febb759e640 1 -- start start 2023-10-18T09:07:14.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.408+0000 7febb759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feba00039a0 0x7febb813ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.408+0000 7febb759e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febb814ec40 0x7febb8152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.408+0000 7febb759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febb813b4f0 0x7febb813b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.408+0000 7febb759e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb8157230 con 0x7feba00039a0 2023-10-18T09:07:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.408+0000 7febb759e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb81574d0 con 0x7febb813b4f0 2023-10-18T09:07:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.408+0000 7febb759e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb81577b0 con 0x7febb814ec40 2023-10-18T09:07:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.408+0000 7febb659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feba00039a0 0x7febb813ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.408+0000 7febb659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feba00039a0 0x7febb813ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53830/0 (socket says 172.21.15.40:53830) 2023-10-18T09:07:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.408+0000 7febb659c640 1 -- 172.21.15.40:0/1359937143 learned_addr learned my addr 172.21.15.40:0/1359937143 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.408+0000 7febb5d9b640 1 --2- 172.21.15.40:0/1359937143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febb814ec40 0x7febb8152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.408+0000 7febb6d9d640 1 --2- 172.21.15.40:0/1359937143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febb813b4f0 0x7febb813b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.408+0000 7febb659c640 1 -- 172.21.15.40:0/1359937143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febb814ec40 msgr2=0x7febb8152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.408+0000 7febb659c640 1 --2- 172.21.15.40:0/1359937143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febb814ec40 0x7febb8152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.408+0000 7febb659c640 1 -- 172.21.15.40:0/1359937143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febb813b4f0 msgr2=0x7febb813b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.408+0000 7febb659c640 1 --2- 172.21.15.40:0/1359937143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febb813b4f0 0x7febb813b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.408+0000 7febb659c640 1 -- 172.21.15.40:0/1359937143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb8157a90 con 0x7feba00039a0 2023-10-18T09:07:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.409+0000 7febb659c640 1 --2- 172.21.15.40:0/1359937143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feba00039a0 0x7febb813ac10 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7feba8001930 tx=0x7feba8001430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.409+0000 7feba6ffd640 1 -- 172.21.15.40:0/1359937143 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8043070 con 0x7feba00039a0 2023-10-18T09:07:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.409+0000 7feba6ffd640 1 -- 172.21.15.40:0/1359937143 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feba8006050 con 0x7feba00039a0 2023-10-18T09:07:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.409+0000 7feba6ffd640 1 -- 172.21.15.40:0/1359937143 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8006440 con 0x7feba00039a0 2023-10-18T09:07:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.409+0000 7febb759e640 1 -- 172.21.15.40:0/1359937143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb8157d70 con 0x7feba00039a0 2023-10-18T09:07:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.409+0000 7febb759e640 1 -- 172.21.15.40:0/1359937143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb813b150 con 0x7feba00039a0 2023-10-18T09:07:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.410+0000 7febb759e640 1 -- 172.21.15.40:0/1359937143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb8066ea0 con 0x7feba00039a0 2023-10-18T09:07:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.411+0000 7feba6ffd640 1 -- 172.21.15.40:0/1359937143 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feba8031030 con 0x7feba00039a0 2023-10-18T09:07:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.412+0000 7feba6ffd640 1 --2- 172.21.15.40:0/1359937143 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb9806ae40 0x7feb9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:14.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.413+0000 7feba6ffd640 1 -- 172.21.15.40:0/1359937143 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feba80aa300 con 0x7feba00039a0 2023-10-18T09:07:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.413+0000 7febb5d9b640 1 --2- 172.21.15.40:0/1359937143 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb9806ae40 0x7feb9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.414+0000 7febb5d9b640 1 --2- 172.21.15.40:0/1359937143 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb9806ae40 0x7feb9806d300 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7febb8155990 tx=0x7febac005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:14.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.417+0000 7feba6ffd640 1 -- 172.21.15.40:0/1359937143 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feba8075790 con 0x7feba00039a0 2023-10-18T09:07:14.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:14.605+0000 7febb759e640 1 -- 172.21.15.40:0/1359937143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7febb806f610 con 0x7feba00039a0 2023-10-18T09:07:15.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.332+0000 7feba6ffd640 1 -- 172.21.15.40:0/1359937143 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v176) v1 ==== 138+0+0 (secure 0 0 0) 0x7feba8078db0 con 0x7feba00039a0 2023-10-18T09:07:15.335 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-10-18T09:07:15.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.335+0000 7febb759e640 1 -- 172.21.15.40:0/1359937143 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb9806ae40 msgr2=0x7feb9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:15.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.335+0000 7febb759e640 1 --2- 172.21.15.40:0/1359937143 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb9806ae40 0x7feb9806d300 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7febb8155990 tx=0x7febac005f90 comp rx=0 tx=0).stop 2023-10-18T09:07:15.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.335+0000 7febb759e640 1 -- 172.21.15.40:0/1359937143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feba00039a0 msgr2=0x7febb813ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:15.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.335+0000 7febb759e640 1 --2- 172.21.15.40:0/1359937143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feba00039a0 0x7febb813ac10 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7feba8001930 tx=0x7feba8001430 comp rx=0 tx=0).stop 2023-10-18T09:07:15.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.336+0000 7febb759e640 1 -- 172.21.15.40:0/1359937143 shutdown_connections 2023-10-18T09:07:15.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.336+0000 7febb759e640 1 --2- 172.21.15.40:0/1359937143 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7feb9806ae40 0x7feb9806d300 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:15.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.336+0000 7febb759e640 1 --2- 172.21.15.40:0/1359937143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febb813b4f0 0x7febb813b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:15.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.336+0000 7febb759e640 1 --2- 172.21.15.40:0/1359937143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febb814ec40 0x7febb8152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:15.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.336+0000 7febb759e640 1 --2- 172.21.15.40:0/1359937143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feba00039a0 0x7febb813ac10 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:15.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.336+0000 7febb759e640 1 -- 172.21.15.40:0/1359937143 >> 172.21.15.40:0/1359937143 conn(0x7febb806b7d0 msgr2=0x7febb80ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:15.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.336+0000 7febb759e640 1 -- 172.21.15.40:0/1359937143 shutdown_connections 2023-10-18T09:07:15.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.336+0000 7febb759e640 1 -- 172.21.15.40:0/1359937143 wait complete. 2023-10-18T09:07:15.364 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T09:07:15.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.597+0000 7fe593278640 1 -- 172.21.15.40:0/2222428096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58c0ce080 msgr2=0x7fe58c15bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:15.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.597+0000 7fe593278640 1 --2- 172.21.15.40:0/2222428096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58c0ce080 0x7fe58c15bc20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe57c005fd0 tx=0x7fe57c02be20 comp rx=0 tx=0).stop 2023-10-18T09:07:15.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.598+0000 7fe593278640 1 -- 172.21.15.40:0/2222428096 shutdown_connections 2023-10-18T09:07:15.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.598+0000 7fe593278640 1 --2- 172.21.15.40:0/2222428096 >> v2:172.21.15.144:3300/0 conn(0x7fe58c15c160 0x7fe58c0b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:15.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.598+0000 7fe593278640 1 --2- 172.21.15.40:0/2222428096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58c0ce080 0x7fe58c15bc20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:15.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.598+0000 7fe593278640 1 -- 172.21.15.40:0/2222428096 >> 172.21.15.40:0/2222428096 conn(0x7fe58c0663b0 msgr2=0x7fe58c0b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:15.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.598+0000 7fe593278640 1 -- 172.21.15.40:0/2222428096 shutdown_connections 2023-10-18T09:07:15.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.598+0000 7fe593278640 1 -- 172.21.15.40:0/2222428096 wait complete. 2023-10-18T09:07:15.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.600+0000 7fe593278640 1 Processor -- start 2023-10-18T09:07:15.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.600+0000 7fe593278640 1 -- start start 2023-10-18T09:07:15.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.601+0000 7fe593278640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0af500 0x7fe58c155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:15.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.601+0000 7fe593278640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58c0ce080 0x7fe58c141220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:15.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.601+0000 7fe593278640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe58c15c160 0x7fe58c153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:15.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.601+0000 7fe593278640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c154870 con 0x7fe58c0af500 2023-10-18T09:07:15.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.601+0000 7fe593278640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c154b50 con 0x7fe58c15c160 2023-10-18T09:07:15.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.601+0000 7fe593278640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c154e30 con 0x7fe58c0ce080 2023-10-18T09:07:15.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.601+0000 7fe592276640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0af500 0x7fe58c155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:15.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.601+0000 7fe592276640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0af500 0x7fe58c155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46304/0 (socket says 172.21.15.40:46304) 2023-10-18T09:07:15.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.601+0000 7fe592276640 1 -- 172.21.15.40:0/4246264549 learned_addr learned my addr 172.21.15.40:0/4246264549 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:15.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.601+0000 7fe591a75640 1 --2- 172.21.15.40:0/4246264549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58c0ce080 0x7fe58c141220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:15.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.601+0000 7fe592a77640 1 --2- 172.21.15.40:0/4246264549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe58c15c160 0x7fe58c153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:15.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.602+0000 7fe592276640 1 -- 172.21.15.40:0/4246264549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58c0ce080 msgr2=0x7fe58c141220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:15.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.602+0000 7fe592276640 1 --2- 172.21.15.40:0/4246264549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58c0ce080 0x7fe58c141220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:15.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.602+0000 7fe592276640 1 -- 172.21.15.40:0/4246264549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe58c15c160 msgr2=0x7fe58c153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:15.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.602+0000 7fe592276640 1 --2- 172.21.15.40:0/4246264549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe58c15c160 0x7fe58c153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:15.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.602+0000 7fe592276640 1 -- 172.21.15.40:0/4246264549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c009560 con 0x7fe58c0af500 2023-10-18T09:07:15.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.602+0000 7fe592276640 1 --2- 172.21.15.40:0/4246264549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0af500 0x7fe58c155910 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fe58800c870 tx=0x7fe58800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:15.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.602+0000 7fe582ffd640 1 -- 172.21.15.40:0/4246264549 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588003b70 con 0x7fe58c0af500 2023-10-18T09:07:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.602+0000 7fe582ffd640 1 -- 172.21.15.40:0/4246264549 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe588003cf0 con 0x7fe58c0af500 2023-10-18T09:07:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.602+0000 7fe582ffd640 1 -- 172.21.15.40:0/4246264549 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588003ea0 con 0x7fe58c0af500 2023-10-18T09:07:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.602+0000 7fe593278640 1 -- 172.21.15.40:0/4246264549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58c155110 con 0x7fe58c0af500 2023-10-18T09:07:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.603+0000 7fe593278640 1 -- 172.21.15.40:0/4246264549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58c141760 con 0x7fe58c0af500 2023-10-18T09:07:15.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.604+0000 7fe593278640 1 -- 172.21.15.40:0/4246264549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe58c073570 con 0x7fe58c0af500 2023-10-18T09:07:15.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.605+0000 7fe582ffd640 1 -- 172.21.15.40:0/4246264549 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe58800aab0 con 0x7fe58c0af500 2023-10-18T09:07:15.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.606+0000 7fe582ffd640 1 --2- 172.21.15.40:0/4246264549 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe57006ae40 0x7fe57006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:15.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.606+0000 7fe582ffd640 1 -- 172.21.15.40:0/4246264549 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fe588089460 con 0x7fe58c0af500 2023-10-18T09:07:15.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.606+0000 7fe591a75640 1 --2- 172.21.15.40:0/4246264549 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe57006ae40 0x7fe57006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:15.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.608+0000 7fe591a75640 1 --2- 172.21.15.40:0/4246264549 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe57006ae40 0x7fe57006d300 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe57c035000 tx=0x7fe57c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:15.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.611+0000 7fe582ffd640 1 -- 172.21.15.40:0/4246264549 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5880546f0 con 0x7fe58c0af500 2023-10-18T09:07:15.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:15.796+0000 7fe593278640 1 -- 172.21.15.40:0/4246264549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe58c0701a0 con 0x7fe58c0af500 2023-10-18T09:07:16.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.338+0000 7fe582ffd640 1 -- 172.21.15.40:0/4246264549 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v177) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe588057d10 con 0x7fe58c0af500 2023-10-18T09:07:16.341 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-10-18T09:07:16.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.342+0000 7fe593278640 1 -- 172.21.15.40:0/4246264549 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe57006ae40 msgr2=0x7fe57006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:16.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.342+0000 7fe593278640 1 --2- 172.21.15.40:0/4246264549 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe57006ae40 0x7fe57006d300 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe57c035000 tx=0x7fe57c004730 comp rx=0 tx=0).stop 2023-10-18T09:07:16.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.342+0000 7fe593278640 1 -- 172.21.15.40:0/4246264549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0af500 msgr2=0x7fe58c155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:16.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.342+0000 7fe593278640 1 --2- 172.21.15.40:0/4246264549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0af500 0x7fe58c155910 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fe58800c870 tx=0x7fe58800cd30 comp rx=0 tx=0).stop 2023-10-18T09:07:16.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.342+0000 7fe593278640 1 -- 172.21.15.40:0/4246264549 shutdown_connections 2023-10-18T09:07:16.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.342+0000 7fe593278640 1 --2- 172.21.15.40:0/4246264549 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe57006ae40 0x7fe57006d300 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:16.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.342+0000 7fe593278640 1 --2- 172.21.15.40:0/4246264549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe58c15c160 0x7fe58c153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:16.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.342+0000 7fe593278640 1 --2- 172.21.15.40:0/4246264549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58c0ce080 0x7fe58c141220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:16.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.342+0000 7fe593278640 1 --2- 172.21.15.40:0/4246264549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0af500 0x7fe58c155910 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:16.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.342+0000 7fe593278640 1 -- 172.21.15.40:0/4246264549 >> 172.21.15.40:0/4246264549 conn(0x7fe58c0663b0 msgr2=0x7fe58c152320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:16.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.342+0000 7fe593278640 1 -- 172.21.15.40:0/4246264549 shutdown_connections 2023-10-18T09:07:16.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.342+0000 7fe593278640 1 -- 172.21.15.40:0/4246264549 wait complete. 2023-10-18T09:07:16.376 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T09:07:16.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.578+0000 7f57ed93c640 1 -- 172.21.15.40:0/783420040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d40039a0 msgr2=0x7f57d4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:16.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.578+0000 7f57ed93c640 1 --2- 172.21.15.40:0/783420040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57d40039a0 0x7f57d4003dc0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f57d8001650 tx=0x7f57d40057f0 comp rx=0 tx=0).stop 2023-10-18T09:07:16.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.578+0000 7f57e77fe640 1 -- 172.21.15.40:0/783420040 reap_dead start 2023-10-18T09:07:16.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.578+0000 7f57ed93c640 1 -- 172.21.15.40:0/783420040 shutdown_connections 2023-10-18T09:07:16.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.578+0000 7f57ed93c640 1 -- 172.21.15.40:0/783420040 >> 172.21.15.40:0/783420040 conn(0x7f57e8069720 msgr2=0x7f57e8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:16.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.578+0000 7f57ed93c640 1 -- 172.21.15.40:0/783420040 shutdown_connections 2023-10-18T09:07:16.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.578+0000 7f57ed93c640 1 -- 172.21.15.40:0/783420040 wait complete. 2023-10-18T09:07:16.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.580+0000 7f57ed93c640 1 Processor -- start 2023-10-18T09:07:16.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.580+0000 7f57ed93c640 1 -- start start 2023-10-18T09:07:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.581+0000 7f57ed93c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d40039a0 0x7f57e8150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.581+0000 7f57ed93c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57e814da80 0x7f57e814c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.581+0000 7f57ed93c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e815d690 0x7f57e813f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:16.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.581+0000 7f57ed93c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e814c780 con 0x7f57e815d690 2023-10-18T09:07:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.581+0000 7f57ed93c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e814c9e0 con 0x7f57e814da80 2023-10-18T09:07:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.581+0000 7f57ed93c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e814ccc0 con 0x7f57d40039a0 2023-10-18T09:07:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.581+0000 7f57e77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e815d690 0x7f57e813f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:16.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.581+0000 7f57e77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e815d690 0x7f57e813f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46332/0 (socket says 172.21.15.40:46332) 2023-10-18T09:07:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.581+0000 7f57e77fe640 1 -- 172.21.15.40:0/3119483178 learned_addr learned my addr 172.21.15.40:0/3119483178 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.582+0000 7f57e6ffd640 1 --2- 172.21.15.40:0/3119483178 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d40039a0 0x7f57e8150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.582+0000 7f57e77fe640 1 -- 172.21.15.40:0/3119483178 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d40039a0 msgr2=0x7f57e8150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:16.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.582+0000 7f57e77fe640 1 --2- 172.21.15.40:0/3119483178 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d40039a0 0x7f57e8150520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:16.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.582+0000 7f57e77fe640 1 -- 172.21.15.40:0/3119483178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57e814da80 msgr2=0x7f57e814c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:16.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.582+0000 7f57e77fe640 1 --2- 172.21.15.40:0/3119483178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57e814da80 0x7f57e814c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:16.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.582+0000 7f57e77fe640 1 -- 172.21.15.40:0/3119483178 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e814cfa0 con 0x7f57e815d690 2023-10-18T09:07:16.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.582+0000 7f57e77fe640 1 --2- 172.21.15.40:0/3119483178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e815d690 0x7f57e813f5a0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f57e806b850 tx=0x7f57e000d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:16.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.582+0000 7f57dffff640 1 -- 172.21.15.40:0/3119483178 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0013070 con 0x7f57e815d690 2023-10-18T09:07:16.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.582+0000 7f57dffff640 1 -- 172.21.15.40:0/3119483178 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f57e0003970 con 0x7f57e815d690 2023-10-18T09:07:16.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.582+0000 7f57ed93c640 1 -- 172.21.15.40:0/3119483178 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e814d280 con 0x7f57e815d690 2023-10-18T09:07:16.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.582+0000 7f57dffff640 1 -- 172.21.15.40:0/3119483178 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0003d60 con 0x7f57e815d690 2023-10-18T09:07:16.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.583+0000 7f57ed93c640 1 -- 172.21.15.40:0/3119483178 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e814df00 con 0x7f57e815d690 2023-10-18T09:07:16.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.584+0000 7f57ed93c640 1 -- 172.21.15.40:0/3119483178 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e8066fd0 con 0x7f57e815d690 2023-10-18T09:07:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.585+0000 7f57dffff640 1 -- 172.21.15.40:0/3119483178 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f57e0004030 con 0x7f57e815d690 2023-10-18T09:07:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.586+0000 7f57dffff640 1 --2- 172.21.15.40:0/3119483178 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57c806ae40 0x7f57c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.586+0000 7f57dffff640 1 -- 172.21.15.40:0/3119483178 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f57e0089dd0 con 0x7f57e815d690 2023-10-18T09:07:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.587+0000 7f57e6ffd640 1 --2- 172.21.15.40:0/3119483178 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57c806ae40 0x7f57c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:16.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.588+0000 7f57e6ffd640 1 --2- 172.21.15.40:0/3119483178 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57c806ae40 0x7f57c806d300 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f57d8034000 tx=0x7f57d8039040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:16.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.591+0000 7f57dffff640 1 -- 172.21.15.40:0/3119483178 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57e008e070 con 0x7f57e815d690 2023-10-18T09:07:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:16.830+0000 7f57ed93c640 1 -- 172.21.15.40:0/3119483178 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f57e80705d0 con 0x7f57e815d690 2023-10-18T09:07:17.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.360+0000 7f57dffff640 1 -- 172.21.15.40:0/3119483178 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v78) v1 ==== 174+0+0 (secure 0 0 0) 0x7f57e0054e40 con 0x7f57e815d690 2023-10-18T09:07:17.363 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 20 and data pool 21 2023-10-18T09:07:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.364+0000 7f57ed93c640 1 -- 172.21.15.40:0/3119483178 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57c806ae40 msgr2=0x7f57c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.364+0000 7f57ed93c640 1 --2- 172.21.15.40:0/3119483178 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57c806ae40 0x7f57c806d300 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f57d8034000 tx=0x7f57d8039040 comp rx=0 tx=0).stop 2023-10-18T09:07:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.364+0000 7f57ed93c640 1 -- 172.21.15.40:0/3119483178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e815d690 msgr2=0x7f57e813f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:17.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.364+0000 7f57ed93c640 1 --2- 172.21.15.40:0/3119483178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e815d690 0x7f57e813f5a0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f57e806b850 tx=0x7f57e000d7b0 comp rx=0 tx=0).stop 2023-10-18T09:07:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.367+0000 7f57ed93c640 1 -- 172.21.15.40:0/3119483178 shutdown_connections 2023-10-18T09:07:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.367+0000 7f57ed93c640 1 --2- 172.21.15.40:0/3119483178 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f57c806ae40 0x7f57c806d300 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:17.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.367+0000 7f57ed93c640 1 --2- 172.21.15.40:0/3119483178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e815d690 0x7f57e813f5a0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.367+0000 7f57ed93c640 1 --2- 172.21.15.40:0/3119483178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57e814da80 0x7f57e814c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.367+0000 7f57ed93c640 1 --2- 172.21.15.40:0/3119483178 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57d40039a0 0x7f57e8150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:17.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.367+0000 7f57ed93c640 1 -- 172.21.15.40:0/3119483178 >> 172.21.15.40:0/3119483178 conn(0x7f57e8069720 msgr2=0x7f57e8156810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.368+0000 7f57ed93c640 1 -- 172.21.15.40:0/3119483178 shutdown_connections 2023-10-18T09:07:17.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.368+0000 7f57ed93c640 1 -- 172.21.15.40:0/3119483178 wait complete. 2023-10-18T09:07:17.402 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:07:17.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.595+0000 7f282f2a3640 1 -- 172.21.15.40:0/1477910946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f282806d6f0 msgr2=0x7f28280b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:17.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.595+0000 7f282f2a3640 1 --2- 172.21.15.40:0/1477910946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f282806d6f0 0x7f28280b15b0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f282806f590 tx=0x7f282402bda0 comp rx=0 tx=0).stop 2023-10-18T09:07:17.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.595+0000 7f282f2a3640 1 -- 172.21.15.40:0/1477910946 shutdown_connections 2023-10-18T09:07:17.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.595+0000 7f282f2a3640 1 --2- 172.21.15.40:0/1477910946 >> v2:172.21.15.144:3300/0 conn(0x7f28280c9f00 0x7f2828157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:17.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.595+0000 7f282f2a3640 1 --2- 172.21.15.40:0/1477910946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28280b0630 0x7f28280c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:17.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.595+0000 7f282f2a3640 1 --2- 172.21.15.40:0/1477910946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f282806d6f0 0x7f28280b15b0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:17.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.595+0000 7f282f2a3640 1 -- 172.21.15.40:0/1477910946 >> 172.21.15.40:0/1477910946 conn(0x7f2828069720 msgr2=0x7f2828069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:17.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.595+0000 7f282f2a3640 1 -- 172.21.15.40:0/1477910946 shutdown_connections 2023-10-18T09:07:17.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.595+0000 7f282f2a3640 1 -- 172.21.15.40:0/1477910946 wait complete. 2023-10-18T09:07:17.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.597+0000 7f282f2a3640 1 Processor -- start 2023-10-18T09:07:17.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.597+0000 7f282f2a3640 1 -- start start 2023-10-18T09:07:17.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.598+0000 7f282f2a3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f282806d6f0 0x7f282814bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:17.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.598+0000 7f282f2a3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28280b0630 0x7f2828157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:17.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.598+0000 7f282f2a3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28280c9f00 0x7f282815a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:17.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.598+0000 7f282f2a3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282814c900 con 0x7f282806d6f0 2023-10-18T09:07:17.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.598+0000 7f282f2a3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282814cbe0 con 0x7f28280b0630 2023-10-18T09:07:17.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.598+0000 7f282f2a3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282814cec0 con 0x7f28280c9f00 2023-10-18T09:07:17.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.598+0000 7f282e2a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f282806d6f0 0x7f282814bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:17.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.598+0000 7f282e2a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f282806d6f0 0x7f282814bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46350/0 (socket says 172.21.15.40:46350) 2023-10-18T09:07:17.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.598+0000 7f282e2a1640 1 -- 172.21.15.40:0/416217811 learned_addr learned my addr 172.21.15.40:0/416217811 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:17.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.599+0000 7f282e2a1640 1 -- 172.21.15.40:0/416217811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28280c9f00 msgr2=0x7f282815a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:17.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.599+0000 7f282eaa2640 1 --2- 172.21.15.40:0/416217811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28280c9f00 0x7f282815a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.599+0000 7f282daa0640 1 --2- 172.21.15.40:0/416217811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28280b0630 0x7f2828157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.599+0000 7f282e2a1640 1 --2- 172.21.15.40:0/416217811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28280c9f00 0x7f282815a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.599+0000 7f282e2a1640 1 -- 172.21.15.40:0/416217811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28280b0630 msgr2=0x7f2828157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.599+0000 7f282e2a1640 1 --2- 172.21.15.40:0/416217811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28280b0630 0x7f2828157d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.599+0000 7f282e2a1640 1 -- 172.21.15.40:0/416217811 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2824009560 con 0x7f282806d6f0 2023-10-18T09:07:17.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.599+0000 7f282eaa2640 1 --2- 172.21.15.40:0/416217811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28280c9f00 0x7f282815a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:17.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.599+0000 7f282e2a1640 1 --2- 172.21.15.40:0/416217811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f282806d6f0 0x7f282814bbb0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f2824009850 tx=0x7f282402dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:17.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.599+0000 7f281effd640 1 -- 172.21.15.40:0/416217811 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824005650 con 0x7f282806d6f0 2023-10-18T09:07:17.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.599+0000 7f281effd640 1 -- 172.21.15.40:0/416217811 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f28240038f0 con 0x7f282806d6f0 2023-10-18T09:07:17.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.599+0000 7f281effd640 1 -- 172.21.15.40:0/416217811 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824003ce0 con 0x7f282806d6f0 2023-10-18T09:07:17.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.599+0000 7f282f2a3640 1 -- 172.21.15.40:0/416217811 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282814d1a0 con 0x7f282806d6f0 2023-10-18T09:07:17.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.600+0000 7f282f2a3640 1 -- 172.21.15.40:0/416217811 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282814c0f0 con 0x7f282806d6f0 2023-10-18T09:07:17.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.601+0000 7f282f2a3640 1 -- 172.21.15.40:0/416217811 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f282814d6c0 con 0x7f282806d6f0 2023-10-18T09:07:17.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.602+0000 7f281effd640 1 -- 172.21.15.40:0/416217811 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f282403e450 con 0x7f282806d6f0 2023-10-18T09:07:17.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.603+0000 7f281effd640 1 --2- 172.21.15.40:0/416217811 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f280c06ae40 0x7f280c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.603+0000 7f281effd640 1 -- 172.21.15.40:0/416217811 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28240abfc0 con 0x7f282806d6f0 2023-10-18T09:07:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.603+0000 7f282daa0640 1 --2- 172.21.15.40:0/416217811 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f280c06ae40 0x7f280c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.605+0000 7f282daa0640 1 --2- 172.21.15.40:0/416217811 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f280c06ae40 0x7f280c06d300 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f28280ad4d0 tx=0x7f2818008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:17.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.607+0000 7f281effd640 1 -- 172.21.15.40:0/416217811 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2824077180 con 0x7f282806d6f0 2023-10-18T09:07:17.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.802+0000 7f282f2a3640 1 -- 172.21.15.40:0/416217811 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2828066e00 con 0x7f282806d6f0 2023-10-18T09:07:17.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.804+0000 7f281effd640 1 -- 172.21.15.40:0/416217811 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f282407a7a0 con 0x7f282806d6f0 2023-10-18T09:07:17.807 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:07:17.808 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":178,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:07:16.831884+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:07:14.660142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-18T09:07:15.853973+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:07:17.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.807+0000 7f282f2a3640 1 -- 172.21.15.40:0/416217811 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f280c06ae40 msgr2=0x7f280c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:17.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.807+0000 7f282f2a3640 1 --2- 172.21.15.40:0/416217811 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f280c06ae40 0x7f280c06d300 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f28280ad4d0 tx=0x7f2818008040 comp rx=0 tx=0).stop 2023-10-18T09:07:17.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.807+0000 7f282f2a3640 1 -- 172.21.15.40:0/416217811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f282806d6f0 msgr2=0x7f282814bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:17.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.807+0000 7f282f2a3640 1 --2- 172.21.15.40:0/416217811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f282806d6f0 0x7f282814bbb0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f2824009850 tx=0x7f282402dd70 comp rx=0 tx=0).stop 2023-10-18T09:07:17.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.807+0000 7f282f2a3640 1 -- 172.21.15.40:0/416217811 shutdown_connections 2023-10-18T09:07:17.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.807+0000 7f282f2a3640 1 --2- 172.21.15.40:0/416217811 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f280c06ae40 0x7f280c06d300 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:17.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.807+0000 7f282f2a3640 1 --2- 172.21.15.40:0/416217811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28280c9f00 0x7f282815a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:17.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.807+0000 7f282f2a3640 1 --2- 172.21.15.40:0/416217811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28280b0630 0x7f2828157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:17.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.807+0000 7f282f2a3640 1 --2- 172.21.15.40:0/416217811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f282806d6f0 0x7f282814bbb0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:17.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.807+0000 7f282f2a3640 1 -- 172.21.15.40:0/416217811 >> 172.21.15.40:0/416217811 conn(0x7f2828069720 msgr2=0x7f28281558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:17.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.807+0000 7f282f2a3640 1 -- 172.21.15.40:0/416217811 shutdown_connections 2023-10-18T09:07:17.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:17.807+0000 7f282f2a3640 1 -- 172.21.15.40:0/416217811 wait complete. 2023-10-18T09:07:17.828 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:07:18.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.061+0000 7f6e5759e640 1 -- 172.21.15.40:0/34891915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e44003f40 msgr2=0x7f6e440047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:18.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.061+0000 7f6e5759e640 1 --2- 172.21.15.40:0/34891915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e44003f40 0x7f6e440047c0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009ed0 tx=0x7f6e440057f0 comp rx=0 tx=0).stop 2023-10-18T09:07:18.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.061+0000 7f6e56d9d640 1 -- 172.21.15.40:0/34891915 reap_dead start 2023-10-18T09:07:18.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.061+0000 7f6e5759e640 1 -- 172.21.15.40:0/34891915 shutdown_connections 2023-10-18T09:07:18.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.061+0000 7f6e5759e640 1 -- 172.21.15.40:0/34891915 >> 172.21.15.40:0/34891915 conn(0x7f6e58069720 msgr2=0x7f6e58069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:18.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.061+0000 7f6e5759e640 1 -- 172.21.15.40:0/34891915 shutdown_connections 2023-10-18T09:07:18.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.061+0000 7f6e5759e640 1 -- 172.21.15.40:0/34891915 wait complete. 2023-10-18T09:07:18.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.063+0000 7f6e5759e640 1 Processor -- start 2023-10-18T09:07:18.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.063+0000 7f6e5759e640 1 -- start start 2023-10-18T09:07:18.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.064+0000 7f6e5759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e44003f40 0x7f6e580c27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:18.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.064+0000 7f6e5759e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e5813bbb0 0x7f6e580c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:18.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.064+0000 7f6e5759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e580ab290 0x7f6e580ab710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:18.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.064+0000 7f6e5759e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580c2310 con 0x7f6e580ab290 2023-10-18T09:07:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.064+0000 7f6e5759e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580abc50 con 0x7f6e44003f40 2023-10-18T09:07:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.064+0000 7f6e5759e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580ac1c0 con 0x7f6e5813bbb0 2023-10-18T09:07:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.064+0000 7f6e56d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e580ab290 0x7f6e580ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.064+0000 7f6e56d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e580ab290 0x7f6e580ab710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46372/0 (socket says 172.21.15.40:46372) 2023-10-18T09:07:18.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.064+0000 7f6e56d9d640 1 -- 172.21.15.40:0/2928991965 learned_addr learned my addr 172.21.15.40:0/2928991965 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:18.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.065+0000 7f6e5659c640 1 --2- 172.21.15.40:0/2928991965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e44003f40 0x7f6e580c27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:18.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.065+0000 7f6e55d9b640 1 --2- 172.21.15.40:0/2928991965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e5813bbb0 0x7f6e580c2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:18.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.065+0000 7f6e56d9d640 1 -- 172.21.15.40:0/2928991965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e5813bbb0 msgr2=0x7f6e580c2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:18.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.065+0000 7f6e56d9d640 1 --2- 172.21.15.40:0/2928991965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e5813bbb0 0x7f6e580c2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.065+0000 7f6e56d9d640 1 -- 172.21.15.40:0/2928991965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e44003f40 msgr2=0x7f6e580c27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.065+0000 7f6e56d9d640 1 --2- 172.21.15.40:0/2928991965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e44003f40 0x7f6e580c27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.065+0000 7f6e56d9d640 1 -- 172.21.15.40:0/2928991965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e580abee0 con 0x7f6e580ab290 2023-10-18T09:07:18.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.065+0000 7f6e56d9d640 1 --2- 172.21.15.40:0/2928991965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e580ab290 0x7f6e580ab710 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f6e5806c860 tx=0x7f6e50010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:18.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.065+0000 7f6e4effd640 1 -- 172.21.15.40:0/2928991965 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5000ec30 con 0x7f6e580ab290 2023-10-18T09:07:18.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.066+0000 7f6e4effd640 1 -- 172.21.15.40:0/2928991965 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6e5000ed90 con 0x7f6e580ab290 2023-10-18T09:07:18.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.066+0000 7f6e4effd640 1 -- 172.21.15.40:0/2928991965 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50005280 con 0x7f6e580ab290 2023-10-18T09:07:18.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.066+0000 7f6e5759e640 1 -- 172.21.15.40:0/2928991965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e581591f0 con 0x7f6e580ab290 2023-10-18T09:07:18.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.066+0000 7f6e5759e640 1 -- 172.21.15.40:0/2928991965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e581597b0 con 0x7f6e580ab290 2023-10-18T09:07:18.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.067+0000 7f6e5759e640 1 -- 172.21.15.40:0/2928991965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e58073250 con 0x7f6e580ab290 2023-10-18T09:07:18.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.068+0000 7f6e4effd640 1 -- 172.21.15.40:0/2928991965 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6e500053e0 con 0x7f6e580ab290 2023-10-18T09:07:18.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.069+0000 7f6e4effd640 1 --2- 172.21.15.40:0/2928991965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6e3806ae40 0x7f6e3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:18.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.069+0000 7f6e4effd640 1 -- 172.21.15.40:0/2928991965 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e50016070 con 0x7f6e580ab290 2023-10-18T09:07:18.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.070+0000 7f6e5659c640 1 --2- 172.21.15.40:0/2928991965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6e3806ae40 0x7f6e3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:18.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.071+0000 7f6e5659c640 1 --2- 172.21.15.40:0/2928991965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6e3806ae40 0x7f6e3806d300 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6e58073970 tx=0x7f6e480015c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:18.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.074+0000 7f6e4effd640 1 -- 172.21.15.40:0/2928991965 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e50054d50 con 0x7f6e580ab290 2023-10-18T09:07:18.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.270+0000 7f6e5759e640 1 -- 172.21.15.40:0/2928991965 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e5806d900 con 0x7f6e580ab290 2023-10-18T09:07:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.273+0000 7f6e4effd640 1 -- 172.21.15.40:0/2928991965 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f6e50058370 con 0x7f6e580ab290 2023-10-18T09:07:18.277 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:07:18.278 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":178,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:07:16.831884+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:07:14.660142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-18T09:07:15.853973+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:07:18.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.276+0000 7f6e5759e640 1 -- 172.21.15.40:0/2928991965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6e3806ae40 msgr2=0x7f6e3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:18.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.276+0000 7f6e5759e640 1 --2- 172.21.15.40:0/2928991965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6e3806ae40 0x7f6e3806d300 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6e58073970 tx=0x7f6e480015c0 comp rx=0 tx=0).stop 2023-10-18T09:07:18.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.276+0000 7f6e5759e640 1 -- 172.21.15.40:0/2928991965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e580ab290 msgr2=0x7f6e580ab710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:18.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.276+0000 7f6e5759e640 1 --2- 172.21.15.40:0/2928991965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e580ab290 0x7f6e580ab710 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f6e5806c860 tx=0x7f6e50010da0 comp rx=0 tx=0).stop 2023-10-18T09:07:18.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.276+0000 7f6e5759e640 1 -- 172.21.15.40:0/2928991965 shutdown_connections 2023-10-18T09:07:18.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.276+0000 7f6e5759e640 1 --2- 172.21.15.40:0/2928991965 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6e3806ae40 0x7f6e3806d300 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:18.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.276+0000 7f6e5759e640 1 --2- 172.21.15.40:0/2928991965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e580ab290 0x7f6e580ab710 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:18.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.276+0000 7f6e5759e640 1 --2- 172.21.15.40:0/2928991965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6e5813bbb0 0x7f6e580c2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:18.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.276+0000 7f6e5759e640 1 --2- 172.21.15.40:0/2928991965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6e44003f40 0x7f6e580c27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:18.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.276+0000 7f6e5759e640 1 -- 172.21.15.40:0/2928991965 >> 172.21.15.40:0/2928991965 conn(0x7f6e58069720 msgr2=0x7f6e58070140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:18.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.277+0000 7f6e5759e640 1 -- 172.21.15.40:0/2928991965 shutdown_connections 2023-10-18T09:07:18.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.277+0000 7f6e5759e640 1 -- 172.21.15.40:0/2928991965 wait complete. 2023-10-18T09:07:18.300 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T09:07:18.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.536+0000 7f1f849e9640 1 -- 172.21.15.40:0/3503098662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f8015af70 msgr2=0x7f1f8006c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:18.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.536+0000 7f1f849e9640 1 --2- 172.21.15.40:0/3503098662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f8015af70 0x7f1f8006c100 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f1f800aef60 tx=0x7f1f7402bda0 comp rx=0 tx=0).stop 2023-10-18T09:07:18.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.536+0000 7f1f849e9640 1 -- 172.21.15.40:0/3503098662 shutdown_connections 2023-10-18T09:07:18.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.536+0000 7f1f849e9640 1 --2- 172.21.15.40:0/3503098662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f8015af70 0x7f1f8006c100 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:18.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.536+0000 7f1f849e9640 1 --2- 172.21.15.40:0/3503098662 >> v2:172.21.15.144:3300/0 conn(0x7f1f800c9620 0x7f1f8006bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:18.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.536+0000 7f1f849e9640 1 -- 172.21.15.40:0/3503098662 >> 172.21.15.40:0/3503098662 conn(0x7f1f8006ca10 msgr2=0x7f1f8006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:18.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.537+0000 7f1f849e9640 1 -- 172.21.15.40:0/3503098662 shutdown_connections 2023-10-18T09:07:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.537+0000 7f1f849e9640 1 -- 172.21.15.40:0/3503098662 wait complete. 2023-10-18T09:07:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.538+0000 7f1f849e9640 1 Processor -- start 2023-10-18T09:07:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.538+0000 7f1f849e9640 1 -- start start 2023-10-18T09:07:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.539+0000 7f1f849e9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f80070b60 0x7f1f8014ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.539+0000 7f1f849e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f800c9620 0x7f1f801589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:18.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.539+0000 7f1f849e9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f8015af70 0x7f1f801464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.539+0000 7f1f849e9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8014f3c0 con 0x7f1f800c9620 2023-10-18T09:07:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.539+0000 7f1f849e9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8014f6a0 con 0x7f1f8015af70 2023-10-18T09:07:18.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.539+0000 7f1f849e9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8014f980 con 0x7f1f80070b60 2023-10-18T09:07:18.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.540+0000 7f1f7effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f800c9620 0x7f1f801589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.540+0000 7f1f7effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f800c9620 0x7f1f801589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46394/0 (socket says 172.21.15.40:46394) 2023-10-18T09:07:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.540+0000 7f1f7effd640 1 -- 172.21.15.40:0/197170461 learned_addr learned my addr 172.21.15.40:0/197170461 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.540+0000 7f1f7f7fe640 1 --2- 172.21.15.40:0/197170461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f80070b60 0x7f1f8014ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.540+0000 7f1f7effd640 1 -- 172.21.15.40:0/197170461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f80070b60 msgr2=0x7f1f8014ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:18.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.540+0000 7f1f7effd640 1 --2- 172.21.15.40:0/197170461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f80070b60 0x7f1f8014ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.540+0000 7f1f7effd640 1 -- 172.21.15.40:0/197170461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f8015af70 msgr2=0x7f1f801464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.540+0000 7f1f7effd640 1 --2- 172.21.15.40:0/197170461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f8015af70 0x7f1f801464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.540+0000 7f1f7effd640 1 -- 172.21.15.40:0/197170461 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f74009560 con 0x7f1f800c9620 2023-10-18T09:07:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.540+0000 7f1f7effd640 1 --2- 172.21.15.40:0/197170461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f800c9620 0x7f1f801589e0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009850 tx=0x7f1f7402dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:18.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.541+0000 7f1f67fff640 1 -- 172.21.15.40:0/197170461 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74003800 con 0x7f1f800c9620 2023-10-18T09:07:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.541+0000 7f1f67fff640 1 -- 172.21.15.40:0/197170461 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1f74003a50 con 0x7f1f800c9620 2023-10-18T09:07:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.541+0000 7f1f67fff640 1 -- 172.21.15.40:0/197170461 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74003e20 con 0x7f1f800c9620 2023-10-18T09:07:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.541+0000 7f1f849e9640 1 -- 172.21.15.40:0/197170461 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8014fc60 con 0x7f1f800c9620 2023-10-18T09:07:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.541+0000 7f1f849e9640 1 -- 172.21.15.40:0/197170461 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f801469f0 con 0x7f1f800c9620 2023-10-18T09:07:18.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.542+0000 7f1f849e9640 1 -- 172.21.15.40:0/197170461 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f80066e00 con 0x7f1f800c9620 2023-10-18T09:07:18.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.543+0000 7f1f67fff640 1 -- 172.21.15.40:0/197170461 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1f7403e450 con 0x7f1f800c9620 2023-10-18T09:07:18.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.544+0000 7f1f67fff640 1 --2- 172.21.15.40:0/197170461 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f6c06ae40 0x7f1f6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:18.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.544+0000 7f1f67fff640 1 -- 172.21.15.40:0/197170461 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f740ac2a0 con 0x7f1f800c9620 2023-10-18T09:07:18.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.545+0000 7f1f7f7fe640 1 --2- 172.21.15.40:0/197170461 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f6c06ae40 0x7f1f6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:18.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.546+0000 7f1f7f7fe640 1 --2- 172.21.15.40:0/197170461 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f6c06ae40 0x7f1f6c06d300 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1f70005fd0 tx=0x7f1f70005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:18.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.548+0000 7f1f67fff640 1 -- 172.21.15.40:0/197170461 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f74077220 con 0x7f1f800c9620 2023-10-18T09:07:18.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:18.781+0000 7f1f849e9640 1 -- 172.21.15.40:0/197170461 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1f800af7c0 con 0x7f1f800c9620 2023-10-18T09:07:19.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.404+0000 7f1f67fff640 1 -- 172.21.15.40:0/197170461 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1f740420b0 con 0x7f1f800c9620 2023-10-18T09:07:19.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.407+0000 7f1f849e9640 1 -- 172.21.15.40:0/197170461 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f6c06ae40 msgr2=0x7f1f6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:19.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.407+0000 7f1f849e9640 1 --2- 172.21.15.40:0/197170461 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f6c06ae40 0x7f1f6c06d300 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1f70005fd0 tx=0x7f1f70005850 comp rx=0 tx=0).stop 2023-10-18T09:07:19.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.407+0000 7f1f849e9640 1 -- 172.21.15.40:0/197170461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f800c9620 msgr2=0x7f1f801589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:19.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.407+0000 7f1f849e9640 1 --2- 172.21.15.40:0/197170461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f800c9620 0x7f1f801589e0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009850 tx=0x7f1f7402dad0 comp rx=0 tx=0).stop 2023-10-18T09:07:19.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.408+0000 7f1f849e9640 1 -- 172.21.15.40:0/197170461 shutdown_connections 2023-10-18T09:07:19.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.408+0000 7f1f849e9640 1 --2- 172.21.15.40:0/197170461 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1f6c06ae40 0x7f1f6c06d300 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:19.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.408+0000 7f1f849e9640 1 --2- 172.21.15.40:0/197170461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f8015af70 0x7f1f801464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:19.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.408+0000 7f1f849e9640 1 --2- 172.21.15.40:0/197170461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f800c9620 0x7f1f801589e0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:19.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.408+0000 7f1f849e9640 1 --2- 172.21.15.40:0/197170461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f80070b60 0x7f1f8014ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:19.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.408+0000 7f1f849e9640 1 -- 172.21.15.40:0/197170461 >> 172.21.15.40:0/197170461 conn(0x7f1f8006ca10 msgr2=0x7f1f80070f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:19.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.408+0000 7f1f849e9640 1 -- 172.21.15.40:0/197170461 shutdown_connections 2023-10-18T09:07:19.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.408+0000 7f1f849e9640 1 -- 172.21.15.40:0/197170461 wait complete. 2023-10-18T09:07:19.431 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T09:07:19.432 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T09:07:19.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.659+0000 7f0a51ff4640 1 -- 172.21.15.40:0/1670569148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a34004d00 msgr2=0x7f0a34005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:19.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.659+0000 7f0a51ff4640 1 --2- 172.21.15.40:0/1670569148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a34004d00 0x7f0a34005180 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009b60 tx=0x7f0a340057f0 comp rx=0 tx=0).stop 2023-10-18T09:07:19.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.659+0000 7f0a517f3640 1 -- 172.21.15.40:0/1670569148 reap_dead start 2023-10-18T09:07:19.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.659+0000 7f0a51ff4640 1 -- 172.21.15.40:0/1670569148 shutdown_connections 2023-10-18T09:07:19.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.659+0000 7f0a51ff4640 1 -- 172.21.15.40:0/1670569148 >> 172.21.15.40:0/1670569148 conn(0x7f0a4c06ca10 msgr2=0x7f0a4c06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:19.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.659+0000 7f0a51ff4640 1 -- 172.21.15.40:0/1670569148 shutdown_connections 2023-10-18T09:07:19.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.659+0000 7f0a51ff4640 1 -- 172.21.15.40:0/1670569148 wait complete. 2023-10-18T09:07:19.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.661+0000 7f0a51ff4640 1 Processor -- start 2023-10-18T09:07:19.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.661+0000 7f0a51ff4640 1 -- start start 2023-10-18T09:07:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.662+0000 7f0a51ff4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a34004d00 0x7f0a4c0c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.662+0000 7f0a51ff4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a4c0c6830 0x7f0a4c0c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.662+0000 7f0a51ff4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a4c0c84d0 0x7f0a4c0c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:19.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.662+0000 7f0a51ff4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0c8e90 con 0x7f0a4c0c6830 2023-10-18T09:07:19.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.662+0000 7f0a51ff4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c156000 con 0x7f0a34004d00 2023-10-18T09:07:19.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.662+0000 7f0a51ff4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c1562e0 con 0x7f0a4c0c84d0 2023-10-18T09:07:19.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.662+0000 7f0a43fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a4c0c6830 0x7f0a4c0c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:19.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.662+0000 7f0a43fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a4c0c6830 0x7f0a4c0c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46410/0 (socket says 172.21.15.40:46410) 2023-10-18T09:07:19.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.662+0000 7f0a43fff640 1 -- 172.21.15.40:0/3843974730 learned_addr learned my addr 172.21.15.40:0/3843974730 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:19.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.663+0000 7f0a50ff2640 1 --2- 172.21.15.40:0/3843974730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a34004d00 0x7f0a4c0c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:19.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.663+0000 7f0a43fff640 1 -- 172.21.15.40:0/3843974730 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a4c0c84d0 msgr2=0x7f0a4c0c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:19.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.663+0000 7f0a43fff640 1 --2- 172.21.15.40:0/3843974730 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a4c0c84d0 0x7f0a4c0c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:19.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.663+0000 7f0a43fff640 1 -- 172.21.15.40:0/3843974730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a34004d00 msgr2=0x7f0a4c0c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.663+0000 7f0a43fff640 1 --2- 172.21.15.40:0/3843974730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a34004d00 0x7f0a4c0c62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.663+0000 7f0a43fff640 1 -- 172.21.15.40:0/3843974730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c1565c0 con 0x7f0a4c0c6830 2023-10-18T09:07:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.663+0000 7f0a50ff2640 1 --2- 172.21.15.40:0/3843974730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a34004d00 0x7f0a4c0c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:19.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.663+0000 7f0a43fff640 1 --2- 172.21.15.40:0/3843974730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a4c0c6830 0x7f0a4c0c7f90 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c00e910 tx=0x7f0a3c00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.663+0000 7f0a417fa640 1 -- 172.21.15.40:0/3843974730 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c00cc60 con 0x7f0a4c0c6830 2023-10-18T09:07:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.663+0000 7f0a417fa640 1 -- 172.21.15.40:0/3843974730 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0a3c004030 con 0x7f0a4c0c6830 2023-10-18T09:07:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.663+0000 7f0a417fa640 1 -- 172.21.15.40:0/3843974730 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c004400 con 0x7f0a4c0c6830 2023-10-18T09:07:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.663+0000 7f0a51ff4640 1 -- 172.21.15.40:0/3843974730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a4c1568a0 con 0x7f0a4c0c6830 2023-10-18T09:07:19.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.664+0000 7f0a51ff4640 1 -- 172.21.15.40:0/3843974730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a4c1347d0 con 0x7f0a4c0c6830 2023-10-18T09:07:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.665+0000 7f0a51ff4640 1 -- 172.21.15.40:0/3843974730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a4c066e00 con 0x7f0a4c0c6830 2023-10-18T09:07:19.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.666+0000 7f0a417fa640 1 -- 172.21.15.40:0/3843974730 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0a3c0035f0 con 0x7f0a4c0c6830 2023-10-18T09:07:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.666+0000 7f0a417fa640 1 --2- 172.21.15.40:0/3843974730 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a3806ae70 0x7f0a3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.666+0000 7f0a417fa640 1 -- 172.21.15.40:0/3843974730 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a3c089fe0 con 0x7f0a4c0c6830 2023-10-18T09:07:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.667+0000 7f0a50ff2640 1 --2- 172.21.15.40:0/3843974730 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a3806ae70 0x7f0a3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:19.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.668+0000 7f0a50ff2640 1 --2- 172.21.15.40:0/3843974730 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a3806ae70 0x7f0a3806d330 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0a480015f0 tx=0x7f0a48009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:19.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.671+0000 7f0a417fa640 1 -- 172.21.15.40:0/3843974730 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0a3c054f70 con 0x7f0a4c0c6830 2023-10-18T09:07:19.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:19.908+0000 7f0a51ff4640 1 -- 172.21.15.40:0/3843974730 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0a4c14e900 con 0x7f0a4c0c6830 2023-10-18T09:07:20.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.415+0000 7f0a417fa640 1 -- 172.21.15.40:0/3843974730 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0a3c01f020 con 0x7f0a4c0c6830 2023-10-18T09:07:20.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.419+0000 7f0a51ff4640 1 -- 172.21.15.40:0/3843974730 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a3806ae70 msgr2=0x7f0a3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:20.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.419+0000 7f0a51ff4640 1 --2- 172.21.15.40:0/3843974730 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a3806ae70 0x7f0a3806d330 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0a480015f0 tx=0x7f0a48009040 comp rx=0 tx=0).stop 2023-10-18T09:07:20.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.419+0000 7f0a51ff4640 1 -- 172.21.15.40:0/3843974730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a4c0c6830 msgr2=0x7f0a4c0c7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:20.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.419+0000 7f0a51ff4640 1 --2- 172.21.15.40:0/3843974730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a4c0c6830 0x7f0a4c0c7f90 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c00e910 tx=0x7f0a3c00edd0 comp rx=0 tx=0).stop 2023-10-18T09:07:20.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.419+0000 7f0a51ff4640 1 -- 172.21.15.40:0/3843974730 shutdown_connections 2023-10-18T09:07:20.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.419+0000 7f0a51ff4640 1 --2- 172.21.15.40:0/3843974730 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0a3806ae70 0x7f0a3806d330 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:20.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.419+0000 7f0a51ff4640 1 --2- 172.21.15.40:0/3843974730 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a4c0c84d0 0x7f0a4c0c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:20.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.419+0000 7f0a51ff4640 1 --2- 172.21.15.40:0/3843974730 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a4c0c6830 0x7f0a4c0c7f90 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:20.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.419+0000 7f0a51ff4640 1 --2- 172.21.15.40:0/3843974730 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a34004d00 0x7f0a4c0c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:20.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.419+0000 7f0a51ff4640 1 -- 172.21.15.40:0/3843974730 >> 172.21.15.40:0/3843974730 conn(0x7f0a4c06ca10 msgr2=0x7f0a4c06bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:20.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.419+0000 7f0a51ff4640 1 -- 172.21.15.40:0/3843974730 shutdown_connections 2023-10-18T09:07:20.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.419+0000 7f0a51ff4640 1 -- 172.21.15.40:0/3843974730 wait complete. 2023-10-18T09:07:20.443 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T09:07:20.443 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:07:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.665+0000 7fa5028fc640 1 -- 172.21.15.40:0/1396404046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4fc0ca0d0 msgr2=0x7fa4fc157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.665+0000 7fa5028fc640 1 --2- 172.21.15.40:0/1396404046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4fc0ca0d0 0x7fa4fc157d40 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc06aeb0 tx=0x7fa4f402bd70 comp rx=0 tx=0).stop 2023-10-18T09:07:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.665+0000 7fa5028fc640 1 -- 172.21.15.40:0/1396404046 shutdown_connections 2023-10-18T09:07:20.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.665+0000 7fa5028fc640 1 --2- 172.21.15.40:0/1396404046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4fc0ca0d0 0x7fa4fc157d40 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:20.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.665+0000 7fa5028fc640 1 --2- 172.21.15.40:0/1396404046 >> v2:172.21.15.144:3300/0 conn(0x7fa4fc06d6f0 0x7fa4fc0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:20.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.665+0000 7fa5028fc640 1 -- 172.21.15.40:0/1396404046 >> 172.21.15.40:0/1396404046 conn(0x7fa4fc069720 msgr2=0x7fa4fc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:20.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.665+0000 7fa5028fc640 1 -- 172.21.15.40:0/1396404046 shutdown_connections 2023-10-18T09:07:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.665+0000 7fa5028fc640 1 -- 172.21.15.40:0/1396404046 wait complete. 2023-10-18T09:07:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.667+0000 7fa5028fc640 1 Processor -- start 2023-10-18T09:07:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.667+0000 7fa5028fc640 1 -- start start 2023-10-18T09:07:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.668+0000 7fa5028fc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4fc06d6f0 0x7fa4fc0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.668+0000 7fa5028fc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa4fc0b0630 0x7fa4fc0adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.668+0000 7fa5028fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4fc0ca0d0 0x7fa4fc142870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.668+0000 7fa5028fc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc0af9e0 con 0x7fa4fc0ca0d0 2023-10-18T09:07:20.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.668+0000 7fa5028fc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc0ae020 con 0x7fa4fc06d6f0 2023-10-18T09:07:20.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.668+0000 7fa5028fc640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc151b00 con 0x7fa4fc0b0630 2023-10-18T09:07:20.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.669+0000 7fa5020fb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4fc0ca0d0 0x7fa4fc142870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:20.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.669+0000 7fa5020fb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4fc0ca0d0 0x7fa4fc142870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46418/0 (socket says 172.21.15.40:46418) 2023-10-18T09:07:20.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.669+0000 7fa5020fb640 1 -- 172.21.15.40:0/3936468942 learned_addr learned my addr 172.21.15.40:0/3936468942 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.669+0000 7fa5018fa640 1 --2- 172.21.15.40:0/3936468942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4fc06d6f0 0x7fa4fc0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.669+0000 7fa5020fb640 1 -- 172.21.15.40:0/3936468942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa4fc0b0630 msgr2=0x7fa4fc0adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.669+0000 7fa5020fb640 1 --2- 172.21.15.40:0/3936468942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa4fc0b0630 0x7fa4fc0adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.669+0000 7fa5020fb640 1 -- 172.21.15.40:0/3936468942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4fc06d6f0 msgr2=0x7fa4fc0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.669+0000 7fa5020fb640 1 --2- 172.21.15.40:0/3936468942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4fc06d6f0 0x7fa4fc0af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.669+0000 7fa5020fb640 1 -- 172.21.15.40:0/3936468942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f4009490 con 0x7fa4fc0ca0d0 2023-10-18T09:07:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.669+0000 7fa5020fb640 1 --2- 172.21.15.40:0/3936468942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4fc0ca0d0 0x7fa4fc142870 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009920 tx=0x7fa4f40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.670+0000 7fa4f27fc640 1 -- 172.21.15.40:0/3936468942 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4003ea0 con 0x7fa4fc0ca0d0 2023-10-18T09:07:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.670+0000 7fa4f27fc640 1 -- 172.21.15.40:0/3936468942 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa4f4005020 con 0x7fa4fc0ca0d0 2023-10-18T09:07:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.670+0000 7fa4f27fc640 1 -- 172.21.15.40:0/3936468942 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4005430 con 0x7fa4fc0ca0d0 2023-10-18T09:07:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.670+0000 7fa5028fc640 1 -- 172.21.15.40:0/3936468942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4fc151de0 con 0x7fa4fc0ca0d0 2023-10-18T09:07:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.670+0000 7fa5028fc640 1 -- 172.21.15.40:0/3936468942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4fc1523a0 con 0x7fa4fc0ca0d0 2023-10-18T09:07:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.671+0000 7fa5028fc640 1 -- 172.21.15.40:0/3936468942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4fc066e00 con 0x7fa4fc0ca0d0 2023-10-18T09:07:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.672+0000 7fa4f27fc640 1 -- 172.21.15.40:0/3936468942 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa4f40047a0 con 0x7fa4fc0ca0d0 2023-10-18T09:07:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.674+0000 7fa4f27fc640 1 --2- 172.21.15.40:0/3936468942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa4e006ae40 0x7fa4e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.674+0000 7fa4f27fc640 1 -- 172.21.15.40:0/3936468942 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4f40abac0 con 0x7fa4fc0ca0d0 2023-10-18T09:07:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.674+0000 7fa5018fa640 1 --2- 172.21.15.40:0/3936468942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa4e006ae40 0x7fa4e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:20.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.675+0000 7fa5018fa640 1 --2- 172.21.15.40:0/3936468942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa4e006ae40 0x7fa4e006d300 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8005fd0 tx=0x7fa4f8009870 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:20.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.678+0000 7fa4f27fc640 1 -- 172.21.15.40:0/3936468942 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa4f4076b60 con 0x7fa4fc0ca0d0 2023-10-18T09:07:20.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.912+0000 7fa5028fc640 1 -- 172.21.15.40:0/3936468942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4fc0caf90 con 0x7fa4fc0ca0d0 2023-10-18T09:07:20.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.913+0000 7fa4f27fc640 1 -- 172.21.15.40:0/3936468942 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fa4f4042090 con 0x7fa4fc0ca0d0 2023-10-18T09:07:20.917 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:07:20.917 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6823,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3592705338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3592705338},{"type":"v1","addr":"172.21.15.40:6835","nonce":3592705338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":6829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1076939595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1076939595},{"type":"v1","addr":"172.21.15.144:6836","nonce":1076939595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":6832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2483821333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2483821333},{"type":"v1","addr":"172.21.15.144:6837","nonce":2483821333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:07:16.832438+0000","modified":"2023-10-18T09:07:19.910479+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8826},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8826":{"gid":8826,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/137474609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":137474609},{"type":"v1","addr":"172.21.15.40:6834","nonce":137474609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-18T09:07:20.917 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 82 2023-10-18T09:07:20.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.915+0000 7fa5028fc640 1 -- 172.21.15.40:0/3936468942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa4e006ae40 msgr2=0x7fa4e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:20.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.915+0000 7fa5028fc640 1 --2- 172.21.15.40:0/3936468942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa4e006ae40 0x7fa4e006d300 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8005fd0 tx=0x7fa4f8009870 comp rx=0 tx=0).stop 2023-10-18T09:07:20.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.915+0000 7fa5028fc640 1 -- 172.21.15.40:0/3936468942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4fc0ca0d0 msgr2=0x7fa4fc142870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:20.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.915+0000 7fa5028fc640 1 --2- 172.21.15.40:0/3936468942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4fc0ca0d0 0x7fa4fc142870 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009920 tx=0x7fa4f40038a0 comp rx=0 tx=0).stop 2023-10-18T09:07:20.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.916+0000 7fa5028fc640 1 -- 172.21.15.40:0/3936468942 shutdown_connections 2023-10-18T09:07:20.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.916+0000 7fa5028fc640 1 --2- 172.21.15.40:0/3936468942 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa4e006ae40 0x7fa4e006d300 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:20.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.916+0000 7fa5028fc640 1 --2- 172.21.15.40:0/3936468942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4fc0ca0d0 0x7fa4fc142870 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:20.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.916+0000 7fa5028fc640 1 --2- 172.21.15.40:0/3936468942 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa4fc0b0630 0x7fa4fc0adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:20.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.916+0000 7fa5028fc640 1 --2- 172.21.15.40:0/3936468942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4fc06d6f0 0x7fa4fc0af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:20.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.916+0000 7fa5028fc640 1 -- 172.21.15.40:0/3936468942 >> 172.21.15.40:0/3936468942 conn(0x7fa4fc069720 msgr2=0x7fa4fc153c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:20.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.916+0000 7fa5028fc640 1 -- 172.21.15.40:0/3936468942 shutdown_connections 2023-10-18T09:07:20.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:20.916+0000 7fa5028fc640 1 -- 172.21.15.40:0/3936468942 wait complete. 2023-10-18T09:07:20.940 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:07:21.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.161+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2738597687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f80b0630 msgr2=0x7ff9f80c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:21.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.161+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/2738597687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f80b0630 0x7ff9f80c99c0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009830 tx=0x7ff9e002bd60 comp rx=0 tx=0).stop 2023-10-18T09:07:21.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.162+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2738597687 shutdown_connections 2023-10-18T09:07:21.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.162+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/2738597687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f80b0630 0x7ff9f80c99c0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:21.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.162+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/2738597687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f806d6f0 0x7ff9f80b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:21.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.162+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2738597687 >> 172.21.15.40:0/2738597687 conn(0x7ff9f8069720 msgr2=0x7ff9f8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:21.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.162+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2738597687 shutdown_connections 2023-10-18T09:07:21.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.162+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2738597687 wait complete. 2023-10-18T09:07:21.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.163+0000 7ff9f7fff640 1 Processor -- start 2023-10-18T09:07:21.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.164+0000 7ff9f7fff640 1 -- start start 2023-10-18T09:07:21.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.165+0000 7ff9f7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9f806d6f0 0x7ff9f8156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:21.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.165+0000 7ff9f7fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f80b0630 0x7ff9f81573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:21.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.165+0000 7ff9f7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f80c9f00 0x7ff9f81f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:21.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.165+0000 7ff9f7fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f81578f0 con 0x7ff9f80c9f00 2023-10-18T09:07:21.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.165+0000 7ff9f7fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f8142b00 con 0x7ff9f806d6f0 2023-10-18T09:07:21.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.165+0000 7ff9f7fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f812ebb0 con 0x7ff9f80b0630 2023-10-18T09:07:21.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.165+0000 7ff9f77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f80c9f00 0x7ff9f81f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:21.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.165+0000 7ff9f77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f80c9f00 0x7ff9f81f3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46434/0 (socket says 172.21.15.40:46434) 2023-10-18T09:07:21.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.165+0000 7ff9f77fe640 1 -- 172.21.15.40:0/2870217596 learned_addr learned my addr 172.21.15.40:0/2870217596 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:21.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.166+0000 7ff9ee7fc640 1 --2- 172.21.15.40:0/2870217596 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f80b0630 0x7ff9f81573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:21.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.166+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/2870217596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9f806d6f0 0x7ff9f8156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:21.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.166+0000 7ff9f77fe640 1 -- 172.21.15.40:0/2870217596 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f80b0630 msgr2=0x7ff9f81573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:21.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.166+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/2870217596 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f80b0630 0x7ff9f81573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:21.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.166+0000 7ff9f77fe640 1 -- 172.21.15.40:0/2870217596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9f806d6f0 msgr2=0x7ff9f8156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:21.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.166+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/2870217596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9f806d6f0 0x7ff9f8156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:21.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.166+0000 7ff9f77fe640 1 -- 172.21.15.40:0/2870217596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e0009490 con 0x7ff9f80c9f00 2023-10-18T09:07:21.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.166+0000 7ff9ee7fc640 1 --2- 172.21.15.40:0/2870217596 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f80b0630 0x7ff9f81573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:21.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.166+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/2870217596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9f806d6f0 0x7ff9f8156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.166+0000 7ff9f77fe640 1 --2- 172.21.15.40:0/2870217596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f80c9f00 0x7ff9f81f3030 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000c740 tx=0x7ff9f000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.166+0000 7ff9f4ff9640 1 -- 172.21.15.40:0/2870217596 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0003da0 con 0x7ff9f80c9f00 2023-10-18T09:07:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.166+0000 7ff9f4ff9640 1 -- 172.21.15.40:0/2870217596 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff9f000ce50 con 0x7ff9f80c9f00 2023-10-18T09:07:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.166+0000 7ff9f4ff9640 1 -- 172.21.15.40:0/2870217596 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00052b0 con 0x7ff9f80c9f00 2023-10-18T09:07:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.166+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2870217596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f8156700 con 0x7ff9f80c9f00 2023-10-18T09:07:21.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.167+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2870217596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f8156cc0 con 0x7ff9f80c9f00 2023-10-18T09:07:21.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.168+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2870217596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f812ed40 con 0x7ff9f80c9f00 2023-10-18T09:07:21.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.171+0000 7ff9f4ff9640 1 -- 172.21.15.40:0/2870217596 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff9f0005410 con 0x7ff9f80c9f00 2023-10-18T09:07:21.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.173+0000 7ff9f4ff9640 1 --2- 172.21.15.40:0/2870217596 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9d806ae40 0x7ff9d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.173+0000 7ff9f4ff9640 1 -- 172.21.15.40:0/2870217596 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9f0089540 con 0x7ff9f80c9f00 2023-10-18T09:07:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.174+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/2870217596 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9d806ae40 0x7ff9d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:21.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.175+0000 7ff9f6ffd640 1 --2- 172.21.15.40:0/2870217596 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9d806ae40 0x7ff9d806d300 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009b40 tx=0x7ff9e8005e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:21.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.177+0000 7ff9f4ff9640 1 -- 172.21.15.40:0/2870217596 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff9f00545d0 con 0x7ff9f80c9f00 2023-10-18T09:07:21.316 INFO:tasks.ceph.mds.c.smithi040.stderr:2023-10-18T09:07:21.312+0000 7f2641eea640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T09:07:21.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.371+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2870217596 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9f81641b0 con 0x7ff9f80c9f00 2023-10-18T09:07:21.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.375+0000 7ff9f4ff9640 1 -- 172.21.15.40:0/2870217596 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v179) v1 ==== 74+0+16592 (secure 0 0 0) 0x7ff9f0017020 con 0x7ff9f80c9f00 2023-10-18T09:07:21.378 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:07:21.379 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":179,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:07:18.360351+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:07:14.660142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-18T09:07:15.853973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:07:21.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.377+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2870217596 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9d806ae40 msgr2=0x7ff9d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:21.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.377+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/2870217596 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9d806ae40 0x7ff9d806d300 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009b40 tx=0x7ff9e8005e50 comp rx=0 tx=0).stop 2023-10-18T09:07:21.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.377+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2870217596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f80c9f00 msgr2=0x7ff9f81f3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:21.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.377+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/2870217596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f80c9f00 0x7ff9f81f3030 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000c740 tx=0x7ff9f000cc00 comp rx=0 tx=0).stop 2023-10-18T09:07:21.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.378+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2870217596 shutdown_connections 2023-10-18T09:07:21.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.378+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/2870217596 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9d806ae40 0x7ff9d806d300 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:21.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.378+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/2870217596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f80c9f00 0x7ff9f81f3030 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:21.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.378+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/2870217596 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f80b0630 0x7ff9f81573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:21.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.378+0000 7ff9f7fff640 1 --2- 172.21.15.40:0/2870217596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9f806d6f0 0x7ff9f8156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:21.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.378+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2870217596 >> 172.21.15.40:0/2870217596 conn(0x7ff9f8069720 msgr2=0x7ff9f81549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:21.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.378+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2870217596 shutdown_connections 2023-10-18T09:07:21.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.378+0000 7ff9f7fff640 1 -- 172.21.15.40:0/2870217596 wait complete. 2023-10-18T09:07:21.401 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T09:07:21.402 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:07:21.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.636+0000 7fe46e328640 1 -- 172.21.15.40:0/1611085171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4540047c0 msgr2=0x7fe454004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.636+0000 7fe46e328640 1 --2- 172.21.15.40:0/1611085171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4540047c0 0x7fe454004c40 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fe45800bb20 tx=0x7fe4540057f0 comp rx=0 tx=0).stop 2023-10-18T09:07:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.636+0000 7fe46db27640 1 -- 172.21.15.40:0/1611085171 reap_dead start 2023-10-18T09:07:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.636+0000 7fe46e328640 1 -- 172.21.15.40:0/1611085171 shutdown_connections 2023-10-18T09:07:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.636+0000 7fe46e328640 1 -- 172.21.15.40:0/1611085171 >> 172.21.15.40:0/1611085171 conn(0x7fe468069720 msgr2=0x7fe468069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:21.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.636+0000 7fe46e328640 1 -- 172.21.15.40:0/1611085171 shutdown_connections 2023-10-18T09:07:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.636+0000 7fe46e328640 1 -- 172.21.15.40:0/1611085171 wait complete. 2023-10-18T09:07:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.638+0000 7fe46e328640 1 Processor -- start 2023-10-18T09:07:21.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.638+0000 7fe46e328640 1 -- start start 2023-10-18T09:07:21.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.639+0000 7fe46e328640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4540047c0 0x7fe468148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.639+0000 7fe46e328640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe468154f20 0x7fe468153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.639+0000 7fe46e328640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4681553a0 0x7fe468153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.639+0000 7fe46e328640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe468156870 con 0x7fe4540047c0 2023-10-18T09:07:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.639+0000 7fe46e328640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe468154a90 con 0x7fe4681553a0 2023-10-18T09:07:21.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.639+0000 7fe46e328640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe468153e80 con 0x7fe468154f20 2023-10-18T09:07:21.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.639+0000 7fe46d326640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4540047c0 0x7fe468148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:21.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.639+0000 7fe46d326640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4540047c0 0x7fe468148c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46454/0 (socket says 172.21.15.40:46454) 2023-10-18T09:07:21.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.639+0000 7fe46d326640 1 -- 172.21.15.40:0/2541468032 learned_addr learned my addr 172.21.15.40:0/2541468032 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:21.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.640+0000 7fe46cb25640 1 --2- 172.21.15.40:0/2541468032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe468154f20 0x7fe468153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:21.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.640+0000 7fe46db27640 1 --2- 172.21.15.40:0/2541468032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4681553a0 0x7fe468153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:21.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.640+0000 7fe46d326640 1 -- 172.21.15.40:0/2541468032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe468154f20 msgr2=0x7fe468153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:21.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.640+0000 7fe46d326640 1 --2- 172.21.15.40:0/2541468032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe468154f20 0x7fe468153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:21.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.640+0000 7fe46d326640 1 -- 172.21.15.40:0/2541468032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4681553a0 msgr2=0x7fe468153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:21.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.640+0000 7fe46d326640 1 --2- 172.21.15.40:0/2541468032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4681553a0 0x7fe468153850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:21.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.640+0000 7fe46d326640 1 -- 172.21.15.40:0/2541468032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe468154160 con 0x7fe4540047c0 2023-10-18T09:07:21.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.640+0000 7fe46d326640 1 --2- 172.21.15.40:0/2541468032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4540047c0 0x7fe468148c80 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fe46400e600 tx=0x7fe46400eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:21.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.640+0000 7fe45dffb640 1 -- 172.21.15.40:0/2541468032 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46400c9a0 con 0x7fe4540047c0 2023-10-18T09:07:21.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.640+0000 7fe45dffb640 1 -- 172.21.15.40:0/2541468032 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe46400cbf0 con 0x7fe4540047c0 2023-10-18T09:07:21.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.640+0000 7fe45dffb640 1 -- 172.21.15.40:0/2541468032 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464011460 con 0x7fe4540047c0 2023-10-18T09:07:21.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.640+0000 7fe46e328640 1 -- 172.21.15.40:0/2541468032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe468154440 con 0x7fe4540047c0 2023-10-18T09:07:21.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.640+0000 7fe46e328640 1 -- 172.21.15.40:0/2541468032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46815afb0 con 0x7fe4540047c0 2023-10-18T09:07:21.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.642+0000 7fe46e328640 1 -- 172.21.15.40:0/2541468032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe468066e00 con 0x7fe4540047c0 2023-10-18T09:07:21.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.643+0000 7fe45dffb640 1 -- 172.21.15.40:0/2541468032 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe4640115c0 con 0x7fe4540047c0 2023-10-18T09:07:21.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.644+0000 7fe45dffb640 1 --2- 172.21.15.40:0/2541468032 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe44c06ae40 0x7fe44c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:21.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.644+0000 7fe45dffb640 1 -- 172.21.15.40:0/2541468032 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe464089f00 con 0x7fe4540047c0 2023-10-18T09:07:21.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.645+0000 7fe46cb25640 1 --2- 172.21.15.40:0/2541468032 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe44c06ae40 0x7fe44c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:21.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.646+0000 7fe46cb25640 1 --2- 172.21.15.40:0/2541468032 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe44c06ae40 0x7fe44c06d300 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe458037000 tx=0x7fe45802e040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:21.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.648+0000 7fe45dffb640 1 -- 172.21.15.40:0/2541468032 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe464054fa0 con 0x7fe4540047c0 2023-10-18T09:07:21.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.841+0000 7fe46e328640 1 -- 172.21.15.40:0/2541468032 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe468154bd0 con 0x7fe44c06ae40 2023-10-18T09:07:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.865+0000 7fe45dffb640 1 -- 172.21.15.40:0/2541468032 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281945 (secure 0 0 0) 0x7fe45803f480 con 0x7fe44c06ae40 2023-10-18T09:07:21.872 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:07:21.874 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:07:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.871+0000 7fe46e328640 1 -- 172.21.15.40:0/2541468032 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe44c06ae40 msgr2=0x7fe44c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.871+0000 7fe46e328640 1 --2- 172.21.15.40:0/2541468032 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe44c06ae40 0x7fe44c06d300 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe458037000 tx=0x7fe45802e040 comp rx=0 tx=0).stop 2023-10-18T09:07:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.871+0000 7fe46e328640 1 -- 172.21.15.40:0/2541468032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4540047c0 msgr2=0x7fe468148c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.871+0000 7fe46e328640 1 --2- 172.21.15.40:0/2541468032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4540047c0 0x7fe468148c80 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fe46400e600 tx=0x7fe46400eac0 comp rx=0 tx=0).stop 2023-10-18T09:07:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.871+0000 7fe46e328640 1 -- 172.21.15.40:0/2541468032 shutdown_connections 2023-10-18T09:07:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.871+0000 7fe46e328640 1 --2- 172.21.15.40:0/2541468032 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe44c06ae40 0x7fe44c06d300 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.871+0000 7fe46e328640 1 --2- 172.21.15.40:0/2541468032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4681553a0 0x7fe468153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.871+0000 7fe46e328640 1 --2- 172.21.15.40:0/2541468032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe468154f20 0x7fe468153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.871+0000 7fe46e328640 1 --2- 172.21.15.40:0/2541468032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4540047c0 0x7fe468148c80 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:21.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.871+0000 7fe46e328640 1 -- 172.21.15.40:0/2541468032 >> 172.21.15.40:0/2541468032 conn(0x7fe468069720 msgr2=0x7fe46814adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.871+0000 7fe46e328640 1 -- 172.21.15.40:0/2541468032 shutdown_connections 2023-10-18T09:07:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:21.872+0000 7fe46e328640 1 -- 172.21.15.40:0/2541468032 wait complete. 2023-10-18T09:07:21.895 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":541,"stamp":"2023-10-18T09:07:21.163496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9062,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553437,"statfs":{"total":1600000000,"available":1590720485,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.827232"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.380074+0000","last_change":"2023-10-18T09:07:17.379695+0000","last_active":"2023-10-18T09:07:17.380074+0000","last_peered":"2023-10-18T09:07:17.380074+0000","last_clean":"2023-10-18T09:07:17.380074+0000","last_became_active":"2023-10-18T09:07:17.379366+0000","last_became_peered":"2023-10-18T09:07:17.379366+0000","last_unstale":"2023-10-18T09:07:17.380074+0000","last_undegraded":"2023-10-18T09:07:17.380074+0000","last_fullsized":"2023-10-18T09:07:17.380074+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:06:46.902419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365435+0000","last_change":"2023-10-18T09:07:17.385274+0000","last_active":"2023-10-18T09:07:18.365435+0000","last_peered":"2023-10-18T09:07:18.365435+0000","last_clean":"2023-10-18T09:07:18.365435+0000","last_became_active":"2023-10-18T09:07:17.385043+0000","last_became_peered":"2023-10-18T09:07:17.385043+0000","last_unstale":"2023-10-18T09:07:18.365435+0000","last_undegraded":"2023-10-18T09:07:18.365435+0000","last_fullsized":"2023-10-18T09:07:18.365435+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:10.523692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370896+0000","last_change":"2023-10-18T09:07:17.395122+0000","last_active":"2023-10-18T09:07:18.370896+0000","last_peered":"2023-10-18T09:07:18.370896+0000","last_clean":"2023-10-18T09:07:18.370896+0000","last_became_active":"2023-10-18T09:07:17.392503+0000","last_became_peered":"2023-10-18T09:07:17.392503+0000","last_unstale":"2023-10-18T09:07:18.370896+0000","last_undegraded":"2023-10-18T09:07:18.370896+0000","last_fullsized":"2023-10-18T09:07:18.370896+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:21:20.537938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.392796+0000","last_change":"2023-10-18T09:07:17.392552+0000","last_active":"2023-10-18T09:07:17.392796+0000","last_peered":"2023-10-18T09:07:17.392796+0000","last_clean":"2023-10-18T09:07:17.392796+0000","last_became_active":"2023-10-18T09:07:17.392049+0000","last_became_peered":"2023-10-18T09:07:17.392049+0000","last_unstale":"2023-10-18T09:07:17.392796+0000","last_undegraded":"2023-10-18T09:07:17.392796+0000","last_fullsized":"2023-10-18T09:07:17.392796+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:52:34.109568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742910+0000","last_change":"2023-10-18T09:07:17.399034+0000","last_active":"2023-10-18T09:07:18.742910+0000","last_peered":"2023-10-18T09:07:18.742910+0000","last_clean":"2023-10-18T09:07:18.742910+0000","last_became_active":"2023-10-18T09:07:17.398592+0000","last_became_peered":"2023-10-18T09:07:17.398592+0000","last_unstale":"2023-10-18T09:07:18.742910+0000","last_undegraded":"2023-10-18T09:07:18.742910+0000","last_fullsized":"2023-10-18T09:07:18.742910+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:09:18.970120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.395378+0000","last_change":"2023-10-18T09:07:17.395304+0000","last_active":"2023-10-18T09:07:17.395378+0000","last_peered":"2023-10-18T09:07:17.395378+0000","last_clean":"2023-10-18T09:07:17.395378+0000","last_became_active":"2023-10-18T09:07:17.394753+0000","last_became_peered":"2023-10-18T09:07:17.394753+0000","last_unstale":"2023-10-18T09:07:17.395378+0000","last_undegraded":"2023-10-18T09:07:17.395378+0000","last_fullsized":"2023-10-18T09:07:17.395378+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:07.611935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.741664+0000","last_change":"2023-10-18T09:07:17.387822+0000","last_active":"2023-10-18T09:07:18.741664+0000","last_peered":"2023-10-18T09:07:18.741664+0000","last_clean":"2023-10-18T09:07:18.741664+0000","last_became_active":"2023-10-18T09:07:17.387431+0000","last_became_peered":"2023-10-18T09:07:17.387431+0000","last_unstale":"2023-10-18T09:07:18.741664+0000","last_undegraded":"2023-10-18T09:07:18.741664+0000","last_fullsized":"2023-10-18T09:07:18.741664+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:41:56.193471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.390476+0000","last_change":"2023-10-18T09:07:17.390398+0000","last_active":"2023-10-18T09:07:17.390476+0000","last_peered":"2023-10-18T09:07:17.390476+0000","last_clean":"2023-10-18T09:07:17.390476+0000","last_became_active":"2023-10-18T09:07:17.390168+0000","last_became_peered":"2023-10-18T09:07:17.390168+0000","last_unstale":"2023-10-18T09:07:17.390476+0000","last_undegraded":"2023-10-18T09:07:17.390476+0000","last_fullsized":"2023-10-18T09:07:17.390476+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:46:23.898025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365283+0000","last_change":"2023-10-18T09:07:17.367980+0000","last_active":"2023-10-18T09:07:18.365283+0000","last_peered":"2023-10-18T09:07:18.365283+0000","last_clean":"2023-10-18T09:07:18.365283+0000","last_became_active":"2023-10-18T09:07:17.367695+0000","last_became_peered":"2023-10-18T09:07:17.367695+0000","last_unstale":"2023-10-18T09:07:18.365283+0000","last_undegraded":"2023-10-18T09:07:18.365283+0000","last_fullsized":"2023-10-18T09:07:18.365283+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:54.405679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.368755+0000","last_change":"2023-10-18T09:07:17.392047+0000","last_active":"2023-10-18T09:07:18.368755+0000","last_peered":"2023-10-18T09:07:18.368755+0000","last_clean":"2023-10-18T09:07:18.368755+0000","last_became_active":"2023-10-18T09:07:17.391271+0000","last_became_peered":"2023-10-18T09:07:17.391271+0000","last_unstale":"2023-10-18T09:07:18.368755+0000","last_undegraded":"2023-10-18T09:07:18.368755+0000","last_fullsized":"2023-10-18T09:07:18.368755+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:08.967354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.389830+0000","last_change":"2023-10-18T09:07:17.389715+0000","last_active":"2023-10-18T09:07:17.389830+0000","last_peered":"2023-10-18T09:07:17.389830+0000","last_clean":"2023-10-18T09:07:17.389830+0000","last_became_active":"2023-10-18T09:07:17.389450+0000","last_became_peered":"2023-10-18T09:07:17.389450+0000","last_unstale":"2023-10-18T09:07:17.389830+0000","last_undegraded":"2023-10-18T09:07:17.389830+0000","last_fullsized":"2023-10-18T09:07:17.389830+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:20:23.944169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.368289+0000","last_change":"2023-10-18T09:07:17.398342+0000","last_active":"2023-10-18T09:07:18.368289+0000","last_peered":"2023-10-18T09:07:18.368289+0000","last_clean":"2023-10-18T09:07:18.368289+0000","last_became_active":"2023-10-18T09:07:17.398068+0000","last_became_peered":"2023-10-18T09:07:17.398068+0000","last_unstale":"2023-10-18T09:07:18.368289+0000","last_undegraded":"2023-10-18T09:07:18.368289+0000","last_fullsized":"2023-10-18T09:07:18.368289+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:31:10.222264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742029+0000","last_change":"2023-10-18T09:07:17.401770+0000","last_active":"2023-10-18T09:07:18.742029+0000","last_peered":"2023-10-18T09:07:18.742029+0000","last_clean":"2023-10-18T09:07:18.742029+0000","last_became_active":"2023-10-18T09:07:17.401523+0000","last_became_peered":"2023-10-18T09:07:17.401523+0000","last_unstale":"2023-10-18T09:07:18.742029+0000","last_undegraded":"2023-10-18T09:07:18.742029+0000","last_fullsized":"2023-10-18T09:07:18.742029+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:34.862962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.391795+0000","last_change":"2023-10-18T09:07:17.389057+0000","last_active":"2023-10-18T09:07:17.391795+0000","last_peered":"2023-10-18T09:07:17.391795+0000","last_clean":"2023-10-18T09:07:17.391795+0000","last_became_active":"2023-10-18T09:07:17.388825+0000","last_became_peered":"2023-10-18T09:07:17.388825+0000","last_unstale":"2023-10-18T09:07:17.391795+0000","last_undegraded":"2023-10-18T09:07:17.391795+0000","last_fullsized":"2023-10-18T09:07:17.391795+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:00.445724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.369106+0000","last_change":"2023-10-18T09:07:17.386782+0000","last_active":"2023-10-18T09:07:18.369106+0000","last_peered":"2023-10-18T09:07:18.369106+0000","last_clean":"2023-10-18T09:07:18.369106+0000","last_became_active":"2023-10-18T09:07:17.383799+0000","last_became_peered":"2023-10-18T09:07:17.383799+0000","last_unstale":"2023-10-18T09:07:18.369106+0000","last_undegraded":"2023-10-18T09:07:18.369106+0000","last_fullsized":"2023-10-18T09:07:18.369106+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:48.932155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.379573+0000","last_change":"2023-10-18T09:07:17.379296+0000","last_active":"2023-10-18T09:07:17.379573+0000","last_peered":"2023-10-18T09:07:17.379573+0000","last_clean":"2023-10-18T09:07:17.379573+0000","last_became_active":"2023-10-18T09:07:17.379011+0000","last_became_peered":"2023-10-18T09:07:17.379011+0000","last_unstale":"2023-10-18T09:07:17.379573+0000","last_undegraded":"2023-10-18T09:07:17.379573+0000","last_fullsized":"2023-10-18T09:07:17.379573+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:16.378262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.368727+0000","last_change":"2023-10-18T09:07:17.395988+0000","last_active":"2023-10-18T09:07:18.368727+0000","last_peered":"2023-10-18T09:07:18.368727+0000","last_clean":"2023-10-18T09:07:18.368727+0000","last_became_active":"2023-10-18T09:07:17.395724+0000","last_became_peered":"2023-10-18T09:07:17.395724+0000","last_unstale":"2023-10-18T09:07:18.368727+0000","last_undegraded":"2023-10-18T09:07:18.368727+0000","last_fullsized":"2023-10-18T09:07:18.368727+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:39:57.360245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370531+0000","last_change":"2023-10-18T09:07:17.395692+0000","last_active":"2023-10-18T09:07:18.370531+0000","last_peered":"2023-10-18T09:07:18.370531+0000","last_clean":"2023-10-18T09:07:18.370531+0000","last_became_active":"2023-10-18T09:07:17.392786+0000","last_became_peered":"2023-10-18T09:07:17.392786+0000","last_unstale":"2023-10-18T09:07:18.370531+0000","last_undegraded":"2023-10-18T09:07:18.370531+0000","last_fullsized":"2023-10-18T09:07:18.370531+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:52.299550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.375290+0000","last_change":"2023-10-18T09:07:17.375158+0000","last_active":"2023-10-18T09:07:17.375290+0000","last_peered":"2023-10-18T09:07:17.375290+0000","last_clean":"2023-10-18T09:07:17.375290+0000","last_became_active":"2023-10-18T09:07:17.374911+0000","last_became_peered":"2023-10-18T09:07:17.374911+0000","last_unstale":"2023-10-18T09:07:17.375290+0000","last_undegraded":"2023-10-18T09:07:17.375290+0000","last_fullsized":"2023-10-18T09:07:17.375290+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:32:45.339436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.376030+0000","last_change":"2023-10-18T09:07:17.375892+0000","last_active":"2023-10-18T09:07:17.376030+0000","last_peered":"2023-10-18T09:07:17.376030+0000","last_clean":"2023-10-18T09:07:17.376030+0000","last_became_active":"2023-10-18T09:07:17.375639+0000","last_became_peered":"2023-10-18T09:07:17.375639+0000","last_unstale":"2023-10-18T09:07:17.376030+0000","last_undegraded":"2023-10-18T09:07:17.376030+0000","last_fullsized":"2023-10-18T09:07:17.376030+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:52:46.689238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.368748+0000","last_change":"2023-10-18T09:07:17.390459+0000","last_active":"2023-10-18T09:07:18.368748+0000","last_peered":"2023-10-18T09:07:18.368748+0000","last_clean":"2023-10-18T09:07:18.368748+0000","last_became_active":"2023-10-18T09:07:17.388674+0000","last_became_peered":"2023-10-18T09:07:17.388674+0000","last_unstale":"2023-10-18T09:07:18.368748+0000","last_undegraded":"2023-10-18T09:07:18.368748+0000","last_fullsized":"2023-10-18T09:07:18.368748+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:28:57.800636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.387182+0000","last_change":"2023-10-18T09:07:17.386576+0000","last_active":"2023-10-18T09:07:17.387182+0000","last_peered":"2023-10-18T09:07:17.387182+0000","last_clean":"2023-10-18T09:07:17.387182+0000","last_became_active":"2023-10-18T09:07:17.385339+0000","last_became_peered":"2023-10-18T09:07:17.385339+0000","last_unstale":"2023-10-18T09:07:17.387182+0000","last_undegraded":"2023-10-18T09:07:17.387182+0000","last_fullsized":"2023-10-18T09:07:17.387182+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:48.205449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.387163+0000","last_change":"2023-10-18T09:07:17.385674+0000","last_active":"2023-10-18T09:07:17.387163+0000","last_peered":"2023-10-18T09:07:17.387163+0000","last_clean":"2023-10-18T09:07:17.387163+0000","last_became_active":"2023-10-18T09:07:17.385410+0000","last_became_peered":"2023-10-18T09:07:17.385410+0000","last_unstale":"2023-10-18T09:07:17.387163+0000","last_undegraded":"2023-10-18T09:07:17.387163+0000","last_fullsized":"2023-10-18T09:07:17.387163+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:31.617560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370339+0000","last_change":"2023-10-18T09:07:17.390960+0000","last_active":"2023-10-18T09:07:18.370339+0000","last_peered":"2023-10-18T09:07:18.370339+0000","last_clean":"2023-10-18T09:07:18.370339+0000","last_became_active":"2023-10-18T09:07:17.390732+0000","last_became_peered":"2023-10-18T09:07:17.390732+0000","last_unstale":"2023-10-18T09:07:18.370339+0000","last_undegraded":"2023-10-18T09:07:18.370339+0000","last_fullsized":"2023-10-18T09:07:18.370339+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:01:17.686685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.374129+0000","last_change":"2023-10-18T09:07:17.374049+0000","last_active":"2023-10-18T09:07:17.374129+0000","last_peered":"2023-10-18T09:07:17.374129+0000","last_clean":"2023-10-18T09:07:17.374129+0000","last_became_active":"2023-10-18T09:07:17.373789+0000","last_became_peered":"2023-10-18T09:07:17.373789+0000","last_unstale":"2023-10-18T09:07:17.374129+0000","last_undegraded":"2023-10-18T09:07:17.374129+0000","last_fullsized":"2023-10-18T09:07:17.374129+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:26:42.168261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.376512+0000","last_change":"2023-10-18T09:07:17.376444+0000","last_active":"2023-10-18T09:07:17.376512+0000","last_peered":"2023-10-18T09:07:17.376512+0000","last_clean":"2023-10-18T09:07:17.376512+0000","last_became_active":"2023-10-18T09:07:17.376174+0000","last_became_peered":"2023-10-18T09:07:17.376174+0000","last_unstale":"2023-10-18T09:07:17.376512+0000","last_undegraded":"2023-10-18T09:07:17.376512+0000","last_fullsized":"2023-10-18T09:07:17.376512+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:35:49.646389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.395196+0000","last_change":"2023-10-18T09:07:17.395124+0000","last_active":"2023-10-18T09:07:17.395196+0000","last_peered":"2023-10-18T09:07:17.395196+0000","last_clean":"2023-10-18T09:07:17.395196+0000","last_became_active":"2023-10-18T09:07:17.394549+0000","last_became_peered":"2023-10-18T09:07:17.394549+0000","last_unstale":"2023-10-18T09:07:17.395196+0000","last_undegraded":"2023-10-18T09:07:17.395196+0000","last_fullsized":"2023-10-18T09:07:17.395196+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:02.434291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.741061+0000","last_change":"2023-10-18T09:07:17.396740+0000","last_active":"2023-10-18T09:07:18.741061+0000","last_peered":"2023-10-18T09:07:18.741061+0000","last_clean":"2023-10-18T09:07:18.741061+0000","last_became_active":"2023-10-18T09:07:17.396468+0000","last_became_peered":"2023-10-18T09:07:17.396468+0000","last_unstale":"2023-10-18T09:07:18.741061+0000","last_undegraded":"2023-10-18T09:07:18.741061+0000","last_fullsized":"2023-10-18T09:07:18.741061+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:57:51.612360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.382460+0000","last_change":"2023-10-18T09:07:17.382369+0000","last_active":"2023-10-18T09:07:17.382460+0000","last_peered":"2023-10-18T09:07:17.382460+0000","last_clean":"2023-10-18T09:07:17.382460+0000","last_became_active":"2023-10-18T09:07:17.381496+0000","last_became_peered":"2023-10-18T09:07:17.381496+0000","last_unstale":"2023-10-18T09:07:17.382460+0000","last_undegraded":"2023-10-18T09:07:17.382460+0000","last_fullsized":"2023-10-18T09:07:17.382460+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:22:26.250817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.385576+0000","last_change":"2023-10-18T09:07:17.384909+0000","last_active":"2023-10-18T09:07:17.385576+0000","last_peered":"2023-10-18T09:07:17.385576+0000","last_clean":"2023-10-18T09:07:17.385576+0000","last_became_active":"2023-10-18T09:07:17.384561+0000","last_became_peered":"2023-10-18T09:07:17.384561+0000","last_unstale":"2023-10-18T09:07:17.385576+0000","last_undegraded":"2023-10-18T09:07:17.385576+0000","last_fullsized":"2023-10-18T09:07:17.385576+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:19:19.451511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365387+0000","last_change":"2023-10-18T09:07:17.388521+0000","last_active":"2023-10-18T09:07:18.365387+0000","last_peered":"2023-10-18T09:07:18.365387+0000","last_clean":"2023-10-18T09:07:18.365387+0000","last_became_active":"2023-10-18T09:07:17.388256+0000","last_became_peered":"2023-10-18T09:07:17.388256+0000","last_unstale":"2023-10-18T09:07:18.365387+0000","last_undegraded":"2023-10-18T09:07:18.365387+0000","last_fullsized":"2023-10-18T09:07:18.365387+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:52.299550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.388691+0000","last_change":"2023-10-18T09:07:17.388582+0000","last_active":"2023-10-18T09:07:17.388691+0000","last_peered":"2023-10-18T09:07:17.388691+0000","last_clean":"2023-10-18T09:07:17.388691+0000","last_became_active":"2023-10-18T09:07:17.387812+0000","last_became_peered":"2023-10-18T09:07:17.387812+0000","last_unstale":"2023-10-18T09:07:17.388691+0000","last_undegraded":"2023-10-18T09:07:17.388691+0000","last_fullsized":"2023-10-18T09:07:17.388691+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:17.813007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370681+0000","last_change":"2023-10-18T09:07:17.399479+0000","last_active":"2023-10-18T09:07:18.370681+0000","last_peered":"2023-10-18T09:07:18.370681+0000","last_clean":"2023-10-18T09:07:18.370681+0000","last_became_active":"2023-10-18T09:07:17.399165+0000","last_became_peered":"2023-10-18T09:07:17.399165+0000","last_unstale":"2023-10-18T09:07:18.370681+0000","last_undegraded":"2023-10-18T09:07:18.370681+0000","last_fullsized":"2023-10-18T09:07:18.370681+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:25:17.355528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"178'4","reported_seq":16,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370385+0000","last_change":"2023-10-18T09:07:16.364042+0000","last_active":"2023-10-18T09:07:18.370385+0000","last_peered":"2023-10-18T09:07:18.370385+0000","last_clean":"2023-10-18T09:07:18.370385+0000","last_became_active":"2023-10-18T09:07:16.361240+0000","last_became_peered":"2023-10-18T09:07:16.361240+0000","last_unstale":"2023-10-18T09:07:18.370385+0000","last_undegraded":"2023-10-18T09:07:18.370385+0000","last_fullsized":"2023-10-18T09:07:18.370385+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:00:04.045218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365497+0000","last_change":"2023-10-18T09:07:17.383827+0000","last_active":"2023-10-18T09:07:18.365497+0000","last_peered":"2023-10-18T09:07:18.365497+0000","last_clean":"2023-10-18T09:07:18.365497+0000","last_became_active":"2023-10-18T09:07:17.383487+0000","last_became_peered":"2023-10-18T09:07:17.383487+0000","last_unstale":"2023-10-18T09:07:18.365497+0000","last_undegraded":"2023-10-18T09:07:18.365497+0000","last_fullsized":"2023-10-18T09:07:18.365497+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:21:20.537938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"178'1","reported_seq":13,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365456+0000","last_change":"2023-10-18T09:07:16.360111+0000","last_active":"2023-10-18T09:07:18.365456+0000","last_peered":"2023-10-18T09:07:18.365456+0000","last_clean":"2023-10-18T09:07:18.365456+0000","last_became_active":"2023-10-18T09:07:16.359724+0000","last_became_peered":"2023-10-18T09:07:16.359724+0000","last_unstale":"2023-10-18T09:07:18.365456+0000","last_undegraded":"2023-10-18T09:07:18.365456+0000","last_fullsized":"2023-10-18T09:07:18.365456+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:42:08.267937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.387475+0000","last_change":"2023-10-18T09:07:17.387271+0000","last_active":"2023-10-18T09:07:17.387475+0000","last_peered":"2023-10-18T09:07:17.387475+0000","last_clean":"2023-10-18T09:07:17.387475+0000","last_became_active":"2023-10-18T09:07:17.387002+0000","last_became_peered":"2023-10-18T09:07:17.387002+0000","last_unstale":"2023-10-18T09:07:17.387475+0000","last_undegraded":"2023-10-18T09:07:17.387475+0000","last_fullsized":"2023-10-18T09:07:17.387475+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:14.476512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"178'2","reported_seq":13,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.397483+0000","last_change":"2023-10-18T09:07:16.361045+0000","last_active":"2023-10-18T09:07:17.397483+0000","last_peered":"2023-10-18T09:07:17.397483+0000","last_clean":"2023-10-18T09:07:17.397483+0000","last_became_active":"2023-10-18T09:07:16.360550+0000","last_became_peered":"2023-10-18T09:07:16.360550+0000","last_unstale":"2023-10-18T09:07:17.397483+0000","last_undegraded":"2023-10-18T09:07:17.397483+0000","last_fullsized":"2023-10-18T09:07:17.397483+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:39:03.300238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.369441+0000","last_change":"2023-10-18T09:07:17.399057+0000","last_active":"2023-10-18T09:07:18.369441+0000","last_peered":"2023-10-18T09:07:18.369441+0000","last_clean":"2023-10-18T09:07:18.369441+0000","last_became_active":"2023-10-18T09:07:17.398309+0000","last_became_peered":"2023-10-18T09:07:17.398309+0000","last_unstale":"2023-10-18T09:07:18.369441+0000","last_undegraded":"2023-10-18T09:07:18.369441+0000","last_fullsized":"2023-10-18T09:07:18.369441+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:48.708074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"178'2","reported_seq":13,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.398633+0000","last_change":"2023-10-18T09:07:16.362256+0000","last_active":"2023-10-18T09:07:17.398633+0000","last_peered":"2023-10-18T09:07:17.398633+0000","last_clean":"2023-10-18T09:07:17.398633+0000","last_became_active":"2023-10-18T09:07:16.361241+0000","last_became_peered":"2023-10-18T09:07:16.361241+0000","last_unstale":"2023-10-18T09:07:17.398633+0000","last_undegraded":"2023-10-18T09:07:17.398633+0000","last_fullsized":"2023-10-18T09:07:17.398633+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:11:10.278131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365701+0000","last_change":"2023-10-18T09:07:17.396091+0000","last_active":"2023-10-18T09:07:18.365701+0000","last_peered":"2023-10-18T09:07:18.365701+0000","last_clean":"2023-10-18T09:07:18.365701+0000","last_became_active":"2023-10-18T09:07:17.395738+0000","last_became_peered":"2023-10-18T09:07:17.395738+0000","last_unstale":"2023-10-18T09:07:18.365701+0000","last_undegraded":"2023-10-18T09:07:18.365701+0000","last_fullsized":"2023-10-18T09:07:18.365701+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:00.445724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"178'1","reported_seq":12,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.389964+0000","last_change":"2023-10-18T09:07:16.361126+0000","last_active":"2023-10-18T09:07:17.389964+0000","last_peered":"2023-10-18T09:07:17.389964+0000","last_clean":"2023-10-18T09:07:17.389964+0000","last_became_active":"2023-10-18T09:07:16.360744+0000","last_became_peered":"2023-10-18T09:07:16.360744+0000","last_unstale":"2023-10-18T09:07:17.389964+0000","last_undegraded":"2023-10-18T09:07:17.389964+0000","last_fullsized":"2023-10-18T09:07:17.389964+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:22:41.018905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.397708+0000","last_change":"2023-10-18T09:07:17.395261+0000","last_active":"2023-10-18T09:07:17.397708+0000","last_peered":"2023-10-18T09:07:17.397708+0000","last_clean":"2023-10-18T09:07:17.397708+0000","last_became_active":"2023-10-18T09:07:17.394823+0000","last_became_peered":"2023-10-18T09:07:17.394823+0000","last_unstale":"2023-10-18T09:07:17.397708+0000","last_undegraded":"2023-10-18T09:07:17.397708+0000","last_fullsized":"2023-10-18T09:07:17.397708+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:04.413480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"178'1","reported_seq":13,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742940+0000","last_change":"2023-10-18T09:07:16.367941+0000","last_active":"2023-10-18T09:07:18.742940+0000","last_peered":"2023-10-18T09:07:18.742940+0000","last_clean":"2023-10-18T09:07:18.742940+0000","last_became_active":"2023-10-18T09:07:16.367443+0000","last_became_peered":"2023-10-18T09:07:16.367443+0000","last_unstale":"2023-10-18T09:07:18.742940+0000","last_undegraded":"2023-10-18T09:07:18.742940+0000","last_fullsized":"2023-10-18T09:07:18.742940+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:28:55.503205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.366030+0000","last_change":"2023-10-18T09:07:17.390110+0000","last_active":"2023-10-18T09:07:18.366030+0000","last_peered":"2023-10-18T09:07:18.366030+0000","last_clean":"2023-10-18T09:07:18.366030+0000","last_became_active":"2023-10-18T09:07:17.389398+0000","last_became_peered":"2023-10-18T09:07:17.389398+0000","last_unstale":"2023-10-18T09:07:18.366030+0000","last_undegraded":"2023-10-18T09:07:18.366030+0000","last_fullsized":"2023-10-18T09:07:18.366030+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:48.205449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.373009+0000","last_change":"2023-10-18T09:07:16.365435+0000","last_active":"2023-10-18T09:07:17.373009+0000","last_peered":"2023-10-18T09:07:17.373009+0000","last_clean":"2023-10-18T09:07:17.373009+0000","last_became_active":"2023-10-18T09:07:16.365125+0000","last_became_peered":"2023-10-18T09:07:16.365125+0000","last_unstale":"2023-10-18T09:07:17.373009+0000","last_undegraded":"2023-10-18T09:07:17.373009+0000","last_fullsized":"2023-10-18T09:07:17.373009+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:55.519487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742308+0000","last_change":"2023-10-18T09:07:17.391978+0000","last_active":"2023-10-18T09:07:18.742308+0000","last_peered":"2023-10-18T09:07:18.742308+0000","last_clean":"2023-10-18T09:07:18.742308+0000","last_became_active":"2023-10-18T09:07:17.389095+0000","last_became_peered":"2023-10-18T09:07:17.389095+0000","last_unstale":"2023-10-18T09:07:18.742308+0000","last_undegraded":"2023-10-18T09:07:18.742308+0000","last_fullsized":"2023-10-18T09:07:18.742308+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:59.823948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.383758+0000","last_change":"2023-10-18T09:07:16.365242+0000","last_active":"2023-10-18T09:07:17.383758+0000","last_peered":"2023-10-18T09:07:17.383758+0000","last_clean":"2023-10-18T09:07:17.383758+0000","last_became_active":"2023-10-18T09:07:16.364983+0000","last_became_peered":"2023-10-18T09:07:16.364983+0000","last_unstale":"2023-10-18T09:07:17.383758+0000","last_undegraded":"2023-10-18T09:07:17.383758+0000","last_fullsized":"2023-10-18T09:07:17.383758+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:14.895783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742809+0000","last_change":"2023-10-18T09:07:17.390161+0000","last_active":"2023-10-18T09:07:18.742809+0000","last_peered":"2023-10-18T09:07:18.742809+0000","last_clean":"2023-10-18T09:07:18.742809+0000","last_became_active":"2023-10-18T09:07:17.389869+0000","last_became_peered":"2023-10-18T09:07:17.389869+0000","last_unstale":"2023-10-18T09:07:18.742809+0000","last_undegraded":"2023-10-18T09:07:18.742809+0000","last_fullsized":"2023-10-18T09:07:18.742809+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:01.734509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365503+0000","last_change":"2023-10-18T09:07:16.356639+0000","last_active":"2023-10-18T09:07:18.365503+0000","last_peered":"2023-10-18T09:07:18.365503+0000","last_clean":"2023-10-18T09:07:18.365503+0000","last_became_active":"2023-10-18T09:07:16.356359+0000","last_became_peered":"2023-10-18T09:07:16.356359+0000","last_unstale":"2023-10-18T09:07:18.365503+0000","last_undegraded":"2023-10-18T09:07:18.365503+0000","last_fullsized":"2023-10-18T09:07:18.365503+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:07:56.622798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.369108+0000","last_change":"2023-10-18T09:07:17.381272+0000","last_active":"2023-10-18T09:07:18.369108+0000","last_peered":"2023-10-18T09:07:18.369108+0000","last_clean":"2023-10-18T09:07:18.369108+0000","last_became_active":"2023-10-18T09:07:17.380821+0000","last_became_peered":"2023-10-18T09:07:17.380821+0000","last_unstale":"2023-10-18T09:07:18.369108+0000","last_undegraded":"2023-10-18T09:07:18.369108+0000","last_fullsized":"2023-10-18T09:07:18.369108+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:03.047350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.369602+0000","last_change":"2023-10-18T09:07:16.368087+0000","last_active":"2023-10-18T09:07:18.369602+0000","last_peered":"2023-10-18T09:07:18.369602+0000","last_clean":"2023-10-18T09:07:18.369602+0000","last_became_active":"2023-10-18T09:07:16.367805+0000","last_became_peered":"2023-10-18T09:07:16.367805+0000","last_unstale":"2023-10-18T09:07:18.369602+0000","last_undegraded":"2023-10-18T09:07:18.369602+0000","last_fullsized":"2023-10-18T09:07:18.369602+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:37:29.546569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.381861+0000","last_change":"2023-10-18T09:07:17.381404+0000","last_active":"2023-10-18T09:07:17.381861+0000","last_peered":"2023-10-18T09:07:17.381861+0000","last_clean":"2023-10-18T09:07:17.381861+0000","last_became_active":"2023-10-18T09:07:17.381033+0000","last_became_peered":"2023-10-18T09:07:17.381033+0000","last_unstale":"2023-10-18T09:07:17.381861+0000","last_undegraded":"2023-10-18T09:07:17.381861+0000","last_fullsized":"2023-10-18T09:07:17.381861+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:02:48.453087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"178'6","reported_seq":18,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.741077+0000","last_change":"2023-10-18T09:07:16.360840+0000","last_active":"2023-10-18T09:07:18.741077+0000","last_peered":"2023-10-18T09:07:18.741077+0000","last_clean":"2023-10-18T09:07:18.741077+0000","last_became_active":"2023-10-18T09:07:16.360429+0000","last_became_peered":"2023-10-18T09:07:16.360429+0000","last_unstale":"2023-10-18T09:07:18.741077+0000","last_undegraded":"2023-10-18T09:07:18.741077+0000","last_fullsized":"2023-10-18T09:07:18.741077+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:37:29.546569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.369699+0000","last_change":"2023-10-18T09:07:17.391046+0000","last_active":"2023-10-18T09:07:18.369699+0000","last_peered":"2023-10-18T09:07:18.369699+0000","last_clean":"2023-10-18T09:07:18.369699+0000","last_became_active":"2023-10-18T09:07:17.390433+0000","last_became_peered":"2023-10-18T09:07:17.390433+0000","last_unstale":"2023-10-18T09:07:18.369699+0000","last_undegraded":"2023-10-18T09:07:18.369699+0000","last_fullsized":"2023-10-18T09:07:18.369699+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:21:24.245975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"178'2","reported_seq":14,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370958+0000","last_change":"2023-10-18T09:07:16.359716+0000","last_active":"2023-10-18T09:07:18.370958+0000","last_peered":"2023-10-18T09:07:18.370958+0000","last_clean":"2023-10-18T09:07:18.370958+0000","last_became_active":"2023-10-18T09:07:16.359462+0000","last_became_peered":"2023-10-18T09:07:16.359462+0000","last_unstale":"2023-10-18T09:07:18.370958+0000","last_undegraded":"2023-10-18T09:07:18.370958+0000","last_fullsized":"2023-10-18T09:07:18.370958+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:38:23.509498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365809+0000","last_change":"2023-10-18T09:07:17.382465+0000","last_active":"2023-10-18T09:07:18.365809+0000","last_peered":"2023-10-18T09:07:18.365809+0000","last_clean":"2023-10-18T09:07:18.365809+0000","last_became_active":"2023-10-18T09:07:17.382227+0000","last_became_peered":"2023-10-18T09:07:17.382227+0000","last_unstale":"2023-10-18T09:07:18.365809+0000","last_undegraded":"2023-10-18T09:07:18.365809+0000","last_fullsized":"2023-10-18T09:07:18.365809+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:04.413480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.369917+0000","last_change":"2023-10-18T09:07:16.367854+0000","last_active":"2023-10-18T09:07:18.369917+0000","last_peered":"2023-10-18T09:07:18.369917+0000","last_clean":"2023-10-18T09:07:18.369917+0000","last_became_active":"2023-10-18T09:07:16.367402+0000","last_became_peered":"2023-10-18T09:07:16.367402+0000","last_unstale":"2023-10-18T09:07:18.369917+0000","last_undegraded":"2023-10-18T09:07:18.369917+0000","last_fullsized":"2023-10-18T09:07:18.369917+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:07:56.622798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742291+0000","last_change":"2023-10-18T09:07:17.401333+0000","last_active":"2023-10-18T09:07:18.742291+0000","last_peered":"2023-10-18T09:07:18.742291+0000","last_clean":"2023-10-18T09:07:18.742291+0000","last_became_active":"2023-10-18T09:07:17.400762+0000","last_became_peered":"2023-10-18T09:07:17.400762+0000","last_unstale":"2023-10-18T09:07:18.742291+0000","last_undegraded":"2023-10-18T09:07:18.742291+0000","last_fullsized":"2023-10-18T09:07:18.742291+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:32:48.511798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.365345+0000","last_change":"2023-10-18T09:07:16.353725+0000","last_active":"2023-10-18T09:07:17.365345+0000","last_peered":"2023-10-18T09:07:17.365345+0000","last_clean":"2023-10-18T09:07:17.365345+0000","last_became_active":"2023-10-18T09:07:16.353299+0000","last_became_peered":"2023-10-18T09:07:16.353299+0000","last_unstale":"2023-10-18T09:07:17.365345+0000","last_undegraded":"2023-10-18T09:07:17.365345+0000","last_fullsized":"2023-10-18T09:07:17.365345+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:52:39.133251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370232+0000","last_change":"2023-10-18T09:07:17.394017+0000","last_active":"2023-10-18T09:07:18.370232+0000","last_peered":"2023-10-18T09:07:18.370232+0000","last_clean":"2023-10-18T09:07:18.370232+0000","last_became_active":"2023-10-18T09:07:17.393461+0000","last_became_peered":"2023-10-18T09:07:17.393461+0000","last_unstale":"2023-10-18T09:07:18.370232+0000","last_undegraded":"2023-10-18T09:07:18.370232+0000","last_fullsized":"2023-10-18T09:07:18.370232+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:54.405679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"178'8","reported_seq":19,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.397911+0000","last_change":"2023-10-18T09:07:16.366985+0000","last_active":"2023-10-18T09:07:17.397911+0000","last_peered":"2023-10-18T09:07:17.397911+0000","last_clean":"2023-10-18T09:07:17.397911+0000","last_became_active":"2023-10-18T09:07:16.366843+0000","last_became_peered":"2023-10-18T09:07:16.366843+0000","last_unstale":"2023-10-18T09:07:17.397911+0000","last_undegraded":"2023-10-18T09:07:17.397911+0000","last_fullsized":"2023-10-18T09:07:17.397911+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:28:55.503205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.369791+0000","last_change":"2023-10-18T09:07:17.384155+0000","last_active":"2023-10-18T09:07:18.369791+0000","last_peered":"2023-10-18T09:07:18.369791+0000","last_clean":"2023-10-18T09:07:18.369791+0000","last_became_active":"2023-10-18T09:07:17.377696+0000","last_became_peered":"2023-10-18T09:07:17.377696+0000","last_unstale":"2023-10-18T09:07:18.369791+0000","last_undegraded":"2023-10-18T09:07:18.369791+0000","last_fullsized":"2023-10-18T09:07:18.369791+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:10:09.706171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"178'1","reported_seq":13,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742455+0000","last_change":"2023-10-18T09:07:16.363546+0000","last_active":"2023-10-18T09:07:18.742455+0000","last_peered":"2023-10-18T09:07:18.742455+0000","last_clean":"2023-10-18T09:07:18.742455+0000","last_became_active":"2023-10-18T09:07:16.363306+0000","last_became_peered":"2023-10-18T09:07:16.363306+0000","last_unstale":"2023-10-18T09:07:18.742455+0000","last_undegraded":"2023-10-18T09:07:18.742455+0000","last_fullsized":"2023-10-18T09:07:18.742455+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:52:45.673846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.386768+0000","last_change":"2023-10-18T09:07:17.384710+0000","last_active":"2023-10-18T09:07:17.386768+0000","last_peered":"2023-10-18T09:07:17.386768+0000","last_clean":"2023-10-18T09:07:17.386768+0000","last_became_active":"2023-10-18T09:07:17.384238+0000","last_became_peered":"2023-10-18T09:07:17.384238+0000","last_unstale":"2023-10-18T09:07:17.386768+0000","last_undegraded":"2023-10-18T09:07:17.386768+0000","last_fullsized":"2023-10-18T09:07:17.386768+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:11.829238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"178'2","reported_seq":14,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.366094+0000","last_change":"2023-10-18T09:07:16.362989+0000","last_active":"2023-10-18T09:07:18.366094+0000","last_peered":"2023-10-18T09:07:18.366094+0000","last_clean":"2023-10-18T09:07:18.366094+0000","last_became_active":"2023-10-18T09:07:16.362622+0000","last_became_peered":"2023-10-18T09:07:16.362622+0000","last_unstale":"2023-10-18T09:07:18.366094+0000","last_undegraded":"2023-10-18T09:07:18.366094+0000","last_fullsized":"2023-10-18T09:07:18.366094+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:21:23.230582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370575+0000","last_change":"2023-10-18T09:07:17.393889+0000","last_active":"2023-10-18T09:07:18.370575+0000","last_peered":"2023-10-18T09:07:18.370575+0000","last_clean":"2023-10-18T09:07:18.370575+0000","last_became_active":"2023-10-18T09:07:17.393116+0000","last_became_peered":"2023-10-18T09:07:17.393116+0000","last_unstale":"2023-10-18T09:07:18.370575+0000","last_undegraded":"2023-10-18T09:07:18.370575+0000","last_fullsized":"2023-10-18T09:07:18.370575+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:10.523692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"178'1","reported_seq":12,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.389605+0000","last_change":"2023-10-18T09:07:16.361903+0000","last_active":"2023-10-18T09:07:17.389605+0000","last_peered":"2023-10-18T09:07:17.389605+0000","last_clean":"2023-10-18T09:07:17.389605+0000","last_became_active":"2023-10-18T09:07:16.361524+0000","last_became_peered":"2023-10-18T09:07:16.361524+0000","last_unstale":"2023-10-18T09:07:17.389605+0000","last_undegraded":"2023-10-18T09:07:17.389605+0000","last_fullsized":"2023-10-18T09:07:17.389605+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:26:45.011516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.397868+0000","last_change":"2023-10-18T09:07:17.395628+0000","last_active":"2023-10-18T09:07:17.397868+0000","last_peered":"2023-10-18T09:07:17.397868+0000","last_clean":"2023-10-18T09:07:17.397868+0000","last_became_active":"2023-10-18T09:07:17.395076+0000","last_became_peered":"2023-10-18T09:07:17.395076+0000","last_unstale":"2023-10-18T09:07:17.397868+0000","last_undegraded":"2023-10-18T09:07:17.397868+0000","last_fullsized":"2023-10-18T09:07:17.397868+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:28.831307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365926+0000","last_change":"2023-10-18T09:07:16.365594+0000","last_active":"2023-10-18T09:07:18.365926+0000","last_peered":"2023-10-18T09:07:18.365926+0000","last_clean":"2023-10-18T09:07:18.365926+0000","last_became_active":"2023-10-18T09:07:16.365311+0000","last_became_peered":"2023-10-18T09:07:16.365311+0000","last_unstale":"2023-10-18T09:07:18.365926+0000","last_undegraded":"2023-10-18T09:07:18.365926+0000","last_fullsized":"2023-10-18T09:07:18.365926+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:42.215263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.385558+0000","last_change":"2023-10-18T09:07:17.384730+0000","last_active":"2023-10-18T09:07:17.385558+0000","last_peered":"2023-10-18T09:07:17.385558+0000","last_clean":"2023-10-18T09:07:17.385558+0000","last_became_active":"2023-10-18T09:07:17.384271+0000","last_became_peered":"2023-10-18T09:07:17.384271+0000","last_unstale":"2023-10-18T09:07:17.385558+0000","last_undegraded":"2023-10-18T09:07:17.385558+0000","last_fullsized":"2023-10-18T09:07:17.385558+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:21:20.537938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"178'2","reported_seq":14,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.741309+0000","last_change":"2023-10-18T09:07:16.366691+0000","last_active":"2023-10-18T09:07:18.741309+0000","last_peered":"2023-10-18T09:07:18.741309+0000","last_clean":"2023-10-18T09:07:18.741309+0000","last_became_active":"2023-10-18T09:07:16.366485+0000","last_became_peered":"2023-10-18T09:07:16.366485+0000","last_unstale":"2023-10-18T09:07:18.741309+0000","last_undegraded":"2023-10-18T09:07:18.741309+0000","last_fullsized":"2023-10-18T09:07:18.741309+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:34.513833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.741210+0000","last_change":"2023-10-18T09:07:17.388640+0000","last_active":"2023-10-18T09:07:18.741210+0000","last_peered":"2023-10-18T09:07:18.741210+0000","last_clean":"2023-10-18T09:07:18.741210+0000","last_became_active":"2023-10-18T09:07:17.388193+0000","last_became_peered":"2023-10-18T09:07:17.388193+0000","last_unstale":"2023-10-18T09:07:18.741210+0000","last_undegraded":"2023-10-18T09:07:18.741210+0000","last_fullsized":"2023-10-18T09:07:18.741210+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:49.542450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.371217+0000","last_change":"2023-10-18T09:07:16.364501+0000","last_active":"2023-10-18T09:07:18.371217+0000","last_peered":"2023-10-18T09:07:18.371217+0000","last_clean":"2023-10-18T09:07:18.371217+0000","last_became_active":"2023-10-18T09:07:16.362564+0000","last_became_peered":"2023-10-18T09:07:16.362564+0000","last_unstale":"2023-10-18T09:07:18.371217+0000","last_undegraded":"2023-10-18T09:07:18.371217+0000","last_fullsized":"2023-10-18T09:07:18.371217+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:14:01.348219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.387324+0000","last_change":"2023-10-18T09:07:17.387235+0000","last_active":"2023-10-18T09:07:17.387324+0000","last_peered":"2023-10-18T09:07:17.387324+0000","last_clean":"2023-10-18T09:07:17.387324+0000","last_became_active":"2023-10-18T09:07:17.386101+0000","last_became_peered":"2023-10-18T09:07:17.386101+0000","last_unstale":"2023-10-18T09:07:17.387324+0000","last_undegraded":"2023-10-18T09:07:17.387324+0000","last_fullsized":"2023-10-18T09:07:17.387324+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:37:35.491056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370015+0000","last_change":"2023-10-18T09:07:16.363088+0000","last_active":"2023-10-18T09:07:18.370015+0000","last_peered":"2023-10-18T09:07:18.370015+0000","last_clean":"2023-10-18T09:07:18.370015+0000","last_became_active":"2023-10-18T09:07:16.362052+0000","last_became_peered":"2023-10-18T09:07:16.362052+0000","last_unstale":"2023-10-18T09:07:18.370015+0000","last_undegraded":"2023-10-18T09:07:18.370015+0000","last_fullsized":"2023-10-18T09:07:18.370015+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:48.527057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.741998+0000","last_change":"2023-10-18T09:07:17.387447+0000","last_active":"2023-10-18T09:07:18.741998+0000","last_peered":"2023-10-18T09:07:18.741998+0000","last_clean":"2023-10-18T09:07:18.741998+0000","last_became_active":"2023-10-18T09:07:17.387146+0000","last_became_peered":"2023-10-18T09:07:17.387146+0000","last_unstale":"2023-10-18T09:07:18.741998+0000","last_undegraded":"2023-10-18T09:07:18.741998+0000","last_fullsized":"2023-10-18T09:07:18.741998+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:31:10.222264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370473+0000","last_change":"2023-10-18T09:07:16.365625+0000","last_active":"2023-10-18T09:07:18.370473+0000","last_peered":"2023-10-18T09:07:18.370473+0000","last_clean":"2023-10-18T09:07:18.370473+0000","last_became_active":"2023-10-18T09:07:16.362765+0000","last_became_peered":"2023-10-18T09:07:16.362765+0000","last_unstale":"2023-10-18T09:07:18.370473+0000","last_undegraded":"2023-10-18T09:07:18.370473+0000","last_fullsized":"2023-10-18T09:07:18.370473+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:18.645811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742625+0000","last_change":"2023-10-18T09:07:17.401163+0000","last_active":"2023-10-18T09:07:18.742625+0000","last_peered":"2023-10-18T09:07:18.742625+0000","last_clean":"2023-10-18T09:07:18.742625+0000","last_became_active":"2023-10-18T09:07:17.400406+0000","last_became_peered":"2023-10-18T09:07:17.400406+0000","last_unstale":"2023-10-18T09:07:18.742625+0000","last_undegraded":"2023-10-18T09:07:18.742625+0000","last_fullsized":"2023-10-18T09:07:18.742625+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:12:41.408408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"178'2","reported_seq":14,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.368981+0000","last_change":"2023-10-18T09:07:16.364405+0000","last_active":"2023-10-18T09:07:18.368981+0000","last_peered":"2023-10-18T09:07:18.368981+0000","last_clean":"2023-10-18T09:07:18.368981+0000","last_became_active":"2023-10-18T09:07:16.363029+0000","last_became_peered":"2023-10-18T09:07:16.363029+0000","last_unstale":"2023-10-18T09:07:18.368981+0000","last_undegraded":"2023-10-18T09:07:18.368981+0000","last_fullsized":"2023-10-18T09:07:18.368981+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:05:41.563036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370920+0000","last_change":"2023-10-18T09:07:17.393745+0000","last_active":"2023-10-18T09:07:18.370920+0000","last_peered":"2023-10-18T09:07:18.370920+0000","last_clean":"2023-10-18T09:07:18.370920+0000","last_became_active":"2023-10-18T09:07:17.392751+0000","last_became_peered":"2023-10-18T09:07:17.392751+0000","last_unstale":"2023-10-18T09:07:18.370920+0000","last_undegraded":"2023-10-18T09:07:18.370920+0000","last_fullsized":"2023-10-18T09:07:18.370920+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:00.445724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.365782+0000","last_change":"2023-10-18T09:07:16.363218+0000","last_active":"2023-10-18T09:07:17.365782+0000","last_peered":"2023-10-18T09:07:17.365782+0000","last_clean":"2023-10-18T09:07:17.365782+0000","last_became_active":"2023-10-18T09:07:16.362938+0000","last_became_peered":"2023-10-18T09:07:16.362938+0000","last_unstale":"2023-10-18T09:07:17.365782+0000","last_undegraded":"2023-10-18T09:07:17.365782+0000","last_fullsized":"2023-10-18T09:07:17.365782+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:50:23.934782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365902+0000","last_change":"2023-10-18T09:07:17.387337+0000","last_active":"2023-10-18T09:07:18.365902+0000","last_peered":"2023-10-18T09:07:18.365902+0000","last_clean":"2023-10-18T09:07:18.365902+0000","last_became_active":"2023-10-18T09:07:17.387062+0000","last_became_peered":"2023-10-18T09:07:17.387062+0000","last_unstale":"2023-10-18T09:07:18.365902+0000","last_undegraded":"2023-10-18T09:07:18.365902+0000","last_fullsized":"2023-10-18T09:07:18.365902+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:37:35.491056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.367018+0000","last_change":"2023-10-18T09:07:16.361647+0000","last_active":"2023-10-18T09:07:17.367018+0000","last_peered":"2023-10-18T09:07:17.367018+0000","last_clean":"2023-10-18T09:07:17.367018+0000","last_became_active":"2023-10-18T09:07:16.361384+0000","last_became_peered":"2023-10-18T09:07:16.361384+0000","last_unstale":"2023-10-18T09:07:17.367018+0000","last_undegraded":"2023-10-18T09:07:17.367018+0000","last_fullsized":"2023-10-18T09:07:17.367018+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:20.810697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.386909+0000","last_change":"2023-10-18T09:07:17.386465+0000","last_active":"2023-10-18T09:07:17.386909+0000","last_peered":"2023-10-18T09:07:17.386909+0000","last_clean":"2023-10-18T09:07:17.386909+0000","last_became_active":"2023-10-18T09:07:17.386086+0000","last_became_peered":"2023-10-18T09:07:17.386086+0000","last_unstale":"2023-10-18T09:07:17.386909+0000","last_undegraded":"2023-10-18T09:07:17.386909+0000","last_fullsized":"2023-10-18T09:07:17.386909+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:11:05.152957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.366510+0000","last_change":"2023-10-18T09:07:16.361128+0000","last_active":"2023-10-18T09:07:17.366510+0000","last_peered":"2023-10-18T09:07:17.366510+0000","last_clean":"2023-10-18T09:07:17.366510+0000","last_became_active":"2023-10-18T09:07:16.360821+0000","last_became_peered":"2023-10-18T09:07:16.360821+0000","last_unstale":"2023-10-18T09:07:17.366510+0000","last_undegraded":"2023-10-18T09:07:17.366510+0000","last_fullsized":"2023-10-18T09:07:17.366510+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:41:54.930255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.386797+0000","last_change":"2023-10-18T09:07:17.384882+0000","last_active":"2023-10-18T09:07:17.386797+0000","last_peered":"2023-10-18T09:07:17.386797+0000","last_clean":"2023-10-18T09:07:17.386797+0000","last_became_active":"2023-10-18T09:07:17.384677+0000","last_became_peered":"2023-10-18T09:07:17.384677+0000","last_unstale":"2023-10-18T09:07:17.386797+0000","last_undegraded":"2023-10-18T09:07:17.386797+0000","last_fullsized":"2023-10-18T09:07:17.386797+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:43:24.684032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.741802+0000","last_change":"2023-10-18T09:07:16.362111+0000","last_active":"2023-10-18T09:07:18.741802+0000","last_peered":"2023-10-18T09:07:18.741802+0000","last_clean":"2023-10-18T09:07:18.741802+0000","last_became_active":"2023-10-18T09:07:16.361862+0000","last_became_peered":"2023-10-18T09:07:16.361862+0000","last_unstale":"2023-10-18T09:07:18.741802+0000","last_undegraded":"2023-10-18T09:07:18.741802+0000","last_fullsized":"2023-10-18T09:07:18.741802+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:35:34.092911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742330+0000","last_change":"2023-10-18T09:07:17.395505+0000","last_active":"2023-10-18T09:07:18.742330+0000","last_peered":"2023-10-18T09:07:18.742330+0000","last_clean":"2023-10-18T09:07:18.742330+0000","last_became_active":"2023-10-18T09:07:17.393306+0000","last_became_peered":"2023-10-18T09:07:17.393306+0000","last_unstale":"2023-10-18T09:07:18.742330+0000","last_undegraded":"2023-10-18T09:07:18.742330+0000","last_fullsized":"2023-10-18T09:07:18.742330+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:28:09.467521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.369215+0000","last_change":"2023-10-18T09:07:16.364731+0000","last_active":"2023-10-18T09:07:18.369215+0000","last_peered":"2023-10-18T09:07:18.369215+0000","last_clean":"2023-10-18T09:07:18.369215+0000","last_became_active":"2023-10-18T09:07:16.364127+0000","last_became_peered":"2023-10-18T09:07:16.364127+0000","last_unstale":"2023-10-18T09:07:18.369215+0000","last_undegraded":"2023-10-18T09:07:18.369215+0000","last_fullsized":"2023-10-18T09:07:18.369215+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:58:53.695998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.371003+0000","last_change":"2023-10-18T09:07:17.379531+0000","last_active":"2023-10-18T09:07:18.371003+0000","last_peered":"2023-10-18T09:07:18.371003+0000","last_clean":"2023-10-18T09:07:18.371003+0000","last_became_active":"2023-10-18T09:07:17.378878+0000","last_became_peered":"2023-10-18T09:07:17.378878+0000","last_unstale":"2023-10-18T09:07:18.371003+0000","last_undegraded":"2023-10-18T09:07:18.371003+0000","last_fullsized":"2023-10-18T09:07:18.371003+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:38.394044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.371372+0000","last_change":"2023-10-18T09:07:16.366629+0000","last_active":"2023-10-18T09:07:18.371372+0000","last_peered":"2023-10-18T09:07:18.371372+0000","last_clean":"2023-10-18T09:07:18.371372+0000","last_became_active":"2023-10-18T09:07:16.365920+0000","last_became_peered":"2023-10-18T09:07:16.365920+0000","last_unstale":"2023-10-18T09:07:18.371372+0000","last_undegraded":"2023-10-18T09:07:18.371372+0000","last_fullsized":"2023-10-18T09:07:18.371372+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:27:53.056702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742812+0000","last_change":"2023-10-18T09:07:17.389853+0000","last_active":"2023-10-18T09:07:18.742812+0000","last_peered":"2023-10-18T09:07:18.742812+0000","last_clean":"2023-10-18T09:07:18.742812+0000","last_became_active":"2023-10-18T09:07:17.389487+0000","last_became_peered":"2023-10-18T09:07:17.389487+0000","last_unstale":"2023-10-18T09:07:18.742812+0000","last_undegraded":"2023-10-18T09:07:18.742812+0000","last_fullsized":"2023-10-18T09:07:18.742812+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:52:54.846421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365636+0000","last_change":"2023-10-18T09:07:16.359958+0000","last_active":"2023-10-18T09:07:18.365636+0000","last_peered":"2023-10-18T09:07:18.365636+0000","last_clean":"2023-10-18T09:07:18.365636+0000","last_became_active":"2023-10-18T09:07:16.359412+0000","last_became_peered":"2023-10-18T09:07:16.359412+0000","last_unstale":"2023-10-18T09:07:18.365636+0000","last_undegraded":"2023-10-18T09:07:18.365636+0000","last_fullsized":"2023-10-18T09:07:18.365636+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:38:23.509498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.741520+0000","last_change":"2023-10-18T09:07:17.388720+0000","last_active":"2023-10-18T09:07:18.741520+0000","last_peered":"2023-10-18T09:07:18.741520+0000","last_clean":"2023-10-18T09:07:18.741520+0000","last_became_active":"2023-10-18T09:07:17.388468+0000","last_became_peered":"2023-10-18T09:07:17.388468+0000","last_unstale":"2023-10-18T09:07:18.741520+0000","last_undegraded":"2023-10-18T09:07:18.741520+0000","last_fullsized":"2023-10-18T09:07:18.741520+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:14:02.363611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"178'3","reported_seq":15,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.371149+0000","last_change":"2023-10-18T09:07:16.366385+0000","last_active":"2023-10-18T09:07:18.371149+0000","last_peered":"2023-10-18T09:07:18.371149+0000","last_clean":"2023-10-18T09:07:18.371149+0000","last_became_active":"2023-10-18T09:07:16.363981+0000","last_became_peered":"2023-10-18T09:07:16.363981+0000","last_unstale":"2023-10-18T09:07:18.371149+0000","last_undegraded":"2023-10-18T09:07:18.371149+0000","last_fullsized":"2023-10-18T09:07:18.371149+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:39:35.876064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":338,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.373440+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:07:17.373440+0000","last_peered":"2023-10-18T09:07:17.373440+0000","last_clean":"2023-10-18T09:07:17.373440+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:07:17.373440+0000","last_undegraded":"2023-10-18T09:07:17.373440+0000","last_fullsized":"2023-10-18T09:07:17.373440+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574997,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1568,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193743,"statfs":{"total":200000000,"available":198393703,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.1040000000000001,"15min":1.0640000000000001},"min":{"1min":0.80000000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.52,"5min":1.635,"15min":1.6990000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.087,"15min":1.0720000000000001},"min":{"1min":0.89500000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.371,"5min":1.724,"15min":1.97},"last":0.79200000000000004}]},{"osd":1,"last update":"Wed Oct 18 09:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.1990000000000001,"15min":1.167},"min":{"1min":0.89500000000000002,"5min":0.67200000000000004,"15min":0.61799999999999999},"max":{"1min":1.591,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.222,"5min":1.1299999999999999,"15min":1.135},"min":{"1min":0.93899999999999995,"5min":0.72799999999999998,"15min":0.58799999999999997},"max":{"1min":1.661,"5min":1.661,"15min":1.7989999999999999},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.345,"15min":1.2909999999999999},"min":{"1min":0.96699999999999997,"5min":0.82199999999999995,"15min":0.65400000000000003},"max":{"1min":1.617,"5min":1.8169999999999999,"15min":1.9730000000000001},"last":0.88},{"interface":"front","average":{"1min":1.212,"5min":1.2490000000000001,"15min":1.2330000000000001},"min":{"1min":0.88400000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.532,"5min":1.863,"15min":1.863},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.306,"15min":1.28},"min":{"1min":0.92400000000000004,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.7330000000000001,"5min":1.98,"15min":1.98},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.3720000000000001,"15min":1.349},"min":{"1min":1.131,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7110000000000001,"5min":1.7210000000000001,"15min":1.9930000000000001},"last":0.94899999999999995}]},{"osd":4,"last update":"Wed Oct 18 09:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.202,"15min":1.2050000000000001},"min":{"1min":0.85499999999999998,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5589999999999999,"5min":1.923,"15min":2.1070000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.333,"15min":1.359},"min":{"1min":0.98999999999999999,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5669999999999999,"5min":2.0369999999999999,"15min":2.5169999999999999},"last":0.98499999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3240000000000001,"15min":1.3460000000000001},"min":{"1min":0.85999999999999999,"5min":0.622,"15min":0.622},"max":{"1min":1.7589999999999999,"5min":1.7989999999999999,"15min":2.0720000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.3879999999999999,"15min":1.413},"min":{"1min":1.1439999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.615,"5min":1.956,"15min":2.0270000000000001},"last":0.56000000000000005}]},{"osd":6,"last update":"Wed Oct 18 09:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.48,"5min":1.4690000000000001,"15min":1.4750000000000001},"min":{"1min":1.103,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.6779999999999999,"5min":2.0779999999999998,"15min":2.3959999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.484,"15min":1.4950000000000001},"min":{"1min":1.127,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.7350000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.82199999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574997,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":991,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194320,"statfs":{"total":200000000,"available":198984577,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.252,"15min":1.1910000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.60299999999999998},"max":{"1min":1.595,"5min":1.643,"15min":1.6850000000000001},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3580000000000001,"15min":1.3080000000000001},"min":{"1min":1.0720000000000001,"5min":0.81799999999999995,"15min":0.64700000000000002},"max":{"1min":1.528,"5min":1.899,"15min":2.0990000000000002},"last":1.0720000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1439999999999999,"15min":1.115},"min":{"1min":0.78000000000000003,"5min":0.63900000000000001,"15min":0.63800000000000001},"max":{"1min":1.5,"5min":1.722,"15min":1.722},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.236,"15min":1.216},"min":{"1min":0.78500000000000003,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.774,"5min":1.8169999999999999,"15min":2.9609999999999999},"last":1.123}]},{"osd":2,"last update":"Wed Oct 18 09:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.202,"15min":1.1850000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.69899999999999995},"max":{"1min":1.6040000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.292,"15min":1.2470000000000001},"min":{"1min":0.86199999999999999,"5min":0.73399999999999999,"15min":0.66300000000000003},"max":{"1min":1.663,"5min":1.75,"15min":2.1779999999999999},"last":0.86199999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.288,"15min":1.2749999999999999},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.79200000000000004},"max":{"1min":1.7050000000000001,"5min":1.8160000000000001,"15min":2.1589999999999998},"last":1.613},{"interface":"front","average":{"1min":1.355,"5min":1.3600000000000001,"15min":1.2889999999999999},"min":{"1min":1.0049999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.6339999999999999,"5min":1.994,"15min":1.994},"last":1.3839999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3400000000000001,"15min":1.2609999999999999},"min":{"1min":0.93400000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.794,"5min":1.9770000000000001,"15min":2.444},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.343,"15min":1.3029999999999999},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.55000000000000004},"max":{"1min":1.6799999999999999,"5min":1.9159999999999999,"15min":2.4039999999999999},"last":1.5129999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4379999999999999,"15min":1.4079999999999999},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.58899999999999997},"max":{"1min":1.8340000000000001,"5min":2.016,"15min":2.032},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.387,"15min":1.3069999999999999},"min":{"1min":0.88600000000000001,"5min":0.55100000000000005,"15min":0.49199999999999999},"max":{"1min":1.593,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.1779999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4570000000000001,"15min":1.425},"min":{"1min":1.016,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.708,"5min":2.028,"15min":2.3540000000000001},"last":1.016},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.552,"15min":1.534},"min":{"1min":1.1890000000000001,"5min":1.087,"15min":0.68500000000000005},"max":{"1min":1.9319999999999999,"5min":2.1440000000000001,"15min":2.794},"last":1.7130000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574998,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":991,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194320,"statfs":{"total":200000000,"available":198984701,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.3480000000000001,"15min":1.2989999999999999},"min":{"1min":1.0669999999999999,"5min":0.68300000000000005,"15min":0.63400000000000001},"max":{"1min":1.6439999999999999,"5min":9.609,"15min":9.609},"last":1.722},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.321,"15min":1.3149999999999999},"min":{"1min":1.0820000000000001,"5min":0.754,"15min":0.60799999999999998},"max":{"1min":1.6339999999999999,"5min":2.0430000000000001,"15min":5.3029999999999999},"last":1.694}]},{"osd":1,"last update":"Wed Oct 18 09:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.3089999999999999,"15min":1.244},"min":{"1min":0.83999999999999997,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.885,"5min":11.68,"15min":11.68},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.198,"15min":1.216},"min":{"1min":0.92700000000000005,"5min":0.70099999999999996,"15min":0.67000000000000004},"max":{"1min":1.5800000000000001,"5min":1.885,"15min":3.7050000000000001},"last":1.004}]},{"osd":2,"last update":"Wed Oct 18 09:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.252,"15min":1.256},"min":{"1min":1.012,"5min":0.76800000000000002,"15min":0.755},"max":{"1min":1.6579999999999999,"5min":1.732,"15min":2.2090000000000001},"last":1.667},{"interface":"front","average":{"1min":1.29,"5min":1.2769999999999999,"15min":1.274},"min":{"1min":0.94499999999999995,"5min":0.79000000000000004,"15min":0.72399999999999998},"max":{"1min":1.698,"5min":1.7270000000000001,"15min":3.8180000000000001},"last":1.46}]},{"osd":3,"last update":"Wed Oct 18 09:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.3029999999999999,"15min":1.26},"min":{"1min":0.95599999999999996,"5min":0.72299999999999998,"15min":0.65400000000000003},"max":{"1min":1.409,"5min":11.797000000000001,"15min":11.797000000000001},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.3919999999999999,"15min":1.3620000000000001},"min":{"1min":1.125,"5min":0.747,"15min":0.747},"max":{"1min":2.105,"5min":2.105,"15min":2.3420000000000001},"last":1.381}]},{"osd":4,"last update":"Wed Oct 18 09:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.351,"15min":1.387},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.028,"5min":2.0680000000000001,"15min":2.4449999999999998},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.278,"15min":1.383},"min":{"1min":0.91400000000000003,"5min":0.83699999999999997,"15min":0.64100000000000001},"max":{"1min":1.7949999999999999,"5min":2.1360000000000001,"15min":5.4580000000000002},"last":1.635}]},{"osd":6,"last update":"Wed Oct 18 09:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.2330000000000001,"15min":1.3160000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.67400000000000004},"max":{"1min":1.7410000000000001,"5min":2.1030000000000002,"15min":5.141},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.569,"5min":1.5329999999999999,"15min":1.5409999999999999},"min":{"1min":1.1910000000000001,"5min":0.82499999999999996,"15min":0.71099999999999997},"max":{"1min":2.161,"5min":2.161,"15min":2.657},"last":1.52}]},{"osd":7,"last update":"Wed Oct 18 09:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.5720000000000001,"15min":1.5820000000000001},"min":{"1min":1.073,"5min":1.0529999999999999,"15min":0.89500000000000002},"max":{"1min":1.976,"5min":11.829000000000001,"15min":11.829000000000001},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.385,"15min":1.454},"min":{"1min":0.96099999999999997,"5min":0.93300000000000005,"15min":0.73399999999999999},"max":{"1min":1.8560000000000001,"5min":2.6419999999999999,"15min":4.8419999999999996},"last":1.496}]}]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":984,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194328,"statfs":{"total":200000000,"available":198992042,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:06:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1559999999999999,"15min":1.1319999999999999},"min":{"1min":0.83499999999999996,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.54,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.1579999999999999,"15min":1.1519999999999999},"min":{"1min":0.98199999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5920000000000001,"5min":3.234,"15min":3.234},"last":1.0129999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:06:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.1950000000000001,"15min":1.1339999999999999},"min":{"1min":0.82799999999999996,"5min":0.64700000000000002,"15min":0.60799999999999998},"max":{"1min":1.9690000000000001,"5min":3.718,"15min":3.718},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.238,"15min":1.2190000000000001},"min":{"1min":0.97799999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5660000000000001,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.135}]},{"osd":2,"last update":"Wed Oct 18 09:06:28 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2310000000000001,"15min":1.1990000000000001},"min":{"1min":0.89900000000000002,"5min":0.71799999999999997,"15min":0.56699999999999995},"max":{"1min":1.905,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3300000000000001,"15min":1.3049999999999999},"min":{"1min":0.99399999999999999,"5min":0.79500000000000004,"15min":0.58899999999999997},"max":{"1min":1.679,"5min":3.532,"15min":3.532},"last":0.95499999999999996}]},{"osd":3,"last update":"Wed Oct 18 09:06:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.282,"15min":1.2829999999999999},"min":{"1min":1.032,"5min":0.73899999999999999,"15min":0.67000000000000004},"max":{"1min":1.669,"5min":3.363,"15min":3.363},"last":1.369},{"interface":"front","average":{"1min":1.399,"5min":1.403,"15min":1.4039999999999999},"min":{"1min":1.089,"5min":0.84999999999999998,"15min":0.68899999999999995},"max":{"1min":1.714,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":1.512}]},{"osd":5,"last update":"Wed Oct 18 09:06:28 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.3,"15min":1.2969999999999999},"min":{"1min":0.78500000000000003,"5min":0.55600000000000005,"15min":0.498},"max":{"1min":1.8200000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":1.417},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3260000000000001,"15min":1.329},"min":{"1min":0.91700000000000004,"5min":0.60099999999999998,"15min":0.38400000000000001},"max":{"1min":1.663,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":1.544}]},{"osd":6,"last update":"Wed Oct 18 09:06:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.3779999999999999,"15min":1.393},"min":{"1min":0.92800000000000005,"5min":0.67400000000000004,"15min":0.51600000000000001},"max":{"1min":1.766,"5min":4.4429999999999996,"15min":5.0049999999999999},"last":1.276},{"interface":"front","average":{"1min":1.383,"5min":1.478,"15min":1.5109999999999999},"min":{"1min":0.90100000000000002,"5min":0.47499999999999998,"15min":0.45400000000000001},"max":{"1min":1.7829999999999999,"5min":4.1239999999999997,"15min":4.407},"last":1.321}]},{"osd":7,"last update":"Wed Oct 18 09:06:28 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4339999999999999,"15min":1.464},"min":{"1min":1.0429999999999999,"5min":0.51200000000000001,"15min":0.438},"max":{"1min":1.6859999999999999,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":1.49},{"interface":"front","average":{"1min":1.456,"5min":1.5489999999999999,"15min":1.542},"min":{"1min":1.028,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.8520000000000001,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":1.5669999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574999,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":991,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194320,"statfs":{"total":200000000,"available":198984633,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:06:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.92000000000000004,"15min":0.87},"min":{"1min":0.66800000000000004,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1739999999999999,"5min":4.774,"15min":5.5730000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.181,"5min":0.97499999999999998,"15min":0.95299999999999996},"min":{"1min":0.66700000000000004,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.863,"5min":3.2370000000000001,"15min":6.5599999999999996},"last":0.90700000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:06:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.0049999999999999,"15min":0.998},"min":{"1min":0.754,"5min":0.39100000000000001,"15min":0.36799999999999999},"max":{"1min":1.7190000000000001,"5min":2.2650000000000001,"15min":6.8099999999999996},"last":1.155},{"interface":"front","average":{"1min":1.478,"5min":1.2749999999999999,"15min":1.2170000000000001},"min":{"1min":1.089,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":3.1920000000000002},"last":1.298}]},{"osd":2,"last update":"Wed Oct 18 09:06:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.131,"15min":1.0840000000000001},"min":{"1min":0.86499999999999999,"5min":0.46800000000000003,"15min":0.45300000000000001},"max":{"1min":1.357,"5min":2.6269999999999998,"15min":7.4580000000000002},"last":1.044},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2350000000000001,"15min":1.1599999999999999},"min":{"1min":0.998,"5min":0.52700000000000002,"15min":0.51400000000000001},"max":{"1min":1.639,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.101}]},{"osd":4,"last update":"Wed Oct 18 09:06:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.357,"15min":1.371},"min":{"1min":1.161,"5min":0.65400000000000003,"15min":0.53700000000000003},"max":{"1min":1.6890000000000001,"5min":3.1139999999999999,"15min":18.664999999999999},"last":1.744},{"interface":"front","average":{"1min":1.415,"5min":1.349,"15min":1.268},"min":{"1min":1.121,"5min":0.878,"15min":0.64000000000000001},"max":{"1min":1.7909999999999999,"5min":2.0289999999999999,"15min":7.0700000000000003},"last":1.4490000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:06:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.365,"15min":1.302},"min":{"1min":1.073,"5min":0.626,"15min":0.626},"max":{"1min":1.7190000000000001,"5min":1.925,"15min":6.9139999999999997},"last":1.637},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4770000000000001,"15min":1.454},"min":{"1min":1.111,"5min":0.72999999999999998,"15min":0.56899999999999995},"max":{"1min":1.8089999999999999,"5min":4.5540000000000003,"15min":16.390999999999998},"last":1.216}]},{"osd":6,"last update":"Wed Oct 18 09:06:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5109999999999999,"15min":1.4830000000000001},"min":{"1min":1.2150000000000001,"5min":0.77200000000000002,"15min":0.58199999999999996},"max":{"1min":2.2839999999999998,"5min":3.375,"15min":7.7270000000000003},"last":1.681},{"interface":"front","average":{"1min":1.633,"5min":1.514,"15min":1.446},"min":{"1min":1.3740000000000001,"5min":0.872,"15min":0.60199999999999998},"max":{"1min":1.8839999999999999,"5min":2.4860000000000002,"15min":7.242},"last":1.8100000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:06:20 2023","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":1.621,"15min":1.488},"min":{"1min":1.393,"5min":0.84099999999999997,"15min":0.70099999999999996},"max":{"1min":1.8959999999999999,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":1.349},{"interface":"front","average":{"1min":1.706,"5min":1.6140000000000001,"15min":1.55},"min":{"1min":1.3540000000000001,"5min":0.82099999999999995,"15min":0.622},"max":{"1min":2.375,"5min":5.4900000000000002,"15min":12.946},"last":1.8660000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":986,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194325,"statfs":{"total":200000000,"available":198989341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.86499999999999999,"15min":0.89400000000000002},"min":{"1min":0.71099999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.52,"5min":1.5660000000000001,"15min":4.2210000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.044,"5min":1.014,"15min":1.0169999999999999},"min":{"1min":0.747,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.5860000000000001,"5min":4.3959999999999999,"15min":4.3959999999999999},"last":0.94399999999999995}]},{"osd":1,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.125,"15min":1.0780000000000001},"min":{"1min":0.82099999999999995,"5min":0.55500000000000005,"15min":0.49099999999999999},"max":{"1min":1.5640000000000001,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.081,"15min":1.101},"min":{"1min":0.80200000000000005,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":1.46,"5min":2.448,"15min":2.5350000000000001},"last":1.282}]},{"osd":3,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.159,"15min":1.1339999999999999},"min":{"1min":0.87,"5min":0.48299999999999998,"15min":0.48199999999999998},"max":{"1min":1.696,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.3069999999999999,"15min":1.3899999999999999},"min":{"1min":1.1539999999999999,"5min":0.60199999999999998,"15min":0.54100000000000004},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":7.5570000000000004},"last":1.1020000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.4259999999999999,"15min":1.4510000000000001},"min":{"1min":1.28,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.8799999999999999,"5min":2.3809999999999998,"15min":7.6189999999999998},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.474,"5min":1.411,"15min":1.4399999999999999},"min":{"1min":1.0609999999999999,"5min":0.89200000000000002,"15min":0.86599999999999999},"max":{"1min":1.6819999999999999,"5min":3.423,"15min":3.423},"last":1.3540000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3700000000000001,"15min":1.3089999999999999},"min":{"1min":1.139,"5min":0.754,"15min":0.61599999999999999},"max":{"1min":1.6399999999999999,"5min":10.507,"15min":10.507},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4039999999999999,"15min":1.4910000000000001},"min":{"1min":1.147,"5min":0.81599999999999995,"15min":0.81399999999999995},"max":{"1min":1.7370000000000001,"5min":1.972,"15min":7.657},"last":1.7110000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.413,"15min":1.478},"min":{"1min":1.095,"5min":0.95999999999999996,"15min":0.74299999999999999},"max":{"1min":1.754,"5min":2.8410000000000002,"15min":3.5299999999999998},"last":1.843},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.516,"15min":1.6120000000000001},"min":{"1min":1.2330000000000001,"5min":0.80200000000000005,"15min":0.77600000000000002},"max":{"1min":2.2639999999999998,"5min":10.561,"15min":37.771999999999998},"last":1.538}]},{"osd":7,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.514,"15min":1.5449999999999999},"min":{"1min":1.274,"5min":0.93999999999999995,"15min":0.83199999999999996},"max":{"1min":1.7729999999999999,"5min":2.7890000000000001,"15min":3.3620000000000001},"last":1.9359999999999999},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.581,"15min":1.524},"min":{"1min":1.375,"5min":0.90200000000000002,"15min":0.77600000000000002},"max":{"1min":2.375,"5min":10.956,"15min":10.956},"last":1.7549999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":984,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194327,"statfs":{"total":200000000,"available":198991460,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92600000000000005,"15min":0.86499999999999999},"min":{"1min":0.67500000000000004,"5min":0.37,"15min":0.315},"max":{"1min":1.4770000000000001,"5min":1.659,"15min":1.659},"last":1.464},{"interface":"front","average":{"1min":1.268,"5min":1.127,"15min":1.014},"min":{"1min":1.002,"5min":0.441,"15min":0.441},"max":{"1min":1.601,"5min":1.681,"15min":1.7010000000000001},"last":1.0229999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.073,"15min":0.997},"min":{"1min":0.78200000000000003,"5min":0.317,"15min":0.317},"max":{"1min":1.446,"5min":1.589,"15min":1.859},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":0.999,"15min":0.92200000000000004},"min":{"1min":0.81899999999999995,"5min":0.38600000000000001,"15min":0.36699999999999999},"max":{"1min":1.8009999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.3400000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.161,"15min":1.0780000000000001},"min":{"1min":0.94499999999999995,"5min":0.46899999999999997,"15min":0.436},"max":{"1min":1.798,"5min":1.915,"15min":2.1059999999999999},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.1060000000000001,"15min":1.075},"min":{"1min":0.877,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.6659999999999999,"5min":1.736,"15min":3.7010000000000001},"last":0.82099999999999995}]},{"osd":4,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3080000000000001,"15min":1.2829999999999999},"min":{"1min":1.0489999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.657,"5min":1.7430000000000001,"15min":2.0489999999999999},"last":1.556},{"interface":"front","average":{"1min":1.415,"5min":1.387,"15min":1.2909999999999999},"min":{"1min":1.0920000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.702,"5min":1.893,"15min":2.0470000000000002},"last":1.7190000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":1.4630000000000001,"15min":1.369},"min":{"1min":1.161,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.905,"5min":1.984,"15min":3.5750000000000002},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.571,"5min":1.484,"15min":1.417},"min":{"1min":1.3200000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.839,"5min":2.0150000000000001,"15min":2.3769999999999998},"last":1.6499999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.462,"15min":1.3660000000000001},"min":{"1min":1.21,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.9410000000000001,"5min":2.0419999999999998,"15min":2.2050000000000001},"last":1.603},{"interface":"front","average":{"1min":1.528,"5min":1.4399999999999999,"15min":1.3540000000000001},"min":{"1min":1.153,"5min":0.73399999999999999,"15min":0.63100000000000001},"max":{"1min":1.804,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.298}]},{"osd":7,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.526,"15min":1.4379999999999999},"min":{"1min":1.218,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.004,"5min":2.2000000000000002,"15min":2.2229999999999999},"last":1.429},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.492,"15min":1.4330000000000001},"min":{"1min":1.1439999999999999,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.948,"5min":2.2469999999999999,"15min":2.4180000000000001},"last":1.7649999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1562,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193750,"statfs":{"total":200000000,"available":198400028,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83099999999999996,"15min":0.86799999999999999},"min":{"1min":0.66900000000000004,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0249999999999999,"5min":1.3320000000000001,"15min":3.4420000000000002},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.0820000000000001,"15min":1.087},"min":{"1min":0.90100000000000002,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.657,"5min":1.806,"15min":1.8240000000000001},"last":1.016}]},{"osd":2,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.82299999999999995,"15min":0.84399999999999997},"min":{"1min":0.77100000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.0720000000000001,"5min":1.679,"15min":1.679},"last":1.077},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.1439999999999999,"15min":1.141},"min":{"1min":0.95699999999999996,"5min":0.503,"15min":0.437},"max":{"1min":1.6779999999999999,"5min":1.718,"15min":1.929},"last":0.90800000000000003}]},{"osd":3,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.113,"15min":1.2},"min":{"1min":0.95199999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.619,"5min":1.98,"15min":2.7639999999999998},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.369,"15min":1.444},"min":{"1min":1.202,"5min":0.61299999999999999,"15min":0.58399999999999996},"max":{"1min":1.7809999999999999,"5min":2.1200000000000001,"15min":3.5720000000000001},"last":0.81200000000000006}]},{"osd":4,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.1990000000000001,"15min":1.2549999999999999},"min":{"1min":0.90100000000000002,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.696,"5min":1.9990000000000001,"15min":4.492},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.28,"15min":1.2709999999999999},"min":{"1min":1.0980000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.659,"5min":1.927,"15min":1.927},"last":1.2929999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.2729999999999999,"15min":1.339},"min":{"1min":1.018,"5min":0.75600000000000001,"15min":0.64300000000000002},"max":{"1min":1.6439999999999999,"5min":2.1779999999999999,"15min":4.2779999999999996},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.3160000000000001,"15min":1.345},"min":{"1min":1.1799999999999999,"5min":0.74399999999999999,"15min":0.73099999999999998},"max":{"1min":1.6879999999999999,"5min":1.847,"15min":1.8740000000000001},"last":1.6180000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.401,"15min":1.5},"min":{"1min":1.2350000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7769999999999999,"5min":2.2469999999999999,"15min":3.5299999999999998},"last":1.776},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.331,"15min":1.423},"min":{"1min":1.196,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.7230000000000001,"5min":3.1440000000000001,"15min":4.4349999999999996},"last":1.748}]},{"osd":7,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.5029999999999999,"15min":1.579},"min":{"1min":1.3240000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.8779999999999999,"5min":2.2629999999999999,"15min":3.492},"last":1.9370000000000001},{"interface":"front","average":{"1min":1.569,"5min":1.4610000000000001,"15min":1.556},"min":{"1min":1.298,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.7849999999999999,"5min":3.2570000000000001,"15min":4.5419999999999998},"last":2.0550000000000002}]}]}],"pool_statfs":[]}} 2023-10-18T09:07:21.902 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:07:22.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.168+0000 7f38bf305640 1 -- 172.21.15.40:0/1174740085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38b8159920 msgr2=0x7f38b815dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.168+0000 7f38bf305640 1 --2- 172.21.15.40:0/1174740085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38b8159920 0x7f38b815dd30 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009830 tx=0x7f38a802bd60 comp rx=0 tx=0).stop 2023-10-18T09:07:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.168+0000 7f38bf305640 1 -- 172.21.15.40:0/1174740085 shutdown_connections 2023-10-18T09:07:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.168+0000 7f38bf305640 1 --2- 172.21.15.40:0/1174740085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38b8159920 0x7f38b815dd30 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.168+0000 7f38bf305640 1 -- 172.21.15.40:0/1174740085 >> 172.21.15.40:0/1174740085 conn(0x7f38b80cf3e0 msgr2=0x7f38b80cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:22.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.168+0000 7f38bf305640 1 -- 172.21.15.40:0/1174740085 shutdown_connections 2023-10-18T09:07:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.168+0000 7f38bf305640 1 -- 172.21.15.40:0/1174740085 wait complete. 2023-10-18T09:07:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.170+0000 7f38bf305640 1 Processor -- start 2023-10-18T09:07:22.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.170+0000 7f38bf305640 1 -- start start 2023-10-18T09:07:22.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.171+0000 7f38bf305640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38b80b4fc0 0x7f38b80c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.171+0000 7f38bf305640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38b80c99a0 0x7f38b81f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.171+0000 7f38bf305640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38b8159920 0x7f38b80aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.171+0000 7f38bf305640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b81f3310 con 0x7f38b80b4fc0 2023-10-18T09:07:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.171+0000 7f38bf305640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b81f2a10 con 0x7f38b80c99a0 2023-10-18T09:07:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.171+0000 7f38bf305640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b81f2cf0 con 0x7f38b8159920 2023-10-18T09:07:22.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.171+0000 7f38bd0a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38b80b4fc0 0x7f38b80c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.171+0000 7f38bd0a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38b80b4fc0 0x7f38b80c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46474/0 (socket says 172.21.15.40:46474) 2023-10-18T09:07:22.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.171+0000 7f38bd0a1640 1 -- 172.21.15.40:0/4054908233 learned_addr learned my addr 172.21.15.40:0/4054908233 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.172+0000 7f38bc8a0640 1 --2- 172.21.15.40:0/4054908233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38b80c99a0 0x7f38b81f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:22.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.172+0000 7f38bd0a1640 1 -- 172.21.15.40:0/4054908233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38b8159920 msgr2=0x7f38b80aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:22.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.172+0000 7f38bd0a1640 1 --2- 172.21.15.40:0/4054908233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38b8159920 0x7f38b80aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.172+0000 7f38bd0a1640 1 -- 172.21.15.40:0/4054908233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38b80c99a0 msgr2=0x7f38b81f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:22.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.172+0000 7f38bd0a1640 1 --2- 172.21.15.40:0/4054908233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38b80c99a0 0x7f38b81f2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.172+0000 7f38bd0a1640 1 -- 172.21.15.40:0/4054908233 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a8009490 con 0x7f38b80b4fc0 2023-10-18T09:07:22.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.172+0000 7f38bd0a1640 1 --2- 172.21.15.40:0/4054908233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38b80b4fc0 0x7f38b80c58a0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f38b400cc50 tx=0x7f38b400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:22.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.172+0000 7f38adffb640 1 -- 172.21.15.40:0/4054908233 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b400ce60 con 0x7f38b80b4fc0 2023-10-18T09:07:22.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.172+0000 7f38adffb640 1 -- 172.21.15.40:0/4054908233 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f38b4012420 con 0x7f38b80b4fc0 2023-10-18T09:07:22.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.172+0000 7f38adffb640 1 -- 172.21.15.40:0/4054908233 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b4012810 con 0x7f38b80b4fc0 2023-10-18T09:07:22.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.172+0000 7f38bf305640 1 -- 172.21.15.40:0/4054908233 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b81f2fd0 con 0x7f38b80b4fc0 2023-10-18T09:07:22.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.173+0000 7f38bf305640 1 -- 172.21.15.40:0/4054908233 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b81f2640 con 0x7f38b80b4fc0 2023-10-18T09:07:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.174+0000 7f38bf305640 1 -- 172.21.15.40:0/4054908233 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b8066d40 con 0x7f38b80b4fc0 2023-10-18T09:07:22.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.175+0000 7f38adffb640 1 -- 172.21.15.40:0/4054908233 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f38b401a3b0 con 0x7f38b80b4fc0 2023-10-18T09:07:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.175+0000 7f38adffb640 1 --2- 172.21.15.40:0/4054908233 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f389c06ae40 0x7f389c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.176+0000 7f38adffb640 1 -- 172.21.15.40:0/4054908233 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38b4089f20 con 0x7f38b80b4fc0 2023-10-18T09:07:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.176+0000 7f38bc8a0640 1 --2- 172.21.15.40:0/4054908233 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f389c06ae40 0x7f389c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:22.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.177+0000 7f38bc8a0640 1 --2- 172.21.15.40:0/4054908233 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f389c06ae40 0x7f389c06d300 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f38a8036000 tx=0x7f38a8004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:22.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.180+0000 7f38adffb640 1 -- 172.21.15.40:0/4054908233 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38b4054fd0 con 0x7f38b80b4fc0 2023-10-18T09:07:22.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.372+0000 7f38bf305640 1 -- 172.21.15.40:0/4054908233 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38b81f2e60 con 0x7f389c06ae40 2023-10-18T09:07:22.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.394+0000 7f38adffb640 1 -- 172.21.15.40:0/4054908233 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281945 (secure 0 0 0) 0x7f38b81f2e60 con 0x7f389c06ae40 2023-10-18T09:07:22.402 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:07:22.404 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:07:22.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.400+0000 7f38bf305640 1 -- 172.21.15.40:0/4054908233 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f389c06ae40 msgr2=0x7f389c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:22.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.400+0000 7f38bf305640 1 --2- 172.21.15.40:0/4054908233 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f389c06ae40 0x7f389c06d300 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f38a8036000 tx=0x7f38a8004730 comp rx=0 tx=0).stop 2023-10-18T09:07:22.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.400+0000 7f38bf305640 1 -- 172.21.15.40:0/4054908233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38b80b4fc0 msgr2=0x7f38b80c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:22.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.400+0000 7f38bf305640 1 --2- 172.21.15.40:0/4054908233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38b80b4fc0 0x7f38b80c58a0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f38b400cc50 tx=0x7f38b400a430 comp rx=0 tx=0).stop 2023-10-18T09:07:22.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.400+0000 7f38bf305640 1 -- 172.21.15.40:0/4054908233 shutdown_connections 2023-10-18T09:07:22.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.400+0000 7f38bf305640 1 --2- 172.21.15.40:0/4054908233 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f389c06ae40 0x7f389c06d300 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.400+0000 7f38bf305640 1 --2- 172.21.15.40:0/4054908233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38b8159920 0x7f38b80aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.400+0000 7f38bf305640 1 --2- 172.21.15.40:0/4054908233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38b80c99a0 0x7f38b81f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.400+0000 7f38bf305640 1 --2- 172.21.15.40:0/4054908233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38b80b4fc0 0x7f38b80c58a0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.400+0000 7f38bf305640 1 -- 172.21.15.40:0/4054908233 >> 172.21.15.40:0/4054908233 conn(0x7f38b80cf3e0 msgr2=0x7f38b80b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:22.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.400+0000 7f38bf305640 1 -- 172.21.15.40:0/4054908233 shutdown_connections 2023-10-18T09:07:22.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.400+0000 7f38bf305640 1 -- 172.21.15.40:0/4054908233 wait complete. 2023-10-18T09:07:22.424 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":541,"stamp":"2023-10-18T09:07:21.163496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9062,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553437,"statfs":{"total":1600000000,"available":1590720485,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.827232"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.380074+0000","last_change":"2023-10-18T09:07:17.379695+0000","last_active":"2023-10-18T09:07:17.380074+0000","last_peered":"2023-10-18T09:07:17.380074+0000","last_clean":"2023-10-18T09:07:17.380074+0000","last_became_active":"2023-10-18T09:07:17.379366+0000","last_became_peered":"2023-10-18T09:07:17.379366+0000","last_unstale":"2023-10-18T09:07:17.380074+0000","last_undegraded":"2023-10-18T09:07:17.380074+0000","last_fullsized":"2023-10-18T09:07:17.380074+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:06:46.902419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365435+0000","last_change":"2023-10-18T09:07:17.385274+0000","last_active":"2023-10-18T09:07:18.365435+0000","last_peered":"2023-10-18T09:07:18.365435+0000","last_clean":"2023-10-18T09:07:18.365435+0000","last_became_active":"2023-10-18T09:07:17.385043+0000","last_became_peered":"2023-10-18T09:07:17.385043+0000","last_unstale":"2023-10-18T09:07:18.365435+0000","last_undegraded":"2023-10-18T09:07:18.365435+0000","last_fullsized":"2023-10-18T09:07:18.365435+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:10.523692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370896+0000","last_change":"2023-10-18T09:07:17.395122+0000","last_active":"2023-10-18T09:07:18.370896+0000","last_peered":"2023-10-18T09:07:18.370896+0000","last_clean":"2023-10-18T09:07:18.370896+0000","last_became_active":"2023-10-18T09:07:17.392503+0000","last_became_peered":"2023-10-18T09:07:17.392503+0000","last_unstale":"2023-10-18T09:07:18.370896+0000","last_undegraded":"2023-10-18T09:07:18.370896+0000","last_fullsized":"2023-10-18T09:07:18.370896+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:21:20.537938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.392796+0000","last_change":"2023-10-18T09:07:17.392552+0000","last_active":"2023-10-18T09:07:17.392796+0000","last_peered":"2023-10-18T09:07:17.392796+0000","last_clean":"2023-10-18T09:07:17.392796+0000","last_became_active":"2023-10-18T09:07:17.392049+0000","last_became_peered":"2023-10-18T09:07:17.392049+0000","last_unstale":"2023-10-18T09:07:17.392796+0000","last_undegraded":"2023-10-18T09:07:17.392796+0000","last_fullsized":"2023-10-18T09:07:17.392796+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:52:34.109568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742910+0000","last_change":"2023-10-18T09:07:17.399034+0000","last_active":"2023-10-18T09:07:18.742910+0000","last_peered":"2023-10-18T09:07:18.742910+0000","last_clean":"2023-10-18T09:07:18.742910+0000","last_became_active":"2023-10-18T09:07:17.398592+0000","last_became_peered":"2023-10-18T09:07:17.398592+0000","last_unstale":"2023-10-18T09:07:18.742910+0000","last_undegraded":"2023-10-18T09:07:18.742910+0000","last_fullsized":"2023-10-18T09:07:18.742910+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:09:18.970120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.395378+0000","last_change":"2023-10-18T09:07:17.395304+0000","last_active":"2023-10-18T09:07:17.395378+0000","last_peered":"2023-10-18T09:07:17.395378+0000","last_clean":"2023-10-18T09:07:17.395378+0000","last_became_active":"2023-10-18T09:07:17.394753+0000","last_became_peered":"2023-10-18T09:07:17.394753+0000","last_unstale":"2023-10-18T09:07:17.395378+0000","last_undegraded":"2023-10-18T09:07:17.395378+0000","last_fullsized":"2023-10-18T09:07:17.395378+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:54:07.611935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.741664+0000","last_change":"2023-10-18T09:07:17.387822+0000","last_active":"2023-10-18T09:07:18.741664+0000","last_peered":"2023-10-18T09:07:18.741664+0000","last_clean":"2023-10-18T09:07:18.741664+0000","last_became_active":"2023-10-18T09:07:17.387431+0000","last_became_peered":"2023-10-18T09:07:17.387431+0000","last_unstale":"2023-10-18T09:07:18.741664+0000","last_undegraded":"2023-10-18T09:07:18.741664+0000","last_fullsized":"2023-10-18T09:07:18.741664+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:41:56.193471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.390476+0000","last_change":"2023-10-18T09:07:17.390398+0000","last_active":"2023-10-18T09:07:17.390476+0000","last_peered":"2023-10-18T09:07:17.390476+0000","last_clean":"2023-10-18T09:07:17.390476+0000","last_became_active":"2023-10-18T09:07:17.390168+0000","last_became_peered":"2023-10-18T09:07:17.390168+0000","last_unstale":"2023-10-18T09:07:17.390476+0000","last_undegraded":"2023-10-18T09:07:17.390476+0000","last_fullsized":"2023-10-18T09:07:17.390476+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:46:23.898025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365283+0000","last_change":"2023-10-18T09:07:17.367980+0000","last_active":"2023-10-18T09:07:18.365283+0000","last_peered":"2023-10-18T09:07:18.365283+0000","last_clean":"2023-10-18T09:07:18.365283+0000","last_became_active":"2023-10-18T09:07:17.367695+0000","last_became_peered":"2023-10-18T09:07:17.367695+0000","last_unstale":"2023-10-18T09:07:18.365283+0000","last_undegraded":"2023-10-18T09:07:18.365283+0000","last_fullsized":"2023-10-18T09:07:18.365283+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:54.405679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.368755+0000","last_change":"2023-10-18T09:07:17.392047+0000","last_active":"2023-10-18T09:07:18.368755+0000","last_peered":"2023-10-18T09:07:18.368755+0000","last_clean":"2023-10-18T09:07:18.368755+0000","last_became_active":"2023-10-18T09:07:17.391271+0000","last_became_peered":"2023-10-18T09:07:17.391271+0000","last_unstale":"2023-10-18T09:07:18.368755+0000","last_undegraded":"2023-10-18T09:07:18.368755+0000","last_fullsized":"2023-10-18T09:07:18.368755+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:57:08.967354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.389830+0000","last_change":"2023-10-18T09:07:17.389715+0000","last_active":"2023-10-18T09:07:17.389830+0000","last_peered":"2023-10-18T09:07:17.389830+0000","last_clean":"2023-10-18T09:07:17.389830+0000","last_became_active":"2023-10-18T09:07:17.389450+0000","last_became_peered":"2023-10-18T09:07:17.389450+0000","last_unstale":"2023-10-18T09:07:17.389830+0000","last_undegraded":"2023-10-18T09:07:17.389830+0000","last_fullsized":"2023-10-18T09:07:17.389830+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:20:23.944169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.368289+0000","last_change":"2023-10-18T09:07:17.398342+0000","last_active":"2023-10-18T09:07:18.368289+0000","last_peered":"2023-10-18T09:07:18.368289+0000","last_clean":"2023-10-18T09:07:18.368289+0000","last_became_active":"2023-10-18T09:07:17.398068+0000","last_became_peered":"2023-10-18T09:07:17.398068+0000","last_unstale":"2023-10-18T09:07:18.368289+0000","last_undegraded":"2023-10-18T09:07:18.368289+0000","last_fullsized":"2023-10-18T09:07:18.368289+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:31:10.222264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742029+0000","last_change":"2023-10-18T09:07:17.401770+0000","last_active":"2023-10-18T09:07:18.742029+0000","last_peered":"2023-10-18T09:07:18.742029+0000","last_clean":"2023-10-18T09:07:18.742029+0000","last_became_active":"2023-10-18T09:07:17.401523+0000","last_became_peered":"2023-10-18T09:07:17.401523+0000","last_unstale":"2023-10-18T09:07:18.742029+0000","last_undegraded":"2023-10-18T09:07:18.742029+0000","last_fullsized":"2023-10-18T09:07:18.742029+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:34.862962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.391795+0000","last_change":"2023-10-18T09:07:17.389057+0000","last_active":"2023-10-18T09:07:17.391795+0000","last_peered":"2023-10-18T09:07:17.391795+0000","last_clean":"2023-10-18T09:07:17.391795+0000","last_became_active":"2023-10-18T09:07:17.388825+0000","last_became_peered":"2023-10-18T09:07:17.388825+0000","last_unstale":"2023-10-18T09:07:17.391795+0000","last_undegraded":"2023-10-18T09:07:17.391795+0000","last_fullsized":"2023-10-18T09:07:17.391795+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:00.445724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.369106+0000","last_change":"2023-10-18T09:07:17.386782+0000","last_active":"2023-10-18T09:07:18.369106+0000","last_peered":"2023-10-18T09:07:18.369106+0000","last_clean":"2023-10-18T09:07:18.369106+0000","last_became_active":"2023-10-18T09:07:17.383799+0000","last_became_peered":"2023-10-18T09:07:17.383799+0000","last_unstale":"2023-10-18T09:07:18.369106+0000","last_undegraded":"2023-10-18T09:07:18.369106+0000","last_fullsized":"2023-10-18T09:07:18.369106+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:56:48.932155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.379573+0000","last_change":"2023-10-18T09:07:17.379296+0000","last_active":"2023-10-18T09:07:17.379573+0000","last_peered":"2023-10-18T09:07:17.379573+0000","last_clean":"2023-10-18T09:07:17.379573+0000","last_became_active":"2023-10-18T09:07:17.379011+0000","last_became_peered":"2023-10-18T09:07:17.379011+0000","last_unstale":"2023-10-18T09:07:17.379573+0000","last_undegraded":"2023-10-18T09:07:17.379573+0000","last_fullsized":"2023-10-18T09:07:17.379573+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:16.378262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.368727+0000","last_change":"2023-10-18T09:07:17.395988+0000","last_active":"2023-10-18T09:07:18.368727+0000","last_peered":"2023-10-18T09:07:18.368727+0000","last_clean":"2023-10-18T09:07:18.368727+0000","last_became_active":"2023-10-18T09:07:17.395724+0000","last_became_peered":"2023-10-18T09:07:17.395724+0000","last_unstale":"2023-10-18T09:07:18.368727+0000","last_undegraded":"2023-10-18T09:07:18.368727+0000","last_fullsized":"2023-10-18T09:07:18.368727+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:39:57.360245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370531+0000","last_change":"2023-10-18T09:07:17.395692+0000","last_active":"2023-10-18T09:07:18.370531+0000","last_peered":"2023-10-18T09:07:18.370531+0000","last_clean":"2023-10-18T09:07:18.370531+0000","last_became_active":"2023-10-18T09:07:17.392786+0000","last_became_peered":"2023-10-18T09:07:17.392786+0000","last_unstale":"2023-10-18T09:07:18.370531+0000","last_undegraded":"2023-10-18T09:07:18.370531+0000","last_fullsized":"2023-10-18T09:07:18.370531+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:52.299550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.375290+0000","last_change":"2023-10-18T09:07:17.375158+0000","last_active":"2023-10-18T09:07:17.375290+0000","last_peered":"2023-10-18T09:07:17.375290+0000","last_clean":"2023-10-18T09:07:17.375290+0000","last_became_active":"2023-10-18T09:07:17.374911+0000","last_became_peered":"2023-10-18T09:07:17.374911+0000","last_unstale":"2023-10-18T09:07:17.375290+0000","last_undegraded":"2023-10-18T09:07:17.375290+0000","last_fullsized":"2023-10-18T09:07:17.375290+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:32:45.339436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.376030+0000","last_change":"2023-10-18T09:07:17.375892+0000","last_active":"2023-10-18T09:07:17.376030+0000","last_peered":"2023-10-18T09:07:17.376030+0000","last_clean":"2023-10-18T09:07:17.376030+0000","last_became_active":"2023-10-18T09:07:17.375639+0000","last_became_peered":"2023-10-18T09:07:17.375639+0000","last_unstale":"2023-10-18T09:07:17.376030+0000","last_undegraded":"2023-10-18T09:07:17.376030+0000","last_fullsized":"2023-10-18T09:07:17.376030+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:52:46.689238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.368748+0000","last_change":"2023-10-18T09:07:17.390459+0000","last_active":"2023-10-18T09:07:18.368748+0000","last_peered":"2023-10-18T09:07:18.368748+0000","last_clean":"2023-10-18T09:07:18.368748+0000","last_became_active":"2023-10-18T09:07:17.388674+0000","last_became_peered":"2023-10-18T09:07:17.388674+0000","last_unstale":"2023-10-18T09:07:18.368748+0000","last_undegraded":"2023-10-18T09:07:18.368748+0000","last_fullsized":"2023-10-18T09:07:18.368748+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:28:57.800636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.387182+0000","last_change":"2023-10-18T09:07:17.386576+0000","last_active":"2023-10-18T09:07:17.387182+0000","last_peered":"2023-10-18T09:07:17.387182+0000","last_clean":"2023-10-18T09:07:17.387182+0000","last_became_active":"2023-10-18T09:07:17.385339+0000","last_became_peered":"2023-10-18T09:07:17.385339+0000","last_unstale":"2023-10-18T09:07:17.387182+0000","last_undegraded":"2023-10-18T09:07:17.387182+0000","last_fullsized":"2023-10-18T09:07:17.387182+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:48.205449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.387163+0000","last_change":"2023-10-18T09:07:17.385674+0000","last_active":"2023-10-18T09:07:17.387163+0000","last_peered":"2023-10-18T09:07:17.387163+0000","last_clean":"2023-10-18T09:07:17.387163+0000","last_became_active":"2023-10-18T09:07:17.385410+0000","last_became_peered":"2023-10-18T09:07:17.385410+0000","last_unstale":"2023-10-18T09:07:17.387163+0000","last_undegraded":"2023-10-18T09:07:17.387163+0000","last_fullsized":"2023-10-18T09:07:17.387163+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:06:31.617560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370339+0000","last_change":"2023-10-18T09:07:17.390960+0000","last_active":"2023-10-18T09:07:18.370339+0000","last_peered":"2023-10-18T09:07:18.370339+0000","last_clean":"2023-10-18T09:07:18.370339+0000","last_became_active":"2023-10-18T09:07:17.390732+0000","last_became_peered":"2023-10-18T09:07:17.390732+0000","last_unstale":"2023-10-18T09:07:18.370339+0000","last_undegraded":"2023-10-18T09:07:18.370339+0000","last_fullsized":"2023-10-18T09:07:18.370339+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:01:17.686685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.374129+0000","last_change":"2023-10-18T09:07:17.374049+0000","last_active":"2023-10-18T09:07:17.374129+0000","last_peered":"2023-10-18T09:07:17.374129+0000","last_clean":"2023-10-18T09:07:17.374129+0000","last_became_active":"2023-10-18T09:07:17.373789+0000","last_became_peered":"2023-10-18T09:07:17.373789+0000","last_unstale":"2023-10-18T09:07:17.374129+0000","last_undegraded":"2023-10-18T09:07:17.374129+0000","last_fullsized":"2023-10-18T09:07:17.374129+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:26:42.168261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.376512+0000","last_change":"2023-10-18T09:07:17.376444+0000","last_active":"2023-10-18T09:07:17.376512+0000","last_peered":"2023-10-18T09:07:17.376512+0000","last_clean":"2023-10-18T09:07:17.376512+0000","last_became_active":"2023-10-18T09:07:17.376174+0000","last_became_peered":"2023-10-18T09:07:17.376174+0000","last_unstale":"2023-10-18T09:07:17.376512+0000","last_undegraded":"2023-10-18T09:07:17.376512+0000","last_fullsized":"2023-10-18T09:07:17.376512+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:35:49.646389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.395196+0000","last_change":"2023-10-18T09:07:17.395124+0000","last_active":"2023-10-18T09:07:17.395196+0000","last_peered":"2023-10-18T09:07:17.395196+0000","last_clean":"2023-10-18T09:07:17.395196+0000","last_became_active":"2023-10-18T09:07:17.394549+0000","last_became_peered":"2023-10-18T09:07:17.394549+0000","last_unstale":"2023-10-18T09:07:17.395196+0000","last_undegraded":"2023-10-18T09:07:17.395196+0000","last_fullsized":"2023-10-18T09:07:17.395196+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:02.434291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.741061+0000","last_change":"2023-10-18T09:07:17.396740+0000","last_active":"2023-10-18T09:07:18.741061+0000","last_peered":"2023-10-18T09:07:18.741061+0000","last_clean":"2023-10-18T09:07:18.741061+0000","last_became_active":"2023-10-18T09:07:17.396468+0000","last_became_peered":"2023-10-18T09:07:17.396468+0000","last_unstale":"2023-10-18T09:07:18.741061+0000","last_undegraded":"2023-10-18T09:07:18.741061+0000","last_fullsized":"2023-10-18T09:07:18.741061+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:57:51.612360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.382460+0000","last_change":"2023-10-18T09:07:17.382369+0000","last_active":"2023-10-18T09:07:17.382460+0000","last_peered":"2023-10-18T09:07:17.382460+0000","last_clean":"2023-10-18T09:07:17.382460+0000","last_became_active":"2023-10-18T09:07:17.381496+0000","last_became_peered":"2023-10-18T09:07:17.381496+0000","last_unstale":"2023-10-18T09:07:17.382460+0000","last_undegraded":"2023-10-18T09:07:17.382460+0000","last_fullsized":"2023-10-18T09:07:17.382460+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:22:26.250817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.385576+0000","last_change":"2023-10-18T09:07:17.384909+0000","last_active":"2023-10-18T09:07:17.385576+0000","last_peered":"2023-10-18T09:07:17.385576+0000","last_clean":"2023-10-18T09:07:17.385576+0000","last_became_active":"2023-10-18T09:07:17.384561+0000","last_became_peered":"2023-10-18T09:07:17.384561+0000","last_unstale":"2023-10-18T09:07:17.385576+0000","last_undegraded":"2023-10-18T09:07:17.385576+0000","last_fullsized":"2023-10-18T09:07:17.385576+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:19:19.451511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365387+0000","last_change":"2023-10-18T09:07:17.388521+0000","last_active":"2023-10-18T09:07:18.365387+0000","last_peered":"2023-10-18T09:07:18.365387+0000","last_clean":"2023-10-18T09:07:18.365387+0000","last_became_active":"2023-10-18T09:07:17.388256+0000","last_became_peered":"2023-10-18T09:07:17.388256+0000","last_unstale":"2023-10-18T09:07:18.365387+0000","last_undegraded":"2023-10-18T09:07:18.365387+0000","last_fullsized":"2023-10-18T09:07:18.365387+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:43:52.299550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.388691+0000","last_change":"2023-10-18T09:07:17.388582+0000","last_active":"2023-10-18T09:07:17.388691+0000","last_peered":"2023-10-18T09:07:17.388691+0000","last_clean":"2023-10-18T09:07:17.388691+0000","last_became_active":"2023-10-18T09:07:17.387812+0000","last_became_peered":"2023-10-18T09:07:17.387812+0000","last_unstale":"2023-10-18T09:07:17.388691+0000","last_undegraded":"2023-10-18T09:07:17.388691+0000","last_fullsized":"2023-10-18T09:07:17.388691+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:34:17.813007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370681+0000","last_change":"2023-10-18T09:07:17.399479+0000","last_active":"2023-10-18T09:07:18.370681+0000","last_peered":"2023-10-18T09:07:18.370681+0000","last_clean":"2023-10-18T09:07:18.370681+0000","last_became_active":"2023-10-18T09:07:17.399165+0000","last_became_peered":"2023-10-18T09:07:17.399165+0000","last_unstale":"2023-10-18T09:07:18.370681+0000","last_undegraded":"2023-10-18T09:07:18.370681+0000","last_fullsized":"2023-10-18T09:07:18.370681+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:25:17.355528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"178'4","reported_seq":16,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370385+0000","last_change":"2023-10-18T09:07:16.364042+0000","last_active":"2023-10-18T09:07:18.370385+0000","last_peered":"2023-10-18T09:07:18.370385+0000","last_clean":"2023-10-18T09:07:18.370385+0000","last_became_active":"2023-10-18T09:07:16.361240+0000","last_became_peered":"2023-10-18T09:07:16.361240+0000","last_unstale":"2023-10-18T09:07:18.370385+0000","last_undegraded":"2023-10-18T09:07:18.370385+0000","last_fullsized":"2023-10-18T09:07:18.370385+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:00:04.045218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365497+0000","last_change":"2023-10-18T09:07:17.383827+0000","last_active":"2023-10-18T09:07:18.365497+0000","last_peered":"2023-10-18T09:07:18.365497+0000","last_clean":"2023-10-18T09:07:18.365497+0000","last_became_active":"2023-10-18T09:07:17.383487+0000","last_became_peered":"2023-10-18T09:07:17.383487+0000","last_unstale":"2023-10-18T09:07:18.365497+0000","last_undegraded":"2023-10-18T09:07:18.365497+0000","last_fullsized":"2023-10-18T09:07:18.365497+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:21:20.537938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"178'1","reported_seq":13,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365456+0000","last_change":"2023-10-18T09:07:16.360111+0000","last_active":"2023-10-18T09:07:18.365456+0000","last_peered":"2023-10-18T09:07:18.365456+0000","last_clean":"2023-10-18T09:07:18.365456+0000","last_became_active":"2023-10-18T09:07:16.359724+0000","last_became_peered":"2023-10-18T09:07:16.359724+0000","last_unstale":"2023-10-18T09:07:18.365456+0000","last_undegraded":"2023-10-18T09:07:18.365456+0000","last_fullsized":"2023-10-18T09:07:18.365456+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:42:08.267937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.387475+0000","last_change":"2023-10-18T09:07:17.387271+0000","last_active":"2023-10-18T09:07:17.387475+0000","last_peered":"2023-10-18T09:07:17.387475+0000","last_clean":"2023-10-18T09:07:17.387475+0000","last_became_active":"2023-10-18T09:07:17.387002+0000","last_became_peered":"2023-10-18T09:07:17.387002+0000","last_unstale":"2023-10-18T09:07:17.387475+0000","last_undegraded":"2023-10-18T09:07:17.387475+0000","last_fullsized":"2023-10-18T09:07:17.387475+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:33:14.476512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"178'2","reported_seq":13,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.397483+0000","last_change":"2023-10-18T09:07:16.361045+0000","last_active":"2023-10-18T09:07:17.397483+0000","last_peered":"2023-10-18T09:07:17.397483+0000","last_clean":"2023-10-18T09:07:17.397483+0000","last_became_active":"2023-10-18T09:07:16.360550+0000","last_became_peered":"2023-10-18T09:07:16.360550+0000","last_unstale":"2023-10-18T09:07:17.397483+0000","last_undegraded":"2023-10-18T09:07:17.397483+0000","last_fullsized":"2023-10-18T09:07:17.397483+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:39:03.300238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.369441+0000","last_change":"2023-10-18T09:07:17.399057+0000","last_active":"2023-10-18T09:07:18.369441+0000","last_peered":"2023-10-18T09:07:18.369441+0000","last_clean":"2023-10-18T09:07:18.369441+0000","last_became_active":"2023-10-18T09:07:17.398309+0000","last_became_peered":"2023-10-18T09:07:17.398309+0000","last_unstale":"2023-10-18T09:07:18.369441+0000","last_undegraded":"2023-10-18T09:07:18.369441+0000","last_fullsized":"2023-10-18T09:07:18.369441+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:16:48.708074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"178'2","reported_seq":13,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.398633+0000","last_change":"2023-10-18T09:07:16.362256+0000","last_active":"2023-10-18T09:07:17.398633+0000","last_peered":"2023-10-18T09:07:17.398633+0000","last_clean":"2023-10-18T09:07:17.398633+0000","last_became_active":"2023-10-18T09:07:16.361241+0000","last_became_peered":"2023-10-18T09:07:16.361241+0000","last_unstale":"2023-10-18T09:07:17.398633+0000","last_undegraded":"2023-10-18T09:07:17.398633+0000","last_fullsized":"2023-10-18T09:07:17.398633+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:11:10.278131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365701+0000","last_change":"2023-10-18T09:07:17.396091+0000","last_active":"2023-10-18T09:07:18.365701+0000","last_peered":"2023-10-18T09:07:18.365701+0000","last_clean":"2023-10-18T09:07:18.365701+0000","last_became_active":"2023-10-18T09:07:17.395738+0000","last_became_peered":"2023-10-18T09:07:17.395738+0000","last_unstale":"2023-10-18T09:07:18.365701+0000","last_undegraded":"2023-10-18T09:07:18.365701+0000","last_fullsized":"2023-10-18T09:07:18.365701+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:00.445724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"178'1","reported_seq":12,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.389964+0000","last_change":"2023-10-18T09:07:16.361126+0000","last_active":"2023-10-18T09:07:17.389964+0000","last_peered":"2023-10-18T09:07:17.389964+0000","last_clean":"2023-10-18T09:07:17.389964+0000","last_became_active":"2023-10-18T09:07:16.360744+0000","last_became_peered":"2023-10-18T09:07:16.360744+0000","last_unstale":"2023-10-18T09:07:17.389964+0000","last_undegraded":"2023-10-18T09:07:17.389964+0000","last_fullsized":"2023-10-18T09:07:17.389964+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:22:41.018905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.397708+0000","last_change":"2023-10-18T09:07:17.395261+0000","last_active":"2023-10-18T09:07:17.397708+0000","last_peered":"2023-10-18T09:07:17.397708+0000","last_clean":"2023-10-18T09:07:17.397708+0000","last_became_active":"2023-10-18T09:07:17.394823+0000","last_became_peered":"2023-10-18T09:07:17.394823+0000","last_unstale":"2023-10-18T09:07:17.397708+0000","last_undegraded":"2023-10-18T09:07:17.397708+0000","last_fullsized":"2023-10-18T09:07:17.397708+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:04.413480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"178'1","reported_seq":13,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742940+0000","last_change":"2023-10-18T09:07:16.367941+0000","last_active":"2023-10-18T09:07:18.742940+0000","last_peered":"2023-10-18T09:07:18.742940+0000","last_clean":"2023-10-18T09:07:18.742940+0000","last_became_active":"2023-10-18T09:07:16.367443+0000","last_became_peered":"2023-10-18T09:07:16.367443+0000","last_unstale":"2023-10-18T09:07:18.742940+0000","last_undegraded":"2023-10-18T09:07:18.742940+0000","last_fullsized":"2023-10-18T09:07:18.742940+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:28:55.503205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.366030+0000","last_change":"2023-10-18T09:07:17.390110+0000","last_active":"2023-10-18T09:07:18.366030+0000","last_peered":"2023-10-18T09:07:18.366030+0000","last_clean":"2023-10-18T09:07:18.366030+0000","last_became_active":"2023-10-18T09:07:17.389398+0000","last_became_peered":"2023-10-18T09:07:17.389398+0000","last_unstale":"2023-10-18T09:07:18.366030+0000","last_undegraded":"2023-10-18T09:07:18.366030+0000","last_fullsized":"2023-10-18T09:07:18.366030+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:20:48.205449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.373009+0000","last_change":"2023-10-18T09:07:16.365435+0000","last_active":"2023-10-18T09:07:17.373009+0000","last_peered":"2023-10-18T09:07:17.373009+0000","last_clean":"2023-10-18T09:07:17.373009+0000","last_became_active":"2023-10-18T09:07:16.365125+0000","last_became_peered":"2023-10-18T09:07:16.365125+0000","last_unstale":"2023-10-18T09:07:17.373009+0000","last_undegraded":"2023-10-18T09:07:17.373009+0000","last_fullsized":"2023-10-18T09:07:17.373009+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:55.519487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742308+0000","last_change":"2023-10-18T09:07:17.391978+0000","last_active":"2023-10-18T09:07:18.742308+0000","last_peered":"2023-10-18T09:07:18.742308+0000","last_clean":"2023-10-18T09:07:18.742308+0000","last_became_active":"2023-10-18T09:07:17.389095+0000","last_became_peered":"2023-10-18T09:07:17.389095+0000","last_unstale":"2023-10-18T09:07:18.742308+0000","last_undegraded":"2023-10-18T09:07:18.742308+0000","last_fullsized":"2023-10-18T09:07:18.742308+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:29:59.823948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.383758+0000","last_change":"2023-10-18T09:07:16.365242+0000","last_active":"2023-10-18T09:07:17.383758+0000","last_peered":"2023-10-18T09:07:17.383758+0000","last_clean":"2023-10-18T09:07:17.383758+0000","last_became_active":"2023-10-18T09:07:16.364983+0000","last_became_peered":"2023-10-18T09:07:16.364983+0000","last_unstale":"2023-10-18T09:07:17.383758+0000","last_undegraded":"2023-10-18T09:07:17.383758+0000","last_fullsized":"2023-10-18T09:07:17.383758+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:14.895783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742809+0000","last_change":"2023-10-18T09:07:17.390161+0000","last_active":"2023-10-18T09:07:18.742809+0000","last_peered":"2023-10-18T09:07:18.742809+0000","last_clean":"2023-10-18T09:07:18.742809+0000","last_became_active":"2023-10-18T09:07:17.389869+0000","last_became_peered":"2023-10-18T09:07:17.389869+0000","last_unstale":"2023-10-18T09:07:18.742809+0000","last_undegraded":"2023-10-18T09:07:18.742809+0000","last_fullsized":"2023-10-18T09:07:18.742809+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:01.734509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365503+0000","last_change":"2023-10-18T09:07:16.356639+0000","last_active":"2023-10-18T09:07:18.365503+0000","last_peered":"2023-10-18T09:07:18.365503+0000","last_clean":"2023-10-18T09:07:18.365503+0000","last_became_active":"2023-10-18T09:07:16.356359+0000","last_became_peered":"2023-10-18T09:07:16.356359+0000","last_unstale":"2023-10-18T09:07:18.365503+0000","last_undegraded":"2023-10-18T09:07:18.365503+0000","last_fullsized":"2023-10-18T09:07:18.365503+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:07:56.622798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.369108+0000","last_change":"2023-10-18T09:07:17.381272+0000","last_active":"2023-10-18T09:07:18.369108+0000","last_peered":"2023-10-18T09:07:18.369108+0000","last_clean":"2023-10-18T09:07:18.369108+0000","last_became_active":"2023-10-18T09:07:17.380821+0000","last_became_peered":"2023-10-18T09:07:17.380821+0000","last_unstale":"2023-10-18T09:07:18.369108+0000","last_undegraded":"2023-10-18T09:07:18.369108+0000","last_fullsized":"2023-10-18T09:07:18.369108+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:13:03.047350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.369602+0000","last_change":"2023-10-18T09:07:16.368087+0000","last_active":"2023-10-18T09:07:18.369602+0000","last_peered":"2023-10-18T09:07:18.369602+0000","last_clean":"2023-10-18T09:07:18.369602+0000","last_became_active":"2023-10-18T09:07:16.367805+0000","last_became_peered":"2023-10-18T09:07:16.367805+0000","last_unstale":"2023-10-18T09:07:18.369602+0000","last_undegraded":"2023-10-18T09:07:18.369602+0000","last_fullsized":"2023-10-18T09:07:18.369602+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:37:29.546569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.381861+0000","last_change":"2023-10-18T09:07:17.381404+0000","last_active":"2023-10-18T09:07:17.381861+0000","last_peered":"2023-10-18T09:07:17.381861+0000","last_clean":"2023-10-18T09:07:17.381861+0000","last_became_active":"2023-10-18T09:07:17.381033+0000","last_became_peered":"2023-10-18T09:07:17.381033+0000","last_unstale":"2023-10-18T09:07:17.381861+0000","last_undegraded":"2023-10-18T09:07:17.381861+0000","last_fullsized":"2023-10-18T09:07:17.381861+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:02:48.453087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"178'6","reported_seq":18,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.741077+0000","last_change":"2023-10-18T09:07:16.360840+0000","last_active":"2023-10-18T09:07:18.741077+0000","last_peered":"2023-10-18T09:07:18.741077+0000","last_clean":"2023-10-18T09:07:18.741077+0000","last_became_active":"2023-10-18T09:07:16.360429+0000","last_became_peered":"2023-10-18T09:07:16.360429+0000","last_unstale":"2023-10-18T09:07:18.741077+0000","last_undegraded":"2023-10-18T09:07:18.741077+0000","last_fullsized":"2023-10-18T09:07:18.741077+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:37:29.546569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.369699+0000","last_change":"2023-10-18T09:07:17.391046+0000","last_active":"2023-10-18T09:07:18.369699+0000","last_peered":"2023-10-18T09:07:18.369699+0000","last_clean":"2023-10-18T09:07:18.369699+0000","last_became_active":"2023-10-18T09:07:17.390433+0000","last_became_peered":"2023-10-18T09:07:17.390433+0000","last_unstale":"2023-10-18T09:07:18.369699+0000","last_undegraded":"2023-10-18T09:07:18.369699+0000","last_fullsized":"2023-10-18T09:07:18.369699+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:21:24.245975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"178'2","reported_seq":14,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370958+0000","last_change":"2023-10-18T09:07:16.359716+0000","last_active":"2023-10-18T09:07:18.370958+0000","last_peered":"2023-10-18T09:07:18.370958+0000","last_clean":"2023-10-18T09:07:18.370958+0000","last_became_active":"2023-10-18T09:07:16.359462+0000","last_became_peered":"2023-10-18T09:07:16.359462+0000","last_unstale":"2023-10-18T09:07:18.370958+0000","last_undegraded":"2023-10-18T09:07:18.370958+0000","last_fullsized":"2023-10-18T09:07:18.370958+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:38:23.509498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365809+0000","last_change":"2023-10-18T09:07:17.382465+0000","last_active":"2023-10-18T09:07:18.365809+0000","last_peered":"2023-10-18T09:07:18.365809+0000","last_clean":"2023-10-18T09:07:18.365809+0000","last_became_active":"2023-10-18T09:07:17.382227+0000","last_became_peered":"2023-10-18T09:07:17.382227+0000","last_unstale":"2023-10-18T09:07:18.365809+0000","last_undegraded":"2023-10-18T09:07:18.365809+0000","last_fullsized":"2023-10-18T09:07:18.365809+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:28:04.413480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.369917+0000","last_change":"2023-10-18T09:07:16.367854+0000","last_active":"2023-10-18T09:07:18.369917+0000","last_peered":"2023-10-18T09:07:18.369917+0000","last_clean":"2023-10-18T09:07:18.369917+0000","last_became_active":"2023-10-18T09:07:16.367402+0000","last_became_peered":"2023-10-18T09:07:16.367402+0000","last_unstale":"2023-10-18T09:07:18.369917+0000","last_undegraded":"2023-10-18T09:07:18.369917+0000","last_fullsized":"2023-10-18T09:07:18.369917+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:07:56.622798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742291+0000","last_change":"2023-10-18T09:07:17.401333+0000","last_active":"2023-10-18T09:07:18.742291+0000","last_peered":"2023-10-18T09:07:18.742291+0000","last_clean":"2023-10-18T09:07:18.742291+0000","last_became_active":"2023-10-18T09:07:17.400762+0000","last_became_peered":"2023-10-18T09:07:17.400762+0000","last_unstale":"2023-10-18T09:07:18.742291+0000","last_undegraded":"2023-10-18T09:07:18.742291+0000","last_fullsized":"2023-10-18T09:07:18.742291+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:32:48.511798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.365345+0000","last_change":"2023-10-18T09:07:16.353725+0000","last_active":"2023-10-18T09:07:17.365345+0000","last_peered":"2023-10-18T09:07:17.365345+0000","last_clean":"2023-10-18T09:07:17.365345+0000","last_became_active":"2023-10-18T09:07:16.353299+0000","last_became_peered":"2023-10-18T09:07:16.353299+0000","last_unstale":"2023-10-18T09:07:17.365345+0000","last_undegraded":"2023-10-18T09:07:17.365345+0000","last_fullsized":"2023-10-18T09:07:17.365345+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:52:39.133251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370232+0000","last_change":"2023-10-18T09:07:17.394017+0000","last_active":"2023-10-18T09:07:18.370232+0000","last_peered":"2023-10-18T09:07:18.370232+0000","last_clean":"2023-10-18T09:07:18.370232+0000","last_became_active":"2023-10-18T09:07:17.393461+0000","last_became_peered":"2023-10-18T09:07:17.393461+0000","last_unstale":"2023-10-18T09:07:18.370232+0000","last_undegraded":"2023-10-18T09:07:18.370232+0000","last_fullsized":"2023-10-18T09:07:18.370232+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:08:54.405679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"178'8","reported_seq":19,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.397911+0000","last_change":"2023-10-18T09:07:16.366985+0000","last_active":"2023-10-18T09:07:17.397911+0000","last_peered":"2023-10-18T09:07:17.397911+0000","last_clean":"2023-10-18T09:07:17.397911+0000","last_became_active":"2023-10-18T09:07:16.366843+0000","last_became_peered":"2023-10-18T09:07:16.366843+0000","last_unstale":"2023-10-18T09:07:17.397911+0000","last_undegraded":"2023-10-18T09:07:17.397911+0000","last_fullsized":"2023-10-18T09:07:17.397911+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:28:55.503205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.369791+0000","last_change":"2023-10-18T09:07:17.384155+0000","last_active":"2023-10-18T09:07:18.369791+0000","last_peered":"2023-10-18T09:07:18.369791+0000","last_clean":"2023-10-18T09:07:18.369791+0000","last_became_active":"2023-10-18T09:07:17.377696+0000","last_became_peered":"2023-10-18T09:07:17.377696+0000","last_unstale":"2023-10-18T09:07:18.369791+0000","last_undegraded":"2023-10-18T09:07:18.369791+0000","last_fullsized":"2023-10-18T09:07:18.369791+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:10:09.706171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"178'1","reported_seq":13,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742455+0000","last_change":"2023-10-18T09:07:16.363546+0000","last_active":"2023-10-18T09:07:18.742455+0000","last_peered":"2023-10-18T09:07:18.742455+0000","last_clean":"2023-10-18T09:07:18.742455+0000","last_became_active":"2023-10-18T09:07:16.363306+0000","last_became_peered":"2023-10-18T09:07:16.363306+0000","last_unstale":"2023-10-18T09:07:18.742455+0000","last_undegraded":"2023-10-18T09:07:18.742455+0000","last_fullsized":"2023-10-18T09:07:18.742455+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:52:45.673846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.386768+0000","last_change":"2023-10-18T09:07:17.384710+0000","last_active":"2023-10-18T09:07:17.386768+0000","last_peered":"2023-10-18T09:07:17.386768+0000","last_clean":"2023-10-18T09:07:17.386768+0000","last_became_active":"2023-10-18T09:07:17.384238+0000","last_became_peered":"2023-10-18T09:07:17.384238+0000","last_unstale":"2023-10-18T09:07:17.386768+0000","last_undegraded":"2023-10-18T09:07:17.386768+0000","last_fullsized":"2023-10-18T09:07:17.386768+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:39:11.829238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"178'2","reported_seq":14,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.366094+0000","last_change":"2023-10-18T09:07:16.362989+0000","last_active":"2023-10-18T09:07:18.366094+0000","last_peered":"2023-10-18T09:07:18.366094+0000","last_clean":"2023-10-18T09:07:18.366094+0000","last_became_active":"2023-10-18T09:07:16.362622+0000","last_became_peered":"2023-10-18T09:07:16.362622+0000","last_unstale":"2023-10-18T09:07:18.366094+0000","last_undegraded":"2023-10-18T09:07:18.366094+0000","last_fullsized":"2023-10-18T09:07:18.366094+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:21:23.230582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370575+0000","last_change":"2023-10-18T09:07:17.393889+0000","last_active":"2023-10-18T09:07:18.370575+0000","last_peered":"2023-10-18T09:07:18.370575+0000","last_clean":"2023-10-18T09:07:18.370575+0000","last_became_active":"2023-10-18T09:07:17.393116+0000","last_became_peered":"2023-10-18T09:07:17.393116+0000","last_unstale":"2023-10-18T09:07:18.370575+0000","last_undegraded":"2023-10-18T09:07:18.370575+0000","last_fullsized":"2023-10-18T09:07:18.370575+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:10.523692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"178'1","reported_seq":12,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.389605+0000","last_change":"2023-10-18T09:07:16.361903+0000","last_active":"2023-10-18T09:07:17.389605+0000","last_peered":"2023-10-18T09:07:17.389605+0000","last_clean":"2023-10-18T09:07:17.389605+0000","last_became_active":"2023-10-18T09:07:16.361524+0000","last_became_peered":"2023-10-18T09:07:16.361524+0000","last_unstale":"2023-10-18T09:07:17.389605+0000","last_undegraded":"2023-10-18T09:07:17.389605+0000","last_fullsized":"2023-10-18T09:07:17.389605+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:26:45.011516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.397868+0000","last_change":"2023-10-18T09:07:17.395628+0000","last_active":"2023-10-18T09:07:17.397868+0000","last_peered":"2023-10-18T09:07:17.397868+0000","last_clean":"2023-10-18T09:07:17.397868+0000","last_became_active":"2023-10-18T09:07:17.395076+0000","last_became_peered":"2023-10-18T09:07:17.395076+0000","last_unstale":"2023-10-18T09:07:17.397868+0000","last_undegraded":"2023-10-18T09:07:17.397868+0000","last_fullsized":"2023-10-18T09:07:17.397868+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:28.831307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365926+0000","last_change":"2023-10-18T09:07:16.365594+0000","last_active":"2023-10-18T09:07:18.365926+0000","last_peered":"2023-10-18T09:07:18.365926+0000","last_clean":"2023-10-18T09:07:18.365926+0000","last_became_active":"2023-10-18T09:07:16.365311+0000","last_became_peered":"2023-10-18T09:07:16.365311+0000","last_unstale":"2023-10-18T09:07:18.365926+0000","last_undegraded":"2023-10-18T09:07:18.365926+0000","last_fullsized":"2023-10-18T09:07:18.365926+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:52:42.215263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.385558+0000","last_change":"2023-10-18T09:07:17.384730+0000","last_active":"2023-10-18T09:07:17.385558+0000","last_peered":"2023-10-18T09:07:17.385558+0000","last_clean":"2023-10-18T09:07:17.385558+0000","last_became_active":"2023-10-18T09:07:17.384271+0000","last_became_peered":"2023-10-18T09:07:17.384271+0000","last_unstale":"2023-10-18T09:07:17.385558+0000","last_undegraded":"2023-10-18T09:07:17.385558+0000","last_fullsized":"2023-10-18T09:07:17.385558+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:21:20.537938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"178'2","reported_seq":14,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.741309+0000","last_change":"2023-10-18T09:07:16.366691+0000","last_active":"2023-10-18T09:07:18.741309+0000","last_peered":"2023-10-18T09:07:18.741309+0000","last_clean":"2023-10-18T09:07:18.741309+0000","last_became_active":"2023-10-18T09:07:16.366485+0000","last_became_peered":"2023-10-18T09:07:16.366485+0000","last_unstale":"2023-10-18T09:07:18.741309+0000","last_undegraded":"2023-10-18T09:07:18.741309+0000","last_fullsized":"2023-10-18T09:07:18.741309+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:53:34.513833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.741210+0000","last_change":"2023-10-18T09:07:17.388640+0000","last_active":"2023-10-18T09:07:18.741210+0000","last_peered":"2023-10-18T09:07:18.741210+0000","last_clean":"2023-10-18T09:07:18.741210+0000","last_became_active":"2023-10-18T09:07:17.388193+0000","last_became_peered":"2023-10-18T09:07:17.388193+0000","last_unstale":"2023-10-18T09:07:18.741210+0000","last_undegraded":"2023-10-18T09:07:18.741210+0000","last_fullsized":"2023-10-18T09:07:18.741210+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:49.542450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.371217+0000","last_change":"2023-10-18T09:07:16.364501+0000","last_active":"2023-10-18T09:07:18.371217+0000","last_peered":"2023-10-18T09:07:18.371217+0000","last_clean":"2023-10-18T09:07:18.371217+0000","last_became_active":"2023-10-18T09:07:16.362564+0000","last_became_peered":"2023-10-18T09:07:16.362564+0000","last_unstale":"2023-10-18T09:07:18.371217+0000","last_undegraded":"2023-10-18T09:07:18.371217+0000","last_fullsized":"2023-10-18T09:07:18.371217+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:14:01.348219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.387324+0000","last_change":"2023-10-18T09:07:17.387235+0000","last_active":"2023-10-18T09:07:17.387324+0000","last_peered":"2023-10-18T09:07:17.387324+0000","last_clean":"2023-10-18T09:07:17.387324+0000","last_became_active":"2023-10-18T09:07:17.386101+0000","last_became_peered":"2023-10-18T09:07:17.386101+0000","last_unstale":"2023-10-18T09:07:17.387324+0000","last_undegraded":"2023-10-18T09:07:17.387324+0000","last_fullsized":"2023-10-18T09:07:17.387324+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:37:35.491056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370015+0000","last_change":"2023-10-18T09:07:16.363088+0000","last_active":"2023-10-18T09:07:18.370015+0000","last_peered":"2023-10-18T09:07:18.370015+0000","last_clean":"2023-10-18T09:07:18.370015+0000","last_became_active":"2023-10-18T09:07:16.362052+0000","last_became_peered":"2023-10-18T09:07:16.362052+0000","last_unstale":"2023-10-18T09:07:18.370015+0000","last_undegraded":"2023-10-18T09:07:18.370015+0000","last_fullsized":"2023-10-18T09:07:18.370015+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:18:48.527057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.741998+0000","last_change":"2023-10-18T09:07:17.387447+0000","last_active":"2023-10-18T09:07:18.741998+0000","last_peered":"2023-10-18T09:07:18.741998+0000","last_clean":"2023-10-18T09:07:18.741998+0000","last_became_active":"2023-10-18T09:07:17.387146+0000","last_became_peered":"2023-10-18T09:07:17.387146+0000","last_unstale":"2023-10-18T09:07:18.741998+0000","last_undegraded":"2023-10-18T09:07:18.741998+0000","last_fullsized":"2023-10-18T09:07:18.741998+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:31:10.222264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370473+0000","last_change":"2023-10-18T09:07:16.365625+0000","last_active":"2023-10-18T09:07:18.370473+0000","last_peered":"2023-10-18T09:07:18.370473+0000","last_clean":"2023-10-18T09:07:18.370473+0000","last_became_active":"2023-10-18T09:07:16.362765+0000","last_became_peered":"2023-10-18T09:07:16.362765+0000","last_unstale":"2023-10-18T09:07:18.370473+0000","last_undegraded":"2023-10-18T09:07:18.370473+0000","last_fullsized":"2023-10-18T09:07:18.370473+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:57:18.645811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742625+0000","last_change":"2023-10-18T09:07:17.401163+0000","last_active":"2023-10-18T09:07:18.742625+0000","last_peered":"2023-10-18T09:07:18.742625+0000","last_clean":"2023-10-18T09:07:18.742625+0000","last_became_active":"2023-10-18T09:07:17.400406+0000","last_became_peered":"2023-10-18T09:07:17.400406+0000","last_unstale":"2023-10-18T09:07:18.742625+0000","last_undegraded":"2023-10-18T09:07:18.742625+0000","last_fullsized":"2023-10-18T09:07:18.742625+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:12:41.408408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"178'2","reported_seq":14,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.368981+0000","last_change":"2023-10-18T09:07:16.364405+0000","last_active":"2023-10-18T09:07:18.368981+0000","last_peered":"2023-10-18T09:07:18.368981+0000","last_clean":"2023-10-18T09:07:18.368981+0000","last_became_active":"2023-10-18T09:07:16.363029+0000","last_became_peered":"2023-10-18T09:07:16.363029+0000","last_unstale":"2023-10-18T09:07:18.368981+0000","last_undegraded":"2023-10-18T09:07:18.368981+0000","last_fullsized":"2023-10-18T09:07:18.368981+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:05:41.563036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.370920+0000","last_change":"2023-10-18T09:07:17.393745+0000","last_active":"2023-10-18T09:07:18.370920+0000","last_peered":"2023-10-18T09:07:18.370920+0000","last_clean":"2023-10-18T09:07:18.370920+0000","last_became_active":"2023-10-18T09:07:17.392751+0000","last_became_peered":"2023-10-18T09:07:17.392751+0000","last_unstale":"2023-10-18T09:07:18.370920+0000","last_undegraded":"2023-10-18T09:07:18.370920+0000","last_fullsized":"2023-10-18T09:07:18.370920+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:15:00.445724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.365782+0000","last_change":"2023-10-18T09:07:16.363218+0000","last_active":"2023-10-18T09:07:17.365782+0000","last_peered":"2023-10-18T09:07:17.365782+0000","last_clean":"2023-10-18T09:07:17.365782+0000","last_became_active":"2023-10-18T09:07:16.362938+0000","last_became_peered":"2023-10-18T09:07:16.362938+0000","last_unstale":"2023-10-18T09:07:17.365782+0000","last_undegraded":"2023-10-18T09:07:17.365782+0000","last_fullsized":"2023-10-18T09:07:17.365782+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:50:23.934782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365902+0000","last_change":"2023-10-18T09:07:17.387337+0000","last_active":"2023-10-18T09:07:18.365902+0000","last_peered":"2023-10-18T09:07:18.365902+0000","last_clean":"2023-10-18T09:07:18.365902+0000","last_became_active":"2023-10-18T09:07:17.387062+0000","last_became_peered":"2023-10-18T09:07:17.387062+0000","last_unstale":"2023-10-18T09:07:18.365902+0000","last_undegraded":"2023-10-18T09:07:18.365902+0000","last_fullsized":"2023-10-18T09:07:18.365902+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:37:35.491056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.367018+0000","last_change":"2023-10-18T09:07:16.361647+0000","last_active":"2023-10-18T09:07:17.367018+0000","last_peered":"2023-10-18T09:07:17.367018+0000","last_clean":"2023-10-18T09:07:17.367018+0000","last_became_active":"2023-10-18T09:07:16.361384+0000","last_became_peered":"2023-10-18T09:07:16.361384+0000","last_unstale":"2023-10-18T09:07:17.367018+0000","last_undegraded":"2023-10-18T09:07:17.367018+0000","last_fullsized":"2023-10-18T09:07:17.367018+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:01:20.810697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.386909+0000","last_change":"2023-10-18T09:07:17.386465+0000","last_active":"2023-10-18T09:07:17.386909+0000","last_peered":"2023-10-18T09:07:17.386909+0000","last_clean":"2023-10-18T09:07:17.386909+0000","last_became_active":"2023-10-18T09:07:17.386086+0000","last_became_peered":"2023-10-18T09:07:17.386086+0000","last_unstale":"2023-10-18T09:07:17.386909+0000","last_undegraded":"2023-10-18T09:07:17.386909+0000","last_fullsized":"2023-10-18T09:07:17.386909+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:11:05.152957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.366510+0000","last_change":"2023-10-18T09:07:16.361128+0000","last_active":"2023-10-18T09:07:17.366510+0000","last_peered":"2023-10-18T09:07:17.366510+0000","last_clean":"2023-10-18T09:07:17.366510+0000","last_became_active":"2023-10-18T09:07:16.360821+0000","last_became_peered":"2023-10-18T09:07:16.360821+0000","last_unstale":"2023-10-18T09:07:17.366510+0000","last_undegraded":"2023-10-18T09:07:17.366510+0000","last_fullsized":"2023-10-18T09:07:17.366510+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:41:54.930255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.386797+0000","last_change":"2023-10-18T09:07:17.384882+0000","last_active":"2023-10-18T09:07:17.386797+0000","last_peered":"2023-10-18T09:07:17.386797+0000","last_clean":"2023-10-18T09:07:17.386797+0000","last_became_active":"2023-10-18T09:07:17.384677+0000","last_became_peered":"2023-10-18T09:07:17.384677+0000","last_unstale":"2023-10-18T09:07:17.386797+0000","last_undegraded":"2023-10-18T09:07:17.386797+0000","last_fullsized":"2023-10-18T09:07:17.386797+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:43:24.684032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.741802+0000","last_change":"2023-10-18T09:07:16.362111+0000","last_active":"2023-10-18T09:07:18.741802+0000","last_peered":"2023-10-18T09:07:18.741802+0000","last_clean":"2023-10-18T09:07:18.741802+0000","last_became_active":"2023-10-18T09:07:16.361862+0000","last_became_peered":"2023-10-18T09:07:16.361862+0000","last_unstale":"2023-10-18T09:07:18.741802+0000","last_undegraded":"2023-10-18T09:07:18.741802+0000","last_fullsized":"2023-10-18T09:07:18.741802+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:35:34.092911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742330+0000","last_change":"2023-10-18T09:07:17.395505+0000","last_active":"2023-10-18T09:07:18.742330+0000","last_peered":"2023-10-18T09:07:18.742330+0000","last_clean":"2023-10-18T09:07:18.742330+0000","last_became_active":"2023-10-18T09:07:17.393306+0000","last_became_peered":"2023-10-18T09:07:17.393306+0000","last_unstale":"2023-10-18T09:07:18.742330+0000","last_undegraded":"2023-10-18T09:07:18.742330+0000","last_fullsized":"2023-10-18T09:07:18.742330+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:28:09.467521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.369215+0000","last_change":"2023-10-18T09:07:16.364731+0000","last_active":"2023-10-18T09:07:18.369215+0000","last_peered":"2023-10-18T09:07:18.369215+0000","last_clean":"2023-10-18T09:07:18.369215+0000","last_became_active":"2023-10-18T09:07:16.364127+0000","last_became_peered":"2023-10-18T09:07:16.364127+0000","last_unstale":"2023-10-18T09:07:18.369215+0000","last_undegraded":"2023-10-18T09:07:18.369215+0000","last_fullsized":"2023-10-18T09:07:18.369215+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:58:53.695998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.371003+0000","last_change":"2023-10-18T09:07:17.379531+0000","last_active":"2023-10-18T09:07:18.371003+0000","last_peered":"2023-10-18T09:07:18.371003+0000","last_clean":"2023-10-18T09:07:18.371003+0000","last_became_active":"2023-10-18T09:07:17.378878+0000","last_became_peered":"2023-10-18T09:07:17.378878+0000","last_unstale":"2023-10-18T09:07:18.371003+0000","last_undegraded":"2023-10-18T09:07:18.371003+0000","last_fullsized":"2023-10-18T09:07:18.371003+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:38.394044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.371372+0000","last_change":"2023-10-18T09:07:16.366629+0000","last_active":"2023-10-18T09:07:18.371372+0000","last_peered":"2023-10-18T09:07:18.371372+0000","last_clean":"2023-10-18T09:07:18.371372+0000","last_became_active":"2023-10-18T09:07:16.365920+0000","last_became_peered":"2023-10-18T09:07:16.365920+0000","last_unstale":"2023-10-18T09:07:18.371372+0000","last_undegraded":"2023-10-18T09:07:18.371372+0000","last_fullsized":"2023-10-18T09:07:18.371372+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:27:53.056702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.742812+0000","last_change":"2023-10-18T09:07:17.389853+0000","last_active":"2023-10-18T09:07:18.742812+0000","last_peered":"2023-10-18T09:07:18.742812+0000","last_clean":"2023-10-18T09:07:18.742812+0000","last_became_active":"2023-10-18T09:07:17.389487+0000","last_became_peered":"2023-10-18T09:07:17.389487+0000","last_unstale":"2023-10-18T09:07:18.742812+0000","last_undegraded":"2023-10-18T09:07:18.742812+0000","last_fullsized":"2023-10-18T09:07:18.742812+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:52:54.846421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.365636+0000","last_change":"2023-10-18T09:07:16.359958+0000","last_active":"2023-10-18T09:07:18.365636+0000","last_peered":"2023-10-18T09:07:18.365636+0000","last_clean":"2023-10-18T09:07:18.365636+0000","last_became_active":"2023-10-18T09:07:16.359412+0000","last_became_peered":"2023-10-18T09:07:16.359412+0000","last_unstale":"2023-10-18T09:07:18.365636+0000","last_undegraded":"2023-10-18T09:07:18.365636+0000","last_fullsized":"2023-10-18T09:07:18.365636+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:38:23.509498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":11,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.741520+0000","last_change":"2023-10-18T09:07:17.388720+0000","last_active":"2023-10-18T09:07:18.741520+0000","last_peered":"2023-10-18T09:07:18.741520+0000","last_clean":"2023-10-18T09:07:18.741520+0000","last_became_active":"2023-10-18T09:07:17.388468+0000","last_became_peered":"2023-10-18T09:07:17.388468+0000","last_unstale":"2023-10-18T09:07:18.741520+0000","last_undegraded":"2023-10-18T09:07:18.741520+0000","last_fullsized":"2023-10-18T09:07:18.741520+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:16.332098+0000","last_clean_scrub_stamp":"2023-10-18T09:07:16.332098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:14:02.363611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"178'3","reported_seq":15,"reported_epoch":179,"state":"active+clean","last_fresh":"2023-10-18T09:07:18.371149+0000","last_change":"2023-10-18T09:07:16.366385+0000","last_active":"2023-10-18T09:07:18.371149+0000","last_peered":"2023-10-18T09:07:18.371149+0000","last_clean":"2023-10-18T09:07:18.371149+0000","last_became_active":"2023-10-18T09:07:16.363981+0000","last_became_peered":"2023-10-18T09:07:16.363981+0000","last_unstale":"2023-10-18T09:07:18.371149+0000","last_undegraded":"2023-10-18T09:07:18.371149+0000","last_fullsized":"2023-10-18T09:07:18.371149+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:07:15.316705+0000","last_clean_scrub_stamp":"2023-10-18T09:07:15.316705+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:39:35.876064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":338,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-10-18T09:07:17.373440+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:07:17.373440+0000","last_peered":"2023-10-18T09:07:17.373440+0000","last_clean":"2023-10-18T09:07:17.373440+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:07:17.373440+0000","last_undegraded":"2023-10-18T09:07:17.373440+0000","last_fullsized":"2023-10-18T09:07:17.373440+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574997,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1568,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193743,"statfs":{"total":200000000,"available":198393703,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.1040000000000001,"15min":1.0640000000000001},"min":{"1min":0.80000000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.52,"5min":1.635,"15min":1.6990000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.087,"15min":1.0720000000000001},"min":{"1min":0.89500000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.371,"5min":1.724,"15min":1.97},"last":0.79200000000000004}]},{"osd":1,"last update":"Wed Oct 18 09:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.1990000000000001,"15min":1.167},"min":{"1min":0.89500000000000002,"5min":0.67200000000000004,"15min":0.61799999999999999},"max":{"1min":1.591,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.222,"5min":1.1299999999999999,"15min":1.135},"min":{"1min":0.93899999999999995,"5min":0.72799999999999998,"15min":0.58799999999999997},"max":{"1min":1.661,"5min":1.661,"15min":1.7989999999999999},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.345,"15min":1.2909999999999999},"min":{"1min":0.96699999999999997,"5min":0.82199999999999995,"15min":0.65400000000000003},"max":{"1min":1.617,"5min":1.8169999999999999,"15min":1.9730000000000001},"last":0.88},{"interface":"front","average":{"1min":1.212,"5min":1.2490000000000001,"15min":1.2330000000000001},"min":{"1min":0.88400000000000001,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.532,"5min":1.863,"15min":1.863},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.306,"15min":1.28},"min":{"1min":0.92400000000000004,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.7330000000000001,"5min":1.98,"15min":1.98},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.3720000000000001,"15min":1.349},"min":{"1min":1.131,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7110000000000001,"5min":1.7210000000000001,"15min":1.9930000000000001},"last":0.94899999999999995}]},{"osd":4,"last update":"Wed Oct 18 09:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.202,"15min":1.2050000000000001},"min":{"1min":0.85499999999999998,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5589999999999999,"5min":1.923,"15min":2.1070000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.333,"15min":1.359},"min":{"1min":0.98999999999999999,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5669999999999999,"5min":2.0369999999999999,"15min":2.5169999999999999},"last":0.98499999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3240000000000001,"15min":1.3460000000000001},"min":{"1min":0.85999999999999999,"5min":0.622,"15min":0.622},"max":{"1min":1.7589999999999999,"5min":1.7989999999999999,"15min":2.0720000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.3879999999999999,"15min":1.413},"min":{"1min":1.1439999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.615,"5min":1.956,"15min":2.0270000000000001},"last":0.56000000000000005}]},{"osd":6,"last update":"Wed Oct 18 09:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.48,"5min":1.4690000000000001,"15min":1.4750000000000001},"min":{"1min":1.103,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.6779999999999999,"5min":2.0779999999999998,"15min":2.3959999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.484,"15min":1.4950000000000001},"min":{"1min":1.127,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.7350000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.82199999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574997,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":991,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194320,"statfs":{"total":200000000,"available":198984577,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.252,"15min":1.1910000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.60299999999999998},"max":{"1min":1.595,"5min":1.643,"15min":1.6850000000000001},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3580000000000001,"15min":1.3080000000000001},"min":{"1min":1.0720000000000001,"5min":0.81799999999999995,"15min":0.64700000000000002},"max":{"1min":1.528,"5min":1.899,"15min":2.0990000000000002},"last":1.0720000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1439999999999999,"15min":1.115},"min":{"1min":0.78000000000000003,"5min":0.63900000000000001,"15min":0.63800000000000001},"max":{"1min":1.5,"5min":1.722,"15min":1.722},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.236,"15min":1.216},"min":{"1min":0.78500000000000003,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.774,"5min":1.8169999999999999,"15min":2.9609999999999999},"last":1.123}]},{"osd":2,"last update":"Wed Oct 18 09:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.202,"15min":1.1850000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.69899999999999995},"max":{"1min":1.6040000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.292,"15min":1.2470000000000001},"min":{"1min":0.86199999999999999,"5min":0.73399999999999999,"15min":0.66300000000000003},"max":{"1min":1.663,"5min":1.75,"15min":2.1779999999999999},"last":0.86199999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.288,"15min":1.2749999999999999},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.79200000000000004},"max":{"1min":1.7050000000000001,"5min":1.8160000000000001,"15min":2.1589999999999998},"last":1.613},{"interface":"front","average":{"1min":1.355,"5min":1.3600000000000001,"15min":1.2889999999999999},"min":{"1min":1.0049999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.6339999999999999,"5min":1.994,"15min":1.994},"last":1.3839999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3400000000000001,"15min":1.2609999999999999},"min":{"1min":0.93400000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.794,"5min":1.9770000000000001,"15min":2.444},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.343,"15min":1.3029999999999999},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.55000000000000004},"max":{"1min":1.6799999999999999,"5min":1.9159999999999999,"15min":2.4039999999999999},"last":1.5129999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4379999999999999,"15min":1.4079999999999999},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.58899999999999997},"max":{"1min":1.8340000000000001,"5min":2.016,"15min":2.032},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.387,"15min":1.3069999999999999},"min":{"1min":0.88600000000000001,"5min":0.55100000000000005,"15min":0.49199999999999999},"max":{"1min":1.593,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.1779999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:07:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4570000000000001,"15min":1.425},"min":{"1min":1.016,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.708,"5min":2.028,"15min":2.3540000000000001},"last":1.016},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.552,"15min":1.534},"min":{"1min":1.1890000000000001,"5min":1.087,"15min":0.68500000000000005},"max":{"1min":1.9319999999999999,"5min":2.1440000000000001,"15min":2.794},"last":1.7130000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574998,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":991,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194320,"statfs":{"total":200000000,"available":198984701,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.3480000000000001,"15min":1.2989999999999999},"min":{"1min":1.0669999999999999,"5min":0.68300000000000005,"15min":0.63400000000000001},"max":{"1min":1.6439999999999999,"5min":9.609,"15min":9.609},"last":1.722},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.321,"15min":1.3149999999999999},"min":{"1min":1.0820000000000001,"5min":0.754,"15min":0.60799999999999998},"max":{"1min":1.6339999999999999,"5min":2.0430000000000001,"15min":5.3029999999999999},"last":1.694}]},{"osd":1,"last update":"Wed Oct 18 09:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.3089999999999999,"15min":1.244},"min":{"1min":0.83999999999999997,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.885,"5min":11.68,"15min":11.68},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.198,"15min":1.216},"min":{"1min":0.92700000000000005,"5min":0.70099999999999996,"15min":0.67000000000000004},"max":{"1min":1.5800000000000001,"5min":1.885,"15min":3.7050000000000001},"last":1.004}]},{"osd":2,"last update":"Wed Oct 18 09:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.252,"15min":1.256},"min":{"1min":1.012,"5min":0.76800000000000002,"15min":0.755},"max":{"1min":1.6579999999999999,"5min":1.732,"15min":2.2090000000000001},"last":1.667},{"interface":"front","average":{"1min":1.29,"5min":1.2769999999999999,"15min":1.274},"min":{"1min":0.94499999999999995,"5min":0.79000000000000004,"15min":0.72399999999999998},"max":{"1min":1.698,"5min":1.7270000000000001,"15min":3.8180000000000001},"last":1.46}]},{"osd":3,"last update":"Wed Oct 18 09:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.3029999999999999,"15min":1.26},"min":{"1min":0.95599999999999996,"5min":0.72299999999999998,"15min":0.65400000000000003},"max":{"1min":1.409,"5min":11.797000000000001,"15min":11.797000000000001},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.3919999999999999,"15min":1.3620000000000001},"min":{"1min":1.125,"5min":0.747,"15min":0.747},"max":{"1min":2.105,"5min":2.105,"15min":2.3420000000000001},"last":1.381}]},{"osd":4,"last update":"Wed Oct 18 09:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.351,"15min":1.387},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.028,"5min":2.0680000000000001,"15min":2.4449999999999998},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.278,"15min":1.383},"min":{"1min":0.91400000000000003,"5min":0.83699999999999997,"15min":0.64100000000000001},"max":{"1min":1.7949999999999999,"5min":2.1360000000000001,"15min":5.4580000000000002},"last":1.635}]},{"osd":6,"last update":"Wed Oct 18 09:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.2330000000000001,"15min":1.3160000000000001},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.67400000000000004},"max":{"1min":1.7410000000000001,"5min":2.1030000000000002,"15min":5.141},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.569,"5min":1.5329999999999999,"15min":1.5409999999999999},"min":{"1min":1.1910000000000001,"5min":0.82499999999999996,"15min":0.71099999999999997},"max":{"1min":2.161,"5min":2.161,"15min":2.657},"last":1.52}]},{"osd":7,"last update":"Wed Oct 18 09:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.5720000000000001,"15min":1.5820000000000001},"min":{"1min":1.073,"5min":1.0529999999999999,"15min":0.89500000000000002},"max":{"1min":1.976,"5min":11.829000000000001,"15min":11.829000000000001},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.385,"15min":1.454},"min":{"1min":0.96099999999999997,"5min":0.93300000000000005,"15min":0.73399999999999999},"max":{"1min":1.8560000000000001,"5min":2.6419999999999999,"15min":4.8419999999999996},"last":1.496}]}]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":984,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194328,"statfs":{"total":200000000,"available":198992042,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:06:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1559999999999999,"15min":1.1319999999999999},"min":{"1min":0.83499999999999996,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.54,"5min":3.1819999999999999,"15min":3.1819999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.1579999999999999,"15min":1.1519999999999999},"min":{"1min":0.98199999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5920000000000001,"5min":3.234,"15min":3.234},"last":1.0129999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:06:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.1950000000000001,"15min":1.1339999999999999},"min":{"1min":0.82799999999999996,"5min":0.64700000000000002,"15min":0.60799999999999998},"max":{"1min":1.9690000000000001,"5min":3.718,"15min":3.718},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.238,"15min":1.2190000000000001},"min":{"1min":0.97799999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5660000000000001,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.135}]},{"osd":2,"last update":"Wed Oct 18 09:06:28 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2310000000000001,"15min":1.1990000000000001},"min":{"1min":0.89900000000000002,"5min":0.71799999999999997,"15min":0.56699999999999995},"max":{"1min":1.905,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3300000000000001,"15min":1.3049999999999999},"min":{"1min":0.99399999999999999,"5min":0.79500000000000004,"15min":0.58899999999999997},"max":{"1min":1.679,"5min":3.532,"15min":3.532},"last":0.95499999999999996}]},{"osd":3,"last update":"Wed Oct 18 09:06:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.282,"15min":1.2829999999999999},"min":{"1min":1.032,"5min":0.73899999999999999,"15min":0.67000000000000004},"max":{"1min":1.669,"5min":3.363,"15min":3.363},"last":1.369},{"interface":"front","average":{"1min":1.399,"5min":1.403,"15min":1.4039999999999999},"min":{"1min":1.089,"5min":0.84999999999999998,"15min":0.68899999999999995},"max":{"1min":1.714,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":1.512}]},{"osd":5,"last update":"Wed Oct 18 09:06:28 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.3,"15min":1.2969999999999999},"min":{"1min":0.78500000000000003,"5min":0.55600000000000005,"15min":0.498},"max":{"1min":1.8200000000000001,"5min":3.6160000000000001,"15min":3.6160000000000001},"last":1.417},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3260000000000001,"15min":1.329},"min":{"1min":0.91700000000000004,"5min":0.60099999999999998,"15min":0.38400000000000001},"max":{"1min":1.663,"5min":4.0780000000000003,"15min":4.0780000000000003},"last":1.544}]},{"osd":6,"last update":"Wed Oct 18 09:06:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.3779999999999999,"15min":1.393},"min":{"1min":0.92800000000000005,"5min":0.67400000000000004,"15min":0.51600000000000001},"max":{"1min":1.766,"5min":4.4429999999999996,"15min":5.0049999999999999},"last":1.276},{"interface":"front","average":{"1min":1.383,"5min":1.478,"15min":1.5109999999999999},"min":{"1min":0.90100000000000002,"5min":0.47499999999999998,"15min":0.45400000000000001},"max":{"1min":1.7829999999999999,"5min":4.1239999999999997,"15min":4.407},"last":1.321}]},{"osd":7,"last update":"Wed Oct 18 09:06:28 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4339999999999999,"15min":1.464},"min":{"1min":1.0429999999999999,"5min":0.51200000000000001,"15min":0.438},"max":{"1min":1.6859999999999999,"5min":4.5170000000000003,"15min":4.5170000000000003},"last":1.49},{"interface":"front","average":{"1min":1.456,"5min":1.5489999999999999,"15min":1.542},"min":{"1min":1.028,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.8520000000000001,"5min":4.5460000000000003,"15min":4.5460000000000003},"last":1.5669999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574999,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":991,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194320,"statfs":{"total":200000000,"available":198984633,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:06:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.92000000000000004,"15min":0.87},"min":{"1min":0.66800000000000004,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.1739999999999999,"5min":4.774,"15min":5.5730000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.181,"5min":0.97499999999999998,"15min":0.95299999999999996},"min":{"1min":0.66700000000000004,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.863,"5min":3.2370000000000001,"15min":6.5599999999999996},"last":0.90700000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:06:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.0049999999999999,"15min":0.998},"min":{"1min":0.754,"5min":0.39100000000000001,"15min":0.36799999999999999},"max":{"1min":1.7190000000000001,"5min":2.2650000000000001,"15min":6.8099999999999996},"last":1.155},{"interface":"front","average":{"1min":1.478,"5min":1.2749999999999999,"15min":1.2170000000000001},"min":{"1min":1.089,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":3.1920000000000002},"last":1.298}]},{"osd":2,"last update":"Wed Oct 18 09:06:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.131,"15min":1.0840000000000001},"min":{"1min":0.86499999999999999,"5min":0.46800000000000003,"15min":0.45300000000000001},"max":{"1min":1.357,"5min":2.6269999999999998,"15min":7.4580000000000002},"last":1.044},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2350000000000001,"15min":1.1599999999999999},"min":{"1min":0.998,"5min":0.52700000000000002,"15min":0.51400000000000001},"max":{"1min":1.639,"5min":5.0049999999999999,"15min":5.0049999999999999},"last":1.101}]},{"osd":4,"last update":"Wed Oct 18 09:06:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.357,"15min":1.371},"min":{"1min":1.161,"5min":0.65400000000000003,"15min":0.53700000000000003},"max":{"1min":1.6890000000000001,"5min":3.1139999999999999,"15min":18.664999999999999},"last":1.744},{"interface":"front","average":{"1min":1.415,"5min":1.349,"15min":1.268},"min":{"1min":1.121,"5min":0.878,"15min":0.64000000000000001},"max":{"1min":1.7909999999999999,"5min":2.0289999999999999,"15min":7.0700000000000003},"last":1.4490000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:06:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.365,"15min":1.302},"min":{"1min":1.073,"5min":0.626,"15min":0.626},"max":{"1min":1.7190000000000001,"5min":1.925,"15min":6.9139999999999997},"last":1.637},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4770000000000001,"15min":1.454},"min":{"1min":1.111,"5min":0.72999999999999998,"15min":0.56899999999999995},"max":{"1min":1.8089999999999999,"5min":4.5540000000000003,"15min":16.390999999999998},"last":1.216}]},{"osd":6,"last update":"Wed Oct 18 09:06:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.5109999999999999,"15min":1.4830000000000001},"min":{"1min":1.2150000000000001,"5min":0.77200000000000002,"15min":0.58199999999999996},"max":{"1min":2.2839999999999998,"5min":3.375,"15min":7.7270000000000003},"last":1.681},{"interface":"front","average":{"1min":1.633,"5min":1.514,"15min":1.446},"min":{"1min":1.3740000000000001,"5min":0.872,"15min":0.60199999999999998},"max":{"1min":1.8839999999999999,"5min":2.4860000000000002,"15min":7.242},"last":1.8100000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:06:20 2023","interfaces":[{"interface":"back","average":{"1min":1.653,"5min":1.621,"15min":1.488},"min":{"1min":1.393,"5min":0.84099999999999997,"15min":0.70099999999999996},"max":{"1min":1.8959999999999999,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":1.349},{"interface":"front","average":{"1min":1.706,"5min":1.6140000000000001,"15min":1.55},"min":{"1min":1.3540000000000001,"5min":0.82099999999999995,"15min":0.622},"max":{"1min":2.375,"5min":5.4900000000000002,"15min":12.946},"last":1.8660000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":986,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194325,"statfs":{"total":200000000,"available":198989341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.86499999999999999,"15min":0.89400000000000002},"min":{"1min":0.71099999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.52,"5min":1.5660000000000001,"15min":4.2210000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.044,"5min":1.014,"15min":1.0169999999999999},"min":{"1min":0.747,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.5860000000000001,"5min":4.3959999999999999,"15min":4.3959999999999999},"last":0.94399999999999995}]},{"osd":1,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.125,"15min":1.0780000000000001},"min":{"1min":0.82099999999999995,"5min":0.55500000000000005,"15min":0.49099999999999999},"max":{"1min":1.5640000000000001,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.081,"15min":1.101},"min":{"1min":0.80200000000000005,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":1.46,"5min":2.448,"15min":2.5350000000000001},"last":1.282}]},{"osd":3,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.159,"15min":1.1339999999999999},"min":{"1min":0.87,"5min":0.48299999999999998,"15min":0.48199999999999998},"max":{"1min":1.696,"5min":7.6369999999999996,"15min":7.6369999999999996},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.492,"5min":1.3069999999999999,"15min":1.3899999999999999},"min":{"1min":1.1539999999999999,"5min":0.60199999999999998,"15min":0.54100000000000004},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":7.5570000000000004},"last":1.1020000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.4259999999999999,"15min":1.4510000000000001},"min":{"1min":1.28,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.8799999999999999,"5min":2.3809999999999998,"15min":7.6189999999999998},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.474,"5min":1.411,"15min":1.4399999999999999},"min":{"1min":1.0609999999999999,"5min":0.89200000000000002,"15min":0.86599999999999999},"max":{"1min":1.6819999999999999,"5min":3.423,"15min":3.423},"last":1.3540000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3700000000000001,"15min":1.3089999999999999},"min":{"1min":1.139,"5min":0.754,"15min":0.61599999999999999},"max":{"1min":1.6399999999999999,"5min":10.507,"15min":10.507},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4039999999999999,"15min":1.4910000000000001},"min":{"1min":1.147,"5min":0.81599999999999995,"15min":0.81399999999999995},"max":{"1min":1.7370000000000001,"5min":1.972,"15min":7.657},"last":1.7110000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.413,"15min":1.478},"min":{"1min":1.095,"5min":0.95999999999999996,"15min":0.74299999999999999},"max":{"1min":1.754,"5min":2.8410000000000002,"15min":3.5299999999999998},"last":1.843},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.516,"15min":1.6120000000000001},"min":{"1min":1.2330000000000001,"5min":0.80200000000000005,"15min":0.77600000000000002},"max":{"1min":2.2639999999999998,"5min":10.561,"15min":37.771999999999998},"last":1.538}]},{"osd":7,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.514,"15min":1.5449999999999999},"min":{"1min":1.274,"5min":0.93999999999999995,"15min":0.83199999999999996},"max":{"1min":1.7729999999999999,"5min":2.7890000000000001,"15min":3.3620000000000001},"last":1.9359999999999999},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.581,"15min":1.524},"min":{"1min":1.375,"5min":0.90200000000000002,"15min":0.77600000000000002},"max":{"1min":2.375,"5min":10.956,"15min":10.956},"last":1.7549999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":984,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194327,"statfs":{"total":200000000,"available":198991460,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92600000000000005,"15min":0.86499999999999999},"min":{"1min":0.67500000000000004,"5min":0.37,"15min":0.315},"max":{"1min":1.4770000000000001,"5min":1.659,"15min":1.659},"last":1.464},{"interface":"front","average":{"1min":1.268,"5min":1.127,"15min":1.014},"min":{"1min":1.002,"5min":0.441,"15min":0.441},"max":{"1min":1.601,"5min":1.681,"15min":1.7010000000000001},"last":1.0229999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.073,"15min":0.997},"min":{"1min":0.78200000000000003,"5min":0.317,"15min":0.317},"max":{"1min":1.446,"5min":1.589,"15min":1.859},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":0.999,"15min":0.92200000000000004},"min":{"1min":0.81899999999999995,"5min":0.38600000000000001,"15min":0.36699999999999999},"max":{"1min":1.8009999999999999,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.3400000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.161,"15min":1.0780000000000001},"min":{"1min":0.94499999999999995,"5min":0.46899999999999997,"15min":0.436},"max":{"1min":1.798,"5min":1.915,"15min":2.1059999999999999},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.1060000000000001,"15min":1.075},"min":{"1min":0.877,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.6659999999999999,"5min":1.736,"15min":3.7010000000000001},"last":0.82099999999999995}]},{"osd":4,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3080000000000001,"15min":1.2829999999999999},"min":{"1min":1.0489999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.657,"5min":1.7430000000000001,"15min":2.0489999999999999},"last":1.556},{"interface":"front","average":{"1min":1.415,"5min":1.387,"15min":1.2909999999999999},"min":{"1min":1.0920000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.702,"5min":1.893,"15min":2.0470000000000002},"last":1.7190000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":1.4630000000000001,"15min":1.369},"min":{"1min":1.161,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.905,"5min":1.984,"15min":3.5750000000000002},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.571,"5min":1.484,"15min":1.417},"min":{"1min":1.3200000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.839,"5min":2.0150000000000001,"15min":2.3769999999999998},"last":1.6499999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.462,"15min":1.3660000000000001},"min":{"1min":1.21,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.9410000000000001,"5min":2.0419999999999998,"15min":2.2050000000000001},"last":1.603},{"interface":"front","average":{"1min":1.528,"5min":1.4399999999999999,"15min":1.3540000000000001},"min":{"1min":1.153,"5min":0.73399999999999999,"15min":0.63100000000000001},"max":{"1min":1.804,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.298}]},{"osd":7,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.526,"15min":1.4379999999999999},"min":{"1min":1.218,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.004,"5min":2.2000000000000002,"15min":2.2229999999999999},"last":1.429},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.492,"15min":1.4330000000000001},"min":{"1min":1.1439999999999999,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.948,"5min":2.2469999999999999,"15min":2.4180000000000001},"last":1.7649999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1562,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193750,"statfs":{"total":200000000,"available":198400028,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83099999999999996,"15min":0.86799999999999999},"min":{"1min":0.66900000000000004,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0249999999999999,"5min":1.3320000000000001,"15min":3.4420000000000002},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.0820000000000001,"15min":1.087},"min":{"1min":0.90100000000000002,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.657,"5min":1.806,"15min":1.8240000000000001},"last":1.016}]},{"osd":2,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.82299999999999995,"15min":0.84399999999999997},"min":{"1min":0.77100000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.0720000000000001,"5min":1.679,"15min":1.679},"last":1.077},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.1439999999999999,"15min":1.141},"min":{"1min":0.95699999999999996,"5min":0.503,"15min":0.437},"max":{"1min":1.6779999999999999,"5min":1.718,"15min":1.929},"last":0.90800000000000003}]},{"osd":3,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.113,"15min":1.2},"min":{"1min":0.95199999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.619,"5min":1.98,"15min":2.7639999999999998},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.369,"15min":1.444},"min":{"1min":1.202,"5min":0.61299999999999999,"15min":0.58399999999999996},"max":{"1min":1.7809999999999999,"5min":2.1200000000000001,"15min":3.5720000000000001},"last":0.81200000000000006}]},{"osd":4,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.1990000000000001,"15min":1.2549999999999999},"min":{"1min":0.90100000000000002,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.696,"5min":1.9990000000000001,"15min":4.492},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.28,"15min":1.2709999999999999},"min":{"1min":1.0980000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.659,"5min":1.927,"15min":1.927},"last":1.2929999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.2729999999999999,"15min":1.339},"min":{"1min":1.018,"5min":0.75600000000000001,"15min":0.64300000000000002},"max":{"1min":1.6439999999999999,"5min":2.1779999999999999,"15min":4.2779999999999996},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.3160000000000001,"15min":1.345},"min":{"1min":1.1799999999999999,"5min":0.74399999999999999,"15min":0.73099999999999998},"max":{"1min":1.6879999999999999,"5min":1.847,"15min":1.8740000000000001},"last":1.6180000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.401,"15min":1.5},"min":{"1min":1.2350000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7769999999999999,"5min":2.2469999999999999,"15min":3.5299999999999998},"last":1.776},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.331,"15min":1.423},"min":{"1min":1.196,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.7230000000000001,"5min":3.1440000000000001,"15min":4.4349999999999996},"last":1.748}]},{"osd":7,"last update":"Wed Oct 18 09:06:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.5029999999999999,"15min":1.579},"min":{"1min":1.3240000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.8779999999999999,"5min":2.2629999999999999,"15min":3.492},"last":1.9370000000000001},{"interface":"front","average":{"1min":1.569,"5min":1.4610000000000001,"15min":1.556},"min":{"1min":1.298,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.7849999999999999,"5min":3.2570000000000001,"15min":4.5419999999999998},"last":2.0550000000000002}]}]}],"pool_statfs":[]}} 2023-10-18T09:07:22.430 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T09:07:22.430 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T09:07:22.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.708+0000 7fde4b425640 1 -- 172.21.15.40:0/3411218495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde4406d5d0 msgr2=0x7fde440c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:22.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.708+0000 7fde4b425640 1 --2- 172.21.15.40:0/3411218495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde4406d5d0 0x7fde440c9620 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fde4406c6c0 tx=0x7fde4002bce0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.708+0000 7fde4b425640 1 -- 172.21.15.40:0/3411218495 shutdown_connections 2023-10-18T09:07:22.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.708+0000 7fde4b425640 1 --2- 172.21.15.40:0/3411218495 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde4415af70 0x7fde4406bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.708+0000 7fde4b425640 1 --2- 172.21.15.40:0/3411218495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde4406d5d0 0x7fde440c9620 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.708+0000 7fde4b425640 1 -- 172.21.15.40:0/3411218495 >> 172.21.15.40:0/3411218495 conn(0x7fde4406ca10 msgr2=0x7fde4406ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:22.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.708+0000 7fde4b425640 1 -- 172.21.15.40:0/3411218495 shutdown_connections 2023-10-18T09:07:22.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.708+0000 7fde4b425640 1 -- 172.21.15.40:0/3411218495 wait complete. 2023-10-18T09:07:22.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.710+0000 7fde4b425640 1 Processor -- start 2023-10-18T09:07:22.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.710+0000 7fde4b425640 1 -- start start 2023-10-18T09:07:22.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.711+0000 7fde4b425640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde4406c100 0x7fde44157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:22.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.711+0000 7fde4b425640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde4406d5d0 0x7fde44158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:22.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.711+0000 7fde4b425640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde4415af70 0x7fde44153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:22.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.711+0000 7fde4b425640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde44152cb0 con 0x7fde4406c100 2023-10-18T09:07:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.711+0000 7fde4b425640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde441526e0 con 0x7fde4406d5d0 2023-10-18T09:07:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.711+0000 7fde4b425640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde441576d0 con 0x7fde4415af70 2023-10-18T09:07:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.711+0000 7fde4a423640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde4406c100 0x7fde44157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.711+0000 7fde4a423640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde4406c100 0x7fde44157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46490/0 (socket says 172.21.15.40:46490) 2023-10-18T09:07:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.711+0000 7fde4a423640 1 -- 172.21.15.40:0/936470638 learned_addr learned my addr 172.21.15.40:0/936470638 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:22.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.712+0000 7fde4a423640 1 -- 172.21.15.40:0/936470638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde4415af70 msgr2=0x7fde44153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:22.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.712+0000 7fde4a423640 1 --2- 172.21.15.40:0/936470638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde4415af70 0x7fde44153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.712+0000 7fde4a423640 1 -- 172.21.15.40:0/936470638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde4406d5d0 msgr2=0x7fde44158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:22.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.712+0000 7fde49c22640 1 --2- 172.21.15.40:0/936470638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde4406d5d0 0x7fde44158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:22.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.712+0000 7fde4a423640 1 --2- 172.21.15.40:0/936470638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde4406d5d0 0x7fde44158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.712+0000 7fde4a423640 1 -- 172.21.15.40:0/936470638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde40009560 con 0x7fde4406c100 2023-10-18T09:07:22.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.712+0000 7fde4a423640 1 --2- 172.21.15.40:0/936470638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde4406c100 0x7fde44157190 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fde400099c0 tx=0x7fde4002dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.712+0000 7fde3affd640 1 -- 172.21.15.40:0/936470638 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4002dea0 con 0x7fde4406c100 2023-10-18T09:07:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.713+0000 7fde3affd640 1 -- 172.21.15.40:0/936470638 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fde40003800 con 0x7fde4406c100 2023-10-18T09:07:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.713+0000 7fde4b425640 1 -- 172.21.15.40:0/936470638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde44152960 con 0x7fde4406c100 2023-10-18T09:07:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.713+0000 7fde3affd640 1 -- 172.21.15.40:0/936470638 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40003c10 con 0x7fde4406c100 2023-10-18T09:07:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.713+0000 7fde4b425640 1 -- 172.21.15.40:0/936470638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde44156c80 con 0x7fde4406c100 2023-10-18T09:07:22.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.714+0000 7fde4b425640 1 -- 172.21.15.40:0/936470638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde44066e00 con 0x7fde4406c100 2023-10-18T09:07:22.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.715+0000 7fde3affd640 1 -- 172.21.15.40:0/936470638 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fde40003d70 con 0x7fde4406c100 2023-10-18T09:07:22.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.716+0000 7fde3affd640 1 --2- 172.21.15.40:0/936470638 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde2806ae70 0x7fde2806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:22.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.716+0000 7fde3affd640 1 -- 172.21.15.40:0/936470638 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde400abbd0 con 0x7fde4406c100 2023-10-18T09:07:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.717+0000 7fde49c22640 1 --2- 172.21.15.40:0/936470638 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde2806ae70 0x7fde2806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:22.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.718+0000 7fde49c22640 1 --2- 172.21.15.40:0/936470638 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde2806ae70 0x7fde2806d330 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fde340058c0 tx=0x7fde34005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:22.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.721+0000 7fde3affd640 1 -- 172.21.15.40:0/936470638 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde40076c80 con 0x7fde4406c100 2023-10-18T09:07:22.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.985+0000 7fde4b425640 1 -- 172.21.15.40:0/936470638 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fde440c9b60 con 0x7fde4406c100 2023-10-18T09:07:22.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.989+0000 7fde3affd640 1 -- 172.21.15.40:0/936470638 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7fde440c9b60 con 0x7fde4406c100 2023-10-18T09:07:22.993 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2023-10-18T09:07:22.993 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w== 2023-10-18T09:07:22.994 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2023-10-18T09:07:22.994 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2023-10-18T09:07:22.994 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T09:07:22.994 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2023-10-18T09:07:22.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.993+0000 7fde4b425640 1 -- 172.21.15.40:0/936470638 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde2806ae70 msgr2=0x7fde2806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:22.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.993+0000 7fde4b425640 1 --2- 172.21.15.40:0/936470638 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde2806ae70 0x7fde2806d330 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fde340058c0 tx=0x7fde34005850 comp rx=0 tx=0).stop 2023-10-18T09:07:22.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.993+0000 7fde4b425640 1 -- 172.21.15.40:0/936470638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde4406c100 msgr2=0x7fde44157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:22.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.993+0000 7fde4b425640 1 --2- 172.21.15.40:0/936470638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde4406c100 0x7fde44157190 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fde400099c0 tx=0x7fde4002dcb0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.993+0000 7fde4b425640 1 -- 172.21.15.40:0/936470638 shutdown_connections 2023-10-18T09:07:22.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.993+0000 7fde4b425640 1 --2- 172.21.15.40:0/936470638 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fde2806ae70 0x7fde2806d330 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.993+0000 7fde4b425640 1 --2- 172.21.15.40:0/936470638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde4415af70 0x7fde44153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.993+0000 7fde4b425640 1 --2- 172.21.15.40:0/936470638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde4406d5d0 0x7fde44158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.993+0000 7fde4b425640 1 --2- 172.21.15.40:0/936470638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde4406c100 0x7fde44157190 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.993+0000 7fde4b425640 1 -- 172.21.15.40:0/936470638 >> 172.21.15.40:0/936470638 conn(0x7fde4406ca10 msgr2=0x7fde440afd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.994+0000 7fde4b425640 1 -- 172.21.15.40:0/936470638 shutdown_connections 2023-10-18T09:07:22.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:22.994+0000 7fde4b425640 1 -- 172.21.15.40:0/936470638 wait complete. 2023-10-18T09:07:23.015 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:07:23.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.246+0000 7f50f4932640 1 -- 172.21.15.40:0/2931020131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00b15b0 msgr2=0x7f50f00c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.246+0000 7f50f4932640 1 --2- 172.21.15.40:0/2931020131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00b15b0 0x7f50f00c99c0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009830 tx=0x7f50e402bd60 comp rx=0 tx=0).stop 2023-10-18T09:07:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.246+0000 7f50f4932640 1 -- 172.21.15.40:0/2931020131 shutdown_connections 2023-10-18T09:07:23.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.246+0000 7f50f4932640 1 --2- 172.21.15.40:0/2931020131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00b15b0 0x7f50f00c99c0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:23.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.246+0000 7f50f4932640 1 -- 172.21.15.40:0/2931020131 >> 172.21.15.40:0/2931020131 conn(0x7f50f0069720 msgr2=0x7f50f0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:23.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.246+0000 7f50f4932640 1 -- 172.21.15.40:0/2931020131 shutdown_connections 2023-10-18T09:07:23.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.246+0000 7f50f4932640 1 -- 172.21.15.40:0/2931020131 wait complete. 2023-10-18T09:07:23.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.248+0000 7f50f4932640 1 Processor -- start 2023-10-18T09:07:23.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.248+0000 7f50f4932640 1 -- start start 2023-10-18T09:07:23.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.249+0000 7f50f4932640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f006d6f0 0x7f50f00abf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:23.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.249+0000 7f50f4932640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50f00b15b0 0x7f50f015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:23.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.249+0000 7f50f4932640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00c9f00 0x7f50f014f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.249+0000 7f50f4932640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f01f1c80 con 0x7f50f00c9f00 2023-10-18T09:07:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.249+0000 7f50f4932640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f01f2aa0 con 0x7f50f006d6f0 2023-10-18T09:07:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.249+0000 7f50f4932640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f00ac4a0 con 0x7f50f00b15b0 2023-10-18T09:07:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.249+0000 7f50ef59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00c9f00 0x7f50f014f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:23.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.250+0000 7f50ef59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00c9f00 0x7f50f014f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46502/0 (socket says 172.21.15.40:46502) 2023-10-18T09:07:23.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.250+0000 7f50ef59e640 1 -- 172.21.15.40:0/381323373 learned_addr learned my addr 172.21.15.40:0/381323373 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:23.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.250+0000 7f50ee59c640 1 --2- 172.21.15.40:0/381323373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50f00b15b0 0x7f50f015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:23.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.250+0000 7f50ef59e640 1 -- 172.21.15.40:0/381323373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50f00b15b0 msgr2=0x7f50f015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:23.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.250+0000 7f50ef59e640 1 --2- 172.21.15.40:0/381323373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50f00b15b0 0x7f50f015a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:23.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.250+0000 7f50ef59e640 1 -- 172.21.15.40:0/381323373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f006d6f0 msgr2=0x7f50f00abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.250+0000 7f50eed9d640 1 --2- 172.21.15.40:0/381323373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f006d6f0 0x7f50f00abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.250+0000 7f50ef59e640 1 --2- 172.21.15.40:0/381323373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f006d6f0 0x7f50f00abf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.250+0000 7f50ef59e640 1 -- 172.21.15.40:0/381323373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e4009490 con 0x7f50f00c9f00 2023-10-18T09:07:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.250+0000 7f50ef59e640 1 --2- 172.21.15.40:0/381323373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00c9f00 0x7f50f014f950 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f50e800b970 tx=0x7f50e800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.251+0000 7f50d77fe640 1 -- 172.21.15.40:0/381323373 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e800cb40 con 0x7f50f00c9f00 2023-10-18T09:07:23.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.251+0000 7f50d77fe640 1 -- 172.21.15.40:0/381323373 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f50e800cd90 con 0x7f50f00c9f00 2023-10-18T09:07:23.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.251+0000 7f50d77fe640 1 -- 172.21.15.40:0/381323373 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e80146a0 con 0x7f50f00c9f00 2023-10-18T09:07:23.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.251+0000 7f50f4932640 1 -- 172.21.15.40:0/381323373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f01f1e90 con 0x7f50f00c9f00 2023-10-18T09:07:23.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.251+0000 7f50f4932640 1 -- 172.21.15.40:0/381323373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f01f2450 con 0x7f50f00c9f00 2023-10-18T09:07:23.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.252+0000 7f50f4932640 1 -- 172.21.15.40:0/381323373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f01f2c50 con 0x7f50f00c9f00 2023-10-18T09:07:23.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.253+0000 7f50d77fe640 1 -- 172.21.15.40:0/381323373 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f50e801e440 con 0x7f50f00c9f00 2023-10-18T09:07:23.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.254+0000 7f50d77fe640 1 --2- 172.21.15.40:0/381323373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f50dc06ae40 0x7f50dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.254+0000 7f50d77fe640 1 -- 172.21.15.40:0/381323373 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50e80896c0 con 0x7f50f00c9f00 2023-10-18T09:07:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.255+0000 7f50eed9d640 1 --2- 172.21.15.40:0/381323373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f50dc06ae40 0x7f50dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:23.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.256+0000 7f50eed9d640 1 --2- 172.21.15.40:0/381323373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f50dc06ae40 0x7f50dc06d300 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f50e00046b0 tx=0x7f50e0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:23.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.258+0000 7f50d77fe640 1 -- 172.21.15.40:0/381323373 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f50f01f2c50 con 0x7f50f00c9f00 2023-10-18T09:07:23.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.494+0000 7f50f4932640 1 -- 172.21.15.40:0/381323373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50f01f2c50 con 0x7f50f00c9f00 2023-10-18T09:07:23.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.495+0000 7f50d77fe640 1 -- 172.21.15.40:0/381323373 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f50e8054760 con 0x7f50f00c9f00 2023-10-18T09:07:23.499 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:07:23.499 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6823,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3592705338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3592705338},{"type":"v1","addr":"172.21.15.40:6835","nonce":3592705338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":6829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1076939595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1076939595},{"type":"v1","addr":"172.21.15.144:6836","nonce":1076939595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":6832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2483821333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2483821333},{"type":"v1","addr":"172.21.15.144:6837","nonce":2483821333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:07:16.832438+0000","modified":"2023-10-18T09:07:19.910479+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8826},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8826":{"gid":8826,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/137474609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":137474609},{"type":"v1","addr":"172.21.15.40:6834","nonce":137474609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-18T09:07:23.499 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 82 2023-10-18T09:07:23.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.498+0000 7f50f4932640 1 -- 172.21.15.40:0/381323373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f50dc06ae40 msgr2=0x7f50dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:23.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.498+0000 7f50f4932640 1 --2- 172.21.15.40:0/381323373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f50dc06ae40 0x7f50dc06d300 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f50e00046b0 tx=0x7f50e0009040 comp rx=0 tx=0).stop 2023-10-18T09:07:23.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.498+0000 7f50f4932640 1 -- 172.21.15.40:0/381323373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00c9f00 msgr2=0x7f50f014f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:23.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.498+0000 7f50f4932640 1 --2- 172.21.15.40:0/381323373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00c9f00 0x7f50f014f950 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f50e800b970 tx=0x7f50e800be30 comp rx=0 tx=0).stop 2023-10-18T09:07:23.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.498+0000 7f50f4932640 1 -- 172.21.15.40:0/381323373 shutdown_connections 2023-10-18T09:07:23.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.498+0000 7f50f4932640 1 --2- 172.21.15.40:0/381323373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f50dc06ae40 0x7f50dc06d300 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:23.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.498+0000 7f50f4932640 1 --2- 172.21.15.40:0/381323373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50f00c9f00 0x7f50f014f950 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:23.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.498+0000 7f50f4932640 1 --2- 172.21.15.40:0/381323373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f50f00b15b0 0x7f50f015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:23.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.498+0000 7f50f4932640 1 --2- 172.21.15.40:0/381323373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50f006d6f0 0x7f50f00abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:23.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.498+0000 7f50f4932640 1 -- 172.21.15.40:0/381323373 >> 172.21.15.40:0/381323373 conn(0x7f50f0069720 msgr2=0x7f50f0156830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:23.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.498+0000 7f50f4932640 1 -- 172.21.15.40:0/381323373 shutdown_connections 2023-10-18T09:07:23.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:23.498+0000 7f50f4932640 1 -- 172.21.15.40:0/381323373 wait complete. 2023-10-18T09:07:23.923 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T09:07:16.832438+0000', 'modified': '2023-10-18T09:07:19.910479+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8826}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8826': {'gid': 8826, 'name': 'c', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6834/137474609', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6832', 'nonce': 137474609}, {'type': 'v1', 'addr': '172.21.15.40:6834', 'nonce': 137474609}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T09:07:23.924 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T09:07:23.924 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-18T09:07:24.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.168+0000 7f5a0ffff640 1 -- 172.21.15.40:0/2882528585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59fc004ab0 msgr2=0x7f59fc004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:24.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.168+0000 7f5a0ffff640 1 --2- 172.21.15.40:0/2882528585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59fc004ab0 0x7f59fc004ed0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009e20 tx=0x7f59fc006900 comp rx=0 tx=0).stop 2023-10-18T09:07:24.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.168+0000 7f5a0f7fe640 1 -- 172.21.15.40:0/2882528585 reap_dead start 2023-10-18T09:07:24.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.168+0000 7f5a0ffff640 1 -- 172.21.15.40:0/2882528585 shutdown_connections 2023-10-18T09:07:24.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.168+0000 7f5a0ffff640 1 -- 172.21.15.40:0/2882528585 >> 172.21.15.40:0/2882528585 conn(0x7f5a10069720 msgr2=0x7f5a10069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:24.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.168+0000 7f5a0ffff640 1 -- 172.21.15.40:0/2882528585 shutdown_connections 2023-10-18T09:07:24.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.168+0000 7f5a0ffff640 1 -- 172.21.15.40:0/2882528585 wait complete. 2023-10-18T09:07:24.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.170+0000 7f5a0ffff640 1 Processor -- start 2023-10-18T09:07:24.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.170+0000 7f5a0ffff640 1 -- start start 2023-10-18T09:07:24.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.171+0000 7f5a0ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59fc004ab0 0x7f5a10159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.171+0000 7f5a0ffff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a1014b130 0x7f5a1014cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.171+0000 7f5a0ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a1014d010 0x7f5a101e4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.171+0000 7f5a0ffff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1014b680 con 0x7f59fc004ab0 2023-10-18T09:07:24.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.171+0000 7f5a0ffff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1014b960 con 0x7f5a1014d010 2023-10-18T09:07:24.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.171+0000 7f5a0ffff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1014bc40 con 0x7f5a1014b130 2023-10-18T09:07:24.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.171+0000 7f5a0effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59fc004ab0 0x7f5a10159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.172+0000 7f5a0effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59fc004ab0 0x7f5a10159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46520/0 (socket says 172.21.15.40:46520) 2023-10-18T09:07:24.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.172+0000 7f5a0effd640 1 -- 172.21.15.40:0/3574243509 learned_addr learned my addr 172.21.15.40:0/3574243509 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:24.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.172+0000 7f5a0e7fc640 1 --2- 172.21.15.40:0/3574243509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a1014b130 0x7f5a1014cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.172+0000 7f5a0f7fe640 1 --2- 172.21.15.40:0/3574243509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a1014d010 0x7f5a101e4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.172+0000 7f5a0effd640 1 -- 172.21.15.40:0/3574243509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a1014b130 msgr2=0x7f5a1014cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:24.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.172+0000 7f5a0effd640 1 --2- 172.21.15.40:0/3574243509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a1014b130 0x7f5a1014cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.172+0000 7f5a0effd640 1 -- 172.21.15.40:0/3574243509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a1014d010 msgr2=0x7f5a101e4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:24.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.172+0000 7f5a0effd640 1 --2- 172.21.15.40:0/3574243509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a1014d010 0x7f5a101e4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.172+0000 7f5a0effd640 1 -- 172.21.15.40:0/3574243509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1014bf20 con 0x7f59fc004ab0 2023-10-18T09:07:24.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.172+0000 7f5a0f7fe640 1 --2- 172.21.15.40:0/3574243509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a1014d010 0x7f5a101e4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:24.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.172+0000 7f5a0effd640 1 --2- 172.21.15.40:0/3574243509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59fc004ab0 0x7f5a10159fd0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f5a0002ca80 tx=0x7f5a00033c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:24.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.173+0000 7f5a077fe640 1 -- 172.21.15.40:0/3574243509 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00003b70 con 0x7f59fc004ab0 2023-10-18T09:07:24.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.173+0000 7f5a077fe640 1 -- 172.21.15.40:0/3574243509 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5a00003cd0 con 0x7f59fc004ab0 2023-10-18T09:07:24.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.173+0000 7f5a077fe640 1 -- 172.21.15.40:0/3574243509 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00004270 con 0x7f59fc004ab0 2023-10-18T09:07:24.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.173+0000 7f5a0ffff640 1 -- 172.21.15.40:0/3574243509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1014c200 con 0x7f59fc004ab0 2023-10-18T09:07:24.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.173+0000 7f5a0ffff640 1 -- 172.21.15.40:0/3574243509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a101e49d0 con 0x7f59fc004ab0 2023-10-18T09:07:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.175+0000 7f5a077fe640 1 -- 172.21.15.40:0/3574243509 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5a000043d0 con 0x7f59fc004ab0 2023-10-18T09:07:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.176+0000 7f5a077fe640 1 --2- 172.21.15.40:0/3574243509 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f59f006ae40 0x7f59f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.176+0000 7f5a077fe640 1 -- 172.21.15.40:0/3574243509 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a000ac830 con 0x7f59fc004ab0 2023-10-18T09:07:24.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.177+0000 7f5a0e7fc640 1 --2- 172.21.15.40:0/3574243509 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f59f006ae40 0x7f59f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.177+0000 7f5a17e33740 1 Processor -- start 2023-10-18T09:07:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.177+0000 7f5a17e33740 1 -- start start 2023-10-18T09:07:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.178+0000 7f5a0e7fc640 1 --2- 172.21.15.40:0/3574243509 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f59f006ae40 0x7f59f006d300 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5a101e39b0 tx=0x7f59f8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.178+0000 7f5a17e33740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29ce194f0 0x55b29ce170b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.178+0000 7f5a17e33740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29cdf8d30 0x55b29cdd0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.178+0000 7f5a17e33740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdd0f60 0x55b29cdfb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.178+0000 7f5a17e33740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29cd3e740 con 0x55b29cdd0f60 2023-10-18T09:07:24.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.178+0000 7f5a17e33740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29cd3be30 con 0x55b29ce194f0 2023-10-18T09:07:24.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.178+0000 7f5a17e33740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b29cd37720 con 0x55b29cdf8d30 2023-10-18T09:07:24.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.178+0000 7f5a0f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdd0f60 0x55b29cdfb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.178+0000 7f5a0f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdd0f60 0x55b29cdfb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46528/0 (socket says 172.21.15.40:46528) 2023-10-18T09:07:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.179+0000 7f5a0f7fe640 1 -- 172.21.15.40:0/3797025595 learned_addr learned my addr 172.21.15.40:0/3797025595 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.179+0000 7f5a0effd640 1 --2- 172.21.15.40:0/3797025595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29ce194f0 0x55b29ce170b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.179+0000 7f5a0e7fc640 1 --2- 172.21.15.40:0/3797025595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29cdf8d30 0x55b29cdd0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.179+0000 7f5a0f7fe640 1 -- 172.21.15.40:0/3797025595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29cdf8d30 msgr2=0x55b29cdd0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:07:24.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.179+0000 7f5a0f7fe640 1 --2- 172.21.15.40:0/3797025595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29cdf8d30 0x55b29cdd0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.179+0000 7f5a0f7fe640 1 -- 172.21.15.40:0/3797025595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29ce194f0 msgr2=0x55b29ce170b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:07:24.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.179+0000 7f5a0f7fe640 1 --2- 172.21.15.40:0/3797025595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29ce194f0 0x55b29ce170b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.179+0000 7f5a0f7fe640 1 -- 172.21.15.40:0/3797025595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b29ccfb910 con 0x55b29cdd0f60 2023-10-18T09:07:24.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.179+0000 7f5a0f7fe640 1 --2- 172.21.15.40:0/3797025595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdd0f60 0x55b29cdfb700 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f5a100b0120 tx=0x7f5a08007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4391b91f1d2c7d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:24.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.179+0000 7f5a04ff9640 1 -- 172.21.15.40:0/3797025595 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0800b070 con 0x55b29cdd0f60 2023-10-18T09:07:24.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.179+0000 7f5a04ff9640 1 -- 172.21.15.40:0/3797025595 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5a08005d80 con 0x55b29cdd0f60 2023-10-18T09:07:24.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.180+0000 7f5a04ff9640 1 -- 172.21.15.40:0/3797025595 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08007330 con 0x55b29cdd0f60 2023-10-18T09:07:24.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.180+0000 7f5a17e33740 1 -- 172.21.15.40:0/3797025595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdd0f60 msgr2=0x55b29cdfb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:24.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.180+0000 7f5a17e33740 1 --2- 172.21.15.40:0/3797025595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdd0f60 0x55b29cdfb700 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f5a100b0120 tx=0x7f5a08007c70 comp rx=0 tx=0).stop 2023-10-18T09:07:24.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.180+0000 7f5a17e33740 1 -- 172.21.15.40:0/3797025595 shutdown_connections 2023-10-18T09:07:24.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.180+0000 7f5a17e33740 1 --2- 172.21.15.40:0/3797025595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdd0f60 0x55b29cdfb700 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.180+0000 7f5a17e33740 1 --2- 172.21.15.40:0/3797025595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29cdf8d30 0x55b29cdd0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.180+0000 7f5a17e33740 1 --2- 172.21.15.40:0/3797025595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29ce194f0 0x55b29ce170b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.180+0000 7f5a17e33740 1 -- 172.21.15.40:0/3797025595 >> 172.21.15.40:0/3797025595 conn(0x55b29cdcde10 msgr2=0x55b29ce16700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.180+0000 7f5a17e33740 1 -- 172.21.15.40:0/3797025595 shutdown_connections 2023-10-18T09:07:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.180+0000 7f5a17e33740 1 -- 172.21.15.40:0/3797025595 wait complete. 2023-10-18T09:07:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.181+0000 7f5a17e33740 1 -- start start 2023-10-18T09:07:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.181+0000 7f5a17e33740 1 Processor -- start 2023-10-18T09:07:24.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.182+0000 7f5a17e33740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdf8d30 0x55b29ce10340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.182+0000 7f5a17e33740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29ce194f0 0x55b29ce12b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.182+0000 7f5a17e33740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29cde5d60 0x55b29ce13050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.182+0000 7f5a17e33740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29ccd5970 con 0x55b29cdf8d30 2023-10-18T09:07:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.182+0000 7f5a17e33740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29ccd5610 con 0x55b29cde5d60 2023-10-18T09:07:24.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.182+0000 7f5a17e33740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b29ca9bf20 con 0x55b29ce194f0 2023-10-18T09:07:24.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.182+0000 7f5a17e33740 10 client.0 _ll_register_callbacks cb 0x55b29ccf90c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:07:24.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.183+0000 7f5a0effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdf8d30 0x55b29ce10340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.183+0000 7f5a0effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdf8d30 0x55b29ce10340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46534/0 (socket says 172.21.15.40:46534) 2023-10-18T09:07:24.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.183+0000 7f5a0effd640 1 -- 172.21.15.40:0/1725557853 learned_addr learned my addr 172.21.15.40:0/1725557853 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:24.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.183+0000 7f5a0effd640 1 -- 172.21.15.40:0/1725557853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29ce194f0 msgr2=0x55b29ce12b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:07:24.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.183+0000 7f5a0effd640 1 --2- 172.21.15.40:0/1725557853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29ce194f0 0x55b29ce12b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.183+0000 7f5a0effd640 1 -- 172.21.15.40:0/1725557853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29cde5d60 msgr2=0x55b29ce13050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:07:24.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.183+0000 7f5a0f7fe640 1 --2- 172.21.15.40:0/1725557853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29cde5d60 0x55b29ce13050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.183+0000 7f5a0effd640 1 --2- 172.21.15.40:0/1725557853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29cde5d60 0x55b29ce13050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.183+0000 7f5a0effd640 1 -- 172.21.15.40:0/1725557853 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1015b2c0 con 0x55b29cdf8d30 2023-10-18T09:07:24.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.183+0000 7f5a0f7fe640 1 --2- 172.21.15.40:0/1725557853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29cde5d60 0x55b29ce13050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:24.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.183+0000 7f5a0effd640 1 --2- 172.21.15.40:0/1725557853 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdf8d30 0x55b29ce10340 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f5a00077bb0 tx=0x7f5a0007bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93abd7341ece340c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:24.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.183+0000 7f59edffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T09:07:24.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.184+0000 7f59edffb640 1 -- 172.21.15.40:0/1725557853 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0007f800 con 0x55b29cdf8d30 2023-10-18T09:07:24.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.184+0000 7f59edffb640 1 -- 172.21.15.40:0/1725557853 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(179..179 src has 1..179) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a000839b0 con 0x55b29cdf8d30 2023-10-18T09:07:24.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.184+0000 7f5a17e33740 1 -- 172.21.15.40:0/1725557853 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b29cd30700 con 0x55b29cdf8d30 2023-10-18T09:07:24.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.184+0000 7f59edffb640 1 -- 172.21.15.40:0/1725557853 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a00003e40 con 0x55b29cdf8d30 2023-10-18T09:07:24.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.184+0000 7f5a17e33740 10 client.8979 fetch_fsmap learned FSMap version 82 2023-10-18T09:07:24.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.184+0000 7f5a17e33740 1 -- 172.21.15.40:0/1725557853 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55b29cd35da0 con 0x55b29cdf8d30 2023-10-18T09:07:24.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.184+0000 7f59edffb640 1 -- 172.21.15.40:0/1725557853 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55b29cd30700 con 0x55b29cdf8d30 2023-10-18T09:07:24.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.184+0000 7f59edffb640 1 -- 172.21.15.40:0/1725557853 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5a00044020 con 0x55b29cdf8d30 2023-10-18T09:07:24.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.184+0000 7f5a17e33740 10 client.8979 fetch_fsmap finished waiting for FSMap version 82 2023-10-18T09:07:24.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.184+0000 7f5a17e33740 10 client.8979 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T09:07:24.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.184+0000 7f5a17e33740 20 client.8979 populate_metadata read hostname 'smithi040' 2023-10-18T09:07:24.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.184+0000 7f5a17e33740 1 --2- 172.21.15.40:0/1725557853 >> [v2:172.21.15.40:6832/137474609,v1:172.21.15.40:6834/137474609] conn(0x55b29ce15700 0x55b29ce114f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.184+0000 7f5a17e33740 4 client.8979 mds_command: new command op to 8826 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:07:24.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.184+0000 7f5a17e33740 1 -- 172.21.15.40:0/1725557853 --> [v2:172.21.15.40:6832/137474609,v1:172.21.15.40:6834/137474609] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b29cd30420 con 0x55b29ce15700 2023-10-18T09:07:24.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.185+0000 7f5a0e7fc640 1 --2- 172.21.15.40:0/1725557853 >> [v2:172.21.15.40:6832/137474609,v1:172.21.15.40:6834/137474609] conn(0x55b29ce15700 0x55b29ce114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.185+0000 7f5a0e7fc640 1 --2- 172.21.15.40:0/1725557853 >> [v2:172.21.15.40:6832/137474609,v1:172.21.15.40:6834/137474609] conn(0x55b29ce15700 0x55b29ce114f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcbebcc7e9c05cbb server_cookie=23e617729e8306b8 in_seq=0 out_seq=0 2023-10-18T09:07:24.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.185+0000 7f59edffb640 10 client.8979 ms_handle_connect on v2:172.21.15.40:6832/137474609 2023-10-18T09:07:24.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.186+0000 7f59edffb640 1 -- 172.21.15.40:0/1725557853 <== mds.0 v2:172.21.15.40:6832/137474609 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b29cd30420 con 0x55b29ce15700 2023-10-18T09:07:24.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.186+0000 7f59edffb640 10 client.8979 handle_command_reply: tid=0 2023-10-18T09:07:24.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.186+0000 7f5a17e33740 1 client.8979 shutdown 2023-10-18T09:07:24.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.186+0000 7f5a17e33740 1 -- 172.21.15.40:0/1725557853 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdf8d30 msgr2=0x55b29ce10340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:24.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.186+0000 7f5a17e33740 1 --2- 172.21.15.40:0/1725557853 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdf8d30 0x55b29ce10340 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f5a00077bb0 tx=0x7f5a0007bdb0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.186+0000 7f5a17e33740 1 -- 172.21.15.40:0/1725557853 shutdown_connections 2023-10-18T09:07:24.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.186+0000 7f5a17e33740 1 --2- 172.21.15.40:0/1725557853 >> [v2:172.21.15.40:6832/137474609,v1:172.21.15.40:6834/137474609] conn(0x55b29ce15700 0x55b29ce114f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.186+0000 7f5a17e33740 1 --2- 172.21.15.40:0/1725557853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29cde5d60 0x55b29ce13050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.186+0000 7f5a17e33740 1 --2- 172.21.15.40:0/1725557853 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29ce194f0 0x55b29ce12b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.186+0000 7f5a17e33740 1 --2- 172.21.15.40:0/1725557853 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdf8d30 0x55b29ce10340 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.186+0000 7f59edffb640 0 client.8979 ms_handle_reset on v2:172.21.15.40:6832/137474609 2023-10-18T09:07:24.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.186+0000 7f5a17e33740 1 -- 172.21.15.40:0/1725557853 >> 172.21.15.40:0/1725557853 conn(0x55b29cdd0f60 msgr2=0x55b29cdcac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:24.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.187+0000 7f5a17e33740 1 -- 172.21.15.40:0/1725557853 shutdown_connections 2023-10-18T09:07:24.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.187+0000 7f5a17e33740 1 -- 172.21.15.40:0/1725557853 wait complete. 2023-10-18T09:07:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.187+0000 7f5a17e33740 20 client.8979 trim_cache size 0 max 16384 2023-10-18T09:07:24.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.204+0000 7f5a17e33740 1 Processor -- start 2023-10-18T09:07:24.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.204+0000 7f5a17e33740 1 -- start start 2023-10-18T09:07:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.205+0000 7f5a17e33740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29cde5d60 0x55b29ce140e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.205+0000 7f5a17e33740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdf8d30 0x55b29cdd0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.205+0000 7f5a17e33740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29ce194f0 0x55b29cdd06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.205+0000 7f5a17e33740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29ccf6f10 con 0x55b29cdf8d30 2023-10-18T09:07:24.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.205+0000 7f5a17e33740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29cde5b70 con 0x55b29ce194f0 2023-10-18T09:07:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.205+0000 7f5a17e33740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b29cdd2800 con 0x55b29cde5d60 2023-10-18T09:07:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.205+0000 7f5a0e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdf8d30 0x55b29cdd0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.205+0000 7f5a0e7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdf8d30 0x55b29cdd0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46544/0 (socket says 172.21.15.40:46544) 2023-10-18T09:07:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.205+0000 7f5a0e7fc640 1 -- 172.21.15.40:0/2610092039 learned_addr learned my addr 172.21.15.40:0/2610092039 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.205+0000 7f5a0e7fc640 1 -- 172.21.15.40:0/2610092039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29cde5d60 msgr2=0x55b29ce140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:07:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.205+0000 7f5a0e7fc640 1 --2- 172.21.15.40:0/2610092039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29cde5d60 0x55b29ce140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.205+0000 7f5a0e7fc640 1 -- 172.21.15.40:0/2610092039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29ce194f0 msgr2=0x55b29cdd06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:07:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.205+0000 7f5a0e7fc640 1 --2- 172.21.15.40:0/2610092039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29ce194f0 0x55b29cdd06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.205+0000 7f5a0e7fc640 1 -- 172.21.15.40:0/2610092039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a10152da0 con 0x55b29cdf8d30 2023-10-18T09:07:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.206+0000 7f5a0e7fc640 1 --2- 172.21.15.40:0/2610092039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdf8d30 0x55b29cdd0160 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x55b29ce15660 tx=0x7f59f8016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511eadb7a5c47759 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.206+0000 7f5a057fa640 1 -- 172.21.15.40:0/2610092039 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f80224c0 con 0x55b29cdf8d30 2023-10-18T09:07:24.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.206+0000 7f5a057fa640 1 -- 172.21.15.40:0/2610092039 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f59f8026070 con 0x55b29cdf8d30 2023-10-18T09:07:24.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.206+0000 7f5a057fa640 1 -- 172.21.15.40:0/2610092039 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f800de90 con 0x55b29cdf8d30 2023-10-18T09:07:24.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.207+0000 7f5a17e33740 1 -- 172.21.15.40:0/2610092039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdf8d30 msgr2=0x55b29cdd0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:24.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.207+0000 7f5a17e33740 1 --2- 172.21.15.40:0/2610092039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdf8d30 0x55b29cdd0160 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x55b29ce15660 tx=0x7f59f8016040 comp rx=0 tx=0).stop 2023-10-18T09:07:24.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.207+0000 7f5a17e33740 1 -- 172.21.15.40:0/2610092039 shutdown_connections 2023-10-18T09:07:24.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.207+0000 7f5a17e33740 1 --2- 172.21.15.40:0/2610092039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29ce194f0 0x55b29cdd06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.207+0000 7f5a17e33740 1 --2- 172.21.15.40:0/2610092039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29cdf8d30 0x55b29cdd0160 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.207+0000 7f5a17e33740 1 --2- 172.21.15.40:0/2610092039 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29cde5d60 0x55b29ce140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.207+0000 7f5a17e33740 1 -- 172.21.15.40:0/2610092039 >> 172.21.15.40:0/2610092039 conn(0x55b29cdd0f60 msgr2=0x55b29cdc7530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:24.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.207+0000 7f5a17e33740 1 -- 172.21.15.40:0/2610092039 shutdown_connections 2023-10-18T09:07:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.207+0000 7f5a17e33740 1 -- 172.21.15.40:0/2610092039 wait complete. 2023-10-18T09:07:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.207+0000 7f5a17e33740 1 -- start start 2023-10-18T09:07:24.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.208+0000 7f5a17e33740 1 Processor -- start 2023-10-18T09:07:24.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.209+0000 7f5a17e33740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29cde5d60 0x55b29ce864e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.209+0000 7f5a17e33740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29cdf8d30 0x55b29ce293c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.209+0000 7f5a17e33740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29ce194f0 0x55b29ce29900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.209+0000 7f5a17e33740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29cdd0b60 con 0x55b29ce194f0 2023-10-18T09:07:24.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.209+0000 7f5a17e33740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29ce19970 con 0x55b29cde5d60 2023-10-18T09:07:24.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.209+0000 7f5a17e33740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b29ce19c50 con 0x55b29cdf8d30 2023-10-18T09:07:24.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.209+0000 7f5a17e33740 10 client.0 _ll_register_callbacks cb 0x7f5a080016b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:07:24.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.209+0000 7f5a0f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29ce194f0 0x55b29ce29900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.209+0000 7f5a0f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29ce194f0 0x55b29ce29900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46552/0 (socket says 172.21.15.40:46552) 2023-10-18T09:07:24.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.209+0000 7f5a0f7fe640 1 -- 172.21.15.40:0/3861280777 learned_addr learned my addr 172.21.15.40:0/3861280777 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:24.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.209+0000 7f5a0effd640 1 --2- 172.21.15.40:0/3861280777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29cde5d60 0x55b29ce864e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.209+0000 7f5a0e7fc640 1 --2- 172.21.15.40:0/3861280777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29cdf8d30 0x55b29ce293c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.210+0000 7f5a0f7fe640 1 -- 172.21.15.40:0/3861280777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29cdf8d30 msgr2=0x55b29ce293c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:07:24.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.210+0000 7f5a0f7fe640 1 --2- 172.21.15.40:0/3861280777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29cdf8d30 0x55b29ce293c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.210+0000 7f5a0f7fe640 1 -- 172.21.15.40:0/3861280777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29cde5d60 msgr2=0x55b29ce864e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:07:24.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.210+0000 7f5a0f7fe640 1 --2- 172.21.15.40:0/3861280777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29cde5d60 0x55b29ce864e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.210+0000 7f5a0f7fe640 1 -- 172.21.15.40:0/3861280777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f801c040 con 0x55b29ce194f0 2023-10-18T09:07:24.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.210+0000 7f5a0f7fe640 1 --2- 172.21.15.40:0/3861280777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29ce194f0 0x55b29ce29900 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f5a08015040 tx=0x7f5a08007300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e13609b5c0b4904 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:24.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.210+0000 7f59edffb640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T09:07:24.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.210+0000 7f59edffb640 1 -- 172.21.15.40:0/3861280777 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0800b040 con 0x55b29ce194f0 2023-10-18T09:07:24.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.210+0000 7f59edffb640 1 -- 172.21.15.40:0/3861280777 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(179..179 src has 1..179) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a0800d6d0 con 0x55b29ce194f0 2023-10-18T09:07:24.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.210+0000 7f5a17e33740 1 -- 172.21.15.40:0/3861280777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a08005d80 con 0x55b29ce194f0 2023-10-18T09:07:24.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.211+0000 7f59edffb640 1 -- 172.21.15.40:0/3861280777 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a0800d8b0 con 0x55b29ce194f0 2023-10-18T09:07:24.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.211+0000 7f5a17e33740 10 client.8985 fetch_fsmap learned FSMap version 82 2023-10-18T09:07:24.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.211+0000 7f5a17e33740 1 -- 172.21.15.40:0/3861280777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55b29ccd4b80 con 0x55b29ce194f0 2023-10-18T09:07:24.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.211+0000 7f59edffb640 1 -- 172.21.15.40:0/3861280777 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5a08005d80 con 0x55b29ce194f0 2023-10-18T09:07:24.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.211+0000 7f59edffb640 1 -- 172.21.15.40:0/3861280777 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5a0800da10 con 0x55b29ce194f0 2023-10-18T09:07:24.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.211+0000 7f5a17e33740 10 client.8985 fetch_fsmap finished waiting for FSMap version 82 2023-10-18T09:07:24.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.211+0000 7f5a17e33740 10 client.8985 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T09:07:24.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.211+0000 7f5a17e33740 20 client.8985 populate_metadata read hostname 'smithi040' 2023-10-18T09:07:24.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.211+0000 7f5a17e33740 1 --2- 172.21.15.40:0/3861280777 >> [v2:172.21.15.40:6832/137474609,v1:172.21.15.40:6834/137474609] conn(0x55b29ce17ab0 0x55b29ce17e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.211+0000 7f5a17e33740 4 client.8985 mds_command: new command op to 8826 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:07:24.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.211+0000 7f5a17e33740 1 -- 172.21.15.40:0/3861280777 --> [v2:172.21.15.40:6832/137474609,v1:172.21.15.40:6834/137474609] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b29ca396c0 con 0x55b29ce17ab0 2023-10-18T09:07:24.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.211+0000 7f5a0effd640 1 --2- 172.21.15.40:0/3861280777 >> [v2:172.21.15.40:6832/137474609,v1:172.21.15.40:6834/137474609] conn(0x55b29ce17ab0 0x55b29ce17e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.212+0000 7f5a0effd640 1 --2- 172.21.15.40:0/3861280777 >> [v2:172.21.15.40:6832/137474609,v1:172.21.15.40:6834/137474609] conn(0x55b29ce17ab0 0x55b29ce17e90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f98400852c70492b server_cookie=347c0feca80358ff in_seq=0 out_seq=0 2023-10-18T09:07:24.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.212+0000 7f59edffb640 10 client.8985 ms_handle_connect on v2:172.21.15.40:6832/137474609 2023-10-18T09:07:24.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.212+0000 7f59edffb640 1 -- 172.21.15.40:0/3861280777 <== mds.0 v2:172.21.15.40:6832/137474609 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b29ca396c0 con 0x55b29ce17ab0 2023-10-18T09:07:24.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.212+0000 7f59edffb640 10 client.8985 handle_command_reply: tid=0 2023-10-18T09:07:24.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.212+0000 7f5a17e33740 1 client.8985 shutdown 2023-10-18T09:07:24.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.212+0000 7f5a17e33740 1 -- 172.21.15.40:0/3861280777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29ce194f0 msgr2=0x55b29ce29900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:24.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.212+0000 7f5a17e33740 1 --2- 172.21.15.40:0/3861280777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29ce194f0 0x55b29ce29900 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f5a08015040 tx=0x7f5a08007300 comp rx=0 tx=0).stop 2023-10-18T09:07:24.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.212+0000 7f5a17e33740 1 -- 172.21.15.40:0/3861280777 shutdown_connections 2023-10-18T09:07:24.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.212+0000 7f5a17e33740 1 --2- 172.21.15.40:0/3861280777 >> [v2:172.21.15.40:6832/137474609,v1:172.21.15.40:6834/137474609] conn(0x55b29ce17ab0 0x55b29ce17e90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.212+0000 7f5a17e33740 1 --2- 172.21.15.40:0/3861280777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55b29ce194f0 0x55b29ce29900 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.212+0000 7f5a17e33740 1 --2- 172.21.15.40:0/3861280777 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b29cdf8d30 0x55b29ce293c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.212+0000 7f5a17e33740 1 --2- 172.21.15.40:0/3861280777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b29cde5d60 0x55b29ce864e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.212+0000 7f59edffb640 0 client.8985 ms_handle_reset on v2:172.21.15.40:6832/137474609 2023-10-18T09:07:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.212+0000 7f5a17e33740 1 -- 172.21.15.40:0/3861280777 >> 172.21.15.40:0/3861280777 conn(0x55b29cdd0f60 msgr2=0x55b29cde4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.213+0000 7f5a17e33740 1 -- 172.21.15.40:0/3861280777 shutdown_connections 2023-10-18T09:07:24.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.213+0000 7f5a17e33740 1 -- 172.21.15.40:0/3861280777 wait complete. 2023-10-18T09:07:24.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.213+0000 7f5a17e33740 20 client.8985 trim_cache size 0 max 16384 2023-10-18T09:07:24.228 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:07:24.229 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:07:24.229 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-10-18T09:07:24.229 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 8826, 2023-10-18T09:07:24.229 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-10-18T09:07:24.229 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-10-18T09:07:24.230 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-10-18T09:07:24.230 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.8509528130000001, 2023-10-18T09:07:24.230 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 82, 2023-10-18T09:07:24.230 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 179, 2023-10-18T09:07:24.230 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 179, 2023-10-18T09:07:24.231 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 13.738450013 2023-10-18T09:07:24.231 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:07:24.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.213+0000 7f5a0ffff640 1 -- 172.21.15.40:0/3574243509 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f59f006ae40 msgr2=0x7f59f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:24.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.213+0000 7f5a0ffff640 1 --2- 172.21.15.40:0/3574243509 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f59f006ae40 0x7f59f006d300 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5a101e39b0 tx=0x7f59f8009040 comp rx=0 tx=0).stop 2023-10-18T09:07:24.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.213+0000 7f5a0ffff640 1 -- 172.21.15.40:0/3574243509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59fc004ab0 msgr2=0x7f5a10159fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:24.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.213+0000 7f5a0ffff640 1 --2- 172.21.15.40:0/3574243509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59fc004ab0 0x7f5a10159fd0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f5a0002ca80 tx=0x7f5a00033c70 comp rx=0 tx=0).stop 2023-10-18T09:07:24.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.213+0000 7f5a0ffff640 1 -- 172.21.15.40:0/3574243509 shutdown_connections 2023-10-18T09:07:24.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.213+0000 7f5a0ffff640 1 --2- 172.21.15.40:0/3574243509 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f59f006ae40 0x7f59f006d300 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.213+0000 7f5a0ffff640 1 --2- 172.21.15.40:0/3574243509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a1014d010 0x7f5a101e4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.213+0000 7f5a0ffff640 1 --2- 172.21.15.40:0/3574243509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5a1014b130 0x7f5a1014cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.213+0000 7f5a0ffff640 1 --2- 172.21.15.40:0/3574243509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59fc004ab0 0x7f5a10159fd0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.213+0000 7f5a0ffff640 1 -- 172.21.15.40:0/3574243509 >> 172.21.15.40:0/3574243509 conn(0x7f5a10069720 msgr2=0x7f5a101e5860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:24.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.214+0000 7f5a0ffff640 1 -- 172.21.15.40:0/3574243509 shutdown_connections 2023-10-18T09:07:24.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.214+0000 7f5a0ffff640 1 -- 172.21.15.40:0/3574243509 wait complete. 2023-10-18T09:07:24.235 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T09:07:24.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.465+0000 7f3ee50c9640 1 -- 172.21.15.40:0/3520905547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee0158310 msgr2=0x7f3ee00b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:24.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.465+0000 7f3ee50c9640 1 --2- 172.21.15.40:0/3520905547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee0158310 0x7f3ee00b1770 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8005fd0 tx=0x7f3ed802c2e0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.466+0000 7f3ee50c9640 1 -- 172.21.15.40:0/3520905547 shutdown_connections 2023-10-18T09:07:24.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.466+0000 7f3ee50c9640 1 --2- 172.21.15.40:0/3520905547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee0158310 0x7f3ee00b1770 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.466+0000 7f3ee50c9640 1 -- 172.21.15.40:0/3520905547 >> 172.21.15.40:0/3520905547 conn(0x7f3ee0069720 msgr2=0x7f3ee0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:24.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.466+0000 7f3ee50c9640 1 -- 172.21.15.40:0/3520905547 shutdown_connections 2023-10-18T09:07:24.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.466+0000 7f3ee50c9640 1 -- 172.21.15.40:0/3520905547 wait complete. 2023-10-18T09:07:24.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.467+0000 7f3ee50c9640 1 Processor -- start 2023-10-18T09:07:24.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.468+0000 7f3ee50c9640 1 -- start start 2023-10-18T09:07:24.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.468+0000 7f3ee50c9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee006d710 0x7f3ee01595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.469+0000 7f3ee50c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee0151cb0 0x7f3ee0143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.469+0000 7f3ee50c9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee0158310 0x7f3ee014bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.469+0000 7f3ee50c9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee00ab620 con 0x7f3ee0151cb0 2023-10-18T09:07:24.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.469+0000 7f3ee50c9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee0157560 con 0x7f3ee006d710 2023-10-18T09:07:24.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.469+0000 7f3ee50c9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee00aa880 con 0x7f3ee0158310 2023-10-18T09:07:24.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.469+0000 7f3ede59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee0151cb0 0x7f3ee0143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.469+0000 7f3ede59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee0151cb0 0x7f3ee0143250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46580/0 (socket says 172.21.15.40:46580) 2023-10-18T09:07:24.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.469+0000 7f3ede59c640 1 -- 172.21.15.40:0/2655993648 learned_addr learned my addr 172.21.15.40:0/2655993648 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:24.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.469+0000 7f3eded9d640 1 --2- 172.21.15.40:0/2655993648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee006d710 0x7f3ee01595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.469+0000 7f3edf59e640 1 --2- 172.21.15.40:0/2655993648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee0158310 0x7f3ee014bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.469+0000 7f3ede59c640 1 -- 172.21.15.40:0/2655993648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee0158310 msgr2=0x7f3ee014bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:24.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.469+0000 7f3ede59c640 1 --2- 172.21.15.40:0/2655993648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee0158310 0x7f3ee014bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.469+0000 7f3ede59c640 1 -- 172.21.15.40:0/2655993648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee006d710 msgr2=0x7f3ee01595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:24.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.469+0000 7f3ede59c640 1 --2- 172.21.15.40:0/2655993648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee006d710 0x7f3ee01595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.469+0000 7f3ede59c640 1 -- 172.21.15.40:0/2655993648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed8009900 con 0x7f3ee0151cb0 2023-10-18T09:07:24.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.470+0000 7f3ede59c640 1 --2- 172.21.15.40:0/2655993648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee0151cb0 0x7f3ee0143250 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400c410 tx=0x7f3ed400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:24.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.470+0000 7f3ec77fe640 1 -- 172.21.15.40:0/2655993648 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed40035f0 con 0x7f3ee0151cb0 2023-10-18T09:07:24.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.470+0000 7f3ec77fe640 1 -- 172.21.15.40:0/2655993648 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3ed4003750 con 0x7f3ee0151cb0 2023-10-18T09:07:24.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.470+0000 7f3ec77fe640 1 -- 172.21.15.40:0/2655993648 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4003ba0 con 0x7f3ee0151cb0 2023-10-18T09:07:24.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.470+0000 7f3ee50c9640 1 -- 172.21.15.40:0/2655993648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee00aab60 con 0x7f3ee0151cb0 2023-10-18T09:07:24.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.470+0000 7f3ee50c9640 1 -- 172.21.15.40:0/2655993648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee00ab120 con 0x7f3ee0151cb0 2023-10-18T09:07:24.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.471+0000 7f3ee50c9640 1 -- 172.21.15.40:0/2655993648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee00ab2e0 con 0x7f3ee0151cb0 2023-10-18T09:07:24.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.472+0000 7f3ec77fe640 1 -- 172.21.15.40:0/2655993648 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3ed4003d00 con 0x7f3ee0151cb0 2023-10-18T09:07:24.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.473+0000 7f3ec77fe640 1 --2- 172.21.15.40:0/2655993648 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ecc06ae40 0x7f3ecc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:24.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.473+0000 7f3ec77fe640 1 -- 172.21.15.40:0/2655993648 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ed4088a90 con 0x7f3ee0151cb0 2023-10-18T09:07:24.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.474+0000 7f3eded9d640 1 --2- 172.21.15.40:0/2655993648 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ecc06ae40 0x7f3ecc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:24.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.475+0000 7f3eded9d640 1 --2- 172.21.15.40:0/2655993648 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ecc06ae40 0x7f3ecc06d300 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0000f60 tx=0x7f3ed0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:24.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.478+0000 7f3ec77fe640 1 -- 172.21.15.40:0/2655993648 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ed4053aa0 con 0x7f3ee0151cb0 2023-10-18T09:07:24.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.709+0000 7f3ee50c9640 1 -- 172.21.15.40:0/2655993648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3ee0066fd0 con 0x7f3ee0151cb0 2023-10-18T09:07:24.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.710+0000 7f3ec77fe640 1 -- 172.21.15.40:0/2655993648 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3ed4017070 con 0x7f3ee0151cb0 2023-10-18T09:07:24.714 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T09:07:24.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.713+0000 7f3ee50c9640 1 -- 172.21.15.40:0/2655993648 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ecc06ae40 msgr2=0x7f3ecc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:24.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.713+0000 7f3ee50c9640 1 --2- 172.21.15.40:0/2655993648 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ecc06ae40 0x7f3ecc06d300 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0000f60 tx=0x7f3ed0009040 comp rx=0 tx=0).stop 2023-10-18T09:07:24.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.713+0000 7f3ee50c9640 1 -- 172.21.15.40:0/2655993648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee0151cb0 msgr2=0x7f3ee0143250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:24.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.713+0000 7f3ee50c9640 1 --2- 172.21.15.40:0/2655993648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee0151cb0 0x7f3ee0143250 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400c410 tx=0x7f3ed400c8d0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.713+0000 7f3ee50c9640 1 -- 172.21.15.40:0/2655993648 shutdown_connections 2023-10-18T09:07:24.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.713+0000 7f3ee50c9640 1 --2- 172.21.15.40:0/2655993648 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f3ecc06ae40 0x7f3ecc06d300 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.713+0000 7f3ee50c9640 1 --2- 172.21.15.40:0/2655993648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee0158310 0x7f3ee014bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.713+0000 7f3ee50c9640 1 --2- 172.21.15.40:0/2655993648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ee0151cb0 0x7f3ee0143250 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.713+0000 7f3ee50c9640 1 --2- 172.21.15.40:0/2655993648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee006d710 0x7f3ee01595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:24.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.713+0000 7f3ee50c9640 1 -- 172.21.15.40:0/2655993648 >> 172.21.15.40:0/2655993648 conn(0x7f3ee0069720 msgr2=0x7f3ee014e070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:24.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.714+0000 7f3ee50c9640 1 -- 172.21.15.40:0/2655993648 shutdown_connections 2023-10-18T09:07:24.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:24.714+0000 7f3ee50c9640 1 -- 172.21.15.40:0/2655993648 wait complete. 2023-10-18T09:07:24.735 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T09:07:24.735 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T09:07:24.735 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-10-18T09:07:24.736 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T09:07:24.736 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T09:07:24.736 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T09:07:24.736 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T09:07:24.737 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T09:07:24.737 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:07:24.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:24.738 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-10-18T09:07:24.796 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T09:07:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T09:07:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T09:07:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:07:24.797 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-18T09:07:24.798 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:07:24.798 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:07:24.798 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:07:24.798 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-10-18T09:07:24.798 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:07:24.799 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:07:24.799 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-10-18T09:07:24.799 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T09:07:24.799 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-10-18T09:07:24.799 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-10-18T09:07:24.800 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-18T09:07:24.800 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 41680sec preferred_lft 41680sec 2023-10-18T09:07:24.800 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2023-10-18T09:07:24.800 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:07:24.800 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:07:24.801 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-10-18T09:07:24.801 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-10-18T09:07:24.801 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T09:07:24.801 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:07:24.801 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T09:07:24.801 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:07:24.802 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2026:95ff:fea9:a463/64 scope link 2023-10-18T09:07:24.802 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:07:24.802 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:07:24.802 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T09:07:24.802 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::80bd:e5ff:fe0f:6f68/64 scope link 2023-10-18T09:07:24.803 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:07:24.803 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:07:24.803 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether ea:2b:21:f5:06:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T09:07:24.803 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::e82b:21ff:fef5:6af/64 scope link 2023-10-18T09:07:24.803 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:07:24.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:24.806 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T09:07:24.904 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-10-18T09:07:24.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:24.908 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-10-18T09:07:24.967 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-10-18T09:07:24.968 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T09:07:24.968 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-18T09:07:24.968 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-18T09:07:24.968 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T09:07:24.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:24.969 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T09:07:24.970 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T09:07:24.970 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-18T09:07:24.970 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-18T09:07:24.971 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-18T09:07:24.971 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T09:07:25.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:25.175 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-18T09:07:25.230 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T09:07:25.230 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T09:07:25.231 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:07:25.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:25.234 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-10-18T09:07:25.336 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:07:25.337 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T09:07:25.394 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:07:25.394 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:07:25.395 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T09:07:25.452 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-18T09:07:25.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:25.453 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:07:25.591 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:07:25.592 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:07:25.625 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:07:25.625 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:07:25.625 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T09:07:25.626 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T09:07:25.626 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T09:07:25.626 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-18T09:07:25.627 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2023-10-18T09:07:25.627 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T09:07:25.627 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T09:07:25.627 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T09:07:25.630 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T09:07:25.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:25.631 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T09:07:25.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:25.724 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:07:25.793 INFO:teuthology.orchestra.run.smithi040.stdout:client6233 2023-10-18T09:07:25.794 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:07:25.794 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:07:25.851 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T09:07:26.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.081+0000 7fcf6b59e640 1 -- 172.21.15.40:0/672853311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6c15f4e0 msgr2=0x7fcf6c0b1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:26.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.081+0000 7fcf6b59e640 1 --2- 172.21.15.40:0/672853311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6c15f4e0 0x7fcf6c0b1ec0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c065410 tx=0x7fcf6402c230 comp rx=0 tx=0).stop 2023-10-18T09:07:26.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.081+0000 7fcf6b59e640 1 -- 172.21.15.40:0/672853311 shutdown_connections 2023-10-18T09:07:26.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.081+0000 7fcf6b59e640 1 --2- 172.21.15.40:0/672853311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6c15f4e0 0x7fcf6c0b1ec0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:26.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.081+0000 7fcf6b59e640 1 --2- 172.21.15.40:0/672853311 >> v2:172.21.15.144:3300/0 conn(0x7fcf6c06e050 0x7fcf6c15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:26.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.081+0000 7fcf6b59e640 1 -- 172.21.15.40:0/672853311 >> 172.21.15.40:0/672853311 conn(0x7fcf6c06fd30 msgr2=0x7fcf6c070140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:26.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.081+0000 7fcf6b59e640 1 -- 172.21.15.40:0/672853311 shutdown_connections 2023-10-18T09:07:26.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.081+0000 7fcf6b59e640 1 -- 172.21.15.40:0/672853311 wait complete. 2023-10-18T09:07:26.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.083+0000 7fcf6b59e640 1 Processor -- start 2023-10-18T09:07:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.083+0000 7fcf6b59e640 1 -- start start 2023-10-18T09:07:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.084+0000 7fcf6b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6c06e050 0x7fcf6c148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.084+0000 7fcf6b59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf6c15b390 0x7fcf6c0ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.084+0000 7fcf6b59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf6c15f4e0 0x7fcf6c0c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.084+0000 7fcf6b59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c14c760 con 0x7fcf6c06e050 2023-10-18T09:07:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.084+0000 7fcf6b59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c0cca30 con 0x7fcf6c15b390 2023-10-18T09:07:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.084+0000 7fcf6b59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c0cbe80 con 0x7fcf6c15f4e0 2023-10-18T09:07:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.084+0000 7fcf6a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6c06e050 0x7fcf6c148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.084+0000 7fcf6a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6c06e050 0x7fcf6c148d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57762/0 (socket says 172.21.15.40:57762) 2023-10-18T09:07:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.084+0000 7fcf6a59c640 1 -- 172.21.15.40:0/3196524488 learned_addr learned my addr 172.21.15.40:0/3196524488 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.085+0000 7fcf69d9b640 1 --2- 172.21.15.40:0/3196524488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf6c15b390 0x7fcf6c0ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:26.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.085+0000 7fcf6ad9d640 1 --2- 172.21.15.40:0/3196524488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf6c15f4e0 0x7fcf6c0c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:26.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.085+0000 7fcf6a59c640 1 -- 172.21.15.40:0/3196524488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf6c15f4e0 msgr2=0x7fcf6c0c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:26.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.085+0000 7fcf6a59c640 1 --2- 172.21.15.40:0/3196524488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf6c15f4e0 0x7fcf6c0c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:26.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.085+0000 7fcf6a59c640 1 -- 172.21.15.40:0/3196524488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf6c15b390 msgr2=0x7fcf6c0ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:26.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.085+0000 7fcf6a59c640 1 --2- 172.21.15.40:0/3196524488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf6c15b390 0x7fcf6c0ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:26.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.085+0000 7fcf6a59c640 1 -- 172.21.15.40:0/3196524488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf64009900 con 0x7fcf6c06e050 2023-10-18T09:07:26.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.085+0000 7fcf6ad9d640 1 --2- 172.21.15.40:0/3196524488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf6c15f4e0 0x7fcf6c0c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:26.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.085+0000 7fcf6a59c640 1 --2- 172.21.15.40:0/3196524488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6c06e050 0x7fcf6c148d90 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00c430 tx=0x7fcf5c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:26.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.086+0000 7fcf5affd640 1 -- 172.21.15.40:0/3196524488 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c011790 con 0x7fcf6c06e050 2023-10-18T09:07:26.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.086+0000 7fcf5affd640 1 -- 172.21.15.40:0/3196524488 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcf5c0119e0 con 0x7fcf6c06e050 2023-10-18T09:07:26.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.086+0000 7fcf5affd640 1 -- 172.21.15.40:0/3196524488 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c011e30 con 0x7fcf6c06e050 2023-10-18T09:07:26.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.086+0000 7fcf6b59e640 1 -- 172.21.15.40:0/3196524488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6c0cc100 con 0x7fcf6c06e050 2023-10-18T09:07:26.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.086+0000 7fcf6b59e640 1 -- 172.21.15.40:0/3196524488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6c0cc6c0 con 0x7fcf6c06e050 2023-10-18T09:07:26.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.087+0000 7fcf6b59e640 1 -- 172.21.15.40:0/3196524488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf6c073570 con 0x7fcf6c06e050 2023-10-18T09:07:26.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.088+0000 7fcf5affd640 1 -- 172.21.15.40:0/3196524488 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcf5c01c460 con 0x7fcf6c06e050 2023-10-18T09:07:26.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.089+0000 7fcf5affd640 1 --2- 172.21.15.40:0/3196524488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf4c06ae70 0x7fcf4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:26.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.090+0000 7fcf5affd640 1 -- 172.21.15.40:0/3196524488 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf5c0898f0 con 0x7fcf6c06e050 2023-10-18T09:07:26.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.090+0000 7fcf69d9b640 1 --2- 172.21.15.40:0/3196524488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf4c06ae70 0x7fcf4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:26.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.091+0000 7fcf69d9b640 1 --2- 172.21.15.40:0/3196524488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf4c06ae70 0x7fcf4c06d330 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fcf60005fd0 tx=0x7fcf60005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:26.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.094+0000 7fcf5affd640 1 -- 172.21.15.40:0/3196524488 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcf5c054900 con 0x7fcf6c06e050 2023-10-18T09:07:26.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.327+0000 7fcf6b59e640 1 -- 172.21.15.40:0/3196524488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcf6c070990 con 0x7fcf6c06e050 2023-10-18T09:07:26.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.328+0000 7fcf5affd640 1 -- 172.21.15.40:0/3196524488 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcf5c057f20 con 0x7fcf6c06e050 2023-10-18T09:07:26.331 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T09:07:26.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.330+0000 7fcf6b59e640 1 -- 172.21.15.40:0/3196524488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf4c06ae70 msgr2=0x7fcf4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.330+0000 7fcf6b59e640 1 --2- 172.21.15.40:0/3196524488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf4c06ae70 0x7fcf4c06d330 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fcf60005fd0 tx=0x7fcf60005850 comp rx=0 tx=0).stop 2023-10-18T09:07:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.330+0000 7fcf6b59e640 1 -- 172.21.15.40:0/3196524488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6c06e050 msgr2=0x7fcf6c148d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.330+0000 7fcf6b59e640 1 --2- 172.21.15.40:0/3196524488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6c06e050 0x7fcf6c148d90 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00c430 tx=0x7fcf5c00c8f0 comp rx=0 tx=0).stop 2023-10-18T09:07:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.331+0000 7fcf6b59e640 1 -- 172.21.15.40:0/3196524488 shutdown_connections 2023-10-18T09:07:26.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.331+0000 7fcf6b59e640 1 --2- 172.21.15.40:0/3196524488 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcf4c06ae70 0x7fcf4c06d330 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:26.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.331+0000 7fcf6b59e640 1 --2- 172.21.15.40:0/3196524488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf6c15f4e0 0x7fcf6c0c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:26.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.331+0000 7fcf6b59e640 1 --2- 172.21.15.40:0/3196524488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf6c15b390 0x7fcf6c0ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:26.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.331+0000 7fcf6b59e640 1 --2- 172.21.15.40:0/3196524488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6c06e050 0x7fcf6c148d90 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:26.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.331+0000 7fcf6b59e640 1 -- 172.21.15.40:0/3196524488 >> 172.21.15.40:0/3196524488 conn(0x7fcf6c06fd30 msgr2=0x7fcf6c159a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:26.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.331+0000 7fcf6b59e640 1 -- 172.21.15.40:0/3196524488 shutdown_connections 2023-10-18T09:07:26.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:26.331+0000 7fcf6b59e640 1 -- 172.21.15.40:0/3196524488 wait complete. 2023-10-18T09:07:26.353 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T09:07:26.354 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T09:07:26.354 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-10-18T09:07:26.354 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-18T09:07:26.355 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T09:07:26.355 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T09:07:26.355 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T09:07:26.356 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T09:07:26.356 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:07:26.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:26.357 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-10-18T09:07:26.418 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T09:07:26.419 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T09:07:26.419 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T09:07:26.419 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:07:26.419 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-18T09:07:26.420 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:07:26.420 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:07:26.420 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:07:26.420 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2023-10-18T09:07:26.420 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:07:26.421 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:07:26.421 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2023-10-18T09:07:26.421 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:07:26.421 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-10-18T09:07:26.421 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T09:07:26.422 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-10-18T09:07:26.422 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T09:07:26.422 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 41671sec preferred_lft 41671sec 2023-10-18T09:07:26.422 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2023-10-18T09:07:26.423 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:07:26.423 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T09:07:26.423 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff 2023-10-18T09:07:26.423 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T09:07:26.423 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:07:26.424 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1493:a0ff:fed2:2c1/64 scope link 2023-10-18T09:07:26.424 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:07:26.424 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:07:26.424 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 4a:6e:a9:f0:92:62 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T09:07:26.424 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::486e:a9ff:fef0:9262/64 scope link 2023-10-18T09:07:26.425 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:07:26.425 INFO:teuthology.orchestra.run.smithi144.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:07:26.425 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T09:07:26.425 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::345a:c5ff:fe54:de6e/64 scope link 2023-10-18T09:07:26.425 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:07:26.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:26.427 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T09:07:26.520 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-10-18T09:07:26.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:26.525 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-10-18T09:07:26.579 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-10-18T09:07:26.579 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T09:07:26.580 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T09:07:26.580 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T09:07:26.580 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T09:07:26.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:26.581 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T09:07:26.581 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T09:07:26.582 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T09:07:26.582 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T09:07:26.582 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T09:07:26.583 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T09:07:26.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:26.763 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-18T09:07:26.816 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T09:07:26.816 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-18T09:07:26.817 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:07:26.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:26.818 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2023-10-18T09:07:26.910 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:07:26.911 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-18T09:07:26.964 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:07:26.965 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:07:26.965 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-18T09:07:27.017 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-18T09:07:27.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:27.018 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:07:27.164 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:07:27.164 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:07:27.199 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:07:27.199 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:07:27.199 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T09:07:27.199 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with new device syntax: 1@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T09:07:27.200 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T09:07:27.201 INFO:teuthology.orchestra.run.smithi144.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-18T09:07:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: You just mounted a file system that supports labels which does not 2023-10-18T09:07:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T09:07:27.201 INFO:teuthology.orchestra.run.smithi144.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T09:07:27.202 INFO:teuthology.orchestra.run.smithi144.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T09:07:27.207 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T09:07:27.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:27.208 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-18T09:07:27.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:07:27.287 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:07:27.337 INFO:teuthology.orchestra.run.smithi144.stdout:client9003 2023-10-18T09:07:27.337 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:07:27.338 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:07:27.384 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:07:27.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.625+0000 7f18f7bd0640 1 -- 172.21.15.40:0/1228525430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f006e460 msgr2=0x7f18f00b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:27.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.625+0000 7f18f7bd0640 1 --2- 172.21.15.40:0/1228525430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f006e460 0x7f18f00b2a50 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009830 tx=0x7f18e002bdf0 comp rx=0 tx=0).stop 2023-10-18T09:07:27.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.625+0000 7f18f7bd0640 1 -- 172.21.15.40:0/1228525430 shutdown_connections 2023-10-18T09:07:27.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.625+0000 7f18f7bd0640 1 --2- 172.21.15.40:0/1228525430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f006e460 0x7f18f00b2a50 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:27.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.625+0000 7f18f7bd0640 1 -- 172.21.15.40:0/1228525430 >> 172.21.15.40:0/1228525430 conn(0x7f18f00655f0 msgr2=0x7f18f00659d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:27.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.625+0000 7f18f7bd0640 1 -- 172.21.15.40:0/1228525430 shutdown_connections 2023-10-18T09:07:27.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.625+0000 7f18f7bd0640 1 -- 172.21.15.40:0/1228525430 wait complete. 2023-10-18T09:07:27.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.627+0000 7f18f7bd0640 1 Processor -- start 2023-10-18T09:07:27.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.627+0000 7f18f7bd0640 1 -- start start 2023-10-18T09:07:27.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.628+0000 7f18f7bd0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18f006e460 0x7f18f00a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:27.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.628+0000 7f18f7bd0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18f00ae700 0x7f18f01562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:27.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.628+0000 7f18f7bd0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f0142200 0x7f18f0156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:27.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.628+0000 7f18f7bd0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f00a8c90 con 0x7f18f0142200 2023-10-18T09:07:27.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.628+0000 7f18f7bd0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f00a76b0 con 0x7f18f006e460 2023-10-18T09:07:27.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.628+0000 7f18f7bd0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f00a7990 con 0x7f18f00ae700 2023-10-18T09:07:27.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.629+0000 7f18f616d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f0142200 0x7f18f0156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:27.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.629+0000 7f18f616d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f0142200 0x7f18f0156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57770/0 (socket says 172.21.15.40:57770) 2023-10-18T09:07:27.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.629+0000 7f18f616d640 1 -- 172.21.15.40:0/151836736 learned_addr learned my addr 172.21.15.40:0/151836736 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:27.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.629+0000 7f18f516b640 1 --2- 172.21.15.40:0/151836736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18f00ae700 0x7f18f01562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:27.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.629+0000 7f18f616d640 1 -- 172.21.15.40:0/151836736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18f00ae700 msgr2=0x7f18f01562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:27.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.629+0000 7f18f616d640 1 --2- 172.21.15.40:0/151836736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18f00ae700 0x7f18f01562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:27.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.629+0000 7f18f616d640 1 -- 172.21.15.40:0/151836736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18f006e460 msgr2=0x7f18f00a6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:27.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.629+0000 7f18f616d640 1 --2- 172.21.15.40:0/151836736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18f006e460 0x7f18f00a6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:27.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.629+0000 7f18f616d640 1 -- 172.21.15.40:0/151836736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e0009490 con 0x7f18f0142200 2023-10-18T09:07:27.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.629+0000 7f18f516b640 1 --2- 172.21.15.40:0/151836736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18f00ae700 0x7f18f01562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:27.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.629+0000 7f18f616d640 1 --2- 172.21.15.40:0/151836736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f0142200 0x7f18f0156800 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f18e800c760 tx=0x7f18e800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:27.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.630+0000 7f18e67fc640 1 -- 172.21.15.40:0/151836736 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e8003da0 con 0x7f18f0142200 2023-10-18T09:07:27.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.630+0000 7f18e67fc640 1 -- 172.21.15.40:0/151836736 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f18e800ce70 con 0x7f18f0142200 2023-10-18T09:07:27.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.630+0000 7f18e67fc640 1 -- 172.21.15.40:0/151836736 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e80052b0 con 0x7f18f0142200 2023-10-18T09:07:27.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.630+0000 7f18f7bd0640 1 -- 172.21.15.40:0/151836736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f00a7c70 con 0x7f18f0142200 2023-10-18T09:07:27.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.630+0000 7f18f7bd0640 1 -- 172.21.15.40:0/151836736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f00a73c0 con 0x7f18f0142200 2023-10-18T09:07:27.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.631+0000 7f18f7bd0640 1 -- 172.21.15.40:0/151836736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f0066fd0 con 0x7f18f0142200 2023-10-18T09:07:27.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.632+0000 7f18e67fc640 1 -- 172.21.15.40:0/151836736 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f18e8005410 con 0x7f18f0142200 2023-10-18T09:07:27.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.633+0000 7f18e67fc640 1 --2- 172.21.15.40:0/151836736 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18d406ae40 0x7f18d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:27.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.634+0000 7f18e67fc640 1 -- 172.21.15.40:0/151836736 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18e8089740 con 0x7f18f0142200 2023-10-18T09:07:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.634+0000 7f18f596c640 1 --2- 172.21.15.40:0/151836736 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18d406ae40 0x7f18d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:27.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.635+0000 7f18f596c640 1 --2- 172.21.15.40:0/151836736 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18d406ae40 0x7f18d406d300 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f18ec000f60 tx=0x7f18ec0079b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:27.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.639+0000 7f18e67fc640 1 -- 172.21.15.40:0/151836736 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18e8054750 con 0x7f18f0142200 2023-10-18T09:07:27.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.873+0000 7f18f7bd0640 1 -- 172.21.15.40:0/151836736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18f0159640 con 0x7f18f0142200 2023-10-18T09:07:27.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.874+0000 7f18e67fc640 1 -- 172.21.15.40:0/151836736 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f18e8057d70 con 0x7f18f0142200 2023-10-18T09:07:27.878 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:07:27.878 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6823,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3592705338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3592705338},{"type":"v1","addr":"172.21.15.40:6835","nonce":3592705338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":6829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1076939595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1076939595},{"type":"v1","addr":"172.21.15.144:6836","nonce":1076939595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":6832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2483821333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2483821333},{"type":"v1","addr":"172.21.15.144:6837","nonce":2483821333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:07:16.832438+0000","modified":"2023-10-18T09:07:19.910479+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8826},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8826":{"gid":8826,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/137474609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":137474609},{"type":"v1","addr":"172.21.15.40:6834","nonce":137474609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-18T09:07:27.879 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 82 2023-10-18T09:07:27.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.877+0000 7f18f7bd0640 1 -- 172.21.15.40:0/151836736 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18d406ae40 msgr2=0x7f18d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:27.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.877+0000 7f18f7bd0640 1 --2- 172.21.15.40:0/151836736 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18d406ae40 0x7f18d406d300 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f18ec000f60 tx=0x7f18ec0079b0 comp rx=0 tx=0).stop 2023-10-18T09:07:27.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.877+0000 7f18f7bd0640 1 -- 172.21.15.40:0/151836736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f0142200 msgr2=0x7f18f0156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:27.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.877+0000 7f18f7bd0640 1 --2- 172.21.15.40:0/151836736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f0142200 0x7f18f0156800 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f18e800c760 tx=0x7f18e800cc20 comp rx=0 tx=0).stop 2023-10-18T09:07:27.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.877+0000 7f18f7bd0640 1 -- 172.21.15.40:0/151836736 shutdown_connections 2023-10-18T09:07:27.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.877+0000 7f18f7bd0640 1 --2- 172.21.15.40:0/151836736 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f18d406ae40 0x7f18d406d300 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:27.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.877+0000 7f18f7bd0640 1 --2- 172.21.15.40:0/151836736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f0142200 0x7f18f0156800 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:27.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.877+0000 7f18f7bd0640 1 --2- 172.21.15.40:0/151836736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18f00ae700 0x7f18f01562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:27.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.877+0000 7f18f7bd0640 1 --2- 172.21.15.40:0/151836736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18f006e460 0x7f18f00a6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:27.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.877+0000 7f18f7bd0640 1 -- 172.21.15.40:0/151836736 >> 172.21.15.40:0/151836736 conn(0x7f18f00655f0 msgr2=0x7f18f0158bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:27.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.878+0000 7f18f7bd0640 1 -- 172.21.15.40:0/151836736 shutdown_connections 2023-10-18T09:07:27.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:27.878+0000 7f18f7bd0640 1 -- 172.21.15.40:0/151836736 wait complete. 2023-10-18T09:07:27.899 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-18T09:07:28.132 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":8826,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.751067837000001,"mdsmap_epoch":82,"osdmap_epoch":179,"osdmap_epoch_barrier":179,"uptime":17.638605033000001} 2023-10-18T09:07:28.133 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 8826, "mdsmap_epoch": 82, "osdmap_epoch": 179, "osdmap_epoch_barrier": 179, "rank_uptime": 10.751067837, "state": "up:active", "uptime": 17.638605033, "want_state": "up:active", "whoami": 0 } 2023-10-18T09:07:28.133 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-18T09:07:28.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.363+0000 7f9abe46b640 1 -- 172.21.15.40:0/2763961767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab806d680 msgr2=0x7f9ab80c80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:28.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.363+0000 7f9abe46b640 1 --2- 172.21.15.40:0/2763961767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab806d680 0x7f9ab80c80f0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8005fd0 tx=0x7f9aa802d0d0 comp rx=0 tx=0).stop 2023-10-18T09:07:28.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.363+0000 7f9abe46b640 1 -- 172.21.15.40:0/2763961767 shutdown_connections 2023-10-18T09:07:28.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.363+0000 7f9abe46b640 1 --2- 172.21.15.40:0/2763961767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80c9850 0x7f9ab80c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:28.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.363+0000 7f9abe46b640 1 --2- 172.21.15.40:0/2763961767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab806d680 0x7f9ab80c80f0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:28.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.363+0000 7f9abe46b640 1 -- 172.21.15.40:0/2763961767 >> 172.21.15.40:0/2763961767 conn(0x7f9ab80aec00 msgr2=0x7f9ab80aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:28.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.363+0000 7f9abe46b640 1 -- 172.21.15.40:0/2763961767 shutdown_connections 2023-10-18T09:07:28.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.363+0000 7f9abe46b640 1 -- 172.21.15.40:0/2763961767 wait complete. 2023-10-18T09:07:28.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.365+0000 7f9abe46b640 1 Processor -- start 2023-10-18T09:07:28.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.365+0000 7f9abe46b640 1 -- start start 2023-10-18T09:07:28.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.366+0000 7f9abe46b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab806b7d0 0x7f9ab80addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:28.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.366+0000 7f9abe46b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab806d680 0x7f9ab80b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:28.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.366+0000 7f9abe46b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80c9850 0x7f9ab814f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:28.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.366+0000 7f9abe46b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab8151490 con 0x7f9ab806d680 2023-10-18T09:07:28.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.366+0000 7f9abe46b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80b2150 con 0x7f9ab806b7d0 2023-10-18T09:07:28.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.366+0000 7f9abe46b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80b2430 con 0x7f9ab80c9850 2023-10-18T09:07:28.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.366+0000 7f9aaffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab806d680 0x7f9ab80b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:28.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.366+0000 7f9aaffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab806d680 0x7f9ab80b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57782/0 (socket says 172.21.15.40:57782) 2023-10-18T09:07:28.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.366+0000 7f9aaffff640 1 -- 172.21.15.40:0/2051182807 learned_addr learned my addr 172.21.15.40:0/2051182807 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:07:28.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.366+0000 7f9ab7fff640 1 --2- 172.21.15.40:0/2051182807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab806b7d0 0x7f9ab80addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:28.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.366+0000 7f9aaffff640 1 -- 172.21.15.40:0/2051182807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80c9850 msgr2=0x7f9ab814f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.366+0000 7f9abca08640 1 --2- 172.21.15.40:0/2051182807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80c9850 0x7f9ab814f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.367+0000 7f9aaffff640 1 --2- 172.21.15.40:0/2051182807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80c9850 0x7f9ab814f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.367+0000 7f9aaffff640 1 -- 172.21.15.40:0/2051182807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab806b7d0 msgr2=0x7f9ab80addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.367+0000 7f9aaffff640 1 --2- 172.21.15.40:0/2051182807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab806b7d0 0x7f9ab80addf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.367+0000 7f9aaffff640 1 -- 172.21.15.40:0/2051182807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa800a670 con 0x7f9ab806d680 2023-10-18T09:07:28.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.367+0000 7f9ab7fff640 1 --2- 172.21.15.40:0/2051182807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab806b7d0 0x7f9ab80addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:28.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.367+0000 7f9abca08640 1 --2- 172.21.15.40:0/2051182807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80c9850 0x7f9ab814f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:07:28.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.367+0000 7f9aaffff640 1 --2- 172.21.15.40:0/2051182807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab806d680 0x7f9ab80b1790 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000d6d0 tx=0x7f9aa000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:28.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.367+0000 7f9ab57fa640 1 -- 172.21.15.40:0/2051182807 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0005220 con 0x7f9ab806d680 2023-10-18T09:07:28.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.367+0000 7f9ab57fa640 1 -- 172.21.15.40:0/2051182807 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9aa0005490 con 0x7f9ab806d680 2023-10-18T09:07:28.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.367+0000 7f9ab57fa640 1 -- 172.21.15.40:0/2051182807 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0009a80 con 0x7f9ab806d680 2023-10-18T09:07:28.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.367+0000 7f9abe46b640 1 -- 172.21.15.40:0/2051182807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab80b2710 con 0x7f9ab806d680 2023-10-18T09:07:28.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.367+0000 7f9abe46b640 1 -- 172.21.15.40:0/2051182807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab80ae330 con 0x7f9ab806d680 2023-10-18T09:07:28.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.369+0000 7f9abe46b640 1 -- 172.21.15.40:0/2051182807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab8066fd0 con 0x7f9ab806d680 2023-10-18T09:07:28.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.370+0000 7f9ab57fa640 1 -- 172.21.15.40:0/2051182807 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9aa00047a0 con 0x7f9ab806d680 2023-10-18T09:07:28.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.370+0000 7f9ab57fa640 1 --2- 172.21.15.40:0/2051182807 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a9806ae40 0x7f9a9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:07:28.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.371+0000 7f9ab57fa640 1 -- 172.21.15.40:0/2051182807 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9aa0089a20 con 0x7f9ab806d680 2023-10-18T09:07:28.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.371+0000 7f9ab7fff640 1 --2- 172.21.15.40:0/2051182807 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a9806ae40 0x7f9a9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:07:28.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.372+0000 7f9ab7fff640 1 --2- 172.21.15.40:0/2051182807 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a9806ae40 0x7f9a9806d300 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8036000 tx=0x7f9aa8004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:07:28.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.375+0000 7f9ab57fa640 1 -- 172.21.15.40:0/2051182807 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9aa0054ac0 con 0x7f9ab806d680 2023-10-18T09:07:28.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:28.568+0000 7f9abe46b640 1 -- 172.21.15.40:0/2051182807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f9ab80cc8d0 con 0x7f9ab806d680 2023-10-18T09:07:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:29.199+0000 7f9ab57fa640 1 -- 172.21.15.40:0/2051182807 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v180) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9aa0017020 con 0x7f9ab806d680 2023-10-18T09:07:29.203 INFO:teuthology.orchestra.run.smithi040.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-18T09:07:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:29.202+0000 7f9abe46b640 1 -- 172.21.15.40:0/2051182807 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a9806ae40 msgr2=0x7f9a9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:29.202+0000 7f9abe46b640 1 --2- 172.21.15.40:0/2051182807 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a9806ae40 0x7f9a9806d300 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8036000 tx=0x7f9aa8004730 comp rx=0 tx=0).stop 2023-10-18T09:07:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:29.202+0000 7f9abe46b640 1 -- 172.21.15.40:0/2051182807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab806d680 msgr2=0x7f9ab80b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:07:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:29.202+0000 7f9abe46b640 1 --2- 172.21.15.40:0/2051182807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab806d680 0x7f9ab80b1790 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000d6d0 tx=0x7f9aa000db90 comp rx=0 tx=0).stop 2023-10-18T09:07:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:29.202+0000 7f9abe46b640 1 -- 172.21.15.40:0/2051182807 shutdown_connections 2023-10-18T09:07:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:29.202+0000 7f9abe46b640 1 --2- 172.21.15.40:0/2051182807 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a9806ae40 0x7f9a9806d300 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:29.202+0000 7f9abe46b640 1 --2- 172.21.15.40:0/2051182807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ab80c9850 0x7f9ab814f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:29.202+0000 7f9abe46b640 1 --2- 172.21.15.40:0/2051182807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ab806d680 0x7f9ab80b1790 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:29.202+0000 7f9abe46b640 1 --2- 172.21.15.40:0/2051182807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ab806b7d0 0x7f9ab80addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:07:29.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:29.202+0000 7f9abe46b640 1 -- 172.21.15.40:0/2051182807 >> 172.21.15.40:0/2051182807 conn(0x7f9ab80aec00 msgr2=0x7f9ab8157cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:07:29.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:29.203+0000 7f9abe46b640 1 -- 172.21.15.40:0/2051182807 shutdown_connections 2023-10-18T09:07:29.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:07:29.203+0000 7f9abe46b640 1 -- 172.21.15.40:0/2051182807 wait complete. 2023-10-18T09:07:29.226 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-18T09:07:29.470 INFO:teuthology.orchestra.run.smithi040.stdout:{"osd_mon_report_interval":"5"} 2023-10-18T09:07:29.471 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-18T09:07:29.472 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-18T09:07:29.709 INFO:teuthology.orchestra.run.smithi040.stdout:{"mon_tick_interval":"5"} 2023-10-18T09:07:29.710 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-18T09:07:29.710 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2023-10-18T09:07:29.711 DEBUG:teuthology.orchestra.run.smithi040:> stdin-killer -- python3 -c ' 2023-10-18T09:07:29.711 DEBUG:teuthology.orchestra.run.smithi040:> import time 2023-10-18T09:07:29.711 DEBUG:teuthology.orchestra.run.smithi040:> import datetime 2023-10-18T09:07:29.712 DEBUG:teuthology.orchestra.run.smithi040:> import subprocess 2023-10-18T09:07:29.712 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-10-18T09:07:29.712 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:07:29.713 DEBUG:teuthology.orchestra.run.smithi040:> # Write some buffered data through before going full, all should be well 2023-10-18T09:07:29.713 DEBUG:teuthology.orchestra.run.smithi040:> print("writing some data through which we expect to succeed") 2023-10-18T09:07:29.713 DEBUG:teuthology.orchestra.run.smithi040:> bytes = 0 2023-10-18T09:07:29.714 DEBUG:teuthology.orchestra.run.smithi040:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-18T09:07:29.714 DEBUG:teuthology.orchestra.run.smithi040:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-18T09:07:29.715 DEBUG:teuthology.orchestra.run.smithi040:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2023-10-18T09:07:29.715 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(f) 2023-10-18T09:07:29.715 DEBUG:teuthology.orchestra.run.smithi040:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-18T09:07:29.716 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:07:29.716 DEBUG:teuthology.orchestra.run.smithi040:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-18T09:07:29.716 DEBUG:teuthology.orchestra.run.smithi040:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-18T09:07:29.717 DEBUG:teuthology.orchestra.run.smithi040:> # buffered IO, the error should always only appear from fsync and not 2023-10-18T09:07:29.717 DEBUG:teuthology.orchestra.run.smithi040:> # from write 2023-10-18T09:07:29.717 DEBUG:teuthology.orchestra.run.smithi040:> full = False 2023-10-18T09:07:29.718 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:07:29.718 DEBUG:teuthology.orchestra.run.smithi040:> for n in range(0, int(32 * 1.1)): 2023-10-18T09:07:29.718 DEBUG:teuthology.orchestra.run.smithi040:> try: 2023-10-18T09:07:29.719 DEBUG:teuthology.orchestra.run.smithi040:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-18T09:07:29.719 DEBUG:teuthology.orchestra.run.smithi040:> print("wrote bytes via buffered write, moving on to fsync") 2023-10-18T09:07:29.719 DEBUG:teuthology.orchestra.run.smithi040:> except OSError as e: 2023-10-18T09:07:29.720 DEBUG:teuthology.orchestra.run.smithi040:> if False: 2023-10-18T09:07:29.720 DEBUG:teuthology.orchestra.run.smithi040:> print("Unexpected error %s from write() instead of fsync()" % e) 2023-10-18T09:07:29.720 DEBUG:teuthology.orchestra.run.smithi040:> raise 2023-10-18T09:07:29.721 DEBUG:teuthology.orchestra.run.smithi040:> else: 2023-10-18T09:07:29.721 DEBUG:teuthology.orchestra.run.smithi040:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-18T09:07:29.721 DEBUG:teuthology.orchestra.run.smithi040:> full = True 2023-10-18T09:07:29.722 DEBUG:teuthology.orchestra.run.smithi040:> break 2023-10-18T09:07:29.722 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:07:29.722 DEBUG:teuthology.orchestra.run.smithi040:> try: 2023-10-18T09:07:29.723 DEBUG:teuthology.orchestra.run.smithi040:> os.fsync(f) 2023-10-18T09:07:29.723 DEBUG:teuthology.orchestra.run.smithi040:> print("fsync'"'"'ed successfully") 2023-10-18T09:07:29.723 DEBUG:teuthology.orchestra.run.smithi040:> except OSError as e: 2023-10-18T09:07:29.724 DEBUG:teuthology.orchestra.run.smithi040:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-18T09:07:29.724 DEBUG:teuthology.orchestra.run.smithi040:> full = True 2023-10-18T09:07:29.724 DEBUG:teuthology.orchestra.run.smithi040:> break 2023-10-18T09:07:29.725 DEBUG:teuthology.orchestra.run.smithi040:> else: 2023-10-18T09:07:29.725 DEBUG:teuthology.orchestra.run.smithi040:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-18T09:07:29.725 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:07:29.726 DEBUG:teuthology.orchestra.run.smithi040:> if n > 32 * 0.9: 2023-10-18T09:07:29.726 DEBUG:teuthology.orchestra.run.smithi040:> # Be cautious in the last region where we expect to hit 2023-10-18T09:07:29.726 DEBUG:teuthology.orchestra.run.smithi040:> # the full condition, so that we don'"'"'t overshoot too dramatically 2023-10-18T09:07:29.727 DEBUG:teuthology.orchestra.run.smithi040:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-18T09:07:29.727 DEBUG:teuthology.orchestra.run.smithi040:> time.sleep(15.0) 2023-10-18T09:07:29.728 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:07:29.728 DEBUG:teuthology.orchestra.run.smithi040:> if not full: 2023-10-18T09:07:29.728 DEBUG:teuthology.orchestra.run.smithi040:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2023-10-18T09:07:29.729 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:07:29.729 DEBUG:teuthology.orchestra.run.smithi040:> # close() should not raise an error because we already caught it in 2023-10-18T09:07:29.729 DEBUG:teuthology.orchestra.run.smithi040:> # fsync. There shouldn'"'"'t have been any more writeback errors 2023-10-18T09:07:29.730 DEBUG:teuthology.orchestra.run.smithi040:> # since then because all IOs got cancelled on the full flag. 2023-10-18T09:07:29.730 DEBUG:teuthology.orchestra.run.smithi040:> print("calling close") 2023-10-18T09:07:29.730 DEBUG:teuthology.orchestra.run.smithi040:> os.close(f) 2023-10-18T09:07:29.731 DEBUG:teuthology.orchestra.run.smithi040:> print("close() did not raise error") 2023-10-18T09:07:29.731 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-18T09:07:29.733 DEBUG:teuthology.orchestra.run.smithi040:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-18T09:07:29.733 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-10-18T09:07:37.830 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:07:37.840 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:08:07.988 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:08:07.998 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:08:31.044 INFO:teuthology.orchestra.run.smithi040.stdout:writing some data through which we expect to succeed 2023-10-18T09:08:31.045 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-18T09:08:31.045 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.045 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.045 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 1.00 MB 2023-10-18T09:08:31.046 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.046 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.046 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 2.00 MB 2023-10-18T09:08:31.046 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.046 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.047 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 3.00 MB 2023-10-18T09:08:31.047 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.047 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.047 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 4.00 MB 2023-10-18T09:08:31.047 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.048 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.048 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 5.00 MB 2023-10-18T09:08:31.048 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.048 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.048 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 6.00 MB 2023-10-18T09:08:31.049 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.049 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.049 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 7.00 MB 2023-10-18T09:08:31.049 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.050 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.050 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 8.00 MB 2023-10-18T09:08:31.050 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.050 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.050 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 9.00 MB 2023-10-18T09:08:31.051 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.051 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.051 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 10.00 MB 2023-10-18T09:08:31.051 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.051 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.052 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 11.00 MB 2023-10-18T09:08:31.052 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.052 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.052 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 12.00 MB 2023-10-18T09:08:31.052 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.053 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.053 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 13.00 MB 2023-10-18T09:08:31.053 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.053 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.053 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 14.00 MB 2023-10-18T09:08:31.054 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.054 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.054 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 15.00 MB 2023-10-18T09:08:31.054 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.054 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.055 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 16.00 MB 2023-10-18T09:08:31.055 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.055 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.055 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 17.00 MB 2023-10-18T09:08:31.055 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.056 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.056 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 18.00 MB 2023-10-18T09:08:31.056 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.056 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.056 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 19.00 MB 2023-10-18T09:08:31.057 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.057 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.057 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 20.00 MB 2023-10-18T09:08:31.057 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.057 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.058 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 21.00 MB 2023-10-18T09:08:31.058 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.058 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.058 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 22.00 MB 2023-10-18T09:08:31.058 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.059 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.059 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 23.00 MB 2023-10-18T09:08:31.059 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.059 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.059 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 24.00 MB 2023-10-18T09:08:31.059 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.060 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.060 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 25.00 MB 2023-10-18T09:08:31.060 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.060 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.060 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 26.00 MB 2023-10-18T09:08:31.061 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.061 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.061 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 27.00 MB 2023-10-18T09:08:31.061 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.061 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.062 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 28.00 MB 2023-10-18T09:08:31.062 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.062 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.062 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 29.00 MB 2023-10-18T09:08:31.062 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.063 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.063 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 30.00 MB 2023-10-18T09:08:31.063 INFO:teuthology.orchestra.run.smithi040.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-18T09:08:31.063 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.063 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.063 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 31.00 MB 2023-10-18T09:08:31.064 INFO:teuthology.orchestra.run.smithi040.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-18T09:08:31.064 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.064 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.064 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 32.00 MB 2023-10-18T09:08:31.064 INFO:teuthology.orchestra.run.smithi040.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-18T09:08:31.064 INFO:teuthology.orchestra.run.smithi040.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-18T09:08:31.065 INFO:teuthology.orchestra.run.smithi040.stdout:fsync'ed successfully 2023-10-18T09:08:31.065 INFO:teuthology.orchestra.run.smithi040.stdout:Not full yet after 33.00 MB 2023-10-18T09:08:31.065 INFO:teuthology.orchestra.run.smithi040.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-18T09:08:31.065 INFO:teuthology.orchestra.run.smithi040.stdout:Reached fullness after 33.00 MB 2023-10-18T09:08:31.065 INFO:teuthology.orchestra.run.smithi040.stdout:calling close 2023-10-18T09:08:31.066 INFO:teuthology.orchestra.run.smithi040.stdout:close() did not raise error 2023-10-18T09:08:31.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T09:08:31.079 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T09:08:31.079 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:08:31.212 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:08:31.348 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T09:08:31.349 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:08:31.349 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:08:31.412 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:08:31.413 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:08:31.471 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:08:31.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.705+0000 7f5d51d28640 1 -- 172.21.15.40:0/1506856215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d3c0047c0 msgr2=0x7f5d3c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:31.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.705+0000 7f5d51d28640 1 --2- 172.21.15.40:0/1506856215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d3c0047c0 0x7f5d3c004c40 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800abb0 tx=0x7f5d3c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:08:31.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.705+0000 7f5d51527640 1 -- 172.21.15.40:0/1506856215 reap_dead start 2023-10-18T09:08:31.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.706+0000 7f5d51d28640 1 -- 172.21.15.40:0/1506856215 shutdown_connections 2023-10-18T09:08:31.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.706+0000 7f5d51d28640 1 -- 172.21.15.40:0/1506856215 >> 172.21.15.40:0/1506856215 conn(0x7f5d4c069720 msgr2=0x7f5d4c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:31.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.706+0000 7f5d51d28640 1 -- 172.21.15.40:0/1506856215 shutdown_connections 2023-10-18T09:08:31.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.706+0000 7f5d51d28640 1 -- 172.21.15.40:0/1506856215 wait complete. 2023-10-18T09:08:31.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.707+0000 7f5d51d28640 1 Processor -- start 2023-10-18T09:08:31.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.708+0000 7f5d51d28640 1 -- start start 2023-10-18T09:08:31.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.709+0000 7f5d51d28640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d3c0047c0 0x7f5d4c1e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:31.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.709+0000 7f5d51d28640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d4c1f5a40 0x7f5d4c0aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:31.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.709+0000 7f5d51d28640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d4c0af240 0x7f5d4c1f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:31.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.709+0000 7f5d51d28640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c1f1680 con 0x7f5d3c0047c0 2023-10-18T09:08:31.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.709+0000 7f5d51d28640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c0adc60 con 0x7f5d4c0af240 2023-10-18T09:08:31.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.709+0000 7f5d51d28640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c0adf40 con 0x7f5d4c1f5a40 2023-10-18T09:08:31.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.709+0000 7f5d50d26640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d3c0047c0 0x7f5d4c1e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:31.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.709+0000 7f5d50d26640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d3c0047c0 0x7f5d4c1e65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60116/0 (socket says 172.21.15.40:60116) 2023-10-18T09:08:31.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.709+0000 7f5d50d26640 1 -- 172.21.15.40:0/2109834179 learned_addr learned my addr 172.21.15.40:0/2109834179 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:31.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.709+0000 7f5d4bfff640 1 --2- 172.21.15.40:0/2109834179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d4c1f5a40 0x7f5d4c0aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:31.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.709+0000 7f5d51527640 1 --2- 172.21.15.40:0/2109834179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d4c0af240 0x7f5d4c1f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:31.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.710+0000 7f5d50d26640 1 -- 172.21.15.40:0/2109834179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d4c1f5a40 msgr2=0x7f5d4c0aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:31.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.710+0000 7f5d50d26640 1 --2- 172.21.15.40:0/2109834179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d4c1f5a40 0x7f5d4c0aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:31.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.710+0000 7f5d50d26640 1 -- 172.21.15.40:0/2109834179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d4c0af240 msgr2=0x7f5d4c1f4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:31.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.710+0000 7f5d50d26640 1 --2- 172.21.15.40:0/2109834179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d4c0af240 0x7f5d4c1f4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:31.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.710+0000 7f5d50d26640 1 -- 172.21.15.40:0/2109834179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c0ae220 con 0x7f5d3c0047c0 2023-10-18T09:08:31.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.710+0000 7f5d50d26640 1 --2- 172.21.15.40:0/2109834179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d3c0047c0 0x7f5d4c1e65c0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000d920 tx=0x7f5d4000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:31.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.710+0000 7f5d497fa640 1 -- 172.21.15.40:0/2109834179 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d400062e0 con 0x7f5d3c0047c0 2023-10-18T09:08:31.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.710+0000 7f5d497fa640 1 -- 172.21.15.40:0/2109834179 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5d40006530 con 0x7f5d3c0047c0 2023-10-18T09:08:31.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.710+0000 7f5d497fa640 1 -- 172.21.15.40:0/2109834179 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4000ba90 con 0x7f5d3c0047c0 2023-10-18T09:08:31.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.710+0000 7f5d51d28640 1 -- 172.21.15.40:0/2109834179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4c0ae500 con 0x7f5d3c0047c0 2023-10-18T09:08:31.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.710+0000 7f5d51d28640 1 -- 172.21.15.40:0/2109834179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4c1e6b00 con 0x7f5d3c0047c0 2023-10-18T09:08:31.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.712+0000 7f5d51d28640 1 -- 172.21.15.40:0/2109834179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4c066e00 con 0x7f5d3c0047c0 2023-10-18T09:08:31.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.713+0000 7f5d497fa640 1 -- 172.21.15.40:0/2109834179 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5d4000bbf0 con 0x7f5d3c0047c0 2023-10-18T09:08:31.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.714+0000 7f5d497fa640 1 --2- 172.21.15.40:0/2109834179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d3006ae40 0x7f5d3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:31.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.714+0000 7f5d497fa640 1 -- 172.21.15.40:0/2109834179 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5d40089e00 con 0x7f5d3c0047c0 2023-10-18T09:08:31.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.715+0000 7f5d497fa640 1 -- 172.21.15.40:0/2109834179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=184}) v3 -- 0x7f5d300729d0 con 0x7f5d3c0047c0 2023-10-18T09:08:31.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.715+0000 7f5d4bfff640 1 --2- 172.21.15.40:0/2109834179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d3006ae40 0x7f5d3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:31.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.716+0000 7f5d4bfff640 1 --2- 172.21.15.40:0/2109834179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d3006ae40 0x7f5d3006d300 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5d38036000 tx=0x7f5d38035040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:31.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.719+0000 7f5d497fa640 1 -- 172.21.15.40:0/2109834179 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d40014070 con 0x7f5d3c0047c0 2023-10-18T09:08:31.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.914+0000 7f5d51d28640 1 -- 172.21.15.40:0/2109834179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d4c0caf90 con 0x7f5d3c0047c0 2023-10-18T09:08:31.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.918+0000 7f5d497fa640 1 -- 172.21.15.40:0/2109834179 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+16667 (secure 0 0 0) 0x7f5d40054ec0 con 0x7f5d3c0047c0 2023-10-18T09:08:31.920 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:08:31.921 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":183,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:08:06.185791+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:07:14.660142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-18T09:07:15.853973+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.2","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:08:31.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.920+0000 7f5d51d28640 1 -- 172.21.15.40:0/2109834179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d3006ae40 msgr2=0x7f5d3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:31.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.920+0000 7f5d51d28640 1 --2- 172.21.15.40:0/2109834179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d3006ae40 0x7f5d3006d300 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5d38036000 tx=0x7f5d38035040 comp rx=0 tx=0).stop 2023-10-18T09:08:31.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.920+0000 7f5d51d28640 1 -- 172.21.15.40:0/2109834179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d3c0047c0 msgr2=0x7f5d4c1e65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:31.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.920+0000 7f5d51d28640 1 --2- 172.21.15.40:0/2109834179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d3c0047c0 0x7f5d4c1e65c0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000d920 tx=0x7f5d4000dde0 comp rx=0 tx=0).stop 2023-10-18T09:08:31.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.920+0000 7f5d51d28640 1 -- 172.21.15.40:0/2109834179 shutdown_connections 2023-10-18T09:08:31.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.920+0000 7f5d51d28640 1 --2- 172.21.15.40:0/2109834179 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d3006ae40 0x7f5d3006d300 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:31.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.920+0000 7f5d51d28640 1 --2- 172.21.15.40:0/2109834179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d4c0af240 0x7f5d4c1f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:31.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.920+0000 7f5d51d28640 1 --2- 172.21.15.40:0/2109834179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d4c1f5a40 0x7f5d4c0aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:31.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.920+0000 7f5d51d28640 1 --2- 172.21.15.40:0/2109834179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d3c0047c0 0x7f5d4c1e65c0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:31.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.920+0000 7f5d51d28640 1 -- 172.21.15.40:0/2109834179 >> 172.21.15.40:0/2109834179 conn(0x7f5d4c069720 msgr2=0x7f5d4c0abba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:31.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.921+0000 7f5d51d28640 1 -- 172.21.15.40:0/2109834179 shutdown_connections 2023-10-18T09:08:31.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:31.921+0000 7f5d51d28640 1 -- 172.21.15.40:0/2109834179 wait complete. 2023-10-18T09:08:31.943 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:07:14.660142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '178', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:07:15.853973+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:08:31.944 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:08:32.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.182+0000 7f649672c640 1 -- 172.21.15.40:0/2075396625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6480004a80 msgr2=0x7f6480004e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:32.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.182+0000 7f649672c640 1 --2- 172.21.15.40:0/2075396625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6480004a80 0x7f6480004e80 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f648400c6c0 tx=0x7f6480006900 comp rx=0 tx=0).stop 2023-10-18T09:08:32.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.182+0000 7f649672c640 1 -- 172.21.15.40:0/2075396625 shutdown_connections 2023-10-18T09:08:32.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.182+0000 7f649672c640 1 --2- 172.21.15.40:0/2075396625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6480004a80 0x7f6480004e80 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:32.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.182+0000 7f649672c640 1 -- 172.21.15.40:0/2075396625 >> 172.21.15.40:0/2075396625 conn(0x7f6490069720 msgr2=0x7f6490069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:32.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.182+0000 7f649672c640 1 -- 172.21.15.40:0/2075396625 shutdown_connections 2023-10-18T09:08:32.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.182+0000 7f649672c640 1 -- 172.21.15.40:0/2075396625 wait complete. 2023-10-18T09:08:32.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.184+0000 7f649672c640 1 Processor -- start 2023-10-18T09:08:32.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.184+0000 7f649672c640 1 -- start start 2023-10-18T09:08:32.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.185+0000 7f649672c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6480004a80 0x7f6490154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:32.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.185+0000 7f649672c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64900af3f0 0x7f6490153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.185+0000 7f649672c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f649013f1d0 0x7f649013f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.185+0000 7f649672c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649013fbc0 con 0x7f64900af3f0 2023-10-18T09:08:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.185+0000 7f649672c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6490155030 con 0x7f649013f1d0 2023-10-18T09:08:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.185+0000 7f649672c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6490155310 con 0x7f6480004a80 2023-10-18T09:08:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.185+0000 7f6494f29640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64900af3f0 0x7f6490153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.185+0000 7f6494f29640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64900af3f0 0x7f6490153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60148/0 (socket says 172.21.15.40:60148) 2023-10-18T09:08:32.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.185+0000 7f6494f29640 1 -- 172.21.15.40:0/4245679147 learned_addr learned my addr 172.21.15.40:0/4245679147 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:32.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.186+0000 7f6495f2b640 1 --2- 172.21.15.40:0/4245679147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f649013f1d0 0x7f649013f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.186+0000 7f649572a640 1 --2- 172.21.15.40:0/4245679147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6480004a80 0x7f6490154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.186+0000 7f6494f29640 1 -- 172.21.15.40:0/4245679147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6480004a80 msgr2=0x7f6490154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.186+0000 7f6494f29640 1 --2- 172.21.15.40:0/4245679147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6480004a80 0x7f6490154230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.186+0000 7f6494f29640 1 -- 172.21.15.40:0/4245679147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f649013f1d0 msgr2=0x7f649013f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:32.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.186+0000 7f6494f29640 1 --2- 172.21.15.40:0/4245679147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f649013f1d0 0x7f649013f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:32.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.186+0000 7f6494f29640 1 -- 172.21.15.40:0/4245679147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64901555f0 con 0x7f64900af3f0 2023-10-18T09:08:32.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.186+0000 7f6495f2b640 1 --2- 172.21.15.40:0/4245679147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f649013f1d0 0x7f649013f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:32.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.186+0000 7f6494f29640 1 --2- 172.21.15.40:0/4245679147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64900af3f0 0x7f6490153c40 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f647c00df60 tx=0x7f647c00df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:32.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.186+0000 7f648dffb640 1 -- 172.21.15.40:0/4245679147 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c01a070 con 0x7f64900af3f0 2023-10-18T09:08:32.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.186+0000 7f648dffb640 1 -- 172.21.15.40:0/4245679147 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f647c005020 con 0x7f64900af3f0 2023-10-18T09:08:32.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.186+0000 7f648dffb640 1 -- 172.21.15.40:0/4245679147 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c012040 con 0x7f64900af3f0 2023-10-18T09:08:32.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.186+0000 7f649672c640 1 -- 172.21.15.40:0/4245679147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64901558d0 con 0x7f64900af3f0 2023-10-18T09:08:32.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.187+0000 7f649672c640 1 -- 172.21.15.40:0/4245679147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6490154a20 con 0x7f64900af3f0 2023-10-18T09:08:32.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.188+0000 7f649672c640 1 -- 172.21.15.40:0/4245679147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6490066e00 con 0x7f64900af3f0 2023-10-18T09:08:32.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.189+0000 7f648dffb640 1 -- 172.21.15.40:0/4245679147 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f647c005430 con 0x7f64900af3f0 2023-10-18T09:08:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.190+0000 7f648dffb640 1 --2- 172.21.15.40:0/4245679147 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f647406ae70 0x7f647406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.190+0000 7f648dffb640 1 -- 172.21.15.40:0/4245679147 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f647c08b340 con 0x7f64900af3f0 2023-10-18T09:08:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.190+0000 7f648dffb640 1 -- 172.21.15.40:0/4245679147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=184}) v3 -- 0x7f6474072a00 con 0x7f64900af3f0 2023-10-18T09:08:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.191+0000 7f649572a640 1 --2- 172.21.15.40:0/4245679147 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f647406ae70 0x7f647406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.192+0000 7f649572a640 1 --2- 172.21.15.40:0/4245679147 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f647406ae70 0x7f647406d330 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6484034000 tx=0x7f648402e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.195+0000 7f648dffb640 1 -- 172.21.15.40:0/4245679147 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f647c056460 con 0x7f64900af3f0 2023-10-18T09:08:32.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.391+0000 7f649672c640 1 -- 172.21.15.40:0/4245679147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f649015acd0 con 0x7f64900af3f0 2023-10-18T09:08:32.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.392+0000 7f648dffb640 1 -- 172.21.15.40:0/4245679147 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v183) v1 ==== 93+0+10 (secure 0 0 0) 0x7f647c059a80 con 0x7f64900af3f0 2023-10-18T09:08:32.394 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:08:32.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.394+0000 7f649672c640 1 -- 172.21.15.40:0/4245679147 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f647406ae70 msgr2=0x7f647406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:32.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.394+0000 7f649672c640 1 --2- 172.21.15.40:0/4245679147 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f647406ae70 0x7f647406d330 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6484034000 tx=0x7f648402e000 comp rx=0 tx=0).stop 2023-10-18T09:08:32.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.394+0000 7f649672c640 1 -- 172.21.15.40:0/4245679147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64900af3f0 msgr2=0x7f6490153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:32.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.394+0000 7f649672c640 1 --2- 172.21.15.40:0/4245679147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64900af3f0 0x7f6490153c40 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f647c00df60 tx=0x7f647c00df90 comp rx=0 tx=0).stop 2023-10-18T09:08:32.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.394+0000 7f649672c640 1 -- 172.21.15.40:0/4245679147 shutdown_connections 2023-10-18T09:08:32.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.394+0000 7f649672c640 1 --2- 172.21.15.40:0/4245679147 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f647406ae70 0x7f647406d330 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:32.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.394+0000 7f649672c640 1 --2- 172.21.15.40:0/4245679147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f649013f1d0 0x7f649013f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:32.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.394+0000 7f649672c640 1 --2- 172.21.15.40:0/4245679147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64900af3f0 0x7f6490153c40 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:32.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.394+0000 7f649672c640 1 --2- 172.21.15.40:0/4245679147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6480004a80 0x7f6490154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:32.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.394+0000 7f649672c640 1 -- 172.21.15.40:0/4245679147 >> 172.21.15.40:0/4245679147 conn(0x7f6490069720 msgr2=0x7f64900ccb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:32.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.395+0000 7f649672c640 1 -- 172.21.15.40:0/4245679147 shutdown_connections 2023-10-18T09:08:32.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.395+0000 7f649672c640 1 -- 172.21.15.40:0/4245679147 wait complete. 2023-10-18T09:08:32.417 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:08:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.649+0000 7f02c6a1a640 1 -- 172.21.15.40:0/2996714714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c015eac0 msgr2=0x7f02c00cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.649+0000 7f02c6a1a640 1 --2- 172.21.15.40:0/2996714714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c015eac0 0x7f02c00cff70 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f02c0069c40 tx=0x7f02bc02bda0 comp rx=0 tx=0).stop 2023-10-18T09:08:32.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.650+0000 7f02c6a1a640 1 -- 172.21.15.40:0/2996714714 shutdown_connections 2023-10-18T09:08:32.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.650+0000 7f02c6a1a640 1 --2- 172.21.15.40:0/2996714714 >> v2:172.21.15.144:3300/0 conn(0x7f02c015eea0 0x7f02c00b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:32.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.650+0000 7f02c6a1a640 1 --2- 172.21.15.40:0/2996714714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c015eac0 0x7f02c00cff70 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:32.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.650+0000 7f02c6a1a640 1 -- 172.21.15.40:0/2996714714 >> 172.21.15.40:0/2996714714 conn(0x7f02c009a610 msgr2=0x7f02c00d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:32.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.650+0000 7f02c6a1a640 1 -- 172.21.15.40:0/2996714714 shutdown_connections 2023-10-18T09:08:32.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.650+0000 7f02c6a1a640 1 -- 172.21.15.40:0/2996714714 wait complete. 2023-10-18T09:08:32.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.652+0000 7f02c6a1a640 1 Processor -- start 2023-10-18T09:08:32.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.652+0000 7f02c6a1a640 1 -- start start 2023-10-18T09:08:32.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.653+0000 7f02c6a1a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02c00d19b0 0x7f02c0159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:32.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.653+0000 7f02c6a1a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02c015eac0 0x7f02c00b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:32.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.654+0000 7f02c6a1a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c015eea0 0x7f02c01f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:32.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.654+0000 7f02c6a1a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c00b62d0 con 0x7f02c015eea0 2023-10-18T09:08:32.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.654+0000 7f02c6a1a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c00c7cc0 con 0x7f02c00d19b0 2023-10-18T09:08:32.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.654+0000 7f02c6a1a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c00b4370 con 0x7f02c015eac0 2023-10-18T09:08:32.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.654+0000 7f02c6219640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c015eea0 0x7f02c01f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:32.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.654+0000 7f02c6219640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c015eea0 0x7f02c01f30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60172/0 (socket says 172.21.15.40:60172) 2023-10-18T09:08:32.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.654+0000 7f02c6219640 1 -- 172.21.15.40:0/1523273714 learned_addr learned my addr 172.21.15.40:0/1523273714 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:32.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.654+0000 7f02c5a18640 1 --2- 172.21.15.40:0/1523273714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02c00d19b0 0x7f02c0159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:32.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.654+0000 7f02c5217640 1 --2- 172.21.15.40:0/1523273714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02c015eac0 0x7f02c00b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:32.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.655+0000 7f02c6219640 1 -- 172.21.15.40:0/1523273714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02c015eac0 msgr2=0x7f02c00b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.655+0000 7f02c6219640 1 --2- 172.21.15.40:0/1523273714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02c015eac0 0x7f02c00b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:32.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.655+0000 7f02c6219640 1 -- 172.21.15.40:0/1523273714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02c00d19b0 msgr2=0x7f02c0159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.655+0000 7f02c6219640 1 --2- 172.21.15.40:0/1523273714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02c00d19b0 0x7f02c0159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.655+0000 7f02c6219640 1 -- 172.21.15.40:0/1523273714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc009560 con 0x7f02c015eea0 2023-10-18T09:08:32.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.655+0000 7f02c6219640 1 --2- 172.21.15.40:0/1523273714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c015eea0 0x7f02c01f30e0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f02b800c780 tx=0x7f02b800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.655+0000 7f02b67fc640 1 -- 172.21.15.40:0/1523273714 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8003d80 con 0x7f02c015eea0 2023-10-18T09:08:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.655+0000 7f02b67fc640 1 -- 172.21.15.40:0/1523273714 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f02b800ce90 con 0x7f02c015eea0 2023-10-18T09:08:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.655+0000 7f02b67fc640 1 -- 172.21.15.40:0/1523273714 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b80052d0 con 0x7f02c015eea0 2023-10-18T09:08:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.655+0000 7f02c6a1a640 1 -- 172.21.15.40:0/1523273714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c00b59a0 con 0x7f02c015eea0 2023-10-18T09:08:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.655+0000 7f02c6a1a640 1 -- 172.21.15.40:0/1523273714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c00b5f60 con 0x7f02c015eea0 2023-10-18T09:08:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.657+0000 7f02c6a1a640 1 -- 172.21.15.40:0/1523273714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c0071da0 con 0x7f02c015eea0 2023-10-18T09:08:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.658+0000 7f02b67fc640 1 -- 172.21.15.40:0/1523273714 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f02b8005430 con 0x7f02c015eea0 2023-10-18T09:08:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.659+0000 7f02b67fc640 1 --2- 172.21.15.40:0/1523273714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f02a406ae40 0x7f02a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.659+0000 7f02b67fc640 1 -- 172.21.15.40:0/1523273714 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f02b8088c90 con 0x7f02c015eea0 2023-10-18T09:08:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.659+0000 7f02b67fc640 1 -- 172.21.15.40:0/1523273714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=184}) v3 -- 0x7f02a40729d0 con 0x7f02c015eea0 2023-10-18T09:08:32.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.659+0000 7f02c5a18640 1 --2- 172.21.15.40:0/1523273714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f02a406ae40 0x7f02a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.661+0000 7f02c5a18640 1 --2- 172.21.15.40:0/1523273714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f02a406ae40 0x7f02a406d300 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f02bc035000 tx=0x7f02bc004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:32.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.663+0000 7f02b67fc640 1 -- 172.21.15.40:0/1523273714 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f02b8017060 con 0x7f02c015eea0 2023-10-18T09:08:32.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.856+0000 7f02c6a1a640 1 -- 172.21.15.40:0/1523273714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f02c00d0b20 con 0x7f02c015eea0 2023-10-18T09:08:32.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.856+0000 7f02b67fc640 1 -- 172.21.15.40:0/1523273714 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v183) v1 ==== 104+0+11 (secure 0 0 0) 0x7f02b8017020 con 0x7f02c015eea0 2023-10-18T09:08:32.859 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:08:32.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.859+0000 7f02c6a1a640 1 -- 172.21.15.40:0/1523273714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f02a406ae40 msgr2=0x7f02a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:32.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.859+0000 7f02c6a1a640 1 --2- 172.21.15.40:0/1523273714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f02a406ae40 0x7f02a406d300 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f02bc035000 tx=0x7f02bc004730 comp rx=0 tx=0).stop 2023-10-18T09:08:32.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.859+0000 7f02c6a1a640 1 -- 172.21.15.40:0/1523273714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c015eea0 msgr2=0x7f02c01f30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:32.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.859+0000 7f02c6a1a640 1 --2- 172.21.15.40:0/1523273714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c015eea0 0x7f02c01f30e0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f02b800c780 tx=0x7f02b800cc40 comp rx=0 tx=0).stop 2023-10-18T09:08:32.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.859+0000 7f02c6a1a640 1 -- 172.21.15.40:0/1523273714 shutdown_connections 2023-10-18T09:08:32.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.859+0000 7f02c6a1a640 1 --2- 172.21.15.40:0/1523273714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f02a406ae40 0x7f02a406d300 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:32.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.859+0000 7f02c6a1a640 1 --2- 172.21.15.40:0/1523273714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c015eea0 0x7f02c01f30e0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:32.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.859+0000 7f02c6a1a640 1 --2- 172.21.15.40:0/1523273714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02c015eac0 0x7f02c00b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:32.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.859+0000 7f02c6a1a640 1 --2- 172.21.15.40:0/1523273714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02c00d19b0 0x7f02c0159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:32.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.859+0000 7f02c6a1a640 1 -- 172.21.15.40:0/1523273714 >> 172.21.15.40:0/1523273714 conn(0x7f02c009a610 msgr2=0x7f02c015cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:32.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.859+0000 7f02c6a1a640 1 -- 172.21.15.40:0/1523273714 shutdown_connections 2023-10-18T09:08:32.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:32.860+0000 7f02c6a1a640 1 -- 172.21.15.40:0/1523273714 wait complete. 2023-10-18T09:08:32.881 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:08:33.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.118+0000 7fa0b2bb6640 1 -- 172.21.15.40:0/1292227807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac06c410 msgr2=0x7fa0ac070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:33.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.118+0000 7fa0b2bb6640 1 --2- 172.21.15.40:0/1292227807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac06c410 0x7fa0ac070d40 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fa098009830 tx=0x7fa09802bd50 comp rx=0 tx=0).stop 2023-10-18T09:08:33.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.118+0000 7fa0b2bb6640 1 -- 172.21.15.40:0/1292227807 shutdown_connections 2023-10-18T09:08:33.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.118+0000 7fa0b2bb6640 1 --2- 172.21.15.40:0/1292227807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0ac071280 0x7fa0ac0cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:33.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.118+0000 7fa0b2bb6640 1 --2- 172.21.15.40:0/1292227807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac06c410 0x7fa0ac070d40 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:33.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.118+0000 7fa0b2bb6640 1 -- 172.21.15.40:0/1292227807 >> 172.21.15.40:0/1292227807 conn(0x7fa0ac069720 msgr2=0x7fa0ac069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:33.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.118+0000 7fa0b2bb6640 1 -- 172.21.15.40:0/1292227807 shutdown_connections 2023-10-18T09:08:33.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.118+0000 7fa0b2bb6640 1 -- 172.21.15.40:0/1292227807 wait complete. 2023-10-18T09:08:33.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.120+0000 7fa0b2bb6640 1 Processor -- start 2023-10-18T09:08:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.120+0000 7fa0b2bb6640 1 -- start start 2023-10-18T09:08:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.121+0000 7fa0b2bb6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac071280 0x7fa0ac133420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:33.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.121+0000 7fa0b2bb6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0ac13ec50 0x7fa0ac1589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.121+0000 7fa0b2bb6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0ac1f59a0 0x7fa0ac12d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.121+0000 7fa0b2bb6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac130950 con 0x7fa0ac071280 2023-10-18T09:08:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.121+0000 7fa0b2bb6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac1f5580 con 0x7fa0ac13ec50 2023-10-18T09:08:33.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.121+0000 7fa0b2bb6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac133e30 con 0x7fa0ac1f59a0 2023-10-18T09:08:33.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.121+0000 7fa0b0952640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac071280 0x7fa0ac133420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:33.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.121+0000 7fa0b0952640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac071280 0x7fa0ac133420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60182/0 (socket says 172.21.15.40:60182) 2023-10-18T09:08:33.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.121+0000 7fa0b0952640 1 -- 172.21.15.40:0/3606891212 learned_addr learned my addr 172.21.15.40:0/3606891212 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:33.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.121+0000 7fa0abfff640 1 --2- 172.21.15.40:0/3606891212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0ac13ec50 0x7fa0ac1589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:33.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.121+0000 7fa0b0952640 1 -- 172.21.15.40:0/3606891212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0ac1f59a0 msgr2=0x7fa0ac12d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:33.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.121+0000 7fa0b1153640 1 --2- 172.21.15.40:0/3606891212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0ac1f59a0 0x7fa0ac12d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:33.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.121+0000 7fa0b0952640 1 --2- 172.21.15.40:0/3606891212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0ac1f59a0 0x7fa0ac12d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:33.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.122+0000 7fa0b0952640 1 -- 172.21.15.40:0/3606891212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0ac13ec50 msgr2=0x7fa0ac1589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.122+0000 7fa0b0952640 1 --2- 172.21.15.40:0/3606891212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0ac13ec50 0x7fa0ac1589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.122+0000 7fa0b0952640 1 -- 172.21.15.40:0/3606891212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa098009490 con 0x7fa0ac071280 2023-10-18T09:08:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.122+0000 7fa0b1153640 1 --2- 172.21.15.40:0/3606891212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0ac1f59a0 0x7fa0ac12d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:33.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.122+0000 7fa0abfff640 1 --2- 172.21.15.40:0/3606891212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0ac13ec50 0x7fa0ac1589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:08:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.122+0000 7fa0b0952640 1 --2- 172.21.15.40:0/3606891212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac071280 0x7fa0ac133420 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000c3c0 tx=0x7fa0a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.122+0000 7fa0a97fa640 1 -- 172.21.15.40:0/3606891212 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0011790 con 0x7fa0ac071280 2023-10-18T09:08:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.122+0000 7fa0a97fa640 1 -- 172.21.15.40:0/3606891212 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa0a00119e0 con 0x7fa0ac071280 2023-10-18T09:08:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.122+0000 7fa0a97fa640 1 -- 172.21.15.40:0/3606891212 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0011e10 con 0x7fa0ac071280 2023-10-18T09:08:33.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.122+0000 7fa0b2bb6640 1 -- 172.21.15.40:0/3606891212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ac0aaa20 con 0x7fa0ac071280 2023-10-18T09:08:33.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.122+0000 7fa0b2bb6640 1 -- 172.21.15.40:0/3606891212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ac1f51a0 con 0x7fa0ac071280 2023-10-18T09:08:33.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.123+0000 7fa0b2bb6640 1 -- 172.21.15.40:0/3606891212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ac1f56c0 con 0x7fa0ac071280 2023-10-18T09:08:33.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.124+0000 7fa0a97fa640 1 -- 172.21.15.40:0/3606891212 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa0a001c460 con 0x7fa0ac071280 2023-10-18T09:08:33.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.125+0000 7fa0a97fa640 1 --2- 172.21.15.40:0/3606891212 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa09006ae90 0x7fa09006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:33.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.125+0000 7fa0a97fa640 1 -- 172.21.15.40:0/3606891212 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fa0a0089500 con 0x7fa0ac071280 2023-10-18T09:08:33.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.126+0000 7fa0a97fa640 1 -- 172.21.15.40:0/3606891212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=184}) v3 -- 0x7fa090072a20 con 0x7fa0ac071280 2023-10-18T09:08:33.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.126+0000 7fa0abfff640 1 --2- 172.21.15.40:0/3606891212 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa09006ae90 0x7fa09006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:33.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.127+0000 7fa0abfff640 1 --2- 172.21.15.40:0/3606891212 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa09006ae90 0x7fa09006d350 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa098036000 tx=0x7fa098004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:33.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.129+0000 7fa0a97fa640 1 -- 172.21.15.40:0/3606891212 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0ac1f56c0 con 0x7fa0ac071280 2023-10-18T09:08:33.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.328+0000 7fa0b2bb6640 1 -- 172.21.15.40:0/3606891212 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0ac1f56c0 con 0x7fa0ac071280 2023-10-18T09:08:33.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.328+0000 7fa0a97fa640 1 -- 172.21.15.40:0/3606891212 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v183) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa0ac1f56c0 con 0x7fa0ac071280 2023-10-18T09:08:33.331 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:08:33.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.330+0000 7fa0b2bb6640 1 -- 172.21.15.40:0/3606891212 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa09006ae90 msgr2=0x7fa09006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:33.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.330+0000 7fa0b2bb6640 1 --2- 172.21.15.40:0/3606891212 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa09006ae90 0x7fa09006d350 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa098036000 tx=0x7fa098004730 comp rx=0 tx=0).stop 2023-10-18T09:08:33.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.331+0000 7fa0b2bb6640 1 -- 172.21.15.40:0/3606891212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac071280 msgr2=0x7fa0ac133420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:33.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.331+0000 7fa0b2bb6640 1 --2- 172.21.15.40:0/3606891212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac071280 0x7fa0ac133420 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000c3c0 tx=0x7fa0a000c880 comp rx=0 tx=0).stop 2023-10-18T09:08:33.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.331+0000 7fa0b2bb6640 1 -- 172.21.15.40:0/3606891212 shutdown_connections 2023-10-18T09:08:33.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.331+0000 7fa0b2bb6640 1 --2- 172.21.15.40:0/3606891212 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa09006ae90 0x7fa09006d350 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:33.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.331+0000 7fa0b2bb6640 1 --2- 172.21.15.40:0/3606891212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0ac1f59a0 0x7fa0ac12d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:33.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.331+0000 7fa0b2bb6640 1 --2- 172.21.15.40:0/3606891212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0ac13ec50 0x7fa0ac1589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.331+0000 7fa0b2bb6640 1 --2- 172.21.15.40:0/3606891212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0ac071280 0x7fa0ac133420 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.331+0000 7fa0b2bb6640 1 -- 172.21.15.40:0/3606891212 >> 172.21.15.40:0/3606891212 conn(0x7fa0ac069720 msgr2=0x7fa0ac13f030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.331+0000 7fa0b2bb6640 1 -- 172.21.15.40:0/3606891212 shutdown_connections 2023-10-18T09:08:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.331+0000 7fa0b2bb6640 1 -- 172.21.15.40:0/3606891212 wait complete. 2023-10-18T09:08:33.352 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:08:33.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.582+0000 7f18be2ea640 1 -- 172.21.15.40:0/1507389005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a40039a0 msgr2=0x7f18a4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:33.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.582+0000 7f18be2ea640 1 --2- 172.21.15.40:0/1507389005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18a40039a0 0x7f18a4003dc0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f18b400bb20 tx=0x7f18a40057f0 comp rx=0 tx=0).stop 2023-10-18T09:08:33.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.582+0000 7f18bdae9640 1 -- 172.21.15.40:0/1507389005 reap_dead start 2023-10-18T09:08:33.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.582+0000 7f18be2ea640 1 -- 172.21.15.40:0/1507389005 shutdown_connections 2023-10-18T09:08:33.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.582+0000 7f18be2ea640 1 -- 172.21.15.40:0/1507389005 >> 172.21.15.40:0/1507389005 conn(0x7f18b8069720 msgr2=0x7f18b8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:33.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.582+0000 7f18be2ea640 1 -- 172.21.15.40:0/1507389005 shutdown_connections 2023-10-18T09:08:33.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.582+0000 7f18be2ea640 1 -- 172.21.15.40:0/1507389005 wait complete. 2023-10-18T09:08:33.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.584+0000 7f18be2ea640 1 Processor -- start 2023-10-18T09:08:33.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.584+0000 7f18be2ea640 1 -- start start 2023-10-18T09:08:33.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18be2ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a40039a0 0x7f18b8154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:33.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18be2ea640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18b8147ae0 0x7f18b8149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18be2ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b814ec70 0x7f18b8149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18be2ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b814f210 con 0x7f18b814ec70 2023-10-18T09:08:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18be2ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b814f4f0 con 0x7f18a40039a0 2023-10-18T09:08:33.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18be2ea640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b814f7d0 con 0x7f18b8147ae0 2023-10-18T09:08:33.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18bdae9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b814ec70 0x7f18b8149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:33.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18bdae9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b814ec70 0x7f18b8149770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60200/0 (socket says 172.21.15.40:60200) 2023-10-18T09:08:33.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18bdae9640 1 -- 172.21.15.40:0/1040707102 learned_addr learned my addr 172.21.15.40:0/1040707102 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:33.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18bd2e8640 1 --2- 172.21.15.40:0/1040707102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a40039a0 0x7f18b8154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:33.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18bdae9640 1 -- 172.21.15.40:0/1040707102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18b8147ae0 msgr2=0x7f18b8149230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:33.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18bcae7640 1 --2- 172.21.15.40:0/1040707102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18b8147ae0 0x7f18b8149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:33.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18bdae9640 1 --2- 172.21.15.40:0/1040707102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18b8147ae0 0x7f18b8149230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:33.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18bdae9640 1 -- 172.21.15.40:0/1040707102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a40039a0 msgr2=0x7f18b8154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:33.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18bdae9640 1 --2- 172.21.15.40:0/1040707102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a40039a0 0x7f18b8154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:33.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18bdae9640 1 -- 172.21.15.40:0/1040707102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b814fab0 con 0x7f18b814ec70 2023-10-18T09:08:33.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.585+0000 7f18bd2e8640 1 --2- 172.21.15.40:0/1040707102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a40039a0 0x7f18b8154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:33.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.586+0000 7f18bdae9640 1 --2- 172.21.15.40:0/1040707102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b814ec70 0x7f18b8149770 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f18b806c550 tx=0x7f18b00069b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:33.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.586+0000 7f18a9ffb640 1 -- 172.21.15.40:0/1040707102 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b0011870 con 0x7f18b814ec70 2023-10-18T09:08:33.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.586+0000 7f18a9ffb640 1 -- 172.21.15.40:0/1040707102 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f18b0011ae0 con 0x7f18b814ec70 2023-10-18T09:08:33.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.586+0000 7f18a9ffb640 1 -- 172.21.15.40:0/1040707102 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b00103c0 con 0x7f18b814ec70 2023-10-18T09:08:33.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.586+0000 7f18be2ea640 1 -- 172.21.15.40:0/1040707102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b814fd90 con 0x7f18b814ec70 2023-10-18T09:08:33.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.586+0000 7f18be2ea640 1 -- 172.21.15.40:0/1040707102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b81502b0 con 0x7f18b814ec70 2023-10-18T09:08:33.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.587+0000 7f18be2ea640 1 -- 172.21.15.40:0/1040707102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b814ffb0 con 0x7f18b814ec70 2023-10-18T09:08:33.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.588+0000 7f18a9ffb640 1 -- 172.21.15.40:0/1040707102 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f18b0010520 con 0x7f18b814ec70 2023-10-18T09:08:33.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.589+0000 7f18a9ffb640 1 --2- 172.21.15.40:0/1040707102 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f189c06ae40 0x7f189c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:33.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.589+0000 7f18a9ffb640 1 -- 172.21.15.40:0/1040707102 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f18b008a0f0 con 0x7f18b814ec70 2023-10-18T09:08:33.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.589+0000 7f18a9ffb640 1 -- 172.21.15.40:0/1040707102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=184}) v3 -- 0x7f189c0729d0 con 0x7f18b814ec70 2023-10-18T09:08:33.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.590+0000 7f18bd2e8640 1 --2- 172.21.15.40:0/1040707102 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f189c06ae40 0x7f189c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:33.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.591+0000 7f18bd2e8640 1 --2- 172.21.15.40:0/1040707102 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f189c06ae40 0x7f189c06d300 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f18b400bb20 tx=0x7f18b402e040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:33.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.593+0000 7f18a9ffb640 1 -- 172.21.15.40:0/1040707102 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18b00550e0 con 0x7f18b814ec70 2023-10-18T09:08:33.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.798+0000 7f18be2ea640 1 -- 172.21.15.40:0/1040707102 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f18b8150470 con 0x7f18b814ec70 2023-10-18T09:08:33.801 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.799+0000 7f18a9ffb640 1 -- 172.21.15.40:0/1040707102 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v183) v1 ==== 80+0+0 (secure 0 0 0) 0x7f18b0058700 con 0x7f18b814ec70 2023-10-18T09:08:33.802 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T09:08:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.801+0000 7f18be2ea640 1 -- 172.21.15.40:0/1040707102 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f189c06ae40 msgr2=0x7f189c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.801+0000 7f18be2ea640 1 --2- 172.21.15.40:0/1040707102 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f189c06ae40 0x7f189c06d300 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f18b400bb20 tx=0x7f18b402e040 comp rx=0 tx=0).stop 2023-10-18T09:08:33.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.801+0000 7f18be2ea640 1 -- 172.21.15.40:0/1040707102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b814ec70 msgr2=0x7f18b8149770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.801+0000 7f18be2ea640 1 --2- 172.21.15.40:0/1040707102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b814ec70 0x7f18b8149770 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f18b806c550 tx=0x7f18b00069b0 comp rx=0 tx=0).stop 2023-10-18T09:08:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.802+0000 7f18be2ea640 1 -- 172.21.15.40:0/1040707102 shutdown_connections 2023-10-18T09:08:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.802+0000 7f18be2ea640 1 --2- 172.21.15.40:0/1040707102 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f189c06ae40 0x7f189c06d300 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.802+0000 7f18be2ea640 1 --2- 172.21.15.40:0/1040707102 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18b814ec70 0x7f18b8149770 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.802+0000 7f18be2ea640 1 --2- 172.21.15.40:0/1040707102 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18b8147ae0 0x7f18b8149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:33.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.802+0000 7f18be2ea640 1 --2- 172.21.15.40:0/1040707102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18a40039a0 0x7f18b8154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:33.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.802+0000 7f18be2ea640 1 -- 172.21.15.40:0/1040707102 >> 172.21.15.40:0/1040707102 conn(0x7f18b8069720 msgr2=0x7f18b814a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:33.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.802+0000 7f18be2ea640 1 -- 172.21.15.40:0/1040707102 shutdown_connections 2023-10-18T09:08:33.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:33.802+0000 7f18be2ea640 1 -- 172.21.15.40:0/1040707102 wait complete. 2023-10-18T09:08:33.824 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-18T09:08:33.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:08:33.825 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-18T09:08:33.940 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T09:08:33.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:08:33.941 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T09:08:33.999 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:08:33.999 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:08:34.061 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:08:34.062 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:08:34.115 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:08:34.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.357+0000 7f59af8c2640 1 -- 172.21.15.40:0/2501054665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5994003970 msgr2=0x7f5994003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:34.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.357+0000 7f59af8c2640 1 --2- 172.21.15.40:0/2501054665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5994003970 0x7f5994003d90 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f59a4006ef0 tx=0x7f59940057f0 comp rx=0 tx=0).stop 2023-10-18T09:08:34.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.357+0000 7f59ade5f640 1 -- 172.21.15.40:0/2501054665 reap_dead start 2023-10-18T09:08:34.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.358+0000 7f59af8c2640 1 -- 172.21.15.40:0/2501054665 shutdown_connections 2023-10-18T09:08:34.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.358+0000 7f59af8c2640 1 -- 172.21.15.40:0/2501054665 >> 172.21.15.40:0/2501054665 conn(0x7f59a806cee0 msgr2=0x7f59a8069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:34.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.358+0000 7f59af8c2640 1 -- 172.21.15.40:0/2501054665 shutdown_connections 2023-10-18T09:08:34.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.358+0000 7f59af8c2640 1 -- 172.21.15.40:0/2501054665 wait complete. 2023-10-18T09:08:34.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.359+0000 7f59af8c2640 1 Processor -- start 2023-10-18T09:08:34.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.360+0000 7f59af8c2640 1 -- start start 2023-10-18T09:08:34.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.361+0000 7f59af8c2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5994003970 0x7f59a81579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:34.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.361+0000 7f59af8c2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59a81e6970 0x7f59a81e6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:34.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.361+0000 7f59af8c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59a8154400 0x7f59a8154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:34.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.361+0000 7f59af8c2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a81f5db0 con 0x7f5994003970 2023-10-18T09:08:34.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.361+0000 7f59af8c2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a81e7310 con 0x7f59a8154400 2023-10-18T09:08:34.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.361+0000 7f59af8c2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a8154da0 con 0x7f59a81e6970 2023-10-18T09:08:34.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.361+0000 7f59ad65e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5994003970 0x7f59a81579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:34.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.361+0000 7f59ad65e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5994003970 0x7f59a81579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60224/0 (socket says 172.21.15.40:60224) 2023-10-18T09:08:34.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.361+0000 7f59ad65e640 1 -- 172.21.15.40:0/2998405547 learned_addr learned my addr 172.21.15.40:0/2998405547 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:34.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.361+0000 7f59ad65e640 1 -- 172.21.15.40:0/2998405547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59a81e6970 msgr2=0x7f59a81e6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:34.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.361+0000 7f59ad65e640 1 --2- 172.21.15.40:0/2998405547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59a81e6970 0x7f59a81e6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:34.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.361+0000 7f59ad65e640 1 -- 172.21.15.40:0/2998405547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59a8154400 msgr2=0x7f59a8154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:34.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.361+0000 7f59ad65e640 1 --2- 172.21.15.40:0/2998405547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59a8154400 0x7f59a8154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:34.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.361+0000 7f59ad65e640 1 -- 172.21.15.40:0/2998405547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a81f4e70 con 0x7f5994003970 2023-10-18T09:08:34.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.362+0000 7f59ad65e640 1 --2- 172.21.15.40:0/2998405547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5994003970 0x7f59a81579d0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f59a402c7f0 tx=0x7f59a40061e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:34.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.362+0000 7f5999ffb640 1 -- 172.21.15.40:0/2998405547 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a40037f0 con 0x7f5994003970 2023-10-18T09:08:34.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.362+0000 7f5999ffb640 1 -- 172.21.15.40:0/2998405547 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f59a4003a40 con 0x7f5994003970 2023-10-18T09:08:34.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.362+0000 7f5999ffb640 1 -- 172.21.15.40:0/2998405547 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4003db0 con 0x7f5994003970 2023-10-18T09:08:34.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.362+0000 7f59af8c2640 1 -- 172.21.15.40:0/2998405547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a81f5130 con 0x7f5994003970 2023-10-18T09:08:34.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.362+0000 7f59af8c2640 1 -- 172.21.15.40:0/2998405547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a81f56f0 con 0x7f5994003970 2023-10-18T09:08:34.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.363+0000 7f59af8c2640 1 -- 172.21.15.40:0/2998405547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a81e50d0 con 0x7f5994003970 2023-10-18T09:08:34.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.364+0000 7f5999ffb640 1 -- 172.21.15.40:0/2998405547 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f59a4032070 con 0x7f5994003970 2023-10-18T09:08:34.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.365+0000 7f5999ffb640 1 --2- 172.21.15.40:0/2998405547 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f598c06ae40 0x7f598c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:34.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.365+0000 7f5999ffb640 1 -- 172.21.15.40:0/2998405547 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f59a40aae20 con 0x7f5994003970 2023-10-18T09:08:34.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.365+0000 7f5999ffb640 1 -- 172.21.15.40:0/2998405547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=184}) v3 -- 0x7f598c0729d0 con 0x7f5994003970 2023-10-18T09:08:34.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.366+0000 7f59ace5d640 1 --2- 172.21.15.40:0/2998405547 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f598c06ae40 0x7f598c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:34.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.367+0000 7f59ace5d640 1 --2- 172.21.15.40:0/2998405547 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f598c06ae40 0x7f598c06d300 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f599c005bc0 tx=0x7f599c005a70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:34.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.369+0000 7f5999ffb640 1 -- 172.21.15.40:0/2998405547 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f59a4044070 con 0x7f5994003970 2023-10-18T09:08:34.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.583+0000 7f59af8c2640 1 -- 172.21.15.40:0/2998405547 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59a81f5db0 con 0x7f5994003970 2023-10-18T09:08:34.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.585+0000 7f5999ffb640 1 -- 172.21.15.40:0/2998405547 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+16667 (secure 0 0 0) 0x7f59a4075e10 con 0x7f5994003970 2023-10-18T09:08:34.588 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:08:34.588 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":183,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:08:06.185791+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:07:14.660142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-18T09:07:15.853973+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.2","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:08:34.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.587+0000 7f59af8c2640 1 -- 172.21.15.40:0/2998405547 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f598c06ae40 msgr2=0x7f598c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:34.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.587+0000 7f59af8c2640 1 --2- 172.21.15.40:0/2998405547 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f598c06ae40 0x7f598c06d300 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f599c005bc0 tx=0x7f599c005a70 comp rx=0 tx=0).stop 2023-10-18T09:08:34.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.587+0000 7f59af8c2640 1 -- 172.21.15.40:0/2998405547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5994003970 msgr2=0x7f59a81579d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:34.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.587+0000 7f59af8c2640 1 --2- 172.21.15.40:0/2998405547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5994003970 0x7f59a81579d0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f59a402c7f0 tx=0x7f59a40061e0 comp rx=0 tx=0).stop 2023-10-18T09:08:34.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.588+0000 7f59af8c2640 1 -- 172.21.15.40:0/2998405547 shutdown_connections 2023-10-18T09:08:34.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.588+0000 7f59af8c2640 1 --2- 172.21.15.40:0/2998405547 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f598c06ae40 0x7f598c06d300 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:34.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.588+0000 7f59af8c2640 1 --2- 172.21.15.40:0/2998405547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59a8154400 0x7f59a8154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:34.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.588+0000 7f59af8c2640 1 --2- 172.21.15.40:0/2998405547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59a81e6970 0x7f59a81e6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:34.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.588+0000 7f59af8c2640 1 --2- 172.21.15.40:0/2998405547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5994003970 0x7f59a81579d0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:34.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.588+0000 7f59af8c2640 1 -- 172.21.15.40:0/2998405547 >> 172.21.15.40:0/2998405547 conn(0x7f59a806cee0 msgr2=0x7f59a80b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:34.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.588+0000 7f59af8c2640 1 -- 172.21.15.40:0/2998405547 shutdown_connections 2023-10-18T09:08:34.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.588+0000 7f59af8c2640 1 -- 172.21.15.40:0/2998405547 wait complete. 2023-10-18T09:08:34.611 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:07:14.660142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '178', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:07:15.853973+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:08:34.612 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:08:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.839+0000 7f8bcb59e640 1 -- 172.21.15.40:0/3029069854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bcc06af40 msgr2=0x7f8bcc0b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.839+0000 7f8bcb59e640 1 --2- 172.21.15.40:0/3029069854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bcc06af40 0x7f8bcc0b1300 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0cefe0 tx=0x7f8bc402bd70 comp rx=0 tx=0).stop 2023-10-18T09:08:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.839+0000 7f8bcb59e640 1 -- 172.21.15.40:0/3029069854 shutdown_connections 2023-10-18T09:08:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.839+0000 7f8bcb59e640 1 --2- 172.21.15.40:0/3029069854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bcc06af40 0x7f8bcc0b1300 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.839+0000 7f8bcb59e640 1 --2- 172.21.15.40:0/3029069854 >> v2:172.21.15.144:3300/0 conn(0x7f8bcc1584b0 0x7f8bcc06dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.839+0000 7f8bcb59e640 1 -- 172.21.15.40:0/3029069854 >> 172.21.15.40:0/3029069854 conn(0x7f8bcc0cdf70 msgr2=0x7f8bcc0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.839+0000 7f8bcb59e640 1 -- 172.21.15.40:0/3029069854 shutdown_connections 2023-10-18T09:08:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.839+0000 7f8bcb59e640 1 -- 172.21.15.40:0/3029069854 wait complete. 2023-10-18T09:08:34.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.841+0000 7f8bcb59e640 1 Processor -- start 2023-10-18T09:08:34.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.841+0000 7f8bcb59e640 1 -- start start 2023-10-18T09:08:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.842+0000 7f8bcb59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bcc06af40 0x7f8bcc14fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.842+0000 7f8bcb59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bcc1584b0 0x7f8bcc0c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.842+0000 7f8bcb59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bcc15c870 0x7f8bcc0ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.842+0000 7f8bcb59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc150530 con 0x7f8bcc15c870 2023-10-18T09:08:34.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.842+0000 7f8bcb59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0c9660 con 0x7f8bcc1584b0 2023-10-18T09:08:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.842+0000 7f8bcb59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc1f20b0 con 0x7f8bcc06af40 2023-10-18T09:08:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.842+0000 7f8bcad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bcc15c870 0x7f8bcc0ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.842+0000 7f8bcad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bcc15c870 0x7f8bcc0ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60230/0 (socket says 172.21.15.40:60230) 2023-10-18T09:08:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.842+0000 7f8bcad9d640 1 -- 172.21.15.40:0/235525054 learned_addr learned my addr 172.21.15.40:0/235525054 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.843+0000 7f8bcad9d640 1 -- 172.21.15.40:0/235525054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bcc06af40 msgr2=0x7f8bcc14fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.843+0000 7f8bca59c640 1 --2- 172.21.15.40:0/235525054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bcc06af40 0x7f8bcc14fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.843+0000 7f8bc9d9b640 1 --2- 172.21.15.40:0/235525054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bcc1584b0 0x7f8bcc0c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.843+0000 7f8bcad9d640 1 --2- 172.21.15.40:0/235525054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bcc06af40 0x7f8bcc14fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.843+0000 7f8bcad9d640 1 -- 172.21.15.40:0/235525054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bcc1584b0 msgr2=0x7f8bcc0c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.843+0000 7f8bcad9d640 1 --2- 172.21.15.40:0/235525054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bcc1584b0 0x7f8bcc0c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.843+0000 7f8bcad9d640 1 -- 172.21.15.40:0/235525054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc4009490 con 0x7f8bcc15c870 2023-10-18T09:08:34.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.843+0000 7f8bcad9d640 1 --2- 172.21.15.40:0/235525054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bcc15c870 0x7f8bcc0ada30 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009920 tx=0x7f8bc4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:34.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.843+0000 7f8bc2ffd640 1 -- 172.21.15.40:0/235525054 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4003e20 con 0x7f8bcc15c870 2023-10-18T09:08:34.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.843+0000 7f8bc2ffd640 1 -- 172.21.15.40:0/235525054 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8bc4034440 con 0x7f8bcc15c870 2023-10-18T09:08:34.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.843+0000 7f8bc2ffd640 1 -- 172.21.15.40:0/235525054 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4034830 con 0x7f8bcc15c870 2023-10-18T09:08:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.843+0000 7f8bcb59e640 1 -- 172.21.15.40:0/235525054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bcc1f2390 con 0x7f8bcc15c870 2023-10-18T09:08:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.844+0000 7f8bcb59e640 1 -- 172.21.15.40:0/235525054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bcc1f1d60 con 0x7f8bcc15c870 2023-10-18T09:08:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.845+0000 7f8bcb59e640 1 -- 172.21.15.40:0/235525054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bcc070740 con 0x7f8bcc15c870 2023-10-18T09:08:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.846+0000 7f8bc2ffd640 1 -- 172.21.15.40:0/235525054 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8bc403b040 con 0x7f8bcc15c870 2023-10-18T09:08:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.847+0000 7f8bc2ffd640 1 --2- 172.21.15.40:0/235525054 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8bac06ae40 0x7f8bac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.847+0000 7f8bc2ffd640 1 -- 172.21.15.40:0/235525054 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8bc40abbb0 con 0x7f8bcc15c870 2023-10-18T09:08:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.847+0000 7f8bc2ffd640 1 -- 172.21.15.40:0/235525054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=184}) v3 -- 0x7f8bac0729d0 con 0x7f8bcc15c870 2023-10-18T09:08:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.847+0000 7f8bca59c640 1 --2- 172.21.15.40:0/235525054 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8bac06ae40 0x7f8bac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.849+0000 7f8bca59c640 1 --2- 172.21.15.40:0/235525054 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8bac06ae40 0x7f8bac06d300 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc005c40 tx=0x7f8bbc005bd0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:34.851+0000 7f8bc2ffd640 1 -- 172.21.15.40:0/235525054 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8bc4039060 con 0x7f8bcc15c870 2023-10-18T09:08:35.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.044+0000 7f8bcb59e640 1 -- 172.21.15.40:0/235525054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8bcc1f28b0 con 0x7f8bcc15c870 2023-10-18T09:08:35.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.045+0000 7f8bc2ffd640 1 -- 172.21.15.40:0/235525054 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v183) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8bcc1f28b0 con 0x7f8bcc15c870 2023-10-18T09:08:35.047 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:08:35.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.047+0000 7f8bcb59e640 1 -- 172.21.15.40:0/235525054 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8bac06ae40 msgr2=0x7f8bac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:35.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.047+0000 7f8bcb59e640 1 --2- 172.21.15.40:0/235525054 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8bac06ae40 0x7f8bac06d300 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc005c40 tx=0x7f8bbc005bd0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.047+0000 7f8bcb59e640 1 -- 172.21.15.40:0/235525054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bcc15c870 msgr2=0x7f8bcc0ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:35.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.047+0000 7f8bcb59e640 1 --2- 172.21.15.40:0/235525054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bcc15c870 0x7f8bcc0ada30 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009920 tx=0x7f8bc4003820 comp rx=0 tx=0).stop 2023-10-18T09:08:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.047+0000 7f8bcb59e640 1 -- 172.21.15.40:0/235525054 shutdown_connections 2023-10-18T09:08:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.048+0000 7f8bcb59e640 1 --2- 172.21.15.40:0/235525054 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8bac06ae40 0x7f8bac06d300 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.048+0000 7f8bcb59e640 1 --2- 172.21.15.40:0/235525054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bcc15c870 0x7f8bcc0ada30 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.048+0000 7f8bcb59e640 1 --2- 172.21.15.40:0/235525054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bcc1584b0 0x7f8bcc0c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.048+0000 7f8bcb59e640 1 --2- 172.21.15.40:0/235525054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8bcc06af40 0x7f8bcc14fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.048+0000 7f8bcb59e640 1 -- 172.21.15.40:0/235525054 >> 172.21.15.40:0/235525054 conn(0x7f8bcc0cdf70 msgr2=0x7f8bcc06da80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.048+0000 7f8bcb59e640 1 -- 172.21.15.40:0/235525054 shutdown_connections 2023-10-18T09:08:35.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.048+0000 7f8bcb59e640 1 -- 172.21.15.40:0/235525054 wait complete. 2023-10-18T09:08:35.070 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:08:35.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.308+0000 7ff1d1bae640 1 -- 172.21.15.40:0/873171613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1cc06c2d0 msgr2=0x7ff1cc070b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:35.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.308+0000 7ff1d1bae640 1 --2- 172.21.15.40:0/873171613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1cc06c2d0 0x7ff1cc070b60 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009830 tx=0x7ff1b802bcc0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.308+0000 7ff1d1bae640 1 -- 172.21.15.40:0/873171613 shutdown_connections 2023-10-18T09:08:35.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.308+0000 7ff1d1bae640 1 --2- 172.21.15.40:0/873171613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1cc15af70 0x7ff1cc0710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.308+0000 7ff1d1bae640 1 --2- 172.21.15.40:0/873171613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1cc06c2d0 0x7ff1cc070b60 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.308+0000 7ff1d1bae640 1 -- 172.21.15.40:0/873171613 >> 172.21.15.40:0/873171613 conn(0x7ff1cc06ca10 msgr2=0x7ff1cc06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:35.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.308+0000 7ff1d1bae640 1 -- 172.21.15.40:0/873171613 shutdown_connections 2023-10-18T09:08:35.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.308+0000 7ff1d1bae640 1 -- 172.21.15.40:0/873171613 wait complete. 2023-10-18T09:08:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.310+0000 7ff1d1bae640 1 Processor -- start 2023-10-18T09:08:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.310+0000 7ff1d1bae640 1 -- start start 2023-10-18T09:08:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.311+0000 7ff1d1bae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1cc06c2d0 0x7ff1cc14f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:35.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.311+0000 7ff1d1bae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1cc0c9620 0x7ff1cc159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.311+0000 7ff1d1bae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1cc15af70 0x7ff1cc158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.311+0000 7ff1d1bae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc15a540 con 0x7ff1cc15af70 2023-10-18T09:08:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.311+0000 7ff1d1bae640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc159f90 con 0x7ff1cc0c9620 2023-10-18T09:08:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.311+0000 7ff1d1bae640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc15a260 con 0x7ff1cc06c2d0 2023-10-18T09:08:35.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.312+0000 7ff1d13ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1cc15af70 0x7ff1cc158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:35.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.312+0000 7ff1d13ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1cc15af70 0x7ff1cc158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46950/0 (socket says 172.21.15.40:46950) 2023-10-18T09:08:35.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.312+0000 7ff1d13ad640 1 -- 172.21.15.40:0/3137927214 learned_addr learned my addr 172.21.15.40:0/3137927214 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:35.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.312+0000 7ff1cbfff640 1 --2- 172.21.15.40:0/3137927214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1cc0c9620 0x7ff1cc159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:35.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.312+0000 7ff1d0bac640 1 --2- 172.21.15.40:0/3137927214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1cc06c2d0 0x7ff1cc14f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:35.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.312+0000 7ff1d13ad640 1 -- 172.21.15.40:0/3137927214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1cc06c2d0 msgr2=0x7ff1cc14f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:35.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.312+0000 7ff1d13ad640 1 --2- 172.21.15.40:0/3137927214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1cc06c2d0 0x7ff1cc14f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.312+0000 7ff1d13ad640 1 -- 172.21.15.40:0/3137927214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1cc0c9620 msgr2=0x7ff1cc159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:35.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.312+0000 7ff1d13ad640 1 --2- 172.21.15.40:0/3137927214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1cc0c9620 0x7ff1cc159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.312+0000 7ff1d13ad640 1 -- 172.21.15.40:0/3137927214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8009490 con 0x7ff1cc15af70 2023-10-18T09:08:35.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.313+0000 7ff1d13ad640 1 --2- 172.21.15.40:0/3137927214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1cc15af70 0x7ff1cc158c90 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400cad0 tx=0x7ff1c400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:35.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.313+0000 7ff1c97fa640 1 -- 172.21.15.40:0/3137927214 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c400ae20 con 0x7ff1cc15af70 2023-10-18T09:08:35.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.313+0000 7ff1c97fa640 1 -- 172.21.15.40:0/3137927214 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff1c400f450 con 0x7ff1cc15af70 2023-10-18T09:08:35.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.313+0000 7ff1c97fa640 1 -- 172.21.15.40:0/3137927214 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c400f8c0 con 0x7ff1cc15af70 2023-10-18T09:08:35.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.313+0000 7ff1d1bae640 1 -- 172.21.15.40:0/3137927214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc1581b0 con 0x7ff1cc15af70 2023-10-18T09:08:35.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.313+0000 7ff1d1bae640 1 -- 172.21.15.40:0/3137927214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1cc158770 con 0x7ff1cc15af70 2023-10-18T09:08:35.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.314+0000 7ff1d1bae640 1 -- 172.21.15.40:0/3137927214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1cc158010 con 0x7ff1cc15af70 2023-10-18T09:08:35.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.315+0000 7ff1c97fa640 1 -- 172.21.15.40:0/3137927214 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff1c400fa20 con 0x7ff1cc15af70 2023-10-18T09:08:35.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.316+0000 7ff1c97fa640 1 --2- 172.21.15.40:0/3137927214 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1b006ae40 0x7ff1b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:35.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.316+0000 7ff1c97fa640 1 -- 172.21.15.40:0/3137927214 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff1c408a2c0 con 0x7ff1cc15af70 2023-10-18T09:08:35.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.316+0000 7ff1c97fa640 1 -- 172.21.15.40:0/3137927214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=184}) v3 -- 0x7ff1b00729d0 con 0x7ff1cc15af70 2023-10-18T09:08:35.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.317+0000 7ff1d0bac640 1 --2- 172.21.15.40:0/3137927214 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1b006ae40 0x7ff1b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:35.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.318+0000 7ff1d0bac640 1 --2- 172.21.15.40:0/3137927214 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1b006ae40 0x7ff1b006d300 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0001650 tx=0x7ff1c0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:35.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.321+0000 7ff1c97fa640 1 -- 172.21.15.40:0/3137927214 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff1cc158010 con 0x7ff1cc15af70 2023-10-18T09:08:35.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.518+0000 7ff1d1bae640 1 -- 172.21.15.40:0/3137927214 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1cc158010 con 0x7ff1cc15af70 2023-10-18T09:08:35.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.519+0000 7ff1c97fa640 1 -- 172.21.15.40:0/3137927214 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v183) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff1cc158010 con 0x7ff1cc15af70 2023-10-18T09:08:35.521 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:08:35.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.521+0000 7ff1d1bae640 1 -- 172.21.15.40:0/3137927214 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1b006ae40 msgr2=0x7ff1b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:35.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.521+0000 7ff1d1bae640 1 --2- 172.21.15.40:0/3137927214 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1b006ae40 0x7ff1b006d300 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0001650 tx=0x7ff1c0009040 comp rx=0 tx=0).stop 2023-10-18T09:08:35.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.522+0000 7ff1d1bae640 1 -- 172.21.15.40:0/3137927214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1cc15af70 msgr2=0x7ff1cc158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:35.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.522+0000 7ff1d1bae640 1 --2- 172.21.15.40:0/3137927214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1cc15af70 0x7ff1cc158c90 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400cad0 tx=0x7ff1c400cf90 comp rx=0 tx=0).stop 2023-10-18T09:08:35.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.522+0000 7ff1d1bae640 1 -- 172.21.15.40:0/3137927214 shutdown_connections 2023-10-18T09:08:35.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.522+0000 7ff1d1bae640 1 --2- 172.21.15.40:0/3137927214 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff1b006ae40 0x7ff1b006d300 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.522+0000 7ff1d1bae640 1 --2- 172.21.15.40:0/3137927214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1cc15af70 0x7ff1cc158c90 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.522+0000 7ff1d1bae640 1 --2- 172.21.15.40:0/3137927214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1cc0c9620 0x7ff1cc159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.522+0000 7ff1d1bae640 1 --2- 172.21.15.40:0/3137927214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1cc06c2d0 0x7ff1cc14f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.522+0000 7ff1d1bae640 1 -- 172.21.15.40:0/3137927214 >> 172.21.15.40:0/3137927214 conn(0x7ff1cc06ca10 msgr2=0x7ff1cc0b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:35.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.522+0000 7ff1d1bae640 1 -- 172.21.15.40:0/3137927214 shutdown_connections 2023-10-18T09:08:35.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.522+0000 7ff1d1bae640 1 -- 172.21.15.40:0/3137927214 wait complete. 2023-10-18T09:08:35.545 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:08:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.778+0000 7fe8610a3640 1 -- 172.21.15.40:0/3764169565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe85c0af310 msgr2=0x7fe85c06f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.778+0000 7fe8610a3640 1 --2- 172.21.15.40:0/3764169565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe85c0af310 0x7fe85c06f510 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fe850009830 tx=0x7fe85002bc20 comp rx=0 tx=0).stop 2023-10-18T09:08:35.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.778+0000 7fe8610a3640 1 -- 172.21.15.40:0/3764169565 shutdown_connections 2023-10-18T09:08:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.778+0000 7fe8610a3640 1 --2- 172.21.15.40:0/3764169565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe85c1589c0 0x7fe85c0cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.778+0000 7fe8610a3640 1 --2- 172.21.15.40:0/3764169565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe85c0af310 0x7fe85c06f510 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.778+0000 7fe8610a3640 1 -- 172.21.15.40:0/3764169565 >> 172.21.15.40:0/3764169565 conn(0x7fe85c06ca10 msgr2=0x7fe85c06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.779+0000 7fe8610a3640 1 -- 172.21.15.40:0/3764169565 shutdown_connections 2023-10-18T09:08:35.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.779+0000 7fe8610a3640 1 -- 172.21.15.40:0/3764169565 wait complete. 2023-10-18T09:08:35.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.780+0000 7fe8610a3640 1 Processor -- start 2023-10-18T09:08:35.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.780+0000 7fe8610a3640 1 -- start start 2023-10-18T09:08:35.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.781+0000 7fe8610a3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe85c0af310 0x7fe85c143720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:35.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.781+0000 7fe8610a3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe85c1585e0 0x7fe85c164f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:35.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.781+0000 7fe8610a3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe85c1589c0 0x7fe85c1654d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:35.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.781+0000 7fe8610a3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c1f2d90 con 0x7fe85c1589c0 2023-10-18T09:08:35.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.781+0000 7fe8610a3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c1f3910 con 0x7fe85c1585e0 2023-10-18T09:08:35.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.781+0000 7fe8610a3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c1f2fe0 con 0x7fe85c0af310 2023-10-18T09:08:35.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.782+0000 7fe8608a2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe85c1589c0 0x7fe85c1654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:35.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.782+0000 7fe8608a2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe85c1589c0 0x7fe85c1654d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46960/0 (socket says 172.21.15.40:46960) 2023-10-18T09:08:35.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.782+0000 7fe8608a2640 1 -- 172.21.15.40:0/2220237920 learned_addr learned my addr 172.21.15.40:0/2220237920 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:35.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.782+0000 7fe8608a2640 1 -- 172.21.15.40:0/2220237920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe85c0af310 msgr2=0x7fe85c143720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:35.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.782+0000 7fe8608a2640 1 --2- 172.21.15.40:0/2220237920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe85c0af310 0x7fe85c143720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.782+0000 7fe8608a2640 1 -- 172.21.15.40:0/2220237920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe85c1585e0 msgr2=0x7fe85c164f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:35.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.782+0000 7fe8608a2640 1 --2- 172.21.15.40:0/2220237920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe85c1585e0 0x7fe85c164f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.782+0000 7fe8608a2640 1 -- 172.21.15.40:0/2220237920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe850009490 con 0x7fe85c1589c0 2023-10-18T09:08:35.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.782+0000 7fe8608a2640 1 --2- 172.21.15.40:0/2220237920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe85c1589c0 0x7fe85c1654d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fe85400eab0 tx=0x7fe85400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.783+0000 7fe858ff9640 1 -- 172.21.15.40:0/2220237920 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85400cea0 con 0x7fe85c1589c0 2023-10-18T09:08:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.783+0000 7fe858ff9640 1 -- 172.21.15.40:0/2220237920 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe854004b20 con 0x7fe85c1589c0 2023-10-18T09:08:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.783+0000 7fe858ff9640 1 -- 172.21.15.40:0/2220237920 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854005020 con 0x7fe85c1589c0 2023-10-18T09:08:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.783+0000 7fe8610a3640 1 -- 172.21.15.40:0/2220237920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85c1f32c0 con 0x7fe85c1589c0 2023-10-18T09:08:35.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.783+0000 7fe8610a3640 1 -- 172.21.15.40:0/2220237920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85c143c60 con 0x7fe85c1589c0 2023-10-18T09:08:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.784+0000 7fe8610a3640 1 -- 172.21.15.40:0/2220237920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85c066e00 con 0x7fe85c1589c0 2023-10-18T09:08:35.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.785+0000 7fe858ff9640 1 -- 172.21.15.40:0/2220237920 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe854005180 con 0x7fe85c1589c0 2023-10-18T09:08:35.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.786+0000 7fe858ff9640 1 --2- 172.21.15.40:0/2220237920 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe84806ae70 0x7fe84806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:35.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.786+0000 7fe858ff9640 1 -- 172.21.15.40:0/2220237920 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe854014070 con 0x7fe85c1589c0 2023-10-18T09:08:35.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.787+0000 7fe858ff9640 1 -- 172.21.15.40:0/2220237920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=184}) v3 -- 0x7fe848072a00 con 0x7fe85c1589c0 2023-10-18T09:08:35.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.787+0000 7fe85bfff640 1 --2- 172.21.15.40:0/2220237920 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe84806ae70 0x7fe84806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:35.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.788+0000 7fe85bfff640 1 --2- 172.21.15.40:0/2220237920 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe84806ae70 0x7fe84806d330 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe84c000f60 tx=0x7fe84c001950 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:35.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.790+0000 7fe858ff9640 1 -- 172.21.15.40:0/2220237920 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe854055360 con 0x7fe85c1589c0 2023-10-18T09:08:35.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.983+0000 7fe8610a3640 1 -- 172.21.15.40:0/2220237920 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe85c0c9f60 con 0x7fe85c1589c0 2023-10-18T09:08:35.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.984+0000 7fe858ff9640 1 -- 172.21.15.40:0/2220237920 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v183) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe854058980 con 0x7fe85c1589c0 2023-10-18T09:08:35.986 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:08:35.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.986+0000 7fe8610a3640 1 -- 172.21.15.40:0/2220237920 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe84806ae70 msgr2=0x7fe84806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.986+0000 7fe8610a3640 1 --2- 172.21.15.40:0/2220237920 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe84806ae70 0x7fe84806d330 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe84c000f60 tx=0x7fe84c001950 comp rx=0 tx=0).stop 2023-10-18T09:08:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.986+0000 7fe8610a3640 1 -- 172.21.15.40:0/2220237920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe85c1589c0 msgr2=0x7fe85c1654d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.986+0000 7fe8610a3640 1 --2- 172.21.15.40:0/2220237920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe85c1589c0 0x7fe85c1654d0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fe85400eab0 tx=0x7fe85400ef70 comp rx=0 tx=0).stop 2023-10-18T09:08:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.986+0000 7fe8610a3640 1 -- 172.21.15.40:0/2220237920 shutdown_connections 2023-10-18T09:08:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.986+0000 7fe8610a3640 1 --2- 172.21.15.40:0/2220237920 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe84806ae70 0x7fe84806d330 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.986+0000 7fe8610a3640 1 --2- 172.21.15.40:0/2220237920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe85c1589c0 0x7fe85c1654d0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.986+0000 7fe8610a3640 1 --2- 172.21.15.40:0/2220237920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe85c1585e0 0x7fe85c164f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.986+0000 7fe8610a3640 1 --2- 172.21.15.40:0/2220237920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe85c0af310 0x7fe85c143720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.986+0000 7fe8610a3640 1 -- 172.21.15.40:0/2220237920 >> 172.21.15.40:0/2220237920 conn(0x7fe85c06ca10 msgr2=0x7fe85c06cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:35.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.986+0000 7fe8610a3640 1 -- 172.21.15.40:0/2220237920 shutdown_connections 2023-10-18T09:08:35.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:35.986+0000 7fe8610a3640 1 -- 172.21.15.40:0/2220237920 wait complete. 2023-10-18T09:08:36.008 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:08:36.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.238+0000 7f31fffff640 1 -- 172.21.15.40:0/1037541663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e80039a0 msgr2=0x7f31e8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:36.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.238+0000 7f31fffff640 1 --2- 172.21.15.40:0/1037541663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e80039a0 0x7f31e8003dc0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f31f0006fd0 tx=0x7f31e80057f0 comp rx=0 tx=0).stop 2023-10-18T09:08:36.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.238+0000 7f31ff7fe640 1 -- 172.21.15.40:0/1037541663 reap_dead start 2023-10-18T09:08:36.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.238+0000 7f31fffff640 1 -- 172.21.15.40:0/1037541663 shutdown_connections 2023-10-18T09:08:36.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.238+0000 7f31fffff640 1 -- 172.21.15.40:0/1037541663 >> 172.21.15.40:0/1037541663 conn(0x7f320009a580 msgr2=0x7f32000d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:36.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.238+0000 7f31fffff640 1 -- 172.21.15.40:0/1037541663 shutdown_connections 2023-10-18T09:08:36.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.238+0000 7f31fffff640 1 -- 172.21.15.40:0/1037541663 wait complete. 2023-10-18T09:08:36.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.240+0000 7f31fffff640 1 Processor -- start 2023-10-18T09:08:36.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.240+0000 7f31fffff640 1 -- start start 2023-10-18T09:08:36.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.241+0000 7f31fffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31e80039a0 0x7f3200161450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:36.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.241+0000 7f31fffff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3200159700 0x7f3200158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:36.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.241+0000 7f31fffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32000c5d90 0x7f32000af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:36.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.241+0000 7f31fffff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3200160fc0 con 0x7f32000c5d90 2023-10-18T09:08:36.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.241+0000 7f31fffff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3200161990 con 0x7f31e80039a0 2023-10-18T09:08:36.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.241+0000 7f31fffff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32000c6c50 con 0x7f3200159700 2023-10-18T09:08:36.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.241+0000 7f31ff7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32000c5d90 0x7f32000af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:36.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.242+0000 7f31ff7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32000c5d90 0x7f32000af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46978/0 (socket says 172.21.15.40:46978) 2023-10-18T09:08:36.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.242+0000 7f31ff7fe640 1 -- 172.21.15.40:0/2160008561 learned_addr learned my addr 172.21.15.40:0/2160008561 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:36.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.242+0000 7f31feffd640 1 --2- 172.21.15.40:0/2160008561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31e80039a0 0x7f3200161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:36.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.242+0000 7f31fe7fc640 1 --2- 172.21.15.40:0/2160008561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3200159700 0x7f3200158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:36.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.242+0000 7f31ff7fe640 1 -- 172.21.15.40:0/2160008561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3200159700 msgr2=0x7f3200158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:36.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.242+0000 7f31ff7fe640 1 --2- 172.21.15.40:0/2160008561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3200159700 0x7f3200158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:36.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.242+0000 7f31ff7fe640 1 -- 172.21.15.40:0/2160008561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31e80039a0 msgr2=0x7f3200161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:36.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.242+0000 7f31ff7fe640 1 --2- 172.21.15.40:0/2160008561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31e80039a0 0x7f3200161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:36.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.242+0000 7f31ff7fe640 1 -- 172.21.15.40:0/2160008561 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32000c6690 con 0x7f32000c5d90 2023-10-18T09:08:36.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.242+0000 7f31ff7fe640 1 --2- 172.21.15.40:0/2160008561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32000c5d90 0x7f32000af230 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f32000cd820 tx=0x7f31f800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:36.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.242+0000 7f31e77fe640 1 -- 172.21.15.40:0/2160008561 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8005d40 con 0x7f32000c5d90 2023-10-18T09:08:36.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.243+0000 7f31e77fe640 1 -- 172.21.15.40:0/2160008561 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f31f800ced0 con 0x7f32000c5d90 2023-10-18T09:08:36.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.243+0000 7f31fffff640 1 -- 172.21.15.40:0/2160008561 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32000c6970 con 0x7f32000c5d90 2023-10-18T09:08:36.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.243+0000 7f31e77fe640 1 -- 172.21.15.40:0/2160008561 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f800a870 con 0x7f32000c5d90 2023-10-18T09:08:36.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.243+0000 7f31fffff640 1 -- 172.21.15.40:0/2160008561 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3200158e80 con 0x7f32000c5d90 2023-10-18T09:08:36.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.244+0000 7f31fffff640 1 -- 172.21.15.40:0/2160008561 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3200073570 con 0x7f32000c5d90 2023-10-18T09:08:36.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.245+0000 7f31e77fe640 1 -- 172.21.15.40:0/2160008561 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f31f800a9d0 con 0x7f32000c5d90 2023-10-18T09:08:36.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.246+0000 7f31e77fe640 1 --2- 172.21.15.40:0/2160008561 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31ec06ae70 0x7f31ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:36.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.246+0000 7f31e77fe640 1 -- 172.21.15.40:0/2160008561 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f31f8088cd0 con 0x7f32000c5d90 2023-10-18T09:08:36.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.247+0000 7f31feffd640 1 --2- 172.21.15.40:0/2160008561 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31ec06ae70 0x7f31ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:36.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.248+0000 7f31feffd640 1 --2- 172.21.15.40:0/2160008561 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31ec06ae70 0x7f31ec06d330 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f31f0034000 tx=0x7f31f002e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:36.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.250+0000 7f31e77fe640 1 -- 172.21.15.40:0/2160008561 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f31f8053dd0 con 0x7f32000c5d90 2023-10-18T09:08:36.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.460+0000 7f31fffff640 1 -- 172.21.15.40:0/2160008561 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f32000b1950 con 0x7f32000c5d90 2023-10-18T09:08:36.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.461+0000 7f31e77fe640 1 -- 172.21.15.40:0/2160008561 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v184) v1 ==== 80+0+0 (secure 0 0 0) 0x7f31f8017070 con 0x7f32000c5d90 2023-10-18T09:08:36.464 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T09:08:36.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.463+0000 7f31fffff640 1 -- 172.21.15.40:0/2160008561 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31ec06ae70 msgr2=0x7f31ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:36.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.463+0000 7f31fffff640 1 --2- 172.21.15.40:0/2160008561 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31ec06ae70 0x7f31ec06d330 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f31f0034000 tx=0x7f31f002e000 comp rx=0 tx=0).stop 2023-10-18T09:08:36.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.463+0000 7f31fffff640 1 -- 172.21.15.40:0/2160008561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32000c5d90 msgr2=0x7f32000af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:36.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.463+0000 7f31fffff640 1 --2- 172.21.15.40:0/2160008561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32000c5d90 0x7f32000af230 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f32000cd820 tx=0x7f31f800cb90 comp rx=0 tx=0).stop 2023-10-18T09:08:36.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.464+0000 7f31fffff640 1 -- 172.21.15.40:0/2160008561 shutdown_connections 2023-10-18T09:08:36.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.464+0000 7f31fffff640 1 --2- 172.21.15.40:0/2160008561 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f31ec06ae70 0x7f31ec06d330 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:36.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.464+0000 7f31fffff640 1 --2- 172.21.15.40:0/2160008561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32000c5d90 0x7f32000af230 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:36.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.464+0000 7f31fffff640 1 --2- 172.21.15.40:0/2160008561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3200159700 0x7f3200158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:36.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.464+0000 7f31fffff640 1 --2- 172.21.15.40:0/2160008561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31e80039a0 0x7f3200161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:36.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.464+0000 7f31fffff640 1 -- 172.21.15.40:0/2160008561 >> 172.21.15.40:0/2160008561 conn(0x7f320009a580 msgr2=0x7f320015b320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:36.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.464+0000 7f31fffff640 1 -- 172.21.15.40:0/2160008561 shutdown_connections 2023-10-18T09:08:36.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:36.464+0000 7f31fffff640 1 -- 172.21.15.40:0/2160008561 wait complete. 2023-10-18T09:08:36.485 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-18T09:08:36.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:08:36.486 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-18T09:08:36.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T09:08:36.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:08:36.604 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T09:08:36.660 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:08:36.661 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:08:36.725 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:08:36.726 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:08:36.726 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:08:36.787 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:08:36.788 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:08:37.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.030+0000 7f1ab3810640 1 -- 172.21.15.40:0/705651141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aac1586a0 msgr2=0x7f1aac0cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:37.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.030+0000 7f1ab3810640 1 --2- 172.21.15.40:0/705651141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aac1586a0 0x7f1aac0cc990 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009830 tx=0x7f1a9c02bd70 comp rx=0 tx=0).stop 2023-10-18T09:08:37.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.030+0000 7f1ab3810640 1 -- 172.21.15.40:0/705651141 shutdown_connections 2023-10-18T09:08:37.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.030+0000 7f1ab3810640 1 --2- 172.21.15.40:0/705651141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aac1586a0 0x7f1aac0cc990 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.030+0000 7f1ab3810640 1 -- 172.21.15.40:0/705651141 >> 172.21.15.40:0/705651141 conn(0x7f1aac06cee0 msgr2=0x7f1aac069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:37.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.030+0000 7f1ab3810640 1 -- 172.21.15.40:0/705651141 shutdown_connections 2023-10-18T09:08:37.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.030+0000 7f1ab3810640 1 -- 172.21.15.40:0/705651141 wait complete. 2023-10-18T09:08:37.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.032+0000 7f1ab3810640 1 Processor -- start 2023-10-18T09:08:37.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.032+0000 7f1ab3810640 1 -- start start 2023-10-18T09:08:37.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.033+0000 7f1ab3810640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aac06d710 0x7f1aac14d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:37.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.033+0000 7f1ab3810640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aac0cced0 0x7f1aac149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:37.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.033+0000 7f1ab3810640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aac1586a0 0x7f1aac145500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:37.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.033+0000 7f1ab3810640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac149770 con 0x7f1aac06d710 2023-10-18T09:08:37.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.033+0000 7f1ab3810640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac145a40 con 0x7f1aac0cced0 2023-10-18T09:08:37.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.033+0000 7f1ab3810640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac149a40 con 0x7f1aac1586a0 2023-10-18T09:08:37.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.033+0000 7f1ab15ac640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aac06d710 0x7f1aac14d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:37.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.033+0000 7f1ab15ac640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aac06d710 0x7f1aac14d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46990/0 (socket says 172.21.15.40:46990) 2023-10-18T09:08:37.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.033+0000 7f1ab15ac640 1 -- 172.21.15.40:0/1675782256 learned_addr learned my addr 172.21.15.40:0/1675782256 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:37.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.034+0000 7f1ab0dab640 1 --2- 172.21.15.40:0/1675782256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aac0cced0 0x7f1aac149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:37.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.034+0000 7f1ab15ac640 1 -- 172.21.15.40:0/1675782256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aac1586a0 msgr2=0x7f1aac145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:37.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.034+0000 7f1ab15ac640 1 --2- 172.21.15.40:0/1675782256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aac1586a0 0x7f1aac145500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.034+0000 7f1ab15ac640 1 -- 172.21.15.40:0/1675782256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aac0cced0 msgr2=0x7f1aac149230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:37.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.034+0000 7f1ab15ac640 1 --2- 172.21.15.40:0/1675782256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aac0cced0 0x7f1aac149230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.034+0000 7f1ab15ac640 1 -- 172.21.15.40:0/1675782256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c009490 con 0x7f1aac06d710 2023-10-18T09:08:37.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.034+0000 7f1ab15ac640 1 --2- 172.21.15.40:0/1675782256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aac06d710 0x7f1aac14d1c0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800c4b0 tx=0x7f1aa800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.034+0000 7f1aa1ffb640 1 -- 172.21.15.40:0/1675782256 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8005480 con 0x7f1aac06d710 2023-10-18T09:08:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.035+0000 7f1aa1ffb640 1 -- 172.21.15.40:0/1675782256 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1aa80036e0 con 0x7f1aac06d710 2023-10-18T09:08:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.035+0000 7f1aa1ffb640 1 -- 172.21.15.40:0/1675782256 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8003b30 con 0x7f1aac06d710 2023-10-18T09:08:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.035+0000 7f1ab3810640 1 -- 172.21.15.40:0/1675782256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aac149d20 con 0x7f1aac06d710 2023-10-18T09:08:37.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.035+0000 7f1ab3810640 1 -- 172.21.15.40:0/1675782256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aac0ca440 con 0x7f1aac06d710 2023-10-18T09:08:37.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.036+0000 7f1ab3810640 1 -- 172.21.15.40:0/1675782256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aac0c9fc0 con 0x7f1aac06d710 2023-10-18T09:08:37.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.037+0000 7f1aa1ffb640 1 -- 172.21.15.40:0/1675782256 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1aa8003c90 con 0x7f1aac06d710 2023-10-18T09:08:37.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.038+0000 7f1aa1ffb640 1 --2- 172.21.15.40:0/1675782256 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1a9006ae40 0x7f1a9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:37.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.039+0000 7f1aa1ffb640 1 -- 172.21.15.40:0/1675782256 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1aa8088ee0 con 0x7f1aac06d710 2023-10-18T09:08:37.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.039+0000 7f1ab0dab640 1 --2- 172.21.15.40:0/1675782256 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1a9006ae40 0x7f1a9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:37.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.040+0000 7f1ab0dab640 1 --2- 172.21.15.40:0/1675782256 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1a9006ae40 0x7f1a9006d300 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c036000 tx=0x7f1a9c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:37.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.043+0000 7f1aa1ffb640 1 -- 172.21.15.40:0/1675782256 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1aac0c9fc0 con 0x7f1aac06d710 2023-10-18T09:08:37.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.273+0000 7f1ab3810640 1 -- 172.21.15.40:0/1675782256 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1aac0c9fc0 con 0x7f1aac06d710 2023-10-18T09:08:37.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.275+0000 7f1aa1ffb640 1 -- 172.21.15.40:0/1675782256 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f1aac0c9fc0 con 0x7f1aac06d710 2023-10-18T09:08:37.277 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:08:37.277 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6823,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3592705338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3592705338},{"type":"v1","addr":"172.21.15.40:6835","nonce":3592705338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":6829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1076939595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1076939595},{"type":"v1","addr":"172.21.15.144:6836","nonce":1076939595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":6832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2483821333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2483821333},{"type":"v1","addr":"172.21.15.144:6837","nonce":2483821333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:07:16.832438+0000","modified":"2023-10-18T09:07:19.910479+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8826},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8826":{"gid":8826,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/137474609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":137474609},{"type":"v1","addr":"172.21.15.40:6834","nonce":137474609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-18T09:08:37.278 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 82 2023-10-18T09:08:37.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.277+0000 7f1ab3810640 1 -- 172.21.15.40:0/1675782256 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1a9006ae40 msgr2=0x7f1a9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:37.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.277+0000 7f1ab3810640 1 --2- 172.21.15.40:0/1675782256 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1a9006ae40 0x7f1a9006d300 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c036000 tx=0x7f1a9c004730 comp rx=0 tx=0).stop 2023-10-18T09:08:37.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.278+0000 7f1ab3810640 1 -- 172.21.15.40:0/1675782256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aac06d710 msgr2=0x7f1aac14d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:37.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.278+0000 7f1ab3810640 1 --2- 172.21.15.40:0/1675782256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aac06d710 0x7f1aac14d1c0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800c4b0 tx=0x7f1aa800c970 comp rx=0 tx=0).stop 2023-10-18T09:08:37.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.278+0000 7f1ab3810640 1 -- 172.21.15.40:0/1675782256 shutdown_connections 2023-10-18T09:08:37.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.278+0000 7f1ab3810640 1 --2- 172.21.15.40:0/1675782256 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f1a9006ae40 0x7f1a9006d300 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.278+0000 7f1ab3810640 1 --2- 172.21.15.40:0/1675782256 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1aac1586a0 0x7f1aac145500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.278+0000 7f1ab3810640 1 --2- 172.21.15.40:0/1675782256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aac0cced0 0x7f1aac149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.278+0000 7f1ab3810640 1 --2- 172.21.15.40:0/1675782256 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1aac06d710 0x7f1aac14d1c0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.278+0000 7f1ab3810640 1 -- 172.21.15.40:0/1675782256 >> 172.21.15.40:0/1675782256 conn(0x7f1aac06cee0 msgr2=0x7f1aac150a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:37.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.278+0000 7f1ab3810640 1 -- 172.21.15.40:0/1675782256 shutdown_connections 2023-10-18T09:08:37.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.278+0000 7f1ab3810640 1 -- 172.21.15.40:0/1675782256 wait complete. 2023-10-18T09:08:37.303 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:08:37.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.517+0000 7f2097fff640 1 -- 172.21.15.40:0/2244231991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2080003970 msgr2=0x7f2080003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:37.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.517+0000 7f2097fff640 1 --2- 172.21.15.40:0/2244231991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2080003970 0x7f2080003d90 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f208800ac60 tx=0x7f20800057f0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.518+0000 7f2097fff640 1 -- 172.21.15.40:0/2244231991 shutdown_connections 2023-10-18T09:08:37.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.518+0000 7f2097fff640 1 --2- 172.21.15.40:0/2244231991 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2080003970 0x7f2080003d90 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.518+0000 7f2097fff640 1 -- 172.21.15.40:0/2244231991 >> 172.21.15.40:0/2244231991 conn(0x7f2098069720 msgr2=0x7f2098069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.518+0000 7f2097fff640 1 -- 172.21.15.40:0/2244231991 shutdown_connections 2023-10-18T09:08:37.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.518+0000 7f2097fff640 1 -- 172.21.15.40:0/2244231991 wait complete. 2023-10-18T09:08:37.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.520+0000 7f2097fff640 1 Processor -- start 2023-10-18T09:08:37.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.520+0000 7f2097fff640 1 -- start start 2023-10-18T09:08:37.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.521+0000 7f2097fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2080003970 0x7f20981f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:37.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.521+0000 7f2097fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2098149080 0x7f20981e25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:37.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.521+0000 7f2097fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981e2b00 0x7f20981e4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:37.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.521+0000 7f2097fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20981e5900 con 0x7f20981e2b00 2023-10-18T09:08:37.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.521+0000 7f2097fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20981e2f80 con 0x7f2098149080 2023-10-18T09:08:37.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.521+0000 7f2097fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20981e5620 con 0x7f2080003970 2023-10-18T09:08:37.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.521+0000 7f20977fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981e2b00 0x7f20981e4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:37.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.521+0000 7f20977fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981e2b00 0x7f20981e4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47004/0 (socket says 172.21.15.40:47004) 2023-10-18T09:08:37.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.521+0000 7f20977fe640 1 -- 172.21.15.40:0/2327663309 learned_addr learned my addr 172.21.15.40:0/2327663309 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:37.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.521+0000 7f2096ffd640 1 --2- 172.21.15.40:0/2327663309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2080003970 0x7f20981f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:37.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.521+0000 7f20967fc640 1 --2- 172.21.15.40:0/2327663309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2098149080 0x7f20981e25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:37.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.521+0000 7f20977fe640 1 -- 172.21.15.40:0/2327663309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2080003970 msgr2=0x7f20981f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:37.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.521+0000 7f20977fe640 1 --2- 172.21.15.40:0/2327663309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2080003970 0x7f20981f2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.521+0000 7f20977fe640 1 -- 172.21.15.40:0/2327663309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2098149080 msgr2=0x7f20981e25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:37.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.521+0000 7f20977fe640 1 --2- 172.21.15.40:0/2327663309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2098149080 0x7f20981e25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.521+0000 7f20977fe640 1 -- 172.21.15.40:0/2327663309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209815bef0 con 0x7f20981e2b00 2023-10-18T09:08:37.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.522+0000 7f20977fe640 1 --2- 172.21.15.40:0/2327663309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981e2b00 0x7f20981e4930 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f20980b0100 tx=0x7f209000eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:37.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.522+0000 7f207f7fe640 1 -- 172.21.15.40:0/2327663309 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209000ca30 con 0x7f20981e2b00 2023-10-18T09:08:37.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.522+0000 7f207f7fe640 1 -- 172.21.15.40:0/2327663309 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f209000cc80 con 0x7f20981e2b00 2023-10-18T09:08:37.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.522+0000 7f2097fff640 1 -- 172.21.15.40:0/2327663309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20981f2080 con 0x7f20981e2b00 2023-10-18T09:08:37.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.522+0000 7f207f7fe640 1 -- 172.21.15.40:0/2327663309 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090011460 con 0x7f20981e2b00 2023-10-18T09:08:37.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.522+0000 7f2097fff640 1 -- 172.21.15.40:0/2327663309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20981f2640 con 0x7f20981e2b00 2023-10-18T09:08:37.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.523+0000 7f2097fff640 1 -- 172.21.15.40:0/2327663309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209806c230 con 0x7f20981e2b00 2023-10-18T09:08:37.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.524+0000 7f207f7fe640 1 -- 172.21.15.40:0/2327663309 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f20900115c0 con 0x7f20981e2b00 2023-10-18T09:08:37.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.525+0000 7f207f7fe640 1 --2- 172.21.15.40:0/2327663309 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f208406ae40 0x7f208406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:37.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.525+0000 7f207f7fe640 1 -- 172.21.15.40:0/2327663309 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2090089ea0 con 0x7f20981e2b00 2023-10-18T09:08:37.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.526+0000 7f2096ffd640 1 --2- 172.21.15.40:0/2327663309 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f208406ae40 0x7f208406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:37.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.527+0000 7f2096ffd640 1 --2- 172.21.15.40:0/2327663309 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f208406ae40 0x7f208406d300 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2088038000 tx=0x7f20880046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:37.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.529+0000 7f207f7fe640 1 -- 172.21.15.40:0/2327663309 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2090054f20 con 0x7f20981e2b00 2023-10-18T09:08:37.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.724+0000 7f2097fff640 1 -- 172.21.15.40:0/2327663309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2098066e00 con 0x7f20981e2b00 2023-10-18T09:08:37.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.727+0000 7f207f7fe640 1 -- 172.21.15.40:0/2327663309 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+16648 (secure 0 0 0) 0x7f209001f020 con 0x7f20981e2b00 2023-10-18T09:08:37.730 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:08:37.731 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":184,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:08:36.196873+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:07:14.660142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-18T09:07:15.853973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.2","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:08:37.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.730+0000 7f2097fff640 1 -- 172.21.15.40:0/2327663309 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f208406ae40 msgr2=0x7f208406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:37.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.730+0000 7f2097fff640 1 --2- 172.21.15.40:0/2327663309 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f208406ae40 0x7f208406d300 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2088038000 tx=0x7f20880046c0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.730+0000 7f2097fff640 1 -- 172.21.15.40:0/2327663309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981e2b00 msgr2=0x7f20981e4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:37.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.730+0000 7f2097fff640 1 --2- 172.21.15.40:0/2327663309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981e2b00 0x7f20981e4930 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f20980b0100 tx=0x7f209000eae0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.730+0000 7f2097fff640 1 -- 172.21.15.40:0/2327663309 shutdown_connections 2023-10-18T09:08:37.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.730+0000 7f2097fff640 1 --2- 172.21.15.40:0/2327663309 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f208406ae40 0x7f208406d300 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.730+0000 7f2097fff640 1 --2- 172.21.15.40:0/2327663309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20981e2b00 0x7f20981e4930 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.730+0000 7f2097fff640 1 --2- 172.21.15.40:0/2327663309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2098149080 0x7f20981e25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.730+0000 7f2097fff640 1 --2- 172.21.15.40:0/2327663309 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2080003970 0x7f20981f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.730+0000 7f2097fff640 1 -- 172.21.15.40:0/2327663309 >> 172.21.15.40:0/2327663309 conn(0x7f2098069720 msgr2=0x7f2098069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:37.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.730+0000 7f2097fff640 1 -- 172.21.15.40:0/2327663309 shutdown_connections 2023-10-18T09:08:37.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.730+0000 7f2097fff640 1 -- 172.21.15.40:0/2327663309 wait complete. 2023-10-18T09:08:37.752 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:07:14.660142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '178', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:07:15.853973+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:08:37.752 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:08:37.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.987+0000 7f840b59e640 1 -- 172.21.15.40:0/1756255121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f840c0b16f0 msgr2=0x7f840c097df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:37.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.987+0000 7f840b59e640 1 --2- 172.21.15.40:0/1756255121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f840c0b16f0 0x7f840c097df0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f840c06a1c0 tx=0x7f83f402bd70 comp rx=0 tx=0).stop 2023-10-18T09:08:37.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.987+0000 7f840b59e640 1 -- 172.21.15.40:0/1756255121 shutdown_connections 2023-10-18T09:08:37.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.987+0000 7f840b59e640 1 --2- 172.21.15.40:0/1756255121 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f840c0b16f0 0x7f840c097df0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.987+0000 7f840b59e640 1 --2- 172.21.15.40:0/1756255121 >> v2:172.21.15.144:3300/0 conn(0x7f840c097a10 0x7f840c0ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.987+0000 7f840b59e640 1 -- 172.21.15.40:0/1756255121 >> 172.21.15.40:0/1756255121 conn(0x7f840c0b0ad0 msgr2=0x7f840c0b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:37.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.987+0000 7f840b59e640 1 -- 172.21.15.40:0/1756255121 shutdown_connections 2023-10-18T09:08:37.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.987+0000 7f840b59e640 1 -- 172.21.15.40:0/1756255121 wait complete. 2023-10-18T09:08:37.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.990+0000 7f840b59e640 1 Processor -- start 2023-10-18T09:08:37.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.990+0000 7f840b59e640 1 -- start start 2023-10-18T09:08:37.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.991+0000 7f840b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f840c06aa40 0x7f840c149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:37.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.991+0000 7f840b59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c097a10 0x7f840c06d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:37.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.991+0000 7f840b59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f840c0b16f0 0x7f840c148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:37.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.991+0000 7f840b59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c14a4d0 con 0x7f840c06aa40 2023-10-18T09:08:37.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.991+0000 7f840b59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0c7b40 con 0x7f840c0b16f0 2023-10-18T09:08:37.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.991+0000 7f840b59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c1f5be0 con 0x7f840c097a10 2023-10-18T09:08:37.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.992+0000 7f840a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f840c06aa40 0x7f840c149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:37.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.992+0000 7f840a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f840c06aa40 0x7f840c149f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47012/0 (socket says 172.21.15.40:47012) 2023-10-18T09:08:37.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.992+0000 7f840a59c640 1 -- 172.21.15.40:0/2203855393 learned_addr learned my addr 172.21.15.40:0/2203855393 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:37.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.992+0000 7f8409d9b640 1 --2- 172.21.15.40:0/2203855393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c097a10 0x7f840c06d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:37.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.992+0000 7f840ad9d640 1 --2- 172.21.15.40:0/2203855393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f840c0b16f0 0x7f840c148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:37.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.992+0000 7f840a59c640 1 -- 172.21.15.40:0/2203855393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c097a10 msgr2=0x7f840c06d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:37.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.992+0000 7f840a59c640 1 --2- 172.21.15.40:0/2203855393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c097a10 0x7f840c06d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.992+0000 7f840a59c640 1 -- 172.21.15.40:0/2203855393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f840c0b16f0 msgr2=0x7f840c148ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:37.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.992+0000 7f840a59c640 1 --2- 172.21.15.40:0/2203855393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f840c0b16f0 0x7f840c148ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:37.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.992+0000 7f840a59c640 1 -- 172.21.15.40:0/2203855393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f4009490 con 0x7f840c06aa40 2023-10-18T09:08:37.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.993+0000 7f840a59c640 1 --2- 172.21.15.40:0/2203855393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f840c06aa40 0x7f840c149f90 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00eab0 tx=0x7f83fc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:37.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.993+0000 7f8402ffd640 1 -- 172.21.15.40:0/2203855393 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc00ce40 con 0x7f840c06aa40 2023-10-18T09:08:37.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.993+0000 7f8402ffd640 1 -- 172.21.15.40:0/2203855393 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f83fc011560 con 0x7f840c06aa40 2023-10-18T09:08:37.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.993+0000 7f8402ffd640 1 -- 172.21.15.40:0/2203855393 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc011910 con 0x7f840c06aa40 2023-10-18T09:08:38.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.993+0000 7f840b59e640 1 -- 172.21.15.40:0/2203855393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840c0c7e00 con 0x7f840c06aa40 2023-10-18T09:08:38.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.993+0000 7f840b59e640 1 -- 172.21.15.40:0/2203855393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840c06de90 con 0x7f840c06aa40 2023-10-18T09:08:38.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.995+0000 7f840b59e640 1 -- 172.21.15.40:0/2203855393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840c0c79a0 con 0x7f840c06aa40 2023-10-18T09:08:38.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.995+0000 7f8402ffd640 1 -- 172.21.15.40:0/2203855393 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f83fc011a70 con 0x7f840c06aa40 2023-10-18T09:08:38.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.997+0000 7f8402ffd640 1 --2- 172.21.15.40:0/2203855393 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83ec06ae40 0x7f83ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:38.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.997+0000 7f8402ffd640 1 -- 172.21.15.40:0/2203855393 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f83fc08a3a0 con 0x7f840c06aa40 2023-10-18T09:08:38.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.997+0000 7f8409d9b640 1 --2- 172.21.15.40:0/2203855393 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83ec06ae40 0x7f83ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:38.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:37.998+0000 7f8409d9b640 1 --2- 172.21.15.40:0/2203855393 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83ec06ae40 0x7f83ec06d300 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f83f4036000 tx=0x7f83f4004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:38.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.001+0000 7f8402ffd640 1 -- 172.21.15.40:0/2203855393 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f840c0c79a0 con 0x7f840c06aa40 2023-10-18T09:08:38.135 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:08:38.142 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:08:38.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.199+0000 7f840b59e640 1 -- 172.21.15.40:0/2203855393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f840c0c79a0 con 0x7f840c06aa40 2023-10-18T09:08:38.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.200+0000 7f8402ffd640 1 -- 172.21.15.40:0/2203855393 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v184) v1 ==== 93+0+10 (secure 0 0 0) 0x7f840c0c79a0 con 0x7f840c06aa40 2023-10-18T09:08:38.203 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:08:38.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.202+0000 7f840b59e640 1 -- 172.21.15.40:0/2203855393 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83ec06ae40 msgr2=0x7f83ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:38.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.202+0000 7f840b59e640 1 --2- 172.21.15.40:0/2203855393 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83ec06ae40 0x7f83ec06d300 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f83f4036000 tx=0x7f83f4004730 comp rx=0 tx=0).stop 2023-10-18T09:08:38.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.202+0000 7f840b59e640 1 -- 172.21.15.40:0/2203855393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f840c06aa40 msgr2=0x7f840c149f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:38.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.202+0000 7f840b59e640 1 --2- 172.21.15.40:0/2203855393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f840c06aa40 0x7f840c149f90 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00eab0 tx=0x7f83fc00ef70 comp rx=0 tx=0).stop 2023-10-18T09:08:38.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.202+0000 7f840b59e640 1 -- 172.21.15.40:0/2203855393 shutdown_connections 2023-10-18T09:08:38.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.202+0000 7f840b59e640 1 --2- 172.21.15.40:0/2203855393 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f83ec06ae40 0x7f83ec06d300 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.202+0000 7f840b59e640 1 --2- 172.21.15.40:0/2203855393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f840c0b16f0 0x7f840c148ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.202+0000 7f840b59e640 1 --2- 172.21.15.40:0/2203855393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c097a10 0x7f840c06d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.202+0000 7f840b59e640 1 --2- 172.21.15.40:0/2203855393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f840c06aa40 0x7f840c149f90 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.202+0000 7f840b59e640 1 -- 172.21.15.40:0/2203855393 >> 172.21.15.40:0/2203855393 conn(0x7f840c0b0ad0 msgr2=0x7f840c15a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:38.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.202+0000 7f840b59e640 1 -- 172.21.15.40:0/2203855393 shutdown_connections 2023-10-18T09:08:38.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.202+0000 7f840b59e640 1 -- 172.21.15.40:0/2203855393 wait complete. 2023-10-18T09:08:38.224 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:08:38.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.440+0000 7f5295fe9640 1 -- 172.21.15.40:0/1049767923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52900ac430 msgr2=0x7f5290155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.440+0000 7f5295fe9640 1 --2- 172.21.15.40:0/1049767923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52900ac430 0x7f5290155330 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f5288005fd0 tx=0x7f528802c240 comp rx=0 tx=0).stop 2023-10-18T09:08:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.440+0000 7f5295fe9640 1 -- 172.21.15.40:0/1049767923 shutdown_connections 2023-10-18T09:08:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.440+0000 7f5295fe9640 1 --2- 172.21.15.40:0/1049767923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52900ac430 0x7f5290155330 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.440+0000 7f5295fe9640 1 --2- 172.21.15.40:0/1049767923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52900cc630 0x7f529006baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.440+0000 7f5295fe9640 1 -- 172.21.15.40:0/1049767923 >> 172.21.15.40:0/1049767923 conn(0x7f529006ca10 msgr2=0x7f529006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.440+0000 7f5295fe9640 1 -- 172.21.15.40:0/1049767923 shutdown_connections 2023-10-18T09:08:38.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.440+0000 7f5295fe9640 1 -- 172.21.15.40:0/1049767923 wait complete. 2023-10-18T09:08:38.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.442+0000 7f5295fe9640 1 Processor -- start 2023-10-18T09:08:38.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.442+0000 7f5295fe9640 1 -- start start 2023-10-18T09:08:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.443+0000 7f5295fe9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529006c030 0x7f5290151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:38.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.443+0000 7f5295fe9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52900ac430 0x7f529014b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.443+0000 7f5295fe9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52900cc630 0x7f52900cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.443+0000 7f5295fe9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5290151190 con 0x7f529006c030 2023-10-18T09:08:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.443+0000 7f5295fe9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529015d8b0 con 0x7f52900ac430 2023-10-18T09:08:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.443+0000 7f5295fe9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52901513a0 con 0x7f52900cc630 2023-10-18T09:08:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.444+0000 7f5294fe7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529006c030 0x7f5290151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.444+0000 7f5294fe7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529006c030 0x7f5290151e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47020/0 (socket says 172.21.15.40:47020) 2023-10-18T09:08:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.444+0000 7f5294fe7640 1 -- 172.21.15.40:0/1955067445 learned_addr learned my addr 172.21.15.40:0/1955067445 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.444+0000 7f5294fe7640 1 -- 172.21.15.40:0/1955067445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52900cc630 msgr2=0x7f52900cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:38.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.444+0000 7f5294fe7640 1 --2- 172.21.15.40:0/1955067445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52900cc630 0x7f52900cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.444+0000 7f5294fe7640 1 -- 172.21.15.40:0/1955067445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52900ac430 msgr2=0x7f529014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.444+0000 7f5294fe7640 1 --2- 172.21.15.40:0/1955067445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52900ac430 0x7f529014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.444+0000 7f5294fe7640 1 -- 172.21.15.40:0/1955067445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5288009900 con 0x7f529006c030 2023-10-18T09:08:38.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.444+0000 7f5294fe7640 1 --2- 172.21.15.40:0/1955067445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529006c030 0x7f5290151e80 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f528400c970 tx=0x7f528400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:38.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.445+0000 7f528d7fa640 1 -- 172.21.15.40:0/1955067445 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284007bb0 con 0x7f529006c030 2023-10-18T09:08:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.445+0000 7f528d7fa640 1 -- 172.21.15.40:0/1955067445 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5284007e00 con 0x7f529006c030 2023-10-18T09:08:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.445+0000 7f528d7fa640 1 -- 172.21.15.40:0/1955067445 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528400f6b0 con 0x7f529006c030 2023-10-18T09:08:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.445+0000 7f5295fe9640 1 -- 172.21.15.40:0/1955067445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529006f6b0 con 0x7f529006c030 2023-10-18T09:08:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.445+0000 7f5295fe9640 1 -- 172.21.15.40:0/1955067445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5290151960 con 0x7f529006c030 2023-10-18T09:08:38.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.446+0000 7f5295fe9640 1 -- 172.21.15.40:0/1955067445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5290066e00 con 0x7f529006c030 2023-10-18T09:08:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.447+0000 7f528d7fa640 1 -- 172.21.15.40:0/1955067445 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f528400f810 con 0x7f529006c030 2023-10-18T09:08:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.448+0000 7f528d7fa640 1 --2- 172.21.15.40:0/1955067445 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f527c06ae40 0x7f527c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:38.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.448+0000 7f528d7fa640 1 -- 172.21.15.40:0/1955067445 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5284089ee0 con 0x7f529006c030 2023-10-18T09:08:38.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.449+0000 7f528ffff640 1 --2- 172.21.15.40:0/1955067445 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f527c06ae40 0x7f527c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:38.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.450+0000 7f528ffff640 1 --2- 172.21.15.40:0/1955067445 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f527c06ae40 0x7f527c06d300 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5290154220 tx=0x7f52800079b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:38.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.452+0000 7f528d7fa640 1 -- 172.21.15.40:0/1955067445 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5284054ed0 con 0x7f529006c030 2023-10-18T09:08:38.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.644+0000 7f5295fe9640 1 -- 172.21.15.40:0/1955067445 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52900b2470 con 0x7f529006c030 2023-10-18T09:08:38.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.645+0000 7f528d7fa640 1 -- 172.21.15.40:0/1955067445 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v184) v1 ==== 104+0+11 (secure 0 0 0) 0x7f528401d020 con 0x7f529006c030 2023-10-18T09:08:38.648 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:08:38.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.647+0000 7f5295fe9640 1 -- 172.21.15.40:0/1955067445 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f527c06ae40 msgr2=0x7f527c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:38.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.647+0000 7f5295fe9640 1 --2- 172.21.15.40:0/1955067445 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f527c06ae40 0x7f527c06d300 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5290154220 tx=0x7f52800079b0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.647+0000 7f5295fe9640 1 -- 172.21.15.40:0/1955067445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529006c030 msgr2=0x7f5290151e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:38.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.647+0000 7f5295fe9640 1 --2- 172.21.15.40:0/1955067445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529006c030 0x7f5290151e80 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f528400c970 tx=0x7f528400ce30 comp rx=0 tx=0).stop 2023-10-18T09:08:38.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.648+0000 7f5295fe9640 1 -- 172.21.15.40:0/1955067445 shutdown_connections 2023-10-18T09:08:38.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.648+0000 7f5295fe9640 1 --2- 172.21.15.40:0/1955067445 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f527c06ae40 0x7f527c06d300 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.648+0000 7f5295fe9640 1 --2- 172.21.15.40:0/1955067445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52900cc630 0x7f52900cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.648+0000 7f5295fe9640 1 --2- 172.21.15.40:0/1955067445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52900ac430 0x7f529014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.648+0000 7f5295fe9640 1 --2- 172.21.15.40:0/1955067445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f529006c030 0x7f5290151e80 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.648+0000 7f5295fe9640 1 -- 172.21.15.40:0/1955067445 >> 172.21.15.40:0/1955067445 conn(0x7f529006ca10 msgr2=0x7f52900cb650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:38.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.648+0000 7f5295fe9640 1 -- 172.21.15.40:0/1955067445 shutdown_connections 2023-10-18T09:08:38.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.648+0000 7f5295fe9640 1 -- 172.21.15.40:0/1955067445 wait complete. 2023-10-18T09:08:38.671 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:08:38.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.909+0000 7f341f59e640 1 -- 172.21.15.40:0/181633870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342006fec0 msgr2=0x7f3420163030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:38.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.909+0000 7f341f59e640 1 --2- 172.21.15.40:0/181633870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342006fec0 0x7f3420163030 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f3418005fd0 tx=0x7f341802c2f0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.909+0000 7f341f59e640 1 -- 172.21.15.40:0/181633870 shutdown_connections 2023-10-18T09:08:38.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.909+0000 7f341f59e640 1 --2- 172.21.15.40:0/181633870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342006fec0 0x7f3420163030 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.909+0000 7f341f59e640 1 -- 172.21.15.40:0/181633870 >> 172.21.15.40:0/181633870 conn(0x7f342006ca10 msgr2=0x7f342006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:38.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.909+0000 7f341f59e640 1 -- 172.21.15.40:0/181633870 shutdown_connections 2023-10-18T09:08:38.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.909+0000 7f341f59e640 1 -- 172.21.15.40:0/181633870 wait complete. 2023-10-18T09:08:38.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.911+0000 7f341f59e640 1 Processor -- start 2023-10-18T09:08:38.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.911+0000 7f341f59e640 1 -- start start 2023-10-18T09:08:38.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f341f59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f342006c180 0x7f3420149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f341f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342006fec0 0x7f34200b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f341f59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3420155eb0 0x7f3420158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f341f59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3420151040 con 0x7f342006fec0 2023-10-18T09:08:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f341f59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3420151300 con 0x7f342006c180 2023-10-18T09:08:38.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f341f59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34201515e0 con 0x7f3420155eb0 2023-10-18T09:08:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f3417fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342006fec0 0x7f34200b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f3417fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342006fec0 0x7f34200b1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47032/0 (socket says 172.21.15.40:47032) 2023-10-18T09:08:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f3417fff640 1 -- 172.21.15.40:0/2854540168 learned_addr learned my addr 172.21.15.40:0/2854540168 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f3417fff640 1 -- 172.21.15.40:0/2854540168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3420155eb0 msgr2=0x7f3420158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f341ed9d640 1 --2- 172.21.15.40:0/2854540168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3420155eb0 0x7f3420158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f3417fff640 1 --2- 172.21.15.40:0/2854540168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3420155eb0 0x7f3420158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f341e59c640 1 --2- 172.21.15.40:0/2854540168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f342006c180 0x7f3420149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f3417fff640 1 -- 172.21.15.40:0/2854540168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f342006c180 msgr2=0x7f3420149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:38.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f3417fff640 1 --2- 172.21.15.40:0/2854540168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f342006c180 0x7f3420149c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:38.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f3417fff640 1 -- 172.21.15.40:0/2854540168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3418009900 con 0x7f342006fec0 2023-10-18T09:08:38.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.912+0000 7f341e59c640 1 --2- 172.21.15.40:0/2854540168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f342006c180 0x7f3420149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:38.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.913+0000 7f3417fff640 1 --2- 172.21.15.40:0/2854540168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342006fec0 0x7f34200b1470 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f340800c450 tx=0x7f340800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:38.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.913+0000 7f3416ffd640 1 -- 172.21.15.40:0/2854540168 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34080052d0 con 0x7f342006fec0 2023-10-18T09:08:38.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.913+0000 7f3416ffd640 1 -- 172.21.15.40:0/2854540168 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3408005430 con 0x7f342006fec0 2023-10-18T09:08:38.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.913+0000 7f3416ffd640 1 -- 172.21.15.40:0/2854540168 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3408012890 con 0x7f342006fec0 2023-10-18T09:08:38.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.913+0000 7f341f59e640 1 -- 172.21.15.40:0/2854540168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34201518c0 con 0x7f342006fec0 2023-10-18T09:08:38.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.913+0000 7f341f59e640 1 -- 172.21.15.40:0/2854540168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3420151e80 con 0x7f342006fec0 2023-10-18T09:08:38.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.914+0000 7f341f59e640 1 -- 172.21.15.40:0/2854540168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34200cb070 con 0x7f342006fec0 2023-10-18T09:08:38.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.916+0000 7f3416ffd640 1 -- 172.21.15.40:0/2854540168 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f34080035f0 con 0x7f342006fec0 2023-10-18T09:08:38.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.917+0000 7f3416ffd640 1 --2- 172.21.15.40:0/2854540168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f340006ae70 0x7f340006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:38.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.917+0000 7f3416ffd640 1 -- 172.21.15.40:0/2854540168 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f34080892c0 con 0x7f342006fec0 2023-10-18T09:08:38.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.917+0000 7f341e59c640 1 --2- 172.21.15.40:0/2854540168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f340006ae70 0x7f340006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:38.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.918+0000 7f341e59c640 1 --2- 172.21.15.40:0/2854540168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f340006ae70 0x7f340006d330 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f341000a830 tx=0x7f3410008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:38.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:38.921+0000 7f3416ffd640 1 -- 172.21.15.40:0/2854540168 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3408054340 con 0x7f342006fec0 2023-10-18T09:08:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.112+0000 7f341f59e640 1 -- 172.21.15.40:0/2854540168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f34200cc410 con 0x7f342006fec0 2023-10-18T09:08:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.113+0000 7f3416ffd640 1 -- 172.21.15.40:0/2854540168 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v184) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3408057960 con 0x7f342006fec0 2023-10-18T09:08:39.116 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:08:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.115+0000 7f341f59e640 1 -- 172.21.15.40:0/2854540168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f340006ae70 msgr2=0x7f340006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.115+0000 7f341f59e640 1 --2- 172.21.15.40:0/2854540168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f340006ae70 0x7f340006d330 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f341000a830 tx=0x7f3410008040 comp rx=0 tx=0).stop 2023-10-18T09:08:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.115+0000 7f341f59e640 1 -- 172.21.15.40:0/2854540168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342006fec0 msgr2=0x7f34200b1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.115+0000 7f341f59e640 1 --2- 172.21.15.40:0/2854540168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342006fec0 0x7f34200b1470 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f340800c450 tx=0x7f340800c910 comp rx=0 tx=0).stop 2023-10-18T09:08:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.115+0000 7f341f59e640 1 -- 172.21.15.40:0/2854540168 shutdown_connections 2023-10-18T09:08:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.115+0000 7f341f59e640 1 --2- 172.21.15.40:0/2854540168 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f340006ae70 0x7f340006d330 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.115+0000 7f341f59e640 1 --2- 172.21.15.40:0/2854540168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3420155eb0 0x7f3420158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.115+0000 7f341f59e640 1 --2- 172.21.15.40:0/2854540168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f342006fec0 0x7f34200b1470 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.115+0000 7f341f59e640 1 --2- 172.21.15.40:0/2854540168 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f342006c180 0x7f3420149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.115+0000 7f341f59e640 1 -- 172.21.15.40:0/2854540168 >> 172.21.15.40:0/2854540168 conn(0x7f342006ca10 msgr2=0x7f3420152c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.115+0000 7f341f59e640 1 -- 172.21.15.40:0/2854540168 shutdown_connections 2023-10-18T09:08:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.116+0000 7f341f59e640 1 -- 172.21.15.40:0/2854540168 wait complete. 2023-10-18T09:08:39.137 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:08:39.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.372+0000 7f28adb73640 1 -- 172.21.15.40:0/2760733285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a81586a0 msgr2=0x7f28a80b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:39.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.372+0000 7f28adb73640 1 --2- 172.21.15.40:0/2760733285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a81586a0 0x7f28a80b0bc0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f289c009830 tx=0x7f289c02bd50 comp rx=0 tx=0).stop 2023-10-18T09:08:39.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.372+0000 7f28adb73640 1 -- 172.21.15.40:0/2760733285 shutdown_connections 2023-10-18T09:08:39.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.372+0000 7f28adb73640 1 --2- 172.21.15.40:0/2760733285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28a80b1100 0x7f28a81625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.372+0000 7f28adb73640 1 --2- 172.21.15.40:0/2760733285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a81586a0 0x7f28a80b0bc0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.372+0000 7f28adb73640 1 -- 172.21.15.40:0/2760733285 >> 172.21.15.40:0/2760733285 conn(0x7f28a806cee0 msgr2=0x7f28a8069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:39.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.372+0000 7f28adb73640 1 -- 172.21.15.40:0/2760733285 shutdown_connections 2023-10-18T09:08:39.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.372+0000 7f28adb73640 1 -- 172.21.15.40:0/2760733285 wait complete. 2023-10-18T09:08:39.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.374+0000 7f28adb73640 1 Processor -- start 2023-10-18T09:08:39.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.374+0000 7f28adb73640 1 -- start start 2023-10-18T09:08:39.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.375+0000 7f28adb73640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28a806d710 0x7f28a80abe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.375+0000 7f28adb73640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28a80b1100 0x7f28a8154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.375+0000 7f28adb73640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a81586a0 0x7f28a81551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.375+0000 7f28adb73640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a80a6470 con 0x7f28a81586a0 2023-10-18T09:08:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.375+0000 7f28adb73640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a813e900 con 0x7f28a806d710 2023-10-18T09:08:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.375+0000 7f28adb73640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a813e670 con 0x7f28a80b1100 2023-10-18T09:08:39.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.375+0000 7f28a7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a81586a0 0x7f28a81551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:39.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.375+0000 7f28a7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a81586a0 0x7f28a81551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47056/0 (socket says 172.21.15.40:47056) 2023-10-18T09:08:39.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.375+0000 7f28a7fff640 1 -- 172.21.15.40:0/2233729869 learned_addr learned my addr 172.21.15.40:0/2233729869 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:39.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.375+0000 7f28a77fe640 1 --2- 172.21.15.40:0/2233729869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28a806d710 0x7f28a80abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:39.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.376+0000 7f28a7fff640 1 -- 172.21.15.40:0/2233729869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28a80b1100 msgr2=0x7f28a8154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:39.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.376+0000 7f28a7fff640 1 --2- 172.21.15.40:0/2233729869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28a80b1100 0x7f28a8154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.376+0000 7f28a7fff640 1 -- 172.21.15.40:0/2233729869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28a806d710 msgr2=0x7f28a80abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:39.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.376+0000 7f28a7fff640 1 --2- 172.21.15.40:0/2233729869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28a806d710 0x7f28a80abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.376+0000 7f28a7fff640 1 -- 172.21.15.40:0/2233729869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c009490 con 0x7f28a81586a0 2023-10-18T09:08:39.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.376+0000 7f28a7fff640 1 --2- 172.21.15.40:0/2233729869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a81586a0 0x7f28a81551c0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f28a000cce0 tx=0x7f28a000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:39.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.376+0000 7f288ffff640 1 -- 172.21.15.40:0/2233729869 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a000f450 con 0x7f28a81586a0 2023-10-18T09:08:39.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.376+0000 7f288ffff640 1 -- 172.21.15.40:0/2233729869 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f28a000f6c0 con 0x7f28a81586a0 2023-10-18T09:08:39.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.376+0000 7f28adb73640 1 -- 172.21.15.40:0/2233729869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a81522c0 con 0x7f28a81586a0 2023-10-18T09:08:39.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.376+0000 7f288ffff640 1 -- 172.21.15.40:0/2233729869 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a000fad0 con 0x7f28a81586a0 2023-10-18T09:08:39.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.376+0000 7f28adb73640 1 -- 172.21.15.40:0/2233729869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a813f160 con 0x7f28a81586a0 2023-10-18T09:08:39.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.377+0000 7f28adb73640 1 -- 172.21.15.40:0/2233729869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a813f2f0 con 0x7f28a81586a0 2023-10-18T09:08:39.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.379+0000 7f288ffff640 1 -- 172.21.15.40:0/2233729869 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f28a000fc30 con 0x7f28a81586a0 2023-10-18T09:08:39.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.380+0000 7f288ffff640 1 --2- 172.21.15.40:0/2233729869 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f289406ae70 0x7f289406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:39.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.380+0000 7f288ffff640 1 -- 172.21.15.40:0/2233729869 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f28a008a2e0 con 0x7f28a81586a0 2023-10-18T09:08:39.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.381+0000 7f28a77fe640 1 --2- 172.21.15.40:0/2233729869 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f289406ae70 0x7f289406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:39.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.382+0000 7f28a77fe640 1 --2- 172.21.15.40:0/2233729869 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f289406ae70 0x7f289406d330 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f28980046b0 tx=0x7f2898009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:39.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.385+0000 7f288ffff640 1 -- 172.21.15.40:0/2233729869 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f28a00552e0 con 0x7f28a81586a0 2023-10-18T09:08:39.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.620+0000 7f28adb73640 1 -- 172.21.15.40:0/2233729869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28a8161f80 con 0x7f28a81586a0 2023-10-18T09:08:39.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.621+0000 7f288ffff640 1 -- 172.21.15.40:0/2233729869 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f28a001d090 con 0x7f28a81586a0 2023-10-18T09:08:39.625 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:08:39.625 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6823,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3592705338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3592705338},{"type":"v1","addr":"172.21.15.40:6835","nonce":3592705338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":6829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1076939595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1076939595},{"type":"v1","addr":"172.21.15.144:6836","nonce":1076939595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":6832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2483821333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2483821333},{"type":"v1","addr":"172.21.15.144:6837","nonce":2483821333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:07:16.832438+0000","modified":"2023-10-18T09:07:19.910479+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8826},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8826":{"gid":8826,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/137474609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":137474609},{"type":"v1","addr":"172.21.15.40:6834","nonce":137474609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-18T09:08:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 82 2023-10-18T09:08:39.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.624+0000 7f28adb73640 1 -- 172.21.15.40:0/2233729869 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f289406ae70 msgr2=0x7f289406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:39.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.624+0000 7f28adb73640 1 --2- 172.21.15.40:0/2233729869 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f289406ae70 0x7f289406d330 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f28980046b0 tx=0x7f2898009040 comp rx=0 tx=0).stop 2023-10-18T09:08:39.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.624+0000 7f28adb73640 1 -- 172.21.15.40:0/2233729869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a81586a0 msgr2=0x7f28a81551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:39.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.624+0000 7f28adb73640 1 --2- 172.21.15.40:0/2233729869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a81586a0 0x7f28a81551c0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f28a000cce0 tx=0x7f28a000a620 comp rx=0 tx=0).stop 2023-10-18T09:08:39.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.624+0000 7f28adb73640 1 -- 172.21.15.40:0/2233729869 shutdown_connections 2023-10-18T09:08:39.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.624+0000 7f28adb73640 1 --2- 172.21.15.40:0/2233729869 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f289406ae70 0x7f289406d330 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.624+0000 7f28adb73640 1 --2- 172.21.15.40:0/2233729869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28a81586a0 0x7f28a81551c0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.624+0000 7f28adb73640 1 --2- 172.21.15.40:0/2233729869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28a80b1100 0x7f28a8154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.624+0000 7f28adb73640 1 --2- 172.21.15.40:0/2233729869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28a806d710 0x7f28a80abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.624+0000 7f28adb73640 1 -- 172.21.15.40:0/2233729869 >> 172.21.15.40:0/2233729869 conn(0x7f28a806cee0 msgr2=0x7f28a8069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:39.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.625+0000 7f28adb73640 1 -- 172.21.15.40:0/2233729869 shutdown_connections 2023-10-18T09:08:39.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.625+0000 7f28adb73640 1 -- 172.21.15.40:0/2233729869 wait complete. 2023-10-18T09:08:39.648 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:08:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.878+0000 7f93f9533640 1 -- 172.21.15.40:0/233743882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93f406bc50 msgr2=0x7f93f406c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.878+0000 7f93f9533640 1 --2- 172.21.15.40:0/233743882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93f406bc50 0x7f93f406c0a0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f93f406c7a0 tx=0x7f93ec02c260 comp rx=0 tx=0).stop 2023-10-18T09:08:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.878+0000 7f93f9533640 1 -- 172.21.15.40:0/233743882 shutdown_connections 2023-10-18T09:08:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.878+0000 7f93f9533640 1 --2- 172.21.15.40:0/233743882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93f406bc50 0x7f93f406c0a0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.878+0000 7f93f9533640 1 --2- 172.21.15.40:0/233743882 >> v2:172.21.15.144:3300/0 conn(0x7f93f41586a0 0x7f93f406b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.878+0000 7f93f9533640 1 --2- 172.21.15.40:0/233743882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f406d710 0x7f93f4151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.878+0000 7f93f9533640 1 -- 172.21.15.40:0/233743882 >> 172.21.15.40:0/233743882 conn(0x7f93f406cee0 msgr2=0x7f93f4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.878+0000 7f93f9533640 1 -- 172.21.15.40:0/233743882 shutdown_connections 2023-10-18T09:08:39.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.879+0000 7f93f9533640 1 -- 172.21.15.40:0/233743882 wait complete. 2023-10-18T09:08:39.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.880+0000 7f93f9533640 1 Processor -- start 2023-10-18T09:08:39.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.880+0000 7f93f9533640 1 -- start start 2023-10-18T09:08:39.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.881+0000 7f93f9533640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f406bc50 0x7f93f4147de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:39.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.881+0000 7f93f9533640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93f406d710 0x7f93f41497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:39.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.881+0000 7f93f9533640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f41586a0 0x7f93f41566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:39.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.881+0000 7f93f9533640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f4148750 con 0x7f93f406d710 2023-10-18T09:08:39.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.881+0000 7f93f9533640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f4148a30 con 0x7f93f41586a0 2023-10-18T09:08:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.881+0000 7f93f9533640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f4148d10 con 0x7f93f406bc50 2023-10-18T09:08:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.882+0000 7f93ebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93f406d710 0x7f93f41497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.882+0000 7f93ebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93f406d710 0x7f93f41497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47082/0 (socket says 172.21.15.40:47082) 2023-10-18T09:08:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.882+0000 7f93ebfff640 1 -- 172.21.15.40:0/1484455197 learned_addr learned my addr 172.21.15.40:0/1484455197 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.882+0000 7f93f37fe640 1 --2- 172.21.15.40:0/1484455197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f41586a0 0x7f93f41566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.882+0000 7f93f2ffd640 1 --2- 172.21.15.40:0/1484455197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f406bc50 0x7f93f4147de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.882+0000 7f93ebfff640 1 -- 172.21.15.40:0/1484455197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f406bc50 msgr2=0x7f93f4147de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.882+0000 7f93ebfff640 1 --2- 172.21.15.40:0/1484455197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f406bc50 0x7f93f4147de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.882+0000 7f93ebfff640 1 -- 172.21.15.40:0/1484455197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f41586a0 msgr2=0x7f93f41566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.882+0000 7f93ebfff640 1 --2- 172.21.15.40:0/1484455197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f41586a0 0x7f93f41566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.882+0000 7f93ebfff640 1 -- 172.21.15.40:0/1484455197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec009900 con 0x7f93f406d710 2023-10-18T09:08:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.882+0000 7f93f2ffd640 1 --2- 172.21.15.40:0/1484455197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f406bc50 0x7f93f4147de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.883+0000 7f93ebfff640 1 --2- 172.21.15.40:0/1484455197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93f406d710 0x7f93f41497f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00d740 tx=0x7f93dc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.883+0000 7f93eb7fe640 1 -- 172.21.15.40:0/1484455197 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc009c30 con 0x7f93f406d710 2023-10-18T09:08:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.883+0000 7f93eb7fe640 1 -- 172.21.15.40:0/1484455197 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f93dc009e80 con 0x7f93f406d710 2023-10-18T09:08:39.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.883+0000 7f93eb7fe640 1 -- 172.21.15.40:0/1484455197 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0106c0 con 0x7f93f406d710 2023-10-18T09:08:39.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.883+0000 7f93f9533640 1 -- 172.21.15.40:0/1484455197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f4148ff0 con 0x7f93f406d710 2023-10-18T09:08:39.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.883+0000 7f93f9533640 1 -- 172.21.15.40:0/1484455197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f4148320 con 0x7f93f406d710 2023-10-18T09:08:39.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.884+0000 7f93f9533640 1 -- 172.21.15.40:0/1484455197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f4066fd0 con 0x7f93f406d710 2023-10-18T09:08:39.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.885+0000 7f93eb7fe640 1 -- 172.21.15.40:0/1484455197 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f93dc010820 con 0x7f93f406d710 2023-10-18T09:08:39.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.886+0000 7f93eb7fe640 1 --2- 172.21.15.40:0/1484455197 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f93d406ae40 0x7f93d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:39.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.886+0000 7f93eb7fe640 1 -- 172.21.15.40:0/1484455197 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f93dc089b70 con 0x7f93f406d710 2023-10-18T09:08:39.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.887+0000 7f93f2ffd640 1 --2- 172.21.15.40:0/1484455197 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f93d406ae40 0x7f93d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:39.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.888+0000 7f93f2ffd640 1 --2- 172.21.15.40:0/1484455197 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f93d406ae40 0x7f93d406d300 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f93e4005fd0 tx=0x7f93e40094d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:39.890+0000 7f93eb7fe640 1 -- 172.21.15.40:0/1484455197 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93dc054bf0 con 0x7f93f406d710 2023-10-18T09:08:40.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.086+0000 7f93f9533640 1 -- 172.21.15.40:0/1484455197 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93f406fb50 con 0x7f93f406d710 2023-10-18T09:08:40.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.088+0000 7f93eb7fe640 1 -- 172.21.15.40:0/1484455197 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+16648 (secure 0 0 0) 0x7f93dc021090 con 0x7f93f406d710 2023-10-18T09:08:40.092 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:08:40.092 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":184,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:08:36.196873+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:07:14.660142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-18T09:07:15.853973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.2","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:08:40.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.091+0000 7f93f9533640 1 -- 172.21.15.40:0/1484455197 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f93d406ae40 msgr2=0x7f93d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:40.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.091+0000 7f93f9533640 1 --2- 172.21.15.40:0/1484455197 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f93d406ae40 0x7f93d406d300 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f93e4005fd0 tx=0x7f93e40094d0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.091+0000 7f93f9533640 1 -- 172.21.15.40:0/1484455197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93f406d710 msgr2=0x7f93f41497f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:40.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.091+0000 7f93f9533640 1 --2- 172.21.15.40:0/1484455197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93f406d710 0x7f93f41497f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00d740 tx=0x7f93dc00dc00 comp rx=0 tx=0).stop 2023-10-18T09:08:40.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.091+0000 7f93f9533640 1 -- 172.21.15.40:0/1484455197 shutdown_connections 2023-10-18T09:08:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.091+0000 7f93f9533640 1 --2- 172.21.15.40:0/1484455197 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f93d406ae40 0x7f93d406d300 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.091+0000 7f93f9533640 1 --2- 172.21.15.40:0/1484455197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93f41586a0 0x7f93f41566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.091+0000 7f93f9533640 1 --2- 172.21.15.40:0/1484455197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93f406d710 0x7f93f41497f0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.091+0000 7f93f9533640 1 --2- 172.21.15.40:0/1484455197 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93f406bc50 0x7f93f4147de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.091+0000 7f93f9533640 1 -- 172.21.15.40:0/1484455197 >> 172.21.15.40:0/1484455197 conn(0x7f93f406cee0 msgr2=0x7f93f4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:40.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.091+0000 7f93f9533640 1 -- 172.21.15.40:0/1484455197 shutdown_connections 2023-10-18T09:08:40.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.091+0000 7f93f9533640 1 -- 172.21.15.40:0/1484455197 wait complete. 2023-10-18T09:08:40.115 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T09:08:40.115 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:08:40.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.346+0000 7f85c90c9640 1 -- 172.21.15.40:0/2457184944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85ac00a410 msgr2=0x7f85ac00c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:40.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.346+0000 7f85c90c9640 1 --2- 172.21.15.40:0/2457184944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85ac00a410 0x7f85ac00c8d0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0043e0 tx=0x7f85ac0037d0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.346+0000 7f85c90c9640 1 -- 172.21.15.40:0/2457184944 shutdown_connections 2023-10-18T09:08:40.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.346+0000 7f85c90c9640 1 --2- 172.21.15.40:0/2457184944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85ac00a410 0x7f85ac00c8d0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.346+0000 7f85c90c9640 1 -- 172.21.15.40:0/2457184944 >> 172.21.15.40:0/2457184944 conn(0x7f85c406cee0 msgr2=0x7f85c4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:40.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.347+0000 7f85c90c9640 1 -- 172.21.15.40:0/2457184944 shutdown_connections 2023-10-18T09:08:40.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.347+0000 7f85c90c9640 1 -- 172.21.15.40:0/2457184944 wait complete. 2023-10-18T09:08:40.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.348+0000 7f85c90c9640 1 Processor -- start 2023-10-18T09:08:40.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.349+0000 7f85c90c9640 1 -- start start 2023-10-18T09:08:40.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.349+0000 7f85c90c9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85ac00a410 0x7f85c40ab350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:40.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.349+0000 7f85c90c9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85c4150960 0x7f85c413f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:40.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.349+0000 7f85c90c9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85c4156bd0 0x7f85c4157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:40.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.349+0000 7f85c90c9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c413dc50 con 0x7f85c4150960 2023-10-18T09:08:40.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.349+0000 7f85c90c9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c4157590 con 0x7f85c4156bd0 2023-10-18T09:08:40.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.349+0000 7f85c90c9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c40ab890 con 0x7f85ac00a410 2023-10-18T09:08:40.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.350+0000 7f85c259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85c4150960 0x7f85c413f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:40.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.350+0000 7f85c259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85c4150960 0x7f85c413f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47094/0 (socket says 172.21.15.40:47094) 2023-10-18T09:08:40.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.350+0000 7f85c259c640 1 -- 172.21.15.40:0/4230166653 learned_addr learned my addr 172.21.15.40:0/4230166653 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:40.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.350+0000 7f85c2d9d640 1 --2- 172.21.15.40:0/4230166653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85ac00a410 0x7f85c40ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:40.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.350+0000 7f85c359e640 1 --2- 172.21.15.40:0/4230166653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85c4156bd0 0x7f85c4157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:40.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.350+0000 7f85c259c640 1 -- 172.21.15.40:0/4230166653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85ac00a410 msgr2=0x7f85c40ab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:40.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.350+0000 7f85c259c640 1 --2- 172.21.15.40:0/4230166653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85ac00a410 0x7f85c40ab350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.350+0000 7f85c259c640 1 -- 172.21.15.40:0/4230166653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85c4156bd0 msgr2=0x7f85c4157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:40.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.350+0000 7f85c259c640 1 --2- 172.21.15.40:0/4230166653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85c4156bd0 0x7f85c4157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.350+0000 7f85c259c640 1 -- 172.21.15.40:0/4230166653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c415c1d0 con 0x7f85c4150960 2023-10-18T09:08:40.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.351+0000 7f85c259c640 1 --2- 172.21.15.40:0/4230166653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85c4150960 0x7f85c413f860 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f85b800f7d0 tx=0x7f85b800fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.351+0000 7f85ab7fe640 1 -- 172.21.15.40:0/4230166653 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b800fee0 con 0x7f85c4150960 2023-10-18T09:08:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.351+0000 7f85ab7fe640 1 -- 172.21.15.40:0/4230166653 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f85b800bd40 con 0x7f85c4150960 2023-10-18T09:08:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.351+0000 7f85ab7fe640 1 -- 172.21.15.40:0/4230166653 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8004290 con 0x7f85c4150960 2023-10-18T09:08:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.351+0000 7f85c90c9640 1 -- 172.21.15.40:0/4230166653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c4140190 con 0x7f85c4150960 2023-10-18T09:08:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.351+0000 7f85c90c9640 1 -- 172.21.15.40:0/4230166653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c4140750 con 0x7f85c4150960 2023-10-18T09:08:40.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.352+0000 7f85c90c9640 1 -- 172.21.15.40:0/4230166653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c40afba0 con 0x7f85c4150960 2023-10-18T09:08:40.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.353+0000 7f85ab7fe640 1 -- 172.21.15.40:0/4230166653 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f85b80035f0 con 0x7f85c4150960 2023-10-18T09:08:40.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.354+0000 7f85ab7fe640 1 --2- 172.21.15.40:0/4230166653 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f85b006ae40 0x7f85b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:40.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.354+0000 7f85ab7fe640 1 -- 172.21.15.40:0/4230166653 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f85b8089f50 con 0x7f85c4150960 2023-10-18T09:08:40.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.355+0000 7f85c2d9d640 1 --2- 172.21.15.40:0/4230166653 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f85b006ae40 0x7f85b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:40.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.356+0000 7f85c2d9d640 1 --2- 172.21.15.40:0/4230166653 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f85b006ae40 0x7f85b006d300 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f85b40018e0 tx=0x7f85b4000ed0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:40.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.359+0000 7f85ab7fe640 1 -- 172.21.15.40:0/4230166653 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f85b8055080 con 0x7f85c4150960 2023-10-18T09:08:40.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.590+0000 7f85c90c9640 1 -- 172.21.15.40:0/4230166653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85c406fb50 con 0x7f85c4150960 2023-10-18T09:08:40.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.591+0000 7f85ab7fe640 1 -- 172.21.15.40:0/4230166653 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f85b8020090 con 0x7f85c4150960 2023-10-18T09:08:40.595 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:08:40.595 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6823,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3592705338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3592705338},{"type":"v1","addr":"172.21.15.40:6835","nonce":3592705338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":6829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1076939595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1076939595},{"type":"v1","addr":"172.21.15.144:6836","nonce":1076939595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":6832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2483821333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2483821333},{"type":"v1","addr":"172.21.15.144:6837","nonce":2483821333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:07:16.832438+0000","modified":"2023-10-18T09:07:19.910479+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8826},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8826":{"gid":8826,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/137474609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":137474609},{"type":"v1","addr":"172.21.15.40:6834","nonce":137474609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-18T09:08:40.596 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 82 2023-10-18T09:08:40.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.594+0000 7f85c90c9640 1 -- 172.21.15.40:0/4230166653 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f85b006ae40 msgr2=0x7f85b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:40.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.594+0000 7f85c90c9640 1 --2- 172.21.15.40:0/4230166653 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f85b006ae40 0x7f85b006d300 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f85b40018e0 tx=0x7f85b4000ed0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.594+0000 7f85c90c9640 1 -- 172.21.15.40:0/4230166653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85c4150960 msgr2=0x7f85c413f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:40.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.594+0000 7f85c90c9640 1 --2- 172.21.15.40:0/4230166653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85c4150960 0x7f85c413f860 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f85b800f7d0 tx=0x7f85b800fc90 comp rx=0 tx=0).stop 2023-10-18T09:08:40.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.594+0000 7f85c90c9640 1 -- 172.21.15.40:0/4230166653 shutdown_connections 2023-10-18T09:08:40.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.594+0000 7f85c90c9640 1 --2- 172.21.15.40:0/4230166653 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f85b006ae40 0x7f85b006d300 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.594+0000 7f85c90c9640 1 --2- 172.21.15.40:0/4230166653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85c4156bd0 0x7f85c4157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.594+0000 7f85c90c9640 1 --2- 172.21.15.40:0/4230166653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85c4150960 0x7f85c413f860 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.594+0000 7f85c90c9640 1 --2- 172.21.15.40:0/4230166653 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85ac00a410 0x7f85c40ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.594+0000 7f85c90c9640 1 -- 172.21.15.40:0/4230166653 >> 172.21.15.40:0/4230166653 conn(0x7f85c406cee0 msgr2=0x7f85c40aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:40.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.594+0000 7f85c90c9640 1 -- 172.21.15.40:0/4230166653 shutdown_connections 2023-10-18T09:08:40.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.594+0000 7f85c90c9640 1 -- 172.21.15.40:0/4230166653 wait complete. 2023-10-18T09:08:40.618 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:08:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.852+0000 7fb037fff640 1 -- 172.21.15.40:0/2130237833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380b15b0 msgr2=0x7fb0380c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.852+0000 7fb037fff640 1 --2- 172.21.15.40:0/2130237833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380b15b0 0x7fb0380c99c0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009830 tx=0x7fb02c02bd60 comp rx=0 tx=0).stop 2023-10-18T09:08:40.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.852+0000 7fb037fff640 1 -- 172.21.15.40:0/2130237833 shutdown_connections 2023-10-18T09:08:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.852+0000 7fb037fff640 1 --2- 172.21.15.40:0/2130237833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380b15b0 0x7fb0380c99c0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.852+0000 7fb037fff640 1 -- 172.21.15.40:0/2130237833 >> 172.21.15.40:0/2130237833 conn(0x7fb038069720 msgr2=0x7fb038069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.853+0000 7fb037fff640 1 -- 172.21.15.40:0/2130237833 shutdown_connections 2023-10-18T09:08:40.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.853+0000 7fb037fff640 1 -- 172.21.15.40:0/2130237833 wait complete. 2023-10-18T09:08:40.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.854+0000 7fb037fff640 1 Processor -- start 2023-10-18T09:08:40.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.854+0000 7fb037fff640 1 -- start start 2023-10-18T09:08:40.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.855+0000 7fb037fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb03806d6f0 0x7fb03814bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:40.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.855+0000 7fb037fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380b15b0 0x7fb038158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:40.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.855+0000 7fb037fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0380c9f00 0x7fb03814fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:40.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.855+0000 7fb037fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03815a5c0 con 0x7fb0380b15b0 2023-10-18T09:08:40.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.855+0000 7fb037fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0381599b0 con 0x7fb03806d6f0 2023-10-18T09:08:40.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.855+0000 7fb037fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb038159c90 con 0x7fb0380c9f00 2023-10-18T09:08:40.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.856+0000 7fb0367fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380b15b0 0x7fb038158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:40.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.856+0000 7fb0367fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380b15b0 0x7fb038158fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47110/0 (socket says 172.21.15.40:47110) 2023-10-18T09:08:40.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.856+0000 7fb0367fc640 1 -- 172.21.15.40:0/4057022714 learned_addr learned my addr 172.21.15.40:0/4057022714 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:40.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.856+0000 7fb0367fc640 1 -- 172.21.15.40:0/4057022714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0380c9f00 msgr2=0x7fb03814fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:40.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.856+0000 7fb0367fc640 1 --2- 172.21.15.40:0/4057022714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0380c9f00 0x7fb03814fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.856+0000 7fb0367fc640 1 -- 172.21.15.40:0/4057022714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb03806d6f0 msgr2=0x7fb03814bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:40.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.856+0000 7fb0367fc640 1 --2- 172.21.15.40:0/4057022714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb03806d6f0 0x7fb03814bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:40.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.856+0000 7fb0367fc640 1 -- 172.21.15.40:0/4057022714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c009490 con 0x7fb0380b15b0 2023-10-18T09:08:40.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.856+0000 7fb0367fc640 1 --2- 172.21.15.40:0/4057022714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380b15b0 0x7fb038158fd0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009950 tx=0x7fb02c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:40.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.857+0000 7fb01ffff640 1 -- 172.21.15.40:0/4057022714 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c003ea0 con 0x7fb0380b15b0 2023-10-18T09:08:40.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.857+0000 7fb01ffff640 1 -- 172.21.15.40:0/4057022714 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb02c005020 con 0x7fb0380b15b0 2023-10-18T09:08:40.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.857+0000 7fb01ffff640 1 -- 172.21.15.40:0/4057022714 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c005410 con 0x7fb0380b15b0 2023-10-18T09:08:40.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.857+0000 7fb037fff640 1 -- 172.21.15.40:0/4057022714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb038159f70 con 0x7fb0380b15b0 2023-10-18T09:08:40.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.857+0000 7fb037fff640 1 -- 172.21.15.40:0/4057022714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03814c2f0 con 0x7fb0380b15b0 2023-10-18T09:08:40.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.858+0000 7fb037fff640 1 -- 172.21.15.40:0/4057022714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb038066e00 con 0x7fb0380b15b0 2023-10-18T09:08:40.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.859+0000 7fb01ffff640 1 -- 172.21.15.40:0/4057022714 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb02c0047a0 con 0x7fb0380b15b0 2023-10-18T09:08:40.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.860+0000 7fb01ffff640 1 --2- 172.21.15.40:0/4057022714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb02406ae40 0x7fb02406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:40.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.861+0000 7fb01ffff640 1 -- 172.21.15.40:0/4057022714 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb02c0ac380 con 0x7fb0380b15b0 2023-10-18T09:08:40.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.861+0000 7fb036ffd640 1 --2- 172.21.15.40:0/4057022714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb02406ae40 0x7fb02406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:40.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.862+0000 7fb036ffd640 1 --2- 172.21.15.40:0/4057022714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb02406ae40 0x7fb02406d300 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb028003cc0 tx=0x7fb0280079b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:40.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:40.865+0000 7fb01ffff640 1 -- 172.21.15.40:0/4057022714 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb02c077400 con 0x7fb0380b15b0 2023-10-18T09:08:41.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.098+0000 7fb037fff640 1 -- 172.21.15.40:0/4057022714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb038073240 con 0x7fb0380b15b0 2023-10-18T09:08:41.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.099+0000 7fb01ffff640 1 -- 172.21.15.40:0/4057022714 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fb02c07aa20 con 0x7fb0380b15b0 2023-10-18T09:08:41.103 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:08:41.103 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6823,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3592705338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3592705338},{"type":"v1","addr":"172.21.15.40:6835","nonce":3592705338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":6829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1076939595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1076939595},{"type":"v1","addr":"172.21.15.144:6836","nonce":1076939595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":6832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2483821333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2483821333},{"type":"v1","addr":"172.21.15.144:6837","nonce":2483821333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:07:16.832438+0000","modified":"2023-10-18T09:07:19.910479+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8826},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8826":{"gid":8826,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6834/137474609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":137474609},{"type":"v1","addr":"172.21.15.40:6834","nonce":137474609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-18T09:08:41.103 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 82 2023-10-18T09:08:41.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.102+0000 7fb037fff640 1 -- 172.21.15.40:0/4057022714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb02406ae40 msgr2=0x7fb02406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.102+0000 7fb037fff640 1 --2- 172.21.15.40:0/4057022714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb02406ae40 0x7fb02406d300 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb028003cc0 tx=0x7fb0280079b0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.102+0000 7fb037fff640 1 -- 172.21.15.40:0/4057022714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380b15b0 msgr2=0x7fb038158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.102+0000 7fb037fff640 1 --2- 172.21.15.40:0/4057022714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380b15b0 0x7fb038158fd0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009950 tx=0x7fb02c0038a0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.102+0000 7fb037fff640 1 -- 172.21.15.40:0/4057022714 shutdown_connections 2023-10-18T09:08:41.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.102+0000 7fb037fff640 1 --2- 172.21.15.40:0/4057022714 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb02406ae40 0x7fb02406d300 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.102+0000 7fb037fff640 1 --2- 172.21.15.40:0/4057022714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb0380c9f00 0x7fb03814fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.102+0000 7fb037fff640 1 --2- 172.21.15.40:0/4057022714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0380b15b0 0x7fb038158fd0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.102+0000 7fb037fff640 1 --2- 172.21.15.40:0/4057022714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb03806d6f0 0x7fb03814bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.102+0000 7fb037fff640 1 -- 172.21.15.40:0/4057022714 >> 172.21.15.40:0/4057022714 conn(0x7fb038069720 msgr2=0x7fb038156a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:41.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.102+0000 7fb037fff640 1 -- 172.21.15.40:0/4057022714 shutdown_connections 2023-10-18T09:08:41.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.102+0000 7fb037fff640 1 -- 172.21.15.40:0/4057022714 wait complete. 2023-10-18T09:08:41.126 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:08:41.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.360+0000 7f5d747e7640 1 -- 172.21.15.40:0/3403840919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d580039c0 msgr2=0x7f5d58003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:41.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.360+0000 7f5d747e7640 1 --2- 172.21.15.40:0/3403840919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d580039c0 0x7f5d58003de0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009b50 tx=0x7f5d580057f0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.360+0000 7f5d747e7640 1 -- 172.21.15.40:0/3403840919 shutdown_connections 2023-10-18T09:08:41.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.360+0000 7f5d747e7640 1 --2- 172.21.15.40:0/3403840919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d580039c0 0x7f5d58003de0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.360+0000 7f5d747e7640 1 -- 172.21.15.40:0/3403840919 >> 172.21.15.40:0/3403840919 conn(0x7f5d6c06cee0 msgr2=0x7f5d6c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:41.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.360+0000 7f5d747e7640 1 -- 172.21.15.40:0/3403840919 shutdown_connections 2023-10-18T09:08:41.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.360+0000 7f5d747e7640 1 -- 172.21.15.40:0/3403840919 wait complete. 2023-10-18T09:08:41.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.362+0000 7f5d747e7640 1 Processor -- start 2023-10-18T09:08:41.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.362+0000 7f5d747e7640 1 -- start start 2023-10-18T09:08:41.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.363+0000 7f5d747e7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d580039c0 0x7f5d6c1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:41.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.363+0000 7f5d747e7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d6c15d3e0 0x7f5d6c15d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:41.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.363+0000 7f5d747e7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d6c154820 0x7f5d6c154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:41.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.363+0000 7f5d747e7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c15dd60 con 0x7f5d580039c0 2023-10-18T09:08:41.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.363+0000 7f5d747e7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c155550 con 0x7f5d6c154820 2023-10-18T09:08:41.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.363+0000 7f5d747e7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c155830 con 0x7f5d6c15d3e0 2023-10-18T09:08:41.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.363+0000 7f5d72583640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d580039c0 0x7f5d6c1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:41.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.363+0000 7f5d72583640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d580039c0 0x7f5d6c1568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47136/0 (socket says 172.21.15.40:47136) 2023-10-18T09:08:41.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.363+0000 7f5d72583640 1 -- 172.21.15.40:0/2174103304 learned_addr learned my addr 172.21.15.40:0/2174103304 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:41.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.364+0000 7f5d72583640 1 -- 172.21.15.40:0/2174103304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d6c15d3e0 msgr2=0x7f5d6c15d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:41.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.364+0000 7f5d72583640 1 --2- 172.21.15.40:0/2174103304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d6c15d3e0 0x7f5d6c15d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.364+0000 7f5d72583640 1 -- 172.21.15.40:0/2174103304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d6c154820 msgr2=0x7f5d6c154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:41.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.364+0000 7f5d72583640 1 --2- 172.21.15.40:0/2174103304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d6c154820 0x7f5d6c154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.364+0000 7f5d72583640 1 -- 172.21.15.40:0/2174103304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c155b10 con 0x7f5d580039c0 2023-10-18T09:08:41.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.364+0000 7f5d72583640 1 --2- 172.21.15.40:0/2174103304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d580039c0 0x7f5d6c1568c0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009b50 tx=0x7f5d68033bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:41.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.364+0000 7f5d62ffd640 1 -- 172.21.15.40:0/2174103304 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68004230 con 0x7f5d580039c0 2023-10-18T09:08:41.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.364+0000 7f5d62ffd640 1 -- 172.21.15.40:0/2174103304 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5d68004390 con 0x7f5d580039c0 2023-10-18T09:08:41.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.364+0000 7f5d62ffd640 1 -- 172.21.15.40:0/2174103304 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d68003850 con 0x7f5d580039c0 2023-10-18T09:08:41.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.364+0000 7f5d747e7640 1 -- 172.21.15.40:0/2174103304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6c155df0 con 0x7f5d580039c0 2023-10-18T09:08:41.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.364+0000 7f5d747e7640 1 -- 172.21.15.40:0/2174103304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6c156e00 con 0x7f5d580039c0 2023-10-18T09:08:41.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.366+0000 7f5d747e7640 1 -- 172.21.15.40:0/2174103304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d6c155f80 con 0x7f5d580039c0 2023-10-18T09:08:41.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.366+0000 7f5d62ffd640 1 -- 172.21.15.40:0/2174103304 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5d680039b0 con 0x7f5d580039c0 2023-10-18T09:08:41.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.367+0000 7f5d62ffd640 1 --2- 172.21.15.40:0/2174103304 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d5006ae70 0x7f5d5006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:41.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.368+0000 7f5d62ffd640 1 -- 172.21.15.40:0/2174103304 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5d680ac240 con 0x7f5d580039c0 2023-10-18T09:08:41.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.368+0000 7f5d71d82640 1 --2- 172.21.15.40:0/2174103304 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d5006ae70 0x7f5d5006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:41.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.369+0000 7f5d71d82640 1 --2- 172.21.15.40:0/2174103304 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d5006ae70 0x7f5d5006d330 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c0b0a20 tx=0x7f5d5c001600 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:41.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.372+0000 7f5d62ffd640 1 -- 172.21.15.40:0/2174103304 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d680771a0 con 0x7f5d580039c0 2023-10-18T09:08:41.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.569+0000 7f5d747e7640 1 -- 172.21.15.40:0/2174103304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d6c066fd0 con 0x7f5d580039c0 2023-10-18T09:08:41.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.573+0000 7f5d62ffd640 1 -- 172.21.15.40:0/2174103304 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+16648 (secure 0 0 0) 0x7f5d6804b090 con 0x7f5d580039c0 2023-10-18T09:08:41.577 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:08:41.577 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":184,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:08:36.196873+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:07:14.660142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-18T09:07:15.853973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.2","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:08:41.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.575+0000 7f5d747e7640 1 -- 172.21.15.40:0/2174103304 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d5006ae70 msgr2=0x7f5d5006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:41.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.575+0000 7f5d747e7640 1 --2- 172.21.15.40:0/2174103304 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d5006ae70 0x7f5d5006d330 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c0b0a20 tx=0x7f5d5c001600 comp rx=0 tx=0).stop 2023-10-18T09:08:41.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.575+0000 7f5d747e7640 1 -- 172.21.15.40:0/2174103304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d580039c0 msgr2=0x7f5d6c1568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:41.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.575+0000 7f5d747e7640 1 --2- 172.21.15.40:0/2174103304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d580039c0 0x7f5d6c1568c0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009b50 tx=0x7f5d68033bd0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.575+0000 7f5d747e7640 1 -- 172.21.15.40:0/2174103304 shutdown_connections 2023-10-18T09:08:41.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.575+0000 7f5d747e7640 1 --2- 172.21.15.40:0/2174103304 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5d5006ae70 0x7f5d5006d330 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.575+0000 7f5d747e7640 1 --2- 172.21.15.40:0/2174103304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d6c154820 0x7f5d6c154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.575+0000 7f5d747e7640 1 --2- 172.21.15.40:0/2174103304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d6c15d3e0 0x7f5d6c15d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.575+0000 7f5d747e7640 1 --2- 172.21.15.40:0/2174103304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d580039c0 0x7f5d6c1568c0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.576+0000 7f5d747e7640 1 -- 172.21.15.40:0/2174103304 >> 172.21.15.40:0/2174103304 conn(0x7f5d6c06cee0 msgr2=0x7f5d6c06f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:41.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.576+0000 7f5d747e7640 1 -- 172.21.15.40:0/2174103304 shutdown_connections 2023-10-18T09:08:41.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.576+0000 7f5d747e7640 1 -- 172.21.15.40:0/2174103304 wait complete. 2023-10-18T09:08:41.600 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T09:08:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.827+0000 7fa2affff640 1 -- 172.21.15.40:0/2439896965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c006290 msgr2=0x7fa29c00ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:41.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.827+0000 7fa2affff640 1 --2- 172.21.15.40:0/2439896965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c006290 0x7fa29c00ca90 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009b50 tx=0x7fa29c006900 comp rx=0 tx=0).stop 2023-10-18T09:08:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.828+0000 7fa2affff640 1 -- 172.21.15.40:0/2439896965 shutdown_connections 2023-10-18T09:08:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.828+0000 7fa2affff640 1 --2- 172.21.15.40:0/2439896965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c006290 0x7fa29c00ca90 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.828+0000 7fa2affff640 1 -- 172.21.15.40:0/2439896965 >> 172.21.15.40:0/2439896965 conn(0x7fa2b0069720 msgr2=0x7fa2b0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:41.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.828+0000 7fa2affff640 1 -- 172.21.15.40:0/2439896965 shutdown_connections 2023-10-18T09:08:41.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.828+0000 7fa2affff640 1 -- 172.21.15.40:0/2439896965 wait complete. 2023-10-18T09:08:41.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.830+0000 7fa2affff640 1 Processor -- start 2023-10-18T09:08:41.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.830+0000 7fa2affff640 1 -- start start 2023-10-18T09:08:41.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.831+0000 7fa2affff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c006290 0x7fa2b0151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:41.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.831+0000 7fa2affff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b0156150 0x7fa2b01517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:41.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.831+0000 7fa2affff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b0151cf0 0x7fa2b00aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:41.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.831+0000 7fa2affff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b0150de0 con 0x7fa29c006290 2023-10-18T09:08:41.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.831+0000 7fa2affff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b01504d0 con 0x7fa2b0156150 2023-10-18T09:08:41.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.831+0000 7fa2affff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b00ac0c0 con 0x7fa2b0151cf0 2023-10-18T09:08:41.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.831+0000 7fa2aeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c006290 0x7fa2b0151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:41.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.831+0000 7fa2aeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c006290 0x7fa2b0151270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47150/0 (socket says 172.21.15.40:47150) 2023-10-18T09:08:41.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.831+0000 7fa2aeffd640 1 -- 172.21.15.40:0/1335471930 learned_addr learned my addr 172.21.15.40:0/1335471930 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:41.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.831+0000 7fa2ae7fc640 1 --2- 172.21.15.40:0/1335471930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b0156150 0x7fa2b01517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:41.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.831+0000 7fa2aeffd640 1 -- 172.21.15.40:0/1335471930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b0151cf0 msgr2=0x7fa2b00aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:41.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.831+0000 7fa2aeffd640 1 --2- 172.21.15.40:0/1335471930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b0151cf0 0x7fa2b00aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.831+0000 7fa2aeffd640 1 -- 172.21.15.40:0/1335471930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b0156150 msgr2=0x7fa2b01517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:41.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.831+0000 7fa2aeffd640 1 --2- 172.21.15.40:0/1335471930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b0156150 0x7fa2b01517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:41.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.831+0000 7fa2aeffd640 1 -- 172.21.15.40:0/1335471930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b0150790 con 0x7fa29c006290 2023-10-18T09:08:41.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.832+0000 7fa2aeffd640 1 --2- 172.21.15.40:0/1335471930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c006290 0x7fa2b0151270 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000ca70 tx=0x7fa2a000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:41.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.832+0000 7fa2a77fe640 1 -- 172.21.15.40:0/1335471930 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0003d80 con 0x7fa29c006290 2023-10-18T09:08:41.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.832+0000 7fa2a77fe640 1 -- 172.21.15.40:0/1335471930 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa2a0005070 con 0x7fa29c006290 2023-10-18T09:08:41.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.832+0000 7fa2affff640 1 -- 172.21.15.40:0/1335471930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b0150a70 con 0x7fa29c006290 2023-10-18T09:08:41.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.832+0000 7fa2a77fe640 1 -- 172.21.15.40:0/1335471930 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00054a0 con 0x7fa29c006290 2023-10-18T09:08:41.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.832+0000 7fa2affff640 1 -- 172.21.15.40:0/1335471930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b0152170 con 0x7fa29c006290 2023-10-18T09:08:41.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.833+0000 7fa2affff640 1 -- 172.21.15.40:0/1335471930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b0066e00 con 0x7fa29c006290 2023-10-18T09:08:41.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.835+0000 7fa2a77fe640 1 -- 172.21.15.40:0/1335471930 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa2a0005600 con 0x7fa29c006290 2023-10-18T09:08:41.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.836+0000 7fa2a77fe640 1 --2- 172.21.15.40:0/1335471930 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa29006ae40 0x7fa29006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:41.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.836+0000 7fa2a77fe640 1 -- 172.21.15.40:0/1335471930 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fa2a0089260 con 0x7fa29c006290 2023-10-18T09:08:41.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.837+0000 7fa2ae7fc640 1 --2- 172.21.15.40:0/1335471930 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa29006ae40 0x7fa29006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:41.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.838+0000 7fa2ae7fc640 1 --2- 172.21.15.40:0/1335471930 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa29006ae40 0x7fa29006d300 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0156080 tx=0x7fa2980068e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:41.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:41.841+0000 7fa2a77fe640 1 -- 172.21.15.40:0/1335471930 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2a00542e0 con 0x7fa29c006290 2023-10-18T09:08:42.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.071+0000 7fa2affff640 1 -- 172.21.15.40:0/1335471930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa2b00c95d0 con 0x7fa29c006290 2023-10-18T09:08:42.075 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:08:42.072+0000 7f6a0f710640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T09:08:42.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.076+0000 7fa2a77fe640 1 -- 172.21.15.40:0/1335471930 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa2a0057900 con 0x7fa29c006290 2023-10-18T09:08:42.080 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T09:08:42.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.080+0000 7fa2affff640 1 -- 172.21.15.40:0/1335471930 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa29006ae40 msgr2=0x7fa29006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:42.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.080+0000 7fa2affff640 1 --2- 172.21.15.40:0/1335471930 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa29006ae40 0x7fa29006d300 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0156080 tx=0x7fa2980068e0 comp rx=0 tx=0).stop 2023-10-18T09:08:42.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.080+0000 7fa2affff640 1 -- 172.21.15.40:0/1335471930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c006290 msgr2=0x7fa2b0151270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:42.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.080+0000 7fa2affff640 1 --2- 172.21.15.40:0/1335471930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c006290 0x7fa2b0151270 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000ca70 tx=0x7fa2a000cf30 comp rx=0 tx=0).stop 2023-10-18T09:08:42.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.080+0000 7fa2affff640 1 -- 172.21.15.40:0/1335471930 shutdown_connections 2023-10-18T09:08:42.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.080+0000 7fa2affff640 1 --2- 172.21.15.40:0/1335471930 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa29006ae40 0x7fa29006d300 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:42.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.080+0000 7fa2affff640 1 --2- 172.21.15.40:0/1335471930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2b0151cf0 0x7fa2b00aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:42.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.080+0000 7fa2affff640 1 --2- 172.21.15.40:0/1335471930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2b0156150 0x7fa2b01517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:42.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.080+0000 7fa2affff640 1 --2- 172.21.15.40:0/1335471930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c006290 0x7fa2b0151270 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:42.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.080+0000 7fa2affff640 1 -- 172.21.15.40:0/1335471930 >> 172.21.15.40:0/1335471930 conn(0x7fa2b0069720 msgr2=0x7fa2b014fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:42.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.080+0000 7fa2affff640 1 -- 172.21.15.40:0/1335471930 shutdown_connections 2023-10-18T09:08:42.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.080+0000 7fa2affff640 1 -- 172.21.15.40:0/1335471930 wait complete. 2023-10-18T09:08:42.104 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T09:08:42.199 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T09:08:42.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.318+0000 7fa6e48e5640 1 -- 172.21.15.40:0/1938682968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e0071240 msgr2=0x7fa6e00cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:42.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.318+0000 7fa6e48e5640 1 --2- 172.21.15.40:0/1938682968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e0071240 0x7fa6e00cc1c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8005fd0 tx=0x7fa6d802be20 comp rx=0 tx=0).stop 2023-10-18T09:08:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.318+0000 7fa6e48e5640 1 -- 172.21.15.40:0/1938682968 shutdown_connections 2023-10-18T09:08:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.318+0000 7fa6e48e5640 1 --2- 172.21.15.40:0/1938682968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e0071240 0x7fa6e00cc1c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.318+0000 7fa6e48e5640 1 --2- 172.21.15.40:0/1938682968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6e013f360 0x7fa6e0070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.319+0000 7fa6e48e5640 1 -- 172.21.15.40:0/1938682968 >> 172.21.15.40:0/1938682968 conn(0x7fa6e0069720 msgr2=0x7fa6e0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:42.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.319+0000 7fa6e48e5640 1 -- 172.21.15.40:0/1938682968 shutdown_connections 2023-10-18T09:08:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.319+0000 7fa6e48e5640 1 -- 172.21.15.40:0/1938682968 wait complete. 2023-10-18T09:08:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.320+0000 7fa6e48e5640 1 Processor -- start 2023-10-18T09:08:42.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.321+0000 7fa6e48e5640 1 -- start start 2023-10-18T09:08:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.321+0000 7fa6e48e5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6e0071240 0x7fa6e0138a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:42.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.321+0000 7fa6e48e5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6e013c550 0x7fa6e00ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:42.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.322+0000 7fa6e48e5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e013f360 0x7fa6e00abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:42.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.322+0000 7fa6e48e5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e013db60 con 0x7fa6e013c550 2023-10-18T09:08:42.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.322+0000 7fa6e48e5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e013cf50 con 0x7fa6e013f360 2023-10-18T09:08:42.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.322+0000 7fa6e48e5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e013d230 con 0x7fa6e0071240 2023-10-18T09:08:42.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.322+0000 7fa6d5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6e013c550 0x7fa6e00ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.322+0000 7fa6d5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6e013c550 0x7fa6e00ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47182/0 (socket says 172.21.15.40:47182) 2023-10-18T09:08:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.322+0000 7fa6d5d9b640 1 -- 172.21.15.40:0/2868143652 learned_addr learned my addr 172.21.15.40:0/2868143652 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.322+0000 7fa6ded9d640 1 --2- 172.21.15.40:0/2868143652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e013f360 0x7fa6e00abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:42.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.322+0000 7fa6de59c640 1 --2- 172.21.15.40:0/2868143652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6e0071240 0x7fa6e0138a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.322+0000 7fa6d5d9b640 1 -- 172.21.15.40:0/2868143652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6e0071240 msgr2=0x7fa6e0138a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.322+0000 7fa6d5d9b640 1 --2- 172.21.15.40:0/2868143652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6e0071240 0x7fa6e0138a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.322+0000 7fa6d5d9b640 1 -- 172.21.15.40:0/2868143652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e013f360 msgr2=0x7fa6e00abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.322+0000 7fa6d5d9b640 1 --2- 172.21.15.40:0/2868143652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e013f360 0x7fa6e00abe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.322+0000 7fa6d5d9b640 1 -- 172.21.15.40:0/2868143652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c8009490 con 0x7fa6e013c550 2023-10-18T09:08:42.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.323+0000 7fa6d5d9b640 1 --2- 172.21.15.40:0/2868143652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6e013c550 0x7fa6e00ab8d0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800e910 tx=0x7fa6c800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.323+0000 7fa6d77fe640 1 -- 172.21.15.40:0/2868143652 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c800cd80 con 0x7fa6e013c550 2023-10-18T09:08:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.323+0000 7fa6d77fe640 1 -- 172.21.15.40:0/2868143652 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa6c8010430 con 0x7fa6e013c550 2023-10-18T09:08:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.323+0000 7fa6d77fe640 1 -- 172.21.15.40:0/2868143652 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8010820 con 0x7fa6e013c550 2023-10-18T09:08:42.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.323+0000 7fa6e48e5640 1 -- 172.21.15.40:0/2868143652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d8009560 con 0x7fa6e013c550 2023-10-18T09:08:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.323+0000 7fa6e48e5640 1 -- 172.21.15.40:0/2868143652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e013d7f0 con 0x7fa6e013c550 2023-10-18T09:08:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.325+0000 7fa6e48e5640 1 -- 172.21.15.40:0/2868143652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e013dd10 con 0x7fa6e013c550 2023-10-18T09:08:42.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.328+0000 7fa6d77fe640 1 -- 172.21.15.40:0/2868143652 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa6c8010980 con 0x7fa6e013c550 2023-10-18T09:08:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.329+0000 7fa6d77fe640 1 --2- 172.21.15.40:0/2868143652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa6c006ae70 0x7fa6c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:42.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.329+0000 7fa6d77fe640 1 -- 172.21.15.40:0/2868143652 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fa6c8014070 con 0x7fa6e013c550 2023-10-18T09:08:42.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.329+0000 7fa6de59c640 1 --2- 172.21.15.40:0/2868143652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa6c006ae70 0x7fa6c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:42.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.331+0000 7fa6de59c640 1 --2- 172.21.15.40:0/2868143652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa6c006ae70 0x7fa6c006d330 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00012e0 tx=0x7fa6d0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:42.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.333+0000 7fa6d77fe640 1 -- 172.21.15.40:0/2868143652 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa6c8055380 con 0x7fa6e013c550 2023-10-18T09:08:42.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:42.563+0000 7fa6e48e5640 1 -- 172.21.15.40:0/2868143652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa6e0066fd0 con 0x7fa6e013c550 2023-10-18T09:08:43.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.080+0000 7fa6d77fe640 1 -- 172.21.15.40:0/2868143652 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v84) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa6c80589a0 con 0x7fa6e013c550 2023-10-18T09:08:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.084+0000 7fa6e48e5640 1 -- 172.21.15.40:0/2868143652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa6c006ae70 msgr2=0x7fa6c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.084+0000 7fa6e48e5640 1 --2- 172.21.15.40:0/2868143652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa6c006ae70 0x7fa6c006d330 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00012e0 tx=0x7fa6d0009040 comp rx=0 tx=0).stop 2023-10-18T09:08:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.084+0000 7fa6e48e5640 1 -- 172.21.15.40:0/2868143652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6e013c550 msgr2=0x7fa6e00ab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:43.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.084+0000 7fa6e48e5640 1 --2- 172.21.15.40:0/2868143652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6e013c550 0x7fa6e00ab8d0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800e910 tx=0x7fa6c800edd0 comp rx=0 tx=0).stop 2023-10-18T09:08:43.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.084+0000 7fa6e48e5640 1 -- 172.21.15.40:0/2868143652 shutdown_connections 2023-10-18T09:08:43.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.084+0000 7fa6e48e5640 1 --2- 172.21.15.40:0/2868143652 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa6c006ae70 0x7fa6c006d330 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:43.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.084+0000 7fa6e48e5640 1 --2- 172.21.15.40:0/2868143652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e013f360 0x7fa6e00abe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:43.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.084+0000 7fa6e48e5640 1 --2- 172.21.15.40:0/2868143652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6e013c550 0x7fa6e00ab8d0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:43.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.084+0000 7fa6e48e5640 1 --2- 172.21.15.40:0/2868143652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa6e0071240 0x7fa6e0138a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:43.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.084+0000 7fa6e48e5640 1 -- 172.21.15.40:0/2868143652 >> 172.21.15.40:0/2868143652 conn(0x7fa6e0069720 msgr2=0x7fa6e0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:43.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.085+0000 7fa6e48e5640 1 -- 172.21.15.40:0/2868143652 shutdown_connections 2023-10-18T09:08:43.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.085+0000 7fa6e48e5640 1 -- 172.21.15.40:0/2868143652 wait complete. 2023-10-18T09:08:43.107 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T09:08:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.333+0000 7fe72f59e640 1 -- 172.21.15.40:0/3884256662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7300cc950 msgr2=0x7fe7300b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:43.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.333+0000 7fe72f59e640 1 --2- 172.21.15.40:0/3884256662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7300cc950 0x7fe7300b1780 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fe73006a9c0 tx=0x7fe72802be30 comp rx=0 tx=0).stop 2023-10-18T09:08:43.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.334+0000 7fe72f59e640 1 -- 172.21.15.40:0/3884256662 shutdown_connections 2023-10-18T09:08:43.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.334+0000 7fe72f59e640 1 --2- 172.21.15.40:0/3884256662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7300cc950 0x7fe7300b1780 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:43.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.334+0000 7fe72f59e640 1 --2- 172.21.15.40:0/3884256662 >> v2:172.21.15.144:3300/0 conn(0x7fe73014e350 0x7fe73006b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:43.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.334+0000 7fe72f59e640 1 -- 172.21.15.40:0/3884256662 >> 172.21.15.40:0/3884256662 conn(0x7fe7300aeb90 msgr2=0x7fe7300aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:43.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.334+0000 7fe72f59e640 1 -- 172.21.15.40:0/3884256662 shutdown_connections 2023-10-18T09:08:43.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.334+0000 7fe72f59e640 1 -- 172.21.15.40:0/3884256662 wait complete. 2023-10-18T09:08:43.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.336+0000 7fe72f59e640 1 Processor -- start 2023-10-18T09:08:43.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.336+0000 7fe72f59e640 1 -- start start 2023-10-18T09:08:43.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.337+0000 7fe72f59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe73006fbc0 0x7fe730140500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:43.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.337+0000 7fe72f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7300cc950 0x7fe7300cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:43.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.337+0000 7fe72f59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe73014e350 0x7fe73013e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:43.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.337+0000 7fe72f59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7301401b0 con 0x7fe7300cc950 2023-10-18T09:08:43.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.337+0000 7fe72f59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73014d710 con 0x7fe73014e350 2023-10-18T09:08:43.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.337+0000 7fe72f59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73013fba0 con 0x7fe73006fbc0 2023-10-18T09:08:43.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.338+0000 7fe72dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7300cc950 0x7fe7300cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:43.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.338+0000 7fe72dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7300cc950 0x7fe7300cb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47202/0 (socket says 172.21.15.40:47202) 2023-10-18T09:08:43.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.338+0000 7fe72dd9b640 1 -- 172.21.15.40:0/1098331396 learned_addr learned my addr 172.21.15.40:0/1098331396 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:43.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.338+0000 7fe72e59c640 1 --2- 172.21.15.40:0/1098331396 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe73006fbc0 0x7fe730140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:43.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.338+0000 7fe72dd9b640 1 -- 172.21.15.40:0/1098331396 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe73006fbc0 msgr2=0x7fe730140500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:43.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.338+0000 7fe72ed9d640 1 --2- 172.21.15.40:0/1098331396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe73014e350 0x7fe73013e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:43.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.338+0000 7fe72dd9b640 1 --2- 172.21.15.40:0/1098331396 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe73006fbc0 0x7fe730140500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:43.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.338+0000 7fe72dd9b640 1 -- 172.21.15.40:0/1098331396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe73014e350 msgr2=0x7fe73013e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:43.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.338+0000 7fe72dd9b640 1 --2- 172.21.15.40:0/1098331396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe73014e350 0x7fe73013e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:43.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.338+0000 7fe72dd9b640 1 -- 172.21.15.40:0/1098331396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe728009560 con 0x7fe7300cc950 2023-10-18T09:08:43.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.339+0000 7fe72dd9b640 1 --2- 172.21.15.40:0/1098331396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7300cc950 0x7fe7300cb6f0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fe71800c870 tx=0x7fe71800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:43.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.339+0000 7fe726ffd640 1 -- 172.21.15.40:0/1098331396 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718003b70 con 0x7fe7300cc950 2023-10-18T09:08:43.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.339+0000 7fe726ffd640 1 -- 172.21.15.40:0/1098331396 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe718003cf0 con 0x7fe7300cc950 2023-10-18T09:08:43.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.339+0000 7fe726ffd640 1 -- 172.21.15.40:0/1098331396 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718003ea0 con 0x7fe7300cc950 2023-10-18T09:08:43.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.339+0000 7fe72f59e640 1 -- 172.21.15.40:0/1098331396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73014d430 con 0x7fe7300cc950 2023-10-18T09:08:43.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.339+0000 7fe72f59e640 1 -- 172.21.15.40:0/1098331396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73014cbb0 con 0x7fe7300cc950 2023-10-18T09:08:43.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.340+0000 7fe72f59e640 1 -- 172.21.15.40:0/1098331396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe730140360 con 0x7fe7300cc950 2023-10-18T09:08:43.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.341+0000 7fe726ffd640 1 -- 172.21.15.40:0/1098331396 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe71800aab0 con 0x7fe7300cc950 2023-10-18T09:08:43.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.342+0000 7fe726ffd640 1 --2- 172.21.15.40:0/1098331396 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe71006ae40 0x7fe71006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:43.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.343+0000 7fe726ffd640 1 -- 172.21.15.40:0/1098331396 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe718089720 con 0x7fe7300cc950 2023-10-18T09:08:43.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.343+0000 7fe72e59c640 1 --2- 172.21.15.40:0/1098331396 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe71006ae40 0x7fe71006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:43.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.344+0000 7fe72e59c640 1 --2- 172.21.15.40:0/1098331396 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe71006ae40 0x7fe71006d300 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe72000b9a0 tx=0x7fe720005c20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:43.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.347+0000 7fe726ffd640 1 -- 172.21.15.40:0/1098331396 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe730140360 con 0x7fe7300cc950 2023-10-18T09:08:43.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:43.531+0000 7fe72f59e640 1 -- 172.21.15.40:0/1098331396 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe7300af1c0 con 0x7fe7300cc950 2023-10-18T09:08:44.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.291+0000 7fe726ffd640 1 -- 172.21.15.40:0/1098331396 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v186) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe718054770 con 0x7fe7300cc950 2023-10-18T09:08:44.295 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-10-18T09:08:44.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.296+0000 7fe72f59e640 1 -- 172.21.15.40:0/1098331396 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe71006ae40 msgr2=0x7fe71006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:44.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.296+0000 7fe72f59e640 1 --2- 172.21.15.40:0/1098331396 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe71006ae40 0x7fe71006d300 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe72000b9a0 tx=0x7fe720005c20 comp rx=0 tx=0).stop 2023-10-18T09:08:44.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.296+0000 7fe72f59e640 1 -- 172.21.15.40:0/1098331396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7300cc950 msgr2=0x7fe7300cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:44.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.296+0000 7fe72f59e640 1 --2- 172.21.15.40:0/1098331396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7300cc950 0x7fe7300cb6f0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fe71800c870 tx=0x7fe71800cd30 comp rx=0 tx=0).stop 2023-10-18T09:08:44.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.296+0000 7fe72f59e640 1 -- 172.21.15.40:0/1098331396 shutdown_connections 2023-10-18T09:08:44.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.296+0000 7fe72f59e640 1 --2- 172.21.15.40:0/1098331396 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe71006ae40 0x7fe71006d300 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:44.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.296+0000 7fe72f59e640 1 --2- 172.21.15.40:0/1098331396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe73014e350 0x7fe73013e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:44.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.296+0000 7fe72f59e640 1 --2- 172.21.15.40:0/1098331396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7300cc950 0x7fe7300cb6f0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:44.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.296+0000 7fe72f59e640 1 --2- 172.21.15.40:0/1098331396 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe73006fbc0 0x7fe730140500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:44.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.296+0000 7fe72f59e640 1 -- 172.21.15.40:0/1098331396 >> 172.21.15.40:0/1098331396 conn(0x7fe7300aeb90 msgr2=0x7fe730072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:44.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.297+0000 7fe72f59e640 1 -- 172.21.15.40:0/1098331396 shutdown_connections 2023-10-18T09:08:44.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.298+0000 7fe72f59e640 1 -- 172.21.15.40:0/1098331396 wait complete. 2023-10-18T09:08:44.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T09:08:44.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.548+0000 7f93589cf640 1 -- 172.21.15.40:0/632048201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f935415b920 msgr2=0x7f935415bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:44.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.548+0000 7f93589cf640 1 --2- 172.21.15.40:0/632048201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f935415b920 0x7f935415bd70 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f934c005fd0 tx=0x7f934c02c120 comp rx=0 tx=0).stop 2023-10-18T09:08:44.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.548+0000 7f93589cf640 1 -- 172.21.15.40:0/632048201 shutdown_connections 2023-10-18T09:08:44.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.548+0000 7f93589cf640 1 --2- 172.21.15.40:0/632048201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f935415b920 0x7f935415bd70 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:44.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.548+0000 7f93589cf640 1 --2- 172.21.15.40:0/632048201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9354070470 0x7f9354070920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.548+0000 7f93589cf640 1 -- 172.21.15.40:0/632048201 >> 172.21.15.40:0/632048201 conn(0x7f93540aeb90 msgr2=0x7f93540aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.548+0000 7f93589cf640 1 -- 172.21.15.40:0/632048201 shutdown_connections 2023-10-18T09:08:44.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.548+0000 7f93589cf640 1 -- 172.21.15.40:0/632048201 wait complete. 2023-10-18T09:08:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.550+0000 7f93589cf640 1 Processor -- start 2023-10-18T09:08:44.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.550+0000 7f93589cf640 1 -- start start 2023-10-18T09:08:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.551+0000 7f93589cf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f935406d290 0x7f93540c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:44.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.551+0000 7f93589cf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9354070470 0x7f93540c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.551+0000 7f93589cf640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f935415b920 0x7f9354148720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.551+0000 7f93589cf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93540ca7f0 con 0x7f935406d290 2023-10-18T09:08:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.551+0000 7f93589cf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93540caad0 con 0x7f9354070470 2023-10-18T09:08:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.551+0000 7f93589cf640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93540cadb0 con 0x7f935415b920 2023-10-18T09:08:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.551+0000 7f9352d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f935406d290 0x7f93540c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:44.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.551+0000 7f9352d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f935406d290 0x7f93540c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47218/0 (socket says 172.21.15.40:47218) 2023-10-18T09:08:44.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.551+0000 7f9352d9d640 1 -- 172.21.15.40:0/3501319796 learned_addr learned my addr 172.21.15.40:0/3501319796 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:44.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.552+0000 7f9352d9d640 1 -- 172.21.15.40:0/3501319796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f935415b920 msgr2=0x7f9354148720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:44.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.552+0000 7f9352d9d640 1 --2- 172.21.15.40:0/3501319796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f935415b920 0x7f9354148720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:44.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.552+0000 7f9352d9d640 1 -- 172.21.15.40:0/3501319796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9354070470 msgr2=0x7f93540c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:44.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.552+0000 7f9352d9d640 1 --2- 172.21.15.40:0/3501319796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9354070470 0x7f93540c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:44.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.552+0000 7f9352d9d640 1 -- 172.21.15.40:0/3501319796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c009900 con 0x7f935406d290 2023-10-18T09:08:44.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.552+0000 7f9352d9d640 1 --2- 172.21.15.40:0/3501319796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f935406d290 0x7f93540c8690 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f934400c430 tx=0x7f934400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:44.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.552+0000 7f934b7fe640 1 -- 172.21.15.40:0/3501319796 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344011790 con 0x7f935406d290 2023-10-18T09:08:44.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.552+0000 7f934b7fe640 1 -- 172.21.15.40:0/3501319796 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f93440119e0 con 0x7f935406d290 2023-10-18T09:08:44.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.552+0000 7f934b7fe640 1 -- 172.21.15.40:0/3501319796 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9344011e30 con 0x7f935406d290 2023-10-18T09:08:44.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.552+0000 7f93589cf640 1 -- 172.21.15.40:0/3501319796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93540cb090 con 0x7f935406d290 2023-10-18T09:08:44.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.553+0000 7f93589cf640 1 -- 172.21.15.40:0/3501319796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93540ca410 con 0x7f935406d290 2023-10-18T09:08:44.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.554+0000 7f93589cf640 1 -- 172.21.15.40:0/3501319796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93540cb5b0 con 0x7f935406d290 2023-10-18T09:08:44.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.555+0000 7f934b7fe640 1 -- 172.21.15.40:0/3501319796 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f934401c460 con 0x7f935406d290 2023-10-18T09:08:44.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.556+0000 7f934b7fe640 1 --2- 172.21.15.40:0/3501319796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f933406ae70 0x7f933406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:44.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.556+0000 7f934b7fe640 1 -- 172.21.15.40:0/3501319796 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f9344089780 con 0x7f935406d290 2023-10-18T09:08:44.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.556+0000 7f934bfff640 1 --2- 172.21.15.40:0/3501319796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f933406ae70 0x7f933406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:44.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.557+0000 7f934bfff640 1 --2- 172.21.15.40:0/3501319796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f933406ae70 0x7f933406d330 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f935415df90 tx=0x7f933c005bf0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:44.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.561+0000 7f934b7fe640 1 -- 172.21.15.40:0/3501319796 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93540cb5b0 con 0x7f935406d290 2023-10-18T09:08:44.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:44.745+0000 7f93589cf640 1 -- 172.21.15.40:0/3501319796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f93540cb5b0 con 0x7f935406d290 2023-10-18T09:08:45.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.305+0000 7f934b7fe640 1 -- 172.21.15.40:0/3501319796 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v187) v1 ==== 169+0+0 (secure 0 0 0) 0x7f93540cb5b0 con 0x7f935406d290 2023-10-18T09:08:45.308 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-10-18T09:08:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.309+0000 7f93589cf640 1 -- 172.21.15.40:0/3501319796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f933406ae70 msgr2=0x7f933406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.309+0000 7f93589cf640 1 --2- 172.21.15.40:0/3501319796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f933406ae70 0x7f933406d330 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f935415df90 tx=0x7f933c005bf0 comp rx=0 tx=0).stop 2023-10-18T09:08:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.309+0000 7f93589cf640 1 -- 172.21.15.40:0/3501319796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f935406d290 msgr2=0x7f93540c8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.309+0000 7f93589cf640 1 --2- 172.21.15.40:0/3501319796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f935406d290 0x7f93540c8690 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f934400c430 tx=0x7f934400c8f0 comp rx=0 tx=0).stop 2023-10-18T09:08:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.309+0000 7f93589cf640 1 -- 172.21.15.40:0/3501319796 shutdown_connections 2023-10-18T09:08:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.309+0000 7f93589cf640 1 --2- 172.21.15.40:0/3501319796 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f933406ae70 0x7f933406d330 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:45.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.309+0000 7f93589cf640 1 --2- 172.21.15.40:0/3501319796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f935415b920 0x7f9354148720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:45.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.309+0000 7f93589cf640 1 --2- 172.21.15.40:0/3501319796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9354070470 0x7f93540c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:45.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.309+0000 7f93589cf640 1 --2- 172.21.15.40:0/3501319796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f935406d290 0x7f93540c8690 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:45.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.309+0000 7f93589cf640 1 -- 172.21.15.40:0/3501319796 >> 172.21.15.40:0/3501319796 conn(0x7f93540aeb90 msgr2=0x7f9354159440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:45.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.312+0000 7f93589cf640 1 -- 172.21.15.40:0/3501319796 shutdown_connections 2023-10-18T09:08:45.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.312+0000 7f93589cf640 1 -- 172.21.15.40:0/3501319796 wait complete. 2023-10-18T09:08:45.338 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 22 2023-10-18T09:08:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.561+0000 7f5359728640 1 -- 172.21.15.40:0/2338535807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53540b0630 msgr2=0x7f53540c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.561+0000 7f5359728640 1 --2- 172.21.15.40:0/2338535807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53540b0630 0x7f53540c99c0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f535406aeb0 tx=0x7f533c02bd70 comp rx=0 tx=0).stop 2023-10-18T09:08:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.561+0000 7f5359728640 1 -- 172.21.15.40:0/2338535807 shutdown_connections 2023-10-18T09:08:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.561+0000 7f5359728640 1 --2- 172.21.15.40:0/2338535807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53540b0630 0x7f53540c99c0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:45.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.561+0000 7f5359728640 1 --2- 172.21.15.40:0/2338535807 >> v2:172.21.15.144:3300/0 conn(0x7f535406d6f0 0x7f53540b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:45.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.561+0000 7f5359728640 1 -- 172.21.15.40:0/2338535807 >> 172.21.15.40:0/2338535807 conn(0x7f5354069720 msgr2=0x7f5354069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:45.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.561+0000 7f5359728640 1 -- 172.21.15.40:0/2338535807 shutdown_connections 2023-10-18T09:08:45.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.561+0000 7f5359728640 1 -- 172.21.15.40:0/2338535807 wait complete. 2023-10-18T09:08:45.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.563+0000 7f5359728640 1 Processor -- start 2023-10-18T09:08:45.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.563+0000 7f5359728640 1 -- start start 2023-10-18T09:08:45.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.564+0000 7f5359728640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f535406d6f0 0x7f5354141c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:45.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.564+0000 7f5359728640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53540b0630 0x7f53540a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:45.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.564+0000 7f5359728640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53540c9f00 0x7f53540af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:45.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.564+0000 7f5359728640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53541f3990 con 0x7f535406d6f0 2023-10-18T09:08:45.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.564+0000 7f5359728640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53540a69b0 con 0x7f53540c9f00 2023-10-18T09:08:45.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.564+0000 7f5359728640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53541f2d10 con 0x7f53540b0630 2023-10-18T09:08:45.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.564+0000 7f5353fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f535406d6f0 0x7f5354141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.564+0000 7f5353fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f535406d6f0 0x7f5354141c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50830/0 (socket says 172.21.15.40:50830) 2023-10-18T09:08:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.564+0000 7f5353fff640 1 -- 172.21.15.40:0/2961863373 learned_addr learned my addr 172.21.15.40:0/2961863373 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.564+0000 7f5353fff640 1 -- 172.21.15.40:0/2961863373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53540b0630 msgr2=0x7f53540a6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.564+0000 7f5353fff640 1 --2- 172.21.15.40:0/2961863373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53540b0630 0x7f53540a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:45.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.564+0000 7f5353fff640 1 -- 172.21.15.40:0/2961863373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53540c9f00 msgr2=0x7f53540af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:45.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.564+0000 7f5358f27640 1 --2- 172.21.15.40:0/2961863373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53540c9f00 0x7f53540af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:45.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.565+0000 7f5353fff640 1 --2- 172.21.15.40:0/2961863373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53540c9f00 0x7f53540af890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:45.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.565+0000 7f5353fff640 1 -- 172.21.15.40:0/2961863373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c009490 con 0x7f535406d6f0 2023-10-18T09:08:45.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.565+0000 7f5353fff640 1 --2- 172.21.15.40:0/2961863373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f535406d6f0 0x7f5354141c30 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f534400ddc0 tx=0x7f534400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:45.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.565+0000 7f53517fa640 1 -- 172.21.15.40:0/2961863373 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534400e650 con 0x7f535406d6f0 2023-10-18T09:08:45.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.565+0000 7f53517fa640 1 -- 172.21.15.40:0/2961863373 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f534400e8a0 con 0x7f535406d6f0 2023-10-18T09:08:45.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.565+0000 7f53517fa640 1 -- 172.21.15.40:0/2961863373 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534400ecb0 con 0x7f535406d6f0 2023-10-18T09:08:45.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.565+0000 7f5359728640 1 -- 172.21.15.40:0/2961863373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53541f2ff0 con 0x7f535406d6f0 2023-10-18T09:08:45.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.565+0000 7f5359728640 1 -- 172.21.15.40:0/2961863373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53541f35b0 con 0x7f535406d6f0 2023-10-18T09:08:45.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.567+0000 7f5359728640 1 -- 172.21.15.40:0/2961863373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5354066e00 con 0x7f535406d6f0 2023-10-18T09:08:45.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.568+0000 7f53517fa640 1 -- 172.21.15.40:0/2961863373 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5344016440 con 0x7f535406d6f0 2023-10-18T09:08:45.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.569+0000 7f53517fa640 1 --2- 172.21.15.40:0/2961863373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f533406ae40 0x7f533406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:45.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.569+0000 7f53517fa640 1 -- 172.21.15.40:0/2961863373 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5344089b80 con 0x7f535406d6f0 2023-10-18T09:08:45.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.569+0000 7f534bfff640 1 --2- 172.21.15.40:0/2961863373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f533406ae40 0x7f533406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:45.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.571+0000 7f534bfff640 1 --2- 172.21.15.40:0/2961863373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f533406ae40 0x7f533406d300 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f533c036000 tx=0x7f533c0046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:45.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.573+0000 7f53517fa640 1 -- 172.21.15.40:0/2961863373 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5344054fe0 con 0x7f535406d6f0 2023-10-18T09:08:45.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.770+0000 7f5359728640 1 -- 172.21.15.40:0/2961863373 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config reset", "num": 22} v 0) v1 -- 0x7f53540cc110 con 0x7f535406d6f0 2023-10-18T09:08:45.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.778+0000 7f53517fa640 1 -- 172.21.15.40:0/2961863373 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 22}]=0 v40) v1 ==== 71+0+0 (secure 0 0 0) 0x7f534401b020 con 0x7f535406d6f0 2023-10-18T09:08:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.781+0000 7f5359728640 1 -- 172.21.15.40:0/2961863373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f533406ae40 msgr2=0x7f533406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.781+0000 7f5359728640 1 --2- 172.21.15.40:0/2961863373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f533406ae40 0x7f533406d300 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f533c036000 tx=0x7f533c0046c0 comp rx=0 tx=0).stop 2023-10-18T09:08:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.781+0000 7f5359728640 1 -- 172.21.15.40:0/2961863373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f535406d6f0 msgr2=0x7f5354141c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.781+0000 7f5359728640 1 --2- 172.21.15.40:0/2961863373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f535406d6f0 0x7f5354141c30 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f534400ddc0 tx=0x7f534400a620 comp rx=0 tx=0).stop 2023-10-18T09:08:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.781+0000 7f5359728640 1 -- 172.21.15.40:0/2961863373 shutdown_connections 2023-10-18T09:08:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.781+0000 7f5359728640 1 --2- 172.21.15.40:0/2961863373 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f533406ae40 0x7f533406d300 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.781+0000 7f5359728640 1 --2- 172.21.15.40:0/2961863373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f53540c9f00 0x7f53540af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.781+0000 7f5359728640 1 --2- 172.21.15.40:0/2961863373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f53540b0630 0x7f53540a6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.781+0000 7f5359728640 1 --2- 172.21.15.40:0/2961863373 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f535406d6f0 0x7f5354141c30 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.781+0000 7f5359728640 1 -- 172.21.15.40:0/2961863373 >> 172.21.15.40:0/2961863373 conn(0x7f5354069720 msgr2=0x7f535414d730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.781+0000 7f5359728640 1 -- 172.21.15.40:0/2961863373 shutdown_connections 2023-10-18T09:08:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:45.782+0000 7f5359728640 1 -- 172.21.15.40:0/2961863373 wait complete. 2023-10-18T09:08:45.804 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2023-10-18T09:08:46.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.021+0000 7f5f23a10640 1 -- 172.21.15.40:0/3916563144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1c15b760 msgr2=0x7f5f1c0cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:46.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.021+0000 7f5f23a10640 1 --2- 172.21.15.40:0/3916563144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1c15b760 0x7f5f1c0cfbc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009ca0 tx=0x7f5f1802c360 comp rx=0 tx=0).stop 2023-10-18T09:08:46.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.022+0000 7f5f23a10640 1 -- 172.21.15.40:0/3916563144 shutdown_connections 2023-10-18T09:08:46.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.022+0000 7f5f23a10640 1 --2- 172.21.15.40:0/3916563144 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f1c0d0100 0x7f5f1c0d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:46.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.022+0000 7f5f23a10640 1 --2- 172.21.15.40:0/3916563144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1c15b760 0x7f5f1c0cfbc0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:46.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.022+0000 7f5f23a10640 1 -- 172.21.15.40:0/3916563144 >> 172.21.15.40:0/3916563144 conn(0x7f5f1c09a9a0 msgr2=0x7f5f1c088fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:46.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.022+0000 7f5f23a10640 1 -- 172.21.15.40:0/3916563144 shutdown_connections 2023-10-18T09:08:46.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.022+0000 7f5f23a10640 1 -- 172.21.15.40:0/3916563144 wait complete. 2023-10-18T09:08:46.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.024+0000 7f5f23a10640 1 Processor -- start 2023-10-18T09:08:46.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.024+0000 7f5f23a10640 1 -- start start 2023-10-18T09:08:46.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.025+0000 7f5f23a10640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1c0d0100 0x7f5f1c153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:46.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.025+0000 7f5f23a10640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f1c0d19c0 0x7f5f1c0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:46.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.025+0000 7f5f23a10640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1c15b760 0x7f5f1c0afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:46.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.025+0000 7f5f23a10640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c1e1900 con 0x7f5f1c15b760 2023-10-18T09:08:46.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.025+0000 7f5f23a10640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c153740 con 0x7f5f1c0d0100 2023-10-18T09:08:46.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.025+0000 7f5f23a10640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c1e0a30 con 0x7f5f1c0d19c0 2023-10-18T09:08:46.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.025+0000 7f5f21fad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1c15b760 0x7f5f1c0afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:46.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.025+0000 7f5f21fad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1c15b760 0x7f5f1c0afc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50840/0 (socket says 172.21.15.40:50840) 2023-10-18T09:08:46.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.025+0000 7f5f21fad640 1 -- 172.21.15.40:0/112735068 learned_addr learned my addr 172.21.15.40:0/112735068 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:46.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.025+0000 7f5f20fab640 1 --2- 172.21.15.40:0/112735068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f1c0d19c0 0x7f5f1c0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:46.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.025+0000 7f5f217ac640 1 --2- 172.21.15.40:0/112735068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1c0d0100 0x7f5f1c153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:46.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.026+0000 7f5f21fad640 1 -- 172.21.15.40:0/112735068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f1c0d19c0 msgr2=0x7f5f1c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:46.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.026+0000 7f5f21fad640 1 --2- 172.21.15.40:0/112735068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f1c0d19c0 0x7f5f1c0af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:46.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.026+0000 7f5f21fad640 1 -- 172.21.15.40:0/112735068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1c0d0100 msgr2=0x7f5f1c153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:46.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.026+0000 7f5f21fad640 1 --2- 172.21.15.40:0/112735068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1c0d0100 0x7f5f1c153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:46.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.026+0000 7f5f21fad640 1 -- 172.21.15.40:0/112735068 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f18009900 con 0x7f5f1c15b760 2023-10-18T09:08:46.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.026+0000 7f5f217ac640 1 --2- 172.21.15.40:0/112735068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1c0d0100 0x7f5f1c153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:46.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.026+0000 7f5f21fad640 1 --2- 172.21.15.40:0/112735068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1c15b760 0x7f5f1c0afc20 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400c3c0 tx=0x7f5f1400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:46.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.026+0000 7f5f11ffb640 1 -- 172.21.15.40:0/112735068 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f140052d0 con 0x7f5f1c15b760 2023-10-18T09:08:46.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.026+0000 7f5f11ffb640 1 -- 172.21.15.40:0/112735068 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5f14007020 con 0x7f5f1c15b760 2023-10-18T09:08:46.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.026+0000 7f5f11ffb640 1 -- 172.21.15.40:0/112735068 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14007430 con 0x7f5f1c15b760 2023-10-18T09:08:46.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.026+0000 7f5f23a10640 1 -- 172.21.15.40:0/112735068 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c1e0c80 con 0x7f5f1c15b760 2023-10-18T09:08:46.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.026+0000 7f5f23a10640 1 -- 172.21.15.40:0/112735068 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c1e1240 con 0x7f5f1c15b760 2023-10-18T09:08:46.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.028+0000 7f5f23a10640 1 -- 172.21.15.40:0/112735068 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c1e1a40 con 0x7f5f1c15b760 2023-10-18T09:08:46.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.029+0000 7f5f11ffb640 1 -- 172.21.15.40:0/112735068 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5f140067d0 con 0x7f5f1c15b760 2023-10-18T09:08:46.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.030+0000 7f5f11ffb640 1 --2- 172.21.15.40:0/112735068 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f0006ae40 0x7f5f0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:46.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.030+0000 7f5f11ffb640 1 -- 172.21.15.40:0/112735068 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5f14088840 con 0x7f5f1c15b760 2023-10-18T09:08:46.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.031+0000 7f5f217ac640 1 --2- 172.21.15.40:0/112735068 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f0006ae40 0x7f5f0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:46.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.032+0000 7f5f217ac640 1 --2- 172.21.15.40:0/112735068 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f0006ae40 0x7f5f0006d300 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5f18036000 tx=0x7f5f18004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:46.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.034+0000 7f5f11ffb640 1 -- 172.21.15.40:0/112735068 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f1c1e1a40 con 0x7f5f1c15b760 2023-10-18T09:08:46.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:46.307+0000 7f5f23a10640 1 -- 172.21.15.40:0/112735068 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7f5f1c1e1a40 con 0x7f5f1c15b760 2023-10-18T09:08:47.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.311+0000 7f5f11ffb640 1 -- 172.21.15.40:0/112735068 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v650) v1 ==== 131+0+0 (secure 0 0 0) 0x7f5f1c1e1a40 con 0x7f5f1c15b760 2023-10-18T09:08:47.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.314+0000 7f5f23a10640 1 -- 172.21.15.40:0/112735068 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f0006ae40 msgr2=0x7f5f0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:47.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.315+0000 7f5f23a10640 1 --2- 172.21.15.40:0/112735068 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f0006ae40 0x7f5f0006d300 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5f18036000 tx=0x7f5f18004ab0 comp rx=0 tx=0).stop 2023-10-18T09:08:47.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.315+0000 7f5f23a10640 1 -- 172.21.15.40:0/112735068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1c15b760 msgr2=0x7f5f1c0afc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:47.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.315+0000 7f5f23a10640 1 --2- 172.21.15.40:0/112735068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1c15b760 0x7f5f1c0afc20 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400c3c0 tx=0x7f5f1400c880 comp rx=0 tx=0).stop 2023-10-18T09:08:47.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.315+0000 7f5f23a10640 1 -- 172.21.15.40:0/112735068 shutdown_connections 2023-10-18T09:08:47.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.315+0000 7f5f23a10640 1 --2- 172.21.15.40:0/112735068 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5f0006ae40 0x7f5f0006d300 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:47.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.315+0000 7f5f23a10640 1 --2- 172.21.15.40:0/112735068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f1c15b760 0x7f5f1c0afc20 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:47.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.315+0000 7f5f23a10640 1 --2- 172.21.15.40:0/112735068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f1c0d19c0 0x7f5f1c0af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:47.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.315+0000 7f5f23a10640 1 --2- 172.21.15.40:0/112735068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1c0d0100 0x7f5f1c153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:47.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.315+0000 7f5f23a10640 1 -- 172.21.15.40:0/112735068 >> 172.21.15.40:0/112735068 conn(0x7f5f1c09a9a0 msgr2=0x7f5f1c0b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:47.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.315+0000 7f5f23a10640 1 -- 172.21.15.40:0/112735068 shutdown_connections 2023-10-18T09:08:47.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.315+0000 7f5f23a10640 1 -- 172.21.15.40:0/112735068 wait complete. 2023-10-18T09:08:47.340 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2023-10-18T09:08:47.340 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) 2023-10-18T09:08:47.341 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:08:47.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.574+0000 7f74ae5b6640 1 -- 172.21.15.40:0/1846332025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a815a2e0 msgr2=0x7f74a80cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:47.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.574+0000 7f74ae5b6640 1 --2- 172.21.15.40:0/1846332025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a815a2e0 0x7f74a80cbc20 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f74a80cb4b0 tx=0x7f749802bd70 comp rx=0 tx=0).stop 2023-10-18T09:08:47.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.574+0000 7f74ae5b6640 1 -- 172.21.15.40:0/1846332025 shutdown_connections 2023-10-18T09:08:47.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.574+0000 7f74ae5b6640 1 --2- 172.21.15.40:0/1846332025 >> v2:172.21.15.144:3300/0 conn(0x7f74a815a6c0 0x7f74a806b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:47.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.574+0000 7f74ae5b6640 1 --2- 172.21.15.40:0/1846332025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a815a2e0 0x7f74a80cbc20 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:47.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.574+0000 7f74ae5b6640 1 -- 172.21.15.40:0/1846332025 >> 172.21.15.40:0/1846332025 conn(0x7f74a806cee0 msgr2=0x7f74a8069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:47.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.574+0000 7f74ae5b6640 1 -- 172.21.15.40:0/1846332025 shutdown_connections 2023-10-18T09:08:47.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.574+0000 7f74ae5b6640 1 -- 172.21.15.40:0/1846332025 wait complete. 2023-10-18T09:08:47.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.576+0000 7f74ae5b6640 1 Processor -- start 2023-10-18T09:08:47.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.576+0000 7f74ae5b6640 1 -- start start 2023-10-18T09:08:47.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.577+0000 7f74ae5b6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a815a2e0 0x7f74a813f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:47.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.577+0000 7f74ae5b6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a815a6c0 0x7f74a8142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:47.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.577+0000 7f74ae5b6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a81590e0 0x7f74a8151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:47.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.577+0000 7f74ae5b6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a813f770 con 0x7f74a81590e0 2023-10-18T09:08:47.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.577+0000 7f74ae5b6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a81f3040 con 0x7f74a815a6c0 2023-10-18T09:08:47.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.577+0000 7f74ae5b6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a81429b0 con 0x7f74a815a2e0 2023-10-18T09:08:47.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.577+0000 7f74acb53640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a81590e0 0x7f74a8151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:47.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.577+0000 7f74acb53640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a81590e0 0x7f74a8151e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50870/0 (socket says 172.21.15.40:50870) 2023-10-18T09:08:47.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.577+0000 7f74acb53640 1 -- 172.21.15.40:0/1827737362 learned_addr learned my addr 172.21.15.40:0/1827737362 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:47.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.578+0000 7f74a7fff640 1 --2- 172.21.15.40:0/1827737362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a815a2e0 0x7f74a813f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:47.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.578+0000 7f74a77fe640 1 --2- 172.21.15.40:0/1827737362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a815a6c0 0x7f74a8142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:47.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.578+0000 7f74acb53640 1 -- 172.21.15.40:0/1827737362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a815a2e0 msgr2=0x7f74a813f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:47.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.578+0000 7f74acb53640 1 --2- 172.21.15.40:0/1827737362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a815a2e0 0x7f74a813f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:47.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.578+0000 7f74acb53640 1 -- 172.21.15.40:0/1827737362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a815a6c0 msgr2=0x7f74a8142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:47.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.578+0000 7f74acb53640 1 --2- 172.21.15.40:0/1827737362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a815a6c0 0x7f74a8142470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:47.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.578+0000 7f74acb53640 1 -- 172.21.15.40:0/1827737362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7498009490 con 0x7f74a81590e0 2023-10-18T09:08:47.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.578+0000 7f74a7fff640 1 --2- 172.21.15.40:0/1827737362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a815a2e0 0x7f74a813f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:47.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.578+0000 7f74acb53640 1 --2- 172.21.15.40:0/1827737362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a81590e0 0x7f74a8151e80 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f74a000ca80 tx=0x7f74a000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:47.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.578+0000 7f74a4ff9640 1 -- 172.21.15.40:0/1827737362 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a0007d90 con 0x7f74a81590e0 2023-10-18T09:08:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.578+0000 7f74a4ff9640 1 -- 172.21.15.40:0/1827737362 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f74a0003990 con 0x7f74a81590e0 2023-10-18T09:08:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.578+0000 7f74a4ff9640 1 -- 172.21.15.40:0/1827737362 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a0003da0 con 0x7f74a81590e0 2023-10-18T09:08:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.578+0000 7f74ae5b6640 1 -- 172.21.15.40:0/1827737362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a81f3250 con 0x7f74a81590e0 2023-10-18T09:08:47.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.579+0000 7f74ae5b6640 1 -- 172.21.15.40:0/1827737362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a81f3810 con 0x7f74a81590e0 2023-10-18T09:08:47.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.580+0000 7f74ae5b6640 1 -- 172.21.15.40:0/1827737362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a806a5a0 con 0x7f74a81590e0 2023-10-18T09:08:47.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.581+0000 7f74a4ff9640 1 -- 172.21.15.40:0/1827737362 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f74a0004030 con 0x7f74a81590e0 2023-10-18T09:08:47.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.581+0000 7f74a4ff9640 1 --2- 172.21.15.40:0/1827737362 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f749406ae70 0x7f749406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:47.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.582+0000 7f74a4ff9640 1 -- 172.21.15.40:0/1827737362 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f74a0089e70 con 0x7f74a81590e0 2023-10-18T09:08:47.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.582+0000 7f74a7fff640 1 --2- 172.21.15.40:0/1827737362 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f749406ae70 0x7f749406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:47.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.583+0000 7f74a7fff640 1 --2- 172.21.15.40:0/1827737362 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f749406ae70 0x7f749406d330 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7498036000 tx=0x7f7498004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:47.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.586+0000 7f74a4ff9640 1 -- 172.21.15.40:0/1827737362 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f74a0055180 con 0x7f74a81590e0 2023-10-18T09:08:47.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.771+0000 7f74ae5b6640 1 -- 172.21.15.40:0/1827737362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74a8066fd0 con 0x7f74a81590e0 2023-10-18T09:08:47.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.772+0000 7f74a4ff9640 1 -- 172.21.15.40:0/1827737362 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+13007 (secure 0 0 0) 0x7f74a001d090 con 0x7f74a81590e0 2023-10-18T09:08:47.776 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:08:47.776 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":187,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:08:45.293624+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6834/137474609":"2023-10-19T09:08:42.073766+0000","172.21.15.40:6832/137474609":"2023-10-19T09:08:42.073766+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:08:47.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.775+0000 7f74ae5b6640 1 -- 172.21.15.40:0/1827737362 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f749406ae70 msgr2=0x7f749406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:47.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.775+0000 7f74ae5b6640 1 --2- 172.21.15.40:0/1827737362 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f749406ae70 0x7f749406d330 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7498036000 tx=0x7f7498004730 comp rx=0 tx=0).stop 2023-10-18T09:08:47.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.775+0000 7f74ae5b6640 1 -- 172.21.15.40:0/1827737362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a81590e0 msgr2=0x7f74a8151e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:47.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.775+0000 7f74ae5b6640 1 --2- 172.21.15.40:0/1827737362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a81590e0 0x7f74a8151e80 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f74a000ca80 tx=0x7f74a000cf40 comp rx=0 tx=0).stop 2023-10-18T09:08:47.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.775+0000 7f74ae5b6640 1 -- 172.21.15.40:0/1827737362 shutdown_connections 2023-10-18T09:08:47.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.775+0000 7f74ae5b6640 1 --2- 172.21.15.40:0/1827737362 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f749406ae70 0x7f749406d330 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:47.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.775+0000 7f74ae5b6640 1 --2- 172.21.15.40:0/1827737362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a81590e0 0x7f74a8151e80 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:47.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.775+0000 7f74ae5b6640 1 --2- 172.21.15.40:0/1827737362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a815a6c0 0x7f74a8142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:47.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.775+0000 7f74ae5b6640 1 --2- 172.21.15.40:0/1827737362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a815a2e0 0x7f74a813f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:47.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.775+0000 7f74ae5b6640 1 -- 172.21.15.40:0/1827737362 >> 172.21.15.40:0/1827737362 conn(0x7f74a806cee0 msgr2=0x7f74a80c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:47.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.775+0000 7f74ae5b6640 1 -- 172.21.15.40:0/1827737362 shutdown_connections 2023-10-18T09:08:47.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:47.775+0000 7f74ae5b6640 1 -- 172.21.15.40:0/1827737362 wait complete. 2023-10-18T09:08:47.796 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:08:47.796 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:08:48.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.015+0000 7f252aef3640 1 -- 172.21.15.40:0/1319533546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25240c98a0 msgr2=0x7f252406f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:48.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.015+0000 7f252aef3640 1 --2- 172.21.15.40:0/1319533546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25240c98a0 0x7f252406f510 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f25240ca340 tx=0x7f252002be40 comp rx=0 tx=0).stop 2023-10-18T09:08:48.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.015+0000 7f252aef3640 1 -- 172.21.15.40:0/1319533546 shutdown_connections 2023-10-18T09:08:48.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.015+0000 7f252aef3640 1 --2- 172.21.15.40:0/1319533546 >> v2:172.21.15.144:3300/0 conn(0x7f25241405a0 0x7f252406fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:48.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.015+0000 7f252aef3640 1 --2- 172.21.15.40:0/1319533546 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25240c98a0 0x7f252406f510 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:48.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.015+0000 7f252aef3640 1 -- 172.21.15.40:0/1319533546 >> 172.21.15.40:0/1319533546 conn(0x7f2524069720 msgr2=0x7f2524069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:48.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.015+0000 7f252aef3640 1 -- 172.21.15.40:0/1319533546 shutdown_connections 2023-10-18T09:08:48.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.016+0000 7f252aef3640 1 -- 172.21.15.40:0/1319533546 wait complete. 2023-10-18T09:08:48.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.017+0000 7f252aef3640 1 Processor -- start 2023-10-18T09:08:48.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.017+0000 7f252aef3640 1 -- start start 2023-10-18T09:08:48.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.018+0000 7f252aef3640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25240c98a0 0x7f2524156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:48.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.018+0000 7f252aef3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25241405a0 0x7f25241f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:48.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.018+0000 7f252aef3640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2524140b50 0x7f2524134e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:48.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.018+0000 7f252aef3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2524135370 con 0x7f25241405a0 2023-10-18T09:08:48.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.018+0000 7f252aef3640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2524156bb0 con 0x7f2524140b50 2023-10-18T09:08:48.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.018+0000 7f252aef3640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25240ae5d0 con 0x7f25240c98a0 2023-10-18T09:08:48.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.019+0000 7f25296f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25241405a0 0x7f25241f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:48.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.019+0000 7f25296f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25241405a0 0x7f25241f61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50882/0 (socket says 172.21.15.40:50882) 2023-10-18T09:08:48.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.019+0000 7f25296f0640 1 -- 172.21.15.40:0/1275627165 learned_addr learned my addr 172.21.15.40:0/1275627165 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:48.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.019+0000 7f2529ef1640 1 --2- 172.21.15.40:0/1275627165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25240c98a0 0x7f2524156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:48.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.019+0000 7f252a6f2640 1 --2- 172.21.15.40:0/1275627165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2524140b50 0x7f2524134e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:48.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.019+0000 7f25296f0640 1 -- 172.21.15.40:0/1275627165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25240c98a0 msgr2=0x7f2524156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:48.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.019+0000 7f25296f0640 1 --2- 172.21.15.40:0/1275627165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25240c98a0 0x7f2524156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:48.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.019+0000 7f25296f0640 1 -- 172.21.15.40:0/1275627165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2524140b50 msgr2=0x7f2524134e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:48.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.019+0000 7f25296f0640 1 --2- 172.21.15.40:0/1275627165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2524140b50 0x7f2524134e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:48.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.019+0000 7f25296f0640 1 -- 172.21.15.40:0/1275627165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2520009560 con 0x7f25241405a0 2023-10-18T09:08:48.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.020+0000 7f25296f0640 1 --2- 172.21.15.40:0/1275627165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25241405a0 0x7f25241f61a0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f251400d740 tx=0x7f251400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:48.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.020+0000 7f251a7fc640 1 -- 172.21.15.40:0/1275627165 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514009c30 con 0x7f25241405a0 2023-10-18T09:08:48.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.020+0000 7f251a7fc640 1 -- 172.21.15.40:0/1275627165 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2514009e80 con 0x7f25241405a0 2023-10-18T09:08:48.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.020+0000 7f251a7fc640 1 -- 172.21.15.40:0/1275627165 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514005260 con 0x7f25241405a0 2023-10-18T09:08:48.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.020+0000 7f252aef3640 1 -- 172.21.15.40:0/1275627165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25240ae8b0 con 0x7f25241405a0 2023-10-18T09:08:48.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.020+0000 7f252aef3640 1 -- 172.21.15.40:0/1275627165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25240aee70 con 0x7f25241405a0 2023-10-18T09:08:48.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.021+0000 7f252aef3640 1 -- 172.21.15.40:0/1275627165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f252413ad00 con 0x7f25241405a0 2023-10-18T09:08:48.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.022+0000 7f251a7fc640 1 -- 172.21.15.40:0/1275627165 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f25140053c0 con 0x7f25241405a0 2023-10-18T09:08:48.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.023+0000 7f251a7fc640 1 --2- 172.21.15.40:0/1275627165 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f250806ae70 0x7f250806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:48.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.023+0000 7f251a7fc640 1 -- 172.21.15.40:0/1275627165 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f25140895c0 con 0x7f25241405a0 2023-10-18T09:08:48.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.024+0000 7f2529ef1640 1 --2- 172.21.15.40:0/1275627165 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f250806ae70 0x7f250806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:48.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.025+0000 7f2529ef1640 1 --2- 172.21.15.40:0/1275627165 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f250806ae70 0x7f250806d330 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2520035000 tx=0x7f2520004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:48.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.028+0000 7f251a7fc640 1 -- 172.21.15.40:0/1275627165 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f25140548d0 con 0x7f25241405a0 2023-10-18T09:08:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.222+0000 7f252aef3640 1 -- 172.21.15.40:0/1275627165 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2524066e00 con 0x7f25241405a0 2023-10-18T09:08:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.223+0000 7f251a7fc640 1 -- 172.21.15.40:0/1275627165 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v187) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25140170a0 con 0x7f25241405a0 2023-10-18T09:08:48.227 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:08:48.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.226+0000 7f252aef3640 1 -- 172.21.15.40:0/1275627165 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f250806ae70 msgr2=0x7f250806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:48.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.226+0000 7f252aef3640 1 --2- 172.21.15.40:0/1275627165 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f250806ae70 0x7f250806d330 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2520035000 tx=0x7f2520004730 comp rx=0 tx=0).stop 2023-10-18T09:08:48.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.226+0000 7f252aef3640 1 -- 172.21.15.40:0/1275627165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25241405a0 msgr2=0x7f25241f61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:48.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.226+0000 7f252aef3640 1 --2- 172.21.15.40:0/1275627165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25241405a0 0x7f25241f61a0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f251400d740 tx=0x7f251400dc00 comp rx=0 tx=0).stop 2023-10-18T09:08:48.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.226+0000 7f252aef3640 1 -- 172.21.15.40:0/1275627165 shutdown_connections 2023-10-18T09:08:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.226+0000 7f252aef3640 1 --2- 172.21.15.40:0/1275627165 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f250806ae70 0x7f250806d330 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.226+0000 7f252aef3640 1 --2- 172.21.15.40:0/1275627165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2524140b50 0x7f2524134e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.226+0000 7f252aef3640 1 --2- 172.21.15.40:0/1275627165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25241405a0 0x7f25241f61a0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.226+0000 7f252aef3640 1 --2- 172.21.15.40:0/1275627165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25240c98a0 0x7f2524156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.226+0000 7f252aef3640 1 -- 172.21.15.40:0/1275627165 >> 172.21.15.40:0/1275627165 conn(0x7f2524069720 msgr2=0x7f252413b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:48.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.226+0000 7f252aef3640 1 -- 172.21.15.40:0/1275627165 shutdown_connections 2023-10-18T09:08:48.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.227+0000 7f252aef3640 1 -- 172.21.15.40:0/1275627165 wait complete. 2023-10-18T09:08:48.249 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2023-10-18T09:08:48.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.478+0000 7f39c9e13640 1 -- 172.21.15.40:0/1159603296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ac005380 msgr2=0x7f39ac00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:48.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.478+0000 7f39c9e13640 1 --2- 172.21.15.40:0/1159603296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ac005380 0x7f39ac00b980 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009e20 tx=0x7f39ac0057f0 comp rx=0 tx=0).stop 2023-10-18T09:08:48.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.478+0000 7f39c9e13640 1 -- 172.21.15.40:0/1159603296 shutdown_connections 2023-10-18T09:08:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.478+0000 7f39c9e13640 1 --2- 172.21.15.40:0/1159603296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ac005380 0x7f39ac00b980 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.478+0000 7f39c9e13640 1 -- 172.21.15.40:0/1159603296 >> 172.21.15.40:0/1159603296 conn(0x7f39c408ad00 msgr2=0x7f39c409a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.478+0000 7f39c9e13640 1 -- 172.21.15.40:0/1159603296 shutdown_connections 2023-10-18T09:08:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.479+0000 7f39c9e13640 1 -- 172.21.15.40:0/1159603296 wait complete. 2023-10-18T09:08:48.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.480+0000 7f39c9e13640 1 Processor -- start 2023-10-18T09:08:48.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.480+0000 7f39c9e13640 1 -- start start 2023-10-18T09:08:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.481+0000 7f39c9e13640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39ac005380 0x7f39c40c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.481+0000 7f39c9e13640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39c40c6550 0x7f39c415e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.481+0000 7f39c9e13640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c4155dd0 0x7f39c40c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.481+0000 7f39c9e13640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c40c60c0 con 0x7f39c4155dd0 2023-10-18T09:08:48.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.481+0000 7f39c9e13640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c40c69b0 con 0x7f39c40c6550 2023-10-18T09:08:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.481+0000 7f39c9e13640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c40c54b0 con 0x7f39ac005380 2023-10-18T09:08:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.482+0000 7f39c9612640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c4155dd0 0x7f39c40c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.482+0000 7f39c9612640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c4155dd0 0x7f39c40c72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50890/0 (socket says 172.21.15.40:50890) 2023-10-18T09:08:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.482+0000 7f39c9612640 1 -- 172.21.15.40:0/1676849609 learned_addr learned my addr 172.21.15.40:0/1676849609 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:48.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.482+0000 7f39c3fff640 1 --2- 172.21.15.40:0/1676849609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39c40c6550 0x7f39c415e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.482+0000 7f39c8e11640 1 --2- 172.21.15.40:0/1676849609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39ac005380 0x7f39c40c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.482+0000 7f39c9612640 1 -- 172.21.15.40:0/1676849609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39ac005380 msgr2=0x7f39c40c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.482+0000 7f39c9612640 1 --2- 172.21.15.40:0/1676849609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39ac005380 0x7f39c40c4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.482+0000 7f39c9612640 1 -- 172.21.15.40:0/1676849609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39c40c6550 msgr2=0x7f39c415e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.482+0000 7f39c9612640 1 --2- 172.21.15.40:0/1676849609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39c40c6550 0x7f39c415e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.482+0000 7f39c9612640 1 -- 172.21.15.40:0/1676849609 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c40c5790 con 0x7f39c4155dd0 2023-10-18T09:08:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.482+0000 7f39c8e11640 1 --2- 172.21.15.40:0/1676849609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39ac005380 0x7f39c40c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:08:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.483+0000 7f39c9612640 1 --2- 172.21.15.40:0/1676849609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c4155dd0 0x7f39c40c72c0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009e20 tx=0x7f39bc005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.483+0000 7f39c17fa640 1 -- 172.21.15.40:0/1676849609 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc005940 con 0x7f39c4155dd0 2023-10-18T09:08:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.483+0000 7f39c17fa640 1 -- 172.21.15.40:0/1676849609 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f39bc003a60 con 0x7f39c4155dd0 2023-10-18T09:08:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.483+0000 7f39c9e13640 1 -- 172.21.15.40:0/1676849609 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c40c5a70 con 0x7f39c4155dd0 2023-10-18T09:08:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.483+0000 7f39c17fa640 1 -- 172.21.15.40:0/1676849609 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc003e50 con 0x7f39c4155dd0 2023-10-18T09:08:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.483+0000 7f39c9e13640 1 -- 172.21.15.40:0/1676849609 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c415e660 con 0x7f39c4155dd0 2023-10-18T09:08:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.484+0000 7f39c9e13640 1 -- 172.21.15.40:0/1676849609 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c4071da0 con 0x7f39c4155dd0 2023-10-18T09:08:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.485+0000 7f39c17fa640 1 -- 172.21.15.40:0/1676849609 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f39bc004030 con 0x7f39c4155dd0 2023-10-18T09:08:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.487+0000 7f39c17fa640 1 --2- 172.21.15.40:0/1676849609 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39b006ae40 0x7f39b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:48.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.487+0000 7f39c17fa640 1 -- 172.21.15.40:0/1676849609 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f39bc0abd30 con 0x7f39c4155dd0 2023-10-18T09:08:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.488+0000 7f39c8e11640 1 --2- 172.21.15.40:0/1676849609 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39b006ae40 0x7f39b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.489+0000 7f39c8e11640 1 --2- 172.21.15.40:0/1676849609 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39b006ae40 0x7f39b006d300 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f39b40038f0 tx=0x7f39b4006cd0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.491+0000 7f39c17fa640 1 -- 172.21.15.40:0/1676849609 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39bc077040 con 0x7f39c4155dd0 2023-10-18T09:08:48.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:48.753+0000 7f39c9e13640 1 -- 172.21.15.40:0/1676849609 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7f39c4066e00 con 0x7f39c4155dd0 2023-10-18T09:08:49.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.317+0000 7f39c17fa640 1 -- 172.21.15.40:0/1676849609 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v652) v1 ==== 138+0+0 (secure 0 0 0) 0x7f39bc037020 con 0x7f39c4155dd0 2023-10-18T09:08:49.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.321+0000 7f39c9e13640 1 -- 172.21.15.40:0/1676849609 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39b006ae40 msgr2=0x7f39b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:49.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.321+0000 7f39c9e13640 1 --2- 172.21.15.40:0/1676849609 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39b006ae40 0x7f39b006d300 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f39b40038f0 tx=0x7f39b4006cd0 comp rx=0 tx=0).stop 2023-10-18T09:08:49.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.321+0000 7f39c9e13640 1 -- 172.21.15.40:0/1676849609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c4155dd0 msgr2=0x7f39c40c72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:49.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.321+0000 7f39c9e13640 1 --2- 172.21.15.40:0/1676849609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c4155dd0 0x7f39c40c72c0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f39bc009e20 tx=0x7f39bc005340 comp rx=0 tx=0).stop 2023-10-18T09:08:49.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.321+0000 7f39c9e13640 1 -- 172.21.15.40:0/1676849609 shutdown_connections 2023-10-18T09:08:49.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.321+0000 7f39c9e13640 1 --2- 172.21.15.40:0/1676849609 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f39b006ae40 0x7f39b006d300 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:49.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.321+0000 7f39c9e13640 1 --2- 172.21.15.40:0/1676849609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39c4155dd0 0x7f39c40c72c0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:49.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.321+0000 7f39c9e13640 1 --2- 172.21.15.40:0/1676849609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39c40c6550 0x7f39c415e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:49.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.321+0000 7f39c9e13640 1 --2- 172.21.15.40:0/1676849609 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39ac005380 0x7f39c40c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:49.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.321+0000 7f39c9e13640 1 -- 172.21.15.40:0/1676849609 >> 172.21.15.40:0/1676849609 conn(0x7f39c408ad00 msgr2=0x7f39c40ca410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:49.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.321+0000 7f39c9e13640 1 -- 172.21.15.40:0/1676849609 shutdown_connections 2023-10-18T09:08:49.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.321+0000 7f39c9e13640 1 -- 172.21.15.40:0/1676849609 wait complete. 2023-10-18T09:08:49.347 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-18T09:08:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.570+0000 7fbfe73ae640 1 -- 172.21.15.40:0/2826294014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfcc0039c0 msgr2=0x7fbfcc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.570+0000 7fbfe73ae640 1 --2- 172.21.15.40:0/2826294014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfcc0039c0 0x7fbfcc003de0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009b50 tx=0x7fbfcc0057f0 comp rx=0 tx=0).stop 2023-10-18T09:08:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.570+0000 7fbfe73ae640 1 -- 172.21.15.40:0/2826294014 shutdown_connections 2023-10-18T09:08:49.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.570+0000 7fbfe73ae640 1 --2- 172.21.15.40:0/2826294014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfcc0039c0 0x7fbfcc003de0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:49.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.570+0000 7fbfe73ae640 1 -- 172.21.15.40:0/2826294014 >> 172.21.15.40:0/2826294014 conn(0x7fbfe00aed70 msgr2=0x7fbfe00af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:49.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.571+0000 7fbfe73ae640 1 -- 172.21.15.40:0/2826294014 shutdown_connections 2023-10-18T09:08:49.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.571+0000 7fbfe73ae640 1 -- 172.21.15.40:0/2826294014 wait complete. 2023-10-18T09:08:49.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.572+0000 7fbfe73ae640 1 Processor -- start 2023-10-18T09:08:49.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.573+0000 7fbfe73ae640 1 -- start start 2023-10-18T09:08:49.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.573+0000 7fbfe73ae640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfcc0039c0 0x7fbfe01f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:49.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.573+0000 7fbfe73ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe01f27b0 0x7fbfe01f1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:49.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.573+0000 7fbfe73ae640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfe01ef970 0x7fbfe00c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:49.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.574+0000 7fbfe73ae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe01f1b70 con 0x7fbfe01f27b0 2023-10-18T09:08:49.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.574+0000 7fbfe73ae640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe01f34d0 con 0x7fbfcc0039c0 2023-10-18T09:08:49.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.574+0000 7fbfe73ae640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe01f37b0 con 0x7fbfe01ef970 2023-10-18T09:08:49.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.574+0000 7fbfe5bab640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe01f27b0 0x7fbfe01f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:49.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.574+0000 7fbfe5bab640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe01f27b0 0x7fbfe01f1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50900/0 (socket says 172.21.15.40:50900) 2023-10-18T09:08:49.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.574+0000 7fbfe5bab640 1 -- 172.21.15.40:0/3243367174 learned_addr learned my addr 172.21.15.40:0/3243367174 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:49.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.574+0000 7fbfe63ac640 1 --2- 172.21.15.40:0/3243367174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfcc0039c0 0x7fbfe01f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:49.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.574+0000 7fbfe6bad640 1 --2- 172.21.15.40:0/3243367174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfe01ef970 0x7fbfe00c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:49.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.574+0000 7fbfe5bab640 1 -- 172.21.15.40:0/3243367174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfe01ef970 msgr2=0x7fbfe00c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:49.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.574+0000 7fbfe5bab640 1 --2- 172.21.15.40:0/3243367174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfe01ef970 0x7fbfe00c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:49.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.574+0000 7fbfe5bab640 1 -- 172.21.15.40:0/3243367174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfcc0039c0 msgr2=0x7fbfe01f2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:49.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.574+0000 7fbfe5bab640 1 --2- 172.21.15.40:0/3243367174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfcc0039c0 0x7fbfe01f2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:49.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.575+0000 7fbfe5bab640 1 -- 172.21.15.40:0/3243367174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe01f3a90 con 0x7fbfe01f27b0 2023-10-18T09:08:49.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.575+0000 7fbfe63ac640 1 --2- 172.21.15.40:0/3243367174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfcc0039c0 0x7fbfe01f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:08:49.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.575+0000 7fbfe5bab640 1 --2- 172.21.15.40:0/3243367174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe01f27b0 0x7fbfe01f1630 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0004cc0 tx=0x7fbfd0001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:49.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.575+0000 7fbfd6ffd640 1 -- 172.21.15.40:0/3243367174 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0011660 con 0x7fbfe01f27b0 2023-10-18T09:08:49.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.575+0000 7fbfd6ffd640 1 -- 172.21.15.40:0/3243367174 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbfd00118b0 con 0x7fbfe01f27b0 2023-10-18T09:08:49.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.575+0000 7fbfe73ae640 1 -- 172.21.15.40:0/3243367174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe01f3d70 con 0x7fbfe01f27b0 2023-10-18T09:08:49.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.575+0000 7fbfd6ffd640 1 -- 172.21.15.40:0/3243367174 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0011c80 con 0x7fbfe01f27b0 2023-10-18T09:08:49.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.575+0000 7fbfe73ae640 1 -- 172.21.15.40:0/3243367174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe01f2c30 con 0x7fbfe01f27b0 2023-10-18T09:08:49.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.577+0000 7fbfe73ae640 1 -- 172.21.15.40:0/3243367174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe0066e00 con 0x7fbfe01f27b0 2023-10-18T09:08:49.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.578+0000 7fbfd6ffd640 1 -- 172.21.15.40:0/3243367174 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbfd001a400 con 0x7fbfe01f27b0 2023-10-18T09:08:49.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.579+0000 7fbfd6ffd640 1 --2- 172.21.15.40:0/3243367174 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbfc406ae70 0x7fbfc406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:49.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.579+0000 7fbfd6ffd640 1 -- 172.21.15.40:0/3243367174 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbfd0089ff0 con 0x7fbfe01f27b0 2023-10-18T09:08:49.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.580+0000 7fbfe63ac640 1 --2- 172.21.15.40:0/3243367174 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbfc406ae70 0x7fbfc406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:49.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.581+0000 7fbfe63ac640 1 --2- 172.21.15.40:0/3243367174 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbfc406ae70 0x7fbfc406d330 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc036000 tx=0x7fbfdc035040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:49.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.584+0000 7fbfd6ffd640 1 -- 172.21.15.40:0/3243367174 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbfd00bcca0 con 0x7fbfe01f27b0 2023-10-18T09:08:49.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.776+0000 7fbfe73ae640 1 -- 172.21.15.40:0/3243367174 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7fbfe006c0d0 con 0x7fbfe01f27b0 2023-10-18T09:08:49.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.776+0000 7fbfd6ffd640 1 -- 172.21.15.40:0/3243367174 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v40) v1 ==== 59+0+1452 (secure 0 0 0) 0x7fbfd0055340 con 0x7fbfe01f27b0 2023-10-18T09:08:49.780 INFO:teuthology.orchestra.run.smithi040.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-18T09:08:49.780 INFO:teuthology.orchestra.run.smithi040.stdout:mon advanced mon_op_complaint_time 120 2023-10-18T09:08:49.781 INFO:teuthology.orchestra.run.smithi040.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-18T09:08:49.781 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds 20/20 2023-10-18T09:08:49.781 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_mds_balancer 20/20 2023-10-18T09:08:49.781 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced debug_ms 1/1 2023-10-18T09:08:49.782 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_fragment_size_max 10000 2023-10-18T09:08:49.782 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_merge_size 5 2023-10-18T09:08:49.782 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_bits 3 2023-10-18T09:08:49.782 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_bal_split_size 100 2023-10-18T09:08:49.782 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_frag true 2023-10-18T09:08:49.783 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_debug_scatterstat true 2023-10-18T09:08:49.783 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-18T09:08:49.783 INFO:teuthology.orchestra.run.smithi040.stdout:mds dev mds_verify_scatter true 2023-10-18T09:08:49.783 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-18T09:08:49.783 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_mon_op_timeout 900 2023-10-18T09:08:49.784 INFO:teuthology.orchestra.run.smithi040.stdout:mds advanced rados_osd_op_timeout 900 2023-10-18T09:08:49.784 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced client_mount_timeout 600 2023-10-18T09:08:49.784 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_client 20/20 2023-10-18T09:08:49.784 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced debug_ms 1/1 2023-10-18T09:08:49.785 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_mon_op_timeout 900 2023-10-18T09:08:49.785 INFO:teuthology.orchestra.run.smithi040.stdout:client advanced rados_osd_op_timeout 900 2023-10-18T09:08:49.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.779+0000 7fbfe73ae640 1 -- 172.21.15.40:0/3243367174 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbfc406ae70 msgr2=0x7fbfc406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:49.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.779+0000 7fbfe73ae640 1 --2- 172.21.15.40:0/3243367174 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbfc406ae70 0x7fbfc406d330 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc036000 tx=0x7fbfdc035040 comp rx=0 tx=0).stop 2023-10-18T09:08:49.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.779+0000 7fbfe73ae640 1 -- 172.21.15.40:0/3243367174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe01f27b0 msgr2=0x7fbfe01f1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:49.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.779+0000 7fbfe73ae640 1 --2- 172.21.15.40:0/3243367174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe01f27b0 0x7fbfe01f1630 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0004cc0 tx=0x7fbfd0001630 comp rx=0 tx=0).stop 2023-10-18T09:08:49.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.779+0000 7fbfe73ae640 1 -- 172.21.15.40:0/3243367174 shutdown_connections 2023-10-18T09:08:49.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.779+0000 7fbfe73ae640 1 --2- 172.21.15.40:0/3243367174 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbfc406ae70 0x7fbfc406d330 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:49.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.779+0000 7fbfe73ae640 1 --2- 172.21.15.40:0/3243367174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfe01ef970 0x7fbfe00c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:49.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.779+0000 7fbfe73ae640 1 --2- 172.21.15.40:0/3243367174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfe01f27b0 0x7fbfe01f1630 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:49.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.779+0000 7fbfe73ae640 1 --2- 172.21.15.40:0/3243367174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfcc0039c0 0x7fbfe01f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:49.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.779+0000 7fbfe73ae640 1 -- 172.21.15.40:0/3243367174 >> 172.21.15.40:0/3243367174 conn(0x7fbfe00aed70 msgr2=0x7fbfe00c9940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:49.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.779+0000 7fbfe73ae640 1 -- 172.21.15.40:0/3243367174 shutdown_connections 2023-10-18T09:08:49.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:49.779+0000 7fbfe73ae640 1 -- 172.21.15.40:0/3243367174 wait complete. 2023-10-18T09:08:49.802 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-18T09:08:50.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.035+0000 7ff63d4b6640 1 -- 172.21.15.40:0/825753363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff63806c180 msgr2=0x7ff6380cb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:50.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.035+0000 7ff63d4b6640 1 --2- 172.21.15.40:0/825753363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff63806c180 0x7ff6380cb3b0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7ff6380aee70 tx=0x7ff63002c2f0 comp rx=0 tx=0).stop 2023-10-18T09:08:50.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.035+0000 7ff63d4b6640 1 -- 172.21.15.40:0/825753363 shutdown_connections 2023-10-18T09:08:50.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.035+0000 7ff63d4b6640 1 --2- 172.21.15.40:0/825753363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff63806c180 0x7ff6380cb3b0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:50.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.035+0000 7ff63d4b6640 1 --2- 172.21.15.40:0/825753363 >> v2:172.21.15.144:3300/0 conn(0x7ff638159430 0x7ff6380b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:50.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.035+0000 7ff63d4b6640 1 -- 172.21.15.40:0/825753363 >> 172.21.15.40:0/825753363 conn(0x7ff63806ca10 msgr2=0x7ff63806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:50.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.035+0000 7ff63d4b6640 1 -- 172.21.15.40:0/825753363 shutdown_connections 2023-10-18T09:08:50.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.035+0000 7ff63d4b6640 1 -- 172.21.15.40:0/825753363 wait complete. 2023-10-18T09:08:50.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.037+0000 7ff63d4b6640 1 Processor -- start 2023-10-18T09:08:50.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.037+0000 7ff63d4b6640 1 -- start start 2023-10-18T09:08:50.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.038+0000 7ff63d4b6640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63806c180 0x7ff638151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:50.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.038+0000 7ff63d4b6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff638159430 0x7ff638159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:50.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.038+0000 7ff63d4b6640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff63815cda0 0x7ff638159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:50.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.038+0000 7ff63d4b6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6381f5bd0 con 0x7ff638159430 2023-10-18T09:08:50.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.038+0000 7ff63d4b6640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6380c8f90 con 0x7ff63806c180 2023-10-18T09:08:50.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.038+0000 7ff63d4b6640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6381f5270 con 0x7ff63815cda0 2023-10-18T09:08:50.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.038+0000 7ff62ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff638159430 0x7ff638159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:50.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.039+0000 7ff62ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff638159430 0x7ff638159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50920/0 (socket says 172.21.15.40:50920) 2023-10-18T09:08:50.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.039+0000 7ff62ffff640 1 -- 172.21.15.40:0/2320094185 learned_addr learned my addr 172.21.15.40:0/2320094185 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:50.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.039+0000 7ff637fff640 1 --2- 172.21.15.40:0/2320094185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63806c180 0x7ff638151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:50.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.039+0000 7ff63ccb5640 1 --2- 172.21.15.40:0/2320094185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff63815cda0 0x7ff638159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:50.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.039+0000 7ff62ffff640 1 -- 172.21.15.40:0/2320094185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff63815cda0 msgr2=0x7ff638159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:50.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.039+0000 7ff62ffff640 1 --2- 172.21.15.40:0/2320094185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff63815cda0 0x7ff638159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:50.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.039+0000 7ff62ffff640 1 -- 172.21.15.40:0/2320094185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63806c180 msgr2=0x7ff638151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:50.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.039+0000 7ff62ffff640 1 --2- 172.21.15.40:0/2320094185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63806c180 0x7ff638151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:50.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.039+0000 7ff62ffff640 1 -- 172.21.15.40:0/2320094185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff630009900 con 0x7ff638159430 2023-10-18T09:08:50.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.039+0000 7ff637fff640 1 --2- 172.21.15.40:0/2320094185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63806c180 0x7ff638151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:50.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.039+0000 7ff62ffff640 1 --2- 172.21.15.40:0/2320094185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff638159430 0x7ff638159860 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7ff62000c400 tx=0x7ff62000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:50.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.039+0000 7ff6357fa640 1 -- 172.21.15.40:0/2320094185 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6200052d0 con 0x7ff638159430 2023-10-18T09:08:50.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.040+0000 7ff6357fa640 1 -- 172.21.15.40:0/2320094185 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff6200035f0 con 0x7ff638159430 2023-10-18T09:08:50.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.040+0000 7ff6357fa640 1 -- 172.21.15.40:0/2320094185 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff620003a00 con 0x7ff638159430 2023-10-18T09:08:50.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.040+0000 7ff63d4b6640 1 -- 172.21.15.40:0/2320094185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6381f5ff0 con 0x7ff638159430 2023-10-18T09:08:50.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.040+0000 7ff63d4b6640 1 -- 172.21.15.40:0/2320094185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6381f5510 con 0x7ff638159430 2023-10-18T09:08:50.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.041+0000 7ff63d4b6640 1 -- 172.21.15.40:0/2320094185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6381f5d10 con 0x7ff638159430 2023-10-18T09:08:50.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.042+0000 7ff6357fa640 1 -- 172.21.15.40:0/2320094185 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff620003b60 con 0x7ff638159430 2023-10-18T09:08:50.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.043+0000 7ff6357fa640 1 --2- 172.21.15.40:0/2320094185 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff61806ae40 0x7ff61806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:50.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.043+0000 7ff6357fa640 1 -- 172.21.15.40:0/2320094185 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff620088ac0 con 0x7ff638159430 2023-10-18T09:08:50.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.044+0000 7ff637fff640 1 --2- 172.21.15.40:0/2320094185 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff61806ae40 0x7ff61806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:50.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.045+0000 7ff637fff640 1 --2- 172.21.15.40:0/2320094185 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff61806ae40 0x7ff61806d300 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff628009b40 tx=0x7ff628005e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:50.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.048+0000 7ff6357fa640 1 -- 172.21.15.40:0/2320094185 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff6381f5d10 con 0x7ff638159430 2023-10-18T09:08:50.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.239+0000 7ff63d4b6640 1 -- 172.21.15.40:0/2320094185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff6381f5d10 con 0x7ff638159430 2023-10-18T09:08:50.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.245+0000 7ff6357fa640 1 -- 172.21.15.40:0/2320094185 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff6381f5d10 con 0x7ff638159430 2023-10-18T09:08:50.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.248+0000 7ff63d4b6640 1 -- 172.21.15.40:0/2320094185 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff61806ae40 msgr2=0x7ff61806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:50.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.248+0000 7ff63d4b6640 1 --2- 172.21.15.40:0/2320094185 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff61806ae40 0x7ff61806d300 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff628009b40 tx=0x7ff628005e50 comp rx=0 tx=0).stop 2023-10-18T09:08:50.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.248+0000 7ff63d4b6640 1 -- 172.21.15.40:0/2320094185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff638159430 msgr2=0x7ff638159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:50.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.248+0000 7ff63d4b6640 1 --2- 172.21.15.40:0/2320094185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff638159430 0x7ff638159860 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7ff62000c400 tx=0x7ff62000c8c0 comp rx=0 tx=0).stop 2023-10-18T09:08:50.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.248+0000 7ff63d4b6640 1 -- 172.21.15.40:0/2320094185 shutdown_connections 2023-10-18T09:08:50.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.248+0000 7ff63d4b6640 1 --2- 172.21.15.40:0/2320094185 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff61806ae40 0x7ff61806d300 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:50.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.248+0000 7ff63d4b6640 1 --2- 172.21.15.40:0/2320094185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff63815cda0 0x7ff638159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:50.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.248+0000 7ff63d4b6640 1 --2- 172.21.15.40:0/2320094185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff638159430 0x7ff638159860 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:50.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.248+0000 7ff63d4b6640 1 --2- 172.21.15.40:0/2320094185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff63806c180 0x7ff638151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:50.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.248+0000 7ff63d4b6640 1 -- 172.21.15.40:0/2320094185 >> 172.21.15.40:0/2320094185 conn(0x7ff63806ca10 msgr2=0x7ff6380aa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:50.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.248+0000 7ff63d4b6640 1 -- 172.21.15.40:0/2320094185 shutdown_connections 2023-10-18T09:08:50.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:50.249+0000 7ff63d4b6640 1 -- 172.21.15.40:0/2320094185 wait complete. 2023-10-18T09:08:50.271 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T09:08:50.272 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:08:50.395 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:08:50.516 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T09:08:50.516 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:08:50.516 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:08:50.583 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:08:50.584 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:08:50.584 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:08:50.642 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:08:50.643 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:08:50.643 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:08:50.707 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:08:50.708 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:08:50.708 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:08:50.767 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:08:50.767 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-18T09:08:50.768 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:08:51.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.001+0000 7f586d9b7640 1 -- 172.21.15.40:0/827676637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58680b1100 msgr2=0x7f58681625a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:51.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.001+0000 7f586d9b7640 1 --2- 172.21.15.40:0/827676637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58680b1100 0x7f58681625a0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f5860005fd0 tx=0x7f586002c0f0 comp rx=0 tx=0).stop 2023-10-18T09:08:51.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.001+0000 7f586d9b7640 1 -- 172.21.15.40:0/827676637 shutdown_connections 2023-10-18T09:08:51.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.001+0000 7f586d9b7640 1 --2- 172.21.15.40:0/827676637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58680b1100 0x7f58681625a0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:51.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.001+0000 7f586d9b7640 1 --2- 172.21.15.40:0/827676637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f58681586a0 0x7f58680b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:51.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.001+0000 7f586d9b7640 1 -- 172.21.15.40:0/827676637 >> 172.21.15.40:0/827676637 conn(0x7f586806cee0 msgr2=0x7f5868069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:51.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.001+0000 7f586d9b7640 1 -- 172.21.15.40:0/827676637 shutdown_connections 2023-10-18T09:08:51.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.001+0000 7f586d9b7640 1 -- 172.21.15.40:0/827676637 wait complete. 2023-10-18T09:08:51.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.003+0000 7f586d9b7640 1 Processor -- start 2023-10-18T09:08:51.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.003+0000 7f586d9b7640 1 -- start start 2023-10-18T09:08:51.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f586d9b7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f586806d710 0x7f5868139670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:51.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f586d9b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58680b1100 0x7f5868149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f586d9b7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58681586a0 0x7f5868149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f586d9b7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58680ae9e0 con 0x7f58680b1100 2023-10-18T09:08:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f586d9b7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58680ad940 con 0x7f58681586a0 2023-10-18T09:08:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f586d9b7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58680adc20 con 0x7f586806d710 2023-10-18T09:08:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f58667fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58680b1100 0x7f5868149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:51.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f58667fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58680b1100 0x7f5868149940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50940/0 (socket says 172.21.15.40:50940) 2023-10-18T09:08:51.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f58667fc640 1 -- 172.21.15.40:0/1083717303 learned_addr learned my addr 172.21.15.40:0/1083717303 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:51.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f5866ffd640 1 --2- 172.21.15.40:0/1083717303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f586806d710 0x7f5868139670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:51.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f58677fe640 1 --2- 172.21.15.40:0/1083717303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58681586a0 0x7f5868149e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:51.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f58667fc640 1 -- 172.21.15.40:0/1083717303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f586806d710 msgr2=0x7f5868139670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:51.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f58667fc640 1 --2- 172.21.15.40:0/1083717303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f586806d710 0x7f5868139670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:51.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f58667fc640 1 -- 172.21.15.40:0/1083717303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58681586a0 msgr2=0x7f5868149e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:51.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f58667fc640 1 --2- 172.21.15.40:0/1083717303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58681586a0 0x7f5868149e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:51.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f58667fc640 1 -- 172.21.15.40:0/1083717303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5860009840 con 0x7f58680b1100 2023-10-18T09:08:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.004+0000 7f5866ffd640 1 --2- 172.21.15.40:0/1083717303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f586806d710 0x7f5868139670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.005+0000 7f58667fc640 1 --2- 172.21.15.40:0/1083717303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58680b1100 0x7f5868149940 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f585c00b410 tx=0x7f585c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.005+0000 7f584ffff640 1 -- 172.21.15.40:0/1083717303 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c011830 con 0x7f58680b1100 2023-10-18T09:08:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.005+0000 7f584ffff640 1 -- 172.21.15.40:0/1083717303 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f585c011aa0 con 0x7f58680b1100 2023-10-18T09:08:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.005+0000 7f584ffff640 1 -- 172.21.15.40:0/1083717303 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c010380 con 0x7f58680b1100 2023-10-18T09:08:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.005+0000 7f586d9b7640 1 -- 172.21.15.40:0/1083717303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58680adf00 con 0x7f58680b1100 2023-10-18T09:08:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.005+0000 7f586d9b7640 1 -- 172.21.15.40:0/1083717303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58680ae4c0 con 0x7f58680b1100 2023-10-18T09:08:51.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.006+0000 7f586d9b7640 1 -- 172.21.15.40:0/1083717303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58680ae6e0 con 0x7f58680b1100 2023-10-18T09:08:51.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.007+0000 7f584ffff640 1 -- 172.21.15.40:0/1083717303 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f585c0104e0 con 0x7f58680b1100 2023-10-18T09:08:51.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.008+0000 7f584ffff640 1 --2- 172.21.15.40:0/1083717303 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f585406ae40 0x7f585406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:51.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.009+0000 7f584ffff640 1 -- 172.21.15.40:0/1083717303 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f585c0897c0 con 0x7f58680b1100 2023-10-18T09:08:51.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.009+0000 7f5866ffd640 1 --2- 172.21.15.40:0/1083717303 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f585406ae40 0x7f585406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:51.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.010+0000 7f5866ffd640 1 --2- 172.21.15.40:0/1083717303 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f585406ae40 0x7f585406d300 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f58580046b0 tx=0x7f5858009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:51.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.013+0000 7f584ffff640 1 -- 172.21.15.40:0/1083717303 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f585c054ad0 con 0x7f58680b1100 2023-10-18T09:08:51.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.246+0000 7f586d9b7640 1 -- 172.21.15.40:0/1083717303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f586815c170 con 0x7f58680b1100 2023-10-18T09:08:51.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.247+0000 7f584ffff640 1 -- 172.21.15.40:0/1083717303 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f585c015e90 con 0x7f58680b1100 2023-10-18T09:08:51.251 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:08:51.251 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":84,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6823,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/3592705338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3592705338},{"type":"v1","addr":"172.21.15.40:6835","nonce":3592705338}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":6829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/1076939595","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1076939595},{"type":"v1","addr":"172.21.15.144:6836","nonce":1076939595}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":6832,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/2483821333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":2483821333},{"type":"v1","addr":"172.21.15.144:6837","nonce":2483821333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9180,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/526876923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":526876923},{"type":"v1","addr":"172.21.15.40:6834","nonce":526876923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84}],"filesystems":[]} 2023-10-18T09:08:51.251 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 84 2023-10-18T09:08:51.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.249+0000 7f586d9b7640 1 -- 172.21.15.40:0/1083717303 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f585406ae40 msgr2=0x7f585406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:51.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.249+0000 7f586d9b7640 1 --2- 172.21.15.40:0/1083717303 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f585406ae40 0x7f585406d300 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f58580046b0 tx=0x7f5858009040 comp rx=0 tx=0).stop 2023-10-18T09:08:51.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.249+0000 7f586d9b7640 1 -- 172.21.15.40:0/1083717303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58680b1100 msgr2=0x7f5868149940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:51.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.249+0000 7f586d9b7640 1 --2- 172.21.15.40:0/1083717303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58680b1100 0x7f5868149940 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f585c00b410 tx=0x7f585c00b8d0 comp rx=0 tx=0).stop 2023-10-18T09:08:51.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.249+0000 7f586d9b7640 1 -- 172.21.15.40:0/1083717303 shutdown_connections 2023-10-18T09:08:51.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.249+0000 7f586d9b7640 1 --2- 172.21.15.40:0/1083717303 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f585406ae40 0x7f585406d300 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:51.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.249+0000 7f586d9b7640 1 --2- 172.21.15.40:0/1083717303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58681586a0 0x7f5868149e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:51.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.249+0000 7f586d9b7640 1 --2- 172.21.15.40:0/1083717303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58680b1100 0x7f5868149940 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:51.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.249+0000 7f586d9b7640 1 --2- 172.21.15.40:0/1083717303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f586806d710 0x7f5868139670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:51.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.250+0000 7f586d9b7640 1 -- 172.21.15.40:0/1083717303 >> 172.21.15.40:0/1083717303 conn(0x7f586806cee0 msgr2=0x7f5868069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:51.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.250+0000 7f586d9b7640 1 -- 172.21.15.40:0/1083717303 shutdown_connections 2023-10-18T09:08:51.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.250+0000 7f586d9b7640 1 -- 172.21.15.40:0/1083717303 wait complete. 2023-10-18T09:08:51.273 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-18T09:08:51.273 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-18T09:08:51.273 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T09:08:51.274 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:08:51.274 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-18T09:08:51.274 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-18T09:08:51.275 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T09:08:51.275 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:08:51.275 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-18T09:08:51.275 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-18T09:08:51.276 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T09:08:51.276 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:08:51.276 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-18T09:08:51.276 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-18T09:08:51.277 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T09:08:51.277 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:08:51.295 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T09:08:51.296 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-18T09:08:51.299 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T09:08:51.300 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-18T09:08:51.301 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T09:08:51.301 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-18T09:08:51.302 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T09:08:51.303 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-18T09:08:51.347 INFO:tasks.ceph.mds.d:Started 2023-10-18T09:08:51.348 DEBUG:teuthology.parallel:result is None 2023-10-18T09:08:51.351 INFO:tasks.ceph.mds.a:Started 2023-10-18T09:08:51.352 DEBUG:teuthology.parallel:result is None 2023-10-18T09:08:51.355 INFO:tasks.ceph.mds.b:Started 2023-10-18T09:08:51.356 DEBUG:teuthology.parallel:result is None 2023-10-18T09:08:51.359 INFO:tasks.ceph.mds.c:Started 2023-10-18T09:08:51.359 DEBUG:teuthology.parallel:result is None 2023-10-18T09:08:51.360 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-18T09:08:51.510 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2023-10-18T09:08:51.510 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2023-10-18T09:08:51.511 INFO:tasks.ceph.mds.d.smithi144.stderr:2023-10-18T09:08:51.509+0000 7f280ff58640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-18T09:08:51.520 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-10-18T09:08:51.521 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T09:08:51.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.559+0000 7f5363ec9640 1 -- 172.21.15.40:0/3527336320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f535c0b1b90 msgr2=0x7f535c0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:51.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.559+0000 7f5363ec9640 1 --2- 172.21.15.40:0/3527336320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f535c0b1b90 0x7f535c0cd2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f534c009860 tx=0x7f534c02bd50 comp rx=0 tx=0).stop 2023-10-18T09:08:51.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.560+0000 7f5363ec9640 1 -- 172.21.15.40:0/3527336320 shutdown_connections 2023-10-18T09:08:51.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.560+0000 7f5363ec9640 1 --2- 172.21.15.40:0/3527336320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f535c0b1b90 0x7f535c0cd2f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:51.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.560+0000 7f5363ec9640 1 --2- 172.21.15.40:0/3527336320 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f535c15eaa0 0x7f535c0ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:51.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.560+0000 7f5363ec9640 1 -- 172.21.15.40:0/3527336320 >> 172.21.15.40:0/3527336320 conn(0x7f535c06cee0 msgr2=0x7f535c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:51.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.560+0000 7f5363ec9640 1 -- 172.21.15.40:0/3527336320 shutdown_connections 2023-10-18T09:08:51.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.560+0000 7f5363ec9640 1 -- 172.21.15.40:0/3527336320 wait complete. 2023-10-18T09:08:51.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.561+0000 7f5363ec9640 1 Processor -- start 2023-10-18T09:08:51.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.561+0000 7f5363ec9640 1 -- start start 2023-10-18T09:08:51.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.562+0000 7f5363ec9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f535c06ffe0 0x7f535c1469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:51.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.562+0000 7f5363ec9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f535c0b1b90 0x7f535c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:51.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.562+0000 7f5363ec9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f535c15eaa0 0x7f535c0ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:51.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.562+0000 7f5363ec9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c156fb0 con 0x7f535c06ffe0 2023-10-18T09:08:51.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.562+0000 7f5363ec9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c156430 con 0x7f535c0b1b90 2023-10-18T09:08:51.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.562+0000 7f5363ec9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c156710 con 0x7f535c15eaa0 2023-10-18T09:08:51.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.563+0000 7f5361c65640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f535c06ffe0 0x7f535c1469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:51.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.563+0000 7f5361c65640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f535c06ffe0 0x7f535c1469f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50982/0 (socket says 172.21.15.40:50982) 2023-10-18T09:08:51.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.563+0000 7f5361c65640 1 -- 172.21.15.40:0/3262028204 learned_addr learned my addr 172.21.15.40:0/3262028204 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:51.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.563+0000 7f5362466640 1 --2- 172.21.15.40:0/3262028204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f535c15eaa0 0x7f535c0ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:51.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.563+0000 7f5361c65640 1 -- 172.21.15.40:0/3262028204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f535c15eaa0 msgr2=0x7f535c0ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:51.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.563+0000 7f5361464640 1 --2- 172.21.15.40:0/3262028204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f535c0b1b90 0x7f535c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:51.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.563+0000 7f5361c65640 1 --2- 172.21.15.40:0/3262028204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f535c15eaa0 0x7f535c0ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:51.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.563+0000 7f5361c65640 1 -- 172.21.15.40:0/3262028204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f535c0b1b90 msgr2=0x7f535c0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:51.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.563+0000 7f5361c65640 1 --2- 172.21.15.40:0/3262028204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f535c0b1b90 0x7f535c0cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:51.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.563+0000 7f5361c65640 1 -- 172.21.15.40:0/3262028204 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c009490 con 0x7f535c06ffe0 2023-10-18T09:08:51.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.563+0000 7f5362466640 1 --2- 172.21.15.40:0/3262028204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f535c15eaa0 0x7f535c0ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:51.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.563+0000 7f5361464640 1 --2- 172.21.15.40:0/3262028204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f535c0b1b90 0x7f535c0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:51.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.564+0000 7f5361c65640 1 --2- 172.21.15.40:0/3262028204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f535c06ffe0 0x7f535c1469f0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f535800e790 tx=0x7f535800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:51.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.564+0000 7f53527fc640 1 -- 172.21.15.40:0/3262028204 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5358009580 con 0x7f535c06ffe0 2023-10-18T09:08:51.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.564+0000 7f53527fc640 1 -- 172.21.15.40:0/3262028204 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f535800cdb0 con 0x7f535c06ffe0 2023-10-18T09:08:51.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.564+0000 7f53527fc640 1 -- 172.21.15.40:0/3262028204 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5358010690 con 0x7f535c06ffe0 2023-10-18T09:08:51.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.564+0000 7f5363ec9640 1 -- 172.21.15.40:0/3262028204 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c1569f0 con 0x7f535c06ffe0 2023-10-18T09:08:51.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.564+0000 7f5363ec9640 1 -- 172.21.15.40:0/3262028204 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535c146f30 con 0x7f535c06ffe0 2023-10-18T09:08:51.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.565+0000 7f5363ec9640 1 -- 172.21.15.40:0/3262028204 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535c066fd0 con 0x7f535c06ffe0 2023-10-18T09:08:51.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.566+0000 7f53527fc640 1 -- 172.21.15.40:0/3262028204 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f53580107f0 con 0x7f535c06ffe0 2023-10-18T09:08:51.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.567+0000 7f53527fc640 1 --2- 172.21.15.40:0/3262028204 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f534006ae40 0x7f534006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:51.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.567+0000 7f53527fc640 1 -- 172.21.15.40:0/3262028204 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5358014070 con 0x7f535c06ffe0 2023-10-18T09:08:51.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.568+0000 7f5361464640 1 --2- 172.21.15.40:0/3262028204 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f534006ae40 0x7f534006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:51.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.569+0000 7f5361464640 1 --2- 172.21.15.40:0/3262028204 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f534006ae40 0x7f534006d300 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f534c036000 tx=0x7f534c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:51.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.572+0000 7f53527fc640 1 -- 172.21.15.40:0/3262028204 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5358051ca0 con 0x7f535c06ffe0 2023-10-18T09:08:51.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:51.773+0000 7f5363ec9640 1 -- 172.21.15.40:0/3262028204 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f535c0aa950 con 0x7f535c06ffe0 2023-10-18T09:08:53.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.239+0000 7f53527fc640 1 -- 172.21.15.40:0/3262028204 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v192) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5358055130 con 0x7f535c06ffe0 2023-10-18T09:08:53.243 INFO:teuthology.orchestra.run.smithi040.stderr: removed all blocklist entries 2023-10-18T09:08:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.244+0000 7f5363ec9640 1 -- 172.21.15.40:0/3262028204 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f534006ae40 msgr2=0x7f534006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.244+0000 7f5363ec9640 1 --2- 172.21.15.40:0/3262028204 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f534006ae40 0x7f534006d300 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f534c036000 tx=0x7f534c004730 comp rx=0 tx=0).stop 2023-10-18T09:08:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.244+0000 7f5363ec9640 1 -- 172.21.15.40:0/3262028204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f535c06ffe0 msgr2=0x7f535c1469f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.244+0000 7f5363ec9640 1 --2- 172.21.15.40:0/3262028204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f535c06ffe0 0x7f535c1469f0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f535800e790 tx=0x7f535800ec50 comp rx=0 tx=0).stop 2023-10-18T09:08:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.245+0000 7f5363ec9640 1 -- 172.21.15.40:0/3262028204 shutdown_connections 2023-10-18T09:08:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.245+0000 7f5363ec9640 1 --2- 172.21.15.40:0/3262028204 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f534006ae40 0x7f534006d300 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.245+0000 7f5363ec9640 1 --2- 172.21.15.40:0/3262028204 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f535c15eaa0 0x7f535c0ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:53.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.245+0000 7f5363ec9640 1 --2- 172.21.15.40:0/3262028204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f535c0b1b90 0x7f535c0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:53.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.245+0000 7f5363ec9640 1 --2- 172.21.15.40:0/3262028204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f535c06ffe0 0x7f535c1469f0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:53.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.245+0000 7f5363ec9640 1 -- 172.21.15.40:0/3262028204 >> 172.21.15.40:0/3262028204 conn(0x7f535c06cee0 msgr2=0x7f535c158290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:53.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.245+0000 7f5363ec9640 1 -- 172.21.15.40:0/3262028204 shutdown_connections 2023-10-18T09:08:53.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.245+0000 7f5363ec9640 1 -- 172.21.15.40:0/3262028204 wait complete. 2023-10-18T09:08:53.268 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-18T09:08:53.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.492+0000 7f5ddcf5e640 1 -- 172.21.15.40:0/2780376029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd80ca160 msgr2=0x7f5dd80b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:53.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.492+0000 7f5ddcf5e640 1 --2- 172.21.15.40:0/2780376029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd80ca160 0x7f5dd80b0a80 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f5dd806f590 tx=0x7f5dcc02bdb0 comp rx=0 tx=0).stop 2023-10-18T09:08:53.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.492+0000 7f5ddcf5e640 1 -- 172.21.15.40:0/2780376029 shutdown_connections 2023-10-18T09:08:53.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.492+0000 7f5ddcf5e640 1 --2- 172.21.15.40:0/2780376029 >> v2:172.21.15.144:3300/0 conn(0x7f5dd81425a0 0x7f5dd80cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:53.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.492+0000 7f5ddcf5e640 1 --2- 172.21.15.40:0/2780376029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd80ca160 0x7f5dd80b0a80 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:53.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.492+0000 7f5ddcf5e640 1 -- 172.21.15.40:0/2780376029 >> 172.21.15.40:0/2780376029 conn(0x7f5dd8069720 msgr2=0x7f5dd8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:53.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.493+0000 7f5ddcf5e640 1 -- 172.21.15.40:0/2780376029 shutdown_connections 2023-10-18T09:08:53.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.493+0000 7f5ddcf5e640 1 -- 172.21.15.40:0/2780376029 wait complete. 2023-10-18T09:08:53.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.494+0000 7f5ddcf5e640 1 Processor -- start 2023-10-18T09:08:53.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.494+0000 7f5ddcf5e640 1 -- start start 2023-10-18T09:08:53.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.495+0000 7f5ddcf5e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd81420f0 0x7f5dd813dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:53.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.495+0000 7f5ddcf5e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5dd81425a0 0x7f5dd80708c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:53.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.495+0000 7f5ddcf5e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5dd8159950 0x7f5dd815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:53.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.495+0000 7f5ddcf5e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd815a9f0 con 0x7f5dd81420f0 2023-10-18T09:08:53.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.495+0000 7f5ddcf5e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd8159de0 con 0x7f5dd8159950 2023-10-18T09:08:53.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.495+0000 7f5ddcf5e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd815a0c0 con 0x7f5dd81425a0 2023-10-18T09:08:53.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.496+0000 7f5dd77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd81420f0 0x7f5dd813dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:53.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.496+0000 7f5dd7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5dd8159950 0x7f5dd815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:53.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.496+0000 7f5dd77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd81420f0 0x7f5dd813dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50994/0 (socket says 172.21.15.40:50994) 2023-10-18T09:08:53.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.496+0000 7f5dd77fe640 1 -- 172.21.15.40:0/1513291793 learned_addr learned my addr 172.21.15.40:0/1513291793 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:53.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.496+0000 7f5dd6ffd640 1 --2- 172.21.15.40:0/1513291793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5dd81425a0 0x7f5dd80708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:53.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.496+0000 7f5dd77fe640 1 -- 172.21.15.40:0/1513291793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5dd81425a0 msgr2=0x7f5dd80708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:53.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.496+0000 7f5dd77fe640 1 --2- 172.21.15.40:0/1513291793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5dd81425a0 0x7f5dd80708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:53.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.496+0000 7f5dd77fe640 1 -- 172.21.15.40:0/1513291793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5dd8159950 msgr2=0x7f5dd815ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:53.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.496+0000 7f5dd77fe640 1 --2- 172.21.15.40:0/1513291793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5dd8159950 0x7f5dd815ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:53.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.496+0000 7f5dd77fe640 1 -- 172.21.15.40:0/1513291793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc009560 con 0x7f5dd81420f0 2023-10-18T09:08:53.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.497+0000 7f5dd77fe640 1 --2- 172.21.15.40:0/1513291793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd81420f0 0x7f5dd813dc40 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800c430 tx=0x7f5dc800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:53.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.497+0000 7f5dbffff640 1 -- 172.21.15.40:0/1513291793 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8011790 con 0x7f5dd81420f0 2023-10-18T09:08:53.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.497+0000 7f5dbffff640 1 -- 172.21.15.40:0/1513291793 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5dc80119e0 con 0x7f5dd81420f0 2023-10-18T09:08:53.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.497+0000 7f5ddcf5e640 1 -- 172.21.15.40:0/1513291793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd815a3a0 con 0x7f5dd81420f0 2023-10-18T09:08:53.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.497+0000 7f5dbffff640 1 -- 172.21.15.40:0/1513291793 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8011e30 con 0x7f5dd81420f0 2023-10-18T09:08:53.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.497+0000 7f5ddcf5e640 1 -- 172.21.15.40:0/1513291793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd813e180 con 0x7f5dd81420f0 2023-10-18T09:08:53.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.498+0000 7f5ddcf5e640 1 -- 172.21.15.40:0/1513291793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd80c98c0 con 0x7f5dd81420f0 2023-10-18T09:08:53.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.499+0000 7f5dbffff640 1 -- 172.21.15.40:0/1513291793 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5dc801c460 con 0x7f5dd81420f0 2023-10-18T09:08:53.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.500+0000 7f5dbffff640 1 --2- 172.21.15.40:0/1513291793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5dc406ae40 0x7f5dc406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:53.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.500+0000 7f5dbffff640 1 -- 172.21.15.40:0/1513291793 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5dc80893b0 con 0x7f5dd81420f0 2023-10-18T09:08:53.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.501+0000 7f5dd6ffd640 1 --2- 172.21.15.40:0/1513291793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5dc406ae40 0x7f5dc406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:53.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.502+0000 7f5dd6ffd640 1 --2- 172.21.15.40:0/1513291793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5dc406ae40 0x7f5dc406d300 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc035000 tx=0x7f5dcc004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:53.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.505+0000 7f5dbffff640 1 -- 172.21.15.40:0/1513291793 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5dc8054850 con 0x7f5dd81420f0 2023-10-18T09:08:53.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.759+0000 7f5ddcf5e640 1 -- 172.21.15.40:0/1513291793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5dd813e310 con 0x7f5dd81420f0 2023-10-18T09:08:53.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.760+0000 7f5dbffff640 1 -- 172.21.15.40:0/1513291793 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5dc8057e70 con 0x7f5dd81420f0 2023-10-18T09:08:53.764 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:08:53.764 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:08:53.764 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_dump": [ 2023-10-18T09:08:53.764 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.764 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.a", 2023-10-18T09:08:53.765 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lYsb5CxAAGButZ6+Usc62kJ0ksxZrYw==", 2023-10-18T09:08:53.765 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:08:53.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:08:53.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:08:53.766 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:08:53.766 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.766 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.766 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.766 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.b", 2023-10-18T09:08:53.767 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lAqlhMBAApAhxCYB9d9bz1GibADDRSQ==", 2023-10-18T09:08:53.767 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:08:53.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:08:53.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:08:53.768 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:08:53.768 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.768 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.768 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.768 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.c", 2023-10-18T09:08:53.768 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBOnS9lPVh+HhAAzucSx+ksPDBkqJGa8+pyJw==", 2023-10-18T09:08:53.769 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:08:53.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:08:53.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:08:53.769 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:08:53.769 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.770 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.770 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.770 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mds.d", 2023-10-18T09:08:53.770 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lP7QCBhAAog7qDCgGVv0T7WKsRkATkA==", 2023-10-18T09:08:53.770 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:08:53.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:08:53.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:08:53.771 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:08:53.771 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.771 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.772 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.772 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.0", 2023-10-18T09:08:53.772 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lVx+hAhAA6ZpIyZKhbqy8tV84piGILA==", 2023-10-18T09:08:53.772 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.772 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:08:53.772 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:08:53.773 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:08:53.773 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.773 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.773 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.773 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.1", 2023-10-18T09:08:53.773 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lk9LwExAA7zFzQOn2hcvnck2cOZZ7AA==", 2023-10-18T09:08:53.774 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:08:53.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:08:53.774 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:08:53.774 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.775 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.775 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.775 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.2", 2023-10-18T09:08:53.775 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9lK930JRAA1GyKRkIpXPHHGqY8owYWRQ==", 2023-10-18T09:08:53.775 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:08:53.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:08:53.776 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:08:53.776 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.776 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.776 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.776 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.3", 2023-10-18T09:08:53.777 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBUnS9liBWyNRAAMHBBdG5nbrGQupqvUtuKMA==", 2023-10-18T09:08:53.777 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:08:53.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:08:53.777 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:08:53.777 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.778 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.778 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.778 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.4", 2023-10-18T09:08:53.778 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9loDoOAhAAjHZub4wnB7dCr86rxAtI5g==", 2023-10-18T09:08:53.778 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:08:53.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:08:53.779 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:08:53.779 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.779 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.779 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.780 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.5", 2023-10-18T09:08:53.780 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lhqNfERAAw410g3DwSXrK23YkxVBFVw==", 2023-10-18T09:08:53.780 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:08:53.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:08:53.781 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:08:53.781 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.781 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.781 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.781 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.6", 2023-10-18T09:08:53.781 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9lRRScIRAAg0pmfxCQBCPYodj1Lko+Lw==", 2023-10-18T09:08:53.782 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:08:53.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:08:53.782 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:08:53.782 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.783 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.783 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "osd.7", 2023-10-18T09:08:53.783 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBZnS9ld1dXMhAAyovlrKOhVuynEiuPTgFcjQ==", 2023-10-18T09:08:53.783 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow profile osd", 2023-10-18T09:08:53.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile osd", 2023-10-18T09:08:53.784 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:08:53.784 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.784 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.784 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.0", 2023-10-18T09:08:53.785 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w==", 2023-10-18T09:08:53.785 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:08:53.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow r", 2023-10-18T09:08:53.785 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-18T09:08:53.785 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.786 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.786 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.1", 2023-10-18T09:08:53.786 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lAB3oKhAAD43WFdC23WGF6WUi4JKsvg==", 2023-10-18T09:08:53.786 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:08:53.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:08:53.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:08:53.787 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:08:53.787 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.788 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.2", 2023-10-18T09:08:53.788 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lu57fHxAAdzsVU2KKsWRkaARRS8TGjw==", 2023-10-18T09:08:53.788 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:08:53.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:08:53.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:08:53.789 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:08:53.789 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.789 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.3", 2023-10-18T09:08:53.790 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBPnS9lygWsNRAA1OziXxk7hBtTw2Aq8OlSKQ==", 2023-10-18T09:08:53.790 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow", 2023-10-18T09:08:53.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow r", 2023-10-18T09:08:53.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow rw", 2023-10-18T09:08:53.791 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow rwx" 2023-10-18T09:08:53.791 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.791 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.791 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "client.admin", 2023-10-18T09:08:53.791 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBMnS9lnrUjNhAAHP2eezRHF4/Jgox7iPwy7g==", 2023-10-18T09:08:53.792 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:08:53.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": "allow *", 2023-10-18T09:08:53.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow *", 2023-10-18T09:08:53.792 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:08:53.792 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.793 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.x", 2023-10-18T09:08:53.793 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9lCf/xOhAAJHVoKIswWUcVGKQpiaJ12A==", 2023-10-18T09:08:53.793 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:08:53.794 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T09:08:53.794 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:08:53.794 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.794 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:08:53.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:53.794 INFO:teuthology.orchestra.run.smithi040.stdout: "entity": "mgr.y", 2023-10-18T09:08:53.795 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "AQBNnS9loxRXLhAAQK1n1/4PB35umHxOKQzz7Q==", 2023-10-18T09:08:53.795 INFO:teuthology.orchestra.run.smithi040.stdout: "caps": { 2023-10-18T09:08:53.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": "allow *", 2023-10-18T09:08:53.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": "allow profile mgr", 2023-10-18T09:08:53.795 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": "allow *" 2023-10-18T09:08:53.795 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.796 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:53.796 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:08:53.796 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:08:53.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.763+0000 7f5ddcf5e640 1 -- 172.21.15.40:0/1513291793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5dc406ae40 msgr2=0x7f5dc406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:53.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.763+0000 7f5ddcf5e640 1 --2- 172.21.15.40:0/1513291793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5dc406ae40 0x7f5dc406d300 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc035000 tx=0x7f5dcc004730 comp rx=0 tx=0).stop 2023-10-18T09:08:53.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.763+0000 7f5ddcf5e640 1 -- 172.21.15.40:0/1513291793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd81420f0 msgr2=0x7f5dd813dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:53.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.763+0000 7f5ddcf5e640 1 --2- 172.21.15.40:0/1513291793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd81420f0 0x7f5dd813dc40 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800c430 tx=0x7f5dc800c8f0 comp rx=0 tx=0).stop 2023-10-18T09:08:53.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.763+0000 7f5ddcf5e640 1 -- 172.21.15.40:0/1513291793 shutdown_connections 2023-10-18T09:08:53.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.763+0000 7f5ddcf5e640 1 --2- 172.21.15.40:0/1513291793 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5dc406ae40 0x7f5dc406d300 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:53.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.763+0000 7f5ddcf5e640 1 --2- 172.21.15.40:0/1513291793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5dd8159950 0x7f5dd815ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:53.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.763+0000 7f5ddcf5e640 1 --2- 172.21.15.40:0/1513291793 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5dd81425a0 0x7f5dd80708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.763+0000 7f5ddcf5e640 1 --2- 172.21.15.40:0/1513291793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dd81420f0 0x7f5dd813dc40 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.763+0000 7f5ddcf5e640 1 -- 172.21.15.40:0/1513291793 >> 172.21.15.40:0/1513291793 conn(0x7f5dd8069720 msgr2=0x7f5dd80aa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.763+0000 7f5ddcf5e640 1 -- 172.21.15.40:0/1513291793 shutdown_connections 2023-10-18T09:08:53.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:53.763+0000 7f5ddcf5e640 1 -- 172.21.15.40:0/1513291793 wait complete. 2023-10-18T09:08:53.800 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:08:54.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.022+0000 7f43dffff640 1 -- 172.21.15.40:0/3182777507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00d0020 msgr2=0x7f43e00ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:54.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.022+0000 7f43dffff640 1 --2- 172.21.15.40:0/3182777507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00d0020 0x7f43e00ae960 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f43e00b0f20 tx=0x7f43d802bfe0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.022+0000 7f43dffff640 1 -- 172.21.15.40:0/3182777507 shutdown_connections 2023-10-18T09:08:54.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.022+0000 7f43dffff640 1 --2- 172.21.15.40:0/3182777507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00d0020 0x7f43e00ae960 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.022+0000 7f43dffff640 1 --2- 172.21.15.40:0/3182777507 >> v2:172.21.15.144:3300/0 conn(0x7f43e015b640 0x7f43e00d1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.022+0000 7f43dffff640 1 -- 172.21.15.40:0/3182777507 >> 172.21.15.40:0/3182777507 conn(0x7f43e008ad00 msgr2=0x7f43e009a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:54.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.022+0000 7f43dffff640 1 -- 172.21.15.40:0/3182777507 shutdown_connections 2023-10-18T09:08:54.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.022+0000 7f43dffff640 1 -- 172.21.15.40:0/3182777507 wait complete. 2023-10-18T09:08:54.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.024+0000 7f43dffff640 1 Processor -- start 2023-10-18T09:08:54.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.024+0000 7f43dffff640 1 -- start start 2023-10-18T09:08:54.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.025+0000 7f43dffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43e00b01e0 0x7f43e00b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:54.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.025+0000 7f43dffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00d0020 0x7f43e01f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:54.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.025+0000 7f43dffff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43e015b640 0x7f43e01585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:54.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.025+0000 7f43dffff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00b6bc0 con 0x7f43e00d0020 2023-10-18T09:08:54.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.025+0000 7f43dffff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00b6600 con 0x7f43e00b01e0 2023-10-18T09:08:54.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.025+0000 7f43dffff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e00b68e0 con 0x7f43e015b640 2023-10-18T09:08:54.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.025+0000 7f43d7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00d0020 0x7f43e01f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:54.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.025+0000 7f43d7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00d0020 0x7f43e01f4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51014/0 (socket says 172.21.15.40:51014) 2023-10-18T09:08:54.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.025+0000 7f43d7fff640 1 -- 172.21.15.40:0/2878016962 learned_addr learned my addr 172.21.15.40:0/2878016962 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:54.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.026+0000 7f43deffd640 1 --2- 172.21.15.40:0/2878016962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43e00b01e0 0x7f43e00b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:54.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.026+0000 7f43df7fe640 1 --2- 172.21.15.40:0/2878016962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43e015b640 0x7f43e01585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:54.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.026+0000 7f43d7fff640 1 -- 172.21.15.40:0/2878016962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43e015b640 msgr2=0x7f43e01585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:54.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.026+0000 7f43d7fff640 1 --2- 172.21.15.40:0/2878016962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43e015b640 0x7f43e01585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.026+0000 7f43d7fff640 1 -- 172.21.15.40:0/2878016962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43e00b01e0 msgr2=0x7f43e00b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:54.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.026+0000 7f43d7fff640 1 --2- 172.21.15.40:0/2878016962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43e00b01e0 0x7f43e00b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.026+0000 7f43d7fff640 1 -- 172.21.15.40:0/2878016962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d8009840 con 0x7f43e00d0020 2023-10-18T09:08:54.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.026+0000 7f43deffd640 1 --2- 172.21.15.40:0/2878016962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43e00b01e0 0x7f43e00b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:54.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.026+0000 7f43df7fe640 1 --2- 172.21.15.40:0/2878016962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43e015b640 0x7f43e01585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:54.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.026+0000 7f43d7fff640 1 --2- 172.21.15.40:0/2878016962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00d0020 0x7f43e01f4fd0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c490 tx=0x7f43c800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:54.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.026+0000 7f43d77fe640 1 -- 172.21.15.40:0/2878016962 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8012830 con 0x7f43e00d0020 2023-10-18T09:08:54.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.026+0000 7f43d77fe640 1 -- 172.21.15.40:0/2878016962 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f43c8012a80 con 0x7f43e00d0020 2023-10-18T09:08:54.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.026+0000 7f43dffff640 1 -- 172.21.15.40:0/2878016962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e009e970 con 0x7f43e00d0020 2023-10-18T09:08:54.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.026+0000 7f43d77fe640 1 -- 172.21.15.40:0/2878016962 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8011380 con 0x7f43e00d0020 2023-10-18T09:08:54.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.026+0000 7f43dffff640 1 -- 172.21.15.40:0/2878016962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e009ef30 con 0x7f43e00d0020 2023-10-18T09:08:54.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.028+0000 7f43dffff640 1 -- 172.21.15.40:0/2878016962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e0071da0 con 0x7f43e00d0020 2023-10-18T09:08:54.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.028+0000 7f43d77fe640 1 -- 172.21.15.40:0/2878016962 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f43c80114e0 con 0x7f43e00d0020 2023-10-18T09:08:54.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.029+0000 7f43d77fe640 1 --2- 172.21.15.40:0/2878016962 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f43c006ae40 0x7f43c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:54.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.029+0000 7f43d77fe640 1 -- 172.21.15.40:0/2878016962 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f43c8089290 con 0x7f43e00d0020 2023-10-18T09:08:54.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.030+0000 7f43deffd640 1 --2- 172.21.15.40:0/2878016962 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f43c006ae40 0x7f43c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:54.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.031+0000 7f43deffd640 1 --2- 172.21.15.40:0/2878016962 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f43c006ae40 0x7f43c006d300 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009aa0 tx=0x7f43d0005db0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:54.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.033+0000 7f43d77fe640 1 -- 172.21.15.40:0/2878016962 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f43c80545f0 con 0x7f43e00d0020 2023-10-18T09:08:54.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.230+0000 7f43dffff640 1 -- 172.21.15.40:0/2878016962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43e009e7d0 con 0x7f43e00d0020 2023-10-18T09:08:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.232+0000 7f43d77fe640 1 -- 172.21.15.40:0/2878016962 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+12880 (secure 0 0 0) 0x7f43e009e7d0 con 0x7f43e00d0020 2023-10-18T09:08:54.235 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:08:54.235 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":192,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:08:53.237674+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:08:54.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.235+0000 7f43dffff640 1 -- 172.21.15.40:0/2878016962 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f43c006ae40 msgr2=0x7f43c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:54.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.235+0000 7f43dffff640 1 --2- 172.21.15.40:0/2878016962 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f43c006ae40 0x7f43c006d300 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009aa0 tx=0x7f43d0005db0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.235+0000 7f43dffff640 1 -- 172.21.15.40:0/2878016962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00d0020 msgr2=0x7f43e01f4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:54.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.235+0000 7f43dffff640 1 --2- 172.21.15.40:0/2878016962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00d0020 0x7f43e01f4fd0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c490 tx=0x7f43c800c950 comp rx=0 tx=0).stop 2023-10-18T09:08:54.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.235+0000 7f43dffff640 1 -- 172.21.15.40:0/2878016962 shutdown_connections 2023-10-18T09:08:54.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.235+0000 7f43dffff640 1 --2- 172.21.15.40:0/2878016962 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f43c006ae40 0x7f43c006d300 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.235+0000 7f43dffff640 1 --2- 172.21.15.40:0/2878016962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43e015b640 0x7f43e01585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.235+0000 7f43dffff640 1 --2- 172.21.15.40:0/2878016962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00d0020 0x7f43e01f4fd0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.235+0000 7f43dffff640 1 --2- 172.21.15.40:0/2878016962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43e00b01e0 0x7f43e00b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.235+0000 7f43dffff640 1 -- 172.21.15.40:0/2878016962 >> 172.21.15.40:0/2878016962 conn(0x7f43e008ad00 msgr2=0x7f43e015a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:54.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.235+0000 7f43dffff640 1 -- 172.21.15.40:0/2878016962 shutdown_connections 2023-10-18T09:08:54.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.235+0000 7f43dffff640 1 -- 172.21.15.40:0/2878016962 wait complete. 2023-10-18T09:08:54.258 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:08:54.258 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:08:54.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.488+0000 7fcdd7fff640 1 -- 172.21.15.40:0/2767745355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc4003970 msgr2=0x7fcdc4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:54.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.488+0000 7fcdd7fff640 1 --2- 172.21.15.40:0/2767745355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc4003970 0x7fcdc4003d90 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8006fd0 tx=0x7fcdc40057f0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.488+0000 7fcdd77fe640 1 -- 172.21.15.40:0/2767745355 reap_dead start 2023-10-18T09:08:54.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.488+0000 7fcdd7fff640 1 -- 172.21.15.40:0/2767745355 shutdown_connections 2023-10-18T09:08:54.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.488+0000 7fcdd7fff640 1 -- 172.21.15.40:0/2767745355 >> 172.21.15.40:0/2767745355 conn(0x7fcdd8069720 msgr2=0x7fcdd8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:54.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.488+0000 7fcdd7fff640 1 -- 172.21.15.40:0/2767745355 shutdown_connections 2023-10-18T09:08:54.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.488+0000 7fcdd7fff640 1 -- 172.21.15.40:0/2767745355 wait complete. 2023-10-18T09:08:54.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.490+0000 7fcdd7fff640 1 Processor -- start 2023-10-18T09:08:54.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.490+0000 7fcdd7fff640 1 -- start start 2023-10-18T09:08:54.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.491+0000 7fcdd7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc4003970 0x7fcdd8156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:54.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.491+0000 7fcdd7fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdd8156b70 0x7fcdd80cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:54.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.491+0000 7fcdd7fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdd81e5300 0x7fcdd81e5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:54.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.491+0000 7fcdd7fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd81e5cd0 con 0x7fcdc4003970 2023-10-18T09:08:54.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.491+0000 7fcdd7fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd80cadc0 con 0x7fcdd8156b70 2023-10-18T09:08:54.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.491+0000 7fcdd7fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd8157070 con 0x7fcdd81e5300 2023-10-18T09:08:54.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.492+0000 7fcdd6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc4003970 0x7fcdd8156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:54.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.492+0000 7fcdd6ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc4003970 0x7fcdd8156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51030/0 (socket says 172.21.15.40:51030) 2023-10-18T09:08:54.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.492+0000 7fcdd6ffd640 1 -- 172.21.15.40:0/825711306 learned_addr learned my addr 172.21.15.40:0/825711306 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:54.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.492+0000 7fcdcffff640 1 --2- 172.21.15.40:0/825711306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdd8156b70 0x7fcdd80cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:54.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.492+0000 7fcdd77fe640 1 --2- 172.21.15.40:0/825711306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdd81e5300 0x7fcdd81e5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:54.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.492+0000 7fcdd6ffd640 1 -- 172.21.15.40:0/825711306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdd81e5300 msgr2=0x7fcdd81e5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:54.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.492+0000 7fcdd6ffd640 1 --2- 172.21.15.40:0/825711306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdd81e5300 0x7fcdd81e5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.492+0000 7fcdd6ffd640 1 -- 172.21.15.40:0/825711306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdd8156b70 msgr2=0x7fcdd80cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:54.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.492+0000 7fcdd6ffd640 1 --2- 172.21.15.40:0/825711306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdd8156b70 0x7fcdd80cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.492+0000 7fcdd6ffd640 1 -- 172.21.15.40:0/825711306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd80cb080 con 0x7fcdc4003970 2023-10-18T09:08:54.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.492+0000 7fcdcffff640 1 --2- 172.21.15.40:0/825711306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdd8156b70 0x7fcdd80cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:54.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.492+0000 7fcdd77fe640 1 --2- 172.21.15.40:0/825711306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdd81e5300 0x7fcdd81e5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:54.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.493+0000 7fcdd6ffd640 1 --2- 172.21.15.40:0/825711306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc4003970 0x7fcdd8156570 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8000c00 tx=0x7fcdc8001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:54.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.493+0000 7fcdcf7fe640 1 -- 172.21.15.40:0/825711306 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc80062e0 con 0x7fcdc4003970 2023-10-18T09:08:54.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.493+0000 7fcdcf7fe640 1 -- 172.21.15.40:0/825711306 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcdc8006440 con 0x7fcdc4003970 2023-10-18T09:08:54.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.493+0000 7fcdcf7fe640 1 -- 172.21.15.40:0/825711306 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc80035f0 con 0x7fcdc4003970 2023-10-18T09:08:54.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.493+0000 7fcdd7fff640 1 -- 172.21.15.40:0/825711306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd80cb360 con 0x7fcdc4003970 2023-10-18T09:08:54.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.493+0000 7fcdd7fff640 1 -- 172.21.15.40:0/825711306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd80cc5b0 con 0x7fcdc4003970 2023-10-18T09:08:54.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.494+0000 7fcdd7fff640 1 -- 172.21.15.40:0/825711306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd80cd230 con 0x7fcdc4003970 2023-10-18T09:08:54.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.495+0000 7fcdcf7fe640 1 -- 172.21.15.40:0/825711306 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcdc8031030 con 0x7fcdc4003970 2023-10-18T09:08:54.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.496+0000 7fcdcf7fe640 1 --2- 172.21.15.40:0/825711306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcdb806ae70 0x7fcdb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:54.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.496+0000 7fcdcf7fe640 1 -- 172.21.15.40:0/825711306 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcdc80aafb0 con 0x7fcdc4003970 2023-10-18T09:08:54.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.497+0000 7fcdcffff640 1 --2- 172.21.15.40:0/825711306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcdb806ae70 0x7fcdb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:54.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.498+0000 7fcdcffff640 1 --2- 172.21.15.40:0/825711306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcdb806ae70 0x7fcdb806d330 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0005fd0 tx=0x7fcdc000e580 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:54.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.500+0000 7fcdcf7fe640 1 -- 172.21.15.40:0/825711306 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcdc80ddca0 con 0x7fcdc4003970 2023-10-18T09:08:54.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.690+0000 7fcdd7fff640 1 -- 172.21.15.40:0/825711306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcdd80cd230 con 0x7fcdc4003970 2023-10-18T09:08:54.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.691+0000 7fcdcf7fe640 1 -- 172.21.15.40:0/825711306 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v192) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcdc8079f70 con 0x7fcdc4003970 2023-10-18T09:08:54.695 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:08:54.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.693+0000 7fcdd7fff640 1 -- 172.21.15.40:0/825711306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcdb806ae70 msgr2=0x7fcdb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:54.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.693+0000 7fcdd7fff640 1 --2- 172.21.15.40:0/825711306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcdb806ae70 0x7fcdb806d330 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0005fd0 tx=0x7fcdc000e580 comp rx=0 tx=0).stop 2023-10-18T09:08:54.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.693+0000 7fcdd7fff640 1 -- 172.21.15.40:0/825711306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc4003970 msgr2=0x7fcdd8156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:54.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.693+0000 7fcdd7fff640 1 --2- 172.21.15.40:0/825711306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc4003970 0x7fcdd8156570 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8000c00 tx=0x7fcdc8001650 comp rx=0 tx=0).stop 2023-10-18T09:08:54.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.694+0000 7fcdd7fff640 1 -- 172.21.15.40:0/825711306 shutdown_connections 2023-10-18T09:08:54.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.694+0000 7fcdd7fff640 1 --2- 172.21.15.40:0/825711306 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fcdb806ae70 0x7fcdb806d330 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.694+0000 7fcdd7fff640 1 --2- 172.21.15.40:0/825711306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcdd81e5300 0x7fcdd81e5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.694+0000 7fcdd7fff640 1 --2- 172.21.15.40:0/825711306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcdd8156b70 0x7fcdd80cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.694+0000 7fcdd7fff640 1 --2- 172.21.15.40:0/825711306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdc4003970 0x7fcdd8156570 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.694+0000 7fcdd7fff640 1 -- 172.21.15.40:0/825711306 >> 172.21.15.40:0/825711306 conn(0x7fcdd8069720 msgr2=0x7fcdd80ada90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:54.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.694+0000 7fcdd7fff640 1 -- 172.21.15.40:0/825711306 shutdown_connections 2023-10-18T09:08:54.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.694+0000 7fcdd7fff640 1 -- 172.21.15.40:0/825711306 wait complete. 2023-10-18T09:08:54.716 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T09:08:54.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.942+0000 7f4b99975640 1 -- 172.21.15.40:0/1089068816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b940d0020 msgr2=0x7f4b940d1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:54.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.942+0000 7f4b99975640 1 --2- 172.21.15.40:0/1089068816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b940d0020 0x7f4b940d1a90 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f4b940ae440 tx=0x7f4b8002bc60 comp rx=0 tx=0).stop 2023-10-18T09:08:54.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.943+0000 7f4b99975640 1 -- 172.21.15.40:0/1089068816 shutdown_connections 2023-10-18T09:08:54.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.943+0000 7f4b99975640 1 --2- 172.21.15.40:0/1089068816 >> v2:172.21.15.144:3300/0 conn(0x7f4b940b01e0 0x7f4b940ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.943+0000 7f4b99975640 1 --2- 172.21.15.40:0/1089068816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b940d0020 0x7f4b940d1a90 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.943+0000 7f4b99975640 1 -- 172.21.15.40:0/1089068816 >> 172.21.15.40:0/1089068816 conn(0x7f4b9408ad00 msgr2=0x7f4b9409a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:54.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.943+0000 7f4b99975640 1 -- 172.21.15.40:0/1089068816 shutdown_connections 2023-10-18T09:08:54.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.943+0000 7f4b99975640 1 -- 172.21.15.40:0/1089068816 wait complete. 2023-10-18T09:08:54.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.945+0000 7f4b99975640 1 Processor -- start 2023-10-18T09:08:54.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.945+0000 7f4b99975640 1 -- start start 2023-10-18T09:08:54.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.946+0000 7f4b99975640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b940b01e0 0x7f4b9415dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:54.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.946+0000 7f4b99975640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b940d0020 0x7f4b940b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:54.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.946+0000 7f4b99975640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b9415b640 0x7f4b94159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:54.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.946+0000 7f4b99975640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b940b6c60 con 0x7f4b9415b640 2023-10-18T09:08:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.946+0000 7f4b99975640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b940b6050 con 0x7f4b940d0020 2023-10-18T09:08:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.946+0000 7f4b99975640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b940b6330 con 0x7f4b940b01e0 2023-10-18T09:08:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.946+0000 7f4b99174640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b9415b640 0x7f4b94159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.946+0000 7f4b99174640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b9415b640 0x7f4b94159910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51108/0 (socket says 172.21.15.40:51108) 2023-10-18T09:08:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.946+0000 7f4b99174640 1 -- 172.21.15.40:0/3926065573 learned_addr learned my addr 172.21.15.40:0/3926065573 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:54.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.946+0000 7f4b98973640 1 --2- 172.21.15.40:0/3926065573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b940b01e0 0x7f4b9415dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:54.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.946+0000 7f4b93fff640 1 --2- 172.21.15.40:0/3926065573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b940d0020 0x7f4b940b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:54.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.947+0000 7f4b99174640 1 -- 172.21.15.40:0/3926065573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b940b01e0 msgr2=0x7f4b9415dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:54.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.947+0000 7f4b99174640 1 --2- 172.21.15.40:0/3926065573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b940b01e0 0x7f4b9415dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.947+0000 7f4b99174640 1 -- 172.21.15.40:0/3926065573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b940d0020 msgr2=0x7f4b940b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:54.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.947+0000 7f4b99174640 1 --2- 172.21.15.40:0/3926065573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b940d0020 0x7f4b940b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:54.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.947+0000 7f4b99174640 1 -- 172.21.15.40:0/3926065573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b80009490 con 0x7f4b9415b640 2023-10-18T09:08:54.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.947+0000 7f4b99174640 1 --2- 172.21.15.40:0/3926065573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b9415b640 0x7f4b94159910 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00eaa0 tx=0x7f4b8c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:54.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.947+0000 7f4b917fa640 1 -- 172.21.15.40:0/3926065573 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c00cdb0 con 0x7f4b9415b640 2023-10-18T09:08:54.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.947+0000 7f4b99975640 1 -- 172.21.15.40:0/3926065573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b940b6610 con 0x7f4b9415b640 2023-10-18T09:08:54.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.947+0000 7f4b917fa640 1 -- 172.21.15.40:0/3926065573 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4b8c004b20 con 0x7f4b9415b640 2023-10-18T09:08:54.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.947+0000 7f4b917fa640 1 -- 172.21.15.40:0/3926065573 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c005020 con 0x7f4b9415b640 2023-10-18T09:08:54.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.947+0000 7f4b99975640 1 -- 172.21.15.40:0/3926065573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b94159e50 con 0x7f4b9415b640 2023-10-18T09:08:54.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.949+0000 7f4b99975640 1 -- 172.21.15.40:0/3926065573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b94071da0 con 0x7f4b9415b640 2023-10-18T09:08:54.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.950+0000 7f4b917fa640 1 -- 172.21.15.40:0/3926065573 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4b8c005180 con 0x7f4b9415b640 2023-10-18T09:08:54.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.951+0000 7f4b917fa640 1 --2- 172.21.15.40:0/3926065573 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b7806ae70 0x7f4b7806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:54.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.951+0000 7f4b917fa640 1 -- 172.21.15.40:0/3926065573 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4b8c089e50 con 0x7f4b9415b640 2023-10-18T09:08:54.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.952+0000 7f4b98973640 1 --2- 172.21.15.40:0/3926065573 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b7806ae70 0x7f4b7806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:54.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.953+0000 7f4b98973640 1 --2- 172.21.15.40:0/3926065573 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b7806ae70 0x7f4b7806d330 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4b88007fd0 tx=0x7f4b88009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:54.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:54.959+0000 7f4b917fa640 1 -- 172.21.15.40:0/3926065573 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4b8c055130 con 0x7f4b9415b640 2023-10-18T09:08:55.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.154+0000 7f4b99975640 1 -- 172.21.15.40:0/3926065573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4b94066e00 con 0x7f4b9415b640 2023-10-18T09:08:55.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.155+0000 7f4b917fa640 1 -- 172.21.15.40:0/3926065573 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v192) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4b8c058750 con 0x7f4b9415b640 2023-10-18T09:08:55.159 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:08:55.159 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-10-18T09:08:55.159 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:08:55.159 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-10-18T09:08:55.159 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-10-18T09:08:55.159 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:08:55.160 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-10-18T09:08:55.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.157+0000 7f4b99975640 1 -- 172.21.15.40:0/3926065573 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b7806ae70 msgr2=0x7f4b7806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.157+0000 7f4b99975640 1 --2- 172.21.15.40:0/3926065573 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b7806ae70 0x7f4b7806d330 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4b88007fd0 tx=0x7f4b88009040 comp rx=0 tx=0).stop 2023-10-18T09:08:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.157+0000 7f4b99975640 1 -- 172.21.15.40:0/3926065573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b9415b640 msgr2=0x7f4b94159910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.157+0000 7f4b99975640 1 --2- 172.21.15.40:0/3926065573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b9415b640 0x7f4b94159910 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00eaa0 tx=0x7f4b8c00ef60 comp rx=0 tx=0).stop 2023-10-18T09:08:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.158+0000 7f4b99975640 1 -- 172.21.15.40:0/3926065573 shutdown_connections 2023-10-18T09:08:55.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.158+0000 7f4b99975640 1 --2- 172.21.15.40:0/3926065573 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b7806ae70 0x7f4b7806d330 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:55.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.158+0000 7f4b99975640 1 --2- 172.21.15.40:0/3926065573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b9415b640 0x7f4b94159910 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:55.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.158+0000 7f4b99975640 1 --2- 172.21.15.40:0/3926065573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b940d0020 0x7f4b940b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:55.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.158+0000 7f4b99975640 1 --2- 172.21.15.40:0/3926065573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b940b01e0 0x7f4b9415dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:55.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.158+0000 7f4b99975640 1 -- 172.21.15.40:0/3926065573 >> 172.21.15.40:0/3926065573 conn(0x7f4b9408ad00 msgr2=0x7f4b940cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.158+0000 7f4b99975640 1 -- 172.21.15.40:0/3926065573 shutdown_connections 2023-10-18T09:08:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.158+0000 7f4b99975640 1 -- 172.21.15.40:0/3926065573 wait complete. 2023-10-18T09:08:55.180 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T09:08:55.180 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T09:08:55.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.411+0000 7f577e7f0640 1 -- 172.21.15.40:0/3517920800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57780c9850 msgr2=0x7f57780c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.411+0000 7f577e7f0640 1 --2- 172.21.15.40:0/3517920800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57780c9850 0x7f57780c9ca0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f576c005fd0 tx=0x7f576c02c0f0 comp rx=0 tx=0).stop 2023-10-18T09:08:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.411+0000 7f577e7f0640 1 -- 172.21.15.40:0/3517920800 shutdown_connections 2023-10-18T09:08:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.411+0000 7f577e7f0640 1 --2- 172.21.15.40:0/3517920800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57780c9850 0x7f57780c9ca0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.411+0000 7f577e7f0640 1 -- 172.21.15.40:0/3517920800 >> 172.21.15.40:0/3517920800 conn(0x7f57780aec00 msgr2=0x7f57780aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:55.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.411+0000 7f577e7f0640 1 -- 172.21.15.40:0/3517920800 shutdown_connections 2023-10-18T09:08:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.411+0000 7f577e7f0640 1 -- 172.21.15.40:0/3517920800 wait complete. 2023-10-18T09:08:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.413+0000 7f577e7f0640 1 Processor -- start 2023-10-18T09:08:55.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.413+0000 7f577e7f0640 1 -- start start 2023-10-18T09:08:55.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.414+0000 7f577e7f0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f577806d680 0x7f5778150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.414+0000 7f577e7f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57780c9850 0x7f577813e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.414+0000 7f577e7f0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57780cb620 0x7f577806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.414+0000 7f577e7f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577814efa0 con 0x7f57780c9850 2023-10-18T09:08:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.414+0000 7f577e7f0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577813c230 con 0x7f57780cb620 2023-10-18T09:08:55.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.414+0000 7f577e7f0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577813eab0 con 0x7f577806d680 2023-10-18T09:08:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.415+0000 7f57777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57780c9850 0x7f577813e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.415+0000 7f57777fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57780c9850 0x7f577813e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51124/0 (socket says 172.21.15.40:51124) 2023-10-18T09:08:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.415+0000 7f57777fe640 1 -- 172.21.15.40:0/2837765665 learned_addr learned my addr 172.21.15.40:0/2837765665 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.415+0000 7f5777fff640 1 --2- 172.21.15.40:0/2837765665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f577806d680 0x7f5778150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:55.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.415+0000 7f577cd8d640 1 --2- 172.21.15.40:0/2837765665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57780cb620 0x7f577806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.416+0000 7f57777fe640 1 -- 172.21.15.40:0/2837765665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f577806d680 msgr2=0x7f5778150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.416+0000 7f57777fe640 1 --2- 172.21.15.40:0/2837765665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f577806d680 0x7f5778150b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.416+0000 7f57777fe640 1 -- 172.21.15.40:0/2837765665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57780cb620 msgr2=0x7f577806ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.416+0000 7f57777fe640 1 --2- 172.21.15.40:0/2837765665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57780cb620 0x7f577806ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:55.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.416+0000 7f57777fe640 1 -- 172.21.15.40:0/2837765665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c009840 con 0x7f57780c9850 2023-10-18T09:08:55.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.416+0000 7f57777fe640 1 --2- 172.21.15.40:0/2837765665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57780c9850 0x7f577813e570 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f576c036040 tx=0x7f576c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:55.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.416+0000 7f5774ff9640 1 -- 172.21.15.40:0/2837765665 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c0046f0 con 0x7f57780c9850 2023-10-18T09:08:55.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.416+0000 7f5774ff9640 1 -- 172.21.15.40:0/2837765665 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f576c0345a0 con 0x7f57780c9850 2023-10-18T09:08:55.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.416+0000 7f5774ff9640 1 -- 172.21.15.40:0/2837765665 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c0349b0 con 0x7f57780c9850 2023-10-18T09:08:55.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.416+0000 7f577e7f0640 1 -- 172.21.15.40:0/2837765665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577813ed50 con 0x7f57780c9850 2023-10-18T09:08:55.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.417+0000 7f577e7f0640 1 -- 172.21.15.40:0/2837765665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577813f310 con 0x7f57780c9850 2023-10-18T09:08:55.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.418+0000 7f577e7f0640 1 -- 172.21.15.40:0/2837765665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5778066fd0 con 0x7f57780c9850 2023-10-18T09:08:55.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.419+0000 7f5774ff9640 1 -- 172.21.15.40:0/2837765665 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f576c034b10 con 0x7f57780c9850 2023-10-18T09:08:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.420+0000 7f5774ff9640 1 --2- 172.21.15.40:0/2837765665 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f576406ae40 0x7f576406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.420+0000 7f5774ff9640 1 -- 172.21.15.40:0/2837765665 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f576c0ac460 con 0x7f57780c9850 2023-10-18T09:08:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.420+0000 7f5777fff640 1 --2- 172.21.15.40:0/2837765665 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f576406ae40 0x7f576406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.421+0000 7f5777fff640 1 --2- 172.21.15.40:0/2837765665 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f576406ae40 0x7f576406d300 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5768001270 tx=0x7f5768007b40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:55.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.424+0000 7f5774ff9640 1 -- 172.21.15.40:0/2837765665 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f576c077860 con 0x7f57780c9850 2023-10-18T09:08:55.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:55.618+0000 7f577e7f0640 1 -- 172.21.15.40:0/2837765665 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f57780cacb0 con 0x7f57780c9850 2023-10-18T09:08:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.234+0000 7f5774ff9640 1 -- 172.21.15.40:0/2837765665 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v193) v1 ==== 138+0+0 (secure 0 0 0) 0x7f576c07ae80 con 0x7f57780c9850 2023-10-18T09:08:56.238 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-10-18T09:08:56.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.239+0000 7f577e7f0640 1 -- 172.21.15.40:0/2837765665 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f576406ae40 msgr2=0x7f576406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:56.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.239+0000 7f577e7f0640 1 --2- 172.21.15.40:0/2837765665 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f576406ae40 0x7f576406d300 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5768001270 tx=0x7f5768007b40 comp rx=0 tx=0).stop 2023-10-18T09:08:56.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.239+0000 7f577e7f0640 1 -- 172.21.15.40:0/2837765665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57780c9850 msgr2=0x7f577813e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:56.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.239+0000 7f577e7f0640 1 --2- 172.21.15.40:0/2837765665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57780c9850 0x7f577813e570 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f576c036040 tx=0x7f576c0042f0 comp rx=0 tx=0).stop 2023-10-18T09:08:56.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.239+0000 7f577e7f0640 1 -- 172.21.15.40:0/2837765665 shutdown_connections 2023-10-18T09:08:56.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.239+0000 7f577e7f0640 1 --2- 172.21.15.40:0/2837765665 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f576406ae40 0x7f576406d300 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:56.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.239+0000 7f577e7f0640 1 --2- 172.21.15.40:0/2837765665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57780cb620 0x7f577806ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:56.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.239+0000 7f577e7f0640 1 --2- 172.21.15.40:0/2837765665 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57780c9850 0x7f577813e570 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:56.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.239+0000 7f577e7f0640 1 --2- 172.21.15.40:0/2837765665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f577806d680 0x7f5778150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:56.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.239+0000 7f577e7f0640 1 -- 172.21.15.40:0/2837765665 >> 172.21.15.40:0/2837765665 conn(0x7f57780aec00 msgr2=0x7f57780710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:56.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.239+0000 7f577e7f0640 1 -- 172.21.15.40:0/2837765665 shutdown_connections 2023-10-18T09:08:56.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.239+0000 7f577e7f0640 1 -- 172.21.15.40:0/2837765665 wait complete. 2023-10-18T09:08:56.266 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T09:08:56.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.496+0000 7f592b59e640 1 -- 172.21.15.40:0/2801933503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c15af70 msgr2=0x7f592c0710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:56.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.496+0000 7f592b59e640 1 --2- 172.21.15.40:0/2801933503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c15af70 0x7f592c0710a0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f592c0cdf20 tx=0x7f592402bcf0 comp rx=0 tx=0).stop 2023-10-18T09:08:56.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.496+0000 7f592b59e640 1 -- 172.21.15.40:0/2801933503 shutdown_connections 2023-10-18T09:08:56.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.496+0000 7f592b59e640 1 --2- 172.21.15.40:0/2801933503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c15af70 0x7f592c0710a0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:56.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.496+0000 7f592b59e640 1 --2- 172.21.15.40:0/2801933503 >> v2:172.21.15.144:3300/0 conn(0x7f592c06c2d0 0x7f592c070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:56.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.496+0000 7f592b59e640 1 -- 172.21.15.40:0/2801933503 >> 172.21.15.40:0/2801933503 conn(0x7f592c06ca10 msgr2=0x7f592c06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:56.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.496+0000 7f592b59e640 1 -- 172.21.15.40:0/2801933503 shutdown_connections 2023-10-18T09:08:56.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.496+0000 7f592b59e640 1 -- 172.21.15.40:0/2801933503 wait complete. 2023-10-18T09:08:56.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.498+0000 7f592b59e640 1 Processor -- start 2023-10-18T09:08:56.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.498+0000 7f592b59e640 1 -- start start 2023-10-18T09:08:56.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.499+0000 7f592b59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f592c06c2d0 0x7f592c152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:56.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.499+0000 7f592b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c0c9620 0x7f592c0a7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:56.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.499+0000 7f592b59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f592c15af70 0x7f592c13e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:56.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.499+0000 7f592b59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c153340 con 0x7f592c0c9620 2023-10-18T09:08:56.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.499+0000 7f592b59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c13ee90 con 0x7f592c15af70 2023-10-18T09:08:56.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.499+0000 7f592b59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592c13f170 con 0x7f592c06c2d0 2023-10-18T09:08:56.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.500+0000 7f5921d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c0c9620 0x7f592c0a7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:56.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.500+0000 7f5921d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c0c9620 0x7f592c0a7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51150/0 (socket says 172.21.15.40:51150) 2023-10-18T09:08:56.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.500+0000 7f5921d9b640 1 -- 172.21.15.40:0/1748917988 learned_addr learned my addr 172.21.15.40:0/1748917988 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:56.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.500+0000 7f592ad9d640 1 --2- 172.21.15.40:0/1748917988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f592c15af70 0x7f592c13e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:56.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.500+0000 7f5921d9b640 1 -- 172.21.15.40:0/1748917988 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f592c06c2d0 msgr2=0x7f592c152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:56.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.500+0000 7f5921d9b640 1 --2- 172.21.15.40:0/1748917988 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f592c06c2d0 0x7f592c152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:56.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.500+0000 7f5921d9b640 1 -- 172.21.15.40:0/1748917988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f592c15af70 msgr2=0x7f592c13e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:56.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.500+0000 7f5921d9b640 1 --2- 172.21.15.40:0/1748917988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f592c15af70 0x7f592c13e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:56.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.500+0000 7f5921d9b640 1 -- 172.21.15.40:0/1748917988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5924009490 con 0x7f592c0c9620 2023-10-18T09:08:56.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.500+0000 7f5921d9b640 1 --2- 172.21.15.40:0/1748917988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c0c9620 0x7f592c0a7ea0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f591400ea80 tx=0x7f591400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:56.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.501+0000 7f59237fe640 1 -- 172.21.15.40:0/1748917988 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591400ce40 con 0x7f592c0c9620 2023-10-18T09:08:56.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.501+0000 7f59237fe640 1 -- 172.21.15.40:0/1748917988 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5914011560 con 0x7f592c0c9620 2023-10-18T09:08:56.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.501+0000 7f59237fe640 1 -- 172.21.15.40:0/1748917988 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914011950 con 0x7f592c0c9620 2023-10-18T09:08:56.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.501+0000 7f592b59e640 1 -- 172.21.15.40:0/1748917988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c13f450 con 0x7f592c0c9620 2023-10-18T09:08:56.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.501+0000 7f592b59e640 1 -- 172.21.15.40:0/1748917988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c13fa10 con 0x7f592c0c9620 2023-10-18T09:08:56.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.502+0000 7f592b59e640 1 -- 172.21.15.40:0/1748917988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c066e00 con 0x7f592c0c9620 2023-10-18T09:08:56.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.503+0000 7f59237fe640 1 -- 172.21.15.40:0/1748917988 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5914011ab0 con 0x7f592c0c9620 2023-10-18T09:08:56.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.504+0000 7f59237fe640 1 --2- 172.21.15.40:0/1748917988 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f590c06ae40 0x7f590c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:56.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.504+0000 7f59237fe640 1 -- 172.21.15.40:0/1748917988 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f591408a1a0 con 0x7f592c0c9620 2023-10-18T09:08:56.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.505+0000 7f592a59c640 1 --2- 172.21.15.40:0/1748917988 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f590c06ae40 0x7f590c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:56.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.506+0000 7f592a59c640 1 --2- 172.21.15.40:0/1748917988 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f590c06ae40 0x7f590c06d300 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f591c001600 tx=0x7f591c007910 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:56.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.510+0000 7f59237fe640 1 -- 172.21.15.40:0/1748917988 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5914058e20 con 0x7f592c0c9620 2023-10-18T09:08:56.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:56.706+0000 7f592b59e640 1 -- 172.21.15.40:0/1748917988 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f592c0ade70 con 0x7f592c0c9620 2023-10-18T09:08:57.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.241+0000 7f59237fe640 1 -- 172.21.15.40:0/1748917988 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v194) v1 ==== 170+0+0 (secure 0 0 0) 0x7f59140589b0 con 0x7f592c0c9620 2023-10-18T09:08:57.245 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-10-18T09:08:57.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.245+0000 7f592b59e640 1 -- 172.21.15.40:0/1748917988 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f590c06ae40 msgr2=0x7f590c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:57.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.245+0000 7f592b59e640 1 --2- 172.21.15.40:0/1748917988 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f590c06ae40 0x7f590c06d300 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f591c001600 tx=0x7f591c007910 comp rx=0 tx=0).stop 2023-10-18T09:08:57.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.246+0000 7f592b59e640 1 -- 172.21.15.40:0/1748917988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c0c9620 msgr2=0x7f592c0a7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:57.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.246+0000 7f592b59e640 1 --2- 172.21.15.40:0/1748917988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c0c9620 0x7f592c0a7ea0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f591400ea80 tx=0x7f591400ef40 comp rx=0 tx=0).stop 2023-10-18T09:08:57.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.246+0000 7f592b59e640 1 -- 172.21.15.40:0/1748917988 shutdown_connections 2023-10-18T09:08:57.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.246+0000 7f592b59e640 1 --2- 172.21.15.40:0/1748917988 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f590c06ae40 0x7f590c06d300 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:57.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.246+0000 7f592b59e640 1 --2- 172.21.15.40:0/1748917988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f592c15af70 0x7f592c13e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:57.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.246+0000 7f592b59e640 1 --2- 172.21.15.40:0/1748917988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c0c9620 0x7f592c0a7ea0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:57.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.246+0000 7f592b59e640 1 --2- 172.21.15.40:0/1748917988 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f592c06c2d0 0x7f592c152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:57.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.246+0000 7f592b59e640 1 -- 172.21.15.40:0/1748917988 >> 172.21.15.40:0/1748917988 conn(0x7f592c06ca10 msgr2=0x7f592c0ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.247+0000 7f592b59e640 1 -- 172.21.15.40:0/1748917988 shutdown_connections 2023-10-18T09:08:57.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.248+0000 7f592b59e640 1 -- 172.21.15.40:0/1748917988 wait complete. 2023-10-18T09:08:57.283 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T09:08:57.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.485+0000 7fabeafbb640 1 -- 172.21.15.40:0/1085768277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabe4157800 msgr2=0x7fabe40b1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:57.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.485+0000 7fabeafbb640 1 --2- 172.21.15.40:0/1085768277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabe4157800 0x7fabe40b1cc0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009860 tx=0x7fabd002bd60 comp rx=0 tx=0).stop 2023-10-18T09:08:57.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.485+0000 7fabeafbb640 1 -- 172.21.15.40:0/1085768277 shutdown_connections 2023-10-18T09:08:57.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.485+0000 7fabeafbb640 1 --2- 172.21.15.40:0/1085768277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabe4157800 0x7fabe40b1cc0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:57.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.485+0000 7fabeafbb640 1 -- 172.21.15.40:0/1085768277 >> 172.21.15.40:0/1085768277 conn(0x7fabe4069720 msgr2=0x7fabe4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:57.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.485+0000 7fabeafbb640 1 -- 172.21.15.40:0/1085768277 shutdown_connections 2023-10-18T09:08:57.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.485+0000 7fabeafbb640 1 -- 172.21.15.40:0/1085768277 wait complete. 2023-10-18T09:08:57.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.487+0000 7fabeafbb640 1 Processor -- start 2023-10-18T09:08:57.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.487+0000 7fabeafbb640 1 -- start start 2023-10-18T09:08:57.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.488+0000 7fabeafbb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabe406d660 0x7fabe40ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:57.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.488+0000 7fabeafbb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabe406f570 0x7fabe41e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:57.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.488+0000 7fabeafbb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabe4157800 0x7fabe41e3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:57.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.488+0000 7fabeafbb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe40ad640 con 0x7fabe4157800 2023-10-18T09:08:57.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.488+0000 7fabeafbb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe41e37d0 con 0x7fabe406d660 2023-10-18T09:08:57.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.488+0000 7fabeafbb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe41e3ab0 con 0x7fabe406f570 2023-10-18T09:08:57.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.488+0000 7fabe9558640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabe4157800 0x7fabe41e3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:57.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.488+0000 7fabe9558640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabe4157800 0x7fabe41e3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51168/0 (socket says 172.21.15.40:51168) 2023-10-18T09:08:57.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.488+0000 7fabe9558640 1 -- 172.21.15.40:0/415015335 learned_addr learned my addr 172.21.15.40:0/415015335 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:57.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.489+0000 7fabe8d57640 1 --2- 172.21.15.40:0/415015335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabe406d660 0x7fabe40ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:57.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.489+0000 7fabe3fff640 1 --2- 172.21.15.40:0/415015335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabe406f570 0x7fabe41e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:57.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.489+0000 7fabe9558640 1 -- 172.21.15.40:0/415015335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabe406f570 msgr2=0x7fabe41e2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:57.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.489+0000 7fabe9558640 1 --2- 172.21.15.40:0/415015335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabe406f570 0x7fabe41e2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:57.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.489+0000 7fabe9558640 1 -- 172.21.15.40:0/415015335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabe406d660 msgr2=0x7fabe40ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:57.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.489+0000 7fabe9558640 1 --2- 172.21.15.40:0/415015335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabe406d660 0x7fabe40ade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:57.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.489+0000 7fabe9558640 1 -- 172.21.15.40:0/415015335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd0009490 con 0x7fabe4157800 2023-10-18T09:08:57.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.489+0000 7fabe3fff640 1 --2- 172.21.15.40:0/415015335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabe406f570 0x7fabe41e2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:57.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.489+0000 7fabe9558640 1 --2- 172.21.15.40:0/415015335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabe4157800 0x7fabe41e3170 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00cbb0 tx=0x7fabdc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:57.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.489+0000 7fabe17fa640 1 -- 172.21.15.40:0/415015335 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc00ad40 con 0x7fabe4157800 2023-10-18T09:08:57.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.490+0000 7fabe17fa640 1 -- 172.21.15.40:0/415015335 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fabdc012420 con 0x7fabe4157800 2023-10-18T09:08:57.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.490+0000 7fabe17fa640 1 -- 172.21.15.40:0/415015335 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc012870 con 0x7fabe4157800 2023-10-18T09:08:57.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.490+0000 7fabeafbb640 1 -- 172.21.15.40:0/415015335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe4157d00 con 0x7fabe4157800 2023-10-18T09:08:57.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.490+0000 7fabeafbb640 1 -- 172.21.15.40:0/415015335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe41582c0 con 0x7fabe4157800 2023-10-18T09:08:57.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.491+0000 7fabeafbb640 1 -- 172.21.15.40:0/415015335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe41584b0 con 0x7fabe4157800 2023-10-18T09:08:57.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.492+0000 7fabe17fa640 1 -- 172.21.15.40:0/415015335 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fabdc01a3b0 con 0x7fabe4157800 2023-10-18T09:08:57.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.494+0000 7fabe17fa640 1 --2- 172.21.15.40:0/415015335 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fabc806ae40 0x7fabc806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:57.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.494+0000 7fabe17fa640 1 -- 172.21.15.40:0/415015335 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fabdc089d20 con 0x7fabe4157800 2023-10-18T09:08:57.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.495+0000 7fabe8d57640 1 --2- 172.21.15.40:0/415015335 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fabc806ae40 0x7fabc806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:57.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.496+0000 7fabe8d57640 1 --2- 172.21.15.40:0/415015335 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fabc806ae40 0x7fabc806d300 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fabd8001640 tx=0x7fabd8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:57.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.499+0000 7fabe17fa640 1 -- 172.21.15.40:0/415015335 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fabdc054d90 con 0x7fabe4157800 2023-10-18T09:08:57.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:57.728+0000 7fabeafbb640 1 -- 172.21.15.40:0/415015335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fabe40cc450 con 0x7fabe4157800 2023-10-18T09:08:58.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.264+0000 7fabe17fa640 1 -- 172.21.15.40:0/415015335 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v86) v1 ==== 174+0+0 (secure 0 0 0) 0x7fabe40cc450 con 0x7fabe4157800 2023-10-18T09:08:58.268 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 22 and data pool 23 2023-10-18T09:08:58.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.269+0000 7fabeafbb640 1 -- 172.21.15.40:0/415015335 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fabc806ae40 msgr2=0x7fabc806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:58.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.269+0000 7fabeafbb640 1 --2- 172.21.15.40:0/415015335 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fabc806ae40 0x7fabc806d300 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fabd8001640 tx=0x7fabd8009040 comp rx=0 tx=0).stop 2023-10-18T09:08:58.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.269+0000 7fabeafbb640 1 -- 172.21.15.40:0/415015335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabe4157800 msgr2=0x7fabe41e3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:58.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.269+0000 7fabeafbb640 1 --2- 172.21.15.40:0/415015335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabe4157800 0x7fabe41e3170 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fabdc00cbb0 tx=0x7fabdc00a430 comp rx=0 tx=0).stop 2023-10-18T09:08:58.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.270+0000 7fabeafbb640 1 -- 172.21.15.40:0/415015335 shutdown_connections 2023-10-18T09:08:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.270+0000 7fabeafbb640 1 --2- 172.21.15.40:0/415015335 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fabc806ae40 0x7fabc806d300 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.270+0000 7fabeafbb640 1 --2- 172.21.15.40:0/415015335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabe4157800 0x7fabe41e3170 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.270+0000 7fabeafbb640 1 --2- 172.21.15.40:0/415015335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabe406f570 0x7fabe41e2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.270+0000 7fabeafbb640 1 --2- 172.21.15.40:0/415015335 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabe406d660 0x7fabe40ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.270+0000 7fabeafbb640 1 -- 172.21.15.40:0/415015335 >> 172.21.15.40:0/415015335 conn(0x7fabe4069720 msgr2=0x7fabe40ca170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:58.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.270+0000 7fabeafbb640 1 -- 172.21.15.40:0/415015335 shutdown_connections 2023-10-18T09:08:58.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.270+0000 7fabeafbb640 1 -- 172.21.15.40:0/415015335 wait complete. 2023-10-18T09:08:58.305 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:08:58.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.494+0000 7fa8cc856640 1 -- 172.21.15.40:0/164054895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c806d710 msgr2=0x7fa8c8151bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:58.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.494+0000 7fa8cc856640 1 --2- 172.21.15.40:0/164054895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c806d710 0x7fa8c8151bd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8005fd0 tx=0x7fa8b802be10 comp rx=0 tx=0).stop 2023-10-18T09:08:58.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.495+0000 7fa8cc856640 1 -- 172.21.15.40:0/164054895 shutdown_connections 2023-10-18T09:08:58.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.495+0000 7fa8cc856640 1 --2- 172.21.15.40:0/164054895 >> v2:172.21.15.144:3300/0 conn(0x7fa8c81586a0 0x7fa8c806b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.495+0000 7fa8cc856640 1 --2- 172.21.15.40:0/164054895 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c806d710 0x7fa8c8151bd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.495+0000 7fa8cc856640 1 -- 172.21.15.40:0/164054895 >> 172.21.15.40:0/164054895 conn(0x7fa8c806cee0 msgr2=0x7fa8c8069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:58.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.495+0000 7fa8cc856640 1 -- 172.21.15.40:0/164054895 shutdown_connections 2023-10-18T09:08:58.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.495+0000 7fa8cc856640 1 -- 172.21.15.40:0/164054895 wait complete. 2023-10-18T09:08:58.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.496+0000 7fa8cc856640 1 Processor -- start 2023-10-18T09:08:58.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.497+0000 7fa8cc856640 1 -- start start 2023-10-18T09:08:58.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.498+0000 7fa8cc856640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c806bc50 0x7fa8c814d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.498+0000 7fa8cc856640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8c806d710 0x7fa8c81498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.498+0000 7fa8cc856640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c81586a0 0x7fa8c814f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.498+0000 7fa8cc856640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c814d9c0 con 0x7fa8c806d710 2023-10-18T09:08:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.498+0000 7fa8cc856640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c814dc20 con 0x7fa8c81586a0 2023-10-18T09:08:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.498+0000 7fa8cc856640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c814df00 con 0x7fa8c806bc50 2023-10-18T09:08:58.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.498+0000 7fa8c5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8c806d710 0x7fa8c81498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:58.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.498+0000 7fa8c5d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8c806d710 0x7fa8c81498c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51182/0 (socket says 172.21.15.40:51182) 2023-10-18T09:08:58.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.498+0000 7fa8c5d9b640 1 -- 172.21.15.40:0/2755085593 learned_addr learned my addr 172.21.15.40:0/2755085593 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:58.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.498+0000 7fa8c659c640 1 --2- 172.21.15.40:0/2755085593 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c806bc50 0x7fa8c814d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:58.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.498+0000 7fa8c6d9d640 1 --2- 172.21.15.40:0/2755085593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c81586a0 0x7fa8c814f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:58.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.498+0000 7fa8c5d9b640 1 -- 172.21.15.40:0/2755085593 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c806bc50 msgr2=0x7fa8c814d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:58.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.498+0000 7fa8c5d9b640 1 --2- 172.21.15.40:0/2755085593 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c806bc50 0x7fa8c814d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.498+0000 7fa8c5d9b640 1 -- 172.21.15.40:0/2755085593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c81586a0 msgr2=0x7fa8c814f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:58.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.498+0000 7fa8c5d9b640 1 --2- 172.21.15.40:0/2755085593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c81586a0 0x7fa8c814f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.499+0000 7fa8c5d9b640 1 -- 172.21.15.40:0/2755085593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b8009560 con 0x7fa8c806d710 2023-10-18T09:08:58.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.499+0000 7fa8c6d9d640 1 --2- 172.21.15.40:0/2755085593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c81586a0 0x7fa8c814f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:08:58.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.499+0000 7fa8c5d9b640 1 --2- 172.21.15.40:0/2755085593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8c806d710 0x7fa8c81498c0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00ea90 tx=0x7fa8bc00ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:58.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.499+0000 7fa8b6ffd640 1 -- 172.21.15.40:0/2755085593 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc00cc80 con 0x7fa8c806d710 2023-10-18T09:08:58.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.499+0000 7fa8b6ffd640 1 -- 172.21.15.40:0/2755085593 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa8bc016410 con 0x7fa8c806d710 2023-10-18T09:08:58.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.499+0000 7fa8b6ffd640 1 -- 172.21.15.40:0/2755085593 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc016820 con 0x7fa8c806d710 2023-10-18T09:08:58.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.499+0000 7fa8cc856640 1 -- 172.21.15.40:0/2755085593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c815c1d0 con 0x7fa8c806d710 2023-10-18T09:08:58.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.499+0000 7fa8cc856640 1 -- 172.21.15.40:0/2755085593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c814e4c0 con 0x7fa8c806d710 2023-10-18T09:08:58.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.501+0000 7fa8cc856640 1 -- 172.21.15.40:0/2755085593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c8066fd0 con 0x7fa8c806d710 2023-10-18T09:08:58.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.502+0000 7fa8b6ffd640 1 -- 172.21.15.40:0/2755085593 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa8bc016980 con 0x7fa8c806d710 2023-10-18T09:08:58.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.503+0000 7fa8b6ffd640 1 --2- 172.21.15.40:0/2755085593 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa8a806ae40 0x7fa8a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:58.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.503+0000 7fa8b6ffd640 1 -- 172.21.15.40:0/2755085593 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8bc089c40 con 0x7fa8c806d710 2023-10-18T09:08:58.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.504+0000 7fa8c659c640 1 --2- 172.21.15.40:0/2755085593 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa8a806ae40 0x7fa8a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:58.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.505+0000 7fa8c659c640 1 --2- 172.21.15.40:0/2755085593 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa8a806ae40 0x7fa8a806d300 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8035000 tx=0x7fa8b8004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:58.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.508+0000 7fa8b6ffd640 1 -- 172.21.15.40:0/2755085593 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa8bc054d00 con 0x7fa8c806d710 2023-10-18T09:08:58.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.702+0000 7fa8cc856640 1 -- 172.21.15.40:0/2755085593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8c81511a0 con 0x7fa8c806d710 2023-10-18T09:08:58.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.704+0000 7fa8b6ffd640 1 -- 172.21.15.40:0/2755085593 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+16597 (secure 0 0 0) 0x7fa8bc01b020 con 0x7fa8c806d710 2023-10-18T09:08:58.707 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:08:58.708 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":195,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:08:57.730772+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:08:55.675479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-18T09:08:56.761888+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:08:58.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.706+0000 7fa8cc856640 1 -- 172.21.15.40:0/2755085593 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa8a806ae40 msgr2=0x7fa8a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:58.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.706+0000 7fa8cc856640 1 --2- 172.21.15.40:0/2755085593 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa8a806ae40 0x7fa8a806d300 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8035000 tx=0x7fa8b8004730 comp rx=0 tx=0).stop 2023-10-18T09:08:58.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.706+0000 7fa8cc856640 1 -- 172.21.15.40:0/2755085593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8c806d710 msgr2=0x7fa8c81498c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.706+0000 7fa8cc856640 1 --2- 172.21.15.40:0/2755085593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8c806d710 0x7fa8c81498c0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00ea90 tx=0x7fa8bc00ef50 comp rx=0 tx=0).stop 2023-10-18T09:08:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.706+0000 7fa8cc856640 1 -- 172.21.15.40:0/2755085593 shutdown_connections 2023-10-18T09:08:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.706+0000 7fa8cc856640 1 --2- 172.21.15.40:0/2755085593 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fa8a806ae40 0x7fa8a806d300 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.706+0000 7fa8cc856640 1 --2- 172.21.15.40:0/2755085593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8c81586a0 0x7fa8c814f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.706+0000 7fa8cc856640 1 --2- 172.21.15.40:0/2755085593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8c806d710 0x7fa8c81498c0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.706+0000 7fa8cc856640 1 --2- 172.21.15.40:0/2755085593 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8c806bc50 0x7fa8c814d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.706+0000 7fa8cc856640 1 -- 172.21.15.40:0/2755085593 >> 172.21.15.40:0/2755085593 conn(0x7fa8c806cee0 msgr2=0x7fa8c8070150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:58.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.707+0000 7fa8cc856640 1 -- 172.21.15.40:0/2755085593 shutdown_connections 2023-10-18T09:08:58.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.707+0000 7fa8cc856640 1 -- 172.21.15.40:0/2755085593 wait complete. 2023-10-18T09:08:58.729 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:08:58.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.959+0000 7f2ece690640 1 -- 172.21.15.40:0/639156554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec81425a0 msgr2=0x7f2ec80c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:58.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.959+0000 7f2ece690640 1 --2- 172.21.15.40:0/639156554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec81425a0 0x7f2ec80c9a50 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f2ec80ccea0 tx=0x7f2eb802bdb0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.959+0000 7f2ece690640 1 -- 172.21.15.40:0/639156554 shutdown_connections 2023-10-18T09:08:58.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.959+0000 7f2ece690640 1 --2- 172.21.15.40:0/639156554 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec81425a0 0x7f2ec80c9a50 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.959+0000 7f2ece690640 1 --2- 172.21.15.40:0/639156554 >> v2:172.21.15.144:3300/0 conn(0x7f2ec81420f0 0x7f2ec80b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.959+0000 7f2ece690640 1 -- 172.21.15.40:0/639156554 >> 172.21.15.40:0/639156554 conn(0x7f2ec8069720 msgr2=0x7f2ec8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:58.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.959+0000 7f2ece690640 1 -- 172.21.15.40:0/639156554 shutdown_connections 2023-10-18T09:08:58.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.959+0000 7f2ece690640 1 -- 172.21.15.40:0/639156554 wait complete. 2023-10-18T09:08:58.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.962+0000 7f2ece690640 1 Processor -- start 2023-10-18T09:08:58.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.962+0000 7f2ece690640 1 -- start start 2023-10-18T09:08:58.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.963+0000 7f2ece690640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ec80c9f90 0x7f2ec8158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.963+0000 7f2ece690640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec81420f0 0x7f2ec8159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.963+0000 7f2ece690640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ec81425a0 0x7f2ec8070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.963+0000 7f2ece690640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec80c29f0 con 0x7f2ec81420f0 2023-10-18T09:08:58.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.963+0000 7f2ece690640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec815a210 con 0x7f2ec81425a0 2023-10-18T09:08:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.963+0000 7f2ece690640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec815a4f0 con 0x7f2ec80c9f90 2023-10-18T09:08:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.963+0000 7f2ecce8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec81420f0 0x7f2ec8159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.963+0000 7f2ecce8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec81420f0 0x7f2ec8159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51188/0 (socket says 172.21.15.40:51188) 2023-10-18T09:08:58.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.963+0000 7f2ecce8d640 1 -- 172.21.15.40:0/2925267719 learned_addr learned my addr 172.21.15.40:0/2925267719 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:58.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.964+0000 7f2ecce8d640 1 -- 172.21.15.40:0/2925267719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ec80c9f90 msgr2=0x7f2ec8158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:58.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.964+0000 7f2ecce8d640 1 --2- 172.21.15.40:0/2925267719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ec80c9f90 0x7f2ec8158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.964+0000 7f2ecce8d640 1 -- 172.21.15.40:0/2925267719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ec81425a0 msgr2=0x7f2ec8070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:58.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.964+0000 7f2ecce8d640 1 --2- 172.21.15.40:0/2925267719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ec81425a0 0x7f2ec8070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:58.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.964+0000 7f2ecce8d640 1 -- 172.21.15.40:0/2925267719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb8009560 con 0x7f2ec81420f0 2023-10-18T09:08:58.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.964+0000 7f2ecce8d640 1 --2- 172.21.15.40:0/2925267719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec81420f0 0x7f2ec8159950 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009850 tx=0x7f2eb802dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:58.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.965+0000 7f2ebdffb640 1 -- 172.21.15.40:0/2925267719 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8005650 con 0x7f2ec81420f0 2023-10-18T09:08:58.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.965+0000 7f2ebdffb640 1 -- 172.21.15.40:0/2925267719 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2eb80038f0 con 0x7f2ec81420f0 2023-10-18T09:08:58.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.965+0000 7f2ebdffb640 1 -- 172.21.15.40:0/2925267719 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8003ce0 con 0x7f2ec81420f0 2023-10-18T09:08:58.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.965+0000 7f2ece690640 1 -- 172.21.15.40:0/2925267719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec815d9a0 con 0x7f2ec81420f0 2023-10-18T09:08:58.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.965+0000 7f2ece690640 1 -- 172.21.15.40:0/2925267719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec815aab0 con 0x7f2ec81420f0 2023-10-18T09:08:58.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.966+0000 7f2ece690640 1 -- 172.21.15.40:0/2925267719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec80c2b60 con 0x7f2ec81420f0 2023-10-18T09:08:58.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.967+0000 7f2ebdffb640 1 -- 172.21.15.40:0/2925267719 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2eb803e450 con 0x7f2ec81420f0 2023-10-18T09:08:58.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.968+0000 7f2ebdffb640 1 --2- 172.21.15.40:0/2925267719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2eac06ae40 0x7f2eac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.968+0000 7f2ebdffb640 1 -- 172.21.15.40:0/2925267719 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2eb80abfd0 con 0x7f2ec81420f0 2023-10-18T09:08:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.969+0000 7f2ecd68e640 1 --2- 172.21.15.40:0/2925267719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2eac06ae40 0x7f2eac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:58.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.970+0000 7f2ecd68e640 1 --2- 172.21.15.40:0/2925267719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2eac06ae40 0x7f2eac06d300 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009710 tx=0x7f2ec4006c90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:58.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:58.972+0000 7f2ebdffb640 1 -- 172.21.15.40:0/2925267719 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2eb8077070 con 0x7f2ec81420f0 2023-10-18T09:08:59.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.165+0000 7f2ece690640 1 -- 172.21.15.40:0/2925267719 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ec815ac40 con 0x7f2ec81420f0 2023-10-18T09:08:59.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.167+0000 7f2ebdffb640 1 -- 172.21.15.40:0/2925267719 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+16597 (secure 0 0 0) 0x7f2eb8042090 con 0x7f2ec81420f0 2023-10-18T09:08:59.171 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:08:59.171 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":195,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:08:57.730772+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:08:55.675479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-18T09:08:56.761888+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:08:59.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.170+0000 7f2ece690640 1 -- 172.21.15.40:0/2925267719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2eac06ae40 msgr2=0x7f2eac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:59.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.170+0000 7f2ece690640 1 --2- 172.21.15.40:0/2925267719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2eac06ae40 0x7f2eac06d300 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009710 tx=0x7f2ec4006c90 comp rx=0 tx=0).stop 2023-10-18T09:08:59.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.170+0000 7f2ece690640 1 -- 172.21.15.40:0/2925267719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec81420f0 msgr2=0x7f2ec8159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:59.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.170+0000 7f2ece690640 1 --2- 172.21.15.40:0/2925267719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec81420f0 0x7f2ec8159950 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009850 tx=0x7f2eb802dd80 comp rx=0 tx=0).stop 2023-10-18T09:08:59.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.170+0000 7f2ece690640 1 -- 172.21.15.40:0/2925267719 shutdown_connections 2023-10-18T09:08:59.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.170+0000 7f2ece690640 1 --2- 172.21.15.40:0/2925267719 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2eac06ae40 0x7f2eac06d300 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:59.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.170+0000 7f2ece690640 1 --2- 172.21.15.40:0/2925267719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ec81425a0 0x7f2ec8070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:59.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.170+0000 7f2ece690640 1 --2- 172.21.15.40:0/2925267719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ec81420f0 0x7f2ec8159950 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:59.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.170+0000 7f2ece690640 1 --2- 172.21.15.40:0/2925267719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ec80c9f90 0x7f2ec8158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:59.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.170+0000 7f2ece690640 1 -- 172.21.15.40:0/2925267719 >> 172.21.15.40:0/2925267719 conn(0x7f2ec8069720 msgr2=0x7f2ec8072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:59.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.170+0000 7f2ece690640 1 -- 172.21.15.40:0/2925267719 shutdown_connections 2023-10-18T09:08:59.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.171+0000 7f2ece690640 1 -- 172.21.15.40:0/2925267719 wait complete. 2023-10-18T09:08:59.193 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T09:08:59.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.424+0000 7ff9052b4640 1 -- 172.21.15.40:0/2357121481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e80047c0 msgr2=0x7ff8e8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:59.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.424+0000 7ff9052b4640 1 --2- 172.21.15.40:0/2357121481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e80047c0 0x7ff8e8004c40 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4005fd0 tx=0x7ff8e80057f0 comp rx=0 tx=0).stop 2023-10-18T09:08:59.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.424+0000 7ff9052b4640 1 -- 172.21.15.40:0/2357121481 shutdown_connections 2023-10-18T09:08:59.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.424+0000 7ff9052b4640 1 --2- 172.21.15.40:0/2357121481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8e80047c0 0x7ff8e8004c40 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:59.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.424+0000 7ff9052b4640 1 -- 172.21.15.40:0/2357121481 >> 172.21.15.40:0/2357121481 conn(0x7ff90006ca10 msgr2=0x7ff90006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:08:59.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.424+0000 7ff9052b4640 1 -- 172.21.15.40:0/2357121481 shutdown_connections 2023-10-18T09:08:59.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.424+0000 7ff9052b4640 1 -- 172.21.15.40:0/2357121481 wait complete. 2023-10-18T09:08:59.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.426+0000 7ff9052b4640 1 Processor -- start 2023-10-18T09:08:59.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.426+0000 7ff9052b4640 1 -- start start 2023-10-18T09:08:59.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.427+0000 7ff9052b4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e80047c0 0x7ff900152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:59.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.427+0000 7ff9052b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9001f34a0 0x7ff900157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:59.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.427+0000 7ff9052b4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff90014a8d0 0x7ff9001544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.427+0000 7ff9052b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff900158280 con 0x7ff9001f34a0 2023-10-18T09:08:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.427+0000 7ff9052b4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff900158530 con 0x7ff90014a8d0 2023-10-18T09:08:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.427+0000 7ff9052b4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff900158810 con 0x7ff8e80047c0 2023-10-18T09:08:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.427+0000 7ff8ff7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9001f34a0 0x7ff900157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:59.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.427+0000 7ff8ff7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9001f34a0 0x7ff900157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51200/0 (socket says 172.21.15.40:51200) 2023-10-18T09:08:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.427+0000 7ff8ff7fe640 1 -- 172.21.15.40:0/1176360770 learned_addr learned my addr 172.21.15.40:0/1176360770 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:08:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.428+0000 7ff8fffff640 1 --2- 172.21.15.40:0/1176360770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e80047c0 0x7ff900152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.428+0000 7ff8ff7fe640 1 -- 172.21.15.40:0/1176360770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e80047c0 msgr2=0x7ff900152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.428+0000 7ff8ff7fe640 1 --2- 172.21.15.40:0/1176360770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e80047c0 0x7ff900152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:59.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.428+0000 7ff8ff7fe640 1 -- 172.21.15.40:0/1176360770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff90014a8d0 msgr2=0x7ff9001544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:08:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.428+0000 7ff8ff7fe640 1 --2- 172.21.15.40:0/1176360770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff90014a8d0 0x7ff9001544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:08:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.428+0000 7ff8ff7fe640 1 -- 172.21.15.40:0/1176360770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff900158af0 con 0x7ff9001f34a0 2023-10-18T09:08:59.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.428+0000 7ff8ff7fe640 1 --2- 172.21.15.40:0/1176360770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9001f34a0 0x7ff900157d40 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7ff8f402ca00 tx=0x7ff8f4033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:59.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.428+0000 7ff8fcff9640 1 -- 172.21.15.40:0/1176360770 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f403d820 con 0x7ff9001f34a0 2023-10-18T09:08:59.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.428+0000 7ff8fcff9640 1 -- 172.21.15.40:0/1176360770 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff8f403d980 con 0x7ff9001f34a0 2023-10-18T09:08:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.428+0000 7ff9052b4640 1 -- 172.21.15.40:0/1176360770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff900159a00 con 0x7ff9001f34a0 2023-10-18T09:08:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.428+0000 7ff8fcff9640 1 -- 172.21.15.40:0/1176360770 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f403dd50 con 0x7ff9001f34a0 2023-10-18T09:08:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.428+0000 7ff9052b4640 1 -- 172.21.15.40:0/1176360770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90014af60 con 0x7ff9001f34a0 2023-10-18T09:08:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.430+0000 7ff9052b4640 1 -- 172.21.15.40:0/1176360770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff900159860 con 0x7ff9001f34a0 2023-10-18T09:08:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.430+0000 7ff8fcff9640 1 -- 172.21.15.40:0/1176360770 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff8f4045400 con 0x7ff9001f34a0 2023-10-18T09:08:59.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.431+0000 7ff8fcff9640 1 --2- 172.21.15.40:0/1176360770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff8ec06ae40 0x7ff8ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:08:59.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.431+0000 7ff8fcff9640 1 -- 172.21.15.40:0/1176360770 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8f40ac7b0 con 0x7ff9001f34a0 2023-10-18T09:08:59.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.432+0000 7ff8fffff640 1 --2- 172.21.15.40:0/1176360770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff8ec06ae40 0x7ff8ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:08:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.433+0000 7ff8fffff640 1 --2- 172.21.15.40:0/1176360770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff8ec06ae40 0x7ff8ec06d300 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff900063540 tx=0x7ff8f00068e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:08:59.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.436+0000 7ff8fcff9640 1 -- 172.21.15.40:0/1176360770 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff900159860 con 0x7ff9001f34a0 2023-10-18T09:08:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:08:59.670+0000 7ff9052b4640 1 -- 172.21.15.40:0/1176360770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff900159860 con 0x7ff9001f34a0 2023-10-18T09:09:00.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.310+0000 7ff8fcff9640 1 -- 172.21.15.40:0/1176360770 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v89) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff900159860 con 0x7ff9001f34a0 2023-10-18T09:09:00.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.316+0000 7ff9052b4640 1 -- 172.21.15.40:0/1176360770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff8ec06ae40 msgr2=0x7ff8ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:00.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.316+0000 7ff9052b4640 1 --2- 172.21.15.40:0/1176360770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff8ec06ae40 0x7ff8ec06d300 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff900063540 tx=0x7ff8f00068e0 comp rx=0 tx=0).stop 2023-10-18T09:09:00.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.316+0000 7ff9052b4640 1 -- 172.21.15.40:0/1176360770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9001f34a0 msgr2=0x7ff900157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:00.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.316+0000 7ff9052b4640 1 --2- 172.21.15.40:0/1176360770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9001f34a0 0x7ff900157d40 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7ff8f402ca00 tx=0x7ff8f4033c90 comp rx=0 tx=0).stop 2023-10-18T09:09:00.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.316+0000 7ff9052b4640 1 -- 172.21.15.40:0/1176360770 shutdown_connections 2023-10-18T09:09:00.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.316+0000 7ff9052b4640 1 --2- 172.21.15.40:0/1176360770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff8ec06ae40 0x7ff8ec06d300 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:00.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.316+0000 7ff9052b4640 1 --2- 172.21.15.40:0/1176360770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff90014a8d0 0x7ff9001544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:00.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.316+0000 7ff9052b4640 1 --2- 172.21.15.40:0/1176360770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9001f34a0 0x7ff900157d40 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:00.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.316+0000 7ff9052b4640 1 --2- 172.21.15.40:0/1176360770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8e80047c0 0x7ff900152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:00.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.316+0000 7ff9052b4640 1 -- 172.21.15.40:0/1176360770 >> 172.21.15.40:0/1176360770 conn(0x7ff90006ca10 msgr2=0x7ff9000ab060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:00.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.316+0000 7ff9052b4640 1 -- 172.21.15.40:0/1176360770 shutdown_connections 2023-10-18T09:09:00.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.316+0000 7ff9052b4640 1 -- 172.21.15.40:0/1176360770 wait complete. 2023-10-18T09:09:00.340 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-18T09:09:00.340 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-18T09:09:00.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.570+0000 7f0f06a1a640 1 -- 172.21.15.40:0/258807190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0006c180 msgr2=0x7f0f000cb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:00.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.570+0000 7f0f06a1a640 1 --2- 172.21.15.40:0/258807190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0006c180 0x7f0f000cb3b0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8005fd0 tx=0x7f0ef802c240 comp rx=0 tx=0).stop 2023-10-18T09:09:00.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.570+0000 7f0f06a1a640 1 -- 172.21.15.40:0/258807190 shutdown_connections 2023-10-18T09:09:00.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.570+0000 7f0f06a1a640 1 --2- 172.21.15.40:0/258807190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0006c180 0x7f0f000cb3b0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:00.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.570+0000 7f0f06a1a640 1 --2- 172.21.15.40:0/258807190 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f00159430 0x7f0f000b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:00.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.570+0000 7f0f06a1a640 1 -- 172.21.15.40:0/258807190 >> 172.21.15.40:0/258807190 conn(0x7f0f0006ca10 msgr2=0x7f0f0006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:00.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.570+0000 7f0f06a1a640 1 -- 172.21.15.40:0/258807190 shutdown_connections 2023-10-18T09:09:00.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.570+0000 7f0f06a1a640 1 -- 172.21.15.40:0/258807190 wait complete. 2023-10-18T09:09:00.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.572+0000 7f0f06a1a640 1 Processor -- start 2023-10-18T09:09:00.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.572+0000 7f0f06a1a640 1 -- start start 2023-10-18T09:09:00.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.574+0000 7f0f06a1a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0006c180 0x7f0f00159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:00.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.574+0000 7f0f06a1a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f00159430 0x7f0f001575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:00.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.574+0000 7f0f06a1a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f0015cda0 0x7f0f000ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:00.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.574+0000 7f0f06a1a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0015af40 con 0x7f0f0006c180 2023-10-18T09:09:00.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.574+0000 7f0f06a1a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f00157b00 con 0x7f0f00159430 2023-10-18T09:09:00.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.574+0000 7f0f06a1a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0015a2c0 con 0x7f0f0015cda0 2023-10-18T09:09:00.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.574+0000 7f0f05a18640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0006c180 0x7f0f00159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:00.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.574+0000 7f0f05a18640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0006c180 0x7f0f00159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51212/0 (socket says 172.21.15.40:51212) 2023-10-18T09:09:00.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.574+0000 7f0f05a18640 1 -- 172.21.15.40:0/3549166088 learned_addr learned my addr 172.21.15.40:0/3549166088 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.574+0000 7f0f05217640 1 --2- 172.21.15.40:0/3549166088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f00159430 0x7f0f001575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.574+0000 7f0f06219640 1 --2- 172.21.15.40:0/3549166088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f0015cda0 0x7f0f000ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.575+0000 7f0f05a18640 1 -- 172.21.15.40:0/3549166088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f0015cda0 msgr2=0x7f0f000ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.575+0000 7f0f05a18640 1 --2- 172.21.15.40:0/3549166088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f0015cda0 0x7f0f000ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.575+0000 7f0f05a18640 1 -- 172.21.15.40:0/3549166088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f00159430 msgr2=0x7f0f001575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.575+0000 7f0f05a18640 1 --2- 172.21.15.40:0/3549166088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f00159430 0x7f0f001575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.575+0000 7f0f05a18640 1 -- 172.21.15.40:0/3549166088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef8009900 con 0x7f0f0006c180 2023-10-18T09:09:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.575+0000 7f0f05a18640 1 --2- 172.21.15.40:0/3549166088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0006c180 0x7f0f00159c70 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00b4a0 tx=0x7f0efc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.575+0000 7f0ef67fc640 1 -- 172.21.15.40:0/3549166088 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc010610 con 0x7f0f0006c180 2023-10-18T09:09:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.575+0000 7f0ef67fc640 1 -- 172.21.15.40:0/3549166088 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0efc010860 con 0x7f0f0006c180 2023-10-18T09:09:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.575+0000 7f0ef67fc640 1 -- 172.21.15.40:0/3549166088 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc010c70 con 0x7f0f0006c180 2023-10-18T09:09:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.575+0000 7f0f06a1a640 1 -- 172.21.15.40:0/3549166088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0015a5a0 con 0x7f0f0006c180 2023-10-18T09:09:00.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.576+0000 7f0f06a1a640 1 -- 172.21.15.40:0/3549166088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0015ab60 con 0x7f0f0006c180 2023-10-18T09:09:00.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.577+0000 7f0f06a1a640 1 -- 172.21.15.40:0/3549166088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0015b080 con 0x7f0f0006c180 2023-10-18T09:09:00.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.578+0000 7f0ef67fc640 1 -- 172.21.15.40:0/3549166088 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0efc017360 con 0x7f0f0006c180 2023-10-18T09:09:00.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.579+0000 7f0ef67fc640 1 --2- 172.21.15.40:0/3549166088 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ee406ae70 0x7f0ee406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:00.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.579+0000 7f0ef67fc640 1 -- 172.21.15.40:0/3549166088 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0efc0893f0 con 0x7f0f0006c180 2023-10-18T09:09:00.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.580+0000 7f0f05217640 1 --2- 172.21.15.40:0/3549166088 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ee406ae70 0x7f0ee406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:00.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.581+0000 7f0f05217640 1 --2- 172.21.15.40:0/3549166088 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ee406ae70 0x7f0ee406d330 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0f00146b20 tx=0x7f0ef0009060 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:00.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.583+0000 7f0ef67fc640 1 -- 172.21.15.40:0/3549166088 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0f0015b080 con 0x7f0f0006c180 2023-10-18T09:09:00.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:00.821+0000 7f0f06a1a640 1 -- 172.21.15.40:0/3549166088 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0f0015b080 con 0x7f0f0006c180 2023-10-18T09:09:01.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.320+0000 7f0ef67fc640 1 -- 172.21.15.40:0/3549166088 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v90) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0f0015b080 con 0x7f0f0006c180 2023-10-18T09:09:01.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.324+0000 7f0f06a1a640 1 -- 172.21.15.40:0/3549166088 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ee406ae70 msgr2=0x7f0ee406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:01.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.324+0000 7f0f06a1a640 1 --2- 172.21.15.40:0/3549166088 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ee406ae70 0x7f0ee406d330 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0f00146b20 tx=0x7f0ef0009060 comp rx=0 tx=0).stop 2023-10-18T09:09:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.324+0000 7f0f06a1a640 1 -- 172.21.15.40:0/3549166088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0006c180 msgr2=0x7f0f00159c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.324+0000 7f0f06a1a640 1 --2- 172.21.15.40:0/3549166088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0006c180 0x7f0f00159c70 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00b4a0 tx=0x7f0efc00b960 comp rx=0 tx=0).stop 2023-10-18T09:09:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.324+0000 7f0f06a1a640 1 -- 172.21.15.40:0/3549166088 shutdown_connections 2023-10-18T09:09:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.324+0000 7f0f06a1a640 1 --2- 172.21.15.40:0/3549166088 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0ee406ae70 0x7f0ee406d330 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:01.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.324+0000 7f0f06a1a640 1 --2- 172.21.15.40:0/3549166088 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f0015cda0 0x7f0f000ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.324+0000 7f0f06a1a640 1 --2- 172.21.15.40:0/3549166088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f00159430 0x7f0f001575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.324+0000 7f0f06a1a640 1 --2- 172.21.15.40:0/3549166088 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f0006c180 0x7f0f00159c70 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.324+0000 7f0f06a1a640 1 -- 172.21.15.40:0/3549166088 >> 172.21.15.40:0/3549166088 conn(0x7f0f0006ca10 msgr2=0x7f0f0006cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.324+0000 7f0f06a1a640 1 -- 172.21.15.40:0/3549166088 shutdown_connections 2023-10-18T09:09:01.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.324+0000 7f0f06a1a640 1 -- 172.21.15.40:0/3549166088 wait complete. 2023-10-18T09:09:01.348 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-18T09:09:01.349 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:09:01.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.576+0000 7fb363a32640 1 -- 172.21.15.40:0/2505479954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb35c15eaa0 msgr2=0x7fb35c0ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:01.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.576+0000 7fb363a32640 1 --2- 172.21.15.40:0/2505479954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb35c15eaa0 0x7fb35c0ccdb0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fb358009830 tx=0x7fb35802be80 comp rx=0 tx=0).stop 2023-10-18T09:09:01.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.576+0000 7fb363a32640 1 -- 172.21.15.40:0/2505479954 shutdown_connections 2023-10-18T09:09:01.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.576+0000 7fb363a32640 1 --2- 172.21.15.40:0/2505479954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb35c15eaa0 0x7fb35c0ccdb0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:01.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.576+0000 7fb363a32640 1 -- 172.21.15.40:0/2505479954 >> 172.21.15.40:0/2505479954 conn(0x7fb35c06cee0 msgr2=0x7fb35c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.576+0000 7fb363a32640 1 -- 172.21.15.40:0/2505479954 shutdown_connections 2023-10-18T09:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.576+0000 7fb363a32640 1 -- 172.21.15.40:0/2505479954 wait complete. 2023-10-18T09:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.578+0000 7fb363a32640 1 Processor -- start 2023-10-18T09:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.578+0000 7fb363a32640 1 -- start start 2023-10-18T09:09:01.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.579+0000 7fb363a32640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb35c0b12a0 0x7fb35c1f1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:01.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.579+0000 7fb363a32640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb35c0b1b90 0x7fb35c0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:01.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.579+0000 7fb363a32640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb35c15eaa0 0x7fb35c1e6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:01.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.579+0000 7fb363a32640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c1f1ea0 con 0x7fb35c15eaa0 2023-10-18T09:09:01.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.579+0000 7fb363a32640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c14a1c0 con 0x7fb35c0b1b90 2023-10-18T09:09:01.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.579+0000 7fb363a32640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c0caa50 con 0x7fb35c0b12a0 2023-10-18T09:09:01.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.579+0000 7fb361fcf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb35c15eaa0 0x7fb35c1e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:01.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.580+0000 7fb361fcf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb35c15eaa0 0x7fb35c1e6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51220/0 (socket says 172.21.15.40:51220) 2023-10-18T09:09:01.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.580+0000 7fb361fcf640 1 -- 172.21.15.40:0/4208795267 learned_addr learned my addr 172.21.15.40:0/4208795267 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:01.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.580+0000 7fb360fcd640 1 --2- 172.21.15.40:0/4208795267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb35c0b1b90 0x7fb35c0ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:01.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.580+0000 7fb3617ce640 1 --2- 172.21.15.40:0/4208795267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb35c0b12a0 0x7fb35c1f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:01.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.580+0000 7fb361fcf640 1 -- 172.21.15.40:0/4208795267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb35c0b12a0 msgr2=0x7fb35c1f1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:01.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.580+0000 7fb361fcf640 1 --2- 172.21.15.40:0/4208795267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb35c0b12a0 0x7fb35c1f1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:01.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.580+0000 7fb361fcf640 1 -- 172.21.15.40:0/4208795267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb35c0b1b90 msgr2=0x7fb35c0ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:01.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.580+0000 7fb361fcf640 1 --2- 172.21.15.40:0/4208795267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb35c0b1b90 0x7fb35c0ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:01.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.580+0000 7fb361fcf640 1 -- 172.21.15.40:0/4208795267 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb358009490 con 0x7fb35c15eaa0 2023-10-18T09:09:01.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.580+0000 7fb360fcd640 1 --2- 172.21.15.40:0/4208795267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb35c0b1b90 0x7fb35c0ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:01.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.580+0000 7fb361fcf640 1 --2- 172.21.15.40:0/4208795267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb35c15eaa0 0x7fb35c1e6090 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fb35400cb40 tx=0x7fb35400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:01.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.581+0000 7fb351ffb640 1 -- 172.21.15.40:0/4208795267 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35400aca0 con 0x7fb35c15eaa0 2023-10-18T09:09:01.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.581+0000 7fb351ffb640 1 -- 172.21.15.40:0/4208795267 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb354012420 con 0x7fb35c15eaa0 2023-10-18T09:09:01.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.581+0000 7fb351ffb640 1 -- 172.21.15.40:0/4208795267 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354012830 con 0x7fb35c15eaa0 2023-10-18T09:09:01.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.581+0000 7fb363a32640 1 -- 172.21.15.40:0/4208795267 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35c1495b0 con 0x7fb35c15eaa0 2023-10-18T09:09:01.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.581+0000 7fb363a32640 1 -- 172.21.15.40:0/4208795267 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35c149b70 con 0x7fb35c15eaa0 2023-10-18T09:09:01.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.582+0000 7fb363a32640 1 -- 172.21.15.40:0/4208795267 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35c14a370 con 0x7fb35c15eaa0 2023-10-18T09:09:01.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.583+0000 7fb351ffb640 1 -- 172.21.15.40:0/4208795267 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb35401b400 con 0x7fb35c15eaa0 2023-10-18T09:09:01.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.584+0000 7fb351ffb640 1 --2- 172.21.15.40:0/4208795267 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb34006ae40 0x7fb34006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:01.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.584+0000 7fb351ffb640 1 -- 172.21.15.40:0/4208795267 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb35408a100 con 0x7fb35c15eaa0 2023-10-18T09:09:01.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.585+0000 7fb3617ce640 1 --2- 172.21.15.40:0/4208795267 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb34006ae40 0x7fb34006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:01.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.586+0000 7fb3617ce640 1 --2- 172.21.15.40:0/4208795267 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb34006ae40 0x7fb34006d300 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb358036000 tx=0x7fb358004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:01.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.589+0000 7fb351ffb640 1 -- 172.21.15.40:0/4208795267 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb35c14a370 con 0x7fb35c15eaa0 2023-10-18T09:09:01.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.825+0000 7fb363a32640 1 -- 172.21.15.40:0/4208795267 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb35c14a330 con 0x7fb35c15eaa0 2023-10-18T09:09:01.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.826+0000 7fb351ffb640 1 -- 172.21.15.40:0/4208795267 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4923 (secure 0 0 0) 0x7fb3540551a0 con 0x7fb35c15eaa0 2023-10-18T09:09:01.830 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:01.830 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2038256442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2038256442},{"type":"v1","addr":"172.21.15.144:6836","nonce":2038256442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3006484507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3006484507},{"type":"v1","addr":"172.21.15.144:6837","nonce":3006484507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1542000032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1542000032},{"type":"v1","addr":"172.21.15.40:6834","nonce":1542000032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:08:57.731109+0000","modified":"2023-10-18T09:09:00.823289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3976883051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3976883051},{"type":"v1","addr":"172.21.15.40:6835","nonce":3976883051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-18T09:09:01.830 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 90 2023-10-18T09:09:01.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.829+0000 7fb363a32640 1 -- 172.21.15.40:0/4208795267 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb34006ae40 msgr2=0x7fb34006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:01.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.829+0000 7fb363a32640 1 --2- 172.21.15.40:0/4208795267 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb34006ae40 0x7fb34006d300 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb358036000 tx=0x7fb358004730 comp rx=0 tx=0).stop 2023-10-18T09:09:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.829+0000 7fb363a32640 1 -- 172.21.15.40:0/4208795267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb35c15eaa0 msgr2=0x7fb35c1e6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.829+0000 7fb363a32640 1 --2- 172.21.15.40:0/4208795267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb35c15eaa0 0x7fb35c1e6090 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fb35400cb40 tx=0x7fb35400a060 comp rx=0 tx=0).stop 2023-10-18T09:09:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.829+0000 7fb363a32640 1 -- 172.21.15.40:0/4208795267 shutdown_connections 2023-10-18T09:09:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.829+0000 7fb363a32640 1 --2- 172.21.15.40:0/4208795267 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb34006ae40 0x7fb34006d300 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.829+0000 7fb363a32640 1 --2- 172.21.15.40:0/4208795267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb35c15eaa0 0x7fb35c1e6090 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:01.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.829+0000 7fb363a32640 1 --2- 172.21.15.40:0/4208795267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb35c0b1b90 0x7fb35c0ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:01.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.829+0000 7fb363a32640 1 --2- 172.21.15.40:0/4208795267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb35c0b12a0 0x7fb35c1f1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:01.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.829+0000 7fb363a32640 1 -- 172.21.15.40:0/4208795267 >> 172.21.15.40:0/4208795267 conn(0x7fb35c06cee0 msgr2=0x7fb35c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:01.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.829+0000 7fb363a32640 1 -- 172.21.15.40:0/4208795267 shutdown_connections 2023-10-18T09:09:01.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:01.829+0000 7fb363a32640 1 -- 172.21.15.40:0/4208795267 wait complete. 2023-10-18T09:09:01.851 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:09:02.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.083+0000 7f6c3ffff640 1 -- 172.21.15.40:0/1561562150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c2c003970 msgr2=0x7f6c2c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.083+0000 7f6c3ffff640 1 --2- 172.21.15.40:0/1561562150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c2c003970 0x7f6c2c003d90 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000bb20 tx=0x7f6c2c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.083+0000 7f6c3f7fe640 1 -- 172.21.15.40:0/1561562150 reap_dead start 2023-10-18T09:09:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.083+0000 7f6c3ffff640 1 -- 172.21.15.40:0/1561562150 shutdown_connections 2023-10-18T09:09:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.083+0000 7f6c3ffff640 1 -- 172.21.15.40:0/1561562150 >> 172.21.15.40:0/1561562150 conn(0x7f6c4008ad00 msgr2=0x7f6c4009a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:02.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.083+0000 7f6c3ffff640 1 -- 172.21.15.40:0/1561562150 shutdown_connections 2023-10-18T09:09:02.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.083+0000 7f6c3ffff640 1 -- 172.21.15.40:0/1561562150 wait complete. 2023-10-18T09:09:02.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.085+0000 7f6c3ffff640 1 Processor -- start 2023-10-18T09:09:02.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.085+0000 7f6c3ffff640 1 -- start start 2023-10-18T09:09:02.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.086+0000 7f6c3ffff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c2c003970 0x7f6c400b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:02.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.086+0000 7f6c3ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c40150350 0x7f6c400ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:02.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.086+0000 7f6c3ffff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c400ccc70 0x7f6c400cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:02.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.086+0000 7f6c3ffff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400cd630 con 0x7f6c40150350 2023-10-18T09:09:02.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.086+0000 7f6c3ffff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c401507d0 con 0x7f6c400ccc70 2023-10-18T09:09:02.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.086+0000 7f6c3ffff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400ca6e0 con 0x7f6c2c003970 2023-10-18T09:09:02.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.086+0000 7f6c37fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c40150350 0x7f6c400ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:02.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.086+0000 7f6c37fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c40150350 0x7f6c400ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51236/0 (socket says 172.21.15.40:51236) 2023-10-18T09:09:02.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.086+0000 7f6c37fff640 1 -- 172.21.15.40:0/4079884571 learned_addr learned my addr 172.21.15.40:0/4079884571 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:02.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.087+0000 7f6c37fff640 1 -- 172.21.15.40:0/4079884571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c2c003970 msgr2=0x7f6c400b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:02.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.087+0000 7f6c3effd640 1 --2- 172.21.15.40:0/4079884571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c2c003970 0x7f6c400b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:02.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.087+0000 7f6c37fff640 1 --2- 172.21.15.40:0/4079884571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c2c003970 0x7f6c400b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:02.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.087+0000 7f6c3f7fe640 1 --2- 172.21.15.40:0/4079884571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c400ccc70 0x7f6c400cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:02.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.087+0000 7f6c37fff640 1 -- 172.21.15.40:0/4079884571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c400ccc70 msgr2=0x7f6c400cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:02.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.087+0000 7f6c37fff640 1 --2- 172.21.15.40:0/4079884571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c400ccc70 0x7f6c400cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:02.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.087+0000 7f6c37fff640 1 -- 172.21.15.40:0/4079884571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c400b44d0 con 0x7f6c40150350 2023-10-18T09:09:02.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.087+0000 7f6c3f7fe640 1 --2- 172.21.15.40:0/4079884571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c400ccc70 0x7f6c400cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.087+0000 7f6c37fff640 1 --2- 172.21.15.40:0/4079884571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c40150350 0x7f6c400ca1a0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f6c28010a40 tx=0x7f6c28010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.088+0000 7f6c377fe640 1 -- 172.21.15.40:0/4079884571 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2800ed70 con 0x7f6c40150350 2023-10-18T09:09:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.088+0000 7f6c377fe640 1 -- 172.21.15.40:0/4079884571 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6c2800eed0 con 0x7f6c40150350 2023-10-18T09:09:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.088+0000 7f6c377fe640 1 -- 172.21.15.40:0/4079884571 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28004290 con 0x7f6c40150350 2023-10-18T09:09:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.088+0000 7f6c3ffff640 1 -- 172.21.15.40:0/4079884571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c400b47b0 con 0x7f6c40150350 2023-10-18T09:09:02.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.088+0000 7f6c3ffff640 1 -- 172.21.15.40:0/4079884571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c400b4d70 con 0x7f6c40150350 2023-10-18T09:09:02.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.089+0000 7f6c3ffff640 1 -- 172.21.15.40:0/4079884571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c40071da0 con 0x7f6c40150350 2023-10-18T09:09:02.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.090+0000 7f6c377fe640 1 -- 172.21.15.40:0/4079884571 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6c280035f0 con 0x7f6c40150350 2023-10-18T09:09:02.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.091+0000 7f6c377fe640 1 --2- 172.21.15.40:0/4079884571 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6c2006ae70 0x7f6c2006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:02.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.091+0000 7f6c377fe640 1 -- 172.21.15.40:0/4079884571 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c28016070 con 0x7f6c40150350 2023-10-18T09:09:02.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.092+0000 7f6c3effd640 1 --2- 172.21.15.40:0/4079884571 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6c2006ae70 0x7f6c2006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:02.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.093+0000 7f6c3effd640 1 --2- 172.21.15.40:0/4079884571 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6c2006ae70 0x7f6c2006d330 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6c30037000 tx=0x7f6c3002e040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:02.098 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.095+0000 7f6c377fe640 1 -- 172.21.15.40:0/4079884571 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6c28055560 con 0x7f6c40150350 2023-10-18T09:09:02.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.289+0000 7f6c3ffff640 1 -- 172.21.15.40:0/4079884571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c40066e00 con 0x7f6c40150350 2023-10-18T09:09:02.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.291+0000 7f6c377fe640 1 -- 172.21.15.40:0/4079884571 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+16584 (secure 0 0 0) 0x7f6c28058b80 con 0x7f6c40150350 2023-10-18T09:09:02.295 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:02.295 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":196,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:08:59.281665+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:08:55.675479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-18T09:08:56.761888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:09:02.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.294+0000 7f6c3ffff640 1 -- 172.21.15.40:0/4079884571 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6c2006ae70 msgr2=0x7f6c2006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:02.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.294+0000 7f6c3ffff640 1 --2- 172.21.15.40:0/4079884571 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6c2006ae70 0x7f6c2006d330 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6c30037000 tx=0x7f6c3002e040 comp rx=0 tx=0).stop 2023-10-18T09:09:02.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.294+0000 7f6c3ffff640 1 -- 172.21.15.40:0/4079884571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c40150350 msgr2=0x7f6c400ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:02.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.294+0000 7f6c3ffff640 1 --2- 172.21.15.40:0/4079884571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c40150350 0x7f6c400ca1a0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f6c28010a40 tx=0x7f6c28010f00 comp rx=0 tx=0).stop 2023-10-18T09:09:02.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.294+0000 7f6c3ffff640 1 -- 172.21.15.40:0/4079884571 shutdown_connections 2023-10-18T09:09:02.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.294+0000 7f6c3ffff640 1 --2- 172.21.15.40:0/4079884571 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6c2006ae70 0x7f6c2006d330 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:02.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.294+0000 7f6c3ffff640 1 --2- 172.21.15.40:0/4079884571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c400ccc70 0x7f6c400cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:02.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.294+0000 7f6c3ffff640 1 --2- 172.21.15.40:0/4079884571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c40150350 0x7f6c400ca1a0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:02.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.294+0000 7f6c3ffff640 1 --2- 172.21.15.40:0/4079884571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c2c003970 0x7f6c400b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:02.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.294+0000 7f6c3ffff640 1 -- 172.21.15.40:0/4079884571 >> 172.21.15.40:0/4079884571 conn(0x7f6c4008ad00 msgr2=0x7f6c400cbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:02.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.294+0000 7f6c3ffff640 1 -- 172.21.15.40:0/4079884571 shutdown_connections 2023-10-18T09:09:02.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.294+0000 7f6c3ffff640 1 -- 172.21.15.40:0/4079884571 wait complete. 2023-10-18T09:09:02.316 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T09:09:02.317 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:09:02.345 INFO:tasks.ceph.mds.c.smithi040.stderr:2023-10-18T09:09:02.342+0000 7f3ed6808640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T09:09:02.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.552+0000 7f8ae1864640 1 -- 172.21.15.40:0/250541828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8adc151bd0 msgr2=0x7f8adc1586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:02.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.552+0000 7f8ae1864640 1 --2- 172.21.15.40:0/250541828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8adc151bd0 0x7f8adc1586a0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f8acc005fd0 tx=0x7f8acc02bd90 comp rx=0 tx=0).stop 2023-10-18T09:09:02.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.552+0000 7f8ae1864640 1 -- 172.21.15.40:0/250541828 shutdown_connections 2023-10-18T09:09:02.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.552+0000 7f8ae1864640 1 --2- 172.21.15.40:0/250541828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8adc0b0bc0 0x7f8adc0b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:02.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.552+0000 7f8ae1864640 1 --2- 172.21.15.40:0/250541828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8adc151bd0 0x7f8adc1586a0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:02.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.552+0000 7f8ae1864640 1 -- 172.21.15.40:0/250541828 >> 172.21.15.40:0/250541828 conn(0x7f8adc06cee0 msgr2=0x7f8adc069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:02.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.552+0000 7f8ae1864640 1 -- 172.21.15.40:0/250541828 shutdown_connections 2023-10-18T09:09:02.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.552+0000 7f8ae1864640 1 -- 172.21.15.40:0/250541828 wait complete. 2023-10-18T09:09:02.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.554+0000 7f8ae1864640 1 Processor -- start 2023-10-18T09:09:02.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.554+0000 7f8ae1864640 1 -- start start 2023-10-18T09:09:02.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.556+0000 7f8ae1864640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8adc06b710 0x7f8adc146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:02.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.556+0000 7f8ae1864640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8adc0b0bc0 0x7f8adc14fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:02.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.556+0000 7f8ae1864640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8adc151bd0 0x7f8adc159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:02.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.556+0000 7f8ae1864640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc14bd20 con 0x7f8adc06b710 2023-10-18T09:09:02.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.556+0000 7f8ae1864640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc14b980 con 0x7f8adc151bd0 2023-10-18T09:09:02.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.556+0000 7f8ae1864640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc0ca160 con 0x7f8adc0b0bc0 2023-10-18T09:09:02.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.556+0000 7f8adaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8adc06b710 0x7f8adc146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:02.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.556+0000 7f8adaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8adc06b710 0x7f8adc146f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51244/0 (socket says 172.21.15.40:51244) 2023-10-18T09:09:02.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.556+0000 7f8adaffd640 1 -- 172.21.15.40:0/4109569193 learned_addr learned my addr 172.21.15.40:0/4109569193 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:02.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.557+0000 7f8adaffd640 1 -- 172.21.15.40:0/4109569193 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8adc0b0bc0 msgr2=0x7f8adc14fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:02.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.557+0000 7f8adaffd640 1 --2- 172.21.15.40:0/4109569193 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8adc0b0bc0 0x7f8adc14fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:02.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.557+0000 7f8adb7fe640 1 --2- 172.21.15.40:0/4109569193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8adc151bd0 0x7f8adc159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:02.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.557+0000 7f8adaffd640 1 -- 172.21.15.40:0/4109569193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8adc151bd0 msgr2=0x7f8adc159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:02.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.557+0000 7f8adaffd640 1 --2- 172.21.15.40:0/4109569193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8adc151bd0 0x7f8adc159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:02.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.557+0000 7f8adaffd640 1 -- 172.21.15.40:0/4109569193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8acc009560 con 0x7f8adc06b710 2023-10-18T09:09:02.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.557+0000 7f8adb7fe640 1 --2- 172.21.15.40:0/4109569193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8adc151bd0 0x7f8adc159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:02.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.557+0000 7f8adaffd640 1 --2- 172.21.15.40:0/4109569193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8adc06b710 0x7f8adc146f30 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009850 tx=0x7f8acc02db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:02.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.557+0000 7f8ad3fff640 1 -- 172.21.15.40:0/4109569193 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc0035f0 con 0x7f8adc06b710 2023-10-18T09:09:02.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.557+0000 7f8ad3fff640 1 -- 172.21.15.40:0/4109569193 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8acc003750 con 0x7f8adc06b710 2023-10-18T09:09:02.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.557+0000 7f8ad3fff640 1 -- 172.21.15.40:0/4109569193 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc003b00 con 0x7f8adc06b710 2023-10-18T09:09:02.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.557+0000 7f8ae1864640 1 -- 172.21.15.40:0/4109569193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc0ca440 con 0x7f8adc06b710 2023-10-18T09:09:02.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.558+0000 7f8ae1864640 1 -- 172.21.15.40:0/4109569193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc147470 con 0x7f8adc06b710 2023-10-18T09:09:02.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.559+0000 7f8ae1864640 1 -- 172.21.15.40:0/4109569193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc14be60 con 0x7f8adc06b710 2023-10-18T09:09:02.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.560+0000 7f8ad3fff640 1 -- 172.21.15.40:0/4109569193 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8acc003c60 con 0x7f8adc06b710 2023-10-18T09:09:02.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.561+0000 7f8ad3fff640 1 --2- 172.21.15.40:0/4109569193 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ac006ae40 0x7f8ac006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:02.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.561+0000 7f8ad3fff640 1 -- 172.21.15.40:0/4109569193 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8acc0ac270 con 0x7f8adc06b710 2023-10-18T09:09:02.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.561+0000 7f8ada7fc640 1 --2- 172.21.15.40:0/4109569193 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ac006ae40 0x7f8ac006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:02.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.562+0000 7f8ada7fc640 1 --2- 172.21.15.40:0/4109569193 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ac006ae40 0x7f8ac006d300 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8adc0cba30 tx=0x7f8ac4005f70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:02.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.565+0000 7f8ad3fff640 1 -- 172.21.15.40:0/4109569193 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8acc077430 con 0x7f8adc06b710 2023-10-18T09:09:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.757+0000 7f8ae1864640 1 -- 172.21.15.40:0/4109569193 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8adc147600 con 0x7f8ac006ae40 2023-10-18T09:09:02.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.781+0000 7f8ad3fff640 1 -- 172.21.15.40:0/4109569193 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282496 (secure 0 0 0) 0x7f8adc147600 con 0x7f8ac006ae40 2023-10-18T09:09:02.788 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:02.790 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:09:02.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.788+0000 7f8ae1864640 1 -- 172.21.15.40:0/4109569193 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ac006ae40 msgr2=0x7f8ac006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:02.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.788+0000 7f8ae1864640 1 --2- 172.21.15.40:0/4109569193 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ac006ae40 0x7f8ac006d300 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8adc0cba30 tx=0x7f8ac4005f70 comp rx=0 tx=0).stop 2023-10-18T09:09:02.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.788+0000 7f8ae1864640 1 -- 172.21.15.40:0/4109569193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8adc06b710 msgr2=0x7f8adc146f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:02.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.788+0000 7f8ae1864640 1 --2- 172.21.15.40:0/4109569193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8adc06b710 0x7f8adc146f30 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009850 tx=0x7f8acc02db10 comp rx=0 tx=0).stop 2023-10-18T09:09:02.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.789+0000 7f8ae1864640 1 -- 172.21.15.40:0/4109569193 shutdown_connections 2023-10-18T09:09:02.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.789+0000 7f8ae1864640 1 --2- 172.21.15.40:0/4109569193 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8ac006ae40 0x7f8ac006d300 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:02.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.789+0000 7f8ae1864640 1 --2- 172.21.15.40:0/4109569193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8adc151bd0 0x7f8adc159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:02.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.789+0000 7f8ae1864640 1 --2- 172.21.15.40:0/4109569193 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8adc0b0bc0 0x7f8adc14fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:02.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.789+0000 7f8ae1864640 1 --2- 172.21.15.40:0/4109569193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8adc06b710 0x7f8adc146f30 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:02.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.789+0000 7f8ae1864640 1 -- 172.21.15.40:0/4109569193 >> 172.21.15.40:0/4109569193 conn(0x7f8adc06cee0 msgr2=0x7f8adc1566f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:02.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.789+0000 7f8ae1864640 1 -- 172.21.15.40:0/4109569193 shutdown_connections 2023-10-18T09:09:02.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:02.789+0000 7f8ae1864640 1 -- 172.21.15.40:0/4109569193 wait complete. 2023-10-18T09:09:02.811 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":608,"stamp":"2023-10-18T09:09:01.202721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552672,"statfs":{"total":1600000000,"available":1589936237,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.964182"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.303087+0000","last_change":"2023-10-18T09:08:58.301123+0000","last_active":"2023-10-18T09:08:59.303087+0000","last_peered":"2023-10-18T09:08:59.303087+0000","last_clean":"2023-10-18T09:08:59.303087+0000","last_became_active":"2023-10-18T09:08:58.299127+0000","last_became_peered":"2023-10-18T09:08:58.299127+0000","last_unstale":"2023-10-18T09:08:59.303087+0000","last_undegraded":"2023-10-18T09:08:59.303087+0000","last_fullsized":"2023-10-18T09:08:59.303087+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:04.216360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.295891+0000","last_change":"2023-10-18T09:08:58.293508+0000","last_active":"2023-10-18T09:08:58.295891+0000","last_peered":"2023-10-18T09:08:58.295891+0000","last_clean":"2023-10-18T09:08:58.295891+0000","last_became_active":"2023-10-18T09:08:58.289859+0000","last_became_peered":"2023-10-18T09:08:58.289859+0000","last_unstale":"2023-10-18T09:08:58.295891+0000","last_undegraded":"2023-10-18T09:08:58.295891+0000","last_fullsized":"2023-10-18T09:08:58.295891+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:02:58.590071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.305851+0000","last_change":"2023-10-18T09:08:58.305759+0000","last_active":"2023-10-18T09:08:58.305851+0000","last_peered":"2023-10-18T09:08:58.305851+0000","last_clean":"2023-10-18T09:08:58.305851+0000","last_became_active":"2023-10-18T09:08:58.303077+0000","last_became_peered":"2023-10-18T09:08:58.303077+0000","last_unstale":"2023-10-18T09:08:58.305851+0000","last_undegraded":"2023-10-18T09:08:58.305851+0000","last_fullsized":"2023-10-18T09:08:58.305851+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:42:33.712181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.284332+0000","last_change":"2023-10-18T09:08:58.284253+0000","last_active":"2023-10-18T09:08:58.284332+0000","last_peered":"2023-10-18T09:08:58.284332+0000","last_clean":"2023-10-18T09:08:58.284332+0000","last_became_active":"2023-10-18T09:08:58.283823+0000","last_became_peered":"2023-10-18T09:08:58.283823+0000","last_unstale":"2023-10-18T09:08:58.284332+0000","last_undegraded":"2023-10-18T09:08:58.284332+0000","last_fullsized":"2023-10-18T09:08:58.284332+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:54:01.850390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.303265+0000","last_change":"2023-10-18T09:08:58.303187+0000","last_active":"2023-10-18T09:08:58.303265+0000","last_peered":"2023-10-18T09:08:58.303265+0000","last_clean":"2023-10-18T09:08:58.303265+0000","last_became_active":"2023-10-18T09:08:58.302925+0000","last_became_peered":"2023-10-18T09:08:58.302925+0000","last_unstale":"2023-10-18T09:08:58.303265+0000","last_undegraded":"2023-10-18T09:08:58.303265+0000","last_fullsized":"2023-10-18T09:08:58.303265+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:04:51.602293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.287796+0000","last_change":"2023-10-18T09:08:58.286998+0000","last_active":"2023-10-18T09:08:58.287796+0000","last_peered":"2023-10-18T09:08:58.287796+0000","last_clean":"2023-10-18T09:08:58.287796+0000","last_became_active":"2023-10-18T09:08:58.286729+0000","last_became_peered":"2023-10-18T09:08:58.286729+0000","last_unstale":"2023-10-18T09:08:58.287796+0000","last_undegraded":"2023-10-18T09:08:58.287796+0000","last_fullsized":"2023-10-18T09:08:58.287796+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:09:38.541576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.298787+0000","last_change":"2023-10-18T09:08:58.298547+0000","last_active":"2023-10-18T09:08:58.298787+0000","last_peered":"2023-10-18T09:08:58.298787+0000","last_clean":"2023-10-18T09:08:58.298787+0000","last_became_active":"2023-10-18T09:08:58.298285+0000","last_became_peered":"2023-10-18T09:08:58.298285+0000","last_unstale":"2023-10-18T09:08:58.298787+0000","last_undegraded":"2023-10-18T09:08:58.298787+0000","last_fullsized":"2023-10-18T09:08:58.298787+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:20.940715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.300490+0000","last_change":"2023-10-18T09:08:58.300390+0000","last_active":"2023-10-18T09:08:58.300490+0000","last_peered":"2023-10-18T09:08:58.300490+0000","last_clean":"2023-10-18T09:08:58.300490+0000","last_became_active":"2023-10-18T09:08:58.300072+0000","last_became_peered":"2023-10-18T09:08:58.300072+0000","last_unstale":"2023-10-18T09:08:58.300490+0000","last_undegraded":"2023-10-18T09:08:58.300490+0000","last_fullsized":"2023-10-18T09:08:58.300490+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:11:50.609557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.289301+0000","last_change":"2023-10-18T09:08:58.288886+0000","last_active":"2023-10-18T09:08:58.289301+0000","last_peered":"2023-10-18T09:08:58.289301+0000","last_clean":"2023-10-18T09:08:58.289301+0000","last_became_active":"2023-10-18T09:08:58.288263+0000","last_became_peered":"2023-10-18T09:08:58.288263+0000","last_unstale":"2023-10-18T09:08:58.289301+0000","last_undegraded":"2023-10-18T09:08:58.289301+0000","last_fullsized":"2023-10-18T09:08:58.289301+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:12:02.873800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.291416+0000","last_change":"2023-10-18T09:08:58.291260+0000","last_active":"2023-10-18T09:08:58.291416+0000","last_peered":"2023-10-18T09:08:58.291416+0000","last_clean":"2023-10-18T09:08:58.291416+0000","last_became_active":"2023-10-18T09:08:58.290926+0000","last_became_peered":"2023-10-18T09:08:58.290926+0000","last_unstale":"2023-10-18T09:08:58.291416+0000","last_undegraded":"2023-10-18T09:08:58.291416+0000","last_fullsized":"2023-10-18T09:08:58.291416+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:04.299458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.292753+0000","last_change":"2023-10-18T09:08:58.292667+0000","last_active":"2023-10-18T09:08:58.292753+0000","last_peered":"2023-10-18T09:08:58.292753+0000","last_clean":"2023-10-18T09:08:58.292753+0000","last_became_active":"2023-10-18T09:08:58.292386+0000","last_became_peered":"2023-10-18T09:08:58.292386+0000","last_unstale":"2023-10-18T09:08:58.292753+0000","last_undegraded":"2023-10-18T09:08:58.292753+0000","last_fullsized":"2023-10-18T09:08:58.292753+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:42.176897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.295909+0000","last_change":"2023-10-18T09:08:58.293751+0000","last_active":"2023-10-18T09:08:58.295909+0000","last_peered":"2023-10-18T09:08:58.295909+0000","last_clean":"2023-10-18T09:08:58.295909+0000","last_became_active":"2023-10-18T09:08:58.290425+0000","last_became_peered":"2023-10-18T09:08:58.290425+0000","last_unstale":"2023-10-18T09:08:58.295909+0000","last_undegraded":"2023-10-18T09:08:58.295909+0000","last_fullsized":"2023-10-18T09:08:58.295909+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:39:19.297430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.306112+0000","last_change":"2023-10-18T09:08:58.305992+0000","last_active":"2023-10-18T09:08:58.306112+0000","last_peered":"2023-10-18T09:08:58.306112+0000","last_clean":"2023-10-18T09:08:58.306112+0000","last_became_active":"2023-10-18T09:08:58.303900+0000","last_became_peered":"2023-10-18T09:08:58.303900+0000","last_unstale":"2023-10-18T09:08:58.306112+0000","last_undegraded":"2023-10-18T09:08:58.306112+0000","last_fullsized":"2023-10-18T09:08:58.306112+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:40:05.428277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.302165+0000","last_change":"2023-10-18T09:08:58.302084+0000","last_active":"2023-10-18T09:08:58.302165+0000","last_peered":"2023-10-18T09:08:58.302165+0000","last_clean":"2023-10-18T09:08:58.302165+0000","last_became_active":"2023-10-18T09:08:58.300396+0000","last_became_peered":"2023-10-18T09:08:58.300396+0000","last_unstale":"2023-10-18T09:08:58.302165+0000","last_undegraded":"2023-10-18T09:08:58.302165+0000","last_fullsized":"2023-10-18T09:08:58.302165+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:36.610169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.303190+0000","last_change":"2023-10-18T09:08:58.297855+0000","last_active":"2023-10-18T09:08:59.303190+0000","last_peered":"2023-10-18T09:08:59.303190+0000","last_clean":"2023-10-18T09:08:59.303190+0000","last_became_active":"2023-10-18T09:08:58.289254+0000","last_became_peered":"2023-10-18T09:08:58.289254+0000","last_unstale":"2023-10-18T09:08:59.303190+0000","last_undegraded":"2023-10-18T09:08:59.303190+0000","last_fullsized":"2023-10-18T09:08:59.303190+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:18:07.471451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.299352+0000","last_change":"2023-10-18T09:08:58.299211+0000","last_active":"2023-10-18T09:08:58.299352+0000","last_peered":"2023-10-18T09:08:58.299352+0000","last_clean":"2023-10-18T09:08:58.299352+0000","last_became_active":"2023-10-18T09:08:58.298975+0000","last_became_peered":"2023-10-18T09:08:58.298975+0000","last_unstale":"2023-10-18T09:08:58.299352+0000","last_undegraded":"2023-10-18T09:08:58.299352+0000","last_fullsized":"2023-10-18T09:08:58.299352+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:52:23.186942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.297706+0000","last_change":"2023-10-18T09:08:58.297506+0000","last_active":"2023-10-18T09:08:58.297706+0000","last_peered":"2023-10-18T09:08:58.297706+0000","last_clean":"2023-10-18T09:08:58.297706+0000","last_became_active":"2023-10-18T09:08:58.297289+0000","last_became_peered":"2023-10-18T09:08:58.297289+0000","last_unstale":"2023-10-18T09:08:58.297706+0000","last_undegraded":"2023-10-18T09:08:58.297706+0000","last_fullsized":"2023-10-18T09:08:58.297706+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:24.134042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.296066+0000","last_change":"2023-10-18T09:08:58.285630+0000","last_active":"2023-10-18T09:08:59.296066+0000","last_peered":"2023-10-18T09:08:59.296066+0000","last_clean":"2023-10-18T09:08:59.296066+0000","last_became_active":"2023-10-18T09:08:58.285384+0000","last_became_peered":"2023-10-18T09:08:58.285384+0000","last_unstale":"2023-10-18T09:08:59.296066+0000","last_undegraded":"2023-10-18T09:08:59.296066+0000","last_fullsized":"2023-10-18T09:08:59.296066+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:26:20.040703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.297626+0000","last_change":"2023-10-18T09:08:58.297479+0000","last_active":"2023-10-18T09:08:58.297626+0000","last_peered":"2023-10-18T09:08:58.297626+0000","last_clean":"2023-10-18T09:08:58.297626+0000","last_became_active":"2023-10-18T09:08:58.297242+0000","last_became_peered":"2023-10-18T09:08:58.297242+0000","last_unstale":"2023-10-18T09:08:58.297626+0000","last_undegraded":"2023-10-18T09:08:58.297626+0000","last_fullsized":"2023-10-18T09:08:58.297626+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:40:05.428277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.304510+0000","last_change":"2023-10-18T09:08:58.304105+0000","last_active":"2023-10-18T09:08:58.304510+0000","last_peered":"2023-10-18T09:08:58.304510+0000","last_clean":"2023-10-18T09:08:58.304510+0000","last_became_active":"2023-10-18T09:08:58.303838+0000","last_became_peered":"2023-10-18T09:08:58.303838+0000","last_unstale":"2023-10-18T09:08:58.304510+0000","last_undegraded":"2023-10-18T09:08:58.304510+0000","last_fullsized":"2023-10-18T09:08:58.304510+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:24.134042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.294990+0000","last_change":"2023-10-18T09:08:58.294608+0000","last_active":"2023-10-18T09:08:58.294990+0000","last_peered":"2023-10-18T09:08:58.294990+0000","last_clean":"2023-10-18T09:08:58.294990+0000","last_became_active":"2023-10-18T09:08:58.294339+0000","last_became_peered":"2023-10-18T09:08:58.294339+0000","last_unstale":"2023-10-18T09:08:58.294990+0000","last_undegraded":"2023-10-18T09:08:58.294990+0000","last_fullsized":"2023-10-18T09:08:58.294990+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:59:53.810779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.295871+0000","last_change":"2023-10-18T09:08:58.290937+0000","last_active":"2023-10-18T09:08:58.295871+0000","last_peered":"2023-10-18T09:08:58.295871+0000","last_clean":"2023-10-18T09:08:58.295871+0000","last_became_active":"2023-10-18T09:08:58.285959+0000","last_became_peered":"2023-10-18T09:08:58.285959+0000","last_unstale":"2023-10-18T09:08:58.295871+0000","last_undegraded":"2023-10-18T09:08:58.295871+0000","last_fullsized":"2023-10-18T09:08:58.295871+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:56:48.221743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.287815+0000","last_change":"2023-10-18T09:08:58.287564+0000","last_active":"2023-10-18T09:08:58.287815+0000","last_peered":"2023-10-18T09:08:58.287815+0000","last_clean":"2023-10-18T09:08:58.287815+0000","last_became_active":"2023-10-18T09:08:58.287100+0000","last_became_peered":"2023-10-18T09:08:58.287100+0000","last_unstale":"2023-10-18T09:08:58.287815+0000","last_undegraded":"2023-10-18T09:08:58.287815+0000","last_fullsized":"2023-10-18T09:08:58.287815+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:42:33.712181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.297308+0000","last_change":"2023-10-18T09:08:58.297237+0000","last_active":"2023-10-18T09:08:58.297308+0000","last_peered":"2023-10-18T09:08:58.297308+0000","last_clean":"2023-10-18T09:08:58.297308+0000","last_became_active":"2023-10-18T09:08:58.295901+0000","last_became_peered":"2023-10-18T09:08:58.295901+0000","last_unstale":"2023-10-18T09:08:58.297308+0000","last_undegraded":"2023-10-18T09:08:58.297308+0000","last_fullsized":"2023-10-18T09:08:58.297308+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:29.611460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.306246+0000","last_change":"2023-10-18T09:08:58.306092+0000","last_active":"2023-10-18T09:08:58.306246+0000","last_peered":"2023-10-18T09:08:58.306246+0000","last_clean":"2023-10-18T09:08:58.306246+0000","last_became_active":"2023-10-18T09:08:58.305854+0000","last_became_peered":"2023-10-18T09:08:58.305854+0000","last_unstale":"2023-10-18T09:08:58.306246+0000","last_undegraded":"2023-10-18T09:08:58.306246+0000","last_fullsized":"2023-10-18T09:08:58.306246+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:18.917111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.290419+0000","last_change":"2023-10-18T09:08:58.288905+0000","last_active":"2023-10-18T09:08:58.290419+0000","last_peered":"2023-10-18T09:08:58.290419+0000","last_clean":"2023-10-18T09:08:58.290419+0000","last_became_active":"2023-10-18T09:08:58.286172+0000","last_became_peered":"2023-10-18T09:08:58.286172+0000","last_unstale":"2023-10-18T09:08:58.290419+0000","last_undegraded":"2023-10-18T09:08:58.290419+0000","last_fullsized":"2023-10-18T09:08:58.290419+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:08:58.043466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.302142+0000","last_change":"2023-10-18T09:08:58.301002+0000","last_active":"2023-10-18T09:08:58.302142+0000","last_peered":"2023-10-18T09:08:58.302142+0000","last_clean":"2023-10-18T09:08:58.302142+0000","last_became_active":"2023-10-18T09:08:58.300486+0000","last_became_peered":"2023-10-18T09:08:58.300486+0000","last_unstale":"2023-10-18T09:08:58.302142+0000","last_undegraded":"2023-10-18T09:08:58.302142+0000","last_fullsized":"2023-10-18T09:08:58.302142+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:05:26.437571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.299227+0000","last_change":"2023-10-18T09:08:58.299004+0000","last_active":"2023-10-18T09:08:58.299227+0000","last_peered":"2023-10-18T09:08:58.299227+0000","last_clean":"2023-10-18T09:08:58.299227+0000","last_became_active":"2023-10-18T09:08:58.298766+0000","last_became_peered":"2023-10-18T09:08:58.298766+0000","last_unstale":"2023-10-18T09:08:58.299227+0000","last_undegraded":"2023-10-18T09:08:58.299227+0000","last_fullsized":"2023-10-18T09:08:58.299227+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:08:08.196353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.297774+0000","last_change":"2023-10-18T09:08:58.296363+0000","last_active":"2023-10-18T09:08:58.297774+0000","last_peered":"2023-10-18T09:08:58.297774+0000","last_clean":"2023-10-18T09:08:58.297774+0000","last_became_active":"2023-10-18T09:08:58.294410+0000","last_became_peered":"2023-10-18T09:08:58.294410+0000","last_unstale":"2023-10-18T09:08:58.297774+0000","last_undegraded":"2023-10-18T09:08:58.297774+0000","last_fullsized":"2023-10-18T09:08:58.297774+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:00:04.213601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.304314+0000","last_change":"2023-10-18T09:08:58.303648+0000","last_active":"2023-10-18T09:08:58.304314+0000","last_peered":"2023-10-18T09:08:58.304314+0000","last_clean":"2023-10-18T09:08:58.304314+0000","last_became_active":"2023-10-18T09:08:58.303344+0000","last_became_peered":"2023-10-18T09:08:58.303344+0000","last_unstale":"2023-10-18T09:08:58.304314+0000","last_undegraded":"2023-10-18T09:08:58.304314+0000","last_fullsized":"2023-10-18T09:08:58.304314+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:22.048918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.303475+0000","last_change":"2023-10-18T09:08:58.299835+0000","last_active":"2023-10-18T09:08:58.303475+0000","last_peered":"2023-10-18T09:08:58.303475+0000","last_clean":"2023-10-18T09:08:58.303475+0000","last_became_active":"2023-10-18T09:08:58.296142+0000","last_became_peered":"2023-10-18T09:08:58.296142+0000","last_unstale":"2023-10-18T09:08:58.303475+0000","last_undegraded":"2023-10-18T09:08:58.303475+0000","last_fullsized":"2023-10-18T09:08:58.303475+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:47.664727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295225+0000","last_change":"2023-10-18T09:08:58.287487+0000","last_active":"2023-10-18T09:08:59.295225+0000","last_peered":"2023-10-18T09:08:59.295225+0000","last_clean":"2023-10-18T09:08:59.295225+0000","last_became_active":"2023-10-18T09:08:58.287089+0000","last_became_peered":"2023-10-18T09:08:58.287089+0000","last_unstale":"2023-10-18T09:08:59.295225+0000","last_undegraded":"2023-10-18T09:08:59.295225+0000","last_fullsized":"2023-10-18T09:08:59.295225+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:54:41.020790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.287873+0000","last_change":"2023-10-18T09:08:58.286781+0000","last_active":"2023-10-18T09:08:58.287873+0000","last_peered":"2023-10-18T09:08:58.287873+0000","last_clean":"2023-10-18T09:08:58.287873+0000","last_became_active":"2023-10-18T09:08:58.286541+0000","last_became_peered":"2023-10-18T09:08:58.286541+0000","last_unstale":"2023-10-18T09:08:58.287873+0000","last_undegraded":"2023-10-18T09:08:58.287873+0000","last_fullsized":"2023-10-18T09:08:58.287873+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:07:11.888595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"195'4","reported_seq":16,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.302496+0000","last_change":"2023-10-18T09:08:57.271399+0000","last_active":"2023-10-18T09:08:59.302496+0000","last_peered":"2023-10-18T09:08:59.302496+0000","last_clean":"2023-10-18T09:08:59.302496+0000","last_became_active":"2023-10-18T09:08:57.271095+0000","last_became_peered":"2023-10-18T09:08:57.271095+0000","last_unstale":"2023-10-18T09:08:59.302496+0000","last_undegraded":"2023-10-18T09:08:59.302496+0000","last_fullsized":"2023-10-18T09:08:59.302496+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:23:21.063445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.303427+0000","last_change":"2023-10-18T09:08:58.298966+0000","last_active":"2023-10-18T09:08:58.303427+0000","last_peered":"2023-10-18T09:08:58.303427+0000","last_clean":"2023-10-18T09:08:58.303427+0000","last_became_active":"2023-10-18T09:08:58.298654+0000","last_became_peered":"2023-10-18T09:08:58.298654+0000","last_unstale":"2023-10-18T09:08:58.303427+0000","last_undegraded":"2023-10-18T09:08:58.303427+0000","last_fullsized":"2023-10-18T09:08:58.303427+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:19.770858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"195'1","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.305900+0000","last_change":"2023-10-18T09:08:57.275486+0000","last_active":"2023-10-18T09:08:58.305900+0000","last_peered":"2023-10-18T09:08:58.305900+0000","last_clean":"2023-10-18T09:08:58.305900+0000","last_became_active":"2023-10-18T09:08:57.275259+0000","last_became_peered":"2023-10-18T09:08:57.275259+0000","last_unstale":"2023-10-18T09:08:58.305900+0000","last_undegraded":"2023-10-18T09:08:58.305900+0000","last_fullsized":"2023-10-18T09:08:58.305900+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:53.638412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.290010+0000","last_change":"2023-10-18T09:08:58.289817+0000","last_active":"2023-10-18T09:08:58.290010+0000","last_peered":"2023-10-18T09:08:58.290010+0000","last_clean":"2023-10-18T09:08:58.290010+0000","last_became_active":"2023-10-18T09:08:58.289385+0000","last_became_peered":"2023-10-18T09:08:58.289385+0000","last_unstale":"2023-10-18T09:08:58.290010+0000","last_undegraded":"2023-10-18T09:08:58.290010+0000","last_fullsized":"2023-10-18T09:08:58.290010+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:00:04.213601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"195'2","reported_seq":13,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.299074+0000","last_change":"2023-10-18T09:08:57.274875+0000","last_active":"2023-10-18T09:08:58.299074+0000","last_peered":"2023-10-18T09:08:58.299074+0000","last_clean":"2023-10-18T09:08:58.299074+0000","last_became_active":"2023-10-18T09:08:57.274421+0000","last_became_peered":"2023-10-18T09:08:57.274421+0000","last_unstale":"2023-10-18T09:08:58.299074+0000","last_undegraded":"2023-10-18T09:08:58.299074+0000","last_fullsized":"2023-10-18T09:08:58.299074+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:43:12.302636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.297795+0000","last_change":"2023-10-18T09:08:58.297232+0000","last_active":"2023-10-18T09:08:58.297795+0000","last_peered":"2023-10-18T09:08:58.297795+0000","last_clean":"2023-10-18T09:08:58.297795+0000","last_became_active":"2023-10-18T09:08:58.296971+0000","last_became_peered":"2023-10-18T09:08:58.296971+0000","last_unstale":"2023-10-18T09:08:58.297795+0000","last_undegraded":"2023-10-18T09:08:58.297795+0000","last_fullsized":"2023-10-18T09:08:58.297795+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:43:50.186716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"195'2","reported_seq":13,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.299278+0000","last_change":"2023-10-18T09:08:57.263282+0000","last_active":"2023-10-18T09:08:58.299278+0000","last_peered":"2023-10-18T09:08:58.299278+0000","last_clean":"2023-10-18T09:08:58.299278+0000","last_became_active":"2023-10-18T09:08:57.262973+0000","last_became_peered":"2023-10-18T09:08:57.262973+0000","last_unstale":"2023-10-18T09:08:58.299278+0000","last_undegraded":"2023-10-18T09:08:58.299278+0000","last_fullsized":"2023-10-18T09:08:58.299278+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:15.422049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.283059+0000","last_change":"2023-10-18T09:08:58.282982+0000","last_active":"2023-10-18T09:08:58.283059+0000","last_peered":"2023-10-18T09:08:58.283059+0000","last_clean":"2023-10-18T09:08:58.283059+0000","last_became_active":"2023-10-18T09:08:58.282571+0000","last_became_peered":"2023-10-18T09:08:58.282571+0000","last_unstale":"2023-10-18T09:08:58.283059+0000","last_undegraded":"2023-10-18T09:08:58.283059+0000","last_fullsized":"2023-10-18T09:08:58.283059+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:13:13.077242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"195'1","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.305443+0000","last_change":"2023-10-18T09:08:57.269774+0000","last_active":"2023-10-18T09:08:58.305443+0000","last_peered":"2023-10-18T09:08:58.305443+0000","last_clean":"2023-10-18T09:08:58.305443+0000","last_became_active":"2023-10-18T09:08:57.269450+0000","last_became_peered":"2023-10-18T09:08:57.269450+0000","last_unstale":"2023-10-18T09:08:58.305443+0000","last_undegraded":"2023-10-18T09:08:58.305443+0000","last_fullsized":"2023-10-18T09:08:58.305443+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:29:33.964918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.292885+0000","last_change":"2023-10-18T09:08:58.292782+0000","last_active":"2023-10-18T09:08:58.292885+0000","last_peered":"2023-10-18T09:08:58.292885+0000","last_clean":"2023-10-18T09:08:58.292885+0000","last_became_active":"2023-10-18T09:08:58.292044+0000","last_became_peered":"2023-10-18T09:08:58.292044+0000","last_unstale":"2023-10-18T09:08:58.292885+0000","last_undegraded":"2023-10-18T09:08:58.292885+0000","last_fullsized":"2023-10-18T09:08:58.292885+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:01:45.963996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"195'1","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.295805+0000","last_change":"2023-10-18T09:08:57.260538+0000","last_active":"2023-10-18T09:08:58.295805+0000","last_peered":"2023-10-18T09:08:58.295805+0000","last_clean":"2023-10-18T09:08:58.295805+0000","last_became_active":"2023-10-18T09:08:57.260278+0000","last_became_peered":"2023-10-18T09:08:57.260278+0000","last_unstale":"2023-10-18T09:08:58.295805+0000","last_undegraded":"2023-10-18T09:08:58.295805+0000","last_fullsized":"2023-10-18T09:08:58.295805+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:04:29.565641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.294654+0000","last_change":"2023-10-18T09:08:58.294014+0000","last_active":"2023-10-18T09:08:58.294654+0000","last_peered":"2023-10-18T09:08:58.294654+0000","last_clean":"2023-10-18T09:08:58.294654+0000","last_became_active":"2023-10-18T09:08:58.293595+0000","last_became_peered":"2023-10-18T09:08:58.293595+0000","last_unstale":"2023-10-18T09:08:58.294654+0000","last_undegraded":"2023-10-18T09:08:58.294654+0000","last_fullsized":"2023-10-18T09:08:58.294654+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:12:54.154240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.273287+0000","last_change":"2023-10-18T09:08:57.276594+0000","last_active":"2023-10-18T09:08:58.273287+0000","last_peered":"2023-10-18T09:08:58.273287+0000","last_clean":"2023-10-18T09:08:58.273287+0000","last_became_active":"2023-10-18T09:08:57.276049+0000","last_became_peered":"2023-10-18T09:08:57.276049+0000","last_unstale":"2023-10-18T09:08:58.273287+0000","last_undegraded":"2023-10-18T09:08:58.273287+0000","last_fullsized":"2023-10-18T09:08:58.273287+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:26:57.248352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.285151+0000","last_change":"2023-10-18T09:08:58.283900+0000","last_active":"2023-10-18T09:08:58.285151+0000","last_peered":"2023-10-18T09:08:58.285151+0000","last_clean":"2023-10-18T09:08:58.285151+0000","last_became_active":"2023-10-18T09:08:58.283623+0000","last_became_peered":"2023-10-18T09:08:58.283623+0000","last_unstale":"2023-10-18T09:08:58.285151+0000","last_undegraded":"2023-10-18T09:08:58.285151+0000","last_fullsized":"2023-10-18T09:08:58.285151+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:36.610169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.280483+0000","last_change":"2023-10-18T09:08:57.259163+0000","last_active":"2023-10-18T09:08:58.280483+0000","last_peered":"2023-10-18T09:08:58.280483+0000","last_clean":"2023-10-18T09:08:58.280483+0000","last_became_active":"2023-10-18T09:08:57.258929+0000","last_became_peered":"2023-10-18T09:08:57.258929+0000","last_unstale":"2023-10-18T09:08:58.280483+0000","last_undegraded":"2023-10-18T09:08:58.280483+0000","last_fullsized":"2023-10-18T09:08:58.280483+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:00:30.305343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295869+0000","last_change":"2023-10-18T09:08:58.289895+0000","last_active":"2023-10-18T09:08:59.295869+0000","last_peered":"2023-10-18T09:08:59.295869+0000","last_clean":"2023-10-18T09:08:59.295869+0000","last_became_active":"2023-10-18T09:08:58.289660+0000","last_became_peered":"2023-10-18T09:08:58.289660+0000","last_unstale":"2023-10-18T09:08:59.295869+0000","last_undegraded":"2023-10-18T09:08:59.295869+0000","last_fullsized":"2023-10-18T09:08:59.295869+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:54.648974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.285379+0000","last_change":"2023-10-18T09:08:57.272340+0000","last_active":"2023-10-18T09:08:58.285379+0000","last_peered":"2023-10-18T09:08:58.285379+0000","last_clean":"2023-10-18T09:08:58.285379+0000","last_became_active":"2023-10-18T09:08:57.272135+0000","last_became_peered":"2023-10-18T09:08:57.272135+0000","last_unstale":"2023-10-18T09:08:58.285379+0000","last_undegraded":"2023-10-18T09:08:58.285379+0000","last_fullsized":"2023-10-18T09:08:58.285379+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:58:59.554028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295302+0000","last_change":"2023-10-18T09:08:58.290706+0000","last_active":"2023-10-18T09:08:59.295302+0000","last_peered":"2023-10-18T09:08:59.295302+0000","last_clean":"2023-10-18T09:08:59.295302+0000","last_became_active":"2023-10-18T09:08:58.290345+0000","last_became_peered":"2023-10-18T09:08:58.290345+0000","last_unstale":"2023-10-18T09:08:59.295302+0000","last_undegraded":"2023-10-18T09:08:59.295302+0000","last_fullsized":"2023-10-18T09:08:59.295302+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:39.065783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"195'3","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.290237+0000","last_change":"2023-10-18T09:08:57.272635+0000","last_active":"2023-10-18T09:08:58.290237+0000","last_peered":"2023-10-18T09:08:58.290237+0000","last_clean":"2023-10-18T09:08:58.290237+0000","last_became_active":"2023-10-18T09:08:57.272462+0000","last_became_peered":"2023-10-18T09:08:57.272462+0000","last_unstale":"2023-10-18T09:08:58.290237+0000","last_undegraded":"2023-10-18T09:08:58.290237+0000","last_fullsized":"2023-10-18T09:08:58.290237+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:58.866489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.306382+0000","last_change":"2023-10-18T09:08:58.305865+0000","last_active":"2023-10-18T09:08:58.306382+0000","last_peered":"2023-10-18T09:08:58.306382+0000","last_clean":"2023-10-18T09:08:58.306382+0000","last_became_active":"2023-10-18T09:08:58.304785+0000","last_became_peered":"2023-10-18T09:08:58.304785+0000","last_unstale":"2023-10-18T09:08:58.306382+0000","last_undegraded":"2023-10-18T09:08:58.306382+0000","last_fullsized":"2023-10-18T09:08:58.306382+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:06.196209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"195'6","reported_seq":17,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.307313+0000","last_change":"2023-10-18T09:08:57.272859+0000","last_active":"2023-10-18T09:08:58.307313+0000","last_peered":"2023-10-18T09:08:58.307313+0000","last_clean":"2023-10-18T09:08:58.307313+0000","last_became_active":"2023-10-18T09:08:57.272592+0000","last_became_peered":"2023-10-18T09:08:57.272592+0000","last_unstale":"2023-10-18T09:08:58.307313+0000","last_undegraded":"2023-10-18T09:08:58.307313+0000","last_fullsized":"2023-10-18T09:08:58.307313+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:26:19.030141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295966+0000","last_change":"2023-10-18T09:08:58.285606+0000","last_active":"2023-10-18T09:08:59.295966+0000","last_peered":"2023-10-18T09:08:59.295966+0000","last_clean":"2023-10-18T09:08:59.295966+0000","last_became_active":"2023-10-18T09:08:58.285357+0000","last_became_peered":"2023-10-18T09:08:58.285357+0000","last_unstale":"2023-10-18T09:08:59.295966+0000","last_undegraded":"2023-10-18T09:08:59.295966+0000","last_fullsized":"2023-10-18T09:08:59.295966+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:19:18.416271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"195'2","reported_seq":13,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.297215+0000","last_change":"2023-10-18T09:08:57.256549+0000","last_active":"2023-10-18T09:08:58.297215+0000","last_peered":"2023-10-18T09:08:58.297215+0000","last_clean":"2023-10-18T09:08:58.297215+0000","last_became_active":"2023-10-18T09:08:57.255882+0000","last_became_peered":"2023-10-18T09:08:57.255882+0000","last_unstale":"2023-10-18T09:08:58.297215+0000","last_undegraded":"2023-10-18T09:08:58.297215+0000","last_fullsized":"2023-10-18T09:08:58.297215+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:15:16.826765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.292269+0000","last_change":"2023-10-18T09:08:58.291201+0000","last_active":"2023-10-18T09:08:58.292269+0000","last_peered":"2023-10-18T09:08:58.292269+0000","last_clean":"2023-10-18T09:08:58.292269+0000","last_became_active":"2023-10-18T09:08:58.290943+0000","last_became_peered":"2023-10-18T09:08:58.290943+0000","last_unstale":"2023-10-18T09:08:58.292269+0000","last_undegraded":"2023-10-18T09:08:58.292269+0000","last_fullsized":"2023-10-18T09:08:58.292269+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:28:56.074539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.296023+0000","last_change":"2023-10-18T09:08:57.260749+0000","last_active":"2023-10-18T09:08:59.296023+0000","last_peered":"2023-10-18T09:08:59.296023+0000","last_clean":"2023-10-18T09:08:59.296023+0000","last_became_active":"2023-10-18T09:08:57.260427+0000","last_became_peered":"2023-10-18T09:08:57.260427+0000","last_unstale":"2023-10-18T09:08:59.296023+0000","last_undegraded":"2023-10-18T09:08:59.296023+0000","last_fullsized":"2023-10-18T09:08:59.296023+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:58.193868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.305275+0000","last_change":"2023-10-18T09:08:58.305200+0000","last_active":"2023-10-18T09:08:58.305275+0000","last_peered":"2023-10-18T09:08:58.305275+0000","last_clean":"2023-10-18T09:08:58.305275+0000","last_became_active":"2023-10-18T09:08:58.304940+0000","last_became_peered":"2023-10-18T09:08:58.304940+0000","last_unstale":"2023-10-18T09:08:58.305275+0000","last_undegraded":"2023-10-18T09:08:58.305275+0000","last_fullsized":"2023-10-18T09:08:58.305275+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:35.614777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.285991+0000","last_change":"2023-10-18T09:08:57.274472+0000","last_active":"2023-10-18T09:08:58.285991+0000","last_peered":"2023-10-18T09:08:58.285991+0000","last_clean":"2023-10-18T09:08:58.285991+0000","last_became_active":"2023-10-18T09:08:57.274159+0000","last_became_peered":"2023-10-18T09:08:57.274159+0000","last_unstale":"2023-10-18T09:08:58.285991+0000","last_undegraded":"2023-10-18T09:08:58.285991+0000","last_fullsized":"2023-10-18T09:08:58.285991+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:39:10.454786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.297825+0000","last_change":"2023-10-18T09:08:58.295437+0000","last_active":"2023-10-18T09:08:58.297825+0000","last_peered":"2023-10-18T09:08:58.297825+0000","last_clean":"2023-10-18T09:08:58.297825+0000","last_became_active":"2023-10-18T09:08:58.295108+0000","last_became_peered":"2023-10-18T09:08:58.295108+0000","last_unstale":"2023-10-18T09:08:58.297825+0000","last_undegraded":"2023-10-18T09:08:58.297825+0000","last_fullsized":"2023-10-18T09:08:58.297825+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:59:53.810779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"195'8","reported_seq":19,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.309253+0000","last_change":"2023-10-18T09:08:57.272858+0000","last_active":"2023-10-18T09:08:58.309253+0000","last_peered":"2023-10-18T09:08:58.309253+0000","last_clean":"2023-10-18T09:08:58.309253+0000","last_became_active":"2023-10-18T09:08:57.272590+0000","last_became_peered":"2023-10-18T09:08:57.272590+0000","last_unstale":"2023-10-18T09:08:58.309253+0000","last_undegraded":"2023-10-18T09:08:58.309253+0000","last_fullsized":"2023-10-18T09:08:58.309253+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:50:09.074412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.292853+0000","last_change":"2023-10-18T09:08:58.289757+0000","last_active":"2023-10-18T09:08:58.292853+0000","last_peered":"2023-10-18T09:08:58.292853+0000","last_clean":"2023-10-18T09:08:58.292853+0000","last_became_active":"2023-10-18T09:08:58.287329+0000","last_became_peered":"2023-10-18T09:08:58.287329+0000","last_unstale":"2023-10-18T09:08:58.292853+0000","last_undegraded":"2023-10-18T09:08:58.292853+0000","last_fullsized":"2023-10-18T09:08:58.292853+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:20.114971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"195'1","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.301319+0000","last_change":"2023-10-18T09:08:57.275895+0000","last_active":"2023-10-18T09:08:58.301319+0000","last_peered":"2023-10-18T09:08:58.301319+0000","last_clean":"2023-10-18T09:08:58.301319+0000","last_became_active":"2023-10-18T09:08:57.275641+0000","last_became_peered":"2023-10-18T09:08:57.275641+0000","last_unstale":"2023-10-18T09:08:58.301319+0000","last_undegraded":"2023-10-18T09:08:58.301319+0000","last_fullsized":"2023-10-18T09:08:58.301319+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:16.784281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295409+0000","last_change":"2023-10-18T09:08:58.291480+0000","last_active":"2023-10-18T09:08:59.295409+0000","last_peered":"2023-10-18T09:08:59.295409+0000","last_clean":"2023-10-18T09:08:59.295409+0000","last_became_active":"2023-10-18T09:08:58.291249+0000","last_became_peered":"2023-10-18T09:08:58.291249+0000","last_unstale":"2023-10-18T09:08:59.295409+0000","last_undegraded":"2023-10-18T09:08:59.295409+0000","last_fullsized":"2023-10-18T09:08:59.295409+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:06.196209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"195'2","reported_seq":13,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.296729+0000","last_change":"2023-10-18T09:08:57.267486+0000","last_active":"2023-10-18T09:08:58.296729+0000","last_peered":"2023-10-18T09:08:58.296729+0000","last_clean":"2023-10-18T09:08:58.296729+0000","last_became_active":"2023-10-18T09:08:57.266986+0000","last_became_peered":"2023-10-18T09:08:57.266986+0000","last_unstale":"2023-10-18T09:08:58.296729+0000","last_undegraded":"2023-10-18T09:08:58.296729+0000","last_fullsized":"2023-10-18T09:08:58.296729+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:58:59.554028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.292870+0000","last_change":"2023-10-18T09:08:58.292648+0000","last_active":"2023-10-18T09:08:58.292870+0000","last_peered":"2023-10-18T09:08:58.292870+0000","last_clean":"2023-10-18T09:08:58.292870+0000","last_became_active":"2023-10-18T09:08:58.291687+0000","last_became_peered":"2023-10-18T09:08:58.291687+0000","last_unstale":"2023-10-18T09:08:58.292870+0000","last_undegraded":"2023-10-18T09:08:58.292870+0000","last_fullsized":"2023-10-18T09:08:58.292870+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:58:49.948400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"195'1","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.303295+0000","last_change":"2023-10-18T09:08:57.270447+0000","last_active":"2023-10-18T09:08:59.303295+0000","last_peered":"2023-10-18T09:08:59.303295+0000","last_clean":"2023-10-18T09:08:59.303295+0000","last_became_active":"2023-10-18T09:08:57.269810+0000","last_became_peered":"2023-10-18T09:08:57.269810+0000","last_unstale":"2023-10-18T09:08:59.303295+0000","last_undegraded":"2023-10-18T09:08:59.303295+0000","last_fullsized":"2023-10-18T09:08:59.303295+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:27:36.905203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.295099+0000","last_change":"2023-10-18T09:08:58.295020+0000","last_active":"2023-10-18T09:08:58.295099+0000","last_peered":"2023-10-18T09:08:58.295099+0000","last_clean":"2023-10-18T09:08:58.295099+0000","last_became_active":"2023-10-18T09:08:58.294792+0000","last_became_peered":"2023-10-18T09:08:58.294792+0000","last_unstale":"2023-10-18T09:08:58.295099+0000","last_undegraded":"2023-10-18T09:08:58.295099+0000","last_fullsized":"2023-10-18T09:08:58.295099+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:39:29.385855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.302470+0000","last_change":"2023-10-18T09:08:57.264924+0000","last_active":"2023-10-18T09:08:59.302470+0000","last_peered":"2023-10-18T09:08:59.302470+0000","last_clean":"2023-10-18T09:08:59.302470+0000","last_became_active":"2023-10-18T09:08:57.264637+0000","last_became_peered":"2023-10-18T09:08:57.264637+0000","last_unstale":"2023-10-18T09:08:59.302470+0000","last_undegraded":"2023-10-18T09:08:59.302470+0000","last_fullsized":"2023-10-18T09:08:59.302470+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:08:29.125397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.293185+0000","last_change":"2023-10-18T09:08:58.292779+0000","last_active":"2023-10-18T09:08:58.293185+0000","last_peered":"2023-10-18T09:08:58.293185+0000","last_clean":"2023-10-18T09:08:58.293185+0000","last_became_active":"2023-10-18T09:08:58.292472+0000","last_became_peered":"2023-10-18T09:08:58.292472+0000","last_unstale":"2023-10-18T09:08:58.293185+0000","last_undegraded":"2023-10-18T09:08:58.293185+0000","last_fullsized":"2023-10-18T09:08:58.293185+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:00:51.596748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"195'2","reported_seq":13,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.296500+0000","last_change":"2023-10-18T09:08:57.274036+0000","last_active":"2023-10-18T09:08:58.296500+0000","last_peered":"2023-10-18T09:08:58.296500+0000","last_clean":"2023-10-18T09:08:58.296500+0000","last_became_active":"2023-10-18T09:08:57.273698+0000","last_became_peered":"2023-10-18T09:08:57.273698+0000","last_unstale":"2023-10-18T09:08:58.296500+0000","last_undegraded":"2023-10-18T09:08:58.296500+0000","last_fullsized":"2023-10-18T09:08:58.296500+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:43:01.706671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.290487+0000","last_change":"2023-10-18T09:08:58.290424+0000","last_active":"2023-10-18T09:08:58.290487+0000","last_peered":"2023-10-18T09:08:58.290487+0000","last_clean":"2023-10-18T09:08:58.290487+0000","last_became_active":"2023-10-18T09:08:58.290191+0000","last_became_peered":"2023-10-18T09:08:58.290191+0000","last_unstale":"2023-10-18T09:08:58.290487+0000","last_undegraded":"2023-10-18T09:08:58.290487+0000","last_fullsized":"2023-10-18T09:08:58.290487+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:45:36.037890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.302902+0000","last_change":"2023-10-18T09:08:57.268081+0000","last_active":"2023-10-18T09:08:59.302902+0000","last_peered":"2023-10-18T09:08:59.302902+0000","last_clean":"2023-10-18T09:08:59.302902+0000","last_became_active":"2023-10-18T09:08:57.266320+0000","last_became_peered":"2023-10-18T09:08:57.266320+0000","last_unstale":"2023-10-18T09:08:59.302902+0000","last_undegraded":"2023-10-18T09:08:59.302902+0000","last_fullsized":"2023-10-18T09:08:59.302902+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:29:35.653154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.302741+0000","last_change":"2023-10-18T09:08:58.299498+0000","last_active":"2023-10-18T09:08:59.302741+0000","last_peered":"2023-10-18T09:08:59.302741+0000","last_clean":"2023-10-18T09:08:59.302741+0000","last_became_active":"2023-10-18T09:08:58.295206+0000","last_became_peered":"2023-10-18T09:08:58.295206+0000","last_unstale":"2023-10-18T09:08:59.302741+0000","last_undegraded":"2023-10-18T09:08:59.302741+0000","last_fullsized":"2023-10-18T09:08:59.302741+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:35.502752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.284078+0000","last_change":"2023-10-18T09:08:57.269644+0000","last_active":"2023-10-18T09:08:58.284078+0000","last_peered":"2023-10-18T09:08:58.284078+0000","last_clean":"2023-10-18T09:08:58.284078+0000","last_became_active":"2023-10-18T09:08:57.269200+0000","last_became_peered":"2023-10-18T09:08:57.269200+0000","last_unstale":"2023-10-18T09:08:58.284078+0000","last_undegraded":"2023-10-18T09:08:58.284078+0000","last_fullsized":"2023-10-18T09:08:58.284078+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:07:22.471253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.296941+0000","last_change":"2023-10-18T09:08:58.296811+0000","last_active":"2023-10-18T09:08:58.296941+0000","last_peered":"2023-10-18T09:08:58.296941+0000","last_clean":"2023-10-18T09:08:58.296941+0000","last_became_active":"2023-10-18T09:08:58.296547+0000","last_became_peered":"2023-10-18T09:08:58.296547+0000","last_unstale":"2023-10-18T09:08:58.296941+0000","last_undegraded":"2023-10-18T09:08:58.296941+0000","last_fullsized":"2023-10-18T09:08:58.296941+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:18.917111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.303408+0000","last_change":"2023-10-18T09:08:57.270917+0000","last_active":"2023-10-18T09:08:59.303408+0000","last_peered":"2023-10-18T09:08:59.303408+0000","last_clean":"2023-10-18T09:08:59.303408+0000","last_became_active":"2023-10-18T09:08:57.270200+0000","last_became_peered":"2023-10-18T09:08:57.270200+0000","last_unstale":"2023-10-18T09:08:59.303408+0000","last_undegraded":"2023-10-18T09:08:59.303408+0000","last_fullsized":"2023-10-18T09:08:59.303408+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:12:16.804218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.296212+0000","last_change":"2023-10-18T09:08:58.290176+0000","last_active":"2023-10-18T09:08:59.296212+0000","last_peered":"2023-10-18T09:08:59.296212+0000","last_clean":"2023-10-18T09:08:59.296212+0000","last_became_active":"2023-10-18T09:08:58.289924+0000","last_became_peered":"2023-10-18T09:08:58.289924+0000","last_unstale":"2023-10-18T09:08:59.296212+0000","last_undegraded":"2023-10-18T09:08:59.296212+0000","last_fullsized":"2023-10-18T09:08:59.296212+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:19:48.023027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"195'2","reported_seq":14,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295814+0000","last_change":"2023-10-18T09:08:57.273975+0000","last_active":"2023-10-18T09:08:59.295814+0000","last_peered":"2023-10-18T09:08:59.295814+0000","last_clean":"2023-10-18T09:08:59.295814+0000","last_became_active":"2023-10-18T09:08:57.273716+0000","last_became_peered":"2023-10-18T09:08:57.273716+0000","last_unstale":"2023-10-18T09:08:59.295814+0000","last_undegraded":"2023-10-18T09:08:59.295814+0000","last_fullsized":"2023-10-18T09:08:59.295814+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:19:38.085128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.295857+0000","last_change":"2023-10-18T09:08:58.295726+0000","last_active":"2023-10-18T09:08:58.295857+0000","last_peered":"2023-10-18T09:08:58.295857+0000","last_clean":"2023-10-18T09:08:58.295857+0000","last_became_active":"2023-10-18T09:08:58.295228+0000","last_became_peered":"2023-10-18T09:08:58.295228+0000","last_unstale":"2023-10-18T09:08:58.295857+0000","last_undegraded":"2023-10-18T09:08:58.295857+0000","last_fullsized":"2023-10-18T09:08:58.295857+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:49.455678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.286450+0000","last_change":"2023-10-18T09:08:57.273659+0000","last_active":"2023-10-18T09:08:58.286450+0000","last_peered":"2023-10-18T09:08:58.286450+0000","last_clean":"2023-10-18T09:08:58.286450+0000","last_became_active":"2023-10-18T09:08:57.271740+0000","last_became_peered":"2023-10-18T09:08:57.271740+0000","last_unstale":"2023-10-18T09:08:58.286450+0000","last_undegraded":"2023-10-18T09:08:58.286450+0000","last_fullsized":"2023-10-18T09:08:58.286450+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:09:37.531014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.289993+0000","last_change":"2023-10-18T09:08:58.289782+0000","last_active":"2023-10-18T09:08:58.289993+0000","last_peered":"2023-10-18T09:08:58.289993+0000","last_clean":"2023-10-18T09:08:58.289993+0000","last_became_active":"2023-10-18T09:08:58.289433+0000","last_became_peered":"2023-10-18T09:08:58.289433+0000","last_unstale":"2023-10-18T09:08:58.289993+0000","last_undegraded":"2023-10-18T09:08:58.289993+0000","last_fullsized":"2023-10-18T09:08:58.289993+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:43:50.186716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.277245+0000","last_change":"2023-10-18T09:08:57.274980+0000","last_active":"2023-10-18T09:08:58.277245+0000","last_peered":"2023-10-18T09:08:58.277245+0000","last_clean":"2023-10-18T09:08:58.277245+0000","last_became_active":"2023-10-18T09:08:57.274419+0000","last_became_peered":"2023-10-18T09:08:57.274419+0000","last_unstale":"2023-10-18T09:08:58.277245+0000","last_undegraded":"2023-10-18T09:08:58.277245+0000","last_fullsized":"2023-10-18T09:08:58.277245+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:16:52.127204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295502+0000","last_change":"2023-10-18T09:08:58.280781+0000","last_active":"2023-10-18T09:08:59.295502+0000","last_peered":"2023-10-18T09:08:59.295502+0000","last_clean":"2023-10-18T09:08:59.295502+0000","last_became_active":"2023-10-18T09:08:58.280496+0000","last_became_peered":"2023-10-18T09:08:58.280496+0000","last_unstale":"2023-10-18T09:08:59.295502+0000","last_undegraded":"2023-10-18T09:08:59.295502+0000","last_fullsized":"2023-10-18T09:08:59.295502+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:13:31.701039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295399+0000","last_change":"2023-10-18T09:08:57.269806+0000","last_active":"2023-10-18T09:08:59.295399+0000","last_peered":"2023-10-18T09:08:59.295399+0000","last_clean":"2023-10-18T09:08:59.295399+0000","last_became_active":"2023-10-18T09:08:57.269404+0000","last_became_peered":"2023-10-18T09:08:57.269404+0000","last_unstale":"2023-10-18T09:08:59.295399+0000","last_undegraded":"2023-10-18T09:08:59.295399+0000","last_fullsized":"2023-10-18T09:08:59.295399+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:03.288895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295600+0000","last_change":"2023-10-18T09:08:58.287324+0000","last_active":"2023-10-18T09:08:59.295600+0000","last_peered":"2023-10-18T09:08:59.295600+0000","last_clean":"2023-10-18T09:08:59.295600+0000","last_became_active":"2023-10-18T09:08:58.286756+0000","last_became_peered":"2023-10-18T09:08:58.286756+0000","last_unstale":"2023-10-18T09:08:59.295600+0000","last_undegraded":"2023-10-18T09:08:59.295600+0000","last_fullsized":"2023-10-18T09:08:59.295600+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:47.664727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.290585+0000","last_change":"2023-10-18T09:08:57.269211+0000","last_active":"2023-10-18T09:08:58.290585+0000","last_peered":"2023-10-18T09:08:58.290585+0000","last_clean":"2023-10-18T09:08:58.290585+0000","last_became_active":"2023-10-18T09:08:57.268692+0000","last_became_peered":"2023-10-18T09:08:57.268692+0000","last_unstale":"2023-10-18T09:08:58.290585+0000","last_undegraded":"2023-10-18T09:08:58.290585+0000","last_fullsized":"2023-10-18T09:08:58.290585+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:20.053571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.303452+0000","last_change":"2023-10-18T09:08:58.296575+0000","last_active":"2023-10-18T09:08:58.303452+0000","last_peered":"2023-10-18T09:08:58.303452+0000","last_clean":"2023-10-18T09:08:58.303452+0000","last_became_active":"2023-10-18T09:08:58.295763+0000","last_became_peered":"2023-10-18T09:08:58.295763+0000","last_unstale":"2023-10-18T09:08:58.303452+0000","last_undegraded":"2023-10-18T09:08:58.303452+0000","last_fullsized":"2023-10-18T09:08:58.303452+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:13:31.701039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.286116+0000","last_change":"2023-10-18T09:08:57.267271+0000","last_active":"2023-10-18T09:08:58.286116+0000","last_peered":"2023-10-18T09:08:58.286116+0000","last_clean":"2023-10-18T09:08:58.286116+0000","last_became_active":"2023-10-18T09:08:57.266863+0000","last_became_peered":"2023-10-18T09:08:57.266863+0000","last_unstale":"2023-10-18T09:08:58.286116+0000","last_undegraded":"2023-10-18T09:08:58.286116+0000","last_fullsized":"2023-10-18T09:08:58.286116+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:37:15.001127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.294128+0000","last_change":"2023-10-18T09:08:58.294054+0000","last_active":"2023-10-18T09:08:58.294128+0000","last_peered":"2023-10-18T09:08:58.294128+0000","last_clean":"2023-10-18T09:08:58.294128+0000","last_became_active":"2023-10-18T09:08:58.293786+0000","last_became_peered":"2023-10-18T09:08:58.293786+0000","last_unstale":"2023-10-18T09:08:58.294128+0000","last_undegraded":"2023-10-18T09:08:58.294128+0000","last_fullsized":"2023-10-18T09:08:58.294128+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:38.149676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.273407+0000","last_change":"2023-10-18T09:08:57.271862+0000","last_active":"2023-10-18T09:08:58.273407+0000","last_peered":"2023-10-18T09:08:58.273407+0000","last_clean":"2023-10-18T09:08:58.273407+0000","last_became_active":"2023-10-18T09:08:57.271531+0000","last_became_peered":"2023-10-18T09:08:57.271531+0000","last_unstale":"2023-10-18T09:08:58.273407+0000","last_undegraded":"2023-10-18T09:08:58.273407+0000","last_fullsized":"2023-10-18T09:08:58.273407+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:52:50.292445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.305581+0000","last_change":"2023-10-18T09:08:58.304237+0000","last_active":"2023-10-18T09:08:58.305581+0000","last_peered":"2023-10-18T09:08:58.305581+0000","last_clean":"2023-10-18T09:08:58.305581+0000","last_became_active":"2023-10-18T09:08:58.299066+0000","last_became_peered":"2023-10-18T09:08:58.299066+0000","last_unstale":"2023-10-18T09:08:58.305581+0000","last_undegraded":"2023-10-18T09:08:58.305581+0000","last_fullsized":"2023-10-18T09:08:58.305581+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:59.877051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.289692+0000","last_change":"2023-10-18T09:08:57.264121+0000","last_active":"2023-10-18T09:08:58.289692+0000","last_peered":"2023-10-18T09:08:58.289692+0000","last_clean":"2023-10-18T09:08:58.289692+0000","last_became_active":"2023-10-18T09:08:57.263752+0000","last_became_peered":"2023-10-18T09:08:57.263752+0000","last_unstale":"2023-10-18T09:08:58.289692+0000","last_undegraded":"2023-10-18T09:08:58.289692+0000","last_fullsized":"2023-10-18T09:08:58.289692+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:19:17.405709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295416+0000","last_change":"2023-10-18T09:08:58.287968+0000","last_active":"2023-10-18T09:08:59.295416+0000","last_peered":"2023-10-18T09:08:59.295416+0000","last_clean":"2023-10-18T09:08:59.295416+0000","last_became_active":"2023-10-18T09:08:58.287735+0000","last_became_peered":"2023-10-18T09:08:58.287735+0000","last_unstale":"2023-10-18T09:08:59.295416+0000","last_undegraded":"2023-10-18T09:08:59.295416+0000","last_fullsized":"2023-10-18T09:08:59.295416+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:22.048918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.290427+0000","last_change":"2023-10-18T09:08:57.267689+0000","last_active":"2023-10-18T09:08:58.290427+0000","last_peered":"2023-10-18T09:08:58.290427+0000","last_clean":"2023-10-18T09:08:58.290427+0000","last_became_active":"2023-10-18T09:08:57.266991+0000","last_became_peered":"2023-10-18T09:08:57.266991+0000","last_unstale":"2023-10-18T09:08:58.290427+0000","last_undegraded":"2023-10-18T09:08:58.290427+0000","last_fullsized":"2023-10-18T09:08:58.290427+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:19:47.012465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":355,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.275550+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:08:58.275550+0000","last_peered":"2023-10-18T09:08:58.275550+0000","last_clean":"2023-10-18T09:08:58.275550+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:08:58.275550+0000","last_undegraded":"2023-10-18T09:08:58.275550+0000","last_fullsized":"2023-10-18T09:08:58.275550+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575017,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1667,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193644,"statfs":{"total":200000000,"available":198292395,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0980000000000001,"15min":1.075},"min":{"1min":0.78800000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.522,"5min":1.635,"15min":1.6990000000000001},"last":1.448},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.131,"15min":1.091},"min":{"1min":0.95399999999999996,"5min":0.63300000000000001,"15min":0.63100000000000001},"max":{"1min":1.5940000000000001,"5min":1.724,"15min":1.97},"last":0.93500000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1879999999999999,"15min":1.165},"min":{"1min":0.92200000000000004,"5min":0.67200000000000004,"15min":0.61799999999999999},"max":{"1min":1.4330000000000001,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1379999999999999,"15min":1.1339999999999999},"min":{"1min":0.80300000000000005,"5min":0.63500000000000001,"15min":0.58799999999999997},"max":{"1min":1.3640000000000001,"5min":1.782,"15min":1.7989999999999999},"last":1.22}]},{"osd":2,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.361,"15min":1.3089999999999999},"min":{"1min":0.999,"5min":0.80700000000000005,"15min":0.65400000000000003},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.9730000000000001},"last":1.794},{"interface":"front","average":{"1min":1.276,"5min":1.2490000000000001,"15min":1.24},"min":{"1min":0.90500000000000003,"5min":0.72499999999999998,"15min":0.68300000000000005},"max":{"1min":1.7030000000000001,"5min":1.8169999999999999,"15min":1.863},"last":1.619}]},{"osd":3,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3080000000000001,"15min":1.284},"min":{"1min":1.0289999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6799999999999999,"5min":1.7330000000000001,"15min":1.98},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.377,"15min":1.355},"min":{"1min":1.206,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9930000000000001},"last":1.8640000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2110000000000001,"15min":1.2210000000000001},"min":{"1min":0.90600000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.625,"5min":1.869,"15min":2.1070000000000002},"last":1.7390000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3169999999999999,"15min":1.345},"min":{"1min":0.88400000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":2.5169999999999999},"last":1.665}]},{"osd":5,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.3380000000000001,"15min":1.347},"min":{"1min":1.056,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.71,"5min":1.7989999999999999,"15min":2.0720000000000001},"last":1.51},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.3740000000000001,"15min":1.4059999999999999},"min":{"1min":0.84999999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.736,"5min":1.9490000000000001,"15min":2.0270000000000001},"last":1.968}]},{"osd":6,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.462,"15min":1.4770000000000001},"min":{"1min":1.073,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.893,"5min":2.0139999999999998,"15min":2.3959999999999999},"last":2.0670000000000002},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.4570000000000001,"15min":1.4850000000000001},"min":{"1min":0.99399999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.8520000000000001,"5min":2.0680000000000001,"15min":2.1680000000000001},"last":1.8959999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575017,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1082,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194229,"statfs":{"total":200000000,"available":198891395,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:08:19 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.238,"15min":1.1970000000000001},"min":{"1min":0.79900000000000004,"5min":0.64200000000000002,"15min":0.60299999999999998},"max":{"1min":1.585,"5min":1.643,"15min":1.6850000000000001},"last":1.145},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.371,"15min":1.319},"min":{"1min":1.0960000000000001,"5min":0.88800000000000001,"15min":0.64700000000000002},"max":{"1min":1.7230000000000001,"5min":1.899,"15min":2.0990000000000002},"last":1.2110000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:08:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1739999999999999,"15min":1.121},"min":{"1min":0.69899999999999995,"5min":0.68799999999999994,"15min":0.63800000000000001},"max":{"1min":1.593,"5min":1.722,"15min":1.722},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.2450000000000001,"15min":1.216},"min":{"1min":0.90500000000000003,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.5669999999999999,"5min":1.8169999999999999,"15min":2.9609999999999999},"last":0.876}]},{"osd":2,"last update":"Wed Oct 18 09:08:19 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.2549999999999999,"15min":1.1950000000000001},"min":{"1min":0.93200000000000005,"5min":0.70299999999999996,"15min":0.69899999999999995},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.331,"15min":1.2549999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.66300000000000003},"max":{"1min":1.696,"5min":1.75,"15min":2.1779999999999999},"last":1.0189999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:08:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.2889999999999999,"15min":1.28},"min":{"1min":0.95999999999999996,"5min":0.82999999999999996,"15min":0.79200000000000004},"max":{"1min":1.728,"5min":1.8160000000000001,"15min":2.1589999999999998},"last":1.377},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.377,"15min":1.2969999999999999},"min":{"1min":0.92800000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.643,"5min":1.994,"15min":1.994},"last":0.93600000000000005}]},{"osd":4,"last update":"Wed Oct 18 09:08:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.3120000000000001,"15min":1.2589999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.58299999999999996},"max":{"1min":1.3819999999999999,"5min":1.9770000000000001,"15min":2.444},"last":1.115},{"interface":"front","average":{"1min":1.278,"5min":1.335,"15min":1.3029999999999999},"min":{"1min":0.91400000000000003,"5min":0.82999999999999996,"15min":0.55000000000000004},"max":{"1min":1.6439999999999999,"5min":1.9159999999999999,"15min":2.4039999999999999},"last":1.4870000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:08:19 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.4419999999999999,"15min":1.415},"min":{"1min":1.087,"5min":0.92900000000000005,"15min":0.58899999999999997},"max":{"1min":1.7889999999999999,"5min":2.016,"15min":2.032},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3839999999999999,"15min":1.3149999999999999},"min":{"1min":1.0309999999999999,"5min":0.77100000000000002,"15min":0.49199999999999999},"max":{"1min":1.631,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.3380000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:08:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4730000000000001,"15min":1.429},"min":{"1min":1.1299999999999999,"5min":0.89400000000000002,"15min":0.52000000000000002},"max":{"1min":1.698,"5min":2.028,"15min":2.3540000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.546,"15min":1.5329999999999999},"min":{"1min":1.258,"5min":1.1890000000000001,"15min":0.68500000000000005},"max":{"1min":1.7909999999999999,"5min":2.1440000000000001,"15min":2.794},"last":1.5649999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575018,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1083,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194229,"statfs":{"total":200000000,"available":198890813,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:08:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3169999999999999,"15min":1.3240000000000001},"min":{"1min":0.68799999999999994,"5min":0.64800000000000002,"15min":0.63400000000000001},"max":{"1min":2.302,"5min":2.302,"15min":9.609},"last":1.895},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.383,"15min":1.3300000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.60799999999999998},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":5.3029999999999999},"last":1.3089999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:08:32 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.2470000000000001,"15min":1.258},"min":{"1min":0.90400000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":11.68},"last":1.792},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.2450000000000001,"15min":1.216},"min":{"1min":0.89100000000000001,"5min":0.72799999999999998,"15min":0.67000000000000004},"max":{"1min":1.79,"5min":1.841,"15min":3.7050000000000001},"last":1.0329999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:08:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2749999999999999,"15min":1.262},"min":{"1min":0.90900000000000003,"5min":0.79700000000000004,"15min":0.755},"max":{"1min":1.8029999999999999,"5min":1.8540000000000001,"15min":2.2090000000000001},"last":1.121},{"interface":"front","average":{"1min":1.371,"5min":1.2789999999999999,"15min":1.2709999999999999},"min":{"1min":1.0780000000000001,"5min":0.79000000000000004,"15min":0.72399999999999998},"max":{"1min":1.706,"5min":1.7270000000000001,"15min":3.8180000000000001},"last":1.238}]},{"osd":3,"last update":"Wed Oct 18 09:08:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.216,"15min":1.262},"min":{"1min":0.745,"5min":0.745,"15min":0.65400000000000003},"max":{"1min":1.6719999999999999,"5min":1.9810000000000001,"15min":11.797000000000001},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.395,"15min":1.369},"min":{"1min":0.93899999999999995,"5min":0.92700000000000005,"15min":0.747},"max":{"1min":1.764,"5min":2.105,"15min":2.3420000000000001},"last":1.5549999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:08:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3300000000000001,"15min":1.377},"min":{"1min":0.94999999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.8560000000000001,"5min":2.028,"15min":2.4449999999999998},"last":1.7190000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.3180000000000001,"15min":1.355},"min":{"1min":0.90800000000000003,"5min":0.67700000000000005,"15min":0.64100000000000001},"max":{"1min":1.982,"5min":2.1360000000000001,"15min":5.4580000000000002},"last":1.1779999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:08:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.236,"15min":1.2929999999999999},"min":{"1min":0.80500000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.6599999999999999,"5min":2.1030000000000002,"15min":5.141},"last":1.27},{"interface":"front","average":{"1min":1.573,"5min":1.5600000000000001,"15min":1.538},"min":{"1min":1.1839999999999999,"5min":1.095,"15min":0.71099999999999997},"max":{"1min":1.923,"5min":2.161,"15min":2.657},"last":1.845}]},{"osd":7,"last update":"Wed Oct 18 09:08:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.466,"15min":1.552},"min":{"1min":0.96099999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.76,"5min":2.444,"15min":11.829000000000001},"last":1.504},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.401,"15min":1.4399999999999999},"min":{"1min":1.123,"5min":0.93300000000000005,"15min":0.73399999999999999},"max":{"1min":1.7110000000000001,"5min":2.6419999999999999,"15min":4.8419999999999996},"last":1.452}]}]},{"osd":4,"up_from":13,"seq":55834575017,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1083,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194229,"statfs":{"total":200000000,"available":198890754,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.167,"15min":1.141},"min":{"1min":0.76200000000000001,"5min":0.66300000000000003,"15min":0.45900000000000002},"max":{"1min":1.7410000000000001,"5min":1.8100000000000001,"15min":3.1819999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.1779999999999999,"15min":1.1559999999999999},"min":{"1min":1.0349999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.52,"5min":1.6339999999999999,"15min":3.234},"last":0.89000000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.1930000000000001,"15min":1.143},"min":{"1min":0.94199999999999995,"5min":0.70099999999999996,"15min":0.60799999999999998},"max":{"1min":1.71,"5min":1.9690000000000001,"15min":3.718},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1970000000000001,"15min":1.2070000000000001},"min":{"1min":0.95899999999999996,"5min":0.76300000000000001,"15min":0.51400000000000001},"max":{"1min":1.49,"5min":1.9259999999999999,"15min":3.4350000000000001},"last":1.258}]},{"osd":2,"last update":"Wed Oct 18 09:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.24,"15min":1.2150000000000001},"min":{"1min":0.93400000000000005,"5min":0.71799999999999997,"15min":0.56699999999999995},"max":{"1min":1.629,"5min":1.905,"15min":4.1749999999999998},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.2889999999999999,"15min":1.2989999999999999},"min":{"1min":0.88200000000000001,"5min":0.64500000000000002,"15min":0.58899999999999997},"max":{"1min":1.655,"5min":1.704,"15min":3.532},"last":1.397}]},{"osd":3,"last update":"Wed Oct 18 09:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3220000000000001,"15min":1.294},"min":{"1min":0.97399999999999998,"5min":0.73899999999999999,"15min":0.67000000000000004},"max":{"1min":1.6659999999999999,"5min":1.7310000000000001,"15min":3.363},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.389,"5min":1.3859999999999999,"15min":1.401},"min":{"1min":1.0609999999999999,"5min":0.92200000000000004,"15min":0.68899999999999995},"max":{"1min":1.778,"5min":1.778,"15min":4.3179999999999996},"last":1.306}]},{"osd":5,"last update":"Wed Oct 18 09:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.304,"15min":1.3},"min":{"1min":0.85199999999999998,"5min":0.55600000000000005,"15min":0.498},"max":{"1min":1.71,"5min":1.841,"15min":3.6160000000000001},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.3300000000000001,"15min":1.3260000000000001},"min":{"1min":0.88600000000000001,"5min":0.60099999999999998,"15min":0.38400000000000001},"max":{"1min":1.607,"5min":1.702,"15min":4.0780000000000003},"last":1.111}]},{"osd":6,"last update":"Wed Oct 18 09:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3600000000000001,"15min":1.389},"min":{"1min":0.91000000000000003,"5min":0.67400000000000004,"15min":0.51600000000000001},"max":{"1min":1.6339999999999999,"5min":1.873,"15min":5.0049999999999999},"last":1.599},{"interface":"front","average":{"1min":1.417,"5min":1.454,"15min":1.4890000000000001},"min":{"1min":1.0069999999999999,"5min":0.47499999999999998,"15min":0.45400000000000001},"max":{"1min":1.827,"5min":1.871,"15min":4.407},"last":1.661}]},{"osd":7,"last update":"Wed Oct 18 09:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.4339999999999999,"15min":1.454},"min":{"1min":0.97399999999999998,"5min":0.51200000000000001,"15min":0.438},"max":{"1min":1.7490000000000001,"5min":1.804,"15min":4.5170000000000003},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.452,"5min":1.5109999999999999,"15min":1.5329999999999999},"min":{"1min":1.014,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.6990000000000001,"5min":1.966,"15min":4.5460000000000003},"last":1.7390000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575019,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1088,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194223,"statfs":{"total":200000000,"available":198885083,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:08:25 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86699999999999999,"15min":0.89000000000000001},"min":{"1min":0.67400000000000004,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2430000000000001,"5min":1.827,"15min":5.5730000000000004},"last":2.0230000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":1.0069999999999999,"15min":0.97399999999999998},"min":{"1min":0.69399999999999995,"5min":0.40899999999999997,"15min":0.32700000000000001},"max":{"1min":1.2030000000000001,"5min":1.863,"15min":6.5599999999999996},"last":0.84699999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:08:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.0549999999999999,"15min":1.012},"min":{"1min":0.69899999999999995,"5min":0.52200000000000002,"15min":0.36799999999999999},"max":{"1min":1.526,"5min":2.2650000000000001,"15min":6.8099999999999996},"last":3.6680000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.319,"15min":1.2410000000000001},"min":{"1min":0.998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.6359999999999999,"5min":1.9670000000000001,"15min":3.1920000000000002},"last":3.7069999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:08:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.149,"15min":1.103},"min":{"1min":0.77600000000000002,"5min":0.58999999999999997,"15min":0.45300000000000001},"max":{"1min":1.492,"5min":1.9550000000000001,"15min":7.4580000000000002},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.246,"15min":1.1890000000000001},"min":{"1min":0.60699999999999998,"5min":0.52700000000000002,"15min":0.51400000000000001},"max":{"1min":1.764,"5min":1.8169999999999999,"15min":5.0049999999999999},"last":2.407}]},{"osd":4,"last update":"Wed Oct 18 09:08:25 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.391,"15min":1.3939999999999999},"min":{"1min":1.1240000000000001,"5min":0.65400000000000003,"15min":0.53700000000000003},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":18.664999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3700000000000001,"15min":1.294},"min":{"1min":1.0089999999999999,"5min":0.86499999999999999,"15min":0.64000000000000001},"max":{"1min":1.6359999999999999,"5min":2.0289999999999999,"15min":7.0700000000000003},"last":7.1269999999999998}]},{"osd":5,"last update":"Wed Oct 18 09:08:25 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.3660000000000001,"15min":1.325},"min":{"1min":1.099,"5min":0.626,"15min":0.626},"max":{"1min":1.786,"5min":1.8560000000000001,"15min":6.9139999999999997},"last":3.5819999999999999},{"interface":"front","average":{"1min":1.544,"5min":1.504,"15min":1.49},"min":{"1min":0.97599999999999998,"5min":0.72999999999999998,"15min":0.56899999999999995},"max":{"1min":2.032,"5min":2.6499999999999999,"15min":16.390999999999998},"last":2.5640000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:08:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.546,"15min":1.502},"min":{"1min":0.998,"5min":0.98899999999999999,"15min":0.58199999999999996},"max":{"1min":1.986,"5min":2.2839999999999998,"15min":7.7270000000000003},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.5449999999999999,"15min":1.478},"min":{"1min":1.169,"5min":1.0049999999999999,"15min":0.60199999999999998},"max":{"1min":1.8029999999999999,"5min":2.351,"15min":7.242},"last":1.8300000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:08:25 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.6259999999999999,"15min":1.5309999999999999},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.70099999999999996},"max":{"1min":1.927,"5min":2.8170000000000002,"15min":4.8630000000000004},"last":2.5099999999999998},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.619,"15min":1.5780000000000001},"min":{"1min":1.1859999999999999,"5min":0.82899999999999996,"15min":0.622},"max":{"1min":2.0800000000000001,"5min":2.4590000000000001,"15min":12.946},"last":5.7649999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575018,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1082,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194229,"statfs":{"total":200000000,"available":198891327,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.83499999999999996,"15min":0.874},"min":{"1min":0.47799999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":4.2210000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.032,"5min":0.996,"15min":1.014},"min":{"1min":0.74199999999999999,"5min":0.40200000000000002,"15min":0.33800000000000002},"max":{"1min":1.702,"5min":2.2799999999999998,"15min":4.3959999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.018,"15min":1.0600000000000001},"min":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.49099999999999999},"max":{"1min":1.454,"5min":1.6200000000000001,"15min":7.3390000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.234,"5min":1.0760000000000001,"15min":1.111},"min":{"1min":0.93200000000000005,"5min":0.55900000000000005,"15min":0.46200000000000002},"max":{"1min":1.696,"5min":1.9510000000000001,"15min":2.5350000000000001},"last":0.63300000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1080000000000001,"15min":1.129},"min":{"1min":0.67300000000000004,"5min":0.48299999999999998,"15min":0.48199999999999998},"max":{"1min":1.6850000000000001,"5min":1.696,"15min":7.6369999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.321,"15min":1.359},"min":{"1min":0.73299999999999998,"5min":0.60199999999999998,"15min":0.54100000000000004},"max":{"1min":1.8700000000000001,"5min":1.9139999999999999,"15min":7.5570000000000004},"last":0.74299999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3899999999999999,"15min":1.4299999999999999},"min":{"1min":0.93400000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.016,"5min":2.016,"15min":7.6189999999999998},"last":1.014},{"interface":"front","average":{"1min":1.444,"5min":1.3939999999999999,"15min":1.4330000000000001},"min":{"1min":0.98299999999999998,"5min":0.89200000000000002,"15min":0.86599999999999999},"max":{"1min":1.925,"5min":2.0430000000000001,"15min":3.423},"last":0.79500000000000004}]},{"osd":5,"last update":"Wed Oct 18 09:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2769999999999999,"15min":1.306},"min":{"1min":0.877,"5min":0.82399999999999995,"15min":0.61599999999999999},"max":{"1min":1.6859999999999999,"5min":2.0600000000000001,"15min":10.507},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.419,"5min":1.389,"15min":1.46},"min":{"1min":0.873,"5min":0.81599999999999995,"15min":0.81399999999999995},"max":{"1min":1.978,"5min":1.984,"15min":7.657},"last":1.145}]},{"osd":6,"last update":"Wed Oct 18 09:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.407,"15min":1.466},"min":{"1min":1.0600000000000001,"5min":0.77700000000000002,"15min":0.74299999999999999},"max":{"1min":2.149,"5min":2.2519999999999998,"15min":3.5299999999999998},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.431,"15min":1.6060000000000001},"min":{"1min":0.90100000000000002,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.081,"5min":2.2639999999999998,"15min":37.771999999999998},"last":0.84999999999999998}]},{"osd":7,"last update":"Wed Oct 18 09:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.6140000000000001,"5min":1.4990000000000001,"15min":1.542},"min":{"1min":1.018,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.04,"5min":2.3860000000000001,"15min":3.3620000000000001},"last":0.872},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.486,"15min":1.5209999999999999},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.77600000000000002},"max":{"1min":2.1800000000000002,"5min":2.375,"15min":10.956},"last":1.171}]}]},{"osd":1,"up_from":13,"seq":55834575017,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1081,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194230,"statfs":{"total":200000000,"available":198892402,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.92700000000000005,"15min":0.88600000000000001},"min":{"1min":0.56000000000000005,"5min":0.37,"15min":0.315},"max":{"1min":1.2150000000000001,"5min":1.6519999999999999,"15min":1.659},"last":0.57199999999999995},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.155,"15min":1.046},"min":{"1min":0.753,"5min":0.58899999999999997,"15min":0.441},"max":{"1min":1.6319999999999999,"5min":1.681,"15min":1.7010000000000001},"last":0.55300000000000005}]},{"osd":2,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.085,"15min":1.0149999999999999},"min":{"1min":0.76300000000000001,"5min":0.443,"15min":0.317},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.859},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.022,"5min":0.98599999999999999,"15min":0.93700000000000006},"min":{"1min":0.58699999999999997,"5min":0.443,"15min":0.36699999999999999},"max":{"1min":1.472,"5min":1.8009999999999999,"15min":2.0230000000000001},"last":0.80800000000000005}]},{"osd":3,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.175,"15min":1.099},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.436},"max":{"1min":1.5900000000000001,"5min":1.798,"15min":2.1059999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.101,"15min":1.073},"min":{"1min":0.66400000000000003,"5min":0.47899999999999998,"15min":0.35399999999999998},"max":{"1min":1.53,"5min":1.736,"15min":3.7010000000000001},"last":0.76800000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3149999999999999,"15min":1.294},"min":{"1min":1.1479999999999999,"5min":0.75600000000000001,"15min":0.45700000000000002},"max":{"1min":1.704,"5min":1.7070000000000001,"15min":2.0489999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3480000000000001,"15min":1.3080000000000001},"min":{"1min":0.93200000000000005,"5min":0.755,"15min":0.59699999999999998},"max":{"1min":1.6399999999999999,"5min":1.877,"15min":2.0470000000000002},"last":1.2589999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.492,"15min":1.393},"min":{"1min":1,"5min":0.86499999999999999,"15min":0.56000000000000005},"max":{"1min":1.7929999999999999,"5min":2.0539999999999998,"15min":3.5750000000000002},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.486,"15min":1.4399999999999999},"min":{"1min":1.04,"5min":0.76500000000000001,"15min":0.71699999999999997},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.3769999999999998},"last":1.204}]},{"osd":6,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4339999999999999,"15min":1.3799999999999999},"min":{"1min":1.0700000000000001,"5min":0.79100000000000004,"15min":0.64200000000000002},"max":{"1min":1.7170000000000001,"5min":1.9890000000000001,"15min":2.2050000000000001},"last":1.008},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.45,"15min":1.3819999999999999},"min":{"1min":1.2330000000000001,"5min":0.89000000000000001,"15min":0.63100000000000001},"max":{"1min":2.0859999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.92900000000000005}]},{"osd":7,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.5289999999999999,"15min":1.462},"min":{"1min":1.147,"5min":0.85699999999999998,"15min":0.69799999999999995},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.2229999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.581,"5min":1.49,"15min":1.4530000000000001},"min":{"1min":0.98199999999999998,"5min":0.81100000000000005,"15min":0.66300000000000003},"max":{"1min":2.1240000000000001,"5min":2.1269999999999998,"15min":2.4180000000000001},"last":1.28}]}]},{"osd":0,"up_from":13,"seq":55834575017,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1658,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193654,"statfs":{"total":200000000,"available":198302068,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.84299999999999997,"15min":0.85299999999999998},"min":{"1min":0.72299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2290000000000001,"5min":1.2430000000000001,"15min":3.4420000000000002},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.121,"15min":1.0900000000000001},"min":{"1min":0.93000000000000005,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.6020000000000001,"5min":1.6950000000000001,"15min":1.8240000000000001},"last":1.1879999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.89400000000000002,"15min":0.85899999999999999},"min":{"1min":0.73099999999999998,"5min":0.46300000000000002,"15min":0.375},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.679},"last":0.64900000000000002},{"interface":"front","average":{"1min":1.321,"5min":1.202,"15min":1.1539999999999999},"min":{"1min":0.94799999999999995,"5min":0.623,"15min":0.437},"max":{"1min":1.917,"5min":1.917,"15min":1.929},"last":1.121}]},{"osd":3,"last update":"Wed Oct 18 09:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.173,"15min":1.1759999999999999},"min":{"1min":0.86299999999999999,"5min":0.52100000000000002,"15min":0.49399999999999999},"max":{"1min":1.726,"5min":1.9590000000000001,"15min":2.7639999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.423,"15min":1.4259999999999999},"min":{"1min":1.262,"5min":0.68100000000000005,"15min":0.58399999999999996},"max":{"1min":1.8029999999999999,"5min":2.1200000000000001,"15min":3.5720000000000001},"last":1.4950000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.226,"15min":1.2390000000000001},"min":{"1min":0.97199999999999998,"5min":0.73099999999999998,"15min":0.71099999999999997},"max":{"1min":1.556,"5min":1.9990000000000001,"15min":4.492},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.288,"5min":1.2929999999999999,"15min":1.274},"min":{"1min":1.042,"5min":0.77500000000000002,"15min":0.68700000000000006},"max":{"1min":1.752,"5min":1.752,"15min":1.927},"last":1.157}]},{"osd":5,"last update":"Wed Oct 18 09:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.276,"15min":1.3029999999999999},"min":{"1min":0.91200000000000003,"5min":0.75600000000000001,"15min":0.64300000000000002},"max":{"1min":1.4830000000000001,"5min":2.1779999999999999,"15min":4.2779999999999996},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.353,"15min":1.339},"min":{"1min":1.095,"5min":0.81899999999999995,"15min":0.73099999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8740000000000001},"last":1.2649999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.474,"15min":1.4850000000000001},"min":{"1min":1.244,"5min":0.81000000000000005,"15min":0.76800000000000002},"max":{"1min":2.081,"5min":2.2469999999999999,"15min":3.5299999999999998},"last":1.288},{"interface":"front","average":{"1min":1.381,"5min":1.3640000000000001,"15min":1.3979999999999999},"min":{"1min":1.075,"5min":0.86899999999999999,"15min":0.66700000000000004},"max":{"1min":1.7330000000000001,"5min":1.8720000000000001,"15min":4.4349999999999996},"last":1.3029999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.6699999999999999,"5min":1.5589999999999999,"15min":1.5669999999999999},"min":{"1min":1.347,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.0209999999999999,"5min":2.2629999999999999,"15min":3.492},"last":1.383},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.4910000000000001,"15min":1.5189999999999999},"min":{"1min":1.1850000000000001,"5min":0.88500000000000001,"15min":0.82099999999999995},"max":{"1min":2.1000000000000001,"5min":2.3090000000000002,"15min":4.5419999999999998},"last":0.95999999999999996}]}]}],"pool_statfs":[]}} 2023-10-18T09:09:02.816 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:09:03.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.079+0000 7f5692793640 1 -- 172.21.15.40:0/1023378019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0b15b0 msgr2=0x7f568c0b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:03.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.079+0000 7f5692793640 1 --2- 172.21.15.40:0/1023378019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0b15b0 0x7f568c0b0630 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f568c0caf60 tx=0x7f568802bd70 comp rx=0 tx=0).stop 2023-10-18T09:09:03.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.079+0000 7f5692793640 1 -- 172.21.15.40:0/1023378019 shutdown_connections 2023-10-18T09:09:03.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.079+0000 7f5692793640 1 --2- 172.21.15.40:0/1023378019 >> v2:172.21.15.144:3300/0 conn(0x7f568c0ca0d0 0x7f568c157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.079+0000 7f5692793640 1 --2- 172.21.15.40:0/1023378019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0b15b0 0x7f568c0b0630 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.079+0000 7f5692793640 1 -- 172.21.15.40:0/1023378019 >> 172.21.15.40:0/1023378019 conn(0x7f568c069720 msgr2=0x7f568c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:03.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.079+0000 7f5692793640 1 -- 172.21.15.40:0/1023378019 shutdown_connections 2023-10-18T09:09:03.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.079+0000 7f5692793640 1 -- 172.21.15.40:0/1023378019 wait complete. 2023-10-18T09:09:03.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.081+0000 7f5692793640 1 Processor -- start 2023-10-18T09:09:03.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.081+0000 7f5692793640 1 -- start start 2023-10-18T09:09:03.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.082+0000 7f5692793640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c0b15b0 0x7f568c136680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:03.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.082+0000 7f5692793640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568c0b1e10 0x7f568c0af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:03.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.082+0000 7f5692793640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0ca0d0 0x7f568c155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:03.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.082+0000 7f5692793640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c1f1f30 con 0x7f568c0ca0d0 2023-10-18T09:09:03.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.082+0000 7f5692793640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c1f2210 con 0x7f568c0b1e10 2023-10-18T09:09:03.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.082+0000 7f5692793640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c1f24f0 con 0x7f568c0b15b0 2023-10-18T09:09:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.082+0000 7f5691f92640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0ca0d0 0x7f568c155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.082+0000 7f5691f92640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0ca0d0 0x7f568c155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51262/0 (socket says 172.21.15.40:51262) 2023-10-18T09:09:03.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.082+0000 7f5691f92640 1 -- 172.21.15.40:0/2663261768 learned_addr learned my addr 172.21.15.40:0/2663261768 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.083+0000 7f5690f90640 1 --2- 172.21.15.40:0/2663261768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568c0b1e10 0x7f568c0af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:03.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.083+0000 7f5691791640 1 --2- 172.21.15.40:0/2663261768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c0b15b0 0x7f568c136680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.083+0000 7f5691f92640 1 -- 172.21.15.40:0/2663261768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c0b15b0 msgr2=0x7f568c136680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.083+0000 7f5691f92640 1 --2- 172.21.15.40:0/2663261768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c0b15b0 0x7f568c136680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.083+0000 7f5691f92640 1 -- 172.21.15.40:0/2663261768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568c0b1e10 msgr2=0x7f568c0af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.083+0000 7f5691f92640 1 --2- 172.21.15.40:0/2663261768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568c0b1e10 0x7f568c0af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.083+0000 7f5691f92640 1 -- 172.21.15.40:0/2663261768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5688009490 con 0x7f568c0ca0d0 2023-10-18T09:09:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.083+0000 7f5691f92640 1 --2- 172.21.15.40:0/2663261768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0ca0d0 0x7f568c155ff0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f568400e960 tx=0x7f568400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.084+0000 7f5681ffb640 1 -- 172.21.15.40:0/2663261768 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684004d20 con 0x7f568c0ca0d0 2023-10-18T09:09:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.084+0000 7f5681ffb640 1 -- 172.21.15.40:0/2663261768 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f568400cb20 con 0x7f568c0ca0d0 2023-10-18T09:09:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.084+0000 7f5681ffb640 1 -- 172.21.15.40:0/2663261768 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684005020 con 0x7f568c0ca0d0 2023-10-18T09:09:03.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.084+0000 7f5692793640 1 -- 172.21.15.40:0/2663261768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c1f27d0 con 0x7f568c0ca0d0 2023-10-18T09:09:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.084+0000 7f5692793640 1 -- 172.21.15.40:0/2663261768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c136bc0 con 0x7f568c0ca0d0 2023-10-18T09:09:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.085+0000 7f5692793640 1 -- 172.21.15.40:0/2663261768 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c1f2cf0 con 0x7f568c0ca0d0 2023-10-18T09:09:03.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.086+0000 7f5681ffb640 1 -- 172.21.15.40:0/2663261768 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5684005180 con 0x7f568c0ca0d0 2023-10-18T09:09:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.087+0000 7f5681ffb640 1 --2- 172.21.15.40:0/2663261768 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f567006ae40 0x7f567006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.087+0000 7f5681ffb640 1 -- 172.21.15.40:0/2663261768 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5684089ff0 con 0x7f568c0ca0d0 2023-10-18T09:09:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.088+0000 7f5691791640 1 --2- 172.21.15.40:0/2663261768 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f567006ae40 0x7f567006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:03.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.089+0000 7f5691791640 1 --2- 172.21.15.40:0/2663261768 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f567006ae40 0x7f567006d300 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5688036000 tx=0x7f5688004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:03.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.092+0000 7f5681ffb640 1 -- 172.21.15.40:0/2663261768 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f56840bcca0 con 0x7f568c0ca0d0 2023-10-18T09:09:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.283+0000 7f5692793640 1 -- 172.21.15.40:0/2663261768 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f568c0b1990 con 0x7f567006ae40 2023-10-18T09:09:03.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.306+0000 7f5681ffb640 1 -- 172.21.15.40:0/2663261768 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282496 (secure 0 0 0) 0x7f568802c890 con 0x7f567006ae40 2023-10-18T09:09:03.311 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:03.315 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:09:03.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.313+0000 7f5692793640 1 -- 172.21.15.40:0/2663261768 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f567006ae40 msgr2=0x7f567006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:03.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.314+0000 7f5692793640 1 --2- 172.21.15.40:0/2663261768 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f567006ae40 0x7f567006d300 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5688036000 tx=0x7f5688004730 comp rx=0 tx=0).stop 2023-10-18T09:09:03.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.314+0000 7f5692793640 1 -- 172.21.15.40:0/2663261768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0ca0d0 msgr2=0x7f568c155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:03.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.314+0000 7f5692793640 1 --2- 172.21.15.40:0/2663261768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0ca0d0 0x7f568c155ff0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f568400e960 tx=0x7f568400ee20 comp rx=0 tx=0).stop 2023-10-18T09:09:03.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.314+0000 7f5692793640 1 -- 172.21.15.40:0/2663261768 shutdown_connections 2023-10-18T09:09:03.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.314+0000 7f5692793640 1 --2- 172.21.15.40:0/2663261768 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f567006ae40 0x7f567006d300 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.314+0000 7f5692793640 1 --2- 172.21.15.40:0/2663261768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f568c0ca0d0 0x7f568c155ff0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.314+0000 7f5692793640 1 --2- 172.21.15.40:0/2663261768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f568c0b1e10 0x7f568c0af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.314+0000 7f5692793640 1 --2- 172.21.15.40:0/2663261768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f568c0b15b0 0x7f568c136680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.314+0000 7f5692793640 1 -- 172.21.15.40:0/2663261768 >> 172.21.15.40:0/2663261768 conn(0x7f568c069720 msgr2=0x7f568c14c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:03.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.314+0000 7f5692793640 1 -- 172.21.15.40:0/2663261768 shutdown_connections 2023-10-18T09:09:03.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.314+0000 7f5692793640 1 -- 172.21.15.40:0/2663261768 wait complete. 2023-10-18T09:09:03.338 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":609,"stamp":"2023-10-18T09:09:03.203389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552672,"statfs":{"total":1600000000,"available":1589936237,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002004"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.303087+0000","last_change":"2023-10-18T09:08:58.301123+0000","last_active":"2023-10-18T09:08:59.303087+0000","last_peered":"2023-10-18T09:08:59.303087+0000","last_clean":"2023-10-18T09:08:59.303087+0000","last_became_active":"2023-10-18T09:08:58.299127+0000","last_became_peered":"2023-10-18T09:08:58.299127+0000","last_unstale":"2023-10-18T09:08:59.303087+0000","last_undegraded":"2023-10-18T09:08:59.303087+0000","last_fullsized":"2023-10-18T09:08:59.303087+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:00:04.216360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.295891+0000","last_change":"2023-10-18T09:08:58.293508+0000","last_active":"2023-10-18T09:08:58.295891+0000","last_peered":"2023-10-18T09:08:58.295891+0000","last_clean":"2023-10-18T09:08:58.295891+0000","last_became_active":"2023-10-18T09:08:58.289859+0000","last_became_peered":"2023-10-18T09:08:58.289859+0000","last_unstale":"2023-10-18T09:08:58.295891+0000","last_undegraded":"2023-10-18T09:08:58.295891+0000","last_fullsized":"2023-10-18T09:08:58.295891+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:02:58.590071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.305851+0000","last_change":"2023-10-18T09:08:58.305759+0000","last_active":"2023-10-18T09:08:58.305851+0000","last_peered":"2023-10-18T09:08:58.305851+0000","last_clean":"2023-10-18T09:08:58.305851+0000","last_became_active":"2023-10-18T09:08:58.303077+0000","last_became_peered":"2023-10-18T09:08:58.303077+0000","last_unstale":"2023-10-18T09:08:58.305851+0000","last_undegraded":"2023-10-18T09:08:58.305851+0000","last_fullsized":"2023-10-18T09:08:58.305851+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:42:33.712181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.284332+0000","last_change":"2023-10-18T09:08:58.284253+0000","last_active":"2023-10-18T09:08:58.284332+0000","last_peered":"2023-10-18T09:08:58.284332+0000","last_clean":"2023-10-18T09:08:58.284332+0000","last_became_active":"2023-10-18T09:08:58.283823+0000","last_became_peered":"2023-10-18T09:08:58.283823+0000","last_unstale":"2023-10-18T09:08:58.284332+0000","last_undegraded":"2023-10-18T09:08:58.284332+0000","last_fullsized":"2023-10-18T09:08:58.284332+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:54:01.850390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.303265+0000","last_change":"2023-10-18T09:08:58.303187+0000","last_active":"2023-10-18T09:08:58.303265+0000","last_peered":"2023-10-18T09:08:58.303265+0000","last_clean":"2023-10-18T09:08:58.303265+0000","last_became_active":"2023-10-18T09:08:58.302925+0000","last_became_peered":"2023-10-18T09:08:58.302925+0000","last_unstale":"2023-10-18T09:08:58.303265+0000","last_undegraded":"2023-10-18T09:08:58.303265+0000","last_fullsized":"2023-10-18T09:08:58.303265+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:04:51.602293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.287796+0000","last_change":"2023-10-18T09:08:58.286998+0000","last_active":"2023-10-18T09:08:58.287796+0000","last_peered":"2023-10-18T09:08:58.287796+0000","last_clean":"2023-10-18T09:08:58.287796+0000","last_became_active":"2023-10-18T09:08:58.286729+0000","last_became_peered":"2023-10-18T09:08:58.286729+0000","last_unstale":"2023-10-18T09:08:58.287796+0000","last_undegraded":"2023-10-18T09:08:58.287796+0000","last_fullsized":"2023-10-18T09:08:58.287796+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:09:38.541576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.298787+0000","last_change":"2023-10-18T09:08:58.298547+0000","last_active":"2023-10-18T09:08:58.298787+0000","last_peered":"2023-10-18T09:08:58.298787+0000","last_clean":"2023-10-18T09:08:58.298787+0000","last_became_active":"2023-10-18T09:08:58.298285+0000","last_became_peered":"2023-10-18T09:08:58.298285+0000","last_unstale":"2023-10-18T09:08:58.298787+0000","last_undegraded":"2023-10-18T09:08:58.298787+0000","last_fullsized":"2023-10-18T09:08:58.298787+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:31:20.940715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.300490+0000","last_change":"2023-10-18T09:08:58.300390+0000","last_active":"2023-10-18T09:08:58.300490+0000","last_peered":"2023-10-18T09:08:58.300490+0000","last_clean":"2023-10-18T09:08:58.300490+0000","last_became_active":"2023-10-18T09:08:58.300072+0000","last_became_peered":"2023-10-18T09:08:58.300072+0000","last_unstale":"2023-10-18T09:08:58.300490+0000","last_undegraded":"2023-10-18T09:08:58.300490+0000","last_fullsized":"2023-10-18T09:08:58.300490+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:11:50.609557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.289301+0000","last_change":"2023-10-18T09:08:58.288886+0000","last_active":"2023-10-18T09:08:58.289301+0000","last_peered":"2023-10-18T09:08:58.289301+0000","last_clean":"2023-10-18T09:08:58.289301+0000","last_became_active":"2023-10-18T09:08:58.288263+0000","last_became_peered":"2023-10-18T09:08:58.288263+0000","last_unstale":"2023-10-18T09:08:58.289301+0000","last_undegraded":"2023-10-18T09:08:58.289301+0000","last_fullsized":"2023-10-18T09:08:58.289301+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:12:02.873800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.291416+0000","last_change":"2023-10-18T09:08:58.291260+0000","last_active":"2023-10-18T09:08:58.291416+0000","last_peered":"2023-10-18T09:08:58.291416+0000","last_clean":"2023-10-18T09:08:58.291416+0000","last_became_active":"2023-10-18T09:08:58.290926+0000","last_became_peered":"2023-10-18T09:08:58.290926+0000","last_unstale":"2023-10-18T09:08:58.291416+0000","last_undegraded":"2023-10-18T09:08:58.291416+0000","last_fullsized":"2023-10-18T09:08:58.291416+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:04.299458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.292753+0000","last_change":"2023-10-18T09:08:58.292667+0000","last_active":"2023-10-18T09:08:58.292753+0000","last_peered":"2023-10-18T09:08:58.292753+0000","last_clean":"2023-10-18T09:08:58.292753+0000","last_became_active":"2023-10-18T09:08:58.292386+0000","last_became_peered":"2023-10-18T09:08:58.292386+0000","last_unstale":"2023-10-18T09:08:58.292753+0000","last_undegraded":"2023-10-18T09:08:58.292753+0000","last_fullsized":"2023-10-18T09:08:58.292753+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:17:42.176897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.295909+0000","last_change":"2023-10-18T09:08:58.293751+0000","last_active":"2023-10-18T09:08:58.295909+0000","last_peered":"2023-10-18T09:08:58.295909+0000","last_clean":"2023-10-18T09:08:58.295909+0000","last_became_active":"2023-10-18T09:08:58.290425+0000","last_became_peered":"2023-10-18T09:08:58.290425+0000","last_unstale":"2023-10-18T09:08:58.295909+0000","last_undegraded":"2023-10-18T09:08:58.295909+0000","last_fullsized":"2023-10-18T09:08:58.295909+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:39:19.297430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.306112+0000","last_change":"2023-10-18T09:08:58.305992+0000","last_active":"2023-10-18T09:08:58.306112+0000","last_peered":"2023-10-18T09:08:58.306112+0000","last_clean":"2023-10-18T09:08:58.306112+0000","last_became_active":"2023-10-18T09:08:58.303900+0000","last_became_peered":"2023-10-18T09:08:58.303900+0000","last_unstale":"2023-10-18T09:08:58.306112+0000","last_undegraded":"2023-10-18T09:08:58.306112+0000","last_fullsized":"2023-10-18T09:08:58.306112+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:40:05.428277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.302165+0000","last_change":"2023-10-18T09:08:58.302084+0000","last_active":"2023-10-18T09:08:58.302165+0000","last_peered":"2023-10-18T09:08:58.302165+0000","last_clean":"2023-10-18T09:08:58.302165+0000","last_became_active":"2023-10-18T09:08:58.300396+0000","last_became_peered":"2023-10-18T09:08:58.300396+0000","last_unstale":"2023-10-18T09:08:58.302165+0000","last_undegraded":"2023-10-18T09:08:58.302165+0000","last_fullsized":"2023-10-18T09:08:58.302165+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:36.610169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.303190+0000","last_change":"2023-10-18T09:08:58.297855+0000","last_active":"2023-10-18T09:08:59.303190+0000","last_peered":"2023-10-18T09:08:59.303190+0000","last_clean":"2023-10-18T09:08:59.303190+0000","last_became_active":"2023-10-18T09:08:58.289254+0000","last_became_peered":"2023-10-18T09:08:58.289254+0000","last_unstale":"2023-10-18T09:08:59.303190+0000","last_undegraded":"2023-10-18T09:08:59.303190+0000","last_fullsized":"2023-10-18T09:08:59.303190+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:18:07.471451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.299352+0000","last_change":"2023-10-18T09:08:58.299211+0000","last_active":"2023-10-18T09:08:58.299352+0000","last_peered":"2023-10-18T09:08:58.299352+0000","last_clean":"2023-10-18T09:08:58.299352+0000","last_became_active":"2023-10-18T09:08:58.298975+0000","last_became_peered":"2023-10-18T09:08:58.298975+0000","last_unstale":"2023-10-18T09:08:58.299352+0000","last_undegraded":"2023-10-18T09:08:58.299352+0000","last_fullsized":"2023-10-18T09:08:58.299352+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:52:23.186942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.297706+0000","last_change":"2023-10-18T09:08:58.297506+0000","last_active":"2023-10-18T09:08:58.297706+0000","last_peered":"2023-10-18T09:08:58.297706+0000","last_clean":"2023-10-18T09:08:58.297706+0000","last_became_active":"2023-10-18T09:08:58.297289+0000","last_became_peered":"2023-10-18T09:08:58.297289+0000","last_unstale":"2023-10-18T09:08:58.297706+0000","last_undegraded":"2023-10-18T09:08:58.297706+0000","last_fullsized":"2023-10-18T09:08:58.297706+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:24.134042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.296066+0000","last_change":"2023-10-18T09:08:58.285630+0000","last_active":"2023-10-18T09:08:59.296066+0000","last_peered":"2023-10-18T09:08:59.296066+0000","last_clean":"2023-10-18T09:08:59.296066+0000","last_became_active":"2023-10-18T09:08:58.285384+0000","last_became_peered":"2023-10-18T09:08:58.285384+0000","last_unstale":"2023-10-18T09:08:59.296066+0000","last_undegraded":"2023-10-18T09:08:59.296066+0000","last_fullsized":"2023-10-18T09:08:59.296066+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:26:20.040703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.297626+0000","last_change":"2023-10-18T09:08:58.297479+0000","last_active":"2023-10-18T09:08:58.297626+0000","last_peered":"2023-10-18T09:08:58.297626+0000","last_clean":"2023-10-18T09:08:58.297626+0000","last_became_active":"2023-10-18T09:08:58.297242+0000","last_became_peered":"2023-10-18T09:08:58.297242+0000","last_unstale":"2023-10-18T09:08:58.297626+0000","last_undegraded":"2023-10-18T09:08:58.297626+0000","last_fullsized":"2023-10-18T09:08:58.297626+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:40:05.428277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.304510+0000","last_change":"2023-10-18T09:08:58.304105+0000","last_active":"2023-10-18T09:08:58.304510+0000","last_peered":"2023-10-18T09:08:58.304510+0000","last_clean":"2023-10-18T09:08:58.304510+0000","last_became_active":"2023-10-18T09:08:58.303838+0000","last_became_peered":"2023-10-18T09:08:58.303838+0000","last_unstale":"2023-10-18T09:08:58.304510+0000","last_undegraded":"2023-10-18T09:08:58.304510+0000","last_fullsized":"2023-10-18T09:08:58.304510+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:24.134042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.294990+0000","last_change":"2023-10-18T09:08:58.294608+0000","last_active":"2023-10-18T09:08:58.294990+0000","last_peered":"2023-10-18T09:08:58.294990+0000","last_clean":"2023-10-18T09:08:58.294990+0000","last_became_active":"2023-10-18T09:08:58.294339+0000","last_became_peered":"2023-10-18T09:08:58.294339+0000","last_unstale":"2023-10-18T09:08:58.294990+0000","last_undegraded":"2023-10-18T09:08:58.294990+0000","last_fullsized":"2023-10-18T09:08:58.294990+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:59:53.810779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.295871+0000","last_change":"2023-10-18T09:08:58.290937+0000","last_active":"2023-10-18T09:08:58.295871+0000","last_peered":"2023-10-18T09:08:58.295871+0000","last_clean":"2023-10-18T09:08:58.295871+0000","last_became_active":"2023-10-18T09:08:58.285959+0000","last_became_peered":"2023-10-18T09:08:58.285959+0000","last_unstale":"2023-10-18T09:08:58.295871+0000","last_undegraded":"2023-10-18T09:08:58.295871+0000","last_fullsized":"2023-10-18T09:08:58.295871+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:56:48.221743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.287815+0000","last_change":"2023-10-18T09:08:58.287564+0000","last_active":"2023-10-18T09:08:58.287815+0000","last_peered":"2023-10-18T09:08:58.287815+0000","last_clean":"2023-10-18T09:08:58.287815+0000","last_became_active":"2023-10-18T09:08:58.287100+0000","last_became_peered":"2023-10-18T09:08:58.287100+0000","last_unstale":"2023-10-18T09:08:58.287815+0000","last_undegraded":"2023-10-18T09:08:58.287815+0000","last_fullsized":"2023-10-18T09:08:58.287815+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:42:33.712181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.297308+0000","last_change":"2023-10-18T09:08:58.297237+0000","last_active":"2023-10-18T09:08:58.297308+0000","last_peered":"2023-10-18T09:08:58.297308+0000","last_clean":"2023-10-18T09:08:58.297308+0000","last_became_active":"2023-10-18T09:08:58.295901+0000","last_became_peered":"2023-10-18T09:08:58.295901+0000","last_unstale":"2023-10-18T09:08:58.297308+0000","last_undegraded":"2023-10-18T09:08:58.297308+0000","last_fullsized":"2023-10-18T09:08:58.297308+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:18:29.611460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.306246+0000","last_change":"2023-10-18T09:08:58.306092+0000","last_active":"2023-10-18T09:08:58.306246+0000","last_peered":"2023-10-18T09:08:58.306246+0000","last_clean":"2023-10-18T09:08:58.306246+0000","last_became_active":"2023-10-18T09:08:58.305854+0000","last_became_peered":"2023-10-18T09:08:58.305854+0000","last_unstale":"2023-10-18T09:08:58.306246+0000","last_undegraded":"2023-10-18T09:08:58.306246+0000","last_fullsized":"2023-10-18T09:08:58.306246+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:18.917111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.290419+0000","last_change":"2023-10-18T09:08:58.288905+0000","last_active":"2023-10-18T09:08:58.290419+0000","last_peered":"2023-10-18T09:08:58.290419+0000","last_clean":"2023-10-18T09:08:58.290419+0000","last_became_active":"2023-10-18T09:08:58.286172+0000","last_became_peered":"2023-10-18T09:08:58.286172+0000","last_unstale":"2023-10-18T09:08:58.290419+0000","last_undegraded":"2023-10-18T09:08:58.290419+0000","last_fullsized":"2023-10-18T09:08:58.290419+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:08:58.043466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.302142+0000","last_change":"2023-10-18T09:08:58.301002+0000","last_active":"2023-10-18T09:08:58.302142+0000","last_peered":"2023-10-18T09:08:58.302142+0000","last_clean":"2023-10-18T09:08:58.302142+0000","last_became_active":"2023-10-18T09:08:58.300486+0000","last_became_peered":"2023-10-18T09:08:58.300486+0000","last_unstale":"2023-10-18T09:08:58.302142+0000","last_undegraded":"2023-10-18T09:08:58.302142+0000","last_fullsized":"2023-10-18T09:08:58.302142+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:05:26.437571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.299227+0000","last_change":"2023-10-18T09:08:58.299004+0000","last_active":"2023-10-18T09:08:58.299227+0000","last_peered":"2023-10-18T09:08:58.299227+0000","last_clean":"2023-10-18T09:08:58.299227+0000","last_became_active":"2023-10-18T09:08:58.298766+0000","last_became_peered":"2023-10-18T09:08:58.298766+0000","last_unstale":"2023-10-18T09:08:58.299227+0000","last_undegraded":"2023-10-18T09:08:58.299227+0000","last_fullsized":"2023-10-18T09:08:58.299227+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:08:08.196353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.297774+0000","last_change":"2023-10-18T09:08:58.296363+0000","last_active":"2023-10-18T09:08:58.297774+0000","last_peered":"2023-10-18T09:08:58.297774+0000","last_clean":"2023-10-18T09:08:58.297774+0000","last_became_active":"2023-10-18T09:08:58.294410+0000","last_became_peered":"2023-10-18T09:08:58.294410+0000","last_unstale":"2023-10-18T09:08:58.297774+0000","last_undegraded":"2023-10-18T09:08:58.297774+0000","last_fullsized":"2023-10-18T09:08:58.297774+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:00:04.213601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.304314+0000","last_change":"2023-10-18T09:08:58.303648+0000","last_active":"2023-10-18T09:08:58.304314+0000","last_peered":"2023-10-18T09:08:58.304314+0000","last_clean":"2023-10-18T09:08:58.304314+0000","last_became_active":"2023-10-18T09:08:58.303344+0000","last_became_peered":"2023-10-18T09:08:58.303344+0000","last_unstale":"2023-10-18T09:08:58.304314+0000","last_undegraded":"2023-10-18T09:08:58.304314+0000","last_fullsized":"2023-10-18T09:08:58.304314+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:22.048918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.303475+0000","last_change":"2023-10-18T09:08:58.299835+0000","last_active":"2023-10-18T09:08:58.303475+0000","last_peered":"2023-10-18T09:08:58.303475+0000","last_clean":"2023-10-18T09:08:58.303475+0000","last_became_active":"2023-10-18T09:08:58.296142+0000","last_became_peered":"2023-10-18T09:08:58.296142+0000","last_unstale":"2023-10-18T09:08:58.303475+0000","last_undegraded":"2023-10-18T09:08:58.303475+0000","last_fullsized":"2023-10-18T09:08:58.303475+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:47.664727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295225+0000","last_change":"2023-10-18T09:08:58.287487+0000","last_active":"2023-10-18T09:08:59.295225+0000","last_peered":"2023-10-18T09:08:59.295225+0000","last_clean":"2023-10-18T09:08:59.295225+0000","last_became_active":"2023-10-18T09:08:58.287089+0000","last_became_peered":"2023-10-18T09:08:58.287089+0000","last_unstale":"2023-10-18T09:08:59.295225+0000","last_undegraded":"2023-10-18T09:08:59.295225+0000","last_fullsized":"2023-10-18T09:08:59.295225+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:54:41.020790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.287873+0000","last_change":"2023-10-18T09:08:58.286781+0000","last_active":"2023-10-18T09:08:58.287873+0000","last_peered":"2023-10-18T09:08:58.287873+0000","last_clean":"2023-10-18T09:08:58.287873+0000","last_became_active":"2023-10-18T09:08:58.286541+0000","last_became_peered":"2023-10-18T09:08:58.286541+0000","last_unstale":"2023-10-18T09:08:58.287873+0000","last_undegraded":"2023-10-18T09:08:58.287873+0000","last_fullsized":"2023-10-18T09:08:58.287873+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:07:11.888595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"195'4","reported_seq":16,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.302496+0000","last_change":"2023-10-18T09:08:57.271399+0000","last_active":"2023-10-18T09:08:59.302496+0000","last_peered":"2023-10-18T09:08:59.302496+0000","last_clean":"2023-10-18T09:08:59.302496+0000","last_became_active":"2023-10-18T09:08:57.271095+0000","last_became_peered":"2023-10-18T09:08:57.271095+0000","last_unstale":"2023-10-18T09:08:59.302496+0000","last_undegraded":"2023-10-18T09:08:59.302496+0000","last_fullsized":"2023-10-18T09:08:59.302496+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:23:21.063445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.303427+0000","last_change":"2023-10-18T09:08:58.298966+0000","last_active":"2023-10-18T09:08:58.303427+0000","last_peered":"2023-10-18T09:08:58.303427+0000","last_clean":"2023-10-18T09:08:58.303427+0000","last_became_active":"2023-10-18T09:08:58.298654+0000","last_became_peered":"2023-10-18T09:08:58.298654+0000","last_unstale":"2023-10-18T09:08:58.303427+0000","last_undegraded":"2023-10-18T09:08:58.303427+0000","last_fullsized":"2023-10-18T09:08:58.303427+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:13:19.770858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"195'1","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.305900+0000","last_change":"2023-10-18T09:08:57.275486+0000","last_active":"2023-10-18T09:08:58.305900+0000","last_peered":"2023-10-18T09:08:58.305900+0000","last_clean":"2023-10-18T09:08:58.305900+0000","last_became_active":"2023-10-18T09:08:57.275259+0000","last_became_peered":"2023-10-18T09:08:57.275259+0000","last_unstale":"2023-10-18T09:08:58.305900+0000","last_undegraded":"2023-10-18T09:08:58.305900+0000","last_fullsized":"2023-10-18T09:08:58.305900+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:53.638412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.290010+0000","last_change":"2023-10-18T09:08:58.289817+0000","last_active":"2023-10-18T09:08:58.290010+0000","last_peered":"2023-10-18T09:08:58.290010+0000","last_clean":"2023-10-18T09:08:58.290010+0000","last_became_active":"2023-10-18T09:08:58.289385+0000","last_became_peered":"2023-10-18T09:08:58.289385+0000","last_unstale":"2023-10-18T09:08:58.290010+0000","last_undegraded":"2023-10-18T09:08:58.290010+0000","last_fullsized":"2023-10-18T09:08:58.290010+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:00:04.213601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"195'2","reported_seq":13,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.299074+0000","last_change":"2023-10-18T09:08:57.274875+0000","last_active":"2023-10-18T09:08:58.299074+0000","last_peered":"2023-10-18T09:08:58.299074+0000","last_clean":"2023-10-18T09:08:58.299074+0000","last_became_active":"2023-10-18T09:08:57.274421+0000","last_became_peered":"2023-10-18T09:08:57.274421+0000","last_unstale":"2023-10-18T09:08:58.299074+0000","last_undegraded":"2023-10-18T09:08:58.299074+0000","last_fullsized":"2023-10-18T09:08:58.299074+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:43:12.302636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.297795+0000","last_change":"2023-10-18T09:08:58.297232+0000","last_active":"2023-10-18T09:08:58.297795+0000","last_peered":"2023-10-18T09:08:58.297795+0000","last_clean":"2023-10-18T09:08:58.297795+0000","last_became_active":"2023-10-18T09:08:58.296971+0000","last_became_peered":"2023-10-18T09:08:58.296971+0000","last_unstale":"2023-10-18T09:08:58.297795+0000","last_undegraded":"2023-10-18T09:08:58.297795+0000","last_fullsized":"2023-10-18T09:08:58.297795+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:43:50.186716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"195'2","reported_seq":13,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.299278+0000","last_change":"2023-10-18T09:08:57.263282+0000","last_active":"2023-10-18T09:08:58.299278+0000","last_peered":"2023-10-18T09:08:58.299278+0000","last_clean":"2023-10-18T09:08:58.299278+0000","last_became_active":"2023-10-18T09:08:57.262973+0000","last_became_peered":"2023-10-18T09:08:57.262973+0000","last_unstale":"2023-10-18T09:08:58.299278+0000","last_undegraded":"2023-10-18T09:08:58.299278+0000","last_fullsized":"2023-10-18T09:08:58.299278+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:55:15.422049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.283059+0000","last_change":"2023-10-18T09:08:58.282982+0000","last_active":"2023-10-18T09:08:58.283059+0000","last_peered":"2023-10-18T09:08:58.283059+0000","last_clean":"2023-10-18T09:08:58.283059+0000","last_became_active":"2023-10-18T09:08:58.282571+0000","last_became_peered":"2023-10-18T09:08:58.282571+0000","last_unstale":"2023-10-18T09:08:58.283059+0000","last_undegraded":"2023-10-18T09:08:58.283059+0000","last_fullsized":"2023-10-18T09:08:58.283059+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:13:13.077242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"195'1","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.305443+0000","last_change":"2023-10-18T09:08:57.269774+0000","last_active":"2023-10-18T09:08:58.305443+0000","last_peered":"2023-10-18T09:08:58.305443+0000","last_clean":"2023-10-18T09:08:58.305443+0000","last_became_active":"2023-10-18T09:08:57.269450+0000","last_became_peered":"2023-10-18T09:08:57.269450+0000","last_unstale":"2023-10-18T09:08:58.305443+0000","last_undegraded":"2023-10-18T09:08:58.305443+0000","last_fullsized":"2023-10-18T09:08:58.305443+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:29:33.964918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.292885+0000","last_change":"2023-10-18T09:08:58.292782+0000","last_active":"2023-10-18T09:08:58.292885+0000","last_peered":"2023-10-18T09:08:58.292885+0000","last_clean":"2023-10-18T09:08:58.292885+0000","last_became_active":"2023-10-18T09:08:58.292044+0000","last_became_peered":"2023-10-18T09:08:58.292044+0000","last_unstale":"2023-10-18T09:08:58.292885+0000","last_undegraded":"2023-10-18T09:08:58.292885+0000","last_fullsized":"2023-10-18T09:08:58.292885+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:01:45.963996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"195'1","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.295805+0000","last_change":"2023-10-18T09:08:57.260538+0000","last_active":"2023-10-18T09:08:58.295805+0000","last_peered":"2023-10-18T09:08:58.295805+0000","last_clean":"2023-10-18T09:08:58.295805+0000","last_became_active":"2023-10-18T09:08:57.260278+0000","last_became_peered":"2023-10-18T09:08:57.260278+0000","last_unstale":"2023-10-18T09:08:58.295805+0000","last_undegraded":"2023-10-18T09:08:58.295805+0000","last_fullsized":"2023-10-18T09:08:58.295805+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:04:29.565641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.294654+0000","last_change":"2023-10-18T09:08:58.294014+0000","last_active":"2023-10-18T09:08:58.294654+0000","last_peered":"2023-10-18T09:08:58.294654+0000","last_clean":"2023-10-18T09:08:58.294654+0000","last_became_active":"2023-10-18T09:08:58.293595+0000","last_became_peered":"2023-10-18T09:08:58.293595+0000","last_unstale":"2023-10-18T09:08:58.294654+0000","last_undegraded":"2023-10-18T09:08:58.294654+0000","last_fullsized":"2023-10-18T09:08:58.294654+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:12:54.154240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.273287+0000","last_change":"2023-10-18T09:08:57.276594+0000","last_active":"2023-10-18T09:08:58.273287+0000","last_peered":"2023-10-18T09:08:58.273287+0000","last_clean":"2023-10-18T09:08:58.273287+0000","last_became_active":"2023-10-18T09:08:57.276049+0000","last_became_peered":"2023-10-18T09:08:57.276049+0000","last_unstale":"2023-10-18T09:08:58.273287+0000","last_undegraded":"2023-10-18T09:08:58.273287+0000","last_fullsized":"2023-10-18T09:08:58.273287+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:26:57.248352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.285151+0000","last_change":"2023-10-18T09:08:58.283900+0000","last_active":"2023-10-18T09:08:58.285151+0000","last_peered":"2023-10-18T09:08:58.285151+0000","last_clean":"2023-10-18T09:08:58.285151+0000","last_became_active":"2023-10-18T09:08:58.283623+0000","last_became_peered":"2023-10-18T09:08:58.283623+0000","last_unstale":"2023-10-18T09:08:58.285151+0000","last_undegraded":"2023-10-18T09:08:58.285151+0000","last_fullsized":"2023-10-18T09:08:58.285151+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:36.610169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.280483+0000","last_change":"2023-10-18T09:08:57.259163+0000","last_active":"2023-10-18T09:08:58.280483+0000","last_peered":"2023-10-18T09:08:58.280483+0000","last_clean":"2023-10-18T09:08:58.280483+0000","last_became_active":"2023-10-18T09:08:57.258929+0000","last_became_peered":"2023-10-18T09:08:57.258929+0000","last_unstale":"2023-10-18T09:08:58.280483+0000","last_undegraded":"2023-10-18T09:08:58.280483+0000","last_fullsized":"2023-10-18T09:08:58.280483+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:00:30.305343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295869+0000","last_change":"2023-10-18T09:08:58.289895+0000","last_active":"2023-10-18T09:08:59.295869+0000","last_peered":"2023-10-18T09:08:59.295869+0000","last_clean":"2023-10-18T09:08:59.295869+0000","last_became_active":"2023-10-18T09:08:58.289660+0000","last_became_peered":"2023-10-18T09:08:58.289660+0000","last_unstale":"2023-10-18T09:08:59.295869+0000","last_undegraded":"2023-10-18T09:08:59.295869+0000","last_fullsized":"2023-10-18T09:08:59.295869+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:54.648974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.285379+0000","last_change":"2023-10-18T09:08:57.272340+0000","last_active":"2023-10-18T09:08:58.285379+0000","last_peered":"2023-10-18T09:08:58.285379+0000","last_clean":"2023-10-18T09:08:58.285379+0000","last_became_active":"2023-10-18T09:08:57.272135+0000","last_became_peered":"2023-10-18T09:08:57.272135+0000","last_unstale":"2023-10-18T09:08:58.285379+0000","last_undegraded":"2023-10-18T09:08:58.285379+0000","last_fullsized":"2023-10-18T09:08:58.285379+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:58:59.554028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295302+0000","last_change":"2023-10-18T09:08:58.290706+0000","last_active":"2023-10-18T09:08:59.295302+0000","last_peered":"2023-10-18T09:08:59.295302+0000","last_clean":"2023-10-18T09:08:59.295302+0000","last_became_active":"2023-10-18T09:08:58.290345+0000","last_became_peered":"2023-10-18T09:08:58.290345+0000","last_unstale":"2023-10-18T09:08:59.295302+0000","last_undegraded":"2023-10-18T09:08:59.295302+0000","last_fullsized":"2023-10-18T09:08:59.295302+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:23:39.065783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"195'3","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.290237+0000","last_change":"2023-10-18T09:08:57.272635+0000","last_active":"2023-10-18T09:08:58.290237+0000","last_peered":"2023-10-18T09:08:58.290237+0000","last_clean":"2023-10-18T09:08:58.290237+0000","last_became_active":"2023-10-18T09:08:57.272462+0000","last_became_peered":"2023-10-18T09:08:57.272462+0000","last_unstale":"2023-10-18T09:08:58.290237+0000","last_undegraded":"2023-10-18T09:08:58.290237+0000","last_fullsized":"2023-10-18T09:08:58.290237+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:58.866489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.306382+0000","last_change":"2023-10-18T09:08:58.305865+0000","last_active":"2023-10-18T09:08:58.306382+0000","last_peered":"2023-10-18T09:08:58.306382+0000","last_clean":"2023-10-18T09:08:58.306382+0000","last_became_active":"2023-10-18T09:08:58.304785+0000","last_became_peered":"2023-10-18T09:08:58.304785+0000","last_unstale":"2023-10-18T09:08:58.306382+0000","last_undegraded":"2023-10-18T09:08:58.306382+0000","last_fullsized":"2023-10-18T09:08:58.306382+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:06.196209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"195'6","reported_seq":17,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.307313+0000","last_change":"2023-10-18T09:08:57.272859+0000","last_active":"2023-10-18T09:08:58.307313+0000","last_peered":"2023-10-18T09:08:58.307313+0000","last_clean":"2023-10-18T09:08:58.307313+0000","last_became_active":"2023-10-18T09:08:57.272592+0000","last_became_peered":"2023-10-18T09:08:57.272592+0000","last_unstale":"2023-10-18T09:08:58.307313+0000","last_undegraded":"2023-10-18T09:08:58.307313+0000","last_fullsized":"2023-10-18T09:08:58.307313+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:26:19.030141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295966+0000","last_change":"2023-10-18T09:08:58.285606+0000","last_active":"2023-10-18T09:08:59.295966+0000","last_peered":"2023-10-18T09:08:59.295966+0000","last_clean":"2023-10-18T09:08:59.295966+0000","last_became_active":"2023-10-18T09:08:58.285357+0000","last_became_peered":"2023-10-18T09:08:58.285357+0000","last_unstale":"2023-10-18T09:08:59.295966+0000","last_undegraded":"2023-10-18T09:08:59.295966+0000","last_fullsized":"2023-10-18T09:08:59.295966+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:19:18.416271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"195'2","reported_seq":13,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.297215+0000","last_change":"2023-10-18T09:08:57.256549+0000","last_active":"2023-10-18T09:08:58.297215+0000","last_peered":"2023-10-18T09:08:58.297215+0000","last_clean":"2023-10-18T09:08:58.297215+0000","last_became_active":"2023-10-18T09:08:57.255882+0000","last_became_peered":"2023-10-18T09:08:57.255882+0000","last_unstale":"2023-10-18T09:08:58.297215+0000","last_undegraded":"2023-10-18T09:08:58.297215+0000","last_fullsized":"2023-10-18T09:08:58.297215+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:15:16.826765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.292269+0000","last_change":"2023-10-18T09:08:58.291201+0000","last_active":"2023-10-18T09:08:58.292269+0000","last_peered":"2023-10-18T09:08:58.292269+0000","last_clean":"2023-10-18T09:08:58.292269+0000","last_became_active":"2023-10-18T09:08:58.290943+0000","last_became_peered":"2023-10-18T09:08:58.290943+0000","last_unstale":"2023-10-18T09:08:58.292269+0000","last_undegraded":"2023-10-18T09:08:58.292269+0000","last_fullsized":"2023-10-18T09:08:58.292269+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:28:56.074539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.296023+0000","last_change":"2023-10-18T09:08:57.260749+0000","last_active":"2023-10-18T09:08:59.296023+0000","last_peered":"2023-10-18T09:08:59.296023+0000","last_clean":"2023-10-18T09:08:59.296023+0000","last_became_active":"2023-10-18T09:08:57.260427+0000","last_became_peered":"2023-10-18T09:08:57.260427+0000","last_unstale":"2023-10-18T09:08:59.296023+0000","last_undegraded":"2023-10-18T09:08:59.296023+0000","last_fullsized":"2023-10-18T09:08:59.296023+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:48:58.193868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.305275+0000","last_change":"2023-10-18T09:08:58.305200+0000","last_active":"2023-10-18T09:08:58.305275+0000","last_peered":"2023-10-18T09:08:58.305275+0000","last_clean":"2023-10-18T09:08:58.305275+0000","last_became_active":"2023-10-18T09:08:58.304940+0000","last_became_peered":"2023-10-18T09:08:58.304940+0000","last_unstale":"2023-10-18T09:08:58.305275+0000","last_undegraded":"2023-10-18T09:08:58.305275+0000","last_fullsized":"2023-10-18T09:08:58.305275+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:00:35.614777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.285991+0000","last_change":"2023-10-18T09:08:57.274472+0000","last_active":"2023-10-18T09:08:58.285991+0000","last_peered":"2023-10-18T09:08:58.285991+0000","last_clean":"2023-10-18T09:08:58.285991+0000","last_became_active":"2023-10-18T09:08:57.274159+0000","last_became_peered":"2023-10-18T09:08:57.274159+0000","last_unstale":"2023-10-18T09:08:58.285991+0000","last_undegraded":"2023-10-18T09:08:58.285991+0000","last_fullsized":"2023-10-18T09:08:58.285991+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:39:10.454786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.297825+0000","last_change":"2023-10-18T09:08:58.295437+0000","last_active":"2023-10-18T09:08:58.297825+0000","last_peered":"2023-10-18T09:08:58.297825+0000","last_clean":"2023-10-18T09:08:58.297825+0000","last_became_active":"2023-10-18T09:08:58.295108+0000","last_became_peered":"2023-10-18T09:08:58.295108+0000","last_unstale":"2023-10-18T09:08:58.297825+0000","last_undegraded":"2023-10-18T09:08:58.297825+0000","last_fullsized":"2023-10-18T09:08:58.297825+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:59:53.810779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"195'8","reported_seq":19,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.309253+0000","last_change":"2023-10-18T09:08:57.272858+0000","last_active":"2023-10-18T09:08:58.309253+0000","last_peered":"2023-10-18T09:08:58.309253+0000","last_clean":"2023-10-18T09:08:58.309253+0000","last_became_active":"2023-10-18T09:08:57.272590+0000","last_became_peered":"2023-10-18T09:08:57.272590+0000","last_unstale":"2023-10-18T09:08:58.309253+0000","last_undegraded":"2023-10-18T09:08:58.309253+0000","last_fullsized":"2023-10-18T09:08:58.309253+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:50:09.074412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.292853+0000","last_change":"2023-10-18T09:08:58.289757+0000","last_active":"2023-10-18T09:08:58.292853+0000","last_peered":"2023-10-18T09:08:58.292853+0000","last_clean":"2023-10-18T09:08:58.292853+0000","last_became_active":"2023-10-18T09:08:58.287329+0000","last_became_peered":"2023-10-18T09:08:58.287329+0000","last_unstale":"2023-10-18T09:08:58.292853+0000","last_undegraded":"2023-10-18T09:08:58.292853+0000","last_fullsized":"2023-10-18T09:08:58.292853+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:20.114971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"195'1","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.301319+0000","last_change":"2023-10-18T09:08:57.275895+0000","last_active":"2023-10-18T09:08:58.301319+0000","last_peered":"2023-10-18T09:08:58.301319+0000","last_clean":"2023-10-18T09:08:58.301319+0000","last_became_active":"2023-10-18T09:08:57.275641+0000","last_became_peered":"2023-10-18T09:08:57.275641+0000","last_unstale":"2023-10-18T09:08:58.301319+0000","last_undegraded":"2023-10-18T09:08:58.301319+0000","last_fullsized":"2023-10-18T09:08:58.301319+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:16.784281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295409+0000","last_change":"2023-10-18T09:08:58.291480+0000","last_active":"2023-10-18T09:08:59.295409+0000","last_peered":"2023-10-18T09:08:59.295409+0000","last_clean":"2023-10-18T09:08:59.295409+0000","last_became_active":"2023-10-18T09:08:58.291249+0000","last_became_peered":"2023-10-18T09:08:58.291249+0000","last_unstale":"2023-10-18T09:08:59.295409+0000","last_undegraded":"2023-10-18T09:08:59.295409+0000","last_fullsized":"2023-10-18T09:08:59.295409+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:55:06.196209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"195'2","reported_seq":13,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.296729+0000","last_change":"2023-10-18T09:08:57.267486+0000","last_active":"2023-10-18T09:08:58.296729+0000","last_peered":"2023-10-18T09:08:58.296729+0000","last_clean":"2023-10-18T09:08:58.296729+0000","last_became_active":"2023-10-18T09:08:57.266986+0000","last_became_peered":"2023-10-18T09:08:57.266986+0000","last_unstale":"2023-10-18T09:08:58.296729+0000","last_undegraded":"2023-10-18T09:08:58.296729+0000","last_fullsized":"2023-10-18T09:08:58.296729+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:58:59.554028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.292870+0000","last_change":"2023-10-18T09:08:58.292648+0000","last_active":"2023-10-18T09:08:58.292870+0000","last_peered":"2023-10-18T09:08:58.292870+0000","last_clean":"2023-10-18T09:08:58.292870+0000","last_became_active":"2023-10-18T09:08:58.291687+0000","last_became_peered":"2023-10-18T09:08:58.291687+0000","last_unstale":"2023-10-18T09:08:58.292870+0000","last_undegraded":"2023-10-18T09:08:58.292870+0000","last_fullsized":"2023-10-18T09:08:58.292870+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:58:49.948400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"195'1","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.303295+0000","last_change":"2023-10-18T09:08:57.270447+0000","last_active":"2023-10-18T09:08:59.303295+0000","last_peered":"2023-10-18T09:08:59.303295+0000","last_clean":"2023-10-18T09:08:59.303295+0000","last_became_active":"2023-10-18T09:08:57.269810+0000","last_became_peered":"2023-10-18T09:08:57.269810+0000","last_unstale":"2023-10-18T09:08:59.303295+0000","last_undegraded":"2023-10-18T09:08:59.303295+0000","last_fullsized":"2023-10-18T09:08:59.303295+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:27:36.905203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.295099+0000","last_change":"2023-10-18T09:08:58.295020+0000","last_active":"2023-10-18T09:08:58.295099+0000","last_peered":"2023-10-18T09:08:58.295099+0000","last_clean":"2023-10-18T09:08:58.295099+0000","last_became_active":"2023-10-18T09:08:58.294792+0000","last_became_peered":"2023-10-18T09:08:58.294792+0000","last_unstale":"2023-10-18T09:08:58.295099+0000","last_undegraded":"2023-10-18T09:08:58.295099+0000","last_fullsized":"2023-10-18T09:08:58.295099+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:39:29.385855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.302470+0000","last_change":"2023-10-18T09:08:57.264924+0000","last_active":"2023-10-18T09:08:59.302470+0000","last_peered":"2023-10-18T09:08:59.302470+0000","last_clean":"2023-10-18T09:08:59.302470+0000","last_became_active":"2023-10-18T09:08:57.264637+0000","last_became_peered":"2023-10-18T09:08:57.264637+0000","last_unstale":"2023-10-18T09:08:59.302470+0000","last_undegraded":"2023-10-18T09:08:59.302470+0000","last_fullsized":"2023-10-18T09:08:59.302470+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:08:29.125397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.293185+0000","last_change":"2023-10-18T09:08:58.292779+0000","last_active":"2023-10-18T09:08:58.293185+0000","last_peered":"2023-10-18T09:08:58.293185+0000","last_clean":"2023-10-18T09:08:58.293185+0000","last_became_active":"2023-10-18T09:08:58.292472+0000","last_became_peered":"2023-10-18T09:08:58.292472+0000","last_unstale":"2023-10-18T09:08:58.293185+0000","last_undegraded":"2023-10-18T09:08:58.293185+0000","last_fullsized":"2023-10-18T09:08:58.293185+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:00:51.596748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"195'2","reported_seq":13,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.296500+0000","last_change":"2023-10-18T09:08:57.274036+0000","last_active":"2023-10-18T09:08:58.296500+0000","last_peered":"2023-10-18T09:08:58.296500+0000","last_clean":"2023-10-18T09:08:58.296500+0000","last_became_active":"2023-10-18T09:08:57.273698+0000","last_became_peered":"2023-10-18T09:08:57.273698+0000","last_unstale":"2023-10-18T09:08:58.296500+0000","last_undegraded":"2023-10-18T09:08:58.296500+0000","last_fullsized":"2023-10-18T09:08:58.296500+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:43:01.706671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.290487+0000","last_change":"2023-10-18T09:08:58.290424+0000","last_active":"2023-10-18T09:08:58.290487+0000","last_peered":"2023-10-18T09:08:58.290487+0000","last_clean":"2023-10-18T09:08:58.290487+0000","last_became_active":"2023-10-18T09:08:58.290191+0000","last_became_peered":"2023-10-18T09:08:58.290191+0000","last_unstale":"2023-10-18T09:08:58.290487+0000","last_undegraded":"2023-10-18T09:08:58.290487+0000","last_fullsized":"2023-10-18T09:08:58.290487+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:45:36.037890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.302902+0000","last_change":"2023-10-18T09:08:57.268081+0000","last_active":"2023-10-18T09:08:59.302902+0000","last_peered":"2023-10-18T09:08:59.302902+0000","last_clean":"2023-10-18T09:08:59.302902+0000","last_became_active":"2023-10-18T09:08:57.266320+0000","last_became_peered":"2023-10-18T09:08:57.266320+0000","last_unstale":"2023-10-18T09:08:59.302902+0000","last_undegraded":"2023-10-18T09:08:59.302902+0000","last_fullsized":"2023-10-18T09:08:59.302902+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:29:35.653154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.302741+0000","last_change":"2023-10-18T09:08:58.299498+0000","last_active":"2023-10-18T09:08:59.302741+0000","last_peered":"2023-10-18T09:08:59.302741+0000","last_clean":"2023-10-18T09:08:59.302741+0000","last_became_active":"2023-10-18T09:08:58.295206+0000","last_became_peered":"2023-10-18T09:08:58.295206+0000","last_unstale":"2023-10-18T09:08:59.302741+0000","last_undegraded":"2023-10-18T09:08:59.302741+0000","last_fullsized":"2023-10-18T09:08:59.302741+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:49:35.502752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.284078+0000","last_change":"2023-10-18T09:08:57.269644+0000","last_active":"2023-10-18T09:08:58.284078+0000","last_peered":"2023-10-18T09:08:58.284078+0000","last_clean":"2023-10-18T09:08:58.284078+0000","last_became_active":"2023-10-18T09:08:57.269200+0000","last_became_peered":"2023-10-18T09:08:57.269200+0000","last_unstale":"2023-10-18T09:08:58.284078+0000","last_undegraded":"2023-10-18T09:08:58.284078+0000","last_fullsized":"2023-10-18T09:08:58.284078+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:07:22.471253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.296941+0000","last_change":"2023-10-18T09:08:58.296811+0000","last_active":"2023-10-18T09:08:58.296941+0000","last_peered":"2023-10-18T09:08:58.296941+0000","last_clean":"2023-10-18T09:08:58.296941+0000","last_became_active":"2023-10-18T09:08:58.296547+0000","last_became_peered":"2023-10-18T09:08:58.296547+0000","last_unstale":"2023-10-18T09:08:58.296941+0000","last_undegraded":"2023-10-18T09:08:58.296941+0000","last_fullsized":"2023-10-18T09:08:58.296941+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:30:18.917111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.303408+0000","last_change":"2023-10-18T09:08:57.270917+0000","last_active":"2023-10-18T09:08:59.303408+0000","last_peered":"2023-10-18T09:08:59.303408+0000","last_clean":"2023-10-18T09:08:59.303408+0000","last_became_active":"2023-10-18T09:08:57.270200+0000","last_became_peered":"2023-10-18T09:08:57.270200+0000","last_unstale":"2023-10-18T09:08:59.303408+0000","last_undegraded":"2023-10-18T09:08:59.303408+0000","last_fullsized":"2023-10-18T09:08:59.303408+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:12:16.804218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.296212+0000","last_change":"2023-10-18T09:08:58.290176+0000","last_active":"2023-10-18T09:08:59.296212+0000","last_peered":"2023-10-18T09:08:59.296212+0000","last_clean":"2023-10-18T09:08:59.296212+0000","last_became_active":"2023-10-18T09:08:58.289924+0000","last_became_peered":"2023-10-18T09:08:58.289924+0000","last_unstale":"2023-10-18T09:08:59.296212+0000","last_undegraded":"2023-10-18T09:08:59.296212+0000","last_fullsized":"2023-10-18T09:08:59.296212+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:19:48.023027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"195'2","reported_seq":14,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295814+0000","last_change":"2023-10-18T09:08:57.273975+0000","last_active":"2023-10-18T09:08:59.295814+0000","last_peered":"2023-10-18T09:08:59.295814+0000","last_clean":"2023-10-18T09:08:59.295814+0000","last_became_active":"2023-10-18T09:08:57.273716+0000","last_became_peered":"2023-10-18T09:08:57.273716+0000","last_unstale":"2023-10-18T09:08:59.295814+0000","last_undegraded":"2023-10-18T09:08:59.295814+0000","last_fullsized":"2023-10-18T09:08:59.295814+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:19:38.085128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.295857+0000","last_change":"2023-10-18T09:08:58.295726+0000","last_active":"2023-10-18T09:08:58.295857+0000","last_peered":"2023-10-18T09:08:58.295857+0000","last_clean":"2023-10-18T09:08:58.295857+0000","last_became_active":"2023-10-18T09:08:58.295228+0000","last_became_peered":"2023-10-18T09:08:58.295228+0000","last_unstale":"2023-10-18T09:08:58.295857+0000","last_undegraded":"2023-10-18T09:08:58.295857+0000","last_fullsized":"2023-10-18T09:08:58.295857+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:49.455678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.286450+0000","last_change":"2023-10-18T09:08:57.273659+0000","last_active":"2023-10-18T09:08:58.286450+0000","last_peered":"2023-10-18T09:08:58.286450+0000","last_clean":"2023-10-18T09:08:58.286450+0000","last_became_active":"2023-10-18T09:08:57.271740+0000","last_became_peered":"2023-10-18T09:08:57.271740+0000","last_unstale":"2023-10-18T09:08:58.286450+0000","last_undegraded":"2023-10-18T09:08:58.286450+0000","last_fullsized":"2023-10-18T09:08:58.286450+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:09:37.531014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.289993+0000","last_change":"2023-10-18T09:08:58.289782+0000","last_active":"2023-10-18T09:08:58.289993+0000","last_peered":"2023-10-18T09:08:58.289993+0000","last_clean":"2023-10-18T09:08:58.289993+0000","last_became_active":"2023-10-18T09:08:58.289433+0000","last_became_peered":"2023-10-18T09:08:58.289433+0000","last_unstale":"2023-10-18T09:08:58.289993+0000","last_undegraded":"2023-10-18T09:08:58.289993+0000","last_fullsized":"2023-10-18T09:08:58.289993+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:43:50.186716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.277245+0000","last_change":"2023-10-18T09:08:57.274980+0000","last_active":"2023-10-18T09:08:58.277245+0000","last_peered":"2023-10-18T09:08:58.277245+0000","last_clean":"2023-10-18T09:08:58.277245+0000","last_became_active":"2023-10-18T09:08:57.274419+0000","last_became_peered":"2023-10-18T09:08:57.274419+0000","last_unstale":"2023-10-18T09:08:58.277245+0000","last_undegraded":"2023-10-18T09:08:58.277245+0000","last_fullsized":"2023-10-18T09:08:58.277245+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:16:52.127204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295502+0000","last_change":"2023-10-18T09:08:58.280781+0000","last_active":"2023-10-18T09:08:59.295502+0000","last_peered":"2023-10-18T09:08:59.295502+0000","last_clean":"2023-10-18T09:08:59.295502+0000","last_became_active":"2023-10-18T09:08:58.280496+0000","last_became_peered":"2023-10-18T09:08:58.280496+0000","last_unstale":"2023-10-18T09:08:59.295502+0000","last_undegraded":"2023-10-18T09:08:59.295502+0000","last_fullsized":"2023-10-18T09:08:59.295502+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:13:31.701039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295399+0000","last_change":"2023-10-18T09:08:57.269806+0000","last_active":"2023-10-18T09:08:59.295399+0000","last_peered":"2023-10-18T09:08:59.295399+0000","last_clean":"2023-10-18T09:08:59.295399+0000","last_became_active":"2023-10-18T09:08:57.269404+0000","last_became_peered":"2023-10-18T09:08:57.269404+0000","last_unstale":"2023-10-18T09:08:59.295399+0000","last_undegraded":"2023-10-18T09:08:59.295399+0000","last_fullsized":"2023-10-18T09:08:59.295399+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:31:03.288895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295600+0000","last_change":"2023-10-18T09:08:58.287324+0000","last_active":"2023-10-18T09:08:59.295600+0000","last_peered":"2023-10-18T09:08:59.295600+0000","last_clean":"2023-10-18T09:08:59.295600+0000","last_became_active":"2023-10-18T09:08:58.286756+0000","last_became_peered":"2023-10-18T09:08:58.286756+0000","last_unstale":"2023-10-18T09:08:59.295600+0000","last_undegraded":"2023-10-18T09:08:59.295600+0000","last_fullsized":"2023-10-18T09:08:59.295600+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:16:47.664727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.290585+0000","last_change":"2023-10-18T09:08:57.269211+0000","last_active":"2023-10-18T09:08:58.290585+0000","last_peered":"2023-10-18T09:08:58.290585+0000","last_clean":"2023-10-18T09:08:58.290585+0000","last_became_active":"2023-10-18T09:08:57.268692+0000","last_became_peered":"2023-10-18T09:08:57.268692+0000","last_unstale":"2023-10-18T09:08:58.290585+0000","last_undegraded":"2023-10-18T09:08:58.290585+0000","last_fullsized":"2023-10-18T09:08:58.290585+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:20.053571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.303452+0000","last_change":"2023-10-18T09:08:58.296575+0000","last_active":"2023-10-18T09:08:58.303452+0000","last_peered":"2023-10-18T09:08:58.303452+0000","last_clean":"2023-10-18T09:08:58.303452+0000","last_became_active":"2023-10-18T09:08:58.295763+0000","last_became_peered":"2023-10-18T09:08:58.295763+0000","last_unstale":"2023-10-18T09:08:58.303452+0000","last_undegraded":"2023-10-18T09:08:58.303452+0000","last_fullsized":"2023-10-18T09:08:58.303452+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:13:31.701039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.286116+0000","last_change":"2023-10-18T09:08:57.267271+0000","last_active":"2023-10-18T09:08:58.286116+0000","last_peered":"2023-10-18T09:08:58.286116+0000","last_clean":"2023-10-18T09:08:58.286116+0000","last_became_active":"2023-10-18T09:08:57.266863+0000","last_became_peered":"2023-10-18T09:08:57.266863+0000","last_unstale":"2023-10-18T09:08:58.286116+0000","last_undegraded":"2023-10-18T09:08:58.286116+0000","last_fullsized":"2023-10-18T09:08:58.286116+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:37:15.001127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.294128+0000","last_change":"2023-10-18T09:08:58.294054+0000","last_active":"2023-10-18T09:08:58.294128+0000","last_peered":"2023-10-18T09:08:58.294128+0000","last_clean":"2023-10-18T09:08:58.294128+0000","last_became_active":"2023-10-18T09:08:58.293786+0000","last_became_peered":"2023-10-18T09:08:58.293786+0000","last_unstale":"2023-10-18T09:08:58.294128+0000","last_undegraded":"2023-10-18T09:08:58.294128+0000","last_fullsized":"2023-10-18T09:08:58.294128+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:38.149676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.273407+0000","last_change":"2023-10-18T09:08:57.271862+0000","last_active":"2023-10-18T09:08:58.273407+0000","last_peered":"2023-10-18T09:08:58.273407+0000","last_clean":"2023-10-18T09:08:58.273407+0000","last_became_active":"2023-10-18T09:08:57.271531+0000","last_became_peered":"2023-10-18T09:08:57.271531+0000","last_unstale":"2023-10-18T09:08:58.273407+0000","last_undegraded":"2023-10-18T09:08:58.273407+0000","last_fullsized":"2023-10-18T09:08:58.273407+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:52:50.292445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.305581+0000","last_change":"2023-10-18T09:08:58.304237+0000","last_active":"2023-10-18T09:08:58.305581+0000","last_peered":"2023-10-18T09:08:58.305581+0000","last_clean":"2023-10-18T09:08:58.305581+0000","last_became_active":"2023-10-18T09:08:58.299066+0000","last_became_peered":"2023-10-18T09:08:58.299066+0000","last_unstale":"2023-10-18T09:08:58.305581+0000","last_undegraded":"2023-10-18T09:08:58.305581+0000","last_fullsized":"2023-10-18T09:08:58.305581+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:59.877051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.289692+0000","last_change":"2023-10-18T09:08:57.264121+0000","last_active":"2023-10-18T09:08:58.289692+0000","last_peered":"2023-10-18T09:08:58.289692+0000","last_clean":"2023-10-18T09:08:58.289692+0000","last_became_active":"2023-10-18T09:08:57.263752+0000","last_became_peered":"2023-10-18T09:08:57.263752+0000","last_unstale":"2023-10-18T09:08:58.289692+0000","last_undegraded":"2023-10-18T09:08:58.289692+0000","last_fullsized":"2023-10-18T09:08:58.289692+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:19:17.405709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-10-18T09:08:59.295416+0000","last_change":"2023-10-18T09:08:58.287968+0000","last_active":"2023-10-18T09:08:59.295416+0000","last_peered":"2023-10-18T09:08:59.295416+0000","last_clean":"2023-10-18T09:08:59.295416+0000","last_became_active":"2023-10-18T09:08:58.287735+0000","last_became_peered":"2023-10-18T09:08:58.287735+0000","last_unstale":"2023-10-18T09:08:59.295416+0000","last_undegraded":"2023-10-18T09:08:59.295416+0000","last_fullsized":"2023-10-18T09:08:59.295416+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:57.235484+0000","last_clean_scrub_stamp":"2023-10-18T09:08:57.235484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:27:22.048918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.290427+0000","last_change":"2023-10-18T09:08:57.267689+0000","last_active":"2023-10-18T09:08:58.290427+0000","last_peered":"2023-10-18T09:08:58.290427+0000","last_clean":"2023-10-18T09:08:58.290427+0000","last_became_active":"2023-10-18T09:08:57.266991+0000","last_became_peered":"2023-10-18T09:08:57.266991+0000","last_unstale":"2023-10-18T09:08:58.290427+0000","last_undegraded":"2023-10-18T09:08:58.290427+0000","last_fullsized":"2023-10-18T09:08:58.290427+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:08:56.224922+0000","last_clean_scrub_stamp":"2023-10-18T09:08:56.224922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:19:47.012465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":355,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-18T09:08:58.275550+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:08:58.275550+0000","last_peered":"2023-10-18T09:08:58.275550+0000","last_clean":"2023-10-18T09:08:58.275550+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:08:58.275550+0000","last_undegraded":"2023-10-18T09:08:58.275550+0000","last_fullsized":"2023-10-18T09:08:58.275550+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575017,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1667,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193644,"statfs":{"total":200000000,"available":198292395,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0980000000000001,"15min":1.075},"min":{"1min":0.78800000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.522,"5min":1.635,"15min":1.6990000000000001},"last":1.448},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.131,"15min":1.091},"min":{"1min":0.95399999999999996,"5min":0.63300000000000001,"15min":0.63100000000000001},"max":{"1min":1.5940000000000001,"5min":1.724,"15min":1.97},"last":0.93500000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1879999999999999,"15min":1.165},"min":{"1min":0.92200000000000004,"5min":0.67200000000000004,"15min":0.61799999999999999},"max":{"1min":1.4330000000000001,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1379999999999999,"15min":1.1339999999999999},"min":{"1min":0.80300000000000005,"5min":0.63500000000000001,"15min":0.58799999999999997},"max":{"1min":1.3640000000000001,"5min":1.782,"15min":1.7989999999999999},"last":1.22}]},{"osd":2,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.361,"15min":1.3089999999999999},"min":{"1min":0.999,"5min":0.80700000000000005,"15min":0.65400000000000003},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.9730000000000001},"last":1.794},{"interface":"front","average":{"1min":1.276,"5min":1.2490000000000001,"15min":1.24},"min":{"1min":0.90500000000000003,"5min":0.72499999999999998,"15min":0.68300000000000005},"max":{"1min":1.7030000000000001,"5min":1.8169999999999999,"15min":1.863},"last":1.619}]},{"osd":3,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3080000000000001,"15min":1.284},"min":{"1min":1.0289999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6799999999999999,"5min":1.7330000000000001,"15min":1.98},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.377,"15min":1.355},"min":{"1min":1.206,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9930000000000001},"last":1.8640000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2110000000000001,"15min":1.2210000000000001},"min":{"1min":0.90600000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.625,"5min":1.869,"15min":2.1070000000000002},"last":1.7390000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3169999999999999,"15min":1.345},"min":{"1min":0.88400000000000001,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":2.5169999999999999},"last":1.665}]},{"osd":5,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.3380000000000001,"15min":1.347},"min":{"1min":1.056,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.71,"5min":1.7989999999999999,"15min":2.0720000000000001},"last":1.51},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.3740000000000001,"15min":1.4059999999999999},"min":{"1min":0.84999999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.736,"5min":1.9490000000000001,"15min":2.0270000000000001},"last":1.968}]},{"osd":6,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.462,"15min":1.4770000000000001},"min":{"1min":1.073,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.893,"5min":2.0139999999999998,"15min":2.3959999999999999},"last":2.0670000000000002},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.4570000000000001,"15min":1.4850000000000001},"min":{"1min":0.99399999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.8520000000000001,"5min":2.0680000000000001,"15min":2.1680000000000001},"last":1.8959999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575017,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1082,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194229,"statfs":{"total":200000000,"available":198891395,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:08:19 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.238,"15min":1.1970000000000001},"min":{"1min":0.79900000000000004,"5min":0.64200000000000002,"15min":0.60299999999999998},"max":{"1min":1.585,"5min":1.643,"15min":1.6850000000000001},"last":1.145},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.371,"15min":1.319},"min":{"1min":1.0960000000000001,"5min":0.88800000000000001,"15min":0.64700000000000002},"max":{"1min":1.7230000000000001,"5min":1.899,"15min":2.0990000000000002},"last":1.2110000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:08:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1739999999999999,"15min":1.121},"min":{"1min":0.69899999999999995,"5min":0.68799999999999994,"15min":0.63800000000000001},"max":{"1min":1.593,"5min":1.722,"15min":1.722},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.2450000000000001,"15min":1.216},"min":{"1min":0.90500000000000003,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.5669999999999999,"5min":1.8169999999999999,"15min":2.9609999999999999},"last":0.876}]},{"osd":2,"last update":"Wed Oct 18 09:08:19 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.2549999999999999,"15min":1.1950000000000001},"min":{"1min":0.93200000000000005,"5min":0.70299999999999996,"15min":0.69899999999999995},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.331,"15min":1.2549999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.66300000000000003},"max":{"1min":1.696,"5min":1.75,"15min":2.1779999999999999},"last":1.0189999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:08:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.2889999999999999,"15min":1.28},"min":{"1min":0.95999999999999996,"5min":0.82999999999999996,"15min":0.79200000000000004},"max":{"1min":1.728,"5min":1.8160000000000001,"15min":2.1589999999999998},"last":1.377},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.377,"15min":1.2969999999999999},"min":{"1min":0.92800000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.643,"5min":1.994,"15min":1.994},"last":0.93600000000000005}]},{"osd":4,"last update":"Wed Oct 18 09:08:19 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.3120000000000001,"15min":1.2589999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.58299999999999996},"max":{"1min":1.3819999999999999,"5min":1.9770000000000001,"15min":2.444},"last":1.115},{"interface":"front","average":{"1min":1.278,"5min":1.335,"15min":1.3029999999999999},"min":{"1min":0.91400000000000003,"5min":0.82999999999999996,"15min":0.55000000000000004},"max":{"1min":1.6439999999999999,"5min":1.9159999999999999,"15min":2.4039999999999999},"last":1.4870000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:08:19 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.4419999999999999,"15min":1.415},"min":{"1min":1.087,"5min":0.92900000000000005,"15min":0.58899999999999997},"max":{"1min":1.7889999999999999,"5min":2.016,"15min":2.032},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3839999999999999,"15min":1.3149999999999999},"min":{"1min":1.0309999999999999,"5min":0.77100000000000002,"15min":0.49199999999999999},"max":{"1min":1.631,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.3380000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:08:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4730000000000001,"15min":1.429},"min":{"1min":1.1299999999999999,"5min":0.89400000000000002,"15min":0.52000000000000002},"max":{"1min":1.698,"5min":2.028,"15min":2.3540000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.546,"15min":1.5329999999999999},"min":{"1min":1.258,"5min":1.1890000000000001,"15min":0.68500000000000005},"max":{"1min":1.7909999999999999,"5min":2.1440000000000001,"15min":2.794},"last":1.5649999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575018,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1083,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194229,"statfs":{"total":200000000,"available":198890813,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:08:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3169999999999999,"15min":1.3240000000000001},"min":{"1min":0.68799999999999994,"5min":0.64800000000000002,"15min":0.63400000000000001},"max":{"1min":2.302,"5min":2.302,"15min":9.609},"last":1.895},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.383,"15min":1.3300000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.60799999999999998},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":5.3029999999999999},"last":1.3089999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:08:32 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.2470000000000001,"15min":1.258},"min":{"1min":0.90400000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":11.68},"last":1.792},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.2450000000000001,"15min":1.216},"min":{"1min":0.89100000000000001,"5min":0.72799999999999998,"15min":0.67000000000000004},"max":{"1min":1.79,"5min":1.841,"15min":3.7050000000000001},"last":1.0329999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:08:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2749999999999999,"15min":1.262},"min":{"1min":0.90900000000000003,"5min":0.79700000000000004,"15min":0.755},"max":{"1min":1.8029999999999999,"5min":1.8540000000000001,"15min":2.2090000000000001},"last":1.121},{"interface":"front","average":{"1min":1.371,"5min":1.2789999999999999,"15min":1.2709999999999999},"min":{"1min":1.0780000000000001,"5min":0.79000000000000004,"15min":0.72399999999999998},"max":{"1min":1.706,"5min":1.7270000000000001,"15min":3.8180000000000001},"last":1.238}]},{"osd":3,"last update":"Wed Oct 18 09:08:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.216,"15min":1.262},"min":{"1min":0.745,"5min":0.745,"15min":0.65400000000000003},"max":{"1min":1.6719999999999999,"5min":1.9810000000000001,"15min":11.797000000000001},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.395,"15min":1.369},"min":{"1min":0.93899999999999995,"5min":0.92700000000000005,"15min":0.747},"max":{"1min":1.764,"5min":2.105,"15min":2.3420000000000001},"last":1.5549999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:08:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3300000000000001,"15min":1.377},"min":{"1min":0.94999999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.8560000000000001,"5min":2.028,"15min":2.4449999999999998},"last":1.7190000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.3180000000000001,"15min":1.355},"min":{"1min":0.90800000000000003,"5min":0.67700000000000005,"15min":0.64100000000000001},"max":{"1min":1.982,"5min":2.1360000000000001,"15min":5.4580000000000002},"last":1.1779999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:08:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.236,"15min":1.2929999999999999},"min":{"1min":0.80500000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.6599999999999999,"5min":2.1030000000000002,"15min":5.141},"last":1.27},{"interface":"front","average":{"1min":1.573,"5min":1.5600000000000001,"15min":1.538},"min":{"1min":1.1839999999999999,"5min":1.095,"15min":0.71099999999999997},"max":{"1min":1.923,"5min":2.161,"15min":2.657},"last":1.845}]},{"osd":7,"last update":"Wed Oct 18 09:08:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.466,"15min":1.552},"min":{"1min":0.96099999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.76,"5min":2.444,"15min":11.829000000000001},"last":1.504},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.401,"15min":1.4399999999999999},"min":{"1min":1.123,"5min":0.93300000000000005,"15min":0.73399999999999999},"max":{"1min":1.7110000000000001,"5min":2.6419999999999999,"15min":4.8419999999999996},"last":1.452}]}]},{"osd":4,"up_from":13,"seq":55834575017,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1083,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194229,"statfs":{"total":200000000,"available":198890754,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.167,"15min":1.141},"min":{"1min":0.76200000000000001,"5min":0.66300000000000003,"15min":0.45900000000000002},"max":{"1min":1.7410000000000001,"5min":1.8100000000000001,"15min":3.1819999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.1779999999999999,"15min":1.1559999999999999},"min":{"1min":1.0349999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.52,"5min":1.6339999999999999,"15min":3.234},"last":0.89000000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.1930000000000001,"15min":1.143},"min":{"1min":0.94199999999999995,"5min":0.70099999999999996,"15min":0.60799999999999998},"max":{"1min":1.71,"5min":1.9690000000000001,"15min":3.718},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.1970000000000001,"15min":1.2070000000000001},"min":{"1min":0.95899999999999996,"5min":0.76300000000000001,"15min":0.51400000000000001},"max":{"1min":1.49,"5min":1.9259999999999999,"15min":3.4350000000000001},"last":1.258}]},{"osd":2,"last update":"Wed Oct 18 09:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.24,"15min":1.2150000000000001},"min":{"1min":0.93400000000000005,"5min":0.71799999999999997,"15min":0.56699999999999995},"max":{"1min":1.629,"5min":1.905,"15min":4.1749999999999998},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.2889999999999999,"15min":1.2989999999999999},"min":{"1min":0.88200000000000001,"5min":0.64500000000000002,"15min":0.58899999999999997},"max":{"1min":1.655,"5min":1.704,"15min":3.532},"last":1.397}]},{"osd":3,"last update":"Wed Oct 18 09:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3220000000000001,"15min":1.294},"min":{"1min":0.97399999999999998,"5min":0.73899999999999999,"15min":0.67000000000000004},"max":{"1min":1.6659999999999999,"5min":1.7310000000000001,"15min":3.363},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.389,"5min":1.3859999999999999,"15min":1.401},"min":{"1min":1.0609999999999999,"5min":0.92200000000000004,"15min":0.68899999999999995},"max":{"1min":1.778,"5min":1.778,"15min":4.3179999999999996},"last":1.306}]},{"osd":5,"last update":"Wed Oct 18 09:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.304,"15min":1.3},"min":{"1min":0.85199999999999998,"5min":0.55600000000000005,"15min":0.498},"max":{"1min":1.71,"5min":1.841,"15min":3.6160000000000001},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.3300000000000001,"15min":1.3260000000000001},"min":{"1min":0.88600000000000001,"5min":0.60099999999999998,"15min":0.38400000000000001},"max":{"1min":1.607,"5min":1.702,"15min":4.0780000000000003},"last":1.111}]},{"osd":6,"last update":"Wed Oct 18 09:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3600000000000001,"15min":1.389},"min":{"1min":0.91000000000000003,"5min":0.67400000000000004,"15min":0.51600000000000001},"max":{"1min":1.6339999999999999,"5min":1.873,"15min":5.0049999999999999},"last":1.599},{"interface":"front","average":{"1min":1.417,"5min":1.454,"15min":1.4890000000000001},"min":{"1min":1.0069999999999999,"5min":0.47499999999999998,"15min":0.45400000000000001},"max":{"1min":1.827,"5min":1.871,"15min":4.407},"last":1.661}]},{"osd":7,"last update":"Wed Oct 18 09:08:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.4339999999999999,"15min":1.454},"min":{"1min":0.97399999999999998,"5min":0.51200000000000001,"15min":0.438},"max":{"1min":1.7490000000000001,"5min":1.804,"15min":4.5170000000000003},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.452,"5min":1.5109999999999999,"15min":1.5329999999999999},"min":{"1min":1.014,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.6990000000000001,"5min":1.966,"15min":4.5460000000000003},"last":1.7390000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575019,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1088,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194223,"statfs":{"total":200000000,"available":198885083,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:08:25 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86699999999999999,"15min":0.89000000000000001},"min":{"1min":0.67400000000000004,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.2430000000000001,"5min":1.827,"15min":5.5730000000000004},"last":2.0230000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":1.0069999999999999,"15min":0.97399999999999998},"min":{"1min":0.69399999999999995,"5min":0.40899999999999997,"15min":0.32700000000000001},"max":{"1min":1.2030000000000001,"5min":1.863,"15min":6.5599999999999996},"last":0.84699999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:08:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.0549999999999999,"15min":1.012},"min":{"1min":0.69899999999999995,"5min":0.52200000000000002,"15min":0.36799999999999999},"max":{"1min":1.526,"5min":2.2650000000000001,"15min":6.8099999999999996},"last":3.6680000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.319,"15min":1.2410000000000001},"min":{"1min":0.998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.6359999999999999,"5min":1.9670000000000001,"15min":3.1920000000000002},"last":3.7069999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:08:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.149,"15min":1.103},"min":{"1min":0.77600000000000002,"5min":0.58999999999999997,"15min":0.45300000000000001},"max":{"1min":1.492,"5min":1.9550000000000001,"15min":7.4580000000000002},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.246,"15min":1.1890000000000001},"min":{"1min":0.60699999999999998,"5min":0.52700000000000002,"15min":0.51400000000000001},"max":{"1min":1.764,"5min":1.8169999999999999,"15min":5.0049999999999999},"last":2.407}]},{"osd":4,"last update":"Wed Oct 18 09:08:25 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.391,"15min":1.3939999999999999},"min":{"1min":1.1240000000000001,"5min":0.65400000000000003,"15min":0.53700000000000003},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":18.664999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3700000000000001,"15min":1.294},"min":{"1min":1.0089999999999999,"5min":0.86499999999999999,"15min":0.64000000000000001},"max":{"1min":1.6359999999999999,"5min":2.0289999999999999,"15min":7.0700000000000003},"last":7.1269999999999998}]},{"osd":5,"last update":"Wed Oct 18 09:08:25 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.3660000000000001,"15min":1.325},"min":{"1min":1.099,"5min":0.626,"15min":0.626},"max":{"1min":1.786,"5min":1.8560000000000001,"15min":6.9139999999999997},"last":3.5819999999999999},{"interface":"front","average":{"1min":1.544,"5min":1.504,"15min":1.49},"min":{"1min":0.97599999999999998,"5min":0.72999999999999998,"15min":0.56899999999999995},"max":{"1min":2.032,"5min":2.6499999999999999,"15min":16.390999999999998},"last":2.5640000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:08:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.546,"15min":1.502},"min":{"1min":0.998,"5min":0.98899999999999999,"15min":0.58199999999999996},"max":{"1min":1.986,"5min":2.2839999999999998,"15min":7.7270000000000003},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.5449999999999999,"15min":1.478},"min":{"1min":1.169,"5min":1.0049999999999999,"15min":0.60199999999999998},"max":{"1min":1.8029999999999999,"5min":2.351,"15min":7.242},"last":1.8300000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:08:25 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.6259999999999999,"15min":1.5309999999999999},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.70099999999999996},"max":{"1min":1.927,"5min":2.8170000000000002,"15min":4.8630000000000004},"last":2.5099999999999998},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.619,"15min":1.5780000000000001},"min":{"1min":1.1859999999999999,"5min":0.82899999999999996,"15min":0.622},"max":{"1min":2.0800000000000001,"5min":2.4590000000000001,"15min":12.946},"last":5.7649999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575018,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1082,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194229,"statfs":{"total":200000000,"available":198891327,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.83499999999999996,"15min":0.874},"min":{"1min":0.47799999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":4.2210000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.032,"5min":0.996,"15min":1.014},"min":{"1min":0.74199999999999999,"5min":0.40200000000000002,"15min":0.33800000000000002},"max":{"1min":1.702,"5min":2.2799999999999998,"15min":4.3959999999999999},"last":0.59899999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.018,"15min":1.0600000000000001},"min":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.49099999999999999},"max":{"1min":1.454,"5min":1.6200000000000001,"15min":7.3390000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.234,"5min":1.0760000000000001,"15min":1.111},"min":{"1min":0.93200000000000005,"5min":0.55900000000000005,"15min":0.46200000000000002},"max":{"1min":1.696,"5min":1.9510000000000001,"15min":2.5350000000000001},"last":0.63300000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1080000000000001,"15min":1.129},"min":{"1min":0.67300000000000004,"5min":0.48299999999999998,"15min":0.48199999999999998},"max":{"1min":1.6850000000000001,"5min":1.696,"15min":7.6369999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.321,"15min":1.359},"min":{"1min":0.73299999999999998,"5min":0.60199999999999998,"15min":0.54100000000000004},"max":{"1min":1.8700000000000001,"5min":1.9139999999999999,"15min":7.5570000000000004},"last":0.74299999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3899999999999999,"15min":1.4299999999999999},"min":{"1min":0.93400000000000005,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.016,"5min":2.016,"15min":7.6189999999999998},"last":1.014},{"interface":"front","average":{"1min":1.444,"5min":1.3939999999999999,"15min":1.4330000000000001},"min":{"1min":0.98299999999999998,"5min":0.89200000000000002,"15min":0.86599999999999999},"max":{"1min":1.925,"5min":2.0430000000000001,"15min":3.423},"last":0.79500000000000004}]},{"osd":5,"last update":"Wed Oct 18 09:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2769999999999999,"15min":1.306},"min":{"1min":0.877,"5min":0.82399999999999995,"15min":0.61599999999999999},"max":{"1min":1.6859999999999999,"5min":2.0600000000000001,"15min":10.507},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.419,"5min":1.389,"15min":1.46},"min":{"1min":0.873,"5min":0.81599999999999995,"15min":0.81399999999999995},"max":{"1min":1.978,"5min":1.984,"15min":7.657},"last":1.145}]},{"osd":6,"last update":"Wed Oct 18 09:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.407,"15min":1.466},"min":{"1min":1.0600000000000001,"5min":0.77700000000000002,"15min":0.74299999999999999},"max":{"1min":2.149,"5min":2.2519999999999998,"15min":3.5299999999999998},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.431,"15min":1.6060000000000001},"min":{"1min":0.90100000000000002,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.081,"5min":2.2639999999999998,"15min":37.771999999999998},"last":0.84999999999999998}]},{"osd":7,"last update":"Wed Oct 18 09:08:37 2023","interfaces":[{"interface":"back","average":{"1min":1.6140000000000001,"5min":1.4990000000000001,"15min":1.542},"min":{"1min":1.018,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.04,"5min":2.3860000000000001,"15min":3.3620000000000001},"last":0.872},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.486,"15min":1.5209999999999999},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.77600000000000002},"max":{"1min":2.1800000000000002,"5min":2.375,"15min":10.956},"last":1.171}]}]},{"osd":1,"up_from":13,"seq":55834575017,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1081,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194230,"statfs":{"total":200000000,"available":198892402,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.92700000000000005,"15min":0.88600000000000001},"min":{"1min":0.56000000000000005,"5min":0.37,"15min":0.315},"max":{"1min":1.2150000000000001,"5min":1.6519999999999999,"15min":1.659},"last":0.57199999999999995},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.155,"15min":1.046},"min":{"1min":0.753,"5min":0.58899999999999997,"15min":0.441},"max":{"1min":1.6319999999999999,"5min":1.681,"15min":1.7010000000000001},"last":0.55300000000000005}]},{"osd":2,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.085,"15min":1.0149999999999999},"min":{"1min":0.76300000000000001,"5min":0.443,"15min":0.317},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.859},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.022,"5min":0.98599999999999999,"15min":0.93700000000000006},"min":{"1min":0.58699999999999997,"5min":0.443,"15min":0.36699999999999999},"max":{"1min":1.472,"5min":1.8009999999999999,"15min":2.0230000000000001},"last":0.80800000000000005}]},{"osd":3,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.175,"15min":1.099},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.436},"max":{"1min":1.5900000000000001,"5min":1.798,"15min":2.1059999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.101,"15min":1.073},"min":{"1min":0.66400000000000003,"5min":0.47899999999999998,"15min":0.35399999999999998},"max":{"1min":1.53,"5min":1.736,"15min":3.7010000000000001},"last":0.76800000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3149999999999999,"15min":1.294},"min":{"1min":1.1479999999999999,"5min":0.75600000000000001,"15min":0.45700000000000002},"max":{"1min":1.704,"5min":1.7070000000000001,"15min":2.0489999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3480000000000001,"15min":1.3080000000000001},"min":{"1min":0.93200000000000005,"5min":0.755,"15min":0.59699999999999998},"max":{"1min":1.6399999999999999,"5min":1.877,"15min":2.0470000000000002},"last":1.2589999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.492,"15min":1.393},"min":{"1min":1,"5min":0.86499999999999999,"15min":0.56000000000000005},"max":{"1min":1.7929999999999999,"5min":2.0539999999999998,"15min":3.5750000000000002},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.486,"15min":1.4399999999999999},"min":{"1min":1.04,"5min":0.76500000000000001,"15min":0.71699999999999997},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.3769999999999998},"last":1.204}]},{"osd":6,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4339999999999999,"15min":1.3799999999999999},"min":{"1min":1.0700000000000001,"5min":0.79100000000000004,"15min":0.64200000000000002},"max":{"1min":1.7170000000000001,"5min":1.9890000000000001,"15min":2.2050000000000001},"last":1.008},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.45,"15min":1.3819999999999999},"min":{"1min":1.2330000000000001,"5min":0.89000000000000001,"15min":0.63100000000000001},"max":{"1min":2.0859999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.92900000000000005}]},{"osd":7,"last update":"Wed Oct 18 09:08:33 2023","interfaces":[{"interface":"back","average":{"1min":1.6160000000000001,"5min":1.5289999999999999,"15min":1.462},"min":{"1min":1.147,"5min":0.85699999999999998,"15min":0.69799999999999995},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.2229999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.581,"5min":1.49,"15min":1.4530000000000001},"min":{"1min":0.98199999999999998,"5min":0.81100000000000005,"15min":0.66300000000000003},"max":{"1min":2.1240000000000001,"5min":2.1269999999999998,"15min":2.4180000000000001},"last":1.28}]}]},{"osd":0,"up_from":13,"seq":55834575017,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1658,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193654,"statfs":{"total":200000000,"available":198302068,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.84299999999999997,"15min":0.85299999999999998},"min":{"1min":0.72299999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2290000000000001,"5min":1.2430000000000001,"15min":3.4420000000000002},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.121,"15min":1.0900000000000001},"min":{"1min":0.93000000000000005,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.6020000000000001,"5min":1.6950000000000001,"15min":1.8240000000000001},"last":1.1879999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.89400000000000002,"15min":0.85899999999999999},"min":{"1min":0.73099999999999998,"5min":0.46300000000000002,"15min":0.375},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.679},"last":0.64900000000000002},{"interface":"front","average":{"1min":1.321,"5min":1.202,"15min":1.1539999999999999},"min":{"1min":0.94799999999999995,"5min":0.623,"15min":0.437},"max":{"1min":1.917,"5min":1.917,"15min":1.929},"last":1.121}]},{"osd":3,"last update":"Wed Oct 18 09:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.173,"15min":1.1759999999999999},"min":{"1min":0.86299999999999999,"5min":0.52100000000000002,"15min":0.49399999999999999},"max":{"1min":1.726,"5min":1.9590000000000001,"15min":2.7639999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.423,"15min":1.4259999999999999},"min":{"1min":1.262,"5min":0.68100000000000005,"15min":0.58399999999999996},"max":{"1min":1.8029999999999999,"5min":2.1200000000000001,"15min":3.5720000000000001},"last":1.4950000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.226,"15min":1.2390000000000001},"min":{"1min":0.97199999999999998,"5min":0.73099999999999998,"15min":0.71099999999999997},"max":{"1min":1.556,"5min":1.9990000000000001,"15min":4.492},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.288,"5min":1.2929999999999999,"15min":1.274},"min":{"1min":1.042,"5min":0.77500000000000002,"15min":0.68700000000000006},"max":{"1min":1.752,"5min":1.752,"15min":1.927},"last":1.157}]},{"osd":5,"last update":"Wed Oct 18 09:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.276,"15min":1.3029999999999999},"min":{"1min":0.91200000000000003,"5min":0.75600000000000001,"15min":0.64300000000000002},"max":{"1min":1.4830000000000001,"5min":2.1779999999999999,"15min":4.2779999999999996},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.353,"15min":1.339},"min":{"1min":1.095,"5min":0.81899999999999995,"15min":0.73099999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8740000000000001},"last":1.2649999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.474,"15min":1.4850000000000001},"min":{"1min":1.244,"5min":0.81000000000000005,"15min":0.76800000000000002},"max":{"1min":2.081,"5min":2.2469999999999999,"15min":3.5299999999999998},"last":1.288},{"interface":"front","average":{"1min":1.381,"5min":1.3640000000000001,"15min":1.3979999999999999},"min":{"1min":1.075,"5min":0.86899999999999999,"15min":0.66700000000000004},"max":{"1min":1.7330000000000001,"5min":1.8720000000000001,"15min":4.4349999999999996},"last":1.3029999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:08:34 2023","interfaces":[{"interface":"back","average":{"1min":1.6699999999999999,"5min":1.5589999999999999,"15min":1.5669999999999999},"min":{"1min":1.347,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.0209999999999999,"5min":2.2629999999999999,"15min":3.492},"last":1.383},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.4910000000000001,"15min":1.5189999999999999},"min":{"1min":1.1850000000000001,"5min":0.88500000000000001,"15min":0.82099999999999995},"max":{"1min":2.1000000000000001,"5min":2.3090000000000002,"15min":4.5419999999999998},"last":0.95999999999999996}]}]}],"pool_statfs":[]}} 2023-10-18T09:09:03.343 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T09:09:03.344 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-18T09:09:03.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.616+0000 7f7bc896c640 1 -- 172.21.15.40:0/274604806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc4071240 msgr2=0x7f7bc40cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:03.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.616+0000 7f7bc896c640 1 --2- 172.21.15.40:0/274604806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc4071240 0x7f7bc40cc1c0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009830 tx=0x7f7bbc02bcf0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.616+0000 7f7bc896c640 1 -- 172.21.15.40:0/274604806 shutdown_connections 2023-10-18T09:09:03.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.616+0000 7f7bc896c640 1 --2- 172.21.15.40:0/274604806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc4071240 0x7f7bc40cc1c0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.616+0000 7f7bc896c640 1 --2- 172.21.15.40:0/274604806 >> v2:172.21.15.144:3300/0 conn(0x7f7bc413f360 0x7f7bc4070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.617+0000 7f7bc896c640 1 -- 172.21.15.40:0/274604806 >> 172.21.15.40:0/274604806 conn(0x7f7bc4069720 msgr2=0x7f7bc4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:03.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.617+0000 7f7bc896c640 1 -- 172.21.15.40:0/274604806 shutdown_connections 2023-10-18T09:09:03.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.617+0000 7f7bc896c640 1 -- 172.21.15.40:0/274604806 wait complete. 2023-10-18T09:09:03.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.618+0000 7f7bc896c640 1 Processor -- start 2023-10-18T09:09:03.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.619+0000 7f7bc896c640 1 -- start start 2023-10-18T09:09:03.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.620+0000 7f7bc896c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc4071240 0x7f7bc413d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:03.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.620+0000 7f7bc896c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc413c550 0x7f7bc413ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:03.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.620+0000 7f7bc896c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bc413f360 0x7f7bc4154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:03.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.620+0000 7f7bc896c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc413d9b0 con 0x7f7bc4071240 2023-10-18T09:09:03.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.620+0000 7f7bc896c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc413f1c0 con 0x7f7bc413c550 2023-10-18T09:09:03.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.620+0000 7f7bc896c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc4154490 con 0x7f7bc413f360 2023-10-18T09:09:03.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.620+0000 7f7bc259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc4071240 0x7f7bc413d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:03.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.620+0000 7f7bc259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc4071240 0x7f7bc413d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51292/0 (socket says 172.21.15.40:51292) 2023-10-18T09:09:03.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.620+0000 7f7bc259c640 1 -- 172.21.15.40:0/816693750 learned_addr learned my addr 172.21.15.40:0/816693750 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:03.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.620+0000 7f7bc2d9d640 1 --2- 172.21.15.40:0/816693750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bc413f360 0x7f7bc4154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:03.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.620+0000 7f7bc1d9b640 1 --2- 172.21.15.40:0/816693750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc413c550 0x7f7bc413ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:03.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.620+0000 7f7bc259c640 1 -- 172.21.15.40:0/816693750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bc413f360 msgr2=0x7f7bc4154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:03.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.620+0000 7f7bc259c640 1 --2- 172.21.15.40:0/816693750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bc413f360 0x7f7bc4154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.620+0000 7f7bc259c640 1 -- 172.21.15.40:0/816693750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc413c550 msgr2=0x7f7bc413ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:03.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.620+0000 7f7bc259c640 1 --2- 172.21.15.40:0/816693750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc413c550 0x7f7bc413ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.620+0000 7f7bc259c640 1 -- 172.21.15.40:0/816693750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc009490 con 0x7f7bc4071240 2023-10-18T09:09:03.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.621+0000 7f7bc259c640 1 --2- 172.21.15.40:0/816693750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc4071240 0x7f7bc413d470 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400cbc0 tx=0x7f7bb400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:03.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.621+0000 7f7bbaffd640 1 -- 172.21.15.40:0/816693750 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb400ad40 con 0x7f7bc4071240 2023-10-18T09:09:03.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.621+0000 7f7bbaffd640 1 -- 172.21.15.40:0/816693750 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7bb4012420 con 0x7f7bc4071240 2023-10-18T09:09:03.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.621+0000 7f7bbaffd640 1 -- 172.21.15.40:0/816693750 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4012830 con 0x7f7bc4071240 2023-10-18T09:09:03.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.621+0000 7f7bc896c640 1 -- 172.21.15.40:0/816693750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc413dbe0 con 0x7f7bc4071240 2023-10-18T09:09:03.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.621+0000 7f7bc896c640 1 -- 172.21.15.40:0/816693750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc413e1a0 con 0x7f7bc4071240 2023-10-18T09:09:03.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.623+0000 7f7bc896c640 1 -- 172.21.15.40:0/816693750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc4066fd0 con 0x7f7bc4071240 2023-10-18T09:09:03.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.624+0000 7f7bbaffd640 1 -- 172.21.15.40:0/816693750 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7bb401a3b0 con 0x7f7bc4071240 2023-10-18T09:09:03.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.624+0000 7f7bbaffd640 1 --2- 172.21.15.40:0/816693750 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ba406ae40 0x7f7ba406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:03.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.625+0000 7f7bbaffd640 1 -- 172.21.15.40:0/816693750 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7bb4089f10 con 0x7f7bc4071240 2023-10-18T09:09:03.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.625+0000 7f7bc1d9b640 1 --2- 172.21.15.40:0/816693750 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ba406ae40 0x7f7ba406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:03.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.626+0000 7f7bc1d9b640 1 --2- 172.21.15.40:0/816693750 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ba406ae40 0x7f7ba406d300 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7bac005fd0 tx=0x7f7bac005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:03.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.629+0000 7f7bbaffd640 1 -- 172.21.15.40:0/816693750 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7bb4054f20 con 0x7f7bc4071240 2023-10-18T09:09:03.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.898+0000 7f7bc896c640 1 -- 172.21.15.40:0/816693750 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7bc4159290 con 0x7f7bc4071240 2023-10-18T09:09:03.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.904+0000 7f7bbaffd640 1 -- 172.21.15.40:0/816693750 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7bb40160b0 con 0x7f7bc4071240 2023-10-18T09:09:03.907 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2023-10-18T09:09:03.907 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBPnS9lwGywFBAAh5er25TiHU65Zj4oIxYn+w== 2023-10-18T09:09:03.908 INFO:teuthology.orchestra.run.smithi040.stdout: caps mds = "allow" 2023-10-18T09:09:03.908 INFO:teuthology.orchestra.run.smithi040.stdout: caps mon = "allow r" 2023-10-18T09:09:03.908 INFO:teuthology.orchestra.run.smithi040.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-18T09:09:03.908 INFO:teuthology.orchestra.run.smithi040.stderr:updated caps for client.0 2023-10-18T09:09:03.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.906+0000 7f7bc896c640 1 -- 172.21.15.40:0/816693750 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ba406ae40 msgr2=0x7f7ba406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:03.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.906+0000 7f7bc896c640 1 --2- 172.21.15.40:0/816693750 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ba406ae40 0x7f7ba406d300 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7bac005fd0 tx=0x7f7bac005850 comp rx=0 tx=0).stop 2023-10-18T09:09:03.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.906+0000 7f7bc896c640 1 -- 172.21.15.40:0/816693750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc4071240 msgr2=0x7f7bc413d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:03.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.906+0000 7f7bc896c640 1 --2- 172.21.15.40:0/816693750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc4071240 0x7f7bc413d470 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400cbc0 tx=0x7f7bb400a430 comp rx=0 tx=0).stop 2023-10-18T09:09:03.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.907+0000 7f7bc896c640 1 -- 172.21.15.40:0/816693750 shutdown_connections 2023-10-18T09:09:03.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.907+0000 7f7bc896c640 1 --2- 172.21.15.40:0/816693750 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ba406ae40 0x7f7ba406d300 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.907+0000 7f7bc896c640 1 --2- 172.21.15.40:0/816693750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bc413f360 0x7f7bc4154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.907+0000 7f7bc896c640 1 --2- 172.21.15.40:0/816693750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bc413c550 0x7f7bc413ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.907+0000 7f7bc896c640 1 --2- 172.21.15.40:0/816693750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bc4071240 0x7f7bc413d470 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.907+0000 7f7bc896c640 1 -- 172.21.15.40:0/816693750 >> 172.21.15.40:0/816693750 conn(0x7f7bc4069720 msgr2=0x7f7bc4157a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.907+0000 7f7bc896c640 1 -- 172.21.15.40:0/816693750 shutdown_connections 2023-10-18T09:09:03.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:03.907+0000 7f7bc896c640 1 -- 172.21.15.40:0/816693750 wait complete. 2023-10-18T09:09:03.930 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:09:04.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.158+0000 7fb6230d7640 1 -- 172.21.15.40:0/1127281917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb61c0b15b0 msgr2=0x7fb61c0b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:04.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.158+0000 7fb6230d7640 1 --2- 172.21.15.40:0/1127281917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb61c0b15b0 0x7fb61c0b0630 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fb61c06b050 tx=0x7fb61802c620 comp rx=0 tx=0).stop 2023-10-18T09:09:04.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.158+0000 7fb6230d7640 1 -- 172.21.15.40:0/1127281917 shutdown_connections 2023-10-18T09:09:04.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.158+0000 7fb6230d7640 1 --2- 172.21.15.40:0/1127281917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb61c0b2270 0x7fb61c0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:04.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.158+0000 7fb6230d7640 1 --2- 172.21.15.40:0/1127281917 >> v2:172.21.15.144:3300/0 conn(0x7fb61c0b1e10 0x7fb61c0c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:04.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.158+0000 7fb6230d7640 1 --2- 172.21.15.40:0/1127281917 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb61c0b15b0 0x7fb61c0b0630 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:04.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.158+0000 7fb6230d7640 1 -- 172.21.15.40:0/1127281917 >> 172.21.15.40:0/1127281917 conn(0x7fb61c069720 msgr2=0x7fb61c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:04.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.158+0000 7fb6230d7640 1 -- 172.21.15.40:0/1127281917 shutdown_connections 2023-10-18T09:09:04.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.158+0000 7fb6230d7640 1 -- 172.21.15.40:0/1127281917 wait complete. 2023-10-18T09:09:04.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.160+0000 7fb6230d7640 1 Processor -- start 2023-10-18T09:09:04.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.161+0000 7fb6230d7640 1 -- start start 2023-10-18T09:09:04.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.162+0000 7fb6230d7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb61c0b15b0 0x7fb61c1486e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:04.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.162+0000 7fb6230d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb61c0b1e10 0x7fb61c15a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:04.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.162+0000 7fb6230d7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb61c0b2270 0x7fb61c13ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:04.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.162+0000 7fb6230d7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c15ab70 con 0x7fb61c0b2270 2023-10-18T09:09:04.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.162+0000 7fb6230d7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c149960 con 0x7fb61c0b1e10 2023-10-18T09:09:04.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.162+0000 7fb6230d7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c148d50 con 0x7fb61c0b15b0 2023-10-18T09:09:04.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.162+0000 7fb6228d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb61c0b2270 0x7fb61c13ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:04.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.162+0000 7fb6228d6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb61c0b2270 0x7fb61c13ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51304/0 (socket says 172.21.15.40:51304) 2023-10-18T09:09:04.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.162+0000 7fb6228d6640 1 -- 172.21.15.40:0/1691704224 learned_addr learned my addr 172.21.15.40:0/1691704224 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:04.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.163+0000 7fb6218d4640 1 --2- 172.21.15.40:0/1691704224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb61c0b1e10 0x7fb61c15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:04.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.163+0000 7fb6220d5640 1 --2- 172.21.15.40:0/1691704224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb61c0b15b0 0x7fb61c1486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:04.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.163+0000 7fb6228d6640 1 -- 172.21.15.40:0/1691704224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb61c0b15b0 msgr2=0x7fb61c1486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:04.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.163+0000 7fb6228d6640 1 --2- 172.21.15.40:0/1691704224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb61c0b15b0 0x7fb61c1486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:04.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.163+0000 7fb6228d6640 1 -- 172.21.15.40:0/1691704224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb61c0b1e10 msgr2=0x7fb61c15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:04.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.163+0000 7fb6228d6640 1 --2- 172.21.15.40:0/1691704224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb61c0b1e10 0x7fb61c15a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:04.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.163+0000 7fb6228d6640 1 -- 172.21.15.40:0/1691704224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb618009df0 con 0x7fb61c0b2270 2023-10-18T09:09:04.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.163+0000 7fb6228d6640 1 --2- 172.21.15.40:0/1691704224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb61c0b2270 0x7fb61c13ad50 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fb61400ccb0 tx=0x7fb61400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:04.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.164+0000 7fb60effd640 1 -- 172.21.15.40:0/1691704224 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61400f450 con 0x7fb61c0b2270 2023-10-18T09:09:04.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.164+0000 7fb60effd640 1 -- 172.21.15.40:0/1691704224 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb61400f6c0 con 0x7fb61c0b2270 2023-10-18T09:09:04.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.164+0000 7fb60effd640 1 -- 172.21.15.40:0/1691704224 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61400fad0 con 0x7fb61c0b2270 2023-10-18T09:09:04.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.164+0000 7fb6230d7640 1 -- 172.21.15.40:0/1691704224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61c149030 con 0x7fb61c0b2270 2023-10-18T09:09:04.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.164+0000 7fb6230d7640 1 -- 172.21.15.40:0/1691704224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61c1495f0 con 0x7fb61c0b2270 2023-10-18T09:09:04.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.166+0000 7fb6230d7640 1 -- 172.21.15.40:0/1691704224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61c149b10 con 0x7fb61c0b2270 2023-10-18T09:09:04.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.166+0000 7fb60effd640 1 -- 172.21.15.40:0/1691704224 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb61400fc30 con 0x7fb61c0b2270 2023-10-18T09:09:04.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.167+0000 7fb60effd640 1 --2- 172.21.15.40:0/1691704224 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb60006ae40 0x7fb60006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:04.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.167+0000 7fb60effd640 1 -- 172.21.15.40:0/1691704224 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb614089c80 con 0x7fb61c0b2270 2023-10-18T09:09:04.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.168+0000 7fb6220d5640 1 --2- 172.21.15.40:0/1691704224 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb60006ae40 0x7fb60006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:04.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.169+0000 7fb6220d5640 1 --2- 172.21.15.40:0/1691704224 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb60006ae40 0x7fb60006d300 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb618035000 tx=0x7fb618004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:04.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.171+0000 7fb60effd640 1 -- 172.21.15.40:0/1691704224 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb61c149b10 con 0x7fb61c0b2270 2023-10-18T09:09:04.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.405+0000 7fb6230d7640 1 -- 172.21.15.40:0/1691704224 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb61c149ad0 con 0x7fb61c0b2270 2023-10-18T09:09:04.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.406+0000 7fb60effd640 1 -- 172.21.15.40:0/1691704224 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4923 (secure 0 0 0) 0x7fb61401d090 con 0x7fb61c0b2270 2023-10-18T09:09:04.410 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:04.410 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2038256442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2038256442},{"type":"v1","addr":"172.21.15.144:6836","nonce":2038256442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3006484507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3006484507},{"type":"v1","addr":"172.21.15.144:6837","nonce":3006484507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1542000032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1542000032},{"type":"v1","addr":"172.21.15.40:6834","nonce":1542000032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:08:57.731109+0000","modified":"2023-10-18T09:09:00.823289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3976883051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3976883051},{"type":"v1","addr":"172.21.15.40:6835","nonce":3976883051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-18T09:09:04.411 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 90 2023-10-18T09:09:04.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.409+0000 7fb6230d7640 1 -- 172.21.15.40:0/1691704224 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb60006ae40 msgr2=0x7fb60006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:04.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.409+0000 7fb6230d7640 1 --2- 172.21.15.40:0/1691704224 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb60006ae40 0x7fb60006d300 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb618035000 tx=0x7fb618004730 comp rx=0 tx=0).stop 2023-10-18T09:09:04.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.409+0000 7fb6230d7640 1 -- 172.21.15.40:0/1691704224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb61c0b2270 msgr2=0x7fb61c13ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:04.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.409+0000 7fb6230d7640 1 --2- 172.21.15.40:0/1691704224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb61c0b2270 0x7fb61c13ad50 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fb61400ccb0 tx=0x7fb61400a620 comp rx=0 tx=0).stop 2023-10-18T09:09:04.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.409+0000 7fb6230d7640 1 -- 172.21.15.40:0/1691704224 shutdown_connections 2023-10-18T09:09:04.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.409+0000 7fb6230d7640 1 --2- 172.21.15.40:0/1691704224 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb60006ae40 0x7fb60006d300 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:04.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.409+0000 7fb6230d7640 1 --2- 172.21.15.40:0/1691704224 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb61c0b2270 0x7fb61c13ad50 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:04.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.409+0000 7fb6230d7640 1 --2- 172.21.15.40:0/1691704224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb61c0b1e10 0x7fb61c15a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:04.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.409+0000 7fb6230d7640 1 --2- 172.21.15.40:0/1691704224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb61c0b15b0 0x7fb61c1486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:04.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.409+0000 7fb6230d7640 1 -- 172.21.15.40:0/1691704224 >> 172.21.15.40:0/1691704224 conn(0x7fb61c069720 msgr2=0x7fb61c1572f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:04.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.409+0000 7fb6230d7640 1 -- 172.21.15.40:0/1691704224 shutdown_connections 2023-10-18T09:09:04.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:04.409+0000 7fb6230d7640 1 -- 172.21.15.40:0/1691704224 wait complete. 2023-10-18T09:09:04.835 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T09:08:57.731109+0000', 'modified': '2023-10-18T09:09:00.823289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9282': {'gid': 9282, 'name': 'c', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/3976883051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6833', 'nonce': 3976883051}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 3976883051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T09:09:04.836 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T09:09:04.836 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-18T09:09:05.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.078+0000 7f33ba247640 1 -- 172.21.15.40:0/3071631811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b40ca160 msgr2=0x7f33b40b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:05.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.078+0000 7f33ba247640 1 --2- 172.21.15.40:0/3071631811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b40ca160 0x7f33b40b0a80 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009830 tx=0x7f33a402bd50 comp rx=0 tx=0).stop 2023-10-18T09:09:05.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.078+0000 7f33ba247640 1 -- 172.21.15.40:0/3071631811 shutdown_connections 2023-10-18T09:09:05.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.078+0000 7f33ba247640 1 --2- 172.21.15.40:0/3071631811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33b41425a0 0x7f33b40cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.078+0000 7f33ba247640 1 --2- 172.21.15.40:0/3071631811 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b40ca160 0x7f33b40b0a80 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.078+0000 7f33ba247640 1 -- 172.21.15.40:0/3071631811 >> 172.21.15.40:0/3071631811 conn(0x7f33b4069720 msgr2=0x7f33b4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:05.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.078+0000 7f33ba247640 1 -- 172.21.15.40:0/3071631811 shutdown_connections 2023-10-18T09:09:05.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.078+0000 7f33ba247640 1 -- 172.21.15.40:0/3071631811 wait complete. 2023-10-18T09:09:05.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.080+0000 7f33ba247640 1 Processor -- start 2023-10-18T09:09:05.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.080+0000 7f33ba247640 1 -- start start 2023-10-18T09:09:05.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.081+0000 7f33ba247640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b40ca160 0x7f33b413a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.081+0000 7f33ba247640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b41420f0 0x7f33b40a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.081+0000 7f33ba247640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33b41425a0 0x7f33b40a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.081+0000 7f33ba247640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40a6ef0 con 0x7f33b40ca160 2023-10-18T09:09:05.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.081+0000 7f33ba247640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40a71a0 con 0x7f33b41420f0 2023-10-18T09:09:05.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.081+0000 7f33ba247640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40a7480 con 0x7f33b41425a0 2023-10-18T09:09:05.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.082+0000 7f33b9245640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b40ca160 0x7f33b413a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.082+0000 7f33b9245640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b40ca160 0x7f33b413a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34462/0 (socket says 172.21.15.40:34462) 2023-10-18T09:09:05.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.082+0000 7f33b9245640 1 -- 172.21.15.40:0/4081510540 learned_addr learned my addr 172.21.15.40:0/4081510540 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:05.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.082+0000 7f33b9a46640 1 --2- 172.21.15.40:0/4081510540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33b41425a0 0x7f33b40a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.082+0000 7f33b8a44640 1 --2- 172.21.15.40:0/4081510540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b41420f0 0x7f33b40a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.082+0000 7f33b9245640 1 -- 172.21.15.40:0/4081510540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33b41425a0 msgr2=0x7f33b40a69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:05.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.082+0000 7f33b9245640 1 --2- 172.21.15.40:0/4081510540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33b41425a0 0x7f33b40a69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.082+0000 7f33b9245640 1 -- 172.21.15.40:0/4081510540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b41420f0 msgr2=0x7f33b40a6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:05.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.082+0000 7f33b9245640 1 --2- 172.21.15.40:0/4081510540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b41420f0 0x7f33b40a6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.082+0000 7f33b9245640 1 -- 172.21.15.40:0/4081510540 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a4009490 con 0x7f33b40ca160 2023-10-18T09:09:05.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.082+0000 7f33b9245640 1 --2- 172.21.15.40:0/4081510540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b40ca160 0x7f33b413a8f0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f33b000c4b0 tx=0x7f33b000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:05.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.083+0000 7f33a9ffb640 1 -- 172.21.15.40:0/4081510540 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0005480 con 0x7f33b40ca160 2023-10-18T09:09:05.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.083+0000 7f33a9ffb640 1 -- 172.21.15.40:0/4081510540 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f33b00036e0 con 0x7f33b40ca160 2023-10-18T09:09:05.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.083+0000 7f33a9ffb640 1 -- 172.21.15.40:0/4081510540 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0003b30 con 0x7f33b40ca160 2023-10-18T09:09:05.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.083+0000 7f33ba247640 1 -- 172.21.15.40:0/4081510540 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b40a7760 con 0x7f33b40ca160 2023-10-18T09:09:05.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.083+0000 7f33ba247640 1 -- 172.21.15.40:0/4081510540 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b40a7d20 con 0x7f33b40ca160 2023-10-18T09:09:05.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.085+0000 7f33a9ffb640 1 -- 172.21.15.40:0/4081510540 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f33b0003c90 con 0x7f33b40ca160 2023-10-18T09:09:05.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.087+0000 7f33a9ffb640 1 --2- 172.21.15.40:0/4081510540 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f339806ae40 0x7f339806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.087+0000 7f33a9ffb640 1 -- 172.21.15.40:0/4081510540 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33b0088ea0 con 0x7f33b40ca160 2023-10-18T09:09:05.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.087+0000 7f33be038740 1 Processor -- start 2023-10-18T09:09:05.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.087+0000 7f33be038740 1 -- start start 2023-10-18T09:09:05.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.087+0000 7f33b8a44640 1 --2- 172.21.15.40:0/4081510540 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f339806ae40 0x7f339806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.088+0000 7f33be038740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c346fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.088+0000 7f33be038740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 0x55c90c376b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.088+0000 7f33be038740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c3474f0 0x55c90c377050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.088+0000 7f33be038740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c2b5740 con 0x55c90c373d80 2023-10-18T09:09:05.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.088+0000 7f33be038740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c2b2e30 con 0x55c90c3474f0 2023-10-18T09:09:05.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.088+0000 7f33be038740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c90c2ae720 con 0x55c90c374160 2023-10-18T09:09:05.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.088+0000 7f33b9245640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c346fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.088+0000 7f33b9245640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c346fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34478/0 (socket says 172.21.15.40:34478) 2023-10-18T09:09:05.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.088+0000 7f33b9245640 1 -- 172.21.15.40:0/2985196010 learned_addr learned my addr 172.21.15.40:0/2985196010 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:05.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.089+0000 7f33b8a44640 1 --2- 172.21.15.40:0/2985196010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 0x55c90c376b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.089+0000 7f33b9245640 1 -- 172.21.15.40:0/2985196010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 msgr2=0x55c90c376b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:09:05.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.089+0000 7f33b9a46640 1 --2- 172.21.15.40:0/2985196010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c3474f0 0x55c90c377050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.089+0000 7f33b9245640 1 --2- 172.21.15.40:0/2985196010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 0x55c90c376b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.089+0000 7f33b9245640 1 -- 172.21.15.40:0/2985196010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c3474f0 msgr2=0x55c90c377050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:09:05.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.089+0000 7f33b9245640 1 --2- 172.21.15.40:0/2985196010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c3474f0 0x55c90c377050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.089+0000 7f33b8a44640 1 --2- 172.21.15.40:0/4081510540 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f339806ae40 0x7f339806d300 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f33a4036000 tx=0x7f33a4004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:05.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.089+0000 7f33b9245640 1 -- 172.21.15.40:0/2985196010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c90c272910 con 0x55c90c373d80 2023-10-18T09:09:05.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.089+0000 7f33b9245640 1 --2- 172.21.15.40:0/2985196010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c346fb0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f33b00010c0 tx=0x7f33b0058330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c5305330be31e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:05.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.089+0000 7f3396ffd640 1 -- 172.21.15.40:0/2985196010 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0058ea0 con 0x55c90c373d80 2023-10-18T09:09:05.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.089+0000 7f3396ffd640 1 -- 172.21.15.40:0/2985196010 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f33b0055f10 con 0x55c90c373d80 2023-10-18T09:09:05.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.089+0000 7f3396ffd640 1 -- 172.21.15.40:0/2985196010 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0057060 con 0x55c90c373d80 2023-10-18T09:09:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.090+0000 7f33be038740 1 -- 172.21.15.40:0/2985196010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 msgr2=0x55c90c346fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.090+0000 7f33be038740 1 --2- 172.21.15.40:0/2985196010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c346fb0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f33b00010c0 tx=0x7f33b0058330 comp rx=0 tx=0).stop 2023-10-18T09:09:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.090+0000 7f33be038740 1 -- 172.21.15.40:0/2985196010 shutdown_connections 2023-10-18T09:09:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.090+0000 7f33be038740 1 --2- 172.21.15.40:0/2985196010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c3474f0 0x55c90c377050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.090+0000 7f33be038740 1 --2- 172.21.15.40:0/2985196010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 0x55c90c376b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.090+0000 7f33be038740 1 --2- 172.21.15.40:0/2985196010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c346fb0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.090+0000 7f33be038740 1 -- 172.21.15.40:0/2985196010 >> 172.21.15.40:0/2985196010 conn(0x55c90c349220 msgr2=0x55c90c391b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:05.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.090+0000 7f33be038740 1 -- 172.21.15.40:0/2985196010 shutdown_connections 2023-10-18T09:09:05.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.090+0000 7f33be038740 1 -- 172.21.15.40:0/2985196010 wait complete. 2023-10-18T09:09:05.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.091+0000 7f33be038740 1 -- start start 2023-10-18T09:09:05.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.091+0000 7f33be038740 1 Processor -- start 2023-10-18T09:09:05.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.092+0000 7f33be038740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c371750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.092+0000 7f33be038740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 0x55c90c38a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.092+0000 7f33be038740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c3a17d0 0x55c90c379f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.092+0000 7f33be038740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c24c970 con 0x55c90c373d80 2023-10-18T09:09:05.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.092+0000 7f33be038740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c24c610 con 0x55c90c3a17d0 2023-10-18T09:09:05.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.092+0000 7f33be038740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c90c012f20 con 0x55c90c374160 2023-10-18T09:09:05.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.092+0000 7f33be038740 10 client.0 _ll_register_callbacks cb 0x55c90c2700c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:09:05.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.092+0000 7f33b9245640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c371750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.092+0000 7f33b9245640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c371750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34482/0 (socket says 172.21.15.40:34482) 2023-10-18T09:09:05.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.093+0000 7f33b9245640 1 -- 172.21.15.40:0/3617568635 learned_addr learned my addr 172.21.15.40:0/3617568635 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:05.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.093+0000 7f33b9a46640 1 --2- 172.21.15.40:0/3617568635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c3a17d0 0x55c90c379f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.093+0000 7f33b8a44640 1 --2- 172.21.15.40:0/3617568635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 0x55c90c38a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.093+0000 7f33b9245640 1 -- 172.21.15.40:0/3617568635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 msgr2=0x55c90c38a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:09:05.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.093+0000 7f33b9245640 1 --2- 172.21.15.40:0/3617568635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 0x55c90c38a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.093+0000 7f33b9245640 1 -- 172.21.15.40:0/3617568635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c3a17d0 msgr2=0x55c90c379f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:09:05.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.093+0000 7f33b9245640 1 --2- 172.21.15.40:0/3617568635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c3a17d0 0x55c90c379f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.093+0000 7f33b9245640 1 -- 172.21.15.40:0/3617568635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b40a7d20 con 0x55c90c373d80 2023-10-18T09:09:05.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.093+0000 7f33b8a44640 1 --2- 172.21.15.40:0/3617568635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 0x55c90c38a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:05.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.093+0000 7f33b9245640 1 --2- 172.21.15.40:0/3617568635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c371750 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f33b0057870 tx=0x7f33b0056080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c843bbcb0cb08797 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:05.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.093+0000 7f338ffff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T09:09:05.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.093+0000 7f338ffff640 1 -- 172.21.15.40:0/3617568635 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0058ea0 con 0x55c90c373d80 2023-10-18T09:09:05.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.094+0000 7f338ffff640 1 -- 172.21.15.40:0/3617568635 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33b005fbe0 con 0x55c90c373d80 2023-10-18T09:09:05.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.094+0000 7f33be038740 1 -- 172.21.15.40:0/3617568635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c90c2a7700 con 0x55c90c373d80 2023-10-18T09:09:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.094+0000 7f338ffff640 1 -- 172.21.15.40:0/3617568635 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33b0087070 con 0x55c90c373d80 2023-10-18T09:09:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.094+0000 7f33be038740 10 client.9426 fetch_fsmap learned FSMap version 90 2023-10-18T09:09:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.094+0000 7f33be038740 1 -- 172.21.15.40:0/3617568635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55c90c2acda0 con 0x55c90c373d80 2023-10-18T09:09:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.094+0000 7f338ffff640 1 -- 172.21.15.40:0/3617568635 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55c90c2a7700 con 0x55c90c373d80 2023-10-18T09:09:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.094+0000 7f338ffff640 1 -- 172.21.15.40:0/3617568635 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 90) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f33b0076360 con 0x55c90c373d80 2023-10-18T09:09:05.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.094+0000 7f33be038740 10 client.9426 fetch_fsmap finished waiting for FSMap version 90 2023-10-18T09:09:05.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.094+0000 7f33be038740 10 client.9426 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T09:09:05.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.094+0000 7f33be038740 20 client.9426 populate_metadata read hostname 'smithi040' 2023-10-18T09:09:05.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.094+0000 7f33be038740 1 --2- 172.21.15.40:0/3617568635 >> [v2:172.21.15.40:6833/3976883051,v1:172.21.15.40:6835/3976883051] conn(0x55c90c378e70 0x55c90c379250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.094+0000 7f33be038740 4 client.9426 mds_command: new command op to 9282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:09:05.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.094+0000 7f33be038740 1 -- 172.21.15.40:0/3617568635 --> [v2:172.21.15.40:6833/3976883051,v1:172.21.15.40:6835/3976883051] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c90c2a7420 con 0x55c90c378e70 2023-10-18T09:09:05.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.095+0000 7f33b8a44640 1 --2- 172.21.15.40:0/3617568635 >> [v2:172.21.15.40:6833/3976883051,v1:172.21.15.40:6835/3976883051] conn(0x55c90c378e70 0x55c90c379250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.095+0000 7f33b8a44640 1 --2- 172.21.15.40:0/3617568635 >> [v2:172.21.15.40:6833/3976883051,v1:172.21.15.40:6835/3976883051] conn(0x55c90c378e70 0x55c90c379250 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e288acd36c1118e server_cookie=6d5ba1d900508c25 in_seq=0 out_seq=0 2023-10-18T09:09:05.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.095+0000 7f338ffff640 10 client.9426 ms_handle_connect on v2:172.21.15.40:6833/3976883051 2023-10-18T09:09:05.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.096+0000 7f338ffff640 1 -- 172.21.15.40:0/3617568635 <== mds.0 v2:172.21.15.40:6833/3976883051 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c90c2a7420 con 0x55c90c378e70 2023-10-18T09:09:05.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.096+0000 7f338ffff640 10 client.9426 handle_command_reply: tid=0 2023-10-18T09:09:05.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.096+0000 7f33be038740 1 client.9426 shutdown 2023-10-18T09:09:05.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.096+0000 7f33be038740 1 -- 172.21.15.40:0/3617568635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 msgr2=0x55c90c371750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:05.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.096+0000 7f33be038740 1 --2- 172.21.15.40:0/3617568635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c371750 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f33b0057870 tx=0x7f33b0056080 comp rx=0 tx=0).stop 2023-10-18T09:09:05.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.096+0000 7f33be038740 1 -- 172.21.15.40:0/3617568635 shutdown_connections 2023-10-18T09:09:05.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.096+0000 7f33be038740 1 --2- 172.21.15.40:0/3617568635 >> [v2:172.21.15.40:6833/3976883051,v1:172.21.15.40:6835/3976883051] conn(0x55c90c378e70 0x55c90c379250 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.096+0000 7f33be038740 1 --2- 172.21.15.40:0/3617568635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c3a17d0 0x55c90c379f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.096+0000 7f33be038740 1 --2- 172.21.15.40:0/3617568635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 0x55c90c38a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.096+0000 7f33be038740 1 --2- 172.21.15.40:0/3617568635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c371750 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.096+0000 7f338ffff640 0 client.9426 ms_handle_reset on v2:172.21.15.40:6833/3976883051 2023-10-18T09:09:05.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.096+0000 7f33be038740 1 -- 172.21.15.40:0/3617568635 >> 172.21.15.40:0/3617568635 conn(0x55c90c3474f0 msgr2=0x55c90c34ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:05.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.097+0000 7f33be038740 1 -- 172.21.15.40:0/3617568635 shutdown_connections 2023-10-18T09:09:05.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.097+0000 7f33be038740 1 -- 172.21.15.40:0/3617568635 wait complete. 2023-10-18T09:09:05.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.097+0000 7f33be038740 20 client.9426 trim_cache size 0 max 16384 2023-10-18T09:09:05.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.114+0000 7f33be038740 1 Processor -- start 2023-10-18T09:09:05.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.114+0000 7f33be038740 1 -- start start 2023-10-18T09:09:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.114+0000 7f33be038740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c3896a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.114+0000 7f33be038740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 0x55c90c389be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.115+0000 7f33be038740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c3a17d0 0x55c90c376550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.115+0000 7f33be038740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c341d70 con 0x55c90c373d80 2023-10-18T09:09:05.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.115+0000 7f33be038740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c38a120 con 0x55c90c3a17d0 2023-10-18T09:09:05.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.115+0000 7f33be038740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c90c3ab350 con 0x55c90c374160 2023-10-18T09:09:05.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.115+0000 7f33b9245640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c3896a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.115+0000 7f33b9245640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c3896a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34492/0 (socket says 172.21.15.40:34492) 2023-10-18T09:09:05.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.115+0000 7f33b9245640 1 -- 172.21.15.40:0/3723361113 learned_addr learned my addr 172.21.15.40:0/3723361113 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:05.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.115+0000 7f33b9245640 1 -- 172.21.15.40:0/3723361113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 msgr2=0x55c90c389be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:09:05.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.115+0000 7f33b9245640 1 --2- 172.21.15.40:0/3723361113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 0x55c90c389be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.115+0000 7f33b9a46640 1 --2- 172.21.15.40:0/3723361113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c3a17d0 0x55c90c376550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.115+0000 7f33b9245640 1 -- 172.21.15.40:0/3723361113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c3a17d0 msgr2=0x55c90c376550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:09:05.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.115+0000 7f33b9245640 1 --2- 172.21.15.40:0/3723361113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c3a17d0 0x55c90c376550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.115+0000 7f33b9245640 1 -- 172.21.15.40:0/3723361113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c90c38b380 con 0x55c90c373d80 2023-10-18T09:09:05.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.116+0000 7f33b9245640 1 --2- 172.21.15.40:0/3723361113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c3896a0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f33b0005600 tx=0x7f33b0058330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd2835b18149ada server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:05.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.116+0000 7f33977fe640 1 -- 172.21.15.40:0/3723361113 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0058ea0 con 0x55c90c373d80 2023-10-18T09:09:05.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.116+0000 7f33977fe640 1 -- 172.21.15.40:0/3723361113 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f33b000e040 con 0x55c90c373d80 2023-10-18T09:09:05.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.116+0000 7f33977fe640 1 -- 172.21.15.40:0/3723361113 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b005f9e0 con 0x55c90c373d80 2023-10-18T09:09:05.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.116+0000 7f33be038740 1 -- 172.21.15.40:0/3723361113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 msgr2=0x55c90c3896a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:05.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.116+0000 7f33be038740 1 --2- 172.21.15.40:0/3723361113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c3896a0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f33b0005600 tx=0x7f33b0058330 comp rx=0 tx=0).stop 2023-10-18T09:09:05.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.116+0000 7f33be038740 1 -- 172.21.15.40:0/3723361113 shutdown_connections 2023-10-18T09:09:05.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.116+0000 7f33be038740 1 --2- 172.21.15.40:0/3723361113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c3a17d0 0x55c90c376550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.116+0000 7f33be038740 1 --2- 172.21.15.40:0/3723361113 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 0x55c90c389be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.116+0000 7f33be038740 1 --2- 172.21.15.40:0/3723361113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c373d80 0x55c90c3896a0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.116+0000 7f33be038740 1 -- 172.21.15.40:0/3723361113 >> 172.21.15.40:0/3723361113 conn(0x55c90c3474f0 msgr2=0x55c90c379d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:05.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.116+0000 7f33be038740 1 -- 172.21.15.40:0/3723361113 shutdown_connections 2023-10-18T09:09:05.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.117+0000 7f33be038740 1 -- 172.21.15.40:0/3723361113 wait complete. 2023-10-18T09:09:05.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.117+0000 7f33be038740 1 -- start start 2023-10-18T09:09:05.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.117+0000 7f33be038740 1 Processor -- start 2023-10-18T09:09:05.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.118+0000 7f33be038740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c373d80 0x55c90c3f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.118+0000 7f33be038740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 0x55c90c3f9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.118+0000 7f33be038740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c3a17d0 0x55c90c370af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.118+0000 7f33be038740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c3fa8e0 con 0x55c90c3a17d0 2023-10-18T09:09:05.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.118+0000 7f33be038740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c90c389c40 con 0x55c90c373d80 2023-10-18T09:09:05.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.118+0000 7f33be038740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c90c347a70 con 0x55c90c374160 2023-10-18T09:09:05.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.118+0000 7f33be038740 10 client.0 _ll_register_callbacks cb 0x7f33b0053f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:09:05.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.119+0000 7f33b9a46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c3a17d0 0x55c90c370af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.119+0000 7f33b9a46640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c3a17d0 0x55c90c370af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34506/0 (socket says 172.21.15.40:34506) 2023-10-18T09:09:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.119+0000 7f33b9a46640 1 -- 172.21.15.40:0/961843029 learned_addr learned my addr 172.21.15.40:0/961843029 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.119+0000 7f33b9245640 1 --2- 172.21.15.40:0/961843029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c373d80 0x55c90c3f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.119+0000 7f33b8a44640 1 --2- 172.21.15.40:0/961843029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 0x55c90c3f9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.119+0000 7f33b9a46640 1 -- 172.21.15.40:0/961843029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 msgr2=0x55c90c3f9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:09:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.119+0000 7f33b9a46640 1 --2- 172.21.15.40:0/961843029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 0x55c90c3f9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.119+0000 7f33b9a46640 1 -- 172.21.15.40:0/961843029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c373d80 msgr2=0x55c90c3f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:09:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.119+0000 7f33b9a46640 1 --2- 172.21.15.40:0/961843029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c373d80 0x55c90c3f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.119+0000 7f33b9a46640 1 -- 172.21.15.40:0/961843029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c90c2acda0 con 0x55c90c3a17d0 2023-10-18T09:09:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.119+0000 7f33b9a46640 1 --2- 172.21.15.40:0/961843029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c3a17d0 0x55c90c370af0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0127a0 tx=0x7f33ac012c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c950a72076db9ec server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.119+0000 7f338ffff640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-18T09:09:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.120+0000 7f338ffff640 1 -- 172.21.15.40:0/961843029 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00eda0 con 0x55c90c3a17d0 2023-10-18T09:09:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.120+0000 7f338ffff640 1 -- 172.21.15.40:0/961843029 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33ac004b20 con 0x55c90c3a17d0 2023-10-18T09:09:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.120+0000 7f33be038740 1 -- 172.21.15.40:0/961843029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c90bfb06c0 con 0x55c90c3a17d0 2023-10-18T09:09:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.120+0000 7f338ffff640 1 -- 172.21.15.40:0/961843029 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33ac004ce0 con 0x55c90c3a17d0 2023-10-18T09:09:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.120+0000 7f33be038740 10 client.9432 fetch_fsmap learned FSMap version 90 2023-10-18T09:09:05.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.120+0000 7f33be038740 1 -- 172.21.15.40:0/961843029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55c90c24bb80 con 0x55c90c3a17d0 2023-10-18T09:09:05.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.120+0000 7f338ffff640 1 -- 172.21.15.40:0/961843029 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55c90bfb06c0 con 0x55c90c3a17d0 2023-10-18T09:09:05.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.120+0000 7f338ffff640 1 -- 172.21.15.40:0/961843029 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 90) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f33ac005020 con 0x55c90c3a17d0 2023-10-18T09:09:05.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.120+0000 7f33be038740 10 client.9432 fetch_fsmap finished waiting for FSMap version 90 2023-10-18T09:09:05.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.120+0000 7f33be038740 10 client.9432 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-18T09:09:05.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.120+0000 7f33be038740 20 client.9432 populate_metadata read hostname 'smithi040' 2023-10-18T09:09:05.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.120+0000 7f33be038740 1 --2- 172.21.15.40:0/961843029 >> [v2:172.21.15.40:6833/3976883051,v1:172.21.15.40:6835/3976883051] conn(0x55c90c36ff70 0x55c90c370350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.120+0000 7f33be038740 4 client.9432 mds_command: new command op to 9282 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:09:05.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.120+0000 7f33be038740 1 -- 172.21.15.40:0/961843029 --> [v2:172.21.15.40:6833/3976883051,v1:172.21.15.40:6835/3976883051] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c90c003d10 con 0x55c90c36ff70 2023-10-18T09:09:05.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.121+0000 7f33b9245640 1 --2- 172.21.15.40:0/961843029 >> [v2:172.21.15.40:6833/3976883051,v1:172.21.15.40:6835/3976883051] conn(0x55c90c36ff70 0x55c90c370350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.121+0000 7f33b9245640 1 --2- 172.21.15.40:0/961843029 >> [v2:172.21.15.40:6833/3976883051,v1:172.21.15.40:6835/3976883051] conn(0x55c90c36ff70 0x55c90c370350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20684bc7bceda2b5 server_cookie=19f5d321cc19ca1c in_seq=0 out_seq=0 2023-10-18T09:09:05.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.121+0000 7f338ffff640 10 client.9432 ms_handle_connect on v2:172.21.15.40:6833/3976883051 2023-10-18T09:09:05.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.122+0000 7f338ffff640 1 -- 172.21.15.40:0/961843029 <== mds.0 v2:172.21.15.40:6833/3976883051 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55c90c003d10 con 0x55c90c36ff70 2023-10-18T09:09:05.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.122+0000 7f338ffff640 10 client.9432 handle_command_reply: tid=0 2023-10-18T09:09:05.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.122+0000 7f33be038740 1 client.9432 shutdown 2023-10-18T09:09:05.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.122+0000 7f33be038740 1 -- 172.21.15.40:0/961843029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c3a17d0 msgr2=0x55c90c370af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:05.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.122+0000 7f33be038740 1 --2- 172.21.15.40:0/961843029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c3a17d0 0x55c90c370af0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0127a0 tx=0x7f33ac012c60 comp rx=0 tx=0).stop 2023-10-18T09:09:05.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.122+0000 7f33be038740 1 -- 172.21.15.40:0/961843029 shutdown_connections 2023-10-18T09:09:05.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.122+0000 7f33be038740 1 --2- 172.21.15.40:0/961843029 >> [v2:172.21.15.40:6833/3976883051,v1:172.21.15.40:6835/3976883051] conn(0x55c90c36ff70 0x55c90c370350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.122+0000 7f33be038740 1 --2- 172.21.15.40:0/961843029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x55c90c3a17d0 0x55c90c370af0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.122+0000 7f33be038740 1 --2- 172.21.15.40:0/961843029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c90c374160 0x55c90c3f9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.122+0000 7f338ffff640 0 client.9432 ms_handle_reset on v2:172.21.15.40:6833/3976883051 2023-10-18T09:09:05.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.122+0000 7f33be038740 1 --2- 172.21.15.40:0/961843029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c90c373d80 0x55c90c3f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.122+0000 7f33be038740 1 -- 172.21.15.40:0/961843029 >> 172.21.15.40:0/961843029 conn(0x55c90c3474f0 msgr2=0x55c90c37a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:05.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.122+0000 7f33be038740 1 -- 172.21.15.40:0/961843029 shutdown_connections 2023-10-18T09:09:05.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.122+0000 7f33be038740 1 -- 172.21.15.40:0/961843029 wait complete. 2023-10-18T09:09:05.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.122+0000 7f33be038740 20 client.9432 trim_cache size 0 max 16384 2023-10-18T09:09:05.136 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:09:05.137 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:09:05.137 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-10-18T09:09:05.137 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 9282, 2023-10-18T09:09:05.137 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-10-18T09:09:05.137 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-10-18T09:09:05.138 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-10-18T09:09:05.138 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 6.8558211880000002, 2023-10-18T09:09:05.138 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 90, 2023-10-18T09:09:05.138 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 196, 2023-10-18T09:09:05.138 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 196, 2023-10-18T09:09:05.138 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 13.597670665000001 2023-10-18T09:09:05.139 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:09:05.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.123+0000 7f33ba247640 1 -- 172.21.15.40:0/4081510540 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f339806ae40 msgr2=0x7f339806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:05.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.123+0000 7f33ba247640 1 --2- 172.21.15.40:0/4081510540 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f339806ae40 0x7f339806d300 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f33a4036000 tx=0x7f33a4004730 comp rx=0 tx=0).stop 2023-10-18T09:09:05.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.123+0000 7f33ba247640 1 -- 172.21.15.40:0/4081510540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b40ca160 msgr2=0x7f33b413a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:05.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.123+0000 7f33ba247640 1 --2- 172.21.15.40:0/4081510540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b40ca160 0x7f33b413a8f0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f33b000c4b0 tx=0x7f33b000c970 comp rx=0 tx=0).stop 2023-10-18T09:09:05.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.123+0000 7f33ba247640 1 -- 172.21.15.40:0/4081510540 shutdown_connections 2023-10-18T09:09:05.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.123+0000 7f33ba247640 1 --2- 172.21.15.40:0/4081510540 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f339806ae40 0x7f339806d300 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.140 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.123+0000 7f33ba247640 1 --2- 172.21.15.40:0/4081510540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33b41425a0 0x7f33b40a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.123+0000 7f33ba247640 1 --2- 172.21.15.40:0/4081510540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b41420f0 0x7f33b40a6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.123+0000 7f33ba247640 1 --2- 172.21.15.40:0/4081510540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f33b40ca160 0x7f33b413a8f0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.123+0000 7f33ba247640 1 -- 172.21.15.40:0/4081510540 >> 172.21.15.40:0/4081510540 conn(0x7f33b4069720 msgr2=0x7f33b4158d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:05.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.123+0000 7f33ba247640 1 -- 172.21.15.40:0/4081510540 shutdown_connections 2023-10-18T09:09:05.141 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.123+0000 7f33ba247640 1 -- 172.21.15.40:0/4081510540 wait complete. 2023-10-18T09:09:05.144 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T09:09:05.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.371+0000 7fabca44b640 1 -- 172.21.15.40:0/2681569368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc4157c00 msgr2=0x7fabc4158070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:05.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.371+0000 7fabca44b640 1 --2- 172.21.15.40:0/2681569368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc4157c00 0x7fabc4158070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fabbc005fd0 tx=0x7fabbc02be10 comp rx=0 tx=0).stop 2023-10-18T09:09:05.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.371+0000 7fabca44b640 1 -- 172.21.15.40:0/2681569368 shutdown_connections 2023-10-18T09:09:05.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.371+0000 7fabca44b640 1 --2- 172.21.15.40:0/2681569368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc4157c00 0x7fabc4158070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.371+0000 7fabca44b640 1 --2- 172.21.15.40:0/2681569368 >> v2:172.21.15.144:3300/0 conn(0x7fabc406fea0 0x7fabc4070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.371+0000 7fabca44b640 1 -- 172.21.15.40:0/2681569368 >> 172.21.15.40:0/2681569368 conn(0x7fabc4069720 msgr2=0x7fabc4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:05.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.371+0000 7fabca44b640 1 -- 172.21.15.40:0/2681569368 shutdown_connections 2023-10-18T09:09:05.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.371+0000 7fabca44b640 1 -- 172.21.15.40:0/2681569368 wait complete. 2023-10-18T09:09:05.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.373+0000 7fabca44b640 1 Processor -- start 2023-10-18T09:09:05.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.373+0000 7fabca44b640 1 -- start start 2023-10-18T09:09:05.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.374+0000 7fabca44b640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabc406fea0 0x7fabc4133a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.374+0000 7fabca44b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc4151cb0 0x7fabc40cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.374+0000 7fabca44b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc4157c00 0x7fabc4155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.374+0000 7fabca44b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc4153910 con 0x7fabc4157c00 2023-10-18T09:09:05.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.374+0000 7fabca44b640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc4133f80 con 0x7fabc406fea0 2023-10-18T09:09:05.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.374+0000 7fabca44b640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc4153b20 con 0x7fabc4151cb0 2023-10-18T09:09:05.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.374+0000 7fabc89e8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc4157c00 0x7fabc4155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.374+0000 7fabc89e8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc4157c00 0x7fabc4155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34514/0 (socket says 172.21.15.40:34514) 2023-10-18T09:09:05.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.374+0000 7fabc89e8640 1 -- 172.21.15.40:0/1588962670 learned_addr learned my addr 172.21.15.40:0/1588962670 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:05.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.375+0000 7fabc37fe640 1 --2- 172.21.15.40:0/1588962670 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc4151cb0 0x7fabc40cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.375+0000 7fabc3fff640 1 --2- 172.21.15.40:0/1588962670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabc406fea0 0x7fabc4133a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.375+0000 7fabc89e8640 1 -- 172.21.15.40:0/1588962670 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc4151cb0 msgr2=0x7fabc40cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:05.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.375+0000 7fabc89e8640 1 --2- 172.21.15.40:0/1588962670 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc4151cb0 0x7fabc40cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.375+0000 7fabc89e8640 1 -- 172.21.15.40:0/1588962670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabc406fea0 msgr2=0x7fabc4133a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:05.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.375+0000 7fabc89e8640 1 --2- 172.21.15.40:0/1588962670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabc406fea0 0x7fabc4133a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.375+0000 7fabc89e8640 1 -- 172.21.15.40:0/1588962670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc009560 con 0x7fabc4157c00 2023-10-18T09:09:05.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.375+0000 7fabc3fff640 1 --2- 172.21.15.40:0/1588962670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabc406fea0 0x7fabc4133a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:05.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.375+0000 7fabc37fe640 1 --2- 172.21.15.40:0/1588962670 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc4151cb0 0x7fabc40cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:05.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.375+0000 7fabc89e8640 1 --2- 172.21.15.40:0/1588962670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc4157c00 0x7fabc4155de0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fabbc036040 tx=0x7fabbc02d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:05.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.376+0000 7fabc0ff9640 1 -- 172.21.15.40:0/1588962670 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc02ddf0 con 0x7fabc4157c00 2023-10-18T09:09:05.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.376+0000 7fabc0ff9640 1 -- 172.21.15.40:0/1588962670 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fabbc0036e0 con 0x7fabc4157c00 2023-10-18T09:09:05.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.376+0000 7fabca44b640 1 -- 172.21.15.40:0/1588962670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc4153e00 con 0x7fabc4157c00 2023-10-18T09:09:05.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.376+0000 7fabc0ff9640 1 -- 172.21.15.40:0/1588962670 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc003af0 con 0x7fabc4157c00 2023-10-18T09:09:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.376+0000 7fabca44b640 1 -- 172.21.15.40:0/1588962670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc41543c0 con 0x7fabc4157c00 2023-10-18T09:09:05.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.377+0000 7fabca44b640 1 -- 172.21.15.40:0/1588962670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc406a030 con 0x7fabc4157c00 2023-10-18T09:09:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.378+0000 7fabc0ff9640 1 -- 172.21.15.40:0/1588962670 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fabbc003c50 con 0x7fabc4157c00 2023-10-18T09:09:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.379+0000 7fabc0ff9640 1 --2- 172.21.15.40:0/1588962670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faba406ae40 0x7faba406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.379+0000 7fabc0ff9640 1 -- 172.21.15.40:0/1588962670 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabbc0ac330 con 0x7fabc4157c00 2023-10-18T09:09:05.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.380+0000 7fabc3fff640 1 --2- 172.21.15.40:0/1588962670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faba406ae40 0x7faba406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:05.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.381+0000 7fabc3fff640 1 --2- 172.21.15.40:0/1588962670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faba406ae40 0x7faba406d300 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fabb4000f60 tx=0x7fabb4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:05.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.383+0000 7fabc0ff9640 1 -- 172.21.15.40:0/1588962670 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fabbc077460 con 0x7fabc4157c00 2023-10-18T09:09:05.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.616+0000 7fabca44b640 1 -- 172.21.15.40:0/1588962670 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fabc4066fd0 con 0x7fabc4157c00 2023-10-18T09:09:05.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.617+0000 7fabc0ff9640 1 -- 172.21.15.40:0/1588962670 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7fabbc07aa80 con 0x7fabc4157c00 2023-10-18T09:09:05.620 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T09:09:05.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.619+0000 7fabca44b640 1 -- 172.21.15.40:0/1588962670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faba406ae40 msgr2=0x7faba406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:05.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.619+0000 7fabca44b640 1 --2- 172.21.15.40:0/1588962670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faba406ae40 0x7faba406d300 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fabb4000f60 tx=0x7fabb4009040 comp rx=0 tx=0).stop 2023-10-18T09:09:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.619+0000 7fabca44b640 1 -- 172.21.15.40:0/1588962670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc4157c00 msgr2=0x7fabc4155de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.619+0000 7fabca44b640 1 --2- 172.21.15.40:0/1588962670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc4157c00 0x7fabc4155de0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fabbc036040 tx=0x7fabbc02d9f0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.620+0000 7fabca44b640 1 -- 172.21.15.40:0/1588962670 shutdown_connections 2023-10-18T09:09:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.620+0000 7fabca44b640 1 --2- 172.21.15.40:0/1588962670 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faba406ae40 0x7faba406d300 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.620+0000 7fabca44b640 1 --2- 172.21.15.40:0/1588962670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc4157c00 0x7fabc4155de0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.620+0000 7fabca44b640 1 --2- 172.21.15.40:0/1588962670 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fabc4151cb0 0x7fabc40cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.620+0000 7fabca44b640 1 --2- 172.21.15.40:0/1588962670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabc406fea0 0x7fabc4133a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:05.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.620+0000 7fabca44b640 1 -- 172.21.15.40:0/1588962670 >> 172.21.15.40:0/1588962670 conn(0x7fabc4069720 msgr2=0x7fabc40ae200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:05.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.620+0000 7fabca44b640 1 -- 172.21.15.40:0/1588962670 shutdown_connections 2023-10-18T09:09:05.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:05.620+0000 7fabca44b640 1 -- 172.21.15.40:0/1588962670 wait complete. 2023-10-18T09:09:05.642 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T09:09:05.642 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T09:09:05.643 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-10-18T09:09:05.643 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T09:09:05.643 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-18T09:09:05.643 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T09:09:05.644 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T09:09:05.644 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T09:09:05.644 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:09:05.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:05.645 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-10-18T09:09:05.702 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T09:09:05.702 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T09:09:05.703 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T09:09:05.703 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:09:05.703 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-18T09:09:05.703 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:09:05.703 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:09:05.703 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:09:05.704 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-10-18T09:09:05.704 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:09:05.704 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:09:05.704 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-10-18T09:09:05.704 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T09:09:05.704 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-10-18T09:09:05.705 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-10-18T09:09:05.705 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-18T09:09:05.705 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 41579sec preferred_lft 41579sec 2023-10-18T09:09:05.705 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2023-10-18T09:09:05.705 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:09:05.706 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:09:05.706 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-10-18T09:09:05.706 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-10-18T09:09:05.706 INFO:teuthology.orchestra.run.smithi040.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T09:09:05.706 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:09:05.706 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T09:09:05.707 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:09:05.707 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::2026:95ff:fea9:a463/64 scope link 2023-10-18T09:09:05.707 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:09:05.707 INFO:teuthology.orchestra.run.smithi040.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:09:05.707 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 82:bd:e5:0f:6f:68 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T09:09:05.707 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::80bd:e5ff:fe0f:6f68/64 scope link 2023-10-18T09:09:05.708 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:09:05.708 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:09:05.708 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether ea:2b:21:f5:06:af brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T09:09:05.708 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::e82b:21ff:fef5:6af/64 scope link 2023-10-18T09:09:05.708 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:09:05.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:05.710 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T09:09:05.804 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-10-18T09:09:05.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:05.809 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-10-18T09:09:05.863 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-10-18T09:09:05.863 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T09:09:05.864 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-18T09:09:05.864 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-18T09:09:05.864 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T09:09:05.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:05.865 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-18T09:09:05.866 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-18T09:09:05.866 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-18T09:09:05.866 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-18T09:09:05.867 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-18T09:09:05.867 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-18T09:09:06.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:06.060 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-18T09:09:06.119 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T09:09:06.119 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T09:09:06.120 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:09:06.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:06.121 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-10-18T09:09:06.229 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:09:06.230 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-18T09:09:06.288 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-18T09:09:06.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:09:06.288 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-18T09:09:06.342 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-18T09:09:06.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:06.343 DEBUG:teuthology.orchestra.run.smithi040:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:09:06.492 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:09:06.492 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:09:06.526 INFO:teuthology.orchestra.run.smithi040.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:09:06.527 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:09:06.527 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T09:09:06.527 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: trying mount with new device syntax: 0@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T09:09:06.527 INFO:teuthology.orchestra.run.smithi040.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T09:09:06.531 INFO:teuthology.orchestra.run.smithi040.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-18T09:09:06.531 INFO:teuthology.orchestra.run.smithi040.stdout: You just mounted a file system that supports labels which does not 2023-10-18T09:09:06.531 INFO:teuthology.orchestra.run.smithi040.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T09:09:06.531 INFO:teuthology.orchestra.run.smithi040.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T09:09:06.532 INFO:teuthology.orchestra.run.smithi040.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T09:09:06.535 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T09:09:06.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:06.536 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-18T09:09:06.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:06.631 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:09:06.700 INFO:teuthology.orchestra.run.smithi040.stdout:client6350 2023-10-18T09:09:06.701 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:09:06.701 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:09:06.758 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-18T09:09:06.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.989+0000 7ff7f232c640 1 -- 172.21.15.40:0/1012039437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7ec071310 msgr2=0x7ff7ec06b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.989+0000 7ff7f232c640 1 --2- 172.21.15.40:0/1012039437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7ec071310 0x7ff7ec06b4d0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec141100 tx=0x7ff7e802c620 comp rx=0 tx=0).stop 2023-10-18T09:09:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.989+0000 7ff7f232c640 1 -- 172.21.15.40:0/1012039437 shutdown_connections 2023-10-18T09:09:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.989+0000 7ff7f232c640 1 --2- 172.21.15.40:0/1012039437 >> v2:172.21.15.144:3300/0 conn(0x7ff7ec158210 0x7ff7ec15af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.989+0000 7ff7f232c640 1 --2- 172.21.15.40:0/1012039437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7ec06ba10 0x7ff7ec157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.989+0000 7ff7f232c640 1 --2- 172.21.15.40:0/1012039437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7ec071310 0x7ff7ec06b4d0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:06.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.989+0000 7ff7f232c640 1 -- 172.21.15.40:0/1012039437 >> 172.21.15.40:0/1012039437 conn(0x7ff7ec0aed70 msgr2=0x7ff7ec0af150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:06.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.990+0000 7ff7f232c640 1 -- 172.21.15.40:0/1012039437 shutdown_connections 2023-10-18T09:09:06.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.990+0000 7ff7f232c640 1 -- 172.21.15.40:0/1012039437 wait complete. 2023-10-18T09:09:06.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.991+0000 7ff7f232c640 1 Processor -- start 2023-10-18T09:09:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.992+0000 7ff7f232c640 1 -- start start 2023-10-18T09:09:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.992+0000 7ff7f232c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7ec06ba10 0x7ff7ec1eecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.992+0000 7ff7f232c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7ec071310 0x7ff7ec0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.993+0000 7ff7f232c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7ec158210 0x7ff7ec0cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.993+0000 7ff7f232c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec1ef200 con 0x7ff7ec071310 2023-10-18T09:09:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.993+0000 7ff7f232c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec1f2ad0 con 0x7ff7ec158210 2023-10-18T09:09:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.993+0000 7ff7f232c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec1f2db0 con 0x7ff7ec06ba10 2023-10-18T09:09:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.993+0000 7ff7f0b29640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7ec071310 0x7ff7ec0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.993+0000 7ff7f0b29640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7ec071310 0x7ff7ec0cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34544/0 (socket says 172.21.15.40:34544) 2023-10-18T09:09:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.993+0000 7ff7f0b29640 1 -- 172.21.15.40:0/2914681603 learned_addr learned my addr 172.21.15.40:0/2914681603 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.993+0000 7ff7f132a640 1 --2- 172.21.15.40:0/2914681603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7ec06ba10 0x7ff7ec1eecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.993+0000 7ff7f1b2b640 1 --2- 172.21.15.40:0/2914681603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7ec158210 0x7ff7ec0cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.993+0000 7ff7f0b29640 1 -- 172.21.15.40:0/2914681603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7ec06ba10 msgr2=0x7ff7ec1eecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.993+0000 7ff7f0b29640 1 --2- 172.21.15.40:0/2914681603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7ec06ba10 0x7ff7ec1eecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.993+0000 7ff7f0b29640 1 -- 172.21.15.40:0/2914681603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7ec158210 msgr2=0x7ff7ec0cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.993+0000 7ff7f0b29640 1 --2- 172.21.15.40:0/2914681603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7ec158210 0x7ff7ec0cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.993+0000 7ff7f0b29640 1 -- 172.21.15.40:0/2914681603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e8009df0 con 0x7ff7ec071310 2023-10-18T09:09:07.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.994+0000 7ff7f0b29640 1 --2- 172.21.15.40:0/2914681603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7ec071310 0x7ff7ec0cc040 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00c920 tx=0x7ff7dc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.994+0000 7ff7e1ffb640 1 -- 172.21.15.40:0/2914681603 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc007c70 con 0x7ff7ec071310 2023-10-18T09:09:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.994+0000 7ff7e1ffb640 1 -- 172.21.15.40:0/2914681603 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff7dc00f450 con 0x7ff7ec071310 2023-10-18T09:09:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.994+0000 7ff7e1ffb640 1 -- 172.21.15.40:0/2914681603 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00f860 con 0x7ff7ec071310 2023-10-18T09:09:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.994+0000 7ff7f232c640 1 -- 172.21.15.40:0/2914681603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ec1f3090 con 0x7ff7ec071310 2023-10-18T09:09:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.994+0000 7ff7f232c640 1 -- 172.21.15.40:0/2914681603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ec154670 con 0x7ff7ec071310 2023-10-18T09:09:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.995+0000 7ff7f232c640 1 -- 172.21.15.40:0/2914681603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ec1f35b0 con 0x7ff7ec071310 2023-10-18T09:09:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.996+0000 7ff7e1ffb640 1 -- 172.21.15.40:0/2914681603 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff7dc00f9c0 con 0x7ff7ec071310 2023-10-18T09:09:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.997+0000 7ff7e1ffb640 1 --2- 172.21.15.40:0/2914681603 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7d006ae40 0x7ff7d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.997+0000 7ff7e1ffb640 1 -- 172.21.15.40:0/2914681603 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7dc089f50 con 0x7ff7ec071310 2023-10-18T09:09:07.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.998+0000 7ff7f132a640 1 --2- 172.21.15.40:0/2914681603 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7d006ae40 0x7ff7d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:07.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:06.999+0000 7ff7f132a640 1 --2- 172.21.15.40:0/2914681603 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7d006ae40 0x7ff7d006d300 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8035000 tx=0x7ff7e8004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:07.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:07.002+0000 7ff7e1ffb640 1 -- 172.21.15.40:0/2914681603 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff7ec1f35b0 con 0x7ff7ec071310 2023-10-18T09:09:07.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:07.235+0000 7ff7f232c640 1 -- 172.21.15.40:0/2914681603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff7ec1f35b0 con 0x7ff7ec071310 2023-10-18T09:09:07.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:07.236+0000 7ff7e1ffb640 1 -- 172.21.15.40:0/2914681603 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff7ec1f35b0 con 0x7ff7ec071310 2023-10-18T09:09:07.239 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-18T09:09:07.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:07.239+0000 7ff7f232c640 1 -- 172.21.15.40:0/2914681603 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7d006ae40 msgr2=0x7ff7d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:07.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:07.239+0000 7ff7f232c640 1 --2- 172.21.15.40:0/2914681603 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7d006ae40 0x7ff7d006d300 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8035000 tx=0x7ff7e8004730 comp rx=0 tx=0).stop 2023-10-18T09:09:07.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:07.239+0000 7ff7f232c640 1 -- 172.21.15.40:0/2914681603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7ec071310 msgr2=0x7ff7ec0cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:07.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:07.239+0000 7ff7f232c640 1 --2- 172.21.15.40:0/2914681603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7ec071310 0x7ff7ec0cc040 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00c920 tx=0x7ff7dc00cde0 comp rx=0 tx=0).stop 2023-10-18T09:09:07.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:07.239+0000 7ff7f232c640 1 -- 172.21.15.40:0/2914681603 shutdown_connections 2023-10-18T09:09:07.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:07.239+0000 7ff7f232c640 1 --2- 172.21.15.40:0/2914681603 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff7d006ae40 0x7ff7d006d300 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:07.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:07.239+0000 7ff7f232c640 1 --2- 172.21.15.40:0/2914681603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7ec158210 0x7ff7ec0cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:07.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:07.239+0000 7ff7f232c640 1 --2- 172.21.15.40:0/2914681603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7ec071310 0x7ff7ec0cc040 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:07.239+0000 7ff7f232c640 1 --2- 172.21.15.40:0/2914681603 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7ec06ba10 0x7ff7ec1eecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:07.239+0000 7ff7f232c640 1 -- 172.21.15.40:0/2914681603 >> 172.21.15.40:0/2914681603 conn(0x7ff7ec0aed70 msgr2=0x7ff7ec150cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:07.239+0000 7ff7f232c640 1 -- 172.21.15.40:0/2914681603 shutdown_connections 2023-10-18T09:09:07.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:07.239+0000 7ff7f232c640 1 -- 172.21.15.40:0/2914681603 wait complete. 2023-10-18T09:09:07.262 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T09:09:07.263 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T09:09:07.263 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2023-10-18T09:09:07.263 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-18T09:09:07.263 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-18T09:09:07.264 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T09:09:07.264 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-18T09:09:07.264 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T09:09:07.265 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:09:07.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:07.265 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2023-10-18T09:09:07.328 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T09:09:07.328 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T09:09:07.328 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T09:09:07.328 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:09:07.329 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2023-10-18T09:09:07.329 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:09:07.329 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:09:07.329 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:09:07.329 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2023-10-18T09:09:07.329 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:09:07.330 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:09:07.330 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2023-10-18T09:09:07.330 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:09:07.330 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2023-10-18T09:09:07.330 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T09:09:07.331 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2023-10-18T09:09:07.331 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T09:09:07.331 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 41570sec preferred_lft 41570sec 2023-10-18T09:09:07.331 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2023-10-18T09:09:07.331 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:09:07.331 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-18T09:09:07.332 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff 2023-10-18T09:09:07.332 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-18T09:09:07.332 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:09:07.332 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::1493:a0ff:fed2:2c1/64 scope link 2023-10-18T09:09:07.332 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:09:07.332 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:09:07.333 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 4a:6e:a9:f0:92:62 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T09:09:07.333 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::486e:a9ff:fef0:9262/64 scope link 2023-10-18T09:09:07.333 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:09:07.333 INFO:teuthology.orchestra.run.smithi144.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-18T09:09:07.333 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 36:5a:c5:54:de:6e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T09:09:07.334 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::345a:c5ff:fe54:de6e/64 scope link 2023-10-18T09:09:07.334 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:09:07.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:07.334 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T09:09:07.423 INFO:teuthology.orchestra.run.smithi144.stdout:1 2023-10-18T09:09:07.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:07.428 DEBUG:teuthology.orchestra.run.smithi144:> route 2023-10-18T09:09:07.481 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2023-10-18T09:09:07.482 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T09:09:07.482 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T09:09:07.482 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T09:09:07.482 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T09:09:07.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:07.484 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec bash -c ' 2023-10-18T09:09:07.484 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2023-10-18T09:09:07.484 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T09:09:07.485 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T09:09:07.485 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T09:09:07.485 DEBUG:teuthology.orchestra.run.smithi144:> ') 2023-10-18T09:09:07.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:07.672 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-18T09:09:07.723 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T09:09:07.723 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-18T09:09:07.724 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:09:07.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:07.725 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2023-10-18T09:09:07.819 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:09:07.819 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-18T09:09:07.873 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-18T09:09:07.873 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:09:07.874 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-18T09:09:07.920 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-18T09:09:07.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:07.920 DEBUG:teuthology.orchestra.run.smithi144:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:09:08.099 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:09:08.099 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:09:08.131 INFO:teuthology.orchestra.run.smithi144.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-18T09:09:08.131 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-18T09:09:08.132 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: resolved to: "172.21.15.40:3300,172.21.15.144:3300,172.21.15.144:3301" 2023-10-18T09:09:08.132 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: trying mount with new device syntax: 1@029855e3-538a-4478-b323-f54ab0f3a56d.cephfs=/ 2023-10-18T09:09:08.132 INFO:teuthology.orchestra.run.smithi144.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.40:3300/172.21.15.144:3300/172.21.15.144:3301" will pass to kernel 2023-10-18T09:09:08.135 INFO:teuthology.orchestra.run.smithi144.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-18T09:09:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: You just mounted a file system that supports labels which does not 2023-10-18T09:09:08.135 INFO:teuthology.orchestra.run.smithi144.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-18T09:09:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: applications will generate AVC messages and not be allowed access to 2023-10-18T09:09:08.136 INFO:teuthology.orchestra.run.smithi144.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-18T09:09:08.139 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T09:09:08.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:08.139 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-18T09:09:08.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:08.227 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:09:08.280 INFO:teuthology.orchestra.run.smithi144.stdout:client6356 2023-10-18T09:09:08.281 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:09:08.281 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:09:08.302 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:09:08.311 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:09:08.332 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:09:08.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.553+0000 7f6c3359e640 1 -- 172.21.15.40:0/3222353405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c1c0047c0 msgr2=0x7f6c1c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:08.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.553+0000 7f6c3359e640 1 --2- 172.21.15.40:0/3222353405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c1c0047c0 0x7f6c1c004c40 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800baf0 tx=0x7f6c1c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:08.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.553+0000 7f6c32d9d640 1 -- 172.21.15.40:0/3222353405 reap_dead start 2023-10-18T09:09:08.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.553+0000 7f6c3359e640 1 -- 172.21.15.40:0/3222353405 shutdown_connections 2023-10-18T09:09:08.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.553+0000 7f6c3359e640 1 -- 172.21.15.40:0/3222353405 >> 172.21.15.40:0/3222353405 conn(0x7f6c34069720 msgr2=0x7f6c34069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:08.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.553+0000 7f6c3359e640 1 -- 172.21.15.40:0/3222353405 shutdown_connections 2023-10-18T09:09:08.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.553+0000 7f6c3359e640 1 -- 172.21.15.40:0/3222353405 wait complete. 2023-10-18T09:09:08.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.556+0000 7f6c3359e640 1 Processor -- start 2023-10-18T09:09:08.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.556+0000 7f6c3359e640 1 -- start start 2023-10-18T09:09:08.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.557+0000 7f6c3359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c1c0047c0 0x7f6c341e4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:08.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.557+0000 7f6c3359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c341510e0 0x7f6c34153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:08.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.557+0000 7f6c3359e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c34153780 0x7f6c340aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.557+0000 7f6c3359e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340ab820 con 0x7f6c341510e0 2023-10-18T09:09:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.557+0000 7f6c3359e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340ab260 con 0x7f6c1c0047c0 2023-10-18T09:09:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.557+0000 7f6c3359e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340ab540 con 0x7f6c34153780 2023-10-18T09:09:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.558+0000 7f6c31d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c341510e0 0x7f6c34153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.558+0000 7f6c31d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c341510e0 0x7f6c34153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34570/0 (socket says 172.21.15.40:34570) 2023-10-18T09:09:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.558+0000 7f6c31d9b640 1 -- 172.21.15.40:0/1072944790 learned_addr learned my addr 172.21.15.40:0/1072944790 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.558+0000 7f6c3259c640 1 --2- 172.21.15.40:0/1072944790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c1c0047c0 0x7f6c341e4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.558+0000 7f6c31d9b640 1 -- 172.21.15.40:0/1072944790 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c34153780 msgr2=0x7f6c340aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.558+0000 7f6c31d9b640 1 --2- 172.21.15.40:0/1072944790 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c34153780 0x7f6c340aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:08.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.558+0000 7f6c31d9b640 1 -- 172.21.15.40:0/1072944790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c1c0047c0 msgr2=0x7f6c341e4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:08.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.558+0000 7f6c31d9b640 1 --2- 172.21.15.40:0/1072944790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c1c0047c0 0x7f6c341e4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:08.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.558+0000 7f6c31d9b640 1 -- 172.21.15.40:0/1072944790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c34152a40 con 0x7f6c341510e0 2023-10-18T09:09:08.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.558+0000 7f6c31d9b640 1 --2- 172.21.15.40:0/1072944790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c341510e0 0x7f6c34153240 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f6c2802c6a0 tx=0x7f6c28036720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:08.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.559+0000 7f6c22ffd640 1 -- 172.21.15.40:0/1072944790 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28036de0 con 0x7f6c341510e0 2023-10-18T09:09:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.559+0000 7f6c22ffd640 1 -- 172.21.15.40:0/1072944790 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6c28004120 con 0x7f6c341510e0 2023-10-18T09:09:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.559+0000 7f6c22ffd640 1 -- 172.21.15.40:0/1072944790 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c280035f0 con 0x7f6c341510e0 2023-10-18T09:09:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.559+0000 7f6c3359e640 1 -- 172.21.15.40:0/1072944790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c34152d20 con 0x7f6c341510e0 2023-10-18T09:09:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.559+0000 7f6c3359e640 1 -- 172.21.15.40:0/1072944790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c340aadc0 con 0x7f6c341510e0 2023-10-18T09:09:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.560+0000 7f6c3359e640 1 -- 172.21.15.40:0/1072944790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c341528a0 con 0x7f6c341510e0 2023-10-18T09:09:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.561+0000 7f6c22ffd640 1 -- 172.21.15.40:0/1072944790 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6c28003750 con 0x7f6c341510e0 2023-10-18T09:09:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.562+0000 7f6c22ffd640 1 --2- 172.21.15.40:0/1072944790 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6c1406ae40 0x7f6c1406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.562+0000 7f6c22ffd640 1 -- 172.21.15.40:0/1072944790 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c280ab410 con 0x7f6c341510e0 2023-10-18T09:09:08.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.563+0000 7f6c3259c640 1 --2- 172.21.15.40:0/1072944790 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6c1406ae40 0x7f6c1406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:08.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.564+0000 7f6c3259c640 1 --2- 172.21.15.40:0/1072944790 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6c1406ae40 0x7f6c1406d300 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6c24005fd0 tx=0x7f6c24005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:08.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.567+0000 7f6c22ffd640 1 -- 172.21.15.40:0/1072944790 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6c341528a0 con 0x7f6c341510e0 2023-10-18T09:09:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.802+0000 7f6c3359e640 1 -- 172.21.15.40:0/1072944790 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c341528a0 con 0x7f6c341510e0 2023-10-18T09:09:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.803+0000 7f6c22ffd640 1 -- 172.21.15.40:0/1072944790 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f6c341528a0 con 0x7f6c341510e0 2023-10-18T09:09:08.807 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:08.807 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2038256442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2038256442},{"type":"v1","addr":"172.21.15.144:6836","nonce":2038256442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3006484507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3006484507},{"type":"v1","addr":"172.21.15.144:6837","nonce":3006484507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1542000032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1542000032},{"type":"v1","addr":"172.21.15.40:6834","nonce":1542000032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:08:57.731109+0000","modified":"2023-10-18T09:09:00.823289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3976883051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3976883051},{"type":"v1","addr":"172.21.15.40:6835","nonce":3976883051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-18T09:09:08.808 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 90 2023-10-18T09:09:08.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.806+0000 7f6c3359e640 1 -- 172.21.15.40:0/1072944790 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6c1406ae40 msgr2=0x7f6c1406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:08.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.806+0000 7f6c3359e640 1 --2- 172.21.15.40:0/1072944790 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6c1406ae40 0x7f6c1406d300 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6c24005fd0 tx=0x7f6c24005850 comp rx=0 tx=0).stop 2023-10-18T09:09:08.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.806+0000 7f6c3359e640 1 -- 172.21.15.40:0/1072944790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c341510e0 msgr2=0x7f6c34153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:08.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.806+0000 7f6c3359e640 1 --2- 172.21.15.40:0/1072944790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c341510e0 0x7f6c34153240 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f6c2802c6a0 tx=0x7f6c28036720 comp rx=0 tx=0).stop 2023-10-18T09:09:08.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.806+0000 7f6c3359e640 1 -- 172.21.15.40:0/1072944790 shutdown_connections 2023-10-18T09:09:08.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.806+0000 7f6c3359e640 1 --2- 172.21.15.40:0/1072944790 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f6c1406ae40 0x7f6c1406d300 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:08.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.806+0000 7f6c3359e640 1 --2- 172.21.15.40:0/1072944790 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c34153780 0x7f6c340aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:08.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.806+0000 7f6c3359e640 1 --2- 172.21.15.40:0/1072944790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c341510e0 0x7f6c34153240 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.806+0000 7f6c3359e640 1 --2- 172.21.15.40:0/1072944790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c1c0047c0 0x7f6c341e4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.806+0000 7f6c3359e640 1 -- 172.21.15.40:0/1072944790 >> 172.21.15.40:0/1072944790 conn(0x7f6c34069720 msgr2=0x7f6c34159960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.806+0000 7f6c3359e640 1 -- 172.21.15.40:0/1072944790 shutdown_connections 2023-10-18T09:09:08.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:08.806+0000 7f6c3359e640 1 -- 172.21.15.40:0/1072944790 wait complete. 2023-10-18T09:09:08.827 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-18T09:09:09.064 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":9282,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.778784907,"mdsmap_epoch":90,"osdmap_epoch":196,"osdmap_epoch_barrier":196,"uptime":17.520656601999999} 2023-10-18T09:09:09.065 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 9282, "mdsmap_epoch": 90, "osdmap_epoch": 196, "osdmap_epoch_barrier": 196, "rank_uptime": 10.778784907, "state": "up:active", "uptime": 17.520656602, "want_state": "up:active", "whoami": 0 } 2023-10-18T09:09:09.066 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-18T09:09:09.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.301+0000 7f0b1af6e640 1 -- 172.21.15.40:0/2868222330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b1413c220 msgr2=0x7f0b140cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:09.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.301+0000 7f0b1af6e640 1 --2- 172.21.15.40:0/2868222330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b1413c220 0x7f0b140cc770 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009830 tx=0x7f0b1002bcc0 comp rx=0 tx=0).stop 2023-10-18T09:09:09.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.301+0000 7f0b1af6e640 1 -- 172.21.15.40:0/2868222330 shutdown_connections 2023-10-18T09:09:09.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.301+0000 7f0b1af6e640 1 --2- 172.21.15.40:0/2868222330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b1413c220 0x7f0b140cc770 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:09.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.301+0000 7f0b1af6e640 1 -- 172.21.15.40:0/2868222330 >> 172.21.15.40:0/2868222330 conn(0x7f0b140aeb90 msgr2=0x7f0b140aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:09.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.301+0000 7f0b1af6e640 1 -- 172.21.15.40:0/2868222330 shutdown_connections 2023-10-18T09:09:09.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.301+0000 7f0b1af6e640 1 -- 172.21.15.40:0/2868222330 wait complete. 2023-10-18T09:09:09.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.303+0000 7f0b1af6e640 1 Processor -- start 2023-10-18T09:09:09.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.303+0000 7f0b1af6e640 1 -- start start 2023-10-18T09:09:09.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.304+0000 7f0b1af6e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b140c9830 0x7f0b14156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:09.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.304+0000 7f0b1af6e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b1413c220 0x7f0b1413c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:09.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.304+0000 7f0b1af6e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b1413f1b0 0x7f0b141e43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:09.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.304+0000 7f0b1af6e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1413d220 con 0x7f0b140c9830 2023-10-18T09:09:09.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.304+0000 7f0b1af6e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1413d500 con 0x7f0b1413c220 2023-10-18T09:09:09.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.304+0000 7f0b1af6e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1413d7e0 con 0x7f0b1413f1b0 2023-10-18T09:09:09.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.305+0000 7f0b19f6c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b140c9830 0x7f0b14156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:09.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.305+0000 7f0b19f6c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b140c9830 0x7f0b14156a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34584/0 (socket says 172.21.15.40:34584) 2023-10-18T09:09:09.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.305+0000 7f0b19f6c640 1 -- 172.21.15.40:0/1544846164 learned_addr learned my addr 172.21.15.40:0/1544846164 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:09.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.305+0000 7f0b1976b640 1 --2- 172.21.15.40:0/1544846164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b1413c220 0x7f0b1413c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:09.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.305+0000 7f0b1a76d640 1 --2- 172.21.15.40:0/1544846164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b1413f1b0 0x7f0b141e43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:09.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.305+0000 7f0b19f6c640 1 -- 172.21.15.40:0/1544846164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b1413f1b0 msgr2=0x7f0b141e43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:09.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.305+0000 7f0b19f6c640 1 --2- 172.21.15.40:0/1544846164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b1413f1b0 0x7f0b141e43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:09.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.305+0000 7f0b19f6c640 1 -- 172.21.15.40:0/1544846164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b1413c220 msgr2=0x7f0b1413c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:09.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.305+0000 7f0b19f6c640 1 --2- 172.21.15.40:0/1544846164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b1413c220 0x7f0b1413c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:09.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.305+0000 7f0b19f6c640 1 -- 172.21.15.40:0/1544846164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b10009490 con 0x7f0b140c9830 2023-10-18T09:09:09.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.306+0000 7f0b19f6c640 1 --2- 172.21.15.40:0/1544846164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b140c9830 0x7f0b14156a10 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009780 tx=0x7f0b1002ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:09.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.306+0000 7f0b0a7fc640 1 -- 172.21.15.40:0/1544846164 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10005220 con 0x7f0b140c9830 2023-10-18T09:09:09.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.306+0000 7f0b0a7fc640 1 -- 172.21.15.40:0/1544846164 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0b10005470 con 0x7f0b140c9830 2023-10-18T09:09:09.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.306+0000 7f0b0a7fc640 1 -- 172.21.15.40:0/1544846164 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10003a40 con 0x7f0b140c9830 2023-10-18T09:09:09.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.306+0000 7f0b1af6e640 1 -- 172.21.15.40:0/1544846164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1413dac0 con 0x7f0b140c9830 2023-10-18T09:09:09.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.306+0000 7f0b1af6e640 1 -- 172.21.15.40:0/1544846164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b14156f50 con 0x7f0b140c9830 2023-10-18T09:09:09.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.307+0000 7f0b1af6e640 1 -- 172.21.15.40:0/1544846164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1413dfe0 con 0x7f0b140c9830 2023-10-18T09:09:09.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.308+0000 7f0b0a7fc640 1 -- 172.21.15.40:0/1544846164 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0b1003b040 con 0x7f0b140c9830 2023-10-18T09:09:09.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.309+0000 7f0b0a7fc640 1 --2- 172.21.15.40:0/1544846164 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0af806ae40 0x7f0af806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:09.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.309+0000 7f0b0a7fc640 1 -- 172.21.15.40:0/1544846164 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b100ab8e0 con 0x7f0b140c9830 2023-10-18T09:09:09.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.310+0000 7f0b1976b640 1 --2- 172.21.15.40:0/1544846164 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0af806ae40 0x7f0af806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:09.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.311+0000 7f0b1976b640 1 --2- 172.21.15.40:0/1544846164 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0af806ae40 0x7f0af806d300 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0b140abd60 tx=0x7f0b04009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:09.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.313+0000 7f0b0a7fc640 1 -- 172.21.15.40:0/1544846164 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b1413dfe0 con 0x7f0b140c9830 2023-10-18T09:09:09.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.508+0000 7f0b1af6e640 1 -- 172.21.15.40:0/1544846164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f0b1413dfe0 con 0x7f0b140c9830 2023-10-18T09:09:09.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.867+0000 7f0b0a7fc640 1 -- 172.21.15.40:0/1544846164 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v197) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0b1413dfe0 con 0x7f0b140c9830 2023-10-18T09:09:09.871 INFO:teuthology.orchestra.run.smithi040.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-18T09:09:09.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.871+0000 7f0b1af6e640 1 -- 172.21.15.40:0/1544846164 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0af806ae40 msgr2=0x7f0af806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:09.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.871+0000 7f0b1af6e640 1 --2- 172.21.15.40:0/1544846164 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0af806ae40 0x7f0af806d300 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0b140abd60 tx=0x7f0b04009040 comp rx=0 tx=0).stop 2023-10-18T09:09:09.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.871+0000 7f0b1af6e640 1 -- 172.21.15.40:0/1544846164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b140c9830 msgr2=0x7f0b14156a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:09.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.871+0000 7f0b1af6e640 1 --2- 172.21.15.40:0/1544846164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b140c9830 0x7f0b14156a10 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009780 tx=0x7f0b1002ce60 comp rx=0 tx=0).stop 2023-10-18T09:09:09.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.872+0000 7f0b1af6e640 1 -- 172.21.15.40:0/1544846164 shutdown_connections 2023-10-18T09:09:09.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.872+0000 7f0b1af6e640 1 --2- 172.21.15.40:0/1544846164 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0af806ae40 0x7f0af806d300 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:09.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.872+0000 7f0b1af6e640 1 --2- 172.21.15.40:0/1544846164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b1413f1b0 0x7f0b141e43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:09.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.872+0000 7f0b1af6e640 1 --2- 172.21.15.40:0/1544846164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b1413c220 0x7f0b1413c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:09.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.872+0000 7f0b1af6e640 1 --2- 172.21.15.40:0/1544846164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b140c9830 0x7f0b14156a10 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:09.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.872+0000 7f0b1af6e640 1 -- 172.21.15.40:0/1544846164 >> 172.21.15.40:0/1544846164 conn(0x7f0b140aeb90 msgr2=0x7f0b140aa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:09.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.872+0000 7f0b1af6e640 1 -- 172.21.15.40:0/1544846164 shutdown_connections 2023-10-18T09:09:09.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:09.872+0000 7f0b1af6e640 1 -- 172.21.15.40:0/1544846164 wait complete. 2023-10-18T09:09:09.895 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-18T09:09:10.136 INFO:teuthology.orchestra.run.smithi040.stdout:{"osd_mon_report_interval":"5"} 2023-10-18T09:09:10.136 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-18T09:09:10.137 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2023-10-18T09:09:10.137 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:09:10.138 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:09:10.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:10.191 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2023-10-18T09:09:10.468 INFO:teuthology.orchestra.run.smithi040.stderr:16+0 records in 2023-10-18T09:09:10.469 INFO:teuthology.orchestra.run.smithi040.stderr:16+0 records out 2023-10-18T09:09:10.469 INFO:teuthology.orchestra.run.smithi040.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.212406 s, 79.0 MB/s 2023-10-18T09:09:10.470 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:09:10.471 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:09:10.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:10.527 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2023-10-18T09:09:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:17+0 records in 2023-10-18T09:09:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:17+0 records out 2023-10-18T09:09:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.184162 s, 96.8 MB/s 2023-10-18T09:09:10.768 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-18T09:09:10.769 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:09:10.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.992+0000 7fe372f25640 1 -- 172.21.15.40:0/3709474141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe36c151bd0 msgr2=0x7fe36c1586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:10.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.992+0000 7fe372f25640 1 --2- 172.21.15.40:0/3709474141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe36c151bd0 0x7fe36c1586a0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fe360005fd0 tx=0x7fe36002c610 comp rx=0 tx=0).stop 2023-10-18T09:09:10.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.992+0000 7fe372f25640 1 -- 172.21.15.40:0/3709474141 shutdown_connections 2023-10-18T09:09:10.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.992+0000 7fe372f25640 1 --2- 172.21.15.40:0/3709474141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe36c0b0bc0 0x7fe36c0b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:10.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.992+0000 7fe372f25640 1 --2- 172.21.15.40:0/3709474141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe36c151bd0 0x7fe36c1586a0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:10.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.992+0000 7fe372f25640 1 -- 172.21.15.40:0/3709474141 >> 172.21.15.40:0/3709474141 conn(0x7fe36c06cee0 msgr2=0x7fe36c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:10.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.992+0000 7fe372f25640 1 -- 172.21.15.40:0/3709474141 shutdown_connections 2023-10-18T09:09:10.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.992+0000 7fe372f25640 1 -- 172.21.15.40:0/3709474141 wait complete. 2023-10-18T09:09:10.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.994+0000 7fe372f25640 1 Processor -- start 2023-10-18T09:09:10.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.994+0000 7fe372f25640 1 -- start start 2023-10-18T09:09:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.995+0000 7fe372f25640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe36c06b710 0x7fe36c14ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:11.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.995+0000 7fe372f25640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe36c0b0bc0 0x7fe36c0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:11.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe372f25640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe36c151bd0 0x7fe36c1e38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:11.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe372f25640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1e7090 con 0x7fe36c06b710 2023-10-18T09:09:11.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe372f25640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1e2b10 con 0x7fe36c0b0bc0 2023-10-18T09:09:11.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe372f25640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1e6db0 con 0x7fe36c151bd0 2023-10-18T09:09:11.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe370cc1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe36c06b710 0x7fe36c14ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:11.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe370cc1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe36c06b710 0x7fe36c14ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34612/0 (socket says 172.21.15.40:34612) 2023-10-18T09:09:11.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe370cc1640 1 -- 172.21.15.40:0/3204066809 learned_addr learned my addr 172.21.15.40:0/3204066809 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:11.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe3714c2640 1 --2- 172.21.15.40:0/3204066809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe36c151bd0 0x7fe36c1e38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:11.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe36bfff640 1 --2- 172.21.15.40:0/3204066809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe36c0b0bc0 0x7fe36c0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:11.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe370cc1640 1 -- 172.21.15.40:0/3204066809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe36c151bd0 msgr2=0x7fe36c1e38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:11.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe370cc1640 1 --2- 172.21.15.40:0/3204066809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe36c151bd0 0x7fe36c1e38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:11.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe370cc1640 1 -- 172.21.15.40:0/3204066809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe36c0b0bc0 msgr2=0x7fe36c0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:11.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe370cc1640 1 --2- 172.21.15.40:0/3204066809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe36c0b0bc0 0x7fe36c0cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:11.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe370cc1640 1 -- 172.21.15.40:0/3204066809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe360009df0 con 0x7fe36c06b710 2023-10-18T09:09:11.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe3714c2640 1 --2- 172.21.15.40:0/3204066809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe36c151bd0 0x7fe36c1e38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:11.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe370cc1640 1 --2- 172.21.15.40:0/3204066809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe36c06b710 0x7fe36c14ef40 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fe36000b3c0 tx=0x7fe360009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:11.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe3697fa640 1 -- 172.21.15.40:0/3204066809 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360009950 con 0x7fe36c06b710 2023-10-18T09:09:11.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe3697fa640 1 -- 172.21.15.40:0/3204066809 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe36002ce90 con 0x7fe36c06b710 2023-10-18T09:09:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe3697fa640 1 -- 172.21.15.40:0/3204066809 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360003940 con 0x7fe36c06b710 2023-10-18T09:09:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe372f25640 1 -- 172.21.15.40:0/3204066809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c1e24c0 con 0x7fe36c06b710 2023-10-18T09:09:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.997+0000 7fe372f25640 1 -- 172.21.15.40:0/3204066809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c142f80 con 0x7fe36c06b710 2023-10-18T09:09:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.999+0000 7fe372f25640 1 -- 172.21.15.40:0/3204066809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c066fd0 con 0x7fe36c06b710 2023-10-18T09:09:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:10.999+0000 7fe3697fa640 1 -- 172.21.15.40:0/3204066809 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe360003aa0 con 0x7fe36c06b710 2023-10-18T09:09:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.001+0000 7fe3697fa640 1 --2- 172.21.15.40:0/3204066809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe35006ae40 0x7fe35006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.001+0000 7fe3697fa640 1 -- 172.21.15.40:0/3204066809 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3600abca0 con 0x7fe36c06b710 2023-10-18T09:09:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.001+0000 7fe36bfff640 1 --2- 172.21.15.40:0/3204066809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe35006ae40 0x7fe35006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.002+0000 7fe36bfff640 1 --2- 172.21.15.40:0/3204066809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe35006ae40 0x7fe35006d300 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe358005fd0 tx=0x7fe358005ce0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:11.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.006+0000 7fe3697fa640 1 -- 172.21.15.40:0/3204066809 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe360076d50 con 0x7fe36c06b710 2023-10-18T09:09:11.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.201+0000 7fe372f25640 1 -- 172.21.15.40:0/3204066809 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe36c0cdb60 con 0x7fe36c06b710 2023-10-18T09:09:11.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.204+0000 7fe3697fa640 1 -- 172.21.15.40:0/3204066809 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v197) v1 ==== 81+0+34026 (secure 0 0 0) 0x7fe360041090 con 0x7fe36c06b710 2023-10-18T09:09:11.208 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:11.209 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:09:11.209 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 197, 2023-10-18T09:09:11.209 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:09:11.209 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:09:11.209 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:09:09.858570+0000", 2023-10-18T09:09:11.210 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:09:11.210 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:09:11.210 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:09:11.210 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:09:11.210 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:09:11.211 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:09:11.211 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:09:11.211 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:09:11.211 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:09:11.211 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:11.212 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:09:11.212 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:09:11.212 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:09:11.212 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:09:11.212 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:09:11.213 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 23, 2023-10-18T09:09:11.213 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:09:11.213 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:09:11.213 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:09:11.213 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:09:11.213 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:09:11.214 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:09:11.214 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.214 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:09:11.214 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:09:11.214 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:09:11.215 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:11.215 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:11.215 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:11.215 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:11.215 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:11.215 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:11.216 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:11.216 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:11.216 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:11.216 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:11.216 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:11.217 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:11.217 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:09:11.217 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:09:11.217 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:09:11.217 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:09:11.217 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:09:11.218 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:11.218 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:11.218 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:11.218 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:11.218 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:11.219 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:11.219 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:11.219 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.219 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:09:11.219 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:11.219 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:11.220 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:11.220 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:11.220 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:11.220 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:11.220 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:11.220 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:11.221 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:11.221 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:09:11.221 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:11.221 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:11.221 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:11.222 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:11.222 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:11.222 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:11.222 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:11.222 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:11.223 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:11.223 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:11.223 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:11.223 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:11.223 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:11.223 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:11.224 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:11.224 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:11.224 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.224 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:11.224 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:11.225 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:11.225 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:11.225 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:11.225 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:11.225 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:11.225 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:11.226 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:11.226 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:11.226 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:11.226 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:11.226 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:09:11.227 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:09:11.227 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.227 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:11.227 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:09:11.227 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.227 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:11.228 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:09:11.228 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:09:11.228 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:09:11.228 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:09:11.228 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:09:11.229 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:11.229 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:11.229 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:11.229 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.229 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.229 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.230 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 22, 2023-10-18T09:09:11.230 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:09:11.230 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:08:55.675479+0000", 2023-10-18T09:09:11.230 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:11.230 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:11.231 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:11.231 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:11.231 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:11.231 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:11.231 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:11.232 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:11.232 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:11.232 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:11.232 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:11.232 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:11.232 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:09:11.233 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:09:11.233 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:09:11.233 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:09:11.233 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:09:11.233 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:11.234 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:11.234 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:11.234 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:11.234 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:11.234 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:11.234 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:11.235 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.235 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "195", 2023-10-18T09:09:11.235 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:11.235 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:11.235 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:11.236 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:11.236 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:11.236 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:11.236 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:11.236 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:11.236 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:11.236 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:09:11.237 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:11.237 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:11.237 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:11.237 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:11.237 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:11.237 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:11.238 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:11.238 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:11.238 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:11.238 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:11.238 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:11.239 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:11.239 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:11.239 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:11.239 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:11.239 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:11.239 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.240 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:11.240 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:11.240 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:11.240 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:11.240 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:11.241 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:11.241 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:11.241 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:11.241 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:11.241 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:11.241 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:11.242 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:11.242 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:09:11.242 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:09:11.242 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:09:11.242 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.243 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:11.243 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:09:11.243 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:09:11.243 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.243 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.243 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:11.244 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 2, 2023-10-18T09:09:11.244 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 2, 2023-10-18T09:09:11.244 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:09:11.244 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:09:11.244 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:09:11.244 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:11.245 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:11.245 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:11.245 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.245 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.245 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.245 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 23, 2023-10-18T09:09:11.246 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:09:11.246 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:08:56.761888+0000", 2023-10-18T09:09:11.246 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:11.246 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:11.246 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:11.247 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:11.247 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:11.247 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:11.247 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:11.247 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:11.247 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:11.248 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:11.248 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:11.248 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:11.248 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:09:11.248 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:09:11.248 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:09:11.249 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:09:11.249 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:09:11.249 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:11.249 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:11.249 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:11.249 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:11.250 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:11.250 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:11.250 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:11.250 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.250 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "197", 2023-10-18T09:09:11.251 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:11.251 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:11.251 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:11.251 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:11.251 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:11.251 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:11.252 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:11.252 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:11.252 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:11.252 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 33554432, 2023-10-18T09:09:11.252 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:11.252 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:11.253 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:11.253 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:11.253 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:11.253 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:11.253 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:11.253 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:11.254 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:11.254 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:11.254 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:11.254 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:11.254 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:11.254 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:11.255 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:11.255 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:11.255 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.255 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:11.255 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:11.256 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:11.256 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:11.256 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:11.256 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:11.256 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:11.256 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:11.257 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:11.257 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:11.257 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:11.257 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:11.257 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:09:11.257 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:09:11.258 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.258 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:11.258 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:09:11.258 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:09:11.258 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.259 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.259 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:11.259 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.3799999952316284, 2023-10-18T09:09:11.259 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.3799999952316284, 2023-10-18T09:09:11.259 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:09:11.259 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-18T09:09:11.260 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-18T09:09:11.260 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:11.260 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:11.260 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:11.260 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.261 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.261 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:11.261 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:09:11.261 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.261 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:09:11.261 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:11.262 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:11.262 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:11.262 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:11.262 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:11.262 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:11.263 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:11.263 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:11.263 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:11.263 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:11.263 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:11.263 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:11.264 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.264 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.264 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.264 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:09:11.264 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:11.265 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.265 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.265 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.265 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:09:11.265 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:11.265 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.266 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.266 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.266 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:11.266 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.266 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.267 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.267 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:09:11.267 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:11.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.267 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.267 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.268 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:09:11.268 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:11.268 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.268 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.268 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.268 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:11.269 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.269 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.269 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.269 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:09:11.269 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:11.270 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.270 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.270 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.270 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:09:11.270 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:11.270 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.271 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.271 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:11.271 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.271 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.272 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.272 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:09:11.272 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:11.272 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.272 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.272 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.273 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:09:11.273 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:11.273 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.273 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.273 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.273 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:09:11.274 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:09:11.274 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:09:11.274 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:09:11.274 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:11.274 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:11.274 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:11.275 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.275 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.275 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.275 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:09:11.275 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:11.276 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:11.276 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:11.276 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:11.276 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:11.276 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:11.276 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:11.277 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:11.277 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:11.277 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:11.277 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:11.277 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:11.277 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.278 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.278 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.278 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:09:11.278 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:11.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.279 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.279 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.279 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:09:11.279 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:11.279 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.279 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.280 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:11.280 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.280 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.280 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.280 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:09:11.281 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:11.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.281 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.281 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:09:11.282 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:11.282 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.282 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.282 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:11.283 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.283 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.283 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.283 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:09:11.283 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:11.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.284 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.284 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.284 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:09:11.284 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:11.284 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.284 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.285 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.285 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:11.285 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.285 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.285 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.286 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:09:11.286 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:11.286 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.286 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.286 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.287 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:09:11.287 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:11.287 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.287 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.287 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.287 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:09:11.288 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:09:11.288 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:09:11.288 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:09:11.288 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:11.288 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:11.289 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:11.289 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.289 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.289 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.289 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:09:11.289 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:11.290 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:11.290 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:11.290 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:11.290 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:11.290 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:11.291 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:11.291 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:11.291 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:11.291 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:11.291 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:11.291 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:11.292 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.292 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.292 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.292 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:09:11.292 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:11.292 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.293 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.293 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.293 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:09:11.293 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:11.293 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.293 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.294 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:11.294 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.294 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.294 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.295 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:09:11.295 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:11.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.295 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.295 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:09:11.296 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:11.296 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.296 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.296 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:11.297 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.297 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.297 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:09:11.297 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:11.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.298 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.298 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:09:11.298 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:11.298 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.299 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.299 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:11.299 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.299 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.300 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:09:11.300 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:11.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.300 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.301 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:09:11.301 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:11.301 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.301 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.302 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:09:11.302 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:09:11.302 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:09:11.302 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:09:11.302 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:11.303 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:11.303 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:11.303 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.303 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.303 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:09:11.304 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:11.304 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:11.304 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:11.304 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:11.304 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:11.304 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:11.305 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:11.305 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:11.305 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:11.305 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:11.305 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:11.305 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:11.306 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.306 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.306 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.306 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:09:11.306 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:11.306 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.307 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.307 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.307 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:09:11.307 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:11.307 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.308 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.308 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.308 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:11.308 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.308 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.308 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.309 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:09:11.309 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:11.309 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.309 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.309 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.309 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:09:11.310 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:11.310 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.310 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.310 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:11.311 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.311 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.311 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.311 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:09:11.311 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:11.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.312 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.312 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:09:11.312 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:11.312 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.313 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.313 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:11.313 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.313 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.314 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:09:11.314 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:11.314 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.314 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.314 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.315 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:09:11.315 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:11.315 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.315 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.315 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:09:11.316 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:09:11.316 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:09:11.316 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:09:11.316 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:11.316 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:11.316 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:11.317 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.317 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.317 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:09:11.317 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:11.317 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:11.318 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:11.318 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:11.318 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:11.318 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:11.318 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:11.318 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:11.319 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:11.319 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:11.319 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:11.319 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:11.319 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.320 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.320 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:09:11.320 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:11.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.321 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.321 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:09:11.321 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:11.321 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.321 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.322 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:11.322 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.322 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.322 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.322 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:09:11.323 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:11.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.323 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.323 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.323 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:09:11.323 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:11.324 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.324 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.324 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.324 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:11.324 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.324 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.325 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.325 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:09:11.325 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:11.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.325 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.325 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.326 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:09:11.326 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:11.326 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.326 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.326 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.327 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:11.327 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.327 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.327 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.327 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:09:11.327 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:11.328 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.328 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.328 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.328 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:09:11.328 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:11.328 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.329 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.329 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:09:11.329 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:09:11.329 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:09:11.329 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:09:11.330 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:11.330 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:11.330 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:11.330 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.330 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.330 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.331 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:09:11.331 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:11.331 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:11.331 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:11.331 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:11.332 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:11.332 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:11.332 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:11.332 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:11.332 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:11.332 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:11.333 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:11.333 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:11.333 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.333 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.333 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:09:11.334 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:11.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.334 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.334 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:09:11.335 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:11.335 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.335 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.335 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:11.335 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.336 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.336 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:09:11.336 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:11.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.337 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.337 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:09:11.337 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:11.337 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.337 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.338 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:11.338 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.338 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.338 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:09:11.339 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:11.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.339 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.339 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:09:11.340 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:11.340 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.340 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.340 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:11.341 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.341 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.341 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:09:11.341 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:11.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.342 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.342 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:09:11.342 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:11.342 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.343 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.343 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:09:11.343 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:09:11.343 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:09:11.344 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:09:11.344 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:11.344 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:11.344 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:11.344 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.345 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.345 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.345 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:09:11.345 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:11.345 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:11.345 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:11.346 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:11.346 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:11.346 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:11.346 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:11.346 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:11.347 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:11.347 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:11.347 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:11.347 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:11.347 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.348 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.348 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:09:11.348 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:11.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.348 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.349 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:09:11.349 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:11.349 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.349 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.350 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:11.350 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.350 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.350 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:09:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:09:11.351 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:09:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:09:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.354 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:09:11.355 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:09:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:11.356 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:09:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:09:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:09:11.357 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:09:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:09:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:11.359 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:11.360 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.361 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.362 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:09:11.362 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:11.362 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.362 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.362 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.362 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:09:11.363 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:11.363 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.363 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.363 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.363 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:11.363 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.364 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.364 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.364 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:09:11.364 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:11.364 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.364 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.365 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.365 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:09:11.365 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:11.365 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.365 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.365 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.366 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:11.366 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.366 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.366 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.366 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:09:11.366 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:11.367 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.367 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.367 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.367 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:09:11.367 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:11.368 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.368 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.368 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:11.368 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:11.368 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.369 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:11.369 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:09:11.369 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:11.369 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.369 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.369 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:11.370 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:09:11.370 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:11.370 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.370 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.370 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.371 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:09:11.371 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:09:11.371 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:09:11.371 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:09:11.371 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:11.372 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:11.372 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:11.372 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:11.372 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.372 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:11.373 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:09:11.373 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.373 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:09:11.373 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:11.373 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:11.373 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:11.374 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:11.374 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:11.374 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:09:11.374 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:11.374 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.375 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.375 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:09:11.375 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:11.375 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:11.375 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:11.375 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:11.376 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:11.376 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:09:11.376 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:11.376 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.376 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.377 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:09:11.377 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:11.377 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:11.377 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:11.377 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:11.377 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:11.378 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:09:11.378 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:11.378 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.378 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.378 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:09:11.379 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:11.379 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:11.379 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:11.379 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:11.379 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:11.380 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:09:11.380 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:11.380 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.380 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.380 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:09:11.380 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:11.381 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:11.381 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:11.381 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:11.381 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:11.381 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:09:11.382 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:11.382 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.382 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.382 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:09:11.382 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:11.382 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:11.383 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:11.383 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:11.383 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:11.383 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:09:11.383 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:11.384 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.384 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.384 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:09:11.384 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:11.384 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:11.384 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:11.385 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:11.385 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:11.385 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:09:11.385 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:11.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.386 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:11.386 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:09:11.386 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:11.386 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:11.386 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:11.387 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:11.387 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:11.387 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:09:11.387 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:11.387 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.388 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:11.388 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:09:11.388 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:09:11.388 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:09:11.388 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:09:11.388 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:09:11.389 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:09:11.389 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:09:11.389 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:09:11.389 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:09:11.389 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:09:11.390 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:09:11.390 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:09:11.390 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:09:11.390 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:09:11.390 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:11.391 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:09:11.391 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:09:11.391 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:09:11.391 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:09:11.391 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:09:11.392 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:09:11.392 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:09:11.392 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:09:11.392 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:09:11.392 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:09:11.392 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:09:11.393 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:11.393 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:09:11.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.208+0000 7fe372f25640 1 -- 172.21.15.40:0/3204066809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe35006ae40 msgr2=0x7fe35006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:11.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.208+0000 7fe372f25640 1 --2- 172.21.15.40:0/3204066809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe35006ae40 0x7fe35006d300 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe358005fd0 tx=0x7fe358005ce0 comp rx=0 tx=0).stop 2023-10-18T09:09:11.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.208+0000 7fe372f25640 1 -- 172.21.15.40:0/3204066809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe36c06b710 msgr2=0x7fe36c14ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:11.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.208+0000 7fe372f25640 1 --2- 172.21.15.40:0/3204066809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe36c06b710 0x7fe36c14ef40 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fe36000b3c0 tx=0x7fe360009550 comp rx=0 tx=0).stop 2023-10-18T09:09:11.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.208+0000 7fe372f25640 1 -- 172.21.15.40:0/3204066809 shutdown_connections 2023-10-18T09:09:11.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.208+0000 7fe372f25640 1 --2- 172.21.15.40:0/3204066809 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fe35006ae40 0x7fe35006d300 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:11.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.208+0000 7fe372f25640 1 --2- 172.21.15.40:0/3204066809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe36c151bd0 0x7fe36c1e38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:11.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.208+0000 7fe372f25640 1 --2- 172.21.15.40:0/3204066809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe36c0b0bc0 0x7fe36c0cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:11.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.208+0000 7fe372f25640 1 --2- 172.21.15.40:0/3204066809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe36c06b710 0x7fe36c14ef40 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:11.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.208+0000 7fe372f25640 1 -- 172.21.15.40:0/3204066809 >> 172.21.15.40:0/3204066809 conn(0x7fe36c06cee0 msgr2=0x7fe36c150b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:11.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.208+0000 7fe372f25640 1 -- 172.21.15.40:0/3204066809 shutdown_connections 2023-10-18T09:09:11.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:11.208+0000 7fe372f25640 1 -- 172.21.15.40:0/3204066809 wait complete. 2023-10-18T09:09:11.397 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-18T09:09:16.398 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:09:16.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.638+0000 7fd193d4c640 1 -- 172.21.15.40:0/1695132796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd18c0b0bc0 msgr2=0x7fd18c0b1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:16.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.638+0000 7fd193d4c640 1 --2- 172.21.15.40:0/1695132796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd18c0b0bc0 0x7fd18c0b1040 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fd17c005fd0 tx=0x7fd17c02bd80 comp rx=0 tx=0).stop 2023-10-18T09:09:16.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.638+0000 7fd193d4c640 1 -- 172.21.15.40:0/1695132796 shutdown_connections 2023-10-18T09:09:16.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.638+0000 7fd193d4c640 1 --2- 172.21.15.40:0/1695132796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd18c0b0bc0 0x7fd18c0b1040 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.638+0000 7fd193d4c640 1 --2- 172.21.15.40:0/1695132796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd18c151bd0 0x7fd18c1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.638+0000 7fd193d4c640 1 -- 172.21.15.40:0/1695132796 >> 172.21.15.40:0/1695132796 conn(0x7fd18c06cee0 msgr2=0x7fd18c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.638+0000 7fd193d4c640 1 -- 172.21.15.40:0/1695132796 shutdown_connections 2023-10-18T09:09:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.638+0000 7fd193d4c640 1 -- 172.21.15.40:0/1695132796 wait complete. 2023-10-18T09:09:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.641+0000 7fd193d4c640 1 Processor -- start 2023-10-18T09:09:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.641+0000 7fd193d4c640 1 -- start start 2023-10-18T09:09:16.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.642+0000 7fd193d4c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd18c06b710 0x7fd18c14c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:16.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.642+0000 7fd193d4c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd18c0b0bc0 0x7fd18c14fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:16.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.642+0000 7fd193d4c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd18c151bd0 0x7fd18c142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:16.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.642+0000 7fd193d4c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c143c70 con 0x7fd18c06b710 2023-10-18T09:09:16.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.642+0000 7fd193d4c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c14f360 con 0x7fd18c0b0bc0 2023-10-18T09:09:16.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.642+0000 7fd193d4c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c0ae4c0 con 0x7fd18c151bd0 2023-10-18T09:09:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.642+0000 7fd191ae8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd18c06b710 0x7fd18c14c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.642+0000 7fd191ae8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd18c06b710 0x7fd18c14c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58350/0 (socket says 172.21.15.40:58350) 2023-10-18T09:09:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.642+0000 7fd191ae8640 1 -- 172.21.15.40:0/3100996112 learned_addr learned my addr 172.21.15.40:0/3100996112 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:16.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.643+0000 7fd1912e7640 1 --2- 172.21.15.40:0/3100996112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd18c0b0bc0 0x7fd18c14fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:16.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.643+0000 7fd191ae8640 1 -- 172.21.15.40:0/3100996112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd18c151bd0 msgr2=0x7fd18c142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:16.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.643+0000 7fd1922e9640 1 --2- 172.21.15.40:0/3100996112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd18c151bd0 0x7fd18c142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:16.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.643+0000 7fd191ae8640 1 --2- 172.21.15.40:0/3100996112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd18c151bd0 0x7fd18c142ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:16.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.643+0000 7fd191ae8640 1 -- 172.21.15.40:0/3100996112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd18c0b0bc0 msgr2=0x7fd18c14fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:16.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.643+0000 7fd191ae8640 1 --2- 172.21.15.40:0/3100996112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd18c0b0bc0 0x7fd18c14fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:16.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.643+0000 7fd191ae8640 1 -- 172.21.15.40:0/3100996112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c009560 con 0x7fd18c06b710 2023-10-18T09:09:16.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.643+0000 7fd1922e9640 1 --2- 172.21.15.40:0/3100996112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd18c151bd0 0x7fd18c142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:16.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.643+0000 7fd191ae8640 1 --2- 172.21.15.40:0/3100996112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd18c06b710 0x7fd18c14c780 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fd18800c9a0 tx=0x7fd18800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:16.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.644+0000 7fd1827fc640 1 -- 172.21.15.40:0/3100996112 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd188007bb0 con 0x7fd18c06b710 2023-10-18T09:09:16.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.644+0000 7fd1827fc640 1 -- 172.21.15.40:0/3100996112 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd188007e00 con 0x7fd18c06b710 2023-10-18T09:09:16.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.644+0000 7fd1827fc640 1 -- 172.21.15.40:0/3100996112 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18800f6b0 con 0x7fd18c06b710 2023-10-18T09:09:16.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.644+0000 7fd193d4c640 1 -- 172.21.15.40:0/3100996112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18c14eed0 con 0x7fd18c06b710 2023-10-18T09:09:16.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.644+0000 7fd193d4c640 1 -- 172.21.15.40:0/3100996112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18c0ae710 con 0x7fd18c06b710 2023-10-18T09:09:16.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.645+0000 7fd193d4c640 1 -- 172.21.15.40:0/3100996112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18c066fd0 con 0x7fd18c06b710 2023-10-18T09:09:16.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.646+0000 7fd1827fc640 1 -- 172.21.15.40:0/3100996112 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd18800f810 con 0x7fd18c06b710 2023-10-18T09:09:16.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.647+0000 7fd1827fc640 1 --2- 172.21.15.40:0/3100996112 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd17006ae40 0x7fd17006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:16.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.647+0000 7fd1827fc640 1 -- 172.21.15.40:0/3100996112 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd188089e00 con 0x7fd18c06b710 2023-10-18T09:09:16.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.647+0000 7fd1827fc640 1 -- 172.21.15.40:0/3100996112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=199}) v3 -- 0x7fd170072970 con 0x7fd18c06b710 2023-10-18T09:09:16.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.648+0000 7fd1912e7640 1 --2- 172.21.15.40:0/3100996112 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd17006ae40 0x7fd17006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:16.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.649+0000 7fd1912e7640 1 --2- 172.21.15.40:0/3100996112 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd17006ae40 0x7fd17006d300 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd17c035000 tx=0x7fd17c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:16.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.652+0000 7fd1827fc640 1 -- 172.21.15.40:0/3100996112 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd18801d020 con 0x7fd18c06b710 2023-10-18T09:09:16.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.852+0000 7fd193d4c640 1 -- 172.21.15.40:0/3100996112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd18c0cdb60 con 0x7fd18c06b710 2023-10-18T09:09:16.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.855+0000 7fd1827fc640 1 -- 172.21.15.40:0/3100996112 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v198) v1 ==== 81+0+34045 (secure 0 0 0) 0x7fd188054ea0 con 0x7fd18c06b710 2023-10-18T09:09:16.858 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:16.859 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:09:16.859 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 198, 2023-10-18T09:09:16.859 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:09:16.859 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:09:16.859 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:09:16.215303+0000", 2023-10-18T09:09:16.860 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:09:16.860 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:09:16.860 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:09:16.860 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:09:16.860 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:09:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:09:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:09:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:09:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:09:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:09:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:09:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:09:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:09:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:09:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 23, 2023-10-18T09:09:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:09:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:09:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:09:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:09:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:09:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:09:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:09:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:09:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:09:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:09:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:09:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:09:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:09:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:09:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:09:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:09:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:09:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:09:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:09:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:09:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:09:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:09:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:09:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:09:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 22, 2023-10-18T09:09:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:09:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:08:55.675479+0000", 2023-10-18T09:09:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:09:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:09:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:09:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:09:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:09:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "195", 2023-10-18T09:09:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:09:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:16.888 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:16.888 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:16.888 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:16.888 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:16.888 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:09:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:09:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:09:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:09:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:09:16.893 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.893 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.893 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:16.893 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 2, 2023-10-18T09:09:16.893 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 2, 2023-10-18T09:09:16.893 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:09:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:09:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:09:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 23, 2023-10-18T09:09:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:09:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:08:56.761888+0000", 2023-10-18T09:09:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1027, 2023-10-18T09:09:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-18T09:09:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:09:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:09:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:09:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:09:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:09:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "198", 2023-10-18T09:09:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 33554432, 2023-10-18T09:09:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:16.905 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:16.905 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.905 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:16.905 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:16.905 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:09:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:09:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.908 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:16.908 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:09:16.908 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:09:16.908 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.3799999952316284, 2023-10-18T09:09:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.3799999952316284, 2023-10-18T09:09:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:09:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-18T09:09:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-18T09:09:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:09:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:09:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:16.912 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:16.912 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:16.912 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:16.912 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:16.912 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:16.912 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:09:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.915 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:09:16.915 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:16.915 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.915 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.915 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.915 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:16.916 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.916 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.916 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.916 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:09:16.916 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:16.916 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.917 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.917 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.917 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:09:16.917 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:16.917 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.917 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.918 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.918 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:16.918 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.918 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.918 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.919 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:09:16.919 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:16.919 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.919 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.919 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.920 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:09:16.920 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:16.920 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.920 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.920 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.920 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:16.921 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.921 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.921 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.921 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:09:16.921 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:16.922 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.922 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.922 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.922 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:09:16.922 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:16.922 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.923 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.923 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.923 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:09:16.923 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:09:16.923 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:09:16.924 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:09:16.924 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:16.924 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:16.924 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:16.924 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.924 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.925 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.925 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:09:16.925 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:16.925 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:16.925 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:16.925 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:16.926 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:16.926 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:16.926 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:16.926 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:16.926 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:16.927 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:16.927 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:16.927 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:16.927 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.927 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.928 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:09:16.928 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:16.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.928 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.928 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.929 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:09:16.929 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:16.929 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.929 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.929 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.929 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:16.930 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.930 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.930 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.930 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:09:16.930 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:16.930 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.931 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.931 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.931 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:09:16.931 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:16.931 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.931 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.932 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.932 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:16.932 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.932 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.932 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.933 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:09:16.933 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:16.933 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.933 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.933 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.933 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:09:16.934 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:16.934 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.934 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.934 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.934 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:16.934 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.935 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.935 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.935 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:09:16.935 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:16.935 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.935 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.936 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.936 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:09:16.936 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:16.936 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.936 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.936 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.937 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:09:16.937 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:09:16.937 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:09:16.937 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:09:16.937 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:16.938 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:16.938 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:16.938 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.938 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.938 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.939 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:09:16.939 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:16.939 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:16.939 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:16.939 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:16.941 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:16.941 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:16.941 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:16.941 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:16.942 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:16.942 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:16.942 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:16.942 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:16.942 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.943 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.943 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.943 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:09:16.943 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:16.943 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.944 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.944 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.944 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:09:16.944 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:16.944 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.944 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.945 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.945 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:16.945 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.945 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.945 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.946 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:09:16.946 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:16.946 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.946 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.946 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.946 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:09:16.947 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:16.947 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.947 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.947 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.947 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:16.948 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.948 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.948 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.948 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:09:16.948 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:16.948 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.949 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.949 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.949 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:09:16.949 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:16.949 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.949 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.950 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.950 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:16.950 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.950 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.950 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.951 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:09:16.951 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:16.951 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.951 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.951 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.951 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:09:16.952 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:16.952 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.952 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.952 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.952 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:09:16.953 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:09:16.953 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:09:16.953 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:09:16.953 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:16.953 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:16.953 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:16.954 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.954 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.954 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.954 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:09:16.954 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:16.954 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:16.955 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:16.955 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:16.955 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:16.955 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:16.955 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:16.955 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:16.956 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:16.956 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:16.956 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:16.956 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:16.956 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.956 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.957 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.957 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:09:16.957 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:16.957 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.957 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.957 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.958 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:09:16.958 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:16.958 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.958 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.958 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.959 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:16.959 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.959 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.959 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.959 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:09:16.959 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:16.960 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.960 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.960 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.960 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:09:16.960 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:16.961 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.961 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.961 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.961 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:16.961 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.961 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.962 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.962 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:09:16.962 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:16.962 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.962 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.962 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.963 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:09:16.963 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:16.963 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.963 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.963 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.963 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:16.964 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.964 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.964 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.964 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:09:16.964 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:16.964 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.965 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.965 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.965 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:09:16.965 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:16.965 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.965 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:09:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:09:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:09:16.966 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:09:16.967 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:16.967 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:16.967 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:16.967 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.967 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.967 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.968 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:09:16.968 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:16.968 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:16.968 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:16.968 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:16.968 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:16.969 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:16.969 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:16.969 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:16.969 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:16.969 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:16.970 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:16.970 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:16.970 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.970 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.970 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.970 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:09:16.971 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:16.971 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.971 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.971 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:09:16.971 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:16.972 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.972 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.972 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.972 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:16.972 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.973 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.973 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.973 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:09:16.973 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:16.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.973 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.974 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.974 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:09:16.974 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:16.974 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.974 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.975 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.975 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:16.975 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.975 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.975 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.975 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:09:16.976 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:16.976 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.976 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.976 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.976 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:09:16.976 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:16.977 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.977 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.977 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.977 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:16.977 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.977 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.978 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.978 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:09:16.978 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:16.978 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.978 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.979 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.979 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:09:16.979 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:16.979 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.979 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.979 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.980 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:09:16.980 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:09:16.980 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:09:16.980 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:09:16.980 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:16.980 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:16.981 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:16.981 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.981 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.981 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.981 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:09:16.982 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:16.982 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:16.982 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:16.982 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:16.982 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:16.982 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:16.983 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:16.983 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:16.983 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:16.983 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:16.983 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:16.983 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:16.984 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.984 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.984 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.984 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:09:16.984 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:16.984 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.985 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.985 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.985 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:09:16.985 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:16.985 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.986 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.986 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.986 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:16.986 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.986 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.986 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.987 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:09:16.987 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:16.987 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.987 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.987 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.988 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:09:16.988 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:16.988 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.988 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.988 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.988 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:16.989 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.989 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.989 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.989 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:09:16.989 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:16.990 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.990 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.990 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.990 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:09:16.990 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:16.991 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.991 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.991 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.991 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:16.991 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.991 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.992 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.992 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:09:16.992 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:16.992 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.992 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.992 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:09:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.993 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.994 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:09:16.994 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:09:16.994 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:09:16.994 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:09:16.994 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:16.995 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:16.995 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:16.995 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:16.995 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.995 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.995 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:09:16.996 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:16.996 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:16.996 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:16.996 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:16.996 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:16.996 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:16.997 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:16.997 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:16.997 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:16.997 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:16.997 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:16.998 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:16.998 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:16.998 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.998 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:16.998 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:09:16.999 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:16.999 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:16.999 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:16.999 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:16.999 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:09:17.000 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:17.000 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:17.000 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:17.000 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.000 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:17.000 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:17.001 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.001 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:17.001 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:09:17.001 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:17.001 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.001 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.002 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:17.002 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:09:17.002 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:17.002 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:17.002 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:17.003 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.003 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:17.003 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:17.003 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.003 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:17.003 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:09:17.004 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:17.004 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.004 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.004 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:17.004 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:09:17.004 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:17.005 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:17.005 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:17.005 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.005 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:17.005 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:17.005 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.006 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:17.006 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:09:17.006 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:17.006 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.006 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.006 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:17.007 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:09:17.007 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:17.007 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:17.007 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:17.007 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.007 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:09:17.008 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:09:17.008 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:09:17.008 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:09:17.008 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:17.008 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:17.008 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:17.009 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:17.009 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.009 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.009 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:09:17.009 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:17.010 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:17.010 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:17.010 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:17.010 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:17.010 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:17.011 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:17.011 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:17.011 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:17.011 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:17.011 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:17.011 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:17.012 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:17.012 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.012 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:17.012 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:09:17.013 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:17.013 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.013 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.013 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:17.013 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:09:17.014 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:17.014 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:17.014 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:17.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.014 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:17.015 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:17.015 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.015 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:17.015 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:09:17.015 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:17.016 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.016 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.016 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:17.016 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:09:17.016 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:17.016 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:17.016 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:17.017 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.017 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:17.017 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:17.017 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.017 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:17.018 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:09:17.018 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:17.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.018 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.018 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:17.019 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:09:17.019 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:17.019 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:17.019 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:17.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.020 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:17.020 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:17.020 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.020 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:17.020 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:09:17.020 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:17.021 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.021 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.021 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:17.021 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:09:17.021 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:17.022 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:17.022 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:17.022 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.022 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:09:17.022 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:09:17.023 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:09:17.023 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:09:17.023 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:17.023 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:17.023 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:17.023 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:17.024 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:17.024 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:17.024 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:09:17.024 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.024 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:09:17.025 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:17.025 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:17.025 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:17.025 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:17.025 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:17.026 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:09:17.026 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:17.026 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.026 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.026 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:09:17.027 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:17.027 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:17.027 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:17.027 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:17.027 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:17.028 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:09:17.028 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:17.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.028 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.028 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:09:17.028 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:17.029 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:17.029 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:17.029 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:17.029 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:17.029 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:09:17.030 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:17.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.030 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.030 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:09:17.030 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:17.031 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:17.031 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:17.031 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:17.031 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:17.031 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:09:17.032 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:17.032 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.032 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.032 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:09:17.032 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:17.033 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:17.033 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:17.033 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:17.033 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:17.033 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:09:17.033 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:17.034 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.034 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.034 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:09:17.034 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:17.034 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:17.034 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:17.035 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:17.035 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:17.035 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:09:17.035 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:17.035 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.036 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.036 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:09:17.036 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:17.036 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:17.036 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:17.036 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:17.037 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:17.037 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:09:17.037 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:17.037 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.037 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:17.038 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:09:17.038 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:17.038 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:17.038 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:17.038 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:17.039 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:17.039 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:09:17.039 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:17.039 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:17.039 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:17.039 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:09:17.040 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:09:17.040 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:09:17.040 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:09:17.040 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:09:17.040 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:09:17.041 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:09:17.041 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:09:17.041 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:09:17.041 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:09:17.041 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:09:17.041 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:09:17.042 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:09:17.042 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:09:17.042 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:17.042 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:17.042 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:09:17.043 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:09:17.043 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:09:17.043 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:09:17.043 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:09:17.043 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:09:17.043 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:09:17.044 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:09:17.044 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:09:17.044 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:09:17.044 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:09:17.044 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:17.044 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:09:17.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.858+0000 7fd193d4c640 1 -- 172.21.15.40:0/3100996112 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd17006ae40 msgr2=0x7fd17006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:17.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.858+0000 7fd193d4c640 1 --2- 172.21.15.40:0/3100996112 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd17006ae40 0x7fd17006d300 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd17c035000 tx=0x7fd17c004730 comp rx=0 tx=0).stop 2023-10-18T09:09:17.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.858+0000 7fd193d4c640 1 -- 172.21.15.40:0/3100996112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd18c06b710 msgr2=0x7fd18c14c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:17.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.858+0000 7fd193d4c640 1 --2- 172.21.15.40:0/3100996112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd18c06b710 0x7fd18c14c780 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fd18800c9a0 tx=0x7fd18800ce60 comp rx=0 tx=0).stop 2023-10-18T09:09:17.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.858+0000 7fd193d4c640 1 -- 172.21.15.40:0/3100996112 shutdown_connections 2023-10-18T09:09:17.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.858+0000 7fd193d4c640 1 --2- 172.21.15.40:0/3100996112 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd17006ae40 0x7fd17006d300 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:17.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.858+0000 7fd193d4c640 1 --2- 172.21.15.40:0/3100996112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd18c151bd0 0x7fd18c142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:17.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.858+0000 7fd193d4c640 1 --2- 172.21.15.40:0/3100996112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd18c0b0bc0 0x7fd18c14fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:17.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.858+0000 7fd193d4c640 1 --2- 172.21.15.40:0/3100996112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd18c06b710 0x7fd18c14c780 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:17.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.858+0000 7fd193d4c640 1 -- 172.21.15.40:0/3100996112 >> 172.21.15.40:0/3100996112 conn(0x7fd18c06cee0 msgr2=0x7fd18c15a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:17.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.859+0000 7fd193d4c640 1 -- 172.21.15.40:0/3100996112 shutdown_connections 2023-10-18T09:09:17.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:16.859+0000 7fd193d4c640 1 -- 172.21.15.40:0/3100996112 wait complete. 2023-10-18T09:09:17.049 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-18T09:09:17.049 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:09:17.049 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:09:17.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:17.113 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2023-10-18T09:09:17.587 INFO:teuthology.orchestra.run.smithi040.stderr:dd: fdatasync failed for 'large_file_b': No space left on device 2023-10-18T09:09:17.588 INFO:teuthology.orchestra.run.smithi040.stderr:50+0 records in 2023-10-18T09:09:17.588 INFO:teuthology.orchestra.run.smithi040.stderr:50+0 records out 2023-10-18T09:09:17.588 INFO:teuthology.orchestra.run.smithi040.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.406614 s, 129 MB/s 2023-10-18T09:09:17.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:09:17.589 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:09:17.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.800+0000 7f356bb97640 1 -- 172.21.15.40:0/984877092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35500047c0 msgr2=0x7f3550004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:17.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.800+0000 7f356bb97640 1 --2- 172.21.15.40:0/984877092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35500047c0 0x7f3550004c40 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f3554006f00 tx=0x7f35500057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:17.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.801+0000 7f356a134640 1 -- 172.21.15.40:0/984877092 reap_dead start 2023-10-18T09:09:17.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.801+0000 7f356bb97640 1 -- 172.21.15.40:0/984877092 shutdown_connections 2023-10-18T09:09:17.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.801+0000 7f356bb97640 1 -- 172.21.15.40:0/984877092 >> 172.21.15.40:0/984877092 conn(0x7f35640655f0 msgr2=0x7f35640659d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:17.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.801+0000 7f356bb97640 1 -- 172.21.15.40:0/984877092 shutdown_connections 2023-10-18T09:09:17.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.801+0000 7f356bb97640 1 -- 172.21.15.40:0/984877092 wait complete. 2023-10-18T09:09:17.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.803+0000 7f356bb97640 1 Processor -- start 2023-10-18T09:09:17.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.803+0000 7f356bb97640 1 -- start start 2023-10-18T09:09:17.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.804+0000 7f356bb97640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35500047c0 0x7f3564140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:17.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.804+0000 7f356bb97640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3564155970 0x7f35640c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:17.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.804+0000 7f356bb97640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35640c99c0 0x7f35640c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:17.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.804+0000 7f356bb97640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3564156230 con 0x7f35640c99c0 2023-10-18T09:09:17.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.804+0000 7f356bb97640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35640b21a0 con 0x7f35500047c0 2023-10-18T09:09:17.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.804+0000 7f356bb97640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35640b1be0 con 0x7f3564155970 2023-10-18T09:09:17.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.804+0000 7f356a134640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35640c99c0 0x7f35640c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:17.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.804+0000 7f356a134640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35640c99c0 0x7f35640c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58380/0 (socket says 172.21.15.40:58380) 2023-10-18T09:09:17.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.804+0000 7f356a134640 1 -- 172.21.15.40:0/641598122 learned_addr learned my addr 172.21.15.40:0/641598122 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:17.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.804+0000 7f3569933640 1 --2- 172.21.15.40:0/641598122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35500047c0 0x7f3564140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:17.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.804+0000 7f356a134640 1 -- 172.21.15.40:0/641598122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3564155970 msgr2=0x7f35640c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:17.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.804+0000 7f356a134640 1 --2- 172.21.15.40:0/641598122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3564155970 0x7f35640c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:17.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.804+0000 7f356a134640 1 -- 172.21.15.40:0/641598122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35500047c0 msgr2=0x7f3564140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:17.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.804+0000 7f356a134640 1 --2- 172.21.15.40:0/641598122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35500047c0 0x7f3564140fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:17.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.804+0000 7f356a134640 1 -- 172.21.15.40:0/641598122 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35640b1ec0 con 0x7f35640c99c0 2023-10-18T09:09:17.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.805+0000 7f356a134640 1 --2- 172.21.15.40:0/641598122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35640c99c0 0x7f35640c9e50 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f3550004740 tx=0x7f355c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:17.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.805+0000 7f355a7fc640 1 -- 172.21.15.40:0/641598122 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c004220 con 0x7f35640c99c0 2023-10-18T09:09:17.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.805+0000 7f355a7fc640 1 -- 172.21.15.40:0/641598122 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f355c004470 con 0x7f35640c99c0 2023-10-18T09:09:17.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.805+0000 7f356bb97640 1 -- 172.21.15.40:0/641598122 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3564141990 con 0x7f35640c99c0 2023-10-18T09:09:17.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.805+0000 7f355a7fc640 1 -- 172.21.15.40:0/641598122 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c005260 con 0x7f35640c99c0 2023-10-18T09:09:17.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.805+0000 7f356bb97640 1 -- 172.21.15.40:0/641598122 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3564141510 con 0x7f35640c99c0 2023-10-18T09:09:17.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.806+0000 7f356bb97640 1 -- 172.21.15.40:0/641598122 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35640663a0 con 0x7f35640c99c0 2023-10-18T09:09:17.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.807+0000 7f355a7fc640 1 -- 172.21.15.40:0/641598122 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f355c0053c0 con 0x7f35640c99c0 2023-10-18T09:09:17.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.808+0000 7f355a7fc640 1 --2- 172.21.15.40:0/641598122 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f354806ae40 0x7f354806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:17.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.808+0000 7f355a7fc640 1 -- 172.21.15.40:0/641598122 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f355c014070 con 0x7f35640c99c0 2023-10-18T09:09:17.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.809+0000 7f355a7fc640 1 -- 172.21.15.40:0/641598122 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=199}) v3 -- 0x7f3548072970 con 0x7f35640c99c0 2023-10-18T09:09:17.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.809+0000 7f3569933640 1 --2- 172.21.15.40:0/641598122 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f354806ae40 0x7f354806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:17.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.810+0000 7f3569933640 1 --2- 172.21.15.40:0/641598122 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f354806ae40 0x7f354806d300 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f35640af020 tx=0x7f3560009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:17.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:17.812+0000 7f355a7fc640 1 -- 172.21.15.40:0/641598122 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f355c01c060 con 0x7f35640c99c0 2023-10-18T09:09:18.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.008+0000 7f356bb97640 1 -- 172.21.15.40:0/641598122 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f3564141eb0 con 0x7f35640c99c0 2023-10-18T09:09:18.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.011+0000 7f355a7fc640 1 -- 172.21.15.40:0/641598122 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v198) v1 ==== 81+0+34045 (secure 0 0 0) 0x7f3564141eb0 con 0x7f35640c99c0 2023-10-18T09:09:18.014 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:18.014 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:09:18.014 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 198, 2023-10-18T09:09:18.014 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:09:18.014 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:09:18.015 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:09:16.215303+0000", 2023-10-18T09:09:18.015 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:09:18.015 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:09:18.015 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:09:18.015 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:09:18.016 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:09:18.016 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:09:18.016 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:09:18.016 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:09:18.016 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:09:18.016 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:18.017 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:09:18.017 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:09:18.017 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:09:18.017 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:09:18.017 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:09:18.018 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 23, 2023-10-18T09:09:18.018 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:09:18.018 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:09:18.018 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:09:18.018 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:09:18.018 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:09:18.019 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:09:18.019 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.019 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:09:18.019 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:09:18.019 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:09:18.020 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:18.020 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:18.020 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:18.020 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:18.020 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:18.021 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:18.021 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:18.021 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:18.021 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:18.021 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:18.022 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:18.022 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:18.022 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:09:18.022 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:09:18.022 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:09:18.023 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:09:18.023 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:09:18.023 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:18.023 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:18.023 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:18.024 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:18.024 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:18.024 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:18.024 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:18.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.025 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:09:18.025 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:18.025 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:18.025 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:18.025 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:18.025 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:18.026 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:18.026 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:18.026 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:18.026 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:18.026 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:09:18.027 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:18.027 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:18.027 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:18.027 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:18.027 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:18.028 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:18.028 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:18.028 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:18.028 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:18.028 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:18.028 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:18.029 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:18.029 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:18.029 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:18.029 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:18.029 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:18.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.030 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:18.030 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:18.030 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:18.030 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:18.031 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:18.031 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:18.031 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:18.031 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:18.031 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:18.032 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:18.032 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:18.032 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:18.032 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:09:18.032 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:09:18.032 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.033 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:18.033 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:09:18.033 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.033 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:18.033 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:09:18.034 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:09:18.034 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:09:18.034 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:09:18.034 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:09:18.034 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:18.034 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:18.035 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:18.035 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.035 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.035 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.035 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 22, 2023-10-18T09:09:18.036 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:09:18.036 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:08:55.675479+0000", 2023-10-18T09:09:18.036 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:18.036 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:18.036 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:18.036 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:18.037 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:18.037 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:18.037 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:18.037 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:18.037 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:18.037 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:18.038 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:18.038 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:18.038 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:09:18.038 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:09:18.038 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:09:18.039 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:09:18.039 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:09:18.039 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:18.039 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:18.039 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:18.040 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:18.040 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:18.040 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:18.040 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:18.040 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.040 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "195", 2023-10-18T09:09:18.041 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:18.041 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:18.041 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:18.041 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:18.041 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:18.041 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:18.042 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:18.042 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:18.042 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:18.042 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:09:18.042 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:18.043 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:18.043 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:18.043 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:18.043 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:18.043 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:18.044 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:18.044 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:18.044 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:18.044 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:18.044 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:18.045 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:18.045 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:18.045 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:18.045 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:18.045 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:18.046 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.046 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:18.046 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:18.046 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:18.046 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:18.047 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:18.047 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:18.047 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:18.047 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:18.047 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:18.048 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:18.048 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:18.048 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:18.048 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:09:18.048 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:09:18.048 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:09:18.049 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.049 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:18.049 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:09:18.049 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:09:18.049 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.050 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:18.050 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 2, 2023-10-18T09:09:18.050 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 2, 2023-10-18T09:09:18.050 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:09:18.051 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:09:18.051 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:09:18.051 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:18.051 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:18.051 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:18.051 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.052 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.052 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 23, 2023-10-18T09:09:18.052 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:09:18.052 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:08:56.761888+0000", 2023-10-18T09:09:18.053 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1027, 2023-10-18T09:09:18.053 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-18T09:09:18.053 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:18.053 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:18.053 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:18.053 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:18.054 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:18.054 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:18.054 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:18.054 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:18.054 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:18.055 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:18.055 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:09:18.055 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:09:18.055 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:09:18.055 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:09:18.055 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:09:18.056 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:18.056 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:18.056 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:18.056 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:18.056 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:18.057 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:18.057 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:18.057 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.057 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "198", 2023-10-18T09:09:18.057 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:18.058 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:18.058 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:18.058 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:18.058 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:18.058 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:18.058 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:18.059 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:18.059 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:18.059 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 33554432, 2023-10-18T09:09:18.059 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:18.059 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:18.060 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:18.060 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:18.060 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:18.060 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:18.060 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:18.060 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:18.061 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:18.061 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:18.061 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:18.061 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:18.061 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:18.061 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:18.062 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:18.062 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:18.062 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.062 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:18.062 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:18.062 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:18.063 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:18.063 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:18.063 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:18.063 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:18.063 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:18.064 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:18.064 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:18.064 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:18.064 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:18.064 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:09:18.064 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:09:18.065 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.065 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:18.065 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:09:18.065 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:09:18.065 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.065 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.066 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:18.066 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.3799999952316284, 2023-10-18T09:09:18.066 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.3799999952316284, 2023-10-18T09:09:18.066 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:09:18.066 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-18T09:09:18.067 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-18T09:09:18.067 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:18.067 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:18.067 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:18.067 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.067 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.068 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:18.068 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:09:18.068 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.068 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:09:18.068 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:18.068 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:18.069 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:18.069 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:18.069 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:18.069 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:18.069 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:09:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:09:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:09:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:09:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:09:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:09:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:09:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:09:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:09:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:09:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:09:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:09:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:09:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:09:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:09:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.087 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.087 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.087 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:18.087 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.087 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.087 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.088 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:09:18.088 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:18.088 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.088 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.088 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.088 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:09:18.089 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:18.089 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.089 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.089 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.089 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:18.089 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.090 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.090 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.090 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:09:18.090 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:18.090 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.091 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.091 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.091 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:09:18.091 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:18.091 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.092 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.092 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.092 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:18.092 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.092 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.092 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.093 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:09:18.093 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:18.093 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.093 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.093 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.094 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:09:18.094 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:18.094 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.094 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.094 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:09:18.095 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:09:18.095 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:09:18.095 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:09:18.095 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:18.095 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:18.096 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:18.096 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.096 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:09:18.096 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:18.097 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:18.097 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:18.097 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:18.097 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:18.097 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:18.097 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:18.098 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:18.098 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:18.098 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:18.098 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:18.098 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:18.098 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.099 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.099 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.099 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:09:18.099 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:18.099 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.100 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.100 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.100 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:09:18.100 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:18.100 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.101 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.101 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.101 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:18.101 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.101 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.102 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:09:18.102 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:18.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.102 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.103 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.103 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:09:18.103 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:18.103 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.103 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.104 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:18.104 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.104 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.104 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.104 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:09:18.105 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:18.105 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.105 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.105 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:09:18.105 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:18.105 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.106 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.106 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:18.106 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.106 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.107 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.107 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:09:18.107 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:18.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.107 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.108 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:09:18.108 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:18.108 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.108 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.109 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:09:18.109 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:09:18.109 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:09:18.109 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:09:18.109 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:18.109 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:18.110 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:18.110 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.110 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:09:18.111 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:18.111 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:18.111 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:18.111 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:18.111 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:18.112 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:18.112 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:18.112 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:18.112 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:18.112 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:18.112 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:18.113 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:18.113 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.113 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.113 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.113 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:09:18.113 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:18.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.114 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.114 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:09:18.114 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:18.114 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.115 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.115 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:18.115 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.115 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.116 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:09:18.116 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:18.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.116 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.116 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:09:18.117 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:18.117 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.117 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.117 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:18.117 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.118 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.118 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:09:18.118 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:18.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.119 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.119 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:09:18.119 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:18.119 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.119 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.120 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:18.120 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.120 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.120 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:09:18.121 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:18.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.121 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.121 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:09:18.122 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:18.122 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.122 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.122 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:09:18.122 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:09:18.123 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:09:18.123 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:09:18.123 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:18.123 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:18.123 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:18.123 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:09:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:09:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:09:18.128 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:18.128 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.128 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.128 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.128 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:09:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:09:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:09:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:09:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:09:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:09:18.136 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:18.136 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.136 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.136 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:09:18.136 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:09:18.137 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:09:18.137 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:09:18.137 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:18.137 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:18.137 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:18.137 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.138 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:09:18.138 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:18.138 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:18.139 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:18.139 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:18.139 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:18.139 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:18.139 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:18.140 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:18.140 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:18.140 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:18.140 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:18.140 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:18.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.141 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:09:18.141 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:18.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.142 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.142 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:09:18.142 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:18.143 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.143 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.143 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:18.143 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.144 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.144 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:09:18.144 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:18.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.144 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.145 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:09:18.145 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:18.145 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.145 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.145 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:18.146 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.146 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.146 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.146 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:09:18.146 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:18.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.147 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.147 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:09:18.147 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:18.147 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.147 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.148 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:18.148 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.148 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.149 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:09:18.149 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:18.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.149 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.150 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:09:18.150 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:18.150 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.150 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.150 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:09:18.151 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:09:18.151 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:09:18.151 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:09:18.151 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:18.151 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:18.151 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:18.152 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.152 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:09:18.152 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:18.153 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:18.153 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:18.153 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:18.153 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:18.153 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:18.154 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:18.154 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:18.154 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:18.154 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:18.154 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:18.154 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:18.155 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.155 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.155 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:09:18.156 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:18.156 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.156 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.156 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.156 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:09:18.156 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:18.157 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.157 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.157 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:18.157 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.157 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.158 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.158 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:09:18.158 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:18.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.159 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.159 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:09:18.159 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:18.159 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.159 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.160 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:18.160 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.160 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.160 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.160 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:09:18.161 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:18.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.161 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.161 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:09:18.162 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:18.162 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.162 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.162 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:18.162 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.163 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.163 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:09:18.163 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:18.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.164 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.164 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:09:18.164 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:18.164 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.164 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.165 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:09:18.165 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:09:18.165 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:09:18.165 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:09:18.166 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:18.166 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:18.166 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:18.166 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.166 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.166 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.167 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:09:18.167 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:18.167 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:18.167 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:18.167 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:18.168 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:18.168 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:18.168 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:18.168 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:18.168 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:18.169 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:18.169 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:18.169 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:18.169 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.169 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.170 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.170 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:09:18.170 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:18.170 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.171 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.171 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:09:18.171 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:18.171 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.171 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.171 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.172 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:18.172 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.172 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.172 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:09:18.172 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:18.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.173 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.173 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:09:18.173 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:18.173 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.174 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.174 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:18.174 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.174 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.174 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.175 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:09:18.175 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:18.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.175 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.175 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.175 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:09:18.176 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:18.176 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.176 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.176 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:18.176 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:18.177 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.177 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:18.177 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:09:18.177 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:18.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.177 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.178 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:18.178 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:09:18.178 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:18.178 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.178 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.178 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.179 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:09:18.179 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:09:18.179 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:09:18.179 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:09:18.179 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:18.179 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:18.180 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:18.180 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:18.180 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.180 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:18.180 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:09:18.180 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.181 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:09:18.181 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:18.181 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:18.181 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:18.181 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:18.181 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:18.182 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:09:18.182 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:18.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.182 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.182 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:09:18.182 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:18.183 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:18.183 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:18.183 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:18.183 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:18.183 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:09:18.183 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:09:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:09:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.186 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:09:18.186 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:18.186 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:18.186 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:18.186 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:18.186 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:18.187 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:09:18.187 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:18.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.187 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:09:18.188 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:18.188 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:18.188 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:18.188 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:18.188 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:09:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:09:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:09:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:09:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:09:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:09:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:18.194 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:18.194 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:18.194 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:18.194 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:09:18.194 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:18.194 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:09:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:09:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:09:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:09:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:09:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:09:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:09:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:09:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:09:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:09:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:09:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:09:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:09:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:09:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.198 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:18.198 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:09:18.198 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:09:18.198 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:09:18.198 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:09:18.199 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:09:18.199 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:09:18.199 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:09:18.199 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:09:18.199 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:09:18.200 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:09:18.200 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:09:18.200 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:18.200 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:09:18.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.013+0000 7f356bb97640 1 -- 172.21.15.40:0/641598122 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f354806ae40 msgr2=0x7f354806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:18.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.013+0000 7f356bb97640 1 --2- 172.21.15.40:0/641598122 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f354806ae40 0x7f354806d300 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f35640af020 tx=0x7f3560009040 comp rx=0 tx=0).stop 2023-10-18T09:09:18.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.013+0000 7f356bb97640 1 -- 172.21.15.40:0/641598122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35640c99c0 msgr2=0x7f35640c9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:18.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.013+0000 7f356bb97640 1 --2- 172.21.15.40:0/641598122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35640c99c0 0x7f35640c9e50 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f3550004740 tx=0x7f355c00eb50 comp rx=0 tx=0).stop 2023-10-18T09:09:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.013+0000 7f356bb97640 1 -- 172.21.15.40:0/641598122 shutdown_connections 2023-10-18T09:09:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.013+0000 7f356bb97640 1 --2- 172.21.15.40:0/641598122 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f354806ae40 0x7f354806d300 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.013+0000 7f356bb97640 1 --2- 172.21.15.40:0/641598122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35640c99c0 0x7f35640c9e50 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.013+0000 7f356bb97640 1 --2- 172.21.15.40:0/641598122 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3564155970 0x7f35640c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.013+0000 7f356bb97640 1 --2- 172.21.15.40:0/641598122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35500047c0 0x7f3564140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.014+0000 7f356bb97640 1 -- 172.21.15.40:0/641598122 >> 172.21.15.40:0/641598122 conn(0x7f35640655f0 msgr2=0x7f3564072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.014+0000 7f356bb97640 1 -- 172.21.15.40:0/641598122 shutdown_connections 2023-10-18T09:09:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.014+0000 7f356bb97640 1 -- 172.21.15.40:0/641598122 wait complete. 2023-10-18T09:09:18.205 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:09:18.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.438+0000 7f8215e94640 1 -- 172.21.15.40:0/2200379301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82101586a0 msgr2=0x7f82100cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:18.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.438+0000 7f8215e94640 1 --2- 172.21.15.40:0/2200379301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82101586a0 0x7f82100cc990 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f821006a6d0 tx=0x7f81f802bd70 comp rx=0 tx=0).stop 2023-10-18T09:09:18.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.438+0000 7f8215e94640 1 -- 172.21.15.40:0/2200379301 shutdown_connections 2023-10-18T09:09:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.438+0000 7f8215e94640 1 --2- 172.21.15.40:0/2200379301 >> v2:172.21.15.144:3300/0 conn(0x7f82100cced0 0x7f82100cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.438+0000 7f8215e94640 1 --2- 172.21.15.40:0/2200379301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82101586a0 0x7f82100cc990 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.438+0000 7f8215e94640 1 -- 172.21.15.40:0/2200379301 >> 172.21.15.40:0/2200379301 conn(0x7f821006cee0 msgr2=0x7f8210069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.439+0000 7f8215e94640 1 -- 172.21.15.40:0/2200379301 shutdown_connections 2023-10-18T09:09:18.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.439+0000 7f8215e94640 1 -- 172.21.15.40:0/2200379301 wait complete. 2023-10-18T09:09:18.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.441+0000 7f8215e94640 1 Processor -- start 2023-10-18T09:09:18.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.441+0000 7f8215e94640 1 -- start start 2023-10-18T09:09:18.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.442+0000 7f8215e94640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f821006d710 0x7f82100cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:18.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.443+0000 7f8215e94640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82100cced0 0x7f82101e3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:18.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.443+0000 7f8215e94640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82101586a0 0x7f8210156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:18.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.443+0000 7f8215e94640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82101f21d0 con 0x7f82100cced0 2023-10-18T09:09:18.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.443+0000 7f8215e94640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82101f1b30 con 0x7f821006d710 2023-10-18T09:09:18.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.443+0000 7f8215e94640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82101e46e0 con 0x7f82101586a0 2023-10-18T09:09:18.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.443+0000 7f8206ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82100cced0 0x7f82101e3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:18.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.443+0000 7f8206ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82100cced0 0x7f82101e3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58398/0 (socket says 172.21.15.40:58398) 2023-10-18T09:09:18.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.443+0000 7f8206ffd640 1 -- 172.21.15.40:0/1391043314 learned_addr learned my addr 172.21.15.40:0/1391043314 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:18.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.443+0000 7f820ffff640 1 --2- 172.21.15.40:0/1391043314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82101586a0 0x7f8210156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:18.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.443+0000 7f820f7fe640 1 --2- 172.21.15.40:0/1391043314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f821006d710 0x7f82100cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:18.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.443+0000 7f8206ffd640 1 -- 172.21.15.40:0/1391043314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82101586a0 msgr2=0x7f8210156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:18.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.444+0000 7f8206ffd640 1 --2- 172.21.15.40:0/1391043314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82101586a0 0x7f8210156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:18.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.444+0000 7f8206ffd640 1 -- 172.21.15.40:0/1391043314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f821006d710 msgr2=0x7f82100cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:18.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.444+0000 7f8206ffd640 1 --2- 172.21.15.40:0/1391043314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f821006d710 0x7f82100cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:18.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.444+0000 7f8206ffd640 1 -- 172.21.15.40:0/1391043314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f8009490 con 0x7f82100cced0 2023-10-18T09:09:18.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.444+0000 7f820f7fe640 1 --2- 172.21.15.40:0/1391043314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f821006d710 0x7f82100cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:18.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.444+0000 7f8206ffd640 1 --2- 172.21.15.40:0/1391043314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82100cced0 0x7f82101e3fe0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f81f8000c00 tx=0x7f81f80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:18.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.444+0000 7f820cff9640 1 -- 172.21.15.40:0/1391043314 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8003ea0 con 0x7f82100cced0 2023-10-18T09:09:18.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.444+0000 7f820cff9640 1 -- 172.21.15.40:0/1391043314 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f81f8034550 con 0x7f82100cced0 2023-10-18T09:09:18.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.444+0000 7f820cff9640 1 -- 172.21.15.40:0/1391043314 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8034960 con 0x7f82100cced0 2023-10-18T09:09:18.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.444+0000 7f8215e94640 1 -- 172.21.15.40:0/1391043314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82101e49c0 con 0x7f82100cced0 2023-10-18T09:09:18.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.444+0000 7f8215e94640 1 -- 172.21.15.40:0/1391043314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8210156790 con 0x7f82100cced0 2023-10-18T09:09:18.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.446+0000 7f8215e94640 1 -- 172.21.15.40:0/1391043314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82101f2310 con 0x7f82100cced0 2023-10-18T09:09:18.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.447+0000 7f820cff9640 1 -- 172.21.15.40:0/1391043314 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f81f803b040 con 0x7f82100cced0 2023-10-18T09:09:18.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.447+0000 7f820cff9640 1 --2- 172.21.15.40:0/1391043314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f81f006ae40 0x7f81f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.448+0000 7f820cff9640 1 -- 172.21.15.40:0/1391043314 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81f80abbf0 con 0x7f82100cced0 2023-10-18T09:09:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.448+0000 7f820cff9640 1 -- 172.21.15.40:0/1391043314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=199}) v3 -- 0x7f81f0072970 con 0x7f82100cced0 2023-10-18T09:09:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.448+0000 7f820f7fe640 1 --2- 172.21.15.40:0/1391043314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f81f006ae40 0x7f81f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.449+0000 7f820f7fe640 1 --2- 172.21.15.40:0/1391043314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f81f006ae40 0x7f81f006d300 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f82000046b0 tx=0x7f8200009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:18.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.452+0000 7f820cff9640 1 -- 172.21.15.40:0/1391043314 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f81f8039060 con 0x7f82100cced0 2023-10-18T09:09:18.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.686+0000 7f8215e94640 1 -- 172.21.15.40:0/1391043314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8210156920 con 0x7f82100cced0 2023-10-18T09:09:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.687+0000 7f820cff9640 1 -- 172.21.15.40:0/1391043314 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f81f8041020 con 0x7f82100cced0 2023-10-18T09:09:18.689 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:18.690 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2038256442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2038256442},{"type":"v1","addr":"172.21.15.144:6836","nonce":2038256442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3006484507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3006484507},{"type":"v1","addr":"172.21.15.144:6837","nonce":3006484507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1542000032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1542000032},{"type":"v1","addr":"172.21.15.40:6834","nonce":1542000032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:08:57.731109+0000","modified":"2023-10-18T09:09:00.823289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3976883051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3976883051},{"type":"v1","addr":"172.21.15.40:6835","nonce":3976883051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-18T09:09:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 90 2023-10-18T09:09:18.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.689+0000 7f8215e94640 1 -- 172.21.15.40:0/1391043314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f81f006ae40 msgr2=0x7f81f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:18.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.689+0000 7f8215e94640 1 --2- 172.21.15.40:0/1391043314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f81f006ae40 0x7f81f006d300 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f82000046b0 tx=0x7f8200009040 comp rx=0 tx=0).stop 2023-10-18T09:09:18.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.689+0000 7f8215e94640 1 -- 172.21.15.40:0/1391043314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82100cced0 msgr2=0x7f82101e3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:18.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.689+0000 7f8215e94640 1 --2- 172.21.15.40:0/1391043314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82100cced0 0x7f82101e3fe0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f81f8000c00 tx=0x7f81f80038a0 comp rx=0 tx=0).stop 2023-10-18T09:09:18.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.689+0000 7f8215e94640 1 -- 172.21.15.40:0/1391043314 shutdown_connections 2023-10-18T09:09:18.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.689+0000 7f8215e94640 1 --2- 172.21.15.40:0/1391043314 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f81f006ae40 0x7f81f006d300 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:18.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.689+0000 7f8215e94640 1 --2- 172.21.15.40:0/1391043314 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82101586a0 0x7f8210156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:18.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.689+0000 7f8215e94640 1 --2- 172.21.15.40:0/1391043314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82100cced0 0x7f82101e3fe0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:18.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.689+0000 7f8215e94640 1 --2- 172.21.15.40:0/1391043314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f821006d710 0x7f82100cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:18.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.689+0000 7f8215e94640 1 -- 172.21.15.40:0/1391043314 >> 172.21.15.40:0/1391043314 conn(0x7f821006cee0 msgr2=0x7f82100ae530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:18.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.690+0000 7f8215e94640 1 -- 172.21.15.40:0/1391043314 shutdown_connections 2023-10-18T09:09:18.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:18.690+0000 7f8215e94640 1 -- 172.21.15.40:0/1391043314 wait complete. 2023-10-18T09:09:18.712 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-18T09:09:18.948 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":9282,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":20.663126394999999,"mdsmap_epoch":90,"osdmap_epoch":198,"osdmap_epoch_barrier":198,"uptime":27.404989219000001} 2023-10-18T09:09:18.948 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 9282, "mdsmap_epoch": 90, "osdmap_epoch": 198, "osdmap_epoch_barrier": 198, "rank_uptime": 20.663126395, "state": "up:active", "uptime": 27.404989219, "want_state": "up:active", "whoami": 0 } 2023-10-18T09:09:18.949 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-18T09:09:18.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:18.949 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-18T09:09:19.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:19.008 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-18T09:09:19.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:09:19.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.319+0000 7f40f95e9640 1 -- 172.21.15.40:0/205659905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f40ca0d0 msgr2=0x7f40f4157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:19.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.319+0000 7f40f95e9640 1 --2- 172.21.15.40:0/205659905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f40ca0d0 0x7f40f4157d40 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f40f406aeb0 tx=0x7f40ec02e520 comp rx=0 tx=0).stop 2023-10-18T09:09:19.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.320+0000 7f40f95e9640 1 -- 172.21.15.40:0/205659905 shutdown_connections 2023-10-18T09:09:19.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.320+0000 7f40f95e9640 1 --2- 172.21.15.40:0/205659905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f40ca0d0 0x7f40f4157d40 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:19.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.320+0000 7f40f95e9640 1 --2- 172.21.15.40:0/205659905 >> v2:172.21.15.144:3300/0 conn(0x7f40f406d6f0 0x7f40f40b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:19.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.320+0000 7f40f95e9640 1 -- 172.21.15.40:0/205659905 >> 172.21.15.40:0/205659905 conn(0x7f40f4069720 msgr2=0x7f40f4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:19.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.320+0000 7f40f95e9640 1 -- 172.21.15.40:0/205659905 shutdown_connections 2023-10-18T09:09:19.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.320+0000 7f40f95e9640 1 -- 172.21.15.40:0/205659905 wait complete. 2023-10-18T09:09:19.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.322+0000 7f40f95e9640 1 Processor -- start 2023-10-18T09:09:19.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.323+0000 7f40f95e9640 1 -- start start 2023-10-18T09:09:19.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.324+0000 7f40f95e9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f406d6f0 0x7f40f415a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:19.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.324+0000 7f40f95e9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40f40b0630 0x7f40f40aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:19.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.324+0000 7f40f95e9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40f40ca0d0 0x7f40f4153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:19.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.324+0000 7f40f95e9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f4159bc0 con 0x7f40f406d6f0 2023-10-18T09:09:19.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.324+0000 7f40f95e9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f41f1840 con 0x7f40f40ca0d0 2023-10-18T09:09:19.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.324+0000 7f40f95e9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40ab260 con 0x7f40f40b0630 2023-10-18T09:09:19.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.324+0000 7f40f3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f406d6f0 0x7f40f415a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:19.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.324+0000 7f40f3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f406d6f0 0x7f40f415a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58410/0 (socket says 172.21.15.40:58410) 2023-10-18T09:09:19.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.324+0000 7f40f3fff640 1 -- 172.21.15.40:0/2055691050 learned_addr learned my addr 172.21.15.40:0/2055691050 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:19.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.325+0000 7f40f37fe640 1 --2- 172.21.15.40:0/2055691050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40f40b0630 0x7f40f40aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:19.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.325+0000 7f40f8de8640 1 --2- 172.21.15.40:0/2055691050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40f40ca0d0 0x7f40f4153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:19.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.325+0000 7f40f3fff640 1 -- 172.21.15.40:0/2055691050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40f40b0630 msgr2=0x7f40f40aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:19.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.325+0000 7f40f3fff640 1 --2- 172.21.15.40:0/2055691050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40f40b0630 0x7f40f40aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:19.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.325+0000 7f40f3fff640 1 -- 172.21.15.40:0/2055691050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40f40ca0d0 msgr2=0x7f40f4153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:19.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.325+0000 7f40f3fff640 1 --2- 172.21.15.40:0/2055691050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40f40ca0d0 0x7f40f4153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:19.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.325+0000 7f40f3fff640 1 -- 172.21.15.40:0/2055691050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec00a950 con 0x7f40f406d6f0 2023-10-18T09:09:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.325+0000 7f40f3fff640 1 --2- 172.21.15.40:0/2055691050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f406d6f0 0x7f40f415a610 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f40e400d950 tx=0x7f40e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.326+0000 7f40f0ff9640 1 -- 172.21.15.40:0/2055691050 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4009c30 con 0x7f40f406d6f0 2023-10-18T09:09:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.326+0000 7f40f0ff9640 1 -- 172.21.15.40:0/2055691050 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f40e4009e80 con 0x7f40f406d6f0 2023-10-18T09:09:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.326+0000 7f40f95e9640 1 -- 172.21.15.40:0/2055691050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f40ab540 con 0x7f40f406d6f0 2023-10-18T09:09:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.326+0000 7f40f0ff9640 1 -- 172.21.15.40:0/2055691050 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e40106c0 con 0x7f40f406d6f0 2023-10-18T09:09:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.326+0000 7f40f95e9640 1 -- 172.21.15.40:0/2055691050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f415a0f0 con 0x7f40f406d6f0 2023-10-18T09:09:19.332 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.327+0000 7f40f95e9640 1 -- 172.21.15.40:0/2055691050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f415a2b0 con 0x7f40f406d6f0 2023-10-18T09:09:19.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.328+0000 7f40f0ff9640 1 -- 172.21.15.40:0/2055691050 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f40e4010820 con 0x7f40f406d6f0 2023-10-18T09:09:19.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.329+0000 7f40f0ff9640 1 --2- 172.21.15.40:0/2055691050 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f40dc06ae40 0x7f40dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:19.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.329+0000 7f40f0ff9640 1 -- 172.21.15.40:0/2055691050 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40e4089a70 con 0x7f40f406d6f0 2023-10-18T09:09:19.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.330+0000 7f40f0ff9640 1 -- 172.21.15.40:0/2055691050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=199}) v3 -- 0x7f40dc072970 con 0x7f40f406d6f0 2023-10-18T09:09:19.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.330+0000 7f40f37fe640 1 --2- 172.21.15.40:0/2055691050 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f40dc06ae40 0x7f40dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:19.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.331+0000 7f40f37fe640 1 --2- 172.21.15.40:0/2055691050 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f40dc06ae40 0x7f40dc06d300 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f40f41561b0 tx=0x7f40e0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.333+0000 7f40f0ff9640 1 -- 172.21.15.40:0/2055691050 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40e4017060 con 0x7f40f406d6f0 2023-10-18T09:09:19.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.528+0000 7f40f95e9640 1 -- 172.21.15.40:0/2055691050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f40f4066e00 con 0x7f40f406d6f0 2023-10-18T09:09:19.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.532+0000 7f40f0ff9640 1 -- 172.21.15.40:0/2055691050 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v198) v1 ==== 81+0+34045 (secure 0 0 0) 0x7f40e4017070 con 0x7f40f406d6f0 2023-10-18T09:09:19.535 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:19.535 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:09:19.535 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 198, 2023-10-18T09:09:19.535 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:09:19.535 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:09:19.536 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:09:16.215303+0000", 2023-10-18T09:09:19.536 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:09:19.536 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:09:19.536 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:09:19.537 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:09:19.537 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:09:19.537 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:09:19.537 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:09:19.537 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:09:19.537 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:09:19.538 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:19.538 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:09:19.538 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:09:19.538 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:09:19.538 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:09:19.539 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:09:19.539 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 23, 2023-10-18T09:09:19.539 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:09:19.539 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:09:19.539 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:09:19.540 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:09:19.540 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:09:19.540 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:09:19.540 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.540 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:09:19.541 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:09:19.541 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:09:19.541 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:19.541 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:19.541 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:19.541 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:19.542 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:19.542 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:19.542 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:19.542 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:19.542 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:19.543 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:19.543 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:19.543 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:19.543 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:09:19.543 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:09:19.544 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:09:19.544 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:09:19.544 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:09:19.544 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:19.544 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:19.544 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:19.545 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:19.545 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:19.545 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:19.545 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:19.545 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.546 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:09:19.546 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:19.546 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:19.546 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:19.546 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:19.547 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:19.547 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:19.547 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:19.547 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:19.547 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:19.547 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:09:19.548 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:19.548 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:19.548 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:19.548 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:19.548 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:19.549 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:19.549 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:19.549 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:19.549 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:19.549 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:19.550 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:19.550 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:19.550 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:19.550 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:19.550 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:19.551 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:19.551 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.551 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:19.551 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:19.551 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:19.551 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:19.552 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:19.552 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:19.552 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:19.552 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:19.552 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:19.553 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:19.553 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:19.553 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:19.553 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:09:19.553 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:09:19.554 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.554 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:19.554 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:09:19.554 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.554 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:19.555 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:09:19.555 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:09:19.555 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:09:19.555 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:09:19.555 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:09:19.555 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:19.556 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:19.556 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:19.556 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.556 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.557 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 22, 2023-10-18T09:09:19.557 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:09:19.557 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:08:55.675479+0000", 2023-10-18T09:09:19.557 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:19.557 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:19.558 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:19.558 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:19.558 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:19.558 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:19.558 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:19.559 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:19.559 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:19.559 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:19.559 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:19.559 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:19.559 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:09:19.560 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:09:19.560 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:09:19.560 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:09:19.560 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:09:19.561 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:19.561 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:19.561 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:19.561 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:19.561 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:19.561 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:19.562 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:19.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.562 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "195", 2023-10-18T09:09:19.562 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:19.562 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:19.563 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:19.563 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:19.563 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:19.563 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:19.563 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:19.563 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:19.564 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:19.564 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:09:19.564 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:19.564 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:19.564 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:19.565 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:19.565 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:19.565 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:19.565 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:19.565 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:19.565 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:19.566 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:19.566 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:19.566 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:19.566 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:19.566 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:19.567 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:19.567 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:19.567 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.567 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:19.567 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:19.568 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:19.568 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:19.568 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:19.568 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:19.568 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:19.568 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:19.569 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:19.569 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:19.569 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:19.569 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:19.569 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:09:19.570 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:09:19.570 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:09:19.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.570 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:19.570 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:09:19.571 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:09:19.571 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.571 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.571 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:19.571 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 2, 2023-10-18T09:09:19.571 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 2, 2023-10-18T09:09:19.572 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:09:19.572 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:09:19.572 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:09:19.572 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:19.573 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:19.573 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:19.573 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.573 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.573 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 23, 2023-10-18T09:09:19.574 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:09:19.574 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:08:56.761888+0000", 2023-10-18T09:09:19.574 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1027, 2023-10-18T09:09:19.574 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-18T09:09:19.575 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:19.575 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:19.575 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:19.575 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:19.575 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:19.576 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:19.576 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:19.576 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:19.576 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:19.576 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:19.576 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:09:19.577 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:09:19.577 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:09:19.577 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:09:19.577 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:09:19.577 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:19.578 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:19.578 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:19.578 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:19.578 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:19.578 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:19.578 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:19.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.579 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "198", 2023-10-18T09:09:19.579 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:19.579 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:19.579 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:19.580 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:19.580 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:19.580 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:19.580 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:19.580 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:19.580 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:19.581 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 33554432, 2023-10-18T09:09:19.581 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:19.581 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:19.581 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:19.581 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:19.582 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:19.582 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:19.582 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:19.582 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:19.582 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:19.583 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:19.583 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:19.583 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:19.583 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:19.583 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:19.584 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:19.584 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:19.584 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.584 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:19.584 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:19.585 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:19.585 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:19.585 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:19.585 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:19.585 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:19.586 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:19.586 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:19.586 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:19.586 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:19.586 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:19.587 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:09:19.587 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:09:19.587 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.587 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:19.587 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:09:19.587 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:09:19.588 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.588 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:19.588 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.3799999952316284, 2023-10-18T09:09:19.588 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.3799999952316284, 2023-10-18T09:09:19.588 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:09:19.589 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-18T09:09:19.589 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-18T09:09:19.589 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:19.589 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:19.589 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:19.590 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.590 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.590 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:19.590 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:09:19.590 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.591 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:09:19.591 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:19.591 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:19.593 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:19.593 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:19.593 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:19.594 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:19.594 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:19.594 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:19.594 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:19.594 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:19.595 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:19.595 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:19.595 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.595 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.595 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:09:19.596 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:19.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.596 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.596 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:09:19.596 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:19.597 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.597 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.597 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:19.597 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.598 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.598 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:09:19.598 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:19.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.599 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.599 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:09:19.599 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:19.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.600 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.600 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.600 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:19.600 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.601 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.601 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:09:19.601 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:19.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.601 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.601 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.602 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:09:19.602 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:19.602 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.602 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.603 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:19.603 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.603 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.603 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:09:19.604 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:19.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.604 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.604 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.604 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:09:19.604 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:19.604 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.605 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.605 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:09:19.605 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:09:19.605 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:09:19.606 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:09:19.606 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:19.606 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:19.606 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:19.606 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.607 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:09:19.607 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:19.607 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:19.607 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:19.608 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:19.608 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:19.608 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:19.608 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:19.608 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:19.608 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:19.609 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:19.609 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:19.609 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:19.609 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.609 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.610 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.610 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:09:19.610 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:19.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.610 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.611 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:09:19.611 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:19.611 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.611 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.611 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:19.612 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.612 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.612 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:09:19.612 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:19.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.613 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.613 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:09:19.613 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:19.613 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.614 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.614 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:19.614 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.615 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.615 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:09:19.615 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:19.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.615 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.616 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:09:19.616 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:19.616 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.616 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.617 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:19.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.617 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:09:19.617 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:19.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.618 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.618 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:09:19.618 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:19.619 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.619 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.619 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:09:19.619 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:09:19.619 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:09:19.620 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:09:19.620 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:19.620 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:19.620 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:19.620 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.621 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:09:19.621 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:19.621 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:19.621 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:19.622 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:19.622 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:19.622 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:19.622 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:19.622 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:19.623 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:19.623 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:19.623 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:19.623 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:19.623 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.624 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.624 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:09:19.624 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:19.624 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.624 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.625 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:09:19.625 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:19.625 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.625 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.625 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:19.625 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.626 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.626 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.626 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:09:19.626 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:19.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.627 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.627 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.627 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:09:19.627 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:19.627 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.627 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.628 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:19.628 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.628 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.628 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.628 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:09:19.629 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:19.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.629 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.629 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.629 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:09:19.630 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:19.630 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.630 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.631 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:19.631 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.631 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.631 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.631 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:09:19.631 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:19.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.632 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.632 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.632 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:09:19.632 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:19.632 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.633 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.633 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:09:19.633 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:09:19.633 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:09:19.633 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:09:19.634 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:19.634 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:19.634 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:19.634 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.635 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.635 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:09:19.635 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:19.635 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:19.635 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:19.636 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:19.636 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:19.636 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:19.636 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:19.636 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:19.637 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:19.637 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:19.637 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:19.637 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:19.637 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.638 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.638 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:09:19.638 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:19.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.638 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.639 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.639 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:09:19.639 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:19.639 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.639 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.640 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:19.640 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.640 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.640 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:09:19.641 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:19.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.641 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.641 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:09:19.642 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:19.642 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.642 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.643 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:19.643 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.643 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.643 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:09:19.644 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:19.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.644 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.644 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.644 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:09:19.645 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:19.645 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.645 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.645 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:19.645 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.646 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.646 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.646 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:09:19.646 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:19.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.647 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.647 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:09:19.647 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:19.648 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.648 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.648 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:09:19.648 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:09:19.649 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:09:19.649 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:09:19.649 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:19.649 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:19.649 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:19.650 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.650 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.650 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.650 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:09:19.651 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:19.651 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:19.651 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:19.651 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:19.651 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:19.652 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:19.652 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:19.652 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:19.652 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:19.652 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:19.652 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:19.653 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:19.653 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.653 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.653 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.653 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:09:19.654 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:19.654 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.654 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.654 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.654 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:09:19.655 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:19.655 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.655 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.655 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.655 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:19.655 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.656 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.656 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.656 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:09:19.656 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:19.656 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.657 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.657 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.657 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:09:19.657 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:19.657 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.657 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.658 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.658 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:19.658 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.658 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.658 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.658 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:09:19.659 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:19.659 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.659 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.659 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.659 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:09:19.660 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:19.660 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.660 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.660 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.660 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:19.661 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.661 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.661 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.661 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:09:19.661 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:19.661 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.662 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.662 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.662 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:09:19.662 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:19.662 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.663 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.663 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.663 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:09:19.663 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:09:19.663 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:09:19.663 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:09:19.664 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:19.664 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:19.664 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:19.664 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.664 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.664 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.665 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:09:19.665 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:19.665 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:19.665 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:19.665 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:19.666 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:19.666 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:19.666 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:19.666 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:19.666 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:19.667 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:19.667 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:19.667 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:19.667 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.667 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.667 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.668 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:09:19.668 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:19.668 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.668 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.668 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.669 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:09:19.669 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:19.669 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.669 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.669 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:19.670 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.670 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.670 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.670 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:09:19.670 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:19.671 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.671 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.671 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.671 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:09:19.671 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:19.671 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.672 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.672 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.672 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:19.672 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.672 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.673 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.673 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:09:19.673 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:19.673 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.673 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.674 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.674 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:09:19.674 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:19.674 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.675 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.675 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.675 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:19.675 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.675 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.676 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.676 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:09:19.676 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:19.676 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.676 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.676 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.677 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:09:19.677 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:19.677 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.677 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.677 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.678 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:09:19.678 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:09:19.678 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:09:19.678 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:09:19.678 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:19.679 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:19.679 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:19.679 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.679 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.679 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.679 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:09:19.680 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:19.680 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:19.680 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:19.680 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:19.680 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:19.681 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:19.681 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:19.681 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:19.681 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:19.681 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:19.682 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:19.682 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:19.682 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.682 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.682 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.683 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:09:19.683 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:19.683 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.683 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.683 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.684 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:09:19.684 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:19.684 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.684 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.684 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.685 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:19.685 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.685 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.685 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.685 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:09:19.686 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:19.686 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.686 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.686 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.686 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.687 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:19.688 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.688 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.688 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.688 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:09:19.688 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:19.688 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.689 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.689 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.689 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:09:19.689 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:19.689 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.689 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.690 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.690 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:19.690 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.690 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.690 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.691 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:09:19.691 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:19.691 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.691 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.691 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.692 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:09:19.692 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:19.692 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.692 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.692 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.692 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:09:19.693 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:09:19.693 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:09:19.693 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:09:19.693 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:19.693 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:19.694 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:19.694 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.694 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.694 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.694 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:09:19.695 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:19.695 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:19.695 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:19.695 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:19.695 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:19.696 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:19.696 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:19.696 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:19.696 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:19.696 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:19.697 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:19.697 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:19.697 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.697 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.697 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.697 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:09:19.698 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:19.698 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.698 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.698 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.698 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:09:19.699 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:19.699 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.699 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.699 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.699 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:19.700 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.700 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.700 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.700 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:09:19.700 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:19.701 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.701 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.701 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.701 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:09:19.701 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:19.702 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.702 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.702 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.702 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:19.702 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.703 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.703 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:09:19.703 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:19.703 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.704 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.704 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:09:19.704 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:19.704 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.704 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.705 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:19.705 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:19.705 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.705 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:19.705 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:09:19.706 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:19.706 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.706 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:19.706 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:09:19.707 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:19.707 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.707 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.707 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.707 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:09:19.707 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:09:19.708 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:09:19.708 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:09:19.708 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:19.708 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:19.708 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:19.709 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:19.709 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.709 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:19.709 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:09:19.709 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.710 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:09:19.710 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:19.710 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:19.710 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:19.710 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:19.710 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:19.711 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:09:19.711 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:19.711 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.711 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.711 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:09:19.712 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:19.712 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:19.712 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:19.712 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:19.712 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:19.712 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:09:19.713 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:19.713 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.713 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.713 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:09:19.713 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:19.713 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:19.714 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:19.714 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:19.714 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:19.714 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:09:19.714 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:19.715 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.715 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.715 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:09:19.715 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:19.715 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:19.716 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:19.716 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:19.716 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:19.716 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:09:19.716 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:19.716 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.717 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.717 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:09:19.717 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:19.717 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:19.717 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:19.718 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:19.718 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:19.718 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:09:19.718 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:19.718 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.718 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.719 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:09:19.719 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:19.719 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:19.719 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:19.719 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:19.720 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:19.720 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:09:19.720 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:19.720 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.720 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.721 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:09:19.721 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:19.721 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:19.721 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:19.721 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:19.721 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:19.722 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:09:19.722 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:19.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.722 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:19.722 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:09:19.723 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:19.723 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:19.723 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:19.723 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:19.723 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:19.723 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:09:19.724 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:19.724 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.724 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:19.724 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:09:19.724 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:09:19.724 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:09:19.725 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:09:19.725 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:09:19.725 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:09:19.725 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:09:19.725 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:09:19.725 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:09:19.726 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:09:19.726 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:09:19.726 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:09:19.726 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:09:19.726 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:09:19.727 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.727 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:19.727 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:09:19.727 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:09:19.727 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:09:19.727 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:09:19.728 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:09:19.728 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:09:19.728 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:09:19.728 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:09:19.728 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:09:19.729 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:09:19.729 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:09:19.729 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:19.729 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:09:19.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.534+0000 7f40f95e9640 1 -- 172.21.15.40:0/2055691050 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f40dc06ae40 msgr2=0x7f40dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:19.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.534+0000 7f40f95e9640 1 --2- 172.21.15.40:0/2055691050 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f40dc06ae40 0x7f40dc06d300 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f40f41561b0 tx=0x7f40e0009040 comp rx=0 tx=0).stop 2023-10-18T09:09:19.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.534+0000 7f40f95e9640 1 -- 172.21.15.40:0/2055691050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f406d6f0 msgr2=0x7f40f415a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:19.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.534+0000 7f40f95e9640 1 --2- 172.21.15.40:0/2055691050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f406d6f0 0x7f40f415a610 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f40e400d950 tx=0x7f40e400de10 comp rx=0 tx=0).stop 2023-10-18T09:09:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.534+0000 7f40f95e9640 1 -- 172.21.15.40:0/2055691050 shutdown_connections 2023-10-18T09:09:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.534+0000 7f40f95e9640 1 --2- 172.21.15.40:0/2055691050 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f40dc06ae40 0x7f40dc06d300 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.534+0000 7f40f95e9640 1 --2- 172.21.15.40:0/2055691050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40f40ca0d0 0x7f40f4153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.534+0000 7f40f95e9640 1 --2- 172.21.15.40:0/2055691050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40f40b0630 0x7f40f40aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:19.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.534+0000 7f40f95e9640 1 --2- 172.21.15.40:0/2055691050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f406d6f0 0x7f40f415a610 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.534+0000 7f40f95e9640 1 -- 172.21.15.40:0/2055691050 >> 172.21.15.40:0/2055691050 conn(0x7f40f4069720 msgr2=0x7f40f4155610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.535+0000 7f40f95e9640 1 -- 172.21.15.40:0/2055691050 shutdown_connections 2023-10-18T09:09:19.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:19.535+0000 7f40f95e9640 1 -- 172.21.15.40:0/2055691050 wait complete. 2023-10-18T09:09:19.734 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-18T09:09:24.735 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:09:24.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.974+0000 7f707a0a9640 1 -- 172.21.15.40:0/2622043465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7060005180 msgr2=0x7f706000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:24.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.974+0000 7f707a0a9640 1 --2- 172.21.15.40:0/2622043465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7060005180 0x7f706000b980 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f706c006fd0 tx=0x7f70600057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:24.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.974+0000 7f7073fff640 1 -- 172.21.15.40:0/2622043465 reap_dead start 2023-10-18T09:09:24.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.974+0000 7f707a0a9640 1 -- 172.21.15.40:0/2622043465 shutdown_connections 2023-10-18T09:09:24.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.974+0000 7f707a0a9640 1 -- 172.21.15.40:0/2622043465 >> 172.21.15.40:0/2622043465 conn(0x7f707406cee0 msgr2=0x7f7074069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:24.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.974+0000 7f707a0a9640 1 -- 172.21.15.40:0/2622043465 shutdown_connections 2023-10-18T09:09:24.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.974+0000 7f707a0a9640 1 -- 172.21.15.40:0/2622043465 wait complete. 2023-10-18T09:09:24.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.976+0000 7f707a0a9640 1 Processor -- start 2023-10-18T09:09:24.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.977+0000 7f707a0a9640 1 -- start start 2023-10-18T09:09:24.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.978+0000 7f707a0a9640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7060005180 0x7f707414b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:24.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.978+0000 7f707a0a9640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70741e3760 0x7f70741f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:24.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.978+0000 7f707a0a9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70740cb820 0x7f707413c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:24.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.978+0000 7f707a0a9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70741f1f30 con 0x7f70740cb820 2023-10-18T09:09:24.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.978+0000 7f707a0a9640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70741f14a0 con 0x7f70741e3760 2023-10-18T09:09:24.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.978+0000 7f707a0a9640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70741e3be0 con 0x7f7060005180 2023-10-18T09:09:24.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.978+0000 7f7073fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70740cb820 0x7f707413c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:24.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.978+0000 7f7073fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70740cb820 0x7f707413c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35950/0 (socket says 172.21.15.40:35950) 2023-10-18T09:09:24.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.978+0000 7f7073fff640 1 -- 172.21.15.40:0/339914541 learned_addr learned my addr 172.21.15.40:0/339914541 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:24.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.978+0000 7f70737fe640 1 --2- 172.21.15.40:0/339914541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7060005180 0x7f707414b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:24.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.978+0000 7f706affd640 1 --2- 172.21.15.40:0/339914541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70741e3760 0x7f70741f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:24.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.978+0000 7f7073fff640 1 -- 172.21.15.40:0/339914541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7060005180 msgr2=0x7f707414b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:24.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.979+0000 7f7073fff640 1 --2- 172.21.15.40:0/339914541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7060005180 0x7f707414b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:24.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.979+0000 7f7073fff640 1 -- 172.21.15.40:0/339914541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70741e3760 msgr2=0x7f70741f0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:24.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.979+0000 7f7073fff640 1 --2- 172.21.15.40:0/339914541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70741e3760 0x7f70741f0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:24.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.979+0000 7f7073fff640 1 -- 172.21.15.40:0/339914541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70741f1690 con 0x7f70740cb820 2023-10-18T09:09:24.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.979+0000 7f7073fff640 1 --2- 172.21.15.40:0/339914541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70740cb820 0x7f707413c6e0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f706c00c350 tx=0x7f706c001d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:24.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.979+0000 7f7070ff9640 1 -- 172.21.15.40:0/339914541 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c0014f0 con 0x7f70740cb820 2023-10-18T09:09:24.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.979+0000 7f7070ff9640 1 -- 172.21.15.40:0/339914541 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f706c001650 con 0x7f70740cb820 2023-10-18T09:09:24.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.979+0000 7f7070ff9640 1 -- 172.21.15.40:0/339914541 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c0062b0 con 0x7f70740cb820 2023-10-18T09:09:24.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.979+0000 7f707a0a9640 1 -- 172.21.15.40:0/339914541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70741f1970 con 0x7f70740cb820 2023-10-18T09:09:24.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.979+0000 7f707a0a9640 1 -- 172.21.15.40:0/339914541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70741e1210 con 0x7f70740cb820 2023-10-18T09:09:24.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.980+0000 7f707a0a9640 1 -- 172.21.15.40:0/339914541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70741e1070 con 0x7f70740cb820 2023-10-18T09:09:24.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.982+0000 7f7070ff9640 1 -- 172.21.15.40:0/339914541 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f706c031030 con 0x7f70740cb820 2023-10-18T09:09:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.983+0000 7f7070ff9640 1 --2- 172.21.15.40:0/339914541 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f705406ae40 0x7f705406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.983+0000 7f7070ff9640 1 -- 172.21.15.40:0/339914541 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f706c0aa450 con 0x7f70740cb820 2023-10-18T09:09:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.984+0000 7f7070ff9640 1 -- 172.21.15.40:0/339914541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=199}) v3 -- 0x7f7054072970 con 0x7f70740cb820 2023-10-18T09:09:24.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.984+0000 7f70737fe640 1 --2- 172.21.15.40:0/339914541 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f705406ae40 0x7f705406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:24.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.985+0000 7f70737fe640 1 --2- 172.21.15.40:0/339914541 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f705406ae40 0x7f705406d300 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f706400a380 tx=0x7f7064005ac0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:24.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:24.989+0000 7f7070ff9640 1 -- 172.21.15.40:0/339914541 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f70741e1070 con 0x7f70740cb820 2023-10-18T09:09:25.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:25.188+0000 7f707a0a9640 1 -- 172.21.15.40:0/339914541 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f70741e1070 con 0x7f70740cb820 2023-10-18T09:09:25.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:25.191+0000 7f7070ff9640 1 -- 172.21.15.40:0/339914541 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v198) v1 ==== 81+0+34045 (secure 0 0 0) 0x7f70741e1070 con 0x7f70740cb820 2023-10-18T09:09:25.194 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:25.194 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:09:25.194 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 198, 2023-10-18T09:09:25.194 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:09:25.194 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:09:25.195 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:09:16.215303+0000", 2023-10-18T09:09:25.195 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:09:25.195 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:09:25.195 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:09:25.195 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:09:25.195 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:09:25.196 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:09:25.196 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:09:25.196 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:09:25.196 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:09:25.196 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:25.197 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:09:25.197 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:09:25.197 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:09:25.197 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:09:25.197 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:09:25.197 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 23, 2023-10-18T09:09:25.198 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:09:25.198 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:09:25.198 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:09:25.198 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:09:25.198 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:09:25.199 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:09:25.199 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.199 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:09:25.199 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:09:25.199 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:09:25.200 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:25.200 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:25.200 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:25.200 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:25.200 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:25.201 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:25.201 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:25.201 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:25.201 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:25.201 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:25.201 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:25.202 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:25.202 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:09:25.202 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:09:25.202 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:09:25.202 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:09:25.202 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:09:25.203 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:25.203 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:25.203 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:25.203 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:25.203 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:25.203 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:25.204 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:25.204 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.204 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:09:25.204 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:25.204 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:25.204 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:25.205 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:25.205 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:25.205 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:25.205 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:25.205 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:25.205 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:25.206 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:09:25.206 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:25.206 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:25.206 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:25.206 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:25.207 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:25.207 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:25.207 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:25.207 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:25.207 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:25.207 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:25.208 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:25.208 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:25.208 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:25.208 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:25.208 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:25.208 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:25.209 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.209 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:25.209 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:25.209 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:25.209 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:25.210 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:25.210 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:25.210 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:25.210 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:25.210 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:25.211 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:25.211 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:25.211 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:25.211 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:09:25.212 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:09:25.212 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.212 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:25.212 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:09:25.212 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.212 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:25.213 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:09:25.213 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:09:25.213 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:09:25.213 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:09:25.213 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:09:25.213 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:25.214 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:25.214 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:25.214 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.214 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.214 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.214 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 22, 2023-10-18T09:09:25.215 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:09:25.215 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:08:55.675479+0000", 2023-10-18T09:09:25.215 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:25.215 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:25.215 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:25.215 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:25.216 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:25.216 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:25.216 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:25.216 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:25.216 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:25.216 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:25.217 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:25.217 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:25.217 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:09:25.217 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:09:25.217 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:09:25.217 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:09:25.218 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:09:25.218 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:25.218 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:25.218 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:25.218 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:25.219 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:25.219 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:25.219 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:25.219 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.219 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "195", 2023-10-18T09:09:25.220 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:25.220 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:25.220 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:25.220 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:25.220 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:25.220 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:25.221 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:25.221 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:25.221 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:25.221 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:09:25.221 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:25.221 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:25.222 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:25.222 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:25.222 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:25.222 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:25.222 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:25.222 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:25.223 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:25.223 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:25.223 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:25.223 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:25.223 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:25.223 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:25.224 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:25.224 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:25.224 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.224 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:25.224 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:25.224 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:25.225 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:25.225 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:25.225 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:25.225 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:25.225 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:25.225 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:25.226 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:25.226 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:25.226 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:25.226 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:09:25.226 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:09:25.226 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:09:25.227 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.227 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:25.227 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:09:25.227 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:09:25.227 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.227 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.228 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:25.228 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 2, 2023-10-18T09:09:25.228 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 2, 2023-10-18T09:09:25.228 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:09:25.228 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:09:25.228 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:09:25.229 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:25.229 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:25.229 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:25.229 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.229 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.230 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.230 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 23, 2023-10-18T09:09:25.230 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:09:25.230 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:08:56.761888+0000", 2023-10-18T09:09:25.230 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1027, 2023-10-18T09:09:25.231 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-18T09:09:25.231 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:25.231 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:25.231 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:25.231 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:25.231 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:25.231 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:25.232 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:25.232 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:25.232 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:25.232 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:25.232 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:09:25.232 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:09:25.233 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:09:25.233 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:09:25.233 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:09:25.233 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:25.233 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:25.233 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:25.234 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:25.234 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:25.234 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:25.234 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:25.234 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.234 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "198", 2023-10-18T09:09:25.235 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:25.235 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:25.235 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:25.235 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:25.235 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:25.235 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:25.236 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:25.236 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:25.236 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:25.236 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 33554432, 2023-10-18T09:09:25.236 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:25.236 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:25.237 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:25.237 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:25.237 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:25.237 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:25.237 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:25.237 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:25.238 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:25.238 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:25.238 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:25.238 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:25.238 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:25.238 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:25.239 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:25.239 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:25.239 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.239 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:25.239 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:25.240 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:25.240 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:25.240 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:25.240 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:25.240 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:25.240 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:25.241 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:25.241 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:25.241 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:25.241 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:25.241 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:09:25.241 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:09:25.242 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.242 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:25.242 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:09:25.242 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:09:25.242 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.242 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.243 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:25.243 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.3799999952316284, 2023-10-18T09:09:25.243 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.3799999952316284, 2023-10-18T09:09:25.243 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:09:25.243 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-18T09:09:25.243 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-18T09:09:25.244 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:25.244 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:25.244 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:25.244 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.244 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.244 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:25.245 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:09:25.245 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.245 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:09:25.245 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:25.245 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:25.246 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:25.246 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:25.246 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:25.246 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:25.246 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:25.247 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:25.247 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:25.247 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:25.247 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:25.247 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:25.247 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.248 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.248 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.248 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:09:25.248 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:25.248 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.248 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.249 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.249 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:09:25.249 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:25.249 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.249 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.249 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.250 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:25.250 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.250 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.250 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.250 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:09:25.251 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:25.251 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.251 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.251 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.251 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:09:25.251 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:25.252 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.252 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.252 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.252 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:25.252 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.252 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.253 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.253 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:09:25.253 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:25.253 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.253 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.253 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.254 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:09:25.254 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:25.254 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.254 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.254 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.254 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:25.255 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.255 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.255 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.255 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:09:25.255 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:25.255 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.256 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.256 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.256 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:09:25.256 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:25.256 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.257 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.257 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.257 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:09:25.257 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:09:25.257 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:09:25.257 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:09:25.257 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:25.258 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:25.258 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:25.258 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.258 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.258 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.259 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:09:25.259 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:25.259 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:25.259 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:25.259 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:25.260 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:25.260 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:25.260 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:25.260 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:25.260 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:25.260 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:25.261 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:25.261 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:25.261 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.261 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.261 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.261 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:09:25.262 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:25.262 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.262 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.262 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.262 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:09:25.263 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:25.263 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.263 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.263 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.263 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:25.263 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.264 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.264 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.264 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:09:25.264 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:25.264 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.264 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.265 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.265 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:09:25.265 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:25.265 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.265 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.265 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.266 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:25.266 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.266 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.266 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.266 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:09:25.267 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:25.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.267 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.267 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.267 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:09:25.267 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:25.268 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.268 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.268 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.268 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:25.268 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.268 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.269 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.269 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:09:25.269 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:25.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.269 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.270 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.270 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:09:25.270 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:25.270 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.270 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.271 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:09:25.271 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:09:25.271 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:09:25.271 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:09:25.272 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:25.272 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:25.272 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:25.272 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.272 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.272 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.273 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:09:25.273 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:25.273 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:25.273 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:25.273 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:25.273 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:25.274 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:25.274 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:25.274 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:25.274 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:25.274 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:25.275 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:25.275 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:25.275 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.275 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.275 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.276 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:09:25.276 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:25.276 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.276 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.276 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.277 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:09:25.277 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:25.277 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.277 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.277 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:25.278 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.278 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.278 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.278 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:09:25.278 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:25.279 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.279 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.279 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.279 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:09:25.279 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:25.280 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.280 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.280 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:25.280 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.281 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.281 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:09:25.281 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:25.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.282 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.282 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:09:25.282 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:25.282 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.282 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.283 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:25.283 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.283 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.283 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.284 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:09:25.284 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:25.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.284 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.284 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.284 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:09:25.285 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:25.285 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.285 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.285 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.285 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:09:25.286 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:09:25.286 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:09:25.286 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:09:25.286 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:25.286 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:25.286 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:25.287 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.287 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.287 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.287 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:09:25.287 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:25.288 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:25.288 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:25.288 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:25.288 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:25.288 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:25.289 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:25.289 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:25.289 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:25.289 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:25.289 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:25.290 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:25.290 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.290 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.290 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.290 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:09:25.290 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:25.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.291 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.291 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.291 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:09:25.291 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:25.292 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.292 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.292 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.292 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:25.292 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.292 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.293 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.293 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:09:25.293 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:25.293 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.293 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.294 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.294 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:09:25.294 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:25.294 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.294 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.295 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:25.295 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.295 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.295 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:09:25.296 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:25.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.296 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.296 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.296 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:09:25.297 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:25.297 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.297 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.297 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:25.297 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.298 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.298 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:09:25.298 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:25.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.299 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.299 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:09:25.299 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:25.299 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.299 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.300 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:09:25.300 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:09:25.300 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:09:25.300 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:09:25.301 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:25.301 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:25.301 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:25.301 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.302 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:09:25.302 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:25.302 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:25.302 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:25.303 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:25.303 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:25.303 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:25.303 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:25.303 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:25.303 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:25.304 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:25.304 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:25.304 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:25.304 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.304 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.305 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:09:25.305 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:25.305 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.305 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.305 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.306 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:09:25.306 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:25.306 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.306 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.306 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.306 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:25.307 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.307 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.307 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.307 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:09:25.307 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:25.308 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.308 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.308 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.308 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:09:25.308 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:25.308 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.309 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.309 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.309 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:25.309 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.309 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.310 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.310 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:09:25.310 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:25.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.310 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.311 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.311 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:09:25.311 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:25.311 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.312 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.312 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:25.312 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.313 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.313 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:09:25.313 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:25.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.314 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.314 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:09:25.314 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:25.314 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.314 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.315 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:09:25.315 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:09:25.315 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:09:25.315 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:09:25.316 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:25.316 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:25.316 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:25.316 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.316 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.317 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:09:25.317 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:25.317 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:25.317 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:25.317 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:25.318 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:25.318 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:25.318 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:25.318 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:25.318 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:25.319 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:25.319 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:25.319 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:25.319 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.319 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.319 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.320 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:09:25.320 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:25.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.320 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.321 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:09:25.321 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:25.321 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.321 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.321 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:25.322 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.322 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.322 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.322 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:09:25.322 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:25.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.323 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.323 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.323 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:09:25.323 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:25.323 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.324 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.324 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.324 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:25.324 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.324 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.325 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.325 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:09:25.325 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:25.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.325 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.326 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.326 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:09:25.326 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:25.326 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.326 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.326 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.327 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:25.327 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.327 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.327 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.327 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:09:25.328 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:25.328 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.328 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.328 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.328 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:09:25.328 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:25.329 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.329 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.329 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:09:25.329 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:09:25.330 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:09:25.330 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:09:25.330 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:25.330 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:25.330 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:25.331 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.331 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.331 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.331 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:09:25.331 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:25.331 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:25.332 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:25.332 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:25.332 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:25.332 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:25.332 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:25.332 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:25.333 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:25.333 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:25.333 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:25.333 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:25.333 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.334 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.334 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:09:25.334 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:25.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.335 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.335 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:09:25.335 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:25.335 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.335 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.336 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:25.336 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.336 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.336 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:09:25.337 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:25.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.337 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.337 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:09:25.337 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:25.338 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.338 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.338 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:25.338 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.339 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.339 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:09:25.339 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:25.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.340 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.340 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:09:25.340 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:25.340 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.340 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.341 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:25.341 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.341 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.341 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:09:25.342 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:25.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.342 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.342 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:09:25.343 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:25.343 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.343 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.343 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:09:25.343 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:09:25.344 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:09:25.344 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:09:25.344 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:25.344 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:25.344 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:25.345 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.345 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.345 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.345 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:09:25.345 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:25.345 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:25.346 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:25.346 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:25.346 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:25.346 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:25.346 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:25.346 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:25.347 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:25.347 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:25.347 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:25.347 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:25.347 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.348 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.348 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:09:25.348 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:25.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.348 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.349 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:09:25.349 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:25.349 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.349 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.350 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:25.350 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.350 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.350 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:09:25.351 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:25.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.351 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.351 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:09:25.351 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:25.352 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.352 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.352 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:25.352 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:09:25.353 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:25.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.354 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:09:25.354 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:25.354 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.354 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.355 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:25.355 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:25.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.355 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:25.355 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:09:25.356 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:25.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.356 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:25.356 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:09:25.356 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:25.357 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.357 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.357 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:09:25.357 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:09:25.357 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:09:25.358 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:09:25.358 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:25.358 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:25.358 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:25.358 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:25.359 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.359 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:25.359 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:09:25.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.359 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:09:25.359 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:25.360 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:25.360 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:25.360 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:25.360 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:25.360 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:09:25.360 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:25.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.361 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.361 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:09:25.361 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:25.361 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:25.361 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:25.362 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:25.362 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:25.362 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:09:25.362 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:25.362 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.363 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.363 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:09:25.363 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:25.363 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:25.363 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:25.363 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:25.364 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:25.364 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:09:25.364 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:25.364 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.364 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.364 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:09:25.365 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:25.365 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:25.365 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:25.365 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:25.365 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:25.365 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:09:25.366 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:25.366 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.366 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.366 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:09:25.366 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:25.367 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:25.367 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:25.367 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:25.367 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:25.367 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:09:25.367 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:25.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.368 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.368 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:09:25.368 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:25.368 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:25.369 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:25.369 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:25.369 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:25.369 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:09:25.369 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:25.369 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.370 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.370 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:09:25.370 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:25.370 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:25.370 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:25.371 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:25.371 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:25.371 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:09:25.371 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:25.371 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.372 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:25.372 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:09:25.372 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:25.372 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:25.372 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:25.372 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:25.373 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:25.373 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:09:25.373 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:25.373 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.373 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:25.373 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:09:25.374 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:09:25.374 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:09:25.374 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:09:25.374 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:09:25.374 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:09:25.374 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:09:25.375 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:09:25.375 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:09:25.375 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:09:25.375 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:09:25.375 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:09:25.375 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:09:25.376 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:09:25.376 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.376 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:25.376 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:09:25.376 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:09:25.376 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:09:25.377 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:09:25.377 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:09:25.377 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:09:25.377 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:09:25.377 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:09:25.377 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:09:25.378 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:09:25.378 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:09:25.378 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:25.378 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:09:25.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:25.193+0000 7f707a0a9640 1 -- 172.21.15.40:0/339914541 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f705406ae40 msgr2=0x7f705406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:25.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:25.193+0000 7f707a0a9640 1 --2- 172.21.15.40:0/339914541 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f705406ae40 0x7f705406d300 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f706400a380 tx=0x7f7064005ac0 comp rx=0 tx=0).stop 2023-10-18T09:09:25.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:25.193+0000 7f707a0a9640 1 -- 172.21.15.40:0/339914541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70740cb820 msgr2=0x7f707413c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:25.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:25.193+0000 7f707a0a9640 1 --2- 172.21.15.40:0/339914541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70740cb820 0x7f707413c6e0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f706c00c350 tx=0x7f706c001d20 comp rx=0 tx=0).stop 2023-10-18T09:09:25.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:25.194+0000 7f707a0a9640 1 -- 172.21.15.40:0/339914541 shutdown_connections 2023-10-18T09:09:25.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:25.194+0000 7f707a0a9640 1 --2- 172.21.15.40:0/339914541 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f705406ae40 0x7f705406d300 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:25.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:25.194+0000 7f707a0a9640 1 --2- 172.21.15.40:0/339914541 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70740cb820 0x7f707413c6e0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:25.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:25.194+0000 7f707a0a9640 1 --2- 172.21.15.40:0/339914541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70741e3760 0x7f70741f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:25.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:25.194+0000 7f707a0a9640 1 --2- 172.21.15.40:0/339914541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7060005180 0x7f707414b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:25.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:25.194+0000 7f707a0a9640 1 -- 172.21.15.40:0/339914541 >> 172.21.15.40:0/339914541 conn(0x7f707406cee0 msgr2=0x7f7074070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:25.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:25.194+0000 7f707a0a9640 1 -- 172.21.15.40:0/339914541 shutdown_connections 2023-10-18T09:09:25.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:25.194+0000 7f707a0a9640 1 -- 172.21.15.40:0/339914541 wait complete. 2023-10-18T09:09:25.383 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-18T09:09:30.384 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:09:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.627+0000 7f0f8e0d7640 1 -- 172.21.15.40:0/3424473883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f88157870 msgr2=0x7f0f880ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.627+0000 7f0f8e0d7640 1 --2- 172.21.15.40:0/3424473883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f88157870 0x7f0f880ca350 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0f84005fd0 tx=0x7f0f8402be10 comp rx=0 tx=0).stop 2023-10-18T09:09:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.627+0000 7f0f8e0d7640 1 -- 172.21.15.40:0/3424473883 shutdown_connections 2023-10-18T09:09:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.627+0000 7f0f8e0d7640 1 --2- 172.21.15.40:0/3424473883 >> v2:172.21.15.144:3300/0 conn(0x7f0f880caec0 0x7f0f880cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.627+0000 7f0f8e0d7640 1 --2- 172.21.15.40:0/3424473883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f88157870 0x7f0f880ca350 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.627+0000 7f0f8e0d7640 1 -- 172.21.15.40:0/3424473883 >> 172.21.15.40:0/3424473883 conn(0x7f0f88069720 msgr2=0x7f0f88069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.628+0000 7f0f8e0d7640 1 -- 172.21.15.40:0/3424473883 shutdown_connections 2023-10-18T09:09:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.628+0000 7f0f8e0d7640 1 -- 172.21.15.40:0/3424473883 wait complete. 2023-10-18T09:09:30.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.630+0000 7f0f8e0d7640 1 Processor -- start 2023-10-18T09:09:30.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.630+0000 7f0f8e0d7640 1 -- start start 2023-10-18T09:09:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.631+0000 7f0f8e0d7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f880ca890 0x7f0f88143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.631+0000 7f0f8e0d7640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f880caec0 0x7f0f8814aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.631+0000 7f0f8e0d7640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f88157870 0x7f0f880abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:30.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.631+0000 7f0f8e0d7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8814fcb0 con 0x7f0f880ca890 2023-10-18T09:09:30.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.631+0000 7f0f8e0d7640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8814aff0 con 0x7f0f880caec0 2023-10-18T09:09:30.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.631+0000 7f0f8e0d7640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8814f680 con 0x7f0f88157870 2023-10-18T09:09:30.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.632+0000 7f0f8d0d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f880ca890 0x7f0f88143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.632+0000 7f0f8d0d5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f880ca890 0x7f0f88143bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35966/0 (socket says 172.21.15.40:35966) 2023-10-18T09:09:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.632+0000 7f0f8d0d5640 1 -- 172.21.15.40:0/3629827685 learned_addr learned my addr 172.21.15.40:0/3629827685 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.632+0000 7f0f8d8d6640 1 --2- 172.21.15.40:0/3629827685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f88157870 0x7f0f880abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.632+0000 7f0f8c8d4640 1 --2- 172.21.15.40:0/3629827685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f880caec0 0x7f0f8814aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.632+0000 7f0f8d0d5640 1 -- 172.21.15.40:0/3629827685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f88157870 msgr2=0x7f0f880abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.632+0000 7f0f8d0d5640 1 --2- 172.21.15.40:0/3629827685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f88157870 0x7f0f880abac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.632+0000 7f0f8d0d5640 1 -- 172.21.15.40:0/3629827685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f880caec0 msgr2=0x7f0f8814aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.632+0000 7f0f8d0d5640 1 --2- 172.21.15.40:0/3629827685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f880caec0 0x7f0f8814aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.632+0000 7f0f8d0d5640 1 -- 172.21.15.40:0/3629827685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f84009560 con 0x7f0f880ca890 2023-10-18T09:09:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.633+0000 7f0f8d0d5640 1 --2- 172.21.15.40:0/3629827685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f880ca890 0x7f0f88143bf0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009850 tx=0x7f0f8402da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.633+0000 7f0f7dffb640 1 -- 172.21.15.40:0/3629827685 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8402de50 con 0x7f0f880ca890 2023-10-18T09:09:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.633+0000 7f0f7dffb640 1 -- 172.21.15.40:0/3629827685 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0f840036e0 con 0x7f0f880ca890 2023-10-18T09:09:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.633+0000 7f0f8e0d7640 1 -- 172.21.15.40:0/3629827685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8814b250 con 0x7f0f880ca890 2023-10-18T09:09:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.633+0000 7f0f7dffb640 1 -- 172.21.15.40:0/3629827685 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84003a90 con 0x7f0f880ca890 2023-10-18T09:09:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.633+0000 7f0f8e0d7640 1 -- 172.21.15.40:0/3629827685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8814f940 con 0x7f0f880ca890 2023-10-18T09:09:30.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.635+0000 7f0f8e0d7640 1 -- 172.21.15.40:0/3629827685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8814fb00 con 0x7f0f880ca890 2023-10-18T09:09:30.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.635+0000 7f0f7dffb640 1 -- 172.21.15.40:0/3629827685 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0f84003bf0 con 0x7f0f880ca890 2023-10-18T09:09:30.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.636+0000 7f0f7dffb640 1 --2- 172.21.15.40:0/3629827685 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f6c06ae70 0x7f0f6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:30.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.637+0000 7f0f7dffb640 1 -- 172.21.15.40:0/3629827685 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f840ac2c0 con 0x7f0f880ca890 2023-10-18T09:09:30.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.637+0000 7f0f8c8d4640 1 --2- 172.21.15.40:0/3629827685 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f6c06ae70 0x7f0f6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:30.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.638+0000 7f0f8c8d4640 1 --2- 172.21.15.40:0/3629827685 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f6c06ae70 0x7f0f6c06d330 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0f881435a0 tx=0x7f0f78009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:30.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.643+0000 7f0f7dffb640 1 -- 172.21.15.40:0/3629827685 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0f840772e0 con 0x7f0f880ca890 2023-10-18T09:09:30.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.839+0000 7f0f8e0d7640 1 -- 172.21.15.40:0/3629827685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f0f88066e00 con 0x7f0f880ca890 2023-10-18T09:09:30.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.841+0000 7f0f7dffb640 1 -- 172.21.15.40:0/3629827685 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v199) v1 ==== 81+0+34026 (secure 0 0 0) 0x7f0f84043090 con 0x7f0f880ca890 2023-10-18T09:09:30.844 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:30.844 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:09:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 199, 2023-10-18T09:09:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:09:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:09:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:09:26.219860+0000", 2023-10-18T09:09:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:09:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:09:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:09:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:09:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:09:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:09:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:09:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:09:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:09:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:09:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:09:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:09:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:09:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:09:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 23, 2023-10-18T09:09:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:09:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:09:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:09:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:09:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:09:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:09:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:09:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:09:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:09:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:09:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:09:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:09:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:09:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:09:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:30.855 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:30.855 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.855 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:09:30.855 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:30.855 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:30.855 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:09:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:09:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:09:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:09:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:09:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:09:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:09:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:09:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:09:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 22, 2023-10-18T09:09:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:09:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:08:55.675479+0000", 2023-10-18T09:09:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:30.869 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:30.869 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:30.869 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:09:30.869 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:09:30.869 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:09:30.870 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:09:30.870 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:09:30.870 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:30.870 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:30.870 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:30.871 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:30.871 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:30.871 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:30.871 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:30.871 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.872 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "195", 2023-10-18T09:09:30.872 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:30.872 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:30.872 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:30.872 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:30.872 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:30.873 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:30.873 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:30.873 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:30.873 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:30.873 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:09:30.874 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:30.874 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:30.874 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:30.874 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:30.874 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:30.875 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:30.875 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:30.875 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:30.875 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:30.875 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:30.875 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:30.876 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:30.876 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:30.876 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:30.876 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:30.876 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:30.877 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.877 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:30.877 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:30.877 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:30.877 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:30.877 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:30.878 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:30.878 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:30.878 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:30.878 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:30.878 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:30.879 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:30.879 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:30.879 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:09:30.879 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:09:30.879 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:09:30.880 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.880 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:30.880 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:09:30.880 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:09:30.880 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.881 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:30.881 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 2, 2023-10-18T09:09:30.881 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 2, 2023-10-18T09:09:30.881 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:09:30.881 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:09:30.882 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:09:30.882 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:30.882 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:30.882 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:30.882 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.883 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.883 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 23, 2023-10-18T09:09:30.883 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:09:30.883 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:08:56.761888+0000", 2023-10-18T09:09:30.884 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:30.884 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:30.884 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:30.884 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:30.884 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:30.885 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:30.885 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:30.885 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:30.885 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:30.885 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:30.885 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:30.886 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:30.886 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:09:30.886 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:09:30.886 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:09:30.886 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:09:30.887 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:09:30.887 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:30.887 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:30.887 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:30.887 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:30.887 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:30.888 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:30.888 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:30.888 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.888 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "199", 2023-10-18T09:09:30.888 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:30.889 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:30.889 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:30.889 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:30.889 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:30.889 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:30.890 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:30.890 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:30.890 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:30.890 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 33554432, 2023-10-18T09:09:30.890 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:30.891 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:30.891 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:30.891 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:30.891 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:30.891 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:30.891 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:30.892 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:30.892 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:30.892 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:30.892 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:30.892 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:30.892 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:30.893 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:30.893 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:30.893 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:30.893 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.893 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:30.894 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:30.894 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:30.894 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:30.894 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:30.894 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:30.895 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:30.895 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:30.895 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:30.895 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:30.895 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:30.895 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:30.896 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:09:30.896 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:09:30.896 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.896 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:30.896 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:09:30.896 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:09:30.897 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.897 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:30.897 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.3799999952316284, 2023-10-18T09:09:30.897 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.3799999952316284, 2023-10-18T09:09:30.897 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:09:30.898 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-18T09:09:30.898 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-18T09:09:30.898 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:30.898 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:30.898 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:30.899 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.899 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.899 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:30.899 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:09:30.899 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.899 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:09:30.900 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:30.900 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:30.900 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:30.900 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:30.900 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:30.901 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:30.901 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:30.901 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:30.901 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:30.901 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:30.902 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:30.902 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:30.902 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.902 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.902 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.902 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:09:30.903 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:30.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.903 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.903 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.903 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:09:30.903 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:30.904 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.904 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.904 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:30.904 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.905 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.905 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:09:30.905 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:30.905 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.905 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.905 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.906 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:09:30.906 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:30.906 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.906 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.906 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.907 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:30.907 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.907 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.907 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.907 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:09:30.908 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:30.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.908 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.908 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.908 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:09:30.908 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:30.909 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.909 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.909 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.909 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:30.909 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.910 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.910 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.910 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:09:30.910 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:30.910 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.911 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.911 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.911 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:09:30.911 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:30.911 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.911 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.912 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.912 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:09:30.912 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:09:30.912 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:09:30.912 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:09:30.913 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:30.913 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:30.913 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:30.913 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.913 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.914 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:09:30.914 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:30.914 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:30.914 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:30.914 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:30.915 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:30.915 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:30.915 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:30.915 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:30.915 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:30.916 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:30.916 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:30.916 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:30.916 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.916 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.917 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.917 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:09:30.917 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:30.917 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.917 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.917 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.918 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:09:30.918 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:30.918 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.918 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.918 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.919 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:30.919 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.919 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.919 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.919 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:09:30.920 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:30.920 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.920 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.920 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.920 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:09:30.921 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:30.921 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.921 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.921 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.921 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:30.921 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.922 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.922 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.922 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:09:30.922 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:30.922 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.923 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.923 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.923 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:09:30.923 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:30.923 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.923 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.924 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.924 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:30.924 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.924 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.924 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.924 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:09:30.925 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:30.925 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.925 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.925 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.925 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:09:30.926 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:30.926 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.926 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.926 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.926 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:09:30.926 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:09:30.927 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:09:30.927 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:09:30.927 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:30.927 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:30.927 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:30.928 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.928 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.928 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:09:30.928 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:30.928 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:30.929 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:30.929 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:30.929 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:30.929 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:30.929 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:30.930 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:30.930 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:30.930 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:30.930 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:30.930 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:30.931 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.931 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.931 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.931 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:09:30.931 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:30.932 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.932 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.932 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.932 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:09:30.932 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:30.932 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.933 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.933 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.933 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:30.933 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.933 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.934 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.934 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:09:30.934 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:30.934 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.934 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.935 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.935 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:09:30.935 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:30.935 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.935 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.935 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.936 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:30.936 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.936 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.936 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.936 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:09:30.936 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:30.937 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.937 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.937 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.937 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:09:30.937 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:30.938 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.938 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.938 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.938 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:30.938 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.939 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.939 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.939 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:09:30.939 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:30.939 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.940 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.940 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.940 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:09:30.940 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:30.940 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.941 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.941 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.941 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:09:30.941 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:09:30.941 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:09:30.942 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:09:30.942 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:30.942 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:30.942 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:30.942 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.942 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.943 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.943 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:09:30.943 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:30.943 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:30.943 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:30.944 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:30.944 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:30.944 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:30.944 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:30.944 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:30.945 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:30.945 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:30.945 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:30.945 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:30.945 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.945 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.946 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.946 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:09:30.946 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:30.946 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.946 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.947 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.947 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:09:30.947 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:30.947 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.947 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.947 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.948 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:30.948 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.948 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.948 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.948 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:09:30.948 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:30.949 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.949 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.949 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.949 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:09:30.949 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:30.950 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.950 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.950 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.950 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:30.950 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.951 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.951 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.951 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:09:30.951 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:30.951 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.951 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.952 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.952 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:09:30.952 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:30.952 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.952 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.952 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.953 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:30.953 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.953 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.953 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.953 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:09:30.954 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:30.954 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.954 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.954 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.954 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:09:30.954 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:30.955 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.955 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.955 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.955 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:09:30.955 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:09:30.955 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:09:30.956 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:09:30.956 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:30.956 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:30.956 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:30.956 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.956 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.957 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.957 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:09:30.957 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:30.957 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:30.957 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:30.958 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:30.958 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:30.958 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:30.958 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:30.958 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:30.958 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:30.959 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:30.959 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:30.959 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:30.959 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.959 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.960 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.960 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:09:30.960 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:30.960 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.960 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.961 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.961 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:09:30.961 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:30.961 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.962 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.962 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.962 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:30.962 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.962 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.962 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.963 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:09:30.963 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:30.963 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.963 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.964 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.964 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:09:30.964 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:30.964 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.964 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.965 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.965 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:30.965 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.965 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.965 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.966 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:09:30.966 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:30.966 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.966 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.966 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.967 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:09:30.967 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:30.967 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.967 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.967 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.967 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:30.968 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.968 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.968 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.968 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:09:30.968 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:30.969 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.969 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.969 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.969 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:09:30.969 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:30.970 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.970 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.970 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.970 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:09:30.970 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:09:30.971 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:09:30.971 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:09:30.971 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:30.971 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:30.971 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:30.971 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.972 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.972 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.972 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:09:30.972 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:30.972 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:30.973 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:30.973 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:30.973 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:30.973 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:30.973 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:30.973 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:30.974 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:30.974 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:30.974 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:30.974 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:30.974 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.975 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.975 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.975 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:09:30.975 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:30.975 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.976 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.976 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.976 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:09:30.976 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:30.976 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.976 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.977 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.977 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:30.977 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.977 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.977 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.977 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:09:30.978 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:30.978 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.978 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.978 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.978 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:09:30.979 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:30.979 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.979 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.979 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.979 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:30.980 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.980 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.980 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.980 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:09:30.980 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:30.981 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.981 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.981 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.981 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:09:30.981 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:30.981 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.982 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.982 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.982 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:30.982 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.982 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.983 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.983 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:09:30.983 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:30.983 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.983 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.984 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.984 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:09:30.984 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:30.984 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.984 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.984 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.985 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:09:30.985 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:09:30.985 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:09:30.985 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:09:30.985 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:30.986 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:30.986 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:30.986 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.986 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.986 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.987 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:09:30.987 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:30.987 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:30.987 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:30.987 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:30.987 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:30.988 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:30.988 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:30.988 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:30.988 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:30.988 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:30.989 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:30.989 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:30.989 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.989 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.989 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.989 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:09:30.990 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:30.990 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.990 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.990 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.990 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:09:30.991 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:30.991 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.991 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.991 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.991 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:30.992 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.992 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.992 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.992 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:09:30.992 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:30.992 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.993 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.993 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.993 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:09:30.993 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:30.993 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.994 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.994 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.994 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:30.994 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.994 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.995 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.995 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:09:30.995 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:30.995 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.995 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.996 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.996 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:09:30.996 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:30.996 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.996 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.997 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.997 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:30.997 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:30.997 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.997 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:30.997 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:09:30.998 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:30.998 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.998 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:30.998 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:30.998 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:09:30.999 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:30.999 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:30.999 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:30.999 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:30.999 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:09:30.999 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:09:31.000 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:09:31.000 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:09:31.000 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:31.000 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:31.000 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:31.001 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.001 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.001 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.001 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:09:31.001 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:31.002 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:31.002 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:31.002 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:31.002 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:31.002 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:31.002 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:31.003 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:31.003 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:31.003 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:31.003 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:31.003 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:31.004 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.004 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.004 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.004 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:09:31.004 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:31.005 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.005 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.005 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.005 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:09:31.005 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:31.005 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.006 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.006 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.006 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:31.006 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.006 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.007 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.007 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:09:31.007 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:31.007 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.007 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.007 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.008 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:09:31.008 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:31.008 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.008 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.008 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.009 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:31.009 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.009 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.009 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.009 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:09:31.009 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:31.010 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.010 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.010 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.010 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:09:31.010 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:31.011 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.011 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.011 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:31.011 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.012 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.012 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.012 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:09:31.012 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:31.012 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.013 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.013 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.013 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:09:31.013 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:31.013 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.013 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.014 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:09:31.014 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:09:31.014 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:09:31.014 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:09:31.015 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:31.015 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:31.015 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:31.015 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.015 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.015 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:31.016 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:09:31.016 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.016 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:09:31.016 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:31.016 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:31.017 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:31.017 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:31.017 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:31.017 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:09:31.018 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:31.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.018 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.018 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:09:31.018 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:31.019 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:31.019 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:31.019 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:31.019 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:31.019 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:09:31.020 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:31.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.020 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.020 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:09:31.020 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:31.021 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:31.021 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:31.021 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:31.021 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:31.021 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:09:31.022 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:31.022 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.022 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.022 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:09:31.022 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:31.022 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:31.023 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:31.023 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:31.023 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:31.023 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:09:31.023 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:31.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.024 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.024 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:09:31.024 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:31.024 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:31.025 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:31.025 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:31.025 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:31.025 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:09:31.025 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:31.025 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.026 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.026 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:09:31.026 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:31.026 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:31.026 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:31.027 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:31.027 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:31.027 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:09:31.027 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:31.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.027 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.028 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:09:31.028 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:31.028 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:31.028 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:31.029 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:31.029 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:31.029 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:09:31.029 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:31.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.030 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.030 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:09:31.030 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:31.031 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:31.031 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:31.031 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:31.031 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:31.031 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:09:31.032 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:31.032 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.032 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:31.032 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:09:31.032 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:09:31.033 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:09:31.033 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:09:31.033 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:09:31.033 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:09:31.033 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:09:31.034 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:09:31.034 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:09:31.034 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:09:31.034 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:09:31.035 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:09:31.035 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:09:31.035 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:09:31.035 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.035 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.035 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:09:31.036 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:09:31.036 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:09:31.036 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:09:31.037 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:09:31.037 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:09:31.037 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:09:31.037 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:09:31.038 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:09:31.038 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:09:31.038 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:09:31.038 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.039 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:09:31.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.844+0000 7f0f8e0d7640 1 -- 172.21.15.40:0/3629827685 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f6c06ae70 msgr2=0x7f0f6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:31.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.844+0000 7f0f8e0d7640 1 --2- 172.21.15.40:0/3629827685 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f6c06ae70 0x7f0f6c06d330 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0f881435a0 tx=0x7f0f78009040 comp rx=0 tx=0).stop 2023-10-18T09:09:31.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.844+0000 7f0f8e0d7640 1 -- 172.21.15.40:0/3629827685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f880ca890 msgr2=0x7f0f88143bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:31.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.844+0000 7f0f8e0d7640 1 --2- 172.21.15.40:0/3629827685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f880ca890 0x7f0f88143bf0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009850 tx=0x7f0f8402da50 comp rx=0 tx=0).stop 2023-10-18T09:09:31.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.844+0000 7f0f8e0d7640 1 -- 172.21.15.40:0/3629827685 shutdown_connections 2023-10-18T09:09:31.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.844+0000 7f0f8e0d7640 1 --2- 172.21.15.40:0/3629827685 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0f6c06ae70 0x7f0f6c06d330 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:31.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.844+0000 7f0f8e0d7640 1 --2- 172.21.15.40:0/3629827685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f88157870 0x7f0f880abac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:31.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.844+0000 7f0f8e0d7640 1 --2- 172.21.15.40:0/3629827685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f880caec0 0x7f0f8814aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:31.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.844+0000 7f0f8e0d7640 1 --2- 172.21.15.40:0/3629827685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f880ca890 0x7f0f88143bf0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:31.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.844+0000 7f0f8e0d7640 1 -- 172.21.15.40:0/3629827685 >> 172.21.15.40:0/3629827685 conn(0x7f0f88069720 msgr2=0x7f0f880aa960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:31.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.844+0000 7f0f8e0d7640 1 -- 172.21.15.40:0/3629827685 shutdown_connections 2023-10-18T09:09:31.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:30.844+0000 7f0f8e0d7640 1 -- 172.21.15.40:0/3629827685 wait complete. 2023-10-18T09:09:31.043 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-18T09:09:31.044 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-18T09:09:31.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.284+0000 7f69c495d640 1 -- 172.21.15.40:0/3552595361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69a4004d00 msgr2=0x7f69a4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:31.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.284+0000 7f69c495d640 1 --2- 172.21.15.40:0/3552595361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69a4004d00 0x7f69a4005180 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f69b8005fd0 tx=0x7f69a40057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:31.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.284+0000 7f69bf59e640 1 -- 172.21.15.40:0/3552595361 reap_dead start 2023-10-18T09:09:31.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.284+0000 7f69c495d640 1 -- 172.21.15.40:0/3552595361 shutdown_connections 2023-10-18T09:09:31.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.284+0000 7f69c495d640 1 -- 172.21.15.40:0/3552595361 >> 172.21.15.40:0/3552595361 conn(0x7f69c0069720 msgr2=0x7f69c0069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.284+0000 7f69c495d640 1 -- 172.21.15.40:0/3552595361 shutdown_connections 2023-10-18T09:09:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.284+0000 7f69c495d640 1 -- 172.21.15.40:0/3552595361 wait complete. 2023-10-18T09:09:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.286+0000 7f69c495d640 1 Processor -- start 2023-10-18T09:09:31.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.286+0000 7f69c495d640 1 -- start start 2023-10-18T09:09:31.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.287+0000 7f69c495d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69a4004d00 0x7f69c01f1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:31.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.287+0000 7f69c495d640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69c0151d40 0x7f69c01521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:31.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.287+0000 7f69c495d640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c00acae0 0x7f69c00acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:31.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.287+0000 7f69c495d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c00ae3b0 con 0x7f69a4004d00 2023-10-18T09:09:31.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.287+0000 7f69c495d640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c00aeeb0 con 0x7f69c00acae0 2023-10-18T09:09:31.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.287+0000 7f69c495d640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c00ad4a0 con 0x7f69c0151d40 2023-10-18T09:09:31.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.287+0000 7f69bed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69a4004d00 0x7f69c01f1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:31.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.287+0000 7f69bed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69a4004d00 0x7f69c01f1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35976/0 (socket says 172.21.15.40:35976) 2023-10-18T09:09:31.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.287+0000 7f69bed9d640 1 -- 172.21.15.40:0/616972549 learned_addr learned my addr 172.21.15.40:0/616972549 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:31.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.288+0000 7f69bf59e640 1 --2- 172.21.15.40:0/616972549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c00acae0 0x7f69c00acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:31.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.288+0000 7f69be59c640 1 --2- 172.21.15.40:0/616972549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69c0151d40 0x7f69c01521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:31.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.288+0000 7f69bed9d640 1 -- 172.21.15.40:0/616972549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69c0151d40 msgr2=0x7f69c01521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:31.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.288+0000 7f69bed9d640 1 --2- 172.21.15.40:0/616972549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69c0151d40 0x7f69c01521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:31.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.288+0000 7f69bed9d640 1 -- 172.21.15.40:0/616972549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c00acae0 msgr2=0x7f69c00acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:31.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.288+0000 7f69bed9d640 1 --2- 172.21.15.40:0/616972549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c00acae0 0x7f69c00acf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:31.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.288+0000 7f69bed9d640 1 -- 172.21.15.40:0/616972549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c01526e0 con 0x7f69a4004d00 2023-10-18T09:09:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.288+0000 7f69bf59e640 1 --2- 172.21.15.40:0/616972549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c00acae0 0x7f69c00acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.288+0000 7f69bed9d640 1 --2- 172.21.15.40:0/616972549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69a4004d00 0x7f69c01f1410 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f69b0004e50 tx=0x7f69b0001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.289+0000 7f69b77fe640 1 -- 172.21.15.40:0/616972549 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b00107d0 con 0x7f69a4004d00 2023-10-18T09:09:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.289+0000 7f69b77fe640 1 -- 172.21.15.40:0/616972549 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f69b0010a40 con 0x7f69a4004d00 2023-10-18T09:09:31.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.289+0000 7f69b77fe640 1 -- 172.21.15.40:0/616972549 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b0010e50 con 0x7f69a4004d00 2023-10-18T09:09:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.289+0000 7f69c495d640 1 -- 172.21.15.40:0/616972549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c00ad730 con 0x7f69a4004d00 2023-10-18T09:09:31.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.289+0000 7f69c495d640 1 -- 172.21.15.40:0/616972549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c00adcf0 con 0x7f69a4004d00 2023-10-18T09:09:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.290+0000 7f69c495d640 1 -- 172.21.15.40:0/616972549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c01575e0 con 0x7f69a4004d00 2023-10-18T09:09:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.291+0000 7f69b77fe640 1 -- 172.21.15.40:0/616972549 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f69b001b460 con 0x7f69a4004d00 2023-10-18T09:09:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.292+0000 7f69b77fe640 1 --2- 172.21.15.40:0/616972549 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f69a806ae40 0x7f69a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.292+0000 7f69b77fe640 1 -- 172.21.15.40:0/616972549 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69b008a080 con 0x7f69a4004d00 2023-10-18T09:09:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.293+0000 7f69be59c640 1 --2- 172.21.15.40:0/616972549 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f69a806ae40 0x7f69a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:31.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.294+0000 7f69be59c640 1 --2- 172.21.15.40:0/616972549 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f69a806ae40 0x7f69a806d300 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0017c0 tx=0x7f69ac001dd0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:31.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.297+0000 7f69b77fe640 1 -- 172.21.15.40:0/616972549 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f69b0055130 con 0x7f69a4004d00 2023-10-18T09:09:31.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.491+0000 7f69c495d640 1 -- 172.21.15.40:0/616972549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f69c0073250 con 0x7f69a4004d00 2023-10-18T09:09:31.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.494+0000 7f69b77fe640 1 -- 172.21.15.40:0/616972549 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v199) v1 ==== 81+0+34026 (secure 0 0 0) 0x7f69b0058750 con 0x7f69a4004d00 2023-10-18T09:09:31.497 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:31.498 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:09:31.498 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 199, 2023-10-18T09:09:31.498 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", 2023-10-18T09:09:31.498 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2023-10-18T08:54:55.880346+0000", 2023-10-18T09:09:31.498 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2023-10-18T09:09:26.219860+0000", 2023-10-18T09:09:31.499 INFO:teuthology.orchestra.run.smithi040.stdout: "last_up_change": "2023-10-18T08:55:04.018251+0000", 2023-10-18T09:09:31.499 INFO:teuthology.orchestra.run.smithi040.stdout: "last_in_change": "2023-10-18T08:55:00.991196+0000", 2023-10-18T09:09:31.499 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-18T09:09:31.499 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_num": 5799936, 2023-10-18T09:09:31.500 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_set": [ 2023-10-18T09:09:31.500 INFO:teuthology.orchestra.run.smithi040.stdout: "pglog_hardlimit", 2023-10-18T09:09:31.500 INFO:teuthology.orchestra.run.smithi040.stdout: "purged_snapdirs", 2023-10-18T09:09:31.500 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_deletes", 2023-10-18T09:09:31.500 INFO:teuthology.orchestra.run.smithi040.stdout: "sortbitwise" 2023-10-18T09:09:31.501 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:31.501 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_version": 4, 2023-10-18T09:09:31.501 INFO:teuthology.orchestra.run.smithi040.stdout: "full_ratio": 0.69999998807907104, 2023-10-18T09:09:31.501 INFO:teuthology.orchestra.run.smithi040.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-18T09:09:31.501 INFO:teuthology.orchestra.run.smithi040.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-18T09:09:31.501 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_snapshot": "", 2023-10-18T09:09:31.502 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_max": 23, 2023-10-18T09:09:31.502 INFO:teuthology.orchestra.run.smithi040.stdout: "max_osd": 8, 2023-10-18T09:09:31.502 INFO:teuthology.orchestra.run.smithi040.stdout: "require_min_compat_client": "luminous", 2023-10-18T09:09:31.502 INFO:teuthology.orchestra.run.smithi040.stdout: "min_compat_client": "jewel", 2023-10-18T09:09:31.502 INFO:teuthology.orchestra.run.smithi040.stdout: "require_osd_release": "reef", 2023-10-18T09:09:31.503 INFO:teuthology.orchestra.run.smithi040.stdout: "allow_crimson": false, 2023-10-18T09:09:31.503 INFO:teuthology.orchestra.run.smithi040.stdout: "pools": [ 2023-10-18T09:09:31.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.503 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 1, 2023-10-18T09:09:31.503 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": ".mgr", 2023-10-18T09:09:31.503 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T08:55:04.972118+0000", 2023-10-18T09:09:31.504 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:31.504 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:31.504 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:31.504 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:31.504 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:31.505 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:31.505 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:31.505 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:31.505 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:31.505 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:31.505 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:31.506 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:31.506 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 1, 2023-10-18T09:09:31.506 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 1, 2023-10-18T09:09:31.506 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 1, 2023-10-18T09:09:31.506 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 1, 2023-10-18T09:09:31.507 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 1, 2023-10-18T09:09:31.507 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:31.507 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:31.507 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:31.507 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:31.507 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:31.508 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:31.508 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:31.508 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.508 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "118", 2023-10-18T09:09:31.508 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:31.509 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:31.509 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:31.509 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:31.509 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:31.509 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:31.510 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:31.510 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:31.510 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:31.510 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:09:31.510 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:31.511 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:31.511 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:31.511 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:31.511 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:31.511 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:31.512 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:31.512 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:31.512 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:31.512 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:31.512 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:31.513 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:31.513 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:31.513 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:31.513 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:31.513 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:31.513 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.514 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:31.514 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:31.514 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:31.514 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:31.514 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:31.515 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:31.515 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:31.515 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:31.515 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:31.515 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:31.516 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:31.516 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:31.516 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_max": 32, 2023-10-18T09:09:31.516 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 1 2023-10-18T09:09:31.516 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.517 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:31.517 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": {} 2023-10-18T09:09:31.517 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.517 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:31.517 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 8, 2023-10-18T09:09:31.517 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 8, 2023-10-18T09:09:31.518 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 0.25, 2023-10-18T09:09:31.518 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:09:31.518 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:09:31.518 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:31.518 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:31.519 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:31.519 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.519 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.519 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.520 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 22, 2023-10-18T09:09:31.520 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_metadata", 2023-10-18T09:09:31.520 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:08:55.675479+0000", 2023-10-18T09:09:31.520 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:31.520 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:31.521 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:31.521 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:31.521 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:31.521 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:31.521 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:31.521 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:31.522 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:31.522 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:31.522 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:31.522 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:31.522 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 32, 2023-10-18T09:09:31.523 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 32, 2023-10-18T09:09:31.523 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 32, 2023-10-18T09:09:31.523 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 32, 2023-10-18T09:09:31.523 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 32, 2023-10-18T09:09:31.523 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:31.523 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:31.524 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:31.524 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:31.524 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:31.524 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:31.524 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:31.525 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.525 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "195", 2023-10-18T09:09:31.525 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:31.525 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:31.525 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:31.525 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:31.526 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:31.526 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:31.526 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:31.526 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:31.526 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:31.527 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 0, 2023-10-18T09:09:31.527 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:31.527 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:31.527 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:31.527 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:31.528 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:31.528 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:31.528 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:31.528 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:31.528 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:31.528 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:31.529 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:31.529 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:31.529 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:31.529 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:31.529 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:31.530 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:31.530 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.530 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:31.530 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:31.530 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:31.531 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:31.531 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:31.531 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:31.531 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:31.531 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:31.531 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:31.532 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:31.532 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:31.532 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:31.532 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_bias": 4, 2023-10-18T09:09:31.532 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 16, 2023-10-18T09:09:31.532 INFO:teuthology.orchestra.run.smithi040.stdout: "recovery_priority": 5 2023-10-18T09:09:31.533 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.533 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:31.533 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:09:31.533 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": "cephfs" 2023-10-18T09:09:31.533 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.534 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.534 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:31.534 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 2, 2023-10-18T09:09:31.534 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 2, 2023-10-18T09:09:31.534 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:09:31.534 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 2, 2023-10-18T09:09:31.535 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 2, 2023-10-18T09:09:31.535 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:31.535 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:31.535 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:31.535 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.536 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.536 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.536 INFO:teuthology.orchestra.run.smithi040.stdout: "pool": 23, 2023-10-18T09:09:31.536 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_name": "cephfs_data", 2023-10-18T09:09:31.536 INFO:teuthology.orchestra.run.smithi040.stdout: "create_time": "2023-10-18T09:08:56.761888+0000", 2023-10-18T09:09:31.536 INFO:teuthology.orchestra.run.smithi040.stdout: "flags": 1, 2023-10-18T09:09:31.537 INFO:teuthology.orchestra.run.smithi040.stdout: "flags_names": "hashpspool", 2023-10-18T09:09:31.537 INFO:teuthology.orchestra.run.smithi040.stdout: "type": 1, 2023-10-18T09:09:31.537 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 2, 2023-10-18T09:09:31.537 INFO:teuthology.orchestra.run.smithi040.stdout: "min_size": 1, 2023-10-18T09:09:31.537 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_rule": 0, 2023-10-18T09:09:31.538 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_count": 0, 2023-10-18T09:09:31.538 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_target": 0, 2023-10-18T09:09:31.538 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_barrier": 0, 2023-10-18T09:09:31.538 INFO:teuthology.orchestra.run.smithi040.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-18T09:09:31.538 INFO:teuthology.orchestra.run.smithi040.stdout: "object_hash": 2, 2023-10-18T09:09:31.539 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_autoscale_mode": "off", 2023-10-18T09:09:31.539 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num": 64, 2023-10-18T09:09:31.539 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num": 64, 2023-10-18T09:09:31.539 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_placement_num_target": 64, 2023-10-18T09:09:31.539 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_target": 64, 2023-10-18T09:09:31.540 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_pending": 64, 2023-10-18T09:09:31.540 INFO:teuthology.orchestra.run.smithi040.stdout: "last_pg_merge_meta": { 2023-10-18T09:09:31.540 INFO:teuthology.orchestra.run.smithi040.stdout: "source_pgid": "0.0", 2023-10-18T09:09:31.540 INFO:teuthology.orchestra.run.smithi040.stdout: "ready_epoch": 0, 2023-10-18T09:09:31.541 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_started": 0, 2023-10-18T09:09:31.541 INFO:teuthology.orchestra.run.smithi040.stdout: "last_epoch_clean": 0, 2023-10-18T09:09:31.541 INFO:teuthology.orchestra.run.smithi040.stdout: "source_version": "0'0", 2023-10-18T09:09:31.541 INFO:teuthology.orchestra.run.smithi040.stdout: "target_version": "0'0" 2023-10-18T09:09:31.541 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.541 INFO:teuthology.orchestra.run.smithi040.stdout: "last_change": "199", 2023-10-18T09:09:31.542 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend": "0", 2023-10-18T09:09:31.542 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-18T09:09:31.542 INFO:teuthology.orchestra.run.smithi040.stdout: "last_force_op_resend_preluminous": "0", 2023-10-18T09:09:31.542 INFO:teuthology.orchestra.run.smithi040.stdout: "auid": 0, 2023-10-18T09:09:31.542 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_mode": "selfmanaged", 2023-10-18T09:09:31.543 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_seq": 0, 2023-10-18T09:09:31.543 INFO:teuthology.orchestra.run.smithi040.stdout: "snap_epoch": 0, 2023-10-18T09:09:31.543 INFO:teuthology.orchestra.run.smithi040.stdout: "pool_snaps": [], 2023-10-18T09:09:31.543 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps": "[]", 2023-10-18T09:09:31.543 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_bytes": 33554432, 2023-10-18T09:09:31.544 INFO:teuthology.orchestra.run.smithi040.stdout: "quota_max_objects": 0, 2023-10-18T09:09:31.544 INFO:teuthology.orchestra.run.smithi040.stdout: "tiers": [], 2023-10-18T09:09:31.544 INFO:teuthology.orchestra.run.smithi040.stdout: "tier_of": -1, 2023-10-18T09:09:31.544 INFO:teuthology.orchestra.run.smithi040.stdout: "read_tier": -1, 2023-10-18T09:09:31.544 INFO:teuthology.orchestra.run.smithi040.stdout: "write_tier": -1, 2023-10-18T09:09:31.544 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_mode": "none", 2023-10-18T09:09:31.545 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_bytes": 0, 2023-10-18T09:09:31.545 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_objects": 0, 2023-10-18T09:09:31.545 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-18T09:09:31.545 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-18T09:09:31.545 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-18T09:09:31.546 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_flush_age": 0, 2023-10-18T09:09:31.546 INFO:teuthology.orchestra.run.smithi040.stdout: "cache_min_evict_age": 0, 2023-10-18T09:09:31.546 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profile": "", 2023-10-18T09:09:31.546 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_params": { 2023-10-18T09:09:31.546 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "none" 2023-10-18T09:09:31.547 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.547 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_period": 0, 2023-10-18T09:09:31.547 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_count": 0, 2023-10-18T09:09:31.547 INFO:teuthology.orchestra.run.smithi040.stdout: "use_gmt_hitset": true, 2023-10-18T09:09:31.547 INFO:teuthology.orchestra.run.smithi040.stdout: "min_read_recency_for_promote": 0, 2023-10-18T09:09:31.547 INFO:teuthology.orchestra.run.smithi040.stdout: "min_write_recency_for_promote": 0, 2023-10-18T09:09:31.548 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_grade_decay_rate": 0, 2023-10-18T09:09:31.548 INFO:teuthology.orchestra.run.smithi040.stdout: "hit_set_search_last_n": 0, 2023-10-18T09:09:31.548 INFO:teuthology.orchestra.run.smithi040.stdout: "grade_table": [], 2023-10-18T09:09:31.548 INFO:teuthology.orchestra.run.smithi040.stdout: "stripe_width": 0, 2023-10-18T09:09:31.548 INFO:teuthology.orchestra.run.smithi040.stdout: "expected_num_objects": 0, 2023-10-18T09:09:31.549 INFO:teuthology.orchestra.run.smithi040.stdout: "fast_read": false, 2023-10-18T09:09:31.549 INFO:teuthology.orchestra.run.smithi040.stdout: "options": { 2023-10-18T09:09:31.549 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_num_min": 64, 2023-10-18T09:09:31.549 INFO:teuthology.orchestra.run.smithi040.stdout: "target_size_ratio": 0.89999997615814209 2023-10-18T09:09:31.550 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.550 INFO:teuthology.orchestra.run.smithi040.stdout: "application_metadata": { 2023-10-18T09:09:31.550 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs": { 2023-10-18T09:09:31.550 INFO:teuthology.orchestra.run.smithi040.stdout: "data": "cephfs" 2023-10-18T09:09:31.550 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.551 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.551 INFO:teuthology.orchestra.run.smithi040.stdout: "read_balance": { 2023-10-18T09:09:31.551 INFO:teuthology.orchestra.run.smithi040.stdout: "score_acting": 1.3799999952316284, 2023-10-18T09:09:31.551 INFO:teuthology.orchestra.run.smithi040.stdout: "score_stable": 1.3799999952316284, 2023-10-18T09:09:31.551 INFO:teuthology.orchestra.run.smithi040.stdout: "optimal_score": 1, 2023-10-18T09:09:31.551 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-18T09:09:31.552 INFO:teuthology.orchestra.run.smithi040.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-18T09:09:31.552 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity_weighted": 1, 2023-10-18T09:09:31.552 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity": 1, 2023-10-18T09:09:31.552 INFO:teuthology.orchestra.run.smithi040.stdout: "average_primary_affinity_weighted": 1 2023-10-18T09:09:31.552 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.553 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.553 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:31.553 INFO:teuthology.orchestra.run.smithi040.stdout: "osds": [ 2023-10-18T09:09:31.553 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.553 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:09:31.553 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:31.554 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:31.554 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:31.554 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:31.554 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:31.554 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:31.555 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:31.555 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:31.555 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:31.555 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:31.555 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:31.556 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:31.556 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.556 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.556 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.556 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6800", 2023-10-18T09:09:31.556 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:31.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.557 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.557 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.557 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6801", 2023-10-18T09:09:31.557 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:31.558 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.558 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.558 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:31.558 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.559 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.559 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6802", 2023-10-18T09:09:31.559 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:31.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.560 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.560 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6803", 2023-10-18T09:09:31.560 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:31.560 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.560 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.561 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:31.561 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.561 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.561 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.562 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6806", 2023-10-18T09:09:31.562 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:31.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.562 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.562 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.562 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6807", 2023-10-18T09:09:31.563 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:31.563 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.563 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.563 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:31.564 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.564 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.564 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.564 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6804", 2023-10-18T09:09:31.564 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:31.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.565 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.565 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.565 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6805", 2023-10-18T09:09:31.566 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1070603306 2023-10-18T09:09:31.566 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.566 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.566 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.566 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6801/1070603306", 2023-10-18T09:09:31.567 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6803/1070603306", 2023-10-18T09:09:31.567 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6807/1070603306", 2023-10-18T09:09:31.567 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6805/1070603306", 2023-10-18T09:09:31.567 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:31.567 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:31.568 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:31.568 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.568 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.568 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.568 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:09:31.568 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:31.569 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:31.569 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:31.569 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:31.569 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:31.569 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:31.570 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:31.570 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:31.570 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:31.570 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:31.570 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:31.571 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:31.571 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.571 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.571 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.571 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6808", 2023-10-18T09:09:31.571 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:31.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.572 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.572 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.572 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6809", 2023-10-18T09:09:31.572 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:31.573 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.573 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.573 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:31.573 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.573 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.574 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.574 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6810", 2023-10-18T09:09:31.574 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:31.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.575 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.575 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6811", 2023-10-18T09:09:31.575 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:31.576 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.576 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.576 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.576 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:31.576 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.577 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.577 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.577 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6814", 2023-10-18T09:09:31.578 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:31.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.578 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.579 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.579 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6815", 2023-10-18T09:09:31.579 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:31.579 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.580 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.580 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.580 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:31.581 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.581 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.581 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.581 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6812", 2023-10-18T09:09:31.581 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:31.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.582 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.582 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.582 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6813", 2023-10-18T09:09:31.582 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2762063371 2023-10-18T09:09:31.583 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.583 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.583 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6809/2762063371", 2023-10-18T09:09:31.583 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6811/2762063371", 2023-10-18T09:09:31.584 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6815/2762063371", 2023-10-18T09:09:31.584 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6813/2762063371", 2023-10-18T09:09:31.584 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:31.584 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:31.584 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:31.584 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.585 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.585 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.585 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:09:31.585 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:31.585 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:31.586 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:31.586 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:31.586 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:31.586 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:31.586 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:31.587 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:31.587 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:31.587 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:31.587 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:31.587 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:31.587 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.588 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.588 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.588 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6823", 2023-10-18T09:09:31.588 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:31.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.589 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.589 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.589 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6825", 2023-10-18T09:09:31.589 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:31.589 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.589 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.590 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:31.590 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.590 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.590 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.591 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6826", 2023-10-18T09:09:31.591 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:31.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.591 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.592 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6827", 2023-10-18T09:09:31.592 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:31.592 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.592 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.592 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:31.593 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.593 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.593 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6830", 2023-10-18T09:09:31.593 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:31.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.594 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.594 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.594 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6831", 2023-10-18T09:09:31.594 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:31.595 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.595 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.595 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:31.595 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.596 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.596 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6828", 2023-10-18T09:09:31.596 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:31.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.597 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.597 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6829", 2023-10-18T09:09:31.597 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1187944209 2023-10-18T09:09:31.597 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.597 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.598 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6825/1187944209", 2023-10-18T09:09:31.598 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6827/1187944209", 2023-10-18T09:09:31.598 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6831/1187944209", 2023-10-18T09:09:31.598 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6829/1187944209", 2023-10-18T09:09:31.598 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:31.599 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:31.599 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:31.599 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.600 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:09:31.600 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:31.600 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:31.600 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:31.600 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:31.601 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:31.601 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:31.601 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:31.601 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:31.601 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:31.602 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:31.602 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:31.602 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:31.602 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.603 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.603 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6816", 2023-10-18T09:09:31.603 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:31.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.603 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.604 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6817", 2023-10-18T09:09:31.604 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:31.604 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.604 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.604 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:31.605 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.605 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.605 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6818", 2023-10-18T09:09:31.605 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:31.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.606 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.606 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6819", 2023-10-18T09:09:31.606 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:31.607 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.607 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.607 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:31.607 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.608 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.608 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6822", 2023-10-18T09:09:31.608 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:31.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.609 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.609 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6824", 2023-10-18T09:09:31.609 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:31.609 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.609 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.610 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:31.610 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.610 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.611 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6820", 2023-10-18T09:09:31.611 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:31.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.611 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.611 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.611 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6821", 2023-10-18T09:09:31.612 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3903517270 2023-10-18T09:09:31.612 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.612 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.612 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6817/3903517270", 2023-10-18T09:09:31.613 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.40:6819/3903517270", 2023-10-18T09:09:31.613 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.40:6824/3903517270", 2023-10-18T09:09:31.613 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.40:6821/3903517270", 2023-10-18T09:09:31.613 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:31.613 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:31.613 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:31.614 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.614 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:09:31.614 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:31.615 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:31.615 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:31.615 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:31.615 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:31.615 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:31.615 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:31.616 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:31.616 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:31.616 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:31.616 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:31.616 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:31.616 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.617 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6810", 2023-10-18T09:09:31.617 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:31.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.618 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.618 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6811", 2023-10-18T09:09:31.618 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:31.618 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.618 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.619 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:31.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.620 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.620 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6812", 2023-10-18T09:09:31.620 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:31.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.621 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.621 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6813", 2023-10-18T09:09:31.622 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:31.622 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.622 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.623 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:31.623 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.624 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.624 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6816", 2023-10-18T09:09:31.625 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:31.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.626 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.626 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6817", 2023-10-18T09:09:31.626 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:31.627 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.627 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.628 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:31.628 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.629 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.629 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.629 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6814", 2023-10-18T09:09:31.630 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:31.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.630 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.631 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.631 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6815", 2023-10-18T09:09:31.632 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1718994813 2023-10-18T09:09:31.632 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.632 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.633 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6811/1718994813", 2023-10-18T09:09:31.633 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6813/1718994813", 2023-10-18T09:09:31.634 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6817/1718994813", 2023-10-18T09:09:31.634 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6815/1718994813", 2023-10-18T09:09:31.634 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:31.635 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:31.635 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:31.635 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.636 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.636 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:09:31.637 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:31.637 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:31.638 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:31.638 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:31.638 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:31.639 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:31.639 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:31.639 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:31.640 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:31.640 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:31.640 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:31.641 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:31.641 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.642 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.642 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6802", 2023-10-18T09:09:31.642 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:31.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.643 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.643 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6803", 2023-10-18T09:09:31.643 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:31.643 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.644 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.644 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:31.644 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.644 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.644 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.645 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6804", 2023-10-18T09:09:31.645 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:31.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.645 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.645 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6805", 2023-10-18T09:09:31.646 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:31.646 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.646 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.647 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:31.647 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.647 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.648 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6808", 2023-10-18T09:09:31.648 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:31.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.648 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.648 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.648 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6809", 2023-10-18T09:09:31.649 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:31.649 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.649 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.649 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:31.650 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.650 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.650 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.650 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6806", 2023-10-18T09:09:31.650 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:31.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.651 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.651 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.651 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6807", 2023-10-18T09:09:31.651 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1833904789 2023-10-18T09:09:31.652 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.652 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.652 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.652 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6803/1833904789", 2023-10-18T09:09:31.652 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6805/1833904789", 2023-10-18T09:09:31.652 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6809/1833904789", 2023-10-18T09:09:31.653 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6807/1833904789", 2023-10-18T09:09:31.653 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:31.653 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:31.653 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:31.653 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.653 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.654 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.654 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:09:31.654 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:31.654 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:31.654 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:31.655 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:31.655 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:31.655 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:31.655 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:31.655 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:31.655 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:31.656 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:31.656 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:31.656 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:31.656 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.656 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.656 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.657 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6818", 2023-10-18T09:09:31.657 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:31.657 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.657 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.657 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.657 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6819", 2023-10-18T09:09:31.658 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:31.658 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.658 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.658 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.658 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:31.659 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.659 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.659 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.659 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6820", 2023-10-18T09:09:31.659 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:31.660 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.660 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.660 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.660 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6821", 2023-10-18T09:09:31.660 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:31.661 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.661 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.661 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.661 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:31.661 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.662 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.662 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.662 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6825", 2023-10-18T09:09:31.662 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:31.662 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.663 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.663 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.663 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6827", 2023-10-18T09:09:31.663 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:31.663 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.663 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.664 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.664 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:31.664 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.664 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.664 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.664 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6822", 2023-10-18T09:09:31.665 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:31.665 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.665 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.665 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.666 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6823", 2023-10-18T09:09:31.666 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1988482345 2023-10-18T09:09:31.666 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.666 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.667 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.667 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6819/1988482345", 2023-10-18T09:09:31.667 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6821/1988482345", 2023-10-18T09:09:31.667 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6827/1988482345", 2023-10-18T09:09:31.668 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6823/1988482345", 2023-10-18T09:09:31.668 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:31.668 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:31.668 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:31.668 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.669 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.669 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:09:31.669 INFO:teuthology.orchestra.run.smithi040.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-18T09:09:31.670 INFO:teuthology.orchestra.run.smithi040.stdout: "up": 1, 2023-10-18T09:09:31.670 INFO:teuthology.orchestra.run.smithi040.stdout: "in": 1, 2023-10-18T09:09:31.670 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 1, 2023-10-18T09:09:31.670 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_affinity": 1, 2023-10-18T09:09:31.670 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_begin": 0, 2023-10-18T09:09:31.671 INFO:teuthology.orchestra.run.smithi040.stdout: "last_clean_end": 0, 2023-10-18T09:09:31.671 INFO:teuthology.orchestra.run.smithi040.stdout: "up_from": 13, 2023-10-18T09:09:31.671 INFO:teuthology.orchestra.run.smithi040.stdout: "up_thru": 194, 2023-10-18T09:09:31.671 INFO:teuthology.orchestra.run.smithi040.stdout: "down_at": 0, 2023-10-18T09:09:31.672 INFO:teuthology.orchestra.run.smithi040.stdout: "lost_at": 0, 2023-10-18T09:09:31.672 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2023-10-18T09:09:31.672 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.672 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.672 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.673 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6824", 2023-10-18T09:09:31.673 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:31.673 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.673 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.673 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.674 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6826", 2023-10-18T09:09:31.674 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:31.674 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.674 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.674 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.675 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addrs": { 2023-10-18T09:09:31.675 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.675 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.675 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.675 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6828", 2023-10-18T09:09:31.676 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:31.676 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.676 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.676 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.677 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6829", 2023-10-18T09:09:31.677 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:31.677 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.677 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.677 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.678 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addrs": { 2023-10-18T09:09:31.678 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.678 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.678 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.678 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6832", 2023-10-18T09:09:31.679 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:31.679 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.679 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.679 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.679 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6833", 2023-10-18T09:09:31.680 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:31.680 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.680 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.680 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.680 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addrs": { 2023-10-18T09:09:31.681 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2023-10-18T09:09:31.681 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.681 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2023-10-18T09:09:31.681 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6830", 2023-10-18T09:09:31.681 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:31.682 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.682 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.682 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-18T09:09:31.682 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.144:6831", 2023-10-18T09:09:31.682 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3907866946 2023-10-18T09:09:31.683 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.683 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.683 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.683 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.144:6826/3907866946", 2023-10-18T09:09:31.684 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "172.21.15.144:6829/3907866946", 2023-10-18T09:09:31.684 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_back_addr": "172.21.15.144:6833/3907866946", 2023-10-18T09:09:31.684 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_front_addr": "172.21.15.144:6831/3907866946", 2023-10-18T09:09:31.684 INFO:teuthology.orchestra.run.smithi040.stdout: "state": [ 2023-10-18T09:09:31.684 INFO:teuthology.orchestra.run.smithi040.stdout: "exists", 2023-10-18T09:09:31.685 INFO:teuthology.orchestra.run.smithi040.stdout: "up" 2023-10-18T09:09:31.685 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2023-10-18T09:09:31.685 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.685 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:31.685 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_xinfo": [ 2023-10-18T09:09:31.686 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.686 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 0, 2023-10-18T09:09:31.686 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:31.686 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:31.686 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:31.687 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:31.687 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:31.687 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.261552+0000", 2023-10-18T09:09:31.687 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:31.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.688 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.688 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 1, 2023-10-18T09:09:31.688 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:31.688 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:31.688 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:31.689 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:31.689 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:31.689 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253175+0000", 2023-10-18T09:09:31.689 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:31.690 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.690 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.690 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 2, 2023-10-18T09:09:31.690 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:31.690 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:31.691 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:31.691 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:31.691 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:31.691 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.353254+0000", 2023-10-18T09:09:31.691 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:31.692 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.692 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.692 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 3, 2023-10-18T09:09:31.692 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:31.692 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:31.693 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:31.693 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:31.693 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:31.693 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.272925+0000", 2023-10-18T09:09:31.693 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:31.694 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.694 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.694 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 4, 2023-10-18T09:09:31.694 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:31.694 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:31.695 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:31.695 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:31.695 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:31.695 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.281007+0000", 2023-10-18T09:09:31.695 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:31.696 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.696 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.696 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 5, 2023-10-18T09:09:31.696 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:31.696 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:31.696 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:31.697 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:31.697 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:31.697 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.230949+0000", 2023-10-18T09:09:31.697 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:31.698 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.698 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.698 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 6, 2023-10-18T09:09:31.698 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:31.698 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:31.699 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:31.699 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:31.699 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:31.699 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.204839+0000", 2023-10-18T09:09:31.699 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:31.700 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.700 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-18T09:09:31.700 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": 7, 2023-10-18T09:09:31.700 INFO:teuthology.orchestra.run.smithi040.stdout: "down_stamp": "0.000000", 2023-10-18T09:09:31.700 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_probability": 0, 2023-10-18T09:09:31.701 INFO:teuthology.orchestra.run.smithi040.stdout: "laggy_interval": 0, 2023-10-18T09:09:31.701 INFO:teuthology.orchestra.run.smithi040.stdout: "features": 4540138322906710015, 2023-10-18T09:09:31.701 INFO:teuthology.orchestra.run.smithi040.stdout: "old_weight": 0, 2023-10-18T09:09:31.701 INFO:teuthology.orchestra.run.smithi040.stdout: "last_purged_snaps_scrub": "2023-10-18T08:55:02.253671+0000", 2023-10-18T09:09:31.701 INFO:teuthology.orchestra.run.smithi040.stdout: "dead_epoch": 0 2023-10-18T09:09:31.701 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.702 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2023-10-18T09:09:31.702 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap": [], 2023-10-18T09:09:31.702 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_items": [], 2023-10-18T09:09:31.702 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_upmap_primaries": [], 2023-10-18T09:09:31.703 INFO:teuthology.orchestra.run.smithi040.stdout: "pg_temp": [], 2023-10-18T09:09:31.703 INFO:teuthology.orchestra.run.smithi040.stdout: "primary_temp": [], 2023-10-18T09:09:31.703 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklist": {}, 2023-10-18T09:09:31.703 INFO:teuthology.orchestra.run.smithi040.stdout: "range_blocklist": {}, 2023-10-18T09:09:31.703 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_profiles": { 2023-10-18T09:09:31.703 INFO:teuthology.orchestra.run.smithi040.stdout: "default": { 2023-10-18T09:09:31.704 INFO:teuthology.orchestra.run.smithi040.stdout: "crush-failure-domain": "osd", 2023-10-18T09:09:31.704 INFO:teuthology.orchestra.run.smithi040.stdout: "k": "2", 2023-10-18T09:09:31.704 INFO:teuthology.orchestra.run.smithi040.stdout: "m": "1", 2023-10-18T09:09:31.704 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin": "jerasure", 2023-10-18T09:09:31.704 INFO:teuthology.orchestra.run.smithi040.stdout: "technique": "reed_sol_van" 2023-10-18T09:09:31.705 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-18T09:09:31.705 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_snaps_queue": [], 2023-10-18T09:09:31.705 INFO:teuthology.orchestra.run.smithi040.stdout: "new_removed_snaps": [], 2023-10-18T09:09:31.705 INFO:teuthology.orchestra.run.smithi040.stdout: "new_purged_snaps": [], 2023-10-18T09:09:31.705 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_node_flags": {}, 2023-10-18T09:09:31.706 INFO:teuthology.orchestra.run.smithi040.stdout: "device_class_flags": {}, 2023-10-18T09:09:31.706 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": { 2023-10-18T09:09:31.706 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_enabled": false, 2023-10-18T09:09:31.706 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_bucket_count": 0, 2023-10-18T09:09:31.707 INFO:teuthology.orchestra.run.smithi040.stdout: "degraded_stretch_mode": 0, 2023-10-18T09:09:31.707 INFO:teuthology.orchestra.run.smithi040.stdout: "recovering_stretch_mode": 0, 2023-10-18T09:09:31.707 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode_bucket": 0 2023-10-18T09:09:31.707 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-18T09:09:31.707 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:09:31.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.497+0000 7f69c495d640 1 -- 172.21.15.40:0/616972549 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f69a806ae40 msgr2=0x7f69a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:31.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.497+0000 7f69c495d640 1 --2- 172.21.15.40:0/616972549 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f69a806ae40 0x7f69a806d300 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0017c0 tx=0x7f69ac001dd0 comp rx=0 tx=0).stop 2023-10-18T09:09:31.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.497+0000 7f69c495d640 1 -- 172.21.15.40:0/616972549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69a4004d00 msgr2=0x7f69c01f1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:31.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.497+0000 7f69c495d640 1 --2- 172.21.15.40:0/616972549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69a4004d00 0x7f69c01f1410 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f69b0004e50 tx=0x7f69b0001630 comp rx=0 tx=0).stop 2023-10-18T09:09:31.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.497+0000 7f69c495d640 1 -- 172.21.15.40:0/616972549 shutdown_connections 2023-10-18T09:09:31.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.497+0000 7f69c495d640 1 --2- 172.21.15.40:0/616972549 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f69a806ae40 0x7f69a806d300 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:31.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.497+0000 7f69c495d640 1 --2- 172.21.15.40:0/616972549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c00acae0 0x7f69c00acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:31.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.497+0000 7f69c495d640 1 --2- 172.21.15.40:0/616972549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69c0151d40 0x7f69c01521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:31.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.497+0000 7f69c495d640 1 --2- 172.21.15.40:0/616972549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69a4004d00 0x7f69c01f1410 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:31.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.497+0000 7f69c495d640 1 -- 172.21.15.40:0/616972549 >> 172.21.15.40:0/616972549 conn(0x7f69c0069720 msgr2=0x7f69c0159d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:31.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.497+0000 7f69c495d640 1 -- 172.21.15.40:0/616972549 shutdown_connections 2023-10-18T09:09:31.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.497+0000 7f69c495d640 1 -- 172.21.15.40:0/616972549 wait complete. 2023-10-18T09:09:31.712 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:09:31.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.947+0000 7ff98ea4f640 1 -- 172.21.15.40:0/1257778815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9880b12d0 msgr2=0x7ff9881635b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:31.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.947+0000 7ff98ea4f640 1 --2- 172.21.15.40:0/1257778815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9880b12d0 0x7ff9881635b0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7ff980005fd0 tx=0x7ff98002c0e0 comp rx=0 tx=0).stop 2023-10-18T09:09:31.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.947+0000 7ff98ea4f640 1 -- 172.21.15.40:0/1257778815 shutdown_connections 2023-10-18T09:09:31.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.947+0000 7ff98ea4f640 1 --2- 172.21.15.40:0/1257778815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9880b12d0 0x7ff9881635b0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:31.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.947+0000 7ff98ea4f640 1 -- 172.21.15.40:0/1257778815 >> 172.21.15.40:0/1257778815 conn(0x7ff98806cee0 msgr2=0x7ff988069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:31.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.947+0000 7ff98ea4f640 1 -- 172.21.15.40:0/1257778815 shutdown_connections 2023-10-18T09:09:31.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.947+0000 7ff98ea4f640 1 -- 172.21.15.40:0/1257778815 wait complete. 2023-10-18T09:09:31.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.949+0000 7ff98ea4f640 1 Processor -- start 2023-10-18T09:09:31.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.949+0000 7ff98ea4f640 1 -- start start 2023-10-18T09:09:31.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.951+0000 7ff98ea4f640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff98806d710 0x7ff9881495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:31.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.951+0000 7ff98ea4f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9880b12d0 0x7ff9880c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:31.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.951+0000 7ff98ea4f640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9881586a0 0x7ff98814eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:31.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.951+0000 7ff98ea4f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff988139670 con 0x7ff9880b12d0 2023-10-18T09:09:31.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.951+0000 7ff98ea4f640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9881387a0 con 0x7ff9881586a0 2023-10-18T09:09:31.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.951+0000 7ff98ea4f640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98814a090 con 0x7ff98806d710 2023-10-18T09:09:31.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.951+0000 7ff9877fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9880b12d0 0x7ff9880c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:31.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.951+0000 7ff9877fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9880b12d0 0x7ff9880c9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36008/0 (socket says 172.21.15.40:36008) 2023-10-18T09:09:31.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.951+0000 7ff9877fe640 1 -- 172.21.15.40:0/3202641419 learned_addr learned my addr 172.21.15.40:0/3202641419 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:31.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.951+0000 7ff987fff640 1 --2- 172.21.15.40:0/3202641419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff98806d710 0x7ff9881495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:31.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.951+0000 7ff98cfec640 1 --2- 172.21.15.40:0/3202641419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9881586a0 0x7ff98814eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:31.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.952+0000 7ff9877fe640 1 -- 172.21.15.40:0/3202641419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff98806d710 msgr2=0x7ff9881495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:31.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.952+0000 7ff9877fe640 1 --2- 172.21.15.40:0/3202641419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff98806d710 0x7ff9881495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:31.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.952+0000 7ff9877fe640 1 -- 172.21.15.40:0/3202641419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9881586a0 msgr2=0x7ff98814eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:31.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.952+0000 7ff9877fe640 1 --2- 172.21.15.40:0/3202641419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9881586a0 0x7ff98814eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:31.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.952+0000 7ff9877fe640 1 -- 172.21.15.40:0/3202641419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff980009840 con 0x7ff9880b12d0 2023-10-18T09:09:31.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.952+0000 7ff9877fe640 1 --2- 172.21.15.40:0/3202641419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9880b12d0 0x7ff9880c9fc0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00c430 tx=0x7ff97c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:31.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.952+0000 7ff984ff9640 1 -- 172.21.15.40:0/3202641419 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c011790 con 0x7ff9880b12d0 2023-10-18T09:09:31.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.952+0000 7ff984ff9640 1 -- 172.21.15.40:0/3202641419 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff97c011a00 con 0x7ff9880b12d0 2023-10-18T09:09:31.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.952+0000 7ff984ff9640 1 -- 172.21.15.40:0/3202641419 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c011e10 con 0x7ff9880b12d0 2023-10-18T09:09:31.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.952+0000 7ff98ea4f640 1 -- 172.21.15.40:0/3202641419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9881389f0 con 0x7ff9880b12d0 2023-10-18T09:09:31.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.953+0000 7ff98ea4f640 1 -- 172.21.15.40:0/3202641419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff988138fb0 con 0x7ff9880b12d0 2023-10-18T09:09:31.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.954+0000 7ff98ea4f640 1 -- 172.21.15.40:0/3202641419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9880b0fa0 con 0x7ff9880b12d0 2023-10-18T09:09:31.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.955+0000 7ff984ff9640 1 -- 172.21.15.40:0/3202641419 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff97c01c460 con 0x7ff9880b12d0 2023-10-18T09:09:31.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.957+0000 7ff984ff9640 1 --2- 172.21.15.40:0/3202641419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff97406ae40 0x7ff97406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:31.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.957+0000 7ff984ff9640 1 -- 172.21.15.40:0/3202641419 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff97c089860 con 0x7ff9880b12d0 2023-10-18T09:09:31.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.958+0000 7ff987fff640 1 --2- 172.21.15.40:0/3202641419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff97406ae40 0x7ff97406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:31.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.959+0000 7ff987fff640 1 --2- 172.21.15.40:0/3202641419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff97406ae40 0x7ff97406d300 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff9880aecc0 tx=0x7ff978009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:31.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:31.961+0000 7ff984ff9640 1 -- 172.21.15.40:0/3202641419 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff97c054870 con 0x7ff9880b12d0 2023-10-18T09:09:32.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:32.184+0000 7ff98ea4f640 1 -- 172.21.15.40:0/3202641419 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff988066fd0 con 0x7ff9880b12d0 2023-10-18T09:09:32.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:32.185+0000 7ff984ff9640 1 -- 172.21.15.40:0/3202641419 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4923 (secure 0 0 0) 0x7ff97c057e90 con 0x7ff9880b12d0 2023-10-18T09:09:32.188 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:32.188 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2038256442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2038256442},{"type":"v1","addr":"172.21.15.144:6836","nonce":2038256442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3006484507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3006484507},{"type":"v1","addr":"172.21.15.144:6837","nonce":3006484507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1542000032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1542000032},{"type":"v1","addr":"172.21.15.40:6834","nonce":1542000032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:08:57.731109+0000","modified":"2023-10-18T09:09:00.823289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3976883051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3976883051},{"type":"v1","addr":"172.21.15.40:6835","nonce":3976883051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-18T09:09:32.188 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 90 2023-10-18T09:09:32.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:32.188+0000 7ff98ea4f640 1 -- 172.21.15.40:0/3202641419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff97406ae40 msgr2=0x7ff97406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:32.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:32.188+0000 7ff98ea4f640 1 --2- 172.21.15.40:0/3202641419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff97406ae40 0x7ff97406d300 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff9880aecc0 tx=0x7ff978009040 comp rx=0 tx=0).stop 2023-10-18T09:09:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:32.188+0000 7ff98ea4f640 1 -- 172.21.15.40:0/3202641419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9880b12d0 msgr2=0x7ff9880c9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:32.188+0000 7ff98ea4f640 1 --2- 172.21.15.40:0/3202641419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9880b12d0 0x7ff9880c9fc0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00c430 tx=0x7ff97c00c8f0 comp rx=0 tx=0).stop 2023-10-18T09:09:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:32.188+0000 7ff98ea4f640 1 -- 172.21.15.40:0/3202641419 shutdown_connections 2023-10-18T09:09:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:32.188+0000 7ff98ea4f640 1 --2- 172.21.15.40:0/3202641419 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff97406ae40 0x7ff97406d300 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:32.188+0000 7ff98ea4f640 1 --2- 172.21.15.40:0/3202641419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9881586a0 0x7ff98814eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:32.188+0000 7ff98ea4f640 1 --2- 172.21.15.40:0/3202641419 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9880b12d0 0x7ff9880c9fc0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:32.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:32.188+0000 7ff98ea4f640 1 --2- 172.21.15.40:0/3202641419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff98806d710 0x7ff9881495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:32.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:32.188+0000 7ff98ea4f640 1 -- 172.21.15.40:0/3202641419 >> 172.21.15.40:0/3202641419 conn(0x7ff98806cee0 msgr2=0x7ff988069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:32.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:32.188+0000 7ff98ea4f640 1 -- 172.21.15.40:0/3202641419 shutdown_connections 2023-10-18T09:09:32.192 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:32.188+0000 7ff98ea4f640 1 -- 172.21.15.40:0/3202641419 wait complete. 2023-10-18T09:09:32.208 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-18T09:09:32.440 INFO:teuthology.orchestra.run.smithi040.stdout:{"cluster_fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","whoami":0,"id":9282,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":34.155412974000001,"mdsmap_epoch":90,"osdmap_epoch":199,"osdmap_epoch_barrier":198,"uptime":40.897266180000003} 2023-10-18T09:09:32.440 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "029855e3-538a-4478-b323-f54ab0f3a56d", "fs_name": "cephfs", "id": 9282, "mdsmap_epoch": 90, "osdmap_epoch": 199, "osdmap_epoch_barrier": 198, "rank_uptime": 34.155412974, "state": "up:active", "uptime": 40.89726618, "want_state": "up:active", "whoami": 0 } 2023-10-18T09:09:32.441 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-18T09:09:32.441 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:09:32.442 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:09:32.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:32.498 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-18T09:09:33.005 INFO:teuthology.orchestra.run.smithi040.stderr:50+0 records in 2023-10-18T09:09:33.006 INFO:teuthology.orchestra.run.smithi040.stderr:50+0 records out 2023-10-18T09:09:33.006 INFO:teuthology.orchestra.run.smithi040.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.448624 s, 117 MB/s 2023-10-18T09:09:33.007 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-18T09:09:33.008 DEBUG:teuthology.orchestra.run.smithi040:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:09:33.117 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-18T09:09:33.248 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-18T09:09:33.248 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:09:33.249 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:09:33.311 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:09:33.311 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:09:33.369 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:09:33.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.595+0000 7f57266f1640 1 -- 172.21.15.40:0/3124495032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57201420f0 msgr2=0x7f57200b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:33.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.595+0000 7f57266f1640 1 --2- 172.21.15.40:0/3124495032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57201420f0 0x7f57200b0a80 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f5720071ab0 tx=0x7f571c02bd70 comp rx=0 tx=0).stop 2023-10-18T09:09:33.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.595+0000 7f57266f1640 1 -- 172.21.15.40:0/3124495032 shutdown_connections 2023-10-18T09:09:33.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.595+0000 7f57266f1640 1 --2- 172.21.15.40:0/3124495032 >> v2:172.21.15.144:3300/0 conn(0x7f57201425a0 0x7f57200c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:33.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.595+0000 7f57266f1640 1 --2- 172.21.15.40:0/3124495032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57201420f0 0x7f57200b0a80 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:33.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.595+0000 7f57266f1640 1 -- 172.21.15.40:0/3124495032 >> 172.21.15.40:0/3124495032 conn(0x7f5720069720 msgr2=0x7f5720069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:33.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.595+0000 7f57266f1640 1 -- 172.21.15.40:0/3124495032 shutdown_connections 2023-10-18T09:09:33.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.596+0000 7f57266f1640 1 -- 172.21.15.40:0/3124495032 wait complete. 2023-10-18T09:09:33.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.597+0000 7f57266f1640 1 Processor -- start 2023-10-18T09:09:33.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.597+0000 7f57266f1640 1 -- start start 2023-10-18T09:09:33.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.598+0000 7f57266f1640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57200c9f90 0x7f57200ac010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:33.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.598+0000 7f57266f1640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57201420f0 0x7f5720158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:33.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.598+0000 7f57266f1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57201425a0 0x7f572015bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:33.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.598+0000 7f57266f1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572006f600 con 0x7f57201425a0 2023-10-18T09:09:33.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.598+0000 7f57266f1640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57200ac550 con 0x7f57200c9f90 2023-10-18T09:09:33.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.598+0000 7f57266f1640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572013dd40 con 0x7f57201420f0 2023-10-18T09:09:33.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.599+0000 7f5725ef0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57201425a0 0x7f572015bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:33.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.599+0000 7f5725ef0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57201425a0 0x7f572015bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36026/0 (socket says 172.21.15.40:36026) 2023-10-18T09:09:33.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.599+0000 7f5725ef0640 1 -- 172.21.15.40:0/1755586619 learned_addr learned my addr 172.21.15.40:0/1755586619 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:33.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.599+0000 7f5725ef0640 1 -- 172.21.15.40:0/1755586619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57201420f0 msgr2=0x7f5720158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:33.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.599+0000 7f5724eee640 1 --2- 172.21.15.40:0/1755586619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57201420f0 0x7f5720158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:33.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.599+0000 7f57256ef640 1 --2- 172.21.15.40:0/1755586619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57200c9f90 0x7f57200ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:33.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.599+0000 7f5725ef0640 1 --2- 172.21.15.40:0/1755586619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57201420f0 0x7f5720158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:33.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.599+0000 7f5725ef0640 1 -- 172.21.15.40:0/1755586619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57200c9f90 msgr2=0x7f57200ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:33.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.599+0000 7f5725ef0640 1 --2- 172.21.15.40:0/1755586619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57200c9f90 0x7f57200ac010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:33.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.599+0000 7f5725ef0640 1 -- 172.21.15.40:0/1755586619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c009490 con 0x7f57201425a0 2023-10-18T09:09:33.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.599+0000 7f5724eee640 1 --2- 172.21.15.40:0/1755586619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57201420f0 0x7f5720158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:33.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.599+0000 7f5725ef0640 1 --2- 172.21.15.40:0/1755586619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57201425a0 0x7f572015bb20 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f571800cb10 tx=0x7f571800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:33.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.600+0000 7f5715ffb640 1 -- 172.21.15.40:0/1755586619 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571800ac70 con 0x7f57201425a0 2023-10-18T09:09:33.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.600+0000 7f5715ffb640 1 -- 172.21.15.40:0/1755586619 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5718003970 con 0x7f57201425a0 2023-10-18T09:09:33.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.600+0000 7f57266f1640 1 -- 172.21.15.40:0/1755586619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572013e020 con 0x7f57201425a0 2023-10-18T09:09:33.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.600+0000 7f5715ffb640 1 -- 172.21.15.40:0/1755586619 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5718003d80 con 0x7f57201425a0 2023-10-18T09:09:33.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.600+0000 7f57266f1640 1 -- 172.21.15.40:0/1755586619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572006fb30 con 0x7f57201425a0 2023-10-18T09:09:33.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.601+0000 7f57266f1640 1 -- 172.21.15.40:0/1755586619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f572006fd50 con 0x7f57201425a0 2023-10-18T09:09:33.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.602+0000 7f5715ffb640 1 -- 172.21.15.40:0/1755586619 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5718004030 con 0x7f57201425a0 2023-10-18T09:09:33.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.603+0000 7f5715ffb640 1 --2- 172.21.15.40:0/1755586619 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f570406ae40 0x7f570406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:33.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.603+0000 7f5715ffb640 1 -- 172.21.15.40:0/1755586619 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f571808a040 con 0x7f57201425a0 2023-10-18T09:09:33.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.604+0000 7f57256ef640 1 --2- 172.21.15.40:0/1755586619 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f570406ae40 0x7f570406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:33.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.605+0000 7f57256ef640 1 --2- 172.21.15.40:0/1755586619 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f570406ae40 0x7f570406d300 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f571c036000 tx=0x7f571c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:33.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.608+0000 7f5715ffb640 1 -- 172.21.15.40:0/1755586619 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57180550f0 con 0x7f57201425a0 2023-10-18T09:09:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.803+0000 7f57266f1640 1 -- 172.21.15.40:0/1755586619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57200afa40 con 0x7f57201425a0 2023-10-18T09:09:33.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.805+0000 7f5715ffb640 1 -- 172.21.15.40:0/1755586619 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+16591 (secure 0 0 0) 0x7f5718058710 con 0x7f57201425a0 2023-10-18T09:09:33.808 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:33.808 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":199,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:09:26.219860+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:08:55.675479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-18T09:08:56.761888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:09:33.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.807+0000 7f57266f1640 1 -- 172.21.15.40:0/1755586619 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f570406ae40 msgr2=0x7f570406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.807+0000 7f57266f1640 1 --2- 172.21.15.40:0/1755586619 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f570406ae40 0x7f570406d300 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f571c036000 tx=0x7f571c004730 comp rx=0 tx=0).stop 2023-10-18T09:09:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.807+0000 7f57266f1640 1 -- 172.21.15.40:0/1755586619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57201425a0 msgr2=0x7f572015bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:33.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.807+0000 7f57266f1640 1 --2- 172.21.15.40:0/1755586619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57201425a0 0x7f572015bb20 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f571800cb10 tx=0x7f571800a060 comp rx=0 tx=0).stop 2023-10-18T09:09:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.808+0000 7f57266f1640 1 -- 172.21.15.40:0/1755586619 shutdown_connections 2023-10-18T09:09:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.808+0000 7f57266f1640 1 --2- 172.21.15.40:0/1755586619 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f570406ae40 0x7f570406d300 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.808+0000 7f57266f1640 1 --2- 172.21.15.40:0/1755586619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57201425a0 0x7f572015bb20 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:33.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.808+0000 7f57266f1640 1 --2- 172.21.15.40:0/1755586619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57201420f0 0x7f5720158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.808+0000 7f57266f1640 1 --2- 172.21.15.40:0/1755586619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57200c9f90 0x7f57200ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.808+0000 7f57266f1640 1 -- 172.21.15.40:0/1755586619 >> 172.21.15.40:0/1755586619 conn(0x7f5720069720 msgr2=0x7f5720069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.808+0000 7f57266f1640 1 -- 172.21.15.40:0/1755586619 shutdown_connections 2023-10-18T09:09:33.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:33.808+0000 7f57266f1640 1 -- 172.21.15.40:0/1755586619 wait complete. 2023-10-18T09:09:33.828 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:08:55.675479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '195', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:08:56.761888+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '199', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:09:33.829 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:09:34.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.048+0000 7effb5c02640 1 -- 172.21.15.40:0/3720107201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff98005180 msgr2=0x7eff9800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:34.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.048+0000 7effb5c02640 1 --2- 172.21.15.40:0/3720107201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff98005180 0x7eff9800b980 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7effa8009ed0 tx=0x7eff980057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.048+0000 7effb5c02640 1 -- 172.21.15.40:0/3720107201 shutdown_connections 2023-10-18T09:09:34.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.048+0000 7effb5c02640 1 --2- 172.21.15.40:0/3720107201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff98005180 0x7eff9800b980 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.048+0000 7effb5c02640 1 -- 172.21.15.40:0/3720107201 >> 172.21.15.40:0/3720107201 conn(0x7effb0069720 msgr2=0x7effb0069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:34.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.048+0000 7effb5c02640 1 -- 172.21.15.40:0/3720107201 shutdown_connections 2023-10-18T09:09:34.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.048+0000 7effb5c02640 1 -- 172.21.15.40:0/3720107201 wait complete. 2023-10-18T09:09:34.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.050+0000 7effb5c02640 1 Processor -- start 2023-10-18T09:09:34.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.050+0000 7effb5c02640 1 -- start start 2023-10-18T09:09:34.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.051+0000 7effb5c02640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff98005180 0x7effb00c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:34.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.051+0000 7effb5c02640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effb00ab290 0x7effb013bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:34.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.051+0000 7effb5c02640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effb01f5870 0x7effb01f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:34.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.051+0000 7effb5c02640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb013d520 con 0x7effb01f5870 2023-10-18T09:09:34.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.051+0000 7effb5c02640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb01f6230 con 0x7effb00ab290 2023-10-18T09:09:34.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.051+0000 7effb5c02640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb00ca380 con 0x7eff98005180 2023-10-18T09:09:34.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.052+0000 7effaffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effb01f5870 0x7effb01f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:34.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.052+0000 7effaffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effb01f5870 0x7effb01f5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36040/0 (socket says 172.21.15.40:36040) 2023-10-18T09:09:34.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.052+0000 7effaffff640 1 -- 172.21.15.40:0/2505739025 learned_addr learned my addr 172.21.15.40:0/2505739025 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:34.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.052+0000 7effaeffd640 1 --2- 172.21.15.40:0/2505739025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effb00ab290 0x7effb013bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:34.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.052+0000 7effaffff640 1 -- 172.21.15.40:0/2505739025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff98005180 msgr2=0x7effb00c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:34.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.052+0000 7effaffff640 1 --2- 172.21.15.40:0/2505739025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff98005180 0x7effb00c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.052+0000 7effaffff640 1 -- 172.21.15.40:0/2505739025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effb00ab290 msgr2=0x7effb013bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:34.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.052+0000 7effaffff640 1 --2- 172.21.15.40:0/2505739025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effb00ab290 0x7effb013bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.052+0000 7effaffff640 1 -- 172.21.15.40:0/2505739025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb00ab850 con 0x7effb01f5870 2023-10-18T09:09:34.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.052+0000 7effaffff640 1 --2- 172.21.15.40:0/2505739025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effb01f5870 0x7effb01f5cf0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7effa8009e20 tx=0x7effa8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:34.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.053+0000 7eff97fff640 1 -- 172.21.15.40:0/2505739025 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa803d070 con 0x7effb01f5870 2023-10-18T09:09:34.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.053+0000 7eff97fff640 1 -- 172.21.15.40:0/2505739025 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7effa80042f0 con 0x7effb01f5870 2023-10-18T09:09:34.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.053+0000 7eff97fff640 1 -- 172.21.15.40:0/2505739025 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8004680 con 0x7effb01f5870 2023-10-18T09:09:34.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.053+0000 7effb5c02640 1 -- 172.21.15.40:0/2505739025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb00abb30 con 0x7effb01f5870 2023-10-18T09:09:34.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.053+0000 7effb5c02640 1 -- 172.21.15.40:0/2505739025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb00ac0f0 con 0x7effb01f5870 2023-10-18T09:09:34.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.054+0000 7effb5c02640 1 -- 172.21.15.40:0/2505739025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb0066fd0 con 0x7effb01f5870 2023-10-18T09:09:34.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.055+0000 7eff97fff640 1 -- 172.21.15.40:0/2505739025 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7effa80304c0 con 0x7effb01f5870 2023-10-18T09:09:34.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.056+0000 7eff97fff640 1 --2- 172.21.15.40:0/2505739025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7eff9c06ae40 0x7eff9c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:34.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.056+0000 7eff97fff640 1 -- 172.21.15.40:0/2505739025 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effa80ab4a0 con 0x7effb01f5870 2023-10-18T09:09:34.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.057+0000 7effaf7fe640 1 --2- 172.21.15.40:0/2505739025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7eff9c06ae40 0x7eff9c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:34.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.058+0000 7effaf7fe640 1 --2- 172.21.15.40:0/2505739025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7eff9c06ae40 0x7eff9c06d300 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7effa0001540 tx=0x7effa00079b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:34.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.061+0000 7eff97fff640 1 -- 172.21.15.40:0/2505739025 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7effa80764b0 con 0x7effb01f5870 2023-10-18T09:09:34.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.256+0000 7effb5c02640 1 -- 172.21.15.40:0/2505739025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7effb013cff0 con 0x7effb01f5870 2023-10-18T09:09:34.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.257+0000 7eff97fff640 1 -- 172.21.15.40:0/2505739025 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v199) v1 ==== 93+0+10 (secure 0 0 0) 0x7effa8079ad0 con 0x7effb01f5870 2023-10-18T09:09:34.259 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:09:34.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.260+0000 7effb5c02640 1 -- 172.21.15.40:0/2505739025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7eff9c06ae40 msgr2=0x7eff9c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:34.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.260+0000 7effb5c02640 1 --2- 172.21.15.40:0/2505739025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7eff9c06ae40 0x7eff9c06d300 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7effa0001540 tx=0x7effa00079b0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.260+0000 7effb5c02640 1 -- 172.21.15.40:0/2505739025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effb01f5870 msgr2=0x7effb01f5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:34.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.260+0000 7effb5c02640 1 --2- 172.21.15.40:0/2505739025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effb01f5870 0x7effb01f5cf0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7effa8009e20 tx=0x7effa8003ba0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.260+0000 7effb5c02640 1 -- 172.21.15.40:0/2505739025 shutdown_connections 2023-10-18T09:09:34.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.260+0000 7effb5c02640 1 --2- 172.21.15.40:0/2505739025 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7eff9c06ae40 0x7eff9c06d300 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.260+0000 7effb5c02640 1 --2- 172.21.15.40:0/2505739025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effb01f5870 0x7effb01f5cf0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.260+0000 7effb5c02640 1 --2- 172.21.15.40:0/2505739025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7effb00ab290 0x7effb013bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.260+0000 7effb5c02640 1 --2- 172.21.15.40:0/2505739025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff98005180 0x7effb00c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.260+0000 7effb5c02640 1 -- 172.21.15.40:0/2505739025 >> 172.21.15.40:0/2505739025 conn(0x7effb0069720 msgr2=0x7effb00acd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:34.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.260+0000 7effb5c02640 1 -- 172.21.15.40:0/2505739025 shutdown_connections 2023-10-18T09:09:34.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.260+0000 7effb5c02640 1 -- 172.21.15.40:0/2505739025 wait complete. 2023-10-18T09:09:34.281 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:09:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.518+0000 7ff7863c4640 1 -- 172.21.15.40:0/883203990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff76c004d00 msgr2=0x7ff76c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.518+0000 7ff7863c4640 1 --2- 172.21.15.40:0/883203990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff76c004d00 0x7ff76c005180 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7ff77800bb20 tx=0x7ff76c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.518+0000 7ff785bc3640 1 -- 172.21.15.40:0/883203990 reap_dead start 2023-10-18T09:09:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.518+0000 7ff7863c4640 1 -- 172.21.15.40:0/883203990 shutdown_connections 2023-10-18T09:09:34.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.518+0000 7ff7863c4640 1 -- 172.21.15.40:0/883203990 >> 172.21.15.40:0/883203990 conn(0x7ff78006ca10 msgr2=0x7ff78006ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:34.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.518+0000 7ff7863c4640 1 -- 172.21.15.40:0/883203990 shutdown_connections 2023-10-18T09:09:34.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.518+0000 7ff7863c4640 1 -- 172.21.15.40:0/883203990 wait complete. 2023-10-18T09:09:34.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.520+0000 7ff7863c4640 1 Processor -- start 2023-10-18T09:09:34.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.520+0000 7ff7863c4640 1 -- start start 2023-10-18T09:09:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.521+0000 7ff7863c4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff76c004d00 0x7ff780145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.521+0000 7ff7863c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7800b1670 0x7ff7800b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.521+0000 7ff7863c4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800c39e0 0x7ff7800c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:34.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.521+0000 7ff7863c4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7801460d0 con 0x7ff7800b1670 2023-10-18T09:09:34.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.521+0000 7ff7863c4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800b2030 con 0x7ff76c004d00 2023-10-18T09:09:34.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.521+0000 7ff7863c4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800c1ee0 con 0x7ff7800c39e0 2023-10-18T09:09:34.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.522+0000 7ff784bc1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7800b1670 0x7ff7800b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:34.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.522+0000 7ff784bc1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7800b1670 0x7ff7800b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36056/0 (socket says 172.21.15.40:36056) 2023-10-18T09:09:34.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.522+0000 7ff784bc1640 1 -- 172.21.15.40:0/4019736533 learned_addr learned my addr 172.21.15.40:0/4019736533 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:34.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.522+0000 7ff7853c2640 1 --2- 172.21.15.40:0/4019736533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff76c004d00 0x7ff780145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:34.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.522+0000 7ff785bc3640 1 --2- 172.21.15.40:0/4019736533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800c39e0 0x7ff7800c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:34.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.522+0000 7ff784bc1640 1 -- 172.21.15.40:0/4019736533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800c39e0 msgr2=0x7ff7800c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:34.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.522+0000 7ff784bc1640 1 --2- 172.21.15.40:0/4019736533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800c39e0 0x7ff7800c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.522+0000 7ff784bc1640 1 -- 172.21.15.40:0/4019736533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff76c004d00 msgr2=0x7ff780145b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:34.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.522+0000 7ff784bc1640 1 --2- 172.21.15.40:0/4019736533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff76c004d00 0x7ff780145b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.522+0000 7ff784bc1640 1 -- 172.21.15.40:0/4019736533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800c2a60 con 0x7ff7800b1670 2023-10-18T09:09:34.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.522+0000 7ff785bc3640 1 --2- 172.21.15.40:0/4019736533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800c39e0 0x7ff7800c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:34.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.522+0000 7ff784bc1640 1 --2- 172.21.15.40:0/4019736533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7800b1670 0x7ff7800b1af0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7ff77000fcb0 tx=0x7ff77000d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.523+0000 7ff775ffb640 1 -- 172.21.15.40:0/4019736533 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77000dee0 con 0x7ff7800b1670 2023-10-18T09:09:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.523+0000 7ff775ffb640 1 -- 172.21.15.40:0/4019736533 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff770004e10 con 0x7ff7800b1670 2023-10-18T09:09:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.523+0000 7ff775ffb640 1 -- 172.21.15.40:0/4019736533 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770005260 con 0x7ff7800b1670 2023-10-18T09:09:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.523+0000 7ff7863c4640 1 -- 172.21.15.40:0/4019736533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7800c2130 con 0x7ff7800b1670 2023-10-18T09:09:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.523+0000 7ff7863c4640 1 -- 172.21.15.40:0/4019736533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7800c26f0 con 0x7ff7800b1670 2023-10-18T09:09:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.524+0000 7ff7863c4640 1 -- 172.21.15.40:0/4019736533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff780066e00 con 0x7ff7800b1670 2023-10-18T09:09:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.525+0000 7ff775ffb640 1 -- 172.21.15.40:0/4019736533 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff7700053c0 con 0x7ff7800b1670 2023-10-18T09:09:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.526+0000 7ff775ffb640 1 --2- 172.21.15.40:0/4019736533 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff76406ae70 0x7ff76406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.526+0000 7ff775ffb640 1 -- 172.21.15.40:0/4019736533 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff77008a4f0 con 0x7ff7800b1670 2023-10-18T09:09:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.527+0000 7ff7853c2640 1 --2- 172.21.15.40:0/4019736533 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff76406ae70 0x7ff76406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.528+0000 7ff7853c2640 1 --2- 172.21.15.40:0/4019736533 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff76406ae70 0x7ff76406d330 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0017a0 tx=0x7ff77c001630 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:34.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.531+0000 7ff775ffb640 1 -- 172.21.15.40:0/4019736533 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff7700555a0 con 0x7ff7800b1670 2023-10-18T09:09:34.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.724+0000 7ff7863c4640 1 -- 172.21.15.40:0/4019736533 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff78015aec0 con 0x7ff7800b1670 2023-10-18T09:09:34.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.725+0000 7ff775ffb640 1 -- 172.21.15.40:0/4019736533 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v199) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff770058bc0 con 0x7ff7800b1670 2023-10-18T09:09:34.728 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:09:34.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.728+0000 7ff7863c4640 1 -- 172.21.15.40:0/4019736533 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff76406ae70 msgr2=0x7ff76406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:34.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.728+0000 7ff7863c4640 1 --2- 172.21.15.40:0/4019736533 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff76406ae70 0x7ff76406d330 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0017a0 tx=0x7ff77c001630 comp rx=0 tx=0).stop 2023-10-18T09:09:34.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.728+0000 7ff7863c4640 1 -- 172.21.15.40:0/4019736533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7800b1670 msgr2=0x7ff7800b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:34.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.728+0000 7ff7863c4640 1 --2- 172.21.15.40:0/4019736533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7800b1670 0x7ff7800b1af0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7ff77000fcb0 tx=0x7ff77000d6b0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.728+0000 7ff7863c4640 1 -- 172.21.15.40:0/4019736533 shutdown_connections 2023-10-18T09:09:34.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.728+0000 7ff7863c4640 1 --2- 172.21.15.40:0/4019736533 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff76406ae70 0x7ff76406d330 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.728+0000 7ff7863c4640 1 --2- 172.21.15.40:0/4019736533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7800c39e0 0x7ff7800c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.728+0000 7ff7863c4640 1 --2- 172.21.15.40:0/4019736533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7800b1670 0x7ff7800b1af0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.728+0000 7ff7863c4640 1 --2- 172.21.15.40:0/4019736533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff76c004d00 0x7ff780145b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.728+0000 7ff7863c4640 1 -- 172.21.15.40:0/4019736533 >> 172.21.15.40:0/4019736533 conn(0x7ff78006ca10 msgr2=0x7ff780154e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:34.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.729+0000 7ff7863c4640 1 -- 172.21.15.40:0/4019736533 shutdown_connections 2023-10-18T09:09:34.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.729+0000 7ff7863c4640 1 -- 172.21.15.40:0/4019736533 wait complete. 2023-10-18T09:09:34.751 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:09:34.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.988+0000 7f876735e640 1 -- 172.21.15.40:0/1744187787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87600cbf20 msgr2=0x7f87600cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:34.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.988+0000 7f876735e640 1 --2- 172.21.15.40:0/1744187787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87600cbf20 0x7f87600cae80 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f87600caa30 tx=0x7f875802bcf0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.989+0000 7f876735e640 1 -- 172.21.15.40:0/1744187787 shutdown_connections 2023-10-18T09:09:34.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.989+0000 7f876735e640 1 --2- 172.21.15.40:0/1744187787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87600cbf20 0x7f87600cae80 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.989+0000 7f876735e640 1 --2- 172.21.15.40:0/1744187787 >> v2:172.21.15.144:3300/0 conn(0x7f876015b4d0 0x7f87600ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.989+0000 7f876735e640 1 -- 172.21.15.40:0/1744187787 >> 172.21.15.40:0/1744187787 conn(0x7f8760069720 msgr2=0x7f8760069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:34.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.989+0000 7f876735e640 1 -- 172.21.15.40:0/1744187787 shutdown_connections 2023-10-18T09:09:34.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.989+0000 7f876735e640 1 -- 172.21.15.40:0/1744187787 wait complete. 2023-10-18T09:09:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.990+0000 7f876735e640 1 Processor -- start 2023-10-18T09:09:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.991+0000 7f876735e640 1 -- start start 2023-10-18T09:09:34.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.992+0000 7f876735e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87600ca580 0x7f8760147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:34.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.992+0000 7f876735e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87600cbf20 0x7f8760159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:34.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.992+0000 7f876735e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f876015b4d0 0x7f876015c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:34.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.992+0000 7f876735e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87600afbf0 con 0x7f87600ca580 2023-10-18T09:09:34.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.992+0000 7f876735e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8760147b40 con 0x7f87600cbf20 2023-10-18T09:09:34.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.992+0000 7f876735e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876015c040 con 0x7f876015b4d0 2023-10-18T09:09:34.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.992+0000 7f87650fa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87600ca580 0x7f8760147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:34.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.992+0000 7f87650fa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87600ca580 0x7f8760147600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55462/0 (socket says 172.21.15.40:55462) 2023-10-18T09:09:34.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.992+0000 7f87650fa640 1 -- 172.21.15.40:0/1021213802 learned_addr learned my addr 172.21.15.40:0/1021213802 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:34.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.992+0000 7f87648f9640 1 --2- 172.21.15.40:0/1021213802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87600cbf20 0x7f8760159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.992+0000 7f87650fa640 1 -- 172.21.15.40:0/1021213802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f876015b4d0 msgr2=0x7f876015c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.992+0000 7f87650fa640 1 --2- 172.21.15.40:0/1021213802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f876015b4d0 0x7f876015c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.992+0000 7f87650fa640 1 -- 172.21.15.40:0/1021213802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87600cbf20 msgr2=0x7f8760159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.992+0000 7f87650fa640 1 --2- 172.21.15.40:0/1021213802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87600cbf20 0x7f8760159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:34.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.993+0000 7f87650fa640 1 -- 172.21.15.40:0/1021213802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8758009490 con 0x7f87600ca580 2023-10-18T09:09:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.993+0000 7f87648f9640 1 --2- 172.21.15.40:0/1021213802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87600cbf20 0x7f8760159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.993+0000 7f87650fa640 1 --2- 172.21.15.40:0/1021213802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87600ca580 0x7f8760147600 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f875c00ea20 tx=0x7f875c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.993+0000 7f8755ffb640 1 -- 172.21.15.40:0/1021213802 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00cde0 con 0x7f87600ca580 2023-10-18T09:09:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.993+0000 7f8755ffb640 1 -- 172.21.15.40:0/1021213802 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f875c011560 con 0x7f87600ca580 2023-10-18T09:09:34.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.993+0000 7f876735e640 1 -- 172.21.15.40:0/1021213802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8760154060 con 0x7f87600ca580 2023-10-18T09:09:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.993+0000 7f8755ffb640 1 -- 172.21.15.40:0/1021213802 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c011950 con 0x7f87600ca580 2023-10-18T09:09:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.993+0000 7f876735e640 1 -- 172.21.15.40:0/1021213802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8760153d80 con 0x7f87600ca580 2023-10-18T09:09:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.994+0000 7f876735e640 1 -- 172.21.15.40:0/1021213802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8760066fd0 con 0x7f87600ca580 2023-10-18T09:09:35.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.995+0000 7f8755ffb640 1 -- 172.21.15.40:0/1021213802 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f875c011ab0 con 0x7f87600ca580 2023-10-18T09:09:35.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.996+0000 7f8755ffb640 1 --2- 172.21.15.40:0/1021213802 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f874406ae40 0x7f874406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:35.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.997+0000 7f8755ffb640 1 -- 172.21.15.40:0/1021213802 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f875c08a2e0 con 0x7f87600ca580 2023-10-18T09:09:35.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.997+0000 7f87648f9640 1 --2- 172.21.15.40:0/1021213802 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f874406ae40 0x7f874406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:35.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:34.998+0000 7f87648f9640 1 --2- 172.21.15.40:0/1021213802 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f874406ae40 0x7f874406d300 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8760148360 tx=0x7f8750009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:35.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.000+0000 7f8755ffb640 1 -- 172.21.15.40:0/1021213802 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f875c0552f0 con 0x7f87600ca580 2023-10-18T09:09:35.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.194+0000 7f876735e640 1 -- 172.21.15.40:0/1021213802 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f87600cb390 con 0x7f87600ca580 2023-10-18T09:09:35.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.195+0000 7f8755ffb640 1 -- 172.21.15.40:0/1021213802 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v199) v1 ==== 100+0+11 (secure 0 0 0) 0x7f875c058910 con 0x7f87600ca580 2023-10-18T09:09:35.197 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:09:35.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.197+0000 7f876735e640 1 -- 172.21.15.40:0/1021213802 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f874406ae40 msgr2=0x7f874406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:35.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.197+0000 7f876735e640 1 --2- 172.21.15.40:0/1021213802 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f874406ae40 0x7f874406d300 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8760148360 tx=0x7f8750009040 comp rx=0 tx=0).stop 2023-10-18T09:09:35.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.197+0000 7f876735e640 1 -- 172.21.15.40:0/1021213802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87600ca580 msgr2=0x7f8760147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:35.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.197+0000 7f876735e640 1 --2- 172.21.15.40:0/1021213802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87600ca580 0x7f8760147600 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f875c00ea20 tx=0x7f875c00eee0 comp rx=0 tx=0).stop 2023-10-18T09:09:35.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.198+0000 7f876735e640 1 -- 172.21.15.40:0/1021213802 shutdown_connections 2023-10-18T09:09:35.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.198+0000 7f876735e640 1 --2- 172.21.15.40:0/1021213802 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f874406ae40 0x7f874406d300 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:35.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.198+0000 7f876735e640 1 --2- 172.21.15.40:0/1021213802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f876015b4d0 0x7f876015c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:35.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.198+0000 7f876735e640 1 --2- 172.21.15.40:0/1021213802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87600cbf20 0x7f8760159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:35.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.198+0000 7f876735e640 1 --2- 172.21.15.40:0/1021213802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87600ca580 0x7f8760147600 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:35.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.198+0000 7f876735e640 1 -- 172.21.15.40:0/1021213802 >> 172.21.15.40:0/1021213802 conn(0x7f8760069720 msgr2=0x7f876006ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:35.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.198+0000 7f876735e640 1 -- 172.21.15.40:0/1021213802 shutdown_connections 2023-10-18T09:09:35.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.198+0000 7f876735e640 1 -- 172.21.15.40:0/1021213802 wait complete. 2023-10-18T09:09:35.218 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:09:35.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.443+0000 7f352d96c640 1 -- 172.21.15.40:0/2738593645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35280b0630 msgr2=0x7f35280b1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:35.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.443+0000 7f352d96c640 1 --2- 172.21.15.40:0/2738593645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35280b0630 0x7f35280b1e10 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f352806aeb0 tx=0x7f351802bd70 comp rx=0 tx=0).stop 2023-10-18T09:09:35.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.443+0000 7f352d96c640 1 -- 172.21.15.40:0/2738593645 shutdown_connections 2023-10-18T09:09:35.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.443+0000 7f352d96c640 1 --2- 172.21.15.40:0/2738593645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35280b0630 0x7f35280b1e10 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:35.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.443+0000 7f352d96c640 1 --2- 172.21.15.40:0/2738593645 >> v2:172.21.15.144:3300/0 conn(0x7f352806d6f0 0x7f35280b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:35.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.443+0000 7f352d96c640 1 -- 172.21.15.40:0/2738593645 >> 172.21.15.40:0/2738593645 conn(0x7f3528069720 msgr2=0x7f3528069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:35.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.443+0000 7f352d96c640 1 -- 172.21.15.40:0/2738593645 shutdown_connections 2023-10-18T09:09:35.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.443+0000 7f352d96c640 1 -- 172.21.15.40:0/2738593645 wait complete. 2023-10-18T09:09:35.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.445+0000 7f352d96c640 1 Processor -- start 2023-10-18T09:09:35.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.445+0000 7f352d96c640 1 -- start start 2023-10-18T09:09:35.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.446+0000 7f352d96c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f352806d6f0 0x7f3528142870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:35.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.446+0000 7f352d96c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35280b0630 0x7f35281525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:35.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.446+0000 7f352d96c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35280c99c0 0x7f3528152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:35.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.446+0000 7f352d96c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3528142db0 con 0x7f35280b0630 2023-10-18T09:09:35.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.446+0000 7f352d96c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3528153180 con 0x7f35280c99c0 2023-10-18T09:09:35.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.446+0000 7f352d96c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3528153460 con 0x7f352806d6f0 2023-10-18T09:09:35.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.447+0000 7f351ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35280b0630 0x7f35281525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:35.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.447+0000 7f351ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35280b0630 0x7f35281525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55482/0 (socket says 172.21.15.40:55482) 2023-10-18T09:09:35.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.447+0000 7f351ffff640 1 -- 172.21.15.40:0/1960697535 learned_addr learned my addr 172.21.15.40:0/1960697535 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:35.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.447+0000 7f352d16b640 1 --2- 172.21.15.40:0/1960697535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35280c99c0 0x7f3528152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:35.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.447+0000 7f351ffff640 1 -- 172.21.15.40:0/1960697535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f352806d6f0 msgr2=0x7f3528142870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:35.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.447+0000 7f351ffff640 1 --2- 172.21.15.40:0/1960697535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f352806d6f0 0x7f3528142870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:35.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.447+0000 7f351ffff640 1 -- 172.21.15.40:0/1960697535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35280c99c0 msgr2=0x7f3528152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:35.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.447+0000 7f351ffff640 1 --2- 172.21.15.40:0/1960697535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35280c99c0 0x7f3528152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:35.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.447+0000 7f351ffff640 1 -- 172.21.15.40:0/1960697535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3518009490 con 0x7f35280b0630 2023-10-18T09:09:35.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.448+0000 7f351ffff640 1 --2- 172.21.15.40:0/1960697535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35280b0630 0x7f35281525e0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f3518000c00 tx=0x7f35180038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:35.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.448+0000 7f351d7fa640 1 -- 172.21.15.40:0/1960697535 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518003ea0 con 0x7f35280b0630 2023-10-18T09:09:35.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.448+0000 7f351d7fa640 1 -- 172.21.15.40:0/1960697535 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3518034440 con 0x7f35280b0630 2023-10-18T09:09:35.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.448+0000 7f351d7fa640 1 -- 172.21.15.40:0/1960697535 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518034850 con 0x7f35280b0630 2023-10-18T09:09:35.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.448+0000 7f352d96c640 1 -- 172.21.15.40:0/1960697535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35280acf60 con 0x7f35280b0630 2023-10-18T09:09:35.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.448+0000 7f352d96c640 1 -- 172.21.15.40:0/1960697535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35280ad520 con 0x7f35280b0630 2023-10-18T09:09:35.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.449+0000 7f352d96c640 1 -- 172.21.15.40:0/1960697535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3528066e00 con 0x7f35280b0630 2023-10-18T09:09:35.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.450+0000 7f351d7fa640 1 -- 172.21.15.40:0/1960697535 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f351803b040 con 0x7f35280b0630 2023-10-18T09:09:35.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.451+0000 7f351d7fa640 1 --2- 172.21.15.40:0/1960697535 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f351406ae40 0x7f351406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:35.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.451+0000 7f351d7fa640 1 -- 172.21.15.40:0/1960697535 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35180abbb0 con 0x7f35280b0630 2023-10-18T09:09:35.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.452+0000 7f352c96a640 1 --2- 172.21.15.40:0/1960697535 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f351406ae40 0x7f351406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:35.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.453+0000 7f352c96a640 1 --2- 172.21.15.40:0/1960697535 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f351406ae40 0x7f351406d300 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3524005c40 tx=0x7f3524005bd0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:35.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.455+0000 7f351d7fa640 1 -- 172.21.15.40:0/1960697535 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3518076b30 con 0x7f35280b0630 2023-10-18T09:09:35.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.667+0000 7f352d96c640 1 -- 172.21.15.40:0/1960697535 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f35280caf90 con 0x7f35280b0630 2023-10-18T09:09:35.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.668+0000 7f351d7fa640 1 -- 172.21.15.40:0/1960697535 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v199) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3518041080 con 0x7f35280b0630 2023-10-18T09:09:35.671 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T09:09:35.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.671+0000 7f352d96c640 1 -- 172.21.15.40:0/1960697535 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f351406ae40 msgr2=0x7f351406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:35.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.671+0000 7f352d96c640 1 --2- 172.21.15.40:0/1960697535 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f351406ae40 0x7f351406d300 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3524005c40 tx=0x7f3524005bd0 comp rx=0 tx=0).stop 2023-10-18T09:09:35.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.671+0000 7f352d96c640 1 -- 172.21.15.40:0/1960697535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35280b0630 msgr2=0x7f35281525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:35.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.671+0000 7f352d96c640 1 --2- 172.21.15.40:0/1960697535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35280b0630 0x7f35281525e0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f3518000c00 tx=0x7f35180038a0 comp rx=0 tx=0).stop 2023-10-18T09:09:35.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.671+0000 7f352d96c640 1 -- 172.21.15.40:0/1960697535 shutdown_connections 2023-10-18T09:09:35.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.671+0000 7f352d96c640 1 --2- 172.21.15.40:0/1960697535 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f351406ae40 0x7f351406d300 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:35.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.671+0000 7f352d96c640 1 --2- 172.21.15.40:0/1960697535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35280c99c0 0x7f3528152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:35.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.671+0000 7f352d96c640 1 --2- 172.21.15.40:0/1960697535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35280b0630 0x7f35281525e0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:35.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.671+0000 7f352d96c640 1 --2- 172.21.15.40:0/1960697535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f352806d6f0 0x7f3528142870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:35.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.671+0000 7f352d96c640 1 -- 172.21.15.40:0/1960697535 >> 172.21.15.40:0/1960697535 conn(0x7f3528069720 msgr2=0x7f35280cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:35.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.671+0000 7f352d96c640 1 -- 172.21.15.40:0/1960697535 shutdown_connections 2023-10-18T09:09:35.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:35.671+0000 7f352d96c640 1 -- 172.21.15.40:0/1960697535 wait complete. 2023-10-18T09:09:35.693 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-18T09:09:35.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:35.694 DEBUG:teuthology.orchestra.run.smithi040:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-18T09:09:35.837 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T09:09:35.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:35.837 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-18T09:09:35.892 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:09:35.893 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:09:35.940 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:09:35.940 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:09:36.035 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:09:36.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.275+0000 7f74b59d0640 1 -- 172.21.15.40:0/2598615585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7498004ab0 msgr2=0x7f7498004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:36.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.275+0000 7f74b59d0640 1 --2- 172.21.15.40:0/2598615585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7498004ab0 0x7f7498004ed0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009b20 tx=0x7f7498006900 comp rx=0 tx=0).stop 2023-10-18T09:09:36.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.275+0000 7f74af7fe640 1 -- 172.21.15.40:0/2598615585 reap_dead start 2023-10-18T09:09:36.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.275+0000 7f74b59d0640 1 -- 172.21.15.40:0/2598615585 shutdown_connections 2023-10-18T09:09:36.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.275+0000 7f74b59d0640 1 -- 172.21.15.40:0/2598615585 >> 172.21.15.40:0/2598615585 conn(0x7f74b006cee0 msgr2=0x7f74b0069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:36.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.275+0000 7f74b59d0640 1 -- 172.21.15.40:0/2598615585 shutdown_connections 2023-10-18T09:09:36.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.275+0000 7f74b59d0640 1 -- 172.21.15.40:0/2598615585 wait complete. 2023-10-18T09:09:36.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.277+0000 7f74b59d0640 1 Processor -- start 2023-10-18T09:09:36.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.277+0000 7f74b59d0640 1 -- start start 2023-10-18T09:09:36.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.278+0000 7f74b59d0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7498004ab0 0x7f74b0145230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:36.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.278+0000 7f74b59d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b00c2050 0x7f74b014d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:36.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.278+0000 7f74b59d0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74b00c24d0 0x7f74b0140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:36.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.278+0000 7f74b59d0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b01404e0 con 0x7f7498004ab0 2023-10-18T09:09:36.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.278+0000 7f74b59d0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b013f8d0 con 0x7f74b00c2050 2023-10-18T09:09:36.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.278+0000 7f74b59d0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b013fbb0 con 0x7f74b00c24d0 2023-10-18T09:09:36.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.278+0000 7f74aeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7498004ab0 0x7f74b0145230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:36.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.278+0000 7f74aeffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7498004ab0 0x7f74b0145230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55502/0 (socket says 172.21.15.40:55502) 2023-10-18T09:09:36.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.278+0000 7f74aeffd640 1 -- 172.21.15.40:0/3884597208 learned_addr learned my addr 172.21.15.40:0/3884597208 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:36.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.279+0000 7f74af7fe640 1 --2- 172.21.15.40:0/3884597208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74b00c24d0 0x7f74b0140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:36.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.279+0000 7f74aeffd640 1 -- 172.21.15.40:0/3884597208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74b00c24d0 msgr2=0x7f74b0140970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:36.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.279+0000 7f74aeffd640 1 --2- 172.21.15.40:0/3884597208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74b00c24d0 0x7f74b0140970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:36.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.279+0000 7f74aeffd640 1 -- 172.21.15.40:0/3884597208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b00c2050 msgr2=0x7f74b014d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:36.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.279+0000 7f74ae7fc640 1 --2- 172.21.15.40:0/3884597208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b00c2050 0x7f74b014d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:36.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.279+0000 7f74aeffd640 1 --2- 172.21.15.40:0/3884597208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b00c2050 0x7f74b014d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:36.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.279+0000 7f74aeffd640 1 -- 172.21.15.40:0/3884597208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b013fe90 con 0x7f7498004ab0 2023-10-18T09:09:36.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.279+0000 7f74af7fe640 1 --2- 172.21.15.40:0/3884597208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74b00c24d0 0x7f74b0140970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:36.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.279+0000 7f74ae7fc640 1 --2- 172.21.15.40:0/3884597208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b00c2050 0x7f74b014d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:36.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.279+0000 7f74aeffd640 1 --2- 172.21.15.40:0/3884597208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7498004ab0 0x7f74b0145230 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009a70 tx=0x7f74a0033c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:36.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.280+0000 7f7497fff640 1 -- 172.21.15.40:0/3884597208 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a0004220 con 0x7f7498004ab0 2023-10-18T09:09:36.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.280+0000 7f7497fff640 1 -- 172.21.15.40:0/3884597208 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f74a0004380 con 0x7f7498004ab0 2023-10-18T09:09:36.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.280+0000 7f7497fff640 1 -- 172.21.15.40:0/3884597208 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a003e890 con 0x7f7498004ab0 2023-10-18T09:09:36.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.280+0000 7f74b59d0640 1 -- 172.21.15.40:0/3884597208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b0140170 con 0x7f7498004ab0 2023-10-18T09:09:36.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.280+0000 7f74b59d0640 1 -- 172.21.15.40:0/3884597208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b013f560 con 0x7f7498004ab0 2023-10-18T09:09:36.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.281+0000 7f74b59d0640 1 -- 172.21.15.40:0/3884597208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b00731b0 con 0x7f7498004ab0 2023-10-18T09:09:36.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.282+0000 7f7497fff640 1 -- 172.21.15.40:0/3884597208 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f74a003e9f0 con 0x7f7498004ab0 2023-10-18T09:09:36.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.283+0000 7f7497fff640 1 --2- 172.21.15.40:0/3884597208 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f749c06ae40 0x7f749c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:36.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.283+0000 7f7497fff640 1 -- 172.21.15.40:0/3884597208 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74a0033e40 con 0x7f7498004ab0 2023-10-18T09:09:36.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.283+0000 7f7497fff640 1 -- 172.21.15.40:0/3884597208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f749c072970 con 0x7f7498004ab0 2023-10-18T09:09:36.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.284+0000 7f74ae7fc640 1 --2- 172.21.15.40:0/3884597208 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f749c06ae40 0x7f749c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:36.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.285+0000 7f74ae7fc640 1 --2- 172.21.15.40:0/3884597208 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f749c06ae40 0x7f749c06d300 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f74b014df60 tx=0x7f74a4006a90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.288+0000 7f7497fff640 1 -- 172.21.15.40:0/3884597208 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f74a004b020 con 0x7f7498004ab0 2023-10-18T09:09:36.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.484+0000 7f74b59d0640 1 -- 172.21.15.40:0/3884597208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74b006c090 con 0x7f7498004ab0 2023-10-18T09:09:36.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.485+0000 7f7497fff640 1 -- 172.21.15.40:0/3884597208 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+16610 (secure 0 0 0) 0x7f74a0077580 con 0x7f7498004ab0 2023-10-18T09:09:36.488 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:36.488 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":200,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:09:36.223929+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:08:55.675479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-18T09:08:56.761888+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:09:36.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.488+0000 7f74b59d0640 1 -- 172.21.15.40:0/3884597208 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f749c06ae40 msgr2=0x7f749c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:36.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.488+0000 7f74b59d0640 1 --2- 172.21.15.40:0/3884597208 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f749c06ae40 0x7f749c06d300 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f74b014df60 tx=0x7f74a4006a90 comp rx=0 tx=0).stop 2023-10-18T09:09:36.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.488+0000 7f74b59d0640 1 -- 172.21.15.40:0/3884597208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7498004ab0 msgr2=0x7f74b0145230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:36.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.488+0000 7f74b59d0640 1 --2- 172.21.15.40:0/3884597208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7498004ab0 0x7f74b0145230 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009a70 tx=0x7f74a0033c50 comp rx=0 tx=0).stop 2023-10-18T09:09:36.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.489+0000 7f74b59d0640 1 -- 172.21.15.40:0/3884597208 shutdown_connections 2023-10-18T09:09:36.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.489+0000 7f74b59d0640 1 --2- 172.21.15.40:0/3884597208 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f749c06ae40 0x7f749c06d300 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:36.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.489+0000 7f74b59d0640 1 --2- 172.21.15.40:0/3884597208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74b00c24d0 0x7f74b0140970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.489+0000 7f74b59d0640 1 --2- 172.21.15.40:0/3884597208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74b00c2050 0x7f74b014d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.489+0000 7f74b59d0640 1 --2- 172.21.15.40:0/3884597208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7498004ab0 0x7f74b0145230 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.489+0000 7f74b59d0640 1 -- 172.21.15.40:0/3884597208 >> 172.21.15.40:0/3884597208 conn(0x7f74b006cee0 msgr2=0x7f74b01591c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.489+0000 7f74b59d0640 1 -- 172.21.15.40:0/3884597208 shutdown_connections 2023-10-18T09:09:36.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.489+0000 7f74b59d0640 1 -- 172.21.15.40:0/3884597208 wait complete. 2023-10-18T09:09:36.510 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:08:55.675479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '195', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:08:56.761888+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '200', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:09:36.511 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:09:36.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.742+0000 7f4f59639640 1 -- 172.21.15.40:0/3524269150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f5413dc60 msgr2=0x7f4f540af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:36.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.742+0000 7f4f59639640 1 --2- 172.21.15.40:0/3524269150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f5413dc60 0x7f4f540af190 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f4f540ce050 tx=0x7f4f4c02c080 comp rx=0 tx=0).stop 2023-10-18T09:09:36.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.742+0000 7f4f59639640 1 -- 172.21.15.40:0/3524269150 shutdown_connections 2023-10-18T09:09:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.742+0000 7f4f59639640 1 --2- 172.21.15.40:0/3524269150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f5413dc60 0x7f4f540af190 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.742+0000 7f4f59639640 1 --2- 172.21.15.40:0/3524269150 >> v2:172.21.15.144:3300/0 conn(0x7f4f5413c580 0x7f4f5415a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.743+0000 7f4f59639640 1 -- 172.21.15.40:0/3524269150 >> 172.21.15.40:0/3524269150 conn(0x7f4f54069720 msgr2=0x7f4f54069b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.743+0000 7f4f59639640 1 -- 172.21.15.40:0/3524269150 shutdown_connections 2023-10-18T09:09:36.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.743+0000 7f4f59639640 1 -- 172.21.15.40:0/3524269150 wait complete. 2023-10-18T09:09:36.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.744+0000 7f4f59639640 1 Processor -- start 2023-10-18T09:09:36.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.744+0000 7f4f59639640 1 -- start start 2023-10-18T09:09:36.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.745+0000 7f4f59639640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f540aed00 0x7f4f54139bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:36.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.745+0000 7f4f59639640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f5413c580 0x7f4f540afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:36.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.745+0000 7f4f59639640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4f5413dc60 0x7f4f540b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:36.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.745+0000 7f4f59639640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f540acce0 con 0x7f4f540aed00 2023-10-18T09:09:36.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.745+0000 7f4f59639640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f540acfc0 con 0x7f4f5413c580 2023-10-18T09:09:36.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.745+0000 7f4f59639640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f540ad2a0 con 0x7f4f5413dc60 2023-10-18T09:09:36.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.746+0000 7f4f52ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f540aed00 0x7f4f54139bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:36.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.746+0000 7f4f52ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f540aed00 0x7f4f54139bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55530/0 (socket says 172.21.15.40:55530) 2023-10-18T09:09:36.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.746+0000 7f4f52ffd640 1 -- 172.21.15.40:0/164401131 learned_addr learned my addr 172.21.15.40:0/164401131 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:36.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.746+0000 7f4f52ffd640 1 -- 172.21.15.40:0/164401131 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4f5413dc60 msgr2=0x7f4f540b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:36.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.746+0000 7f4f52ffd640 1 --2- 172.21.15.40:0/164401131 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4f5413dc60 0x7f4f540b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:36.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.746+0000 7f4f52ffd640 1 -- 172.21.15.40:0/164401131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f5413c580 msgr2=0x7f4f540afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:36.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.746+0000 7f4f52ffd640 1 --2- 172.21.15.40:0/164401131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f5413c580 0x7f4f540afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:36.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.746+0000 7f4f52ffd640 1 -- 172.21.15.40:0/164401131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c009840 con 0x7f4f540aed00 2023-10-18T09:09:36.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.746+0000 7f4f52ffd640 1 --2- 172.21.15.40:0/164401131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f540aed00 0x7f4f54139bc0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400d950 tx=0x7f4f4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:36.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.747+0000 7f4f4b7fe640 1 -- 172.21.15.40:0/164401131 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44009c30 con 0x7f4f540aed00 2023-10-18T09:09:36.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.747+0000 7f4f4b7fe640 1 -- 172.21.15.40:0/164401131 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4f44009e80 con 0x7f4f540aed00 2023-10-18T09:09:36.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.747+0000 7f4f4b7fe640 1 -- 172.21.15.40:0/164401131 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f440106a0 con 0x7f4f540aed00 2023-10-18T09:09:36.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.747+0000 7f4f59639640 1 -- 172.21.15.40:0/164401131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f540ad580 con 0x7f4f540aed00 2023-10-18T09:09:36.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.747+0000 7f4f59639640 1 -- 172.21.15.40:0/164401131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f540ccf40 con 0x7f4f540aed00 2023-10-18T09:09:36.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.748+0000 7f4f59639640 1 -- 172.21.15.40:0/164401131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f5406bec0 con 0x7f4f540aed00 2023-10-18T09:09:36.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.749+0000 7f4f4b7fe640 1 -- 172.21.15.40:0/164401131 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4f44010800 con 0x7f4f540aed00 2023-10-18T09:09:36.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.750+0000 7f4f4b7fe640 1 --2- 172.21.15.40:0/164401131 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4f3406ae40 0x7f4f3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:36.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.750+0000 7f4f4b7fe640 1 -- 172.21.15.40:0/164401131 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f44089b10 con 0x7f4f540aed00 2023-10-18T09:09:36.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.750+0000 7f4f4b7fe640 1 -- 172.21.15.40:0/164401131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f4f34072970 con 0x7f4f540aed00 2023-10-18T09:09:36.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.751+0000 7f4f4bfff640 1 --2- 172.21.15.40:0/164401131 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4f3406ae40 0x7f4f3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:36.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.752+0000 7f4f4bfff640 1 --2- 172.21.15.40:0/164401131 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4f3406ae40 0x7f4f3406d300 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4f540b0ae0 tx=0x7f4f3c007910 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:36.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.754+0000 7f4f4b7fe640 1 -- 172.21.15.40:0/164401131 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f44017060 con 0x7f4f540aed00 2023-10-18T09:09:36.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.949+0000 7f4f59639640 1 -- 172.21.15.40:0/164401131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f54066fd0 con 0x7f4f540aed00 2023-10-18T09:09:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.950+0000 7f4f4b7fe640 1 -- 172.21.15.40:0/164401131 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v200) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f44017070 con 0x7f4f540aed00 2023-10-18T09:09:36.952 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:09:36.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.952+0000 7f4f59639640 1 -- 172.21.15.40:0/164401131 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4f3406ae40 msgr2=0x7f4f3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.952+0000 7f4f59639640 1 --2- 172.21.15.40:0/164401131 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4f3406ae40 0x7f4f3406d300 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4f540b0ae0 tx=0x7f4f3c007910 comp rx=0 tx=0).stop 2023-10-18T09:09:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.952+0000 7f4f59639640 1 -- 172.21.15.40:0/164401131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f540aed00 msgr2=0x7f4f54139bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.952+0000 7f4f59639640 1 --2- 172.21.15.40:0/164401131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f540aed00 0x7f4f54139bc0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400d950 tx=0x7f4f4400de10 comp rx=0 tx=0).stop 2023-10-18T09:09:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.953+0000 7f4f59639640 1 -- 172.21.15.40:0/164401131 shutdown_connections 2023-10-18T09:09:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.953+0000 7f4f59639640 1 --2- 172.21.15.40:0/164401131 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4f3406ae40 0x7f4f3406d300 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:36.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.953+0000 7f4f59639640 1 --2- 172.21.15.40:0/164401131 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4f5413dc60 0x7f4f540b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:36.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.953+0000 7f4f59639640 1 --2- 172.21.15.40:0/164401131 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f5413c580 0x7f4f540afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:36.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.953+0000 7f4f59639640 1 --2- 172.21.15.40:0/164401131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f540aed00 0x7f4f54139bc0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:36.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.953+0000 7f4f59639640 1 -- 172.21.15.40:0/164401131 >> 172.21.15.40:0/164401131 conn(0x7f4f54069720 msgr2=0x7f4f5406b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:36.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.953+0000 7f4f59639640 1 -- 172.21.15.40:0/164401131 shutdown_connections 2023-10-18T09:09:36.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:36.953+0000 7f4f59639640 1 -- 172.21.15.40:0/164401131 wait complete. 2023-10-18T09:09:36.975 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:09:37.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.205+0000 7f293312c640 1 -- 172.21.15.40:0/3766507155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f292c15af70 msgr2=0x7f292c157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:37.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.205+0000 7f293312c640 1 --2- 172.21.15.40:0/3766507155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f292c15af70 0x7f292c157ad0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f2924005fd0 tx=0x7f292402c050 comp rx=0 tx=0).stop 2023-10-18T09:09:37.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.205+0000 7f293312c640 1 -- 172.21.15.40:0/3766507155 shutdown_connections 2023-10-18T09:09:37.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.205+0000 7f293312c640 1 --2- 172.21.15.40:0/3766507155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f292c15af70 0x7f292c157ad0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.205+0000 7f293312c640 1 --2- 172.21.15.40:0/3766507155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f292c071270 0x7f292c0c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.205+0000 7f293312c640 1 -- 172.21.15.40:0/3766507155 >> 172.21.15.40:0/3766507155 conn(0x7f292c06ca10 msgr2=0x7f292c06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:37.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.205+0000 7f293312c640 1 -- 172.21.15.40:0/3766507155 shutdown_connections 2023-10-18T09:09:37.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.205+0000 7f293312c640 1 -- 172.21.15.40:0/3766507155 wait complete. 2023-10-18T09:09:37.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.207+0000 7f293312c640 1 Processor -- start 2023-10-18T09:09:37.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.207+0000 7f293312c640 1 -- start start 2023-10-18T09:09:37.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.208+0000 7f293312c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f292c06d5d0 0x7f292c0c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:37.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.208+0000 7f293312c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f292c071270 0x7f292c14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:37.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.208+0000 7f293312c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f292c15af70 0x7f292c13e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:37.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.208+0000 7f293312c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292c14d510 con 0x7f292c06d5d0 2023-10-18T09:09:37.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.208+0000 7f293312c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292c1554e0 con 0x7f292c15af70 2023-10-18T09:09:37.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.208+0000 7f293312c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292c1557c0 con 0x7f292c071270 2023-10-18T09:09:37.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.208+0000 7f293212a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f292c06d5d0 0x7f292c0c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:37.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.208+0000 7f293212a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f292c06d5d0 0x7f292c0c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55542/0 (socket says 172.21.15.40:55542) 2023-10-18T09:09:37.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.208+0000 7f293212a640 1 -- 172.21.15.40:0/2966215378 learned_addr learned my addr 172.21.15.40:0/2966215378 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:37.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.209+0000 7f2931929640 1 --2- 172.21.15.40:0/2966215378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f292c071270 0x7f292c14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:37.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.209+0000 7f293212a640 1 -- 172.21.15.40:0/2966215378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f292c071270 msgr2=0x7f292c14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:37.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.209+0000 7f293292b640 1 --2- 172.21.15.40:0/2966215378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f292c15af70 0x7f292c13e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:37.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.209+0000 7f293212a640 1 --2- 172.21.15.40:0/2966215378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f292c071270 0x7f292c14cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.209+0000 7f293212a640 1 -- 172.21.15.40:0/2966215378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f292c15af70 msgr2=0x7f292c13e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:37.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.209+0000 7f293212a640 1 --2- 172.21.15.40:0/2966215378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f292c15af70 0x7f292c13e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.209+0000 7f293212a640 1 -- 172.21.15.40:0/2966215378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2924009840 con 0x7f292c06d5d0 2023-10-18T09:09:37.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.209+0000 7f293212a640 1 --2- 172.21.15.40:0/2966215378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f292c06d5d0 0x7f292c0c89f0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f292800c700 tx=0x7f292800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:37.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.210+0000 7f2922ffd640 1 -- 172.21.15.40:0/2966215378 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292800a840 con 0x7f292c06d5d0 2023-10-18T09:09:37.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.210+0000 7f2922ffd640 1 -- 172.21.15.40:0/2966215378 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f292800aa90 con 0x7f292c06d5d0 2023-10-18T09:09:37.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.210+0000 7f293312c640 1 -- 172.21.15.40:0/2966215378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292c155aa0 con 0x7f292c06d5d0 2023-10-18T09:09:37.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.210+0000 7f2922ffd640 1 -- 172.21.15.40:0/2966215378 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2928012420 con 0x7f292c06d5d0 2023-10-18T09:09:37.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.210+0000 7f293312c640 1 -- 172.21.15.40:0/2966215378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292c156060 con 0x7f292c06d5d0 2023-10-18T09:09:37.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.211+0000 7f293312c640 1 -- 172.21.15.40:0/2966215378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f292c156280 con 0x7f292c06d5d0 2023-10-18T09:09:37.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.212+0000 7f2922ffd640 1 -- 172.21.15.40:0/2966215378 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f292801a3b0 con 0x7f292c06d5d0 2023-10-18T09:09:37.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.213+0000 7f2922ffd640 1 --2- 172.21.15.40:0/2966215378 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f291006ae40 0x7f291006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:37.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.213+0000 7f2922ffd640 1 -- 172.21.15.40:0/2966215378 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2928089730 con 0x7f292c06d5d0 2023-10-18T09:09:37.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.213+0000 7f2922ffd640 1 -- 172.21.15.40:0/2966215378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f2910072970 con 0x7f292c06d5d0 2023-10-18T09:09:37.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.213+0000 7f2931929640 1 --2- 172.21.15.40:0/2966215378 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f291006ae40 0x7f291006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:37.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.215+0000 7f2931929640 1 --2- 172.21.15.40:0/2966215378 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f291006ae40 0x7f291006d300 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f291c005fd0 tx=0x7f291c00c540 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:37.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.217+0000 7f2922ffd640 1 -- 172.21.15.40:0/2966215378 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2928016060 con 0x7f292c06d5d0 2023-10-18T09:09:37.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.411+0000 7f293312c640 1 -- 172.21.15.40:0/2966215378 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f292c0c9b60 con 0x7f292c06d5d0 2023-10-18T09:09:37.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.412+0000 7f2922ffd640 1 -- 172.21.15.40:0/2966215378 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v200) v1 ==== 104+0+11 (secure 0 0 0) 0x7f292c0c9b60 con 0x7f292c06d5d0 2023-10-18T09:09:37.414 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:09:37.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.414+0000 7f293312c640 1 -- 172.21.15.40:0/2966215378 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f291006ae40 msgr2=0x7f291006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:37.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.414+0000 7f293312c640 1 --2- 172.21.15.40:0/2966215378 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f291006ae40 0x7f291006d300 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f291c005fd0 tx=0x7f291c00c540 comp rx=0 tx=0).stop 2023-10-18T09:09:37.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.414+0000 7f293312c640 1 -- 172.21.15.40:0/2966215378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f292c06d5d0 msgr2=0x7f292c0c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:37.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.414+0000 7f293312c640 1 --2- 172.21.15.40:0/2966215378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f292c06d5d0 0x7f292c0c89f0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f292800c700 tx=0x7f292800cbc0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.414+0000 7f293312c640 1 -- 172.21.15.40:0/2966215378 shutdown_connections 2023-10-18T09:09:37.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.414+0000 7f293312c640 1 --2- 172.21.15.40:0/2966215378 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f291006ae40 0x7f291006d300 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.414+0000 7f293312c640 1 --2- 172.21.15.40:0/2966215378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f292c15af70 0x7f292c13e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.414+0000 7f293312c640 1 --2- 172.21.15.40:0/2966215378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f292c071270 0x7f292c14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.414+0000 7f293312c640 1 --2- 172.21.15.40:0/2966215378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f292c06d5d0 0x7f292c0c89f0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.414+0000 7f293312c640 1 -- 172.21.15.40:0/2966215378 >> 172.21.15.40:0/2966215378 conn(0x7f292c06ca10 msgr2=0x7f292c14c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:37.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.415+0000 7f293312c640 1 -- 172.21.15.40:0/2966215378 shutdown_connections 2023-10-18T09:09:37.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.415+0000 7f293312c640 1 -- 172.21.15.40:0/2966215378 wait complete. 2023-10-18T09:09:37.437 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:09:37.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.651+0000 7f5ed4200640 1 -- 172.21.15.40:0/3659616204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ecc0717a0 msgr2=0x7f5ecc0c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:37.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.651+0000 7f5ed4200640 1 --2- 172.21.15.40:0/3659616204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ecc0717a0 0x7f5ecc0c9f60 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc06b500 tx=0x7f5ec402c260 comp rx=0 tx=0).stop 2023-10-18T09:09:37.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.651+0000 7f5ed4200640 1 -- 172.21.15.40:0/3659616204 shutdown_connections 2023-10-18T09:09:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.651+0000 7f5ed4200640 1 --2- 172.21.15.40:0/3659616204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ecc0717a0 0x7f5ecc0c9f60 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.651+0000 7f5ed4200640 1 --2- 172.21.15.40:0/3659616204 >> v2:172.21.15.144:3300/0 conn(0x7f5ecc0c9640 0x7f5ecc0c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.651+0000 7f5ed4200640 1 -- 172.21.15.40:0/3659616204 >> 172.21.15.40:0/3659616204 conn(0x7f5ecc069720 msgr2=0x7f5ecc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.652+0000 7f5ed4200640 1 -- 172.21.15.40:0/3659616204 shutdown_connections 2023-10-18T09:09:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.652+0000 7f5ed4200640 1 -- 172.21.15.40:0/3659616204 wait complete. 2023-10-18T09:09:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.653+0000 7f5ed4200640 1 Processor -- start 2023-10-18T09:09:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.654+0000 7f5ed4200640 1 -- start start 2023-10-18T09:09:37.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.654+0000 7f5ed4200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ecc06bc40 0x7f5ecc0c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:37.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.654+0000 7f5ed4200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc0717a0 0x7f5ecc0aced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:37.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.655+0000 7f5ed4200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ecc0c9640 0x7f5ecc1f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:37.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.655+0000 7f5ed4200640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc0af1e0 con 0x7f5ecc0c9640 2023-10-18T09:09:37.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.655+0000 7f5ed4200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc0ae680 con 0x7f5ecc0717a0 2023-10-18T09:09:37.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.655+0000 7f5ed4200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc0c2e70 con 0x7f5ecc06bc40 2023-10-18T09:09:37.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.655+0000 7f5ed279d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ecc0c9640 0x7f5ecc1f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:37.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.655+0000 7f5ed279d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ecc0c9640 0x7f5ecc1f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55564/0 (socket says 172.21.15.40:55564) 2023-10-18T09:09:37.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.655+0000 7f5ed279d640 1 -- 172.21.15.40:0/2910076430 learned_addr learned my addr 172.21.15.40:0/2910076430 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:37.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.655+0000 7f5ed1f9c640 1 --2- 172.21.15.40:0/2910076430 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ecc06bc40 0x7f5ecc0c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:37.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.655+0000 7f5ed279d640 1 -- 172.21.15.40:0/2910076430 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ecc06bc40 msgr2=0x7f5ecc0c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:37.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.655+0000 7f5ed179b640 1 --2- 172.21.15.40:0/2910076430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc0717a0 0x7f5ecc0aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:37.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.655+0000 7f5ed279d640 1 --2- 172.21.15.40:0/2910076430 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ecc06bc40 0x7f5ecc0c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.655+0000 7f5ed279d640 1 -- 172.21.15.40:0/2910076430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc0717a0 msgr2=0x7f5ecc0aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:37.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.655+0000 7f5ed279d640 1 --2- 172.21.15.40:0/2910076430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc0717a0 0x7f5ecc0aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.655+0000 7f5ed279d640 1 -- 172.21.15.40:0/2910076430 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec4009900 con 0x7f5ecc0c9640 2023-10-18T09:09:37.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.656+0000 7f5ed279d640 1 --2- 172.21.15.40:0/2910076430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ecc0c9640 0x7f5ecc1f5070 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009d60 tx=0x7f5ec4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.656+0000 7f5ec27fc640 1 -- 172.21.15.40:0/2910076430 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4004450 con 0x7f5ecc0c9640 2023-10-18T09:09:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.656+0000 7f5ec27fc640 1 -- 172.21.15.40:0/2910076430 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ec40046a0 con 0x7f5ecc0c9640 2023-10-18T09:09:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.656+0000 7f5ec27fc640 1 -- 172.21.15.40:0/2910076430 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec40406b0 con 0x7f5ecc0c9640 2023-10-18T09:09:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.656+0000 7f5ed4200640 1 -- 172.21.15.40:0/2910076430 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ecc0ad410 con 0x7f5ecc0c9640 2023-10-18T09:09:37.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.656+0000 7f5ed4200640 1 -- 172.21.15.40:0/2910076430 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ecc0c2b90 con 0x7f5ecc0c9640 2023-10-18T09:09:37.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.657+0000 7f5ed4200640 1 -- 172.21.15.40:0/2910076430 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ecc0ae830 con 0x7f5ecc0c9640 2023-10-18T09:09:37.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.658+0000 7f5ec27fc640 1 -- 172.21.15.40:0/2910076430 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5ec400a040 con 0x7f5ecc0c9640 2023-10-18T09:09:37.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.659+0000 7f5ec27fc640 1 --2- 172.21.15.40:0/2910076430 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5eb006ae40 0x7f5eb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:37.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.659+0000 7f5ec27fc640 1 -- 172.21.15.40:0/2910076430 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ec40acac0 con 0x7f5ecc0c9640 2023-10-18T09:09:37.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.660+0000 7f5ec27fc640 1 -- 172.21.15.40:0/2910076430 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f5eb0072970 con 0x7f5ecc0c9640 2023-10-18T09:09:37.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.660+0000 7f5ed1f9c640 1 --2- 172.21.15.40:0/2910076430 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5eb006ae40 0x7f5eb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:37.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.661+0000 7f5ed1f9c640 1 --2- 172.21.15.40:0/2910076430 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5eb006ae40 0x7f5eb006d300 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80058c0 tx=0x7f5ec8005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:37.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.663+0000 7f5ec27fc640 1 -- 172.21.15.40:0/2910076430 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ec4039060 con 0x7f5ecc0c9640 2023-10-18T09:09:37.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.852+0000 7f5ed4200640 1 -- 172.21.15.40:0/2910076430 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ecc066fd0 con 0x7f5ecc0c9640 2023-10-18T09:09:37.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.853+0000 7f5ec27fc640 1 -- 172.21.15.40:0/2910076430 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v200) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ec4044020 con 0x7f5ecc0c9640 2023-10-18T09:09:37.856 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:09:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.855+0000 7f5ed4200640 1 -- 172.21.15.40:0/2910076430 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5eb006ae40 msgr2=0x7f5eb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.855+0000 7f5ed4200640 1 --2- 172.21.15.40:0/2910076430 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5eb006ae40 0x7f5eb006d300 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80058c0 tx=0x7f5ec8005850 comp rx=0 tx=0).stop 2023-10-18T09:09:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.855+0000 7f5ed4200640 1 -- 172.21.15.40:0/2910076430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ecc0c9640 msgr2=0x7f5ecc1f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:37.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.855+0000 7f5ed4200640 1 --2- 172.21.15.40:0/2910076430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ecc0c9640 0x7f5ecc1f5070 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009d60 tx=0x7f5ec4003c20 comp rx=0 tx=0).stop 2023-10-18T09:09:37.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.856+0000 7f5ed4200640 1 -- 172.21.15.40:0/2910076430 shutdown_connections 2023-10-18T09:09:37.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.856+0000 7f5ed4200640 1 --2- 172.21.15.40:0/2910076430 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f5eb006ae40 0x7f5eb006d300 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.856+0000 7f5ed4200640 1 --2- 172.21.15.40:0/2910076430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ecc0c9640 0x7f5ecc1f5070 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.856+0000 7f5ed4200640 1 --2- 172.21.15.40:0/2910076430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ecc0717a0 0x7f5ecc0aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.856+0000 7f5ed4200640 1 --2- 172.21.15.40:0/2910076430 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ecc06bc40 0x7f5ecc0c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.856+0000 7f5ed4200640 1 -- 172.21.15.40:0/2910076430 >> 172.21.15.40:0/2910076430 conn(0x7f5ecc069720 msgr2=0x7f5ecc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.856+0000 7f5ed4200640 1 -- 172.21.15.40:0/2910076430 shutdown_connections 2023-10-18T09:09:37.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:37.856+0000 7f5ed4200640 1 -- 172.21.15.40:0/2910076430 wait complete. 2023-10-18T09:09:37.876 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:09:38.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.105+0000 7f99a71ed640 1 -- 172.21.15.40:0/1858577517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99a015b7d0 msgr2=0x7f99a006d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:38.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.105+0000 7f99a71ed640 1 --2- 172.21.15.40:0/1858577517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99a015b7d0 0x7f99a006d7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f998c005fd0 tx=0x7f998c02be10 comp rx=0 tx=0).stop 2023-10-18T09:09:38.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.106+0000 7f99a71ed640 1 -- 172.21.15.40:0/1858577517 shutdown_connections 2023-10-18T09:09:38.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.106+0000 7f99a71ed640 1 --2- 172.21.15.40:0/1858577517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99a015b7d0 0x7f99a006d7a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:38.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.106+0000 7f99a71ed640 1 --2- 172.21.15.40:0/1858577517 >> v2:172.21.15.144:3300/0 conn(0x7f99a006e190 0x7f99a006e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:38.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.106+0000 7f99a71ed640 1 -- 172.21.15.40:0/1858577517 >> 172.21.15.40:0/1858577517 conn(0x7f99a00b0af0 msgr2=0x7f99a00b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:38.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.106+0000 7f99a71ed640 1 -- 172.21.15.40:0/1858577517 shutdown_connections 2023-10-18T09:09:38.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.106+0000 7f99a71ed640 1 -- 172.21.15.40:0/1858577517 wait complete. 2023-10-18T09:09:38.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.108+0000 7f99a71ed640 1 Processor -- start 2023-10-18T09:09:38.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.108+0000 7f99a71ed640 1 -- start start 2023-10-18T09:09:38.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.109+0000 7f99a71ed640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99a006e190 0x7f99a00c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:38.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.109+0000 7f99a71ed640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99a00ae5d0 0x7f99a0154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:38.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.109+0000 7f99a71ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99a015b7d0 0x7f99a00ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:38.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.109+0000 7f99a71ed640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a00c2b30 con 0x7f99a015b7d0 2023-10-18T09:09:38.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.109+0000 7f99a71ed640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a00ca030 con 0x7f99a00ae5d0 2023-10-18T09:09:38.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.109+0000 7f99a71ed640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a01547c0 con 0x7f99a006e190 2023-10-18T09:09:38.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.109+0000 7f99a69ec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99a015b7d0 0x7f99a00ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:38.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.109+0000 7f99a69ec640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99a015b7d0 0x7f99a00ad1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55576/0 (socket says 172.21.15.40:55576) 2023-10-18T09:09:38.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.109+0000 7f99a69ec640 1 -- 172.21.15.40:0/2101861082 learned_addr learned my addr 172.21.15.40:0/2101861082 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:38.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.109+0000 7f99a61eb640 1 --2- 172.21.15.40:0/2101861082 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99a006e190 0x7f99a00c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:38.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.110+0000 7f99a59ea640 1 --2- 172.21.15.40:0/2101861082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99a00ae5d0 0x7f99a0154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:38.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.110+0000 7f99a69ec640 1 -- 172.21.15.40:0/2101861082 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99a006e190 msgr2=0x7f99a00c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.110+0000 7f99a69ec640 1 --2- 172.21.15.40:0/2101861082 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99a006e190 0x7f99a00c9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.110+0000 7f99a69ec640 1 -- 172.21.15.40:0/2101861082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99a00ae5d0 msgr2=0x7f99a0154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.110+0000 7f99a69ec640 1 --2- 172.21.15.40:0/2101861082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99a00ae5d0 0x7f99a0154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.110+0000 7f99a69ec640 1 -- 172.21.15.40:0/2101861082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c009560 con 0x7f99a015b7d0 2023-10-18T09:09:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.110+0000 7f99a61eb640 1 --2- 172.21.15.40:0/2101861082 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99a006e190 0x7f99a00c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:09:38.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.110+0000 7f99a69ec640 1 --2- 172.21.15.40:0/2101861082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99a015b7d0 0x7f99a00ad1d0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f999800c400 tx=0x7f999800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:38.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.110+0000 7f999effd640 1 -- 172.21.15.40:0/2101861082 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99980052d0 con 0x7f99a015b7d0 2023-10-18T09:09:38.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.110+0000 7f999effd640 1 -- 172.21.15.40:0/2101861082 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f99980035f0 con 0x7f99a015b7d0 2023-10-18T09:09:38.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.111+0000 7f99a71ed640 1 -- 172.21.15.40:0/2101861082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a01f57b0 con 0x7f99a015b7d0 2023-10-18T09:09:38.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.111+0000 7f999effd640 1 -- 172.21.15.40:0/2101861082 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99980039e0 con 0x7f99a015b7d0 2023-10-18T09:09:38.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.111+0000 7f99a71ed640 1 -- 172.21.15.40:0/2101861082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a01f5d70 con 0x7f99a015b7d0 2023-10-18T09:09:38.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.112+0000 7f99a71ed640 1 -- 172.21.15.40:0/2101861082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a00c2ca0 con 0x7f99a015b7d0 2023-10-18T09:09:38.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.113+0000 7f999effd640 1 -- 172.21.15.40:0/2101861082 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9998003b40 con 0x7f99a015b7d0 2023-10-18T09:09:38.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.114+0000 7f999effd640 1 --2- 172.21.15.40:0/2101861082 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f998406ae40 0x7f998406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:38.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.114+0000 7f999effd640 1 -- 172.21.15.40:0/2101861082 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9998088f00 con 0x7f99a015b7d0 2023-10-18T09:09:38.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.115+0000 7f999effd640 1 -- 172.21.15.40:0/2101861082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f9984072970 con 0x7f99a015b7d0 2023-10-18T09:09:38.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.115+0000 7f99a61eb640 1 --2- 172.21.15.40:0/2101861082 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f998406ae40 0x7f998406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:38.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.116+0000 7f99a61eb640 1 --2- 172.21.15.40:0/2101861082 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f998406ae40 0x7f998406d300 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9994005fd0 tx=0x7f9994009500 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:38.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.119+0000 7f999effd640 1 -- 172.21.15.40:0/2101861082 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9998017060 con 0x7f99a015b7d0 2023-10-18T09:09:38.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.333+0000 7f99a71ed640 1 -- 172.21.15.40:0/2101861082 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f99a00ce5b0 con 0x7f99a015b7d0 2023-10-18T09:09:38.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.334+0000 7f999effd640 1 -- 172.21.15.40:0/2101861082 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v200) v1 ==== 80+0+0 (secure 0 0 0) 0x7f99a00c2ca0 con 0x7f99a015b7d0 2023-10-18T09:09:38.336 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-18T09:09:38.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.336+0000 7f99a71ed640 1 -- 172.21.15.40:0/2101861082 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f998406ae40 msgr2=0x7f998406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:38.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.336+0000 7f99a71ed640 1 --2- 172.21.15.40:0/2101861082 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f998406ae40 0x7f998406d300 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9994005fd0 tx=0x7f9994009500 comp rx=0 tx=0).stop 2023-10-18T09:09:38.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.336+0000 7f99a71ed640 1 -- 172.21.15.40:0/2101861082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99a015b7d0 msgr2=0x7f99a00ad1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:38.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.336+0000 7f99a71ed640 1 --2- 172.21.15.40:0/2101861082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99a015b7d0 0x7f99a00ad1d0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f999800c400 tx=0x7f999800c8c0 comp rx=0 tx=0).stop 2023-10-18T09:09:38.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.336+0000 7f99a71ed640 1 -- 172.21.15.40:0/2101861082 shutdown_connections 2023-10-18T09:09:38.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.336+0000 7f99a71ed640 1 --2- 172.21.15.40:0/2101861082 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f998406ae40 0x7f998406d300 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:38.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.336+0000 7f99a71ed640 1 --2- 172.21.15.40:0/2101861082 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99a015b7d0 0x7f99a00ad1d0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:38.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.336+0000 7f99a71ed640 1 --2- 172.21.15.40:0/2101861082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99a00ae5d0 0x7f99a0154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:38.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.336+0000 7f99a71ed640 1 --2- 172.21.15.40:0/2101861082 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f99a006e190 0x7f99a00c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:38.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.336+0000 7f99a71ed640 1 -- 172.21.15.40:0/2101861082 >> 172.21.15.40:0/2101861082 conn(0x7f99a00b0af0 msgr2=0x7f99a01550d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:38.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.337+0000 7f99a71ed640 1 -- 172.21.15.40:0/2101861082 shutdown_connections 2023-10-18T09:09:38.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.337+0000 7f99a71ed640 1 -- 172.21.15.40:0/2101861082 wait complete. 2023-10-18T09:09:38.358 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-18T09:09:38.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:38.359 DEBUG:teuthology.orchestra.run.smithi144:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-18T09:09:38.473 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T09:09:38.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:38.474 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-18T09:09:38.531 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:09:38.531 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:09:38.595 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:09:38.596 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:09:38.596 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:09:38.610 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:09:38.619 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:09:38.658 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:09:38.659 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:09:38.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.872+0000 7fab054e5640 1 -- 172.21.15.40:0/1657734617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaec0047c0 msgr2=0x7faaec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:38.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.872+0000 7fab054e5640 1 --2- 172.21.15.40:0/1657734617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaec0047c0 0x7faaec004c40 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7faae8006f00 tx=0x7faaec0057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.872+0000 7fab054e5640 1 -- 172.21.15.40:0/1657734617 shutdown_connections 2023-10-18T09:09:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.872+0000 7fab054e5640 1 --2- 172.21.15.40:0/1657734617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaec0047c0 0x7faaec004c40 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.872+0000 7fab054e5640 1 -- 172.21.15.40:0/1657734617 >> 172.21.15.40:0/1657734617 conn(0x7fab000aff00 msgr2=0x7fab000b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.872+0000 7fab054e5640 1 -- 172.21.15.40:0/1657734617 shutdown_connections 2023-10-18T09:09:38.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.872+0000 7fab054e5640 1 -- 172.21.15.40:0/1657734617 wait complete. 2023-10-18T09:09:38.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.873+0000 7fab054e5640 1 Processor -- start 2023-10-18T09:09:38.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.874+0000 7fab054e5640 1 -- start start 2023-10-18T09:09:38.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.875+0000 7fab054e5640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaec0047c0 0x7fab001f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:38.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.875+0000 7fab054e5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab000c99f0 0x7fab001508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:38.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.875+0000 7fab054e5640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab000b1140 0x7fab000b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:38.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.875+0000 7fab054e5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab000b1b30 con 0x7fab000c99f0 2023-10-18T09:09:38.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.875+0000 7fab054e5640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab001514f0 con 0x7faaec0047c0 2023-10-18T09:09:38.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.875+0000 7fab054e5640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab000cd040 con 0x7fab000b1140 2023-10-18T09:09:38.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.875+0000 7faaf67fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab000c99f0 0x7fab001508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:38.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.875+0000 7faaf67fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab000c99f0 0x7fab001508a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55594/0 (socket says 172.21.15.40:55594) 2023-10-18T09:09:38.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.875+0000 7faaf67fc640 1 -- 172.21.15.40:0/2326271149 learned_addr learned my addr 172.21.15.40:0/2326271149 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:38.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.875+0000 7faafeffd640 1 --2- 172.21.15.40:0/2326271149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaec0047c0 0x7fab001f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:38.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.875+0000 7faaff7fe640 1 --2- 172.21.15.40:0/2326271149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab000b1140 0x7fab000b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:38.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.875+0000 7faaf67fc640 1 -- 172.21.15.40:0/2326271149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab000b1140 msgr2=0x7fab000b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:38.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.875+0000 7faaf67fc640 1 --2- 172.21.15.40:0/2326271149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab000b1140 0x7fab000b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:38.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.875+0000 7faaf67fc640 1 -- 172.21.15.40:0/2326271149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaec0047c0 msgr2=0x7fab001f15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:38.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.875+0000 7faaf67fc640 1 --2- 172.21.15.40:0/2326271149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaec0047c0 0x7fab001f15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:38.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.875+0000 7faaf67fc640 1 -- 172.21.15.40:0/2326271149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab000cd8e0 con 0x7fab000c99f0 2023-10-18T09:09:38.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.876+0000 7faaff7fe640 1 --2- 172.21.15.40:0/2326271149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab000b1140 0x7fab000b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:38.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.876+0000 7faafeffd640 1 --2- 172.21.15.40:0/2326271149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaec0047c0 0x7fab001f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:38.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.876+0000 7faaf67fc640 1 --2- 172.21.15.40:0/2326271149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab000c99f0 0x7fab001508a0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7faae8006f00 tx=0x7faae80016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:38.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.876+0000 7faaf7fff640 1 -- 172.21.15.40:0/2326271149 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae803c070 con 0x7fab000c99f0 2023-10-18T09:09:38.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.876+0000 7faaf7fff640 1 -- 172.21.15.40:0/2326271149 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faae80035f0 con 0x7fab000c99f0 2023-10-18T09:09:38.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.876+0000 7faaf7fff640 1 -- 172.21.15.40:0/2326271149 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae8003a00 con 0x7fab000c99f0 2023-10-18T09:09:38.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.876+0000 7fab054e5640 1 -- 172.21.15.40:0/2326271149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab001f1b00 con 0x7fab000c99f0 2023-10-18T09:09:38.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.876+0000 7fab054e5640 1 -- 172.21.15.40:0/2326271149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab000cd290 con 0x7fab000c99f0 2023-10-18T09:09:38.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.877+0000 7fab054e5640 1 -- 172.21.15.40:0/2326271149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab00066fd0 con 0x7fab000c99f0 2023-10-18T09:09:38.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.878+0000 7faaf7fff640 1 -- 172.21.15.40:0/2326271149 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faae8032070 con 0x7fab000c99f0 2023-10-18T09:09:38.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.880+0000 7faaf7fff640 1 --2- 172.21.15.40:0/2326271149 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faae006ae70 0x7faae006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:38.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.880+0000 7faaf7fff640 1 -- 172.21.15.40:0/2326271149 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faae80aa960 con 0x7fab000c99f0 2023-10-18T09:09:38.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.880+0000 7faaf7fff640 1 -- 172.21.15.40:0/2326271149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7faae00729a0 con 0x7fab000c99f0 2023-10-18T09:09:38.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.881+0000 7faafeffd640 1 --2- 172.21.15.40:0/2326271149 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faae006ae70 0x7faae006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:38.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.882+0000 7faafeffd640 1 --2- 172.21.15.40:0/2326271149 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faae006ae70 0x7faae006d330 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fab00062d20 tx=0x7faaf0006a70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:38.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:38.884+0000 7faaf7fff640 1 -- 172.21.15.40:0/2326271149 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faae8075a00 con 0x7fab000c99f0 2023-10-18T09:09:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.115+0000 7fab054e5640 1 -- 172.21.15.40:0/2326271149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab00157140 con 0x7fab000c99f0 2023-10-18T09:09:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.117+0000 7faaf7fff640 1 -- 172.21.15.40:0/2326271149 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4923 (secure 0 0 0) 0x7faae8079020 con 0x7fab000c99f0 2023-10-18T09:09:39.119 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:39.120 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2038256442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2038256442},{"type":"v1","addr":"172.21.15.144:6836","nonce":2038256442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3006484507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3006484507},{"type":"v1","addr":"172.21.15.144:6837","nonce":3006484507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1542000032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1542000032},{"type":"v1","addr":"172.21.15.40:6834","nonce":1542000032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:08:57.731109+0000","modified":"2023-10-18T09:09:00.823289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3976883051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3976883051},{"type":"v1","addr":"172.21.15.40:6835","nonce":3976883051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-18T09:09:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 90 2023-10-18T09:09:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.119+0000 7fab054e5640 1 -- 172.21.15.40:0/2326271149 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faae006ae70 msgr2=0x7faae006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.119+0000 7fab054e5640 1 --2- 172.21.15.40:0/2326271149 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faae006ae70 0x7faae006d330 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fab00062d20 tx=0x7faaf0006a70 comp rx=0 tx=0).stop 2023-10-18T09:09:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.119+0000 7fab054e5640 1 -- 172.21.15.40:0/2326271149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab000c99f0 msgr2=0x7fab001508a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.119+0000 7fab054e5640 1 --2- 172.21.15.40:0/2326271149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab000c99f0 0x7fab001508a0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7faae8006f00 tx=0x7faae80016f0 comp rx=0 tx=0).stop 2023-10-18T09:09:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.119+0000 7fab054e5640 1 -- 172.21.15.40:0/2326271149 shutdown_connections 2023-10-18T09:09:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.119+0000 7fab054e5640 1 --2- 172.21.15.40:0/2326271149 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7faae006ae70 0x7faae006d330 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.119+0000 7fab054e5640 1 --2- 172.21.15.40:0/2326271149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab000b1140 0x7fab000b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.119+0000 7fab054e5640 1 --2- 172.21.15.40:0/2326271149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab000c99f0 0x7fab001508a0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.119+0000 7fab054e5640 1 --2- 172.21.15.40:0/2326271149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faaec0047c0 0x7fab001f15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.119+0000 7fab054e5640 1 -- 172.21.15.40:0/2326271149 >> 172.21.15.40:0/2326271149 conn(0x7fab000aff00 msgr2=0x7fab000ca3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.120+0000 7fab054e5640 1 -- 172.21.15.40:0/2326271149 shutdown_connections 2023-10-18T09:09:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.120+0000 7fab054e5640 1 -- 172.21.15.40:0/2326271149 wait complete. 2023-10-18T09:09:39.141 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:09:39.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.368+0000 7fb9b49f0640 1 -- 172.21.15.40:0/2761582434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb99c004d00 msgr2=0x7fb99c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:39.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.368+0000 7fb9b49f0640 1 --2- 172.21.15.40:0/2761582434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb99c004d00 0x7fb99c005180 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80016b0 tx=0x7fb99c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:39.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.368+0000 7fb9aed9d640 1 -- 172.21.15.40:0/2761582434 reap_dead start 2023-10-18T09:09:39.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.369+0000 7fb9b49f0640 1 -- 172.21.15.40:0/2761582434 shutdown_connections 2023-10-18T09:09:39.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.369+0000 7fb9b49f0640 1 -- 172.21.15.40:0/2761582434 >> 172.21.15.40:0/2761582434 conn(0x7fb9b006a780 msgr2=0x7fb9b006ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:39.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.369+0000 7fb9b49f0640 1 -- 172.21.15.40:0/2761582434 shutdown_connections 2023-10-18T09:09:39.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.369+0000 7fb9b49f0640 1 -- 172.21.15.40:0/2761582434 wait complete. 2023-10-18T09:09:39.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.371+0000 7fb9b49f0640 1 Processor -- start 2023-10-18T09:09:39.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.371+0000 7fb9b49f0640 1 -- start start 2023-10-18T09:09:39.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.372+0000 7fb9b49f0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb99c004d00 0x7fb9b0142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:39.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.372+0000 7fb9b49f0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9b0145af0 0x7fb9b00b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:39.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.372+0000 7fb9b49f0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b00b1480 0x7fb9b01f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:39.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.372+0000 7fb9b49f0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b00b1900 con 0x7fb9b00b1480 2023-10-18T09:09:39.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.372+0000 7fb9b49f0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b00c4be0 con 0x7fb99c004d00 2023-10-18T09:09:39.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.372+0000 7fb9b49f0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b00b3ec0 con 0x7fb9b0145af0 2023-10-18T09:09:39.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.372+0000 7fb9aed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b00b1480 0x7fb9b01f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:39.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.372+0000 7fb9aed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b00b1480 0x7fb9b01f5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55606/0 (socket says 172.21.15.40:55606) 2023-10-18T09:09:39.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.372+0000 7fb9aed9d640 1 -- 172.21.15.40:0/643178736 learned_addr learned my addr 172.21.15.40:0/643178736 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:39.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.372+0000 7fb9ae59c640 1 --2- 172.21.15.40:0/643178736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb99c004d00 0x7fb9b0142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:39.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.372+0000 7fb9add9b640 1 --2- 172.21.15.40:0/643178736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9b0145af0 0x7fb9b00b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:39.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.373+0000 7fb9aed9d640 1 -- 172.21.15.40:0/643178736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9b0145af0 msgr2=0x7fb9b00b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:39.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.373+0000 7fb9aed9d640 1 --2- 172.21.15.40:0/643178736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9b0145af0 0x7fb9b00b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:39.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.373+0000 7fb9aed9d640 1 -- 172.21.15.40:0/643178736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb99c004d00 msgr2=0x7fb9b0142d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:39.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.373+0000 7fb9aed9d640 1 --2- 172.21.15.40:0/643178736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb99c004d00 0x7fb9b0142d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.373+0000 7fb9aed9d640 1 -- 172.21.15.40:0/643178736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b0143290 con 0x7fb9b00b1480 2023-10-18T09:09:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.373+0000 7fb9add9b640 1 --2- 172.21.15.40:0/643178736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9b0145af0 0x7fb9b00b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.373+0000 7fb9aed9d640 1 --2- 172.21.15.40:0/643178736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b00b1480 0x7fb9b01f5510 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fb9a802c1a0 tx=0x7fb9a80062e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.373+0000 7fb9a6ffd640 1 -- 172.21.15.40:0/643178736 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8003a00 con 0x7fb9b00b1480 2023-10-18T09:09:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.373+0000 7fb9a6ffd640 1 -- 172.21.15.40:0/643178736 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb9a8003c50 con 0x7fb9b00b1480 2023-10-18T09:09:39.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.373+0000 7fb9a6ffd640 1 -- 172.21.15.40:0/643178736 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a802c530 con 0x7fb9b00b1480 2023-10-18T09:09:39.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.373+0000 7fb9b49f0640 1 -- 172.21.15.40:0/643178736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b00b32b0 con 0x7fb9b00b1480 2023-10-18T09:09:39.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.373+0000 7fb9b49f0640 1 -- 172.21.15.40:0/643178736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b00b3870 con 0x7fb9b00b1480 2023-10-18T09:09:39.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.375+0000 7fb9b49f0640 1 -- 172.21.15.40:0/643178736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b00cff20 con 0x7fb9b00b1480 2023-10-18T09:09:39.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.376+0000 7fb9a6ffd640 1 -- 172.21.15.40:0/643178736 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb9a802c690 con 0x7fb9b00b1480 2023-10-18T09:09:39.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.377+0000 7fb9a6ffd640 1 --2- 172.21.15.40:0/643178736 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb99006ae40 0x7fb99006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:39.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.377+0000 7fb9a6ffd640 1 -- 172.21.15.40:0/643178736 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9a80a9f40 con 0x7fb9b00b1480 2023-10-18T09:09:39.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.377+0000 7fb9a6ffd640 1 -- 172.21.15.40:0/643178736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7fb990072970 con 0x7fb9b00b1480 2023-10-18T09:09:39.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.377+0000 7fb9ae59c640 1 --2- 172.21.15.40:0/643178736 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb99006ae40 0x7fb99006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:39.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.378+0000 7fb9ae59c640 1 --2- 172.21.15.40:0/643178736 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb99006ae40 0x7fb99006d300 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0002230 tx=0x7fb9a00019c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:39.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.382+0000 7fb9a6ffd640 1 -- 172.21.15.40:0/643178736 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb9a8074ff0 con 0x7fb9b00b1480 2023-10-18T09:09:39.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.574+0000 7fb9b49f0640 1 -- 172.21.15.40:0/643178736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9b0066fd0 con 0x7fb9b00b1480 2023-10-18T09:09:39.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.577+0000 7fb9a6ffd640 1 -- 172.21.15.40:0/643178736 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+16610 (secure 0 0 0) 0x7fb9a8078610 con 0x7fb9b00b1480 2023-10-18T09:09:39.580 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:39.580 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":200,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:09:36.223929+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:08:55.675479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-18T09:08:56.761888+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:09:39.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.580+0000 7fb9b49f0640 1 -- 172.21.15.40:0/643178736 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb99006ae40 msgr2=0x7fb99006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:39.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.580+0000 7fb9b49f0640 1 --2- 172.21.15.40:0/643178736 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb99006ae40 0x7fb99006d300 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0002230 tx=0x7fb9a00019c0 comp rx=0 tx=0).stop 2023-10-18T09:09:39.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.580+0000 7fb9b49f0640 1 -- 172.21.15.40:0/643178736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b00b1480 msgr2=0x7fb9b01f5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:39.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.580+0000 7fb9b49f0640 1 --2- 172.21.15.40:0/643178736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b00b1480 0x7fb9b01f5510 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fb9a802c1a0 tx=0x7fb9a80062e0 comp rx=0 tx=0).stop 2023-10-18T09:09:39.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.580+0000 7fb9b49f0640 1 -- 172.21.15.40:0/643178736 shutdown_connections 2023-10-18T09:09:39.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.580+0000 7fb9b49f0640 1 --2- 172.21.15.40:0/643178736 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb99006ae40 0x7fb99006d300 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:39.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.580+0000 7fb9b49f0640 1 --2- 172.21.15.40:0/643178736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9b00b1480 0x7fb9b01f5510 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:39.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.580+0000 7fb9b49f0640 1 --2- 172.21.15.40:0/643178736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9b0145af0 0x7fb9b00b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.580+0000 7fb9b49f0640 1 --2- 172.21.15.40:0/643178736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb99c004d00 0x7fb9b0142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.580+0000 7fb9b49f0640 1 -- 172.21.15.40:0/643178736 >> 172.21.15.40:0/643178736 conn(0x7fb9b006a780 msgr2=0x7fb9b015d400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.580+0000 7fb9b49f0640 1 -- 172.21.15.40:0/643178736 shutdown_connections 2023-10-18T09:09:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.581+0000 7fb9b49f0640 1 -- 172.21.15.40:0/643178736 wait complete. 2023-10-18T09:09:39.603 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T08:55:04.972118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:08:55.675479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '195', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:08:56.761888+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '200', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:09:39.604 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:09:39.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.840+0000 7f008359e640 1 -- 172.21.15.40:0/1983342036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0070005e10 msgr2=0x7f0070006290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:39.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.840+0000 7f008359e640 1 --2- 172.21.15.40:0/1983342036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0070005e10 0x7f0070006290 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f007c001d80 tx=0x7f0070006900 comp rx=0 tx=0).stop 2023-10-18T09:09:39.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.840+0000 7f0082d9d640 1 -- 172.21.15.40:0/1983342036 reap_dead start 2023-10-18T09:09:39.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.840+0000 7f008359e640 1 -- 172.21.15.40:0/1983342036 shutdown_connections 2023-10-18T09:09:39.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.840+0000 7f008359e640 1 -- 172.21.15.40:0/1983342036 >> 172.21.15.40:0/1983342036 conn(0x7f0084069720 msgr2=0x7f0084069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:39.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.840+0000 7f008359e640 1 -- 172.21.15.40:0/1983342036 shutdown_connections 2023-10-18T09:09:39.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.840+0000 7f008359e640 1 -- 172.21.15.40:0/1983342036 wait complete. 2023-10-18T09:09:39.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.842+0000 7f008359e640 1 Processor -- start 2023-10-18T09:09:39.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.842+0000 7f008359e640 1 -- start start 2023-10-18T09:09:39.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.843+0000 7f008359e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0070005e10 0x7f0084138400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:39.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.843+0000 7f008359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0084154b30 0x7f00841559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:39.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.843+0000 7f008359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0084155ef0 0x7f00841584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:39.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.843+0000 7f008359e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00841563a0 con 0x7f0084155ef0 2023-10-18T09:09:39.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.843+0000 7f008359e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00840cbcd0 con 0x7f0084154b30 2023-10-18T09:09:39.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.843+0000 7f008359e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00840cb0c0 con 0x7f0070005e10 2023-10-18T09:09:39.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.844+0000 7f0082d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0084155ef0 0x7f00841584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:39.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.844+0000 7f0082d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0084155ef0 0x7f00841584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55618/0 (socket says 172.21.15.40:55618) 2023-10-18T09:09:39.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.844+0000 7f0082d9d640 1 -- 172.21.15.40:0/1815688704 learned_addr learned my addr 172.21.15.40:0/1815688704 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.844+0000 7f007bfff640 1 --2- 172.21.15.40:0/1815688704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0084154b30 0x7f00841559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.844+0000 7f008259c640 1 --2- 172.21.15.40:0/1815688704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0070005e10 0x7f0084138400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.844+0000 7f0082d9d640 1 -- 172.21.15.40:0/1815688704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0070005e10 msgr2=0x7f0084138400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.844+0000 7f0082d9d640 1 --2- 172.21.15.40:0/1815688704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0070005e10 0x7f0084138400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.844+0000 7f0082d9d640 1 -- 172.21.15.40:0/1815688704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0084154b30 msgr2=0x7f00841559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.844+0000 7f0082d9d640 1 --2- 172.21.15.40:0/1815688704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0084154b30 0x7f00841559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.844+0000 7f0082d9d640 1 -- 172.21.15.40:0/1815688704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00840cb3a0 con 0x7f0084155ef0 2023-10-18T09:09:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.844+0000 7f007bfff640 1 --2- 172.21.15.40:0/1815688704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0084154b30 0x7f00841559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.844+0000 7f008259c640 1 --2- 172.21.15.40:0/1815688704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0070005e10 0x7f0084138400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.844+0000 7f0082d9d640 1 --2- 172.21.15.40:0/1815688704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0084155ef0 0x7f00841584e0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f007c001ea0 tx=0x7f007c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.845+0000 7f007affd640 1 -- 172.21.15.40:0/1815688704 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c02cc90 con 0x7f0084155ef0 2023-10-18T09:09:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.845+0000 7f007affd640 1 -- 172.21.15.40:0/1815688704 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f007c0060b0 con 0x7f0084155ef0 2023-10-18T09:09:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.845+0000 7f007affd640 1 -- 172.21.15.40:0/1815688704 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c0064a0 con 0x7f0084155ef0 2023-10-18T09:09:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.845+0000 7f008359e640 1 -- 172.21.15.40:0/1815688704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00840cb680 con 0x7f0084155ef0 2023-10-18T09:09:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.845+0000 7f008359e640 1 -- 172.21.15.40:0/1815688704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0084138940 con 0x7f0084155ef0 2023-10-18T09:09:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.846+0000 7f008359e640 1 -- 172.21.15.40:0/1815688704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0084066e00 con 0x7f0084155ef0 2023-10-18T09:09:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.847+0000 7f007affd640 1 -- 172.21.15.40:0/1815688704 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f007c006600 con 0x7f0084155ef0 2023-10-18T09:09:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.848+0000 7f007affd640 1 --2- 172.21.15.40:0/1815688704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f006406ae70 0x7f006406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.848+0000 7f007affd640 1 -- 172.21.15.40:0/1815688704 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f007c0aa910 con 0x7f0084155ef0 2023-10-18T09:09:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.848+0000 7f007affd640 1 -- 172.21.15.40:0/1815688704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f00640729a0 con 0x7f0084155ef0 2023-10-18T09:09:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.849+0000 7f008259c640 1 --2- 172.21.15.40:0/1815688704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f006406ae70 0x7f006406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.850+0000 7f008259c640 1 --2- 172.21.15.40:0/1815688704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f006406ae70 0x7f006406d330 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f00840c90b0 tx=0x7f007400e4c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:39.852+0000 7f007affd640 1 -- 172.21.15.40:0/1815688704 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f007c045070 con 0x7f0084155ef0 2023-10-18T09:09:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.048+0000 7f008359e640 1 -- 172.21.15.40:0/1815688704 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00840cbe10 con 0x7f0084155ef0 2023-10-18T09:09:40.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.049+0000 7f007affd640 1 -- 172.21.15.40:0/1815688704 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v200) v1 ==== 93+0+10 (secure 0 0 0) 0x7f007c0759d0 con 0x7f0084155ef0 2023-10-18T09:09:40.054 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-18T09:09:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.052+0000 7f008359e640 1 -- 172.21.15.40:0/1815688704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f006406ae70 msgr2=0x7f006406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.052+0000 7f008359e640 1 --2- 172.21.15.40:0/1815688704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f006406ae70 0x7f006406d330 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f00840c90b0 tx=0x7f007400e4c0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.052+0000 7f008359e640 1 -- 172.21.15.40:0/1815688704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0084155ef0 msgr2=0x7f00841584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.052+0000 7f008359e640 1 --2- 172.21.15.40:0/1815688704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0084155ef0 0x7f00841584e0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f007c001ea0 tx=0x7f007c003ba0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.052+0000 7f008359e640 1 -- 172.21.15.40:0/1815688704 shutdown_connections 2023-10-18T09:09:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.052+0000 7f008359e640 1 --2- 172.21.15.40:0/1815688704 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f006406ae70 0x7f006406d330 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.052+0000 7f008359e640 1 --2- 172.21.15.40:0/1815688704 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0084155ef0 0x7f00841584e0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.052+0000 7f008359e640 1 --2- 172.21.15.40:0/1815688704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0084154b30 0x7f00841559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.052+0000 7f008359e640 1 --2- 172.21.15.40:0/1815688704 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0070005e10 0x7f0084138400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.052+0000 7f008359e640 1 -- 172.21.15.40:0/1815688704 >> 172.21.15.40:0/1815688704 conn(0x7f0084069720 msgr2=0x7f00840b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.052+0000 7f008359e640 1 -- 172.21.15.40:0/1815688704 shutdown_connections 2023-10-18T09:09:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.052+0000 7f008359e640 1 -- 172.21.15.40:0/1815688704 wait complete. 2023-10-18T09:09:40.073 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:09:40.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.309+0000 7f3570eaa640 1 -- 172.21.15.40:0/3347659214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f356c06c030 msgr2=0x7f356c0abd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:40.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.309+0000 7f3570eaa640 1 --2- 172.21.15.40:0/3347659214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f356c06c030 0x7f356c0abd20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3554005fd0 tx=0x7f355402be10 comp rx=0 tx=0).stop 2023-10-18T09:09:40.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.309+0000 7f3570eaa640 1 -- 172.21.15.40:0/3347659214 shutdown_connections 2023-10-18T09:09:40.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.309+0000 7f3570eaa640 1 --2- 172.21.15.40:0/3347659214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f356c06c030 0x7f356c0abd20 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.309+0000 7f3570eaa640 1 --2- 172.21.15.40:0/3347659214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f356c0cc630 0x7f356c06baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.309+0000 7f3570eaa640 1 -- 172.21.15.40:0/3347659214 >> 172.21.15.40:0/3347659214 conn(0x7f356c06ca10 msgr2=0x7f356c06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:40.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.309+0000 7f3570eaa640 1 -- 172.21.15.40:0/3347659214 shutdown_connections 2023-10-18T09:09:40.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.309+0000 7f3570eaa640 1 -- 172.21.15.40:0/3347659214 wait complete. 2023-10-18T09:09:40.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.311+0000 7f3570eaa640 1 Processor -- start 2023-10-18T09:09:40.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.311+0000 7f3570eaa640 1 -- start start 2023-10-18T09:09:40.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.312+0000 7f3570eaa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f356c06c030 0x7f356c1f9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:40.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.312+0000 7f3570eaa640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f356c0ac260 0x7f356c148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:40.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.313+0000 7f3570eaa640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f356c0cc630 0x7f356c15dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:40.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.313+0000 7f3570eaa640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c1562d0 con 0x7f356c06c030 2023-10-18T09:09:40.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.313+0000 7f3570eaa640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c1fa4f0 con 0x7f356c0ac260 2023-10-18T09:09:40.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.313+0000 7f3570eaa640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356c148ac0 con 0x7f356c0cc630 2023-10-18T09:09:40.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.313+0000 7f356b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f356c06c030 0x7f356c1f9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:40.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.313+0000 7f356b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f356c06c030 0x7f356c1f9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55628/0 (socket says 172.21.15.40:55628) 2023-10-18T09:09:40.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.313+0000 7f356b7fe640 1 -- 172.21.15.40:0/1560846365 learned_addr learned my addr 172.21.15.40:0/1560846365 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:40.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.313+0000 7f3563fff640 1 --2- 172.21.15.40:0/1560846365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f356c0ac260 0x7f356c148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:40.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.313+0000 7f356b7fe640 1 -- 172.21.15.40:0/1560846365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f356c0cc630 msgr2=0x7f356c15dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:40.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.313+0000 7f356bfff640 1 --2- 172.21.15.40:0/1560846365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f356c0cc630 0x7f356c15dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:40.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.313+0000 7f356b7fe640 1 --2- 172.21.15.40:0/1560846365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f356c0cc630 0x7f356c15dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.313+0000 7f356b7fe640 1 -- 172.21.15.40:0/1560846365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f356c0ac260 msgr2=0x7f356c148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:40.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.313+0000 7f356b7fe640 1 --2- 172.21.15.40:0/1560846365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f356c0ac260 0x7f356c148580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.313+0000 7f356b7fe640 1 -- 172.21.15.40:0/1560846365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3554009560 con 0x7f356c06c030 2023-10-18T09:09:40.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.313+0000 7f356bfff640 1 --2- 172.21.15.40:0/1560846365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f356c0cc630 0x7f356c15dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:40.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.314+0000 7f356b7fe640 1 --2- 172.21.15.40:0/1560846365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f356c06c030 0x7f356c1f9fb0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f355c00e7c0 tx=0x7f355c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:40.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.314+0000 7f3568ff9640 1 -- 172.21.15.40:0/1560846365 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c009580 con 0x7f356c06c030 2023-10-18T09:09:40.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.314+0000 7f3568ff9640 1 -- 172.21.15.40:0/1560846365 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f355c00eed0 con 0x7f356c06c030 2023-10-18T09:09:40.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.314+0000 7f3568ff9640 1 -- 172.21.15.40:0/1560846365 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c010430 con 0x7f356c06c030 2023-10-18T09:09:40.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.314+0000 7f3570eaa640 1 -- 172.21.15.40:0/1560846365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356c06f6b0 con 0x7f356c06c030 2023-10-18T09:09:40.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.314+0000 7f3570eaa640 1 -- 172.21.15.40:0/1560846365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f356c0a7ce0 con 0x7f356c06c030 2023-10-18T09:09:40.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.315+0000 7f3570eaa640 1 -- 172.21.15.40:0/1560846365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f356c066e00 con 0x7f356c06c030 2023-10-18T09:09:40.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.316+0000 7f3568ff9640 1 -- 172.21.15.40:0/1560846365 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f355c0105b0 con 0x7f356c06c030 2023-10-18T09:09:40.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.317+0000 7f3568ff9640 1 --2- 172.21.15.40:0/1560846365 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f354c06ae40 0x7f354c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:40.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.317+0000 7f3568ff9640 1 -- 172.21.15.40:0/1560846365 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f355c014070 con 0x7f356c06c030 2023-10-18T09:09:40.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.318+0000 7f3568ff9640 1 -- 172.21.15.40:0/1560846365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f354c072970 con 0x7f356c06c030 2023-10-18T09:09:40.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.318+0000 7f3563fff640 1 --2- 172.21.15.40:0/1560846365 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f354c06ae40 0x7f354c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:40.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.319+0000 7f3563fff640 1 --2- 172.21.15.40:0/1560846365 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f354c06ae40 0x7f354c06d300 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3554035000 tx=0x7f3554004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:40.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.322+0000 7f3568ff9640 1 -- 172.21.15.40:0/1560846365 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f355c051ca0 con 0x7f356c06c030 2023-10-18T09:09:40.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.513+0000 7f3570eaa640 1 -- 172.21.15.40:0/1560846365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f356c0afa20 con 0x7f356c06c030 2023-10-18T09:09:40.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.514+0000 7f3568ff9640 1 -- 172.21.15.40:0/1560846365 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v200) v1 ==== 104+0+11 (secure 0 0 0) 0x7f355c058bd0 con 0x7f356c06c030 2023-10-18T09:09:40.516 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-18T09:09:40.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.517+0000 7f3570eaa640 1 -- 172.21.15.40:0/1560846365 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f354c06ae40 msgr2=0x7f354c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:40.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.517+0000 7f3570eaa640 1 --2- 172.21.15.40:0/1560846365 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f354c06ae40 0x7f354c06d300 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3554035000 tx=0x7f3554004730 comp rx=0 tx=0).stop 2023-10-18T09:09:40.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.517+0000 7f3570eaa640 1 -- 172.21.15.40:0/1560846365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f356c06c030 msgr2=0x7f356c1f9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:40.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.517+0000 7f3570eaa640 1 --2- 172.21.15.40:0/1560846365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f356c06c030 0x7f356c1f9fb0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f355c00e7c0 tx=0x7f355c00ec80 comp rx=0 tx=0).stop 2023-10-18T09:09:40.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.517+0000 7f3570eaa640 1 -- 172.21.15.40:0/1560846365 shutdown_connections 2023-10-18T09:09:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.517+0000 7f3570eaa640 1 --2- 172.21.15.40:0/1560846365 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f354c06ae40 0x7f354c06d300 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.517+0000 7f3570eaa640 1 --2- 172.21.15.40:0/1560846365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f356c0cc630 0x7f356c15dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.517+0000 7f3570eaa640 1 --2- 172.21.15.40:0/1560846365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f356c0ac260 0x7f356c148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.517+0000 7f3570eaa640 1 --2- 172.21.15.40:0/1560846365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f356c06c030 0x7f356c1f9fb0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.517+0000 7f3570eaa640 1 -- 172.21.15.40:0/1560846365 >> 172.21.15.40:0/1560846365 conn(0x7f356c06ca10 msgr2=0x7f356c0709d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.517+0000 7f3570eaa640 1 -- 172.21.15.40:0/1560846365 shutdown_connections 2023-10-18T09:09:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.518+0000 7f3570eaa640 1 -- 172.21.15.40:0/1560846365 wait complete. 2023-10-18T09:09:40.539 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:09:40.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.767+0000 7f565c2c2640 1 -- 172.21.15.40:0/671690207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56540cea60 msgr2=0x7f56540d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:40.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.767+0000 7f565c2c2640 1 --2- 172.21.15.40:0/671690207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56540cea60 0x7f56540d1360 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f5644005fd0 tx=0x7f564402bd90 comp rx=0 tx=0).stop 2023-10-18T09:09:40.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.767+0000 7f565c2c2640 1 -- 172.21.15.40:0/671690207 shutdown_connections 2023-10-18T09:09:40.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.767+0000 7f565c2c2640 1 --2- 172.21.15.40:0/671690207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56540cea60 0x7f56540d1360 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.767+0000 7f565c2c2640 1 --2- 172.21.15.40:0/671690207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f565415b670 0x7f56540ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.767+0000 7f565c2c2640 1 -- 172.21.15.40:0/671690207 >> 172.21.15.40:0/671690207 conn(0x7f565408ae20 msgr2=0x7f565409a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:40.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.767+0000 7f565c2c2640 1 -- 172.21.15.40:0/671690207 shutdown_connections 2023-10-18T09:09:40.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.767+0000 7f565c2c2640 1 -- 172.21.15.40:0/671690207 wait complete. 2023-10-18T09:09:40.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.769+0000 7f565c2c2640 1 Processor -- start 2023-10-18T09:09:40.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.769+0000 7f565c2c2640 1 -- start start 2023-10-18T09:09:40.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.770+0000 7f565c2c2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56540cea60 0x7f56540c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:40.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.770+0000 7f565c2c2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56540d18a0 0x7f56541f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:40.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.770+0000 7f565c2c2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f565415b670 0x7f56541f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.770+0000 7f565c2c2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56541f3210 con 0x7f56540d18a0 2023-10-18T09:09:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.770+0000 7f565c2c2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56541f2ee0 con 0x7f565415b670 2023-10-18T09:09:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.770+0000 7f565c2c2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565406bac0 con 0x7f56540cea60 2023-10-18T09:09:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.771+0000 7f565985d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56540d18a0 0x7f56541f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.771+0000 7f565985d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56540d18a0 0x7f56541f5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55646/0 (socket says 172.21.15.40:55646) 2023-10-18T09:09:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.771+0000 7f565985d640 1 -- 172.21.15.40:0/3644299191 learned_addr learned my addr 172.21.15.40:0/3644299191 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.771+0000 7f565a05e640 1 --2- 172.21.15.40:0/3644299191 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56540cea60 0x7f56540c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.771+0000 7f565a85f640 1 --2- 172.21.15.40:0/3644299191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f565415b670 0x7f56541f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.771+0000 7f565985d640 1 -- 172.21.15.40:0/3644299191 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56540cea60 msgr2=0x7f56540c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.771+0000 7f565985d640 1 --2- 172.21.15.40:0/3644299191 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56540cea60 0x7f56540c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.771+0000 7f565985d640 1 -- 172.21.15.40:0/3644299191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f565415b670 msgr2=0x7f56541f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:40.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.771+0000 7f565985d640 1 --2- 172.21.15.40:0/3644299191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f565415b670 0x7f56541f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.771+0000 7f565985d640 1 -- 172.21.15.40:0/3644299191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5644009560 con 0x7f56540d18a0 2023-10-18T09:09:40.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.771+0000 7f565985d640 1 --2- 172.21.15.40:0/3644299191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56540d18a0 0x7f56541f5570 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f5644006010 tx=0x7f564402db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:40.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.772+0000 7f564affd640 1 -- 172.21.15.40:0/3644299191 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56440035f0 con 0x7f56540d18a0 2023-10-18T09:09:40.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.772+0000 7f564affd640 1 -- 172.21.15.40:0/3644299191 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5644003840 con 0x7f56540d18a0 2023-10-18T09:09:40.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.772+0000 7f564affd640 1 -- 172.21.15.40:0/3644299191 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644003c50 con 0x7f56540d18a0 2023-10-18T09:09:40.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.772+0000 7f565c2c2640 1 -- 172.21.15.40:0/3644299191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f565406bda0 con 0x7f56540d18a0 2023-10-18T09:09:40.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.772+0000 7f565c2c2640 1 -- 172.21.15.40:0/3644299191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56540c6380 con 0x7f56540d18a0 2023-10-18T09:09:40.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.773+0000 7f565c2c2640 1 -- 172.21.15.40:0/3644299191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5654066fd0 con 0x7f56540d18a0 2023-10-18T09:09:40.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.774+0000 7f564affd640 1 -- 172.21.15.40:0/3644299191 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f564403e450 con 0x7f56540d18a0 2023-10-18T09:09:40.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.775+0000 7f564affd640 1 --2- 172.21.15.40:0/3644299191 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f563806ae70 0x7f563806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:40.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.775+0000 7f564affd640 1 -- 172.21.15.40:0/3644299191 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56440ac1d0 con 0x7f56540d18a0 2023-10-18T09:09:40.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.775+0000 7f564affd640 1 -- 172.21.15.40:0/3644299191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f56380729a0 con 0x7f56540d18a0 2023-10-18T09:09:40.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.776+0000 7f565a05e640 1 --2- 172.21.15.40:0/3644299191 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f563806ae70 0x7f563806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:40.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.777+0000 7f565a05e640 1 --2- 172.21.15.40:0/3644299191 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f563806ae70 0x7f563806d330 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f5650005fd0 tx=0x7f56500097d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:40.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.779+0000 7f564affd640 1 -- 172.21.15.40:0/3644299191 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5644077290 con 0x7f56540d18a0 2023-10-18T09:09:40.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.975+0000 7f565c2c2640 1 -- 172.21.15.40:0/3644299191 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f565415e9f0 con 0x7f56540d18a0 2023-10-18T09:09:40.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.976+0000 7f564affd640 1 -- 172.21.15.40:0/3644299191 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v200) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5644043030 con 0x7f56540d18a0 2023-10-18T09:09:40.978 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-18T09:09:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.978+0000 7f565c2c2640 1 -- 172.21.15.40:0/3644299191 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f563806ae70 msgr2=0x7f563806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.978+0000 7f565c2c2640 1 --2- 172.21.15.40:0/3644299191 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f563806ae70 0x7f563806d330 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f5650005fd0 tx=0x7f56500097d0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.978+0000 7f565c2c2640 1 -- 172.21.15.40:0/3644299191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56540d18a0 msgr2=0x7f56541f5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:40.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.978+0000 7f565c2c2640 1 --2- 172.21.15.40:0/3644299191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56540d18a0 0x7f56541f5570 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f5644006010 tx=0x7f564402db30 comp rx=0 tx=0).stop 2023-10-18T09:09:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.979+0000 7f565c2c2640 1 -- 172.21.15.40:0/3644299191 shutdown_connections 2023-10-18T09:09:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.979+0000 7f565c2c2640 1 --2- 172.21.15.40:0/3644299191 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f563806ae70 0x7f563806d330 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.979+0000 7f565c2c2640 1 --2- 172.21.15.40:0/3644299191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f565415b670 0x7f56541f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.979+0000 7f565c2c2640 1 --2- 172.21.15.40:0/3644299191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56540d18a0 0x7f56541f5570 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.979+0000 7f565c2c2640 1 --2- 172.21.15.40:0/3644299191 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56540cea60 0x7f56540c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:40.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.979+0000 7f565c2c2640 1 -- 172.21.15.40:0/3644299191 >> 172.21.15.40:0/3644299191 conn(0x7f565408ae20 msgr2=0x7f565415a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:40.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.979+0000 7f565c2c2640 1 -- 172.21.15.40:0/3644299191 shutdown_connections 2023-10-18T09:09:40.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:40.979+0000 7f565c2c2640 1 -- 172.21.15.40:0/3644299191 wait complete. 2023-10-18T09:09:41.000 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:09:41.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.227+0000 7f9a97fff640 1 -- 172.21.15.40:0/1479824085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a84004d00 msgr2=0x7f9a84005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:41.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.227+0000 7f9a97fff640 1 --2- 172.21.15.40:0/1479824085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a84004d00 0x7f9a84005180 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000bb20 tx=0x7f9a840057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.227+0000 7f9a977fe640 1 -- 172.21.15.40:0/1479824085 reap_dead start 2023-10-18T09:09:41.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.227+0000 7f9a97fff640 1 -- 172.21.15.40:0/1479824085 shutdown_connections 2023-10-18T09:09:41.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.227+0000 7f9a97fff640 1 -- 172.21.15.40:0/1479824085 >> 172.21.15.40:0/1479824085 conn(0x7f9a980b0b00 msgr2=0x7f9a980b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:41.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.227+0000 7f9a97fff640 1 -- 172.21.15.40:0/1479824085 shutdown_connections 2023-10-18T09:09:41.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.227+0000 7f9a97fff640 1 -- 172.21.15.40:0/1479824085 wait complete. 2023-10-18T09:09:41.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.229+0000 7f9a97fff640 1 Processor -- start 2023-10-18T09:09:41.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.229+0000 7f9a97fff640 1 -- start start 2023-10-18T09:09:41.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.230+0000 7f9a97fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a84004d00 0x7f9a98158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:41.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.230+0000 7f9a97fff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a98158b70 0x7f9a981f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:41.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.230+0000 7f9a97fff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a981f55e0 0x7f9a981f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:41.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.230+0000 7f9a97fff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a98158ff0 con 0x7f9a84004d00 2023-10-18T09:09:41.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.230+0000 7f9a97fff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9806df20 con 0x7f9a98158b70 2023-10-18T09:09:41.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.230+0000 7f9a97fff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a980c80b0 con 0x7f9a981f55e0 2023-10-18T09:09:41.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.231+0000 7f9a96ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a84004d00 0x7f9a98158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:41.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.231+0000 7f9a96ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a84004d00 0x7f9a98158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55664/0 (socket says 172.21.15.40:55664) 2023-10-18T09:09:41.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.231+0000 7f9a96ffd640 1 -- 172.21.15.40:0/2586247643 learned_addr learned my addr 172.21.15.40:0/2586247643 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:41.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.231+0000 7f9a967fc640 1 --2- 172.21.15.40:0/2586247643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a98158b70 0x7f9a981f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:41.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.231+0000 7f9a96ffd640 1 -- 172.21.15.40:0/2586247643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a981f55e0 msgr2=0x7f9a981f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:41.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.231+0000 7f9a96ffd640 1 --2- 172.21.15.40:0/2586247643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a981f55e0 0x7f9a981f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.231+0000 7f9a96ffd640 1 -- 172.21.15.40:0/2586247643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a98158b70 msgr2=0x7f9a981f50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:41.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.231+0000 7f9a96ffd640 1 --2- 172.21.15.40:0/2586247643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a98158b70 0x7f9a981f50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.231+0000 7f9a96ffd640 1 -- 172.21.15.40:0/2586247643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9806d5f0 con 0x7f9a84004d00 2023-10-18T09:09:41.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.232+0000 7f9a96ffd640 1 --2- 172.21.15.40:0/2586247643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a84004d00 0x7f9a98158630 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800db00 tx=0x7f9a8800b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:41.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.232+0000 7f9a8f7fe640 1 -- 172.21.15.40:0/2586247643 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88003b80 con 0x7f9a84004d00 2023-10-18T09:09:41.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.232+0000 7f9a8f7fe640 1 -- 172.21.15.40:0/2586247643 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9a88003ce0 con 0x7f9a84004d00 2023-10-18T09:09:41.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.232+0000 7f9a8f7fe640 1 -- 172.21.15.40:0/2586247643 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8800baf0 con 0x7f9a84004d00 2023-10-18T09:09:41.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.232+0000 7f9a97fff640 1 -- 172.21.15.40:0/2586247643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9806d8d0 con 0x7f9a84004d00 2023-10-18T09:09:41.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.232+0000 7f9a97fff640 1 -- 172.21.15.40:0/2586247643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a980c6bb0 con 0x7f9a84004d00 2023-10-18T09:09:41.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.233+0000 7f9a97fff640 1 -- 172.21.15.40:0/2586247643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a98066e00 con 0x7f9a84004d00 2023-10-18T09:09:41.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.234+0000 7f9a8f7fe640 1 -- 172.21.15.40:0/2586247643 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9a8800bc50 con 0x7f9a84004d00 2023-10-18T09:09:41.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.236+0000 7f9a8f7fe640 1 --2- 172.21.15.40:0/2586247643 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a7806ae40 0x7f9a7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:41.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.236+0000 7f9a8f7fe640 1 -- 172.21.15.40:0/2586247643 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a880892e0 con 0x7f9a84004d00 2023-10-18T09:09:41.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.236+0000 7f9a8f7fe640 1 -- 172.21.15.40:0/2586247643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f9a78072970 con 0x7f9a84004d00 2023-10-18T09:09:41.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.236+0000 7f9a967fc640 1 --2- 172.21.15.40:0/2586247643 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a7806ae40 0x7f9a7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:41.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.237+0000 7f9a967fc640 1 --2- 172.21.15.40:0/2586247643 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a7806ae40 0x7f9a7806d300 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9a980c6020 tx=0x7f9a8000c2d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:41.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.240+0000 7f9a8f7fe640 1 -- 172.21.15.40:0/2586247643 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9a88019020 con 0x7f9a84004d00 2023-10-18T09:09:41.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.475+0000 7f9a97fff640 1 -- 172.21.15.40:0/2586247643 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a980b1760 con 0x7f9a84004d00 2023-10-18T09:09:41.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.477+0000 7f9a8f7fe640 1 -- 172.21.15.40:0/2586247643 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f9a880542f0 con 0x7f9a84004d00 2023-10-18T09:09:41.479 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:41.480 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2038256442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2038256442},{"type":"v1","addr":"172.21.15.144:6836","nonce":2038256442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3006484507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3006484507},{"type":"v1","addr":"172.21.15.144:6837","nonce":3006484507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1542000032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1542000032},{"type":"v1","addr":"172.21.15.40:6834","nonce":1542000032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:08:57.731109+0000","modified":"2023-10-18T09:09:00.823289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3976883051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3976883051},{"type":"v1","addr":"172.21.15.40:6835","nonce":3976883051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-18T09:09:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 90 2023-10-18T09:09:41.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.479+0000 7f9a97fff640 1 -- 172.21.15.40:0/2586247643 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a7806ae40 msgr2=0x7f9a7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:41.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.479+0000 7f9a97fff640 1 --2- 172.21.15.40:0/2586247643 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a7806ae40 0x7f9a7806d300 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9a980c6020 tx=0x7f9a8000c2d0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.479+0000 7f9a97fff640 1 -- 172.21.15.40:0/2586247643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a84004d00 msgr2=0x7f9a98158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:41.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.479+0000 7f9a97fff640 1 --2- 172.21.15.40:0/2586247643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a84004d00 0x7f9a98158630 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800db00 tx=0x7f9a8800b0e0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.479+0000 7f9a97fff640 1 -- 172.21.15.40:0/2586247643 shutdown_connections 2023-10-18T09:09:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.479+0000 7f9a97fff640 1 --2- 172.21.15.40:0/2586247643 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f9a7806ae40 0x7f9a7806d300 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.479+0000 7f9a97fff640 1 --2- 172.21.15.40:0/2586247643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a981f55e0 0x7f9a981f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.479+0000 7f9a97fff640 1 --2- 172.21.15.40:0/2586247643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a98158b70 0x7f9a981f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.479+0000 7f9a97fff640 1 --2- 172.21.15.40:0/2586247643 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a84004d00 0x7f9a98158630 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.479+0000 7f9a97fff640 1 -- 172.21.15.40:0/2586247643 >> 172.21.15.40:0/2586247643 conn(0x7f9a980b0b00 msgr2=0x7f9a9815f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.480+0000 7f9a97fff640 1 -- 172.21.15.40:0/2586247643 shutdown_connections 2023-10-18T09:09:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.480+0000 7f9a97fff640 1 -- 172.21.15.40:0/2586247643 wait complete. 2023-10-18T09:09:41.499 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:09:41.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.707+0000 7fbfeb59e640 1 -- 172.21.15.40:0/3516139271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec0b0630 msgr2=0x7fbfec0c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:41.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.707+0000 7fbfeb59e640 1 --2- 172.21.15.40:0/3516139271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec0b0630 0x7fbfec0c99c0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fbfec06aeb0 tx=0x7fbfd402bd70 comp rx=0 tx=0).stop 2023-10-18T09:09:41.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.708+0000 7fbfeb59e640 1 -- 172.21.15.40:0/3516139271 shutdown_connections 2023-10-18T09:09:41.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.708+0000 7fbfeb59e640 1 --2- 172.21.15.40:0/3516139271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec0b0630 0x7fbfec0c99c0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.708+0000 7fbfeb59e640 1 --2- 172.21.15.40:0/3516139271 >> v2:172.21.15.144:3300/0 conn(0x7fbfec06d6f0 0x7fbfec0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.708+0000 7fbfeb59e640 1 -- 172.21.15.40:0/3516139271 >> 172.21.15.40:0/3516139271 conn(0x7fbfec069720 msgr2=0x7fbfec069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:41.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.708+0000 7fbfeb59e640 1 -- 172.21.15.40:0/3516139271 shutdown_connections 2023-10-18T09:09:41.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.708+0000 7fbfeb59e640 1 -- 172.21.15.40:0/3516139271 wait complete. 2023-10-18T09:09:41.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.709+0000 7fbfeb59e640 1 Processor -- start 2023-10-18T09:09:41.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.710+0000 7fbfeb59e640 1 -- start start 2023-10-18T09:09:41.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.710+0000 7fbfeb59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec06d6f0 0x7fbfec14cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:41.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.710+0000 7fbfeb59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfec0b0630 0x7fbfec1e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:41.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.710+0000 7fbfeb59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfec0c9f00 0x7fbfec1e12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:41.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.710+0000 7fbfeb59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec1e2670 con 0x7fbfec06d6f0 2023-10-18T09:09:41.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.710+0000 7fbfeb59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec14d070 con 0x7fbfec0b0630 2023-10-18T09:09:41.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.710+0000 7fbfeb59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec155610 con 0x7fbfec0c9f00 2023-10-18T09:09:41.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.711+0000 7fbfea59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec06d6f0 0x7fbfec14cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:41.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.711+0000 7fbfea59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec06d6f0 0x7fbfec14cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55682/0 (socket says 172.21.15.40:55682) 2023-10-18T09:09:41.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.711+0000 7fbfea59c640 1 -- 172.21.15.40:0/3558120770 learned_addr learned my addr 172.21.15.40:0/3558120770 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:41.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.711+0000 7fbfea59c640 1 -- 172.21.15.40:0/3558120770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfec0c9f00 msgr2=0x7fbfec1e12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:41.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.711+0000 7fbfea59c640 1 --2- 172.21.15.40:0/3558120770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfec0c9f00 0x7fbfec1e12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.711+0000 7fbfea59c640 1 -- 172.21.15.40:0/3558120770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfec0b0630 msgr2=0x7fbfec1e0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:41.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.711+0000 7fbfea59c640 1 --2- 172.21.15.40:0/3558120770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfec0b0630 0x7fbfec1e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.711+0000 7fbfea59c640 1 -- 172.21.15.40:0/3558120770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd4009490 con 0x7fbfec06d6f0 2023-10-18T09:09:41.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.711+0000 7fbfea59c640 1 --2- 172.21.15.40:0/3558120770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec06d6f0 0x7fbfec14cb30 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00ddc0 tx=0x7fbfdc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:41.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.712+0000 7fbfe2ffd640 1 -- 172.21.15.40:0/3558120770 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc00e650 con 0x7fbfec06d6f0 2023-10-18T09:09:41.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.712+0000 7fbfe2ffd640 1 -- 172.21.15.40:0/3558120770 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbfdc00e8a0 con 0x7fbfec06d6f0 2023-10-18T09:09:41.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.712+0000 7fbfe2ffd640 1 -- 172.21.15.40:0/3558120770 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc00ecb0 con 0x7fbfec06d6f0 2023-10-18T09:09:41.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.712+0000 7fbfeb59e640 1 -- 172.21.15.40:0/3558120770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfec0aea20 con 0x7fbfec06d6f0 2023-10-18T09:09:41.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.712+0000 7fbfeb59e640 1 -- 172.21.15.40:0/3558120770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfec0aefe0 con 0x7fbfec06d6f0 2023-10-18T09:09:41.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.713+0000 7fbfeb59e640 1 -- 172.21.15.40:0/3558120770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfec066e00 con 0x7fbfec06d6f0 2023-10-18T09:09:41.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.714+0000 7fbfe2ffd640 1 -- 172.21.15.40:0/3558120770 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbfdc016440 con 0x7fbfec06d6f0 2023-10-18T09:09:41.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.715+0000 7fbfe2ffd640 1 --2- 172.21.15.40:0/3558120770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbfcc06ae40 0x7fbfcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:41.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.715+0000 7fbfe2ffd640 1 -- 172.21.15.40:0/3558120770 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfdc089db0 con 0x7fbfec06d6f0 2023-10-18T09:09:41.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.716+0000 7fbfe2ffd640 1 -- 172.21.15.40:0/3558120770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7fbfcc072970 con 0x7fbfec06d6f0 2023-10-18T09:09:41.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.716+0000 7fbfe3fff640 1 --2- 172.21.15.40:0/3558120770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbfcc06ae40 0x7fbfcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:41.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.717+0000 7fbfe3fff640 1 --2- 172.21.15.40:0/3558120770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbfcc06ae40 0x7fbfcc06d300 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4036000 tx=0x7fbfd40046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:41.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.720+0000 7fbfe2ffd640 1 -- 172.21.15.40:0/3558120770 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbfdc01b020 con 0x7fbfec06d6f0 2023-10-18T09:09:41.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.906+0000 7fbfeb59e640 1 -- 172.21.15.40:0/3558120770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfec0caf90 con 0x7fbfec06d6f0 2023-10-18T09:09:41.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.909+0000 7fbfe2ffd640 1 -- 172.21.15.40:0/3558120770 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+16610 (secure 0 0 0) 0x7fbfdc058a30 con 0x7fbfec06d6f0 2023-10-18T09:09:41.911 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:41.912 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":200,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:09:36.223929+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:08:55.675479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-18T09:08:56.761888+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:09:41.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.911+0000 7fbfeb59e640 1 -- 172.21.15.40:0/3558120770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbfcc06ae40 msgr2=0x7fbfcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:41.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.911+0000 7fbfeb59e640 1 --2- 172.21.15.40:0/3558120770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbfcc06ae40 0x7fbfcc06d300 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4036000 tx=0x7fbfd40046c0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.911+0000 7fbfeb59e640 1 -- 172.21.15.40:0/3558120770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec06d6f0 msgr2=0x7fbfec14cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:41.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.911+0000 7fbfeb59e640 1 --2- 172.21.15.40:0/3558120770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec06d6f0 0x7fbfec14cb30 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00ddc0 tx=0x7fbfdc00a620 comp rx=0 tx=0).stop 2023-10-18T09:09:41.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.912+0000 7fbfeb59e640 1 -- 172.21.15.40:0/3558120770 shutdown_connections 2023-10-18T09:09:41.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.912+0000 7fbfeb59e640 1 --2- 172.21.15.40:0/3558120770 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fbfcc06ae40 0x7fbfcc06d300 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.912+0000 7fbfeb59e640 1 --2- 172.21.15.40:0/3558120770 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfec0c9f00 0x7fbfec1e12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.912+0000 7fbfeb59e640 1 --2- 172.21.15.40:0/3558120770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfec0b0630 0x7fbfec1e0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.912+0000 7fbfeb59e640 1 --2- 172.21.15.40:0/3558120770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfec06d6f0 0x7fbfec14cb30 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:41.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.912+0000 7fbfeb59e640 1 -- 172.21.15.40:0/3558120770 >> 172.21.15.40:0/3558120770 conn(0x7fbfec069720 msgr2=0x7fbfec0b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:41.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.912+0000 7fbfeb59e640 1 -- 172.21.15.40:0/3558120770 shutdown_connections 2023-10-18T09:09:41.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:41.912+0000 7fbfeb59e640 1 -- 172.21.15.40:0/3558120770 wait complete. 2023-10-18T09:09:41.932 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T09:09:41.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:09:42.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.143+0000 7f03173d0640 1 -- 172.21.15.40:0/797582377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03100c9f90 msgr2=0x7f031006b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:42.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.143+0000 7f03173d0640 1 --2- 172.21.15.40:0/797582377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03100c9f90 0x7f031006b4d0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f03100ccea0 tx=0x7f030802bd90 comp rx=0 tx=0).stop 2023-10-18T09:09:42.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.144+0000 7f03173d0640 1 -- 172.21.15.40:0/797582377 shutdown_connections 2023-10-18T09:09:42.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.144+0000 7f03173d0640 1 --2- 172.21.15.40:0/797582377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03100c9f90 0x7f031006b4d0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.144+0000 7f03173d0640 1 --2- 172.21.15.40:0/797582377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03101425a0 0x7f03100c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.144+0000 7f03173d0640 1 --2- 172.21.15.40:0/797582377 >> v2:172.21.15.144:3300/0 conn(0x7f03101420f0 0x7f03100b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.144+0000 7f03173d0640 1 -- 172.21.15.40:0/797582377 >> 172.21.15.40:0/797582377 conn(0x7f0310069720 msgr2=0x7f0310069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:42.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.144+0000 7f03173d0640 1 -- 172.21.15.40:0/797582377 shutdown_connections 2023-10-18T09:09:42.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.144+0000 7f03173d0640 1 -- 172.21.15.40:0/797582377 wait complete. 2023-10-18T09:09:42.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.146+0000 7f03173d0640 1 Processor -- start 2023-10-18T09:09:42.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.146+0000 7f03173d0640 1 -- start start 2023-10-18T09:09:42.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.148+0000 7f03173d0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03100c9f90 0x7f0310159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:42.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.148+0000 7f03173d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03101420f0 0x7f031015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:42.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.148+0000 7f03173d0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03101425a0 0x7f031015a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:42.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.148+0000 7f03173d0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031015be30 con 0x7f03100c9f90 2023-10-18T09:09:42.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.148+0000 7f03173d0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031015b020 con 0x7f03101420f0 2023-10-18T09:09:42.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.148+0000 7f03173d0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031015c080 con 0x7f03101425a0 2023-10-18T09:09:42.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.148+0000 7f03163ce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03100c9f90 0x7f0310159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:42.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.148+0000 7f03163ce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03100c9f90 0x7f0310159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55696/0 (socket says 172.21.15.40:55696) 2023-10-18T09:09:42.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.148+0000 7f03163ce640 1 -- 172.21.15.40:0/1467481870 learned_addr learned my addr 172.21.15.40:0/1467481870 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:42.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.148+0000 7f03163ce640 1 -- 172.21.15.40:0/1467481870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03101425a0 msgr2=0x7f031015a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:42.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.148+0000 7f03163ce640 1 --2- 172.21.15.40:0/1467481870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03101425a0 0x7f031015a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.148+0000 7f03163ce640 1 -- 172.21.15.40:0/1467481870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03101420f0 msgr2=0x7f031015a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:42.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.148+0000 7f03163ce640 1 --2- 172.21.15.40:0/1467481870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03101420f0 0x7f031015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.148+0000 7f03163ce640 1 -- 172.21.15.40:0/1467481870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0308009560 con 0x7f03100c9f90 2023-10-18T09:09:42.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.149+0000 7f03163ce640 1 --2- 172.21.15.40:0/1467481870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03100c9f90 0x7f0310159950 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f030c00d770 tx=0x7f030c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:42.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.149+0000 7f0306ffd640 1 -- 172.21.15.40:0/1467481870 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c009c30 con 0x7f03100c9f90 2023-10-18T09:09:42.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.149+0000 7f0306ffd640 1 -- 172.21.15.40:0/1467481870 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f030c009e80 con 0x7f03100c9f90 2023-10-18T09:09:42.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.149+0000 7f03173d0640 1 -- 172.21.15.40:0/1467481870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031015c360 con 0x7f03100c9f90 2023-10-18T09:09:42.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.149+0000 7f0306ffd640 1 -- 172.21.15.40:0/1467481870 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c0106c0 con 0x7f03100c9f90 2023-10-18T09:09:42.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.149+0000 7f03173d0640 1 -- 172.21.15.40:0/1467481870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031015c920 con 0x7f03100c9f90 2023-10-18T09:09:42.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.150+0000 7f03173d0640 1 -- 172.21.15.40:0/1467481870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0310066e00 con 0x7f03100c9f90 2023-10-18T09:09:42.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.151+0000 7f0306ffd640 1 -- 172.21.15.40:0/1467481870 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f030c010820 con 0x7f03100c9f90 2023-10-18T09:09:42.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.152+0000 7f0306ffd640 1 --2- 172.21.15.40:0/1467481870 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f02f406ae40 0x7f02f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:42.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.152+0000 7f0306ffd640 1 -- 172.21.15.40:0/1467481870 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f030c089b90 con 0x7f03100c9f90 2023-10-18T09:09:42.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.152+0000 7f0306ffd640 1 -- 172.21.15.40:0/1467481870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f02f4072970 con 0x7f03100c9f90 2023-10-18T09:09:42.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.153+0000 7f0315bcd640 1 --2- 172.21.15.40:0/1467481870 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f02f406ae40 0x7f02f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:42.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.154+0000 7f0315bcd640 1 --2- 172.21.15.40:0/1467481870 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f02f406ae40 0x7f02f406d300 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f03100ac6e0 tx=0x7f0300005ed0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:42.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.156+0000 7f0306ffd640 1 -- 172.21.15.40:0/1467481870 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f030c021060 con 0x7f03100c9f90 2023-10-18T09:09:42.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.387+0000 7f03173d0640 1 -- 172.21.15.40:0/1467481870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03101e4340 con 0x7f03100c9f90 2023-10-18T09:09:42.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.388+0000 7f0306ffd640 1 -- 172.21.15.40:0/1467481870 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f030c021070 con 0x7f03100c9f90 2023-10-18T09:09:42.391 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:42.392 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2038256442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2038256442},{"type":"v1","addr":"172.21.15.144:6836","nonce":2038256442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3006484507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3006484507},{"type":"v1","addr":"172.21.15.144:6837","nonce":3006484507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1542000032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1542000032},{"type":"v1","addr":"172.21.15.40:6834","nonce":1542000032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:08:57.731109+0000","modified":"2023-10-18T09:09:00.823289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3976883051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3976883051},{"type":"v1","addr":"172.21.15.40:6835","nonce":3976883051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-18T09:09:42.394 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 90 2023-10-18T09:09:42.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.391+0000 7f03173d0640 1 -- 172.21.15.40:0/1467481870 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f02f406ae40 msgr2=0x7f02f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:42.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.391+0000 7f03173d0640 1 --2- 172.21.15.40:0/1467481870 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f02f406ae40 0x7f02f406d300 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f03100ac6e0 tx=0x7f0300005ed0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.391+0000 7f03173d0640 1 -- 172.21.15.40:0/1467481870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03100c9f90 msgr2=0x7f0310159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:42.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.391+0000 7f03173d0640 1 --2- 172.21.15.40:0/1467481870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03100c9f90 0x7f0310159950 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f030c00d770 tx=0x7f030c00dc30 comp rx=0 tx=0).stop 2023-10-18T09:09:42.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.391+0000 7f03173d0640 1 -- 172.21.15.40:0/1467481870 shutdown_connections 2023-10-18T09:09:42.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.391+0000 7f03173d0640 1 --2- 172.21.15.40:0/1467481870 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f02f406ae40 0x7f02f406d300 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.391+0000 7f03173d0640 1 --2- 172.21.15.40:0/1467481870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f03101425a0 0x7f031015a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.391+0000 7f03173d0640 1 --2- 172.21.15.40:0/1467481870 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03101420f0 0x7f031015a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.391+0000 7f03173d0640 1 --2- 172.21.15.40:0/1467481870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03100c9f90 0x7f0310159950 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.391+0000 7f03173d0640 1 -- 172.21.15.40:0/1467481870 >> 172.21.15.40:0/1467481870 conn(0x7f0310069720 msgr2=0x7f03100ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:42.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.391+0000 7f03173d0640 1 -- 172.21.15.40:0/1467481870 shutdown_connections 2023-10-18T09:09:42.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.391+0000 7f03173d0640 1 -- 172.21.15.40:0/1467481870 wait complete. 2023-10-18T09:09:42.411 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:09:42.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.633+0000 7fef7c6f2640 1 -- 172.21.15.40:0/1826787679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef60003970 msgr2=0x7fef60003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:42.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.633+0000 7fef7c6f2640 1 --2- 172.21.15.40:0/1826787679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef60003970 0x7fef60003d70 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fef7000cc00 tx=0x7fef600057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.633+0000 7fef7c6f2640 1 -- 172.21.15.40:0/1826787679 shutdown_connections 2023-10-18T09:09:42.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.634+0000 7fef7c6f2640 1 --2- 172.21.15.40:0/1826787679 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef60003970 0x7fef60003d70 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.634+0000 7fef7c6f2640 1 -- 172.21.15.40:0/1826787679 >> 172.21.15.40:0/1826787679 conn(0x7fef7406cee0 msgr2=0x7fef74069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:42.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.634+0000 7fef7c6f2640 1 -- 172.21.15.40:0/1826787679 shutdown_connections 2023-10-18T09:09:42.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.634+0000 7fef7c6f2640 1 -- 172.21.15.40:0/1826787679 wait complete. 2023-10-18T09:09:42.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.635+0000 7fef7c6f2640 1 Processor -- start 2023-10-18T09:09:42.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.636+0000 7fef7c6f2640 1 -- start start 2023-10-18T09:09:42.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.636+0000 7fef7c6f2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef60003970 0x7fef7414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:42.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.636+0000 7fef7c6f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef74148070 0x7fef7413caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:42.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.636+0000 7fef7c6f2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef7414c8e0 0x7fef7414ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:42.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.637+0000 7fef7c6f2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7414de40 con 0x7fef74148070 2023-10-18T09:09:42.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.637+0000 7fef7c6f2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7413d030 con 0x7fef7414c8e0 2023-10-18T09:09:42.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.637+0000 7fef7c6f2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7414d230 con 0x7fef60003970 2023-10-18T09:09:42.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.637+0000 7fef79c8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef74148070 0x7fef7413caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:42.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.637+0000 7fef79c8d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef74148070 0x7fef7413caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55710/0 (socket says 172.21.15.40:55710) 2023-10-18T09:09:42.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.637+0000 7fef79c8d640 1 -- 172.21.15.40:0/2177299543 learned_addr learned my addr 172.21.15.40:0/2177299543 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:42.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.637+0000 7fef7ac8f640 1 --2- 172.21.15.40:0/2177299543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef7414c8e0 0x7fef7414ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:42.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.637+0000 7fef7a48e640 1 --2- 172.21.15.40:0/2177299543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef60003970 0x7fef7414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:42.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.637+0000 7fef79c8d640 1 -- 172.21.15.40:0/2177299543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef60003970 msgr2=0x7fef7414e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:42.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.637+0000 7fef79c8d640 1 --2- 172.21.15.40:0/2177299543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef60003970 0x7fef7414e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.637+0000 7fef79c8d640 1 -- 172.21.15.40:0/2177299543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef7414c8e0 msgr2=0x7fef7414ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:42.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.637+0000 7fef79c8d640 1 --2- 172.21.15.40:0/2177299543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef7414c8e0 0x7fef7414ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.637+0000 7fef79c8d640 1 -- 172.21.15.40:0/2177299543 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef74158c30 con 0x7fef74148070 2023-10-18T09:09:42.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.638+0000 7fef79c8d640 1 --2- 172.21.15.40:0/2177299543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef74148070 0x7fef7413caf0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fef6400c3f0 tx=0x7fef6400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:42.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.638+0000 7fef6affd640 1 -- 172.21.15.40:0/2177299543 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64011790 con 0x7fef74148070 2023-10-18T09:09:42.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.638+0000 7fef6affd640 1 -- 172.21.15.40:0/2177299543 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fef640119e0 con 0x7fef74148070 2023-10-18T09:09:42.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.638+0000 7fef6affd640 1 -- 172.21.15.40:0/2177299543 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64011df0 con 0x7fef74148070 2023-10-18T09:09:42.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.638+0000 7fef7c6f2640 1 -- 172.21.15.40:0/2177299543 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7414d510 con 0x7fef74148070 2023-10-18T09:09:42.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.638+0000 7fef7c6f2640 1 -- 172.21.15.40:0/2177299543 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7414dad0 con 0x7fef74148070 2023-10-18T09:09:42.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.639+0000 7fef7c6f2640 1 -- 172.21.15.40:0/2177299543 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7414dff0 con 0x7fef74148070 2023-10-18T09:09:42.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.640+0000 7fef6affd640 1 -- 172.21.15.40:0/2177299543 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fef6401c460 con 0x7fef74148070 2023-10-18T09:09:42.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.641+0000 7fef6affd640 1 --2- 172.21.15.40:0/2177299543 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fef5806ae70 0x7fef5806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:42.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.641+0000 7fef6affd640 1 -- 172.21.15.40:0/2177299543 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef64089670 con 0x7fef74148070 2023-10-18T09:09:42.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.641+0000 7fef6affd640 1 -- 172.21.15.40:0/2177299543 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7fef580729a0 con 0x7fef74148070 2023-10-18T09:09:42.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.642+0000 7fef7a48e640 1 --2- 172.21.15.40:0/2177299543 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fef5806ae70 0x7fef5806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:42.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.643+0000 7fef7a48e640 1 --2- 172.21.15.40:0/2177299543 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fef5806ae70 0x7fef5806d330 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fef70038000 tx=0x7fef7002f040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:42.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.646+0000 7fef6affd640 1 -- 172.21.15.40:0/2177299543 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef7414dff0 con 0x7fef74148070 2023-10-18T09:09:42.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.874+0000 7fef7c6f2640 1 -- 172.21.15.40:0/2177299543 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef7414dff0 con 0x7fef74148070 2023-10-18T09:09:42.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.876+0000 7fef6affd640 1 -- 172.21.15.40:0/2177299543 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4923 (secure 0 0 0) 0x7fef7414dff0 con 0x7fef74148070 2023-10-18T09:09:42.879 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:42.879 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":90,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2038256442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2038256442},{"type":"v1","addr":"172.21.15.144:6836","nonce":2038256442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3006484507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3006484507},{"type":"v1","addr":"172.21.15.144:6837","nonce":3006484507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1542000032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1542000032},{"type":"v1","addr":"172.21.15.40:6834","nonce":1542000032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:08:57.731109+0000","modified":"2023-10-18T09:09:00.823289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9282":{"gid":9282,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/3976883051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":3976883051},{"type":"v1","addr":"172.21.15.40:6835","nonce":3976883051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-18T09:09:42.880 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 90 2023-10-18T09:09:42.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.878+0000 7fef7c6f2640 1 -- 172.21.15.40:0/2177299543 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fef5806ae70 msgr2=0x7fef5806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:42.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.878+0000 7fef7c6f2640 1 --2- 172.21.15.40:0/2177299543 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fef5806ae70 0x7fef5806d330 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fef70038000 tx=0x7fef7002f040 comp rx=0 tx=0).stop 2023-10-18T09:09:42.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.878+0000 7fef7c6f2640 1 -- 172.21.15.40:0/2177299543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef74148070 msgr2=0x7fef7413caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:42.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.878+0000 7fef7c6f2640 1 --2- 172.21.15.40:0/2177299543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef74148070 0x7fef7413caf0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fef6400c3f0 tx=0x7fef6400c7f0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.879+0000 7fef7c6f2640 1 -- 172.21.15.40:0/2177299543 shutdown_connections 2023-10-18T09:09:42.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.879+0000 7fef7c6f2640 1 --2- 172.21.15.40:0/2177299543 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fef5806ae70 0x7fef5806d330 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.879+0000 7fef7c6f2640 1 --2- 172.21.15.40:0/2177299543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef7414c8e0 0x7fef7414ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.879+0000 7fef7c6f2640 1 --2- 172.21.15.40:0/2177299543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef74148070 0x7fef7413caf0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.879+0000 7fef7c6f2640 1 --2- 172.21.15.40:0/2177299543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef60003970 0x7fef7414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:42.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.879+0000 7fef7c6f2640 1 -- 172.21.15.40:0/2177299543 >> 172.21.15.40:0/2177299543 conn(0x7fef7406cee0 msgr2=0x7fef7415a710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:42.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.879+0000 7fef7c6f2640 1 -- 172.21.15.40:0/2177299543 shutdown_connections 2023-10-18T09:09:42.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:42.879+0000 7fef7c6f2640 1 -- 172.21.15.40:0/2177299543 wait complete. 2023-10-18T09:09:42.899 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:09:43.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.115+0000 7f8f5ee10640 1 -- 172.21.15.40:0/3920461673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f5815cf30 msgr2=0x7f8f5815d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:43.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.115+0000 7f8f5ee10640 1 --2- 172.21.15.40:0/3920461673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f5815cf30 0x7f8f5815d390 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f8f5806c830 tx=0x7f8f4802bdb0 comp rx=0 tx=0).stop 2023-10-18T09:09:43.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.115+0000 7f8f5ee10640 1 -- 172.21.15.40:0/3920461673 shutdown_connections 2023-10-18T09:09:43.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.115+0000 7f8f5ee10640 1 --2- 172.21.15.40:0/3920461673 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f5815cf30 0x7f8f5815d390 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.115+0000 7f8f5ee10640 1 --2- 172.21.15.40:0/3920461673 >> v2:172.21.15.144:3300/0 conn(0x7f8f5813c470 0x7f8f5813f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.115+0000 7f8f5ee10640 1 -- 172.21.15.40:0/3920461673 >> 172.21.15.40:0/3920461673 conn(0x7f8f5806cee0 msgr2=0x7f8f58069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:43.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.115+0000 7f8f5ee10640 1 -- 172.21.15.40:0/3920461673 shutdown_connections 2023-10-18T09:09:43.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.115+0000 7f8f5ee10640 1 -- 172.21.15.40:0/3920461673 wait complete. 2023-10-18T09:09:43.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.117+0000 7f8f5ee10640 1 Processor -- start 2023-10-18T09:09:43.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.117+0000 7f8f5ee10640 1 -- start start 2023-10-18T09:09:43.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.118+0000 7f8f5ee10640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f580c9b00 0x7f8f5813b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:43.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.118+0000 7f8f5ee10640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f5813c470 0x7f8f5813af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:43.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.118+0000 7f8f5ee10640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f5815cf30 0x7f8f5813c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:43.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.118+0000 7f8f5ee10640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5813cd90 con 0x7f8f5815cf30 2023-10-18T09:09:43.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.118+0000 7f8f5ee10640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5813d010 con 0x7f8f5813c470 2023-10-18T09:09:43.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.118+0000 7f8f5ee10640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5813d2f0 con 0x7f8f580c9b00 2023-10-18T09:09:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.118+0000 7f8f5d3ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f5815cf30 0x7f8f5813c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.118+0000 7f8f5d3ad640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f5815cf30 0x7f8f5813c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55730/0 (socket says 172.21.15.40:55730) 2023-10-18T09:09:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.118+0000 7f8f5d3ad640 1 -- 172.21.15.40:0/376089828 learned_addr learned my addr 172.21.15.40:0/376089828 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.118+0000 7f8f57fff640 1 --2- 172.21.15.40:0/376089828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f5813c470 0x7f8f5813af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:43.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.118+0000 7f8f5cbac640 1 --2- 172.21.15.40:0/376089828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f580c9b00 0x7f8f5813b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.119+0000 7f8f5d3ad640 1 -- 172.21.15.40:0/376089828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f580c9b00 msgr2=0x7f8f5813b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.119+0000 7f8f5d3ad640 1 --2- 172.21.15.40:0/376089828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f580c9b00 0x7f8f5813b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.119+0000 7f8f5d3ad640 1 -- 172.21.15.40:0/376089828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f5813c470 msgr2=0x7f8f5813af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.119+0000 7f8f5d3ad640 1 --2- 172.21.15.40:0/376089828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f5813c470 0x7f8f5813af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.119+0000 7f8f5d3ad640 1 -- 172.21.15.40:0/376089828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f48009560 con 0x7f8f5815cf30 2023-10-18T09:09:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.119+0000 7f8f5cbac640 1 --2- 172.21.15.40:0/376089828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f580c9b00 0x7f8f5813b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.119+0000 7f8f5d3ad640 1 --2- 172.21.15.40:0/376089828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f5815cf30 0x7f8f5813c850 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000c780 tx=0x7f8f5000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.119+0000 7f8f557fa640 1 -- 172.21.15.40:0/376089828 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f50004b60 con 0x7f8f5815cf30 2023-10-18T09:09:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.119+0000 7f8f557fa640 1 -- 172.21.15.40:0/376089828 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8f5000ce90 con 0x7f8f5815cf30 2023-10-18T09:09:43.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.119+0000 7f8f557fa640 1 -- 172.21.15.40:0/376089828 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5000aab0 con 0x7f8f5815cf30 2023-10-18T09:09:43.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.119+0000 7f8f5ee10640 1 -- 172.21.15.40:0/376089828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f580ca4a0 con 0x7f8f5815cf30 2023-10-18T09:09:43.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.119+0000 7f8f5ee10640 1 -- 172.21.15.40:0/376089828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5813d8b0 con 0x7f8f5815cf30 2023-10-18T09:09:43.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.120+0000 7f8f5ee10640 1 -- 172.21.15.40:0/376089828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f58066fd0 con 0x7f8f5815cf30 2023-10-18T09:09:43.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.122+0000 7f8f557fa640 1 -- 172.21.15.40:0/376089828 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8f5000ac10 con 0x7f8f5815cf30 2023-10-18T09:09:43.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.123+0000 7f8f557fa640 1 --2- 172.21.15.40:0/376089828 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8f4406ae40 0x7f8f4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:43.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.124+0000 7f8f557fa640 1 -- 172.21.15.40:0/376089828 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f50089ac0 con 0x7f8f5815cf30 2023-10-18T09:09:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.124+0000 7f8f557fa640 1 -- 172.21.15.40:0/376089828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f8f44072970 con 0x7f8f5815cf30 2023-10-18T09:09:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.124+0000 7f8f5cbac640 1 --2- 172.21.15.40:0/376089828 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8f4406ae40 0x7f8f4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.125+0000 7f8f5cbac640 1 --2- 172.21.15.40:0/376089828 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8f4406ae40 0x7f8f4406d300 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8f581f4bc0 tx=0x7f8f4c005e70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:43.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.128+0000 7f8f557fa640 1 -- 172.21.15.40:0/376089828 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f50054b60 con 0x7f8f5815cf30 2023-10-18T09:09:43.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.314+0000 7f8f5ee10640 1 -- 172.21.15.40:0/376089828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f58069930 con 0x7f8f5815cf30 2023-10-18T09:09:43.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.316+0000 7f8f557fa640 1 -- 172.21.15.40:0/376089828 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+16610 (secure 0 0 0) 0x7f8f50058180 con 0x7f8f5815cf30 2023-10-18T09:09:43.318 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:43.319 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":200,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:09:36.223929+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:08:55.675479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-18T09:08:56.761888+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:09:43.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.318+0000 7f8f5ee10640 1 -- 172.21.15.40:0/376089828 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8f4406ae40 msgr2=0x7f8f4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:43.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.318+0000 7f8f5ee10640 1 --2- 172.21.15.40:0/376089828 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8f4406ae40 0x7f8f4406d300 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8f581f4bc0 tx=0x7f8f4c005e70 comp rx=0 tx=0).stop 2023-10-18T09:09:43.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.318+0000 7f8f5ee10640 1 -- 172.21.15.40:0/376089828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f5815cf30 msgr2=0x7f8f5813c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:43.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.318+0000 7f8f5ee10640 1 --2- 172.21.15.40:0/376089828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f5815cf30 0x7f8f5813c850 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000c780 tx=0x7f8f5000cc40 comp rx=0 tx=0).stop 2023-10-18T09:09:43.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.318+0000 7f8f5ee10640 1 -- 172.21.15.40:0/376089828 shutdown_connections 2023-10-18T09:09:43.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.318+0000 7f8f5ee10640 1 --2- 172.21.15.40:0/376089828 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f8f4406ae40 0x7f8f4406d300 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:43.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.318+0000 7f8f5ee10640 1 --2- 172.21.15.40:0/376089828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f5815cf30 0x7f8f5813c850 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:43.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.318+0000 7f8f5ee10640 1 --2- 172.21.15.40:0/376089828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f5813c470 0x7f8f5813af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:43.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.318+0000 7f8f5ee10640 1 --2- 172.21.15.40:0/376089828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f580c9b00 0x7f8f5813b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:43.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.318+0000 7f8f5ee10640 1 -- 172.21.15.40:0/376089828 >> 172.21.15.40:0/376089828 conn(0x7f8f5806cee0 msgr2=0x7f8f58153520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:43.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.318+0000 7f8f5ee10640 1 -- 172.21.15.40:0/376089828 shutdown_connections 2023-10-18T09:09:43.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.318+0000 7f8f5ee10640 1 -- 172.21.15.40:0/376089828 wait complete. 2023-10-18T09:09:43.340 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T09:09:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.564+0000 7f2cef0b4640 1 -- 172.21.15.40:0/253377722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ce80cd0a0 msgr2=0x7f2ce80cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.564+0000 7f2cef0b4640 1 --2- 172.21.15.40:0/253377722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ce80cd0a0 0x7f2ce80cd4a0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0005fd0 tx=0x7f2ce002c170 comp rx=0 tx=0).stop 2023-10-18T09:09:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.564+0000 7f2cef0b4640 1 -- 172.21.15.40:0/253377722 shutdown_connections 2023-10-18T09:09:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.564+0000 7f2cef0b4640 1 --2- 172.21.15.40:0/253377722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ce80cd0a0 0x7f2ce80cd4a0 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.564+0000 7f2cef0b4640 1 -- 172.21.15.40:0/253377722 >> 172.21.15.40:0/253377722 conn(0x7f2ce806cee0 msgr2=0x7f2ce8069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.564+0000 7f2cef0b4640 1 -- 172.21.15.40:0/253377722 shutdown_connections 2023-10-18T09:09:43.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.564+0000 7f2cef0b4640 1 -- 172.21.15.40:0/253377722 wait complete. 2023-10-18T09:09:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.566+0000 7f2cef0b4640 1 Processor -- start 2023-10-18T09:09:43.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.566+0000 7f2cef0b4640 1 -- start start 2023-10-18T09:09:43.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.567+0000 7f2cef0b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ce806d710 0x7f2ce81475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:43.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.567+0000 7f2cef0b4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ce80cd0a0 0x7f2ce8155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:43.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.567+0000 7f2cef0b4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ce81586a0 0x7f2ce80aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:43.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.567+0000 7f2cef0b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8156cd0 con 0x7f2ce806d710 2023-10-18T09:09:43.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.567+0000 7f2cef0b4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce80ac2c0 con 0x7f2ce80cd0a0 2023-10-18T09:09:43.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.567+0000 7f2cef0b4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8156710 con 0x7f2ce81586a0 2023-10-18T09:09:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.568+0000 7f2cece50640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ce806d710 0x7f2ce81475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.568+0000 7f2cece50640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ce806d710 0x7f2ce81475e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55756/0 (socket says 172.21.15.40:55756) 2023-10-18T09:09:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.568+0000 7f2cece50640 1 -- 172.21.15.40:0/3153377154 learned_addr learned my addr 172.21.15.40:0/3153377154 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.568+0000 7f2ce7fff640 1 --2- 172.21.15.40:0/3153377154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ce80cd0a0 0x7f2ce8155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:43.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.568+0000 7f2ced651640 1 --2- 172.21.15.40:0/3153377154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ce81586a0 0x7f2ce80aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.568+0000 7f2cece50640 1 -- 172.21.15.40:0/3153377154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ce81586a0 msgr2=0x7f2ce80aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.568+0000 7f2cece50640 1 --2- 172.21.15.40:0/3153377154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ce81586a0 0x7f2ce80aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.568+0000 7f2cece50640 1 -- 172.21.15.40:0/3153377154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ce80cd0a0 msgr2=0x7f2ce8155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.568+0000 7f2cece50640 1 --2- 172.21.15.40:0/3153377154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ce80cd0a0 0x7f2ce8155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:43.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.568+0000 7f2cece50640 1 -- 172.21.15.40:0/3153377154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce0009840 con 0x7f2ce806d710 2023-10-18T09:09:43.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.569+0000 7f2cece50640 1 --2- 172.21.15.40:0/3153377154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ce806d710 0x7f2ce81475e0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800c450 tx=0x7f2cd800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:43.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.569+0000 7f2ce57fa640 1 -- 172.21.15.40:0/3153377154 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd80052d0 con 0x7f2ce806d710 2023-10-18T09:09:43.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.569+0000 7f2ce57fa640 1 -- 172.21.15.40:0/3153377154 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2cd8005430 con 0x7f2ce806d710 2023-10-18T09:09:43.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.569+0000 7f2ce57fa640 1 -- 172.21.15.40:0/3153377154 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8012890 con 0x7f2ce806d710 2023-10-18T09:09:43.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.569+0000 7f2cef0b4640 1 -- 172.21.15.40:0/3153377154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce81569f0 con 0x7f2ce806d710 2023-10-18T09:09:43.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.569+0000 7f2cef0b4640 1 -- 172.21.15.40:0/3153377154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce814ff70 con 0x7f2ce806d710 2023-10-18T09:09:43.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.570+0000 7f2cef0b4640 1 -- 172.21.15.40:0/3153377154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce80a8840 con 0x7f2ce806d710 2023-10-18T09:09:43.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.571+0000 7f2ce57fa640 1 -- 172.21.15.40:0/3153377154 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2cd80035f0 con 0x7f2ce806d710 2023-10-18T09:09:43.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.572+0000 7f2ce57fa640 1 --2- 172.21.15.40:0/3153377154 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2cc806ae70 0x7f2cc806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:43.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.572+0000 7f2ce57fa640 1 -- 172.21.15.40:0/3153377154 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cd8089300 con 0x7f2ce806d710 2023-10-18T09:09:43.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.573+0000 7f2ce57fa640 1 -- 172.21.15.40:0/3153377154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f2cc80729a0 con 0x7f2ce806d710 2023-10-18T09:09:43.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.573+0000 7f2ce7fff640 1 --2- 172.21.15.40:0/3153377154 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2cc806ae70 0x7f2cc806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:43.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.574+0000 7f2ce7fff640 1 --2- 172.21.15.40:0/3153377154 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2cc806ae70 0x7f2cc806d330 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80abbb0 tx=0x7f2cd0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:43.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.577+0000 7f2ce57fa640 1 -- 172.21.15.40:0/3153377154 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2cd805ce60 con 0x7f2ce806d710 2023-10-18T09:09:43.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.802+0000 7f2cef0b4640 1 -- 172.21.15.40:0/3153377154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2ce8156520 con 0x7f2ce806d710 2023-10-18T09:09:43.805 INFO:tasks.ceph.mon.a.smithi040.stderr:2023-10-18T09:09:43.803+0000 7f6a0f710640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T09:09:43.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.807+0000 7f2ce57fa640 1 -- 172.21.15.40:0/3153377154 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2ce80a8840 con 0x7f2ce806d710 2023-10-18T09:09:43.810 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T09:09:43.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.808+0000 7f2ce57fa640 1 -- 172.21.15.40:0/3153377154 <== mon.0 v2:172.21.15.40:3300/0 8 ==== osd_map(201..201 src has 1..201) v4 ==== 374+0+0 (secure 0 0 0) 0x7f2cd8057fd0 con 0x7f2ce806d710 2023-10-18T09:09:43.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.809+0000 7f2ce57fa640 1 -- 172.21.15.40:0/3153377154 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=202}) v3 -- 0x7f2cc80733a0 con 0x7f2ce806d710 2023-10-18T09:09:43.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.812+0000 7f2cef0b4640 1 -- 172.21.15.40:0/3153377154 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2cc806ae70 msgr2=0x7f2cc806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:43.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.812+0000 7f2cef0b4640 1 --2- 172.21.15.40:0/3153377154 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2cc806ae70 0x7f2cc806d330 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80abbb0 tx=0x7f2cd0009040 comp rx=0 tx=0).stop 2023-10-18T09:09:43.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.812+0000 7f2cef0b4640 1 -- 172.21.15.40:0/3153377154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ce806d710 msgr2=0x7f2ce81475e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:43.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.812+0000 7f2cef0b4640 1 --2- 172.21.15.40:0/3153377154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ce806d710 0x7f2ce81475e0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800c450 tx=0x7f2cd800c910 comp rx=0 tx=0).stop 2023-10-18T09:09:43.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.813+0000 7f2cef0b4640 1 -- 172.21.15.40:0/3153377154 shutdown_connections 2023-10-18T09:09:43.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.813+0000 7f2cef0b4640 1 --2- 172.21.15.40:0/3153377154 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2cc806ae70 0x7f2cc806d330 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:43.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.813+0000 7f2cef0b4640 1 --2- 172.21.15.40:0/3153377154 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ce81586a0 0x7f2ce80aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:43.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.813+0000 7f2cef0b4640 1 --2- 172.21.15.40:0/3153377154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ce80cd0a0 0x7f2ce8155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:43.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.813+0000 7f2cef0b4640 1 --2- 172.21.15.40:0/3153377154 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ce806d710 0x7f2ce81475e0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:43.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.813+0000 7f2cef0b4640 1 -- 172.21.15.40:0/3153377154 >> 172.21.15.40:0/3153377154 conn(0x7f2ce806cee0 msgr2=0x7f2ce80acbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:43.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.814+0000 7f2cef0b4640 1 -- 172.21.15.40:0/3153377154 shutdown_connections 2023-10-18T09:09:43.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:43.814+0000 7f2cef0b4640 1 -- 172.21.15.40:0/3153377154 wait complete. 2023-10-18T09:09:43.840 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T09:09:43.846 INFO:tasks.ceph.mds.c.smithi040.stderr: -9124> 2023-10-18T09:09:02.342+0000 7f3ed6808640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T09:09:43.916 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-18T09:09:44.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.038+0000 7f2eea8a2640 1 -- 172.21.15.40:0/2808766044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ee413c550 msgr2=0x7f2ee413ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:44.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.038+0000 7f2eea8a2640 1 --2- 172.21.15.40:0/2808766044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ee413c550 0x7f2ee413ec50 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4070340 tx=0x7f2ed402bda0 comp rx=0 tx=0).stop 2023-10-18T09:09:44.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.038+0000 7f2eea8a2640 1 -- 172.21.15.40:0/2808766044 shutdown_connections 2023-10-18T09:09:44.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.038+0000 7f2eea8a2640 1 --2- 172.21.15.40:0/2808766044 >> v2:172.21.15.144:3300/0 conn(0x7f2ee4071410 0x7f2ee40cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:44.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.038+0000 7f2eea8a2640 1 --2- 172.21.15.40:0/2808766044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ee413c550 0x7f2ee413ec50 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:44.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.038+0000 7f2eea8a2640 1 -- 172.21.15.40:0/2808766044 >> 172.21.15.40:0/2808766044 conn(0x7f2ee4069720 msgr2=0x7f2ee4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:44.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.038+0000 7f2eea8a2640 1 -- 172.21.15.40:0/2808766044 shutdown_connections 2023-10-18T09:09:44.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.038+0000 7f2eea8a2640 1 -- 172.21.15.40:0/2808766044 wait complete. 2023-10-18T09:09:44.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.040+0000 7f2eea8a2640 1 Processor -- start 2023-10-18T09:09:44.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.040+0000 7f2eea8a2640 1 -- start start 2023-10-18T09:09:44.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.041+0000 7f2eea8a2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ee4071410 0x7f2ee4154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:44.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.041+0000 7f2eea8a2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ee413c550 0x7f2ee40abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:44.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.041+0000 7f2eea8a2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ee413f190 0x7f2ee413af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:44.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.041+0000 7f2eea8a2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40b0ff0 con 0x7f2ee413c550 2023-10-18T09:09:44.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.041+0000 7f2eea8a2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40b03e0 con 0x7f2ee413f190 2023-10-18T09:09:44.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.041+0000 7f2eea8a2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40b06c0 con 0x7f2ee4071410 2023-10-18T09:09:44.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.041+0000 7f2edbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ee413c550 0x7f2ee40abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:44.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.041+0000 7f2edbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ee413c550 0x7f2ee40abef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55798/0 (socket says 172.21.15.40:55798) 2023-10-18T09:09:44.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.041+0000 7f2edbfff640 1 -- 172.21.15.40:0/672181807 learned_addr learned my addr 172.21.15.40:0/672181807 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:44.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.042+0000 7f2edbfff640 1 -- 172.21.15.40:0/672181807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ee4071410 msgr2=0x7f2ee4154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:44.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.042+0000 7f2edbfff640 1 --2- 172.21.15.40:0/672181807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ee4071410 0x7f2ee4154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:44.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.042+0000 7f2ee8e3f640 1 --2- 172.21.15.40:0/672181807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ee413f190 0x7f2ee413af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:44.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.042+0000 7f2edbfff640 1 -- 172.21.15.40:0/672181807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ee413f190 msgr2=0x7f2ee413af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:44.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.042+0000 7f2edbfff640 1 --2- 172.21.15.40:0/672181807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ee413f190 0x7f2ee413af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:44.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.042+0000 7f2edbfff640 1 -- 172.21.15.40:0/672181807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed4009560 con 0x7f2ee413c550 2023-10-18T09:09:44.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.042+0000 7f2edbfff640 1 --2- 172.21.15.40:0/672181807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ee413c550 0x7f2ee40abef0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00b3e0 tx=0x7f2ecc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:44.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.042+0000 7f2ee17fa640 1 -- 172.21.15.40:0/672181807 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc011830 con 0x7f2ee413c550 2023-10-18T09:09:44.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.042+0000 7f2ee17fa640 1 -- 172.21.15.40:0/672181807 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2ecc011aa0 con 0x7f2ee413c550 2023-10-18T09:09:44.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.042+0000 7f2ee17fa640 1 -- 172.21.15.40:0/672181807 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc010380 con 0x7f2ee413c550 2023-10-18T09:09:44.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.042+0000 7f2eea8a2640 1 -- 172.21.15.40:0/672181807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee40b09a0 con 0x7f2ee413c550 2023-10-18T09:09:44.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.043+0000 7f2eea8a2640 1 -- 172.21.15.40:0/672181807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee40b0070 con 0x7f2ee413c550 2023-10-18T09:09:44.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.044+0000 7f2eea8a2640 1 -- 172.21.15.40:0/672181807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee4066fd0 con 0x7f2ee413c550 2023-10-18T09:09:44.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.045+0000 7f2ee17fa640 1 -- 172.21.15.40:0/672181807 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2ecc0104e0 con 0x7f2ee413c550 2023-10-18T09:09:44.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.045+0000 7f2ee17fa640 1 --2- 172.21.15.40:0/672181807 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ec406ae70 0x7f2ec406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:44.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.046+0000 7f2ee17fa640 1 -- 172.21.15.40:0/672181807 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2ecc089b30 con 0x7f2ee413c550 2023-10-18T09:09:44.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.046+0000 7f2ee17fa640 1 -- 172.21.15.40:0/672181807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=202}) v3 -- 0x7f2ec4072ab0 con 0x7f2ee413c550 2023-10-18T09:09:44.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.046+0000 7f2ee3fff640 1 --2- 172.21.15.40:0/672181807 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ec406ae70 0x7f2ec406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:44.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.047+0000 7f2ee3fff640 1 --2- 172.21.15.40:0/672181807 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ec406ae70 0x7f2ec406d330 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4035000 tx=0x7f2ed40046c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:44.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.050+0000 7f2ee17fa640 1 -- 172.21.15.40:0/672181807 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ecc015020 con 0x7f2ee413c550 2023-10-18T09:09:44.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.280+0000 7f2eea8a2640 1 -- 172.21.15.40:0/672181807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2ee40ada00 con 0x7f2ee413c550 2023-10-18T09:09:44.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.811+0000 7f2ee17fa640 1 -- 172.21.15.40:0/672181807 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2ecc054a50 con 0x7f2ee413c550 2023-10-18T09:09:44.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.814+0000 7f2eea8a2640 1 -- 172.21.15.40:0/672181807 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ec406ae70 msgr2=0x7f2ec406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:44.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.814+0000 7f2eea8a2640 1 --2- 172.21.15.40:0/672181807 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ec406ae70 0x7f2ec406d330 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4035000 tx=0x7f2ed40046c0 comp rx=0 tx=0).stop 2023-10-18T09:09:44.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.815+0000 7f2eea8a2640 1 -- 172.21.15.40:0/672181807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ee413c550 msgr2=0x7f2ee40abef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:44.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.815+0000 7f2eea8a2640 1 --2- 172.21.15.40:0/672181807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ee413c550 0x7f2ee40abef0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00b3e0 tx=0x7f2ecc00b8a0 comp rx=0 tx=0).stop 2023-10-18T09:09:44.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.815+0000 7f2eea8a2640 1 -- 172.21.15.40:0/672181807 shutdown_connections 2023-10-18T09:09:44.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.815+0000 7f2eea8a2640 1 --2- 172.21.15.40:0/672181807 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2ec406ae70 0x7f2ec406d330 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:44.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.815+0000 7f2eea8a2640 1 --2- 172.21.15.40:0/672181807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ee413f190 0x7f2ee413af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:44.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.815+0000 7f2eea8a2640 1 --2- 172.21.15.40:0/672181807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ee413c550 0x7f2ee40abef0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:44.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.815+0000 7f2eea8a2640 1 --2- 172.21.15.40:0/672181807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2ee4071410 0x7f2ee4154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:44.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.815+0000 7f2eea8a2640 1 -- 172.21.15.40:0/672181807 >> 172.21.15.40:0/672181807 conn(0x7f2ee4069720 msgr2=0x7f2ee40acbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:44.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.815+0000 7f2eea8a2640 1 -- 172.21.15.40:0/672181807 shutdown_connections 2023-10-18T09:09:44.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:44.815+0000 7f2eea8a2640 1 -- 172.21.15.40:0/672181807 wait complete. 2023-10-18T09:09:44.839 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T09:09:45.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.071+0000 7f1845154640 1 -- 172.21.15.40:0/1471490324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1840071570 msgr2=0x7f18400cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:45.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.071+0000 7f1845154640 1 --2- 172.21.15.40:0/1471490324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1840071570 0x7f18400cc100 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f184006b290 tx=0x7f183402bda0 comp rx=0 tx=0).stop 2023-10-18T09:09:45.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.071+0000 7f1845154640 1 -- 172.21.15.40:0/1471490324 shutdown_connections 2023-10-18T09:09:45.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.071+0000 7f1845154640 1 --2- 172.21.15.40:0/1471490324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1840071570 0x7f18400cc100 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:45.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.071+0000 7f1845154640 1 --2- 172.21.15.40:0/1471490324 >> v2:172.21.15.144:3300/0 conn(0x7f184015b5f0 0x7f18400ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:45.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.071+0000 7f1845154640 1 -- 172.21.15.40:0/1471490324 >> 172.21.15.40:0/1471490324 conn(0x7f18400aeb90 msgr2=0x7f18400aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:45.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.071+0000 7f1845154640 1 -- 172.21.15.40:0/1471490324 shutdown_connections 2023-10-18T09:09:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.072+0000 7f1845154640 1 -- 172.21.15.40:0/1471490324 wait complete. 2023-10-18T09:09:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.074+0000 7f1845154640 1 Processor -- start 2023-10-18T09:09:45.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.074+0000 7f1845154640 1 -- start start 2023-10-18T09:09:45.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.075+0000 7f1845154640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1840071570 0x7f1840151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.075+0000 7f1845154640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18400b1ce0 0x7f1840146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.075+0000 7f1845154640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f184015b5f0 0x7f1840151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.076+0000 7f1845154640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184014d280 con 0x7f18400b1ce0 2023-10-18T09:09:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.076+0000 7f1845154640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1840154a60 con 0x7f1840071570 2023-10-18T09:09:45.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.076+0000 7f1845154640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18400ac050 con 0x7f184015b5f0 2023-10-18T09:09:45.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.076+0000 7f183f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18400b1ce0 0x7f1840146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:45.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.076+0000 7f183f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18400b1ce0 0x7f1840146950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54076/0 (socket says 172.21.15.40:54076) 2023-10-18T09:09:45.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.076+0000 7f183f7fe640 1 -- 172.21.15.40:0/83352444 learned_addr learned my addr 172.21.15.40:0/83352444 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:45.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.076+0000 7f183ffff640 1 --2- 172.21.15.40:0/83352444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1840071570 0x7f1840151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:45.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.076+0000 7f183f7fe640 1 -- 172.21.15.40:0/83352444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f184015b5f0 msgr2=0x7f1840151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:45.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.076+0000 7f183f7fe640 1 --2- 172.21.15.40:0/83352444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f184015b5f0 0x7f1840151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:45.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.076+0000 7f183f7fe640 1 -- 172.21.15.40:0/83352444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1840071570 msgr2=0x7f1840151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:45.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.076+0000 7f183f7fe640 1 --2- 172.21.15.40:0/83352444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1840071570 0x7f1840151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:45.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.076+0000 7f183f7fe640 1 -- 172.21.15.40:0/83352444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1834009560 con 0x7f18400b1ce0 2023-10-18T09:09:45.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.076+0000 7f183f7fe640 1 --2- 172.21.15.40:0/83352444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18400b1ce0 0x7f1840146950 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f1834009850 tx=0x7f183402dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.077+0000 7f183cff9640 1 -- 172.21.15.40:0/83352444 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834005650 con 0x7f18400b1ce0 2023-10-18T09:09:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.077+0000 7f183cff9640 1 -- 172.21.15.40:0/83352444 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f18340038f0 con 0x7f18400b1ce0 2023-10-18T09:09:45.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.077+0000 7f183cff9640 1 -- 172.21.15.40:0/83352444 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834003ce0 con 0x7f18400b1ce0 2023-10-18T09:09:45.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.077+0000 7f1845154640 1 -- 172.21.15.40:0/83352444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1840154cb0 con 0x7f18400b1ce0 2023-10-18T09:09:45.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.077+0000 7f1845154640 1 -- 172.21.15.40:0/83352444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1840155270 con 0x7f18400b1ce0 2023-10-18T09:09:45.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.078+0000 7f1845154640 1 -- 172.21.15.40:0/83352444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1840066e00 con 0x7f18400b1ce0 2023-10-18T09:09:45.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.079+0000 7f183cff9640 1 -- 172.21.15.40:0/83352444 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f183403e450 con 0x7f18400b1ce0 2023-10-18T09:09:45.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.080+0000 7f183cff9640 1 --2- 172.21.15.40:0/83352444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f182c06ae40 0x7f182c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:45.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.080+0000 7f183cff9640 1 -- 172.21.15.40:0/83352444 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f18340abf60 con 0x7f18400b1ce0 2023-10-18T09:09:45.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.080+0000 7f183cff9640 1 -- 172.21.15.40:0/83352444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=202}) v3 -- 0x7f182c072a80 con 0x7f18400b1ce0 2023-10-18T09:09:45.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.081+0000 7f183ffff640 1 --2- 172.21.15.40:0/83352444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f182c06ae40 0x7f182c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:45.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.082+0000 7f183ffff640 1 --2- 172.21.15.40:0/83352444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f182c06ae40 0x7f182c06d300 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1830005fd0 tx=0x7f1830005bc0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:45.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.085+0000 7f183cff9640 1 -- 172.21.15.40:0/83352444 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1834042060 con 0x7f18400b1ce0 2023-10-18T09:09:45.274 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.271+0000 7f1845154640 1 -- 172.21.15.40:0/83352444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f184006b7a0 con 0x7f18400b1ce0 2023-10-18T09:09:46.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.998+0000 7f183cff9640 1 -- 172.21.15.40:0/83352444 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v202) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1834077160 con 0x7f18400b1ce0 2023-10-18T09:09:46.001 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-10-18T09:09:46.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.999+0000 7f183cff9640 1 -- 172.21.15.40:0/83352444 <== mon.0 v2:172.21.15.40:3300/0 8 ==== osd_map(202..202 src has 1..202) v4 ==== 296+0+0 (secure 0 0 0) 0x7f1834003e40 con 0x7f18400b1ce0 2023-10-18T09:09:46.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:45.999+0000 7f183cff9640 1 -- 172.21.15.40:0/83352444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7f182c070eb0 con 0x7f18400b1ce0 2023-10-18T09:09:46.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.005+0000 7f1845154640 1 -- 172.21.15.40:0/83352444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f182c06ae40 msgr2=0x7f182c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:46.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.005+0000 7f1845154640 1 --2- 172.21.15.40:0/83352444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f182c06ae40 0x7f182c06d300 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1830005fd0 tx=0x7f1830005bc0 comp rx=0 tx=0).stop 2023-10-18T09:09:46.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.005+0000 7f1845154640 1 -- 172.21.15.40:0/83352444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18400b1ce0 msgr2=0x7f1840146950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:46.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.005+0000 7f1845154640 1 --2- 172.21.15.40:0/83352444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18400b1ce0 0x7f1840146950 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f1834009850 tx=0x7f183402dd70 comp rx=0 tx=0).stop 2023-10-18T09:09:46.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.006+0000 7f1845154640 1 -- 172.21.15.40:0/83352444 shutdown_connections 2023-10-18T09:09:46.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.006+0000 7f1845154640 1 --2- 172.21.15.40:0/83352444 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f182c06ae40 0x7f182c06d300 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:46.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.006+0000 7f1845154640 1 --2- 172.21.15.40:0/83352444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f184015b5f0 0x7f1840151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:46.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.006+0000 7f1845154640 1 --2- 172.21.15.40:0/83352444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18400b1ce0 0x7f1840146950 secure :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f1834009850 tx=0x7f183402dd70 comp rx=0 tx=0).stop 2023-10-18T09:09:46.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.006+0000 7f1845154640 1 --2- 172.21.15.40:0/83352444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1840071570 0x7f1840151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:46.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.006+0000 7f1845154640 1 -- 172.21.15.40:0/83352444 >> 172.21.15.40:0/83352444 conn(0x7f18400aeb90 msgr2=0x7f18400cd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:46.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.008+0000 7f1845154640 1 -- 172.21.15.40:0/83352444 shutdown_connections 2023-10-18T09:09:46.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.009+0000 7f1845154640 1 -- 172.21.15.40:0/83352444 wait complete. 2023-10-18T09:09:46.034 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T09:09:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.258+0000 7f4b95b8e640 1 -- 172.21.15.40:0/3786858048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b7c0039a0 msgr2=0x7f4b7c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.258+0000 7f4b95b8e640 1 --2- 172.21.15.40:0/3786858048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b7c0039a0 0x7f4b7c003dc0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000bb50 tx=0x7f4b7c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.258+0000 7f4b8ffff640 1 -- 172.21.15.40:0/3786858048 reap_dead start 2023-10-18T09:09:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.258+0000 7f4b95b8e640 1 -- 172.21.15.40:0/3786858048 shutdown_connections 2023-10-18T09:09:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.258+0000 7f4b95b8e640 1 -- 172.21.15.40:0/3786858048 >> 172.21.15.40:0/3786858048 conn(0x7f4b900aec00 msgr2=0x7f4b900aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.258+0000 7f4b95b8e640 1 -- 172.21.15.40:0/3786858048 shutdown_connections 2023-10-18T09:09:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.258+0000 7f4b95b8e640 1 -- 172.21.15.40:0/3786858048 wait complete. 2023-10-18T09:09:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.260+0000 7f4b95b8e640 1 Processor -- start 2023-10-18T09:09:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.260+0000 7f4b95b8e640 1 -- start start 2023-10-18T09:09:46.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.261+0000 7f4b95b8e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b7c0039a0 0x7f4b901570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:46.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.261+0000 7f4b95b8e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b900ad7d0 0x7f4b901e0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:46.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.261+0000 7f4b95b8e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b90156960 0x7f4b900c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.261+0000 7f4b95b8e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900adc50 con 0x7f4b900ad7d0 2023-10-18T09:09:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.261+0000 7f4b95b8e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b901e0690 con 0x7f4b90156960 2023-10-18T09:09:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.261+0000 7f4b95b8e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b901df390 con 0x7f4b7c0039a0 2023-10-18T09:09:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.261+0000 7f4b87fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b900ad7d0 0x7f4b901e0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.262+0000 7f4b87fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b900ad7d0 0x7f4b901e0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54100/0 (socket says 172.21.15.40:54100) 2023-10-18T09:09:46.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.262+0000 7f4b87fff640 1 -- 172.21.15.40:0/3003536635 learned_addr learned my addr 172.21.15.40:0/3003536635 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:46.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.262+0000 7f4b8ffff640 1 --2- 172.21.15.40:0/3003536635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b90156960 0x7f4b900c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:46.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.262+0000 7f4b8f7fe640 1 --2- 172.21.15.40:0/3003536635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b7c0039a0 0x7f4b901570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:46.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.262+0000 7f4b87fff640 1 -- 172.21.15.40:0/3003536635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b7c0039a0 msgr2=0x7f4b901570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:46.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.262+0000 7f4b87fff640 1 --2- 172.21.15.40:0/3003536635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b7c0039a0 0x7f4b901570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.262+0000 7f4b87fff640 1 -- 172.21.15.40:0/3003536635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b90156960 msgr2=0x7f4b900c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.262+0000 7f4b87fff640 1 --2- 172.21.15.40:0/3003536635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b90156960 0x7f4b900c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.262+0000 7f4b87fff640 1 -- 172.21.15.40:0/3003536635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b901df670 con 0x7f4b900ad7d0 2023-10-18T09:09:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.262+0000 7f4b8ffff640 1 --2- 172.21.15.40:0/3003536635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b90156960 0x7f4b900c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.262+0000 7f4b87fff640 1 --2- 172.21.15.40:0/3003536635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b900ad7d0 0x7f4b901e0150 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800c430 tx=0x7f4b7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.263+0000 7f4b8cff9640 1 -- 172.21.15.40:0/3003536635 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b780052d0 con 0x7f4b900ad7d0 2023-10-18T09:09:46.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.263+0000 7f4b8cff9640 1 -- 172.21.15.40:0/3003536635 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4b780035f0 con 0x7f4b900ad7d0 2023-10-18T09:09:46.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.263+0000 7f4b95b8e640 1 -- 172.21.15.40:0/3003536635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b901df950 con 0x7f4b900ad7d0 2023-10-18T09:09:46.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.263+0000 7f4b8cff9640 1 -- 172.21.15.40:0/3003536635 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78003a00 con 0x7f4b900ad7d0 2023-10-18T09:09:46.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.263+0000 7f4b95b8e640 1 -- 172.21.15.40:0/3003536635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b90157900 con 0x7f4b900ad7d0 2023-10-18T09:09:46.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.264+0000 7f4b95b8e640 1 -- 172.21.15.40:0/3003536635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b90066fd0 con 0x7f4b900ad7d0 2023-10-18T09:09:46.270 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.265+0000 7f4b8cff9640 1 -- 172.21.15.40:0/3003536635 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4b78003b60 con 0x7f4b900ad7d0 2023-10-18T09:09:46.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.266+0000 7f4b8cff9640 1 --2- 172.21.15.40:0/3003536635 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b7006ae70 0x7f4b7006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:46.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.266+0000 7f4b8cff9640 1 -- 172.21.15.40:0/3003536635 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f4b78088c40 con 0x7f4b900ad7d0 2023-10-18T09:09:46.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.266+0000 7f4b8cff9640 1 -- 172.21.15.40:0/3003536635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7f4b70071c40 con 0x7f4b900ad7d0 2023-10-18T09:09:46.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.266+0000 7f4b8f7fe640 1 --2- 172.21.15.40:0/3003536635 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b7006ae70 0x7f4b7006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:46.271 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.268+0000 7f4b8f7fe640 1 --2- 172.21.15.40:0/3003536635 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b7006ae70 0x7f4b7006d330 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4b80037000 tx=0x7f4b8002e040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:46.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.271+0000 7f4b8cff9640 1 -- 172.21.15.40:0/3003536635 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4b78017060 con 0x7f4b900ad7d0 2023-10-18T09:09:46.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:46.474+0000 7f4b95b8e640 1 -- 172.21.15.40:0/3003536635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4b900af7f0 con 0x7f4b900ad7d0 2023-10-18T09:09:47.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.234+0000 7f4b8cff9640 1 -- 172.21.15.40:0/3003536635 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v203) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4b78053ee0 con 0x7f4b900ad7d0 2023-10-18T09:09:47.237 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-10-18T09:09:47.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.235+0000 7f4b8cff9640 1 -- 172.21.15.40:0/3003536635 <== mon.0 v2:172.21.15.40:3300/0 8 ==== osd_map(203..203 src has 1..203) v4 ==== 296+0+0 (secure 0 0 0) 0x7f4b780582a0 con 0x7f4b900ad7d0 2023-10-18T09:09:47.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.235+0000 7f4b8cff9640 1 -- 172.21.15.40:0/3003536635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=204}) v3 -- 0x7f4b700709d0 con 0x7f4b900ad7d0 2023-10-18T09:09:47.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.240+0000 7f4b95b8e640 1 -- 172.21.15.40:0/3003536635 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b7006ae70 msgr2=0x7f4b7006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:47.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.240+0000 7f4b95b8e640 1 --2- 172.21.15.40:0/3003536635 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b7006ae70 0x7f4b7006d330 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f4b80037000 tx=0x7f4b8002e040 comp rx=0 tx=0).stop 2023-10-18T09:09:47.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.240+0000 7f4b95b8e640 1 -- 172.21.15.40:0/3003536635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b900ad7d0 msgr2=0x7f4b901e0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:47.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.240+0000 7f4b95b8e640 1 --2- 172.21.15.40:0/3003536635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b900ad7d0 0x7f4b901e0150 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800c430 tx=0x7f4b7800c8f0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.245+0000 7f4b95b8e640 1 -- 172.21.15.40:0/3003536635 shutdown_connections 2023-10-18T09:09:47.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.245+0000 7f4b95b8e640 1 --2- 172.21.15.40:0/3003536635 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4b7006ae70 0x7f4b7006d330 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.245+0000 7f4b95b8e640 1 --2- 172.21.15.40:0/3003536635 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b90156960 0x7f4b900c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.245+0000 7f4b95b8e640 1 --2- 172.21.15.40:0/3003536635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b900ad7d0 0x7f4b901e0150 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.245+0000 7f4b95b8e640 1 --2- 172.21.15.40:0/3003536635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4b7c0039a0 0x7f4b901570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.245+0000 7f4b95b8e640 1 -- 172.21.15.40:0/3003536635 >> 172.21.15.40:0/3003536635 conn(0x7f4b900aec00 msgr2=0x7f4b900c0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:47.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.251+0000 7f4b95b8e640 1 -- 172.21.15.40:0/3003536635 shutdown_connections 2023-10-18T09:09:47.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.251+0000 7f4b95b8e640 1 -- 172.21.15.40:0/3003536635 wait complete. 2023-10-18T09:09:47.277 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 22 2023-10-18T09:09:47.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.485+0000 7ff9fe275640 1 -- 172.21.15.40:0/47318815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f815c640 msgr2=0x7ff9f8069c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:47.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.485+0000 7ff9fe275640 1 --2- 172.21.15.40:0/47318815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f815c640 0x7ff9f8069c60 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0005fd0 tx=0x7ff9f002c170 comp rx=0 tx=0).stop 2023-10-18T09:09:47.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.485+0000 7ff9fe275640 1 -- 172.21.15.40:0/47318815 shutdown_connections 2023-10-18T09:09:47.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.485+0000 7ff9fe275640 1 --2- 172.21.15.40:0/47318815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f815c640 0x7ff9f8069c60 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.485+0000 7ff9fe275640 1 --2- 172.21.15.40:0/47318815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f806edf0 0x7ff9f8069720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.485+0000 7ff9fe275640 1 -- 172.21.15.40:0/47318815 >> 172.21.15.40:0/47318815 conn(0x7ff9f80afdf0 msgr2=0x7ff9f80b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:47.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.485+0000 7ff9fe275640 1 -- 172.21.15.40:0/47318815 shutdown_connections 2023-10-18T09:09:47.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.485+0000 7ff9fe275640 1 -- 172.21.15.40:0/47318815 wait complete. 2023-10-18T09:09:47.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.487+0000 7ff9fe275640 1 Processor -- start 2023-10-18T09:09:47.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.487+0000 7ff9fe275640 1 -- start start 2023-10-18T09:09:47.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.488+0000 7ff9fe275640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9f815b3d0 0x7ff9f80c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:47.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.488+0000 7ff9fe275640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f815c640 0x7ff9f814c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:47.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.488+0000 7ff9fe275640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f81f2f20 0x7ff9f8156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:47.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.488+0000 7ff9fe275640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f81f34e0 con 0x7ff9f815c640 2023-10-18T09:09:47.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.488+0000 7ff9fe275640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f81f37c0 con 0x7ff9f815b3d0 2023-10-18T09:09:47.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.488+0000 7ff9fe275640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f81f3aa0 con 0x7ff9f81f2f20 2023-10-18T09:09:47.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.489+0000 7ff9fca72640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f815c640 0x7ff9f814c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:47.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.489+0000 7ff9fca72640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f815c640 0x7ff9f814c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54120/0 (socket says 172.21.15.40:54120) 2023-10-18T09:09:47.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.489+0000 7ff9fca72640 1 -- 172.21.15.40:0/1744616318 learned_addr learned my addr 172.21.15.40:0/1744616318 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:47.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.489+0000 7ff9fda74640 1 --2- 172.21.15.40:0/1744616318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f81f2f20 0x7ff9f8156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:47.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.489+0000 7ff9fd273640 1 --2- 172.21.15.40:0/1744616318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9f815b3d0 0x7ff9f80c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:47.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.489+0000 7ff9fca72640 1 -- 172.21.15.40:0/1744616318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f81f2f20 msgr2=0x7ff9f8156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:47.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.489+0000 7ff9fca72640 1 --2- 172.21.15.40:0/1744616318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f81f2f20 0x7ff9f8156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.489+0000 7ff9fca72640 1 -- 172.21.15.40:0/1744616318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9f815b3d0 msgr2=0x7ff9f80c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:47.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.489+0000 7ff9fca72640 1 --2- 172.21.15.40:0/1744616318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9f815b3d0 0x7ff9f80c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.489+0000 7ff9fca72640 1 -- 172.21.15.40:0/1744616318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f0009840 con 0x7ff9f815c640 2023-10-18T09:09:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.490+0000 7ff9fca72640 1 --2- 172.21.15.40:0/1744616318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f815c640 0x7ff9f814c5c0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800c920 tx=0x7ff9e800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.490+0000 7ff9edffb640 1 -- 172.21.15.40:0/1744616318 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8007c70 con 0x7ff9f815c640 2023-10-18T09:09:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.490+0000 7ff9edffb640 1 -- 172.21.15.40:0/1744616318 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff9e800f450 con 0x7ff9f815c640 2023-10-18T09:09:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.490+0000 7ff9fe275640 1 -- 172.21.15.40:0/1744616318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f81f3d80 con 0x7ff9f815c640 2023-10-18T09:09:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.490+0000 7ff9edffb640 1 -- 172.21.15.40:0/1744616318 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e800f860 con 0x7ff9f815c640 2023-10-18T09:09:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.490+0000 7ff9fe275640 1 -- 172.21.15.40:0/1744616318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f80c4070 con 0x7ff9f815c640 2023-10-18T09:09:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.492+0000 7ff9fe275640 1 -- 172.21.15.40:0/1744616318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f81f42a0 con 0x7ff9f815c640 2023-10-18T09:09:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.492+0000 7ff9edffb640 1 -- 172.21.15.40:0/1744616318 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff9e800f9c0 con 0x7ff9f815c640 2023-10-18T09:09:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.493+0000 7ff9edffb640 1 --2- 172.21.15.40:0/1744616318 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9dc06ae40 0x7ff9dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.494+0000 7ff9edffb640 1 -- 172.21.15.40:0/1744616318 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff9e8089ba0 con 0x7ff9f815c640 2023-10-18T09:09:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.494+0000 7ff9fd273640 1 --2- 172.21.15.40:0/1744616318 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9dc06ae40 0x7ff9dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.495+0000 7ff9fd273640 1 --2- 172.21.15.40:0/1744616318 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9dc06ae40 0x7ff9dc06d300 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40046b0 tx=0x7ff9f4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.498+0000 7ff9edffb640 1 -- 172.21.15.40:0/1744616318 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff9f81f42a0 con 0x7ff9f815c640 2023-10-18T09:09:47.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.690+0000 7ff9fe275640 1 -- 172.21.15.40:0/1744616318 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "config reset", "num": 22} v 0) v1 -- 0x7ff9f81f42a0 con 0x7ff9f815c640 2023-10-18T09:09:47.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.696+0000 7ff9edffb640 1 -- 172.21.15.40:0/1744616318 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 22}]=0 v42) v1 ==== 71+0+0 (secure 0 0 0) 0x7ff9f81f42a0 con 0x7ff9f815c640 2023-10-18T09:09:47.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.698+0000 7ff9fe275640 1 -- 172.21.15.40:0/1744616318 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9dc06ae40 msgr2=0x7ff9dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:47.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.699+0000 7ff9fe275640 1 --2- 172.21.15.40:0/1744616318 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9dc06ae40 0x7ff9dc06d300 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40046b0 tx=0x7ff9f4009040 comp rx=0 tx=0).stop 2023-10-18T09:09:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.699+0000 7ff9fe275640 1 -- 172.21.15.40:0/1744616318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f815c640 msgr2=0x7ff9f814c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.699+0000 7ff9fe275640 1 --2- 172.21.15.40:0/1744616318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f815c640 0x7ff9f814c5c0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800c920 tx=0x7ff9e800cde0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.699+0000 7ff9fe275640 1 -- 172.21.15.40:0/1744616318 shutdown_connections 2023-10-18T09:09:47.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.699+0000 7ff9fe275640 1 --2- 172.21.15.40:0/1744616318 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff9dc06ae40 0x7ff9dc06d300 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.699+0000 7ff9fe275640 1 --2- 172.21.15.40:0/1744616318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff9f81f2f20 0x7ff9f8156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.699+0000 7ff9fe275640 1 --2- 172.21.15.40:0/1744616318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9f815c640 0x7ff9f814c5c0 secure :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800c920 tx=0x7ff9e800cde0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.699+0000 7ff9fe275640 1 --2- 172.21.15.40:0/1744616318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9f815b3d0 0x7ff9f80c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.699+0000 7ff9fe275640 1 -- 172.21.15.40:0/1744616318 >> 172.21.15.40:0/1744616318 conn(0x7ff9f80afdf0 msgr2=0x7ff9f81554e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:47.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.699+0000 7ff9fe275640 1 -- 172.21.15.40:0/1744616318 shutdown_connections 2023-10-18T09:09:47.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.699+0000 7ff9fe275640 1 -- 172.21.15.40:0/1744616318 wait complete. 2023-10-18T09:09:47.722 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2023-10-18T09:09:47.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.952+0000 7f64f818a640 1 -- 172.21.15.40:0/2717159067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64dc005180 msgr2=0x7f64dc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:47.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.952+0000 7f64f818a640 1 --2- 172.21.15.40:0/2717159067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64dc005180 0x7f64dc00b980 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f64e80043b0 tx=0x7f64dc0057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.952+0000 7f64f818a640 1 -- 172.21.15.40:0/2717159067 shutdown_connections 2023-10-18T09:09:47.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.952+0000 7f64f818a640 1 --2- 172.21.15.40:0/2717159067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64dc005180 0x7f64dc00b980 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.952+0000 7f64f818a640 1 -- 172.21.15.40:0/2717159067 >> 172.21.15.40:0/2717159067 conn(0x7f64f00aec70 msgr2=0x7f64f00af080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:47.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.953+0000 7f64f818a640 1 -- 172.21.15.40:0/2717159067 shutdown_connections 2023-10-18T09:09:47.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.953+0000 7f64f818a640 1 -- 172.21.15.40:0/2717159067 wait complete. 2023-10-18T09:09:47.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.954+0000 7f64f818a640 1 Processor -- start 2023-10-18T09:09:47.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.955+0000 7f64f818a640 1 -- start start 2023-10-18T09:09:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.955+0000 7f64f818a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64dc005180 0x7f64f013a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:47.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.956+0000 7f64f818a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64f01e6a20 0x7f64f015c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:47.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.956+0000 7f64f818a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64f015cc60 0x7f64f0152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:47.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.956+0000 7f64f818a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f013bba0 con 0x7f64f015cc60 2023-10-18T09:09:47.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.956+0000 7f64f818a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f013b300 con 0x7f64dc005180 2023-10-18T09:09:47.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.956+0000 7f64f818a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f013b5e0 con 0x7f64f01e6a20 2023-10-18T09:09:47.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.956+0000 7f64f6727640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64f015cc60 0x7f64f0152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:47.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.956+0000 7f64f6727640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64f015cc60 0x7f64f0152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54146/0 (socket says 172.21.15.40:54146) 2023-10-18T09:09:47.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.956+0000 7f64f6727640 1 -- 172.21.15.40:0/2595077255 learned_addr learned my addr 172.21.15.40:0/2595077255 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:47.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.956+0000 7f64f5725640 1 --2- 172.21.15.40:0/2595077255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64f01e6a20 0x7f64f015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.956+0000 7f64f5f26640 1 --2- 172.21.15.40:0/2595077255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64dc005180 0x7f64f013a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.956+0000 7f64f6727640 1 -- 172.21.15.40:0/2595077255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64f01e6a20 msgr2=0x7f64f015c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.956+0000 7f64f6727640 1 --2- 172.21.15.40:0/2595077255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64f01e6a20 0x7f64f015c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.956+0000 7f64f6727640 1 -- 172.21.15.40:0/2595077255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64dc005180 msgr2=0x7f64f013a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:47.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.956+0000 7f64f6727640 1 --2- 172.21.15.40:0/2595077255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64dc005180 0x7f64f013a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:47.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.956+0000 7f64f6727640 1 -- 172.21.15.40:0/2595077255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f013b8c0 con 0x7f64f015cc60 2023-10-18T09:09:47.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.957+0000 7f64f6727640 1 --2- 172.21.15.40:0/2595077255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64f015cc60 0x7f64f0152b20 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f64e8004300 tx=0x7f64e802cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:47.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.957+0000 7f64e67fc640 1 -- 172.21.15.40:0/2595077255 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e8006480 con 0x7f64f015cc60 2023-10-18T09:09:47.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.957+0000 7f64e67fc640 1 -- 172.21.15.40:0/2595077255 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f64e8004e90 con 0x7f64f015cc60 2023-10-18T09:09:47.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.957+0000 7f64e67fc640 1 -- 172.21.15.40:0/2595077255 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e8040470 con 0x7f64f015cc60 2023-10-18T09:09:47.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.957+0000 7f64f818a640 1 -- 172.21.15.40:0/2595077255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f013a590 con 0x7f64f015cc60 2023-10-18T09:09:47.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.957+0000 7f64f818a640 1 -- 172.21.15.40:0/2595077255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f0153240 con 0x7f64f015cc60 2023-10-18T09:09:47.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.958+0000 7f64f818a640 1 -- 172.21.15.40:0/2595077255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f0066f10 con 0x7f64f015cc60 2023-10-18T09:09:47.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.959+0000 7f64e67fc640 1 -- 172.21.15.40:0/2595077255 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f64e803d070 con 0x7f64f015cc60 2023-10-18T09:09:47.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.960+0000 7f64e67fc640 1 --2- 172.21.15.40:0/2595077255 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f64d406ae40 0x7f64d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:47.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.961+0000 7f64e67fc640 1 -- 172.21.15.40:0/2595077255 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f64e80aa8e0 con 0x7f64f015cc60 2023-10-18T09:09:47.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.961+0000 7f64f5f26640 1 --2- 172.21.15.40:0/2595077255 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f64d406ae40 0x7f64d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:47.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.962+0000 7f64f5f26640 1 --2- 172.21.15.40:0/2595077255 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f64d406ae40 0x7f64d406d300 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009aa0 tx=0x7f64ec005f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:47.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:47.965+0000 7f64e67fc640 1 -- 172.21.15.40:0/2595077255 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64e8075c80 con 0x7f64f015cc60 2023-10-18T09:09:48.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:48.236+0000 7f64f818a640 1 -- 172.21.15.40:0/2595077255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7f64f00cccb0 con 0x7f64f015cc60 2023-10-18T09:09:49.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:49.014+0000 7f64e67fc640 1 -- 172.21.15.40:0/2595077255 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v705) v1 ==== 135+0+0 (secure 0 0 0) 0x7f64e80792a0 con 0x7f64f015cc60 2023-10-18T09:09:49.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:49.018+0000 7f64f818a640 1 -- 172.21.15.40:0/2595077255 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f64d406ae40 msgr2=0x7f64d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:49.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:49.018+0000 7f64f818a640 1 --2- 172.21.15.40:0/2595077255 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f64d406ae40 0x7f64d406d300 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009aa0 tx=0x7f64ec005f90 comp rx=0 tx=0).stop 2023-10-18T09:09:49.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:49.018+0000 7f64f818a640 1 -- 172.21.15.40:0/2595077255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64f015cc60 msgr2=0x7f64f0152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:49.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:49.018+0000 7f64f818a640 1 --2- 172.21.15.40:0/2595077255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64f015cc60 0x7f64f0152b20 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f64e8004300 tx=0x7f64e802cf80 comp rx=0 tx=0).stop 2023-10-18T09:09:49.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:49.018+0000 7f64f818a640 1 -- 172.21.15.40:0/2595077255 shutdown_connections 2023-10-18T09:09:49.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:49.018+0000 7f64f818a640 1 --2- 172.21.15.40:0/2595077255 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f64d406ae40 0x7f64d406d300 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:49.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:49.018+0000 7f64f818a640 1 --2- 172.21.15.40:0/2595077255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64f015cc60 0x7f64f0152b20 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:49.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:49.018+0000 7f64f818a640 1 --2- 172.21.15.40:0/2595077255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64f01e6a20 0x7f64f015c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:49.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:49.018+0000 7f64f818a640 1 --2- 172.21.15.40:0/2595077255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64dc005180 0x7f64f013a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:49.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:49.018+0000 7f64f818a640 1 -- 172.21.15.40:0/2595077255 >> 172.21.15.40:0/2595077255 conn(0x7f64f00aec70 msgr2=0x7f64f0150020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:49.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:49.019+0000 7f64f818a640 1 -- 172.21.15.40:0/2595077255 shutdown_connections 2023-10-18T09:09:49.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:49.019+0000 7f64f818a640 1 -- 172.21.15.40:0/2595077255 wait complete. 2023-10-18T09:09:49.040 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-10-18T09:09:49.041 INFO:tasks.cephfs_test_runner: 2023-10-18T09:09:49.041 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-18T09:09:49.041 INFO:tasks.cephfs_test_runner:Ran 10 tests in 829.293s 2023-10-18T09:09:49.041 INFO:tasks.cephfs_test_runner: 2023-10-18T09:09:49.042 INFO:tasks.cephfs_test_runner:OK 2023-10-18T09:09:49.042 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-18T09:09:49.051 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-18T09:09:49.132 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-18T09:09:49.132 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:09:49.133 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:09:49.195 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:09:49.195 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-18T09:09:49.196 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:09:49.255 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-18T09:09:49.255 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:09:49.256 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:09:49.317 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:09:49.317 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2023-10-18T09:09:49.317 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:09:49.371 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-18T09:09:49.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:49.372 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-18T09:09:49.435 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-18T09:09:49.435 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-18T09:09:49.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:49.437 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-18T09:09:49.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:49.542 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-18T09:09:49.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:49.637 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2023-10-18T09:09:49.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:49.785 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-18T09:09:49.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:49.833 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2023-10-18T09:09:49.976 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2023-10-18T09:09:49.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:09:49.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:49.980 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-18T09:09:50.042 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-18T09:09:50.042 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-18T09:09:50.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:50.043 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-18T09:09:50.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:50.143 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-18T09:09:50.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:50.240 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2023-10-18T09:09:50.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:50.373 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2023-10-18T09:09:50.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:09:50.421 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2023-10-18T09:09:50.501 INFO:teuthology.orchestra.run.smithi144.stderr:Cannot find device "ceph-brx" 2023-10-18T09:09:50.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:09:50.505 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-18T09:09:50.515 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-18T09:09:50.515 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:09:50.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.758+0000 7fb4997b8640 1 -- 172.21.15.40:0/1316799540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4940b0630 msgr2=0x7fb4940c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:50.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.758+0000 7fb4997b8640 1 --2- 172.21.15.40:0/1316799540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4940b0630 0x7fb4940c99c0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fb47c005fd0 tx=0x7fb47c02bd80 comp rx=0 tx=0).stop 2023-10-18T09:09:50.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.758+0000 7fb4997b8640 1 -- 172.21.15.40:0/1316799540 shutdown_connections 2023-10-18T09:09:50.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.758+0000 7fb4997b8640 1 --2- 172.21.15.40:0/1316799540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4940b0630 0x7fb4940c99c0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:50.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.758+0000 7fb4997b8640 1 --2- 172.21.15.40:0/1316799540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb49406d6f0 0x7fb4940b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:50.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.758+0000 7fb4997b8640 1 -- 172.21.15.40:0/1316799540 >> 172.21.15.40:0/1316799540 conn(0x7fb494069720 msgr2=0x7fb494069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:50.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.758+0000 7fb4997b8640 1 -- 172.21.15.40:0/1316799540 shutdown_connections 2023-10-18T09:09:50.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.758+0000 7fb4997b8640 1 -- 172.21.15.40:0/1316799540 wait complete. 2023-10-18T09:09:50.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.760+0000 7fb4997b8640 1 Processor -- start 2023-10-18T09:09:50.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.760+0000 7fb4997b8640 1 -- start start 2023-10-18T09:09:50.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.761+0000 7fb4997b8640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb49406d6f0 0x7fb49413e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:50.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.761+0000 7fb4997b8640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4940b0630 0x7fb4940cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:50.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.761+0000 7fb4997b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4940c9f00 0x7fb4941533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:50.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.761+0000 7fb4997b8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4940a66f0 con 0x7fb4940c9f00 2023-10-18T09:09:50.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.761+0000 7fb4997b8640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49415a3e0 con 0x7fb49406d6f0 2023-10-18T09:09:50.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.761+0000 7fb4997b8640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4940a9310 con 0x7fb4940b0630 2023-10-18T09:09:50.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.761+0000 7fb498fb7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4940c9f00 0x7fb4941533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:50.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.761+0000 7fb498fb7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4940c9f00 0x7fb4941533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54164/0 (socket says 172.21.15.40:54164) 2023-10-18T09:09:50.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.761+0000 7fb498fb7640 1 -- 172.21.15.40:0/904424729 learned_addr learned my addr 172.21.15.40:0/904424729 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.762+0000 7fb48bfff640 1 --2- 172.21.15.40:0/904424729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4940b0630 0x7fb4940cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.762+0000 7fb493fff640 1 --2- 172.21.15.40:0/904424729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb49406d6f0 0x7fb49413e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:50.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.762+0000 7fb498fb7640 1 -- 172.21.15.40:0/904424729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4940b0630 msgr2=0x7fb4940cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.762+0000 7fb498fb7640 1 --2- 172.21.15.40:0/904424729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4940b0630 0x7fb4940cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.762+0000 7fb498fb7640 1 -- 172.21.15.40:0/904424729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb49406d6f0 msgr2=0x7fb49413e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.762+0000 7fb498fb7640 1 --2- 172.21.15.40:0/904424729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb49406d6f0 0x7fb49413e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.762+0000 7fb498fb7640 1 -- 172.21.15.40:0/904424729 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47c009560 con 0x7fb4940c9f00 2023-10-18T09:09:50.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.762+0000 7fb493fff640 1 --2- 172.21.15.40:0/904424729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb49406d6f0 0x7fb49413e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.762+0000 7fb498fb7640 1 --2- 172.21.15.40:0/904424729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4940c9f00 0x7fb4941533e0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00b990 tx=0x7fb48c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.762+0000 7fb4917fa640 1 -- 172.21.15.40:0/904424729 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c00cba0 con 0x7fb4940c9f00 2023-10-18T09:09:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.762+0000 7fb4917fa640 1 -- 172.21.15.40:0/904424729 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb48c00cdf0 con 0x7fb4940c9f00 2023-10-18T09:09:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.762+0000 7fb4917fa640 1 -- 172.21.15.40:0/904424729 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c0146a0 con 0x7fb4940c9f00 2023-10-18T09:09:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.762+0000 7fb4997b8640 1 -- 172.21.15.40:0/904424729 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4940a8a70 con 0x7fb4940c9f00 2023-10-18T09:09:50.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.763+0000 7fb4997b8640 1 -- 172.21.15.40:0/904424729 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4940a9030 con 0x7fb4940c9f00 2023-10-18T09:09:50.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.764+0000 7fb4997b8640 1 -- 172.21.15.40:0/904424729 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4940a6830 con 0x7fb4940c9f00 2023-10-18T09:09:50.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.765+0000 7fb4917fa640 1 -- 172.21.15.40:0/904424729 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb48c0148c0 con 0x7fb4940c9f00 2023-10-18T09:09:50.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.766+0000 7fb4917fa640 1 --2- 172.21.15.40:0/904424729 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb47406ae40 0x7fb47406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:50.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.766+0000 7fb4917fa640 1 -- 172.21.15.40:0/904424729 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb48c089510 con 0x7fb4940c9f00 2023-10-18T09:09:50.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.766+0000 7fb493fff640 1 --2- 172.21.15.40:0/904424729 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb47406ae40 0x7fb47406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:50.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.768+0000 7fb493fff640 1 --2- 172.21.15.40:0/904424729 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb47406ae40 0x7fb47406d300 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb484006fd0 tx=0x7fb484008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:50.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.770+0000 7fb4917fa640 1 -- 172.21.15.40:0/904424729 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb4940a6830 con 0x7fb4940c9f00 2023-10-18T09:09:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.960+0000 7fb4997b8640 1 -- 172.21.15.40:0/904424729 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4940a6470 con 0x7fb47406ae40 2023-10-18T09:09:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.966+0000 7fb4917fa640 1 -- 172.21.15.40:0/904424729 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47110 (secure 0 0 0) 0x7fb48400ae20 con 0x7fb47406ae40 2023-10-18T09:09:50.970 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:09:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.969+0000 7fb4997b8640 1 -- 172.21.15.40:0/904424729 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb47406ae40 msgr2=0x7fb47406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.969+0000 7fb4997b8640 1 --2- 172.21.15.40:0/904424729 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb47406ae40 0x7fb47406d300 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb484006fd0 tx=0x7fb484008040 comp rx=0 tx=0).stop 2023-10-18T09:09:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.969+0000 7fb4997b8640 1 -- 172.21.15.40:0/904424729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4940c9f00 msgr2=0x7fb4941533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.969+0000 7fb4997b8640 1 --2- 172.21.15.40:0/904424729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4940c9f00 0x7fb4941533e0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00b990 tx=0x7fb48c00be50 comp rx=0 tx=0).stop 2023-10-18T09:09:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.969+0000 7fb4997b8640 1 -- 172.21.15.40:0/904424729 shutdown_connections 2023-10-18T09:09:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.969+0000 7fb4997b8640 1 --2- 172.21.15.40:0/904424729 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb47406ae40 0x7fb47406d300 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:50.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.969+0000 7fb4997b8640 1 --2- 172.21.15.40:0/904424729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4940c9f00 0x7fb4941533e0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:50.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.969+0000 7fb4997b8640 1 --2- 172.21.15.40:0/904424729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4940b0630 0x7fb4940cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:50.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.969+0000 7fb4997b8640 1 --2- 172.21.15.40:0/904424729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb49406d6f0 0x7fb49413e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:50.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.969+0000 7fb4997b8640 1 -- 172.21.15.40:0/904424729 >> 172.21.15.40:0/904424729 conn(0x7fb494069720 msgr2=0x7fb494069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:50.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.969+0000 7fb4997b8640 1 -- 172.21.15.40:0/904424729 shutdown_connections 2023-10-18T09:09:50.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:50.969+0000 7fb4997b8640 1 -- 172.21.15.40:0/904424729 wait complete. 2023-10-18T09:09:50.990 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":639,"stamp":"2023-10-18T09:09:49.221396+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":5,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":26616,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1535883,"statfs":{"total":1600000000,"available":1572744314,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52490893,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-129,"num_write_kb":-97371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002164"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":361,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-18T09:09:36.239190+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:09:36.239190+0000","last_peered":"2023-10-18T09:09:36.239190+0000","last_clean":"2023-10-18T09:09:36.239190+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:09:36.239190+0000","last_undegraded":"2023-10-18T09:09:36.239190+0000","last_fullsized":"2023-10-18T09:09:36.239190+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575026,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9948,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185364,"statfs":{"total":200000000,"available":189813025,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1140000000000001,"15min":1.0840000000000001},"min":{"1min":0.69499999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6990000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.044,"5min":1.129,"15min":1.0940000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.63100000000000001},"max":{"1min":1.5589999999999999,"5min":1.724,"15min":1.97},"last":0.71799999999999997}]},{"osd":1,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1910000000000001,"15min":1.165},"min":{"1min":0.73699999999999999,"5min":0.68899999999999995,"15min":0.61799999999999999},"max":{"1min":1.4950000000000001,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.276},{"interface":"front","average":{"1min":1.121,"5min":1.1539999999999999,"15min":1.135},"min":{"1min":0.72799999999999998,"5min":0.63500000000000001,"15min":0.58799999999999997},"max":{"1min":1.571,"5min":1.782,"15min":1.7989999999999999},"last":1.2290000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.351,"15min":1.3109999999999999},"min":{"1min":0.81299999999999994,"5min":0.80700000000000005,"15min":0.65400000000000003},"max":{"1min":1.794,"5min":1.8140000000000001,"15min":1.9730000000000001},"last":1.327},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2470000000000001,"15min":1.244},"min":{"1min":0.79500000000000004,"5min":0.72499999999999998,"15min":0.68300000000000005},"max":{"1min":1.6870000000000001,"5min":1.8169999999999999,"15min":1.863},"last":1.1779999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.3480000000000001,"15min":1.294},"min":{"1min":0.97099999999999997,"5min":0.90300000000000002,"15min":0.61299999999999999},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.98},"last":1.425},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3979999999999999,"15min":1.359},"min":{"1min":0.88200000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8640000000000001,"5min":1.9219999999999999,"15min":1.9930000000000001},"last":1.3959999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2410000000000001,"15min":1.228},"min":{"1min":0.89500000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.7390000000000001,"5min":1.869,"15min":2.1070000000000002},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.3300000000000001,"15min":1.3400000000000001},"min":{"1min":0.96299999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7470000000000001,"5min":1.9570000000000001,"15min":2.5169999999999999},"last":1.5389999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3480000000000001,"15min":1.345},"min":{"1min":0.78500000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":2.0720000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.393,"15min":1.4079999999999999},"min":{"1min":0.92000000000000004,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.968,"5min":1.968,"15min":2.0270000000000001},"last":1.4670000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.484,"15min":1.482},"min":{"1min":0.83699999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.3959999999999999},"last":1.571},{"interface":"front","average":{"1min":1.528,"5min":1.474,"15min":1.4850000000000001},"min":{"1min":1.1879999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.8959999999999999,"5min":2.0680000000000001,"15min":2.1680000000000001},"last":1.597}]}]},{"osd":6,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2250000000000001,"15min":1.2050000000000001},"min":{"1min":0.80900000000000005,"5min":0.64200000000000002,"15min":0.60299999999999998},"max":{"1min":1.575,"5min":1.643,"15min":1.6850000000000001},"last":1.054},{"interface":"front","average":{"1min":1.327,"5min":1.3799999999999999,"15min":1.325},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.64700000000000002},"max":{"1min":1.796,"5min":1.899,"15min":2.0990000000000002},"last":1.4490000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.181,"15min":1.127},"min":{"1min":0.81499999999999995,"5min":0.69899999999999995,"15min":0.63800000000000001},"max":{"1min":1.605,"5min":1.605,"15min":1.722},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.173,"5min":1.2430000000000001,"15min":1.2150000000000001},"min":{"1min":0.76200000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.627,"5min":1.8169999999999999,"15min":2.9609999999999999},"last":1.609}]},{"osd":2,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.234,"15min":1.1950000000000001},"min":{"1min":0.85199999999999998,"5min":0.70299999999999996,"15min":0.69899999999999995},"max":{"1min":1.4790000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3460000000000001,"15min":1.2629999999999999},"min":{"1min":0.86399999999999999,"5min":0.748,"15min":0.66300000000000003},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":2.1779999999999999},"last":1.175}]},{"osd":3,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.3220000000000001,"15min":1.288},"min":{"1min":1.196,"5min":0.82999999999999996,"15min":0.79200000000000004},"max":{"1min":1.587,"5min":1.8160000000000001,"15min":2.1589999999999998},"last":1.391},{"interface":"front","average":{"1min":1.286,"5min":1.355,"15min":1.3009999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.78600000000000003},"max":{"1min":1.617,"5min":1.8540000000000001,"15min":1.994},"last":1.552}]},{"osd":4,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.3029999999999999,"15min":1.272},"min":{"1min":1.075,"5min":0.80600000000000005,"15min":0.58299999999999996},"max":{"1min":1.8500000000000001,"5min":1.9770000000000001,"15min":2.444},"last":1.742},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.363,"15min":1.3120000000000001},"min":{"1min":0.877,"5min":0.82999999999999996,"15min":0.55000000000000004},"max":{"1min":1.766,"5min":1.9159999999999999,"15min":2.4039999999999999},"last":1.5109999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4510000000000001,"15min":1.423},"min":{"1min":1.204,"5min":0.92900000000000005,"15min":0.58899999999999997},"max":{"1min":1.631,"5min":2.016,"15min":2.032},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3680000000000001,"15min":1.3260000000000001},"min":{"1min":1.087,"5min":0.77100000000000002,"15min":0.49199999999999999},"max":{"1min":1.748,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.655}]},{"osd":7,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.4950000000000001,"15min":1.4390000000000001},"min":{"1min":1.242,"5min":1.016,"15min":0.52000000000000002},"max":{"1min":1.913,"5min":2.028,"15min":2.3540000000000001},"last":1.819},{"interface":"front","average":{"1min":1.617,"5min":1.5640000000000001,"15min":1.5389999999999999},"min":{"1min":1.304,"5min":1.1890000000000001,"15min":0.68500000000000005},"max":{"1min":1.9670000000000001,"5min":2.1440000000000001,"15min":2.794},"last":1.9330000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575028,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3200000000000001,"15min":1.3260000000000001},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.895,"5min":2.302,"15min":9.609},"last":0.876},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3819999999999999,"15min":1.331},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.60799999999999998},"max":{"1min":1.714,"5min":2.0840000000000001,"15min":5.3029999999999999},"last":0.94299999999999995}]},{"osd":1,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2370000000000001,"15min":1.256},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.792,"5min":2.0339999999999998,"15min":11.68},"last":0.998},{"interface":"front","average":{"1min":1.212,"5min":1.246,"15min":1.2130000000000001},"min":{"1min":0.74099999999999999,"5min":0.72799999999999998,"15min":0.67000000000000004},"max":{"1min":1.5009999999999999,"5min":1.79,"15min":3.7050000000000001},"last":1.1160000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.27,"15min":1.2529999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.755},"max":{"1min":1.788,"5min":1.8540000000000001,"15min":2.2090000000000001},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2569999999999999,"15min":1.264},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.72399999999999998},"max":{"1min":1.6359999999999999,"5min":1.706,"15min":3.8180000000000001},"last":1.52}]},{"osd":3,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2110000000000001,"15min":1.256},"min":{"1min":0.79800000000000004,"5min":0.745,"15min":0.65400000000000003},"max":{"1min":1.466,"5min":1.6719999999999999,"15min":11.797000000000001},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.3740000000000001,"15min":1.365},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.747},"max":{"1min":1.6020000000000001,"5min":2.105,"15min":2.3420000000000001},"last":1.1779999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.341,"15min":1.373},"min":{"1min":1.004,"5min":0.755,"15min":0.755},"max":{"1min":1.7190000000000001,"5min":2.028,"15min":2.4449999999999998},"last":1.46},{"interface":"front","average":{"1min":1.409,"5min":1.3380000000000001,"15min":1.345},"min":{"1min":1.1319999999999999,"5min":0.67700000000000005,"15min":0.64100000000000001},"max":{"1min":1.7230000000000001,"5min":1.982,"15min":5.4580000000000002},"last":1.411}]},{"osd":6,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.246,"15min":1.2809999999999999},"min":{"1min":0.93600000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.667,"5min":1.7410000000000001,"15min":5.141},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5409999999999999,"15min":1.53},"min":{"1min":1.2589999999999999,"5min":1.095,"15min":0.71099999999999997},"max":{"1min":1.845,"5min":2.161,"15min":2.657},"last":1.3720000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.468,"15min":1.5389999999999999},"min":{"1min":1.1819999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.9319999999999999,"5min":1.976,"15min":11.829000000000001},"last":1.605},{"interface":"front","average":{"1min":1.379,"5min":1.395,"15min":1.4299999999999999},"min":{"1min":1.0509999999999999,"5min":0.96099999999999997,"15min":0.73399999999999999},"max":{"1min":1.5980000000000001,"5min":1.8560000000000001,"15min":4.8419999999999996},"last":1.341}]}]},{"osd":4,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194182,"statfs":{"total":200000000,"available":198843287,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.1699999999999999,"15min":1.1399999999999999},"min":{"1min":0.73499999999999999,"5min":0.66300000000000003,"15min":0.45900000000000002},"max":{"1min":1.669,"5min":1.7410000000000001,"15min":3.1819999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.1930000000000001,"15min":1.155},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.56899999999999995},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":3.234},"last":1.036}]},{"osd":1,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1870000000000001,"15min":1.1479999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.60799999999999998},"max":{"1min":1.5209999999999999,"5min":1.9690000000000001,"15min":3.718},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.206,"15min":1.206},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.51400000000000001},"max":{"1min":1.802,"5min":1.9259999999999999,"15min":3.4350000000000001},"last":0.753}]},{"osd":2,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.236,"15min":1.216},"min":{"1min":0.79100000000000004,"5min":0.76200000000000001,"15min":0.56699999999999995},"max":{"1min":1.518,"5min":1.905,"15min":4.1749999999999998},"last":1.117},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2769999999999999,"15min":1.294},"min":{"1min":0.96799999999999997,"5min":0.64500000000000002,"15min":0.58899999999999997},"max":{"1min":1.601,"5min":1.6919999999999999,"15min":3.532},"last":0.97699999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.323,"15min":1.2889999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.67000000000000004},"max":{"1min":1.754,"5min":1.754,"15min":3.363},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3879999999999999,"15min":1.399},"min":{"1min":0.96199999999999997,"5min":0.92200000000000004,"15min":0.68899999999999995},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":4.3179999999999996},"last":1.21}]},{"osd":5,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3,"15min":1.3049999999999999},"min":{"1min":1.161,"5min":0.55600000000000005,"15min":0.498},"max":{"1min":1.611,"5min":1.841,"15min":3.6160000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.331,"15min":1.325},"min":{"1min":0.95899999999999996,"5min":0.60099999999999998,"15min":0.38400000000000001},"max":{"1min":1.6799999999999999,"5min":1.702,"15min":4.0780000000000003},"last":1.403}]},{"osd":6,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.349,"15min":1.3939999999999999},"min":{"1min":1.1859999999999999,"5min":0.67400000000000004,"15min":0.51600000000000001},"max":{"1min":1.6299999999999999,"5min":1.766,"15min":5.0049999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.494,"5min":1.4399999999999999,"15min":1.4830000000000001},"min":{"1min":1.0860000000000001,"5min":0.47499999999999998,"15min":0.45400000000000001},"max":{"1min":1.7290000000000001,"5min":1.827,"15min":4.407},"last":1.5900000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.4430000000000001,"15min":1.452},"min":{"1min":1.0129999999999999,"5min":0.51200000000000001,"15min":0.438},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":4.5170000000000003},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.579,"5min":1.5069999999999999,"15min":1.5349999999999999},"min":{"1min":1.323,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.7949999999999999,"5min":1.948,"15min":4.5460000000000003},"last":1.3859999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575028,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9311,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186000,"statfs":{"total":200000000,"available":190464992,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.89300000000000002,"15min":0.90600000000000003},"min":{"1min":0.45800000000000002,"5min":0.42699999999999999,"15min":0.35899999999999999},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":5.5730000000000004},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.028,"15min":0.98499999999999999},"min":{"1min":0.58399999999999996,"5min":0.53200000000000003,"15min":0.32700000000000001},"max":{"1min":1.665,"5min":1.863,"15min":6.5599999999999996},"last":0.86599999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0600000000000001,"15min":1.0229999999999999},"min":{"1min":0.63100000000000001,"5min":0.52200000000000002,"15min":0.36799999999999999},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":6.8099999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.335,"15min":1.258},"min":{"1min":0.61299999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.95999999999999996}]},{"osd":2,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.1659999999999999,"15min":1.117},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":1.7150000000000001,"5min":1.9550000000000001,"15min":7.4580000000000002},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.276,"15min":1.2070000000000001},"min":{"1min":0.65100000000000002,"5min":0.60699999999999998,"15min":0.51400000000000001},"max":{"1min":2.407,"5min":2.407,"15min":5.0049999999999999},"last":0.89700000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.385,"15min":1.3999999999999999},"min":{"1min":0.87,"5min":0.79300000000000004,"15min":0.53700000000000003},"max":{"1min":1.8100000000000001,"5min":2.0859999999999999,"15min":18.664999999999999},"last":1.385},{"interface":"front","average":{"1min":1.6200000000000001,"5min":1.427,"15min":1.327},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.64000000000000001},"max":{"1min":7.1269999999999998,"5min":7.1269999999999998,"15min":7.1269999999999998},"last":1.284}]},{"osd":5,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.3919999999999999,"15min":1.343},"min":{"1min":0.86599999999999999,"5min":0.84299999999999997,"15min":0.626},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":6.9139999999999997},"last":1.218},{"interface":"front","average":{"1min":1.496,"5min":1.514,"15min":1.506},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.56899999999999995},"max":{"1min":2.5640000000000001,"5min":2.6499999999999999,"15min":16.390999999999998},"last":1.2450000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.5569999999999999,"15min":1.512},"min":{"1min":1.0820000000000001,"5min":0.998,"15min":0.58199999999999996},"max":{"1min":2.1040000000000001,"5min":2.2839999999999998,"15min":7.7270000000000003},"last":1.464},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.5469999999999999,"15min":1.492},"min":{"1min":1.069,"5min":1.0049999999999999,"15min":0.60199999999999998},"max":{"1min":1.9410000000000001,"5min":2.0089999999999999,"15min":7.242},"last":1.4339999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.6240000000000001,"15min":1.5489999999999999},"min":{"1min":1.0329999999999999,"5min":0.89600000000000002,"15min":0.70099999999999996},"max":{"1min":2.5099999999999998,"5min":2.8170000000000002,"15min":4.8630000000000004},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.756,"5min":1.6659999999999999,"15min":1.6020000000000001},"min":{"1min":0.84699999999999998,"5min":0.83499999999999996,"15min":0.622},"max":{"1min":5.7649999999999997,"5min":5.7649999999999997,"15min":12.946},"last":1.3029999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.84199999999999997,"15min":0.86499999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.33700000000000002},"max":{"1min":1.464,"5min":1.6000000000000001,"15min":4.2210000000000001},"last":1.077},{"interface":"front","average":{"1min":0.875,"5min":0.99099999999999999,"15min":1},"min":{"1min":0.50800000000000001,"5min":0.497,"15min":0.33800000000000002},"max":{"1min":1.3440000000000001,"5min":2.2799999999999998,"15min":4.3959999999999999},"last":0.65000000000000002}]},{"osd":1,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.97399999999999998,"15min":1.0449999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1699999999999999,"5min":1.6100000000000001,"15min":7.3390000000000004},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.97299999999999998,"5min":1.087,"15min":1.1020000000000001},"min":{"1min":0.61099999999999999,"5min":0.55900000000000005,"15min":0.46200000000000002},"max":{"1min":1.4139999999999999,"5min":1.696,"15min":2.5350000000000001},"last":1.2190000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.115,"15min":1.121},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.847,"5min":1.847,"15min":7.6369999999999996},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.3240000000000001,"15min":1.341},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.54100000000000004},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":7.5570000000000004},"last":1.284}]},{"osd":4,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3939999999999999,"15min":1.4179999999999999},"min":{"1min":0.99299999999999999,"5min":0.93400000000000005,"15min":0.77900000000000003},"max":{"1min":1.9239999999999999,"5min":2.016,"15min":7.6189999999999998},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.397,"15min":1.4219999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.736,"5min":2.0430000000000001,"15min":3.423},"last":1.429}]},{"osd":5,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.26,"15min":1.298},"min":{"1min":0.89000000000000001,"5min":0.82399999999999995,"15min":0.61599999999999999},"max":{"1min":1.601,"5min":2.0600000000000001,"15min":10.507},"last":1.542},{"interface":"front","average":{"1min":1.29,"5min":1.377,"15min":1.4379999999999999},"min":{"1min":0.93600000000000005,"5min":0.873,"15min":0.81399999999999995},"max":{"1min":1.788,"5min":1.984,"15min":7.657},"last":1.4850000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.419,"15min":1.45},"min":{"1min":0.84599999999999997,"5min":0.77700000000000002,"15min":0.74299999999999999},"max":{"1min":1.772,"5min":2.2519999999999998,"15min":3.5299999999999998},"last":1.639},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.4239999999999999,"15min":1.599},"min":{"1min":0.84399999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.835,"5min":2.2639999999999998,"15min":37.771999999999998},"last":1.3380000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.5069999999999999,"15min":1.53},"min":{"1min":0.872,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9330000000000001,"5min":2.3860000000000001,"15min":3.3620000000000001},"last":1.732},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.482,"15min":1.512},"min":{"1min":0.86299999999999999,"5min":0.84299999999999997,"15min":0.77600000000000002},"max":{"1min":1.96,"5min":2.375,"15min":10.956},"last":1.6930000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575027,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194182,"statfs":{"total":200000000,"available":198843287,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.91300000000000003,"15min":0.89300000000000002},"min":{"1min":0.57199999999999995,"5min":0.378,"15min":0.315},"max":{"1min":1.3740000000000001,"5min":1.4770000000000001,"15min":1.659},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1379999999999999,"15min":1.0580000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.441},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.0189999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0720000000000001,"15min":1.018},"min":{"1min":0.49199999999999999,"5min":0.48399999999999999,"15min":0.317},"max":{"1min":1.357,"5min":1.6100000000000001,"15min":1.859},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.97699999999999998,"15min":0.94499999999999995},"min":{"1min":0.55300000000000005,"5min":0.443,"15min":0.36699999999999999},"max":{"1min":1.3580000000000001,"5min":1.8009999999999999,"15min":2.0230000000000001},"last":1.6970000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.181,"15min":1.1040000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.436},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.48},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.085,"15min":1.073},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.35399999999999998},"max":{"1min":1.4370000000000001,"5min":1.6659999999999999,"15min":3.7010000000000001},"last":1.1539999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.296,"15min":1.2929999999999999},"min":{"1min":0.81499999999999995,"5min":0.75600000000000001,"15min":0.45700000000000002},"max":{"1min":1.97,"5min":1.97,"15min":2.0489999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.335,"15min":1.3129999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.59699999999999998},"max":{"1min":1.8149999999999999,"5min":1.877,"15min":2.0470000000000002},"last":1.875}]},{"osd":5,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.478,"15min":1.401},"min":{"1min":0.90400000000000003,"5min":0.86499999999999999,"15min":0.56000000000000005},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":3.5750000000000002},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.476,"15min":1.448},"min":{"1min":0.90000000000000002,"5min":0.86399999999999999,"15min":0.71699999999999997},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.3769999999999998},"last":1.542}]},{"osd":6,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4099999999999999,"15min":1.3839999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.64200000000000002},"max":{"1min":2.044,"5min":2.044,"15min":2.2050000000000001},"last":1.9890000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.4610000000000001,"15min":1.3959999999999999},"min":{"1min":0.92900000000000005,"5min":0.89000000000000001,"15min":0.63100000000000001},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.6559999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.538,"15min":1.4730000000000001},"min":{"1min":0.95399999999999996,"5min":0.85699999999999998,"15min":0.69799999999999995},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.4790000000000001,"15min":1.4590000000000001},"min":{"1min":1,"5min":0.81100000000000005,"15min":0.66300000000000003},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.4180000000000001},"last":1.6200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575027,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193606,"statfs":{"total":200000000,"available":198252919,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.52100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.1779999999999999,"5min":1.2410000000000001,"15min":3.4420000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.071,"5min":1.091,"15min":1.089},"min":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.476,"5min":1.657,"15min":1.8240000000000001},"last":0.88400000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.90200000000000002,"15min":0.85999999999999999},"min":{"1min":0.59799999999999998,"5min":0.56299999999999994,"15min":0.375},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.679},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2290000000000001,"15min":1.161},"min":{"1min":0.78400000000000003,"5min":0.623,"15min":0.437},"max":{"1min":1.607,"5min":1.917,"15min":1.929},"last":0.98899999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.167,"15min":1.1599999999999999},"min":{"1min":0.56799999999999995,"5min":0.52100000000000002,"15min":0.49399999999999999},"max":{"1min":1.7210000000000001,"5min":1.9590000000000001,"15min":2.7639999999999998},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.4159999999999999,"15min":1.413},"min":{"1min":0.85099999999999998,"5min":0.68100000000000005,"15min":0.58399999999999996},"max":{"1min":1.77,"5min":2.0150000000000001,"15min":3.5720000000000001},"last":1.196}]},{"osd":4,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2170000000000001,"15min":1.2270000000000001},"min":{"1min":0.80800000000000005,"5min":0.73099999999999998,"15min":0.71099999999999997},"max":{"1min":1.6579999999999999,"5min":1.7370000000000001,"15min":4.492},"last":1.014},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2829999999999999,"15min":1.274},"min":{"1min":0.93600000000000005,"5min":0.77500000000000002,"15min":0.68700000000000006},"max":{"1min":1.552,"5min":1.752,"15min":1.927},"last":1.2529999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.252,"15min":1.2889999999999999},"min":{"1min":0.97899999999999998,"5min":0.75600000000000001,"15min":0.64300000000000002},"max":{"1min":1.625,"5min":1.6439999999999999,"15min":4.2779999999999996},"last":1.407},{"interface":"front","average":{"1min":1.325,"5min":1.347,"15min":1.3340000000000001},"min":{"1min":1.03,"5min":0.81899999999999995,"15min":0.73099999999999998},"max":{"1min":1.583,"5min":1.8169999999999999,"15min":1.8740000000000001},"last":1.135}]},{"osd":6,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.48,"15min":1.4730000000000001},"min":{"1min":1.0620000000000001,"5min":0.81000000000000005,"15min":0.76800000000000002},"max":{"1min":2.0459999999999998,"5min":2.2469999999999999,"15min":3.5299999999999998},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.359,"15min":1.3819999999999999},"min":{"1min":0.95099999999999996,"5min":0.86899999999999999,"15min":0.66700000000000004},"max":{"1min":1.661,"5min":1.8720000000000001,"15min":4.4349999999999996},"last":1.2969999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.5489999999999999,"15min":1.5569999999999999},"min":{"1min":1.1850000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.982,"5min":2.133,"15min":3.492},"last":1.329},{"interface":"front","average":{"1min":1.399,"5min":1.472,"15min":1.4950000000000001},"min":{"1min":0.95999999999999996,"5min":0.88500000000000001,"15min":0.82099999999999995},"max":{"1min":1.776,"5min":2.1000000000000001,"15min":4.5419999999999998},"last":1.5}]}]}],"pool_statfs":[]}} 2023-10-18T09:09:50.992 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:09:51.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.213+0000 7ff3c69ed640 1 -- 172.21.15.40:0/1595838264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c00c8c40 msgr2=0x7ff3c015f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:51.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.213+0000 7ff3c69ed640 1 --2- 172.21.15.40:0/1595838264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c00c8c40 0x7ff3c015f320 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005fd0 tx=0x7ff3b802c0f0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.213+0000 7ff3c69ed640 1 -- 172.21.15.40:0/1595838264 shutdown_connections 2023-10-18T09:09:51.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.213+0000 7ff3c69ed640 1 --2- 172.21.15.40:0/1595838264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c00c8c40 0x7ff3c015f320 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.213+0000 7ff3c69ed640 1 --2- 172.21.15.40:0/1595838264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3c015b3e0 0x7ff3c0070f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.213+0000 7ff3c69ed640 1 -- 172.21.15.40:0/1595838264 >> 172.21.15.40:0/1595838264 conn(0x7ff3c00ca960 msgr2=0x7ff3c00cad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:51.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.213+0000 7ff3c69ed640 1 -- 172.21.15.40:0/1595838264 shutdown_connections 2023-10-18T09:09:51.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.213+0000 7ff3c69ed640 1 -- 172.21.15.40:0/1595838264 wait complete. 2023-10-18T09:09:51.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.215+0000 7ff3c69ed640 1 Processor -- start 2023-10-18T09:09:51.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.215+0000 7ff3c69ed640 1 -- start start 2023-10-18T09:09:51.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.216+0000 7ff3c69ed640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3c00c8c40 0x7ff3c00c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:51.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.216+0000 7ff3c69ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c015b3e0 0x7ff3c0150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:51.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.216+0000 7ff3c69ed640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3c015eec0 0x7ff3c00c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:51.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.216+0000 7ff3c69ed640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c00c68e0 con 0x7ff3c015b3e0 2023-10-18T09:09:51.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.216+0000 7ff3c69ed640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c00c6bc0 con 0x7ff3c015eec0 2023-10-18T09:09:51.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.216+0000 7ff3c69ed640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c00c6ea0 con 0x7ff3c00c8c40 2023-10-18T09:09:51.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.216+0000 7ff3c51ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c015b3e0 0x7ff3c0150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:51.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.216+0000 7ff3c51ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c015b3e0 0x7ff3c0150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54176/0 (socket says 172.21.15.40:54176) 2023-10-18T09:09:51.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.216+0000 7ff3c51ea640 1 -- 172.21.15.40:0/2854486707 learned_addr learned my addr 172.21.15.40:0/2854486707 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:51.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.217+0000 7ff3c51ea640 1 -- 172.21.15.40:0/2854486707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3c00c8c40 msgr2=0x7ff3c00c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:51.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.217+0000 7ff3c51ea640 1 --2- 172.21.15.40:0/2854486707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3c00c8c40 0x7ff3c00c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.217+0000 7ff3c51ea640 1 -- 172.21.15.40:0/2854486707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3c015eec0 msgr2=0x7ff3c00c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:51.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.217+0000 7ff3c51ea640 1 --2- 172.21.15.40:0/2854486707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3c015eec0 0x7ff3c00c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.217+0000 7ff3c51ea640 1 -- 172.21.15.40:0/2854486707 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b8009840 con 0x7ff3c015b3e0 2023-10-18T09:09:51.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.217+0000 7ff3c51ea640 1 --2- 172.21.15.40:0/2854486707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c015b3e0 0x7ff3c0150530 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000c390 tx=0x7ff3b000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:51.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.217+0000 7ff3b67fc640 1 -- 172.21.15.40:0/2854486707 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0011790 con 0x7ff3c015b3e0 2023-10-18T09:09:51.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.217+0000 7ff3b67fc640 1 -- 172.21.15.40:0/2854486707 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff3b00119e0 con 0x7ff3c015b3e0 2023-10-18T09:09:51.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.217+0000 7ff3b67fc640 1 -- 172.21.15.40:0/2854486707 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0011df0 con 0x7ff3c015b3e0 2023-10-18T09:09:51.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.217+0000 7ff3c69ed640 1 -- 172.21.15.40:0/2854486707 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c00c7180 con 0x7ff3c015b3e0 2023-10-18T09:09:51.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.217+0000 7ff3c69ed640 1 -- 172.21.15.40:0/2854486707 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c00c61f0 con 0x7ff3c015b3e0 2023-10-18T09:09:51.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.219+0000 7ff3c69ed640 1 -- 172.21.15.40:0/2854486707 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c00c76a0 con 0x7ff3c015b3e0 2023-10-18T09:09:51.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.219+0000 7ff3b67fc640 1 -- 172.21.15.40:0/2854486707 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff3b001c460 con 0x7ff3c015b3e0 2023-10-18T09:09:51.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.220+0000 7ff3b67fc640 1 --2- 172.21.15.40:0/2854486707 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff3a406ae40 0x7ff3a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:51.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.221+0000 7ff3b67fc640 1 -- 172.21.15.40:0/2854486707 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff3b0089130 con 0x7ff3c015b3e0 2023-10-18T09:09:51.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.221+0000 7ff3c59eb640 1 --2- 172.21.15.40:0/2854486707 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff3a406ae40 0x7ff3a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:51.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.222+0000 7ff3c59eb640 1 --2- 172.21.15.40:0/2854486707 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff3a406ae40 0x7ff3a406d300 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0058c0 tx=0x7ff3bc005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:51.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.225+0000 7ff3b67fc640 1 -- 172.21.15.40:0/2854486707 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff3c00c76a0 con 0x7ff3c015b3e0 2023-10-18T09:09:51.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.416+0000 7ff3c69ed640 1 -- 172.21.15.40:0/2854486707 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff3c00c6d30 con 0x7ff3a406ae40 2023-10-18T09:09:51.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.422+0000 7ff3b67fc640 1 -- 172.21.15.40:0/2854486707 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47142 (secure 0 0 0) 0x7ff3c00c6d30 con 0x7ff3a406ae40 2023-10-18T09:09:51.425 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:51.425 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:09:51.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.425+0000 7ff3c69ed640 1 -- 172.21.15.40:0/2854486707 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff3a406ae40 msgr2=0x7ff3a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:51.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.425+0000 7ff3c69ed640 1 --2- 172.21.15.40:0/2854486707 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff3a406ae40 0x7ff3a406d300 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0058c0 tx=0x7ff3bc005850 comp rx=0 tx=0).stop 2023-10-18T09:09:51.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.425+0000 7ff3c69ed640 1 -- 172.21.15.40:0/2854486707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c015b3e0 msgr2=0x7ff3c0150530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:51.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.425+0000 7ff3c69ed640 1 --2- 172.21.15.40:0/2854486707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c015b3e0 0x7ff3c0150530 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000c390 tx=0x7ff3b000c850 comp rx=0 tx=0).stop 2023-10-18T09:09:51.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.425+0000 7ff3c69ed640 1 -- 172.21.15.40:0/2854486707 shutdown_connections 2023-10-18T09:09:51.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.425+0000 7ff3c69ed640 1 --2- 172.21.15.40:0/2854486707 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7ff3a406ae40 0x7ff3a406d300 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.425+0000 7ff3c69ed640 1 --2- 172.21.15.40:0/2854486707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3c015eec0 0x7ff3c00c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.425+0000 7ff3c69ed640 1 --2- 172.21.15.40:0/2854486707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3c015b3e0 0x7ff3c0150530 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.425+0000 7ff3c69ed640 1 --2- 172.21.15.40:0/2854486707 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3c00c8c40 0x7ff3c00c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.425+0000 7ff3c69ed640 1 -- 172.21.15.40:0/2854486707 >> 172.21.15.40:0/2854486707 conn(0x7ff3c00ca960 msgr2=0x7ff3c015b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:51.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.425+0000 7ff3c69ed640 1 -- 172.21.15.40:0/2854486707 shutdown_connections 2023-10-18T09:09:51.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.425+0000 7ff3c69ed640 1 -- 172.21.15.40:0/2854486707 wait complete. 2023-10-18T09:09:51.445 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":640,"stamp":"2023-10-18T09:09:51.221925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10194,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552305,"statfs":{"total":1600000000,"available":1589560465,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52490893,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-129,"num_write_kb":-97371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.410276"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":363,"reported_epoch":202,"state":"active+clean","last_fresh":"2023-10-18T09:09:46.006422+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:09:46.006422+0000","last_peered":"2023-10-18T09:09:46.006422+0000","last_clean":"2023-10-18T09:09:46.006422+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:09:46.006422+0000","last_undegraded":"2023-10-18T09:09:46.006422+0000","last_fullsized":"2023-10-18T09:09:46.006422+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575027,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193605,"statfs":{"total":200000000,"available":198251900,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1140000000000001,"15min":1.0840000000000001},"min":{"1min":0.69499999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6990000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.044,"5min":1.129,"15min":1.0940000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.63100000000000001},"max":{"1min":1.5589999999999999,"5min":1.724,"15min":1.97},"last":1.3839999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1910000000000001,"15min":1.165},"min":{"1min":0.73699999999999999,"5min":0.68899999999999995,"15min":0.61799999999999999},"max":{"1min":1.4950000000000001,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.24},{"interface":"front","average":{"1min":1.121,"5min":1.1539999999999999,"15min":1.135},"min":{"1min":0.72799999999999998,"5min":0.63500000000000001,"15min":0.58799999999999997},"max":{"1min":1.571,"5min":1.782,"15min":1.7989999999999999},"last":1.1379999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.351,"15min":1.3109999999999999},"min":{"1min":0.81299999999999994,"5min":0.80700000000000005,"15min":0.65400000000000003},"max":{"1min":1.794,"5min":1.8140000000000001,"15min":1.9730000000000001},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2470000000000001,"15min":1.244},"min":{"1min":0.79500000000000004,"5min":0.72499999999999998,"15min":0.68300000000000005},"max":{"1min":1.6870000000000001,"5min":1.8169999999999999,"15min":1.863},"last":1.5700000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.3480000000000001,"15min":1.294},"min":{"1min":0.97099999999999997,"5min":0.90300000000000002,"15min":0.61299999999999999},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.98},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3979999999999999,"15min":1.359},"min":{"1min":0.88200000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8640000000000001,"5min":1.9219999999999999,"15min":1.9930000000000001},"last":1.2090000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2410000000000001,"15min":1.228},"min":{"1min":0.89500000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.7390000000000001,"5min":1.869,"15min":2.1070000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.3300000000000001,"15min":1.3400000000000001},"min":{"1min":0.96299999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7470000000000001,"5min":1.9570000000000001,"15min":2.5169999999999999},"last":1.3380000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3480000000000001,"15min":1.345},"min":{"1min":0.78500000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":2.0720000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.393,"15min":1.4079999999999999},"min":{"1min":0.92000000000000004,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.968,"5min":1.968,"15min":2.0270000000000001},"last":1.488}]},{"osd":6,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.484,"15min":1.482},"min":{"1min":0.83699999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.3959999999999999},"last":1.661},{"interface":"front","average":{"1min":1.528,"5min":1.474,"15min":1.4850000000000001},"min":{"1min":1.1879999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.8959999999999999,"5min":2.0680000000000001,"15min":2.1680000000000001},"last":1.5169999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2250000000000001,"15min":1.2050000000000001},"min":{"1min":0.80900000000000005,"5min":0.64200000000000002,"15min":0.60299999999999998},"max":{"1min":1.575,"5min":1.643,"15min":1.6850000000000001},"last":1.054},{"interface":"front","average":{"1min":1.327,"5min":1.3799999999999999,"15min":1.325},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.64700000000000002},"max":{"1min":1.796,"5min":1.899,"15min":2.0990000000000002},"last":1.4490000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.181,"15min":1.127},"min":{"1min":0.81499999999999995,"5min":0.69899999999999995,"15min":0.63800000000000001},"max":{"1min":1.605,"5min":1.605,"15min":1.722},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.173,"5min":1.2430000000000001,"15min":1.2150000000000001},"min":{"1min":0.76200000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.627,"5min":1.8169999999999999,"15min":2.9609999999999999},"last":1.609}]},{"osd":2,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.234,"15min":1.1950000000000001},"min":{"1min":0.85199999999999998,"5min":0.70299999999999996,"15min":0.69899999999999995},"max":{"1min":1.4790000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3460000000000001,"15min":1.2629999999999999},"min":{"1min":0.86399999999999999,"5min":0.748,"15min":0.66300000000000003},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":2.1779999999999999},"last":1.175}]},{"osd":3,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.3220000000000001,"15min":1.288},"min":{"1min":1.196,"5min":0.82999999999999996,"15min":0.79200000000000004},"max":{"1min":1.587,"5min":1.8160000000000001,"15min":2.1589999999999998},"last":1.391},{"interface":"front","average":{"1min":1.286,"5min":1.355,"15min":1.3009999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.78600000000000003},"max":{"1min":1.617,"5min":1.8540000000000001,"15min":1.994},"last":1.552}]},{"osd":4,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.3029999999999999,"15min":1.272},"min":{"1min":1.075,"5min":0.80600000000000005,"15min":0.58299999999999996},"max":{"1min":1.8500000000000001,"5min":1.9770000000000001,"15min":2.444},"last":1.742},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.363,"15min":1.3120000000000001},"min":{"1min":0.877,"5min":0.82999999999999996,"15min":0.55000000000000004},"max":{"1min":1.766,"5min":1.9159999999999999,"15min":2.4039999999999999},"last":1.5109999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4510000000000001,"15min":1.423},"min":{"1min":1.204,"5min":0.92900000000000005,"15min":0.58899999999999997},"max":{"1min":1.631,"5min":2.016,"15min":2.032},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3680000000000001,"15min":1.3260000000000001},"min":{"1min":1.087,"5min":0.77100000000000002,"15min":0.49199999999999999},"max":{"1min":1.748,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.655}]},{"osd":7,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.4950000000000001,"15min":1.4390000000000001},"min":{"1min":1.242,"5min":1.016,"15min":0.52000000000000002},"max":{"1min":1.913,"5min":2.028,"15min":2.3540000000000001},"last":1.819},{"interface":"front","average":{"1min":1.617,"5min":1.5640000000000001,"15min":1.5389999999999999},"min":{"1min":1.304,"5min":1.1890000000000001,"15min":0.68500000000000005},"max":{"1min":1.9670000000000001,"5min":2.1440000000000001,"15min":2.794},"last":1.9330000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575028,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3200000000000001,"15min":1.3260000000000001},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.895,"5min":2.302,"15min":9.609},"last":0.876},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3819999999999999,"15min":1.331},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.60799999999999998},"max":{"1min":1.714,"5min":2.0840000000000001,"15min":5.3029999999999999},"last":0.94299999999999995}]},{"osd":1,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2370000000000001,"15min":1.256},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.792,"5min":2.0339999999999998,"15min":11.68},"last":0.998},{"interface":"front","average":{"1min":1.212,"5min":1.246,"15min":1.2130000000000001},"min":{"1min":0.74099999999999999,"5min":0.72799999999999998,"15min":0.67000000000000004},"max":{"1min":1.5009999999999999,"5min":1.79,"15min":3.7050000000000001},"last":1.1160000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.27,"15min":1.2529999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.755},"max":{"1min":1.788,"5min":1.8540000000000001,"15min":2.2090000000000001},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2569999999999999,"15min":1.264},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.72399999999999998},"max":{"1min":1.6359999999999999,"5min":1.706,"15min":3.8180000000000001},"last":1.52}]},{"osd":3,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2110000000000001,"15min":1.256},"min":{"1min":0.79800000000000004,"5min":0.745,"15min":0.65400000000000003},"max":{"1min":1.466,"5min":1.6719999999999999,"15min":11.797000000000001},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.3740000000000001,"15min":1.365},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.747},"max":{"1min":1.6020000000000001,"5min":2.105,"15min":2.3420000000000001},"last":1.1779999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.341,"15min":1.373},"min":{"1min":1.004,"5min":0.755,"15min":0.755},"max":{"1min":1.7190000000000001,"5min":2.028,"15min":2.4449999999999998},"last":1.46},{"interface":"front","average":{"1min":1.409,"5min":1.3380000000000001,"15min":1.345},"min":{"1min":1.1319999999999999,"5min":0.67700000000000005,"15min":0.64100000000000001},"max":{"1min":1.7230000000000001,"5min":1.982,"15min":5.4580000000000002},"last":1.411}]},{"osd":6,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.246,"15min":1.2809999999999999},"min":{"1min":0.93600000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.667,"5min":1.7410000000000001,"15min":5.141},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5409999999999999,"15min":1.53},"min":{"1min":1.2589999999999999,"5min":1.095,"15min":0.71099999999999997},"max":{"1min":1.845,"5min":2.161,"15min":2.657},"last":1.3720000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.468,"15min":1.5389999999999999},"min":{"1min":1.1819999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.9319999999999999,"5min":1.976,"15min":11.829000000000001},"last":1.605},{"interface":"front","average":{"1min":1.379,"5min":1.395,"15min":1.4299999999999999},"min":{"1min":1.0509999999999999,"5min":0.96099999999999997,"15min":0.73399999999999999},"max":{"1min":1.5980000000000001,"5min":1.8560000000000001,"15min":4.8419999999999996},"last":1.341}]}]},{"osd":4,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194182,"statfs":{"total":200000000,"available":198843287,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.1699999999999999,"15min":1.1399999999999999},"min":{"1min":0.73499999999999999,"5min":0.66300000000000003,"15min":0.45900000000000002},"max":{"1min":1.669,"5min":1.7410000000000001,"15min":3.1819999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.1930000000000001,"15min":1.155},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.56899999999999995},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":3.234},"last":1.036}]},{"osd":1,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1870000000000001,"15min":1.1479999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.60799999999999998},"max":{"1min":1.5209999999999999,"5min":1.9690000000000001,"15min":3.718},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.206,"15min":1.206},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.51400000000000001},"max":{"1min":1.802,"5min":1.9259999999999999,"15min":3.4350000000000001},"last":0.753}]},{"osd":2,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.236,"15min":1.216},"min":{"1min":0.79100000000000004,"5min":0.76200000000000001,"15min":0.56699999999999995},"max":{"1min":1.518,"5min":1.905,"15min":4.1749999999999998},"last":1.117},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2769999999999999,"15min":1.294},"min":{"1min":0.96799999999999997,"5min":0.64500000000000002,"15min":0.58899999999999997},"max":{"1min":1.601,"5min":1.6919999999999999,"15min":3.532},"last":0.97699999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.323,"15min":1.2889999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.67000000000000004},"max":{"1min":1.754,"5min":1.754,"15min":3.363},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3879999999999999,"15min":1.399},"min":{"1min":0.96199999999999997,"5min":0.92200000000000004,"15min":0.68899999999999995},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":4.3179999999999996},"last":1.21}]},{"osd":5,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3,"15min":1.3049999999999999},"min":{"1min":1.161,"5min":0.55600000000000005,"15min":0.498},"max":{"1min":1.611,"5min":1.841,"15min":3.6160000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.331,"15min":1.325},"min":{"1min":0.95899999999999996,"5min":0.60099999999999998,"15min":0.38400000000000001},"max":{"1min":1.6799999999999999,"5min":1.702,"15min":4.0780000000000003},"last":1.403}]},{"osd":6,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.349,"15min":1.3939999999999999},"min":{"1min":1.1859999999999999,"5min":0.67400000000000004,"15min":0.51600000000000001},"max":{"1min":1.6299999999999999,"5min":1.766,"15min":5.0049999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.494,"5min":1.4399999999999999,"15min":1.4830000000000001},"min":{"1min":1.0860000000000001,"5min":0.47499999999999998,"15min":0.45400000000000001},"max":{"1min":1.7290000000000001,"5min":1.827,"15min":4.407},"last":1.5900000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.4430000000000001,"15min":1.452},"min":{"1min":1.0129999999999999,"5min":0.51200000000000001,"15min":0.438},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":4.5170000000000003},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.579,"5min":1.5069999999999999,"15min":1.5349999999999999},"min":{"1min":1.323,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.7949999999999999,"5min":1.948,"15min":4.5460000000000003},"last":1.3859999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575029,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.89300000000000002,"15min":0.90600000000000003},"min":{"1min":0.45800000000000002,"5min":0.42699999999999999,"15min":0.35899999999999999},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":5.5730000000000004},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.028,"15min":0.98499999999999999},"min":{"1min":0.58399999999999996,"5min":0.53200000000000003,"15min":0.32700000000000001},"max":{"1min":1.665,"5min":1.863,"15min":6.5599999999999996},"last":1.0629999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0600000000000001,"15min":1.0229999999999999},"min":{"1min":0.63100000000000001,"5min":0.52200000000000002,"15min":0.36799999999999999},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":6.8099999999999996},"last":0.872},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.335,"15min":1.258},"min":{"1min":0.61299999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":1.4650000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.1659999999999999,"15min":1.117},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":1.7150000000000001,"5min":1.9550000000000001,"15min":7.4580000000000002},"last":1.351},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.276,"15min":1.2070000000000001},"min":{"1min":0.65100000000000002,"5min":0.60699999999999998,"15min":0.51400000000000001},"max":{"1min":2.407,"5min":2.407,"15min":5.0049999999999999},"last":1.2070000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.385,"15min":1.3999999999999999},"min":{"1min":0.87,"5min":0.79300000000000004,"15min":0.53700000000000003},"max":{"1min":1.8100000000000001,"5min":2.0859999999999999,"15min":18.664999999999999},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.6200000000000001,"5min":1.427,"15min":1.327},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.64000000000000001},"max":{"1min":7.1269999999999998,"5min":7.1269999999999998,"15min":7.1269999999999998},"last":1.2649999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.3919999999999999,"15min":1.343},"min":{"1min":0.86599999999999999,"5min":0.84299999999999997,"15min":0.626},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":6.9139999999999997},"last":1.665},{"interface":"front","average":{"1min":1.496,"5min":1.514,"15min":1.506},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.56899999999999995},"max":{"1min":2.5640000000000001,"5min":2.6499999999999999,"15min":16.390999999999998},"last":1.5840000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.5569999999999999,"15min":1.512},"min":{"1min":1.0820000000000001,"5min":0.998,"15min":0.58199999999999996},"max":{"1min":2.1040000000000001,"5min":2.2839999999999998,"15min":7.7270000000000003},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.5469999999999999,"15min":1.492},"min":{"1min":1.069,"5min":1.0049999999999999,"15min":0.60199999999999998},"max":{"1min":1.9410000000000001,"5min":2.0089999999999999,"15min":7.242},"last":1.5309999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.6240000000000001,"15min":1.5489999999999999},"min":{"1min":1.0329999999999999,"5min":0.89600000000000002,"15min":0.70099999999999996},"max":{"1min":2.5099999999999998,"5min":2.8170000000000002,"15min":4.8630000000000004},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.756,"5min":1.6659999999999999,"15min":1.6020000000000001},"min":{"1min":0.84699999999999998,"5min":0.83499999999999996,"15min":0.622},"max":{"1min":5.7649999999999997,"5min":5.7649999999999997,"15min":12.946},"last":1.6299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.84199999999999997,"15min":0.86499999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.33700000000000002},"max":{"1min":1.464,"5min":1.6000000000000001,"15min":4.2210000000000001},"last":1.077},{"interface":"front","average":{"1min":0.875,"5min":0.99099999999999999,"15min":1},"min":{"1min":0.50800000000000001,"5min":0.497,"15min":0.33800000000000002},"max":{"1min":1.3440000000000001,"5min":2.2799999999999998,"15min":4.3959999999999999},"last":0.65000000000000002}]},{"osd":1,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.97399999999999998,"15min":1.0449999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1699999999999999,"5min":1.6100000000000001,"15min":7.3390000000000004},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.97299999999999998,"5min":1.087,"15min":1.1020000000000001},"min":{"1min":0.61099999999999999,"5min":0.55900000000000005,"15min":0.46200000000000002},"max":{"1min":1.4139999999999999,"5min":1.696,"15min":2.5350000000000001},"last":1.2190000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.115,"15min":1.121},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.847,"5min":1.847,"15min":7.6369999999999996},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.3240000000000001,"15min":1.341},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.54100000000000004},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":7.5570000000000004},"last":1.284}]},{"osd":4,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3939999999999999,"15min":1.4179999999999999},"min":{"1min":0.99299999999999999,"5min":0.93400000000000005,"15min":0.77900000000000003},"max":{"1min":1.9239999999999999,"5min":2.016,"15min":7.6189999999999998},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.397,"15min":1.4219999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.736,"5min":2.0430000000000001,"15min":3.423},"last":1.429}]},{"osd":5,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.26,"15min":1.298},"min":{"1min":0.89000000000000001,"5min":0.82399999999999995,"15min":0.61599999999999999},"max":{"1min":1.601,"5min":2.0600000000000001,"15min":10.507},"last":1.542},{"interface":"front","average":{"1min":1.29,"5min":1.377,"15min":1.4379999999999999},"min":{"1min":0.93600000000000005,"5min":0.873,"15min":0.81399999999999995},"max":{"1min":1.788,"5min":1.984,"15min":7.657},"last":1.4850000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.419,"15min":1.45},"min":{"1min":0.84599999999999997,"5min":0.77700000000000002,"15min":0.74299999999999999},"max":{"1min":1.772,"5min":2.2519999999999998,"15min":3.5299999999999998},"last":1.639},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.4239999999999999,"15min":1.599},"min":{"1min":0.84399999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.835,"5min":2.2639999999999998,"15min":37.771999999999998},"last":1.3380000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.5069999999999999,"15min":1.53},"min":{"1min":0.872,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9330000000000001,"5min":2.3860000000000001,"15min":3.3620000000000001},"last":1.732},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.482,"15min":1.512},"min":{"1min":0.86299999999999999,"5min":0.84299999999999997,"15min":0.77600000000000002},"max":{"1min":1.96,"5min":2.375,"15min":10.956},"last":1.6930000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575027,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194182,"statfs":{"total":200000000,"available":198843287,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.91300000000000003,"15min":0.89300000000000002},"min":{"1min":0.57199999999999995,"5min":0.378,"15min":0.315},"max":{"1min":1.3740000000000001,"5min":1.4770000000000001,"15min":1.659},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1379999999999999,"15min":1.0580000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.441},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.0189999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0720000000000001,"15min":1.018},"min":{"1min":0.49199999999999999,"5min":0.48399999999999999,"15min":0.317},"max":{"1min":1.357,"5min":1.6100000000000001,"15min":1.859},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.97699999999999998,"15min":0.94499999999999995},"min":{"1min":0.55300000000000005,"5min":0.443,"15min":0.36699999999999999},"max":{"1min":1.3580000000000001,"5min":1.8009999999999999,"15min":2.0230000000000001},"last":1.6970000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.181,"15min":1.1040000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.436},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.48},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.085,"15min":1.073},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.35399999999999998},"max":{"1min":1.4370000000000001,"5min":1.6659999999999999,"15min":3.7010000000000001},"last":1.1539999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.296,"15min":1.2929999999999999},"min":{"1min":0.81499999999999995,"5min":0.75600000000000001,"15min":0.45700000000000002},"max":{"1min":1.97,"5min":1.97,"15min":2.0489999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.335,"15min":1.3129999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.59699999999999998},"max":{"1min":1.8149999999999999,"5min":1.877,"15min":2.0470000000000002},"last":1.875}]},{"osd":5,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.478,"15min":1.401},"min":{"1min":0.90400000000000003,"5min":0.86499999999999999,"15min":0.56000000000000005},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":3.5750000000000002},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.476,"15min":1.448},"min":{"1min":0.90000000000000002,"5min":0.86399999999999999,"15min":0.71699999999999997},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.3769999999999998},"last":1.542}]},{"osd":6,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4099999999999999,"15min":1.3839999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.64200000000000002},"max":{"1min":2.044,"5min":2.044,"15min":2.2050000000000001},"last":1.9890000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.4610000000000001,"15min":1.3959999999999999},"min":{"1min":0.92900000000000005,"5min":0.89000000000000001,"15min":0.63100000000000001},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.6559999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.538,"15min":1.4730000000000001},"min":{"1min":0.95399999999999996,"5min":0.85699999999999998,"15min":0.69799999999999995},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.4790000000000001,"15min":1.4590000000000001},"min":{"1min":1,"5min":0.81100000000000005,"15min":0.66300000000000003},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.4180000000000001},"last":1.6200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575027,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193606,"statfs":{"total":200000000,"available":198252919,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.52100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.1779999999999999,"5min":1.2410000000000001,"15min":3.4420000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.071,"5min":1.091,"15min":1.089},"min":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.476,"5min":1.657,"15min":1.8240000000000001},"last":0.88400000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.90200000000000002,"15min":0.85999999999999999},"min":{"1min":0.59799999999999998,"5min":0.56299999999999994,"15min":0.375},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.679},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2290000000000001,"15min":1.161},"min":{"1min":0.78400000000000003,"5min":0.623,"15min":0.437},"max":{"1min":1.607,"5min":1.917,"15min":1.929},"last":0.98899999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.167,"15min":1.1599999999999999},"min":{"1min":0.56799999999999995,"5min":0.52100000000000002,"15min":0.49399999999999999},"max":{"1min":1.7210000000000001,"5min":1.9590000000000001,"15min":2.7639999999999998},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.4159999999999999,"15min":1.413},"min":{"1min":0.85099999999999998,"5min":0.68100000000000005,"15min":0.58399999999999996},"max":{"1min":1.77,"5min":2.0150000000000001,"15min":3.5720000000000001},"last":1.196}]},{"osd":4,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2170000000000001,"15min":1.2270000000000001},"min":{"1min":0.80800000000000005,"5min":0.73099999999999998,"15min":0.71099999999999997},"max":{"1min":1.6579999999999999,"5min":1.7370000000000001,"15min":4.492},"last":1.014},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2829999999999999,"15min":1.274},"min":{"1min":0.93600000000000005,"5min":0.77500000000000002,"15min":0.68700000000000006},"max":{"1min":1.552,"5min":1.752,"15min":1.927},"last":1.2529999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.252,"15min":1.2889999999999999},"min":{"1min":0.97899999999999998,"5min":0.75600000000000001,"15min":0.64300000000000002},"max":{"1min":1.625,"5min":1.6439999999999999,"15min":4.2779999999999996},"last":1.407},{"interface":"front","average":{"1min":1.325,"5min":1.347,"15min":1.3340000000000001},"min":{"1min":1.03,"5min":0.81899999999999995,"15min":0.73099999999999998},"max":{"1min":1.583,"5min":1.8169999999999999,"15min":1.8740000000000001},"last":1.135}]},{"osd":6,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.48,"15min":1.4730000000000001},"min":{"1min":1.0620000000000001,"5min":0.81000000000000005,"15min":0.76800000000000002},"max":{"1min":2.0459999999999998,"5min":2.2469999999999999,"15min":3.5299999999999998},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.359,"15min":1.3819999999999999},"min":{"1min":0.95099999999999996,"5min":0.86899999999999999,"15min":0.66700000000000004},"max":{"1min":1.661,"5min":1.8720000000000001,"15min":4.4349999999999996},"last":1.2969999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.5489999999999999,"15min":1.5569999999999999},"min":{"1min":1.1850000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.982,"5min":2.133,"15min":3.492},"last":1.329},{"interface":"front","average":{"1min":1.399,"5min":1.472,"15min":1.4950000000000001},"min":{"1min":0.95999999999999996,"5min":0.88500000000000001,"15min":0.82099999999999995},"max":{"1min":1.776,"5min":2.1000000000000001,"15min":4.5419999999999998},"last":1.5}]}]}],"pool_statfs":[]}} 2023-10-18T09:09:51.447 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-18T09:09:51.448 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:09:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.676+0000 7f0b7ed5a640 1 -- 172.21.15.40:0/310977436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b780cd9a0 msgr2=0x7f0b780b1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.676+0000 7f0b7ed5a640 1 --2- 172.21.15.40:0/310977436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b780cd9a0 0x7f0b780b1040 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0b70005fd0 tx=0x7f0b7002c2d0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.676+0000 7f0b7ed5a640 1 -- 172.21.15.40:0/310977436 shutdown_connections 2023-10-18T09:09:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.676+0000 7f0b7ed5a640 1 --2- 172.21.15.40:0/310977436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b780cd9a0 0x7f0b780b1040 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.676+0000 7f0b7ed5a640 1 --2- 172.21.15.40:0/310977436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b78151bd0 0x7f0b781586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.676+0000 7f0b7ed5a640 1 -- 172.21.15.40:0/310977436 >> 172.21.15.40:0/310977436 conn(0x7f0b7806cee0 msgr2=0x7f0b78069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.676+0000 7f0b7ed5a640 1 -- 172.21.15.40:0/310977436 shutdown_connections 2023-10-18T09:09:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.676+0000 7f0b7ed5a640 1 -- 172.21.15.40:0/310977436 wait complete. 2023-10-18T09:09:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.678+0000 7f0b7ed5a640 1 Processor -- start 2023-10-18T09:09:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.678+0000 7f0b7ed5a640 1 -- start start 2023-10-18T09:09:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.679+0000 7f0b7ed5a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b780b0bc0 0x7f0b781e67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.679+0000 7f0b7ed5a640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b780cd9a0 0x7f0b780aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.679+0000 7f0b7ed5a640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b78151bd0 0x7f0b780af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.679+0000 7f0b7ed5a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b78149f70 con 0x7f0b780b0bc0 2023-10-18T09:09:51.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.679+0000 7f0b7ed5a640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7814c900 con 0x7f0b78151bd0 2023-10-18T09:09:51.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.679+0000 7f0b7ed5a640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7814cbe0 con 0x7f0b780cd9a0 2023-10-18T09:09:51.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.679+0000 7f0b7caf6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b780b0bc0 0x7f0b781e67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:51.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.679+0000 7f0b7caf6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b780b0bc0 0x7f0b781e67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54188/0 (socket says 172.21.15.40:54188) 2023-10-18T09:09:51.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.679+0000 7f0b7caf6640 1 -- 172.21.15.40:0/4179301067 learned_addr learned my addr 172.21.15.40:0/4179301067 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:51.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.680+0000 7f0b7caf6640 1 -- 172.21.15.40:0/4179301067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b780cd9a0 msgr2=0x7f0b780aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:51.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.680+0000 7f0b7d2f7640 1 --2- 172.21.15.40:0/4179301067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b78151bd0 0x7f0b780af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:51.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.680+0000 7f0b6ffff640 1 --2- 172.21.15.40:0/4179301067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b780cd9a0 0x7f0b780aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:51.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.680+0000 7f0b7caf6640 1 --2- 172.21.15.40:0/4179301067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b780cd9a0 0x7f0b780aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.680+0000 7f0b7caf6640 1 -- 172.21.15.40:0/4179301067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b78151bd0 msgr2=0x7f0b780af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:51.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.680+0000 7f0b7caf6640 1 --2- 172.21.15.40:0/4179301067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b78151bd0 0x7f0b780af300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.680+0000 7f0b7caf6640 1 -- 172.21.15.40:0/4179301067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b70009900 con 0x7f0b780b0bc0 2023-10-18T09:09:51.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.680+0000 7f0b6ffff640 1 --2- 172.21.15.40:0/4179301067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b780cd9a0 0x7f0b780aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:51.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.680+0000 7f0b7d2f7640 1 --2- 172.21.15.40:0/4179301067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b78151bd0 0x7f0b780af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:51.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.680+0000 7f0b7caf6640 1 --2- 172.21.15.40:0/4179301067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b780b0bc0 0x7f0b781e67c0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400ea90 tx=0x7f0b7400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:51.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.681+0000 7f0b6d7fa640 1 -- 172.21.15.40:0/4179301067 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7400cc80 con 0x7f0b780b0bc0 2023-10-18T09:09:51.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.681+0000 7f0b6d7fa640 1 -- 172.21.15.40:0/4179301067 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0b74016410 con 0x7f0b780b0bc0 2023-10-18T09:09:51.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.681+0000 7f0b7ed5a640 1 -- 172.21.15.40:0/4179301067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b7814cec0 con 0x7f0b780b0bc0 2023-10-18T09:09:51.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.681+0000 7f0b6d7fa640 1 -- 172.21.15.40:0/4179301067 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74016820 con 0x7f0b780b0bc0 2023-10-18T09:09:51.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.681+0000 7f0b7ed5a640 1 -- 172.21.15.40:0/4179301067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b7814d480 con 0x7f0b780b0bc0 2023-10-18T09:09:51.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.682+0000 7f0b7ed5a640 1 -- 172.21.15.40:0/4179301067 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b78066fd0 con 0x7f0b780b0bc0 2023-10-18T09:09:51.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.683+0000 7f0b6d7fa640 1 -- 172.21.15.40:0/4179301067 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0b74016980 con 0x7f0b780b0bc0 2023-10-18T09:09:51.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.684+0000 7f0b6d7fa640 1 --2- 172.21.15.40:0/4179301067 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0b6406ae70 0x7f0b6406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:51.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.685+0000 7f0b6d7fa640 1 -- 172.21.15.40:0/4179301067 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0b740898f0 con 0x7f0b780b0bc0 2023-10-18T09:09:51.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.685+0000 7f0b6ffff640 1 --2- 172.21.15.40:0/4179301067 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0b6406ae70 0x7f0b6406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:51.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.686+0000 7f0b6ffff640 1 --2- 172.21.15.40:0/4179301067 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0b6406ae70 0x7f0b6406d330 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0b78149c10 tx=0x7f0b68009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:51.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.689+0000 7f0b6d7fa640 1 -- 172.21.15.40:0/4179301067 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b74054cd0 con 0x7f0b780b0bc0 2023-10-18T09:09:51.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.881+0000 7f0b7ed5a640 1 -- 172.21.15.40:0/4179301067 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b7814cd50 con 0x7f0b6406ae70 2023-10-18T09:09:51.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.888+0000 7f0b6d7fa640 1 -- 172.21.15.40:0/4179301067 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47142 (secure 0 0 0) 0x7f0b7814cd50 con 0x7f0b6406ae70 2023-10-18T09:09:51.891 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:51.891 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:09:51.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.891+0000 7f0b7ed5a640 1 -- 172.21.15.40:0/4179301067 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0b6406ae70 msgr2=0x7f0b6406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:51.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.891+0000 7f0b7ed5a640 1 --2- 172.21.15.40:0/4179301067 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0b6406ae70 0x7f0b6406d330 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0b78149c10 tx=0x7f0b68009040 comp rx=0 tx=0).stop 2023-10-18T09:09:51.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.891+0000 7f0b7ed5a640 1 -- 172.21.15.40:0/4179301067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b780b0bc0 msgr2=0x7f0b781e67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:51.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.891+0000 7f0b7ed5a640 1 --2- 172.21.15.40:0/4179301067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b780b0bc0 0x7f0b781e67c0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400ea90 tx=0x7f0b7400ef50 comp rx=0 tx=0).stop 2023-10-18T09:09:51.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.891+0000 7f0b7ed5a640 1 -- 172.21.15.40:0/4179301067 shutdown_connections 2023-10-18T09:09:51.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.891+0000 7f0b7ed5a640 1 --2- 172.21.15.40:0/4179301067 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f0b6406ae70 0x7f0b6406d330 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.891+0000 7f0b7ed5a640 1 --2- 172.21.15.40:0/4179301067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b78151bd0 0x7f0b780af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.891+0000 7f0b7ed5a640 1 --2- 172.21.15.40:0/4179301067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b780cd9a0 0x7f0b780aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.891+0000 7f0b7ed5a640 1 --2- 172.21.15.40:0/4179301067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b780b0bc0 0x7f0b781e67c0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:51.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.891+0000 7f0b7ed5a640 1 -- 172.21.15.40:0/4179301067 >> 172.21.15.40:0/4179301067 conn(0x7f0b7806cee0 msgr2=0x7f0b78070190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:51.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.891+0000 7f0b7ed5a640 1 -- 172.21.15.40:0/4179301067 shutdown_connections 2023-10-18T09:09:51.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:51.891+0000 7f0b7ed5a640 1 -- 172.21.15.40:0/4179301067 wait complete. 2023-10-18T09:09:51.915 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":640,"stamp":"2023-10-18T09:09:51.221925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10194,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552305,"statfs":{"total":1600000000,"available":1589560465,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52490893,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-129,"num_write_kb":-97371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.410276"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":363,"reported_epoch":202,"state":"active+clean","last_fresh":"2023-10-18T09:09:46.006422+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:09:46.006422+0000","last_peered":"2023-10-18T09:09:46.006422+0000","last_clean":"2023-10-18T09:09:46.006422+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:09:46.006422+0000","last_undegraded":"2023-10-18T09:09:46.006422+0000","last_fullsized":"2023-10-18T09:09:46.006422+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575027,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193605,"statfs":{"total":200000000,"available":198251900,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1140000000000001,"15min":1.0840000000000001},"min":{"1min":0.69499999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6990000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.044,"5min":1.129,"15min":1.0940000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.63100000000000001},"max":{"1min":1.5589999999999999,"5min":1.724,"15min":1.97},"last":1.3839999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1910000000000001,"15min":1.165},"min":{"1min":0.73699999999999999,"5min":0.68899999999999995,"15min":0.61799999999999999},"max":{"1min":1.4950000000000001,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.24},{"interface":"front","average":{"1min":1.121,"5min":1.1539999999999999,"15min":1.135},"min":{"1min":0.72799999999999998,"5min":0.63500000000000001,"15min":0.58799999999999997},"max":{"1min":1.571,"5min":1.782,"15min":1.7989999999999999},"last":1.1379999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.351,"15min":1.3109999999999999},"min":{"1min":0.81299999999999994,"5min":0.80700000000000005,"15min":0.65400000000000003},"max":{"1min":1.794,"5min":1.8140000000000001,"15min":1.9730000000000001},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2470000000000001,"15min":1.244},"min":{"1min":0.79500000000000004,"5min":0.72499999999999998,"15min":0.68300000000000005},"max":{"1min":1.6870000000000001,"5min":1.8169999999999999,"15min":1.863},"last":1.5700000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.3480000000000001,"15min":1.294},"min":{"1min":0.97099999999999997,"5min":0.90300000000000002,"15min":0.61299999999999999},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.98},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3979999999999999,"15min":1.359},"min":{"1min":0.88200000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8640000000000001,"5min":1.9219999999999999,"15min":1.9930000000000001},"last":1.2090000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2410000000000001,"15min":1.228},"min":{"1min":0.89500000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.7390000000000001,"5min":1.869,"15min":2.1070000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.3300000000000001,"15min":1.3400000000000001},"min":{"1min":0.96299999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7470000000000001,"5min":1.9570000000000001,"15min":2.5169999999999999},"last":1.3380000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3480000000000001,"15min":1.345},"min":{"1min":0.78500000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":2.0720000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.393,"15min":1.4079999999999999},"min":{"1min":0.92000000000000004,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.968,"5min":1.968,"15min":2.0270000000000001},"last":1.488}]},{"osd":6,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.484,"15min":1.482},"min":{"1min":0.83699999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.3959999999999999},"last":1.661},{"interface":"front","average":{"1min":1.528,"5min":1.474,"15min":1.4850000000000001},"min":{"1min":1.1879999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.8959999999999999,"5min":2.0680000000000001,"15min":2.1680000000000001},"last":1.5169999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2250000000000001,"15min":1.2050000000000001},"min":{"1min":0.80900000000000005,"5min":0.64200000000000002,"15min":0.60299999999999998},"max":{"1min":1.575,"5min":1.643,"15min":1.6850000000000001},"last":1.054},{"interface":"front","average":{"1min":1.327,"5min":1.3799999999999999,"15min":1.325},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.64700000000000002},"max":{"1min":1.796,"5min":1.899,"15min":2.0990000000000002},"last":1.4490000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.181,"15min":1.127},"min":{"1min":0.81499999999999995,"5min":0.69899999999999995,"15min":0.63800000000000001},"max":{"1min":1.605,"5min":1.605,"15min":1.722},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.173,"5min":1.2430000000000001,"15min":1.2150000000000001},"min":{"1min":0.76200000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.627,"5min":1.8169999999999999,"15min":2.9609999999999999},"last":1.609}]},{"osd":2,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.234,"15min":1.1950000000000001},"min":{"1min":0.85199999999999998,"5min":0.70299999999999996,"15min":0.69899999999999995},"max":{"1min":1.4790000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3460000000000001,"15min":1.2629999999999999},"min":{"1min":0.86399999999999999,"5min":0.748,"15min":0.66300000000000003},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":2.1779999999999999},"last":1.175}]},{"osd":3,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.3220000000000001,"15min":1.288},"min":{"1min":1.196,"5min":0.82999999999999996,"15min":0.79200000000000004},"max":{"1min":1.587,"5min":1.8160000000000001,"15min":2.1589999999999998},"last":1.391},{"interface":"front","average":{"1min":1.286,"5min":1.355,"15min":1.3009999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.78600000000000003},"max":{"1min":1.617,"5min":1.8540000000000001,"15min":1.994},"last":1.552}]},{"osd":4,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.3029999999999999,"15min":1.272},"min":{"1min":1.075,"5min":0.80600000000000005,"15min":0.58299999999999996},"max":{"1min":1.8500000000000001,"5min":1.9770000000000001,"15min":2.444},"last":1.742},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.363,"15min":1.3120000000000001},"min":{"1min":0.877,"5min":0.82999999999999996,"15min":0.55000000000000004},"max":{"1min":1.766,"5min":1.9159999999999999,"15min":2.4039999999999999},"last":1.5109999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4510000000000001,"15min":1.423},"min":{"1min":1.204,"5min":0.92900000000000005,"15min":0.58899999999999997},"max":{"1min":1.631,"5min":2.016,"15min":2.032},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3680000000000001,"15min":1.3260000000000001},"min":{"1min":1.087,"5min":0.77100000000000002,"15min":0.49199999999999999},"max":{"1min":1.748,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.655}]},{"osd":7,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.4950000000000001,"15min":1.4390000000000001},"min":{"1min":1.242,"5min":1.016,"15min":0.52000000000000002},"max":{"1min":1.913,"5min":2.028,"15min":2.3540000000000001},"last":1.819},{"interface":"front","average":{"1min":1.617,"5min":1.5640000000000001,"15min":1.5389999999999999},"min":{"1min":1.304,"5min":1.1890000000000001,"15min":0.68500000000000005},"max":{"1min":1.9670000000000001,"5min":2.1440000000000001,"15min":2.794},"last":1.9330000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575028,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3200000000000001,"15min":1.3260000000000001},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.895,"5min":2.302,"15min":9.609},"last":0.876},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3819999999999999,"15min":1.331},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.60799999999999998},"max":{"1min":1.714,"5min":2.0840000000000001,"15min":5.3029999999999999},"last":0.94299999999999995}]},{"osd":1,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2370000000000001,"15min":1.256},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.792,"5min":2.0339999999999998,"15min":11.68},"last":0.998},{"interface":"front","average":{"1min":1.212,"5min":1.246,"15min":1.2130000000000001},"min":{"1min":0.74099999999999999,"5min":0.72799999999999998,"15min":0.67000000000000004},"max":{"1min":1.5009999999999999,"5min":1.79,"15min":3.7050000000000001},"last":1.1160000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.27,"15min":1.2529999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.755},"max":{"1min":1.788,"5min":1.8540000000000001,"15min":2.2090000000000001},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2569999999999999,"15min":1.264},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.72399999999999998},"max":{"1min":1.6359999999999999,"5min":1.706,"15min":3.8180000000000001},"last":1.52}]},{"osd":3,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2110000000000001,"15min":1.256},"min":{"1min":0.79800000000000004,"5min":0.745,"15min":0.65400000000000003},"max":{"1min":1.466,"5min":1.6719999999999999,"15min":11.797000000000001},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.3740000000000001,"15min":1.365},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.747},"max":{"1min":1.6020000000000001,"5min":2.105,"15min":2.3420000000000001},"last":1.1779999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.341,"15min":1.373},"min":{"1min":1.004,"5min":0.755,"15min":0.755},"max":{"1min":1.7190000000000001,"5min":2.028,"15min":2.4449999999999998},"last":1.46},{"interface":"front","average":{"1min":1.409,"5min":1.3380000000000001,"15min":1.345},"min":{"1min":1.1319999999999999,"5min":0.67700000000000005,"15min":0.64100000000000001},"max":{"1min":1.7230000000000001,"5min":1.982,"15min":5.4580000000000002},"last":1.411}]},{"osd":6,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.246,"15min":1.2809999999999999},"min":{"1min":0.93600000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.667,"5min":1.7410000000000001,"15min":5.141},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5409999999999999,"15min":1.53},"min":{"1min":1.2589999999999999,"5min":1.095,"15min":0.71099999999999997},"max":{"1min":1.845,"5min":2.161,"15min":2.657},"last":1.3720000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.468,"15min":1.5389999999999999},"min":{"1min":1.1819999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.9319999999999999,"5min":1.976,"15min":11.829000000000001},"last":1.605},{"interface":"front","average":{"1min":1.379,"5min":1.395,"15min":1.4299999999999999},"min":{"1min":1.0509999999999999,"5min":0.96099999999999997,"15min":0.73399999999999999},"max":{"1min":1.5980000000000001,"5min":1.8560000000000001,"15min":4.8419999999999996},"last":1.341}]}]},{"osd":4,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194182,"statfs":{"total":200000000,"available":198843287,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.1699999999999999,"15min":1.1399999999999999},"min":{"1min":0.73499999999999999,"5min":0.66300000000000003,"15min":0.45900000000000002},"max":{"1min":1.669,"5min":1.7410000000000001,"15min":3.1819999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.1930000000000001,"15min":1.155},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.56899999999999995},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":3.234},"last":1.036}]},{"osd":1,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1870000000000001,"15min":1.1479999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.60799999999999998},"max":{"1min":1.5209999999999999,"5min":1.9690000000000001,"15min":3.718},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.206,"15min":1.206},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.51400000000000001},"max":{"1min":1.802,"5min":1.9259999999999999,"15min":3.4350000000000001},"last":0.753}]},{"osd":2,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.236,"15min":1.216},"min":{"1min":0.79100000000000004,"5min":0.76200000000000001,"15min":0.56699999999999995},"max":{"1min":1.518,"5min":1.905,"15min":4.1749999999999998},"last":1.117},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2769999999999999,"15min":1.294},"min":{"1min":0.96799999999999997,"5min":0.64500000000000002,"15min":0.58899999999999997},"max":{"1min":1.601,"5min":1.6919999999999999,"15min":3.532},"last":0.97699999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.323,"15min":1.2889999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.67000000000000004},"max":{"1min":1.754,"5min":1.754,"15min":3.363},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3879999999999999,"15min":1.399},"min":{"1min":0.96199999999999997,"5min":0.92200000000000004,"15min":0.68899999999999995},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":4.3179999999999996},"last":1.21}]},{"osd":5,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3,"15min":1.3049999999999999},"min":{"1min":1.161,"5min":0.55600000000000005,"15min":0.498},"max":{"1min":1.611,"5min":1.841,"15min":3.6160000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.331,"15min":1.325},"min":{"1min":0.95899999999999996,"5min":0.60099999999999998,"15min":0.38400000000000001},"max":{"1min":1.6799999999999999,"5min":1.702,"15min":4.0780000000000003},"last":1.403}]},{"osd":6,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.349,"15min":1.3939999999999999},"min":{"1min":1.1859999999999999,"5min":0.67400000000000004,"15min":0.51600000000000001},"max":{"1min":1.6299999999999999,"5min":1.766,"15min":5.0049999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.494,"5min":1.4399999999999999,"15min":1.4830000000000001},"min":{"1min":1.0860000000000001,"5min":0.47499999999999998,"15min":0.45400000000000001},"max":{"1min":1.7290000000000001,"5min":1.827,"15min":4.407},"last":1.5900000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.4430000000000001,"15min":1.452},"min":{"1min":1.0129999999999999,"5min":0.51200000000000001,"15min":0.438},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":4.5170000000000003},"last":1.7070000000000001},{"interface":"front","average":{"1min":1.579,"5min":1.5069999999999999,"15min":1.5349999999999999},"min":{"1min":1.323,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.7949999999999999,"5min":1.948,"15min":4.5460000000000003},"last":1.3859999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575029,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.89300000000000002,"15min":0.90600000000000003},"min":{"1min":0.45800000000000002,"5min":0.42699999999999999,"15min":0.35899999999999999},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":5.5730000000000004},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.028,"15min":0.98499999999999999},"min":{"1min":0.58399999999999996,"5min":0.53200000000000003,"15min":0.32700000000000001},"max":{"1min":1.665,"5min":1.863,"15min":6.5599999999999996},"last":1.0629999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0600000000000001,"15min":1.0229999999999999},"min":{"1min":0.63100000000000001,"5min":0.52200000000000002,"15min":0.36799999999999999},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":6.8099999999999996},"last":0.872},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.335,"15min":1.258},"min":{"1min":0.61299999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":1.4650000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.1659999999999999,"15min":1.117},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":1.7150000000000001,"5min":1.9550000000000001,"15min":7.4580000000000002},"last":1.351},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.276,"15min":1.2070000000000001},"min":{"1min":0.65100000000000002,"5min":0.60699999999999998,"15min":0.51400000000000001},"max":{"1min":2.407,"5min":2.407,"15min":5.0049999999999999},"last":1.2070000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.385,"15min":1.3999999999999999},"min":{"1min":0.87,"5min":0.79300000000000004,"15min":0.53700000000000003},"max":{"1min":1.8100000000000001,"5min":2.0859999999999999,"15min":18.664999999999999},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.6200000000000001,"5min":1.427,"15min":1.327},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.64000000000000001},"max":{"1min":7.1269999999999998,"5min":7.1269999999999998,"15min":7.1269999999999998},"last":1.2649999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.3919999999999999,"15min":1.343},"min":{"1min":0.86599999999999999,"5min":0.84299999999999997,"15min":0.626},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":6.9139999999999997},"last":1.665},{"interface":"front","average":{"1min":1.496,"5min":1.514,"15min":1.506},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.56899999999999995},"max":{"1min":2.5640000000000001,"5min":2.6499999999999999,"15min":16.390999999999998},"last":1.5840000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.5569999999999999,"15min":1.512},"min":{"1min":1.0820000000000001,"5min":0.998,"15min":0.58199999999999996},"max":{"1min":2.1040000000000001,"5min":2.2839999999999998,"15min":7.7270000000000003},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.5469999999999999,"15min":1.492},"min":{"1min":1.069,"5min":1.0049999999999999,"15min":0.60199999999999998},"max":{"1min":1.9410000000000001,"5min":2.0089999999999999,"15min":7.242},"last":1.5309999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.6240000000000001,"15min":1.5489999999999999},"min":{"1min":1.0329999999999999,"5min":0.89600000000000002,"15min":0.70099999999999996},"max":{"1min":2.5099999999999998,"5min":2.8170000000000002,"15min":4.8630000000000004},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.756,"5min":1.6659999999999999,"15min":1.6020000000000001},"min":{"1min":0.84699999999999998,"5min":0.83499999999999996,"15min":0.622},"max":{"1min":5.7649999999999997,"5min":5.7649999999999997,"15min":12.946},"last":1.6299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.84199999999999997,"15min":0.86499999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.33700000000000002},"max":{"1min":1.464,"5min":1.6000000000000001,"15min":4.2210000000000001},"last":1.077},{"interface":"front","average":{"1min":0.875,"5min":0.99099999999999999,"15min":1},"min":{"1min":0.50800000000000001,"5min":0.497,"15min":0.33800000000000002},"max":{"1min":1.3440000000000001,"5min":2.2799999999999998,"15min":4.3959999999999999},"last":0.65000000000000002}]},{"osd":1,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.97399999999999998,"15min":1.0449999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1699999999999999,"5min":1.6100000000000001,"15min":7.3390000000000004},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.97299999999999998,"5min":1.087,"15min":1.1020000000000001},"min":{"1min":0.61099999999999999,"5min":0.55900000000000005,"15min":0.46200000000000002},"max":{"1min":1.4139999999999999,"5min":1.696,"15min":2.5350000000000001},"last":1.2190000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.115,"15min":1.121},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.847,"5min":1.847,"15min":7.6369999999999996},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.3240000000000001,"15min":1.341},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.54100000000000004},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":7.5570000000000004},"last":1.284}]},{"osd":4,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3939999999999999,"15min":1.4179999999999999},"min":{"1min":0.99299999999999999,"5min":0.93400000000000005,"15min":0.77900000000000003},"max":{"1min":1.9239999999999999,"5min":2.016,"15min":7.6189999999999998},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.397,"15min":1.4219999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.736,"5min":2.0430000000000001,"15min":3.423},"last":1.429}]},{"osd":5,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.26,"15min":1.298},"min":{"1min":0.89000000000000001,"5min":0.82399999999999995,"15min":0.61599999999999999},"max":{"1min":1.601,"5min":2.0600000000000001,"15min":10.507},"last":1.542},{"interface":"front","average":{"1min":1.29,"5min":1.377,"15min":1.4379999999999999},"min":{"1min":0.93600000000000005,"5min":0.873,"15min":0.81399999999999995},"max":{"1min":1.788,"5min":1.984,"15min":7.657},"last":1.4850000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.419,"15min":1.45},"min":{"1min":0.84599999999999997,"5min":0.77700000000000002,"15min":0.74299999999999999},"max":{"1min":1.772,"5min":2.2519999999999998,"15min":3.5299999999999998},"last":1.639},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.4239999999999999,"15min":1.599},"min":{"1min":0.84399999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.835,"5min":2.2639999999999998,"15min":37.771999999999998},"last":1.3380000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.5069999999999999,"15min":1.53},"min":{"1min":0.872,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9330000000000001,"5min":2.3860000000000001,"15min":3.3620000000000001},"last":1.732},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.482,"15min":1.512},"min":{"1min":0.86299999999999999,"5min":0.84299999999999997,"15min":0.77600000000000002},"max":{"1min":1.96,"5min":2.375,"15min":10.956},"last":1.6930000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575027,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194182,"statfs":{"total":200000000,"available":198843287,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.91300000000000003,"15min":0.89300000000000002},"min":{"1min":0.57199999999999995,"5min":0.378,"15min":0.315},"max":{"1min":1.3740000000000001,"5min":1.4770000000000001,"15min":1.659},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1379999999999999,"15min":1.0580000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.441},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.0189999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0720000000000001,"15min":1.018},"min":{"1min":0.49199999999999999,"5min":0.48399999999999999,"15min":0.317},"max":{"1min":1.357,"5min":1.6100000000000001,"15min":1.859},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.97699999999999998,"15min":0.94499999999999995},"min":{"1min":0.55300000000000005,"5min":0.443,"15min":0.36699999999999999},"max":{"1min":1.3580000000000001,"5min":1.8009999999999999,"15min":2.0230000000000001},"last":1.6970000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.181,"15min":1.1040000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.436},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.48},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.085,"15min":1.073},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.35399999999999998},"max":{"1min":1.4370000000000001,"5min":1.6659999999999999,"15min":3.7010000000000001},"last":1.1539999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.296,"15min":1.2929999999999999},"min":{"1min":0.81499999999999995,"5min":0.75600000000000001,"15min":0.45700000000000002},"max":{"1min":1.97,"5min":1.97,"15min":2.0489999999999999},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.335,"15min":1.3129999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.59699999999999998},"max":{"1min":1.8149999999999999,"5min":1.877,"15min":2.0470000000000002},"last":1.875}]},{"osd":5,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.478,"15min":1.401},"min":{"1min":0.90400000000000003,"5min":0.86499999999999999,"15min":0.56000000000000005},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":3.5750000000000002},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.476,"15min":1.448},"min":{"1min":0.90000000000000002,"5min":0.86399999999999999,"15min":0.71699999999999997},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.3769999999999998},"last":1.542}]},{"osd":6,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4099999999999999,"15min":1.3839999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.64200000000000002},"max":{"1min":2.044,"5min":2.044,"15min":2.2050000000000001},"last":1.9890000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.4610000000000001,"15min":1.3959999999999999},"min":{"1min":0.92900000000000005,"5min":0.89000000000000001,"15min":0.63100000000000001},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.6559999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.538,"15min":1.4730000000000001},"min":{"1min":0.95399999999999996,"5min":0.85699999999999998,"15min":0.69799999999999995},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.4790000000000001,"15min":1.4590000000000001},"min":{"1min":1,"5min":0.81100000000000005,"15min":0.66300000000000003},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.4180000000000001},"last":1.6200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575027,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193606,"statfs":{"total":200000000,"available":198252919,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.52100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.1779999999999999,"5min":1.2410000000000001,"15min":3.4420000000000002},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.071,"5min":1.091,"15min":1.089},"min":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.476,"5min":1.657,"15min":1.8240000000000001},"last":0.88400000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.90200000000000002,"15min":0.85999999999999999},"min":{"1min":0.59799999999999998,"5min":0.56299999999999994,"15min":0.375},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.679},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2290000000000001,"15min":1.161},"min":{"1min":0.78400000000000003,"5min":0.623,"15min":0.437},"max":{"1min":1.607,"5min":1.917,"15min":1.929},"last":0.98899999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.167,"15min":1.1599999999999999},"min":{"1min":0.56799999999999995,"5min":0.52100000000000002,"15min":0.49399999999999999},"max":{"1min":1.7210000000000001,"5min":1.9590000000000001,"15min":2.7639999999999998},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.4159999999999999,"15min":1.413},"min":{"1min":0.85099999999999998,"5min":0.68100000000000005,"15min":0.58399999999999996},"max":{"1min":1.77,"5min":2.0150000000000001,"15min":3.5720000000000001},"last":1.196}]},{"osd":4,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2170000000000001,"15min":1.2270000000000001},"min":{"1min":0.80800000000000005,"5min":0.73099999999999998,"15min":0.71099999999999997},"max":{"1min":1.6579999999999999,"5min":1.7370000000000001,"15min":4.492},"last":1.014},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2829999999999999,"15min":1.274},"min":{"1min":0.93600000000000005,"5min":0.77500000000000002,"15min":0.68700000000000006},"max":{"1min":1.552,"5min":1.752,"15min":1.927},"last":1.2529999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.252,"15min":1.2889999999999999},"min":{"1min":0.97899999999999998,"5min":0.75600000000000001,"15min":0.64300000000000002},"max":{"1min":1.625,"5min":1.6439999999999999,"15min":4.2779999999999996},"last":1.407},{"interface":"front","average":{"1min":1.325,"5min":1.347,"15min":1.3340000000000001},"min":{"1min":1.03,"5min":0.81899999999999995,"15min":0.73099999999999998},"max":{"1min":1.583,"5min":1.8169999999999999,"15min":1.8740000000000001},"last":1.135}]},{"osd":6,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.48,"15min":1.4730000000000001},"min":{"1min":1.0620000000000001,"5min":0.81000000000000005,"15min":0.76800000000000002},"max":{"1min":2.0459999999999998,"5min":2.2469999999999999,"15min":3.5299999999999998},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.359,"15min":1.3819999999999999},"min":{"1min":0.95099999999999996,"5min":0.86899999999999999,"15min":0.66700000000000004},"max":{"1min":1.661,"5min":1.8720000000000001,"15min":4.4349999999999996},"last":1.2969999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.5489999999999999,"15min":1.5569999999999999},"min":{"1min":1.1850000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.982,"5min":2.133,"15min":3.492},"last":1.329},{"interface":"front","average":{"1min":1.399,"5min":1.472,"15min":1.4950000000000001},"min":{"1min":0.95999999999999996,"5min":0.88500000000000001,"15min":0.82099999999999995},"max":{"1min":1.776,"5min":2.1000000000000001,"15min":4.5419999999999998},"last":1.5}]}]}],"pool_statfs":[]}} 2023-10-18T09:09:51.917 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:09:52.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.144+0000 7fd73b59e640 1 -- 172.21.15.40:0/2209225584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73c0ab2a0 msgr2=0x7fd73c0cb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:52.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.144+0000 7fd73b59e640 1 --2- 172.21.15.40:0/2209225584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73c0ab2a0 0x7fd73c0cb3d0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fd73c066670 tx=0x7fd73402c2b0 comp rx=0 tx=0).stop 2023-10-18T09:09:52.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.144+0000 7fd73b59e640 1 -- 172.21.15.40:0/2209225584 shutdown_connections 2023-10-18T09:09:52.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.144+0000 7fd73b59e640 1 --2- 172.21.15.40:0/2209225584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73c0ab2a0 0x7fd73c0cb3d0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:52.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.144+0000 7fd73b59e640 1 --2- 172.21.15.40:0/2209225584 >> v2:172.21.15.144:3300/0 conn(0x7fd73c152560 0x7fd73c070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:52.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.144+0000 7fd73b59e640 1 -- 172.21.15.40:0/2209225584 >> 172.21.15.40:0/2209225584 conn(0x7fd73c069720 msgr2=0x7fd73c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:52.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.144+0000 7fd73b59e640 1 -- 172.21.15.40:0/2209225584 shutdown_connections 2023-10-18T09:09:52.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.144+0000 7fd73b59e640 1 -- 172.21.15.40:0/2209225584 wait complete. 2023-10-18T09:09:52.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.146+0000 7fd73b59e640 1 Processor -- start 2023-10-18T09:09:52.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.146+0000 7fd73b59e640 1 -- start start 2023-10-18T09:09:52.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.147+0000 7fd73b59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd73c071440 0x7fd73c0a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:52.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.147+0000 7fd73b59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73c0ab2a0 0x7fd73c1f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:52.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.147+0000 7fd73b59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd73c152560 0x7fd73c15de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:52.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.147+0000 7fd73b59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c140bd0 con 0x7fd73c0ab2a0 2023-10-18T09:09:52.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.147+0000 7fd73b59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c1fa1e0 con 0x7fd73c152560 2023-10-18T09:09:52.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.147+0000 7fd73b59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c1fa4c0 con 0x7fd73c071440 2023-10-18T09:09:52.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.147+0000 7fd731d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73c0ab2a0 0x7fd73c1f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:52.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.147+0000 7fd731d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73c0ab2a0 0x7fd73c1f9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54204/0 (socket says 172.21.15.40:54204) 2023-10-18T09:09:52.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.147+0000 7fd731d9b640 1 -- 172.21.15.40:0/1955483981 learned_addr learned my addr 172.21.15.40:0/1955483981 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:52.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.148+0000 7fd73a59c640 1 --2- 172.21.15.40:0/1955483981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd73c071440 0x7fd73c0a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:52.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.148+0000 7fd73ad9d640 1 --2- 172.21.15.40:0/1955483981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd73c152560 0x7fd73c15de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:52.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.148+0000 7fd731d9b640 1 -- 172.21.15.40:0/1955483981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd73c071440 msgr2=0x7fd73c0a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:52.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.148+0000 7fd731d9b640 1 --2- 172.21.15.40:0/1955483981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd73c071440 0x7fd73c0a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:52.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.148+0000 7fd731d9b640 1 -- 172.21.15.40:0/1955483981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd73c152560 msgr2=0x7fd73c15de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:52.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.148+0000 7fd731d9b640 1 --2- 172.21.15.40:0/1955483981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd73c152560 0x7fd73c15de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:52.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.148+0000 7fd731d9b640 1 -- 172.21.15.40:0/1955483981 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd734009900 con 0x7fd73c0ab2a0 2023-10-18T09:09:52.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.148+0000 7fd731d9b640 1 --2- 172.21.15.40:0/1955483981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73c0ab2a0 0x7fd73c1f9720 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fd72400c430 tx=0x7fd72400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:52.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.149+0000 7fd7337fe640 1 -- 172.21.15.40:0/1955483981 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd724011790 con 0x7fd73c0ab2a0 2023-10-18T09:09:52.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.149+0000 7fd7337fe640 1 -- 172.21.15.40:0/1955483981 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd7240119e0 con 0x7fd73c0ab2a0 2023-10-18T09:09:52.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.149+0000 7fd7337fe640 1 -- 172.21.15.40:0/1955483981 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd724011e30 con 0x7fd73c0ab2a0 2023-10-18T09:09:52.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.149+0000 7fd73b59e640 1 -- 172.21.15.40:0/1955483981 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73c140de0 con 0x7fd73c0ab2a0 2023-10-18T09:09:52.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.149+0000 7fd73b59e640 1 -- 172.21.15.40:0/1955483981 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73c0aa4b0 con 0x7fd73c0ab2a0 2023-10-18T09:09:52.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.150+0000 7fd73b59e640 1 -- 172.21.15.40:0/1955483981 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd73c141300 con 0x7fd73c0ab2a0 2023-10-18T09:09:52.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.151+0000 7fd7337fe640 1 -- 172.21.15.40:0/1955483981 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd72401c460 con 0x7fd73c0ab2a0 2023-10-18T09:09:52.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.152+0000 7fd7337fe640 1 --2- 172.21.15.40:0/1955483981 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd71c06ae40 0x7fd71c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:52.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.152+0000 7fd7337fe640 1 -- 172.21.15.40:0/1955483981 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd7240894f0 con 0x7fd73c0ab2a0 2023-10-18T09:09:52.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.153+0000 7fd73a59c640 1 --2- 172.21.15.40:0/1955483981 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd71c06ae40 0x7fd71c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:52.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.154+0000 7fd73a59c640 1 --2- 172.21.15.40:0/1955483981 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd71c06ae40 0x7fd71c06d300 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd72c005fd0 tx=0x7fd72c005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:52.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.156+0000 7fd7337fe640 1 -- 172.21.15.40:0/1955483981 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd73c141300 con 0x7fd73c0ab2a0 2023-10-18T09:09:52.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.353+0000 7fd73b59e640 1 -- 172.21.15.40:0/1955483981 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd73c141300 con 0x7fd73c0ab2a0 2023-10-18T09:09:52.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.355+0000 7fd7337fe640 1 -- 172.21.15.40:0/1955483981 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+13009 (secure 0 0 0) 0x7fd73c141300 con 0x7fd73c0ab2a0 2023-10-18T09:09:52.358 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:52.358 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":203,"fsid":"029855e3-538a-4478-b323-f54ab0f3a56d","created":"2023-10-18T08:54:55.880346+0000","modified":"2023-10-18T09:09:47.230957+0000","last_up_change":"2023-10-18T08:55:04.018251+0000","last_in_change":"2023-10-18T08:55:00.991196+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T08:55:04.972118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6801","nonce":1070603306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6803","nonce":1070603306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6807","nonce":1070603306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":1070603306},{"type":"v1","addr":"172.21.15.40:6805","nonce":1070603306}]},"public_addr":"172.21.15.40:6801/1070603306","cluster_addr":"172.21.15.40:6803/1070603306","heartbeat_back_addr":"172.21.15.40:6807/1070603306","heartbeat_front_addr":"172.21.15.40:6805/1070603306","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6809","nonce":2762063371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6811","nonce":2762063371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6815","nonce":2762063371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2762063371},{"type":"v1","addr":"172.21.15.40:6813","nonce":2762063371}]},"public_addr":"172.21.15.40:6809/2762063371","cluster_addr":"172.21.15.40:6811/2762063371","heartbeat_back_addr":"172.21.15.40:6815/2762063371","heartbeat_front_addr":"172.21.15.40:6813/2762063371","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6825","nonce":1187944209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6827","nonce":1187944209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6831","nonce":1187944209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1187944209},{"type":"v1","addr":"172.21.15.40:6829","nonce":1187944209}]},"public_addr":"172.21.15.40:6825/1187944209","cluster_addr":"172.21.15.40:6827/1187944209","heartbeat_back_addr":"172.21.15.40:6831/1187944209","heartbeat_front_addr":"172.21.15.40:6829/1187944209","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6817","nonce":3903517270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6819","nonce":3903517270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6824","nonce":3903517270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3903517270},{"type":"v1","addr":"172.21.15.40:6821","nonce":3903517270}]},"public_addr":"172.21.15.40:6817/3903517270","cluster_addr":"172.21.15.40:6819/3903517270","heartbeat_back_addr":"172.21.15.40:6824/3903517270","heartbeat_front_addr":"172.21.15.40:6821/3903517270","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6811","nonce":1718994813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6813","nonce":1718994813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6817","nonce":1718994813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1718994813},{"type":"v1","addr":"172.21.15.144:6815","nonce":1718994813}]},"public_addr":"172.21.15.144:6811/1718994813","cluster_addr":"172.21.15.144:6813/1718994813","heartbeat_back_addr":"172.21.15.144:6817/1718994813","heartbeat_front_addr":"172.21.15.144:6815/1718994813","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6803","nonce":1833904789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6805","nonce":1833904789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6809","nonce":1833904789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":1833904789},{"type":"v1","addr":"172.21.15.144:6807","nonce":1833904789}]},"public_addr":"172.21.15.144:6803/1833904789","cluster_addr":"172.21.15.144:6805/1833904789","heartbeat_back_addr":"172.21.15.144:6809/1833904789","heartbeat_front_addr":"172.21.15.144:6807/1833904789","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6819","nonce":1988482345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6821","nonce":1988482345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6825","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6827","nonce":1988482345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":1988482345},{"type":"v1","addr":"172.21.15.144:6823","nonce":1988482345}]},"public_addr":"172.21.15.144:6819/1988482345","cluster_addr":"172.21.15.144:6821/1988482345","heartbeat_back_addr":"172.21.15.144:6827/1988482345","heartbeat_front_addr":"172.21.15.144:6823/1988482345","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6826","nonce":3907866946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6829","nonce":3907866946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6833","nonce":3907866946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":3907866946},{"type":"v1","addr":"172.21.15.144:6831","nonce":3907866946}]},"public_addr":"172.21.15.144:6826/3907866946","cluster_addr":"172.21.15.144:6829/3907866946","heartbeat_back_addr":"172.21.15.144:6833/3907866946","heartbeat_front_addr":"172.21.15.144:6831/3907866946","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.261552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.353254+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.272925+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.281007+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.230949+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.204839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T08:55:02.253671+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6835/3976883051":"2023-10-19T09:09:43.803676+0000","172.21.15.40:6833/3976883051":"2023-10-19T09:09:43.803676+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:09:52.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.358+0000 7fd73b59e640 1 -- 172.21.15.40:0/1955483981 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd71c06ae40 msgr2=0x7fd71c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:52.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.358+0000 7fd73b59e640 1 --2- 172.21.15.40:0/1955483981 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd71c06ae40 0x7fd71c06d300 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd72c005fd0 tx=0x7fd72c005850 comp rx=0 tx=0).stop 2023-10-18T09:09:52.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.358+0000 7fd73b59e640 1 -- 172.21.15.40:0/1955483981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73c0ab2a0 msgr2=0x7fd73c1f9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:52.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.358+0000 7fd73b59e640 1 --2- 172.21.15.40:0/1955483981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73c0ab2a0 0x7fd73c1f9720 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fd72400c430 tx=0x7fd72400c8f0 comp rx=0 tx=0).stop 2023-10-18T09:09:52.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.359+0000 7fd73b59e640 1 -- 172.21.15.40:0/1955483981 shutdown_connections 2023-10-18T09:09:52.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.359+0000 7fd73b59e640 1 --2- 172.21.15.40:0/1955483981 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fd71c06ae40 0x7fd71c06d300 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:52.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.359+0000 7fd73b59e640 1 --2- 172.21.15.40:0/1955483981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd73c152560 0x7fd73c15de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:52.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.359+0000 7fd73b59e640 1 --2- 172.21.15.40:0/1955483981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd73c0ab2a0 0x7fd73c1f9720 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:52.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.359+0000 7fd73b59e640 1 --2- 172.21.15.40:0/1955483981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd73c071440 0x7fd73c0a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:52.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.359+0000 7fd73b59e640 1 -- 172.21.15.40:0/1955483981 >> 172.21.15.40:0/1955483981 conn(0x7fd73c069720 msgr2=0x7fd73c14ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:52.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.359+0000 7fd73b59e640 1 -- 172.21.15.40:0/1955483981 shutdown_connections 2023-10-18T09:09:52.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:52.359+0000 7fd73b59e640 1 -- 172.21.15.40:0/1955483981 wait complete. 2023-10-18T09:09:53.380 INFO:tasks.ceph:Scrubbing osd.0 2023-10-18T09:09:53.381 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-18T09:09:53.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.620+0000 7f74ae8ea640 1 -- 172.21.15.40:0/874250602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74940047c0 msgr2=0x7f7494004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:53.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.620+0000 7f74ae8ea640 1 --2- 172.21.15.40:0/874250602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74940047c0 0x7f7494004c40 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f7498009b50 tx=0x7f74940057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:53.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.620+0000 7f74ae0e9640 1 -- 172.21.15.40:0/874250602 reap_dead start 2023-10-18T09:09:53.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.620+0000 7f74ae8ea640 1 -- 172.21.15.40:0/874250602 shutdown_connections 2023-10-18T09:09:53.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.620+0000 7f74ae8ea640 1 -- 172.21.15.40:0/874250602 >> 172.21.15.40:0/874250602 conn(0x7f74a80aeb90 msgr2=0x7f74a80aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:53.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.620+0000 7f74ae8ea640 1 -- 172.21.15.40:0/874250602 shutdown_connections 2023-10-18T09:09:53.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.620+0000 7f74ae8ea640 1 -- 172.21.15.40:0/874250602 wait complete. 2023-10-18T09:09:53.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.622+0000 7f74ae8ea640 1 Processor -- start 2023-10-18T09:09:53.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.622+0000 7f74ae8ea640 1 -- start start 2023-10-18T09:09:53.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.623+0000 7f74ae8ea640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74940047c0 0x7f74a81f1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.623+0000 7f74ae8ea640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a814a7b0 0x7f74a814cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.623+0000 7f74ae8ea640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a814c210 0x7f74a80ad360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.623+0000 7f74ae8ea640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a814a530 con 0x7f74940047c0 2023-10-18T09:09:53.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.623+0000 7f74ae8ea640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a81f21a0 con 0x7f74a814a7b0 2023-10-18T09:09:53.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.623+0000 7f74ae8ea640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a81f2480 con 0x7f74a814c210 2023-10-18T09:09:53.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.624+0000 7f74ad8e8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74940047c0 0x7f74a81f1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:53.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.624+0000 7f74ad8e8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74940047c0 0x7f74a81f1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54222/0 (socket says 172.21.15.40:54222) 2023-10-18T09:09:53.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.624+0000 7f74ad8e8640 1 -- 172.21.15.40:0/3081991556 learned_addr learned my addr 172.21.15.40:0/3081991556 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:53.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.624+0000 7f74ad0e7640 1 --2- 172.21.15.40:0/3081991556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a814a7b0 0x7f74a814cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:53.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.624+0000 7f74ad8e8640 1 -- 172.21.15.40:0/3081991556 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a814c210 msgr2=0x7f74a80ad360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:53.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.624+0000 7f74ad8e8640 1 --2- 172.21.15.40:0/3081991556 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74a814c210 0x7f74a80ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:53.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.624+0000 7f74ad8e8640 1 -- 172.21.15.40:0/3081991556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a814a7b0 msgr2=0x7f74a814cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:53.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.624+0000 7f74ad8e8640 1 --2- 172.21.15.40:0/3081991556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a814a7b0 0x7f74a814cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.624+0000 7f74ad8e8640 1 -- 172.21.15.40:0/3081991556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a81f2760 con 0x7f74940047c0 2023-10-18T09:09:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.624+0000 7f74ad8e8640 1 --2- 172.21.15.40:0/3081991556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74940047c0 0x7f74a81f1bc0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f74a400d920 tx=0x7f74a400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.625+0000 7f749e7fc640 1 -- 172.21.15.40:0/3081991556 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4006360 con 0x7f74940047c0 2023-10-18T09:09:53.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.625+0000 7f749e7fc640 1 -- 172.21.15.40:0/3081991556 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f74a400b7b0 con 0x7f74940047c0 2023-10-18T09:09:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.625+0000 7f749e7fc640 1 -- 172.21.15.40:0/3081991556 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a400bb60 con 0x7f74940047c0 2023-10-18T09:09:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.625+0000 7f74ae8ea640 1 -- 172.21.15.40:0/3081991556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a81f2a40 con 0x7f74940047c0 2023-10-18T09:09:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.625+0000 7f74ae8ea640 1 -- 172.21.15.40:0/3081991556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a814ac30 con 0x7f74940047c0 2023-10-18T09:09:53.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.626+0000 7f74ae8ea640 1 -- 172.21.15.40:0/3081991556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f74a80cc240 con 0x7f74940047c0 2023-10-18T09:09:53.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.627+0000 7f749e7fc640 1 -- 172.21.15.40:0/3081991556 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f74a400bcc0 con 0x7f74940047c0 2023-10-18T09:09:53.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.628+0000 7f749e7fc640 1 --2- 172.21.15.40:0/3081991556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f748c06ae40 0x7f748c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:53.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.628+0000 7f749e7fc640 1 -- 172.21.15.40:0/3081991556 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f74a4089cf0 con 0x7f74940047c0 2023-10-18T09:09:53.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.628+0000 7f749e7fc640 1 --2- 172.21.15.40:0/3081991556 >> [v2:172.21.15.40:6800/1070603306,v1:172.21.15.40:6801/1070603306] conn(0x7f748c070ca0 0x7f748c0730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:53.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.628+0000 7f749e7fc640 1 -- 172.21.15.40:0/3081991556 --> [v2:172.21.15.40:6800/1070603306,v1:172.21.15.40:6801/1070603306] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f748c073770 con 0x7f748c070ca0 2023-10-18T09:09:53.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.628+0000 7f749e7fc640 1 -- 172.21.15.40:0/3081991556 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74a408c250 con 0x7f74940047c0 2023-10-18T09:09:53.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.629+0000 7f74ad0e7640 1 --2- 172.21.15.40:0/3081991556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f748c06ae40 0x7f748c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:53.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.629+0000 7f74ae0e9640 1 --2- 172.21.15.40:0/3081991556 >> [v2:172.21.15.40:6800/1070603306,v1:172.21.15.40:6801/1070603306] conn(0x7f748c070ca0 0x7f748c0730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:53.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.629+0000 7f74ae0e9640 1 --2- 172.21.15.40:0/3081991556 >> [v2:172.21.15.40:6800/1070603306,v1:172.21.15.40:6801/1070603306] conn(0x7f748c070ca0 0x7f748c0730c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:53.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.630+0000 7f74ad0e7640 1 --2- 172.21.15.40:0/3081991556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f748c06ae40 0x7f748c06d300 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7498036000 tx=0x7f7498035040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:53.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.630+0000 7f749e7fc640 1 -- 172.21.15.40:0/3081991556 <== osd.0 v2:172.21.15.40:6800/1070603306 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f748c073770 con 0x7f748c070ca0 2023-10-18T09:09:53.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.643+0000 7f74ae8ea640 1 -- 172.21.15.40:0/3081991556 --> [v2:172.21.15.40:6800/1070603306,v1:172.21.15.40:6801/1070603306] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f74a8066e00 con 0x7f748c070ca0 2023-10-18T09:09:53.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.644+0000 7f749e7fc640 1 -- 172.21.15.40:0/3081991556 <== osd.0 v2:172.21.15.40:6800/1070603306 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f74a8066e00 con 0x7f748c070ca0 2023-10-18T09:09:53.646 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:09:53.646 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-18T09:09:53.647 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:09:53.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.644+0000 7f74ae8ea640 1 -- 172.21.15.40:0/3081991556 >> [v2:172.21.15.40:6800/1070603306,v1:172.21.15.40:6801/1070603306] conn(0x7f748c070ca0 msgr2=0x7f748c0730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:53.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.644+0000 7f74ae8ea640 1 --2- 172.21.15.40:0/3081991556 >> [v2:172.21.15.40:6800/1070603306,v1:172.21.15.40:6801/1070603306] conn(0x7f748c070ca0 0x7f748c0730c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:53.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.644+0000 7f74ae8ea640 1 -- 172.21.15.40:0/3081991556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f748c06ae40 msgr2=0x7f748c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:53.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.644+0000 7f74ae8ea640 1 --2- 172.21.15.40:0/3081991556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f748c06ae40 0x7f748c06d300 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7498036000 tx=0x7f7498035040 comp rx=0 tx=0).stop 2023-10-18T09:09:53.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.644+0000 7f74ae8ea640 1 -- 172.21.15.40:0/3081991556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74940047c0 msgr2=0x7f74a81f1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:53.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.644+0000 7f74ae8ea640 1 --2- 172.21.15.40:0/3081991556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74940047c0 0x7f74a81f1bc0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f74a400d920 tx=0x7f74a400dde0 comp rx=0 tx=0).stop 2023-10-18T09:09:53.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.644+0000 7f74ae0e9640 1 -- 172.21.15.40:0/3081991556 reap_dead start 2023-10-18T09:09:53.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.645+0000 7f74ae8ea640 1 -- 172.21.15.40:0/3081991556 shutdown_connections 2023-10-18T09:09:53.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.645+0000 7f74ae8ea640 1 -- 172.21.15.40:0/3081991556 >> 172.21.15.40:0/3081991556 conn(0x7f74a80aeb90 msgr2=0x7f74a81437d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:53.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.645+0000 7f74ae8ea640 1 -- 172.21.15.40:0/3081991556 shutdown_connections 2023-10-18T09:09:53.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.645+0000 7f74ae8ea640 1 -- 172.21.15.40:0/3081991556 wait complete. 2023-10-18T09:09:53.664 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-18T09:09:53.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.892+0000 7fb29e1f4640 1 -- 172.21.15.40:0/1496716701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980ca890 msgr2=0x7fb2980b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:53.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.892+0000 7fb29e1f4640 1 --2- 172.21.15.40:0/1496716701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980ca890 0x7fb2980b1d10 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fb29806a5a0 tx=0x7fb28802bda0 comp rx=0 tx=0).stop 2023-10-18T09:09:53.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.893+0000 7fb29e1f4640 1 -- 172.21.15.40:0/1496716701 shutdown_connections 2023-10-18T09:09:53.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.893+0000 7fb29e1f4640 1 --2- 172.21.15.40:0/1496716701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980ca890 0x7fb2980b1d10 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:53.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.893+0000 7fb29e1f4640 1 --2- 172.21.15.40:0/1496716701 >> v2:172.21.15.144:3300/0 conn(0x7fb298157870 0x7fb2980ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:53.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.893+0000 7fb29e1f4640 1 -- 172.21.15.40:0/1496716701 >> 172.21.15.40:0/1496716701 conn(0x7fb298069720 msgr2=0x7fb298069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:53.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.893+0000 7fb29e1f4640 1 -- 172.21.15.40:0/1496716701 shutdown_connections 2023-10-18T09:09:53.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.893+0000 7fb29e1f4640 1 -- 172.21.15.40:0/1496716701 wait complete. 2023-10-18T09:09:53.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.895+0000 7fb29e1f4640 1 Processor -- start 2023-10-18T09:09:53.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.895+0000 7fb29e1f4640 1 -- start start 2023-10-18T09:09:53.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.896+0000 7fb29e1f4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980b2250 0x7fb298159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:53.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.896+0000 7fb29e1f4640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2980ca890 0x7fb298152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:53.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.896+0000 7fb29e1f4640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb298157870 0x7fb2980c2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:53.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.896+0000 7fb29e1f4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb298152d60 con 0x7fb2980b2250 2023-10-18T09:09:53.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.896+0000 7fb29e1f4640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2981f4fc0 con 0x7fb2980ca890 2023-10-18T09:09:53.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.896+0000 7fb29e1f4640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2981f52a0 con 0x7fb298157870 2023-10-18T09:09:53.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.896+0000 7fb29d1f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980b2250 0x7fb298159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.896+0000 7fb29d1f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980b2250 0x7fb298159380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54230/0 (socket says 172.21.15.40:54230) 2023-10-18T09:09:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.896+0000 7fb29d1f2640 1 -- 172.21.15.40:0/1775917966 learned_addr learned my addr 172.21.15.40:0/1775917966 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.896+0000 7fb29c9f1640 1 --2- 172.21.15.40:0/1775917966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2980ca890 0x7fb298152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.897+0000 7fb29d9f3640 1 --2- 172.21.15.40:0/1775917966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb298157870 0x7fb2980c2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.897+0000 7fb29d1f2640 1 -- 172.21.15.40:0/1775917966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb298157870 msgr2=0x7fb2980c2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:53.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.897+0000 7fb29d1f2640 1 --2- 172.21.15.40:0/1775917966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb298157870 0x7fb2980c2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:53.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.897+0000 7fb29d1f2640 1 -- 172.21.15.40:0/1775917966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2980ca890 msgr2=0x7fb298152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:53.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.897+0000 7fb29d1f2640 1 --2- 172.21.15.40:0/1775917966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2980ca890 0x7fb298152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:53.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.897+0000 7fb29d1f2640 1 -- 172.21.15.40:0/1775917966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb288009560 con 0x7fb2980b2250 2023-10-18T09:09:53.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.897+0000 7fb29d1f2640 1 --2- 172.21.15.40:0/1775917966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980b2250 0x7fb298159380 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fb29400c970 tx=0x7fb29400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.897+0000 7fb28dffb640 1 -- 172.21.15.40:0/1775917966 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294007bb0 con 0x7fb2980b2250 2023-10-18T09:09:53.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.898+0000 7fb28dffb640 1 -- 172.21.15.40:0/1775917966 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb294007e00 con 0x7fb2980b2250 2023-10-18T09:09:53.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.898+0000 7fb29e1f4640 1 -- 172.21.15.40:0/1775917966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2981f5580 con 0x7fb2980b2250 2023-10-18T09:09:53.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.898+0000 7fb28dffb640 1 -- 172.21.15.40:0/1775917966 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29400f6b0 con 0x7fb2980b2250 2023-10-18T09:09:53.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.898+0000 7fb29e1f4640 1 -- 172.21.15.40:0/1775917966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2981f4c00 con 0x7fb2980b2250 2023-10-18T09:09:53.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.899+0000 7fb29e1f4640 1 -- 172.21.15.40:0/1775917966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb298066e00 con 0x7fb2980b2250 2023-10-18T09:09:53.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.900+0000 7fb28dffb640 1 -- 172.21.15.40:0/1775917966 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb29400f810 con 0x7fb2980b2250 2023-10-18T09:09:53.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.901+0000 7fb28dffb640 1 --2- 172.21.15.40:0/1775917966 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb27c06ae40 0x7fb27c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:53.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.901+0000 7fb28dffb640 1 -- 172.21.15.40:0/1775917966 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb294089bb0 con 0x7fb2980b2250 2023-10-18T09:09:53.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.901+0000 7fb29c9f1640 1 --2- 172.21.15.40:0/1775917966 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb27c06ae40 0x7fb27c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:53.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.903+0000 7fb29c9f1640 1 --2- 172.21.15.40:0/1775917966 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb27c06ae40 0x7fb27c06d300 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb288035000 tx=0x7fb288004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:53.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:53.905+0000 7fb28dffb640 1 -- 172.21.15.40:0/1775917966 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb294054f50 con 0x7fb2980b2250 2023-10-18T09:09:54.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.099+0000 7fb29e1f4640 1 -- 172.21.15.40:0/1775917966 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fb29806b4d0 con 0x7fb27c06ae40 2023-10-18T09:09:54.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.100+0000 7fb28dffb640 1 -- 172.21.15.40:0/1775917966 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb288004340 con 0x7fb27c06ae40 2023-10-18T09:09:54.103 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 0 to deep-scrub 2023-10-18T09:09:54.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.103+0000 7fb29e1f4640 1 -- 172.21.15.40:0/1775917966 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb27c06ae40 msgr2=0x7fb27c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:54.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.103+0000 7fb29e1f4640 1 --2- 172.21.15.40:0/1775917966 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb27c06ae40 0x7fb27c06d300 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb288035000 tx=0x7fb288004730 comp rx=0 tx=0).stop 2023-10-18T09:09:54.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.103+0000 7fb29e1f4640 1 -- 172.21.15.40:0/1775917966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980b2250 msgr2=0x7fb298159380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:54.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.103+0000 7fb29e1f4640 1 --2- 172.21.15.40:0/1775917966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980b2250 0x7fb298159380 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fb29400c970 tx=0x7fb29400ce30 comp rx=0 tx=0).stop 2023-10-18T09:09:54.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.103+0000 7fb29e1f4640 1 -- 172.21.15.40:0/1775917966 shutdown_connections 2023-10-18T09:09:54.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.103+0000 7fb29e1f4640 1 --2- 172.21.15.40:0/1775917966 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb27c06ae40 0x7fb27c06d300 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.103+0000 7fb29e1f4640 1 --2- 172.21.15.40:0/1775917966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb298157870 0x7fb2980c2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.103+0000 7fb29e1f4640 1 --2- 172.21.15.40:0/1775917966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2980ca890 0x7fb298152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.103+0000 7fb29e1f4640 1 --2- 172.21.15.40:0/1775917966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2980b2250 0x7fb298159380 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.103+0000 7fb29e1f4640 1 -- 172.21.15.40:0/1775917966 >> 172.21.15.40:0/1775917966 conn(0x7fb298069720 msgr2=0x7fb2980cc600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:54.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.103+0000 7fb29e1f4640 1 -- 172.21.15.40:0/1775917966 shutdown_connections 2023-10-18T09:09:54.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.103+0000 7fb29e1f4640 1 -- 172.21.15.40:0/1775917966 wait complete. 2023-10-18T09:09:54.124 INFO:tasks.ceph:Scrubbing osd.1 2023-10-18T09:09:54.124 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-18T09:09:54.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.347+0000 7f7af6e40640 1 -- 172.21.15.40:0/1856593229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af00b2110 msgr2=0x7f7af00b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:54.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.347+0000 7f7af6e40640 1 --2- 172.21.15.40:0/1856593229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af00b2110 0x7f7af00b0670 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8005fd0 tx=0x7f7ae802d400 comp rx=0 tx=0).stop 2023-10-18T09:09:54.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.347+0000 7f7af6e40640 1 -- 172.21.15.40:0/1856593229 shutdown_connections 2023-10-18T09:09:54.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.347+0000 7f7af6e40640 1 --2- 172.21.15.40:0/1856593229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af00b2110 0x7f7af00b0670 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.347+0000 7f7af6e40640 1 --2- 172.21.15.40:0/1856593229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7af006c6b0 0x7f7af00b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.347+0000 7f7af6e40640 1 -- 172.21.15.40:0/1856593229 >> 172.21.15.40:0/1856593229 conn(0x7f7af00aeb90 msgr2=0x7f7af00aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:54.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.347+0000 7f7af6e40640 1 -- 172.21.15.40:0/1856593229 shutdown_connections 2023-10-18T09:09:54.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.347+0000 7f7af6e40640 1 -- 172.21.15.40:0/1856593229 wait complete. 2023-10-18T09:09:54.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.349+0000 7f7af6e40640 1 Processor -- start 2023-10-18T09:09:54.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.349+0000 7f7af6e40640 1 -- start start 2023-10-18T09:09:54.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.350+0000 7f7af6e40640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7af006c6b0 0x7f7af013a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:54.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.350+0000 7f7af6e40640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af00b2110 0x7f7af0149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:54.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.350+0000 7f7af6e40640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7af0155350 0x7f7af014d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:54.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.350+0000 7f7af6e40640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af0148660 con 0x7f7af00b2110 2023-10-18T09:09:54.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.350+0000 7f7af6e40640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af0148940 con 0x7f7af0155350 2023-10-18T09:09:54.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.350+0000 7f7af6e40640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af0148c20 con 0x7f7af006c6b0 2023-10-18T09:09:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.350+0000 7f7af563d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af00b2110 0x7f7af0149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.350+0000 7f7af563d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af00b2110 0x7f7af0149700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54260/0 (socket says 172.21.15.40:54260) 2023-10-18T09:09:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.350+0000 7f7af563d640 1 -- 172.21.15.40:0/918283093 learned_addr learned my addr 172.21.15.40:0/918283093 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.351+0000 7f7af5e3e640 1 --2- 172.21.15.40:0/918283093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7af006c6b0 0x7f7af013a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.351+0000 7f7af663f640 1 --2- 172.21.15.40:0/918283093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7af0155350 0x7f7af014d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:54.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.351+0000 7f7af563d640 1 -- 172.21.15.40:0/918283093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7af006c6b0 msgr2=0x7f7af013a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:54.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.351+0000 7f7af563d640 1 --2- 172.21.15.40:0/918283093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7af006c6b0 0x7f7af013a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.351+0000 7f7af563d640 1 -- 172.21.15.40:0/918283093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7af0155350 msgr2=0x7f7af014d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:54.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.351+0000 7f7af563d640 1 --2- 172.21.15.40:0/918283093 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7af0155350 0x7f7af014d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.351+0000 7f7af563d640 1 -- 172.21.15.40:0/918283093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae800a950 con 0x7f7af00b2110 2023-10-18T09:09:54.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.351+0000 7f7af5e3e640 1 --2- 172.21.15.40:0/918283093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7af006c6b0 0x7f7af013a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:54.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.351+0000 7f7af563d640 1 --2- 172.21.15.40:0/918283093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af00b2110 0x7f7af0149700 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000c920 tx=0x7f7ae000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:54.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.351+0000 7f7ae67fc640 1 -- 172.21.15.40:0/918283093 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0007c70 con 0x7f7af00b2110 2023-10-18T09:09:54.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.352+0000 7f7ae67fc640 1 -- 172.21.15.40:0/918283093 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7ae000f450 con 0x7f7af00b2110 2023-10-18T09:09:54.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.352+0000 7f7af6e40640 1 -- 172.21.15.40:0/918283093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af0148f00 con 0x7f7af00b2110 2023-10-18T09:09:54.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.352+0000 7f7ae67fc640 1 -- 172.21.15.40:0/918283093 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae000f860 con 0x7f7af00b2110 2023-10-18T09:09:54.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.352+0000 7f7af6e40640 1 -- 172.21.15.40:0/918283093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af0147d90 con 0x7f7af00b2110 2023-10-18T09:09:54.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.352+0000 7f7af6e40640 1 -- 172.21.15.40:0/918283093 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7af0066e00 con 0x7f7af00b2110 2023-10-18T09:09:54.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.354+0000 7f7ae67fc640 1 -- 172.21.15.40:0/918283093 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7ae000f9c0 con 0x7f7af00b2110 2023-10-18T09:09:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.355+0000 7f7ae67fc640 1 --2- 172.21.15.40:0/918283093 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ad406ae40 0x7f7ad406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.355+0000 7f7ae67fc640 1 -- 172.21.15.40:0/918283093 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7ae0089c40 con 0x7f7af00b2110 2023-10-18T09:09:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.355+0000 7f7ae67fc640 1 --2- 172.21.15.40:0/918283093 >> [v2:172.21.15.40:6808/2762063371,v1:172.21.15.40:6809/2762063371] conn(0x7f7ad4070ca0 0x7f7ad40730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.355+0000 7f7ae67fc640 1 -- 172.21.15.40:0/918283093 --> [v2:172.21.15.40:6808/2762063371,v1:172.21.15.40:6809/2762063371] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7ad4073770 con 0x7f7ad4070ca0 2023-10-18T09:09:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.355+0000 7f7ae67fc640 1 -- 172.21.15.40:0/918283093 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ae008e240 con 0x7f7af00b2110 2023-10-18T09:09:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.355+0000 7f7af663f640 1 --2- 172.21.15.40:0/918283093 >> [v2:172.21.15.40:6808/2762063371,v1:172.21.15.40:6809/2762063371] conn(0x7f7ad4070ca0 0x7f7ad40730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.356+0000 7f7af5e3e640 1 --2- 172.21.15.40:0/918283093 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ad406ae40 0x7f7ad406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.356+0000 7f7af663f640 1 --2- 172.21.15.40:0/918283093 >> [v2:172.21.15.40:6808/2762063371,v1:172.21.15.40:6809/2762063371] conn(0x7f7ad4070ca0 0x7f7ad40730c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.357+0000 7f7ae67fc640 1 -- 172.21.15.40:0/918283093 <== osd.1 v2:172.21.15.40:6808/2762063371 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f7ad4073770 con 0x7f7ad4070ca0 2023-10-18T09:09:54.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.357+0000 7f7af5e3e640 1 --2- 172.21.15.40:0/918283093 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ad406ae40 0x7f7ad406d300 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7aec001630 tx=0x7f7aec009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:54.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.369+0000 7f7af6e40640 1 -- 172.21.15.40:0/918283093 --> [v2:172.21.15.40:6808/2762063371,v1:172.21.15.40:6809/2762063371] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7af006d4b0 con 0x7f7ad4070ca0 2023-10-18T09:09:54.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.370+0000 7f7ae67fc640 1 -- 172.21.15.40:0/918283093 <== osd.1 v2:172.21.15.40:6808/2762063371 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f7af006d4b0 con 0x7f7ad4070ca0 2023-10-18T09:09:54.372 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:09:54.372 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-18T09:09:54.373 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:09:54.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.370+0000 7f7af6e40640 1 -- 172.21.15.40:0/918283093 >> [v2:172.21.15.40:6808/2762063371,v1:172.21.15.40:6809/2762063371] conn(0x7f7ad4070ca0 msgr2=0x7f7ad40730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:54.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.370+0000 7f7af6e40640 1 --2- 172.21.15.40:0/918283093 >> [v2:172.21.15.40:6808/2762063371,v1:172.21.15.40:6809/2762063371] conn(0x7f7ad4070ca0 0x7f7ad40730c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.370+0000 7f7af6e40640 1 -- 172.21.15.40:0/918283093 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ad406ae40 msgr2=0x7f7ad406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:54.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.370+0000 7f7af6e40640 1 --2- 172.21.15.40:0/918283093 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7ad406ae40 0x7f7ad406d300 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7aec001630 tx=0x7f7aec009040 comp rx=0 tx=0).stop 2023-10-18T09:09:54.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.370+0000 7f7af6e40640 1 -- 172.21.15.40:0/918283093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af00b2110 msgr2=0x7f7af0149700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:54.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.370+0000 7f7af6e40640 1 --2- 172.21.15.40:0/918283093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7af00b2110 0x7f7af0149700 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000c920 tx=0x7f7ae000cde0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.370+0000 7f7af663f640 1 -- 172.21.15.40:0/918283093 reap_dead start 2023-10-18T09:09:54.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.370+0000 7f7af6e40640 1 -- 172.21.15.40:0/918283093 shutdown_connections 2023-10-18T09:09:54.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.370+0000 7f7af6e40640 1 -- 172.21.15.40:0/918283093 >> 172.21.15.40:0/918283093 conn(0x7f7af00aeb90 msgr2=0x7f7af006fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:54.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.370+0000 7f7af6e40640 1 -- 172.21.15.40:0/918283093 shutdown_connections 2023-10-18T09:09:54.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.371+0000 7f7af6e40640 1 -- 172.21.15.40:0/918283093 wait complete. 2023-10-18T09:09:54.387 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-18T09:09:54.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.615+0000 7f7e68cfc640 1 -- 172.21.15.40:0/464799878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e6413f5a0 msgr2=0x7f7e640c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:54.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.615+0000 7f7e68cfc640 1 --2- 172.21.15.40:0/464799878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e6413f5a0 0x7f7e640c9b00 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009830 tx=0x7f7e5802bdf0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.615+0000 7f7e68cfc640 1 -- 172.21.15.40:0/464799878 shutdown_connections 2023-10-18T09:09:54.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.615+0000 7f7e68cfc640 1 --2- 172.21.15.40:0/464799878 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e6413f5a0 0x7f7e640c9b00 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.615+0000 7f7e68cfc640 1 -- 172.21.15.40:0/464799878 >> 172.21.15.40:0/464799878 conn(0x7f7e6406cee0 msgr2=0x7f7e64069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.615+0000 7f7e68cfc640 1 -- 172.21.15.40:0/464799878 shutdown_connections 2023-10-18T09:09:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.615+0000 7f7e68cfc640 1 -- 172.21.15.40:0/464799878 wait complete. 2023-10-18T09:09:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.617+0000 7f7e68cfc640 1 Processor -- start 2023-10-18T09:09:54.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.617+0000 7f7e68cfc640 1 -- start start 2023-10-18T09:09:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.618+0000 7f7e68cfc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e640ca040 0x7f7e640ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.618+0000 7f7e68cfc640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e6413c470 0x7f7e6413b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.618+0000 7f7e68cfc640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e6413f5a0 0x7f7e6413d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.618+0000 7f7e68cfc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e640aa880 con 0x7f7e640ca040 2023-10-18T09:09:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.618+0000 7f7e68cfc640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e640ae3e0 con 0x7f7e6413f5a0 2023-10-18T09:09:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.618+0000 7f7e68cfc640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e64071520 con 0x7f7e6413c470 2023-10-18T09:09:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.619+0000 7f7e6259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e640ca040 0x7f7e640ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.619+0000 7f7e6259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e640ca040 0x7f7e640ac930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54278/0 (socket says 172.21.15.40:54278) 2023-10-18T09:09:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.619+0000 7f7e6259c640 1 -- 172.21.15.40:0/3805242761 learned_addr learned my addr 172.21.15.40:0/3805242761 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:54.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.619+0000 7f7e62d9d640 1 --2- 172.21.15.40:0/3805242761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e6413f5a0 0x7f7e6413d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:54.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.619+0000 7f7e61d9b640 1 --2- 172.21.15.40:0/3805242761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e6413c470 0x7f7e6413b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:54.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.619+0000 7f7e6259c640 1 -- 172.21.15.40:0/3805242761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e6413c470 msgr2=0x7f7e6413b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:54.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.619+0000 7f7e6259c640 1 --2- 172.21.15.40:0/3805242761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e6413c470 0x7f7e6413b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.619+0000 7f7e6259c640 1 -- 172.21.15.40:0/3805242761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e6413f5a0 msgr2=0x7f7e6413d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:54.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.619+0000 7f7e6259c640 1 --2- 172.21.15.40:0/3805242761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e6413f5a0 0x7f7e6413d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.619+0000 7f7e6259c640 1 -- 172.21.15.40:0/3805242761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e58009490 con 0x7f7e640ca040 2023-10-18T09:09:54.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.620+0000 7f7e6259c640 1 --2- 172.21.15.40:0/3805242761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e640ca040 0x7f7e640ac930 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f7e5400c450 tx=0x7f7e5400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:54.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.620+0000 7f7e52ffd640 1 -- 172.21.15.40:0/3805242761 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e540052d0 con 0x7f7e640ca040 2023-10-18T09:09:54.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.620+0000 7f7e52ffd640 1 -- 172.21.15.40:0/3805242761 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7e54005430 con 0x7f7e640ca040 2023-10-18T09:09:54.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.620+0000 7f7e52ffd640 1 -- 172.21.15.40:0/3805242761 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e54012890 con 0x7f7e640ca040 2023-10-18T09:09:54.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.620+0000 7f7e68cfc640 1 -- 172.21.15.40:0/3805242761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e640ace70 con 0x7f7e640ca040 2023-10-18T09:09:54.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.620+0000 7f7e68cfc640 1 -- 172.21.15.40:0/3805242761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e640704f0 con 0x7f7e640ca040 2023-10-18T09:09:54.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.621+0000 7f7e68cfc640 1 -- 172.21.15.40:0/3805242761 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e64066fd0 con 0x7f7e640ca040 2023-10-18T09:09:54.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.622+0000 7f7e52ffd640 1 -- 172.21.15.40:0/3805242761 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7e540035f0 con 0x7f7e640ca040 2023-10-18T09:09:54.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.624+0000 7f7e52ffd640 1 --2- 172.21.15.40:0/3805242761 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7e4406ae70 0x7f7e4406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:54.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.624+0000 7f7e52ffd640 1 -- 172.21.15.40:0/3805242761 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7e54088f70 con 0x7f7e640ca040 2023-10-18T09:09:54.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.625+0000 7f7e61d9b640 1 --2- 172.21.15.40:0/3805242761 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7e4406ae70 0x7f7e4406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:54.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.626+0000 7f7e61d9b640 1 --2- 172.21.15.40:0/3805242761 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7e4406ae70 0x7f7e4406d330 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7e58036000 tx=0x7f7e58004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:54.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.628+0000 7f7e52ffd640 1 -- 172.21.15.40:0/3805242761 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e54054310 con 0x7f7e640ca040 2023-10-18T09:09:54.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.819+0000 7f7e68cfc640 1 -- 172.21.15.40:0/3805242761 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e640cc3f0 con 0x7f7e4406ae70 2023-10-18T09:09:54.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.820+0000 7f7e52ffd640 1 -- 172.21.15.40:0/3805242761 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7e58004140 con 0x7f7e4406ae70 2023-10-18T09:09:54.823 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 1 to deep-scrub 2023-10-18T09:09:54.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.823+0000 7f7e68cfc640 1 -- 172.21.15.40:0/3805242761 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7e4406ae70 msgr2=0x7f7e4406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:54.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.823+0000 7f7e68cfc640 1 --2- 172.21.15.40:0/3805242761 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7e4406ae70 0x7f7e4406d330 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7e58036000 tx=0x7f7e58004730 comp rx=0 tx=0).stop 2023-10-18T09:09:54.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.823+0000 7f7e68cfc640 1 -- 172.21.15.40:0/3805242761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e640ca040 msgr2=0x7f7e640ac930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:54.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.823+0000 7f7e68cfc640 1 --2- 172.21.15.40:0/3805242761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e640ca040 0x7f7e640ac930 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f7e5400c450 tx=0x7f7e5400c910 comp rx=0 tx=0).stop 2023-10-18T09:09:54.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.823+0000 7f7e68cfc640 1 -- 172.21.15.40:0/3805242761 shutdown_connections 2023-10-18T09:09:54.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.823+0000 7f7e68cfc640 1 --2- 172.21.15.40:0/3805242761 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f7e4406ae70 0x7f7e4406d330 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.823+0000 7f7e68cfc640 1 --2- 172.21.15.40:0/3805242761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e6413f5a0 0x7f7e6413d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.823+0000 7f7e68cfc640 1 --2- 172.21.15.40:0/3805242761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e6413c470 0x7f7e6413b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.823+0000 7f7e68cfc640 1 --2- 172.21.15.40:0/3805242761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e640ca040 0x7f7e640ac930 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:54.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.823+0000 7f7e68cfc640 1 -- 172.21.15.40:0/3805242761 >> 172.21.15.40:0/3805242761 conn(0x7f7e6406cee0 msgr2=0x7f7e640aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:54.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.823+0000 7f7e68cfc640 1 -- 172.21.15.40:0/3805242761 shutdown_connections 2023-10-18T09:09:54.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:54.823+0000 7f7e68cfc640 1 -- 172.21.15.40:0/3805242761 wait complete. 2023-10-18T09:09:54.846 INFO:tasks.ceph:Scrubbing osd.2 2023-10-18T09:09:54.846 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-18T09:09:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.080+0000 7f51a8cf2640 1 -- 172.21.15.40:0/2457399970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5190004ab0 msgr2=0x7f5190004ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.080+0000 7f51a8cf2640 1 --2- 172.21.15.40:0/2457399970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5190004ab0 0x7f5190004ed0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f5194009b20 tx=0x7f5190006900 comp rx=0 tx=0).stop 2023-10-18T09:09:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.080+0000 7f51a2d9d640 1 -- 172.21.15.40:0/2457399970 reap_dead start 2023-10-18T09:09:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.080+0000 7f51a8cf2640 1 -- 172.21.15.40:0/2457399970 shutdown_connections 2023-10-18T09:09:55.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.080+0000 7f51a8cf2640 1 -- 172.21.15.40:0/2457399970 >> 172.21.15.40:0/2457399970 conn(0x7f51a40aec00 msgr2=0x7f51a40aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:55.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.080+0000 7f51a8cf2640 1 -- 172.21.15.40:0/2457399970 shutdown_connections 2023-10-18T09:09:55.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.080+0000 7f51a8cf2640 1 -- 172.21.15.40:0/2457399970 wait complete. 2023-10-18T09:09:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.082+0000 7f51a8cf2640 1 Processor -- start 2023-10-18T09:09:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.082+0000 7f51a8cf2640 1 -- start start 2023-10-18T09:09:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.083+0000 7f51a8cf2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5190004ab0 0x7f51a4153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.083+0000 7f51a8cf2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51a4152390 0x7f51a4151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.083+0000 7f51a8cf2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a40c40e0 0x7f51a40c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.083+0000 7f51a8cf2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a40c4aa0 con 0x7f51a40c40e0 2023-10-18T09:09:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.083+0000 7f51a8cf2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a4146470 con 0x7f5190004ab0 2023-10-18T09:09:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.083+0000 7f51a8cf2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a40c7160 con 0x7f51a4152390 2023-10-18T09:09:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.084+0000 7f51a2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a40c40e0 0x7f51a40c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.084+0000 7f51a2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a40c40e0 0x7f51a40c4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37892/0 (socket says 172.21.15.40:37892) 2023-10-18T09:09:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.084+0000 7f51a2d9d640 1 -- 172.21.15.40:0/3302533007 learned_addr learned my addr 172.21.15.40:0/3302533007 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.084+0000 7f51a259c640 1 --2- 172.21.15.40:0/3302533007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5190004ab0 0x7f51a4153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.084+0000 7f51a1d9b640 1 --2- 172.21.15.40:0/3302533007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51a4152390 0x7f51a4151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.084+0000 7f51a2d9d640 1 -- 172.21.15.40:0/3302533007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51a4152390 msgr2=0x7f51a4151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.084+0000 7f51a2d9d640 1 --2- 172.21.15.40:0/3302533007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51a4152390 0x7f51a4151b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.084+0000 7f51a2d9d640 1 -- 172.21.15.40:0/3302533007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5190004ab0 msgr2=0x7f51a4153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.084+0000 7f51a2d9d640 1 --2- 172.21.15.40:0/3302533007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5190004ab0 0x7f51a4153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.084+0000 7f51a2d9d640 1 -- 172.21.15.40:0/3302533007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a41543a0 con 0x7f51a40c40e0 2023-10-18T09:09:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.084+0000 7f51a259c640 1 --2- 172.21.15.40:0/3302533007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5190004ab0 0x7f51a4153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:55.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.084+0000 7f51a2d9d640 1 --2- 172.21.15.40:0/3302533007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a40c40e0 0x7f51a40c4560 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f51a40b3160 tx=0x7f519c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.085+0000 7f519affd640 1 -- 172.21.15.40:0/3302533007 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c007340 con 0x7f51a40c40e0 2023-10-18T09:09:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.085+0000 7f519affd640 1 -- 172.21.15.40:0/3302533007 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f519c007590 con 0x7f51a40c40e0 2023-10-18T09:09:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.085+0000 7f519affd640 1 -- 172.21.15.40:0/3302533007 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c005a20 con 0x7f51a40c40e0 2023-10-18T09:09:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.085+0000 7f51a8cf2640 1 -- 172.21.15.40:0/3302533007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a40c6e80 con 0x7f51a40c40e0 2023-10-18T09:09:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.085+0000 7f51a8cf2640 1 -- 172.21.15.40:0/3302533007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a4158a90 con 0x7f51a40c40e0 2023-10-18T09:09:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.086+0000 7f51a8cf2640 1 -- 172.21.15.40:0/3302533007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f51a4066fd0 con 0x7f51a40c40e0 2023-10-18T09:09:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.087+0000 7f519affd640 1 -- 172.21.15.40:0/3302533007 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f519c005b80 con 0x7f51a40c40e0 2023-10-18T09:09:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.088+0000 7f519affd640 1 --2- 172.21.15.40:0/3302533007 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f518406ae70 0x7f518406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.088+0000 7f519affd640 1 -- 172.21.15.40:0/3302533007 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f519c088210 con 0x7f51a40c40e0 2023-10-18T09:09:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.088+0000 7f519affd640 1 --2- 172.21.15.40:0/3302533007 >> [v2:172.21.15.40:6823/1187944209,v1:172.21.15.40:6825/1187944209] conn(0x7f5184070cd0 0x7f51840730f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.088+0000 7f519affd640 1 -- 172.21.15.40:0/3302533007 --> [v2:172.21.15.40:6823/1187944209,v1:172.21.15.40:6825/1187944209] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f51840737a0 con 0x7f5184070cd0 2023-10-18T09:09:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.088+0000 7f519affd640 1 -- 172.21.15.40:0/3302533007 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f519c08a7f0 con 0x7f51a40c40e0 2023-10-18T09:09:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.089+0000 7f51a1d9b640 1 --2- 172.21.15.40:0/3302533007 >> [v2:172.21.15.40:6823/1187944209,v1:172.21.15.40:6825/1187944209] conn(0x7f5184070cd0 0x7f51840730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.089+0000 7f51a259c640 1 --2- 172.21.15.40:0/3302533007 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f518406ae70 0x7f518406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.089+0000 7f51a1d9b640 1 --2- 172.21.15.40:0/3302533007 >> [v2:172.21.15.40:6823/1187944209,v1:172.21.15.40:6825/1187944209] conn(0x7f5184070cd0 0x7f51840730f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.090+0000 7f519affd640 1 -- 172.21.15.40:0/3302533007 <== osd.2 v2:172.21.15.40:6823/1187944209 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f51840737a0 con 0x7f5184070cd0 2023-10-18T09:09:55.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.090+0000 7f51a259c640 1 --2- 172.21.15.40:0/3302533007 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f518406ae70 0x7f518406d330 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5194036000 tx=0x7f5194035040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:55.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.103+0000 7f51a8cf2640 1 -- 172.21.15.40:0/3302533007 --> [v2:172.21.15.40:6823/1187944209,v1:172.21.15.40:6825/1187944209] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f51a40ae5c0 con 0x7f5184070cd0 2023-10-18T09:09:55.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.103+0000 7f519affd640 1 -- 172.21.15.40:0/3302533007 <== osd.2 v2:172.21.15.40:6823/1187944209 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f51a40ae5c0 con 0x7f5184070cd0 2023-10-18T09:09:55.106 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:09:55.106 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-18T09:09:55.106 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:09:55.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.104+0000 7f51a8cf2640 1 -- 172.21.15.40:0/3302533007 >> [v2:172.21.15.40:6823/1187944209,v1:172.21.15.40:6825/1187944209] conn(0x7f5184070cd0 msgr2=0x7f51840730f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.104+0000 7f51a8cf2640 1 --2- 172.21.15.40:0/3302533007 >> [v2:172.21.15.40:6823/1187944209,v1:172.21.15.40:6825/1187944209] conn(0x7f5184070cd0 0x7f51840730f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.104+0000 7f51a8cf2640 1 -- 172.21.15.40:0/3302533007 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f518406ae70 msgr2=0x7f518406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.104+0000 7f51a8cf2640 1 --2- 172.21.15.40:0/3302533007 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f518406ae70 0x7f518406d330 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5194036000 tx=0x7f5194035040 comp rx=0 tx=0).stop 2023-10-18T09:09:55.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.104+0000 7f51a8cf2640 1 -- 172.21.15.40:0/3302533007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a40c40e0 msgr2=0x7f51a40c4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.104+0000 7f51a8cf2640 1 --2- 172.21.15.40:0/3302533007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51a40c40e0 0x7f51a40c4560 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f51a40b3160 tx=0x7f519c00c820 comp rx=0 tx=0).stop 2023-10-18T09:09:55.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.104+0000 7f51a2d9d640 1 -- 172.21.15.40:0/3302533007 reap_dead start 2023-10-18T09:09:55.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.104+0000 7f51a8cf2640 1 -- 172.21.15.40:0/3302533007 shutdown_connections 2023-10-18T09:09:55.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.104+0000 7f51a8cf2640 1 -- 172.21.15.40:0/3302533007 >> 172.21.15.40:0/3302533007 conn(0x7f51a40aec00 msgr2=0x7f51a40c3680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:55.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.104+0000 7f51a8cf2640 1 -- 172.21.15.40:0/3302533007 shutdown_connections 2023-10-18T09:09:55.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.104+0000 7f51a8cf2640 1 -- 172.21.15.40:0/3302533007 wait complete. 2023-10-18T09:09:55.124 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-18T09:09:55.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.351+0000 7f15a31ce640 1 -- 172.21.15.40:0/4032896079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f159c0b12d0 msgr2=0x7f159c1625a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.351+0000 7f15a31ce640 1 --2- 172.21.15.40:0/4032896079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f159c0b12d0 0x7f159c1625a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1594005fd0 tx=0x7f159402c2d0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.351+0000 7f15a31ce640 1 -- 172.21.15.40:0/4032896079 shutdown_connections 2023-10-18T09:09:55.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.351+0000 7f15a31ce640 1 --2- 172.21.15.40:0/4032896079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f159c0b12d0 0x7f159c1625a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.351+0000 7f15a31ce640 1 --2- 172.21.15.40:0/4032896079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f159c06d710 0x7f159c151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.351+0000 7f15a31ce640 1 -- 172.21.15.40:0/4032896079 >> 172.21.15.40:0/4032896079 conn(0x7f159c06cee0 msgr2=0x7f159c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:55.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.352+0000 7f15a31ce640 1 -- 172.21.15.40:0/4032896079 shutdown_connections 2023-10-18T09:09:55.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.352+0000 7f15a31ce640 1 -- 172.21.15.40:0/4032896079 wait complete. 2023-10-18T09:09:55.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.353+0000 7f15a31ce640 1 Processor -- start 2023-10-18T09:09:55.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.353+0000 7f15a31ce640 1 -- start start 2023-10-18T09:09:55.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.354+0000 7f15a31ce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f159c06d710 0x7f159c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:55.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.354+0000 7f15a31ce640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f159c0b12d0 0x7f159c14c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.354+0000 7f15a31ce640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f159c1586a0 0x7f159c13aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.354+0000 7f15a31ce640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c14d010 con 0x7f159c06d710 2023-10-18T09:09:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.354+0000 7f15a31ce640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c14cd40 con 0x7f159c0b12d0 2023-10-18T09:09:55.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.354+0000 7f15a31ce640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159c14e430 con 0x7f159c1586a0 2023-10-18T09:09:55.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.355+0000 7f15a0f6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f159c06d710 0x7f159c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:55.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.355+0000 7f15a176b640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f159c1586a0 0x7f159c13aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:55.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.355+0000 7f15a0f6a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f159c06d710 0x7f159c06b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37894/0 (socket says 172.21.15.40:37894) 2023-10-18T09:09:55.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.355+0000 7f15a0f6a640 1 -- 172.21.15.40:0/3999997361 learned_addr learned my addr 172.21.15.40:0/3999997361 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:55.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.355+0000 7f1593fff640 1 --2- 172.21.15.40:0/3999997361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f159c0b12d0 0x7f159c14c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:55.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.356+0000 7f15a0f6a640 1 -- 172.21.15.40:0/3999997361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f159c1586a0 msgr2=0x7f159c13aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.356+0000 7f15a0f6a640 1 --2- 172.21.15.40:0/3999997361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f159c1586a0 0x7f159c13aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.356+0000 7f15a0f6a640 1 -- 172.21.15.40:0/3999997361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f159c0b12d0 msgr2=0x7f159c14c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.356+0000 7f15a0f6a640 1 --2- 172.21.15.40:0/3999997361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f159c0b12d0 0x7f159c14c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.356+0000 7f15a0f6a640 1 -- 172.21.15.40:0/3999997361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1594009900 con 0x7f159c06d710 2023-10-18T09:09:55.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.356+0000 7f15a0f6a640 1 --2- 172.21.15.40:0/3999997361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f159c06d710 0x7f159c06b970 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f159800e9f0 tx=0x7f159800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:55.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.356+0000 7f15917fa640 1 -- 172.21.15.40:0/3999997361 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598009700 con 0x7f159c06d710 2023-10-18T09:09:55.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.356+0000 7f15917fa640 1 -- 172.21.15.40:0/3999997361 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f159800ce70 con 0x7f159c06d710 2023-10-18T09:09:55.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.356+0000 7f15a31ce640 1 -- 172.21.15.40:0/3999997361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159c14e710 con 0x7f159c06d710 2023-10-18T09:09:55.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.356+0000 7f15917fa640 1 -- 172.21.15.40:0/3999997361 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598010670 con 0x7f159c06d710 2023-10-18T09:09:55.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.356+0000 7f15a31ce640 1 -- 172.21.15.40:0/3999997361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159c14ecd0 con 0x7f159c06d710 2023-10-18T09:09:55.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.357+0000 7f15a31ce640 1 -- 172.21.15.40:0/3999997361 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159c066fd0 con 0x7f159c06d710 2023-10-18T09:09:55.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.359+0000 7f15917fa640 1 -- 172.21.15.40:0/3999997361 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f15980107d0 con 0x7f159c06d710 2023-10-18T09:09:55.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.360+0000 7f15917fa640 1 --2- 172.21.15.40:0/3999997361 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f158806ae40 0x7f158806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:55.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.360+0000 7f15917fa640 1 -- 172.21.15.40:0/3999997361 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1598014070 con 0x7f159c06d710 2023-10-18T09:09:55.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.360+0000 7f1593fff640 1 --2- 172.21.15.40:0/3999997361 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f158806ae40 0x7f158806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:55.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.362+0000 7f1593fff640 1 --2- 172.21.15.40:0/3999997361 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f158806ae40 0x7f158806d300 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f159c161af0 tx=0x7f158c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:55.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.364+0000 7f15917fa640 1 -- 172.21.15.40:0/3999997361 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1598051ca0 con 0x7f159c06d710 2023-10-18T09:09:55.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.555+0000 7f15a31ce640 1 -- 172.21.15.40:0/3999997361 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f159c14e5a0 con 0x7f158806ae40 2023-10-18T09:09:55.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.556+0000 7f15917fa640 1 -- 172.21.15.40:0/3999997361 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f159c14e5a0 con 0x7f158806ae40 2023-10-18T09:09:55.559 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 2 to deep-scrub 2023-10-18T09:09:55.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.559+0000 7f15a31ce640 1 -- 172.21.15.40:0/3999997361 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f158806ae40 msgr2=0x7f158806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.559+0000 7f15a31ce640 1 --2- 172.21.15.40:0/3999997361 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f158806ae40 0x7f158806d300 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f159c161af0 tx=0x7f158c009040 comp rx=0 tx=0).stop 2023-10-18T09:09:55.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.559+0000 7f15a31ce640 1 -- 172.21.15.40:0/3999997361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f159c06d710 msgr2=0x7f159c06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.559+0000 7f15a31ce640 1 --2- 172.21.15.40:0/3999997361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f159c06d710 0x7f159c06b970 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f159800e9f0 tx=0x7f159800eeb0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.559+0000 7f15a31ce640 1 -- 172.21.15.40:0/3999997361 shutdown_connections 2023-10-18T09:09:55.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.559+0000 7f15a31ce640 1 --2- 172.21.15.40:0/3999997361 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f158806ae40 0x7f158806d300 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.559+0000 7f15a31ce640 1 --2- 172.21.15.40:0/3999997361 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f159c1586a0 0x7f159c13aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.559+0000 7f15a31ce640 1 --2- 172.21.15.40:0/3999997361 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f159c0b12d0 0x7f159c14c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.559+0000 7f15a31ce640 1 --2- 172.21.15.40:0/3999997361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f159c06d710 0x7f159c06b970 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.559+0000 7f15a31ce640 1 -- 172.21.15.40:0/3999997361 >> 172.21.15.40:0/3999997361 conn(0x7f159c06cee0 msgr2=0x7f159c0b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:55.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.559+0000 7f15a31ce640 1 -- 172.21.15.40:0/3999997361 shutdown_connections 2023-10-18T09:09:55.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.559+0000 7f15a31ce640 1 -- 172.21.15.40:0/3999997361 wait complete. 2023-10-18T09:09:55.580 INFO:tasks.ceph:Scrubbing osd.3 2023-10-18T09:09:55.580 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-18T09:09:55.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.805+0000 7f608759e640 1 -- 172.21.15.40:0/2765117635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60880b20a0 msgr2=0x7f60880ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.805+0000 7f608759e640 1 --2- 172.21.15.40:0/2765117635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60880b20a0 0x7f60880ccd20 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f6070009900 tx=0x7f607002bd70 comp rx=0 tx=0).stop 2023-10-18T09:09:55.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.805+0000 7f608759e640 1 -- 172.21.15.40:0/2765117635 shutdown_connections 2023-10-18T09:09:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.805+0000 7f608759e640 1 --2- 172.21.15.40:0/2765117635 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60880c8a50 0x7f60880cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.805+0000 7f608759e640 1 --2- 172.21.15.40:0/2765117635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60880b20a0 0x7f60880ccd20 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.805+0000 7f608759e640 1 -- 172.21.15.40:0/2765117635 >> 172.21.15.40:0/2765117635 conn(0x7f608809a5b0 msgr2=0x7f60880d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.805+0000 7f608759e640 1 -- 172.21.15.40:0/2765117635 shutdown_connections 2023-10-18T09:09:55.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.805+0000 7f608759e640 1 -- 172.21.15.40:0/2765117635 wait complete. 2023-10-18T09:09:55.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.807+0000 7f608759e640 1 Processor -- start 2023-10-18T09:09:55.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.807+0000 7f608759e640 1 -- start start 2023-10-18T09:09:55.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.808+0000 7f608759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60880b20a0 0x7f6088150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:55.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.808+0000 7f608759e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60880c8a50 0x7f608814fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:55.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.808+0000 7f608759e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f608815ebe0 0x7f608815b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:55.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.808+0000 7f608759e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60880c7280 con 0x7f60880b20a0 2023-10-18T09:09:55.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.808+0000 7f608759e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60880c7520 con 0x7f60880c8a50 2023-10-18T09:09:55.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.808+0000 7f608759e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60880c7800 con 0x7f608815ebe0 2023-10-18T09:09:55.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.809+0000 7f608659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60880b20a0 0x7f6088150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:55.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.809+0000 7f608659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60880b20a0 0x7f6088150510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37914/0 (socket says 172.21.15.40:37914) 2023-10-18T09:09:55.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.809+0000 7f608659c640 1 -- 172.21.15.40:0/3079662109 learned_addr learned my addr 172.21.15.40:0/3079662109 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:55.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.809+0000 7f6085d9b640 1 --2- 172.21.15.40:0/3079662109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60880c8a50 0x7f608814fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:55.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.809+0000 7f6086d9d640 1 --2- 172.21.15.40:0/3079662109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f608815ebe0 0x7f608815b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:55.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.809+0000 7f608659c640 1 -- 172.21.15.40:0/3079662109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f608815ebe0 msgr2=0x7f608815b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.809+0000 7f608659c640 1 --2- 172.21.15.40:0/3079662109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f608815ebe0 0x7f608815b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.809+0000 7f608659c640 1 -- 172.21.15.40:0/3079662109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60880c8a50 msgr2=0x7f608814fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.809+0000 7f608659c640 1 --2- 172.21.15.40:0/3079662109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60880c8a50 0x7f608814fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.809+0000 7f608659c640 1 -- 172.21.15.40:0/3079662109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6070009560 con 0x7f60880b20a0 2023-10-18T09:09:55.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.809+0000 7f6085d9b640 1 --2- 172.21.15.40:0/3079662109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60880c8a50 0x7f608814fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:55.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.809+0000 7f6086d9d640 1 --2- 172.21.15.40:0/3079662109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f608815ebe0 0x7f608815b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:55.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.810+0000 7f608659c640 1 --2- 172.21.15.40:0/3079662109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60880b20a0 0x7f6088150510 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f607800ba50 tx=0x7f607800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:55.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.810+0000 7f607effd640 1 -- 172.21.15.40:0/3079662109 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607800cbe0 con 0x7f60880b20a0 2023-10-18T09:09:55.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.810+0000 7f607effd640 1 -- 172.21.15.40:0/3079662109 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f607800ce70 con 0x7f60880b20a0 2023-10-18T09:09:55.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.810+0000 7f607effd640 1 -- 172.21.15.40:0/3079662109 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6078014680 con 0x7f60880b20a0 2023-10-18T09:09:55.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.810+0000 7f608759e640 1 -- 172.21.15.40:0/3079662109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60880c7ae0 con 0x7f60880b20a0 2023-10-18T09:09:55.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.810+0000 7f608759e640 1 -- 172.21.15.40:0/3079662109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6088150a50 con 0x7f60880b20a0 2023-10-18T09:09:55.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.811+0000 7f608759e640 1 -- 172.21.15.40:0/3079662109 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f60880c7690 con 0x7f60880b20a0 2023-10-18T09:09:55.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.812+0000 7f607effd640 1 -- 172.21.15.40:0/3079662109 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f60780147e0 con 0x7f60880b20a0 2023-10-18T09:09:55.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.813+0000 7f607effd640 1 --2- 172.21.15.40:0/3079662109 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f606806ae40 0x7f606806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:55.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.813+0000 7f607effd640 1 -- 172.21.15.40:0/3079662109 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6078089570 con 0x7f60880b20a0 2023-10-18T09:09:55.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.813+0000 7f607effd640 1 --2- 172.21.15.40:0/3079662109 >> [v2:172.21.15.40:6816/3903517270,v1:172.21.15.40:6817/3903517270] conn(0x7f6068070ca0 0x7f60680730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:55.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.813+0000 7f607effd640 1 -- 172.21.15.40:0/3079662109 --> [v2:172.21.15.40:6816/3903517270,v1:172.21.15.40:6817/3903517270] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6068073770 con 0x7f6068070ca0 2023-10-18T09:09:55.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.813+0000 7f607effd640 1 -- 172.21.15.40:0/3079662109 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f607808bb50 con 0x7f60880b20a0 2023-10-18T09:09:55.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.814+0000 7f6086d9d640 1 --2- 172.21.15.40:0/3079662109 >> [v2:172.21.15.40:6816/3903517270,v1:172.21.15.40:6817/3903517270] conn(0x7f6068070ca0 0x7f60680730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:55.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.814+0000 7f6085d9b640 1 --2- 172.21.15.40:0/3079662109 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f606806ae40 0x7f606806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:55.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.814+0000 7f6086d9d640 1 --2- 172.21.15.40:0/3079662109 >> [v2:172.21.15.40:6816/3903517270,v1:172.21.15.40:6817/3903517270] conn(0x7f6068070ca0 0x7f60680730c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:55.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.815+0000 7f6085d9b640 1 --2- 172.21.15.40:0/3079662109 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f606806ae40 0x7f606806d300 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6070035000 tx=0x7f6070004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:55.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.815+0000 7f607effd640 1 -- 172.21.15.40:0/3079662109 <== osd.3 v2:172.21.15.40:6816/3903517270 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f6068073770 con 0x7f6068070ca0 2023-10-18T09:09:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.828+0000 7f608759e640 1 -- 172.21.15.40:0/3079662109 --> [v2:172.21.15.40:6816/3903517270,v1:172.21.15.40:6817/3903517270] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f60880ce060 con 0x7f6068070ca0 2023-10-18T09:09:55.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.829+0000 7f607effd640 1 -- 172.21.15.40:0/3079662109 <== osd.3 v2:172.21.15.40:6816/3903517270 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f6080009c10 con 0x7f6068070ca0 2023-10-18T09:09:55.831 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:09:55.831 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-18T09:09:55.831 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:09:55.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.829+0000 7f608759e640 1 -- 172.21.15.40:0/3079662109 >> [v2:172.21.15.40:6816/3903517270,v1:172.21.15.40:6817/3903517270] conn(0x7f6068070ca0 msgr2=0x7f60680730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.829+0000 7f608759e640 1 --2- 172.21.15.40:0/3079662109 >> [v2:172.21.15.40:6816/3903517270,v1:172.21.15.40:6817/3903517270] conn(0x7f6068070ca0 0x7f60680730c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:55.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.829+0000 7f608759e640 1 -- 172.21.15.40:0/3079662109 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f606806ae40 msgr2=0x7f606806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.829+0000 7f608759e640 1 --2- 172.21.15.40:0/3079662109 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f606806ae40 0x7f606806d300 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6070035000 tx=0x7f6070004730 comp rx=0 tx=0).stop 2023-10-18T09:09:55.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.829+0000 7f608759e640 1 -- 172.21.15.40:0/3079662109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60880b20a0 msgr2=0x7f6088150510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:55.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.829+0000 7f608759e640 1 --2- 172.21.15.40:0/3079662109 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60880b20a0 0x7f6088150510 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f607800ba50 tx=0x7f607800bf10 comp rx=0 tx=0).stop 2023-10-18T09:09:55.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.829+0000 7f6086d9d640 1 -- 172.21.15.40:0/3079662109 reap_dead start 2023-10-18T09:09:55.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.830+0000 7f608759e640 1 -- 172.21.15.40:0/3079662109 shutdown_connections 2023-10-18T09:09:55.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.830+0000 7f608759e640 1 -- 172.21.15.40:0/3079662109 >> 172.21.15.40:0/3079662109 conn(0x7f608809a5b0 msgr2=0x7f6088072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:55.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.830+0000 7f608759e640 1 -- 172.21.15.40:0/3079662109 shutdown_connections 2023-10-18T09:09:55.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:55.830+0000 7f608759e640 1 -- 172.21.15.40:0/3079662109 wait complete. 2023-10-18T09:09:55.851 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-18T09:09:56.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.081+0000 7f548894c640 1 -- 172.21.15.40:0/2486622533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5484151bd0 msgr2=0x7f54841586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.081+0000 7f548894c640 1 --2- 172.21.15.40:0/2486622533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5484151bd0 0x7f54841586a0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f5474009830 tx=0x7f547402bd60 comp rx=0 tx=0).stop 2023-10-18T09:09:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.081+0000 7f548894c640 1 -- 172.21.15.40:0/2486622533 shutdown_connections 2023-10-18T09:09:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.081+0000 7f548894c640 1 --2- 172.21.15.40:0/2486622533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f54840b0bc0 0x7f54840b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.081+0000 7f548894c640 1 --2- 172.21.15.40:0/2486622533 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5484151bd0 0x7f54841586a0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.081+0000 7f548894c640 1 -- 172.21.15.40:0/2486622533 >> 172.21.15.40:0/2486622533 conn(0x7f548406cee0 msgr2=0x7f5484069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.081+0000 7f548894c640 1 -- 172.21.15.40:0/2486622533 shutdown_connections 2023-10-18T09:09:56.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.081+0000 7f548894c640 1 -- 172.21.15.40:0/2486622533 wait complete. 2023-10-18T09:09:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.083+0000 7f548894c640 1 Processor -- start 2023-10-18T09:09:56.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.083+0000 7f548894c640 1 -- start start 2023-10-18T09:09:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.084+0000 7f548894c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f548406b710 0x7f5484138a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.084+0000 7f548894c640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54840b0bc0 0x7f548414ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.084+0000 7f548894c640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5484151bd0 0x7f5484145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:56.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.084+0000 7f548894c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5484145630 con 0x7f548406b710 2023-10-18T09:09:56.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.084+0000 7f548894c640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5484144a20 con 0x7f54840b0bc0 2023-10-18T09:09:56.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.084+0000 7f548894c640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5484144d00 con 0x7f5484151bd0 2023-10-18T09:09:56.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.085+0000 7f548259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f548406b710 0x7f5484138a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:56.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.085+0000 7f548259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f548406b710 0x7f5484138a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37922/0 (socket says 172.21.15.40:37922) 2023-10-18T09:09:56.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.085+0000 7f548259c640 1 -- 172.21.15.40:0/2009308951 learned_addr learned my addr 172.21.15.40:0/2009308951 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:56.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.085+0000 7f5481d9b640 1 --2- 172.21.15.40:0/2009308951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54840b0bc0 0x7f548414ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:56.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.085+0000 7f5482d9d640 1 --2- 172.21.15.40:0/2009308951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5484151bd0 0x7f5484145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:56.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.085+0000 7f548259c640 1 -- 172.21.15.40:0/2009308951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5484151bd0 msgr2=0x7f5484145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:56.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.085+0000 7f548259c640 1 --2- 172.21.15.40:0/2009308951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5484151bd0 0x7f5484145ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.085+0000 7f548259c640 1 -- 172.21.15.40:0/2009308951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54840b0bc0 msgr2=0x7f548414ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.085+0000 7f548259c640 1 --2- 172.21.15.40:0/2009308951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54840b0bc0 0x7f548414ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.085+0000 7f548259c640 1 -- 172.21.15.40:0/2009308951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5474009490 con 0x7f548406b710 2023-10-18T09:09:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.085+0000 7f5481d9b640 1 --2- 172.21.15.40:0/2009308951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54840b0bc0 0x7f548414ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:56.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.085+0000 7f548259c640 1 --2- 172.21.15.40:0/2009308951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f548406b710 0x7f5484138a40 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f5474000c00 tx=0x7f54740038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.086+0000 7f5472ffd640 1 -- 172.21.15.40:0/2009308951 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474003ea0 con 0x7f548406b710 2023-10-18T09:09:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.086+0000 7f5472ffd640 1 -- 172.21.15.40:0/2009308951 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5474034440 con 0x7f548406b710 2023-10-18T09:09:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.086+0000 7f5472ffd640 1 -- 172.21.15.40:0/2009308951 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474034850 con 0x7f548406b710 2023-10-18T09:09:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.086+0000 7f548894c640 1 -- 172.21.15.40:0/2009308951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5484144fe0 con 0x7f548406b710 2023-10-18T09:09:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.086+0000 7f548894c640 1 -- 172.21.15.40:0/2009308951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5484138f80 con 0x7f548406b710 2023-10-18T09:09:56.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.087+0000 7f548894c640 1 -- 172.21.15.40:0/2009308951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5484066fd0 con 0x7f548406b710 2023-10-18T09:09:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.088+0000 7f5472ffd640 1 -- 172.21.15.40:0/2009308951 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f54740349b0 con 0x7f548406b710 2023-10-18T09:09:56.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.089+0000 7f5472ffd640 1 --2- 172.21.15.40:0/2009308951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f546406ae40 0x7f546406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:56.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.089+0000 7f5472ffd640 1 -- 172.21.15.40:0/2009308951 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f54740abc70 con 0x7f548406b710 2023-10-18T09:09:56.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.090+0000 7f5481d9b640 1 --2- 172.21.15.40:0/2009308951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f546406ae40 0x7f546406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:56.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.091+0000 7f5481d9b640 1 --2- 172.21.15.40:0/2009308951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f546406ae40 0x7f546406d300 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f54840b01a0 tx=0x7f5478005e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:56.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.094+0000 7f5472ffd640 1 -- 172.21.15.40:0/2009308951 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5474076ef0 con 0x7f548406b710 2023-10-18T09:09:56.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.288+0000 7f548894c640 1 -- 172.21.15.40:0/2009308951 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5484151fb0 con 0x7f546406ae40 2023-10-18T09:09:56.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.290+0000 7f5472ffd640 1 -- 172.21.15.40:0/2009308951 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f547800eb60 con 0x7f546406ae40 2023-10-18T09:09:56.292 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 3 to deep-scrub 2023-10-18T09:09:56.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.292+0000 7f548894c640 1 -- 172.21.15.40:0/2009308951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f546406ae40 msgr2=0x7f546406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:56.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.292+0000 7f548894c640 1 --2- 172.21.15.40:0/2009308951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f546406ae40 0x7f546406d300 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f54840b01a0 tx=0x7f5478005e50 comp rx=0 tx=0).stop 2023-10-18T09:09:56.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.292+0000 7f548894c640 1 -- 172.21.15.40:0/2009308951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f548406b710 msgr2=0x7f5484138a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:56.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.292+0000 7f548894c640 1 --2- 172.21.15.40:0/2009308951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f548406b710 0x7f5484138a40 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f5474000c00 tx=0x7f54740038a0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.292+0000 7f548894c640 1 -- 172.21.15.40:0/2009308951 shutdown_connections 2023-10-18T09:09:56.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.292+0000 7f548894c640 1 --2- 172.21.15.40:0/2009308951 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f546406ae40 0x7f546406d300 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.292+0000 7f548894c640 1 --2- 172.21.15.40:0/2009308951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5484151bd0 0x7f5484145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.292+0000 7f548894c640 1 --2- 172.21.15.40:0/2009308951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f54840b0bc0 0x7f548414ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.292+0000 7f548894c640 1 --2- 172.21.15.40:0/2009308951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f548406b710 0x7f5484138a40 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.292+0000 7f548894c640 1 -- 172.21.15.40:0/2009308951 >> 172.21.15.40:0/2009308951 conn(0x7f548406cee0 msgr2=0x7f5484069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:56.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.293+0000 7f548894c640 1 -- 172.21.15.40:0/2009308951 shutdown_connections 2023-10-18T09:09:56.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.293+0000 7f548894c640 1 -- 172.21.15.40:0/2009308951 wait complete. 2023-10-18T09:09:56.315 INFO:tasks.ceph:Scrubbing osd.4 2023-10-18T09:09:56.315 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-18T09:09:56.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.547+0000 7f717f59e640 1 -- 172.21.15.40:0/1198880344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71800ae6c0 msgr2=0x7f71800b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:56.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.547+0000 7f717f59e640 1 --2- 172.21.15.40:0/1198880344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71800ae6c0 0x7f71800b1b10 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f7178005fd0 tx=0x7f717802bd90 comp rx=0 tx=0).stop 2023-10-18T09:09:56.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.548+0000 7f717f59e640 1 -- 172.21.15.40:0/1198880344 shutdown_connections 2023-10-18T09:09:56.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.548+0000 7f717f59e640 1 --2- 172.21.15.40:0/1198880344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71800ae6c0 0x7f71800b1b10 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.548+0000 7f717f59e640 1 --2- 172.21.15.40:0/1198880344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71800cee50 0x7f71800afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.548+0000 7f717f59e640 1 -- 172.21.15.40:0/1198880344 >> 172.21.15.40:0/1198880344 conn(0x7f718009a570 msgr2=0x7f71800d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:56.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.548+0000 7f717f59e640 1 -- 172.21.15.40:0/1198880344 shutdown_connections 2023-10-18T09:09:56.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.548+0000 7f717f59e640 1 -- 172.21.15.40:0/1198880344 wait complete. 2023-10-18T09:09:56.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.550+0000 7f717f59e640 1 Processor -- start 2023-10-18T09:09:56.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.550+0000 7f717f59e640 1 -- start start 2023-10-18T09:09:56.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.551+0000 7f717f59e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71800ae6c0 0x7f71800c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:56.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.551+0000 7f717f59e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71800cee50 0x7f718006b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:56.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.551+0000 7f717f59e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f718015eb10 0x7f718006bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:56.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.551+0000 7f717f59e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71800cce40 con 0x7f71800ae6c0 2023-10-18T09:09:56.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.551+0000 7f717f59e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71800cc230 con 0x7f71800cee50 2023-10-18T09:09:56.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.551+0000 7f717f59e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71800cc510 con 0x7f718015eb10 2023-10-18T09:09:56.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.551+0000 7f717e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71800ae6c0 0x7f71800c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:56.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.551+0000 7f717e59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71800ae6c0 0x7f71800c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37940/0 (socket says 172.21.15.40:37940) 2023-10-18T09:09:56.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.551+0000 7f717e59c640 1 -- 172.21.15.40:0/97407816 learned_addr learned my addr 172.21.15.40:0/97407816 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:56.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.551+0000 7f717e59c640 1 -- 172.21.15.40:0/97407816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f718015eb10 msgr2=0x7f718006bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:56.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.551+0000 7f717e59c640 1 --2- 172.21.15.40:0/97407816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f718015eb10 0x7f718006bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.551+0000 7f717e59c640 1 -- 172.21.15.40:0/97407816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71800cee50 msgr2=0x7f718006b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:56.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.551+0000 7f717e59c640 1 --2- 172.21.15.40:0/97407816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71800cee50 0x7f718006b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.551+0000 7f717e59c640 1 -- 172.21.15.40:0/97407816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7178009560 con 0x7f71800ae6c0 2023-10-18T09:09:56.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.552+0000 7f717e59c640 1 --2- 172.21.15.40:0/97407816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71800ae6c0 0x7f71800c7c60 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f717000c7c0 tx=0x7f717000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:56.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.552+0000 7f7176ffd640 1 -- 172.21.15.40:0/97407816 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717000aa50 con 0x7f71800ae6c0 2023-10-18T09:09:56.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.552+0000 7f7176ffd640 1 -- 172.21.15.40:0/97407816 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f717000aca0 con 0x7f71800ae6c0 2023-10-18T09:09:56.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.552+0000 7f7176ffd640 1 -- 172.21.15.40:0/97407816 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7170012680 con 0x7f71800ae6c0 2023-10-18T09:09:56.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.552+0000 7f717f59e640 1 -- 172.21.15.40:0/97407816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71800cc7f0 con 0x7f71800ae6c0 2023-10-18T09:09:56.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.552+0000 7f717f59e640 1 -- 172.21.15.40:0/97407816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71800cbe70 con 0x7f71800ae6c0 2023-10-18T09:09:56.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.553+0000 7f717f59e640 1 -- 172.21.15.40:0/97407816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f71800aeb50 con 0x7f71800ae6c0 2023-10-18T09:09:56.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.554+0000 7f7176ffd640 1 -- 172.21.15.40:0/97407816 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f717001b400 con 0x7f71800ae6c0 2023-10-18T09:09:56.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.555+0000 7f7176ffd640 1 --2- 172.21.15.40:0/97407816 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f716006ae40 0x7f716006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:56.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.555+0000 7f7176ffd640 1 -- 172.21.15.40:0/97407816 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7170089810 con 0x7f71800ae6c0 2023-10-18T09:09:56.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.555+0000 7f7176ffd640 1 --2- 172.21.15.40:0/97407816 >> [v2:172.21.15.144:6810/1718994813,v1:172.21.15.144:6811/1718994813] conn(0x7f7160070ca0 0x7f71600730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.555+0000 7f7176ffd640 1 -- 172.21.15.40:0/97407816 --> [v2:172.21.15.144:6810/1718994813,v1:172.21.15.144:6811/1718994813] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7160073770 con 0x7f7160070ca0 2023-10-18T09:09:56.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.555+0000 7f7176ffd640 1 -- 172.21.15.40:0/97407816 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f717008bdf0 con 0x7f71800ae6c0 2023-10-18T09:09:56.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.556+0000 7f717dd9b640 1 --2- 172.21.15.40:0/97407816 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f716006ae40 0x7f716006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:56.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.556+0000 7f717ed9d640 1 --2- 172.21.15.40:0/97407816 >> [v2:172.21.15.144:6810/1718994813,v1:172.21.15.144:6811/1718994813] conn(0x7f7160070ca0 0x7f71600730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:56.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.557+0000 7f717ed9d640 1 --2- 172.21.15.40:0/97407816 >> [v2:172.21.15.144:6810/1718994813,v1:172.21.15.144:6811/1718994813] conn(0x7f7160070ca0 0x7f71600730c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:56.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.557+0000 7f717dd9b640 1 --2- 172.21.15.40:0/97407816 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f716006ae40 0x7f716006d300 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7168005fd0 tx=0x7f7168005850 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:56.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.559+0000 7f7176ffd640 1 -- 172.21.15.40:0/97407816 <== osd.4 v2:172.21.15.144:6810/1718994813 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f7160073770 con 0x7f7160070ca0 2023-10-18T09:09:56.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.572+0000 7f717f59e640 1 -- 172.21.15.40:0/97407816 --> [v2:172.21.15.144:6810/1718994813,v1:172.21.15.144:6811/1718994813] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7180067020 con 0x7f7160070ca0 2023-10-18T09:09:56.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.573+0000 7f7176ffd640 1 -- 172.21.15.40:0/97407816 <== osd.4 v2:172.21.15.144:6810/1718994813 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f7180067020 con 0x7f7160070ca0 2023-10-18T09:09:56.576 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:09:56.576 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-18T09:09:56.576 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:09:56.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.574+0000 7f717f59e640 1 -- 172.21.15.40:0/97407816 >> [v2:172.21.15.144:6810/1718994813,v1:172.21.15.144:6811/1718994813] conn(0x7f7160070ca0 msgr2=0x7f71600730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:56.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.574+0000 7f717f59e640 1 --2- 172.21.15.40:0/97407816 >> [v2:172.21.15.144:6810/1718994813,v1:172.21.15.144:6811/1718994813] conn(0x7f7160070ca0 0x7f71600730c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.574+0000 7f717f59e640 1 -- 172.21.15.40:0/97407816 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f716006ae40 msgr2=0x7f716006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:56.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.574+0000 7f717f59e640 1 --2- 172.21.15.40:0/97407816 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f716006ae40 0x7f716006d300 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7168005fd0 tx=0x7f7168005850 comp rx=0 tx=0).stop 2023-10-18T09:09:56.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.574+0000 7f717f59e640 1 -- 172.21.15.40:0/97407816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71800ae6c0 msgr2=0x7f71800c7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:56.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.574+0000 7f717f59e640 1 --2- 172.21.15.40:0/97407816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71800ae6c0 0x7f71800c7c60 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f717000c7c0 tx=0x7f717000cc80 comp rx=0 tx=0).stop 2023-10-18T09:09:56.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.574+0000 7f717ed9d640 1 -- 172.21.15.40:0/97407816 reap_dead start 2023-10-18T09:09:56.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.574+0000 7f717f59e640 1 -- 172.21.15.40:0/97407816 shutdown_connections 2023-10-18T09:09:56.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.574+0000 7f717f59e640 1 -- 172.21.15.40:0/97407816 >> 172.21.15.40:0/97407816 conn(0x7f718009a570 msgr2=0x7f71800b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:56.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.575+0000 7f717f59e640 1 -- 172.21.15.40:0/97407816 shutdown_connections 2023-10-18T09:09:56.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.575+0000 7f717f59e640 1 -- 172.21.15.40:0/97407816 wait complete. 2023-10-18T09:09:56.596 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-18T09:09:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.824+0000 7f2dc2844640 1 -- 172.21.15.40:0/434093600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dbc06b710 msgr2=0x7f2dbc06bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.824+0000 7f2dc2844640 1 --2- 172.21.15.40:0/434093600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dbc06b710 0x7f2dbc06bb80 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f2db4009be0 tx=0x7f2db402bfa0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.825+0000 7f2dc2844640 1 -- 172.21.15.40:0/434093600 shutdown_connections 2023-10-18T09:09:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.825+0000 7f2dc2844640 1 --2- 172.21.15.40:0/434093600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dbc06b710 0x7f2dbc06bb80 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.825+0000 7f2dc2844640 1 --2- 172.21.15.40:0/434093600 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2dbc0cdbb0 0x7f2dbc15a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.825+0000 7f2dc2844640 1 -- 172.21.15.40:0/434093600 >> 172.21.15.40:0/434093600 conn(0x7f2dbc06cee0 msgr2=0x7f2dbc069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:56.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.825+0000 7f2dc2844640 1 -- 172.21.15.40:0/434093600 shutdown_connections 2023-10-18T09:09:56.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.825+0000 7f2dc2844640 1 -- 172.21.15.40:0/434093600 wait complete. 2023-10-18T09:09:56.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.827+0000 7f2dc2844640 1 Processor -- start 2023-10-18T09:09:56.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.827+0000 7f2dc2844640 1 -- start start 2023-10-18T09:09:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.829+0000 7f2dc2844640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dbc06b710 0x7f2dbc14b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:56.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.829+0000 7f2dc2844640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2dbc0cbc20 0x7f2dbc1e6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:56.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.829+0000 7f2dc2844640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dbc0cdbb0 0x7f2dbc1e6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:56.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.829+0000 7f2dc2844640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc1e7010 con 0x7f2dbc06b710 2023-10-18T09:09:56.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.829+0000 7f2dc2844640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc1e72f0 con 0x7f2dbc0cdbb0 2023-10-18T09:09:56.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.829+0000 7f2dc2844640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc1e75d0 con 0x7f2dbc0cbc20 2023-10-18T09:09:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.829+0000 7f2dbbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dbc06b710 0x7f2dbc14b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.829+0000 7f2dbbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dbc06b710 0x7f2dbc14b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37964/0 (socket says 172.21.15.40:37964) 2023-10-18T09:09:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.829+0000 7f2dbbfff640 1 -- 172.21.15.40:0/2103656152 learned_addr learned my addr 172.21.15.40:0/2103656152 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.829+0000 7f2dbb7fe640 1 --2- 172.21.15.40:0/2103656152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2dbc0cbc20 0x7f2dbc1e6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.829+0000 7f2dc0de1640 1 --2- 172.21.15.40:0/2103656152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dbc0cdbb0 0x7f2dbc1e6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.829+0000 7f2dbbfff640 1 -- 172.21.15.40:0/2103656152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2dbc0cbc20 msgr2=0x7f2dbc1e6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:56.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.829+0000 7f2dbbfff640 1 --2- 172.21.15.40:0/2103656152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2dbc0cbc20 0x7f2dbc1e6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.829+0000 7f2dbbfff640 1 -- 172.21.15.40:0/2103656152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dbc0cdbb0 msgr2=0x7f2dbc1e6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:56.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.830+0000 7f2dbbfff640 1 --2- 172.21.15.40:0/2103656152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dbc0cdbb0 0x7f2dbc1e6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:56.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.830+0000 7f2dbbfff640 1 -- 172.21.15.40:0/2103656152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db4009840 con 0x7f2dbc06b710 2023-10-18T09:09:56.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.830+0000 7f2dc0de1640 1 --2- 172.21.15.40:0/2103656152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dbc0cdbb0 0x7f2dbc1e6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:56.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.830+0000 7f2dbbfff640 1 --2- 172.21.15.40:0/2103656152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dbc06b710 0x7f2dbc14b230 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00e790 tx=0x7f2dac00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:56.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.830+0000 7f2db8ff9640 1 -- 172.21.15.40:0/2103656152 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac009580 con 0x7f2dbc06b710 2023-10-18T09:09:56.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.830+0000 7f2db8ff9640 1 -- 172.21.15.40:0/2103656152 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2dac00cdb0 con 0x7f2dbc06b710 2023-10-18T09:09:56.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.830+0000 7f2db8ff9640 1 -- 172.21.15.40:0/2103656152 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac010690 con 0x7f2dbc06b710 2023-10-18T09:09:56.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.830+0000 7f2dc2844640 1 -- 172.21.15.40:0/2103656152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dbc1e78b0 con 0x7f2dbc06b710 2023-10-18T09:09:56.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.831+0000 7f2dc2844640 1 -- 172.21.15.40:0/2103656152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dbc14b770 con 0x7f2dbc06b710 2023-10-18T09:09:56.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.832+0000 7f2dc2844640 1 -- 172.21.15.40:0/2103656152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dbc066fd0 con 0x7f2dbc06b710 2023-10-18T09:09:56.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.833+0000 7f2db8ff9640 1 -- 172.21.15.40:0/2103656152 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2dac0107f0 con 0x7f2dbc06b710 2023-10-18T09:09:56.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.834+0000 7f2db8ff9640 1 --2- 172.21.15.40:0/2103656152 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2da806ae70 0x7f2da806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:56.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.834+0000 7f2db8ff9640 1 -- 172.21.15.40:0/2103656152 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2dac014070 con 0x7f2dbc06b710 2023-10-18T09:09:56.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.834+0000 7f2dbb7fe640 1 --2- 172.21.15.40:0/2103656152 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2da806ae70 0x7f2da806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:56.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.835+0000 7f2dbb7fe640 1 --2- 172.21.15.40:0/2103656152 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2da806ae70 0x7f2da806d330 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc1f4f20 tx=0x7f2db0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:56.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:56.838+0000 7f2db8ff9640 1 -- 172.21.15.40:0/2103656152 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2dac051ca0 con 0x7f2dbc06b710 2023-10-18T09:09:57.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.034+0000 7f2dc2844640 1 -- 172.21.15.40:0/2103656152 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2dbc1e7460 con 0x7f2da806ae70 2023-10-18T09:09:57.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.035+0000 7f2db8ff9640 1 -- 172.21.15.40:0/2103656152 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2dbc1e7460 con 0x7f2da806ae70 2023-10-18T09:09:57.037 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 4 to deep-scrub 2023-10-18T09:09:57.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.037+0000 7f2dc2844640 1 -- 172.21.15.40:0/2103656152 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2da806ae70 msgr2=0x7f2da806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:57.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.037+0000 7f2dc2844640 1 --2- 172.21.15.40:0/2103656152 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2da806ae70 0x7f2da806d330 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc1f4f20 tx=0x7f2db0009040 comp rx=0 tx=0).stop 2023-10-18T09:09:57.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.037+0000 7f2dc2844640 1 -- 172.21.15.40:0/2103656152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dbc06b710 msgr2=0x7f2dbc14b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:57.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.037+0000 7f2dc2844640 1 --2- 172.21.15.40:0/2103656152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dbc06b710 0x7f2dbc14b230 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00e790 tx=0x7f2dac00ec50 comp rx=0 tx=0).stop 2023-10-18T09:09:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.037+0000 7f2dc2844640 1 -- 172.21.15.40:0/2103656152 shutdown_connections 2023-10-18T09:09:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.037+0000 7f2dc2844640 1 --2- 172.21.15.40:0/2103656152 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f2da806ae70 0x7f2da806d330 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.037+0000 7f2dc2844640 1 --2- 172.21.15.40:0/2103656152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2dbc0cdbb0 0x7f2dbc1e6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.037+0000 7f2dc2844640 1 --2- 172.21.15.40:0/2103656152 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2dbc0cbc20 0x7f2dbc1e6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.037+0000 7f2dc2844640 1 --2- 172.21.15.40:0/2103656152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dbc06b710 0x7f2dbc14b230 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.037+0000 7f2dc2844640 1 -- 172.21.15.40:0/2103656152 >> 172.21.15.40:0/2103656152 conn(0x7f2dbc06cee0 msgr2=0x7f2dbc0aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:57.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.038+0000 7f2dc2844640 1 -- 172.21.15.40:0/2103656152 shutdown_connections 2023-10-18T09:09:57.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.038+0000 7f2dc2844640 1 -- 172.21.15.40:0/2103656152 wait complete. 2023-10-18T09:09:57.058 INFO:tasks.ceph:Scrubbing osd.5 2023-10-18T09:09:57.058 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-18T09:09:57.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.296+0000 7f661359e640 1 -- 172.21.15.40:0/3583727654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65fc0039a0 msgr2=0x7f65fc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:57.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.296+0000 7f661359e640 1 --2- 172.21.15.40:0/3583727654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65fc0039a0 0x7f65fc003dc0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f6604001680 tx=0x7f65fc0057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.296+0000 7f6612d9d640 1 -- 172.21.15.40:0/3583727654 reap_dead start 2023-10-18T09:09:57.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.296+0000 7f661359e640 1 -- 172.21.15.40:0/3583727654 shutdown_connections 2023-10-18T09:09:57.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.296+0000 7f661359e640 1 -- 172.21.15.40:0/3583727654 >> 172.21.15.40:0/3583727654 conn(0x7f66140aeb90 msgr2=0x7f66140aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:57.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.296+0000 7f661359e640 1 -- 172.21.15.40:0/3583727654 shutdown_connections 2023-10-18T09:09:57.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.296+0000 7f661359e640 1 -- 172.21.15.40:0/3583727654 wait complete. 2023-10-18T09:09:57.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.298+0000 7f661359e640 1 Processor -- start 2023-10-18T09:09:57.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.298+0000 7f661359e640 1 -- start start 2023-10-18T09:09:57.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.299+0000 7f661359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65fc0039a0 0x7f66141543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.299+0000 7f661359e640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66141f2900 0x7f661415a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.299+0000 7f661359e640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f661415a670 0x7f6614156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.299+0000 7f661359e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6614158da0 con 0x7f65fc0039a0 2023-10-18T09:09:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.299+0000 7f661359e640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661415b2a0 con 0x7f66141f2900 2023-10-18T09:09:57.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.299+0000 7f661359e640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66141548e0 con 0x7f661415a670 2023-10-18T09:09:57.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.300+0000 7f661259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65fc0039a0 0x7f66141543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:57.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.300+0000 7f661259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65fc0039a0 0x7f66141543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37988/0 (socket says 172.21.15.40:37988) 2023-10-18T09:09:57.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.300+0000 7f661259c640 1 -- 172.21.15.40:0/271202758 learned_addr learned my addr 172.21.15.40:0/271202758 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:57.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.300+0000 7f6612d9d640 1 --2- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f661415a670 0x7f6614156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:57.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.300+0000 7f6611d9b640 1 --2- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66141f2900 0x7f661415a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:57.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.300+0000 7f661259c640 1 -- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f661415a670 msgr2=0x7f6614156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:57.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.300+0000 7f661259c640 1 --2- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f661415a670 0x7f6614156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.300+0000 7f661259c640 1 -- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66141f2900 msgr2=0x7f661415a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.300+0000 7f661259c640 1 --2- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66141f2900 0x7f661415a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.300+0000 7f661259c640 1 -- 172.21.15.40:0/271202758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6614158120 con 0x7f65fc0039a0 2023-10-18T09:09:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.300+0000 7f6611d9b640 1 --2- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66141f2900 0x7f661415a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.301+0000 7f661259c640 1 --2- 172.21.15.40:0/271202758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65fc0039a0 0x7f66141543a0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f66040016e0 tx=0x7f6604003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.301+0000 7f6602ffd640 1 -- 172.21.15.40:0/271202758 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660403c070 con 0x7f65fc0039a0 2023-10-18T09:09:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.301+0000 7f6602ffd640 1 -- 172.21.15.40:0/271202758 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6604006260 con 0x7f65fc0039a0 2023-10-18T09:09:57.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.301+0000 7f6602ffd640 1 -- 172.21.15.40:0/271202758 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604006540 con 0x7f65fc0039a0 2023-10-18T09:09:57.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.301+0000 7f661359e640 1 -- 172.21.15.40:0/271202758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6614158400 con 0x7f65fc0039a0 2023-10-18T09:09:57.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.301+0000 7f661359e640 1 -- 172.21.15.40:0/271202758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66141589c0 con 0x7f65fc0039a0 2023-10-18T09:09:57.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.302+0000 7f661359e640 1 -- 172.21.15.40:0/271202758 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f661415c030 con 0x7f65fc0039a0 2023-10-18T09:09:57.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.303+0000 7f6602ffd640 1 -- 172.21.15.40:0/271202758 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f660402e6c0 con 0x7f65fc0039a0 2023-10-18T09:09:57.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.304+0000 7f6602ffd640 1 --2- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f65f406ae40 0x7f65f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:57.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.304+0000 7f6602ffd640 1 -- 172.21.15.40:0/271202758 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f66040ab410 con 0x7f65fc0039a0 2023-10-18T09:09:57.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.304+0000 7f6602ffd640 1 --2- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:6802/1833904789,v1:172.21.15.144:6803/1833904789] conn(0x7f65f4070ca0 0x7f65f40730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:57.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.304+0000 7f6602ffd640 1 -- 172.21.15.40:0/271202758 --> [v2:172.21.15.144:6802/1833904789,v1:172.21.15.144:6803/1833904789] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65f4073770 con 0x7f65f4070ca0 2023-10-18T09:09:57.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.304+0000 7f6602ffd640 1 -- 172.21.15.40:0/271202758 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66040ad990 con 0x7f65fc0039a0 2023-10-18T09:09:57.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.305+0000 7f6611d9b640 1 --2- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f65f406ae40 0x7f65f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:57.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.305+0000 7f6612d9d640 1 --2- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:6802/1833904789,v1:172.21.15.144:6803/1833904789] conn(0x7f65f4070ca0 0x7f65f40730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:57.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.306+0000 7f6611d9b640 1 --2- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f65f406ae40 0x7f65f406d300 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f66080017c0 tx=0x7f6608001b90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:57.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.306+0000 7f6612d9d640 1 --2- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:6802/1833904789,v1:172.21.15.144:6803/1833904789] conn(0x7f65f4070ca0 0x7f65f40730c0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:57.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.308+0000 7f6602ffd640 1 -- 172.21.15.40:0/271202758 <== osd.5 v2:172.21.15.144:6802/1833904789 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f65f4073770 con 0x7f65f4070ca0 2023-10-18T09:09:57.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.321+0000 7f661359e640 1 -- 172.21.15.40:0/271202758 --> [v2:172.21.15.144:6802/1833904789,v1:172.21.15.144:6803/1833904789] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6614158b50 con 0x7f65f4070ca0 2023-10-18T09:09:57.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.322+0000 7f6602ffd640 1 -- 172.21.15.40:0/271202758 <== osd.5 v2:172.21.15.144:6802/1833904789 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f6614158b50 con 0x7f65f4070ca0 2023-10-18T09:09:57.326 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:09:57.326 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-18T09:09:57.326 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:09:57.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.323+0000 7f661359e640 1 -- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:6802/1833904789,v1:172.21.15.144:6803/1833904789] conn(0x7f65f4070ca0 msgr2=0x7f65f40730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:57.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.323+0000 7f661359e640 1 --2- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:6802/1833904789,v1:172.21.15.144:6803/1833904789] conn(0x7f65f4070ca0 0x7f65f40730c0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.323+0000 7f661359e640 1 -- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f65f406ae40 msgr2=0x7f65f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:57.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.323+0000 7f661359e640 1 --2- 172.21.15.40:0/271202758 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f65f406ae40 0x7f65f406d300 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f66080017c0 tx=0x7f6608001b90 comp rx=0 tx=0).stop 2023-10-18T09:09:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.323+0000 7f661359e640 1 -- 172.21.15.40:0/271202758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65fc0039a0 msgr2=0x7f66141543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.323+0000 7f661359e640 1 --2- 172.21.15.40:0/271202758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65fc0039a0 0x7f66141543a0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f66040016e0 tx=0x7f6604003e70 comp rx=0 tx=0).stop 2023-10-18T09:09:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.323+0000 7f6612d9d640 1 -- 172.21.15.40:0/271202758 reap_dead start 2023-10-18T09:09:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.323+0000 7f661359e640 1 -- 172.21.15.40:0/271202758 shutdown_connections 2023-10-18T09:09:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.323+0000 7f661359e640 1 -- 172.21.15.40:0/271202758 >> 172.21.15.40:0/271202758 conn(0x7f66140aeb90 msgr2=0x7f6614070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.323+0000 7f661359e640 1 -- 172.21.15.40:0/271202758 shutdown_connections 2023-10-18T09:09:57.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.323+0000 7f661359e640 1 -- 172.21.15.40:0/271202758 wait complete. 2023-10-18T09:09:57.343 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-18T09:09:57.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.572+0000 7f421d322640 1 -- 172.21.15.40:0/1940083568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42180c9f90 msgr2=0x7f421806b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.572+0000 7f421d322640 1 --2- 172.21.15.40:0/1940083568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42180c9f90 0x7f421806b4d0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f421806af60 tx=0x7f421002c240 comp rx=0 tx=0).stop 2023-10-18T09:09:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.572+0000 7f421d322640 1 -- 172.21.15.40:0/1940083568 shutdown_connections 2023-10-18T09:09:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.572+0000 7f421d322640 1 --2- 172.21.15.40:0/1940083568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42180c9f90 0x7f421806b4d0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.572+0000 7f421d322640 1 --2- 172.21.15.40:0/1940083568 >> v2:172.21.15.144:3300/0 conn(0x7f42181425a0 0x7f42180c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.572+0000 7f421d322640 1 --2- 172.21.15.40:0/1940083568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42181420f0 0x7f42180b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.572+0000 7f421d322640 1 -- 172.21.15.40:0/1940083568 >> 172.21.15.40:0/1940083568 conn(0x7f4218069720 msgr2=0x7f4218069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:57.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.572+0000 7f421d322640 1 -- 172.21.15.40:0/1940083568 shutdown_connections 2023-10-18T09:09:57.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.572+0000 7f421d322640 1 -- 172.21.15.40:0/1940083568 wait complete. 2023-10-18T09:09:57.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.574+0000 7f421d322640 1 Processor -- start 2023-10-18T09:09:57.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.574+0000 7f421d322640 1 -- start start 2023-10-18T09:09:57.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.575+0000 7f421d322640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42180c9f90 0x7f421815c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:57.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.575+0000 7f421d322640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42181420f0 0x7f42181e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:57.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.575+0000 7f421d322640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42181425a0 0x7f4218158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:57.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.575+0000 7f421d322640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42181e7b70 con 0x7f42180c9f90 2023-10-18T09:09:57.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.575+0000 7f421d322640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42181e7d80 con 0x7f42181420f0 2023-10-18T09:09:57.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.575+0000 7f421d322640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42181e8060 con 0x7f42181425a0 2023-10-18T09:09:57.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.576+0000 7f4217fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42180c9f90 0x7f421815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:57.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.576+0000 7f4217fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42180c9f90 0x7f421815c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38002/0 (socket says 172.21.15.40:38002) 2023-10-18T09:09:57.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.576+0000 7f4217fff640 1 -- 172.21.15.40:0/1419616556 learned_addr learned my addr 172.21.15.40:0/1419616556 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.576+0000 7f4217fff640 1 -- 172.21.15.40:0/1419616556 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42181425a0 msgr2=0x7f4218158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.576+0000 7f420ffff640 1 --2- 172.21.15.40:0/1419616556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42181420f0 0x7f42181e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.576+0000 7f421cb21640 1 --2- 172.21.15.40:0/1419616556 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42181425a0 0x7f4218158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:57.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.576+0000 7f4217fff640 1 --2- 172.21.15.40:0/1419616556 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42181425a0 0x7f4218158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.576+0000 7f4217fff640 1 -- 172.21.15.40:0/1419616556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42181420f0 msgr2=0x7f42181e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:57.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.576+0000 7f4217fff640 1 --2- 172.21.15.40:0/1419616556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42181420f0 0x7f42181e7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.576+0000 7f4217fff640 1 -- 172.21.15.40:0/1419616556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4210009900 con 0x7f42180c9f90 2023-10-18T09:09:57.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.576+0000 7f421cb21640 1 --2- 172.21.15.40:0/1419616556 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42181425a0 0x7f4218158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:57.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.576+0000 7f420ffff640 1 --2- 172.21.15.40:0/1419616556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42181420f0 0x7f42181e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:57.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.577+0000 7f4217fff640 1 --2- 172.21.15.40:0/1419616556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42180c9f90 0x7f421815c880 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f420800e9f0 tx=0x7f420800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:57.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.577+0000 7f42157fa640 1 -- 172.21.15.40:0/1419616556 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42080096a0 con 0x7f42180c9f90 2023-10-18T09:09:57.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.577+0000 7f42157fa640 1 -- 172.21.15.40:0/1419616556 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f420800ccc0 con 0x7f42180c9f90 2023-10-18T09:09:57.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.577+0000 7f42157fa640 1 -- 172.21.15.40:0/1419616556 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208005280 con 0x7f42180c9f90 2023-10-18T09:09:57.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.577+0000 7f421d322640 1 -- 172.21.15.40:0/1419616556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421806f7a0 con 0x7f42180c9f90 2023-10-18T09:09:57.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.577+0000 7f421d322640 1 -- 172.21.15.40:0/1419616556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4218158a70 con 0x7f42180c9f90 2023-10-18T09:09:57.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.578+0000 7f421d322640 1 -- 172.21.15.40:0/1419616556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f421806f5e0 con 0x7f42180c9f90 2023-10-18T09:09:57.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.579+0000 7f42157fa640 1 -- 172.21.15.40:0/1419616556 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f42080053e0 con 0x7f42180c9f90 2023-10-18T09:09:57.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.580+0000 7f42157fa640 1 --2- 172.21.15.40:0/1419616556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f41f806ae40 0x7f41f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:57.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.580+0000 7f42157fa640 1 -- 172.21.15.40:0/1419616556 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4208014070 con 0x7f42180c9f90 2023-10-18T09:09:57.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.581+0000 7f420ffff640 1 --2- 172.21.15.40:0/1419616556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f41f806ae40 0x7f41f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:57.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.582+0000 7f420ffff640 1 --2- 172.21.15.40:0/1419616556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f41f806ae40 0x7f41f806d300 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4200005fd0 tx=0x7f4200005d20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:57.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.584+0000 7f42157fa640 1 -- 172.21.15.40:0/1419616556 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4208058c10 con 0x7f42180c9f90 2023-10-18T09:09:57.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.777+0000 7f421d322640 1 -- 172.21.15.40:0/1419616556 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f4218066e00 con 0x7f41f806ae40 2023-10-18T09:09:57.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.778+0000 7f42157fa640 1 -- 172.21.15.40:0/1419616556 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4218066e00 con 0x7f41f806ae40 2023-10-18T09:09:57.782 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 5 to deep-scrub 2023-10-18T09:09:57.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.781+0000 7f421d322640 1 -- 172.21.15.40:0/1419616556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f41f806ae40 msgr2=0x7f41f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:57.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.781+0000 7f421d322640 1 --2- 172.21.15.40:0/1419616556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f41f806ae40 0x7f41f806d300 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4200005fd0 tx=0x7f4200005d20 comp rx=0 tx=0).stop 2023-10-18T09:09:57.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.781+0000 7f421d322640 1 -- 172.21.15.40:0/1419616556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42180c9f90 msgr2=0x7f421815c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:57.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.781+0000 7f421d322640 1 --2- 172.21.15.40:0/1419616556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42180c9f90 0x7f421815c880 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f420800e9f0 tx=0x7f420800eeb0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.781+0000 7f421d322640 1 -- 172.21.15.40:0/1419616556 shutdown_connections 2023-10-18T09:09:57.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.781+0000 7f421d322640 1 --2- 172.21.15.40:0/1419616556 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f41f806ae40 0x7f41f806d300 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.781+0000 7f421d322640 1 --2- 172.21.15.40:0/1419616556 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f42181425a0 0x7f4218158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.781+0000 7f421d322640 1 --2- 172.21.15.40:0/1419616556 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42181420f0 0x7f42181e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.781+0000 7f421d322640 1 --2- 172.21.15.40:0/1419616556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42180c9f90 0x7f421815c880 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:57.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.781+0000 7f421d322640 1 -- 172.21.15.40:0/1419616556 >> 172.21.15.40:0/1419616556 conn(0x7f4218069720 msgr2=0x7f42180ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:57.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.781+0000 7f421d322640 1 -- 172.21.15.40:0/1419616556 shutdown_connections 2023-10-18T09:09:57.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:57.781+0000 7f421d322640 1 -- 172.21.15.40:0/1419616556 wait complete. 2023-10-18T09:09:57.801 INFO:tasks.ceph:Scrubbing osd.6 2023-10-18T09:09:57.801 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-18T09:09:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.021+0000 7fc38d9d0640 1 -- 172.21.15.40:0/1666658666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc38806d550 msgr2=0x7fc38806b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.021+0000 7fc38d9d0640 1 --2- 172.21.15.40:0/1666658666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc38806d550 0x7fc38806b710 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fc37c005fd0 tx=0x7fc37c02d0d0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.021+0000 7fc38d9d0640 1 -- 172.21.15.40:0/1666658666 shutdown_connections 2023-10-18T09:09:58.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.021+0000 7fc38d9d0640 1 --2- 172.21.15.40:0/1666658666 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc38814e350 0x7fc3880cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.021+0000 7fc38d9d0640 1 --2- 172.21.15.40:0/1666658666 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc38806d550 0x7fc38806b710 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.021+0000 7fc38d9d0640 1 -- 172.21.15.40:0/1666658666 >> 172.21.15.40:0/1666658666 conn(0x7fc3880aeb90 msgr2=0x7fc3880aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:58.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.021+0000 7fc38d9d0640 1 -- 172.21.15.40:0/1666658666 shutdown_connections 2023-10-18T09:09:58.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.021+0000 7fc38d9d0640 1 -- 172.21.15.40:0/1666658666 wait complete. 2023-10-18T09:09:58.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.024+0000 7fc38d9d0640 1 Processor -- start 2023-10-18T09:09:58.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.024+0000 7fc38d9d0640 1 -- start start 2023-10-18T09:09:58.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.025+0000 7fc38d9d0640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc38806d550 0x7fc388147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:58.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.025+0000 7fc38d9d0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3880cc780 0x7fc38806f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:58.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.025+0000 7fc38d9d0640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc38814e350 0x7fc38806f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:58.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.025+0000 7fc38d9d0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880ad7c0 con 0x7fc3880cc780 2023-10-18T09:09:58.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.025+0000 7fc38d9d0640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880adaa0 con 0x7fc38814e350 2023-10-18T09:09:58.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.025+0000 7fc38d9d0640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880add80 con 0x7fc38806d550 2023-10-18T09:09:58.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.026+0000 7fc387fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3880cc780 0x7fc38806f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:58.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.026+0000 7fc387fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3880cc780 0x7fc38806f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38018/0 (socket says 172.21.15.40:38018) 2023-10-18T09:09:58.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.026+0000 7fc387fff640 1 -- 172.21.15.40:0/3771026501 learned_addr learned my addr 172.21.15.40:0/3771026501 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:58.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.026+0000 7fc38c9ce640 1 --2- 172.21.15.40:0/3771026501 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc38806d550 0x7fc388147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:58.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.026+0000 7fc38d1cf640 1 --2- 172.21.15.40:0/3771026501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc38814e350 0x7fc38806f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:58.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.026+0000 7fc387fff640 1 -- 172.21.15.40:0/3771026501 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc38806d550 msgr2=0x7fc388147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.026+0000 7fc387fff640 1 --2- 172.21.15.40:0/3771026501 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc38806d550 0x7fc388147b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.026+0000 7fc387fff640 1 -- 172.21.15.40:0/3771026501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc38814e350 msgr2=0x7fc38806f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.026+0000 7fc387fff640 1 --2- 172.21.15.40:0/3771026501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc38814e350 0x7fc38806f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.026+0000 7fc387fff640 1 -- 172.21.15.40:0/3771026501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c00a670 con 0x7fc3880cc780 2023-10-18T09:09:58.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.026+0000 7fc387fff640 1 --2- 172.21.15.40:0/3771026501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3880cc780 0x7fc38806f460 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fc37400d6d0 tx=0x7fc37400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:58.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.027+0000 7fc3857fa640 1 -- 172.21.15.40:0/3771026501 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc374005220 con 0x7fc3880cc780 2023-10-18T09:09:58.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.027+0000 7fc3857fa640 1 -- 172.21.15.40:0/3771026501 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc374005490 con 0x7fc3880cc780 2023-10-18T09:09:58.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.027+0000 7fc3857fa640 1 -- 172.21.15.40:0/3771026501 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc374009a80 con 0x7fc3880cc780 2023-10-18T09:09:58.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.027+0000 7fc38d9d0640 1 -- 172.21.15.40:0/3771026501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3880ae060 con 0x7fc3880cc780 2023-10-18T09:09:58.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.027+0000 7fc38d9d0640 1 -- 172.21.15.40:0/3771026501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3880acb40 con 0x7fc3880cc780 2023-10-18T09:09:58.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.028+0000 7fc38d9d0640 1 -- 172.21.15.40:0/3771026501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc3880adc10 con 0x7fc3880cc780 2023-10-18T09:09:58.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.029+0000 7fc3857fa640 1 -- 172.21.15.40:0/3771026501 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc3740047a0 con 0x7fc3880cc780 2023-10-18T09:09:58.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.030+0000 7fc3857fa640 1 --2- 172.21.15.40:0/3771026501 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc36c06ae40 0x7fc36c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:58.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.030+0000 7fc3857fa640 1 -- 172.21.15.40:0/3771026501 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc3740896b0 con 0x7fc3880cc780 2023-10-18T09:09:58.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.030+0000 7fc3857fa640 1 --2- 172.21.15.40:0/3771026501 >> [v2:172.21.15.144:6818/1988482345,v1:172.21.15.144:6819/1988482345] conn(0x7fc36c070ca0 0x7fc36c0730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:58.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.031+0000 7fc3857fa640 1 -- 172.21.15.40:0/3771026501 --> [v2:172.21.15.144:6818/1988482345,v1:172.21.15.144:6819/1988482345] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc36c073770 con 0x7fc36c070ca0 2023-10-18T09:09:58.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.031+0000 7fc3857fa640 1 -- 172.21.15.40:0/3771026501 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc37408bc90 con 0x7fc3880cc780 2023-10-18T09:09:58.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.031+0000 7fc38c9ce640 1 --2- 172.21.15.40:0/3771026501 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc36c06ae40 0x7fc36c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:58.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.031+0000 7fc38d1cf640 1 --2- 172.21.15.40:0/3771026501 >> [v2:172.21.15.144:6818/1988482345,v1:172.21.15.144:6819/1988482345] conn(0x7fc36c070ca0 0x7fc36c0730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:58.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.032+0000 7fc38d1cf640 1 --2- 172.21.15.40:0/3771026501 >> [v2:172.21.15.144:6818/1988482345,v1:172.21.15.144:6819/1988482345] conn(0x7fc36c070ca0 0x7fc36c0730c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:58.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.033+0000 7fc38c9ce640 1 --2- 172.21.15.40:0/3771026501 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc36c06ae40 0x7fc36c06d300 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc37c036000 tx=0x7fc37c004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:58.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.034+0000 7fc3857fa640 1 -- 172.21.15.40:0/3771026501 <== osd.6 v2:172.21.15.144:6818/1988482345 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7fc36c073770 con 0x7fc36c070ca0 2023-10-18T09:09:58.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.047+0000 7fc38d9d0640 1 -- 172.21.15.40:0/3771026501 --> [v2:172.21.15.144:6818/1988482345,v1:172.21.15.144:6819/1988482345] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc38814e8c0 con 0x7fc36c070ca0 2023-10-18T09:09:58.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.048+0000 7fc3857fa640 1 -- 172.21.15.40:0/3771026501 <== osd.6 v2:172.21.15.144:6818/1988482345 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7fc380005680 con 0x7fc36c070ca0 2023-10-18T09:09:58.051 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:09:58.052 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-18T09:09:58.052 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:09:58.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.049+0000 7fc38d9d0640 1 -- 172.21.15.40:0/3771026501 >> [v2:172.21.15.144:6818/1988482345,v1:172.21.15.144:6819/1988482345] conn(0x7fc36c070ca0 msgr2=0x7fc36c0730c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.049+0000 7fc38d9d0640 1 --2- 172.21.15.40:0/3771026501 >> [v2:172.21.15.144:6818/1988482345,v1:172.21.15.144:6819/1988482345] conn(0x7fc36c070ca0 0x7fc36c0730c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.049+0000 7fc38d9d0640 1 -- 172.21.15.40:0/3771026501 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc36c06ae40 msgr2=0x7fc36c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.049+0000 7fc38d9d0640 1 --2- 172.21.15.40:0/3771026501 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc36c06ae40 0x7fc36c06d300 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc37c036000 tx=0x7fc37c004730 comp rx=0 tx=0).stop 2023-10-18T09:09:58.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.049+0000 7fc38d9d0640 1 -- 172.21.15.40:0/3771026501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3880cc780 msgr2=0x7fc38806f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.049+0000 7fc38d9d0640 1 --2- 172.21.15.40:0/3771026501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3880cc780 0x7fc38806f460 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fc37400d6d0 tx=0x7fc37400db90 comp rx=0 tx=0).stop 2023-10-18T09:09:58.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.049+0000 7fc38d1cf640 1 -- 172.21.15.40:0/3771026501 reap_dead start 2023-10-18T09:09:58.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.049+0000 7fc38d9d0640 1 -- 172.21.15.40:0/3771026501 shutdown_connections 2023-10-18T09:09:58.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.049+0000 7fc38d9d0640 1 -- 172.21.15.40:0/3771026501 >> 172.21.15.40:0/3771026501 conn(0x7fc3880aeb90 msgr2=0x7fc3880ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:58.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.049+0000 7fc38d9d0640 1 -- 172.21.15.40:0/3771026501 shutdown_connections 2023-10-18T09:09:58.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.049+0000 7fc38d9d0640 1 -- 172.21.15.40:0/3771026501 wait complete. 2023-10-18T09:09:58.074 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-18T09:09:58.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.305+0000 7fc4cde59640 1 -- 172.21.15.40:0/1330422803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c80ca0d0 msgr2=0x7fc4c8157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.305+0000 7fc4cde59640 1 --2- 172.21.15.40:0/1330422803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c80ca0d0 0x7fc4c8157d40 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80caf60 tx=0x7fc4c002c230 comp rx=0 tx=0).stop 2023-10-18T09:09:58.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.305+0000 7fc4cde59640 1 -- 172.21.15.40:0/1330422803 shutdown_connections 2023-10-18T09:09:58.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.305+0000 7fc4cde59640 1 --2- 172.21.15.40:0/1330422803 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c80ca0d0 0x7fc4c8157d40 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.305+0000 7fc4cde59640 1 --2- 172.21.15.40:0/1330422803 >> v2:172.21.15.144:3300/0 conn(0x7fc4c80b15b0 0x7fc4c80b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.305+0000 7fc4cde59640 1 -- 172.21.15.40:0/1330422803 >> 172.21.15.40:0/1330422803 conn(0x7fc4c8069720 msgr2=0x7fc4c8069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.305+0000 7fc4cde59640 1 -- 172.21.15.40:0/1330422803 shutdown_connections 2023-10-18T09:09:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.305+0000 7fc4cde59640 1 -- 172.21.15.40:0/1330422803 wait complete. 2023-10-18T09:09:58.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.307+0000 7fc4cde59640 1 Processor -- start 2023-10-18T09:09:58.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.307+0000 7fc4cde59640 1 -- start start 2023-10-18T09:09:58.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.308+0000 7fc4cde59640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c80b15b0 0x7fc4c814ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:58.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.308+0000 7fc4cde59640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c80b1e10 0x7fc4c8154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:58.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.308+0000 7fc4cde59640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4c80ca0d0 0x7fc4c80cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:58.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.308+0000 7fc4cde59640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c81539b0 con 0x7fc4c80b1e10 2023-10-18T09:09:58.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.308+0000 7fc4cde59640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c81533f0 con 0x7fc4c80ca0d0 2023-10-18T09:09:58.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.308+0000 7fc4cde59640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c81536d0 con 0x7fc4c80b15b0 2023-10-18T09:09:58.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.308+0000 7fc4c7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c80b1e10 0x7fc4c8154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:58.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.309+0000 7fc4c7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c80b1e10 0x7fc4c8154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38042/0 (socket says 172.21.15.40:38042) 2023-10-18T09:09:58.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.309+0000 7fc4c7fff640 1 -- 172.21.15.40:0/317529558 learned_addr learned my addr 172.21.15.40:0/317529558 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:58.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.309+0000 7fc4cce57640 1 --2- 172.21.15.40:0/317529558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c80b15b0 0x7fc4c814ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:58.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.309+0000 7fc4cd658640 1 --2- 172.21.15.40:0/317529558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4c80ca0d0 0x7fc4c80cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:58.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.309+0000 7fc4c7fff640 1 -- 172.21.15.40:0/317529558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c80b15b0 msgr2=0x7fc4c814ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.309+0000 7fc4c7fff640 1 --2- 172.21.15.40:0/317529558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c80b15b0 0x7fc4c814ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.309+0000 7fc4c7fff640 1 -- 172.21.15.40:0/317529558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4c80ca0d0 msgr2=0x7fc4c80cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.309+0000 7fc4c7fff640 1 --2- 172.21.15.40:0/317529558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4c80ca0d0 0x7fc4c80cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.309+0000 7fc4c7fff640 1 -- 172.21.15.40:0/317529558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c0009900 con 0x7fc4c80b1e10 2023-10-18T09:09:58.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.309+0000 7fc4cd658640 1 --2- 172.21.15.40:0/317529558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4c80ca0d0 0x7fc4c80cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:58.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.309+0000 7fc4c7fff640 1 --2- 172.21.15.40:0/317529558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c80b1e10 0x7fc4c8154040 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400c430 tx=0x7fc4b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:58.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.310+0000 7fc4c57fa640 1 -- 172.21.15.40:0/317529558 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4011790 con 0x7fc4c80b1e10 2023-10-18T09:09:58.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.310+0000 7fc4c57fa640 1 -- 172.21.15.40:0/317529558 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc4b40119e0 con 0x7fc4c80b1e10 2023-10-18T09:09:58.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.310+0000 7fc4c57fa640 1 -- 172.21.15.40:0/317529558 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4011e30 con 0x7fc4c80b1e10 2023-10-18T09:09:58.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.310+0000 7fc4cde59640 1 -- 172.21.15.40:0/317529558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c815bef0 con 0x7fc4c80b1e10 2023-10-18T09:09:58.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.310+0000 7fc4cde59640 1 -- 172.21.15.40:0/317529558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c80a6880 con 0x7fc4c80b1e10 2023-10-18T09:09:58.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.311+0000 7fc4cde59640 1 -- 172.21.15.40:0/317529558 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c80a6aa0 con 0x7fc4c80b1e10 2023-10-18T09:09:58.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.313+0000 7fc4c57fa640 1 -- 172.21.15.40:0/317529558 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc4b401c460 con 0x7fc4c80b1e10 2023-10-18T09:09:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.314+0000 7fc4c57fa640 1 --2- 172.21.15.40:0/317529558 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc4ac06ae40 0x7fc4ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.314+0000 7fc4c57fa640 1 -- 172.21.15.40:0/317529558 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc4b40894f0 con 0x7fc4c80b1e10 2023-10-18T09:09:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.314+0000 7fc4cce57640 1 --2- 172.21.15.40:0/317529558 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc4ac06ae40 0x7fc4ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.316+0000 7fc4cce57640 1 --2- 172.21.15.40:0/317529558 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc4ac06ae40 0x7fc4ac06d300 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc005e10 tx=0x7fc4bc005da0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.319+0000 7fc4c57fa640 1 -- 172.21.15.40:0/317529558 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc4b4054890 con 0x7fc4c80b1e10 2023-10-18T09:09:58.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.516+0000 7fc4cde59640 1 -- 172.21.15.40:0/317529558 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4c80b2270 con 0x7fc4ac06ae40 2023-10-18T09:09:58.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.517+0000 7fc4c57fa640 1 -- 172.21.15.40:0/317529558 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc4bc00a7e0 con 0x7fc4ac06ae40 2023-10-18T09:09:58.520 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 6 to deep-scrub 2023-10-18T09:09:58.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.520+0000 7fc4cde59640 1 -- 172.21.15.40:0/317529558 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc4ac06ae40 msgr2=0x7fc4ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.520+0000 7fc4cde59640 1 --2- 172.21.15.40:0/317529558 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc4ac06ae40 0x7fc4ac06d300 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc005e10 tx=0x7fc4bc005da0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.520+0000 7fc4cde59640 1 -- 172.21.15.40:0/317529558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c80b1e10 msgr2=0x7fc4c8154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.520+0000 7fc4cde59640 1 --2- 172.21.15.40:0/317529558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c80b1e10 0x7fc4c8154040 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400c430 tx=0x7fc4b400c8f0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.520+0000 7fc4cde59640 1 -- 172.21.15.40:0/317529558 shutdown_connections 2023-10-18T09:09:58.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.520+0000 7fc4cde59640 1 --2- 172.21.15.40:0/317529558 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc4ac06ae40 0x7fc4ac06d300 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.520+0000 7fc4cde59640 1 --2- 172.21.15.40:0/317529558 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4c80ca0d0 0x7fc4c80cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.520+0000 7fc4cde59640 1 --2- 172.21.15.40:0/317529558 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4c80b1e10 0x7fc4c8154040 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.520+0000 7fc4cde59640 1 --2- 172.21.15.40:0/317529558 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c80b15b0 0x7fc4c814ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.520+0000 7fc4cde59640 1 -- 172.21.15.40:0/317529558 >> 172.21.15.40:0/317529558 conn(0x7fc4c8069720 msgr2=0x7fc4c8154910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:58.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.520+0000 7fc4cde59640 1 -- 172.21.15.40:0/317529558 shutdown_connections 2023-10-18T09:09:58.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.520+0000 7fc4cde59640 1 -- 172.21.15.40:0/317529558 wait complete. 2023-10-18T09:09:58.543 INFO:tasks.ceph:Scrubbing osd.7 2023-10-18T09:09:58.543 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-18T09:09:58.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.778+0000 7fc04bef2640 1 -- 172.21.15.40:0/2153227744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc030004d00 msgr2=0x7fc030005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.778+0000 7fc04bef2640 1 --2- 172.21.15.40:0/2153227744 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc030004d00 0x7fc030005180 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00c670 tx=0x7fc0300057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.778+0000 7fc04a48f640 1 -- 172.21.15.40:0/2153227744 reap_dead start 2023-10-18T09:09:58.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.779+0000 7fc04bef2640 1 -- 172.21.15.40:0/2153227744 shutdown_connections 2023-10-18T09:09:58.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.779+0000 7fc04bef2640 1 -- 172.21.15.40:0/2153227744 >> 172.21.15.40:0/2153227744 conn(0x7fc0440aec00 msgr2=0x7fc0440aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:58.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.779+0000 7fc04bef2640 1 -- 172.21.15.40:0/2153227744 shutdown_connections 2023-10-18T09:09:58.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.779+0000 7fc04bef2640 1 -- 172.21.15.40:0/2153227744 wait complete. 2023-10-18T09:09:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.780+0000 7fc04bef2640 1 Processor -- start 2023-10-18T09:09:58.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.781+0000 7fc04bef2640 1 -- start start 2023-10-18T09:09:58.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.782+0000 7fc04bef2640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc030004d00 0x7fc04414d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:58.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.782+0000 7fc04bef2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc044155cf0 0x7fc044152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:58.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.782+0000 7fc04bef2640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0441546e0 0x7fc0441e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:58.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.782+0000 7fc04bef2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc044154c50 con 0x7fc044155cf0 2023-10-18T09:09:58.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.782+0000 7fc04bef2640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc044154f30 con 0x7fc030004d00 2023-10-18T09:09:58.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.782+0000 7fc04bef2640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc044155210 con 0x7fc0441546e0 2023-10-18T09:09:58.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.782+0000 7fc04948d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc044155cf0 0x7fc044152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:58.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.782+0000 7fc04948d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc044155cf0 0x7fc044152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38046/0 (socket says 172.21.15.40:38046) 2023-10-18T09:09:58.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.782+0000 7fc04948d640 1 -- 172.21.15.40:0/2094155776 learned_addr learned my addr 172.21.15.40:0/2094155776 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:58.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.782+0000 7fc049c8e640 1 --2- 172.21.15.40:0/2094155776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc030004d00 0x7fc04414d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:58.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.782+0000 7fc04a48f640 1 --2- 172.21.15.40:0/2094155776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0441546e0 0x7fc0441e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:58.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.783+0000 7fc04948d640 1 -- 172.21.15.40:0/2094155776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0441546e0 msgr2=0x7fc0441e3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.783+0000 7fc04948d640 1 --2- 172.21.15.40:0/2094155776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0441546e0 0x7fc0441e3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.783+0000 7fc04948d640 1 -- 172.21.15.40:0/2094155776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc030004d00 msgr2=0x7fc04414d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.783+0000 7fc04948d640 1 --2- 172.21.15.40:0/2094155776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc030004d00 0x7fc04414d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.783+0000 7fc04948d640 1 -- 172.21.15.40:0/2094155776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0441554f0 con 0x7fc044155cf0 2023-10-18T09:09:58.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.783+0000 7fc04948d640 1 --2- 172.21.15.40:0/2094155776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc044155cf0 0x7fc044152a40 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fc03400c500 tx=0x7fc03400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:58.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.783+0000 7fc03a7fc640 1 -- 172.21.15.40:0/2094155776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0340052d0 con 0x7fc044155cf0 2023-10-18T09:09:58.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.783+0000 7fc03a7fc640 1 -- 172.21.15.40:0/2094155776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc034005430 con 0x7fc044155cf0 2023-10-18T09:09:58.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.783+0000 7fc03a7fc640 1 -- 172.21.15.40:0/2094155776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034012890 con 0x7fc044155cf0 2023-10-18T09:09:58.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.783+0000 7fc04bef2640 1 -- 172.21.15.40:0/2094155776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0441557d0 con 0x7fc044155cf0 2023-10-18T09:09:58.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.783+0000 7fc04bef2640 1 -- 172.21.15.40:0/2094155776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc044152f80 con 0x7fc044155cf0 2023-10-18T09:09:58.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.784+0000 7fc04bef2640 1 -- 172.21.15.40:0/2094155776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc0440c93e0 con 0x7fc044155cf0 2023-10-18T09:09:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.786+0000 7fc03a7fc640 1 -- 172.21.15.40:0/2094155776 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc0340129f0 con 0x7fc044155cf0 2023-10-18T09:09:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.787+0000 7fc03a7fc640 1 --2- 172.21.15.40:0/2094155776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc02806ae70 0x7fc02806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.787+0000 7fc03a7fc640 1 -- 172.21.15.40:0/2094155776 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc0340890c0 con 0x7fc044155cf0 2023-10-18T09:09:58.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.787+0000 7fc03a7fc640 1 --2- 172.21.15.40:0/2094155776 >> [v2:172.21.15.144:6824/3907866946,v1:172.21.15.144:6826/3907866946] conn(0x7fc028070cd0 0x7fc0280730f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.787+0000 7fc03a7fc640 1 -- 172.21.15.40:0/2094155776 --> [v2:172.21.15.144:6824/3907866946,v1:172.21.15.144:6826/3907866946] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc0280737a0 con 0x7fc028070cd0 2023-10-18T09:09:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.787+0000 7fc03a7fc640 1 -- 172.21.15.40:0/2094155776 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=203) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc03408b620 con 0x7fc044155cf0 2023-10-18T09:09:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.787+0000 7fc049c8e640 1 --2- 172.21.15.40:0/2094155776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc02806ae70 0x7fc02806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.788+0000 7fc04a48f640 1 --2- 172.21.15.40:0/2094155776 >> [v2:172.21.15.144:6824/3907866946,v1:172.21.15.144:6826/3907866946] conn(0x7fc028070cd0 0x7fc0280730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:58.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.789+0000 7fc049c8e640 1 --2- 172.21.15.40:0/2094155776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc02806ae70 0x7fc02806d330 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc04000cb10 tx=0x7fc040005e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.789+0000 7fc04a48f640 1 --2- 172.21.15.40:0/2094155776 >> [v2:172.21.15.144:6824/3907866946,v1:172.21.15.144:6826/3907866946] conn(0x7fc028070cd0 0x7fc0280730f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:58.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.791+0000 7fc03a7fc640 1 -- 172.21.15.40:0/2094155776 <== osd.7 v2:172.21.15.144:6824/3907866946 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7fc0280737a0 con 0x7fc028070cd0 2023-10-18T09:09:58.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.804+0000 7fc04bef2640 1 -- 172.21.15.40:0/2094155776 --> [v2:172.21.15.144:6824/3907866946,v1:172.21.15.144:6826/3907866946] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc044152660 con 0x7fc028070cd0 2023-10-18T09:09:58.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.805+0000 7fc03a7fc640 1 -- 172.21.15.40:0/2094155776 <== osd.7 v2:172.21.15.144:6824/3907866946 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7fc044152660 con 0x7fc028070cd0 2023-10-18T09:09:58.809 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-18T09:09:58.809 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-18T09:09:58.809 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-18T09:09:58.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.806+0000 7fc04bef2640 1 -- 172.21.15.40:0/2094155776 >> [v2:172.21.15.144:6824/3907866946,v1:172.21.15.144:6826/3907866946] conn(0x7fc028070cd0 msgr2=0x7fc0280730f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.806+0000 7fc04bef2640 1 --2- 172.21.15.40:0/2094155776 >> [v2:172.21.15.144:6824/3907866946,v1:172.21.15.144:6826/3907866946] conn(0x7fc028070cd0 0x7fc0280730f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.806+0000 7fc04bef2640 1 -- 172.21.15.40:0/2094155776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc02806ae70 msgr2=0x7fc02806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.806+0000 7fc04bef2640 1 --2- 172.21.15.40:0/2094155776 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fc02806ae70 0x7fc02806d330 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc04000cb10 tx=0x7fc040005e50 comp rx=0 tx=0).stop 2023-10-18T09:09:58.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.806+0000 7fc04bef2640 1 -- 172.21.15.40:0/2094155776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc044155cf0 msgr2=0x7fc044152a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:58.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.806+0000 7fc04bef2640 1 --2- 172.21.15.40:0/2094155776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc044155cf0 0x7fc044152a40 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fc03400c500 tx=0x7fc03400c9c0 comp rx=0 tx=0).stop 2023-10-18T09:09:58.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.806+0000 7fc04a48f640 1 -- 172.21.15.40:0/2094155776 reap_dead start 2023-10-18T09:09:58.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.806+0000 7fc04bef2640 1 -- 172.21.15.40:0/2094155776 shutdown_connections 2023-10-18T09:09:58.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.806+0000 7fc04bef2640 1 -- 172.21.15.40:0/2094155776 >> 172.21.15.40:0/2094155776 conn(0x7fc0440aec00 msgr2=0x7fc04414f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:58.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.806+0000 7fc04bef2640 1 -- 172.21.15.40:0/2094155776 shutdown_connections 2023-10-18T09:09:58.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:58.806+0000 7fc04bef2640 1 -- 172.21.15.40:0/2094155776 wait complete. 2023-10-18T09:09:58.828 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-18T09:09:59.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.061+0000 7f10b5835640 1 -- 172.21.15.40:0/3748141079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1098005180 msgr2=0x7f109800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:59.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.061+0000 7f10b5835640 1 --2- 172.21.15.40:0/3748141079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1098005180 0x7f109800b980 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f10a8006f00 tx=0x7f10980057f0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.061+0000 7f10b5835640 1 -- 172.21.15.40:0/3748141079 shutdown_connections 2023-10-18T09:09:59.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.061+0000 7f10b5835640 1 --2- 172.21.15.40:0/3748141079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1098005180 0x7f109800b980 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.061+0000 7f10b5835640 1 -- 172.21.15.40:0/3748141079 >> 172.21.15.40:0/3748141079 conn(0x7f10b008aca0 msgr2=0x7f10b009a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:59.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.061+0000 7f10b5835640 1 -- 172.21.15.40:0/3748141079 shutdown_connections 2023-10-18T09:09:59.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.061+0000 7f10b5835640 1 -- 172.21.15.40:0/3748141079 wait complete. 2023-10-18T09:09:59.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.063+0000 7f10b5835640 1 Processor -- start 2023-10-18T09:09:59.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.063+0000 7f10b5835640 1 -- start start 2023-10-18T09:09:59.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.064+0000 7f10b5835640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1098005180 0x7f10b00c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:59.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.064+0000 7f10b5835640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b01f51f0 0x7f10b00c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:59.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.064+0000 7f10b5835640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10b01f1510 0x7f10b00af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:59.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.064+0000 7f10b5835640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b014cf90 con 0x7f10b01f1510 2023-10-18T09:09:59.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.064+0000 7f10b5835640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b01f60e0 con 0x7f1098005180 2023-10-18T09:09:59.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.064+0000 7f10b5835640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b00c9ba0 con 0x7f10b01f51f0 2023-10-18T09:09:59.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.064+0000 7f10b5034640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10b01f1510 0x7f10b00af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:59.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.064+0000 7f10b5034640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10b01f1510 0x7f10b00af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38070/0 (socket says 172.21.15.40:38070) 2023-10-18T09:09:59.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.064+0000 7f10b5034640 1 -- 172.21.15.40:0/2831620284 learned_addr learned my addr 172.21.15.40:0/2831620284 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:59.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.065+0000 7f10b4833640 1 --2- 172.21.15.40:0/2831620284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1098005180 0x7f10b00c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:59.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.065+0000 7f10affff640 1 --2- 172.21.15.40:0/2831620284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b01f51f0 0x7f10b00c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:59.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.065+0000 7f10b5034640 1 -- 172.21.15.40:0/2831620284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b01f51f0 msgr2=0x7f10b00c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:59.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.065+0000 7f10b5034640 1 --2- 172.21.15.40:0/2831620284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b01f51f0 0x7f10b00c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.065+0000 7f10b5034640 1 -- 172.21.15.40:0/2831620284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1098005180 msgr2=0x7f10b00c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:59.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.065+0000 7f10b5034640 1 --2- 172.21.15.40:0/2831620284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1098005180 0x7f10b00c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.065+0000 7f10b5034640 1 -- 172.21.15.40:0/2831620284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b00ce640 con 0x7f10b01f1510 2023-10-18T09:09:59.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.065+0000 7f10affff640 1 --2- 172.21.15.40:0/2831620284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b01f51f0 0x7f10b00c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:59.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.065+0000 7f10b4833640 1 --2- 172.21.15.40:0/2831620284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1098005180 0x7f10b00c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:09:59.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.065+0000 7f10b5034640 1 --2- 172.21.15.40:0/2831620284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10b01f1510 0x7f10b00af4b0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f10a800c290 tx=0x7f10a8006630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:59.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.065+0000 7f10ad7fa640 1 -- 172.21.15.40:0/2831620284 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a80037f0 con 0x7f10b01f1510 2023-10-18T09:09:59.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.065+0000 7f10ad7fa640 1 -- 172.21.15.40:0/2831620284 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f10a8003a40 con 0x7f10b01f1510 2023-10-18T09:09:59.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.065+0000 7f10ad7fa640 1 -- 172.21.15.40:0/2831620284 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8003e30 con 0x7f10b01f1510 2023-10-18T09:09:59.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.065+0000 7f10b5835640 1 -- 172.21.15.40:0/2831620284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b01f5840 con 0x7f10b01f1510 2023-10-18T09:09:59.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.066+0000 7f10b5835640 1 -- 172.21.15.40:0/2831620284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b01f5e00 con 0x7f10b01f1510 2023-10-18T09:09:59.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.067+0000 7f10b5835640 1 -- 172.21.15.40:0/2831620284 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b0071da0 con 0x7f10b01f1510 2023-10-18T09:09:59.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.068+0000 7f10ad7fa640 1 -- 172.21.15.40:0/2831620284 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f10a8032070 con 0x7f10b01f1510 2023-10-18T09:09:59.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.069+0000 7f10ad7fa640 1 --2- 172.21.15.40:0/2831620284 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f109c06ae70 0x7f109c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:59.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.069+0000 7f10ad7fa640 1 -- 172.21.15.40:0/2831620284 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f10a803c070 con 0x7f10b01f1510 2023-10-18T09:09:59.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.069+0000 7f10b4833640 1 --2- 172.21.15.40:0/2831620284 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f109c06ae70 0x7f109c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:59.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.070+0000 7f10b4833640 1 --2- 172.21.15.40:0/2831620284 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f109c06ae70 0x7f109c06d330 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f10a0001cd0 tx=0x7f10a0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:59.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.073+0000 7f10ad7fa640 1 -- 172.21.15.40:0/2831620284 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10a8075530 con 0x7f10b01f1510 2023-10-18T09:09:59.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.269+0000 7f10b5835640 1 -- 172.21.15.40:0/2831620284 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f10b015ef70 con 0x7f109c06ae70 2023-10-18T09:09:59.273 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.270+0000 7f10ad7fa640 1 -- 172.21.15.40:0/2831620284 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f10b015ef70 con 0x7f109c06ae70 2023-10-18T09:09:59.273 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 7 to deep-scrub 2023-10-18T09:09:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.273+0000 7f10b5835640 1 -- 172.21.15.40:0/2831620284 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f109c06ae70 msgr2=0x7f109c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.273+0000 7f10b5835640 1 --2- 172.21.15.40:0/2831620284 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f109c06ae70 0x7f109c06d330 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f10a0001cd0 tx=0x7f10a0009040 comp rx=0 tx=0).stop 2023-10-18T09:09:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.273+0000 7f10b5835640 1 -- 172.21.15.40:0/2831620284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10b01f1510 msgr2=0x7f10b00af4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.273+0000 7f10b5835640 1 --2- 172.21.15.40:0/2831620284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10b01f1510 0x7f10b00af4b0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f10a800c290 tx=0x7f10a8006630 comp rx=0 tx=0).stop 2023-10-18T09:09:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.273+0000 7f10b5835640 1 -- 172.21.15.40:0/2831620284 shutdown_connections 2023-10-18T09:09:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.273+0000 7f10b5835640 1 --2- 172.21.15.40:0/2831620284 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f109c06ae70 0x7f109c06d330 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.273+0000 7f10b5835640 1 --2- 172.21.15.40:0/2831620284 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10b01f1510 0x7f10b00af4b0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.273+0000 7f10b5835640 1 --2- 172.21.15.40:0/2831620284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f10b01f51f0 0x7f10b00c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.273+0000 7f10b5835640 1 --2- 172.21.15.40:0/2831620284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1098005180 0x7f10b00c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.273+0000 7f10b5835640 1 -- 172.21.15.40:0/2831620284 >> 172.21.15.40:0/2831620284 conn(0x7f10b008aca0 msgr2=0x7f10b00ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:59.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.273+0000 7f10b5835640 1 -- 172.21.15.40:0/2831620284 shutdown_connections 2023-10-18T09:09:59.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.273+0000 7f10b5835640 1 -- 172.21.15.40:0/2831620284 wait complete. 2023-10-18T09:09:59.294 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:09:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.519+0000 7f4ddbfff640 1 -- 172.21.15.40:0/2570930246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ddc06d6f0 msgr2=0x7f4ddc0b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.519+0000 7f4ddbfff640 1 --2- 172.21.15.40:0/2570930246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ddc06d6f0 0x7f4ddc0b15b0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc06f590 tx=0x7f4dcc02c250 comp rx=0 tx=0).stop 2023-10-18T09:09:59.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.519+0000 7f4ddbfff640 1 -- 172.21.15.40:0/2570930246 shutdown_connections 2023-10-18T09:09:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.519+0000 7f4ddbfff640 1 --2- 172.21.15.40:0/2570930246 >> v2:172.21.15.144:3300/0 conn(0x7f4ddc0c99c0 0x7f4ddc0c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.520+0000 7f4ddbfff640 1 --2- 172.21.15.40:0/2570930246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ddc0b0630 0x7f4ddc0b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.520+0000 7f4ddbfff640 1 --2- 172.21.15.40:0/2570930246 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ddc06d6f0 0x7f4ddc0b15b0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.520+0000 7f4ddbfff640 1 -- 172.21.15.40:0/2570930246 >> 172.21.15.40:0/2570930246 conn(0x7f4ddc069720 msgr2=0x7f4ddc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:59.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.520+0000 7f4ddbfff640 1 -- 172.21.15.40:0/2570930246 shutdown_connections 2023-10-18T09:09:59.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.520+0000 7f4ddbfff640 1 -- 172.21.15.40:0/2570930246 wait complete. 2023-10-18T09:09:59.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.522+0000 7f4ddbfff640 1 Processor -- start 2023-10-18T09:09:59.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.522+0000 7f4ddbfff640 1 -- start start 2023-10-18T09:09:59.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.523+0000 7f4ddbfff640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ddc06d6f0 0x7f4ddc14c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:59.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.523+0000 7f4ddbfff640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ddc0b0630 0x7f4ddc0aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:59.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.523+0000 7f4ddbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ddc0c99c0 0x7f4ddc0cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:59.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.523+0000 7f4ddbfff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc158310 con 0x7f4ddc0c99c0 2023-10-18T09:09:59.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.523+0000 7f4ddbfff640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc0ab160 con 0x7f4ddc0b0630 2023-10-18T09:09:59.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.523+0000 7f4ddbfff640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc0ab440 con 0x7f4ddc06d6f0 2023-10-18T09:09:59.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.523+0000 7f4ddb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ddc0c99c0 0x7f4ddc0cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:59.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.523+0000 7f4ddb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ddc0c99c0 0x7f4ddc0cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38088/0 (socket says 172.21.15.40:38088) 2023-10-18T09:09:59.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.523+0000 7f4ddb7fe640 1 -- 172.21.15.40:0/1294868372 learned_addr learned my addr 172.21.15.40:0/1294868372 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:09:59.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.523+0000 7f4dd27fc640 1 --2- 172.21.15.40:0/1294868372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ddc0b0630 0x7f4ddc0aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:59.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.523+0000 7f4ddaffd640 1 --2- 172.21.15.40:0/1294868372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ddc06d6f0 0x7f4ddc14c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:59.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.524+0000 7f4ddb7fe640 1 -- 172.21.15.40:0/1294868372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ddc06d6f0 msgr2=0x7f4ddc14c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:59.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.524+0000 7f4ddb7fe640 1 --2- 172.21.15.40:0/1294868372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ddc06d6f0 0x7f4ddc14c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.524+0000 7f4ddb7fe640 1 -- 172.21.15.40:0/1294868372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ddc0b0630 msgr2=0x7f4ddc0aa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:59.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.524+0000 7f4ddb7fe640 1 --2- 172.21.15.40:0/1294868372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ddc0b0630 0x7f4ddc0aa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.524+0000 7f4ddb7fe640 1 -- 172.21.15.40:0/1294868372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc009900 con 0x7f4ddc0c99c0 2023-10-18T09:09:59.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.524+0000 7f4ddb7fe640 1 --2- 172.21.15.40:0/1294868372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ddc0c99c0 0x7f4ddc0cb3e0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400b410 tx=0x7f4dd400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:59.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.524+0000 7f4dd3fff640 1 -- 172.21.15.40:0/1294868372 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4011830 con 0x7f4ddc0c99c0 2023-10-18T09:09:59.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.524+0000 7f4dd3fff640 1 -- 172.21.15.40:0/1294868372 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4dd4011aa0 con 0x7f4ddc0c99c0 2023-10-18T09:09:59.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.524+0000 7f4dd3fff640 1 -- 172.21.15.40:0/1294868372 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4010380 con 0x7f4ddc0c99c0 2023-10-18T09:09:59.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.524+0000 7f4ddbfff640 1 -- 172.21.15.40:0/1294868372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ddc157fa0 con 0x7f4ddc0c99c0 2023-10-18T09:09:59.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.524+0000 7f4ddbfff640 1 -- 172.21.15.40:0/1294868372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ddc0ab720 con 0x7f4ddc0c99c0 2023-10-18T09:09:59.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.526+0000 7f4ddbfff640 1 -- 172.21.15.40:0/1294868372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ddc158480 con 0x7f4ddc0c99c0 2023-10-18T09:09:59.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.527+0000 7f4dd3fff640 1 -- 172.21.15.40:0/1294868372 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4dd40104e0 con 0x7f4ddc0c99c0 2023-10-18T09:09:59.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.528+0000 7f4dd3fff640 1 --2- 172.21.15.40:0/1294868372 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4dbc06ae40 0x7f4dbc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:09:59.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.528+0000 7f4dd3fff640 1 -- 172.21.15.40:0/1294868372 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4dd40896b0 con 0x7f4ddc0c99c0 2023-10-18T09:09:59.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.528+0000 7f4ddaffd640 1 --2- 172.21.15.40:0/1294868372 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4dbc06ae40 0x7f4dbc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:09:59.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.529+0000 7f4ddaffd640 1 --2- 172.21.15.40:0/1294868372 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4dbc06ae40 0x7f4dbc06d300 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc036000 tx=0x7f4dcc004ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:09:59.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.532+0000 7f4dd3fff640 1 -- 172.21.15.40:0/1294868372 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4dd4054a60 con 0x7f4ddc0c99c0 2023-10-18T09:09:59.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.724+0000 7f4ddbfff640 1 -- 172.21.15.40:0/1294868372 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ddc0b0a90 con 0x7f4dbc06ae40 2023-10-18T09:09:59.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.730+0000 7f4dd3fff640 1 -- 172.21.15.40:0/1294868372 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47156 (secure 0 0 0) 0x7f4dcc003c20 con 0x7f4dbc06ae40 2023-10-18T09:09:59.734 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:09:59.734 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:09:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.733+0000 7f4ddbfff640 1 -- 172.21.15.40:0/1294868372 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4dbc06ae40 msgr2=0x7f4dbc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.733+0000 7f4ddbfff640 1 --2- 172.21.15.40:0/1294868372 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4dbc06ae40 0x7f4dbc06d300 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc036000 tx=0x7f4dcc004ab0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.733+0000 7f4ddbfff640 1 -- 172.21.15.40:0/1294868372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ddc0c99c0 msgr2=0x7f4ddc0cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:09:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.733+0000 7f4ddbfff640 1 --2- 172.21.15.40:0/1294868372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ddc0c99c0 0x7f4ddc0cb3e0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400b410 tx=0x7f4dd400b8d0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.733+0000 7f4ddbfff640 1 -- 172.21.15.40:0/1294868372 shutdown_connections 2023-10-18T09:09:59.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.733+0000 7f4ddbfff640 1 --2- 172.21.15.40:0/1294868372 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f4dbc06ae40 0x7f4dbc06d300 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.733+0000 7f4ddbfff640 1 --2- 172.21.15.40:0/1294868372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ddc0c99c0 0x7f4ddc0cb3e0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.733+0000 7f4ddbfff640 1 --2- 172.21.15.40:0/1294868372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ddc0b0630 0x7f4ddc0aa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.733+0000 7f4ddbfff640 1 --2- 172.21.15.40:0/1294868372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ddc06d6f0 0x7f4ddc14c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:09:59.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.733+0000 7f4ddbfff640 1 -- 172.21.15.40:0/1294868372 >> 172.21.15.40:0/1294868372 conn(0x7f4ddc069720 msgr2=0x7f4ddc15a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:09:59.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.734+0000 7f4ddbfff640 1 -- 172.21.15.40:0/1294868372 shutdown_connections 2023-10-18T09:09:59.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:09:59.734+0000 7f4ddbfff640 1 -- 172.21.15.40:0/1294868372 wait complete. 2023-10-18T09:09:59.755 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":644,"stamp":"2023-10-18T09:09:59.224058+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10194,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552305,"statfs":{"total":1600000000,"available":1589560465,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.987184"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":363,"reported_epoch":202,"state":"active+clean","last_fresh":"2023-10-18T09:09:46.006422+0000","last_change":"2023-10-18T08:55:06.276397+0000","last_active":"2023-10-18T09:09:46.006422+0000","last_peered":"2023-10-18T09:09:46.006422+0000","last_clean":"2023-10-18T09:09:46.006422+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:09:46.006422+0000","last_undegraded":"2023-10-18T09:09:46.006422+0000","last_fullsized":"2023-10-18T09:09:46.006422+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T08:55:05.040385+0000","last_clean_scrub_stamp":"2023-10-18T08:55:05.040385+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:34:05.821841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575028,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193605,"statfs":{"total":200000000,"available":198251900,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1140000000000001,"15min":1.0840000000000001},"min":{"1min":0.69499999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6990000000000001},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.129,"15min":1.0940000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.63100000000000001},"max":{"1min":1.5589999999999999,"5min":1.724,"15min":1.97},"last":1.107}]},{"osd":1,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1910000000000001,"15min":1.165},"min":{"1min":0.73699999999999999,"5min":0.68899999999999995,"15min":0.61799999999999999},"max":{"1min":1.4950000000000001,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.1539999999999999,"15min":1.135},"min":{"1min":0.72799999999999998,"5min":0.63500000000000001,"15min":0.58799999999999997},"max":{"1min":1.571,"5min":1.782,"15min":1.7989999999999999},"last":1.1719999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.351,"15min":1.3109999999999999},"min":{"1min":0.81299999999999994,"5min":0.80700000000000005,"15min":0.65400000000000003},"max":{"1min":1.794,"5min":1.8140000000000001,"15min":1.9730000000000001},"last":1.264},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2470000000000001,"15min":1.244},"min":{"1min":0.79500000000000004,"5min":0.72499999999999998,"15min":0.68300000000000005},"max":{"1min":1.6870000000000001,"5min":1.8169999999999999,"15min":1.863},"last":1.427}]},{"osd":3,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.3480000000000001,"15min":1.294},"min":{"1min":0.97099999999999997,"5min":0.90300000000000002,"15min":0.61299999999999999},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.98},"last":1.456},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3979999999999999,"15min":1.359},"min":{"1min":0.88200000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8640000000000001,"5min":1.9219999999999999,"15min":1.9930000000000001},"last":1.3480000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2410000000000001,"15min":1.228},"min":{"1min":0.89500000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.7390000000000001,"5min":1.869,"15min":2.1070000000000002},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.3300000000000001,"15min":1.3400000000000001},"min":{"1min":0.96299999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7470000000000001,"5min":1.9570000000000001,"15min":2.5169999999999999},"last":1.228}]},{"osd":5,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3480000000000001,"15min":1.345},"min":{"1min":0.78500000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":2.0720000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.393,"15min":1.4079999999999999},"min":{"1min":0.92000000000000004,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.968,"5min":1.968,"15min":2.0270000000000001},"last":1.298}]},{"osd":6,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.484,"15min":1.482},"min":{"1min":0.83699999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.3959999999999999},"last":1.524},{"interface":"front","average":{"1min":1.528,"5min":1.474,"15min":1.4850000000000001},"min":{"1min":1.1879999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.8959999999999999,"5min":2.0680000000000001,"15min":2.1680000000000001},"last":1.3879999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2250000000000001,"15min":1.2050000000000001},"min":{"1min":0.80900000000000005,"5min":0.64200000000000002,"15min":0.60299999999999998},"max":{"1min":1.575,"5min":1.643,"15min":1.6850000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.327,"5min":1.3799999999999999,"15min":1.325},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.64700000000000002},"max":{"1min":1.796,"5min":1.899,"15min":2.0990000000000002},"last":1.0269999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.181,"15min":1.127},"min":{"1min":0.81499999999999995,"5min":0.69899999999999995,"15min":0.63800000000000001},"max":{"1min":1.605,"5min":1.605,"15min":1.722},"last":1.202},{"interface":"front","average":{"1min":1.173,"5min":1.2430000000000001,"15min":1.2150000000000001},"min":{"1min":0.76200000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.627,"5min":1.8169999999999999,"15min":2.9609999999999999},"last":1.415}]},{"osd":2,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.234,"15min":1.1950000000000001},"min":{"1min":0.85199999999999998,"5min":0.70299999999999996,"15min":0.69899999999999995},"max":{"1min":1.4790000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3460000000000001,"15min":1.2629999999999999},"min":{"1min":0.86399999999999999,"5min":0.748,"15min":0.66300000000000003},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":2.1779999999999999},"last":1.1419999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.3220000000000001,"15min":1.288},"min":{"1min":1.196,"5min":0.82999999999999996,"15min":0.79200000000000004},"max":{"1min":1.587,"5min":1.8160000000000001,"15min":2.1589999999999998},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.355,"15min":1.3009999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.78600000000000003},"max":{"1min":1.617,"5min":1.8540000000000001,"15min":1.994},"last":1.3580000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.3029999999999999,"15min":1.272},"min":{"1min":1.075,"5min":0.80600000000000005,"15min":0.58299999999999996},"max":{"1min":1.8500000000000001,"5min":1.9770000000000001,"15min":2.444},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.363,"15min":1.3120000000000001},"min":{"1min":0.877,"5min":0.82999999999999996,"15min":0.55000000000000004},"max":{"1min":1.766,"5min":1.9159999999999999,"15min":2.4039999999999999},"last":1.4610000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4510000000000001,"15min":1.423},"min":{"1min":1.204,"5min":0.92900000000000005,"15min":0.58899999999999997},"max":{"1min":1.631,"5min":2.016,"15min":2.032},"last":1.544},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3680000000000001,"15min":1.3260000000000001},"min":{"1min":1.087,"5min":0.77100000000000002,"15min":0.49199999999999999},"max":{"1min":1.748,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.5069999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.4950000000000001,"15min":1.4390000000000001},"min":{"1min":1.242,"5min":1.016,"15min":0.52000000000000002},"max":{"1min":1.913,"5min":2.028,"15min":2.3540000000000001},"last":1.573},{"interface":"front","average":{"1min":1.617,"5min":1.5640000000000001,"15min":1.5389999999999999},"min":{"1min":1.304,"5min":1.1890000000000001,"15min":0.68500000000000005},"max":{"1min":1.9670000000000001,"5min":2.1440000000000001,"15min":2.794},"last":1.649}]}]},{"osd":5,"up_from":13,"seq":55834575030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3200000000000001,"15min":1.3260000000000001},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.895,"5min":2.302,"15min":9.609},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3819999999999999,"15min":1.331},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.60799999999999998},"max":{"1min":1.714,"5min":2.0840000000000001,"15min":5.3029999999999999},"last":1.429}]},{"osd":1,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2370000000000001,"15min":1.256},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.792,"5min":2.0339999999999998,"15min":11.68},"last":1.466},{"interface":"front","average":{"1min":1.212,"5min":1.246,"15min":1.2130000000000001},"min":{"1min":0.74099999999999999,"5min":0.72799999999999998,"15min":0.67000000000000004},"max":{"1min":1.5009999999999999,"5min":1.79,"15min":3.7050000000000001},"last":1.087}]},{"osd":2,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.27,"15min":1.2529999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.755},"max":{"1min":1.788,"5min":1.8540000000000001,"15min":2.2090000000000001},"last":1.333},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2569999999999999,"15min":1.264},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.72399999999999998},"max":{"1min":1.6359999999999999,"5min":1.706,"15min":3.8180000000000001},"last":1.3700000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2110000000000001,"15min":1.256},"min":{"1min":0.79800000000000004,"5min":0.745,"15min":0.65400000000000003},"max":{"1min":1.466,"5min":1.6719999999999999,"15min":11.797000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.3740000000000001,"15min":1.365},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.747},"max":{"1min":1.6020000000000001,"5min":2.105,"15min":2.3420000000000001},"last":1.5700000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.341,"15min":1.373},"min":{"1min":1.004,"5min":0.755,"15min":0.755},"max":{"1min":1.7190000000000001,"5min":2.028,"15min":2.4449999999999998},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.409,"5min":1.3380000000000001,"15min":1.345},"min":{"1min":1.1319999999999999,"5min":0.67700000000000005,"15min":0.64100000000000001},"max":{"1min":1.7230000000000001,"5min":1.982,"15min":5.4580000000000002},"last":1.268}]},{"osd":6,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.246,"15min":1.2809999999999999},"min":{"1min":0.93600000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.667,"5min":1.7410000000000001,"15min":5.141},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5409999999999999,"15min":1.53},"min":{"1min":1.2589999999999999,"5min":1.095,"15min":0.71099999999999997},"max":{"1min":1.845,"5min":2.161,"15min":2.657},"last":1.2190000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.468,"15min":1.5389999999999999},"min":{"1min":1.1819999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.9319999999999999,"5min":1.976,"15min":11.829000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.395,"15min":1.4299999999999999},"min":{"1min":1.0509999999999999,"5min":0.96099999999999997,"15min":0.73399999999999999},"max":{"1min":1.5980000000000001,"5min":1.8560000000000001,"15min":4.8419999999999996},"last":1.3080000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194182,"statfs":{"total":200000000,"available":198843287,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.1699999999999999,"15min":1.1399999999999999},"min":{"1min":0.73499999999999999,"5min":0.66300000000000003,"15min":0.45900000000000002},"max":{"1min":1.669,"5min":1.7410000000000001,"15min":3.1819999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.165,"5min":1.1930000000000001,"15min":1.155},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.56899999999999995},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":3.234},"last":0.82299999999999995}]},{"osd":1,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1870000000000001,"15min":1.1479999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.60799999999999998},"max":{"1min":1.5209999999999999,"5min":1.9690000000000001,"15min":3.718},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.206,"15min":1.206},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.51400000000000001},"max":{"1min":1.802,"5min":1.9259999999999999,"15min":3.4350000000000001},"last":1.085}]},{"osd":2,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.236,"15min":1.216},"min":{"1min":0.79100000000000004,"5min":0.76200000000000001,"15min":0.56699999999999995},"max":{"1min":1.518,"5min":1.905,"15min":4.1749999999999998},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2769999999999999,"15min":1.294},"min":{"1min":0.96799999999999997,"5min":0.64500000000000002,"15min":0.58899999999999997},"max":{"1min":1.601,"5min":1.6919999999999999,"15min":3.532},"last":1.2310000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.323,"15min":1.2889999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.67000000000000004},"max":{"1min":1.754,"5min":1.754,"15min":3.363},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3879999999999999,"15min":1.399},"min":{"1min":0.96199999999999997,"5min":0.92200000000000004,"15min":0.68899999999999995},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":4.3179999999999996},"last":1.2969999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3,"15min":1.3049999999999999},"min":{"1min":1.161,"5min":0.55600000000000005,"15min":0.498},"max":{"1min":1.611,"5min":1.841,"15min":3.6160000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.331,"15min":1.325},"min":{"1min":0.95899999999999996,"5min":0.60099999999999998,"15min":0.38400000000000001},"max":{"1min":1.6799999999999999,"5min":1.702,"15min":4.0780000000000003},"last":1.1339999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.349,"15min":1.3939999999999999},"min":{"1min":1.1859999999999999,"5min":0.67400000000000004,"15min":0.51600000000000001},"max":{"1min":1.6299999999999999,"5min":1.766,"15min":5.0049999999999999},"last":1.494},{"interface":"front","average":{"1min":1.494,"5min":1.4399999999999999,"15min":1.4830000000000001},"min":{"1min":1.0860000000000001,"5min":0.47499999999999998,"15min":0.45400000000000001},"max":{"1min":1.7290000000000001,"5min":1.827,"15min":4.407},"last":1.2809999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.4430000000000001,"15min":1.452},"min":{"1min":1.0129999999999999,"5min":0.51200000000000001,"15min":0.438},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":4.5170000000000003},"last":1.202},{"interface":"front","average":{"1min":1.579,"5min":1.5069999999999999,"15min":1.5349999999999999},"min":{"1min":1.323,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.7949999999999999,"5min":1.948,"15min":4.5460000000000003},"last":1.593}]}]},{"osd":3,"up_from":13,"seq":55834575030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.89300000000000002,"15min":0.90600000000000003},"min":{"1min":0.45800000000000002,"5min":0.42699999999999999,"15min":0.35899999999999999},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":5.5730000000000004},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.028,"15min":0.98499999999999999},"min":{"1min":0.58399999999999996,"5min":0.53200000000000003,"15min":0.32700000000000001},"max":{"1min":1.665,"5min":1.863,"15min":6.5599999999999996},"last":0.88800000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0600000000000001,"15min":1.0229999999999999},"min":{"1min":0.63100000000000001,"5min":0.52200000000000002,"15min":0.36799999999999999},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":6.8099999999999996},"last":1.234},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.335,"15min":1.258},"min":{"1min":0.61299999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":1.2869999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.1659999999999999,"15min":1.117},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":1.7150000000000001,"5min":1.9550000000000001,"15min":7.4580000000000002},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.276,"15min":1.2070000000000001},"min":{"1min":0.65100000000000002,"5min":0.60699999999999998,"15min":0.51400000000000001},"max":{"1min":2.407,"5min":2.407,"15min":5.0049999999999999},"last":1.405}]},{"osd":4,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.385,"15min":1.3999999999999999},"min":{"1min":0.87,"5min":0.79300000000000004,"15min":0.53700000000000003},"max":{"1min":1.8100000000000001,"5min":2.0859999999999999,"15min":18.664999999999999},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.6200000000000001,"5min":1.427,"15min":1.327},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.64000000000000001},"max":{"1min":7.1269999999999998,"5min":7.1269999999999998,"15min":7.1269999999999998},"last":1.5589999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.3919999999999999,"15min":1.343},"min":{"1min":0.86599999999999999,"5min":0.84299999999999997,"15min":0.626},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":6.9139999999999997},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.514,"15min":1.506},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.56899999999999995},"max":{"1min":2.5640000000000001,"5min":2.6499999999999999,"15min":16.390999999999998},"last":1.133}]},{"osd":6,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.5569999999999999,"15min":1.512},"min":{"1min":1.0820000000000001,"5min":0.998,"15min":0.58199999999999996},"max":{"1min":2.1040000000000001,"5min":2.2839999999999998,"15min":7.7270000000000003},"last":1.524},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.5469999999999999,"15min":1.492},"min":{"1min":1.069,"5min":1.0049999999999999,"15min":0.60199999999999998},"max":{"1min":1.9410000000000001,"5min":2.0089999999999999,"15min":7.242},"last":1.4670000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.6240000000000001,"15min":1.5489999999999999},"min":{"1min":1.0329999999999999,"5min":0.89600000000000002,"15min":0.70099999999999996},"max":{"1min":2.5099999999999998,"5min":2.8170000000000002,"15min":4.8630000000000004},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.756,"5min":1.6659999999999999,"15min":1.6020000000000001},"min":{"1min":0.84699999999999998,"5min":0.83499999999999996,"15min":0.622},"max":{"1min":5.7649999999999997,"5min":5.7649999999999997,"15min":12.946},"last":1.4379999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.84199999999999997,"15min":0.86499999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.33700000000000002},"max":{"1min":1.464,"5min":1.6000000000000001,"15min":4.2210000000000001},"last":1.228},{"interface":"front","average":{"1min":0.875,"5min":0.99099999999999999,"15min":1},"min":{"1min":0.50800000000000001,"5min":0.497,"15min":0.33800000000000002},"max":{"1min":1.3440000000000001,"5min":2.2799999999999998,"15min":4.3959999999999999},"last":1.0009999999999999}]},{"osd":1,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.97399999999999998,"15min":1.0449999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1699999999999999,"5min":1.6100000000000001,"15min":7.3390000000000004},"last":1.4510000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":1.087,"15min":1.1020000000000001},"min":{"1min":0.61099999999999999,"5min":0.55900000000000005,"15min":0.46200000000000002},"max":{"1min":1.4139999999999999,"5min":1.696,"15min":2.5350000000000001},"last":1.415}]},{"osd":3,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.115,"15min":1.121},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.847,"5min":1.847,"15min":7.6369999999999996},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.246,"5min":1.3240000000000001,"15min":1.341},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.54100000000000004},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":7.5570000000000004},"last":1.6759999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3939999999999999,"15min":1.4179999999999999},"min":{"1min":0.99299999999999999,"5min":0.93400000000000005,"15min":0.77900000000000003},"max":{"1min":1.9239999999999999,"5min":2.016,"15min":7.6189999999999998},"last":1.609},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.397,"15min":1.4219999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.736,"5min":2.0430000000000001,"15min":3.423},"last":1.113}]},{"osd":5,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.26,"15min":1.298},"min":{"1min":0.89000000000000001,"5min":0.82399999999999995,"15min":0.61599999999999999},"max":{"1min":1.601,"5min":2.0600000000000001,"15min":10.507},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.29,"5min":1.377,"15min":1.4379999999999999},"min":{"1min":0.93600000000000005,"5min":0.873,"15min":0.81399999999999995},"max":{"1min":1.788,"5min":1.984,"15min":7.657},"last":2.0790000000000002}]},{"osd":6,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.419,"15min":1.45},"min":{"1min":0.84599999999999997,"5min":0.77700000000000002,"15min":0.74299999999999999},"max":{"1min":1.772,"5min":2.2519999999999998,"15min":3.5299999999999998},"last":1.306},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.4239999999999999,"15min":1.599},"min":{"1min":0.84399999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.835,"5min":2.2639999999999998,"15min":37.771999999999998},"last":1.048}]},{"osd":7,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.5069999999999999,"15min":1.53},"min":{"1min":0.872,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9330000000000001,"5min":2.3860000000000001,"15min":3.3620000000000001},"last":1.494},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.482,"15min":1.512},"min":{"1min":0.86299999999999999,"5min":0.84299999999999997,"15min":0.77600000000000002},"max":{"1min":1.96,"5min":2.375,"15min":10.956},"last":1.3640000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194182,"statfs":{"total":200000000,"available":198843287,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.91300000000000003,"15min":0.89300000000000002},"min":{"1min":0.57199999999999995,"5min":0.378,"15min":0.315},"max":{"1min":1.3740000000000001,"5min":1.4770000000000001,"15min":1.659},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1379999999999999,"15min":1.0580000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.441},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.73899999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0720000000000001,"15min":1.018},"min":{"1min":0.49199999999999999,"5min":0.48399999999999999,"15min":0.317},"max":{"1min":1.357,"5min":1.6100000000000001,"15min":1.859},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.97699999999999998,"15min":0.94499999999999995},"min":{"1min":0.55300000000000005,"5min":0.443,"15min":0.36699999999999999},"max":{"1min":1.3580000000000001,"5min":1.8009999999999999,"15min":2.0230000000000001},"last":0.51200000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.181,"15min":1.1040000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.436},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.085,"15min":1.073},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.35399999999999998},"max":{"1min":1.4370000000000001,"5min":1.6659999999999999,"15min":3.7010000000000001},"last":0.68200000000000005}]},{"osd":4,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.296,"15min":1.2929999999999999},"min":{"1min":0.81499999999999995,"5min":0.75600000000000001,"15min":0.45700000000000002},"max":{"1min":1.97,"5min":1.97,"15min":2.0489999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.335,"15min":1.3129999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.59699999999999998},"max":{"1min":1.8149999999999999,"5min":1.877,"15min":2.0470000000000002},"last":1.0820000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.478,"15min":1.401},"min":{"1min":0.90400000000000003,"5min":0.86499999999999999,"15min":0.56000000000000005},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":3.5750000000000002},"last":1.331},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.476,"15min":1.448},"min":{"1min":0.90000000000000002,"5min":0.86399999999999999,"15min":0.71699999999999997},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.3769999999999998},"last":1.3700000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4099999999999999,"15min":1.3839999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.64200000000000002},"max":{"1min":2.044,"5min":2.044,"15min":2.2050000000000001},"last":1.276},{"interface":"front","average":{"1min":1.46,"5min":1.4610000000000001,"15min":1.3959999999999999},"min":{"1min":0.92900000000000005,"5min":0.89000000000000001,"15min":0.63100000000000001},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.028}]},{"osd":7,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.538,"15min":1.4730000000000001},"min":{"1min":0.95399999999999996,"5min":0.85699999999999998,"15min":0.69799999999999995},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.421,"5min":1.4790000000000001,"15min":1.4590000000000001},"min":{"1min":1,"5min":0.81100000000000005,"15min":0.66300000000000003},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.4180000000000001},"last":1.129}]}]},{"osd":0,"up_from":13,"seq":55834575029,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193606,"statfs":{"total":200000000,"available":198252919,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.52100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.1779999999999999,"5min":1.2410000000000001,"15min":3.4420000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.071,"5min":1.091,"15min":1.089},"min":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.476,"5min":1.657,"15min":1.8240000000000001},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.90200000000000002,"15min":0.85999999999999999},"min":{"1min":0.59799999999999998,"5min":0.56299999999999994,"15min":0.375},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.679},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2290000000000001,"15min":1.161},"min":{"1min":0.78400000000000003,"5min":0.623,"15min":0.437},"max":{"1min":1.607,"5min":1.917,"15min":1.929},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.167,"15min":1.1599999999999999},"min":{"1min":0.56799999999999995,"5min":0.52100000000000002,"15min":0.49399999999999999},"max":{"1min":1.7210000000000001,"5min":1.9590000000000001,"15min":2.7639999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.4159999999999999,"15min":1.413},"min":{"1min":0.85099999999999998,"5min":0.68100000000000005,"15min":0.58399999999999996},"max":{"1min":1.77,"5min":2.0150000000000001,"15min":3.5720000000000001},"last":0.752}]},{"osd":4,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2170000000000001,"15min":1.2270000000000001},"min":{"1min":0.80800000000000005,"5min":0.73099999999999998,"15min":0.71099999999999997},"max":{"1min":1.6579999999999999,"5min":1.7370000000000001,"15min":4.492},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2829999999999999,"15min":1.274},"min":{"1min":0.93600000000000005,"5min":0.77500000000000002,"15min":0.68700000000000006},"max":{"1min":1.552,"5min":1.752,"15min":1.927},"last":1.077}]},{"osd":5,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.252,"15min":1.2889999999999999},"min":{"1min":0.97899999999999998,"5min":0.75600000000000001,"15min":0.64300000000000002},"max":{"1min":1.625,"5min":1.6439999999999999,"15min":4.2779999999999996},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.347,"15min":1.3340000000000001},"min":{"1min":1.03,"5min":0.81899999999999995,"15min":0.73099999999999998},"max":{"1min":1.583,"5min":1.8169999999999999,"15min":1.8740000000000001},"last":0.82999999999999996}]},{"osd":6,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.48,"15min":1.4730000000000001},"min":{"1min":1.0620000000000001,"5min":0.81000000000000005,"15min":0.76800000000000002},"max":{"1min":2.0459999999999998,"5min":2.2469999999999999,"15min":3.5299999999999998},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.302,"5min":1.359,"15min":1.3819999999999999},"min":{"1min":0.95099999999999996,"5min":0.86899999999999999,"15min":0.66700000000000004},"max":{"1min":1.661,"5min":1.8720000000000001,"15min":4.4349999999999996},"last":0.95799999999999996}]},{"osd":7,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.5489999999999999,"15min":1.5569999999999999},"min":{"1min":1.1850000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.982,"5min":2.133,"15min":3.492},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.399,"5min":1.472,"15min":1.4950000000000001},"min":{"1min":0.95999999999999996,"5min":0.88500000000000001,"15min":0.82099999999999995},"max":{"1min":1.776,"5min":2.1000000000000001,"15min":4.5419999999999998},"last":0.871}]}]}],"pool_statfs":[]}} 2023-10-18T09:09:59.758 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-18T08:55:05.040385+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=8, tm_min=55, tm_sec=5, tm_wday=2, tm_yday=291, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=18, tm_hour=9, tm_min=9, tm_sec=52, tm_wday=2, tm_yday=291, tm_isdst=0) 2023-10-18T09:09:59.759 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-18T09:10:08.773 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:10:08.783 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:10:19.760 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:10:20.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.002+0000 7f23f8ffb640 1 -- 172.21.15.40:0/268240712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f406d6f0 msgr2=0x7f23f40b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:20.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.002+0000 7f23f8ffb640 1 --2- 172.21.15.40:0/268240712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f406d6f0 0x7f23f40b15b0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f23f406aeb0 tx=0x7f23e402bd60 comp rx=0 tx=0).stop 2023-10-18T09:10:20.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.002+0000 7f23f8ffb640 1 -- 172.21.15.40:0/268240712 shutdown_connections 2023-10-18T09:10:20.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.002+0000 7f23f8ffb640 1 --2- 172.21.15.40:0/268240712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23f40ca0d0 0x7f23f4157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.002+0000 7f23f8ffb640 1 --2- 172.21.15.40:0/268240712 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f406d6f0 0x7f23f40b15b0 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.002+0000 7f23f8ffb640 1 -- 172.21.15.40:0/268240712 >> 172.21.15.40:0/268240712 conn(0x7f23f4069720 msgr2=0x7f23f4069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:20.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.002+0000 7f23f8ffb640 1 -- 172.21.15.40:0/268240712 shutdown_connections 2023-10-18T09:10:20.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.002+0000 7f23f8ffb640 1 -- 172.21.15.40:0/268240712 wait complete. 2023-10-18T09:10:20.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.004+0000 7f23f8ffb640 1 Processor -- start 2023-10-18T09:10:20.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.004+0000 7f23f8ffb640 1 -- start start 2023-10-18T09:10:20.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.005+0000 7f23f8ffb640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23f406d6f0 0x7f23f4150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:20.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.005+0000 7f23f8ffb640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23f40b0630 0x7f23f414be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:20.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.005+0000 7f23f8ffb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f40ca0d0 0x7f23f4158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:20.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.005+0000 7f23f8ffb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f4151b70 con 0x7f23f40ca0d0 2023-10-18T09:10:20.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.005+0000 7f23f8ffb640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f414c350 con 0x7f23f406d6f0 2023-10-18T09:10:20.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.005+0000 7f23f8ffb640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f4150ef0 con 0x7f23f40b0630 2023-10-18T09:10:20.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.005+0000 7f23f3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f40ca0d0 0x7f23f4158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:20.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.005+0000 7f23f3fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f40ca0d0 0x7f23f4158990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45304/0 (socket says 172.21.15.40:45304) 2023-10-18T09:10:20.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.005+0000 7f23f3fff640 1 -- 172.21.15.40:0/1704491981 learned_addr learned my addr 172.21.15.40:0/1704491981 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:10:20.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.006+0000 7f23f37fe640 1 --2- 172.21.15.40:0/1704491981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23f406d6f0 0x7f23f4150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:20.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.006+0000 7f23ebfff640 1 --2- 172.21.15.40:0/1704491981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23f40b0630 0x7f23f414be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:20.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.006+0000 7f23f3fff640 1 -- 172.21.15.40:0/1704491981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23f40b0630 msgr2=0x7f23f414be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:20.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.006+0000 7f23f3fff640 1 --2- 172.21.15.40:0/1704491981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23f40b0630 0x7f23f414be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.006+0000 7f23f3fff640 1 -- 172.21.15.40:0/1704491981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23f406d6f0 msgr2=0x7f23f4150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:20.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.006+0000 7f23f3fff640 1 --2- 172.21.15.40:0/1704491981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23f406d6f0 0x7f23f4150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.006+0000 7f23f3fff640 1 -- 172.21.15.40:0/1704491981 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e4009560 con 0x7f23f40ca0d0 2023-10-18T09:10:20.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.006+0000 7f23ebfff640 1 --2- 172.21.15.40:0/1704491981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23f40b0630 0x7f23f414be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:10:20.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.006+0000 7f23f3fff640 1 --2- 172.21.15.40:0/1704491981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f40ca0d0 0x7f23f4158990 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00ddf0 tx=0x7f23ec00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:20.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.006+0000 7f23f0ff9640 1 -- 172.21.15.40:0/1704491981 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec00dee0 con 0x7f23f40ca0d0 2023-10-18T09:10:20.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.007+0000 7f23f0ff9640 1 -- 172.21.15.40:0/1704491981 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f23ec004e10 con 0x7f23f40ca0d0 2023-10-18T09:10:20.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.007+0000 7f23f0ff9640 1 -- 172.21.15.40:0/1704491981 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec00e8c0 con 0x7f23f40ca0d0 2023-10-18T09:10:20.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.007+0000 7f23f8ffb640 1 -- 172.21.15.40:0/1704491981 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f41511d0 con 0x7f23f40ca0d0 2023-10-18T09:10:20.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.007+0000 7f23f8ffb640 1 -- 172.21.15.40:0/1704491981 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f4151790 con 0x7f23f40ca0d0 2023-10-18T09:10:20.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.008+0000 7f23f8ffb640 1 -- 172.21.15.40:0/1704491981 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f4151cb0 con 0x7f23f40ca0d0 2023-10-18T09:10:20.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.009+0000 7f23f0ff9640 1 -- 172.21.15.40:0/1704491981 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f23ec00ea20 con 0x7f23f40ca0d0 2023-10-18T09:10:20.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.010+0000 7f23f0ff9640 1 --2- 172.21.15.40:0/1704491981 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f23d406ae70 0x7f23d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:20.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.010+0000 7f23f0ff9640 1 -- 172.21.15.40:0/1704491981 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f23ec089dc0 con 0x7f23f40ca0d0 2023-10-18T09:10:20.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.010+0000 7f23f37fe640 1 --2- 172.21.15.40:0/1704491981 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f23d406ae70 0x7f23d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:20.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.012+0000 7f23f37fe640 1 --2- 172.21.15.40:0/1704491981 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f23d406ae70 0x7f23d406d330 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f23e4035000 tx=0x7f23e4004730 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:20.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.015+0000 7f23f0ff9640 1 -- 172.21.15.40:0/1704491981 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f23f4151cb0 con 0x7f23f40ca0d0 2023-10-18T09:10:20.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.205+0000 7f23f8ffb640 1 -- 172.21.15.40:0/1704491981 --> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f23f4151cb0 con 0x7f23d406ae70 2023-10-18T09:10:20.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.212+0000 7f23f0ff9640 1 -- 172.21.15.40:0/1704491981 <== mgr.4114 v2:172.21.15.144:6800/3843888630 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47012 (secure 0 0 0) 0x7f23f4151cb0 con 0x7f23d406ae70 2023-10-18T09:10:20.216 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:10:20.217 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-18T09:10:20.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.215+0000 7f23f8ffb640 1 -- 172.21.15.40:0/1704491981 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f23d406ae70 msgr2=0x7f23d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:20.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.215+0000 7f23f8ffb640 1 --2- 172.21.15.40:0/1704491981 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f23d406ae70 0x7f23d406d330 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f23e4035000 tx=0x7f23e4004730 comp rx=0 tx=0).stop 2023-10-18T09:10:20.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.215+0000 7f23f8ffb640 1 -- 172.21.15.40:0/1704491981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f40ca0d0 msgr2=0x7f23f4158990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:20.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.215+0000 7f23f8ffb640 1 --2- 172.21.15.40:0/1704491981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f40ca0d0 0x7f23f4158990 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00ddf0 tx=0x7f23ec00b730 comp rx=0 tx=0).stop 2023-10-18T09:10:20.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.215+0000 7f23f8ffb640 1 -- 172.21.15.40:0/1704491981 shutdown_connections 2023-10-18T09:10:20.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.215+0000 7f23f8ffb640 1 --2- 172.21.15.40:0/1704491981 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f23d406ae70 0x7f23d406d330 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.215+0000 7f23f8ffb640 1 --2- 172.21.15.40:0/1704491981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23f40ca0d0 0x7f23f4158990 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.215+0000 7f23f8ffb640 1 --2- 172.21.15.40:0/1704491981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23f40b0630 0x7f23f414be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.215+0000 7f23f8ffb640 1 --2- 172.21.15.40:0/1704491981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23f406d6f0 0x7f23f4150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.215+0000 7f23f8ffb640 1 -- 172.21.15.40:0/1704491981 >> 172.21.15.40:0/1704491981 conn(0x7f23f4069720 msgr2=0x7f23f4154670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:20.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.215+0000 7f23f8ffb640 1 -- 172.21.15.40:0/1704491981 shutdown_connections 2023-10-18T09:10:20.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.216+0000 7f23f8ffb640 1 -- 172.21.15.40:0/1704491981 wait complete. 2023-10-18T09:10:20.239 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":654,"stamp":"2023-10-18T09:10:19.229551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10194,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552305,"statfs":{"total":1600000000,"available":1589560465,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003263"},"pg_stats":[{"pgid":"1.0","version":"203'119","reported_seq":374,"reported_epoch":203,"state":"active+clean","last_fresh":"2023-10-18T09:10:00.889839+0000","last_change":"2023-10-18T09:10:00.889949+0000","last_active":"2023-10-18T09:10:00.889839+0000","last_peered":"2023-10-18T09:10:00.889839+0000","last_clean":"2023-10-18T09:10:00.889839+0000","last_became_active":"2023-10-18T08:55:06.275922+0000","last_became_peered":"2023-10-18T08:55:06.275922+0000","last_unstale":"2023-10-18T09:10:00.889839+0000","last_undegraded":"2023-10-18T09:10:00.889839+0000","last_fullsized":"2023-10-18T09:10:00.889839+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"203'119","last_scrub_stamp":"2023-10-18T09:10:00.889746+0000","last_deep_scrub":"203'119","last_deep_scrub_stamp":"2023-10-18T09:10:00.889746+0000","last_clean_scrub_stamp":"2023-10-18T09:10:00.889746+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:46:06.014568+0000","scrub_duration":0.0062323509999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575032,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193605,"statfs":{"total":200000000,"available":198251900,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1140000000000001,"15min":1.0840000000000001},"min":{"1min":0.69499999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6990000000000001},"last":1.345},{"interface":"front","average":{"1min":1.044,"5min":1.129,"15min":1.0940000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.63100000000000001},"max":{"1min":1.5589999999999999,"5min":1.724,"15min":1.97},"last":1.109}]},{"osd":1,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1910000000000001,"15min":1.165},"min":{"1min":0.73699999999999999,"5min":0.68899999999999995,"15min":0.61799999999999999},"max":{"1min":1.4950000000000001,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.1539999999999999,"15min":1.135},"min":{"1min":0.72799999999999998,"5min":0.63500000000000001,"15min":0.58799999999999997},"max":{"1min":1.571,"5min":1.782,"15min":1.7989999999999999},"last":0.997}]},{"osd":2,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.351,"15min":1.3109999999999999},"min":{"1min":0.81299999999999994,"5min":0.80700000000000005,"15min":0.65400000000000003},"max":{"1min":1.794,"5min":1.8140000000000001,"15min":1.9730000000000001},"last":1.234},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2470000000000001,"15min":1.244},"min":{"1min":0.79500000000000004,"5min":0.72499999999999998,"15min":0.68300000000000005},"max":{"1min":1.6870000000000001,"5min":1.8169999999999999,"15min":1.863},"last":1.3759999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.3480000000000001,"15min":1.294},"min":{"1min":0.97099999999999997,"5min":0.90300000000000002,"15min":0.61299999999999999},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.98},"last":1.446},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3979999999999999,"15min":1.359},"min":{"1min":0.88200000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8640000000000001,"5min":1.9219999999999999,"15min":1.9930000000000001},"last":1.173}]},{"osd":4,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2410000000000001,"15min":1.228},"min":{"1min":0.89500000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.7390000000000001,"5min":1.869,"15min":2.1070000000000002},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.337,"5min":1.3300000000000001,"15min":1.3400000000000001},"min":{"1min":0.96299999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7470000000000001,"5min":1.9570000000000001,"15min":2.5169999999999999},"last":1.258}]},{"osd":5,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3480000000000001,"15min":1.345},"min":{"1min":0.78500000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":2.0720000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.393,"15min":1.4079999999999999},"min":{"1min":0.92000000000000004,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.968,"5min":1.968,"15min":2.0270000000000001},"last":1.431}]},{"osd":6,"last update":"Wed Oct 18 09:09:36 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.484,"15min":1.482},"min":{"1min":0.83699999999999997,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.3959999999999999},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.528,"5min":1.474,"15min":1.4850000000000001},"min":{"1min":1.1879999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.8959999999999999,"5min":2.0680000000000001,"15min":2.1680000000000001},"last":1.5289999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575033,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2250000000000001,"15min":1.2050000000000001},"min":{"1min":0.80900000000000005,"5min":0.64200000000000002,"15min":0.60299999999999998},"max":{"1min":1.575,"5min":1.643,"15min":1.6850000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.327,"5min":1.3799999999999999,"15min":1.325},"min":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.64700000000000002},"max":{"1min":1.796,"5min":1.899,"15min":2.0990000000000002},"last":1.6140000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.181,"15min":1.127},"min":{"1min":0.81499999999999995,"5min":0.69899999999999995,"15min":0.63800000000000001},"max":{"1min":1.605,"5min":1.605,"15min":1.722},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.173,"5min":1.2430000000000001,"15min":1.2150000000000001},"min":{"1min":0.76200000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.627,"5min":1.8169999999999999,"15min":2.9609999999999999},"last":1.7010000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.234,"15min":1.1950000000000001},"min":{"1min":0.85199999999999998,"5min":0.70299999999999996,"15min":0.69899999999999995},"max":{"1min":1.4790000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3460000000000001,"15min":1.2629999999999999},"min":{"1min":0.86399999999999999,"5min":0.748,"15min":0.66300000000000003},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":2.1779999999999999},"last":1.3520000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.3220000000000001,"15min":1.288},"min":{"1min":1.196,"5min":0.82999999999999996,"15min":0.79200000000000004},"max":{"1min":1.587,"5min":1.8160000000000001,"15min":2.1589999999999998},"last":1.534},{"interface":"front","average":{"1min":1.286,"5min":1.355,"15min":1.3009999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.78600000000000003},"max":{"1min":1.617,"5min":1.8540000000000001,"15min":1.994},"last":1.4299999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.3029999999999999,"15min":1.272},"min":{"1min":1.075,"5min":0.80600000000000005,"15min":0.58299999999999996},"max":{"1min":1.8500000000000001,"5min":1.9770000000000001,"15min":2.444},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.363,"15min":1.3120000000000001},"min":{"1min":0.877,"5min":0.82999999999999996,"15min":0.55000000000000004},"max":{"1min":1.766,"5min":1.9159999999999999,"15min":2.4039999999999999},"last":1.4690000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.4510000000000001,"15min":1.423},"min":{"1min":1.204,"5min":0.92900000000000005,"15min":0.58899999999999997},"max":{"1min":1.631,"5min":2.016,"15min":2.032},"last":1.216},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3680000000000001,"15min":1.3260000000000001},"min":{"1min":1.087,"5min":0.77100000000000002,"15min":0.49199999999999999},"max":{"1min":1.748,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.5680000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:09:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.4950000000000001,"15min":1.4390000000000001},"min":{"1min":1.242,"5min":1.016,"15min":0.52000000000000002},"max":{"1min":1.913,"5min":2.028,"15min":2.3540000000000001},"last":1.7909999999999999},{"interface":"front","average":{"1min":1.617,"5min":1.5640000000000001,"15min":1.5389999999999999},"min":{"1min":1.304,"5min":1.1890000000000001,"15min":0.68500000000000005},"max":{"1min":1.9670000000000001,"5min":2.1440000000000001,"15min":2.794},"last":1.595}]}]},{"osd":5,"up_from":13,"seq":55834575034,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.3200000000000001,"15min":1.3260000000000001},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.895,"5min":2.302,"15min":9.609},"last":1.635},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3819999999999999,"15min":1.331},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.60799999999999998},"max":{"1min":1.714,"5min":2.0840000000000001,"15min":5.3029999999999999},"last":1.2030000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2370000000000001,"15min":1.256},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.792,"5min":2.0339999999999998,"15min":11.68},"last":1.306},{"interface":"front","average":{"1min":1.212,"5min":1.246,"15min":1.2130000000000001},"min":{"1min":0.74099999999999999,"5min":0.72799999999999998,"15min":0.67000000000000004},"max":{"1min":1.5009999999999999,"5min":1.79,"15min":3.7050000000000001},"last":1.4179999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.27,"15min":1.2529999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.755},"max":{"1min":1.788,"5min":1.8540000000000001,"15min":2.2090000000000001},"last":1.468},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2569999999999999,"15min":1.264},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.72399999999999998},"max":{"1min":1.6359999999999999,"5min":1.706,"15min":3.8180000000000001},"last":1.3360000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2110000000000001,"15min":1.256},"min":{"1min":0.79800000000000004,"5min":0.745,"15min":0.65400000000000003},"max":{"1min":1.466,"5min":1.6719999999999999,"15min":11.797000000000001},"last":1.048},{"interface":"front","average":{"1min":1.272,"5min":1.3740000000000001,"15min":1.365},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.747},"max":{"1min":1.6020000000000001,"5min":2.105,"15min":2.3420000000000001},"last":1.5680000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.341,"15min":1.373},"min":{"1min":1.004,"5min":0.755,"15min":0.755},"max":{"1min":1.7190000000000001,"5min":2.028,"15min":2.4449999999999998},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.3380000000000001,"15min":1.345},"min":{"1min":1.1319999999999999,"5min":0.67700000000000005,"15min":0.64100000000000001},"max":{"1min":1.7230000000000001,"5min":1.982,"15min":5.4580000000000002},"last":1.083}]},{"osd":6,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.246,"15min":1.2809999999999999},"min":{"1min":0.93600000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.667,"5min":1.7410000000000001,"15min":5.141},"last":1.129},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.5409999999999999,"15min":1.53},"min":{"1min":1.2589999999999999,"5min":1.095,"15min":0.71099999999999997},"max":{"1min":1.845,"5min":2.161,"15min":2.657},"last":1.3700000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:09:34 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.468,"15min":1.5389999999999999},"min":{"1min":1.1819999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.9319999999999999,"5min":1.976,"15min":11.829000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.395,"15min":1.4299999999999999},"min":{"1min":1.0509999999999999,"5min":0.96099999999999997,"15min":0.73399999999999999},"max":{"1min":1.5980000000000001,"5min":1.8560000000000001,"15min":4.8419999999999996},"last":1.244}]}]},{"osd":4,"up_from":13,"seq":55834575033,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194182,"statfs":{"total":200000000,"available":198843287,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.1699999999999999,"15min":1.1399999999999999},"min":{"1min":0.73499999999999999,"5min":0.66300000000000003,"15min":0.45900000000000002},"max":{"1min":1.669,"5min":1.7410000000000001,"15min":3.1819999999999999},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.1930000000000001,"15min":1.155},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.56899999999999995},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":3.234},"last":0.80900000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1870000000000001,"15min":1.1479999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.60799999999999998},"max":{"1min":1.5209999999999999,"5min":1.9690000000000001,"15min":3.718},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.206,"15min":1.206},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.51400000000000001},"max":{"1min":1.802,"5min":1.9259999999999999,"15min":3.4350000000000001},"last":1.3580000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.236,"15min":1.216},"min":{"1min":0.79100000000000004,"5min":0.76200000000000001,"15min":0.56699999999999995},"max":{"1min":1.518,"5min":1.905,"15min":4.1749999999999998},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.2769999999999999,"15min":1.294},"min":{"1min":0.96799999999999997,"5min":0.64500000000000002,"15min":0.58899999999999997},"max":{"1min":1.601,"5min":1.6919999999999999,"15min":3.532},"last":1.496}]},{"osd":3,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.323,"15min":1.2889999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.67000000000000004},"max":{"1min":1.754,"5min":1.754,"15min":3.363},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3879999999999999,"15min":1.399},"min":{"1min":0.96199999999999997,"5min":0.92200000000000004,"15min":0.68899999999999995},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":4.3179999999999996},"last":1.603}]},{"osd":5,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3,"15min":1.3049999999999999},"min":{"1min":1.161,"5min":0.55600000000000005,"15min":0.498},"max":{"1min":1.611,"5min":1.841,"15min":3.6160000000000001},"last":1.573},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.331,"15min":1.325},"min":{"1min":0.95899999999999996,"5min":0.60099999999999998,"15min":0.38400000000000001},"max":{"1min":1.6799999999999999,"5min":1.702,"15min":4.0780000000000003},"last":1.1319999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.349,"15min":1.3939999999999999},"min":{"1min":1.1859999999999999,"5min":0.67400000000000004,"15min":0.51600000000000001},"max":{"1min":1.6299999999999999,"5min":1.766,"15min":5.0049999999999999},"last":1.542},{"interface":"front","average":{"1min":1.494,"5min":1.4399999999999999,"15min":1.4830000000000001},"min":{"1min":1.0860000000000001,"5min":0.47499999999999998,"15min":0.45400000000000001},"max":{"1min":1.7290000000000001,"5min":1.827,"15min":4.407},"last":1.631}]},{"osd":7,"last update":"Wed Oct 18 09:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.4430000000000001,"15min":1.452},"min":{"1min":1.0129999999999999,"5min":0.51200000000000001,"15min":0.438},"max":{"1min":1.8320000000000001,"5min":1.8320000000000001,"15min":4.5170000000000003},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.579,"5min":1.5069999999999999,"15min":1.5349999999999999},"min":{"1min":1.323,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.7949999999999999,"5min":1.948,"15min":4.5460000000000003},"last":1.653}]}]},{"osd":3,"up_from":13,"seq":55834575034,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.89300000000000002,"15min":0.90600000000000003},"min":{"1min":0.45800000000000002,"5min":0.42699999999999999,"15min":0.35899999999999999},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":5.5730000000000004},"last":1.171},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.028,"15min":0.98499999999999999},"min":{"1min":0.58399999999999996,"5min":0.53200000000000003,"15min":0.32700000000000001},"max":{"1min":1.665,"5min":1.863,"15min":6.5599999999999996},"last":0.94899999999999995}]},{"osd":1,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0600000000000001,"15min":1.0229999999999999},"min":{"1min":0.63100000000000001,"5min":0.52200000000000002,"15min":0.36799999999999999},"max":{"1min":3.6680000000000001,"5min":3.6680000000000001,"15min":6.8099999999999996},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.335,"15min":1.258},"min":{"1min":0.61299999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":1.359}]},{"osd":2,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.1659999999999999,"15min":1.117},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45300000000000001},"max":{"1min":1.7150000000000001,"5min":1.9550000000000001,"15min":7.4580000000000002},"last":1.073},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.276,"15min":1.2070000000000001},"min":{"1min":0.65100000000000002,"5min":0.60699999999999998,"15min":0.51400000000000001},"max":{"1min":2.407,"5min":2.407,"15min":5.0049999999999999},"last":1.4390000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.385,"15min":1.3999999999999999},"min":{"1min":0.87,"5min":0.79300000000000004,"15min":0.53700000000000003},"max":{"1min":1.8100000000000001,"5min":2.0859999999999999,"15min":18.664999999999999},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.6200000000000001,"5min":1.427,"15min":1.327},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.64000000000000001},"max":{"1min":7.1269999999999998,"5min":7.1269999999999998,"15min":7.1269999999999998},"last":1.2430000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.3919999999999999,"15min":1.343},"min":{"1min":0.86599999999999999,"5min":0.84299999999999997,"15min":0.626},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":6.9139999999999997},"last":1.522},{"interface":"front","average":{"1min":1.496,"5min":1.514,"15min":1.506},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.56899999999999995},"max":{"1min":2.5640000000000001,"5min":2.6499999999999999,"15min":16.390999999999998},"last":1.498}]},{"osd":6,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.5569999999999999,"15min":1.512},"min":{"1min":1.0820000000000001,"5min":0.998,"15min":0.58199999999999996},"max":{"1min":2.1040000000000001,"5min":2.2839999999999998,"15min":7.7270000000000003},"last":1.581},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.5469999999999999,"15min":1.492},"min":{"1min":1.069,"5min":1.0049999999999999,"15min":0.60199999999999998},"max":{"1min":1.9410000000000001,"5min":2.0089999999999999,"15min":7.242},"last":1.409}]},{"osd":7,"last update":"Wed Oct 18 09:09:29 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.6240000000000001,"15min":1.5489999999999999},"min":{"1min":1.0329999999999999,"5min":0.89600000000000002,"15min":0.70099999999999996},"max":{"1min":2.5099999999999998,"5min":2.8170000000000002,"15min":4.8630000000000004},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.756,"5min":1.6659999999999999,"15min":1.6020000000000001},"min":{"1min":0.84699999999999998,"5min":0.83499999999999996,"15min":0.622},"max":{"1min":5.7649999999999997,"5min":5.7649999999999997,"15min":12.946},"last":1.3149999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575034,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194181,"statfs":{"total":200000000,"available":198842268,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.84199999999999997,"15min":0.86499999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.33700000000000002},"max":{"1min":1.464,"5min":1.6000000000000001,"15min":4.2210000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.99099999999999999,"15min":1},"min":{"1min":0.50800000000000001,"5min":0.497,"15min":0.33800000000000002},"max":{"1min":1.3440000000000001,"5min":2.2799999999999998,"15min":4.3959999999999999},"last":0.96399999999999997}]},{"osd":1,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.97399999999999998,"15min":1.0449999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1699999999999999,"5min":1.6100000000000001,"15min":7.3390000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":1.087,"15min":1.1020000000000001},"min":{"1min":0.61099999999999999,"5min":0.55900000000000005,"15min":0.46200000000000002},"max":{"1min":1.4139999999999999,"5min":1.696,"15min":2.5350000000000001},"last":1.1459999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.115,"15min":1.121},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.847,"5min":1.847,"15min":7.6369999999999996},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.246,"5min":1.3240000000000001,"15min":1.341},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.54100000000000004},"max":{"1min":2.0990000000000002,"5min":2.0990000000000002,"15min":7.5570000000000004},"last":1.0169999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3939999999999999,"15min":1.4179999999999999},"min":{"1min":0.99299999999999999,"5min":0.93400000000000005,"15min":0.77900000000000003},"max":{"1min":1.9239999999999999,"5min":2.016,"15min":7.6189999999999998},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.397,"15min":1.4219999999999999},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.736,"5min":2.0430000000000001,"15min":3.423},"last":1.6890000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.26,"15min":1.298},"min":{"1min":0.89000000000000001,"5min":0.82399999999999995,"15min":0.61599999999999999},"max":{"1min":1.601,"5min":2.0600000000000001,"15min":10.507},"last":1.282},{"interface":"front","average":{"1min":1.29,"5min":1.377,"15min":1.4379999999999999},"min":{"1min":0.93600000000000005,"5min":0.873,"15min":0.81399999999999995},"max":{"1min":1.788,"5min":1.984,"15min":7.657},"last":1.343}]},{"osd":6,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.419,"15min":1.45},"min":{"1min":0.84599999999999997,"5min":0.77700000000000002,"15min":0.74299999999999999},"max":{"1min":1.772,"5min":2.2519999999999998,"15min":3.5299999999999998},"last":1.407},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.4239999999999999,"15min":1.599},"min":{"1min":0.84399999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.835,"5min":2.2639999999999998,"15min":37.771999999999998},"last":1.6319999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.5069999999999999,"15min":1.53},"min":{"1min":0.872,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9330000000000001,"5min":2.3860000000000001,"15min":3.3620000000000001},"last":1.54},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.482,"15min":1.512},"min":{"1min":0.86299999999999999,"5min":0.84299999999999997,"15min":0.77600000000000002},"max":{"1min":1.96,"5min":2.375,"15min":10.956},"last":1.456}]}]},{"osd":1,"up_from":13,"seq":55834575033,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194182,"statfs":{"total":200000000,"available":198843287,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.91300000000000003,"15min":0.89300000000000002},"min":{"1min":0.57199999999999995,"5min":0.378,"15min":0.315},"max":{"1min":1.3740000000000001,"5min":1.4770000000000001,"15min":1.659},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1379999999999999,"15min":1.0580000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.441},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.238}]},{"osd":2,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":1.0720000000000001,"15min":1.018},"min":{"1min":0.49199999999999999,"5min":0.48399999999999999,"15min":0.317},"max":{"1min":1.357,"5min":1.6100000000000001,"15min":1.859},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.97699999999999998,"15min":0.94499999999999995},"min":{"1min":0.55300000000000005,"5min":0.443,"15min":0.36699999999999999},"max":{"1min":1.3580000000000001,"5min":1.8009999999999999,"15min":2.0230000000000001},"last":1.0629999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.181,"15min":1.1040000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.436},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.181},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.085,"15min":1.073},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.35399999999999998},"max":{"1min":1.4370000000000001,"5min":1.6659999999999999,"15min":3.7010000000000001},"last":0.91100000000000003}]},{"osd":4,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.296,"15min":1.2929999999999999},"min":{"1min":0.81499999999999995,"5min":0.75600000000000001,"15min":0.45700000000000002},"max":{"1min":1.97,"5min":1.97,"15min":2.0489999999999999},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.335,"15min":1.3129999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.59699999999999998},"max":{"1min":1.8149999999999999,"5min":1.877,"15min":2.0470000000000002},"last":1.52}]},{"osd":5,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.478,"15min":1.401},"min":{"1min":0.90400000000000003,"5min":0.86499999999999999,"15min":0.56000000000000005},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":3.5750000000000002},"last":1.609},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.476,"15min":1.448},"min":{"1min":0.90000000000000002,"5min":0.86399999999999999,"15min":0.71699999999999997},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.3769999999999998},"last":1.681}]},{"osd":6,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4099999999999999,"15min":1.3839999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.64200000000000002},"max":{"1min":2.044,"5min":2.044,"15min":2.2050000000000001},"last":1.448},{"interface":"front","average":{"1min":1.46,"5min":1.4610000000000001,"15min":1.3959999999999999},"min":{"1min":0.92900000000000005,"5min":0.89000000000000001,"15min":0.63100000000000001},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.492}]},{"osd":7,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.538,"15min":1.4730000000000001},"min":{"1min":0.95399999999999996,"5min":0.85699999999999998,"15min":0.69799999999999995},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.413},{"interface":"front","average":{"1min":1.421,"5min":1.4790000000000001,"15min":1.4590000000000001},"min":{"1min":1,"5min":0.81100000000000005,"15min":0.66300000000000003},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.4180000000000001},"last":1.375}]}]},{"osd":0,"up_from":13,"seq":55834575033,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193606,"statfs":{"total":200000000,"available":198252919,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.52100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.1779999999999999,"5min":1.2410000000000001,"15min":3.4420000000000002},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.071,"5min":1.091,"15min":1.089},"min":{"1min":0.56100000000000005,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.476,"5min":1.657,"15min":1.8240000000000001},"last":1.302}]},{"osd":2,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.90200000000000002,"15min":0.85999999999999999},"min":{"1min":0.59799999999999998,"5min":0.56299999999999994,"15min":0.375},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.679},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.2290000000000001,"15min":1.161},"min":{"1min":0.78400000000000003,"5min":0.623,"15min":0.437},"max":{"1min":1.607,"5min":1.917,"15min":1.929},"last":1.0840000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.167,"15min":1.1599999999999999},"min":{"1min":0.56799999999999995,"5min":0.52100000000000002,"15min":0.49399999999999999},"max":{"1min":1.7210000000000001,"5min":1.9590000000000001,"15min":2.7639999999999998},"last":1.647},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.4159999999999999,"15min":1.413},"min":{"1min":0.85099999999999998,"5min":0.68100000000000005,"15min":0.58399999999999996},"max":{"1min":1.77,"5min":2.0150000000000001,"15min":3.5720000000000001},"last":1.792}]},{"osd":4,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2170000000000001,"15min":1.2270000000000001},"min":{"1min":0.80800000000000005,"5min":0.73099999999999998,"15min":0.71099999999999997},"max":{"1min":1.6579999999999999,"5min":1.7370000000000001,"15min":4.492},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2829999999999999,"15min":1.274},"min":{"1min":0.93600000000000005,"5min":0.77500000000000002,"15min":0.68700000000000006},"max":{"1min":1.552,"5min":1.752,"15min":1.927},"last":1.2390000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.252,"15min":1.2889999999999999},"min":{"1min":0.97899999999999998,"5min":0.75600000000000001,"15min":0.64300000000000002},"max":{"1min":1.625,"5min":1.6439999999999999,"15min":4.2779999999999996},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.347,"15min":1.3340000000000001},"min":{"1min":1.03,"5min":0.81899999999999995,"15min":0.73099999999999998},"max":{"1min":1.583,"5min":1.8169999999999999,"15min":1.8740000000000001},"last":1.4490000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.48,"15min":1.4730000000000001},"min":{"1min":1.0620000000000001,"5min":0.81000000000000005,"15min":0.76800000000000002},"max":{"1min":2.0459999999999998,"5min":2.2469999999999999,"15min":3.5299999999999998},"last":1.504},{"interface":"front","average":{"1min":1.302,"5min":1.359,"15min":1.3819999999999999},"min":{"1min":0.95099999999999996,"5min":0.86899999999999999,"15min":0.66700000000000004},"max":{"1min":1.661,"5min":1.8720000000000001,"15min":4.4349999999999996},"last":1.595}]},{"osd":7,"last update":"Wed Oct 18 09:09:35 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.5489999999999999,"15min":1.5569999999999999},"min":{"1min":1.1850000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.982,"5min":2.133,"15min":3.492},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.472,"15min":1.4950000000000001},"min":{"1min":0.95999999999999996,"5min":0.88500000000000001,"15min":0.82099999999999995},"max":{"1min":1.776,"5min":2.1000000000000001,"15min":4.5419999999999998},"last":1.415}]}]}],"pool_statfs":[]}} 2023-10-18T09:10:20.241 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-18T09:10:20.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.450+0000 7f8502f97640 1 -- 172.21.15.40:0/3413220830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84ec003f40 msgr2=0x7f84ec0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:20.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.450+0000 7f8502f97640 1 --2- 172.21.15.40:0/3413220830 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84ec003f40 0x7f84ec0047c0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f84e8006f40 tx=0x7f84ec0057f0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.450+0000 7f8501534640 1 -- 172.21.15.40:0/3413220830 reap_dead start 2023-10-18T09:10:20.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.450+0000 7f8502f97640 1 -- 172.21.15.40:0/3413220830 shutdown_connections 2023-10-18T09:10:20.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.450+0000 7f8502f97640 1 -- 172.21.15.40:0/3413220830 >> 172.21.15.40:0/3413220830 conn(0x7f84fc069720 msgr2=0x7f84fc069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:20.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.450+0000 7f8502f97640 1 -- 172.21.15.40:0/3413220830 shutdown_connections 2023-10-18T09:10:20.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.450+0000 7f8502f97640 1 -- 172.21.15.40:0/3413220830 wait complete. 2023-10-18T09:10:20.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.452+0000 7f8502f97640 1 Processor -- start 2023-10-18T09:10:20.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.452+0000 7f8502f97640 1 -- start start 2023-10-18T09:10:20.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.453+0000 7f8502f97640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84ec003f40 0x7f84fc0b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:20.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.453+0000 7f8502f97640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84fc151200 0x7f84fc13a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:20.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.453+0000 7f8502f97640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84fc151990 0x7f84fc151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:20.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.453+0000 7f8502f97640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc152350 con 0x7f84ec003f40 2023-10-18T09:10:20.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.453+0000 7f8502f97640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc0b17e0 con 0x7f84fc151200 2023-10-18T09:10:20.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.453+0000 7f8502f97640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc1550c0 con 0x7f84fc151990 2023-10-18T09:10:20.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.453+0000 7f8500d33640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84ec003f40 0x7f84fc0b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:20.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.454+0000 7f8500d33640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84ec003f40 0x7f84fc0b12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45320/0 (socket says 172.21.15.40:45320) 2023-10-18T09:10:20.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.454+0000 7f8500d33640 1 -- 172.21.15.40:0/3047054915 learned_addr learned my addr 172.21.15.40:0/3047054915 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:10:20.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.454+0000 7f84fbfff640 1 --2- 172.21.15.40:0/3047054915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84fc151200 0x7f84fc13a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:20.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.454+0000 7f8500d33640 1 -- 172.21.15.40:0/3047054915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84fc151990 msgr2=0x7f84fc151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:20.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.454+0000 7f8500d33640 1 --2- 172.21.15.40:0/3047054915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84fc151990 0x7f84fc151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.454+0000 7f8500d33640 1 -- 172.21.15.40:0/3047054915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84fc151200 msgr2=0x7f84fc13a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:20.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.454+0000 7f8500d33640 1 --2- 172.21.15.40:0/3047054915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84fc151200 0x7f84fc13a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.454+0000 7f8500d33640 1 -- 172.21.15.40:0/3047054915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc0b1a60 con 0x7f84ec003f40 2023-10-18T09:10:20.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.454+0000 7f8500d33640 1 --2- 172.21.15.40:0/3047054915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84ec003f40 0x7f84fc0b12a0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f84f0010a10 tx=0x7f84f0010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:20.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.455+0000 7f84f97fa640 1 -- 172.21.15.40:0/3047054915 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f0009720 con 0x7f84ec003f40 2023-10-18T09:10:20.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.455+0000 7f84f97fa640 1 -- 172.21.15.40:0/3047054915 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f84f000ed20 con 0x7f84ec003f40 2023-10-18T09:10:20.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.455+0000 7f8502f97640 1 -- 172.21.15.40:0/3047054915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84fc0b1d40 con 0x7f84ec003f40 2023-10-18T09:10:20.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.455+0000 7f84f97fa640 1 -- 172.21.15.40:0/3047054915 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f0005280 con 0x7f84ec003f40 2023-10-18T09:10:20.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.455+0000 7f8502f97640 1 -- 172.21.15.40:0/3047054915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc0b2020 con 0x7f84ec003f40 2023-10-18T09:10:20.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.456+0000 7f8502f97640 1 -- 172.21.15.40:0/3047054915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84fc066fd0 con 0x7f84ec003f40 2023-10-18T09:10:20.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.457+0000 7f84f97fa640 1 -- 172.21.15.40:0/3047054915 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f84f00053e0 con 0x7f84ec003f40 2023-10-18T09:10:20.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.458+0000 7f84f97fa640 1 --2- 172.21.15.40:0/3047054915 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f84e006ae40 0x7f84e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:20.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.458+0000 7f84f97fa640 1 -- 172.21.15.40:0/3047054915 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f84f0016070 con 0x7f84ec003f40 2023-10-18T09:10:20.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.459+0000 7f84fbfff640 1 --2- 172.21.15.40:0/3047054915 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f84e006ae40 0x7f84e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:20.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.460+0000 7f84fbfff640 1 --2- 172.21.15.40:0/3047054915 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f84e006ae40 0x7f84e006d300 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f84e8034000 tx=0x7f84e8039040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:20.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.462+0000 7f84f97fa640 1 -- 172.21.15.40:0/3047054915 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84f00554f0 con 0x7f84ec003f40 2023-10-18T09:10:20.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.662+0000 7f8502f97640 1 -- 172.21.15.40:0/3047054915 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f84fc0cbc70 con 0x7f84ec003f40 2023-10-18T09:10:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.667+0000 7f84f97fa640 1 -- 172.21.15.40:0/3047054915 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v43)=0 v43) v1 ==== 123+0+0 (secure 0 0 0) 0x7f84f0058b10 con 0x7f84ec003f40 2023-10-18T09:10:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.668+0000 7f84f97fa640 1 -- 172.21.15.40:0/3047054915 <== mon.0 v2:172.21.15.40:3300/0 8 ==== config(6 keys) v1 ==== 172+0+0 (secure 0 0 0) 0x7f84f000eeb0 con 0x7f84ec003f40 2023-10-18T09:10:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.671+0000 7f8502f97640 1 -- 172.21.15.40:0/3047054915 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f84e006ae40 msgr2=0x7f84e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.671+0000 7f8502f97640 1 --2- 172.21.15.40:0/3047054915 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f84e006ae40 0x7f84e006d300 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f84e8034000 tx=0x7f84e8039040 comp rx=0 tx=0).stop 2023-10-18T09:10:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.672+0000 7f8502f97640 1 -- 172.21.15.40:0/3047054915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84ec003f40 msgr2=0x7f84fc0b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.672+0000 7f8502f97640 1 --2- 172.21.15.40:0/3047054915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84ec003f40 0x7f84fc0b12a0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f84f0010a10 tx=0x7f84f0010e10 comp rx=0 tx=0).stop 2023-10-18T09:10:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.672+0000 7f8502f97640 1 -- 172.21.15.40:0/3047054915 shutdown_connections 2023-10-18T09:10:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.672+0000 7f8502f97640 1 --2- 172.21.15.40:0/3047054915 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7f84e006ae40 0x7f84e006d300 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.672+0000 7f8502f97640 1 --2- 172.21.15.40:0/3047054915 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84fc151990 0x7f84fc151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.672+0000 7f8502f97640 1 --2- 172.21.15.40:0/3047054915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84fc151200 0x7f84fc13a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.672+0000 7f8502f97640 1 --2- 172.21.15.40:0/3047054915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84ec003f40 0x7f84fc0b12a0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.672+0000 7f8502f97640 1 -- 172.21.15.40:0/3047054915 >> 172.21.15.40:0/3047054915 conn(0x7f84fc069720 msgr2=0x7f84fc13b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.672+0000 7f8502f97640 1 -- 172.21.15.40:0/3047054915 shutdown_connections 2023-10-18T09:10:20.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.672+0000 7f8502f97640 1 -- 172.21.15.40:0/3047054915 wait complete. 2023-10-18T09:10:20.694 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T09:10:20.695 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:10:20.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.911+0000 7fb074acf640 1 -- 172.21.15.40:0/503071582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb05c004d00 msgr2=0x7fb05c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:20.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.911+0000 7fb074acf640 1 --2- 172.21.15.40:0/503071582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb05c004d00 0x7fb05c005180 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fb06800c260 tx=0x7fb05c0057f0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.911+0000 7fb06ed9d640 1 -- 172.21.15.40:0/503071582 reap_dead start 2023-10-18T09:10:20.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.912+0000 7fb074acf640 1 -- 172.21.15.40:0/503071582 shutdown_connections 2023-10-18T09:10:20.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.912+0000 7fb074acf640 1 -- 172.21.15.40:0/503071582 >> 172.21.15.40:0/503071582 conn(0x7fb07006cee0 msgr2=0x7fb070069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:20.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.912+0000 7fb074acf640 1 -- 172.21.15.40:0/503071582 shutdown_connections 2023-10-18T09:10:20.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.912+0000 7fb074acf640 1 -- 172.21.15.40:0/503071582 wait complete. 2023-10-18T09:10:20.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.914+0000 7fb074acf640 1 Processor -- start 2023-10-18T09:10:20.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.914+0000 7fb074acf640 1 -- start start 2023-10-18T09:10:20.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.915+0000 7fb074acf640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb05c004d00 0x7fb07014bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:20.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.915+0000 7fb074acf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb07014c330 0x7fb0700afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:20.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.915+0000 7fb074acf640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb07014c7b0 0x7fb0700b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:20.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.915+0000 7fb074acf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0701f5cc0 con 0x7fb07014c330 2023-10-18T09:10:20.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.915+0000 7fb074acf640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07014cd10 con 0x7fb05c004d00 2023-10-18T09:10:20.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.915+0000 7fb074acf640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0701f5390 con 0x7fb07014c7b0 2023-10-18T09:10:20.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.915+0000 7fb06dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb07014c330 0x7fb0700afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:20.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.915+0000 7fb06dd9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb07014c330 0x7fb0700afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45352/0 (socket says 172.21.15.40:45352) 2023-10-18T09:10:20.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.915+0000 7fb06dd9b640 1 -- 172.21.15.40:0/3157220255 learned_addr learned my addr 172.21.15.40:0/3157220255 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-18T09:10:20.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.915+0000 7fb06e59c640 1 --2- 172.21.15.40:0/3157220255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb05c004d00 0x7fb07014bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:20.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.915+0000 7fb06ed9d640 1 --2- 172.21.15.40:0/3157220255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb07014c7b0 0x7fb0700b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:20.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.916+0000 7fb06dd9b640 1 -- 172.21.15.40:0/3157220255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb07014c7b0 msgr2=0x7fb0700b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:20.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.916+0000 7fb06dd9b640 1 --2- 172.21.15.40:0/3157220255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb07014c7b0 0x7fb0700b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.916+0000 7fb06dd9b640 1 -- 172.21.15.40:0/3157220255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb05c004d00 msgr2=0x7fb07014bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:20.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.916+0000 7fb06dd9b640 1 --2- 172.21.15.40:0/3157220255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb05c004d00 0x7fb07014bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:20.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.916+0000 7fb06dd9b640 1 -- 172.21.15.40:0/3157220255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0701f5670 con 0x7fb07014c330 2023-10-18T09:10:20.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.916+0000 7fb06dd9b640 1 --2- 172.21.15.40:0/3157220255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb07014c330 0x7fb0700afed0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fb05800d9b0 tx=0x7fb05800de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:20.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.917+0000 7fb066ffd640 1 -- 172.21.15.40:0/3157220255 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058008c80 con 0x7fb07014c330 2023-10-18T09:10:20.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.917+0000 7fb066ffd640 1 -- 172.21.15.40:0/3157220255 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(6 keys) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb058004020 con 0x7fb07014c330 2023-10-18T09:10:20.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.917+0000 7fb066ffd640 1 -- 172.21.15.40:0/3157220255 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058004410 con 0x7fb07014c330 2023-10-18T09:10:20.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.917+0000 7fb074acf640 1 -- 172.21.15.40:0/3157220255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0701f5950 con 0x7fb07014c330 2023-10-18T09:10:20.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.917+0000 7fb074acf640 1 -- 172.21.15.40:0/3157220255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0701f6330 con 0x7fb07014c330 2023-10-18T09:10:20.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.918+0000 7fb074acf640 1 -- 172.21.15.40:0/3157220255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb070066fd0 con 0x7fb07014c330 2023-10-18T09:10:20.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.919+0000 7fb066ffd640 1 -- 172.21.15.40:0/3157220255 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb058005020 con 0x7fb07014c330 2023-10-18T09:10:20.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.921+0000 7fb066ffd640 1 --2- 172.21.15.40:0/3157220255 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb05006ae40 0x7fb05006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:20.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.921+0000 7fb066ffd640 1 -- 172.21.15.40:0/3157220255 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb058013070 con 0x7fb07014c330 2023-10-18T09:10:20.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.922+0000 7fb06e59c640 1 --2- 172.21.15.40:0/3157220255 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb05006ae40 0x7fb05006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:20.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.923+0000 7fb06e59c640 1 --2- 172.21.15.40:0/3157220255 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb05006ae40 0x7fb05006d300 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb060005fd0 tx=0x7fb06000a750 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:20.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:20.924+0000 7fb066ffd640 1 -- 172.21.15.40:0/3157220255 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb0580549c0 con 0x7fb07014c330 2023-10-18T09:10:21.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:21.158+0000 7fb074acf640 1 -- 172.21.15.40:0/3157220255 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb07006fb50 con 0x7fb07014c330 2023-10-18T09:10:21.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:21.159+0000 7fb066ffd640 1 -- 172.21.15.40:0/3157220255 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fb05801b090 con 0x7fb07014c330 2023-10-18T09:10:21.163 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-18T09:10:21.163 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6836/2038256442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2038256442},{"type":"v1","addr":"172.21.15.144:6836","nonce":2038256442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":7024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6837/3006484507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6835","nonce":3006484507},{"type":"v1","addr":"172.21.15.144:6837","nonce":3006484507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9279,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6834/1542000032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1542000032},{"type":"v1","addr":"172.21.15.40:6834","nonce":1542000032}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9660,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/2710898005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6833","nonce":2710898005},{"type":"v1","addr":"172.21.15.40:6835","nonce":2710898005}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[]} 2023-10-18T09:10:21.163 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 92 2023-10-18T09:10:21.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:21.162+0000 7fb074acf640 1 -- 172.21.15.40:0/3157220255 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb05006ae40 msgr2=0x7fb05006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:21.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:21.162+0000 7fb074acf640 1 --2- 172.21.15.40:0/3157220255 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb05006ae40 0x7fb05006d300 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb060005fd0 tx=0x7fb06000a750 comp rx=0 tx=0).stop 2023-10-18T09:10:21.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:21.162+0000 7fb074acf640 1 -- 172.21.15.40:0/3157220255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb07014c330 msgr2=0x7fb0700afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:21.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:21.162+0000 7fb074acf640 1 --2- 172.21.15.40:0/3157220255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb07014c330 0x7fb0700afed0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fb05800d9b0 tx=0x7fb05800de70 comp rx=0 tx=0).stop 2023-10-18T09:10:21.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:21.162+0000 7fb074acf640 1 -- 172.21.15.40:0/3157220255 shutdown_connections 2023-10-18T09:10:21.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:21.162+0000 7fb074acf640 1 --2- 172.21.15.40:0/3157220255 >> [v2:172.21.15.144:6800/3843888630,v1:172.21.15.144:6801/3843888630] conn(0x7fb05006ae40 0x7fb05006d300 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:21.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:21.162+0000 7fb074acf640 1 --2- 172.21.15.40:0/3157220255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb07014c7b0 0x7fb0700b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:21.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:21.162+0000 7fb074acf640 1 --2- 172.21.15.40:0/3157220255 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb07014c330 0x7fb0700afed0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:21.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:21.162+0000 7fb074acf640 1 --2- 172.21.15.40:0/3157220255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb05c004d00 0x7fb07014bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:21.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:21.162+0000 7fb074acf640 1 -- 172.21.15.40:0/3157220255 >> 172.21.15.40:0/3157220255 conn(0x7fb07006cee0 msgr2=0x7fb070071200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:21.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:21.162+0000 7fb074acf640 1 -- 172.21.15.40:0/3157220255 shutdown_connections 2023-10-18T09:10:21.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-18T09:10:21.162+0000 7fb074acf640 1 -- 172.21.15.40:0/3157220255 wait complete. 2023-10-18T09:10:21.185 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-18T09:10:21.186 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-18T09:10:21.186 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-18T09:10:21.186 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:21.207 INFO:tasks.ceph.mds.a:Stopped 2023-10-18T09:10:21.207 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-18T09:10:21.207 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:21.280 INFO:tasks.ceph.mds.c:Stopped 2023-10-18T09:10:21.281 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-18T09:10:21.281 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:21.301 INFO:tasks.ceph.mds.b:Stopped 2023-10-18T09:10:21.302 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-18T09:10:21.302 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:21.374 INFO:tasks.ceph.mds.d:Stopped 2023-10-18T09:10:21.375 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-18T09:10:21.375 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-18T09:10:21.376 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:26.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2023-10-18T09:10:27.478 INFO:tasks.ceph.osd.0:Stopped 2023-10-18T09:10:27.478 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-18T09:10:27.479 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:27.566 INFO:tasks.ceph.osd.1:Stopped 2023-10-18T09:10:27.567 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-18T09:10:27.567 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:32.896 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-10-18T09:10:32.896 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2023-10-18T09:10:33.668 INFO:tasks.ceph.osd.2:Stopped 2023-10-18T09:10:33.669 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-18T09:10:33.669 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:33.746 INFO:tasks.ceph.osd.3:Stopped 2023-10-18T09:10:33.746 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-18T09:10:33.747 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:33.828 INFO:tasks.ceph.osd.4:Stopped 2023-10-18T09:10:33.829 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-18T09:10:33.829 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:33.875 INFO:tasks.ceph.osd.5:Stopped 2023-10-18T09:10:33.875 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-18T09:10:33.875 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:33.964 INFO:tasks.ceph.osd.6:Stopped 2023-10-18T09:10:33.965 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-18T09:10:33.965 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:34.061 INFO:tasks.ceph.osd.7:Stopped 2023-10-18T09:10:34.062 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-18T09:10:34.063 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-18T09:10:34.063 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:34.160 INFO:tasks.ceph.mgr.y:Stopped 2023-10-18T09:10:34.161 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-18T09:10:34.161 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:34.231 INFO:tasks.ceph.mgr.x:Stopped 2023-10-18T09:10:34.231 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-18T09:10:34.232 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-18T09:10:34.232 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:34.284 INFO:tasks.ceph.mon.a:Stopped 2023-10-18T09:10:34.285 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-18T09:10:34.285 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:34.318 INFO:tasks.ceph.mon.b:Stopped 2023-10-18T09:10:34.318 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-18T09:10:34.319 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:10:34.389 INFO:tasks.ceph.mon.c:Stopped 2023-10-18T09:10:34.389 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-18T09:10:34.390 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-18T09:10:34.512 INFO:teuthology.orchestra.run.smithi040.stdout:2023-10-18T08:55:03.058812+0000 osd.4 (osd.4) 3 : cluster [WRN] OSD bench result of 75039.513932 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.4. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2023-10-18T09:10:34.513 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-18T09:10:34.514 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-18T09:10:34.608 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-18T09:10:34.704 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-18T09:10:34.798 INFO:teuthology.orchestra.run.smithi040.stdout:2023-10-18T08:55:03.058812+0000 osd.4 (osd.4) 3 : cluster [WRN] OSD bench result of 75039.513932 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.4. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2023-10-18T09:10:34.799 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T09:10:34.799 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-18T09:10:35.074 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T09:10:35.075 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-18T09:10:35.243 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T09:10:35.243 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-18T09:10:35.405 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T09:10:35.405 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-18T09:10:35.596 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T09:10:35.596 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-18T09:10:35.829 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T09:10:35.830 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-18T09:10:35.980 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T09:10:35.980 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-18T09:10:36.159 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T09:10:36.160 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-18T09:10:36.325 INFO:tasks.ceph:Archiving mon data... 2023-10-18T09:10:36.326 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719/data/mon.a.tgz 2023-10-18T09:10:36.327 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2023-10-18T09:10:36.389 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.cktp4a1k34 2023-10-18T09:10:36.390 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.cktp4a1k34 2023-10-18T09:10:36.624 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.cktp4a1k34 2023-10-18T09:10:36.790 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.cktp4a1k34 is 531KB 2023-10-18T09:10:36.909 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.cktp4a1k34 2023-10-18T09:10:36.972 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719/data/mon.b.tgz 2023-10-18T09:10:36.972 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2023-10-18T09:10:37.033 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.35jZg7lovo 2023-10-18T09:10:37.033 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.35jZg7lovo 2023-10-18T09:10:37.272 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.35jZg7lovo 2023-10-18T09:10:37.430 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.35jZg7lovo is 565KB 2023-10-18T09:10:37.541 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.35jZg7lovo 2023-10-18T09:10:37.599 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719/data/mon.c.tgz 2023-10-18T09:10:37.600 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2023-10-18T09:10:37.652 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.lguuTr15Dt 2023-10-18T09:10:37.652 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.lguuTr15Dt 2023-10-18T09:10:37.875 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.lguuTr15Dt 2023-10-18T09:10:37.898 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2023-10-18T09:10:38.027 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.lguuTr15Dt is 565KB 2023-10-18T09:10:38.137 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.lguuTr15Dt 2023-10-18T09:10:38.194 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-18T09:10:38.195 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-18T09:10:38.204 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-18T09:10:38.296 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-18T09:10:38.296 DEBUG:teuthology.orchestra.run.smithi040:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-18T09:10:38.305 DEBUG:teuthology.orchestra.run.smithi144:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-18T09:10:38.400 INFO:teuthology.orchestra.run.smithi144.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-18T09:10:38.402 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-18T09:10:38.414 INFO:tasks.ceph:Archiving crash dumps... 2023-10-18T09:10:38.417 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719/remote/smithi040/crash 2023-10-18T09:10:38.418 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-18T09:10:38.494 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719/remote/smithi144/crash 2023-10-18T09:10:38.496 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-18T09:10:38.587 INFO:tasks.ceph:Shutting down logrotate 2023-10-18T09:10:38.587 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:10:38.596 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-18T09:10:38.771 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-18T09:10:38.881 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-18T09:10:38.982 INFO:tasks.ceph:Compressing logs... 2023-10-18T09:10:38.983 DEBUG:teuthology.orchestra.run.smithi040:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-18T09:10:38.992 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-18T09:10:39.083 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-18T09:10:39.084 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.89521.log 2023-10-18T09:10:39.085 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.86941.log 2023-10-18T09:10:39.086 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100148.log 2023-10-18T09:10:39.086 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.1.89521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.84278.log 2023-10-18T09:10:39.087 INFO:teuthology.orchestra.run.smithi144.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.89521.log.gz 2023-10-18T09:10:39.087 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84238.log 2023-10-18T09:10:39.088 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.1.86941.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.86941.log.gz 2023-10-18T09:10:39.088 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83266.log 2023-10-18T09:10:39.089 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95098.log 2023-10-18T09:10:39.089 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.100148.log.gz 2023-10-18T09:10:39.090 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.1.84278.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.84278.log.gz 2023-10-18T09:10:39.090 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.86386.log 2023-10-18T09:10:39.091 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82829.log 2023-10-18T09:10:39.091 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.84238.log.gz 2023-10-18T09:10:39.092 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95098.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.95098.log.gz 2023-10-18T09:10:39.093 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.83266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83095.log 2023-10-18T09:10:39.093 INFO:teuthology.orchestra.run.smithi144.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.83266.log.gz 2023-10-18T09:10:39.093 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.1.86386.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.86386.log.gz 2023-10-18T09:10:39.093 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.88971.log 2023-10-18T09:10:39.093 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.83095.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.83095.log.gz 2023-10-18T09:10:39.094 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83175.log 2023-10-18T09:10:39.096 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.1.88971.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.88971.log.gz 2023-10-18T09:10:39.096 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83215.log 2023-10-18T09:10:39.096 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.83175.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-18T09:10:39.096 INFO:teuthology.orchestra.run.smithi144.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.83175.log.gz 2023-10-18T09:10:39.096 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.83215.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-18T09:10:39.097 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102747.log 2023-10-18T09:10:39.097 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82829.log.gz 2023-10-18T09:10:39.097 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111749.log 2023-10-18T09:10:39.098 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.102747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99922.log 2023-10-18T09:10:39.098 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102747.log.gz 2023-10-18T09:10:39.098 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119867.log 2023-10-18T09:10:39.098 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.111749.log.gz 2023-10-18T09:10:39.099 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.99922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112624.log 2023-10-18T09:10:39.099 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.99922.log.gz 2023-10-18T09:10:39.101 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104893.log 2023-10-18T09:10:39.101 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119867.log.gz 2023-10-18T09:10:39.101 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.112624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106724.log 2023-10-18T09:10:39.101 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112624.log.gz 2023-10-18T09:10:39.101 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92676.log 2023-10-18T09:10:39.102 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.104893.log.gz 2023-10-18T09:10:39.103 INFO:teuthology.orchestra.run.smithi144.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.83215.log.gz 2023-10-18T09:10:39.103 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83346.log 2023-10-18T09:10:39.104 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83135.log 2023-10-18T09:10:39.104 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.83346.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.83346.log.gz 2023-10-18T09:10:39.104 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.85784.log 2023-10-18T09:10:39.104 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.83135.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-18T09:10:39.104 INFO:teuthology.orchestra.run.smithi144.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.83135.log.gz 2023-10-18T09:10:39.105 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-18T09:10:39.105 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.1.85784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83055.log 2023-10-18T09:10:39.105 INFO:teuthology.orchestra.run.smithi144.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.85784.log.gz 2023-10-18T09:10:39.107 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83306.log 2023-10-18T09:10:39.107 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.83055.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.83055.log.gz 2023-10-18T09:10:39.107 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-18T09:10:39.107 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.83306.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-18T09:10:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93478.log 2023-10-18T09:10:39.109 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106724.log.gz 2023-10-18T09:10:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124618.log 2023-10-18T09:10:39.109 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.92676.log.gz 2023-10-18T09:10:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93478.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.93478.log.gz --verbose 2023-10-18T09:10:39.110 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.0.115758.log 2023-10-18T09:10:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116890.log 2023-10-18T09:10:39.110 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.124618.log.gz 2023-10-18T09:10:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.115758.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.96783.log -- replaced with /var/log/ceph/ceph-client.0.115758.log.gz 2023-10-18T09:10:39.110 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-10-18T09:10:39.111 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116890.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.116890.log.gz 2023-10-18T09:10:39.111 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97093.log 2023-10-18T09:10:39.111 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89959.log 2023-10-18T09:10:39.111 INFO:teuthology.orchestra.run.smithi040.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.96783.log.gz 2023-10-18T09:10:39.113 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.7.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.83306.log.gz 2023-10-18T09:10:39.114 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.84671.log 2023-10-18T09:10:39.114 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-18T09:10:39.114 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.3.84671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.85285.log 2023-10-18T09:10:39.114 INFO:teuthology.orchestra.run.smithi144.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.84671.log.gz 2023-10-18T09:10:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97093.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97093.log.gz 2023-10-18T09:10:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93210.log 2023-10-18T09:10:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113501.log 2023-10-18T09:10:39.116 INFO:teuthology.orchestra.run.smithi040.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.89959.log.gz 2023-10-18T09:10:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93210.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.93210.log.gz 2023-10-18T09:10:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-18T09:10:39.116 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123358.log 2023-10-18T09:10:39.117 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113501.log.gz 2023-10-18T09:10:39.117 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91744.log 2023-10-18T09:10:39.119 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.88475.log 2023-10-18T09:10:39.119 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.1.85285.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.85285.log.gz 2023-10-18T09:10:39.119 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.87483.log 2023-10-18T09:10:39.120 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.1.88475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.88475.log.gz 2023-10-18T09:10:39.120 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.1.87483.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.87483.log.gz 2023-10-18T09:10:39.120 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-18T09:10:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123358.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.123358.log.gz 2023-10-18T09:10:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83407.log 2023-10-18T09:10:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84500.log 2023-10-18T09:10:39.121 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91744.log.gz 2023-10-18T09:10:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83407.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.83407.log.gz 2023-10-18T09:10:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93688.log 2023-10-18T09:10:39.123 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-18T09:10:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96705.log 2023-10-18T09:10:39.124 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.84500.log.gz 2023-10-18T09:10:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93688.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.93688.log.gz 2023-10-18T09:10:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84756.log 2023-10-18T09:10:39.124 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113171.log 2023-10-18T09:10:39.125 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96705.log.gz 2023-10-18T09:10:39.125 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84756.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.84756.log.gz 2023-10-18T09:10:39.125 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84670.log 2023-10-18T09:10:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120138.log 2023-10-18T09:10:39.126 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113171.log.gz 2023-10-18T09:10:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84670.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.84670.log.gz 2023-10-18T09:10:39.126 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124744.log 2023-10-18T09:10:39.128 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115253.log 2023-10-18T09:10:39.128 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.120138.log.gz 2023-10-18T09:10:39.128 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.90034.log 2023-10-18T09:10:39.128 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-18T09:10:39.129 INFO:teuthology.orchestra.run.smithi144.stderr: 92.7% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-18T09:10:39.129 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.1.90034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.90034.log.gz 2023-10-18T09:10:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97765.log 2023-10-18T09:10:39.130 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.124744.log.gz 2023-10-18T09:10:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.115253.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.115253.log.gz 2023-10-18T09:10:39.130 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111581.log 2023-10-18T09:10:39.132 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.87979.log 2023-10-18T09:10:39.132 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-18T09:10:39.132 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95266.log 2023-10-18T09:10:39.133 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97765.log.gz 2023-10-18T09:10:39.133 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111581.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111581.log.gz 2023-10-18T09:10:39.133 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123400.log 2023-10-18T09:10:39.134 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90954.log 2023-10-18T09:10:39.134 INFO:teuthology.orchestra.run.smithi040.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.95266.log.gz 2023-10-18T09:10:39.135 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123400.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.123400.log.gz -5 2023-10-18T09:10:39.135 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.97261.log 2023-10-18T09:10:39.136 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.1.87979.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.87979.log.gz 2023-10-18T09:10:39.136 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90954.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.90954.log.gz 2023-10-18T09:10:39.136 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113979.log 2023-10-18T09:10:39.137 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85218.log 2023-10-18T09:10:39.138 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97261.log.gz 2023-10-18T09:10:39.139 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113979.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113979.log.gz 2023-10-18T09:10:39.139 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110013.log 2023-10-18T09:10:39.140 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114843.log 2023-10-18T09:10:39.140 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.85218.log.gz 2023-10-18T09:10:39.141 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110013.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.110013.log.gz 2023-10-18T09:10:39.142 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113543.log 2023-10-18T09:10:39.143 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87687.log 2023-10-18T09:10:39.143 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114843.log.gz 2023-10-18T09:10:39.144 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113543.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113543.log.gz 2023-10-18T09:10:39.144 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104725.log 2023-10-18T09:10:39.145 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.87687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103119.log 2023-10-18T09:10:39.146 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87687.log.gz 2023-10-18T09:10:39.146 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104725.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.104725.log.gz 2023-10-18T09:10:39.147 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114063.log 2023-10-18T09:10:39.148 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90751.log 2023-10-18T09:10:39.148 INFO:teuthology.orchestra.run.smithi040.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.103119.log.gz 2023-10-18T09:10:39.149 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124366.log 2023-10-18T09:10:39.150 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.114063.log.gz 2023-10-18T09:10:39.150 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90751.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.90751.log.gz 2023-10-18T09:10:39.151 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114502.log 2023-10-18T09:10:39.152 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89104.log 2023-10-18T09:10:39.152 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.124366.log.gz 2023-10-18T09:10:39.153 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114502.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114502.log.gz 2023-10-18T09:10:39.153 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99718.log 2023-10-18T09:10:39.155 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89104.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.89104.log.gz 2023-10-18T09:10:39.155 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97723.log 2023-10-18T09:10:39.156 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.99718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120899.log 2023-10-18T09:10:39.156 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99718.log.gz 2023-10-18T09:10:39.157 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97723.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97723.log.gz 2023-10-18T09:10:39.157 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96909.log 2023-10-18T09:10:39.158 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109787.log 2023-10-18T09:10:39.159 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.120899.log.gz 2023-10-18T09:10:39.160 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96909.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.96909.log.gz 2023-10-18T09:10:39.160 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102097.log 2023-10-18T09:10:39.161 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.109787.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109787.log.gz 2023-10-18T09:10:39.162 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119455.log 2023-10-18T09:10:39.162 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.102097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110981.log 2023-10-18T09:10:39.163 INFO:teuthology.orchestra.run.smithi040.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.102097.log.gz 2023-10-18T09:10:39.164 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107636.log 2023-10-18T09:10:39.164 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119455.log.gz 2023-10-18T09:10:39.165 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112837.log 2023-10-18T09:10:39.165 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110981.log.gz 2023-10-18T09:10:39.166 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107636.log: gzip 87.3% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.107636.log.gz -- /var/log/ceph/ceph-client.admin.120857.log 2023-10-18T09:10:39.167 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-10-18T09:10:39.168 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.112837.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112837.log.gz 2023-10-18T09:10:39.168 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124282.log 2023-10-18T09:10:39.169 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120857.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.120857.log.gz 2023-10-18T09:10:39.169 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110470.log 2023-10-18T09:10:39.170 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112582.log 2023-10-18T09:10:39.170 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.124282.log.gz 2023-10-18T09:10:39.171 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110470.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.110470.log.gz 2023-10-18T09:10:39.171 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124408.log 2023-10-18T09:10:39.172 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.112582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83447.log 2023-10-18T09:10:39.173 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112582.log.gz 2023-10-18T09:10:39.173 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124408.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.124408.log.gz 2023-10-18T09:10:39.174 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83452.log 2023-10-18T09:10:39.174 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83447.log: gzip 85.9% -- replaced with /var/log/ceph/ceph-client.admin.83447.log.gz 2023-10-18T09:10:39.175 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.94505.log 2023-10-18T09:10:39.175 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83452.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.83452.log.gz 2023-10-18T09:10:39.176 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113255.log 2023-10-18T09:10:39.177 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94505.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94505.log.gz 2023-10-18T09:10:39.177 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119125.log 2023-10-18T09:10:39.178 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113255.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.113255.log.gz 2023-10-18T09:10:39.178 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98550.log 2023-10-18T09:10:39.179 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119125.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.119125.log.gz 2023-10-18T09:10:39.179 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106210.log 2023-10-18T09:10:39.180 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98550.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.98550.log.gz 2023-10-18T09:10:39.181 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111833.log 2023-10-18T09:10:39.182 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104851.log 2023-10-18T09:10:39.182 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106210.log.gz 2023-10-18T09:10:39.183 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111833.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.111833.log.gz 2023-10-18T09:10:39.183 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86746.log 2023-10-18T09:10:39.184 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111147.log 2023-10-18T09:10:39.184 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104851.log.gz 2023-10-18T09:10:39.184 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86746.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86746.log.gz 2023-10-18T09:10:39.185 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95014.log 2023-10-18T09:10:39.186 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111147.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.111147.log.gz 2023-10-18T09:10:39.186 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84924.log 2023-10-18T09:10:39.187 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95014.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95014.log.gz 2023-10-18T09:10:39.187 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113769.log 2023-10-18T09:10:39.188 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101369.log 2023-10-18T09:10:39.188 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.84924.log.gz 2023-10-18T09:10:39.189 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113769.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.113769.log.gz 2023-10-18T09:10:39.190 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92339.log 2023-10-18T09:10:39.191 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101369.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101369.log.gz 2023-10-18T09:10:39.191 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113213.log 2023-10-18T09:10:39.192 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92339.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.92339.log.gz 2023-10-18T09:10:39.192 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122742.log 2023-10-18T09:10:39.193 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110325.log 87.8% 2023-10-18T09:10:39.193 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113213.log.gz 2023-10-18T09:10:39.194 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122742.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.122742.log.gz 2023-10-18T09:10:39.195 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117016.log 2023-10-18T09:10:39.196 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110325.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110325.log.gz 2023-10-18T09:10:39.196 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89146.log 2023-10-18T09:10:39.196 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117016.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117016.log.gz 2023-10-18T09:10:39.197 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97681.log 2023-10-18T09:10:39.198 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89146.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.89146.log.gz 2023-10-18T09:10:39.198 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92964.log 2023-10-18T09:10:39.199 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97681.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97681.log.gz 2023-10-18T09:10:39.199 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106624.log 2023-10-18T09:10:39.200 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92964.log: gzip 87.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.92964.log.gz 2023-10-18T09:10:39.201 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.83384.log 2023-10-18T09:10:39.201 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106624.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.106624.log.gz 2023-10-18T09:10:39.202 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122658.log 2023-10-18T09:10:39.203 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116974.log 2023-10-18T09:10:39.203 INFO:teuthology.orchestra.run.smithi040.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.83384.log.gz 2023-10-18T09:10:39.203 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122658.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.122658.log.gz 2023-10-18T09:10:39.204 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97827.log 2023-10-18T09:10:39.205 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116974.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.116974.log.gz 2023-10-18T09:10:39.206 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99030.log 2023-10-18T09:10:39.206 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97827.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97827.log.gz 2023-10-18T09:10:39.207 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88159.log 2023-10-18T09:10:39.208 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.99030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106766.log 2023-10-18T09:10:39.208 INFO:teuthology.orchestra.run.smithi040.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.99030.log.gz 2023-10-18T09:10:39.209 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88159.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88159.log.gz 2023-10-18T09:10:39.209 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103827.log 2023-10-18T09:10:39.211 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101285.log 2023-10-18T09:10:39.211 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106766.log.gz 2023-10-18T09:10:39.212 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103827.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103827.log.gz 2023-10-18T09:10:39.212 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100840.log 2023-10-18T09:10:39.214 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101285.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.101285.log.gz 2023-10-18T09:10:39.214 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93520.log 2023-10-18T09:10:39.215 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100840.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100840.log.gz 2023-10-18T09:10:39.216 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mon.c.log: 92.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-18T09:10:39.216 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114418.log 2023-10-18T09:10:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93520.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93520.log.gz 2023-10-18T09:10:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98862.log 2023-10-18T09:10:39.218 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114418.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114418.log.gz 2023-10-18T09:10:39.218 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93730.log 2023-10-18T09:10:39.219 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98862.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.98862.log.gz 2023-10-18T09:10:39.219 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93772.log 2023-10-18T09:10:39.220 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102055.log 2023-10-18T09:10:39.220 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93730.log.gz 2023-10-18T09:10:39.221 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84542.log 2023-10-18T09:10:39.221 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.93772.log.gz 2023-10-18T09:10:39.222 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.102055.log: gzip -5 --verbose -- 87.4% /var/log/ceph/ceph-client.admin.114927.log 2023-10-18T09:10:39.223 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.102055.log.gz 2023-10-18T09:10:39.224 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84542.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.84542.log.gz 2023-10-18T09:10:39.224 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91965.log 2023-10-18T09:10:39.225 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111411.log 2023-10-18T09:10:39.225 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114927.log.gz 2023-10-18T09:10:39.226 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116637.log 2023-10-18T09:10:39.226 INFO:teuthology.orchestra.run.smithi040.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.91965.log.gz 2023-10-18T09:10:39.227 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85176.log 2023-10-18T09:10:39.227 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.111411.log.gz 2023-10-18T09:10:39.228 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116637.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.116637.log.gz 2023-10-18T09:10:39.228 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114801.log 2023-10-18T09:10:39.229 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85176.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.85176.log.gz 2023-10-18T09:10:39.230 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108917.log 2023-10-18T09:10:39.231 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124534.log 2023-10-18T09:10:39.231 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.114801.log.gz 2023-10-18T09:10:39.232 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.108917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103911.log 2023-10-18T09:10:39.232 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.108917.log.gz 2023-10-18T09:10:39.233 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92880.log 2023-10-18T09:10:39.233 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.124534.log.gz 2023-10-18T09:10:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103911.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.103911.log.gz 2023-10-18T09:10:39.234 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122616.log 2023-10-18T09:10:39.235 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95140.log 2023-10-18T09:10:39.235 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.92880.log.gz 2023-10-18T09:10:39.236 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122616.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122616.log.gz 2023-10-18T09:10:39.237 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.119993.log 2023-10-18T09:10:39.237 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115083.log 2023-10-18T09:10:39.237 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.95140.log.gz 2023-10-18T09:10:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119993.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.119993.log.gz 2023-10-18T09:10:39.238 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88412.log 2023-10-18T09:10:39.239 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.115083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107060.log 2023-10-18T09:10:39.240 INFO:teuthology.orchestra.run.smithi040.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.115083.log.gz 2023-10-18T09:10:39.241 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88412.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88412.log.gz 2023-10-18T09:10:39.241 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115421.log 2023-10-18T09:10:39.242 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90996.log 2023-10-18T09:10:39.242 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107060.log.gz 2023-10-18T09:10:39.243 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.115421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122160.log 2023-10-18T09:10:39.243 INFO:teuthology.orchestra.run.smithi040.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.115421.log.gz 2023-10-18T09:10:39.244 INFO:teuthology.orchestra.run.smithi144.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-18T09:10:39.244 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104357.log 2023-10-18T09:10:39.245 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.90996.log.gz 2023-10-18T09:10:39.245 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122160.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.122160.log.gz 2023-10-18T09:10:39.245 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119497.log 2023-10-18T09:10:39.246 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86236.log 2023-10-18T09:10:39.246 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.104357.log.gz 2023-10-18T09:10:39.247 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119497.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.119497.log.gz 2023-10-18T09:10:39.247 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97135.log 2023-10-18T09:10:39.248 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90331.log 2023-10-18T09:10:39.249 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86236.log.gz 2023-10-18T09:10:39.250 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118374.log 2023-10-18T09:10:39.250 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97135.log.gz 2023-10-18T09:10:39.251 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90331.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90331.log.gz 2023-10-18T09:10:39.251 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121207.log 2023-10-18T09:10:39.252 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.118374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116763.log 2023-10-18T09:10:39.252 INFO:teuthology.orchestra.run.smithi040.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.118374.log.gz 2023-10-18T09:10:39.253 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.121207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85568.log 2023-10-18T09:10:39.253 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121207.log.gz 2023-10-18T09:10:39.254 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116763.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.116763.log.gz 2023-10-18T09:10:39.254 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89042.log 2023-10-18T09:10:39.256 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120731.log 2023-10-18T09:10:39.256 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.85568.log.gz 2023-10-18T09:10:39.257 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105314.log 2023-10-18T09:10:39.257 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.89042.log.gz 2023-10-18T09:10:39.258 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124492.log 2023-10-18T09:10:39.258 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.120731.log.gz 2023-10-18T09:10:39.259 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.105314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.105314.log.gz 2023-10-18T09:10:39.259 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95224.log 2023-10-18T09:10:39.260 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117287.log 2023-10-18T09:10:39.260 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.124492.log.gz 2023-10-18T09:10:39.261 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97345.log 2023-10-18T09:10:39.261 INFO:teuthology.orchestra.run.smithi040.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.95224.log.gz 2023-10-18T09:10:39.262 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117610.log 2023-10-18T09:10:39.262 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.117287.log.gz 2023-10-18T09:10:39.263 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101039.log 2023-10-18T09:10:39.263 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97345.log.gz 2023-10-18T09:10:39.264 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117610.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.117610.log.gz 2023-10-18T09:10:39.264 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99760.log 2023-10-18T09:10:39.265 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84171.log 2023-10-18T09:10:39.266 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.101039.log.gz 2023-10-18T09:10:39.266 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.99760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121123.log 2023-10-18T09:10:39.267 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.99760.log.gz 2023-10-18T09:10:39.268 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84171.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.84171.log.gz 2023-10-18T09:10:39.268 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107415.log 2023-10-18T09:10:39.269 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.121123.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.121123.log.gz 2023-10-18T09:10:39.269 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122341.log 2023-10-18T09:10:39.270 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107415.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.107415.log.gz 2023-10-18T09:10:39.271 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120987.log 2023-10-18T09:10:39.271 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122341.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.122341.log.gz 2023-10-18T09:10:39.272 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103514.log 2023-10-18T09:10:39.272 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92296.log 2023-10-18T09:10:39.273 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.120987.log.gz 2023-10-18T09:10:39.273 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103514.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103514.log.gz 2023-10-18T09:10:39.274 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124786.log 2023-10-18T09:10:39.274 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111497.log 87.0% 2023-10-18T09:10:39.275 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92296.log.gz 2023-10-18T09:10:39.276 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124786.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.124786.log.gz 2023-10-18T09:10:39.276 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90873.log 2023-10-18T09:10:39.277 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120390.log 2023-10-18T09:10:39.277 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.111497.log.gz 2023-10-18T09:10:39.278 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90873.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90873.log.gz 2023-10-18T09:10:39.278 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99838.log 2023-10-18T09:10:39.279 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110241.log 2023-10-18T09:10:39.279 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.120390.log.gz 2023-10-18T09:10:39.280 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.99838.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.99838.log.gz 2023-10-18T09:10:39.280 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103869.log 2023-10-18T09:10:39.281 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112939.log 2023-10-18T09:10:39.281 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110241.log.gz 2023-10-18T09:10:39.282 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103869.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.103869.log.gz 2023-10-18T09:10:39.282 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102951.log 2023-10-18T09:10:39.283 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.112939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98904.log 2023-10-18T09:10:39.283 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112939.log.gz 2023-10-18T09:10:39.284 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.102951.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102951.log.gz 2023-10-18T09:10:39.284 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84798.log 2023-10-18T09:10:39.285 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112025.log 2023-10-18T09:10:39.285 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.98904.log.gz 2023-10-18T09:10:39.286 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84798.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.84798.log.gz 2023-10-18T09:10:39.286 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98718.log 2023-10-18T09:10:39.287 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.112025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124576.log 2023-10-18T09:10:39.288 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112025.log.gz 2023-10-18T09:10:39.288 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98718.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.98718.log.gz 2023-10-18T09:10:39.289 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83073.log 2023-10-18T09:10:39.289 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82846.log 2023-10-18T09:10:39.290 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124576.log.gz 2023-10-18T09:10:39.291 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83073.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.83073.log.gz 2023-10-18T09:10:39.291 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104527.log 2023-10-18T09:10:39.292 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82846.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.82846.log.gz 2023-10-18T09:10:39.292 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122467.log 2023-10-18T09:10:39.293 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104527.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104527.log.gz 2023-10-18T09:10:39.293 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107247.log 2023-10-18T09:10:39.294 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103743.log 2023-10-18T09:10:39.294 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122467.log.gz 2023-10-18T09:10:39.299 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107247.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.107247.log.gz 2023-10-18T09:10:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117898.log 2023-10-18T09:10:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103743.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103743.log.gz 2023-10-18T09:10:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103077.log 2023-10-18T09:10:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117898.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117898.log.gz 2023-10-18T09:10:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121249.log 2023-10-18T09:10:39.300 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85134.log 2023-10-18T09:10:39.301 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103077.log.gz 2023-10-18T09:10:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.121249.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.121249.log.gz 2023-10-18T09:10:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115125.log 2023-10-18T09:10:39.302 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123526.log 2023-10-18T09:10:39.303 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.85134.log.gz 2023-10-18T09:10:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.115125.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115125.log.gz 2023-10-18T09:10:39.303 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94718.log 2023-10-18T09:10:39.304 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123526.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.123526.log.gz 2023-10-18T09:10:39.304 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91599.log 2023-10-18T09:10:39.304 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94718.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.94718.log.gz 2023-10-18T09:10:39.305 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113459.log 2023-10-18T09:10:39.305 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91599.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.91599.log.gz 2023-10-18T09:10:39.305 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118164.log 2023-10-18T09:10:39.306 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114597.log 2023-10-18T09:10:39.307 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113459.log.gz 2023-10-18T09:10:39.307 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.118164.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.118164.log.gz 2023-10-18T09:10:39.308 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99880.log 2023-10-18T09:10:39.309 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114597.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114597.log.gz 2023-10-18T09:10:39.309 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108460.log 2023-10-18T09:10:39.310 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.99880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124943.log 2023-10-18T09:10:39.310 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99880.log.gz 2023-10-18T09:10:39.311 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.108460.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.108460.log.gz 2023-10-18T09:10:39.311 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85386.log 2023-10-18T09:10:39.312 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121459.log 2023-10-18T09:10:39.312 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.124943.log.gz 2023-10-18T09:10:39.313 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85386.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.85386.log.gz 2023-10-18T09:10:39.313 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89275.log 2023-10-18T09:10:39.314 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.121459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117515.log 2023-10-18T09:10:39.314 INFO:teuthology.orchestra.run.smithi040.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.121459.log.gz 2023-10-18T09:10:39.315 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89275.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.89275.log.gz 2023-10-18T09:10:39.315 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93294.log 2023-10-18T09:10:39.316 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94249.log 2023-10-18T09:10:39.317 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117515.log.gz 2023-10-18T09:10:39.317 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93294.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.93294.log.gz 2023-10-18T09:10:39.318 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118935.log 2023-10-18T09:10:39.319 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94249.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94249.log.gz 2023-10-18T09:10:39.319 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100486.log 2023-10-18T09:10:39.320 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.118935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120432.log 2023-10-18T09:10:39.321 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.118935.log.gz 2023-10-18T09:10:39.321 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100486.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.100486.log.gz 2023-10-18T09:10:39.322 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89701.log 2023-10-18T09:10:39.323 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98044.log 2023-10-18T09:10:39.323 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.120432.log.gz 2023-10-18T09:10:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89701.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.89701.log.gz 2023-10-18T09:10:39.324 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97513.log 2023-10-18T09:10:39.325 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101242.log 2023-10-18T09:10:39.326 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98044.log.gz 2023-10-18T09:10:39.326 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97513.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97513.log.gz 2023-10-18T09:10:39.327 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106582.log 2023-10-18T09:10:39.328 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101242.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101242.log.gz 2023-10-18T09:10:39.328 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115836.log 2023-10-18T09:10:39.329 INFO:teuthology.orchestra.run.smithi144.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-18T09:10:39.330 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86832.log 2023-10-18T09:10:39.330 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106582.log.gz 2023-10-18T09:10:39.331 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.115836.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.115836.log.gz 2023-10-18T09:10:39.331 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122425.log 2023-10-18T09:10:39.331 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96825.log 2023-10-18T09:10:39.332 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86832.log.gz 2023-10-18T09:10:39.332 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122425.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.122425.log.gz 2023-10-18T09:10:39.333 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84457.log 2023-10-18T09:10:39.333 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106892.log 2023-10-18T09:10:39.334 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96825.log.gz 2023-10-18T09:10:39.334 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84457.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.84457.log.gz 2023-10-18T09:10:39.335 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83015.log 2023-10-18T09:10:39.335 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100316.log 2023-10-18T09:10:39.336 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106892.log.gz 2023-10-18T09:10:39.336 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83015.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.83015.log.gz 2023-10-18T09:10:39.337 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118206.log 2023-10-18T09:10:39.337 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100274.log 2023-10-18T09:10:39.338 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100316.log.gz 2023-10-18T09:10:39.339 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.118206.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.118206.log.gz 2023-10-18T09:10:39.339 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118078.log 2023-10-18T09:10:39.339 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101845.log 2023-10-18T09:10:39.340 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.100274.log.gz 2023-10-18T09:10:39.340 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.118078.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.118078.log.gz 2023-10-18T09:10:39.341 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90163.log 2023-10-18T09:10:39.341 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88874.log 2023-10-18T09:10:39.342 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.101845.log.gz 2023-10-18T09:10:39.343 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90163.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.90163.log.gz 2023-10-18T09:10:39.343 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98946.log 2023-10-18T09:10:39.344 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88874.log: 86.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.88874.log.gz -5 2023-10-18T09:10:39.344 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.98424.log 2023-10-18T09:10:39.345 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98946.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98946.log.gz 2023-10-18T09:10:39.345 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108875.log 2023-10-18T09:10:39.346 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98424.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.98424.log.gz 2023-10-18T09:10:39.346 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117118.log 2023-10-18T09:10:39.347 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.108875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109253.log 2023-10-18T09:10:39.348 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.108875.log.gz 2023-10-18T09:10:39.348 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117118.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117118.log.gz 2023-10-18T09:10:39.348 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120815.log 2023-10-18T09:10:39.349 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.109253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98676.log 2023-10-18T09:10:39.349 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.109253.log.gz 2023-10-18T09:10:39.350 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120815.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.120815.log.gz 2023-10-18T09:10:39.351 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120264.log 2023-10-18T09:10:39.351 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86520.log 2023-10-18T09:10:39.351 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.98676.log.gz 2023-10-18T09:10:39.352 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114759.log 2023-10-18T09:10:39.353 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.120264.log.gz 2023-10-18T09:10:39.353 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119251.log 2023-10-18T09:10:39.353 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.86520.log.gz 2023-10-18T09:10:39.354 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114759.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114759.log.gz 2023-10-18T09:10:39.354 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95765.log 2023-10-18T09:10:39.356 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101457.log 2023-10-18T09:10:39.356 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119251.log.gz 2023-10-18T09:10:39.357 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95765.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.95765.log.gz 2023-10-18T09:10:39.357 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123586.log 2023-10-18T09:10:39.358 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101457.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.101457.log.gz 2023-10-18T09:10:39.359 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86149.log 2023-10-18T09:10:39.360 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117058.log 2023-10-18T09:10:39.360 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.123586.log.gz 2023-10-18T09:10:39.361 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123946.log 2023-10-18T09:10:39.361 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86149.log.gz 2023-10-18T09:10:39.362 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120649.log 2023-10-18T09:10:39.363 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.117058.log.gz 2023-10-18T09:10:39.363 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97869.log 2023-10-18T09:10:39.363 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.123946.log.gz 2023-10-18T09:10:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120649.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.120649.log.gz 2023-10-18T09:10:39.364 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85904.log 2023-10-18T09:10:39.365 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97869.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97869.log.gz 2023-10-18T09:10:39.366 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99096.log 2023-10-18T09:10:39.366 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108208.log 2023-10-18T09:10:39.367 INFO:teuthology.orchestra.run.smithi040.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.85904.log.gz 2023-10-18T09:10:39.367 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.99096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88958.log 2023-10-18T09:10:39.368 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99096.log.gz 2023-10-18T09:10:39.369 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.108208.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108208.log.gz 2023-10-18T09:10:39.369 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91838.log 2023-10-18T09:10:39.370 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104399.log 2023-10-18T09:10:39.370 INFO:teuthology.orchestra.run.smithi040.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.88958.log.gz 2023-10-18T09:10:39.371 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104152.log 2023-10-18T09:10:39.371 INFO:teuthology.orchestra.run.smithi040.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.91838.log.gz 2023-10-18T09:10:39.372 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97219.log 2023-10-18T09:10:39.372 INFO:teuthology.orchestra.run.smithi040.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.104399.log.gz 2023-10-18T09:10:39.373 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104441.log 2023-10-18T09:10:39.373 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.104152.log.gz 2023-10-18T09:10:39.374 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115015.log 2023-10-18T09:10:39.374 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97219.log.gz 2023-10-18T09:10:39.375 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104441.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104441.log.gz 2023-10-18T09:10:39.375 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114166.log 2023-10-18T09:10:39.376 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.115015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87084.log 2023-10-18T09:10:39.377 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.115015.log.gz 2023-10-18T09:10:39.377 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104483.log 2023-10-18T09:10:39.378 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114166.log.gz 2023-10-18T09:10:39.379 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.87084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94972.log 2023-10-18T09:10:39.379 INFO:teuthology.orchestra.run.smithi040.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.87084.log.gz 2023-10-18T09:10:39.380 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104483.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104483.log.gz 2023-10-18T09:10:39.380 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109871.log 2023-10-18T09:10:39.381 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94972.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94972.log.gz 2023-10-18T09:10:39.381 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103701.log 2023-10-18T09:10:39.382 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.109871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100064.log 2023-10-18T09:10:39.382 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109871.log.gz 2023-10-18T09:10:39.383 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93604.log 2023-10-18T09:10:39.383 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103701.log.gz 2023-10-18T09:10:39.384 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97471.log 2023-10-18T09:10:39.384 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100064.log.gz 2023-10-18T09:10:39.385 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100944.log 2023-10-18T09:10:39.385 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.93604.log.gz 2023-10-18T09:10:39.387 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97471.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97471.log.gz 2023-10-18T09:10:39.387 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118248.log 2023-10-18T09:10:39.388 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100944.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.100944.log.gz 2023-10-18T09:10:39.388 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94103.log 2023-10-18T09:10:39.389 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.118248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.99283.log 2023-10-18T09:10:39.389 INFO:teuthology.orchestra.run.smithi040.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.118248.log.gz 2023-10-18T09:10:39.390 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94103.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-18T09:10:39.390 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94103.log.gz 2023-10-18T09:10:39.391 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.99283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.99283.log.gz 2023-10-18T09:10:39.391 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112308.log 2023-10-18T09:10:39.392 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101887.log 2023-10-18T09:10:39.393 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.112308.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112308.log.gz 2023-10-18T09:10:39.394 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96307.log 2023-10-18T09:10:39.395 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101887.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.101887.log.gz 2023-10-18T09:10:39.395 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105019.log 2023-10-18T09:10:39.396 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96307.log: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-18T09:10:39.396 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96307.log.gz 2023-10-18T09:10:39.396 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110680.log 2023-10-18T09:10:39.397 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.105019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101507.log 2023-10-18T09:10:39.397 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.105019.log.gz 2023-10-18T09:10:39.398 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110680.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110680.log.gz 2023-10-18T09:10:39.399 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107289.log 2023-10-18T09:10:39.400 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101507.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101507.log.gz 2023-10-18T09:10:39.400 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105746.log 2023-10-18T09:10:39.401 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100654.log 2023-10-18T09:10:39.401 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107289.log.gz 2023-10-18T09:10:39.402 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.105746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111623.log 2023-10-18T09:10:39.402 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.105746.log.gz 2023-10-18T09:10:39.403 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91249.log 2023-10-18T09:10:39.403 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100654.log.gz 2023-10-18T09:10:39.404 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111623.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.111623.log.gz 2023-10-18T09:10:39.404 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92402.log 2023-10-18T09:10:39.405 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96561.log 2023-10-18T09:10:39.406 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.91249.log.gz 2023-10-18T09:10:39.406 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88622.log 2023-10-18T09:10:39.407 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.92402.log.gz 2023-10-18T09:10:39.408 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96561.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.96561.log.gz 2023-10-18T09:10:39.408 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91081.log 2023-10-18T09:10:39.409 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88622.log: gzip -5 87.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.88622.log.gz /var/log/ceph/ceph-client.admin.107102.log 2023-10-18T09:10:39.409 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-10-18T09:10:39.410 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106682.log 2023-10-18T09:10:39.410 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91081.log.gz 2023-10-18T09:10:39.411 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107102.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.107102.log.gz 2023-10-18T09:10:39.411 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85887.log 2023-10-18T09:10:39.412 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106682.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.106682.log.gz 2023-10-18T09:10:39.412 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.87552.log 2023-10-18T09:10:39.413 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116537.log 2023-10-18T09:10:39.413 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.85887.log.gz 2023-10-18T09:10:39.414 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.87552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.87552.log.gz 2023-10-18T09:10:39.414 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103598.log 2023-10-18T09:10:39.415 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116537.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.116537.log.gz 2023-10-18T09:10:39.416 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110638.log 2023-10-18T09:10:39.417 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100232.log 2023-10-18T09:10:39.417 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.103598.log.gz 2023-10-18T09:10:39.418 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101327.log 2023-10-18T09:10:39.418 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110638.log.gz 2023-10-18T09:10:39.419 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123484.log 2023-10-18T09:10:39.419 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.100232.log.gz 2023-10-18T09:10:39.420 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95723.log 2023-10-18T09:10:39.420 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101327.log.gz 2023-10-18T09:10:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123484.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.123484.log.gz 2023-10-18T09:10:39.421 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113727.log 2023-10-18T09:10:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110139.log 2023-10-18T09:10:39.423 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95723.log.gz 2023-10-18T09:10:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120222.log 2023-10-18T09:10:39.423 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113727.log.gz 2023-10-18T09:10:39.424 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110139.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.110139.log.gz 2023-10-18T09:10:39.425 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85560.log 2023-10-18T09:10:39.426 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111275.log 2023-10-18T09:10:39.426 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.120222.log.gz 2023-10-18T09:10:39.427 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105392.log 2023-10-18T09:10:39.427 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.85560.log.gz 2023-10-18T09:10:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111275.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.111275.log.gz 2023-10-18T09:10:39.428 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96223.log 2023-10-18T09:10:39.430 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.105392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98760.log 2023-10-18T09:10:39.430 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105392.log.gz 2023-10-18T09:10:39.431 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107205.log 2023-10-18T09:10:39.431 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96223.log.gz 2023-10-18T09:10:39.432 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98760.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.98760.log.gz 2023-10-18T09:10:39.432 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83281.log 2023-10-18T09:10:39.433 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.121712.log 2023-10-18T09:10:39.433 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.107205.log.gz 2023-10-18T09:10:39.434 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123442.log 2023-10-18T09:10:39.435 INFO:teuthology.orchestra.run.smithi040.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.83281.log.gz 2023-10-18T09:10:39.435 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.121712.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.121712.log.gz 2023-10-18T09:10:39.436 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102163.log 2023-10-18T09:10:39.437 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91207.log 2023-10-18T09:10:39.438 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123442.log.gz 2023-10-18T09:10:39.439 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.102163.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.102163.log.gz --verbose 2023-10-18T09:10:39.439 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.122083.log 2023-10-18T09:10:39.440 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91207.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91207.log.gz 2023-10-18T09:10:39.440 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123190.log 2023-10-18T09:10:39.441 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83126.log 2023-10-18T09:10:39.441 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.122083.log.gz 2023-10-18T09:10:39.442 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123190.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.123190.log.gz 2023-10-18T09:10:39.442 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108164.log 2023-10-18T09:10:39.444 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83126.log: gzip -5 --verbose -- 86.5% /var/log/ceph/ceph-client.admin.104315.log 2023-10-18T09:10:39.444 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83126.log.gz 2023-10-18T09:10:39.445 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.108164.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.108164.log.gz 2023-10-18T09:10:39.445 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96181.log 2023-10-18T09:10:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124198.log 2023-10-18T09:10:39.446 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104315.log.gz 2023-10-18T09:10:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96181.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96181.log.gz 2023-10-18T09:10:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117203.log 2023-10-18T09:10:39.449 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.102350.log 2023-10-18T09:10:39.449 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.124198.log.gz 2023-10-18T09:10:39.450 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117203.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.117203.log.gz 2023-10-18T09:10:39.450 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118416.log 2023-10-18T09:10:39.451 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.102350.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.102350.log.gz 2023-10-18T09:10:39.451 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104935.log 2023-10-18T09:10:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.118416.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.118416.log.gz 2023-10-18T09:10:39.452 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109625.log 2023-10-18T09:10:39.454 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104935.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.104935.log.gz 2023-10-18T09:10:39.454 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109745.log 2023-10-18T09:10:39.455 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.109625.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109625.log.gz 2023-10-18T09:10:39.455 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93832.log 2023-10-18T09:10:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.109745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117245.log 2023-10-18T09:10:39.456 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109745.log.gz 2023-10-18T09:10:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93832.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93832.log.gz 2023-10-18T09:10:39.457 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86208.log 2023-10-18T09:10:39.459 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113643.log 2023-10-18T09:10:39.460 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117245.log.gz 2023-10-18T09:10:39.460 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86208.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.86208.log.gz 2023-10-18T09:10:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100006.log 2023-10-18T09:10:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89401.log 2023-10-18T09:10:39.462 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.113643.log.gz 2023-10-18T09:10:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100006.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100006.log.gz 2023-10-18T09:10:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105434.log 2023-10-18T09:10:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89401.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.89401.log.gz 2023-10-18T09:10:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117329.log 2023-10-18T09:10:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.105434.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.105434.log.gz 2023-10-18T09:10:39.466 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92031.log 2023-10-18T09:10:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117329.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117329.log.gz 2023-10-18T09:10:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117161.log 2023-10-18T09:10:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103177.log 2023-10-18T09:10:39.468 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.92031.log.gz 2023-10-18T09:10:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117161.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.117161.log.gz 2023-10-18T09:10:39.469 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105127.log 2023-10-18T09:10:39.471 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103177.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.103177.log.gz 2023-10-18T09:10:39.471 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116411.log 2023-10-18T09:10:39.472 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.105127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123316.log 2023-10-18T09:10:39.472 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.105127.log.gz 2023-10-18T09:10:39.473 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116411.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116411.log.gz 2023-10-18T09:10:39.473 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114334.log 2023-10-18T09:10:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88664.log 2023-10-18T09:10:39.475 INFO:teuthology.orchestra.run.smithi040.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.123316.log.gz 2023-10-18T09:10:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114334.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114334.log.gz 2023-10-18T09:10:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112540.log 2023-10-18T09:10:39.477 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91419.log 2023-10-18T09:10:39.477 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88664.log.gz 2023-10-18T09:10:39.478 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.112540.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.112540.log.gz 2023-10-18T09:10:39.479 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85884.log 2023-10-18T09:10:39.480 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94590.log 2023-10-18T09:10:39.480 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91419.log.gz 2023-10-18T09:10:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85884.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.85884.log.gz 2023-10-18T09:10:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104015.log 2023-10-18T09:10:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94590.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94590.log.gzgzip 2023-10-18T09:10:39.483 INFO:teuthology.orchestra.run.smithi040.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.103658.log 2023-10-18T09:10:39.484 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119167.log 2023-10-18T09:10:39.484 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104015.log.gz 2023-10-18T09:10:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103658.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103658.log.gz 2023-10-18T09:10:39.485 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86240.log 2023-10-18T09:10:39.486 INFO:teuthology.orchestra.run.smithi144.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-18T09:10:39.486 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119167.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119167.log.gz 2023-10-18T09:10:39.487 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113023.log 2023-10-18T09:10:39.488 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86240.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86240.log.gz 2023-10-18T09:10:39.488 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.81291.log 2023-10-18T09:10:39.489 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116679.log 2023-10-18T09:10:39.490 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.113023.log.gz 2023-10-18T09:10:39.490 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.tmp-client.admin.81291.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.81291.log.gz 2023-10-18T09:10:39.491 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119681.log 2023-10-18T09:10:39.492 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116679.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.116679.log.gz 2023-10-18T09:10:39.492 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123148.log 2023-10-18T09:10:39.493 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107018.log 2023-10-18T09:10:39.493 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119681.log.gz 2023-10-18T09:10:39.494 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94632.log 2023-10-18T09:10:39.494 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123148.log.gz 2023-10-18T09:10:39.495 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107018.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107018.log.gz 2023-10-18T09:10:39.496 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115571.log 2023-10-18T09:10:39.497 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88580.log 2023-10-18T09:10:39.498 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94632.log.gz 2023-10-18T09:10:39.498 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.115571.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115571.log.gz 2023-10-18T09:10:39.498 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124861.log 2023-10-18T09:10:39.499 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118753.log 2023-10-18T09:10:39.500 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.88580.log.gz 2023-10-18T09:10:39.500 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124861.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.124861.log.gz 2023-10-18T09:10:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85558.log 2023-10-18T09:10:39.501 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.118753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118753.log.gz 2023-10-18T09:10:39.502 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88328.log 2023-10-18T09:10:39.503 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85558.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.85558.log.gz 2023-10-18T09:10:39.503 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102831.log 2023-10-18T09:10:39.504 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104589.log 2023-10-18T09:10:39.505 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.88328.log.gz 2023-10-18T09:10:39.505 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.102831.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102831.log.gz 2023-10-18T09:10:39.506 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105587.log 2023-10-18T09:10:39.507 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104589.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104589.log.gz 2023-10-18T09:10:39.507 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83925.log 2023-10-18T09:10:39.508 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.105587.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105587.log.gz 2023-10-18T09:10:39.508 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.92218.log 2023-10-18T09:10:39.509 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121079.log 2023-10-18T09:10:39.509 INFO:teuthology.orchestra.run.smithi040.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.83925.log.gz 2023-10-18T09:10:39.510 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.92218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.92218.log.gz 2023-10-18T09:10:39.511 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92581.log 2023-10-18T09:10:39.512 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.121079.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.121079.log.gz 2023-10-18T09:10:39.512 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.112230.log 2023-10-18T09:10:39.513 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92581.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.92581.log.gz 2023-10-18T09:10:39.513 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89359.log 2023-10-18T09:10:39.514 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.112230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.112230.log.gz 2023-10-18T09:10:39.514 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90373.log 2023-10-18T09:10:39.515 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89359.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.89359.log.gz 2023-10-18T09:10:39.516 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85302.log 2023-10-18T09:10:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90373.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.90373.log.gz 2023-10-18T09:10:39.517 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95182.log 2023-10-18T09:10:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85302.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.85302.log.gz 2023-10-18T09:10:39.518 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-18T09:10:39.519 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94344.log 2023-10-18T09:10:39.520 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.95182.log.gz 2023-10-18T09:10:39.520 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105061.log 2023-10-18T09:10:39.522 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94344.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94344.log.gz 2023-10-18T09:10:39.522 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93646.log 2023-10-18T09:10:39.523 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.105061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120348.log 2023-10-18T09:10:39.524 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93646.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.105061.log.gz 2023-10-18T09:10:39.525 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93646.log.gz 2023-10-18T09:10:39.525 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100570.log 2023-10-18T09:10:39.527 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115463.log 2023-10-18T09:10:39.527 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.120348.log.gz 2023-10-18T09:10:39.528 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100570.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.100570.log.gz 2023-10-18T09:10:39.529 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117940.log 2023-10-18T09:10:39.530 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.115463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116579.log 2023-10-18T09:10:39.531 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.115463.log.gz 2023-10-18T09:10:39.531 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117940.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.117940.log.gz 2023-10-18T09:10:39.532 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124324.log 2023-10-18T09:10:39.534 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117984.log 2023-10-18T09:10:39.534 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116579.log.gz 2023-10-18T09:10:39.535 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124324.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.124324.log.gz 2023-10-18T09:10:39.535 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87042.log 2023-10-18T09:10:39.537 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90499.log 2023-10-18T09:10:39.537 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.117984.log.gz 2023-10-18T09:10:39.538 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.87042.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.87042.log.gz 2023-10-18T09:10:39.539 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91039.log 2023-10-18T09:10:39.540 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89875.log 2023-10-18T09:10:39.541 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90499.log.gz 2023-10-18T09:10:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91039.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91039.log.gz 2023-10-18T09:10:39.542 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106850.log 2023-10-18T09:10:39.544 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100528.log 2023-10-18T09:10:39.544 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.89875.log.gz 2023-10-18T09:10:39.545 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106850.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.106850.log.gz 2023-10-18T09:10:39.545 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97597.log 2023-10-18T09:10:39.547 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122041.log 2023-10-18T09:10:39.547 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100528.log.gz 2023-10-18T09:10:39.548 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97597.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97597.log.gz 2023-10-18T09:10:39.550 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121375.log 2023-10-18T09:10:39.551 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122041.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.122041.log.gz 2023-10-18T09:10:39.552 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114250.log 2023-10-18T09:10:39.553 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.121375.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.121375.log.gz 2023-10-18T09:10:39.553 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85898.log 2023-10-18T09:10:39.555 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111104.log 2023-10-18T09:10:39.555 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.114250.log.gz 2023-10-18T09:10:39.556 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85898.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.85898.log.gz 2023-10-18T09:10:39.556 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91333.log 2023-10-18T09:10:39.558 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111104.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.111104.log.gz 2023-10-18T09:10:39.558 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113297.log 2023-10-18T09:10:39.559 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91333.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91333.log.gz 2023-10-18T09:10:39.560 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94930.log 2023-10-18T09:10:39.561 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119413.log 2023-10-18T09:10:39.562 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113297.log.gz 2023-10-18T09:10:39.563 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94930.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.94930.log.gz 2023-10-18T09:10:39.563 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118831.log 2023-10-18T09:10:39.565 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98124.log 2023-10-18T09:10:39.565 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119413.log.gz 2023-10-18T09:10:39.566 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.118831.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118831.log.gz 2023-10-18T09:10:39.567 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107457.log 2023-10-18T09:10:39.568 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117690.log 2023-10-18T09:10:39.568 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98124.log.gz 2023-10-18T09:10:39.569 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107457.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107457.log.gz 2023-10-18T09:10:39.570 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90289.log 2023-10-18T09:10:39.571 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84714.log 2023-10-18T09:10:39.572 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117690.log.gz 2023-10-18T09:10:39.573 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90289.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90289.log.gz 2023-10-18T09:10:39.573 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116495.log 2023-10-18T09:10:39.575 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103430.log 2023-10-18T09:10:39.575 INFO:teuthology.orchestra.run.smithi040.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.84714.log.gz 2023-10-18T09:10:39.576 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116495.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.116495.log.gz 2023-10-18T09:10:39.576 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118332.log 2023-10-18T09:10:39.578 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95392.log 2023-10-18T09:10:39.578 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103430.log.gz 2023-10-18T09:10:39.579 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.118332.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.118332.log.gz 2023-10-18T09:10:39.580 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86916.log 2023-10-18T09:10:39.581 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100190.log 2023-10-18T09:10:39.582 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86916.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.95392.log.gz 2023-10-18T09:10:39.583 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.86916.log.gz 2023-10-18T09:10:39.584 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88244.log 2023-10-18T09:10:39.585 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118566.log 2023-10-18T09:10:39.585 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100190.log.gz 2023-10-18T09:10:39.586 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88244.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.88244.log.gz 2023-10-18T09:10:39.587 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94207.log 2023-10-18T09:10:39.588 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.118566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88706.log 2023-10-18T09:10:39.589 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118566.log.gz 2023-10-18T09:10:39.590 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94207.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94207.log.gz 2023-10-18T09:10:39.590 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86646.log 2023-10-18T09:10:39.592 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100358.log 2023-10-18T09:10:39.592 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88706.log.gz 2023-10-18T09:10:39.593 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86646.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86646.log.gz 2023-10-18T09:10:39.593 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88454.log 2023-10-18T09:10:39.595 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114376.log 2023-10-18T09:10:39.595 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.100358.log.gz 2023-10-18T09:10:39.596 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88454.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88454.log.gz 2023-10-18T09:10:39.596 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102789.log 2023-10-18T09:10:39.598 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116165.log 2023-10-18T09:10:39.598 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114376.log.gz 2023-10-18T09:10:39.599 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.102789.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102789.log.gz 2023-10-18T09:10:39.600 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120474.log 2023-10-18T09:10:39.601 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110512.log 2023-10-18T09:10:39.601 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116165.log.gz 2023-10-18T09:10:39.603 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120474.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.120474.log.gz 2023-10-18T09:10:39.603 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92497.log 2023-10-18T09:10:39.605 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98634.log 2023-10-18T09:10:39.605 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.110512.log.gz 2023-10-18T09:10:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92497.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.92497.log.gz 2023-10-18T09:10:39.606 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89532.log 2023-10-18T09:10:39.608 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91881.log 2023-10-18T09:10:39.608 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.98634.log.gz 2023-10-18T09:10:39.609 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89532.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89532.log.gz 2023-10-18T09:10:39.609 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120306.log 2023-10-18T09:10:39.611 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106540.log 2023-10-18T09:10:39.611 INFO:teuthology.orchestra.run.smithi040.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.91881.log.gz 2023-10-18T09:10:39.612 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120306.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.120306.log.gz 2023-10-18T09:10:39.613 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94001.log 2023-10-18T09:10:39.614 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102470.log 2023-10-18T09:10:39.615 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106540.log.gz 2023-10-18T09:10:39.615 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94001.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.94001.log.gz 2023-10-18T09:10:39.616 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102993.log 2023-10-18T09:10:39.618 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.102470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107162.log 2023-10-18T09:10:39.618 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.102470.log.gz 2023-10-18T09:10:39.619 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.102993.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.102993.log.gz 2023-10-18T09:10:39.619 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97303.log 2023-10-18T09:10:39.621 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97639.log 2023-10-18T09:10:39.621 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.107162.log.gz 2023-10-18T09:10:39.622 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97303.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97303.log.gz 2023-10-18T09:10:39.622 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93252.log 2023-10-18T09:10:39.624 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118458.log 2023-10-18T09:10:39.624 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97639.log.gz 2023-10-18T09:10:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93252.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93252.log.gz 2023-10-18T09:10:39.625 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111707.log 2023-10-18T09:10:39.627 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.118458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89574.log 2023-10-18T09:10:39.627 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.118458.log.gz 2023-10-18T09:10:39.628 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111707.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.111707.log.gz 2023-10-18T09:10:39.628 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91375.log 2023-10-18T09:10:39.630 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98592.log 2023-10-18T09:10:39.630 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.89574.log.gz 2023-10-18T09:10:39.631 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91375.log: gzip 87.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.91375.log.gz -- 2023-10-18T09:10:39.632 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.123629.log 2023-10-18T09:10:39.632 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98592.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.98592.log.gz 2023-10-18T09:10:39.633 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93562.log 2023-10-18T09:10:39.634 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93436.log 2023-10-18T09:10:39.635 INFO:teuthology.orchestra.run.smithi040.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.123629.log.gz 2023-10-18T09:10:39.635 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93562.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93562.log.gz 2023-10-18T09:10:39.636 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91659.log 2023-10-18T09:10:39.637 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118290.log 2023-10-18T09:10:39.638 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93436.log.gz 2023-10-18T09:10:39.638 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91659.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91659.log.gz 2023-10-18T09:10:39.639 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96072.log 2023-10-18T09:10:39.640 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.118290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111875.log 2023-10-18T09:10:39.641 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118290.log.gz 2023-10-18T09:10:39.642 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96072.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.96072.log.gz 2023-10-18T09:10:39.642 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112412.log 2023-10-18T09:10:39.644 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85260.log 2023-10-18T09:10:39.644 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.111875.log.gz 2023-10-18T09:10:39.645 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.112412.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.112412.log.gz 2023-10-18T09:10:39.645 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88748.log 2023-10-18T09:10:39.646 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108418.log 2023-10-18T09:10:39.647 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.85260.log.gz 2023-10-18T09:10:39.647 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88748.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88748.log.gz 2023-10-18T09:10:39.648 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99361.log 2023-10-18T09:10:39.649 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.108418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118873.log 2023-10-18T09:10:39.649 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.108418.log.gz 2023-10-18T09:10:39.650 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.99361.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.99361.log.gz 2023-10-18T09:10:39.650 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106498.log 2023-10-18T09:10:39.651 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.118873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111665.log 2023-10-18T09:10:39.651 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.118873.log.gz 2023-10-18T09:10:39.652 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106498.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106498.log.gz 2023-10-18T09:10:39.653 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83488.log 2023-10-18T09:10:39.654 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110367.log 2023-10-18T09:10:39.654 INFO:teuthology.orchestra.run.smithi040.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.111665.log.gz 2023-10-18T09:10:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83488.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.83488.log.gz 2023-10-18T09:10:39.655 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94674.log 2023-10-18T09:10:39.656 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101551.log 2023-10-18T09:10:39.656 INFO:teuthology.orchestra.run.smithi040.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.110367.log.gz 2023-10-18T09:10:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94674.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.94674.log.gz 2023-10-18T09:10:39.657 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101761.log 2023-10-18T09:10:39.658 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93048.log 2023-10-18T09:10:39.659 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101551.log.gz 2023-10-18T09:10:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101761.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.101761.log.gz 2023-10-18T09:10:39.660 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85563.log 2023-10-18T09:10:39.661 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86790.log 2023-10-18T09:10:39.661 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93048.log.gz 2023-10-18T09:10:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85563.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.85563.log.gz 2023-10-18T09:10:39.662 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82918.log 2023-10-18T09:10:39.663 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85551.log 2023-10-18T09:10:39.663 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86790.log.gz 2023-10-18T09:10:39.664 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82918.log: 76.7% -- replaced with /var/log/ceph/ceph-client.admin.82918.log.gz 2023-10-18T09:10:39.665 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114208.log 2023-10-18T09:10:39.666 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111791.log 2023-10-18T09:10:39.666 INFO:teuthology.orchestra.run.smithi040.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.85551.log.gz 2023-10-18T09:10:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114208.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114208.log.gz 2023-10-18T09:10:39.667 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115379.log 2023-10-18T09:10:39.668 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115940.log 2023-10-18T09:10:39.668 INFO:teuthology.orchestra.run.smithi040.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.111791.log.gz 2023-10-18T09:10:39.669 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.115379.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.115379.log.gz 2023-10-18T09:10:39.670 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90541.log 2023-10-18T09:10:39.671 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.115940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99964.log 2023-10-18T09:10:39.671 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.115940.log.gz 2023-10-18T09:10:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90541.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90541.log.gz 2023-10-18T09:10:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124030.log 2023-10-18T09:10:39.673 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.99964.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99964.log.gz 2023-10-18T09:10:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105977.log 2023-10-18T09:10:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124030.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.124030.log.gz 2023-10-18T09:10:39.675 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83435.log 2023-10-18T09:10:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.105977.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105977.log.gz 2023-10-18T09:10:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85559.log 2023-10-18T09:10:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83435.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.83435.log.gz 2023-10-18T09:10:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90247.log 2023-10-18T09:10:39.679 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95458.log 2023-10-18T09:10:39.679 INFO:teuthology.orchestra.run.smithi040.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.85559.log.gz 2023-10-18T09:10:39.680 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90247.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90247.log.gz 2023-10-18T09:10:39.680 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96951.log 2023-10-18T09:10:39.681 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124114.log 2023-10-18T09:10:39.682 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.95458.log.gz 2023-10-18T09:10:39.682 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96951.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.96951.log.gz 2023-10-18T09:10:39.683 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120180.log 2023-10-18T09:10:39.684 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124114.log: 86.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.124114.log.gz -5 2023-10-18T09:10:39.684 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.110764.log 2023-10-18T09:10:39.685 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120180.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.120180.log.gz 2023-10-18T09:10:39.686 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102013.log 2023-10-18T09:10:39.687 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101803.log 2023-10-18T09:10:39.687 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.110764.log.gz 2023-10-18T09:10:39.688 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.102013.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.102013.log.gz 2023-10-18T09:10:39.688 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-18T09:10:39.689 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107816.log 2023-10-18T09:10:39.690 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.101803.log.gz 2023-10-18T09:10:39.691 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103035.log 2023-10-18T09:10:39.692 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107816.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.107816.log.gz 2023-10-18T09:10:39.693 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107984.log 2023-10-18T09:10:39.693 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107373.log 2023-10-18T09:10:39.694 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103035.log.gz 2023-10-18T09:10:39.695 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107984.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.107984.log.gz 2023-10-18T09:10:39.695 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124240.log 2023-10-18T09:10:39.696 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106252.log 2023-10-18T09:10:39.697 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107373.log.gz 2023-10-18T09:10:39.698 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124240.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.124240.log.gz 2023-10-18T09:10:39.698 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124660.log 2023-10-18T09:10:39.699 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105704.log 2023-10-18T09:10:39.699 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106252.log.gz 2023-10-18T09:10:39.701 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124660.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.124660.log.gz 2023-10-18T09:10:39.701 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109929.log 2023-10-18T09:10:39.702 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.105704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94762.log 2023-10-18T09:10:39.702 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.105704.log.gz 2023-10-18T09:10:39.703 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.109929.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109929.log.gz 2023-10-18T09:10:39.703 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110199.log 2023-10-18T09:10:39.704 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91794.log 2023-10-18T09:10:39.705 INFO:teuthology.orchestra.run.smithi040.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.94762.log.gz 2023-10-18T09:10:39.706 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110199.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110199.log.gz 2023-10-18T09:10:39.706 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113937.log 2023-10-18T09:10:39.707 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89317.log 2023-10-18T09:10:39.708 INFO:teuthology.orchestra.run.smithi040.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.91794.log.gz 2023-10-18T09:10:39.708 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113937.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113937.log.gz 2023-10-18T09:10:39.709 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88202.log 2023-10-18T09:10:39.710 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89317.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89317.log.gz 2023-10-18T09:10:39.710 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84626.log 2023-10-18T09:10:39.711 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116721.log 2023-10-18T09:10:39.712 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88202.log.gz 2023-10-18T09:10:39.712 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84626.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.84626.log.gz 2023-10-18T09:10:39.713 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89791.log 2023-10-18T09:10:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116721.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116721.log.gz 2023-10-18T09:10:39.714 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109829.log 2023-10-18T09:10:39.715 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113339.log 2023-10-18T09:10:39.716 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.89791.log.gz 2023-10-18T09:10:39.717 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.109829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86874.log 2023-10-18T09:10:39.717 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.109829.log.gz 2023-10-18T09:10:39.719 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90793.log 2023-10-18T09:10:39.719 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.113339.log.gz 2023-10-18T09:10:39.720 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86874.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86874.log.gz 2023-10-18T09:10:39.720 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88496.log 2023-10-18T09:10:39.721 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97429.log 2023-10-18T09:10:39.721 INFO:teuthology.orchestra.run.smithi040.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.90793.log.gz 2023-10-18T09:10:39.722 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88496.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.88496.log.gz 2023-10-18T09:10:39.723 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88370.log 2023-10-18T09:10:39.723 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119909.log 2023-10-18T09:10:39.724 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97429.log.gz 2023-10-18T09:10:39.725 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88370.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88370.log.gz 2023-10-18T09:10:39.725 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108544.log 2023-10-18T09:10:39.726 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119909.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.119909.log.gz 2023-10-18T09:10:39.727 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116249.log 2023-10-18T09:10:39.728 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.108544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123106.log 2023-10-18T09:10:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116249.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.108544.log.gz 2023-10-18T09:10:39.729 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.116249.log.gz 2023-10-18T09:10:39.729 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98338.log 2023-10-18T09:10:39.730 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110097.log 2023-10-18T09:10:39.731 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.123106.log.gz 2023-10-18T09:10:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98338.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.98338.log.gz 2023-10-18T09:10:39.731 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82930.log 2023-10-18T09:10:39.732 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104681.log 2023-10-18T09:10:39.733 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.110097.log.gz 2023-10-18T09:10:39.733 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82930.log: 76.6% -- replaced with /var/log/ceph/ceph-client.admin.82930.log.gz 2023-10-18T09:10:39.734 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114123.log 2023-10-18T09:10:39.735 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88538.log 2023-10-18T09:10:39.735 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104681.log.gz 2023-10-18T09:10:39.736 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114123.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.114123.log.gz 2023-10-18T09:10:39.736 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86958.log 2023-10-18T09:10:39.737 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106019.log 2023-10-18T09:10:39.738 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88538.log.gz 2023-10-18T09:10:39.738 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86958.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86958.log.gz 2023-10-18T09:10:39.739 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107716.log 2023-10-18T09:10:39.740 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85562.log 2023-10-18T09:10:39.740 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106019.log.gz 2023-10-18T09:10:39.741 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107716.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.107716.log.gz 2023-10-18T09:10:39.741 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89659.log 2023-10-18T09:10:39.742 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85050.log 2023-10-18T09:10:39.742 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.85562.log.gz 2023-10-18T09:10:39.743 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89659.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.89659.log.gz 2023-10-18T09:10:39.744 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83221.log 2023-10-18T09:10:39.745 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86562.log 2023-10-18T09:10:39.745 INFO:teuthology.orchestra.run.smithi040.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.85050.log.gz 2023-10-18T09:10:39.746 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83221.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.83221.log.gz 2023-10-18T09:10:39.746 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83246.log 2023-10-18T09:10:39.747 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111319.log 2023-10-18T09:10:39.747 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86562.log.gz 2023-10-18T09:10:39.748 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83246.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.83246.log.gz 2023-10-18T09:10:39.748 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100401.log 2023-10-18T09:10:39.749 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85903.log 2023-10-18T09:10:39.750 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111319.log.gz 2023-10-18T09:10:39.750 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100401.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100401.log.gz 2023-10-18T09:10:39.751 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100756.log 2023-10-18T09:10:39.751 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101719.log 2023-10-18T09:10:39.752 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.85903.log.gz 2023-10-18T09:10:39.753 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100756.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100756.log.gz 2023-10-18T09:10:39.753 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103785.log 2023-10-18T09:10:39.754 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97051.log 2023-10-18T09:10:39.754 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101719.log.gz 2023-10-18T09:10:39.755 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103785.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.103785.log.gz 2023-10-18T09:10:39.755 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96867.log 2023-10-18T09:10:39.756 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112879.log 2023-10-18T09:10:39.757 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97051.log.gz 2023-10-18T09:10:39.758 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96867.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.96867.log.gz 2023-10-18T09:10:39.758 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111917.log 2023-10-18T09:10:39.759 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.112879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108122.log 2023-10-18T09:10:39.759 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.112879.log.gz 2023-10-18T09:10:39.760 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111917.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.111917.log.gz 2023-10-18T09:10:39.760 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117473.log 2023-10-18T09:10:39.761 INFO:teuthology.orchestra.run.smithi040.stderr:gzip/var/log/ceph/ceph-client.admin.108122.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.91123.log 2023-10-18T09:10:39.762 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108122.log.gz 2023-10-18T09:10:39.762 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117473.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117473.log.gz 2023-10-18T09:10:39.763 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102428.log 2023-10-18T09:10:39.764 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93394.log 2023-10-18T09:10:39.764 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91123.log.gz 2023-10-18T09:10:39.765 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.102428.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102428.log.gz 2023-10-18T09:10:39.765 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110283.log 2023-10-18T09:10:39.766 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101119.log 2023-10-18T09:10:39.766 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93394.log.gz 2023-10-18T09:10:39.767 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110283.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.110283.log.gz 2023-10-18T09:10:39.767 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96519.log 2023-10-18T09:10:39.769 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96663.log 2023-10-18T09:10:39.769 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101119.log.gz 2023-10-18T09:10:39.770 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96519.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96519.log.gz 2023-10-18T09:10:39.770 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98166.log 2023-10-18T09:10:39.771 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105875.log 2023-10-18T09:10:39.771 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96663.log.gz 2023-10-18T09:10:39.772 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98166.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98166.log.gz 2023-10-18T09:10:39.772 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99403.log 2023-10-18T09:10:39.773 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.105875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103262.log 2023-10-18T09:10:39.774 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.105875.log.gz 2023-10-18T09:10:39.774 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.99403.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99403.log.gz 2023-10-18T09:10:39.775 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111189.log 2023-10-18T09:10:39.775 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113811.log 2023-10-18T09:10:39.776 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103262.log.gz 2023-10-18T09:10:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111189.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111189.log.gz 2023-10-18T09:10:39.777 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123671.log 2023-10-18T09:10:39.778 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115505.log 2023-10-18T09:10:39.778 INFO:teuthology.orchestra.run.smithi040.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.113811.log.gz 2023-10-18T09:10:39.779 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123671.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.123671.log.gz 2023-10-18T09:10:39.779 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121037.log 2023-10-18T09:10:39.780 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.115505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110596.log 2023-10-18T09:10:39.781 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.121037.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.115505.log.gz 2023-10-18T09:10:39.782 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.121037.log.gz 2023-10-18T09:10:39.782 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115211.log 2023-10-18T09:10:39.783 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107541.log 2023-10-18T09:10:39.783 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110596.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.110596.log.gz 2023-10-18T09:10:39.784 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.115211.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.115211.log.gz 2023-10-18T09:10:39.784 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97177.log 2023-10-18T09:10:39.785 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115878.log 2023-10-18T09:10:39.786 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.107541.log.gz 2023-10-18T09:10:39.786 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97177.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97177.log.gz 2023-10-18T09:10:39.787 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95955.log 2023-10-18T09:10:39.788 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.115878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97387.log 2023-10-18T09:10:39.788 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.115878.log.gz 2023-10-18T09:10:39.789 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95955.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95955.log.gz 2023-10-18T09:10:39.789 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98210.log 2023-10-18T09:10:39.790 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113895.log 2023-10-18T09:10:39.791 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97387.log.gz 2023-10-18T09:10:39.791 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98210.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98210.log.gz 2023-10-18T09:10:39.791 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84882.log 2023-10-18T09:10:39.792 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117856.log 2023-10-18T09:10:39.793 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113895.log.gz 2023-10-18T09:10:39.793 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84882.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.84882.log.gz 2023-10-18T09:10:39.794 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121417.log 2023-10-18T09:10:39.795 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100106.log 2023-10-18T09:10:39.795 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117856.log.gz 2023-10-18T09:10:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.121417.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121417.log.gz 2023-10-18T09:10:39.796 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108376.log 2023-10-18T09:10:39.797 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108610.log 2023-10-18T09:10:39.798 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100106.log.gz 2023-10-18T09:10:39.798 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.108376.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108376.log.gz 2023-10-18T09:10:39.799 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111539.log 2023-10-18T09:10:39.800 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.108610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90415.log 2023-10-18T09:10:39.800 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108610.log.gz 2023-10-18T09:10:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111539.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.111539.log.gz 2023-10-18T09:10:39.801 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83457.log 2023-10-18T09:10:39.802 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104809.log 2023-10-18T09:10:39.802 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.90415.log.gz 2023-10-18T09:10:39.803 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83457.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.83457.log.gz 2023-10-18T09:10:39.803 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112350.log 2023-10-18T09:10:39.804 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121850.log 2023-10-18T09:10:39.805 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.104809.log.gz 2023-10-18T09:10:39.805 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.112350.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.112350.log.gz 2023-10-18T09:10:39.806 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110427.log 2023-10-18T09:10:39.806 INFO:teuthology.orchestra.run.smithi144.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-18T09:10:39.807 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.121850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124901.log 2023-10-18T09:10:39.807 INFO:teuthology.orchestra.run.smithi040.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.121850.log.gz 2023-10-18T09:10:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110427.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.110427.log.gz 2023-10-18T09:10:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120569.log 2023-10-18T09:10:39.809 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121291.log 2023-10-18T09:10:39.810 INFO:teuthology.orchestra.run.smithi040.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.124901.log.gz 2023-10-18T09:10:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120569.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.120569.log.gz 2023-10-18T09:10:39.811 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118028.log 2023-10-18T09:10:39.812 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.121291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97964.log 2023-10-18T09:10:39.812 INFO:teuthology.orchestra.run.smithi040.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.121291.log.gz 2023-10-18T09:10:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.118028.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.118028.log.gz 2023-10-18T09:10:39.813 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83356.log 2023-10-18T09:10:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109158.log 2023-10-18T09:10:39.814 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97964.log.gz 2023-10-18T09:10:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83356.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.83356.log.gz 2023-10-18T09:10:39.816 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94846.log 2023-10-18T09:10:39.817 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.109158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96621.log 2023-10-18T09:10:39.817 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109158.log.gz 2023-10-18T09:10:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94846.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94846.log.gz 2023-10-18T09:10:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85092.log 2023-10-18T09:10:39.819 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109457.log 2023-10-18T09:10:39.819 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96621.log.gz 2023-10-18T09:10:39.820 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85092.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.85092.log.gz 2023-10-18T09:10:39.820 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104977.log 2023-10-18T09:10:39.821 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.109457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106336.log 2023-10-18T09:10:39.822 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.109457.log.gz 2023-10-18T09:10:39.822 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104977.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104977.log.gz 2023-10-18T09:10:39.823 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89617.log 2023-10-18T09:10:39.824 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122383.log 2023-10-18T09:10:39.824 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.106336.log.gz 2023-10-18T09:10:39.825 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89617.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.89617.log.gz 2023-10-18T09:10:39.825 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-18T09:10:39.826 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96030.log 2023-10-18T09:10:39.827 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122383.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122383.log.gz 2023-10-18T09:10:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111369.log 2023-10-18T09:10:39.829 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96030.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.96030.log.gz 2023-10-18T09:10:39.829 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121525.log 2023-10-18T09:10:39.830 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121912.log 2023-10-18T09:10:39.831 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.111369.log.gz 2023-10-18T09:10:39.832 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.121525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103219.log 2023-10-18T09:10:39.832 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.121525.log.gz 2023-10-18T09:10:39.833 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.121912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110554.log 2023-10-18T09:10:39.834 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.121912.log.gz 2023-10-18T09:10:39.835 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111454.log 2023-10-18T09:10:39.835 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103219.log.gz 2023-10-18T09:10:39.837 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98382.log 2023-10-18T09:10:39.837 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110554.log.gz 2023-10-18T09:10:39.837 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91701.log 2023-10-18T09:10:39.838 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111454.log.gz 2023-10-18T09:10:39.838 INFO:teuthology.orchestra.run.smithi040.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-18T09:10:39.839 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93959.log 2023-10-18T09:10:39.839 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.98382.log.gz 2023-10-18T09:10:39.840 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100696.log 2023-10-18T09:10:39.841 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91701.log.gz 2023-10-18T09:10:39.841 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93959.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93959.log.gz 2023-10-18T09:10:39.841 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101971.log 2023-10-18T09:10:39.842 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88790.log 2023-10-18T09:10:39.843 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100696.log.gz 2023-10-18T09:10:39.843 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101971.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.101971.log.gz 2023-10-18T09:10:39.844 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100444.log 2023-10-18T09:10:39.845 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101593.log 2023-10-18T09:10:39.845 INFO:teuthology.orchestra.run.smithi040.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.88790.log.gz 2023-10-18T09:10:39.846 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100444.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.100444.log.gz 2023-10-18T09:10:39.846 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112981.log 2023-10-18T09:10:39.847 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84334.log 2023-10-18T09:10:39.848 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.101593.log.gz 2023-10-18T09:10:39.848 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.112981.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112981.log.gz 2023-10-18T09:10:39.849 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110722.log 2023-10-18T09:10:39.850 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86478.log 2023-10-18T09:10:39.850 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.84334.log.gz 2023-10-18T09:10:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110722.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.110722.log.gz 2023-10-18T09:10:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119951.log 2023-10-18T09:10:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89833.log 2023-10-18T09:10:39.852 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86478.log.gz 2023-10-18T09:10:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119951.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.119951.log.gz 2023-10-18T09:10:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94145.log 2023-10-18T09:10:39.854 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107858.log 2023-10-18T09:10:39.855 INFO:teuthology.orchestra.run.smithi040.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.89833.log.gz 2023-10-18T09:10:39.855 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94145.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.94145.log.gz 2023-10-18T09:10:39.856 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84966.log 2023-10-18T09:10:39.857 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95308.log 2023-10-18T09:10:39.857 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.107858.log.gz 2023-10-18T09:10:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84966.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.84966.log.gz 2023-10-18T09:10:39.858 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102873.log 2023-10-18T09:10:39.859 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108250.log 2023-10-18T09:10:39.860 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.95308.log.gz 2023-10-18T09:10:39.860 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.102873.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.102873.log.gz 2023-10-18T09:10:39.861 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114460.log 2023-10-18T09:10:39.862 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.108250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90457.log 2023-10-18T09:10:39.862 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.108250.log.gz 2023-10-18T09:10:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114460.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114460.log.gz 2023-10-18T09:10:39.863 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114677.log 2023-10-18T09:10:39.864 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98508.log 2023-10-18T09:10:39.864 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.90457.log.gz 2023-10-18T09:10:39.865 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114677.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114677.log.gz 2023-10-18T09:10:39.866 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114885.log 2023-10-18T09:10:39.866 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103953.log 2023-10-18T09:10:39.867 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98508.log.gz 2023-10-18T09:10:39.867 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114885.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114885.log.gz 2023-10-18T09:10:39.868 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105629.log 2023-10-18T09:10:39.869 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123232.log 2023-10-18T09:10:39.869 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.103953.log.gz 2023-10-18T09:10:39.870 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.105629.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105629.log.gz 2023-10-18T09:10:39.870 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114971.log 2023-10-18T09:10:39.871 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106976.log 2023-10-18T09:10:39.872 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.123232.log.gz 2023-10-18T09:10:39.872 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114971.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.114971.log.gz 2023-10-18T09:10:39.873 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105935.log 2023-10-18T09:10:39.874 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85897.log 2023-10-18T09:10:39.874 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106976.log.gz 2023-10-18T09:10:39.875 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.105935.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105935.log.gz 2023-10-18T09:10:39.876 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116291.log 2023-10-18T09:10:39.877 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85897.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.85897.log.gz 2023-10-18T09:10:39.877 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113417.log 2023-10-18T09:10:39.878 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116291.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116291.log.gz 2023-10-18T09:10:39.879 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111062.log 2023-10-18T09:10:39.879 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113417.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.113417.log.gz 2023-10-18T09:10:39.880 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116333.log 2023-10-18T09:10:39.881 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90037.log 2023-10-18T09:10:39.882 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111062.log.gz 2023-10-18T09:10:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116333.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116333.log.gz 2023-10-18T09:10:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94888.log 2023-10-18T09:10:39.885 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90037.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90037.log.gz 2023-10-18T09:10:39.885 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101635.log 2023-10-18T09:10:39.886 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94888.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.94888.log.gz 2023-10-18T09:10:39.886 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106808.log 2023-10-18T09:10:39.887 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101635.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101635.log.gz 2023-10-18T09:10:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103304.log 2023-10-18T09:10:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119639.log 2023-10-18T09:10:39.889 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.106808.log.gz 2023-10-18T09:10:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103304.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103304.log.gz 2023-10-18T09:10:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92922.log 2023-10-18T09:10:39.891 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86215.log 2023-10-18T09:10:39.892 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.119639.log.gz 2023-10-18T09:10:39.893 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.92922.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.92922.log.gz 2023-10-18T09:10:39.893 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103388.log 2023-10-18T09:10:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86215.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86215.log.gz 2023-10-18T09:10:39.894 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122988.log 2023-10-18T09:10:39.895 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109971.log 2023-10-18T09:10:39.896 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.103388.log.gz 2023-10-18T09:10:39.897 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122988.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122988.log.gz 2023-10-18T09:10:39.897 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89000.log 2023-10-18T09:10:39.898 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.109971.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109971.log.gz 2023-10-18T09:10:39.898 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88286.log 2023-10-18T09:10:39.899 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109703.log 2023-10-18T09:10:39.900 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.89000.log.gz 2023-10-18T09:10:39.901 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88286.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.88286.log.gz 2023-10-18T09:10:39.901 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-18T09:10:39.902 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.109703.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.109703.log.gz 2023-10-18T09:10:39.902 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123274.log 2023-10-18T09:10:39.903 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98252.log 2023-10-18T09:10:39.905 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123274.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.123274.log.gz 2023-10-18T09:10:39.905 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111231.log 2023-10-18T09:10:39.906 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98252.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.98252.log.gz 2023-10-18T09:10:39.907 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98466.log 2023-10-18T09:10:39.908 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111231.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111231.log.gz 2023-10-18T09:10:39.908 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124450.log 2023-10-18T09:10:39.909 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116453.log 2023-10-18T09:10:39.909 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98466.log.gz 2023-10-18T09:10:39.910 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124450.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.124450.log.gz 2023-10-18T09:10:39.911 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83160.log 2023-10-18T09:10:39.912 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116453.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.116453.log.gz 2023-10-18T09:10:39.912 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104232.log 2023-10-18T09:10:39.913 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85899.log 2023-10-18T09:10:39.914 INFO:teuthology.orchestra.run.smithi040.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.83160.log.gz 2023-10-18T09:10:39.915 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104232.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104232.log.gz 2023-10-18T09:10:39.915 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106934.log 2023-10-18T09:10:39.916 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85899.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85899.log.gz 2023-10-18T09:10:39.916 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122946.log 2023-10-18T09:10:39.918 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106934.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106934.log.gz 2023-10-18T09:10:39.918 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93006.log 2023-10-18T09:10:39.919 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122946.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122946.log.gz 2023-10-18T09:10:39.920 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108028.log 2023-10-18T09:10:39.921 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93006.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.93006.log.gz 2023-10-18T09:10:39.921 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116848.log 2023-10-18T09:10:39.922 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.108028.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.108028.log.gz 2023-10-18T09:10:39.923 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114021.log 2023-10-18T09:10:39.924 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116848.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.116848.log.gz 2023-10-18T09:10:39.924 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95913.log 2023-10-18T09:10:39.925 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114021.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114021.log.gz 2023-10-18T09:10:39.926 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86238.log 2023-10-18T09:10:39.927 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95913.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.95913.log.gz 2023-10-18T09:10:39.927 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90667.log 2023-10-18T09:10:39.928 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86238.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86238.log.gz 2023-10-18T09:10:39.929 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110901.log 2023-10-18T09:10:39.930 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90667.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.90667.log.gz 2023-10-18T09:10:39.930 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89917.log 2023-10-18T09:10:39.931 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110901.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.110901.log.gz 2023-10-18T09:10:39.932 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116805.log 2023-10-18T09:10:39.933 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82989.log 2023-10-18T09:10:39.933 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.89917.log.gz 2023-10-18T09:10:39.934 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116805.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116805.log.gz 2023-10-18T09:10:39.934 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89444.log 2023-10-18T09:10:39.935 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.82989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119293.log 2023-10-18T09:10:39.935 INFO:teuthology.orchestra.run.smithi040.stderr: 78.0% -- replaced with /var/log/ceph/ceph-client.admin.82989.log.gz 2023-10-18T09:10:39.936 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.89444.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.89444.log.gz 2023-10-18T09:10:39.937 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104057.log 2023-10-18T09:10:39.938 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108502.log 2023-10-18T09:10:39.938 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.119293.log.gz 2023-10-18T09:10:39.939 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104057.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.104057.log.gz 2023-10-18T09:10:39.939 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100798.log 2023-10-18T09:10:39.941 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.108502.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.108502.log.gz 2023-10-18T09:10:39.941 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94424.log 2023-10-18T09:10:39.942 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100798.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100798.log.gz 2023-10-18T09:10:39.942 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112741.log 2023-10-18T09:10:39.944 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94424.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94424.log.gz 2023-10-18T09:10:39.944 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113685.log 2023-10-18T09:10:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.112741.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.112741.log.gz 2023-10-18T09:10:39.945 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114292.log 2023-10-18T09:10:39.947 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113685.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.113685.log.gz 2023-10-18T09:10:39.947 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113853.log 2023-10-18T09:10:39.948 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.114292.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.114292.log.gz 2023-10-18T09:10:39.949 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-18T09:10:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.113853.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.113853.log.gz 2023-10-18T09:10:39.950 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99676.log 2023-10-18T09:10:39.951 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109074.log 2023-10-18T09:10:39.953 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.99676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122904.log 2023-10-18T09:10:39.953 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99676.log.gz 2023-10-18T09:10:39.954 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.109074.log: 87.0% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109074.log.gz 2023-10-18T09:10:39.955 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2023-10-18T09:10:39.955 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122700.log 2023-10-18T09:10:39.956 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85567.log 2023-10-18T09:10:39.956 INFO:teuthology.orchestra.run.smithi040.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.122904.log.gz 2023-10-18T09:10:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122700.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.122700.log.gz 2023-10-18T09:10:39.957 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121165.log 2023-10-18T09:10:39.958 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85567.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-18T09:10:39.959 INFO:teuthology.orchestra.run.smithi040.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.85567.log.gz 2023-10-18T09:10:39.961 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.121165.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121165.log.gz 2023-10-18T09:10:39.962 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94061.log 2023-10-18T09:10:39.962 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109541.log 2023-10-18T09:10:39.963 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119209.log 2023-10-18T09:10:39.963 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94061.log.gz 2023-10-18T09:10:39.964 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.109541.log: gzip 87.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.109541.log.gz --verbose 2023-10-18T09:10:39.964 INFO:teuthology.orchestra.run.smithi040.stderr: -- /var/log/ceph/ceph-client.admin.101413.log 2023-10-18T09:10:39.965 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119209.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-18T09:10:39.966 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.119209.log.gz 2023-10-18T09:10:39.967 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101413.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.101413.log.gz 2023-10-18T09:10:39.967 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123030.log 2023-10-18T09:10:39.968 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119783.log 2023-10-18T09:10:39.972 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123030.log: /var/log/ceph/ceph-client.admin.119783.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.119783.log.gz 2023-10-18T09:10:39.972 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.123030.log.gz 2023-10-18T09:10:39.972 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84415.log 2023-10-18T09:10:39.974 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122784.log 2023-10-18T09:10:39.975 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84415.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.84415.log.gz 2023-10-18T09:10:39.976 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110055.log 2023-10-18T09:10:39.977 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122784.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.122784.log.gz 2023-10-18T09:10:39.978 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116932.log 2023-10-18T09:10:39.979 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110055.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.110055.log.gz 2023-10-18T09:10:39.980 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97009.log 2023-10-18T09:10:39.980 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116932.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.116932.log.gz 2023-10-18T09:10:39.981 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101929.log 2023-10-18T09:10:39.983 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97009.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97009.log.gz 2023-10-18T09:10:39.984 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120035.log 2023-10-18T09:10:39.985 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101929.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.101929.log.gz 2023-10-18T09:10:39.985 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108334.log 2023-10-18T09:10:39.987 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120035.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.120035.log.gz 2023-10-18T09:10:39.987 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115169.log 2023-10-18T09:10:39.989 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.108334.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108334.log.gz 2023-10-18T09:10:39.989 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104639.log 2023-10-18T09:10:39.991 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.115169.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.115169.log.gz 2023-10-18T09:10:39.992 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124702.log 2023-10-18T09:10:39.992 INFO:teuthology.orchestra.run.smithi144.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-18T09:10:39.993 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104639.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104639.log.gz 2023-10-18T09:10:39.994 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117371.log 2023-10-18T09:10:39.995 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124702.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.124702.log.gz 2023-10-18T09:10:39.996 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88832.log 2023-10-18T09:10:39.997 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117371.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117371.log.gz 2023-10-18T09:10:39.997 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85008.log 2023-10-18T09:10:39.999 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88832.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88832.log.gz 2023-10-18T09:10:40.000 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123988.log 2023-10-18T09:10:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85008.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.85008.log.gz 2023-10-18T09:10:40.001 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124156.log 2023-10-18T09:10:40.003 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123988.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.123988.log.gz 2023-10-18T09:10:40.004 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109499.log 2023-10-18T09:10:40.005 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124156.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.124156.log.gz 2023-10-18T09:10:40.006 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106456.log 2023-10-18T09:10:40.007 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.109499.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109499.log.gz 2023-10-18T09:10:40.007 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119597.log 2023-10-18T09:10:40.008 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106456.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106456.log.gz 2023-10-18T09:10:40.009 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120773.log 2023-10-18T09:10:40.010 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119597.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.119597.log.gz 2023-10-18T09:10:40.011 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107900.log 2023-10-18T09:10:40.012 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120773.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.120773.log.gz 2023-10-18T09:10:40.013 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96349.log 2023-10-18T09:10:40.014 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107900.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107900.log.gz 2023-10-18T09:10:40.015 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87222.log 2023-10-18T09:10:40.016 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96349.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96349.log.gz 2023-10-18T09:10:40.017 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93168.log 2023-10-18T09:10:40.018 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.87222.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.87222.log.gz 2023-10-18T09:10:40.019 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107942.log 2023-10-18T09:10:40.020 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93168.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93168.log.gz 2023-10-18T09:10:40.021 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108979.log 2023-10-18T09:10:40.023 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107942.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.107942.log.gz 2023-10-18T09:10:40.023 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96391.log 2023-10-18T09:10:40.024 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.108979.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.108979.log.gz 2023-10-18T09:10:40.025 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112699.log 2023-10-18T09:10:40.026 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96391.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.96391.log.gz 2023-10-18T09:10:40.027 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.87987.log 2023-10-18T09:10:40.028 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.112699.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.112699.log.gz 2023-10-18T09:10:40.029 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90625.log 2023-10-18T09:10:40.030 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.2.87987.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.87987.log.gz 2023-10-18T09:10:40.031 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88117.log 2023-10-18T09:10:40.032 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90625.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.90625.log.gz 2023-10-18T09:10:40.033 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116207.log 2023-10-18T09:10:40.034 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88117.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88117.log.gz 2023-10-18T09:10:40.035 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85344.log 2023-10-18T09:10:40.036 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.116207.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.116207.log.gz 2023-10-18T09:10:40.037 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109116.log 2023-10-18T09:10:40.038 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85344.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.85344.log.gz 2023-10-18T09:10:40.039 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103346.log 2023-10-18T09:10:40.040 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.109116.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109116.log.gz 2023-10-18T09:10:40.040 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98294.log 2023-10-18T09:10:40.042 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103346.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103346.log.gz 2023-10-18T09:10:40.042 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93917.log/var/log/ceph/ceph-client.admin.98294.log: 2023-10-18T09:10:40.043 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.98294.log.gz 2023-10-18T09:10:40.044 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90079.log 2023-10-18T09:10:40.045 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93917.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.93917.log.gz 2023-10-18T09:10:40.047 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119371.log 2023-10-18T09:10:40.047 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90079.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90079.log.gz 2023-10-18T09:10:40.048 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90205.log 2023-10-18T09:10:40.050 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119371.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.119371.log.gz 2023-10-18T09:10:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121333.log 2023-10-18T09:10:40.051 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90205.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.90205.log.gz 2023-10-18T09:10:40.052 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122202.log 2023-10-18T09:10:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.121333.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.121333.log.gz 2023-10-18T09:10:40.054 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99592.log 2023-10-18T09:10:40.055 INFO:teuthology.orchestra.run.smithi144.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-18T09:10:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122202.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122202.log.gz 2023-10-18T09:10:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109583.log 2023-10-18T09:10:40.057 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.99592.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99592.log.gz 2023-10-18T09:10:40.058 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100902.log 2023-10-18T09:10:40.060 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.109583.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109583.log.gz 2023-10-18T09:10:40.060 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122862.log 2023-10-18T09:10:40.061 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100902.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.100902.log.gz 2023-10-18T09:10:40.062 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86209.log 2023-10-18T09:10:40.063 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122862.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122862.log.gz 2023-10-18T09:10:40.064 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90121.log 2023-10-18T09:10:40.065 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86209.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86209.log.gz 2023-10-18T09:10:40.066 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84840.log 2023-10-18T09:10:40.067 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90121.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90121.log.gz 2023-10-18T09:10:40.068 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106061.log 2023-10-18T09:10:40.069 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84840.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.84840.log.gz 2023-10-18T09:10:40.070 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86239.log 2023-10-18T09:10:40.071 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106061.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106061.log.gz 2023-10-18T09:10:40.072 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93352.log 2023-10-18T09:10:40.073 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86239.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.86239.log.gz 2023-10-18T09:10:40.074 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97555.log 2023-10-18T09:10:40.075 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93352.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93352.log.gz 2023-10-18T09:10:40.076 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111959.log 2023-10-18T09:10:40.077 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.97555.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.97555.log.gz 2023-10-18T09:10:40.078 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92539.log 2023-10-18T09:10:40.080 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.111959.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.111959.log.gz 2023-10-18T09:10:40.081 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.92539.log: -- /var/log/ceph/ceph-client.admin.123713.log 2023-10-18T09:10:40.081 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92539.log.gz 2023-10-18T09:10:40.082 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108072.log 2023-10-18T09:10:40.084 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123713.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.123713.log.gz 2023-10-18T09:10:40.085 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84584.log 2023-10-18T09:10:40.085 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.108072.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108072.log.gz 2023-10-18T09:10:40.087 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87000.log 2023-10-18T09:10:40.088 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.84584.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.84584.log.gz 2023-10-18T09:10:40.089 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117772.log 2023-10-18T09:10:40.090 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.87000.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.87000.log.gz 2023-10-18T09:10:40.091 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85894.log 2023-10-18T09:10:40.091 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117772.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117772.log.gz 2023-10-18T09:10:40.092 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95056.log 2023-10-18T09:10:40.094 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.85894.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85894.log.gz 2023-10-18T09:10:40.095 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94804.log 2023-10-18T09:10:40.095 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95056.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.95056.log.gz 2023-10-18T09:10:40.096 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86688.log 2023-10-18T09:10:40.098 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94804.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94804.log.gz 2023-10-18T09:10:40.099 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102705.log 2023-10-18T09:10:40.100 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86688.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86688.log.gz 2023-10-18T09:10:40.101 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103556.log 2023-10-18T09:10:40.102 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.102705.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.102705.log.gz 2023-10-18T09:10:40.103 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101200.log 2023-10-18T09:10:40.104 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.103556.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103556.log.gz 2023-10-18T09:10:40.105 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103472.log 2023-10-18T09:10:40.106 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101200.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.101200.log.gz 2023-10-18T09:10:40.107 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.103472.log: -- /var/log/ceph/ceph-client.admin.117413.log 2023-10-18T09:10:40.108 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103472.log.gz 2023-10-18T09:10:40.109 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118500.log 2023-10-18T09:10:40.110 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117413.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.117413.log.gz 2023-10-18T09:10:40.111 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110806.log 2023-10-18T09:10:40.113 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.118500.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.118500.log.gz 2023-10-18T09:10:40.113 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117814.log 2023-10-18T09:10:40.114 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.110806.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.110806.log.gz 2023-10-18T09:10:40.115 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124072.log 2023-10-18T09:10:40.116 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.117814.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117814.log.gz 2023-10-18T09:10:40.117 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119741.log 2023-10-18T09:10:40.118 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.124072.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.124072.log.gz 2023-10-18T09:10:40.119 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91555.log 2023-10-18T09:10:40.120 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119741.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.119741.log.gz 2023-10-18T09:10:40.121 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123755.log 2023-10-18T09:10:40.122 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91555.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.91555.log.gz 2023-10-18T09:10:40.123 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96265.log 2023-10-18T09:10:40.124 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.123755.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.123755.log.gz 2023-10-18T09:10:40.125 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.95645.log 2023-10-18T09:10:40.126 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.96265.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.96265.log.gz 2023-10-18T09:10:40.127 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106294.log 2023-10-18T09:10:40.128 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.95645.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.95645.log.gz 2023-10-18T09:10:40.129 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120087.log 2023-10-18T09:10:40.131 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106294.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.106294.log.gz 2023-10-18T09:10:40.132 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83723.log 2023-10-18T09:10:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120087.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.120087.log.gz 2023-10-18T09:10:40.133 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106378.log 2023-10-18T09:10:40.135 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83723.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.83723.log.gz 2023-10-18T09:10:40.136 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86436.log 2023-10-18T09:10:40.136 INFO:teuthology.orchestra.run.smithi040.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-18T09:10:40.136 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.106378.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106378.log.gz 2023-10-18T09:10:40.137 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83442.log 2023-10-18T09:10:40.138 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86436.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86436.log.gz 2023-10-18T09:10:40.139 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119825.log 2023-10-18T09:10:40.139 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.83442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100612.log 2023-10-18T09:10:40.140 INFO:teuthology.orchestra.run.smithi040.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.83442.log.gz 2023-10-18T09:10:40.141 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119825.log: 87.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93126.log 2023-10-18T09:10:40.141 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.119825.log.gz 2023-10-18T09:10:40.143 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.100612.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95350.log 2023-10-18T09:10:40.143 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100612.log.gz 2023-10-18T09:10:40.145 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90709.log 2023-10-18T09:10:40.145 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93126.log.gz 2023-10-18T09:10:40.145 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.95350.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.95350.log.gz 2023-10-18T09:10:40.146 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104767.log 2023-10-18T09:10:40.147 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90709.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.90709.log.gz 2023-10-18T09:10:40.148 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91463.log 2023-10-18T09:10:40.149 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.104767.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104767.log.gz 2023-10-18T09:10:40.149 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113585.log 2023-10-18T09:10:40.150 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91463.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91463.log.gz 2023-10-18T09:10:40.150 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91165.log 2023-10-18T09:10:40.152 INFO:teuthology.orchestra.run.smithi040.stderr:gzip/var/log/ceph/ceph-client.admin.113585.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.101677.log 2023-10-18T09:10:40.152 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.113585.log.gz 2023-10-18T09:10:40.153 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91165.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-18T09:10:40.153 INFO:teuthology.orchestra.run.smithi040.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.91165.log.gz 2023-10-18T09:10:40.155 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.101677.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122299.log 2023-10-18T09:10:40.155 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101677.log.gz 2023-10-18T09:10:40.156 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98988.log 2023-10-18T09:10:40.158 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.122299.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122299.log.gz 2023-10-18T09:10:40.159 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98988.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98988.log.gz 2023-10-18T09:10:40.159 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108292.log 2023-10-18T09:10:40.160 INFO:teuthology.orchestra.run.smithi144.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-18T09:10:40.161 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93875.log 2023-10-18T09:10:40.162 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.108292.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.108292.log.gz 2023-10-18T09:10:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.93875.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.93875.log.gz 2023-10-18T09:10:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91291.log 2023-10-18T09:10:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120943.log 2023-10-18T09:10:40.167 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91291.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.91291.log.gz 2023-10-18T09:10:40.169 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.120943.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.120943.log.gz 2023-10-18T09:10:40.169 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115295.log 2023-10-18T09:10:40.171 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119539.log 2023-10-18T09:10:40.172 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.115295.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115295.log.gz 2023-10-18T09:10:40.174 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.119539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121790.log 2023-10-18T09:10:40.175 INFO:teuthology.orchestra.run.smithi040.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.119539.log.gz 2023-10-18T09:10:40.177 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.121790.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121790.log.gz 2023-10-18T09:10:40.177 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99634.log 2023-10-18T09:10:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86604.log 2023-10-18T09:10:40.180 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.99634.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.99634.log.gz 2023-10-18T09:10:40.182 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.86604.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86604.log.gz 2023-10-18T09:10:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94547.log 2023-10-18T09:10:40.184 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98802.log 2023-10-18T09:10:40.185 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.94547.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94547.log.gz 2023-10-18T09:10:40.187 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.98802.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98802.log.gz 2023-10-18T09:10:40.187 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91513.log 2023-10-18T09:10:40.189 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118120.log 2023-10-18T09:10:40.190 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91513.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91513.log.gz 2023-10-18T09:10:40.192 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88916.log 2023-10-18T09:10:40.192 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.118120.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.118120.log.gz 2023-10-18T09:10:40.193 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91923.log 2023-10-18T09:10:40.195 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.88916.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.88916.log.gz 2023-10-18T09:10:40.196 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108797.log 2023-10-18T09:10:40.197 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.91923.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.91923.log.gz 2023-10-18T09:10:40.198 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115337.log 2023-10-18T09:10:40.198 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.108797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.108797.log.gz 2023-10-18T09:10:40.200 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90583.log 2023-10-18T09:10:40.200 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.115337.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.115337.log.gz 2023-10-18T09:10:40.201 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107499.log 2023-10-18T09:10:40.202 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.90583.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.90583.log.gz 2023-10-18T09:10:40.203 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107331.log 2023-10-18T09:10:40.204 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107499.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107499.log.gz 2023-10-18T09:10:40.206 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.107331.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107331.log.gz 2023-10-18T09:10:40.208 INFO:teuthology.orchestra.run.smithi144.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-18T09:10:40.213 INFO:teuthology.orchestra.run.smithi144.stderr: 2023-10-18T09:10:40.213 INFO:teuthology.orchestra.run.smithi144.stderr:real 0m1.169s 2023-10-18T09:10:40.213 INFO:teuthology.orchestra.run.smithi144.stderr:user 0m5.038s 2023-10-18T09:10:40.214 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m0.541s 2023-10-18T09:10:40.232 INFO:teuthology.orchestra.run.smithi040.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-18T09:10:40.491 INFO:teuthology.orchestra.run.smithi040.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-18T09:10:40.708 INFO:teuthology.orchestra.run.smithi040.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-18T09:10:40.734 INFO:teuthology.orchestra.run.smithi040.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-18T09:10:40.867 INFO:teuthology.orchestra.run.smithi040.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-18T09:10:41.033 INFO:teuthology.orchestra.run.smithi040.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-18T09:10:41.038 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-10-18T09:10:41.038 INFO:teuthology.orchestra.run.smithi040.stderr:real 0m1.995s 2023-10-18T09:10:41.038 INFO:teuthology.orchestra.run.smithi040.stderr:user 0m5.701s 2023-10-18T09:10:41.038 INFO:teuthology.orchestra.run.smithi040.stderr:sys 0m2.661s 2023-10-18T09:10:41.039 INFO:tasks.ceph:Archiving logs... 2023-10-18T09:10:41.040 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719/remote/smithi040/log 2023-10-18T09:10:41.041 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-18T09:10:42.417 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719/remote/smithi144/log 2023-10-18T09:10:42.418 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-18T09:10:43.158 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-18T09:10:43.167 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-18T09:10:43.168 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-18T09:10:43.177 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-18T09:10:43.326 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-18T09:10:43.335 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-18T09:10:43.336 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T09:10:43.345 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T09:10:43.396 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2023-10-18T09:10:43.402 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2023-10-18T09:10:43.407 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T09:10:43.407 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2023-10-18T09:10:43.407 INFO:teuthology.orchestra.run.smithi040.stdout:^* hv01.front.sepia.ceph.com 3 6 377 51 +2484us[+2479us] +/- 54ms 2023-10-18T09:10:43.408 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 117 -1646us[-1679us] +/- 58ms 2023-10-18T09:10:43.408 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 -1189us[-1189us] +/- 68ms 2023-10-18T09:10:43.408 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:10:43.411 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T09:10:43.412 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2023-10-18T09:10:43.412 INFO:teuthology.orchestra.run.smithi144.stdout:^* hv01.front.sepia.ceph.com 3 6 377 51 +2425us[+2426us] +/- 54ms 2023-10-18T09:10:43.412 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 49 -1685us[-1685us] +/- 59ms 2023-10-18T09:10:43.412 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 -1244us[-1242us] +/- 68ms 2023-10-18T09:10:43.412 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:10:43.413 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-18T09:10:43.423 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-18T09:10:43.423 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-18T09:10:43.431 DEBUG:teuthology.orchestra.run.smithi040:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-18T09:10:43.738 DEBUG:teuthology.orchestra.run.smithi144:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-18T09:10:43.959 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T09:10:44.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:10:44.078 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T09:10:44.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:10:44.186 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-18T09:10:44.196 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-18T09:10:44.204 INFO:teuthology.task.internal:Duration was 1712.643281 seconds 2023-10-18T09:10:44.204 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-18T09:10:44.213 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-18T09:10:44.213 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-18T09:10:44.223 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-18T09:10:44.396 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T09:10:44.400 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T09:10:44.818 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-18T09:10:44.819 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2023-10-18T09:10:44.819 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-18T09:10:44.892 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2023-10-18T09:10:44.893 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-18T09:10:44.979 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-18T09:10:44.980 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-18T09:10:44.988 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-18T09:10:45.090 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-18T09:10:45.091 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T09:10:45.097 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T09:10:45.320 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-18T09:10:45.332 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-18T09:10:45.332 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-18T09:10:45.436 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-18T09:10:45.536 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-18T09:10:45.546 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-18T09:10:45.556 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-18T09:10:45.666 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2023-10-18T09:10:45.669 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2023-10-18T09:10:45.723 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-18T09:10:45.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:10:45.776 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-18T09:10:45.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:10:45.831 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-18T09:10:45.841 INFO:teuthology.task.internal:Transferring archived files... 2023-10-18T09:10:45.842 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719/remote/smithi040 2023-10-18T09:10:45.842 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-18T09:10:45.961 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719/remote/smithi144 2023-10-18T09:10:45.962 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-18T09:10:46.072 INFO:teuthology.task.internal:Removing archive directory... 2023-10-18T09:10:46.072 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-18T09:10:46.082 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-18T09:10:46.137 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-18T09:10:46.149 INFO:teuthology.task.internal:Not uploading archives. 2023-10-18T09:10:46.149 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-18T09:10:46.158 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-18T09:10:46.158 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-18T09:10:46.164 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-18T09:10:46.219 INFO:teuthology.orchestra.run.smithi040.stdout: 393251 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 18 09:10 /home/ubuntu/cephtest 2023-10-18T09:10:46.225 INFO:teuthology.orchestra.run.smithi144.stdout: 393762 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 18 09:10 /home/ubuntu/cephtest 2023-10-18T09:10:46.228 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-18T09:10:46.240 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-18T09:10:46.259 INFO:teuthology.nuke:Checking targets against current locks 2023-10-18T09:10:46.286 DEBUG:teuthology.nuke:shortname: smithi040 2023-10-18T09:10:46.289 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T09:10:46.316 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 08:33:22.094210', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPru1b2IMuAaQvQ9WGQJurxzltIS6eqOeara4xXFpywLI6lhSRJD685TuJEGHY9LjnxtK5bfzffsA2ujB/Fq6ZE='} 2023-10-18T09:10:46.319 DEBUG:teuthology.nuke:shortname: smithi144 2023-10-18T09:10:46.319 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T09:10:46.341 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430719', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 08:33:22.095952', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDKHTBvPWMvPC4sZtXUB9Xz9kdZblgeqYUCDaBSavudWyKwn8Od7f360DtIrNn+UGrWbBln0nEK2o6UC5gDSAS4='} 2023-10-18T09:10:46.367 INFO:teuthology.orchestra.console.smithi040:Power off 2023-10-18T09:10:46.368 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-18T09:10:46.390 DEBUG:teuthology.orchestra.console.smithi040:power off output: Chassis Power Control: Down/Off 2023-10-18T09:10:46.391 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:10:46.405 INFO:teuthology.orchestra.console.smithi144:Power off 2023-10-18T09:10:46.406 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-18T09:10:46.414 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2023-10-18T09:10:46.426 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2023-10-18T09:10:46.426 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:10:46.447 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-10-18T09:10:50.415 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:10:50.449 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:10:50.524 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2023-10-18T09:10:50.558 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-10-18T09:10:54.526 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:10:54.560 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:10:54.636 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2023-10-18T09:10:54.671 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-10-18T09:10:58.638 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:10:58.672 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:10:58.748 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off 2023-10-18T09:10:58.782 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2023-10-18T09:10:58.850 INFO:teuthology.orchestra.console.smithi040:Power off completed 2023-10-18T09:10:58.883 INFO:teuthology.orchestra.console.smithi144:Power off completed 2023-10-18T09:10:58.988 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 client.1-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} duration: 1712.6432807445526 failure_reason: '"2023-10-18T08:55:03.058812+0000 osd.4 (osd.4) 3 : cluster [WRN] OSD bench result of 75039.513932 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.4. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd''s IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2023-10-18T09:10:58.989 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-18T09:10:59.077 INFO:teuthology.run:FAIL